Skip to contentSkip to conversation

slub: Fixes freepointer encoding for single free

PATCH Superseded Nicolas Bouchinet 4 participants 17 messages Apr 24, 2024 – Apr 30, 2024

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 slub: Fixes freepointer encoding for single free

From: Nicolas Bouchinet <[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
---
 mm/slub.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -1
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
 void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
 	       unsigned long addr)
 {
+	bool init = false;
+
 	memcg_slab_free_hook(s, slab, &object, 1);
+	init = slab_want_init_on_free(s);
 
-	if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+	if (likely(slab_free_hook(s, object, init))) {
+		if (init)
+			set_freepointer(s, object, NULL);
 		do_slab_free(s, slab, object, object, 1, addr);
+	}
 }
 
 static __fastpath_inline
Signature · 1 line
2.44.0
#1 lore raw
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet <[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.

Quoted 6 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

Thanks.

Quoted 49 lines

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
---
 mm/slub.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -1
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
 void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
 	       unsigned long addr)
 {
+	bool init = false;
+
 	memcg_slab_free_hook(s, slab, &object, 1);
+	init = slab_want_init_on_free(s);
 
-	if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+	if (likely(slab_free_hook(s, object, init))) {
+		if (init)
+			set_freepointer(s, object, NULL);
 		do_slab_free(s, slab, object, object, 1, addr);
+	}
 }
 
 static __fastpath_inline
#2 lore raw

Hy,

First of all, thanks a lot for your time.

Quoted 14 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

Quoted 6 lines

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

The patch could also be optimized a bit by restricting set_freepointer() call to the CONFIG_SLAB_FREELIST_HARDENED option value.

Thanks again, Nicolas

Quoted 51 lines

Thanks.

Quoted 48 lines

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
---
  mm/slub.c | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -1
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
  void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
  	       unsigned long addr)
  {
+	bool init = false;
+
  	memcg_slab_free_hook(s, slab, &object, 1);
+	init = slab_want_init_on_free(s);
 
-	if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+	if (likely(slab_free_hook(s, object, init))) {
+		if (init)
+			set_freepointer(s, object, NULL);
  		do_slab_free(s, slab, object, object, 1, addr);
+	}
  }
 
  static __fastpath_inline
#3 lore raw
Quoted 23 lines · Nicolas Bouchinet

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 6 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Quoted 3 lines

The patch could also be optimized a bit by restricting set_freepointer() call to the CONFIG_SLAB_FREELIST_HARDENED option value.

Yeah. Maybe memcg_alloc_abort_single() needs this too.

Thanks.

Quoted 51 lines

Thanks again, Nicolas

Quoted 49 lines

Thanks.

Quoted 46 lines

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:
[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed

---
  mm/slub.c | 8 +++++++-

  1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -0
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
   void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
              unsigned long addr)
   {
+    bool init = false;
+
       memcg_slab_free_hook(s, slab, &object, 1);
+    init = slab_want_init_on_free(s);
   -    if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+    if (likely(slab_free_hook(s, object, init))) {
+        if (init)
+            set_freepointer(s, object, NULL);
           do_slab_free(s, slab, object, object, 1, addr);
+    }
   }

    static __fastpath_inline

#13 lore raw
In reply to Nicolas Bouchinet
Quoted 20 lines · Nicolas Bouchinet

From: Nicolas Bouchinet <[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

Quoted 7 lines

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c

If init_on_free is set, slab_free_hook() zeros the object first, then do_slab_free() calls
set_freepointer() to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related to init_on_free?

The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from CONFIG_SLAB_FREELIST_HARDENED enabled?

Thanks,
Xiongwei

Quoted 39 lines
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
---
 mm/slub.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -2
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
 void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
               unsigned long addr)
 {
+       bool init = false;
+
        memcg_slab_free_hook(s, slab, &object, 1);
+       init = slab_want_init_on_free(s);
 
-       if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+       if (likely(slab_free_hook(s, object, init))) {
+               if (init)
+                       set_freepointer(s, object, NULL);
                do_slab_free(s, slab, object, object, 1, addr);
+       }
 }
 
 static __fastpath_inline
--

2.44.0

#14 lore raw
Quoted 28 lines · Xiongwei Song
Quoted 19 lines · Nicolas Bouchinet

From: Nicolas Bouchinet <[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

Quoted 7 lines

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
Quoted 8 lines

If init_on_free is set, slab_free_hook() zeros the object first, then do_slab_free() calls
set_freepointer() to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related to init_on_free?

The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from CONFIG_SLAB_FREELIST_HARDENED enabled?

My understanding of the bug is that slab_free_hook() indeed zeroes the object and its metadata first, then calls do_slab_free() and directly calls __slab_free(), head an tail parameters being set to the object.

If slub_debug=F (SLAB_CONSISTENCY_CHECKS) is set, the following call path can be executed :

free_to_partial_list() ->

free_debug_processing() ->

free_consistency_checks() ->

check_object() ->

check_valid_pointer(get_freepointer())

When check_valid_pointer() is called, its object parameter is first decoded by get_freepointer(), if CONFIG_SLAB_FREELIST_HARDENED is enabled, zeroed data is then decoded by freelist_ptr_decode() using the (ptr.v ^ s->random ^ swab(ptr_addr)) operation.

Does that makes sense to you or do you think I'm missing something ?

Best regards,

Nicolas

Quoted 41 lines

Xiongwei

Quoted 39 lines
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
---
  mm/slub.c | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -2
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
  void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
                unsigned long addr)
  {
+       bool init = false;
+
         memcg_slab_free_hook(s, slab, &object, 1);
+       init = slab_want_init_on_free(s);
 
-       if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+       if (likely(slab_free_hook(s, object, init))) {
+               if (init)
+                       set_freepointer(s, object, NULL);
                 do_slab_free(s, slab, object, object, 1, addr);
+       }
  }
 
  static __fastpath_inline
--

2.44.0

#15 lore raw
Quoted 48 lines · Nicolas Bouchinet
Quoted 28 lines · Xiongwei Song
Quoted 19 lines · Nicolas Bouchinet

From: Nicolas Bouchinet <[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

Quoted 7 lines

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
Quoted 8 lines

If init_on_free is set, slab_free_hook() zeros the object first, then do_slab_free() calls
set_freepointer() to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related to init_on_free?

The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from CONFIG_SLAB_FREELIST_HARDENED enabled?

My understanding of the bug is that slab_free_hook() indeed zeroes the object and its metadata first, then calls do_slab_free() and directly calls __slab_free(), head an tail parameters being set to the object.

If slub_debug=F (SLAB_CONSISTENCY_CHECKS) is set, the following call path can be executed :

free_to_partial_list() ->

Quoted 7 lines

free_debug_processing() ->

free_consistency_checks() ->

check_object() ->

check_valid_pointer(get_freepointer())

I understand the call path. I meant here the freepointer is not ZERO but an illegal
value( fp=0x7ee4f480ce0ecd7c), then check_valid_pointer return 1 with it's last line
and then check_object() printed out the error message. I'm not sure if I misunderstood you.

Thank,
Xiongwei

#16 lore raw
Quoted 50 lines · Xiongwei Song
Quoted 48 lines · Nicolas Bouchinet
Quoted 28 lines · Xiongwei Song
Quoted 19 lines · Nicolas Bouchinet

From: Nicolas Bouchinet <[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

Quoted 7 lines

dmesg sample log:

[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
Quoted 8 lines

If init_on_free is set, slab_free_hook() zeros the object first, then do_slab_free() calls
set_freepointer() to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related to init_on_free?

The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from CONFIG_SLAB_FREELIST_HARDENED enabled?

My understanding of the bug is that slab_free_hook() indeed zeroes the object and its metadata first, then calls do_slab_free() and directly calls __slab_free(), head an tail parameters being set to the object.

If slub_debug=F (SLAB_CONSISTENCY_CHECKS) is set, the following call path can be executed :

free_to_partial_list() ->

free_debug_processing() ->

Quoted 8 lines
Quoted 5 lines

free_consistency_checks() ->

check_object() ->

check_valid_pointer(get_freepointer())

I understand the call path. I meant here the freepointer is not ZERO but an illegal
value( fp=0x7ee4f480ce0ecd7c),

Yes this is the reason of this patch. The freepointer is an illegal value because the memory range where it sits has been overridden by zeroes, set_freepointer() is never called and thus the freepointer is never properly set.

 The illegal value is obtained after zeroes has been decoded by get_freepointer()->freelist_ptr_decode().

Quoted 4 lines

then check_valid_pointer return 1 with
it's last line
and then check_object() printed out the error message. I'm not sure if I misunderstood you.

check_valid_pointer() returns 0 since object < base, the object being the decoded fp (0x7ee4f480ce0ecd7c < 0xffff.* base addr), hence check_object() returns 0, not 1.

This is why the "Object at 0x%p not freed" slab_fix() is called in free_debug_processing().

Quoted 4 lines

Thank,
Xiongwei

#4 lore raw
In reply to Chengming Zhou
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 26 lines
Quoted 26 lines

Hy,

First of all, thanks a lot for your time.

Quoted 20 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

Quoted 6 lines
Quoted 3 lines

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

Quoted 59 lines
Quoted 3 lines

The patch could also be optimized a bit by restricting set_freepointer() call to the CONFIG_SLAB_FREELIST_HARDENED option value.

Yeah. Maybe memcg_alloc_abort_single() needs this too.

Thanks.

Quoted 51 lines

Thanks again, Nicolas

Quoted 49 lines

Thanks.

Quoted 46 lines

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:
[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed

---
  mm/slub.c | 8 +++++++-

  1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -0
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
   void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
              unsigned long addr)
   {
+    bool init = false;
+
       memcg_slab_free_hook(s, slab, &object, 1);
+    init = slab_want_init_on_free(s);
   -    if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+    if (likely(slab_free_hook(s, object, init))) {
+        if (init)
+            set_freepointer(s, object, NULL);
           do_slab_free(s, slab, object, object, 1, addr);
+    }
   }

    static __fastpath_inline

#5 lore raw

Hi Vlastimil,

thanks for your review and your proposal.

Quoted 43 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 23 lines
Quoted 23 lines

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.
y not suitable for a stable-candidate fix we need

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow

Quoted 7 lines

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

It seems like a good alternative to me, I'll push a V2 patch with those changes.

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

Quoted 49 lines
Quoted 49 lines
Quoted 3 lines

The patch could also be optimized a bit by restricting set_freepointer() call to the CONFIG_SLAB_FREELIST_HARDENED option value.

Yeah. Maybe memcg_alloc_abort_single() needs this too.

Thanks.

Quoted 42 lines

Thanks again, Nicolas

Quoted 40 lines

Thanks.

Quoted 38 lines

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:
[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed

---
   mm/slub.c | 8 +++++++-
   1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +3 -0
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
   void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
              unsigned long addr)
   {
+    bool init = false;
+
       memcg_slab_free_hook(s, slab, &object, 1);
+    init = slab_want_init_on_free(s);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

mm/slub.c static __fastpath_inline · line 4357

Quoted 8 lines
mm/slub.c +4 -0
   -    if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+    if (likely(slab_free_hook(s, object, init))) {
+        if (init)
+            set_freepointer(s, object, NULL);
           do_slab_free(s, slab, object, object, 1, addr);
+    }
   }
     static __fastpath_inline

Thanks again for your review,

Nicolas

#6 lore raw
Quoted 52 lines · Nicolas Bouchinet

Hi Vlastimil,

thanks for your review and your proposal.

Quoted 47 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 23 lines
Quoted 23 lines

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.
y not suitable for a stable-candidate fix we need

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to
it, which is a buffer overflow

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

Quoted 8 lines
Quoted 4 lines

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

It seems like a good alternative to me, I'll push a V2 patch with those changes.

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Quoted 63 lines
Quoted 51 lines
Quoted 51 lines
Quoted 4 lines

The patch could also be optimized a bit by restricting set_freepointer() call to the CONFIG_SLAB_FREELIST_HARDENED option value.

Yeah. Maybe memcg_alloc_abort_single() needs this too.

Thanks.

Quoted 43 lines

Thanks again, Nicolas

Quoted 41 lines

Thanks.

Quoted 39 lines

Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.

To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.

dmesg sample log:
[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B           T ): Freepointer corrupt
[   10.712695]
-----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed

---
   mm/slub.c | 8 +++++++-

   1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +3 -0
index 3aa12b9b323d9..71dbff9ad8f17 100644
@@ -4342,10 +4342,16 @@ static __fastpath_inline
    void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
               unsigned long addr)
    {
+    bool init = false;
+
        memcg_slab_free_hook(s, slab, &object, 1);
+    init = slab_want_init_on_free(s);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
mm/slub.c +4 -0
    -    if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+    if (likely(slab_free_hook(s, object, init))) {
+        if (init)
+            set_freepointer(s, object, NULL);
            do_slab_free(s, slab, object, object, 1, addr);
+    }
    }

     static __fastpath_inline

Thanks again for your review,

Nicolas

#7 lore raw
Quoted 41 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet

Hi Vlastimil,

thanks for your review and your proposal.

Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 30 lines
Quoted 23 lines

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.
y not suitable for a stable-candidate fix we need

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 5 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow

Ah, right, this ability seems important for debugging overflow problem.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

Agree.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

Quoted 7 lines
Quoted 3 lines

It seems like a good alternative to me, I'll push a V2 patch with those changes.

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)

mm/slub.c +2 -1
index 3e33ff900d35..3f250a167cb5 100644
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s,
 static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
                                                   void *obj)
 {
-       if (unlikely(slab_want_init_on_free(s)) && obj)
+       if (unlikely(slab_want_init_on_free(s)) && obj &&
+           !freeptr_outside_object(s))
                memset((void *)((char *)kasan_reset_tag(obj) + s->offset),
                        0, sizeof(void *));
 }

Thanks!

#8 lore raw
Quoted 83 lines · Chengming Zhou
Quoted 40 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet

Hi Vlastimil,

thanks for your review and your proposal.

Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 30 lines
Quoted 23 lines

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.
y not suitable for a stable-candidate fix we need

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 5 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow

Ah, right, this ability seems important for debugging overflow problem.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

Agree.

Quoted 10 lines
Quoted 7 lines
Quoted 4 lines

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

It seems like a good alternative to me, I'll push a V2 patch with those changes.

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)

mm/slub.c +2 -1
index 3e33ff900d35..3f250a167cb5 100644
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s,
  static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
                                                    void *obj)
  {
-       if (unlikely(slab_want_init_on_free(s)) && obj)
+       if (unlikely(slab_want_init_on_free(s)) && obj &&
+           !freeptr_outside_object(s))
                 memset((void *)((char *)kasan_reset_tag(obj) + s->offset),
                         0, sizeof(void *));
  }
Quoted 1 line

Thanks!

Indeed since check_object() avoids objects for which freepointer is in the object and since val is equal to SLUB_RED_ACTIVE in our specific case it should work. Do you want me to add you as Co-authored ?

Best regards,

Nicolas

#9 lore raw
Quoted 85 lines · Nicolas Bouchinet
Quoted 83 lines · Chengming Zhou
Quoted 40 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet

Hi Vlastimil,

thanks for your review and your proposal.

Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 30 lines
Quoted 23 lines

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.
y not suitable for a stable-candidate fix we need

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 5 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow

Ah, right, this ability seems important for debugging overflow problem.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

Agree.

Quoted 10 lines
Quoted 7 lines
Quoted 4 lines

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

It seems like a good alternative to me, I'll push a V2 patch with those changes.

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)

mm/slub.c +2 -1
index 3e33ff900d35..3f250a167cb5 100644
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s,
   static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
                                                     void *obj)
   {
-       if (unlikely(slab_want_init_on_free(s)) && obj)
+       if (unlikely(slab_want_init_on_free(s)) && obj &&
+           !freeptr_outside_object(s))
                  memset((void *)((char *)kasan_reset_tag(obj) + s->offset),
                          0, sizeof(void *));
   }
Quoted 4 lines
Quoted 1 line

Thanks!

Indeed since check_object() avoids objects for which freepointer is in the object and since val is equal to SLUB_RED_ACTIVE in our specific case it should work. Do you want me to add you as Co-authored ?

Ok, it's great. Thanks!

#10 lore raw
Quoted 84 lines · Chengming Zhou
Quoted 83 lines · Nicolas Bouchinet
Quoted 82 lines · Chengming Zhou
Quoted 40 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet

Hi Vlastimil,

thanks for your review and your proposal.

Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou

On 2024/4/25 23:02, Nicolas Bouchinet wrote:

Thanks for finding the bug and the fix!

Quoted 30 lines
Quoted 23 lines

Hy,

First of all, thanks a lot for your time.

Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet

From: Nicolas Bouchinet<[email protected]>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().

Right.
y not suitable for a stable-candidate fix we need

Quoted 5 lines

If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.

Quoted 5 lines
Quoted 2 lines

But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?

The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").

I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.

Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:

- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()

- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow

Ah, right, this ability seems important for debugging overflow problem.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.

Agree.

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.

It seems like a good alternative to me, I'll push a V2 patch with those changes.

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)

mm/slub.c +2 -1
index 3e33ff900d35..3f250a167cb5 100644
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s,
   static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
                                                     void *obj)
   {
-       if (unlikely(slab_want_init_on_free(s)) && obj)
+       if (unlikely(slab_want_init_on_free(s)) && obj &&
+           !freeptr_outside_object(s))
                  memset((void *)((char *)kasan_reset_tag(obj) + s->offset),
                          0, sizeof(void *));
   }
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

Thanks!

Indeed since check_object() avoids objects for which freepointer is in the object and since val is equal to SLUB_RED_ACTIVE in our specific case it should work. Do you want me to add you as Co-authored ?

Ok, it's great. Thanks!

Now I think of it, doesn't it seems a bit odd to only properly init_on_free object's freepointer only if it's inside the object ? IMHO it is equally necessary to avoid information leaking about the freepointer whether it is inside or outside the object. I think it break the semantic of the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") ?

Thanks.

#11 lore raw
Quoted 31 lines · Nicolas Bouchinet
Quoted 25 lines · Chengming Zhou
Quoted 24 lines · Nicolas Bouchinet
Quoted 23 lines · Chengming Zhou
Quoted 4 lines · Nicolas Bouchinet
Quoted 2 lines

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)

mm/slub.c +2 -1
index 3e33ff900d35..3f250a167cb5 100644
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s,
   static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
                                                     void *obj)
   {
-       if (unlikely(slab_want_init_on_free(s)) && obj)
+       if (unlikely(slab_want_init_on_free(s)) && obj &&
+           !freeptr_outside_object(s))
                  memset((void *)((char *)kasan_reset_tag(obj) + s->offset),
                          0, sizeof(void *));
   }
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

Thanks!

Indeed since check_object() avoids objects for which freepointer is in the object and since val is equal to SLUB_RED_ACTIVE in our specific case it should work. Do you want me to add you as Co-authored ?

Ok, it's great. Thanks!

Quoted 7 lines

Now I think of it, doesn't it seems a bit odd to only properly init_on_free object's freepointer only if it's inside the object ? IMHO it is equally necessary to avoid information leaking about the freepointer whether it is inside or outside the object.
I think it break the semantic of the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") ?

Hm, AFAIU, wiping inside object prevents misuse of some buggy kernel code that would allocate and accidentally leak prior content (including the in-object freepointer) somewhere the attacker can read. Now for wiping the freepointer outside the object to be useful it would have assume said leak-prone code to additionally be reading past the allocated object size, i.e. a read buffer overflow. That to me seems to be a much more rare combination, and also in that case such code could also likely read even further past the object, i.e. leak the next object's data? IOW I don't think it buys us much additional security protection in practice?

Quoted 2 lines

Thanks.

#12 lore raw
Quoted 37 lines · Vlastimil Babka
Quoted 28 lines · Nicolas Bouchinet
Quoted 27 lines · Chengming Zhou
Quoted 26 lines · Nicolas Bouchinet
Quoted 22 lines · Chengming Zhou
Quoted 3 lines · Nicolas Bouchinet
Quoted 1 line

I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.

I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.

Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)

mm/slub.c +2 -1
index 3e33ff900d35..3f250a167cb5 100644
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s,
    static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
                                                      void *obj)
    {
-       if (unlikely(slab_want_init_on_free(s)) && obj)
+       if (unlikely(slab_want_init_on_free(s)) && obj &&
+           !freeptr_outside_object(s))
                   memset((void *)((char *)kasan_reset_tag(obj) + s->offset),
                           0, sizeof(void *));
    }
Quoted 1 line

Thanks!

Indeed since check_object() avoids objects for which freepointer is in the object and since val is equal to SLUB_RED_ACTIVE in our specific case it should work. Do you want me to add you as Co-authored ?

Quoted 8 lines
Quoted 1 line

Ok, it's great. Thanks!

Now I think of it, doesn't it seems a bit odd to only properly init_on_free object's freepointer only if it's inside the object ? IMHO it is equally necessary to avoid information leaking about the freepointer whether it is inside or outside the object.
I think it break the semantic of the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") ?

Quoted 10 lines

Hm, AFAIU, wiping inside object prevents misuse of some buggy kernel code that would allocate and accidentally leak prior content (including the in-object freepointer) somewhere the attacker can read. Now for wiping the freepointer outside the object to be useful it would have assume said leak-prone code to additionally be reading past the allocated object size, i.e. a read buffer overflow. That to me seems to be a much more rare combination, and also in that case such code could also likely read even further past the object, i.e. leak the next object's data? IOW I don't think it buys us much additional security protection in practice?

Moreover, with CONFIG_SLAB_FREELIST_HARDENED activated, freepointers are encoded and harder to exploit.

Quoted 3 lines
Quoted 2 lines

Thanks.