slub: Fixes freepointer encoding for single free
Subsystems (2)
PATCH 1/1 slub: Fixes freepointer encoding for single free
From: Nicolas Bouchinet <[email protected]>
Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook().
If init_on_free is set, slab_free_hook() zeroes the object. Afterward, if slub_debug=F and CONFIG_SLAB_FREELIST_HARDENED are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object().
Object's freepointer thus needs to be properly set using set_freepointer() after init_on_free.
To reproduce, set slub_debug=FU init_on_free=1 log_level=7 on the command line of a kernel build with CONFIG_SLAB_FREELIST_HARDENED=y.
dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c [ 10.716698] [ 10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035667c: 00 00 00 00 .... [ 10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
--- mm/slub.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
mm/slub.c +7 -1
@@ -4342,10 +4342,16 @@ static __fastpath_inlinevoid slab_free (struct kmem_cache *s,struct slab *slab,void *object,unsigned long addr) { +bool init = false; +memcg_slab_free_hook (s, slab, &object,1 ); + init =slab_want_init_on_free (s); -if (likely (slab_free_hook (s, object,slab_want_init_on_free (s)))) +if (likely (slab_free_hook (s, object, init))) { +if (init) +set_freepointer (s, object,NULL );do_slab_free (s, slab, object, object,1 , addr); + } } static __fastpath_inline
Signature · 1 line
2.44.0
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet <[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().
Right.
Quoted 6 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().
IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here slab_free_hook() zeroed the freepointer and caused the problem.
But why we should zero the memory outside the object_size? It seems more reasonable to only zero the object_size when init_on_free is set?
Thanks.
Quoted 49 lines
Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c [ 10.716698] [ 10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035667c: 00 00 00 00 .... [ 10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freedSigned-off-by Nicolas Bouchinet <[email protected]>--- mm/slub.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +7
-1@@ -4342,10 +4342,16 @@ static __fastpath_inlinevoid slab_free (struct kmem_cache *s,struct slab *slab,void *object,unsigned long addr) { +bool init = false; +memcg_slab_free_hook (s, slab, &object,1 ); + init =slab_want_init_on_free (s); -if (likely (slab_free_hook (s, object,slab_want_init_on_free (s)))) +if (likely (slab_free_hook (s, object, init))) { +if (init) +set_freepointer (s, object,NULL );do_slab_free (s, slab, object, object,1 , addr); + } } static __fastpath_inline
Hy,
First of all, thanks a lot for your time.
Quoted 14 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
Quoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().
Quoted 6 lines
IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?
The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in init_on_free initial Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").
I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
The patch could also be optimized a bit by restricting set_freepointer() call to the CONFIG_SLAB_FREELIST_HARDENED option value.
Thanks again, Nicolas
Quoted 51 lines
Thanks.
Quoted 48 lines
Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c [ 10.716698] [ 10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035667c: 00 00 00 00 .... [ 10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freedSigned-off-by Nicolas Bouchinet<[email protected]>--- mm/slub.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +7
-1@@ -4342,10 +4342,16 @@ static __fastpath_inlinevoid slab_free (struct kmem_cache *s,struct slab *slab,void *object,unsigned long addr) { +bool init = false; +memcg_slab_free_hook (s, slab, &object,1 ); + init =slab_want_init_on_free (s); -if (likely (slab_free_hook (s, object,slab_want_init_on_free (s)))) +if (likely (slab_free_hook (s, object, init))) { +if (init) +set_freepointer (s, object,NULL );do_slab_free (s, slab, object, object,1 , addr); + } } static __fastpath_inline
Quoted 23 lines · Nicolas Bouchinet
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
Quoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.
Quoted 6 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Quoted 3 lines
The patch could also be optimized a bit by restricting
set_freepointer()call to theCONFIG_SLAB_FREELIST_HARDENEDoption value.
Yeah. Maybe memcg_alloc_abort_single() needs this too.
Thanks.
Quoted 51 lines
Thanks again, Nicolas
Quoted 49 lines
Thanks.
Quoted 46 lines
Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.dmesg sample log:
[ 10.708715] =============================================================================
[ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt
[ 10.712695] -----------------------------------------------------------------------------
[ 10.712695]
[ 10.712695] Slab0xffffd8bdc400d580objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object0xffff9d9a80356600@offset=1536 fp=0x7ee4f480ce0ecd7c
[ 10.716698]
[ 10.716698] Bytes b4ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035667c: 00 00 00 00 ....
[ 10.724696] FIX kmalloc-rnd-05-32: Object at0xffff9d9a80356600not freedSigned-off-by Nicolas Bouchinet<[email protected]>---
mm/slub.c| 8 +++++++-1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +7
-0@@ -4342,10 +4342,16 @@ static __fastpath_inline void slab_free(struct kmem_cache *s, struct slab *slab, void *object, unsigned long addr) { + bool init = false; + memcg_slab_free_hook(s, slab, &object, 1); + init = slab_want_init_on_free(s); - if (likely(slab_free_hook(s, object, slab_want_init_on_free(s)))) + if (likely(slab_free_hook(s, object, init))) { + if (init) + set_freepointer(s, object, NULL); do_slab_free(s, slab, object, object, 1, addr); + } }static
__fastpath_inline
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 26 lines
Quoted 26 lines
Hy,
First of all, thanks a lot for your time.
Quoted 20 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
Quoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?
Quoted 6 lines
Quoted 3 lines
The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of init_on_free:
- the freeing itself will always set it one way or another, in this case free_to_partial_list() will do set_freepointer() after free_debug_processing()
- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow
So the best option to me would be to adjust the init in slab_free_hook() to avoid the outside-object freepointer similarly to how it avoids the red zone.
We'll still not have the buffer overflow detection ability for bulk free where slab_free_freelist_hook() will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.
Quoted 59 lines
Quoted 3 lines
The patch could also be optimized a bit by restricting
set_freepointer()call to theCONFIG_SLAB_FREELIST_HARDENEDoption value.Yeah. Maybe
memcg_alloc_abort_single()needs this too.Thanks.
Quoted 51 lines
Thanks again, Nicolas
Quoted 49 lines
Thanks.
Quoted 46 lines
Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.dmesg sample log:
[ 10.708715] =============================================================================
[ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt
[ 10.712695] -----------------------------------------------------------------------------
[ 10.712695]
[ 10.712695] Slab0xffffd8bdc400d580objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object0xffff9d9a80356600@offset=1536 fp=0x7ee4f480ce0ecd7c
[ 10.716698]
[ 10.716698] Bytes b4ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035667c: 00 00 00 00 ....
[ 10.724696] FIX kmalloc-rnd-05-32: Object at0xffff9d9a80356600not freedSigned-off-by Nicolas Bouchinet<[email protected]>---
mm/slub.c| 8 +++++++-1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +7
-0@@ -4342,10 +4342,16 @@ static __fastpath_inline void slab_free(struct kmem_cache *s, struct slab *slab, void *object, unsigned long addr) { + bool init = false; + memcg_slab_free_hook(s, slab, &object, 1); + init = slab_want_init_on_free(s); - if (likely(slab_free_hook(s, object, slab_want_init_on_free(s)))) + if (likely(slab_free_hook(s, object, init))) { + if (init) + set_freepointer(s, object, NULL); do_slab_free(s, slab, object, object, 1, addr); + } }static
__fastpath_inline
Hi Vlastimil,
thanks for your review and your proposal.
Quoted 43 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 23 lines
Quoted 23 lines
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
y not suitable for a stable-candidate fix we needQuoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of
init_on_free:- the freeing itself will always set it one way or another, in this case
free_to_partial_list()will doset_freepointer()afterfree_debug_processing()- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow
Quoted 7 lines
So the best option to me would be to adjust the init in
slab_free_hook()to avoid the outside-object freepointer similarly to how it avoids the red zone.We'll still not have the buffer overflow detection ability for bulk free where
slab_free_freelist_hook()will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.
It seems like a good alternative to me, I'll push a V2 patch with those changes.
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
Quoted 49 lines
Quoted 49 lines
Quoted 3 lines
The patch could also be optimized a bit by restricting
set_freepointer()call to theCONFIG_SLAB_FREELIST_HARDENEDoption value.Yeah. Maybe
memcg_alloc_abort_single()needs this too.Thanks.
Quoted 42 lines
Thanks again, Nicolas
Quoted 40 lines
Thanks.
Quoted 38 lines
Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.dmesg sample log:
[ 10.708715] =============================================================================
[ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt
[ 10.712695] -----------------------------------------------------------------------------
[ 10.712695]
[ 10.712695] Slab0xffffd8bdc400d580objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object0xffff9d9a80356600@offset=1536 fp=0x7ee4f480ce0ecd7c
[ 10.716698]
[ 10.716698] Bytes b4ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035667c: 00 00 00 00 ....
[ 10.724696] FIX kmalloc-rnd-05-32: Object at0xffff9d9a80356600not freedSigned-off-by Nicolas Bouchinet<[email protected]>--- mm/slub.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +3
-0@@ -4342,10 +4342,16 @@ static __fastpath_inline void slab_free(struct kmem_cache *s, struct slab *slab, void *object, unsigned long addr) { + bool init = false; + memcg_slab_free_hook(s, slab, &object, 1); + init = slab_want_init_on_free(s);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
mm/slub.c static __fastpath_inline · line 4357
Quoted 8 lines
mm/slub.c +4
-0- if (likely(slab_free_hook(s, object, slab_want_init_on_free(s)))) + if (likely(slab_free_hook(s, object, init))) { + if (init) + set_freepointer(s, object, NULL); do_slab_free(s, slab, object, object, 1, addr); + } } static __fastpath_inline
Thanks again for your review,
Nicolas
Quoted 52 lines · Nicolas Bouchinet
Hi Vlastimil,
thanks for your review and your proposal.
Quoted 47 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 23 lines
Quoted 23 lines
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
y not suitable for a stable-candidate fix we needQuoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of
init_on_free:- the freeing itself will always set it one way or another, in this case
free_to_partial_list()will doset_freepointer()afterfree_debug_processing()- we lose the ability to detect if the allocated slab object's user wrote to
it, which is a buffer overflowSo the best option to me would be to adjust the init in
slab_free_hook()to avoid the outside-object freepointer similarly to how it avoids the red zone.
Quoted 8 lines
Quoted 4 lines
We'll still not have the buffer overflow detection ability for bulk free where
slab_free_freelist_hook()will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.It seems like a good alternative to me, I'll push a V2 patch with those changes.
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation if init_on_free is set in order to return a clean initialized object to the caller.
Quoted 63 lines
Quoted 51 lines
Quoted 51 lines
Quoted 4 lines
The patch could also be optimized a bit by restricting
set_freepointer()call to theCONFIG_SLAB_FREELIST_HARDENEDoption value.Yeah. Maybe
memcg_alloc_abort_single()needs this too.Thanks.
Quoted 43 lines
Thanks again, Nicolas
Quoted 41 lines
Thanks.
Quoted 39 lines
Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.dmesg sample log:
[ 10.708715] =============================================================================
[ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt
[ 10.712695]
-----------------------------------------------------------------------------
[ 10.712695]
[ 10.712695] Slab0xffffd8bdc400d580objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object0xffff9d9a80356600@offset=1536 fp=0x7ee4f480ce0ecd7c
[ 10.716698]
[ 10.716698] Bytes b4ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.720703] Objectffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 10.724696] Paddingffff9d9a8035667c: 00 00 00 00 ....
[ 10.724696] FIX kmalloc-rnd-05-32: Object at0xffff9d9a80356600not freedSigned-off-by Nicolas Bouchinet<[email protected]>---
mm/slub.c| 8 +++++++-1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +3
-0@@ -4342,10 +4342,16 @@ static __fastpath_inline void slab_free(struct kmem_cache *s, struct slab *slab, void *object, unsigned long addr) { + bool init = false; + memcg_slab_free_hook(s, slab, &object, 1); + init = slab_want_init_on_free(s);Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
mm/slub.c +4
-0- if (likely(slab_free_hook(s, object, slab_want_init_on_free(s)))) + if (likely(slab_free_hook(s, object, init))) { + if (init) + set_freepointer(s, object, NULL); do_slab_free(s, slab, object, object, 1, addr); + } }static
__fastpath_inlineThanks again for your review,
Nicolas
Quoted 41 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet
Hi Vlastimil,
thanks for your review and your proposal.
Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 30 lines
Quoted 23 lines
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
y not suitable for a stable-candidate fix we needQuoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.Quoted 5 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of
init_on_free:- the freeing itself will always set it one way or another, in this case
free_to_partial_list()will doset_freepointer()afterfree_debug_processing()- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow
Ah, right, this ability seems important for debugging overflow problem.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
So the best option to me would be to adjust the init in
slab_free_hook()to avoid the outside-object freepointer similarly to how it avoids the red zone.
Agree.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
We'll still not have the buffer overflow detection ability for bulk free where
slab_free_freelist_hook()will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.
Quoted 7 lines
Quoted 3 lines
It seems like a good alternative to me, I'll push a V2 patch with those changes.
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit
0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation ifinit_on_freeis set in order to return a clean initialized object to the caller.
Good catch! You may need to change maybe_wipe_obj_freeptr() too, I haven't tested this, not sure whether it works for you. :)
mm/slub.c +2 -1
@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,void *obj) { -if (unlikely (slab_want_init_on_free (s)) && obj) +if (unlikely (slab_want_init_on_free (s)) && obj && + !freeptr_outside_object (s))memset ((void *)((char *)kasan_reset_tag (obj) + s->offset),0 ,sizeof (void *)); }
Thanks!
Quoted 83 lines · Chengming Zhou
Quoted 40 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet
Hi Vlastimil,
thanks for your review and your proposal.
Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 30 lines
Quoted 23 lines
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
y not suitable for a stable-candidate fix we needQuoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.Quoted 5 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of
init_on_free:- the freeing itself will always set it one way or another, in this case
free_to_partial_list()will doset_freepointer()afterfree_debug_processing()- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow
Ah, right, this ability seems important for debugging overflow problem.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So the best option to me would be to adjust the init in
slab_free_hook()to avoid the outside-object freepointer similarly to how it avoids the red zone.Agree.
Quoted 10 lines
Quoted 7 lines
Quoted 4 lines
We'll still not have the buffer overflow detection ability for bulk free where
slab_free_freelist_hook()will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.It seems like a good alternative to me, I'll push a V2 patch with those changes.
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit
0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation ifinit_on_freeis set in order to return a clean initialized object to the caller.Good catch! You may need to change
maybe_wipe_obj_freeptr()too, I haven't tested this, not sure whether it works for you. :)mm/slub.c +2
-1@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,void *obj) { -if (unlikely (slab_want_init_on_free (s)) && obj) +if (unlikely (slab_want_init_on_free (s)) && obj && + !freeptr_outside_object (s))memset ((void *)((char *)kasan_reset_tag (obj) + s->offset),0 ,sizeof (void *)); }
Quoted 1 line
Thanks!
Indeed since check_object() avoids objects for which freepointer is in the object and since val is equal to SLUB_RED_ACTIVE in our specific case it should work. Do you want me to add you as Co-authored ?
Best regards,
Nicolas
Quoted 85 lines · Nicolas Bouchinet
Quoted 83 lines · Chengming Zhou
Quoted 40 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet
Hi Vlastimil,
thanks for your review and your proposal.
Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 30 lines
Quoted 23 lines
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
y not suitable for a stable-candidate fix we needQuoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.Quoted 5 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of
init_on_free:- the freeing itself will always set it one way or another, in this case
free_to_partial_list()will doset_freepointer()afterfree_debug_processing()- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow
Ah, right, this ability seems important for debugging overflow problem.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So the best option to me would be to adjust the init in
slab_free_hook()to avoid the outside-object freepointer similarly to how it avoids the red zone.Agree.
Quoted 10 lines
Quoted 7 lines
Quoted 4 lines
We'll still not have the buffer overflow detection ability for bulk free where
slab_free_freelist_hook()will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.It seems like a good alternative to me, I'll push a V2 patch with those changes.
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit
0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation ifinit_on_freeis set in order to return a clean initialized object to the caller.Good catch! You may need to change
maybe_wipe_obj_freeptr()too, I haven't tested this, not sure whether it works for you. :)mm/slub.c +2
-1@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) { - if (unlikely(slab_want_init_on_free(s)) && obj) + if (unlikely(slab_want_init_on_free(s)) && obj && + !freeptr_outside_object(s)) memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); }
Quoted 4 lines
Quoted 1 line
Thanks!
Indeed since
check_object()avoids objects for which freepointer is in the object and since val is equal toSLUB_RED_ACTIVEin our specific case it should work. Do you want me to add you as Co-authored ?
Ok, it's great. Thanks!
Quoted 84 lines · Chengming Zhou
Quoted 83 lines · Nicolas Bouchinet
Quoted 82 lines · Chengming Zhou
Quoted 40 lines · Nicolas Bouchinet
Quoted 39 lines · Nicolas Bouchinet
Hi Vlastimil,
thanks for your review and your proposal.
Quoted 34 lines · Vlastimil Babka
Quoted 1 line · Chengming Zhou
On 2024/4/25 23:02, Nicolas Bouchinet wrote:
Thanks for finding the bug and the fix!
Quoted 30 lines
Quoted 23 lines
Hy,
First of all, thanks a lot for your time.
Quoted 18 lines · Chengming Zhou
Quoted 6 lines · Nicolas Bouchinet
From: Nicolas Bouchinet<[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().Right.
y not suitable for a stable-candidate fix we needQuoted 5 lines
If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().IIUC, the "freepointer" can be checked on the free path only when it's outside the object memory. Here
slab_free_hook()zeroed the freepointer and caused the problem.Quoted 5 lines
Quoted 2 lines
But why we should zero the memory outside the
object_size? It seems more reasonable to only zero theobject_sizewheninit_on_freeis set?The original purpose was to avoid leaking information through the object and its metadata / tracking information as described in
init_on_freeinitial Commit6471384af2a6("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options").I have to admit I didn't read the entire lore about the original patchset yet, though it could be interesting to know a bit more the threat models, specifically regarding the object metadata init.
Thank you for the reference! I also don't get why it needs to zero the metadata and tracking information.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Hmm taking a step back, it seems really suboptimal to initialize the outside-object freepointer as part of
init_on_free:- the freeing itself will always set it one way or another, in this case
free_to_partial_list()will doset_freepointer()afterfree_debug_processing()- we lose the ability to detect if the allocated slab object's user wrote to it, which is a buffer overflow
Ah, right, this ability seems important for debugging overflow problem.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So the best option to me would be to adjust the init in
slab_free_hook()to avoid the outside-object freepointer similarly to how it avoids the red zone.Agree.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
We'll still not have the buffer overflow detection ability for bulk free where
slab_free_freelist_hook()will set the free pointer before we reach the checks, but changing that is most likely not worth the trouble, and especially not suitable for a stable-candidate fix we need here.It seems like a good alternative to me, I'll push a V2 patch with those changes.
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit
0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation ifinit_on_freeis set in order to return a clean initialized object to the caller.Good catch! You may need to change
maybe_wipe_obj_freeptr()too, I haven't tested this, not sure whether it works for you. :)mm/slub.c +2
-1@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) { - if (unlikely(slab_want_init_on_free(s)) && obj) + if (unlikely(slab_want_init_on_free(s)) && obj && + !freeptr_outside_object(s)) memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); }
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
Thanks!
Indeed since
check_object()avoids objects for which freepointer is in the object and since val is equal toSLUB_RED_ACTIVEin our specific case it should work. Do you want me to add you as Co-authored ?Ok, it's great. Thanks!
Now I think of it, doesn't it seems a bit odd to only properly init_on_free object's freepointer only if it's inside the object ? IMHO it is equally necessary to avoid information leaking about the freepointer whether it is inside or outside the object. I think it break the semantic of the commit 0f181f9fbea8bc7ea ("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") ?
Thanks.
Quoted 31 lines · Nicolas Bouchinet
Quoted 25 lines · Chengming Zhou
Quoted 24 lines · Nicolas Bouchinet
Quoted 23 lines · Chengming Zhou
Quoted 4 lines · Nicolas Bouchinet
Quoted 2 lines
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit
0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation ifinit_on_freeis set in order to return a clean initialized object to the caller.Good catch! You may need to change
maybe_wipe_obj_freeptr()too, I haven't tested this, not sure whether it works for you. :)mm/slub.c +2
-1@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) { - if (unlikely(slab_want_init_on_free(s)) && obj) + if (unlikely(slab_want_init_on_free(s)) && obj && + !freeptr_outside_object(s)) memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); }Quoted 4 lines
Quoted 3 lines
Quoted 1 line
Thanks!
Indeed since
check_object()avoids objects for which freepointer is in the object and since val is equal toSLUB_RED_ACTIVEin our specific case it should work. Do you want me to add you as Co-authored ?Ok, it's great. Thanks!
Quoted 7 lines
Now I think of it, doesn't it seems a bit odd to only properly
init_on_freeobject's freepointer only if it's inside the object ? IMHO it is equally necessary to avoid information leaking about the freepointer whether it is inside or outside the object.
I think it break the semantic of the commit0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") ?
Hm, AFAIU, wiping inside object prevents misuse of some buggy kernel code that would allocate and accidentally leak prior content (including the in-object freepointer) somewhere the attacker can read. Now for wiping the freepointer outside the object to be useful it would have assume said leak-prone code to additionally be reading past the allocated object size, i.e. a read buffer overflow. That to me seems to be a much more rare combination, and also in that case such code could also likely read even further past the object, i.e. leak the next object's data? IOW I don't think it buys us much additional security protection in practice?
Quoted 2 lines
Thanks.
Quoted 37 lines · Vlastimil Babka
Quoted 28 lines · Nicolas Bouchinet
Quoted 27 lines · Chengming Zhou
Quoted 26 lines · Nicolas Bouchinet
Quoted 22 lines · Chengming Zhou
Quoted 3 lines · Nicolas Bouchinet
Quoted 1 line
I help maintaining the Linux-Hardened patchset in which we have a slab object canary feature that helps detecting overflows. It is located just after the object freepointer.
I've tried a patch where the freepointer is avoided but it results in the same bug. It seems that the commit
0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") inits the freepointer on allocation ifinit_on_freeis set in order to return a clean initialized object to the caller.Good catch! You may need to change
maybe_wipe_obj_freeptr()too, I haven't tested this, not sure whether it works for you. :)mm/slub.c +2
-1@@ -3796,7 +3796,8 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) { - if (unlikely(slab_want_init_on_free(s)) && obj) + if (unlikely(slab_want_init_on_free(s)) && obj && + !freeptr_outside_object(s)) memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); }Quoted 1 line
Thanks!
Indeed since
check_object()avoids objects for which freepointer is in the object and since val is equal toSLUB_RED_ACTIVEin our specific case it should work. Do you want me to add you as Co-authored ?Quoted 8 lines
Quoted 1 line
Ok, it's great. Thanks!
Now I think of it, doesn't it seems a bit odd to only properly
init_on_freeobject's freepointer only if it's inside the object ? IMHO it is equally necessary to avoid information leaking about the freepointer whether it is inside or outside the object.
I think it break the semantic of the commit0f181f9fbea8bc7ea("mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations") ?
Quoted 10 lines
Hm, AFAIU, wiping inside object prevents misuse of some buggy kernel code that would allocate and accidentally leak prior content (including the in-object freepointer) somewhere the attacker can read. Now for wiping the freepointer outside the object to be useful it would have assume said leak-prone code to additionally be reading past the allocated object size, i.e. a read buffer overflow. That to me seems to be a much more rare combination, and also in that case such code could also likely read even further past the object, i.e. leak the next object's data? IOW I don't think it buys us much additional security protection in practice?
Moreover, with CONFIG_SLAB_FREELIST_HARDENED activated, freepointers are encoded and harder to exploit.
Quoted 3 lines
Quoted 2 lines
Thanks.
Quoted 20 lines · Nicolas Bouchinet
From: Nicolas Bouchinet <[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.
Quoted 7 lines
dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
If init_on_free is set, slab_free_hook() zeros the object first, then do_slab_free() callsset_freepointer() to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related to init_on_free?
The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from CONFIG_SLAB_FREELIST_HARDENED enabled?
Thanks,
Xiongwei
Quoted 39 lines
[ 10.716698] [ 10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035667c: 00 00 00 00 .... [ 10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freedSigned-off-by Nicolas Bouchinet <[email protected]>--- mm/slub.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +7
-2@@ -4342,10 +4342,16 @@ static __fastpath_inlinevoid slab_free (struct kmem_cache *s,struct slab *slab,void *object,unsigned long addr) { +bool init = false; +memcg_slab_free_hook (s, slab, &object,1 ); + init =slab_want_init_on_free (s); -if (likely (slab_free_hook (s, object,slab_want_init_on_free (s)))) +if (likely (slab_free_hook (s, object, init))) { +if (init) +set_freepointer (s, object,NULL );do_slab_free (s, slab, object, object,1 , addr); + } } static __fastpath_inline --2.44.0
Quoted 28 lines · Xiongwei Song
Quoted 19 lines · Nicolas Bouchinet
From: Nicolas Bouchinet <[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.Quoted 7 lines
dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
Quoted 8 lines
If
init_on_freeis set,slab_free_hook()zeros the object first, thendo_slab_free()callsset_freepointer()to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related toinit_on_free?The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
CONFIG_SLAB_FREELIST_HARDENEDenabled?
My understanding of the bug is that slab_free_hook() indeed zeroes the object and its metadata first, then calls do_slab_free() and directly calls __slab_free(), head an tail parameters being set to the object.
If slub_debug=F (SLAB_CONSISTENCY_CHECKS) is set, the following call path can be executed :
free_to_partial_list() ->
free_debug_processing() ->
free_consistency_checks() ->
check_object() ->
check_valid_pointer(get_freepointer())
When check_valid_pointer() is called, its object parameter is first decoded by get_freepointer(), if CONFIG_SLAB_FREELIST_HARDENED is enabled, zeroed data is then decoded by freelist_ptr_decode() using the (ptr.v ^ s->random ^ swab(ptr_addr)) operation.
Does that makes sense to you or do you think I'm missing something ?
Best regards,
Nicolas
Quoted 41 lines
Xiongwei
Quoted 39 lines
[ 10.716698] [ 10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035667c: 00 00 00 00 .... [ 10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freedSigned-off-by Nicolas Bouchinet <[email protected]>--- mm/slub.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/slub.c +7
-2@@ -4342,10 +4342,16 @@ static __fastpath_inlinevoid slab_free (struct kmem_cache *s,struct slab *slab,void *object,unsigned long addr) { +bool init = false; +memcg_slab_free_hook (s, slab, &object,1 ); + init =slab_want_init_on_free (s); -if (likely (slab_free_hook (s, object,slab_want_init_on_free (s)))) +if (likely (slab_free_hook (s, object, init))) { +if (init) +set_freepointer (s, object,NULL );do_slab_free (s, slab, object, object,1 , addr); + } } static __fastpath_inline --2.44.0
Quoted 48 lines · Nicolas Bouchinet
Quoted 28 lines · Xiongwei Song
Quoted 19 lines · Nicolas Bouchinet
From: Nicolas Bouchinet <[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.Quoted 7 lines
dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7cQuoted 8 lines
If
init_on_freeis set,slab_free_hook()zeros the object first, thendo_slab_free()callsset_freepointer()to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related toinit_on_free?The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
CONFIG_SLAB_FREELIST_HARDENEDenabled?My understanding of the bug is that
slab_free_hook()indeed zeroes the object and its metadata first, then callsdo_slab_free()and directly calls__slab_free(), head an tail parameters being set to the object.If
slub_debug=F(SLAB_CONSISTENCY_CHECKS) is set, the following call path can be executed :
free_to_partial_list()->
Quoted 7 lines
free_debug_processing()->
free_consistency_checks()->
check_object()->check_valid_pointer(get_freepointer())
I understand the call path. I meant here the freepointer is not ZERO but an illegal
value( fp=0x7ee4f480ce0ecd7c), then check_valid_pointer return 1 with it's last line
and then check_object() printed out the error message. I'm not sure if I misunderstood you.
Thank,
Xiongwei
Quoted 50 lines · Xiongwei Song
Quoted 48 lines · Nicolas Bouchinet
Quoted 28 lines · Xiongwei Song
Quoted 19 lines · Nicolas Bouchinet
From: Nicolas Bouchinet <[email protected]>
Commit
284f17ac13fe("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functionsslab_free()andslab_free_bulk()which leadsslab_free()to callslab_free_hook()directly instead ofslab_free_freelist_hook().If
init_on_freeis set,slab_free_hook()zeroes the object. Afterward, ifslub_debug=FandCONFIG_SLAB_FREELIST_HARDENEDare set, thedo_slab_free()slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection incheck_object().Object's freepointer thus needs to be properly set using
set_freepointer()afterinit_on_free.To reproduce, set
slub_debug=FU init_on_free=1 log_level=7on the command line of a kernel build withCONFIG_SLAB_FREELIST_HARDENED=y.Quoted 7 lines
dmesg sample log:
[ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7cQuoted 8 lines
If
init_on_freeis set,slab_free_hook()zeros the object first, thendo_slab_free()callsset_freepointer()to set the fp value, so there are 8 bytes non-zero at the moment?
Hence, the issue is not related toinit_on_free?The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
CONFIG_SLAB_FREELIST_HARDENEDenabled?My understanding of the bug is that
slab_free_hook()indeed zeroes the object and its metadata first, then callsdo_slab_free()and directly calls__slab_free(), head an tail parameters being set to the object.If
slub_debug=F(SLAB_CONSISTENCY_CHECKS) is set, the following call path can be executed :
free_to_partial_list()->
free_debug_processing()->
Quoted 8 lines
Quoted 5 lines
free_consistency_checks()->
check_object()->check_valid_pointer(get_freepointer())
I understand the call path. I meant here the freepointer is not ZERO but an illegal
value( fp=0x7ee4f480ce0ecd7c),
Yes this is the reason of this patch. The freepointer is an illegal value because the memory range where it sits has been overridden by zeroes, set_freepointer() is never called and thus the freepointer is never properly set.
The illegal value is obtained after zeroes has been decoded by get_freepointer()->freelist_ptr_decode().
Quoted 4 lines
then
check_valid_pointerreturn 1 with
it's last line
and thencheck_object()printed out the error message. I'm not sure if I misunderstood you.
check_valid_pointer() returns 0 since object < base, the object being the decoded fp (0x7ee4f480ce0ecd7c < 0xffff.* base addr), hence check_object() returns 0, not 1.
This is why the "Object at 0x%p not freed" slab_fix() is called in free_debug_processing().
Quoted 4 lines
Thank,
Xiongwei