mm: slub: re-initialize randomized freelist sequence in calculate_sizes
Subsystems (2)
PATCH 1/1 mm: slub: re-initialize randomized freelist sequence in calculate_sizes
From: Sahara <[email protected]>
Slab cache flags are exported to sysfs and are allowed to get modified from userspace. Some of those may call calculate_sizes function because the changed flag can take an effect on slab object size and layout, which means kmem_cache may have different order and objects. The freelist pointer corruption occurs if some slab flags are modified while CONFIG_SLAB_FREELIST_RANDOM is turned on.
$ echo 0 > /sys/kernel/slab/zs_handle/store_user $ echo 0 > /sys/kernel/slab/zspage/store_user $ mkswap /dev/block/zram0 $ swapon /dev/block/zram0 -p 32758
=============================================================================
BUG zs_handle (Not tainted): Freepointer corrupt
-----------------------------------------------------------------------------
Disabling lock debugging due to kernel taint
INFO: Slab 0xffffffbf29603600 objects=102 used=102 fp=0x0000000000000000 flags=0x0200
INFO: Object 0xffffffca580d8d78 @offset=3448 fp=0xffffffca580d8ed0
Redzone 00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
Object 0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
Redzone 000000008fd80359: bb bb bb bb bb bb bb bb ........
Padding 00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
In this example, an Android device tries to use zram as a swap and to turn off store_user in order to reduce the slub object size. When calculate_sizes is called in kmem_cache_open, size, order and objects for zs_handle is:
size:360, order:0, objects:22
However, if the SLAB_STORE_USER bit is cleared in store_user_store:
size: 56, order:1, objects:73
All the size, order, and objects is changed by calculate_sizes(), but the size of the random_seq array is still old value(22). As a result, out-of-bound array access can occur at shuffle_freelist() when slab allocation is requested.
This patch fixes the problem by re-allocating the random_seq array with re-calculated correct objects value.
--- mm/slub.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
mm/slub.c +16 -7
@@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)if (oo_objects (s->oo) >oo_objects (s->max)) s->max = s->oo; -return !!oo_objects (s->oo); +if (!oo_objects (s->oo)) +return 0 ; + +/* +* Initialize the pre-computed randomized freelist if slab is up. +* If the randomized freelist random_seq is already initialized, +* free and re-initialize it with re-calculated value. +*/ +if (slab_state >=UP ) { +if (s->random_seq) +cache_random_seq_destroy (s); +if (init_cache_random_seq (s)) +return 0 ; + } + +return 1 ; }static int kmem_cache_open (struct kmem_cache *s,slab_flags_t flags)
@@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) s->remote_node_defrag_ratio =1000 ;#endif -/* Initialize the pre-computed randomized freelist if slab is up */ -if (slab_state >=UP ) { -if (init_cache_random_seq (s)) - goto error; - } -if (!init_kmem_cache_nodes (s)) goto error;
Signature · 1 line
2.17.1
Andrew Morton
maintainer(cc Thomas and linux-mm)
Quoted 94 lines · On Sat, 8 Aug 2020 13:50:30 +0400 [email protected] wrote:
From: Sahara <[email protected]>
Slab cache flags are exported to sysfs and are allowed to get modified from userspace. Some of those may call
calculate_sizesfunction because the changed flag can take an effect on slab object size and layout, which meanskmem_cachemay have different order and objects. The freelist pointer corruption occurs if some slab flags are modified whileCONFIG_SLAB_FREELIST_RANDOMis turned on.$ echo 0 > /sys/kernel/slab/zs_handle/store_user $ echo 0 > /sys/kernel/slab/zspage/store_user $ mkswap /dev/block/zram0 $ swapon /dev/block/zram0 -p 32758=============================================================================
BUGzs_handle(Not tainted): Freepointer corrupt
-----------------------------------------------------------------------------Disabling lock debugging due to kernel taint
INFO: Slab0xffffffbf29603600objects=102 used=102 fp=0x0000000000000000 flags=0x0200
INFO: Object0xffffffca580d8d78@offset=3448 fp=0xffffffca580d8ed0Redzone
00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
Object0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
Redzone000000008fd80359: bb bb bb bb bb bb bb bb ........
Padding00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZIn this example, an Android device tries to use zram as a swap and to turn off
store_userin order to reduce the slub object size. Whencalculate_sizesis called inkmem_cache_open, size, order and objects forzs_handleis:
size:360, order:0, objects:22
However, if theSLAB_STORE_USERbit is cleared instore_user_store:
size: 56, order:1, objects:73All the size, order, and objects is changed by
calculate_sizes(), but the size of therandom_seqarray is still old value(22). As a result, out-of-bound array access can occur atshuffle_freelist()when slab allocation is requested.This patch fixes the problem by re-allocating the
random_seqarray with re-calculated correct objects value.Fixes 210e7a43fa905 ("mm: SLUB freelist randomization")Reported-by Ari-Pekka Verta <[email protected]>Reported-by Timo Simola <[email protected]>Signed-off-by Sahara <[email protected]>--- mm/slub.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)mm/slub.c +16
-7@@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)if (oo_objects (s->oo) >oo_objects (s->max)) s->max = s->oo; -return !!oo_objects (s->oo); +if (!oo_objects (s->oo)) +return 0 ; + +/* +* Initialize the pre-computed randomized freelist if slab is up. +* If the randomized freelist random_seq is already initialized, +* free and re-initialize it with re-calculated value. +*/ +if (slab_state >=UP ) { +if (s->random_seq) +cache_random_seq_destroy (s); +if (init_cache_random_seq (s)) +return 0 ; + } + +return 1 ; }static int kmem_cache_open (struct kmem_cache *s,slab_flags_t flags)@@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) s->remote_node_defrag_ratio =1000 ;#endif -/* Initialize the pre-computed randomized freelist if slab is up */ -if (slab_state >=UP ) { -if (init_cache_random_seq (s)) - goto error; - } -if (!init_kmem_cache_nodes (s)) goto error;Signature · 1 line
2.17.1
Thomas Garnier
Reviewed-by I am not super familiar with the sysfs setup here but the random
sequence should work as expected. One minor comment below.
Quoted 65 lines · Andrew Morton
(cc Thomas and linux-mm)
Quoted 59 lines · On Sat, 8 Aug 2020 13:50:30 +0400 [email protected] wrote:
From: Sahara <[email protected]>
Slab cache flags are exported to sysfs and are allowed to get modified from userspace. Some of those may call
calculate_sizesfunction because the changed flag can take an effect on slab object size and layout, which meanskmem_cachemay have different order and objects. The freelist pointer corruption occurs if some slab flags are modified whileCONFIG_SLAB_FREELIST_RANDOMis turned on.$ echo 0 > /sys/kernel/slab/zs_handle/store_user $ echo 0 > /sys/kernel/slab/zspage/store_user $ mkswap /dev/block/zram0 $ swapon /dev/block/zram0 -p 32758=============================================================================
BUGzs_handle(Not tainted): Freepointer corrupt
-----------------------------------------------------------------------------Disabling lock debugging due to kernel taint
INFO: Slab0xffffffbf29603600objects=102 used=102 fp=0x0000000000000000 flags=0x0200
INFO: Object0xffffffca580d8d78@offset=3448 fp=0xffffffca580d8ed0Redzone
00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
Object0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
Redzone000000008fd80359: bb bb bb bb bb bb bb bb ........
Padding00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZIn this example, an Android device tries to use zram as a swap and to turn off
store_userin order to reduce the slub object size. Whencalculate_sizesis called inkmem_cache_open, size, order and objects forzs_handleis:
size:360, order:0, objects:22
However, if theSLAB_STORE_USERbit is cleared instore_user_store:
size: 56, order:1, objects:73All the size, order, and objects is changed by
calculate_sizes(), but the size of therandom_seqarray is still old value(22). As a result, out-of-bound array access can occur atshuffle_freelist()when slab allocation is requested.This patch fixes the problem by re-allocating the
random_seqarray with re-calculated correct objects value.Fixes 210e7a43fa905 ("mm: SLUB freelist randomization")Reported-by Ari-Pekka Verta <[email protected]>Reported-by Timo Simola <[email protected]>Signed-off-by Sahara <[email protected]>--- mm/slub.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)mm/slub.c +0
-0@@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)if (oo_objects (s->oo) >oo_objects (s->max)) s->max = s->oo;
Quoted 12 lines
mm/slub.c static int calculate_sizes(struct kmem_cache *s, int forced_order) · line 3717
Quoted 12 lines
mm/slub.c +11
-1-return !!oo_objects (s->oo); +if (!oo_objects (s->oo)) +return 0 ; + +/* +* Initialize the pre-computed randomized freelist if slab is up. +* If the randomized freelist random_seq is already initialized, +* free and re-initialize it with re-calculated value. +*/ +if (slab_state >=UP ) { +if (s->random_seq) +cache_random_seq_destroy (s);
kfree(NULL) is a noop, so you don't need to check s->random_seq.
Quoted 21 lines
Quoted 21 lines
mm/slub.c +5
-6+if (init_cache_random_seq (s)) +return 0 ; + } + +return 1 ; }static int kmem_cache_open (struct kmem_cache *s,slab_flags_t flags)@@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) s->remote_node_defrag_ratio =1000 ;#endif -/* Initialize the pre-computed randomized freelist if slab is up */ -if (slab_state >=UP ) { -if (init_cache_random_seq (s)) - goto error; - } -if (!init_kmem_cache_nodes (s)) goto error;
Quoted 2 lines
Quoted 2 lines
--
2.17.1
--
Thomas
Quoted 1 line · On Sat, Aug 08, 2020 at 01:50:30PM +0400, [email protected] wrote:
From: Sahara <[email protected]>
Hi!
Quoted 8 lines
Slab cache flags are exported to sysfs and are allowed to get modified from userspace. Some of those may call
calculate_sizesfunction because the changed flag can take an effect on slab object size and layout, which meanskmem_cachemay have different order and objects. The freelist pointer corruption occurs if some slab flags are modified whileCONFIG_SLAB_FREELIST_RANDOMis turned on.
Quoted 7 lines
$ echo 0 > /sys/kernel/slab/zs_handle/store_user $ echo 0 > /sys/kernel/slab/zspage/store_user $ mkswap /dev/block/zram0 $ swapon /dev/block/zram0 -p 32758=============================================================================
BUGzs_handle(Not tainted): Freepointer corrupt
The problems here are actually larger than just the freelist pointers, so this was actually solved by just making these parameters not writable at runtime:
https://git.kernel.org/linus/ad38b5b1131e2a0e5c46724847da2e1eba31fb68
I wonder if perhaps the above patch needs to be explicitly sent to the -stable trees?
-Kees
Quoted 79 lines
-----------------------------------------------------------------------------
Disabling lock debugging due to kernel taint
INFO: Slab0xffffffbf29603600objects=102 used=102 fp=0x0000000000000000 flags=0x0200
INFO: Object0xffffffca580d8d78@offset=3448 fp=0xffffffca580d8ed0Redzone
00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
Object0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
Redzone000000008fd80359: bb bb bb bb bb bb bb bb ........
Padding00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZIn this example, an Android device tries to use zram as a swap and to turn off
store_userin order to reduce the slub object size. Whencalculate_sizesis called inkmem_cache_open, size, order and objects forzs_handleis:
size:360, order:0, objects:22
However, if theSLAB_STORE_USERbit is cleared instore_user_store:
size: 56, order:1, objects:73All the size, order, and objects is changed by
calculate_sizes(), but the size of therandom_seqarray is still old value(22). As a result, out-of-bound array access can occur atshuffle_freelist()when slab allocation is requested.This patch fixes the problem by re-allocating the
random_seqarray with re-calculated correct objects value.Fixes 210e7a43fa905 ("mm: SLUB freelist randomization")Reported-by Ari-Pekka Verta <[email protected]>Reported-by Timo Simola <[email protected]>Signed-off-by Sahara <[email protected]>--- mm/slub.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)mm/slub.c +16
-7@@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)if (oo_objects (s->oo) >oo_objects (s->max)) s->max = s->oo; -return !!oo_objects (s->oo); +if (!oo_objects (s->oo)) +return 0 ; + +/* +* Initialize the pre-computed randomized freelist if slab is up. +* If the randomized freelist random_seq is already initialized, +* free and re-initialize it with re-calculated value. +*/ +if (slab_state >=UP ) { +if (s->random_seq) +cache_random_seq_destroy (s); +if (init_cache_random_seq (s)) +return 0 ; + } + +return 1 ; }static int kmem_cache_open (struct kmem_cache *s,slab_flags_t flags)@@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) s->remote_node_defrag_ratio =1000 ;#endif -/* Initialize the pre-computed randomized freelist if slab is up */ -if (slab_state >=UP ) { -if (init_cache_random_seq (s)) - goto error; - } -if (!init_kmem_cache_nodes (s)) goto error;Signature · 1 line
2.17.1
Signature · 1 line
Kees Cook