kasan: solve redzone overwritten issue at debug
Subsystems (4)
v6 cover kasan: solve redzone overwritten issue at debug
From: Yee Lee <[email protected]>
Issue: In SLUB debug, hwtag kasan_unpoison() would overwrite the redzone in those objects with unaligned size.
The first patch Introduces slub_debug_enable_unlikely() to check the state of debug mode.
The second patch Adds memzero_explict() to separate the initialization for such condition. The new code path is executed about 1.1% during nromal booting process. The penalty is acceptable since it only works in debug mode.
=============
Exp: QEMUv5.2(+mte)/SLUB_debug mode
code path exec : 941/80854 (1.1%)
Changed since v6:
- Move helper instead of introducing a new one.
Changed since v5:
- Fix format
Changed since v4:
- Introduce slub_debug_enable_unlikly() to check the debug state.
- Include "slab.h" and Add slub_debug_enable_unlikly() to lead the condition statement.
- Add comment block about this new code path in source code.
Changed since v3:
- Apply IS_ENABLED to wrap codes under SLUB debug mode.
- Replace memset() by memzero_explict().
---
Marco Elver (1):
mm: move helper to check slub_debug_enabled
Yee Lee (1):
kasan: Add memzero int for unaligned size at DEBUG
mm/kasan/kasan.h | 12 ++++++++++++ mm/slab.h | 15 +++++++++++---- mm/slub.c | 14 -------------- 3 files changed, 23 insertions(+), 18 deletions(-)
Signature · 1 line
2.18.0
PATCH v6 1/2 mm: move helper to check slub_debug_enabled
From: Marco Elver <[email protected]>
Move the helper to check slub_debug_enabled, so that we can confine the use of #ifdef outside slub.c as well.
--- mm/slab.h | 15 +++++++++++---- mm/slub.c | 14 -------------- 2 files changed, 11 insertions(+), 18 deletions(-)
mm/slab.h +11 -4
@@ -216,10 +216,18 @@ DECLARE_STATIC_KEY_FALSE(slub_debug_enabled);#endif extern void print_tracking (struct kmem_cache *s,void *object);long validate_slab_cache (struct kmem_cache *s); +static inline bool __slub_debug_enabled (void ) +{ +return static_branch_unlikely (&slub_debug_enabled); +}#else static inline void print_tracking (struct kmem_cache *s,void *object) { } +static inline bool __slub_debug_enabled (void ) +{ +return false; +}#endif /*
@@ -229,11 +237,10 @@ static inline void print_tracking(struct kmem_cache *s, void *object)*/ static inline bool kmem_cache_debug_flags (struct kmem_cache *s,slab_flags_t flags) { -#ifdef CONFIG_SLUB_DEBUG -VM_WARN_ON_ONCE (!(flags &SLAB_DEBUG_FLAGS )); -if (static_branch_unlikely (&slub_debug_enabled)) +if (IS_ENABLED (CONFIG_SLUB_DEBUG )) +VM_WARN_ON_ONCE (!(flags &SLAB_DEBUG_FLAGS )); +if (__slub_debug_enabled ())return s->flags & flags; -#endif return false; }
mm/slub.c +0 -14
@@ -119,25 +119,11 @@*/ #ifdef CONFIG_SLUB_DEBUG -#ifdef CONFIG_SLUB_DEBUG_ON DEFINE_STATIC_KEY_TRUE (slub_debug_enabled);#else DEFINE_STATIC_KEY_FALSE (slub_debug_enabled);#endif - -static inline bool __slub_debug_enabled (void ) -{ -return static_branch_unlikely (&slub_debug_enabled); -} - -#else /* CONFIG_SLUB_DEBUG */ - -static inline bool __slub_debug_enabled (void ) -{ -return false; -} -#endif /* CONFIG_SLUB_DEBUG */ static inline bool kmem_cache_debug (struct kmem_cache *s)
Signature · 1 line
2.18.0
PATCH v6 2/2 kasan: Add memzero int for unaligned size at DEBUG
In reply to [email protected]From: Yee Lee <[email protected]>
Issue: when SLUB debug is on, hwtag kasan_unpoison() would overwrite the redzone of object with unaligned size.
An additional memzero_explicit() path is added to replacing init by hwtag instruction for those unaligned size at SLUB debug mode.
The penalty is acceptable since they are only enabled in debug mode, not production builds. A block of comment is added for explanation.
--- mm/kasan/kasan.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)
mm/kasan/kasan.h +12 -0
@@ -9,6 +9,7 @@#ifdef CONFIG_KASAN_HW_TAGS #include <linux/static_key.h> +#include "../slab.h" DECLARE_STATIC_KEY_FALSE (kasan_flag_stacktrace);extern bool kasan_flag_async__ro_after_init ;
@@ -387,6 +388,17 @@ static inline void kasan_unpoison(const void *addr, size_t size, bool init)if (WARN_ON ((unsigned long )addr &KASAN_GRANULE_MASK ))return ; +/* +* Explicitly initialize the memory with the precise object size to +* avoid overwriting the SLAB redzone. This disables initialization in +* the arch code and may thus lead to performance penalty. The penalty +* is accepted since SLAB redzones aren't enabled in production builds. +*/ +if (__slub_debug_enabled () && + init && ((unsigned long )size &KASAN_GRANULE_MASK )) { + init = false; +memzero_explicit ((void *)addr, size); + } size =round_up (size,KASAN_GRANULE_SIZE );hw_set_mem_tag_range ((void *)addr, size, tag, init);
Signature · 1 line
2.18.0
Re: [PATCH v6 2/2] kasan: Add memzero int for unaligned size at DEBUG
Quoted 11 lines · On Mon, 5 Jul 2021 at 12:33, <[email protected]> wrote:
From: Yee Lee <[email protected]>
Issue: when SLUB debug is on, hwtag
kasan_unpoison()would overwrite the redzone of object with unaligned size.An additional
memzero_explicit()path is added to replacing init by hwtag instruction for those unaligned size at SLUB debug mode.The penalty is acceptable since they are only enabled in debug mode, not production builds. A block of comment is added for explanation.
Quoted 7 lines
Cc Andrey Ryabinin <[email protected]>Cc Alexander Potapenko <[email protected]>Cc Dmitry Vyukov <[email protected]>Cc Andrew Morton <[email protected]>Suggested-by Marco Elver <[email protected]>Suggested-by Andrey Konovalov <[email protected]>Signed-off-by Yee Lee <[email protected]>
Thank you!
Quoted 36 lines
--- mm/kasan/kasan.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)mm/kasan/kasan.h +12
-1@@ -9,6 +9,7 @@#ifdef CONFIG_KASAN_HW_TAGS #include <linux/static_key.h> +#include "../slab.h" DECLARE_STATIC_KEY_FALSE (kasan_flag_stacktrace);extern bool kasan_flag_async__ro_after_init ;@@ -387,6 +388,17 @@ static inline void kasan_unpoison(const void *addr, size_t size, bool init)if (WARN_ON ((unsigned long )addr &KASAN_GRANULE_MASK ))return ; +/* +* Explicitly initialize the memory with the precise object size to +* avoid overwriting the SLAB redzone. This disables initialization in +* the arch code and may thus lead to performance penalty. The penalty +* is accepted since SLAB redzones aren't enabled in production builds. +*/ +if (__slub_debug_enabled () && + init && ((unsigned long )size &KASAN_GRANULE_MASK )) { + init = false; +memzero_explicit ((void *)addr, size); + } size =round_up (size,KASAN_GRANULE_SIZE );hw_set_mem_tag_range ((void *)addr, size, tag, init); --2.18.0
Andrey Konovalov
reviewerRe: [PATCH v6 2/2] kasan: Add memzero int for unaligned size at DEBUG
In reply to [email protected]Quoted 36 lines · On Mon, Jul 5, 2021 at 12:33 PM <[email protected]> wrote:
From: Yee Lee <[email protected]>
Issue: when SLUB debug is on, hwtag
kasan_unpoison()would overwrite the redzone of object with unaligned size.An additional
memzero_explicit()path is added to replacing init by hwtag instruction for those unaligned size at SLUB debug mode.The penalty is acceptable since they are only enabled in debug mode, not production builds. A block of comment is added for explanation.
Cc Andrey Ryabinin <[email protected]>Cc Alexander Potapenko <[email protected]>Cc Dmitry Vyukov <[email protected]>Cc Andrew Morton <[email protected]>Suggested-by Marco Elver <[email protected]>Suggested-by Andrey Konovalov <[email protected]>Signed-off-by Yee Lee <[email protected]>--- mm/kasan/kasan.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)mm/kasan/kasan.h +1
-0@@ -9,6 +9,7 @@#ifdef CONFIG_KASAN_HW_TAGS #include <linux/static_key.h> +#include "../slab.h" DECLARE_STATIC_KEY_FALSE (kasan_flag_stacktrace);extern bool kasan_flag_async__ro_after_init ;
mm/kasan/kasan.h static inline void kasan_unpoison(const void *addr, size_t size, bool init) · line 397
Quoted 10 lines
mm/kasan/kasan.h +7
-0@@ -387,6 +388,17 @@ static inline void kasan_unpoison(const void *addr, size_t size, bool init)if (WARN_ON ((unsigned long )addr &KASAN_GRANULE_MASK ))return ; +/* +* Explicitly initialize the memory with the precise object size to +* avoid overwriting the SLAB redzone. This disables initialization in +* the arch code and may thus lead to performance penalty. The penalty +* is accepted since SLAB redzones aren't enabled in production builds. +*/ + if (__slub_debug_enabled() &&
What happened to slub_debug_enabled_unlikely()? Was it renamed? Why? I didn't receive patch #1 of v6 (nor of v5).
Quoted 10 lines
mm/kasan/kasan.h +4
-1+ init && ((unsigned long)size & KASAN_GRANULE_MASK)) { + init = false; +memzero_explicit ((void *)addr, size); + } size =round_up (size,KASAN_GRANULE_SIZE );hw_set_mem_tag_range ((void *)addr, size, tag, init); --2.18.0
Re: [PATCH v6 2/2] kasan: Add memzero int for unaligned size at DEBUG
On Mon, 5 Jul 2021 at 13:12, Andrey Konovalov <[email protected]> wrote:
[...]
Quoted 10 lines
mm/kasan/kasan.h static inline void kasan_unpoison(const void *addr, size_t size, bool init) · line 397
Quoted 7 lines
mm/kasan/kasan.h +7
-0+/* +* Explicitly initialize the memory with the precise object size to +* avoid overwriting the SLAB redzone. This disables initialization in +* the arch code and may thus lead to performance penalty. The penalty +* is accepted since SLAB redzones aren't enabled in production builds. +*/ + if (__slub_debug_enabled() &&What happened to
slub_debug_enabled_unlikely()? Was it renamed? Why? I didn't receive patch #1 of v6 (nor of v5).
Somebody had the same idea with the helper:
https://lkml.kernel.org/r/[email protected]
and Matthew didn't like the _unlikely() prefix.
Which meant we should just move the existing helper introduced in the merge window.
Patch 1/2: https://lkml.kernel.org/r/[email protected]
Thanks,
-- Marco
Andrey Konovalov
reviewerRe: [PATCH v6 2/2] kasan: Add memzero int for unaligned size at DEBUG
Quoted 14 lines · Marco Elver
On Mon, 5 Jul 2021 at 13:12, Andrey Konovalov <[email protected]> wrote:
[...]Quoted 10 lines
mm/kasan/kasan.h static inline void kasan_unpoison(const void *addr, size_t size, bool init) · line 397
Quoted 7 lines
mm/kasan/kasan.h +7
-0+/* +* Explicitly initialize the memory with the precise object size to +* avoid overwriting the SLAB redzone. This disables initialization in +* the arch code and may thus lead to performance penalty. The penalty +* is accepted since SLAB redzones aren't enabled in production builds. +*/ + if (__slub_debug_enabled() &&What happened to
slub_debug_enabled_unlikely()? Was it renamed? Why? I didn't receive patch #1 of v6 (nor of v5).
Quoted 8 lines
Somebody had the same idea with the helper:
https://lkml.kernel.org/r/[email protected]
and Matthew didn't like the_unlikely()prefix.Which meant we should just move the existing helper introduced in the merge window.
Patch 1/2: https://lkml.kernel.org/r/[email protected]
Got it. Thank you, Marco!
Re: [PATCH v6 2/2] kasan: Add memzero int for unaligned size at DEBUG
Thank you, Macro. I thought members in "suggested-by" would be put in the list as well... And thank you guys for the review these days.
@Andrew Motom
Hi Andrew,
Could you help to push the patches? We are dealing with the issue and would backport to Android porject right after the action. Appreciated!
BR,
Yee
Quoted 33 lines · Andrey Konovalov
Quoted 24 lines · Marco Elver
On Mon, 5 Jul 2021 at 13:12, Andrey Konovalov <[email protected]
Quoted 1 line
wrote:
[...]
Quoted 12 lines
mm/kasan/kasan.h static inline void kasan_unpoison(const void *addr, size_t size, bool init) · line 397
Quoted 11 lines
(quoted diff) +2
-0+ /* + * Explicitly initialize the memory with the preciseobject size to
+ * avoid overwriting the SLAB redzone. This disables
initialization in
+ * the arch code and may thus lead to performance
penalty. The penalty
+ * is accepted since SLAB redzones aren't enabled in
production builds.mm/kasan/kasan.h +2
-0+*/ + if (__slub_debug_enabled() &&Quoted 3 lines
What happened to
slub_debug_enabled_unlikely()? Was it renamed? Why? I
didn't receive patch #1 of v6 (nor of v5).Somebody had the same idea with the helper:
https://lkml.kernel.org/r/[email protected]
and Matthew didn't like the_unlikely()prefix.Quoted 6 lines
Which meant we should just move the existing helper introduced in the
merge window.Got it. Thank you, Marco!
Re: [PATCH v6 2/2] kasan: Add memzero int for unaligned size at DEBUG
In reply to [email protected]Quoted 54 lines · On Mon, Jul 5, 2021 at 12:33 PM <[email protected]> wrote:
From: Yee Lee <[email protected]>
Issue: when SLUB debug is on, hwtag
kasan_unpoison()would overwrite the redzone of object with unaligned size.An additional
memzero_explicit()path is added to replacing init by hwtag instruction for those unaligned size at SLUB debug mode.The penalty is acceptable since they are only enabled in debug mode, not production builds. A block of comment is added for explanation.
Cc Andrey Ryabinin <[email protected]>Cc Alexander Potapenko <[email protected]>Cc Dmitry Vyukov <[email protected]>Cc Andrew Morton <[email protected]>Suggested-by Marco Elver <[email protected]>Suggested-by Andrey Konovalov <[email protected]>Signed-off-by Yee Lee <[email protected]>--- mm/kasan/kasan.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)mm/kasan/kasan.h +12
-1@@ -9,6 +9,7 @@#ifdef CONFIG_KASAN_HW_TAGS #include <linux/static_key.h> +#include "../slab.h" DECLARE_STATIC_KEY_FALSE (kasan_flag_stacktrace);extern bool kasan_flag_async__ro_after_init ;@@ -387,6 +388,17 @@ static inline void kasan_unpoison(const void *addr, size_t size, bool init)if (WARN_ON ((unsigned long )addr &KASAN_GRANULE_MASK ))return ; +/* +* Explicitly initialize the memory with the precise object size to +* avoid overwriting the SLAB redzone. This disables initialization in +* the arch code and may thus lead to performance penalty. The penalty +* is accepted since SLAB redzones aren't enabled in production builds. +*/ +if (__slub_debug_enabled () && + init && ((unsigned long )size &KASAN_GRANULE_MASK )) { + init = false; +memzero_explicit ((void *)addr, size); + } size =round_up (size,KASAN_GRANULE_SIZE );hw_set_mem_tag_range ((void *)addr, size, tag, init); --
Quoted 2 lines
2.18.0