riscv: fix race when vmap stack overflow
PATCH v3 1/1 riscv: fix race when vmap stack overflow
Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.
To solve this race, we introduce spin_shadow_stack atomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means the shadow_stack is being used; when the var is cleared, it means the shadow_stack isn't being used.
---
Since v2:
- use REG_AMOSWAP
- add comment to the purpose of smp_store_release()
Since v1:
- use smp_store_release directly
- use unsigned int instead of atomic_t
arch/riscv/include/asm/asm.h | 1 + arch/riscv/kernel/entry.S | 4 ++++ arch/riscv/kernel/traps.c | 9 +++++++++ 3 files changed, 14 insertions(+)
arch/riscv/include/asm/asm.h +1 -0
@@ -23,6 +23,7 @@#define REG_L __REG_SEL(ld, lw)#define REG_S __REG_SEL(sd, sw)#define REG_SC __REG_SEL(sc.d, sc.w) +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w)#define REG_ASM __REG_SEL(.dword, .word)#define SZREG __REG_SEL(8, 4)#define LGREG __REG_SEL(3, 2)
arch/riscv/kernel/entry.S +4 -0
@@ -404,6 +404,10 @@ handle_syscall_trace_exit: #ifdef CONFIG_VMAP_STACK handle_kernel_stack_overflow: +1: la sp, spin_shadow_stack + REG_AMOSWAP sp, sp, (sp) + bnez sp, 1b + la sp, shadow_stack addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
arch/riscv/kernel/traps.c +9 -0
@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)OVERFLOW_STACK_SIZE ; } +unsigned long spin_shadow_stack; + asmlinkage void handle_bad_stack(struct pt_regs *regs) {unsigned long tsk_stk = (unsigned long )current->stack;unsigned long ovf_stk = (unsigned long )this_cpu_ptr (overflow_stack); +/* +* to ensure spin flag is set after the sp is used in entry.S: +* //load per-cpu overflow stack. +* REG_L sp, -8(sp) +*/ +smp_store_release (&spin_shadow_stack,0 ); +console_verbose ();pr_emerg ("Insufficient stack space to handle exception!\n" );
Signature · 1 line
2.37.2
Quoted 85 lines · Jisheng Zhang
Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the
get_overflow_stack()to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.To solve this race, we introduce
spin_shadow_stackatomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means theshadow_stackis being used; when the var is cleared, it means theshadow_stackisn't being used.Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")Signed-off-by Jisheng Zhang <[email protected]>Suggested-by Guo Ren <[email protected]>---
Since v2:
- useREG_AMOSWAP
- add comment to the purpose ofsmp_store_release()Since v1:
- usesmp_store_releasedirectly
- use unsigned int instead ofatomic_tarch/riscv/include/asm/asm.h | 1 + arch/riscv/kernel/entry.S | 4 ++++ arch/riscv/kernel/traps.c | 9 +++++++++ 3 files changed, 14 insertions(+)arch/riscv/include/asm/asm.h +1
-0@@ -23,6 +23,7 @@#define REG_L __REG_SEL(ld, lw)#define REG_S __REG_SEL(sd, sw)#define REG_SC __REG_SEL(sc.d, sc.w) +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w)#define REG_ASM __REG_SEL(.dword, .word)#define SZREG __REG_SEL(8, 4)#define LGREG __REG_SEL(3, 2)arch/riscv/kernel/entry.S +4
-0@@ -404,6 +404,10 @@ handle_syscall_trace_exit: #ifdef CONFIG_VMAP_STACK handle_kernel_stack_overflow: +1: la sp, spin_shadow_stack + REG_AMOSWAP sp, sp, (sp) + bnez sp, 1b + la sp, shadow_stack addi sp, sp, SHADOW_OVERFLOW_STACK_SIZEarch/riscv/kernel/traps.c +9
-1@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)OVERFLOW_STACK_SIZE ; } +unsigned long spin_shadow_stack; + asmlinkage void handle_bad_stack(struct pt_regs *regs) {unsigned long tsk_stk = (unsigned long )current->stack;unsigned long ovf_stk = (unsigned long )this_cpu_ptr (overflow_stack); +/* +* to ensure spin flag is set after the sp is used in entry.S: +* //load per-cpu overflow stack. +* REG_L sp, -8(sp) +*/ +smp_store_release (&spin_shadow_stack,0 ); +console_verbose ();pr_emerg ("Insufficient stack space to handle exception!\n" ); --2.37.2
Signature · 2 lines
Best Regards Guo Ren
Quoted 2 lines · Guo Ren
Reviewed-by Guo Ren <[email protected]>
Gentle ping? This patch is still missing in mainline.
Quoted 91 lines · Jisheng Zhang
Quoted 85 lines
Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the
get_overflow_stack()to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.To solve this race, we introduce
spin_shadow_stackatomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means theshadow_stackis being used; when the var is cleared, it means theshadow_stackisn't being used.Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")Signed-off-by Jisheng Zhang <[email protected]>Suggested-by Guo Ren <[email protected]>---
Since v2:
- useREG_AMOSWAP
- add comment to the purpose ofsmp_store_release()Since v1:
- usesmp_store_releasedirectly
- use unsigned int instead ofatomic_tarch/riscv/include/asm/asm.h | 1 + arch/riscv/kernel/entry.S | 4 ++++ arch/riscv/kernel/traps.c | 9 +++++++++ 3 files changed, 14 insertions(+)arch/riscv/include/asm/asm.h +1
-0@@ -23,6 +23,7 @@#define REG_L __REG_SEL(ld, lw)#define REG_S __REG_SEL(sd, sw)#define REG_SC __REG_SEL(sc.d, sc.w) +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w)#define REG_ASM __REG_SEL(.dword, .word)#define SZREG __REG_SEL(8, 4)#define LGREG __REG_SEL(3, 2)arch/riscv/kernel/entry.S +4
-0@@ -404,6 +404,10 @@ handle_syscall_trace_exit: #ifdef CONFIG_VMAP_STACK handle_kernel_stack_overflow: +1: la sp, spin_shadow_stack + REG_AMOSWAP sp, sp, (sp) + bnez sp, 1b + la sp, shadow_stack addi sp, sp, SHADOW_OVERFLOW_STACK_SIZEarch/riscv/kernel/traps.c +9
-1@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)OVERFLOW_STACK_SIZE ; } +unsigned long spin_shadow_stack; + asmlinkage void handle_bad_stack(struct pt_regs *regs) {unsigned long tsk_stk = (unsigned long )current->stack;unsigned long ovf_stk = (unsigned long )this_cpu_ptr (overflow_stack); +/* +* to ensure spin flag is set after the sp is used in entry.S: +* //load per-cpu overflow stack. +* REG_L sp, -8(sp) +*/ +smp_store_release (&spin_shadow_stack,0 ); +console_verbose ();pr_emerg ("Insufficient stack space to handle exception!\n" ); --2.37.2
Signature · 2 lines
Best Regards Guo Ren
Palmer Dabbelt
maintainerQuoted 5 lines · On Wed, 23 Nov 2022 07:23:51 PST (-0800), [email protected] wrote:
Quoted 2 lines · Guo Ren
Reviewed-by Guo Ren <[email protected]>Gentle ping? This patch is still missing in mainline.
Sorry, this one took a few looks to figure out. I just sent out a v4, it's pretty much the same thing but by the time I'd figured out what this was doing it seemed easier to just write up the comments directly.
https://lore.kernel.org/r/[email protected]/
I'll try and watch the threads, LMK if I screwed anything up. I've already got this queued up in staging so it's being tested, should be fast to take it onto fixes.
Thanks!
Quoted 92 lines
Quoted 92 lines · Jisheng Zhang
Quoted 83 lines
Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the
get_overflow_stack()to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.To solve this race, we introduce
spin_shadow_stackatomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means theshadow_stackis being used; when the var is cleared, it means theshadow_stackisn't being used.Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")Signed-off-by Jisheng Zhang <[email protected]>Suggested-by Guo Ren <[email protected]>---
Since v2:
- useREG_AMOSWAP
- add comment to the purpose ofsmp_store_release()Since v1:
- usesmp_store_releasedirectly
- use unsigned int instead ofatomic_tarch/riscv/include/asm/asm.h | 1 + arch/riscv/kernel/entry.S | 4 ++++ arch/riscv/kernel/traps.c | 9 +++++++++ 3 files changed, 14 insertions(+)arch/riscv/include/asm/asm.h +1
-0@@ -23,6 +23,7 @@#define REG_L __REG_SEL(ld, lw)#define REG_S __REG_SEL(sd, sw)#define REG_SC __REG_SEL(sc.d, sc.w) +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w)#define REG_ASM __REG_SEL(.dword, .word)#define SZREG __REG_SEL(8, 4)#define LGREG __REG_SEL(3, 2)arch/riscv/kernel/entry.S +4
-0@@ -404,6 +404,10 @@ handle_syscall_trace_exit: #ifdef CONFIG_VMAP_STACK handle_kernel_stack_overflow: +1: la sp, spin_shadow_stack + REG_AMOSWAP sp, sp, (sp) + bnez sp, 1b + la sp, shadow_stack addi sp, sp, SHADOW_OVERFLOW_STACK_SIZEarch/riscv/kernel/traps.c +9
-1@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)OVERFLOW_STACK_SIZE ; } +unsigned long spin_shadow_stack; + asmlinkage void handle_bad_stack(struct pt_regs *regs) {unsigned long tsk_stk = (unsigned long )current->stack;unsigned long ovf_stk = (unsigned long )this_cpu_ptr (overflow_stack); +/* +* to ensure spin flag is set after the sp is used in entry.S: +* //load per-cpu overflow stack. +* REG_L sp, -8(sp) +*/ +smp_store_release (&spin_shadow_stack,0 ); +console_verbose ();pr_emerg ("Insufficient stack space to handle exception!\n" ); --Quoted 2 lines
2.37.2
--
Best Regards
Guo Ren