Skip to contentSkip to conversation

riscv: fix race when vmap stack overflow

PATCH In review Jisheng Zhang 3 participants 4 messages Oct 30, 2022 – Nov 30, 2022 RISC-V

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 riscv: fix race when vmap stack overflow

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means the shadow_stack is being used; when the var is cleared, it means the shadow_stack isn't being used.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")

---
Since v2:
- use REG_AMOSWAP
- add comment to the purpose of smp_store_release()

Since v1:
- use smp_store_release directly
- use unsigned int instead of atomic_t

 arch/riscv/include/asm/asm.h | 1 +
 arch/riscv/kernel/entry.S    | 4 ++++
 arch/riscv/kernel/traps.c    | 9 +++++++++
 3 files changed, 14 insertions(+)
arch/riscv/include/asm/asm.h +1 -0
index 1b471ff73178..acf563072b8b 100644
@@ -23,6 +23,7 @@
 #define REG_L		__REG_SEL(ld, lw)
 #define REG_S		__REG_SEL(sd, sw)
 #define REG_SC		__REG_SEL(sc.d, sc.w)
+#define REG_AMOSWAP	__REG_SEL(amoswap.d, amoswap.w)
 #define REG_ASM		__REG_SEL(.dword, .word)
 #define SZREG		__REG_SEL(8, 4)
 #define LGREG		__REG_SEL(3, 2)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..ea6b78dac739 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:	la sp, spin_shadow_stack
+	REG_AMOSWAP sp, sp, (sp)
+	bnez sp, 1b
+
 	la sp, shadow_stack
 	addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +9 -0
index f3e96d60a2ff..dea47f329708 100644
@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)
 		OVERFLOW_STACK_SIZE;
 }
 
+unsigned long spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
 	unsigned long tsk_stk = (unsigned long)current->stack;
 	unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+	/*
+	 * to ensure spin flag is set after the sp is used in entry.S:
+	 * //load per-cpu overflow stack.
+	 * REG_L sp, -8(sp)
+	 */
+	smp_store_release(&spin_shadow_stack, 0);
+
 	console_verbose();
 
 	pr_emerg("Insufficient stack space to handle exception!\n");
Signature · 1 line
2.37.2
Reviewed-by on v3 1/1
#1 lore raw
Quoted 85 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means the shadow_stack is being used; when the var is cleared, it means the shadow_stack isn't being used.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")

---
Since v2:
- use REG_AMOSWAP
- add comment to the purpose of smp_store_release()

Since v1:
- use smp_store_release directly
- use unsigned int instead of atomic_t

 arch/riscv/include/asm/asm.h | 1 +
 arch/riscv/kernel/entry.S    | 4 ++++
 arch/riscv/kernel/traps.c    | 9 +++++++++
 3 files changed, 14 insertions(+)
arch/riscv/include/asm/asm.h +1 -0
index 1b471ff73178..acf563072b8b 100644
@@ -23,6 +23,7 @@
 #define REG_L          __REG_SEL(ld, lw)
 #define REG_S          __REG_SEL(sd, sw)
 #define REG_SC         __REG_SEL(sc.d, sc.w)
+#define REG_AMOSWAP    __REG_SEL(amoswap.d, amoswap.w)
 #define REG_ASM                __REG_SEL(.dword, .word)
 #define SZREG          __REG_SEL(8, 4)
 #define LGREG          __REG_SEL(3, 2)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..ea6b78dac739 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       REG_AMOSWAP sp, sp, (sp)
+       bnez sp, 1b
+
        la sp, shadow_stack
        addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +9 -1
index f3e96d60a2ff..dea47f329708 100644
@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)
                OVERFLOW_STACK_SIZE;
 }
 
+unsigned long spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
        unsigned long tsk_stk = (unsigned long)current->stack;
        unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+       /*
+        * to ensure spin flag is set after the sp is used in entry.S:
+        * //load per-cpu overflow stack.
+        * REG_L sp, -8(sp)
+        */
+       smp_store_release(&spin_shadow_stack, 0);
+
        console_verbose();
 
        pr_emerg("Insufficient stack space to handle exception!\n");
--

2.37.2

Signature · 2 lines
Best Regards
 Guo Ren
#2 lore raw
Quoted 2 lines · Guo Ren

Gentle ping? This patch is still missing in mainline.

Quoted 91 lines · Jisheng Zhang
Quoted 85 lines

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means the shadow_stack is being used; when the var is cleared, it means the shadow_stack isn't being used.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")

---
Since v2:
- use REG_AMOSWAP
- add comment to the purpose of smp_store_release()

Since v1:
- use smp_store_release directly
- use unsigned int instead of atomic_t

 arch/riscv/include/asm/asm.h | 1 +
 arch/riscv/kernel/entry.S    | 4 ++++
 arch/riscv/kernel/traps.c    | 9 +++++++++
 3 files changed, 14 insertions(+)
arch/riscv/include/asm/asm.h +1 -0
index 1b471ff73178..acf563072b8b 100644
@@ -23,6 +23,7 @@
 #define REG_L          __REG_SEL(ld, lw)
 #define REG_S          __REG_SEL(sd, sw)
 #define REG_SC         __REG_SEL(sc.d, sc.w)
+#define REG_AMOSWAP    __REG_SEL(amoswap.d, amoswap.w)
 #define REG_ASM                __REG_SEL(.dword, .word)
 #define SZREG          __REG_SEL(8, 4)
 #define LGREG          __REG_SEL(3, 2)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..ea6b78dac739 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       REG_AMOSWAP sp, sp, (sp)
+       bnez sp, 1b
+
        la sp, shadow_stack
        addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +9 -1
index f3e96d60a2ff..dea47f329708 100644
@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)
                OVERFLOW_STACK_SIZE;
 }
 
+unsigned long spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
        unsigned long tsk_stk = (unsigned long)current->stack;
        unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+       /*
+        * to ensure spin flag is set after the sp is used in entry.S:
+        * //load per-cpu overflow stack.
+        * REG_L sp, -8(sp)
+        */
+       smp_store_release(&spin_shadow_stack, 0);
+
        console_verbose();
 
        pr_emerg("Insufficient stack space to handle exception!\n");
--

2.37.2

Signature · 2 lines
Best Regards
 Guo Ren
maintainer
#3 lore raw
Quoted 5 lines · On Wed, 23 Nov 2022 07:23:51 PST (-0800), [email protected] wrote:
Quoted 2 lines · Guo Ren

Gentle ping? This patch is still missing in mainline.

Sorry, this one took a few looks to figure out. I just sent out a v4, it's pretty much the same thing but by the time I'd figured out what this was doing it seemed easier to just write up the comments directly.

https://lore.kernel.org/r/[email protected]/

I'll try and watch the threads, LMK if I screwed anything up. I've already got this queued up in staging so it's being tested, should be fast to take it onto fixes.

Thanks!

Quoted 92 lines
Quoted 92 lines · Jisheng Zhang
Quoted 83 lines

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means the shadow_stack is being used; when the var is cleared, it means the shadow_stack isn't being used.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")

---
Since v2:
- use REG_AMOSWAP
- add comment to the purpose of smp_store_release()

Since v1:
- use smp_store_release directly
- use unsigned int instead of atomic_t

 arch/riscv/include/asm/asm.h | 1 +
 arch/riscv/kernel/entry.S    | 4 ++++
 arch/riscv/kernel/traps.c    | 9 +++++++++
 3 files changed, 14 insertions(+)
arch/riscv/include/asm/asm.h +1 -0
index 1b471ff73178..acf563072b8b 100644
@@ -23,6 +23,7 @@
 #define REG_L          __REG_SEL(ld, lw)
 #define REG_S          __REG_SEL(sd, sw)
 #define REG_SC         __REG_SEL(sc.d, sc.w)
+#define REG_AMOSWAP    __REG_SEL(amoswap.d, amoswap.w)
 #define REG_ASM                __REG_SEL(.dword, .word)
 #define SZREG          __REG_SEL(8, 4)
 #define LGREG          __REG_SEL(3, 2)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..ea6b78dac739 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       REG_AMOSWAP sp, sp, (sp)
+       bnez sp, 1b
+
        la sp, shadow_stack
        addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +9 -1
index f3e96d60a2ff..dea47f329708 100644
@@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void)
                OVERFLOW_STACK_SIZE;
 }
 
+unsigned long spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
        unsigned long tsk_stk = (unsigned long)current->stack;
        unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+       /*
+        * to ensure spin flag is set after the sp is used in entry.S:
+        * //load per-cpu overflow stack.
+        * REG_L sp, -8(sp)
+        */
+       smp_store_release(&spin_shadow_stack, 0);
+
        console_verbose();
 
        pr_emerg("Insufficient stack space to handle exception!\n");
--
Quoted 2 lines

2.37.2

--
Best Regards
Guo Ren