Skip to contentSkip to conversation

riscv: fix race when vmap stack overflow

PATCH Superseded Jisheng Zhang 4 participants 14 messages Oct 19, 2022 – Oct 21, 2022 RISC-V

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 riscv: fix race when vmap stack overflow

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..7b924b16792b 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:	la sp, spin_shadow_stack
+	amoswap.w sp, sp, (sp)
+	bnez sp, 1b
+
 	la sp, shadow_stack
 	addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +4 -0
index f3e96d60a2ff..88a54947dffb 100644
@@ -221,11 +221,15 @@ asmlinkage unsigned long get_overflow_stack(void)
 		OVERFLOW_STACK_SIZE;
 }
 
+atomic_t spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
 	unsigned long tsk_stk = (unsigned long)current->stack;
 	unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+	atomic_set_release(&spin_shadow_stack, 0);
+
 	console_verbose();
 
 	pr_emerg("Insufficient stack space to handle exception!\n");
Signature · 1 line
2.37.2
Reviewed-by on v1 1/1
#1 lore raw
Quoted 56 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..7b924b16792b 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       amoswap.w sp, sp, (sp)
+       bnez sp, 1b
+
        la sp, shadow_stack
        addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +4 -1
index f3e96d60a2ff..88a54947dffb 100644
@@ -221,11 +221,15 @@ asmlinkage unsigned long get_overflow_stack(void)
                OVERFLOW_STACK_SIZE;
 }
 
+atomic_t spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
        unsigned long tsk_stk = (unsigned long)current->stack;
        unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+       atomic_set_release(&spin_shadow_stack, 0);
+
        console_verbose();
 
        pr_emerg("Insufficient stack space to handle exception!\n");
--

2.37.2

Signature · 2 lines
Best Regards
 Guo Ren
#2 lore raw
In reply to Jisheng Zhang
Quoted 23 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +0 -0
index b9eda3fcbd6d..7b924b16792b 100644

arch/riscv/kernel/entry.S handle_syscall_trace_exit: · line 408

Quoted 5 lines
arch/riscv/kernel/entry.S +2 -0
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       amoswap.w sp, sp, (sp)

If CONFIG_64BIT=y, it would be broken. Because we only hold 32bit of the sp, and the next loop would get the wrong sp value of &spin_shadow_stack.
Here is the correction.
-----

arch/riscv/include/asm/asm.h +1 -0
index 1b471ff73178..acf563072b8b 100644
@@ -23,6 +23,7 @@
 #define REG_L          __REG_SEL(ld, lw)
 #define REG_S          __REG_SEL(sd, sw)
 #define REG_SC         __REG_SEL(sc.d, sc.w)
+#define REG_AMOSWAP    __REG_SEL(amoswap.d, amoswap.w)
 #define REG_ASM                __REG_SEL(.dword, .word)
 #define SZREG          __REG_SEL(8, 4)
 #define LGREG          __REG_SEL(3, 2)
arch/riscv/kernel/entry.S +2 -0
index b9eda3fcbd6d..ea6b78dac739 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       /* Reuse the address as the spin value, so they must be all

XLEN's width. */
+ REG_AMOSWAP sp, sp, (sp)
+ bnez sp, 1b
+
la sp, shadow_stack
addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE

arch/riscv/kernel/traps.c +4 -0
index f3e96d60a2ff..9e6cc0d63833 100644
@@ -221,11 +221,15 @@ asmlinkage unsigned long get_overflow_stack(void)
                OVERFLOW_STACK_SIZE;
 }
 
+unsigned long spin_shadow_stack = 0;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
        unsigned long tsk_stk = (unsigned long)current->stack;
        unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+       smp_store_release(&spin_shadow_stack, 0);
+
        console_verbose();
 
        pr_emerg("Insufficient stack space to handle exception!\n");
Quoted 26 lines
arch/riscv/kernel/entry.S +2 -0
+       bnez sp, 1b
+
        la sp, shadow_stack
        addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +4 -1
index f3e96d60a2ff..88a54947dffb 100644
@@ -221,11 +221,15 @@ asmlinkage unsigned long get_overflow_stack(void)
                OVERFLOW_STACK_SIZE;
 }
 
+atomic_t spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
        unsigned long tsk_stk = (unsigned long)current->stack;
        unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+       atomic_set_release(&spin_shadow_stack, 0);
+
        console_verbose();
 
        pr_emerg("Insufficient stack space to handle exception!\n");
--
Quoted 2 lines

2.37.2

--
Best Regards
Guo Ren

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#3 lore raw
Quoted 25 lines · Guo Ren
Quoted 24 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +0 -0
index b9eda3fcbd6d..7b924b16792b 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
Quoted 7 lines

arch/riscv/kernel/entry.S handle_syscall_trace_exit:

Quoted 4 lines
arch/riscv/kernel/entry.S +2 -0
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       amoswap.w sp, sp, (sp)

If CONFIG_64BIT=y, it would be broken. Because we only hold 32bit of the sp, and the next loop would get the wrong sp value of &spin_shadow_stack.

Hi Guo,

Don't worry about it. the spin_shadow_stack is just a flag used for "spin", if hart is allowed to used the shadow_stack, we load its address in next instruction by "la sp, shadow_stack". But I agree with use unsigned int instead of atomic_t, and use smp_store_release directly. V2 has been sent out, could you please review it?

Thanks

#5 lore raw
In reply to Jisheng Zhang

Hi Jisheng,

Quoted 39 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..7b924b16792b 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:	la sp, spin_shadow_stack
+	amoswap.w sp, sp, (sp)
+	bnez sp, 1b
+
 	la sp, shadow_stack
 	addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +0 -0
index f3e96d60a2ff..88a54947dffb 100644
@@ -221,11 +221,15 @@ asmlinkage unsigned long get_overflow_stack(void)
 		OVERFLOW_STACK_SIZE;
 }

arch/riscv/kernel/traps.c asmlinkage unsigned long get_overflow_stack(void) · line 232

Quoted 10 lines
arch/riscv/kernel/traps.c +4 -0
 
+atomic_t spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
 	unsigned long tsk_stk = (unsigned long)current->stack;
 	unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
+	atomic_set_release(&spin_shadow_stack, 0);
+

Have not really looked the details: should there be a matching acquire?

Andrea

Quoted 6 lines
arch/riscv/kernel/traps.c +0 -0
 	console_verbose();
 
 	pr_emerg("Insufficient stack space to handle exception!\n");
Signature · 1 line
2.37.2
#4 lore raw
In reply to Jisheng Zhang
Quoted 35 lines · Jisheng Zhang
Quoted 25 lines · Guo Ren
Quoted 24 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +0 -0
index b9eda3fcbd6d..7b924b16792b 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
Quoted 7 lines

arch/riscv/kernel/entry.S handle_syscall_trace_exit:

Quoted 4 lines
arch/riscv/kernel/entry.S +2 -0
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:     la sp, spin_shadow_stack
+       amoswap.w sp, sp, (sp)

If CONFIG_64BIT=y, it would be broken. Because we only hold 32bit of the sp, and the next loop would get the wrong sp value of &spin_shadow_stack.

Quoted 5 lines

Hi Guo,

Don't worry about it. the spin_shadow_stack is just a flag used for "spin", if hart is allowed to used the shadow_stack, we load its address in next instruction by "la sp, shadow_stack".

Haha, yes, my brain is at fault :)

Quoted 3 lines

But I agree with use unsigned int instead of atomic_t, and use smp_store_release directly. V2 has been sent out, could you please review it?

Okay

Quoted 2 lines

Thanks

Signature · 2 lines
Best Regards
 Guo Ren
#6 lore raw
In reply to Andrea Parri
Quoted 51 lines · Andrea Parri

Hi Jisheng,

Quoted 47 lines · Jisheng Zhang

Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time.

To solve this race, we introduce spin_shadow_stack atomic var, which will make the shadow stack usage serialized.

Fixes 31da94c25aea ("riscv: add VMAP_STACK overflow detection")
---
 arch/riscv/kernel/entry.S | 4 ++++
 arch/riscv/kernel/traps.c | 4 ++++
 2 files changed, 8 insertions(+)
arch/riscv/kernel/entry.S +4 -0
index b9eda3fcbd6d..7b924b16792b 100644
@@ -404,6 +404,10 @@ handle_syscall_trace_exit:
 
 #ifdef CONFIG_VMAP_STACK
 handle_kernel_stack_overflow:
+1:   la sp, spin_shadow_stack
+     amoswap.w sp, sp, (sp)
+     bnez sp, 1b
+
      la sp, shadow_stack
      addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
 
arch/riscv/kernel/traps.c +2 -0
index f3e96d60a2ff..88a54947dffb 100644
@@ -221,11 +221,15 @@ asmlinkage unsigned long get_overflow_stack(void)
              OVERFLOW_STACK_SIZE;
 }
 
+atomic_t spin_shadow_stack;
+
 asmlinkage void handle_bad_stack(struct pt_regs *regs)
 {
      unsigned long tsk_stk = (unsigned long)current->stack;
      unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
 
Quoted 4 lines

arch/riscv/kernel/traps.c asmlinkage unsigned long get_overflow_stack(void) · line 232

Quoted 2 lines
arch/riscv/kernel/traps.c +2 -0
+     atomic_set_release(&spin_shadow_stack, 0);
+

Have not really looked the details: should there be a matching acquire?

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

The following memory operations order is not important, because we just care about sp value.

Also, we use relax amoswap before, because sp has naturelly dependency. But giving them RCsc is okay here, because we don't care about performance here.
eg:
handle_kernel_stack_overflow:

+1:     la sp, spin_shadow_stack
+       amoswap.w.aqrl sp, sp, (sp)
+       bnez sp, 1b

+
....
+ smp_store_release(&spin_shadow_stack, 0);
+ smp_mb();

Quoted 10 lines

Andrea

Quoted 6 lines
arch/riscv/kernel/traps.c +0 -1
      console_verbose();
 
      pr_emerg("Insufficient stack space to handle exception!\n");
--

2.37.2

--
Best Regards
Guo Ren

#7 lore raw
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

+ atomic_set_release(&spin_shadow_stack, 0);

Have not really looked the details: should there be a matching acquire?

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

Quoted 6 lines

The following memory operations order is not important, because we just care about sp value.

Also, we use relax amoswap before, because sp has naturelly dependency. But giving them RCsc is okay here, because we don't care about performance here.

Thanks for the clarification.

I'm not really suggesting to add unneeded synchronization, even more so in local/private constructs as in this case. It just felt odd to see the release without a pairing acquire, so I asked. ;-)

Thanks,
Andrea

Review

Quoted 9 lines

eg:
handle_kernel_stack_overflow:
+1: la sp, spin_shadow_stack

(quoted diff) +5 -0
+       amoswap.w.aqrl sp, sp, (sp)
+       bnez sp, 1b
+
....
+     smp_store_release(&spin_shadow_stack, 0);
+     smp_mb();

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#8 lore raw
Quoted 15 lines · Andrea Parri
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

+ atomic_set_release(&spin_shadow_stack, 0);

Have not really looked the details: should there be a matching acquire?

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

Quoted 6 lines

The following memory operations order is not important, because we just care about sp value.

Also, we use relax amoswap before, because sp has naturelly dependency. But giving them RCsc is okay here, because we don't care about performance here.

Quoted 20 lines

Thanks for the clarification.

I'm not really suggesting to add unneeded synchronization, even more so in local/private constructs as in this case. It just felt odd to see the release without a pairing acquire, so I asked. ;-)

Thanks,
Andrea

Quoted 9 lines

eg:
handle_kernel_stack_overflow:
+1: la sp, spin_shadow_stack

(quoted diff) +5 -0
+       amoswap.w.aqrl sp, sp, (sp)
+       bnez sp, 1b
+
....
+     smp_store_release(&spin_shadow_stack, 0);
+     smp_mb();

Hi Andrea, Zhang, Guo:

	Consider this implementation:)

	smp_store_mb(&spin_shadow_stack, 0);

Thanks,
Tong.

Quoted 6 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#9 lore raw

Hi Tong,

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

	Consider this implementation:)

	smp_store_mb(&spin_shadow_stack, 0);

smp_store_mb() has "WRITE_ONCE(); smp_mb()" semantics; so it doesn't guarantee that the store to spin_shadow_stack is ordered after program -order earlier memory accesses.

Andrea

#10 lore raw
Quoted 8 lines · Andrea Parri

Hi Tong,

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

Consider this implementation:)

Quoted 8 lines
Quoted 1 line
	smp_store_mb(&spin_shadow_stack, 0);

smp_store_mb() has "WRITE_ONCE(); smp_mb()" semantics; so it doesn't guarantee that the store to spin_shadow_stack is ordered after program -order earlier memory accesses.

Andrea
.

Hi Andrea:

IIUC, the earlier memory access amoswap.aqrl, here .aqrl guarantee it.
But anyway, consider we don't care about performance here, using smp_store_release()(add barrier()) surely right.

Thanks,
Tong.

#13 lore raw
In reply to Andrea Parri
Quoted 16 lines · Andrea Parri
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

+ atomic_set_release(&spin_shadow_stack, 0);

Have not really looked the details: should there be a matching acquire?

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

Quoted 6 lines

The following memory operations order is not important, because we just care about sp value.

Also, we use relax amoswap before, because sp has naturelly dependency. But giving them RCsc is okay here, because we don't care about performance here.

Quoted 5 lines

Thanks for the clarification.

I'm not really suggesting to add unneeded synchronization, even more so in local/private constructs as in this case. It just felt odd to see the release without a pairing acquire, so I asked. ;-)

Okay, let's keep:

handle_kernel_stack_overflow:

+1:     la sp, spin_shadow_stack
+       amoswap.w sp, sp, (sp)
+       bnez sp, 1b

+
....
+ smp_store_release(&spin_shadow_stack, 0);

Quoted 14 lines

Thanks,
Andrea

Quoted 9 lines

eg:
handle_kernel_stack_overflow:
+1: la sp, spin_shadow_stack

(quoted diff) +5 -0
+       amoswap.w.aqrl sp, sp, (sp)
+       bnez sp, 1b
+
....
+     smp_store_release(&spin_shadow_stack, 0);
+     smp_mb();
Signature · 2 lines
Best Regards
 Guo Ren
#11 lore raw
In reply to Tong Tiangen
Quoted 18 lines · Tong Tiangen
Quoted 14 lines · Andrea Parri

Hi Tong,

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

Consider this implementation:)

smp_store_mb(&spin_shadow_stack, 0);

smp_store_mb() has "WRITE_ONCE(); smp_mb()" semantics; so it doesn't guarantee that the store to spin_shadow_stack is ordered after program -order earlier memory accesses.

Quoted 8 lines
Quoted 2 lines

Andrea
.

Hi Andrea:

IIUC, the earlier memory access amoswap.aqrl, here .aqrl guarantee it. But anyway, consider we don't care about performance here, using smp_store_release()(add barrier()) surely right.

We use smp_store_release() is for:

        //load per-cpu overflow stack
        REG_L sp, -8(sp)

Not amoswap.

Actually, amoswap.aqrl guarantees nothing because all instructions depend on the sp register.

Quoted 3 lines

Thanks,
Tong.

Signature · 2 lines
Best Regards
 Guo Ren
#12 lore raw
Quoted 22 lines · Guo Ren
Quoted 21 lines · Tong Tiangen
Quoted 16 lines · Andrea Parri

Hi Tong,

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I use atomic_set_release here, because I need earlier memory operations finished to make sure the sp is ready then set the spin flag.

Consider this implementation:)

smp_store_mb(&spin_shadow_stack, 0);

smp_store_mb() has "WRITE_ONCE(); smp_mb()" semantics; so it doesn't guarantee that the store to spin_shadow_stack is ordered after program -order earlier memory accesses.

Andrea
.

Quoted 8 lines
Quoted 5 lines

Hi Andrea:

IIUC, the earlier memory access amoswap.aqrl, here .aqrl guarantee it. But anyway, consider we don't care about performance here, using smp_store_release()(add barrier()) surely right.

We use smp_store_release() is for:

         //load per-cpu overflow stack
         REG_L sp, -8(sp)

Oh, missing that, you're right. This guarantee the spin flag is set after the sp is used.

Quoted 5 lines

Not amoswap.

Actually, amoswap.aqrl guarantees nothing because all instructions depend on the sp register.

right.

Thanks,
Tong.

Quoted 7 lines
Quoted 4 lines

Thanks,
Tong.