Skip to contentSkip to conversation

riscv: asid: Fixup stale TLB entry cause application crash

PATCH Superseded [email protected] 2 participants 4 messages Nov 9, 2022 – Nov 10, 2022 RISC-V

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 riscv: asid: Fixup stale TLB entry cause application crash

From: Guo Ren <[email protected]>

After use_asid_allocator is enabled, the userspace application will crash by stale TLB entries. Because only using cpumask_clear_cpu without local_flush_tlb_all couldn't guarantee CPU's TLB entries were fresh. Then set_mm_asid would cause the user space application to get a stale value by stale TLB entry, but set_mm_noasid is okay.

Here is the symptom of the bug:
unhandled signal 11 code 0x1 (coredump)

   0x0000003fd6d22524 <+4>:     auipc   s0,0x70
   0x0000003fd6d22528 <+8>:     ld      s0,-148(s0) # 0x3fd6d92490
=> 0x0000003fd6d2252c <+12>:    ld      a5,0(s0)

(gdb) i r s0
s0 0x8082ed1cc3198b21 0x8082ed1cc3198b21
(gdb) x /2x 0x3fd6d92490
0x3fd6d92490: 0xd80ac8a8 0x0000003f
The core dump file shows that register s0 is wrong, but the value in memory is correct. Because 'ld s0, -148(s0)' used a stale mapping entry in TLB and got a wrong result from an incorrect physical address.

When the task ran on CPU0, which loaded/speculative-loaded the value of address(0x3fd6d92490), then the first version of the mapping entry was PTWed into CPU0's TLB.
When the task switched from CPU0 to CPU1 (No local_tlb_flush_all here by asid), it happened to write a value on the address (0x3fd6d92490). It caused do_page_fault -> wp_page_copy -> ptep_clear_flush -> ptep_get_and_clear & flush_tlb_page.
The flush_tlb_page used mm_cpumask(mm) to determine which CPUs need TLB flush, but CPU0 had cleared the CPU0's mm_cpumask in the previous switch_mm. So we only flushed the CPU1 TLB and set the second version mapping of the PTE. When the task switched from CPU1 to CPU0 again, CPU0 still used a stale TLB mapping entry which contained a wrong target physical address. It raised a bug when the task happened to read that value.

The solution is to keep all CPUs' footmarks of cpumask(mm) in switch_mm, which could prevent losing pieces of stuff during TLB flush.

Fixes 65d4b9c53017 ("RISC-V: Implement ASID allocator")
Cc Anup Patel <[email protected]>
Cc Palmer Dabbelt <[email protected]>

---
Changes in v2:
- Fixup nommu compile problem (Thx Conor, Also Reported-by: kernel test robot <[email protected]>)
- Keep cpumask_clear_cpu for noasid

---
 arch/riscv/mm/context.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/riscv/mm/context.c +5 -1
index 7acbfbd14557..f58e4b211595 100644
@@ -317,7 +317,11 @@ void switch_mm(struct mm_struct *prev, struct mm_struct *next,
 	 */
 	cpu = smp_processor_id();
 
-	cpumask_clear_cpu(cpu, mm_cpumask(prev));
+#ifdef CONFIG_MMU
+	if (!static_branch_unlikely(&use_asid_allocator))
+#endif
+		cpumask_clear_cpu(cpu, mm_cpumask(prev));
+
 	cpumask_set_cpu(cpu, mm_cpumask(next));
 
 	set_mm(next, cpu);
Signature · 1 line
2.36.1
#1 lore raw
Quoted 58 lines · On Wed, Nov 09, 2022 at 12:40:56AM -0500, [email protected] wrote:

From: Guo Ren <[email protected]>

After use_asid_allocator is enabled, the userspace application will crash by stale TLB entries. Because only using cpumask_clear_cpu without local_flush_tlb_all couldn't guarantee CPU's TLB entries were fresh. Then set_mm_asid would cause the user space application to get a stale value by stale TLB entry, but set_mm_noasid is okay.

Here is the symptom of the bug:
unhandled signal 11 code 0x1 (coredump)

   0x0000003fd6d22524 <+4>:     auipc   s0,0x70
   0x0000003fd6d22528 <+8>:     ld      s0,-148(s0) # 0x3fd6d92490
=> 0x0000003fd6d2252c <+12>:    ld      a5,0(s0)

(gdb) i r s0
s0 0x8082ed1cc3198b21 0x8082ed1cc3198b21
(gdb) x /2x 0x3fd6d92490
0x3fd6d92490: 0xd80ac8a8 0x0000003f
The core dump file shows that register s0 is wrong, but the value in memory is correct. Because 'ld s0, -148(s0)' used a stale mapping entry in TLB and got a wrong result from an incorrect physical address.

When the task ran on CPU0, which loaded/speculative-loaded the value of address(0x3fd6d92490), then the first version of the mapping entry was PTWed into CPU0's TLB.
When the task switched from CPU0 to CPU1 (No local_tlb_flush_all here by asid), it happened to write a value on the address (0x3fd6d92490). It caused do_page_fault -> wp_page_copy -> ptep_clear_flush -> ptep_get_and_clear & flush_tlb_page.
The flush_tlb_page used mm_cpumask(mm) to determine which CPUs need TLB flush, but CPU0 had cleared the CPU0's mm_cpumask in the previous switch_mm. So we only flushed the CPU1 TLB and set the second version mapping of the PTE. When the task switched from CPU1 to CPU0 again, CPU0 still used a stale TLB mapping entry which contained a wrong target physical address. It raised a bug when the task happened to read that value.

The solution is to keep all CPUs' footmarks of cpumask(mm) in switch_mm, which could prevent losing pieces of stuff during TLB flush.

Fixes 65d4b9c53017 ("RISC-V: Implement ASID allocator")
Cc Anup Patel <[email protected]>
Cc Palmer Dabbelt <[email protected]>

---
Changes in v2:
- Fixup nommu compile problem (Thx Conor, Also Reported-by: kernel test robot <[email protected]>)
- Keep cpumask_clear_cpu for noasid

---
 arch/riscv/mm/context.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/riscv/mm/context.c +0 -0
index 7acbfbd14557..f58e4b211595 100644

arch/riscv/mm/context.c void switch_mm(struct mm_struct *prev, struct mm_struct *next, · line 322

Quoted 7 lines
arch/riscv/mm/context.c +3 -1
@@ -317,7 +317,11 @@ void switch_mm(struct mm_struct *prev, struct mm_struct *next,
 	 */
 	cpu = smp_processor_id();
 
-	cpumask_clear_cpu(cpu, mm_cpumask(prev));
+#ifdef CONFIG_MMU
+	if (!static_branch_unlikely(&use_asid_allocator))
+#endif

That's not very pretty. Can't we just do the following, instead?

arch/riscv/mm/context.c +2 -1
index 7acbfbd14557..ace419761e31 100644
@@ -16,10 +16,11 @@
 #include <asm/cacheflush.h>
 #include <asm/mmu_context.h>
 
-#ifdef CONFIG_MMU
 
 DEFINE_STATIC_KEY_FALSE(use_asid_allocator);
 
+#ifdef CONFIG_MMU
+
 static unsigned long asid_bits;
 static unsigned long num_asids;
 static unsigned long asid_mask;

Thanks,
drew

Quoted 13 lines
arch/riscv/mm/context.c +2 -0
+		cpumask_clear_cpu(cpu, mm_cpumask(prev));
+
 	cpumask_set_cpu(cpu, mm_cpumask(next));
 
 	set_mm(next, cpu);
Signature · 7 lines
2.36.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#2 lore raw
Quoted 75 lines · Andrew Jones
Quoted 58 lines · On Wed, Nov 09, 2022 at 12:40:56AM -0500, [email protected] wrote:

From: Guo Ren <[email protected]>

After use_asid_allocator is enabled, the userspace application will crash by stale TLB entries. Because only using cpumask_clear_cpu without local_flush_tlb_all couldn't guarantee CPU's TLB entries were fresh. Then set_mm_asid would cause the user space application to get a stale value by stale TLB entry, but set_mm_noasid is okay.

Here is the symptom of the bug:
unhandled signal 11 code 0x1 (coredump)

   0x0000003fd6d22524 <+4>:     auipc   s0,0x70
   0x0000003fd6d22528 <+8>:     ld      s0,-148(s0) # 0x3fd6d92490
=> 0x0000003fd6d2252c <+12>:    ld      a5,0(s0)

(gdb) i r s0
s0 0x8082ed1cc3198b21 0x8082ed1cc3198b21
(gdb) x /2x 0x3fd6d92490
0x3fd6d92490: 0xd80ac8a8 0x0000003f
The core dump file shows that register s0 is wrong, but the value in memory is correct. Because 'ld s0, -148(s0)' used a stale mapping entry in TLB and got a wrong result from an incorrect physical address.

When the task ran on CPU0, which loaded/speculative-loaded the value of address(0x3fd6d92490), then the first version of the mapping entry was PTWed into CPU0's TLB.
When the task switched from CPU0 to CPU1 (No local_tlb_flush_all here by asid), it happened to write a value on the address (0x3fd6d92490). It caused do_page_fault -> wp_page_copy -> ptep_clear_flush -> ptep_get_and_clear & flush_tlb_page.
The flush_tlb_page used mm_cpumask(mm) to determine which CPUs need TLB flush, but CPU0 had cleared the CPU0's mm_cpumask in the previous switch_mm. So we only flushed the CPU1 TLB and set the second version mapping of the PTE. When the task switched from CPU1 to CPU0 again, CPU0 still used a stale TLB mapping entry which contained a wrong target physical address. It raised a bug when the task happened to read that value.

The solution is to keep all CPUs' footmarks of cpumask(mm) in switch_mm, which could prevent losing pieces of stuff during TLB flush.

Fixes 65d4b9c53017 ("RISC-V: Implement ASID allocator")
Cc Anup Patel <[email protected]>
Cc Palmer Dabbelt <[email protected]>

---
Changes in v2:
- Fixup nommu compile problem (Thx Conor, Also Reported-by: kernel test robot <[email protected]>)
- Keep cpumask_clear_cpu for noasid

---
 arch/riscv/mm/context.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/riscv/mm/context.c +0 -0
index 7acbfbd14557..f58e4b211595 100644

arch/riscv/mm/context.c void switch_mm(struct mm_struct *prev, struct mm_struct *next, · line 322

Quoted 7 lines
arch/riscv/mm/context.c +3 -1
@@ -317,7 +317,11 @@ void switch_mm(struct mm_struct *prev, struct mm_struct *next,
       */
      cpu = smp_processor_id();
 
-     cpumask_clear_cpu(cpu, mm_cpumask(prev));
+#ifdef CONFIG_MMU
+     if (!static_branch_unlikely(&use_asid_allocator))
+#endif

That's not very pretty. Can't we just do the following, instead?

arch/riscv/mm/context.c +0 -0
index 7acbfbd14557..ace419761e31 100644

arch/riscv/mm/context.c line 20

Quoted 6 lines
arch/riscv/mm/context.c +0 -1
@@ -16,10 +16,11 @@
 #include <asm/cacheflush.h>
 #include <asm/mmu_context.h>
 
-#ifdef CONFIG_MMU
 
 DEFINE_STATIC_KEY_FALSE(use_asid_allocator);

Define use_asid_allocator in nommu part? How about:

arch/riscv/mm/context.c +11 -6
index 7acbfbd14557..ed3f8de7ef97 100644
@@ -205,12 +205,16 @@ static void set_mm_noasid(struct mm_struct *mm)
        local_flush_tlb_all();
 }
 
-static inline void set_mm(struct mm_struct *mm, unsigned int cpu)
+static inline void set_mm(struct mm_struct *prev,
+                         struct mm_struct *next, unsigned int cpu)
 {
-       if (static_branch_unlikely(&use_asid_allocator))
-               set_mm_asid(mm, cpu);
-       else
-               set_mm_noasid(mm);
+       cpumask_set_cpu(cpu, mm_cpumask(next));
+       if (static_branch_unlikely(&use_asid_allocator)) {
+               set_mm_asid(next, cpu);
+       } else {
+               cpumask_clear_cpu(cpu, mm_cpumask(prev));
+               set_mm_noasid(next);
+       }
 }
 
 static int __init asids_init(void)
@@ -264,7 +268,8 @@ static int __init asids_init(void)
 }
 early_initcall(asids_init);
 #else
-static inline void set_mm(struct mm_struct *mm, unsigned int cpu)
+static inline void set_mm(struct mm_struct *prev,
+                         struct mm_struct *next, unsigned int cpu)
 {
        /* Nothing to do here when there is no MMU */
 }
@@ -317,10 +322,7 @@ void switch_mm(struct mm_struct *prev, struct

mm_struct *next,

         */
        cpu = smp_processor_id();

- cpumask_clear_cpu(cpu, mm_cpumask(prev));
- cpumask_set_cpu(cpu, mm_cpumask(next));
-
- set_mm(next, cpu);
+ set_mm(prev, next, cpu);

        flush_icache_deferred(next, cpu);
Quoted 24 lines
arch/riscv/mm/context.c +2 -0
+#ifdef CONFIG_MMU
+
 static unsigned long asid_bits;
 static unsigned long num_asids;
 static unsigned long asid_mask;
 

Thanks,
drew

Quoted 13 lines
arch/riscv/mm/context.c +2 -1
+             cpumask_clear_cpu(cpu, mm_cpumask(prev));
+
      cpumask_set_cpu(cpu, mm_cpumask(next));
 
      set_mm(next, cpu);
--

2.36.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Signature · 2 lines
Best Regards
 Guo Ren
#3 lore raw
Quoted 19 lines · Guo Ren
Quoted 18 lines · Andrew Jones

Review

Quoted 5 lines · On Wed, Nov 09, 2022 at 12:40:56AM -0500, [email protected] wrote:
(quoted diff) +3 -1
-     cpumask_clear_cpu(cpu, mm_cpumask(prev));
+#ifdef CONFIG_MMU
+     if (!static_branch_unlikely(&use_asid_allocator))
+#endif

That's not very pretty. Can't we just do the following, instead?

arch/riscv/mm/context.c +0 -0
index 7acbfbd14557..ace419761e31 100644
@@ -16,10 +16,11 @@
 #include <asm/cacheflush.h>
 #include <asm/mmu_context.h>
 
Quoted 4 lines

arch/riscv/mm/context.c line 20

Quoted 3 lines
arch/riscv/mm/context.c +0 -1
-#ifdef CONFIG_MMU
 
 DEFINE_STATIC_KEY_FALSE(use_asid_allocator);

Define use_asid_allocator in nommu part? How about:

Yeah, I was thinking it'll just always be a false static branch in the nommu case, but I like your proposal below better.

Thanks,
drew

Quoted 80 lines
arch/riscv/mm/context.c +12 -10
index 7acbfbd14557..ed3f8de7ef97 100644
@@ -205,12 +205,16 @@ static void set_mm_noasid(struct mm_struct *mm)
        local_flush_tlb_all();
 }
 
-static inline void set_mm(struct mm_struct *mm, unsigned int cpu)
+static inline void set_mm(struct mm_struct *prev,
+                         struct mm_struct *next, unsigned int cpu)
 {
-       if (static_branch_unlikely(&use_asid_allocator))
-               set_mm_asid(mm, cpu);
-       else
-               set_mm_noasid(mm);
+       cpumask_set_cpu(cpu, mm_cpumask(next));
+       if (static_branch_unlikely(&use_asid_allocator)) {
+               set_mm_asid(next, cpu);
+       } else {
+               cpumask_clear_cpu(cpu, mm_cpumask(prev));
+               set_mm_noasid(next);
+       }
 }
 
 static int __init asids_init(void)
@@ -264,7 +268,8 @@ static int __init asids_init(void)
 }
 early_initcall(asids_init);
 #else
-static inline void set_mm(struct mm_struct *mm, unsigned int cpu)
+static inline void set_mm(struct mm_struct *prev,
+                         struct mm_struct *next, unsigned int cpu)
 {
        /* Nothing to do here when there is no MMU */
 }
@@ -317,10 +322,7 @@ void switch_mm(struct mm_struct *prev, struct
 mm_struct *next,
         */
        cpu = smp_processor_id();
 
-       cpumask_clear_cpu(cpu, mm_cpumask(prev));
-       cpumask_set_cpu(cpu, mm_cpumask(next));
-
-       set_mm(next, cpu);
+       set_mm(prev, next, cpu);
 
        flush_icache_deferred(next, cpu);
Quoted 25 lines
arch/riscv/mm/context.c +2 -0
+#ifdef CONFIG_MMU
+
 static unsigned long asid_bits;
 static unsigned long num_asids;
 static unsigned long asid_mask;
 

Thanks,
drew

Quoted 13 lines
(quoted diff) +2 -1
+             cpumask_clear_cpu(cpu, mm_cpumask(prev));
+
      cpumask_set_cpu(cpu, mm_cpumask(next));
 
      set_mm(next, cpu);
--

2.36.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Signature · 2 lines
Best Regards
 Guo Ren