riscv: mm: execute local TLB flush after populating vmemmap
PATCH 1/1 riscv: mm: execute local TLB flush after populating vmemmap
The spare_init() calls memmap_populate() many times to create VA to PA mapping for the VMEMMAP area, where all "strcut page" are located once CONFIG_SPARSEMEM_VMEMMAP is defined. These "struct page" are later initialized in the zone_sizes_init() function. However, during this process, no sfence.vma instruction is executed for this VMEMMAP area. This omission may cause the hart to fail to perform page table work because some data related to the address translation is invisible to the hart. To solve this issue, the local_flush_tlb_kernel_range() is called right after the spare_init() to execute a sfence.vma instruction for the VMEMMAP area, ensuring that all data related to the address translation is visible to the hart.
--- arch/riscv/include/asm/tlbflush.h | 7 +++++++ arch/riscv/mm/init.c | 5 +++++ 2 files changed, 12 insertions(+)
arch/riscv/include/asm/tlbflush.h +7 -0
@@ -61,4 +61,11 @@ static inline void flush_tlb_kernel_range(unsigned long start, flush_tlb_all(); } +/* Flush a range of kernel pages without broadcasting */ +static inline void local_flush_tlb_kernel_range (unsigned long start, +unsigned long end) +{ +local_flush_tlb_all (); +} +#endif /* _ASM_RISCV_TLBFLUSH_H */
arch/riscv/mm/init.c +5 -0
@@ -1231,6 +1231,10 @@ void __init misc_mem_init(void)early_memtest (min_low_pfn <<PAGE_SHIFT , max_low_pfn <<PAGE_SHIFT );arch_numa_init ();sparse_init (); +#ifdef CONFIG_SPARSEMEM_VMEMMAP +/* The entire VMEMMAP region has been propulated. Flush TLB for this region */ +local_flush_tlb_kernel_range (VMEMMAP_START ,VMEMMAP_END ); +#endif zone_sizes_init ();reserve_crashkernel ();memblock_dump_all ();
@@ -1240,6 +1244,7 @@ void __init misc_mem_init(void)int __meminit vmemmap_populate (unsigned long start,unsigned long end,int node,struct vmem_altmap *altmap) { +/* Defer the required TLB flush until the entire VMEMMAP region has been populated */ return vmemmap_populate_basepages (start, end, node,NULL ); }#endif
Signature · 7 lines
2.25.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 2 lines · Vincent Chen
The
spare_init()callsmemmap_populate()many times to create VA to PA mapping for the VMEMMAP area, where all "strcut page" are located once
struct
Quoted 4 lines
CONFIG_SPARSEMEM_VMEMMAPis defined. These "struct page" are later initialized in thezone_sizes_init()function. However, during this process, no sfence.vma instruction is executed for this VMEMMAP area. This omission may cause the hart to fail to perform page table work
s/work/walk/ ?
Quoted 35 lines
because some data related to the address translation is invisible to the hart. To solve this issue, the
local_flush_tlb_kernel_range()is called right after thespare_init()to execute a sfence.vma instruction for the VMEMMAP area, ensuring that all data related to the address translation is visible to the hart.Fixes d95f1a542c3d ("RISC-V: Implement sparsemem")Signed-off-by Vincent Chen <[email protected]>Reviewed-by Alexandre Ghiti <[email protected]>--- arch/riscv/include/asm/tlbflush.h | 7 +++++++ arch/riscv/mm/init.c | 5 +++++ 2 files changed, 12 insertions(+)arch/riscv/include/asm/tlbflush.h +7
-0@@ -61,4 +61,11 @@ static inline void flush_tlb_kernel_range(unsigned long start, flush_tlb_all(); } +/* Flush a range of kernel pages without broadcasting */ +static inline void local_flush_tlb_kernel_range (unsigned long start, +unsigned long end) +{ +local_flush_tlb_all (); +} +#endif /* _ASM_RISCV_TLBFLUSH_H */ arch/riscv/mm/init.c +0
-0
arch/riscv/mm/init.c void __init misc_mem_init(void) · line 1235
Quoted 5 lines
arch/riscv/mm/init.c +2
-0@@ -1231,6 +1231,10 @@ void __init misc_mem_init(void)early_memtest (min_low_pfn <<PAGE_SHIFT , max_low_pfn <<PAGE_SHIFT );arch_numa_init ();sparse_init (); +#ifdef CONFIG_SPARSEMEM_VMEMMAP +/* The entire VMEMMAP region has been propulated. Flush TLB for this region */
populated
Quoted 16 lines
arch/riscv/mm/init.c +3
-0+local_flush_tlb_kernel_range (VMEMMAP_START ,VMEMMAP_END ); +#endif zone_sizes_init ();reserve_crashkernel ();memblock_dump_all ();@@ -1240,6 +1244,7 @@ void __init misc_mem_init(void)int __meminit vmemmap_populate (unsigned long start,unsigned long end,int node,struct vmem_altmap *altmap) { +/* Defer the required TLB flush until the entire VMEMMAP region has been populated */ return vmemmap_populate_basepages (start, end, node,NULL ); }#endif Signature · 1 line
2.25.1
Besides the typos,
Thanks,
drew
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 67 lines · Andrew Jones
Quoted 2 lines · Vincent Chen
The
spare_init()callsmemmap_populate()many times to create VA to PA mapping for the VMEMMAP area, where all "strcut page" are located oncestruct
Quoted 4 lines
CONFIG_SPARSEMEM_VMEMMAPis defined. These "struct page" are later initialized in thezone_sizes_init()function. However, during this process, no sfence.vma instruction is executed for this VMEMMAP area. This omission may cause the hart to fail to perform page table works/work/walk/ ?
Quoted 35 lines
because some data related to the address translation is invisible to the hart. To solve this issue, the
local_flush_tlb_kernel_range()is called right after thespare_init()to execute a sfence.vma instruction for the VMEMMAP area, ensuring that all data related to the address translation is visible to the hart.Fixes d95f1a542c3d ("RISC-V: Implement sparsemem")Signed-off-by Vincent Chen <[email protected]>Reviewed-by Alexandre Ghiti <[email protected]>--- arch/riscv/include/asm/tlbflush.h | 7 +++++++ arch/riscv/mm/init.c | 5 +++++ 2 files changed, 12 insertions(+)arch/riscv/include/asm/tlbflush.h +7
-0@@ -61,4 +61,11 @@ static inline void flush_tlb_kernel_range(unsigned long start, flush_tlb_all(); } +/* Flush a range of kernel pages without broadcasting */ +static inline void local_flush_tlb_kernel_range (unsigned long start, +unsigned long end) +{ +local_flush_tlb_all (); +} +#endif /* _ASM_RISCV_TLBFLUSH_H */ arch/riscv/mm/init.c +0
-0arch/riscv/mm/init.c void __init misc_mem_init(void) · line 1235
Quoted 5 lines
arch/riscv/mm/init.c +2
-0@@ -1231,6 +1231,10 @@ void __init misc_mem_init(void)early_memtest (min_low_pfn <<PAGE_SHIFT , max_low_pfn <<PAGE_SHIFT );arch_numa_init ();sparse_init (); +#ifdef CONFIG_SPARSEMEM_VMEMMAP +/* The entire VMEMMAP region has been propulated. Flush TLB for this region */ populated
Quoted 10 lines
arch/riscv/mm/init.c +3
-0+local_flush_tlb_kernel_range (VMEMMAP_START ,VMEMMAP_END ); +#endif zone_sizes_init ();reserve_crashkernel ();memblock_dump_all ();@@ -1240,6 +1244,7 @@ void __init misc_mem_init(void)int __meminit vmemmap_populate (unsigned long start,unsigned long end,int node,struct vmem_altmap *altmap) { +/* Defer the required TLB flush until the entire VMEMMAP region has been populated */
Quoted 8 lines
Quoted 6 lines
arch/riscv/mm/init.c +0
-1return vmemmap_populate_basepages(start, end, node, NULL); }#endif --2.25.1
Besides the typos,
Thank you for finding these typos. It's really embarrassing. I will fix these issues in the patch for v3.
Thanks,
Vincent
Quoted 5 lines
Reviewed-by Andrew Jones <[email protected]>Thanks,
drew
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv