Skip to contentSkip to conversation

arm64: tlbflush: Ensure start/end of address range are aligned to stride

PATCH In review Will Deacon 2 participants 3 messages Jun 11, 2019 – Jun 12, 2019 arm64

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm64: tlbflush: Ensure start/end of address range are aligned to stride

Since commit 3d65b6bbc01e ("arm64: tlbi: Set MAX_TLBI_OPS to PTRS_PER_PTE"), we resort to per-ASID invalidation when attempting to perform more than PTRS_PER_PTE invalidation instructions in a single call to __flush_tlb_range(). Whilst this is beneficial, the mmu_gather code does not ensure that the end address of the range is rounded-up to the stride when freeing intermediate page tables in pXX_free_tlb(), which defeats our range checking.

Align the bounds passed into __flush_tlb_range().

Cc Catalin Marinas <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/arm64/include/asm/tlbflush.h | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/include/asm/tlbflush.h +3 -0
index 3a1870228946..dff8f9ea5754 100644
@@ -195,6 +195,9 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma,
 	unsigned long asid = ASID(vma->vm_mm);
 	unsigned long addr;
 
+	start = round_down(start, stride);
+	end = round_up(end, stride);
+
 	if ((end - start) >= (MAX_TLBI_OPS * stride)) {
 		flush_tlb_mm(vma->vm_mm);
 		return;
Signature · 7 lines
2.11.0


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#1 lore raw

Hi Will,

Quoted 8 lines · Will Deacon

Since commit 3d65b6bbc01e ("arm64: tlbi: Set MAX_TLBI_OPS to PTRS_PER_PTE"), we resort to per-ASID invalidation when attempting to perform more than PTRS_PER_PTE invalidation instructions in a single call to __flush_tlb_range(). Whilst this is beneficial, the mmu_gather code does not ensure that the end address of the range is rounded-up to the stride when freeing intermediate page tables in pXX_free_tlb(), which defeats our range checking.

Quoted 5 lines

Align the bounds passed into __flush_tlb_range().

Cc Catalin Marinas <[email protected]>
Cc Peter Zijlstra <[email protected]>

Thanks for the patch,I will test the patch tomorrow my local time as it's late here, and will update you when I get the results.

Thanks
Hanjun

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Tested-by Reviewed-by
#2 lore raw
Quoted 11 lines · Hanjun Guo

Hi Will,

Quoted 8 lines · Will Deacon

Since commit 3d65b6bbc01e ("arm64: tlbi: Set MAX_TLBI_OPS to PTRS_PER_PTE"), we resort to per-ASID invalidation when attempting to perform more than PTRS_PER_PTE invalidation instructions in a single call to __flush_tlb_range(). Whilst this is beneficial, the mmu_gather code does not ensure that the end address of the range is rounded-up to the stride when freeing intermediate page tables in pXX_free_tlb(), which defeats our range checking.

Quoted 8 lines
Quoted 5 lines

Align the bounds passed into __flush_tlb_range().

Cc Catalin Marinas <[email protected]>
Cc Peter Zijlstra <[email protected]>

Thanks for the patch,I will test the patch tomorrow my local time as it's late here, and will update you when I get the results.

I tested this patch on top of 5.2-rc1, and on the Kunpeng920 ARM64 server platform, with test case I reported before, I can see about 100% speedup for munmap() (from about 47us to 25us), that's great!

Thanks
Hanjun

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel