ARM: Add TLB flushing for both entries in a PMD
Subsystems (2)
PATCH 1/1 ARM: Add TLB flushing for both entries in a PMD
Linux uses two PMD entries for a PTE with the classic page table format, covering 2MB range. However, the __pte_free_tlb() function only adds a single TLB flush corresponding to 1MB range covering 'addr'. On Cortex-A15, level 1 entries can be cached by the TLB independently of the level 2 entries and without additional flushing a PMD entry would be left pointing at the wrong PTE. The patch limits the TLB flushing range to two 4KB pages around the 1MB boundary within PMD.
--- arch/arm/include/asm/tlb.h | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-)
arch/arm/include/asm/tlb.h +9 -1
@@ -198,7 +198,15 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, unsigned long addr) {pgtable_page_dtor (pte); -tlb_add_flush (tlb, addr); + +/* +* With the classic ARM MMU, a pte page has two corresponding pmd +* entries, each covering 1MB. +*/ + addr &=PMD_MASK ; +tlb_add_flush (tlb, addr +SZ_1M -PAGE_SIZE ); +tlb_add_flush (tlb, addr +SZ_1M ); +tlb_remove_page (tlb, pte); }
PATCH 1/1 ARM: Add TLB flushing for both entries in a PMD
Hi,
I have tested this patch on several exynos machines which have a9 cores and it worked fine.
Though I'm not sure that android boot and running simple applications are enough test for this patch.
arch/arm/include/asm/tlb.h static inline void __pte_free_tlb(struct mmu_gather *tlb,
Quoted 19 lines · Catalin Marinas
Linux uses two PMD entries for a PTE with the classic page table format, covering 2MB range. However, the
__pte_free_tlb()function only adds a single TLB flush corresponding to 1MB range covering 'addr'. On Cortex-A15, level 1 entries can be cached by the TLB independently of the level 2 entries and without additional flushing a PMD entry would be left pointing at the wrong PTE. The patch limits the TLB flushing range to two 4KB pages around the 1MB boundary within PMD.Signed-off-by Catalin Marinas <[email protected]>--- arch/arm/include/asm/tlb.h | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-)arch/arm/include/asm/tlb.h +0
-0@@ -198,7 +198,15 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb,
pgtable_t pte,
arch/arm/include/asm/tlb.h static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, · line 207
Quoted 11 lines
arch/arm/include/asm/tlb.h +7
-1unsigned long addr) {pgtable_page_dtor (pte); -tlb_add_flush (tlb, addr); + +/* +* With the classic ARM MMU, a pte page has two corresponding pmd +* entries, each covering 1MB. +*/ + addr &=PMD_MASK ; +tlb_add_flush (tlb, addr +SZ_1M -PAGE_SIZE );
The address of tlb_add_flush is modified from 'addr' to 'addr + SZ_1M - PAGE_SIZE'.
Is the previous implementation wrong or just the address of tlb_add_flush is not important
if it's in the same PMD?
arch/arm/include/asm/tlb.h static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, · line 212
Quoted 5 lines
arch/arm/include/asm/tlb.h +2
-0+tlb_add_flush (tlb, addr +SZ_1M ); +tlb_remove_page (tlb, pte); }
Regards,
Changhwan
PATCH 1/1 ARM: Add TLB flushing for both entries in a PMD
Quoted 4 lines · Changhwan Youn
I have tested this patch on several exynos machines which have a9 cores and it worked fine.
Though I'm not sure that android boot and running simple applications are enough test for this patch.
Thanks for testing but the A9 would work fine without this patch. The problem is on A15 where level 1 page table entries (pgd) are cached by the TLB independently of level 2 entries (pte). The original code is only flushing one entry in level 1 rather than 2.
The problem was visible with a stress-test application doing mmap/munmap and then continuously getting level 2 translation fault even if the page table looked fine to the kernel (because of stale level 1 entry in the TLB).
LPAE is not affected, only the classic page table format.
Signature · 1 line
Catalin
PATCH 1/1 ARM: Add TLB flushing for both entries in a PMD
Quoted 10 lines · Catalin Marinas
To: Changhwan Youn
Cc: linux-arm-kernel at lists.infradead.org
Subject: Re: [PATCH] ARM: Add TLB flushing for both entries in a PMDQuoted 4 lines · Changhwan Youn
I have tested this patch on several exynos machines which have a9 cores and it worked fine.
Though I'm not sure that android boot and running simple applications are enough test for this patch.
Quoted 4 lines
Thanks for testing but the A9 would work fine without this patch. The problem is on A15 where level 1 page table entries (pgd) are cached by the TLB independently of level 2 entries (pte). The original code is only flushing one entry in level 1 rather than 2.
Thank you for the answer.
The one thing I don't understand is why A9 works fine without this patch. I know that A9 has worked fine without this patch.
It seems that without this patch, invalid VA->PA mapping can remains in TLB and this can cause wrong PA access by user process.
Can you explain why there's no wrong PA access in A9?
Thank you.
Quoted 8 lines
The problem was visible with a stress-test application doing mmap/munmap and then continuously getting level 2 translation fault even if the page table looked fine to the kernel (because of stale level 1 entry in the TLB).
LPAE is not affected, only the classic page table format.
Quoted 7 lines
--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Regards,
Changhwan
PATCH 1/1 ARM: Add TLB flushing for both entries in a PMD
Quoted 16 lines · Changhwan Youn
Quoted 10 lines · Catalin Marinas
To: Changhwan Youn
Cc: linux-arm-kernel at lists.infradead.org
Subject: Re: [PATCH] ARM: Add TLB flushing for both entries in a PMDQuoted 4 lines · Changhwan Youn
I have tested this patch on several exynos machines which have a9 cores and it worked fine.
Though I'm not sure that android boot and running simple applications are enough test for this patch.Quoted 4 lines
Thanks for testing but the A9 would work fine without this patch. The problem is on A15 where level 1 page table entries (pgd) are cached by the TLB independently of level 2 entries (pte). The original code is only flushing one entry in level 1 rather than 2.
Quoted 6 lines
Thank you for the answer.
The one thing I don't understand is why A9 works fine without this patch. I know that A9 has worked fine without this patch.
It seems that without this patch, invalidVA->PAmapping can remains in TLB and this can cause wrong PA access by user process.
Can you explain why there's no wrong PA access in A9?
This patch only fixes a bug in the pte_free_tlb() function, used when freeing the page tables and its goal is to make sure pmd level entries to not point to an already freed pte. Unmapping user VA->PA mappings happens at the pte level and page TLBs are flushed separately. If you look at the original code, it only adds a TLB flush for a single address while a pte table covers 2MB.
In theory, on A9 SMP there could also be an issue but given the timing and memory ordering conditions that's impossible in practice. Basically, the requirement is that clearing of the pmd entry on one CPU does not become visible to another CPU before the corresponding pte page has been re-used. The A9 only caches a page table walk in the TLB if there is a complete VA->PA translation.
Signature · 1 line
Catalin
PATCH 1/1 ARM: Add TLB flushing for both entries in a PMD
So practically Cortex A9 doesn't need tlb_add_flush() in pte_free_tlb(). I was a little confused because of the address of tlb_add_flush().
I really appreciate your answering.
On Friday, November 25, 2011 7:18 PM, Catalin Marinas wrote
Quoted 31 lines · Changhwan Youn
Quoted 16 lines · Catalin Marinas
Quoted 10 lines
To: Changhwan Youn
Cc: linux-arm-kernel at lists.infradead.org
Subject: Re: [PATCH] ARM: Add TLB flushing for both entries in a PMDQuoted 4 lines · Changhwan Youn
I have tested this patch on several exynos machines which have a9 cores and it worked fine.
Though I'm not sure that android boot and running simple applications are enough test for this patch.Quoted 4 lines
Thanks for testing but the A9 would work fine without this patch. The problem is on A15 where level 1 page table entries (pgd) are cached by the TLB independently of level 2 entries (pte). The original code is only flushing one entry in level 1 rather than 2.
Quoted 6 lines
Thank you for the answer.
The one thing I don't understand is why A9 works fine without this patch. I know that A9 has worked fine without this patch.
It seems that without this patch, invalidVA->PAmapping can remains in TLB and this can cause wrong PA access by user process.
Can you explain why there's no wrong PA access in A9?This patch only fixes a bug in the
pte_free_tlb()function, used when freeing the page tables and its goal is to make sure pmd level entries to not point to an already freed pte. Unmapping userVA->PAmappings happens at the pte level and page TLBs are flushed separately. If you look at the original code, it only adds a TLB flush for a single address while a pte table covers 2MB.
Quoted 6 lines
In theory, on A9 SMP there could also be an issue but given the timing and memory ordering conditions that's impossible in practice. Basically, the requirement is that clearing of the pmd entry on one CPU does not become visible to another CPU before the corresponding pte page has been re-used. The A9 only caches a page table walk in the TLB if there is a complete
VA->PAtranslation.
Regards,
Changhwan