powerpc/mm: Use tlbiel only if we ever ran on the current cpu

PATCH Applied Aneesh Kumar K.V 3 participants 4 messages Oct 24, 2016 – Oct 27, 2016

Subsystems (2)

PowerPC MMU gather and TLB invalidation

lore

Share & tools

lore

Aneesh Kumar K.V

author

2016-10-24 03:20 UTC #0 lore raw

PATCH 1/1 powerpc/mm: Use tlbiel only if we ever ran on the current cpu

Before this patch, we used tlbiel, if we ever ran only on this core. That was mostly derived from the nohash usage of the same. But the ISA 3.0 clarifies tlbiel such that

"All TLB entries that have all of the following properties are made invalid on the thread executing the tlbiel instruction"

Hence use tlbiel, if we only ever ran on just the current cpu.

Signed-off-by Aneesh Kumar K.V <[email protected]>

---

NOTE: May be the nohash usage also need to switch to mm_is_thread_local ? I didn't check the spec, hence I left it unchanged.

I am also not adding a Fixes: line because there is no one commit. But this is needed for radix enabled kernel. So a backport to 4.8 is needed.

 arch/powerpc/include/asm/tlb.h | 12 ++++++++++++
 arch/powerpc/mm/tlb-radix.c    |  8 ++++----
 2 files changed, 16 insertions(+), 4 deletions(-)

arch/powerpc/include/asm/tlb.h +12 -0

index f6f68f73e858..99e1397b71da 100644
@@ -52,11 +52,23 @@ static inline int mm_is_core_local(struct mm_struct *mm)
 	return cpumask_subset(mm_cpumask(mm),
 			      topology_sibling_cpumask(smp_processor_id()));
 }
+
+static inline int mm_is_thread_local(struct mm_struct *mm)
+{
+	return cpumask_equal(mm_cpumask(mm),
+			      cpumask_of(smp_processor_id()));
+}
+
 #else
 static inline int mm_is_core_local(struct mm_struct *mm)
 {
 	return 1;
 }
+
+static inline int mm_is_thread_local(struct mm_struct *mm)
+{
+	return 1;
+}
 #endif
 
 #endif /* __KERNEL__ */

arch/powerpc/mm/tlb-radix.c +4 -4

index 0e49ec541ab5..bda8c43be78a 100644
@@ -175,7 +175,7 @@ void radix__flush_tlb_mm(struct mm_struct *mm)
 	if (unlikely(pid == MMU_NO_CONTEXT))
 		goto no_context;
 
-	if (!mm_is_core_local(mm)) {
+	if (!mm_is_thread_local(mm)) {
 		int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
 
 		if (lock_tlbie)

@@ -201,7 +201,7 @@ void radix__flush_tlb_pwc(struct mmu_gather *tlb, unsigned long addr)
 	if (unlikely(pid == MMU_NO_CONTEXT))
 		goto no_context;
 
-	if (!mm_is_core_local(mm)) {
+	if (!mm_is_thread_local(mm)) {
 		int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
 
 		if (lock_tlbie)

@@ -226,7 +226,7 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,
 	pid = mm ? mm->context.id : 0;
 	if (unlikely(pid == MMU_NO_CONTEXT))
 		goto bail;
-	if (!mm_is_core_local(mm)) {
+	if (!mm_is_thread_local(mm)) {
 		int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
 
 		if (lock_tlbie)

@@ -321,7 +321,7 @@ void radix__flush_tlb_range_psize(struct mm_struct *mm, unsigned long start,
 {
 	unsigned long pid;
 	unsigned long addr;
-	int local = mm_is_core_local(mm);
+	int local = mm_is_thread_local(mm);
 	unsigned long ap = mmu_get_ap(psize);
 	int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
 	unsigned long page_size = 1UL << mmu_psize_defs[psize].shift;

Signature · 1 line

2.10.1

Balbir Singh

2016-10-24 03:40 UTC #1 lore raw

Quoted 9 lines · Aneesh Kumar K.V

Before this patch, we used tlbiel, if we ever ran only on this core. That was mostly derived from the nohash usage of the same. But the ISA 3.0 clarifies tlbiel such that
"All TLB entries that have all of the following properties are made invalid on the thread executing the tlbiel instruction"
Hence use tlbiel, if we only ever ran on just the current cpu.

Could you clarify the impact. The impact I see is that it could lead to us thinking we invalidated the TLB across the core whereas we did it only on the current thread? This could leave others threads in the same core with invalid TLB's, if cpumask reported we ran on other threads in the same core?

Balbir Singh.

Aneesh Kumar K.V

author

2016-10-24 06:01 UTC #2 lore raw

Quoted 8 lines · Balbir Singh

Quoted 7 lines · Aneesh Kumar K.V
Before this patch, we used tlbiel, if we ever ran only on this core. That was mostly derived from the nohash usage of the same. But the ISA 3.0 clarifies tlbiel such that
"All TLB entries that have all of the following properties are made invalid on the thread executing the tlbiel instruction"

Quoted 8 lines

Quoted 2 lines
Hence use tlbiel, if we only ever ran on just the current cpu.
Could you clarify the impact. The impact I see is that it could lead to us thinking we invalidated the TLB across the core whereas we did it only on the current thread? This could leave others threads in the same core with invalid TLB's, if cpumask reported we ran on other threads in the same core?

Correct.

-aneesh

Michael Ellerman

Applied

2016-10-27 11:38 UTC #3 lore raw

In reply to Aneesh Kumar K.V

Quoted 10 lines · Aneesh Kumar K.V

Before this patch, we used tlbiel, if we ever ran only on this core. That was mostly derived from the nohash usage of the same. But the ISA 3.0 clarifies tlbiel such that
"All TLB entries that have all of the following properties are made invalid on the thread executing the tlbiel instruction"
Hence use tlbiel, if we only ever ran on just the current cpu.
Signed-off-by Aneesh Kumar K.V <[email protected]>

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/bd77c4498616e27d5725b5959d880c

cheers