Skip to contentSkip to conversation

x86/mm/fault: Put pgd_lock in its own cacheline

PATCH In review Waiman Long 2 participants 2 messages Jun 10, 2015 – Jun 11, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/mm/fault: Put pgd_lock in its own cacheline

On a large system with many cores, massive creation or destruction of processes/threads can sometime cause a fair amount of spinlock contention in the pgd_lock used by the pgd_alloc() and pgd_free() functions. This patch tries to reduce false cacheline sharing by putting the pgd_lock in its own cacheline which help to reduce contention on the lock.

---
 arch/x86/mm/fault.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/mm/fault.c +1 -1
index 181c53b..fae48df 100644
@@ -185,7 +185,7 @@ force_sig_info_fault(int si_signo, int si_code, unsigned long address,
 	force_sig_info(si_signo, &info, tsk);
 }
 
-DEFINE_SPINLOCK(pgd_lock);
+__cacheline_aligned_in_smp DEFINE_SPINLOCK(pgd_lock);
 LIST_HEAD(pgd_list);
 
 #ifdef CONFIG_X86_32
Signature · 1 line
1.7.1
#1 lore raw
Quoted 17 lines · Waiman Long

On a large system with many cores, massive creation or destruction of processes/threads can sometime cause a fair amount of spinlock contention in the pgd_lock used by the pgd_alloc() and pgd_free() functions. This patch tries to reduce false cacheline sharing by putting the pgd_lock in its own cacheline which help to reduce contention on the lock.

---
 arch/x86/mm/fault.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/mm/fault.c +0 -0
index 181c53b..fae48df 100644

arch/x86/mm/fault.c force_sig_info_fault(int si_signo, int si_code, unsigned long address, · line 189

Quoted 6 lines
arch/x86/mm/fault.c +1 -1
@@ -185,7 +185,7 @@ force_sig_info_fault(int si_signo, int si_code, unsigned long address,
 	force_sig_info(si_signo, &info, tsk);
 }
 
-DEFINE_SPINLOCK(pgd_lock);
+__cacheline_aligned_in_smp DEFINE_SPINLOCK(pgd_lock);
 LIST_HEAD(pgd_list);

Yeah, so if you are seeing pgd_lock contention we could do a lot more to improve the performance of pgd allocation than just putting it on a separate cacheline: we can actually eliminate pgd_list and turn all pgd_list users into RCU walkers of the tasklist lock, and make pgd_alloc()/pgd_free() lockless on x86.

I've done a series that implements this, I'll post it soon, please stay tuned!

Thanks,

Ingo