x86/mm/fault: Put pgd_lock in its own cacheline
PATCH 1/1 x86/mm/fault: Put pgd_lock in its own cacheline
On a large system with many cores, massive creation or destruction of processes/threads can sometime cause a fair amount of spinlock contention in the pgd_lock used by the pgd_alloc() and pgd_free() functions. This patch tries to reduce false cacheline sharing by putting the pgd_lock in its own cacheline which help to reduce contention on the lock.
--- arch/x86/mm/fault.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/mm/fault.c +1 -1
@@ -185,7 +185,7 @@ force_sig_info_fault(int si_signo, int si_code, unsigned long address,force_sig_info (si_signo, &info, tsk); } -DEFINE_SPINLOCK (pgd_lock); +__cacheline_aligned_in_smp DEFINE_SPINLOCK (pgd_lock );LIST_HEAD (pgd_list);#ifdef CONFIG_X86_32
Signature · 1 line
1.7.1
Quoted 17 lines · Waiman Long
On a large system with many cores, massive creation or destruction of processes/threads can sometime cause a fair amount of spinlock contention in the
pgd_lockused by thepgd_alloc()andpgd_free()functions. This patch tries to reduce false cacheline sharing by putting thepgd_lockin its own cacheline which help to reduce contention on the lock.Signed-off-by Waiman Long <[email protected]>--- arch/x86/mm/fault.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)arch/x86/mm/fault.c +0
-0
arch/x86/mm/fault.c force_sig_info_fault(int si_signo, int si_code, unsigned long address, · line 189
Quoted 6 lines
arch/x86/mm/fault.c +1
-1@@ -185,7 +185,7 @@ force_sig_info_fault(int si_signo, int si_code, unsigned long address,force_sig_info (si_signo, &info, tsk); } -DEFINE_SPINLOCK (pgd_lock); +__cacheline_aligned_in_smp DEFINE_SPINLOCK (pgd_lock );LIST_HEAD (pgd_list);
Yeah, so if you are seeing pgd_lock contention we could do a lot more to improve the performance of pgd allocation than just putting it on a separate cacheline: we can actually eliminate pgd_list and turn all pgd_list users into RCU walkers of the tasklist lock, and make pgd_alloc()/pgd_free() lockless on x86.
I've done a series that implements this, I'll post it soon, please stay tuned!
Thanks,
Ingo