sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty
RFC PATCH 1/1 sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty
On archs except x86, CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE is not defined. We found membarrier_mm_sync_core_before_usermode() looks like this when compiled by gcc10:
if (current->mm != mm) return; atomic_read(&mm->membarrier_state);
This memory access is unnecessary. Remove it to improve performance.
--- include/linux/sched/mm.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/linux/sched/mm.h +6 -0
@@ -401,6 +401,7 @@ enum {#include <asm/membarrier.h> #endif +#ifdef CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE static inline void membarrier_mm_sync_core_before_usermode (struct mm_struct *mm) {if (current->mm != mm)
@@ -410,6 +411,11 @@ static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)return ; sync_core_before_usermode(); } +#else +static inline void membarrier_mm_sync_core_before_usermode (struct mm_struct *mm) +{ +} +#endif extern void membarrier_exec_mmap (struct mm_struct *mm);
Signature · 1 line
2.33.0
We've run schbench and found wakeup latency on some arm64 machines worse than others. Perf shows there's a hotspot on atomic_read(&mm->membarrier_state);
We're still working for the real reason behind it (maybe cache or sth hardware related), and we do see remove this function can help improve performance.
Thanks.
Quoted 39 lines · Tianchen Ding
On archs except x86,
CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODEis not defined. We foundmembarrier_mm_sync_core_before_usermode()looks like this when compiled by gcc10:if (current->mm != mm) return; atomic_read(&mm->membarrier_state);This memory access is unnecessary. Remove it to improve performance.
Signed-off-by Tianchen Ding <[email protected]>--- include/linux/sched/mm.h | 6 ++++++ 1 file changed, 6 insertions(+)include/linux/sched/mm.h +6
-0@@ -401,6 +401,7 @@ enum {#include <asm/membarrier.h> #endif +#ifdef CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE static inline void membarrier_mm_sync_core_before_usermode (struct mm_struct *mm) {if (current->mm != mm)@@ -410,6 +411,11 @@ static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)return ; sync_core_before_usermode(); } +#else +static inline void membarrier_mm_sync_core_before_usermode (struct mm_struct *mm) +{ +} +#endif extern void membarrier_exec_mmap (struct mm_struct *mm);