Skip to contentSkip to conversation

sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty

RFC PATCH In review Tianchen Ding 1 participant 2 messages Apr 2, 2022 – Apr 6, 2022

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty

On archs except x86, CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE is not defined. We found membarrier_mm_sync_core_before_usermode() looks like this when compiled by gcc10:

	if (current->mm != mm)
		return;
	atomic_read(&mm->membarrier_state);

This memory access is unnecessary. Remove it to improve performance.

---
 include/linux/sched/mm.h | 6 ++++++
 1 file changed, 6 insertions(+)
include/linux/sched/mm.h +6 -0
index a80356e9dc69..3ded68d9f913 100644
@@ -401,6 +401,7 @@ enum {
 #include <asm/membarrier.h>
 #endif
 
+#ifdef CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
 static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
 {
 	if (current->mm != mm)
@@ -410,6 +411,11 @@ static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
 		return;
 	sync_core_before_usermode();
 }
+#else
+static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
+{
+}
+#endif
 
 extern void membarrier_exec_mmap(struct mm_struct *mm);
 
Signature · 1 line
2.33.0
#1 lore raw

We've run schbench and found wakeup latency on some arm64 machines worse than others. Perf shows there's a hotspot on atomic_read(&mm->membarrier_state);

We're still working for the real reason behind it (maybe cache or sth hardware related), and we do see remove this function can help improve performance.

Thanks.

Quoted 39 lines · Tianchen Ding

On archs except x86, CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE is not defined. We found membarrier_mm_sync_core_before_usermode() looks like this when compiled by gcc10:

	if (current->mm != mm)
		return;
	atomic_read(&mm->membarrier_state);

This memory access is unnecessary. Remove it to improve performance.

---
  include/linux/sched/mm.h | 6 ++++++
  1 file changed, 6 insertions(+)
include/linux/sched/mm.h +6 -0
index a80356e9dc69..3ded68d9f913 100644
@@ -401,6 +401,7 @@ enum {
  #include <asm/membarrier.h>
  #endif
 
+#ifdef CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
  static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
  {
  	if (current->mm != mm)
@@ -410,6 +411,11 @@ static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
  		return;
  	sync_core_before_usermode();
  }
+#else
+static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
+{
+}
+#endif
 
  extern void membarrier_exec_mmap(struct mm_struct *mm);