ARM: spinlock: ensure we have a compiler barrier before sev
Subsystems (2)
PATCH 1/1 ARM: spinlock: ensure we have a compiler barrier before sev
When unlocking a spinlock, we require the following, strictly ordered sequence of events:
<barrier> /* dmb */ <unlock> <barrier> /* dsb */ <sev>
Whilst the code does indeed reflect this in terms of the architecture, the final <barrier> + <sev> have been contracted into a single inline asm without a "memory" clobber, therefore the compiler is at liberty to reorder the unlock to the end of the above sequence. In such a case, a waiting CPU may be woken up before the lock has been unlocked, leading to extremely poor performance.
This patch reworks the dsb_sev() function to make use of the dsb() macro and ensure ordering against the unlock.
--- arch/arm/include/asm/spinlock.h | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-)
arch/arm/include/asm/spinlock.h +3 -12
@@ -37,18 +37,9 @@static inline void dsb_sev (void ) { -#if __LINUX_ARM_ARCH__ >=7 - __asm__ __volatile__ ( -"dsb ishst\n" -SEV - ); -#else - __asm__ __volatile__ ( -"mcr p15, 0, %0, c7, c10, 4\n" -SEV - : :"r" (0 ) - ); -#endif + +dsb (ishst); + __asm__(SEV); }/*
Signature · 1 line
1.8.2.2
PATCH 1/1 ARM: spinlock: ensure we have a compiler barrier before sev
Quoted 15 lines · Will Deacon
When unlocking a spinlock, we require the following, strictly ordered sequence of events:
<barrier> /* dmb */ <unlock> <barrier> /* dsb */ <sev>Whilst the code does indeed reflect this in terms of the architecture, the final <barrier> + <sev> have been contracted into a single inline asm without a "memory" clobber, therefore the compiler is at liberty to reorder the unlock to the end of the above sequence. In such a case, a waiting CPU may be woken up before the lock has been unlocked, leading to extremely poor performance.
Quoted 6 lines
This patch reworks the
dsb_sev()function to make use of thedsb()macro and ensure ordering against the unlock.Cc <[email protected]>Reported-by Mark Rutland <[email protected]>Signed-off-by Will Deacon <[email protected]>
FWIW: Reviewed-by: Christoffer Dall <[email protected]>