ARC: avoid unwanted gcc optimizations in atomic operations
Subsystems (3)
PATCH 1/1 ARC: avoid unwanted gcc optimizations in atomic operations
From: Pavel Kozlov <[email protected]>
Notify a compiler about write operations and prevent unwanted optimizations. Add the "memory" clobber to the clobber list.
An obvious problem with unwanted compiler optimizations appeared after the cpumask optimization commit 596ff4a09b89 ("cpumask: re-introduce constant-sized cpumask optimizations").
After this commit the SMP kernels for ARC no longer loads because of failed assert in the percpu allocator initialization routine:
percpu: BUG: failure at mm/percpu.c:2981/pcpu_build_alloc_info()!
The write operation performed by the scond instruction in the atomic inline asm code is not properly passed to the compiler. The compiler cannot correctly optimize a nested loop that runs through the cpumask in the pcpu_build_alloc_info() function.
Add the "memory" clobber to fix this.
--- arch/arc/include/asm/atomic-llsc.h | 6 +++--- arch/arc/include/asm/atomic64-arcv2.h | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-)
arch/arc/include/asm/atomic-llsc.h +3 -3
@@ -18,7 +18,7 @@ static inline void arch_atomic_##op(int i, atomic_t *v) \ : [val] "=&r" (val) /* Early clobber to prevent reg reuse */ \: [ctr] " r" (&v->counter), /* Not " m": llock only supports reg direct addr mode */ \ [i] " ir" (i) \ - : "cc"); \ + : "cc", "memory"); \ } \ #define ATOMIC_OP_RETURN(op, asm_op) \
@@ -34,7 +34,7 @@ static inline int arch_atomic_##op##_return_relaxed(int i, atomic_t *v) \ : [val] "=&r" (val) \: [ctr] " r" (&v->counter), \ [i] " ir" (i) \ -: " cc"); \ +: " cc", " memory"); \ \ return val; \ }
@@ -56,7 +56,7 @@ static inline int arch_atomic_fetch_##op##_relaxed(int i, atomic_t *v) \ [orig] "=&r" (orig) \: [ctr] " r" (&v->counter), \ [i] " ir" (i) \ -: " cc"); \ +: " cc", " memory"); \ \ return orig; \ }
arch/arc/include/asm/atomic64-arcv2.h +3 -3
@@ -60,7 +60,7 @@ static inline void arch_atomic64_##op(s64 a, atomic64_t *v) \" bnz 1b \n" \ : "=&r"(val) \: " r"(&v->counter), " ir"(a) \ - : "cc"); \ + : "cc", "memory"); \ } \ #define ATOMIC64_OP_RETURN(op, op1, op2) \
@@ -77,7 +77,7 @@ static inline s64 arch_atomic64_##op##_return_relaxed(s64 a, atomic64_t *v) \" bnz 1b \n" \ : [val] "=&r"(val) \: " r"(&v->counter), " ir"(a) \ -: " cc"); /* memory clobber comes from smp_mb() */ \ +: " cc", " memory"); \ \ return val; \ }
@@ -99,7 +99,7 @@ static inline s64 arch_atomic64_fetch_##op##_relaxed(s64 a, atomic64_t *v) \ " bnz 1b \n" \ : "=&r"(orig), "=&r"(val) \ : "r"(&v->counter), "ir"(a) \ - : "cc"); /* memory clobber comes from smp_mb() */ \ + : "cc", "memory"); \ \return orig; \ }
Signature · 7 lines
2.25.1 _______________________________________________ linux-snps-arc mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-snps-arc
Quoted 19 lines · On 8/15/23 08:11, [email protected] wrote:
From: Pavel Kozlov<[email protected]>
Notify a compiler about write operations and prevent unwanted optimizations. Add the "memory" clobber to the clobber list.
An obvious problem with unwanted compiler optimizations appeared after the cpumask optimization commit
596ff4a09b89("cpumask: re-introduce constant-sized cpumask optimizations").After this commit the SMP kernels for ARC no longer loads because of failed assert in the percpu allocator initialization routine:
percpu: BUG: failure at mm/percpu.c:2981/pcpu_build_alloc_info()!
The write operation performed by the scond instruction in the atomic inline asm code is not properly passed to the compiler. The compiler cannot correctly optimize a nested loop that runs through the cpumask in the
pcpu_build_alloc_info()function.
Quoted 5 lines
Add the "memory" clobber to fix this.
Cc <[email protected]> # v6.3+Signed-off-by Pavel Kozlov<[email protected]>
Before that commit, atomic ops could elide memory clobber because the trailing smp_mb() did that anyways. However after that commit, the smp_mb() was optional for relaxed variants and thus needs clobber.
_______________________________________________
linux-snps-arc mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-snps-arc