Introduce __xchg, non-atomic xchg
Subsystems (23)
v5 cover Introduce __xchg, non-atomic xchg
Hi all,
The helper is tiny and there are advices we can live without it, so I want to present few arguments why it would be good to have it:
1. Code readability/simplification/number of lines:
- decreases number of lines,
- it often eliminates local variables,
- for real examples see patches 3+.
2. Presence of similar helpers in other somehow related languages/libs:
a) Rust[1]: 'replace' from std::mem module, there is also 'take' helper (__xchg(&x, 0)), which is the same as private helper in i915 - fetch_and_zero, see latest patch.
b) C++ [2]: 'exchange' from utility header.
If the idea is OK there are still 2 questions to answer:
1. Name of the helper, __xchg follows kernel conventions, but for me Rust names are also OK.
2. Where to put the helper:
a) as in this patchset include/linux/non-atomic/xchg.h, proposed by Andy Shevchenko,
b) include/linux/utils.h ? any better name? Some kind of container for simple helpers.
All __xchg conversions were performed using cocci script, then manually adjusted if necessary.
There is lot of places it can be used in, I have just chosen some of them. I can provide cocci script to detect others (not all), if necessary.
Changes:
v2: squashed all __xchg -> __arch_xchg t one patch (Arnd)
v3: fixed alpha/xchg_local ([email protected])
v4: adjusted indentation (Heiko)
v5: added more __xchg conversions - patches 3-6, added tags
[1]: https://doc.rust-lang.org/std/mem/index.html
[2]: https://en.cppreference.com/w/cpp/header/utility
Regards
Andrzej
Andrzej Hajda (7):
arch: rename all internal names __xchg to __arch_xchg linux/include: add non-atomic version of xchg arch/*/uprobes: simplify arch_uretprobe_hijack_return_addr
llist: simplify __llist_del_all
io_uring: use __xchg if possible
qed: use __xchg if possible
drm/i915/gt: use __xchg instead of internal helper
arch/alpha/include/asm/cmpxchg.h | 10 +++++----- arch/arc/include/asm/cmpxchg.h | 4 ++-- arch/arm/include/asm/cmpxchg.h | 7 ++++--- arch/arm/probes/uprobes/core.c | 8 ++------ arch/arm64/include/asm/cmpxchg.h | 7 +++---- arch/arm64/kernel/probes/uprobes.c | 9 ++------- arch/csky/kernel/probes/uprobes.c | 9 ++------- arch/hexagon/include/asm/cmpxchg.h | 10 +++++----- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 ++-- arch/loongarch/include/asm/cmpxchg.h | 4 ++-- arch/m68k/include/asm/cmpxchg.h | 6 +++--- arch/mips/include/asm/cmpxchg.h | 4 ++-- arch/mips/kernel/uprobes.c | 10 ++-------- arch/openrisc/include/asm/cmpxchg.h | 10 +++++----- arch/parisc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/kernel/uprobes.c | 10 ++-------- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 ++-- arch/riscv/kernel/probes/uprobes.c | 9 ++------- arch/s390/include/asm/cmpxchg.h | 8 ++++---- arch/s390/kernel/uprobes.c | 7 ++----- arch/sh/include/asm/cmpxchg.h | 4 ++-- arch/sparc/include/asm/cmpxchg_32.h | 4 ++-- arch/sparc/include/asm/cmpxchg_64.h | 6 +++--- arch/sparc/kernel/uprobes.c | 7 ++----- arch/xtensa/include/asm/cmpxchg.h | 4 ++-- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +- .../gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++-- .../drm/i915/gt/intel_execlists_submission.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gsc.c | 2 +- drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gt_pm.c | 2 +- drivers/gpu/drm/i915/gt/intel_lrc.c | 6 +++--- drivers/gpu/drm/i915/gt/intel_migrate.c | 2 +- drivers/gpu/drm/i915/gt/intel_rc6.c | 2 +- drivers/gpu/drm/i915/gt/intel_rps.c | 2 +- drivers/gpu/drm/i915/gt/selftest_context.c | 2 +- .../drm/i915/gt/selftest_ring_submission.c | 2 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 2 +- drivers/gpu/drm/i915/i915_utils.h | 1 + include/linux/llist.h | 6 ++---- include/linux/non-atomic/xchg.h | 19 +++++++++++++++++++ include/linux/qed/qed_chain.h | 19 +++++++------------ io_uring/io_uring.c | 7 ++----- io_uring/slist.h | 6 ++---- 51 files changed, 126 insertions(+), 155 deletions(-) create mode 100644 include/linux/non-atomic/xchg.h
Signature · 1 line
2.34.1
PATCH v5 1/7 arch: rename all internal names __xchg to __arch_xchg
__xchg will be used for non-atomic xchg macro.
---
v2: squashed all arch patches into one
v3: fixed alpha/xchg_local, thx to [email protected]
v4: adjusted indentation (Heiko)
--- arch/alpha/include/asm/cmpxchg.h | 10 +++++----- arch/arc/include/asm/cmpxchg.h | 4 ++-- arch/arm/include/asm/cmpxchg.h | 7 ++++--- arch/arm64/include/asm/cmpxchg.h | 7 +++---- arch/hexagon/include/asm/cmpxchg.h | 10 +++++----- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 ++-- arch/loongarch/include/asm/cmpxchg.h | 4 ++-- arch/m68k/include/asm/cmpxchg.h | 6 +++--- arch/mips/include/asm/cmpxchg.h | 4 ++-- arch/openrisc/include/asm/cmpxchg.h | 10 +++++----- arch/parisc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/include/asm/cmpxchg.h | 4 ++-- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 ++-- arch/s390/include/asm/cmpxchg.h | 8 ++++---- arch/sh/include/asm/cmpxchg.h | 4 ++-- arch/sparc/include/asm/cmpxchg_32.h | 4 ++-- arch/sparc/include/asm/cmpxchg_64.h | 6 +++--- arch/xtensa/include/asm/cmpxchg.h | 4 ++-- 20 files changed, 54 insertions(+), 54 deletions(-)
arch/alpha/include/asm/cmpxchg.h +5 -5
@@ -6,15 +6,15 @@* Atomic exchange routines. */ -#define ____xchg(type, args...) __xchg ## type ## _local(args) +#define ____xchg(type, args...) __arch_xchg ## type ## _local(args) #define ____cmpxchg(type, args...) __cmpxchg ## type ## _local(args)#include <asm/xchg.h> #define xchg_local (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_local((ptr), (unsigned long)_x_, \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg_local((ptr), (unsigned long)_x_,\ + sizeof(*(ptr))); \ })#define arch_cmpxchg_local (ptr, o, n) \
@@ -34,7 +34,7 @@#undef ____xchg#undef ____cmpxchg -#define ____xchg(type, args...) __xchg ##type(args) +#define ____xchg(type, args...) __arch_xchg ##type(args) #define ____cmpxchg(type, args...) __cmpxchg ##type(args)#include <asm/xchg.h>
@@ -48,7 +48,7 @@ __typeof__(*(ptr)) _x_ = (x); \smp_mb (); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \smp_mb (); \ __ret; \ })
arch/arc/include/asm/cmpxchg.h +2 -2
@@ -85,7 +85,7 @@*/ #ifdef CONFIG_ARC_HAS_LLSC -#define __xchg (ptr, val) \ +#define __arch_xchg (ptr, val) \ ({ \ __asm__ __volatile__( \ " ex %0, [%1] \n"/* set new value */ \
@@ -102,7 +102,7 @@ \switch (sizeof (*(_p_))) { \case 4 : \ - _val_ =__xchg (_p_, _val_); \ + _val_ =__arch_xchg (_p_, _val_); \break ; \default : \BUILD_BUG (); \
arch/arm/include/asm/cmpxchg.h +4 -3
@@ -25,7 +25,8 @@#define swp_is_buggy#endif -static inline unsigned long __xchg (unsigned long x,volatile void *ptr,int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {extern void __bad_xchg (volatile void *,int );unsigned long ret;
@@ -115,8 +116,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size }#define arch_xchg_relaxed (ptr, x) ({ \ - (__typeof__(*(ptr)))__xchg((unsigned long)(x), (ptr), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr)))__arch_xchg((unsigned long)(x), (ptr), \ + sizeof(*(ptr))); \ })#include <asm-generic/cmpxchg-local.h>
arch/arm64/include/asm/cmpxchg.h +3 -4
@@ -62,9 +62,8 @@ __XCHG_CASE( , , mb_, 64, dmb ish, nop, , a, l, "memory")#undef __XCHG_CASE#define __XCHG_GEN (sfx) \ -static __always_inline unsigned long __xchg##sfx(unsigned long x, \ - volatile void *ptr, \ - int size) \ +static __always_inline unsigned long \ +__arch_xchg##sfx(unsigned long x, volatile void *ptr, int size) \ { \ switch (size) { \ case 1: \
@@ -93,7 +92,7 @@ __XCHG_GEN(_mb) ({ \ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
arch/hexagon/include/asm/cmpxchg.h +5 -5
@@ -9,7 +9,7 @@#define _ASM_CMPXCHG_H/* -* __xchg - atomically exchange a register and a memory location +* __arch_xchg - atomically exchange a register and a memory location * @x: value to swap * @ptr: pointer to memory * @size: size of the value
@@ -19,8 +19,8 @@* Note: there was an errata for V2 about .new's and memw_locked. * */ -static inline unsigned long __xchg (unsigned long x,volatile void *ptr, -int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {unsigned long retval;
@@ -42,8 +42,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr,* Atomically swap the contents of a register with memory. Should be atomic * between multiple CPU's and within interrupts on the same CPU. */ -#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__xchg((unsigned long)(v), (ptr), \ - sizeof(*(ptr)))) +#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(v), (ptr), \ + sizeof(*(ptr))))/* * see rt-mutex-design.txt; cmpxchg supposedly checks if *ptr == A and swaps.
arch/ia64/include/asm/cmpxchg.h +1 -1
@@ -5,7 +5,7 @@#include <uapi/asm/cmpxchg.h> #define arch_xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#define arch_cmpxchg (ptr, o, n) cmpxchg_acq((ptr), (o), (n))#define arch_cmpxchg64 (ptr, o, n) cmpxchg_acq((ptr), (o), (n))
arch/ia64/include/uapi/asm/cmpxchg.h +2 -2
@@ -27,7 +27,7 @@*/ extern void ia64_xchg_called_with_bad_pointer (void ); -#define __xchg (x, ptr, size) \ +#define __arch_xchg (x, ptr, size) \ ({ \ unsigned long __xchg_result; \ \
@@ -55,7 +55,7 @@ extern void ia64_xchg_called_with_bad_pointer(void);#ifndef __KERNEL__#define xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#endif /*
arch/loongarch/include/asm/cmpxchg.h +2 -2
@@ -62,7 +62,7 @@ static inline unsigned int __xchg_small(volatile void *ptr, unsigned int val, }static __always_inline unsigned long -__xchg (volatile void *ptr,unsigned long x,int size) +__arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -87,7 +87,7 @@ __xchg(volatile void *ptr, unsigned long x, int size) __typeof__(*(ptr)) __res; \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \ __res; \ })
arch/m68k/include/asm/cmpxchg.h +3 -3
@@ -9,7 +9,7 @@extern unsigned long __invalid_xchg_size (unsigned long ,volatile void *,int );#ifndef CONFIG_RMW_INSNS -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {unsigned long flags, tmp;
@@ -40,7 +40,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz return x; }#else -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
@@ -75,7 +75,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz }#endif -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})#include <asm-generic/cmpxchg-local.h>
arch/mips/include/asm/cmpxchg.h +2 -2
@@ -68,7 +68,7 @@ extern unsigned long __xchg_small(volatile void *ptr, unsigned long val, unsigned int size);static __always_inline -unsigned long __xchg (volatile void *ptr,unsigned long x,int size) +unsigned long __arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -102,7 +102,7 @@ unsigned long __xchg(volatile void *ptr, unsigned long x, int size)smp_mb__before_llsc (); \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \smp_llsc_mb (); \ \
arch/openrisc/include/asm/cmpxchg.h +5 -5
@@ -147,8 +147,8 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, extern unsigned long __xchg_called_with_bad_pointer(void) __compiletime_error("Bad argument size for xchg"); -static inline unsigned long __xchg (volatile void *ptr,unsigned long with, -int size) +static inline unsigned long +__arch_xchg (volatile void *ptr,unsigned long with,int size) {switch (size) {case 1 :
@@ -163,9 +163,9 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long with,#define arch_xchg (ptr, with) \ ({ \ - (__typeof__(*(ptr))) __xchg((ptr), \ - (unsigned long)(with), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), \ + (unsigned long)(with), \ + sizeof(*(ptr))); \ })#endif /* __ASM_OPENRISC_CMPXCHG_H */
arch/parisc/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@ extern unsigned long __xchg64(unsigned long, volatile unsigned long *);/* optimizer better get rid of switch since size is a constant */ static inline unsigned long -__xchg (unsigned long x,volatile void *ptr,int size) +__arch_xchg (unsigned long x,volatile void *ptr,int size) {switch (size) {#ifdef CONFIG_64BIT
@@ -49,7 +49,7 @@ __xchg(unsigned long x, volatile void *ptr, int size) __typeof__(*(ptr)) __ret; \ __typeof__(*(ptr)) _x_ = (x); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ __ret; \ })
arch/powerpc/include/asm/cmpxchg.h +2 -2
@@ -229,7 +229,7 @@ __xchg_local(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_local (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" );return x; }
@@ -248,7 +248,7 @@ __xchg_relaxed(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_relaxed (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_relaxed" );return x; }#define arch_xchg_local (ptr,x) \
arch/riscv/include/asm/atomic.h +1 -1
@@ -261,7 +261,7 @@ c_t arch_atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \ static __always_inline \ c_t arch_atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ { \ - return __xchg(&(v->counter), n, size); \ + return __arch_xchg(&(v->counter), n, size); \ } \ static __always_inline \ c_t arch_atomic##prefix##_cmpxchg_relaxed(atomic##prefix##_t *v, \
arch/riscv/include/asm/cmpxchg.h +2 -2
@@ -114,7 +114,7 @@ _x_, sizeof(*(ptr))); \ }) -#define __xchg (ptr, new, size) \ +#define __arch_xchg (ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(new) __new = (new); \
@@ -143,7 +143,7 @@#define arch_xchg (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg((ptr), _x_, sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ })#define xchg32 (ptr, x) \
arch/s390/include/asm/cmpxchg.h +4 -4
@@ -14,8 +14,8 @@void __xchg_called_with_bad_pointer (void ); -static __always_inline unsigned long __xchg (unsigned long x, -unsigned long address,int size) +static __always_inline unsigned long +__arch_xchg (unsigned long x,unsigned long address,int size) {unsigned long old;int shift;
@@ -77,8 +77,8 @@ static __always_inline unsigned long __xchg(unsigned long x, __typeof__(*(ptr)) __ret; \ \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )(x), (unsigned long )(ptr), \ -sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )(x), (unsigned long )(ptr), \ +sizeof (*(ptr))); \ __ret; \ })
arch/sh/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@extern void __xchg_called_with_bad_pointer (void ); -#define __xchg (ptr, x, size) \ +#define __arch_xchg (ptr, x, size) \ ({ \ unsigned long __xchg__res; \ volatile void *__xchg_ptr = (ptr); \
@@ -46,7 +46,7 @@ extern void __xchg_called_with_bad_pointer(void); })#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((ptr),(unsigned long)(x), sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((ptr),(unsigned long)(x), sizeof(*(ptr))))/* This function doesn't exist, so you'll get a linker error * if something tries to do an invalid cmpxchg(). */
arch/sparc/include/asm/cmpxchg_32.h +2 -2
@@ -15,7 +15,7 @@unsigned long __xchg_u32 (volatile u32 *m,u32 new);void __xchg_called_with_bad_pointer (void ); -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, int size) +static inline unsigned long __arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 4 :
@@ -25,7 +25,7 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, intreturn x; } -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})/* Emulate cmpxchg() the same way we emulate atomics, * by hashing the object address and indexing into an array
arch/sparc/include/asm/cmpxchg_64.h +3 -3
@@ -55,7 +55,7 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long#define arch_xchg (ptr,x) \ ({ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
@@ -87,8 +87,8 @@ xchg16(__volatile__ unsigned short *m, unsigned short val)return (load32 & mask) >> bit_shift; } -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, -int size) +static inline unsigned long +__arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 2 :
arch/xtensa/include/asm/cmpxchg.h +2 -2
@@ -170,7 +170,7 @@ static inline unsigned long xchg_u32(volatile int * m, unsigned long val) }#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))static inline u32 xchg_small (volatile void *ptr,u32 x,int size) {
@@ -203,7 +203,7 @@ static inline u32 xchg_small(volatile void *ptr, u32 x, int size)extern void __xchg_called_with_bad_pointer (void );static __inline__unsigned long -__xchg (unsigned long x,volatile void * ptr,int size) +__arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
Signature · 1 line
2.34.1
PATCH v5 2/7 linux/include: add non-atomic version of xchg
The pattern of setting variable with new value and returning old one is very common in kernel. Usually atomicity of the operation is not required, so xchg seems to be suboptimal and confusing in such cases.
--- include/linux/non-atomic/xchg.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 include/linux/non-atomic/xchg.h
include/linux/non-atomic/xchg.h new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_NON_ATOMIC_XCHG_H +#define _LINUX_NON_ATOMIC_XCHG_H + +/** +* __xchg - set variable pointed by @ptr to @val, return old value +* @ptr: pointer to affected variable +* @val: value to be written +* +* This is non-atomic variant of xchg. +*/ +#define __xchg (ptr, val) ({ \ + __auto_type __ptr = ptr; \ + __auto_type __t = *__ptr; \ + *__ptr = (val); \ + __t; \ +}) + +#endif
Signature · 1 line
2.34.1
Re: [Intel-gfx] [PATCH v5 2/7] linux/include: add non-atomic version of xchg
Hi Andrzej,
Quoted 7 lines · Andrzej Hajda
The pattern of setting variable with new value and returning old one is very common in kernel. Usually atomicity of the operation is not required, so xchg seems to be suboptimal and confusing in such cases.
Signed-off-by Andrzej Hajda <[email protected]>Reviewed-by Andy Shevchenko <[email protected]>
Thanks,
Andi
[tip: locking/core] linux/include: add non-atomic version of xchg
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 6e58fca8f12015d783df9d155bbb969a09a29e64 Gitweb: https://git.kernel.org/tip/6e58fca8f12015d783df9d155bbb969a09a29e64 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:45 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:03:58 +01:00
linux/include: add non-atomic version of xchg
The pattern of setting variable with new value and returning old one is very common in kernel. Usually atomicity of the operation is not required, so xchg seems to be suboptimal and confusing in such cases.
--- include/linux/non-atomic/xchg.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 include/linux/non-atomic/xchg.h
include/linux/non-atomic/xchg.h new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_NON_ATOMIC_XCHG_H +#define _LINUX_NON_ATOMIC_XCHG_H + +/** +* __xchg - set variable pointed by @ptr to @val, return old value +* @ptr: pointer to affected variable +* @val: value to be written +* +* This is non-atomic variant of xchg. +*/ +#define __xchg (ptr, val) ({ \ + __auto_type __ptr = ptr; \ + __auto_type __t = *__ptr; \ + *__ptr = (val); \ + __t; \ +}) + +#endif
PATCH v5 3/7 arch/*/uprobes: simplify arch_uretprobe_hijack_return_addr
In reply to Andrzej HajdaIn all architectures, except x86, arch_uretprobe_hijack_return_addr is just __xchg.
--- arch/arm/probes/uprobes/core.c | 8 ++------ arch/arm64/kernel/probes/uprobes.c | 9 ++------- arch/csky/kernel/probes/uprobes.c | 9 ++------- arch/mips/kernel/uprobes.c | 10 ++-------- arch/powerpc/kernel/uprobes.c | 10 ++-------- arch/riscv/kernel/probes/uprobes.c | 9 ++------- arch/s390/kernel/uprobes.c | 7 ++----- arch/sparc/kernel/uprobes.c | 7 ++----- 8 files changed, 16 insertions(+), 53 deletions(-)
arch/arm/probes/uprobes/core.c +2 -6
@@ -9,6 +9,7 @@#include <linux/highmem.h> #include <linux/sched.h> #include <linux/uprobes.h> +#include <linux/non-atomic/xchg.h> #include <linux/notifier.h> #include <asm/opcodes.h>
@@ -61,12 +62,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long orig_ret_vaddr; - - orig_ret_vaddr = regs->ARM_lr; -/* Replace the return addr with trampoline addr */ - regs->ARM_lr = trampoline_vaddr; -return orig_ret_vaddr; +return __xchg (®s->ARM_lr, trampoline_vaddr); } int arch_uprobe_analyze_insn(struct arch_uprobe *auprobe, struct mm_struct *mm,
arch/arm64/kernel/probes/uprobes.c +2 -7
@@ -3,6 +3,7 @@* Copyright (C) 2014-2016 Pratyush Anand <[email protected]> */ #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/ptrace.h> #include <linux/uprobes.h> #include <asm/cacheflush.h>
@@ -150,13 +151,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long orig_ret_vaddr; - - orig_ret_vaddr =procedure_link_pointer (regs); -/* Replace the return addr with trampoline addr */ -procedure_link_pointer_set (regs, trampoline_vaddr); - -return orig_ret_vaddr; +return __xchg (&procedure_link_pointer (regs), trampoline_vaddr); } int arch_uprobe_exception_notify(struct notifier_block *self,
arch/csky/kernel/probes/uprobes.c +2 -7
@@ -3,6 +3,7 @@* Copyright (C) 2014-2016 Pratyush Anand <[email protected]> */ #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/ptrace.h> #include <linux/uprobes.h> #include <asm/cacheflush.h>
@@ -123,13 +124,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long ra; - - ra = regs->lr; - - regs->lr = trampoline_vaddr; - -return ra; +return __xchg (®s->lr, trampoline_vaddr); } int arch_uprobe_exception_notify(struct notifier_block *self,
arch/mips/kernel/uprobes.c +2 -8
@@ -2,6 +2,7 @@#include <linux/highmem.h> #include <linux/kdebug.h> #include <linux/types.h> +#include <linux/non-atomic/xchg.h> #include <linux/notifier.h> #include <linux/sched.h> #include <linux/uprobes.h>
@@ -197,14 +198,7 @@ void arch_uprobe_abort_xol(struct arch_uprobe *aup,unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline_vaddr,struct pt_regs *regs) { -unsigned long ra; - - ra = regs->regs[31 ]; - -/* Replace the return address with the trampoline address */ - regs->regs[31 ] = trampoline_vaddr; - -return ra; +return __xchg (®s->regs[31 ], trampoline_vaddr); }/**
arch/powerpc/kernel/uprobes.c +2 -8
@@ -7,6 +7,7 @@* Adapted from the x86 port by Ananth N Mavinakayanahalli <[email protected]> */ #include <linux/kernel.h> +#include <linux/non-atomic/xchg.h> #include <linux/sched.h> #include <linux/ptrace.h> #include <linux/uprobes.h>
@@ -197,14 +198,7 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline_vaddr,struct pt_regs *regs) { -unsigned long orig_ret_vaddr; - - orig_ret_vaddr = regs->link; - -/* Replace the return addr with trampoline addr */ - regs->link = trampoline_vaddr; - -return orig_ret_vaddr; +return __xchg (®s->link, trampoline_vaddr); } bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check ctx,
arch/riscv/kernel/probes/uprobes.c +2 -7
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0-only #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/ptrace.h> #include <linux/uprobes.h>
@@ -122,13 +123,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long ra; - - ra = regs->ra; - - regs->ra = trampoline_vaddr; - -return ra; +return __xchg (®s->ra, trampoline_vaddr); } int arch_uprobe_exception_notify(struct notifier_block *self,
arch/s390/kernel/uprobes.c +2 -5
@@ -11,6 +11,7 @@#include <linux/compat.h> #include <linux/kdebug.h> #include <linux/sched/task_stack.h> +#include <linux/non-atomic/xchg.h> #include <asm/switch_to.h> #include <asm/facility.h>
@@ -144,11 +145,7 @@ void arch_uprobe_abort_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline,struct pt_regs *regs) { -unsigned long orig; - - orig = regs->gprs[14 ]; - regs->gprs[14 ] = trampoline; -return orig; +return __xchg (®s->gprs[14 ], trampoline); } bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check ctx,
arch/sparc/kernel/uprobes.c +2 -5
@@ -11,6 +11,7 @@#include <linux/kernel.h> #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/uprobes.h> #include <linux/uaccess.h> #include <linux/sched.h> /* For struct task_struct */
@@ -310,9 +311,5 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long orig_ret_vaddr = regs->u_regs[UREG_I7 ]; - - regs->u_regs[UREG_I7 ] = trampoline_vaddr-8 ; - -return orig_ret_vaddr +8 ; +return __xchg (®s->u_regs[UREG_I7 ], trampoline_vaddr -8 ) +8 ; }
Signature · 1 line
2.34.1
Re: [Intel-gfx] [PATCH v5 3/7] arch/*/uprobes: simplify arch_uretprobe_hijack_return_addr
Hi Andrzej,
Quoted 4 lines · Andrzej Hajda
In all architectures, except x86,
arch_uretprobe_hijack_return_addris just__xchg.Signed-off-by Andrzej Hajda <[email protected]>
Thanks,
Andi
[tip: locking/core] arch/*/uprobes: simplify arch_uretprobe_hijack_return_addr
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 26ace5d28d3645efe735c3c2ff0b80dd17c85cb2 Gitweb: https://git.kernel.org/tip/26ace5d28d3645efe735c3c2ff0b80dd17c85cb2 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:46 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:03:59 +01:00
arch/*/uprobes: simplify arch_uretprobe_hijack_return_addr
In all architectures, except x86, arch_uretprobe_hijack_return_addr is just __xchg.
--- arch/arm/probes/uprobes/core.c | 8 ++------ arch/arm64/kernel/probes/uprobes.c | 9 ++------- arch/csky/kernel/probes/uprobes.c | 9 ++------- arch/mips/kernel/uprobes.c | 10 ++-------- arch/powerpc/kernel/uprobes.c | 10 ++-------- arch/riscv/kernel/probes/uprobes.c | 9 ++------- arch/s390/kernel/uprobes.c | 7 ++----- arch/sparc/kernel/uprobes.c | 7 ++----- 8 files changed, 16 insertions(+), 53 deletions(-)
arch/arm/probes/uprobes/core.c +2 -6
@@ -9,6 +9,7 @@#include <linux/highmem.h> #include <linux/sched.h> #include <linux/uprobes.h> +#include <linux/non-atomic/xchg.h> #include <linux/notifier.h> #include <asm/opcodes.h>
@@ -61,12 +62,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long orig_ret_vaddr; - - orig_ret_vaddr = regs->ARM_lr; -/* Replace the return addr with trampoline addr */ - regs->ARM_lr = trampoline_vaddr; -return orig_ret_vaddr; +return __xchg (®s->ARM_lr, trampoline_vaddr); } int arch_uprobe_analyze_insn(struct arch_uprobe *auprobe, struct mm_struct *mm,
arch/arm64/kernel/probes/uprobes.c +2 -7
@@ -3,6 +3,7 @@* Copyright (C) 2014-2016 Pratyush Anand <[email protected]> */ #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/ptrace.h> #include <linux/uprobes.h> #include <asm/cacheflush.h>
@@ -150,13 +151,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long orig_ret_vaddr; - - orig_ret_vaddr =procedure_link_pointer (regs); -/* Replace the return addr with trampoline addr */ -procedure_link_pointer_set (regs, trampoline_vaddr); - -return orig_ret_vaddr; +return __xchg (&procedure_link_pointer (regs), trampoline_vaddr); } int arch_uprobe_exception_notify(struct notifier_block *self,
arch/csky/kernel/probes/uprobes.c +2 -7
@@ -3,6 +3,7 @@* Copyright (C) 2014-2016 Pratyush Anand <[email protected]> */ #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/ptrace.h> #include <linux/uprobes.h> #include <asm/cacheflush.h>
@@ -123,13 +124,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long ra; - - ra = regs->lr; - - regs->lr = trampoline_vaddr; - -return ra; +return __xchg (®s->lr, trampoline_vaddr); } int arch_uprobe_exception_notify(struct notifier_block *self,
arch/mips/kernel/uprobes.c +2 -8
@@ -2,6 +2,7 @@#include <linux/highmem.h> #include <linux/kdebug.h> #include <linux/types.h> +#include <linux/non-atomic/xchg.h> #include <linux/notifier.h> #include <linux/sched.h> #include <linux/uprobes.h>
@@ -197,14 +198,7 @@ void arch_uprobe_abort_xol(struct arch_uprobe *aup,unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline_vaddr,struct pt_regs *regs) { -unsigned long ra; - - ra = regs->regs[31 ]; - -/* Replace the return address with the trampoline address */ - regs->regs[31 ] = trampoline_vaddr; - -return ra; +return __xchg (®s->regs[31 ], trampoline_vaddr); }/**
arch/powerpc/kernel/uprobes.c +2 -8
@@ -7,6 +7,7 @@* Adapted from the x86 port by Ananth N Mavinakayanahalli <[email protected]> */ #include <linux/kernel.h> +#include <linux/non-atomic/xchg.h> #include <linux/sched.h> #include <linux/ptrace.h> #include <linux/uprobes.h>
@@ -197,14 +198,7 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline_vaddr,struct pt_regs *regs) { -unsigned long orig_ret_vaddr; - - orig_ret_vaddr = regs->link; - -/* Replace the return addr with trampoline addr */ - regs->link = trampoline_vaddr; - -return orig_ret_vaddr; +return __xchg (®s->link, trampoline_vaddr); } bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check ctx,
arch/riscv/kernel/probes/uprobes.c +2 -7
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0-only #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/ptrace.h> #include <linux/uprobes.h>
@@ -122,13 +123,7 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long ra; - - ra = regs->ra; - - regs->ra = trampoline_vaddr; - -return ra; +return __xchg (®s->ra, trampoline_vaddr); } int arch_uprobe_exception_notify(struct notifier_block *self,
arch/s390/kernel/uprobes.c +2 -5
@@ -11,6 +11,7 @@#include <linux/compat.h> #include <linux/kdebug.h> #include <linux/sched/task_stack.h> +#include <linux/non-atomic/xchg.h> #include <asm/switch_to.h> #include <asm/facility.h>
@@ -144,11 +145,7 @@ void arch_uprobe_abort_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline,struct pt_regs *regs) { -unsigned long orig; - - orig = regs->gprs[14 ]; - regs->gprs[14 ] = trampoline; -return orig; +return __xchg (®s->gprs[14 ], trampoline); } bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check ctx,
arch/sparc/kernel/uprobes.c +2 -5
@@ -11,6 +11,7 @@#include <linux/kernel.h> #include <linux/highmem.h> +#include <linux/non-atomic/xchg.h> #include <linux/uprobes.h> #include <linux/uaccess.h> #include <linux/sched.h> /* For struct task_struct */
@@ -310,9 +311,5 @@ unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { -unsigned long orig_ret_vaddr = regs->u_regs[UREG_I7 ]; - - regs->u_regs[UREG_I7 ] = trampoline_vaddr-8 ; - -return orig_ret_vaddr +8 ; +return __xchg (®s->u_regs[UREG_I7 ], trampoline_vaddr -8 ) +8 ; }
PATCH v5 4/7 llist: simplify __llist_del_all
In reply to Andrzej Hajdallist_del_all uses xchg, let's use __xchg here.
--- include/linux/llist.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
include/linux/llist.h +2 -4
@@ -50,6 +50,7 @@#include <linux/atomic.h> #include <linux/container_of.h> +#include <linux/non-atomic/xchg.h> #include <linux/stddef.h> #include <linux/types.h>
@@ -241,10 +242,7 @@ static inline struct llist_node *llist_del_all(struct llist_head *head)static inline struct llist_node *__llist_del_all (struct llist_head *head) { -struct llist_node *first = head->first; - - head->first =NULL ; -return first; +return __xchg (&head->first,NULL ); }extern struct llist_node *llist_del_first (struct llist_head *head);
Signature · 1 line
2.34.1
Re: [Intel-gfx] [PATCH v5 4/7] llist: simplify __llist_del_all
Hi Andrzej,
Quoted 3 lines · Andrzej Hajda
llist_del_alluses xchg, let's use__xchghere.Signed-off-by Andrzej Hajda <[email protected]>
Thanks,
Andi
[tip: locking/core] llist: simplify __llist_del_all
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: ce27b24cbf7f62b74c4cbf807a06f42a14ccf981 Gitweb: https://git.kernel.org/tip/ce27b24cbf7f62b74c4cbf807a06f42a14ccf981 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:47 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:03:59 +01:00
llist: simplify __llist_del_all
llist_del_all uses xchg, let's use __xchg here.
--- include/linux/llist.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
include/linux/llist.h +2 -4
@@ -50,6 +50,7 @@#include <linux/atomic.h> #include <linux/container_of.h> +#include <linux/non-atomic/xchg.h> #include <linux/stddef.h> #include <linux/types.h>
@@ -241,10 +242,7 @@ static inline struct llist_node *llist_del_all(struct llist_head *head)static inline struct llist_node *__llist_del_all (struct llist_head *head) { -struct llist_node *first = head->first; - - head->first =NULL ; -return first; +return __xchg (&head->first,NULL ); }extern struct llist_node *llist_del_first (struct llist_head *head);
PATCH v5 5/7 io_uring: use __xchg if possible
In reply to Andrzej HajdaRecently introduced helper simplifies the code.
--- io_uring/io_uring.c | 7 ++----- io_uring/slist.h | 6 ++---- 2 files changed, 4 insertions(+), 9 deletions(-)
io_uring/io_uring.c +2 -5
@@ -54,6 +54,7 @@#include <linux/fdtable.h> #include <linux/mm.h> #include <linux/mman.h> +#include <linux/non-atomic/xchg.h> #include <linux/percpu.h> #include <linux/slab.h> #include <linux/bvec.h>
@@ -1095,8 +1096,6 @@ static void __io_req_find_next_prep(struct io_kiocb *req)static inline struct io_kiocb *io_req_find_next (struct io_kiocb *req) { -struct io_kiocb *nxt; -/* * If LINK is set, we have dependent requests in this chain. If we * didn't fail this request, queue the first one up, moving any other
@@ -1105,9 +1104,7 @@ static inline struct io_kiocb *io_req_find_next(struct io_kiocb *req)*/ if (unlikely (req->flags &IO_DISARM_MASK ))__io_req_find_next_prep (req); - nxt = req->link; - req->link =NULL ; -return nxt; +return __xchg (&req->link,NULL ); }static void ctx_flush_and_put (struct io_ring_ctx *ctx,bool *locked)
io_uring/slist.h +2 -4
@@ -2,6 +2,7 @@#define INTERNAL_IO_SLIST_H #include <linux/io_uring_types.h> +#include <linux/non-atomic/xchg.h> #define wq_list_for_each (pos, prv, head) \ for (pos = (head)->first, prv = NULL; pos; prv = pos, pos = (pos)->next)
@@ -121,10 +122,7 @@ static inline void wq_list_del(struct io_wq_work_list *list,static inline struct io_wq_work_node *wq_stack_extract (struct io_wq_work_node *stack) { -struct io_wq_work_node *node = stack->next; - - stack->next = node->next; -return node; +return __xchg (&stack->next, stack->next->next); }static inline struct io_wq_work *wq_next_work (struct io_wq_work *work)
Signature · 1 line
2.34.1
Re: [Intel-gfx] [PATCH v5 5/7] io_uring: use __xchg if possible
Hi Andrzej,
Quoted 3 lines · Andrzej Hajda
Recently introduced helper simplifies the code.
Signed-off-by Andrzej Hajda <[email protected]>
Thanks,
Andi
[tip: locking/core] io_uring: use __xchg if possible
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: bd1e2670ebd10ad727b92ddd424cca270314f5ee Gitweb: https://git.kernel.org/tip/bd1e2670ebd10ad727b92ddd424cca270314f5ee Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:48 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:03:59 +01:00
io_uring: use __xchg if possible
Recently introduced helper simplifies the code.
--- io_uring/io_uring.c | 7 ++----- io_uring/slist.h | 6 ++---- 2 files changed, 4 insertions(+), 9 deletions(-)
io_uring/io_uring.c +2 -5
@@ -54,6 +54,7 @@#include <linux/fdtable.h> #include <linux/mm.h> #include <linux/mman.h> +#include <linux/non-atomic/xchg.h> #include <linux/percpu.h> #include <linux/slab.h> #include <linux/bvec.h>
@@ -1131,8 +1132,6 @@ static void __io_req_find_next_prep(struct io_kiocb *req)static inline struct io_kiocb *io_req_find_next (struct io_kiocb *req) { -struct io_kiocb *nxt; -/* * If LINK is set, we have dependent requests in this chain. If we * didn't fail this request, queue the first one up, moving any other
@@ -1141,9 +1140,7 @@ static inline struct io_kiocb *io_req_find_next(struct io_kiocb *req)*/ if (unlikely (req->flags &IO_DISARM_MASK ))__io_req_find_next_prep (req); - nxt = req->link; - req->link =NULL ; -return nxt; +return __xchg (&req->link,NULL ); }static void ctx_flush_and_put (struct io_ring_ctx *ctx,bool *locked)
io_uring/slist.h +2 -4
@@ -2,6 +2,7 @@#define INTERNAL_IO_SLIST_H #include <linux/io_uring_types.h> +#include <linux/non-atomic/xchg.h> #define wq_list_for_each (pos, prv, head) \ for (pos = (head)->first, prv = NULL; pos; prv = pos, pos = (pos)->next)
@@ -99,10 +100,7 @@ static inline void wq_list_del(struct io_wq_work_list *list,static inline struct io_wq_work_node *wq_stack_extract (struct io_wq_work_node *stack) { -struct io_wq_work_node *node = stack->next; - - stack->next = node->next; -return node; +return __xchg (&stack->next, stack->next->next); }static inline struct io_wq_work *wq_next_work (struct io_wq_work *work)
PATCH v5 6/7 qed: use __xchg if possible
In reply to Andrzej HajdaRecently introduced helper simplifies the code.
--- include/linux/qed/qed_chain.h | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-)
include/linux/qed/qed_chain.h +7 -12
@@ -11,6 +11,7 @@#include <asm/byteorder.h> #include <linux/kernel.h> #include <linux/list.h> +#include <linux/non-atomic/xchg.h> #include <linux/sizes.h> #include <linux/slab.h> #include <linux/qed/common_hsi.h>
@@ -368,7 +369,7 @@ static inline void qed_chain_return_produced(struct qed_chain *p_chain)*/ static inline void *qed_chain_produce (struct qed_chain *p_chain) { -void *p_ret =NULL , *p_prod_idx, *p_prod_page_idx; +void *p_prod_idx, *p_prod_page_idx;if (is_chain_u16 (p_chain)) { if ((p_chain->u.chain16.prod_idx &
@@ -390,11 +391,8 @@ static inline void *qed_chain_produce(struct qed_chain *p_chain) p_chain->u.chain32.prod_idx++; } - p_ret = p_chain->p_prod_elem; - p_chain->p_prod_elem = (void *)(((u8 *)p_chain->p_prod_elem) + - p_chain->elem_size); - -return p_ret; +return __xchg (&p_chain->p_prod_elem, + (u8 *)p_chain->p_prod_elem + p_chain->elem_size); }/**
@@ -439,7 +437,7 @@ static inline void qed_chain_recycle_consumed(struct qed_chain *p_chain)*/ static inline void *qed_chain_consume (struct qed_chain *p_chain) { -void *p_ret =NULL , *p_cons_idx, *p_cons_page_idx; +void *p_cons_idx, *p_cons_page_idx;if (is_chain_u16 (p_chain)) { if ((p_chain->u.chain16.cons_idx &
@@ -461,11 +459,8 @@ static inline void *qed_chain_consume(struct qed_chain *p_chain) p_chain->u.chain32.cons_idx++; } - p_ret = p_chain->p_cons_elem; - p_chain->p_cons_elem = (void *)(((u8 *)p_chain->p_cons_elem) + - p_chain->elem_size); - -return p_ret; +return __xchg (&p_chain->p_cons_elem, + (u8 *)p_chain->p_cons_elem + p_chain->elem_size); }/**
Signature · 1 line
2.34.1
Re: [Intel-gfx] [PATCH v5 6/7] qed: use __xchg if possible
Hi Andrzej,
Quoted 3 lines · Andrzej Hajda
Recently introduced helper simplifies the code.
Signed-off-by Andrzej Hajda <[email protected]>
Thanks,
Andi
[tip: locking/core] qed: use __xchg if possible
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 9b579a841f9032a231e9b14a23dc6699cbe6d311 Gitweb: https://git.kernel.org/tip/9b579a841f9032a231e9b14a23dc6699cbe6d311 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:49 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:03:59 +01:00
qed: use __xchg if possible
Recently introduced helper simplifies the code.
--- include/linux/qed/qed_chain.h | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-)
include/linux/qed/qed_chain.h +7 -12
@@ -11,6 +11,7 @@#include <asm/byteorder.h> #include <linux/kernel.h> #include <linux/list.h> +#include <linux/non-atomic/xchg.h> #include <linux/sizes.h> #include <linux/slab.h> #include <linux/qed/common_hsi.h>
@@ -368,7 +369,7 @@ static inline void qed_chain_return_produced(struct qed_chain *p_chain)*/ static inline void *qed_chain_produce (struct qed_chain *p_chain) { -void *p_ret =NULL , *p_prod_idx, *p_prod_page_idx; +void *p_prod_idx, *p_prod_page_idx;if (is_chain_u16 (p_chain)) { if ((p_chain->u.chain16.prod_idx &
@@ -390,11 +391,8 @@ static inline void *qed_chain_produce(struct qed_chain *p_chain) p_chain->u.chain32.prod_idx++; } - p_ret = p_chain->p_prod_elem; - p_chain->p_prod_elem = (void *)(((u8 *)p_chain->p_prod_elem) + - p_chain->elem_size); - -return p_ret; +return __xchg (&p_chain->p_prod_elem, + (u8 *)p_chain->p_prod_elem + p_chain->elem_size); }/**
@@ -439,7 +437,7 @@ static inline void qed_chain_recycle_consumed(struct qed_chain *p_chain)*/ static inline void *qed_chain_consume (struct qed_chain *p_chain) { -void *p_ret =NULL , *p_cons_idx, *p_cons_page_idx; +void *p_cons_idx, *p_cons_page_idx;if (is_chain_u16 (p_chain)) { if ((p_chain->u.chain16.cons_idx &
@@ -461,11 +459,8 @@ static inline void *qed_chain_consume(struct qed_chain *p_chain) p_chain->u.chain32.cons_idx++; } - p_ret = p_chain->p_cons_elem; - p_chain->p_cons_elem = (void *)(((u8 *)p_chain->p_cons_elem) + - p_chain->elem_size); - -return p_ret; +return __xchg (&p_chain->p_cons_elem, + (u8 *)p_chain->p_cons_elem + p_chain->elem_size); }/**
PATCH v5 7/7 drm/i915/gt: use __xchg instead of internal helper
In reply to Andrzej HajdaPrefer core helper if available.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +- drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gsc.c | 2 +- drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gt_pm.c | 2 +- drivers/gpu/drm/i915/gt/intel_lrc.c | 6 +++--- drivers/gpu/drm/i915/gt/intel_migrate.c | 2 +- drivers/gpu/drm/i915/gt/intel_rc6.c | 2 +- drivers/gpu/drm/i915/gt/intel_rps.c | 2 +- drivers/gpu/drm/i915/gt/selftest_context.c | 2 +- drivers/gpu/drm/i915/gt/selftest_ring_submission.c | 2 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 2 +- drivers/gpu/drm/i915/i915_utils.h | 1 + 18 files changed, 24 insertions(+), 23 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +1 -1
@@ -1042,7 +1042,7 @@ static void cleanup_status_page(struct intel_engine_cs *engine)/* Prevent writes into HWSP after returning the page to the system */ intel_engine_set_hwsp_writemask (engine, ~0u ); - vma =fetch_and_zero (&engine->status_page.vma); + vma =__xchg (&engine->status_page.vma,0 );if (!vma)return ;
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c +2 -2
@@ -229,7 +229,7 @@ static void heartbeat(struct work_struct *wrk)mutex_unlock (&ce->timeline->mutex); out:if (!engine->i915->params.enable_hangcheck || !next_heartbeat (engine)) -i915_request_put (fetch_and_zero (&engine->heartbeat.systole)); +i915_request_put (__xchg (&engine->heartbeat.systole,0 ));intel_engine_pm_put (engine); }
@@ -244,7 +244,7 @@ void intel_engine_unpark_heartbeat(struct intel_engine_cs *engine)void intel_engine_park_heartbeat (struct intel_engine_cs *engine) {if (cancel_delayed_work (&engine->heartbeat.work)) -i915_request_put (fetch_and_zero (&engine->heartbeat.systole)); +i915_request_put (__xchg (&engine->heartbeat.systole,0 )); }void intel_gt_unpark_heartbeats (struct intel_gt *gt)
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +2 -2
@@ -3199,7 +3199,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)RB_CLEAR_NODE (rb);spin_lock (&ve->base.sched_engine->lock); - rq =fetch_and_zero (&ve->request); + rq =__xchg (&ve->request,NULL );if (rq) {if (i915_request_mark_eio (rq)) { rq->engine = engine;
@@ -3604,7 +3604,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)spin_lock_irq (&ve->base.sched_engine->lock); - old =fetch_and_zero (&ve->request); + old =__xchg (&ve->request,NULL );if (old) {GEM_BUG_ON (!__i915_request_is_complete (old));__i915_request_submit (old);
drivers/gpu/drm/i915/gt/intel_ggtt.c +2 -2
@@ -684,7 +684,7 @@ static void fini_aliasing_ppgtt(struct i915_ggtt *ggtt) {struct i915_ppgtt *ppgtt; - ppgtt =fetch_and_zero (&ggtt->alias); + ppgtt =__xchg (&ggtt->alias,NULL );if (!ppgtt)return ;
@@ -1238,7 +1238,7 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm) was_bound);if (obj) {/* only used during resume => exclusive access */ - write_domain_objs |=fetch_and_zero (&obj->write_domain); + write_domain_objs |=__xchg (&obj->write_domain,0 ); obj->read_domains |=I915_GEM_DOMAIN_GTT ; } }
drivers/gpu/drm/i915/gt/intel_gsc.c +1 -1
@@ -70,7 +70,7 @@ gsc_ext_om_alloc(struct intel_gsc *gsc, struct intel_gsc_intf *intf, size_t sizestatic void gsc_ext_om_destroy (struct intel_gsc_intf *intf) { -struct drm_i915_gem_object *obj =fetch_and_zero (&intf->gem_obj); +struct drm_i915_gem_object *obj =__xchg (&intf->gem_obj,0 );if (!obj)return ;
drivers/gpu/drm/i915/gt/intel_gt.c +2 -2
@@ -753,7 +753,7 @@ int intel_gt_init(struct intel_gt *gt)intel_uc_fini (>->uc); err_engines:intel_engines_release (gt); -i915_vm_put (fetch_and_zero (>->vm)); +i915_vm_put (__xchg (>->vm,0 )); err_pm:intel_gt_pm_fini (gt);intel_gt_fini_scratch (gt);
@@ -800,7 +800,7 @@ void intel_gt_driver_release(struct intel_gt *gt) {struct i915_address_space *vm; - vm =fetch_and_zero (>->vm); + vm =__xchg (>->vm,0 );if (vm)/* FIXME being called twice on error paths :( */ i915_vm_put (vm);
drivers/gpu/drm/i915/gt/intel_gt_pm.c +1 -1
@@ -124,7 +124,7 @@ static int __gt_unpark(struct intel_wakeref *wf)static int __gt_park (struct intel_wakeref *wf) {struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref); -intel_wakeref_t wakeref =fetch_and_zero (>->awake); +intel_wakeref_t wakeref =__xchg (>->awake,0 );struct drm_i915_private *i915 = gt->i915;GT_TRACE (gt,"\n" );
drivers/gpu/drm/i915/gt/intel_lrc.c +3 -3
@@ -1144,7 +1144,7 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)static struct intel_timeline *pinned_timeline (struct intel_context *ce,struct intel_engine_cs *engine) { -struct intel_timeline *tl =fetch_and_zero (&ce->timeline); +struct intel_timeline *tl =__xchg (&ce->timeline,0 );return intel_timeline_create_from_engine (engine,page_unmask_bits (tl)); }
@@ -1261,8 +1261,8 @@ void lrc_fini(struct intel_context *ce)if (!ce->state)return ; -intel_ring_put (fetch_and_zero (&ce->ring)); -i915_vma_put (fetch_and_zero (&ce->state)); +intel_ring_put (__xchg (&ce->ring,0 )); +i915_vma_put (__xchg (&ce->state,0 )); }void lrc_destroy (struct kref *kref)
drivers/gpu/drm/i915/gt/intel_migrate.c +1 -1
@@ -1147,7 +1147,7 @@ void intel_migrate_fini(struct intel_migrate *m) {struct intel_context *ce; - ce =fetch_and_zero (&m->context); + ce =__xchg (&m->context,0 );if (!ce)return ;
drivers/gpu/drm/i915/gt/intel_rc6.c +1 -1
@@ -702,7 +702,7 @@ void intel_rc6_fini(struct intel_rc6 *rc6)intel_rc6_disable (rc6); - pctx =fetch_and_zero (&rc6->pctx); + pctx =__xchg (&rc6->pctx,0 );if (pctx)i915_gem_object_put (pctx);
drivers/gpu/drm/i915/gt/intel_rps.c +1 -1
@@ -1831,7 +1831,7 @@ static void rps_work(struct work_struct *work)u32 pm_iir =0 ;spin_lock_irq (gt->irq_lock); - pm_iir =fetch_and_zero (&rps->pm_iir) & rps->pm_events; + pm_iir =__xchg (&rps->pm_iir,0 ) & rps->pm_events; client_boost =atomic_read (&rps->num_waiters);spin_unlock_irq (gt->irq_lock);
drivers/gpu/drm/i915/gt/selftest_context.c +1 -1
@@ -171,7 +171,7 @@ static int live_context_size(void *arg)* active state is sufficient, we are only checking that we * don't use more than we planned. */ - saved =fetch_and_zero (&engine->default_state); + saved =__xchg (&engine->default_state,0 );/* Overlaps with the execlists redzone */ engine->context_size +=I915_GTT_PAGE_SIZE ;
drivers/gpu/drm/i915/gt/selftest_ring_submission.c +1 -1
@@ -269,7 +269,7 @@ static int live_ctx_switch_wa(void *arg)if (IS_GRAPHICS_VER (gt->i915,4 ,5 ))continue ;/* MI_STORE_DWORD is privileged! */ - saved_wa =fetch_and_zero (&engine->wa_ctx.vma); + saved_wa =__xchg (&engine->wa_ctx.vma,0 );intel_engine_pm_get (engine); err =__live_ctx_switch_wa (engine);
drivers/gpu/drm/i915/gt/selftest_timeline.c +1 -1
@@ -892,7 +892,7 @@ static int create_watcher(struct hwsp_watcher *w,static int check_watcher (struct hwsp_watcher *w,const char *name,bool (*op)(u32 hwsp,u32 seqno)) { -struct i915_request *rq =fetch_and_zero (&w->rq); +struct i915_request *rq =__xchg (&w->rq,NULL );u32 offset, end;int err;
drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.c +1 -1
@@ -110,7 +110,7 @@ void intel_gsc_uc_fini(struct intel_gsc_uc *gsc)flush_work (&gsc->work);if (gsc->ce) -intel_engine_destroy_pinned_context (fetch_and_zero (&gsc->ce)); +intel_engine_destroy_pinned_context (__xchg (&gsc->ce,NULL ));i915_vma_unpin_and_release (&gsc->local,0 );
drivers/gpu/drm/i915/gt/uc/intel_uc.c +1 -1
@@ -169,7 +169,7 @@ static void __uc_capture_load_err_log(struct intel_uc *uc)static void __uc_free_load_err_log (struct intel_uc *uc) { -struct drm_i915_gem_object *log =fetch_and_zero (&uc->load_err_log); +struct drm_i915_gem_object *log =__xchg (&uc->load_err_log,NULL );if (log)i915_gem_object_put (log);
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +1 -1
@@ -1119,7 +1119,7 @@ void intel_uc_fw_cleanup_fetch(struct intel_uc_fw *uc_fw)if (!intel_uc_fw_is_available (uc_fw))return ; -i915_gem_object_put (fetch_and_zero (&uc_fw->obj)); +i915_gem_object_put (__xchg (&uc_fw->obj,NULL ));intel_uc_fw_change_status (uc_fw,INTEL_UC_FIRMWARE_SELECTED ); }
drivers/gpu/drm/i915/i915_utils.h +1 -0
@@ -26,6 +26,7 @@#define __I915_UTILS_H#include <linux/list.h> +#include <linux/non-atomic/xchg.h> #include <linux/overflow.h> #include <linux/sched.h> #include <linux/string_helpers.h>
Signature · 1 line
2.34.1
Re: [Intel-gfx] [PATCH v5 7/7] drm/i915/gt: use __xchg instead of internal helper
Hi Andrzej,
Quoted 3 lines · Andrzej Hajda
Prefer core helper if available.
Signed-off-by Andrzej Hajda <[email protected]>
Thanks,
Andi
[tip: locking/core] drm/i915/gt: use __xchg instead of internal helper
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 63802adef67c6ea9a7fe4ce19328cc6e37f0abd7 Gitweb: https://git.kernel.org/tip/63802adef67c6ea9a7fe4ce19328cc6e37f0abd7 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:50 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:04:00 +01:00
drm/i915/gt: use __xchg instead of internal helper
Prefer core helper if available.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +- drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gsc.c | 2 +- drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gt_pm.c | 2 +- drivers/gpu/drm/i915/gt/intel_lrc.c | 6 +++--- drivers/gpu/drm/i915/gt/intel_migrate.c | 2 +- drivers/gpu/drm/i915/gt/intel_rc6.c | 2 +- drivers/gpu/drm/i915/gt/intel_rps.c | 2 +- drivers/gpu/drm/i915/gt/selftest_context.c | 2 +- drivers/gpu/drm/i915/gt/selftest_ring_submission.c | 2 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 2 +- drivers/gpu/drm/i915/i915_utils.h | 1 + 17 files changed, 23 insertions(+), 22 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +1 -1
@@ -1042,7 +1042,7 @@ static void cleanup_status_page(struct intel_engine_cs *engine)/* Prevent writes into HWSP after returning the page to the system */ intel_engine_set_hwsp_writemask (engine, ~0u ); - vma =fetch_and_zero (&engine->status_page.vma); + vma =__xchg (&engine->status_page.vma,0 );if (!vma)return ;
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c +2 -2
@@ -229,7 +229,7 @@ unlock:mutex_unlock (&ce->timeline->mutex); out:if (!engine->i915->params.enable_hangcheck || !next_heartbeat (engine)) -i915_request_put (fetch_and_zero (&engine->heartbeat.systole)); +i915_request_put (__xchg (&engine->heartbeat.systole,0 ));intel_engine_pm_put (engine); }
@@ -244,7 +244,7 @@ void intel_engine_unpark_heartbeat(struct intel_engine_cs *engine)void intel_engine_park_heartbeat (struct intel_engine_cs *engine) {if (cancel_delayed_work (&engine->heartbeat.work)) -i915_request_put (fetch_and_zero (&engine->heartbeat.systole)); +i915_request_put (__xchg (&engine->heartbeat.systole,0 )); }void intel_gt_unpark_heartbeats (struct intel_gt *gt)
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +2 -2
@@ -3199,7 +3199,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)RB_CLEAR_NODE (rb);spin_lock (&ve->base.sched_engine->lock); - rq =fetch_and_zero (&ve->request); + rq =__xchg (&ve->request,NULL );if (rq) {if (i915_request_mark_eio (rq)) { rq->engine = engine;
@@ -3604,7 +3604,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)spin_lock_irq (&ve->base.sched_engine->lock); - old =fetch_and_zero (&ve->request); + old =__xchg (&ve->request,NULL );if (old) {GEM_BUG_ON (!__i915_request_is_complete (old));__i915_request_submit (old);
drivers/gpu/drm/i915/gt/intel_ggtt.c +2 -2
@@ -685,7 +685,7 @@ static void fini_aliasing_ppgtt(struct i915_ggtt *ggtt) {struct i915_ppgtt *ppgtt; - ppgtt =fetch_and_zero (&ggtt->alias); + ppgtt =__xchg (&ggtt->alias,NULL );if (!ppgtt)return ;
@@ -1239,7 +1239,7 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm) was_bound);if (obj) {/* only used during resume => exclusive access */ - write_domain_objs |=fetch_and_zero (&obj->write_domain); + write_domain_objs |=__xchg (&obj->write_domain,0 ); obj->read_domains |=I915_GEM_DOMAIN_GTT ; } }
drivers/gpu/drm/i915/gt/intel_gsc.c +1 -1
@@ -70,7 +70,7 @@ out_put:static void gsc_ext_om_destroy (struct intel_gsc_intf *intf) { -struct drm_i915_gem_object *obj =fetch_and_zero (&intf->gem_obj); +struct drm_i915_gem_object *obj =__xchg (&intf->gem_obj,0 );if (!obj)return ;
drivers/gpu/drm/i915/gt/intel_gt.c +2 -2
@@ -753,7 +753,7 @@ err_uc_init:intel_uc_fini (>->uc); err_engines:intel_engines_release (gt); -i915_vm_put (fetch_and_zero (>->vm)); +i915_vm_put (__xchg (>->vm,0 )); err_pm:intel_gt_pm_fini (gt);intel_gt_fini_scratch (gt);
@@ -800,7 +800,7 @@ void intel_gt_driver_release(struct intel_gt *gt) {struct i915_address_space *vm; - vm =fetch_and_zero (>->vm); + vm =__xchg (>->vm,0 );if (vm)/* FIXME being called twice on error paths :( */ i915_vm_put (vm);
drivers/gpu/drm/i915/gt/intel_gt_pm.c +1 -1
@@ -124,7 +124,7 @@ static int __gt_unpark(struct intel_wakeref *wf)static int __gt_park (struct intel_wakeref *wf) {struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref); -intel_wakeref_t wakeref =fetch_and_zero (>->awake); +intel_wakeref_t wakeref =__xchg (>->awake,0 );struct drm_i915_private *i915 = gt->i915;GT_TRACE (gt,"\n" );
drivers/gpu/drm/i915/gt/intel_lrc.c +3 -3
@@ -1109,7 +1109,7 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)static struct intel_timeline *pinned_timeline (struct intel_context *ce,struct intel_engine_cs *engine) { -struct intel_timeline *tl =fetch_and_zero (&ce->timeline); +struct intel_timeline *tl =__xchg (&ce->timeline,0 );return intel_timeline_create_from_engine (engine,page_unmask_bits (tl)); }
@@ -1226,8 +1226,8 @@ void lrc_fini(struct intel_context *ce)if (!ce->state)return ; -intel_ring_put (fetch_and_zero (&ce->ring)); -i915_vma_put (fetch_and_zero (&ce->state)); +intel_ring_put (__xchg (&ce->ring,0 )); +i915_vma_put (__xchg (&ce->state,0 )); }void lrc_destroy (struct kref *kref)
drivers/gpu/drm/i915/gt/intel_migrate.c +1 -1
@@ -1147,7 +1147,7 @@ void intel_migrate_fini(struct intel_migrate *m) {struct intel_context *ce; - ce =fetch_and_zero (&m->context); + ce =__xchg (&m->context,0 );if (!ce)return ;
drivers/gpu/drm/i915/gt/intel_rc6.c +1 -1
@@ -702,7 +702,7 @@ void intel_rc6_fini(struct intel_rc6 *rc6)intel_rc6_disable (rc6); - pctx =fetch_and_zero (&rc6->pctx); + pctx =__xchg (&rc6->pctx,0 );if (pctx)i915_gem_object_put (pctx);
drivers/gpu/drm/i915/gt/intel_rps.c +1 -1
@@ -1832,7 +1832,7 @@ static void rps_work(struct work_struct *work)u32 pm_iir =0 ;spin_lock_irq (gt->irq_lock); - pm_iir =fetch_and_zero (&rps->pm_iir) & rps->pm_events; + pm_iir =__xchg (&rps->pm_iir,0 ) & rps->pm_events; client_boost =atomic_read (&rps->num_waiters);spin_unlock_irq (gt->irq_lock);
drivers/gpu/drm/i915/gt/selftest_context.c +1 -1
@@ -171,7 +171,7 @@ static int live_context_size(void *arg)* active state is sufficient, we are only checking that we * don't use more than we planned. */ - saved =fetch_and_zero (&engine->default_state); + saved =__xchg (&engine->default_state,0 );/* Overlaps with the execlists redzone */ engine->context_size +=I915_GTT_PAGE_SIZE ;
drivers/gpu/drm/i915/gt/selftest_ring_submission.c +1 -1
@@ -269,7 +269,7 @@ static int live_ctx_switch_wa(void *arg)if (IS_GRAPHICS_VER (gt->i915,4 ,5 ))continue ;/* MI_STORE_DWORD is privileged! */ - saved_wa =fetch_and_zero (&engine->wa_ctx.vma); + saved_wa =__xchg (&engine->wa_ctx.vma,0 );intel_engine_pm_get (engine); err =__live_ctx_switch_wa (engine);
drivers/gpu/drm/i915/gt/selftest_timeline.c +1 -1
@@ -892,7 +892,7 @@ static int create_watcher(struct hwsp_watcher *w,static int check_watcher (struct hwsp_watcher *w,const char *name,bool (*op)(u32 hwsp,u32 seqno)) { -struct i915_request *rq =fetch_and_zero (&w->rq); +struct i915_request *rq =__xchg (&w->rq,NULL );u32 offset, end;int err;
drivers/gpu/drm/i915/gt/uc/intel_uc.c +1 -1
@@ -171,7 +171,7 @@ static void __uc_capture_load_err_log(struct intel_uc *uc)static void __uc_free_load_err_log (struct intel_uc *uc) { -struct drm_i915_gem_object *log =fetch_and_zero (&uc->load_err_log); +struct drm_i915_gem_object *log =__xchg (&uc->load_err_log,NULL );if (log)i915_gem_object_put (log);
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +1 -1
@@ -1119,7 +1119,7 @@ void intel_uc_fw_cleanup_fetch(struct intel_uc_fw *uc_fw)if (!intel_uc_fw_is_available (uc_fw))return ; -i915_gem_object_put (fetch_and_zero (&uc_fw->obj)); +i915_gem_object_put (__xchg (&uc_fw->obj,NULL ));intel_uc_fw_change_status (uc_fw,INTEL_UC_FIRMWARE_SELECTED ); }
drivers/gpu/drm/i915/i915_utils.h +1 -0
@@ -26,6 +26,7 @@#define __I915_UTILS_H#include <linux/list.h> +#include <linux/non-atomic/xchg.h> #include <linux/overflow.h> #include <linux/sched.h> #include <linux/string_helpers.h>
[tip: locking/core] drm/i915/gt: use __xchg instead of internal helper
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: f7faedffa92c35afbcfbf0a3dd73079e16b2b87f Gitweb: https://git.kernel.org/tip/f7faedffa92c35afbcfbf0a3dd73079e16b2b87f Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:50 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 14 Mar 2023 10:53:57 +01:00
drm/i915/gt: use __xchg instead of internal helper
Prefer core helper if available.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +- drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gsc.c | 2 +- drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gt_pm.c | 2 +- drivers/gpu/drm/i915/gt/intel_lrc.c | 6 +++--- drivers/gpu/drm/i915/gt/intel_migrate.c | 2 +- drivers/gpu/drm/i915/gt/intel_rc6.c | 2 +- drivers/gpu/drm/i915/gt/intel_rps.c | 2 +- drivers/gpu/drm/i915/gt/selftest_context.c | 2 +- drivers/gpu/drm/i915/gt/selftest_ring_submission.c | 2 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 2 +- drivers/gpu/drm/i915/i915_utils.h | 1 + 17 files changed, 23 insertions(+), 22 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +1 -1
@@ -1042,7 +1042,7 @@ static void cleanup_status_page(struct intel_engine_cs *engine)/* Prevent writes into HWSP after returning the page to the system */ intel_engine_set_hwsp_writemask (engine, ~0u ); - vma =fetch_and_zero (&engine->status_page.vma); + vma =__xchg (&engine->status_page.vma,0 );if (!vma)return ;
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c +2 -2
@@ -229,7 +229,7 @@ unlock:mutex_unlock (&ce->timeline->mutex); out:if (!engine->i915->params.enable_hangcheck || !next_heartbeat (engine)) -i915_request_put (fetch_and_zero (&engine->heartbeat.systole)); +i915_request_put (__xchg (&engine->heartbeat.systole,0 ));intel_engine_pm_put (engine); }
@@ -244,7 +244,7 @@ void intel_engine_unpark_heartbeat(struct intel_engine_cs *engine)void intel_engine_park_heartbeat (struct intel_engine_cs *engine) {if (cancel_delayed_work (&engine->heartbeat.work)) -i915_request_put (fetch_and_zero (&engine->heartbeat.systole)); +i915_request_put (__xchg (&engine->heartbeat.systole,0 )); }void intel_gt_unpark_heartbeats (struct intel_gt *gt)
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +2 -2
@@ -3199,7 +3199,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)RB_CLEAR_NODE (rb);spin_lock (&ve->base.sched_engine->lock); - rq =fetch_and_zero (&ve->request); + rq =__xchg (&ve->request,NULL );if (rq) {if (i915_request_mark_eio (rq)) { rq->engine = engine;
@@ -3604,7 +3604,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)spin_lock_irq (&ve->base.sched_engine->lock); - old =fetch_and_zero (&ve->request); + old =__xchg (&ve->request,NULL );if (old) {GEM_BUG_ON (!__i915_request_is_complete (old));__i915_request_submit (old);
drivers/gpu/drm/i915/gt/intel_ggtt.c +2 -2
@@ -685,7 +685,7 @@ static void fini_aliasing_ppgtt(struct i915_ggtt *ggtt) {struct i915_ppgtt *ppgtt; - ppgtt =fetch_and_zero (&ggtt->alias); + ppgtt =__xchg (&ggtt->alias,NULL );if (!ppgtt)return ;
@@ -1239,7 +1239,7 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm) was_bound);if (obj) {/* only used during resume => exclusive access */ - write_domain_objs |=fetch_and_zero (&obj->write_domain); + write_domain_objs |=__xchg (&obj->write_domain,0 ); obj->read_domains |=I915_GEM_DOMAIN_GTT ; } }
drivers/gpu/drm/i915/gt/intel_gsc.c +1 -1
@@ -70,7 +70,7 @@ out_put:static void gsc_ext_om_destroy (struct intel_gsc_intf *intf) { -struct drm_i915_gem_object *obj =fetch_and_zero (&intf->gem_obj); +struct drm_i915_gem_object *obj =__xchg (&intf->gem_obj,0 );if (!obj)return ;
drivers/gpu/drm/i915/gt/intel_gt.c +2 -2
@@ -753,7 +753,7 @@ err_uc_init:intel_uc_fini (>->uc); err_engines:intel_engines_release (gt); -i915_vm_put (fetch_and_zero (>->vm)); +i915_vm_put (__xchg (>->vm,0 )); err_pm:intel_gt_pm_fini (gt);intel_gt_fini_scratch (gt);
@@ -800,7 +800,7 @@ void intel_gt_driver_release(struct intel_gt *gt) {struct i915_address_space *vm; - vm =fetch_and_zero (>->vm); + vm =__xchg (>->vm,0 );if (vm)/* FIXME being called twice on error paths :( */ i915_vm_put (vm);
drivers/gpu/drm/i915/gt/intel_gt_pm.c +1 -1
@@ -124,7 +124,7 @@ static int __gt_unpark(struct intel_wakeref *wf)static int __gt_park (struct intel_wakeref *wf) {struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref); -intel_wakeref_t wakeref =fetch_and_zero (>->awake); +intel_wakeref_t wakeref =__xchg (>->awake,0 );struct drm_i915_private *i915 = gt->i915;GT_TRACE (gt,"\n" );
drivers/gpu/drm/i915/gt/intel_lrc.c +3 -3
@@ -1109,7 +1109,7 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)static struct intel_timeline *pinned_timeline (struct intel_context *ce,struct intel_engine_cs *engine) { -struct intel_timeline *tl =fetch_and_zero (&ce->timeline); +struct intel_timeline *tl =__xchg (&ce->timeline,0 );return intel_timeline_create_from_engine (engine,page_unmask_bits (tl)); }
@@ -1226,8 +1226,8 @@ void lrc_fini(struct intel_context *ce)if (!ce->state)return ; -intel_ring_put (fetch_and_zero (&ce->ring)); -i915_vma_put (fetch_and_zero (&ce->state)); +intel_ring_put (__xchg (&ce->ring,0 )); +i915_vma_put (__xchg (&ce->state,0 )); }void lrc_destroy (struct kref *kref)
drivers/gpu/drm/i915/gt/intel_migrate.c +1 -1
@@ -1147,7 +1147,7 @@ void intel_migrate_fini(struct intel_migrate *m) {struct intel_context *ce; - ce =fetch_and_zero (&m->context); + ce =__xchg (&m->context,0 );if (!ce)return ;
drivers/gpu/drm/i915/gt/intel_rc6.c +1 -1
@@ -702,7 +702,7 @@ void intel_rc6_fini(struct intel_rc6 *rc6)intel_rc6_disable (rc6); - pctx =fetch_and_zero (&rc6->pctx); + pctx =__xchg (&rc6->pctx,0 );if (pctx)i915_gem_object_put (pctx);
drivers/gpu/drm/i915/gt/intel_rps.c +1 -1
@@ -1832,7 +1832,7 @@ static void rps_work(struct work_struct *work)u32 pm_iir =0 ;spin_lock_irq (gt->irq_lock); - pm_iir =fetch_and_zero (&rps->pm_iir) & rps->pm_events; + pm_iir =__xchg (&rps->pm_iir,0 ) & rps->pm_events; client_boost =atomic_read (&rps->num_waiters);spin_unlock_irq (gt->irq_lock);
drivers/gpu/drm/i915/gt/selftest_context.c +1 -1
@@ -171,7 +171,7 @@ static int live_context_size(void *arg)* active state is sufficient, we are only checking that we * don't use more than we planned. */ - saved =fetch_and_zero (&engine->default_state); + saved =__xchg (&engine->default_state,0 );/* Overlaps with the execlists redzone */ engine->context_size +=I915_GTT_PAGE_SIZE ;
drivers/gpu/drm/i915/gt/selftest_ring_submission.c +1 -1
@@ -269,7 +269,7 @@ static int live_ctx_switch_wa(void *arg)if (IS_GRAPHICS_VER (gt->i915,4 ,5 ))continue ;/* MI_STORE_DWORD is privileged! */ - saved_wa =fetch_and_zero (&engine->wa_ctx.vma); + saved_wa =__xchg (&engine->wa_ctx.vma,0 );intel_engine_pm_get (engine); err =__live_ctx_switch_wa (engine);
drivers/gpu/drm/i915/gt/selftest_timeline.c +1 -1
@@ -892,7 +892,7 @@ static int create_watcher(struct hwsp_watcher *w,static int check_watcher (struct hwsp_watcher *w,const char *name,bool (*op)(u32 hwsp,u32 seqno)) { -struct i915_request *rq =fetch_and_zero (&w->rq); +struct i915_request *rq =__xchg (&w->rq,NULL );u32 offset, end;int err;
drivers/gpu/drm/i915/gt/uc/intel_uc.c +1 -1
@@ -171,7 +171,7 @@ static void __uc_capture_load_err_log(struct intel_uc *uc)static void __uc_free_load_err_log (struct intel_uc *uc) { -struct drm_i915_gem_object *log =fetch_and_zero (&uc->load_err_log); +struct drm_i915_gem_object *log =__xchg (&uc->load_err_log,NULL );if (log)i915_gem_object_put (log);
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +1 -1
@@ -1119,7 +1119,7 @@ void intel_uc_fw_cleanup_fetch(struct intel_uc_fw *uc_fw)if (!intel_uc_fw_is_available (uc_fw))return ; -i915_gem_object_put (fetch_and_zero (&uc_fw->obj)); +i915_gem_object_put (__xchg (&uc_fw->obj,NULL ));intel_uc_fw_change_status (uc_fw,INTEL_UC_FIRMWARE_SELECTED ); }
drivers/gpu/drm/i915/i915_utils.h +1 -0
@@ -26,6 +26,7 @@#define __I915_UTILS_H#include <linux/list.h> +#include <linux/non-atomic/xchg.h> #include <linux/overflow.h> #include <linux/sched.h> #include <linux/string_helpers.h>
Re: [Intel-gfx] [PATCH v5 1/7] arch: rename all internal names __xchg to __arch_xchg
In reply to Andrzej HajdaHi Andrzej,
Quoted 6 lines · Andrzej Hajda
__xchgwill be used for non-atomic xchg macro.Signed-off-by Andrzej Hajda <[email protected]>Reviewed-by Arnd Bergmann <[email protected]>
Thanks,
Andi
[tip: locking/core] arch: rename all internal names __xchg to __arch_xchg
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: a8596dda1fbf7ec82d0e2ee86e44257ec654e7e5 Gitweb: https://git.kernel.org/tip/a8596dda1fbf7ec82d0e2ee86e44257ec654e7e5 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:44 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 11 Mar 2023 14:03:58 +01:00
arch: rename all internal names __xchg to __arch_xchg
__xchg will be used for non-atomic xchg macro.
--- arch/alpha/include/asm/cmpxchg.h | 10 +++++----- arch/arc/include/asm/cmpxchg.h | 4 ++-- arch/arm/include/asm/cmpxchg.h | 7 ++++--- arch/arm64/include/asm/cmpxchg.h | 7 +++---- arch/hexagon/include/asm/cmpxchg.h | 10 +++++----- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 ++-- arch/loongarch/include/asm/cmpxchg.h | 4 ++-- arch/m68k/include/asm/cmpxchg.h | 6 +++--- arch/mips/include/asm/cmpxchg.h | 4 ++-- arch/openrisc/include/asm/cmpxchg.h | 10 +++++----- arch/parisc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/include/asm/cmpxchg.h | 4 ++-- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 ++-- arch/s390/include/asm/cmpxchg.h | 8 ++++---- arch/sh/include/asm/cmpxchg.h | 4 ++-- arch/sparc/include/asm/cmpxchg_32.h | 4 ++-- arch/sparc/include/asm/cmpxchg_64.h | 6 +++--- arch/xtensa/include/asm/cmpxchg.h | 4 ++-- 20 files changed, 54 insertions(+), 54 deletions(-)
arch/alpha/include/asm/cmpxchg.h +5 -5
@@ -6,15 +6,15 @@* Atomic exchange routines. */ -#define ____xchg(type, args...) __xchg ## type ## _local(args) +#define ____xchg(type, args...) __arch_xchg ## type ## _local(args) #define ____cmpxchg(type, args...) __cmpxchg ## type ## _local(args)#include <asm/xchg.h> #define xchg_local (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_local((ptr), (unsigned long)_x_, \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg_local((ptr), (unsigned long)_x_,\ + sizeof(*(ptr))); \ })#define arch_cmpxchg_local (ptr, o, n) \
@@ -34,7 +34,7 @@#undef ____xchg#undef ____cmpxchg -#define ____xchg(type, args...) __xchg ##type(args) +#define ____xchg(type, args...) __arch_xchg ##type(args) #define ____cmpxchg(type, args...) __cmpxchg ##type(args)#include <asm/xchg.h>
@@ -48,7 +48,7 @@ __typeof__(*(ptr)) _x_ = (x); \smp_mb (); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \smp_mb (); \ __ret; \ })
arch/arc/include/asm/cmpxchg.h +2 -2
@@ -85,7 +85,7 @@*/ #ifdef CONFIG_ARC_HAS_LLSC -#define __xchg (ptr, val) \ +#define __arch_xchg (ptr, val) \ ({ \ __asm__ __volatile__( \ " ex %0, [%1] \n"/* set new value */ \
@@ -102,7 +102,7 @@ \switch (sizeof (*(_p_))) { \case 4 : \ - _val_ =__xchg (_p_, _val_); \ + _val_ =__arch_xchg (_p_, _val_); \break ; \default : \BUILD_BUG (); \
arch/arm/include/asm/cmpxchg.h +4 -3
@@ -25,7 +25,8 @@#define swp_is_buggy#endif -static inline unsigned long __xchg (unsigned long x,volatile void *ptr,int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {extern void __bad_xchg (volatile void *,int );unsigned long ret;
@@ -115,8 +116,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size }#define arch_xchg_relaxed (ptr, x) ({ \ - (__typeof__(*(ptr)))__xchg((unsigned long)(x), (ptr), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr)))__arch_xchg((unsigned long)(x), (ptr), \ + sizeof(*(ptr))); \ })#include <asm-generic/cmpxchg-local.h>
arch/arm64/include/asm/cmpxchg.h +3 -4
@@ -62,9 +62,8 @@ __XCHG_CASE( , , mb_, 64, dmb ish, nop, , a, l, "memory")#undef __XCHG_CASE#define __XCHG_GEN (sfx) \ -static __always_inline unsigned long __xchg##sfx(unsigned long x, \ - volatile void *ptr, \ - int size) \ +static __always_inline unsigned long \ +__arch_xchg##sfx(unsigned long x, volatile void *ptr, int size) \ { \ switch (size) { \ case 1: \
@@ -93,7 +92,7 @@ __XCHG_GEN(_mb) ({ \ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
arch/hexagon/include/asm/cmpxchg.h +5 -5
@@ -9,7 +9,7 @@#define _ASM_CMPXCHG_H/* -* __xchg - atomically exchange a register and a memory location +* __arch_xchg - atomically exchange a register and a memory location * @x: value to swap * @ptr: pointer to memory * @size: size of the value
@@ -19,8 +19,8 @@* Note: there was an errata for V2 about .new's and memw_locked. * */ -static inline unsigned long __xchg (unsigned long x,volatile void *ptr, -int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {unsigned long retval;
@@ -42,8 +42,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr,* Atomically swap the contents of a register with memory. Should be atomic * between multiple CPU's and within interrupts on the same CPU. */ -#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__xchg((unsigned long)(v), (ptr), \ - sizeof(*(ptr)))) +#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(v), (ptr), \ + sizeof(*(ptr))))/* * see rt-mutex-design.txt; cmpxchg supposedly checks if *ptr == A and swaps.
arch/ia64/include/asm/cmpxchg.h +1 -1
@@ -5,7 +5,7 @@#include <uapi/asm/cmpxchg.h> #define arch_xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#define arch_cmpxchg (ptr, o, n) cmpxchg_acq((ptr), (o), (n))#define arch_cmpxchg64 (ptr, o, n) cmpxchg_acq((ptr), (o), (n))
arch/ia64/include/uapi/asm/cmpxchg.h +2 -2
@@ -23,7 +23,7 @@*/ extern void ia64_xchg_called_with_bad_pointer (void ); -#define __xchg (x, ptr, size) \ +#define __arch_xchg (x, ptr, size) \ ({ \ unsigned long __xchg_result; \ \
@@ -51,7 +51,7 @@ extern void ia64_xchg_called_with_bad_pointer(void);#ifndef __KERNEL__#define xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#endif /*
arch/loongarch/include/asm/cmpxchg.h +2 -2
@@ -62,7 +62,7 @@ static inline unsigned int __xchg_small(volatile void *ptr, unsigned int val, }static __always_inline unsigned long -__xchg (volatile void *ptr,unsigned long x,int size) +__arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -87,7 +87,7 @@ __xchg(volatile void *ptr, unsigned long x, int size) __typeof__(*(ptr)) __res; \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \ __res; \ })
arch/m68k/include/asm/cmpxchg.h +3 -3
@@ -9,7 +9,7 @@extern unsigned long __invalid_xchg_size (unsigned long ,volatile void *,int );#ifndef CONFIG_RMW_INSNS -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {unsigned long flags, tmp;
@@ -40,7 +40,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz return x; }#else -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
@@ -75,7 +75,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz }#endif -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})#include <asm-generic/cmpxchg-local.h>
arch/mips/include/asm/cmpxchg.h +2 -2
@@ -68,7 +68,7 @@ extern unsigned long __xchg_small(volatile void *ptr, unsigned long val, unsigned int size);static __always_inline -unsigned long __xchg (volatile void *ptr,unsigned long x,int size) +unsigned long __arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -102,7 +102,7 @@ unsigned long __xchg(volatile void *ptr, unsigned long x, int size)smp_mb__before_llsc (); \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \smp_llsc_mb (); \ \
arch/openrisc/include/asm/cmpxchg.h +5 -5
@@ -147,8 +147,8 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, extern unsigned long __xchg_called_with_bad_pointer(void) __compiletime_error("Bad argument size for xchg"); -static inline unsigned long __xchg (volatile void *ptr,unsigned long with, -int size) +static inline unsigned long +__arch_xchg (volatile void *ptr,unsigned long with,int size) {switch (size) {case 1 :
@@ -163,9 +163,9 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long with,#define arch_xchg (ptr, with) \ ({ \ - (__typeof__(*(ptr))) __xchg((ptr), \ - (unsigned long)(with), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), \ + (unsigned long)(with), \ + sizeof(*(ptr))); \ })#endif /* __ASM_OPENRISC_CMPXCHG_H */
arch/parisc/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@ extern unsigned long __xchg64(unsigned long, volatile unsigned long *);/* optimizer better get rid of switch since size is a constant */ static inline unsigned long -__xchg (unsigned long x,volatile void *ptr,int size) +__arch_xchg (unsigned long x,volatile void *ptr,int size) {switch (size) {#ifdef CONFIG_64BIT
@@ -49,7 +49,7 @@ __xchg(unsigned long x, volatile void *ptr, int size) __typeof__(*(ptr)) __ret; \ __typeof__(*(ptr)) _x_ = (x); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ __ret; \ })
arch/powerpc/include/asm/cmpxchg.h +2 -2
@@ -229,7 +229,7 @@ __xchg_local(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_local (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" );return x; }
@@ -248,7 +248,7 @@ __xchg_relaxed(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_relaxed (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_relaxed" );return x; }#define arch_xchg_local (ptr,x) \
arch/riscv/include/asm/atomic.h +1 -1
@@ -261,7 +261,7 @@ c_t arch_atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \ static __always_inline \ c_t arch_atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ { \ - return __xchg(&(v->counter), n, size); \ + return __arch_xchg(&(v->counter), n, size); \ } \ static __always_inline \ c_t arch_atomic##prefix##_cmpxchg_relaxed(atomic##prefix##_t *v, \
arch/riscv/include/asm/cmpxchg.h +2 -2
@@ -114,7 +114,7 @@ _x_, sizeof(*(ptr))); \ }) -#define __xchg (ptr, new, size) \ +#define __arch_xchg (ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(new) __new = (new); \
@@ -143,7 +143,7 @@#define arch_xchg (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg((ptr), _x_, sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ })#define xchg32 (ptr, x) \
arch/s390/include/asm/cmpxchg.h +4 -4
@@ -14,8 +14,8 @@void __xchg_called_with_bad_pointer (void ); -static __always_inline unsigned long __xchg (unsigned long x, -unsigned long address,int size) +static __always_inline unsigned long +__arch_xchg (unsigned long x,unsigned long address,int size) {unsigned long old;int shift;
@@ -77,8 +77,8 @@ static __always_inline unsigned long __xchg(unsigned long x, __typeof__(*(ptr)) __ret; \ \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )(x), (unsigned long )(ptr), \ -sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )(x), (unsigned long )(ptr), \ +sizeof (*(ptr))); \ __ret; \ })
arch/sh/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@extern void __xchg_called_with_bad_pointer (void ); -#define __xchg (ptr, x, size) \ +#define __arch_xchg (ptr, x, size) \ ({ \ unsigned long __xchg__res; \ volatile void *__xchg_ptr = (ptr); \
@@ -46,7 +46,7 @@ extern void __xchg_called_with_bad_pointer(void); })#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((ptr),(unsigned long)(x), sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((ptr),(unsigned long)(x), sizeof(*(ptr))))/* This function doesn't exist, so you'll get a linker error * if something tries to do an invalid cmpxchg(). */
arch/sparc/include/asm/cmpxchg_32.h +2 -2
@@ -15,7 +15,7 @@unsigned long __xchg_u32 (volatile u32 *m,u32 new);void __xchg_called_with_bad_pointer (void ); -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, int size) +static inline unsigned long __arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 4 :
@@ -25,7 +25,7 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, intreturn x; } -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})/* Emulate cmpxchg() the same way we emulate atomics, * by hashing the object address and indexing into an array
arch/sparc/include/asm/cmpxchg_64.h +3 -3
@@ -55,7 +55,7 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long#define arch_xchg (ptr,x) \ ({ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
@@ -87,8 +87,8 @@ xchg16(__volatile__ unsigned short *m, unsigned short val)return (load32 & mask) >> bit_shift; } -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, -int size) +static inline unsigned long +__arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 2 :
arch/xtensa/include/asm/cmpxchg.h +2 -2
@@ -170,7 +170,7 @@ static inline unsigned long xchg_u32(volatile int * m, unsigned long val) }#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))static inline u32 xchg_small (volatile void *ptr,u32 x,int size) {
@@ -203,7 +203,7 @@ static inline u32 xchg_small(volatile void *ptr, u32 x, int size)extern void __xchg_called_with_bad_pointer (void );static __inline__unsigned long -__xchg (unsigned long x,volatile void * ptr,int size) +__arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
[tip: locking/core] locking/arch: Rename all internal __xchg() names to __arch_xchg()
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 08750b2a0b33b26d80cbb99c235326202a2ec75c Gitweb: https://git.kernel.org/tip/08750b2a0b33b26d80cbb99c235326202a2ec75c Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:44 +01:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Sat, 29 Apr 2023 08:51:01 +02:00
locking/arch: Rename all internal __xchg() names to __arch_xchg()
Decrease the probability of this internal facility to be used by driver code.
--- arch/alpha/include/asm/cmpxchg.h | 10 +++++----- arch/arc/include/asm/cmpxchg.h | 4 ++-- arch/arm/include/asm/cmpxchg.h | 7 ++++--- arch/arm64/include/asm/cmpxchg.h | 7 +++---- arch/hexagon/include/asm/cmpxchg.h | 10 +++++----- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 ++-- arch/loongarch/include/asm/cmpxchg.h | 4 ++-- arch/m68k/include/asm/cmpxchg.h | 6 +++--- arch/mips/include/asm/cmpxchg.h | 4 ++-- arch/openrisc/include/asm/cmpxchg.h | 10 +++++----- arch/parisc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/include/asm/cmpxchg.h | 4 ++-- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 ++-- arch/s390/include/asm/cmpxchg.h | 8 ++++---- arch/sh/include/asm/cmpxchg.h | 4 ++-- arch/sparc/include/asm/cmpxchg_32.h | 4 ++-- arch/sparc/include/asm/cmpxchg_64.h | 6 +++--- arch/xtensa/include/asm/cmpxchg.h | 4 ++-- 20 files changed, 54 insertions(+), 54 deletions(-)
arch/alpha/include/asm/cmpxchg.h +5 -5
@@ -6,15 +6,15 @@* Atomic exchange routines. */ -#define ____xchg(type, args...) __xchg ## type ## _local(args) +#define ____xchg(type, args...) __arch_xchg ## type ## _local(args) #define ____cmpxchg(type, args...) __cmpxchg ## type ## _local(args)#include <asm/xchg.h> #define xchg_local (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_local((ptr), (unsigned long)_x_, \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg_local((ptr), (unsigned long)_x_,\ + sizeof(*(ptr))); \ })#define arch_cmpxchg_local (ptr, o, n) \
@@ -34,7 +34,7 @@#undef ____xchg#undef ____cmpxchg -#define ____xchg(type, args...) __xchg ##type(args) +#define ____xchg(type, args...) __arch_xchg ##type(args) #define ____cmpxchg(type, args...) __cmpxchg ##type(args)#include <asm/xchg.h>
@@ -48,7 +48,7 @@ __typeof__(*(ptr)) _x_ = (x); \smp_mb (); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \smp_mb (); \ __ret; \ })
arch/arc/include/asm/cmpxchg.h +2 -2
@@ -85,7 +85,7 @@*/ #ifdef CONFIG_ARC_HAS_LLSC -#define __xchg (ptr, val) \ +#define __arch_xchg (ptr, val) \ ({ \ __asm__ __volatile__( \ " ex %0, [%1] \n"/* set new value */ \
@@ -102,7 +102,7 @@ \switch (sizeof (*(_p_))) { \case 4 : \ - _val_ =__xchg (_p_, _val_); \ + _val_ =__arch_xchg (_p_, _val_); \break ; \default : \BUILD_BUG (); \
arch/arm/include/asm/cmpxchg.h +4 -3
@@ -25,7 +25,8 @@#define swp_is_buggy#endif -static inline unsigned long __xchg (unsigned long x,volatile void *ptr,int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {extern void __bad_xchg (volatile void *,int );unsigned long ret;
@@ -115,8 +116,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size }#define arch_xchg_relaxed (ptr, x) ({ \ - (__typeof__(*(ptr)))__xchg((unsigned long)(x), (ptr), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr)))__arch_xchg((unsigned long)(x), (ptr), \ + sizeof(*(ptr))); \ })#include <asm-generic/cmpxchg-local.h>
arch/arm64/include/asm/cmpxchg.h +3 -4
@@ -62,9 +62,8 @@ __XCHG_CASE( , , mb_, 64, dmb ish, nop, , a, l, "memory")#undef __XCHG_CASE#define __XCHG_GEN (sfx) \ -static __always_inline unsigned long __xchg##sfx(unsigned long x, \ - volatile void *ptr, \ - int size) \ +static __always_inline unsigned long \ +__arch_xchg##sfx(unsigned long x, volatile void *ptr, int size) \ { \ switch (size) { \ case 1: \
@@ -93,7 +92,7 @@ __XCHG_GEN(_mb) ({ \ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
arch/hexagon/include/asm/cmpxchg.h +5 -5
@@ -9,7 +9,7 @@#define _ASM_CMPXCHG_H/* -* __xchg - atomically exchange a register and a memory location +* __arch_xchg - atomically exchange a register and a memory location * @x: value to swap * @ptr: pointer to memory * @size: size of the value
@@ -19,8 +19,8 @@* Note: there was an errata for V2 about .new's and memw_locked. * */ -static inline unsigned long __xchg (unsigned long x,volatile void *ptr, -int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {unsigned long retval;
@@ -42,8 +42,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr,* Atomically swap the contents of a register with memory. Should be atomic * between multiple CPU's and within interrupts on the same CPU. */ -#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__xchg((unsigned long)(v), (ptr), \ - sizeof(*(ptr)))) +#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(v), (ptr), \ + sizeof(*(ptr))))/* * see rt-mutex-design.txt; cmpxchg supposedly checks if *ptr == A and swaps.
arch/ia64/include/asm/cmpxchg.h +1 -1
@@ -5,7 +5,7 @@#include <uapi/asm/cmpxchg.h> #define arch_xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#define arch_cmpxchg (ptr, o, n) cmpxchg_acq((ptr), (o), (n))#define arch_cmpxchg64 (ptr, o, n) cmpxchg_acq((ptr), (o), (n))
arch/ia64/include/uapi/asm/cmpxchg.h +2 -2
@@ -23,7 +23,7 @@*/ extern void ia64_xchg_called_with_bad_pointer (void ); -#define __xchg (x, ptr, size) \ +#define __arch_xchg (x, ptr, size) \ ({ \ unsigned long __xchg_result; \ \
@@ -51,7 +51,7 @@ extern void ia64_xchg_called_with_bad_pointer(void);#ifndef __KERNEL__#define xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#endif /*
arch/loongarch/include/asm/cmpxchg.h +2 -2
@@ -62,7 +62,7 @@ static inline unsigned int __xchg_small(volatile void *ptr, unsigned int val, }static __always_inline unsigned long -__xchg (volatile void *ptr,unsigned long x,int size) +__arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -87,7 +87,7 @@ __xchg(volatile void *ptr, unsigned long x, int size) __typeof__(*(ptr)) __res; \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \ __res; \ })
arch/m68k/include/asm/cmpxchg.h +3 -3
@@ -9,7 +9,7 @@extern unsigned long __invalid_xchg_size (unsigned long ,volatile void *,int );#ifndef CONFIG_RMW_INSNS -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {unsigned long flags, tmp;
@@ -40,7 +40,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz return x; }#else -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
@@ -75,7 +75,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz }#endif -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})#include <asm-generic/cmpxchg-local.h>
arch/mips/include/asm/cmpxchg.h +2 -2
@@ -68,7 +68,7 @@ extern unsigned long __xchg_small(volatile void *ptr, unsigned long val, unsigned int size);static __always_inline -unsigned long __xchg (volatile void *ptr,unsigned long x,int size) +unsigned long __arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -102,7 +102,7 @@ unsigned long __xchg(volatile void *ptr, unsigned long x, int size)smp_mb__before_llsc (); \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \smp_llsc_mb (); \ \
arch/openrisc/include/asm/cmpxchg.h +5 -5
@@ -147,8 +147,8 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, extern unsigned long __xchg_called_with_bad_pointer(void) __compiletime_error("Bad argument size for xchg"); -static inline unsigned long __xchg (volatile void *ptr,unsigned long with, -int size) +static inline unsigned long +__arch_xchg (volatile void *ptr,unsigned long with,int size) {switch (size) {case 1 :
@@ -163,9 +163,9 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long with,#define arch_xchg (ptr, with) \ ({ \ - (__typeof__(*(ptr))) __xchg((ptr), \ - (unsigned long)(with), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), \ + (unsigned long)(with), \ + sizeof(*(ptr))); \ })#endif /* __ASM_OPENRISC_CMPXCHG_H */
arch/parisc/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@ extern unsigned long __xchg64(unsigned long, volatile unsigned long *);/* optimizer better get rid of switch since size is a constant */ static inline unsigned long -__xchg (unsigned long x,volatile void *ptr,int size) +__arch_xchg (unsigned long x,volatile void *ptr,int size) {switch (size) {#ifdef CONFIG_64BIT
@@ -49,7 +49,7 @@ __xchg(unsigned long x, volatile void *ptr, int size) __typeof__(*(ptr)) __ret; \ __typeof__(*(ptr)) _x_ = (x); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ __ret; \ })
arch/powerpc/include/asm/cmpxchg.h +2 -2
@@ -229,7 +229,7 @@ __xchg_local(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_local (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" );return x; }
@@ -248,7 +248,7 @@ __xchg_relaxed(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_relaxed (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_relaxed" );return x; }#define arch_xchg_local (ptr,x) \
arch/riscv/include/asm/atomic.h +1 -1
@@ -261,7 +261,7 @@ c_t arch_atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \ static __always_inline \ c_t arch_atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ { \ - return __xchg(&(v->counter), n, size); \ + return __arch_xchg(&(v->counter), n, size); \ } \ static __always_inline \ c_t arch_atomic##prefix##_cmpxchg_relaxed(atomic##prefix##_t *v, \
arch/riscv/include/asm/cmpxchg.h +2 -2
@@ -114,7 +114,7 @@ _x_, sizeof(*(ptr))); \ }) -#define __xchg (ptr, new, size) \ +#define __arch_xchg (ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(new) __new = (new); \
@@ -143,7 +143,7 @@#define arch_xchg (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg((ptr), _x_, sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ })#define xchg32 (ptr, x) \
arch/s390/include/asm/cmpxchg.h +4 -4
@@ -14,8 +14,8 @@void __xchg_called_with_bad_pointer (void ); -static __always_inline unsigned long __xchg (unsigned long x, -unsigned long address,int size) +static __always_inline unsigned long +__arch_xchg (unsigned long x,unsigned long address,int size) {unsigned long old;int shift;
@@ -77,8 +77,8 @@ static __always_inline unsigned long __xchg(unsigned long x, __typeof__(*(ptr)) __ret; \ \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )(x), (unsigned long )(ptr), \ -sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )(x), (unsigned long )(ptr), \ +sizeof (*(ptr))); \ __ret; \ })
arch/sh/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@extern void __xchg_called_with_bad_pointer (void ); -#define __xchg (ptr, x, size) \ +#define __arch_xchg (ptr, x, size) \ ({ \ unsigned long __xchg__res; \ volatile void *__xchg_ptr = (ptr); \
@@ -46,7 +46,7 @@ extern void __xchg_called_with_bad_pointer(void); })#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((ptr),(unsigned long)(x), sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((ptr),(unsigned long)(x), sizeof(*(ptr))))/* This function doesn't exist, so you'll get a linker error * if something tries to do an invalid cmpxchg(). */
arch/sparc/include/asm/cmpxchg_32.h +2 -2
@@ -15,7 +15,7 @@unsigned long __xchg_u32 (volatile u32 *m,u32 new);void __xchg_called_with_bad_pointer (void ); -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, int size) +static inline unsigned long __arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 4 :
@@ -25,7 +25,7 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, intreturn x; } -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})/* Emulate cmpxchg() the same way we emulate atomics, * by hashing the object address and indexing into an array
arch/sparc/include/asm/cmpxchg_64.h +3 -3
@@ -55,7 +55,7 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long#define arch_xchg (ptr,x) \ ({ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
@@ -87,8 +87,8 @@ xchg16(__volatile__ unsigned short *m, unsigned short val)return (load32 & mask) >> bit_shift; } -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, -int size) +static inline unsigned long +__arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 2 :
arch/xtensa/include/asm/cmpxchg.h +2 -2
@@ -170,7 +170,7 @@ static inline unsigned long xchg_u32(volatile int * m, unsigned long val) }#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))static inline u32 xchg_small (volatile void *ptr,u32 x,int size) {
@@ -203,7 +203,7 @@ static inline u32 xchg_small(volatile void *ptr, u32 x, int size)extern void __xchg_called_with_bad_pointer (void );static __inline__unsigned long -__xchg (unsigned long x,volatile void * ptr,int size) +__arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
[tip: locking/core] locking/arch: Rename all internal __xchg() names to __arch_xchg()
In reply to Andrzej HajdaThe following commit has been merged into the locking/core branch of tip:
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 068550631fbe0b7fb41625cea6fb204fdc8cb224 Gitweb: https://git.kernel.org/tip/068550631fbe0b7fb41625cea6fb204fdc8cb224 Author: Andrzej Hajda <[email protected]> AuthorDate: Wed, 18 Jan 2023 16:44:44 +01:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Sat, 29 Apr 2023 09:08:44 +02:00
locking/arch: Rename all internal __xchg() names to __arch_xchg()
Decrease the probability of this internal facility to be used by driver code.
--- arch/alpha/include/asm/cmpxchg.h | 10 +++++----- arch/arc/include/asm/cmpxchg.h | 4 ++-- arch/arm/include/asm/cmpxchg.h | 7 ++++--- arch/arm64/include/asm/cmpxchg.h | 7 +++---- arch/hexagon/include/asm/cmpxchg.h | 10 +++++----- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 ++-- arch/loongarch/include/asm/cmpxchg.h | 4 ++-- arch/m68k/include/asm/cmpxchg.h | 6 +++--- arch/mips/include/asm/cmpxchg.h | 4 ++-- arch/openrisc/include/asm/cmpxchg.h | 10 +++++----- arch/parisc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/include/asm/cmpxchg.h | 4 ++-- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 ++-- arch/s390/include/asm/cmpxchg.h | 8 ++++---- arch/sh/include/asm/cmpxchg.h | 4 ++-- arch/sparc/include/asm/cmpxchg_32.h | 4 ++-- arch/sparc/include/asm/cmpxchg_64.h | 6 +++--- arch/xtensa/include/asm/cmpxchg.h | 4 ++-- 20 files changed, 54 insertions(+), 54 deletions(-)
arch/alpha/include/asm/cmpxchg.h +5 -5
@@ -6,15 +6,15 @@* Atomic exchange routines. */ -#define ____xchg(type, args...) __xchg ## type ## _local(args) +#define ____xchg(type, args...) __arch_xchg ## type ## _local(args) #define ____cmpxchg(type, args...) __cmpxchg ## type ## _local(args)#include <asm/xchg.h> #define xchg_local (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_local((ptr), (unsigned long)_x_, \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg_local((ptr), (unsigned long)_x_,\ + sizeof(*(ptr))); \ })#define arch_cmpxchg_local (ptr, o, n) \
@@ -34,7 +34,7 @@#undef ____xchg#undef ____cmpxchg -#define ____xchg(type, args...) __xchg ##type(args) +#define ____xchg(type, args...) __arch_xchg ##type(args) #define ____cmpxchg(type, args...) __cmpxchg ##type(args)#include <asm/xchg.h>
@@ -48,7 +48,7 @@ __typeof__(*(ptr)) _x_ = (x); \smp_mb (); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )_x_,sizeof (*(ptr))); \smp_mb (); \ __ret; \ })
arch/arc/include/asm/cmpxchg.h +2 -2
@@ -85,7 +85,7 @@*/ #ifdef CONFIG_ARC_HAS_LLSC -#define __xchg (ptr, val) \ +#define __arch_xchg (ptr, val) \ ({ \ __asm__ __volatile__( \ " ex %0, [%1] \n"/* set new value */ \
@@ -102,7 +102,7 @@ \switch (sizeof (*(_p_))) { \case 4 : \ - _val_ =__xchg (_p_, _val_); \ + _val_ =__arch_xchg (_p_, _val_); \break ; \default : \BUILD_BUG (); \
arch/arm/include/asm/cmpxchg.h +4 -3
@@ -25,7 +25,8 @@#define swp_is_buggy#endif -static inline unsigned long __xchg (unsigned long x,volatile void *ptr,int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {extern void __bad_xchg (volatile void *,int );unsigned long ret;
@@ -115,8 +116,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size }#define arch_xchg_relaxed (ptr, x) ({ \ - (__typeof__(*(ptr)))__xchg((unsigned long)(x), (ptr), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr)))__arch_xchg((unsigned long)(x), (ptr), \ + sizeof(*(ptr))); \ })#include <asm-generic/cmpxchg-local.h>
arch/arm64/include/asm/cmpxchg.h +3 -4
@@ -62,9 +62,8 @@ __XCHG_CASE( , , mb_, 64, dmb ish, nop, , a, l, "memory")#undef __XCHG_CASE#define __XCHG_GEN (sfx) \ -static __always_inline unsigned long __xchg##sfx(unsigned long x, \ - volatile void *ptr, \ - int size) \ +static __always_inline unsigned long \ +__arch_xchg##sfx(unsigned long x, volatile void *ptr, int size) \ { \ switch (size) { \ case 1: \
@@ -93,7 +92,7 @@ __XCHG_GEN(_mb) ({ \ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
arch/hexagon/include/asm/cmpxchg.h +5 -5
@@ -9,7 +9,7 @@#define _ASM_CMPXCHG_H/* -* __xchg - atomically exchange a register and a memory location +* __arch_xchg - atomically exchange a register and a memory location * @x: value to swap * @ptr: pointer to memory * @size: size of the value
@@ -19,8 +19,8 @@* Note: there was an errata for V2 about .new's and memw_locked. * */ -static inline unsigned long __xchg (unsigned long x,volatile void *ptr, -int size) +static inline unsigned long +__arch_xchg (unsigned long x,volatile void *ptr,int size) {unsigned long retval;
@@ -42,8 +42,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr,* Atomically swap the contents of a register with memory. Should be atomic * between multiple CPU's and within interrupts on the same CPU. */ -#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__xchg((unsigned long)(v), (ptr), \ - sizeof(*(ptr)))) +#define arch_xchg (ptr, v) ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(v), (ptr), \ + sizeof(*(ptr))))/* * see rt-mutex-design.txt; cmpxchg supposedly checks if *ptr == A and swaps.
arch/ia64/include/asm/cmpxchg.h +1 -1
@@ -5,7 +5,7 @@#include <uapi/asm/cmpxchg.h> #define arch_xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#define arch_cmpxchg (ptr, o, n) cmpxchg_acq((ptr), (o), (n))#define arch_cmpxchg64 (ptr, o, n) cmpxchg_acq((ptr), (o), (n))
arch/ia64/include/uapi/asm/cmpxchg.h +2 -2
@@ -23,7 +23,7 @@*/ extern void ia64_xchg_called_with_bad_pointer (void ); -#define __xchg (x, ptr, size) \ +#define __arch_xchg (x, ptr, size) \ ({ \ unsigned long __xchg_result; \ \
@@ -51,7 +51,7 @@ extern void ia64_xchg_called_with_bad_pointer(void);#ifndef __KERNEL__#define xchg (ptr, x) \ -({(__typeof__(*(ptr))) __xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));}) +({(__typeof__(*(ptr))) __arch_xchg((unsigned long) (x), (ptr), sizeof(*(ptr)));})#endif /*
arch/loongarch/include/asm/cmpxchg.h +2 -2
@@ -62,7 +62,7 @@ static inline unsigned int __xchg_small(volatile void *ptr, unsigned int val, }static __always_inline unsigned long -__xchg (volatile void *ptr,unsigned long x,int size) +__arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -87,7 +87,7 @@ __xchg(volatile void *ptr, unsigned long x, int size) __typeof__(*(ptr)) __res; \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \ __res; \ })
arch/m68k/include/asm/cmpxchg.h +3 -3
@@ -9,7 +9,7 @@extern unsigned long __invalid_xchg_size (unsigned long ,volatile void *,int );#ifndef CONFIG_RMW_INSNS -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {unsigned long flags, tmp;
@@ -40,7 +40,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz return x; }#else -static inline unsigned long __xchg (unsigned long x,volatile void * ptr,int size) +static inline unsigned long __arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
@@ -75,7 +75,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void * ptr, int siz }#endif -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})#include <asm-generic/cmpxchg-local.h>
arch/mips/include/asm/cmpxchg.h +2 -2
@@ -68,7 +68,7 @@ extern unsigned long __xchg_small(volatile void *ptr, unsigned long val, unsigned int size);static __always_inline -unsigned long __xchg (volatile void *ptr,unsigned long x,int size) +unsigned long __arch_xchg (volatile void *ptr,unsigned long x,int size) {switch (size) {case 1 :
@@ -102,7 +102,7 @@ unsigned long __xchg(volatile void *ptr, unsigned long x, int size)smp_mb__before_llsc (); \ \ __res = (__typeof__ (*(ptr ))) \ -__xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ +__arch_xchg ((ptr), (unsigned long )(x),sizeof (*(ptr))); \ \smp_llsc_mb (); \ \
arch/openrisc/include/asm/cmpxchg.h +5 -5
@@ -147,8 +147,8 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, extern unsigned long __xchg_called_with_bad_pointer(void) __compiletime_error("Bad argument size for xchg"); -static inline unsigned long __xchg (volatile void *ptr,unsigned long with, -int size) +static inline unsigned long +__arch_xchg (volatile void *ptr,unsigned long with,int size) {switch (size) {case 1 :
@@ -163,9 +163,9 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long with,#define arch_xchg (ptr, with) \ ({ \ - (__typeof__(*(ptr))) __xchg((ptr), \ - (unsigned long)(with), \ - sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), \ + (unsigned long)(with), \ + sizeof(*(ptr))); \ })#endif /* __ASM_OPENRISC_CMPXCHG_H */
arch/parisc/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@ extern unsigned long __xchg64(unsigned long, volatile unsigned long *);/* optimizer better get rid of switch since size is a constant */ static inline unsigned long -__xchg (unsigned long x,volatile void *ptr,int size) +__arch_xchg (unsigned long x,volatile void *ptr,int size) {switch (size) {#ifdef CONFIG_64BIT
@@ -49,7 +49,7 @@ __xchg(unsigned long x, volatile void *ptr, int size) __typeof__(*(ptr)) __ret; \ __typeof__(*(ptr)) _x_ = (x); \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )_x_, (ptr),sizeof (*(ptr))); \ __ret; \ })
arch/powerpc/include/asm/cmpxchg.h +2 -2
@@ -229,7 +229,7 @@ __xchg_local(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_local (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" );return x; }
@@ -248,7 +248,7 @@ __xchg_relaxed(void *ptr, unsigned long x, unsigned int size)return __xchg_u64_relaxed (ptr, x);#endif } -BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_local" ); +BUILD_BUG_ON_MSG (1 ,"Unsupported size for __xchg_relaxed" );return x; }#define arch_xchg_local (ptr,x) \
arch/riscv/include/asm/atomic.h +1 -1
@@ -261,7 +261,7 @@ c_t arch_atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \ static __always_inline \ c_t arch_atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ { \ - return __xchg(&(v->counter), n, size); \ + return __arch_xchg(&(v->counter), n, size); \ } \ static __always_inline \ c_t arch_atomic##prefix##_cmpxchg_relaxed(atomic##prefix##_t *v, \
arch/riscv/include/asm/cmpxchg.h +2 -2
@@ -114,7 +114,7 @@ _x_, sizeof(*(ptr))); \ }) -#define __xchg (ptr, new, size) \ +#define __arch_xchg (ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(new) __new = (new); \
@@ -143,7 +143,7 @@#define arch_xchg (ptr, x) \ ({ \ __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg((ptr), _x_, sizeof(*(ptr))); \ + (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ })#define xchg32 (ptr, x) \
arch/s390/include/asm/cmpxchg.h +4 -4
@@ -14,8 +14,8 @@void __xchg_called_with_bad_pointer (void ); -static __always_inline unsigned long __xchg (unsigned long x, -unsigned long address,int size) +static __always_inline unsigned long +__arch_xchg (unsigned long x,unsigned long address,int size) {unsigned long old;int shift;
@@ -77,8 +77,8 @@ static __always_inline unsigned long __xchg(unsigned long x, __typeof__(*(ptr)) __ret; \ \ __ret = (__typeof__ (*(ptr ))) \ -__xchg ((unsigned long )(x), (unsigned long )(ptr), \ -sizeof (*(ptr))); \ +__arch_xchg ((unsigned long )(x), (unsigned long )(ptr), \ +sizeof (*(ptr))); \ __ret; \ })
arch/sh/include/asm/cmpxchg.h +2 -2
@@ -22,7 +22,7 @@extern void __xchg_called_with_bad_pointer (void ); -#define __xchg (ptr, x, size) \ +#define __arch_xchg (ptr, x, size) \ ({ \ unsigned long __xchg__res; \ volatile void *__xchg_ptr = (ptr); \
@@ -46,7 +46,7 @@ extern void __xchg_called_with_bad_pointer(void); })#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((ptr),(unsigned long)(x), sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((ptr),(unsigned long)(x), sizeof(*(ptr))))/* This function doesn't exist, so you'll get a linker error * if something tries to do an invalid cmpxchg(). */
arch/sparc/include/asm/cmpxchg_32.h +2 -2
@@ -15,7 +15,7 @@unsigned long __xchg_u32 (volatile u32 *m,u32 new);void __xchg_called_with_bad_pointer (void ); -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, int size) +static inline unsigned long __arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 4 :
@@ -25,7 +25,7 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, intreturn x; } -#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));}) +#define arch_xchg (ptr,x) ({(__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr)));})/* Emulate cmpxchg() the same way we emulate atomics, * by hashing the object address and indexing into an array
arch/sparc/include/asm/cmpxchg_64.h +3 -3
@@ -55,7 +55,7 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long#define arch_xchg (ptr,x) \ ({ __typeof__(*(ptr)) __ret; \ __ret = (__typeof__(*(ptr))) \ - __xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ + __arch_xchg((unsigned long)(x), (ptr), sizeof(*(ptr))); \ __ret; \ })
@@ -87,8 +87,8 @@ xchg16(__volatile__ unsigned short *m, unsigned short val)return (load32 & mask) >> bit_shift; } -static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr, -int size) +static inline unsigned long +__arch_xchg(unsigned long x, __volatile__ void * ptr, int size) {switch (size) {case 2 :
arch/xtensa/include/asm/cmpxchg.h +2 -2
@@ -170,7 +170,7 @@ static inline unsigned long xchg_u32(volatile int * m, unsigned long val) }#define arch_xchg (ptr,x) \ - ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr)))) + ((__typeof__(*(ptr)))__arch_xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))static inline u32 xchg_small (volatile void *ptr,u32 x,int size) {
@@ -203,7 +203,7 @@ static inline u32 xchg_small(volatile void *ptr, u32 x, int size)extern void __xchg_called_with_bad_pointer (void );static __inline__unsigned long -__xchg (unsigned long x,volatile void * ptr,int size) +__arch_xchg (unsigned long x,volatile void * ptr,int size) {switch (size) {case 1 :
Hi,
Ping on the series.
Arnd, Andrew is there anything more I can do to push the process forward?
Regards
Andrzej
Quoted 109 lines · Andrzej Hajda
Hi all,
The helper is tiny and there are advices we can live without it, so I want to present few arguments why it would be good to have it:
1. Code readability/simplification/number of lines:
- decreases number of lines,
- it often eliminates local variables,
- for real examples see patches 3+.2. Presence of similar helpers in other somehow related languages/libs:
a) Rust[1]: 'replace' from
std::memmodule, there is also 'take' helper (__xchg(&x, 0)), which is the same as private helper in i915 -fetch_and_zero, see latest patch.
b) C++ [2]: 'exchange' from utility header.If the idea is OK there are still 2 questions to answer:
1. Name of the helper,
__xchgfollows kernel conventions, but for me Rust names are also OK.
2. Where to put the helper:
a) as in this patchsetinclude/linux/non-atomic/xchg.h, proposed by Andy Shevchenko,
b)include/linux/utils.h? any better name? Some kind of container for simple helpers.All
__xchgconversions were performed using cocci script, then manually adjusted if necessary.There is lot of places it can be used in, I have just chosen some of them. I can provide cocci script to detect others (not all), if necessary.
Changes:
v2: squashed all__xchg->__arch_xchgt one patch (Arnd)
v3: fixed alpha/xchg_local ([email protected])
v4: adjusted indentation (Heiko)
v5: added more__xchgconversions - patches 3-6, added tags[1]: https://doc.rust-lang.org/std/mem/index.html
[2]: https://en.cppreference.com/w/cpp/header/utilityRegards
AndrzejAndrzej Hajda (7):
arch: rename all internal names__xchgto__arch_xchglinux/include: add non-atomic version of xchgarch/*/uprobes: simplifyarch_uretprobe_hijack_return_addr
llist: simplify__llist_del_all
io_uring: use__xchgif possible
qed: use__xchgif possible
drm/i915/gt: use__xchginstead of internal helperarch/alpha/include/asm/cmpxchg.h | 10 +++++----- arch/arc/include/asm/cmpxchg.h | 4 ++-- arch/arm/include/asm/cmpxchg.h | 7 ++++--- arch/arm/probes/uprobes/core.c | 8 ++------ arch/arm64/include/asm/cmpxchg.h | 7 +++---- arch/arm64/kernel/probes/uprobes.c | 9 ++------- arch/csky/kernel/probes/uprobes.c | 9 ++------- arch/hexagon/include/asm/cmpxchg.h | 10 +++++----- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 ++-- arch/loongarch/include/asm/cmpxchg.h | 4 ++-- arch/m68k/include/asm/cmpxchg.h | 6 +++--- arch/mips/include/asm/cmpxchg.h | 4 ++-- arch/mips/kernel/uprobes.c | 10 ++-------- arch/openrisc/include/asm/cmpxchg.h | 10 +++++----- arch/parisc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/include/asm/cmpxchg.h | 4 ++-- arch/powerpc/kernel/uprobes.c | 10 ++-------- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 ++-- arch/riscv/kernel/probes/uprobes.c | 9 ++------- arch/s390/include/asm/cmpxchg.h | 8 ++++---- arch/s390/kernel/uprobes.c | 7 ++----- arch/sh/include/asm/cmpxchg.h | 4 ++-- arch/sparc/include/asm/cmpxchg_32.h | 4 ++-- arch/sparc/include/asm/cmpxchg_64.h | 6 +++--- arch/sparc/kernel/uprobes.c | 7 ++----- arch/xtensa/include/asm/cmpxchg.h | 4 ++-- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +- .../gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++-- .../drm/i915/gt/intel_execlists_submission.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gsc.c | 2 +- drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++-- drivers/gpu/drm/i915/gt/intel_gt_pm.c | 2 +- drivers/gpu/drm/i915/gt/intel_lrc.c | 6 +++--- drivers/gpu/drm/i915/gt/intel_migrate.c | 2 +- drivers/gpu/drm/i915/gt/intel_rc6.c | 2 +- drivers/gpu/drm/i915/gt/intel_rps.c | 2 +- drivers/gpu/drm/i915/gt/selftest_context.c | 2 +- .../drm/i915/gt/selftest_ring_submission.c | 2 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 2 +- drivers/gpu/drm/i915/i915_utils.h | 1 + include/linux/llist.h | 6 ++---- include/linux/non-atomic/xchg.h | 19 +++++++++++++++++++ include/linux/qed/qed_chain.h | 19 +++++++------------ io_uring/io_uring.c | 7 ++----- io_uring/slist.h | 6 ++---- 51 files changed, 126 insertions(+), 155 deletions(-)create mode 100644
include/linux/non-atomic/xchg.h
Peter Zijlstra
maintainerQuoted 8 lines · Andrzej Hajda
Andrzej Hajda (7):
arch: rename all internal names__xchgto__arch_xchglinux/include: add non-atomic version of xchgarch/*/uprobes: simplifyarch_uretprobe_hijack_return_addr
llist: simplify__llist_del_all
io_uring: use__xchgif possible
qed: use__xchgif possible
drm/i915/gt: use__xchginstead of internal helper
Nothing crazy in here I suppose, I somewhat wonder why you went through the trouble, but meh.
You want me to take this through te locking tree (for the next cycle, not this one) where I normally take atomic things or does someone else want this?
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Andrzej Hajda
Andrzej Hajda (7):
arch: rename all internal names__xchgto__arch_xchglinux/include: add non-atomic version of xchgarch/*/uprobes: simplifyarch_uretprobe_hijack_return_addr
llist: simplify__llist_del_all
io_uring: use__xchgif possible
qed: use__xchgif possible
drm/i915/gt: use__xchginstead of internal helper
Quoted 2 lines
Nothing crazy in here I suppose, I somewhat wonder why you went through the trouble, but meh.
If you are asking why I have proposed this patchset, then the answer is simple, 1st I've tried to find a way to move internal i915 helper to core (see patch 7). Then I was looking for possible other users of this helper. And apparently there are many of them, patches 3-7 shows some.
Quoted 4 lines
You want me to take this through te locking tree (for the next cycle, not this one) where I normally take atomic things or does someone else want this?
If you could take it I will be happy.
Regards
Andrzej
Peter Zijlstra
maintainerQuoted 23 lines · Andrzej Hajda
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Andrzej Hajda
Andrzej Hajda (7):
arch: rename all internal names__xchgto__arch_xchglinux/include: add non-atomic version of xchgarch/*/uprobes: simplifyarch_uretprobe_hijack_return_addr
llist: simplify__llist_del_all
io_uring: use__xchgif possible
qed: use__xchgif possible
drm/i915/gt: use__xchginstead of internal helperQuoted 2 lines
Nothing crazy in here I suppose, I somewhat wonder why you went through the trouble, but meh.
If you are asking why I have proposed this patchset, then the answer is simple, 1st I've tried to find a way to move internal i915 helper to core (see patch 7).
Then I was looking for possible other users of this helper. And apparently there are many of them, patches 3-7 shows some.Quoted 1 line
Quoted 5 lines
Quoted 3 lines
You want me to take this through te locking tree (for the next cycle, not this one) where I normally take atomic things or does someone else want this?
If you could take it I will be happy.
OK, I'll go queue it in tip/locking/core after -rc1. Thanks!
Jani Nikula
maintainerQuoted 24 lines · Peter Zijlstra
Quoted 23 lines · Andrzej Hajda
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Andrzej Hajda
Andrzej Hajda (7):
arch: rename all internal names__xchgto__arch_xchglinux/include: add non-atomic version of xchgarch/*/uprobes: simplifyarch_uretprobe_hijack_return_addr
llist: simplify__llist_del_all
io_uring: use__xchgif possible
qed: use__xchgif possible
drm/i915/gt: use__xchginstead of internal helperQuoted 2 lines
Nothing crazy in here I suppose, I somewhat wonder why you went through the trouble, but meh.
If you are asking why I have proposed this patchset, then the answer is simple, 1st I've tried to find a way to move internal i915 helper to core (see patch 7).
Then I was looking for possible other users of this helper. And apparently there are many of them, patches 3-7 shows some.Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
You want me to take this through te locking tree (for the next cycle, not this one) where I normally take atomic things or does someone else want this?
If you could take it I will be happy.
OK, I'll go queue it in tip/locking/core after -rc1. Thanks!
Is this where the series fell between the cracks, or was there some follow-up that I missed?
I think this would still be useful. Andrzej, would you mind rebasing and resending if there are no objections?
BR,
Jani.
Signature · 1 line
Jani Nikula, Intel
Quoted 25 lines · Jani Nikula
Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · Andrzej Hajda
Quoted 10 lines · Peter Zijlstra
Quoted 8 lines · Andrzej Hajda
Andrzej Hajda (7):
arch: rename all internal names__xchgto__arch_xchglinux/include: add non-atomic version of xchgarch/*/uprobes: simplifyarch_uretprobe_hijack_return_addr
llist: simplify__llist_del_all
io_uring: use__xchgif possible
qed: use__xchgif possible
drm/i915/gt: use__xchginstead of internal helperQuoted 2 lines
Nothing crazy in here I suppose, I somewhat wonder why you went through the trouble, but meh.
If you are asking why I have proposed this patchset, then the answer is simple, 1st I've tried to find a way to move internal i915 helper to core (see patch 7).
Then I was looking for possible other users of this helper. And apparently there are many of them, patches 3-7 shows some.Quoted 3 lines
You want me to take this through te locking tree (for the next cycle, not this one) where I normally take atomic things or does someone else want this?
Quoted 7 lines
Quoted 2 lines
Quoted 1 line
If you could take it I will be happy.
OK, I'll go queue it in tip/locking/core after -rc1. Thanks!
Is this where the series fell between the cracks, or was there some follow-up that I missed?
I think this would still be useful. Andrzej, would you mind rebasing and resending if there are no objections?
The patchset was rejected/dropped by Linus at the pull-request stage.
He didn't like many things, but the most __xchg name. However he was quite positive about i915 name fetch_and_zero. I can try to revive patchset with fetch_and_zero, and maybe fetch_and_set, instead of __xchg.
Regards
Andrzej
Quoted 5 lines
BR,
Jani.