Rework & improve riscv cmpxchg.h and atomic.h
Subsystems (2)
v4 cover Rework & improve riscv cmpxchg.h and atomic.h
While studying riscv's cmpxchg.h file, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.
When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.
Also, did the same kind of work on atomic.c.
After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.
Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)
I did compare the generated asm on a test.c that contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.
Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".
Thanks!
Leo
Changes since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from _mask to _masked for patches 4 & 5
- __rc variable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/
Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/
Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/
Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/
Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/
Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/
Leonardo Bras (5):
riscv/cmpxchg: Deduplicate xchg() asm functions riscv/cmpxchg: Deduplicate cmpxchg() asm and macros riscv/atomic.h : Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2
arch/riscv/include/asm/atomic.h | 164 ++++++------- arch/riscv/include/asm/cmpxchg.h | 394 ++++++++++--------------------- 2 files changed, 195 insertions(+), 363 deletions(-)
Signature · 7 lines
2.41.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v4 1/5 riscv/cmpxchg: Deduplicate xchg() asm functions
In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm.
This is usually bad, as it means any change may be done in up to 8 different places.
Unify those versions by creating a new define with enough parameters to generate any version of the previous 8.
Then unify the result under a more general define, and simplify arch_xchg* generation.
(This did not cause any change in generated asm)
--- arch/riscv/include/asm/cmpxchg.h | 136 +++++-------------------------- 1 file changed, 22 insertions(+), 114 deletions(-)
arch/riscv/include/asm/cmpxchg.h +22 -114
@@ -11,60 +11,30 @@#include <asm/barrier.h> #include <asm/fence.h> -#define __xchg_relaxed (ptr, new, size) \ +#define __arch_xchg (sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed (ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=r" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) -#define __xchg_acquire (ptr, new, size) \ +#define _arch_xchg (ptr, new, sfx, prepend, append) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ + __typeof__(*(ptr)) __new = (new); \ __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(ptr) __ptr = (ptr); \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \
@@ -72,79 +42,17 @@ __ret; \ }) -#define arch_xchg_acquire (ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed (ptr, x) \ + _arch_xchg(ptr, x, "", "", "") -#define __xchg_release (ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire (ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER)#define arch_xchg_release (ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg (ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "")#define arch_xchg (ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "")#define xchg32 (ptr, x) \ ({ \
Signature · 7 lines
2.41.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
kernel test robot
build botRe: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions
kernel test robot noticed the following build warnings:
Hi Leonardo,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v6.5-rc5 next-20230809] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Leonardo-Bras/riscv-cmpxchg-Deduplicate-xchg-asm-functions/20230809-101624 base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core patch link: https://lore.kernel.org/r/20230809021311.1390578-3-leobras%40redhat.com patch subject: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions config: riscv-randconfig-r072-20230809 (https://download.01.org/0day-ci/archive/20230810/[email protected]/config) compiler: riscv32-linux-gcc (GCC) 12.3.0 reproduce: (https://download.01.org/0day-ci/archive/20230810/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
sparse warnings: (new ones prefixed by >>)
WARNING: invalid argument to '-march': '_zihintpause'
>> net/ipv4/tcp_cong.c:300:24: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected struct tcp_congestion_ops const [noderef] __rcu *__new @@ got struct tcp_congestion_ops *[assigned] ca @@
net/ipv4/tcp_cong.c:300:24: sparse: expected struct tcp_congestion_ops const [noderef] __rcu *__new
net/ipv4/tcp_cong.c:300:24: sparse: got struct tcp_congestion_ops *[assigned] ca
net/ipv4/tcp_cong.c:300:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct tcp_congestion_ops const *prev @@ got struct tcp_congestion_ops const [noderef] __rcu *[assigned] __ret @@
net/ipv4/tcp_cong.c:300:22: sparse: expected struct tcp_congestion_ops const *prev
net/ipv4/tcp_cong.c:300:22: sparse: got struct tcp_congestion_ops const [noderef] __rcu *[assigned] __ret
vim +300 net/ipv4/tcp_cong.c
317a76f9a44b43 Stephen Hemminger 2005-06-23 281
317a76f9a44b43 Stephen Hemminger 2005-06-23 282 /* Used by sysctl to change default congestion control */
6670e152447732 Stephen Hemminger 2017-11-14 283 int tcp_set_default_congestion_control(struct net *net, const char *name)
317a76f9a44b43 Stephen Hemminger 2005-06-23 284 {
317a76f9a44b43 Stephen Hemminger 2005-06-23 285 struct tcp_congestion_ops *ca;
6670e152447732 Stephen Hemminger 2017-11-14 286 const struct tcp_congestion_ops *prev;
6670e152447732 Stephen Hemminger 2017-11-14 287 int ret;
317a76f9a44b43 Stephen Hemminger 2005-06-23 288
6670e152447732 Stephen Hemminger 2017-11-14 289 rcu_read_lock();
6670e152447732 Stephen Hemminger 2017-11-14 290 ca = tcp_ca_find_autoload(net, name);
6670e152447732 Stephen Hemminger 2017-11-14 291 if (!ca) {
6670e152447732 Stephen Hemminger 2017-11-14 292 ret = -ENOENT;
0baf26b0fcd74b Martin KaFai Lau 2020-01-08 293 } else if (!bpf_try_module_get(ca, ca->owner)) {
6670e152447732 Stephen Hemminger 2017-11-14 294 ret = -EBUSY;
8d432592f30fcc Jonathon Reinhart 2021-05-01 295 } else if (!net_eq(net, &init_net) &&
8d432592f30fcc Jonathon Reinhart 2021-05-01 296 !(ca->flags & TCP_CONG_NON_RESTRICTED)) {
8d432592f30fcc Jonathon Reinhart 2021-05-01 297 /* Only init netns can set default to a restricted algorithm */
8d432592f30fcc Jonathon Reinhart 2021-05-01 298 ret = -EPERM;
6670e152447732 Stephen Hemminger 2017-11-14 299 } else {
6670e152447732 Stephen Hemminger 2017-11-14 @300 prev = xchg(&net->ipv4.tcp_congestion_control, ca);
6670e152447732 Stephen Hemminger 2017-11-14 301 if (prev)
0baf26b0fcd74b Martin KaFai Lau 2020-01-08 302 bpf_module_put(prev, prev->owner);
317a76f9a44b43 Stephen Hemminger 2005-06-23 303
6670e152447732 Stephen Hemminger 2017-11-14 304 ca->flags |= TCP_CONG_NON_RESTRICTED;
317a76f9a44b43 Stephen Hemminger 2005-06-23 305 ret = 0;
317a76f9a44b43 Stephen Hemminger 2005-06-23 306 }
6670e152447732 Stephen Hemminger 2017-11-14 307 rcu_read_unlock();
317a76f9a44b43 Stephen Hemminger 2005-06-23 308
317a76f9a44b43 Stephen Hemminger 2005-06-23 309 return ret;
317a76f9a44b43 Stephen Hemminger 2005-06-23 310 }
317a76f9a44b43 Stephen Hemminger 2005-06-23 311 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions
I was not aware Linux supported 32-bit RISC-V.
IIUC, compared to previous code, I am missing a (__typeof__(*(p))) in the output of my code. Should be easily fixed.
Quoted 12 lines · kernel test robot
Hi Leonardo,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v6.5-rc5 next-20230809] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
Quoted 7 lines
url: https://github.com/intel-lab-lkp/linux/commits/Leonardo-Bras/riscv-cmpxchg-Deduplicate-xchg-asm-functions/20230809-101624 base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core patch link: https://lore.kernel.org/r/20230809021311.1390578-3-leobras%40redhat.com patch subject: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions config: riscv-randconfig-r072-20230809 (https://download.01.org/0day-ci/archive/20230810/[email protected]/config) compiler: riscv32-linux-gcc (GCC) 12.3.0 reproduce: (https://download.01.org/0day-ci/archive/20230810/[email protected]/reproduce)
For some reason, following this reproduce causes this bug to happen before reproduction:
$ COMPILER_INSTALL_PATH=/root/0day/ COMPILER=gcc-12.3.0 ~/bin/make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=/out/ ARCH=riscv SHELL=/bin/bash net/ipv4/ [...] HOSTCC scripts/selinux/genheaders/genheaders CC scripts/mod/empty.o HOSTCC scripts/mod/mk_elfconfig CC scripts/mod/devicetable-offsets.s CHECK /linux/scripts/mod/empty.c sparse: invalid argument to '-march': '_zicsr_zifencei_zihintpause'
Seems like the compiler error.
Anyway, I tested the 64-bit version, seems ok now.
Quoted 53 lines
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/sparse warnings: (new ones prefixed by >>)
WARNING: invalid argument to '-march': '_zihintpause'Quoted 1 line
Quoted 1 line
net/ipv4/tcp_cong.c:300:24: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected structtcp_congestion_opsconst [noderef]__rcu*__new@@ got structtcp_congestion_ops*[assigned] ca @@
net/ipv4/tcp_cong.c:300:24: sparse: expected structtcp_congestion_opsconst [noderef]__rcu*__new
net/ipv4/tcp_cong.c:300:24: sparse: got structtcp_congestion_ops*[assigned] ca
net/ipv4/tcp_cong.c:300:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected structtcp_congestion_opsconst *prev @@ got structtcp_congestion_opsconst [noderef]__rcu*[assigned]__ret@@
net/ipv4/tcp_cong.c:300:22: sparse: expected structtcp_congestion_opsconst *prev
net/ipv4/tcp_cong.c:300:22: sparse: got structtcp_congestion_opsconst [noderef]__rcu*[assigned]__retvim +300
net/ipv4/tcp_cong.c317a76f9a44b43 Stephen Hemminger 2005-06-23 281 317a76f9a44b43 Stephen Hemminger 2005-06-23 282 /* Used by sysctl to change default congestion control */ 6670e152447732 Stephen Hemminger 2017-11-14 283 int tcp_set_default_congestion_control(struct net *net, const char *name) 317a76f9a44b43 Stephen Hemminger 2005-06-23 284 { 317a76f9a44b43 Stephen Hemminger 2005-06-23 285 struct tcp_congestion_ops *ca; 6670e152447732 Stephen Hemminger 2017-11-14 286 const struct tcp_congestion_ops *prev; 6670e152447732 Stephen Hemminger 2017-11-14 287 int ret; 317a76f9a44b43 Stephen Hemminger 2005-06-23 288 6670e152447732 Stephen Hemminger 2017-11-14 289 rcu_read_lock(); 6670e152447732 Stephen Hemminger 2017-11-14 290 ca = tcp_ca_find_autoload(net, name); 6670e152447732 Stephen Hemminger 2017-11-14 291 if (!ca) { 6670e152447732 Stephen Hemminger 2017-11-14 292 ret = -ENOENT; 0baf26b0fcd74b Martin KaFai Lau 2020-01-08 293 } else if (!bpf_try_module_get(ca, ca->owner)) { 6670e152447732 Stephen Hemminger 2017-11-14 294 ret = -EBUSY; 8d432592f30fcc Jonathon Reinhart 2021-05-01 295 } else if (!net_eq(net, &init_net) && 8d432592f30fcc Jonathon Reinhart 2021-05-01 296 !(ca->flags & TCP_CONG_NON_RESTRICTED)) { 8d432592f30fcc Jonathon Reinhart 2021-05-01 297 /* Only init netns can set default to a restricted algorithm */ 8d432592f30fcc Jonathon Reinhart 2021-05-01 298 ret = -EPERM; 6670e152447732 Stephen Hemminger 2017-11-14 299 } else { 6670e152447732 Stephen Hemminger 2017-11-14 @300 prev = xchg(&net->ipv4.tcp_congestion_control, ca); 6670e152447732 Stephen Hemminger 2017-11-14 301 if (prev) 0baf26b0fcd74b Martin KaFai Lau 2020-01-08 302 bpf_module_put(prev, prev->owner); 317a76f9a44b43 Stephen Hemminger 2005-06-23 303 6670e152447732 Stephen Hemminger 2017-11-14 304 ca->flags |= TCP_CONG_NON_RESTRICTED; 317a76f9a44b43 Stephen Hemminger 2005-06-23 305 ret = 0; 317a76f9a44b43 Stephen Hemminger 2005-06-23 306 } 6670e152447732 Stephen Hemminger 2017-11-14 307 rcu_read_unlock(); 317a76f9a44b43 Stephen Hemminger 2005-06-23 308 317a76f9a44b43 Stephen Hemminger 2005-06-23 309 return ret; 317a76f9a44b43 Stephen Hemminger 2005-06-23 310 } 317a76f9a44b43 Stephen Hemminger 2005-06-23 311--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Re: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions
v5: https://lore.kernel.org/all/[email protected]/
Quoted 98 lines · Leonardo Bras Soares Passos
I was not aware Linux supported 32-bit RISC-V.
IIUC, compared to previous code, I am missing a (__typeof__(*(p))) in the output of my code. Should be easily fixed.
Quoted 12 lines · kernel test robot
Hi Leonardo,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v6.5-rc5 next-20230809] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]Quoted 7 lines
url: https://github.com/intel-lab-lkp/linux/commits/Leonardo-Bras/riscv-cmpxchg-Deduplicate-xchg-asm-functions/20230809-101624 base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core patch link: https://lore.kernel.org/r/20230809021311.1390578-3-leobras%40redhat.com patch subject: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions config: riscv-randconfig-r072-20230809 (https://download.01.org/0day-ci/archive/20230810/[email protected]/config) compiler: riscv32-linux-gcc (GCC) 12.3.0 reproduce: (https://download.01.org/0day-ci/archive/20230810/[email protected]/reproduce)For some reason, following this reproduce causes this bug to happen before reproduction:
$ COMPILER_INSTALL_PATH=/root/0day/ COMPILER=gcc-12.3.0 ~/bin/make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=/out/ ARCH=riscv SHELL=/bin/bash net/ipv4/ [...] HOSTCC scripts/selinux/genheaders/genheaders CC scripts/mod/empty.o HOSTCC scripts/mod/mk_elfconfig CC scripts/mod/devicetable-offsets.s CHECK /linux/scripts/mod/empty.c sparse: invalid argument to '-march': '_zicsr_zifencei_zihintpause'Seems like the compiler error.
Anyway, I tested the 64-bit version, seems ok now.
Quoted 53 lines
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/sparse warnings: (new ones prefixed by >>)
WARNING: invalid argument to '-march': '_zihintpause'Quoted 1 line
Quoted 1 line
net/ipv4/tcp_cong.c:300:24: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected structtcp_congestion_opsconst [noderef]__rcu*__new@@ got structtcp_congestion_ops*[assigned] ca @@
net/ipv4/tcp_cong.c:300:24: sparse: expected structtcp_congestion_opsconst [noderef]__rcu*__new
net/ipv4/tcp_cong.c:300:24: sparse: got structtcp_congestion_ops*[assigned] ca
net/ipv4/tcp_cong.c:300:22: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected structtcp_congestion_opsconst *prev @@ got structtcp_congestion_opsconst [noderef]__rcu*[assigned]__ret@@
net/ipv4/tcp_cong.c:300:22: sparse: expected structtcp_congestion_opsconst *prev
net/ipv4/tcp_cong.c:300:22: sparse: got structtcp_congestion_opsconst [noderef]__rcu*[assigned]__retvim +300
net/ipv4/tcp_cong.c317a76f9a44b43 Stephen Hemminger 2005-06-23 281 317a76f9a44b43 Stephen Hemminger 2005-06-23 282 /* Used by sysctl to change default congestion control */ 6670e152447732 Stephen Hemminger 2017-11-14 283 int tcp_set_default_congestion_control(struct net *net, const char *name) 317a76f9a44b43 Stephen Hemminger 2005-06-23 284 { 317a76f9a44b43 Stephen Hemminger 2005-06-23 285 struct tcp_congestion_ops *ca; 6670e152447732 Stephen Hemminger 2017-11-14 286 const struct tcp_congestion_ops *prev; 6670e152447732 Stephen Hemminger 2017-11-14 287 int ret; 317a76f9a44b43 Stephen Hemminger 2005-06-23 288 6670e152447732 Stephen Hemminger 2017-11-14 289 rcu_read_lock(); 6670e152447732 Stephen Hemminger 2017-11-14 290 ca = tcp_ca_find_autoload(net, name); 6670e152447732 Stephen Hemminger 2017-11-14 291 if (!ca) { 6670e152447732 Stephen Hemminger 2017-11-14 292 ret = -ENOENT; 0baf26b0fcd74b Martin KaFai Lau 2020-01-08 293 } else if (!bpf_try_module_get(ca, ca->owner)) { 6670e152447732 Stephen Hemminger 2017-11-14 294 ret = -EBUSY; 8d432592f30fcc Jonathon Reinhart 2021-05-01 295 } else if (!net_eq(net, &init_net) && 8d432592f30fcc Jonathon Reinhart 2021-05-01 296 !(ca->flags & TCP_CONG_NON_RESTRICTED)) { 8d432592f30fcc Jonathon Reinhart 2021-05-01 297 /* Only init netns can set default to a restricted algorithm */ 8d432592f30fcc Jonathon Reinhart 2021-05-01 298 ret = -EPERM; 6670e152447732 Stephen Hemminger 2017-11-14 299 } else { 6670e152447732 Stephen Hemminger 2017-11-14 @300 prev = xchg(&net->ipv4.tcp_congestion_control, ca); 6670e152447732 Stephen Hemminger 2017-11-14 301 if (prev) 0baf26b0fcd74b Martin KaFai Lau 2020-01-08 302 bpf_module_put(prev, prev->owner); 317a76f9a44b43 Stephen Hemminger 2005-06-23 303 6670e152447732 Stephen Hemminger 2017-11-14 304 ca->flags |= TCP_CONG_NON_RESTRICTED; 317a76f9a44b43 Stephen Hemminger 2005-06-23 305 ret = 0; 317a76f9a44b43 Stephen Hemminger 2005-06-23 306 } 6670e152447732 Stephen Hemminger 2017-11-14 307 rcu_read_unlock(); 317a76f9a44b43 Stephen Hemminger 2005-06-23 308 317a76f9a44b43 Stephen Hemminger 2005-06-23 309 return ret; 317a76f9a44b43 Stephen Hemminger 2005-06-23 310 } 317a76f9a44b43 Stephen Hemminger 2005-06-23 311--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
RFC PATCH v4 2/5 riscv/cmpxchg: Deduplicate cmpxchg() asm and macros
In reply to Leonardo BrasIn this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm.
This is usually bad, as it means any change may be done in up to 8 different places.
Unify those versions by creating a new define with enough parameters to generate any version of the previous 8.
Then unify the result under a more general define, and simplify arch_cmpxchg* generation
(This did not cause any change in generated asm)
--- arch/riscv/include/asm/cmpxchg.h | 187 +++++-------------------------- 1 file changed, 29 insertions(+), 158 deletions(-)
arch/riscv/include/asm/cmpxchg.h +29 -158
@@ -71,127 +71,40 @@* store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed (ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) -#define arch_cmpxchg_relaxed (ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) -#define __cmpxchg_acquire (ptr, old, new, size) \ +#define __arch_cmpxchg (lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_acquire (ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=&r" (r), "=&r" (__rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) -#define __cmpxchg_release (ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(*(ptr)) __old = (old); \ __typeof__(*(ptr)) __new = (new); \ __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ + \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \
@@ -199,62 +112,20 @@ __ret; \ }) -#define arch_cmpxchg_release (ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed (ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") -#define __cmpxchg (ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire (ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) + +#define arch_cmpxchg_release (ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "")#define arch_cmpxchg (ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n")#define arch_cmpxchg_local (ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n))#define arch_cmpxchg64 (ptr, o, n) \ ({ \
Signature · 7 lines
2.41.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v4 3/5 riscv/atomic.h : Deduplicate arch_atomic.*
In reply to Leonardo BrasSome functions use mostly the same asm for 32-bit and 64-bit versions.
Make a macro that is generic enough and avoid code duplication.
(This did not cause any change in generated asm)
--- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-)
arch/riscv/include/asm/atomic.h +76 -88
@@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i)#undef ATOMIC_FETCH_OP#undef ATOMIC_OP_RETURN +#define _arch_atomic_fetch_add_unless (_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) +/* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u) {int prev, rc; - __asm__ __volatile__ ( -"0: lr.w %[p], %[c]\n" -" beq %[p], %[u], 1f\n" -" add %[rc], %[p], %[a]\n" -" sc.w.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - :"memory" ); +_arch_atomic_fetch_add_unless (prev, rc, v->counter, a, u,"w" ); +return prev; }#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
@@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_unless(atomic64_t *v, s64 a,s64 prev;long rc; - __asm__ __volatile__ ( -"0: lr.d %[p], %[c]\n" -" beq %[p], %[u], 1f\n" -" add %[rc], %[p], %[a]\n" -" sc.d.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - :"memory" ); +_arch_atomic_fetch_add_unless (prev, rc, v->counter, a, u,"d" ); +return prev; }#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless#endif +#define _arch_atomic_inc_unless_negative (_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) {int prev, rc; - __asm__ __volatile__ ( -"0: lr.w %[p], %[c]\n" -" bltz %[p], 1f\n" -" addi %[rc], %[p], 1\n" -" sc.w.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - :"memory" ); +_arch_atomic_inc_unless_negative (prev, rc, v->counter,"w" ); +return !(prev <0 ); }#define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative +#define _arch_atomic_dec_unless_positive (_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) {int prev, rc; - __asm__ __volatile__ ( -"0: lr.w %[p], %[c]\n" -" bgtz %[p], 1f\n" -" addi %[rc], %[p], -1\n" -" sc.w.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - :"memory" ); +_arch_atomic_dec_unless_positive (prev, rc, v->counter,"w" ); +return !(prev >0 ); }#define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive +#define _arch_atomic_dec_if_positive (_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) {int prev, rc; - __asm__ __volatile__ ( -"0: lr.w %[p], %[c]\n" -" addi %[rc], %[p], -1\n" -" bltz %[rc], 1f\n" -" sc.w.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - :"memory" ); +_arch_atomic_dec_if_positive (prev, rc, v->counter,"w" ); +return prev -1 ; }
@@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_negative(atomic64_t *v)s64 prev;long rc; - __asm__ __volatile__ ( -"0: lr.d %[p], %[c]\n" -" bltz %[p], 1f\n" -" addi %[rc], %[p], 1\n" -" sc.d.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - :"memory" ); +_arch_atomic_inc_unless_negative (prev, rc, v->counter,"d" ); +return !(prev <0 ); }
@@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_positive(atomic64_t *v)s64 prev;long rc; - __asm__ __volatile__ ( -"0: lr.d %[p], %[c]\n" -" bgtz %[p], 1f\n" -" addi %[rc], %[p], -1\n" -" sc.d.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - :"memory" ); +_arch_atomic_dec_unless_positive (prev, rc, v->counter,"d" ); +return !(prev >0 ); }
@@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positive(atomic64_t *v)s64 prev;long rc; - __asm__ __volatile__ ( -"0: lr.d %[p], %[c]\n" -" addi %[rc], %[p], -1\n" -" bltz %[rc], 1f\n" -" sc.d.rl %[rc], %[rc], %[c]\n" -" bnez %[rc], 0b\n" -" fence rw, rw\n" -"1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - :"memory" ); +_arch_atomic_dec_if_positive (prev, rc, v->counter,"d" ); +return prev -1 ; }
Signature · 7 lines
2.41.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v4 4/5 riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2
In reply to Leonardo Brascmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.
Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures.
--- arch/riscv/include/asm/cmpxchg.h | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
arch/riscv/include/asm/cmpxchg.h +34 -0
@@ -72,6 +72,35 @@* indicated by comparing RETURN with OLD. */ +#define __arch_cmpxchg_masked (sc_sfx, prepend, append, r, p, o, n) \ +({ \ + u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ + ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx = (ulong)(n) << __s; \ + ulong __oldx = (ulong)(o) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z5\n" \ + " bne %1, %z3, 1f\n" \ + " and %1, %0, %z6\n" \ + " or %1, %1, %z4\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" ((long)__oldx), "rJ" (__newx), \ + "rJ" (__mask), "rJ" (~__mask) \ + : "memory"); \ + \ + r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ +})#define __arch_cmpxchg (lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \
@@ -98,6 +127,11 @@ __typeof__(*(ptr)) __ret; \ \switch (sizeof (*__ptr)) { \ +case 1 : \ +case 2 : \ +__arch_cmpxchg_masked (sc_sfx, prepend, append, \ + __ret, __ptr, __old, __new); \ +break ; \case 4 : \__arch_cmpxchg (".w" ,".w" sc_sfx, prepend, append, \ __ret, __ptr, (long), __old, __new); \
Signature · 7 lines
2.41.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v4 5/5 riscv/cmpxchg: Implement xchg for variables of size 1 and 2
In reply to Leonardo Brasxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.
Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.
--- arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)
arch/riscv/include/asm/cmpxchg.h +31 -0
@@ -11,6 +11,31 @@#include <asm/barrier.h> #include <asm/fence.h> +#define __arch_xchg_masked (prepend, append, r, p, n) \ +({ \ + u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ + ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx = (ulong)(n) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z4\n" \ + " or %1, %1, %z3\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" (__newx), "rJ" (~__mask) \ + : "memory"); \ + \ + r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) +#define __arch_xchg (sfx, prepend, append, r, p, n) \ ({ \ __asm__ __volatile__ ( \
@@ -27,7 +52,13 @@ __typeof__(*(ptr)) __new = (new); \ __typeof__(*(ptr)) __ret; \__typeof__ (ptr) __ptr = (ptr); \ + \switch (sizeof (*__ptr)) { \ +case 1 : \ +case 2 : \ +__arch_xchg_masked (prepend, append, \ + __ret, __ptr, __new); \ +break ; \case 4 : \__arch_xchg (".w" sfx, prepend, append, \ __ret, __ptr, __new); \
Signature · 7 lines
2.41.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 18 lines · Leonardo Bras
While studying riscv's
cmpxchg.hfile, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.
Also, did the same kind of work on
atomic.c.After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.
Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)
Quoted 6 lines
I did compare the generated asm on a
test.cthat contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".
This is the tree that I used:
https://gitlab.com/LeoBras/linux/-/commits/guo_qspinlock_v11
Quoted 45 lines
Thanks!
LeoChanges since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from_maskto_maskedfor patches 4 & 5
-__rcvariable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/Changes since squashed cmpxchg RFCv1:
- Unified withatomic.cpatchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/Leonardo Bras (5):
riscv/cmpxchg: Deduplicatexchg()asm functions riscv/cmpxchg: Deduplicatecmpxchg()asm and macrosriscv/atomic.h: Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2
Quoted 7 lines
arch/riscv/include/asm/atomic.h | 164 ++++++------- arch/riscv/include/asm/cmpxchg.h | 394 ++++++++++--------------------- 2 files changed, 195 insertions(+), 363 deletions(-)--
2.41.0
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv