riscv/barrier: Define __smp_{store_release,load_acquire}
PATCH 1/1 riscv/barrier: Define __smp_{store_release,load_acquire}
Introduce __smp_{store_release,load_acquire}, and rely on the generic definitions for smp_{store_release,load_acquire}. This avoids the use of full ("rw,rw") fences on SMP.
--- arch/riscv/include/asm/barrier.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
arch/riscv/include/asm/barrier.h +15 -0
@@ -38,6 +38,21 @@#define __smp_rmb () RISCV_FENCE(r,r)#define __smp_wmb () RISCV_FENCE(w,w) +#define __smp_store_release (p, v) \ +do { \ + compiletime_assert_atomic_type(*p); \ + RISCV_FENCE(rw,w); \ + WRITE_ONCE(*p, v); \ +} while (0) + +#define __smp_load_acquire (p) \ +({ \ + typeof(*p) ___p1 = READ_ONCE(*p); \ + compiletime_assert_atomic_type(*p); \ + RISCV_FENCE(r,rw); \ + ___p1; \ +}) +/* * This is a very specific barrier: it's currently only used in two places in * the kernel, both in the scheduler. See include/linux/spinlock.h for the two
Signature · 1 line
2.7.4
Quoted 25 lines · parri.andrea at gmail.com
Introduce __smp_{store_release,load_acquire}, and rely on the generic definitions for smp_{store_release,load_acquire}. This avoids the use of full ("rw,rw") fences on SMP.
Signed-off-by Andrea Parri <[email protected]>--- arch/riscv/include/asm/barrier.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)arch/riscv/include/asm/barrier.h +8
-0@@ -38,6 +38,21 @@#define __smp_rmb () RISCV_FENCE(r,r)#define __smp_wmb () RISCV_FENCE(w,w) +#define __smp_store_release (p, v) \ +do { \ + compiletime_assert_atomic_type(*p); \ + RISCV_FENCE(rw,w); \ + WRITE_ONCE(*p, v); \ +} while (0) + +#define __smp_load_acquire (p) \
arch/riscv/include/asm/barrier.h line 58
Quoted 10 lines
arch/riscv/include/asm/barrier.h +7
-0+({ \ + typeof(*p) ___p1 = READ_ONCE(*p); \ +compiletime_assert_atomic_type (*p); \ +RISCV_FENCE (r,rw); \ + ___p1; \ +}) +/* * This is a very specific barrier: it's currently only used in two places in * the kernel, both in the scheduler. See include/linux/spinlock.h for the two
I'm adding Daniel just in case I misunderstood what's going on here, but these look good to me. As this is a non-trivial memory model change I'm going to let it bake in linux-next for a bit just so it gets some visibility.
Thanks!
Quoted 33 lines · Palmer Dabbelt
Quoted 32 lines · parri.andrea at gmail.com
Introduce __smp_{store_release,load_acquire}, and rely on the generic definitions for smp_{store_release,load_acquire}. This avoids the use of full ("rw,rw") fences on SMP.
Signed-off-by Andrea Parri <[email protected]>---
?arch/riscv/include/asm/barrier.h | 15 +++++++++++++++ ?1 file changed, 15 insertions(+)arch/riscv/include/asm/barrier.h +15
-0@@ -38,6 +38,21 @@ ?#define __smp_rmb()??? RISCV_FENCE(r,r) ?#define __smp_wmb()??? RISCV_FENCE(w,w) +#define __smp_store_release (p, v)??????????????????? \ +do {??????????????????????????????????? \ +??? compiletime_assert_atomic_type(*p);??????????????? \ +??? RISCV_FENCE(rw,w);??????????????????????? \ +??? WRITE_ONCE(*p, v);??????????????????????? \ +} while (0) + +#define __smp_load_acquire (p)??????????????????????? \ +({??????????????????????????????????? \ +??? typeof(*p) ___p1 = READ_ONCE(*p);??????????????? \ +??? compiletime_assert_atomic_type(*p);??????????????? \ +??? RISCV_FENCE(r,rw);??????????????????????? \ +??? ___p1;??????????????????????????????? \ +}) +
Quoted 8 lines
Quoted 3 lines
?/*
? * This is a very specific barrier: it's currently only used in two places in ? * the kernel, both in the scheduler.? Seeinclude/linux/spinlock.hfor the twoI'm adding Daniel just in case I misunderstood what's going on here, but these look good to me. As this is a non-trivial memory model change I'm going to let it bake in linux-next for a bit just so it gets some visibility.
Looks good to me too. In particular, it also covers the Write->release(p)->acquire(p)->Write ordering that we were debating in the broader LKMM thread, which is good.
Dan
Quoted 2 lines
Thanks
Quoted 43 lines · Daniel Lustig
Quoted 33 lines · Palmer Dabbelt
Quoted 32 lines · parri.andrea at gmail.com
Introduce __smp_{store_release,load_acquire}, and rely on the generic definitions for smp_{store_release,load_acquire}. This avoids the use of full ("rw,rw") fences on SMP.
Signed-off-by Andrea Parri <[email protected]>---
?arch/riscv/include/asm/barrier.h | 15 +++++++++++++++ ?1 file changed, 15 insertions(+)arch/riscv/include/asm/barrier.h +15
-0@@ -38,6 +38,21 @@ ?#define __smp_rmb()??? RISCV_FENCE(r,r) ?#define __smp_wmb()??? RISCV_FENCE(w,w) +#define __smp_store_release (p, v)??????????????????? \ +do {??????????????????????????????????? \ +??? compiletime_assert_atomic_type(*p);??????????????? \ +??? RISCV_FENCE(rw,w);??????????????????????? \ +??? WRITE_ONCE(*p, v);??????????????????????? \ +} while (0) + +#define __smp_load_acquire (p)??????????????????????? \ +({??????????????????????????????????? \ +??? typeof(*p) ___p1 = READ_ONCE(*p);??????????????? \ +??? compiletime_assert_atomic_type(*p);??????????????? \ +??? RISCV_FENCE(r,rw);??????????????????????? \ +??? ___p1;??????????????????????????????? \ +}) +Quoted 8 lines
Quoted 3 lines
?/*
? * This is a very specific barrier: it's currently only used in two places in ? * the kernel, both in the scheduler.? Seeinclude/linux/spinlock.hfor the twoI'm adding Daniel just in case I misunderstood what's going on here, but these look good to me. As this is a non-trivial memory model change I'm going to let it bake in linux-next for a bit just so it gets some visibility.
Quoted 3 lines
Looks good to me too. In particular, it also covers the Write->release(p)->acquire(p)->Write ordering that we were debating in the broader LKMM thread, which is good.
Yeah, I think that other changes would be required to completely cover the issues debated in that thread: I plan to prepare and to post a new series/RFC to address those (unless someone precedes me of course ;-).
Andrea
Quoted 5 lines
Dan
Quoted 2 lines
Thanks