sparc64 cmpxchg is not a full memory barrier anymore ?
Hi David,
I took a look at the current sparc64 cmpxchg implementation in the Linux kernel and stumbled on this commit:
commit 293666b7a17cb7a389fc274980439212386a19c4
Author: David S. Miller <[email protected]> Date: Sat Nov 15 13:33:25 2008 -0800
sparc64: Stop using memory barriers for atomics and locks.
The kernel always executes in the TSO memory model now, so none of this stuff is necessary any more.
With helpful feedback from Nick Piggin.
Reading p. 152 A.9 Compare and Swap, I see that the cas instruction does not imply a memory barrier.
Reading http://docs.sun.com/app/docs/doc/801-6678/6i11oelck?a=view
I see that:
"TSO guarantees that the store, FLUSH, and atomic load-store instructions of all processors appear to be executed by memory serially in a single order called the memory order. Furthermore, the sequence of store, FLUSH, and atomic load-store instructions in the memory order for a given processor is identical to the sequence in which they were issued by the processor."
So it provides no guarantee whatsoever wrt loads. However, reading Documentation/memory-barriers.txt:
"Whilst they are technically interprocessor interaction considerations, atomic operations are noted specially as some of them imply full memory barriers and some don't, but they're very heavily relied on as a group throughout the kernel." -> cmpxchg is part of them.
The same applies to the other atomic instructions we find in this list. How is the correct ordering of loads wrt to cmxchg (and other atomic ops) still ensured by this modification?
Thanks,
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Sparc64 support added to Userspace RCU
By the way, I just added sparc64 support to the Userspace RCU library, available at: git://lttng.org/userspace-rcu.git
Interesting files concerning sparc64 are:
urcu/arch_sparc64.hurcu/uatomic_arch_sparc64.h
Direct links:
http://www.lttng.org/cgi-bin/gitweb.cgi?p=userspace-rcu.git;a=blob;f=urcu/arch_sparc64.h;hb=HEAD
http://www.lttng.org/cgi-bin/gitweb.cgi?p=userspace-rcu.git;a=blob;f=urcu/uatomic_arch_sparc64.h;hb=HEAD
Please note that this code targets user-space, not kernel-space.
Feedback is welcome,
Thanks,
Mathieu
Quoted 51 lines · * Mathieu Desnoyers ([email protected]) wrote:
Hi David,
I took a look at the current sparc64 cmpxchg implementation in the Linux kernel and stumbled on this commit:
commit
293666b7a17cb7a389fc274980439212386a19c4Author: David S. Miller <[email protected]> Date: Sat Nov 15 13:33:25 2008 -0800sparc64: Stop using memory barriers for atomics and locks.
The kernel always executes in the TSO memory model now, so none of this stuff is necessary any more.
With helpful feedback from Nick Piggin.
Signed-off-by David S. Miller <[email protected]>Reading p. 152 A.9 Compare and Swap, I see that the cas instruction does not imply a memory barrier.
Reading http://docs.sun.com/app/docs/doc/801-6678/6i11oelck?a=view
I see that:
"TSO guarantees that the store, FLUSH, and atomic load-store instructions of all processors appear to be executed by memory serially in a single order called the memory order. Furthermore, the sequence of store, FLUSH, and atomic load-store instructions in the memory order for a given processor is identical to the sequence in which they were issued by the processor."
So it provides no guarantee whatsoever wrt loads. However, reading
Documentation/memory-barriers.txt:"Whilst they are technically interprocessor interaction considerations, atomic operations are noted specially as some of them imply full memory barriers and some don't, but they're very heavily relied on as a group throughout the kernel." -> cmpxchg is part of them.
The same applies to the other atomic instructions we find in this list. How is the correct ordering of loads wrt to cmxchg (and other atomic ops) still ensured by this modification?
Thanks,
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Re: Sparc64 support added to Userspace RCU
From: Mathieu Desnoyers <[email protected]>
Date: Thu, 22 Oct 2009 15:57:53 -0400
Quoted 1 line
Feedback is welcome,
__sparc_v8__ does not mean 64-bit longs or pointers as implied here:
#ifndef __SIZEOF_LONG__
#if (defined(__sparc_v8__) || defined(__sparc_v9__)) #define __SIZEOF_LONG__ 8
#else
#define __SIZEOF_LONG__ 4
#endif
#endif
"v8" was the last 32-bit standard of the cpu architecture.
Otherwise looks fine.
Although this port won't be very useful until it can be built in 32-bit mode. As most applications on sparc64 systems are still 32-bit. Like on powerpc, people only rarely build things 64-bit on sparc64.
You need to build with "-Wa,-Av9a" on the GCC command line for that case so that the assembler allows the v9 instructions emitted by your macros, even when building for 32-bit.
Hope this helps.
Re: Sparc64 support added to Userspace RCU
Quoted 24 lines · * David Miller ([email protected]) wrote:
From: Mathieu Desnoyers <[email protected]>
Date: Thu, 22 Oct 2009 15:57:53 -0400Quoted 1 line
Feedback is welcome,
__sparc_v8__does not mean 64-bit longs or pointers as implied here:#ifndef
__SIZEOF_LONG__
#if (defined(__sparc_v8__)||defined(__sparc_v9__)) #define__SIZEOF_LONG__8
#else
#define__SIZEOF_LONG__4
#endif
#endif"v8" was the last 32-bit standard of the cpu architecture.
Otherwise looks fine.
Although this port won't be very useful until it can be built in 32-bit mode. As most applications on sparc64 systems are still 32-bit. Like on powerpc, people only rarely build things 64-bit on sparc64.
Quoted 5 lines
You need to build with "-Wa,-Av9a" on the GCC command line for that case so that the assembler allows the v9 instructions emitted by your macros, even when building for 32-bit.
Hope this helps.
Yes ! Thanks a lot for this information. I'll push the following fix in my tree.
Mathieu
README +4 -1
@@ -18,11 +18,14 @@ BUILDING Forcing a 32-bit build with 386 backward compatibility: * CFLAGS="-m32 -g -O2" ./configure --host=i386-pc-linux-gnu + Forcing a 32-bit build for Sparcv9 (typical for Sparc v9) + * CFLAGS="-m32 -Wa,-Av9a -g -O2" ./configure + ARCHITECTURES SUPPORTED ----------------------- Currently, x86 (i386, i486, i586, i686), x86 64-bit, PowerPC 32/64, S390, S390x -and Sparc64 are supported. Only tested on Linux so far, but should +and Sparcv9 32/64 are supported. Only tested on Linux so far, but should theoretically work on other operating systems. QUICK START GUIDE
urcu/uatomic_arch_sparc64.h +1 -1
@@ -24,7 +24,7 @@#include <urcu/system.h> #ifndef __SIZEOF_LONG__ -#if (defined(__sparc_v8__) || defined(__sparc_v9__)) +#ifdef __LP64__#define __SIZEOF_LONG__ 8#else #define __SIZEOF_LONG__ 4
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
From: Mathieu Desnoyers <[email protected]>
Date: Thu, 22 Oct 2009 14:32:42 -0400
Quoted 3 lines
The same applies to the other atomic instructions we find in this list. How is the correct ordering of loads wrt to cmxchg (and other atomic ops) still ensured by this modification?
All actual sparc64 chips implement more strict ordering than the V9 specification permits. The memory barriers were just nops and actually not doing anything more than the chip already does for us.
Quoted 11 lines · * David Miller ([email protected]) wrote:
From: Mathieu Desnoyers <[email protected]>
Date: Thu, 22 Oct 2009 14:32:42 -0400Quoted 3 lines
The same applies to the other atomic instructions we find in this list. How is the correct ordering of loads wrt to cmxchg (and other atomic ops) still ensured by this modification?
All actual sparc64 chips implement more strict ordering than the V9 specification permits. The memory barriers were just nops and actually not doing anything more than the chip already does for us.
OK. Perhaps adding a comment to that effect near sparc mb() implementation would be appropriate ?
Thanks,
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
From: Mathieu Desnoyers <[email protected]>
Date: Fri, 23 Oct 2009 08:33:27 -0400
Quoted 2 lines
OK. Perhaps adding a comment to that effect near sparc
mb()implementation would be appropriate ?
Sure, I'll take care of that when I get back from Japan.