x86/spinlocks optional for 3.1
Subsystems (2)
Hi Linus,
Sorry for this late pull; testing on this got delayed due to my and Ingo's scheduling and the desire to make sure that this would not cause a performance regression.
At this time we have good confidence in it, but if you feel this is too late feel free to drop it and we'll do it for 3.2.
-hpa
The following changes since commit 02f8c6aee8df3cdc935e9bdd4f2d020306035dbe:
Linux 3.0 (2011-07-21 19:17:23 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git x86-spinlocks-for-linus
H. Peter Anvin (1):
x86, ticketlock: Use asm volatile for __ticket_unlock_release()
Jeremy Fitzhardinge (8):
x86, ticketlock: Clean up types and accessors
x86, ticketlock: Convert spin loop to C
x86, ticketlock: Use C for __ticket_spin_unlock
x86, ticketlock: Make large and small ticket versions of spin_lock the same x86, ticketlock: Make __ticket_spin_lock common
x86, ticketlock: Make __ticket_spin_trylock common
x86: Add xadd helper macro
x86, ticketlock: Use xadd helper
arch/x86/include/asm/cmpxchg_32.h | 21 +++++ arch/x86/include/asm/cmpxchg_64.h | 26 ++++++ arch/x86/include/asm/spinlock.h | 140 +++++++++++---------------------- arch/x86/include/asm/spinlock_types.h | 22 +++++- 4 files changed, 114 insertions(+), 95 deletions(-)
arch/x86/include/asm/cmpxchg_32.h +21 -0
@@ -280,4 +280,25 @@ static inline unsigned long cmpxchg_386(volatile void *ptr, unsigned long old,#endif +#define xadd (ptr, inc) \ + do { \ + switch (sizeof(*(ptr))) { \ + case 1: \ + asm volatile (LOCK_PREFIX "xaddb %b0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + case 2: \ + asm volatile (LOCK_PREFIX "xaddw %w0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + case 4: \ + asm volatile (LOCK_PREFIX "xaddl %0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + } \ + } while(0) +#endif /* _ASM_X86_CMPXCHG_32_H */
arch/x86/include/asm/cmpxchg_64.h +26 -0
@@ -151,4 +151,30 @@ extern void __cmpxchg_wrong_size(void);cmpxchg_local ((ptr), (o), (n)); \ }) +#define xadd (ptr, inc) \ + do { \ + switch (sizeof(*(ptr))) { \ + case 1: \ + asm volatile (LOCK_PREFIX "xaddb %b0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + case 2: \ + asm volatile (LOCK_PREFIX "xaddw %w0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + case 4: \ + asm volatile (LOCK_PREFIX "xaddl %0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + case 8: \ + asm volatile (LOCK_PREFIX "xaddq %q0, %1\n" \ + : "+r" (inc), "+m" (*(ptr)) \ + : : "memory", "cc"); \ + break; \ + } \ + } while(0) +#endif /* _ASM_X86_CMPXCHG_64_H */
arch/x86/include/asm/spinlock.h +47 -93
177 lines not on this page · raw message
arch/x86/include/asm/spinlock_types.h +20 -2
@@ -5,11 +5,29 @@# error "please don't include this file directly"#endif +#include <linux/types.h> + +#if (CONFIG_NR_CPUS <256 ) +typedef u8 __ticket_t ; +typedef u16 __ticketpair_t ; +#else +typedef u16 __ticket_t ; +typedef u32 __ticketpair_t ; +#endif + +#define TICKET_SHIFT (sizeof(__ticket_t) * 8) +#define TICKET_MASK ((__ticket_t)((1 << TICKET_SHIFT) - 1)) +typedef struct arch_spinlock { -unsigned int slock; +union { +__ticketpair_t head_tail; +struct __raw_tickets { +__ticket_t head, tail; + } tickets; + }; }arch_spinlock_t ; -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } +#define __ARCH_SPIN_LOCK_UNLOCKED { { 0 } } typedef struct { unsigned int lock;
Quoted 7 lines · H. Peter Anvin
Sorry for this late pull; testing on this got delayed due to my and Ingo's scheduling and the desire to make sure that this would not cause a performance regression.
At this time we have good confidence in it, but if you feel this is too late feel free to drop it and we'll do it for 3.2.
Hmm. I don't mind it per se, but I hate how this patch that is meant to combine the inline asm versions into one C version actually creates *more* lines, and creates *more* inline asm differences between x86-32/64 due to the xadd.
So quite frankly, I think the patch in this form is totally self-defeating. I'm not pulling something that is supposed to clean things up, but just adds more ugliness in some other place.
If that xadd implementation could perhaps be shared, I wouldn't hate it so much. As it is, I really don't see the point in pulling this.
Linus
H. Peter Anvin
reviewerQuoted 4 lines · Linus Torvalds
If that xadd implementation could perhaps be shared, I wouldn't hate it so much. As it is, I really don't see the point in pulling this.
OK, no problem. We'll rework it for 3.2.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · Linus Torvalds
If that xadd implementation could perhaps be shared, I wouldn't hate it so much. As it is, I really don't see the point in pulling this.
OK, no problem. We'll rework it for 3.2.
Agreed. The xadd helper is actually identical between the two, aside from the 64-bit form. I was following the form of cmpchg_32/64.h, but to be honest I don't understand why they're separate. Is it some remaining TBD unification?
J