Skip to contentSkip to conversation

64-bit futexes: x86 support

PATCH In review Ulrich Drepper 4 participants 10 messages May 31, 2008 – Jun 2, 2008 x86

lore
Share & tools
lore
#0 lore raw

PATCH 3/3 64-bit futexes: x86 support

This patch adds 64-bit futex support for x86. As explained in the introduction, this requires a new system call. The remaining changes are similarly to the x86-64 changes: enable support for FUTEX_WAKE_OP.

[Sorry Andrew, but syscalls unfortunately have to be added at the end. So you'll have to live with the conflicts.]

 arch/x86/ia32/ia32entry.S          |    1 
 arch/x86/ia32/sys_ia32.c           |   46 ++++++++++++++++++++++++++++++
 arch/x86/kernel/sys_i386_32.c      |   48 ++++++++++++++++++++++++++++++++
 arch/x86/kernel/syscall_table_32.S |    1 
 include/asm-x86/futex.h            |   55 +++++++++++++++++++++++++++++++++++--
 include/asm-x86/types.h            |    2 -
 include/asm-x86/unistd_32.h        |    1 
 scripts/checksyscalls.sh           |    1 
 8 files changed, 151 insertions(+), 4 deletions(-)
arch/x86/ia32/ia32entry.S +1 -0
index b5e329d..776b6a2 100644
@@ -731,4 +731,5 @@ ia32_sys_call_table:
 	.quad sys32_fallocate
 	.quad compat_sys_timerfd_settime	/* 325 */
 	.quad compat_sys_timerfd_gettime
+	.quad sys32_futex64
 ia32_syscall_end:
arch/x86/ia32/sys_ia32.c +46 -0

55 lines not on this page · raw message

arch/x86/kernel/sys_i386_32.c +48 -0

51 lines not on this page · raw message

arch/x86/kernel/syscall_table_32.S +1 -0
index adff556..5b1b5d9 100644
@@ -326,3 +326,4 @@ ENTRY(sys_call_table)
 	.long sys_fallocate
 	.long sys_timerfd_settime	/* 325 */
 	.long sys_timerfd_gettime
+	.long sys_futex64
include/asm-x86/unistd_32.h +1 -0
index 8317d94..8be8e19 100644
@@ -332,6 +332,7 @@
 #define __NR_fallocate		324
 #define __NR_timerfd_settime	325
 #define __NR_timerfd_gettime	326
+#define __NR_futex64		327
 
 #ifdef __KERNEL__
 
scripts/checksyscalls.sh +1 -0
index 366f8c7..a03b34c 100755
@@ -98,6 +98,7 @@ cat << EOF
 #define __IGNORE_setgid32
 #define __IGNORE_setfsuid32
 #define __IGNORE_setfsgid32
+#define __IGNORE_futex64
 
 /* sync_file_range had a stupid ABI. Allow sync_file_range2 instead */
 #ifdef __NR_sync_file_range2
include/asm-x86/types.h +0 -2
@@ -30,10 +30,8 @@
 typedef u32 dma_addr_t;
 #endif
 
-#ifdef CONFIG_X86_64
 typedef u64 futex_val_t;
 #define __have_futex_val_t
-#endif
 
 #endif /* __ASSEMBLY__ */
 #endif /* __KERNEL__ */
include/asm-x86/futex.h +53 -2

73 lines not on this page · raw message

#1 lore raw

include/asm-x86/futex.h static inline int futex_atomic_op_inuser · line 132

Quoted 7 lines · Ulrich Drepper
include/asm-x86/futex.h +6 -0
+#ifdef CONFIG_X86_32
+	/* Not all 32-bit machines support 8-byte cmpxchg.  We just
+	   unconditionally perform a runtime check for the feature.  */
+	if ((op & FUTEX_FLAG_64) && !boot_cpu_has(X86_FEATURE_CX8))
+		return -ENOSYS;
+#endif

Also, doesn't this mean that effectively you have given up on any CPU before the Pentium entirely?

Not that I want to take the patches _anyway_, but it seems doubly stupid. Those things are going to be UP machines, so you could have just emulated it with no correctness issues by disabling preemption.

Linus

#2 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 2 lines · Linus Torvalds

Also, doesn't this mean that effectively you have given up on any CPU before the Pentium entirely?

Yes, but only for the speed-up. Not for the functionality. All programs will work just fine.

Quoted 3 lines

Not that I want to take the patches _anyway_, but it seems doubly stupid. Those things are going to be UP machines, so you could have just emulated it with no correctness issues by disabling preemption.

First, you're thinking too much in kernel terms. At userlevel I cannot disable interrupts etc. I cannot read 64 bits atomically on those processors. Second, the old implementation is not significantly slower in normal programs on UP machines. It's really not a big issue.

In any case, robbing 64-bit architectures of the possible performance gain just because there are ancient versions which cannot do it is just plain stupid. I know you cannot suggest that.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkhAw5QACgkQ2ijCOnn/RHTRRwCgv37woIjA5RJji4XamBQVqCyF VMwAoIauSEdAlfK4pBJhMkIKirwEBw0z
=rDJc
-----END PGP SIGNATURE-----

#3 lore raw
Quoted 4 lines · Ulrich Drepper

In any case, robbing 64-bit architectures of the possible performance gain just because there are ancient versions which cannot do it is just plain stupid. I know you cannot suggest that.

I'm suggesting that there are better approaches than forcing yet another new interface onto this thing. As mentioned, the kernel has had 32-bit rwsemaphores for a long time. Yes, there is "extra information" outside the 32-bit field, but I suspect you simply haven't looked hard enough at just using the existing futex code.

You certainly didn't even read - or understand - my emails, since you thought I was talking about regular semaphores and ignored the "rw" part entirely (not to mention the fact that I talked about reader counts etc, which is why I think you didn't even read the email).

Linus

#4 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 5 lines · Linus Torvalds

I'm suggesting that there are better approaches than forcing yet another new interface onto this thing. As mentioned, the kernel has had 32-bit rwsemaphores for a long time. Yes, there is "extra information" outside the 32-bit field, but I suspect you simply haven't looked hard enough at just using the existing futex code.

I guess have to expect to expect to be insulted by you. What else is new? I think it's safe to say that nobody has looked more deeply at the implementation possibilities with futexes. I don't know how many different primitives I've written over the years. If you think you can do it better, well, that's then a challenge to you. Come up with something better.

The rwlocks cannot be handled as efficiently with 32-bit futexes.

I stand by that. Proof me wrong. You don't have to write code, just describe it. But don't publish it if

- - the uncontended reader *unconditionally* uses more than one memory access (atomic of course)

- - the contended readers uses the one access above plus one or two compare-and-exchange depending on the rwlock preferring readers or writers. The last compare-and-exchange has to be repeated for spurious wakeups which are highly unlikely since we only wake what is ready to use.

- - the uncontended writer needs more than one single compare-and-exchange plus one memory read plus one write (both not atomic)

- - the contended writer needs more than the above plus one additional compare-and-exchange for every time we delay in futex_wait (again, highly unlikely to have spurious wakeups)

- - the unlock operation needs more than one read plus either one (repeated) compare-and-exchange or another atomic operation plus one memory write for writer locks

- - you can handle timedlock variants which don't manage to get the lock efficiently without waking up anyone else

That's it.

And just to point out some more fun: in a highly contended case this is a performance killer (MESI protocol states):

               core #1                           core #2

         instr     cache state               instr    cache state

         load      -> S

         store     -> M

                   -> S                       load    -> S

                                              store   -> M

I.e., of the cache line with the rwlock is in one core's cache and you first load a value from that cache line just to modify it right after that, you're paying a terrible price.

Have fun coming up with something which is as good or better. But shall we say there is a deadline? Otherwise we'd wait too long.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkhA0zUACgkQ2ijCOnn/RHRI2QCbBGCcjES6qMJYay1WRVLdSAAk lHEAnA754SGDsjuzvCYg2ZBrEUIktxYV
=NYYc
-----END PGP SIGNATURE-----

#5 lore raw
Quoted 2 lines · Ulrich Drepper

I guess have to expect to expect to be insulted by you.

Guess why? What comes around goes around.

You just dismissed (without apparently even understanding them) my concerns about a patch that comes out of the blue, when I ask whether you really looked into trying to do it other ways.

You didn't even post the code that would _use_ this. You just blindly claim that you're the only one who can know things like this and can do serialization, and that your way is the only approach.

Color me not impressed.

I did not insult you until you didn't even understand my first email. We have had an efficient 32-bit rwlock in the kernel for a long time. So I (quite naturally) questioned your statement (that had no backing up what-so-ever) that it cannot be done.

Sure, futex use and user space is not the same as the kernel code. It's possible that you really do need 64-bit futexes. But quite frankly, the only thing you have proven so far is that you're an ass.

Linus

#6 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 3 lines · Linus Torvalds

You just blindly
claim that you're the only one who can know things like this and can do serialization, and that your way is the only approach.

Nonsense. As you wrote yourself, you're only guessing about your approaches. You didn't check what we currently do (which is the extra data stored protected by locks) and you suggest this scheme.

Then you suggest hand-wavingly other schemes and I explain why they cannot work as efficiently. I never doubted that they cannot work at all. If performance wouldn't be important I would not have to change the current implementation which works semantically just fine.

Quoted 3 lines

Sure, futex use and user space is not the same as the kernel code. It's possible that you really do need 64-bit futexes. But quite frankly, the only thing you have proven so far is that you're an ass.

Well, I don't know what you're reading into what I wrote. I've only explained what is wrong with each of your proposals and arguments. If pointing out the flaws makes me an ass then so be it.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkhA2AcACgkQ2ijCOnn/RHTiPQCgzWqXP1EW8HID9FuzbgQa70fL THsAoLJvUKJO+hqFM3Hd3cHlOCIKhRoF
=ab2b
-----END PGP SIGNATURE-----

#7 lore raw

From: Ulrich Drepper <[email protected]>
Date: Fri, 30 May 2008 21:45:59 -0700

Quoted 9 lines

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 3 lines · Linus Torvalds

You just blindly
claim that you're the only one who can know things like this and can do serialization, and that your way is the only approach.

Nonsense.

No, Ulrich. Eveyone who ever tries to work with you, and comes away frustrated, has this exact impression.

Whether it's intentional or not, that doesn't matter, people perceive you as an arrogant know it all who is extremely difficult to work with. And you can be sure that projects that you are the primary maintainer of, such as GLIBC, suffer because of it.

#8 lore raw
In reply to Ulrich Drepper
Quoted 3 lines · Ulrich Drepper

This patch adds 64-bit futex support for x86. As explained in the introduction, this requires a new system call. The remaining changes are similarly to the x86-64 changes: enable support for FUTEX_WAKE_OP.

great work - the first two patches look good and we've created a new -tip topic branch for it, tip/core/futex-64bit.

but this 32-bit x86 patch has problems, it needed 3 fixes to even build (see below), then it stopped with various failures related to the assembly constraints:

kernel/futex.c: In function 'do_futex':
include/asm/futex.h:155: error: can't find a register in class 'GENERAL_REGS' while reloading 'asm' [...]

Ingo

-------------------->
Subject: 64 bit futexes: fixes
From: Ingo Molnar <[email protected]>
Date: Mon Jun 02 10:07:08 CEST 2008

- various build fixes on 32-bit.

---
 arch/x86/kernel/sys_i386_32.c |    1 +
 include/asm-x86/futex.h       |    4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/kernel/sys_i386_32.c +1 -0
Index: linux/arch/x86/kernel/sys_i386_32.c
===================================================================
@@ -18,6 +18,7 @@
 #include <linux/file.h>
 #include <linux/utsname.h>
 #include <linux/ipc.h>
+#include <linux/futex.h>
 
 #include <asm/uaccess.h>
 #include <asm/unistd.h>
include/asm-x86/futex.h +2 -2
Index: linux/include/asm-x86/futex.h
===================================================================
@@ -65,7 +65,7 @@
 		     _ASM_EXTABLE(1b, 5b)			\
 		     _ASM_EXTABLE(2b, 5b)			\
 		     _ASM_EXTABLE(3b, 5b)			\
-		     : "=&a" (oldvaltemp1), "=&d" (oldvaltmep2),\
+		     : "=&a" (oldvaltemp1), "=&d" (oldvaltemp2),\
 		       "=&r" (ret),				\
 		       "+m" (((u32 __user *) uaddr)[0]),	\
 		       "+m" (((u32 __user *) uaddr)[1]),	\
@@ -127,7 +127,7 @@ static inline int futex_atomic_op_inuser
 #ifdef CONFIG_X86_32
 	/* Not all 32-bit machines support 8-byte cmpxchg.  We just
 	   unconditionally perform a runtime check for the feature.  */
-	if ((op & FUTEX_FLAG_64) && !boot_cpu_has(X86_FEATURE_CX8))
+	if ((op & FUTEX_64_FLAG) && !boot_cpu_has(X86_FEATURE_CX8))
 		return -ENOSYS;
 #endif
 
#9 lore raw
Quoted 7 lines · Ingo Molnar
Quoted 4 lines

This patch adds 64-bit futex support for x86. As explained in the introduction, this requires a new system call. The remaining changes are similarly to the x86-64 changes: enable support for FUTEX_WAKE_OP.

great work - the first two patches look good and we've created a new -tip topic branch for it, tip/core/futex-64bit.

the topic is not yet offered for linux-next though, until the API discussion with Linus is resolved one way or another :-)

Ingo