Skip to contentSkip to conversation

Fix the atomic compare and swap function on hppa

PATCH Aurelien Jarno 4 participants 7 messages May 20, 2007 – May 23, 2007 9 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 11 lines · Aurelien Jarno

Hi,

I have finally tracked down the "error while loading shared libraries:
internal error: symidx out of range of fptr table" problem.

It comes from a bug in atomic.h, the errno number is compared with the wrong value. The kernel returns -EAGAIN (ie -11), but the value in the cmpib instruction is EAGAIN (11).

The patch below fixes the problem.

Oops this patch is reversed.

Please find a new version below.

2007-05-20 Aurelien Jarno <[email protected]>

* sysdeps/unix/sysv/linux/hppa/bits/atomic.h: check for -EAGAIN instead of EAGAIN.

sysdeps/unix/sysv/linux/hppa/bits/atomic.h renamed from sysdeps/unix/sysv/linux/hppa/bits/atomic.h.orig +1 -1
@@ -56,7 +56,7 @@
 #define LWS_CAS "0"
 /* Note r31 is the link register */
 #define LWS_CLOBBER "r1", "r26", "r25", "r24", "r23", "r22", "r21", "r20", "r28", "r31", "memory"
-#define ASM_EAGAIN "11" 
+#define ASM_EAGAIN "-11" 
 
 #if __ASSUME_LWS_CAS
 /* The only basic operation needed is compare and exchange.  */
Signature · 8 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Re: [parisc-linux] Re: [PATCH] Fix the atomic compare and swap function

Quoted 2 lines

Review

Quoted 2 lines
(quoted diff) +1 -1
-#define ASM_EAGAIN "11"
+#define ASM_EAGAIN "-11"

If the check is wrong, I would change the compare instead of the define. Error numbers are normally defined as positive values.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

Re: [parisc-linux] Re: [PATCH] Fix the atomic compare and swap function

Quoted 6 lines · John David Anglin
Quoted 2 lines

Review

Quoted 2 lines
(quoted diff) +1 -1
-#define ASM_EAGAIN "11"
+#define ASM_EAGAIN "-11"

If the check is wrong, I would change the compare instead of the define. Error numbers are normally defined as positive values.

Then, what about this patch?

sysdeps/unix/sysv/linux/hppa/bits/atomic.h renamed from sysdeps/unix/sysv/linux/hppa/bits/atomic.h.orig +1 -1
@@ -71,10 +71,10 @@
 	"copy	%5, %%r24			\n\t"			\
 	"ble	" LWS "(%%sr2, %%r0)		\n\t"			\
 	"ldi	" LWS_CAS ", %%r20		\n\t"			\
+        "sub	%%r0, %%r21, %%r21		\n\t"			\
 	"cmpib,=,n " ASM_EAGAIN ",%%r21,0b	\n\t"			\
 	"nop					\n\t"			\
 	"stw	%%r28, %0			\n\t"			\
-        "sub	%%r0, %%r21, %%r21		\n\t"			\
 	"stw	%%r21, %1			\n\t"			\
 	: "=m" (lws_ret), "=m" (lws_errno), "=m" (*mem)			\
         : "r" (mem), "r" (oldval), "r" (newval)				\
Signature · 8 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#7 lore raw
In reply to a message not in the archive
Quoted 23 lines · Carlos O'Donell
Quoted 8 lines · Aurelien Jarno

2007-05-20 Aurelien Jarno <[email protected]>

* sysdeps/unix/sysv/linux/hppa/bits/atomic.h: check for -EAGAIN instead of EAGAIN.

sysdeps/unix/sysv/linux/hppa/bits/atomic.h 2007-05-20 03:24:41.000000000 +0200 line 62

Quoted 8 lines
sysdeps/unix/sysv/linux/hppa/bits/atomic.h 2007-05-20 03:24:41.000000000 +0200 renamed from sysdeps/unix/sysv/linux/hppa/bits/atomic.h.orig 2007-05-20 03:28:39.000000000 +0200 +1 -1
@@ -56,7 +56,7 @@
 #define LWS_CAS "0"
 /* Note r31 is the link register */
 #define LWS_CLOBBER "r1", "r26", "r25", "r24", "r23", "r22", "r21", "r20", "r28", "r31", "memory"
-#define ASM_EAGAIN "11"
+#define ASM_EAGAIN "-11"
 
 #if __ASSUME_LWS_CAS
 /* The only basic operation needed is compare and exchange.  */

Aurelian,

Good catch! I applaud your effort! I'll test the patch this week and get it into cvs.

Quoted 8 lines

If the kernel is returning EAGAIN it means there are multiple threads calling LWS CAS on addresses whose hash selects the same kernel spinlock (there is an array of 16 for minimal scalability). The kernel never spins and expects the caller to spin in userpace.

The verified the original test code I wrote does check for "-11", and I did run the kernel tests on an SMP machine with multiple threads, so I know that LWS CAS works on SMP.

Well we still have a problem, the failure occurs less often, but still sometimes. Apparently it is cause by the kernel returning -EDEADLOCK. How should that be handled? The same way as -EAGAIN?

Cheers,
Aurelien

Signature · 8 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#10 lore raw
Quoted 5 lines · Carlos O'Donell
Quoted 3 lines · Aurelien Jarno

Well we still have a problem, the failure occurs less often, but still sometimes. Apparently it is cause by the kernel returning -EDEADLOCK. How should that be handled? The same way as -EAGAIN?

Quoted 9 lines

No.

The only way you can get -EDEADLOCK back is to build a kernel with ENABLE_LWS_DEBUG, this is strictly a developer only feature, please do not enable this.

Please verify that ENABLE_LWS_DEBUG is defined as 0 in:
arch/parisc/kernel/syscall.S:467 #define ENABLE_LWS_DEBUG 0

As discussed on IRC, there is actually a bug in the kernel:

  #define ENABLE_LWS_DEBUG 0
  ...
  # ifdef ENABLE_LWS_DEBUG

ENABLE_LWS_DEBUG is defined as 0, so the debug code is enabled.

Please find below a patch which fixes the -EAGAIN problem by harcoding the value directly into the code. It also adds a workaround for unfixed kernels which return -EDEADLOCK.

Cheers,
Aurelien

2007-05-22 Aurelien Jarno <[email protected]>

* sysdeps/unix/sysv/linux/hppa/bits/atomic.h: check for -11 (-EAGAIN) instead of 11. Loop again when the kernel returns error -45 (-EDEADLOCK) to workaround a kernel bug (debugging code enabled).

sysdeps/unix/sysv/linux/hppa/bits/atomic.h renamed from sysdeps/unix/sysv/linux/hppa/bits/atomic.h.orig +10 -3
@@ -56,10 +56,14 @@
 #define LWS_CAS "0"
 /* Note r31 is the link register */
 #define LWS_CLOBBER "r1", "r26", "r25", "r24", "r23", "r22", "r21", "r20", "r28", "r31", "memory"
-#define ASM_EAGAIN "11" 
 
 #if __ASSUME_LWS_CAS
-/* The only basic operation needed is compare and exchange.  */
+/* The only basic operation needed is compare and exchange.
+   -11 corresponds to -EAGAIN, and -45 to -EDEADLOCK. The constants
+   must stay the same to avoid breaking backwards compatibility.
+
+   The check for -EDEADLOCK workarounds a kernel bug (debugging code
+   enabled by default). */
 # define atomic_compare_and_exchange_val_acq(mem, newval, oldval) 	\
   ({									\
      volatile int lws_errno = EFAULT;					\
@@ -71,7 +75,10 @@
 	"copy	%5, %%r24			\n\t"			\
 	"ble	" LWS "(%%sr2, %%r0)		\n\t"			\
 	"ldi	" LWS_CAS ", %%r20		\n\t"			\
-	"cmpib,=,n " ASM_EAGAIN ",%%r21,0b	\n\t"			\
+	"cmpib,=,n -11,%%r21,0b			\n\t"			\
+	"nop 					\n\t"			\
+	"ldi	-45, %%r24			\n\t"			\
+	"cmpb,=,n %%r24,%%r21,0b		\n\t"			\
 	"nop					\n\t"			\
 	"stw	%%r28, %0			\n\t"			\
         "sub	%%r0, %%r21, %%r21		\n\t"			\
Signature · 8 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#12 lore raw
Quoted 6 lines · Aurelien Jarno

As discussed on IRC, there is actually a bug in the kernel:

  #define ENABLE_LWS_DEBUG 0
  ...
  # ifdef ENABLE_LWS_DEBUG

Patch to switch #ifdef to #if $x == 1 or the #define to #undef welcome.

Review

Quoted 12 lines

ENABLE_LWS_DEBUG is defined as 0, so the debug code is enabled.

Please find below a patch which fixes the -EAGAIN problem by harcoding the value directly into the code. It also adds a workaround for unfixed kernels which return -EDEADLOCK.

(quoted diff) +6 -0
+/* The only basic operation needed is compare and exchange.
+   -11 corresponds to -EAGAIN, and -45 to -EDEADLOCK. The constants
+   must stay the same to avoid breaking backwards compatibility.
+
+   The check for -EDEADLOCK workarounds a kernel bug (debugging code
+   enabled by default). */

This is stupid. Why not just use the symbol and then #undef it immediately afterwards? For extra paranoia name it LWS_RET_EAGAIN or something.

Putting magic numbers in the code is irritating and makes it more difficult to read it on a glance.

Regards,
Kyle
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]