Skip to contentSkip to conversation

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

PATCH Superseded kernel test robot 4 participants 26 messages Jun 23, 2024 – Jun 30, 2024 x86

lore
Share & tools
lore

kernel test robot

author build bot
#0 lore raw
tree:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
head:   5f583a3162ffd9f7999af76b8ab634ce2dac9f90
commit: 95ece48165c136b96fae0f6144f55cbf8b24aeb9 locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions
date:   2 months ago
config: i386-randconfig-061-20240623 (https://download.01.org/0day-ci/archive/20240623/[email protected]/config)
compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240623/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from kernel/bpf/core.c:21:
In file included from include/linux/filter.h:8:
In file included from include/linux/atomic.h:7:
In file included from arch/x86/include/asm/atomic.h:8:
In file included from arch/x86/include/asm/cmpxchg.h:143:

>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
     149 |         return __arch_try_cmpxchg64_emu(ptr, oldp, new);
         |                ^
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^
   In file included from kernel/bpf/core.c:21:
   In file included from include/linux/filter.h:8:
   In file included from include/linux/atomic.h:7:
   In file included from arch/x86/include/asm/atomic.h:8:
   In file included from arch/x86/include/asm/cmpxchg.h:143:
>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^
   In file included from kernel/bpf/core.c:21:
   In file included from include/linux/filter.h:8:
   In file included from include/linux/atomic.h:7:
   In file included from arch/x86/include/asm/atomic.h:8:
   In file included from arch/x86/include/asm/cmpxchg.h:143:
>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^
   In file included from kernel/bpf/core.c:21:
   In file included from include/linux/filter.h:8:
   In file included from include/linux/atomic.h:7:
   In file included from arch/x86/include/asm/atomic.h:8:
   In file included from arch/x86/include/asm/cmpxchg.h:143:
>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^
   In file included from kernel/bpf/core.c:21:
   In file included from include/linux/filter.h:8:
   In file included from include/linux/atomic.h:7:
   In file included from arch/x86/include/asm/atomic.h:8:
   In file included from arch/x86/include/asm/cmpxchg.h:143:
>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^
   In file included from kernel/bpf/core.c:21:
   In file included from include/linux/filter.h:8:
   In file included from include/linux/atomic.h:7:
   In file included from arch/x86/include/asm/atomic.h:8:
   In file included from arch/x86/include/asm/cmpxchg.h:143:
>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^
   In file included from kernel/bpf/core.c:21:
   In file included from include/linux/filter.h:8:
   In file included from include/linux/atomic.h:7:
   In file included from arch/x86/include/asm/atomic.h:8:
   In file included from arch/x86/include/asm/cmpxchg.h:143:
>> arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available
   arch/x86/include/asm/cmpxchg_32.h:131:15: note: expanded from macro '__arch_try_cmpxchg64_emu'
     131 |         asm volatile(ALTERNATIVE(LOCK_PREFIX_HERE                       \
         |                      ^
   arch/x86/include/asm/alternative.h:218:2: note: expanded from macro 'ALTERNATIVE'
     218 |         OLDINSTR(oldinstr, 1)                                           \
         |         ^
   arch/x86/include/asm/alternative.h:168:2: note: expanded from macro 'OLDINSTR'
     168 |         "# ALT: oldnstr\n"                                              \
         |         ^

7 errors generated.

vim +149 arch/x86/include/asm/cmpxchg_32.h

aef95dac9ce4f2 Uros Bizjak 2024-04-08  146  
aef95dac9ce4f2 Uros Bizjak 2024-04-08  147  static __always_inline bool arch_try_cmpxchg64(volatile u64 *ptr, u64 *oldp, u64 new)
aef95dac9ce4f2 Uros Bizjak 2024-04-08  148  {
aef95dac9ce4f2 Uros Bizjak 2024-04-08 @149  	return __arch_try_cmpxchg64_emu(ptr, oldp, new);
aef95dac9ce4f2 Uros Bizjak 2024-04-08  150  }
aef95dac9ce4f2 Uros Bizjak 2024-04-08  151  #define arch_try_cmpxchg64 arch_try_cmpxchg64

aef95dac9ce4f2 Uros Bizjak 2024-04-08 152

:::::: The code at line 149 was first introduced by commit
:::::: aef95dac9ce4f271cc43195ffc175114ed934cbe locking/atomic/x86: Introduce arch_try_cmpxchg64() for !CONFIG_X86_CMPXCHG64

:::::: TO: Uros Bizjak <[email protected]>
:::::: CC: Ingo Molnar <[email protected]>

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#1 lore raw
Quoted 16 lines · kernel test robot
tree:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
head:   5f583a3162ffd9f7999af76b8ab634ce2dac9f90
commit: 95ece48165c136b96fae0f6144f55cbf8b24aeb9 locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions
date:   2 months ago
config: i386-randconfig-061-20240623 (https://download.01.org/0day-ci/archive/20240623/[email protected]/config)
compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240623/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

Quoted 7 lines

In file included from kernel/bpf/core.c:21:
In file included from include/linux/filter.h:8:
In file included from include/linux/atomic.h:7:
In file included from arch/x86/include/asm/atomic.h:8:
In file included from arch/x86/include/asm/cmpxchg.h:143:

Quoted 1 line
Quoted 1 line

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

     149 |         return __arch_try_cmpxchg64_emu(ptr, oldp, new);

The referred patch actually mitigates the issue, as explained in details here [1].

[1] https://lore.kernel.org/lkml/CAFULd4b8kRgy0p3pKwheLRHx7rX+4RuY-45_Zt-CCj5sCpGjCw@mail.gmail.com/

Based on the above explanation, the build will break even harder without the patch, because *TWO* additional registers are needed to carry the value to be compared over cmpxchg8b instruction. The trouble that register allocator goes through can be seen at [2] and [3].

[2] https://lore.kernel.org/lkml/[email protected]/
[3] https://lore.kernel.org/lkml/[email protected]/

Up to now, the atomic64_{and,or,xor} functions (and their derivatives) were/are seldom used in the kernel. Recently, their usage increased, so the register pressure situation around cmpxchg8b instruction worsened. For some reason, x86 implementations of these functions are declared as inline functions, in contrast with much more used atomic64_{inc,dec,add,sub}, which are declared as outline functions. While using try_cmpxchg() instead of cmpxchg() helps, it can only go so far.

The real and reliable fix is to re-declare x86 arch_atomic64_{and,or,xor} functions as outline functions in the same way x86 arch_atomic64_{inc,dec,add,sub} are declared. The infrastructure is already there, the assembly macros just need to be generalized slightly to handle additional asm primitives.

I started this fix some time ago [4] by optimizing the infrastructure first that could relax some register pressure, but there was no reaction from x86 maintainers. I don't want to waste my time on patches that get ignored, so I dropped the ball.

[4] https://lore.kernel.org/lkml/[email protected]/

So, without some interest from x86 maintainers, the issue will remain unfixed. If they want x86_32 arch to die then inline locking primitives involving cmpxchg8b are one of the nails in its coffin.

Uros.

#2 lore raw
Quoted 3 lines · Uros Bizjak

The referred patch actually mitigates the issue, as explained in details here [1].

Clearly not.

It might help things in theory, but in practice it clearly just makes things worse, and causes problems for clang.

And the thing about practice is that it always trumps theory.

Quoted 3 lines

So, without some interest from x86 maintainers, the issue will remain unfixed. If they want x86_32 arch to die then inline locking primitives involving cmpxchg8b are one of the nails in its coffin.

What? No.

That commit will get reverted. It caused problems, and now you are blaming others.

You can't just go "I caused issues, but the x86 maintainers aren't interested in this area, so it's their fault".

And no, "improved code generation with one compiler version" does not then mean "other compilers don't matter".

Now, I do suspect that the issue is some very random thing that could be fixed by massaging the inline asm a bit.

For example, why does that 32-bit __arch_try_cmpxchg64() do this:

        if (unlikely(!ret))                                             \
                *(_oldp) = o.full;                                      \

when I think it would be simpler and more straightforward to just do that *(_oldp) = o.full unconditionally? The "dereference" is purely syntactic, the intent is that the compiler will just use a register for this all (well, two registers on 32-bit), and making it conditional makes things worse, because now that "oldp" register cannot be the same as "o" itself.

I didn't look very closely into this, but maybe that would just fix code generation. And maybe it wouldn't. If there's no other fix somebody comes up with, we just have to revert.

Linus

#3 lore raw
Quoted 8 lines · Linus Torvalds

For example, why does that 32-bit __arch_try_cmpxchg64() do this:

        if (unlikely(!ret))                                             \
                *(_oldp) = o.full;                                      \

when I think it would be simpler and more straightforward to just do that *(_oldp) = o.full unconditionally?

I can repro the clang issue, and no, removing the conditional doesn't fix it.

A plain revert of

95ece48165c1 ("locking/atomic/x86: Rewrite x86_32
arch_atomic64_{,fetch}_{and,or,xor}() functions")

does fix it.

The problem does *NOT* happen with a defconfig, so it's clearly triggered by some horror in that config:

    https://download.01.org/0day-ci/archive/20240623/[email protected]/config

and a quick config bisection seems to imply that it might just be

X86_MINIMUM_CPU_FAMILY 6 -> 4

from setting CONFIG_MGEODEGX1=y instead of M686.

Hmm. While I'm not willing to leave x86-32 behind, I personally have thought for a couple of years that we should leave Pentium and earlier behind. We should just require cmpxchg64b support.

It does look like the complication at least partly comes from the emulation alternative, but I didn't go look any closer.

Linus

#4 lore raw
In reply to Linus Torvalds
Quoted 17 lines · Linus Torvalds
Quoted 3 lines · Uros Bizjak

The referred patch actually mitigates the issue, as explained in details here [1].

Clearly not.

It might help things in theory, but in practice it clearly just makes things worse, and causes problems for clang.

And the thing about practice is that it always trumps theory.

Quoted 3 lines

So, without some interest from x86 maintainers, the issue will remain unfixed. If they want x86_32 arch to die then inline locking primitives involving cmpxchg8b are one of the nails in its coffin.

Quoted 7 lines

What? No.

That commit will get reverted. It caused problems, and now you are blaming others.

You can't just go "I caused issues, but the x86 maintainers aren't interested in this area, so it's their fault".

I disagree with the above. There is no one to blame, let alone blaming others.

The cmpxchg8b is a complex beast, It needs 4 input registers (a pair of them inout) and optionally 2 address registers. When the frame pointer is active, the instruction can consume all available registers on x86_32, so the compiler has to work extremely hard to satisfy the required insn constraints (and also pass live values over the instruction). The two examples I quoted in my previous mail just illustrate this.

Quoted 2 lines

And no, "improved code generation with one compiler version" does not then mean "other compilers don't matter".

Using try_cmpxchg *improves* the situation, as there is a double-word value that does not have to be live over the insn. So, leaving the situation as it is will just trigger the issue later. We had exactly these problems in GCC with __atomic builtins, and GCC learned how to handle this extreme register pressure.

Quoted 8 lines

Now, I do suspect that the issue is some very random thing that could be fixed by massaging the inline asm a bit.

For example, why does that 32-bit __arch_try_cmpxchg64() do this:

        if (unlikely(!ret))                                             \
                *(_oldp) = o.full;                                      \
Quoted 6 lines

when I think it would be simpler and more straightforward to just do that *(_oldp) = o.full unconditionally? The "dereference" is purely syntactic, the intent is that the compiler will just use a register for this all (well, two registers on 32-bit), and making it conditional makes things worse, because now that "oldp" register cannot be the same as "o" itself.

The primitive is written in this way because we would like to use it in a cmpxchg loop, as well as as a condition in an if clause. Without the above referred condition, the compiler will emit a dead store in the later case. I have tested this macro extensively, and the above is really the best we can do, it results in a tight cmpxchg loop and without unneeded moves/stores in code without loops.

Quoted 3 lines

I didn't look very closely into this, but maybe that would just fix code generation. And maybe it wouldn't. If there's no other fix somebody comes up with, we just have to revert.

As said in my previous message, if the compiler can't handle __try_cmpxchg implementation, the most straightforward and reliable solution is to implement atomic64_{and,or,xor} as outline functions,

Uros.

#5 lore raw
Quoted 2 lines · Uros Bizjak

I disagree with the above.

Your disagreement just doesn't matter.

We don't introduce regressions and then blame others.

There's a very clear rule in kernel development: things that break other things ARE NOT FIXES.

EVER.

They get reverted, or the thing they broke gets fixed.

This is not debatable, or something you can "disagree"' with. This is how we work, and if you disagree with that, you had better get out of kernel development.

Quoted 3 lines

As said in my previous message, if the compiler can't handle __try_cmpxchg implementation, the most straightforward and reliable solution is to implement atomic64_{and,or,xor} as outline functions,

You also seemed to say that nobody was doing it.

Which means "revert". Because that I can do trivially.

Now, from having looked a bit at this, I can point you to the differences introduced by having to have the emulation fallback.

And I would personally be ok with leaving Pentium etc entirely behind, and getting rid of said fallback, but that's an entirely separate thing that will require much more discussion.

Some people still wanted to support old 486 clones not that long ago.

Linus

#6 lore raw
Quoted 3 lines · Linus Torvalds

Now, from having looked a bit at this, I can point you to the differences introduced by having to have the emulation fallback.

Ahh.

Itr does all the same things *and* it has

    "S" (_ptr)

as an added register pressure, because if we take the "call cmpxchg8b_emu", we need the address in a fixed place.

I don't see any immediately obvious workaround.

Linus

#7 lore raw
Quoted 9 lines · Linus Torvalds
Quoted 3 lines · Linus Torvalds

Now, from having looked a bit at this, I can point you to the differences introduced by having to have the emulation fallback.

Ahh.

Quoted 8 lines

Itr does all the same things *and* it has

    "S" (_ptr)

as an added register pressure, because if we take the "call cmpxchg8b_emu", we need the address in a fixed place.

I don't see any immediately obvious workaround.

A real fix, not only a workaround, is to rewrite asm arguments to something like (untested, but "should work"TM):

#define __arch_cmpxchg64_emu(_ptr, _old, _new, _lock_loc, _lock) \
({ \
union __u64_halves o = { .full = (_old), }, \
n = { .full = (_new), }; \
\
asm volatile(ALTERNATIVE(_lock_loc \
"call cmpxchg8b_emu", \

_lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
    : "+a" (o.low), "+d" (o.high) \
    : [ptr] "S" (_ptr), \
      "b" (n.low), "c" (n.high) \
    : "memory"); \

\
o.full; \
})

(and in a similar way for __arch_try_cmpxchg64_emu).

We already have a memory clobber in the asm template, so to solve the register pressure issue, just reuse the pointer in %esi to also form the memory address of cmpxchg8b. GCC is smart enough to do this by itself, but the current asm template indeed doesn't force "other" compilers to do so.

Uros.

#8 lore raw
Quoted 3 lines · Uros Bizjak

A real fix, not only a workaround, is to rewrite asm arguments to something like (untested, but "should work"TM):

Sadly, I already tried that, and it didn't help.

Linus

#9 lore raw
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Uros Bizjak

A real fix, not only a workaround, is to rewrite asm arguments to something like (untested, but "should work"TM):

Sadly, I already tried that, and it didn't help.

Bah. I _had_ tried that, but I had tried it on __arch_cmpxchg64_emu(), not on the "try" version.

And no, it hadn't helped.

But doing it *right*, and also doing __arch_try_cmpxchg64_emu() *does* actually help.

Well, at least the attached patch compiles with the bad config and clang.

I'm not certain it does the right thing, because I did more than just the %esi games to try to simplify it.

It's *ENTIRELY* possible that I screwed up. Just in this thread, I have looked at the wrong inline asm at least twice now, so I'm not feeling super-competent right now.

I'm sending the patch out in the hope that another set of eyes will make it actually better.

Linus

 arch/x86/include/asm/cmpxchg_32.h | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)
arch/x86/include/asm/cmpxchg_32.h +12 -16
index ed2797f132ce..b19985520f3c 100644
@@ -88,18 +88,17 @@ static __always_inline bool __try_cmpxchg64_local(volatile u64 *ptr, u64 *oldp,
 
 #define __arch_cmpxchg64_emu(_ptr, _old, _new, _lock_loc, _lock)	\
 ({									\
-	union __u64_halves o = { .full = (_old), },			\
-			   n = { .full = (_new), };			\
+	__u64 o = (_old);						\
+	union __u64_halves n = { .full = (_new), };			\
 									\
 	asm volatile(ALTERNATIVE(_lock_loc				\
 				 "call cmpxchg8b_emu",			\
-				 _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
-		     : [ptr] "+m" (*(_ptr)),				\
-		       "+a" (o.low), "+d" (o.high)			\
-		     : "b" (n.low), "c" (n.high), "S" (_ptr)		\
+				 _lock "cmpxchg8b 0(%[ptr])", X86_FEATURE_CX8) \
+		     : "+A" (o)						\
+		     : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)	\
 		     : "memory");					\
 									\
-	o.full;								\
+	o;								\
 })
 
 static __always_inline u64 arch_cmpxchg64(volatile u64 *ptr, u64 old, u64 new)
@@ -116,22 +115,19 @@ static __always_inline u64 arch_cmpxchg64_local(volatile u64 *ptr, u64 old, u64
 
 #define __arch_try_cmpxchg64_emu(_ptr, _oldp, _new, _lock_loc, _lock)	\
 ({									\
-	union __u64_halves o = { .full = *(_oldp), },			\
-			   n = { .full = (_new), };			\
+	__u64 o = *(_oldp);						\
+	union __u64_halves n = { .full = (_new), };			\
 	bool ret;							\
 									\
 	asm volatile(ALTERNATIVE(_lock_loc				\
 				 "call cmpxchg8b_emu",			\
-				 _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
+				 _lock "cmpxchg8b 0(%[ptr])", X86_FEATURE_CX8) \
 		     CC_SET(e)						\
-		     : CC_OUT(e) (ret),					\
-		       [ptr] "+m" (*(_ptr)),				\
-		       "+a" (o.low), "+d" (o.high)			\
-		     : "b" (n.low), "c" (n.high), "S" (_ptr)		\
+		     : CC_OUT(e) (ret), "+A" (o)			\
+		     : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)	\
 		     : "memory");					\
 									\
-	if (unlikely(!ret))						\
-		*(_oldp) = o.full;					\
+	*(_oldp) = o;							\
 									\
 	likely(ret);							\
 })
#10 lore raw
Quoted 24 lines · Linus Torvalds
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Uros Bizjak

A real fix, not only a workaround, is to rewrite asm arguments to something like (untested, but "should work"TM):

Sadly, I already tried that, and it didn't help.

Bah. I _had_ tried that, but I had tried it on __arch_cmpxchg64_emu(), not on the "try" version.

And no, it hadn't helped.

But doing it *right*, and also doing __arch_try_cmpxchg64_emu() *does* actually help.

Well, at least the attached patch compiles with the bad config and clang.

I'm not certain it does the right thing, because I did more than just the %esi games to try to simplify it.

Quoted 6 lines

It's *ENTIRELY* possible that I screwed up. Just in this thread, I have looked at the wrong inline asm at least twice now, so I'm not feeling super-competent right now.

I'm sending the patch out in the hope that another set of eyes will make it actually better.

+ _lock "cmpxchg8b 0(%[ptr])", X86_FEATURE_CX8) \

This can be just:

+ _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \

- if (unlikely(!ret)) \
- *(_oldp) = o.full; \
+ *(_oldp) = o; \

This one should really update only when cmpxchg fails. The relation between cmpxchg and try_cmpxchg is documented in Documentation/atomic_t.txt, section CMPXCHG vs TRY_CMPXCHG.

Does the compilation still crash with the condition?

Uros.

#11 lore raw

[ Sorry for not being hugely responsive the last few days - I was on the road for a family get-together on the east coast, so I spent time on airplanes and hotels and I don't particularly enjoy working with a laptop ]

Quoted 10 lines · Uros Bizjak
Quoted 4 lines

I'm sending the patch out in the hope that another set of eyes will make it actually better.

+ _lock "cmpxchg8b 0(%[ptr])", X86_FEATURE_CX8) \

This can be just:

+ _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \

Thanks, yup, will fix.

Quoted 5 lines

- if (unlikely(!ret)) \
- *(_oldp) = o.full; \
+ *(_oldp) = o; \

This one should really update only when cmpxchg fails.

The thing is, when cmpxchg doesn't fail, then oldp should already be "old", no?

I mean, by the very definition, atomic_try_cmpxchg() can *not* be successful if the new value didn't match the old one.

I mean, just look at the very doc you point to - the "definition" is

  bool atomic_try_cmpxchg(atomic_t *ptr, int *oldp, int new)
  {
    int ret, old = *oldp;
    ret = atomic_cmpxchg(ptr, old, new);
    if (ret != old)
      *oldp = ret;
    return ret == old;
  }

iow, it only returns success of "ret == old", and "old" by definition is "the contents of oldp".

(Here "oldp" is a local variable, not something that can be changing).

So I *think* the whole

    if (ret != old)
      *oldp = ret;

is actually counter-productive, and could/should be just that simpler unconditional *oldp = ret, because you have two cases:

- ret == old: the assignment doesn't change anything and is a no-op

- ret !=- old: the assignment needs to be done

but doing it *unconditionally* means that now as fat as the compiler is concerned, the original *oldp value is unconditionally dead, which sounds to me like it should be good for register allocation (the context here being that it _looks_ like a pointer access, but it's really meant to be a "in-out argument in a register").

Now, in practice, I suspect that everybody checks the return value and "old" is never used afterwards in the success case, so in that sense this doesn't matter and it's all dead regardless.

But it seems to be a complication in the docs and the implementation.

Of course, I may be missing something completely obvious, and/or you have some subtle code generation reason why you prefer the conditional there. Feel free to explain,

Linus

#12 lore raw

PATCH 1/1 x86-32: fix cmpxchg8b_emu build error with clang

The kernel test robot reported that clang no longer compiles the 32-bit x86 kernel in some configurations due to commit 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions").

The build fails with

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

and the reason seems to be that not only does the cmpxchg8b instruction need four fixed registers (EDX:EAX and ECX:EBX), with the emulation fallback the inline asm also wants a fifth fixed register for the address (it uses %esi for that, but that's just a software convention with cmpxchg8b_emu).

Avoiding using another pointer input to the asm (and just forcing it to use the "0(%esi)" addressing that we end up requiring for the sw fallback) seems to fix the issue.

Fixes 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions")

---

Added commit message, and updated the asm to use '%a[ptr]' instead of writing out the addressing by hand.

Still doing the 'oldp' writeback unconmditionally. The code generation for the case I checked were the same for both clang and gcc, but until Uros hits me with the big clue-hammer, I think it's the simpler code that leaves room for potentially better optimizations too.

This falls solidly in the "looks ok to me, but still untested" category for me. It fixes the clang build issue in my build testing, but I no longer have a 32-bit test environment, so no actual runtime testing.

 arch/x86/include/asm/cmpxchg_32.h | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)
arch/x86/include/asm/cmpxchg_32.h +12 -16
index ed2797f132ce..4444a8292c7a 100644
@@ -88,18 +88,17 @@ static __always_inline bool __try_cmpxchg64_local(volatile u64 *ptr, u64 *oldp,
 
 #define __arch_cmpxchg64_emu(_ptr, _old, _new, _lock_loc, _lock)	\
 ({									\
-	union __u64_halves o = { .full = (_old), },			\
-			   n = { .full = (_new), };			\
+	__u64 o = (_old);						\
+	union __u64_halves n = { .full = (_new), };			\
 									\
 	asm volatile(ALTERNATIVE(_lock_loc				\
 				 "call cmpxchg8b_emu",			\
-				 _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
-		     : [ptr] "+m" (*(_ptr)),				\
-		       "+a" (o.low), "+d" (o.high)			\
-		     : "b" (n.low), "c" (n.high), "S" (_ptr)		\
+				 _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
+		     : "+A" (o)						\
+		     : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)	\
 		     : "memory");					\
 									\
-	o.full;								\
+	o;								\
 })
 
 static __always_inline u64 arch_cmpxchg64(volatile u64 *ptr, u64 old, u64 new)
@@ -116,22 +115,19 @@ static __always_inline u64 arch_cmpxchg64_local(volatile u64 *ptr, u64 old, u64
 
 #define __arch_try_cmpxchg64_emu(_ptr, _oldp, _new, _lock_loc, _lock)	\
 ({									\
-	union __u64_halves o = { .full = *(_oldp), },			\
-			   n = { .full = (_new), };			\
+	__u64 o = *(_oldp);						\
+	union __u64_halves n = { .full = (_new), };			\
 	bool ret;							\
 									\
 	asm volatile(ALTERNATIVE(_lock_loc				\
 				 "call cmpxchg8b_emu",			\
-				 _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
+				 _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
 		     CC_SET(e)						\
-		     : CC_OUT(e) (ret),					\
-		       [ptr] "+m" (*(_ptr)),				\
-		       "+a" (o.low), "+d" (o.high)			\
-		     : "b" (n.low), "c" (n.high), "S" (_ptr)		\
+		     : CC_OUT(e) (ret), "+A" (o)			\
+		     : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)	\
 		     : "memory");					\
 									\
-	if (unlikely(!ret))						\
-		*(_oldp) = o.full;					\
+	*(_oldp) = o;							\
 									\
 	likely(ret);							\
 })
Signature · 1 line
2.45.1.209.gc6f12300df
#13 lore raw

Re: [PATCH] x86-32: fix cmpxchg8b_emu build error with clang

Quoted 28 lines · Linus Torvalds

The kernel test robot reported that clang no longer compiles the 32-bit x86 kernel in some configurations due to commit 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions").

The build fails with

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

and the reason seems to be that not only does the cmpxchg8b instruction need four fixed registers (EDX:EAX and ECX:EBX), with the emulation fallback the inline asm also wants a fifth fixed register for the address (it uses %esi for that, but that's just a software convention with cmpxchg8b_emu).

Avoiding using another pointer input to the asm (and just forcing it to use the "0(%esi)" addressing that we end up requiring for the sw fallback) seems to fix the issue.

Fixes 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions")

---

Quoted 7 lines

Added commit message, and updated the asm to use '%a[ptr]' instead of writing out the addressing by hand.

Still doing the 'oldp' writeback unconmditionally. The code generation for the case I checked were the same for both clang and gcc, but until Uros hits me with the big clue-hammer, I think it's the simpler code that leaves room for potentially better optimizations too.

You probably want to look at 44fe84459faf1 ("locking/atomic: Fix atomic_try_cmpxchg() semantics") [1] and the long LKML discussion at [2].

--quote--
This code is broken with the current implementation, the problem is with unconditional update of *__po.

In case of success it writes the same value back into *__po, but in case of cmpxchg success we might have lose ownership of some memory locations and potentially over what __po has pointed to. The same holds for the re-read of *__po. "
--/quote--

[1] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?id=44fe84459faf1a7781595b7c64cd36daf2f2827d
[2] https://lore.kernel.org/lkml/CACT4Y+bG+a0w6j6v1AmBE7fqqMSPyPEm4QimCzCouicmHT8FqA@mail.gmail.com/

Uros.

Quoted 69 lines

This falls solidly in the "looks ok to me, but still untested" category for me. It fixes the clang build issue in my build testing, but I no longer have a 32-bit test environment, so no actual runtime testing.

 arch/x86/include/asm/cmpxchg_32.h | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)
arch/x86/include/asm/cmpxchg_32.h +12 -17
index ed2797f132ce..4444a8292c7a 100644
@@ -88,18 +88,17 @@ static __always_inline bool __try_cmpxchg64_local(volatile u64 *ptr, u64 *oldp,
 
 #define __arch_cmpxchg64_emu(_ptr, _old, _new, _lock_loc, _lock)       \
 ({                                                                     \
-       union __u64_halves o = { .full = (_old), },                     \
-                          n = { .full = (_new), };                     \
+       __u64 o = (_old);                                               \
+       union __u64_halves n = { .full = (_new), };                     \
                                                                        \
        asm volatile(ALTERNATIVE(_lock_loc                              \
                                 "call cmpxchg8b_emu",                  \
-                                _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
-                    : [ptr] "+m" (*(_ptr)),                            \
-                      "+a" (o.low), "+d" (o.high)                      \
-                    : "b" (n.low), "c" (n.high), "S" (_ptr)            \
+                                _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
+                    : "+A" (o)                                         \
+                    : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)      \
                     : "memory");                                       \
                                                                        \
-       o.full;                                                         \
+       o;                                                              \
 })
 
 static __always_inline u64 arch_cmpxchg64(volatile u64 *ptr, u64 old, u64 new)
@@ -116,22 +115,19 @@ static __always_inline u64 arch_cmpxchg64_local(volatile u64 *ptr, u64 old, u64
 
 #define __arch_try_cmpxchg64_emu(_ptr, _oldp, _new, _lock_loc, _lock)  \
 ({                                                                     \
-       union __u64_halves o = { .full = *(_oldp), },                   \
-                          n = { .full = (_new), };                     \
+       __u64 o = *(_oldp);                                             \
+       union __u64_halves n = { .full = (_new), };                     \
        bool ret;                                                       \
                                                                        \
        asm volatile(ALTERNATIVE(_lock_loc                              \
                                 "call cmpxchg8b_emu",                  \
-                                _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
+                                _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
                     CC_SET(e)                                          \
-                    : CC_OUT(e) (ret),                                 \
-                      [ptr] "+m" (*(_ptr)),                            \
-                      "+a" (o.low), "+d" (o.high)                      \
-                    : "b" (n.low), "c" (n.high), "S" (_ptr)            \
+                    : CC_OUT(e) (ret), "+A" (o)                        \
+                    : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)      \
                     : "memory");                                       \
                                                                        \
-       if (unlikely(!ret))                                             \
-               *(_oldp) = o.full;                                      \
+       *(_oldp) = o;                                                   \
                                                                        \
        likely(ret);                                                    \
 })
--

2.45.1.209.gc6f12300df

#14 lore raw

Re: [PATCH] x86-32: fix cmpxchg8b_emu build error with clang

Quoted 13 lines · Uros Bizjak
Quoted 10 lines · Linus Torvalds

The kernel test robot reported that clang no longer compiles the 32-bit x86 kernel in some configurations due to commit 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions").

The build fails with

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

Quoted 8 lines
Quoted 8 lines

and the reason seems to be that not only does the cmpxchg8b instruction need four fixed registers (EDX:EAX and ECX:EBX), with the emulation fallback the inline asm also wants a fifth fixed register for the address (it uses %esi for that, but that's just a software convention with cmpxchg8b_emu).

Avoiding using another pointer input to the asm (and just forcing it to use the "0(%esi)" addressing that we end up requiring for the sw

A nit: offset 0 is required only for %ebp, so the above should read "(%esi)".

Quoted 41 lines
Quoted 10 lines

fallback) seems to fix the issue.

Fixes 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions")

---

Quoted 7 lines

Added commit message, and updated the asm to use '%a[ptr]' instead of writing out the addressing by hand.

Still doing the 'oldp' writeback unconmditionally. The code generation for the case I checked were the same for both clang and gcc, but until Uros hits me with the big clue-hammer, I think it's the simpler code that leaves room for potentially better optimizations too.

You probably want to look at 44fe84459faf1 ("locking/atomic: Fix atomic_try_cmpxchg() semantics") [1] and the long LKML discussion at [2].

--quote--
This code is broken with the current implementation, the problem is with unconditional update of *__po.

In case of success it writes the same value back into *__po, but in case of cmpxchg success we might have lose ownership of some memory locations and potentially over what __po has pointed to. The same holds for the re-read of *__po. "
--/quote--

[1] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?id=44fe84459faf1a7781595b7c64cd36daf2f2827d
[2] https://lore.kernel.org/lkml/CACT4Y+bG+a0w6j6v1AmBE7fqqMSPyPEm4QimCzCouicmHT8FqA@mail.gmail.com/

Uros.

Quoted 4 lines

This falls solidly in the "looks ok to me, but still untested" category for me. It fixes the clang build issue in my build testing, but I no
longer have a 32-bit test environment, so no actual runtime testing. \

...

Quoted 4 lines

arch/x86/include/asm/cmpxchg_32.h static __always_inline u64 arch_cmpxchg64_local(volatile u64 *ptr, u64 old, u64 · line 130

Quoted 4 lines
                                                                        \
arch/x86/include/asm/cmpxchg_32.h +1 -2
-       if (unlikely(!ret))                                             \
-               *(_oldp) = o.full;                                      \
+       *(_oldp) = o;                                                   \

With the above part changed to:

       if (unlikely(!ret))                                             \
-               *(_oldp) = o.full;                                      \
+               *(_oldp) = o;                                           \
Reviewed-and-Tested-by Uros Bizjak <[email protected]>

Runtime tested with .config provided by test robot on qemu-i386 with both clang and GCC compiler:

LKP: ttyS0: 229: Kernel tests: Boot OK!

Thanks,
Uros.

#15 lore raw

Re: [PATCH] x86-32: fix cmpxchg8b_emu build error with clang

In reply to Uros Bizjak
Quoted 10 lines · Uros Bizjak
Quoted 6 lines

Still doing the 'oldp' writeback unconmditionally. The code generation for the case I checked were the same for both clang and gcc, but until Uros hits me with the big clue-hammer, I think it's the simpler code that leaves room for potentially better optimizations too.

You probably want to look at 44fe84459faf1 ("locking/atomic: Fix atomic_try_cmpxchg() semantics") [1] and the long LKML discussion at [2].

Christ. That use should be invalid.

The only _atomic_ pointer is "_ptr", not "old". Anybody who gives something that can change during the operation in "old" is basically already doing random things.

Quoted 3 lines

--quote--
This code is broken with the current implementation, the problem is with unconditional update of *__po.

I think the only thing broken is that quote, and the crazy expectation that "old" can change.

But obviously, I had completely forgotten that whole discussion from seven years ago.

I don't actually find a single use of that invalid code sequence where somebody would pass a non-private pointer as "oldp". So I really think that part of the whole discussion was bogus to begin with, and presumably from some other code base.

IOW, I think that example of a "classical lock-free stack push" is just broken.

That said, I can't find a case where it would matter for code generation (every use will always do a conditional branch based on the result, so the conditional assignment is practically speaking always "static" anyway by the time you do branch following.

So I'll just send out a minimal patch with *only* the %esi changes.

Linus

#16 lore raw
In reply to Linus Torvalds
Quoted 20 lines · Linus Torvalds

[ Sorry for not being hugely responsive the last few days - I was on the road for a family get-together on the east coast, so I spent time on airplanes and hotels and I don't particularly enjoy working with a laptop ]

Quoted 10 lines · Uros Bizjak
Quoted 4 lines

I'm sending the patch out in the hope that another set of eyes will make it actually better.

+ _lock "cmpxchg8b 0(%[ptr])", X86_FEATURE_CX8) \

This can be just:

+ _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \

Thanks, yup, will fix.

Quoted 7 lines
Quoted 5 lines

- if (unlikely(!ret)) \
- *(_oldp) = o.full; \
+ *(_oldp) = o; \

This one should really update only when cmpxchg fails.

The thing is, when cmpxchg doesn't fail, then oldp should already be "old", no?

Also replying here with the reply to your proposed patch, to document a subtle issue with try_cmpxchg:

You probably want to look at 44fe84459faf1 ("locking/atomic: Fix atomic_try_cmpxchg() semantics") [1] and the long LKML discussion at [2].

--quote--
This code is broken with the current implementation, the problem is with unconditional update of *__po.

In case of success it writes the same value back into *__po, but in case of cmpxchg success we might have lose ownership of some memory locations and potentially over what __po has pointed to. The same holds for the re-read of *__po.
--/quote--

[1] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?id=44fe84459faf1a7781595b7c64cd36daf2f2827d
[2] https://lore.kernel.org/lkml/CACT4Y+bG+a0w6j6v1AmBE7fqqMSPyPEm4QimCzCouicmHT8FqA@mail.gmail.com/

Uros.

Quoted 49 lines

I mean, by the very definition, atomic_try_cmpxchg() can *not* be successful if the new value didn't match the old one.

I mean, just look at the very doc you point to - the "definition" is

  bool atomic_try_cmpxchg(atomic_t *ptr, int *oldp, int new)
  {
    int ret, old = *oldp;
    ret = atomic_cmpxchg(ptr, old, new);
    if (ret != old)
      *oldp = ret;
    return ret == old;
  }

iow, it only returns success of "ret == old", and "old" by definition is "the contents of oldp".

(Here "oldp" is a local variable, not something that can be changing).

So I *think* the whole

    if (ret != old)
      *oldp = ret;

is actually counter-productive, and could/should be just that simpler unconditional *oldp = ret, because you have two cases:

- ret == old: the assignment doesn't change anything and is a no-op

- ret !=- old: the assignment needs to be done

but doing it *unconditionally* means that now as fat as the compiler is concerned, the original *oldp value is unconditionally dead, which sounds to me like it should be good for register allocation (the context here being that it _looks_ like a pointer access, but it's really meant to be a "in-out argument in a register").

Now, in practice, I suspect that everybody checks the return value and "old" is never used afterwards in the success case, so in that sense this doesn't matter and it's all dead regardless.

But it seems to be a complication in the docs and the implementation.

Of course, I may be missing something completely obvious, and/or you have some subtle code generation reason why you prefer the conditional there. Feel free to explain,

Linus

#17 lore raw
In reply to Linus Torvalds
Quoted 1 line · Linus Torvalds

The thing is, when cmpxchg doesn't fail, then oldp should already be "old", no?

Correct.

Quoted 19 lines

I mean, by the very definition, atomic_try_cmpxchg() can *not* be successful if the new value didn't match the old one.

I mean, just look at the very doc you point to - the "definition" is

  bool atomic_try_cmpxchg(atomic_t *ptr, int *oldp, int new)
  {
    int ret, old = *oldp;
    ret = atomic_cmpxchg(ptr, old, new);
    if (ret != old)
      *oldp = ret;
    return ret == old;
  }

iow, it only returns success of "ret == old", and "old" by definition is "the contents of oldp".

(Here "oldp" is a local variable, not something that can be changing).

Quoted 7 lines

So I *think* the whole

    if (ret != old)
      *oldp = ret;

is actually counter-productive, and could/should be just that simpler unconditional *oldp = ret,

IIRC the reason I added that conditional is because at the time the GCC compiler I tried it on generated slightly better code like this.

ISTR it emitting some superfluous assignments with the unconditional store variant. Typically what seemed to happen is that since the cmpxchg() user would have a loop termination on ret == old, it was able to recognise it only needed that assignment in the failure case. Without the condition on it would also do that assignment in the success case.

But yeah, otherwise it doesn't matter.

#18 lore raw
Quoted 28 lines · Peter Zijlstra
Quoted 1 line · Linus Torvalds

The thing is, when cmpxchg doesn't fail, then oldp should already be "old", no?

Correct.

Quoted 21 lines

I mean, by the very definition, atomic_try_cmpxchg() can *not* be successful if the new value didn't match the old one.

I mean, just look at the very doc you point to - the "definition" is

  bool atomic_try_cmpxchg(atomic_t *ptr, int *oldp, int new)
  {
    int ret, old = *oldp;
    ret = atomic_cmpxchg(ptr, old, new);
    if (ret != old)
      *oldp = ret;
    return ret == old;
  }

iow, it only returns success of "ret == old", and "old" by definition is "the contents of oldp".

(Here "oldp" is a local variable, not something that can be changing).

So I *think* the whole

Quoted 8 lines
Quoted 5 lines
    if (ret != old)
      *oldp = ret;

is actually counter-productive, and could/should be just that simpler unconditional *oldp = ret,

IIRC the reason I added that conditional is because at the time the GCC compiler I tried it on generated slightly better code like this.

Please see the thread at [1].

[1] https://lore.kernel.org/lkml/CACT4Y+bG+a0w6j6v1AmBE7fqqMSPyPEm4QimCzCouicmHT8FqA@mail.gmail.com/

Uros.

Quoted 8 lines

ISTR it emitting some superfluous assignments with the unconditional store variant. Typically what seemed to happen is that since the cmpxchg() user would have a loop termination on ret == old, it was able to recognise it only needed that assignment in the failure case. Without the condition on it would also do that assignment in the success case.

But yeah, otherwise it doesn't matter.

#20 lore raw

PATCH v2 1/1 x86-32: fix cmpxchg8b_emu build error with clang

In reply to Linus Torvalds

The kernel test robot reported that clang no longer compiles the 32-bit x86 kernel in some configurations due to commit 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions").

The build fails with

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

and the reason seems to be that not only does the cmpxchg8b instruction need four fixed registers (EDX:EAX and ECX:EBX), with the emulation fallback the inline asm also wants a fifth fixed register for the address (it uses %esi for that, but that's just a software convention with cmpxchg8b_emu).

Avoiding using another pointer input to the asm (and just forcing it to use the "0(%esi)" addressing that we end up requiring for the sw fallback) seems to fix the issue.

Fixes 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions")

---

This is the minimal patch, with the other simplifications removed.

I still think that the case where somebody passes in a "oldp" that can change during the cmpxchg is entirely broken, and is not actually valid.

If it was valid, we'd have to use READ_ONCE() and WRITE_ONCE() in the "oldp" updates anyway just to make verification tools happy. We don't.

But hey, that simplification doesn't matter for fixing this build issue, so let's keep the change minimal.

 arch/x86/include/asm/cmpxchg_32.h | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)
arch/x86/include/asm/cmpxchg_32.h +5 -7
index ed2797f132ce..62cef2113ca7 100644
@@ -93,10 +93,9 @@ static __always_inline bool __try_cmpxchg64_local(volatile u64 *ptr, u64 *oldp,
 									\
 	asm volatile(ALTERNATIVE(_lock_loc				\
 				 "call cmpxchg8b_emu",			\
-				 _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
-		     : [ptr] "+m" (*(_ptr)),				\
-		       "+a" (o.low), "+d" (o.high)			\
-		     : "b" (n.low), "c" (n.high), "S" (_ptr)		\
+				 _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
+		     : "+a" (o.low), "+d" (o.high)			\
+		     : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)	\
 		     : "memory");					\
 									\
 	o.full;								\
@@ -122,12 +121,11 @@ static __always_inline u64 arch_cmpxchg64_local(volatile u64 *ptr, u64 old, u64
 									\
 	asm volatile(ALTERNATIVE(_lock_loc				\
 				 "call cmpxchg8b_emu",			\
-				 _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
+				 _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
 		     CC_SET(e)						\
 		     : CC_OUT(e) (ret),					\
-		       [ptr] "+m" (*(_ptr)),				\
 		       "+a" (o.low), "+d" (o.high)			\
-		     : "b" (n.low), "c" (n.high), "S" (_ptr)		\
+		     : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)	\
 		     : "memory");					\
 									\
 	if (unlikely(!ret))						\
Signature · 1 line
2.45.1.209.gc6f12300df
#21 lore raw

Re: [PATCH v2] x86-32: fix cmpxchg8b_emu build error with clang

Quoted 20 lines · Linus Torvalds

The kernel test robot reported that clang no longer compiles the 32-bit x86 kernel in some configurations due to commit 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions").

The build fails with

arch/x86/include/asm/cmpxchg_32.h:149:9: error: inline assembly requires more registers than available

and the reason seems to be that not only does the cmpxchg8b instruction need four fixed registers (EDX:EAX and ECX:EBX), with the emulation fallback the inline asm also wants a fifth fixed register for the address (it uses %esi for that, but that's just a software convention with cmpxchg8b_emu).

Avoiding using another pointer input to the asm (and just forcing it to use the "0(%esi)" addressing that we end up requiring for the sw fallback) seems to fix the issue.

Quoted 6 lines
Fixes 95ece48165c1 ("locking/atomic/x86: Rewrite x86_32 arch_atomic64_{,fetch}_{and,or,xor}() functions")
Reviewed-and-Tested-by Uros Bizjak <[email protected]>
Quoted 52 lines

---

This is the minimal patch, with the other simplifications removed.

I still think that the case where somebody passes in a "oldp" that can change during the cmpxchg is entirely broken, and is not actually valid.

If it was valid, we'd have to use READ_ONCE() and WRITE_ONCE() in the "oldp" updates anyway just to make verification tools happy. We don't.

But hey, that simplification doesn't matter for fixing this build issue, so let's keep the change minimal.

 arch/x86/include/asm/cmpxchg_32.h | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)
arch/x86/include/asm/cmpxchg_32.h +5 -8
index ed2797f132ce..62cef2113ca7 100644
@@ -93,10 +93,9 @@ static __always_inline bool __try_cmpxchg64_local(volatile u64 *ptr, u64 *oldp,
                                                                        \
        asm volatile(ALTERNATIVE(_lock_loc                              \
                                 "call cmpxchg8b_emu",                  \
-                                _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
-                    : [ptr] "+m" (*(_ptr)),                            \
-                      "+a" (o.low), "+d" (o.high)                      \
-                    : "b" (n.low), "c" (n.high), "S" (_ptr)            \
+                                _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
+                    : "+a" (o.low), "+d" (o.high)                      \
+                    : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)      \
                     : "memory");                                       \
                                                                        \
        o.full;                                                         \
@@ -122,12 +121,11 @@ static __always_inline u64 arch_cmpxchg64_local(volatile u64 *ptr, u64 old, u64
                                                                        \
        asm volatile(ALTERNATIVE(_lock_loc                              \
                                 "call cmpxchg8b_emu",                  \
-                                _lock "cmpxchg8b %[ptr]", X86_FEATURE_CX8) \
+                                _lock "cmpxchg8b %a[ptr]", X86_FEATURE_CX8) \
                     CC_SET(e)                                          \
                     : CC_OUT(e) (ret),                                 \
-                      [ptr] "+m" (*(_ptr)),                            \
                       "+a" (o.low), "+d" (o.high)                      \
-                    : "b" (n.low), "c" (n.high), "S" (_ptr)            \
+                    : "b" (n.low), "c" (n.high), [ptr] "S" (_ptr)      \
                     : "memory");                                       \
                                                                        \
        if (unlikely(!ret))                                             \
--

2.45.1.209.gc6f12300df

#22 lore raw

Re: [PATCH v2] x86-32: fix cmpxchg8b_emu build error with clang

Quoted 2 lines · Uros Bizjak
Reviewed-and-Tested-by Uros Bizjak <[email protected]>

I ended up committing this directly for rc6 as commit 769327258a14 ("x86-32: fix cmpxchg8b_emu build error with clang") since nobody else seemed to react much and it looked obvious enough.

Linus

#23 lore raw
In reply to Linus Torvalds
Quoted 12 lines · Linus Torvalds
Quoted 2 lines · Uros Bizjak

I disagree with the above.

Your disagreement just doesn't matter.

We don't introduce regressions and then blame others.

There's a very clear rule in kernel development: things that break other things ARE NOT FIXES.

Quoted 7 lines

EVER.

They get reverted, or the thing they broke gets fixed.

This is not debatable, or something you can "disagree"' with. This is how we work, and if you disagree with that, you had better get out of kernel development.

I disagree with *I tried to shift the blame to others*! I take full responsibility for my patch, and I'm perfectly capable of fixing the breakage with an alternative approach.

I'm OK with the revert, but it won't fix the underlying problem. Please see the definition of __arch_cmpxchg64_emu - it forces the address to %esi registers in the same way as __arch_try_cmpxchg64_emu. Effectively, the compiler allocates 5 input registers just for the instruction.

The alternative is to implement atomic64_{and,or,xor} as an outline function, in the same way as e.g. arch_atomic64_add is implemented. This will avoid the call to __arch_try_cmpxchg64_emu, and the whole "instruction" will use just:

__alternative_atomic64(add, add_return,

      ASM_OUTPUT2("+A" (i), "+c" (v)),
      ASM_NO_INPUT_CLOBBER("memory"));
Quoted 2 lines

Now, from having looked a bit at this, I can point you to the differences introduced by having to have the emulation fallback.

Yes, I know this - I also (runtime!) tested the emulation, but with GCC only.

Quoted 5 lines

And I would personally be ok with leaving Pentium etc entirely behind, and getting rid of said fallback, but that's an entirely separate thing that will require much more discussion.

Some people still wanted to support old 486 clones not that long ago.

This can be achieved by implementing atomic64_{and,or,xor} as an outline function.

Uros.

#24 lore raw
Quoted 4 lines · Uros Bizjak

I disagree with *I tried to shift the blame to others*! I take full responsibility for my patch, and I'm perfectly capable of fixing the breakage with an alternative approach.

Ah, good. This is something I'm a bit sensitive about, just because there's been so many arguments over it over the years, so now I go into "preemptive nuclear mode" when the regression issue comes up.

Sorry.

Quoted 5 lines

I'm OK with the revert, but it won't fix the underlying problem. Please see the definition of __arch_cmpxchg64_emu - it forces the address to %esi registers in the same way as __arch_try_cmpxchg64_emu. Effectively, the compiler allocates 5 input registers just for the instruction.

Oh, I entirely agree that this is a "random compiler implementation" issue, and then the code around it makes all the difference.

Quoted 4 lines
Quoted 2 lines

Now, from having looked a bit at this, I can point you to the differences introduced by having to have the emulation fallback.

Yes, I know this - I also (runtime!) tested the emulation, but with GCC only.

Yeah, crossed emails, I started out just doing the "let's see what the config difference is", and only after that realized that I had looked at the wrong code for cmpxchg (ie I had looked at the simpler native case).

Quoted 2 lines

This can be achieved by implementing atomic64_{and,or,xor} as an outline function.

Yes, but then a lot of the whole point of commit 95ece48165c1 goes away, doesn't it?

Or were you suggesting the out-of-line code only for the emulation case? That would work.

Linus

#25 lore raw
Quoted 5 lines · Linus Torvalds

Ah, good. This is something I'm a bit sensitive about, just because there's been so many arguments over it over the years, so now I go into "preemptive nuclear mode" when the regression issue comes up.

Sorry.

Also sorry from my side if my disagreement was understood as a criticism of the kernel development process. I'm totally OK with it (but ignored patches indeed cause a bit of frustration...).

Quoted 19 lines
Quoted 5 lines

I'm OK with the revert, but it won't fix the underlying problem. Please see the definition of __arch_cmpxchg64_emu - it forces the address to %esi registers in the same way as __arch_try_cmpxchg64_emu. Effectively, the compiler allocates 5 input registers just for the instruction.

Oh, I entirely agree that this is a "random compiler implementation" issue, and then the code around it makes all the difference.

Quoted 4 lines
Quoted 2 lines

Now, from having looked a bit at this, I can point you to the differences introduced by having to have the emulation fallback.

Yes, I know this - I also (runtime!) tested the emulation, but with GCC only.

Yeah, crossed emails, I started out just doing the "let's see what the config difference is", and only after that realized that I had looked at the wrong code for cmpxchg (ie I had looked at the simpler native case).

Quoted 5 lines
Quoted 2 lines

This can be achieved by implementing atomic64_{and,or,xor} as an outline function.

Yes, but then a lot of the whole point of commit 95ece48165c1 goes away, doesn't it?

True, because this commit pushed one of the compilers over the edge.

Quoted 2 lines

Or were you suggesting the out-of-line code only for the emulation case? That would work.

I am suggesting simply following the approach of arch_atomic64_{add,sub}{_return} in atomic64_32.h. These functions are used extensively in the kernel, and if they didn't cause any problems, then we can rightfully expect that the new ones also won't. This approach will relax the register pressure, so we won't expect magic from the compiler.

I can provide a patch series with the revert and a fix in a couple of days (I'll be away from the keyboard for a short time).

Uros.