Regression due to 0c44c2d0f459 x86: Use asm goto to implement better modify_and_test() functions
Subsystems (3)
H. Peter Anvin
reviewerHi guys,
0c44c2d0f459 x86: Use asm goto to implement better modify_and_test() functions
causes a regression, because it incorrectly changed the constraints of bitops.
Specifically, the GEN_BINARY_RMWcc() hardcodes a constraint as "er", but it needs to be "Ir" for the bitops themselves. "I" is correct (as opposed to "J" even on 64 bits, because we only generate the 64-bit version when we have a register operand.
Unfortunately there isn't a way we can get gcc+gas to generate a version with an offset pointer.
-hpa
Quoted 8 lines · H. Peter Anvin
Hi guys,
0c44c2d0f459x86: Use asm goto to implement bettermodify_and_test()functionscauses a regression, because it incorrectly changed the constraints of bitops.
Quoted 7 lines
Specifically, the
GEN_BINARY_RMWcc()hardcodes a constraint as "er", but it needs to be "Ir" for the bitops themselves. "I" is correct (as opposed to "J" even on 64 bits, because we only generate the 64-bit version when we have a register operand.Unfortunately there isn't a way we can get gcc+gas to generate a version with an offset pointer.
Does the regression manifest itself in any actual breakage - if yes, how does it look like? (People experiencing similar symptoms will be helped by seeing a fix matching their problems.)
Thanks,
ngo
H. Peter Anvin
reviewerQuoted 16 lines · Ingo Molnar
Quoted 13 lines · H. Peter Anvin
Hi guys,
0c44c2d0f459x86: Use asm goto to implement bettermodify_and_test()functionscauses a regression, because it incorrectly changed the constraints of bitops.
Specifically, the
GEN_BINARY_RMWcc()hardcodes a constraint as "er", but it needs to be "Ir" for the bitops themselves. "I" is correct (as opposed to "J" even on 64 bits, because we only generate the 64-bit version when we have a register operand.
Quoted 7 lines
Quoted 2 lines
Unfortunately there isn't a way we can get gcc+gas to generate a version with an offset pointer.
Does the regression manifest itself in any actual breakage - if yes, how does it look like? (People experiencing similar symptoms will be helped by seeing a fix matching their problems.)
It was discovered because it caused a build failure in a not-yet-submitted driver patch. This happens when someone uses test_and_set_bit() or another similar operation on a fixed bit index above 255; the assembler throws an error at that point and the build fails.
*HOWEVER*, for bit indicies in the range 32-255, the current code will instead silently miscompile, as the CPU will truncate the argument to 5 bits. I don't know if there are any such instances in the current kernel, but it is entirely possible there is, with unknown but potentially disastrous results.
-hpa
Quoted 30 lines · H. Peter Anvin
Quoted 16 lines · Ingo Molnar
Quoted 13 lines · H. Peter Anvin
Hi guys,
0c44c2d0f459x86: Use asm goto to implement bettermodify_and_test()functionscauses a regression, because it incorrectly changed the constraints of bitops.
Specifically, the
GEN_BINARY_RMWcc()hardcodes a constraint as "er", but it needs to be "Ir" for the bitops themselves. "I" is correct (as opposed to "J" even on 64 bits, because we only generate the 64-bit version when we have a register operand.Quoted 7 lines
Quoted 2 lines
Unfortunately there isn't a way we can get gcc+gas to generate a version with an offset pointer.
Does the regression manifest itself in any actual breakage - if yes, how does it look like? (People experiencing similar symptoms will be helped by seeing a fix matching their problems.)
It was discovered because it caused a build failure in a not-yet-submitted driver patch. This happens when someone uses
test_and_set_bit()or another similar operation on a fixed bit index above 255; the assembler throws an error at that point and the build fails.
Quoted 5 lines
*HOWEVER*, for bit indicies in the range 32-255, the current code will instead silently miscompile, as the CPU will truncate the argument to 5 bits. I don't know if there are any such instances in the current kernel, but it is entirely possible there is, with unknown but potentially disastrous results.
I'm seeing reports from folks using vfio for PCI device assignment through QEMU that hugepages aren't being released on VM shutdown and reverting 0c44c2d0f459 resolves the problem. I haven't confirmed this myself nor do I pretend to understand the changes this commit makes. Thanks,
Alex
tip-bot for H. Peter Anvin
merge bot[tip:x86/urgent] x86, bitops: Correct the assembly constraints to testing bitops
In reply to H. Peter Anvinx86, bitops: Correct the assembly constraints to testing bitops
Commit-ID: e0f6dec35f9286e78879fe1ac92803fd69fc4fdc Gitweb: http://git.kernel.org/tip/e0f6dec35f9286e78879fe1ac92803fd69fc4fdc Author: H. Peter Anvin <[email protected]> AuthorDate: Wed, 4 Dec 2013 14:31:28 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 4 Dec 2013 14:31:28 -0800
x86, bitops: Correct the assembly constraints to testing bitops
In checkin:
0c44c2d0f459 x86: Use asm goto to implement better modify_and_test() functions
the various functions which do modify and test were unified and optimized using "asm goto". However, this change missed the detail that the bitops require an "Ir" constraint rather than an "er" constraint ("I" = integer constant from 0-31, "e" = signed 32-bit integer constant). This would cause code to miscompile if these functions were used on constant bit positions 32-255 and the build to fail if used on constant bit positions above 255.
Add the constraints as a parameter to the GEN_BINARY_RMWcc() macro to avoid this problem.
--- arch/x86/include/asm/atomic.h | 4 ++-- arch/x86/include/asm/atomic64_64.h | 4 ++-- arch/x86/include/asm/bitops.h | 6 +++--- arch/x86/include/asm/local.h | 4 ++-- arch/x86/include/asm/rmwcc.h | 8 ++++---- 5 files changed, 13 insertions(+), 13 deletions(-)
arch/x86/include/asm/atomic.h +2 -2
@@ -77,7 +77,7 @@ static inline void atomic_sub(int i, atomic_t *v)*/ static inline int atomic_sub_and_test (int i,atomic_t *v) { -GEN_BINARY_RMWcc (LOCK_PREFIX "subl" , v->counter, i,"%0" ,"e" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "subl" , v->counter,"er" , i,"%0" ,"e" ); }/**
@@ -141,7 +141,7 @@ static inline int atomic_inc_and_test(atomic_t *v)*/ static inline int atomic_add_negative (int i,atomic_t *v) { -GEN_BINARY_RMWcc (LOCK_PREFIX "addl" , v->counter, i,"%0" ,"s" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "addl" , v->counter,"er" , i,"%0" ,"s" ); }/**
arch/x86/include/asm/atomic64_64.h +2 -2
@@ -72,7 +72,7 @@ static inline void atomic64_sub(long i, atomic64_t *v)*/ static inline int atomic64_sub_and_test (long i,atomic64_t *v) { -GEN_BINARY_RMWcc (LOCK_PREFIX "subq" , v->counter, i,"%0" ,"e" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "subq" , v->counter,"er" , i,"%0" ,"e" ); }/**
@@ -138,7 +138,7 @@ static inline int atomic64_inc_and_test(atomic64_t *v)*/ static inline int atomic64_add_negative (long i,atomic64_t *v) { -GEN_BINARY_RMWcc (LOCK_PREFIX "addq" , v->counter, i,"%0" ,"s" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "addq" , v->counter,"er" , i,"%0" ,"s" ); }/**
arch/x86/include/asm/bitops.h +3 -3
@@ -205,7 +205,7 @@ static inline void change_bit(long nr, volatile unsigned long *addr)*/ static inline int test_and_set_bit (long nr,volatile unsigned long *addr) { -GEN_BINARY_RMWcc (LOCK_PREFIX "bts" , *addr, nr,"%0" ,"c" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "bts" , *addr,"Ir" , nr,"%0" ,"c" ); }/**
@@ -251,7 +251,7 @@ static inline int __test_and_set_bit(long nr, volatile unsigned long *addr)*/ static inline int test_and_clear_bit (long nr,volatile unsigned long *addr) { -GEN_BINARY_RMWcc (LOCK_PREFIX "btr" , *addr, nr,"%0" ,"c" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "btr" , *addr,"Ir" , nr,"%0" ,"c" ); }/**
@@ -304,7 +304,7 @@ static inline int __test_and_change_bit(long nr, volatile unsigned long *addr)*/ static inline int test_and_change_bit (long nr,volatile unsigned long *addr) { -GEN_BINARY_RMWcc (LOCK_PREFIX "btc" , *addr, nr,"%0" ,"c" ); +GEN_BINARY_RMWcc (LOCK_PREFIX "btc" , *addr,"Ir" , nr,"%0" ,"c" ); } static __always_inline int constant_test_bit(long nr, const volatile unsigned long *addr)
arch/x86/include/asm/local.h +2 -2
@@ -52,7 +52,7 @@ static inline void local_sub(long i, local_t *l)*/ static inline int local_sub_and_test (long i,local_t *l) { -GEN_BINARY_RMWcc (_ASM_SUB, l->a.counter, i,"%0" ,"e" ); +GEN_BINARY_RMWcc (_ASM_SUB, l->a.counter,"er" , i,"%0" ,"e" ); }/**
@@ -92,7 +92,7 @@ static inline int local_inc_and_test(local_t *l)*/ static inline int local_add_negative (long i,local_t *l) { -GEN_BINARY_RMWcc (_ASM_ADD, l->a.counter, i,"%0" ,"s" ); +GEN_BINARY_RMWcc (_ASM_ADD, l->a.counter,"er" , i,"%0" ,"s" ); }/**
arch/x86/include/asm/rmwcc.h +4 -4
@@ -16,8 +16,8 @@ cc_label: \#define GEN_UNARY_RMWcc (op, var, arg0, cc) \ __GEN_RMWcc(op " " arg0, var, cc) -#define GEN_BINARY_RMWcc (op, var, val, arg0, cc) \ - __GEN_RMWcc(op " %1, " arg0, var, cc, "er" (val)) +#define GEN_BINARY_RMWcc (op, var, vcon, val, arg0, cc) \ + __GEN_RMWcc(op " %1, " arg0, var, cc, vcon (val))#else /* !CC_HAVE_ASM_GOTO */
@@ -33,8 +33,8 @@ do { \#define GEN_UNARY_RMWcc (op, var, arg0, cc) \ __GEN_RMWcc(op " " arg0, var, cc) -#define GEN_BINARY_RMWcc (op, var, val, arg0, cc) \ - __GEN_RMWcc(op " %2, " arg0, var, cc, "er" (val)) +#define GEN_BINARY_RMWcc (op, var, vcon, val, arg0, cc) \ + __GEN_RMWcc(op " %2, " arg0, var, cc, vcon (val))#endif /* CC_HAVE_ASM_GOTO */