using gcc built-ins for bitops?
Subsystems (2)
Hi,
gcc 3.4 gained support for several typical bitops as builtin directives. Using these over inline asm has a few advantages:
* gcc can optimize constants into these better
* gcc can reorder and schedule the code better
* gcc can allocate registers etc better for the code
The question is if we consider it desirable to go down this road or not. In order to help that discussion I've attached a patch below that switches the i386 ffz() function to the gcc builtin version, conditional on gcc having support for this. Before I go down the road of converting more functions and/or architectures.... is this worth doing?
Greetings,
Arjan van de Ven
include/asm-i386/bitops.h renamed from include/asm-i386/bitops.h~ +8 -0
@@ -344,6 +344,8 @@* * Undefined if no zero exists, so code should check against ~0UL first. */ + +#ifndef HAVE_BUILTIN_CTZL static inline unsigned long ffz (unsigned long word) { __asm__("bsfl %1,%0"
@@ -351,6 +353,12 @@ :"r" (~word)); return word; } +#else +static inline unsigned long ffz (unsigned long word) +{ +return __builtin_ctzl (~word); +} +#endif /** * __ffs - find first bit in word.
include/linux/compiler-gcc3.h renamed from include/linux/compiler-gcc3.h~ +5 -0
@@ -19,6 +19,11 @@# define __attribute_used__ __attribute__((__unused__))#endif +#if __GNUC_MINOR__ >=4 +#define HAVE_BUILTIN_CTZL +#endif + +#define __attribute_pure__ __attribute__((pure))#define __attribute_const__ __attribute__((__const__))
Quoted 7 lines · Arjan van de Ven
gcc 3.4 gained support for several typical bitops as builtin directives.
Using these over inline asm has a few advantages:
* gcc can optimize constants into these better
* gcc can reorder and schedule the code better
* gcc can allocate registers etc better for the code
Quoted 5 lines
The question is if we consider it desirable to go down this road or not. In order to help that discussion I've attached a patch below that switches the i386
ffz()function to the gcc builtin version, conditional on gcc having support for this. Before I go down the road of converting more functions and/or architectures.... is this worth doing?
I guess it depends on the resulting code size and quality. Some extra conversions would be needed for that.
For the implementation it would be nice to have the old-style implementations in one header and the new-style ones in a separate header. That would create a bit of an all-or-nothing situation, but that should be OK?
Quoted 4 lines
+static inline unsigned long ffz (unsigned long word) +{
+ return__builtin_ctzl(~word);
+}
eww, whitepsace innovations.
static inline unsigned long ffz(unsigned long word)
{
return __builtin_ctzl(~word);
}Quoted 4 lines · Andrew Morton
Quoted 1 line
and/or architectures.... is this worth doing?
I guess it depends on the resulting code size and quality. Some extra conversions would be needed for that.
ok I'll see if I can get some detailed info on that
Quoted 5 lines
For the implementation it would be nice to have the old-style implementations in one header and the new-style ones in a separate header. That would create a bit of an all-or-nothing situation, but that should be OK?
Perhaps. It's not impossible that say gcc 3.5 will add a few more builtins even that then allow more functions to be converted, otoh that shouldn't be impossible to cope with. I'll have a look to see how it pans out.
Greetings,
Arjan van de Ven
Quoted 3 lines
Perhaps. It's not impossible that say gcc 3.5 will add a few more builtins even that then allow more functions to be converted, otoh that shouldn't be impossible to cope with. I'll have a look to see how it pans out.
You could have an asm-generic/bitops-builtin.h and arch's could #include that after defining all the HAVE_BUILTIN_xxx macros they want. I suspect not all architectures will get the most correct built-ins (e.g. the arch may be able to optimize better than gcc's builtin is doing).
Signature · 4 lines
Debian - http://www.debian.org/ Linux 1394 - http://www.linux1394.org/ Subversion - http://subversion.tigris.org/ WatchGuard - http://www.watchguard.com/
Quoted 8 lines · Andrew Morton
Quoted 7 lines · Arjan van de Ven
gcc 3.4 gained support for several typical bitops as builtin directives.
Using these over inline asm has a few advantages:
* gcc can optimize constants into these better
* gcc can reorder and schedule the code better
* gcc can allocate registers etc better for the code
Quoted 8 lines
Quoted 5 lines
The question is if we consider it desirable to go down this road or not. In order to help that discussion I've attached a patch below that switches the i386
ffz()function to the gcc builtin version, conditional on gcc having support for this. Before I go down the road of converting more functions and/or architectures.... is this worth doing?I guess it depends on the resulting code size and quality. Some extra conversions would be needed for that.
for ffz() the exact same assembly instructions are generated in the cases I looked at (kernel/signal.c); eg no extra code at all.
Quoted 4 lines · Andrew Morton
For the implementation it would be nice to have the old-style implementations in one header and the new-style ones in a separate header. That would create a bit of an all-or-nothing situation, but that should be OK?
In addition I stuck those in asm-generic since they no longer are architecture specific....
include/asm-i386/bitops.h +6 -47
@@ -338,34 +338,6 @@ static inline int find_first_bit(const u*/ int find_next_bit (const unsigned long *addr,int size,int offset); -/** -* ffz - find first zero in word. -* @word: The word to search -* -* Undefined if no zero exists, so code should check against ~0UL first. -*/ -static inline unsigned long ffz (unsigned long word) -{ - __asm__("bsfl %1,%0" - :"=r" (word) - :"r" (~word)); -return word; -} - -/** -* __ffs - find first bit in word. -* @word: The word to search -* -* Undefined if no bit exists, so code should check against 0 first. -*/ -static inline unsigned long __ffs (unsigned long word) -{ - __asm__("bsfl %1,%0" - :"=r" (word) - :"rm" (word)); -return word; -} -/* * fls: find last bit set. */
@@ -374,6 +346,12 @@ static inline unsigned long __ffs(unsign#ifdef __KERNEL__ +#ifdef USE_BUILTIN_BITOPS +#include <asm-generic/bitops_gcc.h> +#else +#include <asm/bitops_asm.h> +#endif +/* * Every architecture must define this function. It's the fastest * way of searching a 140-bit bitmap where the first 100 bits are
@@ -394,25 +372,6 @@ static inline int sched_find_first_bit(c }/** -* ffs - find first bit set -* @x: the word to search -* -* This is defined the same way as -* the libc and compiler builtin ffs routines, therefore -* differs in spirit from the above ffz (man ffs). -*/ -static inline int ffs (int x) -{ -int r; - - __asm__("bsfl %1,%0\n\t" -"jnz 1f\n\t" -"movl $-1,%0\n" -"1:" :"=r" (r) :"rm" (x)); -return r+1 ; -} - -/** * hweightN - returns the hamming weight of a N-bit word * @x: the word to weigh *
include/asm-generic/bitops_gcc.h +61 -0
61 lines not on this page · raw message
include/asm-i386/bitops_asm.h +56 -0
56 lines not on this page · raw message
include/linux/compiler-gcc+.h +1 -0
@@ -6,6 +6,7 @@*/ #include <linux/compiler-gcc.h> +#define USE_BUILTIN_BITOPS #define inline __inline__ __attribute__((always_inline))#define __inline__ __inline__ __attribute__((always_inline))#define __inline __inline__ __attribute__((always_inline))
include/linux/compiler-gcc3.h +4 -0
@@ -19,6 +19,10 @@# define __attribute_used__ __attribute__((__unused__))#endif +#if __GNUC_MINOR__ >=4 +#define USE_BUILTIN_BITOPS +#endif +#define __attribute_pure__ __attribute__((pure))#define __attribute_const__ __attribute__((__const__))
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines · Andrew Morton
For the implementation it would be nice to have the old-style implementations in one header and the new-style ones in a separate header. That would create a bit of an all-or-nothing situation, but that should be OK?
In addition I stuck those in asm-generic since they no longer are architecture specific....
This is not going to work.
Say on x86_64, __builtin_ctzl (~word) ends up __ctzdi2 (~word) call in GCC 3.4.x, which is not defined in the kernel (in 3.5 it will be bsfq). On a bunch of arches which don't have an instruction for ffz operation it will always result in a library call.
Jakub
Quoted 7 lines · Jakub Jelinek
Quoted 6 lines · Arjan van de Ven
Quoted 4 lines · Andrew Morton
For the implementation it would be nice to have the old-style implementations in one header and the new-style ones in a separate header. That would create a bit of an all-or-nothing situation, but that should be OK?
Quoted 8 lines
Quoted 2 lines
In addition I stuck those in asm-generic since they no longer are architecture specific....
This is not going to work.
Say onx86_64,__builtin_ctzl(~word) ends up__ctzdi2(~word) call in GCC 3.4.x, which is not defined in the kernel (in 3.5 it will be bsfq). On a bunch of arches which don't have an instruction for ffz operation it will always result in a library call.
It's actually fine; the architecture first needs to include this file and there it can use the proper ifdefs; the functions themselves don't matter, only when they can be used, and the arch still controls that.
Quoted 4 lines · Arjan van de Ven
It's actually fine; the architecture first needs to include this file and there it can use the proper ifdefs; the functions themselves don't matter, only when they can be used, and the arch still controls that.
And my argument is: what the hell does this _buy_ us, except for extra complexity, and even more code dependence on different versions of gcc.
I don't want the extra code-paths and magic #ifdef's unless there's a clear improvement somewhere. And quite frankly, I don't see that being the case for something like ffs.
Linus
I see a list of these gcc bitop builtins at the bottom of the page:
http://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
Looks like you can find the position of the first 1 bit, the length of the leading or trailing seq of 0 bits, the hamming weight (popcount) and the parity, each for int, long and long long.
I just add this for the benefit of others.
As to your primary question - is this worth doing - I don't have an answer.
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373Quoted 4 lines · Paul Jackson
I see a list of these gcc bitop builtins at the bottom of the page:
Quoted 8 lines
Looks like you can find the position of the first 1 bit, the length of the leading or trailing seq of 0 bits, the hamming weight (popcount) and the parity, each for int, long and long long.
I just add this for the benefit of others.
As to your primary question - is this worth doing - I don't have an answer.
It is, for 2 reasons:
1) unlike __asm, GCC knows how to schedule the instructions in the builtins
2) GCC will handle stuff like ffz (16) at compile time rather than runtime
But, all the builtins are not natively supported on every architecture, if there is no arch support, it falls back to a libgcc library function, which the kernel probably wants to avoid.
E.g. popcount on i386, etc.
Jakub
Quoted 7 lines · Jakub Jelinek
Quoted 3 lines
As to your primary question - is this worth doing - I don't have an answer.
It is, for 2 reasons:
1) unlike__asm, GCC knows how to schedule the instructions in the builtins
2) GCC will handle stuff like ffz (16) at compile time rather than runtime
I'd argue that unless the gcc code can be shown to be clearly better, it is NOT worth doing.
Adding support for built-ins generates a noticeable maintenance burden, since we'll still have to support older gcc's and architectures where gcc does WORSE than we do. And quite frankly, I doubt you'll find any cases where gcc does any better in any measurable way.
In other words, the rule about gcc builtins should be:
- use them only if they are old enough that the huge majority (possibly all) of users get them. This is partly because gcc has frankly been buggy, and often makes assumptions that simply may not be true for the kernel (ie "it's ok to use library routines").
- only use them if you can show a measurable improvement. Theoretical arguments simply don't count. Theoretical arguments is why gcc-3.x is a lot slower than 2.95 and is apparently still not generating appreciably better code for the kernel (and no, don't bother pointing me to spec runs, I just don't care. The kernel is what I care about).
So far, the only case where they have been worth it is likely the "memcpy()" stuff, and even there our previous macros were doing an almost equivalent job (but were arguably so ugly that it was worth making the change).
For something like ffs/popcount/etc, I do not see _any_ point to compiler support. There just aren't any kernel uses that make it worth it. Sounds like a total micro-optimization for some spec benchmark.
Linus
Quoted 45 lines · Arjan van de Ven
Hi,
gcc 3.4 gained support for several typical bitops as builtin directives. Using these over inline asm has a few advantages:
* gcc can optimize constants into these better
* gcc can reorder and schedule the code better
* gcc can allocate registers etc better for the codeThe question is if we consider it desirable to go down this road or not. In order to help that discussion I've attached a patch below that switches the i386
ffz()function to the gcc builtin version, conditional on gcc having support for this. Before I go down the road of converting more functions and/or architectures.... is this worth doing?Greetings,
Arjan van de Veninclude/asm-i386/bitops.h renamed from include/asm-i386/bitops.h~ +8
-0@@ -344,6 +344,8 @@* * Undefined if no zero exists, so code should check against ~0UL first. */ + +#ifndef HAVE_BUILTIN_CTZL static inline unsigned long ffz (unsigned long word) { __asm__("bsfl %1,%0"@@ -351,6 +353,12 @@ :"r" (~word)); return word; } +#else +static inline unsigned long ffz (unsigned long word) +{ +return __builtin_ctzl (~word); +} +#endif /** * __ffs - find first bit in word.
include/linux/compiler-gcc3.h line 22
Quoted 4 lines
include/linux/compiler-gcc3.h renamed from include/linux/compiler-gcc3.h~ +1
-0@@ -19,6 +19,11 @@# define __attribute_used__ __attribute__((__unused__))#endif +#if __GNUC_MINOR__ >=4
Please do no test only on minor. People have been arguing whether the next major release of GCC should be called 3.5 or 4.0 since tree-ssa has been merged.
This said, I'd rather use GCC intrinsics when they can perform the task, especially the ones that expand to more than one machine instruction.
include/linux/compiler-gcc3.h line 24
Quoted 2 lines
include/linux/compiler-gcc3.h renamed from include/linux/compiler-gcc3.h~ +2
-0+#define HAVE_BUILTIN_CTZL +#endif
Gabriel
Quoted 11 lines · Gabriel Paubert
include/linux/compiler-gcc3.h line 22
Quoted 7 lines
include/linux/compiler-gcc3.h renamed from include/linux/compiler-gcc3.h~ +1
-0@@ -19,6 +19,11 @@# define __attribute_used__ __attribute__((__unused__))#endif +#if __GNUC_MINOR__ >=4 Please do no test only on minor. People have been arguing whether the next major release of GCC should be called 3.5 or 4.0 since tree-ssa has been merged.
this header is only uses for the gcc major == 3. The header for ==4 and later has the define unconditional, and the file for major ==2 doesn't have it at all.