headers_check fixes]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Ingo Molnar
Should i perhaps not bother with the stuff below? Cannot turn off
CONFIG_HEADERS_CHECKin my builds because it can cause build failures.
I really hate the patch. I think it's fundamentally flawed. I hate scripts that test for things that are readable, and encourage people to then write crap instead.
The thing is, the headers_check stuff is just wrong if it causes these things, and I'd rather just turn it off.
If those
#ifdef CONFIG_XYZ
things result in problems, then we should just make the rule be that we turn that kind of string into
#if 0
automatically when exporting the kernel headers. IOW, just about _anything_ that headers_check complains about automatically is something that should just be _fixed_ automatically at header install time rather than make the code harder to read.
So I think it makes our headers worse. Code like
> +#ifdef __KERNEL__ > +# ifdef CONFIG_X86_BSWAP > +# define __X86_BSWAP > +# endif /* CONFIG_X86_BSWAP */ > +#endif /* __KERNEL__ */
just doesn't make sense. It doesn't make sense _inside_ the kernel, and it doesn't make sense _outside_ it either.
As far as I can tell, the header install script could literally just do something like run 'sed' over the headers as it installs them, and do something like
sed 's/\<CONFIG_[A-Z0-9_]*\>/__kernel_only__/g'
which I realize is not really the complete/correct solution (ie you could write a nicer thing that does a better job), but my point here is that rather than have scripts that _whine_ about these kinds of trivial things and cause people to write less readable header files, we should just make sure that if we can recognize them so easily, we can just fix them instead.
End result: headers that don't suck.
If the damn headers-check isn't working for people, then let's turn the thing off, not make our code look worse.
There are parts of the patches that look fine, like moving __KERNEL__ checks around a bit, and changing <asm/types.h> to <linux/types.h> which looks correct _both_ in a kernel and in a user context, but I dislike the stupid parts.
Linus
H. Peter Anvin
reviewerQuoted 9 lines · Linus Torvalds
So I think it makes our headers worse. Code like
> +#ifdef __KERNEL__ > +# ifdef CONFIG_X86_BSWAP > +# define __X86_BSWAP > +# endif /* CONFIG_X86_BSWAP */ > +#endif /* __KERNEL__ */
Quoted 8 lines
just doesn't make sense. It doesn't make sense _inside_ the kernel, and it doesn't make sense _outside_ it either.
As far as I can tell, the header install script could literally just do something like run 'sed' over the headers as it installs them, and do something like
sed 's/\<CONFIG_[A-Z0-9_]*\>/__kernel_only__/g'which I realize is not really the complete/correct solution (ie you could write a nicer thing that does a better job), but my point here is that rather than have scripts that _whine_ about these kinds of trivial things and cause people to write less readable header files, we should just make sure that if we can recognize them so easily, we can just fix them instead.
We already run the headers through unifdef, so this should be trivial to add.
The intent of headers_check is to try to catch people who put things that depend on CONFIG_* stuff in exported headers (which, as we have seen, have been too sadly common.) If we declare that the export process will treat all CONFIG_* as undefined, we do lose some coverage but potentially end up with cleaner code. Not sure which is worse...
-hpa
Quoted 6 lines · H. Peter Anvin
The intent of
headers_checkis to try to catch people who put things that depend on CONFIG_* stuff in exported headers (which, as we have seen, have been too sadly common.) If we declare that the export process will treat all CONFIG_* as undefined, we do lose some coverage but potentially end up with cleaner code. Not sure which is worse...
Do you think the "fix headers_check" patches spend lots of time analyzing things? I bet no. They just try to make the warning go away, so you don't actually end up with any more "coverage" anyway. Quite the reverse - instead of having a simple rule ("CONFIG_xyz options simply do not exist in user space"), you end up having ad-hoc hacks on a per-fix basis.
Linus
H. Peter Anvin
reviewerQuoted 3 lines · Linus Torvalds
Do you think the "fix
headers_check" patches spend lots of time analyzing things? I bet no. They just try to make the warning go away, so you don't actually end up with any more "coverage" anyway. Quite the reverse - instead of having a simple rule ("CONFIG_xyzoptions simply do not exist in user space"), you end up having ad-hoc hacks on a per-fix basis.
This is probably true. I think we should add this as one more of the preprocessing rules which we really should just do, as well as automatic mangling of integer types.
-hpa
Quoted 9 lines · H. Peter Anvin
Quoted 7 lines · Linus Torvalds
Do you think the "fix
headers_check" patches spend lots of time analyzing things? I bet no. They just try to make the warning go away, so you don't actually end up with any more "coverage" anyway. Quite the reverse - instead of having a simple rule ("CONFIG_xyzoptions simply do not exist in user space"), you end up having ad-hoc hacks on a per-fix basis.
Quoted 3 lines
This is probably true. I think we should add this as one more of the preprocessing rules which we really should just do, as well as automatic mangling of integer types.
Btw, the really scary thing is that I bet there are programs out there that "know" about kernel internals, and do things like
#define CONFIG_SMP 1 #define __KERNEL__ 1 #include <asm/atomic.h>
in order to get the atomic helpers from the kernel, and using CONFIG_xyz markers to force the exact version they want.
And we will inevitably always end up breaking stuff like that. Nothing we can do about it - in the end, users can do infinitely odd things and know about our internals, and whatever changes we do will occasionally break some of the more incestuous code.
Linus
H. Peter Anvin
reviewerQuoted 7 lines · Linus Torvalds
Quoted 4 lines
Quoted 1 line
This is probably true. I think we should add this as one more of the preprocessing rules which we really should just do, as well as automatic mangling of integer types.
Btw, the really scary thing is that I bet there are programs out there that "know" about kernel internals, and do things like
Quoted 8 lines
#define CONFIG_SMP 1 #define __KERNEL__ 1 #include <asm/atomic.h>in order to get the atomic helpers from the kernel, and using
CONFIG_xyzmarkers to force the exact version they want.And we will inevitably always end up breaking stuff like that. Nothing we can do about it - in the end, users can do infinitely odd things and know about our internals, and whatever changes we do will occasionally break some of the more incestuous code.
Yes, that's just PEBKAC.
-hpa
Quoted 4 lines · Linus Torvalds
There are parts of the patches that look fine, like moving
__KERNEL__checks around a bit, and changing <asm/types.h> to <linux/types.h> which looks correct _both_ in a kernel and in a user context, but I dislike the stupid parts.
okay - that was the general thinking and this went through a few iterations already - not enough it appears. The CONFIG_* thing was something that looked somewhat dubious to me too - changing it to those random __foo symbols didnt seem like an improvement.
Jaswider, would you mind re-doing the tree filtering out the CONFIG_* changes? I'll go over the end result once more to make sure it has no changes that make the code look worse.
I still think what i expressed elsewhere in these threads on lkml:
'exporting' 75,000 lines of random kernel headers to user-space is really stretching the term - kernel-space is unaware of it in 90% of the cases. "spilling our guts to user-space" would be a more fair description.
It would be much better if we exported _much_ less and reduced our cross-section to user-space. Also, the include/linux/Kbuild rules are all but transparent: it would also be nice if whatever we exported was be visible straight in the header itself, to make it obvious to people who modify/extend those files that those definitions are going to be exported to user-space.
Some __user_export tag on structures perhaps? I have no good ideas here - #ifdefs are ugly and tags obscure the purity of the code.
Ingo
Quoted 10 lines · Ingo Molnar
It would be much better if we exported _much_ less and reduced our cross-section to user-space. Also, the
include/linux/Kbuildrules are all but transparent: it would also be nice if whatever we exported was be visible straight in the header itself, to make it obvious to people who modify/extend those files that those definitions are going to be exported to user-space.Some
__user_exporttag on structures perhaps? I have no good ideas here - #ifdefs are ugly and tags obscure the purity of the code.
I agree, and we probably could do so with some sparse extension (just make the rule be that in order to make a __user pointer, the base type needs to have been tagged with __user_visible). So we _could_ do something like that, and have a sane checking model where sparse would warn if it sees a user pointer of something that wasn't marked as being a user-visible type.
But I doubt it's really worth it. A lot of the usage cases for users end up being about constants etc that we really can't check sanely and automatically. And some historical user-space usage is literally just about user space finding the kernel headers really helpful (ie using the list.h headers for user-space lists, or the inline asms for atomic stuff for threaded apps that don't rely on ptrace locking).
Linus
Quoted 9 lines · Ingo Molnar
It would be much better if we exported _much_ less and reduced our cross-section to user-space. Also, the
include/linux/Kbuildrules are all but transparent: it would also be nice if whatever we exported was be visible straight in the header itself, to make it obvious to people who modify/extend those files that those definitions are going to be exported to user-space.Some
__user_exporttag on structures perhaps? I have no good ideas here - #ifdefs are ugly and tags obscure the purity of the code.
Something that might or might not be doable in practice, but at least it's a suggestion that I haven't seen elsewhere:
Create an include/user/ directory that contains a "mirror" of the include/ directory _structure_, so that random exported header include/linux/foo.h now has two parts -- include/linux/foo.h and include/user/linux/foo.h.
- include/user/linux/foo.h contains the definitions that are needed by both kernel and userspace
- include/linux/foo.h contains the definitions that are needed only by the kernel
- include/linux/foo.h can simply #include <user/linux/foo.h> and no other change (to source files which _use_ this header) is necessary
- no dependency on a kernel header will exist in a "user" header -- that's how it is now, but this way is more explicit
- the whole include/user/ can be shipped verbatim to /usr/include (or wherever it is needed)
- no #ifdef __KERNEL__ or #ifdef CONFIG_ stuff in the "user" headers; no stripping or unifdefing is needed
- it's easier to see exactly what is being exported
Of course, obvious disadvantages are:
- less readable in the sense that what used to be in one file is now spread across two
- the split itself would probably require a tremendous effort
- other things?
It's just an idea...
Vegard
Signature · 4 lines
"The animistic metaphor of the bug that maliciously sneaked in while the programmer was not looking is intellectually dishonest as it disguises that the error is the programmer's own creation." -- E. W. Dijkstra, EWD1036
Hello Ingo,
Quoted 4 lines · Ingo Molnar
Jaswider, would you mind re-doing the tree filtering out the CONFIG_* changes? I'll go over the end result once more to make sure it has no changes that make the code look worse.
Sure, why not ;-)
But in your -tip tree you already have these patches. Are you going to revert them or should I make patches for linus tree.
--
JSR
Quoted 4 lines · Jaswinder Singh Rajput
Hello Ingo,
On Wed, 2009-01-28 at 00:31 +0100, Ingo Molnar wrote:
Quoted 8 lines
Quoted 3 lines
Jaswider, would you mind re-doing the tree filtering out the CONFIG_* changes? I'll go over the end result once more to make sure it has no changes that make the code look worse.
Sure, why not ;-)
But in your -tip tree you already have these patches. Are you going to revert them or should I make patches for linus tree.
yes, i'll replace that branch with your tree - the bad commits are intermixed with the good commits so reverting does not help (there would be way too many ugly reverts).
As long as you base your patches on Linus's -git tree i'll be able to sort it all out.
Ingo
Quoted 8 lines · Linus Torvalds
Quoted 3 lines · Ingo Molnar
Should i perhaps not bother with the stuff below? Cannot turn off
CONFIG_HEADERS_CHECKin my builds because it can cause build failures.So I think it makes our headers worse. Code like
Quoted 9 lines
> +#ifdef __KERNEL__ > +# ifdef CONFIG_X86_BSWAP > +# define __X86_BSWAP > +# endif /* CONFIG_X86_BSWAP */ > +#endif /* __KERNEL__ */just doesn't make sense. It doesn't make sense _inside_ the kernel, and it doesn't make sense _outside_ it either.
Or just wrap all of asm/swab.h in #ifdef __KERNEL__.
Harvey
Quoted 4 lines · Linus Torvalds
The thing is, the
headers_checkstuff is just wrong if it causes these things, and I'd rather just turn it off.
Basic idea of headers_check is:
1. clean header files so that they do not export useless things in userspace, like:
usr/include/linux/elf-fdpic.h:62: extern's make no sense in userspace
2. provide sufficient stuff like:
usr/include/asm/swab.h:7: found __[us]{8,16,32,64} type without #include <linux/types.h>
3. warns about exporting kernel space defines which are not valid in userspace and is always false for userspace usr/include/asm/swab.h:10: leaks CONFIG_X86 to userspace where it is not valid
And by headers_check tools we also able to find blunder mistakes which are very difficult to find by naked eye like:
usr/include/asm-generic/fcntl.h:127: leaks CONFIG_64BIT to userspace where it is not valid
for userspace this will be always false.
and some files was totally covered with ifdefs CONFIG_* and exported which are useless like:
usr/include/linux/if_frad.h:29: leaks CONFIG_DLCI to userspace where it is not valid
Quoted 9 lines
If those
#ifdef CONFIG_XYZthings result in problems, then we should just make the rule be that we turn that kind of string into
#if 0
This will looks ugly in userspace with so many #if 0 and make impossible to read the code.
Quoted 7 lines
automatically when exporting the kernel headers. IOW, just about _anything_ that
headers_checkcomplains about automatically is something that should just be _fixed_ automatically at header install time rather than make the code harder to read.So I think it makes our headers worse. Code like
Quoted 9 lines
> +#ifdef __KERNEL__ > +# ifdef CONFIG_X86_BSWAP > +# define __X86_BSWAP > +# endif /* CONFIG_X86_BSWAP */ > +#endif /* __KERNEL__ */just doesn't make sense. It doesn't make sense _inside_ the kernel, and it doesn't make sense _outside_ it either.
my earlier patch was like this:
arch/x86/include/asm/swab.h +4 -1
@@ -1,12 +1,15 @@#ifndef _ASM_X86_SWAB_H#define _ASM_X86_SWAB_H -#include <asm/types.h> +#include <linux/types.h> +#ifdef __KERNEL__#include <linux/compiler.h> +#endif /* __KERNEL__ */ static inline __attribute_const__ __u32 __arch_swab32(__u32 val) {#ifdef __i386__ +#ifdef __KERNEL__# ifdef CONFIG_X86_BSWAP asm("bswap %0" :"=r" (val) :"0" (val));# else
@@ -16,7 +19,13 @@ static inline __attribute_const__ __u32
__arch_swab32(__u32 val)
: "=q" (val)
: "0" (val));
# endif
-
+#else /* __KERNEL__ */
+ asm("xchgb %b0,%h0\n\t" /* swap lower bytes */
+ "rorl $16,%0\n\t" /* swap words */
+ "xchgb %b0,%h0" /* swap higher bytes */
+ : "=q" (val)
+ : "0" (val));
+#endif /* __KERNEL__ */
#else /* __i386__ */
asm("bswapl %0"
: "=r" (val)
@@ -37,6 +46,7 @@ static inline __attribute_const__ __u64
__arch_swab64(__u64 val)
__u64 u;
} v;
v.u = val;
+#ifdef __KERNEL__
# ifdef CONFIG_X86_BSWAP
asm("bswapl %0 ; bswapl %1 ; xchgl %0,%1"
: "=r" (v.s.a), "=r" (v.s.b)
@@ -48,6 +58,13 @@ static inline __attribute_const__ __u64
__arch_swab64(__u64 val)
: "=r" (v.s.a), "=r" (v.s.b)
: "0" (v.s.a), "1" (v.s.b));
# endif
+#else /* __KERNEL__ */
+ v.s.a = __arch_swab32(v.s.a);
+ v.s.b = __arch_swab32(v.s.b);
+ asm("xchgl %0,%1"
+ : "=r" (v.s.a), "=r" (v.s.b)
+ : "0" (v.s.a), "1" (v.s.b));
+#endif /* __KERNEL__ */
return v.u;
#else /* __i386__ */
asm("bswapq %0"to get rid of multiple blocks so I used above technique.
If we do not want to export __arch_swab32 and __arch_swab64 then we can put whole block in #ifdef __KERNEL__ then we will get rid of above solution.
--
JSR
Quoted 3 lines · Jaswinder Singh Rajput
If we do not want to export
__arch_swab32and__arch_swab64then we can put whole block in #ifdef__KERNEL__then we will get rid of above solution.
For the specific x86 swab code, that would certainly be the simplest way, user space should not be using those inline assemblies either way.
I think the more interesting question is whether we want to export *any* inline helpers that are not part of the ABI to user space. We already killed most of them (spinlocks, atomics, ...) and what remains is basically just the byteorder code. All that is required for the ABI is the information whether the system is big- or little-endian, but not all the rest.
Arnd <><
Quoted 5 lines · Arnd Bergmann
For the specific x86 swab code, that would certainly be the simplest way, user space should not be using those inline assemblies either way.
Quoted 7 lines
I think the more interesting question is whether we want to export *any* inline helpers that are not part of the ABI to user space. We already killed most of them (spinlocks, atomics, ...) and what remains is basically just the byteorder code. All that is required for the ABI is the information whether the system is big- or little-endian, but not all the rest.
In general, no. The byteswap API is a legacy exception.
-hpa
Quoted 14 lines · H. Peter Anvin
Quoted 5 lines · Arnd Bergmann
For the specific x86 swab code, that would certainly be the simplest way, user space should not be using those inline assemblies either way.
Quoted 7 lines
I think the more interesting question is whether we want to export *any* inline helpers that are not part of the ABI to user space. We already killed most of them (spinlocks, atomics, ...) and what remains is basically just the byteorder code. All that is required for the ABI is the information whether the system is big- or little-endian, but not all the rest.
Quoted 2 lines
In general, no. The byteswap API is a legacy exception.
But now that swab.h has been separated out, we could just stop exporting the asm/swab.h bits while still providing a generic C-based implementation to userspace.
Harvey
Quoted 7 lines · Harvey Harrison
Quoted 2 lines · H. Peter Anvin
In general, no. The byteswap API is a legacy exception.
But now that
swab.hhas been separated out, we could just stop exporting theasm/swab.hbits while still providing a generic C-based implementation to userspace.
Well, the _reason_ the byteswap stuff has been interesting to user space is that the kernel did it better than the alternatives. Rather than having purely "work with big-endian data" (the networking htonl etc functions), the kernel had good and fairly optimized handling of various different forms of byte order handling.
Which is why people wanted to use it in the first place - and which is why then doing just the generic C-based thing doesn't really fix the issue. Things may compile, but they kind of lost the point.
Linus
Quoted 16 lines · Linus Torvalds
Quoted 7 lines · Harvey Harrison
Quoted 2 lines · H. Peter Anvin
In general, no. The byteswap API is a legacy exception.
But now that
swab.hhas been separated out, we could just stop exporting theasm/swab.hbits while still providing a generic C-based implementation to userspace.Well, the _reason_ the byteswap stuff has been interesting to user space is that the kernel did it better than the alternatives. Rather than having purely "work with big-endian data" (the networking htonl etc functions), the kernel had good and fairly optimized handling of various different forms of byte order handling.
Quoted 3 lines
Which is why people wanted to use it in the first place - and which is why then doing just the generic C-based thing doesn't really fix the issue. Things may compile, but they kind of lost the point.
There was at least some discussion on the gcc-list in November about recognizing the byteswap idiom and inserting optimized instructions if available...so hopefully in the future this just fixes itself.
For now, the problem is with arches like X86 that need to test the availability of an instruction. So the arch versions could be unconditionally offered on X86_64 and the lowest-common denominator (no BSWAP) on X86-32. If we still want the optimized (BSWAP) versions on X86-32 the tests will have to use the compiler arch flags as opposed to CONFIG_BSWAP....which is probably not worth the trouble.
Harvey
H. Peter Anvin
reviewerQuoted 7 lines · Harvey Harrison
For now, the problem is with arches like X86 that need to test the availability of an instruction. So the arch versions could be unconditionally offered on
X86_64and the lowest-common denominator (no BSWAP) on X86-32. If we still want the optimized (BSWAP) versions on X86-32 the tests will have to use the compiler arch flags as opposed to CONFIG_BSWAP....which is probably not worth the trouble.
Well, that's how the headers were originally written, I believe.
CONFIG_BSWAP can be replaced with the __i486__ macro, and since most distros compile for i586 or i686 these days, it might actually make sense for this one particular case.
-hpa
PATCH 1/1 x86: do not expose CONFIG_BSWAP to userspace
Use ifdef __i486__ to ensure the BSWAP instruction is available on 32-bit x86.
---
HPA,
I'm afraid my knowledge of gcc compiler flags for various models is lacking, I used i486 as suggested, just wanted to make sure I understood you corectly.
arch/x86/include/asm/swab.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/include/asm/swab.h +2 -2
@@ -7,7 +7,7 @@ static inline __attribute_const__ __u32 __arch_swab32(__u32 val) {#ifdef __i386__ -# ifdef CONFIG_X86_BSWAP +# ifdef __i486__ asm("bswap %0" :"=r" (val) :"0" (val));# else asm("xchgb %b0,%h0\n\t" /* swap lower bytes */
@@ -37,7 +37,7 @@ static inline __attribute_const__ __u64 __arch_swab64(__u64 val)__u64 u; } v; v.u = val; -# ifdef CONFIG_X86_BSWAP +# ifdef __i486__ asm("bswapl %0 ; bswapl %1 ; xchgl %0,%1" :"=r" (v.s.a),"=r" (v.s.b) :"0" (v.s.a),"1" (v.s.b));
Signature · 1 line
1.6.1.401.gf39d5
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 11 lines · Harvey Harrison
Use ifdef
__i486__to ensure the BSWAP instruction is available on 32-bit x86.Signed-off-by Harvey Harrison <[email protected]>---
HPA,I'm afraid my knowledge of gcc compiler flags for various models is lacking, I used i486 as suggested, just wanted to make sure I understood you corectly.
I really don't think this works.
Try this:
cpp -dM -m32 -march=pentium t.c | grep i486
and see the marked absense of __i486__ anywhere.
At least CONFIG_X86_BSWAP gives the right behaviour for the kernel.
Linus
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 16 lines · Linus Torvalds
Quoted 11 lines · Harvey Harrison
Use ifdef
__i486__to ensure the BSWAP instruction is available on 32-bit x86.Signed-off-by Harvey Harrison <[email protected]>---
HPA,I'm afraid my knowledge of gcc compiler flags for various models is lacking, I used i486 as suggested, just wanted to make sure I understood you corectly.
I really don't think this works.
Quoted 7 lines
Try this:
cpp -dM -m32 -march=pentium t.c | grep i486and see the marked absense of
__i486__anywhere.At least
CONFIG_X86_BSWAPgives the right behaviour for the kernel.
You're right, it doesn't work. Perhaps just exporting the bswap version all the time and i386-only people can worry about patching it out?
Harvey
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 19 lines · Harvey Harrison
Quoted 18 lines · Linus Torvalds
Quoted 11 lines · Harvey Harrison
Use ifdef
__i486__to ensure the BSWAP instruction is available on 32-bit x86.Signed-off-by Harvey Harrison <[email protected]>---
HPA,I'm afraid my knowledge of gcc compiler flags for various models is lacking, I used i486 as suggested, just wanted to make sure I understood you corectly.
I really don't think this works.
Try this:
Quoted 8 lines
Quoted 5 lines
cpp -dM -m32 -march=pentium t.c | grep i486and see the marked absense of
__i486__anywhere.At least
CONFIG_X86_BSWAPgives the right behaviour for the kernel.You're right, it doesn't work. Perhaps just exporting the bswap version all the time and i386-only people can worry about patching it out?
And break i386 in a nasty way? Please don't.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
H. Peter Anvin
reviewerRe: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
In reply to Harvey HarrisonQuoted 3 lines · Harvey Harrison
I'm afraid my knowledge of gcc compiler flags for various models is lacking, I used i486 as suggested, just wanted to make sure I understood you corectly.
You did, but I misremembered... instead of having the __i386__, __i486__, __i586__, __i686__ being an additive chain as would make sense, gcc just has __i386__ plus whichever corresponds to the -march= option. I keep forgetting this because it's just so incredibly dumb.
Bloody hell. This really f*cks thing up.
What's worse, they seem to simply be adding new options, so at this point you'd actually need something like:
# if defined(__i486__) || defined(__i586__) || defined(__i686__) || \ defined(__core2__) || defined(__k8__) || defined(__amdfam10__)
Worse, there isn't any kind of macro that can be used to compare for a negative (i.e. not i386).
This obviously is screaming to be abstracted away into a header of its own, but it really can't be done cleanly as far as I can tell because of this particular piece of major gcc braindamage.
So, one ends up doing something like:
#ifdef __i486__
# define __CPU_HAVE_BSWAP #endif #ifdef __i586__ # define __CPU_HAVE_BSWAP #endif
... and so on, and have to keep this up to date with the latest inventions of the gcc people. *Sob.*
-hpa
-hpa
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 12 lines · H. Peter Anvin
Quoted 3 lines · Harvey Harrison
I'm afraid my knowledge of gcc compiler flags for various models is lacking, I used i486 as suggested, just wanted to make sure I understood you corectly.
You did, but I misremembered... instead of having the
__i386__,__i486__,__i586__,__i686__being an additive chain as would make sense, gcc just has__i386__plus whichever corresponds to the -march= option. I keep forgetting this because it's just so incredibly dumb.Bloody hell. This really f*cks thing up.
Quoted 8 lines
What's worse, they seem to simply be adding new options, so at this point you'd actually need something like:
# if defined(__i486__) || defined(__i586__) || defined(__i686__) || \ defined(__core2__) || defined(__k8__) || defined(__amdfam10__)Worse, there isn't any kind of macro that can be used to compare for a negative (i.e. not i386).
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Quoted 8 lines
This obviously is screaming to be abstracted away into a header of its own, but it really can't be done cleanly as far as I can tell because of this particular piece of major gcc braindamage.
So, one ends up doing something like:
#ifdef
__i486__
Quoted 8 lines
# define __CPU_HAVE_BSWAP #endif #ifdef __i586__ # define __CPU_HAVE_BSWAP #endif... and so on, and have to keep this up to date with the latest inventions of the gcc people. *Sob.*
Unpleasant indeed. Is there a byteswap builtin in gcc? At least AVR32 seems to use it, but perhaps it's not generally exposed...perhaps we could ask the gcc-folk?
Harvey
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 3 lines · Harvey Harrison
Is there a byteswap builtin in gcc? At least AVR32
seems to use it, but perhaps it's not generally exposed...perhaps we could ask the gcc-folk?
Yes, GCC has byteswap builtins on x86 documented as follows:
-- Built-in Function: int32_t __builtin_bswap32 (int32_t x)
Returns X with the order of the bytes reversed; for example,
`0xaabbccdd' becomes `0xddccbbaa'. Byte here always means exactly
8 bits. -- Built-in Function: int64_t __builtin_bswap64 (int64_t x)
Similar to `__builtin_bswap32', except the argument and return types are 64-bit.
These were only added as of GCC 4.3 though.
Signature · 2 lines
Ben Pfaff http://benpfaff.org
H. Peter Anvin
reviewerRe: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 13 lines · Ben Pfaff
Quoted 3 lines · Harvey Harrison
Is there a byteswap builtin in gcc? At least AVR32
seems to use it, but perhaps it's not generally exposed...perhaps we could ask the gcc-folk?Yes, GCC has byteswap builtins on x86 documented as follows:
-- Built-in Function:
int32_t__builtin_bswap32(int32_tx)Returns X with the order of the bytes reversed; for example, `0xaabbccdd' becomes `0xddccbbaa'. Byte here always means exactly 8 bits.
Quoted 5 lines
-- Built-in Function:
int64_t__builtin_bswap64(int64_tx)
Similar to `__builtin_bswap32', except the argument and return types are 64-bit.These were only added as of GCC 4.3 though.
It would make sense to use them if gcc >= 4.3 (both for userspace and kernel space.) IMO, for older compilers we can just punt on the enhancement for userspace and fall back to the i386 code.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
H. Peter Anvin
reviewerRe: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
In reply to Harvey HarrisonQuoted 4 lines · Harvey Harrison
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Well, we already have MOVBE coming up, too...
-hpa
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 8 lines · H. Peter Anvin
Quoted 4 lines · Harvey Harrison
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Well, we already have MOVBE coming up, too...
Is someone already working on an __arch_swab{16|32|64}p to use them?
Harvey
H. Peter Anvin
reviewerRe: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 10 lines · Harvey Harrison
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · Harvey Harrison
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Well, we already have MOVBE coming up, too...
Is someone already working on an __arch_swab{16|32|64}p to use them?
Not that I know of, but it's trivial enough. They can also be used for all-register swapping, too, with the advantage that you get register decoupling.
-hpa
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Harvey Harrison
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · Harvey Harrison
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Well, we already have MOVBE coming up, too...
Quoted 6 lines
Quoted 2 lines
Is someone already working on an __arch_swab{16|32|64}p to use them?
Not that I know of, but it's trivial enough. They can also be used for all-register swapping, too, with the advantage that you get register decoupling.
I just realized that gcc-4.3 and higher have the __builtin_bswap{16,32,64} functions on x86, which are supposed to do the right thing on any platform. Maybe a patch like the one below can solve this for both the kernel and for other users of the byteorder headers. Unfortunately, I could not find out whether the builtins are available on all other platforms as well.
arch/x86/include/asm/swab.h +13 -2
@@ -4,9 +4,17 @@#include <linux/types.h> #include <linux/compiler.h> +#ifdef __CHECKER__ +extern unsigned long __builtin_bswap_32 (unsigned long x); +extern unsigned long long __builtin_bswap_64 (unsigned long long x); +#endif + static inline __attribute_const__ __u32 __arch_swab32(__u32 val) { -#ifdef __i386__ +#if defined(__GNUC__) && ((__GNUC__ >4 ) || \ + ((__GNUC__ ==4 ) && (__GNUC_MINOR__ >=3 ))) + val =__builtin_bswap32 (val); +#elif defined (__i386__)# ifdef CONFIG_X86_BSWAP asm("bswap %0" :"=r" (val) :"0" (val));# else
@@ -28,7 +36,10 @@ static inline __attribute_const__ __u32 __arch_swab32(__u32 val) static inline __attribute_const__ __u64 __arch_swab64(__u64 val) { -#ifdef __i386__ +#if defined(__GNUC__) && ((__GNUC__ > 4) || \ + ((__GNUC__ == 4) && (__GNUC_MINOR__ >= 3))) + return __builtin_bswap64(val); +#elif defined (__i386__) union { struct { __u32 a;
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
In reply to Harvey HarrisonQuoted 2 lines · Harvey Harrison
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Or just emulate it alongside CMPXCHG and XADD? ;)
Maciej
H. Peter Anvin
reviewerRe: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 6 lines · Maciej W. Rozycki
Quoted 2 lines · Harvey Harrison
Well, that's unfortunate, how about we just export the BSWAP version unconditionally and hope pure i386 just goes away someday?
Or just emulate it alongside CMPXCHG and XADD? ;)
That would work unless someone actually cares about 386 or 486 in the embedded space. I think at least 486EX is still used as an embedded platform.
The kernel can obviously provide this service (although it slows down the #UD handler), but anyone who actually cares about these chips would not really want them.
At some point, we may want to ask ourselves how long keeping 386 support (with it's broken WP handling) alive.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Re: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
In reply to H. Peter AnvinQuoted 11 lines · H. Peter Anvin
So, one ends up doing something like:
#ifdef
__i486__# define __CPU_HAVE_BSWAP #endif #ifdef __i586__ # define __CPU_HAVE_BSWAP #endif... and so on, and have to keep this up to date with the latest inventions of the gcc people. *Sob.*
Well, to put this into perspective: The bswap inline assembly was introduced in Linux-1.3.51 "Greased Weasel", back in 1995 and at no time it was ever visible to user space, unless the code manually included <linux/config.h> (which we broke) or defined CONFIG_M486, CONFIG_X86_BSWAP and/or __KERNEL__, depending on the kernel version.
I take this as a strong indication that user space applications won't generally expect to get the bswap instruction from including the kernel headers. For the longest time, we actually had
/* For avoiding bswap on i386 */
#ifdef __KERNEL__
#include <linux/config.h>
#endif
which I read as explicitly using the portable i386 version for all user space.
Arnd <><
H. Peter Anvin
reviewerRe: [PATCH] x86: do not expose CONFIG_BSWAP to userspace
Quoted 5 lines · Arnd Bergmann
I take this as a strong indication that user space applications won't generally expect to get the bswap instruction from including the kernel headers. For the longest time, we actually had
Quoted 8 lines
/* For avoiding bswap on i386 */
#ifdef__KERNEL__
#include <linux/config.h>
#endifwhich I read as explicitly using the portable i386 version for all user space.
This is true, although it would be nice to be able to generate BSWAP now when most distros compile with -march=i586 or higher. The reason noone cared for the longest time was that noone wanted to compile userspace for anything other than the i386 baseline.
-hpa
Quoted 17 lines · Linus Torvalds
Quoted 7 lines · Harvey Harrison
Quoted 2 lines · H. Peter Anvin
In general, no. The byteswap API is a legacy exception.
But now that
swab.hhas been separated out, we could just stop exporting theasm/swab.hbits while still providing a generic C-based implementation to userspace.Well, the _reason_ the byteswap stuff has been interesting to user space is that the kernel did it better than the alternatives. Rather than having purely "work with big-endian data" (the networking htonl etc functions), the kernel had good and fairly optimized handling of various different forms of byte order handling.
Quoted 3 lines
Which is why people wanted to use it in the first place - and which is why then doing just the generic C-based thing doesn't really fix the issue. Things may compile, but they kind of lost the point.
The right fix then is to provide these optimized versions as part of their libc variant, not to keep exporting our internal versions to userland. Because the internal version may be subject to runtime patching and api changes etc.
Sam
H. Peter Anvin
reviewerQuoted 5 lines · Harvey Harrison
But now that
swab.hhas been separated out, we could just stop exporting theasm/swab.hbits while still providing a generic C-based implementation to userspace.
That makes sense if and only if the generic C-based implementation doesn't suck.
-hpa
Quoted 21 lines · Linus Torvalds
Quoted 3 lines · Ingo Molnar
Should i perhaps not bother with the stuff below? Cannot turn off
CONFIG_HEADERS_CHECKin my builds because it can cause build failures.I really hate the patch. I think it's fundamentally flawed. I hate scripts that test for things that are readable, and encourage people to then write crap instead.
The thing is, the
headers_checkstuff is just wrong if it causes these things, and I'd rather just turn it off.If those
#ifdef CONFIG_XYZthings result in problems, then we should just make the rule be that we turn that kind of string into
Quoted 6 lines
#if 0automatically when exporting the kernel headers. IOW, just about _anything_ that
headers_checkcomplains about automatically is something that should just be _fixed_ automatically at header install time rather than make the code harder to read.
Almost every single patch that makes code harder to read while removing CONFIG_ references from the exported headers just take the wrong approach.
I read later in this thread that Ingo is planning to filter out these patches so we can get the other ones in - good.
What we should try to do is to look at the bigger picture. Sometimes the right answer is that this whole file should not be exported or maybe only half should be exported.
But due to the fact that we have no established way to seperate what is internal and what is exported *on the file level* people keep sprinkling #ifdef __KERNEL__ all over the header files.
I checked include/linux - we have 494 places where we reference __KERNEL__ And that is in 237 files.
We export 354 files from this dir - so we already have > 100 files that is exported to userspace without modifications.
So rather than start sprinkling the exported headers with "if 0" then we should encourage to clean them and when accetable move them to a dedicated abi directory.
This may need some cooperation with external parties such as glibc and other libc variants. And we could improve here.
Sam