More accessible usage of custom flags
PATCH 1/1 More accessible usage of custom flags
I submitted this patch to the zen-sources Gentoo community and got much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around, so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)? One can indeed add one's own flags in the Makefile, but this method is a good deal friendlier. Granted, this could be misused by ricers and idiots, but they'll get themselves into that mess all of their own fault and we'll all go on our merry ways. It just seems that much use could be made out of this, both in terms of (sane) optimizations and easier access to enhanced debugging opportunities.
I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and
~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
I believe one of the very simple things that can be addressed is to make the kernel more "accessible" without harming its integrity or functionality. This involves trying to fill the gap between those two types of people, allowing there to be more understanding, configuration, and (down the line) development opportunities within the kernel to better ease these people into learning enough to begin contributing back. More developers can only be a Good Thing (tm). Though a haughty and abstract opinion/goal/thought of mine, this patch conforms to it and I think that with minimal effort such a vision could be implemented. Whilst all other kernel devs are hacking away at the rough and tumultuous innards of the kernel, a few people of less confidence and experience (people like me) could polish the outside to make it more appealing to more people. So that eventually they too will sink below the surface and join the party.
Anyway, I'm not quite sure that my implementation is as clean as it could be and any feedback is certainly welcome. This is my first time posting to LKML, so I fully expect to be shot down anyway.
~Nicholas (Alex) Marquez
<[email protected]>
______
init/Kconfig 2008-02-16 20:53:48.000000000 -0500 renamed from init/Kconfig 2008-02-16 20:24:22.000000000 -0500 +46 -0
@@ -487,6 +487,52 @@ option. If problems are observed, a gcc upgrade may be needed. If unsure, say N. + +menu "Custom flags" + +config CUSTOM_CFLAGS + string "Custom CFLAGS for kernel" + default "" + help + You can use this to easily set custom gcc CFLAGS to be used for the + entire kernel (including modules). + + ONLY ADD CUSTOM CFLAGS IF YOU KNOW WHAT YOU ARE DOING + + If unsure, leave blank. + +config CUSTOM_LDFLAGS + string "Custom LDFLAGS for kernel" + default "" + help + You can use this to easily set custom gcc LDFLAGS to be used for the + entire kernel (including modules). + + ONLY ADD CUSTOM LDFLAGS IF YOU KNOW WHAT YOU ARE DOING + + If unsure, leave blank. + +config CUSTOM_AFLAGS + string "Custom AFLAGS for kernel" + default "" + help + You can use this to easily set custom gcc AFLAGS to be used for the + entire kernel (including modules). + + ONLY ADD CUSTOM AFLAGS IF YOU KNOW WHAT YOU ARE DOING + + If unsure, leave blank. + +config CUSTOM_MAKEFLAGS + string "Custom MAKEFLAGS for kernel" + default "" + help + You can use this to easily set custom MAKEFLAGS to be used for building + the entire kernel. + + If unsure, leave blank. + +endmenu config SYSCTL bool
Makefile 2008-02-16 20:35:55.000000000 -0500 renamed from Makefile 2008-02-16 20:15:58.000000000 -0500 +6 -7
@@ -14,7 +14,7 @@ # o use make's built-in rules and variables # (this increases performance and avoids hard-to-debug behaviour); # o print "Entering directory ..."; -MAKEFLAGS += -rR --no-print-directory +MAKEFLAGS += -rR --no-print-directory $(CUSTOM_MAKEFLAGS) # We are using a recursive build, so we need to do a little thinking # to get the ordering right.
@@ -315,11 +315,11 @@ CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(CF) MODFLAGS = -DMODULE -CFLAGS_MODULE = $(MODFLAGS) -AFLAGS_MODULE = $(MODFLAGS) -LDFLAGS_MODULE = -CFLAGS_KERNEL = -AFLAGS_KERNEL = +CFLAGS_MODULE = $(MODFLAGS) $(CUSTOM_CFLAGS) +AFLAGS_MODULE = $(MODFLAGS) $(CUSTOM_AFLAGS) +LDFLAGS_MODULE = $(CUSTOM_LDFLAGS) +CFLAGS_KERNEL = $(CUSTOM_CFLAGS) +AFLAGS_KERNEL = $(CUSTOM_AFLAGS)
# Use LINUXINCLUDE when you must reference the include/ directory.
@@ -525,6 +525,10 @@
KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once) endif
+# Apply custom flags
+KBUILD_CFLAGS += $(CUSTOM_CFLAGS)
+KBUILD_AFLAGS += $(CUSTOM_AFLAGS)
+
# Force gcc to behave correct even for buggy distributions
KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector)
@@ -560,7 +564,7 @@
LDFLAGS_BUILD_ID = $(patsubst -Wl$(comma)%,%,\
$(call ld-option, -Wl$(comma)--build-id,)) LDFLAGS_MODULE += $(LDFLAGS_BUILD_ID)
-LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID) +LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID) $(CUSTOM_LDFLAGS)
# Default kernel image to build when no specific target is given. # KBUILD_IMAGE may be overruled on the command line or
Quoted 4 lines · Nicholas Marquez
I submitted this patch to the zen-sources Gentoo community and got much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around,
Probably it wasn't done by other patchsets. ;-)
Quoted 4 lines
so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)?
Examples, please. Which compiler flags do you want to add to your .config? Speaking of -Os, it's CONFIG_CC_OPTIMIZE_FOR_SIZE .
Quoted 4 lines
One can indeed add one's own flags in the
Makefile, but this method is a good deal friendlier. Granted, this could be misused by ricers and idiots, but they'll get themselves into that mess all of their own fault and we'll all go on our merry ways.
No, they will come here and report bugs they created themselves. And there will be a policy: "too long CONFIG_CUSTOM_CFLAGS -- go away" and so on.
Quoted 2 lines
It just seems that much use could be made out of this, both in terms of (sane) optimizations
Sane optimizations should be added to main Makefile.
Quoted 2 lines
and easier access to enhanced debugging
opportunities.
Which ones exactly?
Quoted 14 lines
I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and
~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
I believe one of the very simple things that can be addressed is to make the kernel more "accessible" without harming its integrity or functionality. This involves trying to fill the gap between those two types of people, allowing there to be more understanding, configuration, and (down the line) development opportunities within the kernel to better ease these people into learning enough to begin contributing back.
Quoted 1 line
More developers can only be a Good Thing (tm).
In general, wrong.
Quoted 8 lines · Alexey Dobriyan
Quoted 4 lines · Nicholas Marquez
I submitted this patch to the zen-sources Gentoo community and got much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around,
Probably it wasn't done by other patchsets. ;-)
Quoted 7 lines
Quoted 4 lines
so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)?
Examples, please. Which compiler flags do you want to add to your .config? Speaking of -Os, it's
CONFIG_CC_OPTIMIZE_FOR_SIZE.
For example: DFORTIFY_SOURCE=2 or ftree-ch, because it's not enabled by Os, but does indeed help often, especially in the case of various kernel codepaths.
As to CONFIG_CC_OPTIMIZE_FOR_SIZE, I realize. "-Os" is just easier to type and I assume that a kernel dev list would map the correct config option. ;)
Quoted 9 lines
Quoted 5 lines
One can indeed add one's own flags in the
Makefile, but this method is a good deal friendlier. Granted, this could be misused by ricers and idiots, but they'll get themselves into that mess all of their own fault and we'll all go on our merry ways.No, they will come here and report bugs they created themselves. And there will be a policy: "too long
CONFIG_CUSTOM_CFLAGS-- go away" and so on.
My point exactly. Enabling experimental or broken options is already ignored pretty well, so how hard is it to just create such a policy and go about shooting down people who've messed up? In all fairness, most people on the mailing list are terse and, shall we say, aggressive. I don't really see there being much of a problem with a few extra faulty bug reports. I feel that the good done would outweigh the slight overhead of a few idiots.
Quoted 25 lines
Quoted 3 lines
It just seems that much use could be made out of this, both in terms of (sane) optimizations
Sane optimizations should be added to main
Makefile.Quoted 2 lines
and easier access to enhanced debugging
opportunities.Which ones exactly?
Quoted 13 lines
I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and
~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
I believe one of the very simple things that can be addressed is to make the kernel more "accessible" without harming its integrity or functionality. This involves trying to fill the gap between those two types of people, allowing there to be more understanding, configuration, and (down the line) development opportunities within the kernel to better ease these people into learning enough to begin contributing back.
Quoted 4 lines
Quoted 1 line
More developers can only be a Good Thing (tm).
In general, wrong.
Erm... what are you responding to? I don't know what that "wrong" refers to. Especially considering that the paragraph you're referencing is an opinion and personal thought.
Does anyone else have any input on this? Tips, suggestions, ideas, comments, constructive criticism, anything at all. I am, however, trying to avoid a flame war.
~Nicholas (Alex) Marquez
<[email protected]>
Quoted 151 lines · Nicholas Marquez
I submitted this patch to the zen-sources Gentoo community and got
much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around, so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)? One can indeed add one's own flags in theMakefile, but this method is a good deal friendlier. Granted, this could be misused by ricers and idiots, but they'll get themselves into that mess all of their own fault and we'll all go on our merry ways. It just seems that much use could be made out of this, both in terms of (sane) optimizations and easier access to enhanced debugging opportunities.I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and
~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
I believe one of the very simple things that can be addressed is to make the kernel more "accessible" without harming its integrity or functionality. This involves trying to fill the gap between those two types of people, allowing there to be more understanding, configuration, and (down the line) development opportunities within the kernel to better ease these people into learning enough to begin contributing back. More developers can only be a Good Thing (tm). Though a haughty and abstract opinion/goal/thought of mine, this patch conforms to it and I think that with minimal effort such a vision could be implemented. Whilst all other kernel devs are hacking away at the rough and tumultuous innards of the kernel, a few people of less confidence and experience (people like me) could polish the outside to make it more appealing to more people. So that eventually they too will sink below the surface and join the party.Anyway, I'm not quite sure that my implementation is as clean as it could be and any feedback is certainly welcome. This is my first time posting to LKML, so I fully expect to be shot down anyway.
~Nicholas (Alex) Marquez
<[email protected]>______
diff -dur original/init/Kconfig modified/init/Kconfig
--- original/init/Kconfig 2008-02-16 20:24:22.000000000 -0500
+++ modified/init/Kconfig 2008-02-16 20:53:48.000000000 -0500
@@ -487,6 +487,52 @@
option. If problems are observed, a gcc upgrade may be needed.If unsure, say N.
+
+menu "Custom flags"
+
+configCUSTOM_CFLAGS+ string "Custom CFLAGS for kernel" + default "" + help + You can use this to easily set custom gcc CFLAGS to be used for the + entire kernel (including modules). + + ONLY ADD CUSTOM CFLAGS IF YOU KNOW WHAT YOU ARE DOING + + If unsure, leave blank. + +config CUSTOM_LDFLAGS + string "Custom LDFLAGS for kernel" + default "" + help + You can use this to easily set custom gcc LDFLAGS to be used for the + entire kernel (including modules). + + ONLY ADD CUSTOM LDFLAGS IF YOU KNOW WHAT YOU ARE DOING + + If unsure, leave blank. + +config CUSTOM_AFLAGS + string "Custom AFLAGS for kernel" + default "" + help + You can use this to easily set custom gcc AFLAGS to be used for the + entire kernel (including modules). + + ONLY ADD CUSTOM AFLAGS IF YOU KNOW WHAT YOU ARE DOING + + If unsure, leave blank. + +config CUSTOM_MAKEFLAGS + string "Custom MAKEFLAGS for kernel" + default "" + help+ You can use this to easily set custom MAKEFLAGS to be used for building
+ the entire kernel.
+
+ If unsure, leave blank.
+
+endmenuconfig SYSCTL
bool
diff -dur original/Makefile modified/Makefile
--- original/Makefile 2008-02-16 20:15:58.000000000 -0500
+++ modified/Makefile 2008-02-16 20:35:55.000000000 -0500
@@ -14,7 +14,7 @@# o use make's built-in rules and variables # (this increases performance and avoids hard-to-debug behaviour); # o print "Entering directory ..."; -MAKEFLAGS += -rR --no-print-directory +MAKEFLAGS += -rR --no-print-directory $(CUSTOM_MAKEFLAGS)# We are using a recursive build, so we need to do a little thinking
# to get the ordering right. @@ -315,11 +315,11 @@CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__
-Wbitwise $(CF)
MODFLAGS = -DMODULE-CFLAGS_MODULE = $(MODFLAGS) -AFLAGS_MODULE = $(MODFLAGS) -LDFLAGS_MODULE = -CFLAGS_KERNEL = -AFLAGS_KERNEL = +CFLAGS_MODULE = $(MODFLAGS) $(CUSTOM_CFLAGS) +AFLAGS_MODULE = $(MODFLAGS) $(CUSTOM_AFLAGS) +LDFLAGS_MODULE = $(CUSTOM_LDFLAGS)+CFLAGS_KERNEL = $(CUSTOM_CFLAGS)
+AFLAGS_KERNEL = $(CUSTOM_AFLAGS)# Use LINUXINCLUDE when you must reference the
include/directory.
@@ -525,6 +525,10 @@
KBUILD_CFLAGS+= $(call cc-option, -fno-inline-functions-called-once) endif+# Apply custom flags
+KBUILD_CFLAGS += $(CUSTOM_CFLAGS)
+KBUILD_AFLAGS += $(CUSTOM_AFLAGS)
+
# Force gcc to behave correct even for buggy distributions
KBUILD_CFLAGS+= $(call cc-option, -fno-stack-protector)@@ -560,7 +564,7 @@
LDFLAGS_BUILD_ID = $(patsubst -Wl$(comma)%,%,\ $(call ld-option, -Wl$(comma)--build-id,))
LDFLAGS_MODULE+= $(LDFLAGS_BUILD_ID)
-LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID) +LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID) $(CUSTOM_LDFLAGS)# Default kernel image to build when no specific target is given. #
KBUILD_IMAGEmay be overruled on the command line or
Quoted 3 lines · Nicholas Marquez
Does anyone else have any input on this? Tips, suggestions, ideas, comments, constructive criticism, anything at all. I am, however, trying to avoid a flame war.
I missed it the first time you posted it, so thanks for resending.
Quoted 10 lines
Quoted 8 lines · Nicholas Marquez
I submitted this patch to the zen-sources Gentoo community and got
much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around, so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)? One can indeed add one's own flags in theMakefile, but this method is a good deal friendlier.
Does using this patch reduce the safety of the kernel build at all? i.e. is it more likely to build different parts of the kernel with incompatible options with this patch, compared to if the options were specified in the Makefile?
Quoted 6 lines
Quoted 6 lines
I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and ~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
I'm currently cleaning up the Kconfig help descriptions to try to help the second group of people. Also I think there's a spectrum of capabilities involved. I'd like kernel building to be a fun and useful activity for a wider group of people :-)
Quoted 4 lines
Quoted 4 lines
Whilst all other kernel devs are hacking away
at the rough and tumultuous innards of the kernel, a few people of less confidence and experience (people like me) could polish the outside to make it more appealing to more people.
Heh, that includes me too.
Quoted 4 lines
Quoted 4 lines
______
diff -dur original/init/Kconfig modified/init/Kconfig
--- original/init/Kconfig 2008-02-16 20:24:22.000000000 -0500
+++ modified/init/Kconfig 2008-02-16 20:53:48.000000000 -0500
I've just started using StGit and it makes it a lot easier to make, send and manage patches. You should take a look at it.
Quoted 17 lines
Quoted 11 lines
@@ -487,6 +487,52 @@
option. If problems are observed, a gcc upgrade may be needed.If unsure, say N.
+
+menu "Custom flags"
+
+configCUSTOM_CFLAGS+ string "Custom CFLAGS for kernel" + default "" + helpQuoted 6 lines
+ You can use this to easily set custom gcc CFLAGS to be used for the
+ entire kernel (including modules).
+
+ ONLY ADD CUSTOM CFLAGS IF YOU KNOW WHAT YOU ARE DOING
+
+ If unsure, leave blank.
Sorry, but I don't think we need shouting. There are a few options already which have bad results if you unknowingly choose the wrong option, like enabling EMBEDDED and turning off the BLOCK layer, yet the help description doesn't shout.
Can I suggest:
help
You can use this to easily set custom gcc CFLAGS to be used for the entire kernel build process (including modules).
Compiler flags can be used for increasing or reducing optimisation, enabling debugging, cross compilation and so on.
If unsure, leave blank.
The boilerplate "If unsure, leave blank" covers (or is meant to cover) the situation where the user does not know what they are doing. It says "If you don't know what you are doing, just leave this field blank, and everything will be OK" ... except it is put in a non-threatening, non-demeaning way.
The paragraph in the middle enhances the "help" component of the help description, by saying why you might want to use that option.
Quoted 6 lines
Quoted 6 lines
+config
CUSTOM_LDFLAGS
[...]
+configCUSTOM_AFLAGS
[...]
+configCUSTOM_MAKEFLAGS
[...]
Similar comments apply.
Nick.
Signature · 2 lines
PGP Key ID = 0x418487E7 http://www.nick-andrew.net/ PGP Key fingerprint = B3ED 6894 8E49 1770 C24A 67E3 6266 6EB9 4184 87E7
First of all, thanks for the input!
Quoted 19 lines · Nick Andrew
Quoted 3 lines · Nicholas Marquez
Does anyone else have any input on this? Tips, suggestions, ideas, comments, constructive criticism, anything at all. I am, however, trying to avoid a flame war.
I missed it the first time you posted it, so thanks for resending.
Quoted 10 lines
Quoted 8 lines · Nicholas Marquez
I submitted this patch to the zen-sources Gentoo community and got
much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around, so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)? One can indeed add one's own flags in theMakefile, but this method is a good deal friendlier.
Quoted 4 lines
Does using this patch reduce the safety of the kernel build at all? i.e. is it more likely to build different parts of the kernel with incompatible options with this patch, compared to if the options were specified in the
Makefile?
There are a few places in the Makefile where options would need to be entered. This just brings those places together in a clean manner and allows for one to set the flags via the configuration interface. Though it would be excellent if the kernel had more gcc micro-optimization (facilitated by such features as the upcoming gcc 4.3's -finstrument-functions-exclude-function-list option), general CFLAGS used for the entire kernel do not impact the safety of the kernel build, assuming that the options are sane. It would be interesting, however, to see an initiative to tune gcc flags for important functions and sections within the kernel for optimal performance given some sort of envisioned usage. (*hint, kernel devs, hint*)
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and ~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
Quoted 4 lines
I'm currently cleaning up the
Kconfighelp descriptions to try to help the second group of people. Also I think there's a spectrum of capabilities involved. I'd like kernel building to be a fun and useful activity for a wider group of people :-)
I praise and applaud your efforts.
Quoted 37 lines
Quoted 6 lines
Quoted 6 lines
Whilst all other kernel devs are hacking away
at the rough and tumultuous innards of the kernel, a few people of less confidence and experience (people like me) could polish the outside to make it more appealing to more people.Heh, that includes me too.
Quoted 4 lines
Quoted 4 lines
______
diff -dur original/init/Kconfig modified/init/Kconfig
--- original/init/Kconfig 2008-02-16 20:24:22.000000000 -0500
+++ modified/init/Kconfig 2008-02-16 20:53:48.000000000 -0500I've just started using StGit and it makes it a lot easier to make, send and manage patches. You should take a look at it.
Quoted 17 lines
Quoted 11 lines
@@ -487,6 +487,52 @@
option. If problems are observed, a gcc upgrade may be needed.If unsure, say N.
+
+menu "Custom flags"
+
+configCUSTOM_CFLAGS+ string "Custom CFLAGS for kernel" + default "" + helpQuoted 6 lines
+ You can use this to easily set custom gcc CFLAGS to be used for the
+ entire kernel (including modules).
+
+ ONLY ADD CUSTOM CFLAGS IF YOU KNOW WHAT YOU ARE DOING
+
+ If unsure, leave blank.
Quoted 4 lines
Sorry, but I don't think we need shouting. There are a few options already which have bad results if you unknowingly choose the wrong option, like enabling EMBEDDED and turning off the BLOCK layer, yet the help description doesn't shout.
Sorry about that. I didn't mean to portray it as "shouting" so much as a very visible and stern warning.
Quoted 8 lines
Can I suggest:
help
You can use this to easily set custom gcc CFLAGS to be used for the entire kernel build process (including modules).
Quoted 5 lines
Compiler flags can be used for increasing or reducing optimisation, enabling debugging, cross compilation and so on.
If unsure, leave blank.
Something else that could be included is a link to the manual for the (currently used) gcc's command flags
(http://gcc.gnu.org/onlinedocs/gcc-4.2.3/gcc/index.html#toc_Invoking-GCC)
or at least to the online docs, considering that specifying one specific gcc version's manual would yield various issues.
Quoted 9 lines
The boilerplate "If unsure, leave blank" covers (or is meant to cover) the situation where the user does not know what they are doing. It says "If you don't know what you are doing, just leave this field blank, and everything will be OK" ... except it is put in a non-threatening, non-demeaning way.
The paragraph in the middle enhances the "help" component of the help description, by saying why you might want to use that option.
I feel that perhaps it is a bit silly to put in such description, as one using the flags should only do so in knowing specifically which ones they wish to use and why ahead of time. However, it cannot hurt to put in more help documentation, so it's a fine suggestion.
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
+config
CUSTOM_LDFLAGS
[...]
+configCUSTOM_AFLAGS
[...]
+configCUSTOM_MAKEFLAGS
[...]
Quoted 7 lines
Similar comments apply.
Nick.
--
PGP Key ID =0x418487E7http://www.nick-andrew.net/
PGP Key fingerprint = B3ED 6894 8E49 1770 C24A 67E3 6266 6EB9 4184 87E7
Can anyone tell me if the implementation I chose looks alright? The kernel has an, understandably, large, complex, and tad messy buildsystem, so it is troublesome to discover whether all the appropriate codepaths are being hit that involve using these flags.
~Nicholas (Alex) Marquez
<[email protected]>
Quoted 5 lines · Nicholas Marquez
First of all, thanks for the input!
Quoted 1 line · Nick Andrew
On Sat, Feb 23, 2008 at 04:28:50PM -0500, Nicholas Marquez wrote:
Quoted 12 lines
There are a few places in the
Makefilewhere options would need to be entered. This just brings those places together in a clean manner and allows for one to set the flags via the configuration interface. Though it would be excellent if the kernel had more gcc micro-optimization (facilitated by such features as the upcoming gcc 4.3's -finstrument-functions-exclude-function-list option), general CFLAGS used for the entire kernel do not impact the safety of the kernel build, assuming that the options are sane. It would be interesting, however, to see an initiative to tune gcc flags for important functions and sections within the kernel for optimal performance given some sort of envisioned usage. (*hint, kernel devs, hint*)
If there are some "common" optimisation flags or sets of flags then maybe they could be turned on/off by invididual config options? There's already CC_OPTIMIZE_FOR_SIZE sorta tagged as experimental. But then, enumerating the allowable flags may limit the user's choices whereas I take it you'd like to expand them.
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 17 lines
@@ -487,6 +487,52 @@
option. If problems are observed, a gcc upgrade may be needed.If unsure, say N.
+
+menu "Custom flags"
+
+configCUSTOM_CFLAGS+ string "Custom CFLAGS for kernel" + default "" + help+ You can use this to easily set custom gcc CFLAGS to be used for the
+ entire kernel (including modules).
+
+ ONLY ADD CUSTOM CFLAGS IF YOU KNOW WHAT YOU ARE DOING
+
+ If unsure, leave blank.
Quoted 7 lines
Quoted 4 lines
Sorry, but I don't think we need shouting. There are a few options already which have bad results if you unknowingly choose the wrong option, like enabling EMBEDDED and turning off the BLOCK layer, yet the help description doesn't shout.
Sorry about that. I didn't mean to portray it as "shouting" so much as a very visible and stern warning.
I'm hoping we don't need any stern warnings in the kernel configuration. Although I expect configuring bogus gcc flags is going to break the build far worse than selecting bad options otherwise. If the user makes a mistake in CUSTOM_CFLAGS is the build going to fail with a message "error in CUSTOM_CFLAGS" or is it going to spew out pages of gcc error messages? It is nice if error messages can point back to the source of the error, so the user doesn't have to look through the entire kernel configuration for what they did wrong.
Quoted 5 lines
Something else that could be included is a link to the manual for the (currently used) gcc's command flags
(http://gcc.gnu.org/onlinedocs/gcc-4.2.3/gcc/index.html#toc_Invoking-GCC)
or at least to the online docs, considering that specifying one specific gcc version's manual would yield various issues.
"info gcc" will give all that information in far more detail than I have time to read.
Quoted 6 lines
Quoted 6 lines
The boilerplate "If unsure, leave blank" covers (or is meant to cover) the situation where the user does not know what they are doing. It says "If you don't know what you are doing, just leave this field blank, and everything will be OK" ... except it is put in a non-threatening, non-demeaning way.
Quoted 7 lines
Quoted 2 lines
The paragraph in the middle enhances the "help" component of the help description, by saying why you might want to use that option.
I feel that perhaps it is a bit silly to put in such description, as one using the flags should only do so in knowing specifically which ones they wish to use and why ahead of time. However, it cannot hurt to put in more help documentation, so it's a fine suggestion.
Well, it can't be both silly and a fine suggestion. Does it dumb down the kernel configuration process too much?
I don't want a single person to say "I can't configure this kernel because I don't know what to answer for the CUSTOM_CFLAGS question" and I think the "If unsure, leave blank" at the bottom covers that problem. But I assume you want to encourage experimentation; there will be a person with a bit higher level of skill who might want to start tweaking. Here's something they can start tweaking.
Nick.
Signature · 2 lines
PGP Key ID = 0x418487E7 http://www.nick-andrew.net/ PGP Key fingerprint = B3ED 6894 8E49 1770 C24A 67E3 6266 6EB9 4184 87E7
Quoted 20 lines · Nick Andrew
Quoted 6 lines · Nicholas Marquez
First of all, thanks for the input!
Quoted 1 line · Nick Andrew
On Sat, Feb 23, 2008 at 04:28:50PM -0500, Nicholas Marquez wrote:
Quoted 12 lines
There are a few places in the
Makefilewhere options would need to be entered. This just brings those places together in a clean manner and allows for one to set the flags via the configuration interface. Though it would be excellent if the kernel had more gcc micro-optimization (facilitated by such features as the upcoming gcc 4.3's -finstrument-functions-exclude-function-list option), general CFLAGS used for the entire kernel do not impact the safety of the kernel build, assuming that the options are sane. It would be interesting, however, to see an initiative to tune gcc flags for important functions and sections within the kernel for optimal performance given some sort of envisioned usage. (*hint, kernel devs, hint*)
Quoted 5 lines
If there are some "common" optimisation flags or sets of flags then maybe they could be turned on/off by invididual config options? There's already
CC_OPTIMIZE_FOR_SIZEsorta tagged as experimental. But then, enumerating the allowable flags may limit the user's choices whereas I take it you'd like to expand them.
Yes, I would like to expand them. If possibly, I'd like to take out the individual config options for gcc flags and integrate them into the help description. I feel that the redundancy would not only be ugly, but possibly confusing.
- Show quoted text -
Quoted 28 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 19 lines
@@ -487,6 +487,52 @@
option. If problems are observed, a gcc upgrade may be needed.If unsure, say N.
+
+menu "Custom flags"
+
+configCUSTOM_CFLAGS+ string "Custom CFLAGS for kernel" + default "" + help+ You can use this to easily set custom gcc CFLAGS to be used for the
+ entire kernel (including modules).
+
+ ONLY ADD CUSTOM CFLAGS IF YOU KNOW WHAT YOU ARE DOING
+
+ If unsure, leave blank.Quoted 7 lines
Quoted 4 lines
Sorry, but I don't think we need shouting. There are a few options already which have bad results if you unknowingly choose the wrong option, like enabling EMBEDDED and turning off the BLOCK layer, yet the help description doesn't shout.
Sorry about that. I didn't mean to portray it as "shouting" so much as a very visible and stern warning.
Quoted 8 lines
I'm hoping we don't need any stern warnings in the kernel configuration. Although I expect configuring bogus gcc flags is going to break the build far worse than selecting bad options otherwise. If the user makes a mistake in
CUSTOM_CFLAGSis the build going to fail with a message "error inCUSTOM_CFLAGS" or is it going to spew out pages of gcc error messages? It is nice if error messages can point back to the source of the error, so the user doesn't have to look through the entire kernel configuration for what they did wrong.
I hadn't thought of that. It would certainly be more useful than a stern warning. ^^; Have you any ideas on how to implement such a thing? I would imagine a configure-script-like test on the flags to make sure that gcc could take them correctly would be in order. Then one could also capture any warnings or errors that gcc outputs (such as redundant flags, improper invocation, etc.), fail the build, and output the messages. As far as capturing dangerous flags for the kernel (such as -ffast-math or -fmerge-all-constants), there could be a blacklist... but that would depend on the gcc version used... :/ Perhaps the automated catching of errors and warnings would be best alone.
Quoted 10 lines
Quoted 7 lines
Something else that could be included is a link to the manual for the (currently used) gcc's command flags
(http://gcc.gnu.org/onlinedocs/gcc-4.2.3/gcc/index.html#toc_Invoking-GCC)
or at least to the online docs, considering that specifying one specific gcc version's manual would yield various issues."info gcc" will give all that information in far more detail than I have time to read.
I believe you misunderstand. I mean to include the link in the help message. Like how the links to the appropriate sites for device drivers are included in their help messages.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
The boilerplate "If unsure, leave blank" covers (or is meant to cover) the situation where the user does not know what they are doing. It says "If you don't know what you are doing, just leave this field blank, and everything will be OK" ... except it is put in a non-threatening, non-demeaning way.
The paragraph in the middle enhances the "help" component of the help description, by saying why you might want to use that option.
Quoted 7 lines
Quoted 4 lines
I feel that perhaps it is a bit silly to put in such description, as one using the flags should only do so in knowing specifically which ones they wish to use and why ahead of time. However, it cannot hurt to put in more help documentation, so it's a fine suggestion.
Well, it can't be both silly and a fine suggestion. Does it dumb down the kernel configuration process too much?
No, I feel that both adjectives are perfectly reconcilable, at least in this case. Again, more documentation couldn't hurt. "Dumbing down" kernel configuration may be less aesthetically pleasing to purists, but that is a tiny price to pay for usability and a widening of kernel builders.
Quoted 9 lines
I don't want a single person to say "I can't configure this kernel because I don't know what to answer for the
CUSTOM_CFLAGSquestion" and I think the "If unsure, leave blank" at the bottom covers that problem. But I assume you want to encourage experimentation; there will be a person with a bit higher level of skill who might want to start tweaking. Here's something they can start tweaking.
Safe experimentation, yes. So I agree with you that adding various statements in the help message can only do good. Sorry for the confusion.
- Hide quoted text -
Quoted 6 lines
Nick.
--
PGP Key ID =0x418487E7http://www.nick-andrew.net/
PGP Key fingerprint = B3ED 6894 8E49 1770 C24A 67E3 6266 6EB9 4184 87E7
PS to Nick: Sorry for the double-send. Didn't CC lkml; a forward would be ugly.
Quoted 4 lines · Nicholas Marquez
Yes, I would like to expand them. If possibly, I'd like to take out the individual config options for gcc flags and integrate them into the help description. I feel that the redundancy would not only be ugly, but possibly confusing.
If you allow CUSTOM_CFLAGS and friends (as an open-ended set of flags for the compiler) then yes, having config options like CC_OPTIMIZE_FOR_SIZE is redundant and should be removed.
Alternately, don't have a CUSTOM_CFLAGS and expand on the set of config options which set individual compiler flags. They could be in their own menu, a bunch of checkboxes to tweak GCC's various optimisation algorithms and/or warnings. That's certainly going to be friendlier for the not-very-experienced kernel builder, if less flexible than CUSTOM_CFLAGS.
Quoted 9 lines
Quoted 8 lines
I'm hoping we don't need any stern warnings in the kernel configuration. Although I expect configuring bogus gcc flags is going to break the build far worse than selecting bad options otherwise. If the user makes a mistake in
CUSTOM_CFLAGSis the build going to fail with a message "error inCUSTOM_CFLAGS" or is it going to spew out pages of gcc error messages? It is nice if error messages can point back to the source of the error, so the user doesn't have to look through the entire kernel configuration for what they did wrong.
Quoted 9 lines
I hadn't thought of that. It would certainly be more useful than a stern warning. ^^; Have you any ideas on how to implement such a thing? I would imagine a configure-script-like test on the flags to make sure that gcc could take them correctly would be in order. Then one could also capture any warnings or errors that gcc outputs (such as redundant flags, improper invocation, etc.), fail the build, and output the messages. As far as capturing dangerous flags for the kernel (such as -ffast-math or -fmerge-all-constants), there could be a blacklist... but that would depend on the gcc version used... :/
I think that's a very interesting idea. Yes, you can test the flags by compiling and linking a simple program. If it fails, your flags are bad. You can stop the build right there and say "fix your flags".
The build could also capture the GCC warning/error output and post-process it. Certain types of warnings might be suppressed, or cause the build to fail (based on selected config options). My kernel builds always show a handful of "may be used uninitialised" or similar ... I just assume that those warnings are harmless. It is best of course that these warnings don't occur at all, and I expect kernel developers to fix them when they get a chance. The idea in this paragraph would be desirable to me, independent of how the CFLAGS are specified.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Something else that could be included is a link to the manual for the (currently used) gcc's command flags
(http://gcc.gnu.org/onlinedocs/gcc-4.2.3/gcc/index.html#toc_Invoking-GCC)
or at least to the online docs, considering that specifying one specific gcc version's manual would yield various issues.
Quoted 6 lines
Quoted 2 lines
"info gcc" will give all that information in far more detail than I have time to read.
I believe you misunderstand. I mean to include the link in the help message. Like how the links to the appropriate sites for device drivers are included in their help messages.
No, I think I understood correctly. If you tell the user to go to that long URL then you run the risk of pointing to the documentation for a different version of GCC. On the other hand, if you tell them to type "info gcc" then they will see the documentation for the version of GCC which they have installed. And it's also a lot friendlier to write:
"Type 'info gcc' to see a full list of the compiler options".
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
I feel that perhaps it is a bit silly to put in such description, as one using the flags should only do so in knowing specifically which ones they wish to use and why ahead of time. However, it cannot hurt to put in more help documentation, so it's a fine suggestion.
Quoted 8 lines
Quoted 2 lines
Well, it can't be both silly and a fine suggestion. Does it dumb down the kernel configuration process too much?
No, I feel that both adjectives are perfectly reconcilable, at least in this case. Again, more documentation couldn't hurt. "Dumbing down" kernel configuration may be less aesthetically pleasing to purists, but that is a tiny price to pay for usability and a widening of kernel builders.
I'm looking to find a balance among the competing objectives. Time will tell with any improvements. I'm also hoping for an aesthetically pleasing config menu -- perhaps someone else can take on the challenge of restructuring it to improve the ordering, structure and names of options.
Quoted 2 lines
PS to Nick: Sorry for the double-send. Didn't CC lkml; a forward would be ugly.
No prob.
Nick.
Signature · 2 lines
PGP Key ID = 0x418487E7 http://www.nick-andrew.net/ PGP Key fingerprint = B3ED 6894 8E49 1770 C24A 67E3 6266 6EB9 4184 87E7
Quoted 10 lines · Nick Andrew
Quoted 4 lines · Nicholas Marquez
Yes, I would like to expand them. If possibly, I'd like to take out the individual config options for gcc flags and integrate them into the help description. I feel that the redundancy would not only be ugly, but possibly confusing.
If you allow
CUSTOM_CFLAGSand friends (as an open-ended set of flags for the compiler) then yes, having config options likeCC_OPTIMIZE_FOR_SIZEis redundant and should be removed.
Quoted 6 lines
Alternately, don't have a
CUSTOM_CFLAGSand expand on the set of config options which set individual compiler flags. They could be in their own menu, a bunch of checkboxes to tweak GCC's various optimisation algorithms and/or warnings. That's certainly going to be friendlier for the not-very-experienced kernel builder, if less flexible thanCUSTOM_CFLAGS.
The problem with this is that it is very static. As you are well aware, options differ between gcc versions. Some options also differ in result depending on the gcc version; for example, -ftree-vectorize in gcc 3.x versus gcc 4.x. I believe that it would be cleaner to have the custom flag menu and suggest some flags for specific uses, such as -Os, in the help description. It might look something like:
help
You can use this to easily set custom gcc CFLAGS to be used for the entire kernel build process (including modules).
Examples include:
~"-Os" to optimize for size, i.e. for an embedded system or for more efficient cache usage
~"-s" to strip all symbol table and relocation information
Other compiler flags can be used for increasing or reducing optimisation, enabling debugging, cross compilation and so on. Please use "info gcc" or "man gcc" for more reference.
If unsure, leave blank.
And for LDFLAGS, examples include:
~"-S" to strip all debugging symbols, which are usually unnecessary unless you intend to debug the kernel
~"-s" to strip all symbol information from the kernel
And for MAKEFLAGS, examples could include:
~"-j{n}" where n is the number of jobs you'd like to run for make. This will make the kernel building process be greatly faster. A good rule of thumb is for n to be 1 + the number of processor cores that are on the compiling machine. ~"-K" to attempt to continue compilation, even if errors are encountered, running until all other compilation paths (that do not depend on any that have errored) complete.
These are of course, just a few examples, but it serves to show the idea.
Quoted 25 lines
Quoted 11 lines
Quoted 10 lines
I'm hoping we don't need any stern warnings in the kernel configuration. Although I expect configuring bogus gcc flags is going to break the build far worse than selecting bad options otherwise. If the user makes a mistake in
CUSTOM_CFLAGSis the build going to fail with a message "error inCUSTOM_CFLAGS" or is it going to spew out pages of gcc error messages? It is nice if error messages can point back to the source of the error, so the user doesn't have to look through the entire kernel configuration for what they did wrong.Quoted 9 lines
I hadn't thought of that. It would certainly be more useful than a stern warning. ^^; Have you any ideas on how to implement such a thing? I would imagine a configure-script-like test on the flags to make sure that gcc could take them correctly would be in order. Then one could also capture any warnings or errors that gcc outputs (such as redundant flags, improper invocation, etc.), fail the build, and output the messages. As far as capturing dangerous flags for the kernel (such as -ffast-math or -fmerge-all-constants), there could be a blacklist... but that would depend on the gcc version used... :/
I think that's a very interesting idea. Yes, you can test the flags by compiling and linking a simple program. If it fails, your flags are bad. You can stop the build right there and say "fix your flags".
Quoted 9 lines
The build could also capture the GCC warning/error output and post-process it. Certain types of warnings might be suppressed, or cause the build to fail (based on selected config options). My kernel builds always show a handful of "may be used uninitialised" or similar ... I just assume that those warnings are harmless. It is best of course that these warnings don't occur at all, and I expect kernel developers to fix them when they get a chance. The idea in this paragraph would be desirable to me, independent of how the CFLAGS are specified.
Glad you agree. ;)
Quoted 38 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Something else that could be included is a link to the manual for the (currently used) gcc's command flags
(http://gcc.gnu.org/onlinedocs/gcc-4.2.3/gcc/index.html#toc_Invoking-GCC)
or at least to the online docs, considering that specifying one specific gcc version's manual would yield various issues.Quoted 6 lines
Quoted 2 lines
"info gcc" will give all that information in far more detail than I have time to read.
I believe you misunderstand. I mean to include the link in the help message. Like how the links to the appropriate sites for device drivers are included in their help messages.
No, I think I understood correctly. If you tell the user to go to that long URL then you run the risk of pointing to the documentation for a different version of GCC. On the other hand, if you tell them to type "info gcc" then they will see the documentation for the version of GCC which they have installed. And it's also a lot friendlier to write:
"Type 'info gcc' to see a full list of the compiler options".Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
I feel that perhaps it is a bit silly to put in such description, as one using the flags should only do so in knowing specifically which ones they wish to use and why ahead of time. However, it cannot hurt to put in more help documentation, so it's a fine suggestion.
Quoted 8 lines
Quoted 2 lines
Well, it can't be both silly and a fine suggestion. Does it dumb down the kernel configuration process too much?
No, I feel that both adjectives are perfectly reconcilable, at least in this case. Again, more documentation couldn't hurt. "Dumbing down" kernel configuration may be less aesthetically pleasing to purists, but that is a tiny price to pay for usability and a widening of kernel builders.
Quoted 4 lines
I'm looking to find a balance among the competing objectives. Time will tell with any improvements. I'm also hoping for an aesthetically pleasing config menu -- perhaps someone else can take on the challenge of restructuring it to improve the ordering, structure and names of options.
Is anyone else out there reading these messages? Cleaning and "upgrading" the kernel's configuration and build systems is something that could both use a lot of input and a lot of helping hands. ;) If you've any patches for usability, aesthetics, optimization, configuration or build robustness, etc., now would be the time to share.
Quoted 14 lines
Quoted 4 lines
PS to Nick: Sorry for the double-send. Didn't CC lkml; a forward would be ugly.
No prob.
Nick.
--
PGP Key ID =0x418487E7http://www.nick-andrew.net/
PGP Key fingerprint = B3ED 6894 8E49 1770 C24A 67E3 6266 6EB9 4184 87E7
Quoted 35 lines · Nicholas Marquez
I submitted this patch to the zen-sources Gentoo community and got much praise and has promptly been included. This kind of thing have very likely already been done in other patchsets, but I haven't seen it around, so I've gone ahead and made one. The idea is that one can enable -Os and various other options transparently through standard kernel configuration, so why bar the builder from any other options to pass on to gcc (et al)? One can indeed add one's own flags in the
Makefile, but this method is a good deal friendlier. Granted, this could be misused by ricers and idiots, but they'll get themselves into that mess all of their own fault and we'll all go on our merry ways. It just seems that much use could be made out of this, both in terms of (sane) optimizations and easier access to enhanced debugging opportunities.I see that people who build a Linux kernel are largely of two types:
~the ones that understand and know enough that they could, with some nudging and learning, become bonafide kernel devs and
~the ones that understand it to some very basic degree and can get through configuring it without too much trouble (though with limited understanding)
I believe one of the very simple things that can be addressed is to make the kernel more "accessible" without harming its integrity or functionality. This involves trying to fill the gap between those two types of people, allowing there to be more understanding, configuration, and (down the line) development opportunities within the kernel to better ease these people into learning enough to begin contributing back. More developers can only be a Good Thing (tm). Though a haughty and abstract opinion/goal/thought of mine, this patch conforms to it and I think that with minimal effort such a vision could be implemented. Whilst all other kernel devs are hacking away at the rough and tumultuous innards of the kernel, a few people of less confidence and experience (people like me) could polish the outside to make it more appealing to more people. So that eventually they too will sink below the surface and join the party.
Quoted 4 lines
Anyway, I'm not quite sure that my implementation is as clean as it could be and any feedback is certainly welcome. This is my first time posting to LKML, so I fully expect to be shot down anyway.
It seems to make access to these features easier, and I really don't see that it is going to create a higher number of posts here, the "I did X and..." posts might go up, and the "How do I..." post might go down, and I doubt either will change enough to notice.
Signature · 3 lines
Bill Davidsen <[email protected]> "We have more to fear from the bungling of the incompetent than from the machinations of the wicked." - from Slashdot