Clarify CONFIG_DEBUG_INFO's bloaty nature
Subsystems (2)
Borislav Petkov
maintainerPATCH 1/1 Clarify CONFIG_DEBUG_INFO's bloaty nature
From: Borislav Petkov <[email protected]>
Hold down when it makes sense to enable CONFIG_DEBUG_INFO and when CONFIG_KALLSYMS suffices.
--- lib/Kconfig.debug | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/Kconfig.debug +5 -1
@@ -126,7 +126,11 @@ config DEBUG_INFO This adds debug symbols to the kernel and modules (gcc -g), and is needed if you intend to use kernel crashdump or binary object tools like crash, kgdb, LKCD, gdb, etc on the kernel. - Say Y here only if you plan to debug the kernel. + + If you only want to have resolved symbols in kernel traces and + are not going to need support for those tools above, you don't need + to enable this as it is a huge bloat and build slowdown; + enable CONFIG_KALLSYMS instead. If unsure, say N.
Signature · 1 line
1.8.5.2.192.g7794a68
lib/Kconfig.debug config DEBUG_INFO · line 132
Quoted 13 lines · Borislav Petkov
lib/Kconfig.debug +4
-1@@ -126,7 +126,11 @@ config DEBUG_INFO This adds debug symbols to the kernel and modules (gcc -g), and is needed if you intend to use kernel crashdump or binary object tools like crash, kgdb, LKCD, gdb, etc on the kernel. - Say Y here only if you plan to debug the kernel. + + If you only want to have resolved symbols in kernel traces and + are not going to need support for those tools above, you don't need + to enable this as it is a huge bloat and build slowdown;
How do you define "huge bloat" if the size of vmlinux doesn't increase? Would some people consider that to be acceptable but now mysteriously confused because we don't know what "huge bloat" you're referring to?
It also begs the question about the meaning of DEBUG_INFO_REDUCED and this comment doesn't even allow us to see that it's an option.
So this doesn't look good.
Quoted 4 lines
lib/Kconfig.debug +1
-0+ enable CONFIG_KALLSYMS instead. If unsure, say N.
Quoted 2 lines · David Rientjes
How do you define "huge bloat" if the size of vmlinux doesn't increase?
Don't be silly. The size of all the object files increase *hugely*.
This is my fs/builtin.o in my normal config:
-rw-rw-r--. 1 torvalds torvalds 2838613 Feb 3 14:32 fs/built-in.o
and with the *reduced* debug info, ie .config diff as follows:
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_INFO_REDUCED=y
I get this:
-rw-rw-r--. 1 torvalds torvalds 11793317 Feb 3 14:33 fs/built-in.o
iow, that "reduced" debug info is still more than 4x bigger than a non-debug-info kernel.
David, if you don't think that debug-info bloats the build, you're in huge *huge* denial. CONFIG_DEBUG_INFO (even with the "reduced" thing) is ABSOLUTELY HORRIBLE CRAP.
Quoted 2 lines
Would some people consider that to be acceptable but now mysteriously confused because we don't know what "huge bloat" you're referring to?
That >4x size increase doesn't match your idea of "huge bloat"?
The object files being much bigger really does screw you especially on laptops that often have less memory and less capable IO subsystems. The final links in particular tend to be much bigger and use much more memory.
I suspect a lot of people are in denial about just how *horrible* the overhead of debug builds are. And yeah, if you have oodles of memory to cache things, it's not too bad. But you really want to have *lots* of memory, because otherwise you won't be caching all those object files in RAM, and your build easily becomes IO bound at link time. A factor of four size difference will not be helping your poor slow laptop disk..
Linus
Andrew Morton
maintainerQuoted 5 lines · On Mon, 3 Feb 2014 14:47:15 -0800 Linus Torvalds…
Quoted 2 lines · David Rientjes
How do you define "huge bloat" if the size of vmlinux doesn't increase?
Don't be silly. The size of all the object files increase *hugely*.
yup, I disable this in my allmodconfig testing, to great effect.
That being said, I do think the text should make clear that the bloat is a compile-time impact and not a runtime one. Something like
lib/Kconfig.debug renamed from lib/Kconfig.debug~lib-kconfigdebug-clarify-config_debug_infos-bloaty-nature-fix +3 -3
@@ -128,9 +128,9 @@ config DEBUG_INFO tools like crash, kgdb, LKCD, gdb, etc on the kernel. If you only want to have resolved symbols in kernel traces and - are not going to need support for those tools above, you don't need - to enable this as it is a huge bloat and build slowdown; - enable CONFIG_KALLSYMS instead. + are not going to need support for the above tools, you don't need + to enable this. It hugely bloat object files' on-disk sizes and slows + the build. Enable CONFIG_KALLSYMS instead. If unsure, say N.
_
Quoted 14 lines · Andrew Morton
Quoted 3 lines
Quoted 1 line
How do you define "huge bloat" if the size of vmlinux doesn't increase?
Don't be silly. The size of all the object files increase *hugely*.
yup, I disable this in my allmodconfig testing, to great effect.
That being said, I do think the text should make clear that the bloat is a compile-time impact and not a runtime one. Something like
lib/Kconfig.debug renamed from lib/Kconfig.debug~lib-kconfigdebug-clarify-config_debug_infos-bloaty-nature-fix +0
-0@@ -128,9 +128,9 @@ config DEBUG_INFO tools like crash, kgdb, LKCD, gdb, etc on the kernel.
lib/Kconfig.debug config DEBUG_INFO · line 136
Quoted 10 lines
lib/Kconfig.debug renamed from lib/Kconfig.debug~lib-kconfigdebug-clarify-config_debug_infos-bloaty-nature-fix +3
-3If you only want to have resolved symbols in kernel traces and - are not going to need support for those tools above, you don't need - to enable this as it is a huge bloat and build slowdown; - enable CONFIG_KALLSYMS instead. + are not going to need support for the above tools, you don't need + to enable this. It hugely bloat object files' on-disk sizes and slows + the build. Enable CONFIG_KALLSYMS instead. If unsure, say N.
Um, I still don't think this is right since CONFIG_DEBUG_INFO_REDUCED requires you to say yes here and that config option actually specifies the tools for which it is effective. Are you suggesting that we decouple CONFIG_DEBUG_INFO from CONFIG_DEBUG_INFO_REDUCED and make CONFIG_DEBUG_INFO select the reduced variant? Or... what?
Quoted 24 lines · David Rientjes
Quoted 22 lines · Andrew Morton
Quoted 3 lines
Quoted 1 line
How do you define "huge bloat" if the size of vmlinux doesn't increase?
Don't be silly. The size of all the object files increase *hugely*.
yup, I disable this in my allmodconfig testing, to great effect.
That being said, I do think the text should make clear that the bloat is a compile-time impact and not a runtime one. Something like
lib/Kconfig.debug renamed from lib/Kconfig.debug~lib-kconfigdebug-clarify-config_debug_infos-bloaty-nature-fix +3
-3@@ -128,9 +128,9 @@ config DEBUG_INFO tools like crash, kgdb, LKCD, gdb, etc on the kernel. If you only want to have resolved symbols in kernel traces and - are not going to need support for those tools above, you don't need - to enable this as it is a huge bloat and build slowdown; - enable CONFIG_KALLSYMS instead. + are not going to need support for the above tools, you don't need + to enable this. It hugely bloat object files' on-disk sizes and slows + the build. Enable CONFIG_KALLSYMS instead.
Quoted 9 lines
lib/Kconfig.debug config DEBUG_INFO · line 136
Quoted 2 lines
lib/Kconfig.debug renamed from lib/Kconfig.debug~lib-kconfigdebug-clarify-config_debug_infos-bloaty-nature-fix +0
-0If unsure, say N.Um, I still don't think this is right since
CONFIG_DEBUG_INFO_REDUCEDrequires you to say yes here and that config option actually specifies the tools for which it is effective. Are you suggesting that we decoupleCONFIG_DEBUG_INFOfromCONFIG_DEBUG_INFO_REDUCEDand makeCONFIG_DEBUG_INFOselect the reduced variant? Or... what?
I think the right solution is what I suggested in another recent discussion:
"So to reign in debuginfo in allyesconfig/allmodconfig builds we could
do something like:
config SAVE_TIME_AND_DISK_SPACE
bool "Faster and leaner build object files: compile without debug info" default y
choice
prompt "Choose DEBUGINFO bloat level" depends on !SAVE_TIME_AND_DISK_SPACE default DEBUG_INFO_REDUCED
help
This option allows to select the debuginfo model.
config DEBUG_INFO_REDUCED
bool "Reduced amount of debugging information, 4x regular .o size"
config DEBUG_INFO
bool "Full debuginfo, 10x regular .o size, 'planet killer' version"
endchoice
That would default to Y and would disable debuginfo by default."
Thanks,
Ingo
Quoted 4 lines · Ingo Molnar
config
SAVE_TIME_AND_DISK_SPACE
bool "Faster and leaner build object files: compile without debug info" default y
Thinking about it, we already have this. It's called "CONFIG_COMPILE_TEST".
It very much gets set by all{yes,mod}config, and it also says "we're interested in testing the compile, not so much running the result".
So let's just make the DEBUG_INFO depend on !COMPILE_TEST.
Trivial oneliner patch (independent of the discussion about improving the documentation), so I'll just apply it.
I can't imagine that this would be controversial:
config DEBUG_INFO
bool "Compile the kernel with debug info"
- depends on DEBUG_KERNEL
+ depends on DEBUG_KERNEL && !COMPILE_TEST
help
Hm?
Linus
Quoted 6 lines · Linus Torvalds
Trivial oneliner patch (independent of the discussion about improving the documentation), so I'll just apply it.
I can't imagine that this would be controversial:
Quoted 7 lines
config
DEBUG_INFO
bool "Compile the kernel with debug info"
- depends onDEBUG_KERNEL
+ depends onDEBUG_KERNEL&& !COMPILE_TEST
helpHm?
I have used debug info on allyesconfig in the past to run statistics on the resulting binary. It's still useful, even though quite slow. Make it depend on !EXPERT or somesuch? I agree for normal usages it's fine.
BTW you probably also want to do the same for the x86 instruction decoder test. It seems to be the biggest compile time waster in allyesconfig and I usually disable it too.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
Quoted 4 lines · Andi Kleen
I have used debug info on allyesconfig in the past to run statistics on the resulting binary. It's still useful, even though quite slow. Make it depend on !EXPERT or somesuch? I agree for normal usages it's fine.
EXPERT is useless, people set it randomly.
I'd suggest that for any statistics gathering, you just disable COMPILE_TEST. You can do that by just editing the .config file after doing the allyesconfig, or by pre-seeding the allyesconfig with COMPILE_TEST disabled.
Your use-case is rather special, after all. I do *not* believe that we should make one of the goals of "make all{yes,mod}config" be that kind of odd specialized thing.
Quoted 3 lines
BTW you probably also want to do the same for the x86 instruction decoder test. It seems to be the biggest compile time waster in allyesconfig and I usually disable it too.
Agreed, that should be disabled too by default. Although in that case I think the COMPILE_TEST combination makes more sense, since it's a coverage issue.
Suggestions for symbols?
Linus
PATCH 1/1 x86: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
Quoted 7 lines · Linus Torvalds
Quoted 3 lines
BTW you probably also want to do the same for the x86 instruction decoder test. It seems to be the biggest compile time waster in allyesconfig and I usually disable it too.
Agreed, that should be disabled too by default. Although in that case I think the
COMPILE_TESTcombination makes more sense, since it's a coverage issue.
I'd say randconfig will cover it often enough, and the failure is also borderline build coverage related: you cannot really make the decoder test fail via source level changes, only with changes in the build environment, so I agree with Andi that we can disable this one too - via the patch below.
Thanks,
Ingo
arch/x86/Kconfig.debug +5 -0
@@ -184,6 +184,11 @@ config HAVE_MMIOTRACE_SUPPORT config X86_DECODER_SELFTEST bool "x86 instruction decoder selftest" depends on DEBUG_KERNEL && KPROBES + # + # It can take some time to validate the image, + # make sure {allyes|allmod}config doesn't enable it: + # + depends on !COMPILE_TEST ---help--- Perform x86 instruction decoder selftests at build time. This option is useful for checking the sanity of x86 instruction
Re: [PATCH] x86: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
Quoted 11 lines · Ingo Molnar
Quoted 7 lines · Linus Torvalds
Quoted 3 lines
BTW you probably also want to do the same for the x86 instruction decoder test. It seems to be the biggest compile time waster in allyesconfig and I usually disable it too.
Agreed, that should be disabled too by default. Although in that case I think the
COMPILE_TESTcombination makes more sense, since it's a coverage issue.
Quoted 5 lines
I'd say randconfig will cover it often enough, and the failure is also borderline build coverage related: you cannot really make the decoder test fail via source level changes, only with changes in the build environment, so I agree with Andi that we can disable this one too - via the patch below.
For the record, even with a (currently) modern CPU such as a Xeon E5-2630 @ 2.30GHz, the script takes 4m30s on 3.14-rc1 i386 allyesconfig. And since it runs single threaded at the end of the build, it really adds that amount to the wall time spent on the build. So enthusiastically Acked, FWIW.
Paul.
--
Quoted 28 lines
Thanks,
Ingo
Signed-off-by Ingo Molnar <[email protected]>arch/x86/Kconfig.debug +5
-1@@ -184,6 +184,11 @@ config HAVE_MMIOTRACE_SUPPORT config X86_DECODER_SELFTEST bool "x86 instruction decoder selftest" depends on DEBUG_KERNEL && KPROBES + # + # It can take some time to validate the image, + # make sure {allyes|allmod}config doesn't enable it: + # + depends on !COMPILE_TEST ---help--- Perform x86 instruction decoder selftests at build time. This option is useful for checking the sanity of x86 instruction --To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
Quoted 5 lines
For the record, even with a (currently) modern CPU such as a Xeon E5-2630 @ 2.30GHz, the script takes 4m30s on 3.14-rc1 i386 allyesconfig. And since it runs single threaded at the end of the build, it really adds that amount to the wall time spent on the build. So enthusiastically Acked, FWIW.
I suppose we could just multi thread it. It should "embarassingly" scale without problems. But turning it off by default is a first step.
-Andi
tip-bot for Ingo Molnar
merge bot[tip:x86/urgent] x86/debug: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
In reply to Ingo Molnarx86/debug: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
Commit-ID: 8c778d541bec8a41cbd8fa73b7cba90b2d6606c1 Gitweb: http://git.kernel.org/tip/8c778d541bec8a41cbd8fa73b7cba90b2d6606c1 Author: Ingo Molnar <[email protected]> AuthorDate: Wed, 5 Feb 2014 06:51:37 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Sun, 9 Feb 2014 15:18:42 +0100
x86/debug: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
* Linus Torvalds <[email protected]> wrote:
Andi Kleen suggested to turn off CONFIG_X86_DECODER_SELFTEST=y in allyesconfig, because it costs a lot of CPU time.
(In practice randconfig builds will cover this test frequently
enough to provide build coverage.)
--- arch/x86/Kconfig.debug | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/Kconfig.debug +5 -0
@@ -185,6 +185,11 @@ config X86_DECODER_SELFTEST bool "x86 instruction decoder selftest" depends on DEBUG_KERNEL && KPROBES depends on !COMPILE_TEST + # + # It can take some time to validate the image, + # make sure {allyes|allmod}config doesn't enable it: + # + depends on !COMPILE_TEST ---help--- Perform x86 instruction decoder selftests at build time. This option is useful for checking the sanity of x86 instruction
Re: [PATCH] x86: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
In reply to Ingo MolnarHi Ingo,
Quoted 30 lines · On Wed, 5 Feb 2014 06:51:37 +0100 Ingo Molnar <[email protected]> wrote:
Quoted 7 lines · Linus Torvalds
Quoted 3 lines
BTW you probably also want to do the same for the x86 instruction decoder test. It seems to be the biggest compile time waster in allyesconfig and I usually disable it too.
Agreed, that should be disabled too by default. Although in that case I think the
COMPILE_TESTcombination makes more sense, since it's a coverage issue.I'd say randconfig will cover it often enough, and the failure is also borderline build coverage related: you cannot really make the decoder test fail via source level changes, only with changes in the build environment, so I agree with Andi that we can disable this one too - via the patch below.
Thanks,
Ingo
Signed-off-by Ingo Molnar <[email protected]>arch/x86/Kconfig.debug +0
-0@@ -184,6 +184,11 @@ config HAVE_MMIOTRACE_SUPPORT config X86_DECODER_SELFTEST
arch/x86/Kconfig.debug config HAVE_MMIOTRACE_SUPPORT · line 194
Quoted 10 lines
arch/x86/Kconfig.debug +5
-0bool "x86 instruction decoder selftest" depends on DEBUG_KERNEL && KPROBES + # + # It can take some time to validate the image, + # make sure {allyes|allmod}config doesn't enable it: + # + depends on !COMPILE_TEST ---help--- Perform x86 instruction decoder selftests at build time. This option is useful for checking the sanity of x86 instruction
When this got applied to the tip tree, Linus had already applied an earlier version as commit f8f202348208, and this one (commit 8c778d541bec) now duplicates the actual dependency in !COMPILE_TEST (but does add a useful comment) ...
Signature · 2 lines
Cheers, Stephen Rothwell <[email protected]>
Re: [PATCH] x86: Disable CONFIG_X86_DECODER_SELFTEST in allmod/allyesconfigs
Quoted 40 lines · Stephen Rothwell
Hi Ingo,
Quoted 37 lines · On Wed, 5 Feb 2014 06:51:37 +0100 Ingo Molnar <[email protected]> wrote:
Quoted 7 lines · Linus Torvalds
Quoted 3 lines
BTW you probably also want to do the same for the x86 instruction decoder test. It seems to be the biggest compile time waster in allyesconfig and I usually disable it too.
Agreed, that should be disabled too by default. Although in that case I think the
COMPILE_TESTcombination makes more sense, since it's a coverage issue.I'd say randconfig will cover it often enough, and the failure is also borderline build coverage related: you cannot really make the decoder test fail via source level changes, only with changes in the build environment, so I agree with Andi that we can disable this one too - via the patch below.
Thanks,
Ingo
Signed-off-by Ingo Molnar <[email protected]>arch/x86/Kconfig.debug +5
-0@@ -184,6 +184,11 @@ config HAVE_MMIOTRACE_SUPPORT config X86_DECODER_SELFTEST bool "x86 instruction decoder selftest" depends on DEBUG_KERNEL && KPROBES + # + # It can take some time to validate the image, + # make sure {allyes|allmod}config doesn't enable it: + # + depends on !COMPILE_TEST
Quoted 8 lines
arch/x86/Kconfig.debug config HAVE_MMIOTRACE_SUPPORT · line 194
Quoted 3 lines
arch/x86/Kconfig.debug +0
-0---help--- Perform x86 instruction decoder selftests at build time. This option is useful for checking the sanity of x86 instructionWhen this got applied to the tip tree, Linus had already applied an earlier version as commit
f8f202348208, and this one (commit8c778d541bec) now duplicates the actual dependency in !COMPILE_TEST (but does add a useful comment) ...
Ok, I've zapped it from x86/urgent, thanks for letting me know!
Ingo
Quoted 18 lines · Linus Torvalds
Quoted 4 lines · Ingo Molnar
config
SAVE_TIME_AND_DISK_SPACE
bool "Faster and leaner build object files: compile without debug info" default yThinking about it, we already have this. It's called "
CONFIG_COMPILE_TEST".It very much gets set by all{yes,mod}config, and it also says "we're interested in testing the compile, not so much running the result".
So let's just make the
DEBUG_INFOdepend on !COMPILE_TEST.Trivial oneliner patch (independent of the discussion about improving the documentation), so I'll just apply it.
I can't imagine that this would be controversial:
Quoted 7 lines
config
DEBUG_INFO
bool "Compile the kernel with debug info"
- depends onDEBUG_KERNEL
+ depends onDEBUG_KERNEL&& !COMPILE_TEST
helpHm?
Fully agreed.
Thanks,
Ingo
Borislav Petkov
maintainerQuoted 16 lines · Andrew Morton
Quoted 5 lines · On Mon, 3 Feb 2014 14:47:15 -0800 Linus Torvalds…
Quoted 2 lines · David Rientjes
How do you define "huge bloat" if the size of vmlinux doesn't increase?
Don't be silly. The size of all the object files increase *hugely*.
yup, I disable this in my allmodconfig testing, to great effect.
That being said, I do think the text should make clear that the bloat is a compile-time impact and not a runtime one. Something like
lib/Kconfig.debug config DEBUG_INFO · line 133
Quoted 9 lines
lib/Kconfig.debug renamed from lib/Kconfig.debug~lib-kconfigdebug-clarify-config_debug_infos-bloaty-nature-fix +3
-3@@ -128,9 +128,9 @@ config DEBUG_INFO tools like crash, kgdb, LKCD, gdb, etc on the kernel. If you only want to have resolved symbols in kernel traces and - are not going to need support for those tools above, you don't need - to enable this as it is a huge bloat and build slowdown; - enable CONFIG_KALLSYMS instead. + are not going to need support for the above tools, you don't need + to enable this. It hugely bloat object files' on-disk sizes and slows + the build. Enable CONFIG_KALLSYMS instead.
Yes, this is better.
Andrew, can you add that or you want me to send a new version?
Thanks.
Signature · 5 lines
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--