build system: section garbage collection for vmlinux
Subsystems (2)
cover build system: section garbage collection for vmlinux
Build system: section garbage collection for vmlinux
Newer gcc and binutils can do dead code and data removal at link time. It is achieved using combination of -ffunction-sections -fdata-sections options for gcc and --gc-sections for ld.
Theory of operation:
Option -ffunction-sections instructs gcc to place each function (including static ones) in it's own section named .text.function_name instead of placing all functions in one big .text section.
At link time, ld normally coalesce all such sections into one output section .text again. It is achieved by having *(.text.*) spec along with *(.text) spec in built-in linker scripts.
If ld is invoked with --gc-sections, it tracks references, starting from entry point and marks all input sections which are reachable from there. Then it discards all input sections which are not marked.
This isn't buying much if you have one big .text section per .o module, because even one referenced function will pull in entire section. You need -ffunction-sections in order to split .text into per-function sections and make --gc-sections much more useful.
-fdata-sections is analogous: it places each global or static variable into .data.variable_name, .rodata.variable_name or .bss.variable_name.
How to use it in kernel:
First, we need to adapt existing code for new section names. Basically, we need to stop using section names of the form .text.xxxx
.data.xxxx
.rodata.xxxx
.bss.xxxx
in the kernel for - otherwise section placement done by kernel's custom linker scripts produces broken vmlinux and vdso images.
Second, kernel linker scripts need to be adapted by adding KEEP(xxx) directives around sections which are not directly referenced, but are nevertheless used (initcalls, altinstructions, etc).
These patches fix section names and add CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.
Patches are conservative and mark a lot of things with KEEP() directive in linker script, inhibiting GC for them. Size savings are modest:
text data bss dec hex filename
5159478 1005139 406784 6571401 644589 linux-2.6.23-rc4.org/vmlinux 5131822 996090 401439 6529351 63a147 linux-2.6.23-rc4.gc/vmlinux
In this particular case, 402 objects were discarded, saving 42 kb.
Linker is unable to discard more because current infrastructure is a bit flawed in this regard. It prevents some unused code from being detected. In particular:KEEP(__ex_table) -> .fixup -> get_user and friends KEEP(.smp_locks) -> lock prefixes
This is an experimental build where KEEPs for them were removed:
text data bss dec hex filename
5131822 996090 401439 6529351 63a147 vmlinux 5065681 996090 401439 6463210 629eea vmlinux.sans_KEEP
52k of difference are due to __ex_table and .smp_locks being removed, the remaining ~13k is genuinely unused .text.* sections.
Patches were run-tested on x86_64, and likely do not work on any other arch (need to add KEEP() to arch/*/kernel/vmlinux.lds.S for each arch). --
vda
PATCH 1/3 build system: section garbage collection for vmlinux
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.
Part 1: fix section names over entire source (all arches).
Patch is big and boring global s/.text.lock/.text_lock/ type thing.
You can regenerate it using attachedlinux-2.6.23-rc4.0.fixname.sh
(e.g. if you need to rebase to later kernel).
--
vda
PATCH 2/3 build system: section garbage collection for vmlinux
Quoted 8 lines · Denys Vlasenko
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.Part 1: fix section names over entire source (all arches).
Part 2: fix x86_64 vdso linker script to not produce broken vdso image with gcc -ffunction-sections -fdata-sections.
Does not affect normal build.
--
vda
arch/x86_64/vdso/vdso.lds.S +3 -3
@@ -26,11 +26,11 @@ is insufficient, ld -shared will barf. Just increase it here. */ . = VDSO_PRELINK + VDSO_TEXT_OFFSET; - .text : { *(.text) } :text + .text : { *(.text) *(.text.*) } :text .text_ptr : { *(.text_ptr) } :text . = VDSO_PRELINK + 0x900; - .data : { *(.data) } :text - .bss : { *(.bss) } :text + .data : { *(.data) *(.data.*) } :text + .bss : { *(.bss) *(.bss.*) } :text .altinstructions : { *(.altinstructions) } :text .altinstr_replacement : { *(.altinstr_replacement) } :text
PATCH 3/3 build system: section garbage collection for vmlinux
Quoted 8 lines · Denys Vlasenko
Quoted 7 lines · Denys Vlasenko
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.
Quoted 4 lines
Quoted 1 line
Part 1: fix section names over entire source (all arches).
Part 2: fix
x86_64vdso linker script to not produce broken vdso image with gcc -ffunction-sections -fdata-sections.
Part 3:
Makefile:
init/Kconfig:
add config DISCARD_UNUSED_SECTIONS with appropriate big scary warning. It enables gcc and ld options for section garbage collection.
arch/x86_64/kernel/vmlinux.lds.S:include/asm-generic/vmlinux.lds.h:
add KEEP and SORT_BY_ALIGNMENT directives, as needed.
arch/frv/Makefile:
had half-baked option similar to DISCARD_UNUSED_SECTIONS, replace it.
DISCARD_UNUSED_SECTIONS=n should be safe for all arches.
DISCARD_UNUSED_SECTIONS=y is usable only for x86_64 at the moment. --
vda
Makefile +5 -0
@@ -508,6 +508,11 @@ CFLAGS += $(call cc-option, -fn NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include) CHECKFLAGS += $(NOSTDINC_FLAGS) +ifdef CONFIG_DISCARD_UNUSED_SECTIONS +CFLAGS += $(call cc-option, -ffunction-sections -fdata-sections) +LDFLAGS_vmlinux += $(call ld-option, -gc-sections) +endif + # warn about C99 declaration after statement CFLAGS += $(call cc-option,-Wdeclaration-after-statement,)
arch/frv/Makefile +3 -1
@@ -52,7 +52,9 @@ endif #LDFLAGS_vmlinux := -Map linkmap.txt -ifdef CONFIG_GC_SECTIONS +# Is this needed? We do this already in kernel's top-level Makefile. +# $(LINKFLAGS) seems to be unused. +ifdef CONFIG_DISCARD_UNUSED_SECTIONS CFLAGS += -ffunction-sections -fdata-sections LINKFLAGS += --gc-sections endif
arch/x86_64/kernel/vmlinux.lds.S +29 -26
158 lines not on this page · raw message
include/asm-generic/vmlinux.lds.h +46 -37
210 lines not on this page · raw message
init/Kconfig +14 -0
@@ -347,6 +347,20 @@ config CC_OPTIMIZE_FOR_SIZE If unsure, say N. +config DISCARD_UNUSED_SECTIONS + bool "Discard unused code/data sections (DANGEROUS)" + default n + depends on EXPERIMENTAL + help + Enabling this option will pass --ffunction-sections -fdata-sections + to gcc and --gc-sections to ld, resulting in a smaller kernel. + + WARNING: --gc-sections support is very new and considered highly + experimental for now. You need at least binutils 2.18, + and even then surprises are likely. + + If unsure, say N. + config SYSCTL bool
Quoted 7 lines · Denys Vlasenko
Part 3:
Makefile:
init/Kconfig:
add configDISCARD_UNUSED_SECTIONSwith appropriate big scary warning. It enables gcc and ld options for section garbage collection.
At it typically happens, last-minute "obviously correct" change was a mistake.
This doesn't work as intended:
LDFLAGS_vmlinux += $(call ld-option, --gc-sections)
With the above line, --gc-sections doesn't get added, and vmlinux is not garbage collected.
It must be
LDFLAGS_vmlinux += --gc-sections
Sorry.
--
vda
Quoted 9 lines · Denys Vlasenko
Quoted 7 lines · Denys Vlasenko
Part 3:
Makefile:
init/Kconfig:
add configDISCARD_UNUSED_SECTIONSwith appropriate big scary warning. It enables gcc and ld options for section garbage collection.
Quoted 8 lines
At it typically happens, last-minute "obviously correct" change was a mistake.
This doesn't work as intended:
LDFLAGS_vmlinux+= $(call ld-option, --gc-sections)With the above line, --gc-sections doesn't get added, and vmlinux is not garbage collected.
Did you find out why it does not work?
Quoted 4 lines
It must be
LDFLAGS_vmlinux+= --gc-sections
Doing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
I have already tried to introduce such a preparatioanl link
of vmlinux - patch attached.
I skipped this patch because suprisingly it was no win for a kernelbuild. ld seems not to be more effective using a prelinked vmlinux.o compared to several .o files.
I know ths prelinked vmlinux.o does not include the init stuff but that is so little that it is not interesting for your patch.
Sam
Makefile +15 -25
@@ -592,7 +592,7 @@ libs-y := $(libs-y1) $(libs-y2) # +-< $(vmlinux-init) # | +--< init/version.o + more # | -# +--< $(vmlinux-main) +# +--< vmlinux.o # | +--< driver/built-in.o mm/built-in.o + more # | # +-< kallsyms.o (see description in CONFIG_KALLSYMS section)
@@ -608,17 +608,16 @@ libs-y := $(libs-y1) $(libs-y2) vmlinux-init := $(head-y) $(init-y) vmlinux-main := $(core-y) $(libs-y) $(drivers-y) $(net-y) -vmlinux-all := $(vmlinux-init) $(vmlinux-main) vmlinux-lds := arch/$(ARCH)/kernel/vmlinux.lds -export KBUILD_VMLINUX_OBJS := $(vmlinux-all) +export KBUILD_VMLINUX_OBJS := $(vmlinux-init) vmlinux.o # Rule to link vmlinux - also used during CONFIG_KALLSYMS # May be overridden by arch/$(ARCH)/Makefile quiet_cmd_vmlinux__ ?= LD $@ cmd_vmlinux__ ?= $(LD) $(LDFLAGS) $(LDFLAGS_vmlinux) -o $@ \ -T $(vmlinux-lds) $(vmlinux-init) \ - --start-group $(vmlinux-main) --end-group \ - $(filter-out $(vmlinux-lds) $(vmlinux-init) $(vmlinux-main) vmlinux.o FORCE ,$^) + --start-group vmlinux.o --end-group \ + $(filter-out $(vmlinux-lds) $(vmlinux-init) vmlinux.o FORCE ,$^) # Generate new vmlinux version quiet_cmd_vmlinux_version = GEN .version
@@ -718,13 +717,13 @@ quiet_cmd_kallsyms = KSYM $@ $(call cmd,kallsyms) # .tmp_vmlinux1 must be complete except kallsyms, so update vmlinux version -.tmp_vmlinux1: $(vmlinux-lds) $(vmlinux-all) FORCE +.tmp_vmlinux1: $(vmlinux-lds) $(vmlinux-init) vmlinux.o FORCE $(call if_changed_rule,ksym_ld) -.tmp_vmlinux2: $(vmlinux-lds) $(vmlinux-all) .tmp_kallsyms1.o FORCE +.tmp_vmlinux2: $(vmlinux-lds) $(vmlinux-init) vmlinux.o .tmp_kallsyms1.o FORCE $(call if_changed,vmlinux__) -.tmp_vmlinux3: $(vmlinux-lds) $(vmlinux-all) .tmp_kallsyms2.o FORCE +.tmp_vmlinux3: $(vmlinux-lds) $(vmlinux-init) vmlinux.o .tmp_kallsyms2.o FORCE $(call if_changed,vmlinux__) # Needs to visit scripts/ before $(KALLSYMS) can be used.
@@ -742,31 +741,22 @@ debug_kallsyms: .tmp_map$(last_kallsyms) endif # ifdef CONFIG_KALLSYMS -# Do modpost on a prelinked vmlinux. The finally linked vmlinux has -# relevant sections renamed as per the linker script. -quiet_cmd_vmlinux-modpost = LD $@ - cmd_vmlinux-modpost = $(LD) $(LDFLAGS) -r -o $@ \ - $(vmlinux-init) --start-group $(vmlinux-main) --end-group \ - $(filter-out $(vmlinux-init) $(vmlinux-main) $(vmlinux-lds) FORCE ,$^) -define rule_vmlinux-modpost - : - +$(call cmd,vmlinux-modpost) - $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost $@ - $(Q)echo 'cmd_$@ := $(cmd_vmlinux-modpost)' > $(dot-target).cmd -endef +# Link the kernel except init parts - for use in subsequent links +quiet_cmd_vmlinux-main = LD $@ + cmd_vmlinux-main = $(LD) $(LDFLAGS) -r -o $@ \ + --start-group $(filter-out FORCE, $^) --end-group + +vmlinux.o: $(vmlinux-main) FORCE + $(call if_changed,vmlinux-main) # vmlinux image - including updated kernel symbols -vmlinux: $(vmlinux-lds) $(vmlinux-init) $(vmlinux-main) $(kallsyms.o) vmlinux.o FORCE +vmlinux: $(vmlinux-lds) $(vmlinux-init) $(kallsyms.o) vmlinux.o FORCE ifdef CONFIG_HEADERS_CHECK $(Q)$(MAKE) -f $(srctree)/Makefile headers_check endif - $(call vmlinux-modpost) $(call if_changed_rule,vmlinux__) $(Q)rm -f .old_version -vmlinux.o: $(vmlinux-lds) $(vmlinux-init) $(vmlinux-main) $(kallsyms.o) FORCE - $(call if_changed_rule,vmlinux-modpost) - # The actual objects are generated when descending, # make sure no implicit rule kicks in $(sort $(vmlinux-init) $(vmlinux-main)) $(vmlinux-lds): $(vmlinux-dirs) ;
Quoted 12 lines · Sam Ravnborg
Quoted 11 lines · Denys Vlasenko
Quoted 7 lines · Denys Vlasenko
Part 3:
Makefile:
init/Kconfig:
add configDISCARD_UNUSED_SECTIONSwith appropriate big scary warning. It enables gcc and ld options for section garbage collection.At it typically happens, last-minute "obviously correct" change was a mistake.
Quoted 8 lines
Quoted 6 lines
This doesn't work as intended:
LDFLAGS_vmlinux+= $(call ld-option, --gc-sections)With the above line, --gc-sections doesn't get added, and vmlinux is not garbage collected.
Did you find out why it does not work?
ld-option is designed to test whether -Wl,-Wsomething will work on gcc commandline. I am adding --gc-sections to ld commandline.
Quoted 9 lines
Quoted 4 lines
It must be
LDFLAGS_vmlinux+= --gc-sectionsDoing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
Yes, this will speed up things a bit.
However, for me build time is totally dominated by CC stages, not LD. I don't have 32 core CPU yet :(
--
vda
Quoted 9 lines · Denys Vlasenko
Quoted 5 lines
Doing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
Yes, this will speed up things a bit.
However, for me build time is totally dominated by CC stages, not LD.
A typical developer workflow is editing a single file
and then build the kernel.
Here CC has much less impact than LD - and this is the case we shall optimize for.
Sam
Quoted 7 lines
Quoted 5 lines
Doing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
Yes, this will speed up things a bit.
If we do the --gc-sections trick during the preparational link then we do not use the arch supplied linker script.
Will it be possible to create a dedicated linker script that is valid for all architectures and which only include the KEEP() directives for the diverse sections?
That would also free us from modifying a lot of arch linker scripts uglifying them even more than they are today.
Sam
Quoted 6 lines · Sam Ravnborg
Quoted 6 lines
Quoted 5 lines
Doing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
Quoted 7 lines
Quoted 1 line
Yes, this will speed up things a bit.
If we do the --gc-sections trick during the preparational link then we do not use the arch supplied linker script.
Will it be possible to create a dedicated linker script that is valid for all architectures and which only include theKEEP()directives for the diverse sections?
Unfortunately, -r and --gc-sections don't mix.
x86_64-pc-linux-gnu-ld: --gc-sections and -r may not be used together --
vda
Quoted 15 lines · Denys Vlasenko
Quoted 6 lines · Sam Ravnborg
Quoted 6 lines
Quoted 5 lines
Doing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
Quoted 7 lines
Quoted 1 line
Yes, this will speed up things a bit.
If we do the --gc-sections trick during the preparational link then we do not use the arch supplied linker script.
Will it be possible to create a dedicated linker script that is valid for all architectures and which only include theKEEP()directives for the diverse sections?
Quoted 3 lines
Unfortunately, -r and --gc-sections don't mix.
x86_64-pc-linux-gnu-ld: --gc-sections and -r may not be used together
OK - so much for that optimization :-(
But then we need to annotate ALL arch linker script before introducing this. And that bring me back to that we should put some sanity into these first.
Sam
Quoted 16 lines · Sam Ravnborg
Quoted 15 lines · Denys Vlasenko
Quoted 13 lines · Sam Ravnborg
Quoted 7 lines
Quoted 5 lines
Doing a normal kernel build will link vmlinux three or four times. If we introduce --gc-sections we should add a preparational link of vmlinux where we use --gc-sections and skip it for the rest of the links assuming that --gc-sections takes some time for ld to do.
Yes, this will speed up things a bit.
If we do the --gc-sections trick during the preparational link then we do not use the arch supplied linker script.
Will it be possible to create a dedicated linker script that is valid for all architectures and which only include theKEEP()directives for the diverse sections?
Quoted 8 lines
Quoted 3 lines
Unfortunately, -r and --gc-sections don't mix.
x86_64-pc-linux-gnu-ld: --gc-sections and -r may not be used together
OK - so much for that optimization :-(
But then we need to annotate ALL arch linker script before introducing this. And that bring me back to that we should put some sanity into these first.
I was working with x86_64 ld script and am willing to clean it up a bit.
I can impelment and test DISCARD_UNUSED_SECTIONS for x86_64 and later for i386.
Other arches can follow when they find it interesting/worthwhile.
At first, big scary warning under "config DISCARD_UNUSED_SECTIONS" should be enough to make people avoid it for production boxes, I hope.
Should I send next round of patches to you or to Andrew? --
vda
Quoted 8 lines
Quoted 2 lines
But then we need to annotate ALL arch linker script before introducing this. And that bring me back to that we should put some sanity into these first.
I was working with
x86_64ld script and am willing to clean it up a bit.I can impelment and test
DISCARD_UNUSED_SECTIONSforx86_64and later for i386.
Quoted 6 lines
Other arches can follow when they find it interesting/worthwhile.
At first, big scary warning under "config
DISCARD_UNUSED_SECTIONS" should be enough to make people avoid it for production boxes, I hope.Should I send next round of patches to you or to Andrew?
Please send them to me with appropriate cc's.
Sam
Quoted 11 lines · Denys Vlasenko
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.Part 1: fix section names over entire source (all arches).
Patch is big and boring global s/.text.lock/.text_lock/ type thing.
The normal naming scheme seems to be:
.<usage>.text so in your example it would be: .lock.text See the naming og init and exit sections (that was renamed during 2.5 to be compatible with -ffunction-sections).
Sam
Quoted 10 lines · Sam Ravnborg
Quoted 9 lines · Denys Vlasenko
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.Part 1: fix section names over entire source (all arches).
Quoted 7 lines
Quoted 2 lines
Patch is big and boring global s/.text.lock/.text_lock/ type thing.
The normal naming scheme seems to be:
.<usage>.text so in your example it would be: .lock.text See the naming og init and exit sections (that was renamed during 2.5 to be compatible with -ffunction-sections).
Thanks, will do it that way. I plan to re-submit patches for inclusion into 2.6.24.
--
vda
Quoted 14 lines · Denys Vlasenko
Quoted 13 lines · Sam Ravnborg
Quoted 11 lines · Denys Vlasenko
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.Part 1: fix section names over entire source (all arches).
Patch is big and boring global s/.text.lock/.text_lock/ type thing.
Quoted 7 lines
Quoted 4 lines
The normal naming scheme seems to be:
.<usage>.text so in your example it would be: .lock.text See the naming og init and exit sections (that was renamed during 2.5 to be compatible with -ffunction-sections).Thanks, will do it that way. I plan to re-submit patches for inclusion into 2.6.24.
This should simmer in -mm for a few weeks at minimum before hitting mainline. I would cosider it 2.6.25 material.
We could use the 2.6.23 timeframe to bring up all the linker script file to a level where adding all the KEEP's are mostly done in the generic vmlinux.h file.
Sam
Quoted 10 lines · Sam Ravnborg
Quoted 9 lines · Denys Vlasenko
Quoted 5 lines · Denys Vlasenko
These patches fix section names and add
CONFIG_DISCARD_UNUSED_SECTIONS. It is not enabled unconditionally because only newest binutils have ld --gc-sections which is stable enough for kernel use.
IOW: this is an experimental feature for now.Part 1: fix section names over entire source (all arches).
Quoted 7 lines
Quoted 2 lines
Patch is big and boring global s/.text.lock/.text_lock/ type thing.
The normal naming scheme seems to be:
.<usage>.text so in your example it would be: .lock.text See the naming of init and exit sections (that was renamed during 2.5 to be compatible with -ffunction-sections).
Well, there seems to be a problem at least with .bss:
http://sourceware.org/bugzilla/show_bug.cgi?id=5006
With __attribute__((section(".bss.page_aligned"))) gcc will produce .bss.page_aligned section with NOBITS attribute, purely on the basis of section name starting by '.bss.'
With __attribute__((section(".bss_page_aligned"))), section will get PROGBITS attribute instead.
Combining NOBITS and PROGBITS sections into one .bss section is not funny.
IOW: at least for bss, we _must_ use ".bss.xxx" names.
I propose (and will implement in next round of patches) .bss.k.page_aligned ('k' for 'kernel').
Lickily, we alctually have only one special bss section on kernel today.
Sam, my question - should I also do the same for text/rodata/data, just for paranoid reasons?
--
vda
* Wed, 5 Sep 2007 14:43:21 +0100
* User-Agent: KMail/1.9.1
Quoted 3 lines
Build system: section garbage collection for vmlinux
Maybe this is just a test suit to get finish with make XYZ static?
____
Quoted 6 lines · Oleg Verych
* Wed, 5 Sep 2007 14:43:21 +0100
* User-Agent: KMail/1.9.1Quoted 2 lines
Build system: section garbage collection for vmlinux
Maybe this is just a test suit to get finish with
make XYZ static?
They are vaguely connected in a sense that unused function which is not marked static doesn't generate gcc warning, but will be discarded by --gc-sections. "make XYZ static" also tends to find them - you make function static, you recompile the file, and gcc informs you that the function is not used at all!
This happened to me when I did aic7xxx patches.
You may yse --print-gc-sections to see the list of discarded sections. --
vda
Quoted 14 lines · Denys Vlasenko
Quoted 6 lines · Oleg Verych
* Wed, 5 Sep 2007 14:43:21 +0100
* User-Agent: KMail/1.9.1Quoted 2 lines
Build system: section garbage collection for vmlinux
Maybe this is just a test suit to get finish with
make XYZ static?They are vaguely connected in a sense that unused function which is not marked static doesn't generate gcc warning, but will be discarded by --gc-sections. "make XYZ static" also tends to find them - you make function static, you recompile the file, and gcc informs you that the function is not used at all!
Quoted 3 lines
This happened to me when I did aic7xxx patches.
You may yse --print-gc-sections to see the list of discarded sections.
Anyway, this is gccism/binutilizm. That about other possible/future options?
Give me example, please, why function must be non static if not used. If usage requires kconfig tuning, then this is a better way to go, than to adopt yet another GNU/Luxury.
____
Quoted 15 lines · Oleg Verych
Quoted 14 lines · Denys Vlasenko
Quoted 6 lines · Oleg Verych
* Wed, 5 Sep 2007 14:43:21 +0100
* User-Agent: KMail/1.9.1Quoted 2 lines
Build system: section garbage collection for vmlinux
Maybe this is just a test suit to get finish with
make XYZ static?They are vaguely connected in a sense that unused function which is not marked static doesn't generate gcc warning, but will be discarded by --gc-sections. "make XYZ static" also tends to find them - you make function static, you recompile the file, and gcc informs you that the function is not used at all!
Quoted 6 lines
Quoted 3 lines
This happened to me when I did aic7xxx patches.
You may yse --print-gc-sections to see the list of discarded sections.
Anyway, this is gccism/binutilizm. That about other possible/future options?
The kernel requires GNU gcc and GNU binutils, and if you want to use other tools for building the kernel they have to be sufficiently compatible.
Quoted 1 line
Give me example, please, why function must be non static if not used.
s/not used/not used in this kernel configuration/
Quoted 2 lines
If usage requires kconfig tuning, then this is a better way to go, than to adopt yet another GNU/Luxury.
The alternative would be to use an unmaintainable amount of #ifdef's.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedQuoted 10 lines · Oleg Verych
Quoted 10 lines
Quoted 3 lines
Quoted 1 line
Build system: section garbage collection for vmlinux
Maybe this is just a test suit to get finish with
make XYZ static?They are vaguely connected in a sense that unused function which is not marked static doesn't generate gcc warning, but will be discarded by --gc-sections. "make XYZ static" also tends to find them - you make function static, you recompile the file, and gcc informs you that the function is not used at all!
Quoted 8 lines
Quoted 3 lines
This happened to me when I did aic7xxx patches.
You may yse --print-gc-sections to see the list of discarded sections.
Anyway, this is gccism/binutilizm. That about other possible/future options?
Give me example, please, why function must be non static if not used.
Where do you see I'm saying that they must be non-static? I'm all for marking functions static. I just did it for aic7xxx.
Quoted 1 line
If usage requires kconfig tuning, then this is a better way to go,
We already do it, but we don't have enough developers to audit every driver for every possible combination of config options. As a result, there always be some amount of unused functions and data. Using --gc-sections will discard that.
Quoted 1 line
than to adopt yet another GNU/Luxury.
Actually, this is how linkers should have worked long ago. Borland's Turbo Pascal was doing it ten+ years ago.
I don't understand why you are opposed to toolchain helping humans to get optimized result. But it's fine with me. I won't force anyone to select CONFIG_DISCARD_UNUSED_SECTIONS. --
vda
On Thu, Sep 06, 2007 at 11:55:46AM +0100, Denys Vlasenko wrote:
[]
Quoted 11 lines
Quoted 1 line
Give me example, please, why function must be non static if not used.
Where do you see I'm saying that they must be non-static? I'm all for marking functions static. I just did it for aic7xxx.
Quoted 1 line
If usage requires kconfig tuning, then this is a better way to go,
We already do it, but we don't have enough developers to audit every driver for every possible combination of config options. As a result, there always be some amount of unused functions and data. Using --gc-sections will discard that.
You've did a tool. Documenting this tool to have it available for testers/janitors/maintainers is a better way, than to have all that opinions/problems with merging-to-mainline.
Quoted 8 lines
Quoted 1 line
than to adopt yet another GNU/Luxury.
Actually, this is how linkers should have worked long ago. Borland's Turbo Pascal was doing it ten+ years ago.
I don't understand why you are opposed to toolchain helping humans to get optimized result. But it's fine with me. I won't force anyone to select
CONFIG_DISCARD_UNUSED_SECTIONS.
That's why. It's treating symptoms, isn't it?
____
Quoted 9 lines · Oleg Verych
On Thu, Sep 06, 2007 at 11:55:46AM +0100, Denys Vlasenko wrote:
[]Quoted 7 lines
Quoted 1 line
Give me example, please, why function must be non static if not used.
Where do you see I'm saying that they must be non-static? I'm all for marking functions static. I just did it for aic7xxx.
Quoted 1 line
If usage requires kconfig tuning, then this is a better way to go,
Quoted 8 lines
Quoted 4 lines
We already do it, but we don't have enough developers to audit every driver for every possible combination of config options. As a result, there always be some amount of unused functions and data. Using --gc-sections will discard that.
You've did a tool. Documenting this tool to have it available for testers/janitors/maintainers is a better way, than to have all that opinions/problems with merging-to-mainline.
There is no problem with his patch.
His patch improves the build process.
And there's no reason to hide this from the users.
Quoted 10 lines
Quoted 8 lines
Quoted 1 line
than to adopt yet another GNU/Luxury.
Actually, this is how linkers should have worked long ago. Borland's Turbo Pascal was doing it ten+ years ago.
I don't understand why you are opposed to toolchain helping humans to get optimized result. But it's fine with me. I won't force anyone to select
CONFIG_DISCARD_UNUSED_SECTIONS.That's why. It's treating symptoms, isn't it?
There's nothing that requires treatment.
It's a matter of fact that the kernel takes advantages from some features of GNU binutils and GNU gcc that might not be available in other versions of these tools.
Whether you like it or not - that's not going to change.
But don't continue arguing about something where you won't win with words - it's open source, so you can always create a fork of the Linux kernel that builds with whatever toolchain you want.
The only way you could convince other people from your point of view would be if your forked version of the kernel would contain advantages that convince many users to use your kernel rather than Linus' one.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedOn Thu, Sep 06, 2007 at 02:21:43PM +0200, Adrian Bunk wrote:
[]
Quoted 7 lines
Quoted 3 lines
You've did a tool. Documenting this tool to have it available for testers/janitors/maintainers is a better way, than to have all that opinions/problems with merging-to-mainline.
There is no problem with his patch.
His patch improves the build process.
I would like to know timing, btw. Size, especially shown 1%, doesn't matter if link time increased dramatically. `Allyes' config, when i had fast and rammish machine was terrible thing (last winter). If 32 cores/cpus is will of author, then i'm even more suspicious.
Quoted 1 line
And there's no reason to hide this from the users.
Patch? Did i said patch?
Ah, patch. Yes -- hide it, because it against LKML's rules. I can provide ftp for such things, easily.
I said tool _and_ documentation. Because if developers don't know about `static' code or _data_ and cann't find out that, then small description is more than welcome, i think.
But, tool. Hide it also, becasue it's kind of thing to be shamed of (:
== untested, for demonstation only ==
SED_REM='
/\.text\./s|\.text\.|.text_|g; /\.data\./s|\.data\.|.data_|g; /\.bss\.p/s|\.bss\.p|.bss_p|g; # for .bss.page_aligned only
'
for place in linux/arch/* linux/kernel linux/include/asm-* do case $place
*cris) ADDON='/\.text\.__/n;' ;;
*powerpc) ADDON='/\.data\.rel/n;' ;;
*parisc) ADDON='/\.data\.vm[p0]/n;' ;;
*frv) ADDON='/\.bss\.stack/n;';;
esac
sed -i -e "$ADDON$SED_REM" find $place -type f
done
done
== ==
[]
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
I don't understand why you are opposed to toolchain helping humans to get optimized result. But it's fine with me. I won't force anyone to select
CONFIG_DISCARD_UNUSED_SECTIONS.That's why. It's treating symptoms, isn't it?
There's nothing that requires treatment.
[Help for] The developers/contributors of those drivers, no?
Quoted 5 lines
It's a matter of fact that the kernel takes advantages from some features of GNU binutils and GNU gcc that might not be available in other versions of these tools.
Whether you like it or not - that's not going to change.
I don't like fast, one-sided solutions.
Quoted 3 lines
But don't continue arguing about something where you won't win with words - it's open source, so you can always create a fork of the Linux kernel that builds with whatever toolchain you want.
I just want to have review process to be real not only in C hacking.
Quoted 8 lines
The only way you could convince other people from your point of view would be if your forked version of the kernel would contain advantages that convince many users to use your kernel rather than Linus' one.
cu
Adrian
____
Quoted 6 lines · Oleg Verych
On Thu, Sep 06, 2007 at 02:21:43PM +0200, Adrian Bunk wrote:
[]Quoted 4 lines
Quoted 3 lines
You've did a tool. Documenting this tool to have it available for testers/janitors/maintainers is a better way, than to have all that opinions/problems with merging-to-mainline.
Quoted 8 lines
Quoted 3 lines
There is no problem with his patch.
His patch improves the build process.
I would like to know timing, btw. Size, especially shown 1%, doesn't matter if link time increased dramatically. `Allyes' config, when i had fast and rammish machine was terrible thing (last winter). If 32 cores/cpus is will of author, then i'm even more suspicious.
For non-developers size and speed of the kernel matter much more than compile time.
When you go towards embedded systems with limited resources a 1% size decrease would often be worth it even if it would (hypothetically) increase the compile time by a factor of 10.
Quoted 11 lines
...
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
I don't understand why you are opposed to toolchain helping humans to get optimized result. But it's fine with me. I won't force anyone to select
CONFIG_DISCARD_UNUSED_SECTIONS.That's why. It's treating symptoms, isn't it?
There's nothing that requires treatment.
[Help for] The developers/contributors of those drivers, no?
...
They did everything right.
You should better try to understand the problem first before behaving as if you knew everything better than everyone else...
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed* Thu, 6 Sep 2007 22:39:31 +0200
[]
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
His patch improves the build process.
I would like to know timing, btw. Size, especially shown 1%, doesn't matter if link time increased dramatically. `Allyes' config, when i had fast and rammish machine was terrible thing (last winter). If 32 cores/cpus is will of author, then i'm even more suspicious.
For non-developers size and speed of the kernel matter much more than compile time.
I'm talking about benefits for the process (developers, testers) and the result (end users, dogs eating own food :).
Quoted 3 lines
When you go towards embedded systems with limited resources a 1% size decrease would often be worth it even if it would (hypothetically) increase the compile time by a factor of 10.
text data bss dec hex filename 5159478 1005139 406784 6571401 644589 linux-2.6.23-rc4.org/vmlinux 5131822 996090 401439 6529351 63a147 linux-2.6.23-rc4.gc/vmlinux
Are this numbers show embedded target? I think no. Also time factor of *10* can be spent more productively reviewing actual code of parts, that are going to be embedded, no?
[]
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
There's nothing that requires treatment.
[Help for] The developers/contributors of those drivers, no?
...They did everything right.
You should better try to understand the problem first before behaving as if you knew everything better than everyone else...
OK, thank you very much. Now, describe what problem you are talking about, please. I see non.
____
Quoted 10 lines · Oleg Verych
* Thu, 6 Sep 2007 22:39:31 +0200
[]
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
His patch improves the build process.
I would like to know timing, btw. Size, especially shown 1%, doesn't matter if link time increased dramatically. `Allyes' config, when i had fast and rammish machine was terrible thing (last winter). If 32 cores/cpus is will of author, then i'm even more suspicious.
Quoted 5 lines
Quoted 2 lines
For non-developers size and speed of the kernel matter much more than compile time.
I'm talking about benefits for the process (developers, testers) and the result (end users, dogs eating own food :).
Your claim was that link time was more important than code size, and that claim is in many cases wrong.
Quoted 11 lines
Quoted 3 lines
When you go towards embedded systems with limited resources a 1% size decrease would often be worth it even if it would (hypothetically) increase the compile time by a factor of 10.
text data bss dec hex filename 5159478 1005139 406784 6571401 644589 linux-2.6.23-rc4.org/vmlinux 5131822 996090 401439 6529351 63a147 linux-2.6.23-rc4.gc/vmlinuxAre this numbers show embedded target? I think no. Also time factor of *10* can be spent more productively reviewing actual code of parts, that are going to be embedded, no?
First of all, please lookup the word "hypothetically" in a dictionary.
And code review and Denys' patch have cumulative effects since his patch results in improvements that can't be resonably done other than at the ld and/or gcc level.
Quoted 6 lines
[]
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
There's nothing that requires treatment.
[Help for] The developers/contributors of those drivers, no?
...
Quoted 7 lines
Quoted 4 lines
They did everything right.
You should better try to understand the problem first before behaving as if you knew everything better than everyone else...
OK, thank you very much. Now, describe what problem you are talking about, please. I see non.
If you don't understand what the patches in this thread are about then you shouldn't have started commenting on this thread...
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed* Thu, 6 Sep 2007 23:19:55 +0200
Quoted 9 lines · Oleg Verych
Quoted 7 lines
* Thu, 6 Sep 2007 22:39:31 +0200
[]
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
His patch improves the build process.
I would like to know timing, btw. Size, especially shown 1%, doesn't matter if link time increased dramatically. `Allyes' config, when i
*if*
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
had fast and rammish machine was terrible thing (last winter). If 32 cores/cpus is will of author, then i'm even more suspicious.
For non-developers size and speed of the kernel matter much more than compile time.
I'm talking about benefits for the process (developers, testers) and the result (end users, dogs eating own food :).
Your claim was that link time was more important than code size, and that claim is in many cases wrong.
I just noted, that maybe (*if*) build/link time have been affected. There was an example of size reduction, why not to have timings also?
I guess, developer can spend time tuning written driver with that option/patch. But what you will write in the help message for testers/users? In this case build time is important obviously. Runtime isn't affected at all, except, maybe, ~1% increase in bzImage unzipping.
Whatever.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
When you go towards embedded systems with limited resources a 1% size decrease would often be worth it even if it would (hypothetically) increase the compile time by a factor of 10.
text data bss dec hex filename 5159478 1005139 406784 6571401 644589 linux-2.6.23-rc4.org/vmlinux 5131822 996090 401439 6529351 63a147 linux-2.6.23-rc4.gc/vmlinux
Quoted 5 lines
Quoted 3 lines
Are this numbers show embedded target? I think no. Also time factor of *10* can be spent more productively reviewing actual code of parts, that are going to be embedded, no?
First of all, please lookup the word "hypothetically" in a dictionary.
Do you mean hand-waving?
Whatever.
Quoted 3 lines
And code review and Denys' patch have cumulative effects since his patch results in improvements that can't be resonably done other than at the ld and/or gcc level.
I was talking about introducing such things in development process. Current kconfig may be not flexible, it must not lead to further problems and silver-bullet solutions.
Quoted 8 lines
Quoted 8 lines
[]
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
There's nothing that requires treatment.
[Help for] The developers/contributors of those drivers, no?
...They did everything right.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
You should better try to understand the problem first before behaving as if you knew everything better than everyone else...
OK, thank you very much. Now, describe what problem you are talking about, please. I see non.
If you don't understand what the patches in this thread are about then you shouldn't have started commenting on this thread...
Not first time i see, what i should do. Thank you very much, Adrian! You know better, what i know. Great.
Then say from the beginning that you're not interested in reviewing and view-exchanging process, you know better, what i should do. Thus, i will not waste my time explaining anything.
Whatever.
____
Quoted 9 lines · Oleg Verych
...
Quoted 3 lines
And code review and Denys' patch have cumulative effects since his patch results in improvements that can't be resonably done other than at the ld and/or gcc level.
I was talking about introducing such things in development process. Current kconfig may be not flexible, it must not lead to further problems and silver-bullet solutions.
...
The patches in this thread are not in any way related to any problems with kconfig...
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedQuoted 12 lines · Oleg Verych
Quoted 4 lines · Denys Vlasenko
We already do it, but we don't have enough developers to audit every driver for every possible combination of config options. As a result, there always be some amount of unused functions and data. Using --gc-sections will discard that.
You've did a tool. Documenting this tool to have it available for testers/janitors/maintainers is a better way, than to have all that opinions/problems with merging-to-mainline.
Quoted 2 lines
Quoted 1 line
than to adopt yet another GNU/Luxury.
Quoted 8 lines
Quoted 6 lines
Actually, this is how linkers should have worked long ago. Borland's Turbo Pascal was doing it ten+ years ago.
I don't understand why you are opposed to toolchain helping humans to get optimized result. But it's fine with me. I won't force anyone to select
CONFIG_DISCARD_UNUSED_SECTIONS.That's why. It's treating symptoms, isn't it?
If I understand you correctly, you seem to think that this work of identifying every piece of code which can end up unused under specific combination of CONFIGs, and #ifdef'ing it out, should be done by people, not machines.
I disagree.
Allyesconfig kernel has ~1700 unused functions/data objects, and it is only *one* of possible .configs.
There is more than 2900 CONFIG options in kernel, giving you about 3^2000 possible permutations.
If you find it interesting to work on making them
to not have unused functions, more power to you,
and good luck convincing people to accept tons
of additional #ifdefs.
--
vda
Quoted 10 lines · Denys Vlasenko
Build system: section garbage collection for vmlinux
Newer gcc and binutils can do dead code and data removal at link time. It is achieved using combination of -ffunction-sections -fdata-sections options for gcc and --gc-sections for ld.
Theory of operation:
Quoted 7 lines
Option -ffunction-sections instructs gcc to place each function (including static ones) in it's own section named .text.function_name instead of placing all functions in one big .text section.
At link time, ld normally coalesce all such sections into one output section .text again. It is achieved by having *(.text.*) spec along with *(.text) spec in built-in linker scripts.
You version doesn't work with CONFIG_MODULES right?
Also, why do you need this patch,
[PATCH 1/3] build system: section garbage collection for vmlinux
Daniel
Quoted 15 lines · Daniel Walker
Quoted 14 lines · Denys Vlasenko
Build system: section garbage collection for vmlinux
Newer gcc and binutils can do dead code and data removal at link time. It is achieved using combination of -ffunction-sections -fdata-sections options for gcc and --gc-sections for ld.
Theory of operation:
Option -ffunction-sections instructs gcc to place each function (including static ones) in it's own section named .text.function_name instead of placing all functions in one big .text section.
Quoted 5 lines
Quoted 3 lines
At link time, ld normally coalesce all such sections into one output section .text again. It is achieved by having *(.text.*) spec along with *(.text) spec in built-in linker scripts.
You version doesn't work with
CONFIG_MODULESright?
It works with CONFIG_MODULES.
Quoted 3 lines
Also, why do you need this patch,
[PATCH 1/3] build system: section garbage collection for vmlinux
Because otherwise, for example, .data.percpu sections we already have get picked up by *(.data.*), and then *(.data.percpu) end up empty:
__per_cpu_start = .;
*(.data.percpu)
*(.data.percpu.shared_aligned)
__per_cpu_end = .;and all hell breaks loose.
We need to stop using sections named like
.text.xxxx
.data.xxxx
.rodata.xxxx
.bss.xxxx
--
vda
Quoted 16 lines · Denys Vlasenko
Quoted 15 lines · Daniel Walker
Quoted 14 lines · Denys Vlasenko
Build system: section garbage collection for vmlinux
Newer gcc and binutils can do dead code and data removal at link time. It is achieved using combination of -ffunction-sections -fdata-sections options for gcc and --gc-sections for ld.
Theory of operation:
Option -ffunction-sections instructs gcc to place each function (including static ones) in it's own section named .text.function_name instead of placing all functions in one big .text section.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
At link time, ld normally coalesce all such sections into one output section .text again. It is achieved by having *(.text.*) spec along with *(.text) spec in built-in linker scripts.
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.
Really? Take a look at this version,
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you .
Daniel
Quoted 11 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
This will discard EXPORT_SYMBOLs potentially used by out-of-tree modules.
I also saw ~10% size reductions, but then at run-time test modules failed to load, they didn't find needed symbols.
OTOH if I know that I am not going to be using such modules, then this can be done. Will require another CONFIG_xxx, though. --
vda
Quoted 7 lines · Denys Vlasenko
Quoted 6 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
Quoted 8 lines
Quoted 5 lines
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
This will discard
EXPORT_SYMBOLspotentially used by out-of-tree modules.
Right , so it doesn't work with modules..
Daniel
Quoted 10 lines · Daniel Walker
Quoted 9 lines · Denys Vlasenko
Quoted 8 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
This will discard
EXPORT_SYMBOLspotentially used by out-of-tree modules.Right, so it doesn't work with modules..
What does "it" stand for in this sentence?
My patch was tested to work in my limited testing, but it is very conservative.
I can't talk for Marcelo, but his patch probably worked too, it just didn't guarantee that you can install kernel, and then compile and load external module. Wel, it will compile, but maybe will fail to load.
It sounds like *in-tree modules* were loading
just fine for Marcelo.
--
vda
Quoted 1 line · Denys Vlasenko
What does "it" stand for in this sentence?
"it" is your patches, and I think we got to bottom of it .. "it" (i.e. your patches) don't actually work with modules, which is what you originally contended ..
Quoted 7 lines
My patch was tested to work in my limited testing, but it is very conservative.
I can't talk for Marcelo, but his patch probably worked too, it just didn't guarantee that you can install kernel, and then compile and load external module. Wel, it will compile, but maybe will fail to load.
At least you should modify your Kconfig changes so that you don't allow people to select your new option unless they have CONFIG_MODULES off..
Daniel
Quoted 7 lines · Daniel Walker
Quoted 1 line · Denys Vlasenko
What does "it" stand for in this sentence?
"it" is your patches, and I think we got to bottom of it .. "it" (i.e. your patches) don't actually work with modules, which is what you originally contended ..
Kernel builds, loads and runs with my patches, and modules load just fine.
Quoted 10 lines
Quoted 7 lines
My patch was tested to work in my limited testing, but it is very conservative.
I can't talk for Marcelo, but his patch probably worked too, it just didn't guarantee that you can install kernel, and then compile and load external module. Wel, it will compile, but maybe will fail to load.
At least you should modify your
Kconfigchanges so that you don't allow people to select your new option unless they haveCONFIG_MODULESoff..
You seem to have interesting definition of "desn't work". --
vda
Quoted 10 lines · Denys Vlasenko
Quoted 7 lines · Daniel Walker
Quoted 1 line · Denys Vlasenko
What does "it" stand for in this sentence?
"it" is your patches, and I think we got to bottom of it .. "it" (i.e. your patches) don't actually work with modules, which is what you originally contended ..
Kernel builds, loads and runs with my patches, and modules load just fine.
Ok, so I guess we're not clear on this point .. In your last email you said that module exports might get removed? Is that what you intended to say or not?
Daniel
Quoted 10 lines · Daniel Walker
Quoted 9 lines · Denys Vlasenko
Quoted 7 lines · Daniel Walker
Quoted 1 line · Denys Vlasenko
What does "it" stand for in this sentence?
"it" is your patches, and I think we got to bottom of it .. "it" (i.e. your patches) don't actually work with modules, which is what you originally contended ..
Quoted 5 lines
Quoted 1 line
Kernel builds, loads and runs with my patches, and modules load just fine.
Ok, so I guess we're not clear on this point .. In your last email you said that module exports might get removed? Is that what you intended to say or not?
I said this:
Quoted 13 lines · Denys Vlasenko
Quoted 11 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
Quoted 5 lines
This will discard
EXPORT_SYMBOLspotentially used by out-of-tree modules.I also saw ~10% size reductions, but then at run-time test modules failed to load, they didn't find needed symbols.
A bit extended version:
In the process in making it work I saw ~10% vmlinux size reductions (which basically matches what Marcelo says) when I wasn't retaining sections needed for EXPORT_SYMBOLs, but module loading didn't work.
Thus I fixed that by adding KEEP() directives so that EXPORT_SYMBOLs are never discarded. This was just one of many fixes until kernel started to actually boot and work.
I did that before I posted patches to lkml.
IOW: posted patches are not broken versus module loading. --
vda
Quoted 6 lines · Denys Vlasenko
A bit extended version:
In the process in making it work I saw ~10% vmlinux size reductions (which basically matches what Marcelo says) when I wasn't retaining sections needed for
EXPORT_SYMBOLs, but module loading didn't work.
Quoted 6 lines
Thus I fixed that by adding
KEEP()directives so thatEXPORT_SYMBOLsare never discarded. This was just one of many fixes until kernel started to actually boot and work.I did that before I posted patches to lkml.
IOW: posted patches are not broken versus module loading.
Ok, this is more like the explanation I was looking for..
During this thread you seemed to indicate the patches you release reduced the kernel ~10% , but now your saying that was pre-release , right?
Daniel
Hi Daniel.
Quoted 8 lines
Quoted 2 lines
I did that before I posted patches to lkml.
IOW: posted patches are not broken versus module loading.Ok, this is more like the explanation I was looking for..
During this thread you seemed to indicate the patches you release reduced the kernel ~10% , but now your saying that was pre-release , right?
What are you after here?
If you read the inital post you see the actual savings and you also can read that it works wiht modules. The precentage can be calculated from these numbers if you are interested.
As explained later this patch does NOT remove exported symbols since they may be used by modules that are not included in the actual build.
And the patch works for x86_64.
So I'm a bit puzzeled what you are trying to bring forward here. And please read carefully the initial posting again...
Sam
Quoted 11 lines · Sam Ravnborg
Hi Daniel.
Quoted 8 lines
Quoted 2 lines
I did that before I posted patches to lkml.
IOW: posted patches are not broken versus module loading.Ok, this is more like the explanation I was looking for..
During this thread you seemed to indicate the patches you release reduced the kernel ~10% , but now your saying that was pre-release , right?
Quoted 5 lines
What are you after here?
If you read the inital post you see the actual savings and you also can read that it works wiht modules. The precentage can be calculated from these numbers if you are interested.
Right, but he contradicted that during the course of this thread.. Which is why I'm asking about it..
Daniel
Quoted 11 lines · Daniel Walker
Quoted 10 lines · Denys Vlasenko
A bit extended version:
In the process in making it work I saw ~10% vmlinux size reductions (which basically matches what Marcelo says) when I wasn't retaining sections needed for
EXPORT_SYMBOLs, but module loading didn't work.Thus I fixed that by adding
KEEP()directives so thatEXPORT_SYMBOLsare never discarded. This was just one of many fixes until kernel started to actually boot and work.
Quoted 8 lines
Quoted 2 lines
I did that before I posted patches to lkml.
IOW: posted patches are not broken versus module loading.Ok, this is more like the explanation I was looking for..
During this thread you seemed to indicate the patches you release reduced the kernel ~10% , but now your saying that was pre-release , right?
CONFIG_MODULE=n will save ~10%CONFIG_MODULE=y - ~1%
Exact figure depends on .config (whether you happen to include especially "fat" code or not).
I want to explain a bit where I am coming from. I am working on busybox, and last release made busybox smaller by "whopping" 2%. This is the result of a hundred or so of small code and data shrinks.
It basically means that I am close to the point of diminishing returns trying to make busybox smaller, and memory wastage on the running embedded system is now elsewhere - including kernel.
--
vda
Quoted 27 lines · Denys Vlasenko
Quoted 11 lines · Daniel Walker
Quoted 10 lines · Denys Vlasenko
A bit extended version:
In the process in making it work I saw ~10% vmlinux size reductions (which basically matches what Marcelo says) when I wasn't retaining sections needed for
EXPORT_SYMBOLs, but module loading didn't work.Thus I fixed that by adding
KEEP()directives so thatEXPORT_SYMBOLsare never discarded. This was just one of many fixes until kernel started to actually boot and work.Quoted 8 lines
Quoted 2 lines
I did that before I posted patches to lkml.
IOW: posted patches are not broken versus module loading.Ok, this is more like the explanation I was looking for..
During this thread you seemed to indicate the patches you release reduced the kernel ~10% , but now your saying that was pre-release , right?
CONFIG_MODULE=nwill save ~10%CONFIG_MODULE=y- ~1%Exact figure depends on .config (whether you happen to include especially "fat" code or not).
Quoted 7 lines
I want to explain a bit where I am coming from. I am working on busybox, and last release made busybox smaller by "whopping" 2%. This is the result of a hundred or so of small code and data shrinks.
It basically means that I am close to the point of diminishing returns trying to make busybox smaller, and memory wastage on the running embedded system is now elsewhere - including kernel.
I think this type of pruning is a good thing, you could even say the biggest bit of low hanging fruit in terms of size reduction.
I think your patches are good, but need some work. There are still some changes that could reduce the kernel further (i.e. when modules are used) .. So I'm not trying to discourage you, but you set off some alarms with me early in the thread.. Which caused this to drag out..
Daniel
Quoted 13 lines · Denys Vlasenko
Quoted 11 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
Quoted 8 lines
This will discard
EXPORT_SYMBOLspotentially used by out-of-tree modules.I also saw ~10% size reductions, but then at run-time test modules failed to load, they didn't find needed symbols.
OTOH if I know that I am not going to be using such modules, then this can be done. Will require another
CONFIG_xxx, though.
One point to keep in mind is that the space penalty of CONFIG_MODULES=y is so big that CONFIG_MODULES=n is actually the most interesting case for small systems that really need small kernels.
Quoted 1 line
vda
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedQuoted 20 lines · Adrian Bunk
Quoted 19 lines · Denys Vlasenko
Quoted 11 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
This will discard
EXPORT_SYMBOLspotentially used by out-of-tree modules.I also saw ~10% size reductions, but then at run-time test modules failed to load, they didn't find needed symbols.
Quoted 6 lines
Quoted 2 lines
OTOH if I know that I am not going to be using such modules, then this can be done. Will require another
CONFIG_xxx, though.One point to keep in mind is that the space penalty of
CONFIG_MODULES=yis so big thatCONFIG_MODULES=nis actually the most interesting case for small systems that really need small kernels.
Marcelo's version actual deals with the CONFIG_MODULES=y penalty , which is interesting to me .. It removes symbols added for CONFIG_MODULES which actually aren't used .. So CONFIG_MODULES=y is just as interesting as without (to me at least..).
Daniel
Quoted 24 lines · Daniel Walker
Quoted 23 lines · Adrian Bunk
Quoted 21 lines · Denys Vlasenko
Quoted 11 lines · Daniel Walker
Quoted 3 lines
Quoted 1 line
You version doesn't work with
CONFIG_MODULESright?It works with
CONFIG_MODULES.Really? Take a look at this version,
http://lkml.org/lkml/2006/6/4/169
Marcello had to implement a two pass build to add back symbol used in modules which got removed from the main kernel.. You don't appear to do that. Marcelo also claims better size reduction than you.
This will discard
EXPORT_SYMBOLspotentially used by out-of-tree modules.I also saw ~10% size reductions, but then at run-time test modules failed to load, they didn't find needed symbols.
OTOH if I know that I am not going to be using such modules, then this can be done. Will require another
CONFIG_xxx, though.
Quoted 8 lines
Quoted 3 lines
One point to keep in mind is that the space penalty of
CONFIG_MODULES=yis so big thatCONFIG_MODULES=nis actually the most interesting case for small systems that really need small kernels.Marcelo's version actual deals with the
CONFIG_MODULES=ypenalty , which is interesting to me .. It removes symbols added forCONFIG_MODULESwhich actually aren't used .. SoCONFIG_MODULES=yis just as interesting as without (to me at least..).
There's still stuff like kernel/module.c or the additional space each used EXPORT_SYMBOL takes that make CONFIG_MODULES=n kernels smaller.
But it depends on the use case:
If you are aiming for the smallest possible runtime memory usage CONFIG_MODULES=n is the best choice, while for some applications where the bzimage (or similar) size is for some reason limited but the size of the modules doesn't matter the approach you mention might be the best.
Quoted 1 line
Daniel
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedQuoted 7 lines · Denys Vlasenko
Build system: section garbage collection for vmlinux
Newer gcc and binutils can do dead code and data removal at link time. It is achieved using combination of -ffunction-sections -fdata-sections options for gcc and --gc-sections for ld.
...
In the long term I'd like us to be able to compile the whole (or at least most of) the kernel with one "-combine -fwhole-program" gcc call which should bring the same positive effect plus enables gcc to do additional optimizations.
That's meant as a remark, not against your patch (which is for a lower hanging fruit).
Quoted 1 line
vda
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed