new kallsyms approach
Subsystems (8)
PATCH 1/1 new kallsyms approach
(Note: Patch only attached due to size restrictions.)
This patch provides al alternative to the pre-exisiting kallsyms code. That code, from a kernel debugger perspective at least, suffers from incomplete information, making it impossible to
(a) disambiguate multiple static functions of the same name (in different
source files),
(b) determine a complete set of attributes for a symbol (namely, the symbol's size, but also its type, which gets converted to an nm-like one-
character representation), and
(c) retain full section information
This new approach basically makes handling core kernel and module symbols
the same, by retrieving the kernel's section, symbol, and string tables
rather than parsing the system map.
At once it adds the functionality to strip unneeded symbols from modules,
which results in non-neglectable space savings for typical distributions
(which large amounts of modules).
(Note: Patch only attached due to size restrictions.)
This patch provides al alternative to the pre-exisiting kallsyms code. That code, from a kernel debugger perspective at least, suffers from incomplete information, making it impossible to
(a) disambiguate multiple static functions of the same name (in different source files),
(b) determine a complete set of attributes for a symbol (namely, the symbol's size, but also its type, which gets converted to an nm-like one- character representation), and
(c) retain full section information
This new approach basically makes handling core kernel and module symbols the same, by retrieving the kernel's section, symbol, and string tables rather than parsing the system map.
At once it adds the functionality to strip unneeded symbols from modules, which results in non-neglectable space savings for typical distributions (which large amounts of modules).
arch/alpha/defconfig +1 -0
@@ -32,6 +32,7 @@ CONFIG_LOG_BUF_SHIFT=14 # CONFIG_IKCONFIG is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y CONFIG_KALLSYMS_ALL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_FUTEX=y
arch/frv/defconfig +1 -0
@@ -33,6 +33,7 @@ CONFIG_SYSCTL=y # CONFIG_IKCONFIG is not set CONFIG_EMBEDDED=y CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_ALL is not set # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_PRINTK=y
arch/i386/defconfig +2 -0
@@ -28,6 +28,8 @@ CONFIG_HOTPLUG=y # CONFIG_IKCONFIG is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y +# CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_FUTEX=y CONFIG_EPOLL=y CONFIG_IOSCHED_NOOP=y
arch/i386/kernel/entry.S +43 -6
236 lines not on this page · raw message
arch/ia64/defconfig +1 -0
@@ -29,6 +29,7 @@ CONFIG_IKCONFIG_PROC=y # CONFIG_CPUSETS is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y CONFIG_KALLSYMS_ALL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_PRINTK=y
arch/ia64/Makefile +0 -1
@@ -9,7 +9,6 @@ # NM := $(CROSS_COMPILE)nm -B -READELF := $(CROSS_COMPILE)readelf export AWK
arch/m68k/defconfig +1 -0
@@ -32,6 +32,7 @@ CONFIG_KOBJECT_UEVENT=y # CONFIG_IKCONFIG is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_PRINTK=y CONFIG_BUG=y
arch/mips/defconfig +1 -0
@@ -32,6 +32,7 @@ CONFIG_IKCONFIG=y CONFIG_IKCONFIG_PROC=y CONFIG_EMBEDDED=y CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_FUTEX=y CONFIG_EPOLL=y
arch/ppc64/defconfig +1 -0
@@ -39,6 +39,7 @@ CONFIG_IKCONFIG_PROC=y CONFIG_CPUSETS=y # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_ALL is not set # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_PRINTK=y
arch/s390/defconfig +1 -0
@@ -35,6 +35,7 @@ CONFIG_IKCONFIG_PROC=y # CONFIG_CPUSETS is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_ALL is not set # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_PRINTK=y
arch/sparc/defconfig +2 -0
@@ -28,7 +28,9 @@ CONFIG_LOG_BUF_SHIFT=14 # CONFIG_IKCONFIG is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_ALL is not set +# CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_FUTEX=y CONFIG_EPOLL=y CONFIG_IOSCHED_NOOP=y
arch/sparc64/defconfig +1 -0
@@ -30,6 +30,7 @@ CONFIG_KOBJECT_UEVENT=y # CONFIG_IKCONFIG is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_ALL is not set # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_FUTEX=y
arch/um/defconfig +1 -0
@@ -70,6 +70,7 @@ CONFIG_IKCONFIG=y CONFIG_IKCONFIG_PROC=y # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y # CONFIG_KALLSYMS_ALL is not set CONFIG_KALLSYMS_EXTRA_PASS=y CONFIG_PRINTK=y
arch/x86_64/defconfig +1 -0
@@ -39,6 +39,7 @@ CONFIG_IKCONFIG_PROC=y # CONFIG_CPUSETS is not set # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y +CONFIG_KALLSYMS_TRADITIONAL=y CONFIG_KALLSYMS_ALL=y # CONFIG_KALLSYMS_EXTRA_PASS is not set CONFIG_PRINTK=y
arch/x86_64/kernel/entry.S +43 -5
235 lines not on this page · raw message
include/linux/kallsyms.h +27 -0
@@ -19,6 +19,25 @@ const char *kallsyms_lookup(unsigned lonunsigned long *offset, char **modname, char *namebuf); +#ifdef CONFIG_MODULES + +#if defined(Elf_Shdr) && defined(Elf_Sym) +char kallsyms_elf_type (const Elf_Sym *sym, +const Elf_Shdr *sechdrs, +const char *secstrings); +#endif + +struct module ; +void module_init_done (struct module *); + +unsigned long kallsyms_find_symname (struct module *,const char *); + +const char *kallsyms_get_ksymbol (struct module *, +unsigned long addr, +unsigned long *size, +unsigned long *offset); +#endif +/* Replace "%s" in format with address, if found */ extern void __print_symbol (const char *fmt,unsigned long address);
@@ -64,4 +83,12 @@ do { \__print_symbol (fmt, addr); \ } while(0) +#if defined(CONFIG_KALLSYMS ) && !defined(CONFIG_KALLSYMS_TRADITIONAL ) +void kallsyms_early_init (void ); +void kallsyms_init_done (void ); +#else +#define kallsyms_early_init () ((void)0) +#define kallsyms_init_done () ((void)0) +#endif +#endif /*_LINUX_KALLSYMS_H*/
include/linux/linkage.h +11 -0
@@ -33,6 +33,17 @@ALIGN ; \ name: +#ifdef CONFIG_KALLSYMS +#define ENDPROC (name) \ + .type name, @function; \ + END(name) +#define END (name) \ + .size name, .-name +#else +#define ENDPROC (name) +#define END (name) +#endif +#endif #define NORET_TYPE /**/
include/linux/module.h +8 -1
@@ -301,7 +301,14 @@ struct module/* We keep the symbol and string tables for kallsyms. */ Elf_Sym *symtab;unsigned long num_symtab; -char *strtab; +const char *strtab; + +# ifndef CONFIG_KALLSYMS_TRADITIONAL +/* And also the sections and their string table. */ +Elf_Shdr *sections; +unsigned long num_sections; +const char *shstrtab; +# endif /* Section attributes */ struct module_sect_attrs *sect_attrs;
init/Kconfig +29 -12
52 lines not on this page · raw message
init/main.c +2 -0
@@ -468,6 +468,7 @@ asmlinkage void __init start_kernel(void &unknown_bootoption);sort_main_extable ();trap_init (); +kallsyms_early_init ();rcu_init ();init_IRQ ();pidhash_init ();
@@ -690,6 +691,7 @@ static int init(void * unused)* we're essentially up and running. Get rid of the * initmem segments and start the user-mode stuff.. */ +kallsyms_init_done ();free_initmem ();unlock_kernel (); system_state =SYSTEM_RUNNING ;
kernel/kallsyms.c +306 -34
473 lines not on this page · raw message
kernel/module.c +138 -119
377 lines not on this page · raw message
Makefile +105 -15
157 lines not on this page · raw message
scripts/genksyms/genksyms.c +17 -5
@@ -43,7 +43,7 @@ FILE *debugfile;int cur_line =1 ;char *cur_filename, *output_directory; -int flag_debug, flag_dump_defs, flag_warnings; +int flag_debug, flag_dump_defs, flag_warnings, flag_inline_asm;static int errors;static int nsyms;
@@ -457,8 +457,16 @@ export_symbol(const char *name)if (flag_dump_defs)fputs (">\n" , debugfile); -/* Used as a linker script. */ -printf ("__crc_%s = 0x%08lx ;\n" , name, crc); +if (!flag_inline_asm) + { +/* Used as a linker script. */ +printf ("__crc_%s = 0x%08lx ;\n" , name, crc); + } +else + { +/* Used as inline assembly. */ +printf ("__asm__(\".equiv __crc_%s, 0x%08lx\");\n" , name, crc); + } } }
@@ -529,6 +537,7 @@ main(int argc, char **argv)#ifdef __GNU_LIBRARY__struct option long_opts[] = { + {"asm" ,0 ,0 ,'a' }, {"debug" ,0 ,0 ,'d' }, {"warnings" ,0 ,0 ,'w' }, {"quiet", 0, 0, 'q'},
@@ -538,13 +547,16 @@ main(int argc, char **argv) {0, 0, 0, 0} }; -while ((o =getopt_long (argc, argv,"dwqVDk:p:" , +while ((o =getopt_long (argc, argv,"adwqVDk:p:" , &long_opts[0 ],NULL )) !=EOF ) #else /* __GNU_LIBRARY__ */ - while ((o = getopt(argc, argv, "dwqVDk:p:")) != EOF) + while ((o = getopt(argc, argv, "adwqVDk:p:")) != EOF)#endif /* __GNU_LIBRARY__ */ switch (o) { +case 'a' : + flag_inline_asm =1 ; +break ;case 'd' : flag_debug++;break ;
scripts/kallsyms.c +65 -27
144 lines not on this page · raw message
scripts/kallsyms.strip +18 -0
@@ -0,0 +1,18 @@ +<*> +__compound_literal[$.][0-9]* +__crc_[a-zA-Z_]* +__exitcall_[a-zA-Z_]* +__func__[$.][0-9]* +__FUNCTION__[$.][0-9]* +gcc[0-9]_compiled[$.] +__initcall_[a-zA-Z_]* +__kcrctab_[a-zA-Z_]* +__kstrtab_[a-zA-Z_]* +__ksymtab_[a-zA-Z_]* +__mod_[a-zA-Z_]*[0-9] +__module_depends +__param_[a-zA-Z_]* +__pci_fixup_[A-Z]* +__PRETTY_FUNCTION__[$.][0-9]* +__setup_[a-zA-Z_0-9]* +____versions
scripts/Makefile +1 -0
@@ -8,6 +8,7 @@ # conmakehash: Create arrays for initializing the kernel console tables hostprogs-$(CONFIG_KALLSYMS) += kallsyms +HOSTCFLAGS_kallsyms.o := -D ARCH_$(SUBARCH) hostprogs-$(CONFIG_LOGO) += pnmtologo hostprogs-$(CONFIG_VT) += conmakehash hostprogs-$(CONFIG_PROM_CONSOLE) += conmakehash
scripts/Makefile.build +20 -11
@@ -153,23 +153,26 @@ else # are done. # o otherwise, we calculate symbol versions using the good old # genksyms on the preprocessed source and postprocess them in a way -# that they are usable as a linker script -# o generate <file>.o from .tmp_<file>.o using the linker to -# replace the unresolved symbols __crc_exported_symbol with -# the actual value of the checksum generated by genksyms +# that they are usable as assembly source +# o recompile <file>.o from <file>.c forcing inclusion of assembler +# directives defining the actual values of __crc_*, followed by +# objcopy-ing them to force these symbols to be local to permit +# stripping them during the final module linking stage. cmd_cc_o_c = $(CC) $(c_flags) -c -o $(@D)/.tmp_$(@F) $< cmd_modversions = \ if $(OBJDUMP) -h $(@D)/.tmp_$(@F) | grep -q __ksymtab; then \ - $(CPP) -D__GENKSYMS__ $(c_flags) $< \ - | $(GENKSYMS) \ - > $(@D)/.tmp_$(@F:.o=.ver); \ - \ - $(LD) $(LDFLAGS) -r -o $@ $(@D)/.tmp_$(@F) \ - -T $(@D)/.tmp_$(@F:.o=.ver); \ - rm -f $(@D)/.tmp_$(@F) $(@D)/.tmp_$(@F:.o=.ver); \ + if $(CPP) -D__GENKSYMS__ $(c_flags) $< \ + | $(GENKSYMS) -a > $(@D)/.tmp_$(@F:.o=.ver) \ + && $(CC) $(c_flags) -c -w -include $(@D)/.tmp_$(@F:.o=.ver) -o $@ $< \ + && $(OBJCOPY) -L '__crc_*' -w $@; then \ + rm -f $(@D)/.tmp_$(@F); \ + else \ + rm -f $@; exit 1; \ + fi; \ else \ mv $(@D)/.tmp_$(@F) $@; \ + rm -f $(@D)/.tmp_$(@F:.o=.ver); \ fi; endif
@@ -180,6 +183,12 @@ define rule_cc_o_c $(cmd_cc_o_c); \ $(cmd_modversions) \ scripts/basic/fixdep $(depfile) $@ '$(subst ','\'',$(cmd_cc_o_c))' > $(@D)/.$(@F).tmp; \ + if [ -r $(@D)/.tmp_$(@F:.o=.ver) ]; then \ + echo >> $(@D)/.$(@F).tmp; \ + echo '$@: $(GENKSYMS)' >> $(@D)/.$(@F).tmp; \ + echo '$(GENKSYMS):: ;' >> $(@D)/.$(@F).tmp; \ + rm $(@D)/.tmp_$(@F:.o=.ver); \ + fi; \ rm -f $(depfile); \ mv -f $(@D)/.$(@F).tmp $(@D)/.$(@F).cmd endef
scripts/Makefile.lib +2 -0
@@ -7,6 +7,8 @@ comma := , empty := space := $(empty) $(empty) +open := ( +close := ) # Backward compatibility - to be removed... extra-y += $(EXTRA_TARGETS)
scripts/Makefile.modinst +2 -1
@@ -17,7 +17,8 @@ __modinst: $(modules) @: quiet_cmd_modules_install = INSTALL $@ - cmd_modules_install = mkdir -p $(2); cp $@ $(2) + cmd_modules_install = mkdir -p $(2); \ + cp $(firstword $(wildcard $@s) $@) $(2)/$(basename $(@F)).ko # Modules built outside the kernel source tree go into extra by default INSTALL_MOD_DIR ?= extra
scripts/Makefile.modpost +62 -6
102 lines not on this page · raw message
Quoted 11 lines · Jan Beulich
This patch provides al alternative to the pre-exisiting kallsyms code. That code, from a kernel debugger perspective at least, suffers from incomplete information, making it impossible to
(a) disambiguate multiple static functions of the same name (in different
source files),
(b) determine a complete set of attributes for a symbol (namely, the symbol's size, but also its type, which gets converted to an nm-like one-
character representation), and
(c) retain full section information
I don't think it's a good idea to have two different ways to do kallsyms. Either we should always use your new way in standard KALLSYMS or not do it at all.
The major decision factor is how much bloat it adds. Can you post before/after numbers of binary size?
If the difference is >5% or so - are there ways to recover the difference?
-Andi
Quoted 3 lines
I don't think it's a good idea to have two different ways to do kallsyms. Either we should always use your new way in standard KALLSYMS or not do it at all.
I agree, but I wanted to retain the old mechanism not the least because of the space constraints you mention.
Quoted 2 lines
The major decision factor is how much bloat it adds. Can you post before/after numbers of binary size?
i386 vmlinux KALLSYMS_ALL=n: old=5782934 new=6106675 (5.60% increase) i386 vmlinux KALLSYMS_ALL=y: old=6049118 new=6524411 (7.85% increase) x86-64 vmlinux KALLSYMS_ALL=n: old=8593125 new=8962128 (4.30% increase)
x86-64 vmlinux KALLSYMS_ALL=y: old=8797853 new=9117704 (3.64% increase)
I shall note additionally that not using KALLSYMS_ALL with the new approach increases the build time somewhat because the stripping of the data symbols in this scheme is rather inefficient (a binutils limitation as I would call it). KALLSYMS_ALL=y, however, is mostly suited for kernel debugging, while KALLSYMS_ALL=n seems the best choice if just caring about symbolic stack traces, which seems to be another argument for keeping both mechanisms.
Quoted 2 lines
If the difference is >5% or so - are there ways to recover the difference?
There are certainly ways, but I'm not sure how much they would cost. One thing would be to teach the linker to not only optimize the section string table, but also the (normal) symbol string one. Decreasing the symbol table size might be more difficult - only st_size and only on 64-bit archs seems to lend itself to reduction (as any symbol's size can clearly be expected to be less than 4G). But that would result in alignment problems, so I don't think that'd be a good idea either.
Jan