Implement DWARF modversions
Subsystems (5)
Sami Tolvanen
maintainerv8 cover Implement DWARF modversions
Hi,
Here's v8 of the DWARF modversions series. The main motivation is modversions support for Rust, which is important for distributions like Android that are about to ship Rust kernel modules. Per Luis' request [1], v2 dropped the Rust specific bits from the series and instead added the feature as an option for the entire kernel to make it easier to evaluate the benefits of this approach, and to get better test coverage. Matt is addressing Rust modversion_info compatibility issues in a separate patch set [2] that depends on this series, and actually allows modversions to be enabled with Rust.
Short background: Unlike C, Rust source code doesn't have sufficient information about the final ABI, as the compiler has considerable freedom in adjusting structure layout, for example, which makes using a source code parser like genksyms a non-starter. Based on earlier feedback, this series uses DWARF debugging information for computing versions. DWARF is an established and a relatively stable format, which includes all the necessary ABI details, and adding a CONFIG_DEBUG_INFO dependency for Rust symbol versioning seems like a reasonable trade-off as most distributions already enable it.
The first 15 patches add gendwarfksyms, a tool for computing symbol versions from DWARF. When passed a list of exported symbols and object files, the tool generates an expanded type string for each symbol and computes symbol versions. gendwarfksyms is written in C, uses libdw to process DWARF, and zlib for CRC32. Patch 16 ensures that debugging information is present where we need it, patch 17 adds gendwarfksyms as an alternative to genksyms, and the last patch adds documentation.
v8 is based on 6.13-rc5, and for your convenience, the series is also available here:
https://github.com/samitolvanen/linux/commits/gendwarfksyms-v8
If you also want to test the series with actual Rust modules, this branch adds v13 of Matt's modversion_info series:
https://github.com/samitolvanen/linux/commits/rustmodversions-v8
Sami
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/
---
v8:
- Rebased on v6.13-rc5.
- Changed opts[] to static const and reformatted the array. (Patch 1)
- Moved a stray comment to patch 1. (Patch 2)
v7: https://lore.kernel.org/lkml/[email protected]/
- Added a Kconfig description to actually allow build testing each patch. (Patch 1)
- Added a MAINTAINERS entry for gendwarfksyms. (Patch 1)
- Cleaned up includes and fixed a build issue with asprintf not always being present without _GNU_SOURCE. (Patches 1, 11, and 13)
- Adjusted hash table sizes for lower overhead as we're not right now concerned with processing vmlinux.o, and thus need to deal with a smaller number of symbols. (Patches 1, 4, 9, 11, and 13)
- Added error handling for gelf_getsymshndx failures. (Patch 2)
- Renamed the is_declaration function to is_kabi_definition for clarity and reversed the logic per Petr's suggestion. (Patch 9)
- Dropped the redundant default Kconfig tag. (Patch 17)
- Fixed a documentation typo. (Patch 18)
v6: https://lore.kernel.org/lkml/[email protected]/
- Dropped pointer expansion limits as this affects version
stability when exported symbols are removed. (Patch 9)
- Changed local type definitions (in .c files) that are opaque to external users to be treated as declarations even if a definition is available. (Patch 9)
- Switched to zlib's CRC32 implementation per Masahiro's suggestion. (Patch 12)
- Renamed struct_declonly kABI rule to simply declonly, as it applies also to unions and enums, added a new rule for overriding enumerator values, and refactored the examples. (Patch 13)
- Added --stable support for renamed structure members and also added examples. (Patch 14)
- Rebased on linux-kbuild/for-next for Masahiro's symtypes build rule clean-ups. (Patch 17)
- Updated the documentation reflect --stable changes. (Patch 18)
v5: https://lore.kernel.org/lkml/[email protected]/
- Rebased on v6.12-rc5.
- Fixed an issue with limiting structure expansion, and applied Petr's clean-up. (Patch 10)
- Dropped an unnecessary return statement in error path. (Patch
12)
- Addressed several other smaller issues Petr brought up. (Patches 13, 14, and 15)
- Added a KBUILD_GENDWARFKSYMS_STABLE flag to enable --stable for the entire kernel build. (Patch 18)
- Updated documentation to include KBUILD flags. (Patch 19)
- Picked up Reviewed-by tags from v4.
v4: https://lore.kernel.org/lkml/[email protected]/
- Rebased on v6.12-rc2, which now includes all the prerequisites.
- Dropped unnecessary name_only parameter for symbols.c::for_each and cleaned up error handling. (Patch 3)
- Fixed anonymous scope handling to ensure unnamed DIEs don't get names. (Patch 4)
- Added non-variant children to variant_type output, and included DW_AT_discr_value attributes for variants. (Patch 9)
- Added another symbol pointer test case. (Patch 16)
- Picked up (Acked|Reviewed)-by tags from v3.
v3: https://lore.kernel.org/lkml/[email protected]/
- Updated SPX license headers.
- Squashed the first two patches in v2 and tried to reduce churn as much as reasonable.
- Dropped patch 18 from v2 ("x86/asm-prototypes: Include <asm/ptrace.h>") as it's addressed by a separate patch.
- Changed the error handling code to immediately terminate instead of propagating the errors back to main, which cleaned up the code quite a bit.
- Switched to the list and hashtable implementations in scripts and dropped the remaining tools/include dependencies. Added a couple missing list macros. (patch 1)
- Moved the genksyms CRC32 implementation to scripts/include and dropped the duplicate code. (patches 2 and 14)
- Switched from ad-hoc command line parsing to getopt_long (patch 3).
- Added structure member and function parameter names to the DIE output to match genksyms behavior, and tweaked the symtypes format to be more parser-friendly in general based on Petr's suggestions.
- Replaced the declaration-only struct annotations with more generic kABI stability rules that allow source code annotations to be used where #ifndef __GENKSYMS__ was previously used. Added support for rules that can be used to exclude enumerators from versioning. (patch 16)
- Per Miroslav's suggestion, added an option to hide structure members from versioning when they're added to existing alignment holes, for example. (patch 16)
- Per Greg's request, added documentation and example macros for the --stable features, and a couple of test cases. (patches 15, 16, and
20)
- Fixed making symtypes files, which need to depend on .o files with gendwarfksyms. (patch 19)
- Addressed several other smaller issues that Petr and Masahiro kindly pointed out during the v2 review.
v2: https://lore.kernel.org/lkml/[email protected]/
- Per Luis' request, dropped Rust-specific patches and added gendwarfksyms as an alternative to genksyms for the entire
kernel.
- Added support for missing DWARF features needed to handle also non-Rust code.
- Changed symbol address matching to use the symbol table information instead of relying on addresses in DWARF.
- Added __gendwarfksyms_ptr patches to ensure the compiler emits the necessary type information in DWARF even for symbols that are defined in other TUs.
- Refactored debugging output and moved the more verbose output behind --dump* flags.
- Added a --symtypes flag for generating a genksyms-style symtypes output based on Petr's feedback, and refactored symbol version calculations to be based on symtypes instead of raw --dump-dies output.
- Based on feedback from Greg and Petr, added --stable flag and support for reserved data structure fields and declaration-onl structures. Also added examples for using these features.
- Added a GENDWARFKSYMS option and hooked up kbuild support for both C and assembly code. Note that with gendwarfksyms, we have to actually build a temporary .o file for calculating assembly modversions.
v1: https://lore.kernel.org/lkml/[email protected]/
---
Sami Tolvanen (18):
tools: Add gendwarfksyms
gendwarfksyms: Add address matching
gendwarfksyms: Expand base_type
gendwarfksyms: Add a cache for processed DIEs
gendwarfksyms: Expand type modifiers and typedefs
gendwarfksyms: Expand subroutine_type
gendwarfksyms: Expand array_type
gendwarfksyms: Expand structure types
gendwarfksyms: Limit structure expansion
gendwarfksyms: Add die_map debugging
gendwarfksyms: Add symtypes output
gendwarfksyms: Add symbol versioning
gendwarfksyms: Add support for kABI rules
gendwarfksyms: Add support for reserved and ignored fields
gendwarfksyms: Add support for symbol type pointers
export: Add __gendwarfksyms_ptr_ references to exported symbols
kbuild: Add gendwarfksyms as an alternative to genksyms Documentation/kbuild: Add DWARF module versioning
Documentation/kbuild/gendwarfksyms.rst | 308 ++++++ Documentation/kbuild/index.rst | 1 + MAINTAINERS | 7 + include/linux/export.h | 15 + kernel/module/Kconfig | 30 + scripts/Makefile | 3 +- scripts/Makefile.build | 35 +- scripts/gendwarfksyms/.gitignore | 2 + scripts/gendwarfksyms/Makefile | 12 + scripts/gendwarfksyms/cache.c | 51 + scripts/gendwarfksyms/die.c | 166 +++ scripts/gendwarfksyms/dwarf.c | 1159 ++++++++++++++++++++ scripts/gendwarfksyms/examples/kabi.h | 157 +++ scripts/gendwarfksyms/examples/kabi_ex.c | 30 + scripts/gendwarfksyms/examples/kabi_ex.h | 263 +++++ scripts/gendwarfksyms/examples/symbolptr.c | 33 + scripts/gendwarfksyms/gendwarfksyms.c | 187 ++++ scripts/gendwarfksyms/gendwarfksyms.h | 296 +++++ scripts/gendwarfksyms/kabi.c | 336 ++++++ scripts/gendwarfksyms/symbols.c | 341 ++++++ scripts/gendwarfksyms/types.c | 481 ++++++++ 21 files changed, 3906 insertions(+), 7 deletions(-) create mode 100644 Documentation/kbuild/gendwarfksyms.rst create mode 100644 scripts/gendwarfksyms/.gitignore create mode 100644 scripts/gendwarfksyms/Makefile create mode 100644 scripts/gendwarfksyms/cache.c create mode 100644 scripts/gendwarfksyms/die.c create mode 100644 scripts/gendwarfksyms/dwarf.c create mode 100644 scripts/gendwarfksyms/examples/kabi.h create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.c create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.h create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h create mode 100644 scripts/gendwarfksyms/kabi.c create mode 100644 scripts/gendwarfksyms/symbols.c create mode 100644 scripts/gendwarfksyms/types.c
base-commit: fc033cf25e612e840e545f8d5ad2edd6ba613ed5
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 1/18 tools: Add gendwarfksyms
Add a basic DWARF parser, which uses libdw to traverse the debugging information in an object file and looks for functions and variables. In follow-up patches, this will be expanded to produce symbol versions for CONFIG_MODVERSIONS from DWARF.
--- MAINTAINERS | 7 ++ kernel/module/Kconfig | 8 ++ scripts/Makefile | 1 + scripts/gendwarfksyms/.gitignore | 2 + scripts/gendwarfksyms/Makefile | 8 ++ scripts/gendwarfksyms/dwarf.c | 166 ++++++++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.c | 128 ++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 95 +++++++++++++++ scripts/gendwarfksyms/symbols.c | 98 +++++++++++++++ 9 files changed, 513 insertions(+) create mode 100644 scripts/gendwarfksyms/.gitignore create mode 100644 scripts/gendwarfksyms/Makefile create mode 100644 scripts/gendwarfksyms/dwarf.c create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h create mode 100644 scripts/gendwarfksyms/symbols.c
MAINTAINERS +7 -0
@@ -9550,6 +9550,13 @@ W: https://linuxtv.org T: git git://linuxtv.org/media.git F: drivers/media/radio/radio-gemtek* +GENDWARFKSYMS +M: Sami Tolvanen <[email protected]> +L: [email protected] +L: [email protected] +S: Maintained +F: scripts/gendwarfksyms/ + GENERIC ARCHITECTURE TOPOLOGY M: Sudeep Holla <[email protected]> L: [email protected]
kernel/module/Kconfig +8 -0
@@ -169,6 +169,14 @@ config MODVERSIONS make them incompatible with the kernel you are running. If unsure, say N. +config GENDWARFKSYMS + bool "gendwarfksyms (from debugging information)" + depends on DEBUG_INFO + # Requires full debugging information, split DWARF not supported. + depends on !DEBUG_INFO_REDUCED && !DEBUG_INFO_SPLIT + # Requires ELF object files. + depends on !LTO + config ASM_MODVERSIONS bool default HAVE_ASM_MODVERSIONS && MODVERSIONS
scripts/Makefile +1 -0
@@ -54,6 +54,7 @@ targets += module.lds subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins subdir-$(CONFIG_MODVERSIONS) += genksyms +subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms subdir-$(CONFIG_SECURITY_SELINUX) += selinux subdir-$(CONFIG_SECURITY_IPE) += ipe
scripts/gendwarfksyms/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +/gendwarfksyms
scripts/gendwarfksyms/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +hostprogs-always-y += gendwarfksyms + +gendwarfksyms-objs += gendwarfksyms.o +gendwarfksyms-objs += dwarf.o +gendwarfksyms-objs += symbols.o + +HOSTLDLIBS_gendwarfksyms := -ldw -lelf
scripts/gendwarfksyms/dwarf.c new file +166 -0
166 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.c new file +128 -0
128 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.h new file +95 -0
95 lines not on this page · raw message
scripts/gendwarfksyms/symbols.c new file +98 -0
98 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 2/18 gendwarfksyms: Add address matching
In reply to Sami TolvanenThe compiler may choose not to emit type information in DWARF for all aliases, but it's possible for each alias to be exported separately. To ensure we find type information for the aliases as well, read {section, address} tuples from the symbol table and match symbols also by address.
--- scripts/gendwarfksyms/gendwarfksyms.c | 2 + scripts/gendwarfksyms/gendwarfksyms.h | 13 +++ scripts/gendwarfksyms/symbols.c | 161 ++++++++++++++++++++++++++ 3 files changed, 176 insertions(+)
scripts/gendwarfksyms/gendwarfksyms.c +2 -0
@@ -105,6 +105,8 @@ int main(int argc, char **argv)error ("open failed for '%s': %s" , argv[n],strerror (errno)); +symbol_read_symtab (fd); + dwfl =dwfl_begin (&callbacks);if (!dwfl) error("dwfl_begin failed for '%s': %s", argv[n],
scripts/gendwarfksyms/gendwarfksyms.h +13 -0
@@ -63,14 +63,27 @@ extern int dump_dies;* symbols.c */ +static inline unsigned int addr_hash (uintptr_t addr) +{ +return hash_ptr ((const void *)addr); +} + +struct symbol_addr { +uint32_t section; +Elf64_Addr address; +}; +struct symbol {const char *name; +struct symbol_addr addr; +struct hlist_node addr_hash;struct hlist_node name_hash; };typedef void (*symbol_callback_t )(struct symbol *,void *arg);void symbol_read_exports (FILE *file); +void symbol_read_symtab (int fd);struct symbol *symbol_get (const char *name);void symbol_free (void );
scripts/gendwarfksyms/symbols.c +161 -0
196 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 3/18 gendwarfksyms: Expand base_type
In reply to Sami TolvanenStart making gendwarfksyms more useful by adding support for expanding DW_TAG_base_type types and basic DWARF attributes.
Example:
$ echo loops_per_jiffy | \
scripts/gendwarfksyms/gendwarfksyms \
--debug --dump-dies vmlinux.o
...
gendwarfksyms: process_symbol: loops_per_jiffy
variable base_type unsigned long byte_size(8) encoding(7)
...--- scripts/gendwarfksyms/dwarf.c | 160 ++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+)
scripts/gendwarfksyms/dwarf.c +160 -0
@@ -3,8 +3,21 @@* Copyright (C) 2024 Google LLC */ +#include <inttypes.h> +#include <stdarg.h> #include "gendwarfksyms.h" +#define DEFINE_GET_ATTR (attr, type) \ + static bool get_##attr##_attr(Dwarf_Die *die, unsigned int id, \ + type *value) \ + { \ + Dwarf_Attribute da; \ + return dwarf_attr(die, id, &da) && \ + !dwarf_form##attr(&da, value); \ + } + +DEFINE_GET_ATTR(udata, Dwarf_Word) +static bool get_ref_die_attr (Dwarf_Die *die,unsigned int id,Dwarf_Die *value) {Dwarf_Attribute da;
@@ -67,6 +80,109 @@ static void process(const char *s)fputs (s, stderr); } +#define MAX_FMT_BUFFER_SIZE 128 + +static void process_fmt (const char *fmt, ...) +{ +char buf[MAX_FMT_BUFFER_SIZE ]; +va_list args; + +va_start (args, fmt); + +if (checkp (vsnprintf (buf,sizeof (buf), fmt, args)) >=sizeof (buf)) +error ("vsnprintf overflow: increase MAX_FMT_BUFFER_SIZE" ); + +process (buf); +va_end (args); +} + +#define MAX_FQN_SIZE 64 + +/* Get a fully qualified name from DWARF scopes */ +static char *get_fqn (Dwarf_Die *die) +{ +const char *list[MAX_FQN_SIZE ]; +Dwarf_Die *scopes =NULL ; +bool has_name = false; +char *fqn =NULL ; +char *p; +int count =0 ; +int len =0 ; +int res; +int i; + + res =checkp (dwarf_getscopes_die (die, &scopes)); +if (!res) { + list[count] =get_name_attr (die); + +if (!list[count]) +return NULL ; + + len +=strlen (list[count]); + count++; + + goto done; + } + +for (i = res -1 ; i >=0 && count <MAX_FQN_SIZE ; i--) { +if (dwarf_tag (&scopes[i]) == DW_TAG_compile_unit) +continue ; + + list[count] =get_name_attr (&scopes[i]); + +if (list[count]) { + has_name = true; + }else { + list[count] ="<anonymous>" ; + has_name = false; + } + + len +=strlen (list[count]); + count++; + +if (i >0 ) { + list[count++] ="::" ; + len +=2 ; + } + } + +free (scopes); + +if (count ==MAX_FQN_SIZE ) +warn ("increase MAX_FQN_SIZE: reached the maximum" ); + +/* Consider the DIE unnamed if the last scope doesn't have a name */ +if (!has_name) +return NULL ; +done: + fqn =xmalloc (len +1 ); + *fqn ='\0' ; + + p = fqn; +for (i =0 ; i < count; i++) + p =stpcpy (p, list[i]); + +return fqn; +} + +static void process_fqn (Dwarf_Die *die) +{ +process (" " ); +process (get_fqn (die) ?:"" ); +} + +#define DEFINE_PROCESS_UDATA_ATTRIBUTE (attribute) \ + static void process_##attribute##_attr(Dwarf_Die *die) \ + { \ + Dwarf_Word value; \ + if (get_udata_attr(die, DW_AT_##attribute, &value)) \ + process_fmt(" " #attribute "(%" PRIu64 ")", value); \ + } + +DEFINE_PROCESS_UDATA_ATTRIBUTE (alignment) +DEFINE_PROCESS_UDATA_ATTRIBUTE (byte_size ) +DEFINE_PROCESS_UDATA_ATTRIBUTE (encoding) + boolmatch_all (Dwarf_Die *die) {return true;
@@ -93,6 +209,49 @@ int process_die_container(struct state *state, Dwarf_Die *die,return 0 ; } +static int process_type (struct state *state,Dwarf_Die *die); + +static void process_type_attr (struct state *state,Dwarf_Die *die) +{ +Dwarf_Die type; + +if (get_ref_die_attr (die, DW_AT_type, &type)) { +check (process_type (state, &type)); +return ; + } + +/* Compilers can omit DW_AT_type -- print out 'void' to clarify */ +process ("base_type void" ); +} + +static void process_base_type (struct state *state,Dwarf_Die *die) +{ +process ("base_type" ); +process_fqn (die); +process_byte_size_attr (die); +process_encoding_attr (die); +process_alignment_attr (die); +} + +#define PROCESS_TYPE (type) \ + case DW_TAG_##type##_type: \ + process_##type##_type(state, die); \ + break; + +static int process_type (struct state *state,Dwarf_Die *die) +{ +int tag =dwarf_tag (die); + +switch (tag) { + PROCESS_TYPE(base) + default: + debug("unimplemented type: %x", tag); +break ; + } + +return 0 ; +} +/* * Exported symbol processing */
@@ -119,6 +278,7 @@ static void process_subprogram(struct state *state, Dwarf_Die *die)static int __process_variable (struct state *state,Dwarf_Die *die) {process ("variable " ); +process_type_attr (state, die);return 0 ; }
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 4/18 gendwarfksyms: Add a cache for processed DIEs
In reply to Sami TolvanenBasic types in DWARF repeat frequently and traversing the DIEs using libdw is relatively slow. Add a simple hashtable based cache for the processed DIEs.
--- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/die.c | 143 ++++++++++++++++++++++++++ scripts/gendwarfksyms/dwarf.c | 136 +++++++++++++++++------- scripts/gendwarfksyms/gendwarfksyms.c | 6 ++ scripts/gendwarfksyms/gendwarfksyms.h | 63 +++++++++++- 5 files changed, 308 insertions(+), 41 deletions(-) create mode 100644 scripts/gendwarfksyms/die.c
scripts/gendwarfksyms/Makefile +1 -0
@@ -2,6 +2,7 @@ hostprogs-always-y += gendwarfksyms gendwarfksyms-objs += gendwarfksyms.o +gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o
scripts/gendwarfksyms/die.c new file +143 -0
143 lines not on this page · raw message
scripts/gendwarfksyms/dwarf.c +98 -38
244 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.c +6 -0
@@ -43,6 +43,10 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name,debug ("%s" , name); dbg =dwfl_module_getdwarf (mod, &dwbias); +/* +* Look for exported symbols in each CU, follow the DIE tree, and add +* the entries to die_map. +*/ do { res = dwarf_get_units(dbg, cu, &cu, NULL, NULL, &cudie, NULL); if (res < 0)
@@ -53,6 +57,8 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name,process_cu (&cudie); }while (cu); +die_map_free (); +return DWARF_CB_OK ; }
scripts/gendwarfksyms/gendwarfksyms.h +60 -3
78 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 5/18 gendwarfksyms: Expand type modifiers and typedefs
In reply to Sami TolvanenAdd support for expanding DWARF type modifiers, such as pointers, const values etc., and typedefs. These types all have DW_AT_type attribute pointing to the underlying type, and thus produce similar output.
Also add linebreaks and indentation to debugging output to make it more readable.
--- scripts/gendwarfksyms/die.c | 12 +++++ scripts/gendwarfksyms/dwarf.c | 67 +++++++++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 5 ++ 3 files changed, 84 insertions(+)
scripts/gendwarfksyms/die.c +12 -0
@@ -130,6 +130,18 @@ void die_map_add_string(struct die *cd, const char *str) df->type =FRAGMENT_STRING ; } +void die_map_add_linebreak (struct die *cd,int linebreak) +{ +struct die_fragment *df; + +if (!cd) +return ; + + df =append_item (cd); + df->data.linebreak = linebreak; + df->type =FRAGMENT_LINEBREAK ; +} +void die_map_add_die (struct die *cd,struct die *child) {struct die_fragment *df;
scripts/gendwarfksyms/dwarf.c +67 -0
98 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.h +5 -0
@@ -59,6 +59,9 @@ extern int dump_dies;/* Error == negative values */ #define checkp (expr) __check(expr, __res < 0) +/* Consistent aliases (DW_TAG_<type>_type) for DWARF tags */ +#define DW_TAG_typedef_type DW_TAG_typedef +/* * symbols.c */
@@ -100,6 +103,7 @@ enum die_state {enum die_fragment_type {FRAGMENT_EMPTY ,FRAGMENT_STRING , +FRAGMENT_LINEBREAK ,FRAGMENT_DIE };
@@ -107,6 +111,7 @@ struct die_fragment {enum die_fragment_type type;union {char *str; +int linebreak;uintptr_t addr; } data;struct list_head list;
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 6/18 gendwarfksyms: Expand subroutine_type
In reply to Sami TolvanenAdd support for expanding DW_TAG_subroutine_type and the parameters in DW_TAG_formal_parameter. Use this to also expand subprograms.
Example output with --dump-dies:
subprogram (
formal_parameter pointer_type {
const_type {
base_type char byte_size(1) encoding(6)
}
}
)
-> base_type unsigned long byte_size(8) encoding(7)--- scripts/gendwarfksyms/dwarf.c | 84 ++++++++++++++++++++++++++- scripts/gendwarfksyms/gendwarfksyms.h | 4 ++ 2 files changed, 85 insertions(+), 3 deletions(-)
scripts/gendwarfksyms/dwarf.c +81 -3
@@ -212,6 +212,15 @@ DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) DEFINE_PROCESS_UDATA_ATTRIBUTE(byte_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) +/* Match functions -- die_match_callback_t */ +#define DEFINE_MATCH(type) \ + static bool match_##type##_type(Dwarf_Die *die) \ + { \ + return dwarf_tag(die) == DW_TAG_##type##_type; \ + } + +DEFINE_MATCH (formal_parameter) + bool match_all(Dwarf_Die *die) {return true;
@@ -224,19 +233,28 @@ int process_die_container(struct state *state, struct die *cache,Dwarf_Die current;int res; +/* Track the first item in lists. */ +if (state) + state->first_list_item = true; + res =checkp (dwarf_child (die, ¤t));while (!res) {if (match (¤t)) {/* <0 = error, 0 = continue, >0 = stop */ res =checkp (func (state, cache, ¤t));if (res) -return res; + goto out; } res =checkp (dwarf_siblingof (¤t, ¤t)); } -return 0 ; + res =0 ; +out: +if (state) + state->first_list_item = false; + +return res; } static int process_type(struct state *state, struct die *parent,
@@ -256,6 +274,40 @@ static void process_type_attr(struct state *state, struct die *cache,process (cache,"base_type void" ); } +static void process_list_comma (struct state *state,struct die *cache) +{ +if (state->first_list_item) { + state->first_list_item = false; + }else { +process (cache," ," ); +process_linebreak (cache,0 ); + } +} + +/* Comma-separated with DW_AT_type */ +static void __process_list_type (struct state *state,struct die *cache, +Dwarf_Die *die,const char *type) +{ +const char *name =get_name_attr (die); + +process_list_comma (state, cache); +process (cache, type); +process_type_attr (state, cache, die); +if (name) { +process (cache," " ); +process (cache, name); + } +} + +#define DEFINE_PROCESS_LIST_TYPE (type) \ + static void process_##type##_type(struct state *state, \ + struct die *cache, Dwarf_Die *die) \ + { \ + __process_list_type(state, cache, die, #type " "); \ + } + +DEFINE_PROCESS_LIST_TYPE (formal_parameter) +/* Container types with DW_AT_type */ static void __process_type(struct state *state, struct die *cache, Dwarf_Die *die,const char *type)
@@ -290,6 +342,29 @@ DEFINE_PROCESS_TYPE(shared) DEFINE_PROCESS_TYPE(volatile) DEFINE_PROCESS_TYPE(typedef) +static void __process_subroutine_type(struct state *state, struct die *cache, + Dwarf_Die *die, const char *type) +{ + process(cache, type); + process(cache, " ("); + process_linebreak(cache, 1); + /* Parameters */ + check(process_die_container(state, cache, die, process_type, + match_formal_parameter_type)); + process_linebreak(cache, -1); + process(cache, ")"); + process_linebreak(cache, 0); + /* Return type */ + process(cache, "-> "); + process_type_attr(state, cache, die); +} + +static void process_subroutine_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + __process_subroutine_type(state, cache, die, "subroutine_type"); +} + static void process_base_type(struct state *state, struct die *cache, Dwarf_Die *die) {
@@ -360,8 +435,11 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) PROCESS_TYPE(rvalue_reference) PROCESS_TYPE(shared) PROCESS_TYPE(volatile) + /* Subtypes */ + PROCESS_TYPE(formal_parameter) /* Other types */ PROCESS_TYPE(base) + PROCESS_TYPE(subroutine) PROCESS_TYPE(typedef) default: debug("unimplemented type: %x", tag);
@@ -391,7 +469,7 @@ static void process_symbol(struct state *state, Dwarf_Die *die,static int __process_subprogram (struct state *state,struct die *cache,Dwarf_Die *die) { -process (cache,"subprogram" ); +__process_subroutine_type (state, cache, die,"subprogram" );return 0 ; }
scripts/gendwarfksyms/gendwarfksyms.h +4 -0
@@ -60,6 +60,7 @@ extern int dump_dies;#define checkp (expr) __check(expr, __res < 0)/* Consistent aliases (DW_TAG_<type>_type) for DWARF tags */ +#define DW_TAG_formal_parameter_type DW_TAG_formal_parameter#define DW_TAG_typedef_type DW_TAG_typedef/*
@@ -154,6 +155,9 @@ void die_map_free(void);struct state {struct symbol *sym;Dwarf_Die die; + +/* List expansion */ +bool first_list_item; }; typedef int (*die_callback_t)(struct state *state, struct die *cache,
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 7/18 gendwarfksyms: Expand array_type
In reply to Sami TolvanenAdd support for expanding DW_TAG_array_type, and the subrange type indicating array size.
Example source code:
const char *s[34];
Output with --dump-dies:
variable array_type[34] {
pointer_type {
const_type {
base_type char byte_size(1) encoding(6)
}
} byte_size(8)
}--- scripts/gendwarfksyms/dwarf.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
scripts/gendwarfksyms/dwarf.c +30 -0
@@ -220,6 +220,7 @@ DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) }DEFINE_MATCH (formal_parameter) +DEFINE_MATCH (subrange ) boolmatch_all (Dwarf_Die *die) {
@@ -342,6 +343,33 @@ DEFINE_PROCESS_TYPE(shared) DEFINE_PROCESS_TYPE(volatile)DEFINE_PROCESS_TYPE (typedef) +static void process_subrange_type (struct state *state,struct die *cache, +Dwarf_Die *die) +{ + Dwarf_Word count = 0; + + if (get_udata_attr(die, DW_AT_count, &count)) + process_fmt(cache, "[%" PRIu64 "]", count); + else if (get_udata_attr(die, DW_AT_upper_bound, &count)) + process_fmt(cache, "[%" PRIu64 "]", count + 1); + else + process(cache, "[]"); +} + +static void process_array_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ +process (cache,"array_type" ); +/* Array size */ +check (process_die_container (state, cache, die, process_type, + match_subrange_type)); +process (cache," {" ); +process_linebreak (cache,1 ); +process_type_attr (state, cache, die); +process_linebreak (cache,-1 ); +process (cache,"}" ); +} +static void __process_subroutine_type (struct state *state,struct die *cache,Dwarf_Die *die,const char *type) {
@@ -437,7 +465,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die)PROCESS_TYPE (volatile) /* Subtypes */ PROCESS_TYPE(formal_parameter) + PROCESS_TYPE(subrange) /* Other types */ + PROCESS_TYPE(array) PROCESS_TYPE(base) PROCESS_TYPE(subroutine) PROCESS_TYPE(typedef)
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 8/18 gendwarfksyms: Expand structure types
In reply to Sami TolvanenRecursively expand DWARF structure types, i.e. structs, unions, and enums. Also include relevant DWARF attributes in type strings to encode structure layout, for example.
Example output with --dump-dies:
subprogram (
formal_parameter structure_type &str {
member pointer_type {
base_type u8 byte_size(1) encoding(7)
} data_ptr data_member_location(0) ,
member base_type usize byte_size(8) encoding(7) length data_member_location(8)
} byte_size(16) alignment(8) msg
)
-> base_type void--- scripts/gendwarfksyms/dwarf.c | 138 +++++++++++++++++++++++++- scripts/gendwarfksyms/gendwarfksyms.h | 5 + 2 files changed, 141 insertions(+), 2 deletions(-)
scripts/gendwarfksyms/dwarf.c +136 -2
@@ -208,9 +208,14 @@ static void process_fqn(struct die *cache, Dwarf_Die *die) value); \ } +DEFINE_PROCESS_UDATA_ATTRIBUTE(accessibility) DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) +DEFINE_PROCESS_UDATA_ATTRIBUTE(bit_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(byte_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) +DEFINE_PROCESS_UDATA_ATTRIBUTE(data_bit_offset) +DEFINE_PROCESS_UDATA_ATTRIBUTE(data_member_location) +DEFINE_PROCESS_UDATA_ATTRIBUTE(discr_value)/* Match functions -- die_match_callback_t */ #define DEFINE_MATCH (type) \
@@ -219,7 +224,9 @@ DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) return dwarf_tag(die) == DW_TAG_##type##_type; \ } +DEFINE_MATCH(enumerator) DEFINE_MATCH(formal_parameter) +DEFINE_MATCH(member) DEFINE_MATCH(subrange) bool match_all(Dwarf_Die *die)
@@ -298,6 +305,10 @@ static void __process_list_type(struct state *state, struct die *cache,process (cache," " );process (cache, name); } +process_accessibility_attr (cache, die); +process_bit_size_attr (cache, die); +process_data_bit_offset_attr (cache, die); +process_data_member_location_attr (cache, die); }#define DEFINE_PROCESS_LIST_TYPE (type) \
@@ -308,6 +319,7 @@ static void __process_list_type(struct state *state, struct die *cache, } DEFINE_PROCESS_LIST_TYPE(formal_parameter) +DEFINE_PROCESS_LIST_TYPE(member) /* Container types with DW_AT_type */ static void __process_type(struct state *state, struct die *cache,
@@ -340,6 +352,7 @@ DEFINE_PROCESS_TYPE(reference) DEFINE_PROCESS_TYPE(restrict) DEFINE_PROCESS_TYPE(rvalue_reference) DEFINE_PROCESS_TYPE(shared) +DEFINE_PROCESS_TYPE(template_type_parameter) DEFINE_PROCESS_TYPE(volatile) DEFINE_PROCESS_TYPE(typedef)
@@ -393,6 +406,107 @@ static void process_subroutine_type(struct state *state, struct die *cache,__process_subroutine_type (state, cache, die,"subroutine_type" ); } +static void process_variant_type (struct state *state,struct die *cache, +Dwarf_Die *die) +{ +process_list_comma (state, cache); +process (cache,"variant {" ); +process_linebreak (cache,1 ); +check (process_die_container (state, cache, die, process_type, + match_member_type)); +process_linebreak (cache,-1 ); +process (cache,"}" ); +process_discr_value_attr (cache, die); +} + +static void process_variant_part_type (struct state *state,struct die *cache, +Dwarf_Die *die) +{ +process_list_comma (state, cache); +process (cache,"variant_part {" ); +process_linebreak (cache,1 ); +check (process_die_container (state, cache, die, process_type, + match_all)); +process_linebreak (cache,-1 ); +process (cache,"}" ); +} + +static int ___process_structure_type (struct state *state,struct die *cache, +Dwarf_Die *die) +{ +switch (dwarf_tag (die)) { +case DW_TAG_member: +case DW_TAG_variant_part: +return check (process_type (state, cache, die)); +case DW_TAG_class_type: +case DW_TAG_enumeration_type: +case DW_TAG_structure_type: +case DW_TAG_template_type_parameter: +case DW_TAG_union_type: +case DW_TAG_subprogram: +/* Skip non-member types, including member functions */ +return 0 ; +default : +error ("unexpected structure_type child: %x" ,dwarf_tag (die)); + } +} + +static void __process_structure_type (struct state *state,struct die *cache, +Dwarf_Die *die,const char *type, +die_callback_t process_func, +die_match_callback_t match_func) +{ +process (cache, type); +process_fqn (cache, die); +process (cache," {" ); +process_linebreak (cache,1 ); + +check (process_die_container (state, cache, die, process_func, + match_func)); + +process_linebreak (cache,-1 ); +process (cache,"}" ); + +process_byte_size_attr (cache, die); +process_alignment_attr (cache, die); +} + +#define DEFINE_PROCESS_STRUCTURE_TYPE (structure) \ + static void process_##structure##_type( \ + struct state *state, struct die *cache, Dwarf_Die *die) \ + { \ + __process_structure_type(state, cache, die, \ + #structure "_type", \ + ___process_structure_type, \ + match_all); \ + } + +DEFINE_PROCESS_STRUCTURE_TYPE (class) +DEFINE_PROCESS_STRUCTURE_TYPE (structure ) +DEFINE_PROCESS_STRUCTURE_TYPE (union) + +static void process_enumerator_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ +Dwarf_Word value; + +process_list_comma (state, cache); +process (cache,"enumerator" ); +process_fqn (cache, die); + +if (get_udata_attr (die, DW_AT_const_value, &value)) { +process (cache," = " ); +process_fmt (cache,"%" PRIu64, value); + } +} + +static void process_enumeration_type (struct state *state,struct die *cache, +Dwarf_Die *die) +{ +__process_structure_type (state, cache, die,"enumeration_type" , + process_type, match_enumerator_type); +} +static void process_base_type (struct state *state,struct die *cache,Dwarf_Die *die) {
@@ -403,6 +517,16 @@ static void process_base_type(struct state *state, struct die *cache,process_alignment_attr (cache, die); } +static void process_unspecified_type (struct state *state,struct die *cache, +Dwarf_Die *die) +{ +/* +* These can be emitted for stand-alone assembly code, which means we +* might run into them in vmlinux.o. +*/ +process (cache,"unspecified_type" ); +} +static void process_cached (struct state *state,struct die *cache,Dwarf_Die *die) {
@@ -463,17 +587,27 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) PROCESS_TYPE(rvalue_reference) PROCESS_TYPE(shared) PROCESS_TYPE(volatile) + /* Container types */ + PROCESS_TYPE(class) + PROCESS_TYPE(structure) + PROCESS_TYPE(union) + PROCESS_TYPE(enumeration) /* Subtypes */ + PROCESS_TYPE(enumerator) PROCESS_TYPE(formal_parameter) + PROCESS_TYPE(member) PROCESS_TYPE(subrange) + PROCESS_TYPE(template_type_parameter) + PROCESS_TYPE(variant) + PROCESS_TYPE(variant_part) /* Other types */ PROCESS_TYPE(array) PROCESS_TYPE(base) PROCESS_TYPE(subroutine) PROCESS_TYPE(typedef) + PROCESS_TYPE(unspecified) default: - debug("unimplemented type: %x", tag); -break ; + error("unexpected type: %x", tag); }/* Update cache state and append to the parent (if any) */
scripts/gendwarfksyms/gendwarfksyms.h +5 -0
@@ -60,8 +60,13 @@ extern int dump_dies;#define checkp (expr) __check(expr, __res < 0)/* Consistent aliases (DW_TAG_<type>_type) for DWARF tags */ +#define DW_TAG_enumerator_type DW_TAG_enumerator#define DW_TAG_formal_parameter_type DW_TAG_formal_parameter +#define DW_TAG_member_type DW_TAG_member +#define DW_TAG_template_type_parameter_type DW_TAG_template_type_parameter#define DW_TAG_typedef_type DW_TAG_typedef +#define DW_TAG_variant_part_type DW_TAG_variant_part +#define DW_TAG_variant_type DW_TAG_variant/* * symbols.c
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 9/18 gendwarfksyms: Limit structure expansion
In reply to Sami TolvanenExpand each structure type only once per exported symbol. This is necessary to support self-referential structures, which would otherwise result in infinite recursion, and it's sufficient for catching ABI changes.
Types defined in .c files are opaque to external users and thus cannot affect the ABI. Consider type definitions in .c files to be declarations to prevent opaque types from changing symbol versions.
--- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/cache.c | 51 +++++++++++ scripts/gendwarfksyms/dwarf.c | 125 ++++++++++++++++++++++++-- scripts/gendwarfksyms/gendwarfksyms.h | 46 ++++++++++ 4 files changed, 215 insertions(+), 8 deletions(-) create mode 100644 scripts/gendwarfksyms/cache.c
scripts/gendwarfksyms/Makefile +1 -0
@@ -2,6 +2,7 @@ hostprogs-always-y += gendwarfksyms gendwarfksyms-objs += gendwarfksyms.o +gendwarfksyms-objs += cache.o gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o
scripts/gendwarfksyms/cache.c new file +51 -0
51 lines not on this page · raw message
scripts/gendwarfksyms/dwarf.c +117 -8
196 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.h +46 -0
74 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 10/18 gendwarfksyms: Add die_map debugging
In reply to Sami TolvanenDebugging the DWARF processing can be somewhat challenging, so add more detailed debugging output for die_map operations. Add the --dump-die-map flag, which adds color coded tags to the output for die_map changes.
--- scripts/gendwarfksyms/dwarf.c | 15 +++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.c | 7 +++++++ scripts/gendwarfksyms/gendwarfksyms.h | 13 +++++++++++++ 3 files changed, 35 insertions(+)
scripts/gendwarfksyms/dwarf.c +15 -0
@@ -146,6 +146,8 @@ static void process(struct die *cache, const char *s)if (dump_dies)fputs (s, stderr); +if (cache) +die_debug_r ("cache %p string '%s'" , cache, s);die_map_add_string (cache, s); }
@@ -594,6 +596,8 @@ static void process_cached(struct state *state, struct die *cache,list_for_each_entry (df, &cache->fragments, list) {switch (df->type) {case FRAGMENT_STRING : +die_debug_b ("cache %p STRING '%s'" , cache, + df->data.str);process (NULL , df->data.str);break ;case FRAGMENT_LINEBREAK :
@@ -603,6 +607,8 @@ static void process_cached(struct state *state, struct die *cache,if (!dwarf_die_addr_die (dwarf_cu_getdwarf (die->cu), (void *)df->data.addr, &child))error ("dwarf_die_addr_die failed" ); +die_debug_b ("cache %p DIE addr %" PRIxPTR" tag %x" , + cache, df->data.addr,dwarf_tag (&child));check (process_type (state,NULL , &child));break ;default :
@@ -671,6 +677,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) cache =die_map_get (die, want_state);if (cache->state == want_state) { +die_debug_g ("cached addr %p tag %x -- %s" , die->addr, tag, +die_state_name (cache->state)); +process_cached (state, cache, die);die_map_add_die (parent, cache);
@@ -678,6 +687,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die)return 0 ; } +die_debug_g ("addr %p tag %x -- %s -> %s" , die->addr, tag, +die_state_name (cache->state),die_state_name (want_state)); +switch (tag) {/* Type modifiers */ PROCESS_TYPE (atomic)
@@ -713,6 +725,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die)error ("unexpected type: %x" , tag); } +die_debug_r ("parent %p cache %p die addr %p tag %x" , parent, cache, + die->addr, tag); +/* Update cache state and append to the parent (if any) */ cache->tag = tag; cache->state = want_state;
scripts/gendwarfksyms/gendwarfksyms.c +7 -0
@@ -19,6 +19,8 @@int debug;/* Dump DIE contents */ int dump_dies; +/* Print debugging information about die_map changes */ +int dump_die_map;static void usage (void ) {
@@ -26,6 +28,7 @@ static void usage(void)"Options:\n" " -d, --debug Print debugging information\n" " --dump-dies Dump DWARF DIE contents\n" +" --dump-die-map Print debugging information about die_map changes\n" " -h, --help Print this message\n" "\n" , stderr);
@@ -75,6 +78,7 @@ int main(int argc, char **argv)static const struct option opts[] = { {"debug" ,0 ,NULL ,'d' }, {"dump-dies" ,0 , &dump_dies,1 }, + {"dump-die-map" ,0 , &dump_die_map,1 }, {"help" ,0 ,NULL ,'h' }, {0 ,0 ,NULL ,0 } };
@@ -95,6 +99,9 @@ int main(int argc, char **argv) } } +if (dump_die_map) + dump_dies =1 ; +if (optind >= argc) {usage ();error ("no input files?" );
scripts/gendwarfksyms/gendwarfksyms.h +13 -0
@@ -21,6 +21,7 @@*/ extern int debug;extern int dump_dies; +extern int dump_die_map;/* * Output helpers
@@ -43,6 +44,18 @@ extern int dump_dies;exit (1 ); \ } while (0) +#define __die_debug (color, format, ...) \ + do { \ + if (dump_dies && dump_die_map) \ + fprintf(stderr, \ + "\033[" #color "m<" format ">\033[39m", \ + __VA_ARGS__); \ + } while (0) + +#define die_debug_r (format, ...) __die_debug(91, format, __VA_ARGS__) +#define die_debug_g (format, ...) __die_debug(92, format, __VA_ARGS__) +#define die_debug_b (format, ...) __die_debug(94, format, __VA_ARGS__) +/* * Error handling helpers */
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 11/18 gendwarfksyms: Add symtypes output
In reply to Sami TolvanenAdd support for producing genksyms-style symtypes files. Process die_map to find the longest expansions for each type, and use symtypes references in type definitions. The basic file format is similar to genksyms, with two notable exceptions:
1. Type names with spaces (common with Rust) in references are wrapped in single quotes. E.g.:
s#'core::result::Result<u8, core::num::error::ParseIntError>'
2. The actual type definition is the simple parsed DWARF format we output with --dump-dies, not the preprocessed C-style format genksyms produces.
--- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/die.c | 11 + scripts/gendwarfksyms/dwarf.c | 1 + scripts/gendwarfksyms/gendwarfksyms.c | 33 ++- scripts/gendwarfksyms/gendwarfksyms.h | 19 ++ scripts/gendwarfksyms/symbols.c | 4 +- scripts/gendwarfksyms/types.c | 363 ++++++++++++++++++++++++++ 7 files changed, 429 insertions(+), 3 deletions(-) create mode 100644 scripts/gendwarfksyms/types.c
scripts/gendwarfksyms/Makefile +1 -0
@@ -6,5 +6,6 @@ gendwarfksyms-objs += cache.o gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o +gendwarfksyms-objs += types.o HOSTLDLIBS_gendwarfksyms := -ldw -lelf
scripts/gendwarfksyms/die.c +11 -0
@@ -22,6 +22,7 @@ static inline unsigned int die_hash(uintptr_t addr, enum die_state state)static void init_die (struct die *cd) { cd->state =DIE_INCOMPLETE ; + cd->mapped = false; cd->fqn =NULL ; cd->tag =-1 ; cd->addr =0 ;
@@ -83,6 +84,16 @@ static void reset_die(struct die *cd)init_die (cd); } +void die_map_for_each (die_map_callback_t func,void *arg) +{ +struct hlist_node *tmp; +struct die *cd; + +hash_for_each_safe (die_map, cd, tmp, hash) { +func (cd, arg); + } +} +void die_map_free (void ) {struct hlist_node *tmp;
scripts/gendwarfksyms/dwarf.c +1 -0
@@ -745,6 +745,7 @@ static void process_symbol(struct state *state, Dwarf_Die *die, {debug ("%s" , state->sym->name);check (process_func (state,NULL , die)); + state->sym->state =SYMBOL_MAPPED ;if (dump_dies)fputs ("\n" , stderr); }
scripts/gendwarfksyms/gendwarfksyms.c +31 -2
@@ -21,6 +21,11 @@ int debug;int dump_dies;/* Print debugging information about die_map changes */ int dump_die_map; +/* Print out type strings (i.e. type_map) */ +int dump_types; +/* Write a symtypes file */ +int symtypes; +static const char *symtypes_file;static void usage (void ) {
@@ -29,6 +34,8 @@ static void usage(void)" -d, --debug Print debugging information\n" " --dump-dies Dump DWARF DIE contents\n" " --dump-die-map Print debugging information about die_map changes\n" +" --dump-types Dump type strings\n" +" -T, --symtypes file Write a symtypes file\n" " -h, --help Print this message\n" "\n" , stderr);
@@ -41,6 +48,7 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name,Dwarf_Die cudie;Dwarf_CU *cu =NULL ;Dwarf *dbg; +FILE *symfile = arg;int res;debug ("%s" , name);
@@ -60,6 +68,10 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name,process_cu (&cudie); }while (cu); +/* +* Use die_map to expand type strings and write them to `symfile`. +*/ +generate_symtypes (symfile);die_map_free ();return DWARF_CB_OK ;
@@ -72,6 +84,7 @@ static const Dwfl_Callbacks callbacks = {int main (int argc,char **argv) { +FILE *symfile =NULL ;unsigned int n;int opt;
@@ -79,17 +92,23 @@ int main(int argc, char **argv) { "debug", 0, NULL, 'd' }, { "dump-dies", 0, &dump_dies, 1 }, { "dump-die-map", 0, &dump_die_map, 1 }, + { "dump-types", 0, &dump_types, 1 }, + { "symtypes", 1, NULL, 'T' }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 } }; -while ((opt =getopt_long (argc, argv,"dh" , opts,NULL )) !=EOF ) { +while ((opt =getopt_long (argc, argv,"dT:h" , opts,NULL )) !=EOF ) {switch (opt) {case 0 :break ;case 'd' : debug =1 ;break ; +case 'T' : + symtypes =1 ; + symtypes_file = optarg; +break ;case 'h' :usage ();return 0 ;
@@ -109,6 +128,13 @@ int main(int argc, char **argv)symbol_read_exports (stdin); +if (symtypes_file) { + symfile =fopen (symtypes_file,"w" ); +if (!symfile) +error ("fopen failed for '%s': %s" , symtypes_file, +strerror (errno)); + } +for (n = optind; n < argc; n++) {Dwfl *dwfl;int fd;
@@ -131,12 +157,15 @@ int main(int argc, char **argv)dwfl_report_end (dwfl,NULL ,NULL ); -if (dwfl_getmodules (dwfl, &process_module,NULL ,0 )) +if (dwfl_getmodules (dwfl, &process_module, symfile,0 ))error ("dwfl_getmodules failed for '%s'" , argv[n]);dwfl_end (dwfl); } +if (symfile) +check (fclose (symfile)); +symbol_free ();return 0 ;
scripts/gendwarfksyms/gendwarfksyms.h +19 -0
@@ -22,6 +22,8 @@extern int debug;extern int dump_dies;extern int dump_die_map; +extern int dump_types; +extern int symtypes;/* * Output helpers
@@ -90,6 +92,11 @@ static inline unsigned int addr_hash(uintptr_t addr)return hash_ptr ((const void *)addr); } +enum symbol_state { +SYMBOL_UNPROCESSED , +SYMBOL_MAPPED , +}; +struct symbol_addr {uint32_t section;Elf64_Addr address;
@@ -100,6 +107,8 @@ struct symbol {struct symbol_addr addr;struct hlist_node addr_hash;struct hlist_node name_hash; +enum symbol_state state; +uintptr_t die_addr; };typedef void (*symbol_callback_t )(struct symbol *,void *arg);
@@ -154,6 +163,7 @@ static inline const char *die_state_name(enum die_state state)struct die {enum die_state state; +bool mapped;char *fqn;int tag;uintptr_t addr;
@@ -161,10 +171,13 @@ struct die {struct hlist_node hash; }; +typedef void (*die_map_callback_t )(struct die *,void *arg); +int __die_map_get (uintptr_t addr,enum die_state state,struct die **res);struct die *die_map_get (Dwarf_Die *die,enum die_state state);void die_map_add_string (struct die *pd,const char *str);void die_map_add_linebreak (struct die *pd,int linebreak); +void die_map_for_each (die_map_callback_t func,void *arg);void die_map_add_die (struct die *pd,struct die *child);void die_map_free (void );
@@ -235,4 +248,10 @@ int process_die_container(struct state *state, struct die *cache,void process_cu (Dwarf_Die *cudie); +/* +* types.c +*/ + +void generate_symtypes (FILE *file); +#endif /* __GENDWARFKSYMS_H */
scripts/gendwarfksyms/symbols.c +3 -1
@@ -92,6 +92,7 @@ void symbol_read_exports(FILE *file) sym =xcalloc (1 ,sizeof (struct symbol )); sym->name = name; sym->addr.section =SHN_UNDEF ; + sym->state =SYMBOL_UNPROCESSED ;hash_add (symbol_names, &sym->name_hash,hash_str (sym->name)); ++nsym;
@@ -107,7 +108,8 @@ static void get_symbol(struct symbol *sym, void *arg) {struct symbol **res = arg; - *res = sym; +if (sym->state ==SYMBOL_UNPROCESSED ) + *res = sym; }struct symbol *symbol_get (const char *name)
scripts/gendwarfksyms/types.c new file +363 -0
363 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 12/18 gendwarfksyms: Add symbol versioning
In reply to Sami TolvanenCalculate symbol versions from the fully expanded type strings in type_map, and output the versions in a genksyms-compatible format.
--- scripts/gendwarfksyms/Makefile | 2 +- scripts/gendwarfksyms/dwarf.c | 25 +++++- scripts/gendwarfksyms/gendwarfksyms.c | 10 ++- scripts/gendwarfksyms/gendwarfksyms.h | 13 ++- scripts/gendwarfksyms/symbols.c | 53 +++++++++++ scripts/gendwarfksyms/types.c | 122 +++++++++++++++++++++++++- 6 files changed, 216 insertions(+), 9 deletions(-)
scripts/gendwarfksyms/Makefile +1 -1
@@ -8,4 +8,4 @@ gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o gendwarfksyms-objs += types.o -HOSTLDLIBS_gendwarfksyms := -ldw -lelf +HOSTLDLIBS_gendwarfksyms := -ldw -lelf -lz
scripts/gendwarfksyms/dwarf.c +23 -2
@@ -740,12 +740,33 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die)/* * Exported symbol processing */ +static struct die *get_symbol_cache (struct state *state,Dwarf_Die *die) +{ +struct die *cache; + + cache =die_map_get (die,DIE_SYMBOL ); + +if (cache->state !=DIE_INCOMPLETE ) +return NULL ;/* We already processed a symbol for this DIE */ + + cache->tag =dwarf_tag (die); +return cache; +} +static void process_symbol (struct state *state,Dwarf_Die *die,die_callback_t process_func) { +struct die *cache; + +symbol_set_die (state->sym, die); + + cache =get_symbol_cache (state, die); +if (!cache) +return ; +debug ("%s" , state->sym->name); -check (process_func (state,NULL , die)); - state->sym->state =SYMBOL_MAPPED ; +check (process_func (state, cache, die)); + cache->state =DIE_SYMBOL ;if (dump_dies)fputs ("\n" , stderr); }
scripts/gendwarfksyms/gendwarfksyms.c +8 -2
@@ -23,6 +23,8 @@ int dump_dies;int dump_die_map;/* Print out type strings (i.e. type_map) */ int dump_types; +/* Print out expanded type strings used for symbol versions */ +int dump_versions;/* Write a symtypes file */ int symtypes;static const char *symtypes_file;
@@ -35,6 +37,7 @@ static void usage(void)" --dump-dies Dump DWARF DIE contents\n" " --dump-die-map Print debugging information about die_map changes\n" " --dump-types Dump type strings\n" +" --dump-versions Dump expanded type strings used for symbol versions\n" " -T, --symtypes file Write a symtypes file\n" " -h, --help Print this message\n" "\n",
@@ -69,9 +72,10 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name, }while (cu);/* -* Use die_map to expand type strings and write them to `symfile`. +* Use die_map to expand type strings, write them to `symfile`, and +* calculate symbol versions. */ -generate_symtypes (symfile); +generate_symtypes_and_versions (symfile);die_map_free ();return DWARF_CB_OK ;
@@ -93,6 +97,7 @@ int main(int argc, char **argv) { "dump-dies", 0, &dump_dies, 1 }, { "dump-die-map", 0, &dump_die_map, 1 }, { "dump-types", 0, &dump_types, 1 }, + { "dump-versions", 0, &dump_versions, 1 }, { "symtypes", 1, NULL, 'T' }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 }
@@ -166,6 +171,7 @@ int main(int argc, char **argv)if (symfile)check (fclose (symfile)); +symbol_print_versions ();symbol_free ();return 0 ;
scripts/gendwarfksyms/gendwarfksyms.h +11 -2
@@ -23,6 +23,7 @@ extern int debug;extern int dump_dies;extern int dump_die_map;extern int dump_types; +extern int dump_versions;extern int symtypes;/*
@@ -95,6 +96,7 @@ static inline unsigned int addr_hash(uintptr_t addr)enum symbol_state {SYMBOL_UNPROCESSED ,SYMBOL_MAPPED , +SYMBOL_PROCESSED };struct symbol_addr {
@@ -109,6 +111,7 @@ struct symbol {struct hlist_node name_hash;enum symbol_state state;uintptr_t die_addr; +unsigned long crc; };typedef void (*symbol_callback_t )(struct symbol *,void *arg);
@@ -116,6 +119,10 @@ typedef void (*symbol_callback_t)(struct symbol *, void *arg);void symbol_read_exports (FILE *file);void symbol_read_symtab (int fd);struct symbol *symbol_get (const char *name); +void symbol_set_die (struct symbol *sym,Dwarf_Die *die); +void symbol_set_crc (struct symbol *sym,unsigned long crc); +void symbol_for_each (symbol_callback_t func,void *arg); +void symbol_print_versions (void );void symbol_free (void );/*
@@ -126,7 +133,8 @@ enum die_state {DIE_INCOMPLETE ,DIE_UNEXPANDED ,DIE_COMPLETE , -DIE_LAST =DIE_COMPLETE +DIE_SYMBOL , +DIE_LAST =DIE_SYMBOL }; enum die_fragment_type {
@@ -156,6 +164,7 @@ static inline const char *die_state_name(enum die_state state) CASE_CONST_TO_STR(DIE_INCOMPLETE) CASE_CONST_TO_STR(DIE_UNEXPANDED) CASE_CONST_TO_STR(DIE_COMPLETE) + CASE_CONST_TO_STR(DIE_SYMBOL) }error ("unexpected die_state: %d" , state);
@@ -252,6 +261,6 @@ void process_cu(Dwarf_Die *cudie);* types.c */ -void generate_symtypes (FILE *file); +void generate_symtypes_and_versions (FILE *file);#endif /* __GENDWARFKSYMS_H */
scripts/gendwarfksyms/symbols.c +53 -0
71 lines not on this page · raw message
scripts/gendwarfksyms/types.c +120 -2
164 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 13/18 gendwarfksyms: Add support for kABI rules
In reply to Sami TolvanenDistributions that want to maintain a stable kABI need the ability to make ABI compatible changes to kernel without affecting symbol versions, either because of LTS updates or backports.
With genksyms, developers would typically hide these changes from version calculation with #ifndef __GENKSYMS__, which would result in the symbol version not changing even though the actual type has changed. When we process precompiled object files, this isn't an option.
To support this use case, add a --stable command line flag that gates kABI stability features that are not needed in mainline kernels, but can be useful for distributions, and add support for kABI rules, which can be used to restrict gendwarfksyms output.
The rules are specified as a set of null-terminated strings stored in the .discard.gendwarfksyms.kabi_rules section. Each rule consists of four strings as follows:
"version\0type\0target\0value"
The version string ensures the structure can be changed in a backwards compatible way. The type string indicates the type of the rule, and target and value strings contain rule-specific data.
Initially support two simple rules:
1. Declaration-only types
A type declaration can change into a full definition when additional includes are pulled in to the TU, which changes the versions of any symbol that references the type. Add support for defining declaration-only types whose definition is not expanded during versioning.
2. Ignored enumerators
It's possible to add new enum fields without changing the ABI, but as the fields are included in symbol versioning, this would change the versions. Add support for ignoring specific fields.
3. Overridden enumerator values
Add support for overriding enumerator values when calculating versions. This may be needed when the last field of the enum is used as a sentinel and new fields must be added before it.
Add examples for using the rules under the examples/ directory.
--- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/dwarf.c | 25 +- scripts/gendwarfksyms/examples/kabi.h | 70 +++++ scripts/gendwarfksyms/examples/kabi_ex.c | 14 + scripts/gendwarfksyms/examples/kabi_ex.h | 64 +++++ scripts/gendwarfksyms/gendwarfksyms.c | 11 +- scripts/gendwarfksyms/gendwarfksyms.h | 14 + scripts/gendwarfksyms/kabi.c | 336 +++++++++++++++++++++++ 8 files changed, 531 insertions(+), 4 deletions(-) create mode 100644 scripts/gendwarfksyms/examples/kabi.h create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.c create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.h create mode 100644 scripts/gendwarfksyms/kabi.c
scripts/gendwarfksyms/Makefile +1 -0
@@ -5,6 +5,7 @@ gendwarfksyms-objs += gendwarfksyms.o gendwarfksyms-objs += cache.o gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o +gendwarfksyms-objs += kabi.o gendwarfksyms-objs += symbols.o gendwarfksyms-objs += types.o
scripts/gendwarfksyms/dwarf.c +22 -3
@@ -120,13 +120,16 @@ static bool is_definition_private(Dwarf_Die *die)return !!res; } -static bool is_kabi_definition (Dwarf_Die *die) +static bool is_kabi_definition (struct die *cache,Dwarf_Die *die) {bool value;if (get_flag_attr (die, DW_AT_declaration, &value) && value)return false; +if (kabi_is_declonly (cache->fqn)) +return false; +return !is_definition_private (die); }
@@ -515,9 +518,10 @@ static void __process_structure_type(struct state *state, struct die *cache,process (cache," {" );process_linebreak (cache,1 ); - expand = state->expand.expand &&is_kabi_definition (die); + expand = state->expand.expand &&is_kabi_definition (cache, die);if (expand) { + state->expand.current_fqn = cache->fqn;check (process_die_container (state, cache, die, process_func, match_func)); }
@@ -548,13 +552,26 @@ DEFINE_PROCESS_STRUCTURE_TYPE(union)static void process_enumerator_type (struct state *state,struct die *cache,Dwarf_Die *die) { +bool overridden = false;Dwarf_Word value; +if (stable) { +/* Get the fqn before we process anything */ +update_fqn (cache, die); + +if (kabi_is_enumerator_ignored (state->expand.current_fqn, + cache->fqn)) +return ; + + overridden =kabi_get_enumerator_value ( + state->expand.current_fqn, cache->fqn, &value); + } +process_list_comma (state, cache);process (cache,"enumerator" );process_fqn (cache, die); -if (get_udata_attr (die, DW_AT_const_value, &value)) { +if (overridden ||get_udata_attr (die, DW_AT_const_value, &value)) {process (cache," = " );process_fmt (cache,"%" PRIu64, value); }
@@ -620,6 +637,7 @@ static void process_cached(struct state *state, struct die *cache,static void state_init (struct state *state) { state->expand.expand = true; + state->expand.current_fqn =NULL ;cache_init (&state->expansion_cache); }
@@ -627,6 +645,7 @@ static void expansion_state_restore(struct expansion_state *state, struct expansion_state *saved) { state->expand = saved->expand; + state->current_fqn = saved->current_fqn; } static void expansion_state_save(struct expansion_state *state,
scripts/gendwarfksyms/examples/kabi.h new file +70 -0
70 lines not on this page · raw message
scripts/gendwarfksyms/examples/kabi_ex.c new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* kabi_ex.c +* +* Copyright (C) 2024 Google LLC +* +* Examples for kABI stability features with --stable. See kabi_ex.h +* for details. +*/ + +#include "kabi_ex.h" + +struct s e0; +enum e e1;
scripts/gendwarfksyms/examples/kabi_ex.h new file +64 -0
64 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.c +10 -1
@@ -25,6 +25,8 @@ int dump_die_map;int dump_types;/* Print out expanded type strings used for symbol versions */ int dump_versions; +/* Support kABI stability features */ +int stable;/* Write a symtypes file */ int symtypes;static const char *symtypes_file;
@@ -38,6 +40,7 @@ static void usage(void)" --dump-die-map Print debugging information about die_map changes\n" " --dump-types Dump type strings\n" " --dump-versions Dump expanded type strings used for symbol versions\n" +" -s, --stable Support kABI stability features\n" " -T, --symtypes file Write a symtypes file\n" " -h, --help Print this message\n" "\n",
@@ -98,18 +101,22 @@ int main(int argc, char **argv) { "dump-die-map", 0, &dump_die_map, 1 }, { "dump-types", 0, &dump_types, 1 }, { "dump-versions", 0, &dump_versions, 1 }, + { "stable", 0, NULL, 's' }, { "symtypes", 1, NULL, 'T' }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 } }; -while ((opt =getopt_long (argc, argv,"dT:h" , opts,NULL )) !=EOF ) { +while ((opt =getopt_long (argc, argv,"dsT:h" , opts,NULL )) !=EOF ) {switch (opt) {case 0 :break ;case 'd' : debug =1 ;break ; +case 's' : + stable =1 ; +break ;case 'T' : symtypes =1 ; symtypes_file = optarg;
@@ -150,6 +157,7 @@ int main(int argc, char **argv) strerror(errno));symbol_read_symtab (fd); +kabi_read_rules (fd); dwfl =dwfl_begin (&callbacks);if (!dwfl)
@@ -166,6 +174,7 @@ int main(int argc, char **argv)error ("dwfl_getmodules failed for '%s'" , argv[n]);dwfl_end (dwfl); +kabi_free (); }if (symfile)
scripts/gendwarfksyms/gendwarfksyms.h +14 -0
@@ -24,6 +24,7 @@ extern int dump_dies;extern int dump_die_map;extern int dump_types;extern int dump_versions; +extern int stable;extern int symtypes;/*
@@ -232,6 +233,7 @@ static inline bool cache_was_expanded(struct cache *cache, void *addr)struct expansion_state {bool expand; +const char *current_fqn; };struct state {
@@ -263,4 +265,16 @@ void process_cu(Dwarf_Die *cudie);void generate_symtypes_and_versions (FILE *file); +/* +* kabi.c +*/ + +bool kabi_is_enumerator_ignored (const char *fqn,const char *field); +bool kabi_get_enumerator_value (const char *fqn,const char *field, +unsigned long *value); +bool kabi_is_declonly (const char *fqn); + +void kabi_read_rules (int fd); +void kabi_free (void ); +#endif /* __GENDWARFKSYMS_H */
scripts/gendwarfksyms/kabi.c new file +336 -0
336 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 14/18 gendwarfksyms: Add support for reserved and ignored fields
In reply to Sami TolvanenDistributions that want to maintain a stable kABI need the ability to make ABI compatible changes to kernel data structures without affecting symbol versions, either because of LTS updates or backports.
With genksyms, developers would typically hide these changes from version calculation with #ifndef __GENKSYMS__, which would result in the symbol version not changing even though the actual type has changed. When we process precompiled object files, this isn't an option.
Change union processing to recognize field name prefixes that allow the user to ignore the union completely during symbol versioning with a __kabi_ignored prefix in a field name, or to replace the type of a placeholder field using a __kabi_reserved field name prefix.
For example, assume we want to add a new field to an existing alignment hole in a data structure, and ignore the new field when calculating symbol versions:
struct struct1 {
int a;
/* a 4-byte alignment hole */
unsigned long b;
};To add int n to the alignment hole, we can add a union that includes a __kabi_ignored field that causes gendwarfksyms to ignore the entire union:
struct struct1 {
int a;
union {
char __kabi_ignored_0;
int n;
};
unsigned long b;
};With --stable, both structs produce the same symbol version.
Alternatively, when a distribution expects future modification to a data structure, they can explicitly add reserved fields:
struct struct2 {
long a;
long __kabi_reserved_0; /* reserved for future use */
};To take the field into use, we can again replace it with a union, with one of the fields keeping the __kabi_reserved name prefix to indicate the original type:
struct struct2 {
long a;
union {
long __kabi_reserved_0;
struct {
int b;
int v;
};
};Here gendwarfksyms --stable replaces the union with the type of the placeholder field when calculating versions.
--- scripts/gendwarfksyms/dwarf.c | 248 ++++++++++++++++++++++- scripts/gendwarfksyms/examples/kabi.h | 87 ++++++++ scripts/gendwarfksyms/examples/kabi_ex.c | 16 ++ scripts/gendwarfksyms/examples/kabi_ex.h | 199 ++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 9 + 5 files changed, 558 insertions(+), 1 deletion(-)
scripts/gendwarfksyms/dwarf.c +247 -1
@@ -3,10 +3,33 @@* Copyright (C) 2024 Google LLC */ +#include <assert.h> #include <inttypes.h> #include <stdarg.h> #include "gendwarfksyms.h" +/* See get_union_kabi_status */ +#define KABI_PREFIX "__kabi_" +#define KABI_PREFIX_LEN (sizeof(KABI_PREFIX) - 1) +#define KABI_RESERVED_PREFIX "reserved" +#define KABI_RESERVED_PREFIX_LEN (sizeof(KABI_RESERVED_PREFIX) - 1) +#define KABI_RENAMED_PREFIX "renamed" +#define KABI_RENAMED_PREFIX_LEN (sizeof(KABI_RENAMED_PREFIX) - 1) +#define KABI_IGNORED_PREFIX "ignored" +#define KABI_IGNORED_PREFIX_LEN (sizeof(KABI_IGNORED_PREFIX) - 1) + +static inline bool is_kabi_prefix (const char *name) +{ +return name && !strncmp (name,KABI_PREFIX ,KABI_PREFIX_LEN ); +} + +enum kabi_status { +/* >0 to stop DIE processing */ +KABI_NORMAL =1 , +KABI_RESERVED , +KABI_IGNORED , +}; +static bool do_linebreak;static int indentation_level;
@@ -353,13 +376,23 @@ static void __process_list_type(struct state *state, struct die *cache, {const char *name =get_name_attr (die); +if (stable) { +if (is_kabi_prefix (name)) + name =NULL ; + state->kabi.orig_name =NULL ; + } +process_list_comma (state, cache);process (cache, type);process_type_attr (state, cache, die); + +if (stable && state->kabi.orig_name) + name = state->kabi.orig_name;if (name) {process (cache," " );process (cache, name); } +process_accessibility_attr (cache, die);process_bit_size_attr (cache, die);process_data_bit_offset_attr (cache, die);
@@ -486,11 +519,208 @@ static void process_variant_part_type(struct state *state, struct die *cache,process (cache,"}" ); } +static int get_kabi_status (Dwarf_Die *die,const char **suffix) +{ +const char *name =get_name_attr (die); + +if (suffix) + *suffix =NULL ; + +if (is_kabi_prefix (name)) { + name +=KABI_PREFIX_LEN ; + +if (!strncmp (name,KABI_RESERVED_PREFIX , +KABI_RESERVED_PREFIX_LEN )) +return KABI_RESERVED ; +if (!strncmp (name,KABI_IGNORED_PREFIX , +KABI_IGNORED_PREFIX_LEN )) +return KABI_IGNORED ; + +if (!strncmp (name,KABI_RENAMED_PREFIX , +KABI_RENAMED_PREFIX_LEN )) { +if (suffix) { + name +=KABI_RENAMED_PREFIX_LEN ; + *suffix = name; + } +return KABI_RESERVED ; + } + } + +return KABI_NORMAL ; +} + +static int check_struct_member_kabi_status (struct state *state, +struct die *__unused,Dwarf_Die *die) +{ +int res; + +assert (dwarf_tag (die) == DW_TAG_member_type); + +/* +* If the union member is a struct, expect the __kabi field to +* be the first member of the structure, i.e..: +* +* union { +* type new_member; +* struct { +* type __kabi_field; +* } +* }; +*/ + res =get_kabi_status (die, &state->kabi.orig_name); + +if (res ==KABI_RESERVED && + !get_ref_die_attr (die, DW_AT_type, &state->kabi.placeholder)) +error ("structure member missing a type?" ); + +return res; +} + +static int check_union_member_kabi_status (struct state *state, +struct die *__unused,Dwarf_Die *die) +{ +Dwarf_Die type; +int res; + +assert (dwarf_tag (die) == DW_TAG_member_type); + +if (!get_ref_die_attr (die, DW_AT_type, &type)) +error ("union member missing a type?" ); + +/* +* We expect a union with two members. Check if either of them +* has a __kabi name prefix, i.e.: +* +* union { +* ... +* type memberN; // <- type, N = {0,1} +* ... +* }; +* +* The member can also be a structure type, in which case we'll +* check the first structure member. +* +* In any case, stop processing after we've seen two members. +*/ + res =get_kabi_status (die, &state->kabi.orig_name); + +if (res ==KABI_RESERVED ) + state->kabi.placeholder = type; +if (res !=KABI_NORMAL ) +return res; + +if (dwarf_tag (&type) == DW_TAG_structure_type) + res =checkp (process_die_container ( + state,NULL , &type, check_struct_member_kabi_status, + match_member_type)); + +if (res <=KABI_NORMAL && ++state->kabi.members <2 ) +return 0 ;/* Continue */ + +return res; +} + +static int get_union_kabi_status (Dwarf_Die *die,Dwarf_Die *placeholder, +const char **orig_name) +{ +struct state state; +int res; + +if (!stable) +return KABI_NORMAL ; + +/* +* To maintain a stable kABI, distributions may choose to reserve +* space in structs for later use by adding placeholder members, +* for example: +* +* struct s { +* u32 a; +* // an 8-byte placeholder for future use +* u64 __kabi_reserved_0; +* }; +* +* When the reserved member is taken into use, the type change +* would normally cause the symbol version to change as well, but +* if the replacement uses the following convention, gendwarfksyms +* continues to use the placeholder type for versioning instead, +* thus maintaining the same symbol version: +* +* struct s { +* u32 a; +* union { +* // placeholder replaced with a new member `b` +* struct t b; +* struct { +* // the placeholder type that is still +* // used for versioning +* u64 __kabi_reserved_0; +* }; +* }; +* }; +* +* I.e., as long as the replaced member is in a union, and the +* placeholder has a __kabi_reserved name prefix, we'll continue +* to use the placeholder type (here u64) for version calculation +* instead of the union type. +* +* It's also possible to ignore new members from versioning if +* they've been added to alignment holes, for example, by +* including them in a union with another member that uses the +* __kabi_ignored name prefix: +* +* struct s { +* u32 a; +* // an alignment hole is used to add `n` +* union { +* u32 n; +* // hide the entire union member from versioning +* u8 __kabi_ignored_0; +* }; +* u64 b; +* }; +* +* Note that the user of this feature is responsible for ensuring +* that the structure actually remains ABI compatible. +*/ +memset (&state.kabi,0 ,sizeof (struct kabi_state )); + + res =checkp (process_die_container (&state,NULL , die, + check_union_member_kabi_status, + match_member_type)); + +if (res ==KABI_RESERVED ) { +if (placeholder) + *placeholder = state.kabi.placeholder; +if (orig_name) + *orig_name = state.kabi.orig_name; + } + +return res; +} + +static bool is_kabi_ignored (Dwarf_Die *die) +{ +Dwarf_Die type; + +if (!stable) +return false; + +if (!get_ref_die_attr (die, DW_AT_type, &type)) +error ("member missing a type?" ); + +return dwarf_tag (&type) == DW_TAG_union_type && +checkp (get_union_kabi_status (&type,NULL ,NULL )) ==KABI_IGNORED ; +} +static int ___process_structure_type (struct state *state,struct die *cache,Dwarf_Die *die) {switch (dwarf_tag (die)) {case DW_TAG_member: +if (is_kabi_ignored (die)) +return 0 ; +return check (process_type (state, cache, die));case DW_TAG_variant_part:return check (process_type (state, cache, die));case DW_TAG_class_type:
@@ -547,7 +777,23 @@ static void __process_structure_type(struct state *state, struct die *cache, DEFINE_PROCESS_STRUCTURE_TYPE(class) DEFINE_PROCESS_STRUCTURE_TYPE(structure) -DEFINE_PROCESS_STRUCTURE_TYPE(union) + +static void process_union_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + Dwarf_Die placeholder; + + int res = checkp(get_union_kabi_status(die, &placeholder, + &state->kabi.orig_name)); + + if (res == KABI_RESERVED) + check(process_type(state, cache, &placeholder)); + if (res > KABI_NORMAL) + return; + + __process_structure_type(state, cache, die, "union_type", + ___process_structure_type, match_all); +} static void process_enumerator_type(struct state *state, struct die *cache, Dwarf_Die *die)
scripts/gendwarfksyms/examples/kabi.h +87 -0
97 lines not on this page · raw message
scripts/gendwarfksyms/examples/kabi_ex.c +16 -0
@@ -12,3 +12,19 @@struct s e0;enum e e1; + +struct ex0a ex0a; +struct ex0b ex0b; +struct ex0c ex0c; + +struct ex1a ex1a; +struct ex1b ex1b; +struct ex1c ex1c; + +struct ex2a ex2a; +struct ex2b ex2b; +struct ex2c ex2c; + +struct ex3a ex3a; +struct ex3b ex3b; +struct ex3c ex3c;
scripts/gendwarfksyms/examples/kabi_ex.h +199 -0
205 lines not on this page · raw message
scripts/gendwarfksyms/gendwarfksyms.h +9 -0
@@ -236,6 +236,12 @@ struct expansion_state {const char *current_fqn; }; +struct kabi_state { +int members; +Dwarf_Die placeholder; +const char *orig_name; +}; +struct state {struct symbol *sym;Dwarf_Die die;
@@ -246,6 +252,9 @@ struct state {/* Structure expansion */ struct expansion_state expand;struct cache expansion_cache; + +/* Reserved or ignored members */ +struct kabi_state kabi; }; typedef int (*die_callback_t)(struct state *state, struct die *cache,
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 15/18 gendwarfksyms: Add support for symbol type pointers
In reply to Sami TolvanenThe compiler may choose not to emit type information in DWARF for external symbols. Clang, for example, does this for symbols not defined in the current TU.
To provide a way to work around this issue, add support for __gendwarfksyms_ptr_<symbol> pointers that force the compiler to emit the necessary type information in DWARF also for the missing symbols.
Example usage:
#define GENDWARFKSYMS_PTR(sym) \
static typeof(sym) *__gendwarfksyms_ptr_##sym __used \
__section(".discard.gendwarfksyms") = &sym;
extern int external_symbol(void);
GENDWARFKSYMS_PTR(external_symbol);--- scripts/gendwarfksyms/dwarf.c | 55 +++++++++++++++++++++- scripts/gendwarfksyms/examples/symbolptr.c | 33 +++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 7 +++ scripts/gendwarfksyms/symbols.c | 27 +++++++++++ 4 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c
scripts/gendwarfksyms/dwarf.c +54 -1
@@ -1061,6 +1061,31 @@ static void process_variable(struct state *state, Dwarf_Die *die)process_symbol (state, die, __process_variable); } +static void save_symbol_ptr (struct state *state) +{ +Dwarf_Die ptr_type; +Dwarf_Die type; + +if (!get_ref_die_attr (&state->die, DW_AT_type, &ptr_type) || +dwarf_tag (&ptr_type) != DW_TAG_pointer_type) +error ("%s must be a pointer type!" , +get_symbol_name (&state->die)); + +if (!get_ref_die_attr (&ptr_type, DW_AT_type, &type)) +error ("%s pointer missing a type attribute?" , +get_symbol_name (&state->die)); + +/* +* Save the symbol pointer DIE in case the actual symbol is +* missing from the DWARF. Clang, for example, intentionally +* omits external symbols from the debugging information. +*/ +if (dwarf_tag (&type) == DW_TAG_subroutine_type) +symbol_set_ptr (state->sym, &type); +else +symbol_set_ptr (state->sym, &ptr_type); +} +static int process_exported_symbols (struct state *unused,struct die *cache,Dwarf_Die *die) {
@@ -1084,7 +1109,9 @@ static int process_exported_symbols(struct state *unused, struct die *cache,state_init (&state); -if (tag == DW_TAG_subprogram) +if (is_symbol_ptr (get_symbol_name (&state.die))) +save_symbol_ptr (&state); +else if (tag == DW_TAG_subprogram)process_subprogram (&state, &state.die);else process_variable (&state, &state.die);
@@ -1097,10 +1124,36 @@ static int process_exported_symbols(struct state *unused, struct die *cache, } } +static void process_symbol_ptr (struct symbol *sym,void *arg) +{ +struct state state; +Dwarf *dwarf = arg; + +if (sym->state !=SYMBOL_UNPROCESSED || !sym->ptr_die_addr) +return ; + +debug ("%s" , sym->name); +state_init (&state); + state.sym = sym; + +if (!dwarf_die_addr_die (dwarf, (void *)sym->ptr_die_addr, &state.die)) +error ("dwarf_die_addr_die failed for symbol ptr: '%s'" , + sym->name); + +if (dwarf_tag (&state.die) == DW_TAG_subroutine_type) +process_subprogram (&state, &state.die); +else +process_variable (&state, &state.die); + +cache_free (&state.expansion_cache); +} +void process_cu (Dwarf_Die *cudie) {check (process_die_container (NULL ,NULL , cudie, process_exported_symbols, match_all)); +symbol_for_each (process_symbol_ptr,dwarf_cu_getdwarf (cudie->cu)); +cache_free (&srcfile_cache); }
scripts/gendwarfksyms/examples/symbolptr.c new file +33 -0
@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (C) 2024 Google LLC +* +* Example for symbol pointers. When compiled with Clang, gendwarfkyms +* uses a symbol pointer for `f`. +* +* $ clang -g -c examples/symbolptr.c -o examples/symbolptr.o +* $ echo -e "f\ng\np" | ./gendwarfksyms -d examples/symbolptr.o +*/ + +/* Kernel macros for userspace testing. */ +#ifndef __used +#define __used __attribute__((__used__)) +#endif +#ifndef __section +#define __section (section) __attribute__((__section__(section))) +#endif + +#define __GENDWARFKSYMS_EXPORT (sym) \ + static typeof(sym) *__gendwarfksyms_ptr_##sym __used \ + __section(".discard.gendwarfksyms") = &sym; + +extern void f (unsigned int arg); +void g (int *arg); +void g (int *arg) {} + +struct s ; +extern struct s *p; + +__GENDWARFKSYMS_EXPORT (f); +__GENDWARFKSYMS_EXPORT (g); +__GENDWARFKSYMS_EXPORT (p);
scripts/gendwarfksyms/gendwarfksyms.h +7 -0
@@ -89,6 +89,10 @@ extern int symtypes;* symbols.c */ +/* See symbols.c:is_symbol_ptr */ +#define SYMBOL_PTR_PREFIX "__gendwarfksyms_ptr_" +#define SYMBOL_PTR_PREFIX_LEN (sizeof(SYMBOL_PTR_PREFIX) - 1) +static inline unsigned int addr_hash (uintptr_t addr) {return hash_ptr ((const void *)addr);
@@ -112,14 +116,17 @@ struct symbol {struct hlist_node name_hash;enum symbol_state state;uintptr_t die_addr; +uintptr_t ptr_die_addr;unsigned long crc; };typedef void (*symbol_callback_t )(struct symbol *,void *arg); +bool is_symbol_ptr (const char *name);void symbol_read_exports (FILE *file);void symbol_read_symtab (int fd);struct symbol *symbol_get (const char *name); +void symbol_set_ptr (struct symbol *sym,Dwarf_Die *ptr);void symbol_set_die (struct symbol *sym,Dwarf_Die *die);void symbol_set_crc (struct symbol *sym,unsigned long crc);void symbol_for_each (symbol_callback_t func,void *arg);
scripts/gendwarfksyms/symbols.c +27 -0
@@ -39,6 +39,20 @@ static unsigned int __for_each_addr(struct symbol *sym, symbol_callback_t func,return processed; } +/* +* For symbols without debugging information (e.g. symbols defined in other +* TUs), we also match __gendwarfksyms_ptr_<symbol_name> symbols, which the +* kernel uses to ensure type information is present in the TU that exports +* the symbol. A __gendwarfksyms_ptr pointer must have the same type as the +* exported symbol, e.g.: +* +* typeof(symname) *__gendwarf_ptr_symname = &symname; +*/ +bool is_symbol_ptr (const char *name) +{ +return name && !strncmp (name,SYMBOL_PTR_PREFIX ,SYMBOL_PTR_PREFIX_LEN ); +} +static unsigned int for_each (const char *name,symbol_callback_t func,void *data) {
@@ -47,6 +61,8 @@ static unsigned int for_each(const char *name, symbol_callback_t func,if (!name || !*name)return 0 ; +if (is_symbol_ptr (name)) + name +=SYMBOL_PTR_PREFIX_LEN ;hash_for_each_possible_safe (symbol_names, match, tmp, name_hash,hash_str (name)) {
@@ -84,6 +100,17 @@ void symbol_set_crc(struct symbol *sym, unsigned long crc)error ("no matching symbols: '%s'" , sym->name); } +static void set_ptr (struct symbol *sym,void *data) +{ + sym->ptr_die_addr = (uintptr_t )((Dwarf_Die *)data)->addr; +} + +void symbol_set_ptr (struct symbol *sym,Dwarf_Die *ptr) +{ +if (for_each (sym->name, set_ptr, ptr) ==0 ) +error ("no matching symbols: '%s'" , sym->name); +} +static void set_die (struct symbol *sym,void *data) { sym->die_addr = (uintptr_t )((Dwarf_Die *)data)->addr;
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 16/18 export: Add __gendwarfksyms_ptr_ references to exported symbols
In reply to Sami TolvanenWith gendwarfksyms, we need each TU where the EXPORT_SYMBOL() macro is used to also contain DWARF type information for the symbols it exports. However, as a TU can also export external symbols and compilers may choose not to emit debugging information for symbols not defined in the current TU, the missing types will result in missing symbol versions. Stand-alone assembly code also doesn't contain type information for exported symbols, so we need to compile a temporary object file with asm-prototypes.h instead, and similarly need to ensure the DWARF in the temporary object file contains the necessary types.
To always emit type information for external exports, add explicit __gendwarfksyms_ptr_<symbol> references to them in EXPORT_SYMBOL(). gendwarfksyms will use the type information for __gendwarfksyms_ptr_* if needed. Discard the pointers from the final binary to avoid further bloat.
--- include/linux/export.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
include/linux/export.h +15 -0
@@ -52,9 +52,24 @@#else +#ifdef CONFIG_GENDWARFKSYMS +/* +* With CONFIG_GENDWARFKSYMS, ensure the compiler emits debugging +* information for all exported symbols, including those defined in +* different TUs, by adding a __gendwarfksyms_ptr_<symbol> pointer +* that's discarded during the final link. +*/ +#define __GENDWARFKSYMS_EXPORT (sym) \ + static typeof(sym) *__gendwarfksyms_ptr_##sym __used \ + __section(".discard.gendwarfksyms") = &sym; +#else +#define __GENDWARFKSYMS_EXPORT (sym) +#endif +#define __EXPORT_SYMBOL (sym, license, ns) \ extern typeof(sym) sym; \ __ADDRESSABLE(sym) \ + __GENDWARFKSYMS_EXPORT(sym) \ asm(__stringify(___EXPORT_SYMBOL(sym, license, ns)))#endif
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 17/18 kbuild: Add gendwarfksyms as an alternative to genksyms
In reply to Sami TolvanenWhen MODVERSIONS is enabled, allow selecting gendwarfksyms as the implementation, but default to genksyms.
--- kernel/module/Kconfig | 22 ++++++++++++++++++++++ scripts/Makefile | 2 +- scripts/Makefile.build | 35 +++++++++++++++++++++++++++++------ 3 files changed, 52 insertions(+), 7 deletions(-)
kernel/module/Kconfig +22 -0
@@ -169,6 +169,22 @@ config MODVERSIONS make them incompatible with the kernel you are running. If unsure, say N. +choice + prompt "Module versioning implementation" + depends on MODVERSIONS + help + Select the tool used to calculate symbol versions for modules. + + If unsure, select GENKSYMS. + +config GENKSYMS + bool "genksyms (from source code)" + help + Calculate symbol versions from pre-processed source code using + genksyms. + + If unsure, say Y. + config GENDWARFKSYMS bool "gendwarfksyms (from debugging information)" depends on DEBUG_INFO
@@ -176,6 +192,12 @@ config GENDWARFKSYMS depends on !DEBUG_INFO_REDUCED && !DEBUG_INFO_SPLIT # Requires ELF object files. depends on !LTO + help + Calculate symbol versions from DWARF debugging information using + gendwarfksyms. Requires DEBUG_INFO to be enabled. + + If unsure, say N. +endchoice config ASM_MODVERSIONS bool
scripts/Makefile +1 -1
@@ -53,7 +53,7 @@ hostprogs += unifdef targets += module.lds subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins -subdir-$(CONFIG_MODVERSIONS) += genksyms +subdir-$(CONFIG_GENKSYMS) += genksyms subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms subdir-$(CONFIG_SECURITY_SELINUX) += selinux subdir-$(CONFIG_SECURITY_IPE) += ipe
scripts/Makefile.build +29 -6
@@ -107,13 +107,24 @@ cmd_cpp_i_c = $(CPP) $(c_flags) -o $@ $< $(obj)/%.i: $(obj)/%.c FORCE $(call if_changed_dep,cpp_i_c) +getexportsymbols = $(NM) $@ | sed -n 's/.* __export_symbol_\(.*\)/$(1)/p' + +gendwarfksyms = $(objtree)/scripts/gendwarfksyms/gendwarfksyms \ + $(if $(KBUILD_SYMTYPES), --symtypes $(@:.o=.symtypes)) \ + $(if $(KBUILD_GENDWARFKSYMS_STABLE), --stable) + genksyms = $(objtree)/scripts/genksyms/genksyms \ $(if $(KBUILD_SYMTYPES), -T $(@:.o=.symtypes)) \ $(if $(KBUILD_PRESERVE), -p) \ $(addprefix -r , $(wildcard $(@:.o=.symref))) # These mirror gensymtypes_S and co below, keep them in synch. +ifdef CONFIG_GENDWARFKSYMS +cmd_gensymtypes_c = $(if $(skip_gendwarfksyms),, \ + $(call getexportsymbols,\1) | $(gendwarfksyms) $@) +else cmd_gensymtypes_c = $(CPP) -D__GENKSYMS__ $(c_flags) $< | $(genksyms) +endif # CONFIG_GENDWARFKSYMS # LLVM assembly # Generate .ll files from .c
@@ -286,14 +297,26 @@ $(obj)/%.rs: $(obj)/%.rs.S FORCE # This is convoluted. The .S file must first be preprocessed to run guards and # expand names, then the resulting exports must be constructed into plain # EXPORT_SYMBOL(symbol); to build our dummy C file, and that gets preprocessed -# to make the genksyms input. +# to make the genksyms input or compiled into an object for gendwarfksyms. # # These mirror gensymtypes_c and co above, keep them in synch. -cmd_gensymtypes_S = \ - { echo "\#include <linux/kernel.h>" ; \ - echo "\#include <asm/asm-prototypes.h>" ; \ - $(NM) $@ | sed -n 's/.* __export_symbol_\(.*\)/EXPORT_SYMBOL(\1);/p' ; } | \ - $(CPP) -D__GENKSYMS__ $(c_flags) -xc - | $(genksyms) +getasmexports = \ + { echo "\#include <linux/kernel.h>" ; \ + echo "\#include <linux/string.h>" ; \ + echo "\#include <asm/asm-prototypes.h>" ; \ + $(call getexportsymbols,EXPORT_SYMBOL(\1);) ; } + +ifdef CONFIG_GENDWARFKSYMS +cmd_gensymtypes_S = \ + $(getasmexports) | \ + $(CC) $(c_flags) -c -o $(@:.o=.gendwarfksyms.o) -xc -; \ + $(call getexportsymbols,\1) | \ + $(gendwarfksyms) $(@:.o=.gendwarfksyms.o) +else +cmd_gensymtypes_S = \ + $(getasmexports) | \ + $(CPP) -D__GENKSYMS__ $(c_flags) -xc - | $(genksyms) +endif # CONFIG_GENDWARFKSYMS quiet_cmd_cpp_s_S = CPP $(quiet_modtag) $@ cmd_cpp_s_S = $(CPP) $(a_flags) -o $@ $<
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Sami Tolvanen
maintainerPATCH v8 18/18 Documentation/kbuild: Add DWARF module versioning
In reply to Sami TolvanenAdd documentation for gendwarfksyms changes, and the kABI stability features that can be useful for distributions even though they're not used in mainline kernels.
--- Documentation/kbuild/gendwarfksyms.rst | 308 +++++++++++++++++++++++++ Documentation/kbuild/index.rst | 1 + 2 files changed, 309 insertions(+) create mode 100644 Documentation/kbuild/gendwarfksyms.rst
Documentation/kbuild/gendwarfksyms.rst new file +308 -0
@@ -0,0 +1,308 @@ +======================= +DWARF module versioning +======================= + +1. Introduction +=============== + +When CONFIG_MODVERSIONS is enabled, symbol versions for modules +are typically calculated from preprocessed source code using the +**genksyms** tool. However, this is incompatible with languages such +as Rust, where the source code has insufficient information about +the resulting ABI. With CONFIG_GENDWARFKSYMS (and CONFIG_DEBUG_INFO) +selected, **gendwarfksyms** is used instead to calculate symbol versions +from the DWARF debugging information, which contains the necessary +details about the final module ABI. + +1.1. Usage +========== + +gendwarfksyms accepts a list of object files on the command line, and a +list of symbol names (one per line) in standard input:: + + Usage: gendwarfksyms [options] elf-object-file ... < symbol-list + + Options: + -d, --debug Print debugging information + --dump-dies Dump DWARF DIE contents + --dump-die-map Print debugging information about die_map changes + --dump-types Dump type strings + --dump-versions Dump expanded type strings used for symbol versions + -s, --stable Support kABI stability features + -T, --symtypes file Write a symtypes file + -h, --help Print this message + + +2. Type information availability +================================ + +While symbols are typically exported in the same translation unit (TU) +where they're defined, it's also perfectly fine for a TU to export +external symbols. For example, this is done when calculating symbol +versions for exports in stand-alone assembly code. + +To ensure the compiler emits the necessary DWARF type information in the +TU where symbols are actually exported, gendwarfksyms adds a pointer +to exported symbols in the `EXPORT_SYMBOL()` macro using the following +macro:: + + #define __GENDWARFKSYMS_EXPORT(sym) \ + static typeof(sym) *__gendwarfksyms_ptr_##sym __used \ + __section(".discard.gendwarfksyms") = &sym; + + +When a symbol pointer is found in DWARF, gendwarfksyms can use its +type for calculating symbol versions even if the symbol is defined +elsewhere. The name of the symbol pointer is expected to start with +`__gendwarfksyms_ptr_`, followed by the name of the exported symbol. + +3. Symtypes output format +========================= + +Similarly to genksyms, gendwarfksyms supports writing a symtypes +file for each processed object that contain types for exported +symbols and each referenced type that was used in calculating symbol +versions. These files can be useful when trying to determine what +exactly caused symbol versions to change between builds. To generate +symtypes files during a kernel build, set `KBUILD_SYMTYPES=1`. + +Matching the existing format, the first column of each line contains +either a type reference or a symbol name. Type references have a +one-letter prefix followed by "#" and the name of the type. Four +reference types are supported:: + + e#<type> = enum + s#<type> = struct + t#<type> = typedef + u#<type> = union + +Type names with spaces in them are wrapped in single quotes, e.g.:: + + s#'core::result::Result<u8, core::num::error::ParseIntError>' + +The rest of the line contains a type string. Unlike with genksyms that +produces C-style type strings, gendwarfksyms uses the same simple parsed +DWARF format produced by **--dump-dies**, but with type references +instead of fully expanded strings. + +4. Maintaining a stable kABI +============================ + +Distribution maintainers often need the ability to make ABI compatible +changes to kernel data structures due to LTS updates or backports. Using +the traditional `#ifndef __GENKSYMS__` to hide these changes from symbol +versioning won't work when processing object files. To support this +use case, gendwarfksyms provides kABI stability features designed to +hide changes that won't affect the ABI when calculating versions. These +features are all gated behind the **--stable** command line flag and are +not used in the mainline kernel. To use stable features during a kernel +build, set `KBUILD_GENDWARFKSYMS_STABLE=1`. + +Examples for using these features are provided in the +**scripts/gendwarfksyms/examples** directory, including helper macros +for source code annotation. Note that as these features are only used to +transform the inputs for symbol versioning, the user is responsible for +ensuring that their changes actually won't break the ABI. + +4.1. kABI rules +=============== + +kABI rules allow distributions to fine-tune certain parts +of gendwarfksyms output and thus control how symbol +versions are calculated. These rules are defined in the +`.discard.gendwarfksyms.kabi_rules` section of the object file and +consist of simple null-terminated strings with the following structure:: + + version\0type\0target\0value\0 + +This string sequence is repeated as many times as needed to express all +the rules. The fields are as follows: + +- `version`: Ensures backward compatibility for future changes to the + structure. Currently expected to be "1". +- `type`: Indicates the type of rule being applied. +- `target`: Specifies the target of the rule, typically the fully + qualified name of the DWARF Debugging Information Entry (DIE). +- `value`: Provides rule-specific data. + +The following helper macro, for example, can be used to specify rules +in the source code:: + + #define __KABI_RULE(hint, target, value) \ + static const char __PASTE(__gendwarfksyms_rule_, \ + __COUNTER__)[] __used __aligned(1) \ + __section(".discard.gendwarfksyms.kabi_rules") = \ + "1\0" #hint "\0" #target "\0" #value + + +Currently, only the rules discussed in this section are supported, but +the format is extensible enough to allow further rules to be added as +need arises. + +4.1.1. Managing definition visibility +===================================== + +A declaration can change into a full definition when additional includes +are pulled into the translation unit. This changes the versions of any +symbol that references the type even if the ABI remains unchanged. As +it may not be possible to drop includes without breaking the build, the +`declonly` rule can be used to specify a type as declaration-only, even +if the debugging information contains the full definition. + +The rule fields are expected to be as follows: + +- `type`: "declonly" +- `target`: The fully qualified name of the target data structure + (as shown in **--dump-dies** output). +- `value`: This field is ignored. + +Using the `__KABI_RULE` macro, this rule can be defined as:: + + #define KABI_DECLONLY(fqn) __KABI_RULE(declonly, fqn, ) + +Example usage:: + + struct s { + /* definition */ + }; + + KABI_DECLONLY(s); + +4.1.2. Adding enumerators +========================= + +For enums, all enumerators and their values are included in calculating +symbol versions, which becomes a problem if we later need to add more +enumerators without changing symbol versions. The `enumerator_ignore` +rule allows us to hide named enumerators from the input. + +The rule fields are expected to be as follows: + +- `type`: "enumerator_ignore" +- `target`: The fully qualified name of the target enum + (as shown in **--dump-dies** output) and the name of the + enumerator field separated by a space. +- `value`: This field is ignored. + +Using the `__KABI_RULE` macro, this rule can be defined as:: + + #define KABI_ENUMERATOR_IGNORE(fqn, field) \ + __KABI_RULE(enumerator_ignore, fqn field, ) + +Example usage:: + + enum e { + A, B, C, D, + }; + + KABI_ENUMERATOR_IGNORE(e, B); + KABI_ENUMERATOR_IGNORE(e, C); + +If the enum additionally includes an end marker and new values must +be added in the middle, we may need to use the old value for the last +enumerator when calculating versions. The `enumerator_value` rule allows +us to override the value of an enumerator for version calculation: + +- `type`: "enumerator_value" +- `target`: The fully qualified name of the target enum + (as shown in **--dump-dies** output) and the name of the + enumerator field separated by a space. +- `value`: Integer value used for the field. + +Using the `__KABI_RULE` macro, this rule can be defined as:: + + #define KABI_ENUMERATOR_VALUE(fqn, field, value) \ + __KABI_RULE(enumerator_value, fqn field, value) + +Example usage:: + + enum e { + A, B, C, LAST, + }; + + KABI_ENUMERATOR_IGNORE(e, C); + KABI_ENUMERATOR_VALUE(e, LAST, 2); + +4.3. Adding structure members +============================= + +Perhaps the most common ABI compatible change is adding a member to a +kernel data structure. When changes to a structure are anticipated, +distribution maintainers can pre-emptively reserve space in the +structure and take it into use later without breaking the ABI. If +changes are needed to data structures without reserved space, existing +alignment holes can potentially be used instead. While kABI rules could +be added for these type of changes, using unions is typically a more +natural method. This section describes gendwarfksyms support for using +reserved space in data structures and hiding members that don't change +the ABI when calculating symbol versions. + +4.3.1. Reserving space and replacing members +============================================ + +Space is typically reserved for later use by appending integer types, or +arrays, to the end of the data structure, but any type can be used. Each +reserved member needs a unique name, but as the actual purpose is usually +not known at the time the space is reserved, for convenience, names that +start with `__kabi_` are left out when calculating symbol versions:: + + struct s { + long a; + long __kabi_reserved_0; /* reserved for future use */ + }; + +The reserved space can be taken into use by wrapping the member in a +union, which includes the original type and the replacement member:: + + struct s { + long a; + union { + long __kabi_reserved_0; /* original type */ + struct b b; /* replaced field */ + }; + }; + +If the `__kabi_` naming scheme was used when reserving space, the name +of the first member of the union must start with `__kabi_reserved`. This +ensures the original type is used when calculating versions, but the name +is again left out. The rest of the union is ignored. + +If we're replacing a member that doesn't follow this naming convention, +we also need to preserve the original name to avoid changing versions, +which we can do by changing the first union member's name to start with +`__kabi_renamed` followed by the original name. + +The examples include `KABI_(RESERVE|USE|REPLACE)*` macros that help +simplify the process and also ensure the replacement member is correctly +aligned and its size won't exceed the reserved space. + +4.3.2. Hiding members +===================== + +Predicting which structures will require changes during the support +timeframe isn't always possible, in which case one might have to resort +to placing new members into existing alignment holes:: + + struct s { + int a; + /* a 4-byte alignment hole */ + unsigned long b; + }; + + +While this won't change the size of the data structure, one needs to +be able to hide the added members from symbol versioning. Similarly +to reserved fields, this can be accomplished by wrapping the added +member to a union where one of the fields has a name starting with +`__kabi_ignored`:: + + struct s { + int a; + union { + char __kabi_ignored_0; + int n; + }; + unsigned long b; + }; + +With **--stable**, both versions produce the same symbol version.
Documentation/kbuild/index.rst +1 -0
@@ -21,6 +21,7 @@ Kernel Build System reproducible-builds gcc-plugins llvm + gendwarfksyms .. only:: subproject and html
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Quoted 41 lines · Sami Tolvanen
Hi,
Here's v8 of the DWARF modversions series. The main motivation is modversions support for Rust, which is important for distributions like Android that are about to ship Rust kernel modules. Per Luis' request [1], v2 dropped the Rust specific bits from the series and instead added the feature as an option for the entire kernel to make it easier to evaluate the benefits of this approach, and to get better test coverage. Matt is addressing Rust
modversion_infocompatibility issues in a separate patch set [2] that depends on this series, and actually allows modversions to be enabled with Rust.Short background: Unlike C, Rust source code doesn't have sufficient information about the final ABI, as the compiler has considerable freedom in adjusting structure layout, for example, which makes using a source code parser like genksyms a non-starter. Based on earlier feedback, this series uses DWARF debugging information for computing versions. DWARF is an established and a relatively stable format, which includes all the necessary ABI details, and adding a
CONFIG_DEBUG_INFOdependency for Rust symbol versioning seems like a reasonable trade-off as most distributions already enable it.The first 15 patches add gendwarfksyms, a tool for computing symbol versions from DWARF. When passed a list of exported symbols and object files, the tool generates an expanded type string for each symbol and computes symbol versions. gendwarfksyms is written in C, uses libdw to process DWARF, and zlib for CRC32. Patch 16 ensures that debugging information is present where we need it, patch 17 adds gendwarfksyms as an alternative to genksyms, and the last patch adds documentation.
v8 is based on 6.13-rc5, and for your convenience, the series is also available here:
https://github.com/samitolvanen/linux/commits/gendwarfksyms-v8
If you also want to test the series with actual Rust modules, this branch adds v13 of Matt's
modversion_infoseries:
Quoted 8 lines
https://github.com/samitolvanen/linux/commits/rustmodversions-v8
Sami
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/
Applied to linux-kbuild.
Thanks.
Signature · 2 lines
Best Regards Masahiro Yamada