Improve objtool jump table handling
Subsystems (3)
v3 cover Improve objtool jump table handling
From: Ard Biesheuvel <[email protected]>
Jump table handling has faded into the background a little due to the fact that jump tables are [currently] disabled when enabling retpoline mitigations and/or IBT on x86.
However, this is likely to come back and bite us later, so it still needs to be addressed. Given the difficulty in identifying jump tables from .rodata references and indirect jump instructions that often have no obvious correlation, it would be better to do this in the compiler.
This series implements [on the objtool side] the suggestion made at GNU Cauldron this year to annotate the indirect jump with a R_X86_64_NONE relocation that refers to the jump table, and ensure that it is covered by a STT_OBJECT symbol whose size accurately reflects the size of the jump table.
This can be wired up in objtool with minimal effort. The only complication is that indirect jumps may be direct jumps in disguise, if they target retpoline thunks. This will result in more than one relocation attached to the same instruction, which needs careful handling in objtool.
Other than that, changes are rather straight-forward.
Patches #6 - #8 update the CRC32C driver, which has a jump table implemented in assembler, to
a) use a relative jump table, for compatibility with linking in PIE mode
b) add the jump table annotation
c) make the jump table more difficult to identify by objtool's existing heuristics, so that it will fail to identify it without the annotation.
Changes since v2:
- drastic refactoring of the annotation handling so that generic users (non-x86) get it as well, with the x86 heuristics moved to a x86 specific source file
- use generic reloc type identifiers where appropriate
- update insn->no_reloc where appropriate
Changes since v1:
- tweak logic in patch #1 to ensure that all jump table entries are covered by the same type of relocation
- use the corrected addend when validating IBT targets
- add patches #2 - #5
Ard Biesheuvel (8):
objtool: Deal with relative jump tables correctly
objtool: Allow arch code to discover jump table size
objtool: Make some helper functions globally accessible
objtool: Move jump table heuristics to a x86 specific source file
objtool: Add generic support for jump table annotations
crypto: x86/crc32c - Use idiomatic relative jump table
crypto: x86/crc32c - Add jump table annotation
crypto: x86/crc32c-intel - Tweaks to make objtool's life harder
arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 50 +++-- tools/objtool/arch/loongarch/special.c | 6 - tools/objtool/arch/powerpc/special.c | 6 - tools/objtool/arch/x86/special.c | 160 ++++++++++++---- tools/objtool/check.c | 199 ++++++++++---------- tools/objtool/include/objtool/check.h | 25 ++- tools/objtool/include/objtool/elf.h | 6 + tools/objtool/include/objtool/special.h | 8 +- 8 files changed, 287 insertions(+), 173 deletions(-)
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
PATCH v3 1/8 objtool: Deal with relative jump tables correctly
From: Ard Biesheuvel <[email protected]>
Relative jump tables contain entries that carry the offset between the target of the jump and the start of the jump table. This permits the use of the PIC idiom of
leaq jump_table(%rip), %tbl
movslq (%tbl,%idx,4), %offset
addq %offset, %tbl
jmp *%tblThe jump table entries are decorated with PC32 relocations, which record the offset of the referenced symbol relative to the target of the relocation, which is the individual entry in the table. This means that only the first entry produces the correct value directly; the subsequent ones need to be corrected to produce the offset relative to the start of the table, by applying an addend.
Given that the referenced symbols are anonymous, and thus already expressed in terms of sections and addends, e.g., .text+0x5df9, the correction is incorporated into the existing addend. The upshot of this is that chasing the reference to find the target instruction needs to take this second addend into account as well.
--- tools/objtool/arch/x86/special.c | 8 ------- tools/objtool/check.c | 24 +++++++++++++++++--- tools/objtool/include/objtool/elf.h | 6 +++++ 3 files changed, 27 insertions(+), 11 deletions(-)
tools/objtool/arch/x86/special.c +0 -8
@@ -150,13 +150,5 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,if (!rodata_reloc)return NULL ; -/* -* Use of RIP-relative switch jumps is quite rare, and -* indicates a rare GCC quirk/bug which can leave dead -* code behind. -*/ -if (reloc_type (text_reloc) ==R_X86_64_PC32 ) - file->ignore_unreachables = true; -return rodata_reloc; }
tools/objtool/check.c +21 -3
@@ -2101,6 +2101,8 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, {struct symbol *pfunc =insn_func (insn)->pfunc;struct reloc *table =insn_jump_table (insn); +unsigned int rtype =reloc_type (table); +bool pcrel = (rtype ==elf_text_rela_type (file->elf));struct instruction *dest_insn;unsigned int prev_offset =0 ;struct reloc *reloc = table;
@@ -2111,13 +2113,18 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn,* instruction. */ for_each_reloc_from (table->sec, reloc) { +unsigned long addend =reloc_addend (reloc);/* Check for the end of the table: */ if (reloc != table && reloc == next_table)break ; +/* Each entry in the jump table should use the same relocation type */ +if (reloc_type (reloc) != rtype) +break ; +/* Make sure the table entries are consecutive: */ -if (prev_offset &&reloc_offset (reloc) != prev_offset +8 ) +if (prev_offset &&reloc_offset (reloc) != prev_offset + (pcrel ?4 :8 ))break ;/* Detect function pointers from contiguous objects: */
@@ -2125,7 +2132,15 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn, reloc_addend(reloc) == pfunc->offset)break ; - dest_insn =find_insn (file, reloc->sym->sec,reloc_addend (reloc)); +/* +* Place-relative jump tables carry offsets relative to the +* start of the jump table, not to the entry itself. So correct +* the addend for the location of the entry in the table. +*/ +if (pcrel) + addend -=reloc_offset (reloc) -reloc_offset (table); + + dest_insn =find_insn (file, reloc->sym->sec, addend);if (!dest_insn)break ;
@@ -2133,6 +2148,9 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn,if (!insn_func (dest_insn) ||insn_func (dest_insn)->pfunc != pfunc)break ; +if (pcrel) + reloc->sym_offset = addend; + alt =malloc (sizeof (*alt));if (!alt) {WARN ("malloc failed" );
@@ -4535,7 +4553,7 @@ static int validate_ibt_data_reloc(struct objtool_file *file,struct instruction *dest; dest =find_insn (file, reloc->sym->sec, - reloc->sym->offset +reloc_addend (reloc)); + reloc->sym->offset +reloc_sym_offset (reloc));if (!dest)return 0 ;
tools/objtool/include/objtool/elf.h +6 -0
@@ -78,6 +78,7 @@ struct reloc {struct section *sec;struct symbol *sym;struct reloc *sym_next_reloc; +s64 sym_offset; };struct elf {
@@ -251,6 +252,11 @@ static inline s64 reloc_addend(struct reloc *reloc)return __get_reloc_field (reloc, r_addend); } +static inline s64 reloc_sym_offset (struct reloc *reloc) +{ +return reloc->sym_offset ?:reloc_addend (reloc); +} +static inline void set_reloc_addend (struct elf *elf,struct reloc *reloc,s64 addend) {__set_reloc_field (reloc, r_addend, addend);
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
PATCH v3 2/8 objtool: Allow arch code to discover jump table size
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
In preparation for adding support for annotated jump tables, where ELF relocations and symbols are used to describe the locations of jump tables in the executable, refactor the jump table discovery logic so the table size can be returned from arch_find_switch_table().
--- tools/objtool/arch/loongarch/special.c | 3 +- tools/objtool/arch/powerpc/special.c | 3 +- tools/objtool/arch/x86/special.c | 4 ++- tools/objtool/check.c | 31 +++++++++++++------- tools/objtool/include/objtool/check.h | 5 +++- tools/objtool/include/objtool/special.h | 3 +- 6 files changed, 33 insertions(+), 16 deletions(-)
tools/objtool/arch/loongarch/special.c +2 -1
@@ -9,7 +9,8 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, }struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn) +struct instruction *insn, +unsigned long *table_size) {return NULL ; }
tools/objtool/arch/powerpc/special.c +2 -1
@@ -13,7 +13,8 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, }struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn) +struct instruction *insn, +unsigned long *table_size) {exit (-1 ); }
tools/objtool/arch/x86/special.c +3 -1
@@ -109,7 +109,8 @@ bool arch_support_alt_relocation(struct special_alt *special_alt,* NOTE: MITIGATION_RETPOLINE made it harder still to decode dynamic jumps. */ struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn) +struct instruction *insn, +unsigned long *table_size) {struct reloc *text_reloc, *rodata_reloc;struct section *table_sec;
@@ -150,5 +151,6 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,if (!rodata_reloc)return NULL ; + *table_size =0 ;return rodata_reloc; }
tools/objtool/check.c +20 -11
@@ -150,6 +150,15 @@ static inline struct reloc *insn_jump_table(struct instruction *insn)return NULL ; } +static inline unsigned long insn_jump_table_size (struct instruction *insn) +{ +if (insn->type ==INSN_JUMP_DYNAMIC || + insn->type ==INSN_CALL_DYNAMIC ) +return insn->_jump_table_size; + +return 0 ; +} +static bool is_jump_table_jump (struct instruction *insn) {struct alt_group *alt_group = insn->alt_group;
@@ -2099,6 +2108,7 @@ static int add_special_section_alts(struct objtool_file *file)static int add_jump_table (struct objtool_file *file,struct instruction *insn,struct reloc *next_table) { +unsigned long table_size =insn_jump_table_size (insn);struct symbol *pfunc =insn_func (insn)->pfunc;struct reloc *table =insn_jump_table (insn);unsigned int rtype =reloc_type (table);
@@ -2116,6 +2126,8 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn,unsigned long addend =reloc_addend (reloc);/* Check for the end of the table: */ +if (table_size &&reloc_offset (reloc) -reloc_offset (table) >= table_size) +break ;if (reloc != table && reloc == next_table)break ;
@@ -2175,12 +2187,12 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn,* find_jump_table() - Given a dynamic jump, find the switch jump table * associated with it. */ -static struct reloc *find_jump_table (struct objtool_file *file, -struct symbol *func, -struct instruction *insn) +static void find_jump_table (struct objtool_file *file,struct symbol *func, +struct instruction *insn) {struct reloc *table_reloc;struct instruction *dest_insn, *orig_insn = insn; +unsigned long table_size;/* * Backward search using the @first_jump_src links, these help avoid
@@ -2201,17 +2213,17 @@ static struct reloc *find_jump_table(struct objtool_file *file, insn->jump_dest->offset > orig_insn->offset)) break; - table_reloc =arch_find_switch_table (file, insn); + table_reloc =arch_find_switch_table (file, insn, &table_size);if (!table_reloc)continue ; dest_insn =find_insn (file, table_reloc->sym->sec,reloc_addend (table_reloc));if (!dest_insn || !insn_func (dest_insn) ||insn_func (dest_insn)->pfunc != func)continue ; -return table_reloc; + orig_insn->_jump_table = table_reloc; + orig_insn->_jump_table_size = table_size; +break ; } - -return NULL ; }/*
@@ -2222,7 +2234,6 @@ static void mark_func_jump_tables(struct objtool_file *file, struct symbol *func) {struct instruction *insn, *last =NULL ; -struct reloc *reloc;func_for_each_insn (file, func, insn) {if (!last)
@@ -2245,9 +2256,7 @@ static void mark_func_jump_tables(struct objtool_file *file,if (insn->type !=INSN_JUMP_DYNAMIC )continue ; - reloc =find_jump_table (file, func, insn); -if (reloc) - insn->_jump_table = reloc; +find_jump_table (file, func, insn); } }
tools/objtool/include/objtool/check.h +4 -1
@@ -71,7 +71,10 @@ struct instruction {struct instruction *first_jump_src;union {struct symbol *_call_dest; -struct reloc *_jump_table; +struct { +struct reloc *_jump_table; +unsigned long _jump_table_size; + }; };struct alternative *alts;struct symbol *sym;
tools/objtool/include/objtool/special.h +2 -1
@@ -38,5 +38,6 @@ bool arch_support_alt_relocation(struct special_alt *special_alt,struct instruction *insn, struct reloc *reloc);struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn); +struct instruction *insn, +unsigned long *table_size);#endif /* _SPECIAL_H */
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
[tip: objtool/core] objtool: Allow arch code to discover jump table size
The following commit has been merged into the objtool/core branch of tip:
The following commit has been merged into the objtool/core branch of tip:
Commit-ID: c3cb6c158c64dc39838208d51dcd06d1990b371d Gitweb: https://git.kernel.org/tip/c3cb6c158c64dc39838208d51dcd06d1990b371d Author: Ard Biesheuvel <[email protected]> AuthorDate: Fri, 11 Oct 2024 19:08:50 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 02 Dec 2024 12:24:29 +01:00
objtool: Allow arch code to discover jump table size
In preparation for adding support for annotated jump tables, where ELF relocations and symbols are used to describe the locations of jump tables in the executable, refactor the jump table discovery logic so the table size can be returned from arch_find_switch_table().
--- tools/objtool/arch/loongarch/special.c | 3 +- tools/objtool/arch/powerpc/special.c | 3 +- tools/objtool/arch/x86/special.c | 4 ++- tools/objtool/check.c | 31 +++++++++++++++--------- tools/objtool/include/objtool/check.h | 5 +++- tools/objtool/include/objtool/special.h | 3 +- 6 files changed, 33 insertions(+), 16 deletions(-)
tools/objtool/arch/loongarch/special.c +2 -1
@@ -9,7 +9,8 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, }struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn) +struct instruction *insn, +unsigned long *table_size) {return NULL ; }
tools/objtool/arch/powerpc/special.c +2 -1
@@ -13,7 +13,8 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, }struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn) +struct instruction *insn, +unsigned long *table_size) {exit (-1 ); }
tools/objtool/arch/x86/special.c +3 -1
@@ -109,7 +109,8 @@ bool arch_support_alt_relocation(struct special_alt *special_alt,* NOTE: MITIGATION_RETPOLINE made it harder still to decode dynamic jumps. */ struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn) +struct instruction *insn, +unsigned long *table_size) {struct reloc *text_reloc, *rodata_reloc;struct section *table_sec;
@@ -158,5 +159,6 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,if (reloc_type (text_reloc) ==R_X86_64_PC32 ) file->ignore_unreachables = true; + *table_size =0 ;return rodata_reloc; }
tools/objtool/check.c +20 -11
@@ -150,6 +150,15 @@ static inline struct reloc *insn_jump_table(struct instruction *insn)return NULL ; } +static inline unsigned long insn_jump_table_size (struct instruction *insn) +{ +if (insn->type ==INSN_JUMP_DYNAMIC || + insn->type ==INSN_CALL_DYNAMIC ) +return insn->_jump_table_size; + +return 0 ; +} +static bool is_jump_table_jump (struct instruction *insn) {struct alt_group *alt_group = insn->alt_group;
@@ -1937,6 +1946,7 @@ out:static int add_jump_table (struct objtool_file *file,struct instruction *insn,struct reloc *next_table) { +unsigned long table_size =insn_jump_table_size (insn);struct symbol *pfunc =insn_func (insn)->pfunc;struct reloc *table =insn_jump_table (insn);struct instruction *dest_insn;
@@ -1951,6 +1961,8 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn,for_each_reloc_from (table->sec, reloc) {/* Check for the end of the table: */ +if (table_size &&reloc_offset (reloc) -reloc_offset (table) >= table_size) +break ;if (reloc != table && reloc == next_table)break ;
@@ -1995,12 +2007,12 @@ static int add_jump_table(struct objtool_file *file, struct instruction *insn,* find_jump_table() - Given a dynamic jump, find the switch jump table * associated with it. */ -static struct reloc *find_jump_table (struct objtool_file *file, -struct symbol *func, -struct instruction *insn) +static void find_jump_table (struct objtool_file *file,struct symbol *func, +struct instruction *insn) {struct reloc *table_reloc;struct instruction *dest_insn, *orig_insn = insn; +unsigned long table_size;/* * Backward search using the @first_jump_src links, these help avoid
@@ -2021,17 +2033,17 @@ static struct reloc *find_jump_table(struct objtool_file *file, insn->jump_dest->offset > orig_insn->offset)) break; - table_reloc =arch_find_switch_table (file, insn); + table_reloc =arch_find_switch_table (file, insn, &table_size);if (!table_reloc)continue ; dest_insn =find_insn (file, table_reloc->sym->sec,reloc_addend (table_reloc));if (!dest_insn || !insn_func (dest_insn) ||insn_func (dest_insn)->pfunc != func)continue ; -return table_reloc; + orig_insn->_jump_table = table_reloc; + orig_insn->_jump_table_size = table_size; +break ; } - -return NULL ; }/*
@@ -2042,7 +2054,6 @@ static void mark_func_jump_tables(struct objtool_file *file, struct symbol *func) {struct instruction *insn, *last =NULL ; -struct reloc *reloc;func_for_each_insn (file, func, insn) {if (!last)
@@ -2065,9 +2076,7 @@ static void mark_func_jump_tables(struct objtool_file *file,if (insn->type !=INSN_JUMP_DYNAMIC )continue ; - reloc =find_jump_table (file, func, insn); -if (reloc) - insn->_jump_table = reloc; +find_jump_table (file, func, insn); } }
tools/objtool/include/objtool/check.h +4 -1
@@ -71,7 +71,10 @@ struct instruction {struct instruction *first_jump_src;union {struct symbol *_call_dest; -struct reloc *_jump_table; +struct { +struct reloc *_jump_table; +unsigned long _jump_table_size; + }; };struct alternative *alts;struct symbol *sym;
tools/objtool/include/objtool/special.h +2 -1
@@ -38,5 +38,6 @@ bool arch_support_alt_relocation(struct special_alt *special_alt,struct instruction *insn, struct reloc *reloc);struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn); +struct instruction *insn, +unsigned long *table_size);#endif /* _SPECIAL_H */
PATCH v3 3/8 objtool: Make some helper functions globally accessible
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Move some helpers around so they can be used from arch specific jump table code that is getting refactored in the next patch.
--- tools/objtool/check.c | 22 ++++---------------- tools/objtool/include/objtool/check.h | 16 ++++++++++++++ 2 files changed, 20 insertions(+), 18 deletions(-)
tools/objtool/check.c +4 -18
@@ -61,8 +61,8 @@ struct instruction *next_insn_same_sec(struct objtool_file *file,return insn; } -static struct instruction *next_insn_same_func (struct objtool_file *file, -struct instruction *insn) +struct instruction *next_insn_same_func (struct objtool_file *file, +struct instruction *insn) {struct instruction *next =next_insn_same_sec (file, insn);struct symbol *func =insn_func (insn);
@@ -93,8 +93,8 @@ static struct instruction *prev_insn_same_sec(struct objtool_file *file,return insn -1 ; } -static struct instruction *prev_insn_same_sym (struct objtool_file *file, -struct instruction *insn) +struct instruction *prev_insn_same_sym (struct objtool_file *file, +struct instruction *insn) {struct instruction *prev =prev_insn_same_sec (file, insn);
@@ -110,11 +110,6 @@ static struct instruction *prev_insn_same_sym(struct objtool_file *file, for_each_sec(file, __sec) \ sec_for_each_insn(file, __sec, insn) -#define func_for_each_insn (file, func, insn) \ - for (insn = find_insn(file, func->sec, func->offset); \ - insn; \ - insn = next_insn_same_func(file, insn)) -#define sym_for_each_insn (file, sym, insn) \ for (insn = find_insn(file, sym->sec, sym->offset); \ insn && insn->offset < sym->offset + sym->len; \
@@ -141,15 +136,6 @@ static inline struct symbol *insn_call_dest(struct instruction *insn)return insn->_call_dest; } -static inline struct reloc *insn_jump_table (struct instruction *insn) -{ -if (insn->type ==INSN_JUMP_DYNAMIC || - insn->type ==INSN_CALL_DYNAMIC ) -return insn->_jump_table; - -return NULL ; -} -static inline unsigned long insn_jump_table_size (struct instruction *insn) { if (insn->type == INSN_JUMP_DYNAMIC ||
tools/objtool/include/objtool/check.h +16 -0
@@ -114,14 +114,30 @@ static inline bool is_jump(struct instruction *insn)return is_static_jump (insn) ||is_dynamic_jump (insn); } +static inline struct reloc *insn_jump_table (struct instruction *insn) +{ +if (insn->type ==INSN_JUMP_DYNAMIC || + insn->type ==INSN_CALL_DYNAMIC ) +return insn->_jump_table; + +return NULL ; +} +struct instruction *find_insn (struct objtool_file *file,struct section *sec,unsigned long offset); +struct instruction *prev_insn_same_sym (struct objtool_file *file,struct instruction *insn);struct instruction *next_insn_same_sec (struct objtool_file *file,struct instruction *insn); +struct instruction *next_insn_same_func (struct objtool_file *file,struct instruction *insn);#define sec_for_each_insn (file, _sec, insn) \ for (insn = find_insn(file, _sec, 0); \ insn && insn->sec == _sec; \ insn = next_insn_same_sec(file, insn)) +#define func_for_each_insn (file, func, insn) \ + for (insn = find_insn(file, func->sec, func->offset); \ + insn; \ + insn = next_insn_same_func(file, insn)) +#endif /* _CHECK_H */
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
PATCH v3 4/8 objtool: Move jump table heuristics to a x86 specific source file
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
In preparation for implementing support for the use of compiler emitted jump table annotations, move the existing code out of the generic sources. This will permit a clean separation between the two approaches, where the old one will not be wired up for architectures other than x86.
--- tools/objtool/arch/loongarch/special.c | 7 -- tools/objtool/arch/powerpc/special.c | 7 -- tools/objtool/arch/x86/special.c | 114 +++++++++++++++++++- tools/objtool/check.c | 112 +------------------ tools/objtool/include/objtool/special.h | 9 +- 5 files changed, 122 insertions(+), 127 deletions(-)
tools/objtool/arch/loongarch/special.c +0 -7
@@ -7,10 +7,3 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, {return false; } - -struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn, -unsigned long *table_size) -{ -return NULL ; -}
tools/objtool/arch/powerpc/special.c +0 -7
@@ -11,10 +11,3 @@ bool arch_support_alt_relocation(struct special_alt *special_alt, {exit (-1 ); } - -struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn, -unsigned long *table_size) -{ -exit (-1 ); -}
tools/objtool/arch/x86/special.c +111 -3
123 lines not on this page · raw message
tools/objtool/check.c +5 -107
131 lines not on this page · raw message
tools/objtool/include/objtool/special.h +6 -3
@@ -37,7 +37,10 @@ void arch_handle_alternative(unsigned short feature, struct special_alt *alt);bool arch_support_alt_relocation (struct special_alt *special_alt,struct instruction *insn,struct reloc *reloc); -struct reloc *arch_find_switch_table (struct objtool_file *file, -struct instruction *insn, -unsigned long *table_size); + +int add_func_jump_tables (struct objtool_file *file,struct symbol *func); + +int add_jump_table (struct objtool_file *file,struct instruction *insn, +struct reloc *next_table); +#endif /* _SPECIAL_H */
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
PATCH v3 5/8 objtool: Add generic support for jump table annotations
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Refactor the jump table handling code so that a generic code path is provided that can identify jump tables attached to indirect jumps based only on compiler provided annotations. This will be used by non-x86 architectures which do not support jump tables at all at this point.
Refactor the x86 code to share the logic that follows relocations on instructions into the .rodata section and finds the associated symbols.
--- tools/objtool/arch/x86/special.c | 46 ++++------ tools/objtool/check.c | 88 +++++++++++++++++++- tools/objtool/include/objtool/check.h | 4 + 3 files changed, 106 insertions(+), 32 deletions(-)
tools/objtool/arch/x86/special.c +17 -29
@@ -112,46 +112,34 @@ static struct reloc *find_switch_table(struct objtool_file *file,struct instruction *insn, unsigned long *table_size) { - struct reloc *text_reloc, *rodata_reloc; -struct section *table_sec; -unsigned long table_offset; - -/* look for a relocation which references .rodata */ - text_reloc =find_reloc_by_dest_range (file->elf, insn->sec, - insn->offset, insn->len); -if (!text_reloc || text_reloc->sym->type !=STT_SECTION || - !text_reloc->sym->sec->rodata) -return NULL ; - - table_offset =reloc_addend (text_reloc); - table_sec = text_reloc->sym->sec; + struct reloc *rodata_reloc; +struct symbol *sym =NULL ; -if (reloc_type (text_reloc) ==R_X86_64_PC32 ) - table_offset +=4 ; +/* +* Each table entry has a rela associated with it. The rela +* should reference text in the same function as the original +* instruction. +*/ + rodata_reloc =find_rodata_sym_reference (file, insn, &sym);/* -* Make sure the .rodata address isn't associated with a -* symbol. GCC jump tables are anonymous data. +* Annotations, if present, are attached to the indirect jump +* instruction directly. In this case, a symbol annotation is +* expected. +* +* Otherwise, make sure the .rodata address isn't associated with +* a symbol. Unannotated GCC jump tables are anonymous data. * * Also support C jump tables which are in the same format as * switch jump tables. For objtool to recognize them, they * need to be placed in the C_JUMP_TABLE_SECTION section. They * have symbols associated with them. */ -if (find_symbol_containing (table_sec, table_offset) && -strcmp (table_sec->name,C_JUMP_TABLE_SECTION )) -return NULL ; - -/* -* Each table entry has a rela associated with it. The rela -* should reference text in the same function as the original -* instruction. -*/ - rodata_reloc =find_reloc_by_dest (file->elf, table_sec, table_offset); -if (!rodata_reloc) +if (insn->type !=INSN_JUMP_DYNAMIC && sym && +strcmp (sym->sec->name,C_JUMP_TABLE_SECTION ))return NULL ; - *table_size =0 ; + *table_size = sym ? sym->len :0 ;return rodata_reloc; }
tools/objtool/check.c +85 -3
109 lines not on this page · raw message
tools/objtool/include/objtool/check.h +4 -0
@@ -140,4 +140,8 @@ struct instruction *next_insn_same_func(struct objtool_file *file, struct instruinsn ; \ insn = next_insn_same_func(file, insn)) +struct reloc *find_rodata_sym_reference (struct objtool_file *file, +struct instruction *insn, +struct symbol **sym); +#endif /* _CHECK_H */
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
PATCH v3 6/8 crypto: x86/crc32c - Use idiomatic relative jump table
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The original crc32c code used a place-relative jump table but with a slightly awkward use of two separate symbols. To help objtool, this was replaced with a bog-standard position dependent jump table call, which was subsequently tweaked to use a RIP-relative reference to the table, but still populate it with absolute 64-bit references.
Given that objtool will need to be taught about the jump table idiom that compilers use when running with -fpie enabled, let's update the jump table in the crc32c code once again to use this standard idiom, where the jump table carries 32-bit references relative to the start of the table, and the destination address can be obtained by adding the two.
--- arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +3 -2
@@ -53,7 +53,7 @@ .endm .macro JMPTBL_ENTRY i -.quad .Lcrc_\i +.long .Lcrc_\i - jump_table .endm .macro JNC_LESS_THAN j
@@ -169,7 +169,8 @@ SYM_FUNC_START(crc_pcl) ## branch into array leaq jump_table(%rip), %bufp - mov (%bufp,%rax,8), %bufp + movslq (%bufp,%rax,4), len + addq len, %bufp JMP_NOSPEC bufp ################################################################
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
Re: [PATCH v3 6/8] crypto: x86/crc32c - Use idiomatic relative jump table
Quoted 14 lines · Ard Biesheuvel
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +1
-1@@ -53,7 +53,7 @@ .endm .macro JMPTBL_ENTRY i -.quad .Lcrc_\i +.long .Lcrc_\i - jump_table .endm .macro JNC_LESS_THAN j
arch/x86/crypto/crc32c-pcl-intel-asm_64.S SYM_FUNC_START(crc_pcl) · line 174
Quoted 7 lines
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +2
-1@@ -169,7 +169,8 @@ SYM_FUNC_START(crc_pcl) ## branch into array leaq jump_table(%rip), %bufp - mov (%bufp,%rax,8), %bufp + movslq (%bufp,%rax,4), len + addq len, %bufp JMP_NOSPEC bufp
I think it would be much better to just get rid of the jump table here.
I have done this at
https://lore.kernel.org/linux-crypto/[email protected]/
- Eric
Re: [PATCH v3 6/8] crypto: x86/crc32c - Use idiomatic relative jump table
Quoted 17 lines · Eric Biggers
Quoted 15 lines · Ard Biesheuvel
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +1
-1@@ -53,7 +53,7 @@ .endm .macro JMPTBL_ENTRY i -.quad .Lcrc_\i +.long .Lcrc_\i - jump_table .endm .macro JNC_LESS_THAN j@@ -169,7 +169,8 @@ SYM_FUNC_START(crc_pcl)
Quoted 9 lines
arch/x86/crypto/crc32c-pcl-intel-asm_64.S SYM_FUNC_START(crc_pcl) · line 174
Quoted 6 lines
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +2
-1## branch into array leaq jump_table(%rip), %bufp - mov (%bufp,%rax,8), %bufp + movslq (%bufp,%rax,4), len + addq len, %bufp JMP_NOSPEC bufpI think it would be much better to just get rid of the jump table here.
Good point.
Quoted 3 lines
I have done this at
https://lore.kernel.org/linux-crypto/[email protected]/
I'll go and take a look - thanks.
PATCH v3 7/8 crypto: x86/crc32c - Add jump table annotation
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Annotate the indirect jump with a relocation that correlates it with the jump table emitted into .rodata. This helps objtool identify the jump table, allowing it to infer the places in the code that are reachable from the jump.
--- arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +3 -0
@@ -171,6 +171,7 @@ SYM_FUNC_START(crc_pcl) leaq jump_table(%rip), %bufp movslq (%bufp,%rax,4), len addq len, %bufp + .reloc ., R_X86_64_NONE, jump_table JMP_NOSPEC bufp ################################################################
@@ -327,6 +328,8 @@ JMPTBL_ENTRY %i i=i+1 .endr +.size jump_table, . - jump_table +.type jump_table, @object ################################################################ ## PCLMULQDQ tables
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
PATCH v3 8/8 crypto: x86/crc32c-intel - Tweaks to make objtool's life harder
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Tweak the asm crc32c asm code so that the jump table is more difficult to decipher for objtool:
- load the address in the prologue
- move the jump table to the middle of .rodata, so that the section offset and the symbol offset differ
- emit an entry following the jump table that is unrelated
- add unconditional ENDBRs so we can test --ibt regardless of the kernel config
Test code only. Not intended for merging.
--- arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 50 +++++++++++--------- 1 file changed, 28 insertions(+), 22 deletions(-)
arch/x86/crypto/crc32c-pcl-intel-asm_64.S +28 -22
@@ -93,10 +93,14 @@ SYM_FUNC_START(crc_pcl) #define crc1 %r9 #define crc2 %r10 + endbr64 + pushq %rbp pushq %rbx pushq %rdi pushq %rsi + leaq jump_table(%rip), %rbp + ## Move crc_init for Linux to a different mov crc_init_arg, crc_init
@@ -168,9 +172,8 @@ SYM_FUNC_START(crc_pcl) xor crc2, crc2 ## branch into array - leaq jump_table(%rip), %bufp - movslq (%bufp,%rax,4), len - addq len, %bufp + movslq (%rbp,%rax,4), %bufp + addq %rbp, %bufp .reloc ., R_X86_64_NONE, jump_table JMP_NOSPEC bufp
@@ -197,7 +200,7 @@ SYM_FUNC_START(crc_pcl) .altmacro LABEL crc_ %i .noaltmacro - ENDBR + endbr64 crc32q -i*8(block_0), crc_init crc32q -i*8(block_1), crc1 crc32q -i*8(block_2), crc2
@@ -207,7 +210,7 @@ LABEL crc_ %i .altmacro LABEL crc_ %i .noaltmacro - ENDBR + endbr64 crc32q -i*8(block_0), crc_init crc32q -i*8(block_1), crc1 # SKIP crc32 -i*8(block_2), crc2 ; Don't do this one yet
@@ -241,7 +244,7 @@ LABEL crc_ %i ################################################################ LABEL crc_ 0 - ENDBR + endbr64 mov tmp, len cmp $128*24, tmp jae .Lfull_block
@@ -311,26 +314,11 @@ LABEL less_than_ %j # less_than_j: Length should be in popq %rsi popq %rdi popq %rbx + popq %rbp RET SYM_FUNC_END(crc_pcl) .section .rodata, "a", @progbits - ################################################################ - ## jump table Table is 129 entries x 2 bytes each - ################################################################ -.align 4 -jump_table: - i=0 -.rept 129 -.altmacro -JMPTBL_ENTRY %i -.noaltmacro - i=i+1 -.endr - -.size jump_table, . - jump_table -.type jump_table, @object - ################################################################ ## PCLMULQDQ tables ## Table is 128 entries x 2 words (8 bytes) each
@@ -465,3 +453,21 @@ K_table: .long 0x45cddf4e, 0xe0ac139e .long 0xacfa3103, 0x6c23e841 .long 0xa51b6135, 0x170076fa + + ################################################################ + ## jump table Table is 129 entries x 2 bytes each + ################################################################ +.align 4 +jump_table: + i=0 +.rept 129 +.altmacro +JMPTBL_ENTRY %i +.noaltmacro + i=i+1 +.endr + +.size jump_table, . - jump_table +.type jump_table, @object + + .long crc_pcl - .
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog