crypto objtool cleanup
Subsystems (3)
Peter Zijlstra
maintainercover crypto objtool cleanup
Hi,
two small patches that fix objtool reports on x86 crypto asm. The first (chacha) is independent and could go through the crypto tree if desired. The second is a fix for a commit that currently still lives in tip/x86/core and should go there.
Peter Zijlstra
maintainerPATCH 1/2 x86/chacha20: Avoid spurious jumps to other functions
The chacha_Nblock_xor_avx512vl() functions all have their own, identical, .LdoneN label, however in one particular spot {2,4} jump to the 8 version instead of their own. Resulting in:
arch/x86/crypto/chacha-x86_64.o: warning: objtool: chacha_2block_xor_avx512vl() falls through to next function chacha_8block_xor_avx512vl() arch/x86/crypto/chacha-x86_64.o: warning: objtool: chacha_4block_xor_avx512vl() falls through to next function chacha_8block_xor_avx512vl()
Make each function consistently use its own done label.
--- arch/x86/crypto/chacha-avx512vl-x86_64.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/crypto/chacha-avx512vl-x86_64.S +2 -2
@@ -172,7 +172,7 @@ SYM_FUNC_START(chacha_2block_xor_avx512v # xor remaining bytes from partial register into output mov %rcx,%rax and $0xf,%rcx - jz .Ldone8 + jz .Ldone2 mov %rax,%r9 and $~0xf,%r9
@@ -438,7 +438,7 @@ SYM_FUNC_START(chacha_4block_xor_avx512v # xor remaining bytes from partial register into output mov %rcx,%rax and $0xf,%rcx - jz .Ldone8 + jz .Ldone4 mov %rax,%r9 and $~0xf,%r9
Re: [PATCH 1/2] x86/chacha20: Avoid spurious jumps to other functions
Quoted 6 lines
The
chacha_Nblock_xor_avx512vl()functions all have their own, identical, .LdoneN label, however in one particular spot {2,4} jump to the 8 version instead of their own. [...]Make each function consistently use its own done label.
Herbert Xu
maintainerRe: [PATCH 1/2] x86/chacha20: Avoid spurious jumps to other functions
In reply to Peter ZijlstraQuoted 7 lines · Peter Zijlstra
The
chacha_Nblock_xor_avx512vl()functions all have their own, identical, .LdoneN label, however in one particular spot {2,4} jump to the 8 version instead of their own. Resulting in:
arch/x86/crypto/chacha-x86_64.o: warning: objtool:chacha_2block_xor_avx512vl()falls through to next functionchacha_8block_xor_avx512vl()arch/x86/crypto/chacha-x86_64.o: warning: objtool:chacha_4block_xor_avx512vl()falls through to next functionchacha_8block_xor_avx512vl()
Quoted 7 lines
Make each function consistently use its own done label.
Reported-by Stephen Rothwell <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- arch/x86/crypto/chacha-avx512vl-x86_64.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Patch applied. Thanks.
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Peter Zijlstra
maintainerPATCH 2/2 objtool: Fix IBT tail-call detection
In reply to Peter ZijlstraObjtool reports:
arch/x86/crypto/poly1305-x86_64.o: warning: objtool: poly1305_blocks_avx() falls through to next function poly1305_blocks_x86_64() arch/x86/crypto/poly1305-x86_64.o: warning: objtool: poly1305_emit_avx() falls through to next function poly1305_emit_x86_64() arch/x86/crypto/poly1305-x86_64.o: warning: objtool: poly1305_blocks_avx2() falls through to next function poly1305_blocks_x86_64()
Which reads like:
0000000000000040 <poly1305_blocks_x86_64>:
40: f3 0f 1e fa endbr64 ...
0000000000000400 <poly1305_blocks_avx>:
400: f3 0f 1e fa endbr64 404: 44 8b 47 14 mov 0x14(%rdi),%r8d 408: 48 81 fa 80 00 00 00 cmp $0x80,%rdx 40f: 73 09 jae 41a <poly1305_blocks_avx+0x1a> 411: 45 85 c0 test %r8d,%r8d 414: 0f 84 2a fc ff ff je 44 <poly1305_blocks_x86_64+0x4> ...
These are simple conditional tail-calls and *should* be recognised as such by objtool, however due to a mistake in commit 08f87a93c8ec ("objtool: Validate IBT assumptions") this is failing.
Specifically, the jump_dest is +4, this means the instruction pointed at will not be ENDBR and as such it will fail the second clause of is_first_func_insn() that was supposed to capture this exact case.
Instead, have is_first_func_insn() look at the previous instruction.
--- tools/objtool/check.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
tools/objtool/check.c +14 -5
@@ -1239,11 +1239,20 @@ static bool same_function(struct instrucreturn insn1->func->pfunc == insn2->func->pfunc; } -static bool is_first_func_insn (struct instruction *insn) +static bool is_first_func_insn (struct objtool_file *file,struct instruction *insn) { -return insn->offset == insn->func->offset || - (insn->type ==INSN_ENDBR && - insn->offset == insn->func->offset + insn->len); +if (insn->offset == insn->func->offset) +return true; + +if (ibt) { +struct instruction *prev =prev_insn_same_sym (file, insn); + +if (prev && prev->type ==INSN_ENDBR && + insn->offset == insn->func->offset + prev->len) +return true; + } + +return false; }/*
@@ -1327,7 +1336,7 @@ static int add_jump_destinations(struct insn->jump_dest->func->pfunc = insn->func; }else if (!same_function (insn, insn->jump_dest) && -is_first_func_insn (insn->jump_dest)) { +is_first_func_insn (file, insn->jump_dest)) {/* internal sibling call (without reloc) */ add_call_dest (file, insn, insn->jump_dest->func, true); }
[tip: x86/urgent] objtool: Fix IBT tail-call detection
The following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: d139bca4b824ffb9731763c31b271a24b595948a Gitweb: https://git.kernel.org/tip/d139bca4b824ffb9731763c31b271a24b595948a Author: Peter Zijlstra <[email protected]> AuthorDate: Tue, 22 Mar 2022 12:33:31 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 05 Apr 2022 10:24:40 +02:00
objtool: Fix IBT tail-call detection
Objtool reports:
arch/x86/crypto/poly1305-x86_64.o: warning: objtool: poly1305_blocks_avx() falls through to next function poly1305_blocks_x86_64() arch/x86/crypto/poly1305-x86_64.o: warning: objtool: poly1305_emit_avx() falls through to next function poly1305_emit_x86_64() arch/x86/crypto/poly1305-x86_64.o: warning: objtool: poly1305_blocks_avx2() falls through to next function poly1305_blocks_x86_64()
Which reads like:
0000000000000040 <poly1305_blocks_x86_64>:
40: f3 0f 1e fa endbr64 ...
0000000000000400 <poly1305_blocks_avx>:
400: f3 0f 1e fa endbr64 404: 44 8b 47 14 mov 0x14(%rdi),%r8d 408: 48 81 fa 80 00 00 00 cmp $0x80,%rdx 40f: 73 09 jae 41a <poly1305_blocks_avx+0x1a> 411: 45 85 c0 test %r8d,%r8d 414: 0f 84 2a fc ff ff je 44 <poly1305_blocks_x86_64+0x4> ...
These are simple conditional tail-calls and *should* be recognised as such by objtool, however due to a mistake in commit 08f87a93c8ec ("objtool: Validate IBT assumptions") this is failing.
Specifically, the jump_dest is +4, this means the instruction pointed at will not be ENDBR and as such it will fail the second clause of is_first_func_insn() that was supposed to capture this exact case.
Instead, have is_first_func_insn() look at the previous instruction.
--- tools/objtool/check.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
tools/objtool/check.c +14 -5
@@ -1239,11 +1239,20 @@ static bool same_function(struct instruction *insn1, struct instruction *insn2)return insn1->func->pfunc == insn2->func->pfunc; } -static bool is_first_func_insn (struct instruction *insn) +static bool is_first_func_insn (struct objtool_file *file,struct instruction *insn) { -return insn->offset == insn->func->offset || - (insn->type ==INSN_ENDBR && - insn->offset == insn->func->offset + insn->len); +if (insn->offset == insn->func->offset) +return true; + +if (ibt) { +struct instruction *prev =prev_insn_same_sym (file, insn); + +if (prev && prev->type ==INSN_ENDBR && + insn->offset == insn->func->offset + prev->len) +return true; + } + +return false; }/*
@@ -1327,7 +1336,7 @@ static int add_jump_destinations(struct objtool_file *file) insn->jump_dest->func->pfunc = insn->func; }else if (!same_function (insn, insn->jump_dest) && -is_first_func_insn (insn->jump_dest)) { +is_first_func_insn (file, insn->jump_dest)) {/* internal sibling call (without reloc) */ add_call_dest (file, insn, insn->jump_dest->func, true); }
Peter Zijlstra
maintainer[PATCH 3/2] x86/poly1305: Fixup SLS
In reply to Peter ZijlstraDue to being a perl generated asm file, it got missed by the mass convertion script.
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_init_x86_64()+0x3a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_x86_64()+0xf2: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_x86_64()+0x37: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_block()+0x6d: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_init_avx()+0x1e8: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0x18a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0xaf8: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_avx()+0x99: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x18a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x776: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x18a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x796: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x10bd: missing int3 after ret
--- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 38 +++++++++++++------------- 1 file changed, 19 insertions(+), 19 deletions(-)
arch/x86/crypto/poly1305-x86_64-cryptogams.pl +19 -19
@@ -297,7 +297,7 @@ ___ $code.=<<___; mov \$1,%eax .Lno_key: - ret + RET ___ &end_function("poly1305_init_x86_64");
@@ -373,7 +373,7 @@ $code.=<<___; .cfi_adjust_cfa_offset -48 .Lno_data: .Lblocks_epilogue: - ret + RET .cfi_endproc ___ &end_function("poly1305_blocks_x86_64");
@@ -399,7 +399,7 @@ $code.=<<___; mov %rax,0($mac) # write result mov %rcx,8($mac) - ret + RET ___ &end_function("poly1305_emit_x86_64"); if ($avx) {
@@ -429,7 +429,7 @@ ___ &poly1305_iteration(); $code.=<<___; pop $ctx - ret + RET .size __poly1305_block,.-__poly1305_block .type __poly1305_init_avx,\@abi-omnipotent
@@ -594,7 +594,7 @@ $code.=<<___; lea -48-64($ctx),$ctx # size [de-]optimization pop %rbp - ret + RET .size __poly1305_init_avx,.-__poly1305_init_avx ___
@@ -747,7 +747,7 @@ $code.=<<___; .cfi_restore %rbp .Lno_data_avx: .Lblocks_avx_epilogue: - ret + RET .cfi_endproc .align 32
@@ -1452,7 +1452,7 @@ $code.=<<___ if (!$win64); ___ $code.=<<___; vzeroupper - ret + RET .cfi_endproc ___ &end_function("poly1305_blocks_avx");
@@ -1508,7 +1508,7 @@ $code.=<<___; mov %rax,0($mac) # write result mov %rcx,8($mac) - ret + RET ___ &end_function("poly1305_emit_avx");
@@ -1675,7 +1675,7 @@ $code.=<<___; .cfi_restore %rbp .Lno_data_avx2$suffix: .Lblocks_avx2_epilogue$suffix: - ret + RET .cfi_endproc .align 32
@@ -2201,7 +2201,7 @@ $code.=<<___ if (!$win64); ___ $code.=<<___; vzeroupper - ret + RET .cfi_endproc ___ if($avx > 2 && $avx512) {
@@ -2792,7 +2792,7 @@ $code.=<<___ if (!$win64); .cfi_def_cfa_register %rsp ___ $code.=<<___; - ret + RET .cfi_endproc ___
@@ -2893,7 +2893,7 @@ $code.=<<___ if ($flavour =~ /elf32/); ___ $code.=<<___; mov \$1,%eax - ret + RET .size poly1305_init_base2_44,.-poly1305_init_base2_44 ___ {
@@ -3010,7 +3010,7 @@ $code.=<<___; jnz .Lblocks_vpmadd52_4x .Lno_data_vpmadd52: - ret + RET .size poly1305_blocks_vpmadd52,.-poly1305_blocks_vpmadd52 ___ }
@@ -3451,7 +3451,7 @@ $code.=<<___; vzeroall .Lno_data_vpmadd52_4x: - ret + RET .size poly1305_blocks_vpmadd52_4x,.-poly1305_blocks_vpmadd52_4x ___ }
@@ -3824,7 +3824,7 @@ $code.=<<___; vzeroall .Lno_data_vpmadd52_8x: - ret + RET .size poly1305_blocks_vpmadd52_8x,.-poly1305_blocks_vpmadd52_8x ___ }
@@ -3861,7 +3861,7 @@ $code.=<<___; mov %rax,0($mac) # write result mov %rcx,8($mac) - ret + RET .size poly1305_emit_base2_44,.-poly1305_emit_base2_44 ___ } } }
@@ -3916,7 +3916,7 @@ $code.=<<___; .Ldone_enc: mov $otp,%rax - ret + RET .size xor128_encrypt_n_pad,.-xor128_encrypt_n_pad .globl xor128_decrypt_n_pad
@@ -3967,7 +3967,7 @@ $code.=<<___; .Ldone_dec: mov $otp,%rax - ret + RET .size xor128_decrypt_n_pad,.-xor128_decrypt_n_pad ___ }
@@ -4109,7 +4109,7 @@ $code.=<<___; pop %rbx pop %rdi pop %rsi - ret + RET .size avx_handler,.-avx_handler .section .pdata
Herbert Xu
maintainerRe: [PATCH 3/2] x86/poly1305: Fixup SLS
Quoted 18 lines · Peter Zijlstra
Due to being a perl generated asm file, it got missed by the mass convertion script.
arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_init_x86_64()+0x3a: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_x86_64()+0xf2: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_x86_64()+0x37: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_block()+0x6d: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_init_avx()+0x1e8: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0x18a: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0xaf8: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_avx()+0x99: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x18a: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x776: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x18a: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x796: missing int3 after retarch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x10bd: missing int3 after ret
Quoted 5 lines
Fixes f94909ceb1ed ("x86: Prepare asm files for straight-line-speculation")Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 38 +++++++++++++------------- 1 file changed, 19 insertions(+), 19 deletions(-)
Patch applied. Thanks.
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Peter Zijlstra
maintainer[PATCH 4/2] objtool: Fix SLS validation for KCOV tail-call replacement
In reply to Peter ZijlstraSince not all compilers have a function attribute to disable KCOV instrumentation, objtool can rewrite KCOV instrumentation in noinstr functions as per commit:
f56dae88a81f ("objtool: Handle __sanitize_cov*() tail calls")
However, this has subtle interaction with the SLS validation from commit:
1cc1e4c8aab4 ("objtool: Add straight-line-speculation validation")
In that when a tail-call instrucion is replaced with a RET an additional INT3 instruction is also written, but is not represented in the decoded instruction stream.
This then leads to false positive missing INT3 objtool warnings in noinstr code.
Instead of adding additional struct instruction objects, mark the RET instruction with retpoline_safe to suppress the warning (since we know there really is an INT3).
--- tools/objtool/check.c | 43 +++++++++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 14 deletions(-)
tools/objtool/check.c +11 -0
@@ -1090,6 +1099,17 @@ static void annotate_call_site(struct ob : arch_nop_insn(insn->len)); insn->type = sibling ?INSN_RETURN :INSN_NOP ; + +if (sibling) { +/* +* We've replaced the tail-call JMP insn by two new +* insn: RET; INT3, except we only have a single struct +* insn here. Mark it retpoline_safe to avoid the SLS +* warning, instead of adding another insn. +*/ + insn->retpoline_safe = true; + } +return ; }
[tip: x86/urgent] objtool: Fix SLS validation for kcov tail-call replacement
The following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 7a53f408902d913cd541b4f8ad7dbcd4961f5b82 Gitweb: https://git.kernel.org/tip/7a53f408902d913cd541b4f8ad7dbcd4961f5b82 Author: Peter Zijlstra <[email protected]> AuthorDate: Wed, 23 Mar 2022 23:35:01 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 05 Apr 2022 10:24:40 +02:00
objtool: Fix SLS validation for kcov tail-call replacement
Since not all compilers have a function attribute to disable KCOV instrumentation, objtool can rewrite KCOV instrumentation in noinstr functions as per commit:
f56dae88a81f ("objtool: Handle __sanitize_cov*() tail calls")
However, this has subtle interaction with the SLS validation from commit:
1cc1e4c8aab4 ("objtool: Add straight-line-speculation validation")
In that when a tail-call instrucion is replaced with a RET an additional INT3 instruction is also written, but is not represented in the decoded instruction stream.
This then leads to false positive missing INT3 objtool warnings in noinstr code.
Instead of adding additional struct instruction objects, mark the RET instruction with retpoline_safe to suppress the warning (since we know there really is an INT3).
--- tools/objtool/check.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/objtool/check.c +11 -0
@@ -1155,6 +1155,17 @@ static void annotate_call_site(struct objtool_file *file, : arch_nop_insn(insn->len)); insn->type = sibling ?INSN_RETURN :INSN_NOP ; + +if (sibling) { +/* +* We've replaced the tail-call JMP insn by two new +* insn: RET; INT3, except we only have a single struct +* insn here. Mark it retpoline_safe to avoid the SLS +* warning, instead of adding another insn. +*/ + insn->retpoline_safe = true; + } +return ; }
Peter Zijlstra
maintainer[PATCH 5/2] x86/sm3: Fixup SLS
In reply to Peter ZijlstraThis missed the big asm update due to being merged through the crypto tree.
--- arch/x86/crypto/sm3-avx-asm_64.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/crypto/sm3-avx-asm_64.S +1 -1
@@ -513,5 +513,5 @@ SYM_FUNC_START(sm3_transform_avx) movq %rbp, %rsp; popq %rbp; - ret; + RET; SYM_FUNC_END(sm3_transform_avx)
Herbert Xu
maintainerRe: [PATCH 5/2] x86/sm3: Fixup SLS
Quoted 9 lines · Peter Zijlstra
This missed the big asm update due to being merged through the crypto tree.
Fixes f94909ceb1ed ("x86: Prepare asm files for straight-line-speculation")Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- arch/x86/crypto/sm3-avx-asm_64.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Patch applied. Thanks.
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt