objtool: Fix SLS checks
Subsystems (5)
Peter Zijlstra
maintainerPATCH 1/1 objtool: Fix SLS checks
Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be INSN_TRAP.
--- tools/objtool/check.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/objtool/check.c +2 -2
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,case INSN_RETURN :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after ret" , insn->sec, insn->offset); }
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,case INSN_JUMP_DYNAMIC :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after indirect jump" , insn->sec, insn->offset); }
Quoted 23 lines · Peter Zijlstra
Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be
INSN_TRAP.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- tools/objtool/check.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/objtool/check.c +1
-1@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,case INSN_RETURN :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after ret" , insn->sec, insn->offset); }
tools/objtool/check.c static int validate_branch(struct objtool_file *file, struct symbol *func, · line 3434
Quoted 8 lines
tools/objtool/check.c +1
-1@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,case INSN_JUMP_DYNAMIC :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after indirect jump" , insn->sec, insn->offset); }
My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this could be good for -urgent.
Here's another SLS improvement I mentioned to you the other day, do you agree we should do this as well?
From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled
Having disabled retpolines doesn't necessarily mean the user doesn't care about straight-line speculation. For example, retpolines are disabled when eIBRS is used.
If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the retpolines disabled cases.
--- arch/x86/include/asm/nospec-branch.h | 13 ++++++++++--- tools/objtool/check.c | 3 --- 2 files changed, 10 insertions(+), 6 deletions(-)
arch/x86/include/asm/nospec-branch.h +10 -3
@@ -75,6 +75,13 @@ .popsection .endm +.macro INDIRECT_JMP reg + jmp *%\reg +#ifdef CONFIG_SLS + int3 +#endif +.endm +/* * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple * indirect jmp/call which may be susceptible to the Spectre variant 2
@@ -82,11 +89,11 @@*/ .macro JMP_NOSPEC reg:req#ifdef CONFIG_RETPOLINE - ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ + ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \ __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE #else - jmp *%\reg + INDIRECT_JMP \reg #endif .endm
tools/objtool/check.c +0 -3
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)for_each_insn (file, insn) { next_insn =next_insn_same_sec (file, insn); -if (insn->retpoline_safe) -continue ; -switch (insn->type) {case INSN_RETURN :if (!next_insn || next_insn->type !=INSN_TRAP ) {
Signature · 1 line
2.34.1
Peter Zijlstra
maintainerQuoted 25 lines · Josh Poimboeuf
Quoted 24 lines · Peter Zijlstra
Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be
INSN_TRAP.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- tools/objtool/check.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/objtool/check.c +1
-1@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,case INSN_RETURN :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after ret" , insn->sec, insn->offset); }@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
Quoted 10 lines
tools/objtool/check.c static int validate_branch(struct objtool_file *file, struct symbol *func, · line 3434
Quoted 7 lines
tools/objtool/check.c +1
-1case INSN_JUMP_DYNAMIC :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after indirect jump" , insn->sec, insn->offset); }My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this could be good for -urgent.
Urgh, I should've looked at that indeed. This didn't find any new sites though; so I don't think this needs to go through urgent.
Quoted 37 lines · Peter Zijlstra
Quoted 25 lines · Josh Poimboeuf
Quoted 24 lines · Peter Zijlstra
Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be
INSN_TRAP.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- tools/objtool/check.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/objtool/check.c +1
-1@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,case INSN_RETURN :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after ret" , insn->sec, insn->offset); }@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,Quoted 10 lines
tools/objtool/check.c static int validate_branch(struct objtool_file *file, struct symbol *func, · line 3434
Quoted 7 lines
tools/objtool/check.c +1
-1case INSN_JUMP_DYNAMIC :if (sls && !insn->retpoline_safe && - next_insn && next_insn->type !=INSN_TRAP ) { + (!next_insn || (next_insn && next_insn->type !=INSN_TRAP ))) {WARN_FUNC ("missing int3 after indirect jump" , insn->sec, insn->offset); }My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this could be good for -urgent.
Quoted 2 lines
Urgh, I should've looked at that indeed. This didn't find any new sites though; so I don't think this needs to go through urgent.
Well to be fair, it was easy to miss since I snuck it in with rewrite.
Signature · 1 line
Josh
Peter Zijlstra
maintainerQuoted 54 lines · Josh Poimboeuf
From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] x86/speculation: Mitigate SLS forJMP_NOSPECwith retpolines disabledHaving disabled retpolines doesn't necessarily mean the user doesn't care about straight-line speculation. For example, retpolines are disabled when eIBRS is used.
If
CONFIG_SLSis enabled, properly mitigate SLS forJMP_NOSPECfor the retpolines disabled cases.Signed-off-by Josh Poimboeuf <[email protected]>--- arch/x86/include/asm/nospec-branch.h | 13 ++++++++++--- tools/objtool/check.c | 3 --- 2 files changed, 10 insertions(+), 6 deletions(-)arch/x86/include/asm/nospec-branch.h +10
-3@@ -75,6 +75,13 @@ .popsection .endm +.macro INDIRECT_JMP reg + jmp *%\reg +#ifdef CONFIG_SLS + int3 +#endif +.endm +/* * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple * indirect jmp/call which may be susceptible to the Spectre variant 2 @@ -82,11 +89,11 @@*/ .macro JMP_NOSPEC reg:req#ifdef CONFIG_RETPOLINE - ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ + ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \ __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE #else - jmp *%\reg + INDIRECT_JMP \reg #endif .endmtools/objtool/check.c +0
-0
tools/objtool/check.c static int validate_sls(struct objtool_file *file) · line 3847
Quoted 9 lines
tools/objtool/check.c +0
-3@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)for_each_insn (file, insn) { next_insn =next_insn_same_sec (file, insn); -if (insn->retpoline_safe) -continue ; -switch (insn->type) {case INSN_RETURN :if (!next_insn || next_insn->type !=INSN_TRAP ) {
Yes, agreed. But perhaps with something like this on top?
---
arch/x86/kernel/alternative.c +11 -0
@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)return ret; i += ret; +#ifdef CONFIG_SLS +/* +* Ideally this would be unconditional, except in case of +* RETPOLINE_LFENCE we don't have sufficient space. Additionally, +* -mharden-sls=all should be extended to emit INT3 after +* direct jumps too, which will then cover that case. +*/ +if (i < insn->length) + bytes[i++] =INT3_INSN_OPCODE ; +#endif +for (; i < insn->length;) bytes[i++] =BYTES_NOP1 ;
arch/x86/lib/retpoline.S +2 -2
@@ -33,9 +33,9 @@ SYM_INNER_LABEL(__x86_indirect_thunk_\reg, SYM_L_GLOBAL) UNWIND_HINT_EMPTY ANNOTATE_NOENDBR - ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ + ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \ __stringify(RETPOLINE \reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg; int3), X86_FEATURE_RETPOLINE_LFENCE + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE .endm
arch/x86/net/bpf_jit_comp.c +4 -0
@@ -418,6 +418,10 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)#endif EMIT2 (0xFF ,0xE0 + reg); +#ifdef CONFIG_SLS +EMIT1 (0xCC ); +#endif + *pprog = prog; }
Quoted 5 lines · Peter Zijlstra
Quoted 5 lines
+++
b/tools/objtool/check.c(quoted diff) +0
-0@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file) for_each_insn(file, insn) { next_insn = next_insn_same_sec(file, insn);
Quoted 8 lines
Review static int validate_sls(struct objtool_file *file) · line 3847
Quoted 6 lines
(quoted diff) +0
-3- if (insn->retpoline_safe) - continue; - switch (insn->type) { case INSN_RETURN: if (!next_insn || next_insn->type != INSN_TRAP) {Yes, agreed. But perhaps with something like this on top?
Yup, I missed those... Looks good. Just one comment:
arch/x86/kernel/alternative.c static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes) · line 461
Quoted 12 lines
+++
b/arch/x86/kernel/alternative.carch/x86/kernel/alternative.c +7
-0@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)return ret; i += ret; +#ifdef CONFIG_SLS +/* +* Ideally this would be unconditional, except in case of +* RETPOLINE_LFENCE we don't have sufficient space. Additionally, +* -mharden-sls=all should be extended to emit INT3 after +* direct jumps too, which will then cover that case. +*/
I don't quite follow this 2nd sentence and how it's related here, since this function doesn't actually deal with direct jumps.
Speaking of, I guess we'll eventually need to hack this SLS mess into jump labels :-/
Quoted 3 lines
arch/x86/kernel/alternative.c +3
-0+if (i < insn->length) + bytes[i++] =INT3_INSN_OPCODE ; +#endif
Signature · 1 line
Josh
Peter Zijlstra
maintainerQuoted 22 lines · Josh Poimboeuf
Quoted 5 lines · Peter Zijlstra
Quoted 5 lines
+++
b/tools/objtool/check.c(quoted diff) +0
-0@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file) for_each_insn(file, insn) { next_insn = next_insn_same_sec(file, insn);Quoted 8 lines
Review static int validate_sls(struct objtool_file *file) · line 3847
Quoted 6 lines
(quoted diff) +0
-3- if (insn->retpoline_safe) - continue; - switch (insn->type) { case INSN_RETURN: if (!next_insn || next_insn->type != INSN_TRAP) {Yes, agreed. But perhaps with something like this on top?
Yup, I missed those... Looks good. Just one comment:
Quoted 5 lines
+++
b/arch/x86/kernel/alternative.carch/x86/kernel/alternative.c +0
-0@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)return ret; i += ret;
Quoted 10 lines
arch/x86/kernel/alternative.c static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes) · line 461
Quoted 7 lines
arch/x86/kernel/alternative.c +7
-0+#ifdef CONFIG_SLS +/* +* Ideally this would be unconditional, except in case of +* RETPOLINE_LFENCE we don't have sufficient space. Additionally, +* -mharden-sls=all should be extended to emit INT3 after +* direct jumps too, which will then cover that case. +*/ I don't quite follow this 2nd sentence and how it's related here, since this function doesn't actually deal with direct jumps.
Ah, my bad. Also, this wrong.
I suppose this wants to be something like:
if (i < insn->length && op == JMP32_INSN_OPCODE) bytes[i++] = INT3_INSN_OPCODE;
So this *can* be a jump, but typically won't be I suppose.
Quoted 2 lines
Speaking of, I guess we'll eventually need to hack this SLS mess into jump labels :-/
Urgh... can't we reason that the straight line case is actually expected to run with the given register state anyway and ignore this?
Josh Poimboeuf
maintainerQuoted 11 lines · Peter Zijlstra
Quoted 2 lines
I don't quite follow this 2nd sentence and how it's related here, since this function doesn't actually deal with direct jumps.
Ah, my bad. Also, this wrong.
I suppose this wants to be something like:
if (i < insn->length && op == JMP32_INSN_OPCODE) bytes[i++] = INT3_INSN_OPCODE;So this *can* be a jump, but typically won't be I suppose.
Yep.
Quoted 5 lines
Quoted 2 lines
Speaking of, I guess we'll eventually need to hack this SLS mess into jump labels :-/
Urgh... can't we reason that the straight line case is actually expected to run with the given register state anyway and ignore this?
Yeah, that makes sense.
So for jump labels the SLS path would probably not be worse than a typical v1-style conditional branch misspeculation into the 'else' path, and we've already given up on worrying about those anyway.
Signature · 1 line
Josh