Skip to contentSkip to conversation

objtool: Fix SLS checks

PATCH In review Peter Zijlstra 3 participants 8 messages Apr 30, 2022 – May 5, 2022

Subsystems (5)
lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

PATCH 1/1 objtool: Fix SLS checks

Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be INSN_TRAP.

---
 tools/objtool/check.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/objtool/check.c +2 -2
index 3f6785415894..3354101ffe34 100644
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_RETURN:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after ret",
 					  insn->sec, insn->offset);
 			}
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_JUMP_DYNAMIC:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after indirect jump",
 					  insn->sec, insn->offset);
 			}
Acked-by on v1 1/1
#1 lore raw
Quoted 23 lines · Peter Zijlstra

Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be INSN_TRAP.

---
 tools/objtool/check.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/objtool/check.c +1 -1
index 3f6785415894..3354101ffe34 100644
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_RETURN:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after ret",
 					  insn->sec, insn->offset);
 			}

tools/objtool/check.c static int validate_branch(struct objtool_file *file, struct symbol *func, · line 3434

Quoted 8 lines
tools/objtool/check.c +1 -1
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_JUMP_DYNAMIC:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after indirect jump",
 					  insn->sec, insn->offset);
 			}

My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this could be good for -urgent.

Here's another SLS improvement I mentioned to you the other day, do you agree we should do this as well?

From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled

Having disabled retpolines doesn't necessarily mean the user doesn't care about straight-line speculation. For example, retpolines are disabled when eIBRS is used.

If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the retpolines disabled cases.

---
 arch/x86/include/asm/nospec-branch.h | 13 ++++++++++---
 tools/objtool/check.c                |  3 ---
 2 files changed, 10 insertions(+), 6 deletions(-)
arch/x86/include/asm/nospec-branch.h +10 -3
index acbaeaf83b61..0648746bf60b 100644
@@ -75,6 +75,13 @@
 	.popsection
 .endm
 
+.macro INDIRECT_JMP reg
+	jmp *%\reg
+#ifdef CONFIG_SLS
+	int3
+#endif
+.endm
+
 /*
  * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple
  * indirect jmp/call which may be susceptible to the Spectre variant 2
@@ -82,11 +89,11 @@
  */
 .macro JMP_NOSPEC reg:req
 #ifdef CONFIG_RETPOLINE
-	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
 		      __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \
-		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE
+		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
 #else
-	jmp	*%\reg
+	INDIRECT_JMP \reg
 #endif
 .endm
 
tools/objtool/check.c +0 -3
index e7983c3e2408..3cf3ad0b5db5 100644
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
 	for_each_insn(file, insn) {
 		next_insn = next_insn_same_sec(file, insn);
 
-		if (insn->retpoline_safe)
-			continue;
-
 		switch (insn->type) {
 		case INSN_RETURN:
 			if (!next_insn || next_insn->type != INSN_TRAP) {
Signature · 1 line
2.34.1
author maintainer
#2 lore raw
Quoted 25 lines · Josh Poimboeuf
Quoted 24 lines · Peter Zijlstra

Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be INSN_TRAP.

---
 tools/objtool/check.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/objtool/check.c +1 -1
index 3f6785415894..3354101ffe34 100644
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_RETURN:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after ret",
 					  insn->sec, insn->offset);
 			}
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
Quoted 10 lines

tools/objtool/check.c static int validate_branch(struct objtool_file *file, struct symbol *func, · line 3434

Quoted 7 lines
tools/objtool/check.c +1 -1
 		case INSN_JUMP_DYNAMIC:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after indirect jump",
 					  insn->sec, insn->offset);
 			}

My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this could be good for -urgent.

Urgh, I should've looked at that indeed. This didn't find any new sites though; so I don't think this needs to go through urgent.

#3 lore raw
Quoted 37 lines · Peter Zijlstra
Quoted 25 lines · Josh Poimboeuf
Quoted 24 lines · Peter Zijlstra

Fix the SLS validation; not having a next instruction is also a fail when the next instruction should be INSN_TRAP.

---
 tools/objtool/check.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/objtool/check.c +1 -1
index 3f6785415894..3354101ffe34 100644
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_RETURN:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after ret",
 					  insn->sec, insn->offset);
 			}
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
Quoted 10 lines

tools/objtool/check.c static int validate_branch(struct objtool_file *file, struct symbol *func, · line 3434

Quoted 7 lines
tools/objtool/check.c +1 -1
 		case INSN_JUMP_DYNAMIC:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after indirect jump",
 					  insn->sec, insn->offset);
 			}

My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this could be good for -urgent.

Quoted 2 lines

Urgh, I should've looked at that indeed. This didn't find any new sites though; so I don't think this needs to go through urgent.

Well to be fair, it was easy to miss since I snuck it in with rewrite.

Signature · 1 line
Josh
author maintainer
#4 lore raw
In reply to Josh Poimboeuf
Quoted 54 lines · Josh Poimboeuf

From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled

Having disabled retpolines doesn't necessarily mean the user doesn't care about straight-line speculation. For example, retpolines are disabled when eIBRS is used.

If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the retpolines disabled cases.

---
 arch/x86/include/asm/nospec-branch.h | 13 ++++++++++---
 tools/objtool/check.c                |  3 ---
 2 files changed, 10 insertions(+), 6 deletions(-)
arch/x86/include/asm/nospec-branch.h +10 -3
index acbaeaf83b61..0648746bf60b 100644
@@ -75,6 +75,13 @@
 	.popsection
 .endm
 
+.macro INDIRECT_JMP reg
+	jmp *%\reg
+#ifdef CONFIG_SLS
+	int3
+#endif
+.endm
+
 /*
  * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple
  * indirect jmp/call which may be susceptible to the Spectre variant 2
@@ -82,11 +89,11 @@
  */
 .macro JMP_NOSPEC reg:req
 #ifdef CONFIG_RETPOLINE
-	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
 		      __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \
-		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE
+		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
 #else
-	jmp	*%\reg
+	INDIRECT_JMP \reg
 #endif
 .endm
 
tools/objtool/check.c +0 -0
index e7983c3e2408..3cf3ad0b5db5 100644

tools/objtool/check.c static int validate_sls(struct objtool_file *file) · line 3847

Quoted 9 lines
tools/objtool/check.c +0 -3
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
 	for_each_insn(file, insn) {
 		next_insn = next_insn_same_sec(file, insn);
 
-		if (insn->retpoline_safe)
-			continue;
-
 		switch (insn->type) {
 		case INSN_RETURN:
 			if (!next_insn || next_insn->type != INSN_TRAP) {

Yes, agreed. But perhaps with something like this on top?

---

arch/x86/kernel/alternative.c +11 -0
index 3c66073e7645..84beeb5297d5 100644
@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
 		return ret;
 	i += ret;
 
+#ifdef CONFIG_SLS
+	/*
+	 * Ideally this would be unconditional, except in case of
+	 * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
+	 * -mharden-sls=all should be extended to emit INT3 after
+	 * direct jumps too, which will then cover that case.
+	 */
+	if (i < insn->length)
+		bytes[i++] = INT3_INSN_OPCODE;
+#endif
+
 	for (; i < insn->length;)
 		bytes[i++] = BYTES_NOP1;
 
arch/x86/lib/retpoline.S +2 -2
index b2b2366885a2..26e742da3129 100644
@@ -33,9 +33,9 @@ SYM_INNER_LABEL(__x86_indirect_thunk_\reg, SYM_L_GLOBAL)
 	UNWIND_HINT_EMPTY
 	ANNOTATE_NOENDBR
 
-	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
 		      __stringify(RETPOLINE \reg), X86_FEATURE_RETPOLINE, \
-		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg; int3), X86_FEATURE_RETPOLINE_LFENCE
+		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
 
 .endm
 
arch/x86/net/bpf_jit_comp.c +4 -0
index 16b6efacf7c6..6929856e7f6d 100644
@@ -418,6 +418,10 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)
 #endif
 	EMIT2(0xFF, 0xE0 + reg);
 
+#ifdef CONFIG_SLS
+	EMIT1(0xCC);
+#endif
+
 	*pprog = prog;
 }
 
#5 lore raw
Quoted 5 lines · Peter Zijlstra
Quoted 5 lines

+++ b/tools/objtool/check.c

(quoted diff) +0 -0
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
 	for_each_insn(file, insn) {
 		next_insn = next_insn_same_sec(file, insn);
 
Quoted 8 lines

Review static int validate_sls(struct objtool_file *file) · line 3847

Quoted 6 lines
(quoted diff) +0 -3
-		if (insn->retpoline_safe)
-			continue;
-
 		switch (insn->type) {
 		case INSN_RETURN:
 			if (!next_insn || next_insn->type != INSN_TRAP) {

Yes, agreed. But perhaps with something like this on top?

Yup, I missed those... Looks good. Just one comment:

arch/x86/kernel/alternative.c static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes) · line 461

Quoted 12 lines

+++ b/arch/x86/kernel/alternative.c

arch/x86/kernel/alternative.c +7 -0
@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
 		return ret;
 	i += ret;
 
+#ifdef CONFIG_SLS
+	/*
+	 * Ideally this would be unconditional, except in case of
+	 * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
+	 * -mharden-sls=all should be extended to emit INT3 after
+	 * direct jumps too, which will then cover that case.
+	 */

I don't quite follow this 2nd sentence and how it's related here, since this function doesn't actually deal with direct jumps.

Speaking of, I guess we'll eventually need to hack this SLS mess into jump labels :-/

Quoted 3 lines
arch/x86/kernel/alternative.c +3 -0
+	if (i < insn->length)
+		bytes[i++] = INT3_INSN_OPCODE;
+#endif
Signature · 1 line
Josh
author maintainer
#6 lore raw
Quoted 22 lines · Josh Poimboeuf
Quoted 5 lines · Peter Zijlstra
Quoted 5 lines

+++ b/tools/objtool/check.c

(quoted diff) +0 -0
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
 	for_each_insn(file, insn) {
 		next_insn = next_insn_same_sec(file, insn);
 
Quoted 8 lines

Review static int validate_sls(struct objtool_file *file) · line 3847

Quoted 6 lines
(quoted diff) +0 -3
-		if (insn->retpoline_safe)
-			continue;
-
 		switch (insn->type) {
 		case INSN_RETURN:
 			if (!next_insn || next_insn->type != INSN_TRAP) {

Yes, agreed. But perhaps with something like this on top?

Yup, I missed those... Looks good. Just one comment:

Quoted 5 lines

+++ b/arch/x86/kernel/alternative.c

arch/x86/kernel/alternative.c +0 -0
@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
 		return ret;
 	i += ret;
 
Quoted 10 lines

arch/x86/kernel/alternative.c static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes) · line 461

Quoted 7 lines
arch/x86/kernel/alternative.c +7 -0
+#ifdef CONFIG_SLS
+	/*
+	 * Ideally this would be unconditional, except in case of
+	 * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
+	 * -mharden-sls=all should be extended to emit INT3 after
+	 * direct jumps too, which will then cover that case.
+	 */

I don't quite follow this 2nd sentence and how it's related here, since this function doesn't actually deal with direct jumps.

Ah, my bad. Also, this wrong.

I suppose this wants to be something like:

	if (i < insn->length && op == JMP32_INSN_OPCODE)
		bytes[i++] = INT3_INSN_OPCODE;

So this *can* be a jump, but typically won't be I suppose.

Quoted 2 lines

Speaking of, I guess we'll eventually need to hack this SLS mess into jump labels :-/

Urgh... can't we reason that the straight line case is actually expected to run with the given register state anyway and ignore this?

maintainer
#7 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 2 lines

I don't quite follow this 2nd sentence and how it's related here, since this function doesn't actually deal with direct jumps.

Ah, my bad. Also, this wrong.

I suppose this wants to be something like:

	if (i < insn->length && op == JMP32_INSN_OPCODE)
		bytes[i++] = INT3_INSN_OPCODE;

So this *can* be a jump, but typically won't be I suppose.

Yep.

Quoted 5 lines
Quoted 2 lines

Speaking of, I guess we'll eventually need to hack this SLS mess into jump labels :-/

Urgh... can't we reason that the straight line case is actually expected to run with the given register state anyway and ignore this?

Yeah, that makes sense.

So for jump labels the SLS path would probably not be worse than a typical v1-style conditional branch misspeculation into the 'else' path, and we've already given up on worrying about those anyway.

Signature · 1 line
Josh