Skip to contentSkip to conversation

x86/kprobes: Fix to check not-kprobe's int3 correctly

PATCH Applied Masami Hiramatsu (Google) 3 participants 6 messages Dec 19, 2022 – Dec 27, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover x86/kprobes: Fix to check not-kprobe's int3 correctly

Hi,

Here is 4th version of the patches to fix kprobes and optprobe with CONFIG_RETHUNK and CONFIG_SLS.
Previous version is here;

https://lore.kernel.org/all/166264927154.775585.16570756675363838701.stgit@devnote2/

I just ported the v3 on the latest -tip tree.

With CONFIG_RETHUNK/CONFIG_SLS and after many efforts, the kernel functions may includes INT3 for stopping speculative execution in the function code block (body) in addition to the gaps between functions.

Since kprobes on x86 has to ensure the probe address is an instruction bondary, it decodes the instructions in the function until the address. If it finds an INT3 which is not embedded by kprobe, it stops decoding because usually the INT3 is used for debugging as a software breakpoint and such INT3 will replace the first byte of an original instruction. Thus the kprobes returns -EILSEQ as below.

 # echo 'p:probe/resched_curr_L21 resched_curr+98' >> dynamic_events 
 sh: write error: Invalid or incomplete multibyte or wide character

Actually, such INT3 can be ignored except the INT3 installed dynamically by kgdb.
To avoid this issue, just check whether the INT3 is owned by kgdb or not and if so, it just stopped and return failure.

With thses fixes, kprobe and optprobe can probe the kernel again with CONFIG_RETHUNK=y.

Thank you,

---

Masami Hiramatsu (Google) (2):
x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK

 arch/x86/kernel/kprobes/core.c |   10 +++++++---
 arch/x86/kernel/kprobes/opt.c  |   28 ++++++++--------------------
 2 files changed, 15 insertions(+), 23 deletions(-)
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
#1 lore raw

PATCH v4 1/2 x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK

From: Masami Hiramatsu (Google) <[email protected]>

Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after RET instruction, kprobes always failes to check the probed instruction boundary by decoding the function body if the probed address is after such sequence. (Note that some conditional code blocks will be placed after function return, if compiler decides it is not on the hot path.)

This is because kprobes expects kgdb puts the INT3 as a software breakpoint and it will replace the original instruction. But these INT3 are not such purpose, it doesn't need to recover the original instruction.

To avoid this issue, kprobes checks whether the INT3 is owned by kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.

Fixes e463a09af2f0 ("x86: Add straight-line-speculation mitigation")
---
 arch/x86/kernel/kprobes/core.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kernel/kprobes/core.c +7 -3
index 66299682b6b7..b36f3c367cb2 100644
@@ -37,6 +37,7 @@
 #include <linux/extable.h>
 #include <linux/kdebug.h>
 #include <linux/kallsyms.h>
+#include <linux/kgdb.h>
 #include <linux/ftrace.h>
 #include <linux/kasan.h>
 #include <linux/moduleloader.h>
@@ -281,12 +282,15 @@ static int can_probe(unsigned long paddr)
 		if (ret < 0)
 			return 0;
 
+#ifdef CONFIG_KGDB
 		/*
-		 * Another debugging subsystem might insert this breakpoint.
-		 * In that case, we can't recover it.
+		 * If there is a dynamically installed kgdb sw breakpoint,
+		 * this function should not be probed.
 		 */
-		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE)
+		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE &&
+		    kgdb_has_hit_break(addr))
 			return 0;
+#endif
 		addr += insn.length;
 	}
 
#2 lore raw

[tip: x86/urgent] x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1993bf97992df2d560287f3c4120eda57426843d
Gitweb:        https://git.kernel.org/tip/1993bf97992df2d560287f3c4120eda57426843d
Author:        Masami Hiramatsu (Google) <[email protected]>
AuthorDate:    Mon, 19 Dec 2022 23:35:10 +09:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 27 Dec 2022 12:51:58 +01:00

x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK

Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after RET instruction, kprobes always failes to check the probed instruction boundary by decoding the function body if the probed address is after such sequence. (Note that some conditional code blocks will be placed after function return, if compiler decides it is not on the hot path.)

This is because kprobes expects kgdb puts the INT3 as a software breakpoint and it will replace the original instruction. But these INT3 are not such purpose, it doesn't need to recover the original instruction.

To avoid this issue, kprobes checks whether the INT3 is owned by kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.

---
 arch/x86/kernel/kprobes/core.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kernel/kprobes/core.c +7 -3
index 6629968..b36f3c3 100644
@@ -37,6 +37,7 @@
 #include <linux/extable.h>
 #include <linux/kdebug.h>
 #include <linux/kallsyms.h>
+#include <linux/kgdb.h>
 #include <linux/ftrace.h>
 #include <linux/kasan.h>
 #include <linux/moduleloader.h>
@@ -281,12 +282,15 @@ static int can_probe(unsigned long paddr)
 		if (ret < 0)
 			return 0;
 
+#ifdef CONFIG_KGDB
 		/*
-		 * Another debugging subsystem might insert this breakpoint.
-		 * In that case, we can't recover it.
+		 * If there is a dynamically installed kgdb sw breakpoint,
+		 * this function should not be probed.
 		 */
-		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE)
+		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE &&
+		    kgdb_has_hit_break(addr))
 			return 0;
+#endif
 		addr += insn.length;
 	}
 
#3 lore raw

PATCH v4 2/2 x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK

In reply to Masami Hiramatsu (Google)

From: Masami Hiramatsu (Google) <[email protected]>

Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after function return, kprobe jump optimization always fails on the functions with such INT3 inside the function body. (It already checks the INT3 padding between functions, but not inside
the function)

To avoid this issue, as same as kprobes, check whether the INT3 comes from kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.

Fixes e463a09af2f0 ("x86: Add straight-line-speculation mitigation")
---
 arch/x86/kernel/kprobes/opt.c |   28 ++++++++--------------------
 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/kprobes/opt.c +8 -20
index e6b8c5362b94..e57e07b0edb6 100644
@@ -15,6 +15,7 @@
 #include <linux/extable.h>
 #include <linux/kdebug.h>
 #include <linux/kallsyms.h>
+#include <linux/kgdb.h>
 #include <linux/ftrace.h>
 #include <linux/objtool.h>
 #include <linux/pgtable.h>
@@ -279,19 +280,6 @@ static int insn_is_indirect_jump(struct insn *insn)
 	return ret;
 }
 
-static bool is_padding_int3(unsigned long addr, unsigned long eaddr)
-{
-	unsigned char ops;
-
-	for (; addr < eaddr; addr++) {
-		if (get_kernel_nofault(ops, (void *)addr) < 0 ||
-		    ops != INT3_INSN_OPCODE)
-			return false;
-	}
-
-	return true;
-}
-
 /* Decode whole function to ensure any instructions don't jump into target */
 static int can_optimize(unsigned long paddr)
 {
@@ -334,15 +322,15 @@ static int can_optimize(unsigned long paddr)
 		ret = insn_decode_kernel(&insn, (void *)recovered_insn);
 		if (ret < 0)
 			return 0;
-
+#ifdef CONFIG_KGDB
 		/*
-		 * In the case of detecting unknown breakpoint, this could be
-		 * a padding INT3 between functions. Let's check that all the
-		 * rest of the bytes are also INT3.
+		 * If there is a dynamically installed kgdb sw breakpoint,
+		 * this function should not be probed.
 		 */
-		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE)
-			return is_padding_int3(addr, paddr - offset + size) ? 1 : 0;
-
+		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE &&
+		    kgdb_has_hit_break(addr))
+			return 0;
+#endif
 		/* Recover address */
 		insn.kaddr = (void *)addr;
 		insn.next_byte = (void *)(addr + insn.length);
#4 lore raw

[tip: x86/urgent] x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     63dc6325ff41ee9e570bde705ac34a39c5dbeb44
Gitweb:        https://git.kernel.org/tip/63dc6325ff41ee9e570bde705ac34a39c5dbeb44
Author:        Masami Hiramatsu (Google) <[email protected]>
AuthorDate:    Mon, 19 Dec 2022 23:35:19 +09:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 27 Dec 2022 12:51:58 +01:00

x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK

Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after function return, kprobe jump optimization always fails on the functions with such INT3 inside the function body. (It already checks the INT3 padding between functions, but not inside
the function)

To avoid this issue, as same as kprobes, check whether the INT3 comes from kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.

---
 arch/x86/kernel/kprobes/opt.c | 28 ++++++++--------------------
 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/kprobes/opt.c +8 -20
index e6b8c53..e57e07b 100644
@@ -15,6 +15,7 @@
 #include <linux/extable.h>
 #include <linux/kdebug.h>
 #include <linux/kallsyms.h>
+#include <linux/kgdb.h>
 #include <linux/ftrace.h>
 #include <linux/objtool.h>
 #include <linux/pgtable.h>
@@ -279,19 +280,6 @@ static int insn_is_indirect_jump(struct insn *insn)
 	return ret;
 }
 
-static bool is_padding_int3(unsigned long addr, unsigned long eaddr)
-{
-	unsigned char ops;
-
-	for (; addr < eaddr; addr++) {
-		if (get_kernel_nofault(ops, (void *)addr) < 0 ||
-		    ops != INT3_INSN_OPCODE)
-			return false;
-	}
-
-	return true;
-}
-
 /* Decode whole function to ensure any instructions don't jump into target */
 static int can_optimize(unsigned long paddr)
 {
@@ -334,15 +322,15 @@ static int can_optimize(unsigned long paddr)
 		ret = insn_decode_kernel(&insn, (void *)recovered_insn);
 		if (ret < 0)
 			return 0;
-
+#ifdef CONFIG_KGDB
 		/*
-		 * In the case of detecting unknown breakpoint, this could be
-		 * a padding INT3 between functions. Let's check that all the
-		 * rest of the bytes are also INT3.
+		 * If there is a dynamically installed kgdb sw breakpoint,
+		 * this function should not be probed.
 		 */
-		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE)
-			return is_padding_int3(addr, paddr - offset + size) ? 1 : 0;
-
+		if (insn.opcode.bytes[0] == INT3_INSN_OPCODE &&
+		    kgdb_has_hit_break(addr))
+			return 0;
+#endif
 		/* Recover address */
 		insn.kaddr = (void *)addr;
 		insn.next_byte = (void *)(addr + insn.length);
#5 lore raw
In reply to Masami Hiramatsu (Google)
Quoted 4 lines · Masami Hiramatsu (Google)

Masami Hiramatsu (Google) (2):
x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK

Thanks, I'll go queue these in x86/urgent after -rc1 happens.