x86/kprobes: Fix to check not-kprobe's int3 correctly
Subsystems (3)
v4 cover x86/kprobes: Fix to check not-kprobe's int3 correctly
Hi,
Here is 4th version of the patches to fix kprobes and optprobe with CONFIG_RETHUNK and CONFIG_SLS.
Previous version is here;
https://lore.kernel.org/all/166264927154.775585.16570756675363838701.stgit@devnote2/
I just ported the v3 on the latest -tip tree.
With CONFIG_RETHUNK/CONFIG_SLS and after many efforts, the kernel functions may includes INT3 for stopping speculative execution in the function code block (body) in addition to the gaps between functions.
Since kprobes on x86 has to ensure the probe address is an instruction bondary, it decodes the instructions in the function until the address. If it finds an INT3 which is not embedded by kprobe, it stops decoding because usually the INT3 is used for debugging as a software breakpoint and such INT3 will replace the first byte of an original instruction. Thus the kprobes returns -EILSEQ as below.
# echo 'p:probe/resched_curr_L21 resched_curr+98' >> dynamic_events sh: write error: Invalid or incomplete multibyte or wide character
Actually, such INT3 can be ignored except the INT3 installed dynamically by kgdb.
To avoid this issue, just check whether the INT3 is owned by kgdb or not and if so, it just stopped and return failure.
With thses fixes, kprobe and optprobe can probe the kernel again with CONFIG_RETHUNK=y.
Thank you,
---
Masami Hiramatsu (Google) (2):
x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK
arch/x86/kernel/kprobes/core.c | 10 +++++++--- arch/x86/kernel/kprobes/opt.c | 28 ++++++++-------------------- 2 files changed, 15 insertions(+), 23 deletions(-)
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
PATCH v4 1/2 x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK
From: Masami Hiramatsu (Google) <[email protected]>
Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after RET instruction, kprobes always failes to check the probed instruction boundary by decoding the function body if the probed address is after such sequence. (Note that some conditional code blocks will be placed after function return, if compiler decides it is not on the hot path.)
This is because kprobes expects kgdb puts the INT3 as a software breakpoint and it will replace the original instruction. But these INT3 are not such purpose, it doesn't need to recover the original instruction.
To avoid this issue, kprobes checks whether the INT3 is owned by kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.
--- arch/x86/kernel/kprobes/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kernel/kprobes/core.c +7 -3
@@ -37,6 +37,7 @@#include <linux/extable.h> #include <linux/kdebug.h> #include <linux/kallsyms.h> +#include <linux/kgdb.h> #include <linux/ftrace.h> #include <linux/kasan.h> #include <linux/moduleloader.h>
@@ -281,12 +282,15 @@ static int can_probe(unsigned long paddr)if (ret <0 )return 0 ; +#ifdef CONFIG_KGDB /* -* Another debugging subsystem might insert this breakpoint. -* In that case, we can't recover it. +* If there is a dynamically installed kgdb sw breakpoint, +* this function should not be probed. */ -if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE ) +if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE && +kgdb_has_hit_break (addr))return 0 ; +#endif addr += insn.length; }
[tip: x86/urgent] x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK
The following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 1993bf97992df2d560287f3c4120eda57426843d Gitweb: https://git.kernel.org/tip/1993bf97992df2d560287f3c4120eda57426843d Author: Masami Hiramatsu (Google) <[email protected]> AuthorDate: Mon, 19 Dec 2022 23:35:10 +09:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 27 Dec 2022 12:51:58 +01:00
x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK
Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after RET instruction, kprobes always failes to check the probed instruction boundary by decoding the function body if the probed address is after such sequence. (Note that some conditional code blocks will be placed after function return, if compiler decides it is not on the hot path.)
This is because kprobes expects kgdb puts the INT3 as a software breakpoint and it will replace the original instruction. But these INT3 are not such purpose, it doesn't need to recover the original instruction.
To avoid this issue, kprobes checks whether the INT3 is owned by kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.
--- arch/x86/kernel/kprobes/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kernel/kprobes/core.c +7 -3
@@ -37,6 +37,7 @@#include <linux/extable.h> #include <linux/kdebug.h> #include <linux/kallsyms.h> +#include <linux/kgdb.h> #include <linux/ftrace.h> #include <linux/kasan.h> #include <linux/moduleloader.h>
@@ -281,12 +282,15 @@ static int can_probe(unsigned long paddr)if (ret <0 )return 0 ; +#ifdef CONFIG_KGDB /* -* Another debugging subsystem might insert this breakpoint. -* In that case, we can't recover it. +* If there is a dynamically installed kgdb sw breakpoint, +* this function should not be probed. */ -if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE ) +if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE && +kgdb_has_hit_break (addr))return 0 ; +#endif addr += insn.length; }
PATCH v4 2/2 x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK
In reply to Masami Hiramatsu (Google)From: Masami Hiramatsu (Google) <[email protected]>
Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after function return, kprobe jump optimization always fails on the functions with such INT3 inside the function body. (It already checks the INT3 padding between functions, but not inside
the function)
To avoid this issue, as same as kprobes, check whether the INT3 comes from kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.
--- arch/x86/kernel/kprobes/opt.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/kprobes/opt.c +8 -20
@@ -15,6 +15,7 @@#include <linux/extable.h> #include <linux/kdebug.h> #include <linux/kallsyms.h> +#include <linux/kgdb.h> #include <linux/ftrace.h> #include <linux/objtool.h> #include <linux/pgtable.h>
@@ -279,19 +280,6 @@ static int insn_is_indirect_jump(struct insn *insn)return ret; } -static bool is_padding_int3 (unsigned long addr,unsigned long eaddr) -{ -unsigned char ops; - -for (; addr < eaddr; addr++) { -if (get_kernel_nofault (ops, (void *)addr) <0 || - ops !=INT3_INSN_OPCODE ) -return false; - } - -return true; -} -/* Decode whole function to ensure any instructions don't jump into target */ static int can_optimize (unsigned long paddr) {
@@ -334,15 +322,15 @@ static int can_optimize(unsigned long paddr) ret =insn_decode_kernel (&insn, (void *)recovered_insn);if (ret <0 )return 0 ; - +#ifdef CONFIG_KGDB /* -* In the case of detecting unknown breakpoint, this could be -* a padding INT3 between functions. Let's check that all the -* rest of the bytes are also INT3. +* If there is a dynamically installed kgdb sw breakpoint, +* this function should not be probed. */ -if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE ) -return is_padding_int3 (addr, paddr - offset + size) ?1 :0 ; - +if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE && +kgdb_has_hit_break (addr)) +return 0 ; +#endif /* Recover address */ insn.kaddr = (void *)addr; insn.next_byte = (void *)(addr + insn.length);
[tip: x86/urgent] x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK
The following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 63dc6325ff41ee9e570bde705ac34a39c5dbeb44 Gitweb: https://git.kernel.org/tip/63dc6325ff41ee9e570bde705ac34a39c5dbeb44 Author: Masami Hiramatsu (Google) <[email protected]> AuthorDate: Mon, 19 Dec 2022 23:35:19 +09:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 27 Dec 2022 12:51:58 +01:00
x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK
Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after function return, kprobe jump optimization always fails on the functions with such INT3 inside the function body. (It already checks the INT3 padding between functions, but not inside
the function)
To avoid this issue, as same as kprobes, check whether the INT3 comes from kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction.
--- arch/x86/kernel/kprobes/opt.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-)
arch/x86/kernel/kprobes/opt.c +8 -20
@@ -15,6 +15,7 @@#include <linux/extable.h> #include <linux/kdebug.h> #include <linux/kallsyms.h> +#include <linux/kgdb.h> #include <linux/ftrace.h> #include <linux/objtool.h> #include <linux/pgtable.h>
@@ -279,19 +280,6 @@ static int insn_is_indirect_jump(struct insn *insn)return ret; } -static bool is_padding_int3 (unsigned long addr,unsigned long eaddr) -{ -unsigned char ops; - -for (; addr < eaddr; addr++) { -if (get_kernel_nofault (ops, (void *)addr) <0 || - ops !=INT3_INSN_OPCODE ) -return false; - } - -return true; -} -/* Decode whole function to ensure any instructions don't jump into target */ static int can_optimize (unsigned long paddr) {
@@ -334,15 +322,15 @@ static int can_optimize(unsigned long paddr) ret =insn_decode_kernel (&insn, (void *)recovered_insn);if (ret <0 )return 0 ; - +#ifdef CONFIG_KGDB /* -* In the case of detecting unknown breakpoint, this could be -* a padding INT3 between functions. Let's check that all the -* rest of the bytes are also INT3. +* If there is a dynamically installed kgdb sw breakpoint, +* this function should not be probed. */ -if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE ) -return is_padding_int3 (addr, paddr - offset + size) ?1 :0 ; - +if (insn.opcode.bytes[0 ] ==INT3_INSN_OPCODE && +kgdb_has_hit_break (addr)) +return 0 ; +#endif /* Recover address */ insn.kaddr = (void *)addr; insn.next_byte = (void *)(addr + insn.length);
Quoted 4 lines · Masami Hiramatsu (Google)
Masami Hiramatsu (Google) (2):
x86/kprobes: Fix kprobes instruction boudary check withCONFIG_RETHUNKx86/kprobes: Fix optprobe optimization check withCONFIG_RETHUNK
Thanks, I'll go queue these in x86/urgent after -rc1 happens.