Skip to contentSkip to conversation

kgdb: x86: fix breakpoint removal problems

PATCH Superseded Florian Rommel 2 participants 5 messages Aug 11, 2024 – Aug 14, 2024

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover kgdb: x86: fix breakpoint removal problems

This series fixes two problems with KGDB on x86 concerning the removal of breakpoints, causing the kernel to hang. Note that breakpoint removal is not only performed when explicitly deleting a breakpoint, but also happens before continuing execution or single stepping.

Florian Rommel (2):
x86/kgdb: convert early breakpoints to poke breakpoints x86/kgdb: fix hang on failed breakpoint removal

 arch/x86/kernel/kgdb.c    | 38 ++++++++++++++++++++++++++++++++++++++
 include/linux/kgdb.h      |  3 +++
 init/main.c               |  1 +
 kernel/debug/debug_core.c |  7 ++++++-
 4 files changed, 48 insertions(+), 1 deletion(-)
Signature · 1 line
2.46.0
#1 lore raw

PATCH 1/2 x86/kgdb: convert early breakpoints to poke breakpoints

On x86, after booting, the kernel text is read-only. Then, KGDB has to use the text_poke mechanism to install software breakpoints. KGDB uses a special (x86-specific) breakpoint type for these kinds of breakpoints (BP_POKE_BREAKPOINT). When removing a breakpoint, KGDB always adheres to the breakpoint's original installment method, which is determined by its type.

Before this fix, early (non-"poke") breakpoints could not be removed after the kernel text was set as read-only since the original code patching mechanism was no longer allowed to remove the breakpoints. Eventually, this even caused the kernel to hang (loop between int3 instruction and the function kgdb_skipexception).

With this patch, we convert early breakpoints to "poke" breakpoints after the kernel text has been made read-only. This makes them removable later.

---
A patch for this problem has already been proposed by Stefan Saecherl and Lorena Kretzschmar [1]. Their solution is different from the one I suggest here (it fixes the problem on removal, not "in advance"). Unfortunately, Lorena and Stefan's patch has not been accepted / the conversation has fallen asleep. One point of criticism concerned possible problems with reused init code pages. This should not be a problem with my patch.
[1] https://lore.kernel.org/all/[email protected]/

 arch/x86/kernel/kgdb.c    | 14 ++++++++++++++
 include/linux/kgdb.h      |  3 +++
 init/main.c               |  1 +
 kernel/debug/debug_core.c |  7 ++++++-
 4 files changed, 24 insertions(+), 1 deletion(-)
arch/x86/kernel/kgdb.c +14 -0
index 9c9faa1634fb..64c332151af7 100644
@@ -623,6 +623,20 @@ int kgdb_arch_init(void)
 	return retval;
 }
 
+void kgdb_after_mark_readonly(void)
+{
+	int i;
+
+	/* Convert all breakpoints in rodata to BP_POKE_BREAKPOINT. */
+	for (i = 0; i < KGDB_MAX_BREAKPOINTS; i++) {
+		if (kgdb_break[i].state != BP_UNDEFINED &&
+		    kgdb_break[i].type == BP_BREAKPOINT &&
+		    is_kernel_text(kgdb_break[i].bpt_addr)) {
+			kgdb_break[i].type = BP_POKE_BREAKPOINT;
+		}
+	}
+}
+
 static void kgdb_hw_overflow_handler(struct perf_event *event,
 		struct perf_sample_data *data, struct pt_regs *regs)
 {
include/linux/kgdb.h +3 -0
index 76e891ee9e37..c07855d388b5 100644
@@ -98,6 +98,8 @@ extern int dbg_set_reg(int regno, void *mem, struct pt_regs *regs);
 # define KGDB_MAX_BREAKPOINTS	1000
 #endif
 
+extern struct kgdb_bkpt kgdb_break[KGDB_MAX_BREAKPOINTS];
+
 #define KGDB_HW_BREAKPOINT	1
 
 /*
@@ -360,6 +362,7 @@ extern bool dbg_is_early;
 extern void __init dbg_late_init(void);
 extern void kgdb_panic(const char *msg);
 extern void kgdb_free_init_mem(void);
+extern void kgdb_after_mark_readonly(void);
 #else /* ! CONFIG_KGDB */
 #define in_dbg_master() (0)
 #define dbg_late_init()
init/main.c +1 -0
index 206acdde51f5..33b6e092fed3 100644
@@ -1441,6 +1441,7 @@ static void mark_readonly(void)
 		mark_rodata_ro();
 		debug_checkwx();
 		rodata_test();
+		kgdb_after_mark_readonly();
 	} else if (IS_ENABLED(CONFIG_STRICT_KERNEL_RWX)) {
 		pr_info("Kernel memory protection disabled.\n");
 	} else if (IS_ENABLED(CONFIG_ARCH_HAS_STRICT_KERNEL_RWX)) {
kernel/debug/debug_core.c +6 -1
index ce1bb2301c06..9dd6b69f1679 100644
@@ -98,7 +98,7 @@ module_param(kgdbreboot, int, 0644);
  * Holds information about breakpoints in a kernel. These breakpoints are
  * added and removed by gdb.
  */
-static struct kgdb_bkpt		kgdb_break[KGDB_MAX_BREAKPOINTS] = {
+struct kgdb_bkpt		kgdb_break[KGDB_MAX_BREAKPOINTS] = {
 	[0 ... KGDB_MAX_BREAKPOINTS-1] = { .state = BP_UNDEFINED }
 };
 
@@ -452,6 +452,11 @@ void kgdb_free_init_mem(void)
 	}
 }
 
+void __weak kgdb_after_mark_readonly(void)
+{
+	/* Weak implementation, may be overridden by arch code */
+}
+
 #ifdef CONFIG_KGDB_KDB
 void kdb_dump_stack_on_cpu(int cpu)
 {
Signature · 1 line
2.46.0
#2 lore raw

PATCH 2/2 x86/kgdb: fix hang on failed breakpoint removal

In reply to Florian Rommel

On x86, occasionally, the removal of a breakpoint (i.e., removal of the int3 instruction) fails because the text_mutex is taken by another CPU (mainly due to the static_key mechanism, I think). The function kgdb_skipexception catches exceptions from these spurious int3 instructions, bails out of KGDB, and continues execution from the previous PC address.

However, this led to an endless loop between the int3 instruction and kgdb_skipexception since the int3 instruction (being still present) triggered again. This effectively caused the system to hang.

With this patch, we try to remove the concerned spurious int3 instruction in kgdb_skipexception before continuing execution. This may take a few attempts until the concurrent holders of the text_mutex have released it, but eventually succeeds and the kernel can continue.

---
 arch/x86/kernel/kgdb.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
arch/x86/kernel/kgdb.c +24 -0
index 64c332151af7..585a7a72af74 100644
@@ -723,7 +723,31 @@ void kgdb_arch_exit(void)
 int kgdb_skipexception(int exception, struct pt_regs *regs)
 {
 	if (exception == 3 && kgdb_isremovedbreak(regs->ip - 1)) {
+		struct kgdb_bkpt *bpt;
+		int i, error;
+
 		regs->ip -= 1;
+
+		/*
+		 * Try to remove the spurious int3 instruction.
+		 * These int3s can result from failed breakpoint removals
+		 * in kgdb_arch_remove_breakpoint.
+		 */
+		for (bpt = NULL, i = 0; i < KGDB_MAX_BREAKPOINTS; i++) {
+			if (kgdb_break[i].bpt_addr == regs->ip &&
+			    kgdb_break[i].state == BP_REMOVED &&
+			    (kgdb_break[i].type == BP_BREAKPOINT ||
+			     kgdb_break[i].type == BP_POKE_BREAKPOINT)) {
+				bpt = &kgdb_break[i];
+				break;
+			}
+		}
+		if (!bpt)
+			return 1;
+		error = kgdb_arch_remove_breakpoint(bpt);
+		if (error)
+			pr_err("skipexception: breakpoint remove failed: %lx\n",
+			       bpt->bpt_addr);
 		return 1;
 	}
 	return 0;
Signature · 1 line
2.46.0
#3 lore raw
In reply to Florian Rommel
Quoted 4 lines · On Mon, 12 Aug 2024 01:22:06 +0200 Florian Rommel <[email protected]> wrote:

This series fixes two problems with KGDB on x86 concerning the removal of breakpoints, causing the kernel to hang. Note that breakpoint removal is not only performed when explicitly deleting a breakpoint, but also happens before continuing execution or single stepping.

Neat. It would be nice to fix earlier kernels; for that it is desirable to identify a Fixes: target. From a quick look it appears this issue is more than a decade old, in which case I don't believe a
Fixes: is needed - our request becomes "please backport to everything".

#4 lore raw
Quoted 7 lines · Andrew Morton
Quoted 4 lines · On Mon, 12 Aug 2024 01:22:06 +0200 Florian Rommel <[email protected]> wrote:

This series fixes two problems with KGDB on x86 concerning the removal of breakpoints, causing the kernel to hang. Note that breakpoint removal is not only performed when explicitly deleting a breakpoint, but also happens before continuing execution or single stepping.

Quoted 5 lines

Neat. It would be nice to fix earlier kernels; for that it is desirable to identify a Fixes: target. From a quick look it appears this issue is more than a decade old, in which case I don't believe a
Fixes: is needed - our request becomes "please backport to everything".

Thanks. There is already a v2 (due to my negligence on the details) and a bit of discussion:
https://lore.kernel.org/all/[email protected]/

Rgeards,
Florian