x86: Catch stray non-kprobe breakpoints

Greetings,

Non kprobe breakpoints in the kernel might lie inside the .kprobes.text section. Such breakpoints can easily be identified by in_kprobes_functions and can be caught early. These are problematic and a warning should be emitted to discourage them (in any rare case, if they actually occur).

For this, a check can route the trap handling of such breakpoints away from kprobe_handler (which ends up calling even more functions marked as __kprobes) from inside kprobe_exceptions_notify.

Quoted 1 line

Non kprobe breakpoints in the kernel might lie inside the .kprobes.text section. Such breakpoints can easily be identified by in_kprobes_functions and can be caught early. These are problematic and a warning should be emitted to discourage them (in any rare case, if they actually occur).

Why? As Masami indicated in an earlier reply, the annotation is to prevent *only* kprobes.

Quoted 1 line

For this, a check can route the trap handling of such breakpoints away from kprobe_handler (which ends up calling even more functions marked as __kprobes) from inside kprobe_exceptions_notify.

Well.. we pass on control of a !kprobe breakpoint to the kernel. This is exactly what permits debuggers like xmon to work fine now.

I don't see any harm in such breakpoints being handled autonomously without any sort of kprobe influence.

Ananth

Quoted 4 lines · Ananth N Mavinakayanahalli

Quoted 1 line

Non kprobe breakpoints in the kernel might lie inside the .kprobes.text section. Such breakpoints can easily be identified by in_kprobes_functions and can be caught early. These are problematic and a warning should be emitted to discourage them (in any rare case, if they actually occur).

Why? As Masami indicated in an earlier reply, the annotation is to prevent *only* kprobes.

May be I'm completely off the mark here, but shouldn't a small subset of this section simply be 'breakpoint-free' rather than 'kprobe-free'? Placing a breakpoint on kprobe_handler (say) can loop into a recursive trap without allowing the debugger's notifier chain to be invoked.

I'm assuming that non-kprobe exception notifiers may (or even should) run after kprobe's notifier callback (kprobe_exceptions_notify).

Quoted 4 lines

Quoted 1 line

For this, a check can route the trap handling of such breakpoints away from kprobe_handler (which ends up calling even more functions marked as __kprobes) from inside kprobe_exceptions_notify.

Well.. we pass on control of a !kprobe breakpoint to the kernel. This is exactly what permits debuggers like xmon to work fine now.

This will still happen. It doesn't stop non-kprobe breakpoints from being handled, wherever they may be.

Quoted 2 lines

I don't see any harm in such breakpoints being handled autonomously without any sort of kprobe influence.

Here's what seems to be happening currently:

int3 (non-kprobe) -> do_int3 ->kprobe_exceptions_notify -> kprobe_handler (passes the buck to the kernel) -> non-krpobe/debugger exception handler.

Here's what the patch will do:

int3 (non-kprobe) -> do_int3 ->kprobe_exceptions_notify -> WARN_ON/kprobe_handler -> non-kprobe/debugger exception handler.

The WARN_ON (and not a BUG_ON) will be hit iff:
(in_kprobes_functions(addr) && !is_jprobe_bkpt(addr))

Placing a breakpoint on kprobe_handler (say) can loop into a recursive trap without allowing the debugger's notifier chain to be invoked. I'm assuming that non-kprobe exception notifiers may (or even should) run after kprobe's notifier callback (kprobe_exceptions_notify).

Quoted 4 lines

May be I'm completely off the mark here, but shouldn't a small subset of this section simply be 'breakpoint-free' rather than 'kprobe-free'? Placing a breakpoint on kprobe_handler (say) can loop into a recursive trap without allowing the debugger's notifier chain to be invoked.

A well heeled debugger will necessarily take care of saving contexts (using techniques like setjmp/longjmp, etc) to help it recover from such nested cases (See xmon for example).

Quoted 2 lines

I'm assuming that non-kprobe exception notifiers may (or even should) run after kprobe's notifier callback (kprobe_exceptions_notify).

Yes, any such notifier is invoked after kprobe's callback as the kprobe notifier is always registered with the highest priority.

Quoted 2 lines

The WARN_ON (and not a BUG_ON) will be hit iff:
(in_kprobes_functions(addr) && !is_jprobe_bkpt(addr))

But that still is unneeded dmesg clutter, IMHO.

Ananth

In that case, why don't you just reduce the priority of kprobe_exceptions_nb? Then, the execution path becomes very simple.

I also like to use a debugger for debugging kprobes. that will help us.

Best Regards,

--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America) Inc.
Software Solutions Division

e-mail: [email protected]

Quoted 2 lines · Masami Hiramatsu

In that case, why don't you just reduce the priority of kprobe_exceptions_nb? Then, the execution path becomes very simple.

Ananth mentioned that the kprobe notifier has to be the first to run.

It still wouldnt allow us to notice breakpoints on places like do_int3 etc.

Quoted 1 line

I also like to use a debugger for debugging kprobes. that will help us.

Hmm...It would increase the code-path leading upto kprobe_handler. That's more territory to be guarded from kprobes.

Quoted 5 lines · Abhishek Sagar

Quoted 2 lines · Masami Hiramatsu

In that case, why don't you just reduce the priority of kprobe_exceptions_nb? Then, the execution path becomes very simple.

Ananth mentioned that the kprobe notifier has to be the first to run.

(Hmm.. I think he has just explained current implementation:)) IMHO, since kprobes itself can not know what the external debugger wants to do, the highest priority should be reserved for those external tools.