stackleak: Disable ftrace for stackleak.c
PATCH 1/1 stackleak: Disable ftrace for stackleak.c
The stackleak_erase() function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enable kprobe_events for stackleak_erase().
Let's disable ftrace for stackleak.c to avoid such situations.
--- kernel/Makefile | 1 + 1 file changed, 1 insertion(+)
kernel/Makefile +1 -0
@@ -18,6 +18,7 @@ obj-$(CONFIG_MULTIUSER) += groups.o ifdef CONFIG_FUNCTION_TRACER # Do not trace internal ftrace files CFLAGS_REMOVE_irq_work.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_stackleak.o = $(CC_FLAGS_FTRACE) endif # Prevents flicker of uninteresting __do_softirq()/__local_bh_disable_ip()
Signature · 1 line
2.7.4
Quoted 3 lines · On Sun, 11 Nov 2018 01:05:30 +0300 Alexander Popov <[email protected]> wrote:
The
stackleak_erase()function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enablekprobe_eventsforstackleak_erase().
Is the issue with kprobes or with function tracing? Because this stops function tracing which I only want disabled if function tracing itself is an issue, not for other things that may use the function tracing infrastructure.
-- Steve
Quoted 11 lines · Steven Rostedt
Quoted 3 lines · On Sun, 11 Nov 2018 01:05:30 +0300 Alexander Popov <[email protected]> wrote:
The
stackleak_erase()function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enablekprobe_eventsforstackleak_erase().Is the issue with kprobes or with function tracing? Because this stops function tracing which I only want disabled if function tracing itself is an issue, not for other things that may use the function tracing infrastructure.
Hello Steven,
I believe that stackleak erasing is not compatible with function tracing itself. That's what the kernel testing robot has hit:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/1
I used kprobe_events just to reproduce the problem:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/4
Best regards,
Alexander
Quoted 13 lines · On Sun, 11 Nov 2018 13:19:45 +0300 Alexander Popov <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 3 lines · On Sun, 11 Nov 2018 01:05:30 +0300 Alexander Popov <[email protected]> wrote:
The
stackleak_erase()function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enablekprobe_eventsforstackleak_erase().Is the issue with kprobes or with function tracing? Because this stops function tracing which I only want disabled if function tracing itself is an issue, not for other things that may use the function tracing infrastructure.
Quoted 8 lines
Hello Steven,
I believe that stackleak erasing is not compatible with function tracing itself. That's what the kernel testing robot has hit:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/1I used
kprobe_eventsjust to reproduce the problem:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/4
Have you tried adding a "notrace" to stackleak_erase()?
Not tracing the entire file is a bit of overkill. There's no reason ftrace can't trace stack_erasing_sysctl() or perhaps even stackleak_track_stack() as that may be very interesting to trace.
-- Steve
Hi Alexander and Steve,
Quoted 22 lines · On Sun, 11 Nov 2018 20:53:51 -0500 Steven Rostedt <[email protected]> wrote:
Quoted 19 lines · On Sun, 11 Nov 2018 13:19:45 +0300 Alexander Popov <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 3 lines · On Sun, 11 Nov 2018 01:05:30 +0300 Alexander Popov <[email protected]> wrote:
The
stackleak_erase()function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enablekprobe_eventsforstackleak_erase().Is the issue with kprobes or with function tracing? Because this stops function tracing which I only want disabled if function tracing itself is an issue, not for other things that may use the function tracing infrastructure.
Hello Steven,
I believe that stackleak erasing is not compatible with function tracing itself. That's what the kernel testing robot has hit:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/1
Quoted 8 lines
Quoted 2 lines
I used
kprobe_eventsjust to reproduce the problem:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/4Have you tried adding a "notrace" to
stackleak_erase()?Not tracing the entire file is a bit of overkill. There's no reason ftrace can't trace
stack_erasing_sysctl()or perhaps evenstackleak_track_stack()as that may be very interesting to trace.
I think it is not enough for stopping kprobes. If you want to stop the kprobes (int3 version) on stackleak_erase(), you should use NOKPROBE_SYMBOL(stackleak_erase), since kprobes can work without ftrace.
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 32 lines · ----- On Nov 11, 2018, at 9:50 PM, Masami Hiramatsu [email protected] wrote:
Hi Alexander and Steve,
Quoted 27 lines · On Sun, 11 Nov 2018 20:53:51 -0500 Steven Rostedt <[email protected]> wrote:
Quoted 13 lines · On Sun, 11 Nov 2018 13:19:45 +0300 Alexander Popov <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 3 lines · On Sun, 11 Nov 2018 01:05:30 +0300 Alexander Popov <[email protected]> wrote:
The
stackleak_erase()function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enablekprobe_eventsforstackleak_erase().Is the issue with kprobes or with function tracing? Because this stops function tracing which I only want disabled if function tracing itself is an issue, not for other things that may use the function tracing infrastructure.
Quoted 8 lines
Hello Steven,
I believe that stackleak erasing is not compatible with function tracing itself. That's what the kernel testing robot has hit:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/1I used
kprobe_eventsjust to reproduce the problem:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/4Have you tried adding a "notrace" to
stackleak_erase()?
Quoted 8 lines
Quoted 3 lines
Not tracing the entire file is a bit of overkill. There's no reason ftrace can't trace
stack_erasing_sysctl()or perhaps evenstackleak_track_stack()as that may be very interesting to trace.I think it is not enough for stopping kprobes. If you want to stop the kprobes (int3 version) on
stackleak_erase(), you should useNOKPROBE_SYMBOL(stackleak_erase),
since kprobes can work without ftrace.
Just to clarify: AFAIU you guys are recommending to add _both_ a "notrace" annotation to stackleak_erase() _and_ a NOKPROBE_SYMBOL(stackleak_erase), so neither function tracing nor kprobes can hook on that function.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 3 lines · On Mon, 12 Nov 2018 09:53:29 -0500 (EST) Mathieu Desnoyers…
Just to clarify: AFAIU you guys are recommending to add _both_ a "notrace" annotation to
stackleak_erase()_and_ aNOKPROBE_SYMBOL(stackleak_erase), so neither function tracing nor kprobes can hook on that function.
Correct.
-- Steve
Hello Steven and Masami,
Thanks for your comments.
Quoted 35 lines · Masami Hiramatsu
Hi Alexander and Steve,
Quoted 30 lines · On Sun, 11 Nov 2018 20:53:51 -0500 Steven Rostedt <[email protected]> wrote:
Quoted 13 lines · On Sun, 11 Nov 2018 13:19:45 +0300 Alexander Popov <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 3 lines · On Sun, 11 Nov 2018 01:05:30 +0300 Alexander Popov <[email protected]> wrote:
The
stackleak_erase()function is called on the trampoline stack at the end of syscall. This stack is not big enough for ftrace operations, e.g. it can be overflowed if we enablekprobe_eventsforstackleak_erase().Is the issue with kprobes or with function tracing? Because this stops function tracing which I only want disabled if function tracing itself is an issue, not for other things that may use the function tracing infrastructure.
Quoted 8 lines
Hello Steven,
I believe that stackleak erasing is not compatible with function tracing itself. That's what the kernel testing robot has hit:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/1I used
kprobe_eventsjust to reproduce the problem:
https://www.openwall.com/lists/kernel-hardening/2018/11/09/4Have you tried adding a "notrace" to
stackleak_erase()?Not tracing the entire file is a bit of overkill. There's no reason ftrace can't trace
stack_erasing_sysctl()or perhaps evenstackleak_track_stack()as that may be very interesting to trace.
Yes, thank you. It's much better.
Quoted 3 lines
I think it is not enough for stopping kprobes. If you want to stop the kprobes (int3 version) on
stackleak_erase(), you should useNOKPROBE_SYMBOL(stackleak_erase), since kprobes can work without ftrace.
Thanks!
I learned how to use kprobes without ftrace and managed to reproduce the problem as well (I modified kprobe_example.c and kretprobe_example.c). NOKPROBE_SYMBOL() allowed to avoid it.
I'll send the patch soon.
By the way, are there any other tracing/instrumentation mechanisms that should be disabled?
Best regards,
Alexander
Quoted 2 lines · On Mon, 12 Nov 2018 19:51:00 +0300 Alexander Popov <[email protected]> wrote:
By the way, are there any other tracing/instrumentation mechanisms that should be disabled?
ftrace and kprobes are pretty much the only ones that currently do self modification of code all over the kernel. Kprobes even more so than ftrace.
-- Steve
Quoted 9 lines · On Mon, 12 Nov 2018 12:21:48 -0500 Steven Rostedt <[email protected]> wrote:
Quoted 2 lines · On Mon, 12 Nov 2018 19:51:00 +0300 Alexander Popov <[email protected]> wrote:
By the way, are there any other tracing/instrumentation mechanisms that should be disabled?
ftrace and kprobes are pretty much the only ones that currently do self modification of code all over the kernel. Kprobes even more so than ftrace.
Right, since kprobes uses int3 or sw breakpoint exception for hooking into the code, it consumes stack much more.
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>