Skip to contentSkip to conversation

ftrace: dynamic enabling/disabling of function calls

PATCH In review Ingo Molnar 2 participants 3 messages Feb 10, 2008 – Feb 11, 2008

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 17/19 ftrace: dynamic enabling/disabling of function calls

From: Steven Rostedt <[email protected]>

This patch adds a feature to dynamically replace the ftrace code with the jmps to allow a kernel with ftrace configured to run as fast as it can without it configured.

The way this works, is on bootup, a ftrace function is registered to record the instruction pointer of all places that call the function.

Later, a kthread is awoken once a second that performs a stop_machine, and replaces all the code that was called with a jmp over the call to ftrace. It only replaces what was found the previous time.

e.g.

call ftrace /* 5 bytes */

is replaced with

jmp 3f /* jmp is 2 bytes and we jump 3 forward */
3:

When we want to enable ftrace for function tracing, the IP recording is removed, and stop_machine is called again to replace all the locations of that were recorded back to the call of ftrace. When it is disabled, we replace the code back to the jmp.

Allocation is done by the kthread. If the ftrace recording function is called, and we don't have any record slots available, then we simply skip that call. Once a second a new page (if needed) is allocated for recording new ftrace function calls. A large batch is allocated at boot up to get most of the calls there.

Because we do this via stop_machine, we don't have to worry about another CPU executing a ftrace call as we modify it. But we do need to worry about NMI's so all functions that might be called via nmi must be annotated with notrace_nmi. When this code is configured in, the NMI code will not call notrace.

---
 arch/x86/kernel/Makefile |    1 
 arch/x86/kernel/ftrace.c |  237 +++++++++++++++++++++++++++++++
 include/linux/ftrace.h   |   18 ++
 kernel/trace/Kconfig     |   17 ++
 kernel/trace/ftrace.c    |  356 ++++++++++++++++++++++++++++++++++++++++++-----
 5 files changed, 597 insertions(+), 32 deletions(-)
arch/x86/kernel/Makefile +1 -0
Index: linux/arch/x86/kernel/Makefile
===================================================================
@@ -54,6 +54,7 @@ obj-$(CONFIG_X86_NUMAQ)		+= numaq_32.o
 obj-$(CONFIG_X86_SUMMIT_NUMA)	+= summit_32.o
 obj-$(CONFIG_X86_VSMP)		+= vsmp_64.o
 obj-$(CONFIG_KPROBES)		+= kprobes.o
+obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
 obj-$(CONFIG_MODULES)		+= module_$(BITS).o
 obj-$(CONFIG_ACPI_SRAT) 	+= srat_32.o
 obj-$(CONFIG_EFI) 		+= efi.o efi_$(BITS).o efi_stub_$(BITS).o
arch/x86/kernel/ftrace.c new file +237 -0

237 lines not on this page · raw message

include/linux/ftrace.h +18 -0
Index: linux/include/linux/ftrace.h
===================================================================
@@ -32,6 +32,24 @@ extern void mcount(void);
 # define clear_ftrace_function(ops) do { } while (0)
 #endif /* CONFIG_FTRACE */
 
+#ifdef CONFIG_DYNAMIC_FTRACE
+# define FTRACE_HASHBITS	10
+# define FTRACE_HASHSIZE	(1<<FTRACE_HASHBITS)
+
+struct dyn_ftrace {
+	struct hlist_node node;
+	unsigned long	  ip;
+};
+
+/* defined in arch */
+extern struct dyn_ftrace *
+ftrace_alloc_shutdown_node(unsigned long ip);
+extern int ftrace_shutdown_arch_init(void);
+extern void ftrace_code_disable(struct dyn_ftrace *rec);
+extern void ftrace_startup_code(void);
+extern void ftrace_shutdown_code(void);
+extern void ftrace_shutdown_replenish(void);
+#endif
 
 #ifdef CONFIG_FRAME_POINTER
 /* TODO: need to fix this for ARM */
kernel/trace/Kconfig +17 -0
Index: linux/kernel/trace/Kconfig
===================================================================
@@ -88,3 +88,20 @@ config CONTEXT_SWITCH_TRACER
 	  This tracer gets called from the context switch and records
 	  all switching of tasks.
 
+config DYNAMIC_FTRACE
+	bool "enable/disable ftrace tracepoints dynamically"
+	depends on FTRACE
+	default y
+	help
+         This option will modify all the calls to ftrace dynamically
+	 (will patch them out of the binary image and replaces them
+	 with a No-Op instruction) as they are called. A table is
+	 created to dynamically enable them again.
+
+	 This way a CONFIG_FTRACE kernel is slightly larger, but otherwise
+	 has native performance as long as no tracing is active.
+
+	 The changes to the code are done by a kernel thread that
+	 wakes up once a second and checks to see if any ftrace calls
+	 were made. If so, it runs stop_machine (stops all CPUS)
+	 and modifies the code to jump over the call to ftrace.
kernel/trace/ftrace.c +324 -32

411 lines not on this page · raw message

#1 lore raw

Hi Ingo and Steven,

Quoted 34 lines · Ingo Molnar

From: Steven Rostedt <[email protected]>

This patch adds a feature to dynamically replace the ftrace code with the jmps to allow a kernel with ftrace configured to run as fast as it can without it configured.

The way this works, is on bootup, a ftrace function is registered to record the instruction pointer of all places that call the function.

Later, a kthread is awoken once a second that performs a stop_machine, and replaces all the code that was called with a jmp over the call to ftrace. It only replaces what was found the previous time.

e.g.

call ftrace /* 5 bytes */

is replaced with

jmp 3f /* jmp is 2 bytes and we jump 3 forward */
3:

When we want to enable ftrace for function tracing, the IP recording is removed, and stop_machine is called again to replace all the locations of that were recorded back to the call of ftrace. When it is disabled, we replace the code back to the jmp.

Allocation is done by the kthread. If the ftrace recording function is called, and we don't have any record slots available, then we simply skip that call. Once a second a new page (if needed) is allocated for recording new ftrace function calls. A large batch is allocated at boot up to get most of the calls there.

Quoted 5 lines

Because we do this via stop_machine, we don't have to worry about another CPU executing a ftrace call as we modify it. But we do need to worry about NMI's so all functions that might be called via nmi must be annotated with notrace_nmi. When this code is configured in, the NMI code will not call notrace.

I'd like to suggest that you can use djprobe-like solution here to eliminate stop_machine.
The djprobe makes a bypass over that call instruction by using a kprobe, and after replacing the call instruction, you can safely remove the bypass. Here is an ols paper about djprobe.
http://www.kernel.org/doc/ols/2007/ols2007v1-pages-189-200.pdf

Thank you,

Signature · 7 lines
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America) Inc.
Software Solutions Division

e-mail: [email protected]
#2 lore raw
Quoted 10 lines · Masami Hiramatsu
Quoted 5 lines

Because we do this via stop_machine, we don't have to worry about another CPU executing a ftrace call as we modify it. But we do need to worry about NMI's so all functions that might be called via nmi must be annotated with notrace_nmi. When this code is configured in, the NMI code will not call notrace.

I'd like to suggest that you can use djprobe-like solution here to eliminate stop_machine. The djprobe makes a bypass over that call instruction by using a kprobe, and after replacing the call instruction, you can safely remove the bypass. [...]

i think the stop_machine() method for code patching is pretty elegant in that it reuses an otherwise rarely used but critical facility. We want it to be fast and easy, and our current ftrace testing shows that it is. So unless there are strong reasons to do otherwise, it would be nice to keep that approach :-)

Ingo