ftrace: handle kernel code remove

The dynamic ftrace feature keeps a table of all the places that call mcount to either disable them (replacing them with nops) or to enable them (calling some trace function).

This table of functions is also displayed to the user interface to let users enable or disable specific functions. This allows users to only trace some functions within the kernel.

When a module or init sections are removed, the pointers to their locations still exist in this table. To protect against faults and writing over other text, fault handling and code comparing is done. When the code is update, the code being replaced is calculated and compared to the actual text that is being replaced, if the code does not match what is expected to be there, the change is not made.

There is a very small chance that the wrong text (or perhaps a data section) could match the call to mcount and an inappropriate modification could be made.

This patch series adds ftrace_release, to allow a module to remove the pointers to the mcount callers in the module from this table.

Also, __init has "notrace" added to it so that text in the init section are not traced. The trace currently can not be enabled until after init anyway, so this should not be a problem.

-- Steve

Seems that freed records can appear in the available_filter_functions list. This patch fixes that.

---
 kernel/trace/ftrace.c |   10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

Index: linux-tip.git/kernel/trace/ftrace.c
===================================================================
@@ -879,15 +879,13 @@ t_next(struct seq_file *m, void *v, loff
 		}
 	} else {
 		rec = &iter->pg->records[iter->idx++];
-		if ((!(iter->flags & FTRACE_ITER_FAILURES) &&
+		if ((rec->flags & FTRACE_FL_FREE) ||
+
+		    (!(iter->flags & FTRACE_ITER_FAILURES) &&
 		     (rec->flags & FTRACE_FL_FAILED)) ||
 
 		    ((iter->flags & FTRACE_ITER_FAILURES) &&
-		     (!(rec->flags & FTRACE_FL_FAILED) ||
-		      (rec->flags & FTRACE_FL_FREE))) ||
-
-		    ((iter->flags & FTRACE_ITER_FILTER) &&
-		     !(rec->flags & FTRACE_FL_FILTER)) ||
+		     !(rec->flags & FTRACE_FL_FAILED)) ||
 
 		    ((iter->flags & FTRACE_ITER_NOTRACE) &&
 		     !(rec->flags & FTRACE_FL_NOTRACE))) {

The notrace define belongs in compiler.h so that it can be used in init.h

---
 include/linux/compiler.h |    2 ++
 include/linux/linkage.h  |    2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

Index: linux-tip.git/include/linux/compiler.h
===================================================================
@@ -44,6 +44,8 @@ extern void __chk_io_ptr(const volatile 
 # error Sorry, your compiler is too old/not recognized.
 #endif
 
+#define notrace __attribute__((no_instrument_function))
+
 /* Intel compiler defines __GNUC__. So we will overwrite implementations
  * coming from above header files here
  */

Index: linux-tip.git/include/linux/linkage.h
===================================================================
@@ -4,8 +4,6 @@
 #include <linux/compiler.h>
 #include <asm/linkage.h>
 
-#define notrace __attribute__((no_instrument_function))
-
 #ifdef __cplusplus
 #define CPP_ASMLINKAGE extern "C"
 #else

When a mcount pointer is recorded into a table, it is used to add or remove calls to mcount (replacing them with nops). If the code is removed via removing a module, the pointers still exist. At modifying the code a check is always made to make sure the code being replaced is the code expected. In-other-words, the code being replaced is compared to what it is expected to be before being replaced.

There is a very small chance that the code being replaced just happens to look like code that calls mcount (very small since the call to mcount is relative). To remove this chance, this patch adds ftrace_release to allow module unloading to remove the pointers to mcount within the module.

Another change for init calls is made to not trace calls marked with __init. The tracing can not be started until after init is done anyway.

[
Rusty, I decided to remove the if statement for you ;-)
]

---
 include/linux/ftrace.h |    2 ++
 include/linux/init.h   |    2 +-
 kernel/module.c        |   12 ++++++++----
 kernel/trace/ftrace.c  |   32 ++++++++++++++++++++++++++++++--
 4 files changed, 41 insertions(+), 7 deletions(-)

Index: linux-tip.git/include/linux/ftrace.h
===================================================================
@@ -144,10 +144,12 @@ ftrace_special(unsigned long arg1, unsig
 #ifdef CONFIG_FTRACE_MCOUNT_RECORD
 extern void ftrace_init(void);
 extern void ftrace_init_module(unsigned long *start, unsigned long *end);
+extern void ftrace_release(void *start, unsigned long size);
 #else
 static inline void ftrace_init(void) { }
 static inline void
 ftrace_init_module(unsigned long *start, unsigned long *end) { }
+static inline void ftrace_release(void *start, unsigned long size) { }
 #endif
 
 #endif /* _LINUX_FTRACE_H */

Index: linux-tip.git/kernel/module.c
===================================================================
@@ -1432,6 +1432,9 @@ static void free_module(struct module *m
 	/* Module unload stuff */
 	module_unload_free(mod);
 
+	/* release any pointers to mcount in this module */
+	ftrace_release(mod->module_core, mod->core_size);
+
 	/* This may be NULL, but that's OK */
 	module_free(mod, mod->module_init);
 	kfree(mod->args);

@@ -1840,6 +1843,7 @@ static struct module *load_module(void _
 	struct module *mod;
 	long err = 0;
 	void *percpu = NULL, *ptr = NULL; /* Stops spurious gcc warning */
+	void *mseg;
 	struct exception_table_entry *extable;
 	mm_segment_t old_fs;
 

@@ -2191,10 +2195,9 @@ static struct module *load_module(void _
 #endif
 	}
 
-	if (mcountindex) {
-		void *mseg = (void *)sechdrs[mcountindex].sh_addr;
-		ftrace_init_module(mseg, mseg + sechdrs[mcountindex].sh_size);
-	}
+	/* sechdrs[0].sh_size is always zero */
+	mseg = (void *)sechdrs[mcountindex].sh_addr;
+	ftrace_init_module(mseg, mseg + sechdrs[mcountindex].sh_size);
 
 	err = module_finalize(hdr, sechdrs, mod);
 	if (err < 0)

@@ -2265,6 +2268,7 @@ static struct module *load_module(void _
  cleanup:
 	kobject_del(&mod->mkobj.kobj);
 	kobject_put(&mod->mkobj.kobj);
+	ftrace_release(mod->module_core, mod->core_size);
  free_unload:
 	module_unload_free(mod);
 	module_free(mod, mod->module_init);

Index: linux-tip.git/kernel/trace/ftrace.c
===================================================================
@@ -294,13 +294,37 @@ static inline void ftrace_del_hash(struc
 
 static void ftrace_free_rec(struct dyn_ftrace *rec)
 {
-	/* no locking, only called from kstop_machine */
-
 	rec->ip = (unsigned long)ftrace_free_records;
 	ftrace_free_records = rec;
 	rec->flags |= FTRACE_FL_FREE;
 }
 
+void ftrace_release(void *start, unsigned long size)
+{
+	struct dyn_ftrace *rec;
+	struct ftrace_page *pg;
+	unsigned long s = (unsigned long)start;
+	unsigned long e = s + size;
+	int i;
+
+	if (!start)
+		return;
+
+	/* No interrupt should call this */
+	spin_lock(&ftrace_lock);
+
+	for (pg = ftrace_pages_start; pg; pg = pg->next) {
+		for (i = 0; i < pg->index; i++) {
+			rec = &pg->records[i];
+
+			if ((rec->ip >= s) && (rec->ip < e))
+				ftrace_free_rec(rec);
+		}
+	}
+	spin_unlock(&ftrace_lock);
+
+}
+
 static struct dyn_ftrace *ftrace_alloc_dyn_node(unsigned long ip)
 {
 	struct dyn_ftrace *rec;

@@ -1534,7 +1558,9 @@ static int ftrace_convert_nops(unsigned 
 	p = start;
 	while (p < end) {
 		addr = ftrace_call_adjust(*p++);
+		spin_lock(&ftrace_lock);
 		ftrace_record_ip(addr);
+		spin_unlock(&ftrace_lock);
 		ftrace_shutdown_replenish();
 	}
 

@@ -1548,6 +1574,8 @@ static int ftrace_convert_nops(unsigned 
 
 void ftrace_init_module(unsigned long *start, unsigned long *end)
 {
+	if (start == end)
+		return;
 	ftrace_convert_nops(start, end);
 }
 

Index: linux-tip.git/include/linux/init.h
===================================================================
@@ -40,7 +40,7 @@
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
-#define __init		__section(.init.text) __cold
+#define __init		__section(.init.text) __cold notrace
 #define __initdata	__section(.init.data)
 #define __initconst	__section(.init.rodata)
 #define __exitdata	__section(.exit.data)

applied to tip/tracing/ftrace - thanks Steve! I merged it into tip/master and started testing it.

Ingo

hm, massive build failures:

ld: Relocatable linking with relocations from format elf32-i386 (init/.tmp_gl_calibrate.o) to format elf64-x86-64 (init/.tmp_mx_calibrate.o) i CC arch/x86/mm/extable.o
objcopy: 'init/.tmp_mx_calibrate.o': No such file
rm: cannot remove `init/.tmp_mx_calibrate.o': No such file or directory
ld: Relocatable linking with relocations from format elf32-i386 (arch/x86/mm/extable.o) to format elf64-x86-64 (arch/x86/mm/.tmp_mx_extable.o) is not supported
mv: cannot stat `arch/x86/mm/.tmp_mx_extable.o': No such file or directory
ld: Relocatable linking with relocations from format elf32-i386 (arch/x86/mm/fault.o) to format elf64-x86-64 (arch/x86/mm/.tmp_mx_fault.o) is not supported

with this config:

http://redhat.com/~mingo/misc/config-Fri_Aug_15_11_34_53_CEST_2008.bad

Ingo

the problem is that i'm building a 32-bit kernel on a 64-bit host, and the scripts/recordmcount.pl script does not create the temporary object files in the proper (32-bit) output ELF format.

Ingo

the crude hack below got me going for now - but this needs a proper solution.

Ingo

--------------------->

From: Ingo Molnar <[email protected]>
Date: Mon, 9 Jun 2008 20:54:22 +0200
Subject: [PATCH] ftrace: scripts/recordmcount.pl cross-build hack

hack around:

ld: Relocatable linking with relocations from format elf32-i386 (init/.tmp_gl_calibrate.o) to format elf64-x86-64 (init/.tmp_mx_calibrate.o) i CC arch/x86/mm/extable.o
objcopy: 'init/.tmp_mx_calibrate.o': No such file
rm: cannot remove `init/.tmp_mx_calibrate.o': No such file or directory
ld: Relocatable linking with relocations from format elf32-i386 (arch/x86/mm/extable.o) to format elf64-x86-64 (arch/x86/mm/.tmp_mx_extable.o) is not supported
mv: cannot stat `arch/x86/mm/.tmp_mx_extable.o': No such file or directory
ld: Relocatable linking with relocations from format elf32-i386 (arch/x86/mm/fault.o) to format elf64-x86-64 (arch/x86/mm/.tmp_mx_fault.o) is not supported

---
 scripts/recordmcount.pl |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

index 44b4b23..41b55e0 100755
@@ -108,6 +108,20 @@ if ($#ARGV < 6) {
 
 my ($arch, $objdump, $objcopy, $cc, $ld, $nm, $rm, $mv, $inputfile) = @ARGV;
 
+if ($arch eq "i386") {
+  $ld = "ld -m elf_i386";
+  $objdump = "objdump -M i386";
+  $objcopy = "objcopy -O elf32-i386";
+  $cc = "gcc -m32";
+}
+
+if ($arch eq "x86_64") {
+  $ld = "ld -m elf_x86_64";
+  $objdump = "objdump -M x86-64";
+  $objcopy = "objcopy -O elf64-i386";
+  $cc = "gcc -m64";
+}
+
 $objdump = "objdump" if ((length $objdump) == 0);
 $objcopy = "objcopy" if ((length $objcopy) == 0);
 $cc = "gcc" if ((length $cc) == 0);

updated hack - it now works on 64-bit too.

( Btw., whoever came up with those fantastic variations of BFD,
architecture name and ELF format names and sprinkled them around various tools with different command line switches and variants and underscore versus dash names, must not be let near any serious open-source code base again, until the end of his life. )

Ingo

------------->

From: Ingo Molnar <[email protected]>
Date: Mon, 9 Jun 2008 20:54:22 +0200
Subject: [PATCH] ftrace: scripts/recordmcount.pl cross-build hack

hack around:

ld: Relocatable linking with relocations from format elf32-i386 (init/.tmp_gl_calibrate.o) to format elf64-x86-64 (init/.tmp_mx_calibrate.o) i CC arch/x86/mm/extable.o
objcopy: 'init/.tmp_mx_calibrate.o': No such file
rm: cannot remove `init/.tmp_mx_calibrate.o': No such file or directory
ld: Relocatable linking with relocations from format elf32-i386 (arch/x86/mm/extable.o) to format elf64-x86-64 (arch/x86/mm/.tmp_mx_extable.o) is not supported
mv: cannot stat `arch/x86/mm/.tmp_mx_extable.o': No such file or directory
ld: Relocatable linking with relocations from format elf32-i386 (arch/x86/mm/fault.o) to format elf64-x86-64 (arch/x86/mm/.tmp_mx_fault.o) is not supported

---
 scripts/recordmcount.pl |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

index 44b4b23..e4922a6 100755
@@ -108,6 +108,20 @@ if ($#ARGV < 6) {
 
 my ($arch, $objdump, $objcopy, $cc, $ld, $nm, $rm, $mv, $inputfile) = @ARGV;
 
+if ($arch eq "i386") {
+  $ld = "ld -m elf_i386";
+  $objdump = "objdump -M i386";
+  $objcopy = "objcopy -O elf32-i386";
+  $cc = "gcc -m32";
+}
+
+if ($arch eq "x86_64") {
+  $ld = "ld -m elf_x86_64";
+  $objdump = "objdump -M x86-64";
+  $objcopy = "objcopy -O elf64-x86-64";
+  $cc = "gcc -m64";
+}
+
 $objdump = "objdump" if ((length $objdump) == 0);
 $objcopy = "objcopy" if ((length $objcopy) == 0);
 $cc = "gcc" if ((length $cc) == 0);

recordmount.pl should use the already define environment variables:
$CC + $CFLAGS
$LD + $LDFLAGS
$OBJCOPY, $OBJCOPYFLAGS
$OBJDUMP

We need to set $OBJCOPYFLAFS to a sensible variable in arch/x86/Makefile And we should introduce $OBJDUMPFLAGS too.

This would solve all the problems - no?

I am limited in time atm - so relying on one of you to create the patch and do some testing.

Sam