Skip to contentSkip to conversation

tracing: Fixes for 5.13

PATCH In review Steven Rostedt 2 participants 7 messages Jun 10, 2021

Subsystems (4)
lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

cover tracing: Fixes for 5.13

Tracing fixes for 5.13:

- Fix the length check in the temp buffer filter

- Fix record mcount handling of st_shndx

- Fix build failure in bootconfig tools for "fallthrough"

- Fix error return of bootconfig apply_xbc() routine

Liangyan (1):
tracing: Correct the length check which causes memory corruption

Masami Hiramatsu (1):
tools/bootconfig: Fix a build error accroding to undefined fallthrough

Peter Zijlstra (1):
recordmcount: Correct st_shndx handling

Steven Rostedt (VMware) (1):
ftrace: Do not blindly read the ip address in ftrace_bug()

Zhen Lei (1):
tools/bootconfig: Fix error return code in apply_xbc()

----

 kernel/trace/ftrace.c                       |  8 +++++++-
 kernel/trace/trace.c                        |  2 +-
 scripts/recordmcount.h                      | 13 +++++++++----
 tools/bootconfig/include/linux/bootconfig.h |  4 ++++
 tools/bootconfig/main.c                     |  1 +
 5 files changed, 22 insertions(+), 6 deletions(-)
author maintainer
#1 lore raw

PATCH 1/5 tools/bootconfig: Fix error return code in apply_xbc()

From: Zhen Lei <[email protected]>

Fix to return a negative error code from the error handling case instead of 0, as done elsewhere in this function.

Fixes a995e6bc0524 ("tools/bootconfig: Fix to check the write failure correctly")
---
 tools/bootconfig/main.c | 1 +
 1 file changed, 1 insertion(+)
tools/bootconfig/main.c +1 -0
index 7362bef1a368..6cd6080cac04 100644
@@ -399,6 +399,7 @@ static int apply_xbc(const char *path, const char *xbc_path)
 	}
 	/* TODO: Ensure the @path is initramfs/initrd image */
 	if (fstat(fd, &stat) < 0) {
+		ret = -errno;
 		pr_err("Failed to get the size of %s\n", path);
 		goto out;
 	}
Signature · 1 line
2.30.2
author maintainer
#2 lore raw

PATCH 2/5 tools/bootconfig: Fix a build error accroding to undefined fallthrough

In reply to Steven Rostedt

From: Masami Hiramatsu <[email protected]>

Since the "fallthrough" is defined only in the kernel, building lib/bootconfig.c as a part of user-space tools causes a build error.

Add a dummy fallthrough to avoid the build error.

Cc Ingo Molnar <[email protected]>
Fixes 4c1ca831adb1 ("Revert "lib: Revert use of fallthrough pseudo-keyword in lib/"")
---
 tools/bootconfig/include/linux/bootconfig.h | 4 ++++
 1 file changed, 4 insertions(+)
tools/bootconfig/include/linux/bootconfig.h +4 -0
index 078cbd2ba651..de7f30f99af3 100644
@@ -4,4 +4,8 @@
 
 #include "../../../../include/linux/bootconfig.h"
 
+#ifndef fallthrough
+# define fallthrough
+#endif
+
 #endif
Signature · 1 line
2.30.2
author maintainer
#3 lore raw

PATCH 3/5 ftrace: Do not blindly read the ip address in ftrace_bug()

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

It was reported that a bug on arm64 caused a bad ip address to be used for updating into a nop in ftrace_init(), but the error path (rightfully) returned -EINVAL and not -EFAULT, as the bug caused more than one error to occur. But because -EINVAL was returned, the ftrace_bug() tried to report what was at the location of the ip address, and read it directly. This caused the machine to panic, as the ip was not pointing to a valid memory address.

Instead, read the ip address with copy_from_kernel_nofault() to safely access the memory, and if it faults, report that the address faulted, otherwise report what was in that location.

Fixes 05736a427f7e1 ("ftrace: warn on failure to disable mcount callers")
---
 kernel/trace/ftrace.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/trace/ftrace.c +7 -1
index 2e8a3fde7104..72ef4dccbcc4 100644
@@ -1967,12 +1967,18 @@ static int ftrace_hash_ipmodify_update(struct ftrace_ops *ops,
 
 static void print_ip_ins(const char *fmt, const unsigned char *p)
 {
+	char ins[MCOUNT_INSN_SIZE];
 	int i;
 
+	if (copy_from_kernel_nofault(ins, p, MCOUNT_INSN_SIZE)) {
+		printk(KERN_CONT "%s[FAULT] %px\n", fmt, p);
+		return;
+	}
+
 	printk(KERN_CONT "%s", fmt);
 
 	for (i = 0; i < MCOUNT_INSN_SIZE; i++)
-		printk(KERN_CONT "%s%02x", i ? ":" : "", p[i]);
+		printk(KERN_CONT "%s%02x", i ? ":" : "", ins[i]);
 }
 
 enum ftrace_bug_type ftrace_bug_type;
Signature · 1 line
2.30.2
author maintainer
#4 lore raw

PATCH 4/5 tracing: Correct the length check which causes memory corruption

In reply to Steven Rostedt

From: Liangyan <[email protected]>

We've suffered from severe kernel crashes due to memory corruption on our production environment, like,

Call Trace:
[1640542.554277] general protection fault: 0000 [#1] SMP PTI
[1640542.554856] CPU: 17 PID: 26996 Comm: python Kdump: loaded Tainted:G
[1640542.556629] RIP: 0010:kmem_cache_alloc+0x90/0x190
[1640542.559074] RSP: 0018:ffffb16faa597df8 EFLAGS: 00010286
[1640542.559587] RAX: 0000000000000000 RBX: 0000000000400200 RCX:
0000000006e931bf
[1640542.560323] RDX: 0000000006e931be RSI: 0000000000400200 RDI:
ffff9a45ff004300
[1640542.560996] RBP: 0000000000400200 R08: 0000000000023420 R09:
0000000000000000
[1640542.561670] R10: 0000000000000000 R11: 0000000000000000 R12:
ffffffff9a20608d
[1640542.562366] R13: ffff9a45ff004300 R14: ffff9a45ff004300 R15:
696c662f65636976
[1640542.563128] FS:  00007f45d7c6f740(0000) GS:ffff9a45ff840000(0000)
knlGS:0000000000000000
[1640542.563937] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1640542.564557] CR2: 00007f45d71311a0 CR3: 000000189d63e004 CR4:
00000000003606e0
[1640542.565279] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[1640542.566069] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[1640542.566742] Call Trace:
[1640542.567009]  anon_vma_clone+0x5d/0x170
[1640542.567417]  __split_vma+0x91/0x1a0
[1640542.567777]  do_munmap+0x2c6/0x320
[1640542.568128]  vm_munmap+0x54/0x70
[1640542.569990]  __x64_sys_munmap+0x22/0x30
[1640542.572005]  do_syscall_64+0x5b/0x1b0
[1640542.573724]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[1640542.575642] RIP: 0033:0x7f45d6e61e27

James Wang has reproduced it stably on the latest 4.19 LTS. After some debugging, we finally proved that it's due to ftrace buffer out-of-bound access using a debug tool as follows:

[   86.775200] BUG: Out-of-bounds write at addr 0xffff88aefe8b7000
[   86.780806]  no_context+0xdf/0x3c0
[   86.784327]  __do_page_fault+0x252/0x470
[   86.788367]  do_page_fault+0x32/0x140
[   86.792145]  page_fault+0x1e/0x30
[   86.795576]  strncpy_from_unsafe+0x66/0xb0
[   86.799789]  fetch_memory_string+0x25/0x40
[   86.804002]  fetch_deref_string+0x51/0x60
[   86.808134]  kprobe_trace_func+0x32d/0x3a0
[   86.812347]  kprobe_dispatcher+0x45/0x50
[   86.816385]  kprobe_ftrace_handler+0x90/0xf0
[   86.820779]  ftrace_ops_assist_func+0xa1/0x140
[   86.825340]  0xffffffffc00750bf
[   86.828603]  do_sys_open+0x5/0x1f0
[   86.832124]  do_syscall_64+0x5b/0x1b0
[   86.835900]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

commit b220c049d519 ("tracing: Check length before giving out the filter buffer") adds length check to protect trace data overflow introduced in 0fc1b09ff1ff, seems that this fix can't prevent overflow entirely, the length check should also take the sizeof entry->array[0] into account, since this array[0] is filled the length of trace data and occupy addtional space and risk overflow.

Cc Ingo Molnar <[email protected]>
Cc Xunlei Pang <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Fixes b220c049d519 ("tracing: Check length before giving out the filter buffer")
---
 kernel/trace/trace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace.c +1 -1
index a21ef9cd2aae..9299057feb56 100644
@@ -2736,7 +2736,7 @@ trace_event_buffer_lock_reserve(struct trace_buffer **current_rb,
 	    (entry = this_cpu_read(trace_buffered_event))) {
 		/* Try to use the per cpu buffer first */
 		val = this_cpu_inc_return(trace_buffered_event_cnt);
-		if ((len < (PAGE_SIZE - sizeof(*entry))) && val == 1) {
+		if ((len < (PAGE_SIZE - sizeof(*entry) - sizeof(entry->array[0]))) && val == 1) {
 			trace_event_setup(entry, type, trace_ctx);
 			entry->array[0] = len;
 			return entry;
Signature · 1 line
2.30.2
author maintainer
#5 lore raw

PATCH 5/5 recordmcount: Correct st_shndx handling

In reply to Steven Rostedt

From: Peter Zijlstra <[email protected]>

One should only use st_shndx when >SHN_UNDEF and <SHN_LORESERVE. When SHN_XINDEX, then use .symtab_shndx. Otherwise use 0.

This handles the case: st_shndx >= SHN_LORESERVE && st_shndx != SHN_XINDEX.

---
 scripts/recordmcount.h | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
scripts/recordmcount.h +9 -4
index f9b19524da11..7e8a11ed5e2f 100644
@@ -194,13 +194,18 @@ static unsigned int get_symindex(Elf_Sym const *sym, Elf32_Word const *symtab,
 	unsigned long offset;
 	int index;
 
-	if (sym->st_shndx != SHN_XINDEX)
+	if (sym->st_shndx > SHN_UNDEF &&
+	    sym->st_shndx < SHN_LORESERVE)
 		return w2(sym->st_shndx);
 
-	offset = (unsigned long)sym - (unsigned long)symtab;
-	index = offset / sizeof(*sym);
+	if (sym->st_shndx == SHN_XINDEX) {
+		offset = (unsigned long)sym - (unsigned long)symtab;
+		index = offset / sizeof(*sym);
 
-	return w(symtab_shndx[index]);
+		return w(symtab_shndx[index]);
+	}
+
+	return 0;
 }
 
 static unsigned int get_shnum(Elf_Ehdr const *ehdr, Elf_Shdr const *shdr0)
Signature · 1 line
2.30.2
#6 lore raw

Re: [for-linus][PATCH 5/5] recordmcount: Correct st_shndx handling

Quoted 9 lines · Steven Rostedt

From: Peter Zijlstra <[email protected]>

One should only use st_shndx when >SHN_UNDEF and <SHN_LORESERVE. When SHN_XINDEX, then use .symtab_shndx. Otherwise use 0.

This handles the case: st_shndx >= SHN_LORESERVE && st_shndx != SHN_XINDEX.

Quoted 7 lines

This is apperently causing trouble for Stephen in -next. Please hold.