Add build ID to stacktraces
Subsystems (12)
v2 cover Add build ID to stacktraces
This series adds the kernel's build ID[1] to the stacktrace header printed in oops messages, warnings, etc. and the build ID for any module that appears in the stacktrace after the module name. The goal is to make the stacktrace more self-contained and descriptive by including the relevant build IDs in the kernel logs when something goes wrong. This can be used by post processing tools like script/decode_stacktrace.sh and kernel developers to easily locate the debug info associated with a kernel crash and line up what line and file things started falling apart at.
To show how this can be used I've included a patch to decode_stacktrace.sh that downloads the debuginfo from a debuginfod server.
This also includes some patches to make the buildid.c file use more const arguments and consolidate logic into buildid.c from kdump. These are left to the end as they were mostly cleanup patches. I don't know who exactly maintains this so I guess Andrew is the best option to merge all this code.
Here's an example lkdtm stacktrace on arm64.
WARNING: CPU: 4 PID: 3255 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE CPU: 4 PID: 3255 Comm: bash Not tainted 5.11 #3 aa23f7a1231c229de205662d5a9e0d4c580f19a1 Hardware name: Google Lazor (rev3+) with KB Backlight (DT) pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--) pc : lkdtm_WARNING+0x28/0x30 [lkdtm] lr : lkdtm_do_action+0x24/0x40 [lkdtm] sp : ffffffc0134fbca0 x29: ffffffc0134fbca0 x28: ffffff92d53ba240 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: ffffffe3622352c0 x23: 0000000000000020 x22: ffffffe362233366 x21: ffffffe3622352e0 x20: ffffffc0134fbde0 x19: 0000000000000008 x18: 0000000000000000 x17: ffffff929b6536fc x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000012 x13: ffffffe380ed892c x12: ffffffe381d05068 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000001 x8 : ffffffe362237000 x7 : aaaaaaaaaaaaaaaa x6 : 0000000000000000 x5 : 0000000000000000 x4 : 0000000000000001 x3 : 0000000000000008 x2 : ffffff93fef25a70 x1 : ffffff93fef15788 x0 : ffffffe3622352e0 Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] direct_entry+0x16c/0x1b4 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] full_proxy_write+0x74/0xa4 vfs_write+0xec/0x2e8 ksys_write+0x84/0xf0 __arm64_sys_write+0x24/0x30 el0_svc_common+0xf4/0x1c0 do_el0_svc_compat+0x28/0x3c el0_svc_compat+0x10/0x1c el0_sync_compat_handler+0xa8/0xcc el0_sync_compat+0x178/0x180 ---[ end trace 3d95032303e59e68 ]---
Changes from v1 (https://lore.kernel.org/r/[email protected]):
* New printk format %pSb and %pSr
* Return binary format instead of hex format string from build ID APIs
* Some new patches to cleanup buildid/decode_stacktrace.sh
* A new patch to decode_stacktrace.sh to parse output
[1] https://fedoraproject.org/wiki/Releases/FeatureBuildId
Stephen Boyd (12):
buildid: Add API to parse build ID out of buffer
buildid: Add method to get running kernel's build ID
dump_stack: Add vmlinux build ID to stack traces
module: Add printk format to add module build ID to stacktraces
arm64: stacktrace: Use %pSb for backtrace printing x86/dumpstack: Use %pSb for backtrace printing scripts/decode_stacktrace.sh: Support debuginfod scripts/decode_stacktrace.sh: Silence stderr messages from
addr2line/nm
scripts/decode_stacktrace.sh: Indicate 'auto' can be used for base
path
buildid: Mark some arguments const
buildid: Fix kernel-doc notation
kdump: Use vmlinux_build_id() to simplify
Documentation/core-api/printk-formats.rst | 9 +++ arch/arm64/kernel/stacktrace.c | 2 +- arch/x86/kernel/dumpstack.c | 4 +- include/linux/buildid.h | 3 + include/linux/crash_core.h | 6 +- include/linux/kallsyms.h | 13 +++- include/linux/module.h | 6 +- kernel/crash_core.c | 41 +---------- kernel/kallsyms.c | 73 ++++++++++++++----- kernel/module.c | 24 +++++- lib/buildid.c | 75 +++++++++++++++---- lib/dump_stack.c | 5 +- lib/vsprintf.c | 6 +- scripts/decode_stacktrace.sh | 89 +++++++++++++++++++---- 14 files changed, 251 insertions(+), 105 deletions(-)
base-commit: a38fd8748464831584a19438cbb3082b5a2dab15
Signature · 7 lines
https://chromeos.dev _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 1/12 buildid: Add API to parse build ID out of buffer
Add an API that can parse the build ID out of a buffer, instead of a vma, to support printing a kernel module's build ID for stack traces.
--- include/linux/buildid.h | 1 + lib/buildid.c | 50 ++++++++++++++++++++++++++++++----------- 2 files changed, 38 insertions(+), 13 deletions(-)
include/linux/buildid.h +1 -0
@@ -8,5 +8,6 @@int build_id_parse (struct vm_area_struct *vma,unsigned char *build_id,__u32 *size); +int build_id_parse_buf (const void *buf,unsigned char *build_id,u32 buf_size);#endif
lib/buildid.c +37 -13
79 lines not on this page · raw message
Signature · 1 line
https://chromeos.dev
PATCH v2 2/12 buildid: Add method to get running kernel's build ID
In reply to Stephen BoydAdd vmlinux_build_id() so that callers can print a hex format string representation of the running kernel's build ID. This will be used in the kdump and dump_stack code so that developers can easily locate the vmlinux debug symbols for a crash/stacktrace.
--- include/linux/buildid.h | 2 ++ lib/buildid.c | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+)
include/linux/buildid.h +2 -0
@@ -10,4 +10,6 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size);int build_id_parse_buf (const void *buf,unsigned char *build_id,u32 buf_size); +const unsigned char *vmlinux_build_id (void ); +#endif
lib/buildid.c +19 -0
@@ -4,6 +4,7 @@#include <linux/elf.h> #include <linux/kernel.h> #include <linux/pagemap.h> +#include <linux/string.h> #define BUILD_ID 3
@@ -171,3 +172,21 @@ int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) {return parse_build_id_buf (build_id,NULL , buf, buf_size); } + +/** +* vmlinux_build_id - Get the running kernel's build ID +* +* Return: Running kernel's build ID +*/ +const unsigned char *vmlinux_build_id (void ) +{ +extern const void __start_notes__weak ; +extern const void __stop_notes__weak ; +unsigned int size = &__stop_notes - &__start_notes; +static unsigned char vmlinux_build_id[BUILD_ID_SIZE_MAX ]; + +if (!memchr_inv (vmlinux_build_id,0 ,BUILD_ID_SIZE_MAX )) +build_id_parse_buf (&__start_notes, vmlinux_build_id, size); + +return vmlinux_build_id; +}
Signature · 7 lines
https://chromeos.dev _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Rasmus Villemoes
reviewerRe: [PATCH v2 02/12] buildid: Add method to get running kernel's build ID
Quoted 56 lines · Stephen Boyd
Add
vmlinux_build_id()so that callers can print a hex format string representation of the running kernel's build ID. This will be used in the kdump anddump_stackcode so that developers can easily locate the vmlinux debug symbols for a crash/stacktrace.Cc Jiri Olsa <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Jessica Yu <[email protected]>Cc Evan Green <[email protected]>Cc Hsin-Yi Wang <[email protected]>Cc Dave Young <[email protected]>Cc Baoquan He <[email protected]>Cc Vivek Goyal <[email protected]>Cc <[email protected]>Signed-off-by Stephen Boyd <[email protected]>--- include/linux/buildid.h | 2 ++ lib/buildid.c | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+)include/linux/buildid.h +2
-0@@ -10,4 +10,6 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size);int build_id_parse_buf (const void *buf,unsigned char *build_id,u32 buf_size); +const unsigned char *vmlinux_build_id (void ); +#endif lib/buildid.c +9
-0@@ -4,6 +4,7 @@#include <linux/elf.h> #include <linux/kernel.h> #include <linux/pagemap.h> +#include <linux/string.h> #define BUILD_ID 3@@ -171,3 +172,21 @@ int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) {return parse_build_id_buf (build_id,NULL , buf, buf_size); } + +/** +* vmlinux_build_id - Get the running kernel's build ID +* +* Return: Running kernel's build ID +*/ +const unsigned char *vmlinux_build_id (void ) +{
lib/buildid.c int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) · line 192
Quoted 10 lines
lib/buildid.c +10
-0+extern const void __start_notes__weak ; +extern const void __stop_notes__weak ; +unsigned int size = &__stop_notes - &__start_notes; +static unsigned char vmlinux_build_id[BUILD_ID_SIZE_MAX ]; + +if (!memchr_inv (vmlinux_build_id,0 ,BUILD_ID_SIZE_MAX )) +build_id_parse_buf (&__start_notes, vmlinux_build_id, size); + +return vmlinux_build_id; +}
Hm, is there any reason to do that initialization lazily and thus need an accessor? If the system is coming down hard, there's a (very very small) risk that one thread starts finding the build id, is in the middle of the memcpy, another thread also ends up wanting the vmlinux build id, sees some non-nul byte, and proceeds to using the partially written vmlinux_build_id.
Perhaps consider just exposing the vmlinux_build_id[] array itself, adding a init_vmlinux_build_id() call somewhere early in start_kernel().
It could then also be made __ro_after_init.
In any case, if you decide to keep the current way, please rename the local variable (just "build_id" is fine) so that it doesn't shadow the very function it resides in, that's very confusing.
Rasmus
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 02/12] buildid: Add method to get running kernel's build ID
Quoting Rasmus Villemoes (2021-03-24 02:24:27)
Quoted 75 lines · Stephen Boyd
Quoted 56 lines
Add
vmlinux_build_id()so that callers can print a hex format string representation of the running kernel's build ID. This will be used in the kdump anddump_stackcode so that developers can easily locate the vmlinux debug symbols for a crash/stacktrace.Cc Jiri Olsa <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Jessica Yu <[email protected]>Cc Evan Green <[email protected]>Cc Hsin-Yi Wang <[email protected]>Cc Dave Young <[email protected]>Cc Baoquan He <[email protected]>Cc Vivek Goyal <[email protected]>Cc <[email protected]>Signed-off-by Stephen Boyd <[email protected]>--- include/linux/buildid.h | 2 ++ lib/buildid.c | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+)include/linux/buildid.h +2
-0@@ -10,4 +10,6 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size);int build_id_parse_buf (const void *buf,unsigned char *build_id,u32 buf_size); +const unsigned char *vmlinux_build_id (void ); +#endif lib/buildid.c +9
-0@@ -4,6 +4,7 @@#include <linux/elf.h> #include <linux/kernel.h> #include <linux/pagemap.h> +#include <linux/string.h> #define BUILD_ID 3@@ -171,3 +172,21 @@ int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) {return parse_build_id_buf (build_id,NULL , buf, buf_size); } + +/** +* vmlinux_build_id - Get the running kernel's build ID +* +* Return: Running kernel's build ID +*/ +const unsigned char *vmlinux_build_id (void ) +{lib/buildid.c int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) · line 192
Quoted 10 lines
lib/buildid.c +10
-0+extern const void __start_notes__weak ; +extern const void __stop_notes__weak ; +unsigned int size = &__stop_notes - &__start_notes; +static unsigned char vmlinux_build_id[BUILD_ID_SIZE_MAX ]; + +if (!memchr_inv (vmlinux_build_id,0 ,BUILD_ID_SIZE_MAX )) +build_id_parse_buf (&__start_notes, vmlinux_build_id, size); + +return vmlinux_build_id; +}Hm, is there any reason to do that initialization lazily and thus need an accessor? If the system is coming down hard, there's a (very very small) risk that one thread starts finding the build id, is in the middle of the memcpy, another thread also ends up wanting the vmlinux build id, sees some non-nul byte, and proceeds to using the partially written
vmlinux_build_id.
Quoted 9 lines
Perhaps consider just exposing the vmlinux_build_id[] array itself, adding a
init_vmlinux_build_id()call somewhere early instart_kernel().It could then also be made
__ro_after_init.In any case, if you decide to keep the current way, please rename the local variable (just "
build_id" is fine) so that it doesn't shadow the very function it resides in, that's very confusing.
No particular reason to do it this way. I'll take that approach to initialize it early in start_kernel() and then expose the array instead. Thanks!
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 3/12 dump_stack: Add vmlinux build ID to stack traces
In reply to Stephen BoydAdd the running kernel's build ID[1] to the stacktrace information header. This makes it simpler for developers to locate the vmlinux with full debuginfo for a particular kernel stacktrace. Combined with scripts/decode_stracktrace.sh, a developer can download the correct vmlinux from a debuginfod[2] server and find the exact file and line number for the functions plus offsets in a stacktrace.
This is especially useful for pstore crash debugging where the kernel crashes are recorded in the pstore logs and the recovery kernel is different or the debuginfo doesn't exist on the device due to space concerns (the data can be large and a security concern). The stacktrace can be analyzed after the crash by using the build ID to find the matching vmlinux and understand where in the function something went wrong.
Example stacktrace from lkdtm:
WARNING: CPU: 4 PID: 3255 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE CPU: 4 PID: 3255 Comm: bash Not tainted 5.11 #3 aa23f7a1231c229de205662d5a9e0d4c580f19a1 Hardware name: Google Lazor (rev3+) with KB Backlight (DT) pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--) pc : lkdtm_WARNING+0x28/0x30 [lkdtm] lr : lkdtm_do_action+0x24/0x40 [lkdtm] sp : ffffffc0134fbca0 x29: ffffffc0134fbca0 x28: ffffff92d53ba240 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: ffffffe3622352c0 x23: 0000000000000020 x22: ffffffe362233366 x21: ffffffe3622352e0 x20: ffffffc0134fbde0 x19: 0000000000000008 x18: 0000000000000000 x17: ffffff929b6536fc x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000012 x13: ffffffe380ed892c x12: ffffffe381d05068 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000001 x8 : ffffffe362237000 x7 : aaaaaaaaaaaaaaaa x6 : 0000000000000000 x5 : 0000000000000000 x4 : 0000000000000001 x3 : 0000000000000008 x2 : ffffff93fef25a70 x1 : ffffff93fef15788 x0 : ffffffe3622352e0 Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm] direct_entry+0x16c/0x1b4 [lkdtm] full_proxy_write+0x74/0xa4 vfs_write+0xec/0x2e8 ksys_write+0x84/0xf0 __arm64_sys_write+0x24/0x30 el0_svc_common+0xf4/0x1c0 do_el0_svc_compat+0x28/0x3c el0_svc_compat+0x10/0x1c el0_sync_compat_handler+0xa8/0xcc el0_sync_compat+0x178/0x180 ---[ end trace 3d95032303e59e68 ]---
The hex string aa23f7a1231c229de205662d5a9e0d4c580f19a1 is the build ID, following the kernel version number.
--- lib/dump_stack.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/dump_stack.c +3 -2
@@ -5,6 +5,7 @@*/ #include <linux/kernel.h> +#include <linux/buildid.h> #include <linux/export.h> #include <linux/sched.h> #include <linux/sched/debug.h>
@@ -45,13 +46,13 @@ void __init dump_stack_set_arch_desc(const char *fmt, ...)*/ void dump_stack_print_info (const char *log_lvl) { -printk ("%sCPU: %d PID: %d Comm: %.20s %s%s %s %.*s\n" , +printk ("%sCPU: %d PID: %d Comm: %.20s %s%s %s %.*s %20phN\n" , log_lvl,raw_smp_processor_id (), current->pid, current->comm,kexec_crash_loaded () ?"Kdump: loaded " :"" ,print_tainted (),init_utsname ()->release, (int )strcspn (init_utsname ()->version," " ), -init_utsname ()->version); +init_utsname ()->version,vmlinux_build_id ());if (dump_stack_arch_desc_str[0 ] !='\0' ) printk("%sHardware name: %s\n",
Signature · 1 line
https://chromeos.dev
Andy Shevchenko
reviewerRe: [PATCH v2 03/12] dump_stack: Add vmlinux build ID to stack traces
Quoted 15 lines · Stephen Boyd
Add the running kernel's build ID[1] to the stacktrace information header. This makes it simpler for developers to locate the vmlinux with full debuginfo for a particular kernel stacktrace. Combined with
scripts/decode_stracktrace.sh, a developer can download the correct vmlinux from a debuginfod[2] server and find the exact file and line number for the functions plus offsets in a stacktrace.This is especially useful for pstore crash debugging where the kernel crashes are recorded in the pstore logs and the recovery kernel is different or the debuginfo doesn't exist on the device due to space concerns (the data can be large and a security concern). The stacktrace can be analyzed after the crash by using the build ID to find the matching vmlinux and understand where in the function something went wrong.
Quoted 1 line
Example stacktrace from lkdtm:
Can we drop the noise from the example and leave only important part(s)?
Quoted 39 lines
WARNING: CPU: 4 PID: 3255 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE CPU: 4 PID: 3255 Comm: bash Not tainted 5.11 #3 aa23f7a1231c229de205662d5a9e0d4c580f19a1 Hardware name: Google Lazor (rev3+) with KB Backlight (DT) pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--) pc : lkdtm_WARNING+0x28/0x30 [lkdtm] lr : lkdtm_do_action+0x24/0x40 [lkdtm] sp : ffffffc0134fbca0 x29: ffffffc0134fbca0 x28: ffffff92d53ba240 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: ffffffe3622352c0 x23: 0000000000000020 x22: ffffffe362233366 x21: ffffffe3622352e0 x20: ffffffc0134fbde0 x19: 0000000000000008 x18: 0000000000000000 x17: ffffff929b6536fc x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000012 x13: ffffffe380ed892c x12: ffffffe381d05068 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000001 x8 : ffffffe362237000 x7 : aaaaaaaaaaaaaaaa x6 : 0000000000000000 x5 : 0000000000000000 x4 : 0000000000000001 x3 : 0000000000000008 x2 : ffffff93fef25a70 x1 : ffffff93fef15788 x0 : ffffffe3622352e0 Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm] direct_entry+0x16c/0x1b4 [lkdtm] full_proxy_write+0x74/0xa4 vfs_write+0xec/0x2e8 ksys_write+0x84/0xf0 __arm64_sys_write+0x24/0x30 el0_svc_common+0xf4/0x1c0 do_el0_svc_compat+0x28/0x3c el0_svc_compat+0x10/0x1c el0_sync_compat_handler+0xa8/0xcc el0_sync_compat+0x178/0x180 ---[ end trace 3d95032303e59e68 ]---The hex string
aa23f7a1231c229de205662d5a9e0d4c580f19a1is the build ID, following the kernel version number.
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [PATCH v2 03/12] dump_stack: Add vmlinux build ID to stack traces
Quoting Andy Shevchenko (2021-03-24 04:22:58)
Quoted 16 lines · Stephen Boyd
Quoted 15 lines
Add the running kernel's build ID[1] to the stacktrace information header. This makes it simpler for developers to locate the vmlinux with full debuginfo for a particular kernel stacktrace. Combined with
scripts/decode_stracktrace.sh, a developer can download the correct vmlinux from a debuginfod[2] server and find the exact file and line number for the functions plus offsets in a stacktrace.This is especially useful for pstore crash debugging where the kernel crashes are recorded in the pstore logs and the recovery kernel is different or the debuginfo doesn't exist on the device due to space concerns (the data can be large and a security concern). The stacktrace can be analyzed after the crash by using the build ID to find the matching vmlinux and understand where in the function something went wrong.
Quoted 3 lines
Quoted 1 line
Example stacktrace from lkdtm:
Can we drop the noise from the example and leave only important part(s)?
Do you mean cut after the CPU line?
Quoted 4 lines
Quoted 4 lines
WARNING: CPU: 4 PID: 3255 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE CPU: 4 PID: 3255 Comm: bash Not tainted 5.11 #3 aa23f7a1231c229de205662d5a9e0d4c580f19a1
PATCH v2 4/12 module: Add printk format to add module build ID to stacktraces
In reply to Stephen BoydLet's make kernel stacktraces easier to identify by including the build ID[1] of a module if the stacktrace is printing a symbol from a module. This makes it simpler for developers to locate a kernel module's full debuginfo for a particular stacktrace. Combined with scripts/decode_stracktrace.sh, a developer can download the matching debuginfo from a debuginfod[2] server and find the exact file and line number for the functions plus offsets in a stacktrace that match the module. This is especially useful for pstore crash debugging where the kernel crashes are recorded in something like console-ramoops and the recovery kernel/modules are different or the debuginfo doesn't exist on the device due to space concerns (the debuginfo can be too large for space limited devices).
Originally, I put this on the %pS format, but that was quickly rejected given that %pS is used in other places such as ftrace where build IDs aren't meaningful. There was some discussions on the list to put every module build ID into the "Modules linked in:" section of the stacktrace message but that quickly becomes very hard to read once you have more than three or four modules linked in. It also provides too much information when we don't expect each module to be traversed in a stacktrace. Having the build ID for modules that aren't important just makes things messy. Splitting it to multiple lines for each module quickly explodes the number of lines printed in an oops too, possibly wrapping the warning off the console. And finally, trying to stash away each module used in a callstack to provide the ID of each symbol printed is cumbersome and would require changes to each architecture to stash away modules and return their build IDs once unwinding has completed.
Instead, we opt for the simpler approach of introducing a new printk format '%p[sS][br]' for "pointer symbolic backtrace with module build ID" and then updating the few places in the architecture layer where the stacktrace is printed to use this new format.
Example:
WARNING: CPU: 3 PID: 3373 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE hci_uart <modules trimmed> CPU: 3 PID: 3373 Comm: bash Not tainted 5.11 #12 a8c0d47f7051f3e6670ceaea724af66a39c6cec8 Hardware name: Google Lazor (rev3+) with KB Backlight (DT) pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--) pc : lkdtm_WARNING+0x28/0x30 [lkdtm] lr : lkdtm_do_action+0x24/0x40 [lkdtm] sp : ffffffc013febca0 x29: ffffffc013febca0 x28: ffffff88d9438040 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: ffffffdd0e9772c0 x23: 0000000000000020 x22: ffffffdd0e975366 x21: ffffffdd0e9772e0 x20: ffffffc013febde0 x19: 0000000000000008 x18: 0000000000000000 x17: 0000000000000000 x16: 0000000000000037 x15: ffffffdd102ab174 x14: 0000000000000003 x13: 0000000000000004 x12: 0000000000000000 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000001 x8 : ffffffdd0e979000 x7 : 0000000000000000 x6 : ffffffdd10ff6b54 x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffffffc013feb938 x2 : ffffff89fef05a70 x1 : ffffff89feef5788 x0 : ffffffdd0e9772e0 Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm 6c2215028606bda50de823490723dc4bc5bf46f9] direct_entry+0x16c/0x1b4 [lkdtm 6c2215028606bda50de823490723dc4bc5bf46f9] full_proxy_write+0x74/0xa4 vfs_write+0xec/0x2e8 ksys_write+0x84/0xf0 __arm64_sys_write+0x24/0x30 el0_svc_common+0xf4/0x1c0 do_el0_svc_compat+0x28/0x3c el0_svc_compat+0x10/0x1c el0_sync_compat_handler+0xa8/0xcc el0_sync_compat+0x178/0x180 ---[ end trace f89bc7f5417cbcc6 ]---
--- Documentation/core-api/printk-formats.rst | 9 +++ include/linux/kallsyms.h | 13 +++- include/linux/module.h | 6 +- kernel/kallsyms.c | 73 ++++++++++++++++------- kernel/module.c | 24 +++++++- lib/vsprintf.c | 6 +- 6 files changed, 106 insertions(+), 25 deletions(-)
Documentation/core-api/printk-formats.rst +9 -0
@@ -114,6 +114,15 @@ used when printing stack backtraces. The specifier takes into consideration the effect of compiler optimisations which may occur when tail-calls are used and marked with the noreturn GCC attribute. +If the pointer is within a module, the module name and optionally build ID is +printed after the symbol name. + +:: + %pS versatile_init+0x0/0x110 [module_name] + %pSb versatile_init+0x0/0x110 [module_name ed5019fdf5e53be37cb1ba7899292d7e143b259e] + %pSr versatile_init+0x9/0x110 [module_name ed5019fdf5e53be37cb1ba7899292d7e143b259e] + (with __builtin_extract_return_addr() translation) + Probed Pointers from BPF / tracing ----------------------------------
include/linux/kallsyms.h +11 -2
@@ -7,6 +7,7 @@#define _LINUX_KALLSYMS_H#include <linux/errno.h> +#include <linux/buildid.h> #include <linux/kernel.h> #include <linux/stddef.h> #include <linux/mm.h>
@@ -15,8 +16,9 @@#include <asm/sections.h> #define KSYM_NAME_LEN 128 -#define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s]") + (KSYM_NAME_LEN - 1) + \ - 2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + 1) +#define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s %s]") + (KSYM_NAME_LEN - 1) + \ + 2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + \ + (BUILD_ID_SIZE_MAX * 2) + 1)struct cred ;struct module ;
@@ -91,6 +93,7 @@ const char *kallsyms_lookup(unsigned long addr,/* Look up a kernel symbol and return it in a text buffer. */ extern int sprint_symbol (char *buffer,unsigned long address); +extern int sprint_symbol_stacktrace (char *buffer,unsigned long address);extern int sprint_symbol_no_offset (char *buffer,unsigned long address);extern int sprint_backtrace (char *buffer,unsigned long address);
@@ -128,6 +131,12 @@ static inline int sprint_symbol(char *buffer, unsigned long addr)return 0 ; } +static inline int sprint_symbol_stacktrace (char *buffer,unsigned long address) +{ + *buffer ='\0' ; +return 0 ; +} +static inline int sprint_symbol_no_offset (char *buffer,unsigned long addr) { *buffer ='\0' ;
include/linux/module.h +5 -1
@@ -11,6 +11,7 @@#include <linux/list.h> #include <linux/stat.h> +#include <linux/buildid.h> #include <linux/compiler.h> #include <linux/cache.h> #include <linux/kmod.h>
@@ -367,6 +368,9 @@ struct module {/* Unique handle for this module */ char name[MODULE_NAME_LEN ]; +/* Module build ID */ +unsigned char build_id[BUILD_ID_SIZE_MAX ]; +/* Sysfs stuff. */ struct module_kobject mkobj;struct module_attribute *modinfo_attrs;
@@ -630,7 +634,7 @@ void *dereference_module_function_descriptor(struct module *mod, void *ptr);const char *module_address_lookup (unsigned long addr,unsigned long *symbolsize,unsigned long *offset, -char **modname, +char **modname,unsigned char **modbuildid,char *namebuf);int lookup_module_symbol_name (unsigned long addr,char *symname);int lookup_module_symbol_attrs (unsigned long addr,unsigned long *size,unsigned long *offset,char *modname,char *name);
kernel/kallsyms.c +53 -20
139 lines not on this page · raw message
kernel/module.c +23 -1
@@ -13,6 +13,7 @@#include <linux/trace_events.h> #include <linux/init.h> #include <linux/kallsyms.h> +#include <linux/buildid.h> #include <linux/file.h> #include <linux/fs.h> #include <linux/sysfs.h>
@@ -2770,6 +2771,20 @@ static void add_kallsyms(struct module *mod, const struct load_info *info) } mod->core_kallsyms.num_symtab = ndst; } + +static void init_build_id (struct module *mod,const struct load_info *info) +{ +const Elf_Shdr *sechdr; +unsigned int i; + +for (i =0 ; i < info->hdr->e_shnum; i++) { + sechdr = &info->sechdrs[i]; +if (!sect_empty (sechdr) && sechdr->sh_type ==SHT_NOTE && + !build_id_parse_buf ((void *)sechdr->sh_addr, mod->build_id, + sechdr->sh_size)) +break ; + } +}#else static inline void layout_symtab (struct module *mod,struct load_info *info) {
@@ -2778,6 +2793,10 @@ static inline void layout_symtab(struct module *mod, struct load_info *info)static void add_kallsyms (struct module *mod,const struct load_info *info) { } + +static void init_build_id (struct module *mod,const struct load_info *info) +{ +}#endif /* CONFIG_KALLSYMS */ static void dynamic_debug_setup (struct module *mod,struct _ddebug *debug,unsigned int num)
@@ -4004,6 +4023,7 @@ static int load_module(struct load_info *info, const char __user *uargs, goto free_arch_cleanup; } +init_build_id (mod, info);dynamic_debug_setup (mod, info->debug, info->num_debug);/* Ftrace init must be called in the MODULE_STATE_UNFORMED state */
@@ -4235,7 +4255,7 @@ void * __weak dereference_module_function_descriptor(struct module *mod,const char *module_address_lookup (unsigned long addr,unsigned long *size,unsigned long *offset, -char **modname, +char **modname,unsigned char **modbuildid,char *namebuf) {const char *ret =NULL ;
@@ -4246,6 +4266,8 @@ const char *module_address_lookup(unsigned long addr,if (mod) {if (modname) *modname = mod->name; +if (modbuildid) + *modbuildid = mod->build_id; ret =find_kallsyms_symbol (mod, addr, size, offset); }
lib/vsprintf.c +5 -1
@@ -961,13 +961,15 @@ char *symbol_string(char *buf, char *end, void *ptr,char sym[KSYM_SYMBOL_LEN ];#endif -if (fmt[1 ] =='R' ) +if (fmt[1 ] =='R' || fmt[1 ] =='r' ) ptr =__builtin_extract_return_addr (ptr); value = (unsigned long )ptr;#ifdef CONFIG_KALLSYMS if (*fmt =='B' )sprint_backtrace (sym, value); +else if (*fmt =='S' && (fmt[1 ] =='b' || fmt[1 ] =='r' )) +sprint_symbol_stacktrace (sym, value);else if (*fmt !='s' )sprint_symbol (sym, value);else
@@ -2129,6 +2131,8 @@ early_param("no_hash_pointers", no_hash_pointers_enable);* - 'S' For symbolic direct pointers (or function descriptors) with offset * - 's' For symbolic direct pointers (or function descriptors) without offset * - '[Ss]R' as above with __builtin_extract_return_addr() translation +* - '[Ss]r' as above with __builtin_extract_return_addr() translation and module build ID +* - '[Ss]b' as above with module build ID (for use in backtraces) * - '[Ff]' %pf and %pF were obsoleted and later removed in favor of * %ps and %pS. Be careful when re-using these specifiers. * - 'B' For backtraced symbolic direct pointers with offset
Signature · 1 line
https://chromeos.dev
Rasmus Villemoes
reviewerRe: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
Quoted 20 lines · Stephen Boyd
kernel/module.c +5
-0@@ -2778,6 +2793,10 @@ static inline void layout_symtab(struct module *mod, struct load_info *info)static void add_kallsyms (struct module *mod,const struct load_info *info) { } + +static void init_build_id (struct module *mod,const struct load_info *info) +{ +}#endif /* CONFIG_KALLSYMS */ static void dynamic_debug_setup (struct module *mod,struct _ddebug *debug,unsigned int num)@@ -4004,6 +4023,7 @@ static int load_module(struct load_info *info, const char __user *uargs, goto free_arch_cleanup; } +init_build_id (mod, info);dynamic_debug_setup (mod, info->debug, info->num_debug);/* Ftrace init must be called in the MODULE_STATE_UNFORMED state */
kernel/module.c void * __weak dereference_module_function_descriptor(struct module *mod, · line 4258
Quoted 5 lines
kernel/module.c +1
-1@@ -4235,7 +4255,7 @@ void * __weak dereference_module_function_descriptor(struct module *mod, const char *module_address_lookup(unsigned long addr, unsigned long *size, unsigned long *offset, - char **modname, + char **modname, unsigned char **modbuildid,
It's an existing defect with modname, but surely this should take a "const unsigned char **modbuildid", no?
Quoted 34 lines
kernel/module.c +2
-0char *namebuf) {const char *ret =NULL ;@@ -4246,6 +4266,8 @@ const char *module_address_lookup(unsigned long addr,if (mod) {if (modname) *modname = mod->name; +if (modbuildid) + *modbuildid = mod->build_id; ret =find_kallsyms_symbol (mod, addr, size, offset); }lib/vsprintf.c +3
-1@@ -961,13 +961,15 @@ char *symbol_string(char *buf, char *end, void *ptr,char sym[KSYM_SYMBOL_LEN ];#endif -if (fmt[1 ] =='R' ) +if (fmt[1 ] =='R' || fmt[1 ] =='r' ) ptr =__builtin_extract_return_addr (ptr); value = (unsigned long )ptr;#ifdef CONFIG_KALLSYMS if (*fmt =='B' )sprint_backtrace (sym, value); +else if (*fmt =='S' && (fmt[1 ] =='b' || fmt[1 ] =='r' )) +sprint_symbol_stacktrace (sym, value);else if (*fmt !='s' )sprint_symbol (sym, value);else
lib/vsprintf.c early_param("no_hash_pointers", no_hash_pointers_enable); · line 2135
Quoted 5 lines
lib/vsprintf.c +2
-0@@ -2129,6 +2131,8 @@ early_param("no_hash_pointers", no_hash_pointers_enable);* - 'S' For symbolic direct pointers (or function descriptors) with offset * - 's' For symbolic direct pointers (or function descriptors) without offset * - '[Ss]R' as above with __builtin_extract_return_addr() translation +* - '[Ss]r' as above with __builtin_extract_return_addr() translation and module build ID +* - '[Ss]b' as above with module build ID (for use in backtraces)
The code doesn't quite match the comment. The lowercase s is not handled (i.e., there's no way to say "without offset, with build ID"). You don't have to fix the code to support that right now, the whole kallsyms vsprintf code needs to be reworked inside-out anyway (having vsnprint call sprint_symbol* which builds the output using snprintf() calls makes me cringe), so please just replace [Ss] by S to make the comment match the code - I notice that you did only document the S variant in printk-formats.rst.
Is there any reason you didn't just make b an optional flag that could be specified with or without R? I suppose the parsing is more difficult with several orthogonal flags (see escaped_string()), but it's a little easier to understand. Dunno, it's not like we're gonna think of 10 other things that could be printed for a symbol, so perhaps it's fine.
Rasmus
Re: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
Quoting Rasmus Villemoes (2021-03-24 02:57:13)
Quoted 20 lines · Stephen Boyd
Quoted 19 lines
kernel/module.c +5
-0@@ -2778,6 +2793,10 @@ static inline void layout_symtab(struct module *mod, struct load_info *info)static void add_kallsyms (struct module *mod,const struct load_info *info) { } + +static void init_build_id (struct module *mod,const struct load_info *info) +{ +}#endif /* CONFIG_KALLSYMS */ static void dynamic_debug_setup (struct module *mod,struct _ddebug *debug,unsigned int num)@@ -4004,6 +4023,7 @@ static int load_module(struct load_info *info, const char __user *uargs, goto free_arch_cleanup; } +init_build_id (mod, info);dynamic_debug_setup (mod, info->debug, info->num_debug);
Quoted 10 lines
kernel/module.c void * __weak dereference_module_function_descriptor(struct module *mod, · line 4258
Quoted 7 lines
kernel/module.c +1
-1/* Ftrace init must be called in the MODULE_STATE_UNFORMED state */ @@ -4235,7 +4255,7 @@ void * __weak dereference_module_function_descriptor(struct module *mod, const char *module_address_lookup(unsigned long addr, unsigned long *size, unsigned long *offset, - char **modname, + char **modname, unsigned char **modbuildid,It's an existing defect with modname, but surely this should take a "const unsigned char **modbuildid", no?
Sure.
Quoted 41 lines
Quoted 35 lines
kernel/module.c +2
-0char *namebuf) {const char *ret =NULL ;@@ -4246,6 +4266,8 @@ const char *module_address_lookup(unsigned long addr,if (mod) {if (modname) *modname = mod->name; +if (modbuildid) + *modbuildid = mod->build_id; ret =find_kallsyms_symbol (mod, addr, size, offset); }lib/vsprintf.c +3
-1@@ -961,13 +961,15 @@ char *symbol_string(char *buf, char *end, void *ptr,char sym[KSYM_SYMBOL_LEN ];#endif -if (fmt[1 ] =='R' ) +if (fmt[1 ] =='R' || fmt[1 ] =='r' ) ptr =__builtin_extract_return_addr (ptr); value = (unsigned long )ptr;#ifdef CONFIG_KALLSYMS if (*fmt =='B' )sprint_backtrace (sym, value); +else if (*fmt =='S' && (fmt[1 ] =='b' || fmt[1 ] =='r' )) +sprint_symbol_stacktrace (sym, value);else if (*fmt !='s' )sprint_symbol (sym, value);else lib/vsprintf.c early_param("no_hash_pointers", no_hash_pointers_enable); · line 2135
Quoted 5 lines
lib/vsprintf.c +2
-0@@ -2129,6 +2131,8 @@ early_param("no_hash_pointers", no_hash_pointers_enable);* - 'S' For symbolic direct pointers (or function descriptors) with offset * - 's' For symbolic direct pointers (or function descriptors) without offset * - '[Ss]R' as above with __builtin_extract_return_addr() translation +* - '[Ss]r' as above with __builtin_extract_return_addr() translation and module build ID +* - '[Ss]b' as above with module build ID (for use in backtraces)
Quoted 8 lines
The code doesn't quite match the comment. The lowercase s is not handled (i.e., there's no way to say "without offset, with build ID"). You don't have to fix the code to support that right now, the whole kallsyms vsprintf code needs to be reworked inside-out anyway (having vsnprint call sprint_symbol* which builds the output using
snprintf()calls makes me cringe), so please just replace [Ss] by S to make the comment match the code - I notice that you did only document the S variant inprintk-formats.rst.
No problem. Will fix this comment.
Quoted 7 lines
Is there any reason you didn't just make b an optional flag that could be specified with or without R? I suppose the parsing is more difficult with several orthogonal flags (see
escaped_string()), but it's a little easier to understand. Dunno, it's not like we're gonna think of 10 other things that could be printed for a symbol, so perhaps it's fine.
I think I follow. So %pSb or %pSRb? If it's easier to understand then sure. I was trying to avoid checking another character beyond fmt[1] but it should be fine if fmt[1] is already 'R'.
Rasmus Villemoes
reviewerRe: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
Quoted 10 lines · Stephen Boyd
Quoting Rasmus Villemoes (2021-03-24 02:57:13)
Quoted 7 lines
Is there any reason you didn't just make b an optional flag that could be specified with or without R? I suppose the parsing is more difficult with several orthogonal flags (see
escaped_string()), but it's a little easier to understand. Dunno, it's not like we're gonna think of 10 other things that could be printed for a symbol, so perhaps it's fine.
Quoted 4 lines
I think I follow. So %pSb or %pSRb? If it's easier to understand then sure. I was trying to avoid checking another character beyond fmt[1] but it should be fine if fmt[1] is already 'R'.
I don't know. On the one hand, it seems sensible to allow such "flag" modifiers to appear independently and in any order. Because what if some day we think of some other property of the symbol we might want to provide access to via a "z" flag; when to allow all combinations of the R, b and z functionality we'd have to use four more random letters to stand for various combinations of those flags. On the other hand, vsprintf.c is already a complete wild west of odd conventions for %p<foo>, and it's not like symbol_string() gets extended every other release, and I can certainly understand the desire to keep the parsing of fmt minimal. So 'r' to mean 'Rb' is ok by me if you prefer that.
Rasmus
Re: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
Quoting Rasmus Villemoes (2021-03-24 15:21:34)
Quoted 16 lines · Stephen Boyd
Quoted 10 lines
Quoting Rasmus Villemoes (2021-03-24 02:57:13)
Quoted 7 lines
Is there any reason you didn't just make b an optional flag that could be specified with or without R? I suppose the parsing is more difficult with several orthogonal flags (see
escaped_string()), but it's a little easier to understand. Dunno, it's not like we're gonna think of 10 other things that could be printed for a symbol, so perhaps it's fine.Quoted 4 lines
I think I follow. So %pSb or %pSRb? If it's easier to understand then sure. I was trying to avoid checking another character beyond fmt[1] but it should be fine if fmt[1] is already 'R'.
Quoted 10 lines
I don't know. On the one hand, it seems sensible to allow such "flag" modifiers to appear independently and in any order. Because what if some day we think of some other property of the symbol we might want to provide access to via a "z" flag; when to allow all combinations of the R, b and z functionality we'd have to use four more random letters to stand for various combinations of those flags. On the other hand,
vsprintf.cis already a complete wild west of odd conventions for %p<foo>, and it's not likesymbol_string()gets extended every other release, and I can certainly understand the desire to keep the parsing of fmt minimal. So 'r' to mean 'Rb' is ok by me if you prefer that.
I'm inclined to use %pSb and %pSRb. The code looks to simpler and I suppose we can worry about different ordering/combination problems if it comes to it.
---8<---
lib/vsprintf.c +2 -0
@@ -968,6 +968,8 @@ char *symbol_string(char *buf, char *end, void *ptr,#ifdef CONFIG_KALLSYMS if (*fmt =='B' )sprint_backtrace (sym, value); +else if (*fmt =='S' && (fmt[1 ] =='b' || (fmt[1 ] =='R' && fmt[2 ] =='b' ))) +sprint_symbol_stacktrace (sym, value);else if (*fmt !='s' )sprint_symbol (sym, value);else
Petr Mladek
maintainerRe: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
Quoted 26 lines · Stephen Boyd
Quoting Rasmus Villemoes (2021-03-24 15:21:34)
Quoted 11 lines · Stephen Boyd
Quoted 10 lines
Quoting Rasmus Villemoes (2021-03-24 02:57:13)
Quoted 7 lines
Is there any reason you didn't just make b an optional flag that could be specified with or without R? I suppose the parsing is more difficult with several orthogonal flags (see
escaped_string()), but it's a little easier to understand. Dunno, it's not like we're gonna think of 10 other things that could be printed for a symbol, so perhaps it's fine.Quoted 3 lines
Quoted 3 lines
I think I follow. So %pSb or %pSRb? If it's easier to understand then sure. I was trying to avoid checking another character beyond fmt[1] but it should be fine if fmt[1] is already 'R'.
I'm inclined to use %pSb and %pSRb. The code looks to simpler and I suppose we can worry about different ordering/combination problems if it comes to it.
---8<---
lib/vsprintf.c +0
-0
lib/vsprintf.c char *symbol_string(char *buf, char *end, void *ptr, · line 975
Quoted 8 lines
lib/vsprintf.c +2
-0@@ -968,6 +968,8 @@ char *symbol_string(char *buf, char *end, void *ptr,#ifdef CONFIG_KALLSYMS if (*fmt =='B' )sprint_backtrace (sym, value); +else if (*fmt =='S' && (fmt[1 ] =='b' || (fmt[1 ] =='R' && fmt[2 ] =='b' ))) +sprint_symbol_stacktrace (sym, value);else if (*fmt !='s' )sprint_symbol (sym, value);else
I prefer the extra "b" flag as well. It helps me to understand and memorize[*] the meaning.
[*] at least short-time memorize ;-)
Best Regards,
Petr
Petr Mladek
maintainerRe: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
In reply to Stephen BoydQuoted 8 lines · Stephen Boyd
Let's make kernel stacktraces easier to identify by including the build ID[1] of a module if the stacktrace is printing a symbol from a module.
Example:
WARNING: CPU: 3 PID: 3373 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE hci_uart <modules trimmed> CPU: 3 PID: 3373 Comm: bash Not tainted 5.11 #12 a8c0d47f7051f3e6670ceaea724af66a39c6cec8
I tested it with "echo l >/proc/sysrq-trigger" and I got the following output:
[ 71.905593] CPU: 0 PID: 1762 Comm: bash Kdump: loaded Not tainted 5.12.0-rc5-default+ #167 00000080ffffffff0000000000000000
00000000
It does not looks like an unique build-id.
Any idea what was going wrong?
include/linux/kallsyms.h const char *kallsyms_lookup(unsigned long addr, · line 98
Quoted 9 lines
include/linux/kallsyms.h +1
-0@@ -91,6 +93,7 @@ const char *kallsyms_lookup(unsigned long addr,/* Look up a kernel symbol and return it in a text buffer. */ extern int sprint_symbol (char *buffer,unsigned long address); +extern int sprint_symbol_stacktrace (char *buffer,unsigned long address);extern int sprint_symbol_no_offset (char *buffer,unsigned long address);extern int sprint_backtrace (char *buffer,unsigned long address);
Could we use a more clear name? It is hard to guess what is the difference between sprint_*_stacktrace() and sprint_backtrace().
What about sprint_symbol_build_id() ?
Quoted 11 lines
kernel/module.c +3
-0@@ -2770,6 +2771,20 @@ static void add_kallsyms(struct module *mod, const struct load_info *info) } mod->core_kallsyms.num_symtab = ndst; } + +static void init_build_id (struct module *mod,const struct load_info *info) +{
kernel/module.c static void add_kallsyms(struct module *mod, const struct load_info *info) · line 2786
Quoted 10 lines
kernel/module.c +10
-0+const Elf_Shdr *sechdr; +unsigned int i; + +for (i =0 ; i < info->hdr->e_shnum; i++) { + sechdr = &info->sechdrs[i]; +if (!sect_empty (sechdr) && sechdr->sh_type ==SHT_NOTE && + !build_id_parse_buf ((void *)sechdr->sh_addr, mod->build_id, + sechdr->sh_size)) +break ; + }
Just to be sure. Is this really reliable way how to find the build ID, please?
kernel/module.c static inline void layout_symtab(struct module *mod, struct load_info *info) · line 2795
Quoted 8 lines
kernel/module.c +1
-0+}#else static inline void layout_symtab (struct module *mod,struct load_info *info) {@@ -2778,6 +2793,10 @@ static inline void layout_symtab(struct module *mod, struct load_info *info)static void add_kallsyms (struct module *mod,const struct load_info *info) { }
Best Regards,
Petr
Re: [PATCH v2 04/12] module: Add printk format to add module build ID to stacktraces
Quoting Petr Mladek (2021-03-30 03:29:24)
Quoted 10 lines · Stephen Boyd
Quoted 8 lines
Let's make kernel stacktraces easier to identify by including the build ID[1] of a module if the stacktrace is printing a symbol from a module.
Example:
WARNING: CPU: 3 PID: 3373 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE hci_uart <modules trimmed> CPU: 3 PID: 3373 Comm: bash Not tainted 5.11 #12 a8c0d47f7051f3e6670ceaea724af66a39c6cec8
Quoted 8 lines
I tested it with "echo l >/proc/sysrq-trigger" and I got the following output:
[ 71.905593] CPU: 0 PID: 1762 Comm: bash Kdump: loaded Not tainted 5.12.0-rc5-default+ #167 00000080ffffffff000000000000000000000000
It does not looks like an unique build-id.
Any idea what was going wrong?
Hmm not sure. What does 'hexdump /sys/kernel/notes' show or 'file vmlinux' parse out of the kernel image?
Quoted 14 lines
include/linux/kallsyms.h const char *kallsyms_lookup(unsigned long addr, · line 98
Quoted 10 lines
include/linux/kallsyms.h +1
-0@@ -91,6 +93,7 @@ const char *kallsyms_lookup(unsigned long addr,/* Look up a kernel symbol and return it in a text buffer. */ extern int sprint_symbol (char *buffer,unsigned long address); +extern int sprint_symbol_stacktrace (char *buffer,unsigned long address);extern int sprint_symbol_no_offset (char *buffer,unsigned long address);extern int sprint_backtrace (char *buffer,unsigned long address);Could we use a more clear name? It is hard to guess what is the difference between sprint_*_stacktrace() and
sprint_backtrace().
Quoted 1 line
What about
sprint_symbol_build_id()?
Sure.
Quoted 18 lines
Quoted 18 lines
kernel/module.c +8
-0@@ -2770,6 +2771,20 @@ static void add_kallsyms(struct module *mod, const struct load_info *info) } mod->core_kallsyms.num_symtab = ndst; } + +static void init_build_id (struct module *mod,const struct load_info *info) +{ +const Elf_Shdr *sechdr; +unsigned int i; + +for (i =0 ; i < info->hdr->e_shnum; i++) { + sechdr = &info->sechdrs[i];
Quoted 9 lines
kernel/module.c static void add_kallsyms(struct module *mod, const struct load_info *info) · line 2786
Quoted 5 lines
kernel/module.c +5
-0+if (!sect_empty (sechdr) && sechdr->sh_type ==SHT_NOTE && + !build_id_parse_buf ((void *)sechdr->sh_addr, mod->build_id, + sechdr->sh_size)) +break ; + }Just to be sure. Is this really reliable way how to find the build ID, please?
The build ID is always part of a note section, so if we parse the notes and find a build ID type of section (see nhdr->n_type == BUILD_ID) with a GNU name then we know we have the right section and will be able to parse the ID out of the buffer.
PATCH v2 5/12 arm64: stacktrace: Use %pSb for backtrace printing
In reply to Stephen BoydLet's use the new printk format to print the stacktrace entry when printing a backtrace to the kernel logs. This will include any module's build ID[1] in it so that offline/crash debugging can easily locate the debuginfo for a module via something like debuginfod[2].
--- arch/arm64/kernel/stacktrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/stacktrace.c +1 -1
@@ -129,7 +129,7 @@ NOKPROBE_SYMBOL(walk_stackframe);static void dump_backtrace_entry (unsigned long where,const char *loglvl) { -printk ("%s %pS\n" , loglvl, (void *)where); +printk ("%s %pSb\n" , loglvl, (void *)where); } void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk,
Signature · 1 line
https://chromeos.dev
PATCH v2 6/12 x86/dumpstack: Use %pSb for backtrace printing
In reply to Stephen BoydLet's use the new printk format to print the stacktrace entry when printing a backtrace to the kernel logs. This will include any module's build ID[1] in it so that offline/crash debugging can easily locate the debuginfo for a module via something like debuginfod[2].
---
arch/x86/kernel/dumpstack.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/dumpstack.c +2 -2
@@ -143,9 +143,9 @@ void show_opcodes(struct pt_regs *regs, const char *loglvl)void show_ip (struct pt_regs *regs,const char *loglvl) {#ifdef CONFIG_X86_32 -printk ("%sEIP: %pS\n" , loglvl, (void *)regs->ip); +printk ("%sEIP: %pSb\n" , loglvl, (void *)regs->ip);#else -printk ("%sRIP: %04x:%pS\n" , loglvl, (int )regs->cs, (void *)regs->ip); +printk ("%sRIP: %04x:%pSb\n" , loglvl, (int )regs->cs, (void *)regs->ip);#endif show_opcodes (regs, loglvl); }
Signature · 1 line
https://chromeos.dev
PATCH v2 7/12 scripts/decode_stacktrace.sh: Support debuginfod
In reply to Stephen BoydNow that stacktraces contain the build ID information we can update this script to use debuginfod-find to locate the debuginfo for the vmlinux and modules automatically. This can replace the existing code that requires specifying a path to vmlinux or tries to find the vmlinux and modules automatically by using the release number. Work it into the script as a fallback option if the vmlinux isn't specified on the commandline.
--- scripts/decode_stacktrace.sh | 81 +++++++++++++++++++++++++++++++----- 1 file changed, 70 insertions(+), 11 deletions(-)
scripts/decode_stacktrace.sh +70 -11
@@ -3,11 +3,10 @@ # (c) 2014, Sasha Levin <[email protected]> #set -x -if [[ $# < 1 ]]; then +usage() { echo "Usage:" echo " $0 -r <release> | <vmlinux> [base path] [modules path]" - exit 1 -fi +} if [[ $1 == "-r" ]] ; then vmlinux=""
@@ -24,6 +23,7 @@ if [[ $1 == "-r" ]] ; then if [[ $vmlinux == "" ]] ; then echo "ERROR! vmlinux image for release $release is not found" >&2 + usage exit 2 fi else
@@ -31,12 +31,35 @@ else basepath=${2-auto} modpath=$3 release="" + debuginfod= + + # Can we use debuginfod-find? + if type debuginfod-find >/dev/null 2>&1 ; then + debuginfod=${1-only} + fi + + if [[ $vmlinux == "" && -z $debuginfod ]] ; then + echo "ERROR! vmlinux image must be specified" >&2 + usage + exit 1 + fi fi declare -A cache declare -A modcache find_module() { + if [[ -n $debuginfod ]] ; then + if [[ -n $modbuildid ]] ; then + debuginfod-find debuginfo $modbuildid && return + fi + + # Only using debuginfod so don't try to find vmlinux module path + if [[ $debuginfod == "only" ]] ; then + return + fi + fi + if [[ "$modpath" != "" ]] ; then for fn in $(find "$modpath" -name "${module//_/[-_]}.ko*") ; do if readelf -WS "$fn" | grep -qwF .debug_line ; then
@@ -150,6 +173,27 @@ parse_symbol() { symbol="$segment$name ($code)" } +debuginfod_get_vmlinux() { + local vmlinux_buildid=${1##* } + + if [[ $vmlinux != "" ]]; then + return + fi + + if [[ $vmlinux_buildid =~ ^[0-9a-f]+ ]]; then + vmlinux=$(debuginfod-find debuginfo $vmlinux_buildid) + if [[ $? -ne 0 ]] ; then + echo "ERROR! vmlinux image not found via debuginfod-find" >&2 + usage + exit 2 + fi + return + fi + echo "ERROR! Build ID for vmlinux not found. Try passing -r or specifying vmlinux" >&2 + usage + exit 2 +} + decode_code() { local scripts=`dirname "${BASH_SOURCE[0]}"`
@@ -157,6 +201,14 @@ decode_code() { } handle_line() { + if [[ $basepath == "auto" && $vmlinux != "" ]] ; then + module="" + symbol="kernel_init+0x0/0x0" + parse_symbol + basepath=${symbol#kernel_init (} + basepath=${basepath%/init/main.c:*)} + fi + local words # Tokenize
@@ -182,16 +234,28 @@ handle_line() { fi done + if [[ ${words[$last]} =~ ^[0-9a-f]+\] ]]; then + words[$last-1]="${words[$last-1]} ${words[$last]}" + unset words[$last] + last=$(( $last - 1 )) + fi + if [[ ${words[$last]} =~ \[([^]]+)\] ]]; then module=${words[$last]} module=${module#\[} module=${module%\]} + modbuildid=${module#* } + module=${module% *} + if [[ $modbuildid == $module ]]; then + modbuildid= + fi symbol=${words[$last-1]} unset words[$last-1] else # The symbol is the last element, process it symbol=${words[$last]} module= + modbuildid= fi unset words[$last]
@@ -201,14 +265,6 @@ handle_line() { echo "${words[@]}" "$symbol $module" } -if [[ $basepath == "auto" ]] ; then - module="" - symbol="kernel_init+0x0/0x0" - parse_symbol - basepath=${symbol#kernel_init (} - basepath=${basepath%/init/main.c:*)} -fi - while read line; do # Let's see if we have an address in the line if [[ $line =~ \[\<([^]]+)\>\] ]] ||
@@ -218,6 +274,9 @@ while read line; do # Is it a code line? elif [[ $line == *Code:* ]]; then decode_code "$line" + # Is it a version line? + elif [[ -n $debuginfod && $line =~ PID:\ [0-9]+\ Comm: ]]; then + debuginfod_get_vmlinux "$line" else # Nothing special in this line, show it as is echo "$line"
Signature · 1 line
https://chromeos.dev
Andy Shevchenko
reviewerRe: [PATCH v2 07/12] scripts/decode_stacktrace.sh: Support debuginfod
Quoted 7 lines · Stephen Boyd
Now that stacktraces contain the build ID information we can update this script to use debuginfod-find to locate the debuginfo for the vmlinux and modules automatically. This can replace the existing code that requires specifying a path to vmlinux or tries to find the vmlinux and modules automatically by using the release number. Work it into the script as a fallback option if the vmlinux isn't specified on the commandline.
One wish for the shell scripts: try to avoid bashisms as much as possible, please.
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [PATCH v2 07/12] scripts/decode_stacktrace.sh: Support debuginfod
Quoting Andy Shevchenko (2021-03-24 04:27:53)
Quoted 9 lines · Stephen Boyd
Quoted 7 lines
Now that stacktraces contain the build ID information we can update this script to use debuginfod-find to locate the debuginfo for the vmlinux and modules automatically. This can replace the existing code that requires specifying a path to vmlinux or tries to find the vmlinux and modules automatically by using the release number. Work it into the script as a fallback option if the vmlinux isn't specified on the commandline.
Quoted 3 lines
One wish for the shell scripts: try to avoid bashisms as much as possible, please.
I wish too but the script has /bin/bash at the start and I was trying to follow similar style to what was already in here. Feel free to send a patch to remove the bashisms.
PATCH v2 8/12 scripts/decode_stacktrace.sh: Silence stderr messages from addr2line/nm
In reply to Stephen BoydSometimes if you're using tools that have linked things improperly or have new features/sections that older tools don't expect you'll see warnings printed to stderr. We don't really care about these warnings, so let's just silence these messages to cleanup output of this script.
--- scripts/decode_stacktrace.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
scripts/decode_stacktrace.sh +3 -3
@@ -74,7 +74,7 @@ find_module() { find_module && return if [[ $release == "" ]] ; then - release=$(gdb -ex 'print init_uts_ns.name.release' -ex 'quit' -quiet -batch "$vmlinux" | sed -n 's/\$1 = "\(.*\)".*/\1/p') + release=$(gdb -ex 'print init_uts_ns.name.release' -ex 'quit' -quiet -batch "$vmlinux" 2>/dev/null | sed -n 's/\$1 = "\(.*\)".*/\1/p') fi for dn in {/usr/lib/debug,}/lib/modules/$release ; do
@@ -128,7 +128,7 @@ parse_symbol() { if [[ "${cache[$module,$name]+isset}" == "isset" ]]; then local base_addr=${cache[$module,$name]} else - local base_addr=$(nm "$objfile" | awk '$3 == "'$name'" && ($2 == "t" || $2 == "T") {print $1; exit}') + local base_addr=$(nm "$objfile" 2>/dev/null | awk '$3 == "'$name'" && ($2 == "t" || $2 == "T") {print $1; exit}') if [[ $base_addr == "" ]] ; then # address not found return
@@ -152,7 +152,7 @@ parse_symbol() { if [[ "${cache[$module,$address]+isset}" == "isset" ]]; then local code=${cache[$module,$address]} else - local code=$(${CROSS_COMPILE}addr2line -i -e "$objfile" "$address") + local code=$(${CROSS_COMPILE}addr2line -i -e "$objfile" "$address" 2>/dev/null) cache[$module,$address]=$code fi
Signature · 1 line
https://chromeos.dev
PATCH v2 9/12 scripts/decode_stacktrace.sh: Indicate 'auto' can be used for base path
In reply to Stephen BoydAdd "auto" to the usage message so that it's a little clearer that you can pass "auto" as the second argument. When passing "auto" the script tries to find the base path automatically instead of requiring it be passed on the commandline. Also use [<variable>] to indicate the variable argument and that it is optional so that we can differentiate from the literal "auto" that should be passed.
--- scripts/decode_stacktrace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/decode_stacktrace.sh +1 -1
@@ -5,7 +5,7 @@ usage() { echo "Usage:" - echo " $0 -r <release> | <vmlinux> [base path] [modules path]" + echo " $0 -r <release> | <vmlinux> [<base path>|auto] [<modules path>]" } if [[ $1 == "-r" ]] ; then
Signature · 1 line
https://chromeos.dev
PATCH v2 10/12 buildid: Mark some arguments const
In reply to Stephen BoydThese arguments are never modified so they can be marked const to indicate as such.
--- lib/buildid.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
lib/buildid.c +4 -4
@@ -47,10 +47,10 @@ static int parse_build_id_buf(unsigned char *build_id,return -EINVAL ; } -static inline int parse_build_id (void *page_addr, +static inline int parse_build_id (const void *page_addr,unsigned char *build_id,__u32 *size, -void *note_start, +const void *note_start,Elf32_Word note_size) {/* check for overflow */
@@ -65,7 +65,7 @@ static inline int parse_build_id(void *page_addr, }/* Parse build ID from 32-bit ELF */ -static int get_build_id_32 (void *page_addr,unsigned char *build_id, +static int get_build_id_32 (const void *page_addr,unsigned char *build_id,__u32 *size) {Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr;
@@ -90,7 +90,7 @@ static int get_build_id_32(void *page_addr, unsigned char *build_id, }/* Parse build ID from 64-bit ELF */ -static int get_build_id_64 (void *page_addr,unsigned char *build_id, +static int get_build_id_64 (const void *page_addr,unsigned char *build_id,__u32 *size) {Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr;
Signature · 1 line
https://chromeos.dev
PATCH v2 11/12 buildid: Fix kernel-doc notation
In reply to Stephen BoydKernel doc should use "Return:" instead of "Returns" to properly reflect the return values.
--- lib/buildid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/buildid.c +1 -1
@@ -120,7 +120,7 @@ static int get_build_id_64(const void *page_addr, unsigned char *build_id,* @build_id: buffer to store build id, at least BUILD_ID_SIZE long * @size: returns actual build id size in case of success * -* Returns 0 on success, otherwise error (< 0). +* Return: 0 on success, -EINVAL otherwise */ int build_id_parse (struct vm_area_struct *vma,unsigned char *build_id,__u32 *size)
Signature · 1 line
https://chromeos.dev
PATCH v2 12/12 kdump: Use vmlinux_build_id() to simplify
In reply to Stephen BoydWe can use the vmlinux_build_id() helper here now instead of open coding it. This consolidates code and possibly avoids calculating the build ID twice in the case of a crash with a stacktrace.
--- include/linux/crash_core.h | 6 +----- kernel/crash_core.c | 41 ++------------------------------------ 2 files changed, 3 insertions(+), 44 deletions(-)
include/linux/crash_core.h +1 -5
@@ -39,7 +39,7 @@ phys_addr_t paddr_vmcoreinfo_note(void);#define VMCOREINFO_OSRELEASE (value) \ vmcoreinfo_append_str("OSRELEASE=%s\n", value)#define VMCOREINFO_BUILD_ID (value) \ - vmcoreinfo_append_str("BUILD-ID=%s\n", value) + vmcoreinfo_append_str("BUILD-ID=%20phN\n", value)#define VMCOREINFO_PAGESIZE (value) \ vmcoreinfo_append_str("PAGESIZE=%ld\n", value)#define VMCOREINFO_SYMBOL (name) \
@@ -69,10 +69,6 @@ extern unsigned char *vmcoreinfo_data;extern size_t vmcoreinfo_size;extern u32 *vmcoreinfo_note; -/* raw contents of kernel .notes section */ -extern const void __start_notes__weak ; -extern const void __stop_notes__weak ; -Elf_Word *append_elf_note (Elf_Word *buf,char *name,unsigned int type,void *data,size_t data_len);void final_note (Elf_Word *buf);
kernel/crash_core.c +2 -39
59 lines not on this page · raw message
Signature · 7 lines
https://chromeos.dev _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 6 lines · Stephen Boyd
x5 : 0000000000000000 x4 : 0000000000000001
x3 : 0000000000000008 x2 :ffffff93fef25a70
x1 :ffffff93fef15788x0 :ffffffe3622352e0Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] direct_entry+0x16c/0x1b4 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e]
Yikes. No, please do not make the backtraces a complete mess for something that serves absolutely no need.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 9 lines · Christoph Hellwig
Quoted 6 lines · Stephen Boyd
x5 : 0000000000000000 x4 : 0000000000000001
x3 : 0000000000000008 x2 :ffffff93fef25a70
x1 :ffffff93fef15788x0 :ffffffe3622352e0Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] direct_entry+0x16c/0x1b4 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e]Yikes. No, please do not make the backtraces a complete mess for something that serves absolutely no need.
Would a "verbose" flag be acceptable solution? Something like write 1 to /sys/kernel/debug/verbose_stack to get the extra info.
I think I see a need for it.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoting peter enderborg (2021-03-25 04:06:17)
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Stephen Boyd
Quoted 6 lines
x5 : 0000000000000000 x4 : 0000000000000001
x3 : 0000000000000008 x2 :ffffff93fef25a70
x1 :ffffff93fef15788x0 :ffffffe3622352e0Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] direct_entry+0x16c/0x1b4 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e]Yikes. No, please do not make the backtraces a complete mess for something that serves absolutely no need.
It serves a need. Please look at the patches to understand that I'm adding the buildid to automatically find the associated debug information on distros.
Quoted 5 lines
Would a "verbose" flag be acceptable solution? Something like write 1 to
/sys/kernel/debug/verbose_stackto get the extra info.I think I see a need for it.
Or a kernel config option and a commandline parameter? That would be OK for me as I said on v1 of this series. I'll add that in for the next patch series given all the distaste for some more hex characters next to the module name.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Petr Mladek
maintainerQuoted 19 lines · Stephen Boyd
Quoting peter enderborg (2021-03-25 04:06:17)
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Stephen Boyd
Quoted 6 lines
x5 : 0000000000000000 x4 : 0000000000000001
x3 : 0000000000000008 x2 :ffffff93fef25a70
x1 :ffffff93fef15788x0 :ffffffe3622352e0Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] direct_entry+0x16c/0x1b4 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e]Yikes. No, please do not make the backtraces a complete mess for something that serves absolutely no need.
It serves a need. Please look at the patches to understand that I'm adding the buildid to automatically find the associated debug information on distros.
Quoted 3 lines
Would a "verbose" flag be acceptable solution? Something like write 1 to
/sys/kernel/debug/verbose_stackto get the extra info.
Quoted 7 lines
Quoted 2 lines
I think I see a need for it.
Or a kernel config option and a commandline parameter? That would be OK for me as I said on v1 of this series. I'll add that in for the next patch series given all the distaste for some more hex characters next to the module name.
IMHO, a build configure option would fit the best here.
It does not make sense to show the ID when the kernel vendor does not have a service to download the related binaries. But it makes sense to show the buildid by default when the provider/distro has the service and want to use the ID when handling bug reports.
We could always add boot/run time options when people really need it.
Best Regards,
Petr
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
A message that is not in the archive (sent off-list or not yet received): [email protected]
HTML mail?
Quoting Konstantin Khlebnikov (2021-03-24 01:23:55)
Quoted 5 lines
24.03.2021, 05:04, "Stephen Boyd" <[email protected]>:
Looks too noisy for me. Maybe print id in the line "Modules linked in:"? I suppose only out-of-tree modules need this?
Please see this note in patch 4:
Originally, I put this on the %pS format, but that was quickly rejected given that %pS is used in other places such as ftrace where build IDs aren't meaningful. There was some discussions on the list to put every module build ID into the "Modules linked in:" section of the stacktrace message but that quickly becomes very hard to read once you have more than three or four modules linked in. It also provides too much information when we don't expect each module to be traversed in a stacktrace. Having the build ID for modules that aren't important just makes things messy. Splitting it to multiple lines for each module quickly explodes the number of lines printed in an oops too, possibly wrapping the warning off the console. And finally, trying to stash away each module used in a callstack to provide the ID of each symbol printed is cumbersome and would require changes to each architecture to stash away modules and return their build IDs once unwinding has completed.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 49 lines · Stephen Boyd
This series adds the kernel's build ID[1] to the stacktrace header printed in oops messages, warnings, etc. and the build ID for any module that appears in the stacktrace after the module name. The goal is to make the stacktrace more self-contained and descriptive by including the relevant build IDs in the kernel logs when something goes wrong. This can be used by post processing tools like
script/decode_stacktrace.shand kernel developers to easily locate the debug info associated with a kernel crash and line up what line and file things started falling apart at.To show how this can be used I've included a patch to
decode_stacktrace.shthat downloads the debuginfo from a debuginfod server.This also includes some patches to make the
buildid.cfile use more const arguments and consolidate logic intobuildid.cfrom kdump. These are left to the end as they were mostly cleanup patches. I don't know who exactly maintains this so I guess Andrew is the best option to merge all this code.Here's an example lkdtm stacktrace on arm64.
WARNING: CPU: 4 PID: 3255 at drivers/misc/lkdtm/bugs.c:83 lkdtm_WARNING+0x28/0x30 [lkdtm] Modules linked in: lkdtm rfcomm algif_hash algif_skcipher af_alg xt_cgroup uinput xt_MASQUERADE CPU: 4 PID: 3255 Comm: bash Not tainted 5.11 #3 aa23f7a1231c229de205662d5a9e0d4c580f19a1 Hardware name: Google Lazor (rev3+) with KB Backlight (DT) pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--) pc : lkdtm_WARNING+0x28/0x30 [lkdtm] lr : lkdtm_do_action+0x24/0x40 [lkdtm] sp : ffffffc0134fbca0 x29: ffffffc0134fbca0 x28: ffffff92d53ba240 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: ffffffe3622352c0 x23: 0000000000000020 x22: ffffffe362233366 x21: ffffffe3622352e0 x20: ffffffc0134fbde0 x19: 0000000000000008 x18: 0000000000000000 x17: ffffff929b6536fc x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000012 x13: ffffffe380ed892c x12: ffffffe381d05068 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000001 x8 : ffffffe362237000 x7 : aaaaaaaaaaaaaaaa x6 : 0000000000000000 x5 : 0000000000000000 x4 : 0000000000000001 x3 : 0000000000000008 x2 : ffffff93fef25a70 x1 : ffffff93fef15788 x0 : ffffffe3622352e0 Call trace: lkdtm_WARNING+0x28/0x30 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] direct_entry+0x16c/0x1b4 [lkdtm ed5019fdf5e53be37cb1ba7899292d7e143b259e] full_proxy_write+0x74/0xa4 vfs_write+0xec/0x2e8
Quoted 8 lines
ksys_write+0x84/0xf0 __arm64_sys_write+0x24/0x30 el0_svc_common+0xf4/0x1c0 do_el0_svc_compat+0x28/0x3c el0_svc_compat+0x10/0x1c el0_sync_compat_handler+0xa8/0xcc el0_sync_compat+0x178/0x180 ---[ end trace 3d95032303e59e68 ]---
How will this work with the ftrace?
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoting peter enderborg (2021-03-25 04:14:31)
Quoted 6 lines
Quoted 3 lines
el0_sync_compat_handler+0xa8/0xcc el0_sync_compat+0x178/0x180 ---[ end trace 3d95032303e59e68 ]---How will this work with the ftrace?
It won't affect ftrace, if that's the question you're asking.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec