tracing - signal tracer, support multiple pids in set_pid_ftrace
cover tracing - signal tracer, support multiple pids in set_pid_ftrace
I was debugging some ptrace applications and got to the idea of tracing signals - signal tracer. There's probably possibility to trace more info than just signal delivery info (like ignored signals deliveries, queue status...).
While doing this I realized it might be worth to have ability to trace more independent processes via set_pid_ftrace file.
Althought I'm sending this as RFC, attached patches work for me. I'm open to any suggestions/comments/ideas.
1/2 - signal trace
2/2 - multiple pids support for set_pid_ftrace file
thanks,
jirka
--- include/linux/signaltrace.h | 22 ++++ include/linux/tracehook.h | 14 +++ kernel/signal.c | 3 + kernel/trace/Kconfig | 11 ++ kernel/trace/Makefile | 1 + kernel/trace/ftrace.c | 226 +++++++++++++++++++++++++++++++------------ kernel/trace/trace.h | 13 ++- kernel/trace/trace_signal.c | 132 +++++++++++++++++++++++++ 8 files changed, 356 insertions(+), 66 deletions(-)
RFC PATCH 1/2 tracing - signal tracer
Implements the tracer for the signals. It is possible to narrow the trace using the "set_ftrace_pid" pid filter file.
Usage (running some of the pthread-tests pkg's binaries):
sh-4.0# echo signal > current_tracer
sh-4.0# /tests/signal-loss
sh-4.0# /tests/ptrace_event_clone
sh-4.0# /tests/ptrace-on-job-control-stopped
sh-4.0# /tests/clone-ptrace
sh-4.0# cat trace
# tracer: signal
#
# TIMESTAMP CPU# FROM-PID TO-PID SIGNAL-NUM
# | | | | | | | |
217.117000 [000] modprobe-8741 khelper-8740 SIGCHLD-17
217.142000 [001] signal-loss-8742 signal-loss-8743 SIGSTOP-19
217.143000 [001] signal-loss-8742 signal-loss-8743 SIGINT-2
218.145000 [001] <idle>-0 signal-loss-8742 SIGALRM-14
218.149000 [001] signal-loss-8742 signal-loss-8743 SIGKILL-9
218.159000 [001] signal-loss-8742 sh-283 SIGCHLD-17
221.108000 [001] modprobe-8745 khelper-8744 SIGCHLD-17
221.145000 [001] cat-8746 sh-283 SIGCHLD-17
254.790000 [000] modprobe-8748 khelper-8747 SIGCHLD-17
254.820000 [001] ptrace_event_cl-8750 ptrace_event_cl-8750 SIGSTOP-19
254.824000 [001] ptrace_event_cl-8749 ptrace_event_cl-8750 SIGKILL-9
254.824000 [001] ptrace_event_cl-8749 ptrace_event_cl-8751 SIGKILL-9
254.827000 [001] ptrace_event_cl-8749 sh-283 SIGCHLD-17
256.399000 [000] modprobe-8753 khelper-8752 SIGCHLD-17
256.448000 [000] cat-8754 sh-283 SIGCHLD-17
333.634000 [000] modprobe-8756 khelper-8755 SIGCHLD-17
333.663000 [000] ptrace-on-job-c-8757 ptrace-on-job-c-8758 SIGSTOP-19
333.664000 [000] ptrace-on-job-c-8757 ptrace-on-job-c-8758 SIGKILL-9
333.666000 [000] ptrace-on-job-c-8757 init-1 SIGCHLD-17
333.666000 [000] ptrace-on-job-c-8757 sh-283 SIGCHLD-17
335.323000 [000] modprobe-8760 khelper-8759 SIGCHLD-17
448.054000 [001] clone-ptrace-8808 clone-ptrace-8808 SIGSTOP-19
448.057000 [001] clone-ptrace-8807 clone-ptrace-8808 SIGKILL-9
448.058000 [001] clone-ptrace-8807 clone-ptrace-8809 SIGKILL-9
448.063000 [001] clone-ptrace-8807 init-1 SIGCHLD-17
448.063000 [001] clone-ptrace-8807 sh-283 SIGCHLD-17
448.064000 [000] clone-ptrace-8809 clone-ptrace-8809 SIGTRAP-5
448.066000 [000] clone-ptrace-8809 init-1 SIGCHLD-17
449.321000 [001] modprobe-8811 khelper-8810 SIGCHLD-17sh-4.0#
wbr,
jirka
--- include/linux/signaltrace.h | 22 +++++++ include/linux/tracehook.h | 14 +++++ kernel/signal.c | 3 + kernel/trace/Kconfig | 11 ++++ kernel/trace/Makefile | 1 + kernel/trace/trace.h | 9 +++ kernel/trace/trace_signal.c | 132 +++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 192 insertions(+), 0 deletions(-) create mode 100644 include/linux/signaltrace.h create mode 100644 kernel/trace/trace_signal.c
include/linux/signaltrace.h new file +22 -0
@@ -0,0 +1,22 @@ +/* +* Copyright (C) 2009 Jiri Olsa <[email protected]> +* +* This file is released under GPL version 2. +*/ + +#ifndef _LINUX_SIGNALTRACE_H +#define _LINUX_SIGNALTRACE_H + +#ifdef __KERNEL__ + +#ifdef CONFIG_SIGNAL_TRACER +void trace_signal (struct task_struct *to,int sig); +#else +static inline void trace_signal (struct task_struct *to,int sig) +{ +} +#endif + +#endif /* __KERNEL__ */ + +#endif /* _LINUX_SIGNALTRACE_H */
include/linux/tracehook.h +14 -0
@@ -49,6 +49,7 @@#include <linux/sched.h> #include <linux/ptrace.h> #include <linux/security.h> +#include <linux/signaltrace.h> struct linux_binprm ;/**
@@ -462,6 +463,19 @@ static inline int tracehook_get_signal(struct task_struct *task, }/** +* tracehook_send_signal +* @to: @current +* @sig: number of signal being sent +*/ +static inline int tracehook_send_signal (struct task_struct *to,int sig) +{ +#ifdef CONFIG_SIGNAL_TRACER +trace_signal (to, sig); +#endif +return 0 ; +} + +/** * tracehook_notify_jctl - report about job control stop/continue * @notify: nonzero if this is the last thread in the group to stop * @why: %CLD_STOPPED or %CLD_CONTINUED
kernel/signal.c +3 -0
@@ -842,6 +842,9 @@ static int __send_signal(int sig, struct siginfo *info, struct task_struct *t,return 0 ; pending = group ? &t->signal->shared_pending : &t->pending; + +tracehook_send_signal (t, sig); +/* * Short-circuit ignored signals and support queuing * exactly one non-rt signal, so that we can get more
kernel/trace/Kconfig +11 -0
@@ -396,6 +396,17 @@ config HW_BRANCH_TRACER This tracer records all branches on the system in a circular buffer giving access to the last N branches for each cpu. +config SIGNAL_TRACER + bool "Trace signals" + select TRACING + help + This tracer records sent signals in a circular buffer. + It is possible to narrow the trace using the "set_ftrace_pid" + pid filter file. + + If unsure, say N. + + config KMEMTRACE bool "Trace SLAB allocations" select GENERIC_TRACER
kernel/trace/Makefile +1 -0
@@ -55,5 +55,6 @@ obj-$(CONFIG_FTRACE_SYSCALLS) += trace_syscalls.o obj-$(CONFIG_EVENT_PROFILE) += trace_event_profile.o obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o obj-$(CONFIG_KSYM_TRACER) += trace_ksym.o +obj-$(CONFIG_SIGNAL_TRACER) += trace_signal.o libftrace-y := ftrace.o
kernel/trace/trace.h +9 -0
@@ -44,6 +44,7 @@ enum trace_type {TRACE_POWER ,TRACE_BLK ,TRACE_KSYM , +TRACE_SIGNAL , __TRACE_LAST_TYPE, };
@@ -218,6 +219,7 @@ extern void __ftrace_bad_type(void); IF_ASSIGN(var, ent, struct kmemtrace_free_entry, \TRACE_KMEM_FREE ); \ IF_ASSIGN(var, ent, struct ksym_trace_entry, TRACE_KSYM);\ + IF_ASSIGN(var, ent, struct signal_trace_entry, TRACE_SIGNAL);\__ftrace_bad_type (); \ }while (0 )
@@ -254,6 +256,13 @@ struct ksym_trace_entry {char cmd[TASK_COMM_LEN ]; }; +/* Signal trace entry. */ +struct signal_trace_entry { +struct trace_entry ent; +int pid; +int sig; +}; +/** * struct tracer - a specific tracer and its callbacks to interact with debugfs * @name: the name chosen to select it on the available_tracers file
kernel/trace/trace_signal.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
1.6.2.5
RFC PATCH 2/2 tracing - support multiple pids in set_pid_ftrace file
In reply to [email protected]Adding the possibility to set more than 1 pid in the set_pid_ftrace file, thus allowing to trace more than 1 independent processes.
Usage:
sh-4.0# echo 284 > ./set_ftrace_pid
sh-4.0# cat ./set_ftrace_pid
284
sh-4.0# echo 1 >> ./set_ftrace_pid
sh-4.0# echo 0 >> ./set_ftrace_pid
sh-4.0# cat ./set_ftrace_pid
swapper tasks
1
284
sh-4.0# echo 4 > ./set_ftrace_pid
sh-4.0# cat ./set_ftrace_pid
4
sh-4.0# echo > ./set_ftrace_pid
sh-4.0# cat ./set_ftrace_pid
no pid
sh-4.0#
wbr,
jirka
--- kernel/trace/ftrace.c | 226 +++++++++++++++++++++++++++++++++++-------------- kernel/trace/trace.h | 4 +- 2 files changed, 164 insertions(+), 66 deletions(-)
kernel/trace/ftrace.c +162 -64
@@ -60,6 +60,13 @@ static int last_ftrace_enabled;/* Quick disabling of function tracer. */ int function_trace_stop; +/* List for set_ftrace_pid's pids */ +LIST_HEAD (ftrace_pids); +struct ftrace_pid { +struct list_head list; +struct pid *pid; +}; +/* * ftrace_disabled is set when an anomaly is discovered. * ftrace_disabled is much stronger than ftrace_enabled.
@@ -155,7 +162,7 @@ static int __register_ftrace_function(struct ftrace_ops *ops)else func = ftrace_list_func; -if (ftrace_pid_trace) { +if (!list_empty (&ftrace_pids)) {set_ftrace_pid_function (func); func = ftrace_pid_func; }
@@ -203,7 +210,7 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)if (ftrace_list->next == &ftrace_list_end) {ftrace_func_t func = ftrace_list->func; -if (ftrace_pid_trace) { +if (!list_empty (&ftrace_pids)) {set_ftrace_pid_function (func); func = ftrace_pid_func; }
@@ -227,7 +234,7 @@ static void ftrace_update_pid_func(void) func = ftrace_trace_function; -if (ftrace_pid_trace) { +if (!list_empty (&ftrace_pids)) {set_ftrace_pid_function (func); func = ftrace_pid_func; }else {
@@ -818,7 +825,6 @@ static __init void ftrace_profile_debugfs(struct dentry *d_tracer)#endif /* CONFIG_FUNCTION_PROFILER */ /* set when tracing only a pid */ -struct pid *ftrace_pid_trace;static struct pid *const ftrace_swapper_pid = &init_struct_pid;#ifdef CONFIG_DYNAMIC_FTRACE
@@ -2795,23 +2801,6 @@ static inline void ftrace_startup_enable(int command) { }# define ftrace_shutdown_sysctl () do { } while (0)#endif /* CONFIG_DYNAMIC_FTRACE */ -static ssize_t -ftrace_pid_read (struct file *file,char __user *ubuf, -size_t cnt,loff_t *ppos) -{ -char buf[64 ]; -int r; - -if (ftrace_pid_trace == ftrace_swapper_pid) - r =sprintf (buf,"swapper tasks\n" ); -else if (ftrace_pid_trace) - r =sprintf (buf,"%u\n" ,pid_vnr (ftrace_pid_trace)); -else - r =sprintf (buf,"no pid\n" ); - -return simple_read_from_buffer (ubuf, cnt, ppos, buf, r); -} -static void clear_ftrace_swapper (void ) {struct task_struct *p;
@@ -2880,11 +2869,141 @@ static void set_ftrace_pid_task(struct pid *pid)set_ftrace_pid (pid); } +static int ftrace_pid_add (int p) +{ +struct pid *pid; +struct ftrace_pid *fpid; +int ret = -EINVAL ; + +mutex_lock (&ftrace_lock); + +if (!p) + pid = ftrace_swapper_pid; +else + pid =find_get_pid (p); + +if (!pid) + goto out; + +list_for_each_entry (fpid, &ftrace_pids, list) +if (fpid->pid == pid) + goto out_put; + + ret = -ENOMEM ; + + fpid =kmalloc (sizeof (*fpid),GFP_KERNEL ); +if (!fpid) + goto out_put; + +list_add (&fpid->list, &ftrace_pids); + fpid->pid = pid; + +set_ftrace_pid_task (pid); + +ftrace_update_pid_func (); +ftrace_startup_enable (0 ); + +mutex_unlock (&ftrace_lock); +return 0 ; + +out_put: +if (pid != ftrace_swapper_pid) +put_pid (pid); + +out: +mutex_unlock (&ftrace_lock); +return ret; +} + +static void ftrace_pid_reset (void ) +{ +struct ftrace_pid *fpid, *safe; + +mutex_lock (&ftrace_lock); +list_for_each_entry_safe (fpid, safe, &ftrace_pids, list) { +struct pid *pid = fpid->pid; + +clear_ftrace_pid_task (&pid); + +if (pid != ftrace_swapper_pid) +put_pid (pid); + +list_del (&fpid->list); +kfree (fpid); + } + +ftrace_update_pid_func (); +ftrace_startup_enable (0 ); + +mutex_unlock (&ftrace_lock); +} + +static void *fpid_start (struct seq_file *m,loff_t *pos) +{ +mutex_lock (&ftrace_lock); + +if (list_empty (&ftrace_pids) && (!*pos)) +return (void * )1 ; + +return seq_list_start (&ftrace_pids, *pos); +} + +static void *fpid_next (struct seq_file *m,void *v,loff_t *pos) +{ +if (v == (void *)1 ) +return NULL ; + +return seq_list_next (v, &ftrace_pids, pos); +} + +static void fpid_stop (struct seq_file *m,void *p) +{ +mutex_unlock (&ftrace_lock); +} + +static int fpid_show (struct seq_file *m,void *v) +{ + const struct ftrace_pid *fpid = list_entry(v, struct ftrace_pid, list); + +if (v == (void *)1 ) { +seq_printf (m,"no pid\n" ); +return 0 ; + } + +if (fpid->pid == ftrace_swapper_pid) +seq_printf (m,"swapper tasks\n" ); +else +seq_printf (m,"%u\n" ,pid_vnr (fpid->pid)); + +return 0 ; +} + +static struct seq_operations ftrace_pid_sops = { + .start = fpid_start, + .next = fpid_next, + .stop = fpid_stop, + .show = fpid_show, +}; + +static int +ftrace_pid_open (struct inode *inode,struct file *file) +{ +int ret =0 ; + +if ((file->f_mode &FMODE_WRITE ) && + (file->f_flags &O_TRUNC )) +ftrace_pid_reset (); + +if (file->f_mode &FMODE_READ ) + ret =seq_open (file, &ftrace_pid_sops); + +return ret; +} +static ssize_t ftrace_pid_write (struct file *filp,const char __user *ubuf,size_t cnt,loff_t *ppos) { -struct pid *pid;char buf[64 ];long val;int ret;
@@ -2897,57 +3016,36 @@ ftrace_pid_write(struct file *filp, const char __user *ubuf, buf[cnt] =0 ; +/* +* Allow the "echo > set_ftrace_pid" to clean +* the pids quietly. +*/ +if (!*ppos && *buf =='\n' && cnt ==1 ) +return 1 ; + ret =strict_strtol (buf,10 , &val);if (ret <0 )return ret; -mutex_lock (&ftrace_lock); -if (val <0 ) { -/* disable pid tracing */ -if (!ftrace_pid_trace) - goto out; - -clear_ftrace_pid_task (&ftrace_pid_trace); - - }else { -/* swapper task is special */ -if (!val) { - pid = ftrace_swapper_pid; -if (pid == ftrace_pid_trace) - goto out; - }else { - pid =find_get_pid (val); - -if (pid == ftrace_pid_trace) { -put_pid (pid); - goto out; - } - } - -if (ftrace_pid_trace) -clear_ftrace_pid_task (&ftrace_pid_trace); - -if (!pid) - goto out; - - ftrace_pid_trace = pid; + ret =ftrace_pid_add (val); -set_ftrace_pid_task (ftrace_pid_trace); - } - -/* update the function call */ -ftrace_update_pid_func (); -ftrace_startup_enable (0 ); - - out: -mutex_unlock (&ftrace_lock); +return ret ? ret : cnt; +} -return cnt; +static int +ftrace_pid_release (struct inode *inode,struct file *file) +{ +if (file->f_mode &FMODE_READ ) +seq_release (inode, file); +return 0 ; }static const struct file_operations ftrace_pid_fops = { - .read = ftrace_pid_read, - .write = ftrace_pid_write, + .open = ftrace_pid_open, + .write = ftrace_pid_write, + .read = seq_read, + .llseek = seq_lseek, + .release = ftrace_pid_release, };static __init int ftrace_init_debugfs (void )
kernel/trace/trace.h +2 -2
@@ -530,12 +530,12 @@ print_graph_function(struct trace_iterator *iter) }#endif /* CONFIG_FUNCTION_GRAPH_TRACER */ -extern struct pid *ftrace_pid_trace; +extern struct list_head ftrace_pids;#ifdef CONFIG_FUNCTION_TRACER static inline int ftrace_trace_task (struct task_struct *task) { -if (!ftrace_pid_trace) +if (list_empty (&ftrace_pids))return 1 ;return test_tsk_trace_trace (task);
Signature · 1 line
1.6.2.5
Steven Rostedt
maintainerQuoted 5 lines · On Mon, 2009-09-14 at 12:14 +0200, [email protected] wrote:
I was debugging some ptrace applications and got to the idea of tracing signals - signal tracer. There's probably possibility to trace more info than just signal delivery info (like ignored signals deliveries, queue status...).
Quoted 7 lines
While doing this I realized it might be worth to have ability to trace more independent processes via
set_pid_ftracefile.Althought I'm sending this as RFC, attached patches work for me. I'm open to any suggestions/comments/ideas.
1/2 - signal trace
Don't make it a plugin. Just put the trace points into the signal handler, and then you enable them via the event tracer.
Quoted 1 line
2/2 - multiple pids support for
set_pid_ftracefile
I took a quick look at this patch and I so far like it. But this will not help you with events, but filtering will. That is:
# echo "common_pid == 1234" > /debug/tracing/events/signals/signal/format
You can do multiple pids with "common_pid == 1234 || common_pid == 1235"
-- Steve
Quoted 2 lines · On Mon, 2009-09-14 at 12:14 +0200, [email protected] wrote:
1/2 - signal trace
2/2 - multiple pids support forset_pid_ftracefile
The whole series several checkpatch errors. Could you fix those before you take the RFC off this?
Daniel
Quoted 10 lines · Daniel Walker
Quoted 2 lines · On Mon, 2009-09-14 at 12:14 +0200, [email protected] wrote:
1/2 - signal trace
2/2 - multiple pids support forset_pid_ftracefileThe whole series several checkpatch errors. Could you fix those before you take the RFC off this?
Daniel
yes, I know about them
I'll fix them for the final version
wbr,
jirka