convert ftrace syscalls to TRACE_EVENT
Subsystems (5)
cover convert ftrace syscalls to TRACE_EVENT
hi,
I brought this up for as RFC before, and people were agreed in principal to this idea. There are a number of advantages to having the syscall tracing implemented as an event:
-can toggle on/off individual syscalls
-as a non-dedicated tracer, it can intermix with other tracing data -by using tracepoints other infrastructures can tie into this implementation -ability to print flag names and not just bits
In terms of the implementation, I've introduced a new 'trace_syscalls' enabled flag in the <debugfs>/tracing/ directory to toggle the TIF_SYSCALL_FTRACE.
so: echo 1 > trace_syscalls to enable, echo 0 > trace_syscalls disable.
The implementation is still a bit rough, but I wanted to make sure I was on the right track before spending more time on this. I've showed an example of printing out a flag using a 'clock_id_toname()' to macro. Steven's print_flags() addition, should make pretty printing flag names much easier.
I believe that this current patch covers the functionality of the current ftrace syscall tracer minus priting variable names and types, which I could add....thus, I hope it can be merged in this state, and we can add pretty printing features over time?
thanks,
-Jason
Jason Baron (2):
-allow TP_printk() to have no args
-syscall tracepoints
arch/x86/kernel/ptrace.c | 8 +- include/asm-generic/syscall.h | 3 + include/trace/events/syscalls.h | 4202 +++++++++++++++++++++++++++++++++++++++ include/trace/ftrace.h | 2 +- include/trace/syscall.h | 6 + kernel/trace/Makefile | 1 - kernel/trace/trace.c | 101 + kernel/trace/trace_syscalls.c | 250 --- 8 files changed, 4318 insertions(+), 255 deletions(-) create mode 100644 include/trace/events/syscalls.h delete mode 100644 kernel/trace/trace_syscalls.c
PATCH 2/2 convert to syscall tracepoints
Implements syscall tracer via tracepoints and TRACE_EVENT(). Introduces a new tracing flag 'trace_syscalls', which must be toggled to enable this feature.
--- arch/x86/kernel/ptrace.c | 8 +- include/asm-generic/syscall.h | 3 + include/trace/events/syscalls.h | 4202 +++++++++++++++++++++++++++++++++++++++ include/trace/syscall.h | 6 + kernel/trace/Makefile | 1 - kernel/trace/trace.c | 101 + kernel/trace/trace_syscalls.c | 250 --- 7 files changed, 4317 insertions(+), 254 deletions(-) create mode 100644 include/trace/events/syscalls.h delete mode 100644 kernel/trace/trace_syscalls.c
arch/x86/kernel/ptrace.c +5 -3
@@ -35,7 +35,9 @@#include <asm/proto.h> #include <asm/ds.h> -#include <trace/syscall.h> +#include <linux/ftrace.h> +#define CREATE_TRACE_POINTS +#include <trace/events/syscalls.h> #include "tls.h"
@@ -1498,7 +1500,7 @@ asmregparm long syscall_trace_enter(struct pt_regs *regs) ret =-1L ;if (unlikely (test_thread_flag (TIF_SYSCALL_FTRACE ))) -ftrace_syscall_enter (regs); +syscall_tracepoints_enter (regs);if (unlikely (current->audit_context)) {if (IS_IA32 )
@@ -1524,7 +1526,7 @@ asmregparm void syscall_trace_leave(struct pt_regs *regs)audit_syscall_exit (AUDITSC_RESULT (regs->ax), regs->ax);if (unlikely (test_thread_flag (TIF_SYSCALL_FTRACE ))) -ftrace_syscall_exit (regs); +syscall_tracepoints_exit (regs);if (test_thread_flag (TIF_SYSCALL_TRACE ))tracehook_report_syscall_exit (regs,0 );
include/asm-generic/syscall.h +3 -0
@@ -22,6 +22,9 @@struct task_struct ;struct pt_regs ; + +struct syscall_metadata *syscall_nr_to_meta (int nr); +/** * syscall_get_nr - find what system call a task is executing * @task: task of interest, must be blocked
include/trace/events/syscalls.h new file +4202 -0
2,965 lines not on this page · raw message
kernel/trace/trace.c +101 -0
101 lines not on this page · raw message
kernel/trace/trace_syscalls.c deleted +0 -250
250 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.6.0.6
Part of the diff is not shown here. View the raw message.
PATCH 1/2 allow TP_printk() to have no args
In reply to Jason BaronThis patch allows TP_printk() to take no args. otherwise there is a stray ',' which causes a compile error.
--- include/trace/ftrace.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
include/trace/ftrace.h +1 -1
@@ -114,7 +114,7 @@#define __entry field#undef TP_printk -#define TP_printk(fmt, args...) fmt "\n", args +#define TP_printk (fmt, ...) fmt "\n", ##__VA_ARGS__#undef __get_str#define __get_str (field) ((char *)__entry + __entry->__str_loc_##field)
Signature · 1 line
1.6.0.6
Re: [PATCH 2/2] convert to syscall tracepoints
In reply to Jason BaronQuoted 8 lines · Jason Baron
Implements syscall tracer via tracepoints and
TRACE_EVENT(). Introduces a new tracing flag 'trace_syscalls', which must be toggled to enable this feature.Signed-off-by Jason Baron <[email protected]>
Quoted 11 lines
--- arch/x86/kernel/ptrace.c | 8 +- include/asm-generic/syscall.h | 3 + include/trace/events/syscalls.h | 4202 +++++++++++++++++++++++++++++++++++++++ include/trace/syscall.h | 6 + kernel/trace/Makefile | 1 - kernel/trace/trace.c | 101 + kernel/trace/trace_syscalls.c | 250 --- 7 files changed, 4317 insertions(+), 254 deletions(-) create mode 100644 include/trace/events/syscalls.h delete mode 100644 kernel/trace/trace_syscalls.c
the functionality is fine and very desired IMO, but the implementation is pretty large and ugly, isnt it?
Wouldnt it be possible to compress this:
include/trace/events/syscalls.h line 494
Quoted 9 lines
include/trace/events/syscalls.h new file +9
-0+#ifdef __NR_setgroups +trace_event_syscall_enter(2, setgroups, int, gidsetsize, gid_t __user *, grouplist); +trace_event_syscall_exit (setgroups); +#define ENTERCASEsetgroups create_syscall_enter(2, setgroups, int, gid_t __user *); +#define EXITCASEsetgroups create_syscall_exit(setgroups); +#else +#define ENTERCASEsetgroups +#define EXITCASEsetgroups +#endif
down to some sane syntax? Do we really want to replicate all the DEFINE_SYSCALL() information again in a separate header? Is there no way to somehow define the tracepoint in-situ, where the DEFINE_SYSCALL macro is present? That would eliminate include/trace/events/syscalls.h altogether.
Ingo
Re: [PATCH 2/2] convert to syscall tracepoints
In reply to Jason BaronQuoted 196 lines · Jason Baron
Implements syscall tracer via tracepoints and
TRACE_EVENT(). Introduces a new tracing flag 'trace_syscalls', which must be toggled to enable this feature.Signed-off-by Jason Baron <[email protected]>--- arch/x86/kernel/ptrace.c | 8 +- include/asm-generic/syscall.h | 3 + include/trace/events/syscalls.h | 4202 +++++++++++++++++++++++++++++++++++++++ include/trace/syscall.h | 6 + kernel/trace/Makefile | 1 - kernel/trace/trace.c | 101 + kernel/trace/trace_syscalls.c | 250 --- 7 files changed, 4317 insertions(+), 254 deletions(-) create mode 100644 include/trace/events/syscalls.h delete mode 100644 kernel/trace/trace_syscalls.carch/x86/kernel/ptrace.c +5
-3@@ -35,7 +35,9 @@#include <asm/proto.h> #include <asm/ds.h> -#include <trace/syscall.h> +#include <linux/ftrace.h> +#define CREATE_TRACE_POINTS +#include <trace/events/syscalls.h> #include "tls.h" @@ -1498,7 +1500,7 @@ asmregparm long syscall_trace_enter(struct pt_regs *regs) ret =-1L ;if (unlikely (test_thread_flag (TIF_SYSCALL_FTRACE ))) -ftrace_syscall_enter (regs); +syscall_tracepoints_enter (regs);if (unlikely (current->audit_context)) {if (IS_IA32 )@@ -1524,7 +1526,7 @@ asmregparm void syscall_trace_leave(struct pt_regs *regs)audit_syscall_exit (AUDITSC_RESULT (regs->ax), regs->ax);if (unlikely (test_thread_flag (TIF_SYSCALL_FTRACE ))) -ftrace_syscall_exit (regs); +syscall_tracepoints_exit (regs);if (test_thread_flag (TIF_SYSCALL_TRACE ))tracehook_report_syscall_exit (regs,0 );include/asm-generic/syscall.h +3
-0@@ -22,6 +22,9 @@struct task_struct ;struct pt_regs ; + +struct syscall_metadata *syscall_nr_to_meta (int nr); +/** * syscall_get_nr - find what system call a task is executing * @task: task of interest, must be blocked include/trace/events/syscalls.h new file +123
-0@@ -0,0 +1,4202 @@ +#if !defined(_TRACE_SYSCALLS_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _TRACE_SYSCALLS_H + +#include <asm/syscall.h> +#include <asm-generic/syscall.h> +#include <linux/tracepoint.h> +#include <trace/syscall.h> + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM syscalls + +struct epoll_event ; +struct iattr ; +struct inode ; +struct iocb ; +struct io_event ; +struct iovec ; +struct itimerspec ; +struct itimerval ; +struct kexec_segment ; +struct linux_dirent ; +struct linux_dirent64 ; +struct list_head ; +struct msgbuf ; +struct msghdr ; +struct msqid_ds ; +struct new_utsname ; +struct nfsctl_arg ; +struct __old_kernel_stat ; +struct pollfd ; +struct rlimit ; +struct rusage ; +struct sched_param ; +struct semaphore ; +struct sembuf ; +struct shmid_ds ; +struct sockaddr ; +struct stat ; +struct stat64 ; +struct statfs ; +struct statfs64 ; +struct __sysctl_args ; +struct sysinfo ; +struct timespec ; +struct timeval ; +struct timex ; +struct timezone ; +struct tms ; +struct utimbuf ; +struct mq_attr ; +struct compat_stat ; +struct compat_timeval ; +struct robust_list_head ; +struct getcpu_cache ; +struct old_linux_dirent ; +struct perf_counter_hw_event ; + +/* misc macros */ + +#define clock_id_toname (id) \ + (id == CLOCK_REALTIME ? "CLOCK_REALTIME" : \ + id == CLOCK_MONOTONIC ? "CLOCK_MONOTONIC" : \ + id == CLOCK_PROCESS_CPUTIME_ID ? "CLOCK_PROCESS_CPUTIME_ID" : \ + id == CLOCK_MONOTONIC_RAW ? "CLOCK_MONOTONIC_RAW" : \ + id == CLOCK_SGI_CYCLE ? "CLOCK_SGI_CYCLE" : \ + "UNKNOWN CLOCK") + +/* enter helper macros */ + +#define MAX_SYS_ARGS 6 + +#define expand_enter_sys_args_0 () +#define expand_enter_sys_args_1 (t1) (t1) sys_args[0] +#define expand_enter_sys_args_2 (t1, t2) expand_enter_sys_args_1(t1), (t2) sys_args[1] +#define expand_enter_sys_args_3 (t1, t2, t3) expand_enter_sys_args_2(t1, t2), (t3) sys_args[2] +#define expand_enter_sys_args_4 (t1, t2, t3, t4) expand_enter_sys_args_3(t1, t2, t3), (t4) sys_args[3] +#define expand_enter_sys_args_5 (t1, t2, t3, t4, t5) expand_enter_sys_args_4(t1, t2, t3, t4), (t5) sys_args[4] +#define expand_enter_sys_args_6 (t1, t2, t3, t4, t5, t6) expand_enter_sys_args_5(t1, t2, t3, t4, t5), (t6) sys_args[5] + +#define create_syscall_enter (n, sysname, ...) \ + case __NR_##sysname: \ + syscall_get_arguments(current, regs, 0, n, sys_args); \ + trace_sysenter_##sysname(expand_enter_sys_args_##n(__VA_ARGS__)); \ + break; + +#define expand_enter_proto_0 () void +#define expand_enter_proto_1 (t1, p1) t1 p1 +#define expand_enter_proto_2 (t2, p2, ...) t2 p2, expand_enter_proto_1(__VA_ARGS__) +#define expand_enter_proto_3 (t3, p3, ...) t3 p3, expand_enter_proto_2(__VA_ARGS__) +#define expand_enter_proto_4 (t4, p4, ...) t4 p4, expand_enter_proto_3(__VA_ARGS__) +#define expand_enter_proto_5 (t5, p5, ...) t5 p5, expand_enter_proto_4(__VA_ARGS__) +#define expand_enter_proto_6 (t6, p6, ...) t6 p6, expand_enter_proto_5(__VA_ARGS__) + +#define expand_enter_args_0 () +#define expand_enter_args_1 (t1, p1) p1 +#define expand_enter_args_2 (t2, p2, ...) p2, expand_enter_args_1(__VA_ARGS__) +#define expand_enter_args_3 (t3, p3, ...) p3, expand_enter_args_2(__VA_ARGS__) +#define expand_enter_args_4 (t4, p4, ...) p4, expand_enter_args_3(__VA_ARGS__) +#define expand_enter_args_5 (t5, p5, ...) p5, expand_enter_args_4(__VA_ARGS__) +#define expand_enter_args_6 (t6, p6, ...) p6, expand_enter_args_5(__VA_ARGS__) + +#define expand_enter_entry_0 () +#define expand_enter_entry_1 (t1, p1) __field(t1, p1) +#define expand_enter_entry_2 (t2, p2, ...) __field(t2, p2) expand_enter_entry_1(__VA_ARGS__) +#define expand_enter_entry_3 (t3, p3, ...) __field(t3, p3) expand_enter_entry_2(__VA_ARGS__) +#define expand_enter_entry_4 (t4, p4, ...) __field(t4, p4) expand_enter_entry_3(__VA_ARGS__) +#define expand_enter_entry_5 (t5, p5, ...) __field(t5, p5) expand_enter_entry_4(__VA_ARGS__) +#define expand_enter_entry_6 (t6, p6, ...) __field(t6, p6) expand_enter_entry_5(__VA_ARGS__) + +#define expand_enter_assign_0 () +#define expand_enter_assign_1 (t1, p1) __entry->p1 = p1; +#define expand_enter_assign_2 (t2, p2, ...) __entry->p2 = p2; expand_enter_assign_1(__VA_ARGS__) +#define expand_enter_assign_3 (t3, p3, ...) __entry->p3 = p3; expand_enter_assign_2(__VA_ARGS__) +#define expand_enter_assign_4 (t4, p4, ...) __entry->p4 = p4; expand_enter_assign_3(__VA_ARGS__) +#define expand_enter_assign_5 (t5, p5, ...) __entry->p5 = p5; expand_enter_assign_4(__VA_ARGS__) +#define expand_enter_assign_6 (t6, p6, ...) __entry->p6 = p6; expand_enter_assign_5(__VA_ARGS__) + +#define expand_enter_printk_1 (t1, p1) (u64)__entry->p1 +#define expand_enter_printk_2 (t2, p2, ...) (u64)__entry->p2, expand_enter_printk_1(__VA_ARGS__) +#define expand_enter_printk_3 (t3, p3, ...) (u64)__entry->p3, expand_enter_printk_2(__VA_ARGS__) +#define expand_enter_printk_4 (t4, p4, ...) (u64)__entry->p4, expand_enter_printk_3(__VA_ARGS__) +#define expand_enter_printk_5 (t5, p5, ...) (u64)__entry->p5, expand_enter_printk_4(__VA_ARGS__) +#define expand_enter_printk_6 (t6, p6, ...) (u64)__entry->p6, expand_enter_printk_5(__VA_ARGS__)
include/trace/events/syscalls.h line 133
Quoted 10 lines
include/trace/events/syscalls.h new file +10
-0+ +#define TP_printk_0 () TP_printk() +#define TP_printk_1 (...) TP_printk("%016Lx", expand_enter_printk_1(__VA_ARGS__)) +#define TP_printk_2 (...) TP_printk("%016Lx %016Lx", expand_enter_printk_2(__VA_ARGS__)) +#define TP_printk_3 (...) TP_printk("%016Lx %016Lx %016Lx", expand_enter_printk_3(__VA_ARGS__)) +#define TP_printk_4 (...) TP_printk("%016Lx %016Lx %016Lx %016Lx", expand_enter_printk_4(__VA_ARGS__)) +#define TP_printk_5 (...) TP_printk("%016Lx %016Lx %016Lx %016Lx %016Lx", \ + expand_enter_printk_5(__VA_ARGS__)) +#define TP_printk_6 (...) TP_printk("%016Lx %016Lx %016Lx %016Lx %016Lx %016Lx", \ + expand_enter_printk_6(__VA_ARGS__))
Hmm, may be just use %p so that it will adapt to the arch len. Anyway we'll need to custom the syscall args printing, once we have these tracepoints.
Quoted 19 lines
include/trace/events/syscalls.h new file +19
-0+ +#define trace_event_syscall_enter (n, name, ...) \ + TRACE_EVENT(sysenter_##name, \ + TP_PROTO(expand_enter_proto_##n(__VA_ARGS__)), \ + TP_ARGS(expand_enter_args_##n(__VA_ARGS__)), \ + TP_STRUCT__entry(expand_enter_entry_##n(__VA_ARGS__)), \ + TP_fast_assign(expand_enter_assign_##n(__VA_ARGS__)), \ + TP_printk_##n(__VA_ARGS__) \ + ); + +/* exit helper macros */ + +#define create_syscall_exit (sysname) \ + case __NR_##sysname: \ + trace_sysexit_##sysname(ret); \ + break; \ + +#define trace_event_syscall_exit (name) \ + TRACE_EVENT(sysexit_##name, \
include/trace/events/syscalls.h line 162
Quoted 10 lines
include/trace/events/syscalls.h new file +10
-0+ TP_PROTO(long ret), \ +TP_ARGS (ret), \ +TP_STRUCT__entry ( \ + __field(long ,retval ) \ + ), \ +TP_fast_assign ( \ + __entry->retval = ret; \ + ), \ +TP_printk ("return value: %ld" , __entry->retval) \ + );
Until there it looks good, these helpers can be applied in SYSCALL_DEFINE(), but I really think the manually written per syscall tracepoints definition is not a good idea.
What you did above may be fine to be integrated inside SYSCALL_DEFINEx() so that we can benefit from the magic of defining each syscall tracepoints in a single generic code.
It will probably require some tuning such as setting the TIF_FTRACE flags from the reg() callback in TRACE_EVENT.
And probably some other things.
Thanks.
Frederic.
Quoted 19 lines
include/trace/events/syscalls.h new file +19
-0+#ifdef __NR_time +trace_event_syscall_enter(1, time, time_t __user *, tloc); +trace_event_syscall_exit (time); +#define ENTERCASEtime create_syscall_enter(1, time, time_t __user *); +#define EXITCASEtime create_syscall_exit(time); +#else +#define ENTERCASEtime +#define EXITCASEtime +#endif + +#ifdef __NR_stime +trace_event_syscall_enter(1, stime, time_t __user *, tptr); +trace_event_syscall_exit (stime); +#define ENTERCASEstime create_syscall_enter(1, stime, time_t __user *); +#define EXITCASEstime create_syscall_exit(stime); +#else +#define ENTERCASEstime +#define EXITCASEstime +#endif
Re: [PATCH 2/2] convert to syscall tracepoints
In reply to Ingo MolnarQuoted 21 lines · Ingo Molnar
the functionality is fine and very desired IMO, but the implementation is pretty large and ugly, isnt it?
Wouldnt it be possible to compress this:
Review
Quoted 9 lines
(quoted diff) +9
-0+#ifdef __NR_setgroups +trace_event_syscall_enter(2, setgroups, int, gidsetsize, gid_t __user *, grouplist); +trace_event_syscall_exit(setgroups); +#define ENTERCASEsetgroups create_syscall_enter(2, setgroups, int, gid_t __user *); +#define EXITCASEsetgroups create_syscall_exit(setgroups); +#else +#define ENTERCASEsetgroups +#define EXITCASEsetgroups +#endifdown to some sane syntax? Do we really want to replicate all the
DEFINE_SYSCALL()information again in a separate header? Is there no way to somehow define the tracepoint in-situ, where theDEFINE_SYSCALLmacro is present? That would eliminateinclude/trace/events/syscalls.haltogether.
Quoted 1 line
Ingo
ok, I've re-worked the implmentation a bit and I've gotten each syscall down to:
#ifdef __NR_setgroupstrace_event_syscall(2, setgroups, int, gidsetsize, gid_t __user *, grouplist);
#endif
However, I'm still creating a separate syscalls.h trace file. I think that doing this via DEFINE_SYSCALL() is going to be difficult. We can easily define the tracepoints via DEFINE_SYSCALL(), but then we are going to need to reference these tracepoints somehow from ptrace.c. I don't see how to do this easily. Further, over time we hope to replace most of the default tracepoints, with customized tracepoints that are more human readable. We need to store these somewhere. In the new implementation I can do this as follows:
#ifdef __NR_clock_settime
#ifdef SYSCALL_OVERRIDE
TRACE_EVENT(sysenter_clock_settime,
TP_PROTO(clockid_t which_clock, struct timespec __user *tp),
TP_ARGS(which_clock, tp),
TP_STRUCT__entry(
__field(clockid_t, which_clock)
__field(struct timespec __user *, tp)
),
TP_fast_assign(
__entry->which_clock = which_clock;
__entry->tp = tp;
),
TP_printk("%s %016Lx",clock_id_toname((u64)__entry->which_clock), (u64)__entry->tp) );trace_event_syscall_exit(clock_settime);
#elsetrace_event_syscall(2, clock_settime, clockid_t, which_clock, struct timespec __user *, tp);
#endif
#endif
The new 2/2 patch is below.
thanks,
-Jason
--- arch/x86/kernel/ptrace.c | 8 +- include/asm-generic/syscall.h | 3 + include/trace/events/syscalls-core.h | 1341 ++++++++++++++++++++++++++++++++++ include/trace/events/syscalls.h | 158 ++++ include/trace/syscall.h | 64 ++ kernel/trace/Makefile | 1 - kernel/trace/trace.c | 100 +++ kernel/trace/trace_syscalls.c | 250 ------- 8 files changed, 1672 insertions(+), 253 deletions(-) create mode 100644 include/trace/events/syscalls-core.h create mode 100644 include/trace/events/syscalls.h delete mode 100644 kernel/trace/trace_syscalls.c
arch/x86/kernel/ptrace.c +6 -2
@@ -35,6 +35,10 @@#include <asm/proto.h> #include <asm/ds.h> +#include <linux/ftrace.h> +#define CREATE_TRACE_POINTS +#include <trace/events/syscalls.h> +#undef CREATE_TRACE_POINTS#include <trace/syscall.h> #include "tls.h"
@@ -1498,7 +1502,7 @@ asmregparm long syscall_trace_enter(struct pt_regs *regs) ret =-1L ;if (unlikely (test_thread_flag (TIF_SYSCALL_FTRACE ))) -ftrace_syscall_enter (regs); +syscall_tracepoints_enter (regs);if (unlikely (current->audit_context)) {if (IS_IA32 )
@@ -1524,7 +1528,7 @@ asmregparm void syscall_trace_leave(struct pt_regs *regs)audit_syscall_exit (AUDITSC_RESULT (regs->ax), regs->ax);if (unlikely (test_thread_flag (TIF_SYSCALL_FTRACE ))) -ftrace_syscall_exit (regs); +syscall_tracepoints_exit (regs);if (test_thread_flag (TIF_SYSCALL_TRACE ))tracehook_report_syscall_exit (regs,0 );
include/asm-generic/syscall.h +3 -0
@@ -22,6 +22,9 @@struct task_struct ;struct pt_regs ; + +struct syscall_metadata *syscall_nr_to_meta (int nr); +/** * syscall_get_nr - find what system call a task is executing * @task: task of interest, must be blocked
include/trace/events/syscalls-core.h new file +1341 -0
1,341 lines not on this page · raw message
include/trace/events/syscalls.h new file +158 -0
158 lines not on this page · raw message
include/trace/syscall.h +64 -0
81 lines not on this page · raw message
kernel/trace/Makefile +0 -1
@@ -48,7 +48,6 @@ obj-$(CONFIG_WORKQUEUE_TRACER) += trace_workqueue.o obj-$(CONFIG_BLK_DEV_IO_TRACE) += blktrace.o obj-$(CONFIG_EVENT_TRACING) += trace_events.o obj-$(CONFIG_EVENT_TRACING) += trace_export.o -obj-$(CONFIG_FTRACE_SYSCALLS) += trace_syscalls.o obj-$(CONFIG_EVENT_PROFILE) += trace_event_profile.o obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o obj-$(CONFIG_EVENT_TRACING) += trace_mm.o
kernel/trace/trace.c +100 -0
118 lines not on this page · raw message
kernel/trace/trace_syscalls.c deleted +0 -250
250 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 2/2] convert to syscall tracepoints
include/trace/events/syscalls-core.h line 11
Quoted 11 lines · Jason Baron
include/trace/events/syscalls-core.h new file +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endif
This could be reduced to a single line: just add a Kconfig entry (say TRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...
As more architectures turn on SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate DEFINE_SYSCALL macros. Why arent those macros re-used?
We dont need too smart pretty-printing i think - we only want to know the field size and the field name - nothing else. Duplicating all those definitions looks outright wrong to me. Do we really, really, really have to do it?
Ingo
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 19 lines · Ingo Molnar
include/trace/events/syscalls-core.h line 11
Quoted 11 lines · Jason Baron
include/trace/events/syscalls-core.h new file +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endif This could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.
true, but this implementation works for all arches now, why would want to slowly add this over time? I think its unnecessary work that could be error prone.
Quoted 4 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?
The DEFINE_SYSCALL() are located all over the code in various .c files. Thus, if we define the tracpoints via the DEFINE_SYSCALL() macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' from ptrace.c. How do I do that? By defining these as I've done in syscalls.h I can include that header file and create an efficient 'case' statement. I've verified in the assembly that the 'case' statments resolve to jumps. Thus, this implementation is pretty efficient. Further, by creating a header file other users of the tracepoints can include that header. If we use DEFINE_SYSCALL() that is not possible. I'd be more than happy to use the DEFINE_SYSCALL() macros, if you could explain how to resolve the above issues...We could write a pre-processor that creates a dynamic header file from the DEFINE_SYSCALL() points? I'm not sure that's simpler...
Quoted 5 lines
We dont need too smart pretty-printing i think - we only want to know the field size and the field name - nothing else. Duplicating all those definitions looks outright wrong to me. Do we really, really, really have to do it?
In terms of pretty-printing, it should be fairly easy to add size and the field name to what I have without bloating the syscalls.h file at all.
thanks,
-Jason
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 20 lines · Jason Baron
Quoted 19 lines · Ingo Molnar
include/trace/events/syscalls-core.h line 11
Quoted 11 lines · Jason Baron
include/trace/events/syscalls-core.h new file +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endif This could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...
Quoted 8 lines
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.true, but this implementation works for all arches now, why would want to slowly add this over time? [...]
Because the current solution is butt-ugly ...
Quoted 1 line
[...] I think its unnecessary work that could be error prone.
This area needs cleanups - making it messier doesnt help. (I've Cc:-ed hpa - he has expressed interest in auto-generating all the syscall related details from another angle ...)
Quoted 5 lines
Quoted 3 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?The
DEFINE_SYSCALL()are located all over the code in various .c files.
yes, and that's good.
Quoted 5 lines
Thus, if we define the tracpoints via the
DEFINE_SYSCALL()macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' fromptrace.c. How do I do that? [...]
And that's bad.
We dont want a per syscall tracepoint call site. AT ALL.
We want to collect the record information, we want to construct /debug/tracing/events/syscalls/ directories with all the proper tracepoint-lookalike entries, and then we want to use the _existing_, _zero overhead_ method implemented by Frederic to get per syscall functionality.
Have you looked at how the syscall attributes information is constructed by using .section tricks? See:kernel/trace/trace_syscalls.c.
Ingo
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 53 lines · Ingo Molnar
Quoted 20 lines · Ingo Molnar
Quoted 19 lines · Jason Baron
include/trace/events/syscalls-core.h line 11
Quoted 12 lines
include/trace/events/syscalls-core.h new file +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endif This could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...Quoted 8 lines
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.true, but this implementation works for all arches now, why would want to slowly add this over time? [...]
Because the current solution is butt-ugly ...
Quoted 1 line
[...] I think its unnecessary work that could be error prone.
This area needs cleanups - making it messier doesnt help. (I've Cc:-ed hpa - he has expressed interest in auto-generating all the syscall related details from another angle ...)
Quoted 5 lines
Quoted 3 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?The
DEFINE_SYSCALL()are located all over the code in various .c files.yes, and that's good.
Quoted 5 lines
Thus, if we define the tracpoints via the
DEFINE_SYSCALL()macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' fromptrace.c. How do I do that? [...]And that's bad.
Quoted 8 lines
We dont want a per syscall tracepoint call site. AT ALL.
We want to collect the record information, we want to construct /debug/tracing/events/syscalls/ directories with all the proper tracepoint-lookalike entries, and then we want to use the _existing_,
_zerooverhead_method implemented by Frederic to get per syscall functionality.
Yes, this can easily be done....but that wasn't the problem I was interested in solving. I wanted a per syscall tracepoint site. I thought I had been making that clear all along...Please notice that the implementation I've proposed obtains the syscall number, and then jumps to the appropriate tracepoint and then exits. Its quite efficient. In fact, I've enabled all of the syscalls using my proposed method and running tbench I'm able to get more throughput then using the current syscall method. I've also done 'getpid()' loops and seen no performance difference between the approaches. I'm happy to run any other benchmarks...
Quoted 4 lines
Have you looked at how the syscall attributes information is constructed by using .section tricks? See:
kernel/trace/trace_syscalls.c.
yes, I believe I understand the problem space. I had been talking about a per-syscall tracepoint all along...maybe I wasn't clear...
thanks,
-Jason
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 3 lines · Jason Baron
Yes, this can easily be done....but that wasn't the problem I was interested in solving. I wanted a per syscall tracepoint site. I thought I had been making that clear all along...
We can do that, but _not in the syscall_.
There's absolutely nothing wrong with having an array of tracepoints/callbacks that is being called by the TIF_FTRACE code. We already have per syscall attributes - that can include a callback too.
Quoted 7 lines
[...] Please notice that the implementation I've proposed obtains the syscall number, and then jumps to the appropriate tracepoint and then exits. Its quite efficient. In fact, I've enabled all of the syscalls using my proposed method and running tbench I'm able to get more throughput then using the current syscall method. I've also done '
getpid()' loops and seen no performance difference between the approaches. I'm happy to run any other benchmarks...
If you cannot see a performance difference in getpid() loop that's possibly because glibc caches getpid() results ...
Try getppid().
Anyway - bloating every syscall with a tracepoint check is out of question and unnecessary. We already have the TIF_ flag based machinery to call back on syscalls - and what we need is the proper type/field info extracted and /debug/tracing/syscalls/* entries created.
Ingo
Re: [PATCH 2/2] convert to syscall tracepoints
In reply to Jason BaronQuoted 79 lines · Jason Baron
Quoted 53 lines · Ingo Molnar
Quoted 20 lines · Ingo Molnar
Quoted 19 lines · Jason Baron
include/trace/events/syscalls-core.h line 11
Quoted 12 lines
include/trace/events/syscalls-core.h new file +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endif This could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...Quoted 8 lines
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.true, but this implementation works for all arches now, why would want to slowly add this over time? [...]
Because the current solution is butt-ugly ...
Quoted 1 line
[...] I think its unnecessary work that could be error prone.
This area needs cleanups - making it messier doesnt help. (I've Cc:-ed hpa - he has expressed interest in auto-generating all the syscall related details from another angle ...)
Quoted 5 lines
Quoted 3 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?The
DEFINE_SYSCALL()are located all over the code in various .c files.yes, and that's good.
Quoted 5 lines
Thus, if we define the tracpoints via the
DEFINE_SYSCALL()macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' fromptrace.c. How do I do that? [...]And that's bad.
Quoted 8 lines
We dont want a per syscall tracepoint call site. AT ALL.
We want to collect the record information, we want to construct /debug/tracing/events/syscalls/ directories with all the proper tracepoint-lookalike entries, and then we want to use the _existing_,
_zerooverhead_method implemented by Frederic to get per syscall functionality.Yes, this can easily be done....but that wasn't the problem I was interested in solving. I wanted a per syscall tracepoint site. I thought I had been making that clear all along...Please notice that the implementation I've proposed obtains the syscall number, and then jumps to the appropriate tracepoint and then exits. Its quite efficient. In fact, I've enabled all of the syscalls using my proposed method and running tbench I'm able to get more throughput then using the current syscall method. I've also done '
getpid()' loops and seen no performance difference between the approaches. I'm happy to run any other benchmarks...Quoted 4 lines
Have you looked at how the syscall attributes information is constructed by using .section tricks? See:
kernel/trace/trace_syscalls.c.
Quoted 6 lines
yes, I believe I understand the problem space. I had been talking about a per-syscall tracepoint all along...maybe I wasn't clear...
thanks,
-Jason
Ok, I understand the problem.
Well, the fact is that we can use the artifact from the current syscall tracer to solve a part of this problem.
Currently the syscall tracer does the following:
- create a section with all syscalls informations, provided by DEFINE_SYSCALL() That includes the size, type, name of parameters.
- map a table during the boot which resolves a syscall number to its information in the syscall metadata section
- uses a generic "trace_syscall()" (or something like that) in ptrace.c (x86) which gather informations from the current syscalls (get from the mapped table) and then send the trace to the ring buffer.
- have a pretty printing (well, not that much actually) callback which, again, retrieve the syscall information from its number after getting the trace from the ring buffer. And then the raw field values aree printed, with the field names, and their types, optionally.
Now what I would suggest to avoid this whole listing of syscalls in your patch is to avoid the use of hardcoded tracepoints.
We can't really use TRACE_EVENT() here without using the listing you did. Instead, you could define a custom struct ftrace_event_call from DEFINE_SYSCALL().
In regfunc() you can turn on TIF_FTRACE (using a refcounter).
The struct trace_event ftrace_event_type can reuse the existing output callback for syscall tracing which retrieve the syscall informations.
void ftrace_raw_event_##call() can be replaced by calling directly the existing generic callback for syscall tracing trace insertion.
And the arch mapping table can resolve a syscall number to its matching event.
The rest of the challenge is the pretty printing because of complex types such as structures pointers, strings, etc...
We can think about it later, the above is more important.
But some ideas if we do that later: through a quick access table that mapps the types and gives us informations about them (should they be dereferenced, how, which size, etc....). Again, this one can be generic and would factorize some work.
Imagine that we have:
void syscall_foo(struct bar *b);
void syscall_foo2(struct bar *b, int blah);
If we hash each complex types such as struct bar* somewhere with the appropriate callback and/or direct informations to handle them, then we can build a hash of this type in the syscall metadata which would give us an index to retrieve the appropriate callback to handle it, or just the size of this field to know how much we hav to copy in the ring buffer.
Better even to avoid more than one level to derefenc in the tracing fast path, we can build the right things on boot, having a bit
for each parameter which tells whether the field has to be dereferenced and which size in this case. We would just need to put these informations for each complex types somewhere and glue them dynamically on each syscall metadata that need it during the boot.
Example:
struct syscall_param_meta {
.name = "struct bar __user *",
.deref = 1,
.size = sizeof(struct bar)
.print = bar_printer
};Put it in a param_meta hash table:
[ hash_to_struct_bar_meta, hash_to_other_complex_type, etc...]
Now we can add some fields in the syscall metadata, such as
.param_name = "struct bar __user*" (added on DEFINE_SYSCALL()) .deref = 0 (filled on boot)
.size = 0 (filled on boot, or by default sizeof(struct bar __user*))
Then on boot, we iterate through the syscalls metadata, hash each types, find the matching syscall param meta, and copy the relevant informations for the fast path (tracing) which are deref and size.
And the slow path (printing) we can retrieve more informations such as the callback to handle this type.
The interesting thing is that we define how to handle this type once and it's propagated for each syscalls that use it.
Hmm?
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 117 lines · Frederic Weisbecker
Quoted 79 lines · Jason Baron
Quoted 53 lines · Ingo Molnar
Quoted 20 lines · Ingo Molnar
Quoted 19 lines · Jason Baron
include/trace/events/syscalls-core.h line 11
Quoted 12 lines
include/trace/events/syscalls-core.h new file +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endif This could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...Quoted 8 lines
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.true, but this implementation works for all arches now, why would want to slowly add this over time? [...]
Because the current solution is butt-ugly ...
Quoted 1 line
[...] I think its unnecessary work that could be error prone.
This area needs cleanups - making it messier doesnt help. (I've Cc:-ed hpa - he has expressed interest in auto-generating all the syscall related details from another angle ...)
Quoted 5 lines
Quoted 3 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?The
DEFINE_SYSCALL()are located all over the code in various .c files.yes, and that's good.
Quoted 5 lines
Thus, if we define the tracpoints via the
DEFINE_SYSCALL()macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' fromptrace.c. How do I do that? [...]And that's bad.
Quoted 8 lines
We dont want a per syscall tracepoint call site. AT ALL.
We want to collect the record information, we want to construct /debug/tracing/events/syscalls/ directories with all the proper tracepoint-lookalike entries, and then we want to use the _existing_,
_zerooverhead_method implemented by Frederic to get per syscall functionality.Yes, this can easily be done....but that wasn't the problem I was interested in solving. I wanted a per syscall tracepoint site. I thought I had been making that clear all along...Please notice that the implementation I've proposed obtains the syscall number, and then jumps to the appropriate tracepoint and then exits. Its quite efficient. In fact, I've enabled all of the syscalls using my proposed method and running tbench I'm able to get more throughput then using the current syscall method. I've also done '
getpid()' loops and seen no performance difference between the approaches. I'm happy to run any other benchmarks...Quoted 4 lines
Have you looked at how the syscall attributes information is constructed by using .section tricks? See:
kernel/trace/trace_syscalls.c.Quoted 6 lines
yes, I believe I understand the problem space. I had been talking about a per-syscall tracepoint all along...maybe I wasn't clear...
thanks,
-Jason
Ok, I understand the problem.
Well, the fact is that we can use the artifact from the current syscall tracer to solve a part of this problem.Currently the syscall tracer does the following:
- create a section with all syscalls informations, provided by
DEFINE_SYSCALL()That includes the size, type, name of parameters.- map a table during the boot which resolves a syscall number to its information in the syscall metadata section
- uses a generic "
trace_syscall()" (or something like that) inptrace.c(x86) which gather informations from the current syscalls (get from the mapped table) and then send the trace to the ring buffer.- have a pretty printing (well, not that much actually) callback which, again, retrieve the syscall information from its number after getting the trace from the ring buffer. And then the raw field values aree printed, with the field names, and their types, optionally.
Now what I would suggest to avoid this whole listing of syscalls in your patch is to avoid the use of hardcoded tracepoints.
We can't really use
TRACE_EVENT()here without using the listing you did. Instead, you could define a custom structftrace_event_callfromDEFINE_SYSCALL().In
regfunc()you can turn onTIF_FTRACE(using a refcounter).
Quoted 9 lines
The struct
trace_eventftrace_event_typecan reuse the existing output callback for syscall tracing which retrieve the syscall informations.void ftrace_raw_event_##call() can be replaced by calling directly the existing generic callback for syscall tracing trace insertion.
And the arch mapping table can resolve a syscall number to its matching event.
hmmm..so I presume this would layer on 2 tracepoints? One in syscall entry and one in exit, presumably passing a 'struct pt_regs'? I think the refcounter would also have to be deeper in the tracepoint infrastructure since the event tracing wouldn't be the only potential user of these tracepoints.
thanks,
-Jason
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 124 lines · Jason Baron
Quoted 123 lines · Frederic Weisbecker
Quoted 79 lines · Jason Baron
Quoted 53 lines · Ingo Molnar
Quoted 20 lines · Ingo Molnar
Quoted 19 lines · Jason Baron
Review
Quoted 12 lines
(quoted diff) +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endifThis could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...Quoted 8 lines
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.true, but this implementation works for all arches now, why would want to slowly add this over time? [...]
Because the current solution is butt-ugly ...
Quoted 1 line
[...] I think its unnecessary work that could be error prone.
This area needs cleanups - making it messier doesnt help. (I've Cc:-ed hpa - he has expressed interest in auto-generating all the syscall related details from another angle ...)
Quoted 5 lines
Quoted 3 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?The
DEFINE_SYSCALL()are located all over the code in various .c files.yes, and that's good.
Quoted 5 lines
Thus, if we define the tracpoints via the
DEFINE_SYSCALL()macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' fromptrace.c. How do I do that? [...]And that's bad.
Quoted 8 lines
We dont want a per syscall tracepoint call site. AT ALL.
We want to collect the record information, we want to construct /debug/tracing/events/syscalls/ directories with all the proper tracepoint-lookalike entries, and then we want to use the _existing_,
_zerooverhead_method implemented by Frederic to get per syscall functionality.Yes, this can easily be done....but that wasn't the problem I was interested in solving. I wanted a per syscall tracepoint site. I thought I had been making that clear all along...Please notice that the implementation I've proposed obtains the syscall number, and then jumps to the appropriate tracepoint and then exits. Its quite efficient. In fact, I've enabled all of the syscalls using my proposed method and running tbench I'm able to get more throughput then using the current syscall method. I've also done '
getpid()' loops and seen no performance difference between the approaches. I'm happy to run any other benchmarks...Quoted 4 lines
Have you looked at how the syscall attributes information is constructed by using .section tricks? See:
kernel/trace/trace_syscalls.c.Quoted 6 lines
yes, I believe I understand the problem space. I had been talking about a per-syscall tracepoint all along...maybe I wasn't clear...
thanks,
-Jason
Ok, I understand the problem.
Well, the fact is that we can use the artifact from the current syscall tracer to solve a part of this problem.Currently the syscall tracer does the following:
- create a section with all syscalls informations, provided by
DEFINE_SYSCALL()That includes the size, type, name of parameters.- map a table during the boot which resolves a syscall number to its information in the syscall metadata section
- uses a generic "
trace_syscall()" (or something like that) inptrace.c(x86) which gather informations from the current syscalls (get from the mapped table) and then send the trace to the ring buffer.- have a pretty printing (well, not that much actually) callback which, again, retrieve the syscall information from its number after getting the trace from the ring buffer. And then the raw field values aree printed, with the field names, and their types, optionally.
Now what I would suggest to avoid this whole listing of syscalls in your patch is to avoid the use of hardcoded tracepoints.
We can't really use
TRACE_EVENT()here without using the listing you did. Instead, you could define a custom structftrace_event_callfromDEFINE_SYSCALL().In
regfunc()you can turn onTIF_FTRACE(using a refcounter).The struct
trace_eventftrace_event_typecan reuse the existing output callback for syscall tracing which retrieve the syscall informations.void ftrace_raw_event_##call() can be replaced by calling directly the existing generic callback for syscall tracing trace insertion.
Quoted 6 lines
Quoted 3 lines
And the arch mapping table can resolve a syscall number to its matching event.
hmmm..so I presume this would layer on 2 tracepoints? One in syscall entry and one in exit, presumably passing a 'struct
pt_regs'?
Exactly, instead of having two tracepoints per syscalls, we would have only two generic and smart enough to handle all syscalls thanks to the syscalls metadata.
Quoted 4 lines
I think
the refcounter would also have to be deeper in the tracepoint infrastructure since the event tracing wouldn't be the only potential user of these tracepoints.
Yeah. Ie we would have two layers of syscalls tracing (de)activation:
the TIF flags to enable the whole syscall tracing, and also a state bit in syscalls metadata. So that we can activate/deactivate each of them independently, like any "normal" tracepoint.
Frederic.
Quoted 4 lines
thanks,
-Jason
Re: [PATCH 2/2] convert to syscall tracepoints
Quoted 147 lines · Frederic Weisbecker
Quoted 124 lines · Jason Baron
Quoted 123 lines · Frederic Weisbecker
Quoted 79 lines · Jason Baron
Quoted 53 lines · Ingo Molnar
Quoted 20 lines · Ingo Molnar
Quoted 19 lines · Jason Baron
Review
Quoted 12 lines
(quoted diff) +11
-0+#ifdef __NR_time +trace_event_syscall(1, time, time_t __user *, tloc); +#endif + +#ifdef __NR_stime +trace_event_syscall(1, stime, time_t __user *, tptr); +#endif + +#ifdef __NR_gettimeofday +trace_event_syscall(2, gettimeofday, struct timeval __user *, tv, struct timezone __user *, tz); +#endifThis could be reduced to a single line: just add a
Kconfigentry (sayTRACE_SYSCALL_TRACEPOINTS) wether an arch supports syscall tracepoints, enable it on a sane arch, make sure it has all the syscalls and list them ...Quoted 8 lines
Quoted 5 lines
As more architectures turn on
SYSCALL_TRACEPOINTS, they'll have to resolve any deviations in syscall entry points. Ideally we'd have one generic table that covers 95% of all syscalls, and the remaining 5% in some architecture specific #ifdef section.true, but this implementation works for all arches now, why would want to slowly add this over time? [...]
Because the current solution is butt-ugly ...
Quoted 1 line
[...] I think its unnecessary work that could be error prone.
This area needs cleanups - making it messier doesnt help. (I've Cc:-ed hpa - he has expressed interest in auto-generating all the syscall related details from another angle ...)
Quoted 5 lines
Quoted 3 lines
But, more generally, i'm not at all convinced that we need _any_ of this enumeration. Look how much the above lines duplicate
DEFINE_SYSCALLmacros. Why arent those macros re-used?The
DEFINE_SYSCALL()are located all over the code in various .c files.yes, and that's good.
Quoted 5 lines
Thus, if we define the tracpoints via the
DEFINE_SYSCALL()macros we are going to have 'static inline functions' (which is how tracepoints are implemented) defined in all these .c files. Now, I need to call all these 'static inline functions' fromptrace.c. How do I do that? [...]And that's bad.
Quoted 8 lines
We dont want a per syscall tracepoint call site. AT ALL.
We want to collect the record information, we want to construct /debug/tracing/events/syscalls/ directories with all the proper tracepoint-lookalike entries, and then we want to use the _existing_,
_zerooverhead_method implemented by Frederic to get per syscall functionality.Yes, this can easily be done....but that wasn't the problem I was interested in solving. I wanted a per syscall tracepoint site. I thought I had been making that clear all along...Please notice that the implementation I've proposed obtains the syscall number, and then jumps to the appropriate tracepoint and then exits. Its quite efficient. In fact, I've enabled all of the syscalls using my proposed method and running tbench I'm able to get more throughput then using the current syscall method. I've also done '
getpid()' loops and seen no performance difference between the approaches. I'm happy to run any other benchmarks...Quoted 4 lines
Have you looked at how the syscall attributes information is constructed by using .section tricks? See:
kernel/trace/trace_syscalls.c.Quoted 6 lines
yes, I believe I understand the problem space. I had been talking about a per-syscall tracepoint all along...maybe I wasn't clear...
thanks,
-Jason
Ok, I understand the problem.
Well, the fact is that we can use the artifact from the current syscall tracer to solve a part of this problem.Currently the syscall tracer does the following:
- create a section with all syscalls informations, provided by
DEFINE_SYSCALL()That includes the size, type, name of parameters.- map a table during the boot which resolves a syscall number to its information in the syscall metadata section
- uses a generic "
trace_syscall()" (or something like that) inptrace.c(x86) which gather informations from the current syscalls (get from the mapped table) and then send the trace to the ring buffer.- have a pretty printing (well, not that much actually) callback which, again, retrieve the syscall information from its number after getting the trace from the ring buffer. And then the raw field values aree printed, with the field names, and their types, optionally.
Now what I would suggest to avoid this whole listing of syscalls in your patch is to avoid the use of hardcoded tracepoints.
We can't really use
TRACE_EVENT()here without using the listing you did. Instead, you could define a custom structftrace_event_callfromDEFINE_SYSCALL().In
regfunc()you can turn onTIF_FTRACE(using a refcounter).The struct
trace_eventftrace_event_typecan reuse the existing output callback for syscall tracing which retrieve the syscall informations.void ftrace_raw_event_##call() can be replaced by calling directly the existing generic callback for syscall tracing trace insertion.
Quoted 6 lines
Quoted 3 lines
And the arch mapping table can resolve a syscall number to its matching event.
hmmm..so I presume this would layer on 2 tracepoints? One in syscall entry and one in exit, presumably passing a 'struct
pt_regs'?Exactly, instead of having two tracepoints per syscalls, we would have only two generic and smart enough to handle all syscalls thanks to the syscalls metadata.
Quoted 4 lines
I think
the refcounter would also have to be deeper in the tracepoint infrastructure since the event tracing wouldn't be the only potential user of these tracepoints.
Quoted 7 lines
Yeah. Ie we would have two layers of syscalls tracing (de)activation:
the TIF flags to enable the whole syscall tracing, and also a state bit in syscalls metadata. So that we can activate/deactivate each of them independently, like any "normal" tracepoint.Frederic.
ok, this is what I was thinking here as well...I'll code this implementation up.
thanks,
-Jason