Trace code and documentation
Subsystems (3)
PATCH 1/2 Trace code and documentation
Trace - Provides tracing primitives
Tom Zanussi <[email protected]>
Martin Hunt <[email protected]>
David Wilder <[email protected]>
--- Documentation/trace.txt | 297 ++++++++++++++++++++++++ include/linux/trace.h | 99 ++++++++ lib/Kconfig | 10 + lib/Makefile | 2 + lib/trace.c | 575 +++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 983 insertions(+), 0 deletions(-)
Documentation/trace.txt new file +297 -0
@@ -0,0 +1,297 @@ +Trace Setup and Control +======================= +In the kernel, the trace interface provides a simple mechanism for +starting and managing data channels (traces) to user space. The +trace interface builds on the relay interface. For a complete +description of the relay interface, please see: +Documentation/filesystems/relay.txt. + +The trace interface provides a single layer in a complete tracing +application. Trace provides a kernel API that can be used for the setup +and control of tracing channels. User of trace must provide a data layer +responsible for formatting and writing data into the trace channels. + +A layered approach to tracing +============================= +A complete kernel tracing application consists of a data provider and +a data consumer. Both provider and consumer contain three layers; each +layer works in tandem with the corresponding layer in the opposite side. +The layers are represented in the following diagram. + +Provider Data layer + Formats raw trace data and provides data-related service. + For example, adding timestamps used by consumer to sort data. + +Provider Control layer + Provided by the trace interface, this layer creates trace channels + and informs the data layer and consumer of the current state + of the trace channels. + +Provider Buffering layer + Provided by relay. This layer buffers data in the + kernel for consumption by the consumer's buffer + layer. + +Provider (in-kernel facility) +----------------------------------------------------------------------------- +Consumer (user application) + + +Consumer Buffer layer + Reads/consumes data from the provider's data buffers. + +Consumer Control layer + Communicates to the provider's control layer to control the state + of the trace channels. + +Consumer Data layer + Sorts and formats data as provided by the provider's data layer. + +The provider is coded as a kernel facility. The consumer is coded as +a user application. + + +Trace - Features +================ +Trace exploits services and features provided by relay. These features +are: +- The creation and destruction of relay channels. +- Buffer management. Overwrite or non-overwrite modes can be selected + as well as global or per-CPU buffering. + +Overwrite mode can be called "flight recorder mode". Flight recorder +mode is selected by setting the TRACE_FLIGHT_CHANNEL flag when +creating trace channels. In flight mode when a tracing buffer is +full, the oldest records in the buffer will be discarded to make room +as new records arrive. In the default non-overwrite mode, new records +may be written only if the buffer has room. In either case, to +prevent data loss, a user space reader must keep the buffers +drained. Trace provides a means to detect the number of records that +have been dropped due to a buffer-full condition (non-overwrite mode +only). + +When per-CPU buffers are used, relay creates one debugfs file for each +running CPU. The user-space consumer of the data is responsible for +reading the per-CPU buffers and collating the records presumably using +a time stamp or sequence number included in the trace records. The +use of global buffers eliminates this extra work of sequencing +records; however the provider's data layer must hold a lock when +writing records. The lock prevents writers running on different CPUs +from overwriting each other's data. However, buffering may be slower +because writes to the buffer are serialized. Global buffering is +selected by setting the TRACE_GLOBAL_CHANNEL flag when creating trace +channels. + +Trace User Interface +=================== +When a trace channel is created and started, the following +directories and files are created in the root of the mounted debugfs. + +/debug (root of the debugfs) + /<trace-root-dir> + /<trace-name> + trace0 ... traceN Per-CPU trace data, one + file per CPU. + + state Start or stop tracing by + by writing the strings + "start" or "stop" to this + file. Read the file to get the + current state. + + dropped The number of records dropped + due to a full-buffer condition, + for non-TRACE_FLIGHT_CHANNELs + only. + + rewind Trigger a rewind by writing + to this file. i.e. start + next read at the beginning + again. Only available for + TRACE_FLIGHT_CHANNELS. + + + nr_sub Number of sub-buffers + in the channel. + + sub_size Size of sub-buffers in + the channnel. + +Trace data is gathered from the trace[0...N] files using one of the +available interfaces provided by relay. + +When using the READ(2) interface, as data is read it is marked as +consumed by the relay subsystem. Therefore, subsequent reads will +only return unconsumed data. + +Trace Kernel API +=============== +An overview of the trace Kernel API is now given. More details of the +API can be found in linux/trace.h. + +The steps a kernel data provider takes to utilize the trace interface are: +1) Set up a trace channel - trace_setup() +2) Start the trace channel - trace_start() +3) Write one or more trace records into the channel (using the relay API). + + Important: When writing a trace record the provider must insure that + preemption is disabled and that trace state is set to "running". a + typical function used to write records into a trace channel should + follow the following semantics: + + rcu_read_lock(); // disables preemption + if (trace_running(trace)){ + relay_write(....); // use any available relay data + function + } + rcu_read_unlock(); // enables preemption + +4) Stop and start tracing as desired - trace_start()/trace_stop() +5) Destroy the trace channel and underlying relay channel - +trace_cleanup(). + +Trace Example +=========== + +This small sample module creates a trace channel. It places a kprobe +on the function do_fork(). The value of current->pid is writen to +the trace channel. + +You can build the kernel module fork_trace.ko using the following +Makefile: +------------------------------------CUT------------------------------------- +obj-m := fork_trace.o +KDIR := /lib/modules/$(shell uname -r)/build +PWD := $(shell pwd) +default: + $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules +clean: + rm -f *.mod.c *.ko *.o +----------------------------------CUT-------------------------------------- + +/* fork_trace.c - An example of using trace in a kprobes module */ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/kprobes.h> +#include <linux/trace.h> + +#define USE_GLOBAL_BUFFERS 1 +#define USE_FLIGHT 1 + +#define PROBE_POINT "do_fork" + +static struct kprobe kp; +struct trace_info *kprobes_trace; + +#ifdef USE_GLOBAL_BUFFERS +static DEFINE_SPINLOCK(trace_lock); +#endif + +#define TRACE_PRINTF_TMPBUF_SIZE (1024) +static char trace_tmpbuf[NR_CPUS][TRACE_PRINTF_TMPBUF_SIZE]; + +void trace_printf(struct trace_info *trace, const char *format, ...) +{ + va_list args; + void *buf; + char *record; + int len=0; + + if (!trace) + return; + + buf = trace_tmpbuf[smp_processor_id()]; + +#ifdef USE_GLOBAL_BUFFERS + spin_lock(&trace_lock); +#endif + + rcu_read_lock(); + if (trace_running(trace)){ + va_start(args, format); + len += vsnprintf(buf+len, TRACE_PRINTF_TMPBUF_SIZE, + format, args); + va_end(args); + + if ((record = relay_reserve(trace->rchan, len))) + memcpy(record, buf, len); + } + rcu_read_unlock(); + +#ifdef USE_GLOBAL_BUFFERS + spin_unlock(&trace_lock); +#endif +} + + +int handler_pre(struct kprobe *p, struct pt_regs *regs) +{ + trace_printf(kprobes_trace,"%d\n", current->pid); + return 0; +} + + +int init_module(void) +{ + u32 flags=0; + +#ifdef USE_GLOBAL_BUFFERS + flags |= TRACE_GLOBAL_CHANNEL; +#endif + +#ifdef USE_FLIGHT + flags |= TRACE_FLIGHT_CHANNEL; +#endif + + /* setup the trace */ + kprobes_trace = trace_setup("trace_example",PROBE_POINT, + 1024,8,flags); + if (!kprobes_trace) + return -1; + + trace_start(kprobes_trace); + + /* setup the kprobe */ + kp.pre_handler = handler_pre; + kp.post_handler = NULL; + kp.fault_handler = NULL; + kp.symbol_name = PROBE_POINT; + if (register_kprobe(&kp) < 0) { + printk("fork_trace: register_kprobe failed\n"); + return -1; + } + return 0; +} + +void cleanup_module(void) +{ + unregister_kprobe(&kp); + trace_stop(kprobes_trace); + trace_cleanup(kprobes_trace); +} + +MODULE_LICENSE("GPL"); +-----------------------------CUT----------------------------------- +How to run the example: +$ mount -t debugfs /debug +$ make +$ insmod fork_trace.ko + +To view the data produced by the module: +$ cat /debug/trace_example/do_fork/trace0 + +Remove the module. +$ rmmod fork_trace + +The function trace_cleanup() is called when the module +is removed. This will cause the TRACE channel to be destroyed and the +corresponding files to disappear from the debug file system. + +Credits +======= +Trace is adapted from blktrace authored by Jens Axboe ([email protected]). + +Major contributions were made by: +Tom Zanussi <[email protected]> +Martin Hunt <[email protected]> +David Wilder <[email protected]>
include/linux/trace.h new file +99 -0
99 lines not on this page · raw message
lib/Kconfig +10 -0
@@ -141,4 +141,14 @@ config HAS_DMA config CHECK_SIGNATURE bool +config TRACE + bool "Trace setup and control" + select RELAY + select DEBUG_FS + help + This option provides support for the setup, teardown and control + of tracing channels from kernel code. It also provides trace + information and control to userspace via a set of debugfs control + files. If unsure, say N. + endmenu
lib/Makefile +2 -0
@@ -69,6 +69,8 @@ lib-$(CONFIG_GENERIC_BUG) += bug.o obj-$(CONFIG_PROFILE_LIKELY) += likely_prof.o +obj-$(CONFIG_TRACE) += trace.o + hostprogs-y := gen_crc32table clean-files := crc32table.h
lib/trace.c new file +575 -0
575 lines not on this page · raw message
Quoted 2 lines · On Thu, 13 Sep 2007 16:43:16 -0700 David Wilder wrote:
[it would be easier to review and make sense of the comments if the patch were inline instead of attached]
Tom Zanussi <[email protected]>
Martin Hunt <[email protected]>
David Wilder <[email protected]>
Above needs to use Signed-off-by: if you want this merged.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
trace.txt:
+When using the READ(2) interface,
s/READ/read/
+ preemption is disabled and that trace state is set to "running". a
s/. a/. A/
+on the function do_fork(). The value of current->pid is writen to
s/writen/written/
+You can build the kernel module fork_trace.ko using the following +Makefile:
There was reportedly a discussion about sample source code in the Documentation/ directory at the kernel summit. Some people want to move it to the util-linux package. If that's not done, I strongly prefer that Makefiles and source files be put into their own sub-directory, not "hidden" inside txt files, so you would end up with something line Documentation/trace/, with trace.txt, and also Documentation/trace/src, with Makefile and fork_trace.c & any other source files.
+Trace is adapted from blktrace authored by Jens Axboe ([email protected]).
MAINTAINERS file says <[email protected]>. He's also <[email protected]>.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/trace.c:
+ * Based on blktrace code, Copyright (C) 2006 Jens Axboe <[email protected]>
Ditto.
+
+
Use just one blank line between functions and/or structs.
Please check the patch with scripts/checkpatch.pl and then evaluate its warnings. Sometimes it makes sense to ignore some of them.
+/**
+ * trace_setup: create a new trace trace handle
+ *
+ * @root: The root directory name in the root of the debugfs
+ * to place trace directories. Created as needed.
Thanks for using kernel-doc; however, don't put a blank line between the function name line and the parameters. Also, the function name line should have a "-" separating the function name and the short description. (multiple places in trace.c)
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Hi David.
A random comment to the code.
Several of the struct file_operations are not declared static as they should be.
Btw. it looks good from a coding style point-of-view.
About the name what about ktrace??
Sam
Andrew Morton
maintainerQuoted 4 lines
Trace - Provides tracing primitives
...
lib/Kconfig config HAS_DMA · line 153
Quoted 10 lines
lib/Kconfig +10
-0+config TRACE + bool "Trace setup and control" + select RELAY + select DEBUG_FS + help + This option provides support for the setup, teardown and control + of tracing channels from kernel code. It also provides trace + information and control to userspace via a set of debugfs control + files. If unsure, say N. +
select is evil - you really want to avoid using it.
The problem is where you select a symbol whose dependencies aren't met. Kconfig resolves this incompatibility by just not selecting the thing you wanted, iirc. So your CONFIG_SYSFS=n, CONFIG_TRACE=y kernel won't build.
Quoted 2 lines
+/*
+ * Based on blktrace code, Copyright (C) 2006 Jens Axboe <[email protected]>
So can we migrate blktrace to using this?
lib/trace.c line 59
Quoted 5 lines
lib/trace.c new file +5
-0+static ssize_t state_write (struct file *filp,const char __user *buffer, +size_t count,loff_t *ppos) +{ +struct trace_info *trace = filp->private_data; +char buf[16 ] = {'\0' };
this initialisation isn't needed and will waste cycles.
Quoted 11 lines
lib/trace.c new file +11
-0+int ret; + +if (trace->flags &TRACE_DISABLE_STATE ) +return -EINVAL ; + +if (count >sizeof (buf) -1 ) +return -EINVAL ; + +if (copy_from_user (buf, buffer, count)) +return -EFAULT ; +
lib/trace.c line 80
Quoted 10 lines
lib/trace.c new file +10
-0+ buf[count] ='\0' ; + +if (strncmp (buf,"start" ,strlen ("start" )) ==0 ) { + ret =trace_start (trace); +if (ret) +return ret; + }else if (strncmp (buffer,"stop" ,strlen ("stop" )) ==0 ) +trace_stop (trace); +else +return -EINVAL ;
What's the above code doing? Trying to cope with trailing chars after
"start" or "stop"? Is that actually needed? It's the \n, I assume?
Quoted 24 lines
lib/trace.c new file +24
-0+return count; +} + + +static struct file_operations state_fops = { + .owner =THIS_MODULE , + .open = state_open, + .read = state_read, + .write = state_write, +}; + + +static void remove_root (struct trace_info *trace) +{ +if (trace->root->root &&simple_empty (trace->root->root)) { +debugfs_remove (trace->root->root); +list_del (&trace->root->list); +kfree (trace->root); + trace->root =NULL ; + } +} + + +static void remove_tree (struct trace_info *trace)
lib/trace.c line 115
Quoted 10 lines
lib/trace.c new file +10
-0+{ +mutex_lock (&trace_mutex); + +debugfs_remove (trace->dir); + +if (--trace->root->users ==0 ) +remove_root (trace); + +mutex_unlock (&trace_mutex); +}
We usually only put a single blank line between functions. Two is just a waste of screen space.
Quoted 6 lines
lib/trace.c new file +5
-0+ + +/* +* Creates the trace_root if it's not found. +*/
lib/trace.c line 252
Quoted 10 lines
lib/trace.c new file +8
-0+static ssize_t sub_size_read (struct file *filp,char __user *buffer, +size_t count,loff_t *ppos) +{ +struct trace_info *trace = filp->private_data; +char buf[32 ]; + +snprintf (buf,sizeof (buf),"%u\n" , + (unsigned int )trace->rchan->subbuf_size);
Use %tu to print a size_t, rather than the typecast.
Quoted 4 lines
lib/trace.c new file +3
-0+return simple_read_from_buffer (buffer, count, ppos, buf,strlen (buf)); +} +
lib/trace.c line 276
Quoted 10 lines
lib/trace.c new file +8
-0+static ssize_t nr_sub_read (struct file *filp,char __user *buffer, +size_t count,loff_t *ppos) +{ +struct trace_info *trace = filp->private_data; +char buf[32 ]; + +snprintf (buf,sizeof (buf),"%u\n" , + (unsigned int )trace->rchan->n_subbufs);
Ditto. (It's unobvious why n_subbufs is a size_t)
Quoted 8 lines
lib/trace.c new file +5
-0+return simple_read_from_buffer (buffer, count, ppos, buf,strlen (buf)); +} + +static void remove_controls (struct trace_info *trace) +{
lib/trace.c line 362
Quoted 10 lines
lib/trace.c new file +10
-0+if (trace->state_file) +debugfs_remove (trace->state_file); +if (trace->dropped_file) +debugfs_remove (trace->dropped_file); +if (trace->reset_consumed_file) +debugfs_remove (trace->reset_consumed_file); +if (trace->nr_sub_file) +debugfs_remove (trace->nr_sub_file); +if (trace->sub_size_file) +debugfs_remove (trace->sub_size_file);
debugfs_remove(NULL) is legal: all the above tests can be removed.
Quoted 28 lines
lib/trace.c new file +25
-0+if (trace->dir) +remove_tree(trace); +} + +* trace_setup: create a new trace trace handle +* +* @root: The root directory name in the root of the debugfs +* to place trace directories. Created as needed. +* @name: Trace directory name, created in @root +* @buf_size: size of the relay sub-buffers +* @buf_nr: number of relay sub-buffers +* @flags: Option selection (see GTSC channel flags definitions) +* default values when flags=0 are: use per-CPU buffering, +* use non-overwrite mode. See Documentation/trace.txt for details. +* +* returns a trace_info handle or NULL, if setup failed. +*/ +struct trace_info *trace_setup (const char *root,const char *name, +u32 buf_size,u32 buf_nr,u32 flags) +{ +struct trace_info *trace =NULL ; + + trace =setup_controls (root, name, flags); +if (!trace) +return NULL ;
lib/trace.c line 484
Quoted 10 lines
lib/trace.c new file +10
-0+ + trace->buf_size = buf_size; + trace->buf_nr = buf_nr; + trace->flags = flags; +mutex_init (&trace->state_mutex); + trace->state =TRACE_SETUP ; + +return trace; +} +EXPORT_SYMBOL_GPL (trace_setup);
It's better for a pointer-returning function to return an ERR_PTR on error, rather than NULL. That way the caller doesn't have to make guesses about why the callee failed when propagating back an error. (See how your init_module gives up and returns -1?)
lib/trace.c line 560
Quoted 13 lines
...
lib/trace.c new file +11
-0+ +/** +* trace_cleanup_channel: destroys the trace channel only +* +* @trace: trace handle to cleanup +*/ +static void trace_cleanup_channel (struct trace_info *trace) +{ +trace_stop (trace); +if (trace->rchan) +relay_close (trace->rchan);
relay_close(NULL) is legal. Please check the whole patch for this.
Quoted 18 lines
lib/trace.c new file +15
-0+ trace->rchan =NULL ; +} + +/** +* trace_cleanup: destroys the trace channel, control files and dir +* +* @trace: trace handle to cleanup +*/ +void trace_cleanup (struct trace_info *trace) +{ +trace_cleanup_channel (trace); +remove_controls (trace); +kfree (trace); +} +EXPORT_SYMBOL_GPL (trace_cleanup);
Quoted 4 lines · On Fri, 14 Sep 2007 18:08:40 -0700 Andrew Morton wrote:
Quoted 4 lines
Trace - Provides tracing primitives
...
Quoted 12 lines
lib/Kconfig config HAS_DMA · line 153
Quoted 10 lines
lib/Kconfig +10
-0+config TRACE + bool "Trace setup and control" + select RELAY + select DEBUG_FS + help + This option provides support for the setup, teardown and control + of tracing channels from kernel code. It also provides trace + information and control to userspace via a set of debugfs control + files. If unsure, say N. +select is evil - you really want to avoid using it.
I checked when I reviewed(?) this patch. There are a few other places that select also (IIRC, the blktrace code does), but most of them use depends on.
Quoted 9 lines
The problem is where you select a symbol whose dependencies aren't met.
Kconfigresolves this incompatibility by just not selecting the thing you wanted, iirc. So yourCONFIG_SYSFS=n,CONFIG_TRACE=ykernel won't build.Quoted 2 lines
+/*
+ * Based on blktrace code, Copyright (C) 2006 Jens Axboe <[email protected]>So can we migrate blktrace to using this?
Quoted 10 lines
lib/trace.c line 252
Quoted 8 lines
lib/trace.c new file +8
-0+static ssize_t sub_size_read (struct file *filp,char __user *buffer, +size_t count,loff_t *ppos) +{ +struct trace_info *trace = filp->private_data; +char buf[32 ]; + +snprintf (buf,sizeof (buf),"%u\n" , + (unsigned int )trace->rchan->subbuf_size);Use %tu to print a
size_t, rather than the typecast.
Eh?
Use %zu to print a size_t. Use %tu to print a ptrdiff_t.
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Quoted 6 lines · Andrew Morton
Quoted 3 lines
+/*
+ * Based on blktrace code, Copyright (C) 2006 Jens Axboe <[email protected]>So can we migrate blktrace to using this?
Yes, a blktrace patch is comming.
Quoted 17 lines
Quoted 17 lines
lib/trace.c new file +17
-0+int ret; + +if (trace->flags &TRACE_DISABLE_STATE ) +return -EINVAL ; + +if (count >sizeof (buf) -1 ) +return -EINVAL ; + +if (copy_from_user (buf, buffer, count)) +return -EFAULT ; + + buf[count] ='\0' ; + +if (strncmp (buf,"start" ,strlen ("start" )) ==0 ) { + ret =trace_start (trace); +if (ret) +return ret;
Quoted 9 lines
lib/trace.c line 80
Quoted 5 lines
lib/trace.c new file +4
-0+ }else if (strncmp (buffer,"stop" ,strlen ("stop" )) ==0 ) +trace_stop (trace); +else +return -EINVAL ;What's the above code doing? Trying to cope with trailing chars after
"start" or "stop"? Is that actually needed? It's the \n, I assume?
Yes, the typical usage is "echo start > state" and echo adds a \n.
Thanks for the comments, I will make the changes and resubmit.
Quoted 6 lines · Sam Ravnborg
Hi David.
A random comment to the code.
Several of the structfile_operationsare not declared static as they should be.
Quoted 7 lines
Btw. it looks good from a coding style point-of-view.
About the name what about ktrace??
Sam
Thanks for the comment. I sure don't want to change the name a forth time, can we live with "trace"?
Quoted 9 lines · David Wilder
Quoted 8 lines · Sam Ravnborg
Hi David.
A random comment to the code.
Several of the structfile_operationsare not declared static as they should be.Btw. it looks good from a coding style point-of-view.
Quoted 7 lines
Quoted 5 lines
About the name what about ktrace??
Sam
Thanks for the comment. I sure don't want to change the name a forth time, can we live with "trace"?
I do not care much about the name so no big deal for me. I was just soo generic..
Sam
Hi David,
Interesting work, but I think we could still enhance it. The interesting things you bring is the trace control though debugfs files, which is clear and simple. (I did it on top of netlink in LTTng, but I don't really care about the mechanism, as long as we have the same flexibility).
* David Wilder ([email protected]) wrote:
[...]
Documentation/trace.txt line 72
Quoted 12 lines
Documentation/trace.txt new file +11
-0+Overwrite mode can be called "flight recorder mode". Flight recorder +mode is selected by setting the TRACE_FLIGHT_CHANNEL flag when +creating trace channels. In flight mode when a tracing buffer is +full, the oldest records in the buffer will be discarded to make room +as new records arrive. In the default non-overwrite mode, new records +may be written only if the buffer has room. In either case, to +prevent data loss, a user space reader must keep the buffers +drained. Trace provides a means to detect the number of records that +have been dropped due to a buffer-full condition (non-overwrite mode +only). +
Since, in the end, we can represent the "flight recorder" as a simple flag, can we imagine setting/unsetting it while the trace is active ?
Also, why is trace creation an in-kernel API ? What about a mkdir in debugfs/trace ? I guess I see that this is because you need to keep a pointer to the created trace so you can record your events it in. Have you thought about keeping a global RCU list of active traces instead ? One could then iterate on every active trace and record information in them without having to bother about which specific trace it has created. However, this should come with the ability to filter in/out events in the handler on a per-trace basis.
The problem with the approach you propose is that it seems to tie a specific event source to one trace channel.
It would be good to have some way to separate:
- event sources (markers/kprobes/...)
- active traces.
Each of them would have an event filter.
- within each trace, the ability to create _multiple_ channels, so we can send the information in high/medium/low event rate channels on a per-event basis. This is really useful to gather hybrid traces made from flight recorder channels (high event rate) and non-over channels (important low rate information required to understand the trace).
- Each trace channel would be either global or per cpu, and would be a flight recorder channel or "normal", non overwrite, channel.
Documentation/trace.txt line 84
Quoted 12 lines
Documentation/trace.txt new file +12
-0+When per-CPU buffers are used, relay creates one debugfs file for each +running CPU. The user-space consumer of the data is responsible for +reading the per-CPU buffers and collating the records presumably using +a time stamp or sequence number included in the trace records. The +use of global buffers eliminates this extra work of sequencing +records; however the provider's data layer must hold a lock when +writing records. The lock prevents writers running on different CPUs +from overwriting each other's data. However, buffering may be slower +because writes to the buffer are serialized. Global buffering is +selected by setting the TRACE_GLOBAL_CHANNEL flag when creating trace +channels. +
We could allocate the trace buffers upon actions that would be independant from trace creation. By doing so, we could then do a
echo 1 > path_to_trace/channel/global
Before we activate the trace or allocate the buffers. I would vote for a
echo 1 > path_to_trace/channel/allocate
So we can separate the trace buffer allocation from trace start (because start operation might have to be done near from the studied events and we want it to be as lightweight as possible).
So, typical usage could be:
cd /mnt/debugfs/trace
mkdir mytrace
cd mytrace
echo 1 > start
The default could be that we create a trace with a "main" set of per-cpu channels. i.e.:
mytrace/main
But then, a mkdir within the mytrace directory could add new custom channels:
in mytrace:
mkdir processes
cd processes
(then set buffer size, nr subbuf, flight vs non flight, global..) echo 1 > allocate
By default, a trace event filter would accept all events. Events could be identified by a name (see markers proposed subsystem_event name). Issuing a :
echo 0 > path_to_trace/filter
would disable all events
echo "event_name" > path_to_trace/filter
would add the event_name to the trace filter
By default, events would be sent into the "main" channel, but echo "event_name" > path_to_trace/channel/filter would send the event in the "channel" channel instead.
We could think of integrating the markers macro into this scheme to describe the events. Instead of doing an explicit trace write in the breakpoint handler, we could simply put a marker (it could even be a branch-free marker if you prefer). In LTTng, upon trace start, I iterate on all the kernel's markers to record, in a "control" channel, all the marker names, their ids (I assign them a 16 bit id), and their format strings. It allows me to parse the trace given only the timestamps, event IDs and event specific data.
Mathieu
Signature · 3 lines
Mathieu Desnoyers Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68