tracing: Add support for in-kernel dynamic event API
Subsystems (2)
v2 cover tracing: Add support for in-kernel dynamic event API
Hi,
This is v2 of the previous 'tracing: Add support for in-kernel synthetic event API', and is largely a rewrite based on suggestions from Masami to expand the scope to include other types of dynamic events such as kprobe events, in addition to the original sythetic event focus.
The functionality of the original API remains available, though it's in a slightly different form due to the use of the new dynevent_cmd API it's now based on. The dynevent_cmd API provides a common dynamic event command generation capability that both the synthetic event API and the kprobe event API are now based on, and there are now test modules demonstrating both the synthetic event and kprobes APIs.
A couple of the patches are snippets from Masami's 'tracing:
bootconfig: Boot-time tracing and Extra boot config' series, and the patch implementing the dynevent_cmd API includes some of the spnprintf() generating code from that patchset.
Because I used Masami's gen_*_cmd() naming suggestion for generating the commands, the previous patchset's generate_*() functions were renamed to trace_*() to avoid confusion, and probably is better naming anyway.
An overview of the user-visible changes in comparison to v1:
- create_synth_event() using an array of synth_desc_fields remains unchanged and works the same way as previously
- gen_synth_cmd() takes a variable-length number of args which represent 'type field;' pairs. Using this with no field args basically replaces the previous 'create_empty_synth_event()'
- The 'add_synth_field()' and 'add_synth_fields()' function from v1 are essentially the same except that they now take a dynevent_cmd instead of a synth_event pointer
- The finalize_synth_event() from v1 is replaced by create_dynevent() in the new version.
- The new v2 API includes some additional functions to initialize the dynevent_cmd - synth_dynevent_cmd() is used to do that. While it's an extra step, it makes it easier to do error handling.
- There's a new trace_synth_event() function that traces a synthetic event using a variable-arg list of values.
- The original generate_synth_event() using an array of values is now called trace_synth_event_array().
- For piecewise event tracing, the original generate_synth_event_start() and generate_synth_event_end() have now been renamed to trace_synth_event_end().
- add_next_synth_val() and add_synth_val() remain the same.
- A similar API and test module demonstrating the API has been added for kprobe events
- Both the synthetic events and kprobe events API is based on the dynevent_cmd API, newly added
- The Documentation for all of the above has been updated
Text from the orginal v1 posting:
I've recently had several requests and suggestions from users to add support for the creation and generation of synthetic events from kernel code such as modules, and not just from the available command line commands.
This patchset adds support for that. The first three patches add some minor preliminary setup, followed by the two main patches that add the ability to create and generate synthetic events from the kernel. The next patch adds a test module that demonstrates actual use of the API and verifies that it works as intended, followed by Documentation.
Special thanks to Artem Bityutskiy, who worked with me over several iterations of the API, and who had many great suggestions on the details of the interface, and pointed out several problems with the code itself.
The following changes since commit d783b3c08c14fccbc4d5ef33a38288ec9b264df7:
tracing: Have the histogram compare functions convert to u64 first (2019-12-11 15:47:14 -0500)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/zanussi/linux-trace.git ftrace/synth-event-gen-v2
Masami Hiramatsu (2):
tracing: Add synth_event_run_command()
tracing: Add trace_kprobe_run_command()
Tom Zanussi (10):
tracing: Add trace_array_find() to find instance trace arrays
tracing: Add get/put_event_file()
tracing: Add delete_synth_event()
tracing: Add dynamic event command creation interface
tracing: Add synthetic event command generation functions
tracing: Add trace_synth_event() and related functions
tracing: Add synth event generation test module
tracing: Add kprobe event command generation functions
tracing: Add kprobe event command generation test module
tracing: Documentation for in-kernel synthetic event API
Documentation/trace/events.rst | 488 ++++++++++++++++++++ include/linux/trace_events.h | 134 ++++++ kernel/trace/Kconfig | 25 ++ kernel/trace/Makefile | 2 + kernel/trace/kprobe_event_gen_test.c | 223 ++++++++++ kernel/trace/synth_event_gen_test.c | 519 +++++++++++++++++++++ kernel/trace/trace.c | 30 +- kernel/trace/trace.h | 1 + kernel/trace/trace_events.c | 304 +++++++++++++ kernel/trace/trace_events_hist.c | 842 ++++++++++++++++++++++++++++++++++- kernel/trace/trace_kprobe.c | 162 +++++++ 11 files changed, 2704 insertions(+), 26 deletions(-) create mode 100644 kernel/trace/kprobe_event_gen_test.c create mode 100644 kernel/trace/synth_event_gen_test.c
Signature · 1 line
2.14.1
PATCH v2 1/12 tracing: Add trace_array_find() to find instance trace arrays
Add a new trace_array_find() function that can be used to find a trace array given the instance name, and replace existing code that does the same thing with it.
Also make it available for use outside of trace.c.
--- kernel/trace/trace.c | 30 ++++++++++++++++++++---------- kernel/trace/trace.h | 1 + 2 files changed, 21 insertions(+), 10 deletions(-)
kernel/trace/trace.c +20 -10
@@ -8455,6 +8455,21 @@ static void update_tracer_options(struct trace_array *tr)mutex_unlock (&trace_types_lock); } +/* Must have trace_types_lock held */ +struct trace_array *trace_array_find (const char *name) +{ +struct trace_array *tr, *found =NULL ; + +list_for_each_entry (tr, &ftrace_trace_arrays, list) { +if (tr->name &&strcmp (tr->name, name) ==0 ) { + found = tr; +break ; + } + } + +return found; +} +static struct trace_array *trace_array_create (const char *name) {struct trace_array *tr;
@@ -8531,10 +8546,8 @@ static int instance_mkdir(const char *name)mutex_lock (&trace_types_lock); ret = -EEXIST ; -list_for_each_entry (tr, &ftrace_trace_arrays, list) { -if (tr->name &&strcmp (tr->name, name) ==0 ) - goto out_unlock; - } +if (trace_array_find (name)) + goto out_unlock; tr =trace_array_create (name);
@@ -8658,12 +8671,9 @@ static int instance_rmdir(const char *name)mutex_lock (&trace_types_lock); ret = -ENODEV ; -list_for_each_entry (tr, &ftrace_trace_arrays, list) { -if (tr->name &&strcmp (tr->name, name) ==0 ) { - ret =__remove_instance (tr); -break ; - } - } + tr =trace_array_find (name); +if (tr) + ret =__remove_instance (tr);mutex_unlock (&trace_types_lock);mutex_unlock (&event_mutex);
kernel/trace/trace.h +1 -0
@@ -346,6 +346,7 @@ extern struct mutex trace_types_lock;extern int trace_array_get (struct trace_array *tr);extern int tracing_check_open_get_tr (struct trace_array *tr); +extern struct trace_array *trace_array_find (const char *name);extern int tracing_set_time_stamp_abs (struct trace_array *tr,bool abs);extern int tracing_set_clock (struct trace_array *tr,const char *clockstr);
Signature · 1 line
2.14.1
PATCH v2 2/12 tracing: Add get/put_event_file()
In reply to Tom ZanussiAdd a function to get an event file and prevent it from going away on module or instance removal.
get_event_file() will find an event file in a given instance (if instance is NULL, it assumes the top trace array) and return it, pinning the instance's trace array as well as the event's module, if applicable, so they won't go away while in use.
put_event_file() does the matching release.
--- include/linux/trace_events.h | 5 +++ kernel/trace/trace_events.c | 87 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+)
include/linux/trace_events.h +5 -0
@@ -348,6 +348,11 @@ enum { EVENT_FILE_FL_WAS_ENABLED_BIT, }; +extern struct trace_event_file *get_event_file (const char *instance, +const char *system, +const char *event); +extern void put_event_file (struct trace_event_file *file); +/* * Event file flags: * ENABLED - The event is enabled
kernel/trace/trace_events.c +87 -0
93 lines not on this page · raw message
Signature · 1 line
2.14.1
Masami Hiramatsu
maintainerRe: [PATCH v2 02/12] tracing: Add get/put_event_file()
Hi Tom,
Quoted 69 lines · On Fri, 10 Jan 2020 14:35:08 -0600 Tom Zanussi <[email protected]> wrote:
Add a function to get an event file and prevent it from going away on module or instance removal.
get_event_file()will find an event file in a given instance (if instance is NULL, it assumes the top trace array) and return it, pinning the instance's trace array as well as the event's module, if applicable, so they won't go away while in use.
put_event_file()does the matching release.Signed-off-by Tom Zanussi <[email protected]>--- include/linux/trace_events.h | 5 +++ kernel/trace/trace_events.c | 87 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+)include/linux/trace_events.h +5
-0@@ -348,6 +348,11 @@ enum { EVENT_FILE_FL_WAS_ENABLED_BIT, }; +extern struct trace_event_file *get_event_file (const char *instance, +const char *system, +const char *event); +extern void put_event_file (struct trace_event_file *file); +/* * Event file flags: * ENABLED - The event is enabled kernel/trace/trace_events.c +29
-0@@ -2535,6 +2535,93 @@ find_event_file(struct trace_array *tr, const char *system, const char *event)return file; } +/** +* get_event_file - Find and return a trace event file +* @instance: The name of the trace instance containing the event +* @system: The name of the system containing the event +* @event: The name of the event +* +* Return a trace event file given the trace instance name, trace +* system, and trace event name. If the instance name is NULL, it +* refers to the top-level trace array. +* +* This function will look it up and return it if found, after calling +* trace_array_get() to prevent the instance from going away, and +* increment the event's module refcount to prevent it from being +* removed. +* +* To release the file, call put_event_file(), which will call +* trace_array_put() and decrement the event's module refcount. +* +* Return: The trace event on success, ERR_PTR otherwise. +*/ +struct trace_event_file *get_event_file (const char *instance, +const char *system, +const char *event) +{ +struct trace_array *tr =top_trace_array (); +struct trace_event_file *file =NULL ; +int ret = -EINVAL ; + +if (instance) {
kernel/trace/trace_events.c find_event_file(struct trace_array *tr, const char *system, const char *event) · line 2576
Quoted 10 lines
kernel/trace/trace_events.c +10
-0+mutex_lock (&trace_types_lock); + tr =trace_array_find (instance); +mutex_unlock (&trace_types_lock); +if (!tr) +return ERR_PTR (ret); + } + + ret =trace_array_get (tr); +if (ret) +return ERR_PTR (ret);
This seems a bit odd. I think we should introduce trace_array_find_get(instance) so that we don't take care of loosing tr there. It should be something like
trace_array_find_get(instance) {
mutex_lock(&trace_types_lock);
tr = trace_array_find(instance);
if (tr)
tr->ref++;
mutex_unlock(&trace_types_lock);
return tr;
}Then we can write it as below;
struct trace_array *tr = top_trace_array();
if (instance) {
tr = trace_array_find_get(instance);
if (!tr)
return -ENOENT;
} else
trace_array_get(tr);/* We never loose top trace_array */
...This will fail only if there is no instance.
BTW, I think the function name should be get_trace_event_file() because "event_file" sounds a bit generic. (or trace_get_event_file() will give us a good namespace)
Thank you,
Quoted 54 lines
kernel/trace/trace_events.c +48
-0+ +mutex_lock (&event_mutex); + + file =find_event_file (tr, system, event); +if (!file) { +trace_array_put (tr); + ret = -EINVAL ; + goto out; + } + +/* Don't let event modules unload while in use */ + ret =try_module_get (file->event_call->mod); +if (!ret) { +trace_array_put (tr); + ret = -EBUSY ; + goto out; + } + + ret =0 ; + out: +mutex_unlock (&event_mutex); + +if (ret) + file =ERR_PTR (ret); + +return file; +} +EXPORT_SYMBOL_GPL (get_event_file); + +/** +* put_event_file - Release a file from get_event_file() +* @file: The trace event file +* +* If a file was retrieved using get_event_file(), this should be +* called when it's no longer needed. It will cancel the previous +* trace_array_get() called by that function, and decrement the +* event's module refcount. +*/ +void put_event_file (struct trace_event_file *file) +{ +trace_array_put (file->tr); + +mutex_lock (&event_mutex); +module_put (file->event_call->mod); +mutex_unlock (&event_mutex); +} +EXPORT_SYMBOL_GPL (put_event_file); +#ifdef CONFIG_DYNAMIC_FTRACE /* Avoid typos */ Signature · 1 line
2.14.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
Re: [PATCH v2 02/12] tracing: Add get/put_event_file()
Hi Masami,
Quoted 119 lines · Masami Hiramatsu
Hi Tom,
Quoted 77 lines · On Fri, 10 Jan 2020 14:35:08 -0600 Tom Zanussi <[email protected]> wrote:
Add a function to get an event file and prevent it from going away on
module or instance removal.
get_event_file()will find an event file in a given instance (if instance is NULL, it assumes the top trace array) and return it, pinning the instance's trace array as well as the event's module, if
applicable, so they won't go away while in use.
put_event_file()does the matching release.Signed-off-by Tom Zanussi <[email protected]>--- include/linux/trace_events.h | 5 +++ kernel/trace/trace_events.c | 87 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+)include/linux/trace_events.h +5
-0@@ -348,6 +348,11 @@ enum { EVENT_FILE_FL_WAS_ENABLED_BIT, }; +extern struct trace_event_file *get_event_file (const char *instance, +const char *system, +const char *event); +extern void put_event_file (struct trace_event_file *file); +/* * Event file flags: * ENABLED - The event is enabled
kernel/trace/trace_events.c +11
-0@@ -2535,6 +2535,93 @@ find_event_file(struct trace_array *tr, const char *system, const char *event) return file; } +/** +* get_event_file - Find and return a trace event file +* @instance: The name of the trace instance containing the event +* @system: The name of the system containing the event +* @event: The name of the event +* +* Return a trace event file given the trace instance name, trace +* system, and trace event name. If the instance name is NULL, it +* refers to the top-level trace array. +* +* This function will look it up and return it if found, after calling
+ *trace_array_get()to prevent the instance from going away, and
+ * increment the event's module refcount to prevent it from being
+ * removed.
+ *
+ * To release the file, callput_event_file(), which will call
+ *trace_array_put()and decrement the event's module refcount.
+ *
+ * Return: The trace event on success,ERR_PTRotherwise.
+ */kernel/trace/trace_events.c +9
-0+struct trace_event_file *get_event_file (const char *instance, +const char *system, +const char *event) +{ +struct trace_array *tr =top_trace_array (); +struct trace_event_file *file =NULL ; +int ret = -EINVAL ; + +if (instance) {kernel/trace/trace_events.c find_event_file(struct trace_array *tr, const char *system, const char *event) · line 2576
Quoted 10 lines
kernel/trace/trace_events.c +10
-0+mutex_lock (&trace_types_lock); + tr =trace_array_find (instance); +mutex_unlock (&trace_types_lock); +if (!tr) +return ERR_PTR (ret); + } + + ret =trace_array_get (tr); +if (ret) +return ERR_PTR (ret);This seems a bit odd. I think we should introduce
trace_array_find_get(instance)so that we don't take care of loosing tr there. It should be something liketrace_array_find_get(instance) { mutex_lock(&trace_types_lock); tr = trace_array_find(instance); if (tr) tr->ref++; mutex_unlock(&trace_types_lock); return tr; }Then we can write it as below;
struct trace_array *tr = top_trace_array(); if (instance) { tr = trace_array_find_get(instance); if (!tr) return -ENOENT; } else trace_array_get(tr);/* We never loose top trace_array */ ...
Quoted 2 lines
This will fail only if there is no instance.
Yes, that's a better idea, will do that in v3.
Quoted 4 lines
BTW, I think the function name should be
get_trace_event_file()because "event_file" sounds a bit generic. (ortrace_get_event_file()will give us a good namespace)
I agree, will use trace_get_event_file().
Thanks,
Tom
Quoted 58 lines
Thank you,
Quoted 54 lines
kernel/trace/trace_events.c +48
-0+ +mutex_lock (&event_mutex); + + file =find_event_file (tr, system, event); +if (!file) { +trace_array_put (tr); + ret = -EINVAL ; + goto out; + } + +/* Don't let event modules unload while in use */ + ret =try_module_get (file->event_call->mod); +if (!ret) { +trace_array_put (tr); + ret = -EBUSY ; + goto out; + } + + ret =0 ; + out: +mutex_unlock (&event_mutex); + +if (ret) + file =ERR_PTR (ret); + +return file; +} +EXPORT_SYMBOL_GPL (get_event_file); + +/** +* put_event_file - Release a file from get_event_file() +* @file: The trace event file +* +* If a file was retrieved using get_event_file(), this should be +* called when it's no longer needed. It will cancel the previous +* trace_array_get() called by that function, and decrement the +* event's module refcount. +*/ +void put_event_file (struct trace_event_file *file) +{ +trace_array_put (file->tr); + +mutex_lock (&event_mutex); +module_put (file->event_call->mod); +mutex_unlock (&event_mutex); +} +EXPORT_SYMBOL_GPL (put_event_file); +#ifdef CONFIG_DYNAMIC_FTRACE /* Avoid typos */ Signature · 1 line
2.14.1
PATCH v2 3/12 tracing: Add delete_synth_event()
In reply to Tom Zanussicreate_or_delete_synth_event() contains code to delete a synthetic event, which would be useful on its own - specifically, it would be useful to allow event-creating modules to call it separately.
Separate out the delete code from that function and create an exported function named delete_synth_event().
--- include/linux/trace_events.h | 2 ++ kernel/trace/trace_events_hist.c | 57 +++++++++++++++++++++++++++++----------- 2 files changed, 43 insertions(+), 16 deletions(-)
include/linux/trace_events.h +2 -0
@@ -353,6 +353,8 @@ extern struct trace_event_file *get_event_file(const char *instance, const char *event);extern void put_event_file (struct trace_event_file *file); +extern int delete_synth_event (const char *name); +/* * Event file flags: * ENABLED - The event is enabled
kernel/trace/trace_events_hist.c +41 -16
70 lines not on this page · raw message
Signature · 1 line
2.14.1
PATCH v2 4/12 tracing: Add dynamic event command creation interface
In reply to Tom ZanussiAdd an interface used to build up dynamic event creation commands, such as synthetic and kprobe events. Interfaces specific to those particular types of events and others can be built on top of this interface.
Command creation is started by first using the dynevent_cmd_init() function to initialize the dynevent_cmd object. Following that, args are appended and optionally checked by the add_dynevent_arg() and add_dynevent_arg_pair() functions, which use objects representing arguments and pairs of arguments, initialized respectively by dynevent_arg_init() and dynevent_arg_pair_init(). Finally, once all args have been successfully added, the command is finalized and actually created using create_dynevent().
The code here for actually printing into the dyn_event->cmd buffer using snprintf() etc was adapted from v4 of Masami's 'tracing/boot:
Add synthetic event support' patch.
--- include/linux/trace_events.h | 54 +++++++++++ kernel/trace/trace_events.c | 217 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 271 insertions(+)
include/linux/trace_events.h +54 -0
@@ -353,6 +353,60 @@ extern struct trace_event_file *get_event_file(const char *instance, const char *event);extern void put_event_file (struct trace_event_file *file); +struct dynevent_cmd ; + +typedef int (*dynevent_check_arg_fn_t )(void *data); +typedef int (*dynevent_create_fn_t )(struct dynevent_cmd *cmd); + +struct dynevent_arg { +const char *str; +char separator;/* e.g. ';', ',', or nothing */ +dynevent_check_arg_fn_t check_arg; +}; + +extern void dynevent_arg_init (struct dynevent_arg *arg, +dynevent_check_arg_fn_t check_arg, +char separator); +extern int add_dynevent_arg (struct dynevent_cmd *cmd, +struct dynevent_arg *arg); + +struct dynevent_arg_pair { +const char *lhs; +const char *rhs; +char operator;/* e.g. '=' or nothing */ +char separator;/* e.g. ';', ',', or nothing */ +dynevent_check_arg_fn_t check_arg; +}; + +extern void dynevent_arg_pair_init (struct dynevent_arg_pair *arg_pair, +dynevent_check_arg_fn_t check_arg, +char operator,char separator); +extern int add_dynevent_arg_pair (struct dynevent_cmd *cmd, +struct dynevent_arg_pair *arg_pair); + +#define MAX_DYNEVENT_CMD_LEN (2048) + +enum dynevent_type { +DYNEVENT_TYPE_SYNTH , +DYNEVENT_TYPE_KPROBE , +}; + +struct dynevent_cmd { +char *buf; +const char *event_name; +int maxlen; +int remaining; +unsigned int n_fields; +enum dynevent_type type; +dynevent_create_fn_t run_command; +void *private_data; +}; + +extern void dynevent_cmd_init (struct dynevent_cmd *cmd,char *buf,int maxlen, +enum dynevent_type type, +dynevent_create_fn_t run_command); +extern int create_dynevent (struct dynevent_cmd *cmd); +extern int delete_synth_event (const char *name);/*
kernel/trace/trace_events.c +217 -0
223 lines not on this page · raw message
Signature · 1 line
2.14.1
Masami Hiramatsu
maintainerRe: [PATCH v2 04/12] tracing: Add dynamic event command creation interface
Hi Tom,
Would you have any reason to export these raw interface to modules? (add_dynevent_arg() and add_dynevent_arg_pair())
I'm OK to export wrapped interfaces, but I can't find any good reason to export these raw interfaces. Especially since the argument format has been defined by each event type (synthetic, kprobe and uprobe), so modules may not be able to add new one...
Thank you,
Quoted 321 lines · On Fri, 10 Jan 2020 14:35:10 -0600 Tom Zanussi <[email protected]> wrote:
Add an interface used to build up dynamic event creation commands, such as synthetic and kprobe events. Interfaces specific to those particular types of events and others can be built on top of this interface.
Command creation is started by first using the
dynevent_cmd_init()function to initialize thedynevent_cmdobject. Following that, args are appended and optionally checked by theadd_dynevent_arg()andadd_dynevent_arg_pair()functions, which use objects representing arguments and pairs of arguments, initialized respectively bydynevent_arg_init()anddynevent_arg_pair_init(). Finally, once all args have been successfully added, the command is finalized and actually created usingcreate_dynevent().The code here for actually printing into the
dyn_event->cmdbuffer usingsnprintf()etc was adapted from v4 of Masami's 'tracing/boot:
Add synthetic event support' patch.Signed-off-by Tom Zanussi <[email protected]>Signed-off-by Masami Hiramatsu <[email protected]>--- include/linux/trace_events.h | 54 +++++++++++ kernel/trace/trace_events.c | 217 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 271 insertions(+)include/linux/trace_events.h +54
-0@@ -353,6 +353,60 @@ extern struct trace_event_file *get_event_file(const char *instance, const char *event);extern void put_event_file (struct trace_event_file *file); +struct dynevent_cmd ; + +typedef int (*dynevent_check_arg_fn_t )(void *data); +typedef int (*dynevent_create_fn_t )(struct dynevent_cmd *cmd); + +struct dynevent_arg { +const char *str; +char separator;/* e.g. ';', ',', or nothing */ +dynevent_check_arg_fn_t check_arg; +}; + +extern void dynevent_arg_init (struct dynevent_arg *arg, +dynevent_check_arg_fn_t check_arg, +char separator); +extern int add_dynevent_arg (struct dynevent_cmd *cmd, +struct dynevent_arg *arg); + +struct dynevent_arg_pair { +const char *lhs; +const char *rhs; +char operator;/* e.g. '=' or nothing */ +char separator;/* e.g. ';', ',', or nothing */ +dynevent_check_arg_fn_t check_arg; +}; + +extern void dynevent_arg_pair_init (struct dynevent_arg_pair *arg_pair, +dynevent_check_arg_fn_t check_arg, +char operator,char separator); +extern int add_dynevent_arg_pair (struct dynevent_cmd *cmd, +struct dynevent_arg_pair *arg_pair); + +#define MAX_DYNEVENT_CMD_LEN (2048) + +enum dynevent_type { +DYNEVENT_TYPE_SYNTH , +DYNEVENT_TYPE_KPROBE , +}; + +struct dynevent_cmd { +char *buf; +const char *event_name; +int maxlen; +int remaining; +unsigned int n_fields; +enum dynevent_type type; +dynevent_create_fn_t run_command; +void *private_data; +}; + +extern void dynevent_cmd_init (struct dynevent_cmd *cmd,char *buf,int maxlen, +enum dynevent_type type, +dynevent_create_fn_t run_command); +extern int create_dynevent (struct dynevent_cmd *cmd); +extern int delete_synth_event (const char *name);/* kernel/trace/trace_events.c +217
-0@@ -3301,6 +3301,223 @@ void __init trace_event_init(void)event_trace_enable (); } +/** +* add_dynevent_arg - Add an arg to a dynevent_cmd +* @cmd: A pointer to the dynevent_cmd struct representing the new event cmd +* @arg: The argument to append to the current cmd +* +* Append an argument to a dynevent_cmd. The argument string will be +* appended to the current cmd string, followed by a separator, if +* applicable. Before the argument is added, the check_arg() +* function, if defined, is called. +* +* The cmd string, separator, and check_arg() function should be set +* using the dynevent_arg_init() before any arguments are added using +* this function. +* +* Return: 0 if successful, error otherwise. +*/ +int add_dynevent_arg (struct dynevent_cmd *cmd, +struct dynevent_arg *arg) +{ +int ret =0 ; +int delta; +char *q; + +if (arg->check_arg) { + ret = arg->check_arg (arg); +if (ret) +return ret; + } + + q = cmd->buf + (cmd->maxlen - cmd->remaining); + + delta =snprintf (q, cmd->remaining," %s%c" , arg->str, arg->separator); +if (delta >= cmd->remaining) { +pr_err ("String is too long: %s\n" , arg->str); +return -E2BIG ; + } + cmd->remaining -= delta; + +return ret; +} +EXPORT_SYMBOL_GPL (add_dynevent_arg); + +/** +* add_dynevent_arg_pair - Add an arg pair to a dynevent_cmd +* @cmd: A pointer to the dynevent_cmd struct representing the new event cmd +* @arg_pair: The argument pair to append to the current cmd +* +* Append an argument pair to a dynevent_cmd. An argument pair +* consists of a left-hand-side argument and a right-hand-side +* argument separated by an operator, which can be whitespace, all +* followed by a separator, if applicable. This can be used to add +* arguments of the form 'type variable_name;' or 'x+y'. +* +* The lhs argument string will be appended to the current cmd string, +* followed by an operator, if applicable, followd by the rhs string, +* followed finally by a separator, if applicable. Before anything is +* added, the check_arg() function, if defined, is called. +* +* The cmd strings, operator, separator, and check_arg() function +* should be set using the dynevent_arg_init() before any arguments +* are added using this function. +* +* Return: 0 if successful, error otherwise. +*/ +int add_dynevent_arg_pair (struct dynevent_cmd *cmd, +struct dynevent_arg_pair *arg_pair) +{ +int ret =0 ; +int delta; +char *q; + +if (arg_pair->check_arg) { + ret = arg_pair->check_arg (arg_pair); +if (ret) +return ret; + } + + q = cmd->buf + (cmd->maxlen - cmd->remaining); + + delta =snprintf (q, cmd->remaining," %s%c" , arg_pair->lhs, + arg_pair->operator); +if (delta >= cmd->remaining) { +pr_err ("field string is too long: %s\n" , arg_pair->lhs); +return -E2BIG ; + } + cmd->remaining -= delta; q += delta; + + delta =snprintf (q, cmd->remaining,"%s%c" , arg_pair->rhs, + arg_pair->separator); +if (delta >= cmd->remaining) { +pr_err ("field string is too long: %s\n" , arg_pair->rhs); +return -E2BIG ; + } + cmd->remaining -= delta; q += delta; + +return ret; +} +EXPORT_SYMBOL_GPL (add_dynevent_arg_pair); + +/** +* dynevent_cmd_init - Initialize a dynevent_cmd object +* @cmd: A pointer to the dynevent_cmd struct representing the cmd +* @buf: A pointer to the buffer to generate the command into +* @maxlen: The length of the buffer the command will be generated into +* @type: The type of the cmd, checked against further operations +* @run_command: The type-specific function that will actually run the command +* +* Initialize a dynevent_cmd. A dynevent_cmd is used to build up and +* run dynamic event creation commands, such as commands for creating +* synthetic and kprobe events. Before calling any of the functions +* used to build the command, a dynevent_cmd object should be +* instantiated and initialized using this function. +* +* The initialization sets things up by saving a pointer to the +* user-supplied buffer and its length via the @buf and @maxlen +* params, and by saving the cmd-specific @type and @run_command +* params which are used to check subsequent dynevent_cmd operations +* and actually run the command when complete. +*/ +void dynevent_cmd_init (struct dynevent_cmd *cmd,char *buf,int maxlen, +enum dynevent_type type, +dynevent_create_fn_t run_command) +{ +memset (cmd,'\0' ,sizeof (*cmd)); + + cmd->buf = buf; + cmd->maxlen = maxlen; + cmd->remaining = cmd->maxlen; + cmd->type = type; + cmd->run_command = run_command; +} +EXPORT_SYMBOL_GPL (dynevent_cmd_init); + +/** +* dynevent_arg_init - Initialize a dynevent_arg object +* @arg: A pointer to the dynevent_arg struct representing the arg +* @check_arg: An (optional) pointer to a function checking arg sanity +* @separator: An (optional) separator, appended after adding the arg +* +* Initialize a dynevent_arg object. A dynevent_arg represents an +* object used to append single arguments to the current command +* string. The @check_arg function, if present, will be used to check +* the sanity of the current arg string (which is directly set by the +* caller). After the arg string is successfully appended to the +* command string, the optional @separator is appended. If no +* separator was specified when initializing the arg, a space will be +* appended. +*/ +void dynevent_arg_init (struct dynevent_arg *arg, +dynevent_check_arg_fn_t check_arg, +char separator) +{ +memset (arg,'\0' ,sizeof (*arg)); + +if (!separator) + separator =' ' ; + arg->separator = separator; + + arg->check_arg = check_arg; +} +EXPORT_SYMBOL_GPL (dynevent_arg_init); + +/** +* dynevent_arg_pair_init - Initialize a dynevent_arg_pair object +* @arg_pair: A pointer to the dynevent_arg_pair struct representing the arg +* @check_arg: An (optional) pointer to a function checking arg sanity +* @operator: An (optional) operator, appended after adding the first arg +* @separator: An (optional) separator, appended after adding the second arg +* +* Initialize a dynevent_arg_pair object. A dynevent_arg_pair +* represents an object used to append argument pairs such as 'type +* variable_name;' or 'x+y' to the current command string. An +* argument pair consists of a left-hand-side argument and a +* right-hand-side argument separated by an operator, which can be +* whitespace, all followed by a separator, if applicable. The +* @check_arg function, if present, will be used to check the sanity +* of the current arg strings (which is directly set by the caller). +* After the first arg string is successfully appended to the command +* string, the optional @operator is appended, followed by the second +* arg and and optional @separator. If no separator was specified +* when initializing the arg, a space will be appended. +*/ +void dynevent_arg_pair_init (struct dynevent_arg_pair *arg_pair, +dynevent_check_arg_fn_t check_arg, +char operator,char separator) +{ +memset (arg_pair,'\0' ,sizeof (*arg_pair)); + +if (!operator) + operator =' ' ; + arg_pair->operator = operator; + +if (!separator) + separator =' ' ; + arg_pair->separator = separator; + + arg_pair->check_arg = check_arg; +} +EXPORT_SYMBOL_GPL (dynevent_arg_pair_init); + +/** +* create_dynevent - Create the dynamic event contained in dynevent_cmd +* @cmd: The dynevent_cmd object containing the dynamic event creation command +* +* Once a dynevent_cmd object has been successfully built up via the +* dynevent_cmd_init(), add_dynevent_arg() and add_dynevent_arg_pair() +* functions, this function runs the final command to actually create +* the event. +* +* Return: 0 if the event was successfully created, error otherwise. +*/ +int create_dynevent (struct dynevent_cmd *cmd) +{ +return cmd->run_command (cmd); +} +EXPORT_SYMBOL_GPL (create_dynevent); +#ifdef CONFIG_EVENT_TRACE_STARTUP_TEST static DEFINE_SPINLOCK (test_spinlock);Signature · 1 line
2.14.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
Re: [PATCH v2 04/12] tracing: Add dynamic event command creation interface
Hi Masami,
Quoted 10 lines · Masami Hiramatsu
Hi Tom,
Would you have any reason to export these raw interface to modules? (
add_dynevent_arg()andadd_dynevent_arg_pair())I'm OK to export wrapped interfaces, but I can't find any good reason to export these raw interfaces. Especially since the argument format has been defined by each event type (synthetic, kprobe and uprobe), so modules may not be able to add new one...
I didn't have a strong reason to export them, other than I thought they might come in handy for some user at some point. But you make some good points, and I'm fine with assuming that only the wrapped interfaces are what we care about. I'll unexport them in the next revision.
Thanks,
Tom
Quoted 382 lines
Thank you,
Quoted 375 lines · On Fri, 10 Jan 2020 14:35:10 -0600 Tom Zanussi <[email protected]> wrote:
Add an interface used to build up dynamic event creation commands, such as synthetic and kprobe events. Interfaces specific to those particular types of events and others can be built on top of this interface.
Command creation is started by first using the
dynevent_cmd_init()function to initialize thedynevent_cmdobject. Following that, args
are appended and optionally checked by theadd_dynevent_arg()andadd_dynevent_arg_pair()functions, which use objects representing arguments and pairs of arguments, initialized respectively bydynevent_arg_init()anddynevent_arg_pair_init(). Finally, once all
args have been successfully added, the command is finalized and actually created usingcreate_dynevent().The code here for actually printing into the
dyn_event->cmdbuffer usingsnprintf()etc was adapted from v4 of Masami's 'tracing/boot:
Add synthetic event support' patch.Signed-off-by Tom Zanussi <[email protected]>Signed-off-by Masami Hiramatsu <[email protected]>--- include/linux/trace_events.h | 54 +++++++++++ kernel/trace/trace_events.c | 217 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 271 insertions(+)include/linux/trace_events.h +8
-0@@ -353,6 +353,60 @@ extern struct trace_event_file *get_event_file(const char *instance, const char *event);extern void put_event_file (struct trace_event_file *file); +struct dynevent_cmd ; + +typedef int (*dynevent_check_arg_fn_t )(void *data); +typedef int (*dynevent_create_fn_t )(struct dynevent_cmd *cmd); + +struct dynevent_arg { +const char *str; +char separator;/* e.g. ';', ',', or nothing */
include/linux/trace_events.h +13
-0+dynevent_check_arg_fn_t check_arg; +}; + +extern void dynevent_arg_init (struct dynevent_arg *arg, +dynevent_check_arg_fn_t check_arg, +char separator); +extern int add_dynevent_arg (struct dynevent_cmd *cmd, +struct dynevent_arg *arg); + +struct dynevent_arg_pair { +const char *lhs; +const char *rhs; +char operator;/* e.g. '=' or nothing */
include/linux/trace_events.h +1
-0+char separator;/* e.g. ';', ',', or nothing */
include/linux/trace_events.h +4
-0+dynevent_check_arg_fn_t check_arg; +}; + +extern void dynevent_arg_pair_init(struct dynevent_arg_pair
*arg_pair,include/linux/trace_events.h +1
-0+ dynevent_check_arg_fn_t
check_arg,include/linux/trace_events.h +3
-0+ char operator, char separator); +extern int add_dynevent_arg_pair (struct dynevent_cmd *cmd, + struct dynevent_arg_pair
*arg_pair);include/linux/trace_events.h +20
-0+ +#define MAX_DYNEVENT_CMD_LEN (2048) + +enum dynevent_type { +DYNEVENT_TYPE_SYNTH , +DYNEVENT_TYPE_KPROBE , +}; + +struct dynevent_cmd { +char *buf; +const char *event_name; +int maxlen; +int remaining; +unsigned int n_fields; +enum dynevent_type type; +dynevent_create_fn_t run_command; +void *private_data; +}; + +extern void dynevent_cmd_init(struct dynevent_cmd *cmd, char *buf,int maxlen,
include/linux/trace_events.h +4
-0+enum dynevent_type type, + dynevent_create_fn_t run_command); +extern int create_dynevent (struct dynevent_cmd *cmd); +extern int delete_synth_event (const char *name);/* kernel/trace/trace_events.c +3
-0@@ -3301,6 +3301,223 @@ void __init trace_event_init(void)event_trace_enable (); } +/** +* add_dynevent_arg - Add an arg to a dynevent_cmd +* @cmd: A pointer to the dynevent_cmd struct representing the new event cmd
+ * @arg: The argument to append to the current cmd
+ *
+ * Append an argument to adynevent_cmd. The argument string will be
+ * appended to the current cmd string, followed by a separator, if
+ * applicable. Before the argument is added, thecheck_arg()
+ * function, if defined, is called.
+ *
+ * The cmd string, separator, andcheck_arg()function should be set
+ * using thedynevent_arg_init()before any arguments are added using
+ * this function.
+ *
+ * Return: 0 if successful, error otherwise.
+ */kernel/trace/trace_events.c +16
-0+int add_dynevent_arg (struct dynevent_cmd *cmd, +struct dynevent_arg *arg) +{ +int ret =0 ; +int delta; +char *q; + +if (arg->check_arg) { + ret = arg->check_arg (arg); +if (ret) +return ret; + } + + q = cmd->buf + (cmd->maxlen - cmd->remaining); + + delta = snprintf(q, cmd->remaining, " %s%c", arg->str,
arg->separator);kernel/trace/trace_events.c +13
-0+if (delta >= cmd->remaining) { +pr_err ("String is too long: %s\n" , arg->str); +return -E2BIG ; + } + cmd->remaining -= delta; + +return ret; +} +EXPORT_SYMBOL_GPL (add_dynevent_arg); + +/** +* add_dynevent_arg_pair - Add an arg pair to a dynevent_cmd +* @cmd: A pointer to the dynevent_cmd struct representing the new event cmd
+ * @arg_pair: The argument pair to append to the current cmd
+ *
+ * Append an argument pair to adynevent_cmd. An argument pair
+ * consists of a left-hand-side argument and a right-hand-side
+ * argument separated by an operator, which can be whitespace, all
+ * followed by a separator, if applicable. This can be used to add
+ * arguments of the form 'typevariable_name;' or 'x+y'.
+ *
+ * The lhs argument string will be appended to the current cmd string,
+ * followed by an operator, if applicable, followd by the rhs string,
+ * followed finally by a separator, if applicable. Before anything is
+ * added, thecheck_arg()function, if defined, is called.
+ *
+ * The cmd strings, operator, separator, andcheck_arg()function
+ * should be set using thedynevent_arg_init()before any arguments
+ * are added using this function.
+ *
+ * Return: 0 if successful, error otherwise.
+ */kernel/trace/trace_events.c +16
-0+int add_dynevent_arg_pair (struct dynevent_cmd *cmd, +struct dynevent_arg_pair *arg_pair) +{ +int ret =0 ; +int delta; +char *q; + +if (arg_pair->check_arg) { + ret = arg_pair->check_arg (arg_pair); +if (ret) +return ret; + } + + q = cmd->buf + (cmd->maxlen - cmd->remaining); + + delta = snprintf(q, cmd->remaining, " %s%c", arg_pair-Quoted 1 line
lhs,
kernel/trace/trace_events.c +3
-0+ arg_pair->operator); +if (delta >= cmd->remaining) { + pr_err("field string is too long: %s\n", arg_pair-Quoted 1 line
lhs);
kernel/trace/trace_events.c +8
-0+return -E2BIG ; + } + cmd->remaining -= delta; q += delta; + + delta =snprintf (q, cmd->remaining,"%s%c" , arg_pair->rhs, + arg_pair->separator); +if (delta >= cmd->remaining) { + pr_err("field string is too long: %s\n", arg_pair-Quoted 1 line
rhs);
kernel/trace/trace_events.c +13
-0+return -E2BIG ; + } + cmd->remaining -= delta; q += delta; + +return ret; +} +EXPORT_SYMBOL_GPL (add_dynevent_arg_pair); + +/** +* dynevent_cmd_init - Initialize a dynevent_cmd object +* @cmd: A pointer to the dynevent_cmd struct representing the cmd +* @buf: A pointer to the buffer to generate the command into +* @maxlen: The length of the buffer the command will be generated into
+ * @type: The type of the cmd, checked against further operations
+ * @run_command: The type-specific function that will actually run the command
+ *
+ * Initialize adynevent_cmd. Adynevent_cmdis used to build up and
+ * run dynamic event creation commands, such as commands for creating
+ * synthetic and kprobe events. Before calling any of the functions
+ * used to build the command, adynevent_cmdobject should be
+ * instantiated and initialized using this function.
+ *
+ * The initialization sets things up by saving a pointer to the
+ * user-supplied buffer and its length via the @buf and @maxlen
+ * params, and by saving the cmd-specific @type and @run_command
+ * params which are used to check subsequentdynevent_cmdoperations
+ * and actually run the command when complete.
+ */
+void dynevent_cmd_init(structdynevent_cmd*cmd, char *buf, int maxlen,kernel/trace/trace_events.c +17
-0+enum dynevent_type type, + dynevent_create_fn_t run_command) +{ + memset(cmd, '\0', sizeof(*cmd)); + + cmd->buf = buf; + cmd->maxlen = maxlen; + cmd->remaining = cmd->maxlen; + cmd->type = type; + cmd->run_command = run_command; +} +EXPORT_SYMBOL_GPL (dynevent_cmd_init); + +/** +* dynevent_arg_init - Initialize a dynevent_arg object +* @arg: A pointer to the dynevent_arg struct representing the arg +* @check_arg: An (optional) pointer to a function checking arg sanity
+ * @separator: An (optional) separator, appended after adding the arg
+ *
+ * Initialize adynevent_argobject. Adynevent_argrepresents an
+ * object used to append single arguments to the current command
+ * string. The @check_arg function, if present, will be used to check
+ * the sanity of the current arg string (which is directly set by the
+ * caller). After the arg string is successfully appended to the
+ * command string, the optional @separator is appended. If no
+ * separator was specified when initializing the arg, a space will be
+ * appended.
+ */kernel/trace/trace_events.c +17
-0+void dynevent_arg_init (struct dynevent_arg *arg, +dynevent_check_arg_fn_t check_arg, +char separator) +{ +memset (arg,'\0' ,sizeof (*arg)); + +if (!separator) + separator =' ' ; + arg->separator = separator; + + arg->check_arg = check_arg; +} +EXPORT_SYMBOL_GPL (dynevent_arg_init); + +/** +* dynevent_arg_pair_init - Initialize a dynevent_arg_pair object +* @arg_pair: A pointer to the dynevent_arg_pair struct representing the arg
+ * @check_arg: An (optional) pointer to a function checking arg sanity
+ * @operator: An (optional) operator, appended after adding the first arg
+ * @separator: An (optional) separator, appended after adding the second arg
+ *
+ * Initialize adynevent_arg_pairobject. Adynevent_arg_pair
+ * represents an object used to append argument pairs such as 'type
+ *variable_name;' or 'x+y' to the current command string. An
+ * argument pair consists of a left-hand-side argument and a
+ * right-hand-side argument separated by an operator, which can be
+ * whitespace, all followed by a separator, if applicable. The
+ * @check_arg function, if present, will be used to check the sanity
+ * of the current arg strings (which is directly set by the caller).
+ * After the first arg string is successfully appended to the command
+ * string, the optional @operator is appended, followed by the second
+ * arg and and optional @separator. If no separator was specified
+ * when initializing the arg, a space will be appended.
+ */kernel/trace/trace_events.c +20
-0+void dynevent_arg_pair_init (struct dynevent_arg_pair *arg_pair, +dynevent_check_arg_fn_t check_arg, +char operator,char separator) +{ +memset (arg_pair,'\0' ,sizeof (*arg_pair)); + +if (!operator) + operator =' ' ; + arg_pair->operator = operator; + +if (!separator) + separator =' ' ; + arg_pair->separator = separator; + + arg_pair->check_arg = check_arg; +} +EXPORT_SYMBOL_GPL (dynevent_arg_pair_init); + +/** +* create_dynevent - Create the dynamic event contained in
dynevent_cmd
+ * @cmd: Thedynevent_cmdobject containing the dynamic event creation command
+ *
+ * Once adynevent_cmdobject has been successfully built up via the
+ *dynevent_cmd_init(),add_dynevent_arg()andadd_dynevent_arg_pair()
+ * functions, this function runs the final command to actually create
+ * the event.
+ *
+ * Return: 0 if the event was successfully created, error otherwise.
+ */kernel/trace/trace_events.c +6
-0+int create_dynevent (struct dynevent_cmd *cmd) +{ +return cmd->run_command (cmd); +} +EXPORT_SYMBOL_GPL (create_dynevent); +#ifdef CONFIG_EVENT_TRACE_STARTUP_TEST static DEFINE_SPINLOCK (test_spinlock);Signature · 1 line
2.14.1
PATCH v2 5/12 tracing: Add synth_event_run_command()
In reply to Tom ZanussiFrom: Masami Hiramatsu <[email protected]>
This snippet was taken from v4 of Masami's 'tracing/boot: Add synthetic event support' patch.
From the original: 'The synthetic node requires "fields" string arraies, which defines the fields as same as tracing/synth_events interface.'
synth_event_run_command() provides the means to execute the synthetic event create command using the synthetic event command string.
--- kernel/trace/trace_events_hist.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/trace/trace_events_hist.c +5 -0
@@ -1390,6 +1390,11 @@ static int create_or_delete_synth_event(int argc, char **argv)return ret == -ECANCELED ? -EINVAL : ret; } +int synth_event_run_command (const char *command) +{ +return trace_run_command (command, create_or_delete_synth_event); +} +static int synth_event_create (int argc,const char **argv) {const char *name = argv[0 ];
Signature · 1 line
2.14.1
Steven Rostedt
maintainerRe: [PATCH v2 05/12] tracing: Add synth_event_run_command()
Quoted 6 lines · On Fri, 10 Jan 2020 14:35:11 -0600 Tom Zanussi <[email protected]> wrote:
From: Masami Hiramatsu <[email protected]>
This snippet was taken from v4 of Masami's 'tracing/boot: Add synthetic event support' patch.
Quoted 1 line
From the original: 'The synthetic node requires "fields" string
Quoted 8 lines
arraies, which defines the fields as same as tracing/synth_events interface.'
synth_event_run_command()provides the means to execute the synthetic event create command using the synthetic event command string.Signed-off-by Masami Hiramatsu <[email protected]>Signed-off-by Tom Zanussi <[email protected]>
If you base your next series off of my for-next branch, you wont need this patch.
-- Steve
Quoted 20 lines
--- kernel/trace/trace_events_hist.c | 5 +++++ 1 file changed, 5 insertions(+)kernel/trace/trace_events_hist.c +5
-0@@ -1390,6 +1390,11 @@ static int create_or_delete_synth_event(int argc, char **argv)return ret == -ECANCELED ? -EINVAL : ret; } +int synth_event_run_command (const char *command) +{ +return trace_run_command (command, create_or_delete_synth_event); +} +static int synth_event_create (int argc,const char **argv) {const char *name = argv[0 ];
PATCH v2 6/12 tracing: Add synthetic event command generation functions
In reply to Tom ZanussiAdd functions used to generate synthetic event commands, built on top of the dynevent_cmd interface.
gen_synth_cmd() is used to create a synthetic event command using a variable arg list and gen_synth_cmd_array() does the same thing but using an array of field descriptors. add_synth_field() and add_synth_fields() can be used to add single fields one by one or as a group. Once all desired fields are added, create_dynevent() is used to actually execute the command and create the event.
create_synth_event() does everything, including creating the event, in a single call.
--- include/linux/trace_events.h | 30 ++++ kernel/trace/trace_events_hist.c | 325 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 352 insertions(+), 3 deletions(-)
include/linux/trace_events.h +30 -0
@@ -409,6 +409,36 @@ extern int create_dynevent(struct dynevent_cmd *cmd);extern int delete_synth_event (const char *name); +extern void synth_dynevent_cmd_init (struct dynevent_cmd *cmd, +char *buf,int maxlen); + +extern int __gen_synth_cmd (struct dynevent_cmd *cmd,const char *name, +struct module *mod, ...); + +#define gen_synth_cmd (cmd, name, mod, ...) \ + __gen_synth_cmd(cmd, name, mod, ## __VA_ARGS__, NULL) + +struct synth_field_desc { +const char *type; +const char *name; +}; + +extern int gen_synth_cmd_array (struct dynevent_cmd *cmd,const char *name, +struct module *mod, +struct synth_field_desc *fields, +unsigned int n_fields); +extern int create_synth_event (const char *name, +struct synth_field_desc *fields, +unsigned int n_fields,struct module *mod); + + +extern int add_synth_field (struct dynevent_cmd *cmd, +const char *type, +const char *name); +extern int add_synth_fields (struct dynevent_cmd *cmd, +struct synth_field_desc *fields, +unsigned int n_fields); +/* * Event file flags: * ENABLED - The event is enabled
kernel/trace/trace_events_hist.c +322 -3
364 lines not on this page · raw message
Signature · 1 line
2.14.1
Steven Rostedt
maintainerRe: [PATCH v2 06/12] tracing: Add synthetic event command generation functions
Quoted 46 lines · On Fri, 10 Jan 2020 14:35:12 -0600 Tom Zanussi <[email protected]> wrote:
Add functions used to generate synthetic event commands, built on top of the
dynevent_cmdinterface.
gen_synth_cmd()is used to create a synthetic event command using a variable arg list andgen_synth_cmd_array()does the same thing but using an array of field descriptors.add_synth_field()andadd_synth_fields()can be used to add single fields one by one or as a group. Once all desired fields are added,create_dynevent()is used to actually execute the command and create the event.
create_synth_event()does everything, including creating the event, in a single call.Signed-off-by Tom Zanussi <[email protected]>--- include/linux/trace_events.h | 30 ++++ kernel/trace/trace_events_hist.c | 325 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 352 insertions(+), 3 deletions(-)include/linux/trace_events.h +19
-0@@ -409,6 +409,36 @@ extern int create_dynevent(struct dynevent_cmd *cmd);extern int delete_synth_event (const char *name); +extern void synth_dynevent_cmd_init (struct dynevent_cmd *cmd, +char *buf,int maxlen); + +extern int __gen_synth_cmd (struct dynevent_cmd *cmd,const char *name, +struct module *mod, ...); + +#define gen_synth_cmd (cmd, name, mod, ...) \ + __gen_synth_cmd(cmd, name, mod, ## __VA_ARGS__, NULL) + +struct synth_field_desc { +const char *type; +const char *name; +}; + +extern int gen_synth_cmd_array (struct dynevent_cmd *cmd,const char *name, +struct module *mod, +struct synth_field_desc *fields, +unsigned int n_fields); +extern int create_synth_event(const char *name,
include/linux/trace_events.h extern int create_dynevent(struct dynevent_cmd *cmd); · line 440
Quoted 10 lines
include/linux/trace_events.h +10
-0+struct synth_field_desc *fields, + unsigned int n_fields, struct module *mod); + + +extern int add_synth_field (struct dynevent_cmd *cmd, +const char *type, +const char *name); +extern int add_synth_fields (struct dynevent_cmd *cmd, +struct synth_field_desc *fields, +unsigned int n_fields);
As these are in a global header and globally visible, let's rename them to be more name space aware.
Have them all start with "synth_event_".
synth_event_gen_cmd_array()synth_event_create()synth_event_add_field()synth_event_add_fields()
Makes it easier to grep for synth_event functions too.
-- Steve
Quoted 4 lines
include/linux/trace_events.h +1
-0+/* * Event file flags: * ENABLED - The event is enabled
Re: [PATCH v2 06/12] tracing: Add synthetic event command generation functions
Hi Steve,
Quoted 74 lines · Steven Rostedt
Quoted 58 lines · On Fri, 10 Jan 2020 14:35:12 -0600 Tom Zanussi <[email protected]> wrote:
Add functions used to generate synthetic event commands, built on top
of thedynevent_cmdinterface.
gen_synth_cmd()is used to create a synthetic event command using a variable arg list andgen_synth_cmd_array()does the same thing but using an array of field descriptors.add_synth_field()andadd_synth_fields()can be used to add single fields one by one or as a
group. Once all desired fields are added,create_dynevent()is used
to actually execute the command and create the event.
create_synth_event()does everything, including creating the event, in
a single call.Signed-off-by Tom Zanussi <[email protected]>--- include/linux/trace_events.h | 30 ++++ kernel/trace/trace_events_hist.c | 325 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 352 insertions(+), 3 deletions(-)include/linux/trace_events.h +21
-0@@ -409,6 +409,36 @@ extern int create_dynevent(struct dynevent_cmd *cmd);extern int delete_synth_event (const char *name); +extern void synth_dynevent_cmd_init (struct dynevent_cmd *cmd, +char *buf,int maxlen); + +extern int __gen_synth_cmd (struct dynevent_cmd *cmd,const char *name, +struct module *mod, ...); + +#define gen_synth_cmd (cmd, name, mod, ...) \ + __gen_synth_cmd(cmd, name, mod, ## __VA_ARGS__, NULL) + +struct synth_field_desc { +const char *type; +const char *name; +}; + +extern int gen_synth_cmd_array (struct dynevent_cmd *cmd,const char *name, +struct module *mod, +struct synth_field_desc *fields, +unsigned int n_fields); +extern int create_synth_event (const char *name, +struct synth_field_desc *fields, + unsigned int n_fields, struct module*mod);
include/linux/trace_events.h extern int create_dynevent(struct dynevent_cmd *cmd); · line 440
Quoted 8 lines
include/linux/trace_events.h +8
-0+ + +extern int add_synth_field (struct dynevent_cmd *cmd, +const char *type, +const char *name); +extern int add_synth_fields (struct dynevent_cmd *cmd, +struct synth_field_desc *fields, +unsigned int n_fields);As these are in a global header and globally visible, let's rename them
to be more name space aware.
Quoted 9 lines
Have them all start with "
synth_event_".
synth_event_gen_cmd_array()synth_event_create()synth_event_add_field()synth_event_add_fields()Makes it easier to grep for
synth_eventfunctions too.
OK, will do.
Thanks,
Tom
Quoted 7 lines
-- Steve
Quoted 4 lines
+
/*
* Event file flags:
* ENABLED - The event is enabled
PATCH v2 7/12 tracing: Add trace_synth_event() and related functions
In reply to Tom ZanussiAdd an exported function named trace_synth_event(), allowing modules or other kernel code to generate synthetic events.
Also added are several functions that allow the same functionality to be broken out in a piecewise fashion, which are useful in situations where generating an event from a full array of values would be cumbersome. Those functions are trace_synth_event_start/end() and add_(next)_synth_val().
--- include/linux/trace_events.h | 25 +++ kernel/trace/trace_events_hist.c | 461 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 486 insertions(+)
include/linux/trace_events.h +25 -0
@@ -439,6 +439,31 @@ extern int add_synth_fields(struct dynevent_cmd *cmd,struct synth_field_desc *fields, unsigned int n_fields); +struct synth_event ; + +struct synth_gen_state { +struct trace_event_buffer fbuffer; +struct synth_trace_event *entry; +struct ring_buffer *buffer; +struct synth_event *event; +unsigned int cur_field; +unsigned int n_u64; +bool enabled; +bool add_next; +bool add_name; +}; + +extern int trace_synth_event (struct trace_event_file *file, +unsigned int n_vals, ...); +extern int trace_synth_event_array (struct trace_event_file *file,u64 *vals, +unsigned int n_vals); +extern int trace_synth_event_start (struct trace_event_file *file, +struct synth_gen_state *gen_state); +extern int add_next_synth_val (u64 val,struct synth_gen_state *gen_state); +extern int add_synth_val (const char *field_name,u64 val, +struct synth_gen_state *gen_state); +extern int trace_synth_event_end (struct synth_gen_state *gen_state); +/* * Event file flags: * ENABLED - The event is enabled
kernel/trace/trace_events_hist.c +461 -0
479 lines not on this page · raw message
Signature · 1 line
2.14.1
Steven Rostedt
maintainerRe: [PATCH v2 07/12] tracing: Add trace_synth_event() and related functions
Quoted 4 lines · On Fri, 10 Jan 2020 14:35:13 -0600 Tom Zanussi <[email protected]> wrote:
include/linux/trace_events.h +4
-0+struct synth_event ; + +struct synth_gen_state { +struct trace_event_buffer fbuffer;
include/linux/trace_events.h extern int add_synth_fields(struct dynevent_cmd *cmd, · line 455
Quoted 10 lines
include/linux/trace_events.h +10
-0+struct synth_trace_event *entry; +struct ring_buffer *buffer; +struct synth_event *event; +unsigned int cur_field; +unsigned int n_u64; +bool enabled; +bool add_next; +bool add_name; +}; +
Yes, please rebase on my for-next branch, as the ring_buffer structure has been renamed, and will break these patches :-/
-- Steve
Re: [PATCH v2 07/12] tracing: Add trace_synth_event() and related functions
Hi Steve,
Quoted 13 lines · Steven Rostedt
Quoted 10 lines · On Fri, 10 Jan 2020 14:35:13 -0600 Tom Zanussi <[email protected]> wrote:
include/linux/trace_events.h +10
-0+struct synth_event ; + +struct synth_gen_state { +struct trace_event_buffer fbuffer; +struct synth_trace_event *entry; +struct ring_buffer *buffer; +struct synth_event *event; +unsigned int cur_field; +unsigned int n_u64; +bool enabled;
Quoted 9 lines
include/linux/trace_events.h extern int add_synth_fields(struct dynevent_cmd *cmd, · line 455
Quoted 4 lines
include/linux/trace_events.h +4
-0+bool add_next; +bool add_name; +}; +Yes, please rebase on my for-next branch, as the
ring_bufferstructure
has been renamed, and will break these patches :-/
OK, I'll rebase on for-next, and will also adapt a couple of Masami's boot trace patches to these.
Thanks,
Tom
Quoted 1 line
-- Steve
PATCH v2 8/12 tracing: Add synth event generation test module
In reply to Tom ZanussiAdd a test module that checks the basic functionality of the in-kernel synthetic event generation API by creating and generating synthetic events from a module.
--- kernel/trace/Kconfig | 13 + kernel/trace/Makefile | 1 + kernel/trace/synth_event_gen_test.c | 519 ++++++++++++++++++++++++++++++++++++ 3 files changed, 533 insertions(+) create mode 100644 kernel/trace/synth_event_gen_test.c
kernel/trace/Kconfig +13 -0
@@ -775,6 +775,19 @@ config PREEMPTIRQ_DELAY_TEST If unsure, say N +config SYNTH_EVENT_GEN_TEST + tristate "Test module for in-kernel synthetic event generation" + depends on HIST_TRIGGERS + help + This option creates a test module to check the base + functionality of in-kernel synthetic event definition and + generation. + + To test, insert the module, and then check the trace buffer + for the generated sample events. + + If unsure, say N. + config TRACE_EVAL_MAP_FILE bool "Show eval mappings for trace events" depends on TRACING
kernel/trace/Makefile +1 -0
@@ -44,6 +44,7 @@ obj-$(CONFIG_TRACING) += trace_stat.o obj-$(CONFIG_TRACING) += trace_printk.o obj-$(CONFIG_TRACING_MAP) += tracing_map.o obj-$(CONFIG_PREEMPTIRQ_DELAY_TEST) += preemptirq_delay_test.o +obj-$(CONFIG_SYNTH_EVENT_GEN_TEST) += synth_event_gen_test.o obj-$(CONFIG_CONTEXT_SWITCH_TRACER) += trace_sched_switch.o obj-$(CONFIG_FUNCTION_TRACER) += trace_functions.o obj-$(CONFIG_PREEMPTIRQ_TRACEPOINTS) += trace_preemptirq.o
kernel/trace/synth_event_gen_test.c new file +519 -0
519 lines not on this page · raw message
Signature · 1 line
2.14.1
PATCH v2 9/12 tracing: Add trace_kprobe_run_command()
In reply to Tom ZanussiFrom: Masami Hiramatsu <[email protected]>
This snippet was taken from v4 of Masami's 'tracing/boot: Add kprobe event support' patch.
trace_kprobe_run_command() provides the means to execute the kprobe event create command using the kprobe event command string, from kernel code.
--- kernel/trace/trace_kprobe.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/trace/trace_kprobe.c +5 -0
@@ -902,6 +902,11 @@ static int create_or_delete_trace_kprobe(int argc, char **argv)return ret == -ECANCELED ? -EINVAL : ret; } +int trace_kprobe_run_command (const char *command) +{ +return trace_run_command (command, create_or_delete_trace_kprobe); +} +static int trace_kprobe_release (struct dyn_event *ev) {struct trace_kprobe *tk =to_trace_kprobe (ev);
Signature · 1 line
2.14.1
PATCH v2 10/12 tracing: Add kprobe event command generation functions
In reply to Tom ZanussiAdd functions used to generate kprobe event commands, built on top of the dynevent_cmd interface.
gen_kprobe_cmd() is used to create a kprobe event command using a variable arg list. add_probe_fields() can be used to add single fields one by one or as a group. Once all desired fields are added, create_dynevent() is used to actually execute the command and create the event.
gen_kprobe_cmd() can be used to create kretprobe commands as well.
--- include/linux/trace_events.h | 18 +++++ kernel/trace/trace_kprobe.c | 161 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 177 insertions(+), 2 deletions(-)
include/linux/trace_events.h +18 -0
@@ -464,6 +464,24 @@ extern int add_synth_val(const char *field_name, u64 val, struct synth_gen_state *gen_state);extern int trace_synth_event_end (struct synth_gen_state *gen_state); +extern void kprobe_dynevent_cmd_init (struct dynevent_cmd *cmd, +char *buf,int maxlen); + +#define gen_kprobe_cmd (cmd, name, loc, ...) \ + __gen_kprobe_cmd(cmd, name, loc, false, ## __VA_ARGS__, NULL) + +#define gen_kretprobe_cmd (cmd, name, loc, ...) \ + __gen_kprobe_cmd(cmd, name, loc, true, ## __VA_ARGS__, NULL) + +extern int __gen_kprobe_cmd (struct dynevent_cmd *cmd,const char *name, +const char *loc,bool kretprobe, ...); + +#define add_probe_fields (cmd, ...) \ + __add_probe_fields(cmd, ## __VA_ARGS__, NULL) + +extern int __add_probe_fields (struct dynevent_cmd *cmd, ...); +extern int delete_kprobe_event (const char *name); +/* * Event file flags: * ENABLED - The event is enabled
kernel/trace/trace_kprobe.c +159 -2
170 lines not on this page · raw message
Signature · 1 line
2.14.1
PATCH v2 11/12 tracing: Add kprobe event command generation test module
In reply to Tom ZanussiAdd a test module that checks the basic functionality of the in-kernel kprobe event command generation API by creating kprobe events from a module.
--- kernel/trace/Kconfig | 12 ++ kernel/trace/Makefile | 1 + kernel/trace/kprobe_event_gen_test.c | 223 +++++++++++++++++++++++++++++++++++ 3 files changed, 236 insertions(+) create mode 100644 kernel/trace/kprobe_event_gen_test.c
kernel/trace/Kconfig +12 -0
@@ -788,6 +788,18 @@ config SYNTH_EVENT_GEN_TEST If unsure, say N. +config KPROBE_EVENT_GEN_TEST + tristate "Test module for in-kernel kprobe event generation" + depends on KPROBE_EVENTS + help + This option creates a test module to check the base + functionality of in-kernel kprobe event definition. + + To test, insert the module, and then check the trace buffer + for the generated kprobe events. + + If unsure, say N. + config TRACE_EVAL_MAP_FILE bool "Show eval mappings for trace events" depends on TRACING
kernel/trace/Makefile +1 -0
@@ -45,6 +45,7 @@ obj-$(CONFIG_TRACING) += trace_printk.o obj-$(CONFIG_TRACING_MAP) += tracing_map.o obj-$(CONFIG_PREEMPTIRQ_DELAY_TEST) += preemptirq_delay_test.o obj-$(CONFIG_SYNTH_EVENT_GEN_TEST) += synth_event_gen_test.o +obj-$(CONFIG_KPROBE_EVENT_GEN_TEST) += kprobe_event_gen_test.o obj-$(CONFIG_CONTEXT_SWITCH_TRACER) += trace_sched_switch.o obj-$(CONFIG_FUNCTION_TRACER) += trace_functions.o obj-$(CONFIG_PREEMPTIRQ_TRACEPOINTS) += trace_preemptirq.o
kernel/trace/kprobe_event_gen_test.c new file +223 -0
223 lines not on this page · raw message
Signature · 1 line
2.14.1
PATCH v2 12/12 tracing: Documentation for in-kernel synthetic event API
In reply to Tom ZanussiAdd Documentation for creating and generating synthetic events from modules.
--- Documentation/trace/events.rst | 488 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 488 insertions(+)
Documentation/trace/events.rst +488 -0
@@ -525,3 +525,491 @@ The following commands are supported: event counts (hitcount). See Documentation/trace/histogram.rst for details and examples. + +6.3 In-kernel trace event API +----------------------------- + +In most cases, the command-line interface to trace events is more than +sufficient. Sometimes, however, applications might find the need for +more complex relationships than can be expressed through a simple +series of linked command-line expressions, or putting together sets of +commands may be simply too cumbersome. An example might be an +application that needs to 'listen' to the trace stream in order to +maintain an in-kernel state machine detecting, for instance, when an +illegal kernel state occurs in the scheduler. + +The trace event subsystem provides an in-kernel API allowing modules +or other kernel code to generate user-defined 'synthetic' events at +will, which can be used to either augment the existing trace stream +and/or signal that a particular important state has occurred. + +A similar in-kernel API is also available for creating kprobe and +kretprobe events. + +Both the synthetic event and k/ret/probe event APIs are built on top +of a lower-level "dynevent_cmd" event command API, which is also +available for more specialized applications, or as the basis of other +higher-level trace event APIs. + +The API provided for these purposes is describe below and allows the +following: + + - dynamically creating synthetic event definitions + - dynamically creating kprobe and kretprobe event definitions + - tracing synthetic events from in-kernel code + - the low-level "dynevent_cmd" API + +6.3.1 Dyamically creating synthetic event definitions +----------------------------------------------------- + +There are a couple ways to create a new synthetic event from a kernel +module or other kernel code. + +The first creates the event in one step, using create_synth_event(). +In this method, the name of the event to create and an array defining +the fields is supplied to create_synth_event(). If successful, a +synthetic event with that name and fields will exist following that +call. For example, to create a new "schedtest" synthetic event: + + ret = create_synth_event("schedtest", sched_fields, + ARRAY_SIZE(sched_fields), THIS_MODULE); + +The sched_fields param in this example points to an array of struct +synth_field_desc, each of which describes an event field by type and +name: + + static struct synth_field_desc sched_fields[] = { + { .type = "pid_t", .name = "next_pid_field" }, + { .type = "char[16]", .name = "next_comm_field" }, + { .type = "u64", .name = "ts_ns" }, + { .type = "u64", .name = "ts_ms" }, + { .type = "unsigned int", .name = "cpu" }, + { .type = "char[64]", .name = "my_string_field" }, + { .type = "int", .name = "my_int_field" }, + }; + +See synth_field_size() for available types. If field_name contains [n] +the field is considered to be an array. + +If the event is created from within a module, a pointer to the module +must be passed to create_synth_event(). This will ensure that the +trace buffer won't contain unreadable events when the module is +removed. + +At this point, the event object is ready to be used for generating new +events. + +In the second method, the event is created in several steps. This +allows events to be created dynamically and without the need to create +and populate an array of fields beforehand. + +To use this method, an empty or partially empty synthetic event should +first be created using gen_synth_cmd(). The name of the event along +with one or more pairs of args each pair representing a 'type +field_name;' field specification should be supplied to this function. +Before calling gen_synth_cmd(), the user should create and initialize +a dynevent_cmd object using synth_dynevent_cmd_init(). + +For example, to create a new "schedtest" synthetic event with two +fields: + + struct dynevent_cmd cmd; + char *buf; + + /* Create a buffer to hold the generated command */ + buf = kzalloc(MAX_DYNEVENT_CMD_LEN, GFP_KERNEL); + + /* Before generating the command, initialize the cmd object */ + synth_dynevent_cmd_init(&cmd, buf, MAX_DYNEVENT_CMD_LEN); + + ret = gen_synth_cmd(&cmd, "schedtest", THIS_MODULE, + "pid_t", "next_pid_field", + "u64", "ts_ns"); + +Once the synthetic event object has been created, it can then be +populated with more fields. Fields are added one by one using +add_synth_field(), supplying the dynevent_cmd object, a field type, +and a field name. For example, to add a new int field named +"intfield", the following call should be made: + + ret = add_synth_field(&cmd, "int", "intfield"); + +See synth_field_size() for available types. If field_name contains [n] +the field is considered to be an array. + +A group of fields can also be added all at once using an array of +synth_field_desc with add_synth_fields(). For example, this would add +just the first four sched_fields: + + ret = add_synth_fields(&cmd, sched_fields, 4); + +Once all the fields have been added, the event should be finalized and +registered by calling the create_dynevent() function: + + ret = create_dynevent(&cmd); + +At this point, the event object is ready to be used for tracing new +events. + +6.3.3 Tracing synthetic events from in-kernel code +-------------------------------------------------- + +To trace a synthetic event, there are several options. The first +option is to trace the event in one call, using trace_synth_event() +with a variable number of values, or trace_synth_event_array() with an +array of values to be set. A second option can be used to avoid the +need for a pre-formed array of values or list of arguments, via +trace_synth_event_start() and trace_synth_event_end() along with +add_next_synth_val() or add_synth_val() to add the values piecewise. + +6.3.3.1 Tracing a synthetic event all at once +--------------------------------------------- + +To trace a synthetic event all at once, the trace_synth_event() or +trace_synth_event_array() functions can be used. + +The trace_synth_event() function is passed the trace_event_file +representing the synthetic event (which can be retrieved using +get_event_file() using the synthetic event name, "synthetic" as the +system name, and the trace instance name (NULL if using the global +trace array)), along with an variable number of u64 args, one for each +synthetic event field, and the number of values being passed. + +So, to generate an event corresponding to the synthetic event +definition above, code like the following could be used: + + ret = trace_synth_event(create_synth_test, 7, /* number of values */ + 444, /* next_pid_field */ + (u64)"clackers", /* next_comm_field */ + 1000000, /* ts_ns */ + 1000, /* ts_ms */ + smp_processor_id(),/* cpu */ + (u64)"Thneed", /* my_string_field */ + 999); /* my_int_field */ + +All vals should be cast to u64, and string vals are just pointers to +strings, cast to u64. Strings will be copied into space reserved in +the event for the string, using these pointers. + +Alternatively, the trace_synth_event_array() function can be used to +accomplish the same thing. It is passed the trace_event_file +representing the synthetic event (which can be retrieved using +get_event_file() using the synthetic event name, "synthetic" as the +system name, and the trace instance name (NULL if using the global +trace array)), along with an array of u64, one for each synthetic +event field. + +Tto generate an event corresponding to the synthetic event definition +above, code like the following could be used: + + u64 vals[7]; + + vals[0] = 777; /* next_pid_field */ + vals[1] = (u64)"tiddlywinks"; /* next_comm_field */ + vals[2] = 1000000; /* ts_ns */ + vals[3] = 1000; /* ts_ms */ + vals[4] = smp_processor_id(); /* cpu */ + vals[5] = (u64)"thneed"; /* my_string_field */ + vals[6] = 398; /* my_int_field */ + +The 'vals' array is just an array of u64, the number of which must +match the number of field in the synthetic event, and which must be in +the same order as the synthetic event fields. + +All vals should be cast to u64, and string vals are just pointers to +strings, cast to u64. Strings will be copied into space reserved in +the event for the string, using these pointers. + +In order to generate a synthetic event, a pointer to the trace event +file is needed. The get_event_file() function can be used to get it - +it will find the file in the given trace instance (in this case NULL +since the top trace array is being used) while at the same time +preventing the instance containing it from going away: + + schedtest_event_file = get_event_file(NULL, "synthetic", + "schedtest"); + +Before generating the event, it should be enabled in some way, +otherwise the synthetic event won't actually show up in the trace +buffer. + +To enable a synthetic event from the kernel, trace_array_set_clr_event() +can be used (which is not specific to synthetic events, so does need +the "synthetic" system name to be specified explicitly). + +To enable the event, pass 'true' to it: + + trace_array_set_clr_event(schedtest_event_file->tr, + "synthetic", "schedtest", true); + +To disable it pass false: + + trace_array_set_clr_event(schedtest_event_file->tr, + "synthetic", "schedtest", false); + +Finally, trace_synth_event_array() can be used to actually trace the +event, which should be visible in the trace buffer afterwards: + + ret = trace_synth_event_array(schedtest_event_file, vals, + ARRAY_SIZE(vals)); + +To remove the synthetic event, the event should be disabled, and the +trace instance should be 'put' back using put_event_file(): + + trace_array_set_clr_event(schedtest_event_file->tr, + "synthetic", "schedtest", false); + put_event_file(schedtest_event_file); + +If those have been successful, delete_synth_event() can be called to +remove the event: + + ret = delete_synth_event("schedtest"); + +6.3.3.1 Tracing a synthetic event piecewise +------------------------------------------- + +To trace a synthetic using the piecewise method described above, the +trace_synth_event_start() function is used to 'open' the synthetic +event trace: + + struct synth_gen_state gen_state; + + ret = trace_synth_event_start(schedtest_event_file, &gen_state); + +It's passed the trace_event_file representing the synthetic event +using the same methods as described above, along with a pointer to a +struct synth_gen_state object, which will be zeroed before use and +used to maintain state between this and following calls. + +Once the event has been opened, which means space for it has been +reserved in the trace buffer, the individual fields can be set. There +are two ways to do that, either one after another for each field in +the event, which requires no lookups, or by name, which does. The +tradeoff is flexibility in doing the assignments vs the cost of a +lookup per field. + +To assign the values one after the other without lookups, +add_next_synth_val() should be used. Each call is passed the same +synth_gen_state object used in the trace_synth_event_start(), along +with the value to set the next field in the event. After each field +is set, the 'cursor' points to the next field, which will be set by +the subsequent call, continuing until all the fields have been set in +order. The same sequence of calls as in the above examples using this +method would be (without error-handling code): + + /* next_pid_field */ + ret = add_next_synth_val(777, &gen_state); + + /* next_comm_field */ + ret = add_next_synth_val((u64)"slinky", &gen_state); + + /* ts_ns */ + ret = add_next_synth_val(1000000, &gen_state); + + /* ts_ms */ + ret = add_next_synth_val(1000, &gen_state); + + /* cpu */ + ret = add_next_synth_val(smp_processor_id(), &gen_state); + + /* my_string_field */ + ret = add_next_synth_val((u64)"thneed_2.01", &gen_state); + + /* my_int_field */ + ret = add_next_synth_val(395, &gen_state); + +To assign the values in any order, add_synth_val() should be used. +Each call is passed the same synth_gen_state object used in the +generate_synth_event_start(), along with the field name of the field +to set and the value to set it to. The same sequence of calls as in +the above examples using this method would be (without error-handling +code): + + ret = add_synth_val("next_pid_field", 777, &gen_state); + ret = add_synth_val("next_comm_field", (u64)"silly putty", &gen_state); + ret = add_synth_val("ts_ns", 1000000, &gen_state); + ret = add_synth_val("ts_ms", 1000, &gen_state); + ret = add_synth_val("cpu", smp_processor_id(), &gen_state); + ret = add_synth_val("my_string_field", (u64)"thneed_9", &gen_state); + ret = add_synth_val("my_int_field", 3999, &gen_state); + +Note that add_next_synth_val() and add_synth_val() are incompatible if +used within the same generation of an event - either one can be used +but not both at the same time. + +Finally, the event won't be actually traced until it's 'closed', +which is done using trace_synth_event_end(), which takes only the +struct synth_gen_state object used in the previous calls: + + ret = trace_synth_event_end(&gen_state); + +Note that trace_synth_event_end() must be called at the end regardless +of whether any of the add calls failed (say due to a bad field name +being passed in). + +6.3.4 Dyamically creating kprobe and kretprobe event definitions +---------------------------------------------------------------- + +To create a kprobe or kretprobe trace event from kernel code, the +gen_kprobe_cmd() or gen_kretprobe() functions can be used. + +To create a kprobe event, an empty or partially empty kprobe event +should first be created using gen_kprobe_cmd(). The name of the event +and the probe location should be specfied along with one or args each +representing a probe field should be supplied to this function. +Before calling gen_kprobe_cmd(), the user should create and initialize +a dynevent_cmd object using kprobe_dynevent_cmd_init(). + +For example, to create a new "schedtest" synthetic event with two +fields: + + struct dynevent_cmd cmd; + char *buf; + + /* Create a buffer to hold the generated command */ + buf = kzalloc(MAX_DYNEVENT_CMD_LEN, GFP_KERNEL); + + /* Before generating the command, initialize the cmd object */ + kpobe_dynevent_cmd_init(&cmd, buf, MAX_DYNEVENT_CMD_LEN); + + /* + * Define the gen_kprobe_test event with the first 2 kprobe + * fields. + */ + ret = gen_kprobe_cmd(&cmd, "gen_kprobe_test", "do_sys_open", + "dfd=%ax", "filename=%dx"); + +Once the kprobe event object has been created, it can then be +populated with more fields. Fields can be added using +add_probe_fields(), supplying the dynevent_cmd object along with a +variable arg list of probe fields. For example, to add a couple +additional fields, the following call could be made: + + ret = add_probe_fields(&cmd, "flags=%cx", "mode=+4($stack)"); + +Once all the fields have been added, the event should be finalized and +registered by calling the create_dynevent() function: + + ret = create_dynevent(&cmd); + +At this point, the event object is ready to be used for tracing new +events. + +Similarly, a kretprobe event can be created using gen_kretprobe_cmd() +with a probe name and location and additional params such as $retval: + + ret = gen_kretprobe_cmd(&cmd, "gen_kretprobe_test", "do_sys_open", + "$retval"); + +Similar to the synthetic event case, code like the following can be +used to enable the newly created kprobe event: + + gen_kprobe_test = get_event_file(NULL, "kprobes", "gen_kprobe_test"); + + ret = trace_array_set_clr_event(gen_kprobe_test->tr, + "kprobes", "gen_kprobe_test", true); + +Finally, also similar to synthetic events, the following code can be +used to give the kprobe event file back and delete the event: + + put_event_file(gen_kprobe_test); + +ret = delete_kprobe_event("gen_kprobe_test"); + +6.3.4 The "dynevent_cmd" low-level API +-------------------------------------- + +Both the in-kernel synthetic event and kprobe interfaces are built on +top of a lower-level "dynevent_cmd" interface. This interface can +also be used on its own by modules and other kernel code, but it's +generally meant to be used as the basis for higher-level interfaces +such as the synthetic and kprobe interfaces, which can be used as +examples. + +The basic idea is simple - it's meant to be used to generate trace +event commands that can then be passed to the command-parsing and +event creation code that already exists to create the corresponding +trace events. + +In a nutshell, the way it works is that the user creates a struct +dynevent_cmd object, then uses a couple functions, add_dynevent_arg() +and add_dynevent_arg_pair() to build up a command string, then finally +causes the command to be executed using the create_dynevent() +function. The details of the interface are described below. + +The first step in building a new command string is to create and +initialize an instance of a dynevent_cmd. Here, for instance, we +create a dynevent_cmd on the stack and initialize it: + + struct dynevent_cmd cmd; + char *buf; + int ret; + + buf = kzalloc(MAX_DYNEVENT_CMD_LEN, GFP_KERNEL); + + dynevent_cmd_init(cmd, buf, maxlen, DYNEVENT_TYPE_FOO, + foo_event_run_command); + +The dynevent_cmd initialization needs to be given a user-specified +buffer and the length of the buffer (MAX_DYNEVENT_CMD_LEN can be used +for this purpose - at 2k it's generally too big to be comfortably put +on the stack, so is dynamically allocated), a dynevent type id, which +is meant to be used to check that further API calls are for the +correct command type, and a pointer to an event-specific run_command() +callback that will be called to actually execute the event-specific +command function. + +Once that's done, the command string can by built up by successive +calls to argument-adding functions. + +To add a single argument, define and initialize a struct dynevent_arg +or struct dynevent_arg_pair object. Here's an example of the simplest +possible arg addition, which is simply to append the given string as +a whitespace-separated argument to the command: + + struct dynevent_arg arg; + + dynevent_arg_init(&arg, NULL, 0); + + arg.str = name; + + ret = add_dynevent_arg(cmd, &arg); + +The arg object is first initialized using dynevent_arg_init() and in +this case the parameters are NULL or 0, which means there's no +optional sanity-checking function or separator appended to the end of +the arg. + +Here's another more complicated example using an 'arg pair', which is +used to create an argument that consists of a couple components added +together as a unit, for example, a 'type field_name;' arg or a simple +expression arg e.g. 'flags=%cx': + + struct dynevent_arg_pair arg_pair; + + dynevent_arg_pair_init(&arg_pair, dynevent_foo_check_arg_fn, 0, ';'); + + arg_pair.lhs = type; + arg_pair.rhs = name; + + ret = add_dynevent_arg_pair(cmd, &arg_pair); + +Again, the arg_pair is first initialized, in this case with a callback +function used to check the sanity of the args (for example, that +neither part of the pair is NULL), along with a character to be used +to add an operator between the pair (here none) and a separator to be +appended onto the end of the arg pair (here ';'). + +Any number of dynevent_add_* calls can be made to build up the string +(until its length surpasses cmd->maxlen). When all the arguments have +been added and the command string is complete, the only thing left to +do is run the command, which happens by simply calling +create_dynevent(): + + ret = create_dynevent(&cmd); + +At that point, if the return value is 0, the dynamic event has been +created and is ready to use. + +See the dynevent_cmd function definitions themselves for the details +of the API.
Signature · 1 line
2.14.1