tracing: Updates for 6.2
Subsystems (7)
Steven Rostedt
maintainercover tracing: Updates for 6.2
Tracing updates for 6.2:
- Fix filter memory leak by calling ftrace_free_filter()
- Initialize trace_printk() earlier so that ftrace_dump_on_oops shows data on early crashs.
- Update the outdated instructions in scripts/tracing/ftrace-bisect.sh
- Add lockdep_is_held() to fix lockdep warning
- Add allocation failure check in create_hist_field()
- Don't initalize pointer that gets set right away in enabled_monitors_write()
- Update MAINTAINER entries
- Fix help messages in Kconfigs
- Fix kernel-doc header for update_preds()
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
ftrace/urgent
Head SHA1: 78020233418518faa72fba11f40e1d53b9e88a2e
Chuang Wang (1):
tracing/osnoise: Use built-in RCU list checking
Colin Ian King (1):
rv: remove redundant initialization of pointer ptr
Mark Rutland (2):
ftrace: Export ftrace_free_filter() to modules
ftrace: Maintain samples/ftrace
Masami Hiramatsu (Google) (1):
bootconfig: Update MAINTAINERS file to add tree and mailing list
Natalia Petrova (1):
trace_events_hist: add check for return value of 'create_hist_field'
Randy Dunlap (3):
tracing: Kconfig: Fix spelling/grammar/punctuation
lib: Kconfig: fix spellos
tracing/filter: fix kernel-doc warnings
Steven Rostedt (Google) (2):
tracing: Make sure trace_printk() can output as soon as it can be used ftrace/scripts: Update the instructions for ftrace-bisect.sh
----
MAINTAINERS | 5 +++++ kernel/trace/Kconfig | 8 +++---- kernel/trace/ftrace.c | 23 ++++++++++++++++++- kernel/trace/rv/rv.c | 2 +- kernel/trace/trace.c | 2 ++ kernel/trace/trace.h | 1 + kernel/trace/trace_events_filter.c | 8 +++---- kernel/trace/trace_events_hist.c | 2 ++ kernel/trace/trace_osnoise.c | 5 ++--- kernel/trace/trace_output.c | 3 +-- lib/Kconfig.debug | 2 +- lib/Kconfig.kcsan | 2 +- samples/ftrace/ftrace-direct-multi-modify.c | 1 + samples/ftrace/ftrace-direct-multi.c | 1 + scripts/tracing/ftrace-bisect.sh | 34 ++++++++++++++++++++++------- 15 files changed, 74 insertions(+), 25 deletions(-)
Steven Rostedt
maintainerPATCH 1/11 ftrace: Export ftrace_free_filter() to modules
From: Mark Rutland <[email protected]>
Setting filters on an ftrace ops results in some memory being allocated for the filter hashes, which must be freed before the ops can be freed. This can be done by removing every individual element of the hash by calling ftrace_set_filter_ip() or ftrace_set_filter_ips() with remove set, but this is somewhat error prone as it's easy to forget to remove an element.
Make it easier to clean this up by exporting ftrace_free_filter(), which can be used to clean up all of the filter hashes after an ftrace_ops has been unregistered.
Using this, fix the ftrace-direct* samples to free hashes prior to being unloaded. All other code either removes individual filters explicitly or is built-in and already calls ftrace_free_filter().
--- kernel/trace/ftrace.c | 23 ++++++++++++++++++++- samples/ftrace/ftrace-direct-multi-modify.c | 1 + samples/ftrace/ftrace-direct-multi.c | 1 + 3 files changed, 24 insertions(+), 1 deletion(-)
kernel/trace/ftrace.c +22 -1
@@ -1248,12 +1248,17 @@ static void free_ftrace_hash_rcu(struct ftrace_hash *hash)call_rcu (&hash->rcu, __free_ftrace_hash_rcu); } +/** +* ftrace_free_filter - remove all filters for an ftrace_ops +* @ops - the ops to remove the filters from +*/ void ftrace_free_filter (struct ftrace_ops *ops) {ftrace_ops_init (ops);free_ftrace_hash (ops->func_hash->filter_hash);free_ftrace_hash (ops->func_hash->notrace_hash); } +EXPORT_SYMBOL_GPL (ftrace_free_filter);static struct ftrace_hash *alloc_ftrace_hash (int size_bits) {
@@ -5839,6 +5844,10 @@ EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi);* * Filters denote which functions should be enabled when tracing is enabled * If @ip is NULL, it fails to update filter. +* +* This can allocate memory which must be freed before @ops can be freed, +* either by removing each filtered addr or by using +* ftrace_free_filter(@ops). */ int ftrace_set_filter_ip (struct ftrace_ops *ops,unsigned long ip,int remove,int reset)
@@ -5858,7 +5867,11 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter_ip);* * Filters denote which functions should be enabled when tracing is enabled * If @ips array or any ip specified within is NULL , it fails to update filter. -*/ +* +* This can allocate memory which must be freed before @ops can be freed, +* either by removing each filtered addr or by using +* ftrace_free_filter(@ops). +*/ int ftrace_set_filter_ips (struct ftrace_ops *ops,unsigned long *ips,unsigned int cnt,int remove,int reset) {
@@ -5900,6 +5913,10 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,* * Filters denote which functions should be enabled when tracing is enabled. * If @buf is NULL and reset is set, all functions will be enabled for tracing. +* +* This can allocate memory which must be freed before @ops can be freed, +* either by removing each filtered addr or by using +* ftrace_free_filter(@ops). */ int ftrace_set_filter (struct ftrace_ops *ops,unsigned char *buf,int len,int reset)
@@ -5919,6 +5936,10 @@ EXPORT_SYMBOL_GPL(ftrace_set_filter);* Notrace Filters denote which functions should not be enabled when tracing * is enabled. If @buf is NULL and reset is set, all functions will be enabled * for tracing. +* +* This can allocate memory which must be freed before @ops can be freed, +* either by removing each filtered addr or by using +* ftrace_free_filter(@ops). */ int ftrace_set_notrace (struct ftrace_ops *ops,unsigned char *buf,int len,int reset)
samples/ftrace/ftrace-direct-multi-modify.c +1 -0
@@ -152,6 +152,7 @@ static void __exit ftrace_direct_multi_exit(void) {kthread_stop (simple_tsk);unregister_ftrace_direct_multi (&direct, my_tramp); +ftrace_free_filter (&direct); }module_init (ftrace_direct_multi_init);
samples/ftrace/ftrace-direct-multi.c +1 -0
@@ -79,6 +79,7 @@ static int __init ftrace_direct_multi_init(void) static void __exit ftrace_direct_multi_exit(void) {unregister_ftrace_direct_multi (&direct, (unsigned long ) my_tramp); +ftrace_free_filter (&direct); }module_init (ftrace_direct_multi_init);
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 2/11 tracing: Make sure trace_printk() can output as soon as it can be used
In reply to Steven RostedtFrom: "Steven Rostedt (Google)" <[email protected]>
Currently trace_printk() can be used as soon as early_trace_init() is called from start_kernel(). But if a crash happens, and "ftrace_dump_on_oops" is set on the kernel command line, all you get will be:
[ 0.456075] <idle>-0 0dN.2. 347519us : Unknown type 6 [ 0.456075] <idle>-0 0dN.2. 353141us : Unknown type 6 [ 0.456075] <idle>-0 0dN.2. 358684us : Unknown type 6
This is because the trace_printk() event (type 6) hasn't been registered yet. That gets done via an early_initcall(), which may be early, but not early enough.
Instead of registering the trace_printk() event (and other ftrace events, which are not trace events) via an early_initcall(), have them registered at the same time that trace_printk() can be used. This way, if there is a crash before early_initcall(), then the trace_printk()s will actually be useful.
--- kernel/trace/trace.c | 2 ++ kernel/trace/trace.h | 1 + kernel/trace/trace_output.c | 3 +-- 3 files changed, 4 insertions(+), 2 deletions(-)
kernel/trace/trace.c +2 -0
@@ -10295,6 +10295,8 @@ void __init early_trace_init(void)static_key_enable (&tracepoint_printk_key.key); }tracer_alloc_buffers (); + +init_events (); }void __init trace_init (void )
kernel/trace/trace.h +1 -0
@@ -1490,6 +1490,7 @@ extern void trace_event_enable_cmd_record(bool enable);extern void trace_event_enable_tgid_record (bool enable);extern int event_trace_init (void ); +extern int init_events (void );extern int event_trace_add_tracer (struct dentry *parent,struct trace_array *tr);extern int event_trace_del_tracer (struct trace_array *tr);extern void __trace_early_add_events (struct trace_array *tr);
kernel/trace/trace_output.c +1 -2
@@ -1535,7 +1535,7 @@ static struct trace_event *events[] __initdata = { NULL }; -__init static int init_events (void ) +__init int init_events (void ) {struct trace_event *event;int i, ret;
@@ -1548,4 +1548,3 @@ __init static int init_events(void)return 0 ; } -early_initcall (init_events);
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 3/11 ftrace/scripts: Update the instructions for ftrace-bisect.sh
In reply to Steven RostedtFrom: "Steven Rostedt (Google)" <[email protected]>
The instructions for the ftrace-bisect.sh script, which is used to find what function is being traced that is causing a kernel crash, and possibly a triple fault reboot, uses the old method. In 5.1, a new feature was added that let the user write in the index into available_filter_functions that maps to the function a user wants to set in set_ftrace_filter (or set_ftrace_notrace). This takes O(1) to set, as suppose to writing a function name, which takes O(n) (where n is the number of functions in available_filter_functions).
The ftrace-bisect.sh requires setting half of the functions in available_filter_functions, which is O(n^2) using the name method to enable and can take several minutes to complete. The number method is O(n) which takes less than a second to complete. Using the number method for any kernel 5.1 and after is the proper way to do the bisect.
Update the usage to reflect the new change, as well as using the /sys/kernel/tracing path instead of the obsolete debugfs path.
--- scripts/tracing/ftrace-bisect.sh | 34 ++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-)
scripts/tracing/ftrace-bisect.sh +26 -8
@@ -12,7 +12,7 @@ # (note, if this is a problem with function_graph tracing, then simply # replace "function" with "function_graph" in the following steps). # -# # cd /sys/kernel/debug/tracing +# # cd /sys/kernel/tracing # # echo schedule > set_ftrace_filter # # echo function > current_tracer #
@@ -20,22 +20,40 @@ # # # echo nop > current_tracer # -# # cat available_filter_functions > ~/full-file +# Starting with v5.1 this can be done with numbers, making it much faster: +# +# The old (slow) way, for kernels before v5.1. +# +# [old-way] # cat available_filter_functions > ~/full-file +# +# [old-way] *** Note *** this process will take several minutes to update the +# [old-way] filters. Setting multiple functions is an O(n^2) operation, and we +# [old-way] are dealing with thousands of functions. So go have coffee, talk +# [old-way] with your coworkers, read facebook. And eventually, this operation +# [old-way] will end. +# +# The new way (using numbers) is an O(n) operation, and usually takes less than a second. +# +# seq `wc -l available_filter_functions | cut -d' ' -f1` > ~/full-file +# +# This will create a sequence of numbers that match the functions in +# available_filter_functions, and when echoing in a number into the +# set_ftrace_filter file, it will enable the corresponding function in +# O(1) time. Making enabling all functions O(n) where n is the number of +# functions to enable. +# +# For either the new or old way, the rest of the operations remain the same. +# # # ftrace-bisect ~/full-file ~/test-file ~/non-test-file # # cat ~/test-file > set_ftrace_filter # -# *** Note *** this will take several minutes. Setting multiple functions is -# an O(n^2) operation, and we are dealing with thousands of functions. So go -# have coffee, talk with your coworkers, read facebook. And eventually, this -# operation will end. -# # # echo function > current_tracer # # If it crashes, we know that ~/test-file has a bad function. # # Reboot back to test kernel. # -# # cd /sys/kernel/debug/tracing +# # cd /sys/kernel/tracing # # mv ~/test-file ~/full-file # # If it didn't crash.
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 4/11 tracing: Kconfig: Fix spelling/grammar/punctuation
In reply to Steven RostedtFrom: Randy Dunlap <[email protected]>
Fix some editorial nits in trace Kconfig.
--- kernel/trace/Kconfig | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/trace/Kconfig +4 -4
@@ -933,8 +933,8 @@ config RING_BUFFER_RECORD_RECURSION default y help The ring buffer has its own internal recursion. Although when - recursion happens it wont cause harm because of the protection, - but it does cause an unwanted overhead. Enabling this option will + recursion happens it won't cause harm because of the protection, + but it does cause unwanted overhead. Enabling this option will place where recursion was detected into the ftrace "recursed_functions" file.
@@ -1017,8 +1017,8 @@ config RING_BUFFER_STARTUP_TEST The test runs for 10 seconds. This will slow your boot time by at least 10 more seconds. - At the end of the test, statics and more checks are done. - It will output the stats of each per cpu buffer. What + At the end of the test, statistics and more checks are done. + It will output the stats of each per cpu buffer: What was written, the sizes, what was read, what was lost, and other similar details.
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 5/11 tracing/osnoise: Use built-in RCU list checking
In reply to Steven RostedtFrom: Chuang Wang <[email protected]>
list_for_each_entry_rcu() has built-in RCU and lock checking.
Pass cond argument to list_for_each_entry_rcu() to silence false lockdep warning when CONFIG_PROVE_RCU_LIST is enabled.
Execute as follow:
[tracing]# echo osnoise > current_tracer
[tracing]# echo 1 > tracing_on
[tracing]# echo 0 > tracing_on
The trace_types_lock is held when osnoise_tracer_stop() or timerlat_tracer_stop() are called in the non-RCU read side section. So, pass lockdep_is_held(&trace_types_lock) to silence false lockdep warning.
--- kernel/trace/trace_osnoise.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
kernel/trace/trace_osnoise.c +2 -3
@@ -147,9 +147,8 @@ static void osnoise_unregister_instance(struct trace_array *tr)* register/unregister serialization is provided by trace's * trace_types_lock. */ -lockdep_assert_held (&trace_types_lock); - -list_for_each_entry_rcu (inst, &osnoise_instances, list) { +list_for_each_entry_rcu (inst, &osnoise_instances, list, +lockdep_is_held (&trace_types_lock)) {if (inst->tr == tr) {list_del_rcu (&inst->list); found =1 ;
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 6/11 trace_events_hist: add check for return value of create_hist_field
In reply to Steven RostedtFrom: Natalia Petrova <[email protected]>
Function 'create_hist_field' is called recursively at trace_events_hist.c:1954 and can return NULL-value that's why we have to check it to avoid null pointer dereference.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
--- kernel/trace/trace_events_hist.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/trace/trace_events_hist.c +2 -0
@@ -1988,6 +1988,8 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, hist_field->fn_num = flags &HIST_FIELD_FL_LOG2 ?HIST_FIELD_FN_LOG2 :HIST_FIELD_FN_BUCKET ; hist_field->operands[0 ] =create_hist_field (hist_data, field, fl,NULL ); +if (!hist_field->operands[0 ]) + goto free; hist_field->size = hist_field->operands[0 ]->size; hist_field->type =kstrdup_const (hist_field->operands[0 ]->type,GFP_KERNEL );if (!hist_field->type)
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 7/11 lib: Kconfig: fix spellos
In reply to Steven RostedtFrom: Randy Dunlap <[email protected]>
Fix spelling in lib/ Kconfig files.
(reported by codespell)
--- lib/Kconfig.debug | 2 +- lib/Kconfig.kcsan | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
lib/Kconfig.debug +1 -1
@@ -1917,7 +1917,7 @@ config FUNCTION_ERROR_INJECTION help Add fault injections into various functions that are annotated with ALLOW_ERROR_INJECTION() in the kernel. BPF may also modify the return - value of theses functions. This is useful to test error paths of code. + value of these functions. This is useful to test error paths of code. If unsure, say N
lib/Kconfig.kcsan +1 -1
@@ -194,7 +194,7 @@ config KCSAN_WEAK_MEMORY Enable support for modeling a subset of weak memory, which allows detecting a subset of data races due to missing memory barriers. - Depends on KCSAN_STRICT, because the options strenghtening certain + Depends on KCSAN_STRICT, because the options strengthening certain plain accesses by default (depending on !KCSAN_STRICT) reduce the ability to detect any data races invoving reordered accesses, in particular reordered writes.
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 8/11 tracing/filter: fix kernel-doc warnings
In reply to Steven RostedtFrom: Randy Dunlap <[email protected]>
Use the 'struct' keyword for a struct's kernel-doc notation and use the correct function parameter name to eliminate kernel-doc warnings:
kernel/trace/trace_events_filter.c:136: warning: cannot understand function prototype: 'struct prog_entry ' kerne/trace/trace_events_filter.c:155: warning: Excess function parameter 'when_to_branch' description in 'update_preds'
Also correct some trivial punctuation problems.
--- kernel/trace/trace_events_filter.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/trace/trace_events_filter.c +4 -4
@@ -128,7 +128,7 @@ static bool is_not(const char *str) }/** -* prog_entry - a singe entry in the filter program +* struct prog_entry - a singe entry in the filter program * @target: Index to jump to on a branch (actually one minus the index) * @when_to_branch: The value of the result of the predicate to do a branch * @pred: The predicate to execute.
@@ -140,16 +140,16 @@ struct prog_entry { };/** -* update_preds- assign a program entry a label target +* update_preds - assign a program entry a label target * @prog: The program array * @N: The index of the current entry in @prog -* @when_to_branch: What to assign a program entry for its branch condition +* @invert: What to assign a program entry for its branch condition * * The program entry at @N has a target that points to the index of a program * entry that can have its target and when_to_branch fields updated. * Update the current program entry denoted by index @N target field to be * that of the updated entry. This will denote the entry to update if -* we are processing an "||" after an "&&" +* we are processing an "||" after an "&&". */ static void update_preds (struct prog_entry *prog,int N ,int invert) {
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 9/11 ftrace: Maintain samples/ftrace
In reply to Steven RostedtFrom: Mark Rutland <[email protected]>
There's no entry in MAINTAINERS for samples/ftrace. Add one so that the FTRACE maintainers are kept in the loop.
--- MAINTAINERS | 1 + 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
@@ -8570,6 +8570,7 @@ F: kernel/trace/fgraph.c F: arch/*/*/*/*ftrace* F: arch/*/*/*ftrace* F: include/*/ftrace.h +F: samples/ftrace FUNGIBLE ETHERNET DRIVERS M: Dimitris Michailidis <[email protected]>
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 10/11 remove redundant initialization of pointer ptr
In reply to Steven RostedtFrom: Colin Ian King <[email protected]>
The pointer ptr is being initialized with a value that is never read, it is being updated later on a call to strim. Remove the extraneous initialization.
--- kernel/trace/rv/rv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/rv/rv.c +1 -1
@@ -516,7 +516,7 @@ static ssize_t enabled_monitors_write(struct file *filp, const char __user *userstruct rv_monitor_def *mdef;int retval = -EINVAL ;bool enable = true; -char *ptr = buff; +char *ptr;int len;if (count <1 || count >MAX_RV_MONITOR_NAME_SIZE +1 )
Signature · 1 line
2.39.0
Steven Rostedt
maintainerPATCH 11/11 bootconfig: Update MAINTAINERS file to add tree and mailing list
In reply to Steven RostedtFrom: "Masami Hiramatsu (Google)" <[email protected]>
Since the bootconfig related changes will be handled on linux-trace tree, add the tree and mailing lists for EXTRA BOOT CONFIG.
--- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+)
MAINTAINERS +4 -0
@@ -7894,7 +7894,11 @@ F: include/linux/extcon/ EXTRA BOOT CONFIG M: Masami Hiramatsu <[email protected]> +L: [email protected] +L: [email protected] +Q: https://patchwork.kernel.org/project/linux-trace-kernel/list/ S: Maintained +T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git F: Documentation/admin-guide/bootconfig.rst F: fs/proc/bootconfig.c F: include/linux/bootconfig.h
Signature · 1 line
2.39.0
Steven Rostedt
maintainerQuoted 2 lines · On Wed, 25 Jan 2023 11:18:24 -0500 Steven Rostedt <[email protected]> wrote:
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
ftrace/urgent
Actually, that will be trace/urgent (had it in the wrong branch :-p)
-- Steve