Skip to contentSkip to conversation

tracing: Add an option to show symbols in _text+offset for function profiler

PATCH Superseded Masami Hiramatsu (Google) 3 participants 6 messages Sep 19, 2025 – Sep 21, 2025

Subsystems (2)

v1 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Masami Hiramatsu (Google)

author maintainer
#0 lore raw

PATCH 1/1 tracing: Add an option to show symbols in _text+offset for function profiler

From: Masami Hiramatsu (Google) <[email protected]>

Function profiler shows the hit count of each function using its symbol name. However, there are some same-name local symbols, which we can not distinguish.
To solve this issue, this introduces an option to show the symbols in "_text+OFFSET" format. This can avoid exposing the random shift of KASLR. The functions in modules are shown as "MODNAME+OFFSET" where the offset is from ".text".

E.g. for the kernel text symbols, specify vmlinux and the output to
addr2line, you can find the actual function and source info;

  $ addr2line -fie vmlinux _text+3078208
  __balance_callbacks
  kernel/sched/core.c:5064

for modules, specify the module file and .text+OFFSET;

  $ addr2line -fie samples/trace_events/trace-events-sample.ko .text+8224
  do_simple_thread_func
  samples/trace_events/trace-events-sample.c:23
---
 kernel/trace/ftrace.c |   26 +++++++++++++++++++++++++-
 kernel/trace/trace.c  |    5 +++--
 kernel/trace/trace.h  |   10 +++++++++-
 3 files changed, 37 insertions(+), 4 deletions(-)
kernel/trace/ftrace.c +25 -1
index a69067367c29..c759cc392fbe 100644
@@ -534,7 +534,9 @@ static int function_stat_headers(struct seq_file *m)
 
 static int function_stat_show(struct seq_file *m, void *v)
 {
+	struct trace_array *tr = trace_get_global_array();
 	struct ftrace_profile *rec = v;
+	const char *refsymbol = NULL;
 	char str[KSYM_SYMBOL_LEN];
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 	static struct trace_seq s;
@@ -554,7 +556,29 @@ static int function_stat_show(struct seq_file *m, void *v)
 		return 0;
 #endif
 
-	kallsyms_lookup(rec->ip, NULL, NULL, NULL, str);
+	if (tr->trace_flags & TRACE_ITER_PROF_TEXT_OFFSET) {
+		long offset;
+
+		if (core_kernel_text(rec->ip)) {
+			refsymbol = "_text";
+			offset = rec->ip - (unsigned long)_text;
+		} else {
+			struct module *mod;
+
+			guard(rcu)();
+			mod = __module_text_address(rec->ip);
+			if (mod) {
+				refsymbol = mod->name;
+				/* Calculate offset from module's text entry address. */
+				offset = rec->ip - (unsigned long)mod->mem[MOD_TEXT].base;
+			}
+		}
+		if (refsymbol)
+			snprintf(str, sizeof(str), "  %s%+ld", refsymbol, offset);
+	}
+	if (!refsymbol)
+		kallsyms_lookup(rec->ip, NULL, NULL, NULL, str);
+
 	seq_printf(m, "  %-30.30s  %10lu", str, rec->counter);
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
kernel/trace/trace.c +3 -2
index b3c94fbaf002..bd82b28b8d2b 100644
@@ -522,7 +522,8 @@ EXPORT_SYMBOL_GPL(unregister_ftrace_export);
 
 /* trace_options that are only supported by global_trace */
 #define TOP_LEVEL_TRACE_FLAGS (TRACE_ITER_PRINTK |			\
-	       TRACE_ITER_PRINTK_MSGONLY | TRACE_ITER_RECORD_CMD)
+	       TRACE_ITER_PRINTK_MSGONLY | TRACE_ITER_RECORD_CMD |	\
+	       TRACE_ITER_PROF_TEXT_OFFSET)
 
 /* trace_flags that are default zero for instances */
 #define ZEROED_TRACE_FLAGS \
@@ -11115,7 +11116,7 @@ __init static int tracer_alloc_buffers(void)
 
 #ifdef CONFIG_FUNCTION_TRACER
 /* Used to set module cached ftrace filtering at boot up */
-__init struct trace_array *trace_get_global_array(void)
+struct trace_array *trace_get_global_array(void)
 {
 	return &global_trace;
 }
kernel/trace/trace.h +9 -1
index 5f4bed5842f9..f698aac73b14 100644
@@ -1359,6 +1359,13 @@ extern int trace_get_user(struct trace_parser *parser, const char __user *ubuf,
 # define STACK_FLAGS
 #endif
 
+#ifdef CONFIG_FUNCTION_PROFILER
+# define PROFILER_FLAGS					\
+		C(PROF_TEXT_OFFSET,	"prof-text-offset"),
+#else
+# define PROFILER_FLAGS
+#endif
+
 /*
  * trace_iterator_flags is an enumeration that defines bit
  * positions into trace_flags that controls the output.
@@ -1397,7 +1404,8 @@ extern int trace_get_user(struct trace_parser *parser, const char __user *ubuf,
 		FUNCTION_FLAGS					\
 		FGRAPH_FLAGS					\
 		STACK_FLAGS					\
-		BRANCH_FLAGS
+		BRANCH_FLAGS					\
+		PROFILER_FLAGS
 
 /*
  * By defining C, we can make TRACE_FLAGS a list of bit names
#1 lore raw
kernel test robot noticed the following build errors:

Hi Masami,

kernel test robot noticed the following build errors:

[auto build test ERROR on trace/for-next]
[also build test ERROR on linus/master v6.17-rc6 next-20250918] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/tracing-Add-an-option-to-show-symbols-in-_text-offset-for-function-profiler/20250919-135733
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/175826135058.101165.7219957344129610147.stgit%40devnote2
patch subject: [PATCH] tracing: Add an option to show symbols in _text+offset for function profiler
config: x86_64-buildonly-randconfig-002-20250919 (https://download.01.org/0day-ci/archive/20250920/[email protected]/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All error/warnings (new ones prefixed by >>):

In file included from kernel/trace/trace_sched_switch.c:15:

>> kernel/trace/trace.h:1422:37: warning: left shift count >= width of type [-Wshift-count-overflow]
    1422 | #define C(a, b) TRACE_ITER_##a = (1 << TRACE_ITER_##a##_BIT)
         |                                     ^~
   kernel/trace/trace.h:1358:17: note: in expansion of macro 'C'
    1358 |                 C(PROF_TEXT_OFFSET,     "prof-text-offset"),
         |                 ^
   kernel/trace/trace.h:1402:17: note: in expansion of macro 'PROFILER_FLAGS'
    1402 |                 PROFILER_FLAGS
         |                 ^~~~~~~~~~~~~~
   kernel/trace/trace.h:1424:29: note: in expansion of macro 'TRACE_FLAGS'
    1424 | enum trace_iterator_flags { TRACE_FLAGS };
         |                             ^~~~~~~~~~~
--
   In file included from kernel/trace/trace.c:58:
>> kernel/trace/trace.h:1422:37: warning: left shift count >= width of type [-Wshift-count-overflow]
    1422 | #define C(a, b) TRACE_ITER_##a = (1 << TRACE_ITER_##a##_BIT)
         |                                     ^~
   kernel/trace/trace.h:1358:17: note: in expansion of macro 'C'
    1358 |                 C(PROF_TEXT_OFFSET,     "prof-text-offset"),
         |                 ^
   kernel/trace/trace.h:1402:17: note: in expansion of macro 'PROFILER_FLAGS'
    1402 |                 PROFILER_FLAGS
         |                 ^~~~~~~~~~~~~~
   kernel/trace/trace.h:1424:29: note: in expansion of macro 'TRACE_FLAGS'
    1424 | enum trace_iterator_flags { TRACE_FLAGS };
         |                             ^~~~~~~~~~~
   In file included from <command-line>:
   In function 'tracer_alloc_buffers',
       inlined from 'early_trace_init' at kernel/trace/trace.c:11144:2:
>> include/linux/compiler_types.h:572:45: error: call to '__compiletime_assert_557' declared with attribute error: BUILD_BUG_ON failed: TRACE_ITER_LAST_BIT > TRACE_FLAGS_MAX_SIZE
     572 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |                                             ^
   include/linux/compiler_types.h:553:25: note: in definition of macro '__compiletime_assert'
     553 |                         prefix ## suffix();                             \
         |                         ^~~~~~
   include/linux/compiler_types.h:572:9: note: in expansion of macro '_compiletime_assert'
     572 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
      39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
         |                                     ^~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:50:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
      50 |         BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
         |         ^~~~~~~~~~~~~~~~
   kernel/trace/trace.c:10983:9: note: in expansion of macro 'BUILD_BUG_ON'
   10983 |         BUILD_BUG_ON(TRACE_ITER_LAST_BIT > TRACE_FLAGS_MAX_SIZE);
         |         ^~~~~~~~~~~~

vim +1422 kernel/trace/trace.h

a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1416) 
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1417) /*
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1418)  * By redefining C, we can make TRACE_FLAGS a list of masks that
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1419)  * use the bits as defined above.
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1420)  */
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1421) #undef C
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29 @1422) #define C(a, b) TRACE_ITER_##a = (1 << TRACE_ITER_##a##_BIT)
a3418a364ec3c8f Steven Rostedt (Red Hat  2015-09-29  1423) 
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#4 lore raw
In reply to Masami Hiramatsu (Google)
kernel test robot noticed the following build errors:

Hi Masami,

kernel test robot noticed the following build errors:

[auto build test ERROR on trace/for-next]
[also build test ERROR on linus/master v6.17-rc6 next-20250918] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/tracing-Add-an-option-to-show-symbols-in-_text-offset-for-function-profiler/20250919-135733
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/175826135058.101165.7219957344129610147.stgit%40devnote2
patch subject: [PATCH] tracing: Add an option to show symbols in _text+offset for function profiler
config: arc-randconfig-001-20250919 (https://download.01.org/0day-ci/archive/20250920/[email protected]/config)
compiler: arc-linux-gcc (GCC) 15.1.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

kernel/trace/trace.c: In function 'create_trace_options_dir':

>> kernel/trace/trace.c:526:16: error: 'TRACE_ITER_PROF_TEXT_OFFSET' undeclared (first use in this function); did you mean 'TRACE_ITER_CONTEXT_INFO'?
     526 |                TRACE_ITER_PROF_TEXT_OFFSET)
         |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
   kernel/trace/trace.c:9316:34: note: in expansion of macro 'TOP_LEVEL_TRACE_FLAGS'
    9316 |                     !((1 << i) & TOP_LEVEL_TRACE_FLAGS))
         |                                  ^~~~~~~~~~~~~~~~~~~~~
   kernel/trace/trace.c:526:16: note: each undeclared identifier is reported only once for each function it appears in
     526 |                TRACE_ITER_PROF_TEXT_OFFSET)
         |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
   kernel/trace/trace.c:9316:34: note: in expansion of macro 'TOP_LEVEL_TRACE_FLAGS'
    9316 |                     !((1 << i) & TOP_LEVEL_TRACE_FLAGS))
         |                                  ^~~~~~~~~~~~~~~~~~~~~

vim +526 kernel/trace/trace.c

512
513 /* trace_flags holds trace_options default values */
514 #define TRACE_DEFAULT_FLAGS \
515 (FUNCTION_DEFAULT_FLAGS | \
516 TRACE_ITER_PRINT_PARENT | TRACE_ITER_PRINTK | \
517 TRACE_ITER_ANNOTATE | TRACE_ITER_CONTEXT_INFO | \
518 TRACE_ITER_RECORD_CMD | TRACE_ITER_OVERWRITE | \
519 TRACE_ITER_IRQ_INFO | TRACE_ITER_MARKERS | \
520 TRACE_ITER_HASH_PTR | TRACE_ITER_TRACE_PRINTK | \
521 TRACE_ITER_COPY_MARKER)
522
523 /* trace_options that are only supported by global_trace */
524 #define TOP_LEVEL_TRACE_FLAGS (TRACE_ITER_PRINTK | \
525 TRACE_ITER_PRINTK_MSGONLY | TRACE_ITER_RECORD_CMD | \

 > 526		       TRACE_ITER_PROF_TEXT_OFFSET)

527

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
maintainer
#2 lore raw
In reply to kernel test robot
Quoted 23 lines · On Sat, 20 Sep 2025 01:09:52 +0800 kernel test robot <[email protected]> wrote:

Hi Masami,

kernel test robot noticed the following build errors:

[auto build test ERROR on trace/for-next]
[also build test ERROR on linus/master v6.17-rc6 next-20250918] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/tracing-Add-an-option-to-show-symbols-in-_text-offset-for-function-profiler/20250919-135733
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/175826135058.101165.7219957344129610147.stgit%40devnote2
patch subject: [PATCH] tracing: Add an option to show symbols in _text+offset for function profiler
config: x86_64-buildonly-randconfig-002-20250919 (https://download.01.org/0day-ci/archive/20250920/[email protected]/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

Quoted 6 lines

All error/warnings (new ones prefixed by >>):

In file included from kernel/trace/trace_sched_switch.c:15:

Quoted 1 line
Quoted 1 line

kernel/trace/trace.h:1422:37: warning: left shift count >= width of type [-Wshift-count-overflow]

    1422 | #define C(a, b) TRACE_ITER_##a = (1 << TRACE_ITER_##a##_BIT)
         |                                     ^~

Ouch! We need to make the option enum64 now!

-- Steve

author maintainer
#5 lore raw
In reply to kernel test robot
Quoted 23 lines · On Sat, 20 Sep 2025 01:09:42 +0800 kernel test robot <[email protected]> wrote:

Hi Masami,

kernel test robot noticed the following build errors:

[auto build test ERROR on trace/for-next]
[also build test ERROR on linus/master v6.17-rc6 next-20250918] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/tracing-Add-an-option-to-show-symbols-in-_text-offset-for-function-profiler/20250919-135733
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/175826135058.101165.7219957344129610147.stgit%40devnote2
patch subject: [PATCH] tracing: Add an option to show symbols in _text+offset for function profiler
config: arc-randconfig-001-20250919 (https://download.01.org/0day-ci/archive/20250920/[email protected]/config)
compiler: arc-linux-gcc (GCC) 15.1.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

Quoted 6 lines

All errors (new ones prefixed by >>):

kernel/trace/trace.c: In function 'create_trace_options_dir':

Quoted 1 line
Quoted 1 line

kernel/trace/trace.c:526:16: error: 'TRACE_ITER_PROF_TEXT_OFFSET' undeclared (first use in this function); did you mean 'TRACE_ITER_CONTEXT_INFO'?

     526 |                TRACE_ITER_PROF_TEXT_OFFSET)
         |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~

Aah, it should be defined always. OK, let me fix that in the next version.

Thanks!

Quoted 33 lines
   kernel/trace/trace.c:9316:34: note: in expansion of macro 'TOP_LEVEL_TRACE_FLAGS'
    9316 |                     !((1 << i) & TOP_LEVEL_TRACE_FLAGS))
         |                                  ^~~~~~~~~~~~~~~~~~~~~
   kernel/trace/trace.c:526:16: note: each undeclared identifier is reported only once for each function it appears in
     526 |                TRACE_ITER_PROF_TEXT_OFFSET)
         |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
   kernel/trace/trace.c:9316:34: note: in expansion of macro 'TOP_LEVEL_TRACE_FLAGS'
    9316 |                     !((1 << i) & TOP_LEVEL_TRACE_FLAGS))
         |                                  ^~~~~~~~~~~~~~~~~~~~~

vim +526 kernel/trace/trace.c

   512	
   513	/* trace_flags holds trace_options default values */
   514	#define TRACE_DEFAULT_FLAGS						\
   515		(FUNCTION_DEFAULT_FLAGS |					\
   516		 TRACE_ITER_PRINT_PARENT | TRACE_ITER_PRINTK |			\
   517		 TRACE_ITER_ANNOTATE | TRACE_ITER_CONTEXT_INFO |		\
   518		 TRACE_ITER_RECORD_CMD | TRACE_ITER_OVERWRITE |			\
   519		 TRACE_ITER_IRQ_INFO | TRACE_ITER_MARKERS |			\
   520		 TRACE_ITER_HASH_PTR | TRACE_ITER_TRACE_PRINTK |		\
   521		 TRACE_ITER_COPY_MARKER)
   522	
   523	/* trace_options that are only supported by global_trace */
   524	#define TOP_LEVEL_TRACE_FLAGS (TRACE_ITER_PRINTK |			\
   525		       TRACE_ITER_PRINTK_MSGONLY | TRACE_ITER_RECORD_CMD |	\
Quoted 1 line

526 TRACE_ITER_PROF_TEXT_OFFSET)

527

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
author maintainer
#3 lore raw
In reply to Steven Rostedt
Quoted 26 lines · On Fri, 19 Sep 2025 13:16:21 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 23 lines · On Sat, 20 Sep 2025 01:09:52 +0800 kernel test robot <[email protected]> wrote:

Hi Masami,

kernel test robot noticed the following build errors:

[auto build test ERROR on trace/for-next]
[also build test ERROR on linus/master v6.17-rc6 next-20250918] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/tracing-Add-an-option-to-show-symbols-in-_text-offset-for-function-profiler/20250919-135733
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/175826135058.101165.7219957344129610147.stgit%40devnote2
patch subject: [PATCH] tracing: Add an option to show symbols in _text+offset for function profiler
config: x86_64-buildonly-randconfig-002-20250919 (https://download.01.org/0day-ci/archive/20250920/[email protected]/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

Quoted 8 lines
Quoted 6 lines

All error/warnings (new ones prefixed by >>):

In file included from kernel/trace/trace_sched_switch.c:15:

Quoted 1 line
Quoted 1 line

kernel/trace/trace.h:1422:37: warning: left shift count >= width of type [-Wshift-count-overflow]

    1422 | #define C(a, b) TRACE_ITER_##a = (1 << TRACE_ITER_##a##_BIT)
         |                                     ^~

Ouch! We need to make the option enum64 now!

Ah, it is an enum, not macro.

Thanks!

Quoted 3 lines

-- Steve

Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>