Skip to contentSkip to conversation

perf hist: Introduce routine to measure lenght of formatted entry

tip:perf/core tip-bot for Arnaldo Carvalho de Melo 3 participants 9 messages Aug 2, 2010 – Nov 25, 2011 perf events

lore
Share & tools
lore
#0 lore raw
Commit-ID:  06daaaba7c211ca6a8227b9a54dbc86dd837f034
Gitweb:     http://git.kernel.org/tip/06daaaba7c211ca6a8227b9a54dbc86dd837f034
Author:     Arnaldo Carvalho de Melo <[email protected]>
AuthorDate: Wed, 21 Jul 2010 17:58:25 -0300
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 27 Jul 2010 11:24:31 -0300

perf hist: Introduce routine to measure lenght of formatted entry

Will be used to figure out the window width needed in the new tree widget.

Cc: Frederic Weisbecker <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
LKML-Reference: <new-submission>

---
 tools/perf/util/hist.c |   27 +++++++++++++++++++++++++++
 tools/perf/util/hist.h |    3 +++
 2 files changed, 30 insertions(+), 0 deletions(-)
tools/perf/util/hist.c +27 -0
index 0bc6790..f93095f 100644
@@ -850,6 +850,33 @@ print_entries:
 	return ret;
 }
 
+/*
+ * See hists__fprintf to match the column widths
+ */
+unsigned int hists__sort_list_width(struct hists *self)
+{
+	struct sort_entry *se;
+	int ret = 9; /* total % */
+
+	if (symbol_conf.show_cpu_utilization) {
+		ret += 7; /* count_sys % */
+		ret += 6; /* count_us % */
+		if (perf_guest) {
+			ret += 13; /* count_guest_sys % */
+			ret += 12; /* count_guest_us % */
+		}
+	}
+
+	if (symbol_conf.show_nr_samples)
+		ret += 11;
+
+	list_for_each_entry(se, &hist_entry__sort_list, list)
+		if (!se->elide)
+			ret += 2 + hists__col_len(self, se->se_width_idx);
+
+	return ret;
+}
+
 static void hists__remove_entry_filter(struct hists *self, struct hist_entry *h,
 				       enum hist_filter filter)
 {
tools/perf/util/hist.h +3 -0
index 92962b2..65a48db 100644
@@ -141,4 +141,7 @@ int hist_entry__tui_annotate(struct hist_entry *self);
 
 int hists__tui_browse_tree(struct rb_root *self, const char *help);
 #endif
+
+unsigned int hists__sort_list_width(struct hists *self);
+
 #endif	/* __PERF_HIST_H */
#1 lore raw

libperf interface stability

Quoted 33 lines · tip-bot for Arnaldo Carvalho de Melo
Commit-ID:  06daaaba7c211ca6a8227b9a54dbc86dd837f034
Gitweb:     http://git.kernel.org/tip/06daaaba7c211ca6a8227b9a54dbc86dd837f034
Author:     Arnaldo Carvalho de Melo <[email protected]>
AuthorDate: Wed, 21 Jul 2010 17:58:25 -0300
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 27 Jul 2010 11:24:31 -0300

perf hist: Introduce routine to measure lenght of formatted entry

Will be used to figure out the window width needed in the new tree widget.

Cc: Frederic Weisbecker <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephane Eranian <[email protected]>
LKML-Reference: <new-submission>

---
 tools/perf/util/hist.c |   27 +++++++++++++++++++++++++++
 tools/perf/util/hist.h |    3 +++
 2 files changed, 30 insertions(+), 0 deletions(-)
tools/perf/util/hist.c +2 -0
index 0bc6790..f93095f 100644
@@ -850,6 +850,33 @@ print_entries:
 	return ret;
 }
 
+/*
+ * See hists__fprintf to match the column widths

tools/perf/util/hist.c print_entries: · line 864

Quoted 10 lines
tools/perf/util/hist.c +10 -0
+ */
+unsigned int hists__sort_list_width(struct hists *self)
+{
+	struct sort_entry *se;
+	int ret = 9; /* total % */
+
+	if (symbol_conf.show_cpu_utilization) {
+		ret += 7; /* count_sys % */
+		ret += 6; /* count_us % */
+		if (perf_guest) {

Arnaldo,

this accesses a global variable that is not part of libperf. Linking it causes undefined references, see below for example code. This applies at least to following symbols: perf_guest, perf_host, use_browser, perf_version_string. To fix this we need to extend the library interface. As a workaround the references must be added to the application using libperf:

bool perf_host;
bool perf_guest;
const char perf_version_string[] = "undefined";
int use_browser = -1;

Also there are library dependencies to -lelf -ldw -lm -lbfd even if its functionality is not used. It would be better to load libraries dynamically then or to have switches to disable code using them.

Is libperf supposed to be a library interface in general, or is it for internal perf tools usage only? It would be good to have a well defined, stable libperf interface for tools other than perf.

-Robert

$ cd tools/perf
$ cat linktest.c 
#include "util/evsel.h"
#include "util/evlist.h"
#include "util/cpumap.h"
#include "util/thread_map.h"

int main(void)

{
	perf_evlist__mmap(NULL, 0, 0);
	return 0;
}
$ gcc -I util/include/ -DNO_NEWT_SUPPORT -DNO_DWARF -lelf -ldw -lm -lbfd linktest.c libperf.a -o linktest
libperf.a(hist.o): In function `hists__sort_list_width':
tools/perf/util/hist.c:1074: undefined reference to `perf_guest'
tools/perf/util/hist.c:1074: undefined reference to `perf_guest'
libperf.a(hist.o): In function `hist_entry__pcnt_snprintf':
tools/perf/util/hist.c:779: undefined reference to `perf_guest'
libperf.a(hist.o): In function `hists__fprintf':
tools/perf/util/hist.c:938: undefined reference to `perf_guest'
tools/perf/util/hist.c:945: undefined reference to `perf_guest'
libperf.a(event.o):tools/perf/util/event.c:697: more undefined references to `perf_guest' follow
libperf.a(event.o): In function `thread__find_addr_map':
tools/perf/util/event.c:681: undefined reference to `perf_host'
tools/perf/util/event.c:673: undefined reference to `perf_host'
tools/perf/util/event.c:707: undefined reference to `perf_host'
tools/perf/util/event.c:684: undefined reference to `perf_guest'
tools/perf/util/event.c:703: undefined reference to `perf_guest'
libperf.a(debug.o): In function `eprintf':
tools/perf/util/debug.c:25: undefined reference to `use_browser'
libperf.a(header.o): In function `do_write_string':
tools/perf/util/header.c:126: undefined reference to `perf_version_string'
libperf.a(header.o): In function `do_write':
tools/perf/util/header.c:94: undefined reference to `perf_version_string'
collect2: ld returned 1 exit status
Quoted 35 lines
tools/perf/util/hist.c +15 -0
+			ret += 13; /* count_guest_sys % */
+			ret += 12; /* count_guest_us % */
+		}
+	}
+
+	if (symbol_conf.show_nr_samples)
+		ret += 11;
+
+	list_for_each_entry(se, &hist_entry__sort_list, list)
+		if (!se->elide)
+			ret += 2 + hists__col_len(self, se->se_width_idx);
+
+	return ret;
+}
+
 static void hists__remove_entry_filter(struct hists *self, struct hist_entry *h,
 				       enum hist_filter filter)
 {
tools/perf/util/hist.h +3 -1
index 92962b2..65a48db 100644
@@ -141,4 +141,7 @@ int hist_entry__tui_annotate(struct hist_entry *self);
 
 int hists__tui_browse_tree(struct rb_root *self, const char *help);
 #endif
+
+unsigned int hists__sort_list_width(struct hists *self);
+
 #endif	/* __PERF_HIST_H */
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 2 lines
Advanced Micro Devices, Inc.
Operating System Research Center
#2 lore raw

Re: libperf interface stability

Em Thu, Nov 24, 2011 at 05:28:45PM +0100, Robert Richter escreveu:

Quoted 20 lines · tip-bot for Arnaldo Carvalho de Melo

tools/perf/util/hist.c print_entries: · line 864

Quoted 4 lines
tools/perf/util/hist.c +4 -0
+	if (symbol_conf.show_cpu_utilization) {
+		ret += 7; /* count_sys % */
+		ret += 6; /* count_us % */
+		if (perf_guest) {

Arnaldo,

this accesses a global variable that is not part of libperf. Linking it causes undefined references, see below for example code. This applies at least to following symbols: perf_guest, perf_host, use_browser, perf_version_string. To fix this we need to extend the library interface. As a workaround the references must be added to the application using libperf:

bool perf_host;
bool perf_guest;
const char perf_version_string[] = "undefined";
int use_browser = -1;

Quoted 5 lines

Also there are library dependencies to -lelf -ldw -lm -lbfd even if its functionality is not used. It would be better to load libraries dynamically then or to have switches to disable code using them.

Is libperf supposed to be a library interface in general, or is it for

No it was not meant to be, I guess, when I started working in perf it was there already, I guess it was made like that by Ingo following some convention found in the git sources from where code/model was borrowed.

Quoted 1 line

internal perf tools usage only?

yes, so far

Quoted 2 lines

It would be good to have a well defined, stable libperf interface for tools other than perf.

Agreed, an effort in this direction was the perf python binding, that so far includes only:

tools/perf/util/setup.py

	sources = ['util/python.c', 'util/ctype.c', 'util/evlist.c',
		   'util/evsel.c', 'util/cpumap.c', 'util/thread_map.c',
		   'util/util.c', 'util/xyarray.c', 'util/cgroup.c',
		   'util/debugfs.c'],

You need https://github.com/acmel/linux/commits/perf/urgent to build it tho, it has a fix for that:

In the end it generates a ~/build/perf/python/perf.so file (I use make -C tools/perf O=/home/acme/build/perf) that has all that is needed for simple tools like tools/perf/python/twatch.py.

I think of it as a precursor for a shared library for use with C and I have been working on untangling references such as the symbol_conf one you mentioned and making the 'perf_evlist' and 'perf_evsel' classes the main way to work with libperf.

I'm working right now on another effort that hopefully will end up easing the development of new tools, in-perf and outside, current goal is to have a strace like tool using using code carved out from tools.

I have a branch that I explicitely mark as unstable where I'm putting what I have at:

https://github.com/acmel/linux/commits/tmp.perf/trace4

I hope to have it in a form that is reasonable to move it to 'perf/core' and then ask Ingo to merge it in the next merge window.

This is how the 'strace' is looking like, removing the error handling to simplify:

static const char *strace__tracepoints[] = {
"raw_syscalls:sys_enter",
"raw_syscalls:sys_exit",
};

static int strace(struct perf_record_opts *opts, const char *argv[])

{
	struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);

	err = perf_evlist__create_maps(evlist, opts->target_pid,
				       opts->target_tid, opts->cpu_list);

	err = perf_evlist__add_tracepoints_array(evlist, strace__tracepoints);

	err = perf_evlist__prepare_workload(evlist, opts, argv);

	perf_evlist__config_attrs(evlist, opts);

	err = perf_evlist__open(evlist, opts->group);

	err = perf_evlist__mmap(evlist, opts->mmap_pages, false);

	perf_evlist__start_workload(evlist);

	/*
	 * WIP: use perf_evlist__read, etc and then feed a
	 * perf_event_ops operations sorted events like is done in
	 * perf_session__process_events. Top is like that but currently
	 * doesn't use perf_session__process_events, we need to make it
	 * use something refactored out from perf_session that does the
	 * ordering of samples perf_session does, etc.
	 */

	perf_evlist__munmap(evlist);

	perf_evlist__delete(evlist);
}

So yes, I agree that at some point we should have a libperf.so, I'm working on it, but there are still several experimentations to do to commit to such an stable library :-\

Jiri Olsa is experimenting with the mmap part, that needs to be reworked to support multiple sample types, etc.

- Arnaldo

Quoted 80 lines

-Robert

$ cd tools/perf
$ cat linktest.c 
#include "util/evsel.h"
#include "util/evlist.h"
#include "util/cpumap.h"
#include "util/thread_map.h"

int main(void)

{
	perf_evlist__mmap(NULL, 0, 0);
	return 0;
}
$ gcc -I util/include/ -DNO_NEWT_SUPPORT -DNO_DWARF -lelf -ldw -lm -lbfd linktest.c libperf.a -o linktest
libperf.a(hist.o): In function `hists__sort_list_width':
tools/perf/util/hist.c:1074: undefined reference to `perf_guest'
tools/perf/util/hist.c:1074: undefined reference to `perf_guest'
libperf.a(hist.o): In function `hist_entry__pcnt_snprintf':
tools/perf/util/hist.c:779: undefined reference to `perf_guest'
libperf.a(hist.o): In function `hists__fprintf':
tools/perf/util/hist.c:938: undefined reference to `perf_guest'
tools/perf/util/hist.c:945: undefined reference to `perf_guest'
libperf.a(event.o):tools/perf/util/event.c:697: more undefined references to `perf_guest' follow
libperf.a(event.o): In function `thread__find_addr_map':
tools/perf/util/event.c:681: undefined reference to `perf_host'
tools/perf/util/event.c:673: undefined reference to `perf_host'
tools/perf/util/event.c:707: undefined reference to `perf_host'
tools/perf/util/event.c:684: undefined reference to `perf_guest'
tools/perf/util/event.c:703: undefined reference to `perf_guest'
libperf.a(debug.o): In function `eprintf':
tools/perf/util/debug.c:25: undefined reference to `use_browser'
libperf.a(header.o): In function `do_write_string':
tools/perf/util/header.c:126: undefined reference to `perf_version_string'
libperf.a(header.o): In function `do_write':
tools/perf/util/header.c:94: undefined reference to `perf_version_string'
collect2: ld returned 1 exit status
Quoted 35 lines
tools/perf/util/hist.c +15 -0
+			ret += 13; /* count_guest_sys % */
+			ret += 12; /* count_guest_us % */
+		}
+	}
+
+	if (symbol_conf.show_nr_samples)
+		ret += 11;
+
+	list_for_each_entry(se, &hist_entry__sort_list, list)
+		if (!se->elide)
+			ret += 2 + hists__col_len(self, se->se_width_idx);
+
+	return ret;
+}
+
 static void hists__remove_entry_filter(struct hists *self, struct hist_entry *h,
 				       enum hist_filter filter)
 {
tools/perf/util/hist.h +3 -1
index 92962b2..65a48db 100644
@@ -141,4 +141,7 @@ int hist_entry__tui_annotate(struct hist_entry *self);
 
 int hists__tui_browse_tree(struct rb_root *self, const char *help);
 #endif
+
+unsigned int hists__sort_list_width(struct hists *self);
+
 #endif	/* __PERF_HIST_H */
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 2 lines
Advanced Micro Devices, Inc.
Operating System Research Center
#3 lore raw

strace based on perf [Re: libperf interface stability]

Quoted 3 lines · Arnaldo Carvalho de Melo

I'm working right now on another effort that hopefully will end up easing the development of new tools, in-perf and outside, current goal is to have a strace like tool using using code carved out from tools.

Interesting. I was thinking about this as well -- and Arun too I believe.

Quoted 9 lines

I have a branch that I explicitely mark as unstable where I'm putting what I have at:

https://github.com/acmel/linux/commits/tmp.perf/trace4

I hope to have it in a form that is reasonable to move it to 'perf/core' and then ask Ingo to merge it in the next merge window.

Quoted 7 lines

This is how the 'strace' is looking like, removing the error handling to simplify:

static const char *strace__tracepoints[] = {
"raw_syscalls:sys_enter",
"raw_syscalls:sys_exit",
};

That answers the quandry about having a zillion event fd's for all the syscalls.

Quoted 25 lines

static int strace(struct perf_record_opts *opts, const char *argv[])

{
	struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);

	err = perf_evlist__create_maps(evlist, opts->target_pid,
				       opts->target_tid, opts->cpu_list);

	err = perf_evlist__add_tracepoints_array(evlist, strace__tracepoints);

	err = perf_evlist__prepare_workload(evlist, opts, argv);

	perf_evlist__config_attrs(evlist, opts);

	err = perf_evlist__open(evlist, opts->group);

	err = perf_evlist__mmap(evlist, opts->mmap_pages, false);

	perf_evlist__start_workload(evlist);

	/*
	 * WIP: use perf_evlist__read, etc and then feed a
	 * perf_event_ops operations sorted events like is done in
	 * perf_session__process_events. Top is like that but currently
	 * doesn't use perf_session__process_events, we need to make it
Quoted 8 lines
	 * use something refactored out from perf_session that does the
	 * ordering of samples perf_session does, etc.
	 */

	perf_evlist__munmap(evlist);

	perf_evlist__delete(evlist);
}

Have you given thought to how you will compute the time spent in the syscalls?

I've been playing around with ideas on my end -- like storing 'last seen times' in struct thread {} and struct perf_evsel{}. For evsel it is CPU based, so something like:

struct perf_evsel {
...
u64 *last_time; /* time this event was last seen */
u32 max_cpu; /* highest cpu slot allocated */
};

And then methods to save/retrieve the time:

void perf_evsel__save_time(struct perf_evsel *evsel,

                                  u64 timestamp, u32 cpu)
{
	if ((cpu > evsel->max_cpu) || (evsel->last_time == NULL)) {
		unsigned int i;

		if (evsel->last_time)
			evsel->last_time = realloc(evsel->last_time,
				(cpu+1) * sizeof(u64));
		else
			evsel->last_time = malloc((cpu+1) * sizeof(u64));

		if (!evsel->last_time)
		    return;

		i = evsel->max_cpu ? evsel->max_cpu + 1 : 0;
		for (; i <= cpu; ++i)
			evsel->last_time[i] = (u64) 0;

		evsel->max_cpu = cpu;
	}

	evsel->last_time[cpu] = timestamp;
}

u64 perf_evsel__get_time(struct perf_evsel *evsel, u32 cpu)

{
	if (!evsel->last_time)
	    return 0;

	if ((cpu > evsel->max_cpu) || (evsel->last_time == NULL))
	    return 0;

	return evsel->last_time[cpu];
}

Saving the times allows to you compute delta-times -- between events and per tasks.

For raw syscalls that gets harder -- now you need to save times per cpu per syscall number. Or maybe save event histories within the thread struct.

I'll find some time to try out your trace branch over the long holiday weekend here.

David

#5 lore raw

Re: strace based on perf [Re: libperf interface stability]

Em Fri, Nov 25, 2011 at 10:33:35AM -0700, David Ahern escreveu:

Quoted 8 lines · David Ahern
Quoted 5 lines · Arnaldo Carvalho de Melo
Quoted 4 lines

I have a branch that I explicitely mark as unstable where I'm putting what I have at:

https://github.com/acmel/linux/commits/tmp.perf/trace4

I don't see the strace command in this branch.

Its not there yet, lemme check some stuff here and I'll push it, but still has ways to go. It'll be on tools/perf/builtin-trace.c

For the reference implementation I'll be picking code/ideas from:

https://github.com/acmel/linux/tree/tmp.perf/trace2

This is what Thomas, Ingo et al came to in '10.

That was done with the tooling infrastructure at that point in time, lotsa happened since then and there are still more to do for me to get into it confortably.

Take a look, probably some of your questions will be answered there.

- Arnaldo

#6 lore raw

Re: libperf interface stability

In reply to Arnaldo Carvalho de Melo
Quoted 17 lines · Arnaldo Carvalho de Melo

Em Thu, Nov 24, 2011 at 05:28:45PM +0100, Robert Richter escreveu:

Quoted 2 lines

It would be good to have a well defined, stable libperf interface for tools other than perf.

Agreed, an effort in this direction was the perf python binding, that so far includes only:

tools/perf/util/setup.py

	sources = ['util/python.c', 'util/ctype.c', 'util/evlist.c',
		   'util/evsel.c', 'util/cpumap.c', 'util/thread_map.c',
		   'util/util.c', 'util/xyarray.c', 'util/cgroup.c',
		   'util/debugfs.c'],

You need https://github.com/acmel/linux/commits/perf/urgent to build it tho, it has a fix for that:

Quoted 8 lines

In the end it generates a ~/build/perf/python/perf.so file (I use make -C tools/perf O=/home/acme/build/perf) that has all that is needed for simple tools like tools/perf/python/twatch.py.

I think of it as a precursor for a shared library for use with C and I have been working on untangling references such as the symbol_conf one you mentioned and making the 'perf_evlist' and 'perf_evsel' classes the main way to work with libperf.

Thanks Arnaldo, will then take this as a base to start with.

-Robert

Signature · 2 lines
Advanced Micro Devices, Inc.
Operating System Research Center
#7 lore raw

Re: libperf interface stability

Em Fri, Nov 25, 2011 at 11:45:35AM +0100, Robert Richter escreveu:

Quoted 22 lines · Arnaldo Carvalho de Melo
Quoted 20 lines

Em Thu, Nov 24, 2011 at 05:28:45PM +0100, Robert Richter escreveu:

Quoted 2 lines

It would be good to have a well defined, stable libperf interface for tools other than perf.

Agreed, an effort in this direction was the perf python binding, that so far includes only:

tools/perf/util/setup.py

	sources = ['util/python.c', 'util/ctype.c', 'util/evlist.c',
		   'util/evsel.c', 'util/cpumap.c', 'util/thread_map.c',
		   'util/util.c', 'util/xyarray.c', 'util/cgroup.c',
		   'util/debugfs.c'],

You need https://github.com/acmel/linux/commits/perf/urgent to build it tho, it has a fix for that:

In the end it generates a ~/build/perf/python/perf.so file (I use make -C tools/perf O=/home/acme/build/perf) that has all that is needed for simple tools like tools/perf/python/twatch.py.

Quoted 6 lines
Quoted 4 lines

I think of it as a precursor for a shared library for use with C and I have been working on untangling references such as the symbol_conf one you mentioned and making the 'perf_evlist' and 'perf_evsel' classes the main way to work with libperf.

Thanks Arnaldo, will then take this as a base to start with.

Good, if you could do something like:

1) get the above list of c files except for util/python.c that is just glue stuff for the python binding and make it the new libperf.a

2) make a perf.a that just is libperf.a + the rest of the .c files today in libperf.a, i.e. perf.a would be the "just for perf tools for now" bundle

3) make the python binding be just this new libperf.a + util/python.c

4) make libperf.so.0.0.1 be just what is in this new libperf.a

I think we would have a good start, committing just to what is in the python binding as the first iteration, then over time we can on a case ba casis, slowly, move more things there.

Then, if you use this new libperf.so.0.0.1 in some tool, lets check if the abstractions in place there are enough for your case, I'd really love to know if they are enough or have any major flaw :-)

Or if you just want to use them directly for initial experimenting, just link against that set of files removing python.c.

- Arnaldo

#8 lore raw

Re: libperf interface stability

Quoted 10 lines · Arnaldo Carvalho de Melo

1) get the above list of c files except for util/python.c that is just glue stuff for the python binding and make it the new libperf.a

2) make a perf.a that just is libperf.a + the rest of the .c files today in libperf.a, i.e. perf.a would be the "just for perf tools for now" bundle

3) make the python binding be just this new libperf.a + util/python.c

4) make libperf.so.0.0.1 be just what is in this new libperf.a

Yeah, I was thinking about something like this.

-Robert

Signature · 2 lines
Advanced Micro Devices, Inc.
Operating System Research Center