perf tools: Speed-up perf report by using multi thread (v2)
Namhyung Kim
maintainercover perf tools: Speed-up perf report by using multi thread (v2)
Hello,
This patchset converts perf report to use multiple threads in order to speed up the processing on large data files. I can see a minimum ~30% of speedup with this change. The code is still experimental and contains many rough edges. But I'd like to share and give some feedbacks.
The main change in this version is using single data file with an index table rather than using multiple files. It seems that single thread performance was improved by this than previous version but multi thread performance remains almost same.
The perf report processes (sample) events like below:
1. preprocess sample to get matching thread/dso/symbol info
2. insert it to hists rbtree (with callchain tree) based on the info
3. optionally collapse hist entries that match given sort key(s)
4. resort hist entries (by overhead) for output
5. display the hist entries
The stage 1 is a preprocessing and mostly act like a read-only operation in that it doesn't change a machine state during the sample processing. Meta events like fork, comm and mmap can change the machine/thread state but symbols can be loaded during the processing (stage 2).
The stage 2 consumes most of the time especially with callchains and
--children option is enabled. And this work can be easily patitioned
as each sample is independent to others. But the resulting hists must be combined/collapsed to a single global hists before going to further steps.
The stage 3 is optional and only needed by certain sort keys - but with stage 2 paralellized, it needs to be done always.
The stage 4 and 5 works on whole hists so must be done serially.
So my approach is like this:
Partially do stage 1 first - but only for meta events that changes machine state. To do this I add a dummy tracking event to perf record and make it collect such meta events only. They are saved as normal data and processed before sample events at perf report time.
This also requires to handle multiple sample data concurrently and to find a corresponding machine state when processing samples. On a large profiling session, many tasks were created and exited so pid might be recycled (even more than once!). To deal with it, I managed to have thread, map_groups and comm in time sorted. The only remaining thing is symbol loading as it's done lazily when sample requires it.
With that being done, the stage 2 can be done by multiple threads. I also save each sample data (per-cpu or per-thread) in separate files during record and then merge them into a single data file with an index table. On perf report time, each region of sample data will be processed by each thread. And symbol loading is protected by a mutex lock.
For DWARF post-unwinding, dso cache data also needs to be protected by a lock and this caused a huge contention. I made it to search the rbtree speculatively first and then, if it didn't find one, search it again under the dso lock. Please take a look at it if it's acceptable.
The patch 1-4 are independent fixes and cleans. The patch 5-14 are to support indexing for data file. With --index option, perf record will create a intermediate directory and then save meta events and sample data to separate files. And finally it'll build an index table and concatenate the data files.
The patch 15-26 are to manage machine and thread state using timestamp so that it can be searched when processing samples. The patch 27-40 are to implement parallel report. And finally I implemented 'perf data index' command to build an index table for a given data file.
This patchset didn't change perf record to use multi-thread. But I think it can be easily done later if needed.
Note that output has a slight difference to original version when compared using indexed data file. But they're mostly unresolved symbols for callchains.
Here is the result:
This is just elapsed time measured by 'perf stat -r 5'.
The data file was recorded during kernel build with fp callchain and size is 2.1GB. The machine has 6 core with hyper-threading enabled and I got a similar result on my laptop too.
perf report --children --no-children + --call-graph none
------------- ------------- -------------------
current 285.708340593 94.317412961 36.707232978 with index 253.322717665 77.079748639 24.892021523 + --multi-thread 174.037760271 44.717308080 8.300466711
This result is with 7.7GB data file using libunwind for callchain.
perf report --children --no-children + --call-graph none
------------- ------------- -------------------
current 247.070444039 196.393820003 5.068489333 with index 149.456483830 108.917644447 3.642109876 + --multi-thread 43.990095636 28.342798882 1.829218561
I guess the speedup of indexed data file came from skipping ordered event layer.
This result is with same file but using libdw for callchain unwind.
perf report --children --no-children + --call-graph none
------------- ------------- -------------------
current 465.661321115 496.153153039 4.629841428 with index 445.712762188 462.146612217 3.535147499 + --multi-thread 215.264706814 29.279996335 1.938137940
On my archlinux system, callchain unwind using libdw is much slower than libunwind. I'm using elfutils version 0.160. Also I don't know why --children takes less time than --no-children. Anyway we can see the --multi-thread performance is much better for each case.
You can get it from 'perf/threaded-v2' branch on my tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Please take a look and play with it. Any comments are welcome! :)
Thanks,
Namhyung
Jiri Olsa (1):
perf tools: Add new perf data command
Namhyung Kim (41):
perf tools: Support to read compressed module from build-id cache perf tools: Do not use __perf_session__process_events() directly perf record: Show precise number of samples
perf header: Set header version correctly
perf tools: Set attr.task bit for a tracking event
perf tools: Use a software dummy event to track task/mmap events perf tools: Use perf_data_file__fd() consistently
perf tools: Add rm_rf() utility function
perf tools: Introduce copyfile_offset() function
perf tools: Create separate mmap for dummy tracking event perf tools: Introduce perf_evlist__mmap_track()
perf tools: Add HEADER_DATA_INDEX feature
perf tools: Handle indexed data file properly
perf record: Add --index option for building index table perf report: Skip dummy tracking event
perf tools: Pass session arg to perf_event__preprocess_sample() perf script: Pass session arg to ->process_event callback perf tools: Introduce thread__comm_time() helpers
perf tools: Add a test case for thread comm handling
perf tools: Use thread__comm_time() when adding hist entries perf tools: Convert dead thread list into rbtree
perf tools: Introduce machine__find*_thread_time()
perf tools: Add a test case for timed thread handling perf tools: Maintain map groups list in a leader thread perf tools: Introduce thread__find_addr_location_time() and friends perf tools: Add a test case for timed map groups handling perf tools: Protect dso symbol loading using a mutex
perf tools: Protect dso cache tree using dso->lock
perf tools: Protect dso cache fd with a mutex
perf session: Pass struct events stats to event processing functions perf hists: Pass hists struct to hist_entry_iter functions perf tools: Move BUILD_ID_SIZE definition to perf.h
perf report: Parallelize perf report using multi-thread perf tools: Add missing_threads rb tree
perf record: Synthesize COMM event for a command line workload perf tools: Fix progress ui to support multi thread
perf report: Add --multi-thread option and config item perf session: Handle index files generally
perf tools: Convert lseek + read to pread
perf callchain: Save eh/debug frame offset for dwarf unwind perf data: Implement 'index' subcommand
tools/perf/Documentation/perf-data.txt | 44 +++ tools/perf/Documentation/perf-record.txt | 4 + tools/perf/Documentation/perf-report.txt | 3 + tools/perf/Makefile.perf | 4 + tools/perf/builtin-annotate.c | 8 +- tools/perf/builtin-data.c | 428 +++++++++++++++++++++ tools/perf/builtin-diff.c | 21 +- tools/perf/builtin-inject.c | 5 +- tools/perf/builtin-mem.c | 6 +- tools/perf/builtin-record.c | 261 +++++++++++-- tools/perf/builtin-report.c | 74 +++- tools/perf/builtin-script.c | 54 ++- tools/perf/builtin-timechart.c | 10 +- tools/perf/builtin-top.c | 7 +- tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/perf.c | 1 + tools/perf/perf.h | 2 + tools/perf/tests/builtin-test.c | 12 + tools/perf/tests/dso-data.c | 5 + tools/perf/tests/dwarf-unwind.c | 8 +- tools/perf/tests/hists_common.c | 3 +- tools/perf/tests/hists_cumulate.c | 6 +- tools/perf/tests/hists_filter.c | 5 +- tools/perf/tests/hists_link.c | 10 +- tools/perf/tests/hists_output.c | 6 +- tools/perf/tests/tests.h | 3 + tools/perf/tests/thread-comm.c | 47 +++ tools/perf/tests/thread-lookup-time.c | 180 +++++++++ tools/perf/tests/thread-mg-share.c | 7 +- tools/perf/tests/thread-mg-time.c | 88 +++++ tools/perf/ui/browsers/hists.c | 30 +- tools/perf/ui/gtk/hists.c | 3 + tools/perf/util/build-id.c | 9 +- tools/perf/util/build-id.h | 2 - tools/perf/util/db-export.c | 6 +- tools/perf/util/db-export.h | 4 +- tools/perf/util/dso.c | 159 +++++--- tools/perf/util/dso.h | 3 + tools/perf/util/event.c | 106 ++++- tools/perf/util/event.h | 13 +- tools/perf/util/evlist.c | 161 ++++++-- tools/perf/util/evlist.h | 22 +- tools/perf/util/evsel.c | 1 + tools/perf/util/evsel.h | 15 + tools/perf/util/header.c | 63 ++- tools/perf/util/header.h | 3 + tools/perf/util/hist.c | 121 ++++-- tools/perf/util/hist.h | 12 +- tools/perf/util/machine.c | 258 +++++++++++-- tools/perf/util/machine.h | 12 +- tools/perf/util/map.c | 1 + tools/perf/util/map.h | 2 + tools/perf/util/ordered-events.c | 4 +- .../perf/util/scripting-engines/trace-event-perl.c | 3 +- .../util/scripting-engines/trace-event-python.c | 5 +- tools/perf/util/session.c | 356 ++++++++++++++--- tools/perf/util/session.h | 11 +- tools/perf/util/symbol-elf.c | 13 +- tools/perf/util/symbol.c | 34 +- tools/perf/util/thread.c | 139 ++++++- tools/perf/util/thread.h | 28 +- tools/perf/util/tool.h | 14 + tools/perf/util/trace-event-scripting.c | 3 +- tools/perf/util/trace-event.h | 3 +- tools/perf/util/unwind-libdw.c | 11 +- tools/perf/util/unwind-libunwind.c | 49 ++- tools/perf/util/util.c | 81 +++- tools/perf/util/util.h | 2 + 69 files changed, 2662 insertions(+), 414 deletions(-) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c create mode 100644 tools/perf/tests/thread-comm.c create mode 100644 tools/perf/tests/thread-lookup-time.c create mode 100644 tools/perf/tests/thread-mg-time.c
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 1/42 perf tools: Support to read compressed module from build-id cache
The commit c00c48fc6e6e ("perf symbols: Preparation for compressed kernel module support") added support for compressed kernel modules but it only supports system path DSOs. When a dso is read from build-id cache, its filename doesn't end with ".gz" but has build-id. In this case, we should fallback to the original dso->name.
--- tools/perf/util/symbol-elf.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/util/symbol-elf.c +8 -5
@@ -574,13 +574,16 @@ static int decompress_kmodule(struct dso *dso, const char *name,const char *ext =strrchr (name,'.' );char tmpbuf[] ="/tmp/perf-kmod-XXXXXX" ; -if ((type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && - type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP ) || - type != dso->symtab_type) +if (type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && + type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP && + type !=DSO_BINARY_TYPE__BUILD_ID_CACHE )return -1 ; -if (!ext || !is_supported_compression (ext +1 )) -return -1 ; +if (!ext || !is_supported_compression (ext +1 )) { + ext =strrchr (dso->name,'.' ); +if (!ext || !is_supported_compression (ext +1 )) +return -1 ; + } fd =mkstemp (tmpbuf);if (fd <0 )
Signature · 1 line
2.2.2
Re: [PATCH 01/42] perf tools: Support to read compressed module from build-id cache
Quoted 16 lines · Namhyung Kim
The commit
c00c48fc6e6e("perf symbols: Preparation for compressed kernel module support") added support for compressed kernel modules but it only supports system path DSOs. When a dso is read from build-id cache, its filename doesn't end with ".gz" but has build-id. In this case, we should fallback to the originaldso->name.Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/symbol-elf.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)tools/perf/util/symbol-elf.c +0
-0
tools/perf/util/symbol-elf.c static int decompress_kmodule(struct dso *dso, const char *name, · line 580
Quoted 10 lines
tools/perf/util/symbol-elf.c +3
-3@@ -574,13 +574,16 @@ static int decompress_kmodule(struct dso *dso, const char *name,const char *ext =strrchr (name,'.' );char tmpbuf[] ="/tmp/perf-kmod-XXXXXX" ; -if ((type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && - type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP ) || - type != dso->symtab_type) +if (type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && + type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP && + type !=DSO_BINARY_TYPE__BUILD_ID_CACHE )return -1 ;
hum, is it possible the type == DSO_BINARY_TYPE__BUILD_ID_CACHE could get in here?
---
for (i = 0; i < DSO_BINARY_TYPE__SYMTAB_CNT; i++) {
struct symsrc *ss = &ss_[ss_pos];
bool next_slot = false;
enum dso_binary_type symtab_type = binary_type_symtab[i];
if (!dso__is_compatible_symtab_type(dso, kmod, symtab_type))
continue;--- ^^^ this check should rule out buildid symtab_type for kmod dso?
symsrc__init(
I wonder wether we should set special type from compressed binaries (as of now), or instead try to decompress anything that looks like it's compressed ;-) it seems more to be more generic and could simplify the code..
jirka
Namhyung Kim
maintainerRe: [PATCH 01/42] perf tools: Support to read compressed module from build-id cache
Quoted 41 lines · Jiri Olsa
Quoted 16 lines · Namhyung Kim
The commit
c00c48fc6e6e("perf symbols: Preparation for compressed kernel module support") added support for compressed kernel modules but it only supports system path DSOs. When a dso is read from build-id cache, its filename doesn't end with ".gz" but has build-id. In this case, we should fallback to the originaldso->name.Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/symbol-elf.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)tools/perf/util/symbol-elf.c +0
-0tools/perf/util/symbol-elf.c static int decompress_kmodule(struct dso *dso, const char *name, · line 580
Quoted 10 lines
tools/perf/util/symbol-elf.c +3
-3@@ -574,13 +574,16 @@ static int decompress_kmodule(struct dso *dso, const char *name,const char *ext =strrchr (name,'.' );char tmpbuf[] ="/tmp/perf-kmod-XXXXXX" ; -if ((type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && - type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP ) || - type != dso->symtab_type) +if (type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && + type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP && + type !=DSO_BINARY_TYPE__BUILD_ID_CACHE )return -1 ;hum, is it possible the type ==
DSO_BINARY_TYPE__BUILD_ID_CACHEcould get in here?---
for (i = 0; i < DSO_BINARY_TYPE__SYMTAB_CNT; i++) { struct symsrc *ss = &ss_[ss_pos]; bool next_slot = false; enum dso_binary_type symtab_type = binary_type_symtab[i]; if (!dso__is_compatible_symtab_type(dso, kmod, symtab_type)) continue;
Quoted 1 line
--- ^^^ this check should rule out buildid
symtab_typefor kmod dso?
AFAICS symtab_type of BUILD_ID_CACHE always returns true in this function.
Quoted 7 lines
symsrc__init(I wonder wether we should set special type from compressed binaries (as of now), or instead try to decompress anything that looks like it's compressed ;-) it seems more to be more generic and could simplify the code..
I don't know. But it seems only kernel modules are compressed now. If user-level dso also supports compression, we need to think about it again..
Thanks,
Namhyung
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf symbols: Support to read compressed module from build-id cache
In reply to Namhyung Kimperf symbols: Support to read compressed module from build-id cache
Commit-ID: 0b064f43001fc2627ff8c3020647b85db040235f Gitweb: http://git.kernel.org/tip/0b064f43001fc2627ff8c3020647b85db040235f Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:06:42 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:56:54 -0300
perf symbols: Support to read compressed module from build-id cache
The commit c00c48fc6e6e ("perf symbols: Preparation for compressed kernel module support") added support for compressed kernel modules but it only supports system path DSOs. When a dso is read from build-id cache, its filename doesn't end with ".gz" but has build-id. In this case, we should fallback to the original dso->name.
--- tools/perf/util/symbol-elf.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/util/symbol-elf.c +8 -5
@@ -574,13 +574,16 @@ static int decompress_kmodule(struct dso *dso, const char *name,const char *ext =strrchr (name,'.' );char tmpbuf[] ="/tmp/perf-kmod-XXXXXX" ; -if ((type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && - type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP ) || - type != dso->symtab_type) +if (type !=DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP && + type !=DSO_BINARY_TYPE__GUEST_KMODULE_COMP && + type !=DSO_BINARY_TYPE__BUILD_ID_CACHE )return -1 ; -if (!ext || !is_supported_compression (ext +1 )) -return -1 ; +if (!ext || !is_supported_compression (ext +1 )) { + ext =strrchr (dso->name,'.' ); +if (!ext || !is_supported_compression (ext +1 )) +return -1 ; + } fd =mkstemp (tmpbuf);if (fd <0 )
Namhyung Kim
maintainerPATCH 2/42 perf tools: Do not use __perf_session__process_events() directly
In reply to Namhyung KimIt's only used for perf record to process build-id because its file size it's not fixed at this time due to remaining header features. However data offset and size is available so that we can use the perf_session__process_events() once we set the file size as the current offset like for now.
--- tools/perf/builtin-record.c | 7 +++---- tools/perf/util/session.c | 6 +++--- tools/perf/util/session.h | 3 --- 3 files changed, 6 insertions(+), 10 deletions(-)
tools/perf/builtin-record.c +3 -4
@@ -194,12 +194,13 @@ static int process_buildids(struct record *rec) {struct perf_data_file *file = &rec->file;struct perf_session *session = rec->session; -u64 start = session->header.data_offset;u64 size =lseek (file->fd,0 ,SEEK_CUR );if (size ==0 )return 0 ; + file->size = size; +/* * During this process, it'll load kernel map and replace the * dso->long_name to a real pathname it found. In this case
@@ -211,9 +212,7 @@ static int process_buildids(struct record *rec)*/ symbol_conf.ignore_vmlinux_buildid = true; -return __perf_session__process_events (session, start, - size - start, - size, &build_id__mark_dso_hit_ops); +return perf_session__process_events (session, &build_id__mark_dso_hit_ops); }static void perf_event__synthesize_guest_os (struct machine *machine,void *data)
tools/perf/util/session.c +3 -3
@@ -1251,9 +1251,9 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif -int __perf_session__process_events (struct perf_session *session, -u64 data_offset,u64 data_size, -u64 file_size,struct perf_tool *tool) +static int __perf_session__process_events (struct perf_session *session, +u64 data_offset,u64 data_size, +u64 file_size,struct perf_tool *tool) {int fd =perf_data_file__fd (session->file);u64 head, page_offset, file_offset, file_pos, size;
tools/perf/util/session.h +0 -3
@@ -49,9 +49,6 @@ int perf_session__peek_event(struct perf_session *session, off_t file_offset,union perf_event **event_ptr, struct perf_sample *sample); -int __perf_session__process_events (struct perf_session *session, -u64 data_offset,u64 data_size,u64 size, -struct perf_tool *tool);int perf_session__process_events (struct perf_session *session,struct perf_tool *tool);
Signature · 1 line
2.2.2
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf tools: Do not use __perf_session__process_events() directly
perf tools: Do not use __perf_session__process_events() directly
Commit-ID: 4ac30cf74b308fb01338e660d3471cd490a7958a Gitweb: http://git.kernel.org/tip/4ac30cf74b308fb01338e660d3471cd490a7958a Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:06:43 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:36:32 -0300
perf tools: Do not use __perf_session__process_events() directly
It's only used for perf record to process build-id because its file size it's not fixed at this time due to remaining header features.
However data offset and size is available so that we can use the perf_session__process_events() once we set the file size as the current offset like for now.
--- tools/perf/builtin-record.c | 7 +++---- tools/perf/util/session.c | 6 +++--- tools/perf/util/session.h | 3 --- 3 files changed, 6 insertions(+), 10 deletions(-)
tools/perf/builtin-record.c +3 -4
@@ -194,12 +194,13 @@ static int process_buildids(struct record *rec) {struct perf_data_file *file = &rec->file;struct perf_session *session = rec->session; -u64 start = session->header.data_offset;u64 size =lseek (file->fd,0 ,SEEK_CUR );if (size ==0 )return 0 ; + file->size = size; +/* * During this process, it'll load kernel map and replace the * dso->long_name to a real pathname it found. In this case
@@ -211,9 +212,7 @@ static int process_buildids(struct record *rec)*/ symbol_conf.ignore_vmlinux_buildid = true; -return __perf_session__process_events (session, start, - size - start, - size, &build_id__mark_dso_hit_ops); +return perf_session__process_events (session, &build_id__mark_dso_hit_ops); }static void perf_event__synthesize_guest_os (struct machine *machine,void *data)
tools/perf/util/session.c +3 -3
@@ -1251,9 +1251,9 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif -int __perf_session__process_events (struct perf_session *session, -u64 data_offset,u64 data_size, -u64 file_size,struct perf_tool *tool) +static int __perf_session__process_events (struct perf_session *session, +u64 data_offset,u64 data_size, +u64 file_size,struct perf_tool *tool) {int fd =perf_data_file__fd (session->file);u64 head, page_offset, file_offset, file_pos, size;
tools/perf/util/session.h +0 -3
@@ -49,9 +49,6 @@ int perf_session__peek_event(struct perf_session *session, off_t file_offset,union perf_event **event_ptr, struct perf_sample *sample); -int __perf_session__process_events (struct perf_session *session, -u64 data_offset,u64 data_size,u64 size, -struct perf_tool *tool);int perf_session__process_events (struct perf_session *session,struct perf_tool *tool);
Namhyung Kim
maintainerPATCH 3/42 perf record: Show precise number of samples
In reply to Namhyung KimAfter perf record finishes, it prints file size and number of samples in the file but this info is wrong since it assumes typical sample size of 24 bytes and divides file size by the value.
However as we post-process recorded samples for build-id, it can show correct number like below. If build-id post-processing is not requested just omit the wrong number of samples.
$ perf record noploop 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.159 MB perf.data (3989 samples) ] $ perf report --stdio -n
# To display the perf.data header info, please use --header/--header-only options.
#
# Samples: 3K of event 'cycles'
# Event count (approx.): 3771330663
#
# Overhead Samples Command Shared Object Symbol
# ........ ............ ....... ................ ..........................
#
99.90% 3982 noploop noploop [.] main
0.09% 1 noploop ld-2.17.so [.] _dl_check_map_versions
0.01% 1 noploop [kernel.vmlinux] [k] setup_arg_pages
0.00% 5 noploop [kernel.vmlinux] [k] intel_pmu_enable_all--- tools/perf/builtin-record.c | 51 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 12 deletions(-)
tools/perf/builtin-record.c +39 -12
@@ -190,6 +190,19 @@ static int record__open(struct record *rec)return rc; } +static int process_sample_event (struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct perf_evsel *evsel, +struct machine *machine) +{ + struct record *rec = container_of(tool, struct record, tool); + + rec->samples++; + +return build_id__mark_dso_hit (tool, event, sample, evsel, machine); +} +static int process_buildids (struct record *rec) {struct perf_data_file *file = &rec->file;
@@ -212,7 +225,7 @@ static int process_buildids(struct record *rec)*/ symbol_conf.ignore_vmlinux_buildid = true; -return perf_session__process_events (session, &build_id__mark_dso_hit_ops); +return perf_session__process_events (session, &rec->tool); }static void perf_event__synthesize_guest_os (struct machine *machine,void *data)
@@ -503,19 +516,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) goto out_child; } -if (!quiet) { +if (!quiet)fprintf (stderr,"[ perf record: Woken up %ld times to write data ]\n" , waking); -/* -* Approximate RIP event size: 24 bytes. -*/ -fprintf (stderr, -"[ perf record: Captured and wrote %.3f MB %s (~%" PRIu64" samples) ]\n" , - (double )rec->bytes_written /1024.0 /1024.0 , - file->path, - rec->bytes_written /24 ); - } - out_child:if (forks) {int exit_status;
@@ -534,6 +537,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) }else status = err; +/* this will be recalculated during process_buildids() */ + rec->samples =0 ; +if (!err && !file->is_pipe) { rec->session->header.data_size += rec->bytes_written;
@@ -543,6 +549,20 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) file->fd, true); } +if (!err && !quiet) { +char samples[128 ]; + +if (rec->samples) +scnprintf (samples,sizeof (samples), +" (%" PRIu64" samples)" , rec->samples); +else + samples[0 ] ='\0' ; + +fprintf (stderr,"[ perf record: Captured and wrote %.3f MB %s%s ]\n" , +perf_data_file__size (file) /1024.0 /1024.0 , + file->path, samples); + } + out_delete_session:perf_session__delete (session);return status;
@@ -719,6 +739,13 @@ static struct record record = { .default_per_cpu = true, }, }, + .tool = { + .sample = process_sample_event, + .fork = perf_event__process_fork, + .comm = perf_event__process_comm, + .mmap = perf_event__process_mmap, + .mmap2 = perf_event__process_mmap2, + }, };#define CALLCHAIN_HELP "setup and enables call-graph (stack chain/backtrace) recording: "
Signature · 1 line
2.2.2
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf record: Show precise number of samples
perf record: Show precise number of samples
Commit-ID: e3d5911221f5cf71e1f0306256d4e42d34a365d2 Gitweb: http://git.kernel.org/tip/e3d5911221f5cf71e1f0306256d4e42d34a365d2 Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:06:44 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:37:20 -0300
perf record: Show precise number of samples
After perf record finishes, it prints file size and number of samples in the file but this info is wrong since it assumes typical sample size of 24 bytes and divides file size by the value.
However as we post-process recorded samples for build-id, it can show correct number like below. If build-id post-processing is not requested just omit the wrong number of samples.
$ perf record noploop 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.159 MB perf.data (3989 samples) ] $ perf report --stdio -n
# To display the perf.data header info, please use --header/--header-only options.
#
# Samples: 3K of event 'cycles'
# Event count (approx.): 3771330663
#
# Overhead Samples Command Shared Object Symbol
# ........ ............ ....... ................ ..........................
#
99.90% 3982 noploop noploop [.] main
0.09% 1 noploop ld-2.17.so [.] _dl_check_map_versions
0.01% 1 noploop [kernel.vmlinux] [k] setup_arg_pages
0.00% 5 noploop [kernel.vmlinux] [k] intel_pmu_enable_all--- tools/perf/builtin-record.c | 51 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 12 deletions(-)
tools/perf/builtin-record.c +39 -12
@@ -190,6 +190,19 @@ out:return rc; } +static int process_sample_event (struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct perf_evsel *evsel, +struct machine *machine) +{ + struct record *rec = container_of(tool, struct record, tool); + + rec->samples++; + +return build_id__mark_dso_hit (tool, event, sample, evsel, machine); +} +static int process_buildids (struct record *rec) {struct perf_data_file *file = &rec->file;
@@ -212,7 +225,7 @@ static int process_buildids(struct record *rec)*/ symbol_conf.ignore_vmlinux_buildid = true; -return perf_session__process_events (session, &build_id__mark_dso_hit_ops); +return perf_session__process_events (session, &rec->tool); }static void perf_event__synthesize_guest_os (struct machine *machine,void *data)
@@ -503,19 +516,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) goto out_child; } -if (!quiet) { +if (!quiet)fprintf (stderr,"[ perf record: Woken up %ld times to write data ]\n" , waking); -/* -* Approximate RIP event size: 24 bytes. -*/ -fprintf (stderr, -"[ perf record: Captured and wrote %.3f MB %s (~%" PRIu64" samples) ]\n" , - (double )rec->bytes_written /1024.0 /1024.0 , - file->path, - rec->bytes_written /24 ); - } - out_child:if (forks) {int exit_status;
@@ -534,6 +537,9 @@ out_child: }else status = err; +/* this will be recalculated during process_buildids() */ + rec->samples =0 ; +if (!err && !file->is_pipe) { rec->session->header.data_size += rec->bytes_written;
@@ -543,6 +549,20 @@ out_child: file->fd, true); } +if (!err && !quiet) { +char samples[128 ]; + +if (rec->samples) +scnprintf (samples,sizeof (samples), +" (%" PRIu64" samples)" , rec->samples); +else + samples[0 ] ='\0' ; + +fprintf (stderr,"[ perf record: Captured and wrote %.3f MB %s%s ]\n" , +perf_data_file__size (file) /1024.0 /1024.0 , + file->path, samples); + } + out_delete_session:perf_session__delete (session);return status;
@@ -719,6 +739,13 @@ static struct record record = { .default_per_cpu = true, }, }, + .tool = { + .sample = process_sample_event, + .fork = perf_event__process_fork, + .comm = perf_event__process_comm, + .mmap = perf_event__process_mmap, + .mmap2 = perf_event__process_mmap2, + }, };#define CALLCHAIN_HELP "setup and enables call-graph (stack chain/backtrace) recording: "
Namhyung Kim
maintainerPATCH 4/42 perf header: Set header version correctly
In reply to Namhyung KimWhen check_magic_endian() is called, it checks the magic number in the perf data file to determine version and endianness. But if it uses a same endian the verison number wasn't updated and makes confusion.
--- tools/perf/util/header.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/header.c +1 -1
@@ -2237,6 +2237,7 @@ static int check_magic_endian(u64 magic, uint64_t hdr_sz,* - unique number to identify actual perf.data files * - encode endianness of file */ + ph->version =PERF_HEADER_VERSION_2 ;/* check magic number with one endianness */ if (magic == __perf_magic2)
@@ -2247,7 +2248,6 @@ static int check_magic_endian(u64 magic, uint64_t hdr_sz,return -1 ; ph->needs_swap = true; - ph->version =PERF_HEADER_VERSION_2 ;return 0 ; }
Signature · 1 line
2.2.2
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf header: Set header version correctly
perf header: Set header version correctly
Commit-ID: f7913971bdad1a72c6158074786babed477d61e2 Gitweb: http://git.kernel.org/tip/f7913971bdad1a72c6158074786babed477d61e2 Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:06:45 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:53:11 -0300
perf header: Set header version correctly
When check_magic_endian() is called, it checks the magic number in the perf data file to determine version and endianness. But if it uses a same endian the verison number wasn't updated and makes confusion.
--- tools/perf/util/header.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/header.c +1 -1
@@ -2237,6 +2237,7 @@ static int check_magic_endian(u64 magic, uint64_t hdr_sz,* - unique number to identify actual perf.data files * - encode endianness of file */ + ph->version =PERF_HEADER_VERSION_2 ;/* check magic number with one endianness */ if (magic == __perf_magic2)
@@ -2247,7 +2248,6 @@ static int check_magic_endian(u64 magic, uint64_t hdr_sz,return -1 ; ph->needs_swap = true; - ph->version =PERF_HEADER_VERSION_2 ;return 0 ; }
Namhyung Kim
maintainerPATCH 5/42 perf tools: Set attr.task bit for a tracking event
In reply to Namhyung KimThe perf_event_attr.task bit is to track task (fork and exit) events but it missed to be set by perf_evsel__config(). While it was not a problem in practice since setting other bits (comm/mmap) ended up being in same result, it'd be good to set it explicitly anyway.
The attr->task is to track task related events (fork/exit) only but other meta events like comm and mmap[2] also needs the task events. So setting attr->comm and/or attr->mmap causes the kernel emits the task events anyway. So the attr->task is only meaningful when other bits are off but I'd like to set it for completeness.
--- tools/perf/util/evsel.c | 1 + 1 file changed, 1 insertion(+)
tools/perf/util/evsel.c +1 -0
@@ -709,6 +709,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)if (opts->sample_weight)perf_evsel__set_sample_bit (evsel,WEIGHT ); + attr->task = track; attr->mmap = track; attr->mmap2 = track && !perf_missing_features.mmap2; attr->comm = track;
Signature · 1 line
2.2.2
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf evsel: Set attr.task bit for a tracking event
perf evsel: Set attr.task bit for a tracking event
Commit-ID: 62e503b7ed98fcdf16308cda0b5378e7840f4339 Gitweb: http://git.kernel.org/tip/62e503b7ed98fcdf16308cda0b5378e7840f4339 Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:06:46 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:54:59 -0300
perf evsel: Set attr.task bit for a tracking event
The perf_event_attr.task bit is to track task (fork and exit) events but it missed to be set by perf_evsel__config(). While it was not a problem in practice since setting other bits (comm/mmap) ended up being in same result, it'd be good to set it explicitly anyway.
The attr->task is to track task related events (fork/exit) only but other meta events like comm and mmap[2] also needs the task events. So setting attr->comm and/or attr->mmap causes the kernel emits the task events anyway. So the attr->task is only meaningful when other bits are off but I'd like to set it for completeness.
--- tools/perf/util/evsel.c | 1 + 1 file changed, 1 insertion(+)
tools/perf/util/evsel.c +1 -0
@@ -709,6 +709,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)if (opts->sample_weight)perf_evsel__set_sample_bit (evsel,WEIGHT ); + attr->task = track; attr->mmap = track; attr->mmap2 = track && !perf_missing_features.mmap2; attr->comm = track;
Namhyung Kim
maintainerPATCH 6/42 perf tools: Use a software dummy event to track task/mmap events
In reply to Namhyung KimAdd APIs for software dummy event to track task/comm/mmap events separately. The perf record will use them to save such events in a separate mmap buffer to make it easy to index. This is a preparation of multi-thread support which will come later.
--- tools/perf/util/evlist.c | 30 ++++++++++++++++++++++++++++++ tools/perf/util/evlist.h | 1 + tools/perf/util/evsel.h | 15 +++++++++++++++ 3 files changed, 46 insertions(+)
tools/perf/util/evlist.c +30 -0
@@ -195,6 +195,36 @@ int perf_evlist__add_default(struct perf_evlist *evlist)return -ENOMEM ; } +int perf_evlist__add_dummy_tracking (struct perf_evlist *evlist) +{ +struct perf_event_attr attr = { + .type =PERF_TYPE_SOFTWARE , + .config =PERF_COUNT_SW_DUMMY , + .exclude_kernel =1 , + }; +struct perf_evsel *evsel; + +event_attr_init (&attr); + + evsel =perf_evsel__new (&attr); +if (evsel ==NULL ) + goto error; + +/* use strdup() because free(evsel) assumes name is allocated */ + evsel->name =strdup ("dummy" ); +if (!evsel->name) + goto error_free; + +perf_evlist__add (evlist, evsel); +perf_evlist__set_tracking_event (evlist, evsel); + +return 0 ; +error_free: +perf_evsel__delete (evsel); +error: +return -ENOMEM ; +} +static int perf_evlist__add_attrs (struct perf_evlist *evlist,struct perf_event_attr *attrs,size_t nr_attrs) {
tools/perf/util/evlist.h +1 -0
@@ -67,6 +67,7 @@ void perf_evlist__delete(struct perf_evlist *evlist);void perf_evlist__add (struct perf_evlist *evlist,struct perf_evsel *entry);int perf_evlist__add_default (struct perf_evlist *evlist); +int perf_evlist__add_dummy_tracking (struct perf_evlist *evlist);int __perf_evlist__add_default_attrs (struct perf_evlist *evlist,struct perf_event_attr *attrs,size_t nr_attrs);
tools/perf/util/evsel.h +15 -0
@@ -331,6 +331,21 @@ static inline bool perf_evsel__is_function_event(struct perf_evsel *evsel)#undef FUNCTION_EVENT } +/** +* perf_evsel__is_dummy_tracking - Return whether given evsel is a dummy +* event for tracking meta events only +* +* @evsel - evsel selector to be tested +* +* Return %true if event is a dummy tracking event +*/ +static inline bool perf_evsel__is_dummy_tracking (struct perf_evsel *evsel) +{ +return evsel->attr.type ==PERF_TYPE_SOFTWARE && + evsel->attr.config ==PERF_COUNT_SW_DUMMY && + evsel->attr.task ==1 && evsel->attr.mmap ==1 ; +} +struct perf_attr_details {bool freq;bool verbose;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 7/42 perf tools: Use perf_data_file__fd() consistently
In reply to Namhyung KimDo not reference file->fd directly since we want hide the implementation details from outside for possible future changes.
--- tools/perf/builtin-inject.c | 5 +++-- tools/perf/builtin-record.c | 14 +++++++------- 2 files changed, 10 insertions(+), 9 deletions(-)
tools/perf/builtin-inject.c +3 -2
@@ -343,6 +343,7 @@ static int __cmd_inject(struct perf_inject *inject)int ret = -EINVAL ;struct perf_session *session = inject->session;struct perf_data_file *file_out = &inject->output; +int fd =perf_data_file__fd (file_out);signal (SIGINT , sig_handler);
@@ -376,7 +377,7 @@ static int __cmd_inject(struct perf_inject *inject) }if (!file_out->is_pipe) -lseek (file_out->fd, session->header.data_offset,SEEK_SET ); +lseek (fd, session->header.data_offset,SEEK_SET ); ret =perf_session__process_events (session, &inject->tool);
@@ -385,7 +386,7 @@ static int __cmd_inject(struct perf_inject *inject)perf_header__set_feat (&session->header,HEADER_BUILD_ID ); session->header.data_size = inject->bytes_written; -perf_session__write_header (session, session->evlist, file_out->fd, true); +perf_session__write_header (session, session->evlist, fd, true); }return ret;
tools/perf/builtin-record.c +7 -7
@@ -208,7 +208,7 @@ static int process_buildids(struct record *rec)struct perf_data_file *file = &rec->file;struct perf_session *session = rec->session; -u64 size =lseek (file->fd,0 ,SEEK_CUR ); +u64 size =lseek (perf_data_file__fd (file),0 ,SEEK_CUR );if (size ==0 )return 0 ;
@@ -334,6 +334,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)struct perf_data_file *file = &rec->file;struct perf_session *session;bool disabled = false, draining = false; +int fd; rec->progname = argv[0 ];
@@ -348,6 +349,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)return -1 ; } + fd =perf_data_file__fd (file); rec->session = session;record__init_features (rec);
@@ -372,12 +374,11 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)perf_header__clear_feat (&session->header,HEADER_GROUP_DESC );if (file->is_pipe) { - err =perf_header__write_pipe (file->fd); + err =perf_header__write_pipe (fd);if (err <0 ) goto out_child; }else { - err =perf_session__write_header (session, rec->evlist, - file->fd, false); + err =perf_session__write_header (session, rec->evlist, fd, false);if (err <0 ) goto out_child; }
@@ -409,7 +410,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)* return this more properly and also * propagate errors that now are calling die() */ - err =perf_event__synthesize_tracing_data (tool, file->fd, rec->evlist, + err =perf_event__synthesize_tracing_data (tool, fd, rec->evlist, process_synthesized_event);if (err <=0 ) {pr_err ("Couldn't record tracing data.\n" );
@@ -545,8 +546,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (!rec->no_buildid)process_buildids (rec); -perf_session__write_header (rec->session, rec->evlist, - file->fd, true); +perf_session__write_header (rec->session, rec->evlist, fd, true); }if (!err && !quiet) {
Signature · 1 line
2.2.2
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf tools: Use perf_data_file__fd() consistently
perf tools: Use perf_data_file__fd() consistently
Commit-ID: 42aa276f40730211383e9a9923416f1fb9841d68 Gitweb: http://git.kernel.org/tip/42aa276f40730211383e9a9923416f1fb9841d68 Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:06:48 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:58:24 -0300
perf tools: Use perf_data_file__fd() consistently
Do not reference file->fd directly since we want hide the implementation details from outside for possible future changes.
--- tools/perf/builtin-inject.c | 5 +++-- tools/perf/builtin-record.c | 14 +++++++------- 2 files changed, 10 insertions(+), 9 deletions(-)
tools/perf/builtin-inject.c +3 -2
@@ -343,6 +343,7 @@ static int __cmd_inject(struct perf_inject *inject)int ret = -EINVAL ;struct perf_session *session = inject->session;struct perf_data_file *file_out = &inject->output; +int fd =perf_data_file__fd (file_out);signal (SIGINT , sig_handler);
@@ -376,7 +377,7 @@ static int __cmd_inject(struct perf_inject *inject) }if (!file_out->is_pipe) -lseek (file_out->fd, session->header.data_offset,SEEK_SET ); +lseek (fd, session->header.data_offset,SEEK_SET ); ret =perf_session__process_events (session, &inject->tool);
@@ -385,7 +386,7 @@ static int __cmd_inject(struct perf_inject *inject)perf_header__set_feat (&session->header,HEADER_BUILD_ID ); session->header.data_size = inject->bytes_written; -perf_session__write_header (session, session->evlist, file_out->fd, true); +perf_session__write_header (session, session->evlist, fd, true); }return ret;
tools/perf/builtin-record.c +7 -7
@@ -208,7 +208,7 @@ static int process_buildids(struct record *rec)struct perf_data_file *file = &rec->file;struct perf_session *session = rec->session; -u64 size =lseek (file->fd,0 ,SEEK_CUR ); +u64 size =lseek (perf_data_file__fd (file),0 ,SEEK_CUR );if (size ==0 )return 0 ;
@@ -334,6 +334,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)struct perf_data_file *file = &rec->file;struct perf_session *session;bool disabled = false, draining = false; +int fd; rec->progname = argv[0 ];
@@ -348,6 +349,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)return -1 ; } + fd =perf_data_file__fd (file); rec->session = session;record__init_features (rec);
@@ -372,12 +374,11 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)perf_header__clear_feat (&session->header,HEADER_GROUP_DESC );if (file->is_pipe) { - err =perf_header__write_pipe (file->fd); + err =perf_header__write_pipe (fd);if (err <0 ) goto out_child; }else { - err =perf_session__write_header (session, rec->evlist, - file->fd, false); + err =perf_session__write_header (session, rec->evlist, fd, false);if (err <0 ) goto out_child; }
@@ -409,7 +410,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)* return this more properly and also * propagate errors that now are calling die() */ - err =perf_event__synthesize_tracing_data (tool, file->fd, rec->evlist, + err =perf_event__synthesize_tracing_data (tool, fd, rec->evlist, process_synthesized_event);if (err <=0 ) {pr_err ("Couldn't record tracing data.\n" );
@@ -545,8 +546,7 @@ out_child:if (!rec->no_buildid)process_buildids (rec); -perf_session__write_header (rec->session, rec->evlist, - file->fd, true); +perf_session__write_header (rec->session, rec->evlist, fd, true); }if (!err && !quiet) {
Namhyung Kim
maintainerPATCH 8/42 perf tools: Add rm_rf() utility function
In reply to Namhyung KimThe rm_rf() function does same as the shell command 'rm -rf' which removes all directory entries recursively.
--- tools/perf/util/util.c | 43 +++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/util.h | 1 + 2 files changed, 44 insertions(+)
tools/perf/util/util.c +43 -0
@@ -72,6 +72,49 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } +int rm_rf (char *path) +{ +DIR *dir; +int ret =0 ; +struct dirent *d; +char namebuf[PATH_MAX ]; + + dir =opendir (path); +if (dir ==NULL ) +return 0 ; + +while ((d =readdir (dir)) !=NULL && !ret) { +struct stat statbuf; + +if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." )) +continue ; + +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , + path, d->d_name); + + ret =stat (namebuf, &statbuf); +if (ret <0 ) { +pr_debug ("stat failed: %s\n" , namebuf); +break ; + } + +if (S_ISREG (statbuf.st_mode)) + ret =unlink (namebuf); +else if (S_ISDIR (statbuf.st_mode)) + ret =rm_rf (namebuf); +else { +pr_debug ("unknown file: %s\n" , namebuf); + ret =-1 ; + } + } +closedir (dir); + +if (ret <0 ) +return ret; + +return rmdir (path); +} +static int slow_copyfile (const char *from,const char *to,mode_t mode) {int err =-1 ;
tools/perf/util/util.h +1 -0
@@ -248,6 +248,7 @@ static inline int sane_case(int x, int high) }int mkdir_p (char *path,mode_t mode); +int rm_rf (char *path);int copyfile (const char *from,const char *to);int copyfile_mode (const char *from,const char *to,mode_t mode);
Signature · 1 line
2.2.2
Re: [PATCH 08/42] perf tools: Add rm_rf() utility function
Quoted 2 lines · Namhyung Kim
The
rm_rf()function does same as the shell command 'rm -rf' which removes all directory entries recursively.
thanks,
jirka
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf tools: Add rm_rf() utility function
perf tools: Add rm_rf() utility function
Commit-ID: 0b1de0be1eac7b23e89cb43c17b02d38ead6b6c8 Gitweb: http://git.kernel.org/tip/0b1de0be1eac7b23e89cb43c17b02d38ead6b6c8 Author: Namhyung Kim <[email protected]> AuthorDate: Mon, 18 May 2015 09:30:17 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 18 May 2015 10:17:34 -0300
perf tools: Add rm_rf() utility function
The rm_rf() function does same as the shell command 'rm -rf' which removes all directory entries recursively.
--- tools/perf/util/util.c | 43 +++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/util.h | 1 + 2 files changed, 44 insertions(+)
tools/perf/util/util.c +43 -0
@@ -72,6 +72,49 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } +int rm_rf (char *path) +{ +DIR *dir; +int ret =0 ; +struct dirent *d; +char namebuf[PATH_MAX ]; + + dir =opendir (path); +if (dir ==NULL ) +return 0 ; + +while ((d =readdir (dir)) !=NULL && !ret) { +struct stat statbuf; + +if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." )) +continue ; + +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , + path, d->d_name); + + ret =stat (namebuf, &statbuf); +if (ret <0 ) { +pr_debug ("stat failed: %s\n" , namebuf); +break ; + } + +if (S_ISREG (statbuf.st_mode)) + ret =unlink (namebuf); +else if (S_ISDIR (statbuf.st_mode)) + ret =rm_rf (namebuf); +else { +pr_debug ("unknown file: %s\n" , namebuf); + ret =-1 ; + } + } +closedir (dir); + +if (ret <0 ) +return ret; + +return rmdir (path); +} +static int slow_copyfile (const char *from,const char *to,mode_t mode) {int err =-1 ;
tools/perf/util/util.h +1 -0
@@ -249,6 +249,7 @@ static inline int sane_case(int x, int high) }int mkdir_p (char *path,mode_t mode); +int rm_rf (char *path);int copyfile (const char *from,const char *to);int copyfile_mode (const char *from,const char *to,mode_t mode);
Namhyung Kim
maintainerPATCH 9/42 perf tools: Introduce copyfile_offset() function
In reply to Namhyung KimThe copyfile_offset() function is to copy source data from given offset to a destination file with an offset. It'll be used to build an indexed data file.
--- tools/perf/util/util.c | 38 +++++++++++++++++++++++++++++--------- tools/perf/util/util.h | 1 + 2 files changed, 30 insertions(+), 9 deletions(-)
tools/perf/util/util.c +29 -9
@@ -145,11 +145,38 @@ static int slow_copyfile(const char *from, const char *to, mode_t mode)return err; } +int copyfile_offset (int ifd,loff_t off_in,int ofd,loff_t off_out,u64 size) +{ +void *ptr; +loff_t pgoff; + + pgoff = off_in & ~(page_size -1 ); + off_in -= pgoff; + + ptr =mmap (NULL , off_in + size,PROT_READ ,MAP_PRIVATE , ifd, pgoff); +if (ptr ==MAP_FAILED ) +return -1 ; + +while (size) { +ssize_t ret =pwrite (ofd, ptr + off_in, size, off_out); +if (ret <0 && errno ==EINTR ) +continue ; +if (ret <=0 ) +break ; + + size -= ret; + off_in += ret; + off_out -= ret; + } +munmap (ptr, off_in + size); + +return size ?-1 :0 ; +} +int copyfile_mode (const char *from,const char *to,mode_t mode) {int fromfd, tofd;struct stat st; -void *addr;int err =-1 ;if (stat (from, &st))
@@ -166,15 +193,8 @@ int copyfile_mode(const char *from, const char *to, mode_t mode)if (tofd <0 ) goto out_close_from; - addr =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fromfd,0 ); -if (addr ==MAP_FAILED ) - goto out_close_to; - -if (write (tofd, addr, st.st_size) == st.st_size) - err =0 ; + err =copyfile_offset (fromfd,0 , tofd,0 , st.st_size); -munmap (addr, st.st_size); -out_close_to:close (tofd);if (err)unlink (to);
tools/perf/util/util.h +1 -0
@@ -251,6 +251,7 @@ int mkdir_p(char *path, mode_t mode);int rm_rf (char *path);int copyfile (const char *from,const char *to);int copyfile_mode (const char *from,const char *to,mode_t mode); +int copyfile_offset (int fromfd,loff_t from_ofs,int tofd,loff_t to_ofs,u64 size);s64 perf_atoll (const char *str);char **argv_split (const char *str,int *argcp);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 10/42 perf tools: Create separate mmap for dummy tracking event
In reply to Namhyung KimWhen indexed data file support is enabled, a dummy tracking event will be used to track metadata (like task, comm and mmap events) for a session and actual samples will be recorded in separate (intermediate) files and then merged (with index table).
Provide separate mmap to the dummy tracking event. The size is fixed to 128KiB (+ 1 page) as the event rate will be lower than samples. I originally wanted to use a single mmap for this but cross-cpu sharing is prohibited so it's per-cpu (or per-task) like normal mmaps.
--- tools/perf/builtin-record.c | 9 +++- tools/perf/util/evlist.c | 122 +++++++++++++++++++++++++++++++++++--------- tools/perf/util/evlist.h | 11 +++- 3 files changed, 117 insertions(+), 25 deletions(-)
tools/perf/builtin-record.c +8 -1
@@ -69,7 +69,7 @@ static int process_synthesized_event(struct perf_tool *tool,static int record__mmap_read (struct record *rec,int idx) { -struct perf_mmap *md = &rec->evlist->mmap[idx]; +struct perf_mmap *md =perf_evlist__mmap_desc (rec->evlist, idx);unsigned int head =perf_mmap__read_head (md);unsigned int old = md->prev;unsigned char *data = md->base + page_size;
@@ -105,6 +105,7 @@ static int record__mmap_read(struct record *rec, int idx) } md->prev = old; +perf_evlist__mmap_consume (rec->evlist, idx); out:return rc;
@@ -275,6 +276,12 @@ static int record__mmap_read_all(struct record *rec) goto out; } } +if (rec->evlist->track_mmap) { +if (record__mmap_read (rec,track_mmap_idx (i)) !=0 ) { + rc =-1 ; + goto out; + } + } }/*
tools/perf/util/evlist.c +99 -23
226 lines not on this page · raw message
tools/perf/util/evlist.h +10 -1
@@ -48,11 +48,14 @@ struct perf_evlist {bool overwrite;struct fdarray pollfd;struct perf_mmap *mmap; +struct perf_mmap *track_mmap;struct thread_map *threads;struct cpu_map *cpus;struct perf_evsel *selected; }; +#define TRACK_MMAP_SIZE (((128 * 1024 / page_size) + 1) * page_size) +struct perf_evsel_str_handler {const char *name;void *handler;
@@ -100,8 +103,8 @@ struct perf_evsel *perf_evlist__id2evsel(struct perf_evlist *evlist, u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);union perf_event *perf_evlist__mmap_read (struct perf_evlist *evlist,int idx); -void perf_evlist__mmap_consume (struct perf_evlist *evlist,int idx); +struct perf_mmap *perf_evlist__mmap_desc (struct perf_evlist *evlist,int idx);int perf_evlist__open (struct perf_evlist *evlist);void perf_evlist__close (struct perf_evlist *evlist);
@@ -211,6 +214,12 @@ bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str);void perf_evlist__to_front (struct perf_evlist *evlist,struct perf_evsel *move_evsel); +/* convert from/to negative idx for track mmaps */ +static inline int track_mmap_idx (int idx) +{ +return -idx -1 ; +} +/** * __evlist__for_each - iterate thru all the evsels * @list: list_head instance to iterate
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 11/42 perf tools: Introduce perf_evlist__mmap_track()
In reply to Namhyung KimThe perf_evlist__mmap_track function creates data mmaps and optionally tracking mmaps for events. It'll be used for perf record to save events in a separate files and build an index table. Checking dummy tracking event in perf_evlist__mmap() alone is not enough as users can specify a dummy event (like in keep tracking testcase) without the index option.
--- tools/perf/builtin-record.c | 3 ++- tools/perf/util/evlist.c | 15 +++++++++------ tools/perf/util/evlist.h | 10 ++++++++-- 3 files changed, 19 insertions(+), 9 deletions(-)
tools/perf/builtin-record.c +2 -1
@@ -169,7 +169,8 @@ static int record__open(struct record *rec) goto out; } -if (perf_evlist__mmap (evlist, opts->mmap_pages, false) <0 ) { +if (perf_evlist__mmap_track (evlist, opts->mmap_pages, false, + false) <0 ) {if (errno ==EPERM ) {pr_err ("Permission error mapping pages.\n" "Consider increasing "
tools/perf/util/evlist.c +9 -6
@@ -835,6 +835,7 @@ static int perf_evlist__alloc_mmap(struct perf_evlist *evlist, bool track_mmap)struct mmap_params {int prot; +bool track;size_t len; };
@@ -884,7 +885,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, fd =FD (evsel, cpu, thread); -if (perf_evsel__is_dummy_tracking (evsel)) { +if (mp->track &&perf_evsel__is_dummy_tracking (evsel)) {struct mmap_params track_mp = { .prot = mp->prot, .len =TRACK_MMAP_SIZE ,
@@ -937,7 +938,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, thread); } -if (perf_evsel__is_dummy_tracking (evsel)) { +if (mp->track &&perf_evsel__is_dummy_tracking (evsel)) {/* restore idx as normal idx (positive) */ idx =track_mmap_idx (idx); }
@@ -1088,10 +1089,11 @@ int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str, }/** -* perf_evlist__mmap - Create mmaps to receive events. +* perf_evlist__mmap_track - Create mmaps to receive events. * @evlist: list of events * @pages: map length in pages * @overwrite: overwrite older events? +* @use_track_mmap: use another mmaps to track meta events * * If @overwrite is %false the user needs to signal event consumption using * perf_mmap__write_tail(). Using perf_evlist__mmap_read() does this
@@ -1099,17 +1101,18 @@ int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str,* * Return: %0 on success, negative error code otherwise. */ -int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages, -bool overwrite) +int perf_evlist__mmap_track (struct perf_evlist *evlist,unsigned int pages, +bool overwrite,bool use_track_mmap) {struct perf_evsel *evsel;const struct cpu_map *cpus = evlist->cpus;const struct thread_map *threads = evlist->threads;struct mmap_params mp = { .prot =PROT_READ | (overwrite ?0 :PROT_WRITE ), + .track = use_track_mmap, }; -if (evlist->mmap ==NULL &&perf_evlist__alloc_mmap (evlist, true) <0 ) +if (evlist->mmap ==NULL &&perf_evlist__alloc_mmap (evlist, mp.track) <0 )return -ENOMEM ;if (evlist->pollfd.entries ==NULL &&perf_evlist__alloc_pollfd (evlist) <0 )
tools/perf/util/evlist.h +8 -2
@@ -127,10 +127,16 @@ int perf_evlist__parse_mmap_pages(const struct option *opt,const char *str, int unset); -int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages, -bool overwrite); +int perf_evlist__mmap_track (struct perf_evlist *evlist,unsigned int pages, +bool overwrite,bool use_track_mmap);void perf_evlist__munmap (struct perf_evlist *evlist); +static inline int perf_evlist__mmap (struct perf_evlist *evlist, +unsigned int pages,bool overwrite) +{ +return perf_evlist__mmap_track (evlist, pages, overwrite, false); +} +void perf_evlist__disable (struct perf_evlist *evlist);void perf_evlist__enable (struct perf_evlist *evlist);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 12/42 perf tools: Add HEADER_DATA_INDEX feature
In reply to Namhyung KimThe HEADER_DATA_INDEX feature is to record index table for sample data so that they can be processed by multiple thread concurrently. Each item is a struct perf_file_section which consists of an offset and size.
--- tools/perf/builtin-data.c | 0 tools/perf/builtin-record.c | 2 ++ tools/perf/util/header.c | 61 +++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/header.h | 3 +++ 4 files changed, 66 insertions(+) create mode 100644 tools/perf/builtin-data.c
tools/perf/builtin-data.c new file +0 -0
tools/perf/builtin-record.c +2 -0
@@ -312,6 +312,8 @@ static void record__init_features(struct record *rec)if (!rec->opts.branch_stack)perf_header__clear_feat (&session->header,HEADER_BRANCH_STACK ); + +perf_header__clear_feat (&session->header,HEADER_DATA_INDEX ); }static volatile int workload_exec_errno;
tools/perf/util/header.c +61 -0
85 lines not on this page · raw message
tools/perf/util/header.h +3 -0
@@ -30,6 +30,7 @@ enum {HEADER_BRANCH_STACK ,HEADER_PMU_MAPPINGS ,HEADER_GROUP_DESC , +HEADER_DATA_INDEX ,HEADER_LAST_FEATURE , HEADER_FEAT_BITS = 256, };
@@ -94,6 +95,8 @@ struct perf_header {bool needs_swap;u64 data_offset;u64 data_size; +struct perf_file_section *index; +u64 nr_index;u64 feat_offset;DECLARE_BITMAP (adds_features,HEADER_FEAT_BITS );struct perf_session_env env;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 13/42 perf tools: Handle indexed data file properly
In reply to Namhyung KimWhen perf detects data file has index table, process header file first and then rest data files in a row. Note that the indexed data is recorded for each cpu/thread separately, it's already ordered with respect to themselves so no need to use the ordered event queue interface.
--- tools/perf/util/session.c | 47 ++++++++++++++++++++++++++++++++++++----------- tools/perf/util/session.h | 5 +++++ 2 files changed, 41 insertions(+), 11 deletions(-)
tools/perf/util/session.c +36 -11
@@ -1251,11 +1251,10 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif -static int __perf_session__process_events (struct perf_session *session, +static int __perf_session__process_events (struct perf_session *session,int fd,u64 data_offset,u64 data_size,u64 file_size,struct perf_tool *tool) { -int fd =perf_data_file__fd (session->file);u64 head, page_offset, file_offset, file_pos, size;int err, mmap_prot, mmap_flags, map_idx =0 ;size_t mmap_size;
@@ -1278,7 +1277,9 @@ static int __perf_session__process_events(struct perf_session *session, mmap_size =MMAP_SIZE ;if (mmap_size > file_size) { mmap_size = file_size; - session->one_mmap = true; + +if (!perf_session__has_index (session)) + session->one_mmap = true; }memset (mmaps,0 ,sizeof (mmaps));
@@ -1360,19 +1361,43 @@ static int __perf_session__process_events(struct perf_session *session,int perf_session__process_events (struct perf_session *session,struct perf_tool *tool) { -u64 size =perf_data_file__size (session->file); -int err; +struct perf_data_file *file = session->file; +u64 size =perf_data_file__size (file); +int err, i;if (perf_session__register_idle_thread (session) ==NULL )return -ENOMEM ; -if (!perf_data_file__is_pipe (session->file)) +if (perf_data_file__is_pipe (file)) +return __perf_session__process_pipe_events (session, tool); + + err =__perf_session__process_events (session, +perf_data_file__fd (file), + session->header.data_offset, + session->header.data_size, + size, tool); + +if (err <0 || !perf_session__has_index (session)) +return err; + +/* +* For indexed data file, events are processed for +* each cpu/thread so it's already ordered. +*/ + tool->ordered_events = false; + +for (i =0 ; i < (int )session->header.nr_index; i++) { +if (!session->header.index[i].size) +continue ; + err =__perf_session__process_events (session, - session->header.data_offset, - session->header.data_size, - size, tool); -else - err =__perf_session__process_pipe_events (session, tool); +perf_data_file__fd (file), + session->header.index[i].offset, + session->header.index[i].size, + size, tool); +if (err <0 ) +break ; + }return err; }
tools/perf/util/session.h +5 -0
@@ -138,4 +138,9 @@ int perf_event__synthesize_id_index(struct perf_tool *tool,struct perf_evlist *evlist, struct machine *machine); +static inline bool perf_session__has_index (struct perf_session *session) +{ +return perf_header__has_feat (&session->header,HEADER_DATA_INDEX ); +} +#endif /* __PERF_SESSION_H */
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 14/42 perf record: Add --index option for building index table
In reply to Namhyung KimThe new --index option will create indexed data file which can be processed by multiple threads parallelly. It saves meta event and sample data in separate files and merges them with an index table.
To build an index table, it needs to know exact offsets and sizes for each sample data. However the offset only can be calculated after the feature data is fixed, and to save feature data it needs to access to the sample data because it needs to mark used DSOs for build-id table.
So I ended up with reserving 1MB hole for the feature data area and then put sample data and calculated offsets. Now an indexed perf data file will look like below:
+---------------------+
| file header |
|---------------------|
| |
| meta events |
| |
|---------------------|
| feature data |
| (contains index) -+--+
|---------------------| |
| ~1MB hole | |
|---------------------| |
| | |
| sample data[1] <-+--+
| | |
|---------------------| |
| | |
| sample data[2] <-|--+
| | |
|---------------------| |
| ... | ...
+---------------------+--- tools/perf/Documentation/perf-record.txt | 4 + tools/perf/builtin-data.c | 0 tools/perf/builtin-record.c | 165 +++++++++++++++++++++++++++++-- tools/perf/perf.h | 1 + tools/perf/util/session.c | 1 + 5 files changed, 161 insertions(+), 10 deletions(-) delete mode 100644 tools/perf/builtin-data.c
tools/perf/Documentation/perf-record.txt +4 -0
@@ -235,6 +235,10 @@ Capture machine state (registers) at interrupt, i.e., on counter overflows for each sample. List of captured registers depends on the architecture. This option is off by default. +--index:: +Build an index table for sample data. This will speed up perf report by +parallel processing. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-data.c deleted +0 -0
tools/perf/builtin-record.c +155 -10
260 lines not on this page · raw message
tools/perf/perf.h +1 -0
@@ -53,6 +53,7 @@ struct record_opts {bool sample_time;bool period;bool sample_intr_regs; +bool index;unsigned int freq;unsigned int mmap_pages;unsigned int user_freq;
tools/perf/util/session.c +1 -0
@@ -173,6 +173,7 @@ void perf_session__delete(struct perf_session *session)machines__exit (&session->machines);if (session->file)perf_data_file__close (session->file); +free (session->header.index);free (session); }
Signature · 1 line
2.2.2
Re: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 27 lines · Namhyung Kim
The new --index option will create indexed data file which can be processed by multiple threads parallelly. It saves meta event and sample data in separate files and merges them with an index table.
To build an index table, it needs to know exact offsets and sizes for each sample data. However the offset only can be calculated after the feature data is fixed, and to save feature data it needs to access to the sample data because it needs to mark used DSOs for build-id table.
So I ended up with reserving 1MB hole for the feature data area and then put sample data and calculated offsets. Now an indexed perf data file will look like below:
+---------------------+ | file header | |---------------------| | | | meta events | | | |---------------------| | feature data | | (contains index) -+--+ |---------------------| | | ~1MB hole | | |---------------------| | | | | | sample data[1] <-+--+
Quoted 8 lines
| | |
|---------------------| |
| | |
| sample data[2] <-|--+
| | |
|---------------------| |
| ... | ...
+---------------------+I also dont see how to store it in a nice way under current header layout, but how about bump up the header version for this feature? ;-)
currently it's:
struct perf_file_header {
u64 magic;
u64 size;
u64 attr_size;
struct perf_file_section attrs;
struct perf_file_section data;
/* event_types is ignored */
struct perf_file_section event_types;
DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
};- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx) this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features position wouldnt depend on the 'data' end as of now and we could easily store after all data is merged:
struct perf_file_header {
u64 magic;
u64 size;
u64 attr_size;
struct perf_file_section features;
DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
};thoughts?
jirka
Adrian Hunter
reviewerRe: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 70 lines · Jiri Olsa
Quoted 27 lines · Namhyung Kim
The new --index option will create indexed data file which can be processed by multiple threads parallelly. It saves meta event and sample data in separate files and merges them with an index table.
To build an index table, it needs to know exact offsets and sizes for each sample data. However the offset only can be calculated after the feature data is fixed, and to save feature data it needs to access to the sample data because it needs to mark used DSOs for build-id table.
So I ended up with reserving 1MB hole for the feature data area and then put sample data and calculated offsets. Now an indexed perf data file will look like below:
+---------------------+ | file header | |---------------------| | | | meta events | | | |---------------------| | feature data | | (contains index) -+--+ |---------------------| | | ~1MB hole | | |---------------------| | | | | | sample data[1] <-+--+Quoted 8 lines
| | | |---------------------| | | | | | sample data[2] <-|--+ | | | |---------------------| | | ... | ... +---------------------+I also dont see how to store it in a nice way under current header layout, but how about bump up the header version for this feature? ;-)
currently it's:
struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section attrs; struct perf_file_section data; /* event_types is ignored */ struct perf_file_section event_types; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx) this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features position wouldnt depend on the 'data' end as of now and we could easily store after all data is merged:struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section features; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };
Quoted 1 line
thoughts?
How come the features are being written before the sample data anyway? I would have expected:
- write the data (update the index in memory)
- write the features (including index)
Re: [PATCH 14/42] perf record: Add --index option for building index table
On Mon, Feb 02, 2015 at 10:34:50AM +0200, Adrian Hunter wrote:
SNIP
Quoted 32 lines
Quoted 32 lines
but how about bump up the header version for this feature? ;-)
currently it's:
struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section attrs; struct perf_file_section data; /* event_types is ignored */ struct perf_file_section event_types; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx) this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features position wouldnt depend on the 'data' end as of now and we could easily store after all data is merged:struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section features; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };
Quoted 7 lines
Quoted 1 line
thoughts?
How come the features are being written before the sample data anyway? I would have expected:
- write the data (update the index in memory)
- write the features (including index)
I think the problem is that the only way how to get features offset right now is via perf_file_header::data.offset + perf_file_headerdata.size, and we still use this section to carry 'sythesized' data, so it needs to have correct size.
I guess we could workaround that by storing the 'perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.
jirka
Adrian Hunter
reviewerRe: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 44 lines · Jiri Olsa
On Mon, Feb 02, 2015 at 10:34:50AM +0200, Adrian Hunter wrote:
SNIP
Quoted 32 lines
Quoted 32 lines
but how about bump up the header version for this feature? ;-)
currently it's:
struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section attrs; struct perf_file_section data; /* event_types is ignored */ struct perf_file_section event_types; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx) this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features position wouldnt depend on the 'data' end as of now and we could easily store after all data is merged:struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section features; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };Quoted 7 lines
Quoted 1 line
thoughts?
How come the features are being written before the sample data anyway? I would have expected:
- write the data (update the index in memory)
- write the features (including index)
Quoted 4 lines
I think the problem is that the only way how to get features offset right now is via
perf_file_header::data.offset+ perf_file_headerdata.size, and we still use this section to carry 'sythesized' data, so it needs to have correct size.
Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
Quoted 3 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.
Would it need to be last? Logically it should precede the data that depends on it.
Re: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 45 lines · Adrian Hunter
Quoted 44 lines · Jiri Olsa
On Mon, Feb 02, 2015 at 10:34:50AM +0200, Adrian Hunter wrote:
SNIP
Quoted 39 lines
Quoted 32 lines
but how about bump up the header version for this feature? ;-)
currently it's:
struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section attrs; struct perf_file_section data; /* event_types is ignored */ struct perf_file_section event_types; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx) this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features position wouldnt depend on the 'data' end as of now and we could easily store after all data is merged:struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section features; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };Quoted 1 line
thoughts?
How come the features are being written before the sample data anyway? I would have expected:
- write the data (update the index in memory)
- write the features (including index)
Quoted 7 lines
Quoted 4 lines
I think the problem is that the only way how to get features offset right now is via
perf_file_header::data.offset+ perf_file_headerdata.size, and we still use this section to carry 'sythesized' data, so it needs to have correct size.Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
thats what I suggested
Quoted 7 lines
Quoted 4 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.Would it need to be last? Logically it should precede the data that depends on it.
i suggested this as a workaround for having features at the end of the file while keeping the current perf data header
jirka
Adrian Hunter
reviewerRe: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 51 lines · Jiri Olsa
Quoted 50 lines · Adrian Hunter
Quoted 48 lines · Jiri Olsa
On Mon, Feb 02, 2015 at 10:34:50AM +0200, Adrian Hunter wrote:
SNIP
Quoted 32 lines
Quoted 32 lines
but how about bump up the header version for this feature? ;-)
currently it's:
struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section attrs; struct perf_file_section data; /* event_types is ignored */ struct perf_file_section event_types; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx) this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features position wouldnt depend on the 'data' end as of now and we could easily store after all data is merged:struct perf_file_header { u64 magic; u64 size; u64 attr_size; struct perf_file_section features; DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); };Quoted 7 lines
Quoted 1 line
thoughts?
How come the features are being written before the sample data anyway? I would have expected:
- write the data (update the index in memory)
- write the features (including index)I think the problem is that the only way how to get features offset right now is via
perf_file_header::data.offset+ perf_file_headerdata.size, and we still use this section to carry 'sythesized' data, so it needs to have correct size.
Quoted 4 lines
Quoted 2 lines
Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
thats what I suggested
No, I meant really lump it all together. i.e. perf_file_header.data.size = total data size
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.Would it need to be last? Logically it should precede the data that depends on it.
i suggested this as a workaround for having features at the end of the file while keeping the current perf data header
Which wouldn't be necessary if you lump it all together?
Re: [PATCH 14/42] perf record: Add --index option for building index table
On Mon, Feb 02, 2015 at 02:07:27PM +0200, Adrian Hunter wrote:
SNIP
Quoted 15 lines
Quoted 5 lines
Quoted 3 lines
Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
thats what I suggested
No, I meant really lump it all together. i.e. perf_file_header.data.size = total data size
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Would it need to be last? Logically it should precede the data that depends on it.
i suggested this as a workaround for having features at the end of the file while keeping the current perf data header
Which wouldn't be necessary if you lump it all together?
yep, that's also an option
jirka
Namhyung Kim
maintainerRe: [PATCH 14/42] perf record: Add --index option for building index table
Hi Jiri and Adrian,
Quoted 22 lines · Jiri Olsa
On Mon, Feb 02, 2015 at 02:07:27PM +0200, Adrian Hunter wrote:
SNIP
Quoted 18 lines
Quoted 5 lines
Quoted 3 lines
Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
thats what I suggested
No, I meant really lump it all together. i.e. perf_file_header.data.size = total data size
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.Would it need to be last? Logically it should precede the data that depends on it.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
i suggested this as a workaround for having features at the end of the file while keeping the current perf data header
Which wouldn't be necessary if you lump it all together?
yep, that's also an option
So we want a single section for the entire data area, right?
I also thought about it. My concern was the holes between each data due to page alignment. If an old tool which doesn't know about the index accesses to the data file, it'd just see a event type of 0 and stop processing.
Maybe the page alignment is not necessary?
Thanks,
Namhyung
Re: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 32 lines · Namhyung Kim
Hi Jiri and Adrian,
Quoted 22 lines · Jiri Olsa
On Mon, Feb 02, 2015 at 02:07:27PM +0200, Adrian Hunter wrote:
SNIP
Quoted 18 lines
Quoted 5 lines
Quoted 3 lines
Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
thats what I suggested
No, I meant really lump it all together. i.e. perf_file_header.data.size = total data size
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.Would it need to be last? Logically it should precede the data that depends on it.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
i suggested this as a workaround for having features at the end of the file while keeping the current perf data header
Which wouldn't be necessary if you lump it all together?
yep, that's also an option
Quoted 8 lines
So we want a single section for the entire data area, right?
I also thought about it. My concern was the holes between each data due to page alignment. If an old tool which doesn't know about the index accesses to the data file, it'd just see a event type of 0 and stop processing.
Maybe the page alignment is not necessary?
seems ok, but how about time ordering.. every time you reach new file data you'll hit 'out of order event' right?
hum, maybe it's not a big deal now when it's just incrementing counter ;-)
jirka
Adrian Hunter
reviewerRe: [PATCH 14/42] perf record: Add --index option for building index table
Quoted 31 lines · Jiri Olsa
Quoted 30 lines · Namhyung Kim
Hi Jiri and Adrian,
Quoted 27 lines · Jiri Olsa
On Mon, Feb 02, 2015 at 02:07:27PM +0200, Adrian Hunter wrote:
SNIP
Quoted 22 lines
Quoted 5 lines
Quoted 3 lines
Why not make it the same as all the other data. i.e. find the start and size via the index? And then just lump all the data together?
thats what I suggested
No, I meant really lump it all together. i.e. perf_file_header.data.size = total data size
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
I guess we could workaround that by storing the '
perf_file_header::data' as the last data section. That would require to treat it the same way as all other data sections, but we could keep current header layout.Would it need to be last? Logically it should precede the data that depends on it.
i suggested this as a workaround for having features at the end of the file while keeping the current perf data header
Which wouldn't be necessary if you lump it all together?
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
yep, that's also an option
So we want a single section for the entire data area, right?
I also thought about it. My concern was the holes between each data due to page alignment. If an old tool which doesn't know about the index accesses to the data file, it'd just see a event type of 0 and stop processing.
Please don't leave holes. Either fill them with a padding event or put the data end-to-end.
Quoted 11 lines
Quoted 2 lines
Maybe the page alignment is not necessary?
seems ok, but how about time ordering.. every time you reach new file data you'll hit 'out of order event' right?
hum, maybe it's not a big deal now when it's just incrementing counter ;-)
jirka
Namhyung Kim
maintainerPATCH 15/42 perf report: Skip dummy tracking event
In reply to Namhyung KimThe dummy tracking event is only for tracking task/comom/mmap events and has no sample data for itself. So no need to report, just skip it.
--- tools/perf/builtin-report.c | 3 +++ tools/perf/ui/browsers/hists.c | 30 ++++++++++++++++++++++++------ tools/perf/ui/gtk/hists.c | 3 +++ 3 files changed, 30 insertions(+), 6 deletions(-)
tools/perf/builtin-report.c +3 -0
@@ -318,6 +318,9 @@ static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,struct hists *hists =evsel__hists (pos);const char *evname =perf_evsel__name (pos); +if (perf_evsel__is_dummy_tracking (pos)) +continue ; +if (symbol_conf.event_group && !perf_evsel__is_group_leader (pos))continue ;
tools/perf/ui/browsers/hists.c +24 -6
@@ -1947,14 +1947,17 @@ static int perf_evsel_menu__run(struct perf_evsel_menu *menu,return key; } -static bool filter_group_entries (struct ui_browser *browser__maybe_unused , -void *entry) +static bool filter_entries (struct ui_browser *browser__maybe_unused , +void *entry) { struct perf_evsel *evsel = list_entry(entry, struct perf_evsel, node);if (symbol_conf.event_group && !perf_evsel__is_group_leader (evsel))return true; +if (perf_evsel__is_dummy_tracking (evsel)) +return true; +return false; }
@@ -1971,7 +1974,7 @@ static int __perf_evlist__tui_browse_hists(struct perf_evlist *evlist, .refresh = ui_browser__list_head_refresh, .seek = ui_browser__list_head_seek, .write = perf_evsel_menu__write, - .filter = filter_group_entries, + .filter = filter_entries, .nr_entries = nr_entries, .priv = evlist, },
@@ -1998,21 +2001,22 @@ int perf_evlist__tui_browse_hists(struct perf_evlist *evlist, const char *help, struct perf_session_env *env) {int nr_entries = evlist->nr_entries; +struct perf_evsel *first =perf_evlist__first (evlist); +struct perf_evsel *pos; single_entry:if (nr_entries ==1 ) { -struct perf_evsel *first =perf_evlist__first (evlist); -return perf_evsel__hists_browse (first, nr_entries, help, false, hbt, min_pcnt, env); }if (symbol_conf.event_group) { -struct perf_evsel *pos; nr_entries =0 ;evlist__for_each (evlist, pos) { +if (perf_evsel__is_dummy_tracking (pos)) +continue ;if (perf_evsel__is_group_leader (pos)) nr_entries++; }
@@ -2021,6 +2025,20 @@ int perf_evlist__tui_browse_hists(struct perf_evlist *evlist, const char *help, goto single_entry; } +evlist__for_each (evlist, pos) { +if (perf_evsel__is_dummy_tracking (pos)) + nr_entries--; + } + +if (nr_entries ==1 ) { +evlist__for_each (evlist, pos) { +if (!perf_evsel__is_dummy_tracking (pos)) { + first = pos; + goto single_entry; + } + } + } +return __perf_evlist__tui_browse_hists (evlist, nr_entries, help, hbt, min_pcnt, env); }
tools/perf/ui/gtk/hists.c +3 -0
@@ -317,6 +317,9 @@ int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist,char buf[512 ];size_t size =sizeof (buf); +if (perf_evsel__is_dummy_tracking (pos)) +continue ; +if (symbol_conf.event_group) {if (!perf_evsel__is_group_leader (pos))continue ;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 16/42 perf tools: Pass session arg to perf_event__preprocess_sample()
In reply to Namhyung KimThe perf_event__preprocess_sample() translates a given ip into a matching symbol. To do that, it first finds a corresponding thread and map in the current thread tree. But for indexed data files, it needs to find a thread (and map) with slightly different APIs using timestamp. So it needs a way to know whether this session deals with an indexed data file or not.
--- tools/perf/builtin-annotate.c | 3 ++- tools/perf/builtin-diff.c | 13 +++++++++---- tools/perf/builtin-mem.c | 6 +++++- tools/perf/builtin-report.c | 3 ++- tools/perf/builtin-script.c | 20 +++++++++++--------- tools/perf/builtin-timechart.c | 10 +++++++--- tools/perf/builtin-top.c | 3 ++- tools/perf/tests/hists_cumulate.c | 2 +- tools/perf/tests/hists_filter.c | 2 +- tools/perf/tests/hists_link.c | 4 ++-- tools/perf/tests/hists_output.c | 2 +- tools/perf/util/event.c | 3 ++- tools/perf/util/event.h | 4 +++- 13 files changed, 48 insertions(+), 27 deletions(-)
tools/perf/builtin-annotate.c +2 -1
@@ -85,7 +85,8 @@ static int process_sample_event(struct perf_tool *tool, struct perf_annotate *ann = container_of(tool, struct perf_annotate, tool);struct addr_location al; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + ann->session) <0 ) {pr_warning ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
tools/perf/builtin-diff.c +9 -4
@@ -42,6 +42,7 @@ struct diff_hpp_fmt { };struct data__file { +struct perf_tool tool;struct perf_session *session;struct perf_data_file file;int idx;
@@ -320,16 +321,18 @@ static int hists__add_entry(struct hists *hists,return -ENOMEM ; } -static int diff__process_sample_event (struct perf_tool *tool__maybe_unused , +static int diff__process_sample_event (struct perf_tool *tool,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel,struct machine *machine) {struct addr_location al; + struct data__file *d = container_of(tool, struct data__file, tool);struct hists *hists =evsel__hists (evsel); -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + d->session) <0 ) {pr_warning ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
@@ -740,14 +743,16 @@ static int __cmd_diff(void)int ret = -EINVAL , i;data__for_each_file (i, d) { - d->session =perf_session__new (&d->file, false, &tool); +memcpy (&d->tool, &tool,sizeof (tool)); + + d->session =perf_session__new (&d->file, false, &d->tool);if (!d->session) {pr_err ("Failed to open %s\n" , d->file.path); ret =-1 ; goto out_delete; } - ret =perf_session__process_events (d->session, &tool); + ret =perf_session__process_events (d->session, &d->tool);if (ret) {pr_err ("Failed to process %s\n" , d->file.path); goto out_delete;
tools/perf/builtin-mem.c +5 -1
@@ -12,6 +12,7 @@struct perf_mem {struct perf_tool tool; +struct perf_session *session;char const *input_name;bool hide_unresolved;bool dump_raw;
@@ -66,7 +67,8 @@ dump_raw_samples(struct perf_tool *tool,struct addr_location al;const char *fmt; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + mem->session) <0 ) {fprintf (stderr,"problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
@@ -129,6 +131,8 @@ static int report_raw_events(struct perf_mem *mem)if (session ==NULL )return -1 ; + mem->session = session; +if (mem->cpu_list) { ret =perf_session__cpu_bitmap (session, mem->cpu_list, mem->cpu_bitmap);
tools/perf/builtin-report.c +2 -1
@@ -142,7 +142,8 @@ static int process_sample_event(struct perf_tool *tool, };int ret; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + rep->session) <0 ) {pr_debug ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
tools/perf/builtin-script.c +11 -9
@@ -542,13 +542,21 @@ static int cleanup_scripting(void)return scripting_ops->stop_script (); } -static int process_sample_event (struct perf_tool *tool__maybe_unused , +struct perf_script { +struct perf_tool tool; +struct perf_session *session; +bool show_task_events; +bool show_mmap_events; +}; + +static int process_sample_event (struct perf_tool *tool,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel,struct machine *machine) {struct addr_location al; + struct perf_script *script = container_of(tool, struct perf_script, tool);struct thread *thread =machine__findnew_thread (machine, sample->pid, sample->tid);
@@ -569,7 +577,8 @@ static int process_sample_event(struct perf_tool *tool __maybe_unused,return 0 ; } -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + script->session) <0 ) {pr_err ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
@@ -586,13 +595,6 @@ static int process_sample_event(struct perf_tool *tool __maybe_unused,return 0 ; } -struct perf_script { -struct perf_tool tool; -struct perf_session *session; -bool show_task_events; -bool show_mmap_events; -}; -static int process_attr (struct perf_tool *tool,union perf_event *event,struct perf_evlist **pevlist) {
tools/perf/builtin-timechart.c +7 -3
@@ -48,6 +48,7 @@ struct wake_event;struct timechart {struct perf_tool tool; +struct perf_session *session;struct per_pid *all_data;struct power_event *power_events;struct wake_event *wake_events;
@@ -469,7 +470,8 @@ static void sched_switch(struct timechart *tchart, int cpu, u64 timestamp,static const char *cat_backtrace (union perf_event *event,struct perf_sample *sample, -struct machine *machine) +struct machine *machine, +struct perf_session *session) {struct addr_location al;unsigned int i;
@@ -488,7 +490,8 @@ static const char *cat_backtrace(union perf_event *event,if (!chain) goto exit; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + session) <0 ) {fprintf (stderr,"problem processing %d event, skipping it.\n" , event->header.type); goto exit;
@@ -567,7 +570,7 @@ static int process_sample_event(struct perf_tool *tool,if (evsel->handler !=NULL ) {tracepoint_handler f = evsel->handler;return f (tchart, evsel, sample, -cat_backtrace (event, sample, machine)); +cat_backtrace (event, sample, machine, tchart->session)); }return 0 ;
@@ -1623,6 +1626,7 @@ static int __cmd_timechart(struct timechart *tchart, const char *output_name) goto out_delete; } + tchart->session = session; ret =perf_session__process_events (session, &tchart->tool);if (ret) goto out_delete;
tools/perf/builtin-top.c +2 -1
@@ -723,7 +723,8 @@ static void perf_event__process_sample(struct perf_tool *tool,if (event->header.misc &PERF_RECORD_MISC_EXACT_IP ) top->exact_samples++; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) +if (perf_event__preprocess_sample (event, machine, &al, sample, + top->session) <0 )return ; if (!top->kptr_restrict_warned &&
tools/perf/tests/hists_cumulate.c +1 -1
@@ -101,7 +101,7 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) sample.callchain = (struct ip_callchain *)fake_callchains[i];if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; if (hist_entry_iter__add(&iter, &al, evsel, &sample,
tools/perf/tests/hists_filter.c +1 -1
@@ -78,7 +78,7 @@ static int add_hist_entries(struct perf_evlist *evlist, sample.ip = fake_samples[i].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; if (hist_entry_iter__add(&iter, &al, evsel, &sample,
tools/perf/tests/hists_link.c +2 -2
@@ -86,7 +86,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) sample.tid = fake_common_samples[k].pid; sample.ip = fake_common_samples[k].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; he = __hists__add_entry(hists, &al, NULL,
@@ -110,7 +110,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) sample.tid = fake_samples[i][k].pid; sample.ip = fake_samples[i][k].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; he = __hists__add_entry(hists, &al, NULL,
tools/perf/tests/hists_output.c +1 -1
@@ -67,7 +67,7 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) sample.ip = fake_samples[i].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; if (hist_entry_iter__add(&iter, &al, evsel, &sample,
tools/perf/util/event.c +2 -1
@@ -822,7 +822,8 @@ void thread__find_addr_location(struct thread *thread,int perf_event__preprocess_sample (const union perf_event *event,struct machine *machine,struct addr_location *al, -struct perf_sample *sample) +struct perf_sample *sample, +struct perf_session *session__maybe_unused ) {u8 cpumode = event->header.misc &PERF_RECORD_MISC_CPUMODE_MASK ; struct thread *thread = machine__findnew_thread(machine, sample->pid,
tools/perf/util/event.h +3 -1
@@ -353,11 +353,13 @@ int perf_event__process(struct perf_tool *tool, struct machine *machine);struct addr_location ; +struct perf_session ;int perf_event__preprocess_sample (const union perf_event *event,struct machine *machine,struct addr_location *al, -struct perf_sample *sample); +struct perf_sample *sample, +struct perf_session *session);struct thread ;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 17/42 perf script: Pass session arg to ->process_event callback
In reply to Namhyung KimSometimes it needs to retrieve symbol info inside a script engine so we need to pass the session pointer to find the symbol correctly as with previous patch.
--- tools/perf/builtin-script.c | 23 ++++++++++++---------- tools/perf/util/db-export.c | 6 ++++-- tools/perf/util/db-export.h | 4 +++- tools/perf/util/event.c | 3 ++- tools/perf/util/event.h | 3 ++- .../perf/util/scripting-engines/trace-event-perl.c | 3 ++- .../util/scripting-engines/trace-event-python.c | 5 +++-- tools/perf/util/trace-event-scripting.c | 3 ++- tools/perf/util/trace-event.h | 3 ++- 9 files changed, 33 insertions(+), 20 deletions(-)
tools/perf/builtin-script.c +13 -10
@@ -377,9 +377,10 @@ static void print_sample_start(struct perf_sample *sample, }static void print_sample_addr (union perf_event *event, -struct perf_sample *sample, -struct thread *thread, -struct perf_event_attr *attr) +struct perf_sample *sample, +struct thread *thread, +struct perf_event_attr *attr, +struct perf_session *session) {struct addr_location al;
@@ -388,7 +389,7 @@ static void print_sample_addr(union perf_event *event,if (!sample_addr_correlates_sym (attr))return ; -perf_event__preprocess_sample_addr (event, sample, thread, &al); +perf_event__preprocess_sample_addr (event, sample, thread, &al, session);if (PRINT_FIELD (SYM )) {printf (" " );
@@ -409,7 +410,8 @@ static void print_sample_bts(union perf_event *event,struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al) + struct addr_location *al, + struct perf_session *session) { struct perf_event_attr *attr = &evsel->attr;bool print_srcline_last = false;
@@ -436,7 +438,7 @@ static void print_sample_bts(union perf_event *event, ((evsel->attr.sample_type & PERF_SAMPLE_ADDR) && !output[attr->type].user_set)) {printf (" => " ); -print_sample_addr (event, sample, thread, attr); +print_sample_addr (event, sample, thread, attr, session); }if (print_srcline_last)
@@ -447,7 +449,7 @@ static void print_sample_bts(union perf_event *event,static void process_event (union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel,struct thread *thread, -struct addr_location *al) +struct addr_location *al,struct perf_session *session) {struct perf_event_attr *attr = &evsel->attr;
@@ -465,7 +467,7 @@ static void process_event(union perf_event *event, struct perf_sample *sample, }if (is_bts_event (attr)) { -print_sample_bts (event, sample, evsel, thread, al); +print_sample_bts (event, sample, evsel, thread, al, session);return ; }
@@ -473,7 +475,7 @@ static void process_event(union perf_event *event, struct perf_sample *sample,event_format__print (evsel->tp_format, sample->cpu, sample->raw_data, sample->raw_size);if (PRINT_FIELD (ADDR )) -print_sample_addr (event, sample, thread, attr); +print_sample_addr (event, sample, thread, attr, session);if (PRINT_FIELD (IP )) {if (!symbol_conf.use_callchain)
@@ -590,7 +592,8 @@ static int process_sample_event(struct perf_tool *tool,if (cpu_list && !test_bit (sample->cpu, cpu_bitmap))return 0 ; - scripting_ops->process_event (event, sample, evsel, thread, &al); + scripting_ops->process_event (event, sample, evsel, thread, &al, + script->session);return 0 ; }
tools/perf/util/db-export.c +4 -2
@@ -282,7 +282,8 @@ int db_export__branch_type(struct db_export *dbe, u32 branch_type,int db_export__sample (struct db_export *dbe,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel, -struct thread *thread,struct addr_location *al) +struct thread *thread,struct addr_location *al, +struct perf_session *session) {struct export_sample es = { .event = event,
@@ -328,7 +329,8 @@ int db_export__sample(struct db_export *dbe, union perf_event *event, sample_addr_correlates_sym(&evsel->attr)) {struct addr_location addr_al; -perf_event__preprocess_sample_addr (event, sample, thread, &addr_al); +perf_event__preprocess_sample_addr (event, sample, thread, + &addr_al, session); err =db_ids_from_al (dbe, &addr_al, &es.addr_dso_db_id, &es.addr_sym_db_id, &es.addr_offset);if (err)
tools/perf/util/db-export.h +3 -1
@@ -29,6 +29,7 @@ struct addr_location;struct call_return_processor ;struct call_path ;struct call_return ; +struct perf_session ;struct export_sample {union perf_event *event;
@@ -97,7 +98,8 @@ int db_export__branch_type(struct db_export *dbe, u32 branch_type, const char *name);int db_export__sample (struct db_export *dbe,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel, -struct thread *thread,struct addr_location *al); +struct thread *thread,struct addr_location *al, +struct perf_session *session);int db_export__branch_types (struct db_export *dbe);
tools/perf/util/event.c +2 -1
@@ -904,7 +904,8 @@ bool sample_addr_correlates_sym(struct perf_event_attr *attr)void perf_event__preprocess_sample_addr (union perf_event *event,struct perf_sample *sample,struct thread *thread, -struct addr_location *al) +struct addr_location *al, +struct perf_session *session__maybe_unused ) {u8 cpumode = event->header.misc &PERF_RECORD_MISC_CPUMODE_MASK ;
tools/perf/util/event.h +2 -1
@@ -368,7 +368,8 @@ bool sample_addr_correlates_sym(struct perf_event_attr *attr);void perf_event__preprocess_sample_addr (union perf_event *event,struct perf_sample *sample,struct thread *thread, -struct addr_location *al); +struct addr_location *al, +struct perf_session *session);const char *perf_event__name (unsigned int id);
tools/perf/util/scripting-engines/trace-event-perl.c +2 -1
@@ -356,7 +356,8 @@ static void perl_process_event(union perf_event *event,struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al __maybe_unused) + struct addr_location *al __maybe_unused, + struct perf_session *session __maybe_unused) {perl_process_tracepoint (sample ,evsel ,thread );perl_process_event_generic (event, sample, evsel);
tools/perf/util/scripting-engines/trace-event-python.c +3 -2
@@ -839,7 +839,8 @@ static void python_process_event(union perf_event *event,struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al) + struct addr_location *al, + struct perf_session *session) { struct tables *tables = &tables_global;
@@ -851,7 +852,7 @@ static void python_process_event(union perf_event *event,default :if (tables->db_export_mode)db_export__sample (&tables->dbe, event, sample, evsel, - thread, al); + thread, al, session);else python_process_general_event (sample, evsel, thread, al); }
tools/perf/util/trace-event-scripting.c +2 -1
@@ -44,7 +44,8 @@ static void process_event_unsupported(union perf_event *event __maybe_unused,struct perf_sample *sample__maybe_unused , struct perf_evsel *evsel __maybe_unused, struct thread *thread __maybe_unused, - struct addr_location *al __maybe_unused) + struct addr_location *al __maybe_unused, + struct perf_session *session __maybe_unused) { }
tools/perf/util/trace-event.h +2 -1
@@ -70,7 +70,8 @@ struct scripting_ops {struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al); + struct addr_location *al, + struct perf_session *session);int (*generate_script) (struct pevent *pevent,const char *outfile); };
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 18/42 perf tools: Introduce thread__comm_time() helpers
In reply to Namhyung KimWhen data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The thread__comm_time() will mostly work even if PERF_SAMPLE_TIME bit is off since in that case, sample->time will be -1 so it'll take the last comm anyway.
--- tools/perf/util/thread.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 2 ++ 2 files changed, 34 insertions(+), 1 deletion(-)
tools/perf/util/thread.c +32 -1
@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; } +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp) +{ +struct comm *comm; + +list_for_each_entry (comm, &thread->comm_list, list) { +if (timestamp >= comm->start) +return comm; + } + +if (list_empty (&thread->comm_list)) +return NULL ; + + return list_last_entry(&thread->comm_list, struct comm, list); +} +int __thread__set_comm (struct thread *thread,const char *str,u64 timestamp,bool exec) {
@@ -118,7 +133,13 @@ int __thread__set_comm(struct thread *thread, const char *str, u64 timestamp, new =comm__new (str, timestamp, exec);if (!new)return -ENOMEM ; -list_add (&new->list, &thread->comm_list); + +/* sort by time */ +list_for_each_entry (curr, &thread->comm_list, list) { +if (timestamp >= curr->start) +break ; + } +list_add_tail (&new->list, &curr->list);if (exec)unwind__flush_access (thread);
@@ -139,6 +160,16 @@ const char *thread__comm_str(const struct thread *thread)return comm__str (comm); } +const char *thread__comm_str_time (const struct thread *thread,u64 timestamp) +{ +const struct comm *comm =thread__comm_time (thread, timestamp); + +if (!comm) +return NULL ; + +return comm__str (comm); +} +/* CHECKME: it should probably better return the max comm len from its comm list */ int thread__comm_len (struct thread *thread) {
tools/perf/util/thread.h +2 -0
@@ -53,7 +53,9 @@ static inline int thread__set_comm(struct thread *thread, const char *comm,int thread__comm_len (struct thread *thread);struct comm *thread__comm (const struct thread *thread);struct comm *thread__exec_comm (const struct thread *thread); +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp);const char *thread__comm_str (const struct thread *thread); +const char *thread__comm_str_time (const struct thread *thread,u64 timestamp);void thread__insert_map (struct thread *thread,struct map *map);int thread__fork (struct thread *thread,struct thread *parent,u64 timestamp);size_t thread__fprintf (struct thread *thread,FILE *fp);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 19/42 perf tools: Add a test case for thread comm handling
In reply to Namhyung KimThe new test case checks various thread comm handling like overridding and time sorting.
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/builtin-test.c | 4 ++++ tools/perf/tests/tests.h | 1 + tools/perf/tests/thread-comm.c | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 53 insertions(+) create mode 100644 tools/perf/tests/thread-comm.c
tools/perf/Makefile.perf +1 -0
@@ -458,6 +458,7 @@ endif LIB_OBJS += $(OUTPUT)tests/mmap-thread-lookup.o LIB_OBJS += $(OUTPUT)tests/thread-mg-share.o LIB_OBJS += $(OUTPUT)tests/switch-tracking.o +LIB_OBJS += $(OUTPUT)tests/thread-comm.o BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
tools/perf/tests/builtin-test.c +4 -0
@@ -167,6 +167,10 @@ static struct test { .func = test__fdarray__add, }, { + .desc = "Test thread comm handling", + .func = test__thread_comm, + }, + { .func = NULL, }, };
tools/perf/tests/tests.h +1 -0
@@ -51,6 +51,7 @@ int test__hists_cumulate(void);int test__switch_tracking (void );int test__fdarray__filter (void );int test__fdarray__add (void ); +int test__thread_comm (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/tests/thread-comm.c new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 20/42 perf tools: Use thread__comm_time() when adding hist entries
In reply to Namhyung KimNow thread->comm can be handled with time properly, use it to find correct comm when adding hist entries.
--- tools/perf/builtin-annotate.c | 5 +++-- tools/perf/builtin-diff.c | 8 ++++---- tools/perf/tests/hists_link.c | 4 ++-- tools/perf/util/hist.c | 19 ++++++++++--------- tools/perf/util/hist.h | 2 +- 5 files changed, 20 insertions(+), 18 deletions(-)
tools/perf/builtin-annotate.c +3 -2
@@ -47,7 +47,7 @@ struct perf_annotate { };static int perf_evsel__add_sample (struct perf_evsel *evsel, -struct perf_sample *sample__maybe_unused , +struct perf_sample *sample,struct addr_location *al,struct perf_annotate *ann) {
@@ -67,7 +67,8 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,return 0 ; } - he =__hists__add_entry (hists, al,NULL ,NULL ,NULL ,1 ,1 ,0 , true); + he =__hists__add_entry (hists, al,NULL ,NULL ,NULL ,1 ,1 ,0 , + sample->time, true);if (he ==NULL )return -ENOMEM ;
tools/perf/builtin-diff.c +4 -4
@@ -313,10 +313,10 @@ static int formula_fprintf(struct hist_entry *he, struct hist_entry *pair,static int hists__add_entry (struct hists *hists,struct addr_location *al,u64 period, -u64 weight,u64 transaction) +u64 weight,u64 transaction,u64 timestamp) {if (__hists__add_entry (hists, al,NULL ,NULL ,NULL , period, weight, - transaction, true) !=NULL ) + transaction, timestamp, true) !=NULL )return 0 ;return -ENOMEM ; }
@@ -338,8 +338,8 @@ static int diff__process_sample_event(struct perf_tool *tool,return -1 ; } -if (hists__add_entry (hists, &al, sample->period, - sample->weight, sample->transaction)) { +if (hists__add_entry (hists, &al, sample->period, sample->weight, + sample->transaction, sample->time)) {pr_warning ("problem incrementing symbol period, skipping event\n" );return -1 ; }
tools/perf/tests/hists_link.c +2 -2
@@ -90,7 +90,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) goto out; he =__hists__add_entry (hists, &al,NULL , -NULL ,NULL ,1 ,1 ,0 , true); +NULL ,NULL ,1 ,1 ,0 ,-1 , true);if (he ==NULL ) goto out;
@@ -114,7 +114,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) goto out; he =__hists__add_entry (hists, &al,NULL , -NULL ,NULL ,1 ,1 ,0 , true); +NULL ,NULL ,1 ,1 ,0 ,-1 , true);if (he ==NULL ) goto out;
tools/perf/util/hist.c +10 -9
@@ -447,11 +447,11 @@ struct hist_entry *__hists__add_entry(struct hists *hists, struct branch_info *bi, struct mem_info *mi, u64 period, u64 weight, u64 transaction, - bool sample_self) + u64 timestamp, bool sample_self) { struct hist_entry entry = { .thread = al->thread, - .comm = thread__comm(al->thread), + .comm = thread__comm_time(al->thread, timestamp), .ms = { .map = al->map, .sym = al->sym,
@@ -509,13 +509,14 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *al {u64 cost;struct mem_info *mi = iter->priv; +struct perf_sample *sample = iter->sample;struct hists *hists =evsel__hists (iter->evsel);struct hist_entry *he;if (mi ==NULL )return -EINVAL ; - cost = iter->sample->weight; + cost = sample->weight;if (!cost) cost =1 ;
@@ -527,7 +528,7 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *al* and the he_stat__add_period() function. */ he =__hists__add_entry (hists, al, iter->parent,NULL , mi, - cost, cost,0 , true); + cost, cost,0 , sample->time, true);if (!he)return -ENOMEM ;
@@ -628,7 +629,7 @@ iter_add_next_branch_entry(struct hist_entry_iter *iter, struct addr_location *a* and not events sampled. Thus we use a pseudo period of 1. */ he =__hists__add_entry (hists, al, iter->parent, &bi[i],NULL , -1 ,1 ,0 , true); +1 ,1 ,0 , iter->sample->time, true);if (he ==NULL )return -ENOMEM ;
@@ -666,7 +667,7 @@ iter_add_single_normal_entry(struct hist_entry_iter *iter, struct addr_location he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , sample->period, sample->weight, - sample->transaction, true); + sample->transaction, sample->time, true);if (he ==NULL )return -ENOMEM ;
@@ -728,7 +729,7 @@ iter_add_single_cumulative_entry(struct hist_entry_iter *iter, he =__hists__add_entry (hists, al, iter->parent,NULL ,NULL , sample->period, sample->weight, - sample->transaction, true); + sample->transaction, sample->time, true);if (he ==NULL )return -ENOMEM ;
@@ -772,7 +773,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter,struct hist_entry he_tmp = { .cpu = al->cpu, .thread = al->thread, - .comm =thread__comm (al->thread), + .comm =thread__comm_time (al->thread, sample->time), .ip = al->addr, .ms = { .map = al->map,
@@ -801,7 +802,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter, he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , sample->period, sample->weight, - sample->transaction, false); + sample->transaction, sample->time, false);if (he ==NULL )return -ENOMEM ;
tools/perf/util/hist.h +1 -1
@@ -109,7 +109,7 @@ struct hist_entry *__hists__add_entry(struct hists *hists,struct branch_info *bi, struct mem_info *mi, u64 period, u64 weight, u64 transaction, - bool sample_self); + u64 timestamp, bool sample_self);int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al,struct perf_evsel *evsel,struct perf_sample *sample,int max_stack_depth,void *arg);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 21/42 perf tools: Convert dead thread list into rbtree
In reply to Namhyung KimCurrently perf maintains dead threads in a linked list but this can be a problem if someone needs to search from it especially in a large session which might have many dead threads. Convert it to a rbtree like normal threads and it'll be used later with multi-file changes.
The list node is now used for chaining dead threads of same tid since it's easier to handle such threads in time order.
--- tools/perf/util/machine.c | 70 +++++++++++++++++++++++++++++++++++++++++------ tools/perf/util/machine.h | 2 +- tools/perf/util/thread.c | 1 + tools/perf/util/thread.h | 11 ++++---- 4 files changed, 68 insertions(+), 16 deletions(-)
tools/perf/util/machine.c +61 -9
@@ -28,7 +28,7 @@ int machine__init(struct machine *machine, const char *root_dir, pid_t pid)dsos__init (&machine->kernel_dsos); machine->threads =RB_ROOT ; -INIT_LIST_HEAD (&machine->dead_threads); + machine->dead_threads =RB_ROOT ; machine->last_match =NULL ; machine->vdso_info =NULL ;
@@ -91,10 +91,22 @@ static void dsos__delete(struct dsos *dsos)void machine__delete_dead_threads (struct machine *machine) { -struct thread *n, *t; +struct rb_node *nd =rb_first (&machine->dead_threads); + +while (nd) { + struct thread *t = rb_entry(nd, struct thread, rb_node); +struct thread *pos; + + nd =rb_next (nd); +rb_erase (&t->rb_node, &machine->dead_threads); + +while (!list_empty (&t->tid_node)) { + pos =list_first_entry (&t->tid_node, + struct thread, tid_node); +list_del (&pos->tid_node); +thread__delete (pos); + } -list_for_each_entry_safe (t, n, &machine->dead_threads, node) { -list_del (&t->node);thread__delete (t); } }
@@ -106,8 +118,8 @@ void machine__delete_threads(struct machine *machine)while (nd) { struct thread *t = rb_entry(nd, struct thread, rb_node); -rb_erase (&t->rb_node, &machine->threads); nd =rb_next (nd); +rb_erase (&t->rb_node, &machine->threads);thread__delete (t); } }
@@ -1238,13 +1250,46 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *eventstatic void machine__remove_thread (struct machine *machine,struct thread *th) { +struct rb_node **p = &machine->dead_threads.rb_node; +struct rb_node *parent =NULL ; +struct thread *pos; + machine->last_match =NULL ;rb_erase (&th->rb_node, &machine->threads); + + th->dead = true; +/* * We may have references to this thread, for instance in some hist_entry -* instances, so just move them to a separate list. +* instances, so just move them to a separate list in rbtree. */ -list_add_tail (&th->node, &machine->dead_threads); +while (*p !=NULL ) { + parent = *p; + pos = rb_entry(parent, struct thread, rb_node); + +if (pos->tid == th->tid) { +struct thread *old; + +/* sort by time */ +list_for_each_entry (old, &pos->tid_node, tid_node) { +if (th->start_time >= old->start_time) { + pos = old; +break ; + } + } + +list_add_tail (&th->tid_node, &pos->tid_node); +return ; + } + +if (th->tid < pos->tid) + p = &(*p)->rb_left; +else + p = &(*p)->rb_right; + } + +rb_link_node (&th->rb_node, parent, p); +rb_insert_color (&th->rb_node, &machine->dead_threads); } int machine__process_fork_event(struct machine *machine, union perf_event *event,
@@ -1649,7 +1694,7 @@ int machine__for_each_thread(struct machine *machine, void *priv) {struct rb_node *nd; -struct thread *thread; +struct thread *thread, *pos;int rc =0 ;for (nd =rb_first (&machine->threads); nd; nd =rb_next (nd)) {
@@ -1659,10 +1704,17 @@ int machine__for_each_thread(struct machine *machine,return rc; } -list_for_each_entry (thread, &machine->dead_threads, node) { +for (nd =rb_first (&machine->dead_threads); nd; nd =rb_next (nd)) { + thread = rb_entry(nd, struct thread, rb_node); rc =fn (thread, priv);if (rc !=0 )return rc; + +list_for_each_entry (pos, &thread->tid_node, tid_node) { + rc =fn (pos, priv); +if (rc !=0 ) +return rc; + } }return rc; }
tools/perf/util/machine.h +1 -1
@@ -30,7 +30,7 @@ struct machine {bool comm_exec;char *root_dir;struct rb_root threads; -struct list_head dead_threads; +struct rb_root dead_threads;struct thread *last_match;struct vdso_info *vdso_info;struct dsos user_dsos;
tools/perf/util/thread.c +1 -0
@@ -38,6 +38,7 @@ struct thread *thread__new(pid_t pid, pid_t tid) thread->ppid =-1 ; thread->cpu =-1 ;INIT_LIST_HEAD (&thread->comm_list); +INIT_LIST_HEAD (&thread->tid_node);if (unwind__prepare_access (thread) <0 ) goto err_thread;
tools/perf/util/thread.h +5 -6
@@ -11,10 +11,8 @@struct thread_stack ;struct thread { -union { -struct rb_node rb_node; -struct list_head node; - }; +struct rb_node rb_node; +struct list_head tid_node;struct map_groups *mg;pid_t pid_;/* Not all tools update this */ pid_t tid;
@@ -22,7 +20,8 @@ struct thread {int cpu;char shortname[3 ];bool comm_set; -bool dead;/* if set thread has exited */ +bool exited;/* if set thread has exited */ +bool dead;/* thread is in dead_threads list */ struct list_head comm_list;int comm_len;u64 db_id;
@@ -39,7 +38,7 @@ int thread__init_map_groups(struct thread *thread, struct machine *machine);void thread__delete (struct thread *thread);static inline void thread__exited (struct thread *thread) { - thread->dead = true; + thread->exited = true; } int __thread__set_comm(struct thread *thread, const char *comm, u64 timestamp,
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 22/42 perf tools: Introduce machine__find*_thread_time()
In reply to Namhyung KimWith data file indexing is enabled, it needs to search thread based on sample time since sample processing is done after other (task, comm and mmap) events are processed. This can be a problem if a session is very long and pid is recycled - in that case it'll only see the last one.
So keep thread start time in it, and search thread based on the time. This patch introduces machine__find{,new}_thread_time() function for this. It'll first search current thread rbtree and then dead thread tree and list. If it couldn't find anyone, it'll create a new thread.
The sample timestamp of 0 means that this is called from synthesized event so just use current rbtree. The timestamp will be -1 if sample didn't record the timestamp so will see current threads automatically.
--- tools/perf/builtin-script.c | 11 ++++- tools/perf/tests/dwarf-unwind.c | 8 ++-- tools/perf/tests/hists_common.c | 3 +- tools/perf/tests/hists_link.c | 2 +- tools/perf/util/event.c | 14 ++++-- tools/perf/util/machine.c | 102 +++++++++++++++++++++++++++++++++++++++- tools/perf/util/machine.h | 8 +++- tools/perf/util/thread.c | 4 ++ tools/perf/util/thread.h | 1 + 9 files changed, 138 insertions(+), 15 deletions(-)
tools/perf/builtin-script.c +9 -2
@@ -559,8 +559,15 @@ static int process_sample_event(struct perf_tool *tool, {struct addr_location al; struct perf_script *script = container_of(tool, struct perf_script, tool); -struct thread *thread =machine__findnew_thread (machine, sample->pid, - sample->tid); +struct thread *thread; + +if (perf_session__has_index (script->session)) + thread =machine__findnew_thread_time (machine, sample->pid, + sample->tid, + sample->time); +else + thread =machine__findnew_thread (machine, sample->pid, + sample->tid);if (thread ==NULL ) { pr_debug("problem processing %d event, skipping it.\n",
tools/perf/tests/dwarf-unwind.c +3 -5
@@ -16,10 +16,10 @@static int mmap_handler (struct perf_tool *tool__maybe_unused ,union perf_event *event, -struct perf_sample *sample__maybe_unused , +struct perf_sample *sample,struct machine *machine) { -return machine__process_mmap2_event (machine, event,NULL ); +return machine__process_mmap2_event (machine, event, sample); }static int init_live_machine (struct machine *machine)
@@ -66,12 +66,10 @@ static int unwind_entry(struct unwind_entry *entry, void *arg) __attribute__ ((noinline))static int unwind_thread (struct thread *thread) { -struct perf_sample sample; +struct perf_sample sample = { .time =-1ULL , };unsigned long cnt =0 ;int err =-1 ; -memset (&sample,0 ,sizeof (sample)); -if (test__arch_unwind_sample (&sample, thread)) {pr_debug ("failed to get unwind sample\n" ); goto out;
tools/perf/tests/hists_common.c +2 -1
@@ -80,6 +80,7 @@ static struct {struct machine *setup_fake_machine (struct machines *machines) {struct machine *machine =machines__find (machines,HOST_KERNEL_ID ); +struct perf_sample sample = { .time =-1ULL , };size_t i;if (machine ==NULL ) {
@@ -113,7 +114,7 @@ struct machine *setup_fake_machine(struct machines *machines)strcpy (fake_mmap_event.mmap.filename, fake_mmap_info[i].filename); -machine__process_mmap_event (machine, &fake_mmap_event,NULL ); +machine__process_mmap_event (machine, &fake_mmap_event, &sample); }for (i =0 ; i <ARRAY_SIZE (fake_symbols); i++) {
tools/perf/tests/hists_link.c +1 -1
@@ -64,7 +64,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)struct perf_evsel *evsel;struct addr_location al;struct hist_entry *he; -struct perf_sample sample = { .period =1 , }; +struct perf_sample sample = { .period =1 , .time =-1ULL , };size_t i =0 , k;/*
tools/perf/util/event.c +11 -3
@@ -9,6 +9,7 @@#include "strlist.h" #include "thread.h" #include "thread_map.h" +#include "session.h" #include "symbol/kallsyms.h" static const char *perf_event__names[] = {
@@ -823,11 +824,18 @@ int perf_event__preprocess_sample(const union perf_event *event,struct machine *machine, struct addr_location *al, struct perf_sample *sample, - struct perf_session *session __maybe_unused) + struct perf_session *session) { u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK; -struct thread *thread =machine__findnew_thread (machine, sample->pid, - sample->tid); +struct thread *thread; + +if (session &&perf_session__has_index (session)) + thread =machine__findnew_thread_time (machine, sample->pid, + sample->tid, + sample->time); +else + thread =machine__findnew_thread (machine, sample->pid, + sample->tid);if (thread ==NULL )return -1 ;
tools/perf/util/machine.c +101 -1
114 lines not on this page · raw message
tools/perf/util/machine.h +6 -2
@@ -68,8 +68,6 @@ static inline bool machine__kernel_ip(struct machine *machine, u64 ip)return ip >= kernel_start; } -struct thread *machine__find_thread (struct machine *machine,pid_t pid, -pid_t tid);struct comm *machine__thread_exec_comm (struct machine *machine,struct thread *thread);
@@ -149,6 +147,12 @@ static inline bool machine__is_host(struct machine *machine)struct thread *machine__findnew_thread (struct machine *machine,pid_t pid,pid_t tid); +struct thread *machine__find_thread (struct machine *machine,pid_t pid, +pid_t tid); +struct thread *machine__findnew_thread_time (struct machine *machine,pid_t pid, +pid_t tid,u64 timestamp); +struct thread *machine__find_thread_time (struct machine *machine,pid_t pid, +pid_t tid,u64 timestamp);size_t machine__fprintf (struct machine *machine,FILE *fp);
tools/perf/util/thread.c +4 -0
@@ -127,6 +127,9 @@ int __thread__set_comm(struct thread *thread, const char *str, u64 timestamp,/* Override the default :tid entry */ if (!thread->comm_set) { +if (!thread->start_time) + thread->start_time = timestamp; + err =comm__override (curr, str, timestamp, exec);if (err)return err;
@@ -228,6 +231,7 @@ int thread__fork(struct thread *thread, struct thread *parent, u64 timestamp) } thread->ppid = parent->tid; + thread->start_time = timestamp;return thread__clone_map_groups (thread, parent); }
tools/perf/util/thread.h +1 -0
@@ -25,6 +25,7 @@ struct thread {struct list_head comm_list;int comm_len;u64 db_id; +u64 start_time;void *priv;struct thread_stack *ts;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 23/42 perf tools: Add a test case for timed thread handling
In reply to Namhyung KimA test case for verifying live and dead thread tree management during time change and new machine__find{,new}_thread_time().
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/builtin-test.c | 4 + tools/perf/tests/tests.h | 1 + tools/perf/tests/thread-lookup-time.c | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 180 insertions(+) create mode 100644 tools/perf/tests/thread-lookup-time.c
tools/perf/Makefile.perf +1 -0
@@ -459,6 +459,7 @@ LIB_OBJS += $(OUTPUT)tests/mmap-thread-lookup.o LIB_OBJS += $(OUTPUT)tests/thread-mg-share.o LIB_OBJS += $(OUTPUT)tests/switch-tracking.o LIB_OBJS += $(OUTPUT)tests/thread-comm.o +LIB_OBJS += $(OUTPUT)tests/thread-lookup-time.o BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
tools/perf/tests/builtin-test.c +4 -0
@@ -171,6 +171,10 @@ static struct test { .func = test__thread_comm, }, { + .desc = "Test thread lookup with time", + .func = test__thread_lookup_time, + }, + { .func = NULL, }, };
tools/perf/tests/tests.h +1 -0
@@ -52,6 +52,7 @@ int test__switch_tracking(void);int test__fdarray__filter (void );int test__fdarray__add (void );int test__thread_comm (void ); +int test__thread_lookup_time (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/tests/thread-lookup-time.c new file +174 -0
174 lines not on this page · raw message
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 24/42 perf tools: Maintain map groups list in a leader thread
In reply to Namhyung KimTo support multi-threaded perf report, we need to maintain time-sorted map groups. Add ->mg_list member to struct thread and sort the list by time. Now leader threads have one more refcnt for map groups in the list so also update the thread-mg-share test case.
Currently only add a new map groups when an exec (comm) event is received.
--- tools/perf/tests/thread-mg-share.c | 7 +++- tools/perf/util/event.c | 2 + tools/perf/util/machine.c | 2 +- tools/perf/util/map.c | 1 + tools/perf/util/map.h | 2 + tools/perf/util/thread.c | 80 +++++++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 3 ++ 7 files changed, 93 insertions(+), 4 deletions(-)
tools/perf/tests/thread-mg-share.c +5 -2
@@ -23,6 +23,9 @@ int test__thread_mg_share(void)* with several threads and checks they properly share and * maintain map groups info (struct map_groups). * +* Note that a leader thread has one more refcnt for its +* (current) map groups. +* * thread group (pid: 0, tids: 0, 1, 2, 3) * other group (pid: 4, tids: 4, 5) */
@@ -43,7 +46,7 @@ int test__thread_mg_share(void) leader && t1 && t2 && t3 && other); mg = leader->mg; -TEST_ASSERT_VAL ("wrong refcnt" , mg->refcnt ==4 ); +TEST_ASSERT_VAL ("wrong refcnt" , mg->refcnt ==5 );/* test the map groups pointer is shared */ TEST_ASSERT_VAL ("map groups don't match" , mg == t1->mg);
@@ -59,7 +62,7 @@ int test__thread_mg_share(void)TEST_ASSERT_VAL ("failed to find other leader" , other_leader); other_mg = other->mg; -TEST_ASSERT_VAL ("wrong refcnt" , other_mg->refcnt ==2 ); +TEST_ASSERT_VAL ("wrong refcnt" , other_mg->refcnt ==3 );TEST_ASSERT_VAL ("map groups don't match" , other_mg == other_leader->mg);
tools/perf/util/event.c +2 -0
@@ -751,6 +751,8 @@ void thread__find_addr_map(struct thread *thread, u8 cpumode,return ; } +BUG_ON (mg ==NULL ); +if (cpumode ==PERF_RECORD_MISC_KERNEL && perf_host) { al->level ='k' ; mg = &machine->kmaps;
tools/perf/util/machine.c +1 -1
@@ -331,7 +331,7 @@ static void machine__update_thread_pid(struct machine *machine, goto out_err;if (!leader->mg) - leader->mg =map_groups__new (machine); +thread__set_map_groups (leader,map_groups__new (machine),0 );if (!leader->mg) goto out_err;
tools/perf/util/map.c +1 -0
@@ -422,6 +422,7 @@ void map_groups__init(struct map_groups *mg, struct machine *machine) } mg->machine = machine; mg->refcnt =1 ; + mg->timestamp =0 ; }static void maps__delete (struct rb_root *maps)
tools/perf/util/map.h +2 -0
@@ -61,7 +61,9 @@ struct map_groups {struct rb_root maps[MAP__NR_TYPES ];struct list_head removed_maps[MAP__NR_TYPES ];struct machine *machine; +u64 timestamp;int refcnt; +struct list_head list; };struct map_groups *map_groups__new (struct machine *machine);
tools/perf/util/thread.c +79 -1
116 lines not on this page · raw message
tools/perf/util/thread.h +3 -0
@@ -14,6 +14,7 @@ struct thread {struct rb_node rb_node;struct list_head tid_node;struct map_groups *mg; +struct list_head mg_list;pid_t pid_;/* Not all tools update this */ pid_t tid;pid_t ppid;
@@ -56,6 +57,8 @@ struct comm *thread__exec_comm(const struct thread *thread);struct comm *thread__comm_time (const struct thread *thread,u64 timestamp);const char *thread__comm_str (const struct thread *thread);const char *thread__comm_str_time (const struct thread *thread,u64 timestamp); +struct map_groups *thread__get_map_groups (struct thread *thread,u64 timestamp); +int thread__set_map_groups (struct thread *thread,struct map_groups *mg,u64 timestamp);void thread__insert_map (struct thread *thread,struct map *map);int thread__fork (struct thread *thread,struct thread *parent,u64 timestamp);size_t thread__fprintf (struct thread *thread,FILE *fp);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 25/42 perf tools: Introduce thread__find_addr_location_time() and friends
In reply to Namhyung KimThe *_time() variants are for find appropriate map (and symbol) at the given time. This is based on the fact that map_groups list is sorted by time in the previous patch.
--- tools/perf/util/event.c | 80 +++++++++++++++++++++++++++++++++----- tools/perf/util/machine.c | 51 ++++++++++++++---------- tools/perf/util/thread.c | 21 ++++++++++ tools/perf/util/thread.h | 10 +++++ tools/perf/util/unwind-libdw.c | 11 +++--- tools/perf/util/unwind-libunwind.c | 18 ++++----- 6 files changed, 146 insertions(+), 45 deletions(-)
tools/perf/util/event.c +70 -10
@@ -732,16 +732,14 @@ int perf_event__process(struct perf_tool *tool __maybe_unused,return machine__process_event (machine, event, sample); } -void thread__find_addr_map (struct thread *thread,u8 cpumode, -enum map_type type,u64 addr, -struct addr_location *al) +static void map_groups__find_addr_map (struct map_groups *mg,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al) { -struct map_groups *mg = thread->mg;struct machine *machine = mg->machine;bool load_map = false; al->machine = machine; - al->thread = thread; al->addr = addr; al->cpumode = cpumode; al->filtered =0 ;
@@ -810,6 +808,36 @@ void thread__find_addr_map(struct thread *thread, u8 cpumode, } } +void thread__find_addr_map (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al) +{ + al->thread = thread; +map_groups__find_addr_map (thread->mg, cpumode, type, addr, al); +} + +void thread__find_addr_map_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp) +{ +struct map_groups *mg; +struct thread *leader; + +if (thread->tid == thread->pid_) + leader = thread; +else + leader =machine__findnew_thread_time (thread->mg->machine, + thread->pid_, + thread->pid_, + timestamp); +BUG_ON (leader ==NULL ); + + mg =thread__get_map_groups (leader, timestamp); + + al->thread = thread; +map_groups__find_addr_map (mg, cpumode, type, addr, al); +} +void thread__find_addr_location (struct thread *thread,u8 cpumode,enum map_type type,u64 addr,struct addr_location *al)
@@ -822,6 +850,21 @@ void thread__find_addr_location(struct thread *thread, al->sym =NULL ; } +void thread__find_addr_location_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp) +{ +struct map_groups *mg; + + mg =thread__get_map_groups (thread, timestamp); +map_groups__find_addr_map (mg, cpumode, type, addr, al); +if (al->map !=NULL ) + al->sym =map__find_symbol (al->map, al->addr, + mg->machine->symbol_filter); +else + al->sym =NULL ; +} + int perf_event__preprocess_sample(const union perf_event *event, struct machine *machine, struct addr_location *al,
@@ -854,7 +897,13 @@ int perf_event__preprocess_sample(const union perf_event *event, machine->vmlinux_maps[MAP__FUNCTION] == NULL)machine__create_kernel_maps (machine); -thread__find_addr_map (thread, cpumode,MAP__FUNCTION , sample->ip, al); +if (session &&perf_session__has_index (session)) +thread__find_addr_map_time (thread, cpumode,MAP__FUNCTION , + sample->ip, al, sample->time); +else +thread__find_addr_map (thread, cpumode,MAP__FUNCTION , + sample->ip, al); +dump_printf (" ...... dso: %s\n" , al->map ? al->map->dso->long_name : al->level =='H' ?"[hypervisor]" :"<not found>" );
@@ -915,15 +964,26 @@ void perf_event__preprocess_sample_addr(union perf_event *event,struct perf_sample *sample, struct thread *thread, struct addr_location *al, - struct perf_session *session __maybe_unused) + struct perf_session *session) { u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK; -thread__find_addr_map (thread, cpumode,MAP__FUNCTION , sample->addr, al); -if (!al->map) -thread__find_addr_map (thread, cpumode,MAP__VARIABLE , +if (session &&perf_session__has_index (session)) +thread__find_addr_map_time (thread, cpumode,MAP__FUNCTION , + sample->addr, al, sample->time); +else +thread__find_addr_map (thread, cpumode,MAP__FUNCTION , sample->addr, al); +if (!al->map) { +if (session &&perf_session__has_index (session)) +thread__find_addr_map_time (thread, cpumode,MAP__VARIABLE , + sample->addr, al, sample->time); +else +thread__find_addr_map (thread, cpumode,MAP__VARIABLE , + sample->addr, al); + } + al->cpu = sample->cpu; al->sym =NULL ;
tools/perf/util/machine.c +30 -21
136 lines not on this page · raw message
tools/perf/util/thread.c +21 -0
@@ -331,3 +331,24 @@ void thread__find_cpumode_addr_location(struct thread *thread,break ; } } + +void thread__find_cpumode_addr_location_time (struct thread *thread, +enum map_type type,u64 addr, +struct addr_location *al, +u64 timestamp) +{ +size_t i; +const u8 const cpumodes[] = { +PERF_RECORD_MISC_USER , +PERF_RECORD_MISC_KERNEL , +PERF_RECORD_MISC_GUEST_USER , +PERF_RECORD_MISC_GUEST_KERNEL + }; + +for (i =0 ; i <ARRAY_SIZE (cpumodes); i++) { +thread__find_addr_location_time (thread, cpumodes[i], type, + addr, al, timestamp); +if (al->map) +break ; + } +}
tools/perf/util/thread.h +10 -0
@@ -66,14 +66,24 @@ size_t thread__fprintf(struct thread *thread, FILE *fp);void thread__find_addr_map (struct thread *thread,u8 cpumode,enum map_type type,u64 addr,struct addr_location *al); +void thread__find_addr_map_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp);void thread__find_addr_location (struct thread *thread,u8 cpumode,enum map_type type,u64 addr,struct addr_location *al); +void thread__find_addr_location_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp);void thread__find_cpumode_addr_location (struct thread *thread,enum map_type type,u64 addr,struct addr_location *al); +void thread__find_cpumode_addr_location_time (struct thread *thread, +enum map_type type,u64 addr, +struct addr_location *al, +u64 timestamp);static inline void *thread__priv (struct thread *thread) {
tools/perf/util/unwind-libdw.c +6 -5
@@ -26,9 +26,10 @@ static int __report_module(struct addr_location *al, u64 ip,Dwfl_Module *mod;struct dso *dso =NULL ; -thread__find_addr_location (ui->thread, -PERF_RECORD_MISC_USER , -MAP__FUNCTION , ip, al); +thread__find_addr_location_time (ui->thread, +PERF_RECORD_MISC_USER , +MAP__FUNCTION , ip, al, + ui->sample->time);if (al->map) dso = al->map->dso;
@@ -89,8 +90,8 @@ static int access_dso_mem(struct unwind_info *ui, Dwarf_Addr addr,struct addr_location al;ssize_t size; -thread__find_addr_map (ui->thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , addr, &al); +thread__find_addr_map_time (ui->thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , addr, &al, ui->sample->time);if (!al.map) {pr_debug ("unwind: no map for %lx\n" , (unsigned long )addr);return -1 ;
tools/perf/util/unwind-libunwind.c +9 -9
@@ -306,8 +306,8 @@ static struct map *find_map(unw_word_t ip, struct unwind_info *ui) {struct addr_location al; -thread__find_addr_map (ui->thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , ip, &al); +thread__find_addr_map_time (ui->thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , ip, &al, ui->sample->time);return al.map; }
@@ -400,8 +400,8 @@ static int access_dso_mem(struct unwind_info *ui, unw_word_t addr,struct addr_location al;ssize_t size; -thread__find_addr_map (ui->thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , addr, &al); +thread__find_addr_map_time (ui->thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , addr, &al, ui->sample->time);if (!al.map) {pr_debug ("unwind: no map for %lx\n" , (unsigned long )addr);return -1 ;
@@ -502,14 +502,14 @@ static void put_unwind_info(unw_addr_space_t __maybe_unused as,pr_debug ("unwind: put_unwind_info called\n" ); } -static int entry (u64 ip,struct thread *thread, +static int entry (u64 ip,struct thread *thread,u64 timestamp,unwind_entry_cb_t cb,void *arg) {struct unwind_entry e;struct addr_location al; -thread__find_addr_location (thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , ip, &al); +thread__find_addr_location_time (thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , ip, &al, timestamp); e.ip = ip; e.map = al.map;
@@ -611,7 +611,7 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,unw_word_t ip;unw_get_reg (&c,UNW_REG_IP , &ip); - ret = ip ?entry (ip, ui->thread, cb, arg) :0 ; + ret = ip ?entry (ip, ui->thread, ui->sample->time, cb, arg) :0 ; }return ret;
@@ -636,7 +636,7 @@ int unwind__get_entries(unwind_entry_cb_t cb, void *arg,if (ret)return ret; - ret =entry (ip, thread, cb, arg); + ret =entry (ip, thread, data->time, cb, arg);if (ret)return -ENOMEM ;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 26/42 perf tools: Add a test case for timed map groups handling
In reply to Namhyung KimA test case for verifying thread->mg and ->mg_list handling during time change and new thread__find_addr_map_time() and friends.
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/tests.h | 1 + tools/perf/tests/thread-mg-time.c | 88 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 94 insertions(+) create mode 100644 tools/perf/tests/thread-mg-time.c
tools/perf/Makefile.perf +1 -0
@@ -460,6 +460,7 @@ LIB_OBJS += $(OUTPUT)tests/thread-mg-share.o LIB_OBJS += $(OUTPUT)tests/switch-tracking.o LIB_OBJS += $(OUTPUT)tests/thread-comm.o LIB_OBJS += $(OUTPUT)tests/thread-lookup-time.o +LIB_OBJS += $(OUTPUT)tests/thread-mg-time.o BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
tools/perf/tests/builtin-test.c +4 -0
@@ -175,6 +175,10 @@ static struct test { .func = test__thread_lookup_time, }, { + .desc = "Test thread map group handling with time", + .func = test__thread_mg_time, + }, + { .func = NULL, }, };
tools/perf/tests/tests.h +1 -0
@@ -53,6 +53,7 @@ int test__fdarray__filter(void);int test__fdarray__add (void );int test__thread_comm (void );int test__thread_lookup_time (void ); +int test__thread_mg_time (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/tests/thread-mg-time.c new file +88 -0
88 lines not on this page · raw message
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 27/42 perf tools: Protect dso symbol loading using a mutex
In reply to Namhyung KimWhen multi-thread support for perf report is enabled, it's possible to access a dso concurrently. Add a new pthread_mutex to protect it from concurrent dso__load().
--- tools/perf/util/dso.c | 2 ++ tools/perf/util/dso.h | 1 + tools/perf/util/symbol.c | 34 ++++++++++++++++++++++++---------- 3 files changed, 27 insertions(+), 10 deletions(-)
tools/perf/util/dso.c +2 -0
@@ -888,6 +888,7 @@ struct dso *dso__new(const char *name)RB_CLEAR_NODE (&dso->rb_node);INIT_LIST_HEAD (&dso->node);INIT_LIST_HEAD (&dso->data.open_entry); +pthread_mutex_init (&dso->lock,NULL ); }return dso;
@@ -917,6 +918,7 @@ void dso__delete(struct dso *dso)dso_cache__free (&dso->data.cache);dso__free_a2l (dso);zfree (&dso->symsrc_filename); +pthread_mutex_destroy (&dso->lock);free (dso); }
tools/perf/util/dso.h +1 -0
@@ -102,6 +102,7 @@ struct dsos { };struct dso { +pthread_mutex_t lock;struct list_head node;struct rb_node rb_node;/* rbtree node sorted by long name */ struct rb_root symbols[MAP__NR_TYPES ];
tools/perf/util/symbol.c +24 -10
@@ -1357,12 +1357,22 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)struct symsrc *syms_ss =NULL , *runtime_ss =NULL ;bool kmod; -dso__set_loaded (dso, map->type); +pthread_mutex_lock (&dso->lock); + +/* check again under the dso->lock */ +if (dso__loaded (dso, map->type)) { + ret =1 ; + goto out; + } + +if (dso->kernel) { +if (dso->kernel ==DSO_TYPE_KERNEL ) + ret =dso__load_kernel_sym (dso, map, filter); +else if (dso->kernel ==DSO_TYPE_GUEST_KERNEL ) + ret =dso__load_guest_kernel_sym (dso, map, filter); -if (dso->kernel ==DSO_TYPE_KERNEL ) -return dso__load_kernel_sym (dso, map, filter); -else if (dso->kernel ==DSO_TYPE_GUEST_KERNEL ) -return dso__load_guest_kernel_sym (dso, map, filter); + goto out; + }if (map->groups && map->groups->machine) machine = map->groups->machine;
@@ -1375,18 +1385,18 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)struct stat st;if (lstat (dso->name, &st) <0 ) -return -1 ; + goto out;if (st.st_uid && (st.st_uid !=geteuid ())) {pr_warning ("File %s not owned by current user or root, " "ignoring it.\n" , dso->name); -return -1 ; + goto out; } ret =dso__load_perf_map (dso, map, filter); dso->symtab_type = ret >0 ?DSO_BINARY_TYPE__JAVA_JIT :DSO_BINARY_TYPE__NOT_FOUND ; -return ret; + goto out; }if (machine)
@@ -1394,7 +1404,7 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter) name =malloc (PATH_MAX );if (!name) -return -1 ; + goto out; kmod = dso->symtab_type ==DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE || dso->symtab_type == DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP ||
@@ -1475,7 +1485,11 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter) out_free:free (name);if (ret <0 &&strstr (dso->name," (deleted)" ) !=NULL ) -return 0 ; + ret =0 ; +out: +dso__set_loaded (dso, map->type); +pthread_mutex_unlock (&dso->lock); +return ret; }
Signature · 1 line
2.2.2
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 27/42] perf tools: Protect dso symbol loading using a mutex
Em Thu, Jan 29, 2015 at 05:07:08PM +0900, Namhyung Kim escreveu:
Quoted 48 lines
When multi-thread support for perf report is enabled, it's possible to access a dso concurrently. Add a new
pthread_mutexto protect it from concurrentdso__load().Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/dso.c | 2 ++ tools/perf/util/dso.h | 1 + tools/perf/util/symbol.c | 34 ++++++++++++++++++++++++---------- 3 files changed, 27 insertions(+), 10 deletions(-)tools/perf/util/dso.c +2
-0@@ -888,6 +888,7 @@ struct dso *dso__new(const char *name)RB_CLEAR_NODE (&dso->rb_node);INIT_LIST_HEAD (&dso->node);INIT_LIST_HEAD (&dso->data.open_entry); +pthread_mutex_init (&dso->lock,NULL ); }return dso;@@ -917,6 +918,7 @@ void dso__delete(struct dso *dso)dso_cache__free (&dso->data.cache);dso__free_a2l (dso);zfree (&dso->symsrc_filename); +pthread_mutex_destroy (&dso->lock);free (dso); }tools/perf/util/dso.h +1
-0@@ -102,6 +102,7 @@ struct dsos { };struct dso { +pthread_mutex_t lock;struct list_head node;struct rb_node rb_node;/* rbtree node sorted by long name */ struct rb_root symbols[MAP__NR_TYPES ];tools/perf/util/symbol.c +0
-0
tools/perf/util/symbol.c int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter) · line 1362
Quoted 7 lines
tools/perf/util/symbol.c +3
-1@@ -1357,12 +1357,22 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)struct symsrc *syms_ss =NULL , *runtime_ss =NULL ;bool kmod; -dso__set_loaded (dso, map->type); +pthread_mutex_lock (&dso->lock); + +/* check again under the dso->lock */
Again? Where was it first checked? Perhaps we should lock there, so that we don't have to do two checks, one unlocked, the other locked?
tools/perf/util/symbol.c int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter) · line 1367
Quoted 5 lines
tools/perf/util/symbol.c +5
-0+if (dso__loaded (dso, map->type)) { + ret =1 ; + goto out; + } +
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 27/42] perf tools: Protect dso symbol loading using a mutex
Hi Arnaldo,
Quoted 52 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 29, 2015 at 05:07:08PM +0900, Namhyung Kim escreveu:
Quoted 51 lines
When multi-thread support for perf report is enabled, it's possible to access a dso concurrently. Add a new
pthread_mutexto protect it from concurrentdso__load().Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/dso.c | 2 ++ tools/perf/util/dso.h | 1 + tools/perf/util/symbol.c | 34 ++++++++++++++++++++++++---------- 3 files changed, 27 insertions(+), 10 deletions(-)tools/perf/util/dso.c +2
-0@@ -888,6 +888,7 @@ struct dso *dso__new(const char *name)RB_CLEAR_NODE (&dso->rb_node);INIT_LIST_HEAD (&dso->node);INIT_LIST_HEAD (&dso->data.open_entry); +pthread_mutex_init (&dso->lock,NULL ); }return dso;@@ -917,6 +918,7 @@ void dso__delete(struct dso *dso)dso_cache__free (&dso->data.cache);dso__free_a2l (dso);zfree (&dso->symsrc_filename); +pthread_mutex_destroy (&dso->lock);free (dso); }tools/perf/util/dso.h +1
-0@@ -102,6 +102,7 @@ struct dsos { };struct dso { +pthread_mutex_t lock;struct list_head node;struct rb_node rb_node;/* rbtree node sorted by long name */ struct rb_root symbols[MAP__NR_TYPES ];tools/perf/util/symbol.c +0
-0@@ -1357,12 +1357,22 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)struct symsrc *syms_ss =NULL , *runtime_ss =NULL ;bool kmod;
Quoted 6 lines
tools/perf/util/symbol.c int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter) · line 1362
Quoted 4 lines
tools/perf/util/symbol.c +3
-1-dso__set_loaded (dso, map->type); +pthread_mutex_lock (&dso->lock); + +/* check again under the dso->lock */ Again? Where was it first checked?
Please see map__load().
Quoted 2 lines
Perhaps we should lock there, so that
we don't have to do two checks, one unlocked, the other locked?
Hmm.. maybe. I just keep it to avoid locking overhead since it'll be called whenever it searches symbols during preprocessing. I didn't measure the overhead but it could be huge IMHO.
Thanks,
Namhyung
Quoted 8 lines
tools/perf/util/symbol.c int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter) · line 1367
Quoted 6 lines
tools/perf/util/symbol.c +5
-0+if (dso__loaded (dso, map->type)) { + ret =1 ; + goto out; + } +- Arnaldo
Namhyung Kim
maintainerPATCH 28/42 perf tools: Protect dso cache tree using dso->lock
In reply to Namhyung KimThe dso cache is accessed during dwarf callchain unwind and it might be processed concurrently when multi-thread report is enabled. Protect it under dso->lock.
Note that it doesn't protect dso_cache__find(). I think it's safe to access to the cache tree without the lock since we don't delete nodes. It it missed an existing node due to rotation, it'll find it during dso_cache__insert() anyway.
--- tools/perf/util/dso.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)
tools/perf/util/dso.c +27 -7
@@ -443,10 +443,12 @@ bool dso__data_status_seen(struct dso *dso, enum dso_data_status_seen by) }static void -dso_cache__free (struct rb_root *root) +dso_cache__free (struct dso *dso) { +struct rb_root *root = &dso->data.cache;struct rb_node *next =rb_first (root); +pthread_mutex_lock (&dso->lock);while (next) {struct dso_cache *cache;
@@ -455,10 +457,12 @@ dso_cache__free(struct rb_root *root)rb_erase (&cache->rb_node, root);free (cache); } +pthread_mutex_unlock (&dso->lock); } -static struct dso_cache *dso_cache__find (const struct rb_root *root,u64 offset) +static struct dso_cache *dso_cache__find (struct dso *dso,u64 offset) { +const struct rb_root *root = &dso->data.cache;struct rb_node *const *p = &root->rb_node;const struct rb_node *parent =NULL ;struct dso_cache *cache;
@@ -477,17 +481,20 @@ static struct dso_cache *dso_cache__find(const struct rb_root *root, u64 offset)else return cache; } +return NULL ; } -static void -dso_cache__insert (struct rb_root *root,struct dso_cache *new) +static struct dso_cache * +dso_cache__insert (struct dso *dso,struct dso_cache *new) { +struct rb_root *root = &dso->data.cache;struct rb_node **p = &root->rb_node;struct rb_node *parent =NULL ;struct dso_cache *cache;u64 offset = new->offset; +pthread_mutex_lock (&dso->lock);while (*p !=NULL ) {u64 end;
@@ -499,10 +506,17 @@ dso_cache__insert(struct rb_root *root, struct dso_cache *new) p = &(*p)->rb_left; else if (offset >= end) p = &(*p)->rb_right; + else + goto out; }rb_link_node (&new->rb_node, parent, p);rb_insert_color (&new->rb_node, root); + + cache =NULL ; +out: +pthread_mutex_unlock (&dso->lock); +return cache; } static ssize_t
@@ -520,6 +534,7 @@ static ssize_t dso_cache__read(struct dso *dso, u64 offset, u8 *data, ssize_t size) { struct dso_cache *cache; +struct dso_cache *old;ssize_t ret; do {
@@ -543,7 +558,12 @@ dso_cache__read(struct dso *dso, u64 offset, u8 *data, ssize_t size) cache->offset = cache_offset; cache->size = ret; -dso_cache__insert (&dso->data.cache, cache); + old =dso_cache__insert (dso, cache); +if (old) { +/* we lose the race */ +free (cache); + cache = old; + } ret =dso_cache__memcpy (cache, offset, data, size);
@@ -560,7 +580,7 @@ static ssize_t dso_cache_read(struct dso *dso, u64 offset, {struct dso_cache *cache; - cache =dso_cache__find (&dso->data.cache, offset); + cache =dso_cache__find (dso, offset);if (cache)return dso_cache__memcpy (cache, offset, data, size);else
@@ -915,7 +935,7 @@ void dso__delete(struct dso *dso) }dso__data_close (dso); -dso_cache__free (&dso->data.cache); +dso_cache__free (dso);dso__free_a2l (dso);zfree (&dso->symsrc_filename);pthread_mutex_destroy (&dso->lock);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 29/42 perf tools: Protect dso cache fd with a mutex
In reply to Namhyung KimWhen dso cache is accessed in multi-thread environment, it's possible to close other dso->data.fd during operation due to open file limit. Protect the file descriptors using a separate mutex.
--- tools/perf/tests/dso-data.c | 5 ++ tools/perf/util/dso.c | 136 +++++++++++++++++++++++++++++--------------- 2 files changed, 94 insertions(+), 47 deletions(-)
tools/perf/tests/dso-data.c +5 -0
@@ -111,6 +111,9 @@ int test__dso_data(void)memset (&machine,0 ,sizeof (machine)); dso =dso__new ((const char *)file); +TEST_ASSERT_VAL ("failed to get dso" , dso); + + dso->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;/* Basic 10 bytes tests. */ for (i =0 ; i <ARRAY_SIZE (offsets); i++) {
@@ -199,6 +202,8 @@ static int dsos__create(int cnt, int size) dsos[i] =dso__new (file);TEST_ASSERT_VAL ("failed to get dso" , dsos[i]); + + dsos[i]->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ; }return 0 ;
tools/perf/util/dso.c +89 -47
249 lines not on this page · raw message
Signature · 1 line
2.2.2
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 29/42] perf tools: Protect dso cache fd with a mutex
Em Thu, Jan 29, 2015 at 05:07:10PM +0900, Namhyung Kim escreveu:
Quoted 25 lines
When dso cache is accessed in multi-thread environment, it's possible to close other
dso->data.fdduring operation due to open file limit. Protect the file descriptors using a separate mutex.Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/tests/dso-data.c | 5 ++ tools/perf/util/dso.c | 136 +++++++++++++++++++++++++++++--------------- 2 files changed, 94 insertions(+), 47 deletions(-)tools/perf/tests/dso-data.c +3
-0@@ -111,6 +111,9 @@ int test__dso_data(void)memset (&machine,0 ,sizeof (machine)); dso =dso__new ((const char *)file); +TEST_ASSERT_VAL ("failed to get dso" , dso); + + dso->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;/* Basic 10 bytes tests. */ for (i =0 ; i <ARRAY_SIZE (offsets); i++) {
tools/perf/tests/dso-data.c static int dsos__create(int cnt, int size) · line 206
Quoted 5 lines
tools/perf/tests/dso-data.c +2
-0@@ -199,6 +202,8 @@ static int dsos__create(int cnt, int size) dsos[i] =dso__new (file);TEST_ASSERT_VAL ("failed to get dso" , dsos[i]); + + dsos[i]->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;
Those two are unrelated, please put them in a separate patch, one that I can even cherrypick ahead of the other patches.
Quoted 16 lines
tools/perf/tests/dso-data.c +0
-0}return 0 ;tools/perf/util/dso.c +1
-0@@ -213,6 +213,7 @@ bool dso__needs_decompress(struct dso *dso)*/ static LIST_HEAD (dso__data_open);static long dso__data_open_cnt; +static pthread_mutex_t dso__data_open_lock =PTHREAD_MUTEX_INITIALIZER ;static void dso__list_add (struct dso *dso) {
tools/perf/util/dso.c static int do_open(char *name) · line 246
Quoted 7 lines
tools/perf/util/dso.c +1
-1@@ -240,7 +241,7 @@ static int do_open(char *name)if (fd >=0 )return fd; -pr_debug ("dso open failed, mmap: %s\n" , +pr_debug ("dso open failed: %s\n" ,strerror_r (errno, sbuf,sizeof (sbuf)));if (!dso__data_open_cnt || errno !=EMFILE )
Ditto, another unrelated patch, please separate.
Quoted 32 lines
tools/perf/util/dso.c +5
-0break ;@@ -382,7 +383,9 @@ static void check_data_close(void)*/ void dso__data_close (struct dso *dso) { +pthread_mutex_lock (&dso__data_open_lock);close_dso (dso); +pthread_mutex_unlock (&dso__data_open_lock); }/** @@ -405,6 +408,8 @@ int dso__data_fd(struct dso *dso, struct machine *machine)if (dso->data.status ==DSO_DATA_STATUS_ERROR )return -1 ; +pthread_mutex_lock (&dso__data_open_lock); +if (dso->data.fd >=0 ) goto out;@@ -427,6 +432,7 @@ int dso__data_fd(struct dso *dso, struct machine *machine)else dso->data.status =DSO_DATA_STATUS_ERROR ; +pthread_mutex_unlock (&dso__data_open_lock);return dso->data.fd; }@@ -531,52 +537,66 @@ dso_cache__memcpy(struct dso_cache *cache, u64 offset, } static ssize_t
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 541
Quoted 10 lines
tools/perf/util/dso.c +2
-5-dso_cache__read(struct dso *dso, u64 offset, u8 *data, ssize_t size) +dso_cache__read(struct dso *dso, struct machine *machine, + u64 offset, u8 *data, ssize_t size) { struct dso_cache *cache;struct dso_cache *old; -ssize_t ret; - - do { - u64 cache_offset;
While I understand that there was no need for this do { } while (0) construct in the first place, removing it in this patch is not interesting, as it is both unrelated to this patch and makes the it harder to review by just looking at the patch :-\ Please refrain from doing this in this patch.
A later patch that does _just_ that could be done, if you feel like doing it.
Quoted 15 lines
tools/perf/util/dso.c +6
-6+ssize_t ret = -EINVAL ; +u64 cache_offset; - ret = -ENOMEM ; + cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); +if (!cache) +return -ENOMEM ; - cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); -if (!cache) -break ; + cache_offset = offset &DSO__DATA_CACHE_MASK ; - cache_offset = offset &DSO__DATA_CACHE_MASK ; - ret = -EINVAL ;
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 561
Quoted 10 lines
tools/perf/util/dso.c +7
-2+pthread_mutex_lock (&dso__data_open_lock); -if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) -break ; +/* +* dso->data.fd might be closed if other thread opened another +* file (dso) due to open file limit (RLIMIT_NOFILE). +*/ +if (dso->data.fd <0 ) { + dso->data.fd =open_dso (dso, machine);
Also please consider adding a backpointer to machine in the dso object, since you need to reopen it, so that we don't have to go on passing machine around to dso_cache__read(), etc.
This probably needs to be done in the patch that makes dso->data.fd to be closed due to limit.
Quoted 181 lines
tools/perf/util/dso.c +67
-33+if (dso->data.fd <0 ) { + ret = -errno; + dso->data.status =DSO_DATA_STATUS_ERROR ; + goto err_unlock; + } + } - ret =read (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE ); -if (ret <=0 ) -break ; +if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) + goto err_unlock; - cache->offset = cache_offset; - cache->size = ret; - old =dso_cache__insert (dso, cache); -if (old) { -/* we lose the race */ -free (cache); - cache = old; - } + ret =read (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE ); +if (ret <=0 ) + goto err_unlock; - ret =dso_cache__memcpy (cache, offset, data, size); +pthread_mutex_unlock (&dso__data_open_lock); - }while (0 ); + cache->offset = cache_offset; + cache->size = ret; + old =dso_cache__insert (dso, cache); +if (old) { +/* we lose the race */ +free (cache); + cache = old; + } + ret =dso_cache__memcpy (cache, offset, data, size);if (ret <=0 )free (cache);return ret; + +err_unlock: +pthread_mutex_unlock (&dso__data_open_lock); +return ret; } -static ssize_t dso_cache_read (struct dso *dso,u64 offset, -u8 *data,ssize_t size) +static ssize_t dso_cache_read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {struct dso_cache *cache;@@ -584,7 +604,7 @@ static ssize_t dso_cache_read(struct dso *dso, u64 offset,if (cache)return dso_cache__memcpy (cache, offset, data, size);else -return dso_cache__read (dso, offset, data, size); +return dso_cache__read (dso, machine, offset, data, size); }/* @@ -592,7 +612,8 @@ static ssize_t dso_cache_read(struct dso *dso, u64 offset,* in the rb_tree. Any read to already cached data is served * by cached data. */ -static ssize_t cached_read (struct dso *dso,u64 offset,u8 *data,ssize_t size) +static ssize_t cached_read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {ssize_t r =0 ;u8 *p = data;@@ -600,7 +621,7 @@ static ssize_t cached_read(struct dso *dso, u64 offset, u8 *data, ssize_t size) do { ssize_t ret; - ret = dso_cache_read(dso, offset, p, size); + ret = dso_cache_read(dso, machine, offset, p, size); if (ret < 0) return ret;@@ -620,21 +641,42 @@ static ssize_t cached_read(struct dso *dso, u64 offset, u8 *data, ssize_t size)return r; } -static int data_file_size (struct dso *dso) +static int data_file_size (struct dso *dso,struct machine *machine) { +int ret =0 ;struct stat st;char sbuf[STRERR_BUFSIZE ]; -if (!dso->data.file_size) { -if (fstat (dso->data.fd, &st)) { -pr_err ("dso mmap failed, fstat: %s\n" , -strerror_r (errno, sbuf,sizeof (sbuf))); -return -1 ; +if (dso->data.file_size) +return 0 ; + +pthread_mutex_lock (&dso__data_open_lock); + +/* +* dso->data.fd might be closed if other thread opened another +* file (dso) due to open file limit (RLIMIT_NOFILE). +*/ +if (dso->data.fd <0 ) { + dso->data.fd =open_dso (dso, machine); +if (dso->data.fd <0 ) { + ret = -errno; + dso->data.status =DSO_DATA_STATUS_ERROR ; + goto out; } - dso->data.file_size = st.st_size; } -return 0 ; +if (fstat (dso->data.fd, &st) <0 ) { + ret = -errno; +pr_err ("dso cache fstat failed: %s\n" , +strerror_r (errno, sbuf,sizeof (sbuf))); + dso->data.status =DSO_DATA_STATUS_ERROR ; + goto out; + } + dso->data.file_size = st.st_size; + +out: +pthread_mutex_unlock (&dso__data_open_lock); +return ret; }/** @@ -652,17 +694,17 @@ off_t dso__data_size(struct dso *dso, struct machine *machine)if (fd <0 )return fd; -if (data_file_size (dso)) +if (data_file_size (dso, machine))return -1 ;/* For now just estimate dso data size is close to file size */ return dso->data.file_size; } -static ssize_t data_read_offset (struct dso *dso,u64 offset, -u8 *data,ssize_t size) +static ssize_t data_read_offset (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) { -if (data_file_size (dso)) +if (data_file_size (dso, machine))return -1 ;/* Check the offset sanity. */ @@ -672,7 +714,7 @@ static ssize_t data_read_offset(struct dso *dso, u64 offset,if (offset + size < offset)return -1 ; -return cached_read (dso, offset, data, size); +return cached_read (dso, machine, offset, data, size); }/** @@ -689,10 +731,10 @@ static ssize_t data_read_offset(struct dso *dso, u64 offset,ssize_t dso__data_read_offset (struct dso *dso,struct machine *machine,u64 offset,u8 *data,ssize_t size) { -if (dso__data_fd (dso, machine) <0 ) +if (dso->data.status ==DSO_DATA_STATUS_ERROR )return -1 ; -return data_read_offset (dso, offset, data, size); +return data_read_offset (dso, machine, offset, data, size); }/** Signature · 1 line
2.2.2
Namhyung Kim
maintainerRe: [PATCH 29/42] perf tools: Protect dso cache fd with a mutex
Quoted 27 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 29, 2015 at 05:07:10PM +0900, Namhyung Kim escreveu:
Quoted 26 lines
When dso cache is accessed in multi-thread environment, it's possible to close other
dso->data.fdduring operation due to open file limit. Protect the file descriptors using a separate mutex.Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/tests/dso-data.c | 5 ++ tools/perf/util/dso.c | 136 +++++++++++++++++++++++++++++--------------- 2 files changed, 94 insertions(+), 47 deletions(-)tools/perf/tests/dso-data.c +3
-0@@ -111,6 +111,9 @@ int test__dso_data(void)memset (&machine,0 ,sizeof (machine)); dso =dso__new ((const char *)file); +TEST_ASSERT_VAL ("failed to get dso" , dso); + + dso->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;/* Basic 10 bytes tests. */ for (i =0 ; i <ARRAY_SIZE (offsets); i++) {@@ -199,6 +202,8 @@ static int dsos__create(int cnt, int size)
Quoted 7 lines
tools/perf/tests/dso-data.c static int dsos__create(int cnt, int size) · line 206
Quoted 4 lines
tools/perf/tests/dso-data.c +2
-0dsos[i] =dso__new (file);TEST_ASSERT_VAL ("failed to get dso" , dsos[i]); + + dsos[i]->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;Those two are unrelated, please put them in a separate patch, one that I can even cherrypick ahead of the other patches.
It's a consequence of changing dso__data_read_offset() not to call dso__data_fd() due to a performance reason. The binary_type was determined during the dso__data_fd() before, but now it needs to be set explicitly for this test.
In the original code, it was called everytime we access to the dso cache just to check an error, I guess. But it's enough to check the status field.
Quoted 20 lines
Quoted 20 lines
tools/perf/tests/dso-data.c +0
-0}return 0 ;tools/perf/util/dso.c +1
-0@@ -213,6 +213,7 @@ bool dso__needs_decompress(struct dso *dso)*/ static LIST_HEAD (dso__data_open);static long dso__data_open_cnt; +static pthread_mutex_t dso__data_open_lock =PTHREAD_MUTEX_INITIALIZER ;static void dso__list_add (struct dso *dso) {@@ -240,7 +241,7 @@ static int do_open(char *name)if (fd >=0 )return fd;
Quoted 6 lines
tools/perf/util/dso.c static int do_open(char *name) · line 246
Quoted 4 lines
tools/perf/util/dso.c +1
-1-pr_debug ("dso open failed, mmap: %s\n" , +pr_debug ("dso open failed: %s\n" ,strerror_r (errno, sbuf,sizeof (sbuf)));if (!dso__data_open_cnt || errno !=EMFILE )Ditto, another unrelated patch, please separate.
Ah, okay. I kept it since it's just a small change. But I'd like to separate if it helps reviewing.
Quoted 41 lines
Quoted 41 lines
tools/perf/util/dso.c +7
-3break ;@@ -382,7 +383,9 @@ static void check_data_close(void)*/ void dso__data_close (struct dso *dso) { +pthread_mutex_lock (&dso__data_open_lock);close_dso (dso); +pthread_mutex_unlock (&dso__data_open_lock); }/** @@ -405,6 +408,8 @@ int dso__data_fd(struct dso *dso, struct machine *machine)if (dso->data.status ==DSO_DATA_STATUS_ERROR )return -1 ; +pthread_mutex_lock (&dso__data_open_lock); +if (dso->data.fd >=0 ) goto out;@@ -427,6 +432,7 @@ int dso__data_fd(struct dso *dso, struct machine *machine)else dso->data.status =DSO_DATA_STATUS_ERROR ; +pthread_mutex_unlock (&dso__data_open_lock);return dso->data.fd; }@@ -531,52 +537,66 @@ dso_cache__memcpy(struct dso_cache *cache, u64 offset, }static ssize_t -dso_cache__read (struct dso *dso,u64 offset,u8 *data,ssize_t size) +dso_cache__read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {struct dso_cache *cache;struct dso_cache *old; -ssize_t ret; -
Quoted 8 lines
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 541
Quoted 2 lines
tools/perf/util/dso.c +0
-2- do { - u64 cache_offset;While I understand that there was no need for this do { } while (0) construct in the first place, removing it in this patch is not interesting, as it is both unrelated to this patch and makes the it harder to review by just looking at the patch :-\ Please refrain from doing this in this patch.
Understood, sorry for bothering! :)
Quoted 3 lines
A later patch that does _just_ that could be done, if you feel like doing it.
Okay.
Quoted 18 lines
Quoted 18 lines
tools/perf/util/dso.c +7
-6+ssize_t ret = -EINVAL ; +u64 cache_offset; - ret = -ENOMEM ; + cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); +if (!cache) +return -ENOMEM ; - cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); -if (!cache) -break ; + cache_offset = offset &DSO__DATA_CACHE_MASK ; - cache_offset = offset &DSO__DATA_CACHE_MASK ; - ret = -EINVAL ; +pthread_mutex_lock (&dso__data_open_lock);
Quoted 12 lines
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 561
Quoted 8 lines
tools/perf/util/dso.c +6
-2-if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) -break ; +/* +* dso->data.fd might be closed if other thread opened another +* file (dso) due to open file limit (RLIMIT_NOFILE). +*/ +if (dso->data.fd <0 ) { + dso->data.fd =open_dso (dso, machine);Also please consider adding a backpointer to machine in the dso object, since you need to reopen it, so that we don't have to go on passing machine around to
dso_cache__read(), etc.
Yeah, it's a pain to passing a machine pointer.
Quoted 3 lines
This probably needs to be done in the patch that makes
dso->data.fdto be closed due to limit.
I don't know which patch you are refering.. It already closes an fd if it reaches the limit - what this patch does is protecting such concurrent open and close when multi-thread is used.
Thanks,
Namhyung
Quoted 182 lines
Quoted 182 lines
tools/perf/util/dso.c +67
-33+if (dso->data.fd <0 ) { + ret = -errno; + dso->data.status =DSO_DATA_STATUS_ERROR ; + goto err_unlock; + } + } - ret =read (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE ); -if (ret <=0 ) -break ; +if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) + goto err_unlock; - cache->offset = cache_offset; - cache->size = ret; - old =dso_cache__insert (dso, cache); -if (old) { -/* we lose the race */ -free (cache); - cache = old; - } + ret =read (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE ); +if (ret <=0 ) + goto err_unlock; - ret =dso_cache__memcpy (cache, offset, data, size); +pthread_mutex_unlock (&dso__data_open_lock); - }while (0 ); + cache->offset = cache_offset; + cache->size = ret; + old =dso_cache__insert (dso, cache); +if (old) { +/* we lose the race */ +free (cache); + cache = old; + } + ret =dso_cache__memcpy (cache, offset, data, size);if (ret <=0 )free (cache);return ret; + +err_unlock: +pthread_mutex_unlock (&dso__data_open_lock); +return ret; } -static ssize_t dso_cache_read (struct dso *dso,u64 offset, -u8 *data,ssize_t size) +static ssize_t dso_cache_read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {struct dso_cache *cache;@@ -584,7 +604,7 @@ static ssize_t dso_cache_read(struct dso *dso, u64 offset,if (cache)return dso_cache__memcpy (cache, offset, data, size);else -return dso_cache__read (dso, offset, data, size); +return dso_cache__read (dso, machine, offset, data, size); }/* @@ -592,7 +612,8 @@ static ssize_t dso_cache_read(struct dso *dso, u64 offset,* in the rb_tree. Any read to already cached data is served * by cached data. */ -static ssize_t cached_read (struct dso *dso,u64 offset,u8 *data,ssize_t size) +static ssize_t cached_read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {ssize_t r =0 ;u8 *p = data;@@ -600,7 +621,7 @@ static ssize_t cached_read(struct dso *dso, u64 offset, u8 *data, ssize_t size) do { ssize_t ret; - ret = dso_cache_read(dso, offset, p, size); + ret = dso_cache_read(dso, machine, offset, p, size); if (ret < 0) return ret;@@ -620,21 +641,42 @@ static ssize_t cached_read(struct dso *dso, u64 offset, u8 *data, ssize_t size)return r; } -static int data_file_size (struct dso *dso) +static int data_file_size (struct dso *dso,struct machine *machine) { +int ret =0 ;struct stat st;char sbuf[STRERR_BUFSIZE ]; -if (!dso->data.file_size) { -if (fstat (dso->data.fd, &st)) { -pr_err ("dso mmap failed, fstat: %s\n" , -strerror_r (errno, sbuf,sizeof (sbuf))); -return -1 ; +if (dso->data.file_size) +return 0 ; + +pthread_mutex_lock (&dso__data_open_lock); + +/* +* dso->data.fd might be closed if other thread opened another +* file (dso) due to open file limit (RLIMIT_NOFILE). +*/ +if (dso->data.fd <0 ) { + dso->data.fd =open_dso (dso, machine); +if (dso->data.fd <0 ) { + ret = -errno; + dso->data.status =DSO_DATA_STATUS_ERROR ; + goto out; } - dso->data.file_size = st.st_size; } -return 0 ; +if (fstat (dso->data.fd, &st) <0 ) { + ret = -errno; +pr_err ("dso cache fstat failed: %s\n" , +strerror_r (errno, sbuf,sizeof (sbuf))); + dso->data.status =DSO_DATA_STATUS_ERROR ; + goto out; + } + dso->data.file_size = st.st_size; + +out: +pthread_mutex_unlock (&dso__data_open_lock); +return ret; }/** @@ -652,17 +694,17 @@ off_t dso__data_size(struct dso *dso, struct machine *machine)if (fd <0 )return fd; -if (data_file_size (dso)) +if (data_file_size (dso, machine))return -1 ;/* For now just estimate dso data size is close to file size */ return dso->data.file_size; } -static ssize_t data_read_offset (struct dso *dso,u64 offset, -u8 *data,ssize_t size) +static ssize_t data_read_offset (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) { -if (data_file_size (dso)) +if (data_file_size (dso, machine))return -1 ;/* Check the offset sanity. */ @@ -672,7 +714,7 @@ static ssize_t data_read_offset(struct dso *dso, u64 offset,if (offset + size < offset)return -1 ; -return cached_read (dso, offset, data, size); +return cached_read (dso, machine, offset, data, size); }/** @@ -689,10 +731,10 @@ static ssize_t data_read_offset(struct dso *dso, u64 offset,ssize_t dso__data_read_offset (struct dso *dso,struct machine *machine,u64 offset,u8 *data,ssize_t size) { -if (dso__data_fd (dso, machine) <0 ) +if (dso->data.status ==DSO_DATA_STATUS_ERROR )return -1 ; -return data_read_offset (dso, offset, data, size); +return data_read_offset (dso, machine, offset, data, size); }/** Signature · 1 line
2.2.2
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 29/42] perf tools: Protect dso cache fd with a mutex
Em Thu, Jan 29, 2015 at 10:19:38PM +0900, Namhyung Kim escreveu:
Quoted 36 lines · Arnaldo Carvalho de Melo
Quoted 27 lines
Em Thu, Jan 29, 2015 at 05:07:10PM +0900, Namhyung Kim escreveu:
Quoted 26 lines
When dso cache is accessed in multi-thread environment, it's possible to close other
dso->data.fdduring operation due to open file limit. Protect the file descriptors using a separate mutex.Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/tests/dso-data.c | 5 ++ tools/perf/util/dso.c | 136 +++++++++++++++++++++++++++++--------------- 2 files changed, 94 insertions(+), 47 deletions(-)tools/perf/tests/dso-data.c +3
-0@@ -111,6 +111,9 @@ int test__dso_data(void)memset (&machine,0 ,sizeof (machine)); dso =dso__new ((const char *)file); +TEST_ASSERT_VAL ("failed to get dso" , dso); + + dso->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;/* Basic 10 bytes tests. */ for (i =0 ; i <ARRAY_SIZE (offsets); i++) {@@ -199,6 +202,8 @@ static int dsos__create(int cnt, int size)Quoted 7 lines
tools/perf/tests/dso-data.c static int dsos__create(int cnt, int size) · line 206
Quoted 4 lines
tools/perf/tests/dso-data.c +2
-0dsos[i] =dso__new (file);TEST_ASSERT_VAL ("failed to get dso" , dsos[i]); + + dsos[i]->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;Those two are unrelated, please put them in a separate patch, one that I can even cherrypick ahead of the other patches.
Quoted 8 lines
It's a consequence of changing
dso__data_read_offset()not to calldso__data_fd()due to a performance reason. Thebinary_typewas determined during thedso__data_fd()before, but now it needs to be set explicitly for this test.In the original code, it was called everytime we access to the dso cache just to check an error, I guess. But it's enough to check the status field.
Are you saying that this test should not rely on some function that is called somewhere down the functions it uses and should instead do as you do above?
I.e. if that is the case, then this stands out as a separate patch, if not, if this is indeed really related to this patch (at first sight it doesn't look like) then this explanation you give should be in the patch comment log.
Quoted 24 lines
Quoted 17 lines
Quoted 17 lines
tools/perf/tests/dso-data.c +0
-0return 0 ;tools/perf/util/dso.c +1
-0@@ -213,6 +213,7 @@ bool dso__needs_decompress(struct dso *dso)*/ static LIST_HEAD (dso__data_open);static long dso__data_open_cnt; +static pthread_mutex_t dso__data_open_lock =PTHREAD_MUTEX_INITIALIZER ;static void dso__list_add (struct dso *dso) {@@ -240,7 +241,7 @@ static int do_open(char *name)if (fd >=0 )return fd;Quoted 6 lines
tools/perf/util/dso.c static int do_open(char *name) · line 246
Quoted 4 lines
tools/perf/util/dso.c +1
-1-pr_debug ("dso open failed, mmap: %s\n" , +pr_debug ("dso open failed: %s\n" ,strerror_r (errno, sbuf,sizeof (sbuf)));if (!dso__data_open_cnt || errno !=EMFILE )Ditto, another unrelated patch, please separate.
Quoted 2 lines
Ah, okay. I kept it since it's just a small change. But I'd like to separate if it helps reviewing.
Thanks
Quoted 49 lines
Quoted 40 lines
Quoted 40 lines
tools/perf/util/dso.c +7
-3break ;@@ -382,7 +383,9 @@ static void check_data_close(void)*/ void dso__data_close (struct dso *dso) { +pthread_mutex_lock (&dso__data_open_lock);close_dso (dso); +pthread_mutex_unlock (&dso__data_open_lock); }/** @@ -405,6 +408,8 @@ int dso__data_fd(struct dso *dso, struct machine *machine)if (dso->data.status ==DSO_DATA_STATUS_ERROR )return -1 ; +pthread_mutex_lock (&dso__data_open_lock); +if (dso->data.fd >=0 ) goto out;@@ -427,6 +432,7 @@ int dso__data_fd(struct dso *dso, struct machine *machine)else dso->data.status =DSO_DATA_STATUS_ERROR ; +pthread_mutex_unlock (&dso__data_open_lock);return dso->data.fd; }@@ -531,52 +537,66 @@ dso_cache__memcpy(struct dso_cache *cache, u64 offset, }static ssize_t -dso_cache__read (struct dso *dso,u64 offset,u8 *data,ssize_t size) +dso_cache__read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {struct dso_cache *cache;struct dso_cache *old; -ssize_t ret; -Quoted 8 lines
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 541
Quoted 2 lines
tools/perf/util/dso.c +0
-2- do { - u64 cache_offset;While I understand that there was no need for this do { } while (0) construct in the first place, removing it in this patch is not interesting, as it is both unrelated to this patch and makes the it harder to review by just looking at the patch :-\ Please refrain from doing this in this patch.
Quoted 1 line
Understood, sorry for bothering! :)
:-)
Quoted 37 lines
Quoted 2 lines
A later patch that does _just_ that could be done, if you feel like doing it.
Okay.
Quoted 18 lines
Quoted 18 lines
tools/perf/util/dso.c +7
-6+ssize_t ret = -EINVAL ; +u64 cache_offset; - ret = -ENOMEM ; + cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); +if (!cache) +return -ENOMEM ; - cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); -if (!cache) -break ; + cache_offset = offset &DSO__DATA_CACHE_MASK ; - cache_offset = offset &DSO__DATA_CACHE_MASK ; - ret = -EINVAL ; +pthread_mutex_lock (&dso__data_open_lock);Quoted 12 lines
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 561
Quoted 8 lines
tools/perf/util/dso.c +6
-2-if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) -break ; +/* +* dso->data.fd might be closed if other thread opened another +* file (dso) due to open file limit (RLIMIT_NOFILE). +*/ +if (dso->data.fd <0 ) { + dso->data.fd =open_dso (dso, machine);Also please consider adding a backpointer to machine in the dso object, since you need to reopen it, so that we don't have to go on passing machine around to
dso_cache__read(), etc.
Quoted 1 line
Yeah, it's a pain to passing a machine pointer.
Hey, so setting of dso->data.fd is protected in this function and we can be sure that it will not be closed _again_ just before we do that lseek and other operations, I guess so, just checking...
Quoted 7 lines
Quoted 3 lines
This probably needs to be done in the patch that makes
dso->data.fdto be closed due to limit.I don't know which patch you are refering.. It already closes an fd if it reaches the limit - what this patch does is protecting such concurrent open and close when multi-thread is used.
Ok then, i.e.:
A) take the lock, close it if over the limit, drop the lock.
B) take the lock, check if it is closed, open if so, use it, drop the lock.
Is that right?
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 29/42] perf tools: Protect dso cache fd with a mutex
Quoted 46 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 29, 2015 at 10:19:38PM +0900, Namhyung Kim escreveu:
Quoted 36 lines · Arnaldo Carvalho de Melo
Quoted 27 lines
Em Thu, Jan 29, 2015 at 05:07:10PM +0900, Namhyung Kim escreveu:
Quoted 26 lines
When dso cache is accessed in multi-thread environment, it's possible to close other
dso->data.fdduring operation due to open file limit. Protect the file descriptors using a separate mutex.Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/tests/dso-data.c | 5 ++ tools/perf/util/dso.c | 136 +++++++++++++++++++++++++++++--------------- 2 files changed, 94 insertions(+), 47 deletions(-)tools/perf/tests/dso-data.c +3
-0@@ -111,6 +111,9 @@ int test__dso_data(void)memset (&machine,0 ,sizeof (machine)); dso =dso__new ((const char *)file); +TEST_ASSERT_VAL ("failed to get dso" , dso); + + dso->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;/* Basic 10 bytes tests. */ for (i =0 ; i <ARRAY_SIZE (offsets); i++) {@@ -199,6 +202,8 @@ static int dsos__create(int cnt, int size)Quoted 7 lines
tools/perf/tests/dso-data.c static int dsos__create(int cnt, int size) · line 206
Quoted 4 lines
tools/perf/tests/dso-data.c +2
-0dsos[i] =dso__new (file);TEST_ASSERT_VAL ("failed to get dso" , dsos[i]); + + dsos[i]->binary_type =DSO_BINARY_TYPE__SYSTEM_PATH_DSO ;Those two are unrelated, please put them in a separate patch, one that I can even cherrypick ahead of the other patches.
Quoted 8 lines
It's a consequence of changing
dso__data_read_offset()not to calldso__data_fd()due to a performance reason. Thebinary_typewas determined during thedso__data_fd()before, but now it needs to be set explicitly for this test.In the original code, it was called everytime we access to the dso cache just to check an error, I guess. But it's enough to check the status field.
Quoted 8 lines
Are you saying that this test should not rely on some function that is called somewhere down the functions it uses and should instead do as you do above?
I.e. if that is the case, then this stands out as a separate patch, if not, if this is indeed really related to this patch (at first sight it doesn't look like) then this explanation you give should be in the patch comment log.
I think it'd be better adding dso__data_fd() after dso__new() as an extra validation step. With that we don't need to specify binary type manually and have a more consistent usage pattern. I'll do it as a separate patch.
Quoted 120 lines
Quoted 25 lines
Quoted 18 lines
Quoted 18 lines
tools/perf/tests/dso-data.c +0
-0return 0 ;tools/perf/util/dso.c +1
-0@@ -213,6 +213,7 @@ bool dso__needs_decompress(struct dso *dso)*/ static LIST_HEAD (dso__data_open);static long dso__data_open_cnt; +static pthread_mutex_t dso__data_open_lock =PTHREAD_MUTEX_INITIALIZER ;static void dso__list_add (struct dso *dso) {@@ -240,7 +241,7 @@ static int do_open(char *name)if (fd >=0 )return fd;Quoted 6 lines
tools/perf/util/dso.c static int do_open(char *name) · line 246
Quoted 4 lines
tools/perf/util/dso.c +1
-1-pr_debug ("dso open failed, mmap: %s\n" , +pr_debug ("dso open failed: %s\n" ,strerror_r (errno, sbuf,sizeof (sbuf)));if (!dso__data_open_cnt || errno !=EMFILE )Ditto, another unrelated patch, please separate.
Quoted 2 lines
Ah, okay. I kept it since it's just a small change. But I'd like to separate if it helps reviewing.
Thanks
Quoted 49 lines
Quoted 40 lines
Quoted 40 lines
tools/perf/util/dso.c +7
-3break ;@@ -382,7 +383,9 @@ static void check_data_close(void)*/ void dso__data_close (struct dso *dso) { +pthread_mutex_lock (&dso__data_open_lock);close_dso (dso); +pthread_mutex_unlock (&dso__data_open_lock); }/** @@ -405,6 +408,8 @@ int dso__data_fd(struct dso *dso, struct machine *machine)if (dso->data.status ==DSO_DATA_STATUS_ERROR )return -1 ; +pthread_mutex_lock (&dso__data_open_lock); +if (dso->data.fd >=0 ) goto out;@@ -427,6 +432,7 @@ int dso__data_fd(struct dso *dso, struct machine *machine)else dso->data.status =DSO_DATA_STATUS_ERROR ; +pthread_mutex_unlock (&dso__data_open_lock);return dso->data.fd; }@@ -531,52 +537,66 @@ dso_cache__memcpy(struct dso_cache *cache, u64 offset, }static ssize_t -dso_cache__read (struct dso *dso,u64 offset,u8 *data,ssize_t size) +dso_cache__read (struct dso *dso,struct machine *machine, +u64 offset,u8 *data,ssize_t size) {struct dso_cache *cache;struct dso_cache *old; -ssize_t ret; -Quoted 8 lines
tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 541
Quoted 2 lines
tools/perf/util/dso.c +0
-2- do { - u64 cache_offset;While I understand that there was no need for this do { } while (0) construct in the first place, removing it in this patch is not interesting, as it is both unrelated to this patch and makes the it harder to review by just looking at the patch :-\ Please refrain from doing this in this patch.
Quoted 1 line
Understood, sorry for bothering! :)
:-)
Quoted 37 lines
Quoted 2 lines
A later patch that does _just_ that could be done, if you feel like doing it.
Okay.
Quoted 30 lines
Quoted 16 lines
tools/perf/util/dso.c +6
-6+ssize_t ret = -EINVAL ; +u64 cache_offset; - ret = -ENOMEM ; + cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); +if (!cache) +return -ENOMEM ; - cache =zalloc (sizeof (*cache) +DSO__DATA_CACHE_SIZE ); -if (!cache) -break ; + cache_offset = offset &DSO__DATA_CACHE_MASK ; - cache_offset = offset &DSO__DATA_CACHE_MASK ; - ret = -EINVAL ;tools/perf/util/dso.c dso_cache__memcpy(struct dso_cache *cache, u64 offset, · line 561
Quoted 10 lines
tools/perf/util/dso.c +7
-2+pthread_mutex_lock (&dso__data_open_lock); -if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) -break ; +/* +* dso->data.fd might be closed if other thread opened another +* file (dso) due to open file limit (RLIMIT_NOFILE). +*/ +if (dso->data.fd <0 ) { + dso->data.fd =open_dso (dso, machine);Also please consider adding a backpointer to machine in the dso object, since you need to reopen it, so that we don't have to go on passing machine around to
dso_cache__read(), etc.
Quoted 5 lines
Quoted 1 line
Yeah, it's a pain to passing a machine pointer.
Hey, so setting of
dso->data.fdis protected in this function and we can be sure that it will not be closed _again_ just before we do that lseek and other operations, I guess so, just checking...
Right. Accessing to dso->data.fd is safe only if it grabs the dso__data_open_lock.
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
This probably needs to be done in the patch that makes
dso->data.fdto be closed due to limit.I don't know which patch you are refering.. It already closes an fd if it reaches the limit - what this patch does is protecting such concurrent open and close when multi-thread is used.
Quoted 8 lines
Ok then, i.e.:
A) take the lock, close it if over the limit, drop the lock.
B) take the lock, check if it is closed, open if so, use it, drop the lock.
Is that right?
It's like this:
A) take the lock, check if it's still open, use it and drop the lock.
B) take the lock, it's closed, reopen it and check if it reaches the limit. Then close the first (kinda in LRU) dso to make next open() succeeded. Use it and drop the lock.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 30/42 perf session: Pass struct events stats to event processing functions
In reply to Namhyung KimPass stats structure so that it can point separate object when used in multi-thread environment.
--- tools/perf/util/ordered-events.c | 4 +- tools/perf/util/session.c | 81 ++++++++++++++++++++++------------------ tools/perf/util/session.h | 1 + 3 files changed, 49 insertions(+), 37 deletions(-)
tools/perf/util/ordered-events.c +3 -1
@@ -183,7 +183,9 @@ static int __ordered_events__flush(struct perf_session *s,if (ret)pr_err ("Can't parse sample, err = %d\n" , ret);else { - ret =perf_session__deliver_event (s, iter->event, &sample, tool, + ret =perf_session__deliver_event (s, &s->stats, + iter->event, + &sample, tool, iter->file_offset);if (ret)return ret;
tools/perf/util/session.c +45 -36
244 lines not on this page · raw message
tools/perf/util/session.h +1 -0
@@ -59,6 +59,7 @@ int perf_session_queue_event(struct perf_session *s, union perf_event *event,void perf_tool__fill_defaults (struct perf_tool *tool);int perf_session__deliver_event (struct perf_session *session, +struct events_stats *stats,union perf_event *event,struct perf_sample *sample,struct perf_tool *tool,u64 file_offset);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 31/42 perf hists: Pass hists struct to hist_entry_iter functions
In reply to Namhyung KimThis is a preparation for perf report multi-thread support. When multi-thread is enable, each thread will have its own hists during the sample processing.
--- tools/perf/builtin-report.c | 4 ++-- tools/perf/builtin-top.c | 4 ++-- tools/perf/tests/hists_cumulate.c | 4 ++-- tools/perf/tests/hists_filter.c | 3 ++- tools/perf/tests/hists_output.c | 4 ++-- tools/perf/util/hist.c | 26 +++++++++++--------------- tools/perf/util/hist.h | 6 ++++-- 7 files changed, 25 insertions(+), 26 deletions(-)
tools/perf/builtin-report.c +2 -2
@@ -167,8 +167,8 @@ static int process_sample_event(struct perf_tool *tool,if (al.map !=NULL ) al.map->dso->hit =1 ; - ret =hist_entry_iter__add (&iter, &al, evsel, sample, rep->max_stack, - rep); + ret =hist_entry_iter__add (&iter,evsel__hists (evsel), evsel, &al, + sample, rep->max_stack, rep);if (ret <0 )pr_debug ("problem adding hist entry, skipping event\n" );
tools/perf/builtin-top.c +2 -2
@@ -784,8 +784,8 @@ static void perf_event__process_sample(struct perf_tool *tool,pthread_mutex_lock (&hists->lock); - err =hist_entry_iter__add (&iter, &al, evsel, sample, - top->max_stack, top); + err =hist_entry_iter__add (&iter,evsel__hists (evsel), evsel, + &al, sample, top->max_stack, top);if (err <0 )pr_err ("Problem incrementing symbol period, skipping event\n" );
tools/perf/tests/hists_cumulate.c +2 -2
@@ -104,8 +104,8 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) &sample, NULL) < 0) goto out; -if (hist_entry_iter__add (&iter, &al, evsel, &sample, -PERF_MAX_STACK_DEPTH ,NULL ) <0 ) +if (hist_entry_iter__add (&iter,evsel__hists (evsel), evsel, &al, + &sample,PERF_MAX_STACK_DEPTH ,NULL ) <0 ) goto out; fake_samples[i].thread = al.thread;
tools/perf/tests/hists_filter.c +2 -1
@@ -81,7 +81,8 @@ static int add_hist_entries(struct perf_evlist *evlist, &sample, NULL) < 0) goto out; -if (hist_entry_iter__add (&iter, &al, evsel, &sample, +if (hist_entry_iter__add (&iter,evsel__hists (evsel), + evsel, &al, &sample,PERF_MAX_STACK_DEPTH ,NULL ) <0 ) goto out;
tools/perf/tests/hists_output.c +2 -2
@@ -70,8 +70,8 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) &sample, NULL) < 0) goto out; -if (hist_entry_iter__add (&iter, &al, evsel, &sample, -PERF_MAX_STACK_DEPTH ,NULL ) <0 ) +if (hist_entry_iter__add (&iter,evsel__hists (evsel), evsel, &al, + &sample,PERF_MAX_STACK_DEPTH ,NULL ) <0 ) goto out; fake_samples[i].thread = al.thread;
tools/perf/util/hist.c +11 -15
@@ -510,7 +510,7 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *alu64 cost;struct mem_info *mi = iter->priv;struct perf_sample *sample = iter->sample; -struct hists *hists =evsel__hists (iter->evsel); +struct hists *hists = iter->hists;struct hist_entry *he;if (mi ==NULL )
@@ -540,8 +540,7 @@ static int iter_finish_mem_entry(struct hist_entry_iter *iter, struct addr_location *al __maybe_unused) { - struct perf_evsel *evsel = iter->evsel; -struct hists *hists =evsel__hists (evsel); + struct hists *hists = iter->hists;struct hist_entry *he = iter->he;int err = -EINVAL ;
@@ -613,8 +612,7 @@ static int iter_add_next_branch_entry(struct hist_entry_iter *iter, struct addr_location *al) { struct branch_info *bi; -struct perf_evsel *evsel = iter->evsel; -struct hists *hists =evsel__hists (evsel); +struct hists *hists = iter->hists;struct hist_entry *he =NULL ;int i = iter->curr;int err =0 ;
@@ -661,11 +659,10 @@ iter_prepare_normal_entry(struct hist_entry_iter *iter __maybe_unused,static int iter_add_single_normal_entry (struct hist_entry_iter *iter,struct addr_location *al) { -struct perf_evsel *evsel = iter->evsel;struct perf_sample *sample = iter->sample;struct hist_entry *he; - he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , + he =__hists__add_entry (iter->hists, al, iter->parent,NULL ,NULL , sample->period, sample->weight, sample->transaction, sample->time, true);if (he ==NULL )
@@ -680,7 +677,6 @@ iter_finish_normal_entry(struct hist_entry_iter *iter, struct addr_location *al __maybe_unused) {struct hist_entry *he = iter->he; -struct perf_evsel *evsel = iter->evsel;struct perf_sample *sample = iter->sample;if (he ==NULL )
@@ -688,7 +684,7 @@ iter_finish_normal_entry(struct hist_entry_iter *iter, iter->he =NULL ; -hists__inc_nr_samples (evsel__hists (evsel), he->filtered); +hists__inc_nr_samples (iter->hists, he->filtered);return hist_entry__append_callchain (he, sample); }
@@ -720,8 +716,7 @@ static int iter_add_single_cumulative_entry(struct hist_entry_iter *iter, struct addr_location *al) { - struct perf_evsel *evsel = iter->evsel; -struct hists *hists =evsel__hists (evsel); + struct hists *hists = iter->hists;struct perf_sample *sample = iter->sample;struct hist_entry **he_cache = iter->priv;struct hist_entry *he;
@@ -766,7 +761,6 @@ static int iter_add_next_cumulative_entry(struct hist_entry_iter *iter, struct addr_location *al) { - struct perf_evsel *evsel = iter->evsel; struct perf_sample *sample = iter->sample;struct hist_entry **he_cache = iter->priv;struct hist_entry *he;
@@ -800,7 +794,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter, } } - he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , + he =__hists__add_entry (iter->hists, al, iter->parent,NULL ,NULL , sample->period, sample->weight, sample->transaction, sample->time, false);if (he ==NULL )
@@ -856,8 +850,9 @@ const struct hist_iter_ops hist_iter_cumulative = { .finish_entry = iter_finish_cumulative_entry, }; -int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al, -struct perf_evsel *evsel,struct perf_sample *sample, +int hist_entry_iter__add (struct hist_entry_iter *iter,struct hists *hists, +struct perf_evsel *evsel,struct addr_location *al, +struct perf_sample *sample,int max_stack_depth,void *arg) {int err, err2;
@@ -867,6 +862,7 @@ int hist_entry_iter__add(struct hist_entry_iter *iter, struct addr_location *al,if (err)return err; + iter->hists = hists; iter->evsel = evsel; iter->sample = sample;
tools/perf/util/hist.h +4 -2
@@ -86,6 +86,7 @@ struct hist_entry_iter {bool hide_unresolved; +struct hists *hists;struct perf_evsel *evsel;struct perf_sample *sample;struct hist_entry *he;
@@ -110,8 +111,9 @@ struct hist_entry *__hists__add_entry(struct hists *hists, struct mem_info *mi, u64 period, u64 weight, u64 transaction, u64 timestamp, bool sample_self); -int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al, -struct perf_evsel *evsel,struct perf_sample *sample, +int hist_entry_iter__add (struct hist_entry_iter *iter,struct hists *hists, +struct perf_evsel *evsel,struct addr_location *al, +struct perf_sample *sample,int max_stack_depth,void *arg);int64_t hist_entry__cmp (struct hist_entry *left,struct hist_entry *right);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 32/42 perf tools: Move BUILD_ID_SIZE definition to perf.h
In reply to Namhyung KimThe util/event.h includes util/build-id.h only for BUILD_ID_SIZE. This is a problem when I include util/event.h from util/tool.h which is also included by util/build-id.h since it now makes a circular dependency resulting in incomplete type error.
--- tools/perf/perf.h | 1 + tools/perf/util/build-id.h | 2 -- tools/perf/util/dso.h | 1 + tools/perf/util/event.h | 1 - 4 files changed, 2 insertions(+), 3 deletions(-)
tools/perf/perf.h +1 -0
@@ -30,6 +30,7 @@ static inline unsigned long long rdclock(void) }#define MAX_NR_CPUS 256 +#define BUILD_ID_SIZE 20extern const char *input_name;extern bool perf_host, perf_guest;
tools/perf/util/build-id.h +0 -2
@@ -1,8 +1,6 @@#ifndef PERF_BUILD_ID_H_ #define PERF_BUILD_ID_H_ 1 -#define BUILD_ID_SIZE 20 -#include "tool.h" #include <linux/types.h>
tools/perf/util/dso.h +1 -0
@@ -7,6 +7,7 @@#include <linux/types.h> #include <linux/bitops.h> #include "map.h" +#include "perf.h" #include "build-id.h" enum dso_binary_type {
tools/perf/util/event.h +0 -1
@@ -6,7 +6,6 @@#include "../perf.h" #include "map.h" -#include "build-id.h" #include "perf_regs.h" struct mmap_event {
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 33/42 perf report: Parallelize perf report using multi-thread
In reply to Namhyung KimIntroduce perf_session__process_events_mt() to enable multi-thread sample processing. It allocates a struct perf_tool_mt and fills needed info in it.
The session and hists event stats are counted for each thread and summed after finishing the processing. Similarly hist entries are added to per-thread hists first and then move to the original hists using hists__mt_resort(). This function reuses hists__collapse_ resort() code so makes sort__need_collapse force to true and skips the collapsing function.
Note that most of preprocessing stage is already done by processing meta events in dummy tracking evsel first. We can find corresponding thread and map based on the sample time and symbol loading and dso cache access is protected by pthread mutex.
--- tools/perf/util/hist.c | 75 +++++++++++++++++++----- tools/perf/util/hist.h | 3 + tools/perf/util/session.c | 141 ++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/session.h | 2 + tools/perf/util/tool.h | 12 ++++ 5 files changed, 220 insertions(+), 13 deletions(-)
tools/perf/util/hist.c +62 -13
@@ -950,7 +950,7 @@ void hist_entry__delete(struct hist_entry *he)* collapse the histogram */ -static bool hists__collapse_insert_entry (struct hists *hists__maybe_unused , +static bool hists__collapse_insert_entry (struct hists *hists,struct rb_root *root,struct hist_entry *he) {
@@ -987,6 +987,13 @@ static bool hists__collapse_insert_entry(struct hists *hists __maybe_unused, } hists->nr_entries++; +/* +* For multi-threaded report, he->hists points to a dummy +* hists in the struct perf_tool_mt. Please see +* perf_session__process_events_mt(). +*/ + he->hists = hists; +rb_link_node (&he->rb_node_in, parent, p);rb_insert_color (&he->rb_node_in, root);return true;
@@ -1014,19 +1021,12 @@ static void hists__apply_filters(struct hists *hists, struct hist_entry *he)hists__filter_entry_by_symbol (hists, he); } -void hists__collapse_resort (struct hists *hists,struct ui_progress *prog) +static void __hists__collapse_resort (struct hists *hists,struct rb_root *root, +struct ui_progress *prog) { -struct rb_root *root;struct rb_node *next;struct hist_entry *n; -if (!sort__need_collapse) -return ; - - hists->nr_entries =0 ; - - root =hists__get_rotate_entries_in (hists); - next =rb_first (root);while (next) {
@@ -1049,6 +1049,27 @@ void hists__collapse_resort(struct hists *hists, struct ui_progress *prog) } } +void hists__collapse_resort (struct hists *hists,struct ui_progress *prog) +{ +struct rb_root *root; + +if (!sort__need_collapse) +return ; + + hists->nr_entries =0 ; + + root =hists__get_rotate_entries_in (hists); +__hists__collapse_resort (hists, root, prog); +} + +void hists__mt_resort (struct hists *dst,struct hists *src) +{ +struct rb_root *root = src->entries_in; + + sort__need_collapse =1 ; +__hists__collapse_resort (dst, root,NULL ); +} +static int hist_entry__sort (struct hist_entry *a,struct hist_entry *b) {struct perf_hpp_fmt *fmt;
@@ -1277,6 +1298,29 @@ void events_stats__inc(struct events_stats *stats, u32 type) ++stats->nr_events[type]; } +void events_stats__add (struct events_stats *dst,struct events_stats *src) +{ +int i; + +#define ADD (_field) dst->_field += src->_field + +ADD (total_period); +ADD (total_non_filtered_period); +ADD (total_lost); +ADD (total_invalid_chains); +ADD (nr_non_filtered_samples); +ADD (nr_lost_warned); +ADD (nr_unknown_events); +ADD (nr_invalid_chains); +ADD (nr_unknown_id); +ADD (nr_unprocessable_samples); + +for (i =0 ; i <PERF_RECORD_HEADER_MAX ; i++) +ADD (nr_events[i]); + +#undef ADD +} +void hists__inc_nr_events (struct hists *hists,u32 type) {events_stats__inc (&hists->stats, type);
@@ -1453,16 +1497,21 @@ int perf_hist_config(const char *var, const char *value)return 0 ; } -static int hists_evsel__init (struct perf_evsel *evsel) +void __hists__init (struct hists *hists) { -struct hists *hists =evsel__hists (evsel); -memset (hists,0 ,sizeof (*hists)); hists->entries_in_array[0 ] = hists->entries_in_array[1 ] =RB_ROOT ; hists->entries_in = &hists->entries_in_array[0 ]; hists->entries_collapsed =RB_ROOT ; hists->entries =RB_ROOT ;pthread_mutex_init (&hists->lock,NULL ); +} + +static int hists_evsel__init (struct perf_evsel *evsel) +{ +struct hists *hists =evsel__hists (evsel); + +__hists__init (hists);return 0 ; }
tools/perf/util/hist.h +3 -0
@@ -124,6 +124,7 @@ int hist_entry__sort_snprintf(struct hist_entry *he, char *bf, size_t size,void hist_entry__delete (struct hist_entry *he);void hists__output_resort (struct hists *hists,struct ui_progress *prog); +void hists__mt_resort (struct hists *dst,struct hists *src);void hists__collapse_resort (struct hists *hists,struct ui_progress *prog);void hists__decay_entries (struct hists *hists,bool zap_user,bool zap_kernel);
@@ -136,6 +137,7 @@ void hists__inc_stats(struct hists *hists, struct hist_entry *h);void hists__inc_nr_events (struct hists *hists,u32 type);void hists__inc_nr_samples (struct hists *hists,bool filtered);void events_stats__inc (struct events_stats *stats,u32 type); +void events_stats__add (struct events_stats *dst,struct events_stats *src);size_t events_stats__fprintf (struct events_stats *stats,FILE *fp); size_t hists__fprintf(struct hists *hists, bool show_header, int max_rows,
@@ -179,6 +181,7 @@ static inline struct hists *evsel__hists(struct perf_evsel *evsel) }int hists__init (void ); +void __hists__init (struct hists *hists);struct perf_hpp {char *buf;
tools/perf/util/session.c +141 -0
147 lines not on this page · raw message
tools/perf/util/session.h +2 -0
@@ -51,6 +51,8 @@ int perf_session__peek_event(struct perf_session *session, off_t file_offset,int perf_session__process_events (struct perf_session *session,struct perf_tool *tool); +int perf_session__process_events_mt (struct perf_session *session, +struct perf_tool *tool,void *arg); int perf_session_queue_event(struct perf_session *s, union perf_event *event, struct perf_tool *tool, struct perf_sample *sample,
tools/perf/util/tool.h +12 -0
@@ -2,6 +2,7 @@#define __PERF_TOOL_H#include <stdbool.h> +#include "util/event.h" struct perf_session ;union perf_event ;
@@ -10,6 +11,7 @@ struct perf_evsel;struct perf_sample ;struct perf_tool ;struct machine ; +struct hists ; typedef int (*event_sample)(struct perf_tool *tool, union perf_event *event, struct perf_sample *sample,
@@ -45,4 +47,14 @@ struct perf_tool { bool ordering_requires_timestamps; }; +struct perf_tool_mt { +struct perf_tool tool; +struct events_stats stats; +struct hists *hists; +struct perf_session *session; +int idx; + +void *priv; +}; +#endif /* __PERF_TOOL_H */
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 34/42 perf tools: Add missing_threads rb tree
In reply to Namhyung KimSometimes it's possible to miss certain meta events like fork/exit and in this case it can fail to find such thread in the machine's rbtree. But adding a thread to the tree is dangerous since it's now executed in multi-thread environment otherwise it'll add an overhead in order to grab a lock for every search. So adds a separate missing_threads tree and protect it with a mutex. It's expected to be accessed only if a thread is not found in a normal tree.
--- tools/perf/tests/thread-lookup-time.c | 8 ++- tools/perf/util/build-id.c | 9 ++- tools/perf/util/machine.c | 129 +++++++++++++++++++++------------- tools/perf/util/machine.h | 2 + tools/perf/util/session.c | 8 +-- tools/perf/util/thread.h | 1 + 6 files changed, 101 insertions(+), 56 deletions(-)
tools/perf/tests/thread-lookup-time.c +7 -1
@@ -7,7 +7,9 @@static int thread__print_cb (struct thread *th,void *arg__maybe_unused ) {printf ("thread: %d, start time: %" PRIu64" %s\n" , - th->tid, th->start_time, th->dead ?"(dead)" :"" ); + th->tid, th->start_time, + th->dead ?"(dead)" : th->exited ?"(exited)" : + th->missing ?"(missing)" :"" );return 0 ; }
@@ -105,6 +107,8 @@ static int lookup_with_timestamp(struct machine *machine) machine__findnew_thread_time(machine, 0, 0, 70000) == t3);machine__delete_threads (machine); +machine__delete_dead_threads (machine); +machine__delete_missing_threads (machine);return 0 ; }
@@ -146,6 +150,8 @@ static int lookup_without_timestamp(struct machine *machine) machine__findnew_thread_time(machine, 0, 0, -1ULL) == t3);machine__delete_threads (machine); +machine__delete_dead_threads (machine); +machine__delete_missing_threads (machine);return 0 ; }
tools/perf/util/build-id.c +8 -1
@@ -60,7 +60,14 @@ static int perf_event__exit_del_thread(struct perf_tool *tool __maybe_unused, event->fork.ppid, event->fork.ptid);if (thread) { -rb_erase (&thread->rb_node, &machine->threads); +if (thread->dead) +rb_erase (&thread->rb_node, &machine->dead_threads); +else if (thread->missing) +rb_erase (&thread->rb_node, &machine->missing_threads); +else +rb_erase (&thread->rb_node, &machine->threads); + +list_del (&thread->tid_node); machine->last_match =NULL ;thread__delete (thread); }
tools/perf/util/machine.c +81 -48
185 lines not on this page · raw message
tools/perf/util/machine.h +2 -0
@@ -31,6 +31,7 @@ struct machine {char *root_dir;struct rb_root threads;struct rb_root dead_threads; +struct rb_root missing_threads;struct thread *last_match;struct vdso_info *vdso_info;struct dsos user_dsos;
@@ -116,6 +117,7 @@ void machines__set_comm_exec(struct machines *machines, bool comm_exec);struct machine *machine__new_host (void );int machine__init (struct machine *machine,const char *root_dir,pid_t pid);void machine__exit (struct machine *machine); +void machine__delete_missing_threads (struct machine *machine);void machine__delete_dead_threads (struct machine *machine);void machine__delete_threads (struct machine *machine);void machine__delete (struct machine *machine);
tools/perf/util/session.c +2 -6
@@ -138,14 +138,11 @@ struct perf_session *perf_session__new(struct perf_data_file *file,return NULL ; } -static void perf_session__delete_dead_threads (struct perf_session *session) -{ -machine__delete_dead_threads (&session->machines.host); -} -static void perf_session__delete_threads (struct perf_session *session) {machine__delete_threads (&session->machines.host); +machine__delete_dead_threads (&session->machines.host); +machine__delete_missing_threads (&session->machines.host); }static void perf_session_env__delete (struct perf_session_env *env)
@@ -167,7 +164,6 @@ static void perf_session_env__delete(struct perf_session_env *env)void perf_session__delete (struct perf_session *session) {perf_session__destroy_kernel_maps (session); -perf_session__delete_dead_threads (session);perf_session__delete_threads (session);perf_session_env__delete (&session->header.env);machines__exit (&session->machines);
tools/perf/util/thread.h +1 -0
@@ -23,6 +23,7 @@ struct thread {bool comm_set;bool exited;/* if set thread has exited */ bool dead;/* thread is in dead_threads list */ +bool missing;/* thread is in missing_threads list */ struct list_head comm_list;int comm_len;u64 db_id;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 35/42 perf record: Synthesize COMM event for a command line workload
In reply to Namhyung KimWhen perf creates a new child to profile, the events are enabled on exec(). And in this case, it doesn't synthesize any event for the child since they'll be generated during exec(). But there's an window between the enabling and the event generation.
It used to be overcome since samples are only in kernel (so we always have the map) and the comm is overridden by a later COMM event. However it won't work anymore since those samples will go to a missing thread now but the COMM event will create a (current) thread. This leads to those early samples (like native_write_msr_safe) not having a comm but pid (like ':15328').
So it needs to synthesize COMM event for the child explicitly before enabling so that it can have a correct comm. But at this time, the comm will be "perf" since it's not exec-ed yet.
--- tools/perf/builtin-record.c | 18 +++++++++++++++++- tools/perf/util/event.c | 8 ++++---- tools/perf/util/event.h | 5 +++++ 3 files changed, 26 insertions(+), 5 deletions(-)
tools/perf/builtin-record.c +17 -1
@@ -607,8 +607,24 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)/* * Let the child rip */ -if (forks) +if (forks) { +union perf_event *comm_event; + + comm_event =malloc (sizeof (*comm_event) + machine->id_hdr_size); +if (comm_event ==NULL ) + goto out_child; + + err =perf_event__synthesize_comm (tool, comm_event, + rec->evlist->threads->map[0 ], + process_synthesized_event, + machine); +free (comm_event); + +if (err <0 ) + goto out_child; +perf_evlist__start_workload (rec->evlist); + }if (opts->initial_delay) {usleep (opts->initial_delay *1000 );
tools/perf/util/event.c +4 -4
@@ -96,10 +96,10 @@ static pid_t perf_event__get_comm_tgid(pid_t pid, char *comm, size_t len)return tgid; } -static pid_t perf_event__synthesize_comm (struct perf_tool *tool, -union perf_event *event,pid_t pid, -perf_event__handler_t process, -struct machine *machine) +pid_t perf_event__synthesize_comm (struct perf_tool *tool, +union perf_event *event,pid_t pid, +perf_event__handler_t process, +struct machine *machine) {size_t size;pid_t tgid;
tools/perf/util/event.h +5 -0
@@ -386,6 +386,11 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,struct machine *machine, bool mmap_data); +pid_t perf_event__synthesize_comm (struct perf_tool *tool, +union perf_event *event,pid_t pid, +perf_event__handler_t process, +struct machine *machine); +size_t perf_event__fprintf_comm (union perf_event *event,FILE *fp);size_t perf_event__fprintf_mmap (union perf_event *event,FILE *fp);size_t perf_event__fprintf_mmap2 (union perf_event *event,FILE *fp);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 36/42 perf tools: Fix progress ui to support multi thread
In reply to Namhyung KimSplit ui_progress struct into global and local one. Each thread updates local struct without lock and only updates global one if meaningful progress is done (with lock).
To do that, pass struct ui_progress to __perf_session__process_event() and set it for the total size of multi-file storage.
--- tools/perf/util/hist.c | 5 ++-- tools/perf/util/hist.h | 3 ++- tools/perf/util/session.c | 63 +++++++++++++++++++++++++++++++++++++---------- tools/perf/util/tool.h | 3 +++ 4 files changed, 58 insertions(+), 16 deletions(-)
tools/perf/util/hist.c +3 -2
@@ -1062,12 +1062,13 @@ void hists__collapse_resort(struct hists *hists, struct ui_progress *prog)__hists__collapse_resort (hists, root, prog); } -void hists__mt_resort (struct hists *dst,struct hists *src) +void hists__mt_resort (struct hists *dst,struct hists *src, +struct ui_progress *prog) {struct rb_root *root = src->entries_in; sort__need_collapse =1 ; -__hists__collapse_resort (dst, root,NULL ); +__hists__collapse_resort (dst, root, prog); }static int hist_entry__sort (struct hist_entry *a,struct hist_entry *b)
tools/perf/util/hist.h +2 -1
@@ -124,7 +124,8 @@ int hist_entry__sort_snprintf(struct hist_entry *he, char *bf, size_t size,void hist_entry__delete (struct hist_entry *he);void hists__output_resort (struct hists *hists,struct ui_progress *prog); -void hists__mt_resort (struct hists *dst,struct hists *src); +void hists__mt_resort (struct hists *dst,struct hists *src, +struct ui_progress *prog);void hists__collapse_resort (struct hists *hists,struct ui_progress *prog);void hists__decay_entries (struct hists *hists,bool zap_user,bool zap_kernel);
tools/perf/util/session.c +50 -13
171 lines not on this page · raw message
tools/perf/util/tool.h +3 -0
@@ -3,6 +3,7 @@#include <stdbool.h> #include "util/event.h" +#include "ui/progress.h" struct perf_session ;union perf_event ;
@@ -52,6 +53,8 @@ struct perf_tool_mt {struct events_stats stats;struct hists *hists;struct perf_session *session; +struct ui_progress prog; +struct ui_progress *global_prog;int idx;void *priv;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 37/42 perf report: Add --multi-thread option and config item
In reply to Namhyung KimThe --multi-thread option is to enable parallel processing so user can force serial processing even for indexed data file. It default to false for now but users also can changes this by setting "report.multi_thread" config option in ~/.perfconfig file.
--- tools/perf/Documentation/perf-report.txt | 3 ++ tools/perf/builtin-report.c | 66 +++++++++++++++++++++++++++----- tools/perf/util/session.c | 1 + 3 files changed, 61 insertions(+), 9 deletions(-)
tools/perf/Documentation/perf-report.txt +3 -0
@@ -318,6 +318,9 @@ OPTIONS --header-only:: Show only perf.data header (forces --stdio). +--multi-thread:: + Speed up report by parallelizing sample processing using multi-thread. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-annotate[1]
tools/perf/builtin-report.c +57 -9
121 lines not on this page · raw message
tools/perf/util/session.c +1 -0
@@ -1586,6 +1586,7 @@ int perf_session__process_events_mt(struct perf_session *session, goto out; } +bool perf_session__has_traces (struct perf_session *session,const char *msg) {struct perf_evsel *evsel;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 38/42 perf session: Handle index files generally
In reply to Namhyung KimThe current code assumes that the number of index item and cpu are matched so it creates that number of threads. But it's not the case of non-system-wide session or data came from different machine.
Just creates threads at most number of online cpus and process data.
--- tools/perf/util/session.c | 68 ++++++++++++++++++++++++++++++++++------------- tools/perf/util/tool.h | 1 - 2 files changed, 50 insertions(+), 19 deletions(-)
tools/perf/util/session.c +50 -18
@@ -1431,26 +1431,51 @@ static struct ui_progress_ops mt_progress__ops = { .update = mt_progress__update, }; +static int perf_session__get_index (struct perf_session *session) +{ +int ret; +static unsigned index; +static pthread_mutex_t idx_lock =PTHREAD_MUTEX_INITIALIZER ; + +pthread_mutex_lock (&idx_lock); +if (index < session->header.nr_index) + ret = index++; +else + ret =-1 ; +pthread_mutex_unlock (&idx_lock); + +return ret; +} +static void *processing_thread_idx (void *arg) {struct perf_tool_mt *mt_tool = arg;struct perf_session *session = mt_tool->session;int fd =perf_data_file__fd (session->file); -u64 offset = session->header.index[mt_tool->idx].offset; -u64 size = session->header.index[mt_tool->idx].size;u64 file_size =perf_data_file__size (session->file); +int idx; -ui_progress__init (&mt_tool->prog, size,"" ); +while ((idx =perf_session__get_index (session)) >=0 ) { +u64 offset = session->header.index[idx].offset; +u64 size = session->header.index[idx].size; +struct perf_tool_mt *mtt = &mt_tool[idx]; -pr_debug ("processing samples using thread [%d]\n" , mt_tool->idx); -if (__perf_session__process_events (session, &mt_tool->stats, - fd, offset, size, file_size, - &mt_tool->tool, &mt_tool->prog) <0 ) { -pr_err ("processing samples failed (thread [%d)\n" , mt_tool->idx); -return NULL ; +if (size ==0 ) +continue ; + +pr_debug ("processing samples [index %d]\n" , idx); + +ui_progress__init (&mtt->prog, size,"" ); + +if (__perf_session__process_events (session, &mtt->stats, + fd, offset, size, file_size, + &mtt->tool, &mtt->prog) <0 ) { +pr_err ("processing samples failed [index %d]\n" , idx); +return NULL ; + } +pr_debug ("processing samples done [index %d]\n" , idx); } -pr_debug ("processing samples done for thread [%d]\n" , mt_tool->idx);return arg; }
@@ -1468,6 +1493,7 @@ int perf_session__process_events_mt(struct perf_session *session,int err, i, k;int nr_index = session->header.nr_index;u64 size =perf_data_file__size (file); +int nr_thread =sysconf (_SC_NPROCESSORS_ONLN);if (perf_session__register_idle_thread (session) ==NULL )return -ENOMEM ;
@@ -1491,10 +1517,6 @@ int perf_session__process_events_mt(struct perf_session *session, ui_progress__ops = &mt_progress__ops; ui_progress__ops->finish = orig_progress__ops->finish; - th_id =calloc (nr_index,sizeof (*th_id)); -if (th_id ==NULL ) - goto out; - mt_tools =calloc (nr_index,sizeof (*mt_tools));if (mt_tools ==NULL ) goto out;
@@ -1513,20 +1535,30 @@ int perf_session__process_events_mt(struct perf_session *session, mt->session = session; mt->tool.ordered_events = false; - mt->idx = i; mt->priv = arg; mt->global_prog = &prog; - -pthread_create (&th_id[i],NULL , processing_thread_idx, mt); } -for (i =0 ; i < nr_index; i++) { +if (nr_thread > nr_index) + nr_thread = nr_index; + + th_id =calloc (nr_thread,sizeof (*th_id)); +if (th_id ==NULL ) + goto out; + +for (i =0 ; i < nr_thread; i++) +pthread_create (&th_id[i],NULL , processing_thread_idx, mt_tools); + +for (i =0 ; i < nr_thread; i++) {pthread_join (th_id[i], (void **)&mt);if (mt ==NULL ) { err = -EINVAL ;continue ; } + } +for (i =0 ; i < nr_index; i++) { + mt = &mt_tools[i];events_stats__add (&session->stats, &mt->stats);evlist__for_each (evlist, evsel) {
tools/perf/util/tool.h +0 -1
@@ -55,7 +55,6 @@ struct perf_tool_mt {struct perf_session *session;struct ui_progress prog;struct ui_progress *global_prog; -int idx;void *priv; };
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 39/42 perf tools: Convert lseek + read to pread
In reply to Namhyung KimWhen dso_cache__read() is called, it reads data from the given offset using lseek + normal read syscall. It can be combined to a single pread syscall.
--- tools/perf/util/dso.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
tools/perf/util/dso.c +1 -4
@@ -566,10 +566,7 @@ dso_cache__read(struct dso *dso, struct machine *machine, } } -if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) - goto err_unlock; - - ret =read (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE ); + ret =pread (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE , cache_offset);if (ret <=0 ) goto err_unlock;
Signature · 1 line
2.2.2
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf symbols: Convert lseek + read to pread
perf symbols: Convert lseek + read to pread
Commit-ID: c52686f9f888d23ca72f1309e86af8e91d075697 Gitweb: http://git.kernel.org/tip/c52686f9f888d23ca72f1309e86af8e91d075697 Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:02:01 -0300 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 17:02:01 -0300
perf symbols: Convert lseek + read to pread
When dso_cache__read() is called, it reads data from the given offset using lseek + normal read syscall. It can be combined to a single pread syscall.
--- tools/perf/util/dso.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
tools/perf/util/dso.c +1 -5
@@ -532,12 +532,8 @@ dso_cache__read(struct dso *dso, u64 offset, u8 *data, ssize_t size)break ; cache_offset = offset &DSO__DATA_CACHE_MASK ; - ret = -EINVAL ; -if (-1 ==lseek (dso->data.fd, cache_offset,SEEK_SET )) -break ; - - ret =read (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE ); + ret =pread (dso->data.fd, cache->data,DSO__DATA_CACHE_SIZE , cache_offset);if (ret <=0 )break ;
Namhyung Kim
maintainerPATCH 40/42 perf callchain: Save eh/debug frame offset for dwarf unwind
In reply to Namhyung KimWhen libunwind tries to resolve callchains it needs to know the offset of .eh_frame_hdr or .debug_frame to access the dso. Since it calls dso__data_fd(), it'll try to grab dso->lock everytime for same information. So save it to dso_data struct and reuse it.
Note that there's a window between dso__data_fd() and actual use of the fd. The fd could be closed by other threads to deal with the open file limit in dso cache code. But I think it's ok since in that case elf_section_offset() will return 0 so it'll be tried in next acess.
--- tools/perf/util/dso.h | 1 + tools/perf/util/unwind-libunwind.c | 31 ++++++++++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-)
tools/perf/util/dso.h +1 -0
@@ -141,6 +141,7 @@ struct dso {u32 status_seen;size_t file_size;struct list_head open_entry; +u64 frame_offset; } data;union {/* Tool specific area */
tools/perf/util/unwind-libunwind.c +20 -11
@@ -266,14 +266,17 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine, u64 *fde_count) {int ret = -EINVAL , fd; -u64 offset; +u64 offset = dso->data.frame_offset; - fd =dso__data_fd (dso, machine); -if (fd <0 ) -return -EINVAL ; +if (offset ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .eh_frame section for unwinding info */ - offset =elf_section_offset (fd,".eh_frame_hdr" ); +/* Check the .eh_frame section for unwinding info */ + offset =elf_section_offset (fd,".eh_frame_hdr" ); + dso->data.frame_offset = offset; + }if (offset) ret = unwind_spec_ehframe(dso, machine, offset,
@@ -287,14 +290,20 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine,static int read_unwind_spec_debug_frame (struct dso *dso,struct machine *machine,u64 *offset) { -int fd =dso__data_fd (dso, machine); +int fd; +u64 ofs = dso->data.frame_offset; -if (fd <0 ) -return -EINVAL ; +if (ofs ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .debug_frame section for unwinding info */ - *offset =elf_section_offset (fd,".debug_frame" ); +/* Check the .debug_frame section for unwinding info */ + ofs =elf_section_offset (fd,".debug_frame" ); + dso->data.frame_offset = ofs; + } + *offset = ofs;if (*offset)return 0 ;
Signature · 1 line
2.2.2
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 40/42] perf callchain: Save eh/debug frame offset for dwarf unwind
Em Thu, Jan 29, 2015 at 05:07:21PM +0900, Namhyung Kim escreveu:
Quoted 9 lines
When libunwind tries to resolve callchains it needs to know the offset of .eh_frame_hdr or .debug_frame to access the dso. Since it calls
dso__data_fd(), it'll try to grabdso->lockeverytime for same information. So save it todso_datastruct and reuse it.Note that there's a window between
dso__data_fd()and actual use of the fd. The fd could be closed by other threads to deal with the open file limit in dso cache code. But I think it's ok since in that caseelf_section_offset()will return 0 so it'll be tried in next acess.
I know that you did this in the context of your multi threading patchkit, but this seems useful even without that patckhit, i.e. this can be cherry picked on the grounds that it speeds up things by caching something that doesn't change, right?
I.e. I'll probably just rewrite the comment and apply it before considering the other patches, so that other people can comment on the other patches, etc.
- Arnaldo
Quoted 74 lines
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/dso.h | 1 + tools/perf/util/unwind-libunwind.c | 31 ++++++++++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-)tools/perf/util/dso.h +1
-0@@ -141,6 +141,7 @@ struct dso {u32 status_seen;size_t file_size;struct list_head open_entry; +u64 frame_offset; } data;union {/* Tool specific area */ tools/perf/util/unwind-libunwind.c +20
-11@@ -266,14 +266,17 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine, u64 *fde_count) {int ret = -EINVAL , fd; -u64 offset; +u64 offset = dso->data.frame_offset; - fd =dso__data_fd (dso, machine); -if (fd <0 ) -return -EINVAL ; +if (offset ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .eh_frame section for unwinding info */ - offset =elf_section_offset (fd,".eh_frame_hdr" ); +/* Check the .eh_frame section for unwinding info */ + offset =elf_section_offset (fd,".eh_frame_hdr" ); + dso->data.frame_offset = offset; + }if (offset) ret = unwind_spec_ehframe(dso, machine, offset,@@ -287,14 +290,20 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine,static int read_unwind_spec_debug_frame (struct dso *dso,struct machine *machine,u64 *offset) { -int fd =dso__data_fd (dso, machine); +int fd; +u64 ofs = dso->data.frame_offset; -if (fd <0 ) -return -EINVAL ; +if (ofs ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .debug_frame section for unwinding info */ - *offset =elf_section_offset (fd,".debug_frame" ); +/* Check the .debug_frame section for unwinding info */ + ofs =elf_section_offset (fd,".debug_frame" ); + dso->data.frame_offset = ofs; + } + *offset = ofs;if (*offset)return 0 ;Signature · 1 line
2.2.2
Namhyung Kim
maintainerRe: [PATCH 40/42] perf callchain: Save eh/debug frame offset for dwarf unwind
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 29, 2015 at 05:07:21PM +0900, Namhyung Kim escreveu:
Quoted 9 lines
When libunwind tries to resolve callchains it needs to know the offset of .eh_frame_hdr or .debug_frame to access the dso. Since it calls
dso__data_fd(), it'll try to grabdso->lockeverytime for same information. So save it todso_datastruct and reuse it.Note that there's a window between
dso__data_fd()and actual use of the fd. The fd could be closed by other threads to deal with the open file limit in dso cache code. But I think it's ok since in that caseelf_section_offset()will return 0 so it'll be tried in next acess.
Quoted 4 lines
I know that you did this in the context of your multi threading patchkit, but this seems useful even without that patckhit, i.e. this can be cherry picked on the grounds that it speeds up things by caching something that doesn't change, right?
Right.
Quoted 4 lines
I.e. I'll probably just rewrite the comment and apply it before considering the other patches, so that other people can comment on the other patches, etc.
Thanks for doing that!
Namhyung
Quoted 77 lines
- Arnaldo
Quoted 74 lines
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/dso.h | 1 + tools/perf/util/unwind-libunwind.c | 31 ++++++++++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-)tools/perf/util/dso.h +1
-0@@ -141,6 +141,7 @@ struct dso {u32 status_seen;size_t file_size;struct list_head open_entry; +u64 frame_offset; } data;union {/* Tool specific area */ tools/perf/util/unwind-libunwind.c +20
-11@@ -266,14 +266,17 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine, u64 *fde_count) {int ret = -EINVAL , fd; -u64 offset; +u64 offset = dso->data.frame_offset; - fd =dso__data_fd (dso, machine); -if (fd <0 ) -return -EINVAL ; +if (offset ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .eh_frame section for unwinding info */ - offset =elf_section_offset (fd,".eh_frame_hdr" ); +/* Check the .eh_frame section for unwinding info */ + offset =elf_section_offset (fd,".eh_frame_hdr" ); + dso->data.frame_offset = offset; + }if (offset) ret = unwind_spec_ehframe(dso, machine, offset,@@ -287,14 +290,20 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine,static int read_unwind_spec_debug_frame (struct dso *dso,struct machine *machine,u64 *offset) { -int fd =dso__data_fd (dso, machine); +int fd; +u64 ofs = dso->data.frame_offset; -if (fd <0 ) -return -EINVAL ; +if (ofs ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .debug_frame section for unwinding info */ - *offset =elf_section_offset (fd,".debug_frame" ); +/* Check the .debug_frame section for unwinding info */ + ofs =elf_section_offset (fd,".debug_frame" ); + dso->data.frame_offset = ofs; + } + *offset = ofs;if (*offset)return 0 ;Signature · 1 line
2.2.2
Re: [PATCH 40/42] perf callchain: Save eh/debug frame offset for dwarf unwind
In reply to Namhyung KimEm Thu, Jan 29, 2015 at 05:07:21PM +0900, Namhyung Kim escreveu:
Quoted 9 lines
When libunwind tries to resolve callchains it needs to know the offset of .eh_frame_hdr or .debug_frame to access the dso. Since it calls
dso__data_fd(), it'll try to grabdso->lockeverytime for same information. So save it todso_datastruct and reuse it.Note that there's a window between
dso__data_fd()and actual use of the fd. The fd could be closed by other threads to deal with the open file limit in dso cache code. But I think it's ok since in that caseelf_section_offset()will return 0 so it'll be tried in next acess.
Thanks, applied after rewriting the changelog to read as:
---
perf callchain: Cache eh/debug frame offset for dwarf unwind
When libunwind tries to resolve callchains it needs to know the offset of .eh_frame_hdr or .debug_frame to access the dso.
Since it will always return the same result for a given DSO, just cache the result as an optimization.
---
- Arnaldo
tip-bot for Namhyung Kim
merge bot[tip:perf/core] perf callchain: Cache eh/ debug frame offset for dwarf unwind
In reply to Namhyung Kimperf callchain: Cache eh/debug frame offset for dwarf unwind
Commit-ID: f1f13af99a903ae873f5373e965508e0486c1c29 Gitweb: http://git.kernel.org/tip/f1f13af99a903ae873f5373e965508e0486c1c29 Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 29 Jan 2015 17:07:21 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 29 Jan 2015 16:20:42 -0300
perf callchain: Cache eh/debug frame offset for dwarf unwind
When libunwind tries to resolve callchains it needs to know the offset of .eh_frame_hdr or .debug_frame to access the dso.
Since it will always return the same result for a given DSO, just cache the result as an optimization.
--- tools/perf/util/dso.h | 1 + tools/perf/util/unwind-libunwind.c | 31 ++++++++++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-)
tools/perf/util/dso.h +1 -0
@@ -139,6 +139,7 @@ struct dso {u32 status_seen;size_t file_size;struct list_head open_entry; +u64 frame_offset; } data;union {/* Tool specific area */
tools/perf/util/unwind-libunwind.c +20 -11
@@ -266,14 +266,17 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine, u64 *fde_count) {int ret = -EINVAL , fd; -u64 offset; +u64 offset = dso->data.frame_offset; - fd =dso__data_fd (dso, machine); -if (fd <0 ) -return -EINVAL ; +if (offset ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .eh_frame section for unwinding info */ - offset =elf_section_offset (fd,".eh_frame_hdr" ); +/* Check the .eh_frame section for unwinding info */ + offset =elf_section_offset (fd,".eh_frame_hdr" ); + dso->data.frame_offset = offset; + }if (offset) ret = unwind_spec_ehframe(dso, machine, offset,
@@ -287,14 +290,20 @@ static int read_unwind_spec_eh_frame(struct dso *dso, struct machine *machine,static int read_unwind_spec_debug_frame (struct dso *dso,struct machine *machine,u64 *offset) { -int fd =dso__data_fd (dso, machine); +int fd; +u64 ofs = dso->data.frame_offset; -if (fd <0 ) -return -EINVAL ; +if (ofs ==0 ) { + fd =dso__data_fd (dso, machine); +if (fd <0 ) +return -EINVAL ; -/* Check the .debug_frame section for unwinding info */ - *offset =elf_section_offset (fd,".debug_frame" ); +/* Check the .debug_frame section for unwinding info */ + ofs =elf_section_offset (fd,".debug_frame" ); + dso->data.frame_offset = ofs; + } + *offset = ofs;if (*offset)return 0 ;
Namhyung Kim
maintainerPATCH 41/42 perf tools: Add new perf data command
In reply to Namhyung KimFrom: Jiri Olsa <[email protected]>
Adding new 'perf data' command to provide operations over data files.
The 'perf data convert' sub command is coming in following patch, but there's possibility for other useful commands like 'perf data ls' (to display perf data file in directory in ls style).
--- tools/perf/Documentation/perf-data.txt | 15 +++++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-data.c | 75 ++++++++++++++++++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/perf.c | 1 + 6 files changed, 94 insertions(+) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c
tools/perf/Documentation/perf-data.txt new file +15 -0
@@ -0,0 +1,15 @@ +perf-data(1) +============== + +NAME +---- +perf-data - Data file related processing + +SYNOPSIS +-------- +[verse] +'perf data' [<common options>] <command> [<options>]", + +DESCRIPTION +----------- +Data file related processing.
tools/perf/Makefile.perf +1 -0
@@ -498,6 +498,7 @@ BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o BUILTIN_OBJS += $(OUTPUT)builtin-inject.o BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o BUILTIN_OBJS += $(OUTPUT)builtin-mem.o +BUILTIN_OBJS += $(OUTPUT)builtin-data.o PERFLIBS = $(LIB_FILE) $(LIBAPIKFS) $(LIBTRACEEVENT)
tools/perf/builtin-data.c new file +75 -0
75 lines not on this page · raw message
tools/perf/builtin.h +1 -0
@@ -37,6 +37,7 @@ extern int cmd_test(int argc, const char **argv, const char *prefix);extern int cmd_trace (int argc,const char **argv,const char *prefix);extern int cmd_inject (int argc,const char **argv,const char *prefix);extern int cmd_mem (int argc,const char **argv,const char *prefix); +extern int cmd_data (int argc,const char **argv,const char *prefix);extern int find_scripts (char **scripts_array,char **scripts_path_array);#endif
tools/perf/command-list.txt +1 -0
@@ -7,6 +7,7 @@ perf-archive mainporcelain common perf-bench mainporcelain common perf-buildid-cache mainporcelain common perf-buildid-list mainporcelain common +perf-data mainporcelain common perf-diff mainporcelain common perf-evlist mainporcelain common perf-inject mainporcelain common
tools/perf/perf.c +1 -0
@@ -62,6 +62,7 @@ static struct cmd_struct commands[] = {#endif { "inject", cmd_inject, 0 }, { "mem", cmd_mem, 0 }, + { "data", cmd_data, 0 }, }; struct pager_config {
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 42/42 perf data: Implement 'index' subcommand
In reply to Namhyung KimThe index command first splits a given data file into intermediate data files and merges them into a final data file with an index table so that it can processed using multi threads. The HEADER_DATA_INDEX feature bit is added to distinguish data file that has an index table.
--- tools/perf/Documentation/perf-data.txt | 29 +++ tools/perf/builtin-data.c | 353 +++++++++++++++++++++++++++++++++ 2 files changed, 382 insertions(+)
tools/perf/Documentation/perf-data.txt +29 -0
@@ -13,3 +13,32 @@ SYNOPSIS DESCRIPTION ----------- Data file related processing. + +COMMANDS +-------- +index:: + Build an index table for data file so that it can be processed + with multiple threads concurrently. + + +OPTIONS for 'split' +--------------------- +-i:: +--input:: + Specify input perf data file path. + +-o:: +--output:: + Specify output perf data directory path. + +-v:: +--verbose:: + Be more verbose (show counter open errors, etc). + +-f:: +--force:: + Don't complain, do it. + +SEE ALSO +-------- +linkperf:perf[1], linkperf:perf-report[1]
tools/perf/builtin-data.c +353 -0
373 lines not on this page · raw message
Signature · 1 line
2.2.2
Arnaldo Carvalho de Melo
maintainerApplied 1-5 and dwarf unwind caching one, will look at the others.
- Arnaldo
Namhyung Kim
maintainercover perf tools: Speed-up perf report by using multi thread (v3)
Hello,
This patchset converts perf report to use multiple threads in order to speed up the processing on large data files. I can see a minimum ~30% of speedup with this change. The code is still experimental and contains many rough edges. But I'd like to share and give some feedbacks.
* changes in v3)
- handle header (metadata) same as sample data (at index 0)
- maintain libunwind address space in map_groups instead of thread
- use *_time API only for indexed data file
- resolve callchain with the *_time API
- use dso__data_get/put_fd() to protect access to fd
- synthesize COMM event for command line workload
* changes in v2)
- rework with single indexed data file rather than multiple files in a directory
The perf report processes (sample) events like below:
1. preprocess sample to get matching thread/dso/symbol info
2. insert it to hists rbtree (with callchain tree) based on the info
3. optionally collapse hist entries that match given sort key(s)
4. resort hist entries (by overhead) for output
5. display the hist entries
The stage 1 is a preprocessing and mostly act like a read-only operation in that it doesn't change a machine state during the sample processing. Meta events like fork, comm and mmap can change the machine/thread state but symbols can be loaded during the processing (stage 2).
The stage 2 consumes most of the time especially with callchains and
--children option is enabled. And this work can be easily patitioned
as each sample is independent to others. But the resulting hists must be combined/collapsed to a single global hists before going to further steps.
The stage 3 is optional and only needed by certain sort keys - but with stage 2 paralellized, it needs to be done always.
The stage 4 and 5 works on whole hists so must be done serially.
So my approach is like this:
Partially do stage 1 first - but only for meta events that changes machine state. To do this I add a dummy tracking event to perf record and make it collect such meta events only. They are saved as normal data and processed before sample events at perf report time.
This also requires to handle multiple sample data concurrently and to find a corresponding machine state when processing samples. On a large profiling session, many tasks were created and exited so pid might be recycled (even more than once!). To deal with it, I managed to have thread, map_groups and comm in time sorted. The only remaining thing is symbol loading as it's done lazily when sample requires it.
With that being done, the stage 2 can be done by multiple threads. I also save each sample data (per-cpu or per-thread) in separate files during record and then merge them into a single data file with an index table. On perf report time, each region of sample data will be processed by each thread. And symbol loading is protected by a mutex lock.
For DWARF post-unwinding, dso cache data also needs to be protected by a lock and this caused a huge contention. I made it to search the rbtree speculatively first and then, if it didn't find one, search it again under the dso lock. Please take a look at it if it's acceptable.
The patch 1-9 are to support indexing for data file. With --index option, perf record will create a intermediate directory and then save meta events and sample data to separate files. And finally it'll build an index table and concatenate the data files (and also remove the intermediate direcotry).
The patch 10-23 are to manage machine and thread state using timestamp so that it can be searched when processing samples. The patch 24-37 are to implement parallel report. And finally I implemented 'perf data index' command to build an index table for a given data file.
This patchset didn't change perf record to use multi-thread. But I think it can be easily done later if needed.
Note that output has a slight difference to original version when compared using indexed data file. But they're mostly unresolved symbols for callchains.
Here is the result:
This is just elapsed time measured by 'perf stat -r 5'.
The data file was recorded during kernel build with fp callchain and size is 2.1GB. The machine has 6 core with hyper-threading enabled and I got a similar result on my laptop too.
perf report --children --no-children + --call-graph none
------------- ------------- -------------------
current 286.213349446 93.753958745 36.860880945 with index 270.158361549 87.963067415 32.896841653 + --multi-thread 166.039011492 43.209152911 8.434560193
This result is with 7.7GB data file using libunwind for callchain.
perf report --children --no-children + --call-graph none
------------- ------------- -------------------
current 150.714039134 111.420099831 5.035423803 with index 152.438739157 112.691612534 3.642109876 + --multi-thread 45.966048256 29.844907087 1.829218561
I guess the speedup of indexed data file came from skipping ordered event layer.
This result is with same file but using libdw for callchain unwind.
perf report --children --no-children + --call-graph none
------------- ------------- -------------------
current 457.507820205 491.520096816 4.831840810 with index 441.140769287 461.993666236 3.767947395 + --multi-thread 219.289176894 171.935294339 1.785351793
On my archlinux system, callchain unwind using libdw is much slower than libunwind. I'm using elfutils version 0.160. Also I don't know why --children takes less time than --no-children. Anyway we can see the --multi-thread performance is much better for each case.
You can get it from 'perf/threaded-v3' branch on my tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Please take a look and play with it. Any comments are welcome! :)
Thanks,
Namhyung
Namhyung Kim (38):
perf tools: Use a software dummy event to track task/mmap events perf tools: Add rm_rf() utility function
perf tools: Introduce copyfile_offset() function
perf tools: Create separate mmap for dummy tracking event perf tools: Introduce perf_evlist__mmap_track()
perf tools: Add HEADER_DATA_INDEX feature
perf tools: Handle indexed data file properly
perf record: Add --index option for building index table perf report: Skip dummy tracking event
perf tools: Pass session arg to perf_event__preprocess_sample() perf script: Pass session arg to ->process_event callback perf tools: Introduce thread__comm_time() helpers
perf tools: Add a test case for thread comm handling
perf tools: Use thread__comm_time() when adding hist entries perf tools: Convert dead thread list into rbtree
perf tools: Introduce machine__find*_thread_time()
perf tools: Add a test case for timed thread handling perf tools: Reducing arguments of hist_entry_iter__add() perf tools: Pass session to hist_entry_iter struct
perf tools: Maintain map groups list in a leader thread perf tools: Introduce session__find_addr_location() and friends perf callchain: Use session__find_addr_location() and friends perf tools: Add a test case for timed map groups handling perf tools: Protect dso symbol loading using a mutex
perf tools: Protect dso cache tree using dso->lock
perf tools: Protect dso cache fd with a mutex
perf callchain: Maintain libunwind's address space in map_groups perf tools: Add dso__data_get/put_fd()
perf session: Pass struct events stats to event processing functions perf hists: Pass hists struct to hist_entry_iter struct perf tools: Move BUILD_ID_SIZE definition to perf.h
perf report: Parallelize perf report using multi-thread perf tools: Add missing_threads rb tree
perf record: Synthesize COMM event for a command line workload perf tools: Fix progress ui to support multi thread
perf report: Add --multi-thread option and config item perf session: Handle index files generally
perf data: Implement 'index' subcommand
tools/perf/Documentation/perf-data.txt | 25 +- tools/perf/Documentation/perf-record.txt | 4 + tools/perf/Documentation/perf-report.txt | 3 + tools/perf/builtin-annotate.c | 8 +- tools/perf/builtin-data.c | 349 ++++++++++++++++++++- tools/perf/builtin-diff.c | 21 +- tools/perf/builtin-mem.c | 6 +- tools/perf/builtin-record.c | 189 ++++++++++- tools/perf/builtin-report.c | 81 ++++- tools/perf/builtin-script.c | 58 ++-- tools/perf/builtin-timechart.c | 10 +- tools/perf/builtin-top.c | 12 +- tools/perf/perf.h | 2 + tools/perf/tests/Build | 3 + tools/perf/tests/builtin-test.c | 12 + tools/perf/tests/dwarf-unwind.c | 14 +- tools/perf/tests/hists_common.c | 3 +- tools/perf/tests/hists_cumulate.c | 9 +- tools/perf/tests/hists_filter.c | 7 +- tools/perf/tests/hists_link.c | 10 +- tools/perf/tests/hists_output.c | 9 +- tools/perf/tests/tests.h | 3 + tools/perf/tests/thread-comm.c | 47 +++ tools/perf/tests/thread-lookup-time.c | 180 +++++++++++ tools/perf/tests/thread-mg-share.c | 7 +- tools/perf/tests/thread-mg-time.c | 88 ++++++ tools/perf/ui/browsers/hists.c | 30 +- tools/perf/ui/gtk/hists.c | 3 + tools/perf/util/build-id.c | 9 +- tools/perf/util/build-id.h | 2 - tools/perf/util/callchain.c | 6 +- tools/perf/util/callchain.h | 4 +- tools/perf/util/db-export.c | 6 +- tools/perf/util/db-export.h | 4 +- tools/perf/util/dso.c | 172 +++++++--- tools/perf/util/dso.h | 11 +- tools/perf/util/event.c | 77 ++++- tools/perf/util/event.h | 13 +- tools/perf/util/evlist.c | 161 ++++++++-- tools/perf/util/evlist.h | 22 +- tools/perf/util/evsel.h | 15 + tools/perf/util/header.c | 61 ++++ tools/perf/util/header.h | 3 + tools/perf/util/hist.c | 126 +++++--- tools/perf/util/hist.h | 9 +- tools/perf/util/machine.c | 287 ++++++++++++++--- tools/perf/util/machine.h | 15 +- tools/perf/util/map.c | 8 + tools/perf/util/map.h | 3 + tools/perf/util/ordered-events.c | 4 +- .../perf/util/scripting-engines/trace-event-perl.c | 3 +- .../util/scripting-engines/trace-event-python.c | 32 +- tools/perf/util/session.c | 345 +++++++++++++++++--- tools/perf/util/session.h | 48 ++- tools/perf/util/symbol.c | 34 +- tools/perf/util/thread.c | 173 +++++++++- tools/perf/util/thread.h | 28 +- tools/perf/util/tool.h | 14 + tools/perf/util/trace-event-scripting.c | 3 +- tools/perf/util/trace-event.h | 3 +- tools/perf/util/unwind-libdw.c | 14 +- tools/perf/util/unwind-libdw.h | 1 + tools/perf/util/unwind-libunwind.c | 98 +++--- tools/perf/util/unwind.h | 18 +- tools/perf/util/util.c | 81 ++++- tools/perf/util/util.h | 2 + 66 files changed, 2670 insertions(+), 438 deletions(-) create mode 100644 tools/perf/tests/thread-comm.c create mode 100644 tools/perf/tests/thread-lookup-time.c create mode 100644 tools/perf/tests/thread-mg-time.c
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 1/38 perf tools: Use a software dummy event to track task/mmap events
Add APIs for software dummy event to track task/comm/mmap events separately. The perf record will use them to save such events in a separate mmap buffer to make it easy to index. This is a preparation of multi-thread support which will come later.
--- tools/perf/util/evlist.c | 30 ++++++++++++++++++++++++++++++ tools/perf/util/evlist.h | 1 + tools/perf/util/evsel.h | 15 +++++++++++++++ 3 files changed, 46 insertions(+)
tools/perf/util/evlist.c +30 -0
@@ -194,6 +194,36 @@ int perf_evlist__add_default(struct perf_evlist *evlist)return -ENOMEM ; } +int perf_evlist__add_dummy_tracking (struct perf_evlist *evlist) +{ +struct perf_event_attr attr = { + .type =PERF_TYPE_SOFTWARE , + .config =PERF_COUNT_SW_DUMMY , + .exclude_kernel =1 , + }; +struct perf_evsel *evsel; + +event_attr_init (&attr); + + evsel =perf_evsel__new (&attr); +if (evsel ==NULL ) + goto error; + +/* use strdup() because free(evsel) assumes name is allocated */ + evsel->name =strdup ("dummy" ); +if (!evsel->name) + goto error_free; + +perf_evlist__add (evlist, evsel); +perf_evlist__set_tracking_event (evlist, evsel); + +return 0 ; +error_free: +perf_evsel__delete (evsel); +error: +return -ENOMEM ; +} +static int perf_evlist__add_attrs (struct perf_evlist *evlist,struct perf_event_attr *attrs,size_t nr_attrs) {
tools/perf/util/evlist.h +1 -0
@@ -68,6 +68,7 @@ void perf_evlist__delete(struct perf_evlist *evlist);void perf_evlist__add (struct perf_evlist *evlist,struct perf_evsel *entry);int perf_evlist__add_default (struct perf_evlist *evlist); +int perf_evlist__add_dummy_tracking (struct perf_evlist *evlist);int __perf_evlist__add_default_attrs (struct perf_evlist *evlist,struct perf_event_attr *attrs,size_t nr_attrs);
tools/perf/util/evsel.h +15 -0
@@ -331,6 +331,21 @@ static inline bool perf_evsel__is_function_event(struct perf_evsel *evsel)#undef FUNCTION_EVENT } +/** +* perf_evsel__is_dummy_tracking - Return whether given evsel is a dummy +* event for tracking meta events only +* +* @evsel - evsel selector to be tested +* +* Return %true if event is a dummy tracking event +*/ +static inline bool perf_evsel__is_dummy_tracking (struct perf_evsel *evsel) +{ +return evsel->attr.type ==PERF_TYPE_SOFTWARE && + evsel->attr.config ==PERF_COUNT_SW_DUMMY && + evsel->attr.task ==1 && evsel->attr.mmap ==1 ; +} +struct perf_attr_details {bool freq;bool verbose;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 2/38 perf tools: Add rm_rf() utility function
In reply to Namhyung KimThe rm_rf() function does same as the shell command 'rm -rf' which removes all directory entries recursively.
--- tools/perf/util/util.c | 43 +++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/util.h | 1 + 2 files changed, 44 insertions(+)
tools/perf/util/util.c +43 -0
@@ -72,6 +72,49 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } +int rm_rf (char *path) +{ +DIR *dir; +int ret =0 ; +struct dirent *d; +char namebuf[PATH_MAX ]; + + dir =opendir (path); +if (dir ==NULL ) +return 0 ; + +while ((d =readdir (dir)) !=NULL && !ret) { +struct stat statbuf; + +if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." )) +continue ; + +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , + path, d->d_name); + + ret =stat (namebuf, &statbuf); +if (ret <0 ) { +pr_debug ("stat failed: %s\n" , namebuf); +break ; + } + +if (S_ISREG (statbuf.st_mode)) + ret =unlink (namebuf); +else if (S_ISDIR (statbuf.st_mode)) + ret =rm_rf (namebuf); +else { +pr_debug ("unknown file: %s\n" , namebuf); + ret =-1 ; + } + } +closedir (dir); + +if (ret <0 ) +return ret; + +return rmdir (path); +} +static int slow_copyfile (const char *from,const char *to,mode_t mode) {int err =-1 ;
tools/perf/util/util.h +1 -0
@@ -249,6 +249,7 @@ static inline int sane_case(int x, int high) }int mkdir_p (char *path,mode_t mode); +int rm_rf (char *path);int copyfile (const char *from,const char *to);int copyfile_mode (const char *from,const char *to,mode_t mode);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 3/38 perf tools: Introduce copyfile_offset() function
In reply to Namhyung KimThe copyfile_offset() function is to copy source data from given offset to a destination file with an offset. It'll be used to build an indexed data file.
--- tools/perf/util/util.c | 38 +++++++++++++++++++++++++++++--------- tools/perf/util/util.h | 1 + 2 files changed, 30 insertions(+), 9 deletions(-)
tools/perf/util/util.c +29 -9
@@ -145,11 +145,38 @@ static int slow_copyfile(const char *from, const char *to, mode_t mode)return err; } +int copyfile_offset (int ifd,loff_t off_in,int ofd,loff_t off_out,u64 size) +{ +void *ptr; +loff_t pgoff; + + pgoff = off_in & ~(page_size -1 ); + off_in -= pgoff; + + ptr =mmap (NULL , off_in + size,PROT_READ ,MAP_PRIVATE , ifd, pgoff); +if (ptr ==MAP_FAILED ) +return -1 ; + +while (size) { +ssize_t ret =pwrite (ofd, ptr + off_in, size, off_out); +if (ret <0 && errno ==EINTR ) +continue ; +if (ret <=0 ) +break ; + + size -= ret; + off_in += ret; + off_out -= ret; + } +munmap (ptr, off_in + size); + +return size ?-1 :0 ; +} +int copyfile_mode (const char *from,const char *to,mode_t mode) {int fromfd, tofd;struct stat st; -void *addr;int err =-1 ;if (stat (from, &st))
@@ -166,15 +193,8 @@ int copyfile_mode(const char *from, const char *to, mode_t mode)if (tofd <0 ) goto out_close_from; - addr =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fromfd,0 ); -if (addr ==MAP_FAILED ) - goto out_close_to; - -if (write (tofd, addr, st.st_size) == st.st_size) - err =0 ; + err =copyfile_offset (fromfd,0 , tofd,0 , st.st_size); -munmap (addr, st.st_size); -out_close_to:close (tofd);if (err)unlink (to);
tools/perf/util/util.h +1 -0
@@ -252,6 +252,7 @@ int mkdir_p(char *path, mode_t mode);int rm_rf (char *path);int copyfile (const char *from,const char *to);int copyfile_mode (const char *from,const char *to,mode_t mode); +int copyfile_offset (int fromfd,loff_t from_ofs,int tofd,loff_t to_ofs,u64 size);s64 perf_atoll (const char *str);char **argv_split (const char *str,int *argcp);
Signature · 1 line
2.2.2
Re: [PATCH 03/38] perf tools: Introduce copyfile_offset() function
Quoted 5 lines · Namhyung Kim
The
copyfile_offset()function is to copy source data from given offset to a destination file with an offset. It'll be used to build an indexed data file.Signed-off-by Namhyung Kim <[email protected]>
thanks,
jirka
Namhyung Kim
maintainerPATCH 4/38 perf tools: Create separate mmap for dummy tracking event
In reply to Namhyung KimWhen indexed data file support is enabled, a dummy tracking event will be used to track metadata (like task, comm and mmap events) for a session and actual samples will be recorded in separate (intermediate) files and then merged (with index table).
Provide separate mmap to the dummy tracking event. The size is fixed to 128KiB (+ 1 page) as the event rate will be lower than samples. I originally wanted to use a single mmap for this but cross-cpu sharing is prohibited so it's per-cpu (or per-task) like normal mmaps.
--- tools/perf/builtin-record.c | 9 +++- tools/perf/util/evlist.c | 122 +++++++++++++++++++++++++++++++++++--------- tools/perf/util/evlist.h | 11 +++- 3 files changed, 117 insertions(+), 25 deletions(-)
tools/perf/builtin-record.c +8 -1
@@ -69,7 +69,7 @@ static int process_synthesized_event(struct perf_tool *tool,static int record__mmap_read (struct record *rec,int idx) { -struct perf_mmap *md = &rec->evlist->mmap[idx]; +struct perf_mmap *md =perf_evlist__mmap_desc (rec->evlist, idx);unsigned int head =perf_mmap__read_head (md);unsigned int old = md->prev;unsigned char *data = md->base + page_size;
@@ -105,6 +105,7 @@ static int record__mmap_read(struct record *rec, int idx) } md->prev = old; +perf_evlist__mmap_consume (rec->evlist, idx); out:return rc;
@@ -275,6 +276,12 @@ static int record__mmap_read_all(struct record *rec) goto out; } } +if (rec->evlist->track_mmap) { +if (record__mmap_read (rec,track_mmap_idx (i)) !=0 ) { + rc =-1 ; + goto out; + } + } }/*
tools/perf/util/evlist.c +99 -23
226 lines not on this page · raw message
tools/perf/util/evlist.h +10 -1
@@ -48,12 +48,15 @@ struct perf_evlist {bool overwrite;struct fdarray pollfd;struct perf_mmap *mmap; +struct perf_mmap *track_mmap;struct thread_map *threads;struct cpu_map *cpus;struct perf_evsel *selected;struct events_stats stats; }; +#define TRACK_MMAP_SIZE (((128 * 1024 / page_size) + 1) * page_size) +struct perf_evsel_str_handler {const char *name;void *handler;
@@ -103,8 +106,8 @@ struct perf_evsel *perf_evlist__id2evsel(struct perf_evlist *evlist, u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);union perf_event *perf_evlist__mmap_read (struct perf_evlist *evlist,int idx); -void perf_evlist__mmap_consume (struct perf_evlist *evlist,int idx); +struct perf_mmap *perf_evlist__mmap_desc (struct perf_evlist *evlist,int idx);int perf_evlist__open (struct perf_evlist *evlist);void perf_evlist__close (struct perf_evlist *evlist);
@@ -214,6 +217,12 @@ bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str);void perf_evlist__to_front (struct perf_evlist *evlist,struct perf_evsel *move_evsel); +/* convert from/to negative idx for track mmaps */ +static inline int track_mmap_idx (int idx) +{ +return -idx -1 ; +} +/** * __evlist__for_each - iterate thru all the evsels * @list: list_head instance to iterate
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 5/38 perf tools: Introduce perf_evlist__mmap_track()
In reply to Namhyung KimThe perf_evlist__mmap_track function creates data mmaps and optionally tracking mmaps for events. It'll be used for perf record to save events in a separate files and build an index table. Checking dummy tracking event in perf_evlist__mmap() alone is not enough as users can specify a dummy event (like in keep tracking testcase) without the index option.
--- tools/perf/builtin-record.c | 3 ++- tools/perf/util/evlist.c | 15 +++++++++------ tools/perf/util/evlist.h | 10 ++++++++-- 3 files changed, 19 insertions(+), 9 deletions(-)
tools/perf/builtin-record.c +2 -1
@@ -169,7 +169,8 @@ static int record__open(struct record *rec) goto out; } -if (perf_evlist__mmap (evlist, opts->mmap_pages, false) <0 ) { +if (perf_evlist__mmap_track (evlist, opts->mmap_pages, false, + false) <0 ) {if (errno ==EPERM ) {pr_err ("Permission error mapping pages.\n" "Consider increasing "
tools/perf/util/evlist.c +9 -6
@@ -834,6 +834,7 @@ static int perf_evlist__alloc_mmap(struct perf_evlist *evlist, bool track_mmap)struct mmap_params {int prot; +bool track;size_t len; };
@@ -883,7 +884,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, fd =FD (evsel, cpu, thread); -if (perf_evsel__is_dummy_tracking (evsel)) { +if (mp->track &&perf_evsel__is_dummy_tracking (evsel)) {struct mmap_params track_mp = { .prot = mp->prot, .len =TRACK_MMAP_SIZE ,
@@ -936,7 +937,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, thread); } -if (perf_evsel__is_dummy_tracking (evsel)) { +if (mp->track &&perf_evsel__is_dummy_tracking (evsel)) {/* restore idx as normal idx (positive) */ idx =track_mmap_idx (idx); }
@@ -1087,10 +1088,11 @@ int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str, }/** -* perf_evlist__mmap - Create mmaps to receive events. +* perf_evlist__mmap_track - Create mmaps to receive events. * @evlist: list of events * @pages: map length in pages * @overwrite: overwrite older events? +* @use_track_mmap: use another mmaps to track meta events * * If @overwrite is %false the user needs to signal event consumption using * perf_mmap__write_tail(). Using perf_evlist__mmap_read() does this
@@ -1098,17 +1100,18 @@ int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str,* * Return: %0 on success, negative error code otherwise. */ -int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages, -bool overwrite) +int perf_evlist__mmap_track (struct perf_evlist *evlist,unsigned int pages, +bool overwrite,bool use_track_mmap) {struct perf_evsel *evsel;const struct cpu_map *cpus = evlist->cpus;const struct thread_map *threads = evlist->threads;struct mmap_params mp = { .prot =PROT_READ | (overwrite ?0 :PROT_WRITE ), + .track = use_track_mmap, }; -if (evlist->mmap ==NULL &&perf_evlist__alloc_mmap (evlist, true) <0 ) +if (evlist->mmap ==NULL &&perf_evlist__alloc_mmap (evlist, mp.track) <0 )return -ENOMEM ;if (evlist->pollfd.entries ==NULL &&perf_evlist__alloc_pollfd (evlist) <0 )
tools/perf/util/evlist.h +8 -2
@@ -130,10 +130,16 @@ int perf_evlist__parse_mmap_pages(const struct option *opt,const char *str, int unset); -int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages, -bool overwrite); +int perf_evlist__mmap_track (struct perf_evlist *evlist,unsigned int pages, +bool overwrite,bool use_track_mmap);void perf_evlist__munmap (struct perf_evlist *evlist); +static inline int perf_evlist__mmap (struct perf_evlist *evlist, +unsigned int pages,bool overwrite) +{ +return perf_evlist__mmap_track (evlist, pages, overwrite, false); +} +void perf_evlist__disable (struct perf_evlist *evlist);void perf_evlist__enable (struct perf_evlist *evlist);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 6/38 perf tools: Add HEADER_DATA_INDEX feature
In reply to Namhyung KimThe HEADER_DATA_INDEX feature is to record index table for sample data so that they can be processed by multiple thread concurrently. Each item is a struct perf_file_section which consists of an offset and size.
--- tools/perf/builtin-record.c | 2 ++ tools/perf/util/header.c | 61 +++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/header.h | 3 +++ 3 files changed, 66 insertions(+)
tools/perf/builtin-record.c +2 -0
@@ -312,6 +312,8 @@ static void record__init_features(struct record *rec)if (!rec->opts.branch_stack)perf_header__clear_feat (&session->header,HEADER_BRANCH_STACK ); + +perf_header__clear_feat (&session->header,HEADER_DATA_INDEX ); }static volatile int workload_exec_errno;
tools/perf/util/header.c +61 -0
85 lines not on this page · raw message
tools/perf/util/header.h +3 -0
@@ -30,6 +30,7 @@ enum {HEADER_BRANCH_STACK ,HEADER_PMU_MAPPINGS ,HEADER_GROUP_DESC , +HEADER_DATA_INDEX ,HEADER_LAST_FEATURE , HEADER_FEAT_BITS = 256, };
@@ -94,6 +95,8 @@ struct perf_header {bool needs_swap;u64 data_offset;u64 data_size; +struct perf_file_section *index; +u64 nr_index;u64 feat_offset;DECLARE_BITMAP (adds_features,HEADER_FEAT_BITS );struct perf_session_env env;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 7/38 perf tools: Handle indexed data file properly
In reply to Namhyung KimWhen perf detects data file has index table, process header part first and then rest data files in a row. Note that the indexed sample data is recorded for each cpu/thread separately, it's already ordered with respect to themselves so no need to use the ordered event queue interface.
--- tools/perf/util/session.c | 62 ++++++++++++++++++++++++++++++++++++++--------- tools/perf/util/session.h | 5 ++++ 2 files changed, 55 insertions(+), 12 deletions(-)
tools/perf/util/session.c +50 -12
@@ -1300,11 +1300,10 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif -static int __perf_session__process_events (struct perf_session *session, +static int __perf_session__process_events (struct perf_session *session,int fd,u64 data_offset,u64 data_size,u64 file_size,struct perf_tool *tool) { -int fd =perf_data_file__fd (session->file);u64 head, page_offset, file_offset, file_pos, size;int err, mmap_prot, mmap_flags, map_idx =0 ;size_t mmap_size;
@@ -1327,7 +1326,9 @@ static int __perf_session__process_events(struct perf_session *session, mmap_size =MMAP_SIZE ;if (mmap_size > file_size) { mmap_size = file_size; - session->one_mmap = true; + +if (!perf_session__has_index (session)) + session->one_mmap = true; }memset (mmaps,0 ,sizeof (mmaps));
@@ -1400,29 +1401,66 @@ static int __perf_session__process_events(struct perf_session *session, err =ordered_events__flush (session, tool,OE_FLUSH__FINAL ); out_err:ui_progress__finish (); -perf_tool__warn_about_errors (tool, &session->evlist->stats);ordered_events__free (&session->ordered_events); session->one_mmap = false;return err; } +static int __perf_session__process_indexed_events (struct perf_session *session, +struct perf_tool *tool) +{ +struct perf_data_file *file = session->file; +u64 size =perf_data_file__size (file); +int err =0 , i; + +for (i =0 ; i < (int )session->header.nr_index; i++) { +struct perf_file_section *index = &session->header.index[i]; + +if (!index->size) +continue ; + +/* +* For indexed data file, samples are processed for +* each cpu/thread so it's already ordered. However +* meta-events at index 0 should be processed in order. +*/ +if (i >0 ) + tool->ordered_events = false; + + err =__perf_session__process_events (session, +perf_data_file__fd (file), + index->offset, index->size, + size, tool); +if (err <0 ) +break ; + } + +perf_tool__warn_about_errors (tool, &session->evlist->stats); +return err; +} +int perf_session__process_events (struct perf_session *session,struct perf_tool *tool) { -u64 size =perf_data_file__size (session->file); +struct perf_data_file *file = session->file; +u64 size =perf_data_file__size (file);int err;if (perf_session__register_idle_thread (session) ==NULL )return -ENOMEM ; -if (!perf_data_file__is_pipe (session->file)) - err =__perf_session__process_events (session, - session->header.data_offset, - session->header.data_size, - size, tool); -else - err =__perf_session__process_pipe_events (session, tool); +if (perf_data_file__is_pipe (file)) +return __perf_session__process_pipe_events (session, tool); +if (perf_session__has_index (session)) +return __perf_session__process_indexed_events (session, tool); + + err =__perf_session__process_events (session, +perf_data_file__fd (file), + session->header.data_offset, + session->header.data_size, + size, tool); +perf_tool__warn_about_errors (tool, &session->evlist->stats);return err; }
tools/perf/util/session.h +5 -0
@@ -137,4 +137,9 @@ int perf_event__synthesize_id_index(struct perf_tool *tool,struct perf_evlist *evlist, struct machine *machine); +static inline bool perf_session__has_index (struct perf_session *session) +{ +return perf_header__has_feat (&session->header,HEADER_DATA_INDEX ); +} +#endif /* __PERF_SESSION_H */
Signature · 1 line
2.2.2
Re: [PATCH 07/38] perf tools: Handle indexed data file properly
Quoted 17 lines · Namhyung Kim
When perf detects data file has index table, process header part first and then rest data files in a row. Note that the indexed sample data is recorded for each cpu/thread separately, it's already ordered with respect to themselves so no need to use the ordered event queue interface.
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/session.c | 62 ++++++++++++++++++++++++++++++++++++++--------- tools/perf/util/session.h | 5 ++++ 2 files changed, 55 insertions(+), 12 deletions(-)tools/perf/util/session.c +0
-0
tools/perf/util/session.c fetch_mmaped_event(struct perf_session *session, · line 1307
Quoted 9 lines
tools/perf/util/session.c +1
-2@@ -1300,11 +1300,10 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif -static int __perf_session__process_events (struct perf_session *session, +static int __perf_session__process_events (struct perf_session *session,int fd,u64 data_offset,u64 data_size,u64 file_size,struct perf_tool *tool) { -int fd =perf_data_file__fd (session->file);
why is 'fd' passed separatelly here? we have single file now and the only 'file::fd' we use is in session, no?
I might be missing some change in later patches ;-)
jirka
Namhyung Kim
maintainerRe: [PATCH 07/38] perf tools: Handle indexed data file properly
Hi Jiri,
Quoted 21 lines · Jiri Olsa
Quoted 20 lines · Namhyung Kim
When perf detects data file has index table, process header part first and then rest data files in a row. Note that the indexed sample data is recorded for each cpu/thread separately, it's already ordered with respect to themselves so no need to use the ordered event queue interface.
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/session.c | 62 ++++++++++++++++++++++++++++++++++++++--------- tools/perf/util/session.h | 5 ++++ 2 files changed, 55 insertions(+), 12 deletions(-)tools/perf/util/session.c +0
-0@@ -1300,11 +1300,10 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif
Quoted 9 lines
tools/perf/util/session.c fetch_mmaped_event(struct perf_session *session, · line 1307
Quoted 6 lines
tools/perf/util/session.c +1
-2-static int __perf_session__process_events (struct perf_session *session, +static int __perf_session__process_events (struct perf_session *session,int fd,u64 data_offset,u64 data_size,u64 file_size,struct perf_tool *tool) { -int fd =perf_data_file__fd (session->file);why is 'fd' passed separatelly here? we have single file now and the only '
file::fd' we use is in session, no?
You're right, it's a leftover from the old code. Will change.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 8/38 perf record: Add --index option for building index table
In reply to Namhyung KimThe new --index option will create indexed data file which can be processed by multiple threads parallelly. It saves meta event and sample data in separate files and merges them with an index table.
If there's an index table in the data file, the HEADER_DATA_INDEX feature bit is set and session->header.index[0] will point to the meta event area, and rest are sample data. It'd look like below:
+---------------------+
| file header |
|---------------------|
| |
| meta events[0] <-+--+
| | |
|---------------------| |
| | |
| sample data[1] <-+--+
| | |
|---------------------| |
| | |
| sample data[2] <-|--+
| | |
|---------------------| |
| ... | ...
|---------------------| |
| feature data | |
| (contains index) -+--+
+---------------------+--- tools/perf/Documentation/perf-record.txt | 4 + tools/perf/builtin-record.c | 161 +++++++++++++++++++++++++++++-- tools/perf/perf.h | 1 + tools/perf/util/session.c | 1 + 4 files changed, 158 insertions(+), 9 deletions(-)
tools/perf/Documentation/perf-record.txt +4 -0
@@ -250,6 +250,10 @@ is off by default. --running-time:: Record running and enabled time for read events (:S) +--index:: +Build an index table for sample data. This will speed up perf report by +parallel processing. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +152 -9
252 lines not on this page · raw message
tools/perf/perf.h +1 -0
@@ -54,6 +54,7 @@ struct record_opts {bool period;bool sample_intr_regs;bool running_time; +bool index;unsigned int freq;unsigned int mmap_pages;unsigned int user_freq;
tools/perf/util/session.c +1 -0
@@ -173,6 +173,7 @@ void perf_session__delete(struct perf_session *session)machines__exit (&session->machines);if (session->file)perf_data_file__close (session->file); +free (session->header.index);free (session); }
Signature · 1 line
2.2.2
Re: [PATCH 08/38] perf record: Add --index option for building index table
On Tue, Mar 03, 2015 at 12:07:20PM +0900, Namhyung Kim wrote:
SNIP
Quoted 37 lines
tools/perf/builtin-record.c +37
-0+static int record__merge_index_files (struct record *rec,int nr_index) +{ +int i; +int ret =-1 ; +u64 offset; +char path[PATH_MAX ]; +struct perf_file_section *idx; +struct perf_data_file *file = &rec->file; +struct perf_session *session = rec->session; +int output_fd =perf_data_file__fd (file); + +/* +1 for header file itself */ + nr_index++; + + idx =calloc (nr_index,sizeof (*idx)); +if (idx ==NULL ) + goto out_close; + + offset =lseek (output_fd,0 ,SEEK_END ); + + idx[0 ].offset = session->header.data_offset; + idx[0 ].size = offset - idx[0 ].offset; + +for (i =1 ; i < nr_index; i++) { +struct stat stbuf; +int fd = rec->fds[i]; + +if (fstat (fd, &stbuf) <0 ) + goto out_close; + + idx[i].offset = offset; + idx[i].size = stbuf.st_size; + + offset += stbuf.st_size; + } + +/* copy sample events */
tools/perf/builtin-record.c static int record__mmap_read(struct record *rec, int idx) · line 211
Quoted 10 lines
tools/perf/builtin-record.c +10
-0+for (i =1 ; i < nr_index; i++) { +int fd = rec->fds[i]; + +if (idx[i].size ==0 ) +continue ; + +if (copyfile_offset (fd,0 , output_fd, idx[i].offset, + idx[i].size) <0 ) + goto out_close; + }
why not do the copy in previous loop as well?
jirka
Namhyung Kim
maintainerRe: [PATCH 08/38] perf record: Add --index option for building index table
Quoted 45 lines · Jiri Olsa
On Tue, Mar 03, 2015 at 12:07:20PM +0900, Namhyung Kim wrote:
SNIP
Quoted 41 lines
tools/perf/builtin-record.c +41
-0+static int record__merge_index_files (struct record *rec,int nr_index) +{ +int i; +int ret =-1 ; +u64 offset; +char path[PATH_MAX ]; +struct perf_file_section *idx; +struct perf_data_file *file = &rec->file; +struct perf_session *session = rec->session; +int output_fd =perf_data_file__fd (file); + +/* +1 for header file itself */ + nr_index++; + + idx =calloc (nr_index,sizeof (*idx)); +if (idx ==NULL ) + goto out_close; + + offset =lseek (output_fd,0 ,SEEK_END ); + + idx[0 ].offset = session->header.data_offset; + idx[0 ].size = offset - idx[0 ].offset; + +for (i =1 ; i < nr_index; i++) { +struct stat stbuf; +int fd = rec->fds[i]; + +if (fstat (fd, &stbuf) <0 ) + goto out_close; + + idx[i].offset = offset; + idx[i].size = stbuf.st_size; + + offset += stbuf.st_size; + } + +/* copy sample events */ +for (i =1 ; i < nr_index; i++) { +int fd = rec->fds[i]; + +if (idx[i].size ==0 )
Quoted 8 lines
tools/perf/builtin-record.c static int record__mmap_read(struct record *rec, int idx) · line 211
Quoted 6 lines
tools/perf/builtin-record.c +6
-0+continue ; + +if (copyfile_offset (fd,0 , output_fd, idx[i].offset, + idx[i].size) <0 ) + goto out_close; + }why not do the copy in previous loop as well?
I will change it in the next version.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 9/38 perf report: Skip dummy tracking event
In reply to Namhyung KimThe dummy tracking event is only for tracking task/comom/mmap events and has no sample data for itself. So no need to report, just skip it.
--- tools/perf/builtin-report.c | 3 +++ tools/perf/ui/browsers/hists.c | 30 ++++++++++++++++++++++++------ tools/perf/ui/gtk/hists.c | 3 +++ 3 files changed, 30 insertions(+), 6 deletions(-)
tools/perf/builtin-report.c +3 -0
@@ -320,6 +320,9 @@ static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,struct hists *hists =evsel__hists (pos);const char *evname =perf_evsel__name (pos); +if (perf_evsel__is_dummy_tracking (pos)) +continue ; +if (symbol_conf.event_group && !perf_evsel__is_group_leader (pos))continue ;
tools/perf/ui/browsers/hists.c +24 -6
@@ -1947,14 +1947,17 @@ static int perf_evsel_menu__run(struct perf_evsel_menu *menu,return key; } -static bool filter_group_entries (struct ui_browser *browser__maybe_unused , -void *entry) +static bool filter_entries (struct ui_browser *browser__maybe_unused , +void *entry) { struct perf_evsel *evsel = list_entry(entry, struct perf_evsel, node);if (symbol_conf.event_group && !perf_evsel__is_group_leader (evsel))return true; +if (perf_evsel__is_dummy_tracking (evsel)) +return true; +return false; }
@@ -1971,7 +1974,7 @@ static int __perf_evlist__tui_browse_hists(struct perf_evlist *evlist, .refresh = ui_browser__list_head_refresh, .seek = ui_browser__list_head_seek, .write = perf_evsel_menu__write, - .filter = filter_group_entries, + .filter = filter_entries, .nr_entries = nr_entries, .priv = evlist, },
@@ -1998,21 +2001,22 @@ int perf_evlist__tui_browse_hists(struct perf_evlist *evlist, const char *help, struct perf_session_env *env) {int nr_entries = evlist->nr_entries; +struct perf_evsel *first =perf_evlist__first (evlist); +struct perf_evsel *pos; single_entry:if (nr_entries ==1 ) { -struct perf_evsel *first =perf_evlist__first (evlist); -return perf_evsel__hists_browse (first, nr_entries, help, false, hbt, min_pcnt, env); }if (symbol_conf.event_group) { -struct perf_evsel *pos; nr_entries =0 ;evlist__for_each (evlist, pos) { +if (perf_evsel__is_dummy_tracking (pos)) +continue ;if (perf_evsel__is_group_leader (pos)) nr_entries++; }
@@ -2021,6 +2025,20 @@ int perf_evlist__tui_browse_hists(struct perf_evlist *evlist, const char *help, goto single_entry; } +evlist__for_each (evlist, pos) { +if (perf_evsel__is_dummy_tracking (pos)) + nr_entries--; + } + +if (nr_entries ==1 ) { +evlist__for_each (evlist, pos) { +if (!perf_evsel__is_dummy_tracking (pos)) { + first = pos; + goto single_entry; + } + } + } +return __perf_evlist__tui_browse_hists (evlist, nr_entries, help, hbt, min_pcnt, env); }
tools/perf/ui/gtk/hists.c +3 -0
@@ -317,6 +317,9 @@ int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist,char buf[512 ];size_t size =sizeof (buf); +if (perf_evsel__is_dummy_tracking (pos)) +continue ; +if (symbol_conf.event_group) {if (!perf_evsel__is_group_leader (pos))continue ;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 10/38 perf tools: Pass session arg to perf_event__preprocess_sample()
In reply to Namhyung KimThe perf_event__preprocess_sample() translates a given ip into a matching symbol. To do that, it first finds a corresponding thread and map in the current thread tree. But for indexed data files, it needs to find a thread (and map) with slightly different APIs using timestamp. So it needs a way to know whether this session deals with an indexed data file or not.
--- tools/perf/builtin-annotate.c | 3 ++- tools/perf/builtin-diff.c | 13 +++++++++---- tools/perf/builtin-mem.c | 6 +++++- tools/perf/builtin-report.c | 3 ++- tools/perf/builtin-script.c | 20 +++++++++++--------- tools/perf/builtin-timechart.c | 10 +++++++--- tools/perf/builtin-top.c | 3 ++- tools/perf/tests/hists_cumulate.c | 2 +- tools/perf/tests/hists_filter.c | 2 +- tools/perf/tests/hists_link.c | 4 ++-- tools/perf/tests/hists_output.c | 2 +- tools/perf/util/event.c | 3 ++- tools/perf/util/event.h | 4 +++- 13 files changed, 48 insertions(+), 27 deletions(-)
tools/perf/builtin-annotate.c +2 -1
@@ -85,7 +85,8 @@ static int process_sample_event(struct perf_tool *tool, struct perf_annotate *ann = container_of(tool, struct perf_annotate, tool);struct addr_location al; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + ann->session) <0 ) {pr_warning ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
tools/perf/builtin-diff.c +9 -4
@@ -42,6 +42,7 @@ struct diff_hpp_fmt { };struct data__file { +struct perf_tool tool;struct perf_session *session;struct perf_data_file file;int idx;
@@ -320,16 +321,18 @@ static int hists__add_entry(struct hists *hists,return -ENOMEM ; } -static int diff__process_sample_event (struct perf_tool *tool__maybe_unused , +static int diff__process_sample_event (struct perf_tool *tool,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel,struct machine *machine) {struct addr_location al; + struct data__file *d = container_of(tool, struct data__file, tool);struct hists *hists =evsel__hists (evsel); -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + d->session) <0 ) {pr_warning ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
@@ -740,14 +743,16 @@ static int __cmd_diff(void)int ret = -EINVAL , i;data__for_each_file (i, d) { - d->session =perf_session__new (&d->file, false, &tool); +memcpy (&d->tool, &tool,sizeof (tool)); + + d->session =perf_session__new (&d->file, false, &d->tool);if (!d->session) {pr_err ("Failed to open %s\n" , d->file.path); ret =-1 ; goto out_delete; } - ret =perf_session__process_events (d->session, &tool); + ret =perf_session__process_events (d->session, &d->tool);if (ret) {pr_err ("Failed to process %s\n" , d->file.path); goto out_delete;
tools/perf/builtin-mem.c +5 -1
@@ -12,6 +12,7 @@struct perf_mem {struct perf_tool tool; +struct perf_session *session;char const *input_name;bool hide_unresolved;bool dump_raw;
@@ -66,7 +67,8 @@ dump_raw_samples(struct perf_tool *tool,struct addr_location al;const char *fmt; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + mem->session) <0 ) {fprintf (stderr,"problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
@@ -129,6 +131,8 @@ static int report_raw_events(struct perf_mem *mem)if (session ==NULL )return -1 ; + mem->session = session; +if (mem->cpu_list) { ret =perf_session__cpu_bitmap (session, mem->cpu_list, mem->cpu_bitmap);
tools/perf/builtin-report.c +2 -1
@@ -142,7 +142,8 @@ static int process_sample_event(struct perf_tool *tool, };int ret; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + rep->session) <0 ) {pr_debug ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
tools/perf/builtin-script.c +11 -9
@@ -542,13 +542,21 @@ static int cleanup_scripting(void)return scripting_ops->stop_script (); } -static int process_sample_event (struct perf_tool *tool__maybe_unused , +struct perf_script { +struct perf_tool tool; +struct perf_session *session; +bool show_task_events; +bool show_mmap_events; +}; + +static int process_sample_event (struct perf_tool *tool,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel,struct machine *machine) {struct addr_location al; + struct perf_script *script = container_of(tool, struct perf_script, tool);struct thread *thread =machine__findnew_thread (machine, sample->pid, sample->tid);
@@ -569,7 +577,8 @@ static int process_sample_event(struct perf_tool *tool __maybe_unused,return 0 ; } -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + script->session) <0 ) {pr_err ("problem processing %d event, skipping it.\n" , event->header.type);return -1 ;
@@ -586,13 +595,6 @@ static int process_sample_event(struct perf_tool *tool __maybe_unused,return 0 ; } -struct perf_script { -struct perf_tool tool; -struct perf_session *session; -bool show_task_events; -bool show_mmap_events; -}; -static int process_attr (struct perf_tool *tool,union perf_event *event,struct perf_evlist **pevlist) {
tools/perf/builtin-timechart.c +7 -3
@@ -48,6 +48,7 @@ struct wake_event;struct timechart {struct perf_tool tool; +struct perf_session *session;struct per_pid *all_data;struct power_event *power_events;struct wake_event *wake_events;
@@ -469,7 +470,8 @@ static void sched_switch(struct timechart *tchart, int cpu, u64 timestamp,static const char *cat_backtrace (union perf_event *event,struct perf_sample *sample, -struct machine *machine) +struct machine *machine, +struct perf_session *session) {struct addr_location al;unsigned int i;
@@ -488,7 +490,8 @@ static const char *cat_backtrace(union perf_event *event,if (!chain) goto exit; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) { +if (perf_event__preprocess_sample (event, machine, &al, sample, + session) <0 ) {fprintf (stderr,"problem processing %d event, skipping it.\n" , event->header.type); goto exit;
@@ -567,7 +570,7 @@ static int process_sample_event(struct perf_tool *tool,if (evsel->handler !=NULL ) {tracepoint_handler f = evsel->handler;return f (tchart, evsel, sample, -cat_backtrace (event, sample, machine)); +cat_backtrace (event, sample, machine, tchart->session)); }return 0 ;
@@ -1623,6 +1626,7 @@ static int __cmd_timechart(struct timechart *tchart, const char *output_name) goto out_delete; } + tchart->session = session; ret =perf_session__process_events (session, &tchart->tool);if (ret) goto out_delete;
tools/perf/builtin-top.c +2 -1
@@ -723,7 +723,8 @@ static void perf_event__process_sample(struct perf_tool *tool,if (event->header.misc &PERF_RECORD_MISC_EXACT_IP ) top->exact_samples++; -if (perf_event__preprocess_sample (event, machine, &al, sample) <0 ) +if (perf_event__preprocess_sample (event, machine, &al, sample, + top->session) <0 )return ; if (!top->kptr_restrict_warned &&
tools/perf/tests/hists_cumulate.c +1 -1
@@ -101,7 +101,7 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) sample.callchain = (struct ip_callchain *)fake_callchains[i];if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; if (hist_entry_iter__add(&iter, &al, evsel, &sample,
tools/perf/tests/hists_filter.c +1 -1
@@ -78,7 +78,7 @@ static int add_hist_entries(struct perf_evlist *evlist, sample.ip = fake_samples[i].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; if (hist_entry_iter__add(&iter, &al, evsel, &sample,
tools/perf/tests/hists_link.c +2 -2
@@ -86,7 +86,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) sample.tid = fake_common_samples[k].pid; sample.ip = fake_common_samples[k].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; he = __hists__add_entry(hists, &al, NULL,
@@ -110,7 +110,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) sample.tid = fake_samples[i][k].pid; sample.ip = fake_samples[i][k].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; he = __hists__add_entry(hists, &al, NULL,
tools/perf/tests/hists_output.c +1 -1
@@ -67,7 +67,7 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) sample.ip = fake_samples[i].ip;if (perf_event__preprocess_sample (&event, machine, &al, - &sample) <0 ) + &sample,NULL ) <0 ) goto out; if (hist_entry_iter__add(&iter, &al, evsel, &sample,
tools/perf/util/event.c +2 -1
@@ -836,7 +836,8 @@ void thread__find_addr_location(struct thread *thread,int perf_event__preprocess_sample (const union perf_event *event,struct machine *machine,struct addr_location *al, -struct perf_sample *sample) +struct perf_sample *sample, +struct perf_session *session__maybe_unused ) {u8 cpumode = event->header.misc &PERF_RECORD_MISC_CPUMODE_MASK ; struct thread *thread = machine__findnew_thread(machine, sample->pid,
tools/perf/util/event.h +3 -1
@@ -353,11 +353,13 @@ int perf_event__process(struct perf_tool *tool, struct machine *machine);struct addr_location ; +struct perf_session ;int perf_event__preprocess_sample (const union perf_event *event,struct machine *machine,struct addr_location *al, -struct perf_sample *sample); +struct perf_sample *sample, +struct perf_session *session);struct thread ;
Signature · 1 line
2.2.2
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 10/38] perf tools: Pass session arg to perf_event__preprocess_sample()
Em Tue, Mar 03, 2015 at 12:07:22PM +0900, Namhyung Kim escreveu:
Quoted 6 lines
The
perf_event__preprocess_sample()translates a given ip into a matching symbol. To do that, it first finds a corresponding thread and map in the current thread tree. But for indexed data files, it needs to find a thread (and map) with slightly different APIs using timestamp. So it needs a way to know whether this session deals with an indexed data file or not.
I need to look into this, but in general, I'm trying to _remove_ accesses to perf_session :-)
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 10/38] perf tools: Pass session arg to perf_event__preprocess_sample()
Quoted 10 lines · Arnaldo Carvalho de Melo
Em Tue, Mar 03, 2015 at 12:07:22PM +0900, Namhyung Kim escreveu:
Quoted 6 lines
The
perf_event__preprocess_sample()translates a given ip into a matching symbol. To do that, it first finds a corresponding thread and map in the current thread tree. But for indexed data files, it needs to find a thread (and map) with slightly different APIs using timestamp. So it needs a way to know whether this session deals with an indexed data file or not.I need to look into this, but in general, I'm trying to _remove_ accesses to
perf_session:-)
The reason is that it needs to know whether the current session has index so that it's accessing the samples in sequence or in random order. If we can save that info at some other place (like a global variable?) it doesn't need to pass session.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 11/38 perf script: Pass session arg to ->process_event callback
In reply to Namhyung KimSometimes it needs to retrieve symbol info inside a script engine so we need to pass the session pointer to find the symbol correctly as with previous patch.
--- tools/perf/builtin-script.c | 23 ++++++++++++---------- tools/perf/util/db-export.c | 6 ++++-- tools/perf/util/db-export.h | 4 +++- tools/perf/util/event.c | 3 ++- tools/perf/util/event.h | 3 ++- .../perf/util/scripting-engines/trace-event-perl.c | 3 ++- .../util/scripting-engines/trace-event-python.c | 5 +++-- tools/perf/util/trace-event-scripting.c | 3 ++- tools/perf/util/trace-event.h | 3 ++- 9 files changed, 33 insertions(+), 20 deletions(-)
tools/perf/builtin-script.c +13 -10
@@ -377,9 +377,10 @@ static void print_sample_start(struct perf_sample *sample, }static void print_sample_addr (union perf_event *event, -struct perf_sample *sample, -struct thread *thread, -struct perf_event_attr *attr) +struct perf_sample *sample, +struct thread *thread, +struct perf_event_attr *attr, +struct perf_session *session) {struct addr_location al;
@@ -388,7 +389,7 @@ static void print_sample_addr(union perf_event *event,if (!sample_addr_correlates_sym (attr))return ; -perf_event__preprocess_sample_addr (event, sample, thread, &al); +perf_event__preprocess_sample_addr (event, sample, thread, &al, session);if (PRINT_FIELD (SYM )) {printf (" " );
@@ -409,7 +410,8 @@ static void print_sample_bts(union perf_event *event,struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al) + struct addr_location *al, + struct perf_session *session) { struct perf_event_attr *attr = &evsel->attr;bool print_srcline_last = false;
@@ -436,7 +438,7 @@ static void print_sample_bts(union perf_event *event, ((evsel->attr.sample_type & PERF_SAMPLE_ADDR) && !output[attr->type].user_set)) {printf (" => " ); -print_sample_addr (event, sample, thread, attr); +print_sample_addr (event, sample, thread, attr, session); }if (print_srcline_last)
@@ -447,7 +449,7 @@ static void print_sample_bts(union perf_event *event,static void process_event (union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel,struct thread *thread, -struct addr_location *al) +struct addr_location *al,struct perf_session *session) {struct perf_event_attr *attr = &evsel->attr;
@@ -465,7 +467,7 @@ static void process_event(union perf_event *event, struct perf_sample *sample, }if (is_bts_event (attr)) { -print_sample_bts (event, sample, evsel, thread, al); +print_sample_bts (event, sample, evsel, thread, al, session);return ; }
@@ -473,7 +475,7 @@ static void process_event(union perf_event *event, struct perf_sample *sample,event_format__print (evsel->tp_format, sample->cpu, sample->raw_data, sample->raw_size);if (PRINT_FIELD (ADDR )) -print_sample_addr (event, sample, thread, attr); +print_sample_addr (event, sample, thread, attr, session);if (PRINT_FIELD (IP )) {if (!symbol_conf.use_callchain)
@@ -590,7 +592,8 @@ static int process_sample_event(struct perf_tool *tool,if (cpu_list && !test_bit (sample->cpu, cpu_bitmap))return 0 ; - scripting_ops->process_event (event, sample, evsel, thread, &al); + scripting_ops->process_event (event, sample, evsel, thread, &al, + script->session);return 0 ; }
tools/perf/util/db-export.c +4 -2
@@ -282,7 +282,8 @@ int db_export__branch_type(struct db_export *dbe, u32 branch_type,int db_export__sample (struct db_export *dbe,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel, -struct thread *thread,struct addr_location *al) +struct thread *thread,struct addr_location *al, +struct perf_session *session) {struct export_sample es = { .event = event,
@@ -328,7 +329,8 @@ int db_export__sample(struct db_export *dbe, union perf_event *event, sample_addr_correlates_sym(&evsel->attr)) {struct addr_location addr_al; -perf_event__preprocess_sample_addr (event, sample, thread, &addr_al); +perf_event__preprocess_sample_addr (event, sample, thread, + &addr_al, session); err =db_ids_from_al (dbe, &addr_al, &es.addr_dso_db_id, &es.addr_sym_db_id, &es.addr_offset);if (err)
tools/perf/util/db-export.h +3 -1
@@ -29,6 +29,7 @@ struct addr_location;struct call_return_processor ;struct call_path ;struct call_return ; +struct perf_session ;struct export_sample {union perf_event *event;
@@ -97,7 +98,8 @@ int db_export__branch_type(struct db_export *dbe, u32 branch_type, const char *name);int db_export__sample (struct db_export *dbe,union perf_event *event,struct perf_sample *sample,struct perf_evsel *evsel, -struct thread *thread,struct addr_location *al); +struct thread *thread,struct addr_location *al, +struct perf_session *session);int db_export__branch_types (struct db_export *dbe);
tools/perf/util/event.c +2 -1
@@ -918,7 +918,8 @@ bool sample_addr_correlates_sym(struct perf_event_attr *attr)void perf_event__preprocess_sample_addr (union perf_event *event,struct perf_sample *sample,struct thread *thread, -struct addr_location *al) +struct addr_location *al, +struct perf_session *session__maybe_unused ) {u8 cpumode = event->header.misc &PERF_RECORD_MISC_CPUMODE_MASK ;
tools/perf/util/event.h +2 -1
@@ -368,7 +368,8 @@ bool sample_addr_correlates_sym(struct perf_event_attr *attr);void perf_event__preprocess_sample_addr (union perf_event *event,struct perf_sample *sample,struct thread *thread, -struct addr_location *al); +struct addr_location *al, +struct perf_session *session);const char *perf_event__name (unsigned int id);
tools/perf/util/scripting-engines/trace-event-perl.c +2 -1
@@ -356,7 +356,8 @@ static void perl_process_event(union perf_event *event,struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al __maybe_unused) + struct addr_location *al __maybe_unused, + struct perf_session *session __maybe_unused) {perl_process_tracepoint (sample ,evsel ,thread );perl_process_event_generic (event, sample, evsel);
tools/perf/util/scripting-engines/trace-event-python.c +3 -2
@@ -839,7 +839,8 @@ static void python_process_event(union perf_event *event,struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al) + struct addr_location *al, + struct perf_session *session) { struct tables *tables = &tables_global;
@@ -851,7 +852,7 @@ static void python_process_event(union perf_event *event,default :if (tables->db_export_mode)db_export__sample (&tables->dbe, event, sample, evsel, - thread, al); + thread, al, session);else python_process_general_event (sample, evsel, thread, al); }
tools/perf/util/trace-event-scripting.c +2 -1
@@ -44,7 +44,8 @@ static void process_event_unsupported(union perf_event *event __maybe_unused,struct perf_sample *sample__maybe_unused , struct perf_evsel *evsel __maybe_unused, struct thread *thread __maybe_unused, - struct addr_location *al __maybe_unused) + struct addr_location *al __maybe_unused, + struct perf_session *session __maybe_unused) { }
tools/perf/util/trace-event.h +2 -1
@@ -73,7 +73,8 @@ struct scripting_ops {struct perf_sample *sample, struct perf_evsel *evsel, struct thread *thread, - struct addr_location *al); + struct addr_location *al, + struct perf_session *session);int (*generate_script) (struct pevent *pevent,const char *outfile); };
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 12/38 perf tools: Introduce thread__comm_time() helpers
In reply to Namhyung KimWhen data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The thread__comm_time() will mostly work even if PERF_SAMPLE_TIME bit is off since in that case, sample->time will be -1 so it'll take the last comm anyway.
--- tools/perf/util/thread.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 2 ++ 2 files changed, 34 insertions(+), 1 deletion(-)
tools/perf/util/thread.c +32 -1
@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; } +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp) +{ +struct comm *comm; + +list_for_each_entry (comm, &thread->comm_list, list) { +if (timestamp >= comm->start) +return comm; + } + +if (list_empty (&thread->comm_list)) +return NULL ; + + return list_last_entry(&thread->comm_list, struct comm, list); +} +int __thread__set_comm (struct thread *thread,const char *str,u64 timestamp,bool exec) {
@@ -118,7 +133,13 @@ int __thread__set_comm(struct thread *thread, const char *str, u64 timestamp, new =comm__new (str, timestamp, exec);if (!new)return -ENOMEM ; -list_add (&new->list, &thread->comm_list); + +/* sort by time */ +list_for_each_entry (curr, &thread->comm_list, list) { +if (timestamp >= curr->start) +break ; + } +list_add_tail (&new->list, &curr->list);if (exec)unwind__flush_access (thread);
@@ -139,6 +160,16 @@ const char *thread__comm_str(const struct thread *thread)return comm__str (comm); } +const char *thread__comm_str_time (const struct thread *thread,u64 timestamp) +{ +const struct comm *comm =thread__comm_time (thread, timestamp); + +if (!comm) +return NULL ; + +return comm__str (comm); +} +/* CHECKME: it should probably better return the max comm len from its comm list */ int thread__comm_len (struct thread *thread) {
tools/perf/util/thread.h +2 -0
@@ -53,7 +53,9 @@ static inline int thread__set_comm(struct thread *thread, const char *comm,int thread__comm_len (struct thread *thread);struct comm *thread__comm (const struct thread *thread);struct comm *thread__exec_comm (const struct thread *thread); +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp);const char *thread__comm_str (const struct thread *thread); +const char *thread__comm_str_time (const struct thread *thread,u64 timestamp);void thread__insert_map (struct thread *thread,struct map *map);int thread__fork (struct thread *thread,struct thread *parent,u64 timestamp);size_t thread__fprintf (struct thread *thread,FILE *fp);
Signature · 1 line
2.2.2
Re: [PATCH 12/38] perf tools: Introduce thread__comm_time() helpers
Quoted 21 lines · Namhyung Kim
When data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The
thread__comm_time()will mostly work even ifPERF_SAMPLE_TIMEbit is off since in that case,sample->timewill be -1 so it'll take the last comm anyway.Cc Frederic Weisbecker <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/thread.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 2 ++ 2 files changed, 34 insertions(+), 1 deletion(-)tools/perf/util/thread.c +0
-0
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 106
Quoted 4 lines
tools/perf/util/thread.c +1
-0@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; } +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp)
Usually thread__comm_foo() would suggest that we return the "foo" from a thread comm. For example thread__comm_len() returns the len of the last thread comm. thread__comm_str() returns the string of the last thread comm.
So thread__comm_time() suggests that we return the timestamp of the default thread comm.
Ideally all thread__comm_foo() accessors should now take a timestamp as an argument. Now there
are quite some callers of such APIs, I'm not sure they will all turn into passing a precise timestamp, but the current callers are interested in the last comm so perhaps those can be turned into thread__last_comm[_str](). The call would gain some clarity.
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 119
Quoted 13 lines
tools/perf/util/thread.c +13
-0+{ +struct comm *comm; + +list_for_each_entry (comm, &thread->comm_list, list) { +if (timestamp >= comm->start) +return comm; + } + +if (list_empty (&thread->comm_list)) +return NULL ; + + return list_last_entry(&thread->comm_list, struct comm, list); +}
Yes, handling the thread's comm lifecycle correctly with fetching the right comm at a given time is the evolution I had in mind. I haven't looked at the rest of your patchset but this change alone seem to go to the right direction.
Thanks!
Quoted 53 lines
tools/perf/util/thread.c +18
-1+int __thread__set_comm (struct thread *thread,const char *str,u64 timestamp,bool exec) {@@ -118,7 +133,13 @@ int __thread__set_comm(struct thread *thread, const char *str, u64 timestamp, new =comm__new (str, timestamp, exec);if (!new)return -ENOMEM ; -list_add (&new->list, &thread->comm_list); + +/* sort by time */ +list_for_each_entry (curr, &thread->comm_list, list) { +if (timestamp >= curr->start) +break ; + } +list_add_tail (&new->list, &curr->list);if (exec)unwind__flush_access (thread);@@ -139,6 +160,16 @@ const char *thread__comm_str(const struct thread *thread)return comm__str (comm); } +const char *thread__comm_str_time (const struct thread *thread,u64 timestamp) +{ +const struct comm *comm =thread__comm_time (thread, timestamp); + +if (!comm) +return NULL ; + +return comm__str (comm); +} +/* CHECKME: it should probably better return the max comm len from its comm list */ int thread__comm_len (struct thread *thread) {tools/perf/util/thread.h +2
-0@@ -53,7 +53,9 @@ static inline int thread__set_comm(struct thread *thread, const char *comm,int thread__comm_len (struct thread *thread);struct comm *thread__comm (const struct thread *thread);struct comm *thread__exec_comm (const struct thread *thread); +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp);const char *thread__comm_str (const struct thread *thread); +const char *thread__comm_str_time (const struct thread *thread,u64 timestamp);void thread__insert_map (struct thread *thread,struct map *map);int thread__fork (struct thread *thread,struct thread *parent,u64 timestamp);size_t thread__fprintf (struct thread *thread,FILE *fp);Signature · 1 line
2.2.2
Namhyung Kim
maintainerRe: [PATCH 12/38] perf tools: Introduce thread__comm_time() helpers
Hi Frederic,
Quoted 25 lines · Frederic Weisbecker
Quoted 24 lines · Namhyung Kim
When data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The
thread__comm_time()will mostly work even ifPERF_SAMPLE_TIMEbit is off since in that case,sample->timewill be -1 so it'll take the last comm anyway.Cc Frederic Weisbecker <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/thread.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 2 ++ 2 files changed, 34 insertions(+), 1 deletion(-)tools/perf/util/thread.c +0
-0@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; }
Quoted 5 lines
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 106
Quoted 1 line
tools/perf/util/thread.c +1
-0+struct comm *thread__comm_time (const struct thread *thread,u64 timestamp)Usually
thread__comm_foo()would suggest that we return the "foo" from a thread comm. For examplethread__comm_len()returns the len of the last thread comm.thread__comm_str()returns the string of the last thread comm.
Ah, okay.
Quoted 6 lines
So
thread__comm_time()suggests that we return the timestamp of the default thread comm.
Ideally allthread__comm_foo()accessors should now take a timestamp as an argument. Now there
are quite some callers of such APIs, I'm not sure they will all turn into passing a precise timestamp, but the current callers are interested in the last comm so perhaps those can be turned into thread__last_comm[_str](). The call would gain some clarity.
I'm fine with this change. Actually I also don't like the _time suffix but couldn't come up with a better name. ;-)
I also think the last comm also be thought as current comm. So how about thread__curr_comm[_str]() then?
Quoted 10 lines
Quoted 10 lines
tools/perf/util/thread.c +9
-0+{ +struct comm *comm; + +list_for_each_entry (comm, &thread->comm_list, list) { +if (timestamp >= comm->start) +return comm; + } + +if (list_empty (&thread->comm_list))
Quoted 8 lines
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 119
Quoted 4 lines
tools/perf/util/thread.c +4
-0+return NULL ; + + return list_last_entry(&thread->comm_list, struct comm, list); +}Yes, handling the thread's comm lifecycle correctly with fetching the right comm at a given time is the evolution I had in mind. I haven't looked at the rest of your patchset but this change alone seem to go to the right direction.
The idea is extending it to find thread and maps so that we can process samples parallelly.
Thanks for your review!
Namhyung
Re: [PATCH 12/38] perf tools: Introduce thread__comm_time() helpers
Quoted 28 lines · Namhyung Kim
Hi Frederic,
Quoted 25 lines · Frederic Weisbecker
Quoted 24 lines · Namhyung Kim
When data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The
thread__comm_time()will mostly work even ifPERF_SAMPLE_TIMEbit is off since in that case,sample->timewill be -1 so it'll take the last comm anyway.Cc Frederic Weisbecker <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/thread.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 2 ++ 2 files changed, 34 insertions(+), 1 deletion(-)tools/perf/util/thread.c +0
-0@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; }
Quoted 7 lines
Quoted 5 lines
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 106
Quoted 1 line
tools/perf/util/thread.c +1
-0+struct comm *thread__comm_time (const struct thread *thread,u64 timestamp)Usually
thread__comm_foo()would suggest that we return the "foo" from a thread comm. For examplethread__comm_len()returns the len of the last thread comm.thread__comm_str()returns the string of the last thread comm.Ah, okay.
I mean, that's just an impression, others may have a different one :o)
Quoted 8 lines
Quoted 7 lines
So
thread__comm_time()suggests that we return the timestamp of the default thread comm.
Ideally allthread__comm_foo()accessors should now take a timestamp as an argument. Now there
are quite some callers of such APIs, I'm not sure they will all turn into passing a precise timestamp, but the current callers are interested in the last comm so perhaps those can be turned into thread__last_comm[_str](). The call would gain some clarity.
Quoted 5 lines
I'm fine with this change. Actually I also don't like the
_timesuffix but couldn't come up with a better name. ;-)I also think the last comm also be thought as current comm. So how about thread__curr_comm[_str]() then?
Yeah, curr works for me too.
Thanks!
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
tools/perf/util/thread.c +9
-0+{ +struct comm *comm; + +list_for_each_entry (comm, &thread->comm_list, list) { +if (timestamp >= comm->start) +return comm; + } + +if (list_empty (&thread->comm_list))Quoted 8 lines
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 119
Quoted 4 lines
tools/perf/util/thread.c +4
-0+return NULL ; + + return list_last_entry(&thread->comm_list, struct comm, list); +}Yes, handling the thread's comm lifecycle correctly with fetching the right comm at a given time is the evolution I had in mind. I haven't looked at the rest of your patchset but this change alone seem to go to the right direction.
Quoted 2 lines
The idea is extending it to find thread and maps so that we can process samples parallelly.
Ok!
Thanks!
Quoted 3 lines
Thanks for your review!
Namhyung
Namhyung Kim
maintainerRe: [PATCH 12/38] perf tools: Introduce thread__comm_time() helpers
Hi Frederic and Arnaldo,
Quoted 35 lines · Frederic Weisbecker
Quoted 34 lines · Namhyung Kim
Hi Frederic,
Quoted 30 lines · Frederic Weisbecker
Quoted 21 lines · Namhyung Kim
When data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The
thread__comm_time()will mostly work even ifPERF_SAMPLE_TIMEbit is off since in that case,sample->timewill be -1 so it'll take the last comm anyway.Cc Frederic Weisbecker <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/thread.c | 33 ++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 2 ++ 2 files changed, 34 insertions(+), 1 deletion(-)tools/perf/util/thread.c +0
-0tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 106
Quoted 4 lines
tools/perf/util/thread.c +1
-0@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; } +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp)Usually
thread__comm_foo()would suggest that we return the "foo" from a thread comm. For examplethread__comm_len()returns the len of the last thread comm.thread__comm_str()returns the string of the last thread comm.
Quoted 3 lines
Quoted 1 line
Ah, okay.
I mean, that's just an impression, others may have a different one :o)
Right. Although I agree with your idea of function naming, I'm not sure it's worth changing every function call site for this - and for similar machine__find(new)_thread()_time.
Arnaldo, What do you think?
Thanks,
Namhyung
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 12/38] perf tools: Introduce thread__comm_time() helpers
Em Fri, Mar 06, 2015 at 01:38:06PM +0900, Namhyung Kim escreveu:
Quoted 22 lines · Frederic Weisbecker
Quoted 20 lines · Namhyung Kim
Quoted 19 lines · Frederic Weisbecker
Quoted 18 lines · Namhyung Kim
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 104
Quoted 13 lines
When data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The
thread__comm_time()will mostly work even ifPERF_SAMPLE_TIMEbit is off since in that case,sample->timewill be -1 so it'll take the last comm anyway.+++
b/tools/perf/util/thread.ctools/perf/util/thread.c +1
-0@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp)Usually
thread__comm_foo()would suggest that we return the "foo" from a thread comm. For examplethread__comm_len()returns the len of the last thread comm.thread__comm_str()returns the string of the last thread comm.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Ah, okay.
I mean, that's just an impression, others may have a different one :o)
Right. Although I agree with your idea of function naming, I'm not sure it's worth changing every function call site for this - and for similar machine__find(new)_thread()_time.
So you need to find a thread using a new search key, namely (pid, tid, time), is that right?
I.e. you process all the meta events so that you can have the lifetime events for the whole session, right?
If that is the case, then I think we should keep the existing functions as is, for tools not yet converted to this new way of processing samples, be it just because there was no time yet to do it or because we may find that it is not adequate for some tools, and introduce the new API, that takes the time as part of the key, so, in addition to:
machine__find_thread
machine__findnew_thread
namely:
machine__find_thread_by_time
machine__findnew_thread_by_time
But thread__comm_time() for me is to get the time of most recent comm for that thread. I.e. I agree with Frédéric, if what you want is to _find_ the comm for a specific key, i.e. time, then I think it should be:
thread__find_comm_by_time(thread, tstamp)
Ah, how do you manage the thread rbtree? First a search by pid/tid and then another by time? Or put it all together and then machine__find_thread becomes just a:
struct thread *machine__find_thread(machine, pid, tid)
{
return machine__find_thread_by_time(machine, pid, tid, -1);
}And it returns the most recent thread for that pid/tid?
I.e. machine__remove_thread() becomes a noop for such use cases and we never delete a struct thread instance?
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 12/38] perf tools: Introduce thread__comm_time() helpers
Hi Arnaldo,
Quoted 25 lines · Arnaldo Carvalho de Melo
Em Fri, Mar 06, 2015 at 01:38:06PM +0900, Namhyung Kim escreveu:
Quoted 23 lines · Frederic Weisbecker
Quoted 22 lines · Namhyung Kim
Quoted 21 lines · Frederic Weisbecker
Quoted 10 lines · Namhyung Kim
Quoted 8 lines
When data file indexing is enabled, it processes all task, comm and mmap events first and then goes to the sample events. So all it sees is the last comm of a thread although it has information at the time of sample.
Sort thread's comm by time so that it can find appropriate comm at the sample time. The
thread__comm_time()will mostly work even ifPERF_SAMPLE_TIMEbit is off since in that case,sample->timewill be -1 so it'll take the last comm anyway.Quoted 8 lines
tools/perf/util/thread.c struct comm *thread__exec_comm(const struct thread *thread) · line 104
Quoted 4 lines
+++
b/tools/perf/util/thread.ctools/perf/util/thread.c +1
-0@@ -103,6 +103,21 @@ struct comm *thread__exec_comm(const struct thread *thread)return last; +struct comm *thread__comm_time (const struct thread *thread,u64 timestamp)Usually
thread__comm_foo()would suggest that we return the "foo" from a thread comm. For examplethread__comm_len()returns the len of the last thread comm.thread__comm_str()returns the string of the last thread comm.Ah, okay.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
I mean, that's just an impression, others may have a different one :o)
Right. Although I agree with your idea of function naming, I'm not sure it's worth changing every function call site for this - and for similar machine__find(new)_thread()_time.
So you need to find a thread using a new search key, namely (pid, tid, time), is that right?
Right.
Quoted 3 lines
I.e. you process all the meta events so that you can have the lifetime events for the whole session, right?
Exactly. :)
Quoted 16 lines
If that is the case, then I think we should keep the existing functions as is, for tools not yet converted to this new way of processing samples, be it just because there was no time yet to do it or because we may find that it is not adequate for some tools, and introduce the new API, that takes the time as part of the key, so, in addition to:
machine__find_thread
machine__findnew_threadnamely:
machine__find_thread_by_time
machine__findnew_thread_by_time
Quoted 6 lines
But
thread__comm_time()for me is to get the time of most recent comm for that thread. I.e. I agree with Frédéric, if what you want is to _find_ the comm for a specific key, i.e. time, then I think it should be:
thread__find_comm_by_time(thread, tstamp)
I'm okay with it, will change!
Quoted 11 lines
Ah, how do you manage the thread rbtree? First a search by pid/tid and then another by time? Or put it all together and then
machine__find_threadbecomes just a:struct thread *machine__find_thread(machine, pid, tid)
{ return machine__find_thread_by_time(machine, pid, tid, -1); }And it returns the most recent thread for that pid/tid?
Yeah, but there's an issue with multi-thread access. The find_thread API is good but findnew_thread API is a problem since we need to protect creation of new thread from multiple concurrent accesses. So basically it needs a lock but since it's in a hotpath it'd hurt the performance IMHO.
So I separate the API - existing functions should be called in a serial execution only. The new APIs allow concurrent accesses in that it searches (current) thread tree first and then searches dead thread tree if not found. Almost all of thread search should success at this stage. Since these operations don't change anything, they don't require locking.
For occasional search failure, it introduced a new tree (called 'missing tree') protected with a lock. Threads are added only if they're not found in the normal (current + dead) tree.
Quoted 3 lines
I.e.
machine__remove_thread()becomes a noop for such use cases and we never delete a struct thread instance?
Right. I just converted to keep removed threads in a rbtree as they can be quite large in a long session.
Thanks,
Namhyung
Quoted 7 lines
- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Namhyung Kim
maintainerPATCH 13/38 perf tools: Add a test case for thread comm handling
In reply to Namhyung KimThe new test case checks various thread comm handling like overridding and time sorting.
--- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 4 ++++ tools/perf/tests/tests.h | 1 + tools/perf/tests/thread-comm.c | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 53 insertions(+) create mode 100644 tools/perf/tests/thread-comm.c
tools/perf/tests/Build +1 -0
@@ -24,6 +24,7 @@ perf-y += bp_signal_overflow.o perf-y += task-exit.o perf-y += sw-clock.o perf-y += mmap-thread-lookup.o +perf-y += thread-comm.o perf-y += thread-mg-share.o perf-y += switch-tracking.o perf-y += keep-tracking.o
tools/perf/tests/builtin-test.c +4 -0
@@ -167,6 +167,10 @@ static struct test { .func = test__fdarray__add, }, { + .desc = "Test thread comm handling", + .func = test__thread_comm, + }, + { .func = NULL, }, };
tools/perf/tests/tests.h +1 -0
@@ -51,6 +51,7 @@ int test__hists_cumulate(void);int test__switch_tracking (void );int test__fdarray__filter (void );int test__fdarray__add (void ); +int test__thread_comm (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/tests/thread-comm.c new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 14/38 perf tools: Use thread__comm_time() when adding hist entries
In reply to Namhyung KimNow thread->comm can be handled with time properly, use it to find correct comm when adding hist entries.
--- tools/perf/builtin-annotate.c | 5 +++-- tools/perf/builtin-diff.c | 8 ++++---- tools/perf/tests/hists_link.c | 4 ++-- tools/perf/util/hist.c | 19 ++++++++++--------- tools/perf/util/hist.h | 2 +- 5 files changed, 20 insertions(+), 18 deletions(-)
tools/perf/builtin-annotate.c +3 -2
@@ -47,7 +47,7 @@ struct perf_annotate { };static int perf_evsel__add_sample (struct perf_evsel *evsel, -struct perf_sample *sample__maybe_unused , +struct perf_sample *sample,struct addr_location *al,struct perf_annotate *ann) {
@@ -67,7 +67,8 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,return 0 ; } - he =__hists__add_entry (hists, al,NULL ,NULL ,NULL ,1 ,1 ,0 , true); + he =__hists__add_entry (hists, al,NULL ,NULL ,NULL ,1 ,1 ,0 , + sample->time, true);if (he ==NULL )return -ENOMEM ;
tools/perf/builtin-diff.c +4 -4
@@ -313,10 +313,10 @@ static int formula_fprintf(struct hist_entry *he, struct hist_entry *pair,static int hists__add_entry (struct hists *hists,struct addr_location *al,u64 period, -u64 weight,u64 transaction) +u64 weight,u64 transaction,u64 timestamp) {if (__hists__add_entry (hists, al,NULL ,NULL ,NULL , period, weight, - transaction, true) !=NULL ) + transaction, timestamp, true) !=NULL )return 0 ;return -ENOMEM ; }
@@ -338,8 +338,8 @@ static int diff__process_sample_event(struct perf_tool *tool,return -1 ; } -if (hists__add_entry (hists, &al, sample->period, - sample->weight, sample->transaction)) { +if (hists__add_entry (hists, &al, sample->period, sample->weight, + sample->transaction, sample->time)) {pr_warning ("problem incrementing symbol period, skipping event\n" );return -1 ; }
tools/perf/tests/hists_link.c +2 -2
@@ -90,7 +90,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) goto out; he =__hists__add_entry (hists, &al,NULL , -NULL ,NULL ,1 ,1 ,0 , true); +NULL ,NULL ,1 ,1 ,0 ,-1 , true);if (he ==NULL ) goto out;
@@ -114,7 +114,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine) goto out; he =__hists__add_entry (hists, &al,NULL , -NULL ,NULL ,1 ,1 ,0 , true); +NULL ,NULL ,1 ,1 ,0 ,-1 , true);if (he ==NULL ) goto out;
tools/perf/util/hist.c +10 -9
@@ -447,11 +447,11 @@ struct hist_entry *__hists__add_entry(struct hists *hists, struct branch_info *bi, struct mem_info *mi, u64 period, u64 weight, u64 transaction, - bool sample_self) + u64 timestamp, bool sample_self) { struct hist_entry entry = { .thread = al->thread, - .comm = thread__comm(al->thread), + .comm = thread__comm_time(al->thread, timestamp), .ms = { .map = al->map, .sym = al->sym,
@@ -509,13 +509,14 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *al {u64 cost;struct mem_info *mi = iter->priv; +struct perf_sample *sample = iter->sample;struct hists *hists =evsel__hists (iter->evsel);struct hist_entry *he;if (mi ==NULL )return -EINVAL ; - cost = iter->sample->weight; + cost = sample->weight;if (!cost) cost =1 ;
@@ -527,7 +528,7 @@ iter_add_single_mem_entry(struct hist_entry_iter *iter, struct addr_location *al* and the he_stat__add_period() function. */ he =__hists__add_entry (hists, al, iter->parent,NULL , mi, - cost, cost,0 , true); + cost, cost,0 , sample->time, true);if (!he)return -ENOMEM ;
@@ -628,7 +629,7 @@ iter_add_next_branch_entry(struct hist_entry_iter *iter, struct addr_location *a* and not events sampled. Thus we use a pseudo period of 1. */ he =__hists__add_entry (hists, al, iter->parent, &bi[i],NULL , -1 ,1 ,0 , true); +1 ,1 ,0 , iter->sample->time, true);if (he ==NULL )return -ENOMEM ;
@@ -666,7 +667,7 @@ iter_add_single_normal_entry(struct hist_entry_iter *iter, struct addr_location he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , sample->period, sample->weight, - sample->transaction, true); + sample->transaction, sample->time, true);if (he ==NULL )return -ENOMEM ;
@@ -728,7 +729,7 @@ iter_add_single_cumulative_entry(struct hist_entry_iter *iter, he =__hists__add_entry (hists, al, iter->parent,NULL ,NULL , sample->period, sample->weight, - sample->transaction, true); + sample->transaction, sample->time, true);if (he ==NULL )return -ENOMEM ;
@@ -772,7 +773,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter,struct hist_entry he_tmp = { .cpu = al->cpu, .thread = al->thread, - .comm =thread__comm (al->thread), + .comm =thread__comm_time (al->thread, sample->time), .ip = al->addr, .ms = { .map = al->map,
@@ -801,7 +802,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter, he =__hists__add_entry (evsel__hists (evsel), al, iter->parent,NULL ,NULL , sample->period, sample->weight, - sample->transaction, false); + sample->transaction, sample->time, false);if (he ==NULL )return -ENOMEM ;
tools/perf/util/hist.h +1 -1
@@ -109,7 +109,7 @@ struct hist_entry *__hists__add_entry(struct hists *hists,struct branch_info *bi, struct mem_info *mi, u64 period, u64 weight, u64 transaction, - bool sample_self); + u64 timestamp, bool sample_self);int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al,struct perf_evsel *evsel,struct perf_sample *sample,int max_stack_depth,void *arg);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 15/38 perf tools: Convert dead thread list into rbtree
In reply to Namhyung KimCurrently perf maintains dead threads in a linked list but this can be a problem if someone needs to search from it especially in a large session which might have many dead threads. Convert it to a rbtree like normal threads and it'll be used later with multi-file changes.
The list node is now used for chaining dead threads of same tid since it's easier to handle such threads in time order.
--- tools/perf/util/machine.c | 70 +++++++++++++++++++++++++++++++++++++++++------ tools/perf/util/machine.h | 2 +- tools/perf/util/thread.c | 1 + tools/perf/util/thread.h | 11 ++++---- 4 files changed, 68 insertions(+), 16 deletions(-)
tools/perf/util/machine.c +61 -9
@@ -28,7 +28,7 @@ int machine__init(struct machine *machine, const char *root_dir, pid_t pid)dsos__init (&machine->kernel_dsos); machine->threads =RB_ROOT ; -INIT_LIST_HEAD (&machine->dead_threads); + machine->dead_threads =RB_ROOT ; machine->last_match =NULL ; machine->vdso_info =NULL ;
@@ -91,10 +91,22 @@ static void dsos__delete(struct dsos *dsos)void machine__delete_dead_threads (struct machine *machine) { -struct thread *n, *t; +struct rb_node *nd =rb_first (&machine->dead_threads); + +while (nd) { + struct thread *t = rb_entry(nd, struct thread, rb_node); +struct thread *pos; + + nd =rb_next (nd); +rb_erase (&t->rb_node, &machine->dead_threads); + +while (!list_empty (&t->tid_node)) { + pos =list_first_entry (&t->tid_node, + struct thread, tid_node); +list_del (&pos->tid_node); +thread__delete (pos); + } -list_for_each_entry_safe (t, n, &machine->dead_threads, node) { -list_del (&t->node);thread__delete (t); } }
@@ -106,8 +118,8 @@ void machine__delete_threads(struct machine *machine)while (nd) { struct thread *t = rb_entry(nd, struct thread, rb_node); -rb_erase (&t->rb_node, &machine->threads); nd =rb_next (nd); +rb_erase (&t->rb_node, &machine->threads);thread__delete (t); } }
@@ -1238,13 +1250,46 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *eventstatic void machine__remove_thread (struct machine *machine,struct thread *th) { +struct rb_node **p = &machine->dead_threads.rb_node; +struct rb_node *parent =NULL ; +struct thread *pos; + machine->last_match =NULL ;rb_erase (&th->rb_node, &machine->threads); + + th->dead = true; +/* * We may have references to this thread, for instance in some hist_entry -* instances, so just move them to a separate list. +* instances, so just move them to a separate list in rbtree. */ -list_add_tail (&th->node, &machine->dead_threads); +while (*p !=NULL ) { + parent = *p; + pos = rb_entry(parent, struct thread, rb_node); + +if (pos->tid == th->tid) { +struct thread *old; + +/* sort by time */ +list_for_each_entry (old, &pos->tid_node, tid_node) { +if (th->start_time >= old->start_time) { + pos = old; +break ; + } + } + +list_add_tail (&th->tid_node, &pos->tid_node); +return ; + } + +if (th->tid < pos->tid) + p = &(*p)->rb_left; +else + p = &(*p)->rb_right; + } + +rb_link_node (&th->rb_node, parent, p); +rb_insert_color (&th->rb_node, &machine->dead_threads); } int machine__process_fork_event(struct machine *machine, union perf_event *event,
@@ -1729,7 +1774,7 @@ int machine__for_each_thread(struct machine *machine, void *priv) {struct rb_node *nd; -struct thread *thread; +struct thread *thread, *pos;int rc =0 ;for (nd =rb_first (&machine->threads); nd; nd =rb_next (nd)) {
@@ -1739,10 +1784,17 @@ int machine__for_each_thread(struct machine *machine,return rc; } -list_for_each_entry (thread, &machine->dead_threads, node) { +for (nd =rb_first (&machine->dead_threads); nd; nd =rb_next (nd)) { + thread = rb_entry(nd, struct thread, rb_node); rc =fn (thread, priv);if (rc !=0 )return rc; + +list_for_each_entry (pos, &thread->tid_node, tid_node) { + rc =fn (pos, priv); +if (rc !=0 ) +return rc; + } }return rc; }
tools/perf/util/machine.h +1 -1
@@ -30,7 +30,7 @@ struct machine {bool comm_exec;char *root_dir;struct rb_root threads; -struct list_head dead_threads; +struct rb_root dead_threads;struct thread *last_match;struct vdso_info *vdso_info;struct dsos user_dsos;
tools/perf/util/thread.c +1 -0
@@ -38,6 +38,7 @@ struct thread *thread__new(pid_t pid, pid_t tid) thread->ppid =-1 ; thread->cpu =-1 ;INIT_LIST_HEAD (&thread->comm_list); +INIT_LIST_HEAD (&thread->tid_node);if (unwind__prepare_access (thread) <0 ) goto err_thread;
tools/perf/util/thread.h +5 -6
@@ -11,10 +11,8 @@struct thread_stack ;struct thread { -union { -struct rb_node rb_node; -struct list_head node; - }; +struct rb_node rb_node; +struct list_head tid_node;struct map_groups *mg;pid_t pid_;/* Not all tools update this */ pid_t tid;
@@ -22,7 +20,8 @@ struct thread {int cpu;char shortname[3 ];bool comm_set; -bool dead;/* if set thread has exited */ +bool exited;/* if set thread has exited */ +bool dead;/* thread is in dead_threads list */ struct list_head comm_list;int comm_len;u64 db_id;
@@ -39,7 +38,7 @@ int thread__init_map_groups(struct thread *thread, struct machine *machine);void thread__delete (struct thread *thread);static inline void thread__exited (struct thread *thread) { - thread->dead = true; + thread->exited = true; } int __thread__set_comm(struct thread *thread, const char *comm, u64 timestamp,
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 16/38 perf tools: Introduce machine__find*_thread_time()
In reply to Namhyung KimWith data file indexing is enabled, it needs to search thread based on sample time since sample processing is done after other (task, comm and mmap) events are processed. This can be a problem if a session is very long and pid is recycled - in that case it'll only see the last one.
So keep thread start time in it, and search thread based on the time. This patch introduces machine__find{,new}_thread_time() function for this. It'll first search current thread rbtree and then dead thread tree and list. If it couldn't find anyone, it'll create a new thread.
The sample timestamp of 0 means that this is called from synthesized event so just use current rbtree. The timestamp will be -1 if sample didn't record the timestamp so will see current threads automatically.
--- tools/perf/builtin-script.c | 11 ++++- tools/perf/tests/dwarf-unwind.c | 8 ++-- tools/perf/tests/hists_common.c | 3 +- tools/perf/tests/hists_link.c | 2 +- tools/perf/util/event.c | 14 ++++-- tools/perf/util/machine.c | 102 +++++++++++++++++++++++++++++++++++++++- tools/perf/util/machine.h | 8 +++- tools/perf/util/thread.c | 4 ++ tools/perf/util/thread.h | 1 + 9 files changed, 138 insertions(+), 15 deletions(-)
tools/perf/builtin-script.c +9 -2
@@ -559,8 +559,15 @@ static int process_sample_event(struct perf_tool *tool, {struct addr_location al; struct perf_script *script = container_of(tool, struct perf_script, tool); -struct thread *thread =machine__findnew_thread (machine, sample->pid, - sample->tid); +struct thread *thread; + +if (perf_session__has_index (script->session)) + thread =machine__findnew_thread_time (machine, sample->pid, + sample->tid, + sample->time); +else + thread =machine__findnew_thread (machine, sample->pid, + sample->tid);if (thread ==NULL ) { pr_debug("problem processing %d event, skipping it.\n",
tools/perf/tests/dwarf-unwind.c +3 -5
@@ -16,10 +16,10 @@static int mmap_handler (struct perf_tool *tool__maybe_unused ,union perf_event *event, -struct perf_sample *sample__maybe_unused , +struct perf_sample *sample,struct machine *machine) { -return machine__process_mmap2_event (machine, event,NULL ); +return machine__process_mmap2_event (machine, event, sample); }static int init_live_machine (struct machine *machine)
@@ -66,12 +66,10 @@ static int unwind_entry(struct unwind_entry *entry, void *arg) __attribute__ ((noinline))static int unwind_thread (struct thread *thread) { -struct perf_sample sample; +struct perf_sample sample = { .time =-1ULL , };unsigned long cnt =0 ;int err =-1 ; -memset (&sample,0 ,sizeof (sample)); -if (test__arch_unwind_sample (&sample, thread)) {pr_debug ("failed to get unwind sample\n" ); goto out;
tools/perf/tests/hists_common.c +2 -1
@@ -80,6 +80,7 @@ static struct {struct machine *setup_fake_machine (struct machines *machines) {struct machine *machine =machines__find (machines,HOST_KERNEL_ID ); +struct perf_sample sample = { .time =-1ULL , };size_t i;if (machine ==NULL ) {
@@ -113,7 +114,7 @@ struct machine *setup_fake_machine(struct machines *machines)strcpy (fake_mmap_event.mmap.filename, fake_mmap_info[i].filename); -machine__process_mmap_event (machine, &fake_mmap_event,NULL ); +machine__process_mmap_event (machine, &fake_mmap_event, &sample); }for (i =0 ; i <ARRAY_SIZE (fake_symbols); i++) {
tools/perf/tests/hists_link.c +1 -1
@@ -64,7 +64,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)struct perf_evsel *evsel;struct addr_location al;struct hist_entry *he; -struct perf_sample sample = { .period =1 , }; +struct perf_sample sample = { .period =1 , .time =-1ULL , };size_t i =0 , k;/*
tools/perf/util/event.c +11 -3
@@ -9,6 +9,7 @@#include "strlist.h" #include "thread.h" #include "thread_map.h" +#include "session.h" #include "symbol/kallsyms.h" static const char *perf_event__names[] = {
@@ -837,11 +838,18 @@ int perf_event__preprocess_sample(const union perf_event *event,struct machine *machine, struct addr_location *al, struct perf_sample *sample, - struct perf_session *session __maybe_unused) + struct perf_session *session) { u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK; -struct thread *thread =machine__findnew_thread (machine, sample->pid, - sample->tid); +struct thread *thread; + +if (session &&perf_session__has_index (session)) + thread =machine__findnew_thread_time (machine, sample->pid, + sample->tid, + sample->time); +else + thread =machine__findnew_thread (machine, sample->pid, + sample->tid);if (thread ==NULL )return -1 ;
tools/perf/util/machine.c +101 -1
114 lines not on this page · raw message
tools/perf/util/machine.h +6 -2
@@ -68,8 +68,6 @@ static inline bool machine__kernel_ip(struct machine *machine, u64 ip)return ip >= kernel_start; } -struct thread *machine__find_thread (struct machine *machine,pid_t pid, -pid_t tid);struct comm *machine__thread_exec_comm (struct machine *machine,struct thread *thread);
@@ -149,6 +147,12 @@ static inline bool machine__is_host(struct machine *machine)struct thread *machine__findnew_thread (struct machine *machine,pid_t pid,pid_t tid); +struct thread *machine__find_thread (struct machine *machine,pid_t pid, +pid_t tid); +struct thread *machine__findnew_thread_time (struct machine *machine,pid_t pid, +pid_t tid,u64 timestamp); +struct thread *machine__find_thread_time (struct machine *machine,pid_t pid, +pid_t tid,u64 timestamp);size_t machine__fprintf (struct machine *machine,FILE *fp);
tools/perf/util/thread.c +4 -0
@@ -127,6 +127,9 @@ int __thread__set_comm(struct thread *thread, const char *str, u64 timestamp,/* Override the default :tid entry */ if (!thread->comm_set) { +if (!thread->start_time) + thread->start_time = timestamp; + err =comm__override (curr, str, timestamp, exec);if (err)return err;
@@ -228,6 +231,7 @@ int thread__fork(struct thread *thread, struct thread *parent, u64 timestamp) } thread->ppid = parent->tid; + thread->start_time = timestamp;return thread__clone_map_groups (thread, parent); }
tools/perf/util/thread.h +1 -0
@@ -25,6 +25,7 @@ struct thread {struct list_head comm_list;int comm_len;u64 db_id; +u64 start_time;void *priv;struct thread_stack *ts;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 17/38 perf tools: Add a test case for timed thread handling
In reply to Namhyung KimA test case for verifying live and dead thread tree management during time change and new machine__find{,new}_thread_time().
--- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 4 + tools/perf/tests/tests.h | 1 + tools/perf/tests/thread-lookup-time.c | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 180 insertions(+) create mode 100644 tools/perf/tests/thread-lookup-time.c
tools/perf/tests/Build +1 -0
@@ -26,6 +26,7 @@ perf-y += sw-clock.o perf-y += mmap-thread-lookup.o perf-y += thread-comm.o perf-y += thread-mg-share.o +perf-y += thread-lookup-time.o perf-y += switch-tracking.o perf-y += keep-tracking.o perf-y += code-reading.o
tools/perf/tests/builtin-test.c +4 -0
@@ -171,6 +171,10 @@ static struct test { .func = test__thread_comm, }, { + .desc = "Test thread lookup with time", + .func = test__thread_lookup_time, + }, + { .func = NULL, }, };
tools/perf/tests/tests.h +1 -0
@@ -52,6 +52,7 @@ int test__switch_tracking(void);int test__fdarray__filter (void );int test__fdarray__add (void );int test__thread_comm (void ); +int test__thread_lookup_time (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
tools/perf/tests/thread-lookup-time.c new file +174 -0
174 lines not on this page · raw message
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 18/38 perf tools: Reducing arguments of hist_entry_iter__add()
In reply to Namhyung KimThe evsel and sample arguments are to set iter for later use. As it also receives an iter as another argument, just set them before calling the function.
--- tools/perf/builtin-report.c | 9 +++++---- tools/perf/builtin-top.c | 7 ++++--- tools/perf/tests/hists_cumulate.c | 6 ++++-- tools/perf/tests/hists_filter.c | 4 +++- tools/perf/tests/hists_output.c | 6 ++++-- tools/perf/util/hist.c | 8 ++------ tools/perf/util/hist.h | 1 - 7 files changed, 22 insertions(+), 19 deletions(-)
tools/perf/builtin-report.c +5 -4
@@ -137,8 +137,10 @@ static int process_sample_event(struct perf_tool *tool, struct report *rep = container_of(tool, struct report, tool);struct addr_location al;struct hist_entry_iter iter = { - .hide_unresolved = rep->hide_unresolved, - .add_entry_cb = hist_iter__report_callback, + .evsel = evsel, + .sample = sample, + .hide_unresolved = rep->hide_unresolved, + .add_entry_cb = hist_iter__report_callback, };int ret;
@@ -167,8 +169,7 @@ static int process_sample_event(struct perf_tool *tool,if (al.map !=NULL ) al.map->dso->hit =1 ; - ret =hist_entry_iter__add (&iter, &al, evsel, sample, rep->max_stack, - rep); + ret =hist_entry_iter__add (&iter, &al, rep->max_stack, rep);if (ret <0 )pr_debug ("problem adding hist entry, skipping event\n" );
tools/perf/builtin-top.c +4 -3
@@ -774,7 +774,9 @@ static void perf_event__process_sample(struct perf_tool *tool,if (al.sym ==NULL || !al.sym->ignore) {struct hists *hists =evsel__hists (evsel);struct hist_entry_iter iter = { - .add_entry_cb = hist_iter__top_callback, + .evsel = evsel, + .sample = sample, + .add_entry_cb = hist_iter__top_callback, };if (symbol_conf.cumulate_callchain)
@@ -784,8 +786,7 @@ static void perf_event__process_sample(struct perf_tool *tool,pthread_mutex_lock (&hists->lock); - err =hist_entry_iter__add (&iter, &al, evsel, sample, - top->max_stack, top); + err =hist_entry_iter__add (&iter, &al, top->max_stack, top);if (err <0 )pr_err ("Problem incrementing symbol period, skipping event\n" );
tools/perf/tests/hists_cumulate.c +4 -2
@@ -87,6 +87,8 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) }, };struct hist_entry_iter iter = { + .evsel = evsel, + .sample = &sample, .hide_unresolved = false, };
@@ -104,8 +106,8 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) &sample, NULL) < 0) goto out; -if (hist_entry_iter__add (&iter, &al, evsel, &sample, -PERF_MAX_STACK_DEPTH ,NULL ) <0 ) +if (hist_entry_iter__add (&iter, &al,PERF_MAX_STACK_DEPTH , +NULL ) <0 ) goto out; fake_samples[i].thread = al.thread;
tools/perf/tests/hists_filter.c +3 -1
@@ -63,6 +63,8 @@ static int add_hist_entries(struct perf_evlist *evlist, }, };struct hist_entry_iter iter = { + .evsel = evsel, + .sample = &sample, .ops = &hist_iter_normal, .hide_unresolved = false, };
@@ -81,7 +83,7 @@ static int add_hist_entries(struct perf_evlist *evlist, &sample, NULL) < 0) goto out; -if (hist_entry_iter__add (&iter, &al, evsel, &sample, +if (hist_entry_iter__add (&iter, &al,PERF_MAX_STACK_DEPTH ,NULL ) <0 ) goto out;
tools/perf/tests/hists_output.c +4 -2
@@ -57,6 +57,8 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) }, };struct hist_entry_iter iter = { + .evsel = evsel, + .sample = &sample, .ops = &hist_iter_normal, .hide_unresolved = false, };
@@ -70,8 +72,8 @@ static int add_hist_entries(struct hists *hists, struct machine *machine) &sample, NULL) < 0) goto out; -if (hist_entry_iter__add (&iter, &al, evsel, &sample, -PERF_MAX_STACK_DEPTH ,NULL ) <0 ) +if (hist_entry_iter__add (&iter, &al,PERF_MAX_STACK_DEPTH , +NULL ) <0 ) goto out; fake_samples[i].thread = al.thread;
tools/perf/util/hist.c +2 -6
@@ -857,19 +857,15 @@ const struct hist_iter_ops hist_iter_cumulative = { };int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al, -struct perf_evsel *evsel,struct perf_sample *sample,int max_stack_depth,void *arg) {int err, err2; - err =sample__resolve_callchain (sample, &iter->parent, evsel, al, - max_stack_depth); + err =sample__resolve_callchain (iter->sample, &iter->parent, + iter->evsel, al, max_stack_depth);if (err)return err; - iter->evsel = evsel; - iter->sample = sample; - err = iter->ops->prepare_entry (iter, al);if (err) goto out;
tools/perf/util/hist.h +0 -1
@@ -111,7 +111,6 @@ struct hist_entry *__hists__add_entry(struct hists *hists, u64 weight, u64 transaction, u64 timestamp, bool sample_self);int hist_entry_iter__add (struct hist_entry_iter *iter,struct addr_location *al, -struct perf_evsel *evsel,struct perf_sample *sample,int max_stack_depth,void *arg);int64_t hist_entry__cmp (struct hist_entry *left,struct hist_entry *right);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 19/38 perf tools: Pass session to hist_entry_iter struct
In reply to Namhyung KimThe session is necessary to determine whether this is an indexed data so that it needs to use timestamp for searching threads/symbols.
--- tools/perf/builtin-report.c | 1 + tools/perf/builtin-top.c | 1 + tools/perf/util/hist.h | 1 + 3 files changed, 3 insertions(+)
tools/perf/builtin-report.c +1 -0
@@ -139,6 +139,7 @@ static int process_sample_event(struct perf_tool *tool,struct hist_entry_iter iter = { .evsel = evsel, .sample = sample, + .session = rep->session, .hide_unresolved = rep->hide_unresolved, .add_entry_cb = hist_iter__report_callback, };
tools/perf/builtin-top.c +1 -0
@@ -776,6 +776,7 @@ static void perf_event__process_sample(struct perf_tool *tool,struct hist_entry_iter iter = { .evsel = evsel, .sample = sample, + .session = top->session, .add_entry_cb = hist_iter__top_callback, };
tools/perf/util/hist.h +1 -0
@@ -88,6 +88,7 @@ struct hist_entry_iter {struct perf_evsel *evsel;struct perf_sample *sample; +struct perf_session *session;struct hist_entry *he;struct symbol *parent;void *priv;
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 20/38 perf tools: Maintain map groups list in a leader thread
In reply to Namhyung KimTo support multi-threaded perf report, we need to maintain time-sorted map groups. Add ->mg_list member to struct thread and sort the list by time. Now leader threads have one more refcnt for map groups in the list so also update the thread-mg-share test case.
Currently only add a new map groups when an exec (comm) event is received.
--- tools/perf/tests/thread-mg-share.c | 7 ++- tools/perf/util/event.c | 2 + tools/perf/util/machine.c | 4 +- tools/perf/util/map.c | 3 ++ tools/perf/util/map.h | 2 + tools/perf/util/thread.c | 108 ++++++++++++++++++++++++++++++++++++- tools/perf/util/thread.h | 3 ++ 7 files changed, 124 insertions(+), 5 deletions(-)
tools/perf/tests/thread-mg-share.c +5 -2
@@ -23,6 +23,9 @@ int test__thread_mg_share(void)* with several threads and checks they properly share and * maintain map groups info (struct map_groups). * +* Note that a leader thread has one more refcnt for its +* (current) map groups. +* * thread group (pid: 0, tids: 0, 1, 2, 3) * other group (pid: 4, tids: 4, 5) */
@@ -43,7 +46,7 @@ int test__thread_mg_share(void) leader && t1 && t2 && t3 && other); mg = leader->mg; -TEST_ASSERT_VAL ("wrong refcnt" , mg->refcnt ==4 ); +TEST_ASSERT_VAL ("wrong refcnt" , mg->refcnt ==5 );/* test the map groups pointer is shared */ TEST_ASSERT_VAL ("map groups don't match" , mg == t1->mg);
@@ -59,7 +62,7 @@ int test__thread_mg_share(void)TEST_ASSERT_VAL ("failed to find other leader" , other_leader); other_mg = other->mg; -TEST_ASSERT_VAL ("wrong refcnt" , other_mg->refcnt ==2 ); +TEST_ASSERT_VAL ("wrong refcnt" , other_mg->refcnt ==3 );TEST_ASSERT_VAL ("map groups don't match" , other_mg == other_leader->mg);
tools/perf/util/event.c +2 -0
@@ -765,6 +765,8 @@ void thread__find_addr_map(struct thread *thread, u8 cpumode,return ; } +BUG_ON (mg ==NULL ); +if (cpumode ==PERF_RECORD_MISC_KERNEL && perf_host) { al->level ='k' ; mg = &machine->kmaps;
tools/perf/util/machine.c +2 -2
@@ -331,7 +331,7 @@ static void machine__update_thread_pid(struct machine *machine, goto out_err;if (!leader->mg) - leader->mg =map_groups__new (machine); +thread__set_map_groups (leader,map_groups__new (machine),0 );if (!leader->mg) goto out_err;
@@ -348,7 +348,7 @@ static void machine__update_thread_pid(struct machine *machine,if (!map_groups__empty (th->mg))pr_err ("Discarding thread maps for %d:%d\n" , th->pid_, th->tid); -map_groups__delete (th->mg); +map_groups__put (th->mg); } th->mg =map_groups__get (leader->mg);
tools/perf/util/map.c +3 -0
@@ -422,6 +422,8 @@ void map_groups__init(struct map_groups *mg, struct machine *machine) } mg->machine = machine; mg->refcnt =1 ; + mg->timestamp =0 ; +INIT_LIST_HEAD (&mg->list); }static void maps__delete (struct rb_root *maps)
@@ -484,6 +486,7 @@ struct map_groups *map_groups__new(struct machine *machine)void map_groups__delete (struct map_groups *mg) {map_groups__exit (mg); +list_del (&mg->list);free (mg); }
tools/perf/util/map.h +2 -0
@@ -61,7 +61,9 @@ struct map_groups {struct rb_root maps[MAP__NR_TYPES ];struct list_head removed_maps[MAP__NR_TYPES ];struct machine *machine; +u64 timestamp;int refcnt; +struct list_head list; };struct map_groups *map_groups__new (struct machine *machine);
tools/perf/util/thread.c +107 -1
150 lines not on this page · raw message
tools/perf/util/thread.h +3 -0
@@ -14,6 +14,7 @@ struct thread {struct rb_node rb_node;struct list_head tid_node;struct map_groups *mg; +struct list_head mg_list;pid_t pid_;/* Not all tools update this */ pid_t tid;pid_t ppid;
@@ -56,6 +57,8 @@ struct comm *thread__exec_comm(const struct thread *thread);struct comm *thread__comm_time (const struct thread *thread,u64 timestamp);const char *thread__comm_str (const struct thread *thread);const char *thread__comm_str_time (const struct thread *thread,u64 timestamp); +struct map_groups *thread__get_map_groups (struct thread *thread,u64 timestamp); +int thread__set_map_groups (struct thread *thread,struct map_groups *mg,u64 timestamp);void thread__insert_map (struct thread *thread,struct map *map);int thread__fork (struct thread *thread,struct thread *parent,u64 timestamp);size_t thread__fprintf (struct thread *thread,FILE *fp);
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 21/38 perf tools: Introduce session__find_addr_location() and friends
In reply to Namhyung KimThese new functions are for find appropriate map (and symbol) at the given time when used with an indexed data file. This is based on the fact that map_groups list is sorted by time in the previous patch.
--- tools/perf/util/event.c | 57 ++++++++++++++++++++++++++++++++++++++--------- tools/perf/util/hist.c | 4 ++-- tools/perf/util/machine.c | 37 ++++++++++++++++++++---------- tools/perf/util/machine.h | 2 ++ tools/perf/util/session.h | 38 +++++++++++++++++++++++++++++++ tools/perf/util/thread.c | 21 +++++++++++++++++ tools/perf/util/thread.h | 10 +++++++++ 7 files changed, 145 insertions(+), 24 deletions(-)
tools/perf/util/event.c +47 -10
@@ -746,16 +746,14 @@ int perf_event__process(struct perf_tool *tool __maybe_unused,return machine__process_event (machine, event, sample); } -void thread__find_addr_map (struct thread *thread,u8 cpumode, -enum map_type type,u64 addr, -struct addr_location *al) +static void map_groups__find_addr_map (struct map_groups *mg,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al) { -struct map_groups *mg = thread->mg;struct machine *machine = mg->machine;bool load_map = false; al->machine = machine; - al->thread = thread; al->addr = addr; al->cpumode = cpumode; al->filtered =0 ;
@@ -824,6 +822,26 @@ void thread__find_addr_map(struct thread *thread, u8 cpumode, } } +void thread__find_addr_map (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al) +{ + al->thread = thread; +map_groups__find_addr_map (thread->mg, cpumode, type, addr, al); +} + +void thread__find_addr_map_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp) +{ +struct map_groups *mg; + + mg =thread__get_map_groups (thread, timestamp); + + al->thread = thread; +map_groups__find_addr_map (mg, cpumode, type, addr, al); +} +void thread__find_addr_location (struct thread *thread,u8 cpumode,enum map_type type,u64 addr,struct addr_location *al)
@@ -836,6 +854,21 @@ void thread__find_addr_location(struct thread *thread, al->sym =NULL ; } +void thread__find_addr_location_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp) +{ +struct map_groups *mg; + + mg =thread__get_map_groups (thread, timestamp); +map_groups__find_addr_map (mg, cpumode, type, addr, al); +if (al->map !=NULL ) + al->sym =map__find_symbol (al->map, al->addr, + mg->machine->symbol_filter); +else + al->sym =NULL ; +} + int perf_event__preprocess_sample(const union perf_event *event, struct machine *machine, struct addr_location *al,
@@ -868,7 +901,9 @@ int perf_event__preprocess_sample(const union perf_event *event, machine->vmlinux_maps[MAP__FUNCTION] == NULL)machine__create_kernel_maps (machine); -thread__find_addr_map (thread, cpumode,MAP__FUNCTION , sample->ip, al); +session__find_addr_map (session, thread, cpumode,MAP__FUNCTION , + sample->ip, al, sample->time); +dump_printf (" ...... dso: %s\n" , al->map ? al->map->dso->long_name : al->level =='H' ?"[hypervisor]" :"<not found>" );
@@ -929,14 +964,16 @@ void perf_event__preprocess_sample_addr(union perf_event *event,struct perf_sample *sample, struct thread *thread, struct addr_location *al, - struct perf_session *session __maybe_unused) + struct perf_session *session) { u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK; -thread__find_addr_map (thread, cpumode,MAP__FUNCTION , sample->addr, al); +session__find_addr_map (session, thread, cpumode,MAP__FUNCTION , + sample->addr, al, sample->time); +if (!al->map) -thread__find_addr_map (thread, cpumode,MAP__VARIABLE , - sample->addr, al); +session__find_addr_map (session, thread, cpumode,MAP__VARIABLE , + sample->addr, al, sample->time); al->cpu = sample->cpu; al->sym =NULL ;
tools/perf/util/hist.c +2 -2
@@ -496,7 +496,7 @@ iter_prepare_mem_entry(struct hist_entry_iter *iter, struct addr_location *al)struct perf_sample *sample = iter->sample;struct mem_info *mi; - mi =sample__resolve_mem (sample, al); + mi =sample__resolve_mem (sample, iter->session, al);if (mi ==NULL )return -ENOMEM ;
@@ -570,7 +570,7 @@ iter_prepare_branch_entry(struct hist_entry_iter *iter, struct addr_location *alstruct branch_info *bi;struct perf_sample *sample = iter->sample; - bi =sample__resolve_bstack (sample, al); + bi =sample__resolve_bstack (sample, iter->session, al);if (!bi)return -ENOMEM ;
tools/perf/util/machine.c +25 -12
@@ -8,6 +8,7 @@#include "sort.h" #include "strlist.h" #include "thread.h" +#include "session.h" #include "vdso.h" #include <stdbool.h> #include <symbol/kallsyms.h>
@@ -1469,9 +1470,10 @@ static bool symbol__match_regex(struct symbol *sym, regex_t *regex)return 0 ; } -static void ip__resolve_ams (struct thread *thread, +static void ip__resolve_ams (struct perf_session *session, +struct thread *thread,struct addr_map_symbol *ams, -u64 ip) +u64 ip,u64 timestamp) {struct addr_location al;
@@ -1483,7 +1485,8 @@ static void ip__resolve_ams(struct thread *thread,* Thus, we have to try consecutively until we find a match * or else, the symbol is unknown */ -thread__find_cpumode_addr_location (thread,MAP__FUNCTION , ip, &al); +session__find_cpumode_addr_location (session, thread,MAP__FUNCTION , + ip, &al, timestamp); ams->addr = ip; ams->al_addr = al.addr;
@@ -1491,21 +1494,25 @@ static void ip__resolve_ams(struct thread *thread, ams->map = al.map; } -static void ip__resolve_data (struct thread *thread, -u8 m,struct addr_map_symbol *ams,u64 addr) +static void ip__resolve_data (struct perf_session *session, +struct thread *thread,u8 m, +struct addr_map_symbol *ams, +u64 addr,u64 timestamp) {struct addr_location al;memset (&al,0 ,sizeof (al)); -thread__find_addr_location (thread, m,MAP__VARIABLE , addr, &al); +session__find_addr_location (session, thread, m,MAP__VARIABLE , addr, + &al, timestamp);if (al.map ==NULL ) {/* * some shared data regions have execute bit set which puts * their mapping in the MAP__FUNCTION type array. * Check there as a fallback option before dropping the sample. */ -thread__find_addr_location (thread, m,MAP__FUNCTION , addr, &al); +session__find_addr_location (session, thread, m,MAP__FUNCTION , + addr, &al, timestamp); } ams->addr = addr;
@@ -1515,6 +1522,7 @@ static void ip__resolve_data(struct thread *thread, }struct mem_info *sample__resolve_mem (struct perf_sample *sample, +struct perf_session *session,struct addr_location *al) {struct mem_info *mi =zalloc (sizeof (*mi));
@@ -1522,8 +1530,10 @@ struct mem_info *sample__resolve_mem(struct perf_sample *sample,if (!mi)return NULL ; -ip__resolve_ams (al->thread, &mi->iaddr, sample->ip); -ip__resolve_data (al->thread, al->cpumode, &mi->daddr, sample->addr); +ip__resolve_ams (session, al->thread, &mi->iaddr, sample->ip, + sample->time); +ip__resolve_data (session, al->thread, al->cpumode, &mi->daddr, + sample->addr, sample->time); mi->data_src.val = sample->data_src;return mi;
@@ -1539,6 +1549,7 @@ static int add_callchain_ip(struct thread *thread, al.filtered =0 ; al.sym =NULL ; +if (branch_history)thread__find_cpumode_addr_location (thread,MAP__FUNCTION , ip, &al);
@@ -1589,6 +1600,7 @@ static int add_callchain_ip(struct thread *thread, }struct branch_info *sample__resolve_bstack (struct perf_sample *sample, +struct perf_session *session,struct addr_location *al) {unsigned int i;
@@ -1599,8 +1611,10 @@ struct branch_info *sample__resolve_bstack(struct perf_sample *sample,return NULL ;for (i =0 ; i < bs->nr; i++) { -ip__resolve_ams (al->thread, &bi[i].to, bs->entries[i].to); -ip__resolve_ams (al->thread, &bi[i].from, bs->entries[i].from); +ip__resolve_ams (session, al->thread, &bi[i].to, + bs->entries[i].to, sample->time); +ip__resolve_ams (session, al->thread, &bi[i].from, + bs->entries[i].from, sample->time); bi[i].flags = bs->entries[i].flags; }return bi;
@@ -1826,7 +1840,6 @@ static int thread__resolve_callchain_sample(struct thread *thread, ip = chain->ips[j]; err =add_callchain_ip (thread, parent, root_al, false, ip); -if (err)return (err <0 ) ? err :0 ; }
tools/perf/util/machine.h +2 -0
@@ -121,8 +121,10 @@ void machine__delete_threads(struct machine *machine);void machine__delete (struct machine *machine);struct branch_info *sample__resolve_bstack (struct perf_sample *sample, +struct perf_session *session,struct addr_location *al);struct mem_info *sample__resolve_mem (struct perf_sample *sample, +struct perf_session *session,struct addr_location *al); int thread__resolve_callchain(struct thread *thread, struct perf_evsel *evsel,
tools/perf/util/session.h +38 -0
@@ -142,4 +142,42 @@ static inline bool perf_session__has_index(struct perf_session *session)return perf_header__has_feat (&session->header,HEADER_DATA_INDEX ); } +static inline void +session__find_addr_map (struct perf_session *session,struct thread *thread, +u8 cpumode,enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp) +{ +if (session &&perf_session__has_index (session)) +thread__find_addr_map_time (thread, cpumode, type, addr, al, + timestamp); +else +thread__find_addr_map (thread, cpumode, type, addr, al); +} + +static inline void +session__find_addr_location (struct perf_session *session,struct thread *thread, +u8 cpumode,enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp) +{ +if (session &&perf_session__has_index (session)) +thread__find_addr_location_time (thread, cpumode, type, addr, al, + timestamp); +else +thread__find_addr_location (thread, cpumode, type, addr, al); +} + +static inline void +session__find_cpumode_addr_location (struct perf_session *session, +struct thread *thread,enum map_type type, +u64 addr,struct addr_location *al, +u64 timestamp) +{ +if (session &&perf_session__has_index (session)) +thread__find_cpumode_addr_location_time (thread, type, addr, al, + timestamp); +else +thread__find_cpumode_addr_location (thread, type, addr, al); +} + +#endif /* __PERF_SESSION_H */
tools/perf/util/thread.c +21 -0
@@ -359,3 +359,24 @@ void thread__find_cpumode_addr_location(struct thread *thread,break ; } } + +void thread__find_cpumode_addr_location_time (struct thread *thread, +enum map_type type,u64 addr, +struct addr_location *al, +u64 timestamp) +{ +size_t i; +const u8 const cpumodes[] = { +PERF_RECORD_MISC_USER , +PERF_RECORD_MISC_KERNEL , +PERF_RECORD_MISC_GUEST_USER , +PERF_RECORD_MISC_GUEST_KERNEL + }; + +for (i =0 ; i <ARRAY_SIZE (cpumodes); i++) { +thread__find_addr_location_time (thread, cpumodes[i], type, + addr, al, timestamp); +if (al->map) +break ; + } +}
tools/perf/util/thread.h +10 -0
@@ -66,14 +66,24 @@ size_t thread__fprintf(struct thread *thread, FILE *fp);void thread__find_addr_map (struct thread *thread,u8 cpumode,enum map_type type,u64 addr,struct addr_location *al); +void thread__find_addr_map_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp);void thread__find_addr_location (struct thread *thread,u8 cpumode,enum map_type type,u64 addr,struct addr_location *al); +void thread__find_addr_location_time (struct thread *thread,u8 cpumode, +enum map_type type,u64 addr, +struct addr_location *al,u64 timestamp);void thread__find_cpumode_addr_location (struct thread *thread,enum map_type type,u64 addr,struct addr_location *al); +void thread__find_cpumode_addr_location_time (struct thread *thread, +enum map_type type,u64 addr, +struct addr_location *al, +u64 timestamp);static inline void *thread__priv (struct thread *thread) {
Signature · 1 line
2.2.2
Namhyung Kim
maintainerPATCH 22/38 perf callchain: Use session__find_addr_location() and friends
In reply to Namhyung KimPass session struct to callchain resolve routines and find correct thread/map/symbol using proper functions.
--- tools/perf/builtin-script.c | 4 +-- tools/perf/tests/dwarf-unwind.c | 2 +- tools/perf/util/callchain.c | 6 ++-- tools/perf/util/callchain.h | 4 ++- tools/perf/util/hist.c | 5 +-- tools/perf/util/machine.c | 41 +++++++++++++--------- tools/perf/util/machine.h | 1 + .../util/scripting-engines/trace-event-python.c | 27 +++++++------- tools/perf/util/session.c | 6 ++-- tools/perf/util/session.h | 2 +- tools/perf/util/unwind-libdw.c | 14 ++++---- tools/perf/util/unwind-libdw.h | 1 + tools/perf/util/unwind-libunwind.c | 32 +++++++++-------- tools/perf/util/unwind.h | 3 +- 14 files changed, 86 insertions(+), 62 deletions(-)
tools/perf/builtin-script.c +2 -2
@@ -429,7 +429,7 @@ static void print_sample_bts(union perf_event *event, print_opts &= ~PRINT_IP_OPT_SRCLINE ; } } -perf_evsel__print_ip (evsel, sample, al, print_opts, +perf_evsel__print_ip (evsel, sample, session, al, print_opts,PERF_MAX_STACK_DEPTH ); }
@@ -483,7 +483,7 @@ static void process_event(union perf_event *event, struct perf_sample *sample,else printf ("\n" ); -perf_evsel__print_ip (evsel, sample, al, +perf_evsel__print_ip (evsel, sample, session, al, output[attr->type].print_ip_opts,PERF_MAX_STACK_DEPTH ); }
tools/perf/tests/dwarf-unwind.c +1 -1
@@ -75,7 +75,7 @@ static int unwind_thread(struct thread *thread) goto out; } - err =unwind__get_entries (unwind_entry, &cnt, thread, + err =unwind__get_entries (unwind_entry, &cnt, thread,NULL , &sample,MAX_STACK );if (err)pr_debug ("unwind failed\n" );
tools/perf/util/callchain.c +4 -2
@@ -757,7 +757,9 @@ int callchain_cursor_append(struct callchain_cursor *cursor,return 0 ; } -int sample__resolve_callchain (struct perf_sample *sample,struct symbol **parent, +int sample__resolve_callchain (struct perf_sample *sample, +struct perf_session *session, +struct symbol **parent,struct perf_evsel *evsel,struct addr_location *al,int max_stack) {
@@ -767,7 +769,7 @@ int sample__resolve_callchain(struct perf_sample *sample, struct symbol **parentif (symbol_conf.use_callchain || symbol_conf.cumulate_callchain || sort__has_parent) {return thread__resolve_callchain (al->thread, evsel, sample, - parent, al, max_stack); + session, parent, al, max_stack); }return 0 ; }
tools/perf/util/callchain.h +3 -1
@@ -165,7 +165,9 @@ struct hist_entry;int record_parse_callchain_opt (const struct option *opt,const char *arg,int unset);int record_callchain_opt (const struct option *opt,const char *arg,int unset); -int sample__resolve_callchain (struct perf_sample *sample,struct symbol **parent, +int sample__resolve_callchain (struct perf_sample *sample, +struct perf_session *session, +struct symbol **parent,struct perf_evsel *evsel,struct addr_location *al,int max_stack);int hist_entry__append_callchain (struct hist_entry *he,struct perf_sample *sample);
tools/perf/util/hist.c +3 -2
@@ -861,8 +861,9 @@ int hist_entry_iter__add(struct hist_entry_iter *iter, struct addr_location *al, {int err, err2; - err =sample__resolve_callchain (iter->sample, &iter->parent, - iter->evsel, al, max_stack_depth); + err =sample__resolve_callchain (iter->sample, iter->session, + &iter->parent, iter->evsel, al, + max_stack_depth);if (err)return err;
tools/perf/util/machine.c +25 -16
121 lines not on this page · raw message
tools/perf/util/machine.h +1 -0
@@ -129,6 +129,7 @@ struct mem_info *sample__resolve_mem(struct perf_sample *sample,int thread__resolve_callchain (struct thread *thread,struct perf_evsel *evsel,struct perf_sample *sample, +struct perf_session *session,struct symbol **parent,struct addr_location *root_al,int max_stack);
tools/perf/util/scripting-engines/trace-event-python.c +15 -12
@@ -298,9 +298,10 @@ static PyObject *get_field_numeric_entry(struct event_format *event, } -static PyObject *python_process_callchain (struct perf_sample *sample, -struct perf_evsel *evsel, -struct addr_location *al) +static PyObject *python_process_callchain (struct perf_session *session, +struct perf_sample *sample, +struct perf_evsel *evsel, +struct addr_location *al) {PyObject *pylist;
@@ -311,9 +312,8 @@ static PyObject *python_process_callchain(struct perf_sample *sample,if (!symbol_conf.use_callchain || !sample->callchain) goto exit; -if (thread__resolve_callchain (al->thread, evsel, - sample,NULL ,NULL , -PERF_MAX_STACK_DEPTH ) !=0 ) { +if (thread__resolve_callchain (al->thread, evsel, sample, session, +NULL ,NULL ,PERF_MAX_STACK_DEPTH ) !=0 ) {pr_err ("Failed to resolve callchain. Skipping\n" ); goto exit; }
@@ -374,7 +374,8 @@ static PyObject *python_process_callchain(struct perf_sample *sample, } -static void python_process_tracepoint (struct perf_sample *sample, +static void python_process_tracepoint (struct perf_session *session, +struct perf_sample *sample,struct perf_evsel *evsel,struct thread *thread,struct addr_location *al)
@@ -424,7 +425,7 @@ static void python_process_tracepoint(struct perf_sample *sample,PyTuple_SetItem (t, n++, context);/* ip unwinding */ - callchain =python_process_callchain (sample, evsel, al); + callchain =python_process_callchain (session, sample, evsel, al);if (handler) {PyTuple_SetItem (t, n++,PyInt_FromLong (cpu));
@@ -759,7 +760,8 @@ static int python_process_call_return(struct call_return *cr, void *data)return db_export__call_return (dbe, cr); } -static void python_process_general_event (struct perf_sample *sample, +static void python_process_general_event (struct perf_session *session, +struct perf_sample *sample,struct perf_evsel *evsel,struct thread *thread,struct addr_location *al)
@@ -822,7 +824,7 @@ static void python_process_general_event(struct perf_sample *sample, }/* ip unwinding */ - callchain =python_process_callchain (sample, evsel, al); + callchain =python_process_callchain (session, sample, evsel, al);pydict_set_item_string_decref (dict,"callchain" , callchain);PyTuple_SetItem (t, n++, dict);
@@ -846,7 +848,7 @@ static void python_process_event(union perf_event *event,switch (evsel->attr.type) {case PERF_TYPE_TRACEPOINT : -python_process_tracepoint (sample, evsel, thread, al); +python_process_tracepoint (session, sample, evsel, thread, al);break ;/* Reserve for future process_hw/sw/raw APIs */ default :
@@ -854,7 +856,8 @@ static void python_process_event(union perf_event *event,db_export__sample (&tables->dbe, event, sample, evsel, thread, al, session); else -python_process_general_event (sample ,evsel ,thread ,al ); +python_process_general_event (session ,sample ,evsel , +thread ,al ); } }
tools/perf/util/session.c +3 -3
@@ -1550,7 +1550,7 @@ struct perf_evsel *perf_session__find_first_evtype(struct perf_session *session, }void perf_evsel__print_ip (struct perf_evsel *evsel,struct perf_sample *sample, -struct addr_location *al, +struct perf_session *session,struct addr_location *al,unsigned int print_opts,unsigned int stack_depth) {struct callchain_cursor_node *node;
@@ -1565,8 +1565,8 @@ void perf_evsel__print_ip(struct perf_evsel *evsel, struct perf_sample *sample,if (symbol_conf.use_callchain && sample->callchain) {struct addr_location node_al; -if (thread__resolve_callchain (al->thread, evsel, - sample,NULL ,NULL , +if (thread__resolve_callchain (al->thread, evsel, sample, + session,NULL ,NULL ,PERF_MAX_STACK_DEPTH ) !=0 ) {if (verbose)error ("Failed to resolve callchain. Skipping\n" );
tools/perf/util/session.h +1 -1
@@ -102,7 +102,7 @@ struct perf_evsel *perf_session__find_first_evtype(struct perf_session *session, unsigned int type);void perf_evsel__print_ip (struct perf_evsel *evsel,struct perf_sample *sample, -struct addr_location *al, +struct perf_session *session,struct addr_location *al,unsigned int print_opts,unsigned int stack_depth); int perf_session__cpu_bitmap(struct perf_session *session,
tools/perf/util/unwind-libdw.c +8 -6
@@ -8,6 +8,7 @@#include "unwind-libdw.h" #include "machine.h" #include "thread.h" +#include "session.h" #include <linux/types.h> #include "event.h" #include "perf_regs.h"
@@ -26,10 +27,9 @@ static int __report_module(struct addr_location *al, u64 ip,Dwfl_Module *mod;struct dso *dso =NULL ; -thread__find_addr_location (ui->thread, -PERF_RECORD_MISC_USER , -MAP__FUNCTION , ip, al); - +session__find_addr_location (ui->session, ui->thread, +PERF_RECORD_MISC_USER ,MAP__FUNCTION , + ip, al, ui->sample->time);if (al->map) dso = al->map->dso;
@@ -89,8 +89,8 @@ static int access_dso_mem(struct unwind_info *ui, Dwarf_Addr addr,struct addr_location al;ssize_t size; -thread__find_addr_map (ui->thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , addr, &al); +session__find_addr_map (ui->session, ui->thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , addr, &al, ui->sample->time);if (!al.map) {pr_debug ("unwind: no map for %lx\n" , (unsigned long )addr);return -1 ;
@@ -165,12 +165,14 @@ frame_callback(Dwfl_Frame *state, void *arg)int unwind__get_entries (unwind_entry_cb_t cb,void *arg,struct thread *thread, +struct perf_session *session,struct perf_sample *data,int max_stack) {struct unwind_info ui = { .sample = data, .thread = thread, + .session = session, .machine = thread->mg->machine, .cb = cb, .arg = arg,
tools/perf/util/unwind-libdw.h +1 -0
@@ -11,6 +11,7 @@ bool libdw__arch_set_initial_registers(Dwfl_Thread *thread, void *arg);struct unwind_info {Dwfl *dwfl;struct perf_sample *sample; +struct perf_session *session;struct machine *machine;struct thread *thread;unwind_entry_cb_t cb;
tools/perf/util/unwind-libunwind.c +17 -15
@@ -86,6 +86,7 @@ UNW_OBJ(dwarf_find_debug_frame) (int found, unw_dyn_info_t *di_debug,struct unwind_info {struct perf_sample *sample; +struct perf_session *session;struct machine *machine;struct thread *thread; };
@@ -315,8 +316,8 @@ static struct map *find_map(unw_word_t ip, struct unwind_info *ui) {struct addr_location al; -thread__find_addr_map (ui->thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , ip, &al); +session__find_addr_map (ui->session, ui->thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , ip, &al, ui->sample->time);return al.map; }
@@ -406,20 +407,19 @@ get_proc_name(unw_addr_space_t __maybe_unused as,static int access_dso_mem (struct unwind_info *ui,unw_word_t addr,unw_word_t *data) { -struct addr_location al; +struct map *map;ssize_t size; -thread__find_addr_map (ui->thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , addr, &al); -if (!al.map) { + map =find_map (addr, ui); +if (!map) {pr_debug ("unwind: no map for %lx\n" , (unsigned long )addr);return -1 ; } -if (!al.map->dso) +if (!map->dso)return -1 ; - size =dso__data_read_addr (al.map->dso, al.map, ui->machine, + size =dso__data_read_addr (map->dso, map, ui->machine, addr, (u8 *) data,sizeof (*data));return !(size ==sizeof (*data));
@@ -511,14 +511,14 @@ static void put_unwind_info(unw_addr_space_t __maybe_unused as,pr_debug ("unwind: put_unwind_info called\n" ); } -static int entry (u64 ip,struct thread *thread, -unwind_entry_cb_t cb,void *arg) +static int entry (u64 ip,struct thread *thread,struct perf_session *session, +u64 timestamp,unwind_entry_cb_t cb,void *arg) {struct unwind_entry e;struct addr_location al; -thread__find_addr_location (thread,PERF_RECORD_MISC_USER , -MAP__FUNCTION , ip, &al); +session__find_addr_location (session, thread,PERF_RECORD_MISC_USER , +MAP__FUNCTION , ip, &al, timestamp); e.ip = ip; e.map = al.map;
@@ -620,20 +620,22 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,unw_word_t ip;unw_get_reg (&c,UNW_REG_IP , &ip); - ret = ip ?entry (ip, ui->thread, cb, arg) :0 ; + ret = ip ?entry (ip, ui->thread, ui->session, + ui->sample->time, cb, arg) :0 ; }return ret; }int unwind__get_entries (unwind_entry_cb_t cb,void *arg, -struct thread *thread, +struct thread *thread,struct perf_session *session,struct perf_sample *data,int max_stack) {u64 ip;struct unwind_info ui = { .sample = data, .thread = thread, + .session = session, .machine = thread->mg->machine, };int ret;
@@ -645,7 +647,7 @@ int unwind__get_entries(unwind_entry_cb_t cb, void *arg,if (ret)return ret; - ret =entry (ip, thread, cb, arg); + ret =entry (ip, thread, session, data->time, cb, arg);if (ret)return -ENOMEM ;
tools/perf/util/unwind.h +2 -1
@@ -16,7 +16,7 @@ typedef int (*unwind_entry_cb_t)(struct unwind_entry *entry, void *arg);#ifdef HAVE_DWARF_UNWIND_SUPPORT int unwind__get_entries (unwind_entry_cb_t cb,void *arg, -struct thread *thread, +struct thread *thread,struct perf_session *session,struct perf_sample *data,int max_stack);/* libunwind specific */ #ifdef HAVE_LIBUNWIND_SUPPORT
@@ -38,6 +38,7 @@ static inline int unwind__get_entries(unwind_entry_cb_t cb __maybe_unused, void *arg __maybe_unused, struct thread *thread __maybe_unused, + struct perf_session *session __maybe_unused, struct perf_sample *data __maybe_unused, int max_stack __maybe_unused) {
Signature · 1 line
2.2.2