maps/threads/dsos memory improvements and fixes
Ian Rogers
reviewerv7 cover maps/threads/dsos memory improvements and fixes
Modify the implementation of maps to not use an rbtree as the container for maps, instead use a sorted array. Improve locking and reference counting issues.
Similar to maps separate out and reimplement threads to use a hashmap for lower memory consumption and faster look up. The fixes a regression in memory usage where reference count checking switched to using non-invasive tree nodes. Reduce its default size by 32 times and improve locking discipline. Also, fix regressions where tids had become unordered to make perf report --tasks andperf trace --summary output easier to read.
Better encapsulate the dsos abstraction. Remove the linked list and rbtree used for faster iteration and log(n) lookup to a sorted array for similar performance but half the memory usage per dso. Improve reference counting and locking discipline, adding reference count checking to dso.
v7:
- rebase to latest perf-tools-next where 22 patches were applied by Arnaldo.
- resolve merge conflicts, in particular with fc044c53b99f ("perf
annotate-data: Add dso->data_types tree") that required more dso accessor functions.
v6 series is here:
https://lore.kernel.org/lkml/[email protected]/
Ian Rogers (25):
perf maps: Switch from rbtree to lazily sorted array for addresses perf maps: Get map before returning in maps__find
perf maps: Get map before returning in maps__find_by_name perf maps: Get map before returning in maps__find_next_entry perf maps: Hide maps internals
perf maps: Locking tidy up of nr_maps
perf dso: Reorder variables to save space in struct dso perf report: Sort child tasks by tid
perf trace: Ignore thread hashing in summary
perf machine: Move fprintf to for_each loop and a callback perf threads: Move threads to its own files
perf threads: Switch from rbtree to hashmap
perf threads: Reduce table size from 256 to 8
perf dsos: Attempt to better abstract dsos internals perf dsos: Tidy reference counting and locking
perf dsos: Add dsos__for_each_dso
perf dso: Move dso functions out of dsos
perf dsos: Switch more loops to dsos__for_each_dso
perf dsos: Switch backing storage to array from rbtree/list perf dsos: Remove __dsos__addnew
perf dsos: Remove __dsos__findnew_link_by_longname_id perf dsos: Switch hand code to bsearch
perf dso: Add reference count checking and accessor functions perf dso: Reference counting related fixes
perf dso: Use container_of to avoid a pointer in dso_data
tools/perf/arch/x86/tests/dwarf-unwind.c | 1 + tools/perf/builtin-annotate.c | 8 +- tools/perf/builtin-buildid-cache.c | 2 +- tools/perf/builtin-buildid-list.c | 18 +- tools/perf/builtin-inject.c | 96 +- tools/perf/builtin-kallsyms.c | 2 +- tools/perf/builtin-mem.c | 4 +- tools/perf/builtin-record.c | 2 +- tools/perf/builtin-report.c | 209 +-- tools/perf/builtin-script.c | 8 +- tools/perf/builtin-top.c | 4 +- tools/perf/builtin-trace.c | 43 +- tools/perf/tests/code-reading.c | 8 +- tools/perf/tests/dso-data.c | 67 +- tools/perf/tests/hists_common.c | 6 +- tools/perf/tests/hists_cumulate.c | 4 +- tools/perf/tests/hists_output.c | 2 +- tools/perf/tests/maps.c | 7 +- tools/perf/tests/symbols.c | 2 +- tools/perf/tests/thread-maps-share.c | 8 +- tools/perf/tests/vmlinux-kallsyms.c | 16 +- tools/perf/ui/browsers/annotate.c | 6 +- tools/perf/ui/browsers/hists.c | 8 +- tools/perf/ui/browsers/map.c | 4 +- tools/perf/util/Build | 1 + tools/perf/util/annotate-data.c | 6 +- tools/perf/util/annotate.c | 45 +- tools/perf/util/auxtrace.c | 2 +- tools/perf/util/block-info.c | 2 +- tools/perf/util/bpf-event.c | 9 +- tools/perf/util/bpf_lock_contention.c | 8 +- tools/perf/util/build-id.c | 136 +- tools/perf/util/build-id.h | 2 - tools/perf/util/callchain.c | 4 +- tools/perf/util/data-convert-json.c | 2 +- tools/perf/util/db-export.c | 6 +- tools/perf/util/dlfilter.c | 12 +- tools/perf/util/dso.c | 469 +++--- tools/perf/util/dso.h | 549 +++++-- tools/perf/util/dsos.c | 529 ++++--- tools/perf/util/dsos.h | 40 +- tools/perf/util/event.c | 12 +- tools/perf/util/header.c | 8 +- tools/perf/util/hist.c | 4 +- tools/perf/util/intel-pt.c | 22 +- tools/perf/util/machine.c | 570 +++----- tools/perf/util/machine.h | 32 +- tools/perf/util/map.c | 73 +- tools/perf/util/maps.c | 1280 +++++++++++------ tools/perf/util/maps.h | 65 +- tools/perf/util/probe-event.c | 26 +- tools/perf/util/rb_resort.h | 5 - .../util/scripting-engines/trace-event-perl.c | 6 +- .../scripting-engines/trace-event-python.c | 21 +- tools/perf/util/session.c | 21 + tools/perf/util/session.h | 2 + tools/perf/util/sort.c | 19 +- tools/perf/util/srcline.c | 65 +- tools/perf/util/symbol-elf.c | 132 +- tools/perf/util/symbol.c | 217 +-- tools/perf/util/symbol_fprintf.c | 4 +- tools/perf/util/synthetic-events.c | 24 +- tools/perf/util/thread.c | 8 +- tools/perf/util/thread.h | 6 - tools/perf/util/threads.c | 186 +++ tools/perf/util/threads.h | 35 + tools/perf/util/unwind-libunwind-local.c | 20 +- tools/perf/util/unwind-libunwind.c | 9 +- tools/perf/util/vdso.c | 56 +- 69 files changed, 3127 insertions(+), 2158 deletions(-) create mode 100644 tools/perf/util/threads.c create mode 100644 tools/perf/util/threads.h
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 1/25 perf maps: Switch from rbtree to lazily sorted array for addresses
Maps is a collection of maps primarily sorted by the starting address of the map. Prior to this change the maps were held in an rbtree requiring 4 pointers per node. Prior to reference count checking, the rbnode was embedded in the map so 3 pointers per node were necessary. This change switches the rbtree to an array lazily sorted by address, much as the array sorting nodes by name. 1 pointer is needed per node, but to avoid excessive resizing the backing array may be twice the number of used elements. Meaning the memory overhead is roughly half that of the rbtree. For a perf record with "--no-bpf-event -g -a" of true, the memory overhead of perf inject is reduce fom 3.3MB to 3MB, so 10% or 300KB is saved.
Map inserts always happen at the end of the array. The code tracks whether the insertion violates the sorting property. O(log n) rb-tree complexity is switched to O(1).
Remove slides the array, so O(log n) rb-tree complexity is degraded to O(n).
A find may need to sort the array using qsort which is O(n*log n), but in general the maps should be sorted and so average performance should be O(log n) as with the rbtree.
An rbtree node consumes a cache line, but with the array 4 nodes fit on a cache line. Iteration is simplified to scanning an array rather than pointer chasing.
Overall it is expected the performance after the change should be comparable to before, but with half of the memory consumed.
To avoid a list and repeated logic around splitting maps, maps__merge_in is rewritten in terms of maps__fixup_overlap_and_insert. maps_merge_in splits the given mapping inserting remaining gaps. maps__fixup_overlap_and_insert splits the existing mappings, then adds the incoming mapping. By adding the new mapping first, then re-inserting the existing mappings the splitting behavior matches.
--- tools/perf/tests/maps.c | 3 + tools/perf/util/map.c | 1 + tools/perf/util/maps.c | 1183 +++++++++++++++++++++++---------------- tools/perf/util/maps.h | 54 +- 4 files changed, 757 insertions(+), 484 deletions(-)
tools/perf/tests/maps.c +3 -0
@@ -156,6 +156,9 @@ static int test__maps__merge_in(struct test_suite *t __maybe_unused, int subtestTEST_ASSERT_VAL ("merge check failed" , !ret);maps__zput (maps); +map__zput (map_kcore1); +map__zput (map_kcore2); +map__zput (map_kcore3);return TEST_OK ; }
tools/perf/util/map.c +1 -0
@@ -168,6 +168,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,if (dso ==NULL ) goto out_delete; +assert (!dso->kernel);map__init (result, start, start + len, pgoff, dso);if (anon || no_dso) {
tools/perf/util/maps.c +709 -474
1,447 lines not on this page · raw message
tools/perf/util/maps.h +44 -10
79 lines not on this page · raw message
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Namhyung Kim
maintainerRe: [PATCH v7 01/25] perf maps: Switch from rbtree to lazily sorted array for addresses
Hi Ian,
Quoted 24 lines · Ian Rogers
Maps is a collection of maps primarily sorted by the starting address of the map. Prior to this change the maps were held in an rbtree requiring 4 pointers per node. Prior to reference count checking, the rbnode was embedded in the map so 3 pointers per node were necessary. This change switches the rbtree to an array lazily sorted by address, much as the array sorting nodes by name. 1 pointer is needed per node, but to avoid excessive resizing the backing array may be twice the number of used elements. Meaning the memory overhead is roughly half that of the rbtree. For a perf record with "--no-bpf-event -g -a" of true, the memory overhead of perf inject is reduce fom 3.3MB to 3MB, so 10% or 300KB is saved.
Map inserts always happen at the end of the array. The code tracks whether the insertion violates the sorting property. O(log n) rb-tree complexity is switched to O(1).
Remove slides the array, so O(log n) rb-tree complexity is degraded to O(n).
A find may need to sort the array using qsort which is
O(n*log n), but in general the maps should be sorted and so average performance should be O(log n) as with the rbtree.
Quoted 6 lines
An rbtree node consumes a cache line, but with the array 4 nodes fit on a cache line. Iteration is simplified to scanning an array rather than pointer chasing.
Overall it is expected the performance after the change should be comparable to before, but with half of the memory consumed.
I don't know how much performance impact it would have but I guess search/iteration would be the most frequent operation. So I like the memory saving it can bring.
Quoted 561 lines
To avoid a list and repeated logic around splitting maps,
maps__merge_inis rewritten in terms ofmaps__fixup_overlap_and_insert.maps_merge_insplits the given mapping inserting remaining gaps.maps__fixup_overlap_and_insertsplits the existing mappings, then adds the incoming mapping. By adding the new mapping first, then re-inserting the existing mappings the splitting behavior matches.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/maps.c | 3 + tools/perf/util/map.c | 1 + tools/perf/util/maps.c | 1183 +++++++++++++++++++++++---------------- tools/perf/util/maps.h | 54 +- 4 files changed, 757 insertions(+), 484 deletions(-)tools/perf/tests/maps.c +3
-0@@ -156,6 +156,9 @@ static int test__maps__merge_in(struct test_suite *t __maybe_unused, int subtestTEST_ASSERT_VAL ("merge check failed" , !ret);maps__zput (maps); +map__zput (map_kcore1); +map__zput (map_kcore2); +map__zput (map_kcore3);return TEST_OK ; }tools/perf/util/map.c +1
-0@@ -168,6 +168,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,if (dso ==NULL ) goto out_delete; +assert (!dso->kernel);map__init (result, start, start + len, pgoff, dso);if (anon || no_dso) {tools/perf/util/maps.c +289
-142@@ -10,286 +10,477 @@#include "ui/ui.h" #include "unwind.h" -struct map_rb_node { -struct rb_node rb_node; -struct map *map; -}; - -#define maps__for_each_entry (maps, map) \ - for (map = maps__first(maps); map; map = map_rb_node__next(map)) +static void check_invariants (const struct maps *maps__maybe_unused ) +{ +#ifndef NDEBUG +assert (RC_CHK_ACCESS (maps)->nr_maps <=RC_CHK_ACCESS (maps)->nr_maps_allocated); +for (unsigned int i =0 ; i <RC_CHK_ACCESS (maps)->nr_maps; i++) { +struct map *map =RC_CHK_ACCESS (maps)->maps_by_address[i]; + +/* Check map is well-formed. */ +assert (map__end (map) ==0 ||map__start (map) <=map__end (map)); +/* Expect at least 1 reference count. */ +assert (refcount_read (map__refcnt (map)) >0 ); + +if (map__dso (map) &&map__dso (map)->kernel) +assert (RC_CHK_EQUAL (map__kmap (map)->kmaps, maps)); + +if (i >0 ) { +struct map *prev =RC_CHK_ACCESS (maps)->maps_by_address[i -1 ]; + +/* If addresses are sorted... */ +if (RC_CHK_ACCESS (maps)->maps_by_address_sorted) { +/* Maps should be in start address order. */ +assert (map__start (prev) <=map__start (map)); +/* +* If the ends of maps aren't broken (during +* construction) then they should be ordered +* too. +*/ +if (!RC_CHK_ACCESS (maps)->ends_broken) { +assert (map__end (prev) <=map__end (map)); +assert (map__end (prev) <=map__start (map) || +map__start (prev) ==map__start (map)); + } + } + } + } +if (RC_CHK_ACCESS (maps)->maps_by_name) { +for (unsigned int i =0 ; i <RC_CHK_ACCESS (maps)->nr_maps; i++) { +struct map *map =RC_CHK_ACCESS (maps)->maps_by_name[i]; -#define maps__for_each_entry_safe (maps, map, next) \ - for (map = maps__first(maps), next = map_rb_node__next(map); map; \ - map = next, next = map_rb_node__next(map)) +/* +* Maps by name maps should be in maps_by_address, so +* the reference count should be higher. +*/ +assert (refcount_read (map__refcnt (map)) >1 ); + } + } +#endif +} -static struct rb_root *maps__entries (struct maps *maps) +static struct map **maps__maps_by_address (const struct maps *maps) { -return &RC_CHK_ACCESS (maps)->entries; +return RC_CHK_ACCESS (maps)->maps_by_address; } -static struct rw_semaphore *maps__lock (struct maps *maps) +static void maps__set_maps_by_address (struct maps *maps,struct map **new) { -return &RC_CHK_ACCESS (maps)->lock; +RC_CHK_ACCESS (maps)->maps_by_address = new; + } -static struct map **maps__maps_by_name (struct maps *maps) +/* Not in the header, to aid reference counting. */ +static struct map **maps__maps_by_name (const struct maps *maps) {return RC_CHK_ACCESS (maps)->maps_by_name; + } -static struct map_rb_node *maps__first (struct maps *maps) +static void maps__set_maps_by_name (struct maps *maps,struct map **new) { -struct rb_node *first =rb_first (maps__entries (maps)); +RC_CHK_ACCESS (maps)->maps_by_name = new; -if (first) - return rb_entry(first, struct map_rb_node, rb_node); -return NULL ; } -static struct map_rb_node *map_rb_node__next (struct map_rb_node *node) +static bool maps__maps_by_address_sorted (const struct maps *maps) { -struct rb_node *next; - -if (!node) -return NULL ; - - next =rb_next (&node->rb_node); +return RC_CHK_ACCESS (maps)->maps_by_address_sorted; +} -if (!next) -return NULL ; +static void maps__set_maps_by_address_sorted (struct maps *maps,bool value) +{ +RC_CHK_ACCESS (maps)->maps_by_address_sorted = value; +} - return rb_entry(next, struct map_rb_node, rb_node); +static bool maps__maps_by_name_sorted (const struct maps *maps) +{ +return RC_CHK_ACCESS (maps)->maps_by_name_sorted; } -static struct map_rb_node *maps__find_node (struct maps *maps,struct map *map) +static void maps__set_maps_by_name_sorted (struct maps *maps,bool value) { -struct map_rb_node *rb_node; +RC_CHK_ACCESS (maps)->maps_by_name_sorted = value; +} -maps__for_each_entry (maps, rb_node) { -if (rb_node->RC_CHK_ACCESS (map) ==RC_CHK_ACCESS (map)) -return rb_node; - } -return NULL ; +static struct rw_semaphore *maps__lock (struct maps *maps) +{ +/* +* When the lock is acquired or released the maps invariants should +* hold. +*/ +check_invariants (maps); +return &RC_CHK_ACCESS (maps)->lock; }static void maps__init (struct maps *maps,struct machine *machine) { -refcount_set (maps__refcnt (maps),1 );init_rwsem (maps__lock (maps)); -RC_CHK_ACCESS (maps)->entries =RB_ROOT ; +RC_CHK_ACCESS (maps)->maps_by_address =NULL ; +RC_CHK_ACCESS (maps)->maps_by_name =NULL ;RC_CHK_ACCESS (maps)->machine = machine; -RC_CHK_ACCESS (maps)->last_search_by_name =NULL ; +#ifdef HAVE_LIBUNWIND_SUPPORT +RC_CHK_ACCESS (maps)->addr_space =NULL ; +RC_CHK_ACCESS (maps)->unwind_libunwind_ops =NULL ; +#endif +refcount_set (maps__refcnt (maps),1 );RC_CHK_ACCESS (maps)->nr_maps =0 ; -RC_CHK_ACCESS (maps)->maps_by_name =NULL ; +RC_CHK_ACCESS (maps)->nr_maps_allocated =0 ; +RC_CHK_ACCESS (maps)->last_search_by_name_idx =0 ; +RC_CHK_ACCESS (maps)->maps_by_address_sorted = true; +RC_CHK_ACCESS (maps)->maps_by_name_sorted = false; } -static void __maps__free_maps_by_name (struct maps *maps) +static void maps__exit (struct maps *maps) { -/* -* Free everything to try to do it from the rbtree in the next search -*/ -for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) -map__put (maps__maps_by_name (maps)[i]); +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); -zfree (&RC_CHK_ACCESS (maps)->maps_by_name); -RC_CHK_ACCESS (maps)->nr_maps_allocated =0 ; +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +map__zput (maps_by_address[i]); +if (maps_by_name) +map__zput (maps_by_name[i]); + } +zfree (&maps_by_address); +zfree (&maps_by_name); +unwind__finish_access (maps); } -static int __maps__insert (struct maps *maps,struct map *map) +struct maps *maps__new (struct machine *machine) { -struct rb_node **p = &maps__entries (maps)->rb_node; -struct rb_node *parent =NULL ; -const u64 ip =map__start (map); -struct map_rb_node *m, *new_rb_node; - - new_rb_node =malloc (sizeof (*new_rb_node)); -if (!new_rb_node) -return -ENOMEM ; - -RB_CLEAR_NODE (&new_rb_node->rb_node); - new_rb_node->map =map__get (map); +struct maps *result; +RC_STRUCT (maps) *maps =zalloc (sizeof (*maps)); -while (*p !=NULL ) { - parent = *p; - m = rb_entry(parent, struct map_rb_node, rb_node); -if (ip <map__start (m->map)) - p = &(*p)->rb_left; -else - p = &(*p)->rb_right; - } +if (ADD_RC_CHK (result, maps)) +maps__init (result, machine); -rb_link_node (&new_rb_node->rb_node, parent, p); -rb_insert_color (&new_rb_node->rb_node,maps__entries (maps)); -return 0 ; +return result; } -int maps__insert (struct maps *maps,struct map *map) +static void maps__delete (struct maps *maps) { -int err; -const struct dso *dso =map__dso (map); - -down_write (maps__lock (maps)); - err =__maps__insert (maps, map); -if (err) - goto out; +maps__exit (maps); +RC_CHK_FREE (maps); +} - ++RC_CHK_ACCESS (maps)->nr_maps; +struct maps *maps__get (struct maps *maps) +{ +struct maps *result; -if (dso && dso->kernel) { -struct kmap *kmap =map__kmap (map); +if (RC_CHK_GET (result, maps)) +refcount_inc (maps__refcnt (maps)); -if (kmap) - kmap->kmaps = maps; -else -pr_err ("Internal error: kernel dso with non kernel map\n" ); - } +return result; +} +void maps__put (struct maps *maps) +{ +if (maps &&refcount_dec_and_test (maps__refcnt (maps))) +maps__delete (maps); +else +RC_CHK_PUT (maps); +} +static void __maps__free_maps_by_name (struct maps *maps) +{/* -* If we already performed some search by name, then we need to add the just -* inserted map and resort. +* Free everything to try to do it from the rbtree in the next search */ -if (maps__maps_by_name (maps)) { -if (maps__nr_maps (maps) >RC_CHK_ACCESS (maps)->nr_maps_allocated) { -int nr_allocate =maps__nr_maps (maps) *2 ; -struct map **maps_by_name =realloc (maps__maps_by_name (maps), - nr_allocate *sizeof (map)); +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) +map__put (maps__maps_by_name (maps)[i]); -if (maps_by_name ==NULL ) { -__maps__free_maps_by_name (maps); - err = -ENOMEM ; - goto out; - } +zfree (&RC_CHK_ACCESS (maps)->maps_by_name); +} -RC_CHK_ACCESS (maps)->maps_by_name = maps_by_name; -RC_CHK_ACCESS (maps)->nr_maps_allocated = nr_allocate; +static int map__start_cmp (const void *a,const void *b) +{ +const struct map *map_a = *(const struct map *const *)a; +const struct map *map_b = *(const struct map *const *)b; +u64 map_a_start =map__start (map_a); +u64 map_b_start =map__start (map_b); + +if (map_a_start == map_b_start) { +u64 map_a_end =map__end (map_a); +u64 map_b_end =map__end (map_b); + +if (map_a_end == map_b_end) { +/* Ensure maps with the same addresses have a fixed order. */ +if (RC_CHK_ACCESS (map_a) ==RC_CHK_ACCESS (map_b)) +return 0 ; +return (intptr_t )RC_CHK_ACCESS (map_a) > (intptr_t )RC_CHK_ACCESS (map_b) + ?1 :-1 ; } -maps__maps_by_name (maps)[maps__nr_maps (maps) -1 ] =map__get (map); -__maps__sort_by_name (maps); +return map_a_end > map_b_end ?1 :-1 ; } - out: -up_write (maps__lock (maps)); -return err; +return map_a_start > map_b_start ?1 :-1 ; } -static void __maps__remove (struct maps *maps,struct map_rb_node *rb_node) +static void __maps__sort_by_address (struct maps *maps) { -rb_erase_init (&rb_node->rb_node,maps__entries (maps)); -map__put (rb_node->map); -free (rb_node); +if (maps__maps_by_address_sorted (maps)) +return ; + +qsort (maps__maps_by_address (maps), +maps__nr_maps (maps), +sizeof (struct map *), + map__start_cmp); +maps__set_maps_by_address_sorted (maps, true); } -void maps__remove (struct maps *maps,struct map *map) +static void maps__sort_by_address (struct maps *maps) { -struct map_rb_node *rb_node; -down_write (maps__lock (maps)); -if (RC_CHK_ACCESS (maps)->last_search_by_name == map) -RC_CHK_ACCESS (maps)->last_search_by_name =NULL ; - - rb_node =maps__find_node (maps, map); -assert (rb_node->RC_CHK_ACCESS (map) ==RC_CHK_ACCESS (map)); -__maps__remove (maps, rb_node); -if (maps__maps_by_name (maps)) -__maps__free_maps_by_name (maps); - --RC_CHK_ACCESS (maps)->nr_maps; +__maps__sort_by_address (maps);up_write (maps__lock (maps)); } -static void __maps__purge (struct maps *maps) +static int map__strcmp (const void *a,const void *b) { -struct map_rb_node *pos, *next; - -if (maps__maps_by_name (maps)) -__maps__free_maps_by_name (maps); +const struct map *map_a = *(const struct map *const *)a; +const struct map *map_b = *(const struct map *const *)b; +const struct dso *dso_a =map__dso (map_a); +const struct dso *dso_b =map__dso (map_b); +int ret =strcmp (dso_a->short_name, dso_b->short_name); -maps__for_each_entry_safe (maps, pos, next) { -rb_erase_init (&pos->rb_node,maps__entries (maps)); -map__put (pos->map); -free (pos); +if (ret ==0 &&RC_CHK_ACCESS (map_a) !=RC_CHK_ACCESS (map_b)) { +/* Ensure distinct but name equal maps have an order. */ +return map__start_cmp (a, b); } +return ret; } -static void maps__exit (struct maps *maps) +static int maps__sort_by_name (struct maps *maps) { +int err =0 ;down_write (maps__lock (maps)); -__maps__purge (maps); +if (!maps__maps_by_name_sorted (maps)) { +struct map **maps_by_name =maps__maps_by_name (maps); + +if (!maps_by_name) { + maps_by_name =malloc (RC_CHK_ACCESS (maps)->nr_maps_allocated * +sizeof (*maps_by_name)); +if (!maps_by_name) + err = -ENOMEM ; +else { +struct map **maps_by_address =maps__maps_by_address (maps); +unsigned int n =maps__nr_maps (maps); + +maps__set_maps_by_name (maps, maps_by_name); +for (unsigned int i =0 ; i < n; i++) + maps_by_name[i] =map__get (maps_by_address[i]); + } + } +if (!err) { +qsort (maps_by_name, +maps__nr_maps (maps), +sizeof (struct map *), + map__strcmp); +maps__set_maps_by_name_sorted (maps, true); + } + }up_write (maps__lock (maps)); +return err; } -bool maps__empty (struct maps *maps) +static unsigned int maps__by_address_index (const struct maps *maps,const struct map *map) { -return !maps__first (maps); +struct map **maps_by_address =maps__maps_by_address (maps); + +if (maps__maps_by_address_sorted (maps)) { +struct map **mapp = +bsearch (&map,maps__maps_by_address (maps),maps__nr_maps (maps), +sizeof (*mapp), map__start_cmp); + +if (mapp) +return mapp - maps_by_address; + }else { +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +if (RC_CHK_ACCESS (maps_by_address[i]) ==RC_CHK_ACCESS (map)) +return i; + } + } +pr_err ("Map missing from maps" ); +return -1 ; } -struct maps *maps__new (struct machine *machine) +static unsigned int maps__by_name_index (const struct maps *maps,const struct map *map) { -struct maps *result; -RC_STRUCT (maps) *maps =zalloc (sizeof (*maps)); +struct map **maps_by_name =maps__maps_by_name (maps); + +if (maps__maps_by_name_sorted (maps)) { +struct map **mapp = +bsearch (&map, maps_by_name,maps__nr_maps (maps), +sizeof (*mapp), map__strcmp); + +if (mapp) +return mapp - maps_by_name; + }else { +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +if (RC_CHK_ACCESS (maps_by_name[i]) ==RC_CHK_ACCESS (map)) +return i; + } + } +pr_err ("Map missing from maps" ); +return -1 ; +} -if (ADD_RC_CHK (result, maps)) -maps__init (result, machine); +static int __maps__insert (struct maps *maps,struct map *new) +{ +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); +const struct dso *dso =map__dso (new); +unsigned int nr_maps =maps__nr_maps (maps); +unsigned int nr_allocate =RC_CHK_ACCESS (maps)->nr_maps_allocated; + +if (nr_maps +1 > nr_allocate) { + nr_allocate = !nr_allocate ?32 : nr_allocate *2 ; + + maps_by_address =realloc (maps_by_address, nr_allocate *sizeof (new)); +if (!maps_by_address) +return -ENOMEM ; + +maps__set_maps_by_address (maps, maps_by_address); +if (maps_by_name) { + maps_by_name =realloc (maps_by_name, nr_allocate *sizeof (new)); +if (!maps_by_name) { +/* +* If by name fails, just disable by name and it will +* recompute next time it is required. +*/ +__maps__free_maps_by_name (maps); + } +maps__set_maps_by_name (maps, maps_by_name); + } +RC_CHK_ACCESS (maps)->nr_maps_allocated = nr_allocate; + } +/* Insert the value at the end. */ + maps_by_address[nr_maps] =map__get (new); +if (maps_by_name) + maps_by_name[nr_maps] =map__get (new); -return result; + nr_maps++; +RC_CHK_ACCESS (maps)->nr_maps = nr_maps; + +/* +* Recompute if things are sorted. If things are inserted in a sorted +* manner, for example by processing /proc/pid/maps, then no +* sorting/resorting will be necessary. +*/ +if (nr_maps ==1 ) { +/* If there's just 1 entry then maps are sorted. */ +maps__set_maps_by_address_sorted (maps, true); +maps__set_maps_by_name_sorted (maps, maps_by_name !=NULL ); + }else { +/* Sorted if maps were already sorted and this map starts after the last one. */ +maps__set_maps_by_address_sorted (maps, +maps__maps_by_address_sorted (maps) && +map__end (maps_by_address[nr_maps -2 ]) <=map__start (new)); +maps__set_maps_by_name_sorted (maps, false); + } +if (map__end (new) <map__start (new))
tools/perf/util/maps.c line 390
Quoted 10 lines
tools/perf/util/maps.c +10
-0+RC_CHK_ACCESS (maps)->ends_broken = true; +if (dso && dso->kernel) { +struct kmap *kmap =map__kmap (new); + +if (kmap) + kmap->kmaps = maps; +else +pr_err ("Internal error: kernel dso with non kernel map\n" ); + } +check_invariants (maps);
Probably not needed as it's checked when you get the lock below.
Quoted 36 lines
tools/perf/util/maps.c +24
-6+return 0 ; } -static void maps__delete (struct maps *maps) +int maps__insert (struct maps *maps,struct map *map) { -maps__exit (maps); -unwind__finish_access (maps); -RC_CHK_FREE (maps); +int ret; + +down_write (maps__lock (maps)); + ret =__maps__insert (maps, map); +up_write (maps__lock (maps)); +return ret; } -struct maps *maps__get (struct maps *maps) +static void __maps__remove (struct maps *maps,struct map *map) { -struct maps *result; +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); +unsigned int nr_maps =maps__nr_maps (maps); +unsigned int address_idx; + +/* Slide later mappings over the one to remove */ + address_idx =maps__by_address_index (maps, map); +map__put (maps_by_address[address_idx]); +memmove (&maps_by_address[address_idx], + &maps_by_address[address_idx +1 ], + (nr_maps - address_idx -1 ) *sizeof (*maps_by_address)); + +if (maps_by_name) { +unsigned int name_idx =maps__by_name_index (maps, map); +
tools/perf/util/maps.c line 428
Quoted 10 lines
tools/perf/util/maps.c +7
-2+map__put (maps_by_name[name_idx]); +memmove (&maps_by_name[name_idx], + &maps_by_name[name_idx +1 ], + (nr_maps - name_idx -1 ) *sizeof (*maps_by_name)); + } -if (RC_CHK_GET (result, maps)) -refcount_inc (maps__refcnt (maps)); + --RC_CHK_ACCESS (maps)->nr_maps; +check_invariants (maps);
Ditto.
Quoted 397 lines
tools/perf/util/maps.c +194
-99+} -return result; +void maps__remove (struct maps *maps,struct map *map) +{ +down_write (maps__lock (maps)); +__maps__remove (maps, map); +up_write (maps__lock (maps)); } -void maps__put (struct maps *maps) +bool maps__empty (struct maps *maps) { -if (maps &&refcount_dec_and_test (maps__refcnt (maps))) -maps__delete (maps); -else -RC_CHK_PUT (maps); +return maps__nr_maps (maps) ==0 ; }int maps__for_each_map (struct maps *maps,int (*cb)(struct map *map,void *data),void *data) { -struct map_rb_node *pos; +bool done = false;int ret =0 ; -down_read (maps__lock (maps)); -maps__for_each_entry (maps, pos) { - ret =cb (pos->map, data); -if (ret) -break ; +/* See locking/sorting note. */ +while (!done) { +down_read (maps__lock (maps)); +if (maps__maps_by_address_sorted (maps)) { +struct map **maps_by_address =maps__maps_by_address (maps); +unsigned int n =maps__nr_maps (maps); + +for (unsigned int i =0 ; i < n; i++) { +struct map *map = maps_by_address[i]; + + ret =cb (map, data); +if (ret) +break ; + } + done = true; + } +up_read (maps__lock (maps)); +if (!done) +maps__sort_by_address (maps); } -up_read (maps__lock (maps));return ret; }void maps__remove_maps (struct maps *maps,bool (*cb)(struct map *map,void *data),void *data) { -struct map_rb_node *pos, *next; -unsigned int start_nr_maps; +struct map **maps_by_address;down_write (maps__lock (maps)); - start_nr_maps =maps__nr_maps (maps); -maps__for_each_entry_safe (maps, pos, next) { -if (cb (pos->map, data)) { -__maps__remove (maps, pos); - --RC_CHK_ACCESS (maps)->nr_maps; - } + maps_by_address =maps__maps_by_address (maps); +for (unsigned int i =0 ; i <maps__nr_maps (maps);) { +if (cb (maps_by_address[i], data)) +__maps__remove (maps, maps_by_address[i]); +else + i++; } -if (maps__maps_by_name (maps) && start_nr_maps !=maps__nr_maps (maps)) -__maps__free_maps_by_name (maps); -up_write (maps__lock (maps)); }@@ -300,7 +491,7 @@ struct symbol *maps__find_symbol(struct maps *maps, u64 addr, struct map **mapp)/* Ensure map is loaded before using map->map_ip */ if (map !=NULL &&map__load (map) >=0 ) {if (mapp !=NULL ) - *mapp = map; + *mapp = map;// TODO: map_put on else path when find returns a get. return map__find_symbol (map,map__map_ip (map, addr)); }@@ -348,7 +539,7 @@ int maps__find_ams(struct maps *maps, struct addr_map_symbol *ams)if (ams->addr <map__start (ams->ms.map) || ams->addr >=map__end (ams->ms.map)) {if (maps ==NULL )return -1 ; - ams->ms.map =maps__find (maps, ams->addr); + ams->ms.map =maps__find (maps, ams->addr);// TODO: map_get if (ams->ms.map ==NULL )return -1 ; }@@ -393,24 +584,28 @@ size_t maps__fprintf(struct maps *maps, FILE *fp)* Find first map where end > map->start. * Same as find_vma() in kernel. */ -static struct rb_node *first_ending_after (struct maps *maps,const struct map *map) +static unsigned int first_ending_after (struct maps *maps,const struct map *map) { -struct rb_root *root; -struct rb_node *next, *first; +struct map **maps_by_address =maps__maps_by_address (maps); +int low =0 , high = (int )maps__nr_maps (maps) -1 , first = high +1 ; + +assert (maps__maps_by_address_sorted (maps)); +if (low <= high &&map__end (maps_by_address[0 ]) >map__start (map)) +return 0 ; - root =maps__entries (maps); - next = root->rb_node; - first =NULL ; -while (next) { - struct map_rb_node *pos = rb_entry(next, struct map_rb_node, rb_node); +while (low <= high) { +int mid = (low + high) /2 ; +struct map *pos = maps_by_address[mid]; -if (map__end (pos->map) >map__start (map)) { - first = next; -if (map__start (pos->map) <=map__start (map)) +if (map__end (pos) >map__start (map)) { + first = mid; +if (map__start (pos) <=map__start (map)) { +/* Entry overlaps map. */ break ; - next = next->rb_left; + } + high = mid -1 ; }else - next = next->rb_right; + low = mid +1 ; }return first; }@@ -419,171 +614,249 @@ static struct rb_node *first_ending_after(struct maps *maps, const struct map *m* Adds new to maps, if new overlaps existing entries then the existing maps are * adjusted or removed so that new fits without overlapping any entries. */ -int maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) +static int __maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) { - -struct rb_node *next; +struct map **maps_by_address;int err =0 ;FILE *fp =debug_file (); -down_write (maps__lock (maps)); +sort_again: +if (!maps__maps_by_address_sorted (maps)) +__maps__sort_by_address (maps); - next =first_ending_after (maps, new); -while (next && !err) { - struct map_rb_node *pos = rb_entry(next, struct map_rb_node, rb_node); - next =rb_next (&pos->rb_node); + maps_by_address =maps__maps_by_address (maps); +/* +* Iterate through entries where the end of the existing entry is +* greater-than the new map's start. +*/ +for (unsigned int i =first_ending_after (maps, new); i <maps__nr_maps (maps); ) { +struct map *pos = maps_by_address[i]; +struct map *before =NULL , *after =NULL ;/* * Stop if current map starts after map->end. * Maps are ordered by start: next will not overlap for sure. */ -if (map__start (pos->map) >=map__end (new)) +if (map__start (pos) >=map__end (new))break ; -if (verbose >=2 ) { - -if (use_browser) { -pr_debug ("overlapping maps in %s (disable tui for more info)\n" , -map__dso (new)->name); - }else { -pr_debug ("overlapping maps:\n" ); -map__fprintf (new, fp); -map__fprintf (pos->map, fp); - } +if (use_browser) { +pr_debug ("overlapping maps in %s (disable tui for more info)\n" , +map__dso (new)->name); + }else if (verbose >=2 ) { +pr_debug ("overlapping maps:\n" ); +map__fprintf (new, fp); +map__fprintf (pos, fp); } -rb_erase_init (&pos->rb_node,maps__entries (maps));/* * Now check if we need to create new maps for areas not * overlapped by the new map: */ -if (map__start (new) >map__start (pos->map)) { -struct map *before =map__clone (pos->map); +if (map__start (new) >map__start (pos)) { +/* Map starts within existing map. Need to shorten the existing map. */ + before =map__clone (pos);if (before ==NULL ) { err = -ENOMEM ; - goto put_map; + goto out_err; } -map__set_end (before,map__start (new)); - err =__maps__insert (maps, before); -if (err) { -map__put (before); - goto put_map; - }if (verbose >=2 && !use_browser)map__fprintf (before, fp); -map__put (before); } - -if (map__end (new) <map__end (pos->map)) { -struct map *after =map__clone (pos->map); +if (map__end (new) <map__end (pos)) { +/* The new map isn't as long as the existing map. */ + after =map__clone (pos);if (after ==NULL ) { +map__zput (before); err = -ENOMEM ; - goto put_map; + goto out_err; }map__set_start (after,map__end (new)); -map__add_pgoff (after,map__end (new) -map__start (pos->map)); -assert (map__map_ip (pos->map,map__end (new)) == -map__map_ip (after,map__end (new))); - err =__maps__insert (maps, after); -if (err) { -map__put (after); - goto put_map; - } +map__add_pgoff (after,map__end (new) -map__start (pos)); +assert (map__map_ip (pos,map__end (new)) == +map__map_ip (after,map__end (new))); +if (verbose >=2 && !use_browser)map__fprintf (after, fp); -map__put (after); } -put_map: -map__put (pos->map); -free (pos); +/* +* If adding one entry, for `before` or `after`, we can replace +* the existing entry. If both `before` and `after` are +* necessary than an insert is needed. If the existing entry +* entirely overlaps the existing entry it can just be removed. +*/ +if (before) { +map__put (maps_by_address[i]); + maps_by_address[i] = before; +/* Maps are still ordered, go to next one. */ + i++; +if (after) { +__maps__insert (maps, after); +map__put (after); +if (!maps__maps_by_address_sorted (maps)) { +/* +* Sorting broken so invariants don't +* hold, sort and go again. +*/ + goto sort_again; + } +/* +* Maps are still ordered, skip after and go to +* next one (terminate loop). +*/ + i++; + } + }else if (after) { +map__put (maps_by_address[i]); + maps_by_address[i] = after; +/* Maps are ordered, go to next one. */ + i++; + }else { +__maps__remove (maps, pos); +/* +* Maps are ordered but no need to increase `i` as the +* later maps were moved down. +*/ + } +check_invariants (maps); }/* Add the map. */ - err =__maps__insert (maps, new); -up_write (maps__lock (maps)); +__maps__insert (maps, new); +out_err:return err; } -int maps__copy_from (struct maps *maps,struct maps *parent) +int maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) {int err; -struct map_rb_node *rb_node; +down_write (maps__lock (maps)); + err =__maps__fixup_overlap_and_insert (maps, new); +up_write (maps__lock (maps)); +return err; +} + +int maps__copy_from (struct maps *dest,struct maps *parent) +{ +/* Note, if struct map were immutable then cloning could use ref counts. */ +struct map **parent_maps_by_address; +int err =0 ; +unsigned int n; + +down_write (maps__lock (dest));down_read (maps__lock (parent)); -maps__for_each_entry (parent, rb_node) { -struct map *new =map__clone (rb_node->map); + parent_maps_by_address =maps__maps_by_address (parent); + n =maps__nr_maps (parent); +if (maps__empty (dest)) { +/* No existing mappings so just copy from parent to avoid reallocs in insert. */ +unsigned int nr_maps_allocated =RC_CHK_ACCESS (parent)->nr_maps_allocated; +struct map **dest_maps_by_address = +malloc (nr_maps_allocated *sizeof (struct map *)); +struct map **dest_maps_by_name =NULL ; -if (new ==NULL ) { +if (!dest_maps_by_address) err = -ENOMEM ; - goto out_unlock; +else { +if (maps__maps_by_name (parent)) { + dest_maps_by_name = +malloc (nr_maps_allocated *sizeof (struct map *)); + } + +RC_CHK_ACCESS (dest)->maps_by_address = dest_maps_by_address; +RC_CHK_ACCESS (dest)->maps_by_name = dest_maps_by_name; +RC_CHK_ACCESS (dest)->nr_maps_allocated = nr_maps_allocated; } - err =unwind__prepare_access (maps, new,NULL ); -if (err) - goto out_unlock; +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos); - err =maps__insert (maps, new); -if (err) - goto out_unlock; +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) { + dest_maps_by_address[i] = new; +if (dest_maps_by_name) + dest_maps_by_name[i] =map__get (new); +RC_CHK_ACCESS (dest)->nr_maps = i +1 ; + } + } +if (err) +map__put (new); + } +maps__set_maps_by_address_sorted (dest,maps__maps_by_address_sorted (parent)); +if (!err) { +RC_CHK_ACCESS (dest)->last_search_by_name_idx = +RC_CHK_ACCESS (parent)->last_search_by_name_idx; +maps__set_maps_by_name_sorted (dest, + dest_maps_by_name && +maps__maps_by_name_sorted (parent)); + }else { +RC_CHK_ACCESS (dest)->last_search_by_name_idx =0 ; +maps__set_maps_by_name_sorted (dest, false); + } + }else { +/* Unexpected copying to a maps containing entries. */ +for (unsigned int i =0 ; !err && i < n; i++) {
tools/perf/util/maps.c static struct rb_node *first_ending_after(struct maps *maps, const struct map *m · line 816
Quoted 10 lines
tools/perf/util/maps.c +8
-1+struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos); -map__put (new); +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) + err =maps__insert (dest, new);
Shouldn't it be __maps__insert()?
Quoted 360 lines
tools/perf/util/maps.c +139
-160+ } +map__put (new); + } } - - err =0 ; -out_unlock:up_read (maps__lock (parent)); +up_write (maps__lock (dest));return err; } -struct map *maps__find (struct maps *maps,u64 ip) +static int map__addr_cmp (const void *key,const void *entry) { -struct rb_node *p; -struct map_rb_node *m; - +const u64 ip = *(const u64 *)key; +const struct map *map = *(const struct map *const *)entry; -down_read (maps__lock (maps)); - - p =maps__entries (maps)->rb_node; -while (p !=NULL ) { - m = rb_entry(p, struct map_rb_node, rb_node); -if (ip <map__start (m->map)) - p = p->rb_left; -else if (ip >=map__end (m->map)) - p = p->rb_right; -else - goto out; - } - - m =NULL ; -out: -up_read (maps__lock (maps)); -return m ? m->map :NULL ; +if (ip <map__start (map)) +return -1 ; +if (ip >=map__end (map)) +return 1 ; +return 0 ; } -static int map__strcmp (const void *a,const void *b) +struct map *maps__find (struct maps *maps,u64 ip) { -const struct map *map_a = *(const struct map **)a; -const struct map *map_b = *(const struct map **)b; -const struct dso *dso_a =map__dso (map_a); -const struct dso *dso_b =map__dso (map_b); -int ret =strcmp (dso_a->short_name, dso_b->short_name); - -if (ret ==0 && map_a != map_b) { -/* -* Ensure distinct but name equal maps have an order in part to -* aid reference counting. -*/ - ret = (int )map__start (map_a) - (int )map__start (map_b); -if (ret ==0 ) - ret = (int )((intptr_t )map_a - (intptr_t )map_b); +struct map *result =NULL ; +bool done = false; + +/* See locking/sorting note. */ +while (!done) { +down_read (maps__lock (maps)); +if (maps__maps_by_address_sorted (maps)) { +struct map **mapp = +bsearch (&ip,maps__maps_by_address (maps),maps__nr_maps (maps), +sizeof (*mapp), map__addr_cmp); + +if (mapp) + result = *mapp;// map__get(*mapp); + done = true; + } +up_read (maps__lock (maps)); +if (!done) +maps__sort_by_address (maps); } - -return ret; +return result; }static int map__strcmp_name (const void *name,const void *b)@@ -593,126 +866,113 @@ static int map__strcmp_name(const void *name, const void *b)return strcmp (name, dso->short_name); } -void __maps__sort_by_name (struct maps *maps) -{ -qsort (maps__maps_by_name (maps),maps__nr_maps (maps),sizeof (struct map *), map__strcmp); -} - -static int map__groups__sort_by_name_from_rbtree (struct maps *maps) -{ -struct map_rb_node *rb_node; -struct map **maps_by_name =realloc (maps__maps_by_name (maps), -maps__nr_maps (maps) *sizeof (struct map *)); -int i =0 ; - -if (maps_by_name ==NULL ) -return -1 ; - -up_read (maps__lock (maps)); -down_write (maps__lock (maps)); - -RC_CHK_ACCESS (maps)->maps_by_name = maps_by_name; -RC_CHK_ACCESS (maps)->nr_maps_allocated =maps__nr_maps (maps); - -maps__for_each_entry (maps, rb_node) - maps_by_name[i++] =map__get (rb_node->map); - -__maps__sort_by_name (maps); - -up_write (maps__lock (maps)); -down_read (maps__lock (maps)); - -return 0 ; -} - -static struct map *__maps__find_by_name (struct maps *maps,const char *name) +struct map *maps__find_by_name (struct maps *maps,const char *name) { -struct map **mapp; +struct map *result =NULL ; +bool done = false; -if (maps__maps_by_name (maps) ==NULL && -map__groups__sort_by_name_from_rbtree (maps)) -return NULL ; +/* See locking/sorting note. */ +while (!done) { +unsigned int i; - mapp =bsearch (name,maps__maps_by_name (maps),maps__nr_maps (maps), -sizeof (*mapp), map__strcmp_name); -if (mapp) -return *mapp; -return NULL ; -} +down_read (maps__lock (maps)); -struct map *maps__find_by_name (struct maps *maps,const char *name) -{ -struct map_rb_node *rb_node; -struct map *map; - -down_read (maps__lock (maps)); +/* First check last found entry. */ + i =RC_CHK_ACCESS (maps)->last_search_by_name_idx; +if (i <maps__nr_maps (maps) &&maps__maps_by_name (maps)) { +struct dso *dso =map__dso (maps__maps_by_name (maps)[i]); +if (dso &&strcmp (dso->short_name, name) ==0 ) { + result =maps__maps_by_name (maps)[i];// TODO: map__get + done = true; + } + } -if (RC_CHK_ACCESS (maps)->last_search_by_name) { -const struct dso *dso =map__dso (RC_CHK_ACCESS (maps)->last_search_by_name); +/* Second search sorted array. */ +if (!done &&maps__maps_by_name_sorted (maps)) { +struct map **mapp = +bsearch (name,maps__maps_by_name (maps),maps__nr_maps (maps), +sizeof (*mapp), map__strcmp_name); -if (strcmp (dso->short_name, name) ==0 ) { - map =RC_CHK_ACCESS (maps)->last_search_by_name; - goto out_unlock; +if (mapp) { + result = *mapp;// TODO: map__get + i = mapp -maps__maps_by_name (maps); +RC_CHK_ACCESS (maps)->last_search_by_name_idx = i; + } + done = true; } - } -/* -* If we have maps->maps_by_name, then the name isn't in the rbtree, -* as maps->maps_by_name mirrors the rbtree when lookups by name are -* made. -*/ - map =__maps__find_by_name (maps, name); -if (map ||maps__maps_by_name (maps) !=NULL ) - goto out_unlock; - -/* Fallback to traversing the rbtree... */ -maps__for_each_entry (maps, rb_node) { -struct dso *dso; - - map = rb_node->map; - dso =map__dso (map); -if (strcmp (dso->short_name, name) ==0 ) { -RC_CHK_ACCESS (maps)->last_search_by_name = map; - goto out_unlock; +up_read (maps__lock (maps)); +if (!done) { +/* Sort and retry binary search. */ +if (maps__sort_by_name (maps)) { +/* +* Memory allocation failed do linear search +* through address sorted maps. +*/ +struct map **maps_by_address; +unsigned int n; + +down_read (maps__lock (maps)); + maps_by_address =maps__maps_by_address (maps); + n =maps__nr_maps (maps); +for (i =0 ; i < n; i++) { +struct map *pos = maps_by_address[i]; +struct dso *dso =map__dso (pos); + +if (dso &&strcmp (dso->short_name, name) ==0 ) { + result = pos;// TODO: map__get +break ; + } + } +up_read (maps__lock (maps)); + done = true; + } } } - map =NULL ; - -out_unlock: -up_read (maps__lock (maps)); -return map; +return result; }struct map *maps__find_next_entry (struct maps *maps,struct map *map) { -struct map_rb_node *rb_node =maps__find_node (maps, map); -struct map_rb_node *next =map_rb_node__next (rb_node); +unsigned int i; +struct map *result =NULL ; -if (next) -return next->map; +down_read (maps__lock (maps)); + i =maps__by_address_index (maps, map); +if (i <maps__nr_maps (maps)) + result =maps__maps_by_address (maps)[i];// TODO: map__get -return NULL ; +up_read (maps__lock (maps)); +return result; }void maps__fixup_end (struct maps *maps) { -struct map_rb_node *prev =NULL , *curr; +struct map **maps_by_address; +unsigned int n;down_write (maps__lock (maps)); +if (!maps__maps_by_address_sorted (maps)) +__maps__sort_by_address (maps); -maps__for_each_entry (maps, curr) { -if (prev && (!map__end (prev->map) ||map__end (prev->map) >map__start (curr->map))) -map__set_end (prev->map,map__start (curr->map)); + maps_by_address =maps__maps_by_address (maps); + n =maps__nr_maps (maps); +for (unsigned int i =1 ; i < n; i++) { +struct map *prev = maps_by_address[i -1 ]; +struct map *curr = maps_by_address[i]; - prev = curr; +if (!map__end (prev) ||map__end (prev) >map__start (curr)) +map__set_end (prev,map__start (curr)); }/* * We still haven't the actual symbols, so guess the * last map final address. */ -if (curr && !map__end (curr->map)) -map__set_end (curr->map, ~0ULL ); +if (n >0 && !map__end (maps_by_address[n -1 ])) +map__set_end (maps_by_address[n -1 ], ~0ULL ); + +RC_CHK_ACCESS (maps)->ends_broken = false;up_write (maps__lock (maps)); }@@ -723,117 +983,92 @@ void maps__fixup_end(struct maps *maps)*/ int maps__merge_in (struct maps *kmaps,struct map *new_map) { -struct map_rb_node *rb_node; -struct rb_node *first; -bool overlaps; -LIST_HEAD (merged); -int err =0 ; +unsigned int first_after_, kmaps__nr_maps; +struct map **kmaps_maps_by_address; +struct map **merged_maps_by_address; +unsigned int merged_nr_maps_allocated; + +/* First try under a read lock. */ +while (true) { +down_read (maps__lock (kmaps)); +if (maps__maps_by_address_sorted (kmaps)) +break ; -down_read (maps__lock (kmaps)); - first =first_ending_after (kmaps, new_map); - rb_node = first ? rb_entry(first, struct map_rb_node, rb_node) : NULL; - overlaps = rb_node &&map__start (rb_node->map) <map__end (new_map); -up_read (maps__lock (kmaps)); +up_read (maps__lock (kmaps)); + +/* First after binary search requires sorted maps. Sort and try again. */ +maps__sort_by_address (kmaps); + } + first_after_ =first_ending_after (kmaps, new_map); + kmaps_maps_by_address =maps__maps_by_address (kmaps); -if (!overlaps) +if (first_after_ >=maps__nr_maps (kmaps) || +map__start (kmaps_maps_by_address[first_after_]) >=map__end (new_map)) { +/* No overlap so regular insert suffices. */ +up_read (maps__lock (kmaps));return maps__insert (kmaps, new_map); + } +up_read (maps__lock (kmaps)); -maps__for_each_entry (kmaps, rb_node) { -struct map *old_map = rb_node->map; +/* Plain insert with a read-lock failed, try again now with the write lock. */ +down_write (maps__lock (kmaps)); +if (!maps__maps_by_address_sorted (kmaps)) +__maps__sort_by_address (kmaps); -/* no overload with this one */ -if (map__end (new_map) <map__start (old_map) || -map__start (new_map) >=map__end (old_map)) -continue ; + first_after_ =first_ending_after (kmaps, new_map); + kmaps_maps_by_address =maps__maps_by_address (kmaps); + kmaps__nr_maps =maps__nr_maps (kmaps); -if (map__start (new_map) <map__start (old_map)) { -/* -* |new...... -* |old.... -*/ -if (map__end (new_map) <map__end (old_map)) { -/* -* |new......| -> |new..| -* |old....| -> |old....| -*/ -map__set_end (new_map,map__start (old_map)); - }else {
tools/perf/util/maps.c void maps__fixup_end(struct maps *maps) · line 1026
Quoted 10 lines
tools/perf/util/maps.c +5
-5-/* -* |new.............| -> |new..| |new..| -* |old....| -> |old....| -*/ -struct map_list_node *m =map_list_node__new (); +if (first_after_ >= kmaps__nr_maps || +map__start (kmaps_maps_by_address[first_after_]) >=map__end (new_map)) { +/* No overlap so regular insert suffices. */ +up_write (maps__lock (kmaps)); +return maps__insert (kmaps, new_map);
I think it could be:
ret = __maps__insert(kmaps, new_map);
up_write(maps__lock(kmaps));
return ret;Quoted 6 lines
tools/perf/util/maps.c +6
-0+ } +/* Array to merge into, possibly 1 more for the sake of new_map. */ + merged_nr_maps_allocated =RC_CHK_ACCESS (kmaps)->nr_maps_allocated; +if (kmaps__nr_maps +1 == merged_nr_maps_allocated) + merged_nr_maps_allocated++; +
tools/perf/util/maps.c void maps__fixup_end(struct maps *maps) · line 1042
Quoted 10 lines
tools/perf/util/maps.c +10
-0+ merged_maps_by_address =malloc (merged_nr_maps_allocated *sizeof (*merged_maps_by_address)); +if (!merged_maps_by_address) { +up_write (maps__lock (kmaps)); +return -ENOMEM ; + } +RC_CHK_ACCESS (kmaps)->maps_by_address = merged_maps_by_address; +RC_CHK_ACCESS (kmaps)->maps_by_address_sorted = true; +zfree (&RC_CHK_ACCESS (kmaps)->maps_by_name); +RC_CHK_ACCESS (kmaps)->maps_by_name_sorted = false; +RC_CHK_ACCESS (kmaps)->nr_maps_allocated = merged_nr_maps_allocated;
Why not use the accessor functions?
Thanks,
Namhyung
Quoted 181 lines
tools/perf/util/maps.c +17
-59-if (!m) { - err = -ENOMEM ; - goto out; - } +/* Copy entries before the new_map that can't overlap. */ +for (unsigned int i =0 ; i < first_after_; i++) + merged_maps_by_address[i] =map__get (kmaps_maps_by_address[i]); - m->map =map__clone (new_map); -if (!m->map) { -free (m); - err = -ENOMEM ; - goto out; - } +RC_CHK_ACCESS (kmaps)->nr_maps = first_after_; -map__set_end (m->map,map__start (old_map)); -list_add_tail (&m->node, &merged); -map__add_pgoff (new_map,map__end (old_map) -map__start (new_map)); -map__set_start (new_map,map__end (old_map)); - } - }else { -/* -* |new...... -* |old.... -*/ -if (map__end (new_map) <map__end (old_map)) { -/* -* |new..| -> x -* |old.........| -> |old.........| -*/ -map__put (new_map); - new_map =NULL ; -break ; - }else { -/* -* |new......| -> |new...| -* |old....| -> |old....| -*/ -map__add_pgoff (new_map,map__end (old_map) -map__start (new_map)); -map__set_start (new_map,map__end (old_map)); - } - } - } +/* Add the new map, it will be split when the later overlapping mappings are added. */ +__maps__insert (kmaps, new_map); -out: -while (!list_empty (&merged)) { -struct map_list_node *old_node; +/* Insert mappings after new_map, splitting new_map in the process. */ +for (unsigned int i = first_after_; i < kmaps__nr_maps; i++) +__maps__fixup_overlap_and_insert (kmaps, kmaps_maps_by_address[i]); - old_node = list_entry(merged.next, struct map_list_node, node); -list_del_init (&old_node->node); -if (!err) - err =maps__insert (kmaps, old_node->map); -map__put (old_node->map); -free (old_node); - } +/* Copy the maps from merged into kmaps. */ +for (unsigned int i =0 ; i < kmaps__nr_maps; i++) +map__zput (kmaps_maps_by_address[i]); -if (new_map) { -if (!err) - err =maps__insert (kmaps, new_map); -map__put (new_map); - } -return err; +free (kmaps_maps_by_address); +up_write (maps__lock (kmaps)); +return 0 ; }void maps__load_first (struct maps *maps) { -struct map_rb_node *first; -down_read (maps__lock (maps)); - first =maps__first (maps); -if (first) -map__load (first->map); +if (maps__nr_maps (maps) >0 ) +map__load (maps__maps_by_address (maps)[0 ]);up_read (maps__lock (maps)); }tools/perf/util/maps.h +44
-11@@ -25,21 +25,56 @@ static inline struct map_list_node *map_list_node__new(void)return malloc (sizeof (struct map_list_node )); } -struct map *maps__find (struct maps *maps,u64 addr); +/* +* Locking/sorting note: +* +* Sorting is done with the write lock, iteration and binary searching happens +* under the read lock requiring being sorted. There is a race between sorting +* releasing the write lock and acquiring the read lock for iteration/searching +* where another thread could insert and break the sorting of the maps. In +* practice inserting maps should be rare meaning that the race shouldn't lead +* to live lock. Removal of maps doesn't break being sorted. +*/ DECLARE_RC_STRUCT (maps) { -struct rb_root entries;struct rw_semaphore lock; -struct machine *machine; -struct map *last_search_by_name; +/** +* @maps_by_address: array of maps sorted by their starting address if +* maps_by_address_sorted is true. +*/ +struct map **maps_by_address; +/** +* @maps_by_name: optional array of maps sorted by their dso name if +* maps_by_name_sorted is true. +*/ struct map **maps_by_name; -refcount_t refcnt; -unsigned int nr_maps; -unsigned int nr_maps_allocated; +struct machine *machine;#ifdef HAVE_LIBUNWIND_SUPPORT -void *addr_space; +void *addr_space;const struct unwind_libunwind_ops *unwind_libunwind_ops;#endif +refcount_t refcnt; +/** +* @nr_maps: number of maps_by_address, and possibly maps_by_name, +* entries that contain maps. +*/ +unsigned int nr_maps; +/** +* @nr_maps_allocated: number of entries in maps_by_address and possibly +* maps_by_name. +*/ +unsigned int nr_maps_allocated; +/** +* @last_search_by_name_idx: cache of last found by name entry's index +* as frequent searches for the same dso name are common. +*/ +unsigned int last_search_by_name_idx; +/** @maps_by_address_sorted: is maps_by_address sorted. */ +bool maps_by_address_sorted; +/** @maps_by_name_sorted: is maps_by_name sorted. */ +bool maps_by_name_sorted; +/** @ends_broken: does the map contain a map where end values are unset/unsorted? */ +bool ends_broken; };#define KMAP_NAME_LEN 256@@ -102,6 +137,7 @@ size_t maps__fprintf(struct maps *maps, FILE *fp);int maps__insert (struct maps *maps,struct map *map);void maps__remove (struct maps *maps,struct map *map); +struct map *maps__find (struct maps *maps,u64 addr);struct symbol *maps__find_symbol (struct maps *maps,u64 addr,struct map **mapp);struct symbol *maps__find_symbol_by_name (struct maps *maps,const char *name,struct map **mapp);@@ -117,8 +153,6 @@ struct map *maps__find_next_entry(struct maps *maps, struct map *map);int maps__merge_in (struct maps *kmaps,struct map *new_map); -void __maps__sort_by_name (struct maps *maps); -void maps__fixup_end (struct maps *maps);void maps__load_first (struct maps *maps); --2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerRe: [PATCH v7 01/25] perf maps: Switch from rbtree to lazily sorted array for addresses
Quoted 604 lines · Namhyung Kim
Hi Ian,
Quoted 24 lines · Ian Rogers
Maps is a collection of maps primarily sorted by the starting address of the map. Prior to this change the maps were held in an rbtree requiring 4 pointers per node. Prior to reference count checking, the rbnode was embedded in the map so 3 pointers per node were necessary. This change switches the rbtree to an array lazily sorted by address, much as the array sorting nodes by name. 1 pointer is needed per node, but to avoid excessive resizing the backing array may be twice the number of used elements. Meaning the memory overhead is roughly half that of the rbtree. For a perf record with "--no-bpf-event -g -a" of true, the memory overhead of perf inject is reduce fom 3.3MB to 3MB, so 10% or 300KB is saved.
Map inserts always happen at the end of the array. The code tracks whether the insertion violates the sorting property. O(log n) rb-tree complexity is switched to O(1).
Remove slides the array, so O(log n) rb-tree complexity is degraded to O(n).
A find may need to sort the array using qsort which is
O(n*log n), but in general the maps should be sorted and so average performance should be O(log n) as with the rbtree.Quoted 6 lines
An rbtree node consumes a cache line, but with the array 4 nodes fit on a cache line. Iteration is simplified to scanning an array rather than pointer chasing.
Overall it is expected the performance after the change should be comparable to before, but with half of the memory consumed.
I don't know how much performance impact it would have but I guess search/iteration would be the most frequent operation. So I like the memory saving it can bring.
Quoted 565 lines
To avoid a list and repeated logic around splitting maps,
maps__merge_inis rewritten in terms ofmaps__fixup_overlap_and_insert.maps_merge_insplits the given mapping inserting remaining gaps.maps__fixup_overlap_and_insertsplits the existing mappings, then adds the incoming mapping. By adding the new mapping first, then re-inserting the existing mappings the splitting behavior matches.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/maps.c | 3 + tools/perf/util/map.c | 1 + tools/perf/util/maps.c | 1183 +++++++++++++++++++++++---------------- tools/perf/util/maps.h | 54 +- 4 files changed, 757 insertions(+), 484 deletions(-)tools/perf/tests/maps.c +3
-0@@ -156,6 +156,9 @@ static int test__maps__merge_in(struct test_suite *t __maybe_unused, int subtestTEST_ASSERT_VAL ("merge check failed" , !ret);maps__zput (maps); +map__zput (map_kcore1); +map__zput (map_kcore2); +map__zput (map_kcore3);return TEST_OK ; }tools/perf/util/map.c +1
-0@@ -168,6 +168,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,if (dso ==NULL ) goto out_delete; +assert (!dso->kernel);map__init (result, start, start + len, pgoff, dso);if (anon || no_dso) {tools/perf/util/maps.c +293
-142@@ -10,286 +10,477 @@#include "ui/ui.h" #include "unwind.h" -struct map_rb_node { -struct rb_node rb_node; -struct map *map; -}; - -#define maps__for_each_entry (maps, map) \ - for (map = maps__first(maps); map; map = map_rb_node__next(map)) +static void check_invariants (const struct maps *maps__maybe_unused ) +{ +#ifndef NDEBUG +assert (RC_CHK_ACCESS (maps)->nr_maps <=RC_CHK_ACCESS (maps)->nr_maps_allocated); +for (unsigned int i =0 ; i <RC_CHK_ACCESS (maps)->nr_maps; i++) { +struct map *map =RC_CHK_ACCESS (maps)->maps_by_address[i]; + +/* Check map is well-formed. */ +assert (map__end (map) ==0 ||map__start (map) <=map__end (map)); +/* Expect at least 1 reference count. */ +assert (refcount_read (map__refcnt (map)) >0 ); + +if (map__dso (map) &&map__dso (map)->kernel) +assert (RC_CHK_EQUAL (map__kmap (map)->kmaps, maps)); + +if (i >0 ) { +struct map *prev =RC_CHK_ACCESS (maps)->maps_by_address[i -1 ]; + +/* If addresses are sorted... */ +if (RC_CHK_ACCESS (maps)->maps_by_address_sorted) { +/* Maps should be in start address order. */ +assert (map__start (prev) <=map__start (map)); +/* +* If the ends of maps aren't broken (during +* construction) then they should be ordered +* too. +*/ +if (!RC_CHK_ACCESS (maps)->ends_broken) { +assert (map__end (prev) <=map__end (map)); +assert (map__end (prev) <=map__start (map) || +map__start (prev) ==map__start (map)); + } + } + } + } +if (RC_CHK_ACCESS (maps)->maps_by_name) { +for (unsigned int i =0 ; i <RC_CHK_ACCESS (maps)->nr_maps; i++) { +struct map *map =RC_CHK_ACCESS (maps)->maps_by_name[i]; -#define maps__for_each_entry_safe (maps, map, next) \ - for (map = maps__first(maps), next = map_rb_node__next(map); map; \ - map = next, next = map_rb_node__next(map)) +/* +* Maps by name maps should be in maps_by_address, so +* the reference count should be higher. +*/ +assert (refcount_read (map__refcnt (map)) >1 ); + } + } +#endif +} -static struct rb_root *maps__entries (struct maps *maps) +static struct map **maps__maps_by_address (const struct maps *maps) { -return &RC_CHK_ACCESS (maps)->entries; +return RC_CHK_ACCESS (maps)->maps_by_address; } -static struct rw_semaphore *maps__lock (struct maps *maps) +static void maps__set_maps_by_address (struct maps *maps,struct map **new) { -return &RC_CHK_ACCESS (maps)->lock; +RC_CHK_ACCESS (maps)->maps_by_address = new; + } -static struct map **maps__maps_by_name (struct maps *maps) +/* Not in the header, to aid reference counting. */ +static struct map **maps__maps_by_name (const struct maps *maps) {return RC_CHK_ACCESS (maps)->maps_by_name; + } -static struct map_rb_node *maps__first (struct maps *maps) +static void maps__set_maps_by_name (struct maps *maps,struct map **new) { -struct rb_node *first =rb_first (maps__entries (maps)); +RC_CHK_ACCESS (maps)->maps_by_name = new; -if (first) - return rb_entry(first, struct map_rb_node, rb_node); -return NULL ; } -static struct map_rb_node *map_rb_node__next (struct map_rb_node *node) +static bool maps__maps_by_address_sorted (const struct maps *maps) { -struct rb_node *next; - -if (!node) -return NULL ; - - next =rb_next (&node->rb_node); +return RC_CHK_ACCESS (maps)->maps_by_address_sorted; +} -if (!next) -return NULL ; +static void maps__set_maps_by_address_sorted (struct maps *maps,bool value) +{ +RC_CHK_ACCESS (maps)->maps_by_address_sorted = value; +} - return rb_entry(next, struct map_rb_node, rb_node); +static bool maps__maps_by_name_sorted (const struct maps *maps) +{ +return RC_CHK_ACCESS (maps)->maps_by_name_sorted; } -static struct map_rb_node *maps__find_node (struct maps *maps,struct map *map) +static void maps__set_maps_by_name_sorted (struct maps *maps,bool value) { -struct map_rb_node *rb_node; +RC_CHK_ACCESS (maps)->maps_by_name_sorted = value; +} -maps__for_each_entry (maps, rb_node) { -if (rb_node->RC_CHK_ACCESS (map) ==RC_CHK_ACCESS (map)) -return rb_node; - } -return NULL ; +static struct rw_semaphore *maps__lock (struct maps *maps) +{ +/* +* When the lock is acquired or released the maps invariants should +* hold. +*/ +check_invariants (maps); +return &RC_CHK_ACCESS (maps)->lock; }static void maps__init (struct maps *maps,struct machine *machine) { -refcount_set (maps__refcnt (maps),1 );init_rwsem (maps__lock (maps)); -RC_CHK_ACCESS (maps)->entries =RB_ROOT ; +RC_CHK_ACCESS (maps)->maps_by_address =NULL ; +RC_CHK_ACCESS (maps)->maps_by_name =NULL ;RC_CHK_ACCESS (maps)->machine = machine; -RC_CHK_ACCESS (maps)->last_search_by_name =NULL ; +#ifdef HAVE_LIBUNWIND_SUPPORT +RC_CHK_ACCESS (maps)->addr_space =NULL ; +RC_CHK_ACCESS (maps)->unwind_libunwind_ops =NULL ; +#endif +refcount_set (maps__refcnt (maps),1 );RC_CHK_ACCESS (maps)->nr_maps =0 ; -RC_CHK_ACCESS (maps)->maps_by_name =NULL ; +RC_CHK_ACCESS (maps)->nr_maps_allocated =0 ; +RC_CHK_ACCESS (maps)->last_search_by_name_idx =0 ; +RC_CHK_ACCESS (maps)->maps_by_address_sorted = true; +RC_CHK_ACCESS (maps)->maps_by_name_sorted = false; } -static void __maps__free_maps_by_name (struct maps *maps) +static void maps__exit (struct maps *maps) { -/* -* Free everything to try to do it from the rbtree in the next search -*/ -for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) -map__put (maps__maps_by_name (maps)[i]); +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); -zfree (&RC_CHK_ACCESS (maps)->maps_by_name); -RC_CHK_ACCESS (maps)->nr_maps_allocated =0 ; +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +map__zput (maps_by_address[i]); +if (maps_by_name) +map__zput (maps_by_name[i]); + } +zfree (&maps_by_address); +zfree (&maps_by_name); +unwind__finish_access (maps); } -static int __maps__insert (struct maps *maps,struct map *map) +struct maps *maps__new (struct machine *machine) { -struct rb_node **p = &maps__entries (maps)->rb_node; -struct rb_node *parent =NULL ; -const u64 ip =map__start (map); -struct map_rb_node *m, *new_rb_node; - - new_rb_node =malloc (sizeof (*new_rb_node)); -if (!new_rb_node) -return -ENOMEM ; - -RB_CLEAR_NODE (&new_rb_node->rb_node); - new_rb_node->map =map__get (map); +struct maps *result; +RC_STRUCT (maps) *maps =zalloc (sizeof (*maps)); -while (*p !=NULL ) { - parent = *p; - m = rb_entry(parent, struct map_rb_node, rb_node); -if (ip <map__start (m->map)) - p = &(*p)->rb_left; -else - p = &(*p)->rb_right; - } +if (ADD_RC_CHK (result, maps)) +maps__init (result, machine); -rb_link_node (&new_rb_node->rb_node, parent, p); -rb_insert_color (&new_rb_node->rb_node,maps__entries (maps)); -return 0 ; +return result; } -int maps__insert (struct maps *maps,struct map *map) +static void maps__delete (struct maps *maps) { -int err; -const struct dso *dso =map__dso (map); - -down_write (maps__lock (maps)); - err =__maps__insert (maps, map); -if (err) - goto out; +maps__exit (maps); +RC_CHK_FREE (maps); +} - ++RC_CHK_ACCESS (maps)->nr_maps; +struct maps *maps__get (struct maps *maps) +{ +struct maps *result; -if (dso && dso->kernel) { -struct kmap *kmap =map__kmap (map); +if (RC_CHK_GET (result, maps)) +refcount_inc (maps__refcnt (maps)); -if (kmap) - kmap->kmaps = maps; -else -pr_err ("Internal error: kernel dso with non kernel map\n" ); - } +return result; +} +void maps__put (struct maps *maps) +{ +if (maps &&refcount_dec_and_test (maps__refcnt (maps))) +maps__delete (maps); +else +RC_CHK_PUT (maps); +} +static void __maps__free_maps_by_name (struct maps *maps) +{/* -* If we already performed some search by name, then we need to add the just -* inserted map and resort. +* Free everything to try to do it from the rbtree in the next search */ -if (maps__maps_by_name (maps)) { -if (maps__nr_maps (maps) >RC_CHK_ACCESS (maps)->nr_maps_allocated) { -int nr_allocate =maps__nr_maps (maps) *2 ; -struct map **maps_by_name =realloc (maps__maps_by_name (maps), - nr_allocate *sizeof (map)); +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) +map__put (maps__maps_by_name (maps)[i]); -if (maps_by_name ==NULL ) { -__maps__free_maps_by_name (maps); - err = -ENOMEM ; - goto out; - } +zfree (&RC_CHK_ACCESS (maps)->maps_by_name); +} -RC_CHK_ACCESS (maps)->maps_by_name = maps_by_name; -RC_CHK_ACCESS (maps)->nr_maps_allocated = nr_allocate; +static int map__start_cmp (const void *a,const void *b) +{ +const struct map *map_a = *(const struct map *const *)a; +const struct map *map_b = *(const struct map *const *)b; +u64 map_a_start =map__start (map_a); +u64 map_b_start =map__start (map_b); + +if (map_a_start == map_b_start) { +u64 map_a_end =map__end (map_a); +u64 map_b_end =map__end (map_b); + +if (map_a_end == map_b_end) { +/* Ensure maps with the same addresses have a fixed order. */ +if (RC_CHK_ACCESS (map_a) ==RC_CHK_ACCESS (map_b)) +return 0 ; +return (intptr_t )RC_CHK_ACCESS (map_a) > (intptr_t )RC_CHK_ACCESS (map_b) + ?1 :-1 ; } -maps__maps_by_name (maps)[maps__nr_maps (maps) -1 ] =map__get (map); -__maps__sort_by_name (maps); +return map_a_end > map_b_end ?1 :-1 ; } - out: -up_write (maps__lock (maps)); -return err; +return map_a_start > map_b_start ?1 :-1 ; } -static void __maps__remove (struct maps *maps,struct map_rb_node *rb_node) +static void __maps__sort_by_address (struct maps *maps) { -rb_erase_init (&rb_node->rb_node,maps__entries (maps)); -map__put (rb_node->map); -free (rb_node); +if (maps__maps_by_address_sorted (maps)) +return ; + +qsort (maps__maps_by_address (maps), +maps__nr_maps (maps), +sizeof (struct map *), + map__start_cmp); +maps__set_maps_by_address_sorted (maps, true); } -void maps__remove (struct maps *maps,struct map *map) +static void maps__sort_by_address (struct maps *maps) { -struct map_rb_node *rb_node; -down_write (maps__lock (maps)); -if (RC_CHK_ACCESS (maps)->last_search_by_name == map) -RC_CHK_ACCESS (maps)->last_search_by_name =NULL ; - - rb_node =maps__find_node (maps, map); -assert (rb_node->RC_CHK_ACCESS (map) ==RC_CHK_ACCESS (map)); -__maps__remove (maps, rb_node); -if (maps__maps_by_name (maps)) -__maps__free_maps_by_name (maps); - --RC_CHK_ACCESS (maps)->nr_maps; +__maps__sort_by_address (maps);up_write (maps__lock (maps)); } -static void __maps__purge (struct maps *maps) +static int map__strcmp (const void *a,const void *b) { -struct map_rb_node *pos, *next; - -if (maps__maps_by_name (maps)) -__maps__free_maps_by_name (maps); +const struct map *map_a = *(const struct map *const *)a; +const struct map *map_b = *(const struct map *const *)b; +const struct dso *dso_a =map__dso (map_a); +const struct dso *dso_b =map__dso (map_b); +int ret =strcmp (dso_a->short_name, dso_b->short_name); -maps__for_each_entry_safe (maps, pos, next) { -rb_erase_init (&pos->rb_node,maps__entries (maps)); -map__put (pos->map); -free (pos); +if (ret ==0 &&RC_CHK_ACCESS (map_a) !=RC_CHK_ACCESS (map_b)) { +/* Ensure distinct but name equal maps have an order. */ +return map__start_cmp (a, b); } +return ret; } -static void maps__exit (struct maps *maps) +static int maps__sort_by_name (struct maps *maps) { +int err =0 ;down_write (maps__lock (maps)); -__maps__purge (maps); +if (!maps__maps_by_name_sorted (maps)) { +struct map **maps_by_name =maps__maps_by_name (maps); + +if (!maps_by_name) { + maps_by_name =malloc (RC_CHK_ACCESS (maps)->nr_maps_allocated * +sizeof (*maps_by_name)); +if (!maps_by_name) + err = -ENOMEM ; +else { +struct map **maps_by_address =maps__maps_by_address (maps); +unsigned int n =maps__nr_maps (maps); + +maps__set_maps_by_name (maps, maps_by_name); +for (unsigned int i =0 ; i < n; i++) + maps_by_name[i] =map__get (maps_by_address[i]); + } + } +if (!err) { +qsort (maps_by_name, +maps__nr_maps (maps), +sizeof (struct map *), + map__strcmp); +maps__set_maps_by_name_sorted (maps, true); + } + }up_write (maps__lock (maps)); +return err; } -bool maps__empty (struct maps *maps) +static unsigned int maps__by_address_index (const struct maps *maps,const struct map *map) { -return !maps__first (maps); +struct map **maps_by_address =maps__maps_by_address (maps); + +if (maps__maps_by_address_sorted (maps)) { +struct map **mapp = +bsearch (&map,maps__maps_by_address (maps),maps__nr_maps (maps), +sizeof (*mapp), map__start_cmp); + +if (mapp) +return mapp - maps_by_address; + }else { +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +if (RC_CHK_ACCESS (maps_by_address[i]) ==RC_CHK_ACCESS (map)) +return i; + } + } +pr_err ("Map missing from maps" ); +return -1 ; } -struct maps *maps__new (struct machine *machine) +static unsigned int maps__by_name_index (const struct maps *maps,const struct map *map) { -struct maps *result; -RC_STRUCT (maps) *maps =zalloc (sizeof (*maps)); +struct map **maps_by_name =maps__maps_by_name (maps); + +if (maps__maps_by_name_sorted (maps)) { +struct map **mapp = +bsearch (&map, maps_by_name,maps__nr_maps (maps), +sizeof (*mapp), map__strcmp); + +if (mapp) +return mapp - maps_by_name; + }else { +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +if (RC_CHK_ACCESS (maps_by_name[i]) ==RC_CHK_ACCESS (map)) +return i; + } + } +pr_err ("Map missing from maps" ); +return -1 ; +} -if (ADD_RC_CHK (result, maps)) -maps__init (result, machine); +static int __maps__insert (struct maps *maps,struct map *new) +{ +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); +const struct dso *dso =map__dso (new); +unsigned int nr_maps =maps__nr_maps (maps); +unsigned int nr_allocate =RC_CHK_ACCESS (maps)->nr_maps_allocated; + +if (nr_maps +1 > nr_allocate) { + nr_allocate = !nr_allocate ?32 : nr_allocate *2 ; + + maps_by_address =realloc (maps_by_address, nr_allocate *sizeof (new)); +if (!maps_by_address) +return -ENOMEM ; + +maps__set_maps_by_address (maps, maps_by_address); +if (maps_by_name) { + maps_by_name =realloc (maps_by_name, nr_allocate *sizeof (new)); +if (!maps_by_name) { +/* +* If by name fails, just disable by name and it will +* recompute next time it is required. +*/ +__maps__free_maps_by_name (maps); + } +maps__set_maps_by_name (maps, maps_by_name); + } +RC_CHK_ACCESS (maps)->nr_maps_allocated = nr_allocate; + } +/* Insert the value at the end. */ + maps_by_address[nr_maps] =map__get (new); +if (maps_by_name) + maps_by_name[nr_maps] =map__get (new); -return result; + nr_maps++; +RC_CHK_ACCESS (maps)->nr_maps = nr_maps; + +/* +* Recompute if things are sorted. If things are inserted in a sorted +* manner, for example by processing /proc/pid/maps, then no +* sorting/resorting will be necessary. +*/ +if (nr_maps ==1 ) { +/* If there's just 1 entry then maps are sorted. */ +maps__set_maps_by_address_sorted (maps, true); +maps__set_maps_by_name_sorted (maps, maps_by_name !=NULL ); + }else { +/* Sorted if maps were already sorted and this map starts after the last one. */ +maps__set_maps_by_address_sorted (maps, +maps__maps_by_address_sorted (maps) && +map__end (maps_by_address[nr_maps -2 ]) <=map__start (new)); +maps__set_maps_by_name_sorted (maps, false); + } +if (map__end (new) <map__start (new)) +RC_CHK_ACCESS (maps)->ends_broken = true; +if (dso && dso->kernel) { +struct kmap *kmap =map__kmap (new); +
Quoted 8 lines
tools/perf/util/maps.c line 390
Quoted 6 lines
tools/perf/util/maps.c +6
-0+if (kmap) + kmap->kmaps = maps; +else +pr_err ("Internal error: kernel dso with non kernel map\n" ); + } +check_invariants (maps);Probably not needed as it's checked when you get the lock below.
Ack. Will remove in v3.
Quoted 43 lines
Quoted 43 lines
tools/perf/util/maps.c +29
-6+return 0 ; } -static void maps__delete (struct maps *maps) +int maps__insert (struct maps *maps,struct map *map) { -maps__exit (maps); -unwind__finish_access (maps); -RC_CHK_FREE (maps); +int ret; + +down_write (maps__lock (maps)); + ret =__maps__insert (maps, map); +up_write (maps__lock (maps)); +return ret; } -struct maps *maps__get (struct maps *maps) +static void __maps__remove (struct maps *maps,struct map *map) { -struct maps *result; +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); +unsigned int nr_maps =maps__nr_maps (maps); +unsigned int address_idx; + +/* Slide later mappings over the one to remove */ + address_idx =maps__by_address_index (maps, map); +map__put (maps_by_address[address_idx]); +memmove (&maps_by_address[address_idx], + &maps_by_address[address_idx +1 ], + (nr_maps - address_idx -1 ) *sizeof (*maps_by_address)); + +if (maps_by_name) { +unsigned int name_idx =maps__by_name_index (maps, map); + +map__put (maps_by_name[name_idx]); +memmove (&maps_by_name[name_idx], + &maps_by_name[name_idx +1 ], + (nr_maps - name_idx -1 ) *sizeof (*maps_by_name)); + }
Quoted 6 lines
tools/perf/util/maps.c line 428
Quoted 4 lines
tools/perf/util/maps.c +2
-2-if (RC_CHK_GET (result, maps)) -refcount_inc (maps__refcnt (maps)); + --RC_CHK_ACCESS (maps)->nr_maps; +check_invariants (maps);Ditto.
Ack.
Quoted 400 lines
Quoted 400 lines
tools/perf/util/maps.c +196
-99+} -return result; +void maps__remove (struct maps *maps,struct map *map) +{ +down_write (maps__lock (maps)); +__maps__remove (maps, map); +up_write (maps__lock (maps)); } -void maps__put (struct maps *maps) +bool maps__empty (struct maps *maps) { -if (maps &&refcount_dec_and_test (maps__refcnt (maps))) -maps__delete (maps); -else -RC_CHK_PUT (maps); +return maps__nr_maps (maps) ==0 ; }int maps__for_each_map (struct maps *maps,int (*cb)(struct map *map,void *data),void *data) { -struct map_rb_node *pos; +bool done = false;int ret =0 ; -down_read (maps__lock (maps)); -maps__for_each_entry (maps, pos) { - ret =cb (pos->map, data); -if (ret) -break ; +/* See locking/sorting note. */ +while (!done) { +down_read (maps__lock (maps)); +if (maps__maps_by_address_sorted (maps)) { +struct map **maps_by_address =maps__maps_by_address (maps); +unsigned int n =maps__nr_maps (maps); + +for (unsigned int i =0 ; i < n; i++) { +struct map *map = maps_by_address[i]; + + ret =cb (map, data); +if (ret) +break ; + } + done = true; + } +up_read (maps__lock (maps)); +if (!done) +maps__sort_by_address (maps); } -up_read (maps__lock (maps));return ret; }void maps__remove_maps (struct maps *maps,bool (*cb)(struct map *map,void *data),void *data) { -struct map_rb_node *pos, *next; -unsigned int start_nr_maps; +struct map **maps_by_address;down_write (maps__lock (maps)); - start_nr_maps =maps__nr_maps (maps); -maps__for_each_entry_safe (maps, pos, next) { -if (cb (pos->map, data)) { -__maps__remove (maps, pos); - --RC_CHK_ACCESS (maps)->nr_maps; - } + maps_by_address =maps__maps_by_address (maps); +for (unsigned int i =0 ; i <maps__nr_maps (maps);) { +if (cb (maps_by_address[i], data)) +__maps__remove (maps, maps_by_address[i]); +else + i++; } -if (maps__maps_by_name (maps) && start_nr_maps !=maps__nr_maps (maps)) -__maps__free_maps_by_name (maps); -up_write (maps__lock (maps)); }@@ -300,7 +491,7 @@ struct symbol *maps__find_symbol(struct maps *maps, u64 addr, struct map **mapp)/* Ensure map is loaded before using map->map_ip */ if (map !=NULL &&map__load (map) >=0 ) {if (mapp !=NULL ) - *mapp = map; + *mapp = map;// TODO: map_put on else path when find returns a get. return map__find_symbol (map,map__map_ip (map, addr)); }@@ -348,7 +539,7 @@ int maps__find_ams(struct maps *maps, struct addr_map_symbol *ams)if (ams->addr <map__start (ams->ms.map) || ams->addr >=map__end (ams->ms.map)) {if (maps ==NULL )return -1 ; - ams->ms.map =maps__find (maps, ams->addr); + ams->ms.map =maps__find (maps, ams->addr);// TODO: map_get if (ams->ms.map ==NULL )return -1 ; }@@ -393,24 +584,28 @@ size_t maps__fprintf(struct maps *maps, FILE *fp)* Find first map where end > map->start. * Same as find_vma() in kernel. */ -static struct rb_node *first_ending_after (struct maps *maps,const struct map *map) +static unsigned int first_ending_after (struct maps *maps,const struct map *map) { -struct rb_root *root; -struct rb_node *next, *first; +struct map **maps_by_address =maps__maps_by_address (maps); +int low =0 , high = (int )maps__nr_maps (maps) -1 , first = high +1 ; + +assert (maps__maps_by_address_sorted (maps)); +if (low <= high &&map__end (maps_by_address[0 ]) >map__start (map)) +return 0 ; - root =maps__entries (maps); - next = root->rb_node; - first =NULL ; -while (next) { - struct map_rb_node *pos = rb_entry(next, struct map_rb_node, rb_node); +while (low <= high) { +int mid = (low + high) /2 ; +struct map *pos = maps_by_address[mid]; -if (map__end (pos->map) >map__start (map)) { - first = next; -if (map__start (pos->map) <=map__start (map)) +if (map__end (pos) >map__start (map)) { + first = mid; +if (map__start (pos) <=map__start (map)) { +/* Entry overlaps map. */ break ; - next = next->rb_left; + } + high = mid -1 ; }else - next = next->rb_right; + low = mid +1 ; }return first; }@@ -419,171 +614,249 @@ static struct rb_node *first_ending_after(struct maps *maps, const struct map *m* Adds new to maps, if new overlaps existing entries then the existing maps are * adjusted or removed so that new fits without overlapping any entries. */ -int maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) +static int __maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) { - -struct rb_node *next; +struct map **maps_by_address;int err =0 ;FILE *fp =debug_file (); -down_write (maps__lock (maps)); +sort_again: +if (!maps__maps_by_address_sorted (maps)) +__maps__sort_by_address (maps); - next =first_ending_after (maps, new); -while (next && !err) { - struct map_rb_node *pos = rb_entry(next, struct map_rb_node, rb_node); - next =rb_next (&pos->rb_node); + maps_by_address =maps__maps_by_address (maps); +/* +* Iterate through entries where the end of the existing entry is +* greater-than the new map's start. +*/ +for (unsigned int i =first_ending_after (maps, new); i <maps__nr_maps (maps); ) { +struct map *pos = maps_by_address[i]; +struct map *before =NULL , *after =NULL ;/* * Stop if current map starts after map->end. * Maps are ordered by start: next will not overlap for sure. */ -if (map__start (pos->map) >=map__end (new)) +if (map__start (pos) >=map__end (new))break ; -if (verbose >=2 ) { - -if (use_browser) { -pr_debug ("overlapping maps in %s (disable tui for more info)\n" , -map__dso (new)->name); - }else { -pr_debug ("overlapping maps:\n" ); -map__fprintf (new, fp); -map__fprintf (pos->map, fp); - } +if (use_browser) { +pr_debug ("overlapping maps in %s (disable tui for more info)\n" , +map__dso (new)->name); + }else if (verbose >=2 ) { +pr_debug ("overlapping maps:\n" ); +map__fprintf (new, fp); +map__fprintf (pos, fp); } -rb_erase_init (&pos->rb_node,maps__entries (maps));/* * Now check if we need to create new maps for areas not * overlapped by the new map: */ -if (map__start (new) >map__start (pos->map)) { -struct map *before =map__clone (pos->map); +if (map__start (new) >map__start (pos)) { +/* Map starts within existing map. Need to shorten the existing map. */ + before =map__clone (pos);if (before ==NULL ) { err = -ENOMEM ; - goto put_map; + goto out_err; } -map__set_end (before,map__start (new)); - err =__maps__insert (maps, before); -if (err) { -map__put (before); - goto put_map; - }if (verbose >=2 && !use_browser)map__fprintf (before, fp); -map__put (before); } - -if (map__end (new) <map__end (pos->map)) { -struct map *after =map__clone (pos->map); +if (map__end (new) <map__end (pos)) { +/* The new map isn't as long as the existing map. */ + after =map__clone (pos);if (after ==NULL ) { +map__zput (before); err = -ENOMEM ; - goto put_map; + goto out_err; }map__set_start (after,map__end (new)); -map__add_pgoff (after,map__end (new) -map__start (pos->map)); -assert (map__map_ip (pos->map,map__end (new)) == -map__map_ip (after,map__end (new))); - err =__maps__insert (maps, after); -if (err) { -map__put (after); - goto put_map; - } +map__add_pgoff (after,map__end (new) -map__start (pos)); +assert (map__map_ip (pos,map__end (new)) == +map__map_ip (after,map__end (new))); +if (verbose >=2 && !use_browser)map__fprintf (after, fp); -map__put (after); } -put_map: -map__put (pos->map); -free (pos); +/* +* If adding one entry, for `before` or `after`, we can replace +* the existing entry. If both `before` and `after` are +* necessary than an insert is needed. If the existing entry +* entirely overlaps the existing entry it can just be removed. +*/ +if (before) { +map__put (maps_by_address[i]); + maps_by_address[i] = before; +/* Maps are still ordered, go to next one. */ + i++; +if (after) { +__maps__insert (maps, after); +map__put (after); +if (!maps__maps_by_address_sorted (maps)) { +/* +* Sorting broken so invariants don't +* hold, sort and go again. +*/ + goto sort_again; + } +/* +* Maps are still ordered, skip after and go to +* next one (terminate loop). +*/ + i++; + } + }else if (after) { +map__put (maps_by_address[i]); + maps_by_address[i] = after; +/* Maps are ordered, go to next one. */ + i++; + }else { +__maps__remove (maps, pos); +/* +* Maps are ordered but no need to increase `i` as the +* later maps were moved down. +*/ + } +check_invariants (maps); }/* Add the map. */ - err =__maps__insert (maps, new); -up_write (maps__lock (maps)); +__maps__insert (maps, new); +out_err:return err; } -int maps__copy_from (struct maps *maps,struct maps *parent) +int maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) {int err; -struct map_rb_node *rb_node; +down_write (maps__lock (maps)); + err =__maps__fixup_overlap_and_insert (maps, new); +up_write (maps__lock (maps)); +return err; +} + +int maps__copy_from (struct maps *dest,struct maps *parent) +{ +/* Note, if struct map were immutable then cloning could use ref counts. */ +struct map **parent_maps_by_address; +int err =0 ; +unsigned int n; + +down_write (maps__lock (dest));down_read (maps__lock (parent)); -maps__for_each_entry (parent, rb_node) { -struct map *new =map__clone (rb_node->map); + parent_maps_by_address =maps__maps_by_address (parent); + n =maps__nr_maps (parent); +if (maps__empty (dest)) { +/* No existing mappings so just copy from parent to avoid reallocs in insert. */ +unsigned int nr_maps_allocated =RC_CHK_ACCESS (parent)->nr_maps_allocated; +struct map **dest_maps_by_address = +malloc (nr_maps_allocated *sizeof (struct map *)); +struct map **dest_maps_by_name =NULL ; -if (new ==NULL ) { +if (!dest_maps_by_address) err = -ENOMEM ; - goto out_unlock; +else { +if (maps__maps_by_name (parent)) { + dest_maps_by_name = +malloc (nr_maps_allocated *sizeof (struct map *)); + } + +RC_CHK_ACCESS (dest)->maps_by_address = dest_maps_by_address; +RC_CHK_ACCESS (dest)->maps_by_name = dest_maps_by_name; +RC_CHK_ACCESS (dest)->nr_maps_allocated = nr_maps_allocated; } - err =unwind__prepare_access (maps, new,NULL ); -if (err) - goto out_unlock; +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos); - err =maps__insert (maps, new); -if (err) - goto out_unlock; +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) { + dest_maps_by_address[i] = new; +if (dest_maps_by_name) + dest_maps_by_name[i] =map__get (new); +RC_CHK_ACCESS (dest)->nr_maps = i +1 ; + } + } +if (err) +map__put (new); + } +maps__set_maps_by_address_sorted (dest,maps__maps_by_address_sorted (parent)); +if (!err) { +RC_CHK_ACCESS (dest)->last_search_by_name_idx = +RC_CHK_ACCESS (parent)->last_search_by_name_idx; +maps__set_maps_by_name_sorted (dest, + dest_maps_by_name && +maps__maps_by_name_sorted (parent)); + }else { +RC_CHK_ACCESS (dest)->last_search_by_name_idx =0 ; +maps__set_maps_by_name_sorted (dest, false); + } + }else { +/* Unexpected copying to a maps containing entries. */ +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos);
Quoted 9 lines
tools/perf/util/maps.c static struct rb_node *first_ending_after(struct maps *maps, const struct map *m · line 816
Quoted 7 lines
tools/perf/util/maps.c +6
-1-map__put (new); +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) + err =maps__insert (dest, new);Shouldn't it be
__maps__insert()?
On entry the read lock is taken on parent but no lock is taken on dest so the locked version is used.
Quoted 373 lines
Quoted 360 lines
tools/perf/util/maps.c +139
-160+ } +map__put (new); + } } - - err =0 ; -out_unlock:up_read (maps__lock (parent)); +up_write (maps__lock (dest));return err; } -struct map *maps__find (struct maps *maps,u64 ip) +static int map__addr_cmp (const void *key,const void *entry) { -struct rb_node *p; -struct map_rb_node *m; - +const u64 ip = *(const u64 *)key; +const struct map *map = *(const struct map *const *)entry; -down_read (maps__lock (maps)); - - p =maps__entries (maps)->rb_node; -while (p !=NULL ) { - m = rb_entry(p, struct map_rb_node, rb_node); -if (ip <map__start (m->map)) - p = p->rb_left; -else if (ip >=map__end (m->map)) - p = p->rb_right; -else - goto out; - } - - m =NULL ; -out: -up_read (maps__lock (maps)); -return m ? m->map :NULL ; +if (ip <map__start (map)) +return -1 ; +if (ip >=map__end (map)) +return 1 ; +return 0 ; } -static int map__strcmp (const void *a,const void *b) +struct map *maps__find (struct maps *maps,u64 ip) { -const struct map *map_a = *(const struct map **)a; -const struct map *map_b = *(const struct map **)b; -const struct dso *dso_a =map__dso (map_a); -const struct dso *dso_b =map__dso (map_b); -int ret =strcmp (dso_a->short_name, dso_b->short_name); - -if (ret ==0 && map_a != map_b) { -/* -* Ensure distinct but name equal maps have an order in part to -* aid reference counting. -*/ - ret = (int )map__start (map_a) - (int )map__start (map_b); -if (ret ==0 ) - ret = (int )((intptr_t )map_a - (intptr_t )map_b); +struct map *result =NULL ; +bool done = false; + +/* See locking/sorting note. */ +while (!done) { +down_read (maps__lock (maps)); +if (maps__maps_by_address_sorted (maps)) { +struct map **mapp = +bsearch (&ip,maps__maps_by_address (maps),maps__nr_maps (maps), +sizeof (*mapp), map__addr_cmp); + +if (mapp) + result = *mapp;// map__get(*mapp); + done = true; + } +up_read (maps__lock (maps)); +if (!done) +maps__sort_by_address (maps); } - -return ret; +return result; }static int map__strcmp_name (const void *name,const void *b)@@ -593,126 +866,113 @@ static int map__strcmp_name(const void *name, const void *b)return strcmp (name, dso->short_name); } -void __maps__sort_by_name (struct maps *maps) -{ -qsort (maps__maps_by_name (maps),maps__nr_maps (maps),sizeof (struct map *), map__strcmp); -} - -static int map__groups__sort_by_name_from_rbtree (struct maps *maps) -{ -struct map_rb_node *rb_node; -struct map **maps_by_name =realloc (maps__maps_by_name (maps), -maps__nr_maps (maps) *sizeof (struct map *)); -int i =0 ; - -if (maps_by_name ==NULL ) -return -1 ; - -up_read (maps__lock (maps)); -down_write (maps__lock (maps)); - -RC_CHK_ACCESS (maps)->maps_by_name = maps_by_name; -RC_CHK_ACCESS (maps)->nr_maps_allocated =maps__nr_maps (maps); - -maps__for_each_entry (maps, rb_node) - maps_by_name[i++] =map__get (rb_node->map); - -__maps__sort_by_name (maps); - -up_write (maps__lock (maps)); -down_read (maps__lock (maps)); - -return 0 ; -} - -static struct map *__maps__find_by_name (struct maps *maps,const char *name) +struct map *maps__find_by_name (struct maps *maps,const char *name) { -struct map **mapp; +struct map *result =NULL ; +bool done = false; -if (maps__maps_by_name (maps) ==NULL && -map__groups__sort_by_name_from_rbtree (maps)) -return NULL ; +/* See locking/sorting note. */ +while (!done) { +unsigned int i; - mapp =bsearch (name,maps__maps_by_name (maps),maps__nr_maps (maps), -sizeof (*mapp), map__strcmp_name); -if (mapp) -return *mapp; -return NULL ; -} +down_read (maps__lock (maps)); -struct map *maps__find_by_name (struct maps *maps,const char *name) -{ -struct map_rb_node *rb_node; -struct map *map; - -down_read (maps__lock (maps)); +/* First check last found entry. */ + i =RC_CHK_ACCESS (maps)->last_search_by_name_idx; +if (i <maps__nr_maps (maps) &&maps__maps_by_name (maps)) { +struct dso *dso =map__dso (maps__maps_by_name (maps)[i]); +if (dso &&strcmp (dso->short_name, name) ==0 ) { + result =maps__maps_by_name (maps)[i];// TODO: map__get + done = true; + } + } -if (RC_CHK_ACCESS (maps)->last_search_by_name) { -const struct dso *dso =map__dso (RC_CHK_ACCESS (maps)->last_search_by_name); +/* Second search sorted array. */ +if (!done &&maps__maps_by_name_sorted (maps)) { +struct map **mapp = +bsearch (name,maps__maps_by_name (maps),maps__nr_maps (maps), +sizeof (*mapp), map__strcmp_name); -if (strcmp (dso->short_name, name) ==0 ) { - map =RC_CHK_ACCESS (maps)->last_search_by_name; - goto out_unlock; +if (mapp) { + result = *mapp;// TODO: map__get + i = mapp -maps__maps_by_name (maps); +RC_CHK_ACCESS (maps)->last_search_by_name_idx = i; + } + done = true; } - } -/* -* If we have maps->maps_by_name, then the name isn't in the rbtree, -* as maps->maps_by_name mirrors the rbtree when lookups by name are -* made. -*/ - map =__maps__find_by_name (maps, name); -if (map ||maps__maps_by_name (maps) !=NULL ) - goto out_unlock; - -/* Fallback to traversing the rbtree... */ -maps__for_each_entry (maps, rb_node) { -struct dso *dso; - - map = rb_node->map; - dso =map__dso (map); -if (strcmp (dso->short_name, name) ==0 ) { -RC_CHK_ACCESS (maps)->last_search_by_name = map; - goto out_unlock; +up_read (maps__lock (maps)); +if (!done) { +/* Sort and retry binary search. */ +if (maps__sort_by_name (maps)) { +/* +* Memory allocation failed do linear search +* through address sorted maps. +*/ +struct map **maps_by_address; +unsigned int n; + +down_read (maps__lock (maps)); + maps_by_address =maps__maps_by_address (maps); + n =maps__nr_maps (maps); +for (i =0 ; i < n; i++) { +struct map *pos = maps_by_address[i]; +struct dso *dso =map__dso (pos); + +if (dso &&strcmp (dso->short_name, name) ==0 ) { + result = pos;// TODO: map__get +break ; + } + } +up_read (maps__lock (maps)); + done = true; + } } } - map =NULL ; - -out_unlock: -up_read (maps__lock (maps)); -return map; +return result; }struct map *maps__find_next_entry (struct maps *maps,struct map *map) { -struct map_rb_node *rb_node =maps__find_node (maps, map); -struct map_rb_node *next =map_rb_node__next (rb_node); +unsigned int i; +struct map *result =NULL ; -if (next) -return next->map; +down_read (maps__lock (maps)); + i =maps__by_address_index (maps, map); +if (i <maps__nr_maps (maps)) + result =maps__maps_by_address (maps)[i];// TODO: map__get -return NULL ; +up_read (maps__lock (maps)); +return result; }void maps__fixup_end (struct maps *maps) { -struct map_rb_node *prev =NULL , *curr; +struct map **maps_by_address; +unsigned int n;down_write (maps__lock (maps)); +if (!maps__maps_by_address_sorted (maps)) +__maps__sort_by_address (maps); -maps__for_each_entry (maps, curr) { -if (prev && (!map__end (prev->map) ||map__end (prev->map) >map__start (curr->map))) -map__set_end (prev->map,map__start (curr->map)); + maps_by_address =maps__maps_by_address (maps); + n =maps__nr_maps (maps); +for (unsigned int i =1 ; i < n; i++) { +struct map *prev = maps_by_address[i -1 ]; +struct map *curr = maps_by_address[i]; - prev = curr; +if (!map__end (prev) ||map__end (prev) >map__start (curr)) +map__set_end (prev,map__start (curr)); }/* * We still haven't the actual symbols, so guess the * last map final address. */ -if (curr && !map__end (curr->map)) -map__set_end (curr->map, ~0ULL ); +if (n >0 && !map__end (maps_by_address[n -1 ])) +map__set_end (maps_by_address[n -1 ], ~0ULL ); + +RC_CHK_ACCESS (maps)->ends_broken = false;up_write (maps__lock (maps)); }@@ -723,117 +983,92 @@ void maps__fixup_end(struct maps *maps)*/ int maps__merge_in (struct maps *kmaps,struct map *new_map) { -struct map_rb_node *rb_node; -struct rb_node *first; -bool overlaps; -LIST_HEAD (merged); -int err =0 ; +unsigned int first_after_, kmaps__nr_maps; +struct map **kmaps_maps_by_address; +struct map **merged_maps_by_address; +unsigned int merged_nr_maps_allocated; + +/* First try under a read lock. */ +while (true) { +down_read (maps__lock (kmaps)); +if (maps__maps_by_address_sorted (kmaps)) +break ; -down_read (maps__lock (kmaps)); - first =first_ending_after (kmaps, new_map); - rb_node = first ? rb_entry(first, struct map_rb_node, rb_node) : NULL; - overlaps = rb_node &&map__start (rb_node->map) <map__end (new_map); -up_read (maps__lock (kmaps)); +up_read (maps__lock (kmaps)); + +/* First after binary search requires sorted maps. Sort and try again. */ +maps__sort_by_address (kmaps); + } + first_after_ =first_ending_after (kmaps, new_map); + kmaps_maps_by_address =maps__maps_by_address (kmaps); -if (!overlaps) +if (first_after_ >=maps__nr_maps (kmaps) || +map__start (kmaps_maps_by_address[first_after_]) >=map__end (new_map)) { +/* No overlap so regular insert suffices. */ +up_read (maps__lock (kmaps));return maps__insert (kmaps, new_map); + } +up_read (maps__lock (kmaps)); -maps__for_each_entry (kmaps, rb_node) { -struct map *old_map = rb_node->map; +/* Plain insert with a read-lock failed, try again now with the write lock. */ +down_write (maps__lock (kmaps)); +if (!maps__maps_by_address_sorted (kmaps)) +__maps__sort_by_address (kmaps); -/* no overload with this one */ -if (map__end (new_map) <map__start (old_map) || -map__start (new_map) >=map__end (old_map)) -continue ; + first_after_ =first_ending_after (kmaps, new_map); + kmaps_maps_by_address =maps__maps_by_address (kmaps); + kmaps__nr_maps =maps__nr_maps (kmaps); -if (map__start (new_map) <map__start (old_map)) { -/* -* |new...... -* |old.... -*/ -if (map__end (new_map) <map__end (old_map)) { -/* -* |new......| -> |new..| -* |old....| -> |old....| -*/ -map__set_end (new_map,map__start (old_map)); - }else {tools/perf/util/maps.c void maps__fixup_end(struct maps *maps) · line 1026
Quoted 10 lines
tools/perf/util/maps.c +5
-5-/* -* |new.............| -> |new..| |new..| -* |old....| -> |old....| -*/ -struct map_list_node *m =map_list_node__new (); +if (first_after_ >= kmaps__nr_maps || +map__start (kmaps_maps_by_address[first_after_]) >=map__end (new_map)) { +/* No overlap so regular insert suffices. */ +up_write (maps__lock (kmaps)); +return maps__insert (kmaps, new_map);I think it could be:
Quoted 3 lines
ret = __maps__insert(kmaps, new_map);
up_write(maps__lock(kmaps));
return ret;Ack. Will change in v3.
Quoted 17 lines
Quoted 6 lines
tools/perf/util/maps.c +6
-0+ } +/* Array to merge into, possibly 1 more for the sake of new_map. */ + merged_nr_maps_allocated =RC_CHK_ACCESS (kmaps)->nr_maps_allocated; +if (kmaps__nr_maps +1 == merged_nr_maps_allocated) + merged_nr_maps_allocated++; +tools/perf/util/maps.c void maps__fixup_end(struct maps *maps) · line 1042
Quoted 10 lines
tools/perf/util/maps.c +10
-0+ merged_maps_by_address =malloc (merged_nr_maps_allocated *sizeof (*merged_maps_by_address)); +if (!merged_maps_by_address) { +up_write (maps__lock (kmaps)); +return -ENOMEM ; + } +RC_CHK_ACCESS (kmaps)->maps_by_address = merged_maps_by_address; +RC_CHK_ACCESS (kmaps)->maps_by_address_sorted = true; +zfree (&RC_CHK_ACCESS (kmaps)->maps_by_name); +RC_CHK_ACCESS (kmaps)->maps_by_name_sorted = false; +RC_CHK_ACCESS (kmaps)->nr_maps_allocated = merged_nr_maps_allocated;
Quoted 1 line
Why not use the accessor functions?
Ack. I've been holding back on accessors that are used once, but I will add them here.
Thanks,
Ian
Quoted 184 lines
Thanks,
NamhyungQuoted 181 lines
tools/perf/util/maps.c +17
-59-if (!m) { - err = -ENOMEM ; - goto out; - } +/* Copy entries before the new_map that can't overlap. */ +for (unsigned int i =0 ; i < first_after_; i++) + merged_maps_by_address[i] =map__get (kmaps_maps_by_address[i]); - m->map =map__clone (new_map); -if (!m->map) { -free (m); - err = -ENOMEM ; - goto out; - } +RC_CHK_ACCESS (kmaps)->nr_maps = first_after_; -map__set_end (m->map,map__start (old_map)); -list_add_tail (&m->node, &merged); -map__add_pgoff (new_map,map__end (old_map) -map__start (new_map)); -map__set_start (new_map,map__end (old_map)); - } - }else { -/* -* |new...... -* |old.... -*/ -if (map__end (new_map) <map__end (old_map)) { -/* -* |new..| -> x -* |old.........| -> |old.........| -*/ -map__put (new_map); - new_map =NULL ; -break ; - }else { -/* -* |new......| -> |new...| -* |old....| -> |old....| -*/ -map__add_pgoff (new_map,map__end (old_map) -map__start (new_map)); -map__set_start (new_map,map__end (old_map)); - } - } - } +/* Add the new map, it will be split when the later overlapping mappings are added. */ +__maps__insert (kmaps, new_map); -out: -while (!list_empty (&merged)) { -struct map_list_node *old_node; +/* Insert mappings after new_map, splitting new_map in the process. */ +for (unsigned int i = first_after_; i < kmaps__nr_maps; i++) +__maps__fixup_overlap_and_insert (kmaps, kmaps_maps_by_address[i]); - old_node = list_entry(merged.next, struct map_list_node, node); -list_del_init (&old_node->node); -if (!err) - err =maps__insert (kmaps, old_node->map); -map__put (old_node->map); -free (old_node); - } +/* Copy the maps from merged into kmaps. */ +for (unsigned int i =0 ; i < kmaps__nr_maps; i++) +map__zput (kmaps_maps_by_address[i]); -if (new_map) { -if (!err) - err =maps__insert (kmaps, new_map); -map__put (new_map); - } -return err; +free (kmaps_maps_by_address); +up_write (maps__lock (kmaps)); +return 0 ; }void maps__load_first (struct maps *maps) { -struct map_rb_node *first; -down_read (maps__lock (maps)); - first =maps__first (maps); -if (first) -map__load (first->map); +if (maps__nr_maps (maps) >0 ) +map__load (maps__maps_by_address (maps)[0 ]);up_read (maps__lock (maps)); }tools/perf/util/maps.h +44
-11@@ -25,21 +25,56 @@ static inline struct map_list_node *map_list_node__new(void)return malloc (sizeof (struct map_list_node )); } -struct map *maps__find (struct maps *maps,u64 addr); +/* +* Locking/sorting note: +* +* Sorting is done with the write lock, iteration and binary searching happens +* under the read lock requiring being sorted. There is a race between sorting +* releasing the write lock and acquiring the read lock for iteration/searching +* where another thread could insert and break the sorting of the maps. In +* practice inserting maps should be rare meaning that the race shouldn't lead +* to live lock. Removal of maps doesn't break being sorted. +*/ DECLARE_RC_STRUCT (maps) { -struct rb_root entries;struct rw_semaphore lock; -struct machine *machine; -struct map *last_search_by_name; +/** +* @maps_by_address: array of maps sorted by their starting address if +* maps_by_address_sorted is true. +*/ +struct map **maps_by_address; +/** +* @maps_by_name: optional array of maps sorted by their dso name if +* maps_by_name_sorted is true. +*/ struct map **maps_by_name; -refcount_t refcnt; -unsigned int nr_maps; -unsigned int nr_maps_allocated; +struct machine *machine;#ifdef HAVE_LIBUNWIND_SUPPORT -void *addr_space; +void *addr_space;const struct unwind_libunwind_ops *unwind_libunwind_ops;#endif +refcount_t refcnt; +/** +* @nr_maps: number of maps_by_address, and possibly maps_by_name, +* entries that contain maps. +*/ +unsigned int nr_maps; +/** +* @nr_maps_allocated: number of entries in maps_by_address and possibly +* maps_by_name. +*/ +unsigned int nr_maps_allocated; +/** +* @last_search_by_name_idx: cache of last found by name entry's index +* as frequent searches for the same dso name are common. +*/ +unsigned int last_search_by_name_idx; +/** @maps_by_address_sorted: is maps_by_address sorted. */ +bool maps_by_address_sorted; +/** @maps_by_name_sorted: is maps_by_name sorted. */ +bool maps_by_name_sorted; +/** @ends_broken: does the map contain a map where end values are unset/unsorted? */ +bool ends_broken; };#define KMAP_NAME_LEN 256@@ -102,6 +137,7 @@ size_t maps__fprintf(struct maps *maps, FILE *fp);int maps__insert (struct maps *maps,struct map *map);void maps__remove (struct maps *maps,struct map *map); +struct map *maps__find (struct maps *maps,u64 addr);struct symbol *maps__find_symbol (struct maps *maps,u64 addr,struct map **mapp);struct symbol *maps__find_symbol_by_name (struct maps *maps,const char *name,struct map **mapp);@@ -117,8 +153,6 @@ struct map *maps__find_next_entry(struct maps *maps, struct map *map);int maps__merge_in (struct maps *kmaps,struct map *new_map); -void __maps__sort_by_name (struct maps *maps); -void maps__fixup_end (struct maps *maps);void maps__load_first (struct maps *maps); --2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerRe: [PATCH v7 01/25] perf maps: Switch from rbtree to lazily sorted array for addresses
Quoted 615 lines · Ian Rogers
Quoted 604 lines · Namhyung Kim
Hi Ian,
Quoted 24 lines · Ian Rogers
Maps is a collection of maps primarily sorted by the starting address of the map. Prior to this change the maps were held in an rbtree requiring 4 pointers per node. Prior to reference count checking, the rbnode was embedded in the map so 3 pointers per node were necessary. This change switches the rbtree to an array lazily sorted by address, much as the array sorting nodes by name. 1 pointer is needed per node, but to avoid excessive resizing the backing array may be twice the number of used elements. Meaning the memory overhead is roughly half that of the rbtree. For a perf record with "--no-bpf-event -g -a" of true, the memory overhead of perf inject is reduce fom 3.3MB to 3MB, so 10% or 300KB is saved.
Map inserts always happen at the end of the array. The code tracks whether the insertion violates the sorting property. O(log n) rb-tree complexity is switched to O(1).
Remove slides the array, so O(log n) rb-tree complexity is degraded to O(n).
A find may need to sort the array using qsort which is
O(n*log n), but in general the maps should be sorted and so average performance should be O(log n) as with the rbtree.Quoted 6 lines
An rbtree node consumes a cache line, but with the array 4 nodes fit on a cache line. Iteration is simplified to scanning an array rather than pointer chasing.
Overall it is expected the performance after the change should be comparable to before, but with half of the memory consumed.
I don't know how much performance impact it would have but I guess search/iteration would be the most frequent operation. So I like the memory saving it can bring.
Quoted 565 lines
To avoid a list and repeated logic around splitting maps,
maps__merge_inis rewritten in terms ofmaps__fixup_overlap_and_insert.maps_merge_insplits the given mapping inserting remaining gaps.maps__fixup_overlap_and_insertsplits the existing mappings, then adds the incoming mapping. By adding the new mapping first, then re-inserting the existing mappings the splitting behavior matches.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/maps.c | 3 + tools/perf/util/map.c | 1 + tools/perf/util/maps.c | 1183 +++++++++++++++++++++++---------------- tools/perf/util/maps.h | 54 +- 4 files changed, 757 insertions(+), 484 deletions(-)tools/perf/tests/maps.c +3
-0@@ -156,6 +156,9 @@ static int test__maps__merge_in(struct test_suite *t __maybe_unused, int subtestTEST_ASSERT_VAL ("merge check failed" , !ret);maps__zput (maps); +map__zput (map_kcore1); +map__zput (map_kcore2); +map__zput (map_kcore3);return TEST_OK ; }tools/perf/util/map.c +1
-0@@ -168,6 +168,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,if (dso ==NULL ) goto out_delete; +assert (!dso->kernel);map__init (result, start, start + len, pgoff, dso);if (anon || no_dso) {tools/perf/util/maps.c +293
-142@@ -10,286 +10,477 @@#include "ui/ui.h" #include "unwind.h" -struct map_rb_node { -struct rb_node rb_node; -struct map *map; -}; - -#define maps__for_each_entry (maps, map) \ - for (map = maps__first(maps); map; map = map_rb_node__next(map)) +static void check_invariants (const struct maps *maps__maybe_unused ) +{ +#ifndef NDEBUG +assert (RC_CHK_ACCESS (maps)->nr_maps <=RC_CHK_ACCESS (maps)->nr_maps_allocated); +for (unsigned int i =0 ; i <RC_CHK_ACCESS (maps)->nr_maps; i++) { +struct map *map =RC_CHK_ACCESS (maps)->maps_by_address[i]; + +/* Check map is well-formed. */ +assert (map__end (map) ==0 ||map__start (map) <=map__end (map)); +/* Expect at least 1 reference count. */ +assert (refcount_read (map__refcnt (map)) >0 ); + +if (map__dso (map) &&map__dso (map)->kernel) +assert (RC_CHK_EQUAL (map__kmap (map)->kmaps, maps)); + +if (i >0 ) { +struct map *prev =RC_CHK_ACCESS (maps)->maps_by_address[i -1 ]; + +/* If addresses are sorted... */ +if (RC_CHK_ACCESS (maps)->maps_by_address_sorted) { +/* Maps should be in start address order. */ +assert (map__start (prev) <=map__start (map)); +/* +* If the ends of maps aren't broken (during +* construction) then they should be ordered +* too. +*/ +if (!RC_CHK_ACCESS (maps)->ends_broken) { +assert (map__end (prev) <=map__end (map)); +assert (map__end (prev) <=map__start (map) || +map__start (prev) ==map__start (map)); + } + } + } + } +if (RC_CHK_ACCESS (maps)->maps_by_name) { +for (unsigned int i =0 ; i <RC_CHK_ACCESS (maps)->nr_maps; i++) { +struct map *map =RC_CHK_ACCESS (maps)->maps_by_name[i]; -#define maps__for_each_entry_safe (maps, map, next) \ - for (map = maps__first(maps), next = map_rb_node__next(map); map; \ - map = next, next = map_rb_node__next(map)) +/* +* Maps by name maps should be in maps_by_address, so +* the reference count should be higher. +*/ +assert (refcount_read (map__refcnt (map)) >1 ); + } + } +#endif +} -static struct rb_root *maps__entries (struct maps *maps) +static struct map **maps__maps_by_address (const struct maps *maps) { -return &RC_CHK_ACCESS (maps)->entries; +return RC_CHK_ACCESS (maps)->maps_by_address; } -static struct rw_semaphore *maps__lock (struct maps *maps) +static void maps__set_maps_by_address (struct maps *maps,struct map **new) { -return &RC_CHK_ACCESS (maps)->lock; +RC_CHK_ACCESS (maps)->maps_by_address = new; + } -static struct map **maps__maps_by_name (struct maps *maps) +/* Not in the header, to aid reference counting. */ +static struct map **maps__maps_by_name (const struct maps *maps) {return RC_CHK_ACCESS (maps)->maps_by_name; + } -static struct map_rb_node *maps__first (struct maps *maps) +static void maps__set_maps_by_name (struct maps *maps,struct map **new) { -struct rb_node *first =rb_first (maps__entries (maps)); +RC_CHK_ACCESS (maps)->maps_by_name = new; -if (first) - return rb_entry(first, struct map_rb_node, rb_node); -return NULL ; } -static struct map_rb_node *map_rb_node__next (struct map_rb_node *node) +static bool maps__maps_by_address_sorted (const struct maps *maps) { -struct rb_node *next; - -if (!node) -return NULL ; - - next =rb_next (&node->rb_node); +return RC_CHK_ACCESS (maps)->maps_by_address_sorted; +} -if (!next) -return NULL ; +static void maps__set_maps_by_address_sorted (struct maps *maps,bool value) +{ +RC_CHK_ACCESS (maps)->maps_by_address_sorted = value; +} - return rb_entry(next, struct map_rb_node, rb_node); +static bool maps__maps_by_name_sorted (const struct maps *maps) +{ +return RC_CHK_ACCESS (maps)->maps_by_name_sorted; } -static struct map_rb_node *maps__find_node (struct maps *maps,struct map *map) +static void maps__set_maps_by_name_sorted (struct maps *maps,bool value) { -struct map_rb_node *rb_node; +RC_CHK_ACCESS (maps)->maps_by_name_sorted = value; +} -maps__for_each_entry (maps, rb_node) { -if (rb_node->RC_CHK_ACCESS (map) ==RC_CHK_ACCESS (map)) -return rb_node; - } -return NULL ; +static struct rw_semaphore *maps__lock (struct maps *maps) +{ +/* +* When the lock is acquired or released the maps invariants should +* hold. +*/ +check_invariants (maps); +return &RC_CHK_ACCESS (maps)->lock; }static void maps__init (struct maps *maps,struct machine *machine) { -refcount_set (maps__refcnt (maps),1 );init_rwsem (maps__lock (maps)); -RC_CHK_ACCESS (maps)->entries =RB_ROOT ; +RC_CHK_ACCESS (maps)->maps_by_address =NULL ; +RC_CHK_ACCESS (maps)->maps_by_name =NULL ;RC_CHK_ACCESS (maps)->machine = machine; -RC_CHK_ACCESS (maps)->last_search_by_name =NULL ; +#ifdef HAVE_LIBUNWIND_SUPPORT +RC_CHK_ACCESS (maps)->addr_space =NULL ; +RC_CHK_ACCESS (maps)->unwind_libunwind_ops =NULL ; +#endif +refcount_set (maps__refcnt (maps),1 );RC_CHK_ACCESS (maps)->nr_maps =0 ; -RC_CHK_ACCESS (maps)->maps_by_name =NULL ; +RC_CHK_ACCESS (maps)->nr_maps_allocated =0 ; +RC_CHK_ACCESS (maps)->last_search_by_name_idx =0 ; +RC_CHK_ACCESS (maps)->maps_by_address_sorted = true; +RC_CHK_ACCESS (maps)->maps_by_name_sorted = false; } -static void __maps__free_maps_by_name (struct maps *maps) +static void maps__exit (struct maps *maps) { -/* -* Free everything to try to do it from the rbtree in the next search -*/ -for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) -map__put (maps__maps_by_name (maps)[i]); +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); -zfree (&RC_CHK_ACCESS (maps)->maps_by_name); -RC_CHK_ACCESS (maps)->nr_maps_allocated =0 ; +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +map__zput (maps_by_address[i]); +if (maps_by_name) +map__zput (maps_by_name[i]); + } +zfree (&maps_by_address); +zfree (&maps_by_name); +unwind__finish_access (maps); } -static int __maps__insert (struct maps *maps,struct map *map) +struct maps *maps__new (struct machine *machine) { -struct rb_node **p = &maps__entries (maps)->rb_node; -struct rb_node *parent =NULL ; -const u64 ip =map__start (map); -struct map_rb_node *m, *new_rb_node; - - new_rb_node =malloc (sizeof (*new_rb_node)); -if (!new_rb_node) -return -ENOMEM ; - -RB_CLEAR_NODE (&new_rb_node->rb_node); - new_rb_node->map =map__get (map); +struct maps *result; +RC_STRUCT (maps) *maps =zalloc (sizeof (*maps)); -while (*p !=NULL ) { - parent = *p; - m = rb_entry(parent, struct map_rb_node, rb_node); -if (ip <map__start (m->map)) - p = &(*p)->rb_left; -else - p = &(*p)->rb_right; - } +if (ADD_RC_CHK (result, maps)) +maps__init (result, machine); -rb_link_node (&new_rb_node->rb_node, parent, p); -rb_insert_color (&new_rb_node->rb_node,maps__entries (maps)); -return 0 ; +return result; } -int maps__insert (struct maps *maps,struct map *map) +static void maps__delete (struct maps *maps) { -int err; -const struct dso *dso =map__dso (map); - -down_write (maps__lock (maps)); - err =__maps__insert (maps, map); -if (err) - goto out; +maps__exit (maps); +RC_CHK_FREE (maps); +} - ++RC_CHK_ACCESS (maps)->nr_maps; +struct maps *maps__get (struct maps *maps) +{ +struct maps *result; -if (dso && dso->kernel) { -struct kmap *kmap =map__kmap (map); +if (RC_CHK_GET (result, maps)) +refcount_inc (maps__refcnt (maps)); -if (kmap) - kmap->kmaps = maps; -else -pr_err ("Internal error: kernel dso with non kernel map\n" ); - } +return result; +} +void maps__put (struct maps *maps) +{ +if (maps &&refcount_dec_and_test (maps__refcnt (maps))) +maps__delete (maps); +else +RC_CHK_PUT (maps); +} +static void __maps__free_maps_by_name (struct maps *maps) +{/* -* If we already performed some search by name, then we need to add the just -* inserted map and resort. +* Free everything to try to do it from the rbtree in the next search */ -if (maps__maps_by_name (maps)) { -if (maps__nr_maps (maps) >RC_CHK_ACCESS (maps)->nr_maps_allocated) { -int nr_allocate =maps__nr_maps (maps) *2 ; -struct map **maps_by_name =realloc (maps__maps_by_name (maps), - nr_allocate *sizeof (map)); +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) +map__put (maps__maps_by_name (maps)[i]); -if (maps_by_name ==NULL ) { -__maps__free_maps_by_name (maps); - err = -ENOMEM ; - goto out; - } +zfree (&RC_CHK_ACCESS (maps)->maps_by_name); +} -RC_CHK_ACCESS (maps)->maps_by_name = maps_by_name; -RC_CHK_ACCESS (maps)->nr_maps_allocated = nr_allocate; +static int map__start_cmp (const void *a,const void *b) +{ +const struct map *map_a = *(const struct map *const *)a; +const struct map *map_b = *(const struct map *const *)b; +u64 map_a_start =map__start (map_a); +u64 map_b_start =map__start (map_b); + +if (map_a_start == map_b_start) { +u64 map_a_end =map__end (map_a); +u64 map_b_end =map__end (map_b); + +if (map_a_end == map_b_end) { +/* Ensure maps with the same addresses have a fixed order. */ +if (RC_CHK_ACCESS (map_a) ==RC_CHK_ACCESS (map_b)) +return 0 ; +return (intptr_t )RC_CHK_ACCESS (map_a) > (intptr_t )RC_CHK_ACCESS (map_b) + ?1 :-1 ; } -maps__maps_by_name (maps)[maps__nr_maps (maps) -1 ] =map__get (map); -__maps__sort_by_name (maps); +return map_a_end > map_b_end ?1 :-1 ; } - out: -up_write (maps__lock (maps)); -return err; +return map_a_start > map_b_start ?1 :-1 ; } -static void __maps__remove (struct maps *maps,struct map_rb_node *rb_node) +static void __maps__sort_by_address (struct maps *maps) { -rb_erase_init (&rb_node->rb_node,maps__entries (maps)); -map__put (rb_node->map); -free (rb_node); +if (maps__maps_by_address_sorted (maps)) +return ; + +qsort (maps__maps_by_address (maps), +maps__nr_maps (maps), +sizeof (struct map *), + map__start_cmp); +maps__set_maps_by_address_sorted (maps, true); } -void maps__remove (struct maps *maps,struct map *map) +static void maps__sort_by_address (struct maps *maps) { -struct map_rb_node *rb_node; -down_write (maps__lock (maps)); -if (RC_CHK_ACCESS (maps)->last_search_by_name == map) -RC_CHK_ACCESS (maps)->last_search_by_name =NULL ; - - rb_node =maps__find_node (maps, map); -assert (rb_node->RC_CHK_ACCESS (map) ==RC_CHK_ACCESS (map)); -__maps__remove (maps, rb_node); -if (maps__maps_by_name (maps)) -__maps__free_maps_by_name (maps); - --RC_CHK_ACCESS (maps)->nr_maps; +__maps__sort_by_address (maps);up_write (maps__lock (maps)); } -static void __maps__purge (struct maps *maps) +static int map__strcmp (const void *a,const void *b) { -struct map_rb_node *pos, *next; - -if (maps__maps_by_name (maps)) -__maps__free_maps_by_name (maps); +const struct map *map_a = *(const struct map *const *)a; +const struct map *map_b = *(const struct map *const *)b; +const struct dso *dso_a =map__dso (map_a); +const struct dso *dso_b =map__dso (map_b); +int ret =strcmp (dso_a->short_name, dso_b->short_name); -maps__for_each_entry_safe (maps, pos, next) { -rb_erase_init (&pos->rb_node,maps__entries (maps)); -map__put (pos->map); -free (pos); +if (ret ==0 &&RC_CHK_ACCESS (map_a) !=RC_CHK_ACCESS (map_b)) { +/* Ensure distinct but name equal maps have an order. */ +return map__start_cmp (a, b); } +return ret; } -static void maps__exit (struct maps *maps) +static int maps__sort_by_name (struct maps *maps) { +int err =0 ;down_write (maps__lock (maps)); -__maps__purge (maps); +if (!maps__maps_by_name_sorted (maps)) { +struct map **maps_by_name =maps__maps_by_name (maps); + +if (!maps_by_name) { + maps_by_name =malloc (RC_CHK_ACCESS (maps)->nr_maps_allocated * +sizeof (*maps_by_name)); +if (!maps_by_name) + err = -ENOMEM ; +else { +struct map **maps_by_address =maps__maps_by_address (maps); +unsigned int n =maps__nr_maps (maps); + +maps__set_maps_by_name (maps, maps_by_name); +for (unsigned int i =0 ; i < n; i++) + maps_by_name[i] =map__get (maps_by_address[i]); + } + } +if (!err) { +qsort (maps_by_name, +maps__nr_maps (maps), +sizeof (struct map *), + map__strcmp); +maps__set_maps_by_name_sorted (maps, true); + } + }up_write (maps__lock (maps)); +return err; } -bool maps__empty (struct maps *maps) +static unsigned int maps__by_address_index (const struct maps *maps,const struct map *map) { -return !maps__first (maps); +struct map **maps_by_address =maps__maps_by_address (maps); + +if (maps__maps_by_address_sorted (maps)) { +struct map **mapp = +bsearch (&map,maps__maps_by_address (maps),maps__nr_maps (maps), +sizeof (*mapp), map__start_cmp); + +if (mapp) +return mapp - maps_by_address; + }else { +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +if (RC_CHK_ACCESS (maps_by_address[i]) ==RC_CHK_ACCESS (map)) +return i; + } + } +pr_err ("Map missing from maps" ); +return -1 ; } -struct maps *maps__new (struct machine *machine) +static unsigned int maps__by_name_index (const struct maps *maps,const struct map *map) { -struct maps *result; -RC_STRUCT (maps) *maps =zalloc (sizeof (*maps)); +struct map **maps_by_name =maps__maps_by_name (maps); + +if (maps__maps_by_name_sorted (maps)) { +struct map **mapp = +bsearch (&map, maps_by_name,maps__nr_maps (maps), +sizeof (*mapp), map__strcmp); + +if (mapp) +return mapp - maps_by_name; + }else { +for (unsigned int i =0 ; i <maps__nr_maps (maps); i++) { +if (RC_CHK_ACCESS (maps_by_name[i]) ==RC_CHK_ACCESS (map)) +return i; + } + } +pr_err ("Map missing from maps" ); +return -1 ; +} -if (ADD_RC_CHK (result, maps)) -maps__init (result, machine); +static int __maps__insert (struct maps *maps,struct map *new) +{ +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); +const struct dso *dso =map__dso (new); +unsigned int nr_maps =maps__nr_maps (maps); +unsigned int nr_allocate =RC_CHK_ACCESS (maps)->nr_maps_allocated; + +if (nr_maps +1 > nr_allocate) { + nr_allocate = !nr_allocate ?32 : nr_allocate *2 ; + + maps_by_address =realloc (maps_by_address, nr_allocate *sizeof (new)); +if (!maps_by_address) +return -ENOMEM ; + +maps__set_maps_by_address (maps, maps_by_address); +if (maps_by_name) { + maps_by_name =realloc (maps_by_name, nr_allocate *sizeof (new)); +if (!maps_by_name) { +/* +* If by name fails, just disable by name and it will +* recompute next time it is required. +*/ +__maps__free_maps_by_name (maps); + } +maps__set_maps_by_name (maps, maps_by_name); + } +RC_CHK_ACCESS (maps)->nr_maps_allocated = nr_allocate; + } +/* Insert the value at the end. */ + maps_by_address[nr_maps] =map__get (new); +if (maps_by_name) + maps_by_name[nr_maps] =map__get (new); -return result; + nr_maps++; +RC_CHK_ACCESS (maps)->nr_maps = nr_maps; + +/* +* Recompute if things are sorted. If things are inserted in a sorted +* manner, for example by processing /proc/pid/maps, then no +* sorting/resorting will be necessary. +*/ +if (nr_maps ==1 ) { +/* If there's just 1 entry then maps are sorted. */ +maps__set_maps_by_address_sorted (maps, true); +maps__set_maps_by_name_sorted (maps, maps_by_name !=NULL ); + }else { +/* Sorted if maps were already sorted and this map starts after the last one. */ +maps__set_maps_by_address_sorted (maps, +maps__maps_by_address_sorted (maps) && +map__end (maps_by_address[nr_maps -2 ]) <=map__start (new)); +maps__set_maps_by_name_sorted (maps, false); + } +if (map__end (new) <map__start (new)) +RC_CHK_ACCESS (maps)->ends_broken = true; +if (dso && dso->kernel) { +struct kmap *kmap =map__kmap (new); +Quoted 8 lines
tools/perf/util/maps.c line 390
Quoted 6 lines
tools/perf/util/maps.c +6
-0+if (kmap) + kmap->kmaps = maps; +else +pr_err ("Internal error: kernel dso with non kernel map\n" ); + } +check_invariants (maps);Probably not needed as it's checked when you get the lock below.
Quoted 1 line
Ack. Will remove in v3.
s/v3/v8/g :-)
Ian
Quoted 1,053 lines
Quoted 43 lines
Quoted 43 lines
tools/perf/util/maps.c +29
-6+return 0 ; } -static void maps__delete (struct maps *maps) +int maps__insert (struct maps *maps,struct map *map) { -maps__exit (maps); -unwind__finish_access (maps); -RC_CHK_FREE (maps); +int ret; + +down_write (maps__lock (maps)); + ret =__maps__insert (maps, map); +up_write (maps__lock (maps)); +return ret; } -struct maps *maps__get (struct maps *maps) +static void __maps__remove (struct maps *maps,struct map *map) { -struct maps *result; +struct map **maps_by_address =maps__maps_by_address (maps); +struct map **maps_by_name =maps__maps_by_name (maps); +unsigned int nr_maps =maps__nr_maps (maps); +unsigned int address_idx; + +/* Slide later mappings over the one to remove */ + address_idx =maps__by_address_index (maps, map); +map__put (maps_by_address[address_idx]); +memmove (&maps_by_address[address_idx], + &maps_by_address[address_idx +1 ], + (nr_maps - address_idx -1 ) *sizeof (*maps_by_address)); + +if (maps_by_name) { +unsigned int name_idx =maps__by_name_index (maps, map); + +map__put (maps_by_name[name_idx]); +memmove (&maps_by_name[name_idx], + &maps_by_name[name_idx +1 ], + (nr_maps - name_idx -1 ) *sizeof (*maps_by_name)); + }Quoted 6 lines
tools/perf/util/maps.c line 428
Quoted 4 lines
tools/perf/util/maps.c +2
-2-if (RC_CHK_GET (result, maps)) -refcount_inc (maps__refcnt (maps)); + --RC_CHK_ACCESS (maps)->nr_maps; +check_invariants (maps);Ditto.
Ack.
Quoted 400 lines
Quoted 400 lines
tools/perf/util/maps.c +196
-99+} -return result; +void maps__remove (struct maps *maps,struct map *map) +{ +down_write (maps__lock (maps)); +__maps__remove (maps, map); +up_write (maps__lock (maps)); } -void maps__put (struct maps *maps) +bool maps__empty (struct maps *maps) { -if (maps &&refcount_dec_and_test (maps__refcnt (maps))) -maps__delete (maps); -else -RC_CHK_PUT (maps); +return maps__nr_maps (maps) ==0 ; }int maps__for_each_map (struct maps *maps,int (*cb)(struct map *map,void *data),void *data) { -struct map_rb_node *pos; +bool done = false;int ret =0 ; -down_read (maps__lock (maps)); -maps__for_each_entry (maps, pos) { - ret =cb (pos->map, data); -if (ret) -break ; +/* See locking/sorting note. */ +while (!done) { +down_read (maps__lock (maps)); +if (maps__maps_by_address_sorted (maps)) { +struct map **maps_by_address =maps__maps_by_address (maps); +unsigned int n =maps__nr_maps (maps); + +for (unsigned int i =0 ; i < n; i++) { +struct map *map = maps_by_address[i]; + + ret =cb (map, data); +if (ret) +break ; + } + done = true; + } +up_read (maps__lock (maps)); +if (!done) +maps__sort_by_address (maps); } -up_read (maps__lock (maps));return ret; }void maps__remove_maps (struct maps *maps,bool (*cb)(struct map *map,void *data),void *data) { -struct map_rb_node *pos, *next; -unsigned int start_nr_maps; +struct map **maps_by_address;down_write (maps__lock (maps)); - start_nr_maps =maps__nr_maps (maps); -maps__for_each_entry_safe (maps, pos, next) { -if (cb (pos->map, data)) { -__maps__remove (maps, pos); - --RC_CHK_ACCESS (maps)->nr_maps; - } + maps_by_address =maps__maps_by_address (maps); +for (unsigned int i =0 ; i <maps__nr_maps (maps);) { +if (cb (maps_by_address[i], data)) +__maps__remove (maps, maps_by_address[i]); +else + i++; } -if (maps__maps_by_name (maps) && start_nr_maps !=maps__nr_maps (maps)) -__maps__free_maps_by_name (maps); -up_write (maps__lock (maps)); }@@ -300,7 +491,7 @@ struct symbol *maps__find_symbol(struct maps *maps, u64 addr, struct map **mapp)/* Ensure map is loaded before using map->map_ip */ if (map !=NULL &&map__load (map) >=0 ) {if (mapp !=NULL ) - *mapp = map; + *mapp = map;// TODO: map_put on else path when find returns a get. return map__find_symbol (map,map__map_ip (map, addr)); }@@ -348,7 +539,7 @@ int maps__find_ams(struct maps *maps, struct addr_map_symbol *ams)if (ams->addr <map__start (ams->ms.map) || ams->addr >=map__end (ams->ms.map)) {if (maps ==NULL )return -1 ; - ams->ms.map =maps__find (maps, ams->addr); + ams->ms.map =maps__find (maps, ams->addr);// TODO: map_get if (ams->ms.map ==NULL )return -1 ; }@@ -393,24 +584,28 @@ size_t maps__fprintf(struct maps *maps, FILE *fp)* Find first map where end > map->start. * Same as find_vma() in kernel. */ -static struct rb_node *first_ending_after (struct maps *maps,const struct map *map) +static unsigned int first_ending_after (struct maps *maps,const struct map *map) { -struct rb_root *root; -struct rb_node *next, *first; +struct map **maps_by_address =maps__maps_by_address (maps); +int low =0 , high = (int )maps__nr_maps (maps) -1 , first = high +1 ; + +assert (maps__maps_by_address_sorted (maps)); +if (low <= high &&map__end (maps_by_address[0 ]) >map__start (map)) +return 0 ; - root =maps__entries (maps); - next = root->rb_node; - first =NULL ; -while (next) { - struct map_rb_node *pos = rb_entry(next, struct map_rb_node, rb_node); +while (low <= high) { +int mid = (low + high) /2 ; +struct map *pos = maps_by_address[mid]; -if (map__end (pos->map) >map__start (map)) { - first = next; -if (map__start (pos->map) <=map__start (map)) +if (map__end (pos) >map__start (map)) { + first = mid; +if (map__start (pos) <=map__start (map)) { +/* Entry overlaps map. */ break ; - next = next->rb_left; + } + high = mid -1 ; }else - next = next->rb_right; + low = mid +1 ; }return first; }@@ -419,171 +614,249 @@ static struct rb_node *first_ending_after(struct maps *maps, const struct map *m* Adds new to maps, if new overlaps existing entries then the existing maps are * adjusted or removed so that new fits without overlapping any entries. */ -int maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) +static int __maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) { - -struct rb_node *next; +struct map **maps_by_address;int err =0 ;FILE *fp =debug_file (); -down_write (maps__lock (maps)); +sort_again: +if (!maps__maps_by_address_sorted (maps)) +__maps__sort_by_address (maps); - next =first_ending_after (maps, new); -while (next && !err) { - struct map_rb_node *pos = rb_entry(next, struct map_rb_node, rb_node); - next =rb_next (&pos->rb_node); + maps_by_address =maps__maps_by_address (maps); +/* +* Iterate through entries where the end of the existing entry is +* greater-than the new map's start. +*/ +for (unsigned int i =first_ending_after (maps, new); i <maps__nr_maps (maps); ) { +struct map *pos = maps_by_address[i]; +struct map *before =NULL , *after =NULL ;/* * Stop if current map starts after map->end. * Maps are ordered by start: next will not overlap for sure. */ -if (map__start (pos->map) >=map__end (new)) +if (map__start (pos) >=map__end (new))break ; -if (verbose >=2 ) { - -if (use_browser) { -pr_debug ("overlapping maps in %s (disable tui for more info)\n" , -map__dso (new)->name); - }else { -pr_debug ("overlapping maps:\n" ); -map__fprintf (new, fp); -map__fprintf (pos->map, fp); - } +if (use_browser) { +pr_debug ("overlapping maps in %s (disable tui for more info)\n" , +map__dso (new)->name); + }else if (verbose >=2 ) { +pr_debug ("overlapping maps:\n" ); +map__fprintf (new, fp); +map__fprintf (pos, fp); } -rb_erase_init (&pos->rb_node,maps__entries (maps));/* * Now check if we need to create new maps for areas not * overlapped by the new map: */ -if (map__start (new) >map__start (pos->map)) { -struct map *before =map__clone (pos->map); +if (map__start (new) >map__start (pos)) { +/* Map starts within existing map. Need to shorten the existing map. */ + before =map__clone (pos);if (before ==NULL ) { err = -ENOMEM ; - goto put_map; + goto out_err; } -map__set_end (before,map__start (new)); - err =__maps__insert (maps, before); -if (err) { -map__put (before); - goto put_map; - }if (verbose >=2 && !use_browser)map__fprintf (before, fp); -map__put (before); } - -if (map__end (new) <map__end (pos->map)) { -struct map *after =map__clone (pos->map); +if (map__end (new) <map__end (pos)) { +/* The new map isn't as long as the existing map. */ + after =map__clone (pos);if (after ==NULL ) { +map__zput (before); err = -ENOMEM ; - goto put_map; + goto out_err; }map__set_start (after,map__end (new)); -map__add_pgoff (after,map__end (new) -map__start (pos->map)); -assert (map__map_ip (pos->map,map__end (new)) == -map__map_ip (after,map__end (new))); - err =__maps__insert (maps, after); -if (err) { -map__put (after); - goto put_map; - } +map__add_pgoff (after,map__end (new) -map__start (pos)); +assert (map__map_ip (pos,map__end (new)) == +map__map_ip (after,map__end (new))); +if (verbose >=2 && !use_browser)map__fprintf (after, fp); -map__put (after); } -put_map: -map__put (pos->map); -free (pos); +/* +* If adding one entry, for `before` or `after`, we can replace +* the existing entry. If both `before` and `after` are +* necessary than an insert is needed. If the existing entry +* entirely overlaps the existing entry it can just be removed. +*/ +if (before) { +map__put (maps_by_address[i]); + maps_by_address[i] = before; +/* Maps are still ordered, go to next one. */ + i++; +if (after) { +__maps__insert (maps, after); +map__put (after); +if (!maps__maps_by_address_sorted (maps)) { +/* +* Sorting broken so invariants don't +* hold, sort and go again. +*/ + goto sort_again; + } +/* +* Maps are still ordered, skip after and go to +* next one (terminate loop). +*/ + i++; + } + }else if (after) { +map__put (maps_by_address[i]); + maps_by_address[i] = after; +/* Maps are ordered, go to next one. */ + i++; + }else { +__maps__remove (maps, pos); +/* +* Maps are ordered but no need to increase `i` as the +* later maps were moved down. +*/ + } +check_invariants (maps); }/* Add the map. */ - err =__maps__insert (maps, new); -up_write (maps__lock (maps)); +__maps__insert (maps, new); +out_err:return err; } -int maps__copy_from (struct maps *maps,struct maps *parent) +int maps__fixup_overlap_and_insert (struct maps *maps,struct map *new) {int err; -struct map_rb_node *rb_node; +down_write (maps__lock (maps)); + err =__maps__fixup_overlap_and_insert (maps, new); +up_write (maps__lock (maps)); +return err; +} + +int maps__copy_from (struct maps *dest,struct maps *parent) +{ +/* Note, if struct map were immutable then cloning could use ref counts. */ +struct map **parent_maps_by_address; +int err =0 ; +unsigned int n; + +down_write (maps__lock (dest));down_read (maps__lock (parent)); -maps__for_each_entry (parent, rb_node) { -struct map *new =map__clone (rb_node->map); + parent_maps_by_address =maps__maps_by_address (parent); + n =maps__nr_maps (parent); +if (maps__empty (dest)) { +/* No existing mappings so just copy from parent to avoid reallocs in insert. */ +unsigned int nr_maps_allocated =RC_CHK_ACCESS (parent)->nr_maps_allocated; +struct map **dest_maps_by_address = +malloc (nr_maps_allocated *sizeof (struct map *)); +struct map **dest_maps_by_name =NULL ; -if (new ==NULL ) { +if (!dest_maps_by_address) err = -ENOMEM ; - goto out_unlock; +else { +if (maps__maps_by_name (parent)) { + dest_maps_by_name = +malloc (nr_maps_allocated *sizeof (struct map *)); + } + +RC_CHK_ACCESS (dest)->maps_by_address = dest_maps_by_address; +RC_CHK_ACCESS (dest)->maps_by_name = dest_maps_by_name; +RC_CHK_ACCESS (dest)->nr_maps_allocated = nr_maps_allocated; } - err =unwind__prepare_access (maps, new,NULL ); -if (err) - goto out_unlock; +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos); - err =maps__insert (maps, new); -if (err) - goto out_unlock; +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) { + dest_maps_by_address[i] = new; +if (dest_maps_by_name) + dest_maps_by_name[i] =map__get (new); +RC_CHK_ACCESS (dest)->nr_maps = i +1 ; + } + } +if (err) +map__put (new); + } +maps__set_maps_by_address_sorted (dest,maps__maps_by_address_sorted (parent)); +if (!err) { +RC_CHK_ACCESS (dest)->last_search_by_name_idx = +RC_CHK_ACCESS (parent)->last_search_by_name_idx; +maps__set_maps_by_name_sorted (dest, + dest_maps_by_name && +maps__maps_by_name_sorted (parent)); + }else { +RC_CHK_ACCESS (dest)->last_search_by_name_idx =0 ; +maps__set_maps_by_name_sorted (dest, false); + } + }else { +/* Unexpected copying to a maps containing entries. */ +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos);Quoted 9 lines
tools/perf/util/maps.c static struct rb_node *first_ending_after(struct maps *maps, const struct map *m · line 816
Quoted 7 lines
tools/perf/util/maps.c +6
-1-map__put (new); +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) + err =maps__insert (dest, new);Shouldn't it be
__maps__insert()?On entry the read lock is taken on parent but no lock is taken on dest so the locked version is used.
Quoted 373 lines
Quoted 360 lines
tools/perf/util/maps.c +139
-160+ } +map__put (new); + } } - - err =0 ; -out_unlock:up_read (maps__lock (parent)); +up_write (maps__lock (dest));return err; } -struct map *maps__find (struct maps *maps,u64 ip) +static int map__addr_cmp (const void *key,const void *entry) { -struct rb_node *p; -struct map_rb_node *m; - +const u64 ip = *(const u64 *)key; +const struct map *map = *(const struct map *const *)entry; -down_read (maps__lock (maps)); - - p =maps__entries (maps)->rb_node; -while (p !=NULL ) { - m = rb_entry(p, struct map_rb_node, rb_node); -if (ip <map__start (m->map)) - p = p->rb_left; -else if (ip >=map__end (m->map)) - p = p->rb_right; -else - goto out; - } - - m =NULL ; -out: -up_read (maps__lock (maps)); -return m ? m->map :NULL ; +if (ip <map__start (map)) +return -1 ; +if (ip >=map__end (map)) +return 1 ; +return 0 ; } -static int map__strcmp (const void *a,const void *b) +struct map *maps__find (struct maps *maps,u64 ip) { -const struct map *map_a = *(const struct map **)a; -const struct map *map_b = *(const struct map **)b; -const struct dso *dso_a =map__dso (map_a); -const struct dso *dso_b =map__dso (map_b); -int ret =strcmp (dso_a->short_name, dso_b->short_name); - -if (ret ==0 && map_a != map_b) { -/* -* Ensure distinct but name equal maps have an order in part to -* aid reference counting. -*/ - ret = (int )map__start (map_a) - (int )map__start (map_b); -if (ret ==0 ) - ret = (int )((intptr_t )map_a - (intptr_t )map_b); +struct map *result =NULL ; +bool done = false; + +/* See locking/sorting note. */ +while (!done) { +down_read (maps__lock (maps)); +if (maps__maps_by_address_sorted (maps)) { +struct map **mapp = +bsearch (&ip,maps__maps_by_address (maps),maps__nr_maps (maps), +sizeof (*mapp), map__addr_cmp); + +if (mapp) + result = *mapp;// map__get(*mapp); + done = true; + } +up_read (maps__lock (maps)); +if (!done) +maps__sort_by_address (maps); } - -return ret; +return result; }static int map__strcmp_name (const void *name,const void *b)@@ -593,126 +866,113 @@ static int map__strcmp_name(const void *name, const void *b)return strcmp (name, dso->short_name); } -void __maps__sort_by_name (struct maps *maps) -{ -qsort (maps__maps_by_name (maps),maps__nr_maps (maps),sizeof (struct map *), map__strcmp); -} - -static int map__groups__sort_by_name_from_rbtree (struct maps *maps) -{ -struct map_rb_node *rb_node; -struct map **maps_by_name =realloc (maps__maps_by_name (maps), -maps__nr_maps (maps) *sizeof (struct map *)); -int i =0 ; - -if (maps_by_name ==NULL ) -return -1 ; - -up_read (maps__lock (maps)); -down_write (maps__lock (maps)); - -RC_CHK_ACCESS (maps)->maps_by_name = maps_by_name; -RC_CHK_ACCESS (maps)->nr_maps_allocated =maps__nr_maps (maps); - -maps__for_each_entry (maps, rb_node) - maps_by_name[i++] =map__get (rb_node->map); - -__maps__sort_by_name (maps); - -up_write (maps__lock (maps)); -down_read (maps__lock (maps)); - -return 0 ; -} - -static struct map *__maps__find_by_name (struct maps *maps,const char *name) +struct map *maps__find_by_name (struct maps *maps,const char *name) { -struct map **mapp; +struct map *result =NULL ; +bool done = false; -if (maps__maps_by_name (maps) ==NULL && -map__groups__sort_by_name_from_rbtree (maps)) -return NULL ; +/* See locking/sorting note. */ +while (!done) { +unsigned int i; - mapp =bsearch (name,maps__maps_by_name (maps),maps__nr_maps (maps), -sizeof (*mapp), map__strcmp_name); -if (mapp) -return *mapp; -return NULL ; -} +down_read (maps__lock (maps)); -struct map *maps__find_by_name (struct maps *maps,const char *name) -{ -struct map_rb_node *rb_node; -struct map *map; - -down_read (maps__lock (maps)); +/* First check last found entry. */ + i =RC_CHK_ACCESS (maps)->last_search_by_name_idx; +if (i <maps__nr_maps (maps) &&maps__maps_by_name (maps)) { +struct dso *dso =map__dso (maps__maps_by_name (maps)[i]); +if (dso &&strcmp (dso->short_name, name) ==0 ) { + result =maps__maps_by_name (maps)[i];// TODO: map__get + done = true; + } + } -if (RC_CHK_ACCESS (maps)->last_search_by_name) { -const struct dso *dso =map__dso (RC_CHK_ACCESS (maps)->last_search_by_name); +/* Second search sorted array. */ +if (!done &&maps__maps_by_name_sorted (maps)) { +struct map **mapp = +bsearch (name,maps__maps_by_name (maps),maps__nr_maps (maps), +sizeof (*mapp), map__strcmp_name); -if (strcmp (dso->short_name, name) ==0 ) { - map =RC_CHK_ACCESS (maps)->last_search_by_name; - goto out_unlock; +if (mapp) { + result = *mapp;// TODO: map__get + i = mapp -maps__maps_by_name (maps); +RC_CHK_ACCESS (maps)->last_search_by_name_idx = i; + } + done = true; } - } -/* -* If we have maps->maps_by_name, then the name isn't in the rbtree, -* as maps->maps_by_name mirrors the rbtree when lookups by name are -* made. -*/ - map =__maps__find_by_name (maps, name); -if (map ||maps__maps_by_name (maps) !=NULL ) - goto out_unlock; - -/* Fallback to traversing the rbtree... */ -maps__for_each_entry (maps, rb_node) { -struct dso *dso; - - map = rb_node->map; - dso =map__dso (map); -if (strcmp (dso->short_name, name) ==0 ) { -RC_CHK_ACCESS (maps)->last_search_by_name = map; - goto out_unlock; +up_read (maps__lock (maps)); +if (!done) { +/* Sort and retry binary search. */ +if (maps__sort_by_name (maps)) { +/* +* Memory allocation failed do linear search +* through address sorted maps. +*/ +struct map **maps_by_address; +unsigned int n; + +down_read (maps__lock (maps)); + maps_by_address =maps__maps_by_address (maps); + n =maps__nr_maps (maps); +for (i =0 ; i < n; i++) { +struct map *pos = maps_by_address[i]; +struct dso *dso =map__dso (pos); + +if (dso &&strcmp (dso->short_name, name) ==0 ) { + result = pos;// TODO: map__get +break ; + } + } +up_read (maps__lock (maps)); + done = true; + } } } - map =NULL ; - -out_unlock: -up_read (maps__lock (maps)); -return map; +return result; }struct map *maps__find_next_entry (struct maps *maps,struct map *map) { -struct map_rb_node *rb_node =maps__find_node (maps, map); -struct map_rb_node *next =map_rb_node__next (rb_node); +unsigned int i; +struct map *result =NULL ; -if (next) -return next->map; +down_read (maps__lock (maps)); + i =maps__by_address_index (maps, map); +if (i <maps__nr_maps (maps)) + result =maps__maps_by_address (maps)[i];// TODO: map__get -return NULL ; +up_read (maps__lock (maps)); +return result; }void maps__fixup_end (struct maps *maps) { -struct map_rb_node *prev =NULL , *curr; +struct map **maps_by_address; +unsigned int n;down_write (maps__lock (maps)); +if (!maps__maps_by_address_sorted (maps)) +__maps__sort_by_address (maps); -maps__for_each_entry (maps, curr) { -if (prev && (!map__end (prev->map) ||map__end (prev->map) >map__start (curr->map))) -map__set_end (prev->map,map__start (curr->map)); + maps_by_address =maps__maps_by_address (maps); + n =maps__nr_maps (maps); +for (unsigned int i =1 ; i < n; i++) { +struct map *prev = maps_by_address[i -1 ]; +struct map *curr = maps_by_address[i]; - prev = curr; +if (!map__end (prev) ||map__end (prev) >map__start (curr)) +map__set_end (prev,map__start (curr)); }/* * We still haven't the actual symbols, so guess the * last map final address. */ -if (curr && !map__end (curr->map)) -map__set_end (curr->map, ~0ULL ); +if (n >0 && !map__end (maps_by_address[n -1 ])) +map__set_end (maps_by_address[n -1 ], ~0ULL ); + +RC_CHK_ACCESS (maps)->ends_broken = false;up_write (maps__lock (maps)); }@@ -723,117 +983,92 @@ void maps__fixup_end(struct maps *maps)*/ int maps__merge_in (struct maps *kmaps,struct map *new_map) { -struct map_rb_node *rb_node; -struct rb_node *first; -bool overlaps; -LIST_HEAD (merged); -int err =0 ; +unsigned int first_after_, kmaps__nr_maps; +struct map **kmaps_maps_by_address; +struct map **merged_maps_by_address; +unsigned int merged_nr_maps_allocated; + +/* First try under a read lock. */ +while (true) { +down_read (maps__lock (kmaps)); +if (maps__maps_by_address_sorted (kmaps)) +break ; -down_read (maps__lock (kmaps)); - first =first_ending_after (kmaps, new_map); - rb_node = first ? rb_entry(first, struct map_rb_node, rb_node) : NULL; - overlaps = rb_node &&map__start (rb_node->map) <map__end (new_map); -up_read (maps__lock (kmaps)); +up_read (maps__lock (kmaps)); + +/* First after binary search requires sorted maps. Sort and try again. */ +maps__sort_by_address (kmaps); + } + first_after_ =first_ending_after (kmaps, new_map); + kmaps_maps_by_address =maps__maps_by_address (kmaps); -if (!overlaps) +if (first_after_ >=maps__nr_maps (kmaps) || +map__start (kmaps_maps_by_address[first_after_]) >=map__end (new_map)) { +/* No overlap so regular insert suffices. */ +up_read (maps__lock (kmaps));return maps__insert (kmaps, new_map); + } +up_read (maps__lock (kmaps)); -maps__for_each_entry (kmaps, rb_node) { -struct map *old_map = rb_node->map; +/* Plain insert with a read-lock failed, try again now with the write lock. */ +down_write (maps__lock (kmaps)); +if (!maps__maps_by_address_sorted (kmaps)) +__maps__sort_by_address (kmaps); -/* no overload with this one */ -if (map__end (new_map) <map__start (old_map) || -map__start (new_map) >=map__end (old_map)) -continue ; + first_after_ =first_ending_after (kmaps, new_map); + kmaps_maps_by_address =maps__maps_by_address (kmaps); + kmaps__nr_maps =maps__nr_maps (kmaps); -if (map__start (new_map) <map__start (old_map)) { -/* -* |new...... -* |old.... -*/ -if (map__end (new_map) <map__end (old_map)) { -/* -* |new......| -> |new..| -* |old....| -> |old....| -*/ -map__set_end (new_map,map__start (old_map)); - }else {tools/perf/util/maps.c void maps__fixup_end(struct maps *maps) · line 1026
Quoted 10 lines
tools/perf/util/maps.c +5
-5-/* -* |new.............| -> |new..| |new..| -* |old....| -> |old....| -*/ -struct map_list_node *m =map_list_node__new (); +if (first_after_ >= kmaps__nr_maps || +map__start (kmaps_maps_by_address[first_after_]) >=map__end (new_map)) { +/* No overlap so regular insert suffices. */ +up_write (maps__lock (kmaps)); +return maps__insert (kmaps, new_map);I think it could be:
Quoted 3 lines
ret = __maps__insert(kmaps, new_map); up_write(maps__lock(kmaps)); return ret;Ack. Will change in v3.
Quoted 17 lines
Quoted 6 lines
tools/perf/util/maps.c +6
-0+ } +/* Array to merge into, possibly 1 more for the sake of new_map. */ + merged_nr_maps_allocated =RC_CHK_ACCESS (kmaps)->nr_maps_allocated; +if (kmaps__nr_maps +1 == merged_nr_maps_allocated) + merged_nr_maps_allocated++; +tools/perf/util/maps.c void maps__fixup_end(struct maps *maps) · line 1042
Quoted 10 lines
tools/perf/util/maps.c +10
-0+ merged_maps_by_address =malloc (merged_nr_maps_allocated *sizeof (*merged_maps_by_address)); +if (!merged_maps_by_address) { +up_write (maps__lock (kmaps)); +return -ENOMEM ; + } +RC_CHK_ACCESS (kmaps)->maps_by_address = merged_maps_by_address; +RC_CHK_ACCESS (kmaps)->maps_by_address_sorted = true; +zfree (&RC_CHK_ACCESS (kmaps)->maps_by_name); +RC_CHK_ACCESS (kmaps)->maps_by_name_sorted = false; +RC_CHK_ACCESS (kmaps)->nr_maps_allocated = merged_nr_maps_allocated;Quoted 1 line
Why not use the accessor functions?
Ack. I've been holding back on accessors that are used once, but I will add them here.
Thanks,
IanQuoted 184 lines
Thanks,
NamhyungQuoted 181 lines
tools/perf/util/maps.c +17
-59-if (!m) { - err = -ENOMEM ; - goto out; - } +/* Copy entries before the new_map that can't overlap. */ +for (unsigned int i =0 ; i < first_after_; i++) + merged_maps_by_address[i] =map__get (kmaps_maps_by_address[i]); - m->map =map__clone (new_map); -if (!m->map) { -free (m); - err = -ENOMEM ; - goto out; - } +RC_CHK_ACCESS (kmaps)->nr_maps = first_after_; -map__set_end (m->map,map__start (old_map)); -list_add_tail (&m->node, &merged); -map__add_pgoff (new_map,map__end (old_map) -map__start (new_map)); -map__set_start (new_map,map__end (old_map)); - } - }else { -/* -* |new...... -* |old.... -*/ -if (map__end (new_map) <map__end (old_map)) { -/* -* |new..| -> x -* |old.........| -> |old.........| -*/ -map__put (new_map); - new_map =NULL ; -break ; - }else { -/* -* |new......| -> |new...| -* |old....| -> |old....| -*/ -map__add_pgoff (new_map,map__end (old_map) -map__start (new_map)); -map__set_start (new_map,map__end (old_map)); - } - } - } +/* Add the new map, it will be split when the later overlapping mappings are added. */ +__maps__insert (kmaps, new_map); -out: -while (!list_empty (&merged)) { -struct map_list_node *old_node; +/* Insert mappings after new_map, splitting new_map in the process. */ +for (unsigned int i = first_after_; i < kmaps__nr_maps; i++) +__maps__fixup_overlap_and_insert (kmaps, kmaps_maps_by_address[i]); - old_node = list_entry(merged.next, struct map_list_node, node); -list_del_init (&old_node->node); -if (!err) - err =maps__insert (kmaps, old_node->map); -map__put (old_node->map); -free (old_node); - } +/* Copy the maps from merged into kmaps. */ +for (unsigned int i =0 ; i < kmaps__nr_maps; i++) +map__zput (kmaps_maps_by_address[i]); -if (new_map) { -if (!err) - err =maps__insert (kmaps, new_map); -map__put (new_map); - } -return err; +free (kmaps_maps_by_address); +up_write (maps__lock (kmaps)); +return 0 ; }void maps__load_first (struct maps *maps) { -struct map_rb_node *first; -down_read (maps__lock (maps)); - first =maps__first (maps); -if (first) -map__load (first->map); +if (maps__nr_maps (maps) >0 ) +map__load (maps__maps_by_address (maps)[0 ]);up_read (maps__lock (maps)); }tools/perf/util/maps.h +44
-11@@ -25,21 +25,56 @@ static inline struct map_list_node *map_list_node__new(void)return malloc (sizeof (struct map_list_node )); } -struct map *maps__find (struct maps *maps,u64 addr); +/* +* Locking/sorting note: +* +* Sorting is done with the write lock, iteration and binary searching happens +* under the read lock requiring being sorted. There is a race between sorting +* releasing the write lock and acquiring the read lock for iteration/searching +* where another thread could insert and break the sorting of the maps. In +* practice inserting maps should be rare meaning that the race shouldn't lead +* to live lock. Removal of maps doesn't break being sorted. +*/ DECLARE_RC_STRUCT (maps) { -struct rb_root entries;struct rw_semaphore lock; -struct machine *machine; -struct map *last_search_by_name; +/** +* @maps_by_address: array of maps sorted by their starting address if +* maps_by_address_sorted is true. +*/ +struct map **maps_by_address; +/** +* @maps_by_name: optional array of maps sorted by their dso name if +* maps_by_name_sorted is true. +*/ struct map **maps_by_name; -refcount_t refcnt; -unsigned int nr_maps; -unsigned int nr_maps_allocated; +struct machine *machine;#ifdef HAVE_LIBUNWIND_SUPPORT -void *addr_space; +void *addr_space;const struct unwind_libunwind_ops *unwind_libunwind_ops;#endif +refcount_t refcnt; +/** +* @nr_maps: number of maps_by_address, and possibly maps_by_name, +* entries that contain maps. +*/ +unsigned int nr_maps; +/** +* @nr_maps_allocated: number of entries in maps_by_address and possibly +* maps_by_name. +*/ +unsigned int nr_maps_allocated; +/** +* @last_search_by_name_idx: cache of last found by name entry's index +* as frequent searches for the same dso name are common. +*/ +unsigned int last_search_by_name_idx; +/** @maps_by_address_sorted: is maps_by_address sorted. */ +bool maps_by_address_sorted; +/** @maps_by_name_sorted: is maps_by_name sorted. */ +bool maps_by_name_sorted; +/** @ends_broken: does the map contain a map where end values are unset/unsorted? */ +bool ends_broken; };#define KMAP_NAME_LEN 256@@ -102,6 +137,7 @@ size_t maps__fprintf(struct maps *maps, FILE *fp);int maps__insert (struct maps *maps,struct map *map);void maps__remove (struct maps *maps,struct map *map); +struct map *maps__find (struct maps *maps,u64 addr);struct symbol *maps__find_symbol (struct maps *maps,u64 addr,struct map **mapp);struct symbol *maps__find_symbol_by_name (struct maps *maps,const char *name,struct map **mapp);@@ -117,8 +153,6 @@ struct map *maps__find_next_entry(struct maps *maps, struct map *map);int maps__merge_in (struct maps *kmaps,struct map *new_map); -void __maps__sort_by_name (struct maps *maps); -void maps__fixup_end (struct maps *maps);void maps__load_first (struct maps *maps); --2.43.0.472.g3155946c3a-goog
Namhyung Kim
maintainerRe: [PATCH v7 01/25] perf maps: Switch from rbtree to lazily sorted array for addresses
In reply to Ian RogersHi Ian,
Sorry for the late reply.
Quoted 2 lines · Ian Rogers
On Thu, Feb 1, 2024 at 6:48 PM Namhyung Kim <[email protected]> wrote:
[SNIP]
Quoted 87 lines
Quoted 77 lines
Quoted 77 lines
tools/perf/util/maps.c +59
-10+int maps__copy_from (struct maps *dest,struct maps *parent) +{ +/* Note, if struct map were immutable then cloning could use ref counts. */ +struct map **parent_maps_by_address; +int err =0 ; +unsigned int n; + +down_write (maps__lock (dest));down_read (maps__lock (parent)); -maps__for_each_entry (parent, rb_node) { -struct map *new =map__clone (rb_node->map); + parent_maps_by_address =maps__maps_by_address (parent); + n =maps__nr_maps (parent); +if (maps__empty (dest)) { +/* No existing mappings so just copy from parent to avoid reallocs in insert. */ +unsigned int nr_maps_allocated =RC_CHK_ACCESS (parent)->nr_maps_allocated; +struct map **dest_maps_by_address = +malloc (nr_maps_allocated *sizeof (struct map *)); +struct map **dest_maps_by_name =NULL ; -if (new ==NULL ) { +if (!dest_maps_by_address) err = -ENOMEM ; - goto out_unlock; +else { +if (maps__maps_by_name (parent)) { + dest_maps_by_name = +malloc (nr_maps_allocated *sizeof (struct map *)); + } + +RC_CHK_ACCESS (dest)->maps_by_address = dest_maps_by_address; +RC_CHK_ACCESS (dest)->maps_by_name = dest_maps_by_name; +RC_CHK_ACCESS (dest)->nr_maps_allocated = nr_maps_allocated; } - err =unwind__prepare_access (maps, new,NULL ); -if (err) - goto out_unlock; +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos); - err =maps__insert (maps, new); -if (err) - goto out_unlock; +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) { + dest_maps_by_address[i] = new; +if (dest_maps_by_name) + dest_maps_by_name[i] =map__get (new); +RC_CHK_ACCESS (dest)->nr_maps = i +1 ; + } + } +if (err) +map__put (new); + } +maps__set_maps_by_address_sorted (dest,maps__maps_by_address_sorted (parent)); +if (!err) { +RC_CHK_ACCESS (dest)->last_search_by_name_idx = +RC_CHK_ACCESS (parent)->last_search_by_name_idx; +maps__set_maps_by_name_sorted (dest, + dest_maps_by_name && +maps__maps_by_name_sorted (parent)); + }else { +RC_CHK_ACCESS (dest)->last_search_by_name_idx =0 ; +maps__set_maps_by_name_sorted (dest, false); + } + }else { +/* Unexpected copying to a maps containing entries. */ +for (unsigned int i =0 ; !err && i < n; i++) { +struct map *pos = parent_maps_by_address[i]; +struct map *new =map__clone (pos);Quoted 9 lines
tools/perf/util/maps.c static struct rb_node *first_ending_after(struct maps *maps, const struct map *m · line 816
Quoted 7 lines
tools/perf/util/maps.c +6
-1-map__put (new); +if (!new) + err = -ENOMEM ; +else { + err =unwind__prepare_access (dest, new,NULL ); +if (!err) + err =maps__insert (dest, new);Shouldn't it be
__maps__insert()?
Quoted 2 lines
On entry the read lock is taken on parent but no lock is taken on dest so the locked version is used.
I think you added the writer lock on dest.
Thanks,
Namhyung
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
tools/perf/util/maps.c +4
-3+ } +map__put (new); + } } - - err =0 ; -out_unlock:up_read (maps__lock (parent)); +up_write (maps__lock (dest));return err; }
Ian Rogers
reviewerPATCH v7 2/25 perf maps: Get map before returning in maps__find
In reply to Ian RogersFinding a map is done under a lock, returning the map without a reference count means it can be removed without notice and causing uses after free. Grab a reference count to the map within the lock region and return this. Fix up locations that need a map__put following this.
--- tools/perf/arch/x86/tests/dwarf-unwind.c | 1 + tools/perf/tests/vmlinux-kallsyms.c | 5 ++--- tools/perf/util/bpf-event.c | 1 + tools/perf/util/event.c | 4 ++-- tools/perf/util/machine.c | 22 ++++++++-------------- tools/perf/util/maps.c | 17 ++++++++++------- tools/perf/util/symbol.c | 3 ++- 7 files changed, 26 insertions(+), 27 deletions(-)
tools/perf/arch/x86/tests/dwarf-unwind.c +1 -0
@@ -34,6 +34,7 @@ static int sample_ustack(struct perf_sample *sample, } stack_size =map__end (map) - sp; +map__put (map); stack_size = stack_size >STACK_SIZE ?STACK_SIZE : stack_size;memcpy (buf, (void *) sp, stack_size);
tools/perf/tests/vmlinux-kallsyms.c +2 -3
@@ -151,10 +151,8 @@ static int test__vmlinux_matches_kallsyms_cb2(struct map *map, void *data)u64 mem_end =map__unmap_ip (args->vmlinux_map,map__end (map)); pair =maps__find (args->kallsyms.kmaps, mem_start); -if (pair ==NULL ||map__priv (pair)) -return 0 ; -if (map__start (pair) == mem_start) { +if (pair !=NULL && !map__priv (pair) &&map__start (pair) == mem_start) {struct dso *dso =map__dso (map);if (!args->header_printed) {
@@ -170,6 +168,7 @@ static int test__vmlinux_matches_kallsyms_cb2(struct map *map, void *data)pr_info (" %s\n" , dso->name);map__set_priv (pair,1 ); } +map__put (pair);return 0 ; }
tools/perf/util/bpf-event.c +1 -0
@@ -63,6 +63,7 @@ static int machine__process_bpf_event_load(struct machine *machine, dso->bpf_prog.id = id; dso->bpf_prog.sub_id = i; dso->bpf_prog.env = env; +map__put (map); } }return 0 ;
tools/perf/util/event.c +2 -2
@@ -511,7 +511,7 @@ size_t perf_event__fprintf_text_poke(union perf_event *event, struct machine *mastruct addr_location al;addr_location__init (&al); - al.map =map__get (maps__find (machine__kernel_maps (machine), tp->addr)); + al.map =maps__find (machine__kernel_maps (machine), tp->addr);if (al.map &&map__load (al.map) >=0 ) { al.addr =map__map_ip (al.map, tp->addr); al.sym =map__find_symbol (al.map, al.addr);
@@ -641,7 +641,7 @@ struct map *thread__find_map(struct thread *thread, u8 cpumode, u64 addr,return NULL ; } al->maps =maps__get (maps); - al->map =map__get (maps__find (maps, al->addr)); + al->map =maps__find (maps, al->addr);if (al->map !=NULL ) {/* * Kernel maps might be changed when loading symbols so loading
tools/perf/util/machine.c +8 -14
@@ -896,7 +896,6 @@ static int machine__process_ksymbol_register(struct machine *machine,struct symbol *sym;struct dso *dso;struct map *map =maps__find (machine__kernel_maps (machine), event->ksymbol.addr); -bool put_map = false;int err =0 ;if (!map) {
@@ -913,12 +912,6 @@ static int machine__process_ksymbol_register(struct machine *machine, err = -ENOMEM ; goto out; } -/* -* The inserted map has a get on it, we need to put to release -* the reference count here, but do it after all accesses are -* done. -*/ - put_map = true;if (event->ksymbol.ksym_type ==PERF_RECORD_KSYMBOL_TYPE_OOL ) { dso->binary_type =DSO_BINARY_TYPE__OOL ; dso->data.file_size = event->ksymbol.len;
@@ -952,8 +945,7 @@ static int machine__process_ksymbol_register(struct machine *machine, }dso__insert_symbol (dso, sym); out: -if (put_map) -map__put (map); +map__put (map);return err; }
@@ -977,7 +969,7 @@ static int machine__process_ksymbol_unregister(struct machine *machine,if (sym)dso__delete_symbol (dso, sym); } - +map__put (map);return 0 ; }
@@ -1005,11 +997,11 @@ int machine__process_text_poke(struct machine *machine, union perf_event *event,perf_event__fprintf_text_poke (event, machine, stdout);if (!event->text_poke.new_len) -return 0 ; + goto out;if (cpumode !=PERF_RECORD_MISC_KERNEL ) {pr_debug ("%s: unsupported cpumode - ignoring\n" , __func__); -return 0 ; + goto out; }if (dso) {
@@ -1032,7 +1024,8 @@ int machine__process_text_poke(struct machine *machine, union perf_event *event,pr_debug ("Failed to find kernel text poke address map for %#" PRI_lx64"\n" , event->text_poke.addr); } - +out: +map__put (map);return 0 ; }
@@ -1300,9 +1293,10 @@ static int machine__map_x86_64_entry_trampolines_cb(struct map *map, void *data)return 0 ; dest_map =maps__find (args->kmaps,map__pgoff (map)); -if (dest_map != map) +if (RC_CHK_ACCESS (dest_map) !=RC_CHK_ACCESS (map))map__set_pgoff (map,map__map_ip (dest_map,map__pgoff (map))); +map__put (dest_map); args->found = true;return 0 ; }
tools/perf/util/maps.c +10 -7
@@ -487,15 +487,18 @@ void maps__remove_maps(struct maps *maps, bool (*cb)(struct map *map, void *datastruct symbol *maps__find_symbol (struct maps *maps,u64 addr,struct map **mapp) {struct map *map =maps__find (maps, addr); +struct symbol *result =NULL ;/* Ensure map is loaded before using map->map_ip */ if (map !=NULL &&map__load (map) >=0 ) { -if (mapp !=NULL ) - *mapp = map;// TODO: map_put on else path when find returns a get. -return map__find_symbol (map,map__map_ip (map, addr)); - } +if (mapp) + *mapp = map; -return NULL ; + result =map__find_symbol (map,map__map_ip (map, addr)); +if (!mapp) +map__put (map); + } +return result; }struct maps__find_symbol_by_name_args {
@@ -539,7 +542,7 @@ int maps__find_ams(struct maps *maps, struct addr_map_symbol *ams)if (ams->addr <map__start (ams->ms.map) || ams->addr >=map__end (ams->ms.map)) {if (maps ==NULL )return -1 ; - ams->ms.map =maps__find (maps, ams->addr);// TODO: map_get + ams->ms.map =maps__find (maps, ams->addr);if (ams->ms.map ==NULL )return -1 ; }
@@ -849,7 +852,7 @@ struct map *maps__find(struct maps *maps, u64 ip) sizeof(*mapp), map__addr_cmp);if (mapp) - result = *mapp;// map__get(*mapp); + result =map__get (*mapp); done = true; }up_read (maps__lock (maps));
tools/perf/util/symbol.c +2 -1
@@ -757,7 +757,6 @@ static int dso__load_all_kallsyms(struct dso *dso, const char *filename)static int maps__split_kallsyms_for_kcore (struct maps *kmaps,struct dso *dso) { -struct map *curr_map;struct symbol *pos;int count =0 ;struct rb_root_cached old_root = dso->symbols;
@@ -770,6 +769,7 @@ static int maps__split_kallsyms_for_kcore(struct maps *kmaps, struct dso *dso) *root =RB_ROOT_CACHED ;while (next) { +struct map *curr_map;struct dso *curr_map_dso;char *module;
@@ -796,6 +796,7 @@ static int maps__split_kallsyms_for_kcore(struct maps *kmaps, struct dso *dso) pos->end -=map__start (curr_map) -map__pgoff (curr_map);symbols__insert (&curr_map_dso->symbols, pos); ++count; +map__put (curr_map); }/* Symbols have been adjusted */
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 3/25 perf maps: Get map before returning in maps__find_by_name
In reply to Ian RogersFinding a map is done under a lock, returning the map without a reference count means it can be removed without notice and causing uses after free. Grab a reference count to the map within the lock region and return this. Fix up locations that need a map__put following this. Also fix some reference counted pointer comparisons.
--- tools/perf/tests/vmlinux-kallsyms.c | 5 +++-- tools/perf/util/machine.c | 6 ++++-- tools/perf/util/maps.c | 6 +++--- tools/perf/util/probe-event.c | 1 + tools/perf/util/symbol-elf.c | 4 +++- tools/perf/util/symbol.c | 18 +++++++++++------- 6 files changed, 25 insertions(+), 15 deletions(-)
tools/perf/tests/vmlinux-kallsyms.c +3 -2
@@ -131,9 +131,10 @@ static int test__vmlinux_matches_kallsyms_cb1(struct map *map, void *data)struct map *pair =maps__find_by_name (args->kallsyms.kmaps, (dso->kernel ? dso->short_name : dso->name)); -if (pair) +if (pair) {map__set_priv (pair,1 ); -else { +map__put (pair); + }else {if (!args->header_printed) {pr_info ("WARN: Maps only in vmlinux:\n" ); args->header_printed = true;
tools/perf/util/machine.c +4 -2
@@ -1537,8 +1537,10 @@ static int maps__set_module_path(struct maps *maps, const char *path, struct kmoreturn 0 ; long_name =strdup (path); -if (long_name ==NULL ) +if (long_name ==NULL ) { +map__put (map);return -ENOMEM ; + } dso =map__dso (map);dso__set_long_name (dso, long_name, true);
@@ -1552,7 +1554,7 @@ static int maps__set_module_path(struct maps *maps, const char *path, struct kmo dso->symtab_type++; dso->comp = m->comp; } - +map__put (map);return 0 ; }
tools/perf/util/maps.c +3 -3
@@ -886,7 +886,7 @@ struct map *maps__find_by_name(struct maps *maps, const char *name)struct dso *dso =map__dso (maps__maps_by_name (maps)[i]);if (dso &&strcmp (dso->short_name, name) ==0 ) { - result =maps__maps_by_name (maps)[i];// TODO: map__get + result =map__get (maps__maps_by_name (maps)[i]); done = true; } }
@@ -898,7 +898,7 @@ struct map *maps__find_by_name(struct maps *maps, const char *name) sizeof(*mapp), map__strcmp_name);if (mapp) { - result = *mapp;// TODO: map__get + result =map__get (*mapp); i = mapp -maps__maps_by_name (maps);RC_CHK_ACCESS (maps)->last_search_by_name_idx = i; }
@@ -923,7 +923,7 @@ struct map *maps__find_by_name(struct maps *maps, const char *name)struct dso *dso =map__dso (pos);if (dso &&strcmp (dso->short_name, name) ==0 ) { - result = pos;// TODO: map__get + result =map__get (pos);break ; } }
tools/perf/util/probe-event.c +1 -0
@@ -358,6 +358,7 @@ static int kernel_get_module_dso(const char *module, struct dso **pdso) map =maps__find_by_name (machine__kernel_maps (host_machine), module_name);if (map) { dso =map__dso (map); +map__put (map); goto found; }pr_debug ("Failed to find module %s.\n" , module);
tools/perf/util/symbol-elf.c +3 -1
@@ -1470,8 +1470,10 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,dso__set_loaded (curr_dso); *curr_mapp = curr_map; *curr_dsop = curr_dso; - }else + }else { *curr_dsop =map__dso (curr_map); +map__put (curr_map); + }return 0 ; }
tools/perf/util/symbol.c +11 -7
@@ -814,7 +814,7 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta, struct map *initial_map) {struct machine *machine; -struct map *curr_map = initial_map; +struct map *curr_map =map__get (initial_map);struct symbol *pos;int count =0 , moved =0 ;struct rb_root_cached *root = &dso->symbols;
@@ -858,13 +858,14 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta,dso__set_loaded (curr_map_dso); } +map__zput (curr_map); curr_map =maps__find_by_name (kmaps, module);if (curr_map ==NULL ) {pr_debug ("%s/proc/{kallsyms,modules} " "inconsistency while looking " "for \"%s\" module!\n" , machine->root_dir, module); - curr_map = initial_map; + curr_map =map__get (initial_map); goto discard_symbol; } curr_map_dso =map__dso (curr_map);
@@ -888,7 +889,7 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta,* symbols at this point. */ goto discard_symbol; - }else if (curr_map != initial_map) { + }else if (!RC_CHK_EQUAL (curr_map, initial_map)) {char dso_name[PATH_MAX ];struct dso *ndso;
@@ -899,7 +900,8 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta, }if (count ==0 ) { - curr_map = initial_map; +map__zput (curr_map); + curr_map =map__get (initial_map); goto add_symbol; }
@@ -913,6 +915,7 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta, kernel_range++); ndso =dso__new (dso_name); +map__zput (curr_map);if (ndso ==NULL )return -1 ;
@@ -926,6 +929,7 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta,map__set_mapping_type (curr_map,MAPPING_TYPE__IDENTITY );if (maps__insert (kmaps, curr_map)) { +map__zput (curr_map);dso__put (ndso);return -1 ; }
@@ -936,7 +940,7 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta, pos->end -= delta; } add_symbol: -if (curr_map != initial_map) { +if (!RC_CHK_EQUAL (curr_map, initial_map)) {struct dso *curr_map_dso =map__dso (curr_map);rb_erase_cached (&pos->rb_node, root);
@@ -951,12 +955,12 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta,symbol__delete (pos); } -if (curr_map != initial_map && +if (!RC_CHK_EQUAL (curr_map, initial_map) && dso->kernel ==DSO_SPACE__KERNEL_GUEST &&machine__is_default_guest (maps__machine (kmaps))) {dso__set_loaded (map__dso (curr_map)); } - +map__put (curr_map);return count + moved; }
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 4/25 perf maps: Get map before returning in maps__find_next_entry
In reply to Ian RogersFinding a map is done under a lock, returning the map without a reference count means it can be removed without notice and causing uses after free. Grab a reference count to the map within the lock region and return this. Fix up locations that need a map__put following this.
--- tools/perf/util/machine.c | 4 +++- tools/perf/util/maps.c | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-)
tools/perf/util/machine.c +3 -1
@@ -1761,8 +1761,10 @@ int machine__create_kernel_maps(struct machine *machine)struct map *next =maps__find_next_entry (machine__kernel_maps (machine),machine__kernel_map (machine)); -if (next) +if (next) {machine__set_kernel_mmap (machine, start,map__start (next)); +map__put (next); + } } out_put:
tools/perf/util/maps.c +1 -1
@@ -943,7 +943,7 @@ struct map *maps__find_next_entry(struct maps *maps, struct map *map)down_read (maps__lock (maps)); i =maps__by_address_index (maps, map);if (i <maps__nr_maps (maps)) - result =maps__maps_by_address (maps)[i];// TODO: map__get + result =map__get (maps__maps_by_address (maps)[i]);up_read (maps__lock (maps));return result;
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 5/25 perf maps: Hide maps internals
In reply to Ian RogersMove the struct into the C file. Add maps__equal to work around exposing the struct for reference count checking. Add accessors for the unwind_libunwind_ops. Move maps_list_node to its only use in symbol.c.
--- tools/perf/tests/thread-maps-share.c | 8 +- tools/perf/util/callchain.c | 2 +- tools/perf/util/maps.c | 96 +++++++++++++++++++++++ tools/perf/util/maps.h | 97 +++--------------------- tools/perf/util/symbol.c | 10 +++ tools/perf/util/thread.c | 2 +- tools/perf/util/unwind-libunwind-local.c | 2 +- tools/perf/util/unwind-libunwind.c | 7 +- 8 files changed, 123 insertions(+), 101 deletions(-)
tools/perf/tests/thread-maps-share.c +4 -4
@@ -46,9 +46,9 @@ static int test__thread_maps_share(struct test_suite *test __maybe_unused, int sTEST_ASSERT_EQUAL ("wrong refcnt" ,refcount_read (maps__refcnt (maps)),4 );/* test the maps pointer is shared */ -TEST_ASSERT_VAL ("maps don't match" ,RC_CHK_EQUAL (maps,thread__maps (t1))); -TEST_ASSERT_VAL ("maps don't match" ,RC_CHK_EQUAL (maps,thread__maps (t2))); -TEST_ASSERT_VAL ("maps don't match" ,RC_CHK_EQUAL (maps,thread__maps (t3))); +TEST_ASSERT_VAL ("maps don't match" ,maps__equal (maps,thread__maps (t1))); +TEST_ASSERT_VAL ("maps don't match" ,maps__equal (maps,thread__maps (t2))); +TEST_ASSERT_VAL ("maps don't match" ,maps__equal (maps,thread__maps (t3)));/* * Verify the other leader was created by previous call.
@@ -73,7 +73,7 @@ static int test__thread_maps_share(struct test_suite *test __maybe_unused, int s other_maps =thread__maps (other);TEST_ASSERT_EQUAL ("wrong refcnt" ,refcount_read (maps__refcnt (other_maps)),2 ); -TEST_ASSERT_VAL ("maps don't match" ,RC_CHK_EQUAL (other_maps,thread__maps (other_leader))); +TEST_ASSERT_VAL ("maps don't match" ,maps__equal (other_maps,thread__maps (other_leader)));/* release thread group */ thread__put (t3);
tools/perf/util/callchain.c +1 -1
@@ -1157,7 +1157,7 @@ int fill_callchain_info(struct addr_location *al, struct callchain_cursor_node *if (al->map ==NULL ) goto out; } -if (RC_CHK_EQUAL (al->maps,machine__kernel_maps (machine))) { +if (maps__equal (al->maps,machine__kernel_maps (machine))) {if (machine__is_host (machine)) { al->cpumode =PERF_RECORD_MISC_KERNEL ; al->level ='k' ;
tools/perf/util/maps.c +96 -0
117 lines not on this page · raw message
tools/perf/util/maps.h +9 -88
125 lines not on this page · raw message
tools/perf/util/symbol.c +10 -0
@@ -63,6 +63,16 @@ struct symbol_conf symbol_conf = { .res_sample = 0, }; +struct map_list_node { +struct list_head node; +struct map *map; +}; + +static struct map_list_node *map_list_node__new (void ) +{ +return malloc (sizeof (struct map_list_node )); +} +static enum dso_binary_type binary_type_symtab[] = {DSO_BINARY_TYPE__KALLSYMS ,DSO_BINARY_TYPE__GUEST_KALLSYMS ,
tools/perf/util/thread.c +1 -1
@@ -383,7 +383,7 @@ static int thread__clone_maps(struct thread *thread, struct thread *parent, boolif (thread__pid (thread) ==thread__pid (parent))return thread__prepare_access (thread); -if (RC_CHK_EQUAL (thread__maps (thread),thread__maps (parent))) { +if (maps__equal (thread__maps (thread),thread__maps (parent))) {pr_debug ("broken map groups on thread %d/%d parent %d/%d\n" ,thread__pid (thread),thread__tid (thread),thread__pid (parent),thread__tid (parent));
tools/perf/util/unwind-libunwind-local.c +1 -1
@@ -706,7 +706,7 @@ static int _unwind__prepare_access(struct maps *maps) {void *addr_space =unw_create_addr_space (&accessors,0 ); -RC_CHK_ACCESS (maps)->addr_space = addr_space; +maps__set_addr_space (maps, addr_space);if (!addr_space) {pr_err ("unwind: Can't create unwind address space.\n" );return -ENOMEM ;
tools/perf/util/unwind-libunwind.c +1 -6
@@ -12,11 +12,6 @@ struct unwind_libunwind_ops __weak *local_unwind_libunwind_ops;struct unwind_libunwind_ops __weak *x86_32_unwind_libunwind_ops;struct unwind_libunwind_ops __weak *arm64_unwind_libunwind_ops; -static void unwind__register_ops (struct maps *maps,struct unwind_libunwind_ops *ops) -{ -RC_CHK_ACCESS (maps)->unwind_libunwind_ops = ops; -} -int unwind__prepare_access (struct maps *maps,struct map *map,bool *initialized) {const char *arch;
@@ -60,7 +55,7 @@ int unwind__prepare_access(struct maps *maps, struct map *map, bool *initializedreturn 0 ; } out_register: -unwind__register_ops (maps, ops); +maps__set_unwind_libunwind_ops (maps, ops); err =maps__unwind_libunwind_ops (maps)->prepare_access (maps);if (initialized)
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 6/25 perf maps: Locking tidy up of nr_maps
In reply to Ian RogersAfter this change maps__nr_maps is only used by tests, existing users are migrated to maps__empty. Compute maps__empty under the read lock.
--- tools/perf/util/machine.c | 2 +- tools/perf/util/maps.c | 10 ++++++++-- tools/perf/util/maps.h | 4 ++-- 3 files changed, 11 insertions(+), 5 deletions(-)
tools/perf/util/machine.c +1 -1
@@ -440,7 +440,7 @@ static struct thread *findnew_guest_code(struct machine *machine,return NULL ;/* Assume maps are set up if there are any */ -if (maps__nr_maps (thread__maps (thread))) +if (!maps__empty (thread__maps (thread)))return thread; host_thread =machine__find_thread (host_machine,-1 , pid);
tools/perf/util/maps.c +8 -2
@@ -528,7 +528,13 @@ void maps__remove(struct maps *maps, struct map *map)bool maps__empty (struct maps *maps) { -return maps__nr_maps (maps) ==0 ; +bool res; + +down_read (maps__lock (maps)); + res =maps__nr_maps (maps) ==0 ; +up_read (maps__lock (maps)); + +return res; }bool maps__equal (struct maps *a,struct maps *b)
@@ -852,7 +858,7 @@ int maps__copy_from(struct maps *dest, struct maps *parent) parent_maps_by_address =maps__maps_by_address (parent); n =maps__nr_maps (parent); -if (maps__empty (dest)) { +if (maps__nr_maps (dest) ==0 ) {/* No existing mappings so just copy from parent to avoid reallocs in insert. */ unsigned int nr_maps_allocated =RC_CHK_ACCESS (parent)->nr_maps_allocated; struct map **dest_maps_by_address =
tools/perf/util/maps.h +2 -2
@@ -43,8 +43,8 @@ int maps__for_each_map(struct maps *maps, int (*cb)(struct map *map, void *data)void maps__remove_maps (struct maps *maps,bool (*cb)(struct map *map,void *data),void *data);struct machine *maps__machine (const struct maps *maps); -unsigned int maps__nr_maps (const struct maps *maps); -refcount_t *maps__refcnt (struct maps *maps); +unsigned int maps__nr_maps (const struct maps *maps);/* Test only. */ +refcount_t *maps__refcnt (struct maps *maps);/* Test only. */ #ifdef HAVE_LIBUNWIND_SUPPORT void *maps__addr_space (const struct maps *maps);
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 7/25 perf dso: Reorder variables to save space in struct dso
In reply to Ian RogersSave 40 bytes and move from 8 to 7 cache lines. Make variable dwfl dependent on being a powerpc build. Squeeze bits of int/enum types when appropriate. Remove holes/padding by reordering variables.
Before:
```
struct dso {
struct mutex lock; /* 0 40 */
struct list_head node; /* 40 16 */
struct rb_node rb_node __attribute__((__aligned__(8))); /* 56 24 */
/* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
struct rb_root * root; /* 80 8 */
struct rb_root_cached symbols; /* 88 16 */
struct symbol * * symbol_names; /* 104 8 */
size_t symbol_names_len; /* 112 8 */
struct rb_root_cached inlined_nodes; /* 120 16 */
/* --- cacheline 2 boundary (128 bytes) was 8 bytes ago --- */
struct rb_root_cached srclines; /* 136 16 */
struct {
u64 addr; /* 152 8 */
struct symbol * symbol; /* 160 8 */
} last_find_result; /* 152 16 */
void * a2l; /* 168 8 */
char * symsrc_filename; /* 176 8 */
unsigned int a2l_fails; /* 184 4 */
enum dso_space_type kernel; /* 188 4 */
/* --- cacheline 3 boundary (192 bytes) --- */
_Bool is_kmod; /* 192 1 */
/* XXX 3 bytes hole, try to pack */
enum dso_swap_type needs_swap; /* 196 4 */
enum dso_binary_type symtab_type; /* 200 4 */
enum dso_binary_type binary_type; /* 204 4 */
enum dso_load_errno load_errno; /* 208 4 */
u8 adjust_symbols:1; /* 212: 0 1 */
u8 has_build_id:1; /* 212: 1 1 */
u8 header_build_id:1; /* 212: 2 1 */
u8 has_srcline:1; /* 212: 3 1 */
u8 hit:1; /* 212: 4 1 */
u8 annotate_warned:1; /* 212: 5 1 */
u8 auxtrace_warned:1; /* 212: 6 1 */
u8 short_name_allocated:1; /* 212: 7 1 */
u8 long_name_allocated:1; /* 213: 0 1 */
u8 is_64_bit:1; /* 213: 1 1 */
/* XXX 6 bits hole, try to pack */
_Bool sorted_by_name; /* 214 1 */
_Bool loaded; /* 215 1 */
u8 rel; /* 216 1 */
/* XXX 7 bytes hole, try to pack */
struct build_id bid; /* 224 32 */
/* --- cacheline 4 boundary (256 bytes) --- */
u64 text_offset; /* 256 8 */
u64 text_end; /* 264 8 */
const char * short_name; /* 272 8 */
const char * long_name; /* 280 8 */
u16 long_name_len; /* 288 2 */
u16 short_name_len; /* 290 2 */
/* XXX 4 bytes hole, try to pack */
void * dwfl; /* 296 8 */
struct auxtrace_cache * auxtrace_cache; /* 304 8 */
int comp; /* 312 4 */
/* XXX 4 bytes hole, try to pack */
/* --- cacheline 5 boundary (320 bytes) --- */
struct {
struct rb_root cache; /* 320 8 */
int fd; /* 328 4 */
int status; /* 332 4 */
u32 status_seen; /* 336 4 */
/* XXX 4 bytes hole, try to pack */
u64 file_size; /* 344 8 */
struct list_head open_entry; /* 352 16 */
u64 elf_base_addr; /* 368 8 */
u64 debug_frame_offset; /* 376 8 */
/* --- cacheline 6 boundary (384 bytes) --- */
u64 eh_frame_hdr_addr; /* 384 8 */
u64 eh_frame_hdr_offset; /* 392 8 */
} data; /* 320 80 */
struct {
u32 id; /* 400 4 */
u32 sub_id; /* 404 4 */
struct perf_env * env; /* 408 8 */
} bpf_prog; /* 400 16 */
union {
void * priv; /* 416 8 */
u64 db_id; /* 416 8 */
}; /* 416 8 */
struct nsinfo * nsinfo; /* 424 8 */
struct dso_id id; /* 432 24 */
/* --- cacheline 7 boundary (448 bytes) was 8 bytes ago --- */
refcount_t refcnt; /* 456 4 */
char name[]; /* 460 0 */
/* size: 464, cachelines: 8, members: 49 */
/* sum members: 440, holes: 4, sum holes: 18 */
/* sum bitfield members: 10 bits, bit holes: 1, sum bit holes: 6 bits */
/* padding: 4 */
/* forced alignments: 1 */
/* last cacheline: 16 bytes */
} __attribute__((__aligned__(8)));
```After:
```
struct dso {
struct mutex lock; /* 0 40 */
struct list_head node; /* 40 16 */
struct rb_node rb_node __attribute__((__aligned__(8))); /* 56 24 */
/* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
struct rb_root * root; /* 80 8 */
struct rb_root_cached symbols; /* 88 16 */
struct symbol * * symbol_names; /* 104 8 */
size_t symbol_names_len; /* 112 8 */
struct rb_root_cached inlined_nodes; /* 120 16 */
/* --- cacheline 2 boundary (128 bytes) was 8 bytes ago --- */
struct rb_root_cached srclines; /* 136 16 */
struct {
u64 addr; /* 152 8 */
struct symbol * symbol; /* 160 8 */
} last_find_result; /* 152 16 */
struct build_id bid; /* 168 32 */
/* --- cacheline 3 boundary (192 bytes) was 8 bytes ago --- */
u64 text_offset; /* 200 8 */
u64 text_end; /* 208 8 */
const char * short_name; /* 216 8 */
const char * long_name; /* 224 8 */
void * a2l; /* 232 8 */
char * symsrc_filename; /* 240 8 */
struct nsinfo * nsinfo; /* 248 8 */
/* --- cacheline 4 boundary (256 bytes) --- */
struct auxtrace_cache * auxtrace_cache; /* 256 8 */
union {
void * priv; /* 264 8 */
u64 db_id; /* 264 8 */
}; /* 264 8 */
struct {
struct perf_env * env; /* 272 8 */
u32 id; /* 280 4 */
u32 sub_id; /* 284 4 */
} bpf_prog; /* 272 16 */
struct {
struct rb_root cache; /* 288 8 */
struct list_head open_entry; /* 296 16 */
u64 file_size; /* 312 8 */
/* --- cacheline 5 boundary (320 bytes) --- */
u64 elf_base_addr; /* 320 8 */
u64 debug_frame_offset; /* 328 8 */
u64 eh_frame_hdr_addr; /* 336 8 */
u64 eh_frame_hdr_offset; /* 344 8 */
int fd; /* 352 4 */
int status; /* 356 4 */
u32 status_seen; /* 360 4 */
} data; /* 288 80 */
/* XXX last struct has 4 bytes of padding */
struct dso_id id; /* 368 24 */
/* --- cacheline 6 boundary (384 bytes) was 8 bytes ago --- */
unsigned int a2l_fails; /* 392 4 */
int comp; /* 396 4 */
refcount_t refcnt; /* 400 4 */
enum dso_load_errno load_errno; /* 404 4 */
u16 long_name_len; /* 408 2 */
u16 short_name_len; /* 410 2 */
enum dso_binary_type symtab_type:8; /* 412: 0 4 */
enum dso_binary_type binary_type:8; /* 412: 8 4 */
enum dso_space_type kernel:2; /* 412:16 4 */
enum dso_swap_type needs_swap:2; /* 412:18 4 */
/* Bitfield combined with next fields */
_Bool is_kmod:1; /* 414: 4 1 */
u8 adjust_symbols:1; /* 414: 5 1 */
u8 has_build_id:1; /* 414: 6 1 */
u8 header_build_id:1; /* 414: 7 1 */
u8 has_srcline:1; /* 415: 0 1 */
u8 hit:1; /* 415: 1 1 */
u8 annotate_warned:1; /* 415: 2 1 */
u8 auxtrace_warned:1; /* 415: 3 1 */
u8 short_name_allocated:1; /* 415: 4 1 */
u8 long_name_allocated:1; /* 415: 5 1 */
u8 is_64_bit:1; /* 415: 6 1 */
/* XXX 1 bit hole, try to pack */
_Bool sorted_by_name; /* 416 1 */
_Bool loaded; /* 417 1 */
u8 rel; /* 418 1 */
char name[]; /* 419 0 */
/* size: 424, cachelines: 7, members: 48 */
/* sum members: 415 */
/* sum bitfield members: 31 bits, bit holes: 1, sum bit holes: 1 bits */
/* padding: 5 */
/* paddings: 1, sum paddings: 4 */
/* forced alignments: 1 */
/* last cacheline: 40 bytes */
} __attribute__((__aligned__(8)));
```--- tools/perf/util/dso.h | 84 +++++++++++++++++++++---------------------- 1 file changed, 42 insertions(+), 42 deletions(-)
tools/perf/util/dso.h +42 -42
@@ -160,66 +160,66 @@ struct dso {u64 addr;struct symbol *symbol; } last_find_result; -void *a2l; -char *symsrc_filename; -unsigned int a2l_fails; -enum dso_space_type kernel; -bool is_kmod; -enum dso_swap_type needs_swap; -enum dso_binary_type symtab_type; -enum dso_binary_type binary_type; -enum dso_load_errno load_errno; - u8 adjust_symbols:1; - u8 has_build_id:1; - u8 header_build_id:1; - u8 has_srcline:1; - u8 hit:1; - u8 annotate_warned:1; - u8 auxtrace_warned:1; - u8 short_name_allocated:1; - u8 long_name_allocated:1; - u8 is_64_bit:1; -bool sorted_by_name; -bool loaded; -u8 rel;struct build_id bid;u64 text_offset;u64 text_end;const char *short_name;const char *long_name; -u16 long_name_len; -u16 short_name_len; +void *a2l; +char *symsrc_filename; +#if defined(__powerpc__)void *dwfl;/* DWARF debug info */ +#endif +struct nsinfo *nsinfo;struct auxtrace_cache *auxtrace_cache; -int comp; - +union {/* Tool specific area */ +void *priv; +u64 db_id; + }; +/* bpf prog information */ +struct { +struct perf_env *env; +u32 id; +u32 sub_id; + } bpf_prog;/* dso data file */ struct {struct rb_root cache; -int fd; -int status; -u32 status_seen; -u64 file_size;struct list_head open_entry; +u64 file_size;u64 elf_base_addr;u64 debug_frame_offset;u64 eh_frame_hdr_addr;u64 eh_frame_hdr_offset; +int fd; +int status; +u32 status_seen; } data; -/* bpf prog information */ -struct { -u32 id; -u32 sub_id; -struct perf_env *env; - } bpf_prog; - -union {/* Tool specific area */ -void *priv; -u64 db_id; - }; -struct nsinfo *nsinfo;struct dso_id id; +unsigned int a2l_fails; +int comp;refcount_t refcnt; +enum dso_load_errno load_errno; +u16 long_name_len; +u16 short_name_len; + enum dso_binary_type symtab_type:8; + enum dso_binary_type binary_type:8; + enum dso_space_type kernel:2; + enum dso_swap_type needs_swap:2; + bool is_kmod:1; + u8 adjust_symbols:1; + u8 has_build_id:1; + u8 header_build_id:1; + u8 has_srcline:1; + u8 hit:1; + u8 annotate_warned:1; + u8 auxtrace_warned:1; + u8 short_name_allocated:1; + u8 long_name_allocated:1; + u8 is_64_bit:1; +bool sorted_by_name; +bool loaded; +u8 rel;char name[]; };
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 8/25 perf report: Sort child tasks by tid
In reply to Ian RogersCommit 91e467bc568f ("perf machine: Use hashtable for machine threads") made the iteration of thread tids unordered. The perf report --tasks output now shows child threads in an order determined by the hashing. For example, in this snippet tid 3 appears after tid 256 even though they have the same ppid 2:
```
$ perf report --tasks
% pid tid ppid comm
0 0 -1 |swapper
2 2 0 | kthreadd
256 256 2 | kworker/12:1H-k
693761 693761 2 | kworker/10:1-mm
1301762 1301762 2 | kworker/1:1-mm_
1302530 1302530 2 | kworker/u32:0-k
3 3 2 | rcu_gp
...
```The output is easier to read if threads appear numerically increasing. To allow for this, read all threads into a list then sort with a comparator that orders by the child task's of the first common parent. The list creation and deletion are created as utilities on machine. The indentation is possible by counting the number of parents a child has.
With this change the output for the same data file is now like:
```
$ perf report --tasks
% pid tid ppid comm
0 0 -1 |swapper
1 1 0 | systemd
823 823 1 | systemd-journal
853 853 1 | systemd-udevd
3230 3230 1 | systemd-timesyn
3236 3236 1 | auditd
3239 3239 3236 | audisp-syslog
3321 3321 1 | accounts-daemon
...
```--- tools/perf/builtin-report.c | 203 ++++++++++++++++++++---------------- tools/perf/util/machine.c | 30 ++++++ tools/perf/util/machine.h | 10 ++ 3 files changed, 155 insertions(+), 88 deletions(-)
tools/perf/builtin-report.c +115 -88
@@ -59,6 +59,7 @@#include <linux/ctype.h> #include <signal.h> #include <linux/bitmap.h> +#include <linux/list_sort.h> #include <linux/string.h> #include <linux/stringify.h> #include <linux/time64.h>
@@ -828,35 +829,6 @@ static void tasks_setup(struct report *rep) rep->tool.no_warn = true; } -struct task { -struct thread *thread; -struct list_head list; -struct list_head children; -}; - -static struct task *tasks_list (struct task *task,struct machine *machine) -{ -struct thread *parent_thread, *thread = task->thread; -struct task *parent_task; - -/* Already listed. */ -if (!list_empty (&task->list)) -return NULL ; - -/* Last one in the chain. */ -if (thread__ppid (thread) ==-1 ) -return task; - - parent_thread =machine__find_thread (machine,-1 ,thread__ppid (thread)); -if (!parent_thread) -return ERR_PTR (-ENOENT ); - - parent_task =thread__priv (parent_thread); -thread__put (parent_thread); -list_add_tail (&task->list, &parent_task->children); -return tasks_list (parent_task, machine); -} -struct maps__fprintf_task_args {int indent;FILE *fp;
@@ -900,89 +872,144 @@ static size_t maps__fprintf_task(struct maps *maps, int indent, FILE *fp)return args.printed; } -static void task__print_level (struct task *task,FILE *fp,int level) +static int thread_level (struct machine *machine,const struct thread *thread) { -struct thread *thread = task->thread; -struct task *child; -int comm_indent =fprintf (fp," %8d %8d %8d |%*s" , -thread__pid (thread),thread__tid (thread), -thread__ppid (thread), level,"" ); +struct thread *parent_thread; +int res; -fprintf (fp,"%s\n" ,thread__comm_str (thread)); +if (thread__tid (thread) <=0 ) +return 0 ; -maps__fprintf_task (thread__maps (thread), comm_indent, fp); +if (thread__ppid (thread) <=0 ) +return 1 ; -if (!list_empty (&task->children)) { -list_for_each_entry (child, &task->children, list) -task__print_level (child, fp, level +1 ); + parent_thread =machine__find_thread (machine,-1 ,thread__ppid (thread)); +if (!parent_thread) { +pr_err ("Missing parent thread of %d\n" ,thread__tid (thread)); +return 0 ; } + res =1 +thread_level (machine, parent_thread); +thread__put (parent_thread); +return res; } -static int tasks_print (struct report *rep,FILE *fp) +static void task__print_level (struct machine *machine,struct thread *thread,FILE *fp) { -struct perf_session *session = rep->session; -struct machine *machine = &session->machines.host; -struct task *tasks, *task; -unsigned int nr =0 , itask =0 , i; -struct rb_node *nd; -LIST_HEAD (list); +int level =thread_level (machine, thread); +int comm_indent =fprintf (fp," %8d %8d %8d |%*s" , +thread__pid (thread),thread__tid (thread), +thread__ppid (thread), level,"" ); -/* -* No locking needed while accessing machine->threads, -* because --tasks is single threaded command. -*/ +fprintf (fp,"%s\n" ,thread__comm_str (thread)); -/* Count all the threads. */ -for (i =0 ; i <THREADS__TABLE_SIZE ; i++) - nr += machine->threads[i].nr; +maps__fprintf_task (thread__maps (thread), comm_indent, fp); +} - tasks =malloc (sizeof (*tasks) * nr); -if (!tasks) -return -ENOMEM ; +static int task_list_cmp (void *priv,const struct list_head *la,const struct list_head *lb) +{ +struct machine *machine = priv; + struct thread_list *task_a = list_entry(la, struct thread_list, list); + struct thread_list *task_b = list_entry(lb, struct thread_list, list); +struct thread *a = task_a->thread; +struct thread *b = task_b->thread; +int level_a, level_b, res; + +/* Compare a and b to root. */ +if (thread__tid (a) ==thread__tid (b)) +return 0 ; -for (i =0 ; i <THREADS__TABLE_SIZE ; i++) { -struct threads *threads = &machine->threads[i]; +if (thread__tid (a) ==0 ) +return -1 ; -for (nd =rb_first_cached (&threads->entries); nd; - nd =rb_next (nd)) { - task = tasks + itask++; +if (thread__tid (b) ==0 ) +return 1 ; - task->thread = rb_entry(nd, struct thread_rb_node, rb_node)->thread; -INIT_LIST_HEAD (&task->children); -INIT_LIST_HEAD (&task->list); -thread__set_priv (task->thread, task); - } +/* If parents match sort by tid. */ +if (thread__ppid (a) ==thread__ppid (b)) { +return thread__tid (a) <thread__tid (b) + ?-1 + : (thread__tid (a) >thread__tid (b) ?1 :0 ); }/* -* Iterate every task down to the unprocessed parent -* and link all in task children list. Task with no -* parent is added into 'list'. +* Find a and b such that if they are a child of each other a and b's +* tid's match, otherwise a and b have a common parent and distinct +* tid's to sort by. First make the depths of the threads match. */ -for (itask =0 ; itask < nr; itask++) { - task = tasks + itask; - -if (!list_empty (&task->list)) -continue ; - - task =tasks_list (task, machine); -if (IS_ERR (task)) { -pr_err ("Error: failed to process tasks\n" ); -free (tasks); -return PTR_ERR (task); + level_a =thread_level (machine, a); + level_b =thread_level (machine, b); + a =thread__get (a); + b =thread__get (b); +for (int i = level_a; i > level_b; i--) { +struct thread *parent =machine__find_thread (machine,-1 ,thread__ppid (a)); + +thread__put (a); +if (!parent) { +pr_err ("Missing parent thread of %d\n" ,thread__tid (a)); +thread__put (b); +return -1 ; } + a = parent; + } +for (int i = level_b; i > level_a; i--) { +struct thread *parent =machine__find_thread (machine,-1 ,thread__ppid (b)); -if (task) -list_add_tail (&task->list, &list); +thread__put (b); +if (!parent) { +pr_err ("Missing parent thread of %d\n" ,thread__tid (b)); +thread__put (a); +return 1 ; + } + b = parent; + } +/* Search up to a common parent. */ +while (thread__ppid (a) !=thread__ppid (b)) { +struct thread *parent; + + parent =machine__find_thread (machine,-1 ,thread__ppid (a)); +thread__put (a); +if (!parent) +pr_err ("Missing parent thread of %d\n" ,thread__tid (a)); + a = parent; + parent =machine__find_thread (machine,-1 ,thread__ppid (b)); +thread__put (b); +if (!parent) +pr_err ("Missing parent thread of %d\n" ,thread__tid (b)); + b = parent; +if (!a || !b) +return !a && !b ?0 : (!a ?-1 :1 ); + } +if (thread__tid (a) ==thread__tid (b)) { +/* a is a child of b or vice-versa, deeper levels appear later. */ + res = level_a < level_b ?-1 : (level_a > level_b ?1 :0 ); + }else { +/* Sort by tid now the parent is the same. */ + res =thread__tid (a) <thread__tid (b) ?-1 :1 ; } +thread__put (a); +thread__put (b); +return res; +} + +static int tasks_print (struct report *rep,FILE *fp) +{ +struct machine *machine = &rep->session->machines.host; +LIST_HEAD (tasks); +int ret; -fprintf (fp,"# %8s %8s %8s %s\n" ,"pid" ,"tid" ,"ppid" ,"comm" ); + ret =machine__thread_list (machine, &tasks); +if (!ret) { +struct thread_list *task; -list_for_each_entry (task, &list, list) -task__print_level (task, fp,0 ); +list_sort (machine, &tasks, task_list_cmp); -free (tasks); -return 0 ; +fprintf (fp,"# %8s %8s %8s %s\n" ,"pid" ,"tid" ,"ppid" ,"comm" ); + +list_for_each_entry (task, &tasks, list) +task__print_level (machine, task->thread, fp); + } +thread_list__delete (&tasks); +return ret; }static int __cmd_report (struct report *rep)
tools/perf/util/machine.c +30 -0
@@ -3261,6 +3261,36 @@ int machines__for_each_thread(struct machines *machines,return rc; } + +static int thread_list_cb (struct thread *thread,void *data) +{ +struct list_head *list = data; +struct thread_list *entry =malloc (sizeof (*entry)); + +if (!entry) +return -ENOMEM ; + + entry->thread =thread__get (thread); +list_add_tail (&entry->list, list); +return 0 ; +} + +int machine__thread_list (struct machine *machine,struct list_head *list) +{ +return machine__for_each_thread (machine, thread_list_cb, list); +} + +void thread_list__delete (struct list_head *list) +{ +struct thread_list *pos, *next; + +list_for_each_entry_safe (pos, next, list, list) { +thread__zput (pos->thread); +list_del (&pos->list); +free (pos); + } +} +pid_t machine__get_current_tid (struct machine *machine,int cpu) {if (cpu <0 || (size_t )cpu >= machine->current_tid_sz)
tools/perf/util/machine.h +10 -0
@@ -280,6 +280,16 @@ int machines__for_each_thread(struct machines *machines,int (*fn)(struct thread *thread,void *p), void *priv); +struct thread_list { +struct list_head list; +struct thread *thread; +}; + +/* Make a list of struct thread_list based on threads in the machine. */ +int machine__thread_list (struct machine *machine,struct list_head *list); +/* Free up the nodes within the thread_list list. */ +void thread_list__delete (struct list_head *list); +pid_t machine__get_current_tid (struct machine *machine,int cpu);int machine__set_current_tid (struct machine *machine,int cpu,pid_t pid,pid_t tid);
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 9/25 perf trace: Ignore thread hashing in summary
In reply to Ian RogersCommit 91e467bc568f ("perf machine: Use hashtable for machine threads") made the iteration of thread tids unordered. The perf trace --summary output sorts and prints each hash bucket, rather than all threads globally. Change this behavior by turn all threads into a list, sort the list by number of trace events then by tids, finally print the list. This also allows the rbtree in threads to be not accessed outside of machine.
--- tools/perf/builtin-trace.c | 41 +++++++++++++++++++++---------------- tools/perf/util/rb_resort.h | 5 ----- 2 files changed, 23 insertions(+), 23 deletions(-)
tools/perf/builtin-trace.c +23 -18
@@ -74,6 +74,7 @@#include <linux/err.h> #include <linux/filter.h> #include <linux/kernel.h> +#include <linux/list_sort.h> #include <linux/random.h> #include <linux/stringify.h> #include <linux/time64.h>
@@ -4312,34 +4313,38 @@ static unsigned long thread__nr_events(struct thread_trace *ttrace)return ttrace ? ttrace->nr_events :0 ; } -DEFINE_RESORT_RB (threads, - (thread__nr_events (thread__priv (a->thread)) < -thread__nr_events (thread__priv (b->thread))), - struct thread *thread; -) +static int trace_nr_events_cmp (void *priv__maybe_unused , +const struct list_head *la, +const struct list_head *lb) { - entry->thread = rb_entry(nd, struct thread_rb_node, rb_node)->thread; + struct thread_list *a = list_entry(la, struct thread_list, list); + struct thread_list *b = list_entry(lb, struct thread_list, list); +unsigned long a_nr_events =thread__nr_events (thread__priv (a->thread)); +unsigned long b_nr_events =thread__nr_events (thread__priv (b->thread)); + +if (a_nr_events != b_nr_events) +return a_nr_events < b_nr_events ?-1 :1 ; + +/* Identical number of threads, place smaller tids first. */ +return thread__tid (a->thread) <thread__tid (b->thread) + ?-1 + : (thread__tid (a->thread) >thread__tid (b->thread) ?1 :0 ); }static size_t trace__fprintf_thread_summary (struct trace *trace,FILE *fp) {size_t printed =trace__fprintf_threads_header (fp); -struct rb_node *nd; -int i; - -for (i =0 ; i <THREADS__TABLE_SIZE ; i++) { -DECLARE_RESORT_RB_MACHINE_THREADS (threads, trace->host, i); +LIST_HEAD (threads); -if (threads ==NULL ) { -fprintf (fp,"%s" ,"Error sorting output by nr_events!\n" ); -return 0 ; - } +if (machine__thread_list (trace->host, &threads) ==0 ) { +struct thread_list *pos; -resort_rb__for_each_entry (nd, threads) - printed +=trace__fprintf_thread (fp, threads_entry->thread, trace); +list_sort (NULL , &threads, trace_nr_events_cmp); -resort_rb__delete (threads); +list_for_each_entry (pos, &threads, list) + printed +=trace__fprintf_thread (fp, pos->thread, trace); } +thread_list__delete (&threads);return printed; }
tools/perf/util/rb_resort.h +0 -5
@@ -143,9 +143,4 @@ struct __name##_sorted *__name = __name##_sorted__newDECLARE_RESORT_RB (__name)(&__ilist->rblist.entries.rb_root, \ __ilist->rblist.nr_entries) -/* For 'struct machine->threads' */ -#define DECLARE_RESORT_RB_MACHINE_THREADS (__name, __machine, hash_bucket) \ - DECLARE_RESORT_RB(__name)(&__machine->threads[hash_bucket].entries.rb_root, \ - __machine->threads[hash_bucket].nr) -#endif /* _PERF_RESORT_RB_H_ */
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 10/25 perf machine: Move fprintf to for_each loop and a callback
In reply to Ian RogersAvoid exposing the threads data structure by switching to the callback machine__for_each_thread approach. machine__fprintf is only used in tests and verbose >3 output so don't turn to list and sort. Add machine__threads_nr to be refactored later.
Note, all existing *_fprintf routines ignore fprintf errors.
--- tools/perf/util/machine.c | 43 ++++++++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 16 deletions(-)
tools/perf/util/machine.c +27 -16
@@ -1113,29 +1113,40 @@ size_t machine__fprintf_vmlinux_path(struct machine *machine, FILE *fp)return printed; } -size_t machine__fprintf (struct machine *machine,FILE *fp) +struct machine_fprintf_cb_args { +FILE *fp; +size_t printed; +}; + +static int machine_fprintf_cb (struct thread *thread,void *data) { -struct rb_node *nd; -size_t ret; -int i; +struct machine_fprintf_cb_args *args = data; -for (i =0 ; i <THREADS__TABLE_SIZE ; i++) { -struct threads *threads = &machine->threads[i]; +/* TODO: handle fprintf errors. */ + args->printed +=thread__fprintf (thread, args->fp); +return 0 ; +} -down_read (&threads->lock); +static size_t machine__threads_nr (const struct machine *machine) +{ +size_t nr =0 ; - ret =fprintf (fp,"Threads: %u\n" , threads->nr); +for (int i =0 ; i <THREADS__TABLE_SIZE ; i++) + nr += machine->threads[i].nr; -for (nd =rb_first_cached (&threads->entries); nd; - nd =rb_next (nd)) { - struct thread *pos = rb_entry(nd, struct thread_rb_node, rb_node)->thread; +return nr; +} - ret +=thread__fprintf (pos, fp); - } +size_t machine__fprintf (struct machine *machine,FILE *fp) +{ +struct machine_fprintf_cb_args args = { + .fp = fp, + .printed =0 , + }; +size_t ret =fprintf (fp,"Threads: %zu\n" ,machine__threads_nr (machine)); -up_read (&threads->lock); - } -return ret; +machine__for_each_thread (machine, machine_fprintf_cb, &args); +return ret + args.printed; }static struct dso *machine__get_kernel (struct machine *machine)
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 11/25 perf threads: Move threads to its own files
In reply to Ian RogersMove threads out of machine and move thread_rb_node into the C file. This hides the implementation of threads from the rest of the code allowing for it to be refactored.
Locking discipline is tightened up in this change.
--- tools/perf/util/Build | 1 + tools/perf/util/bpf_lock_contention.c | 8 +- tools/perf/util/machine.c | 287 ++++---------------------- tools/perf/util/machine.h | 20 +- tools/perf/util/thread.c | 2 +- tools/perf/util/thread.h | 6 - tools/perf/util/threads.c | 244 ++++++++++++++++++++++ tools/perf/util/threads.h | 35 ++++ 8 files changed, 325 insertions(+), 278 deletions(-) create mode 100644 tools/perf/util/threads.c create mode 100644 tools/perf/util/threads.h
tools/perf/util/Build +1 -0
@@ -71,6 +71,7 @@ perf-y += ordered-events.o perf-y += namespaces.o perf-y += comm.o perf-y += thread.o +perf-y += threads.o perf-y += thread_map.o perf-y += parse-events-flex.o perf-y += parse-events-bison.o
tools/perf/util/bpf_lock_contention.c +4 -4
@@ -210,7 +210,7 @@ static const char *lock_contention_get_name(struct lock_contention *con,/* do not update idle comm which contains CPU number */ if (pid) { -struct thread *t =__machine__findnew_thread (machine,/*pid=*/ -1 , pid); +struct thread *t =machine__findnew_thread (machine,/*pid=*/ -1 , pid);if (t ==NULL )return name;
@@ -302,9 +302,9 @@ int lock_contention_read(struct lock_contention *con)return -1 ;if (con->aggr_mode ==LOCK_AGGR_TASK ) { -struct thread *idle =__machine__findnew_thread (machine, -/*pid=*/ 0 , -/*tid=*/ 0 ); +struct thread *idle =machine__findnew_thread (machine, +/*pid=*/ 0 , +/*tid=*/ 0 );thread__set_comm (idle,"swapper" ,/*timestamp=*/ 0 ); }
tools/perf/util/machine.c +38 -249
377 lines not on this page · raw message
tools/perf/util/machine.h +2 -18
@@ -7,6 +7,7 @@#include "maps.h" #include "dsos.h" #include "rwsem.h" +#include "threads.h" struct addr_location ;struct branch_stack ;
@@ -28,16 +29,6 @@ extern const char *ref_reloc_sym_names[];struct vdso_info ; -#define THREADS__TABLE_BITS 8 -#define THREADS__TABLE_SIZE (1 << THREADS__TABLE_BITS) - -struct threads { -struct rb_root_cached entries; -struct rw_semaphore lock; -unsigned int nr; -struct thread *last_match; -}; -struct machine {struct rb_node rb_node;pid_t pid;
@@ -48,7 +39,7 @@ struct machine {char *root_dir;char *mmap_name;char *kallsyms_filename; -struct threads threads[THREADS__TABLE_SIZE ]; +struct threads threads;struct vdso_info *vdso_info;struct perf_env *env;struct dsos dsos;
@@ -69,12 +60,6 @@ struct machine {bool trampolines_mapped; }; -static inline struct threads *machine__threads (struct machine *machine,pid_t tid) -{ -/* Cast it to handle tid == -1 */ -return &machine->threads[(unsigned int )tid %THREADS__TABLE_SIZE ]; -} -/* * The main kernel (vmlinux) map */
@@ -220,7 +205,6 @@ bool machine__is(struct machine *machine, const char *arch);bool machine__normalized_is (struct machine *machine,const char *arch);int machine__nr_cpus_avail (struct machine *machine); -struct thread *__machine__findnew_thread (struct machine *machine,pid_t pid,pid_t tid);struct thread *machine__findnew_thread (struct machine *machine,pid_t pid,pid_t tid);struct dso *machine__findnew_dso_id (struct machine *machine,const char *filename,struct dso_id *id);
tools/perf/util/thread.c +1 -1
@@ -26,7 +26,7 @@ int thread__init_maps(struct thread *thread, struct machine *machine)if (pid ==thread__tid (thread) || pid ==-1 ) {thread__set_maps (thread,maps__new (machine)); }else { -struct thread *leader =__machine__findnew_thread (machine, pid, pid); +struct thread *leader =machine__findnew_thread (machine, pid, pid);if (leader) {thread__set_maps (thread,maps__get (thread__maps (leader)));
tools/perf/util/thread.h +0 -6
@@ -3,7 +3,6 @@#define __PERF_THREAD_H#include <linux/refcount.h> -#include <linux/rbtree.h> #include <linux/list.h> #include <stdio.h> #include <unistd.h>
@@ -30,11 +29,6 @@ struct lbr_stitch {struct callchain_cursor_node *prev_lbr_cursor; }; -struct thread_rb_node { -struct rb_node rb_node; -struct thread *thread; -}; -DECLARE_RC_STRUCT (thread) {/** @maps: mmaps associated with this thread. */ struct maps *maps;
tools/perf/util/threads.c new file +244 -0
244 lines not on this page · raw message
tools/perf/util/threads.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __PERF_THREADS_H +#define __PERF_THREADS_H + +#include <linux/rbtree.h> +#include "rwsem.h" + +struct thread ; + +#define THREADS__TABLE_BITS 8 +#define THREADS__TABLE_SIZE (1 << THREADS__TABLE_BITS) + +struct threads_table_entry { +struct rb_root_cached entries; +struct rw_semaphore lock; +unsigned int nr; +struct thread *last_match; +}; + +struct threads { +struct threads_table_entry table[THREADS__TABLE_SIZE ]; +}; + +void threads__init (struct threads *threads); +void threads__exit (struct threads *threads); +size_t threads__nr (struct threads *threads); +struct thread *threads__find (struct threads *threads,pid_t tid); +struct thread *threads__findnew (struct threads *threads,pid_t pid,pid_t tid,bool *created); +void threads__remove_all_threads (struct threads *threads); +void threads__remove (struct threads *threads,struct thread *thread); +int threads__for_each_thread (struct threads *threads, +int (*fn)(struct thread *thread,void *data), +void *data); + +#endif /* __PERF_THREADS_H */
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 12/25 perf threads: Switch from rbtree to hashmap
In reply to Ian RogersThe rbtree provides a sorting on entries but this is unused. Switch to using hashmap for O(1) rather than O(log n) find/insert/remove complexity.
--- tools/perf/util/threads.c | 146 ++++++++++++-------------------------- tools/perf/util/threads.h | 6 +- 2 files changed, 47 insertions(+), 105 deletions(-)
tools/perf/util/threads.c +44 -102
@@ -3,25 +3,30 @@#include "machine.h" #include "thread.h" -struct thread_rb_node { -struct rb_node rb_node; -struct thread *thread; -}; -static struct threads_table_entry *threads__table (struct threads *threads,pid_t tid) {/* Cast it to handle tid == -1 */ return &threads->table[(unsigned int )tid %THREADS__TABLE_SIZE ]; } +static size_t key_hash (long key,void *ctx__maybe_unused ) +{ +/* The table lookup removes low bit entropy, but this is just ignored here. */ +return key; +} + +static bool key_equal (long key1,long key2,void *ctx__maybe_unused ) +{ +return key1 == key2; +} +void threads__init (struct threads *threads) {for (int i =0 ; i <THREADS__TABLE_SIZE ; i++) {struct threads_table_entry *table = &threads->table[i]; - table->entries =RB_ROOT_CACHED ; +hashmap__init (&table->shard, key_hash, key_equal,NULL );init_rwsem (&table->lock); - table->nr =0 ; table->last_match =NULL ; } }
@@ -32,6 +37,7 @@ void threads__exit(struct threads *threads)for (int i =0 ; i <THREADS__TABLE_SIZE ; i++) {struct threads_table_entry *table = &threads->table[i]; +hashmap__clear (&table->shard);exit_rwsem (&table->lock); } }
@@ -44,7 +50,7 @@ size_t threads__nr(struct threads *threads)struct threads_table_entry *table = &threads->table[i];down_read (&table->lock); - nr += table->nr; + nr +=hashmap__size (&table->shard);up_read (&table->lock); }return nr;
@@ -86,28 +92,13 @@ static void threads_table_entry__set_last_match(struct threads_table_entry *tablstruct thread *threads__find (struct threads *threads,pid_t tid) {struct threads_table_entry *table =threads__table (threads, tid); -struct rb_node **p; -struct thread *res =NULL ; +struct thread *res;down_read (&table->lock); res =__threads_table_entry__get_last_match (table, tid); -if (res) -return res; - - p = &table->entries.rb_root.rb_node; -while (*p !=NULL ) { -struct rb_node *parent = *p; - struct thread *th = rb_entry(parent, struct thread_rb_node, rb_node)->thread; - -if (thread__tid (th) == tid) { - res =thread__get (th); -break ; - } - -if (tid <thread__tid (th)) - p = &(*p)->rb_left; -else - p = &(*p)->rb_right; +if (!res) { +if (hashmap__find (&table->shard, tid, &res)) + res =thread__get (res); }up_read (&table->lock);if (res)
@@ -118,49 +109,25 @@ struct thread *threads__find(struct threads *threads, pid_t tid)struct thread *threads__findnew (struct threads *threads,pid_t pid,pid_t tid,bool *created) {struct threads_table_entry *table =threads__table (threads, tid); -struct rb_node **p; -struct rb_node *parent =NULL ;struct thread *res =NULL ; -struct thread_rb_node *nd; -bool leftmost = true; *created = false;down_write (&table->lock); - p = &table->entries.rb_root.rb_node; -while (*p !=NULL ) { -struct thread *th; - - parent = *p; - th = rb_entry(parent, struct thread_rb_node, rb_node)->thread; - -if (thread__tid (th) == tid) { -__threads_table_entry__set_last_match (table, th); - res =thread__get (th); - goto out_unlock; - } - -if (tid <thread__tid (th)) - p = &(*p)->rb_left; -else { - leftmost = false; - p = &(*p)->rb_right; - } - } - nd =malloc (sizeof (*nd)); -if (nd ==NULL ) - goto out_unlock; res =thread__new (pid, tid); -if (!res) -free (nd); -else { - *created = true; - nd->thread =thread__get (res); -rb_link_node (&nd->rb_node, parent, p); -rb_insert_color_cached (&nd->rb_node, &table->entries, leftmost); - ++table->nr; -__threads_table_entry__set_last_match (table, res); +if (res) { +if (hashmap__add (&table->shard, tid, res)) { +/* Add failed. Assume a race so find other entry. */ +thread__put (res); + res =NULL ; +if (hashmap__find (&table->shard, tid, &res)) + res =thread__get (res); + }else { + res =thread__get (res); + *created = true; + } +if (res) +__threads_table_entry__set_last_match (table, res); } -out_unlock:up_write (&table->lock);return res; }
@@ -169,57 +136,32 @@ void threads__remove_all_threads(struct threads *threads) {for (int i =0 ; i <THREADS__TABLE_SIZE ; i++) {struct threads_table_entry *table = &threads->table[i]; -struct rb_node *nd; +struct hashmap_entry *cur, *tmp; +size_t bkt;down_write (&table->lock);__threads_table_entry__set_last_match (table,NULL ); - nd =rb_first_cached (&table->entries); -while (nd) { - struct thread_rb_node *trb = rb_entry(nd, struct thread_rb_node, rb_node); - - nd =rb_next (nd); -thread__put (trb->thread); -rb_erase_cached (&trb->rb_node, &table->entries); -RB_CLEAR_NODE (&trb->rb_node); - --table->nr; +hashmap__for_each_entry_safe ((&table->shard), cur, tmp, bkt) { +struct thread *old_value; -free (trb); +hashmap__delete (&table->shard, cur->key,/*old_key=*/ NULL , &old_value); +thread__put (old_value); } -assert (table->nr ==0 );up_write (&table->lock); } }void threads__remove (struct threads *threads,struct thread *thread) { -struct rb_node **p;struct threads_table_entry *table =threads__table (threads,thread__tid (thread)); -pid_t tid =thread__tid (thread); +struct thread *old_value;down_write (&table->lock);if (table->last_match &&RC_CHK_EQUAL (table->last_match, thread))__threads_table_entry__set_last_match (table,NULL ); - p = &table->entries.rb_root.rb_node; -while (*p !=NULL ) { -struct rb_node *parent = *p; - struct thread_rb_node *nd = rb_entry(parent, struct thread_rb_node, rb_node); -struct thread *th = nd->thread; - -if (RC_CHK_EQUAL (th, thread)) { -thread__put (nd->thread); -rb_erase_cached (&nd->rb_node, &table->entries); -RB_CLEAR_NODE (&nd->rb_node); - --table->nr; -free (nd); -break ; - } - -if (tid <thread__tid (th)) - p = &(*p)->rb_left; -else - p = &(*p)->rb_right; - } +hashmap__delete (&table->shard,thread__tid (thread),/*old_key=*/ NULL , &old_value); +thread__put (old_value);up_write (&table->lock); }
@@ -229,11 +171,11 @@ int threads__for_each_thread(struct threads *threads, {for (int i =0 ; i <THREADS__TABLE_SIZE ; i++) {struct threads_table_entry *table = &threads->table[i]; -struct rb_node *nd; +struct hashmap_entry *cur; +size_t bkt; -for (nd =rb_first_cached (&table->entries); nd; nd =rb_next (nd)) { - struct thread_rb_node *trb = rb_entry(nd, struct thread_rb_node, rb_node); -int rc =fn (trb->thread, data); +hashmap__for_each_entry ((&table->shard), cur, bkt) { +int rc =fn ((struct thread *)cur->pvalue, data);if (rc !=0 )return rc;
tools/perf/util/threads.h +3 -3
@@ -2,7 +2,7 @@#ifndef __PERF_THREADS_H#define __PERF_THREADS_H -#include <linux/rbtree.h> +#include "hashmap.h" #include "rwsem.h" struct thread ;
@@ -11,9 +11,9 @@ struct thread;#define THREADS__TABLE_SIZE (1 << THREADS__TABLE_BITS)struct threads_table_entry { -struct rb_root_cached entries; +/* Key is tid, value is struct thread. */ +struct hashmap shard;struct rw_semaphore lock; -unsigned int nr;struct thread *last_match; };
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 13/25 perf threads: Reduce table size from 256 to 8
In reply to Ian RogersThe threads data structure is an array of hashmaps, previously rbtrees. The two levels allows for a fixed outer array where access is guarded by rw_semaphores. Commit 91e467bc568f ("perf machine: Use hashtable for machine threads") sized the outer table at 256 entries to avoid future scalability problems, however, this means the threads struct is sized at 30,720 bytes. As the hashmaps allow O(1) access for the common find/insert/remove operations, lower the number of entries to 8. This reduces the size overhead to 960 bytes.
--- tools/perf/util/threads.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/threads.h +1 -1
@@ -7,7 +7,7 @@struct thread ; -#define THREADS__TABLE_BITS 8 +#define THREADS__TABLE_BITS 3#define THREADS__TABLE_SIZE (1 << THREADS__TABLE_BITS)struct threads_table_entry {
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 14/25 perf dsos: Attempt to better abstract dsos internals
In reply to Ian RogersMove functions from machine and build-id to dsos. Pass dsos struct rather than internal state. Rename some functions to better represent which data structure they operate on.
--- tools/perf/builtin-inject.c | 2 +- tools/perf/builtin-record.c | 2 +- tools/perf/util/build-id.c | 38 +--------------------------- tools/perf/util/build-id.h | 2 -- tools/perf/util/dso.h | 6 ----- tools/perf/util/dsos.c | 49 ++++++++++++++++++++++++++++++++++--- tools/perf/util/dsos.h | 19 +++++++++++--- tools/perf/util/machine.c | 40 ++++++------------------------ tools/perf/util/machine.h | 2 ++ tools/perf/util/session.c | 21 ++++++++++++++++ tools/perf/util/session.h | 2 ++ 11 files changed, 97 insertions(+), 86 deletions(-)
tools/perf/builtin-inject.c +1 -1
@@ -2122,7 +2122,7 @@ static int __cmd_inject(struct perf_inject *inject)*/ if (perf_header__has_feat (&session->header,HEADER_BUILD_ID ) && inject->have_auxtrace && !inject->itrace_synth_opts.set) -dsos__hit_all (session); +perf_session__dsos_hit_all (session);/* * The AUX areas have been removed and replaced with * synthesized hardware events, so clear the feature flag.
tools/perf/builtin-record.c +1 -1
@@ -1787,7 +1787,7 @@ record__finish_output(struct record *rec)process_buildids (rec);if (rec->buildid_all) -dsos__hit_all (rec->session); +perf_session__dsos_hit_all (rec->session); }perf_session__write_header (rec->session, rec->evlist, fd, true);
tools/perf/util/build-id.c +1 -37
@@ -390,42 +390,6 @@ int perf_session__write_buildid_table(struct perf_session *session,return err; } -static int __dsos__hit_all (struct list_head *head) -{ -struct dso *pos; - -list_for_each_entry (pos, head, node) - pos->hit = true; - -return 0 ; -} - -static int machine__hit_all_dsos (struct machine *machine) -{ -return __dsos__hit_all (&machine->dsos.head); -} - -int dsos__hit_all (struct perf_session *session) -{ -struct rb_node *nd; -int err; - - err =machine__hit_all_dsos (&session->machines.host); -if (err) -return err; - -for (nd =rb_first_cached (&session->machines.guests); nd; - nd =rb_next (nd)) { - struct machine *pos = rb_entry(nd, struct machine, rb_node); - - err =machine__hit_all_dsos (pos); -if (err) -return err; - } - -return 0 ; -} -void disable_buildid_cache (void ) { no_buildid_cache = true;
@@ -992,7 +956,7 @@ int perf_session__cache_build_ids(struct perf_session *session)static bool machine__read_build_ids (struct machine *machine,bool with_hits) { -return __dsos__read_build_ids (&machine->dsos.head, with_hits); +return __dsos__read_build_ids (&machine->dsos, with_hits); }bool perf_session__read_build_ids (struct perf_session *session,bool with_hits)
tools/perf/util/build-id.h +0 -2
@@ -39,8 +39,6 @@ int build_id__mark_dso_hit(struct perf_tool *tool, union perf_event *event, struct perf_sample *sample, struct evsel *evsel, struct machine *machine); -int dsos__hit_all (struct perf_session *session); -int perf_event__inject_buildid (struct perf_tool *tool,union perf_event *event,struct perf_sample *sample,struct evsel *evsel,struct machine *machine);
tools/perf/util/dso.h +0 -6
@@ -232,12 +232,6 @@ struct dso {#define dso__for_each_symbol (dso, pos, n) \ symbols__for_each_entry(&(dso)->symbols, pos, n) -#define dsos__for_each_with_build_id (pos, head) \ - list_for_each_entry(pos, head, node) \ - if (!pos->has_build_id) \ - continue; \ - else -static inline void dso__set_loaded (struct dso *dso) { dso->loaded = true;
tools/perf/util/dsos.c +46 -3
80 lines not on this page · raw message
tools/perf/util/dsos.h +15 -4
@@ -21,6 +21,15 @@ struct dsos {struct rw_semaphore lock; }; +#define dsos__for_each_with_build_id (pos, head) \ + list_for_each_entry(pos, head, node) \ + if (!pos->has_build_id) \ + continue; \ + else + +void dsos__init (struct dsos *dsos); +void dsos__exit (struct dsos *dsos); +void __dsos__add (struct dsos *dsos,struct dso *dso);void dsos__add (struct dsos *dsos,struct dso *dso);struct dso *__dsos__addnew (struct dsos *dsos,const char *name);
@@ -28,13 +37,15 @@ struct dso *__dsos__find(struct dsos *dsos, const char *name, bool cmp_short);struct dso *dsos__findnew_id (struct dsos *dsos,const char *name,struct dso_id *id); +bool __dsos__read_build_ids (struct dsos *dsos,bool with_hits); +struct dso *__dsos__findnew_link_by_longname_id (struct rb_root *root,struct dso *dso,const char *name,struct dso_id *id); -bool __dsos__read_build_ids (struct list_head *head,bool with_hits); - -size_t __dsos__fprintf_buildid (struct list_head *head,FILE *fp, +size_t __dsos__fprintf_buildid (struct dsos *dsos,FILE *fp,bool (skip )(struct dso *dso,int parm),int parm); -size_t __dsos__fprintf (struct list_head *head,FILE *fp); +size_t __dsos__fprintf (struct dsos *dsos,FILE *fp); + +int __dsos__hit_all (struct dsos *dsos);#endif /* __PERF_DSOS */
tools/perf/util/machine.c +8 -32
@@ -48,13 +48,6 @@ static struct dso *machine__kernel_dso(struct machine *machine)return map__dso (machine->vmlinux_map); } -static void dsos__init (struct dsos *dsos) -{ -INIT_LIST_HEAD (&dsos->head); - dsos->root =RB_ROOT ; -init_rwsem (&dsos->lock); -} -static int machine__set_mmap_name (struct machine *machine) {if (machine__is_host (machine))
@@ -165,28 +158,6 @@ struct machine *machine__new_kallsyms(void)return machine; } -static void dsos__purge (struct dsos *dsos) -{ -struct dso *pos, *n; - -down_write (&dsos->lock); - -list_for_each_entry_safe (pos, n, &dsos->head, node) { -RB_CLEAR_NODE (&pos->rb_node); - pos->root =NULL ; -list_del_init (&pos->node); -dso__put (pos); - } - -up_write (&dsos->lock); -} - -static void dsos__exit (struct dsos *dsos) -{ -dsos__purge (dsos); -exit_rwsem (&dsos->lock); -} -void machine__delete_threads (struct machine *machine) {threads__remove_all_threads (&machine->threads);
@@ -906,11 +877,11 @@ static struct map *machine__addnew_module_map(struct machine *machine, u64 startsize_t machines__fprintf_dsos (struct machines *machines,FILE *fp) {struct rb_node *nd; -size_t ret =__dsos__fprintf (&machines->host.dsos.head, fp); +size_t ret =__dsos__fprintf (&machines->host.dsos, fp);for (nd =rb_first_cached (&machines->guests); nd; nd =rb_next (nd)) { struct machine *pos = rb_entry(nd, struct machine, rb_node); - ret +=__dsos__fprintf (&pos->dsos.head, fp); + ret +=__dsos__fprintf (&pos->dsos, fp); }return ret;
@@ -919,7 +890,7 @@ size_t machines__fprintf_dsos(struct machines *machines, FILE *fp)size_t machine__fprintf_dsos_buildid (struct machine *m,FILE *fp,bool (skip )(struct dso *dso,int parm),int parm) { -return __dsos__fprintf_buildid (&m->dsos.head, fp, skip, parm); +return __dsos__fprintf_buildid (&m->dsos, fp, skip, parm); } size_t machines__fprintf_dsos_buildid(struct machines *machines, FILE *fp,
@@ -3281,3 +3252,8 @@ bool machine__is_lock_function(struct machine *machine, u64 addr)return false; } + +int machine__hit_all_dsos (struct machine *machine) +{ +return __dsos__hit_all (&machine->dsos); +}
tools/perf/util/machine.h +2 -0
@@ -306,4 +306,6 @@ int machine__map_x86_64_entry_trampolines(struct machine *machine,int machine__resolve (struct machine *machine,struct addr_location *al,struct perf_sample *sample); +int machine__hit_all_dsos (struct machine *machine); +#endif /* __PERF_MACHINE_H */
tools/perf/util/session.c +21 -0
@@ -2906,3 +2906,24 @@ int perf_event__process_id_index(struct perf_session *session, }return 0 ; } + +int perf_session__dsos_hit_all (struct perf_session *session) +{ +struct rb_node *nd; +int err; + + err =machine__hit_all_dsos (&session->machines.host); +if (err) +return err; + +for (nd =rb_first_cached (&session->machines.guests); nd; + nd =rb_next (nd)) { + struct machine *pos = rb_entry(nd, struct machine, rb_node); + + err =machine__hit_all_dsos (pos); +if (err) +return err; + } + +return 0 ; +}
tools/perf/util/session.h +2 -0
@@ -154,6 +154,8 @@ int perf_session__deliver_synth_event(struct perf_session *session,union perf_event *event, struct perf_sample *sample); +int perf_session__dsos_hit_all (struct perf_session *session); +int perf_event__process_id_index (struct perf_session *session,union perf_event *event);
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 15/25 perf dsos: Tidy reference counting and locking
In reply to Ian RogersMove more functionality into dsos.c generally from machine, renaming functions to match their new usage. The find function is made to always "get" before returning a dso. Reduce the scope of locks in vdso to match the locking paradigm.
--- tools/perf/util/dsos.c | 73 +++++++++++++++++++++++++++++++++++---- tools/perf/util/dsos.h | 9 ++++- tools/perf/util/machine.c | 62 ++------------------------------- tools/perf/util/map.c | 4 +-- tools/perf/util/vdso.c | 48 +++++++++++-------------- 5 files changed, 97 insertions(+), 99 deletions(-)
tools/perf/util/dsos.c +66 -7
@@ -181,7 +181,7 @@ struct dso *__dsos__findnew_link_by_longname_id(struct rb_root *root, struct dso* at the end of the list of duplicates. */ if (!dso || (dso == this)) -return this;/* Find matching dso */ +return dso__get (this);/* Find matching dso */ /* * The core kernel DSOs may have duplicated long name. * In this case, the short name should be different.
@@ -253,15 +253,20 @@ static struct dso *__dsos__find_id(struct dsos *dsos, const char *name, struct dif (cmp_short) {list_for_each_entry (pos, &dsos->head, node)if (__dso__cmp_short_name (name, id, pos) ==0 ) -return pos; +return dso__get (pos);return NULL ; }return __dsos__findnew_by_longname_id (&dsos->root, name, id); } -struct dso *__dsos__find (struct dsos *dsos,const char *name,bool cmp_short) +struct dso *dsos__find (struct dsos *dsos,const char *name,bool cmp_short) { -return __dsos__find_id (dsos, name,NULL , cmp_short); +struct dso *res; + +down_read (&dsos->lock); + res =__dsos__find_id (dsos, name,NULL , cmp_short); +up_read (&dsos->lock); +return res; }static void dso__set_basename (struct dso *dso)
@@ -303,8 +308,6 @@ static struct dso *__dsos__addnew_id(struct dsos *dsos, const char *name, structif (dso !=NULL ) {__dsos__add (dsos, dso);dso__set_basename (dso); -/* Put dso here because __dsos_add already got it */ -dso__put (dso); }return dso; }
@@ -328,7 +331,7 @@ struct dso *dsos__findnew_id(struct dsos *dsos, const char *name, struct dso_id {struct dso *dso;down_write (&dsos->lock); - dso =dso__get (__dsos__findnew_id (dsos, name, id)); + dso =__dsos__findnew_id (dsos, name, id);up_write (&dsos->lock);return dso; }
@@ -374,3 +377,59 @@ int __dsos__hit_all(struct dsos *dsos)return 0 ; } + +struct dso *dsos__findnew_module_dso (struct dsos *dsos, +struct machine *machine, +struct kmod_path *m, +const char *filename) +{ +struct dso *dso; + +down_write (&dsos->lock); + + dso =__dsos__find_id (dsos, m->name,NULL ,/*cmp_short=*/ true); +if (!dso) { + dso =__dsos__addnew (dsos, m->name); +if (dso ==NULL ) + goto out_unlock; + +dso__set_module_info (dso, m, machine); +dso__set_long_name (dso,strdup (filename), true); + dso->kernel =DSO_SPACE__KERNEL ; + } + +out_unlock: +up_write (&dsos->lock); +return dso; +} + +struct dso *dsos__find_kernel_dso (struct dsos *dsos) +{ +struct dso *dso, *res =NULL ; + +down_read (&dsos->lock); +list_for_each_entry (dso, &dsos->head, node) { +/* +* The cpumode passed to is_kernel_module is not the cpumode of +* *this* event. If we insist on passing correct cpumode to +* is_kernel_module, we should record the cpumode when we adding +* this dso to the linked list. +* +* However we don't really need passing correct cpumode. We +* know the correct cpumode must be kernel mode (if not, we +* should not link it onto kernel_dsos list). +* +* Therefore, we pass PERF_RECORD_MISC_CPUMODE_UNKNOWN. +* is_kernel_module() treats it as a kernel cpumode. +*/ +if (!dso->kernel || +is_kernel_module (dso->long_name, +PERF_RECORD_MISC_CPUMODE_UNKNOWN )) +continue ; + + res =dso__get (dso); +break ; + } +up_read (&dsos->lock); +return res; +}
tools/perf/util/dsos.h +8 -1
@@ -10,6 +10,8 @@struct dso ;struct dso_id ; +struct kmod_path ; +struct machine ;/* * DSOs are put into both a list for fast iteration and rbtree for fast
@@ -33,7 +35,7 @@ void dsos__exit(struct dsos *dsos);void __dsos__add (struct dsos *dsos,struct dso *dso);void dsos__add (struct dsos *dsos,struct dso *dso);struct dso *__dsos__addnew (struct dsos *dsos,const char *name); -struct dso *__dsos__find (struct dsos *dsos,const char *name,bool cmp_short); +struct dso *dsos__find (struct dsos *dsos,const char *name,bool cmp_short);struct dso *dsos__findnew_id (struct dsos *dsos,const char *name,struct dso_id *id);
@@ -48,4 +50,9 @@ size_t __dsos__fprintf(struct dsos *dsos, FILE *fp);int __dsos__hit_all (struct dsos *dsos); +struct dso *dsos__findnew_module_dso (struct dsos *dsos,struct machine *machine, +struct kmod_path *m,const char *filename); + +struct dso *dsos__find_kernel_dso (struct dsos *dsos); +#endif /* __PERF_DSOS */
tools/perf/util/machine.c +2 -60
80 lines not on this page · raw message
tools/perf/util/map.c +1 -3
@@ -196,9 +196,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,* reading the header will have the build ID set and all future mmaps will * have it missing. */ -down_read (&machine->dsos.lock); - header_bid_dso =__dsos__find (&machine->dsos, filename, false); -up_read (&machine->dsos.lock); + header_bid_dso =dsos__find (&machine->dsos, filename, false);if (header_bid_dso && header_bid_dso->header_build_id) {dso__set_build_id (dso, &header_bid_dso->bid); dso->header_build_id =1 ;
tools/perf/util/vdso.c +20 -28
103 lines not on this page · raw message
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 16/25 perf dsos: Add dsos__for_each_dso
In reply to Ian RogersTo better abstract the dsos internals, add dsos__for_each_dso that does a callback on each dso. This also means the read lock can be correctly held.
--- tools/perf/builtin-inject.c | 25 +++++++----- tools/perf/util/build-id.c | 76 ++++++++++++++++++++----------------- tools/perf/util/dsos.c | 16 ++++++++ tools/perf/util/dsos.h | 8 +--- tools/perf/util/machine.c | 40 +++++++++++-------- 5 files changed, 100 insertions(+), 65 deletions(-)
tools/perf/builtin-inject.c +15 -10
@@ -1187,23 +1187,28 @@ static int synthesize_build_id(struct perf_inject *inject, struct dso *dso, pid_ process_build_id, machine); } +static int guest_session__add_build_ids_cb (struct dso *dso,void *data) +{ +struct guest_session *gs = data; + struct perf_inject *inject = container_of(gs, struct perf_inject, guest_session); + +if (!dso->has_build_id) +return 0 ; + +return synthesize_build_id (inject, dso, gs->machine_pid); + +} +static int guest_session__add_build_ids (struct guest_session *gs) { struct perf_inject *inject = container_of(gs, struct perf_inject, guest_session); -struct machine *machine = &gs->session->machines.host; -struct dso *dso; -int ret;/* Build IDs will be put in the Build ID feature section */ perf_header__set_feat (&inject->session->header,HEADER_BUILD_ID ); -dsos__for_each_with_build_id (dso, &machine->dsos.head) { - ret =synthesize_build_id (inject, dso, gs->machine_pid); -if (ret) -return ret; - } - -return 0 ; +return dsos__for_each_dso (&gs->session->machines.host.dsos, + guest_session__add_build_ids_cb, + gs); } static int guest_session__ksymbol_event(struct perf_tool *tool,
tools/perf/util/build-id.c +42 -34
90 lines not on this page · raw message
tools/perf/util/dsos.c +16 -0
@@ -433,3 +433,19 @@ struct dso *dsos__find_kernel_dso(struct dsos *dsos)up_read (&dsos->lock);return res; } + +int dsos__for_each_dso (struct dsos *dsos,int (*cb)(struct dso *dso,void *data),void *data) +{ +struct dso *dso; + +down_read (&dsos->lock); +list_for_each_entry (dso, &dsos->head, node) { +int err; + + err =cb (dso, data); +if (err) +return err; + } +up_read (&dsos->lock); +return 0 ; +}
tools/perf/util/dsos.h +2 -6
@@ -23,12 +23,6 @@ struct dsos {struct rw_semaphore lock; }; -#define dsos__for_each_with_build_id (pos, head) \ - list_for_each_entry(pos, head, node) \ - if (!pos->has_build_id) \ - continue; \ - else -void dsos__init (struct dsos *dsos);void dsos__exit (struct dsos *dsos);
@@ -55,4 +49,6 @@ struct dso *dsos__findnew_module_dso(struct dsos *dsos, struct machine *machine,struct dso *dsos__find_kernel_dso (struct dsos *dsos); +int dsos__for_each_dso (struct dsos *dsos,int (*cb)(struct dso *dso,void *data),void *data); +#endif /* __PERF_DSOS */
tools/perf/util/machine.c +25 -15
@@ -1561,16 +1561,14 @@ int machine__create_kernel_maps(struct machine *machine)return ret; } -static bool machine__uses_kcore (struct machine *machine) +static int machine__uses_kcore_cb (struct dso *dso,void *data__maybe_unused ) { -struct dso *dso; - -list_for_each_entry (dso, &machine->dsos.head, node) { -if (dso__is_kcore (dso)) -return true; - } +return dso__is_kcore (dso) ?1 :0 ; +} -return false; +static bool machine__uses_kcore (struct machine *machine) +{ +return dsos__for_each_dso (&machine->dsos, machine__uses_kcore_cb,NULL ) !=0 ? true : false; } static bool perf_event__is_extra_kernel_mmap(struct machine *machine,
@@ -3136,16 +3134,28 @@ char *machine__resolve_kernel_addr(void *vmachine, unsigned long long *addrp, chreturn sym->name; } +struct machine__for_each_dso_cb_args { +struct machine *machine; +machine__dso_t fn; +void *priv; +}; + +static int machine__for_each_dso_cb (struct dso *dso,void *data) +{ +struct machine__for_each_dso_cb_args *args = data; + +return args->fn (dso, args->machine, args->priv); +} +int machine__for_each_dso (struct machine *machine,machine__dso_t fn,void *priv) { -struct dso *pos; -int err =0 ; +struct machine__for_each_dso_cb_args args = { + .machine = machine, + .fn = fn, + .priv = priv, + }; -list_for_each_entry (pos, &machine->dsos.head, node) { -if (fn (pos, machine, priv)) - err =-1 ; - } -return err; +return dsos__for_each_dso (&machine->dsos, machine__for_each_dso_cb, &args); }int machine__for_each_kernel_map (struct machine *machine,machine__map_t fn,void *priv)
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 17/25 perf dso: Move dso functions out of dsos
In reply to Ian RogersMove dso and dso_id functions to dso.c to match the struct declarations.
--- tools/perf/util/dso.c | 61 ++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/dso.h | 4 +++ tools/perf/util/dsos.c | 61 ------------------------------------------ 3 files changed, 65 insertions(+), 61 deletions(-)
tools/perf/util/dso.c +61 -0
@@ -1269,6 +1269,67 @@ static void dso__set_long_name_id(struct dso *dso, const char *name, struct dso___dsos__findnew_link_by_longname_id (root, dso,NULL , id); } +static int __dso_id__cmp (struct dso_id *a,struct dso_id *b) +{ +if (a->maj > b->maj)return -1 ; +if (a->maj < b->maj)return 1 ; + +if (a->min > b->min)return -1 ; +if (a->min < b->min)return 1 ; + +if (a->ino > b->ino)return -1 ; +if (a->ino < b->ino)return 1 ; + +/* +* Synthesized MMAP events have zero ino_generation, avoid comparing +* them with MMAP events with actual ino_generation. +* +* I found it harmful because the mismatch resulted in a new +* dso that did not have a build ID whereas the original dso did have a +* build ID. The build ID was essential because the object was not found +* otherwise. - Adrian +*/ +if (a->ino_generation && b->ino_generation) { +if (a->ino_generation > b->ino_generation)return -1 ; +if (a->ino_generation < b->ino_generation)return 1 ; + } + +return 0 ; +} + +bool dso_id__empty (struct dso_id *id) +{ +if (!id) +return true; + +return !id->maj && !id->min && !id->ino && !id->ino_generation; +} + +void dso__inject_id (struct dso *dso,struct dso_id *id) +{ + dso->id.maj = id->maj; + dso->id.min = id->min; + dso->id.ino = id->ino; + dso->id.ino_generation = id->ino_generation; +} + +int dso_id__cmp (struct dso_id *a,struct dso_id *b) +{ +/* +* The second is always dso->id, so zeroes if not set, assume passing +* NULL for a means a zeroed id +*/ +if (dso_id__empty (a) ||dso_id__empty (b)) +return 0 ; + +return __dso_id__cmp (a, b); +} + +int dso__cmp_id (struct dso *a,struct dso *b) +{ +return __dso_id__cmp (&a->id, &b->id); +} +void dso__set_long_name (struct dso *dso,const char *name,bool name_allocated) {dso__set_long_name_id (dso, name,NULL , name_allocated);
tools/perf/util/dso.h +4 -0
@@ -237,6 +237,9 @@ static inline void dso__set_loaded(struct dso *dso) dso->loaded = true; } +int dso_id__cmp (struct dso_id *a,struct dso_id *b); +bool dso_id__empty (struct dso_id *id); +struct dso *dso__new_id (const char *name,struct dso_id *id);struct dso *dso__new (const char *name);void dso__delete (struct dso *dso);
@@ -244,6 +247,7 @@ void dso__delete(struct dso *dso);int dso__cmp_id (struct dso *a,struct dso *b);void dso__set_short_name (struct dso *dso,const char *name,bool name_allocated);void dso__set_long_name (struct dso *dso,const char *name,bool name_allocated); +void dso__inject_id (struct dso *dso,struct dso_id *id);int dso__name_len (const struct dso *dso);
tools/perf/util/dsos.c +0 -61
67 lines not on this page · raw message
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 18/25 perf dsos: Switch more loops to dsos__for_each_dso
In reply to Ian RogersSwitch loops within dsos.c, add a version that isn't locked. Switch some unlocked loops to hold the read lock.
--- tools/perf/util/build-id.c | 2 +- tools/perf/util/dsos.c | 258 ++++++++++++++++++++++++------------- tools/perf/util/dsos.h | 8 +- tools/perf/util/machine.c | 8 +- 4 files changed, 174 insertions(+), 102 deletions(-)
tools/perf/util/build-id.c +1 -1
@@ -964,7 +964,7 @@ int perf_session__cache_build_ids(struct perf_session *session)static bool machine__read_build_ids (struct machine *machine,bool with_hits) { -return __dsos__read_build_ids (&machine->dsos, with_hits); +return dsos__read_build_ids (&machine->dsos, with_hits); }bool perf_session__read_build_ids (struct perf_session *session,bool with_hits)
tools/perf/util/dsos.c +165 -93
323 lines not on this page · raw message
tools/perf/util/dsos.h +4 -4
@@ -33,16 +33,16 @@ struct dso *dsos__find(struct dsos *dsos, const char *name, bool cmp_short);struct dso *dsos__findnew_id (struct dsos *dsos,const char *name,struct dso_id *id); -bool __dsos__read_build_ids (struct dsos *dsos,bool with_hits); +bool dsos__read_build_ids (struct dsos *dsos,bool with_hits);struct dso *__dsos__findnew_link_by_longname_id (struct rb_root *root,struct dso *dso,const char *name,struct dso_id *id); -size_t __dsos__fprintf_buildid (struct dsos *dsos,FILE *fp, +size_t dsos__fprintf_buildid (struct dsos *dsos,FILE *fp,bool (skip )(struct dso *dso,int parm),int parm); -size_t __dsos__fprintf (struct dsos *dsos,FILE *fp); +size_t dsos__fprintf (struct dsos *dsos,FILE *fp); -int __dsos__hit_all (struct dsos *dsos); +int dsos__hit_all (struct dsos *dsos);struct dso *dsos__findnew_module_dso (struct dsos *dsos,struct machine *machine,struct kmod_path *m,const char *filename);
tools/perf/util/machine.c +4 -4
@@ -852,11 +852,11 @@ static struct map *machine__addnew_module_map(struct machine *machine, u64 startsize_t machines__fprintf_dsos (struct machines *machines,FILE *fp) {struct rb_node *nd; -size_t ret =__dsos__fprintf (&machines->host.dsos, fp); +size_t ret =dsos__fprintf (&machines->host.dsos, fp);for (nd =rb_first_cached (&machines->guests); nd; nd =rb_next (nd)) { struct machine *pos = rb_entry(nd, struct machine, rb_node); - ret +=__dsos__fprintf (&pos->dsos, fp); + ret +=dsos__fprintf (&pos->dsos, fp); }return ret;
@@ -865,7 +865,7 @@ size_t machines__fprintf_dsos(struct machines *machines, FILE *fp)size_t machine__fprintf_dsos_buildid (struct machine *m,FILE *fp,bool (skip )(struct dso *dso,int parm),int parm) { -return __dsos__fprintf_buildid (&m->dsos, fp, skip, parm); +return dsos__fprintf_buildid (&m->dsos, fp, skip, parm); } size_t machines__fprintf_dsos_buildid(struct machines *machines, FILE *fp,
@@ -3207,5 +3207,5 @@ bool machine__is_lock_function(struct machine *machine, u64 addr)int machine__hit_all_dsos (struct machine *machine) { -return __dsos__hit_all (&machine->dsos); +return dsos__hit_all (&machine->dsos); }
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 19/25 perf dsos: Switch backing storage to array from rbtree/list
In reply to Ian RogersDSOs were held on a list for fast iteration and in an rbtree for fast finds. Switch to using a lazily sorted array where iteration is just iterating through the array and binary searches are the same complexity as searching the rbtree. The find may need to sort the array first which does increase the complexity, but add operations have lower complexity and overall the complexity should remain about the same.
The set name operations on the dso just records that the array is no longer sorted, avoiding complexity in rebalancing the rbtree. Tighter locking discipline is enforced to avoid the array being resorted while long and short names or ids are changed.
The array is smaller in size, replacing 6 pointers with 2, and so even with extra allocated space in the array, the array may be 50% unoccupied, the memory saving should be at least 2x.
--- tools/perf/util/dso.c | 67 +++++++++------ tools/perf/util/dso.h | 10 +-- tools/perf/util/dsos.c | 188 ++++++++++++++++++++++++++--------------- tools/perf/util/dsos.h | 21 +++-- 4 files changed, 177 insertions(+), 109 deletions(-)
tools/perf/util/dso.c +42 -25
@@ -1241,35 +1241,35 @@ struct dso *machine__findnew_kernel(struct machine *machine, const char *name,return dso; } -static void dso__set_long_name_id (struct dso *dso,const char *name,struct dso_id *id,bool name_allocated) +static void dso__set_long_name_id (struct dso *dso,const char *name,bool name_allocated) { -struct rb_root *root = dso->root; +struct dsos *dsos = dso->dsos;if (name ==NULL )return ; -if (dso->long_name_allocated) -free ((char *)dso->long_name); - -if (root) { -rb_erase (&dso->rb_node, root); +if (dsos) {/* -* __dsos__findnew_link_by_longname_id() isn't guaranteed to -* add it back, so a clean removal is required here. +* Need to avoid re-sorting the dsos breaking by non-atomically +* renaming the dso. */ -RB_CLEAR_NODE (&dso->rb_node); - dso->root =NULL ; +down_write (&dsos->lock); } +if (dso->long_name_allocated) +free ((char *)dso->long_name); + dso->long_name = name; dso->long_name_len =strlen (name); dso->long_name_allocated = name_allocated; -if (root) -__dsos__findnew_link_by_longname_id (root, dso,NULL , id); +if (dsos) { + dsos->sorted = false; +up_write (&dsos->lock); + } } -static int __dso_id__cmp (struct dso_id *a,struct dso_id *b) +static int __dso_id__cmp (const struct dso_id *a,const struct dso_id *b) {if (a->maj > b->maj)return -1 ;if (a->maj < b->maj)return 1 ;
@@ -1297,7 +1297,7 @@ static int __dso_id__cmp(struct dso_id *a, struct dso_id *b)return 0 ; } -bool dso_id__empty (struct dso_id *id) +bool dso_id__empty (const struct dso_id *id) {if (!id)return true;
@@ -1305,15 +1305,22 @@ bool dso_id__empty(struct dso_id *id)return !id->maj && !id->min && !id->ino && !id->ino_generation; } -void dso__inject_id (struct dso *dso,struct dso_id *id) +void __dso__inject_id (struct dso *dso,struct dso_id *id) { +struct dsos *dsos = dso->dsos; + +/* dsos write lock held by caller. */ + dso->id.maj = id->maj; dso->id.min = id->min; dso->id.ino = id->ino; dso->id.ino_generation = id->ino_generation; + +if (dsos) + dsos->sorted = false; } -int dso_id__cmp (struct dso_id *a,struct dso_id *b) +int dso_id__cmp (const struct dso_id *a,const struct dso_id *b) {/* * The second is always dso->id, so zeroes if not set, assume passing
@@ -1332,20 +1339,34 @@ int dso__cmp_id(struct dso *a, struct dso *b)void dso__set_long_name (struct dso *dso,const char *name,bool name_allocated) { -dso__set_long_name_id (dso, name,NULL , name_allocated); +dso__set_long_name_id (dso, name, name_allocated); }void dso__set_short_name (struct dso *dso,const char *name,bool name_allocated) { +struct dsos *dsos = dso->dsos; +if (name ==NULL )return ; +if (dsos) { +/* +* Need to avoid re-sorting the dsos breaking by non-atomically +* renaming the dso. +*/ +down_write (&dsos->lock); + }if (dso->short_name_allocated)free ((char *)dso->short_name); dso->short_name = name; dso->short_name_len =strlen (name); dso->short_name_allocated = name_allocated; + +if (dsos) { + dsos->sorted = false; +up_write (&dsos->lock); + } }int dso__name_len (const struct dso *dso)
@@ -1381,7 +1402,7 @@ struct dso *dso__new_id(const char *name, struct dso_id *id)strcpy (dso->name, name);if (id) dso->id = *id; -dso__set_long_name_id (dso, dso->name, id, false); +dso__set_long_name_id (dso, dso->name, false);dso__set_short_name (dso, dso->name, false); dso->symbols =RB_ROOT_CACHED ; dso->symbol_names =NULL ;
@@ -1405,9 +1426,6 @@ struct dso *dso__new_id(const char *name, struct dso_id *id) dso->is_kmod =0 ; dso->needs_swap =DSO_SWAP__UNSET ; dso->comp =COMP_ID__NONE ; -RB_CLEAR_NODE (&dso->rb_node); - dso->root =NULL ; -INIT_LIST_HEAD (&dso->node);INIT_LIST_HEAD (&dso->data.open_entry);mutex_init (&dso->lock);refcount_set (&dso->refcnt,1 );
@@ -1423,9 +1441,8 @@ struct dso *dso__new(const char *name)void dso__delete (struct dso *dso) { -if (!RB_EMPTY_NODE (&dso->rb_node)) -pr_err ("DSO %s is still in rbtree when being deleted!\n" , - dso->long_name); +if (dso->dsos) +pr_err ("DSO %s is still in rbtree when being deleted!\n" , dso->long_name);/* free inlines first, as they reference symbols */ inlines__tree_delete (&dso->inlined_nodes);
tools/perf/util/dso.h +4 -6
@@ -146,9 +146,7 @@ struct auxtrace_cache;struct dso {struct mutex lock; -struct list_head node; -struct rb_node rb_node;/* rbtree node sorted by long name */ -struct rb_root *root;/* root of rbtree that rb_node is in */ +struct dsos *dsos;struct rb_root_cached symbols;struct symbol **symbol_names;size_t symbol_names_len;
@@ -237,8 +235,8 @@ static inline void dso__set_loaded(struct dso *dso) dso->loaded = true; } -int dso_id__cmp (struct dso_id *a,struct dso_id *b); -bool dso_id__empty (struct dso_id *id); +int dso_id__cmp (const struct dso_id *a,const struct dso_id *b); +bool dso_id__empty (const struct dso_id *id);struct dso *dso__new_id (const char *name,struct dso_id *id);struct dso *dso__new (const char *name);
@@ -247,7 +245,7 @@ void dso__delete(struct dso *dso);int dso__cmp_id (struct dso *a,struct dso *b);void dso__set_short_name (struct dso *dso,const char *name,bool name_allocated);void dso__set_long_name (struct dso *dso,const char *name,bool name_allocated); -void dso__inject_id (struct dso *dso,struct dso_id *id); +void __dso__inject_id (struct dso *dso,struct dso_id *id);int dso__name_len (const struct dso *dso);
tools/perf/util/dsos.c +118 -70
291 lines not on this page · raw message
tools/perf/util/dsos.h +13 -8
@@ -14,20 +14,22 @@ struct kmod_path;struct machine ;/* -* DSOs are put into both a list for fast iteration and rbtree for fast -* long name lookup. +* Collection of DSOs as an array for iteration speed, but sorted for O(n) +* lookup. */ struct dsos { -struct list_head head; -struct rb_root root;/* rbtree root sorted by long name */ struct rw_semaphore lock; +struct dso **dsos; +unsigned int cnt; +unsigned int allocated; +bool sorted; };void dsos__init (struct dsos *dsos);void dsos__exit (struct dsos *dsos); -void __dsos__add (struct dsos *dsos,struct dso *dso); -void dsos__add (struct dsos *dsos,struct dso *dso); +int __dsos__add (struct dsos *dsos,struct dso *dso); +int dsos__add (struct dsos *dsos,struct dso *dso);struct dso *__dsos__addnew (struct dsos *dsos,const char *name);struct dso *dsos__find (struct dsos *dsos,const char *name,bool cmp_short);
@@ -35,8 +37,11 @@ struct dso *dsos__findnew_id(struct dsos *dsos, const char *name, struct dso_idbool dsos__read_build_ids (struct dsos *dsos,bool with_hits); -struct dso *__dsos__findnew_link_by_longname_id (struct rb_root *root,struct dso *dso, -const char *name,struct dso_id *id); +struct dso *__dsos__findnew_link_by_longname_id (struct dsos *dsos, +struct dso *dso, +const char *name, +struct dso_id *id, +bool write_locked);size_t dsos__fprintf_buildid (struct dsos *dsos,FILE *fp,bool (skip )(struct dso *dso,int parm),int parm);
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 20/25 perf dsos: Remove __dsos__addnew
In reply to Ian RogersFunction no longer used so remove.
--- tools/perf/util/dsos.c | 5 ----- tools/perf/util/dsos.h | 1 - 2 files changed, 6 deletions(-)
tools/perf/util/dsos.c +0 -5
@@ -341,11 +341,6 @@ static struct dso *__dsos__addnew_id(struct dsos *dsos, const char *name, structreturn dso; } -struct dso *__dsos__addnew (struct dsos *dsos,const char *name) -{ -return __dsos__addnew_id (dsos, name,NULL ); -} -static struct dso *__dsos__findnew_id (struct dsos *dsos,const char *name,struct dso_id *id) {struct dso *dso =__dsos__find_id (dsos, name, id, false,/*write_locked=*/ true);
tools/perf/util/dsos.h +0 -1
@@ -30,7 +30,6 @@ void dsos__exit(struct dsos *dsos);int __dsos__add (struct dsos *dsos,struct dso *dso);int dsos__add (struct dsos *dsos,struct dso *dso); -struct dso *__dsos__addnew (struct dsos *dsos,const char *name);struct dso *dsos__find (struct dsos *dsos,const char *name,bool cmp_short);struct dso *dsos__findnew_id (struct dsos *dsos,const char *name,struct dso_id *id);
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 21/25 perf dsos: Remove __dsos__findnew_link_by_longname_id
In reply to Ian RogersFunction was only called in dsos.c with the dso parameter as NULL. Remove the function and specialize for the dso being NULL case removing other unused functions along the way.
--- tools/perf/util/dsos.c | 51 +++++++++--------------------------------- tools/perf/util/dsos.h | 6 ----- 2 files changed, 10 insertions(+), 47 deletions(-)
tools/perf/util/dsos.c +10 -41
@@ -119,11 +119,6 @@ static int __dso__cmp_short_name(const char *short_name, struct dso_id *id, strureturn rc ?:dso_id__cmp (id, &b->id); } -static int dso__cmp_short_name (struct dso *a,struct dso *b) -{ -return __dso__cmp_short_name (a->short_name, &a->id, b); -} -static int dsos__cmp_long_name_id_short_name (const void *va,const void *vb) {const struct dso *a = *((const struct dso **)va);
@@ -143,20 +138,21 @@ static int dsos__cmp_long_name_id_short_name(const void *va, const void *vb)* Either one of the dso or name parameter must be non-NULL or the * function will not work. */ -struct dso *__dsos__findnew_link_by_longname_id (struct dsos *dsos, -struct dso *dso, -const char *name, -struct dso_id *id, -bool write_locked) +static struct dso *__dsos__find_by_longname_id (struct dsos *dsos, +const char *name, +struct dso_id *id, +bool write_locked) {int low =0 , high = dsos->cnt -1 ;if (!dsos->sorted) {if (!write_locked) { +struct dso *dso; +up_read (&dsos->lock);down_write (&dsos->lock); - dso =__dsos__findnew_link_by_longname_id (dsos, dso, name, id, -/*write_locked=*/ true); + dso =__dsos__find_by_longname_id (dsos, name, id, +/*write_locked=*/ true);up_write (&dsos->lock);down_read (&dsos->lock);return dso;
@@ -166,9 +162,6 @@ struct dso *__dsos__findnew_link_by_longname_id(struct dsos *dsos, dsos->sorted = true; } -if (!name) - name = dso->long_name; -/* * Find node with the matching name */
@@ -178,31 +171,13 @@ struct dso *__dsos__findnew_link_by_longname_id(struct dsos *dsos,int rc =__dso__cmp_long_name (name, id, this);if (rc ==0 ) { -/* -* In case the new DSO is a duplicate of an existing -* one, print a one-time warning & put the new entry -* at the end of the list of duplicates. -*/ -if (!dso || (dso == this)) -return dso__get (this);/* Find matching dso */ -/* -* The core kernel DSOs may have duplicated long name. -* In this case, the short name should be different. -* Comparing the short names to differentiate the DSOs. -*/ - rc =dso__cmp_short_name (dso, this); -if (rc ==0 ) { -pr_err ("Duplicated dso name: %s\n" , name); -return NULL ; - } +return dso__get (this);/* Find matching dso */ }if (rc <0 ) high = mid -1 ;else low = mid +1 ; } -if (dso) -__dsos__add (dsos, dso);return NULL ; }
@@ -240,12 +215,6 @@ int dsos__add(struct dsos *dsos, struct dso *dso)return ret; } -static struct dso *__dsos__findnew_by_longname_id (struct dsos *dsos,const char *name, -struct dso_id *id,bool write_locked) -{ -return __dsos__findnew_link_by_longname_id (dsos,NULL , name, id, write_locked); -} -struct dsos__find_id_cb_args {const char *name;struct dso_id *id;
@@ -279,7 +248,7 @@ static struct dso *__dsos__find_id(struct dsos *dsos, const char *name, struct d__dsos__for_each_dso (dsos, dsos__find_id_cb, &args);return args.res; } - res =__dsos__findnew_by_longname_id (dsos, name, id, write_locked); + res =__dsos__find_by_longname_id (dsos, name, id, write_locked);return res; }
tools/perf/util/dsos.h +0 -6
@@ -36,12 +36,6 @@ struct dso *dsos__findnew_id(struct dsos *dsos, const char *name, struct dso_idbool dsos__read_build_ids (struct dsos *dsos,bool with_hits); -struct dso *__dsos__findnew_link_by_longname_id (struct dsos *dsos, -struct dso *dso, -const char *name, -struct dso_id *id, -bool write_locked); -size_t dsos__fprintf_buildid (struct dsos *dsos,FILE *fp,bool (skip )(struct dso *dso,int parm),int parm);size_t dsos__fprintf (struct dsos *dsos,FILE *fp);
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 22/25 perf dsos: Switch hand code to bsearch
In reply to Ian RogersSwitch to using the bsearch library function rather than having a hand written binary search. Const-ify some static functions to avoid compiler warnings.
--- tools/perf/util/dsos.c | 46 +++++++++++++++++++++++++----------------- 1 file changed, 27 insertions(+), 19 deletions(-)
tools/perf/util/dsos.c +27 -19
@@ -107,13 +107,15 @@ bool dsos__read_build_ids(struct dsos *dsos, bool with_hits)return args.have_build_id; } -static int __dso__cmp_long_name (const char *long_name,struct dso_id *id,struct dso *b) +static int __dso__cmp_long_name (const char *long_name,const struct dso_id *id, +const struct dso *b) {int rc =strcmp (long_name, b->long_name);return rc ?:dso_id__cmp (id, &b->id); } -static int __dso__cmp_short_name (const char *short_name,struct dso_id *id,struct dso *b) +static int __dso__cmp_short_name (const char *short_name,const struct dso_id *id, +const struct dso *b) {int rc =strcmp (short_name, b->short_name);return rc ?:dso_id__cmp (id, &b->id);
@@ -133,6 +135,19 @@ static int dsos__cmp_long_name_id_short_name(const void *va, const void *vb)return rc; } +struct dsos__key { +const char *long_name; +const struct dso_id *id; +}; + +static int dsos__cmp_key_long_name_id (const void *vkey,const void *vdso) +{ +const struct dsos__key *key = vkey; +const struct dso *dso = *((const struct dso **)vdso); + +return __dso__cmp_long_name (key->long_name, key->id, dso); +} +/* * Find a matching entry and/or link current entry to RB tree. * Either one of the dso or name parameter must be non-NULL or the
@@ -143,7 +158,11 @@ static struct dso *__dsos__find_by_longname_id(struct dsos *dsos, struct dso_id *id, bool write_locked) { - int low = 0, high = dsos->cnt - 1; +struct dsos__key key = { + .long_name = name, + .id = id, + }; +struct dso **res;if (!dsos->sorted) {if (!write_locked) {
@@ -162,23 +181,12 @@ static struct dso *__dsos__find_by_longname_id(struct dsos *dsos, dsos->sorted = true; } -/* -* Find node with the matching name -*/ -while (low <= high) { -int mid = (low + high) /2 ; -struct dso *this = dsos->dsos[mid]; -int rc =__dso__cmp_long_name (name, id, this); + res =bsearch (&key, dsos->dsos, dsos->cnt,sizeof (struct dso *), + dsos__cmp_key_long_name_id); +if (!res) +return NULL ; -if (rc ==0 ) { -return dso__get (this);/* Find matching dso */ - } -if (rc <0 ) - high = mid -1 ; -else - low = mid +1 ; - } -return NULL ; +return dso__get (*res); }int __dsos__add (struct dsos *dsos,struct dso *dso)
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 23/25 perf dso: Add reference count checking and accessor functions
In reply to Ian RogersAdd reference count checking to struct dso, this can help with implementing correct reference counting discipline. To avoid RC_CHK_ACCESS everywhere, add accessor functions for the variables in struct dso.
The majority of the change is mechanical in nature and not easy to split up.
--- tools/perf/builtin-annotate.c | 8 +- tools/perf/builtin-buildid-cache.c | 2 +- tools/perf/builtin-buildid-list.c | 18 +- tools/perf/builtin-inject.c | 71 ++- tools/perf/builtin-kallsyms.c | 2 +- tools/perf/builtin-mem.c | 4 +- tools/perf/builtin-report.c | 6 +- tools/perf/builtin-script.c | 8 +- tools/perf/builtin-top.c | 4 +- tools/perf/builtin-trace.c | 2 +- tools/perf/tests/code-reading.c | 8 +- tools/perf/tests/dso-data.c | 11 +- tools/perf/tests/hists_common.c | 6 +- tools/perf/tests/hists_cumulate.c | 4 +- tools/perf/tests/hists_output.c | 2 +- tools/perf/tests/maps.c | 4 +- tools/perf/tests/symbols.c | 2 +- tools/perf/tests/vmlinux-kallsyms.c | 6 +- tools/perf/ui/browsers/annotate.c | 6 +- tools/perf/ui/browsers/hists.c | 8 +- tools/perf/ui/browsers/map.c | 4 +- tools/perf/util/annotate-data.c | 6 +- tools/perf/util/annotate.c | 45 +- tools/perf/util/auxtrace.c | 2 +- tools/perf/util/block-info.c | 2 +- tools/perf/util/bpf-event.c | 8 +- tools/perf/util/build-id.c | 38 +- tools/perf/util/callchain.c | 2 +- tools/perf/util/data-convert-json.c | 2 +- tools/perf/util/db-export.c | 6 +- tools/perf/util/dlfilter.c | 12 +- tools/perf/util/dso.c | 365 +++++++------ tools/perf/util/dso.h | 483 ++++++++++++++++-- tools/perf/util/dsos.c | 54 +- tools/perf/util/event.c | 8 +- tools/perf/util/header.c | 8 +- tools/perf/util/hist.c | 4 +- tools/perf/util/intel-pt.c | 22 +- tools/perf/util/machine.c | 46 +- tools/perf/util/map.c | 69 ++- tools/perf/util/maps.c | 14 +- tools/perf/util/probe-event.c | 25 +- .../util/scripting-engines/trace-event-perl.c | 6 +- .../scripting-engines/trace-event-python.c | 21 +- tools/perf/util/sort.c | 19 +- tools/perf/util/srcline.c | 65 +-- tools/perf/util/symbol-elf.c | 92 ++-- tools/perf/util/symbol.c | 186 +++---- tools/perf/util/symbol_fprintf.c | 4 +- tools/perf/util/synthetic-events.c | 24 +- tools/perf/util/thread.c | 4 +- tools/perf/util/unwind-libunwind-local.c | 18 +- tools/perf/util/unwind-libunwind.c | 2 +- tools/perf/util/vdso.c | 8 +- 54 files changed, 1140 insertions(+), 716 deletions(-)
tools/perf/builtin-annotate.c +4 -4
@@ -217,7 +217,7 @@ static int process_branch_callback(struct evsel *evsel, }if (a.map !=NULL ) -map__dso (a.map)->hit =1 ; +dso__set_hit (map__dso (a.map));hist__account_cycles (sample->branch_stack, al, sample, false,NULL );
@@ -252,7 +252,7 @@ static int evsel__add_sample(struct evsel *evsel, struct perf_sample *sample,if (al->sym !=NULL ) {struct dso *dso =map__dso (al->map); -rb_erase_cached (&al->sym->rb_node, &dso->symbols); +rb_erase_cached (&al->sym->rb_node,dso__symbols (dso));symbol__delete (al->sym);dso__reset_find_symbol_cache (dso); }
@@ -341,7 +341,7 @@ static void print_annotated_data_header(struct hist_entry *he, struct evsel *evs }printf ("Annotate type: '%s' in %s (%d samples):\n" , - he->mem_type->self.type_name, dso->name, nr_samples); + he->mem_type->self.type_name,dso__name (dso), nr_samples);if (evsel__is_group_event (evsel)) {struct evsel *pos;
@@ -487,7 +487,7 @@ static void hists__find_annotations(struct hists *hists, struct hist_entry *he = rb_entry(nd, struct hist_entry, rb_node);struct annotation *notes; -if (he->ms.sym ==NULL ||map__dso (he->ms.map)->annotate_warned) +if (he->ms.sym ==NULL ||dso__annotate_warned (map__dso (he->ms.map))) goto find_next; if (ann->sym_hist_filter &&
tools/perf/builtin-buildid-cache.c +1 -1
@@ -286,7 +286,7 @@ static bool dso__missing_buildid_cache(struct dso *dso, int parm __maybe_unused)pr_warning ("Problems with %s file, consider removing it from the cache\n" , filename); - }else if (memcmp (dso->bid.data, bid.data, bid.size)) { + }else if (memcmp (dso__bid (dso)->data, bid.data, bid.size)) {pr_warning ("Problems with %s file, consider removing it from the cache\n" , filename); }
tools/perf/builtin-buildid-list.c +10 -8
@@ -26,16 +26,18 @@ static int buildid__map_cb(struct map *map, void *arg __maybe_unused) {const struct dso *dso =map__dso (map);char bid_buf[SBUILD_ID_SIZE ]; +const char *dso_long_name =dso__long_name (dso); +const char *dso_short_name =dso__short_name (dso);memset (bid_buf,0 ,sizeof (bid_buf)); -if (dso->has_build_id) -build_id__sprintf (&dso->bid, bid_buf); +if (dso__has_build_id (dso)) +build_id__sprintf (dso__bid_const (dso), bid_buf);printf ("%s %16" PRIx64" %16" PRIx64, bid_buf,map__start (map),map__end (map)); -if (dso->long_name !=NULL ) { -printf (" %s" , dso->long_name); - }else if (dso->short_name !=NULL ) { -printf (" %s" , dso->short_name); - } +if (dso_long_name !=NULL ) +printf (" %s" , dso_long_name); +else if (dso_short_name !=NULL ) +printf (" %s" , dso_short_name); +printf ("\n" );return 0 ;
@@ -76,7 +78,7 @@ static int filename__fprintf_build_id(const char *name, FILE *fp)static bool dso__skip_buildid (struct dso *dso,int with_hits) { -return with_hits && !dso->hit; +return with_hits && !dso__hit (dso); }static int perf_session__list_build_ids (bool force,bool with_hits)
tools/perf/builtin-inject.c +35 -36
166 lines not on this page · raw message
tools/perf/builtin-kallsyms.c +1 -1
@@ -38,7 +38,7 @@ static int __cmd_kallsyms(int argc, const char **argv) dso =map__dso (map);printf ("%s: %s %s %#" PRIx64"-%#" PRIx64" (%#" PRIx64"-%#" PRIx64")\n" , - symbol->name, dso->short_name, dso->long_name, + symbol->name,dso__short_name (dso),dso__long_name (dso),map__unmap_ip (map, symbol->start),map__unmap_ip (map, symbol->end), symbol->start, symbol->end); }
tools/perf/builtin-mem.c +2 -2
@@ -213,7 +213,7 @@ dump_raw_samples(struct perf_tool *tool,if (al.map !=NULL ) { dso =map__dso (al.map);if (dso) - dso->hit =1 ; +dso__set_hit (dso); } field_sep = symbol_conf.field_sep;
@@ -255,7 +255,7 @@ dump_raw_samples(struct perf_tool *tool, symbol_conf.field_sep, sample->data_src, symbol_conf.field_sep, - dso ? dso->long_name :"???" , + dso ?dso__long_name (dso) :"???" , al.sym ? al.sym->name : "???"); out_put:addr_location__exit (&al);
tools/perf/builtin-report.c +3 -3
@@ -322,7 +322,7 @@ static int process_sample_event(struct perf_tool *tool, }if (al.map !=NULL ) -map__dso (al.map)->hit =1 ; +dso__set_hit (map__dso (al.map));if (ui__has_annotation () || rep->symbol_ipc || rep->total_cycles_mode) { hist__account_cycles(sample->branch_stack, &al, sample,
@@ -609,7 +609,7 @@ static void report__warn_kptr_restrict(const struct report *rep)return ;if (kernel_map ==NULL || - (map__dso (kernel_map)->hit && + (dso__hit (map__dso (kernel_map)) && (kernel_kmap->ref_reloc_sym ==NULL || kernel_kmap->ref_reloc_sym->addr ==0 ))) { const char *desc =
@@ -850,7 +850,7 @@ static int maps__fprintf_task_cb(struct map *map, void *data) prot &PROT_EXEC ?'x' :'-' ,map__flags (map) ?'s' :'p' ,map__pgoff (map), - dso->id.ino, dso->name); + dso__id_const(dso)->ino, dso__name(dso));if (ret <0 )return ret;
tools/perf/builtin-script.c +4 -4
@@ -1011,11 +1011,11 @@ static int perf_sample__fprintf_brstackoff(struct perf_sample *sample, to = entries[i].to;if (thread__find_map_fb (thread, sample->cpumode, from, &alf) && - !map__dso (alf.map)->adjust_symbols) + !dso__adjust_symbols (map__dso (alf.map))) from =map__dso_map_ip (alf.map, from);if (thread__find_map_fb (thread, sample->cpumode, to, &alt) && - !map__dso (alt.map)->adjust_symbols) + !dso__adjust_symbols (map__dso (alt.map))) to =map__dso_map_ip (alt.map, to); printed +=fprintf (fp," 0x%" PRIx64, from);
@@ -1076,7 +1076,7 @@ static int grab_bb(u8 *buffer, u64 start, u64 end,pr_debug ("\tcannot resolve %" PRIx64"-%" PRIx64"\n" , start, end); goto out; } -if (dso->data.status ==DSO_DATA_STATUS_ERROR ) { +if (dso__data (dso)->status ==DSO_DATA_STATUS_ERROR ) {pr_debug ("\tcannot resolve %" PRIx64"-%" PRIx64"\n" , start, end); goto out; }
@@ -1088,7 +1088,7 @@ static int grab_bb(u8 *buffer, u64 start, u64 end, len =dso__data_read_offset (dso, machine, offset, (u8 *)buffer, end - start +MAXINSN ); - *is64bit = dso->is_64_bit; + *is64bit =dso__is_64_bit (dso);if (len <=0 )pr_debug ("\tcannot fetch code for block at %" PRIx64"-%" PRIx64"\n" , start, end);
tools/perf/builtin-top.c +2 -2
@@ -129,7 +129,7 @@ static int perf_top__parse_source(struct perf_top *top, struct hist_entry *he)/* * We can't annotate with just /proc/kallsyms */ -if (dso->symtab_type ==DSO_BINARY_TYPE__KALLSYMS && !dso__is_kcore (dso)) { +if (dso__symtab_type (dso) ==DSO_BINARY_TYPE__KALLSYMS && !dso__is_kcore (dso)) {pr_err ("Can't annotate %s: No vmlinux file was found in the " "path\n" , sym->name);sleep (1 );
@@ -182,7 +182,7 @@ static void ui__warn_map_erange(struct map *map, struct symbol *sym, u64 ip)"Tools: %s\n\n" "Not all samples will be on the annotation output.\n\n" "Please report to [email protected]\n" , - ip, dso->long_name,dso__symtab_origin (dso), + ip,dso__long_name (dso),dso__symtab_origin (dso),map__start (map),map__end (map), sym->start, sym->end, sym->binding ==STB_GLOBAL ?'g' : sym->binding == STB_LOCAL ? 'l' : 'w', sym->name,
tools/perf/builtin-trace.c +1 -1
@@ -2903,7 +2903,7 @@ static void print_location(FILE *f, struct perf_sample *sample, {if ((verbose >0 || print_dso) && al->map) -fprintf (f,"%s@" ,map__dso (al->map)->long_name); +fprintf (f,"%s@" ,dso__long_name (map__dso (al->map)));if ((verbose >0 || print_sym) && al->sym) fprintf(f, "%s+0x%" PRIx64, al->sym->name,
tools/perf/tests/code-reading.c +4 -4
@@ -253,9 +253,9 @@ static int read_object_code(u64 addr, size_t len, u8 cpumode, goto out; } dso =map__dso (al.map); -pr_debug ("File is: %s\n" , dso->long_name); +pr_debug ("File is: %s\n" ,dso__long_name (dso)); -if (dso->symtab_type ==DSO_BINARY_TYPE__KALLSYMS && !dso__is_kcore (dso)) { +if (dso__symtab_type (dso) ==DSO_BINARY_TYPE__KALLSYMS && !dso__is_kcore (dso)) {pr_debug ("Unexpected kernel address - skipping\n" ); goto out; }
@@ -274,7 +274,7 @@ static int read_object_code(u64 addr, size_t len, u8 cpumode,* modules to manage long jumps. Check if the ip offset falls in stubs * sections for kernel modules. And skip module address after text end */ -if (dso->is_kmod && al.addr > dso->text_end) { +if (dso__is_kmod (dso) && al.addr >dso__text_end (dso)) {pr_debug ("skipping the module address %#" PRIx64" after text end\n" , al.addr); goto out; }
@@ -315,7 +315,7 @@ static int read_object_code(u64 addr, size_t len, u8 cpumode, state->done[state->done_cnt++] =map__start (al.map); } - objdump_name = dso->long_name; + objdump_name =dso__long_name (dso);if (dso__needs_decompress (dso)) { if (dso__decompress_kmodule_path(dso, objdump_name, decomp_name,
tools/perf/tests/dso-data.c +6 -5
@@ -228,7 +228,8 @@ static void dsos__delete(int cnt)for (i =0 ; i < cnt; i++) {struct dso *dso = dsos[i]; -unlink (dso->name); +dso__data_close (dso); +unlink (dso__name (dso));dso__put (dso); }
@@ -289,14 +290,14 @@ static int test__dso_data_cache(struct test_suite *test __maybe_unused, int subt }/* verify the first one is already open */ -TEST_ASSERT_VAL ("dsos[0] is not open" , dsos[0 ]->data.fd !=-1 ); +TEST_ASSERT_VAL ("dsos[0] is not open" ,dso__data (dsos[0 ])->fd !=-1 );/* open +1 dso to reach the allowed limit */ fd =dso__data_fd (dsos[i], &machine);TEST_ASSERT_VAL ("failed to get fd" , fd >0 );/* should force the first one to be closed */ -TEST_ASSERT_VAL ("failed to close dsos[0]" , dsos[0 ]->data.fd ==-1 ); +TEST_ASSERT_VAL ("failed to close dsos[0]" ,dso__data (dsos[0 ])->fd ==-1 );/* cleanup everything */ dsos__delete (dso_cnt);
@@ -371,7 +372,7 @@ static int test__dso_data_reopen(struct test_suite *test __maybe_unused, int sub* dso_0 should get closed, because we reached * the file descriptor limit */ -TEST_ASSERT_VAL ("failed to close dso_0" , dso_0->data.fd ==-1 ); +TEST_ASSERT_VAL ("failed to close dso_0" ,dso__data (dso_0)->fd ==-1 );/* open dso_0 */ fd =dso__data_fd (dso_0, &machine);
@@ -381,7 +382,7 @@ static int test__dso_data_reopen(struct test_suite *test __maybe_unused, int sub* dso_1 should get closed, because we reached * the file descriptor limit */ -TEST_ASSERT_VAL ("failed to close dso_1" , dso_1->data.fd ==-1 ); +TEST_ASSERT_VAL ("failed to close dso_1" ,dso__data (dso_1)->fd ==-1 );/* cleanup everything */ close (fd_extra);
tools/perf/tests/hists_common.c +3 -3
@@ -146,7 +146,7 @@ struct machine *setup_fake_machine(struct machines *machines) goto out; } -symbols__insert (&dso->symbols, sym); +symbols__insert (dso__symbols (dso), sym); }dso__put (dso);
@@ -183,7 +183,7 @@ void print_hists_in(struct hists *hists)pr_info ("%2d: entry: %-8s [%-8s] %20s: period = %" PRIu64"\n" , i,thread__comm_str (he->thread), - dso->short_name, +dso__short_name (dso), he->ms.sym->name, he->stat.period); }
@@ -212,7 +212,7 @@ void print_hists_out(struct hists *hists)pr_info ("%2d: entry: %8s:%5d [%-8s] %20s: period = %" PRIu64"/%" PRIu64"\n" , i,thread__comm_str (he->thread),thread__tid (he->thread), - dso->short_name, +dso__short_name (dso), he->ms.sym->name, he->stat.period, he->stat_acc ? he->stat_acc->period :0 ); }
tools/perf/tests/hists_cumulate.c +2 -2
@@ -164,11 +164,11 @@ static void put_fake_samples(void)typedef int (*test_fn_t )(struct evsel *,struct machine *);#define COMM (he) (thread__comm_str(he->thread)) -#define DSO (he) (map__dso(he->ms.map)->short_name) +#define DSO (he) (dso__short_name(map__dso(he->ms.map)))#define SYM (he) (he->ms.sym->name)#define CPU (he) (he->cpu)#define DEPTH (he) (he->callchain->max_depth) -#define CDSO (cl) (map__dso(cl->ms.map)->short_name) +#define CDSO (cl) (dso__short_name(map__dso(cl->ms.map)))#define CSYM (cl) (cl->ms.sym->name)struct result {
tools/perf/tests/hists_output.c +1 -1
@@ -129,7 +129,7 @@ static void put_fake_samples(void)typedef int (*test_fn_t )(struct evsel *,struct machine *);#define COMM (he) (thread__comm_str(he->thread)) -#define DSO (he) (map__dso(he->ms.map)->short_name) +#define DSO (he) (dso__short_name(map__dso(he->ms.map)))#define SYM (he) (he->ms.sym->name)#define CPU (he) (he->cpu)#define PID (he) (thread__tid(he->thread))
tools/perf/tests/maps.c +2 -2
@@ -26,7 +26,7 @@ static int check_maps_cb(struct map *map, void *data)if (map__start (map) != merged->start ||map__end (map) != merged->end || -strcmp (map__dso (map)->name, merged->name) || +strcmp (dso__name (map__dso (map)), merged->name) ||refcount_read (map__refcnt (map)) !=1 ) {return 1 ; }
@@ -39,7 +39,7 @@ static int failed_cb(struct map *map, void *data __maybe_unused)pr_debug ("\tstart: %" PRIu64" end: %" PRIu64" name: '%s' refcnt: %d\n" ,map__start (map),map__end (map), -map__dso (map)->name, +dso__name (map__dso (map)),refcount_read (map__refcnt (map)));return 0 ;
tools/perf/tests/symbols.c +1 -1
@@ -72,7 +72,7 @@ static int test_dso(struct dso *dso)if (verbose >1 )dso__fprintf (dso, stderr); -for (nd =rb_first_cached (&dso->symbols); nd; nd =rb_next (nd)) { +for (nd =rb_first_cached (dso__symbols (dso)); nd; nd =rb_next (nd)) { struct symbol *sym = rb_entry(nd, struct symbol, rb_node);if (sym->type !=STT_FUNC && sym->type !=STT_GNU_IFUNC )
tools/perf/tests/vmlinux-kallsyms.c +3 -3
@@ -129,7 +129,7 @@ static int test__vmlinux_matches_kallsyms_cb1(struct map *map, void *data)* cases. */ struct map *pair =maps__find_by_name (args->kallsyms.kmaps, - (dso->kernel ? dso->short_name : dso->name)); + (dso__kernel (dso) ?dso__short_name (dso) :dso__name (dso)));if (pair) {map__set_priv (pair,1 );
@@ -162,11 +162,11 @@ static int test__vmlinux_matches_kallsyms_cb2(struct map *map, void *data) }pr_info ("WARN: %" PRIx64"-%" PRIx64" %" PRIx64" %s in kallsyms as" , -map__start (map),map__end (map),map__pgoff (map), dso->name); +map__start (map),map__end (map),map__pgoff (map),dso__name (dso));if (mem_end !=map__end (pair))pr_info (":\nWARN: *%" PRIx64"-%" PRIx64" %" PRIx64,map__start (pair),map__end (pair),map__pgoff (pair)); -pr_info (" %s\n" , dso->name); +pr_info (" %s\n" ,dso__name (dso));map__set_priv (pair,1 ); }map__put (pair);
tools/perf/ui/browsers/annotate.c +3 -3
@@ -438,7 +438,7 @@ static int sym_title(struct symbol *sym, struct map *map, char *title, size_t sz, int percent_type) {return snprintf (title, sz,"%s %s [Percent: %s]" , sym->name, -map__dso (map)->long_name, +dso__long_name (map__dso (map)),percent_type_str (percent_type)); }
@@ -966,14 +966,14 @@ int symbol__tui_annotate(struct map_symbol *ms, struct evsel *evsel,return -1 ; dso =map__dso (ms->map); -if (dso->annotate_warned) +if (dso__annotate_warned (dso))return -1 ;if (not_annotated) { err =symbol__annotate2 (ms, evsel, &browser.arch);if (err) {char msg[BUFSIZ ]; - dso->annotate_warned = true; +dso__set_annotate_warned (dso);symbol__strerror_disassemble (ms, err, msg,sizeof (msg));ui__error ("Couldn't annotate %s:\n%s" , sym->name, msg); goto out_free_offsets;
tools/perf/ui/browsers/hists.c +4 -4
@@ -2488,7 +2488,7 @@ add_annotate_opt(struct hist_browser *browser __maybe_unused, {struct dso *dso; -if (!ms->map || (dso =map__dso (ms->map)) ==NULL || dso->annotate_warned) +if (!ms->map || (dso =map__dso (ms->map)) ==NULL ||dso__annotate_warned (dso))return 0 ;if (!ms->sym)
@@ -2581,7 +2581,7 @@ static int hists_browser__zoom_map(struct hist_browser *browser, struct map *map }else {struct dso *dso =map__dso (map);ui_helpline__fpush ("To zoom out press ESC or ENTER + \"Zoom out of %s DSO\"" , -__map__is_kernel (map) ?"the Kernel" : dso->short_name); +__map__is_kernel (map) ?"the Kernel" :dso__short_name (dso)); browser->hists->dso_filter = dso;perf_hpp__set_elide (HISTC_DSO , true);pstack__push (browser->pstack, &browser->hists->dso_filter);
@@ -2607,7 +2607,7 @@ add_dso_opt(struct hist_browser *browser, struct popup_action *act,if (asprintf (optstr,"Zoom %s %s DSO (use the 'k' hotkey to zoom directly into the kernel)" , browser->hists->dso_filter ?"out of" :"into" , -__map__is_kernel (map) ?"the Kernel" :map__dso (map)->short_name) <0 ) +__map__is_kernel (map) ?"the Kernel" :dso__short_name (map__dso (map))) <0 )return 0 ; act->ms.map = map;
@@ -3082,7 +3082,7 @@ static int evsel__hists_browse(struct evsel *evsel, int nr_events, const char *hif (!browser->selection || !browser->selection->map || !map__dso (browser->selection->map) || -map__dso (browser->selection->map)->annotate_warned) { +dso__annotate_warned (map__dso (browser->selection->map))) {continue ; }
tools/perf/ui/browsers/map.c +2 -2
@@ -76,7 +76,7 @@ static int map_browser__run(struct map_browser *browser) {int key; -if (ui_browser__show (&browser->b,map__dso (browser->map)->long_name, +if (ui_browser__show (&browser->b,dso__long_name (map__dso (browser->map)),"Press ESC to exit, %s / to search" , verbose >0 ?"" :"restart with -v to use" ) <0 )return -1 ;
@@ -106,7 +106,7 @@ int map__browse(struct map *map) {struct map_browser mb = { .b = { - .entries = &map__dso (map)->symbols, + .entries =dso__symbols (map__dso (map)), .refresh = ui_browser__rb_tree_refresh, .seek = ui_browser__rb_tree_seek, .write = map_browser__write,
tools/perf/util/annotate-data.c +3 -3
@@ -141,7 +141,7 @@ static struct annotated_data_type *dso__findnew_data_type(struct dso *dso,/* Check existing nodes in dso->data_types tree */ key.self.type_name = type_name; key.self.size = size; - node =rb_find (&key, &dso->data_types, data_type_cmp); + node =rb_find (&key,dso__data_types (dso), data_type_cmp);if (node) { result = rb_entry(node, struct annotated_data_type, node);free (type_name);
@@ -162,7 +162,7 @@ static struct annotated_data_type *dso__findnew_data_type(struct dso *dso,if (symbol_conf.annotate_data_member)add_member_types (result, type_die); -rb_add (&result->node, &dso->data_types, data_type_less); +rb_add (&result->node,dso__data_types (dso), data_type_less);return result; }
@@ -288,7 +288,7 @@ struct annotated_data_type *find_data_type(struct map_symbol *ms, u64 ip,Dwarf_Die type_die;u64 pc; - di =debuginfo__new (dso->long_name); + di =debuginfo__new (dso__long_name (dso));if (di ==NULL ) {pr_debug ("cannot get the debug info\n" );return NULL ;
tools/perf/util/annotate.c +22 -23
134 lines not on this page · raw message
tools/perf/util/auxtrace.c +1 -1
@@ -2652,7 +2652,7 @@ static int addr_filter__entire_dso(struct addr_filter *filt, struct dso *dso) } filt->addr =0 ; - filt->size = dso->data.file_size; + filt->size =dso__data (dso)->file_size;return 0 ; }
tools/perf/util/block-info.c +1 -1
@@ -319,7 +319,7 @@ static int block_dso_entry(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,if (map &&map__dso (map)) {return scnprintf (hpp->buf, hpp->size,"%*s" , block_fmt->width, -map__dso (map)->short_name); +dso__short_name (map__dso (map))); } return scnprintf(hpp->buf, hpp->size, "%*s", block_fmt->width,
tools/perf/util/bpf-event.c +4 -4
@@ -59,10 +59,10 @@ static int machine__process_bpf_event_load(struct machine *machine,if (map) {struct dso *dso =map__dso (map); - dso->binary_type =DSO_BINARY_TYPE__BPF_PROG_INFO ; - dso->bpf_prog.id = id; - dso->bpf_prog.sub_id = i; - dso->bpf_prog.env = env; +dso__set_binary_type (dso,DSO_BINARY_TYPE__BPF_PROG_INFO ); +dso__bpf_prog (dso)->id = id; +dso__bpf_prog (dso)->sub_id = i; +dso__bpf_prog (dso)->env = env;map__put (map); } }
tools/perf/util/build-id.c +19 -19
@@ -60,7 +60,7 @@ int build_id__mark_dso_hit(struct perf_tool *tool __maybe_unused,addr_location__init (&al);if (thread__find_map (thread, sample->cpumode, sample->ip, &al)) -map__dso (al.map)->hit =1 ; +dso__set_hit (map__dso (al.map));addr_location__exit (&al);thread__put (thread);
@@ -272,10 +272,10 @@ char *__dso__build_id_filename(const struct dso *dso, char *bf, size_t size,bool alloc = (bf ==NULL );int ret; -if (!dso->has_build_id) +if (!dso__has_build_id (dso))return NULL ; -build_id__sprintf (&dso->bid, sbuild_id); +build_id__sprintf (dso__bid_const (dso), sbuild_id); linkname =build_id_cache__linkname (sbuild_id,NULL ,0 );if (!linkname)return NULL ;
@@ -340,25 +340,25 @@ static int machine__write_buildid_table_cb(struct dso *dso, void *data)size_t name_len;bool in_kernel = false; -if (!dso->has_build_id) +if (!dso__has_build_id (dso))return 0 ; -if (!dso->hit && !dso__is_vdso (dso)) +if (!dso__hit (dso) && !dso__is_vdso (dso))return 0 ;if (dso__is_vdso (dso)) { - name = dso->short_name; - name_len = dso->short_name_len; + name =dso__short_name (dso); + name_len =dso__short_name_len (dso); }else if (dso__is_kcore (dso)) { name = args->machine->mmap_name; name_len =strlen (name); }else { - name = dso->long_name; - name_len = dso->long_name_len; + name =dso__long_name (dso); + name_len =dso__long_name_len (dso); } - in_kernel = dso->kernel ||is_kernel_module (name,PERF_RECORD_MISC_CPUMODE_UNKNOWN ); -return write_buildid (name, name_len, &dso->bid, args->machine->pid, + in_kernel =dso__kernel (dso) ||is_kernel_module (name,PERF_RECORD_MISC_CPUMODE_UNKNOWN ); +return write_buildid (name, name_len,dso__bid (dso), args->machine->pid, in_kernel ? args->kmisc : args->umisc, args->fd); }
@@ -876,11 +876,11 @@ static bool dso__build_id_mismatch(struct dso *dso, const char *name)struct build_id bid;bool ret = false; -mutex_lock (&dso->lock); -if (filename__read_build_id_ns (name, &bid, dso->nsinfo) >=0 ) +mutex_lock (dso__lock (dso)); +if (filename__read_build_id_ns (name, &bid,dso__nsinfo (dso)) >=0 ) ret = !dso__build_id_equal (dso, &bid); -mutex_unlock (&dso->lock); +mutex_unlock (dso__lock (dso));return ret; }
@@ -890,13 +890,13 @@ static int dso__cache_build_id(struct dso *dso, struct machine *machine, {bool is_kallsyms =dso__is_kallsyms (dso);bool is_vdso =dso__is_vdso (dso); -const char *name = dso->long_name; +const char *name =dso__long_name (dso);const char *proper_name =NULL ;const char *root_dir =NULL ;char *allocated_name =NULL ;int ret =0 ; -if (!dso->has_build_id) +if (!dso__has_build_id (dso))return 0 ;if (dso__is_kcore (dso)) {
@@ -921,10 +921,10 @@ static int dso__cache_build_id(struct dso *dso, struct machine *machine,if (!is_kallsyms &&dso__build_id_mismatch (dso, name)) goto out_free; -mutex_lock (&dso->lock); - ret =build_id_cache__add_b (&dso->bid, name, dso->nsinfo, +mutex_lock (dso__lock (dso)); + ret =build_id_cache__add_b (dso__bid (dso), name,dso__nsinfo (dso), is_kallsyms, is_vdso, proper_name, root_dir); -mutex_unlock (&dso->lock); +mutex_unlock (dso__lock (dso)); out_free:free (allocated_name);return ret;
tools/perf/util/callchain.c +1 -1
@@ -1205,7 +1205,7 @@ char *callchain_list__sym_name(struct callchain_list *cl,if (show_dso)scnprintf (bf + printed, bfsize - printed," %s" , cl->ms.map ? -map__dso (cl->ms.map)->short_name : +dso__short_name (map__dso (cl->ms.map)) :"unknown" );return bf;
tools/perf/util/data-convert-json.c +1 -1
@@ -134,7 +134,7 @@ static void output_sample_callchain_entry(struct perf_tool *tool,output_json_key_string (out, false,5 ,"symbol" , al->sym->name);if (dso) { -const char *dso_name = dso->short_name; +const char *dso_name =dso__short_name (dso);if (dso_name &&strlen (dso_name) >0 ) {fputc (',' , out);
tools/perf/util/db-export.c +3 -3
@@ -146,10 +146,10 @@ int db_export__comm_thread(struct db_export *dbe, struct comm *comm,int db_export__dso (struct db_export *dbe,struct dso *dso,struct machine *machine) { -if (dso->db_id) +if (dso__db_id (dso))return 0 ; - dso->db_id = ++dbe->dso_last_db_id; +dso__set_db_id (dso, ++dbe->dso_last_db_id);if (dbe->export_dso)return dbe->export_dso (dbe, dso, machine);
@@ -184,7 +184,7 @@ static int db_ids_from_al(struct db_export *dbe, struct addr_location *al, err =db_export__dso (dbe, dso,maps__machine (al->maps));if (err)return err; - *dso_db_id = dso->db_id; + *dso_db_id =dso__db_id (dso);if (!al->sym) { al->sym =symbol__new (al->addr,0 ,0 ,0 ,"unknown" );
tools/perf/util/dlfilter.c +6 -6
@@ -33,13 +33,13 @@ static void al_to_d_al(struct addr_location *al, struct perf_dlfilter_al *d_al)if (al->map) {struct dso *dso =map__dso (al->map); -if (symbol_conf.show_kernel_path && dso->long_name) - d_al->dso = dso->long_name; +if (symbol_conf.show_kernel_path &&dso__long_name (dso)) + d_al->dso =dso__long_name (dso);else - d_al->dso = dso->name; - d_al->is_64_bit = dso->is_64_bit; - d_al->buildid_size = dso->bid.size; - d_al->buildid = dso->bid.data; + d_al->dso =dso__name (dso); + d_al->is_64_bit =dso__is_64_bit (dso); + d_al->buildid_size =dso__bid (dso)->size; + d_al->buildid =dso__bid (dso)->data; }else { d_al->dso =NULL ; d_al->is_64_bit =0 ;
tools/perf/util/dso.c +195 -170
876 lines not on this page · raw message
tools/perf/util/dso.h +437 -46
541 lines not on this page · raw message
tools/perf/util/dsos.c +27 -27
142 lines not on this page · raw message
tools/perf/util/event.c +4 -4
@@ -726,7 +726,7 @@ int machine__resolve(struct machine *machine, struct addr_location *al, dso = al->map ?map__dso (al->map) :NULL ;dump_printf (" ...... dso: %s\n" , dso - ? dso->long_name + ?dso__long_name (dso) : (al->level =='H' ?"[hypervisor]" :"<not found>" ));if (thread__is_filtered (thread))
@@ -750,10 +750,10 @@ int machine__resolve(struct machine *machine, struct addr_location *al,if (al->map) {if (symbol_conf.dso_list && (!dso || !(strlist__has_entry (symbol_conf.dso_list, - dso->short_name) || - (dso->short_name != dso->long_name && +dso__short_name (dso)) || + (dso__short_name (dso) !=dso__long_name (dso) &&strlist__has_entry (symbol_conf.dso_list, - dso->long_name))))) { +dso__long_name (dso)))))) { al->filtered |= (1 <<HIST_FILTER__DSO ); }
tools/perf/util/header.c +4 -4
@@ -2308,7 +2308,7 @@ static int __event_process_build_id(struct perf_record_header_build_id *bev,build_id__init (&bid, bev->data, size);dso__set_build_id (dso, &bid); - dso->header_build_id =1 ; +dso__set_header_build_id (dso, true);if (dso_space !=DSO_SPACE__USER ) {struct kmod_path m = { .name =NULL , };
@@ -2316,13 +2316,13 @@ static int __event_process_build_id(struct perf_record_header_build_id *bev,if (!kmod_path__parse_name (&m, filename) && m.kmod)dso__set_module_info (dso, &m, machine); - dso->kernel = dso_space; +dso__set_kernel (dso, dso_space);free (m.name); } -build_id__sprintf (&dso->bid, sbuild_id); +build_id__sprintf (dso__bid (dso), sbuild_id);pr_debug ("build id event received for %s: %s [%zu]\n" , - dso->long_name, sbuild_id, size); +dso__long_name (dso), sbuild_id, size);dso__put (dso); }
tools/perf/util/hist.c +2 -2
@@ -2128,7 +2128,7 @@ static bool hists__filter_entry_by_dso(struct hists *hists, struct hist_entry *he) {if (hists->dso_filter !=NULL && - (he->ms.map ==NULL ||map__dso (he->ms.map) != hists->dso_filter)) { + (he->ms.map ==NULL || !RC_CHK_EQUAL (map__dso (he->ms.map), hists->dso_filter))) { he->filtered |= (1 <<HIST_FILTER__DSO );return true; }
@@ -2808,7 +2808,7 @@ int __hists__scnprintf_title(struct hists *hists, char *bf, size_t size, bool sh }if (dso) printed +=scnprintf (bf + printed, size - printed, -", DSO: %s" , dso->short_name); +", DSO: %s" ,dso__short_name (dso));if (socket_id >-1 ) printed +=scnprintf (bf + printed, size - printed,", Processor Socket: %d" , socket_id);
tools/perf/util/intel-pt.c +11 -11
@@ -598,15 +598,15 @@ static struct auxtrace_cache *intel_pt_cache(struct dso *dso,struct auxtrace_cache *c;unsigned int bits; -if (dso->auxtrace_cache) -return dso->auxtrace_cache; +if (dso__auxtrace_cache (dso)) +return dso__auxtrace_cache (dso); bits =intel_pt_cache_size (dso, machine);/* Ignoring cache creation failure */ c =auxtrace_cache__new (bits,sizeof (struct intel_pt_cache_entry ),200 ); - dso->auxtrace_cache = c; +dso__set_auxtrace_cache (dso, c);return c; }
@@ -650,7 +650,7 @@ intel_pt_cache_lookup(struct dso *dso, struct machine *machine, u64 offset)if (!c)return NULL ; -return auxtrace_cache__lookup (dso->auxtrace_cache, offset); +return auxtrace_cache__lookup (dso__auxtrace_cache (dso), offset); } static void intel_pt_cache_invalidate(struct dso *dso, struct machine *machine,
@@ -661,7 +661,7 @@ static void intel_pt_cache_invalidate(struct dso *dso, struct machine *machine,if (!c)return ; -auxtrace_cache__remove (dso->auxtrace_cache, offset); +auxtrace_cache__remove (dso__auxtrace_cache (dso), offset); }static inline bool intel_pt_guest_kernel_ip (uint64_t ip)
@@ -820,8 +820,8 @@ static int intel_pt_walk_next_insn(struct intel_pt_insn *intel_pt_insn, } dso =map__dso (al.map); -if (dso->data.status ==DSO_DATA_STATUS_ERROR && -dso__data_status_seen (dso,DSO_DATA_STATUS_SEEN_ITRACE )) { +if (dso__data (dso)->status ==DSO_DATA_STATUS_ERROR && +dso__data_status_seen (dso,DSO_DATA_STATUS_SEEN_ITRACE )) { ret = -ENOENT ; goto out_ret; }
@@ -854,7 +854,7 @@ static int intel_pt_walk_next_insn(struct intel_pt_insn *intel_pt_insn,/* Load maps to ensure dso->is_64_bit has been updated */ map__load (al.map); - x86_64 = dso->is_64_bit; + x86_64 =dso__is_64_bit (dso);while (1 ) { len = dso__data_read_offset(dso, machine,
@@ -1008,7 +1008,7 @@ static int __intel_pt_pgd_ip(uint64_t ip, void *data) offset =map__map_ip (al.map, ip); - res =intel_pt_match_pgd_ip (ptq->pt, ip, offset,map__dso (al.map)->long_name); + res =intel_pt_match_pgd_ip (ptq->pt, ip, offset,dso__long_name (map__dso (al.map)));addr_location__exit (&al);return res; }
@@ -3416,7 +3416,7 @@ static int intel_pt_text_poke(struct intel_pt *pt, union perf_event *event) } dso =map__dso (al.map); -if (!dso || !dso->auxtrace_cache) +if (!dso || !dso__auxtrace_cache (dso))continue ; offset =map__map_ip (al.map, addr);
@@ -3436,7 +3436,7 @@ static int intel_pt_text_poke(struct intel_pt *pt, union perf_event *event) }else {intel_pt_cache_invalidate (dso, machine, offset);intel_pt_log ("Invalidated instruction cache for %s at %#" PRIx64"\n" , - dso->long_name, addr); +dso__long_name (dso), addr); } } out:
tools/perf/util/machine.c +23 -23
131 lines not on this page · raw message
tools/perf/util/map.c +34 -35
213 lines not on this page · raw message
tools/perf/util/maps.c +7 -7
@@ -76,7 +76,7 @@ static void check_invariants(const struct maps *maps __maybe_unused)/* Expect at least 1 reference count. */ assert (refcount_read (map__refcnt (map)) >0 ); -if (map__dso (map) &&map__dso (map)->kernel) +if (map__dso (map) &&dso__kernel (map__dso (map)))assert (RC_CHK_EQUAL (map__kmap (map)->kmaps, maps));if (i >0 ) {
@@ -331,7 +331,7 @@ static int map__strcmp(const void *a, const void *b)const struct map *map_b = *(const struct map *const *)b;const struct dso *dso_a =map__dso (map_a);const struct dso *dso_b =map__dso (map_b); -int ret =strcmp (dso_a->short_name, dso_b->short_name); +int ret =strcmp (dso__short_name (dso_a),dso__short_name (dso_b));if (ret ==0 &&RC_CHK_ACCESS (map_a) !=RC_CHK_ACCESS (map_b)) {/* Ensure distinct but name equal maps have an order. */
@@ -470,7 +470,7 @@ static int __maps__insert(struct maps *maps, struct map *new) }if (map__end (new) <map__start (new))RC_CHK_ACCESS (maps)->ends_broken = true; -if (dso && dso->kernel) { +if (dso &&dso__kernel (dso)) {struct kmap *kmap =map__kmap (new);if (kmap)
@@ -747,7 +747,7 @@ static int __maps__fixup_overlap_and_insert(struct maps *maps, struct map *new)if (use_browser) {pr_debug ("overlapping maps in %s (disable tui for more info)\n" , -map__dso (new)->name); +dso__name (map__dso (new))); }else if (verbose >=2 ) {pr_debug ("overlapping maps:\n" );map__fprintf (new, fp);
@@ -968,7 +968,7 @@ static int map__strcmp_name(const void *name, const void *b) {const struct dso *dso =map__dso (*(const struct map **)b); -return strcmp (name, dso->short_name); +return strcmp (name,dso__short_name (dso)); }struct map *maps__find_by_name (struct maps *maps,const char *name)
@@ -987,7 +987,7 @@ struct map *maps__find_by_name(struct maps *maps, const char *name)if (i <maps__nr_maps (maps) &&maps__maps_by_name (maps)) {struct dso *dso =map__dso (maps__maps_by_name (maps)[i]); -if (dso &&strcmp (dso->short_name, name) ==0 ) { +if (dso &&strcmp (dso__short_name (dso), name) ==0 ) { result =map__get (maps__maps_by_name (maps)[i]); done = true; }
@@ -1024,7 +1024,7 @@ struct map *maps__find_by_name(struct maps *maps, const char *name)struct map *pos = maps_by_address[i];struct dso *dso =map__dso (pos); -if (dso &&strcmp (dso->short_name, name) ==0 ) {
Lines of this file are not shown. View the raw message.
tools/perf/util/scripting-engines/trace-event-perl.c +3 -3
Lines of this file are not shown. View the raw message.
tools/perf/util/scripting-engines/trace-event-python.c +11 -10
Lines of this file are not shown. View the raw message.
tools/perf/util/srcline.c +34 -31
65 lines not on this page · raw message
tools/perf/util/symbol-elf.c +46 -46
92 lines not on this page · raw message
tools/perf/util/symbol.c +95 -91
186 lines not on this page · raw message
tools/perf/util/unwind-libunwind-local.c +9 -9
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Part of the diff is not shown here. View the raw message.
Ian Rogers
reviewerPATCH v7 24/25 perf dso: Reference counting related fixes
In reply to Ian RogersEnsure gets and puts are better aligned fixing reference couting checking problems.
--- tools/perf/util/machine.c | 4 ++-- tools/perf/util/map.c | 1 + tools/perf/util/symbol-elf.c | 38 +++++++++++++++++------------------- 3 files changed, 21 insertions(+), 22 deletions(-)
tools/perf/util/machine.c +2 -2
@@ -682,7 +682,7 @@ static int machine__process_ksymbol_register(struct machine *machine, struct perf_sample *sample __maybe_unused) {struct symbol *sym; -struct dso *dso; +struct dso *dso =NULL ;struct map *map =maps__find (machine__kernel_maps (machine), event->ksymbol.addr);int err =0 ;
@@ -695,7 +695,6 @@ static int machine__process_ksymbol_register(struct machine *machine, }dso__set_kernel (dso,DSO_SPACE__KERNEL ); map =map__new2 (0 , dso); -dso__put (dso);if (!map) { err = -ENOMEM ; goto out;
@@ -734,6 +733,7 @@ static int machine__process_ksymbol_register(struct machine *machine,dso__insert_symbol (dso, sym); out:map__put (map); +dso__put (dso);return err; }
tools/perf/util/map.c +1 -0
@@ -200,6 +200,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,dso__set_build_id (dso,dso__bid (header_bid_dso));dso__set_header_build_id (dso,1 ); } +dso__put (header_bid_dso); }dso__put (dso); }
tools/perf/util/symbol-elf.c +18 -20
@@ -1366,7 +1366,7 @@ void __weak arch__sym_update(struct symbol *s __maybe_unused,static int dso__process_kernel_symbol (struct dso *dso,struct map *map,GElf_Sym *sym,GElf_Shdr *shdr,struct maps *kmaps,struct kmap *kmap, -struct dso **curr_dsop,struct map **curr_mapp, +struct dso **curr_dsop,const char *section_name,bool adjust_kernel_syms,bool kmodule,bool *remap_kernel) {
@@ -1416,8 +1416,8 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,map__set_pgoff (map, shdr->sh_offset); } - *curr_mapp = map; - *curr_dsop = dso; +dso__put (*curr_dsop); + *curr_dsop =dso__get (dso);return 0 ; }
@@ -1442,10 +1442,10 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,dso__set_binary_type (curr_dso,dso__binary_type (dso));dso__set_adjust_symbols (curr_dso,dso__adjust_symbols (dso)); curr_map =map__new2 (start, curr_dso); -dso__put (curr_dso); -if (curr_map ==NULL ) +if (curr_map ==NULL ) { +dso__put (curr_dso);return -1 ; - + }if (dso__kernel (curr_dso))map__kmap (curr_map)->kmaps = kmaps;
@@ -1459,21 +1459,15 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,dso__set_symtab_type (curr_dso,dso__symtab_type (dso));if (maps__insert (kmaps, curr_map))return -1 ; -/* -* Add it before we drop the reference to curr_map, i.e. while -* we still are sure to have a reference to this DSO via -* *curr_map->dso. -*/ dsos__add (&maps__machine (kmaps)->dsos, curr_dso); -/* kmaps already got it */ -map__put (curr_map);dso__set_loaded (curr_dso); - *curr_mapp = curr_map; +dso__put (*curr_dsop); *curr_dsop = curr_dso; }else { - *curr_dsop =map__dso (curr_map); -map__put (curr_map); +dso__put (*curr_dsop); + *curr_dsop =dso__get (map__dso (curr_map)); } +map__put (curr_map);return 0 ; }
@@ -1484,8 +1478,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss, {struct kmap *kmap =dso__kernel (dso) ?map__kmap (map) :NULL ;struct maps *kmaps = kmap ?map__kmaps (map) :NULL ; -struct map *curr_map = map; -struct dso *curr_dso = dso; +struct dso *curr_dso;Elf_Data *symstrs, *secstrs, *secstrs_run, *secstrs_sym;uint32_t nr_syms;int err =-1 ;
@@ -1586,6 +1579,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss, remap_kernel = true; adjust_kernel_syms =dso__adjust_symbols (dso); } + curr_dso =dso__get (dso);elf_symtab__for_each_symbol (syms, nr_syms, idx, sym) {struct symbol *f;const char *elf_name =elf_sym__name (&sym, symstrs);
@@ -1674,8 +1668,11 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss, --sym.st_value;if (dso__kernel (dso)) { -if (dso__process_kernel_symbol (dso, map, &sym, &shdr, kmaps, kmap, &curr_dso, &curr_map, - section_name, adjust_kernel_syms, kmodule, &remap_kernel)) +if (dso__process_kernel_symbol (dso, map, &sym, &shdr, + kmaps, kmap, &curr_dso, + section_name, + adjust_kernel_syms, + kmodule, &remap_kernel)) goto out_elf_end; }else if ((used_opd && runtime_ss->adjust_symbols) || (!used_opd && syms_ss->adjust_symbols)) {
@@ -1724,6 +1721,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,__symbols__insert (dso__symbols (curr_dso), f,dso__kernel (dso)); nr++; } +dso__put (curr_dso);/* * For misannotated, zeroed, ASM function sizes.
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerPATCH v7 25/25 perf dso: Use container_of to avoid a pointer in dso_data
In reply to Ian RogersThe dso pointer in dso_data is necessary for reference count checking to account for the dso_data forming a global list of open dso's with references to the dso. The dso pointer also allows for the indirection that reference count checking needs. Outside of reference count checking the indirection isn't needed and container_of is more efficient and saves space.
The reference count won't be increased by placing items onto the global list, matching how things were before the reference count checking change, but we assert the dso is in dsos holding it live (and that the set of open dsos is a subset of all dsos for the machine). Update the DSO data tests so that they use a dsos struct to make the invariant true.
--- tools/perf/tests/dso-data.c | 60 ++++++++++++++++++------------------- tools/perf/util/dso.c | 16 +++++++++- tools/perf/util/dso.h | 2 ++ 3 files changed, 46 insertions(+), 32 deletions(-)
tools/perf/tests/dso-data.c +29 -31
@@ -10,6 +10,7 @@#include <sys/resource.h> #include <api/fs/fs.h> #include "dso.h" +#include "dsos.h" #include "machine.h" #include "symbol.h" #include "tests.h"
@@ -123,9 +124,10 @@ static int test__dso_data(struct test_suite *test __maybe_unused, int subtest __TEST_ASSERT_VAL ("No test file" , file);memset (&machine,0 ,sizeof (machine)); +dsos__init (&machine.dsos); - dso =dso__new ((const char *)file); - + dso =dso__new (file); +TEST_ASSERT_VAL ("Failed to add dso" , !dsos__add (&machine.dsos, dso));TEST_ASSERT_VAL ("Failed to access to dso" ,dso__data_fd (dso, &machine) >=0 );
@@ -170,6 +172,7 @@ static int test__dso_data(struct test_suite *test __maybe_unused, int subtest __ }dso__put (dso); +dsos__exit (&machine.dsos);unlink (file);return 0 ; }
@@ -199,41 +202,35 @@ static long open_files_cnt(void)return nr -1 ; } -static struct dso **dsos; - -static int dsos__create (int cnt,int size) +static int dsos__create (int cnt,int size,struct dsos *dsos) {int i; - dsos =malloc (sizeof (*dsos) * cnt); -TEST_ASSERT_VAL ("failed to alloc dsos array" , dsos); +dsos__init (dsos);for (i =0 ; i < cnt; i++) { -char *file; +struct dso *dso; +char *file =test_file (size); - file =test_file (size);TEST_ASSERT_VAL ("failed to get dso file" , file); - - dsos[i] =dso__new (file); -TEST_ASSERT_VAL ("failed to get dso" , dsos[i]); + dso =dso__new (file); +TEST_ASSERT_VAL ("failed to get dso" , dso); +TEST_ASSERT_VAL ("failed to add dso" , !dsos__add (dsos, dso)); +dso__put (dso); }return 0 ; } -static void dsos__delete (int cnt) +static void dsos__delete (struct dsos *dsos) { -int i; - -for (i =0 ; i < cnt; i++) { -struct dso *dso = dsos[i]; +for (unsigned int i =0 ; i < dsos->cnt; i++) { +struct dso *dso = dsos->dsos[i];dso__data_close (dso);unlink (dso__name (dso)); -dso__put (dso); } - -free (dsos); +dsos__exit (dsos); }static int set_fd_limit (int n)
@@ -267,10 +264,10 @@ static int test__dso_data_cache(struct test_suite *test __maybe_unused, int subt/* and this is now our dso open FDs limit */ dso_cnt = limit /2 ;TEST_ASSERT_VAL ("failed to create dsos\n" , - !dsos__create (dso_cnt,TEST_FILE_SIZE )); + !dsos__create (dso_cnt,TEST_FILE_SIZE , &machine.dsos));for (i =0 ; i < (dso_cnt -1 ); i++) { -struct dso *dso = dsos[i]; +struct dso *dso = machine.dsos.dsos[i];/* * Open dsos via dso__data_fd(), it opens the data
@@ -290,17 +287,17 @@ static int test__dso_data_cache(struct test_suite *test __maybe_unused, int subt }/* verify the first one is already open */ -TEST_ASSERT_VAL ("dsos[0] is not open" ,dso__data (dsos[0 ])->fd !=-1 ); +TEST_ASSERT_VAL ("dsos[0] is not open" ,dso__data (machine.dsos.dsos[0 ])->fd !=-1 );/* open +1 dso to reach the allowed limit */ - fd =dso__data_fd (dsos[i], &machine); + fd =dso__data_fd (machine.dsos.dsos[i], &machine);TEST_ASSERT_VAL ("failed to get fd" , fd >0 );/* should force the first one to be closed */ -TEST_ASSERT_VAL ("failed to close dsos[0]" ,dso__data (dsos[0 ])->fd ==-1 ); +TEST_ASSERT_VAL ("failed to close dsos[0]" ,dso__data (machine.dsos.dsos[0 ])->fd ==-1 );/* cleanup everything */ -dsos__delete (dso_cnt); +dsos__delete (&machine.dsos);/* Make sure we did not leak any file descriptor. */ nr_end =open_files_cnt ();
@@ -325,9 +322,9 @@ static int test__dso_data_reopen(struct test_suite *test __maybe_unused, int sublong nr_end, nr =open_files_cnt (), lim =new_limit (3 );int fd, fd_extra; -#define dso_0 (dsos[0]) -#define dso_1 (dsos[1]) -#define dso_2 (dsos[2]) +#define dso_0 (machine.dsos.dsos[0]) +#define dso_1 (machine.dsos.dsos[1]) +#define dso_2 (machine.dsos.dsos[2])/* Rest the internal dso open counter limit. */ reset_fd_limit ();
@@ -347,7 +344,8 @@ static int test__dso_data_reopen(struct test_suite *test __maybe_unused, int subTEST_ASSERT_VAL ("failed to set file limit" , !set_fd_limit ((lim))); -TEST_ASSERT_VAL ("failed to create dsos\n" , !dsos__create (3 ,TEST_FILE_SIZE )); +TEST_ASSERT_VAL ("failed to create dsos\n" , + !dsos__create (3 ,TEST_FILE_SIZE , &machine.dsos));/* open dso_0 */ fd =dso__data_fd (dso_0, &machine);
@@ -386,7 +384,7 @@ static int test__dso_data_reopen(struct test_suite *test __maybe_unused, int sub/* cleanup everything */ close (fd_extra); -dsos__delete (3 ); +dsos__delete (&machine.dsos);/* Make sure we did not leak any file descriptor. */ nr_end =open_files_cnt ();
tools/perf/util/dso.c +15 -1
@@ -497,14 +497,20 @@ static pthread_mutex_t dso__data_open_lock = PTHREAD_MUTEX_INITIALIZER;static void dso__list_add (struct dso *dso) {list_add_tail (&dso__data (dso)->open_entry, &dso__data_open); +#ifdef REFCNT_CHECKING dso__data (dso)->dso =dso__get (dso); +#endif +/* Assume the dso is part of dsos, hence the optional reference count above. */ +assert (dso__dsos (dso)); dso__data_open_cnt++; }static void dso__list_del (struct dso *dso) {list_del_init (&dso__data (dso)->open_entry); +#ifdef REFCNT_CHECKING dso__put (dso__data (dso)->dso); +#endif WARN_ONCE (dso__data_open_cnt <=0 ,"DSO data fd counter out of bounds." ); dso__data_open_cnt--;
@@ -654,9 +660,15 @@ static void close_dso(struct dso *dso)static void close_first_dso (void ) {struct dso_data *dso_data; +struct dso *dso; dso_data = list_first_entry(&dso__data_open, struct dso_data, open_entry); -close_dso (dso_data->dso); +#ifdef REFCNT_CHECKING + dso = dso_data->dso; +#else + dso = container_of(dso_data, struct dso, data); +#endif +close_dso (dso); }static rlim_t get_fd_limit (void )
@@ -1448,7 +1460,9 @@ struct dso *dso__new_id(const char *name, struct dso_id *id) data->fd =-1 ; data->status =DSO_DATA_STATUS_UNKNOWN ;INIT_LIST_HEAD (&data->open_entry); +#ifdef REFCNT_CHECKING data->dso =NULL ;/* Set when on the open_entry list. */ +#endif }return res; }
tools/perf/util/dso.h +2 -0
@@ -147,7 +147,9 @@ struct dso_cache {struct dso_data {struct rb_root cache;struct list_head open_entry; +#ifdef REFCNT_CHECKING struct dso *dso; +#endif int fd;int status;u32 status_seen;
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Ian Rogers
reviewerQuoted 19 lines · Ian Rogers
Modify the implementation of maps to not use an rbtree as the container for maps, instead use a sorted array. Improve locking and reference counting issues.
Similar to maps separate out and reimplement threads to use a hashmap for lower memory consumption and faster look up. The fixes a regression in memory usage where reference count checking switched to using non-invasive tree nodes. Reduce its default size by 32 times and improve locking discipline. Also, fix regressions where tids had become unordered to make
perf report --tasksandperf trace --summaryoutput easier to read.Better encapsulate the dsos abstraction. Remove the linked list and rbtree used for faster iteration and log(n) lookup to a sorted array for similar performance but half the memory usage per dso. Improve reference counting and locking discipline, adding reference count checking to dso.
Quoted 5 lines
v7:
- rebase to latest perf-tools-next where 22 patches were applied by Arnaldo.
- resolve merge conflicts, in particular withfc044c53b99f("perf
annotate-data: Adddso->data_typestree") that required more dso accessor functions.
Ping. No review comments:
https://lore.kernel.org/lkml/[email protected]/
This may start to conflict with Adrian's work:
https://lore.kernel.org/lkml/[email protected]/
but should just need minor get/put cleanup.
Thanks,
Ian
Quoted 106 lines
v6 series is here:
https://lore.kernel.org/lkml/[email protected]/Ian Rogers (25):
perf maps: Switch from rbtree to lazily sorted array for addresses perf maps: Get map before returning inmaps__find
perf maps: Get map before returning inmaps__find_by_nameperf maps: Get map before returning inmaps__find_next_entryperf maps: Hide maps internals
perf maps: Locking tidy up ofnr_maps
perf dso: Reorder variables to save space in struct dso perf report: Sort child tasks by tid
perf trace: Ignore thread hashing in summary
perf machine: Move fprintf tofor_eachloop and a callback perf threads: Move threads to its own files
perf threads: Switch from rbtree to hashmap
perf threads: Reduce table size from 256 to 8
perf dsos: Attempt to better abstract dsos internals perf dsos: Tidy reference counting and locking
perf dsos: Adddsos__for_each_dso
perf dso: Move dso functions out of dsos
perf dsos: Switch more loops todsos__for_each_dso
perf dsos: Switch backing storage to array from rbtree/list perf dsos: Remove__dsos__addnew
perf dsos: Remove__dsos__findnew_link_by_longname_idperf dsos: Switch hand code to bsearch
perf dso: Add reference count checking and accessor functions perf dso: Reference counting related fixes
perf dso: Usecontainer_ofto avoid a pointer indso_datatools/perf/arch/x86/tests/dwarf-unwind.c | 1 + tools/perf/builtin-annotate.c | 8 +- tools/perf/builtin-buildid-cache.c | 2 +- tools/perf/builtin-buildid-list.c | 18 +- tools/perf/builtin-inject.c | 96 +- tools/perf/builtin-kallsyms.c | 2 +- tools/perf/builtin-mem.c | 4 +- tools/perf/builtin-record.c | 2 +- tools/perf/builtin-report.c | 209 +-- tools/perf/builtin-script.c | 8 +- tools/perf/builtin-top.c | 4 +- tools/perf/builtin-trace.c | 43 +- tools/perf/tests/code-reading.c | 8 +- tools/perf/tests/dso-data.c | 67 +- tools/perf/tests/hists_common.c | 6 +- tools/perf/tests/hists_cumulate.c | 4 +- tools/perf/tests/hists_output.c | 2 +- tools/perf/tests/maps.c | 7 +- tools/perf/tests/symbols.c | 2 +- tools/perf/tests/thread-maps-share.c | 8 +- tools/perf/tests/vmlinux-kallsyms.c | 16 +- tools/perf/ui/browsers/annotate.c | 6 +- tools/perf/ui/browsers/hists.c | 8 +- tools/perf/ui/browsers/map.c | 4 +- tools/perf/util/Build | 1 + tools/perf/util/annotate-data.c | 6 +- tools/perf/util/annotate.c | 45 +- tools/perf/util/auxtrace.c | 2 +- tools/perf/util/block-info.c | 2 +- tools/perf/util/bpf-event.c | 9 +- tools/perf/util/bpf_lock_contention.c | 8 +- tools/perf/util/build-id.c | 136 +- tools/perf/util/build-id.h | 2 - tools/perf/util/callchain.c | 4 +- tools/perf/util/data-convert-json.c | 2 +- tools/perf/util/db-export.c | 6 +- tools/perf/util/dlfilter.c | 12 +- tools/perf/util/dso.c | 469 +++--- tools/perf/util/dso.h | 549 +++++-- tools/perf/util/dsos.c | 529 ++++--- tools/perf/util/dsos.h | 40 +- tools/perf/util/event.c | 12 +- tools/perf/util/header.c | 8 +- tools/perf/util/hist.c | 4 +- tools/perf/util/intel-pt.c | 22 +- tools/perf/util/machine.c | 570 +++----- tools/perf/util/machine.h | 32 +- tools/perf/util/map.c | 73 +- tools/perf/util/maps.c | 1280 +++++++++++------ tools/perf/util/maps.h | 65 +- tools/perf/util/probe-event.c | 26 +- tools/perf/util/rb_resort.h | 5 - .../util/scripting-engines/trace-event-perl.c | 6 +- .../scripting-engines/trace-event-python.c | 21 +- tools/perf/util/session.c | 21 + tools/perf/util/session.h | 2 + tools/perf/util/sort.c | 19 +- tools/perf/util/srcline.c | 65 +- tools/perf/util/symbol-elf.c | 132 +- tools/perf/util/symbol.c | 217 +-- tools/perf/util/symbol_fprintf.c | 4 +- tools/perf/util/synthetic-events.c | 24 +- tools/perf/util/thread.c | 8 +- tools/perf/util/thread.h | 6 - tools/perf/util/threads.c | 186 +++ tools/perf/util/threads.h | 35 + tools/perf/util/unwind-libunwind-local.c | 20 +- tools/perf/util/unwind-libunwind.c | 9 +- tools/perf/util/vdso.c | 56 +- 69 files changed, 3127 insertions(+), 2158 deletions(-) create mode 100644 tools/perf/util/threads.c create mode 100644 tools/perf/util/threads.h--
2.43.0.472.g3155946c3a-goog