Skip to content

perf symbol: Reference counting, flat array storage, and LRU shrinking

Ian Rogers linux-perf-users lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest7 patches 9 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7perf symbol: Add accessor functions for struct symbol fieldstools/perf/arch/powerpc/util/skip-callchain-idx.c tools/perf/arch/powerpc/util/sym-handling.c tools/perf/builtin-annotate.c +61 more+504 -3900 replies
2/7perf symbol: Remove symbol_conf.priv_size and negative-offset allocationstools/perf/builtin-c2c.c tools/perf/builtin-report.c tools/perf/tests/builtin-test.c +9 more+219 -950 replies
3/7perf symbol: Switch backing storage from rbtree to struct symbols arraytools/perf/arch/powerpc/util/sym-handling.c tools/perf/builtin-annotate.c tools/perf/builtin-kmem.c +13 more+801 -5900 replies
4/7perf symbol: Add reference counting and DECLARE_RC_STRUCT(symbol)tools/perf/builtin-kallsyms.c tools/perf/builtin-lock.c tools/perf/builtin-script.c +45 more+427 -1900 replies
5/7perf symbol: Add LRU memory shrinking for symbols, DSOs, and machinestools/perf/util/dso.c tools/perf/util/dso.h tools/perf/util/dsos.c +9 more+551 -570 replies
6/7perf session: Periodically shrink symbols and DSOs during event processingtools/perf/builtin-top.c tools/perf/util/session.c tools/perf/util/session.h +2 more+44 -00 replies
7/7perf test symbols: Add tests for symbol and DSO LRU shrinkingtools/perf/tests/symbols.c+228 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH v1 0/7] perf symbol: Reference counting, flat array storage, and LRU shrinking

This series modernizes struct symbol memory management and backing storage in tools/perf to support reference counting (DECLARE_RC_STRUCT(symbol)) and periodic LRU memory shrinking during long or large profiling sessions:

1. Field accessors (Patch 1): Introduces inline getters/setters for struct symbol fields across tools/perf in preparation for wrapping struct symbol with DECLARE_RC_STRUCT(symbol).
2. Remove negative-offset private allocations (Patch 2): Replaces symbol_conf.priv_size and symbol__priv() negative-offset allocations in util/annotate.c and util/db-export.c with hashmaps keyed by symbol pointer (struct annotated_symbol protected by an rw_semaphore, and sym_db_ids on struct db_export).
3. Flat sorted array backing storage (Patch 3): Replaces the dual address-sorted and name-sorted red-black trees (rb_node) in struct dso with a struct symbols container backed by dynamically sorted flat arrays protected by struct rw_semaphore (util/rwsem.h) with Clang thread-safety annotations. Saves 48 bytes per struct symbol (two rb_nodes) and provides cache-friendly binary search (bsearch) and callback-based iteration.
4. Reference counting and DECLARE_RC_STRUCT(symbol) (Patch 4): Wraps struct symbol with DECLARE_RC_STRUCT(symbol) and adds refcount_t refcnt along with symbol__get(), symbol__put(), and symbol__zput(). Lookup functions return a counted reference acquired under symbols->lock, and lifetime holders (addr_location, map_symbol, hist_entry, call_path, block_info, disasm_line, thread_stack, db_export, etc.) properly manage symbol references.
5. LRU memory shrinking (Patch 5): Adds an accessed bit to struct dso and implements symbols__shrink(), dso__shrink_symbols(), dsos__shrink(), machine__shrink(), and machines__shrink(). Unreferenced symbols (refcnt == 1) in cold DSOs are reclaimed, and shrunken DSOs transparently reload on demand via dso__load() while preserving live symbols and their RC_STRUCT(symbol) identity.
6. Periodic session shrinking (Patch 6): Hooks machines__shrink() into perf_session event processing so long-running sessions periodically reclaim cold symbols and DSOs.
7. Unit tests (Patch 7): Adds unit tests in tools/perf/tests/symbols.c covering symbol reference counting, symbols__shrink() retention of active symbols, dsos__shrink() two-pass LRU clock behavior, and on-demand reload preserving live symbol identity.

Ian Rogers (7):
perf symbol: Add accessor functions for struct symbol fields perf symbol: Remove symbol_conf.priv_size and negative-offset
allocations
perf symbol: Switch backing storage from rbtree to struct symbols
array
perf symbol: Add reference counting and DECLARE_RC_STRUCT(symbol) perf symbol: Add LRU memory shrinking for symbols, DSOs, and machines perf session: Periodically shrink symbols and DSOs during event
processing
perf test symbols: Add tests for symbol and DSO LRU shrinking

 .../arch/powerpc/util/skip-callchain-idx.c    |   2 +-
 tools/perf/arch/powerpc/util/sym-handling.c   |  40 +-
 tools/perf/builtin-annotate.c                 |  15 +-
 tools/perf/builtin-c2c.c                      |  14 -
 tools/perf/builtin-diff.c                     |   5 +-
 tools/perf/builtin-kallsyms.c                 |  10 +-
 tools/perf/builtin-kmem.c                     |  66 +-
 tools/perf/builtin-kwork.c                    |   4 +-
 tools/perf/builtin-lock.c                     |  27 +-
 tools/perf/builtin-mem.c                      |   2 +-
 tools/perf/builtin-report.c                   |  14 -
 tools/perf/builtin-sched.c                    |   6 +-
 tools/perf/builtin-script.c                   |  18 +-
 tools/perf/builtin-timechart.c                |   2 +-
 tools/perf/builtin-top.c                      |  29 +-
 tools/perf/builtin-trace.c                    |  12 +-
 tools/perf/tests/builtin-test.c               |   1 -
 tools/perf/tests/dlfilter-test.c              |  12 +-
 tools/perf/tests/dwarf-unwind.c               |   2 +-
 tools/perf/tests/hists_common.c               |   4 +-
 tools/perf/tests/hists_cumulate.c             |   8 +-
 tools/perf/tests/hists_filter.c               |  10 +-
 tools/perf/tests/hists_link.c                 |  16 +-
 tools/perf/tests/hists_output.c               |   9 +-
 tools/perf/tests/kallsyms-split.c             |   5 +-
 tools/perf/tests/pe-file-parsing.c            |   1 +
 tools/perf/tests/symbols.c                    | 289 +++++-
 tools/perf/tests/vmlinux-kallsyms.c           | 191 ++--
 tools/perf/ui/browsers/annotate.c             |  11 +-
 tools/perf/ui/browsers/hists.c                |  17 +-
 tools/perf/ui/browsers/map.c                  |  87 +-
 tools/perf/ui/gtk/annotate.c                  |   7 +-
 tools/perf/ui/stdio/hist.c                    |   6 +-
 tools/perf/util/addr_location.c               |   4 +
 .../perf/util/annotate-arch/annotate-alpha.c  |   2 +-
 .../util/annotate-arch/annotate-loongarch.c   |   8 +-
 tools/perf/util/annotate-arch/annotate-s390.c |   2 +-
 tools/perf/util/annotate-arch/annotate-x86.c  |  14 +-
 tools/perf/util/annotate-data.c               |  14 +-
 tools/perf/util/annotate.c                    | 211 ++++-
 tools/perf/util/annotate.h                    |   6 +-
 tools/perf/util/auxtrace.c                    | 111 ++-
 tools/perf/util/block-info.c                  |  13 +-
 tools/perf/util/bpf_lock_contention.c         |  39 +-
 tools/perf/util/c2c-function.c                |  25 +-
 tools/perf/util/call-path.c                   |  15 +-
 tools/perf/util/callchain.c                   |  19 +-
 tools/perf/util/capstone.c                    |  21 +-
 tools/perf/util/data-convert-json.c           |   4 +-
 tools/perf/util/db-export.c                   |  76 +-
 tools/perf/util/db-export.h                   |   6 +
 tools/perf/util/debug.c                       |   3 +-
 tools/perf/util/disasm.c                      |  57 +-
 tools/perf/util/dlfilter.c                    |  12 +-
 tools/perf/util/dso.c                         | 236 +++--
 tools/perf/util/dso.h                         |  85 +-
 tools/perf/util/dsos.c                        | 100 +++
 tools/perf/util/dsos.h                        |  12 +-
 tools/perf/util/event.c                       |  12 +-
 tools/perf/util/evsel_fprintf.c               |   2 +-
 tools/perf/util/hist.c                        |  30 +-
 tools/perf/util/intel-pt.c                    |  74 +-
 tools/perf/util/libbfd.c                      |   5 +-
 tools/perf/util/libdw.c                       |   5 +-
 tools/perf/util/llvm.c                        |   6 +-
 tools/perf/util/lock-contention.c             |   8 +-
 tools/perf/util/machine.c                     | 143 ++-
 tools/perf/util/machine.h                     |   2 +
 tools/perf/util/map.c                         |  53 +-
 tools/perf/util/map.h                         |  18 +-
 tools/perf/util/map_symbol.c                  |   4 +-
 tools/perf/util/maps.c                        |   3 +-
 tools/perf/util/probe-event.c                 | 135 +--
 tools/perf/util/python.c                      |  20 +-
 .../util/scripting-engines/trace-event-perl.c |   8 +-
 .../scripting-engines/trace-event-python.c    |  38 +-
 tools/perf/util/session.c                     |  31 +
 tools/perf/util/session.h                     |  13 +
 tools/perf/util/sort.c                        |  44 +-
 tools/perf/util/srcline.c                     |  19 +-
 tools/perf/util/symbol-elf.c                  |  22 +-
 tools/perf/util/symbol.c                      | 845 ++++++++++++------
 tools/perf/util/symbol.h                      | 203 ++++-
 tools/perf/util/symbol_conf.h                 |   1 -
 tools/perf/util/symbol_fprintf.c              |  14 +-
 tools/perf/util/thread-stack.c                |  18 +-
 tools/perf/util/threads.c                     |  17 +-
 tools/perf/util/unwind-libdw.c                |   4 +-
 tools/perf/util/unwind-libunwind.c            |   2 +-
 89 files changed, 2644 insertions(+), 1192 deletions(-)
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog