Skip to content

Reference count checker and related fixes

Ian Rogers linux-perf-users lkml 3 versions – Applied

Status from this message: Thanks, applied.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest22 patches 35 repliesApplied
  2. v24 patches 15 repliesSuperseded
  3. v13 patches 2 repliesApplied

Patches in v3 Applied

#Subject and filesSizeRepliesApplied
1/22perf cpumap: Migrate to libperf cpumap apitools/perf/tests/cpumap.c+6 -61 reply
2/22perf cpumap: Use for each looptools/perf/util/pmu.c+6 -81 reply
3/22perf dso: Make lock error check and add BUG_ONstools/perf/util/dso.c tools/perf/util/symbol.c+10 -47 replies
4/22perf dso: Hold lock when accessing nsinfotools/perf/builtin-inject.c tools/perf/util/dso.c tools/perf/util/map.c +2 more+14 -22 replies
5/22perf maps: Use a pointer for kmapstools/perf/arch/x86/util/event.c tools/perf/tests/vmlinux-kallsyms.c tools/perf/util/bpf-event.c +5 more+34 -302 replies
6/22perf test: Use pointer for mapstools/perf/tests/maps.c+10 -103 replies
7/22perf maps: Reduce scope of init and exittools/perf/util/map.c tools/perf/util/maps.h+2 -41 reply
8/22perf maps: Move maps code to own C filetools/perf/util/Build tools/perf/util/map.c tools/perf/util/map.h +1 more+414 -4092 replies
9/22perf map: Add const to map_ip and unmap_iptools/perf/util/map.c tools/perf/util/map.h+23 -161 reply
10/22perf map: Make map__contains_symbol args consttools/perf/util/map.c tools/perf/util/map.h+2 -21 reply
11/22perf map: Move map list node into symboltools/perf/util/map.h tools/perf/util/symbol.c+60 -340 replies
12/22perf maps: Remove rb_node from struct maptools/perf/arch/x86/util/event.c tools/perf/builtin-report.c tools/perf/tests/maps.c +12 more+288 -1814 replies
13/22perf namespaces: Add functions to access nsinfotools/perf/builtin-inject.c tools/perf/builtin-probe.c tools/perf/util/build-id.c +5 more+63 -271 reply
14/22perf maps: Add functions to access mapstools/perf/scripts/python/Perf-Trace-Util/Context.c tools/perf/tests/code-reading.c tools/perf/ui/browsers/hists.c +17 more+170 -1051 reply
15/22perf map: Use functions to access the variables in maptools/perf/arch/s390/annotate/instructions.c tools/perf/arch/x86/tests/dwarf-unwind.c tools/perf/arch/x86/util/event.c +46 more+577 -4894 replies
16/22perf test: Add extra diagnostics to maps testtools/perf/tests/maps.c+36 -150 replies
17/22perf map: Changes to reference countingtools/perf/tests/hists_cumulate.c tools/perf/tests/hists_filter.c tools/perf/tests/hists_link.c +6 more+118 -504 replies
18/22libperf: Add reference count checking macros.tools/lib/perf/include/internal/rc_check.h+94 -00 replies
19/22perf cpumap: Add reference count checkingtools/lib/perf/cpumap.c tools/lib/perf/include/internal/cpumap.h tools/perf/tests/cpumap.c +2 more+75 -680 replies
20/22perf namespaces: Add reference count checkingtools/perf/util/namespaces.c tools/perf/util/namespaces.h+78 -570 replies
21/22perf maps: Add reference count checking.tools/perf/tests/thread-maps-share.c tools/perf/util/maps.c tools/perf/util/maps.h +3 more+60 -530 replies
22/22perf map: Add reference count checkingtools/perf/arch/s390/annotate/instructions.c tools/perf/builtin-top.c tools/perf/tests/hists_link.c +9 more+127 -1140 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH v3 00/22] Reference count checker and related fixes

The perf tool has a class of memory problems where reference counts are used incorrectly. Memory/address sanitizers and valgrind don't provide useful ways to debug these problems, you see a memory leak where the only pertinent information is the original allocation site. What would be more useful is knowing where a get fails to have a corresponding put, where there are double puts, etc.

This work was motivated by the roll-back of:
https://lore.kernel.org/linux-perf-users/[email protected]/
where fixing a missed put resulted in a use-after-free in a different context. There was a sense in fixing the issue that a game of wac-a-mole had been embarked upon in adding missed gets and puts.

The basic approach of the change is to add a level of indirection at the get and put calls. Get allocates a level of indirection that, if no corresponding put is called, becomes a memory leak (and associated stack trace) that leak sanitizer can report. Similarly if two puts are called for the same get, then a double free can be detected by address sanitizer. This can also detect the use after put, which should also yield a segv without a sanitizer.

Adding reference count checking to cpu map was done as a proof of concept, it yielded little other than a location where the use of get could be cleaner by using its result. Reference count checking on nsinfo identified a double free of the indirection layer and the related threads, thereby identifying a data race as discussed here:
https://lore.kernel.org/linux-perf-users/CAP-5=fWZH20L4kv-BwVtGLwR=Em3AOOT+Q4QGivvQuYn5AsPRg@mail.gmail.com/
Accordingly the dso->lock was extended and use to cover the race.

The v3 version addresses problems in v2, in particular using macros to avoid #ifdefs. The v3 version applies the reference count checking approach to two more data structures, maps and map. While maps was straightforward, struct map showed a problem where reference counted thing can be on lists and rb-trees that are oblivious to the reference count. To sanitize this, struct map is changed so that it is referenced by either a list or rb-tree node and not part of it. This simplifies the reference count and the patches have caught and fixed a number of missed or mismatched reference counts relating to struct map.

The patches are arranged so that API refactors and bug fixes appear first, then the reference count checker itself appears. This allows for the refactor and fixes to be applied upstream first, as has already happened with cpumap.

A wider discussion of the approach is on the mailing list:
https://lore.kernel.org/linux-perf-users/[email protected]/T/#mf25ccd7a2e03de92cec29d36e2999a8ab5ec7f88
Comparing it to a past approach:
https://lore.kernel.org/all/[email protected]/
and to ref_tracker:
https://lwn.net/Articles/877603/

Ian Rogers (22):
perf cpumap: Migrate to libperf cpumap api perf cpumap: Use for each loop
perf dso: Make lock error check and add BUG_ONs perf dso: Hold lock when accessing nsinfo perf maps: Use a pointer for kmaps
perf test: Use pointer for maps
perf maps: Reduce scope of init and exit
perf maps: Move maps code to own C file
perf map: Add const to map_ip and unmap_ip perf map: Make map__contains_symbol args const perf map: Move map list node into symbol
perf maps: Remove rb_node from struct map perf namespaces: Add functions to access nsinfo perf maps: Add functions to access maps
perf map: Use functions to access the variables in map perf test: Add extra diagnostics to maps test perf map: Changes to reference counting
libperf: Add reference count checking macros. perf cpumap: Add reference count checking perf namespaces: Add reference count checking perf maps: Add reference count checking.
perf map: Add reference count checking

 tools/lib/perf/cpumap.c                       |  93 +--
 tools/lib/perf/include/internal/cpumap.h      |   4 +-
 tools/lib/perf/include/internal/rc_check.h    |  94 +++
 tools/perf/arch/s390/annotate/instructions.c  |   4 +-
 tools/perf/arch/x86/tests/dwarf-unwind.c      |   2 +-
 tools/perf/arch/x86/util/event.c              |  15 +-
 tools/perf/builtin-annotate.c                 |   8 +-
 tools/perf/builtin-inject.c                   |  14 +-
 tools/perf/builtin-kallsyms.c                 |   6 +-
 tools/perf/builtin-kmem.c                     |   4 +-
 tools/perf/builtin-mem.c                      |   4 +-
 tools/perf/builtin-probe.c                    |   2 +-
 tools/perf/builtin-report.c                   |  26 +-
 tools/perf/builtin-script.c                   |  26 +-
 tools/perf/builtin-top.c                      |  16 +-
 tools/perf/builtin-trace.c                    |   2 +-
 .../scripts/python/Perf-Trace-Util/Context.c  |  14 +-
 tools/perf/tests/code-reading.c               |  32 +-
 tools/perf/tests/cpumap.c                     |  14 +-
 tools/perf/tests/hists_common.c               |   4 +-
 tools/perf/tests/hists_cumulate.c             |  14 +-
 tools/perf/tests/hists_filter.c               |  14 +-
 tools/perf/tests/hists_link.c                 |  18 +-
 tools/perf/tests/hists_output.c               |  12 +-
 tools/perf/tests/maps.c                       |  87 ++-
 tools/perf/tests/mmap-thread-lookup.c         |   3 +-
 tools/perf/tests/thread-maps-share.c          |  29 +-
 tools/perf/tests/vmlinux-kallsyms.c           |  56 +-
 tools/perf/ui/browsers/annotate.c             |   7 +-
 tools/perf/ui/browsers/hists.c                |  21 +-
 tools/perf/ui/browsers/map.c                  |   4 +-
 tools/perf/util/Build                         |   1 +
 tools/perf/util/annotate.c                    |  38 +-
 tools/perf/util/auxtrace.c                    |   2 +-
 tools/perf/util/block-info.c                  |   4 +-
 tools/perf/util/bpf-event.c                   |  10 +-
 tools/perf/util/build-id.c                    |   6 +-
 tools/perf/util/callchain.c                   |  28 +-
 tools/perf/util/cpumap.c                      |  36 +-
 tools/perf/util/data-convert-json.c           |   4 +-
 tools/perf/util/db-export.c                   |  16 +-
 tools/perf/util/dlfilter.c                    |  29 +-
 tools/perf/util/dso.c                         |  21 +-
 tools/perf/util/event.c                       |  30 +-
 tools/perf/util/evsel_fprintf.c               |   4 +-
 tools/perf/util/hist.c                        |  22 +-
 tools/perf/util/intel-pt.c                    |  48 +-
 tools/perf/util/jitdump.c                     |  10 +-
 tools/perf/util/machine.c                     | 252 ++++---
 tools/perf/util/machine.h                     |   8 +-
 tools/perf/util/map.c                         | 629 ++++--------------
 tools/perf/util/map.h                         |  80 ++-
 tools/perf/util/maps.c                        | 475 +++++++++++++
 tools/perf/util/maps.h                        |  69 +-
 tools/perf/util/namespaces.c                  | 158 +++--
 tools/perf/util/namespaces.h                  |  13 +-
 tools/perf/util/pmu.c                         |  18 +-
 tools/perf/util/probe-event.c                 |  58 +-
 .../util/scripting-engines/trace-event-perl.c |   9 +-
 .../scripting-engines/trace-event-python.c    |  14 +-
 tools/perf/util/sort.c                        |  48 +-
 tools/perf/util/symbol-elf.c                  |  59 +-
 tools/perf/util/symbol.c                      | 280 +++++---
 tools/perf/util/symbol_fprintf.c              |   2 +-
 tools/perf/util/synthetic-events.c            |  34 +-
 tools/perf/util/thread-stack.c                |   4 +-
 tools/perf/util/thread.c                      |  40 +-
 tools/perf/util/unwind-libunwind-local.c      |  50 +-
 tools/perf/util/unwind-libunwind.c            |  34 +-
 tools/perf/util/vdso.c                        |   7 +-
 70 files changed, 1941 insertions(+), 1358 deletions(-)
 create mode 100644 tools/lib/perf/include/internal/rc_check.h
 create mode 100644 tools/perf/util/maps.c
Signature · 1 line
2.35.1.265.g69c8d7142f-goog