Skip to content

Add io_dir to avoid memory overhead from opendir

Ian Rogers linux-perf-users lkml 2 versions – Applied

Status from this message: Applied to perf-tools-next, thanks!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest8 patches 4 repliesApplied
  2. v27 patches 7 repliesSuperseded

Patches in v3 Applied

#Subject and filesSizeRepliesApplied
1/8tools lib api: Add io_dir an allocation free readdir alternativetools/lib/api/Makefile tools/lib/api/io_dir.h+105 -10 replies
2/8perf maps: Switch modules tree walk to io_dir__readdirtools/perf/util/machine.c+8 -170 replies
3/8perf pmu: Switch to io_dir__readdirtools/perf/util/pmu.c tools/perf/util/pmus.c+30 -460 replies
4/8perf header: Switch mem topology to io_dir__readdirtools/perf/util/header.c+16 -150 replies
5/8perf events: Remove scandir in thread synthesistools/perf/util/synthetic-events.c+12 -100 replies
6/8perf parse-events: Switch tracepoints to io_dir__readdirtools/perf/util/parse-events.c+19 -130 replies
7/8perf hwmon_pmu: Switch event discovery to io_dir__readdirtools/perf/util/hwmon_pmu.c+17 -250 replies
8/8perf machine: Reuse module path buffertools/perf/util/machine.c+23 -110 replies

Reviews trailers per patch

3 of 8 patches have a Reviewed-by or Acked-by.

PatchNamhyung Kim
1/8
2/8A-b
3/8A-b
4/8
5/8A-b
6/8
7/8
8/8

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH v3 0/8] Add io_dir to avoid memory overhead from opendir

glibc's opendir allocates a minimum of 32kb, when called recursively for a directory tree the memory consumption can add up - nearly 300kb during perf start-up when processing modules. Add a stack allocated variant of readdir sized a little more than 1kb

v3: Rebase on top of Krzysztof Łopatowski's work. Add additional
defines for SYS_getdents64 on all other architectures if its definition is missing. Add a patch to further reduce the stack/memory usage in machine__set_modules_path_dir by appending to a buffer rather than creating a copy.
v2: Remove the feature test and always use a perf supplied getdents64
to workaround an Alpine Linux issue in v1:
https://lore.kernel.org/lkml/[email protected]/
As suggested by Krzysztof Łopatowski <[email protected]> who also pointed to the perf trace performance improvements in start-up time eliminating stat calls can achieve:
https://lore.kernel.org/lkml/[email protected]/
Convert parse-events and hwmon_pmu to use io_dir.
v1: This was previously part of the memory saving change set:
https://lore.kernel.org/lkml/[email protected]/
It is separated here and a feature check and syscall workaround for missing getdents64 added.

Ian Rogers (8):
tools lib api: Add io_dir an allocation free readdir alternative perf maps: Switch modules tree walk to io_dir__readdir perf pmu: Switch to io_dir__readdir
perf header: Switch mem topology to io_dir__readdir perf events: Remove scandir in thread synthesis
perf parse-events: Switch tracepoints to io_dir__readdir perf hwmon_pmu: Switch event discovery to io_dir__readdir perf machine: Reuse module path buffer

 tools/lib/api/Makefile             |   2 +-
 tools/lib/api/io_dir.h             | 104 +++++++++++++++++++++++++++++
 tools/perf/util/header.c           |  31 ++++-----
 tools/perf/util/hwmon_pmu.c        |  42 +++++-------
 tools/perf/util/machine.c          |  57 ++++++++--------
 tools/perf/util/parse-events.c     |  32 +++++----
 tools/perf/util/pmu.c              |  46 ++++++-------
 tools/perf/util/pmus.c             |  30 +++------
 tools/perf/util/synthetic-events.c |  22 +++---
 9 files changed, 229 insertions(+), 137 deletions(-)
 create mode 100644 tools/lib/api/io_dir.h
Signature · 1 line
2.48.1.658.g4767266eb4-goog