Skip to content

Benchmark and improve event synthesis performance

Ian Rogers linux-perf-users lkml 2 versions – Applied

Status from this message: Thanks, applied,

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest5 patches 8 repliesApplied
  2. v15 patches 4 repliesSuperseded

Patches in v2 Applied

#Subject and filesSizeRepliesApplied
1/5perf bench: add event synthesis benchmarktools/perf/Documentation/perf-bench.txt tools/perf/bench/Build tools/perf/bench/bench.h +2 more+117 -21 reply
2/5tools api fs: make xxx__mountpoint() more scalabletools/lib/api/fs/fs.c tools/lib/api/fs/fs.h+29 -01 reply
3/5perf synthetic-events: save 4kb from 2 stack framestools/perf/util/synthetic-events.c+10 -120 replies
4/5tools api: add a lightweight buffered reading apitools/lib/api/io.h+107 -05 replies
5/5perf synthetic events: Remove use of sscanf from /proc readingtools/perf/util/synthetic-events.c+105 -520 replies

Reviews trailers per patch

1 of 5 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchIan RogersJiri Olsa
SeriesA-b
1/5
2/5R-b A-b
3/5
4/5
5/5

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/5] Benchmark and improve event synthesis performance

Event synthesis is performance critical in common tasks using perf. For example, when perf record starts in system wide mode the /proc file system is scanned with events synthesized for each process and all executable mmaps. With large machines and lots of processes, we have seen O(seconds) of wall clock time while synthesis is occurring.

This patch set adds a benchmark for synthesis performance in a new benchmark collection called 'internals'. The benchmark uses the machine__synthesize_threads function, single threaded on the perf process with a 'tool' that just drops the events, to measure how long synthesis takes.

By profiling this benchmark 2 performance bottlenecks were identified, hugetlbfs_mountpoint and stdio. The impact of theses changes are:

Before:
Average synthesis took: 167.616800 usec
Average data synthesis took: 208.655600 usec

After hugetlbfs_mountpoint scalability fix:
Average synthesis took: 120.195100 usec
Average data synthesis took: 156.582300 usec

After removal of stdio in /proc/pid/maps code:
Average synthesis took: 67.189100 usec
Average data synthesis took: 102.451600 usec

Time was measured on an Intel Xeon 6154 compiling with Debian gcc 9.2.1.

v2 of this patch set adds the new benchmark to the perf-bench man page and addresses review comments from Jiri Olsa, thanks!

Two patches in the set were sent to LKML previously but are included here for context around the benchmark performance impact:
https://lore.kernel.org/lkml/[email protected]/T/#u
https://lore.kernel.org/lkml/[email protected]/T/#u

A future area of improvement could be to add the perf top num-thread-synthesize option more widely to other perf commands, and also to benchmark its effectiveness.

Ian Rogers (4):
perf bench: add event synthesis benchmark
perf synthetic-events: save 4kb from 2 stack frames tools api: add a lightweight buffered reading api perf synthetic events: Remove use of sscanf from /proc reading

Stephane Eranian (1):
tools api fs: make xxx__mountpoint() more scalable

 tools/lib/api/fs/fs.c                   |  17 +++
 tools/lib/api/fs/fs.h                   |  12 ++
 tools/lib/api/io.h                      | 107 ++++++++++++++
 tools/perf/Documentation/perf-bench.txt |   8 ++
 tools/perf/bench/Build                  |   2 +-
 tools/perf/bench/bench.h                |   2 +-
 tools/perf/bench/synthesize.c           | 101 ++++++++++++++
 tools/perf/builtin-bench.c              |   6 +
 tools/perf/util/synthetic-events.c      | 177 +++++++++++++++---------
 9 files changed, 367 insertions(+), 65 deletions(-)
 create mode 100644 tools/lib/api/io.h
 create mode 100644 tools/perf/bench/synthesize.c
Signature · 1 line
2.26.0.rc2.310.g2932bb562d-goog