Skip to content

Parallelize BTF type info generating of pahole

Kui-Feng Lee bpf dwarves 3 versions – Applied

Status from this message: Thanks, applied!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest4 patches 20 repliesApplied
  2. v34 patches 6 repliesSuperseded
  3. v23 patches 3 repliesSuperseded

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/4dwarf_loader: Receive per-thread data on worker threads.btf_loader.c ctf_loader.c dwarf_loader.c +4 more+14 -911 replies
2/4dwarf_loader: Prepare and pass per-thread data to worker threads.dwarf_loader.c dwarves.h+50 -131 reply
3/4pahole: Use per-thread btf instances to avoid mutex locking.btf_encoder.c btf_encoder.h pahole.c+124 -87 replies
4/4libbpf: Update libbpf to a new revision.btf_encoder.c btf_loader.c lib/bpf+12 -121 reply

Reviews trailers per patch

All 4 patches have a Reviewed-by or Acked-by.

PatchAndrii Nakryiko
1/4A-b
2/4A-b
3/4A-b
4/4A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH dwarves v4 0/4] Parallelize BTF type info generating of pahole

Creating an instance of btf for each worker thread allows steal-function provided by pahole to add type info on multiple threads without a lock. The main thread merges the results of worker threads to the primary instance.

Copying data from per-thread btf instances to the primary instance is expensive now. However, there is a patch landed at the bpf-next repository. [1] With the patch for bpf-next and this patch, they drop total runtime to 5.4s from 6.0s with "-j4" on my device to generate BTF for Linux.

V4 includes following changes.

- Fix nits and typos.

- Rollback to calling btf__add_btf() at the main thread to simplify code. The reasons for that are additional lock making it complicated and doing w/o reusing btf_encoder being obvious slower.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=d81283d27266
[v1] https://lore.kernel.org/dwarves/[email protected]/
[v2] https://lore.kernel.org/dwarves/[email protected]/
[v3] https://lore.kernel.org/dwarves/[email protected]/

Kui-Feng Lee (4):
dwarf_loader: Receive per-thread data on worker threads.
dwarf_loader: Prepare and pass per-thread data to worker threads.
pahole: Use per-thread btf instances to avoid mutex locking.
libbpf: Update libbpf to a new revision.

 btf_encoder.c  |  25 ++++++----
 btf_encoder.h  |   2 +
 btf_loader.c   |   4 +-
 ctf_loader.c   |   2 +-
 dwarf_loader.c |  59 ++++++++++++++++++-----
 dwarves.h      |   9 +++-
 lib/bpf        |   2 +-
 pahole.c       | 128 +++++++++++++++++++++++++++++++++++++++++++++----
 pdwtags.c      |   3 +-
 pfunct.c       |   4 +-
 10 files changed, 198 insertions(+), 40 deletions(-)
Signature · 1 line
2.30.2