Skip to contentSkip to conversation

addr2line performance

DISCUSSION Tony Jones 3 participants 7 messages Oct 22, 2025 – Nov 4, 2025

lore
Share & tools
lore

Tony Jones

author
#0 lore raw

Hi.

https://bugzilla.kernel.org/show_bug.cgi?id=218996#c1 mentions "there are proposals to move this logic to being from a library again. I think the addr2line processing attempted is valiant but ultimately using a library will be best" (this was when LLVM addr2line support was already present),

Obviously the test patch mentioned in the above bug doesn't address the underlying performance issue, only better classifying a timeout error.

We have a customer bug complaining about the performance of addr2line. I'm seeing cases where it's taking over 20 seconds wall clock to resolve certain addresses which obviously causes the io_timer to trip but the performance of the success path isn't great either.

Obviously BFD isn't an option and (for various reasons) we can't use LLVM for this specific distro so I'm looking at the performance of addr2line.

Can someone point me to the prior "using a library" discussion referenced in the above bug or whatever the current thinking is. I couldn't find anything searching the archive.

Thanks in advance!

Tony

Signature · 2 lines
Tony Jones
SUSE Kernel Performance Team
#1 lore raw
Quoted 18 lines · Tony Jones

Hi.

https://bugzilla.kernel.org/show_bug.cgi?id=218996#c1 mentions "there are proposals to move this logic to being from a library again. I think the addr2line processing attempted is valiant but ultimately using a library will be best" (this was when LLVM addr2line support was already present),

Obviously the test patch mentioned in the above bug doesn't address the underlying performance issue, only better classifying a timeout error.

We have a customer bug complaining about the performance of addr2line. I'm seeing cases where it's taking over 20 seconds wall clock to resolve certain addresses which obviously causes the io_timer to trip but the performance of the success path isn't great either.

Quoted 7 lines

Obviously BFD isn't an option and (for various reasons) we can't use LLVM for this specific distro so I'm looking at the performance of addr2line.

Can someone point me to the prior "using a library" discussion referenced in the above bug or whatever the current thinking is. I couldn't find anything searching the archive.

I'm not sure there is an explicit discussion rather than discussion on individual patch sets. It is something that comes up when fixing the text parsing of addr2line, something that is particularly problematic because of inlining. It comes up wrt removing non-distributable stuff, most recently in:
https://lore.kernel.org/lkml/[email protected]/

Wrt the bug, the --addr2line option for perf script is merged:
https://lore.kernel.org/r/[email protected]

There has also been clean up into how the addr2line implementation is picked, see:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/srcline.c?h=tmp.perf-tools-next#n117
which came from:
https://lore.kernel.org/lkml/[email protected]/

Wrt dependencies, I have been pushing these patches that use dlopen for libLLVM mean installing libLLVM can be done separately from building perf against it:
https://lore.kernel.org/lkml/[email protected]/

So if addr2line is the problem, perhaps use the command line option to switch to the llvm version. If that's not fast enough, and you can't distribute linked against libLLVM, some feedback on the dlopen libLLVM changes would be appreciated!

Thanks,
Ian

Quoted 7 lines

Thanks in advance!

Tony

--
Tony Jones
SUSE Kernel Performance Team

author
#2 lore raw
Quoted 3 lines · Ian Rogers

I'm not sure there is an explicit discussion rather than discussion on individual patch sets.

Thanks for the reply. I thought the original comment in that BZ was talking about plans to remove the current forking of an "addr2line" process and communicating via pipe with some library implementation. Of course what that library would be if not BFD/LLVM was my question. Clearly this was a misread on my part.

I'd seen the patches related to dlopen'ing llvm, backporting this is a possibility since we don't want a rpm requires.

Cheers!

Tony

Signature · 2 lines
Tony Jones
SUSE Kernel Performance Team
#3 lore raw
Quoted 6 lines · Tony Jones
Quoted 3 lines · Ian Rogers

I'm not sure there is an explicit discussion rather than discussion on individual patch sets.

Quoted 8 lines

Thanks for the reply. I thought the original comment in that BZ was talking about plans to remove the current forking of an "addr2line" process and communicating via pipe with some library implementation. Of course what that library would be if not BFD/LLVM was my question. Clearly this was a misread on my part.

I'd seen the patches related to dlopen'ing llvm, backporting this is a possibility since we don't want a rpm required.

That's great to know and motivates the dlopen work! I can resend a rebase of that series, would it be okay to add an Acked-by tag[1] to the series saying it'd be useful for SuSE ?

Thanks,
Ian

[1] https://www.kernel.org/doc/Documentation/process/submitting-patches.rst

Quoted 7 lines

Cheers!

Tony

--
Tony Jones
SUSE Kernel Performance Team

#4 lore raw
In reply to Tony Jones

Hello,

Quoted 17 lines · Tony Jones

Hi.

https://bugzilla.kernel.org/show_bug.cgi?id=218996#c1 mentions "there are proposals to move this logic to being from a library again. I think the addr2line processing attempted is valiant but ultimately using a library will be best" (this was when LLVM addr2line support was already present),

Obviously the test patch mentioned in the above bug doesn't address the underlying performance issue, only better classifying a timeout error.

We have a customer bug complaining about the performance of addr2line. I'm seeing cases where it's taking over 20 seconds wall clock to resolve certain addresses which obviously causes the io_timer to trip but the performance of the success path isn't great either.

Quoted 7 lines

Obviously BFD isn't an option and (for various reasons) we can't use LLVM for this specific distro so I'm looking at the performance of addr2line.

Can someone point me to the prior "using a library" discussion referenced in the above bug or whatever the current thinking is. I couldn't find anything searching the archive.

I think we can just use libdw. I tried it in the past but never finished. :(

Thanks,
Namhyung

author
#5 lore raw
Quoted 2 lines · Namhyung Kim

I think we can just use libdw. I tried it in the past but never finished. :(

This would be interesting.

I asked our toolchain folks if they were aware of libraries (other than libllvm and libbfd) that could replace the addr2line+pipe (and were license compatible) but I've not heard back from them yet.

I'm happy to look into implementing this if you lack the time. LMK.

Tony

Signature · 2 lines
Tony Jones
SUSE Kernel Performance Team
#6 lore raw

Hello,

Quoted 5 lines · Tony Jones
Quoted 2 lines · Namhyung Kim

I think we can just use libdw. I tried it in the past but never finished. :(

Quoted 7 lines

This would be interesting.

I asked our toolchain folks if they were aware of libraries (other than libllvm and libbfd) that could replace the addr2line+pipe (and were license compatible) but I've not heard back from them yet.

I'm happy to look into implementing this if you lack the time. LMK.

Sure, it'd be great if you can work on it. I think it's basically dwarf_lineaddr() followed by dwarf_linesrc() and dwarf_lineno() but you also need to check inlined functions.

And probably you want to cache dbginfo of recent binaries to speed up addr2line processing for the same DSOs.

Thanks,
Namhyung