perf symbol: Speed up demangling of long Rust symbols
cover perf symbol: Speed up demangling of long Rust symbols
dso__demangle_sym() demangles Rust v0 symbols itself instead of using a helper like the other demanglers. When a demangled name doesn't fit the buffer, it grows the buffer by a fixed 32 bytes and reformats the whole name from scratch. The number of attempts grows with the name's length, so the total work is quadratic. Rust programs with deeply nested generics can have very long demangled names, and I noticed this as real time spent in perf report on perf.data recorded from launching Bevy and Typst.
Patch 3 is the actual optimisation, doubling the size of the buffer instead. Patch 5 moves the Rust-specific logic into its own function, matching how the other demanglers are called. I think the cleanup is worthwhile but I understand it comes down to personal taste and the patch can be skipped if maintainers would rather keep dso__demangle_sym() as is.
I used an LLM to review my patches and find the issues with the demangle tests and the unterminated buffer bug.
Testing:
- 'perf test' passes, including the demangle Java, OCaml and Rust suites (patch 3 adds a case for a symbol that exceeds the buffer limit).
- 'perf report' output is unchanged before and after this series on every Rust binary I tried it on.
- Built and tested on x86-64 with a 7.0 kernel.
The series is based on perf-tools-next, commit 0ae6fc78c5ce ("perf
timechart: Remove the unused use_old_power_events variable").
Michal Pluta (5):
perf test demangle: Fail when demangling fails
perf symbol: Don't return an unterminated Rust demangle buffer perf symbol: Grow the Rust demangle buffer geometrically perf symbol: Shrink the demangled Rust buffer to fit perf symbol: Move Rust demangling into its own function
tools/perf/tests/demangle-java-test.c | 1 + tools/perf/tests/demangle-rust-v0-test.c | 13 +++++ tools/perf/util/symbol.c | 60 +++++++++++++++--------- 3 files changed, 52 insertions(+), 22 deletions(-)
base-commit: 0ae6fc78c5ce0dfd18d8712a50f0fd4602eff103
Signature · 1 line
2.43.0
PATCH 1/5 perf test demangle: Fail when demangling fails
The Rust and Java demangle tests only log a NULL result from dso__demangle_sym() and carry on, so a symbol that fails to demangle doesn't fail the test. Set TEST_FAIL.
--- tools/perf/tests/demangle-java-test.c | 1 + tools/perf/tests/demangle-rust-v0-test.c | 1 + 2 files changed, 2 insertions(+)
tools/perf/tests/demangle-java-test.c +1 -0
@@ -33,6 +33,7 @@ static int test__demangle_java(struct test_suite *test __maybe_unused, int subteif (!buf) {pr_debug ("FAILED to demangle: \"%s\"\n \"%s\"\n" , test_cases[i].mangled, test_cases[i].demangled); + ret =TEST_FAIL ;continue ; }if (strcmp (buf, test_cases[i].demangled)) {
tools/perf/tests/demangle-rust-v0-test.c +1 -0
@@ -58,6 +58,7 @@ static int test__demangle_rust(struct test_suite *test __maybe_unused, int subteif (!buf) {pr_debug ("FAILED to demangle: \"%s\"\n \"%s\"\n" , test_cases[i].mangled, test_cases[i].demangled); + ret =TEST_FAIL ;continue ; }if (strcmp (buf, test_cases[i].demangled)) {
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH 1/5] perf test demangle: Fail when demangling fails
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
The Rust and Java demangle tests only log a NULL result from
dso__demangle_sym()and carry on, so a symbol that fails to demangle doesn't fail the test. SetTEST_FAIL.Assisted-by LLMSigned-off-by Michal Pluta <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH 1/5] perf test demangle: Fail when demangling fails
In reply to Michal PlutaQuoted 7 lines · Michal Pluta
The Rust and Java demangle tests only log a NULL result from
dso__demangle_sym()and carry on, so a symbol that fails to demangle doesn't fail the test. SetTEST_FAIL.Assisted-by LLMSigned-off-by Michal Pluta <[email protected]>
Thanks!
Ian
Quoted 32 lines
--- tools/perf/tests/demangle-java-test.c | 1 + tools/perf/tests/demangle-rust-v0-test.c | 1 + 2 files changed, 2 insertions(+)tools/perf/tests/demangle-java-test.c +1
-0@@ -33,6 +33,7 @@ static int test__demangle_java(struct test_suite *test __maybe_unused, int subteif (!buf) {pr_debug ("FAILED to demangle: \"%s\"\n \"%s\"\n" , test_cases[i].mangled, test_cases[i].demangled); + ret =TEST_FAIL ;continue ; }if (strcmp (buf, test_cases[i].demangled)) {tools/perf/tests/demangle-rust-v0-test.c +1
-1@@ -58,6 +58,7 @@ static int test__demangle_rust(struct test_suite *test __maybe_unused, int subteif (!buf) {pr_debug ("FAILED to demangle: \"%s\"\n \"%s\"\n" , test_cases[i].mangled, test_cases[i].demangled); + ret =TEST_FAIL ;continue ; }if (strcmp (buf, test_cases[i].demangled)) { --2.43.0
PATCH 2/5 perf symbol: Don't return an unterminated Rust demangle buffer
In reply to Michal PlutaWhen dso__demangle_sym() runs out of retries or fails to grow the output buffer for a Rust v0 symbol, it returns the buffer and callers blindly use it as the demangled symbol name. However, rust_demangle_display_demangle() doesn't NUL terminate its output when it reports OverflowOverflow, so the buffer isn't a string.
Free the buffer and return NULL in both cases. Callers already fall back to the mangled name when no demangled name is returned.
--- tools/perf/util/symbol.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
tools/perf/util/symbol.c +5 -6
@@ -2762,17 +2762,16 @@ char *dso__demangle_sym(struct dso *dso, int kmodule, const char *elf_name) buf_len < 1024 * 1024; buf_len += 32) {char *tmp =realloc (demangled, buf_len); -if (!tmp) { -/* Failure to grow output buffer, return what is there. */ -return demangled; - } +if (!tmp) +break ; demangled = tmp;if (rust_demangle_display_demangle (&rust_demangle, demangled, buf_len,/*alternate=*/ true) == OverflowOk)return demangled; } -/* Buffer exceeded sensible bounds, return what is there. */ -return demangled; +/* Failure to grow output buffer or buffer exceeded sensible bounds. */ +free (demangled); +return NULL ; } demangled =cxx_demangle_sym (elf_name, verbose >0 , verbose >0 );
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH 2/5] perf symbol: Don't return an unterminated Rust demangle buffer
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
When
dso__demangle_sym()runs out of retries or fails to grow the output buffer for a Rust v0 symbol, it returns the buffer and callers blindly use it as the demangled symbol name. However,rust_demangle_display_demangle()doesn't NUL terminate its output when it reports OverflowOverflow, so the buffer isn't a string.
Quoted 6 lines
Free the buffer and return NULL in both cases. Callers already fall back to the mangled name when no demangled name is returned.
Fixes e20848c317b5 ("perf symbol-elf: Integrate rust-v0 demangling")Assisted-by LLMSigned-off-by Michal Pluta <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH 2/5] perf symbol: Don't return an unterminated Rust demangle buffer
In reply to Michal PlutaQuoted 7 lines · Michal Pluta
When
dso__demangle_sym()runs out of retries or fails to grow the output buffer for a Rust v0 symbol, it returns the buffer and callers blindly use it as the demangled symbol name. However,rust_demangle_display_demangle()doesn't NUL terminate its output when it reports OverflowOverflow, so the buffer isn't a string.
Quoted 6 lines
Free the buffer and return NULL in both cases. Callers already fall back to the mangled name when no demangled name is returned.
Fixes e20848c317b5 ("perf symbol-elf: Integrate rust-v0 demangling")Assisted-by LLMSigned-off-by Michal Pluta <[email protected]>
Thanks,
Ian
Quoted 34 lines
--- tools/perf/util/symbol.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)tools/perf/util/symbol.c +5
-7@@ -2762,17 +2762,16 @@ char *dso__demangle_sym(struct dso *dso, int kmodule, const char *elf_name) buf_len < 1024 * 1024; buf_len += 32) {char *tmp =realloc (demangled, buf_len); -if (!tmp) { -/* Failure to grow output buffer, return what is there. */ -return demangled; - } +if (!tmp) +break ; demangled = tmp;if (rust_demangle_display_demangle (&rust_demangle, demangled, buf_len,/*alternate=*/ true) == OverflowOk)return demangled; } -/* Buffer exceeded sensible bounds, return what is there. */ -return demangled; +/* Failure to grow output buffer or buffer exceeded sensible bounds. */ +free (demangled); +return NULL ; } demangled =cxx_demangle_sym (elf_name, verbose >0 , verbose >0 ); --2.43.0
Re: [PATCH 2/5] perf symbol: Don't return an unterminated Rust demangle buffer
Quoted 9 lines · Ian Rogers
Quoted 6 lines · Michal Pluta
Free the buffer and return NULL in both cases. Callers already fall back to the mangled name when no demangled name is returned.
Fixes e20848c317b5 ("perf symbol-elf: Integrate rust-v0 demangling")Assisted-by LLMSigned-off-by Michal Pluta <[email protected]>Reviewed-by Ian Rogers <[email protected]>
Thanks, applied the first two patches to perf-tools-next, for v7.4.
- Arnaldo
PATCH 3/5 perf symbol: Grow the Rust demangle buffer geometrically
In reply to Michal PlutaWhen a demangled Rust name doesn't fit in the provided buffer, dso__demangle_sym() adds 32 bytes to it and formats the name again from the start. The number of attempts grows with the length of the output, so the total work is quadratic. Names with deeply nested generic types need many attempts, leading to noticeable slowdowns in larger programs.
Double the buffer instead, reaching the maximum buffer limit exactly rather than stopping 32 bytes early. The demangled names are unchanged.
Add a test for a symbol whose expansion exceeds the bound.
--- tools/perf/tests/demangle-rust-v0-test.c | 12 ++++++++++++ tools/perf/util/symbol.c | 10 ++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-)
tools/perf/tests/demangle-rust-v0-test.c +12 -0
@@ -69,6 +69,18 @@ static int test__demangle_rust(struct test_suite *test __maybe_unused, int subtefree (buf); } +/* +* A symbol with more lifetimes bound than fit in the largest buffer +* must fail to demangle rather than give a truncated name. +*/ + buf =dso__demangle_sym (/*dso=*/ NULL ,/*kmodule=*/ 0 ,"_RINvC1a1fFGZZZZZZ_EuE" ); +if (buf) { +pr_debug ("FAILED: symbol larger than the buffer limit demangled to %zu bytes\n" , +strlen (buf)); + ret =TEST_FAIL ; +free (buf); + } +return ret; }
tools/perf/util/symbol.c +8 -2
@@ -2737,29 +2737,35 @@ char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse }#endif /* !HAVE_CXA_DEMANGLE_SUPPORT */ +/* Buffer limit for a demangled Rust symbol name. */ +#define RUST_DEMANGLE_MAX_LEN (1024 * 1024) +char *dso__demangle_sym (struct dso *dso,int kmodule,const char *elf_name) {struct demangle rust_demangle = { .style = DemangleStyleUnknown, };char *demangled =NULL ; +size_t buf_len;/* * We need to figure out if the object was created from C++ sources * DWARF DW_compile_unit has this, but we don't always have access * to it... */ if (!want_demangle ((dso &&dso__kernel (dso)) || kmodule))return demangled;rust_demangle_demangle (elf_name, &rust_demangle);if (rust_demangle_is_known (&rust_demangle)) {/* A rust mangled name. */ if (rust_demangle.mangled_len ==0 )return demangled; -for (size_t buf_len =roundup_pow_of_two (rust_demangle.mangled_len *2 ); - buf_len <1024 *1024 ; buf_len +=32 ) { +for (buf_len =min_t (size_t,roundup_pow_of_two (rust_demangle.mangled_len *2 ), +RUST_DEMANGLE_MAX_LEN ); + buf_len <=RUST_DEMANGLE_MAX_LEN ; + buf_len *=2 ) {char *tmp =realloc (demangled, buf_len);if (!tmp)
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH 3/5] perf symbol: Grow the Rust demangle buffer geometrically
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
When a demangled Rust name doesn't fit in the provided buffer,
dso__demangle_sym()adds 32 bytes to it and formats the name again from the start. The number of attempts grows with the length of the output, so the total work is quadratic. Names with deeply nested generic types need many attempts, leading to noticeable slowdowns in larger programs.
Quoted 7 lines
Double the buffer instead, reaching the maximum buffer limit exactly rather than stopping 32 bytes early. The demangled names are unchanged.
Add a test for a symbol whose expansion exceeds the bound.
Signed-off-by Michal Pluta <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Ian Rogers
reviewerRe: [PATCH 3/5] perf symbol: Grow the Rust demangle buffer geometrically
In reply to Michal PlutaQuoted 50 lines · Michal Pluta
When a demangled Rust name doesn't fit in the provided buffer,
dso__demangle_sym()adds 32 bytes to it and formats the name again from the start. The number of attempts grows with the length of the output, so the total work is quadratic. Names with deeply nested generic types need many attempts, leading to noticeable slowdowns in larger programs.Double the buffer instead, reaching the maximum buffer limit exactly rather than stopping 32 bytes early. The demangled names are unchanged.
Add a test for a symbol whose expansion exceeds the bound.
Signed-off-by Michal Pluta <[email protected]>--- tools/perf/tests/demangle-rust-v0-test.c | 12 ++++++++++++ tools/perf/util/symbol.c | 10 ++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-)tools/perf/tests/demangle-rust-v0-test.c +12
-0@@ -69,6 +69,18 @@ static int test__demangle_rust(struct test_suite *test __maybe_unused, int subtefree (buf); } +/* +* A symbol with more lifetimes bound than fit in the largest buffer +* must fail to demangle rather than give a truncated name. +*/ + buf =dso__demangle_sym (/*dso=*/ NULL ,/*kmodule=*/ 0 ,"_RINvC1a1fFGZZZZZZ_EuE" ); +if (buf) { +pr_debug ("FAILED: symbol larger than the buffer limit demangled to %zu bytes\n" , +strlen (buf)); + ret =TEST_FAIL ; +free (buf); + } +return ret; }tools/perf/util/symbol.c +0
-0@@ -2737,29 +2737,35 @@ char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse }#endif /* !HAVE_CXA_DEMANGLE_SUPPORT */
tools/perf/util/symbol.c char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse · line 2749
Quoted 10 lines
tools/perf/util/symbol.c +4
-0+/* Buffer limit for a demangled Rust symbol name. */ +#define RUST_DEMANGLE_MAX_LEN (1024 * 1024) +char *dso__demangle_sym (struct dso *dso,int kmodule,const char *elf_name) {struct demangle rust_demangle = { .style = DemangleStyleUnknown, };char *demangled =NULL ; +size_t buf_len;
It seems changing the scope of this variable is unnecessary.
Quoted 11 lines
tools/perf/util/symbol.c +0
-0/* * We need to figure out if the object was created from C++ sources * DWARF DW_compile_unit has this, but we don't always have access * to it... */ if (!want_demangle ((dso &&dso__kernel (dso)) || kmodule))return demangled;rust_demangle_demangle (elf_name, &rust_demangle);if (rust_demangle_is_known (&rust_demangle)) {
tools/perf/util/symbol.c char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse · line 2768
Quoted 10 lines
tools/perf/util/symbol.c +4
-2/* A rust mangled name. */ if (rust_demangle.mangled_len ==0 )return demangled; -for (size_t buf_len =roundup_pow_of_two (rust_demangle.mangled_len *2 ); - buf_len <1024 *1024 ; buf_len +=32 ) { +for (buf_len =min_t (size_t,roundup_pow_of_two (rust_demangle.mangled_len *2 ), +RUST_DEMANGLE_MAX_LEN ); + buf_len <=RUST_DEMANGLE_MAX_LEN ; + buf_len *=2 ) {
Thanks for digging into this problem and exploring a fix! Previously, we guessed the demangled length was twice the mangled length, then added 32 bytes for each retry. These were numbers I pulled out of thin air, so I'm glad you've found them to be wrong :-). Could we estimate the initial demangled size better? Could you get data from Bevy and Typst? I'm a little concerned that a demangled symbol of say just over 2KB might require 4KB with this change, instead of 2KB + 32bytes.
Thanks!
Ian
Quoted 6 lines
tools/perf/util/symbol.c +0
-1char *tmp =realloc (demangled, buf_len);if (!tmp) --2.43.0
Namhyung Kim
maintainerRe: [PATCH 3/5] perf symbol: Grow the Rust demangle buffer geometrically
Quoted 86 lines · Ian Rogers
Quoted 50 lines · Michal Pluta
When a demangled Rust name doesn't fit in the provided buffer,
dso__demangle_sym()adds 32 bytes to it and formats the name again from the start. The number of attempts grows with the length of the output, so the total work is quadratic. Names with deeply nested generic types need many attempts, leading to noticeable slowdowns in larger programs.Double the buffer instead, reaching the maximum buffer limit exactly rather than stopping 32 bytes early. The demangled names are unchanged.
Add a test for a symbol whose expansion exceeds the bound.
Signed-off-by Michal Pluta <[email protected]>--- tools/perf/tests/demangle-rust-v0-test.c | 12 ++++++++++++ tools/perf/util/symbol.c | 10 ++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-)tools/perf/tests/demangle-rust-v0-test.c +12
-0@@ -69,6 +69,18 @@ static int test__demangle_rust(struct test_suite *test __maybe_unused, int subtefree (buf); } +/* +* A symbol with more lifetimes bound than fit in the largest buffer +* must fail to demangle rather than give a truncated name. +*/ + buf =dso__demangle_sym (/*dso=*/ NULL ,/*kmodule=*/ 0 ,"_RINvC1a1fFGZZZZZZ_EuE" ); +if (buf) { +pr_debug ("FAILED: symbol larger than the buffer limit demangled to %zu bytes\n" , +strlen (buf)); + ret =TEST_FAIL ; +free (buf); + } +return ret; }tools/perf/util/symbol.c +0
-0@@ -2737,29 +2737,35 @@ char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse }#endif /* !HAVE_CXA_DEMANGLE_SUPPORT */ tools/perf/util/symbol.c char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse · line 2749
Quoted 10 lines
tools/perf/util/symbol.c +4
-0+/* Buffer limit for a demangled Rust symbol name. */ +#define RUST_DEMANGLE_MAX_LEN (1024 * 1024) +char *dso__demangle_sym (struct dso *dso,int kmodule,const char *elf_name) {struct demangle rust_demangle = { .style = DemangleStyleUnknown, };char *demangled =NULL ; +size_t buf_len;It seems changing the scope of this variable is unnecessary.
Quoted 11 lines
tools/perf/util/symbol.c +0
-0/* * We need to figure out if the object was created from C++ sources * DWARF DW_compile_unit has this, but we don't always have access * to it... */ if (!want_demangle ((dso &&dso__kernel (dso)) || kmodule))return demangled;rust_demangle_demangle (elf_name, &rust_demangle);if (rust_demangle_is_known (&rust_demangle)) {tools/perf/util/symbol.c char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse · line 2768
Quoted 10 lines
tools/perf/util/symbol.c +4
-2/* A rust mangled name. */ if (rust_demangle.mangled_len ==0 )return demangled; -for (size_t buf_len =roundup_pow_of_two (rust_demangle.mangled_len *2 ); - buf_len <1024 *1024 ; buf_len +=32 ) { +for (buf_len =min_t (size_t,roundup_pow_of_two (rust_demangle.mangled_len *2 ), +RUST_DEMANGLE_MAX_LEN ); + buf_len <=RUST_DEMANGLE_MAX_LEN ; + buf_len *=2 ) {
Quoted 7 lines
Thanks for digging into this problem and exploring a fix! Previously, we guessed the demangled length was twice the mangled length, then added 32 bytes for each retry. These were numbers I pulled out of thin air, so I'm glad you've found them to be wrong :-). Could we estimate the initial demangled size better? Could you get data from Bevy and Typst? I'm a little concerned that a demangled symbol of say just over 2KB might require 4KB with this change, instead of 2KB + 32bytes.
I guess it's hard to predict a good initial size as backrefs can make long strings easily. If we really care about the memory usage, how about calling realloc() for the actual length at the end?
Thanks,
Namhyung
Re: [PATCH 3/5] perf symbol: Grow the Rust demangle buffer geometrically
Quoted 10 lines · Ian Rogers
It seems changing the scope of this variable is unnecessary.
Thanks for digging into this problem and exploring a fix! Previously, we guessed the demangled length was twice the mangled length, then added 32 bytes for each retry. These were numbers I pulled out of thin air, so I'm glad you've found them to be wrong :-). Could we estimate the initial demangled size better? Could you get data from Bevy and Typst? I'm a little concerned that a demangled symbol of say just over 2KB might require 4KB with this change, instead of 2KB + 32bytes.
Thank you for the reviews. I'll move the buf_len declaration back to where it was before.
For context, the other demanglers in perf don't size their buffer by retrying. The OCaml one allocates len + 1, as the output is never longer than the input. The Java one allocates strlen * 3 + 1 and truncates. The C++ demangler (libiberty) starts with an empty buffer and doubles it, but it writes through a callback so growing doesn't restart the formatting. The Rust one writes into a caller provided buffer and has to start over each time it is too small.
I collected some data from release builds of Bevy and Typst and a few other large(r) Rust projects. For each binary I took the defined symbols and counted the number of attempts that the buffer loop in dso__demangle_sym() makes for them. I ran both the current +32 loop and the doubling one from this series in a small test program. No symbol went over the 1 MiB limit.
Builds (mostly v0 symbols, plus up to 148 C++ ones per binary from linked libraries):
bevy 0f38358f573a cargo +1.98.1 build --example 3d_scene
--release --config profile.release.strip=false
datafusion acf5c89e6454 cargo +1.98.1 build --locked -p datafusion-cli
--profile release-nonlto
materialize 54a6aca7b5f1 bin/environmentd +1.98.1 --build-only --optimized
(clusterd and environmentd)
polars 750bbaa9054c cargo +nightly-2026-09-01 build --locked
-p polars-dylib --profile fast-release
--config profile.fast-release.debug=0
--config profile.fast-release.strip=false
risingwave be77a6149e97 cargo +nightly-2026-06-21 build --locked
-p risingwave_cmd_all --bin risingwave --release
--config profile.release.debug=0
--config profile.release.strip=false
typst 9dfd3a08500b cargo +1.98.1 build --locked -p typst-cli --release
(v0.15.1) --config profile.release.package.typst-cli.strip=falseFirstly some data on how often initial guesses are too small and how many retries they cause. "retried symbols" is the share of symbols whose output did not fit in the initial buffer. The other columns count the extra attempts after the first one ("total"), and the most any single symbol needed ("max"):
symbols retried +32 loop doubling
symbols total max total max
materialize clusterd 295,365 5.66% 3,500,913 17,844 27,960 8
risingwave 1,051,376 0.87% 297,151 1,561 10,180 5
polars libpolars_dylib 276,797 5.61% 247,424 236 16,149 3
bevy 3d_scene 218,042 1.09% 205,030 1,135 3,493 5
materialize environmentd 377,085 0.51% 117,485 1,539 2,302 5
datafusion-cli 175,020 1.10% 24,304 125 1,957 3
typst 36,504 0.12% 907 66 50 2Only 0.1% to 5.7% don't fit in the initial buffer, but with +32 this is very costly. In clusterd it adds up to 3.5 million extra attempts with one symbol needing 17,844 retries. Doubling does the same job in 27,960 total retries and no symbol needs more than 8 retries.
You asked whether a better initial guess would help, so here I keep the +32 loop and change only the initial buffer. Its size is the mangled length multiplied by a factor N and rounded up to a power of two. The code uses 2x today, so 1x is a smaller guess than today and 4x and 8x are larger ones. Each value is the total extra attempts for that binary:
N 1x 2x 4x 8x
materialize clusterd 4,507,480 3,500,913 2,460,193 1,528,046 risingwave 1,306,091 297,151 71,074 25,133 polars libpolars_dylib 1,254,174 247,424 14,214 305 bevy 3d_scene 357,175 205,030 94,936 29,978 materialize environmentd 271,354 117,485 59,697 22,265 datafusion-cli 179,241 24,304 1,003 29
typst 7,066 907 15 0
A bigger guess reduces the retries for every binary, but the symbols that still miss are the ones with many backrefs. In every binary, all the symbols that missed the initial buffer have backrefs, with a median of 53-138 expansions compared to 1-4 for the symbols that fit.
Running the same experiment with the doubling loop from this series:
1x 2x 4x 8x
materialize clusterd 70,399 27,960 11,231 4,073 risingwave 106,878 10,180 1,016 128 polars libpolars_dylib 84,502 16,149 634 9 bevy 3d_scene 15,742 3,493 1,115 192 materialize environmentd 22,668 2,302 365 73 datafusion-cli 15,366 1,957 38 1
typst 2,310 50 5 0
On the 2KB example you mentioned, a symbol that barely misses a power of two could get a buffer up to twice what it needs, but the next patch shrinks it to the exact string length before returning, so the extra memory is only held during the call. I'll move the realloc patch ahead of this one.
One other way I tried to avoid retries is a minimum size for the initial buffer. Here the initial buffer is the larger of this minimum and the current 2x guess, and each cell is the total extra attempts.
First with the +32 loop:
current 1KiB 4KiB 16KiB 64KiB
materialize clusterd 3,500,913 3,500,867 3,181,650 1,425,379 216,505
risingwave 297,151 296,929 151,909 14,567 0 polars libpolars_dylib 247,424 246,715 15,413 0 0 bevy 3d_scene 205,030 204,818 132,911 17,038 0 materialize environmentd 117,485 117,455 79,073 37,531 0 datafusion-cli 24,304 24,277 866 0 0 typst 907 898 2 0 0
And with doubling:
current 1KiB 4KiB 16KiB 64KiB
materialize clusterd 27,960 27,945 18,824 3,260 183 risingwave 10,180 10,117 1,887 63 0 polars libpolars_dylib 16,149 16,043 686 0 0 bevy 3d_scene 3,493 3,472 1,574 64 0 materialize environmentd 2,302 2,282 419 88 0 datafusion-cli 1,957 1,939 27 0 0 typst 50 47 1 0 0
As we can see, a minimum does help but it depends a lot on the size chosen and would mean we incur a large allocation per-symbol albeit only during the call.
Sorry for the long reply, just wanted to get all the data and rationale across.
Fun fact: the largest symbol I encountered was in materialize's clusterd. It is 1,091 bytes mangled and demangles to 575,074 bytes.
Thanks,
Michal
Ian Rogers
reviewerRe: [PATCH 3/5] perf symbol: Grow the Rust demangle buffer geometrically
Quoted 136 lines · Michal Pluta
Quoted 10 lines · Ian Rogers
It seems changing the scope of this variable is unnecessary.
Thanks for digging into this problem and exploring a fix! Previously, we guessed the demangled length was twice the mangled length, then added 32 bytes for each retry. These were numbers I pulled out of thin air, so I'm glad you've found them to be wrong :-). Could we estimate the initial demangled size better? Could you get data from Bevy and Typst? I'm a little concerned that a demangled symbol of say just over 2KB might require 4KB with this change, instead of 2KB + 32bytes.
Thank you for the reviews. I'll move the
buf_lendeclaration back to where it was before.For context, the other demanglers in perf don't size their buffer by retrying. The OCaml one allocates len + 1, as the output is never longer than the input. The Java one allocates strlen * 3 + 1 and truncates. The C++ demangler (libiberty) starts with an empty buffer and doubles it, but it writes through a callback so growing doesn't restart the formatting. The Rust one writes into a caller provided buffer and has to start over each time it is too small.
I collected some data from release builds of Bevy and Typst and a few other large(r) Rust projects. For each binary I took the defined symbols and counted the number of attempts that the buffer loop in
dso__demangle_sym()makes for them. I ran both the current +32 loop and the doubling one from this series in a small test program. No symbol went over the 1 MiB limit.Builds (mostly v0 symbols, plus up to 148 C++ ones per binary from linked libraries):
bevy 0f38358f573a cargo +1.98.1 build --example 3d_scene --release --config profile.release.strip=false datafusion acf5c89e6454 cargo +1.98.1 build --locked -p datafusion-cli --profile release-nonlto materialize 54a6aca7b5f1 bin/environmentd +1.98.1 --build-only --optimized (clusterd and environmentd) polars 750bbaa9054c cargo +nightly-2026-09-01 build --locked -p polars-dylib --profile fast-release --config profile.fast-release.debug=0 --config profile.fast-release.strip=false risingwave be77a6149e97 cargo +nightly-2026-06-21 build --locked -p risingwave_cmd_all --bin risingwave --release --config profile.release.debug=0 --config profile.release.strip=false typst 9dfd3a08500b cargo +1.98.1 build --locked -p typst-cli --release (v0.15.1) --config profile.release.package.typst-cli.strip=falseFirstly some data on how often initial guesses are too small and how many retries they cause. "retried symbols" is the share of symbols whose output did not fit in the initial buffer. The other columns count the extra attempts after the first one ("total"), and the most any single symbol needed ("max"):
symbols retried +32 loop doubling symbols total max total max materialize clusterd 295,365 5.66% 3,500,913 17,844 27,960 8 risingwave 1,051,376 0.87% 297,151 1,561 10,180 5 polars libpolars_dylib 276,797 5.61% 247,424 236 16,149 3 bevy 3d_scene 218,042 1.09% 205,030 1,135 3,493 5 materialize environmentd 377,085 0.51% 117,485 1,539 2,302 5 datafusion-cli 175,020 1.10% 24,304 125 1,957 3 typst 36,504 0.12% 907 66 50 2Only 0.1% to 5.7% don't fit in the initial buffer, but with +32 this is very costly. In clusterd it adds up to 3.5 million extra attempts with one symbol needing 17,844 retries. Doubling does the same job in 27,960 total retries and no symbol needs more than 8 retries.
You asked whether a better initial guess would help, so here I keep the +32 loop and change only the initial buffer. Its size is the mangled length multiplied by a factor N and rounded up to a power of two. The code uses 2x today, so 1x is a smaller guess than today and 4x and 8x are larger ones. Each value is the total extra attempts for that binary:
N 1x 2x 4x 8x
materialize clusterd 4,507,480 3,500,913 2,460,193 1,528,046 risingwave 1,306,091 297,151 71,074 25,133 polars libpolars_dylib 1,254,174 247,424 14,214 305 bevy 3d_scene 357,175 205,030 94,936 29,978 materialize environmentd 271,354 117,485 59,697 22,265 datafusion-cli 179,241 24,304 1,003 29typst 7,066 907 15 0
A bigger guess reduces the retries for every binary, but the symbols that still miss are the ones with many backrefs. In every binary, all the symbols that missed the initial buffer have backrefs, with a median of 53-138 expansions compared to 1-4 for the symbols that fit.
Running the same experiment with the doubling loop from this series:
1x 2x 4x 8x
materialize clusterd 70,399 27,960 11,231 4,073 risingwave 106,878 10,180 1,016 128 polars libpolars_dylib 84,502 16,149 634 9 bevy 3d_scene 15,742 3,493 1,115 192 materialize environmentd 22,668 2,302 365 73 datafusion-cli 15,366 1,957 38 1typst 2,310 50 5 0
On the 2KB example you mentioned, a symbol that barely misses a power of two could get a buffer up to twice what it needs, but the next patch shrinks it to the exact string length before returning, so the extra memory is only held during the call. I'll move the realloc patch ahead of this one.
One other way I tried to avoid retries is a minimum size for the initial buffer. Here the initial buffer is the larger of this minimum and the current 2x guess, and each cell is the total extra attempts.
First with the +32 loop:
current 1KiB 4KiB 16KiB 64KiB
materialize clusterd 3,500,913 3,500,867 3,181,650 1,425,379 216,505risingwave 297,151 296,929 151,909 14,567 0 polars libpolars_dylib 247,424 246,715 15,413 0 0 bevy 3d_scene 205,030 204,818 132,911 17,038 0 materialize environmentd 117,485 117,455 79,073 37,531 0 datafusion-cli 24,304 24,277 866 0 0 typst 907 898 2 0 0And with doubling:
current 1KiB 4KiB 16KiB 64KiBmaterialize clusterd 27,960 27,945 18,824 3,260 183 risingwave 10,180 10,117 1,887 63 0 polars libpolars_dylib 16,149 16,043 686 0 0 bevy 3d_scene 3,493 3,472 1,574 64 0 materialize environmentd 2,302 2,282 419 88 0 datafusion-cli 1,957 1,939 27 0 0 typst 50 47 1 0 0As we can see, a minimum does help but it depends a lot on the size chosen and would mean we incur a large allocation per-symbol albeit only during the call.
Quoted 5 lines
Sorry for the long reply, just wanted to get all the data and rationale across.
Fun fact: the largest symbol I encountered was in materialize's clusterd. It is 1,091 bytes mangled and demangles to 575,074 bytes.
Thanks, Michal and thanks for the long reply and data!
I wasn't sure if you saw any patterns when comparing the unmangled and mangled sizes? What is the typical rust symbol size? I'm concerned that reallocating to return memory to the heap might just be treated as a no-op, so guessing the size correctly would be great. It would also be good if rather than just failing to parse the parser could compute the output size, so we retry a maximum of once (snprintf behaves like this).
Thanks,
Ian
Quoted 2 lines
Thanks,
Michal
PATCH 4/5 perf symbol: Shrink the demangled Rust buffer to fit
In reply to Michal PlutaThe buffer a demangled Rust name is returned in is sized to whatever attempt succeeded, which is usually larger than the string it holds. Shrink it to the string's exact length before returning it.
--- tools/perf/util/symbol.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
tools/perf/util/symbol.c +4 -2
@@ -2772,8 +2772,10 @@ char *dso__demangle_sym(struct dso *dso, int kmodule, const char *elf_name)break ; demangled = tmp;if (rust_demangle_display_demangle (&rust_demangle, demangled, buf_len, -/*alternate=*/ true) == OverflowOk) -return demangled; +/*alternate=*/ true) == OverflowOk) { + tmp =realloc (demangled,strlen (demangled) +1 ); +return tmp ?: demangled; + } }/* Failure to grow output buffer or buffer exceeded sensible bounds. */ free (demangled);
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH 4/5] perf symbol: Shrink the demangled Rust buffer to fit
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines
The buffer a demangled Rust name is returned in is sized to whatever attempt succeeded, which is usually larger than the string it holds. Shrink it to the string's exact length before returning it.
Signed-off-by Michal Pluta <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Namhyung Kim
maintainerRe: [PATCH 4/5] perf symbol: Shrink the demangled Rust buffer to fit
In reply to Michal PlutaQuoted 3 lines · Michal Pluta
The buffer a demangled Rust name is returned in is sized to whatever attempt succeeded, which is usually larger than the string it holds. Shrink it to the string's exact length before returning it.
Oh, you already have the patch! :)
Thanks,
Namhyung
Quoted 26 lines
Signed-off-by Michal Pluta <[email protected]>--- tools/perf/util/symbol.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)tools/perf/util/symbol.c +4
-2@@ -2772,8 +2772,10 @@ char *dso__demangle_sym(struct dso *dso, int kmodule, const char *elf_name)break ; demangled = tmp;if (rust_demangle_display_demangle (&rust_demangle, demangled, buf_len, -/*alternate=*/ true) == OverflowOk) -return demangled; +/*alternate=*/ true) == OverflowOk) { + tmp =realloc (demangled,strlen (demangled) +1 ); +return tmp ?: demangled; + } }/* Failure to grow output buffer or buffer exceeded sensible bounds. */ free (demangled);Signature · 1 line
2.43.0
PATCH 5/5 perf symbol: Move Rust demangling into its own function
In reply to Michal Plutadso__demangle_sym() is the only place that implements demangling logic inline. The other three are one call each to a helper. Move the Rust parsing and buffer growth into rust_demangle_sym(), with the same signature as the other three.
A Rust name that fails to demangle now falls through to the other demanglers, matching how the other demanglers chain into each other. A v0 name can't be misread by any of them, since none accepts its "_R" prefix. A legacy name can only fail here on allocation failure or hitting the output bound, neither of which is language-specific.
--- tools/perf/util/symbol.c | 61 +++++++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 26 deletions(-)
tools/perf/util/symbol.c +35 -26
@@ -2740,47 +2740,56 @@ char *cxx_demangle_sym(const char *str __maybe_unused, bool params __maybe_unuse/* Buffer limit for a demangled Rust symbol name. */ #define RUST_DEMANGLE_MAX_LEN (1024 * 1024) -char *dso__demangle_sym (struct dso *dso,int kmodule,const char *elf_name) +static char *rust_demangle_sym (const char *elf_name) {struct demangle rust_demangle = { .style = DemangleStyleUnknown, };char *demangled =NULL ;size_t buf_len; +rust_demangle_demangle (elf_name, &rust_demangle); +if (!rust_demangle_is_known (&rust_demangle)) +return NULL ; + +if (rust_demangle.mangled_len ==0 ) +return NULL ; + +for (buf_len =min_t (size_t,roundup_pow_of_two (rust_demangle.mangled_len *2 ), +RUST_DEMANGLE_MAX_LEN ); + buf_len <=RUST_DEMANGLE_MAX_LEN ; + buf_len *=2 ) { +char *tmp =realloc (demangled, buf_len); + +if (!tmp) +break ; + demangled = tmp; +if (rust_demangle_display_demangle (&rust_demangle, demangled, buf_len, +/*alternate=*/ true) == OverflowOk) { + tmp =realloc (demangled,strlen (demangled) +1 ); +return tmp ?: demangled; + } + } +/* Failure to grow output buffer or buffer exceeded sensible bounds. */ +free (demangled); +return NULL ; +} + +char *dso__demangle_sym (struct dso *dso,int kmodule,const char *elf_name) +{ +char *demangled =NULL ; +/* * We need to figure out if the object was created from C++ sources * DWARF DW_compile_unit has this, but we don't always have access * to it... */ if (!want_demangle ((dso &&dso__kernel (dso)) || kmodule))return demangled; -rust_demangle_demangle (elf_name, &rust_demangle); -if (rust_demangle_is_known (&rust_demangle)) { -/* A rust mangled name. */ -if (rust_demangle.mangled_len ==0 ) -return demangled; - -for (buf_len =min_t (size_t,roundup_pow_of_two (rust_demangle.mangled_len *2 ), -RUST_DEMANGLE_MAX_LEN ); - buf_len <=RUST_DEMANGLE_MAX_LEN ; - buf_len *=2 ) { -char *tmp =realloc (demangled, buf_len); - -if (!tmp) -break ; - demangled = tmp; -if (rust_demangle_display_demangle (&rust_demangle, demangled, buf_len, -/*alternate=*/ true) == OverflowOk) { - tmp =realloc (demangled,strlen (demangled) +1 ); -return tmp ?: demangled; - } - } -/* Failure to grow output buffer or buffer exceeded sensible bounds. */ -free (demangled); -return NULL ; - } + demangled =rust_demangle_sym (elf_name); +if (demangled) +return demangled; demangled =cxx_demangle_sym (elf_name, verbose >0 , verbose >0 );if (demangled)
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH 5/5] perf symbol: Move Rust demangling into its own function
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines
dso__demangle_sym()is the only place that implements demangling logic inline. The other three are one call each to a helper. Move the Rust parsing and buffer growth intorust_demangle_sym(), with the same signature as the other three.
Quoted 8 lines
A Rust name that fails to demangle now falls through to the other demanglers, matching how the other demanglers chain into each other. A v0 name can't be misread by any of them, since none accepts its "_R" prefix. A legacy name can only fail here on allocation failure or hitting the output bound, neither of which is language-specific.
Assisted-by LLMSigned-off-by Michal Pluta <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Quoted 30 lines · Michal Pluta
dso__demangle_sym()demangles Rust v0 symbols itself instead of using a helper like the other demanglers. When a demangled name doesn't fit the buffer, it grows the buffer by a fixed 32 bytes and reformats the whole name from scratch. The number of attempts grows with the name's length, so the total work is quadratic. Rust programs with deeply nested generics can have very long demangled names, and I noticed this as real time spent in perf report on perf.data recorded from launching Bevy and Typst.Patch 3 is the actual optimisation, doubling the size of the buffer instead. Patch 5 moves the Rust-specific logic into its own function, matching how the other demanglers are called. I think the cleanup is worthwhile but I understand it comes down to personal taste and the patch can be skipped if maintainers would rather keep
dso__demangle_sym()as is.I used an LLM to review my patches and find the issues with the demangle tests and the unterminated buffer bug.
Testing:
- 'perf test' passes, including the demangle Java, OCaml and Rust suites (patch 3 adds a case for a symbol that exceeds the buffer limit).
- 'perf report' output is unchanged before and after this series on every Rust binary I tried it on.
- Built and tested on x86-64 with a 7.0 kernel.The series is based on perf-tools-next, commit
0ae6fc78c5ce("perf
timechart: Remove the unuseduse_old_power_eventsvariable").
Quoted 6 lines
Michal Pluta (5):
perf test demangle: Fail when demangling fails
perf symbol: Don't return an unterminated Rust demangle buffer perf symbol: Grow the Rust demangle buffer geometrically perf symbol: Shrink the demangled Rust buffer to fit perf symbol: Move Rust demangling into its own function
Thanks,
Namhyung
Quoted 11 lines
tools/perf/tests/demangle-java-test.c | 1 + tools/perf/tests/demangle-rust-v0-test.c | 13 +++++ tools/perf/util/symbol.c | 60 +++++++++++++++--------- 3 files changed, 52 insertions(+), 22 deletions(-)base-commit:
0ae6fc78c5ce0dfd18d8712a50f0fd4602eff103Signature · 1 line
2.43.0