perf record: Fix teardown hang on system-wide multi-threaded sessions
Ian Rogers
reviewerPATCH 1/2 perf record: Fix teardown hang on system-wide multi-threaded sessions
Under system-wide (-a) parallel streaming mode (--threads=cpu), background recording threads can be inundated by a continuous firehose of hardware samples generated by the OS. In this state, a background thread's local hit count remains unequal to its sample count, causing it to bypass the blocking fdarray__poll() call entirely on each iteration of its recording loop.
Because the termination check relies on the POLLHUP event status populated specifically by fdarray__poll(), bypassing it prevents the background thread from ever recognizing that its control pipe was closed by the main thread. This traps the background thread in an infinite recording loop, hanging the main thread indefinitely as it awaits a termination acknowledgment that never arrives.
Ensure teardown completion by adding explicit evlist__disable() calls in the main thread's cleanup paths at out_child: and out_child_no_flush:. Additionally, patch fdarray__filter() to respect the fdarray_flag__nonfilterable flag, preventing it from incorrectly setting the background thread's control pipe file descriptor to -1 and clearing its revents mask upon processing termination POLLHUP signals.
--- tools/lib/api/fd/array.c | 6 ++++-- tools/perf/builtin-record.c | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-)
tools/lib/api/fd/array.c +4 -2
@@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { +if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +continue ; +if (!fda->entries[fd].events)continue ;
@@ -132,8 +135,7 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } -if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) - ++nr; + ++nr; }return nr;
tools/perf/builtin-record.c +2 -0
@@ -2890,11 +2890,13 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)record__synthesize_workload (rec, true); out_child: +evlist__disable (rec->evlist);record__stop_threads (rec);record__mmap_read_all (rec, true); goto out_free_threads; out_child_no_flush:/* mmap read already failed — retrying would just fail again */ +evlist__disable (rec->evlist);record__stop_threads (rec); out_free_threads:record__free_thread_data (rec);
Signature · 1 line
2.55.0.795.g602f6c329a-goog
Ian Rogers
reviewerPATCH 2/2 perf cap: If capability is missing still perform root test
Refine the capability checking logic in perf_cap__capable() to in the event the capability is missing do the fallback used_root test. Previously this was only performed when the capability syscall failed.
--- tools/perf/util/cap.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/perf/util/cap.c +12 -2
@@ -20,6 +20,7 @@ bool perf_cap__capable(int cap, bool *used_root) };struct __user_cap_data_struct data[MAX_LINUX_CAPABILITY_U32S ] = {};__u32 cap_val; +bool has_cap = false; *used_root = false;while (syscall (SYS_capget, &header, &data[0 ]) ==-1 ) {
@@ -37,12 +38,21 @@ bool perf_cap__capable(int cap, bool *used_root)if (cap >=32 ) {if (header.version == _LINUX_CAPABILITY_VERSION_3) { cap_val = data[1 ].effective; + has_cap = (cap_val & (1U << (cap &0x1f ))) !=0 ; }else {/* Capability beyond 32 is requested but only 32 are supported. */ -return false; + has_cap = false; } }else { cap_val = data[0 ].effective; + has_cap = (cap_val & (1U << (cap &0x1f ))) !=0 ; } -return (cap_val & (1 << (cap &0x1f ))) !=0 ; + +if (has_cap) { + *used_root = false; +return true; + } + + *used_root = true; +return geteuid () ==0 ; }
Signature · 1 line
2.55.0.795.g602f6c329a-goog
[email protected]
AI reviewRe: [PATCH v1 2/2] perf cap: If capability is missing still perform root test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The capability fallback logic overrides valid capability checks for root users and incorrectly…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The capability fallback logic overrides valid capability checks for root users and incorrectly short-circuits sequential capability checks for non-root users by improperly setting the used_root flag. --
--- Patch [2]: [PATCH v1 2/2] perf cap: If capability is missing still perform root test --- commit 8515c6747c6ffb6d27af14829f03c002334e5c67
Author: Ian Rogers <[email protected]>
perf cap: If capability is missing still perform root test
This commit refines the capability checking logic in perf_cap__capable() to fallback to the used_root test if a capability is missing, rather than only doing so when the capget syscall fails.
Quoted 3 lines
tools/perf/util/cap.c +0
-0
[ ... ]
Quoted 14 lines
tools/perf/util/cap.c +3
-1@@ -37,12 +38,21 @@ bool perf_cap__capable(int cap, bool *used_root)if (cap >=32 ) {if (header.version == _LINUX_CAPABILITY_VERSION_3) { cap_val = data[1 ].effective; + has_cap = (cap_val & (1U << (cap &0x1f ))) !=0 ; }else {/* Capability beyond 32 is requested but only 32 are supported. */ -return false; + has_cap = false; } }else { cap_val = data[0 ].effective; + has_cap = (cap_val & (1U << (cap &0x1f ))) !=0 ; }
tools/perf/util/cap.c bool perf_cap__capable(int cap, bool *used_root) · line 58
Quoted 10 lines
tools/perf/util/cap.c +8
-1-return (cap_val & (1 << (cap &0x1f ))) !=0 ; + +if (has_cap) { + *used_root = false; +return true; + } + + *used_root = true; +return geteuid () ==0 ; }
[Severity: High]
Does setting *used_root = true unconditionally here when a capability is missing break sequential capability checks for non-root users?
For example, if an unprivileged user lacks CAP_SYS_ADMIN, this will return false but set used_root to true. In callers like perf_event_paranoid_check():
tools/perf/util/util.c:perf_event_paranoid_check() {
...
if (!used_root && perf_cap__capable(CAP_PERFMON, &used_root))
...
}If a previous check for CAP_SYS_ADMIN failed and poisoned the used_root flag, won't it cause the subsequent check for CAP_PERFMON to be skipped entirely?
This pattern appears in other locations as well, such as check_bpf_filter_capable() where a failed CAP_BPF check might skip the fallback for non-root users to access root-pinned BPF filters because !used_root evaluates to false.
Additionally, could this override valid capability drops for root users in containers? Because geteuid() == 0 evaluates to true, it seems this might falsely claim root users still possess the capability even if it was explicitly dropped, potentially leading to later failures instead of graceful fallbacks.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Namhyung Kim
maintainerRe: [PATCH v1 2/2] perf cap: If capability is missing still perform root test
Quoted 8 lines · On Fri, Jul 10, 2026 at 05:47:10AM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The capability fallback logic overrides valid capability checks for root users and incorrectly short-circuits sequential capability checks for non-root users by improperly setting theused_rootflag. ----- Patch [2]: [PATCH v1 2/2] perf cap: If capability is missing still perform root test --- commit
8515c6747c6ffb6d27af14829f03c002334e5c67
Author: Ian Rogers <[email protected]>
Quoted 5 lines
perf cap: If capability is missing still perform root test
This commit refines the capability checking logic in
perf_cap__capable()to fallback to theused_roottest if a capability is missing, rather than only doing so when the capget syscall fails.
I think we can process this independently. It's not related to patch 1, right?
Quoted 54 lines
Quoted 4 lines
tools/perf/util/cap.c +0
-0[ ... ]
Quoted 14 lines
tools/perf/util/cap.c +3
-1@@ -37,12 +38,21 @@ bool perf_cap__capable(int cap, bool *used_root)if (cap >=32 ) {if (header.version == _LINUX_CAPABILITY_VERSION_3) { cap_val = data[1 ].effective; + has_cap = (cap_val & (1U << (cap &0x1f ))) !=0 ; }else {/* Capability beyond 32 is requested but only 32 are supported. */ -return false; + has_cap = false; } }else { cap_val = data[0 ].effective; + has_cap = (cap_val & (1U << (cap &0x1f ))) !=0 ; }tools/perf/util/cap.c bool perf_cap__capable(int cap, bool *used_root) · line 58
Quoted 10 lines
tools/perf/util/cap.c +8
-1-return (cap_val & (1 << (cap &0x1f ))) !=0 ; + +if (has_cap) { + *used_root = false; +return true; + } + + *used_root = true; +return geteuid () ==0 ; }[Severity: High]
Does setting*used_root= true unconditionally here when a capability is missing break sequential capability checks for non-root users?For example, if an unprivileged user lacks
CAP_SYS_ADMIN, this will return false but setused_rootto true. In callers likeperf_event_paranoid_check():tools/perf/util/util.c:perf_event_paranoid_check() { ... if (!used_root && perf_cap__capable(CAP_PERFMON, &used_root)) ... }If a previous check for
CAP_SYS_ADMINfailed and poisoned theused_rootflag, won't it cause the subsequent check forCAP_PERFMONto be skipped entirely?This pattern appears in other locations as well, such as
check_bpf_filter_capable()where a failedCAP_BPFcheck might skip the fallback for non-root users to access root-pinned BPF filters because !used_root evaluates to false.
Quoted 5 lines
Additionally, could this override valid capability drops for root users in containers? Because
geteuid()== 0 evaluates to true, it seems this might falsely claim root users still possess the capability even if it was explicitly dropped, potentially leading to later failures instead of graceful fallbacks.
Can we simply remove the used_root parameter from perf_cap__capable()? Normally I think root would have the capability. If not, maybe we need to reject the operation as it may happen in a container setup.
Thanks,
Namhyung
Ian Rogers
reviewerPATCH v2 1/1 perf cap: Remove used_root parameter and simplify capability checks
Refactor perf_cap__capable() to completely remove the used_root out-parameter as requested by the maintainer. Relying on an explicit used_root boolean poisoned sequential capability checks (e.g. failing CAP_SYS_ADMIN checks poisoning the flag for subsequent CAP_PERFMON evaluations for unprivileged users) and created redundant complexity across check_ftrace_capable(), symbol__read_kptr_restrict(), and perf_event_paranoid_check().
Streamline the capability API to perform a pure true/false boolean evaluation. The function checks the Effective set using SYS_capget; if the syscall is missing or fails on legacy kernels, it cleanly falls back to checking EUID == 0. This perfectly preserves modern capability-aware host sessions, guarantees transparent fallback for older kernels, and correctly rejects privileged operations for containerized root processes that have explicitly dropped their capability bounding and permitted sets.
--- tools/perf/builtin-ftrace.c | 13 +++---------- tools/perf/util/bpf-filter.c | 22 +++++++++------------- tools/perf/util/cap.c | 4 +--- tools/perf/util/cap.h | 3 +-- tools/perf/util/symbol.c | 3 +-- tools/perf/util/util.c | 12 +++--------- 6 files changed, 18 insertions(+), 39 deletions(-)
tools/perf/builtin-ftrace.c +3 -10
@@ -72,18 +72,11 @@ static void ftrace__workload_exec_failed_signal(int signo __maybe_unused,static bool check_ftrace_capable (void ) { -bool used_root; - -if (perf_cap__capable (CAP_PERFMON , &used_root)) -return true; - -if (!used_root &&perf_cap__capable (CAP_SYS_ADMIN , &used_root)) +if (perf_cap__capable (CAP_PERFMON ) || +perf_cap__capable (CAP_SYS_ADMIN ))return true; -pr_err ("ftrace only works for %s!\n" , - used_root ?"root" - :"users with the CAP_PERFMON or CAP_SYS_ADMIN capability" - ); +pr_err ("ftrace only works for users with the CAP_PERFMON or CAP_SYS_ADMIN capability!\n" );return false; }
tools/perf/util/bpf-filter.c +9 -13
@@ -629,24 +629,20 @@ struct perf_bpf_filter_expr *perf_bpf_filter_expr__new(enum perf_bpf_filter_termstatic bool check_bpf_filter_capable (void ) { -bool used_root; +int fd; -if (perf_cap__capable (CAP_BPF , &used_root)) +if (perf_cap__capable (CAP_BPF ))return true; -if (!used_root) { -/* Check if root already pinned the filter programs and maps */ -int fd =get_pinned_fd ("filters" ); - -if (fd >=0 ) { -close (fd); -return true; - } +/* Check if root already pinned the filter programs and maps */ + fd =get_pinned_fd ("filters" ); +if (fd >=0 ) { +close (fd); +return true; } -pr_err ("Error: BPF filter only works for %s!\n" -"\tPlease run 'perf record --setup-filter pin' as root first.\n" , - used_root ?"root" :"users with the CAP_BPF capability" ); +pr_err ("Error: BPF filter only works for users with the CAP_BPF capability!\n" +"\tPlease run 'perf record --setup-filter pin' as root first.\n" );return false; }
tools/perf/util/cap.c +1 -3
@@ -12,7 +12,7 @@#define MAX_LINUX_CAPABILITY_U32S _LINUX_CAPABILITY_U32S_3 -bool perf_cap__capable (int cap,bool *used_root) +bool perf_cap__capable (int cap) {struct __user_cap_header_struct header = { .version = _LINUX_CAPABILITY_VERSION_3,
@@ -21,7 +21,6 @@ bool perf_cap__capable(int cap, bool *used_root)struct __user_cap_data_struct data[MAX_LINUX_CAPABILITY_U32S ] = {};__u32 cap_val; - *used_root = false;while (syscall (SYS_capget, &header, &data[0 ]) ==-1 ) {/* Retry, first attempt has set the header.version correctly. */ if (errno == EINVAL && header.version != _LINUX_CAPABILITY_VERSION_3 &&
@@ -29,7 +28,6 @@ bool perf_cap__capable(int cap, bool *used_root)continue ;pr_debug2 ("capget syscall failed (%m) fall back on root check\n" ); - *used_root = true;return geteuid () ==0 ; }
tools/perf/util/cap.h +1 -2
@@ -18,7 +18,6 @@#define CAP_BPF 39#endif -/* Query if a capability is supported, used_root is set if the fallback root check was used. */ -bool perf_cap__capable (int cap,bool *used_root); +bool perf_cap__capable (int cap);#endif /* __PERF_CAP_H */
tools/perf/util/symbol.c +1 -2
@@ -2452,8 +2452,7 @@ static bool symbol__read_kptr_restrict(void) {bool value = false;FILE *fp =fopen ("/proc/sys/kernel/kptr_restrict" ,"r" ); -bool used_root; -bool cap_syslog =perf_cap__capable (CAP_SYSLOG , &used_root); +bool cap_syslog =perf_cap__capable (CAP_SYSLOG );if (fp !=NULL ) {char line[8 ];
tools/perf/util/util.c +3 -9
@@ -378,15 +378,9 @@ int perf_event_paranoid(void)bool perf_event_paranoid_check (int max_level) { -bool used_root; - -if (perf_cap__capable (CAP_SYS_ADMIN , &used_root)) -return true; - -if (!used_root &&perf_cap__capable (CAP_PERFMON , &used_root)) -return true; - -return perf_event_paranoid () <= max_level; +return perf_cap__capable (CAP_SYS_ADMIN ) || +perf_cap__capable (CAP_PERFMON ) || +perf_event_paranoid () <= max_level; }int perf_tip (char **strp,const char *dirpath)
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Ian Rogers
reviewerPATCH v3 1/1 perf cap: Remove used_root parameter and simplify capability checks
Refactor perf_cap__capable() to completely remove the used_root out-parameter as requested by the maintainer. Relying on an explicit used_root boolean poisoned sequential capability checks (e.g. failing CAP_SYS_ADMIN checks poisoning the flag for subsequent CAP_PERFMON evaluations for unprivileged users) and created redundant complexity across check_ftrace_capable(), symbol__read_kptr_restrict(), and perf_event_paranoid_check().
Streamline the capability API to perform a pure true/false boolean evaluation. The function checks the Effective set using SYS_capget; if the syscall is missing or fails on legacy kernels, it cleanly falls back to checking EUID == 0. This perfectly preserves modern capability-aware host sessions, guarantees transparent fallback for older kernels, and correctly rejects privileged operations for containerized root processes that have explicitly dropped their capability bounding and permitted sets.
--- tools/perf/builtin-ftrace.c | 13 +++---------- tools/perf/util/bpf-filter.c | 22 +++++++++------------- tools/perf/util/cap.c | 4 +--- tools/perf/util/cap.h | 3 +-- tools/perf/util/symbol.c | 3 +-- tools/perf/util/util.c | 12 +++--------- 6 files changed, 18 insertions(+), 39 deletions(-)
tools/perf/builtin-ftrace.c +3 -10
@@ -72,18 +72,11 @@ static void ftrace__workload_exec_failed_signal(int signo __maybe_unused,static bool check_ftrace_capable (void ) { -bool used_root; - -if (perf_cap__capable (CAP_PERFMON , &used_root)) -return true; - -if (!used_root &&perf_cap__capable (CAP_SYS_ADMIN , &used_root)) +if (perf_cap__capable (CAP_PERFMON ) || +perf_cap__capable (CAP_SYS_ADMIN ))return true; -pr_err ("ftrace only works for %s!\n" , - used_root ?"root" - :"users with the CAP_PERFMON or CAP_SYS_ADMIN capability" - ); +pr_err ("ftrace only works for users with the CAP_PERFMON or CAP_SYS_ADMIN capability!\n" );return false; }
tools/perf/util/bpf-filter.c +9 -13
@@ -629,24 +629,20 @@ struct perf_bpf_filter_expr *perf_bpf_filter_expr__new(enum perf_bpf_filter_termstatic bool check_bpf_filter_capable (void ) { -bool used_root; +int fd; -if (perf_cap__capable (CAP_BPF , &used_root)) +if (perf_cap__capable (CAP_BPF ))return true; -if (!used_root) { -/* Check if root already pinned the filter programs and maps */ -int fd =get_pinned_fd ("filters" ); - -if (fd >=0 ) { -close (fd); -return true; - } +/* Check if root already pinned the filter programs and maps */ + fd =get_pinned_fd ("filters" ); +if (fd >=0 ) { +close (fd); +return true; } -pr_err ("Error: BPF filter only works for %s!\n" -"\tPlease run 'perf record --setup-filter pin' as root first.\n" , - used_root ?"root" :"users with the CAP_BPF capability" ); +pr_err ("Error: BPF filter only works for users with the CAP_BPF capability!\n" +"\tPlease run 'perf record --setup-filter pin' as root first.\n" );return false; }
tools/perf/util/cap.c +1 -3
@@ -12,7 +12,7 @@#define MAX_LINUX_CAPABILITY_U32S _LINUX_CAPABILITY_U32S_3 -bool perf_cap__capable (int cap,bool *used_root) +bool perf_cap__capable (int cap) {struct __user_cap_header_struct header = { .version = _LINUX_CAPABILITY_VERSION_3,
@@ -21,7 +21,6 @@ bool perf_cap__capable(int cap, bool *used_root)struct __user_cap_data_struct data[MAX_LINUX_CAPABILITY_U32S ] = {};__u32 cap_val; - *used_root = false;while (syscall (SYS_capget, &header, &data[0 ]) ==-1 ) {/* Retry, first attempt has set the header.version correctly. */ if (errno == EINVAL && header.version != _LINUX_CAPABILITY_VERSION_3 &&
@@ -29,7 +28,6 @@ bool perf_cap__capable(int cap, bool *used_root)continue ;pr_debug2 ("capget syscall failed (%m) fall back on root check\n" ); - *used_root = true;return geteuid () ==0 ; }
tools/perf/util/cap.h +1 -2
@@ -18,7 +18,6 @@#define CAP_BPF 39#endif -/* Query if a capability is supported, used_root is set if the fallback root check was used. */ -bool perf_cap__capable (int cap,bool *used_root); +bool perf_cap__capable (int cap);#endif /* __PERF_CAP_H */
tools/perf/util/symbol.c +1 -2
@@ -2452,8 +2452,7 @@ static bool symbol__read_kptr_restrict(void) {bool value = false;FILE *fp =fopen ("/proc/sys/kernel/kptr_restrict" ,"r" ); -bool used_root; -bool cap_syslog =perf_cap__capable (CAP_SYSLOG , &used_root); +bool cap_syslog =perf_cap__capable (CAP_SYSLOG );if (fp !=NULL ) {char line[8 ];
tools/perf/util/util.c +3 -9
@@ -378,15 +378,9 @@ int perf_event_paranoid(void)bool perf_event_paranoid_check (int max_level) { -bool used_root; - -if (perf_cap__capable (CAP_SYS_ADMIN , &used_root)) -return true; - -if (!used_root &&perf_cap__capable (CAP_PERFMON , &used_root)) -return true; - -return perf_event_paranoid () <= max_level; +return perf_cap__capable (CAP_SYS_ADMIN ) || +perf_cap__capable (CAP_PERFMON ) || +perf_event_paranoid () <= max_level; }int perf_tip (char **strp,const char *dirpath)
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v3] perf cap: Remove used_root parameter and simplify capability checks
Quoted 7 lines · Ian Rogers
Refactor
perf_cap__capable()to completely remove theused_rootout-parameter as requested by the maintainer. Relying on an explicitused_rootboolean poisoned sequential capability checks (e.g. failingCAP_SYS_ADMINchecks poisoning the flag for subsequentCAP_PERFMONevaluations for unprivileged users) and created redundant complexity acrosscheck_ftrace_capable(),symbol__read_kptr_restrict(), andperf_event_paranoid_check().
Quoted 7 lines
Streamline the capability API to perform a pure true/false boolean evaluation. The function checks the Effective set using
SYS_capget; if the syscall is missing or fails on legacy kernels, it cleanly falls back to checking EUID == 0. This perfectly preserves modern capability-aware host sessions, guarantees transparent fallback for older kernels, and correctly rejects privileged operations for containerized root processes that have explicitly dropped their capability bounding and permitted sets.
Applied to perf-tools-next, thanks!
Best regards,
Namhyung
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The patch unconditionally skips `fdarray_flag__nonfilterable` file descriptors in…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The patch unconditionally skips fdarray_flag__nonfilterable file descriptors in fdarray__filter(), causing a memory leak of perf_mmap references and an infinite busy loop if a system-wide event or other nonfilterable file descriptor receives an error (e.g., POLLERR, POLLHUP). --
--- Patch [1]: [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions --- commit 6940fed42e3ec7ba75ed80d3ab8c1cc4f0aad35d
Author: Ian Rogers <[email protected]>
perf record: Fix teardown hang on system-wide multi-threaded sessions
This patch attempts to resolve a hang during teardown in system-wide streaming mode by adding explicit evlist disables. It also modifies the fdarray filter logic to bypass nonfilterable file descriptors when processing POLLHUP signals.
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents, · line 120
Quoted 11 lines
tools/lib/api/fd/array.c +3
-0@@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { +if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +continue ; +
[Severity: High]
Does skipping the filter logic for non-filterable file descriptors here cause an infinite poll loop if a system-wide event receives an error?
In tools/lib/perf/evlist.c:mmap_per_evsel(), system-wide event file descriptors are marked with fdarray_flag__nonfilterable:
flgs = evsel->system_wide ? fdarray_flag__nonfilterable : fdarray_flag__default;
If one of these file descriptors encounters an error like POLLERR or POLLHUP, it seems this new continue statement bypasses the block that clears the events mask and invokes the entry_destructor.
Could this cause poll() to continuously return immediately with the same error, leading to a CPU busy loop and leaking perf_mmap references since the destructor is skipped?
Quoted 13 lines
tools/lib/api/fd/array.c +1
-2if (!fda->entries[fd].events)continue ;@@ -126,8 +129,7 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } -if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) - ++nr; + ++nr; }return nr;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Namhyung Kim
maintainerHi Ian,
Quoted 22 lines · Ian Rogers
Under system-wide (-a) parallel streaming mode (--threads=cpu), background recording threads can be inundated by a continuous firehose of hardware samples generated by the OS. In this state, a background thread's local hit count remains unequal to its sample count, causing it to bypass the blocking
fdarray__poll()call entirely on each iteration of its recording loop.Because the termination check relies on the POLLHUP event status populated specifically by
fdarray__poll(), bypassing it prevents the background thread from ever recognizing that its control pipe was closed by the main thread. This traps the background thread in an infinite recording loop, hanging the main thread indefinitely as it awaits a termination acknowledgment that never arrives.Ensure teardown completion by adding explicit
evlist__disable()calls in the main thread's cleanup paths atout_child: andout_child_no_flush:. Additionally, patchfdarray__filter()to respect thefdarray_flag__nonfilterableflag, preventing it from incorrectly setting the background thread's control pipe file descriptor to -1 and clearing its revents mask upon processing termination POLLHUP signals.
Quoted 3 lines
Fixes f94563fac269 ("perf record: fix poll storm when monitored threads exit")Assisted-by Antigravity:gemini-3.5-flashSigned-off-by Ian Rogers <[email protected]>
I didn't know you already sent the fix so I was doing the same. It seems your fix is better as it added evlist__disable(). I'll test this and pick up if it's ok.
Thanks,
Namhyung
Quoted 50 lines
--- tools/lib/api/fd/array.c | 6 ++++-- tools/perf/builtin-record.c | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-)tools/lib/api/fd/array.c +4
-2@@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { +if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +continue ; +if (!fda->entries[fd].events)continue ;@@ -132,8 +135,7 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } -if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) - ++nr; + ++nr; }return nr;tools/perf/builtin-record.c +2
-0@@ -2890,11 +2890,13 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)record__synthesize_workload (rec, true); out_child: +evlist__disable (rec->evlist);record__stop_threads (rec);record__mmap_read_all (rec, true); goto out_free_threads; out_child_no_flush:/* mmap read already failed — retrying would just fail again */ +evlist__disable (rec->evlist);record__stop_threads (rec); out_free_threads:record__free_thread_data (rec);Signature · 1 line
2.55.0.795.g602f6c329a-goog
Re: (subset) [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions
In reply to Ian RogersQuoted 8 lines · Ian Rogers
Under system-wide (-a) parallel streaming mode (--threads=cpu), background recording threads can be inundated by a continuous firehose of hardware samples generated by the OS. In this state, a background thread's local hit count remains unequal to its sample count, causing it to bypass the blocking
fdarray__poll()call entirely on each iteration of its recording loop.[...]
Applied to perf-tools-next, thanks!
Best regards,
Namhyung
Ian Rogers
reviewerPATCH v2 1/1 perf record: Fix destructor invocation and event counting in fdarray__filter
Fix the API-level fdarray filtering logic in fdarray__filter() following the integration of the system-wide recording teardown fix. Rather than bypassing the evaluation loop entirely for nonfilterable file descriptors (which accidentally skips calling registered entry destructors for control pipes and wakeup FDs, while creating poll storms and reference leaks for system-wide events that also have nonfilterable set), separate the lifecycle and masking logic.
Reorder the filtering sequence so that registered entry destructors are safely executed first for all file descriptors. Then, utilize the fdarray_flag__non_perf_event flag to perfectly distinguish and bypass destructive file descriptor zeroing (.fd = -1) and revents mask clearing for background control pipes and timers. This allows system-wide event file descriptors to be correctly destroyed, zeroed, and filtered out upon receiving POLLHUP/POLLERR signals without triggering busy loops, while safely preserving control pipe teardown signaling and destructors.
--- tools/lib/api/fd/array.c | 6 +++--- tools/perf/builtin-record.c | 4 +++- 2 files changed, 6 insertions(+), 4 deletions(-)
tools/lib/api/fd/array.c +3 -3
@@ -115,9 +115,6 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) -continue ; -if (!fda->entries[fd].events)continue ;
@@ -125,6 +122,9 @@ int fdarray__filter(struct fdarray *fda, short revents,if (entry_destructor)entry_destructor (fda, fd, arg); +if (fda->priv[fd].flags & fdarray_flag__non_perf_event) +continue ; +/* * Set fd to -1 so poll() ignores this entry; otherwise * POLLHUP/POLLERR are still reported for events=0 fds
tools/perf/builtin-record.c +3 -1
@@ -1242,7 +1242,9 @@ static int record__alloc_thread_data(struct record *rec, struct evlist *evlist) goto out_free; } ret =fdarray__add (&thread_data[t].pollfd, thread_data[t].pipes.msg[0 ], -POLLIN |POLLERR |POLLHUP , fdarray_flag__nonfilterable); +POLLIN |POLLERR |POLLHUP , + fdarray_flag__nonfilterable | + fdarray_flag__non_perf_event);if (ret <0 ) {pr_err ("Failed to add descriptor to thread[%d] pollfd\n" , t); goto out_free;
Signature · 1 line
2.55.0.141.g00534a21ce-goog
[email protected]
AI reviewRe: [PATCH v2] perf record: Fix destructor invocation and event counting in fdarray__filter
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] perf record and perf trace hang indefinitely when profiling a target process that exits. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] perf record and perf trace hang indefinitely when profiling a target process that exits. --
--- Patch [1]: [PATCH v2] perf record: Fix destructor invocation and event counting in fdarray__filter --- commit 586c1ed32764876e9ecee40639dd9f2a7fcfdcd5
Author: Ian Rogers <[email protected]>
perf record: Fix destructor invocation and event counting in fdarray__filter
This commit modifies the fdarray filtering logic to separate lifecycle and masking logic. It removes the early bypass for nonfilterable file descriptors so their destructors can run, and uses a non_perf_event flag to prevent clearing their masks.
Quoted 15 lines
tools/lib/api/fd/array.c +0
-3@@ -115,9 +115,6 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) -continue ; -if (!fda->entries[fd].events)continue ;
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents, · line 129
Quoted 8 lines
tools/lib/api/fd/array.c +3
-0@@ -125,6 +122,9 @@ int fdarray__filter(struct fdarray *fda, short revents,if (entry_destructor)entry_destructor (fda, fd, arg); +if (fda->priv[fd].flags & fdarray_flag__non_perf_event) +continue ; +/* * Set fd to -1 so poll() ignores this entry; otherwise
[Severity: High]
With the removal of the early continue for nonfilterable file descriptors, does this regression cause the active event count to never reach zero?
When profiling a target process that exits, auxiliary file descriptors like control pipes usually do not match the revents mask (such as POLLHUP). They now bypass the revents check block and unconditionally reach the counter increment at the bottom of the loop:
fdarray__filter()
...
if (fda->entries[fd].revents & revents) {
...
continue;
}
++nr;
}
return nr;
}In the main event loop, __cmd_record() relies on this function returning 0 to transition to the draining state and terminate gracefully:
if (fdarray__filter(&thread->pollfd, POLLERR | POLLHUP,
record__thread_munmap_filtered, NULL) == 0)
draining = true;Since nonfilterable and non-perf-event file descriptors unconditionally increment nr when they don't have matching revents, could this cause perf record and perf trace to hang indefinitely waiting in poll() after the traced process exits?
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v3 1/1 perf record: Fix destructor invocation and event counting in fdarray__filter
In reply to Ian RogersWhen processing POLLHUP or POLLERR for an event in fdarray__filter, the function invokes its destructor callback but fails to re-inject the nonfilterable exclusion masks into the active event count (nr) increment condition. Because auxiliary and control pipe descriptors never enter the POLLHUP event match block, their active event count is unconditionally incremented, preventing the return value from reaching zero on target process exit and trapping __cmd_record() in an infinite drain poll hang.
Fix it by applying both fdarray_flag__nonfilterable and fdarray_flag__non_perf_event exclusion masks into the active event counter increment block at the bottom of the filtering loop.
--- tools/lib/api/fd/array.c | 10 ++++++---- tools/perf/builtin-record.c | 4 +++- 2 files changed, 9 insertions(+), 5 deletions(-)
tools/lib/api/fd/array.c +6 -4
@@ -116,9 +116,6 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) -continue ; -if (!fda->entries[fd].events)continue ;
@@ -126,6 +123,9 @@ int fdarray__filter(struct fdarray *fda, short revents,if (entry_destructor)entry_destructor (fda, fd, arg); +if (fda->priv[fd].flags & fdarray_flag__non_perf_event) +continue ; +/* * Set fd to -1 so poll() ignores this entry; otherwise * POLLHUP/POLLERR are still reported for events=0 fds
@@ -136,7 +136,9 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } - ++nr; +if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable) && + !(fda->priv[fd].flags & fdarray_flag__non_perf_event)) + ++nr; }return nr;
tools/perf/builtin-record.c +3 -1
@@ -1242,7 +1242,9 @@ static int record__alloc_thread_data(struct record *rec, struct evlist *evlist) goto out_free; } ret =fdarray__add (&thread_data[t].pollfd, thread_data[t].pipes.msg[0 ], -POLLIN |POLLERR |POLLHUP , fdarray_flag__nonfilterable); +POLLIN |POLLERR |POLLHUP , + fdarray_flag__nonfilterable | + fdarray_flag__non_perf_event);if (ret <0 ) {pr_err ("Failed to add descriptor to thread[%d] pollfd\n" , t); goto out_free;
Signature · 1 line
2.55.0.229.g6434b31f56-goog
[email protected]
AI reviewRe: [PATCH v3] perf record: Fix destructor invocation and event counting in fdarray__filter
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [High] The patch completely breaks the 'nonfilterable' semantics for system-wide perf events, erroneously causing them to be filtered and their mmap references to be prematurely dropped on POLLHUP.
- [Medium] The patch is based on a fabricated premise regarding the active event count (nr) increment, making its core logical changes redundant while introducing regressions.
Pre-existing issues:
- [High] Unhandled POLLHUP on non_perf_event descriptors causes premature termination due to thread and evlist pollfd index mismatch. --
--- Patch [1]: [PATCH v3] perf record: Fix destructor invocation and event counting in fdarray__filter --- commit 251ec969368be86f9fe7f805b44d3a39b2228d7f
Author: Ian Rogers <[email protected]>
perf record: Fix destructor invocation and event counting in fdarray__filter
This commit modifies fdarray__filter() to change how nonfilterable and non-perf events are handled during poll events. It removes the early exit for nonfilterable descriptors and adds a conditional block before incrementing the active event count.
Quoted 7 lines
When processing POLLHUP or POLLERR for an event in
fdarray__filter, the function invokes its destructor callback but fails to re-inject the nonfilterable exclusion masks into the active event count (nr) increment condition. Because auxiliary and control pipe descriptors never enter the POLLHUP event match block, their active event count is unconditionally incremented, preventing the return value from reaching zero on target process exit and trapping__cmd_record()in an infinite drain poll hang.
[Severity: Medium]
Before this patch, descriptors marked with fdarray_flag__nonfilterable hit a continue at the very top of the loop, skipping the loop body entirely, including the ++nr increment.
It appears they were never unconditionally incrementing the active event count, meaning the patch might be attempting to fix a non-existent bug by modifying the increment conditionally instead of fixing the root cause.
Is the description of the original behavior in the commit message accurate?
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents, · line 121
Quoted 11 lines
tools/lib/api/fd/array.c +0
-3@@ -116,9 +116,6 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) -continue ; -
[Severity: High]
System-wide events are assigned fdarray_flag__nonfilterable but not fdarray_flag__non_perf_event, as seen in mmap_per_evsel():
flgs = evsel->system_wide ? fdarray_flag__nonfilterable :
fdarray_flag__default;Without the early continue, when these events receive a POLLHUP or POLLERR, they will now fall into the revents block below, prematurely trigger their destructor (unmapping the perf event), and have their fd set to -1.
Does removing this early continue break the semantics for system-wide perf events?
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents, · line 128
Quoted 10 lines
tools/lib/api/fd/array.c +3
-0if (!fda->entries[fd].events)continue ;@@ -126,6 +123,9 @@ int fdarray__filter(struct fdarray *fda, short revents,if (entry_destructor)entry_destructor (fda, fd, arg); +if (fda->priv[fd].flags & fdarray_flag__non_perf_event) +continue ; +
[Severity: High]
This isn't a bug introduced by this patch, but when the ctl_fd receives a POLLHUP, it skips setting its fd to -1 in the thread's poll array.
Later, evlist__ctlfd_process() handles the command and calls evlist__finalize_ctlfd(), which explicitly sets the core evlist's poll array fd to -1.
On the next iteration, a mismatch is detected:
tools/perf/builtin-record.c:record__update_evlist_pollfd_from_thread() {
...
pr_err("Thread and evlist pollfd index mismatch\n");
err = -EINVAL;
...
}Could this cause perf record to abort with -EINVAL and data loss when a control descriptor is closed?
Quoted 3 lines
tools/lib/api/fd/array.c +0
-0/* * Set fd to -1 so poll() ignores this entry; otherwise * POLLHUP/POLLERR are still reported for events=0 fds
[ ... ]
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents, · line 142
Quoted 9 lines
tools/lib/api/fd/array.c +3
-1@@ -136,7 +136,9 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } - ++nr; +if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable) && + !(fda->priv[fd].flags & fdarray_flag__non_perf_event)) + ++nr; }
[Severity: Medium]
As noted in the commit message review, nonfilterable descriptors used to skip the loop entirely via the early continue at the top.
Since they were never reaching this point, is this new check redundant?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v4 1/1 perf fdarray: Fix destructor invocation and event counting in fdarray__filter
In reply to Ian RogersWhen processing POLLHUP or POLLERR for an event in fdarray__filter, the function invokes its destructor callback. However, the exact behavior around unhandled POLLHUP events on control pipe descriptors caused premature termination due to thread and evlist pollfd index mismatches. This occurred because they were skipped by the early nonfilterable continue.
Address this by refining the early continue filter to only skip system-wide perf events (which are nonfilterable but not non_perf_event). Control descriptors (non_perf_event) now fall through to appropriately have their fd value unset to -1 on POLLHUP while avoiding their destructors. Finally, maintain the invariant that the active event counter (nr) increments strictly and only for completely filterable events.
--- tools/lib/api/fd/array.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
tools/lib/api/fd/array.c +13 -3
@@ -116,14 +116,23 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +/* +* System-wide perf events are nonfilterable but not non_perf_event. +* We want to skip them entirely and never process revents on them. +*/ +if ((fda->priv[fd].flags & fdarray_flag__nonfilterable) && + !(fda->priv[fd].flags & fdarray_flag__non_perf_event))continue ;if (!fda->entries[fd].events)continue ;if (fda->entries[fd].revents & revents) { -if (entry_destructor) +/* +* Control descriptors are non_perf_event and don't need +* their perf-specific destructors triggered. +*/ +if (entry_destructor && !(fda->priv[fd].flags & fdarray_flag__non_perf_event))entry_destructor (fda, fd, arg);/*
@@ -136,7 +145,8 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } - ++nr; +if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) + ++nr; }return nr;
Signature · 1 line
2.55.0.699.gb54405d56f-goog
Namhyung Kim
maintainerRe: [PATCH v4] perf fdarray: Fix destructor invocation and event counting in fdarray__filter
Quoted 5 lines · Ian Rogers
When processing POLLHUP or POLLERR for an event in
fdarray__filter, the function invokes its destructor callback. However, the exact behavior around unhandled POLLHUP events on control pipe descriptors caused premature termination due to thread and evlist pollfd index mismatches. This occurred because they were skipped by the early nonfilterable continue.
Do you have a concrete scenario to check this behavior so that we can verify the fix? It'd be great if we can add a test case.
Thanks,
Namhyung
Quoted 58 lines
Address this by refining the early continue filter to only skip system-wide perf events (which are nonfilterable but not
non_perf_event). Control descriptors (non_perf_event) now fall through to appropriately have their fd value unset to -1 on POLLHUP while avoiding their destructors. Finally, maintain the invariant that the active event counter (nr) increments strictly and only for completely filterable events.Fixes fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")Assisted-by Gemini:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>--- tools/lib/api/fd/array.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)tools/lib/api/fd/array.c +13
-3@@ -116,14 +116,23 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +/* +* System-wide perf events are nonfilterable but not non_perf_event. +* We want to skip them entirely and never process revents on them. +*/ +if ((fda->priv[fd].flags & fdarray_flag__nonfilterable) && + !(fda->priv[fd].flags & fdarray_flag__non_perf_event))continue ;if (!fda->entries[fd].events)continue ;if (fda->entries[fd].revents & revents) { -if (entry_destructor) +/* +* Control descriptors are non_perf_event and don't need +* their perf-specific destructors triggered. +*/ +if (entry_destructor && !(fda->priv[fd].flags & fdarray_flag__non_perf_event))entry_destructor (fda, fd, arg);/* @@ -136,7 +145,8 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } - ++nr; +if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) + ++nr; }return nr;Signature · 1 line
2.55.0.699.gb54405d56f-goog
Ian Rogers
reviewerRe: [PATCH v4] perf fdarray: Fix destructor invocation and event counting in fdarray__filter
Quoted 10 lines · Namhyung Kim
Quoted 5 lines · Ian Rogers
When processing POLLHUP or POLLERR for an event in
fdarray__filter, the function invokes its destructor callback. However, the exact behavior around unhandled POLLHUP events on control pipe descriptors caused premature termination due to thread and evlist pollfd index mismatches. This occurred because they were skipped by the early nonfilterable continue.Do you have a concrete scenario to check this behavior so that we can verify the fix? It'd be great if we can add a test case.
So the fix is trying to address hangs I see in the TPEBS test. We should be able to add a C unit test. I can do that while I wrangle with Sashiko.
Thanks,
Ian
Quoted 61 lines
Thanks,
NamhyungQuoted 58 lines
Address this by refining the early continue filter to only skip system-wide perf events (which are nonfilterable but not
non_perf_event). Control descriptors (non_perf_event) now fall through to appropriately have their fd value unset to -1 on POLLHUP while avoiding their destructors. Finally, maintain the invariant that the active event counter (nr) increments strictly and only for completely filterable events.Fixes fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")Assisted-by Gemini:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>--- tools/lib/api/fd/array.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)tools/lib/api/fd/array.c +13
-4@@ -116,14 +116,23 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +/* +* System-wide perf events are nonfilterable but not non_perf_event. +* We want to skip them entirely and never process revents on them. +*/ +if ((fda->priv[fd].flags & fdarray_flag__nonfilterable) && + !(fda->priv[fd].flags & fdarray_flag__non_perf_event))continue ;if (!fda->entries[fd].events)continue ;if (fda->entries[fd].revents & revents) { -if (entry_destructor) +/* +* Control descriptors are non_perf_event and don't need +* their perf-specific destructors triggered. +*/ +if (entry_destructor && !(fda->priv[fd].flags & fdarray_flag__non_perf_event))entry_destructor (fda, fd, arg);/* @@ -136,7 +145,8 @@ int fdarray__filter(struct fdarray *fda, short revents,continue ; } - ++nr; +if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) + ++nr; }return nr; --2.55.0.699.gb54405d56f-goog
[email protected]
AI reviewRe: [PATCH v4] perf fdarray: Fix destructor invocation and event counting in fdarray__filter
In reply to Ian RogersThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The refinement to fdarray__filter causes an immediate 'pollfd index mismatch' error and premature…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The refinement to fdarray__filter causes an immediate 'pollfd index mismatch' error and premature termination on control descriptor POLLHUP by prematurely clearing fd and revents. --
--- Patch [1]: [PATCH v4] perf fdarray: Fix destructor invocation and event counting in fdarray__filter --- commit 495f184eb7be01a8af2c7157e4d7f142c89228b0
Author: Ian Rogers <[email protected]>
perf fdarray: Fix destructor invocation and event counting in fdarray__filter
This commit refines the early continue filter in fdarray__filter to only skip system-wide perf events. Control descriptors now fall through to have their fd values unset to -1 on POLLHUP while avoiding their destructors. It also ensures the active event counter increments strictly for completely filterable events.
Quoted 30 lines
tools/lib/api/fd/array.c +11
-2@@ -116,14 +116,23 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->priv[fd].flags & fdarray_flag__nonfilterable) +/* +* System-wide perf events are nonfilterable but not non_perf_event. +* We want to skip them entirely and never process revents on them. +*/ +if ((fda->priv[fd].flags & fdarray_flag__nonfilterable) && + !(fda->priv[fd].flags & fdarray_flag__non_perf_event))continue ;if (!fda->entries[fd].events)continue ;if (fda->entries[fd].revents & revents) { -if (entry_destructor) +/* +* Control descriptors are non_perf_event and don't need +* their perf-specific destructors triggered. +*/ +if (entry_destructor && !(fda->priv[fd].flags & fdarray_flag__non_perf_event))entry_destructor (fda, fd, arg);/*
tools/lib/api/fd/array.c
Quoted 6 lines
tools/lib/api/fd/array.c +0
-0* Set fd to -1 so poll() ignores this entry; otherwise * POLLHUP/POLLERR are still reported for events=0 fds * (POSIX: always checked), causing a poll storm. */ fda->entries[fd].fd =-1 ; fda->entries[fd].revents = fda->entries[fd].events =0 ;
[Severity: High]
Does unsetting the file descriptor to -1 and clearing revents for control descriptors cause a pollfd index mismatch error during teardown?
When __cmd_record() handles a POLLHUP on the control pipe, it calls fdarray__filter() which now mutates the thread's control descriptor:
tools/perf/builtin-record.c:__cmd_record() {
...
if (fdarray__filter(&thread->pollfd, POLLERR | POLLHUP,
record__thread_munmap_filtered, NULL) == 0)
draining = true;
err = record__update_evlist_pollfd_from_thread(rec, rec->evlist, thread);
...
}Immediately after, record__update_evlist_pollfd_from_thread() compares the mutated thread array against evlist->pollfd, which retains the original valid file descriptor. This strict check fails and returns -EINVAL, causing an early ungraceful abort:
tools/perf/builtin-record.c:record__update_evlist_pollfd_from_thread() {
...
if (e_entries[e_pos].fd != t_entries[t_pos].fd ||
e_entries[e_pos].events != t_entries[t_pos].events) {
pr_err("Thread and evlist pollfd index mismatch\n");
err = -EINVAL;
continue;
}
}Furthermore, does clearing revents to 0 here ensure evlist__ctlfd_process() will never see the POLLHUP to perform a clean teardown via evlist__finalize_ctlfd()?
Quoted 9 lines
tools/lib/api/fd/array.c +2
-1continue ; } - ++nr; +if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable)) + ++nr; }return nr;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v5 1/1 perf record: Fix unhandled POLLHUP on non_perf_event descriptors
In reply to Ian RogersWhen processing POLLHUP or POLLERR for an event in fdarray__filter, the function's prior iterations incorrectly mutated non_perf_event control descriptors by zeroing their events when a POLLHUP occurred. This caused premature termination due to index mismatches or hangs since the core evlist logic was prevented from safely finalizing the poll array setup via evlist__ctlfd_process().
Revert the logic in fdarray__filter to cleanly bypass all nonfilterable events as introduced by fb4751e79c45. Instead, fix the underlying logic within record__update_evlist_pollfd_from_thread() in builtin-record.c to sustainably reflect teardown statuses (-1) originating from evlist__finalize_ctlfd(). This correctly mirrors the finalized state to the thread's poll structure avoiding both invalid -EINVAL index crashes and POLLHUP spin loops.
Included is a unit test to enforce that fdarray_flag__nonfilterable items are accurately completely circumvented during revents filtering loops.
--- tools/perf/builtin-record.c | 10 ++++++++++ tools/perf/tests/fdarray.c | 24 ++++++++++++++++++++++++ 2 files changed, 34 insertions(+)
tools/perf/builtin-record.c +10 -0
@@ -1169,6 +1169,16 @@ static int record__update_evlist_pollfd_from_thread(struct record *rec,int e_pos = rec->index_map[i].evlist_pollfd_index;int t_pos = rec->index_map[i].thread_pollfd_index; +if (e_entries[e_pos].fd ==-1 || e_entries[e_pos].events ==0 ) { +/* +* e_entries might have been finalized by evlist__finalize_ctlfd(). +* We must propagate it to t_entries to avoid index mismatches +* and to prevent a poll storm on the next iteration. +*/ + t_entries[t_pos].fd =-1 ; + t_entries[t_pos].events =0 ; + } +if (e_entries[e_pos].fd != t_entries[t_pos].fd || e_entries[e_pos].events != t_entries[t_pos].events) {pr_err ("Thread and evlist pollfd index mismatch\n" );
tools/perf/tests/fdarray.c +24 -0
@@ -80,6 +80,30 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub goto out_delete; } +fdarray__init_revents (fda,POLLHUP ); + fda->priv[2 ].flags = fdarray_flag__nonfilterable; + +pr_debug ("\nfiltering all but fda->entries[2] (nonfilterable):" ); +fdarray__fprintf_prefix (fda,"before" , stderr); + nr_fds =fdarray__filter (fda,POLLHUP ,NULL ,NULL ); +fdarray__fprintf_prefix (fda," after" , stderr); + +if (nr_fds !=0 ) { +pr_debug ("\nfdarray__filter()=%d != 0, should be 0\n" , + nr_fds); + goto out_delete; + } +if (fda->entries[2 ].fd ==-1 ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" ); + goto out_delete; + } + + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
2.55.0.699.gb54405d56f-goog
[email protected]
AI reviewRe: [PATCH v5] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The patch completely omits necessary modifications to `tools/lib/api/fd/array.c`, causing the…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The patch completely omits necessary modifications to tools/lib/api/fd/array.c, causing the newly added unit test to fail deterministically. --
--- Patch [1]: [PATCH v5] perf record: Fix unhandled POLLHUP on non_perf_event descriptors --- commit dd8edd3a80b47efe7508c7fe4d3a8d1e99fd10d2
Author: Ian Rogers <[email protected]>
perf record: Fix unhandled POLLHUP on non_perf_event descriptors
This commit attempts to fix a POLLHUP issue by updating the pollfd state in record__update_evlist_pollfd_from_thread() and reverting logic in fdarray__filter(). It also introduces a unit test to enforce that nonfilterable items bypass the revents filtering loop.
Quoted 4 lines
tools/perf/tests/fdarray.c +0
-0
[ ... ]
Quoted 8 lines
tools/perf/tests/fdarray.c +4
-0@@ -80,6 +80,30 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub goto out_delete; } +fdarray__init_revents (fda,POLLHUP ); + fda->priv[2 ].flags = fdarray_flag__nonfilterable; + +pr_debug ("\nfiltering all but fda->entries[2] (nonfilterable):" );
tools/perf/tests/fdarray.c static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub · line 96
Quoted 10 lines
tools/perf/tests/fdarray.c +10
-0+fdarray__fprintf_prefix (fda,"before" , stderr); + nr_fds =fdarray__filter (fda,POLLHUP ,NULL ,NULL ); +fdarray__fprintf_prefix (fda," after" , stderr); + +if (nr_fds !=0 ) { +pr_debug ("\nfdarray__filter()=%d != 0, should be 0\n" , + nr_fds); + goto out_delete; + } +if (fda->entries[2 ].fd ==-1 ) {
[Severity: Medium]
The commit message mentions reverting logic in fdarray__filter() to bypass nonfilterable events, but the actual modifications to tools/lib/api/fd/array.c appear to be missing from this patch.
Since the logic to bypass nonfilterable items wasn't included, does this new test in test__fdarray__filter() fail deterministically because the underlying behavior hasn't changed?
Quoted 13 lines
tools/perf/tests/fdarray.c +10
-0+pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" ); + goto out_delete; + } + + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v6 1/1 perf record: Fix unhandled POLLHUP on non_perf_event descriptors
In reply to Ian RogersWhen processing POLLHUP on a non_perf_event control descriptor (like ctl_fd), evlist__finalize_ctlfd() is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.
Consequently, on the next iteration of the main recording loop, record__update_evlist_pollfd_from_thread() performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match (-1 != original_fd). This causes an immediate -EINVAL abort and a premature, ungraceful teardown.
Fix the underlying logic within record__update_evlist_pollfd_from_thread() to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.
Additionally, add a unit test that explicitly validates that fdarray__filter() preserves its invariants regarding fdarray_flag__nonfilterable items to guard against future regressions.
--- tools/perf/builtin-record.c | 10 ++++++++++ tools/perf/tests/fdarray.c | 24 ++++++++++++++++++++++++ 2 files changed, 34 insertions(+)
tools/perf/builtin-record.c +10 -0
@@ -1169,6 +1169,16 @@ static int record__update_evlist_pollfd_from_thread(struct record *rec,int e_pos = rec->index_map[i].evlist_pollfd_index;int t_pos = rec->index_map[i].thread_pollfd_index; +if (e_entries[e_pos].fd ==-1 || e_entries[e_pos].events ==0 ) { +/* +* e_entries might have been finalized by evlist__finalize_ctlfd(). +* We must propagate it to t_entries to avoid index mismatches +* and to prevent a poll storm on the next iteration. +*/ + t_entries[t_pos].fd =-1 ; + t_entries[t_pos].events =0 ; + } +if (e_entries[e_pos].fd != t_entries[t_pos].fd || e_entries[e_pos].events != t_entries[t_pos].events) {pr_err ("Thread and evlist pollfd index mismatch\n" );
tools/perf/tests/fdarray.c +24 -0
@@ -80,6 +80,30 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub goto out_delete; } +fdarray__init_revents (fda,POLLHUP ); + fda->priv[2 ].flags = fdarray_flag__nonfilterable; + +pr_debug ("\nfiltering all but fda->entries[2] (nonfilterable):" ); +fdarray__fprintf_prefix (fda,"before" , stderr); + nr_fds =fdarray__filter (fda,POLLHUP ,NULL ,NULL ); +fdarray__fprintf_prefix (fda," after" , stderr); + +if (nr_fds !=0 ) { +pr_debug ("\nfdarray__filter()=%d != 0, should be 0\n" , + nr_fds); + goto out_delete; + } +if (fda->entries[2 ].fd ==-1 ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" ); + goto out_delete; + } + + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
2.55.0.737.g08866a6d13-goog
[email protected]
AI reviewRe: [PATCH v6] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The synchronization logic added to `record__update_evlist_pollfd_from_thread` is flawed and fails…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The synchronization logic added to record__update_evlist_pollfd_from_thread is flawed and fails to prevent the -EINVAL index mismatch abort on POLLHUP.
- [Medium] The newly added unit test test__fdarray__filter fails unconditionally, breaking the perf test suite.
--
--- Patch [1]: [PATCH v6] perf record: Fix unhandled POLLHUP on non_perf_event descriptors --- commit a78a902826b55b27b98ff151dcb9dd5753ba4611
Author: Ian Rogers <[email protected]>
perf record: Fix unhandled POLLHUP on non_perf_event descriptors
This patch attempts to fix a premature teardown during recording sessions by propagating the finalized teardown status (-1) from the core evlist back to the thread's local pollfd array when a POLLHUP is received. It also adds a unit test to verify that fdarray__filter() properly preserves items marked as nonfilterable.
Quoted 7 lines
tools/perf/builtin-record.c +0
-0@@ -1180,6 +1180,16 @@ static int record__update_evlist_pollfd_from_thread(struct record *rec,int e_pos = rec->index_map[i].evlist_pollfd_index;int t_pos = rec->index_map[i].thread_pollfd_index;
tools/perf/builtin-record.c static int record__update_evlist_pollfd_from_thread(struct record *rec, · line 1191
Quoted 10 lines
tools/perf/builtin-record.c +9
-0+if (e_entries[e_pos].fd ==-1 || e_entries[e_pos].events ==0 ) { +/* +* e_entries might have been finalized by evlist__finalize_ctlfd(). +* We must propagate it to t_entries to avoid index mismatches +* and to prevent a poll storm on the next iteration. +*/ + t_entries[t_pos].fd =-1 ; + t_entries[t_pos].events =0 ; + }
[Severity: High]
Does this synchronization logic happen in the correct order to prevent the -EINVAL index mismatch?
Looking at the main loop in __cmd_record(), fdarray__filter() mutates the thread's local pollfd array (zeroing events) before record__update_evlist_pollfd_from_thread() is called:
tools/perf/builtin-record.c:__cmd_record() {
...
if (fdarray__filter(&thread->pollfd, POLLERR | POLLHUP, ...) == 0)
draining = true;
err = record__update_evlist_pollfd_from_thread(rec, rec->evlist, thread);
if (err)
goto out_child;
}
if (evlist__ctlfd_process(rec->evlist, &cmd) > 0) {
...
}Because evlist__ctlfd_process() hasn't finalized the core evlist's array yet, e_entries[e_pos].events will still be POLLIN, bypassing this new check.
Since the thread's events field was just zeroed by fdarray__filter(), the strict equivalence check below (POLLIN != 0) will still fail and abort the session with -EINVAL.
Quoted 4 lines
tools/perf/builtin-record.c +1
-0+if (e_entries[e_pos].fd != t_entries[t_pos].fd || e_entries[e_pos].events != t_entries[t_pos].events) {pr_err ("Thread and evlist pollfd index mismatch\n" );
[ ... ]
Quoted 5 lines
tools/perf/tests/fdarray.c +0
-0@@ -80,6 +80,30 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub
[ ... ]
tools/perf/tests/fdarray.c static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub · line 103
Quoted 8 lines
tools/perf/tests/fdarray.c +8
-0+if (fda->entries[2 ].fd ==-1 ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" ); + goto out_delete; + }
[Severity: Medium]
Will this test fail unconditionally?
Looking at fdarray__filter() in tools/lib/api/fd/array.c, it appears to unconditionally clear the events and revents fields for any entry matching the filter mask before the nonfilterable flag is ever evaluated:
tools/lib/api/fd/array.c:fdarray__filter() {
...
if (fda->entries[fd].revents & revents) {
if (entry_destructor)
entry_destructor(fda, fd, arg);
fda->entries[fd].revents = fda->entries[fd].events = 0;
continue;
}
...
}Because revents is unconditionally zeroed, won't this check always evaluate to true and cause the perf test suite to fail?
Quoted 6 lines
tools/perf/tests/fdarray.c +3
-0+ + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v7 1/1 perf record: Fix unhandled POLLHUP on non_perf_event descriptors
In reply to Ian RogersWhen processing POLLHUP on a non_perf_event control descriptor, evlist__finalize_ctlfd() is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.
Consequently, on the next iteration of the main recording loop, record__update_evlist_pollfd_from_thread() performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match. This causes an immediate -EINVAL abort and a premature teardown.
Fix the underlying logic within record__update_evlist_pollfd_from_thread() to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.
Additionally, add a unit test that explicitly validates that fdarray__filter() preserves its invariants regarding fdarray_flag__nonfilterable items to guard against regressions.
--- tools/perf/builtin-record.c | 15 +++++++++++++++ tools/perf/tests/fdarray.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+)
tools/perf/builtin-record.c +15 -0
@@ -1169,6 +1169,21 @@ static int record__update_evlist_pollfd_from_thread(struct record *rec,int e_pos = rec->index_map[i].evlist_pollfd_index;int t_pos = rec->index_map[i].thread_pollfd_index; +if (e_entries[e_pos].fd ==-1 || e_entries[e_pos].events ==0 ) { +/* +* If the control file descriptor was closed, then evlist__ctlfd_process() +* will have called evlist__finalize_ctlfd() on the PREVIOUS loop iteration +* to cleanly set the core evlist's e_entries[e_pos].fd to -1. +* +* Since nonfilterable items are skipped by fdarray__filter(), the +* thread's local t_entries[t_pos] retains its original state. +* We must explicitly propagate the finalized -1 state to t_entries +* BEFORE evaluating the strict equivalence check below. +*/ + t_entries[t_pos].fd =-1 ; + t_entries[t_pos].events =0 ; + } +if (e_entries[e_pos].fd != t_entries[t_pos].fd || e_entries[e_pos].events != t_entries[t_pos].events) {pr_err ("Thread and evlist pollfd index mismatch\n" );
tools/perf/tests/fdarray.c +32 -0
@@ -80,6 +80,38 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub goto out_delete; } +fdarray__init_revents (fda,POLLHUP ); + fda->priv[2 ].flags = fdarray_flag__nonfilterable; + +pr_debug ("\nfiltering all but fda->entries[2] (nonfilterable):" ); +fdarray__fprintf_prefix (fda,"before" , stderr); + +/* +* Note: fdarray__filter() in tools/lib/api/fd/array.c evaluates the +* fdarray_flag__nonfilterable flag at the very top of its loop via an +* early continue. Therefore, it completely skips all processing for this +* descriptor, guaranteeing its fd, events, and revents fields remain +* entirely untouched by the filter mask evaluation below it. +*/ + nr_fds =fdarray__filter (fda,POLLHUP ,NULL ,NULL ); +fdarray__fprintf_prefix (fda," after" , stderr); + +if (nr_fds !=0 ) { +pr_debug ("\nfdarray__filter()=%d != 0, should be 0\n" , + nr_fds); + goto out_delete; + } +if (fda->entries[2 ].fd ==-1 ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" ); + goto out_delete; + } + + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
2.55.0.766.g2966f0265a-goog
[email protected]
AI reviewRe: [PATCH v7] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The patch fails to fix the `-EINVAL` pollfd mismatch abort and introduces dead code because it…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The patch fails to fix the -EINVAL pollfd mismatch abort and introduces dead code because it relies on an invalid assumption about fdarray__filter().
- [Medium] The newly added unit test test__fdarray__filter asserts an incorrect invariant about fdarray__filter(), causing deterministic test failure. --
--- Patch [1]: [PATCH v7] perf record: Fix unhandled POLLHUP on non_perf_event descriptors --- commit c8946dae2d12736f0f97f47367d99aaa40d80c15
Author: Ian Rogers <[email protected]>
Subject: perf record: Fix unhandled POLLHUP on non_perf_event descriptors
This commit attempts to fix an abort during perf record teardown caused by index mismatches between a thread's localized poll array and the core evlist poll array when handling POLLHUP. It propagates the finalized -1 state from the core evlist back to the thread's local array before the equivalence check. It also adds a unit test to verify that fdarray__filter preserves nonfilterable items.
Quoted 16 lines
tools/perf/builtin-record.c +8
-0@@ -1180,6 +1180,21 @@ static int record__update_evlist_pollfd_from_thread(struct record *rec,int e_pos = rec->index_map[i].evlist_pollfd_index;int t_pos = rec->index_map[i].thread_pollfd_index; +if (e_entries[e_pos].fd ==-1 || e_entries[e_pos].events ==0 ) { +/* +* If the control file descriptor was closed, then evlist__ctlfd_process() +* will have called evlist__finalize_ctlfd() on the PREVIOUS loop iteration +* to cleanly set the core evlist's e_entries[e_pos].fd to -1. +* +* Since nonfilterable items are skipped by fdarray__filter(), the +* thread's local t_entries[t_pos] retains its original state.
tools/perf/builtin-record.c static int record__update_evlist_pollfd_from_thread(struct record *rec, · line 1200
Quoted 10 lines
tools/perf/builtin-record.c +7
-0+* We must explicitly propagate the finalized -1 state to t_entries +* BEFORE evaluating the strict equivalence check below. +*/ + t_entries[t_pos].fd =-1 ; + t_entries[t_pos].events =0 ; + } +if (e_entries[e_pos].fd != t_entries[t_pos].fd || e_entries[e_pos].events != t_entries[t_pos].events) {pr_err ("Thread and evlist pollfd index mismatch\n" );
[Severity: High]
Does this fix actually resolve the mismatch? A POLLHUP event on the control descriptor causes fdarray__filter to unconditionally clear t_entries[t_pos].events to 0.
Since e_entries has not yet been processed by evlist__ctlfd_process, its events is still POLLIN. This means the e_entries condition evaluates to false, making the new block dead code.
Consequently, the strict equivalence check below evaluates to true (POLLIN != 0), causing the function to abort with -EINVAL.
Quoted 25 lines
tools/perf/tests/fdarray.c +17
-0@@ -80,6 +80,38 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub goto out_delete; } +fdarray__init_revents (fda,POLLHUP ); + fda->priv[2 ].flags = fdarray_flag__nonfilterable; + +pr_debug ("\nfiltering all but fda->entries[2] (nonfilterable):" ); +fdarray__fprintf_prefix (fda,"before" , stderr); + +/* +* Note: fdarray__filter() in tools/lib/api/fd/array.c evaluates the +* fdarray_flag__nonfilterable flag at the very top of its loop via an +* early continue. Therefore, it completely skips all processing for this +* descriptor, guaranteeing its fd, events, and revents fields remain +* entirely untouched by the filter mask evaluation below it. +*/ + nr_fds =fdarray__filter (fda,POLLHUP ,NULL ,NULL ); +fdarray__fprintf_prefix (fda," after" , stderr); + +if (nr_fds !=0 ) {
tools/perf/tests/fdarray.c static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub · line 109
Quoted 10 lines
tools/perf/tests/fdarray.c +10
-0+pr_debug ("\nfdarray__filter()=%d != 0, should be 0\n" , + nr_fds); + goto out_delete; + } +if (fda->entries[2 ].fd ==-1 ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" );
[Severity: Medium]
Will this test pass? The fdarray__filter function in tools/lib/api/fd/array.c actively clears events and revents prior to checking the nonfilterable flag:
if (fda->entries[fd].revents & revents) {
...
fda->entries[fd].revents = fda->entries[fd].events = 0;
continue;
}if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable))
This implementation contradicts the test's assumption that the fields remain untouched, causing the fda->entries[2].revents != POLLHUP assertion to deterministically fail.
Quoted 8 lines
tools/perf/tests/fdarray.c +5
-0+ goto out_delete; + } + + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v8 1/1 perf record: Fix unhandled POLLHUP on non_perf_event descriptors
In reply to Ian RogersWhen processing POLLHUP on a non_perf_event control descriptor, evlist__finalize_ctlfd() is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.
Consequently, on the next iteration of the main recording loop, record__update_evlist_pollfd_from_thread() performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match. This causes an immediate -EINVAL abort and a premature teardown.
Fix the underlying logic within record__update_evlist_pollfd_from_thread() to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.
Additionally, add a unit test that explicitly validates that fdarray__filter() preserves its invariants regarding fdarray_flag__nonfilterable items to guard against regressions.
--- tools/lib/api/fd/array.c | 4 ++++ tools/perf/builtin-record.c | 15 +++++++++++++++ tools/perf/tests/fdarray.c | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+)
tools/lib/api/fd/array.c +4 -0
@@ -116,6 +116,10 @@ int fdarray__filter(struct fdarray *fda, short revents,return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { +/* +* Explicitly bypass nonfilterable items (e.g. control descriptors). +* This ensures their fd, events, and revents remain completely untouched. +*/ if (fda->priv[fd].flags & fdarray_flag__nonfilterable)continue ;
tools/perf/builtin-record.c +15 -0
@@ -1169,6 +1169,21 @@ static int record__update_evlist_pollfd_from_thread(struct record *rec,int e_pos = rec->index_map[i].evlist_pollfd_index;int t_pos = rec->index_map[i].thread_pollfd_index; +if (e_entries[e_pos].fd ==-1 || e_entries[e_pos].events ==0 ) { +/* +* If the control file descriptor was closed, then evlist__ctlfd_process() +* will have called evlist__finalize_ctlfd() on the PREVIOUS loop iteration +* to cleanly set the core evlist's e_entries[e_pos].fd to -1. +* +* Since nonfilterable items are skipped by fdarray__filter(), the +* thread's local t_entries[t_pos] retains its original state. +* We must explicitly propagate the finalized -1 state to t_entries +* BEFORE evaluating the strict equivalence check below. +*/ + t_entries[t_pos].fd =-1 ; + t_entries[t_pos].events =0 ; + } +if (e_entries[e_pos].fd != t_entries[t_pos].fd || e_entries[e_pos].events != t_entries[t_pos].events) {pr_err ("Thread and evlist pollfd index mismatch\n" );
tools/perf/tests/fdarray.c +32 -0
@@ -80,6 +80,38 @@ static int test__fdarray__filter(struct test_suite *test __maybe_unused, int sub goto out_delete; } +fdarray__init_revents (fda,POLLHUP ); + fda->priv[2 ].flags = fdarray_flag__nonfilterable; + +pr_debug ("\nfiltering all but fda->entries[2] (nonfilterable):" ); +fdarray__fprintf_prefix (fda,"before" , stderr); + +/* +* Note: fdarray__filter() in tools/lib/api/fd/array.c evaluates the +* fdarray_flag__nonfilterable flag at the very top of its loop via an +* early continue. Therefore, it completely skips all processing for this +* descriptor, guaranteeing its fd, events, and revents fields remain +* entirely untouched by the filter mask evaluation below it. +*/ + nr_fds =fdarray__filter (fda,POLLHUP ,NULL ,NULL ); +fdarray__fprintf_prefix (fda," after" , stderr); + +if (nr_fds !=0 ) { +pr_debug ("\nfdarray__filter()=%d != 0, should be 0\n" , + nr_fds); + goto out_delete; + } +if (fda->entries[2 ].fd ==-1 ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable fd!" ); + goto out_delete; + } +if (fda->entries[2 ].revents !=POLLHUP ) { +pr_debug ("\nfdarray__filter() illegally modified nonfilterable revents!" ); + goto out_delete; + } + + fda->priv[2 ].flags =0 ;/* reset flags */ +pr_debug ("\n" ); err =0 ;
Signature · 1 line
2.55.0.766.g2966f0265a-goog
[email protected]
AI reviewRe: [PATCH v8] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 19 lines
When processing POLLHUP on a
non_perf_eventcontrol descriptor,evlist__finalize_ctlfd()is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.Consequently, on the next iteration of the main recording loop,
record__update_evlist_pollfd_from_thread()performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match. This causes an immediate-EINVALabort and a premature teardown.Fix the underlying logic within
record__update_evlist_pollfd_from_thread()to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.
Quoted 7 lines
Additionally, add a unit test that explicitly validates that
fdarray__filter()preserves its invariants regardingfdarray_flag__nonfilterableitems to guard against regressions.Fixes fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")Assisted-by Gemini:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerRe: [PATCH v8] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Quoted 24 lines · On Thu, Aug 20, 2026 at 2:47 PM <[email protected]> wrote:
Quoted 24 lines
When processing POLLHUP on a
non_perf_eventcontrol descriptor,evlist__finalize_ctlfd()is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.Consequently, on the next iteration of the main recording loop,
record__update_evlist_pollfd_from_thread()performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match. This causes an immediate-EINVALabort and a premature teardown.Fix the underlying logic within
record__update_evlist_pollfd_from_thread()to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.Additionally, add a unit test that explicitly validates that
fdarray__filter()preserves its invariants regardingfdarray_flag__nonfilterableitems to guard against regressions.
Quoted 5 lines
Quoted 3 lines
Fixes fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")Assisted-by Gemini:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>Sashiko has reviewed this patch and found no issues. It looks great!
Random unnecessary comment: v6, v7 and v8 only differ by comments. The comments were added to avoid hallucinations in the issues Sashiko was seeing; I wasn't seeing the issue internally. This is the first time I've encountered this problem.
Thanks,
Ian
Quoted 3 lines
--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerRe: [PATCH v8] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Quoted 26 lines · Ian Rogers
Quoted 24 lines · On Thu, Aug 20, 2026 at 2:47 PM <[email protected]> wrote:
Quoted 24 lines
When processing POLLHUP on a
non_perf_eventcontrol descriptor,evlist__finalize_ctlfd()is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.Consequently, on the next iteration of the main recording loop,
record__update_evlist_pollfd_from_thread()performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match. This causes an immediate-EINVALabort and a premature teardown.Fix the underlying logic within
record__update_evlist_pollfd_from_thread()to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.Additionally, add a unit test that explicitly validates that
fdarray__filter()preserves its invariants regardingfdarray_flag__nonfilterableitems to guard against regressions.
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Fixes fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")Assisted-by Gemini:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>Sashiko has reviewed this patch and found no issues. It looks great!
Ping. Thanks,
Ian
Quoted 12 lines
Random unnecessary comment: v6, v7 and v8 only differ by comments. The comments were added to avoid hallucinations in the issues Sashiko was seeing; I wasn't seeing the issue internally. This is the first time I've encountered this problem.
Thanks,
IanQuoted 3 lines
--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH v8] perf record: Fix unhandled POLLHUP on non_perf_event descriptors
Quoted 27 lines · Ian Rogers
Quoted 26 lines · Ian Rogers
Quoted 24 lines · On Thu, Aug 20, 2026 at 2:47 PM <[email protected]> wrote:
Quoted 24 lines
When processing POLLHUP on a
non_perf_eventcontrol descriptor,evlist__finalize_ctlfd()is invoked to finalize the setup, correctly setting the core evlist's poll array file descriptor to -1. However, this finalized teardown state is never propagated back to the individual thread's local replica of the pollfd array.Consequently, on the next iteration of the main recording loop,
record__update_evlist_pollfd_from_thread()performs a strict equivalence check between the core evlist's array and the thread's localized poll array, detecting that the fd values no longer match. This causes an immediate-EINVALabort and a premature teardown.Fix the underlying logic within
record__update_evlist_pollfd_from_thread()to sustainably propagate the finalized teardown statuses (-1) originating from the core evlist back to the thread's localized poll structure. This correctly maintains synchronization and entirely prevents the unhandled index mismatch crashes.Additionally, add a unit test that explicitly validates that
fdarray__filter()preserves its invariants regardingfdarray_flag__nonfilterableitems to guard against regressions.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Fixes fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")Assisted-by Gemini:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>Sashiko has reviewed this patch and found no issues. It looks great!
Ping. Thanks,
Ian
Thanks, applied to perf-tools-next, for v7.4.
- Arnaldo