perf ftrace latency: Do not read trace files when BPF is used
Namhyung Kim
maintainerPATCH 1/1 perf ftrace latency: Do not read trace files when BPF is used
I've realized that it didn't set up the tracing files when BPF is used so poll() just returns immediately. It ends up with calling poll() unnecessarily in a loop.
BPF still needs the loop to wait for the target process exiting or a signal from users. Let's sleep for 1 msec and check that.
--- tools/perf/builtin-ftrace.c | 5 +++++ 1 file changed, 5 insertions(+)
tools/perf/builtin-ftrace.c +5 -0
@@ -1146,6 +1146,11 @@ static int __cmd_latency(struct perf_ftrace *ftrace) line[0 ] ='\0' ;while (!done) { +if (ftrace->target.use_bpf) { +usleep (1000 ); +continue ; + } +if (poll (&pollfd,1 ,-1 ) <0 )break ;
Signature · 1 line
2.55.0.897.gb25b4bd76c-goog
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] API misuse where IS_ERR() is used to check libbpf attachment functions that return NULL on error, leading to completely bypassed error handling.
- [High] A Time-of-Check to Time-of-Use (TOCTOU) race condition between checking the done flag and blocking in poll() causes the perf ftrace tool to hang indefinitely. --
commit 005a20a206112aa28a48619fcb3cf2e39a48a24a
Author: Namhyung Kim <[email protected]>
perf ftrace latency: Do not read trace files when BPF is used
This commit modifies the perf ftrace latency command to avoid reading tracing files when BPF is in use, as the tracing files are not set up and poll() returns immediately. Instead, it introduces a small sleep loop to wait for the target process to exit or a signal to arrive.
Quoted 5 lines
tools/perf/builtin-ftrace.c +0
-0
tools/perf/builtin-ftrace.c static int __cmd_latency(struct perf_ftrace *ftrace) · line 1155
Quoted 10 lines
tools/perf/builtin-ftrace.c +5
-0@@ -1146,6 +1146,11 @@ static int __cmd_latency(struct perf_ftrace *ftrace) line[0 ] ='\0' ;while (!done) { +if (ftrace->target.use_bpf) { +usleep (1000 ); +continue ; + } +if (poll (&pollfd,1 ,-1 ) <0 )break ;
[Severity: High]
This isn't a bug introduced by this patch, but does this main loop have a race condition between checking the done flag and blocking in poll?
If a terminating signal (like sigchld from a completed workload or sigint from the user) is handled immediately after the while (!done) check but before the poll(&pollfd, 1, -1) syscall, the done flag will be set but the signal is consumed.
The main thread then executes poll() with an infinite timeout (-1).
Since the workload has exited, no new trace events will arrive on trace_fd, causing poll() to block forever and freeze the tool until killed manually.
Could this be avoided by using ppoll() with a signal mask, or by using a timeout instead of blocking indefinitely?
This same pattern also appears to be present in the standard trace command loop in __cmd_ftrace().
While reviewing the BPF functionality for ftrace, I also noticed a potential issue in perf_ftrace__latency_prepare_bpf():
tools/perf/util/bpf_ftrace.c:perf_ftrace__latency_prepare_bpf() {
...
if (func) {
skel->links.func_begin = bpf_program__attach_kprobe(skel->progs.func_begin,
false, func->name);
if (IS_ERR(skel->links.func_begin)) {
pr_err("Failed to attach fentry program\n");
err = PTR_ERR(skel->links.func_begin);
goto out;
}
...
}[Severity: High]
This is a pre-existing issue, but does this error check work as intended?
Functions like bpf_program__attach_kprobe() and bpf_program__attach_raw_tracepoint() typically return NULL on error and set errno.
If the attachment fails, IS_ERR(NULL) evaluates to false, which bypasses this error handling path entirely.
This would cause the tool to falsely report success while silently failing to collect data.
Should these attachment checks compare against NULL instead?
A similar check is also used for skel->links.event_begin later in the same function.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerQuoted 20 lines · Namhyung Kim
I've realized that it didn't set up the tracing files when BPF is used so
poll()just returns immediately. It ends up with callingpoll()unnecessarily in a loop.BPF still needs the loop to wait for the target process exiting or a signal from users. Let's sleep for 1 msec and check that.
Cc Steven Rostedt <[email protected]>Cc Masami Hiramatsu <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/builtin-ftrace.c | 5 +++++ 1 file changed, 5 insertions(+)tools/perf/builtin-ftrace.c +0
-0
tools/perf/builtin-ftrace.c static int __cmd_latency(struct perf_ftrace *ftrace) · line 1153
Quoted 8 lines
tools/perf/builtin-ftrace.c +5
-0@@ -1146,6 +1146,11 @@ static int __cmd_latency(struct perf_ftrace *ftrace) line[0 ] ='\0' ;while (!done) { +if (ftrace->target.use_bpf) { +usleep (1000 ); +continue ; + } +
To avoid the busy waiting could we:
```
// Globaly:
sem_t sig_sem;
...
// Prior to setting up the signal handler:
sem_init(&sig_sem, 0, 0);
...
// Here:
if (ftrace->target.use_bpf) {
sem_wait(&sig_sem);
} else {
while (!done) {
...
}
// In the signal handler:
if (ftrace->target.use_bpf) {
sem_post(&sig_sem);
}
...
// After removing the signal handler:
sem_destroy(&sig_sem);
```Thanks,
Ian
Quoted 6 lines
tools/perf/builtin-ftrace.c +0
-1if (poll (&pollfd,1 ,-1 ) <0 )break ; --2.55.0.897.gb25b4bd76c-goog
Namhyung Kim
maintainerHi Ian,
Quoted 45 lines · Ian Rogers
Quoted 20 lines · Namhyung Kim
I've realized that it didn't set up the tracing files when BPF is used so
poll()just returns immediately. It ends up with callingpoll()unnecessarily in a loop.BPF still needs the loop to wait for the target process exiting or a signal from users. Let's sleep for 1 msec and check that.
Cc Steven Rostedt <[email protected]>Cc Masami Hiramatsu <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/builtin-ftrace.c | 5 +++++ 1 file changed, 5 insertions(+)tools/perf/builtin-ftrace.c +0
-0tools/perf/builtin-ftrace.c static int __cmd_latency(struct perf_ftrace *ftrace) · line 1153
Quoted 8 lines
tools/perf/builtin-ftrace.c +5
-0@@ -1146,6 +1146,11 @@ static int __cmd_latency(struct perf_ftrace *ftrace) line[0 ] ='\0' ;while (!done) { +if (ftrace->target.use_bpf) { +usleep (1000 ); +continue ; + } +To avoid the busy waiting could we:
``` // Globaly: sem_t sig_sem; ... // Prior to setting up the signal handler: sem_init(&sig_sem, 0, 0); ... // Here: if (ftrace->target.use_bpf) { sem_wait(&sig_sem); } else { while (!done) { ... }
Quoted 8 lines
// In the signal handler:
if (ftrace->target.use_bpf) {
sem_post(&sig_sem);
}
...
// After removing the signal handler:
sem_destroy(&sig_sem);
```Looks better, will check.
Thanks,
Namhyung