File descriptor interface for BPF streams
Subsystems (6)
Kumar Kartikeya Dwivedi
maintainerv2 cover File descriptor interface for BPF streams
BPF program stdout and stderr streams can currently be consumed only through BPF_PROG_STREAM_READ_BY_FD. This requires repeated bpf() calls and provides no way to block for output or integrate with poll-based event loops.
Add BPF_PROG_STREAM_OPEN to return a read-only, close-on-exec descriptor for a program stream. Reads block by default, with an option for non-blocking mode. poll and epoll report readable data and report hangup when the program is freed, while allowing buffered data to be drained before EOF. Stream descriptors retain the stream storage without retaining the program itself.
Expose the command through libbpf and switch bpftool tracelog to blocking stream reads. bpftool falls back to BPF_PROG_STREAM_READ_BY_FD on older kernels, where an unknown bpf() command returns EINVAL. The legacy command remains supported.
The first patch fixes a pre-existing issue where empty stream writes allocate elements that escape capacity accounting; the readiness tracking added later relies on every element carrying data.
See commit logs for details.
Changelog:
----------
v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]
* Fold the irq_work and readable-counter patches into the interface patch so no intermediate state wakes waiters directly or derives readiness from the capacity counter. (BPF CI)
* Accept only BPF_F_STREAM_NONBLOCK; BPF_F_RDONLY is no longer accepted since the descriptor is always read-only.
* Allocate streams only for programs loaded through BPF_PROG_LOAD, not for classic BPF filters, JIT subprograms or shim programs.
* Add a patch that skips zero-length stream writes instead of allocating elements that bypass capacity accounting. (Emil, Sashiko)
* Report EOF only when the stream was already dead before it was found empty, so data published right before program teardown is not lost. (Sashiko)
* Document that POLLHUP follows program destruction, not the caller's own release, and that it may lag the final reference drop.
* Drop the llseek operation so lseek fails with ESPIPE like other stream descriptors. (Emil)
* Drop the unreachable length check in the file read path; the VFS caps read sizes below INT_MAX.
* Let __bpf_prog_free() clean up after a failed stream allocation instead of freeing the streams twice, and drop redundant zeroing of the stream counters after kzalloc().
* Explain why wakeups are always deferred through irq_work and drop the batching rationale. (Emil)
* Skip irq_work_sync() at teardown for streams that never queued a notification; on PREEMPT_RT it waits for an RCU grace period that every program free, including cBPF filters, would otherwise pay. (Sashiko)
* Qualify the POLLIN followed by EAGAIN claim to a single reader. (BPF CI)
* Handle SIGINT, SIGHUP and SIGTERM in bpftool so a followed stream exits cleanly, and document the follow behavior. (Emil, BPF CI)
* Drop bpftool's program reference once the stream is open so following ends with EOF when the program is unloaded, and restore the previous signal dispositions afterwards for batch mode. (Sashiko)
* Report bpftool read errors on the fallback path as well.
* Test lseek rejection and empty stream writes.
* Retry the NMI test write on a later sample if the first attempt fails.
* Drop Emil's Reviewed-by from the patches that changed.
* Rebase on bpf-next.
Kumar Kartikeya Dwivedi (5):
bpf: Skip zero-length stream writes
bpf: Add file descriptor interface for program streams
libbpf: Add bpf_prog_stream_open()
bpftool: Read program streams through file descriptors selftests/bpf: Test program stream file descriptors
include/linux/bpf.h | 14 +- include/uapi/linux/bpf.h | 39 ++ kernel/bpf/core.c | 8 +- kernel/bpf/stream.c | 211 ++++++++- kernel/bpf/syscall.c | 29 ++ .../bpftool/Documentation/bpftool-prog.rst | 5 + tools/bpf/bpftool/prog.c | 62 ++- tools/include/uapi/linux/bpf.h | 39 ++ tools/lib/bpf/bpf.c | 19 + tools/lib/bpf/bpf.h | 24 + tools/lib/bpf/libbpf.map | 1 + .../testing/selftests/bpf/prog_tests/stream.c | 415 ++++++++++++++++++ tools/testing/selftests/bpf/progs/stream.c | 20 + 13 files changed, 853 insertions(+), 33 deletions(-)
base-commit: 740eb74b8d2e3b7850e55b15257c8c0b0fc4a125
Signature · 1 line
2.53.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 1/5 bpf: Skip zero-length stream writes
A stream write whose formatted output is empty still allocates a stream element and links it into the stream log. Capacity accounting only counts payload bytes, so such elements never count against the stream limit, and a program that keeps producing empty output allocates elements without bound until a reader drains them.
Return success without allocating anything when there is nothing to write, for both bpf_stream_vprintk() and staged writes. Every element in a stream log then carries at least one byte, which later changes rely on when they derive read readiness from published byte counts.
--- kernel/bpf/stream.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/bpf/stream.c +8 -3
@@ -75,8 +75,13 @@ static void bpf_stream_release_capacity(struct bpf_stream *stream, int len)static int bpf_stream_push_str (struct bpf_stream *stream,const char *str,int len) { -int ret =bpf_stream_consume_capacity (stream, len); +int ret; + +/* Nothing to publish; do not allocate an element for it. */ +if (!len) +return 0 ; + ret =bpf_stream_consume_capacity (stream, len);if (ret)return ret;
@@ -333,8 +338,8 @@ int bpf_stream_stage_printk(struct bpf_stream_stage *ss, const char *fmt, ...)va_start (args, fmt); len =vscnprintf (buf->buf,ARRAY_SIZE (buf->buf), fmt, args);va_end (args); -/* Exclude NULL byte during push. */ - ret =__bpf_stream_push_str (&ss->log, buf->buf, len); +/* Exclude NULL byte during push; skip empty output entirely. */ + ret = len ?__bpf_stream_push_str (&ss->log, buf->buf, len) :0 ;if (!ret) ss->len += len;bpf_put_buffers ();
Signature · 1 line
2.53.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 2/5 bpf: Add file descriptor interface for program streams
In reply to Kumar Kartikeya DwivediThe existing BPF_PROG_STREAM_READ_BY_FD command only supports polling a program stream through repeated bpf() calls. It cannot block for new data or integrate with poll-based event loops.
Add BPF_PROG_STREAM_OPEN to return a read-only, close-on-exec file descriptor for a selected program stream. Reads block by default and BPF_F_STREAM_NONBLOCK, the only accepted flag, provides non-blocking behavior. poll reports readable data and reports hangup once the program has been freed. Like pipes and sockets, the descriptor is not seekable and lseek fails with ESPIPE.
A stream descriptor deliberately does not retain the program. Move each stream into a separately refcounted allocation so program teardown can mark it dead and wake descriptor users while outstanding descriptors drain buffered data safely. Readers sample the dead flag before looking for data, so EOF is reported only when the stream was already dead before it was found empty; data published right before teardown is never skipped.
Only programs loaded through BPF_PROG_LOAD get streams. Classic BPF filters, JIT subprograms and shim programs never write to one, and kernel-side writers already resolve a subprogram to its main program, so those programs no longer carry stream state.
Readiness needs its own counter. Stream capacity is charged before allocation and before an element is published to the stream log, so using that reservation as the read and poll condition can report readable data while no element exists: a blocking reader retries instead of sleeping and a lone non-blocking reader can see POLLIN followed by EAGAIN. Publish bytes with release ordering after adding elements to the lockless log, use acquire loads before consuming them or reporting readiness, limit each read to its readable snapshot and subtract only bytes actually copied. This keeps the aggregate count correct even when concurrent publishers update it out of publication order. With several readers on one stream, readiness remains advisory, as it is for pipes. The capacity counter is kept solely for enforcing the stream size limit.
Wakeups are always deferred through irq_work. Stream writers run in whatever context the program runs in: NMI context for perf_event programs, sections with interrupts disabled inside bpf_spin_lock or rqspinlock critical sections since bpf_stream_vprintk() is KF_SPINLOCK_SAFE, and tracing programs attached anywhere in the kernel, including inside the wait queue and epoll code itself. Waking waiters directly from there can deadlock, and no cheap context check covers every case: on PREEMPT_RT, spinlock_t sections do not disable interrupts, so in_nmi() or irqs_disabled() cannot tell such a program apart from a benign one. Queue an irq_work item after publishing instead, as bpf_ringbuf does. Coalescing of concurrent publications is a side effect that is compatible with poll and epoll semantics: notifications prompt waiters to re-evaluate persistent readable state, and EPOLLET consumers drain until EAGAIN.
Synchronize pending work before releasing the final stream reference so the callback cannot outlive the stream, but only when the work was ever
queued: irq_work_sync() waits for an RCU grace period on PREEMPT_RT and on architectures without an irq_work interrupt.
--- include/linux/bpf.h | 14 ++- include/uapi/linux/bpf.h | 39 +++++++ kernel/bpf/core.c | 8 +- kernel/bpf/stream.c | 200 ++++++++++++++++++++++++++++++--- kernel/bpf/syscall.c | 29 +++++ tools/include/uapi/linux/bpf.h | 39 +++++++ 6 files changed, 306 insertions(+), 23 deletions(-)
include/linux/bpf.h +11 -3
@@ -17,6 +17,7 @@#include <linux/numa.h> #include <linux/mm_types.h> #include <linux/wait.h> +#include <linux/irq_work_types.h> #include <linux/refcount.h> #include <linux/mutex.h> #include <linux/module.h>
@@ -1770,12 +1771,18 @@ enum { };struct bpf_stream { -atomic_t capacity; +refcount_t refcnt; +atomic_t capacity;/* bytes reserved against the stream limit */ +atomic_t readable;/* published bytes available to readers */ struct llist_head log;/* list of in-flight stream elements in LIFO order */ struct mutex lock;/* lock protecting backlog_{head,tail} */ struct llist_node *backlog_head;/* list of in-flight stream elements in FIFO order */ struct llist_node *backlog_tail;/* tail of the list above */ +wait_queue_head_t waitq; +struct irq_work notify_work; +bool notify_used;/* notify_work was queued at least once */ +bool dead; };struct bpf_stream_stage {
@@ -1914,7 +1921,7 @@ struct bpf_prog_aux {struct work_struct work;struct rcu_head rcu; }; -struct bpf_stream stream[2 ]; +struct bpf_stream *stream[2 ];struct mutex st_ops_assoc_mutex;struct bpf_map __rcu *st_ops_assoc; };
@@ -4223,9 +4230,10 @@ void bpf_bprintf_cleanup(struct bpf_bprintf_data *data);int bpf_try_get_buffers (struct bpf_bprintf_buffers **bufs);void bpf_put_buffers (void ); -void bpf_prog_stream_init (struct bpf_prog *prog); +int bpf_prog_stream_init (struct bpf_prog *prog,gfp_t gfp_extra_flags);void bpf_prog_stream_free (struct bpf_prog *prog);int bpf_prog_stream_read (struct bpf_prog *prog,enum bpf_stream_id stream_id,void __user *buf,u32 len); +int bpf_prog_stream_new_fd (struct bpf_prog *prog,enum bpf_stream_id stream_id,u32 flags);void bpf_stream_stage_init (struct bpf_stream_stage *ss);void bpf_stream_stage_free (struct bpf_stream_stage *ss);__printf (2 ,3 )
include/uapi/linux/bpf.h +39 -0
@@ -936,6 +936,33 @@ union bpf_iter_link_info {* 0 on success or -1 if an error occurred (in which case, * *errno* is set appropriately). * +* BPF_PROG_STREAM_OPEN +* Description +* Open a file descriptor for one of the BPF streams associated +* with the program identified by *prog_fd*. The stream is selected +* by *stream_id*. +* +* The returned file descriptor supports **read**\ (2) and +* **poll**\ (2). Reads block while the stream is empty unless +* **BPF_F_STREAM_NONBLOCK** is specified in *flags*. A non-blocking +* read of an empty stream fails with **EAGAIN**. +* +* **poll**\ (2) reports **POLLIN** when data is available and +* **POLLHUP** once the program has been freed, that is, after every +* reference to it, including links and other file descriptors, has +* been dropped. Hangup may lag the final release because program +* teardown is deferred. Buffered data remains readable after +* **POLLHUP** and a read returns zero after all such data has been +* consumed. +* +* The file descriptor is read-only and has the close-on-exec flag +* set. It is not seekable and **lseek**\ (2) fails with **ESPIPE**. +* *flags* may only contain **BPF_F_STREAM_NONBLOCK**. +* +* Return +* A new file descriptor (a nonnegative integer), or -1 if an +* error occurred (in which case, *errno* is set appropriately). +* * NOTES * eBPF objects (maps and programs) can be shared between processes. *
@@ -993,6 +1020,7 @@ enum bpf_cmd {BPF_TOKEN_CREATE ,BPF_PROG_STREAM_READ_BY_FD ,BPF_PROG_ASSOC_STRUCT_OPS , +BPF_PROG_STREAM_OPEN , __MAX_BPF_CMD, BPF_COMMON_ATTRS = 1 << 16, /* Indicate carrying syscall common attrs. */ };
@@ -1524,6 +1552,11 @@ enum { BPF_STREAM_STDERR = 2, }; +/* flags for BPF_PROG_STREAM_OPEN command */ +enum { +BPF_F_STREAM_NONBLOCK = (1U <<0 ), +}; +union bpf_attr {struct {/* anonymous struct used by BPF_MAP_CREATE command */ __u32 map_type;/* one of enum bpf_map_type */
@@ -1950,6 +1983,12 @@ union bpf_attr {__u32 flags; } prog_assoc_struct_ops; +struct { +__u32 prog_fd; +__u32 stream_id; +__u32 flags; + } prog_stream_open; + } __attribute__((aligned(8)));/* The description below is an attempt at providing documentation to eBPF
kernel/bpf/core.c +3 -5
@@ -141,10 +141,6 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flagmutex_init (&fp->aux->dst_mutex);mutex_init (&fp->aux->st_ops_assoc_mutex); -#ifdef CONFIG_BPF_SYSCALL -bpf_prog_stream_init (fp); -#endif -return fp; }
@@ -288,6 +284,9 @@ struct bpf_prog *bpf_prog_realloc(struct bpf_prog *fp_old, unsigned int size,void __bpf_prog_free (struct bpf_prog *fp) {if (fp->aux) { +#ifdef CONFIG_BPF_SYSCALL +bpf_prog_stream_free (fp); +#endif mutex_destroy (&fp->aux->used_maps_mutex);mutex_destroy (&fp->aux->dst_mutex);mutex_destroy (&fp->aux->st_ops_assoc_mutex);
@@ -3073,7 +3072,6 @@ static void bpf_prog_free_deferred(struct work_struct *work) aux = container_of(work, struct bpf_prog_aux, work);#ifdef CONFIG_BPF_SYSCALL bpf_free_kfunc_btf_tab (aux->kfunc_btf_tab); -bpf_prog_stream_free (aux->prog);#endif #ifdef CONFIG_CGROUP_BPF if (aux->cgroup_atype !=CGROUP_BPF_ATTACH_TYPE_INVALID )
kernel/bpf/stream.c +185 -15
284 lines not on this page · raw message
kernel/bpf/syscall.c +29 -0
@@ -3080,6 +3080,10 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at prog->aux->user =get_current_user (); prog->len = attr->insn_cnt; + err =bpf_prog_stream_init (prog,GFP_USER ); +if (err) + goto free_prog; + err = -EFAULT ; if (copy_from_bpfptr(prog->insns, make_bpfptr(attr->insns, uattr.is_kernel),
@@ -6313,6 +6317,28 @@ static int prog_assoc_struct_ops(union bpf_attr *attr)return ret; } +#define BPF_PROG_STREAM_OPEN_LAST_FIELD prog_stream_open.flags + +static int prog_stream_open (union bpf_attr *attr) +{ +struct bpf_prog *prog; +u32 flags = attr->prog_stream_open.flags; +int ret; + +if (CHECK_ATTR (BPF_PROG_STREAM_OPEN )) +return -EINVAL ; +if (flags & ~BPF_F_STREAM_NONBLOCK ) +return -EINVAL ; + + prog =bpf_prog_get (attr->prog_stream_open.prog_fd); +if (IS_ERR (prog)) +return PTR_ERR (prog); + + ret =bpf_prog_stream_new_fd (prog, attr->prog_stream_open.stream_id, flags); +bpf_prog_put (prog); +return ret; +} +static int __sys_bpf (enum bpf_cmd cmd,bpfptr_t uattr,unsigned int size,bpfptr_t uattr_common,unsigned int size_common) {
@@ -6485,6 +6511,9 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size,case BPF_PROG_ASSOC_STRUCT_OPS : err =prog_assoc_struct_ops (&attr);break ; +case BPF_PROG_STREAM_OPEN : + err =prog_stream_open (&attr); +break ;default : err = -EINVAL ;break ;
tools/include/uapi/linux/bpf.h +39 -0
@@ -936,6 +936,33 @@ union bpf_iter_link_info {* 0 on success or -1 if an error occurred (in which case, * *errno* is set appropriately). * +* BPF_PROG_STREAM_OPEN +* Description +* Open a file descriptor for one of the BPF streams associated +* with the program identified by *prog_fd*. The stream is selected +* by *stream_id*. +* +* The returned file descriptor supports **read**\ (2) and +* **poll**\ (2). Reads block while the stream is empty unless +* **BPF_F_STREAM_NONBLOCK** is specified in *flags*. A non-blocking +* read of an empty stream fails with **EAGAIN**. +* +* **poll**\ (2) reports **POLLIN** when data is available and +* **POLLHUP** once the program has been freed, that is, after every +* reference to it, including links and other file descriptors, has +* been dropped. Hangup may lag the final release because program +* teardown is deferred. Buffered data remains readable after +* **POLLHUP** and a read returns zero after all such data has been +* consumed. +* +* The file descriptor is read-only and has the close-on-exec flag +* set. It is not seekable and **lseek**\ (2) fails with **ESPIPE**. +* *flags* may only contain **BPF_F_STREAM_NONBLOCK**. +* +* Return +* A new file descriptor (a nonnegative integer), or -1 if an +* error occurred (in which case, *errno* is set appropriately). +* * NOTES * eBPF objects (maps and programs) can be shared between processes. *
@@ -993,6 +1020,7 @@ enum bpf_cmd {BPF_TOKEN_CREATE ,BPF_PROG_STREAM_READ_BY_FD ,BPF_PROG_ASSOC_STRUCT_OPS , +BPF_PROG_STREAM_OPEN , __MAX_BPF_CMD, BPF_COMMON_ATTRS = 1 << 16, /* Indicate carrying syscall common attrs. */ };
@@ -1524,6 +1552,11 @@ enum { BPF_STREAM_STDERR = 2, }; +/* flags for BPF_PROG_STREAM_OPEN command */ +enum { +BPF_F_STREAM_NONBLOCK = (1U <<0 ), +}; +union bpf_attr {struct {/* anonymous struct used by BPF_MAP_CREATE command */ __u32 map_type;/* one of enum bpf_map_type */
@@ -1950,6 +1983,12 @@ union bpf_attr {__u32 flags; } prog_assoc_struct_ops; +struct { +__u32 prog_fd; +__u32 stream_id; +__u32 flags; + } prog_stream_open; + } __attribute__((aligned(8)));/* The description below is an attempt at providing documentation to eBPF
Signature · 1 line
2.53.0
Re: [PATCH bpf-next v2 2/5] bpf: Add file descriptor interface for program streams
kernel/bpf/stream.c static void bpf_stream_release_capacity(struct bpf_stream *stream, int len) · line 114
Quoted 6 lines · Kumar Kartikeya Dwivedi
kernel/bpf/stream.c +6
-0+static void bpf_stream_publish (struct bpf_stream *stream,int len) +{ +/* Pairs with atomic_read_acquire() in bpf_stream_readable_bytes(). */ + (void )atomic_add_return_release (len, &stream->readable); +bpf_stream_queue_notify (stream); +}
This queues irq_work on every bpf_stream_vprintk() and every stage commit, whether anyone opened the stream or not.
That's an irq_work interrupt per print for every existing user. ringbuf queues it only when the consumer caught up.
The result of atomic_add_return_release() is thrown away. can we queue the work only when it returns len, i.e. when the stream was empty ?
pw-bot: cr
Kumar Kartikeya Dwivedi
maintainerPATCH v2 3/5 libbpf: Add bpf_prog_stream_open()
In reply to Kumar Kartikeya DwivediExpose the BPF_PROG_STREAM_OPEN command through a low-level libbpf wrapper. The options structure carries stream open flags, including non-blocking mode, while leaving room for future extensions.
--- tools/lib/bpf/bpf.c | 19 +++++++++++++++++++ tools/lib/bpf/bpf.h | 24 ++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 44 insertions(+)
tools/lib/bpf/bpf.c +19 -0
@@ -1466,6 +1466,25 @@ int bpf_prog_stream_read(int prog_fd, __u32 stream_id, void *buf, __u32 buf_len,return libbpf_err_errno (err); } +int bpf_prog_stream_open (int prog_fd,__u32 stream_id, +const struct bpf_prog_stream_open_opts *opts) +{ + const size_t attr_sz = offsetofend(union bpf_attr, prog_stream_open); +union bpf_attr attr; +int fd; + +if (!OPTS_VALID (opts, bpf_prog_stream_open_opts)) +return libbpf_err (-EINVAL ); + +memset (&attr,0 , attr_sz); + attr.prog_stream_open.prog_fd = prog_fd; + attr.prog_stream_open.stream_id = stream_id; + attr.prog_stream_open.flags =OPTS_GET (opts, flags,0 ); + + fd =sys_bpf_fd (BPF_PROG_STREAM_OPEN , &attr, attr_sz); +return libbpf_err_errno (fd); +} +int bpf_prog_assoc_struct_ops (int prog_fd,int map_fd,struct bpf_prog_assoc_struct_ops_opts *opts) {
tools/lib/bpf/bpf.h +24 -0
@@ -759,10 +759,34 @@ struct bpf_prog_stream_read_opts {* * @return The number of bytes read, on success; negative error code, otherwise * (errno is also set to the error code) +* +* For blocking reads and polling, prefer **bpf_prog_stream_open**. */ LIBBPF_API int bpf_prog_stream_read(int prog_fd, __u32 stream_id, void *buf, __u32 buf_len,struct bpf_prog_stream_read_opts *opts); +struct bpf_prog_stream_open_opts { +size_t sz; +__u32 flags; + size_t :0; +}; +#define bpf_prog_stream_open_opts__last_field flags + +/** +* @brief **bpf_prog_stream_open** opens a file descriptor for a BPF stream of +* a given BPF program. +* +* @param prog_fd FD for the BPF program whose BPF stream is to be opened. +* @param stream_id ID of the BPF stream to be opened. +* @param opts optional options, can be NULL. BPF_F_STREAM_NONBLOCK requests a +* non-blocking descriptor. +* +* @return A new stream FD, on success; negative error code, otherwise (errno +* is also set to the error code) +*/ +LIBBPF_API int bpf_prog_stream_open(int prog_fd, __u32 stream_id, +const struct bpf_prog_stream_open_opts *opts); +struct bpf_prog_assoc_struct_ops_opts {size_t sz;__u32 flags;
tools/lib/bpf/libbpf.map +1 -0
@@ -459,6 +459,7 @@ LIBBPF_1.7.0 { LIBBPF_1.8.0 { global: bpf_map__attach_cgroup_opts; + bpf_prog_stream_open; bpf_program__add_flags; bpf_program__attach_tracing_multi; bpf_program__clear_flags;
Signature · 1 line
2.53.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 4/5 bpftool: Read program streams through file descriptors
In reply to Kumar Kartikeya DwivediUse bpf_prog_stream_open() in its default blocking mode when following a program stream. This lets bpftool wait for later output instead of exiting as soon as it drains the currently buffered bytes. Flush each chunk as it arrives so redirected output is not held in stdio buffers while the next read blocks.
A stream descriptor does not keep its program alive, so bpftool drops the program descriptor once the stream is open. The command then ends with EOF when the program is unloaded and otherwise runs until interrupted, as bpftool prog tracelog already does for the trace pipe. Install SIGINT, SIGHUP and SIGTERM handlers without SA_RESTART so an interrupted read ends the loop, check the stop flag before each read so a signal that lands while output is being printed ends it as well, and exit successfully in both cases. Restore the previous dispositions afterwards so batch mode keeps its usual signal behavior. Report errors from the fallback read path too.
Fall back to BPF_PROG_STREAM_READ_BY_FD when the open command returns EINVAL. The bpf() syscall uses EINVAL for unknown commands, so this keeps a new bpftool compatible with kernels that provide program streams but predate their file descriptor interface. On such kernels the command still dumps the buffered output and exits. Document both behaviors.
--- .../bpftool/Documentation/bpftool-prog.rst | 5 ++ tools/bpf/bpftool/prog.c | 62 ++++++++++++++++--- 2 files changed, 60 insertions(+), 7 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-prog.rst +5 -0
@@ -186,6 +186,11 @@ bpftool prog tracelog { stdout | stderr } *PROG* error messages to the standard error stream. This facility should be used only for debugging purposes. + On kernels that support opening a stream as a file descriptor, bpftool + keeps printing new output as the program produces it, until the program is + unloaded or <Ctrl+C> is hit. Older kernels only allow dumping the output + buffered so far, after which bpftool exits. + bpftool prog run *PROG* data_in *FILE* [data_out *FILE* [data_size_out *L*]] [ctx_in *FILE* [ctx_out *FILE* [ctx_size_out *M*]]] [repeat *N*] Run BPF program *PROG* in the kernel testing infrastructure for BPF, meaning that the program works on the data and context provided by the
tools/bpf/bpftool/prog.c +55 -7
76 lines not on this page · raw message
Signature · 1 line
2.53.0
Re: [PATCH bpf-next v2 4/5] bpftool: Read program streams through file descriptors
Quoted 5 lines · Kumar Kartikeya Dwivedi
Use
bpf_prog_stream_open()in its default blocking mode when following a program stream. This lets bpftool wait for later output instead of exiting as soon as it drains the currently buffered bytes. Flush each chunk as it arrives so redirected output is not held in stdio buffers while the next read blocks.
So the same 'bpftool prog tracelog stderr PROG' prints what is buffered and exits on old kernels and doesn't return on new ones while the prog is loaded. Scripts that dump the stream will hang.
Probably should be opt-in?
[...]
tools/bpf/bpftool/prog.c enum prog_tracelog_mode { · line 1171
Quoted 2 lines
tools/bpf/bpftool/prog.c +2
-0+while (!stream_stop) { + ret =read (fd, buf,sizeof (buf));
A signal that arrives after the stream_stop check and before read() blocks only sets the flag. read() then sleeps until the next print or the next signal.exit_tracelog() calls exit(0) from the handler.
can we do the same here ?
Then stream_stop and save/restore of old[] are not needed.
Kumar Kartikeya Dwivedi
maintainerPATCH v2 5/5 selftests/bpf: Test program stream file descriptors
In reply to Kumar Kartikeya DwivediExercise validation and normal file semantics of BPF_PROG_STREAM_OPEN. Cover read-only and close-on-exec state, write and lseek rejection, non-blocking reads, partial reads, poll readiness, and a blocking reader woken by new stream data. Also check that an empty stream write produces neither readable data nor poll readiness.
Register the descriptor with EPOLLET, consume the initial data event without draining buffered output, then release the program. Require a second edge carrying POLLIN and POLLHUP, drain the buffered bytes, and verify that poll continues to report HUP before read returns EOF. This ensures program teardown notifies edge-triggered loops even when readability never transitions.
Also block a reader on an empty live stream, release the program, and verify that it wakes and returns EOF. This directly covers the teardown wait condition and its no-lost-wakeup ordering.
Finally, attach a stream-writing program to a hardware perf event and busy-poll its epoll registration until output arrives. This both drives the per-thread event into NMI context and directly verifies that the deferred irq_work notification reaches epoll, without a separate BSS completion handshake.
--- .../testing/selftests/bpf/prog_tests/stream.c | 415 ++++++++++++++++++ tools/testing/selftests/bpf/progs/stream.c | 20 + 2 files changed, 435 insertions(+)
tools/testing/selftests/bpf/prog_tests/stream.c +415 -0
@@ -1,11 +1,17 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2025 Meta Platforms, Inc. and affiliates. */ #include <test_progs.h> +#include <linux/perf_event.h> +#include <poll.h> +#include <sys/epoll.h> #include <sys/mman.h> +#include <sys/syscall.h> #include "stream.skel.h" #include "stream_fail.skel.h" +#define NMI_TIMEOUT_NS (5ULL * 1000 * 1000 * 1000) +void test_stream_failure (void ) {RUN_TESTS (stream_fail);
@@ -62,6 +68,415 @@ void test_stream_syscall(void)stream__destroy (skel); } +static bool stream_fd_trigger (struct bpf_program *prog) +{ +LIBBPF_OPTS (bpf_test_run_opts, opts); +int ret; + + ret =bpf_prog_test_run_opts (bpf_program__fd (prog), &opts); +return ASSERT_OK (ret,"test_run" ) &&ASSERT_OK (opts.retval,"retval" ); +} + +static void test_stream_fd_open (void ) +{ +LIBBPF_OPTS (bpf_prog_stream_open_opts, opts); +struct stream *skel; +int fd, prog_fd; + + skel =stream__open_and_load (); +if (!ASSERT_OK_PTR (skel,"stream__open_and_load" )) +return ; + + prog_fd =bpf_program__fd (skel->progs.stream_syscall); + fd =bpf_prog_stream_open (0 ,BPF_STREAM_STDOUT ,NULL ); +ASSERT_EQ (fd, -EINVAL ,"bad_prog_fd" ); + + fd =bpf_prog_stream_open (prog_fd,0 ,NULL ); +ASSERT_EQ (fd, -ENOENT ,"bad_stream_id" ); + + opts.flags =BPF_F_RDONLY ; + fd =bpf_prog_stream_open (prog_fd,BPF_STREAM_STDOUT , &opts); +ASSERT_EQ (fd, -EINVAL ,"access_flag" ); + + opts.flags =1U <<31 ; + fd =bpf_prog_stream_open (prog_fd,BPF_STREAM_STDOUT , &opts); +ASSERT_EQ (fd, -EINVAL ,"unknown_flag" ); + + fd =bpf_prog_stream_open (prog_fd,BPF_STREAM_STDERR ,NULL ); +if (ASSERT_OK_FD (fd,"stderr" )) +close (fd); + +stream__destroy (skel); +} + +static void test_stream_fd_nonblock (void ) +{ +LIBBPF_OPTS (bpf_prog_stream_open_opts, opts, + .flags = BPF_F_STREAM_NONBLOCK, + ); +struct pollfd pfd = { .events =POLLIN |POLLHUP }; +struct stream *skel; +char buf[4 ] = {}; +int fd, flags, ret; + + skel =stream__open_and_load (); +if (!ASSERT_OK_PTR (skel,"stream__open_and_load" )) +return ; + + fd =bpf_prog_stream_open (bpf_program__fd (skel->progs.stream_syscall), +BPF_STREAM_STDOUT , &opts); +if (!ASSERT_OK_FD (fd,"stream_open" )) + goto out_destroy; + pfd.fd = fd; + + flags =fcntl (fd,F_GETFD ); +ASSERT_GE (flags,0 ,"getfd" ); +ASSERT_NEQ (flags &FD_CLOEXEC ,0 ,"cloexec" ); + flags =fcntl (fd,F_GETFL ); +ASSERT_GE (flags,0 ,"getfl" ); +ASSERT_EQ (flags &O_ACCMODE ,O_RDONLY ,"readonly" ); +ASSERT_NEQ (flags &O_NONBLOCK ,0 ,"nonblock" ); + + ret =write (fd,"x" ,1 ); +ASSERT_EQ (ret,-1 ,"write" ); +ASSERT_EQ (errno,EBADF ,"write_errno" ); + ret =lseek (fd,0 ,SEEK_SET ); +ASSERT_EQ (ret,-1 ,"lseek" ); +ASSERT_EQ (errno,ESPIPE ,"lseek_errno" ); + + ret =poll (&pfd,1 ,0 ); +ASSERT_EQ (ret,0 ,"poll_empty" ); + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,-1 ,"read_empty" ); +ASSERT_EQ (errno,EAGAIN ,"read_empty_errno" ); + +if (!stream_fd_trigger (skel->progs.stream_syscall)) + goto out_close; + ret =poll (&pfd,1 ,0 ); +ASSERT_EQ (ret,1 ,"poll_data" ); +ASSERT_NEQ (pfd.revents &POLLIN ,0 ,"pollin" ); +ASSERT_EQ (pfd.revents &POLLHUP ,0 ,"no_pollhup" ); + + ret =read (fd, buf,2 ); +ASSERT_EQ (ret,2 ,"read_first" ); +ASSERT_OK (memcmp (buf,"fo" ,2 ),"read_first_data" ); + pfd.revents =0 ; + ret =poll (&pfd,1 ,0 ); +ASSERT_EQ (ret,1 ,"poll_partial" ); +ASSERT_NEQ (pfd.revents &POLLIN ,0 ,"pollin_partial" ); + + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,1 ,"read_rest" ); +ASSERT_EQ (buf[0 ],'o' ,"read_rest_data" ); + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,-1 ,"read_drained" ); +ASSERT_EQ (errno,EAGAIN ,"read_drained_errno" ); + +out_close: +close (fd); +out_destroy: +stream__destroy (skel); +} + +static void test_stream_fd_empty (void ) +{ +LIBBPF_OPTS (bpf_prog_stream_open_opts, opts, + .flags = BPF_F_STREAM_NONBLOCK, + ); +struct pollfd pfd = { .events =POLLIN |POLLHUP }; +struct stream *skel; +char buf[4 ]; +int fd, ret; + + skel =stream__open_and_load (); +if (!ASSERT_OK_PTR (skel,"stream__open_and_load" )) +return ; + + fd =bpf_prog_stream_open (bpf_program__fd (skel->progs.stream_empty), +BPF_STREAM_STDOUT , &opts); +if (!ASSERT_OK_FD (fd,"stream_open" )) + goto out_destroy; + pfd.fd = fd; + +/* Empty output produces neither data nor readiness. */ +if (!stream_fd_trigger (skel->progs.stream_empty)) + goto out_close; + ret =poll (&pfd,1 ,0 ); +ASSERT_EQ (ret,0 ,"poll_empty_write" ); + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,-1 ,"read_empty_write" ); +ASSERT_EQ (errno,EAGAIN ,"read_empty_write_errno" ); + +out_close: +close (fd); +out_destroy: +stream__destroy (skel); +} + +struct stream_fd_read_ctx { +int fd; +ssize_t ret; +char buf[4 ]; +}; + +static void *stream_fd_read_thread (void *arg) +{ +struct stream_fd_read_ctx *ctx = arg; + + ctx->ret =read (ctx->fd, ctx->buf,sizeof (ctx->buf)); +return NULL ; +} + +static int stream_fd_timed_join (pthread_t thread) +{ +struct timespec timeout; + +clock_gettime (CLOCK_REALTIME , &timeout); + timeout.tv_sec +=5 ; +return pthread_timedjoin_np (thread,NULL , &timeout); +} + +static void test_stream_fd_blocking (void ) +{ +struct stream_fd_read_ctx ctx = {}; +struct stream *skel; +pthread_t thread; +bool thread_live = false; +int fd, flags, ret; + + skel =stream__open_and_load (); +if (!ASSERT_OK_PTR (skel,"stream__open_and_load" )) +return ; + + fd =bpf_prog_stream_open (bpf_program__fd (skel->progs.stream_syscall), +BPF_STREAM_STDOUT ,NULL ); +if (!ASSERT_OK_FD (fd,"stream_open" )) + goto out_destroy; + ctx.fd = fd; + + flags =fcntl (fd,F_GETFL ); +ASSERT_GE (flags,0 ,"getfl" ); +ASSERT_EQ (flags &O_NONBLOCK ,0 ,"blocking" ); + + ret =pthread_create (&thread,NULL , stream_fd_read_thread, &ctx); +if (!ASSERT_OK (ret,"pthread_create" )) + goto out_close; + thread_live = true; + +usleep (50000 ); + ret =pthread_tryjoin_np (thread,NULL ); +if (!ASSERT_EQ (ret,EBUSY ,"read_blocks" )) { + thread_live = ret !=0 ; + goto out_thread; + } +if (!stream_fd_trigger (skel->progs.stream_syscall)) + goto out_thread; + + ret =stream_fd_timed_join (thread); +if (!ASSERT_OK (ret,"pthread_join" )) + goto out_thread; + thread_live = false; +ASSERT_EQ (ctx.ret,3 ,"read_len" ); +ASSERT_OK (memcmp (ctx.buf,"foo" ,3 ),"read_data" ); + +memset (&ctx,0 ,sizeof (ctx)); + ctx.fd = fd; + ret =pthread_create (&thread,NULL , stream_fd_read_thread, &ctx); +if (!ASSERT_OK (ret,"pthread_create_eof" )) + goto out_close; + thread_live = true; + +usleep (50000 ); + ret =pthread_tryjoin_np (thread,NULL ); +if (!ASSERT_EQ (ret,EBUSY ,"read_eof_blocks" )) { + thread_live = ret !=0 ; + goto out_thread; + } + +stream__destroy (skel); + skel =NULL ; + ret =stream_fd_timed_join (thread); +if (!ASSERT_OK (ret,"pthread_join_eof" )) + goto out_thread; + thread_live = false; +ASSERT_EQ (ctx.ret,0 ,"read_eof" ); + +out_thread: +if (thread_live) { +stream__destroy (skel); + skel =NULL ; + ret =stream_fd_timed_join (thread); +if (ret) { +pthread_cancel (thread); +pthread_join (thread,NULL ); + } + } +out_close: +close (fd); +out_destroy: +stream__destroy (skel); +} + +static void test_stream_fd_hup (void ) +{ +LIBBPF_OPTS (bpf_prog_stream_open_opts, opts, + .flags = BPF_F_STREAM_NONBLOCK, + ); +struct epoll_event event = { + .events =EPOLLIN |EPOLLET , + }; +struct pollfd pfd = { .events =POLLIN |POLLHUP }; +struct stream *skel; +char buf[4 ] = {}; +int epfd =-1 , fd, ret; + + skel =stream__open_and_load (); +if (!ASSERT_OK_PTR (skel,"stream__open_and_load" )) +return ; + + fd =bpf_prog_stream_open (bpf_program__fd (skel->progs.stream_syscall), +BPF_STREAM_STDOUT , &opts); +if (!ASSERT_OK_FD (fd,"stream_open" )) + goto out_destroy; + pfd.fd = fd; + epfd =epoll_create1 (EPOLL_CLOEXEC ); +if (!ASSERT_OK_FD (epfd,"epoll_create" )) + goto out_close; + event.data.fd = fd; + ret =epoll_ctl (epfd,EPOLL_CTL_ADD , fd, &event); +if (!ASSERT_OK (ret,"epoll_ctl" )) + goto out_close; + +if (!stream_fd_trigger (skel->progs.stream_syscall)) + goto out_close; + ret =epoll_wait (epfd, &event,1 ,5000 ); +if (!ASSERT_EQ (ret,1 ,"epoll_wait_data" )) + goto out_close; +ASSERT_NEQ (event.events &EPOLLIN ,0 ,"epollin" ); +ASSERT_EQ (event.events &EPOLLHUP ,0 ,"no_epollhup" ); + +stream__destroy (skel); + skel =NULL ; + event.events =0 ; + ret =epoll_wait (epfd, &event,1 ,5000 ); +if (!ASSERT_EQ (ret,1 ,"epoll_wait_hup" )) + goto out_close; +ASSERT_NEQ (event.events &EPOLLIN ,0 ,"epollin_with_hup" ); +ASSERT_NEQ (event.events &EPOLLHUP ,0 ,"epollhup" ); + + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,3 ,"read_buffered" ); +ASSERT_OK (memcmp (buf,"foo" ,3 ),"read_buffered_data" ); + pfd.revents =0 ; + ret =poll (&pfd,1 ,0 ); +ASSERT_EQ (ret,1 ,"poll_drained_hup" ); +ASSERT_EQ (pfd.revents &POLLIN ,0 ,"no_pollin_after_drain" ); +ASSERT_NEQ (pfd.revents &POLLHUP ,0 ,"pollhup_after_drain" ); + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,0 ,"read_eof" ); + +out_close: +if (epfd >=0 ) +close (epfd); +close (fd); +out_destroy: +stream__destroy (skel); +} + +static void test_stream_fd_nmi_epoll (void ) +{ +LIBBPF_OPTS (bpf_prog_stream_open_opts, opts, + .flags = BPF_F_STREAM_NONBLOCK, + ); +struct perf_event_attr attr = { + .size =sizeof (attr), + .type =PERF_TYPE_HARDWARE , + .config =PERF_COUNT_HW_CPU_CYCLES , + .freq =1 , + .sample_freq =10 , + }; +struct epoll_event event = { + .events =EPOLLIN , + }; +struct bpf_link *link =NULL ; +struct stream *skel; +__u64 deadline; +char buf[4 ] = {}; +int epfd =-1 , fd =-1 , pmu_fd =-1 ; +int ret; + + skel =stream__open_and_load (); +if (!ASSERT_OK_PTR (skel,"stream__open_and_load" )) +return ; + fd =bpf_prog_stream_open (bpf_program__fd (skel->progs.stream_nmi), +BPF_STREAM_STDOUT , &opts); +if (!ASSERT_OK_FD (fd,"stream_open" )) + goto out; + + epfd =epoll_create1 (EPOLL_CLOEXEC ); +if (!ASSERT_OK_FD (epfd,"epoll_create" )) + goto out; + event.data.fd = fd; + ret =epoll_ctl (epfd,EPOLL_CTL_ADD , fd, &event); +if (!ASSERT_OK (ret,"epoll_ctl" )) + goto out; + + pmu_fd =syscall (__NR_perf_event_open, &attr,0 ,-1 ,-1 , +PERF_FLAG_FD_CLOEXEC ); +if (pmu_fd <0 && (errno ==ENOENT || errno ==EOPNOTSUPP )) { +printf ("%s:SKIP:no PERF_COUNT_HW_CPU_CYCLES\n" , __func__); +test__skip (); + goto out; + } +if (!ASSERT_GE (pmu_fd,0 ,"perf_event_open" )) + goto out; + + link =bpf_program__attach_perf_event (skel->progs.stream_nmi, pmu_fd); +if (!ASSERT_OK_PTR (link,"attach_perf_event" )) { + link =NULL ; + goto out; + } + pmu_fd =-1 ; + + deadline =get_time_ns () +NMI_TIMEOUT_NS ; +do { + ret =epoll_wait (epfd, &event,1 ,0 ); + }while (ret ==0 &&get_time_ns () < deadline); +if (!ASSERT_EQ (ret,1 ,"epoll_wait" )) + goto out; +ASSERT_NEQ (event.events &EPOLLIN ,0 ,"epollin" ); + + ret =read (fd, buf,sizeof (buf)); +ASSERT_EQ (ret,3 ,"read_len" ); +ASSERT_OK (memcmp (buf,"nmi" ,3 ),"read_data" ); + +out: +bpf_link__destroy (link); +if (pmu_fd >=0 ) +close (pmu_fd); +if (epfd >=0 ) +close (epfd); +if (fd >=0 ) +close (fd); +stream__destroy (skel); +} + +void test_stream_fd (void ) +{ +if (test__start_subtest ("open" )) +test_stream_fd_open (); +if (test__start_subtest ("nonblock" )) +test_stream_fd_nonblock (); +if (test__start_subtest ("empty" )) +test_stream_fd_empty (); +if (test__start_subtest ("blocking" )) +test_stream_fd_blocking (); +if (test__start_subtest ("hup" )) +test_stream_fd_hup (); +if (test__start_subtest ("nmi_epoll" )) +test_stream_fd_nmi_epoll (); +} +void test_stream_oversize (void ) {LIBBPF_OPTS (bpf_test_run_opts, opts);
tools/testing/selftests/bpf/progs/stream.c +20 -0
@@ -44,6 +44,7 @@ struct { _X64 _X64 _X64 _X64 _X64 _X64 _X64 _X64)int size; +int nmi_stream_prints;u64 fault_addr;void *arena_ptr;
@@ -124,6 +125,25 @@ int stream_syscall(void *ctx)return 0 ; } +SEC("syscall") +__success __retval(0) +int stream_empty(void *ctx) +{ +return bpf_stream_printk (BPF_STDOUT ,"" ); +} + +SEC("perf_event") +int stream_nmi (void *ctx) +{ +if (nmi_stream_prints) +return 0 ; +/* Retry on a later sample if the write failed, e.g. with -EBUSY. */ +if (bpf_stream_printk (BPF_STDOUT ,"nmi" )) +return 0 ; + nmi_stream_prints =1 ; +return 0 ; +} + SEC("syscall") __success __retval(0) int stream_oversize(void *ctx)
Signature · 1 line
2.53.0