BPF controlled io_uring

Add ring_fd to the struct_ops and implement [un]registration.

Signed-off-by Pavel Begunkov <[email protected]>

---
 include/linux/io_uring_types.h |  2 +
 io_uring/bpf.c                 | 69 +++++++++++++++++++++++++++++++++-
 2 files changed, 70 insertions(+), 1 deletion(-)

include/linux/io_uring_types.h +2 -0

index 43432a06d177..3a71ed2d05ea 100644
@@ -274,6 +274,8 @@ struct io_ring_ctx {
 		unsigned int		compat: 1;
 		unsigned int		iowq_limits_set : 1;
 
+		unsigned int		bpf_installed: 1;
+
 		struct task_struct	*submitter_task;
 		struct io_rings		*rings;
 		struct percpu_ref	refs;

io_uring/bpf.c +24 -1

index 24dd2fe9134f..683e87f1a58b 100644
@@ -4,6 +4,7 @@
 #include "bpf.h"
 #include "register.h"
 
+static DEFINE_MUTEX(io_bpf_ctrl_mutex);
 static const struct btf_type *loop_state_type;
 
 static int io_bpf_ops__loop(struct io_ring_ctx *ctx, struct iou_loop_state *ls)

@@ -87,20 +88,86 @@ static int bpf_io_init_member(const struct btf_type *t,
 			       const struct btf_member *member,
 			       void *kdata, const void *udata)
 {
+	u32 moff = __btf_member_bit_offset(t, member) / 8;
+	const struct io_uring_ops *uops = udata;
+	struct io_uring_ops *ops = kdata;
+
+	switch (moff) {
+	case offsetof(struct io_uring_ops, ring_fd):
+		ops->ring_fd = uops->ring_fd;
+		return 1;
+	}
+	return 0;
+}
+
+static int io_install_bpf(struct io_ring_ctx *ctx, struct io_uring_ops *ops)
+{
+	if (ctx->bpf_ops)
+		return -EBUSY;
+	ops->priv = ctx;
+	ctx->bpf_ops = ops;
+	ctx->bpf_installed = 1;
 	return 0;
 }
 
 static int bpf_io_reg(void *kdata, struct bpf_link *link)
 {
-	return -EOPNOTSUPP;
+	struct io_uring_ops *ops = kdata;
+	struct io_ring_ctx *ctx;
+	struct file *file;
+	int ret = -EBUSY;

Quoted 1 line · Pavel Begunkov

Add ring_fd to the struct_ops and implement [un]registration.

Quoted 23 lines

io_uring/bpf.c +17 -1

+static int io_install_bpf(struct io_ring_ctx *ctx, struct io_uring_ops *ops)
+{
+	if (ctx->bpf_ops)
+		return -EBUSY;
+	ops->priv = ctx;
+	ctx->bpf_ops = ops;
+	ctx->bpf_installed = 1;
  	return 0;
  }
 
  static int bpf_io_reg(void *kdata, struct bpf_link *link)
  {
-	return -EOPNOTSUPP;
+	struct io_uring_ops *ops = kdata;
+	struct io_ring_ctx *ctx;
+	struct file *file;
+	int ret = -EBUSY;
+
+	file = io_uring_register_get_file(ops->ring_fd, false);
+	if (IS_ERR(file))
+		return PTR_ERR(file);
+	ctx = file->private_data;
+

io_uring/bpf.c static int bpf_io_init_member(const struct btf_type *t, · line 128

Quoted 4 lines

io_uring/bpf.c +4 -0

+	scoped_guard(mutex, &io_bpf_ctrl_mutex) {
+		guard(mutex)(&ctx->uring_lock);
+		ret = io_install_bpf(ctx, ops);
+	}

I feel per-io-uring struct_ops is less useful, because it means the io_uring application has to be capable of loading/registering struct_ops prog, which often needs privilege.

For example of IO link use case you mentioned, why does the application need to get privilege for running IO link?

Quoted 2 lines · Ming Lei

Quoted 1 line · Pavel Begunkov

Add ring_fd to the struct_ops and implement [un]registration.

...

Quoted 22 lines

Quoted 22 lines

io_uring/bpf.c +17 -1

+static int io_install_bpf(struct io_ring_ctx *ctx, struct io_uring_ops *ops)
+{
+	if (ctx->bpf_ops)
+		return -EBUSY;
+	ops->priv = ctx;
+	ctx->bpf_ops = ops;
+	ctx->bpf_installed = 1;
  	return 0;
  }
  static int bpf_io_reg(void *kdata, struct bpf_link *link)
  {
-	return -EOPNOTSUPP;
+	struct io_uring_ops *ops = kdata;
+	struct io_ring_ctx *ctx;
+	struct file *file;
+	int ret = -EBUSY;
+
+	file = io_uring_register_get_file(ops->ring_fd, false);
+	if (IS_ERR(file))
+		return PTR_ERR(file);
+	ctx = file->private_data;
+

Quoted 8 lines

io_uring/bpf.c static int bpf_io_init_member(const struct btf_type *t, · line 128

Quoted 4 lines

io_uring/bpf.c +4 -0

+	scoped_guard(mutex, &io_bpf_ctrl_mutex) {
+		guard(mutex)(&ctx->uring_lock);
+		ret = io_install_bpf(ctx, ops);
+	}

I feel per-io-uring struct_ops is less useful, because it means the io_uring application has to be capable of loading/registering struct_ops prog, which often needs privilege.

I gave it a thought before, there would need to be a way to pass a program from one (e.g. privileged) task to another, e.g. by putting it into a list on attachment from where it can be imported. That can be extended, and I needed to start somewhere.

Quoted 3 lines

For example of IO link use case you mentioned, why does the application need to get privilege for running IO link?

Links are there to compare with existing features. It's more interesting to allow arbitrary relations / result propagation between requests. Maybe some common patterns can be generalised, but otherwise nothing can be done with this without custom tailored bpf programs.

Quoted 2 lines · Pavel Begunkov

Quoted 1 line · Ming Lei

On Thu, Nov 13, 2025 at 11:59:44AM +0000, Pavel Begunkov wrote:

Quoted 8 lines

Quoted 3 lines

I feel per-io-uring struct_ops is less useful, because it means the io_uring application has to be capable of loading/registering struct_ops prog, which often needs privilege.

I gave it a thought before, there would need to be a way to pass a program from one (e.g. privileged) task to another, e.g. by putting it into a list on attachment from where it can be imported. That can be extended, and I needed to start somewhere.

If any task can ask such privileged task to load bpf program for itself, BPF_UNPRIV_DEFAULT_OFF becomes N actually for bpf controlled io_uring.

Quoted 7 lines

Quoted 2 lines

For example of IO link use case you mentioned, why does the application need to get privilege for running IO link?

Links are there to compare with existing features. It's more interesting to allow arbitrary relations / result propagation between requests. Maybe some common patterns can be generalised, but otherwise nothing can be done with this without custom tailored bpf programs.

I know the motivation, which is one thing covered in my IORING_OP_BPF patch too.

Add a io_uring bpf selftest/example. runner.c sets up a ring and BPF and calls io_uring_enter syscall to run the BPF program. All the execution logic is in basic.bpf.c, which creates a request, waits for its completion and repeats it N=10 times, after which it terminates. The makefile is borrowed from sched_ext.

Note, it doesn't need to be all in BPF and can be intermingled with userspace code. This needs a separate example.

Signed-off-by Pavel Begunkov <[email protected]>

---
 tools/testing/selftests/Makefile             |   3 +-
 tools/testing/selftests/io_uring/Makefile    | 164 +++++++++++++++++++
 tools/testing/selftests/io_uring/basic.bpf.c |  81 +++++++++
 tools/testing/selftests/io_uring/common.h    |   2 +
 tools/testing/selftests/io_uring/runner.c    |  80 +++++++++
 tools/testing/selftests/io_uring/types.bpf.h | 136 +++++++++++++++
 6 files changed, 465 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/io_uring/Makefile
 create mode 100644 tools/testing/selftests/io_uring/basic.bpf.c
 create mode 100644 tools/testing/selftests/io_uring/common.h
 create mode 100644 tools/testing/selftests/io_uring/runner.c
 create mode 100644 tools/testing/selftests/io_uring/types.bpf.h

tools/testing/selftests/Makefile +2 -1

index c46ebdb9b8ef..31dd369a7154 100644
@@ -129,6 +129,7 @@ TARGETS += vfio
 TARGETS += x86
 TARGETS += x86/bugs
 TARGETS += zram
+TARGETS += io_uring
 #Please keep the TARGETS list alphabetically sorted
 # Run "make quicktest=1 run_tests" or
 # "make quicktest=1 kselftest" from top level Makefile

@@ -146,7 +147,7 @@ endif
 # User can optionally provide a TARGETS skiplist. By default we skip
 # targets using BPF since it has cutting edge build time dependencies
 # which require more effort to install.
-SKIP_TARGETS ?= bpf sched_ext
+SKIP_TARGETS ?= bpf sched_ext io_uring
 ifneq ($(SKIP_TARGETS),)
 	TMP := $(filter-out $(SKIP_TARGETS), $(TARGETS))
 	override TARGETS := $(TMP)

tools/testing/selftests/io_uring/Makefile new file +164 -0

new file mode 100644
index 000000000000..7dfba422e5a6
@@ -0,0 +1,164 @@
+# SPDX-License-Identifier: GPL-2.0
+include ../../../build/Build.include
+include ../../../scripts/Makefile.arch
+include ../../../scripts/Makefile.include
+
+TEST_GEN_PROGS := runner
+
+# override lib.mk's default rules
+OVERRIDE_TARGETS := 1
+include ../lib.mk
+
+CURDIR := $(abspath .)
+REPOROOT := $(abspath ../../../..)
+TOOLSDIR := $(REPOROOT)/tools
+LIBDIR := $(TOOLSDIR)/lib
+BPFDIR := $(LIBDIR)/bpf
+TOOLSINCDIR := $(TOOLSDIR)/include
+BPFTOOLDIR := $(TOOLSDIR)/bpf/bpftool
+APIDIR := $(TOOLSINCDIR)/uapi
+GENDIR := $(REPOROOT)/include/generated
+GENHDR := $(GENDIR)/autoconf.h
+
+OUTPUT_DIR := $(OUTPUT)/build
+OBJ_DIR := $(OUTPUT_DIR)/obj
+INCLUDE_DIR := $(OUTPUT_DIR)/include
+BPFOBJ_DIR := $(OBJ_DIR)/libbpf
+IOUOBJ_DIR := $(OBJ_DIR)/io_uring
+LIBBPF_OUTPUT := $(OBJ_DIR)/libbpf/libbpf.a
+BPFOBJ := $(BPFOBJ_DIR)/libbpf.a
+
+DEFAULT_BPFTOOL := $(OUTPUT_DIR)/host/sbin/bpftool
+HOST_OBJ_DIR := $(OBJ_DIR)/host/bpftool
+HOST_LIBBPF_OUTPUT := $(OBJ_DIR)/host/libbpf/
+HOST_LIBBPF_DESTDIR := $(OUTPUT_DIR)/host/
+HOST_DESTDIR := $(OUTPUT_DIR)/host/
+
+VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux)					\
+		     $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux)		\
+		     ../../../../vmlinux					\
+		     /sys/kernel/btf/vmlinux					\
+		     /boot/vmlinux-$(shell uname -r)
+VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))
+ifeq ($(VMLINUX_BTF),)
+$(error Cannot find a vmlinux for VMLINUX_BTF at any of "$(VMLINUX_BTF_PATHS)")
+endif
+
+BPFTOOL ?= $(DEFAULT_BPFTOOL)
+
+ifneq ($(wildcard $(GENHDR)),)
+  GENFLAGS := -DHAVE_GENHDR
+endif
+
+CFLAGS += -g -O2 -rdynamic -pthread -Wall -Werror $(GENFLAGS)			\
+	  -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR)				\
+	  -I$(TOOLSINCDIR) -I$(APIDIR) -I$(CURDIR)/include
+
+# Silence some warnings when compiled with clang
+ifneq ($(LLVM),)
+CFLAGS += -Wno-unused-command-line-argument
+endif
+
+LDFLAGS = -lelf -lz -lpthread -lzstd
+
+IS_LITTLE_ENDIAN = $(shell $(CC) -dM -E - </dev/null |				\
+			grep 'define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__')
+
+# Get Clang's default includes on this system, as opposed to those seen by
+# '-target bpf'. This fixes "missing" files on some architectures/distros,
+# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc.
+#
+# Use '-idirafter': Don't interfere with include mechanics except where the
+# build would have failed anyways.
+define get_sys_includes
+$(shell $(1) $(2) -v -E - </dev/null 2>&1 \
+	| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
+$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}')
+endef
+
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+
+CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
+
+BPF_CFLAGS = -g -D__TARGET_ARCH_$(SRCARCH)					\
+	     $(if $(IS_LITTLE_ENDIAN),-mlittle-endian,-mbig-endian)		\
+	     -I$(CURDIR)/include -I$(CURDIR)/include/bpf-compat			\
+	     -I$(INCLUDE_DIR) -I$(APIDIR) 	\
+	     -I$(REPOROOT)/include						\
+	     $(CLANG_SYS_INCLUDES) 						\
+	     -Wall -Wno-compare-distinct-pointer-types				\
+	     -Wno-incompatible-function-pointer-types				\
+	     -O2 -mcpu=v3
+
+# sort removes libbpf duplicates when not cross-building
+MAKE_DIRS := $(sort $(OBJ_DIR)/libbpf $(OBJ_DIR)/libbpf				\
+	       $(OBJ_DIR)/bpftool $(OBJ_DIR)/resolve_btfids			\
+	       $(HOST_OBJ_DIR) $(INCLUDE_DIR) $(IOUOBJ_DIR))
+
+$(MAKE_DIRS):
+	$(call msg,MKDIR,,$@)
+	$(Q)mkdir -p $@
+
+$(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile)			\
+	   $(APIDIR)/linux/bpf.h						\
+	   | $(OBJ_DIR)/libbpf
+	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(OBJ_DIR)/libbpf/	\
+		    ARCH=$(ARCH) CC="$(CC)" CROSS_COMPILE=$(CROSS_COMPILE)	\
+		    EXTRA_CFLAGS='-g -O0 -fPIC'					\
+		    DESTDIR=$(OUTPUT_DIR) prefix= all install_headers
+
+$(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
+		    $(LIBBPF_OUTPUT) | $(HOST_OBJ_DIR)
+	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
+		    ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)		\
+		    EXTRA_CFLAGS='-g -O0'					\
+		    OUTPUT=$(HOST_OBJ_DIR)/					\
+		    LIBBPF_OUTPUT=$(HOST_LIBBPF_OUTPUT)				\
+		    LIBBPF_DESTDIR=$(HOST_LIBBPF_DESTDIR)			\
+		    prefix= DESTDIR=$(HOST_DESTDIR) install-bin
+
+$(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) | $(INCLUDE_DIR)
+ifeq ($(VMLINUX_H),)
+	$(call msg,GEN,,$@)
+	$(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF) format c > $@
+else
+	$(call msg,CP,,$@)
+	$(Q)cp "$(VMLINUX_H)" $@
+endif
+
+$(IOUOBJ_DIR)/%.bpf.o: %.bpf.c $(INCLUDE_DIR)/vmlinux.h	| $(BPFOBJ) $(IOUOBJ_DIR)
+	$(call msg,CLNG-BPF,,$(notdir $@))
+	$(Q)$(CLANG) $(BPF_CFLAGS) -target bpf -c $< -o $@
+
+$(INCLUDE_DIR)/%.bpf.skel.h: $(IOUOBJ_DIR)/%.bpf.o $(INCLUDE_DIR)/vmlinux.h $(BPFTOOL) | $(INCLUDE_DIR)
+	$(eval sched=$(notdir $@))
+	$(call msg,GEN-SKEL,,$(sched))
+	$(Q)$(BPFTOOL) gen object $(<:.o=.linked1.o) $<
+	$(Q)$(BPFTOOL) gen object $(<:.o=.linked2.o) $(<:.o=.linked1.o)
+	$(Q)$(BPFTOOL) gen object $(<:.o=.linked3.o) $(<:.o=.linked2.o)
+	$(Q)diff $(<:.o=.linked2.o) $(<:.o=.linked3.o)
+	$(Q)$(BPFTOOL) gen skeleton $(<:.o=.linked3.o) name $(subst .bpf.skel.h,,$(sched)) > $@
+	$(Q)$(BPFTOOL) gen subskeleton $(<:.o=.linked3.o) name $(subst .bpf.skel.h,,$(sched)) > $(@:.skel.h=.subskel.h)
+
+override define CLEAN
+	rm -rf $(OUTPUT_DIR)
+	rm -f $(TEST_GEN_PROGS)
+endef
+
+all_test_bpfprogs := $(foreach prog,$(wildcard *.bpf.c),$(INCLUDE_DIR)/$(patsubst %.c,%.skel.h,$(prog)))
+
+$(IOUOBJ_DIR)/runner.o: runner.c $(all_test_bpfprogs) | $(IOUOBJ_DIR) $(BPFOBJ)
+	$(CC) $(CFLAGS) -c $< -o $@
+
+$(OUTPUT)/runner: $(IOUOBJ_DIR)/runner.o $(BPFOBJ)
+	@echo "$(testcase-targets)"
+	echo 111
+	$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
+
+.DEFAULT_GOAL := all
+
+.DELETE_ON_ERROR:
+
+.SECONDARY:

tools/testing/selftests/io_uring/basic.bpf.c new file +56 -0

new file mode 100644
index 000000000000..c7954146ae4d
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/types.h>
+#include <linux/stddef.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "types.bpf.h"
+#include "common.h"
+
+extern int bpf_io_uring_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr) __ksym;
+extern __u8 *bpf_io_uring_get_region(struct io_ring_ctx *ctx, __u32 region_id,
+				     const __u64 rdwr_buf_size) __ksym;
+
+static inline void io_bpf_wait_nr(struct io_ring_ctx *ring,
+				  struct iou_loop_state *ls, int nr)
+{
+	ls->cq_tail = ring->rings->cq.head + nr;
+}
+
+enum {
+	RINGS_REGION_ID		= 0,
+	SQ_REGION_ID		= 1,
+};
+
+char LICENSE[] SEC("license") = "Dual BSD/GPL";
+int reqs_to_run;
+
+SEC("struct_ops.s/link_loop")
+int BPF_PROG(link_loop, struct io_ring_ctx *ring, struct iou_loop_state *ls)
+{
+	struct ring_hdr *sq_hdr, *cq_hdr;
+	struct io_uring_cqe *cqe, *cqes;
+	struct io_uring_sqe *sqes, *sqe;
+	void *rings;
+	int ret;
+
+	sqes = (void *)bpf_io_uring_get_region(ring, SQ_REGION_ID,
+				SQ_ENTRIES * sizeof(struct io_uring_sqe));
+	rings = (void *)bpf_io_uring_get_region(ring, RINGS_REGION_ID,
+				64 + CQ_ENTRIES * sizeof(struct io_uring_cqe));
+	if (!rings || !sqes) {
+		bpf_printk("error: can't get regions");
+		return IOU_LOOP_STOP;
+	}
+
+	sq_hdr = rings;
+	cq_hdr = sq_hdr + 1;
+	cqes = rings + 64;
+
+	if (cq_hdr->tail != cq_hdr->head) {
+		unsigned cq_mask = CQ_ENTRIES - 1;
+
+		cqe = &cqes[cq_hdr->head++ & cq_mask];
+		bpf_printk("found cqe: data %lu res %i",
+			   (unsigned long)cqe->user_data, (int)cqe->res);
+
+		int left = --reqs_to_run;

On Thu, Nov 13, 2025 at 11:59:47AM +0000, Pavel Begunkov wrote:

tools/testing/selftests/io_uring/basic.bpf.c line 66

Quoted 4 lines

tools/testing/selftests/io_uring/basic.bpf.c new file +4 -0

+	bpf_printk("queue nop request, data %lu\n", (unsigned long)reqs_to_run);
+	sqe = &sqes[sq_hdr->tail & (SQ_ENTRIES - 1)];
+	sqe->user_data = reqs_to_run;
+	sq_hdr->tail++;

Looks this way turns io_uring_enter() into pthread-unsafe, does it need to be documented?

Quoted 1 line · Ming Lei

On Thu, Nov 13, 2025 at 11:59:47AM +0000, Pavel Begunkov wrote:

...

Quoted 7 lines

tools/testing/selftests/io_uring/basic.bpf.c line 66

Quoted 4 lines

tools/testing/selftests/io_uring/basic.bpf.c new file +4 -0

+	bpf_printk("queue nop request, data %lu\n", (unsigned long)reqs_to_run);
+	sqe = &sqes[sq_hdr->tail & (SQ_ENTRIES - 1)];
+	sqe->user_data = reqs_to_run;
+	sq_hdr->tail++;

Looks this way turns io_uring_enter() into pthread-unsafe, does it need to be documented?

Assuming you mean parallel io_uring_enter() calls modifying the SQ, it's not different from how it currently is. If you're sharing an io_uring, threads need to sync the use of SQ/CQ.

Quoted 11 lines · Pavel Begunkov

Quoted 1 line · Ming Lei

On Thu, Nov 13, 2025 at 11:59:47AM +0000, Pavel Begunkov wrote:

...

Quoted 7 lines

tools/testing/selftests/io_uring/basic.bpf.c line 66

Quoted 4 lines

tools/testing/selftests/io_uring/basic.bpf.c new file +4 -0

+	bpf_printk("queue nop request, data %lu\n", (unsigned long)reqs_to_run);
+	sqe = &sqes[sq_hdr->tail & (SQ_ENTRIES - 1)];
+	sqe->user_data = reqs_to_run;
+	sq_hdr->tail++;

Looks this way turns io_uring_enter() into pthread-unsafe, does it need to be documented?

Quoted 3 lines

Assuming you mean parallel io_uring_enter() calls modifying the SQ, it's not different from how it currently is. If you're sharing an io_uring, threads need to sync the use of SQ/CQ.

Please see the example:

thread_fn(struct io_uring *ring)

{
	while (true) {
		pthread_mutex_lock(sqe_mutex);
		sqe = io_uring_get_sqe(ring);
		io_uring_prep_op(sqe);
		pthread_mutex_unlock(sqe_mutex);

		io_uring_enter(ring);

		pthread_mutex_lock(cqe_mutex);
		io_uring_wait_cqe(ring, &cqe);
		io_uring_cqe_seen(ring, cqe);
		pthread_mutex_unlock(cqe_mutex);
	}
}

thread_fn is supposed to work concurrently from >1 pthreads:

1) io_uring_enter() is claimed as pthread safe

2) because of userspace lock protection, there is single code path for producing sqe for SQ at same time, and single code path for consuming sqe from io_uring_enter().

With bpf controlled io_uring patches, sqe can be produced from io_uring_enter(), and cqe can be consumed in io_uring_enter() too, there will be race between bpf prog(producing sqe, or consuming cqe) and userspace lock-protected code block.

Quoted 39 lines · Ming Lei

Quoted 8 lines · Pavel Begunkov

Quoted 1 line · Ming Lei

On Thu, Nov 13, 2025 at 11:59:47AM +0000, Pavel Begunkov wrote:

...

Quoted 5 lines

Quoted 4 lines

tools/testing/selftests/io_uring/basic.bpf.c new file +4 -0

+  bpf_printk("queue nop request, data %lu\n", (unsigned long)reqs_to_run);
+  sqe = &sqes[sq_hdr->tail & (SQ_ENTRIES - 1)];
+  sqe->user_data = reqs_to_run;
+  sq_hdr->tail++;

Quoted 6 lines

Quoted 2 lines

Looks this way turns io_uring_enter() into pthread-unsafe, does it need to be documented?

Assuming you mean parallel io_uring_enter() calls modifying the SQ, it's not different from how it currently is. If you're sharing an io_uring, threads need to sync the use of SQ/CQ.

Please see the example:

thread_fn(struct io_uring *ring)

{
      while (true) {
              pthread_mutex_lock(sqe_mutex);
              sqe = io_uring_get_sqe(ring);
              io_uring_prep_op(sqe);
              pthread_mutex_unlock(sqe_mutex);

              io_uring_enter(ring);

              pthread_mutex_lock(cqe_mutex);
              io_uring_wait_cqe(ring, &cqe);
              io_uring_cqe_seen(ring, cqe);
              pthread_mutex_unlock(cqe_mutex);
      }
}

thread_fn is supposed to work concurrently from >1 pthreads:

1) io_uring_enter() is claimed as pthread safe

Quoted 8 lines

2) because of userspace lock protection, there is single code path for producing sqe for SQ at same time, and single code path for consuming sqe from io_uring_enter().

With bpf controlled io_uring patches, sqe can be produced from io_uring_enter(), and cqe can be consumed in io_uring_enter() too, there will be race between bpf prog(producing sqe, or consuming cqe) and userspace lock-protected code block.

BPF is attached by the same process/user that creates io_uring. The guarantees are same as before, the user code (which includes BPF) should protect from concurrent mutations.

In this example, just extend the first critical section to io_uring_enter(). Concurrent io_uring_enter() will be serialised by a mutex anyway. But let me note, that sharing rings is not a great pattern in either case.

On Sat, Nov 22, 2025 at 12:12 AM Pavel Begunkov <[email protected]> wrote:

Quoted 18 lines

Quoted 5 lines

thread_fn is supposed to work concurrently from >1 pthreads:

1) io_uring_enter() is claimed as pthread safe

Quoted 8 lines

2) because of userspace lock protection, there is single code path for producing sqe for SQ at same time, and single code path for consuming sqe from io_uring_enter().

With bpf controlled io_uring patches, sqe can be produced from io_uring_enter(), and cqe can be consumed in io_uring_enter() too, there will be race between bpf prog(producing sqe, or consuming cqe) and userspace lock-protected code block.

BPF is attached by the same process/user that creates io_uring. The guarantees are same as before, the user code (which includes BPF) should protect from concurrent mutations.

Quoted 4 lines

In this example, just extend the first critical section to io_uring_enter(). Concurrent io_uring_enter() will be serialised by a mutex anyway. But let me note, that sharing rings is not a great pattern in either case.

If io_uring_enter() needs to be serialised, it becomes pthread-unsafe,

that is why I mentioned this should be documented, because it is one very big difference introduced in bpf controlled ring.

Quoted 1 line · Ming Lei

On Sat, Nov 22, 2025 at 12:12 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 23 lines

Quoted 22 lines

Quoted 5 lines

thread_fn is supposed to work concurrently from >1 pthreads:

1) io_uring_enter() is claimed as pthread safe

Quoted 8 lines

2) because of userspace lock protection, there is single code path for producing sqe for SQ at same time, and single code path for consuming sqe from io_uring_enter().

With bpf controlled io_uring patches, sqe can be produced from io_uring_enter(), and cqe can be consumed in io_uring_enter() too, there will be race between bpf prog(producing sqe, or consuming cqe) and userspace lock-protected code block.

BPF is attached by the same process/user that creates io_uring. The guarantees are same as before, the user code (which includes BPF) should protect from concurrent mutations.

In this example, just extend the first critical section to io_uring_enter(). Concurrent io_uring_enter() will be serialised by a mutex anyway. But let me note, that sharing rings is not a great pattern in either case.

Quoted 1 line

If io_uring_enter() needs to be serialised, it becomes pthread-unsafe,

The BPF program needs to be synchronised _if_ it races. There are different ways to sync, including from within the program, but not racing in the first place is still the preferred option.

Quoted 3 lines

that is why I mentioned this should be documented, because it is one very big difference introduced in bpf controlled ring.

That can definitely be mentioned as a guide to users, would be a diligent thing to do, but my point is that it doesn't change the contract. SQ/CQ are not protected, and it's the users obligation to synchronise it. With this set it includes BPF programs the user attaches.