io_uring/net: inherit IORING_CQE_F_BUF_MORE across bundle recv retries
PATCH 1/1 io_uring/net: inherit IORING_CQE_F_BUF_MORE across bundle recv retries
When a bundle recv retries inside io_recv_finish(), the merge logic OR the saved cflags from the previous iteration with the cflags returned by the new iteration:
cflags = req->cqe.flags | (cflags & CQE_F_MASK);
Bits listed in CQE_F_MASK are inherited from the new iteration, and all other bits (notably IORING_CQE_F_BUFFER and the buffer ID) come from the saved cflags. Before this change CQE_F_MASK covered only IORING_CQE_F_SOCK_NONEMPTY and IORING_CQE_F_MORE.
When using provided buffer rings (IOU_PBUF_RING_INC) with incremental mode, and bundle recv, io_kbuf_inc_commit() can leave the head ring entry partially consumed, __io_put_kbufs() then sets IORING_CQE_F_BUF_MORE on the returned cflags so userspace knows the buffer ID will be reused for subsequent completions.
Because IORING_CQE_F_BUF_MORE was not in CQE_F_MASK, the merge above silently dropped it whenever the final retry iteration partially consumed the buffer, and the subsequent req->cqe.flags = cflags & ~CQE_F_MASK save would have left a stale IORING_CQE_F_BUF_MORE in the carried-over cflags had one been present. Userspace would then wrongfully advance it ring head past an entry the kernel still uses.
Add IORING_CQE_F_BUF_MORE to CQE_F_MASK so it is both inherited from the new iteration into the user-visible CQE and stripped from the saved cflags between iterations.
A test available in https://github.com/clementleger/liburing/tree/bug_f_buf_more
allows to validate this fix.
--- io_uring/net.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
io_uring/net.c +2 -1
@@ -842,7 +842,8 @@ int io_recvmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) }/* bits to clear in old and inherit in new cflags on bundle retry */ -#define CQE_F_MASK (IORING_CQE_F_SOCK_NONEMPTY|IORING_CQE_F_MORE) +#define CQE_F_MASK (IORING_CQE_F_SOCK_NONEMPTY|IORING_CQE_F_MORE|\ + IORING_CQE_F_BUF_MORE)/* * Finishes io_recv and io_recvmsg.
--
2.53.0-Meta
Jens Axboe
maintainerQuoted 23 lines · Cl?ment L?ger
When a bundle recv retries inside
io_recv_finish(), the merge logic OR the saved cflags from the previous iteration with the cflags returned by the new iteration:
cflags =req->cqe.flags| (cflags &CQE_F_MASK);Bits listed in
CQE_F_MASKare inherited from the new iteration, and all other bits (notablyIORING_CQE_F_BUFFERand the buffer ID) come from the saved cflags. Before this changeCQE_F_MASKcovered onlyIORING_CQE_F_SOCK_NONEMPTYandIORING_CQE_F_MORE.When using provided buffer rings (
IOU_PBUF_RING_INC) with incremental mode, and bundle recv,io_kbuf_inc_commit()can leave the head ring entry partially consumed,__io_put_kbufs()then setsIORING_CQE_F_BUF_MOREon the returned cflags so userspace knows the buffer ID will be reused for subsequent completions.Because
IORING_CQE_F_BUF_MOREwas not inCQE_F_MASK, the merge above silently dropped it whenever the final retry iteration partially consumed the buffer, and the subsequentreq->cqe.flags= cflags & ~CQE_F_MASK save would have left a staleIORING_CQE_F_BUF_MOREin the carried-over cflags had one been present. Userspace would then wrongfully advance it ring head past an entry the kernel still uses.
Quoted 3 lines
Add
IORING_CQE_F_BUF_MOREtoCQE_F_MASKso it is both inherited from the new iteration into the user-visible CQE and stripped from the saved cflags between iterations.
Looks good!
Quoted 3 lines
A test available in
https://github.com/clementleger/liburing/tree/bug_f_buf_more allows to validate this fix.
Can you send in the test case separately for liburing?
Quoted 3 lines
Signed-off-by Cl?ment L?ger <[email protected]>Assisted-by Claude:claude-opus-4.6
I'll add a:
Signature · 1 line
Jens Axboe
Quoted 11 lines · Clément Léger
When a bundle recv retries inside
io_recv_finish(), the merge logic OR the saved cflags from the previous iteration with the cflags returned by the new iteration:
cflags =req->cqe.flags| (cflags &CQE_F_MASK);Bits listed in
CQE_F_MASKare inherited from the new iteration, and all other bits (notablyIORING_CQE_F_BUFFERand the buffer ID) come from the saved cflags. Before this changeCQE_F_MASKcovered onlyIORING_CQE_F_SOCK_NONEMPTYandIORING_CQE_F_MORE.[...]
Applied, thanks!
[1/1] io_uring/net: inherit IORING_CQE_F_BUF_MORE across bundle recv retries
commit: ed46f39c47eb5530a9c161481a2080d3a869cfaf
Best regards,
Signature · 1 line
Jens Axboe