Add support for IORING_CQE_F_SOCK_FULL
Jens Axboe
maintainercover Add support for IORING_CQE_F_SOCK_FULL
Hi,
It can be useful for userspace to know if a send request had to go through poll to complete, as that generally means that the socket was out of space. On the send side, this is pretty trivial to support - we just need to check if the request needed to go through poll to complete.
This reuses the IORING_CQE_F_SOCK_NONEMPTY flag value, which is only valid for recv operations. As IORING_CQE_F_SOCK_FULL only applies on sends, there's no need for separate values for this flag.
Based on an earlier patchset, which utilized REQ_F_POLL_ARMED instead and handled patch 1 a bit differently.
include/linux/io_uring_types.h | 3 +++ include/uapi/linux/io_uring.h | 4 ++++ io_uring/net.c | 19 +++++++++++++------ io_uring/poll.c | 1 + 4 files changed, 21 insertions(+), 6 deletions(-)
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerPATCH 1/2 io_uring/poll: flag request with REQ_F_POLL_TRIGGERED if it went through poll
If a request targeting a pollable file cannot get executed without needing to wait for a POLLIN/POLLOUT trigger, then flag it with REQ_F_POLL_TRIGGERED. Only supported for non-multishot requests, as it's fully expected that multishot will always end up going through poll.
No functional changes in this patch, just in preparation for using this information elsewhere.
--- include/linux/io_uring_types.h | 3 +++ io_uring/poll.c | 1 + 2 files changed, 4 insertions(+)
include/linux/io_uring_types.h +3 -0
@@ -521,6 +521,7 @@ enum {REQ_F_HAS_METADATA_BIT ,REQ_F_IMPORT_BUFFER_BIT ,REQ_F_SQE_COPIED_BIT , +REQ_F_POLL_TRIGGERED_BIT ,/* not a real bit, just to check we're not overflowing the space */ __REQ_F_LAST_BIT,
@@ -612,6 +613,8 @@ enum {REQ_F_IMPORT_BUFFER =IO_REQ_FLAG (REQ_F_IMPORT_BUFFER_BIT ),/* ->sqe_copy() has been called, if necessary */ REQ_F_SQE_COPIED =IO_REQ_FLAG (REQ_F_SQE_COPIED_BIT ), +/* poll was triggered at least once for this request */ + REQ_F_POLL_TRIGGERED = IO_REQ_FLAG(REQ_F_POLL_TRIGGERED_BIT), }; struct io_tw_req {
io_uring/poll.c +1 -0
@@ -419,6 +419,7 @@ static int io_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync, req->flags &= ~REQ_F_DOUBLE_POLL ; else req->flags &= ~REQ_F_SINGLE_POLL; + req->flags |=REQ_F_POLL_TRIGGERED ; }__io_poll_execute (req, mask); }
Signature · 1 line
2.51.0
Jens Axboe
maintainerPATCH 2/2 io_uring/net: add IORING_CQE_F_SOCK_FULL if a send needed to poll arm
In reply to Jens AxboeIf a send/sendmsg/sendzc/sendmsgzc needed to wait for space in the socket to complete, add IORING_CQE_F_SOCK_FULL to the cqe->flags to tell the application about it. This meant that the socket was full when the operation was attempted.
This adds IORING_CQE_F_SOCK_FULL as a new CQE flag. It borrows the value of IORING_CQE_F_SOCK_NONEMPTY, which is a flag only used on the receive side. Hence there can be no confusion on which of the two meanings is included in a given CQE, as the application must know which kind of operation the completion refers to.
--- include/uapi/linux/io_uring.h | 4 ++++ io_uring/net.c | 19 +++++++++++++------ 2 files changed, 17 insertions(+), 6 deletions(-)
include/uapi/linux/io_uring.h +4 -0
@@ -495,6 +495,9 @@ struct io_uring_cqe {* IORING_CQE_F_BUFFER If set, the upper 16 bits are the buffer ID * IORING_CQE_F_MORE If set, parent SQE will generate more CQE entries * IORING_CQE_F_SOCK_NONEMPTY If set, more data to read after socket recv +* IORING_CQE_F_SOCK_FULL If set, the socket was full when this send or +* sendmsg was attempted. Hence it had to wait for POLLOUT +* before being able to complete. * IORING_CQE_F_NOTIF Set for notification CQEs. Can be used to distinct * them from sends. * IORING_CQE_F_BUF_MORE If set, the buffer ID set in the completion will get
@@ -518,6 +521,7 @@ struct io_uring_cqe {#define IORING_CQE_F_BUFFER (1U << 0)#define IORING_CQE_F_MORE (1U << 1)#define IORING_CQE_F_SOCK_NONEMPTY (1U << 2) +#define IORING_CQE_F_SOCK_FULL IORING_CQE_F_SOCK_NONEMPTY#define IORING_CQE_F_NOTIF (1U << 3)#define IORING_CQE_F_BUF_MORE (1U << 4)#define IORING_CQE_F_SKIP (1U << 5)
io_uring/net.c +13 -6
@@ -530,11 +530,21 @@ static inline bool io_send_finish(struct io_kiocb *req,/* Otherwise stop bundle and use the current result. */ finish: +if (req->flags &REQ_F_POLL_TRIGGERED ) + cflags |=IORING_CQE_F_SOCK_FULL ;io_req_set_res (req, sel->val, cflags); sel->val =IOU_COMPLETE ;return true; } +static int io_send_complete (struct io_kiocb *req,int ret,unsigned cflags) +{ +if (req->flags &REQ_F_POLL_TRIGGERED ) + cflags |=IORING_CQE_F_SOCK_FULL ; +io_req_set_res (req, ret, cflags); +return IOU_COMPLETE ; +} +int io_sendmsg (struct io_kiocb *req,unsigned int issue_flags) { struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg);
@@ -580,8 +590,7 @@ int io_sendmsg(struct io_kiocb *req, unsigned int issue_flags) ret += sr->done_io; else if (sr->done_io) ret = sr->done_io; -io_req_set_res (req, ret,0 ); -return IOU_COMPLETE ; +return io_send_complete (req, ret,0 ); } static int io_send_select_buffer(struct io_kiocb *req, unsigned int issue_flags,
@@ -1516,8 +1525,7 @@ int io_send_zc(struct io_kiocb *req, unsigned int issue_flags) zc->notif =NULL ;io_req_msg_cleanup (req,0 ); } -io_req_set_res (req, ret,IORING_CQE_F_MORE ); -return IOU_COMPLETE ; +return io_send_complete (req, ret,IORING_CQE_F_MORE ); }int io_sendmsg_zc (struct io_kiocb *req,unsigned int issue_flags)
@@ -1586,8 +1594,7 @@ int io_sendmsg_zc(struct io_kiocb *req, unsigned int issue_flags) sr->notif =NULL ;io_req_msg_cleanup (req,0 ); } -io_req_set_res (req, ret,IORING_CQE_F_MORE ); -return IOU_COMPLETE ; +return io_send_complete (req, ret,IORING_CQE_F_MORE ); }void io_sendrecv_fail (struct io_kiocb *req)
Signature · 1 line
2.51.0
Quoted 7 lines · Jens Axboe
Hi,
It can be useful for userspace to know if a send request had to go through poll to complete, as that generally means that the socket was out of space. On the send side, this is pretty trivial to support - we just need to check if the request needed to go through poll to complete.
Quoted 6 lines
This reuses the
IORING_CQE_F_SOCK_NONEMPTYflag value, which is only valid for recv operations. AsIORING_CQE_F_SOCK_FULLonly applies on sends, there's no need for separate values for this flag.Based on an earlier patchset, which utilized
REQ_F_POLL_ARMEDinstead and handled patch 1 a bit differently.
FWIW, same comments as last time. REQ_F_POLL_TRIGGERED is set not in the right place. And, with how tcp manages wait queues, you won't be able to use it well for any throttling, as the user will get the flagged CQE long time after, when the queue is already half empty.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 12 lines · Pavel Begunkov
Quoted 11 lines · Jens Axboe
Hi,
It can be useful for userspace to know if a send request had to go through poll to complete, as that generally means that the socket was out of space. On the send side, this is pretty trivial to support - we just need to check if the request needed to go through poll to complete.
This reuses the
IORING_CQE_F_SOCK_NONEMPTYflag value, which is only valid for recv operations. AsIORING_CQE_F_SOCK_FULLonly applies on sends, there's no need for separate values for this flag.
Quoted 7 lines
Quoted 2 lines
Based on an earlier patchset, which utilized
REQ_F_POLL_ARMEDinstead and handled patch 1 a bit differently.FWIW, same comments as last time.
REQ_F_POLL_TRIGGEREDis set not in the right place. And, with how tcp manages wait queues, you won't be able to use it well for any throttling, as the user will get the flagged CQE long time after, when the queue is already half empty.
I'll take a closer look when I'm back, on the road until Monday...
Signature · 1 line
Jens Axboe