Add io_uring_register() based cancel
Jens Axboe
maintainercover Add io_uring_register() based cancel
Hi,
One of the tricky parts of cancelations is work that is handled by io-wq, and the regular cancelation API is naturally async and just returns -EALREADY for work in that state. This means that the work cancelation has been started, but we don't know if it's done yet.
With an async API for cancelation, we really can't do much better than that. This leaves the application canceling work to need to wait for a CQE from the original request.
Add a way to do sync cancel, even for io-wq. Since this isn't a natural fit for an SQE based cancel, add it via io_uring_register(), adding a IORING_REGISTER_SYNC_CANCEL operation. If we get -EALREADY, we wait for completions to come in. When they do, we re-check. Once we get -ENOENT for a lookup that previously gave us -EALREADY, we know the targeted requests have been stopped.
By utilizing the usual ctx->cq_wait to trigger a retry of the cancel operation, we avoid adding any kind of extra overhead to the normal completion path and tracking specific requests.
This gives applications an easy way to cancel any kind of request, and know that buffers associated with them will not be touched by the kernel. This effectively returns ownership of the data back to the application.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerPATCH 1/3 io_uring: have cancelation API accept io_uring_task directly
We just use the io_kiocb passed in to find the io_uring_task, and we already pass in the ctx via cd->ctx anyway.
--- io_uring/cancel.c | 17 +++++++++-------- io_uring/cancel.h | 2 +- io_uring/timeout.c | 2 +- 3 files changed, 11 insertions(+), 10 deletions(-)
io_uring/cancel.c +9 -8
@@ -77,15 +77,15 @@ static int io_async_cancel_one(struct io_uring_task *tctx,return ret; } -int io_try_cancel (struct io_kiocb *req,struct io_cancel_data *cd, +int io_try_cancel (struct io_uring_task *tctx,struct io_cancel_data *cd,unsigned issue_flags) { -struct io_ring_ctx *ctx = req->ctx; +struct io_ring_ctx *ctx = cd->ctx;int ret; -WARN_ON_ONCE (!io_wq_current_is_worker () && req->task != current); +WARN_ON_ONCE (!io_wq_current_is_worker () && tctx != current->io_uring); - ret =io_async_cancel_one (req->task->io_uring, cd); + ret =io_async_cancel_one (tctx, cd);/* * Fall-through even for -EALREADY, as we may have poll armed * that need unarming.
@@ -104,7 +104,6 @@ int io_try_cancel(struct io_kiocb *req, struct io_cancel_data *cd,return ret; } -int io_async_cancel_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe) {struct io_cancel *cancel =io_kiocb_to_cmd (req);
@@ -127,7 +126,8 @@ int io_async_cancel_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)return 0 ; } -static int __io_async_cancel (struct io_cancel_data *cd,struct io_kiocb *req, +static int __io_async_cancel (struct io_cancel_data *cd, +struct io_uring_task *tctx,unsigned int issue_flags) {bool all = cd->flags & (IORING_ASYNC_CANCEL_ALL |IORING_ASYNC_CANCEL_ANY );
@@ -136,7 +136,7 @@ static int __io_async_cancel(struct io_cancel_data *cd, struct io_kiocb *req,int ret, nr =0 ; do { - ret = io_try_cancel(req, cd, issue_flags); + ret = io_try_cancel(tctx, cd, issue_flags); if (ret == -ENOENT) break; if (!all)
@@ -170,6 +170,7 @@ int io_async_cancel(struct io_kiocb *req, unsigned int issue_flags) .flags = cancel->flags, .seq = atomic_inc_return(&req->ctx->cancel_seq), }; +struct io_uring_task *tctx = req->task->io_uring;int ret;if (cd.flags &IORING_ASYNC_CANCEL_FD ) {
@@ -185,7 +186,7 @@ int io_async_cancel(struct io_kiocb *req, unsigned int issue_flags) cd.file = req->file; } - ret =__io_async_cancel (&cd, req, issue_flags); + ret =__io_async_cancel (&cd, tctx, issue_flags); done:if (ret <0 )req_set_fail (req);
io_uring/cancel.h +1 -1
@@ -16,6 +16,6 @@ struct io_cancel_data {int io_async_cancel_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe);int io_async_cancel (struct io_kiocb *req,unsigned int issue_flags); -int io_try_cancel (struct io_kiocb *req,struct io_cancel_data *cd, +int io_try_cancel (struct io_uring_task *tctx,struct io_cancel_data *cd,unsigned int issue_flags);void init_hash_table (struct io_hash_table *table,unsigned size);
io_uring/timeout.c +1 -1
@@ -272,7 +272,7 @@ static void io_req_task_link_timeout(struct io_kiocb *req, bool *locked) .data = prev->cqe.user_data, }; - ret =io_try_cancel (req, &cd, issue_flags); + ret =io_try_cancel (req->task->io_uring, &cd, issue_flags); }io_req_set_res (req, ret ?: -ETIME ,0 );io_req_complete_post (req);
Signature · 1 line
2.35.1
Jens Axboe
maintainerPATCH 2/3 io_uring: add IORING_ASYNC_CANCEL_FD_FIXED cancel flag
In reply to Jens AxboeIn preparation for not having a request to pass in that carries this state, add a separate cancelation flag that allows the caller to ask for a fixed file for cancelation.
--- include/uapi/linux/io_uring.h | 2 ++ io_uring/cancel.c | 9 ++++++--- 2 files changed, 8 insertions(+), 3 deletions(-)
include/uapi/linux/io_uring.h +2 -0
@@ -244,10 +244,12 @@ enum io_uring_op {* IORING_ASYNC_CANCEL_FD Key off 'fd' for cancelation rather than the * request 'user_data' * IORING_ASYNC_CANCEL_ANY Match any request +* IORING_ASYNC_CANCEL_FD_FIXED 'fd' passed in is a fixed descriptor */ #define IORING_ASYNC_CANCEL_ALL (1U << 0)#define IORING_ASYNC_CANCEL_FD (1U << 1)#define IORING_ASYNC_CANCEL_ANY (1U << 2) +#define IORING_ASYNC_CANCEL_FD_FIXED (1U << 3)/* * send/sendmsg and recv/recvmsg flags (sqe->addr2)
io_uring/cancel.c +6 -3
@@ -24,7 +24,7 @@ struct io_cancel { };#define CANCEL_FLAGS (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_FD | \ - IORING_ASYNC_CANCEL_ANY) + IORING_ASYNC_CANCEL_ANY | IORING_ASYNC_CANCEL_FD_FIXED)static bool io_cancel_cb (struct io_wq_work *work,void *data) {
@@ -174,11 +174,14 @@ int io_async_cancel(struct io_kiocb *req, unsigned int issue_flags)int ret;if (cd.flags &IORING_ASYNC_CANCEL_FD ) { -if (req->flags &REQ_F_FIXED_FILE ) +if (req->flags &REQ_F_FIXED_FILE || + cd.flags &IORING_ASYNC_CANCEL_FD_FIXED ) { + req->flags |=REQ_F_FIXED_FILE ; req->file =io_file_get_fixed (req, cancel->fd, issue_flags); -else + }else { req->file =io_file_get_normal (req, cancel->fd); + }if (!req->file) { ret = -EBADF ; goto done;
Signature · 1 line
2.35.1
Jens Axboe
maintainerPATCH 3/3 io_uring: add sync cancelation API through io_uring_register()
In reply to Jens AxboeThe io_uring cancelation API is async, like any other API that we expose there. For the case of finding a request to cancel, or not finding one, it is fully sync in that when submission returns, the CQE for both the cancelation request and the targeted request have been posted to the CQ ring.
However, if the targeted work is being executed by io-wq, the API can only start the act of canceling it. This makes it difficult to use in some circumstances, as the caller then has to wait for the CQEs to come in and match on the same cancelation data there.
Provide a IORING_REGISTER_SYNC_CANCEL command for io_uring_register() that does sync cancelations, always. For the io-wq case, it'll wait for the cancelation to come in before returning. The only expected returns from this API is:
0 Request found and canceled fine.
Quoted 1 line
0 Requests found and canceled. Only happens if asked to
cancel multiple requests, and if the work wasn't in progress.-ENOENT Request not found.-ETIME A timeout on the operation was requested, but the timeout
expired before we could cancel.
and we won't get -EALREADY via this API.
If the timeout value passed in is -1 (tv_sec and tv_nsec), then that means that no timeout is requested. Otherwise, the timespec passed in is the amount of time the sync cancel will wait for a successful cancelation.
--- include/uapi/linux/io_uring.h | 15 +++++ io_uring/cancel.c | 107 ++++++++++++++++++++++++++++++++++ io_uring/cancel.h | 2 + io_uring/io_uring.c | 6 ++ 4 files changed, 130 insertions(+)
include/uapi/linux/io_uring.h +15 -0
@@ -10,6 +10,7 @@#include <linux/fs.h> #include <linux/types.h> +#include <linux/time_types.h> /* * IO submission data structure (Submission Queue Entry)
@@ -425,6 +426,9 @@ enum {IORING_REGISTER_PBUF_RING =22 ,IORING_UNREGISTER_PBUF_RING =23 , +/* sync cancelation API */ +IORING_REGISTER_SYNC_CANCEL =24 , +/* this goes last */ IORING_REGISTER_LAST };
@@ -560,4 +564,15 @@ struct io_uring_getevents_arg { __u64 ts; }; +/* +* Argument for IORING_REGISTER_SYNC_CANCEL +*/ +struct io_uring_sync_cancel_reg { +__u64 addr; +__s32 fd; +__u32 flags; +struct __kernel_timespec timeout; +__u64 pad[4 ]; +}; +#endif
io_uring/cancel.c +107 -0
116 lines not on this page · raw message
io_uring/cancel.h +2 -0
@@ -19,3 +19,5 @@ int io_async_cancel(struct io_kiocb *req, unsigned int issue_flags);int io_try_cancel (struct io_uring_task *tctx,struct io_cancel_data *cd,unsigned int issue_flags);void init_hash_table (struct io_hash_table *table,unsigned size); + +int io_sync_cancel (struct io_ring_ctx *ctx,void __user *arg);
io_uring/io_uring.c +6 -0
@@ -3911,6 +3911,12 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,break ; ret =io_unregister_pbuf_ring (ctx, arg);break ; +case IORING_REGISTER_SYNC_CANCEL : + ret = -EINVAL ; +if (!arg || nr_args !=1 ) +break ; + ret =io_sync_cancel (ctx, arg); +break ;default : ret = -EINVAL ;break ;
Signature · 1 line
2.35.1
Re: [PATCH 3/3] io_uring: add sync cancelation API through io_uring_register()
Hi Jens,
io_uring/cancel.c void init_hash_table(struct io_hash_table *table, unsigned size) · line 285
Quoted 1 line · Jens Axboe
io_uring/cancel.c +1
-0+ cd.seq = atomic_inc_return(&ctx->cancel_seq),
Not sure if it's intentional, but the comma after atomic_inc return() looks unique. While that's valid syntax, I think you want to use a semicolon to end it consistently.
Quoted 4 lines
io_uring/cancel.c +4
-0+ +prepare_to_wait (&ctx->cq_wait, &wait,TASK_INTERRUPTIBLE ); + + ret =__io_sync_cancel (current->io_uring, &cd, sc.fd);
Signature · 1 line
Ammar Faizi
Jens Axboe
maintainerRe: [PATCH 3/3] io_uring: add sync cancelation API through io_uring_register()
Quoted 8 lines · Ammar Faizi
Hi Jens,
Quoted 1 line · Jens Axboe
+ cd.seq =
atomic_inc_return(&ctx->cancel_seq),Not sure if it's intentional, but the comma after
atomic_increturn()looks unique. While that's valid syntax, I think you want to use a semicolon to end it consistently.
Oops. Not intentional, that line was copied from the struct init at the top of that function. I'll fix it up, thanks.
Signature · 1 line
Jens Axboe