resurrect
Subsystems (3)
cover resurrect
Combined approach with tryget, fallbacking to trying completion and doing synchronize_rcu, where the last one is very unlikely to happen and limited by one such per register syscall.
I expect 1/2 to cleanly apply for stable, 2/2 needs to be backported by hand.
Pavel Begunkov (2):
io_uring: wait potential ->release() on resurrect
io_uring: wait for ->release() on rsrc resurrect
fs/io_uring.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-)
Signature · 1 line
2.24.0
PATCH 1/2 io_uring: wait potential ->release() on resurrect
There is a short window where percpu_refs are already turned zero, but we try to do resurrect(). Play nicer and wait for ->release() to happen in this case and proceed as everything is ok. One little downside is that we can ignore signal_pending() on a rare occasion, but someone else should check for it later if needed.
--- fs/io_uring.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
fs/io_uring.c +17 -5
@@ -1104,6 +1104,21 @@ static inline void io_set_resource_node(struct io_kiocb *req) } } +static bool io_refs_resurrect (struct percpu_ref *ref,struct completion *compl) +{ +if (!percpu_ref_tryget (ref)) { +/* already at zero, wait for ->release() */ +if (!try_wait_for_completion (compl)) +synchronize_rcu (); +return false; + } + +percpu_ref_resurrect (ref); +reinit_completion (compl); +percpu_ref_put (ref); +return true; +} +static bool io_match_task (struct io_kiocb *head,struct task_struct *task,struct files_struct *files)
@@ -10094,10 +10109,8 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,mutex_lock (&ctx->uring_lock); -if (ret) { -percpu_ref_resurrect (&ctx->refs); - goto out_quiesce; - } +if (ret &&io_refs_resurrect (&ctx->refs, &ctx->ref_comp)) +return ret; }if (ctx->restricted) {
@@ -10189,7 +10202,6 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,if (io_register_op_must_quiesce (opcode)) {/* bring the ctx back to life */ percpu_ref_reinit (&ctx->refs); -out_quiesce:reinit_completion (&ctx->ref_comp); }return ret;
Signature · 1 line
2.24.0
PATCH 2/2 io_uring: wait for ->release() on rsrc resurrect
In reply to Pavel BegunkovAs with ctx refs, on resurrect wait for potentially concurrently running ->release().
--- fs/io_uring.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
fs/io_uring.c +1 -3
@@ -7368,13 +7368,11 @@ static int io_rsrc_ref_quiesce(struct fixed_rsrc_data *data,flush_delayed_work (&ctx->rsrc_put_work); ret =wait_for_completion_interruptible (&data->done); -if (!ret) +if (!ret || !io_refs_resurrect (&data->refs, &data->done))break ; -percpu_ref_resurrect (&data->refs);io_sqe_rsrc_set_node (ctx, data, backup_node); backup_node =NULL ; -reinit_completion (&data->done);mutex_unlock (&ctx->uring_lock); ret =io_run_task_work_sig ();mutex_lock (&ctx->uring_lock);
Signature · 1 line
2.24.0
Quoted 6 lines · Pavel Begunkov
Combined approach with tryget, fallbacking to trying completion and doing
synchronize_rcu, where the last one is very unlikely to happen and limited by one such per register syscall.I expect 1/2 to cleanly apply for stable, 2/2 needs to be backported by hand.
nevermind this, need to rebase and resend
Quoted 8 lines
Pavel Begunkov (2):
io_uring: wait potential ->release() on resurrect
io_uring: wait for ->release() on rsrc resurrectfs/io_uring.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-)
Signature · 1 line
Pavel Begunkov