io_uring: don't submit sqes when ctx->refs is dying
RFC PATCH 1/1 io_uring: don't submit sqes when ctx->refs is dying
When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:
while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
cond_resched();Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.
In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.
--- fs/io_uring.c | 19 ++++++------------- 1 file changed, 6 insertions(+), 13 deletions(-)
fs/io_uring.c +6 -13
@@ -5980,7 +5980,8 @@ static int io_sq_thread(void *data)* If submit got -EBUSY, flag us as needing the application * to enter the kernel to reap and flush events. */ -if (!to_submit || ret == -EBUSY ) { +if (!to_submit || ret == -EBUSY || +percpu_ref_is_dying (&ctx->refs)) {/* * Drop cur_mm before scheduling, we can't hold it for * long periods (or over schedule()). Do this before
@@ -6027,7 +6028,8 @@ static int io_sq_thread(void *data)smp_mb (); to_submit =io_sqring_entries (ctx); -if (!to_submit || ret == -EBUSY ) { +if (!to_submit || ret == -EBUSY || +percpu_ref_is_dying (&ctx->refs)) {if (kthread_should_park ()) {finish_wait (&ctx->sqo_wait, &wait);break ;
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data) }mutex_lock (&ctx->uring_lock); - ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true); +if (likely (!percpu_ref_is_dying (&ctx->refs))) + ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true);mutex_unlock (&ctx->uring_lock); timeout = jiffies + ctx->sq_thread_idle; }
@@ -7323,16 +7326,6 @@ static void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx)percpu_ref_kill (&ctx->refs);mutex_unlock (&ctx->uring_lock); -/* -* Wait for sq thread to idle, if we have one. It won't spin on new -* work after we've killed the ctx ref above. This is important to do -* before we cancel existing commands, as the thread could otherwise -* be queueing new work post that. If that's work we need to cancel, -* it could cause shutdown to hang. -*/ -while (ctx->sqo_thread && !wq_has_sleeper (&ctx->sqo_wait)) -cond_resched (); -io_kill_timeouts (ctx);io_poll_remove_all (ctx);
Signature · 1 line
2.17.2
Jens Axboe
maintainerQuoted 9 lines · Xiaoguang Wang
When
IORING_SETUP_SQPOLLis enabled,io_ring_ctx_wait_and_kill()will wait for sq thread to idle by busy loop:while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait)) cond_resched();Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.
In this patch, if
ctx->refsis dying, we forbidssq_threadfrom submitting sqes anymore, just discard leftover sqes.
I don't think this really changes anything. What happens if:
fs/io_uring.c static int io_sq_thread(void *data) · line 6059
Quoted 9 lines
fs/io_uring.c +2
-1@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data) }mutex_lock (&ctx->uring_lock); - ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true); +if (likely (!percpu_ref_is_dying (&ctx->refs))) + ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true);mutex_unlock (&ctx->uring_lock); timeout = jiffies + ctx->sq_thread_idle;
You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.
Signature · 1 line
Jens Axboe
hi,
Quoted 16 lines · Xiaoguang Wang
Quoted 9 lines
When
IORING_SETUP_SQPOLLis enabled,io_ring_ctx_wait_and_kill()will wait for sq thread to idle by busy loop:while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait)) cond_resched();Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.
In this patch, if
ctx->refsis dying, we forbidssq_threadfrom submitting sqes anymore, just discard leftover sqes.I don't think this really changes anything. What happens if:
Quoted 3 lines
(quoted diff) +0
-0@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data) }
Quoted 10 lines
fs/io_uring.c static int io_sq_thread(void *data) · line 6059
Quoted 6 lines
fs/io_uring.c +2
-1mutex_lock (&ctx->uring_lock); - ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true); +if (likely (!percpu_ref_is_dying (&ctx->refs))) + ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true);mutex_unlock (&ctx->uring_lock); timeout = jiffies + ctx->sq_thread_idle;You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.
Look at codes, we call percpu_ref_kill() under uring_lock, so isn't it safe to check the refs' dying status? Thanks.
Regards,
Xiaouang Wang
Quoted 1 line
hi,
Quoted 27 lines
hi,
Quoted 13 lines · Xiaoguang Wang
Quoted 9 lines
When
IORING_SETUP_SQPOLLis enabled,io_ring_ctx_wait_and_kill()will wait for sq thread to idle by busy loop:
while (ctx->sqo_thread&& !wq_has_sleeper(&ctx->sqo_wait))cond_resched();
Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.In this patch, if
ctx->refsis dying, we forbidssq_threadfrom submitting sqes anymore, just discard leftover sqes.I don't think this really changes anything. What happens if:
Quoted 12 lines
fs/io_uring.c static int io_sq_thread(void *data) · line 6058
Quoted 8 lines
fs/io_uring.c +2
-1@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data) } mutex_lock(&ctx->uring_lock); - ret = io_submit_sqes(ctx, to_submit, NULL, -1, true); + if (likely(!percpu_ref_is_dying(&ctx->refs))) + ret = io_submit_sqes(ctx, to_submit, NULL, -1, true); mutex_unlock(&ctx->uring_lock); timeout = jiffies + ctx->sq_thread_idle;You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.
Quoted 2 lines
Look at codes, we call
percpu_ref_kill()underuring_lock, so isn't it safe to check the refs' dying status? Thanks.
Cloud you please have a look at my explanation again? Thanks.
Regards,
Xiaoguang Wang
Quoted 6 lines
Regards,
Xiaouang WangQuoted 1 line
Jens Axboe
maintainerQuoted 29 lines · Xiaoguang Wang
hi,
Quoted 27 lines
hi,
Quoted 25 lines · Xiaoguang Wang
Quoted 9 lines
When
IORING_SETUP_SQPOLLis enabled,io_ring_ctx_wait_and_kill()will wait for sq thread to idle by busy loop:while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait)) cond_resched();Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.
In this patch, if
ctx->refsis dying, we forbidssq_threadfrom submitting sqes anymore, just discard leftover sqes.I don't think this really changes anything. What happens if:
fs/io_uring.c static int io_sq_thread(void *data) · line 6058
Quoted 8 lines
fs/io_uring.c +2
-1@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data) }mutex_lock (&ctx->uring_lock); - ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true); +if (likely (!percpu_ref_is_dying (&ctx->refs))) + ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true);mutex_unlock (&ctx->uring_lock); timeout = jiffies + ctx->sq_thread_idle;You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.
Quoted 3 lines
Quoted 2 lines
Look at codes, we call
percpu_ref_kill()underuring_lock, so isn't it safe to check the refs' dying status? Thanks.Cloud you please have a look at my explanation again? Thanks.
Sorry for the delay - you are right, we only kill it inside the ring mutex, so should be safe to check. Can we get by with just the very last check instead of checking all of the cases?
Signature · 1 line
Jens Axboe
hi,
Quoted 30 lines · Xiaoguang Wang
Quoted 29 lines
hi,
Quoted 27 lines
hi,
Quoted 25 lines · Xiaoguang Wang
Quoted 9 lines
When
IORING_SETUP_SQPOLLis enabled,io_ring_ctx_wait_and_kill()will wait for sq thread to idle by busy loop:while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait)) cond_resched();Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.
In this patch, if
ctx->refsis dying, we forbidssq_threadfrom submitting sqes anymore, just discard leftover sqes.I don't think this really changes anything. What happens if:
fs/io_uring.c static int io_sq_thread(void *data) · line 6058
Quoted 8 lines
fs/io_uring.c +2
-1@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data) }mutex_lock (&ctx->uring_lock); - ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true); +if (likely (!percpu_ref_is_dying (&ctx->refs))) + ret =io_submit_sqes (ctx, to_submit,NULL ,-1 , true);mutex_unlock (&ctx->uring_lock); timeout = jiffies + ctx->sq_thread_idle;You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Look at codes, we call
percpu_ref_kill()underuring_lock, so isn't it safe to check the refs' dying status? Thanks.Cloud you please have a look at my explanation again? Thanks.
Sorry for the delay - you are right, we only kill it inside the ring mutex, so should be safe to check. Can we get by with just the very last check instead of checking all of the cases?
Sure, I'll send V2 soon, and thanks for reviewing, I need this patch because I'm writing a prototype to implement idea in this url:
https://lore.kernel.org/io-uring/[email protected]/T/#t
which needs above patch.
Regards,
Xiaoguang Wang