Skip to contentSkip to conversation

io_uring: don't submit sqes when ctx->refs is dying

RFC PATCH Superseded Xiaoguang Wang 2 participants 6 messages May 13, 2020 – May 20, 2020

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 io_uring: don't submit sqes when ctx->refs is dying

When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:

    while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
        cond_resched();

Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.

In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.

---
 fs/io_uring.c | 19 ++++++-------------
 1 file changed, 6 insertions(+), 13 deletions(-)
fs/io_uring.c +6 -13
index ec0aa6957882..6e51140a5722 100644
@@ -5980,7 +5980,8 @@ static int io_sq_thread(void *data)
 		 * If submit got -EBUSY, flag us as needing the application
 		 * to enter the kernel to reap and flush events.
 		 */
-		if (!to_submit || ret == -EBUSY) {
+		if (!to_submit || ret == -EBUSY ||
+		    percpu_ref_is_dying(&ctx->refs)) {
 			/*
 			 * Drop cur_mm before scheduling, we can't hold it for
 			 * long periods (or over schedule()). Do this before
@@ -6027,7 +6028,8 @@ static int io_sq_thread(void *data)
 			smp_mb();
 
 			to_submit = io_sqring_entries(ctx);
-			if (!to_submit || ret == -EBUSY) {
+			if (!to_submit || ret == -EBUSY ||
+			    percpu_ref_is_dying(&ctx->refs)) {
 				if (kthread_should_park()) {
 					finish_wait(&ctx->sqo_wait, &wait);
 					break;
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data)
 		}
 
 		mutex_lock(&ctx->uring_lock);
-		ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
+		if (likely(!percpu_ref_is_dying(&ctx->refs)))
+			ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
 		mutex_unlock(&ctx->uring_lock);
 		timeout = jiffies + ctx->sq_thread_idle;
 	}
@@ -7323,16 +7326,6 @@ static void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx)
 	percpu_ref_kill(&ctx->refs);
 	mutex_unlock(&ctx->uring_lock);
 
-	/*
-	 * Wait for sq thread to idle, if we have one. It won't spin on new
-	 * work after we've killed the ctx ref above. This is important to do
-	 * before we cancel existing commands, as the thread could otherwise
-	 * be queueing new work post that. If that's work we need to cancel,
-	 * it could cause shutdown to hang.
-	 */
-	while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
-		cond_resched();
-
 	io_kill_timeouts(ctx);
 	io_poll_remove_all(ctx);
 
Signature · 1 line
2.17.2
maintainer
#1 lore raw
Quoted 9 lines · Xiaoguang Wang

When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:

    while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
        cond_resched();

Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.

In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.

I don't think this really changes anything. What happens if:

fs/io_uring.c static int io_sq_thread(void *data) · line 6059

Quoted 9 lines
fs/io_uring.c +2 -1
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data)
 		}
 
 		mutex_lock(&ctx->uring_lock);
-		ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
+		if (likely(!percpu_ref_is_dying(&ctx->refs)))
+			ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
 		mutex_unlock(&ctx->uring_lock);
 		timeout = jiffies + ctx->sq_thread_idle;

You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.

Signature · 1 line
Jens Axboe
#2 lore raw

hi,

Quoted 16 lines · Xiaoguang Wang
Quoted 9 lines

When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:

     while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
         cond_resched();

Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.

In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.

I don't think this really changes anything. What happens if:

Quoted 3 lines
(quoted diff) +0 -0
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data)
  		}
 
Quoted 10 lines

fs/io_uring.c static int io_sq_thread(void *data) · line 6059

Quoted 6 lines
fs/io_uring.c +2 -1
  		mutex_lock(&ctx->uring_lock);
-		ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
+		if (likely(!percpu_ref_is_dying(&ctx->refs)))
+			ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
  		mutex_unlock(&ctx->uring_lock);
  		timeout = jiffies + ctx->sq_thread_idle;

You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.

Look at codes, we call percpu_ref_kill() under uring_lock, so isn't it safe to check the refs' dying status? Thanks.

Regards,
Xiaouang Wang

Quoted 1 line
#3 lore raw

hi,

Quoted 27 lines

hi,

Quoted 13 lines · Xiaoguang Wang
Quoted 9 lines

When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:
     while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait)) cond_resched();
Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.

In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.

I don't think this really changes anything. What happens if:

Quoted 12 lines

fs/io_uring.c static int io_sq_thread(void *data) · line 6058

Quoted 8 lines
fs/io_uring.c +2 -1
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data)
           }
           mutex_lock(&ctx->uring_lock);
-        ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
+        if (likely(!percpu_ref_is_dying(&ctx->refs)))
+            ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
           mutex_unlock(&ctx->uring_lock);
           timeout = jiffies + ctx->sq_thread_idle;

You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.

Quoted 2 lines

Look at codes, we call percpu_ref_kill() under uring_lock, so isn't it safe to check the refs' dying status? Thanks.

Cloud you please have a look at my explanation again? Thanks.

Regards,
Xiaoguang Wang

Quoted 6 lines

Regards,
Xiaouang Wang

Quoted 1 line
maintainer
#4 lore raw
Quoted 29 lines · Xiaoguang Wang

hi,

Quoted 27 lines

hi,

Quoted 25 lines · Xiaoguang Wang
Quoted 9 lines

When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:

     while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
         cond_resched();

Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.

In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.

I don't think this really changes anything. What happens if:

fs/io_uring.c static int io_sq_thread(void *data) · line 6058

Quoted 8 lines
fs/io_uring.c +2 -1
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data)
          }
          mutex_lock(&ctx->uring_lock);
-        ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
+        if (likely(!percpu_ref_is_dying(&ctx->refs)))
+            ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
          mutex_unlock(&ctx->uring_lock);
          timeout = jiffies + ctx->sq_thread_idle;

You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.

Quoted 3 lines
Quoted 2 lines

Look at codes, we call percpu_ref_kill() under uring_lock, so isn't it safe to check the refs' dying status? Thanks.

Cloud you please have a look at my explanation again? Thanks.

Sorry for the delay - you are right, we only kill it inside the ring mutex, so should be safe to check. Can we get by with just the very last check instead of checking all of the cases?

Signature · 1 line
Jens Axboe
#5 lore raw

hi,

Quoted 30 lines · Xiaoguang Wang
Quoted 29 lines

hi,

Quoted 27 lines

hi,

Quoted 25 lines · Xiaoguang Wang
Quoted 9 lines

When IORING_SETUP_SQPOLL is enabled, io_ring_ctx_wait_and_kill() will wait for sq thread to idle by busy loop:

      while (ctx->sqo_thread && !wq_has_sleeper(&ctx->sqo_wait))
          cond_resched();

Above codes are not friendly, indeed I think this busy loop will introduce a cpu burst in current cpu, though it maybe short.

In this patch, if ctx->refs is dying, we forbids sq_thread from submitting sqes anymore, just discard leftover sqes.

I don't think this really changes anything. What happens if:

fs/io_uring.c static int io_sq_thread(void *data) · line 6058

Quoted 8 lines
fs/io_uring.c +2 -1
@@ -6051,7 +6053,8 @@ static int io_sq_thread(void *data)
           }
           mutex_lock(&ctx->uring_lock);
-        ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
+        if (likely(!percpu_ref_is_dying(&ctx->refs)))
+            ret = io_submit_sqes(ctx, to_submit, NULL, -1, true);
           mutex_unlock(&ctx->uring_lock);
           timeout = jiffies + ctx->sq_thread_idle;

You check for dying here, but that could change basically while you're checking it. So you're still submitting sqes with a ref that's going away. You've only reduced the window, you haven't eliminated it.

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

Look at codes, we call percpu_ref_kill() under uring_lock, so isn't it safe to check the refs' dying status? Thanks.

Cloud you please have a look at my explanation again? Thanks.

Sorry for the delay - you are right, we only kill it inside the ring mutex, so should be safe to check. Can we get by with just the very last check instead of checking all of the cases?

Sure, I'll send V2 soon, and thanks for reviewing, I need this patch because I'm writing a prototype to implement idea in this url:

    https://lore.kernel.org/io-uring/[email protected]/T/#t

which needs above patch.

Regards,
Xiaoguang Wang

Quoted 1 line