Skip to contentSkip to conversation

inline execution ltimeout optimisation

PATCH Applied Pavel Begunkov 2 participants 5 messages Dec 1, 2023 – Dec 3, 2023 io_uring

lore
Share & tools
lore
#0 lore raw

cover inline execution ltimeout optimisation

Patches are not related but put touch adjacent lines. First we improve a little bit on post issue iopoll checks, and then return back a long lost linked timeout optimisation.

Pavel Begunkov (2):
io_uring: don't check iopoll if request completes
io_uring: optimise ltimeout for inline execution

 io_uring/io_uring.c | 21 ++++++++++++---------
 1 file changed, 12 insertions(+), 9 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/2 io_uring: don't check iopoll if request completes

IOPOLL request should never return IOU_OK, so the following iopoll queueing check in io_issue_sqe() after getting IOU_OK doesn't make any sense as would never turn true. Let's optimise on that and return a bit earlier. It's also much more resilient to potential bugs from mischieving iopoll implementations.

---
 io_uring/io_uring.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
io_uring/io_uring.c +5 -1
index 6ffd7216393b..21e646ef9654 100644
@@ -1896,7 +1896,11 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags)
 			io_req_complete_defer(req);
 		else
 			io_req_complete_post(req, issue_flags);
-	} else if (ret != IOU_ISSUE_SKIP_COMPLETE)
+
+		return 0;
+	}
+
+	if (ret != IOU_ISSUE_SKIP_COMPLETE)
 		return ret;
 
 	/* If the op doesn't have a file, we're not polling for it */
Signature · 1 line
2.43.0
#2 lore raw

PATCH 2/2 io_uring: optimise ltimeout for inline execution

In reply to Pavel Begunkov

At one point in time we had an optimisation that would not spin up a linked timeout timer when the master request successfully completes inline (during the first nowait execution attempt). We somehow lost it, so this patch restores it back.

Note, that it's fine using io_arm_ltimeout() after the io_issue_sqe() completes the request because of delayed completion, but that that adds unwanted overhead.

---
 io_uring/io_uring.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
io_uring/io_uring.c +9 -10
index 21e646ef9654..6212f81ed887 100644
@@ -1900,14 +1900,15 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags)
 		return 0;
 	}
 
-	if (ret != IOU_ISSUE_SKIP_COMPLETE)
-		return ret;
-
-	/* If the op doesn't have a file, we're not polling for it */
-	if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue)
-		io_iopoll_req_issued(req, issue_flags);
+	if (ret == IOU_ISSUE_SKIP_COMPLETE) {
+		ret = 0;
+		io_arm_ltimeout(req);
 
-	return 0;
+		/* If the op doesn't have a file, we're not polling for it */
+		if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue)
+			io_iopoll_req_issued(req, issue_flags);
+	}
+	return ret;
 }
 
 int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts)
@@ -2078,9 +2079,7 @@ static inline void io_queue_sqe(struct io_kiocb *req)
 	 * We async punt it if the file wasn't marked NOWAIT, or if the file
 	 * doesn't support non-blocking read/write attempts
 	 */
-	if (likely(!ret))
-		io_arm_ltimeout(req);
-	else
+	if (unlikely(ret))
 		io_queue_async(req, ret);
 }
 
Signature · 1 line
2.43.0
#3 lore raw

Re: [PATCH for-next 2/2] io_uring: optimise ltimeout for inline execution

Quoted 8 lines · Pavel Begunkov

At one point in time we had an optimisation that would not spin up a linked timeout timer when the master request successfully completes inline (during the first nowait execution attempt). We somehow lost it, so this patch restores it back.

Note, that it's fine using io_arm_ltimeout() after the io_issue_sqe() completes the request because of delayed completion, but that that adds unwanted overhead.

Let's add:

Quoted 44 lines
---
  io_uring/io_uring.c | 19 +++++++++----------
  1 file changed, 9 insertions(+), 10 deletions(-)
io_uring/io_uring.c +9 -10
index 21e646ef9654..6212f81ed887 100644
@@ -1900,14 +1900,15 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags)
  		return 0;
  	}
 
-	if (ret != IOU_ISSUE_SKIP_COMPLETE)
-		return ret;
-
-	/* If the op doesn't have a file, we're not polling for it */
-	if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue)
-		io_iopoll_req_issued(req, issue_flags);
+	if (ret == IOU_ISSUE_SKIP_COMPLETE) {
+		ret = 0;
+		io_arm_ltimeout(req);
 
-	return 0;
+		/* If the op doesn't have a file, we're not polling for it */
+		if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue)
+			io_iopoll_req_issued(req, issue_flags);
+	}
+	return ret;
  }
 
  int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts)
@@ -2078,9 +2079,7 @@ static inline void io_queue_sqe(struct io_kiocb *req)
  	 * We async punt it if the file wasn't marked NOWAIT, or if the file
  	 * doesn't support non-blocking read/write attempts
  	 */
-	if (likely(!ret))
-		io_arm_ltimeout(req);
-	else
+	if (unlikely(ret))
  		io_queue_async(req, ret);
  }
 
Signature · 1 line
Pavel Begunkov
maintainer Applied
#4 lore raw
In reply to Pavel Begunkov
Quoted 9 lines · Pavel Begunkov

Patches are not related but put touch adjacent lines. First we improve a little bit on post issue iopoll checks, and then return back a long lost linked timeout optimisation.

Pavel Begunkov (2):
io_uring: don't check iopoll if request completes
io_uring: optimise ltimeout for inline execution

[...]

Applied, thanks!

[1/2] io_uring: don't check iopoll if request completes
(no commit info)
[2/2] io_uring: optimise ltimeout for inline execution
(no commit info)

Best regards,

Signature · 1 line
Jens Axboe