Skip to contentSkip to conversation

io_uring: kbuf: add comments for some tricky code

PATCH Superseded Hao Xu 2 participants 4 messages Jun 14, 2022 – Jun 17, 2022 io_uring

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Hao Xu

author
#0 lore raw

PATCH 2/2 io_uring: kbuf: add comments for some tricky code

From: Hao Xu <[email protected]>

Add comments to explain why it is always under uring lock when incrementing head in __io_kbuf_recycle. And rectify one comemnt about kbuf consuming in iowq case.

---
 io_uring/kbuf.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
io_uring/kbuf.c +14 -6
index 9cdbc018fd64..37f06456bf30 100644
@@ -50,6 +50,13 @@ void __io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)
 	if (req->flags & REQ_F_BUFFER_RING) {
 		if (req->buf_list) {
 			if (req->flags & REQ_F_PARTIAL_IO) {
+				/*
+				 * if we reach here, uring_lock has been
+				¦* holden. Because in iowq, we already
+				¦* cleared req->buf_list to NULL when got
+				¦* the buffer from the ring, which means
+				¦* we cannot be here in that case.
+				 */
 				req->buf_list->head++;
 				req->buf_list = NULL;
 			} else {
@@ -128,12 +135,13 @@ static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 	if (issue_flags & IO_URING_F_UNLOCKED) {
 		/*
 		 * If we came in unlocked, we have no choice but to consume the
-		 * buffer here. This does mean it'll be pinned until the IO
-		 * completes. But coming in unlocked means we're in io-wq
-		 * context, hence there should be no further retry. For the
-		 * locked case, the caller must ensure to call the commit when
-		 * the transfer completes (or if we get -EAGAIN and must poll
-		 * or retry).
+		 * buffer here otherwise nothing ensures the buffer not being
+		 * used by others. This does mean it'll be pinned until the IO
+		 * completes though coming in unlocked means we're in io-wq
+		 * context and there may be further retries in async hybrid mode.
+		 * For the locked case, the caller must ensure to call the commit
+		 * when the transfer completes (or if we get -EAGAIN and must
+		 * poll or retry).
 		 */
 		req->buf_list = NULL;
 		bl->head++;
Signature · 1 line
2.25.1
author
#1 lore raw

Ping this one..

Quoted 49 lines · Hao Xu

From: Hao Xu <[email protected]>

Add comments to explain why it is always under uring lock when incrementing head in __io_kbuf_recycle. And rectify one comemnt about kbuf consuming in iowq case.

---
  io_uring/kbuf.c | 20 ++++++++++++++------
  1 file changed, 14 insertions(+), 6 deletions(-)
io_uring/kbuf.c +14 -6
index 9cdbc018fd64..37f06456bf30 100644
@@ -50,6 +50,13 @@ void __io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)
  	if (req->flags & REQ_F_BUFFER_RING) {
  		if (req->buf_list) {
  			if (req->flags & REQ_F_PARTIAL_IO) {
+				/*
+				 * if we reach here, uring_lock has been
+				¦* holden. Because in iowq, we already
+				¦* cleared req->buf_list to NULL when got
+				¦* the buffer from the ring, which means
+				¦* we cannot be here in that case.
+				 */
  				req->buf_list->head++;
  				req->buf_list = NULL;
  			} else {
@@ -128,12 +135,13 @@ static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len,
  	if (issue_flags & IO_URING_F_UNLOCKED) {
  		/*
  		 * If we came in unlocked, we have no choice but to consume the
-		 * buffer here. This does mean it'll be pinned until the IO
-		 * completes. But coming in unlocked means we're in io-wq
-		 * context, hence there should be no further retry. For the
-		 * locked case, the caller must ensure to call the commit when
-		 * the transfer completes (or if we get -EAGAIN and must poll
-		 * or retry).
+		 * buffer here otherwise nothing ensures the buffer not being
+		 * used by others. This does mean it'll be pinned until the IO
+		 * completes though coming in unlocked means we're in io-wq
+		 * context and there may be further retries in async hybrid mode.
+		 * For the locked case, the caller must ensure to call the commit
+		 * when the transfer completes (or if we get -EAGAIN and must
+		 * poll or retry).
  		 */
  		req->buf_list = NULL;
  		bl->head++;
maintainer
#2 lore raw
In reply to Hao Xu
Quoted 5 lines · Hao Xu

From: Hao Xu <[email protected]>

Add comments to explain why it is always under uring lock when incrementing head in __io_kbuf_recycle. And rectify one comemnt about kbuf consuming in iowq case.

Was there a 1/2 patch in this series? This one has a subject of 2/2...

Quoted 10 lines
---
 io_uring/kbuf.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
io_uring/kbuf.c +0 -0
index 9cdbc018fd64..37f06456bf30 100644

io_uring/kbuf.c void __io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags) · line 59

Quoted 10 lines
io_uring/kbuf.c +7 -0
@@ -50,6 +50,13 @@ void __io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)
 	if (req->flags & REQ_F_BUFFER_RING) {
 		if (req->buf_list) {
 			if (req->flags & REQ_F_PARTIAL_IO) {
+				/*
+				 * if we reach here, uring_lock has been
+				?* holden. Because in iowq, we already
+				?* cleared req->buf_list to NULL when got
+				?* the buffer from the ring, which means
+				?* we cannot be here in that case.
+				 */

There's a weird character before the '*' in most lines? I'd rephrase the above as:

If we end up here, then the io_uring_lock has been kept held since we retrieved the buffer. For the io-wq case, we already cleared req->buf_list when the buffer was retrieved, hence it cannot be set here for that case.

And make sure it lines up around 80 chars, your lines look very short.

Quoted 7 lines
io_uring/kbuf.c +0 -3
@@ -128,12 +135,13 @@ static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 	if (issue_flags & IO_URING_F_UNLOCKED) {
 		/*
 		 * If we came in unlocked, we have no choice but to consume the
-		 * buffer here. This does mean it'll be pinned until the IO
-		 * completes. But coming in unlocked means we're in io-wq
-		 * context, hence there should be no further retry. For the

io_uring/kbuf.c static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len, · line 144

Quoted 10 lines
io_uring/kbuf.c +7 -3
-		 * locked case, the caller must ensure to call the commit when
-		 * the transfer completes (or if we get -EAGAIN and must poll
-		 * or retry).
+		 * buffer here otherwise nothing ensures the buffer not being
+		 * used by others. This does mean it'll be pinned until the IO
+		 * completes though coming in unlocked means we're in io-wq
+		 * context and there may be further retries in async hybrid mode.
+		 * For the locked case, the caller must ensure to call the commit
+		 * when the transfer completes (or if we get -EAGAIN and must
+		 * poll or retry).

and similarly:

buffer here, otherwise nothing ensures that the buffer won't get used by others. This does mean it'll be pinned until the IO completes, coming in unlocked means we're being called from io-wq context and there may be further retries in async hybrid mode. For the locked case, the caller must call commit when the transfer completes (or if we get -EAGAIN and must poll of retry).

Signature · 1 line
Jens Axboe
author
#3 lore raw
Quoted 8 lines · Jens Axboe
Quoted 5 lines · Hao Xu

From: Hao Xu <[email protected]>

Add comments to explain why it is always under uring lock when incrementing head in __io_kbuf_recycle. And rectify one comemnt about kbuf consuming in iowq case.

Was there a 1/2 patch in this series? This one has a subject of 2/2...

Apologize for this, 1/2 and this should be separate, and I'm not going to send 1/2 for now for some reason. I should have change the subject.

Quoted 25 lines
Quoted 11 lines
---
  io_uring/kbuf.c | 20 ++++++++++++++------
  1 file changed, 14 insertions(+), 6 deletions(-)
io_uring/kbuf.c +0 -0
index 9cdbc018fd64..37f06456bf30 100644

io_uring/kbuf.c void __io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags) · line 59

Quoted 10 lines
io_uring/kbuf.c +7 -0
@@ -50,6 +50,13 @@ void __io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)
  	if (req->flags & REQ_F_BUFFER_RING) {
  		if (req->buf_list) {
  			if (req->flags & REQ_F_PARTIAL_IO) {
+				/*
+				 * if we reach here, uring_lock has been
+				?* holden. Because in iowq, we already
+				?* cleared req->buf_list to NULL when got
+				?* the buffer from the ring, which means
+				?* we cannot be here in that case.
+				 */

There's a weird character before the '*' in most lines? I'd rephrase the above as:

Quoted 6 lines

If we end up here, then the io_uring_lock has been kept held since we retrieved the buffer. For the io-wq case, we already cleared req->buf_list when the buffer was retrieved, hence it cannot be set here for that case.

And make sure it lines up around 80 chars, your lines look very short.

I'll do the change, as well as figuring out the weird char stuff. Thanks.

Quoted 21 lines
Quoted 8 lines
io_uring/kbuf.c +0 -3
@@ -128,12 +135,13 @@ static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len,
  	if (issue_flags & IO_URING_F_UNLOCKED) {
  		/*
  		 * If we came in unlocked, we have no choice but to consume the
-		 * buffer here. This does mean it'll be pinned until the IO
-		 * completes. But coming in unlocked means we're in io-wq
-		 * context, hence there should be no further retry. For the

io_uring/kbuf.c static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len, · line 144

Quoted 10 lines
io_uring/kbuf.c +7 -3
-		 * locked case, the caller must ensure to call the commit when
-		 * the transfer completes (or if we get -EAGAIN and must poll
-		 * or retry).
+		 * buffer here otherwise nothing ensures the buffer not being
+		 * used by others. This does mean it'll be pinned until the IO
+		 * completes though coming in unlocked means we're in io-wq
+		 * context and there may be further retries in async hybrid mode.
+		 * For the locked case, the caller must ensure to call the commit
+		 * when the transfer completes (or if we get -EAGAIN and must
+		 * poll or retry).

and similarly:

Quoted 7 lines

buffer here, otherwise nothing ensures that the buffer won't get used by others. This does mean it'll be pinned until the IO completes, coming in unlocked means we're being called from io-wq context and there may be further retries in async hybrid mode. For the locked case, the caller must call commit when the transfer completes (or if we get -EAGAIN and must poll of retry).

Gotcha.