Skip to contentSkip to conversation

io_uring iter_revert issues

PATCH In review Pavel Begunkov 2 participants 4 messages Jun 3, 2022

Subsystems (2)
Stable branch VFS
lore
Share & tools
lore

Pavel Begunkov

author
#0 lore raw

cover io_uring iter_revert issues

5.10 fixup for 89c2b3b7491820 ("io_uring: reexpand under-reexpanded iters"). We can't just directly cherry-pick them as the code base is quite different, so we also need patch 1/2. Previous attempts to backport 2/2 directly were pulling in too many dependencies only adding more problems.

Pavel Begunkov (2):
io_uring: don't re-import iovecs from callbacks
io_uring: fix using under-expanded iters

 fs/io_uring.c | 47 ++++++-----------------------------------------
 1 file changed, 6 insertions(+), 41 deletions(-)
Signature · 1 line
2.36.1
author
#1 lore raw

PATCH 1/2 io_uring: don't re-import iovecs from callbacks

We can't re-import or modify iterators from iocb callbacks, it's not safe as it might be reverted and/or reexpanded while unwinding stack. It's also not safe to resubmit as io-wq thread will race with stack undwinding for the iterator and other data.

Disallow resubmission from callbacks, it can fail some cases that were handled before, but the possibility of such a failure was a part of the API from the beginning and so it should be fine.

Signed-off-by Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 39 ---------------------------------------
 1 file changed, 39 deletions(-)
fs/io_uring.c +0 -39
index 4330603eae35..aded83f20a15 100644
@@ -2579,45 +2579,6 @@ static void io_complete_rw_common(struct kiocb *kiocb, long res,
 #ifdef CONFIG_BLOCK
 static bool io_resubmit_prep(struct io_kiocb *req, int error)
 {
-	struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs;
-	ssize_t ret = -ECANCELED;
-	struct iov_iter iter;
-	int rw;
-
-	if (error) {
-		ret = error;
-		goto end_req;
-	}
-
-	switch (req->opcode) {
-	case IORING_OP_READV:
-	case IORING_OP_READ_FIXED:
-	case IORING_OP_READ:
-		rw = READ;
-		break;
-	case IORING_OP_WRITEV:
-	case IORING_OP_WRITE_FIXED:
-	case IORING_OP_WRITE:
-		rw = WRITE;
-		break;
-	default:
-		printk_once(KERN_WARNING "io_uring: bad opcode in resubmit %d\n",
-				req->opcode);
-		goto end_req;
-	}
-
-	if (!req->async_data) {
-		ret = io_import_iovec(rw, req, &iovec, &iter, false);
-		if (ret < 0)
-			goto end_req;
-		ret = io_setup_async_rw(req, iovec, inline_vecs, &iter, false);
-		if (!ret)
-			return true;
-		kfree(iovec);
-	} else {
-		return true;
-	}
-end_req:
 	req_set_fail_links(req);
 	return false;
 }
Signature · 1 line
2.36.1
author
#2 lore raw

PATCH 2/2 io_uring: fix using under-expanded iters

In reply to Pavel Begunkov

[ upstream commit cd65869512ab5668a5d16f789bc4da1319c435c4 ]

The issue was first described and addressed in 89c2b3b7491820 ("io_uring: reexpand under-reexpanded iters"), but shortly after reimplemented as.
cd65869512ab56 ("io_uring: use iov_iter state save/restore helpers").

Here we follow the approach from the second patch but without in-callback resubmissions, fixups for not yet supported in 5.10 short read retries and replacing iov_iter_state with iter copies to not pull even more dependencies, and because it's just much simpler.

Signed-off-by Pavel Begunkov <[email protected]>
---
 fs/io_uring.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
fs/io_uring.c +6 -2
index aded83f20a15..b2b5edee1512 100644
@@ -3389,6 +3389,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock,
 	struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs;
 	struct kiocb *kiocb = &req->rw.kiocb;
 	struct iov_iter __iter, *iter = &__iter;
+	struct iov_iter iter_cp;
 	struct io_async_rw *rw = req->async_data;
 	ssize_t io_size, ret, ret2;
 	bool no_async;

@@ -3399,6 +3400,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock,
 	ret = io_import_iovec(READ, req, &iovec, iter, !force_nonblock);
 	if (ret < 0)
 		return ret;
+	iter_cp = *iter;
 	io_size = iov_iter_count(iter);
 	req->result = io_size;
 	ret = 0;

@@ -3434,7 +3436,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock,
 		if (req->file->f_flags & O_NONBLOCK)
 			goto done;
 		/* some cases will consume bytes even on error returns */
-		iov_iter_revert(iter, io_size - iov_iter_count(iter));
+		*iter = iter_cp;
 		ret = 0;
 		goto copy_iov;
 	} else if (ret < 0) {

@@ -3517,6 +3519,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock,
 	struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs;
 	struct kiocb *kiocb = &req->rw.kiocb;
 	struct iov_iter __iter, *iter = &__iter;
+	struct iov_iter iter_cp;
 	struct io_async_rw *rw = req->async_data;
 	ssize_t ret, ret2, io_size;
 

@@ -3526,6 +3529,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock,
 	ret = io_import_iovec(WRITE, req, &iovec, iter, !force_nonblock);
 	if (ret < 0)
 		return ret;
+	iter_cp = *iter;
 	io_size = iov_iter_count(iter);
 	req->result = io_size;
 

@@ -3587,7 +3591,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock,
 	} else {
 copy_iov:
 		/* some cases will consume bytes even on error returns */
-		iov_iter_revert(iter, io_size - iov_iter_count(iter));
+		*iter = iter_cp;
 		ret = io_setup_async_rw(req, iovec, inline_vecs, iter, false);
 		if (!ret)
 			return -EAGAIN;
Signature · 1 line
2.36.1
maintainer
#3 lore raw
In reply to Pavel Begunkov
Quoted 5 lines · Pavel Begunkov

5.10 fixup for 89c2b3b7491820 ("io_uring: reexpand under-reexpanded iters"). We can't just directly cherry-pick them as the code base is quite different, so we also need patch 1/2. Previous attempts to backport 2/2 directly were pulling in too many dependencies only adding more problems.

Quoted 10 lines

Pavel Begunkov (2):
io_uring: don't re-import iovecs from callbacks
io_uring: fix using under-expanded iters

 fs/io_uring.c | 47 ++++++-----------------------------------------
 1 file changed, 6 insertions(+), 41 deletions(-)
Signature · 1 line
2.36.1

All now queued up, thanks.

greg k-h