Skip to contentSkip to conversation

io_uring: mshot read fix for buffer size changes

PATCH Superseded Dylan Yudaken 2 participants 11 messages Nov 5, 2023 – Nov 6, 2023 io_uring

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover io_uring: mshot read fix for buffer size changes

Hi,

This series fixes a bug (will send a liburing patch separately showing it) where the used buffer size is clamped to the minimum of all the previous buffers selected.

It also as part of this forces the multishot read API to set addr & len to 0.
len should probably have some accounting post-processing if it has meaning to set it to non-zero, but I think for a new API it is simpler to overly-constrain it upfront?

addr is useful to force to zero as it will allow some more bits to be used in struct io_rw, which is otherwise full.

Dylan Yudaken (2):
io_uring: do not allow multishot read to set addr or len
io_uring: do not clamp read length for multishot read

 io_uring/rw.c | 9 +++++++++
 1 file changed, 9 insertions(+)
Signature · 1 line
2.41.0
#1 lore raw

PATCH 1/2 io_uring: do not allow multishot read to set addr or len

For addr: this field is not used, since buffer select is forced. But by forcing it to be zero it leaves open future uses of the field.

len is actually usable, you could imagine that you want to receive multishot up to a certain length.
However right now this is not how it is implemented, and it seems safer to force this to be zero.

Fixes fc68fcda0491 ("io_uring/rw: add support for IORING_OP_READ_MULTISHOT")
---
 io_uring/rw.c | 7 +++++++
 1 file changed, 7 insertions(+)
io_uring/rw.c +7 -0
index 1c76de483ef6..ea86498d8769 100644
@@ -111,6 +111,13 @@ int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	rw->len = READ_ONCE(sqe->len);
 	rw->flags = READ_ONCE(sqe->rw_flags);
 
+	if (req->opcode == IORING_OP_READ_MULTISHOT) {
+		if (rw->addr)
+			return -EINVAL;
+		if (rw->len)
+			return -EINVAL;
+	}
+
 	/* Have to do this validation here, as this is in io_read() rw->len might
 	 * have chanaged due to buffer selection
 	 */
Signature · 1 line
2.41.0
maintainer
#2 lore raw

Re: [PATCH 1/2] io_uring: do not allow multishot read to set addr or len

Quoted 19 lines · Dylan Yudaken

For addr: this field is not used, since buffer select is forced. But by forcing it to be zero it leaves open future uses of the field.

len is actually usable, you could imagine that you want to receive multishot up to a certain length.
However right now this is not how it is implemented, and it seems safer to force this to be zero.

Fixes fc68fcda0491 ("io_uring/rw: add support for IORING_OP_READ_MULTISHOT")
---
 io_uring/rw.c | 7 +++++++
 1 file changed, 7 insertions(+)
io_uring/rw.c +0 -0
index 1c76de483ef6..ea86498d8769 100644

io_uring/rw.c int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 119

Quoted 9 lines
io_uring/rw.c +6 -0
@@ -111,6 +111,13 @@ int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	rw->len = READ_ONCE(sqe->len);
 	rw->flags = READ_ONCE(sqe->rw_flags);
 
+	if (req->opcode == IORING_OP_READ_MULTISHOT) {
+		if (rw->addr)
+			return -EINVAL;
+		if (rw->len)
+			return -EINVAL;
+	}

Should we just put these in io_read_mshot_prep() instead? Ala the below. In general I think it'd be nice to have a core prep_rw, and then each variant will have its own prep. Then we can get away from random opcode checking in there.

I do agree with the change in general, just think we can tweak it a bit to make it a bit cleaner.

io_uring/rw.c +4 -0
index 1c76de483ef6..635a1bf5df70 100644
@@ -129,6 +129,7 @@ int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
  */
 int io_read_mshot_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
+	struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);
 	int ret;
 
 	/* must be used with provided buffers */
@@ -139,6 +140,9 @@ int io_read_mshot_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	if (unlikely(ret))
 		return ret;
 
+	if (rw->addr || rw->len)
+		return -EINVAL;
+
 	req->flags |= REQ_F_APOLL_MULTISHOT;
 	return 0;
 }
Signature · 1 line
Jens Axboe
maintainer
#3 lore raw

Re: [PATCH 1/2] io_uring: do not allow multishot read to set addr or len

Quoted 35 lines · Jens Axboe
Quoted 19 lines · Dylan Yudaken

For addr: this field is not used, since buffer select is forced. But by forcing it to be zero it leaves open future uses of the field.

len is actually usable, you could imagine that you want to receive multishot up to a certain length.
However right now this is not how it is implemented, and it seems safer to force this to be zero.

Fixes fc68fcda0491 ("io_uring/rw: add support for IORING_OP_READ_MULTISHOT")
---
 io_uring/rw.c | 7 +++++++
 1 file changed, 7 insertions(+)
io_uring/rw.c +0 -0
index 1c76de483ef6..ea86498d8769 100644

io_uring/rw.c int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 119

Quoted 9 lines
io_uring/rw.c +6 -0
@@ -111,6 +111,13 @@ int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	rw->len = READ_ONCE(sqe->len);
 	rw->flags = READ_ONCE(sqe->rw_flags);
 
+	if (req->opcode == IORING_OP_READ_MULTISHOT) {
+		if (rw->addr)
+			return -EINVAL;
+		if (rw->len)
+			return -EINVAL;
+	}

Should we just put these in io_read_mshot_prep() instead? Ala the below. In general I think it'd be nice to have a core prep_rw, and then each variant will have its own prep. Then we can get away from random opcode checking in there.

Quoted 2 lines

I do agree with the change in general, just think we can tweak it a bit to make it a bit cleaner.

Sent out two cleanups that take it in this direction in general, fwiw.

Signature · 1 line
Jens Axboe
#4 lore raw

Re: [PATCH 1/2] io_uring: do not allow multishot read to set addr or len

Quoted 37 lines · Jens Axboe
Quoted 35 lines · Jens Axboe
Quoted 19 lines · Dylan Yudaken

For addr: this field is not used, since buffer select is forced. But by forcing it to be zero it leaves open future uses of the field.

len is actually usable, you could imagine that you want to receive multishot up to a certain length.
However right now this is not how it is implemented, and it seems safer to force this to be zero.

Fixes fc68fcda0491 ("io_uring/rw: add support for IORING_OP_READ_MULTISHOT")
---
 io_uring/rw.c | 7 +++++++
 1 file changed, 7 insertions(+)
io_uring/rw.c +0 -0
index 1c76de483ef6..ea86498d8769 100644

io_uring/rw.c int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 119

Quoted 9 lines
io_uring/rw.c +6 -0
@@ -111,6 +111,13 @@ int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
     rw->len = READ_ONCE(sqe->len);
     rw->flags = READ_ONCE(sqe->rw_flags);
 
+    if (req->opcode == IORING_OP_READ_MULTISHOT) {
+            if (rw->addr)
+                    return -EINVAL;
+            if (rw->len)
+                    return -EINVAL;
+    }

Should we just put these in io_read_mshot_prep() instead? Ala the below. In general I think it'd be nice to have a core prep_rw, and then each variant will have its own prep. Then we can get away from random opcode checking in there.

Quoted 4 lines
Quoted 2 lines

I do agree with the change in general, just think we can tweak it a bit to make it a bit cleaner.

Sent out two cleanups that take it in this direction in general, fwiw.

Yes - I think this approach is better, will rebase on these

#5 lore raw

PATCH 2/2 io_uring: do not clamp read length for multishot read

In reply to Dylan Yudaken

When doing a multishot read, the code path reuses the old read paths. However this breaks an assumption built into those paths, namely that struct io_rw::len is available for reuse by __io_import_iovec.

For multishot this results in len being set for the first receive call, and then subsequent calls are clamped to that buffer length incorrectly.

Fixes fc68fcda0491 ("io_uring/rw: add support for IORING_OP_READ_MULTISHOT")
---
 io_uring/rw.c | 2 ++
 1 file changed, 2 insertions(+)
io_uring/rw.c +2 -0
index ea86498d8769..b7f7fbc28032 100644
@@ -417,6 +417,8 @@ static struct iovec *__io_import_iovec(int ddir, struct io_kiocb *req,
 
 	if (!io_issue_defs[opcode].vectored || req->flags & REQ_F_BUFFER_SELECT) {
 		if (io_do_buffer_select(req)) {
+			if (opcode == IORING_OP_READ_MULTISHOT)
+				sqe_len = 0;
 			buf = io_buffer_select(req, &sqe_len, issue_flags);
 			if (!buf)
 				return ERR_PTR(-ENOBUFS);
Signature · 1 line
2.41.0
maintainer
#6 lore raw

Re: [PATCH 2/2] io_uring: do not clamp read length for multishot read

Quoted 6 lines · Dylan Yudaken

When doing a multishot read, the code path reuses the old read paths. However this breaks an assumption built into those paths, namely that struct io_rw::len is available for reuse by __io_import_iovec.

For multishot this results in len being set for the first receive call, and then subsequent calls are clamped to that buffer length incorrectly.

Should we just reset this to 0 always in io_read_mshot()? And preferably with a comment added as well as to why that is necessary to avoid repeated clamping.

Signature · 1 line
Jens Axboe
#7 lore raw

Re: [PATCH 2/2] io_uring: do not clamp read length for multishot read

Quoted 6 lines · Jens Axboe
Quoted 4 lines · Dylan Yudaken

When doing a multishot read, the code path reuses the old read paths. However this breaks an assumption built into those paths, namely that struct io_rw::len is available for reuse by __io_import_iovec.

Quoted 6 lines
Quoted 2 lines

For multishot this results in len being set for the first receive call, and then subsequent calls are clamped to that buffer length incorrectly.

Should we just reset this to 0 always in io_read_mshot()? And preferably with a comment added as well as to why that is necessary to avoid repeated clamping.

Unfortunately I don't think (without testing) that will work. Sometimes the request
comes into io_read_mshot with the buffer already selected, and the length cannot
be touched in that case.

We could check if the buffer is set, and if not clear the length I guess. I'm a bit unsure which is better - both seem equally ugly to be honest.

maintainer
#8 lore raw

Re: [PATCH 2/2] io_uring: do not clamp read length for multishot read

Quoted 14 lines · Dylan Yudaken
Quoted 6 lines · On Mon, Nov 6, 2023 at 2:46?PM Jens Axboe <[email protected]> wrote:
Quoted 4 lines · Dylan Yudaken

When doing a multishot read, the code path reuses the old read paths. However this breaks an assumption built into those paths, namely that struct io_rw::len is available for reuse by __io_import_iovec.

Quoted 6 lines
Quoted 2 lines

For multishot this results in len being set for the first receive call, and then subsequent calls are clamped to that buffer length incorrectly.

Should we just reset this to 0 always in io_read_mshot()? And preferably with a comment added as well as to why that is necessary to avoid repeated clamping.

Quoted 8 lines

Unfortunately I don't think (without testing) that will work. Sometimes the request
comes into io_read_mshot with the buffer already selected, and the length cannot
be touched in that case.

We could check if the buffer is set, and if not clear the length I guess. I'm a bit unsure which is better - both seem equally ugly to be honest.

I mean do it at the end when we complete it, so it's reset for the next iteration. But yeah, I'd want to have the test case verify this first :-)

Signature · 1 line
Jens Axboe
maintainer
#9 lore raw

Re: [PATCH 2/2] io_uring: do not clamp read length for multishot read

Quoted 21 lines · Jens Axboe
Quoted 20 lines · Dylan Yudaken
Quoted 6 lines · On Mon, Nov 6, 2023 at 2:46?PM Jens Axboe <[email protected]> wrote:
Quoted 4 lines · Dylan Yudaken

When doing a multishot read, the code path reuses the old read paths. However this breaks an assumption built into those paths, namely that struct io_rw::len is available for reuse by __io_import_iovec.

Quoted 6 lines
Quoted 2 lines

For multishot this results in len being set for the first receive call, and then subsequent calls are clamped to that buffer length incorrectly.

Should we just reset this to 0 always in io_read_mshot()? And preferably with a comment added as well as to why that is necessary to avoid repeated clamping.

Unfortunately I don't think (without testing) that will work. Sometimes the request
comes into io_read_mshot with the buffer already selected, and the length cannot
be touched in that case.

Quoted 6 lines
Quoted 2 lines

We could check if the buffer is set, and if not clear the length I guess. I'm a bit unsure which is better - both seem equally ugly to be honest.

I mean do it at the end when we complete it, so it's reset for the next iteration. But yeah, I'd want to have the test case verify this first :-)

Something ala the below?

io_uring/rw.c +6 -0
index 9e3e56b74e35..9121832eadec 100644
@@ -932,6 +932,12 @@ int io_read_mshot(struct io_kiocb *req, unsigned int issue_flags)
 	 * Any successful return value will keep the multishot read armed.
 	 */
 	if (ret > 0) {
+		/*
+		 * Reset rw->len to 0 again to avoid clamping future mshot
+		 * reads, in case the buffer size varies.
+		 */
+		io_kiocb_to_cmd(req, struct io_rw)->len = 0;
+
 		/*
 		 * Put our buffer and post a CQE. If we fail to post a CQE, then
 		 * jump to the termination path. This request is then done.
Signature · 1 line
Jens Axboe
maintainer
#10 lore raw
In reply to Dylan Yudaken
Quoted 5 lines · Dylan Yudaken

Hi,

This series fixes a bug (will send a liburing patch separately showing it) where the used buffer size is clamped to the minimum of all the previous buffers selected.

Oh, if you have time and once we get v2 in, maybe send a patch for liburing as well to remove 'nbytes' from io_uring_prep_read_multishot() as well?

Signature · 1 line
Jens Axboe