Skip to contentSkip to conversation

io_uring memory accounting fixes

PATCH In review Jens Axboe 2 participants 9 messages Aug 5, 2020 – Aug 6, 2020

Subsystems (3)
lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

cover io_uring memory accounting fixes

First one is something that should go into stable, to avoid discrepancies between accounted and unaccounted memory. The second one is just for 5.9.

Signature · 1 line
Jens Axboe
author maintainer
#1 lore raw

PATCH 1/2 io_uring: set ctx sq/cq entry count earlier

If we hit an earlier error path in io_uring_create(), then we will have accounted memory, but not set ctx->{sq,cq}_entries yet. Then when the ring is torn down in error, we use those values to unaccount the memory.

Ensure we set the ctx entries before we're able to hit a potential error path.

---
 fs/io_uring.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
fs/io_uring.c +4 -2
index 8f96566603f3..0d857f7ca507 100644
@@ -8193,6 +8193,10 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 	struct io_rings *rings;
 	size_t size, sq_array_offset;
 
+	/* make sure these are sane, as we already accounted them */
+	ctx->sq_entries = p->sq_entries;
+	ctx->cq_entries = p->cq_entries;
+
 	size = rings_size(p->sq_entries, p->cq_entries, &sq_array_offset);
 	if (size == SIZE_MAX)
 		return -EOVERFLOW;
@@ -8209,8 +8213,6 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 	rings->cq_ring_entries = p->cq_entries;
 	ctx->sq_mask = rings->sq_ring_mask;
 	ctx->cq_mask = rings->cq_ring_mask;
-	ctx->sq_entries = rings->sq_ring_entries;
-	ctx->cq_entries = rings->cq_ring_entries;
 
 	size = array_size(sizeof(struct io_uring_sqe), p->sq_entries);
 	if (size == SIZE_MAX) {
Signature · 1 line
2.28.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] io_uring: set ctx sq/cq entry count earlier

Quoted 40 lines · Jens Axboe

If we hit an earlier error path in io_uring_create(), then we will have accounted memory, but not set ctx->{sq,cq}_entries yet. Then when the ring is torn down in error, we use those values to unaccount the memory.

Ensure we set the ctx entries before we're able to hit a potential error path.

---
 fs/io_uring.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
fs/io_uring.c +4 -2
index 8f96566603f3..0d857f7ca507 100644
@@ -8193,6 +8193,10 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 	struct io_rings *rings;
 	size_t size, sq_array_offset;
 
+	/* make sure these are sane, as we already accounted them */
+	ctx->sq_entries = p->sq_entries;
+	ctx->cq_entries = p->cq_entries;
+
 	size = rings_size(p->sq_entries, p->cq_entries, &sq_array_offset);
 	if (size == SIZE_MAX)
 		return -EOVERFLOW;
@@ -8209,8 +8213,6 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 	rings->cq_ring_entries = p->cq_entries;
 	ctx->sq_mask = rings->sq_ring_mask;
 	ctx->cq_mask = rings->cq_ring_mask;
-	ctx->sq_entries = rings->sq_ring_entries;
-	ctx->cq_entries = rings->cq_ring_entries;
 
 	size = array_size(sizeof(struct io_uring_sqe), p->sq_entries);
 	if (size == SIZE_MAX) {
Signature · 1 line
2.28.0

While reviewing I was asking if we should move io_account_mem() before io_allocate_scq_urings(), then I saw the second patch :-)

LGTM:

Thanks,
Stefano

author maintainer
#3 lore raw

Re: [PATCH 1/2] io_uring: set ctx sq/cq entry count earlier

Quoted 36 lines · Stefano Garzarella
Quoted 35 lines · Jens Axboe

If we hit an earlier error path in io_uring_create(), then we will have accounted memory, but not set ctx->{sq,cq}_entries yet. Then when the ring is torn down in error, we use those values to unaccount the memory.

Ensure we set the ctx entries before we're able to hit a potential error path.

---
 fs/io_uring.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
fs/io_uring.c +4 -2
index 8f96566603f3..0d857f7ca507 100644
@@ -8193,6 +8193,10 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 	struct io_rings *rings;
 	size_t size, sq_array_offset;
 
+	/* make sure these are sane, as we already accounted them */
+	ctx->sq_entries = p->sq_entries;
+	ctx->cq_entries = p->cq_entries;
+
 	size = rings_size(p->sq_entries, p->cq_entries, &sq_array_offset);
 	if (size == SIZE_MAX)
 		return -EOVERFLOW;
@@ -8209,8 +8213,6 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 	rings->cq_ring_entries = p->cq_entries;
 	ctx->sq_mask = rings->sq_ring_mask;
 	ctx->cq_mask = rings->cq_ring_mask;
-	ctx->sq_entries = rings->sq_ring_entries;
-	ctx->cq_entries = rings->cq_ring_entries;
 
Quoted 8 lines
Quoted 5 lines
fs/io_uring.c +0 -0
 	size = array_size(sizeof(struct io_uring_sqe), p->sq_entries);
 	if (size == SIZE_MAX) {
Signature · 1 line
2.28.0

While reviewing I was asking if we should move io_account_mem() before io_allocate_scq_urings(), then I saw the second patch :-)

Indeed, just split it in two to avoid any extra issues around backporting.

Quoted 1 line

Thanks, added.

Signature · 1 line
Jens Axboe
author maintainer
#4 lore raw

PATCH 2/2 io_uring: account locked memory before potential error case

In reply to Jens Axboe

The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.

---
 fs/io_uring.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
fs/io_uring.c +8 -8
index 0d857f7ca507..7c42f63fbb0a 100644
@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 	ctx->user = user;
 	ctx->creds = get_current_cred();
 
+	/*
+	 * Account memory _before_ installing the file descriptor. Once
+	 * the descriptor is installed, it can get closed at any time.
+	 */
+	io_account_mem(ctx, ring_pages(p->sq_entries, p->cq_entries),
+		       ACCT_LOCKED);
+	ctx->limit_mem = limit_mem;
+
 	ret = io_allocate_scq_urings(ctx, p);
 	if (ret)
 		goto err;
@@ -8377,14 +8385,6 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 		goto err;
 	}
 
-	/*
-	 * Account memory _before_ installing the file descriptor. Once
-	 * the descriptor is installed, it can get closed at any time.
-	 */
-	io_account_mem(ctx, ring_pages(p->sq_entries, p->cq_entries),
-		       ACCT_LOCKED);
-	ctx->limit_mem = limit_mem;
-
 	/*
 	 * Install ring fd as the very last thing, so we don't risk someone
 	 * having closed it before we finish setup
Signature · 1 line
2.28.0
#5 lore raw

Re: [PATCH 2/2] io_uring: account locked memory before potential error case

Quoted 13 lines · Jens Axboe

The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.

---
 fs/io_uring.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
fs/io_uring.c +0 -0
index 0d857f7ca507..7c42f63fbb0a 100644

fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347

Quoted 7 lines
fs/io_uring.c +4 -0
@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 	ctx->user = user;
 	ctx->creds = get_current_cred();
 
+	/*
+	 * Account memory _before_ installing the file descriptor. Once
+	 * the descriptor is installed, it can get closed at any time.
+	 */

What about update a bit the comment?
Maybe adding the commit description in this comment.

Quoted 25 lines
fs/io_uring.c +4 -8
+	io_account_mem(ctx, ring_pages(p->sq_entries, p->cq_entries),
+		       ACCT_LOCKED);
+	ctx->limit_mem = limit_mem;
+
 	ret = io_allocate_scq_urings(ctx, p);
 	if (ret)
 		goto err;
@@ -8377,14 +8385,6 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 		goto err;
 	}
 
-	/*
-	 * Account memory _before_ installing the file descriptor. Once
-	 * the descriptor is installed, it can get closed at any time.
-	 */
-	io_account_mem(ctx, ring_pages(p->sq_entries, p->cq_entries),
-		       ACCT_LOCKED);
-	ctx->limit_mem = limit_mem;
-
 	/*
 	 * Install ring fd as the very last thing, so we don't risk someone
 	 * having closed it before we finish setup
Signature · 1 line
2.28.0

Thanks,
Stefano

author maintainer
#6 lore raw

Re: [PATCH 2/2] io_uring: account locked memory before potential error case

Quoted 17 lines · Stefano Garzarella
Quoted 16 lines · Jens Axboe

The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.

---
 fs/io_uring.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
fs/io_uring.c +0 -0
index 0d857f7ca507..7c42f63fbb0a 100644
@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 	ctx->user = user;
 	ctx->creds = get_current_cred();
 
Quoted 7 lines

fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347

Quoted 4 lines
fs/io_uring.c +4 -0
+	/*
+	 * Account memory _before_ installing the file descriptor. Once
+	 * the descriptor is installed, it can get closed at any time.
+	 */

What about update a bit the comment?
Maybe adding the commit description in this comment.

I updated the comment:

/*
* Account memory _before_ installing the file descriptor. Once
* the descriptor is installed, it can get closed at any time. Also
* do this before hitting the general error path, as ring freeing
* will un-account as well.
*/

Signature · 1 line
Jens Axboe
Reviewed-by
#7 lore raw

Re: [PATCH 2/2] io_uring: account locked memory before potential error case

Quoted 26 lines · Jens Axboe
Quoted 17 lines · Stefano Garzarella
Quoted 16 lines · Jens Axboe

The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.

---
 fs/io_uring.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
fs/io_uring.c +0 -0
index 0d857f7ca507..7c42f63fbb0a 100644
@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 	ctx->user = user;
 	ctx->creds = get_current_cred();
 
Quoted 7 lines

fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347

Quoted 4 lines
fs/io_uring.c +4 -0
+	/*
+	 * Account memory _before_ installing the file descriptor. Once
+	 * the descriptor is installed, it can get closed at any time.
+	 */

What about update a bit the comment?
Maybe adding the commit description in this comment.

Quoted 8 lines

I updated the comment:

/*
* Account memory _before_ installing the file descriptor. Once
* the descriptor is installed, it can get closed at any time. Also
* do this before hitting the general error path, as ring freeing
* will un-account as well.
*/

Now it looks better!

Thanks,
Stefano

author maintainer
#8 lore raw

Re: [PATCH 2/2] io_uring: account locked memory before potential error case

Quoted 36 lines · Stefano Garzarella
Quoted 26 lines · Jens Axboe
Quoted 17 lines · Stefano Garzarella
Quoted 16 lines · Jens Axboe

The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.

---
 fs/io_uring.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
fs/io_uring.c +0 -0
index 0d857f7ca507..7c42f63fbb0a 100644
@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p,
 	ctx->user = user;
 	ctx->creds = get_current_cred();
 
Quoted 7 lines

fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347

Quoted 4 lines
fs/io_uring.c +4 -0
+	/*
+	 * Account memory _before_ installing the file descriptor. Once
+	 * the descriptor is installed, it can get closed at any time.
+	 */

What about update a bit the comment?
Maybe adding the commit description in this comment.

Quoted 8 lines

I updated the comment:

/*
* Account memory _before_ installing the file descriptor. Once
* the descriptor is installed, it can get closed at any time. Also
* do this before hitting the general error path, as ring freeing
* will un-account as well.
*/

Quoted 3 lines

Now it looks better!

Thanks, added.

Signature · 1 line
Jens Axboe