io_uring memory accounting fixes
Subsystems (3)
Jens Axboe
maintainerPATCH 1/2 io_uring: set ctx sq/cq entry count earlier
In reply to Jens AxboeIf we hit an earlier error path in io_uring_create(), then we will have accounted memory, but not set ctx->{sq,cq}_entries yet. Then when the ring is torn down in error, we use those values to unaccount the memory.
Ensure we set the ctx entries before we're able to hit a potential error path.
--- fs/io_uring.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/io_uring.c +4 -2
@@ -8193,6 +8193,10 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,struct io_rings *rings;size_t size, sq_array_offset; +/* make sure these are sane, as we already accounted them */ + ctx->sq_entries = p->sq_entries; + ctx->cq_entries = p->cq_entries; + size =rings_size (p->sq_entries, p->cq_entries, &sq_array_offset);if (size ==SIZE_MAX )return -EOVERFLOW ;
@@ -8209,8 +8213,6 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx, rings->cq_ring_entries = p->cq_entries; ctx->sq_mask = rings->sq_ring_mask; ctx->cq_mask = rings->cq_ring_mask; - ctx->sq_entries = rings->sq_ring_entries; - ctx->cq_entries = rings->cq_ring_entries; size =array_size (sizeof (struct io_uring_sqe ), p->sq_entries);if (size ==SIZE_MAX ) {
Signature · 1 line
2.28.0
Jens Axboe
maintainerPATCH 2/2 io_uring: account locked memory before potential error case
In reply to Jens AxboeThe tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.
--- fs/io_uring.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
fs/io_uring.c +8 -8
@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, ctx->user = user; ctx->creds =get_current_cred (); +/* +* Account memory _before_ installing the file descriptor. Once +* the descriptor is installed, it can get closed at any time. +*/ +io_account_mem (ctx,ring_pages (p->sq_entries, p->cq_entries), +ACCT_LOCKED ); + ctx->limit_mem = limit_mem; + ret =io_allocate_scq_urings (ctx, p);if (ret) goto err;
@@ -8377,14 +8385,6 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, goto err; } -/* -* Account memory _before_ installing the file descriptor. Once -* the descriptor is installed, it can get closed at any time. -*/ -io_account_mem (ctx,ring_pages (p->sq_entries, p->cq_entries), -ACCT_LOCKED ); - ctx->limit_mem = limit_mem; -/* * Install ring fd as the very last thing, so we don't risk someone * having closed it before we finish setup
Signature · 1 line
2.28.0
Jens Axboe
maintainercover io_uring memory accounting fixes
First one is something that should go into stable, to avoid discrepancies between accounted and unaccounted memory. The second one is just for 5.9.
Signature · 1 line
Jens Axboe
Re: [PATCH 1/2] io_uring: set ctx sq/cq entry count earlier
In reply to Jens AxboeQuoted 40 lines · Jens Axboe
If we hit an earlier error path in
io_uring_create(), then we will have accounted memory, but not set ctx->{sq,cq}_entries yet. Then when the ring is torn down in error, we use those values to unaccount the memory.Ensure we set the ctx entries before we're able to hit a potential error path.
Signed-off-by Jens Axboe <[email protected]>--- fs/io_uring.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)fs/io_uring.c +4
-2@@ -8193,6 +8193,10 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,struct io_rings *rings;size_t size, sq_array_offset; +/* make sure these are sane, as we already accounted them */ + ctx->sq_entries = p->sq_entries; + ctx->cq_entries = p->cq_entries; + size =rings_size (p->sq_entries, p->cq_entries, &sq_array_offset);if (size ==SIZE_MAX )return -EOVERFLOW ;@@ -8209,8 +8213,6 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx, rings->cq_ring_entries = p->cq_entries; ctx->sq_mask = rings->sq_ring_mask; ctx->cq_mask = rings->cq_ring_mask; - ctx->sq_entries = rings->sq_ring_entries; - ctx->cq_entries = rings->cq_ring_entries; size =array_size (sizeof (struct io_uring_sqe ), p->sq_entries);if (size ==SIZE_MAX ) {Signature · 1 line
2.28.0
While reviewing I was asking if we should move io_account_mem() before io_allocate_scq_urings(), then I saw the second patch :-)
LGTM:
Thanks,
Stefano
Re: [PATCH 2/2] io_uring: account locked memory before potential error case
In reply to Jens AxboeQuoted 13 lines · Jens Axboe
The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.
Signed-off-by Jens Axboe <[email protected]>--- fs/io_uring.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)fs/io_uring.c +0
-0
fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347
Quoted 7 lines
fs/io_uring.c +4
-0@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, ctx->user = user; ctx->creds =get_current_cred (); +/* +* Account memory _before_ installing the file descriptor. Once +* the descriptor is installed, it can get closed at any time. +*/
What about update a bit the comment?
Maybe adding the commit description in this comment.
Quoted 25 lines
fs/io_uring.c +4
-8+io_account_mem (ctx,ring_pages (p->sq_entries, p->cq_entries), +ACCT_LOCKED ); + ctx->limit_mem = limit_mem; + ret =io_allocate_scq_urings (ctx, p);if (ret) goto err;@@ -8377,14 +8385,6 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, goto err; } -/* -* Account memory _before_ installing the file descriptor. Once -* the descriptor is installed, it can get closed at any time. -*/ -io_account_mem (ctx,ring_pages (p->sq_entries, p->cq_entries), -ACCT_LOCKED ); - ctx->limit_mem = limit_mem; -/* * Install ring fd as the very last thing, so we don't risk someone * having closed it before we finish setup Signature · 1 line
2.28.0
Thanks,
Stefano
Jens Axboe
maintainerRe: [PATCH 2/2] io_uring: account locked memory before potential error case
Quoted 17 lines · Stefano Garzarella
Quoted 16 lines · Jens Axboe
The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.
Signed-off-by Jens Axboe <[email protected]>--- fs/io_uring.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)fs/io_uring.c +0
-0@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, ctx->user = user; ctx->creds =get_current_cred ();
Quoted 7 lines
fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347
Quoted 4 lines
fs/io_uring.c +4
-0+/* +* Account memory _before_ installing the file descriptor. Once +* the descriptor is installed, it can get closed at any time. +*/ What about update a bit the comment?
Maybe adding the commit description in this comment.
I updated the comment:
/*
* Account memory _before_ installing the file descriptor. Once
* the descriptor is installed, it can get closed at any time. Also
* do this before hitting the general error path, as ring freeing
* will un-account as well.
*/
Signature · 1 line
Jens Axboe
Stefano Garzarella
Reviewed-byRe: [PATCH 2/2] io_uring: account locked memory before potential error case
Quoted 26 lines · Jens Axboe
Quoted 17 lines · Stefano Garzarella
Quoted 16 lines · Jens Axboe
The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.
Signed-off-by Jens Axboe <[email protected]>--- fs/io_uring.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)fs/io_uring.c +0
-0@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, ctx->user = user; ctx->creds =get_current_cred ();Quoted 7 lines
fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347
Quoted 4 lines
fs/io_uring.c +4
-0+/* +* Account memory _before_ installing the file descriptor. Once +* the descriptor is installed, it can get closed at any time. +*/ What about update a bit the comment?
Maybe adding the commit description in this comment.
Quoted 8 lines
I updated the comment:
/*
* Account memory _before_ installing the file descriptor. Once
* the descriptor is installed, it can get closed at any time. Also
* do this before hitting the general error path, as ring freeing
* will un-account as well.
*/
Now it looks better!
Thanks,
Stefano
Jens Axboe
maintainerRe: [PATCH 1/2] io_uring: set ctx sq/cq entry count earlier
In reply to Stefano GarzarellaQuoted 36 lines · Stefano Garzarella
Quoted 35 lines · Jens Axboe
If we hit an earlier error path in
io_uring_create(), then we will have accounted memory, but not set ctx->{sq,cq}_entries yet. Then when the ring is torn down in error, we use those values to unaccount the memory.Ensure we set the ctx entries before we're able to hit a potential error path.
Signed-off-by Jens Axboe <[email protected]>--- fs/io_uring.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)fs/io_uring.c +4
-2@@ -8193,6 +8193,10 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx,struct io_rings *rings;size_t size, sq_array_offset; +/* make sure these are sane, as we already accounted them */ + ctx->sq_entries = p->sq_entries; + ctx->cq_entries = p->cq_entries; + size =rings_size (p->sq_entries, p->cq_entries, &sq_array_offset);if (size ==SIZE_MAX )return -EOVERFLOW ;@@ -8209,8 +8213,6 @@ static int io_allocate_scq_urings(struct io_ring_ctx *ctx, rings->cq_ring_entries = p->cq_entries; ctx->sq_mask = rings->sq_ring_mask; ctx->cq_mask = rings->cq_ring_mask; - ctx->sq_entries = rings->sq_ring_entries; - ctx->cq_entries = rings->cq_ring_entries;
Quoted 8 lines
Quoted 5 lines
fs/io_uring.c +0
-0size =array_size (sizeof (struct io_uring_sqe ), p->sq_entries);if (size ==SIZE_MAX ) {Signature · 1 line
2.28.0While reviewing I was asking if we should move
io_account_mem()beforeio_allocate_scq_urings(), then I saw the second patch :-)
Indeed, just split it in two to avoid any extra issues around backporting.
Quoted 1 line
Reviewed-by Stefano Garzarella <[email protected]>
Thanks, added.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerRe: [PATCH 2/2] io_uring: account locked memory before potential error case
In reply to Stefano GarzarellaQuoted 36 lines · Stefano Garzarella
Quoted 26 lines · Jens Axboe
Quoted 17 lines · Stefano Garzarella
Quoted 16 lines · Jens Axboe
The tear down path will always unaccount the memory, so ensure that we have accounted it before hitting any of them.
Signed-off-by Jens Axboe <[email protected]>--- fs/io_uring.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)fs/io_uring.c +0
-0@@ -8341,6 +8341,14 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, ctx->user = user; ctx->creds =get_current_cred ();Quoted 7 lines
fs/io_uring.c static int io_uring_create(unsigned entries, struct io_uring_params *p, · line 8347
Quoted 4 lines
fs/io_uring.c +4
-0+/* +* Account memory _before_ installing the file descriptor. Once +* the descriptor is installed, it can get closed at any time. +*/ What about update a bit the comment?
Maybe adding the commit description in this comment.Quoted 8 lines
I updated the comment:
/*
* Account memory _before_ installing the file descriptor. Once
* the descriptor is installed, it can get closed at any time. Also
* do this before hitting the general error path, as ring freeing
* will un-account as well.
*/
Quoted 3 lines
Now it looks better!
Reviewed-by Stefano Garzarella <[email protected]>
Thanks, added.
Signature · 1 line
Jens Axboe