io_uring/btrfs: remove struct io_uring_cmd_data
Subsystems (2)
v2 cover io_uring/btrfs: remove struct io_uring_cmd_data
btrfs's ->uring_cmd() implementations are the only ones using io_uring_cmd_data to store data that lasts for the lifetime of the uring_cmd. But all uring_cmds have to pay the memory and CPU cost of initializing this field and freeing the pointer if necessary when the uring_cmd ends. There is already a pdu field in struct io_uring_cmd that ->uring_cmd() implementations can use for storage. The only benefit of op_data seems to be that io_uring initializes it, so ->uring_cmd() can read it to tell if there was a previous call to ->uring_cmd().
Introduce a flag IORING_URING_CMD_REISSUE that ->uring_cmd() implementations can use to tell if this is the first call to ->uring_cmd() or a reissue of the uring_cmd. Switch btrfs to use the pdu storage for its btrfs_uring_encoded_data. If IORING_URING_CMD_REISSUE is unset, allocate a new btrfs_uring_encoded_data. If it's set, use the existing one in op_data. Free the btrfs_uring_encoded_data in the btrfs layer instead of relying on io_uring to free op_data. Finally, remove io_uring_cmd_data since it's now unused.
Caleb Sander Mateos (4):
btrfs/ioctl: don't skip accounting in early ENOTTY return io_uring/cmd: introduce IORING_URING_CMD_REISSUE flag btrfs/ioctl: store btrfs_uring_encoded_data in io_btrfs_cmd io_uring/cmd: remove struct io_uring_cmd_data
fs/btrfs/ioctl.c | 41 +++++++++++++++++++++++++----------- include/linux/io_uring/cmd.h | 11 ++-------- io_uring/uring_cmd.c | 18 ++++++---------- io_uring/uring_cmd.h | 1 - 4 files changed, 37 insertions(+), 34 deletions(-)
v2:
- Don't branch twice on -EAGAIN in io_uring_cmd() (Jens)
- Rebase on for-6.17/io_uring
Signature · 1 line
2.45.2
PATCH v2 1/4 btrfs/ioctl: don't skip accounting in early ENOTTY return
btrfs_uring_encoded_read() returns early with -ENOTTY if the uring_cmd is issued with IO_URING_F_COMPAT but the kernel doesn't support compat syscalls. However, this early return bypasses the syscall accounting. goto out_acct instead to ensure the syscall is counted.
--- fs/btrfs/ioctl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/btrfs/ioctl.c +2 -1
@@ -4827,11 +4827,12 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issueif (issue_flags &IO_URING_F_COMPAT ) {#if defined(CONFIG_64BIT ) && defined(CONFIG_COMPAT ) copy_end = offsetofend(struct btrfs_ioctl_encoded_io_args_32, flags);#else -return -ENOTTY ; + ret = -ENOTTY ; + goto out_acct;#endif }else { copy_end = copy_end_kernel; }
Signature · 1 line
2.45.2
PATCH v2 2/4 io_uring/cmd: introduce IORING_URING_CMD_REISSUE flag
In reply to Caleb Sander MateosAdd a flag IORING_URING_CMD_REISSUE that ->uring_cmd() implementations can use to tell whether this is the first or subsequent issue of the uring_cmd. This will allow ->uring_cmd() implementations to store information in the io_uring_cmd's pdu across issues.
--- include/linux/io_uring/cmd.h | 2 ++ io_uring/uring_cmd.c | 6 +++++- 2 files changed, 7 insertions(+), 1 deletion(-)
include/linux/io_uring/cmd.h +2 -0
@@ -6,10 +6,12 @@#include <linux/io_uring_types.h> #include <linux/blk-mq.h> /* only top 8 bits of sqe->uring_cmd_flags for kernel internal use */ #define IORING_URING_CMD_CANCELABLE (1U << 30) +/* io_uring_cmd is being issued again */ +#define IORING_URING_CMD_REISSUE (1U << 31)struct io_uring_cmd {struct file *file;const struct io_uring_sqe *sqe;/* callback to defer completions to task context */
io_uring/uring_cmd.c +5 -1
@@ -259,11 +259,15 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int issue_flags) req->iopoll_start =ktime_get_ns (); } } ret = file->f_op->uring_cmd (ioucmd, issue_flags); -if (ret == -EAGAIN || ret == -EIOCBQUEUED ) +if (ret == -EAGAIN ) { + ioucmd->flags |=IORING_URING_CMD_REISSUE ; +return ret; + } +if (ret == -EIOCBQUEUED )return ret;if (ret <0 )req_set_fail (req);io_req_uring_cleanup (req, issue_flags);io_req_set_res (req, ret,0 );
Signature · 1 line
2.45.2
PATCH v2 3/4 btrfs/ioctl: store btrfs_uring_encoded_data in io_btrfs_cmd
In reply to Caleb Sander Mateosbtrfs is the only user of struct io_uring_cmd_data and its op_data field. Switch its ->uring_cmd() implementations to store the struct btrfs_uring_encoded_data * in the struct io_btrfs_cmd, overlayed with io_uring_cmd's pdu field. This avoids having to touch another cache line to access the struct btrfs_uring_encoded_data *, and allows op_data and struct io_uring_cmd_data to be removed.
--- fs/btrfs/ioctl.c | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-)
fs/btrfs/ioctl.c +27 -11
@@ -4627,10 +4627,17 @@ static int btrfs_ioctl_encoded_write(struct file *file, void __user *argp, booladd_wchar (current, ret);inc_syscw (current);return ret; } +struct btrfs_uring_encoded_data { +struct btrfs_ioctl_encoded_io_args args; +struct iovec iovstack[UIO_FASTIOV ]; +struct iovec *iov; +struct iov_iter iter; +}; +/* * Context that's attached to an encoded read io_uring command, in cmd->pdu. It * contains the fields in btrfs_uring_read_extent that are necessary to finish * off and cleanup the I/O in btrfs_uring_read_finished. */
@@ -4648,10 +4655,11 @@ struct btrfs_uring_priv {int err;bool compressed; };struct io_btrfs_cmd { +struct btrfs_uring_encoded_data *data;struct btrfs_uring_priv *priv; };static void btrfs_uring_read_finished (struct io_uring_cmd *cmd,unsigned int issue_flags) {
@@ -4706,10 +4714,11 @@ static void btrfs_uring_read_finished(struct io_uring_cmd *cmd, unsigned int iss__free_page (priv->pages[index]);kfree (priv->pages);kfree (priv->iov);kfree (priv); +kfree (bc->data); }void btrfs_uring_read_extent_endio (void *ctx,int err) {struct btrfs_uring_priv *priv = ctx;
@@ -4789,17 +4798,10 @@ static int btrfs_uring_read_extent(struct kiocb *iocb, struct iov_iter *iter,btrfs_inode_unlock (inode,BTRFS_ILOCK_SHARED );kfree (priv);return ret; } -struct btrfs_uring_encoded_data { -struct btrfs_ioctl_encoded_io_args args; -struct iovec iovstack[UIO_FASTIOV ]; -struct iovec *iov; -struct iov_iter iter; -}; -static int btrfs_uring_encoded_read (struct io_uring_cmd *cmd,unsigned int issue_flags) { size_t copy_end_kernel = offsetofend(struct btrfs_ioctl_encoded_io_args, flags);size_t copy_end;int ret;
@@ -4811,11 +4813,15 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issueloff_t pos;struct kiocb kiocb;struct extent_state *cached_state =NULL ;u64 start, lockend;void __user *sqe_addr; -struct btrfs_uring_encoded_data *data =io_uring_cmd_get_async_data (cmd)->op_data; + struct io_btrfs_cmd *bc = io_uring_cmd_to_pdu(cmd, struct io_btrfs_cmd); +struct btrfs_uring_encoded_data *data =NULL ; + +if (cmd->flags &IORING_URING_CMD_REISSUE ) + data = bc->data;if (!capable (CAP_SYS_ADMIN )) { ret = -EPERM ; goto out_acct; }
@@ -4841,11 +4847,11 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issueif (!data) { ret = -ENOMEM ; goto out_acct; } -io_uring_cmd_get_async_data (cmd)->op_data = data; + bc->data = data;if (issue_flags &IO_URING_F_COMPAT ) {#if defined(CONFIG_64BIT ) && defined(CONFIG_COMPAT )struct btrfs_ioctl_encoded_io_args_32 args32;
@@ -4939,21 +4945,28 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issue out_acct:if (ret >0 )add_rchar (current, ret);inc_syscr (current); +if (ret != -EIOCBQUEUED && ret != -EAGAIN ) +kfree (data); +return ret; }static int btrfs_uring_encoded_write (struct io_uring_cmd *cmd,unsigned int issue_flags) {loff_t pos;struct kiocb kiocb;struct file *file;ssize_t ret;void __user *sqe_addr; -struct btrfs_uring_encoded_data *data =io_uring_cmd_get_async_data (cmd)->op_data; + struct io_btrfs_cmd *bc = io_uring_cmd_to_pdu(cmd, struct io_btrfs_cmd); +struct btrfs_uring_encoded_data *data =NULL ; + +if (cmd->flags &IORING_URING_CMD_REISSUE ) + data = bc->data;if (!capable (CAP_SYS_ADMIN )) { ret = -EPERM ; goto out_acct; }
@@ -4971,11 +4984,11 @@ static int btrfs_uring_encoded_write(struct io_uring_cmd *cmd, unsigned int issuif (!data) { ret = -ENOMEM ; goto out_acct; } -io_uring_cmd_get_async_data (cmd)->op_data = data; + bc->data = data;if (issue_flags &IO_URING_F_COMPAT ) {#if defined(CONFIG_64BIT ) && defined(CONFIG_COMPAT )struct btrfs_ioctl_encoded_io_args_32 args32;
@@ -5061,10 +5074,13 @@ static int btrfs_uring_encoded_write(struct io_uring_cmd *cmd, unsigned int issukfree (data->iov); out_acct:if (ret >0 )add_wchar (current, ret);inc_syscw (current); + +if (ret != -EAGAIN ) +kfree (data);return ret; }int btrfs_uring_cmd (struct io_uring_cmd *cmd,unsigned int issue_flags) {
Signature · 1 line
2.45.2
PATCH v2 4/4 io_uring/cmd: remove struct io_uring_cmd_data
In reply to Caleb Sander MateosThere are no more users of struct io_uring_cmd_data and its op_data field. Remove it to shave 8 bytes from struct io_async_cmd and eliminate a store and load for every uring_cmd.
--- include/linux/io_uring/cmd.h | 9 --------- io_uring/uring_cmd.c | 12 +----------- io_uring/uring_cmd.h | 1 - 3 files changed, 1 insertion(+), 21 deletions(-)
include/linux/io_uring/cmd.h +0 -9
@@ -19,14 +19,10 @@ struct io_uring_cmd {u32 cmd_op;u32 flags;u8 pdu[32 ];/* available inline for free use */ }; -struct io_uring_cmd_data { -void *op_data; -}; -static inline const void *io_uring_sqe_cmd (const struct io_uring_sqe *sqe) {return sqe->cmd; }
@@ -135,15 +131,10 @@ static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd,static inline struct task_struct *io_uring_cmd_get_task (struct io_uring_cmd *cmd) {return cmd_to_io_kiocb (cmd)->tctx->task; } -static inline struct io_uring_cmd_data *io_uring_cmd_get_async_data (struct io_uring_cmd *cmd) -{ -return cmd_to_io_kiocb (cmd)->async_data; -} -/* * Return uring_cmd's context reference as its context handle for driver to * track per-context resource, such as registered kernel IO buffer */ static inline void *io_uring_cmd_ctx_handle (struct io_uring_cmd *cmd)
io_uring/uring_cmd.c +1 -11
@@ -24,25 +24,19 @@ void io_cmd_cache_free(const void *entry)static void io_req_uring_cleanup (struct io_kiocb *req,unsigned int issue_flags) { struct io_uring_cmd *ioucmd = io_kiocb_to_cmd(req, struct io_uring_cmd);struct io_async_cmd *ac = req->async_data; -struct io_uring_cmd_data *cache = &ac->data; - -if (cache->op_data) { -kfree (cache->op_data); - cache->op_data =NULL ; - }if (issue_flags &IO_URING_F_UNLOCKED )return ;io_alloc_cache_vec_kasan (&ac->vec);if (ac->vec.nr >IO_VEC_CACHE_SOFT_CAP )io_vec_free (&ac->vec); -if (io_alloc_cache_put (&req->ctx->cmd_cache, cache)) { +if (io_alloc_cache_put (&req->ctx->cmd_cache, ac)) { ioucmd->sqe =NULL ; req->async_data =NULL ; req->flags &= ~(REQ_F_ASYNC_DATA |REQ_F_NEED_CLEANUP ); } }
@@ -191,13 +185,10 @@ EXPORT_SYMBOL_GPL(io_uring_cmd_done);int io_uring_cmd_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe) { struct io_uring_cmd *ioucmd = io_kiocb_to_cmd(req, struct io_uring_cmd);struct io_async_cmd *ac; -/* see io_uring_cmd_get_async_data() */ -BUILD_BUG_ON (offsetof(struct io_async_cmd , data) !=0 ); -if (sqe->__pad1)return -EINVAL ; ioucmd->flags =READ_ONCE (sqe->uring_cmd_flags);if (ioucmd->flags & ~IORING_URING_CMD_MASK )
@@ -209,11 +200,10 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) ioucmd->cmd_op =READ_ONCE (sqe->cmd_op); ac =io_uring_alloc_async_data (&req->ctx->cmd_cache, req);if (!ac)return -ENOMEM ; - ac->data.op_data =NULL ; ioucmd->sqe = sqe;return 0 ; }void io_uring_cmd_sqe_copy (struct io_kiocb *req)
io_uring/uring_cmd.h +0 -1
@@ -2,11 +2,10 @@#include <linux/io_uring/cmd.h> #include <linux/io_uring_types.h> struct io_async_cmd { -struct io_uring_cmd_data data;struct iou_vec vec;struct io_uring_sqe sqes[2 ]; };int io_uring_cmd (struct io_kiocb *req,unsigned int issue_flags);
Signature · 1 line
2.45.2
Hi Jens,
Are you satisfied with the updated version of this series? Let me know if there's anything else you'd like to see.
Thanks,
Caleb
Quoted 36 lines · Caleb Sander Mateos
btrfs's ->uring_cmd() implementations are the only ones using
io_uring_cmd_datato store data that lasts for the lifetime of theuring_cmd. But alluring_cmdshave to pay the memory and CPU cost of initializing this field and freeing the pointer if necessary when theuring_cmdends. There is already a pdu field in structio_uring_cmdthat ->uring_cmd() implementations can use for storage. The only benefit ofop_dataseems to be thatio_uringinitializes it, so ->uring_cmd() can read it to tell if there was a previous call to ->uring_cmd().Introduce a flag
IORING_URING_CMD_REISSUEthat ->uring_cmd() implementations can use to tell if this is the first call to ->uring_cmd() or a reissue of theuring_cmd. Switch btrfs to use the pdu storage for itsbtrfs_uring_encoded_data. IfIORING_URING_CMD_REISSUEis unset, allocate a newbtrfs_uring_encoded_data. If it's set, use the existing one inop_data. Free thebtrfs_uring_encoded_datain the btrfs layer instead of relying onio_uringto freeop_data. Finally, removeio_uring_cmd_datasince it's now unused.Caleb Sander Mateos (4):
btrfs/ioctl: don't skip accounting in early ENOTTY returnio_uring/cmd: introduceIORING_URING_CMD_REISSUEflag btrfs/ioctl: storebtrfs_uring_encoded_datainio_btrfs_cmdio_uring/cmd: remove structio_uring_cmd_datafs/btrfs/ioctl.c | 41 +++++++++++++++++++++++++----------- include/linux/io_uring/cmd.h | 11 ++-------- io_uring/uring_cmd.c | 18 ++++++---------- io_uring/uring_cmd.h | 1 - 4 files changed, 37 insertions(+), 34 deletions(-)v2:
- Don't branch twice on-EAGAINinio_uring_cmd()(Jens)
- Rebase on for-6.17/io_uring--
2.45.2
Jens Axboe
maintainerQuoted 3 lines · Caleb Sander Mateos
Hi Jens,
Are you satisfied with the updated version of this series? Let me know if there's anything else you'd like to see.
I'm fine with it, was hoping some of the CC'ed btrfs folks would ack or review the btrfs bits. OOO until late sunday, if I hear nothing else by then, I'll just tentatively stage it in a separate branch for 6.17.
What test cases did you run?
Signature · 1 line
Jens Axboe
David Sterba
Acked-byQuoted 8 lines · Jens Axboe
Quoted 3 lines · Caleb Sander Mateos
Hi Jens,
Are you satisfied with the updated version of this series? Let me know if there's anything else you'd like to see.I'm fine with it, was hoping some of the CC'ed btrfs folks would ack or review the btrfs bits. OOO until late sunday, if I hear nothing else by then, I'll just tentatively stage it in a separate branch for 6.17.
I've taken the first patch to btrfs' for-next but if you want to add it to your queue then git will deal with that too. For the btrfs changes
Jens Axboe
maintainerQuoted 6 lines · David Sterba
Quoted 5 lines · Jens Axboe
Quoted 3 lines · Caleb Sander Mateos
Hi Jens,
Are you satisfied with the updated version of this series? Let me know if there's anything else you'd like to see.
Quoted 8 lines
Quoted 3 lines
I'm fine with it, was hoping some of the CC'ed btrfs folks would ack or review the btrfs bits. OOO until late sunday, if I hear nothing else by then, I'll just tentatively stage it in a separate branch for 6.17.
I've taken the first patch to btrfs' for-next but if you want to add it to your queue then git will deal with that too. For the btrfs changes
Acked-by David Sterba <[email protected]>
Thanks! I guess that works fine, it can go in separately. I've queued up the rest.
Signature · 1 line
Jens Axboe
Re: (subset) [PATCH v2 0/4] io_uring/btrfs: remove struct io_uring_cmd_data
In reply to Caleb Sander MateosQuoted 9 lines · Caleb Sander Mateos
btrfs's ->uring_cmd() implementations are the only ones using
io_uring_cmd_datato store data that lasts for the lifetime of theuring_cmd. But alluring_cmdshave to pay the memory and CPU cost of initializing this field and freeing the pointer if necessary when theuring_cmdends. There is already a pdu field in structio_uring_cmdthat ->uring_cmd() implementations can use for storage. The only benefit ofop_dataseems to be thatio_uringinitializes it, so ->uring_cmd() can read it to tell if there was a previous call to ->uring_cmd().[...]
Applied, thanks!
[2/4] io_uring/cmd: introduce IORING_URING_CMD_REISSUE flag
commit: 733c43f1df34f9185b945e6f12ac00c8556c6dfe
[3/4] btrfs/ioctl: store btrfs_uring_encoded_data in io_btrfs_cmd
commit: 9aad72b4e3f0233e747bb6b1ec05ea71365f4246
[4/4] io_uring/cmd: remove struct io_uring_cmd_data
commit: 2e6dbb25ea15844c8b617260d635731c37c85ac9Best regards,
Signature · 1 line
Jens Axboe