Skip to contentSkip to conversation

nvme-passthru with vectored-io

PATCH In review Kanchan Joshi 4 participants 9 messages Jan 27, 2022 – Jan 28, 2022 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): CGME20220127083033epcas5p3ce9565e4b5e21e897571e48e73e4f9af@epcas5p3.samsung.com

#1 lore raw

cover nvme-passthru with vectored-io

NVMe passthru takes a single buffer pointer.
This enables using multiple buffers for passthru, similiar to readv/writev.

First patch is prep, while second one adds vectored-io abililty to NVME_IOCTL_IO64_CMD.

Kanchan Joshi (2):
block: introduce and export blk_rq_map_user_vec
nvme: add vectored-io support for user passthru

 block/blk-map.c                 | 19 +++++++++++++++++++
 drivers/nvme/host/ioctl.c       |  9 ++++++---
 include/linux/blk-mq.h          |  2 ++
 include/uapi/linux/nvme_ioctl.h |  4 ++++
 4 files changed, 31 insertions(+), 3 deletions(-)
Signature · 1 line
2.25.1
#2 lore raw

PATCH 1/2 block: introduce and export blk_rq_map_user_vec

Similiar to blk_rq_map_user except that it operates on iovec. This is a prep patch.

---
 block/blk-map.c        | 19 +++++++++++++++++++
 include/linux/blk-mq.h |  2 ++
 2 files changed, 21 insertions(+)
block/blk-map.c +19 -0
index 4526adde0156..7fe45df3e580 100644
@@ -577,6 +577,25 @@ int blk_rq_map_user(struct request_queue *q, struct request *rq,
 }
 EXPORT_SYMBOL(blk_rq_map_user);
 
+int blk_rq_map_user_vec(struct request_queue *q, struct request *rq,
+		    struct rq_map_data *map_data, void __user *uvec,
+		    unsigned long nr_vecs, gfp_t gfp_mask)
+{
+	struct iovec fast_iov[UIO_FASTIOV];
+	struct iovec *iov = fast_iov;
+	struct iov_iter iter;
+	int ret;
+
+	ret = import_iovec(rq_data_dir(rq), uvec, nr_vecs, UIO_FASTIOV, &iov, &iter);
+	if (unlikely(ret < 0))
+		return ret;
+	ret = blk_rq_map_user_iov(q, rq, NULL, &iter, gfp_mask);
+	kfree(iov);
+
+	return ret;
+}
+EXPORT_SYMBOL(blk_rq_map_user_vec);
+
 /**
  * blk_rq_unmap_user - unmap a request with user data
  * @bio:	       start of bio list
include/linux/blk-mq.h +2 -0
index d319ffa59354..0fda666d2230 100644
@@ -966,6 +966,8 @@ struct rq_map_data {
 
 int blk_rq_map_user(struct request_queue *, struct request *,
 		struct rq_map_data *, void __user *, unsigned long, gfp_t);
+int blk_rq_map_user_vec(struct request_queue *, struct request *,
+		struct rq_map_data *, void __user *, unsigned long, gfp_t);
 int blk_rq_map_user_iov(struct request_queue *, struct request *,
 		struct rq_map_data *, const struct iov_iter *, gfp_t);
 int blk_rq_unmap_user(struct bio *);
Signature · 1 line
2.25.1
#3 lore raw

Re: [PATCH 1/2] block: introduce and export blk_rq_map_user_vec

Quoted 24 lines · Kanchan Joshi

Similiar to blk_rq_map_user except that it operates on iovec. This is a prep patch.

---
 block/blk-map.c        | 19 +++++++++++++++++++
 include/linux/blk-mq.h |  2 ++
 2 files changed, 21 insertions(+)
block/blk-map.c +6 -0
index 4526adde0156..7fe45df3e580 100644
@@ -577,6 +577,25 @@ int blk_rq_map_user(struct request_queue *q, struct request *rq,
 }
 EXPORT_SYMBOL(blk_rq_map_user);
 
+int blk_rq_map_user_vec(struct request_queue *q, struct request *rq,
+		    struct rq_map_data *map_data, void __user *uvec,
+		    unsigned long nr_vecs, gfp_t gfp_mask)
+{
+	struct iovec fast_iov[UIO_FASTIOV];
+	struct iovec *iov = fast_iov;

block/blk-map.c int blk_rq_map_user(struct request_queue *q, struct request *rq, · line 595

Quoted 10 lines
block/blk-map.c +10 -0
+	struct iov_iter iter;
+	int ret;
+
+	ret = import_iovec(rq_data_dir(rq), uvec, nr_vecs, UIO_FASTIOV, &iov, &iter);
+	if (unlikely(ret < 0))
+		return ret;
+	ret = blk_rq_map_user_iov(q, rq, NULL, &iter, gfp_mask);
+	kfree(iov);
+
+	return ret;

I see very little point in adding this function vs just open coding it.

Also please don't add overly long lines.

#4 lore raw

Re: [PATCH 1/2] block: introduce and export blk_rq_map_user_vec

Quoted 30 lines · Christoph Hellwig
Quoted 28 lines · Kanchan Joshi

Similiar to blk_rq_map_user except that it operates on iovec. This is a prep patch.

---
 block/blk-map.c        | 19 +++++++++++++++++++
 include/linux/blk-mq.h |  2 ++
 2 files changed, 21 insertions(+)
block/blk-map.c +10 -0
index 4526adde0156..7fe45df3e580 100644
@@ -577,6 +577,25 @@ int blk_rq_map_user(struct request_queue *q, struct request *rq,
 }
 EXPORT_SYMBOL(blk_rq_map_user);
 
+int blk_rq_map_user_vec(struct request_queue *q, struct request *rq,
+                 struct rq_map_data *map_data, void __user *uvec,
+                 unsigned long nr_vecs, gfp_t gfp_mask)
+{
+     struct iovec fast_iov[UIO_FASTIOV];
+     struct iovec *iov = fast_iov;
+     struct iov_iter iter;
+     int ret;
+
+     ret = import_iovec(rq_data_dir(rq), uvec, nr_vecs, UIO_FASTIOV, &iov, &iter);
Quoted 8 lines

block/blk-map.c int blk_rq_map_user(struct request_queue *q, struct request *rq, · line 595

Quoted 6 lines
block/blk-map.c +6 -0
+     if (unlikely(ret < 0))
+             return ret;
+     ret = blk_rq_map_user_iov(q, rq, NULL, &iter, gfp_mask);
+     kfree(iov);
+
+     return ret;

I see very little point in adding this function vs just open coding it.

Fine, I will kill this and open-code in nvme instead.

Signature · 1 line
Kanchan
#6 lore raw

PATCH 2/2 nvme: add vectored-io support for user passthru

In reply to Kanchan Joshi

wire up support for passthru that takes an array of buffers (using iovec). Enable it for NVME_IOCTL_IO64_CMD; same ioctl code to be used with following differences -

1. NVME_IOVEC to be set as cmd.flags
2. cmd.addr, base addr of user iovec array
3. cmd.data_len, count of iovec array elements

---
 drivers/nvme/host/ioctl.c       | 9 ++++++---
 include/uapi/linux/nvme_ioctl.h | 4 ++++
 2 files changed, 10 insertions(+), 3 deletions(-)
drivers/nvme/host/ioctl.c +6 -3
index 22314962842d..3a896443e110 100644
@@ -65,6 +65,7 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 	struct bio *bio = NULL;
 	void *meta = NULL;
 	int ret;
+	u8 cmd_flags = cmd->common.flags;
 
 	req = nvme_alloc_request(q, cmd, 0);
 	if (IS_ERR(req))
@@ -75,7 +76,11 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 	nvme_req(req)->flags |= NVME_REQ_USERCMD;
 
 	if (ubuffer && bufflen) {
-		ret = blk_rq_map_user(q, req, NULL, ubuffer, bufflen,
+		if (!(cmd_flags & NVME_IOVEC))
+			ret = blk_rq_map_user(q, req, NULL, ubuffer, bufflen,
+				GFP_KERNEL);
+		else
+			ret = blk_rq_map_user_vec(q, req, NULL, ubuffer, bufflen,
 				GFP_KERNEL);
 		if (ret)
 			goto out;
@@ -246,8 +251,6 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 		return -EACCES;
 	if (copy_from_user(&cmd, ucmd, sizeof(cmd)))
 		return -EFAULT;
-	if (cmd.flags)
-		return -EINVAL;
 	if (!nvme_validate_passthru_nsid(ctrl, ns, cmd.nsid))
 		return -EINVAL;
 
include/uapi/linux/nvme_ioctl.h +4 -0
index d99b5a772698..d4999e3930f8 100644
@@ -9,6 +9,10 @@
 
 #include <linux/types.h>
 
+enum nvme_ioc_flags {
+	NVME_IOVEC	= 1 << 0 /* vectored io */
+};
+
 struct nvme_user_io {
 	__u8	opcode;
 	__u8	flags;
Signature · 1 line
2.25.1
#7 lore raw

Re: [PATCH 2/2] nvme: add vectored-io support for user passthru

Quoted 3 lines · Kanchan Joshi

wire up support for passthru that takes an array of buffers (using iovec). Enable it for NVME_IOCTL_IO64_CMD; same ioctl code to be used with following differences -

Flags overloading for a completely different ABI is a bad idea. Please add a separate ioctl instead.

#8 lore raw

Re: [PATCH 2/2] nvme: add vectored-io support for user passthru

Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Kanchan Joshi

wire up support for passthru that takes an array of buffers (using iovec). Enable it for NVME_IOCTL_IO64_CMD; same ioctl code to be used with following differences -

Flags overloading for a completely different ABI is a bad idea. Please add a separate ioctl instead.

Hope "NVME_IOCTL_IO64_VEC_CMD" sounds fine for the new one?

Signature · 1 line
Kanchan
#9 lore raw

Re: [PATCH 2/2] nvme: add vectored-io support for user passthru

Flags overloading for a completely different ABI is a bad idea.

Quoted 4 lines
Quoted 1 line

Please add a separate ioctl instead.

Hope "NVME_IOCTL_IO64_VEC_CMD" sounds fine for the new one?

sounds reasonable to me..

Quoted 3 lines

--
Kanchan