Skip to contentSkip to conversation

support nvme-tcp SUCCESS flag optimization

PATCH New Sagi Grimberg 3 participants 10 messages Mar 8, 2019 – Mar 12, 2019

Subsystems (2)
lore
Share & tools
lore

Sagi Grimberg

author maintainer
#0 lore raw

cover support nvme-tcp SUCCESS flag optimization

The SUCCESS flag is a bit that exists in the C2HData PDU header and allows a controller to indicate that the command completed successfully from the controller perspective, and by setting it, the controller omits a subsequent completion response capsule PDU.

Its optional usage in the spec, but the host definitly needs to support it given that the controller is allowed to set it anytime.

Thanks to Steve Blightman and Oliver Smith-Denny for raising my attention on this.

Sagi Grimberg (2):
nvme-tcp: support C2HData with SUCCESS flag
nvmet-tcp: implement C2HData SUCCESS optimization

 drivers/nvme/host/tcp.c   | 32 ++++++++++++++++++++++++++++----
 drivers/nvme/target/tcp.c | 24 +++++++++++++++++++++---
 2 files changed, 49 insertions(+), 7 deletions(-)
Signature · 1 line
2.17.1
author maintainer
#1 lore raw

PATCH 1/2 nvme-tcp: support C2HData with SUCCESS flag

A C2HData PDU with the SUCCESS flag set indicates that the I/O was completed by the controller successfully and means that a subsequent completion response capsule PDU will be ommitted.

If we see this flag, fisrt we check that LAST_PDU flag is set as well, and then we complete the request when the data transfer (and data digest verification if its on) is done.

While we're at it, reuse a bit of code with nvme_fail_request.

Suggested-by Oliver Smith-Denny <osmithde at cisco.com>
Signed-off-by Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/tcp.c | 32 ++++++++++++++++++++++++++++----
 1 file changed, 28 insertions(+), 4 deletions(-)
drivers/nvme/host/tcp.c +28 -4
index 208ee518af65..e7e08889865e 100644
@@ -463,6 +463,15 @@ static int nvme_tcp_handle_c2h_data(struct nvme_tcp_queue *queue,
 
 	queue->data_remaining = le32_to_cpu(pdu->data_length);
 
+	if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS &&
+	    unlikely(!(pdu->hdr.flags & NVME_TCP_F_DATA_LAST))) {
+		dev_err(queue->ctrl->ctrl.device,
+			"queue %d tag %#x SUCCESS set but not last PDU\n",
+			nvme_tcp_queue_id(queue), rq->tag);
+		nvme_tcp_error_recovery(&queue->ctrl->ctrl);
+		return -EPROTO;
+	}
+
 	return 0;
 
 }
@@ -618,6 +627,14 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	return ret;
 }
 
+static inline void nvme_tcp_end_request(struct request *rq, __le16 status)
+{
+	union nvme_result res = {};
+
+	nvme_end_request(rq, cpu_to_le16(status << 1), res);
+}
+
+
 static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 			      unsigned int *offset, size_t *len)
 {
@@ -685,6 +702,8 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 			nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
 			queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
 		} else {
+			if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS)
+				nvme_tcp_end_request(rq, NVME_SC_SUCCESS);
 			nvme_tcp_init_recv_ctx(queue);
 		}
 	}
@@ -695,6 +714,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
 		struct sk_buff *skb, unsigned int *offset, size_t *len)
 {
+	struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
 	char *ddgst = (char *)&queue->recv_ddgst;
 	size_t recv_len = min_t(size_t, *len, queue->ddgst_remaining);
 	off_t off = NVME_TCP_DIGEST_LENGTH - queue->ddgst_remaining;
@@ -718,6 +738,13 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
 		return -EIO;
 	}
 
+	if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
+		struct request *rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
+						pdu->command_id);
+
+		nvme_tcp_end_request(rq, NVME_SC_SUCCESS);
+	}
+
 	nvme_tcp_init_recv_ctx(queue);
 	return 0;
 }
@@ -815,10 +842,7 @@ static inline void nvme_tcp_done_send_req(struct nvme_tcp_queue *queue)
 
 static void nvme_tcp_fail_request(struct nvme_tcp_request *req)
 {
-	union nvme_result res = {};
-
-	nvme_end_request(blk_mq_rq_from_pdu(req),
-		cpu_to_le16(NVME_SC_DATA_XFER_ERROR), res);
+	nvme_tcp_end_request(blk_mq_rq_from_pdu(req), NVME_SC_DATA_XFER_ERROR);
 }
 
 static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)
Signature · 1 line
2.17.1
author maintainer
#2 lore raw

PATCH 1/2 nvme-tcp: support C2HData with SUCCESS flag

Quoted 11 lines

A C2HData PDU with the SUCCESS flag set indicates that the I/O was completed by the controller successfully and means that a subsequent completion response capsule PDU will be ommitted.

If we see this flag, fisrt we check that LAST_PDU flag is set as well, and then we complete the request when the data transfer (and data digest verification if its on) is done.

While we're at it, reuse a bit of code with nvme_fail_request.

rrr...

This should also have:

Reported-by Steve Blightman <steve.blightman at oracle.com>
Quoted 3 lines
Suggested-by Oliver Smith-Denny <osmithde at cisco.com>
Signed-off-by Sagi Grimberg <sagi at grimberg.me>

---

#3 lore raw

PATCH 1/2 nvme-tcp: support C2HData with SUCCESS flag

In reply to Sagi Grimberg
Quoted 8 lines · Sagi Grimberg

A C2HData PDU with the SUCCESS flag set indicates that the I/O was completed by the controller successfully and means that a subsequent completion response capsule PDU will be ommitted.

If we see this flag, fisrt we check that LAST_PDU flag is set as well, and then we complete the request when the data transfer (and data digest verification if its on) is done.

Quoted 7 lines

While we're at it, reuse a bit of code with nvme_fail_request.

Suggested-by Oliver Smith-Denny <osmithde at cisco.com>
Signed-off-by Sagi Grimberg <sagi at grimberg.me>
---
  drivers/nvme/host/tcp.c | 32 ++++++++++++++++++++++++++++----
  1 file changed, 28 insertions(+), 4 deletions(-)

Looks good! I don't see any issues.

Tested-and-Reviewed-by: Oliver Smith-Denny

Thanks Sagi!

Oliver

author maintainer
#5 lore raw

PATCH 2/2 nvmet-tcp: implement C2HData SUCCESS optimization

In reply to Sagi Grimberg

TP 8000 says that the use of the SUCCESS flag depends on weather the controller support disabling sq_head pointer updates. Given that we support it by default, makes sense that we go the extra mile to actually use the SUCCESS flag.

When we create the C2HData PDU header, we check if sqhd_disabled is set on our queue, if so, we set the SUCCESS flag in the PDU header and skip sending a completion response capsule.

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/tcp.c | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)
drivers/nvme/target/tcp.c +21 -3
index ad0df786fe93..0a941abf56ec 100644
@@ -371,7 +371,8 @@ static void nvmet_setup_c2h_data_pdu(struct nvmet_tcp_cmd *cmd)
 	cmd->state = NVMET_TCP_SEND_DATA_PDU;
 
 	pdu->hdr.type = nvme_tcp_c2h_data;
-	pdu->hdr.flags = NVME_TCP_F_DATA_LAST;
+	pdu->hdr.flags = NVME_TCP_F_DATA_LAST | (queue->nvme_sq.sqhd_disabled ?
+						NVME_TCP_F_DATA_SUCCESS : 0);
 	pdu->hdr.hlen = sizeof(*pdu);
 	pdu->hdr.pdo = pdu->hdr.hlen + hdgst;
 	pdu->hdr.plen =
@@ -542,8 +543,19 @@ static int nvmet_try_send_data(struct nvmet_tcp_cmd *cmd)
 		cmd->state = NVMET_TCP_SEND_DDGST;
 		cmd->offset = 0;
 	} else {
-		nvmet_setup_response_pdu(cmd);
+		if (queue->nvme_sq.sqhd_disabled) {
+			cmd->queue->snd_cmd = NULL;
+			nvmet_tcp_put_cmd(cmd);
+		} else {
+			nvmet_setup_response_pdu(cmd);
+		}
 	}
+
+	if (queue->nvme_sq.sqhd_disabled) {
+		kfree(cmd->iov);
+		sgl_free(cmd->req.sg);
+	}
+
 	return 1;
 
 }
@@ -619,7 +631,13 @@ static int nvmet_try_send_ddgst(struct nvmet_tcp_cmd *cmd)
 		return ret;
 
 	cmd->offset += ret;
-	nvmet_setup_response_pdu(cmd);
+
+	if (queue->nvme_sq.sqhd_disabled) {
+		cmd->queue->snd_cmd = NULL;
+		nvmet_tcp_put_cmd(cmd);
+	} else {
+		nvmet_setup_response_pdu(cmd);
+	}
 	return 1;
 }
 
Signature · 1 line
2.17.1
#6 lore raw

PATCH 2/2 nvmet-tcp: implement C2HData SUCCESS optimization

Quoted 10 lines · Sagi Grimberg

TP 8000 says that the use of the SUCCESS flag depends on weather the controller support disabling sq_head pointer updates. Given that we support it by default, makes sense that we go the extra mile to actually use the SUCCESS flag.

When we create the C2HData PDU header, we check if sqhd_disabled is set on our queue, if so, we set the SUCCESS flag in the PDU header and skip sending a completion response capsule.

Signed-off-by Sagi Grimberg <sagi at grimberg.me>

Target looks good, too.

Tested-and-Reviewed-by: Oliver Smith-Denny

author maintainer
#8 lore raw

cover support nvme-tcp SUCCESS flag optimization

In reply to Sagi Grimberg

Steve, Oliver, Did you get a chance to test this out?

Quoted 19 lines

The SUCCESS flag is a bit that exists in the C2HData PDU header and allows a controller to indicate that the command completed successfully from the controller perspective, and by setting it, the controller omits a subsequent completion response capsule PDU.

Its optional usage in the spec, but the host definitly needs to support it given that the controller is allowed to set it anytime.

Thanks to Steve Blightman and Oliver Smith-Denny for raising my attention on this.

Sagi Grimberg (2):
nvme-tcp: support C2HData with SUCCESS flag
nvmet-tcp: implement C2HData SUCCESS optimization

  drivers/nvme/host/tcp.c   | 32 ++++++++++++++++++++++++++++----
  drivers/nvme/target/tcp.c | 24 +++++++++++++++++++++---
  2 files changed, 49 insertions(+), 7 deletions(-)
#9 lore raw

cover support nvme-tcp SUCCESS flag optimization

Quoted 1 line · Sagi Grimberg

Steve, Oliver, Did you get a chance to test this out?

Hi Sagi, sorry I was on vacation for a few days. I am planning on testing this patch out right now and I will let you know how it goes!

Thanks,
Oliver