Skip to contentSkip to conversation

nvme: Return BLK_STS_TARGET if the DNR bit is set

PATCH Superseded Hannes Reinecke 5 participants 9 messages Aug 12, 2019 – Aug 13, 2019 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

If the DNR bit is set we should not retry the command, even if the standard status evaluation indicates so.

Signed-off-by Hannes Reinecke <hare at suse.com>
---
 drivers/nvme/host/core.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
drivers/nvme/host/core.c +13 -8
index cc09b81fc7f4..5e1309709917 100644
@@ -267,15 +267,20 @@ void nvme_complete_rq(struct request *req)
 	if (nvme_req(req)->ctrl->kas)
 		nvme_req(req)->ctrl->comp_seen = true;
 
-	if (unlikely(status != BLK_STS_OK && nvme_req_needs_retry(req))) {
-		if ((req->cmd_flags & REQ_NVME_MPATH) &&
-		    blk_path_error(status)) {
-			nvme_failover_req(req);
-			return;
+	if (unlikely(status != BLK_STS_OK)) {
+		if (nvme_req_needs_retry(req)) {
+			if ((req->cmd_flags & REQ_NVME_MPATH) &&
+			    blk_path_error(status)) {
+				nvme_failover_req(req);
+				return;
+			}
+			if (!blk_queue_dying(req->q)) {
+				nvme_retry_req(req);
+				return;
+			}
 		}
-
-		if (!blk_queue_dying(req->q)) {
-			nvme_retry_req(req);
+		if (nvme_req(req)->status & NVME_SC_DNR) {
+			blk_mq_end_request(req, BLK_STS_TARGET);
 			return;
 		}
 	}
Signature · 1 line
2.16.4
#1 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 31 lines · Hannes Reinecke

If the DNR bit is set we should not retry the command, even if the standard status evaluation indicates so.

Signed-off-by Hannes Reinecke <hare at suse.com>
---
  drivers/nvme/host/core.c | 21 +++++++++++++--------
  1 file changed, 13 insertions(+), 8 deletions(-)
drivers/nvme/host/core.c +10 -5
index cc09b81fc7f4..5e1309709917 100644
@@ -267,15 +267,20 @@ void nvme_complete_rq(struct request *req)
  	if (nvme_req(req)->ctrl->kas)
  		nvme_req(req)->ctrl->comp_seen = true;
 
-	if (unlikely(status != BLK_STS_OK && nvme_req_needs_retry(req))) {
-		if ((req->cmd_flags & REQ_NVME_MPATH) &&
-		    blk_path_error(status)) {
-			nvme_failover_req(req);
-			return;
+	if (unlikely(status != BLK_STS_OK)) {
+		if (nvme_req_needs_retry(req)) {
+			if ((req->cmd_flags & REQ_NVME_MPATH) &&
+			    blk_path_error(status)) {
+				nvme_failover_req(req);
+				return;
+			}
+			if (!blk_queue_dying(req->q)) {
+				nvme_retry_req(req);
+				return;

drivers/nvme/host/core.c void nvme_complete_rq(struct request *req) · line 286

Quoted 10 lines
drivers/nvme/host/core.c +3 -3
+			}
  		}
-
-		if (!blk_queue_dying(req->q)) {
-			nvme_retry_req(req);
+		if (nvme_req(req)->status & NVME_SC_DNR) {
+			blk_mq_end_request(req, BLK_STS_TARGET);
  			return;
  		}
  	}

This seems redundant, to re-check the NVME_SC_DNR here, after just checking it in nvme_req_needs_retry().

Ed

maintainer
#2 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 36 lines · Edmund
Quoted 35 lines · Hannes Reinecke

If the DNR bit is set we should not retry the command, even if the standard status evaluation indicates so.

Signed-off-by Hannes Reinecke <hare at suse.com>

---
? drivers/nvme/host/core.c | 21 +++++++++++++-------- ? 1 file changed, 13 insertions(+), 8 deletions(-)

drivers/nvme/host/core.c +11 -8
index cc09b81fc7f4..5e1309709917 100644
@@ -267,15 +267,20 @@ void nvme_complete_rq(struct request *req)
 ????? if (nvme_req(req)->ctrl->kas)
 ????????? nvme_req(req)->ctrl->comp_seen = true;
-??? if (unlikely(status != BLK_STS_OK && nvme_req_needs_retry(req))) {
-??????? if ((req->cmd_flags & REQ_NVME_MPATH) &&
-??????????? blk_path_error(status)) {
-??????????? nvme_failover_req(req);
-??????????? return;
+??? if (unlikely(status != BLK_STS_OK)) {
+??????? if (nvme_req_needs_retry(req)) {
+??????????? if ((req->cmd_flags & REQ_NVME_MPATH) &&
+??????????????? blk_path_error(status)) {
+??????????????? nvme_failover_req(req);
+??????????????? return;
+??????????? }
+??????????? if (!blk_queue_dying(req->q)) {
+??????????????? nvme_retry_req(req);
+??????????????? return;
+??????????? }
 ????????? }
-
-??????? if (!blk_queue_dying(req->q)) {
-??????????? nvme_retry_req(req);
Quoted 8 lines

drivers/nvme/host/core.c void nvme_complete_rq(struct request *req) · line 285

Quoted 5 lines
drivers/nvme/host/core.c +2 -0
+??????? if (nvme_req(req)->status & NVME_SC_DNR) {
+??????????? blk_mq_end_request(req, BLK_STS_TARGET);
 ????????????? return;
 ????????? }
 ????? }

This seems redundant, to re-check the NVME_SC_DNR here, after just checking it in nvme_req_needs_retry().

Indeed, why do we need this at this point? we can simply continue and end the request down in the func (after we trace it as well).

maintainer
#3 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 46 lines · Sagi Grimberg
Quoted 36 lines · Edmund
Quoted 35 lines · Hannes Reinecke

If the DNR bit is set we should not retry the command, even if the standard status evaluation indicates so.

Signed-off-by Hannes Reinecke <hare at suse.com>

---
? drivers/nvme/host/core.c | 21 +++++++++++++-------- ? 1 file changed, 13 insertions(+), 8 deletions(-)

drivers/nvme/host/core.c +11 -8
index cc09b81fc7f4..5e1309709917 100644
@@ -267,15 +267,20 @@ void nvme_complete_rq(struct request *req)
 ????? if (nvme_req(req)->ctrl->kas)
 ????????? nvme_req(req)->ctrl->comp_seen = true;
-??? if (unlikely(status != BLK_STS_OK && nvme_req_needs_retry(req))) {
-??????? if ((req->cmd_flags & REQ_NVME_MPATH) &&
-??????????? blk_path_error(status)) {
-??????????? nvme_failover_req(req);
-??????????? return;
+??? if (unlikely(status != BLK_STS_OK)) {
+??????? if (nvme_req_needs_retry(req)) {
+??????????? if ((req->cmd_flags & REQ_NVME_MPATH) &&
+??????????????? blk_path_error(status)) {
+??????????????? nvme_failover_req(req);
+??????????????? return;
+??????????? }
+??????????? if (!blk_queue_dying(req->q)) {
+??????????????? nvme_retry_req(req);
+??????????????? return;
+??????????? }
 ????????? }
-
-??????? if (!blk_queue_dying(req->q)) {
-??????????? nvme_retry_req(req);
Quoted 8 lines

drivers/nvme/host/core.c void nvme_complete_rq(struct request *req) · line 285

Quoted 5 lines
drivers/nvme/host/core.c +2 -0
+??????? if (nvme_req(req)->status & NVME_SC_DNR) {
+??????????? blk_mq_end_request(req, BLK_STS_TARGET);
 ????????????? return;
 ????????? }
 ????? }

This seems redundant, to re-check the NVME_SC_DNR here, after just checking it in nvme_req_needs_retry().

Quoted 2 lines

Indeed, why do we need this at this point? we can simply continue and end the request down in the func (after we trace it as well).

We need to override the default nvme_error_status() to BLK_STS_TARGET when DNR is set to prevent upper layers from their own retries/failover. I suggested adding that check in the existing unlikely() case, but maybe it's more obvious if we teach nvme_error_status() to default to return BLK_STS_TARGET for this condition?

---

drivers/nvme/host/core.c +3 -0
index 8f3fbe5ca937..50adcdde44d5 100644
@@ -199,6 +199,9 @@ static inline bool nvme_ns_has_pi(struct nvme_ns *ns)
 
 static blk_status_t nvme_error_status(struct request *req)
 {
+	if (unlikely(nvme_req(req)->status & NVME_SC_DNR))
+		return BLK_STS_TARGET;
+
 	switch (nvme_req(req)->status & 0x7ff) {
 	case NVME_SC_SUCCESS:
 		return BLK_STS_OK;

--

maintainer
#4 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 13 lines

We need to override the default nvme_error_status() to BLK_STS_TARGET when DNR is set to prevent upper layers from their own retries/failover. I suggested adding that check in the existing unlikely() case, but maybe it's more obvious if we teach nvme_error_status() to default to return BLK_STS_TARGET for this condition?

---

drivers/nvme/host/core.c +0 -0
index 8f3fbe5ca937..50adcdde44d5 100644

drivers/nvme/host/core.c static inline bool nvme_ns_has_pi(struct nvme_ns *ns)

Quoted 10 lines
drivers/nvme/host/core.c +3 -1
@@ -199,6 +199,9 @@ static inline bool nvme_ns_has_pi(struct nvme_ns *ns)
 
  static blk_status_t nvme_error_status(struct request *req)
  {
+	if (unlikely(nvme_req(req)->status & NVME_SC_DNR))
+		return BLK_STS_TARGET;
+
  	switch (nvme_req(req)->status & 0x7ff) {
  	case NVME_SC_SUCCESS:
  		return BLK_STS_OK;
--

Makes sense to me, Hannes does this address your issue?

#5 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 16 lines · Sagi Grimberg
Quoted 16 lines

We need to override the default nvme_error_status() to BLK_STS_TARGET when DNR is set to prevent upper layers from their own retries/failover. I suggested adding that check in the existing unlikely() case, but maybe it's more obvious if we teach nvme_error_status() to default to return BLK_STS_TARGET for this condition?

---

drivers/nvme/host/core.c +1 -0
index 8f3fbe5ca937..50adcdde44d5 100644
@@ -199,6 +199,9 @@ static inline bool nvme_ns_has_pi(struct nvme_ns *ns)
 ? static blk_status_t nvme_error_status(struct request *req)
 ? {
+??? if (unlikely(nvme_req(req)->status & NVME_SC_DNR))
Quoted 8 lines
Quoted 6 lines
drivers/nvme/host/core.c +2 -0
+??????? return BLK_STS_TARGET;
+
 ????? switch (nvme_req(req)->status & 0x7ff) {

????? case NVME_SC_SUCCESS:
????????? return BLK_STS_OK;

Makes sense to me, Hannes does this address your issue?

Oh, yes, it does.
I was slightly worried as we would override other non-retryable status codes, but as there is no definite mapping which status codes are retryable we can go with this one.

I'll be sending a patch.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare at suse.de                              +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#6 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

In reply to Hannes Reinecke
Quoted 2 lines · Hannes Reinecke

If the DNR bit is set we should not retry the command, even if the standard status evaluation indicates so.

What problem is this even trying to solve? Nothing in the documentation of BLK_STS_TARGET says it should be retried any more or less than other error code.

If you really care about a retryable vs not retryable distinction at the block layer we need to propagate the equivalent of the DNR bit.

#7 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · Hannes Reinecke

If the DNR bit is set we should not retry the command, even if the standard status evaluation indicates so.

Quoted 8 lines

What problem is this even trying to solve? Nothing in the documentation of BLK_STS_TARGET says it should be retried any more or less than other error code.

If you really care about a retryable vs not retryable distinction at the block layer we need to propagate the equivalent of the DNR bit.

That was my worry, too.
But I wasn't sure if that was the direction we should be going. Personally, I'd be more than happy to add a flag for non-retryable errors.

Let's see how it would look like ...

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare at suse.de                              +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#8 lore raw

PATCH v3 1/1 nvme: Return BLK_STS_TARGET if the DNR bit is set

Quoted 5 lines · Hannes Reinecke

That was my worry, too.
But I wasn't sure if that was the direction we should be going. Personally, I'd be more than happy to add a flag for non-retryable errors.

Let's see how it would look like ...

Can you start by explaining what the actual problem is, e.g. what you are doing, what the expected and what the actual behavior is. That also belongs into the commit log.