Skip to contentSkip to conversation

nvme-core: optimize process for error status NVME_SC_CMD_INTERRUPTED

PATCH In review Chao Leng 2 participants 3 messages Jul 24, 2020 – Jul 25, 2020 NVMe

lore
Share & tools
lore

Chao Leng

author
#0 lore raw

PATCH 1/1 nvme-core: optimize process for error status NVME_SC_CMD_INTERRUPTED

The protocol NVM-Express-1.4 define:
Command Interrupted: Command processing was interrupted and the controller is unable to successfully complete the command. The host should retry the command. If this status code is returned, then the controller shall clear the Do Not Retry bit to ‘0’ in the Status field of the CQE (refer to Figure 124). The controller shall not return this status code unless the host has set the Advanced Command Retry Enable (ACRE) field to 1h in the Host Behavior Support feature(refer to section 5.21.1.22).

According the protocol define, NVME_SC_CMD_INTERRUPTED need retry. The error code NVME_SC_CMD_INTERRUPTED should not translate to BLK_STS_TARGET, because if the error code translate to BLK_STS_TARGET, dm-multipah will return error to application. NVME_SC_CMD_INTERRUPTED should translate to BLK_STS_IOERR by default, dm-multipath will fail over to other path retry the io.

We need a mechanism for retry in the current path first in some scenarios such as the target is dealing with an important emergency and need host retry the io after a defined time, because dm-multipath can not get Command Retry Delay (CRD) to control retry delay time. according the protocol define, the error code NVME_SC_CMD_INTERRUPTED is a good choice.

---
 drivers/nvme/host/core.c | 3 +--
 drivers/nvme/host/nvme.h | 6 ++++++
 2 files changed, 7 insertions(+), 2 deletions(-)
drivers/nvme/host/core.c +1 -2
index 66e6f7ca8947..b3307e317ece 100644
@@ -211,7 +211,6 @@ static blk_status_t nvme_error_status(u16 status)
 	case NVME_SC_CAP_EXCEEDED:
 		return BLK_STS_NOSPC;
 	case NVME_SC_LBA_RANGE:
-	case NVME_SC_CMD_INTERRUPTED:
 	case NVME_SC_NS_NOT_READY:
 		return BLK_STS_TARGET;
 	case NVME_SC_BAD_ATTRIBUTES:
@@ -243,7 +242,7 @@ static blk_status_t nvme_error_status(u16 status)
 
 static inline bool nvme_req_needs_retry(struct request *req)
 {
-	if (blk_noretry_request(req))
+	if (!nvme_req_local_retry(req) && blk_noretry_request(req))
 		return false;
 	if (nvme_req(req)->status & NVME_SC_DNR)
 		return false;
drivers/nvme/host/nvme.h +6 -0
index 1de3f9b827aa..3213aae8985c 100644
@@ -725,4 +725,10 @@ void nvme_hwmon_init(struct nvme_ctrl *ctrl);
 static inline void nvme_hwmon_init(struct nvme_ctrl *ctrl) { }
 #endif
 
+static inline bool nvme_req_local_retry(struct request *req)
+{
+	if (nvme_req(req)->status == NVME_SC_CMD_INTERRUPTED)
+		return true;
+	return false;
+}
 #endif /* _NVME_H */
Signature · 7 lines
2.16.4


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#1 lore raw

drivers/nvme/host/core.c static blk_status_t nvme_error_status(u16 status) · line 245

Quoted 7 lines
drivers/nvme/host/core.c +1 -1
@@ -243,7 +242,7 @@ static blk_status_t nvme_error_status(u16 status)
 
  static inline bool nvme_req_needs_retry(struct request *req)
  {
-	if (blk_noretry_request(req))
+	if (!nvme_req_local_retry(req) && blk_noretry_request(req))

Why is this needed?

Quoted 9 lines
drivers/nvme/host/core.c +0 -0
  		return false;
  	if (nvme_req(req)->status & NVME_SC_DNR)
  		return false;
drivers/nvme/host/nvme.h +0 -0
index 1de3f9b827aa..3213aae8985c 100644

drivers/nvme/host/nvme.h void nvme_hwmon_init(struct nvme_ctrl *ctrl); · line 734

Quoted 10 lines
drivers/nvme/host/nvme.h +6 -0
@@ -725,4 +725,10 @@ void nvme_hwmon_init(struct nvme_ctrl *ctrl);
  static inline void nvme_hwmon_init(struct nvme_ctrl *ctrl) { }
  #endif
 
+static inline bool nvme_req_local_retry(struct request *req)
+{
+	if (nvme_req(req)->status == NVME_SC_CMD_INTERRUPTED)
+		return true;
+	return false;
+}
  #endif /* _NVME_H */

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

author
#2 lore raw
Quoted 8 lines · Sagi Grimberg

drivers/nvme/host/core.c static blk_status_t nvme_error_status(u16 status) · line 244

Quoted 6 lines
drivers/nvme/host/core.c +1 -1
@@ -243,7 +242,7 @@ static blk_status_t nvme_error_status(u16 status)
     static inline bool nvme_req_needs_retry(struct request *req)
   {
-    if (blk_noretry_request(req))
+    if (!nvme_req_local_retry(req) && blk_noretry_request(req))

Why is this needed?

Because if work with dm-multipath or other multipath software, the multipath software will set cmd_flags as REQ_FAILFAST_TRANSPORT. If io return with any error, nvme will return io error to block layer, the multipath will set the path fault and retry in other path, but realy need retry in the current path in some scenarios. So we need introduce nvme_req_local_retry to check if need retry in the current path, if needed, do not need checking the cmd_flags through blk_noretry_request. According the protocol define, the error code NVME_SC_CMD_INTERRUPTED is a good choice.

The scenario which need retry in the current path:
Scenario 1: user configures a QoS policy on the storage system. Storage system need to tell host to retry io after a defined time when need control QoS. Host should retry the io in the current path instead of return io to block layer.

Scenario 3:many hosts(more than 100) access the same storage device. I/Os are not delivered at the same time in most cases,however a large number of I/Os may be burst delivered at the same time sometimes. In this case, the storage device can not treat all I/Os, needs host retry in the current path after a defined time.

Scenario 3: storage software upgrade inline. The storage system may need host retry in the current path after software upgrade completed.

Quoted 11 lines
Quoted 11 lines
drivers/nvme/host/core.c +0 -0
           return false;
       if (nvme_req(req)->status & NVME_SC_DNR)
           return false;
drivers/nvme/host/nvme.h +0 -0
index 1de3f9b827aa..3213aae8985c 100644
@@ -725,4 +725,10 @@ void nvme_hwmon_init(struct nvme_ctrl *ctrl);
   static inline void nvme_hwmon_init(struct nvme_ctrl *ctrl) { }
   #endif
Quoted 8 lines

drivers/nvme/host/nvme.h void nvme_hwmon_init(struct nvme_ctrl *ctrl); · line 734

Quoted 8 lines
drivers/nvme/host/nvme.h +5 -0
   +static inline bool nvme_req_local_retry(struct request *req)
+{
+    if (nvme_req(req)->status == NVME_SC_CMD_INTERRUPTED)
+        return true;
+    return false;
+}
   #endif /* _NVME_H */

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme