Skip to contentSkip to conversation

scsi: core: Clear cmd->flags for terminal queuing error

RFC PATCH Superseded John Garry 2 participants 4 messages Sep 8 – Sep 9

Subsystems (2)

v1 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

John Garry

author
#0 lore raw

RFC PATCH 1/1 scsi: core: Clear cmd->flags for terminal queuing error

If a request can't be queued in scsi_queue_rq(), then we error out with a blk_status_t code. Such a scenario may be the scsi_device has its transport offline.

For BLK_STS_RESOURCE or BLK_STS_AGAIN, the request will be requeued in the block layer. For other errors, the block layer ends the request and so we won't see it again (until recycled).

Normally when a request completes scmd->flags is cleared in scsi_end_request().

However for the error scenario mentioned above, scsi_end_request() is not called and so scmd->flags is not cleared. As such, when the request is recycled we see the old value in scmd->flags.

Fix this issue by clearing scmd->flags in the error scenario mentioned.

---
Setting as an RFC as I am not 100% sure that this is correct as there are lots of paths involved. Sashiko pointed out a related issue in
https://lore.kernel.org/linux-scsi/[email protected]/,
but blk_mq_cleanup_rq() -> scsi_cleanup_rq() -> cmd->flags = 0 is only called for by DM code.

drivers/scsi/scsi_lib.c +1 -0
index af27fd3df8d4..295f8b4afaa7 100644
@@ -1967,6 +1967,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 		 */
 		if (req->rq_flags & RQF_DONTPREP)
 			scsi_mq_uninit_cmd(cmd);
+		cmd->flags = 0;
 		scsi_run_queue_async(sdev);
 		break;
 	}
Signature · 1 line
2.43.0
#1 lore raw
Quoted 30 lines · John Garry

If a request can't be queued in scsi_queue_rq(), then we error out with a blk_status_t code. Such a scenario may be the scsi_device has its transport offline.

For BLK_STS_RESOURCE or BLK_STS_AGAIN, the request will be requeued in the block layer. For other errors, the block layer ends the request and so we won't see it again (until recycled).

Normally when a request completes scmd->flags is cleared in scsi_end_request().

However for the error scenario mentioned above, scsi_end_request() is not called and so scmd->flags is not cleared. As such, when the request is recycled we see the old value in scmd->flags.

Fix this issue by clearing scmd->flags in the error scenario mentioned.

---
Setting as an RFC as I am not 100% sure that this is correct as there are lots of paths involved. Sashiko pointed out a related issue in
https://lore.kernel.org/linux-scsi/[email protected]/,
but blk_mq_cleanup_rq() -> scsi_cleanup_rq() -> cmd->flags = 0 is only called for by DM code.

drivers/scsi/scsi_lib.c +0 -0
index af27fd3df8d4..295f8b4afaa7 100644

drivers/scsi/scsi_lib.c static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, · line 1973

Quoted 7 lines
drivers/scsi/scsi_lib.c +1 -0
@@ -1967,6 +1967,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
  		 */
  		if (req->rq_flags & RQF_DONTPREP)
  			scsi_mq_uninit_cmd(cmd);
+		cmd->flags = 0;
  		scsi_run_queue_async(sdev);
  		break;
  	}

Is there agreement that all block drivers are affected that define a .cleanup_rq() callback? Would this be a valid alternative (untested)?

block/blk-mq.c +6 -0
index a26a11c73ee3..0be933e26d67 100644
@@ -2126,6 +2126,12 @@ bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *hctx, struct list_head *list,
 			blk_mq_handle_dev_resource(rq, list);
 			goto out;
 		default:
+			/*
+			 * Make sure to release all allocated resources when
+			 * we hit an error, as we will never see this command
+			 * again.
+			 */
+			blk_mq_cleanup_rq(rq);
 			blk_mq_end_request(rq, ret);
 		}
 	} while (!list_empty(list));
drivers/scsi/scsi_lib.c +0 -7
index 48aab0df30b7..636854bffd1e 100644
@@ -1960,13 +1960,6 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 			cmd->result = DID_NO_CONNECT << 16;
 		else
 			cmd->result = DID_ERROR << 16;
-		/*
-		 * Make sure to release all allocated resources when
-		 * we hit an error, as we will never see this command
-		 * again.
-		 */
-		if (req->rq_flags & RQF_DONTPREP)
-			scsi_mq_uninit_cmd(cmd);
 		scsi_run_queue_async(sdev);
 		break;
 	}

Thanks,

Bart.

author
#2 lore raw
Quoted 26 lines · Bart Van Assche
Quoted 25 lines · John Garry

If a request can't be queued in scsi_queue_rq(), then we error out with a blk_status_t code. Such a scenario may be the scsi_device has its transport offline.

For BLK_STS_RESOURCE or BLK_STS_AGAIN, the request will be requeued in the block layer. For other errors, the block layer ends the request and so we won't see it again (until recycled).

Normally when a request completes scmd->flags is cleared in scsi_end_request().

However for the error scenario mentioned above, scsi_end_request() is not called and so scmd->flags is not cleared. As such, when the request is recycled we see the old value in scmd->flags.

Fix this issue by clearing scmd->flags in the error scenario mentioned.

---
Setting as an RFC as I am not 100% sure that this is correct as there are lots of paths involved. Sashiko pointed out a related issue in
https://lore.kernel.org/linux- scsi/[email protected]/, but blk_mq_cleanup_rq() -> scsi_cleanup_rq() -> cmd->flags = 0 is only called for by DM code.

Quoted 16 lines
Quoted 12 lines
drivers/scsi/scsi_lib.c +1 -0
index af27fd3df8d4..295f8b4afaa7 100644
@@ -1967,6 +1967,7 @@ static blk_status_t scsi_queue_rq(struct 
 blk_mq_hw_ctx *hctx,
            */
           if (req->rq_flags & RQF_DONTPREP)
               scsi_mq_uninit_cmd(cmd);
+        cmd->flags = 0;
           scsi_run_queue_async(sdev);
           break;
Quoted 1 line

      }

Is there agreement that all block drivers are affected that define a .cleanup_rq() callback?

That is only SCSI (which is affected) and SCSI does define that (callback).

Quoted 1 line

Would this be a valid alternative (untested)?

Yes, logically this looks the same. However does scsi_cleanup_rq() need the unsetting of RQF_DONTPREP at ***, below. I suppose that it is not doing any harm if the req is going to end.

static void scsi_cleanup_rq(struct request *rq)

{
	struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(rq);

	cmd->flags = 0;

	if (rq->rq_flags & RQF_DONTPREP) {
		scsi_mq_uninit_cmd(cmd);
		rq->rq_flags &= ~RQF_DONTPREP; ***
	}
}
Quoted 37 lines
block/blk-mq.c +6 -0
index a26a11c73ee3..0be933e26d67 100644
@@ -2126,6 +2126,12 @@ bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx 
 *hctx, struct list_head *list,
              blk_mq_handle_dev_resource(rq, list);
              goto out;
          default:
+            /*
+             * Make sure to release all allocated resources when
+             * we hit an error, as we will never see this command
+             * again.
+             */
+            blk_mq_cleanup_rq(rq);
              blk_mq_end_request(rq, ret);
          }

     } while (!list_empty(list));

drivers/scsi/scsi_lib.c +0 -7
index 48aab0df30b7..636854bffd1e 100644
@@ -1960,13 +1960,6 @@ static blk_status_t scsi_queue_rq(struct 
 blk_mq_hw_ctx *hctx,
              cmd->result = DID_NO_CONNECT << 16;
          else
              cmd->result = DID_ERROR << 16;
-        /*
-         * Make sure to release all allocated resources when
-         * we hit an error, as we will never see this command
-         * again.
-         */
-        if (req->rq_flags & RQF_DONTPREP)
-            scsi_mq_uninit_cmd(cmd);
          scsi_run_queue_async(sdev);
          break;
Quoted 2 lines

     }

cheers,
John

#3 lore raw
Quoted 1 line · John Garry

Yes, logically this looks the same. However does scsi_cleanup_rq() need the unsetting of RQF_DONTPREP at ***, below. I suppose that it is not doing any harm if the req is going to end.

It's not necessary in the scsi_queue_rq() code path that we are discussing but it improves consistency because clearing RQF_DONTPREP reflects that the scsi_mq_uninit_cmd() call undid the changes made by the code that sets RQF_DONTPREP.

Thanks,

Bart.