nvme: cancel requests for real
Subsystems (2)
Keith Busch
maintainerPATCH 1/1 nvme: cancel requests for real
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/nvme/host/core.c +1 -1
@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)return true;nvme_req (req)->status =NVME_SC_HOST_ABORTED_CMD ; -blk_mq_complete_request (req); +while (!blk_mq_complete_request (req));return true; }EXPORT_SYMBOL_GPL (nvme_cancel_request);
Signature · 7 lines
2.24.1 _______________________________________________ linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Hi Keith,
Quoted 15 lines · Keith Busch
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +0
-0
drivers/nvme/host/core.c bool nvme_cancel_request(struct request *req, void *data, bool reserved) · line 313
Quoted 5 lines
drivers/nvme/host/core.c +1
-1@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)return true;nvme_req (req)->status =NVME_SC_HOST_ABORTED_CMD ; -blk_mq_complete_request (req); +while (!blk_mq_complete_request (req));
If the probability is configured to fail every time with 100% probability, we would run into endless loop?
Dongli Zhang
_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 18 lines · Dongli Zhang
Quoted 17 lines · Keith Busch
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +0
-0@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)return true;
Quoted 5 lines
drivers/nvme/host/core.c bool nvme_cancel_request(struct request *req, void *data, bool reserved) · line 313
Quoted 3 lines
drivers/nvme/host/core.c +1
-1nvme_req (req)->status =NVME_SC_HOST_ABORTED_CMD ; -blk_mq_complete_request (req); +while (!blk_mq_complete_request (req));If the probability is configured to fail every time with 100% probability, we would run into endless loop?
If completing requests is not possible, it's already endless. This is just a tighter loop in that case.
_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 16 lines
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +0
-0
drivers/nvme/host/core.c bool nvme_cancel_request(struct request *req, void *data, bool reserved) · line 313
Quoted 5 lines
drivers/nvme/host/core.c +1
-1@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)return true;nvme_req (req)->status =NVME_SC_HOST_ABORTED_CMD ; -blk_mq_complete_request (req); +while (!blk_mq_complete_request (req));
Maybe add cond_resched in the loop? just to play nice in case something is wrong?
_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 11 lines · Sagi Grimberg
Quoted 11 lines
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
Signed-off-by Keith Busch <[email protected]>---
drivers/nvme/host/core.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 13 lines
drivers/nvme/host/core.c bool nvme_cancel_request(struct request *req, · line 313
Quoted 10 lines
drivers/nvme/host/core.c +1
-1@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved) return true; nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD; - blk_mq_complete_request(req); + while (!blk_mq_complete_request(req));Maybe add
cond_reschedin the loop? just to play nice in case something is wrong?
Is this a nvme issue or a block issue? My original fix was in the block layer.
block/blk-timeout.c +0 -0
@@ -22,10 +22,10 @@ static int __init setup_fail_io_timeout(char *str)
int blk_should_fake_timeout(struct request_queue *q) {
- if (!test_bit(QUEUE_FLAG_FAIL_IO, &q->queue_flags)) - return 0;
-
- return should_fail(&fail_io_timeout, 1);
+ if (test_bit(QUEUE_FLAG_FAIL_IO, &q->queue_flags) && + !blk_queue_quiesced(q))
+ return should_fail(&fail_io_timeout, 1); + return 0;
}
static int __init fail_io_timeout_debugfs(void)
Signature · 10 lines
Alan _______________________________________________ linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Christoph Hellwig
maintainerQuoted 5 lines · Alan Adamson
block/blk-timeout.c +0
-0@@ -22,10 +22,10 @@ static int __init setup_fail_io_timeout(char *str)int blk_should_fake_timeout (struct request_queue *q)
block/blk-timeout.c static int __init setup_fail_io_timeout(char *str) · line 29
Quoted 10 lines
block/blk-timeout.c +4
-4{ - if (!test_bit(QUEUE_FLAG_FAIL_IO, &q->queue_flags)) - return 0; - - return should_fail(&fail_io_timeout, 1); + if (test_bit(QUEUE_FLAG_FAIL_IO, &q->queue_flags) && + !blk_queue_quiesced(q)) + return should_fail(&fail_io_timeout, 1); + return 0; }
While that is way better than the loop I think this encodes too way too many assumptions.
_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Christoph Hellwig
maintainerQuoted 15 lines · Keith Busch
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +0
-0
drivers/nvme/host/core.c bool nvme_cancel_request(struct request *req, void *data, bool reserved) · line 314
Quoted 6 lines
drivers/nvme/host/core.c +1
-1@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)return true;nvme_req (req)->status =NVME_SC_HOST_ABORTED_CMD ; -blk_mq_complete_request (req); +while (!blk_mq_complete_request (req));return true;
This is just disgusting. Take a look at how blk_mq_complete_request is implemented, and what might be a better function to export and call here, or any other error handling context. Bonus points for finding a saner name for it.
I'm also thinking that whole idea of randomly not doing anything in blk_mq_complete_request is just a bad idea vs doing targeted hooks in the actual drivers.
_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 18 lines · Christoph Hellwig
Quoted 17 lines · Keith Busch
Once the driver decides to cancel requests, the concept of those requests timing out should no longer exist. Since we can't stop fake timeouts from preventing a forced reclaim, continue completing the same request until the block layer isn't told to pretend that didn't happen.
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +0
-0@@ -310,7 +310,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)return true;
Quoted 6 lines
drivers/nvme/host/core.c bool nvme_cancel_request(struct request *req, void *data, bool reserved) · line 314
Quoted 4 lines
drivers/nvme/host/core.c +1
-1nvme_req (req)->status =NVME_SC_HOST_ABORTED_CMD ; -blk_mq_complete_request (req); +while (!blk_mq_complete_request (req));return true;This is just disgusting.
Thanks for noticing. :)
Quoted 3 lines
I'm also thinking that whole idea of randomly not doing anything in
blk_mq_complete_requestis just a bad idea vs doing targeted hooks in the actual drivers.
Yeah, I don't like fake test code in the middle of production code. Testing code shouldn't modify the code being testing. IMO, leave real stuff alone and create a thunk for error injection, preferably one specific to the low-level driver. I know we can do that on x86, but not so sure about other archs.
_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme