nvme-core: reduce io pause time when fail over
Subsystems (2)
PATCH 1/1 nvme-core: reduce io pause time when fail over
We test nvme over roce fail over with multipath when 1000 namespaces configured, io pause more than 10 seconds. The reason: nvme_stop_queues will quiesce all queues for each namespace when io timeout cause path error. Quiesce queue wait all ongoing dispatches finished through synchronize_rcu, need more than 10 milliseconds for each wait, thus io pause more than 10 seconds.
To reduce io pause time, nvme_stop_queues use blk_mq_quiesce_queue_nowait to quiesce the queue, nvme_stop_queues wait all ongoing dispatches completed after all queues has been quiesced.
--- drivers/nvme/host/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvme/host/core.c +2 -1
@@ -4323,8 +4323,9 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)down_read (&ctrl->namespaces_rwsem);list_for_each_entry (ns, &ctrl->namespaces, list) -blk_mq_quiesce_queue (ns->queue); +blk_mq_quiesce_queue_nowait (ns->queue);up_read (&ctrl->namespaces_rwsem); +synchronize_rcu (); }EXPORT_SYMBOL_GPL (nvme_stop_queues);
Signature · 7 lines
2.16.4 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 3 lines · Chao Leng
To reduce io pause time,
nvme_stop_queuesuseblk_mq_quiesce_queue_nowaitto quiesce the queue,nvme_stop_queueswait all ongoing dispatches completed after all queues has been quiesced.
The comment above blk_mq_quiesce_queue_nowait() wants to remove this function. I'm not sure we should be introducing more users if that's the case.
Using synchronize_rcu() at the end may be looking to much into blk-mq's internal quiesce implementation. It happens to be the right thing to do for non-blocking hctx, so this patch assumes that will be true for any nvme request_queue.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 5 lines
Quoted 4 lines
To reduce io pause time,
nvme_stop_queuesuseblk_mq_quiesce_queue_nowaitto quiesce the queue,nvme_stop_queueswait all ongoing dispatches completed after all queues has been quiesced.
Quoted 8 lines
The comment above
blk_mq_quiesce_queue_nowait()wants to remove this function. I'm not sure we should be introducing more users if that's the case.Using
synchronize_rcu()at the end may be looking to much into blk-mq's internal quiesce implementation. It happens to be the right thing to do for non-blocking hctx, so this patch assumes that will be true for any nvmerequest_queue.
nvme-tcp became a block hctx since we optimize to do network sends from inside queue_rq. So this should either split into two functions or nvme-core needs to look into BLK_MQ_F_BLOCKING and do the right thing.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 9 lines · Sagi Grimberg
Quoted 9 lines
Quoted 4 lines
To reduce io pause time,
nvme_stop_queuesuseblk_mq_quiesce_queue_nowaitto quiesce the queue,nvme_stop_queueswait all ongoing dispatches completed after all queues has been quiesced.The comment above
blk_mq_quiesce_queue_nowait()wants to remove this function. I'm not sure we should be introducing more users if that's the case.
Quoted 8 lines
Quoted 4 lines
Using
synchronize_rcu()at the end may be looking to much into blk-mq's internal quiesce implementation. It happens to be the right thing to do for non-blocking hctx, so this patch assumes that will be true for any nvmerequest_queue.nvme-tcp became a block hctx since we optimize to do network sends from inside
queue_rq. So this should either split into two functions or nvme-core needs to look intoBLK_MQ_F_BLOCKINGand do the right thing.
Another solution:introduce blk_mq_quiesce_queue_async, blk_mq_quiesce_queue_async do not wait all ongoing dispatches completed, if the hctx is not block hctx. The caller such as nvme_stop_queues wait all ongoing dispatches completed after all queues has been quiesced.
---
block/blk-mq.c | 28 ++++++++++++++++++++++++++++ drivers/nvme/host/core.c | 5 ++++- include/linux/blk-mq.h | 1 +
3 files changed, 33 insertions(+), 1 deletion(-)
block/blk-mq.c +0 -0
@@ -235,6 +235,34 @@ void blk_mq_quiesce_queue(struct request_queue *q)
}
EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
+/*
+ * blk_mq_quiesce_queue_async() - do not wait for non block hctx
+ * @q: request queue.
+ *
+ * Note: this function do not wait all ongoing dispatches completed
+ * if the hctx is not block hctx. Should be used to reduce wait time
+ * when quiesce batch queues at the same time. The caller should
+ * synchronous wait until all ongoing dispatches have finished
+ * after all queues has been done this.
+ */
+bool blk_mq_quiesce_queue_async(struct request_queue *q) +{
+ struct blk_mq_hw_ctx *hctx;
+ unsigned int i;
+ bool rcu = false;
+
+ blk_mq_quiesce_queue_nowait(q);
+
+ queue_for_each_hw_ctx(q, hctx, i) {
+ if (hctx->flags & BLK_MQ_F_BLOCKING)
+ synchronize_srcu(hctx->srcu);
+ else
+ rcu = true;
+ }
+ return rcu;
+}
+EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_async);
+
/*
* blk_mq_unquiesce_queue() - counterpart of blk_mq_quiesce_queue()
* @q: request queue.
drivers/nvme/host/core.c +0 -0
@@ -4320,11 +4320,14 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);
void nvme_stop_queues(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ bool rcu = false;
down_read(&ctrl->namespaces_rwsem); list_for_each_entry(ns, &ctrl->namespaces, list) - blk_mq_quiesce_queue(ns->queue);
+ rcu = rcu || blk_mq_quiesce_queue_async(ns->queue); up_read(&ctrl->namespaces_rwsem);
+ if (rcu)
+ synchronize_rcu();
}
EXPORT_SYMBOL_GPL(nvme_stop_queues);
include/linux/blk-mq.h +0 -0
@@ -515,6 +515,7 @@ void blk_mq_start_hw_queues(struct request_queue *q);
void blk_mq_start_stopped_hw_queue(struct blk_mq_hw_ctx *hctx, bool async); void blk_mq_start_stopped_hw_queues(struct request_queue *q, bool async); void blk_mq_quiesce_queue(struct request_queue *q);
+bool blk_mq_quiesce_queue_async(struct request_queue *q);
void blk_mq_unquiesce_queue(struct request_queue *q);
void blk_mq_delay_run_hw_queue(struct blk_mq_hw_ctx *hctx, unsigned long msecs); void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async);
Signature · 4 lines
_______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 6 lines
Quoted 6 lines
Quoted 5 lines
Using
synchronize_rcu()at the end may be looking to much into blk-mq's internal quiesce implementation. It happens to be the right thing to do for non-blocking hctx, so this patch assumes that will be true for any nvmerequest_queue.
Quoted 8 lines
Quoted 3 lines
nvme-tcp became a block hctx since we optimize to do network sends from inside
queue_rq. So this should either split into two functions or nvme-core needs to look intoBLK_MQ_F_BLOCKINGand do the right thing.Another solution:introduce
blk_mq_quiesce_queue_async,blk_mq_quiesce_queue_asyncdo not wait all ongoing dispatches completed, if the hctx is not block hctx. The caller such asnvme_stop_queueswait all ongoing dispatches completed after all queues has been quiesced.
But then you maintain the problem for blocking hctx, so its not a good solution (and also its not a good practice to abuse the return code like you did IMO).
What we need is something like the following untested patches:
[1]:
--
block/blk-mq.c +17 -12
@@ -209,23 +209,12 @@ void blk_mq_quiesce_queue_nowait(struct request_queue *q) }EXPORT_SYMBOL_GPL (blk_mq_quiesce_queue_nowait); -/** -* blk_mq_quiesce_queue() - wait until all ongoing dispatches have finished -* @q: request queue. -* -* Note: this function does not prevent that the struct request end_io() -* callback function is invoked. Once this function is returned, we make -* sure no dispatch can happen until the queue is unquiesced via -* blk_mq_unquiesce_queue(). -*/ -void blk_mq_quiesce_queue (struct request_queue *q) +void blk_mq_quiesce_queue_sync (struct request_queue *q) {struct blk_mq_hw_ctx *hctx;unsigned int i;bool rcu = false; -blk_mq_quiesce_queue_nowait (q); -queue_for_each_hw_ctx (q, hctx, i) {if (hctx->flags &BLK_MQ_F_BLOCKING )synchronize_srcu (hctx->srcu);
@@ -235,6 +224,22 @@ void blk_mq_quiesce_queue(struct request_queue *q)if (rcu)synchronize_rcu (); } +EXPORT_SYMBOL_GPL (blk_mq_quiesce_queue_sync); + +/** +* blk_mq_quiesce_queue() - wait until all ongoing dispatches have finished +* @q: request queue. +* +* Note: this function does not prevent that the struct request end_io() +* callback function is invoked. Once this function is returned, we make +* sure no dispatch can happen until the queue is unquiesced via +* blk_mq_unquiesce_queue(). +*/ +void blk_mq_quiesce_queue (struct request_queue *q) +{ +blk_mq_quiesce_queue_nowait (q); +blk_mq_quiesce_queue_sync (q); +}EXPORT_SYMBOL_GPL (blk_mq_quiesce_queue);/*
include/linux/blk-mq.h +0 -0
@@ -532,6 +532,7 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap); void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int
nr_hw_queues);
void blk_mq_quiesce_queue_nowait(struct request_queue *q);
+void blk_mq_quiesce_queue_sync(struct request_queue *q);
unsigned int blk_mq_rq_cpu(struct request *rq);
--
[2]:
--
drivers/nvme/host/core.c +4 -2
@@ -4535,8 +4535,11 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)down_read (&ctrl->namespaces_rwsem);list_for_each_entry (ns, &ctrl->namespaces, list) -blk_mq_quiesce_queue (ns->queue); +blk_mq_quiesce_queue_nowait (ns->queue); +/* all namespaces share hctxs, so sync just once */ +blk_mq_quiesce_queue_sync (ns->queue);up_read (&ctrl->namespaces_rwsem); + }EXPORT_SYMBOL_GPL (nvme_stop_queues); --
I can send formal patches if you care to test it...
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerdrivers/nvme/host/core.c void nvme_stop_queues(struct nvme_ctrl *ctrl) · line 4540
Quoted 2 lines · Sagi Grimberg
drivers/nvme/host/core.c +2
-0+/* all namespaces share hctxs, so sync just once */ +blk_mq_quiesce_queue_sync (ns->queue);
They don't share hctx's. An hctx is a property of a specific request_queue, and each namespace has their own.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 6 lines
drivers/nvme/host/core.c void nvme_stop_queues(struct nvme_ctrl *ctrl) · line 4540
Quoted 3 lines
drivers/nvme/host/core.c +2
-0+/* all namespaces share hctxs, so sync just once */ +blk_mq_quiesce_queue_sync (ns->queue);They don't share hctx's. An hctx is a property of a specific
request_queue, and each namespace has their own.
ugh, right...
But I still think that we should wait for all the srcus together...
Perhaps we should do call_srcu on all and wait for all in one pass. But I don't want the driver to do this without a proper blk-mq interface..
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme