nvme: don't call blk_mq_{,un}quiesce_tagset when ctrl->tagset is NULL
Christoph Hellwig
maintainerPATCH 1/1 nvme: don't call blk_mq_{,un}quiesce_tagset when ctrl->tagset is NULL
The NVMe drivers support a mode where no tagset is allocated for the I/O queues and only the admin queue is usable. In that case ctrl->tagset is NULL and we must not call the block per-tagset quiesce helpers that dereference it.
---
It turns out we have a bunch of other quiesce calls that can happen when only the admin queue is live. So bring back the original quick fix to check for the tagset pointer in the quiesce helpers.
drivers/nvme/host/core.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/nvme/host/core.c +4 -0
@@ -5215,6 +5215,8 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);void nvme_quiesce_io_queues (struct nvme_ctrl *ctrl) { +if (!ctrl->tagset) +return ;if (!test_and_set_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_quiesce_tagset (ctrl->tagset);else
@@ -5224,6 +5226,8 @@ EXPORT_SYMBOL_GPL(nvme_quiesce_io_queues);void nvme_unquiesce_io_queues (struct nvme_ctrl *ctrl) { +if (!ctrl->tagset) +return ;if (test_and_clear_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_unquiesce_tagset (ctrl->tagset); }
Signature · 1 line
2.30.2
Quoted 40 lines · Christoph Hellwig
The NVMe drivers support a mode where no tagset is allocated for the I/O queues and only the admin queue is usable. In that case
ctrl->tagsetis NULL and we must not call the block per-tagset quiesce helpers that dereference it.Fixes 98d81f0df70c ("nvme: use blk_mq_[un]quiesce_tagset")Reported-by Gerd Bayer <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>---
It turns out we have a bunch of other quiesce calls that can happen when only the admin queue is live. So bring back the original quick fix to check for the tagset pointer in the quiesce helpers.
drivers/nvme/host/core.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/nvme/host/core.c +4
-0@@ -5215,6 +5215,8 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);void nvme_quiesce_io_queues (struct nvme_ctrl *ctrl) { +if (!ctrl->tagset) +return ;if (!test_and_set_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_quiesce_tagset (ctrl->tagset);else @@ -5224,6 +5226,8 @@ EXPORT_SYMBOL_GPL(nvme_quiesce_io_queues);void nvme_unquiesce_io_queues (struct nvme_ctrl *ctrl) { +if (!ctrl->tagset) +return ;if (test_and_clear_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_unquiesce_tagset (ctrl->tagset); }
Chao,
Quoted 4 lines · Chao Leng
Reviewed-by Chao Leng <[email protected]>Quoted 1 line · Christoph Hellwig
The NVMe drivers support a mode where no tagset is allocated for the I/O
Please avoid top posting.
-ck
Sagi Grimberg
maintainerQuoted 22 lines · Christoph Hellwig
The NVMe drivers support a mode where no tagset is allocated for the I/O queues and only the admin queue is usable. In that case
ctrl->tagsetis NULL and we must not call the block per-tagset quiesce helpers that dereference it.Fixes 98d81f0df70c ("nvme: use blk_mq_[un]quiesce_tagset")Reported-by Gerd Bayer <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>---
It turns out we have a bunch of other quiesce calls that can happen when only the admin queue is live. So bring back the original quick fix to check for the tagset pointer in the quiesce helpers.
drivers/nvme/host/core.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/nvme/host/core.c +0
-0
drivers/nvme/host/core.c EXPORT_SYMBOL_GPL(nvme_start_freeze); · line 5219
Quoted 5 lines
drivers/nvme/host/core.c +2
-0@@ -5215,6 +5215,8 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);void nvme_quiesce_io_queues (struct nvme_ctrl *ctrl) { +if (!ctrl->tagset) +return ;
Can this be placed in the pci code instead?
Quoted 12 lines
drivers/nvme/host/core.c +2
-0if (!test_and_set_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_quiesce_tagset (ctrl->tagset);else @@ -5224,6 +5226,8 @@ EXPORT_SYMBOL_GPL(nvme_quiesce_io_queues);void nvme_unquiesce_io_queues (struct nvme_ctrl *ctrl) { +if (!ctrl->tagset) +return ;if (test_and_clear_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_unquiesce_tagset (ctrl->tagset); }
Christoph Hellwig
maintainerQuoted 5 lines · Sagi Grimberg
Review
Quoted 3 lines
{(quoted diff) +2
-0+ if (!ctrl->tagset) + return;Can this be placed in the pci code instead?
There's nothing PCIe-specific about this case. All fabrics drivers support controllers with I/O queues and thus the I/O tagset, e.g. discovery controllers.
Sagi Grimberg
maintainerQuoted 10 lines
Quoted 6 lines
Review
Quoted 4 lines
{(quoted diff) +2
-0+ if (!ctrl->tagset) + return;Can this be placed in the pci code instead?
There's nothing PCIe-specific about this case. All fabrics drivers support controllers with I/O queues and thus the I/O tagset, e.g. discovery controllers.
All the other drivers check that they have IO queues before quiescing/unquiescing them.
Christoph Hellwig
maintainerQuoted 2 lines · Sagi Grimberg
All the other drivers check that they have IO queues before quiescing/unquiescing them.
The core firmware activation code does not.
Sagi Grimberg
maintainerQuoted 5 lines
Quoted 3 lines
All the other drivers check that they have IO queues before quiescing/unquiescing them.
The core firmware activation code does not.
Well, ideally it should, but maybe it would be more readable if we condition on ctrl->queue_count > 1.
Christoph Hellwig
maintainerQuoted 8 lines · Sagi Grimberg
Quoted 5 lines
Quoted 3 lines
All the other drivers check that they have IO queues before quiescing/unquiescing them.
The core firmware activation code does not.
Well, ideally it should, but maybe it would be more readable if we condition on
ctrl->queue_count> 1.
I find this more confusing to be honest, but if we get a consensus for that I can live with it.
Christoph Hellwig
maintainerQuoted 8 lines · Sagi Grimberg
Quoted 5 lines
Quoted 3 lines
All the other drivers check that they have IO queues before quiescing/unquiescing them.
The core firmware activation code does not.
Well, ideally it should, but maybe it would be more readable if we condition on
ctrl->queue_count> 1.
This would be the queue_count version. I have to say I much prefer the tagset check, as it is self-documenting and matches what nvme_queue_scan, nvme_cancel_tagset and nvme_cancel_admin_tagset do.
Christoph Hellwig
maintainerQuoted 4 lines · Christoph Hellwig
This would be the
queue_countversion. I have to say I much prefer the tagset check, as it is self-documenting and matches whatnvme_queue_scan,nvme_cancel_tagsetandnvme_cancel_admin_tagsetdo.
And here is the actual patch:
---
From 167d14d9613014142ee285c2912dadc089d80e86 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <[email protected]>
Date: Wed, 16 Nov 2022 08:14:46 +0100
Subject: nvme: don't call blk_mq_{,un}quiesce_tagset when ctrl->tagset is NULL
The NVMe drivers support a mode where no tagset is allocated for the I/O queues and only the admin queue is usable. In that case ctrl->queue_count is smaller than two and we must not call the block per-tagset quiesce helpers that dereference it.
--- drivers/nvme/host/core.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/nvme/host/core.c +4 -0
@@ -5215,6 +5215,8 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);void nvme_quiesce_io_queues (struct nvme_ctrl *ctrl) { +if (ctrl->queue_count <2 ) +return ;if (!test_and_set_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_quiesce_tagset (ctrl->tagset);else
@@ -5224,6 +5226,8 @@ EXPORT_SYMBOL_GPL(nvme_quiesce_io_queues);void nvme_unquiesce_io_queues (struct nvme_ctrl *ctrl) { +if (ctrl->queue_count <2 ) +return ;if (test_and_clear_bit (NVME_CTRL_STOPPED , &ctrl->flags))blk_mq_unquiesce_tagset (ctrl->tagset); }
Signature · 1 line
2.30.2
Christoph Hellwig
maintainerQuoted 4 lines · Christoph Hellwig
This would be the
queue_countversion. I have to say I much prefer the tagset check, as it is self-documenting and matches whatnvme_queue_scan,nvme_cancel_tagsetandnvme_cancel_admin_tagsetdo.
So no comments so far. Unless I hear more feedback I'd prefer to go with the ->tagset checks that matches what the other code is doing and also is more readable.
Sagi Grimberg
maintainerQuoted 9 lines
Quoted 5 lines
This would be the
queue_countversion. I have to say I much prefer the tagset check, as it is self-documenting and matches whatnvme_queue_scan,nvme_cancel_tagsetandnvme_cancel_admin_tagsetdo.So no comments so far. Unless I hear more feedback I'd prefer to go with the ->tagset checks that matches what the other code is doing and also is more readable.
OK, I'm not opposed to that anymore. We can take care of this area on a different occasion.