fix nvme sqsize on off regression
Subsystems (3)
Christoph Hellwig
maintainerHi all,
this series fixes up the sqsize regression that Hugh reported. The first two patches are the real fixes, and it might make sense if Linus just picks them directly before -rc1. The other two just clean things up a bit to avoid pitfalls like this in the future.
Diffstat:
apple.c | 3 +-- core.c | 21 ++++++++++++++++++--- fabrics.c | 2 +- fc.c | 12 ++++++------ pci.c | 27 ++++++++++++--------------- rdma.c | 23 +++++++++++------------ tcp.c | 12 ++++++------ 7 files changed, 55 insertions(+), 45 deletions(-)
Christoph Hellwig
maintainerPATCH 1/4 nvme: fix setting the queue depth in nvme_alloc_io_tag_set
While the CAP.MQES field in NVMe is a 0s based filed with a natural one off, we also need to account for the queue wrap condition and fix undo the one off again in nvme_alloc_io_tag_set. This was never properly done by the fabrics drivers, but they don't seem to care because there is no actual physical queue that can wrap around, but it became a problem when converting over the PCIe driver. Also add back the BLK_MQ_MAX_DEPTH check that was lost in the same commit.
--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/nvme/host/core.c +1 -1
@@ -4897,7 +4897,7 @@ int nvme_alloc_io_tag_set(struct nvme_ctrl *ctrl, struct blk_mq_tag_set *set,memset (set,0 ,sizeof (*set)); set->ops = ops; - set->queue_depth = ctrl->sqsize +1 ; + set->queue_depth =min_t (unsigned, ctrl->sqsize,BLK_MQ_MAX_DEPTH -1 );/* * Some Apple controllers requires tags to be unique across admin and * the (only) I/O queue, so reserve the first 32 tags of the I/O queue.
Signature · 1 line
2.35.1
Sagi Grimberg
maintainerRe: [PATCH 1/4] nvme: fix setting the queue depth in nvme_alloc_io_tag_set
Quoted 10 lines
While the CAP.MQES field in NVMe is a 0s based filed with a natural one off, we also need to account for the queue wrap condition and fix undo the one off again in
nvme_alloc_io_tag_set. This was never properly done by the fabrics drivers, but they don't seem to care because there is no actual physical queue that can wrap around, but it became a problem when converting over the PCIe driver. Also add back theBLK_MQ_MAX_DEPTHcheck that was lost in the same commit.Fixes 0da7feaa5913 ("nvme-pci: use the tagset alloc/free helpers")
This looks fine, but does this actually fix anything?
How does nvme-pci ever set sqsize anywhere close to BLK_MQ_MAX_DEPTH ? NVME_PCI_MAX_QUEUE_SIZE is 4095...
Quoted 19 lines
Reported-by Hugh Dickins <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +1
-1@@ -4897,7 +4897,7 @@ int nvme_alloc_io_tag_set(struct nvme_ctrl *ctrl, struct blk_mq_tag_set *set,memset (set,0 ,sizeof (*set)); set->ops = ops; - set->queue_depth = ctrl->sqsize +1 ; + set->queue_depth =min_t (unsigned, ctrl->sqsize,BLK_MQ_MAX_DEPTH -1 );/* * Some Apple controllers requires tags to be unique across admin and * the (only) I/O queue, so reserve the first 32 tags of the I/O queue.
Sagi Grimberg
maintainerRe: [PATCH 1/4] nvme: fix setting the queue depth in nvme_alloc_io_tag_set
Quoted 9 lines
Quoted 9 lines
While the CAP.MQES field in NVMe is a 0s based filed with a natural one off, we also need to account for the queue wrap condition and fix undo the one off again in nvme_alloc_io_tag_set. This was never properly done by the fabrics drivers, but they don't seem to care because there is no actual physical queue that can wrap around, but it became a problem when converting over the PCIe driver. Also add back the
BLK_MQ_MAX_DEPTHcheck that was lost in the same commit.
Quoted 5 lines
Quoted 1 line
Fixes 0da7feaa5913 ("nvme-pci: use the tagset alloc/free helpers")This looks fine, but does this actually fix anything?
How does nvme-pci ever set sqsize anywhere close toBLK_MQ_MAX_DEPTH?NVME_PCI_MAX_QUEUE_SIZEis 4095...
Disregard this comment.
But this means that fabrics drivers automatically get 1 less queue entry? with this patch alone?
Quoted 21 lines
Quoted 21 lines
Reported-by Hugh Dickins <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>---
drivers/nvme/host/core.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +1
-2@@ -4897,7 +4897,7 @@ int nvme_alloc_io_tag_set(struct nvme_ctrl *ctrl, struct blk_mq_tag_set *set, memset(set, 0, sizeof(*set)); set->ops = ops; - set->queue_depth = ctrl->sqsize + 1; + set->queue_depth = min_t(unsigned, ctrl->sqsize, BLK_MQ_MAX_DEPTH - 1); /* * Some Apple controllers requires tags to be unique across admin and* the (only) I/O queue, so reserve the first 32 tags of the I/O queue.
Christoph Hellwig
maintainerRe: [PATCH 1/4] nvme: fix setting the queue depth in nvme_alloc_io_tag_set
Quoted 2 lines · Sagi Grimberg
But this means that fabrics drivers automatically get 1 less queue entry? with this patch alone?
Yes. And also with the entire series.
Re: [PATCH 1/4] nvme: fix setting the queue depth in nvme_alloc_io_tag_set
In reply to Christoph HellwigQuoted 11 lines · Christoph Hellwig
While the CAP.MQES field in NVMe is a 0s based filed with a natural one off, we also need to account for the queue wrap condition and fix undo the one off again in
nvme_alloc_io_tag_set. This was never properly done by the fabrics drivers, but they don't seem to care because there is no actual physical queue that can wrap around, but it became a problem when converting over the PCIe driver. Also add back theBLK_MQ_MAX_DEPTHcheck that was lost in the same commit.Fixes 0da7feaa5913 ("nvme-pci: use the tagset alloc/free helpers")Reported-by Hugh Dickins <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>
Quoted 21 lines
--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/nvme/host/core.c +1
-1@@ -4897,7 +4897,7 @@ int nvme_alloc_io_tag_set(struct nvme_ctrl *ctrl, struct blk_mq_tag_set *set,memset (set,0 ,sizeof (*set)); set->ops = ops; - set->queue_depth = ctrl->sqsize +1 ; + set->queue_depth =min_t (unsigned, ctrl->sqsize,BLK_MQ_MAX_DEPTH -1 );/* * Some Apple controllers requires tags to be unique across admin and * the (only) I/O queue, so reserve the first 32 tags of the I/O queue. Signature · 1 line
2.35.1
Christoph Hellwig
maintainerPATCH 2/4 nvme-pci: update sqsize when adjusting the queue depth
In reply to Christoph HellwigUpdate the core sqsize field in addition to the PCIe-specific q_depth field as the core tagset allocation helpers rely on it.
--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/nvme/host/pci.c +5 -4
@@ -2333,10 +2333,12 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)if (dev->cmb_use_sqes) { result =nvme_cmb_qdepth (dev, nr_io_queues,sizeof (struct nvme_command )); -if (result >0 ) +if (result >0 ) { dev->q_depth = result; -else + dev->ctrl.sqsize = result -1 ; + }else { dev->cmb_use_sqes = false; + } } do {
@@ -2537,7 +2539,6 @@ static int nvme_pci_enable(struct nvme_dev *dev) dev->q_depth =min_t (u32,NVME_CAP_MQES (dev->ctrl.cap) +1 , io_queue_depth); - dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ dev->db_stride =1 <<NVME_CAP_STRIDE (dev->ctrl.cap); dev->dbs = dev->bar +4096 ;
@@ -2578,7 +2579,7 @@ static int nvme_pci_enable(struct nvme_dev *dev)dev_warn (dev->ctrl.device,"IO queue depth clamped to %d\n" , dev->q_depth); } - + dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ nvme_map_cmb (dev);
Signature · 1 line
2.35.1
Sagi Grimberg
maintainerRe: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
Quoted 38 lines
Update the core sqsize field in addition to the PCIe-specific
q_depthfield as the core tagset allocation helpers rely on it.Fixes 0da7feaa5913 ("nvme-pci: use the tagset alloc/free helpers")Signed-off-by Christoph Hellwig <[email protected]>--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)drivers/nvme/host/pci.c +4
-3@@ -2333,10 +2333,12 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)if (dev->cmb_use_sqes) { result =nvme_cmb_qdepth (dev, nr_io_queues,sizeof (struct nvme_command )); -if (result >0 ) +if (result >0 ) { dev->q_depth = result; -else + dev->ctrl.sqsize = result -1 ; + }else { dev->cmb_use_sqes = false; + } } do {@@ -2537,7 +2539,6 @@ static int nvme_pci_enable(struct nvme_dev *dev) dev->q_depth =min_t (u32,NVME_CAP_MQES (dev->ctrl.cap) +1 , io_queue_depth); - dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ dev->db_stride =1 <<NVME_CAP_STRIDE (dev->ctrl.cap); dev->dbs = dev->bar +4096 ;
drivers/nvme/host/pci.c static int nvme_pci_enable(struct nvme_dev *dev) · line 2585
Quoted 8 lines
drivers/nvme/host/pci.c +1
-1@@ -2578,7 +2579,7 @@ static int nvme_pci_enable(struct nvme_dev *dev)dev_warn (dev->ctrl.device,"IO queue depth clamped to %d\n" , dev->q_depth); } - + dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ nvme_map_cmb (dev);
OK, I think I understand now.
But if you want to patches 1+2 to be taken before 3+4, I'd make sqsize to be a 0's based value of q_depth after we subtract one entry for queue wraparound. So others would not get affected from patches 1+2. Then in one batch make all assignments to sqsize 1's based like in patch 3 and change nvme_alloc_io_tagset at the same time.
Not super critical, but there is a small chance that if 1+2 are taken before 3+4, someone may complain...
Christoph Hellwig
maintainerRe: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
Quoted 5 lines · Sagi Grimberg
But if you want to patches 1+2 to be taken before 3+4, I'd make sqsize to be a 0's based value of
q_depthafter we subtract one entry for queue wraparound. So others would not get affected from patches 1+2. Then in one batch make all assignments to sqsize 1's based like in patch 3 and changenvme_alloc_io_tagsetat the same time.
1 doe not make any difference in the 0s based vs not values, they just reduce the queue size by 1 intentionally.
Sagi Grimberg
maintainerRe: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
Quoted 9 lines
Quoted 6 lines
But if you want to patches 1+2 to be taken before 3+4, I'd make sqsize to be a 0's based value of
q_depthafter we subtract one entry for queue wraparound. So others would not get affected from patches 1+2. Then in one batch make all assignments to sqsize 1's based like in patch 3 and changenvme_alloc_io_tagsetat the same time.1 doe not make any difference in the 0s based vs not values, they just reduce the queue size by 1 intentionally.
Not sure who is they here...
Anyways, I guess we'll have to wait and see if someone happens to care if his/hers effective queuing is reduced by 1 all of a sudden...
Keith Busch
maintainerRe: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
Quoted 7 lines · Sagi Grimberg
Quoted 7 lines
Quoted 6 lines
But if you want to patches 1+2 to be taken before 3+4, I'd make sqsize to be a 0's based value of
q_depthafter we subtract one entry for queue wraparound. So others would not get affected from patches 1+2. Then in one batch make all assignments to sqsize 1's based like in patch 3 and changenvme_alloc_io_tagsetat the same time.
Quoted 7 lines
Quoted 2 lines
1 doe not make any difference in the 0s based vs not values, they just reduce the queue size by 1 intentionally.
Not sure who is they here...
Anyways, I guess we'll have to wait and see if someone happens to care if his/hers effective queuing is reduced by 1 all of a sudden...
All nvme transports require the host driver leave one queue entry unused. The driver was not spec compliant prior to patches 1 and 2. I'm not sure if that "queue full" definition makes sense for fabrics lacking a shared ring, but the spec says we have to work that way.
Sagi Grimberg
maintainerRe: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
Quoted 12 lines
Quoted 12 lines
Quoted 9 lines
Quoted 6 lines
But if you want to patches 1+2 to be taken before 3+4, I'd make sqsize to be a 0's based value of
q_depthafter we subtract one entry for queue wraparound. So others would not get affected from patches 1+2. Then in one batch make all assignments to sqsize 1's based like in patch 3 and changenvme_alloc_io_tagsetat the same time.1 doe not make any difference in the 0s based vs not values, they just reduce the queue size by 1 intentionally.
Not sure who is they here...
Quoted 7 lines
Quoted 2 lines
Anyways, I guess we'll have to wait and see if someone happens to care if his/hers effective queuing is reduced by 1 all of a sudden...
All nvme transports require the host driver leave one queue entry unused. The driver was not spec compliant prior to patches 1 and 2. I'm not sure if that "queue full" definition makes sense for fabrics lacking a shared ring, but the spec says we have to work that way.
fabrics already reserves one entry for a connect cmd, so effectively it was reserving a slot already...
Keith Busch
maintainerRe: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
Quoted 15 lines · Sagi Grimberg
Quoted 15 lines
Quoted 14 lines
Quoted 9 lines
Quoted 6 lines
But if you want to patches 1+2 to be taken before 3+4, I'd make sqsize to be a 0's based value of
q_depthafter we subtract one entry for queue wraparound. So others would not get affected from patches 1+2. Then in one batch make all assignments to sqsize 1's based like in patch 3 and changenvme_alloc_io_tagsetat the same time.1 doe not make any difference in the 0s based vs not values, they just reduce the queue size by 1 intentionally.
Not sure who is they here...
Anyways, I guess we'll have to wait and see if someone happens to care if his/hers effective queuing is reduced by 1 all of a sudden...
Quoted 7 lines
Quoted 4 lines
All nvme transports require the host driver leave one queue entry unused. The driver was not spec compliant prior to patches 1 and 2. I'm not sure if that "queue full" definition makes sense for fabrics lacking a shared ring, but the spec says we have to work that way.
fabrics already reserves one entry for a connect cmd, so effectively it was reserving a slot already...
nvmf_connect_io_queue() sets the cmd.connect.sqsize to ctrl->sqsize, so doing a +1 on that value for the tagset is definitely not right. A spec complaint target should have been halting and setting CSTS.CFS if we actually used all tags.
Re: [PATCH 2/4] nvme-pci: update sqsize when adjusting the queue depth
In reply to Christoph HellwigQuoted 5 lines · Christoph Hellwig
Update the core sqsize field in addition to the PCIe-specific
q_depthfield as the core tagset allocation helpers rely on it.Fixes 0da7feaa5913 ("nvme-pci: use the tagset alloc/free helpers")Signed-off-by Christoph Hellwig <[email protected]>
(I'm not saying Tested-by, because although the equivalent of this was in my testing last night, I don't believe it would have been hitting the cases which this patch repairs.
And I won't be looking at 3/4 or 4/4 today: but certainly agree that reducing the number of fields which need to be kept in sync is a good step forward.)
Quoted 42 lines
--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)drivers/nvme/host/pci.c +5
-4@@ -2333,10 +2333,12 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)if (dev->cmb_use_sqes) { result =nvme_cmb_qdepth (dev, nr_io_queues,sizeof (struct nvme_command )); -if (result >0 ) +if (result >0 ) { dev->q_depth = result; -else + dev->ctrl.sqsize = result -1 ; + }else { dev->cmb_use_sqes = false; + } } do {@@ -2537,7 +2539,6 @@ static int nvme_pci_enable(struct nvme_dev *dev) dev->q_depth =min_t (u32,NVME_CAP_MQES (dev->ctrl.cap) +1 , io_queue_depth); - dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ dev->db_stride =1 <<NVME_CAP_STRIDE (dev->ctrl.cap); dev->dbs = dev->bar +4096 ;@@ -2578,7 +2579,7 @@ static int nvme_pci_enable(struct nvme_dev *dev)dev_warn (dev->ctrl.device,"IO queue depth clamped to %d\n" , dev->q_depth); } - + dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ nvme_map_cmb (dev);Signature · 1 line
2.35.1
Christoph Hellwig
maintainerPATCH 3/4 nvme: store the actual queue size in ctrl->sqsize
In reply to Christoph HellwigConvert from the strange on the wire 0s based value to a regular size when reading the filed to avoid adjustments all over.
--- drivers/nvme/host/apple.c | 3 +-- drivers/nvme/host/core.c | 19 +++++++++++++++++-- drivers/nvme/host/fabrics.c | 2 +- drivers/nvme/host/fc.c | 12 ++++++------ drivers/nvme/host/pci.c | 4 ++-- drivers/nvme/host/rdma.c | 23 +++++++++++------------ drivers/nvme/host/tcp.c | 12 ++++++------ 7 files changed, 44 insertions(+), 31 deletions(-)
drivers/nvme/host/apple.c +1 -2
@@ -1080,8 +1080,7 @@ static void apple_nvme_reset_work(struct work_struct *work)writeq (anv->ioq.tcb_dma_addr, anv->mmio_nvme +APPLE_NVMMU_IOSQ_TCB_BASE ); - anv->ctrl.sqsize = -APPLE_ANS_MAX_QUEUE_DEPTH -1 ;/* 0's based queue depth */ + anv->ctrl.sqsize =APPLE_ANS_MAX_QUEUE_DEPTH ; anv->ctrl.cap =readq (anv->mmio_nvme +NVME_REG_CAP );dev_dbg (anv->dev,"Enabling controller now" );
drivers/nvme/host/core.c +17 -2
@@ -3242,7 +3242,8 @@ int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl, bool was_suspended)return ret; } - ctrl->sqsize =min_t (u16,NVME_CAP_MQES (ctrl->cap), ctrl->sqsize); +/* CAP.MQES is 0s based */ + ctrl->sqsize =min_t (u16,NVME_CAP_MQES (ctrl->cap) -1 , ctrl->sqsize);if (ctrl->vs >=NVME_VS (1 ,1 ,0 )) ctrl->subsystem =NVME_CAP_NSSRC (ctrl->cap);
@@ -3505,9 +3506,18 @@ static DEVICE_ATTR(field, S_IRUGO, field##_show, NULL);nvme_show_int_function (cntlid);nvme_show_int_function (numa_node);nvme_show_int_function (queue_count); -nvme_show_int_function (sqsize);nvme_show_int_function (kato); +static ssize_t sqsize_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct nvme_ctrl *ctrl =dev_get_drvdata (dev); + +/* Report the 0s based register value for historic reasons: */ +return sysfs_emit (buf,"%d\n" , ctrl->sqsize -1 ); +} +static DEVICE_ATTR(sqsize, S_IRUGO, sqsize_show, NULL); +static ssize_t nvme_sysfs_delete (struct device *dev,struct device_attribute *attr,const char *buf,size_t count)
@@ -4897,7 +4907,12 @@ int nvme_alloc_io_tag_set(struct nvme_ctrl *ctrl, struct blk_mq_tag_set *set,memset (set,0 ,sizeof (*set)); set->ops = ops; + +/* +* Leave an empty slot in the SQ to deal with queue wrap conditions. +*/ set->queue_depth =min_t (unsigned, ctrl->sqsize,BLK_MQ_MAX_DEPTH -1 ); +/* * Some Apple controllers requires tags to be unique across admin and * the (only) I/O queue, so reserve the first 32 tags of the I/O queue.
drivers/nvme/host/fabrics.c +1 -1
@@ -464,7 +464,7 @@ int nvmf_connect_io_queue(struct nvme_ctrl *ctrl, u16 qid) cmd.connect.opcode = nvme_fabrics_command; cmd.connect.fctype = nvme_fabrics_type_connect; cmd.connect.qid =cpu_to_le16 (qid); - cmd.connect.sqsize =cpu_to_le16 (ctrl->sqsize); + cmd.connect.sqsize =cpu_to_le16 (ctrl->sqsize -1 );if (ctrl->opts->disable_sqflow) cmd.connect.cattr |=NVME_CONNECT_DISABLE_SQFLOW ;
drivers/nvme/host/fc.c +6 -6
@@ -2922,11 +2922,11 @@ nvme_fc_create_io_queues(struct nvme_fc_ctrl *ctrl)if (ret)return ret; - ret =nvme_fc_create_hw_io_queues (ctrl, ctrl->ctrl.sqsize +1 ); + ret =nvme_fc_create_hw_io_queues (ctrl, ctrl->ctrl.sqsize);if (ret) goto out_cleanup_tagset; - ret =nvme_fc_connect_io_queues (ctrl, ctrl->ctrl.sqsize +1 ); + ret =nvme_fc_connect_io_queues (ctrl, ctrl->ctrl.sqsize);if (ret) goto out_delete_hw_queues;
@@ -2982,11 +2982,11 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)blk_mq_update_nr_hw_queues (&ctrl->tag_set, nr_io_queues); } - ret =nvme_fc_create_hw_io_queues (ctrl, ctrl->ctrl.sqsize +1 ); + ret =nvme_fc_create_hw_io_queues (ctrl, ctrl->ctrl.sqsize);if (ret) goto out_free_io_queues; - ret =nvme_fc_connect_io_queues (ctrl, ctrl->ctrl.sqsize +1 ); + ret =nvme_fc_connect_io_queues (ctrl, ctrl->ctrl.sqsize);if (ret) goto out_delete_hw_queues;
@@ -3148,7 +3148,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)"to maxcmd\n" , opts->queue_size, ctrl->ctrl.maxcmd); opts->queue_size = ctrl->ctrl.maxcmd; - ctrl->ctrl.sqsize = opts->queue_size -1 ; + ctrl->ctrl.sqsize = opts->queue_size; } ret =nvme_fc_init_aen_ops (ctrl);
@@ -3509,7 +3509,7 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts, lport->ops->max_hw_queues); ctrl->ctrl.queue_count++;/* +1 for admin queue */ - ctrl->ctrl.sqsize = opts->queue_size -1 ; + ctrl->ctrl.sqsize = opts->queue_size; ctrl->ctrl.kato = opts->kato; ctrl->ctrl.cntlid =0xffff ;
drivers/nvme/host/pci.c +2 -2
@@ -2335,7 +2335,7 @@ static int nvme_setup_io_queues(struct nvme_dev *dev) sizeof(struct nvme_command));if (result >0 ) { dev->q_depth = result; - dev->ctrl.sqsize = result -1 ; + dev->ctrl.sqsize = result; }else { dev->cmb_use_sqes = false; }
@@ -2579,7 +2579,7 @@ static int nvme_pci_enable(struct nvme_dev *dev)dev_warn (dev->ctrl.device,"IO queue depth clamped to %d\n" , dev->q_depth); } - dev->ctrl.sqsize = dev->q_depth -1 ;/* 0's based queue depth */ + dev->ctrl.sqsize = dev->q_depth;nvme_map_cmb (dev);
drivers/nvme/host/rdma.c +11 -12
@@ -773,8 +773,7 @@ static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl) }for (i =1 ; i < ctrl->ctrl.queue_count; i++) { - ret =nvme_rdma_alloc_queue (ctrl, i, - ctrl->ctrl.sqsize +1 ); + ret =nvme_rdma_alloc_queue (ctrl, i, ctrl->ctrl.sqsize);if (ret) goto out_free_queues; }
@@ -1059,24 +1058,24 @@ static int nvme_rdma_setup_ctrl(struct nvme_rdma_ctrl *ctrl, bool new) goto destroy_admin; } -if (ctrl->ctrl.opts->queue_size > ctrl->ctrl.sqsize +1 ) { +if (ctrl->ctrl.opts->queue_size > ctrl->ctrl.sqsize) {dev_warn (ctrl->ctrl.device,"queue_size %zu > ctrl sqsize %u, clamping down\n" , - ctrl->ctrl.opts->queue_size, ctrl->ctrl.sqsize +1 ); + ctrl->ctrl.opts->queue_size, ctrl->ctrl.sqsize); } -if (ctrl->ctrl.sqsize +1 >NVME_RDMA_MAX_QUEUE_SIZE ) { +if (ctrl->ctrl.sqsize >NVME_RDMA_MAX_QUEUE_SIZE ) {dev_warn (ctrl->ctrl.device,"ctrl sqsize %u > max queue size %u, clamping down\n" , - ctrl->ctrl.sqsize +1 ,NVME_RDMA_MAX_QUEUE_SIZE ); - ctrl->ctrl.sqsize =NVME_RDMA_MAX_QUEUE_SIZE -1 ; + ctrl->ctrl.sqsize,NVME_RDMA_MAX_QUEUE_SIZE ); + ctrl->ctrl.sqsize =NVME_RDMA_MAX_QUEUE_SIZE ; } -if (ctrl->ctrl.sqsize +1 > ctrl->ctrl.maxcmd) { +if (ctrl->ctrl.sqsize > ctrl->ctrl.maxcmd) {dev_warn (ctrl->ctrl.device,"sqsize %u > ctrl maxcmd %u, clamping down\n" , - ctrl->ctrl.sqsize +1 , ctrl->ctrl.maxcmd); - ctrl->ctrl.sqsize = ctrl->ctrl.maxcmd -1 ; + ctrl->ctrl.sqsize, ctrl->ctrl.maxcmd); + ctrl->ctrl.sqsize = ctrl->ctrl.maxcmd; }if (ctrl->ctrl.sgls & (1 <<20 ))
@@ -1891,7 +1890,7 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue)* 1's based representation of sqsize. */ priv.hrqsize =cpu_to_le16 (queue->queue_size); - priv.hsqsize =cpu_to_le16 (queue->ctrl->ctrl.sqsize); + priv.hsqsize =cpu_to_le16 (queue->ctrl->ctrl.sqsize -1 ); } ret =rdma_connect_locked (queue->cm_id, ¶m);
@@ -2338,7 +2337,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev, ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues + opts->nr_poll_queues +1 ; - ctrl->ctrl.sqsize = opts->queue_size -1 ; + ctrl->ctrl.sqsize = opts->queue_size; ctrl->ctrl.kato = opts->kato; ret = -ENOMEM ;
drivers/nvme/host/tcp.c +6 -6
@@ -2047,16 +2047,16 @@ static int nvme_tcp_setup_ctrl(struct nvme_ctrl *ctrl, bool new) goto destroy_admin; } -if (opts->queue_size > ctrl->sqsize +1 ) +if (opts->queue_size > ctrl->sqsize)dev_warn (ctrl->device,"queue_size %zu > ctrl sqsize %u, clamping down\n" , - opts->queue_size, ctrl->sqsize +1 ); + opts->queue_size, ctrl->sqsize); -if (ctrl->sqsize +1 > ctrl->maxcmd) { +if (ctrl->sqsize > ctrl->maxcmd) {dev_warn (ctrl->device,"sqsize %u > ctrl maxcmd %u, clamping down\n" , - ctrl->sqsize +1 , ctrl->maxcmd); - ctrl->sqsize = ctrl->maxcmd -1 ; + ctrl->sqsize, ctrl->maxcmd); + ctrl->sqsize = ctrl->maxcmd; }if (ctrl->queue_count >1 ) {
@@ -2562,7 +2562,7 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev, ctrl->ctrl.opts = opts; ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues + opts->nr_poll_queues +1 ; - ctrl->ctrl.sqsize = opts->queue_size -1 ; + ctrl->ctrl.sqsize = opts->queue_size; ctrl->ctrl.kato = opts->kato; INIT_DELAYED_WORK(&ctrl->connect_work,
Signature · 1 line
2.35.1
Sagi Grimberg
maintainerRe: [PATCH 3/4] nvme: store the actual queue size in ctrl->sqsize
Quoted 33 lines · Christoph Hellwig
Convert from the strange on the wire 0s based value to a regular size when reading the filed to avoid adjustments all over.
Signed-off-by Christoph Hellwig <[email protected]>--- drivers/nvme/host/apple.c | 3 +-- drivers/nvme/host/core.c | 19 +++++++++++++++++-- drivers/nvme/host/fabrics.c | 2 +- drivers/nvme/host/fc.c | 12 ++++++------ drivers/nvme/host/pci.c | 4 ++-- drivers/nvme/host/rdma.c | 23 +++++++++++------------ drivers/nvme/host/tcp.c | 12 ++++++------ 7 files changed, 44 insertions(+), 31 deletions(-)drivers/nvme/host/apple.c +1
-2@@ -1080,8 +1080,7 @@ static void apple_nvme_reset_work(struct work_struct *work)writeq (anv->ioq.tcb_dma_addr, anv->mmio_nvme +APPLE_NVMMU_IOSQ_TCB_BASE ); - anv->ctrl.sqsize = -APPLE_ANS_MAX_QUEUE_DEPTH -1 ;/* 0's based queue depth */ + anv->ctrl.sqsize =APPLE_ANS_MAX_QUEUE_DEPTH ; anv->ctrl.cap =readq (anv->mmio_nvme +NVME_REG_CAP );dev_dbg (anv->dev,"Enabling controller now" );drivers/nvme/host/core.c +0
-0
drivers/nvme/host/core.c int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl, bool was_suspended) · line 3246
Quoted 6 lines
drivers/nvme/host/core.c +2
-1@@ -3242,7 +3242,8 @@ int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl, bool was_suspended)return ret; } - ctrl->sqsize =min_t (u16,NVME_CAP_MQES (ctrl->cap), ctrl->sqsize); +/* CAP.MQES is 0s based */ + ctrl->sqsize =min_t (u16,NVME_CAP_MQES (ctrl->cap) -1 , ctrl->sqsize);
Shoudn't this be NVME_CAP_MQES(ctrl->cap) + 1 ?
Keith Busch
maintainerRe: [PATCH 3/4] nvme: store the actual queue size in ctrl->sqsize
In reply to Christoph HellwigQuoted 2 lines · Christoph Hellwig
Convert from the strange on the wire 0s based value to a regular size when reading the filed to avoid adjustments all over.
s/filed/field
drivers/nvme/host/core.c int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl, bool was_suspended) · line 3246
Quoted 7 lines
drivers/nvme/host/core.c +2
-1@@ -3242,7 +3242,8 @@ int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl, bool was_suspended)return ret; } - ctrl->sqsize =min_t (u16,NVME_CAP_MQES (ctrl->cap), ctrl->sqsize); +/* CAP.MQES is 0s based */ + ctrl->sqsize =min_t (u16,NVME_CAP_MQES (ctrl->cap) -1 , ctrl->sqsize);
Did you mean '+ 1' instead of '- 1'? The minimum MQES value per spec is 1, giving 2 slots. Subtracting 1 from that possible value would give you sqsize 0.
But I'm confused why this is being changed. The ctrl->sqsize isn't a 0's based value. It's normal 1-based value that tells blk-mq how many tags to allocate.
drivers/nvme/host/core.c static DEVICE_ATTR(field, S_IRUGO, field##_show, NULL); · line 3518
Quoted 8 lines
drivers/nvme/host/core.c +8
-0+static ssize_t sqsize_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct nvme_ctrl *ctrl =dev_get_drvdata (dev); + +/* Report the 0s based register value for historic reasons: */ +return sysfs_emit (buf,"%d\n" , ctrl->sqsize -1 ); +}
Weird indentation here.
Christoph Hellwig
maintainerPATCH 4/4 nvme-pci: remove the dev->q_depth field
In reply to Christoph HellwigThis field duplicates the sqsize field in the common nvme_ctrl structure, so remove it and use the common field instead.
--- drivers/nvme/host/pci.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-)
drivers/nvme/host/pci.c +13 -17
@@ -126,7 +126,6 @@ struct nvme_dev {unsigned max_qid;unsigned io_queues[HCTX_MAX_TYPES ];unsigned int num_vecs; -u32 q_depth;int io_sqes;u32 db_stride;void __iomem *bar;
@@ -1515,7 +1514,7 @@ static void nvme_reap_pending_cqes(struct nvme_dev *dev)static int nvme_cmb_qdepth (struct nvme_dev *dev,int nr_io_queues,int entry_size) { -int q_depth = dev->q_depth; +u16 q_depth = dev->ctrl.sqsize;unsigned q_size_aligned =roundup (q_depth * entry_size,NVME_CTRL_PAGE_SIZE );
@@ -1824,7 +1823,7 @@ static int nvme_create_io_queues(struct nvme_dev *dev)int ret =0 ;for (i = dev->ctrl.queue_count; i <= dev->max_qid; i++) { -if (nvme_alloc_queue (dev, i, dev->q_depth)) { +if (nvme_alloc_queue (dev, i, dev->ctrl.sqsize)) { ret = -ENOMEM ;break ; }
@@ -2333,12 +2332,10 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)if (dev->cmb_use_sqes) { result =nvme_cmb_qdepth (dev, nr_io_queues,sizeof (struct nvme_command )); -if (result >0 ) { - dev->q_depth = result; +if (result >0 ) dev->ctrl.sqsize = result; - }else { +else dev->cmb_use_sqes = false; - } } do {
@@ -2537,8 +2534,8 @@ static int nvme_pci_enable(struct nvme_dev *dev) dev->ctrl.cap =lo_hi_readq (dev->bar +NVME_REG_CAP ); - dev->q_depth =min_t (u32,NVME_CAP_MQES (dev->ctrl.cap) +1 , - io_queue_depth); + dev->ctrl.sqsize =min_t (u32,NVME_CAP_MQES (dev->ctrl.cap) +1 , + io_queue_depth); dev->db_stride =1 <<NVME_CAP_STRIDE (dev->ctrl.cap); dev->dbs = dev->bar +4096 ;
@@ -2557,16 +2554,16 @@ static int nvme_pci_enable(struct nvme_dev *dev)* some MacBook7,1 to avoid controller resets and data loss. */ if (pdev->vendor ==PCI_VENDOR_ID_APPLE && pdev->device ==0x2001 ) { - dev->q_depth =2 ; + dev->ctrl.sqsize =2 ;dev_warn (dev->ctrl.device,"detected Apple NVMe controller, " "set queue depth=%u to work around controller resets\n" , - dev->q_depth); + dev->ctrl.sqsize); }else if (pdev->vendor ==PCI_VENDOR_ID_SAMSUNG && (pdev->device ==0xa821 || pdev->device ==0xa822 ) &&NVME_CAP_MQES (dev->ctrl.cap) ==0 ) { - dev->q_depth =64 ; + dev->ctrl.sqsize =64 ;dev_err (dev->ctrl.device,"detected PM1725 NVMe controller, " -"set queue depth=%u\n" , dev->q_depth); +"set queue depth=%u\n" , dev->ctrl.sqsize); }/*
@@ -2574,12 +2571,11 @@ static int nvme_pci_enable(struct nvme_dev *dev)* big enough so that we get 32 tags for the admin queue */ if ((dev->ctrl.quirks &NVME_QUIRK_SHARED_TAGS ) && - (dev->q_depth < (NVME_AQ_DEPTH +2 ))) { - dev->q_depth =NVME_AQ_DEPTH +2 ; + (dev->ctrl.sqsize <NVME_AQ_DEPTH +2 )) { + dev->ctrl.sqsize =NVME_AQ_DEPTH +2 ;dev_warn (dev->ctrl.device,"IO queue depth clamped to %d\n" , - dev->q_depth); + dev->ctrl.sqsize); } - dev->ctrl.sqsize = dev->q_depth;nvme_map_cmb (dev);
Signature · 1 line
2.35.1
Re: [PATCH 4/4] nvme-pci: remove the dev->q_depth field
Re: (subset) fix nvme sqsize on off regression
In reply to Christoph HellwigQuoted 15 lines · Christoph Hellwig
this series fixes up the sqsize regression that Hugh reported. The first two patches are the real fixes, and it might make sense if Linus just picks them directly before -rc1. The other two just clean things up a bit to avoid pitfalls like this in the future.
Diffstat:
apple.c | 3 +-- core.c | 21 ++++++++++++++++++--- fabrics.c | 2 +- fc.c | 12 ++++++------ pci.c | 27 ++++++++++++--------------- rdma.c | 23 +++++++++++------------ tcp.c | 12 ++++++------ 7 files changed, 55 insertions(+), 45 deletions(-)
Quoted 1 line
[...]
Applied, thanks!
[1/4] nvme: fix setting the queue depth in nvme_alloc_io_tag_set
commit: 33b93727ce90c8db916fb071ed13e90106339754
[2/4] nvme-pci: update sqsize when adjusting the queue depth
commit: 88d356ca41ba1c3effc2d4208dfbd4392f58cd6dBest regards,
Signature · 1 line
Jens Axboe