nvmet pci-epf fixes
Subsystems (2)
cover nvmet pci-epf fixes
Four patches to fix and improve IRQ handling in the PCI endpoint target driver. The issues fixed by the first 2 patches where discovered with testing polling queues (set on the host side). The following 2 patches are small improvements/cleanup in the same IRQ handling area.
Damien Le Moal (4):
nvmet: pci-epf: Clear completion queue IRQ flag on delete
nvmet: pci-epf: Do not fall back to using INTX if not supported
nvmet: pci-epf: Cleanup nvmet_pci_epf_raise_irq()
nvmet: pci-epf: Improve debug message
drivers/nvme/target/pci-epf.c | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)
Signature · 1 line
2.49.0
PATCH 1/4 nvmet: pci-epf: Clear completion queue IRQ flag on delete
The function nvmet_pci_epf_delete_cq() unconditionnally calls nvmet_pci_epf_remove_irq_vector() even for completion queues that do not have interrupts enabled. Furthermore, for completion queues that do have IRQ enabled, deleting and re-creating the completion queue leaves the flag NVMET_PCI_EPF_Q_IRQ_ENABLED set, even if the completion queue is being re-created with IRQ disabled.
Fix these issues by calling nvmet_pci_epf_remove_irq_vector() only if NVMET_PCI_EPF_Q_IRQ_ENABLED is set and make sure to always clear that flag.
--- drivers/nvme/target/pci-epf.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvme/target/pci-epf.c +2 -1
@@ -1344,7 +1344,8 @@ static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid)cancel_delayed_work_sync (&cq->work);nvmet_pci_epf_drain_queue (cq); -nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector); +if (test_and_clear_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) +nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector);nvmet_pci_epf_mem_unmap (ctrl->nvme_epf, &cq->pci_map);return NVME_SC_SUCCESS ;
Signature · 1 line
2.49.0
Re: [PATCH 1/4] nvmet: pci-epf: Clear completion queue IRQ flag on delete
Quoted 1 line · Damien Le Moal
The function
nvmet_pci_epf_delete_cq()unconditionnally calls
s/unconditionnally/unconditionally/
Quoted 22 lines
nvmet_pci_epf_remove_irq_vector()even for completion queues that do not have interrupts enabled. Furthermore, for completion queues that do have IRQ enabled, deleting and re-creating the completion queue leaves the flagNVMET_PCI_EPF_Q_IRQ_ENABLEDset, even if the completion queue is being re-created with IRQ disabled.Fix these issues by calling
nvmet_pci_epf_remove_irq_vector()only ifNVMET_PCI_EPF_Q_IRQ_ENABLEDis set and make sure to always clear that flag.Fixes 0faa0fe6f90e ("nvmet: New NVMe PCI endpoint function target driver")Signed-off-by Damien Le Moal <[email protected]>--- drivers/nvme/target/pci-epf.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/nvme/target/pci-epf.c +0
-0
drivers/nvme/target/pci-epf.c static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid) · line 1348
Quoted 6 lines
drivers/nvme/target/pci-epf.c +2
-1@@ -1344,7 +1344,8 @@ static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid)cancel_delayed_work_sync (&cq->work);nvmet_pci_epf_drain_queue (cq); -nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector); +if (test_and_clear_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) +nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector);
I agree that we should only call nvmet_pci_epf_remove_irq_vector() if NVMET_PCI_EPF_Q_IRQ_ENABLED is set.
However, it looks a bit weird to explicitly only clear only this flag bit. What about the other flag bits?
I would have expected a memset() of flags, or flags = 0; in either .create_cq/.create_sq, or in .delete_cq/.delete_sq.
And it also seems that flag NVMET_PCI_EPF_Q_IS_SQ is unused, so that flag can probably be dropped.
Kind regards,
Niklas
Re: [PATCH 1/4] nvmet: pci-epf: Clear completion queue IRQ flag on delete
Quoted 38 lines · Niklas Cassel
Quoted 1 line · Damien Le Moal
The function
nvmet_pci_epf_delete_cq()unconditionnally callss/unconditionnally/unconditionally/
Quoted 22 lines
nvmet_pci_epf_remove_irq_vector()even for completion queues that do not have interrupts enabled. Furthermore, for completion queues that do have IRQ enabled, deleting and re-creating the completion queue leaves the flagNVMET_PCI_EPF_Q_IRQ_ENABLEDset, even if the completion queue is being re-created with IRQ disabled.Fix these issues by calling
nvmet_pci_epf_remove_irq_vector()only ifNVMET_PCI_EPF_Q_IRQ_ENABLEDis set and make sure to always clear that flag.Fixes 0faa0fe6f90e ("nvmet: New NVMe PCI endpoint function target driver")Signed-off-by Damien Le Moal <[email protected]>--- drivers/nvme/target/pci-epf.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/nvme/target/pci-epf.c +0
-0drivers/nvme/target/pci-epf.c static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid) · line 1348
Quoted 6 lines
drivers/nvme/target/pci-epf.c +2
-1@@ -1344,7 +1344,8 @@ static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid)cancel_delayed_work_sync (&cq->work);nvmet_pci_epf_drain_queue (cq); -nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector); +if (test_and_clear_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) +nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector);I agree that we should only call
nvmet_pci_epf_remove_irq_vector()ifNVMET_PCI_EPF_Q_IRQ_ENABLEDis set.
Quoted 2 lines
However, it looks a bit weird to explicitly only clear only this flag bit. What about the other flag bits?
NVMET_PCI_EPF_Q_LIVE is cleared in the same function in the same manner. The only other flag is NVMET_PCI_EPF_Q_IRQ_ENABLED, which this patch clears.
Quoted 5 lines
I would have expected a
memset()of flags, or flags = 0; in either .create_cq/.create_sq, or in .delete_cq/.delete_sq.And it also seems that flag
NVMET_PCI_EPF_Q_IS_SQis unused, so that flag can probably be dropped.
Good catch. Will add a patch to drop it.
Signature · 2 lines
Damien Le Moal Western Digital Research
Re: [PATCH 1/4] nvmet: pci-epf: Clear completion queue IRQ flag on delete
Quoted 39 lines · Damien Le Moal
Quoted 38 lines · Niklas Cassel
Quoted 1 line · Damien Le Moal
The function
nvmet_pci_epf_delete_cq()unconditionnally callss/unconditionnally/unconditionally/
Quoted 22 lines
nvmet_pci_epf_remove_irq_vector()even for completion queues that do not have interrupts enabled. Furthermore, for completion queues that do have IRQ enabled, deleting and re-creating the completion queue leaves the flagNVMET_PCI_EPF_Q_IRQ_ENABLEDset, even if the completion queue is being re-created with IRQ disabled.Fix these issues by calling
nvmet_pci_epf_remove_irq_vector()only ifNVMET_PCI_EPF_Q_IRQ_ENABLEDis set and make sure to always clear that flag.Fixes 0faa0fe6f90e ("nvmet: New NVMe PCI endpoint function target driver")Signed-off-by Damien Le Moal <[email protected]>--- drivers/nvme/target/pci-epf.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/nvme/target/pci-epf.c +0
-0drivers/nvme/target/pci-epf.c static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid) · line 1348
Quoted 6 lines
drivers/nvme/target/pci-epf.c +2
-1@@ -1344,7 +1344,8 @@ static u16 nvmet_pci_epf_delete_cq(struct nvmet_ctrl *tctrl, u16 cqid)cancel_delayed_work_sync (&cq->work);nvmet_pci_epf_drain_queue (cq); -nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector); +if (test_and_clear_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) +nvmet_pci_epf_remove_irq_vector (ctrl, cq->vector);I agree that we should only call
nvmet_pci_epf_remove_irq_vector()ifNVMET_PCI_EPF_Q_IRQ_ENABLEDis set.
Quoted 8 lines
Quoted 2 lines
However, it looks a bit weird to explicitly only clear only this flag bit. What about the other flag bits?
NVMET_PCI_EPF_Q_LIVEis cleared in the same function in the same manner. The only other flag isNVMET_PCI_EPF_Q_IRQ_ENABLED, which this patch clears.Quoted 2 lines
I would have expected a
memset()of flags, or flags = 0; in either .create_cq/.create_sq, or in .delete_cq/.delete_sq.
You patch looks fine, I was just thinking that it would be slightly more robust to do flags = 0; in e.g. .create_cq/.create_sq, or .delete_cq/.delete_sq.
That way, we remove the possibility of ever introducing a similar bug to what this patch fixes. (I.e if we ever add a new flag, we will not need to remember to also add code that explicitly clears the bit for that new flag, since flags = 0; clears all flags in one go.)
Kind regards,
Niklas
PATCH 2/4 nvmet: pci-epf: Do not fall back to using INTX if not supported
In reply to Damien Le MoalSome endpoint PCIe controllers do not support raising legacy INTX interrupts. This support is indicated by the intx_capable field of struct pci_epc_features. Modify nvmet_pci_epf_raise_irq() to not automatically fallback to trying raising an INTX interrupt after an MSI or MSI-X error if the controller does not support INTX.
--- drivers/nvme/target/pci-epf.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/nvme/target/pci-epf.c +7 -5
@@ -636,14 +636,16 @@ static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl,switch (nvme_epf->irq_type) {case PCI_IRQ_MSIX :case PCI_IRQ_MSI : +/* +* If we fail to raise an MSI or MSI-X interrupr, it is likely +* because the host is using legacy INTX IRQs (e.g. BIOS, +* grub). But we can fallback to the INTX type only if the +* endpoint controller supports this type. +*/ ret =pci_epc_raise_irq (epf->epc, epf->func_no, epf->vfunc_no, nvme_epf->irq_type, cq->vector +1 ); -if (!ret) +if (!ret || !nvme_epf->epc_features->intx_capable)break ; -/* -* If we got an error, it is likely because the host is using -* legacy IRQs (e.g. BIOS, grub). -*/ fallthrough;case PCI_IRQ_INTX : ret = pci_epc_raise_irq(epf->epc, epf->func_no, epf->vfunc_no,
Signature · 1 line
2.49.0
Re: [PATCH 2/4] nvmet: pci-epf: Do not fall back to using INTX if not supported
Quoted 9 lines · Damien Le Moal
Some endpoint PCIe controllers do not support raising legacy INTX interrupts. This support is indicated by the
intx_capablefield of structpci_epc_features. Modifynvmet_pci_epf_raise_irq()to not automatically fallback to trying raising an INTX interrupt after an MSI or MSI-X error if the controller does not support INTX.Fixes 0faa0fe6f90e ("nvmet: New NVMe PCI endpoint function target driver")Signed-off-by Damien Le Moal <[email protected]>
Hm..
0faa0fe6f90e ("nvmet: New NVMe PCI endpoint function target driver") was first included in v6.14-rc1.
4b313c69a38e ("PCI: endpoint: Add intx_capable to epc_features struct") was first included in v6.15-rc1.
Perhaps add a:
Cc: [email protected] # depends on patch introducing intx_capable flag
So that it is not automatically backported.
Quoted 9 lines
--- drivers/nvme/target/pci-epf.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)drivers/nvme/target/pci-epf.c +0
-0
drivers/nvme/target/pci-epf.c static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl, · line 640
Quoted 5 lines
drivers/nvme/target/pci-epf.c +2
-0@@ -636,14 +636,16 @@ static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl,switch (nvme_epf->irq_type) {case PCI_IRQ_MSIX :case PCI_IRQ_MSI : +/* +* If we fail to raise an MSI or MSI-X interrupr, it is likely
s/interrupr/interrupt/
drivers/nvme/target/pci-epf.c static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl, · line 642
Quoted 2 lines
drivers/nvme/target/pci-epf.c +2
-0+* because the host is using legacy INTX IRQs (e.g. BIOS, +* grub). But we can fallback to the INTX type only if the
s/. But/, but/
Quoted 18 lines
drivers/nvme/target/pci-epf.c +3
-5+* endpoint controller supports this type. +*/ ret =pci_epc_raise_irq (epf->epc, epf->func_no, epf->vfunc_no, nvme_epf->irq_type, cq->vector +1 ); -if (!ret) +if (!ret || !nvme_epf->epc_features->intx_capable)break ; -/* -* If we got an error, it is likely because the host is using -* legacy IRQs (e.g. BIOS, grub). -*/ fallthrough;case PCI_IRQ_INTX : ret = pci_epc_raise_irq(epf->epc, epf->func_no, epf->vfunc_no,Signature · 1 line
2.49.0
Re: [PATCH 2/4] nvmet: pci-epf: Do not fall back to using INTX if not supported
Quoted 16 lines · Niklas Cassel
Quoted 9 lines · Damien Le Moal
Some endpoint PCIe controllers do not support raising legacy INTX interrupts. This support is indicated by the
intx_capablefield of structpci_epc_features. Modifynvmet_pci_epf_raise_irq()to not automatically fallback to trying raising an INTX interrupt after an MSI or MSI-X error if the controller does not support INTX.Fixes 0faa0fe6f90e ("nvmet: New NVMe PCI endpoint function target driver")Signed-off-by Damien Le Moal <[email protected]>Hm..
0faa0fe6f90e("nvmet: New NVMe PCI endpoint function target driver") was first included in v6.14-rc1.
Quoted 7 lines
4b313c69a38e("PCI: endpoint: Addintx_capabletoepc_featuresstruct") was first included in v6.15-rc1.Perhaps add a:
Cc: [email protected] # depends on patch introducingintx_capableflagSo that it is not automatically backported.
Good catch. Will drop the Fixes tag. No point in it.
Signature · 2 lines
Damien Le Moal Western Digital Research
PATCH 3/4 nvmet: pci-epf: Cleanup nvmet_pci_epf_raise_irq()
In reply to Damien Le MoalThere is no point in taking the controller irq_lock and calling nvmet_pci_epf_should_raise_irq() for a completion queue which does not have IRQ enabled (NVMET_PCI_EPF_Q_IRQ_ENABLED flag is not set). Move the test for the NVMET_PCI_EPF_Q_IRQ_ENABLED flag out of nvmet_pci_epf_should_raise_irq() to the top of nvmet_pci_epf_raise_irq() to return early when no IRQ should be raised.
Also, use dev_err_ratelimited() to avoid a message storm under load when raising IRQs is failing.
--- drivers/nvme/target/pci-epf.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/nvme/target/pci-epf.c +5 -5
@@ -596,9 +596,6 @@ static bool nvmet_pci_epf_should_raise_irq(struct nvmet_pci_epf_ctrl *ctrl,struct nvmet_pci_epf_irq_vector *iv = cq->iv;bool ret; -if (!test_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) -return false; -/* IRQ coalescing for the admin queue is not allowed. */ if (!cq->qid)return true;
@@ -625,7 +622,8 @@ static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl,struct pci_epf *epf = nvme_epf->epf;int ret =0 ; -if (!test_bit (NVMET_PCI_EPF_Q_LIVE , &cq->flags)) +if (!test_bit (NVMET_PCI_EPF_Q_LIVE , &cq->flags) || + !test_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags))return ;mutex_lock (&ctrl->irq_lock);
@@ -658,7 +656,9 @@ static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl, }if (ret) -dev_err (ctrl->dev,"Failed to raise IRQ (err=%d)\n" , ret); +dev_err_ratelimited (ctrl->dev, +"CQ[%u]: Failed to raise IRQ (err=%d)\n" , + cq->qid, ret); unlock:mutex_unlock (&ctrl->irq_lock);
Signature · 1 line
2.49.0
Re: [PATCH 3/4] nvmet: pci-epf: Cleanup nvmet_pci_epf_raise_irq()
Quoted 54 lines · Damien Le Moal
There is no point in taking the controller
irq_lockand callingnvmet_pci_epf_should_raise_irq()for a completion queue which does not have IRQ enabled (NVMET_PCI_EPF_Q_IRQ_ENABLEDflag is not set). Move the test for theNVMET_PCI_EPF_Q_IRQ_ENABLEDflag out ofnvmet_pci_epf_should_raise_irq()to the top ofnvmet_pci_epf_raise_irq()to return early when no IRQ should be raised.Also, use
dev_err_ratelimited()to avoid a message storm under load when raising IRQs is failing.Signed-off-by Damien Le Moal <[email protected]>--- drivers/nvme/target/pci-epf.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)drivers/nvme/target/pci-epf.c +5
-5@@ -596,9 +596,6 @@ static bool nvmet_pci_epf_should_raise_irq(struct nvmet_pci_epf_ctrl *ctrl,struct nvmet_pci_epf_irq_vector *iv = cq->iv;bool ret; -if (!test_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) -return false; -/* IRQ coalescing for the admin queue is not allowed. */ if (!cq->qid)return true;@@ -625,7 +622,8 @@ static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl,struct pci_epf *epf = nvme_epf->epf;int ret =0 ; -if (!test_bit (NVMET_PCI_EPF_Q_LIVE , &cq->flags)) +if (!test_bit (NVMET_PCI_EPF_Q_LIVE , &cq->flags) || + !test_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags))return ;mutex_lock (&ctrl->irq_lock);@@ -658,7 +656,9 @@ static void nvmet_pci_epf_raise_irq(struct nvmet_pci_epf_ctrl *ctrl, }if (ret) -dev_err (ctrl->dev,"Failed to raise IRQ (err=%d)\n" , ret); +dev_err_ratelimited (ctrl->dev, +"CQ[%u]: Failed to raise IRQ (err=%d)\n" , + cq->qid, ret); unlock:mutex_unlock (&ctrl->irq_lock);Signature · 1 line
2.49.0
Nit: since we no longer perform the NVMET_PCI_EPF_Q_IRQ_ENABLED check in nvmet_pci_epf_should_raise_irq(), should we rename the function to something else? nvmet_pci_epf_irq_allowed() ? Any ideas ?
Re: [PATCH 3/4] nvmet: pci-epf: Cleanup nvmet_pci_epf_raise_irq()
Quoted 3 lines · Niklas Cassel
Nit: since we no longer perform the
NVMET_PCI_EPF_Q_IRQ_ENABLEDcheck innvmet_pci_epf_should_raise_irq(), should we rename the function to something else?nvmet_pci_epf_irq_allowed()? Any ideas ?
nvmet_pci_epf_should_raise_irq() is still in charge of handling IRQ coalescing, returning true if we should raise an IRQ and false if the IRQs are still being coalesced. So the name as is still seems fine to me.
Quoted 1 line
Reviewed-by Niklas Cassel <[email protected]>
Thanks.
Signature · 2 lines
Damien Le Moal Western Digital Research
PATCH 4/4 nvmet: pci-epf: Improve debug message
In reply to Damien Le MoalImprove the debug message of nvmet_pci_epf_create_cq() to indicate if a completion queue IRQ is disabled.
--- drivers/nvme/target/pci-epf.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/nvme/target/pci-epf.c +8 -2
@@ -1321,8 +1321,14 @@ static u16 nvmet_pci_epf_create_cq(struct nvmet_ctrl *tctrl,set_bit (NVMET_PCI_EPF_Q_LIVE , &cq->flags); -dev_dbg (ctrl->dev,"CQ[%u]: %u entries of %zu B, IRQ vector %u\n" , - cqid, qsize, cq->qes, cq->vector); +if (test_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) +dev_dbg (ctrl->dev, +"CQ[%u]: %u entries of %zu B, IRQ vector %u\n" , + cqid, qsize, cq->qes, cq->vector); +else +dev_dbg (ctrl->dev, +"CQ[%u]: %u entries of %zu B, IRQ disabled\n" , + cqid, qsize, cq->qes);return NVME_SC_SUCCESS ;
Signature · 1 line
2.49.0
Re: [PATCH 4/4] nvmet: pci-epf: Improve debug message
Quoted 33 lines · Damien Le Moal
Improve the debug message of
nvmet_pci_epf_create_cq()to indicate if a completion queue IRQ is disabled.Signed-off-by Damien Le Moal <[email protected]>--- drivers/nvme/target/pci-epf.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/nvme/target/pci-epf.c +8
-2@@ -1321,8 +1321,14 @@ static u16 nvmet_pci_epf_create_cq(struct nvmet_ctrl *tctrl,set_bit (NVMET_PCI_EPF_Q_LIVE , &cq->flags); -dev_dbg (ctrl->dev,"CQ[%u]: %u entries of %zu B, IRQ vector %u\n" , - cqid, qsize, cq->qes, cq->vector); +if (test_bit (NVMET_PCI_EPF_Q_IRQ_ENABLED , &cq->flags)) +dev_dbg (ctrl->dev, +"CQ[%u]: %u entries of %zu B, IRQ vector %u\n" , + cqid, qsize, cq->qes, cq->vector); +else +dev_dbg (ctrl->dev, +"CQ[%u]: %u entries of %zu B, IRQ disabled\n" , + cqid, qsize, cq->qes);return NVME_SC_SUCCESS ;Signature · 1 line
2.49.0