hw/nvme: shadow doorbells broken on riscv64
cover hw/nvme: shadow doorbells broken on riscv64
From: Klaus Jensen <[email protected]>
Guenter reports[1] that hw/nvme is broken on riscv64.
This is a regression since 7.1, so this does not warrent an rc5 for 7.2. I'm sure Guenter can carry this patch in his tree, and maybe we can get this out in a stable release.
I really wonder why this issue only shows up on riscv64. We have not observed this on other platforms (yet).
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
Klaus Jensen (1):
hw/nvme: fix missing cq eventidx update
hw/nvme/ctrl.c | 8 ++++++++ 1 file changed, 8 insertions(+)
Signature · 1 line
2.38.1
PATCH 1/1 hw/nvme: fix missing cq eventidx update
From: Klaus Jensen <[email protected]>
Prior to reading the shadow doorbell cq head, we have to update the eventidx. Otherwise, we risk that the driver will skip an mmio doorbell write. This happens on riscv64, as reported by Guenter.
Adding the missing update to the cq eventidx fixes the issue.
--- hw/nvme/ctrl.c | 8 ++++++++ 1 file changed, 8 insertions(+)
hw/nvme/ctrl.c +8 -0
@@ -1331,6 +1331,13 @@ static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, } } +static void nvme_update_cq_eventidx (const NvmeCQueue *cq) +{ +pci_dma_write (&cq->ctrl->parent_obj, cq->ei_addr, &cq->head, +sizeof (cq->head)); +trace_pci_nvme_eventidx_cq (cq->cqid, cq->head); +} +static void nvme_update_cq_head (NvmeCQueue *cq) { pci_dma_read(&cq->ctrl->parent_obj, cq->db_addr, &cq->head,
@@ -1351,6 +1358,7 @@ static void nvme_post_cqes(void *opaque)hwaddr addr;if (n->dbbuf_enabled) { +nvme_update_cq_eventidx (cq);nvme_update_cq_head (cq); }
Signature · 1 line
2.38.1
Re: [PATCH 1/1] hw/nvme: fix missing cq eventidx update
Quoted 20 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Prior to reading the shadow doorbell cq head, we have to update the eventidx. Otherwise, we risk that the driver will skip an mmio doorbell write. This happens on riscv64, as reported by Guenter.
Adding the missing update to the cq eventidx fixes the issue.
Fixes 3f7fe8de3d49 ("hw/nvme: Implement shadow doorbell buffer support")Reported-by Guenter Roeck <[email protected]>Signed-off-by Klaus Jensen <[email protected]>--- hw/nvme/ctrl.c | 8 ++++++++ 1 file changed, 8 insertions(+)hw/nvme/ctrl.c +0
-0
hw/nvme/ctrl.c static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, · line 1336
Quoted 6 lines
hw/nvme/ctrl.c +3
-0@@ -1331,6 +1331,13 @@ static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, } } +static void nvme_update_cq_eventidx (const NvmeCQueue *cq) +{ + pci_dma_write(&cq->ctrl->parent_obj, cq->ei_addr, &cq->head,
'parent_obj' is a private field. Better to use the QOM accessor:
pci_dma_write(PCI_DEVICE(&cq->ctrl), cq->ei_addr, &cq->head,
hw/nvme/ctrl.c static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, · line 1338
Quoted 2 lines
hw/nvme/ctrl.c +2
-0+ sizeof(cq->head)); +trace_pci_nvme_eventidx_cq (cq->cqid, cq->head);
Surprisingly the trace event format was already present in Jinhao respin...
https://lore.kernel.org/all/YqsIh+OUcWnHU3zp@apples/T/
Could we move the event before the call?
Quoted 13 lines
hw/nvme/ctrl.c +3
-0+} +static void nvme_update_cq_head (NvmeCQueue *cq) { pci_dma_read(&cq->ctrl->parent_obj, cq->db_addr, &cq->head,@@ -1351,6 +1358,7 @@ static void nvme_post_cqes(void *opaque)hwaddr addr;if (n->dbbuf_enabled) { +nvme_update_cq_eventidx (cq);nvme_update_cq_head (cq); }
Re: [PATCH 1/1] hw/nvme: fix missing cq eventidx update
Quoted 29 lines · Philippe Mathieu-Daudé
Quoted 20 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Prior to reading the shadow doorbell cq head, we have to update the eventidx. Otherwise, we risk that the driver will skip an mmio doorbell write. This happens on riscv64, as reported by Guenter.
Adding the missing update to the cq eventidx fixes the issue.
Fixes 3f7fe8de3d49 ("hw/nvme: Implement shadow doorbell buffer support")Reported-by Guenter Roeck <[email protected]>Signed-off-by Klaus Jensen <[email protected]>--- hw/nvme/ctrl.c | 8 ++++++++ 1 file changed, 8 insertions(+)hw/nvme/ctrl.c +0
-0hw/nvme/ctrl.c static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, · line 1335
Quoted 5 lines
hw/nvme/ctrl.c +3
-0@@ -1331,6 +1331,13 @@ static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, } } +static void nvme_update_cq_eventidx (const NvmeCQueue *cq) +{ + pci_dma_write(&cq->ctrl->parent_obj, cq->ei_addr, &cq->head,'
parent_obj' is a private field. Better to use the QOM accessor:
Quoted 1 line
pci_dma_write(PCI_DEVICE(&cq->ctrl), cq->ei_addr, &cq->head,
Ah yes, of course. I think we have a couple of other ->parent_obj accesses that we should fix also.
Quoted 9 lines
hw/nvme/ctrl.c static inline void nvme_blk_write(BlockBackend *blk, int64_t offset, · line 1338
Quoted 3 lines
hw/nvme/ctrl.c +2
-0+ sizeof(cq->head)); +trace_pci_nvme_eventidx_cq (cq->cqid, cq->head);Surprisingly the trace event format was already present in Jinhao respin...
https://lore.kernel.org/all/YqsIh+OUcWnHU3zp@apples/T/Could we move the event before the call?
Makes sense.
Quoted 12 lines
Quoted 11 lines
hw/nvme/ctrl.c +3
-0+} +static void nvme_update_cq_head (NvmeCQueue *cq) { pci_dma_read(&cq->ctrl->parent_obj, cq->db_addr, &cq->head,@@ -1351,6 +1358,7 @@ static void nvme_post_cqes(void *opaque)hwaddr addr;if (n->dbbuf_enabled) { +nvme_update_cq_eventidx (cq);nvme_update_cq_head (cq); }
Quoted 7 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Guenter reports[1] that hw/nvme is broken on riscv64.
This is a regression since 7.1, so this does not warrent an rc5 for 7.2. I'm sure Guenter can carry this patch in his tree, and maybe we can get this out in a stable release.
Delaying to 8.0 and adding a 'Cc: qemu-stable@' tag seems wise.
Cc'ing riscv list in case.
Quoted 11 lines
I really wonder why this issue only shows up on riscv64. We have not observed this on other platforms (yet).
[1]: https://lore.kernel.org/qemu-devel/[email protected]/Klaus Jensen (1):
hw/nvme: fix missing cq eventidx updatehw/nvme/ctrl.c | 8 ++++++++ 1 file changed, 8 insertions(+)