Skip to contentSkip to conversation

nvme-pci: add trouble shooting steps for timeouts

PATCH Applied Keith Busch 3 participants 3 messages Jun 6, 2022 – Jun 7, 2022 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme-pci: add trouble shooting steps for timeouts

From: Keith Busch <[email protected]>

Many users have encountered IO timeouts with a CSTS value of 0xffffffff, which indicates a failure to read the register. While there are various potential causes for this observation, faulty NVMe APST has been the culprit quite frequently. Add the recommended troubleshooting steps in the error output when this condition occurs.

---
 drivers/nvme/host/pci.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/nvme/host/pci.c +8 -0
index 48f4f6eb877b..9d963f6cdbae 100644
@@ -1334,6 +1334,14 @@ static void nvme_warn_reset(struct nvme_dev *dev, u32 csts)
 		dev_warn(dev->ctrl.device,
 			 "controller is down; will reset: CSTS=0x%x, PCI_STATUS read failed (%d)\n",
 			 csts, result);
+
+	if (csts != ~0)
+		return;
+
+	dev_warn(dev->ctrl.device,
+		 "Does your device have a faulty power saving mode enabled?\n");
+	dev_warn(dev->ctrl.device,
+		 "Try \"nvme_core.default_ps_max_latency_us=0 pcie_aspm=off\" and report a bug\n");
 }
 
 static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
Signature · 1 line
2.30.2
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Keith Busch

From: Keith Busch <[email protected]>

Many users have encountered IO timeouts with a CSTS value of 0xffffffff, which indicates a failure to read the register. While there are various potential causes for this observation, faulty NVMe APST has been the culprit quite frequently. Add the recommended troubleshooting steps in the error output when this condition occurs.

---

I've seen these reports with CSTS == 0xffffffff many times, looks good, this should help...

-ck