nvme-pci: enhance timeout kernel log
PATCH 1/1 nvme-pci: enhance timeout kernel log
From: Keith Busch <[email protected]>
Kernel configs don't necessarily have opcode decoding, and some opcodes are not even decodable. It is still interesting for debugging SSD issues to know what opcode is timing out, what request type it came from, and the data size (if applicable).
--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/nvme/host/pci.c +5 -4
@@ -1284,6 +1284,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req)struct request *abort_req;struct nvme_command cmd = { };u32 csts =readl (dev->bar +NVME_REG_CSTS ); +u8 opcode;/* If PCI error recovery process is happening, we cannot reset or * the recovery mechanism will surely fail.
@@ -1361,11 +1362,11 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req) cmd.abort.cid =nvme_cid (req); cmd.abort.sqid =cpu_to_le16 (nvmeq->qid); + opcode =nvme_req (req)->cmd->common.opcode,dev_warn (nvmeq->dev->ctrl.device, -"I/O %d (%s) QID %d timeout, aborting\n" , - req->tag, -nvme_get_opcode_str (nvme_req (req)->cmd->common.opcode), - nvmeq->qid); +"I/O %d (%s %d) QID %d timeout, aborting req_op:%u size:%u\n" , + req->tag,nvme_get_opcode_str (opcode), opcode, nvmeq->qid, +req_op (req),blk_rq_bytes (req)); abort_req =blk_mq_alloc_request (dev->ctrl.admin_q,nvme_req_op (&cmd),BLK_MQ_REQ_NOWAIT );
Signature · 1 line
2.34.1
Jens Axboe
maintainerQuoted 27 lines · Keith Busch
From: Keith Busch <[email protected]>
Kernel configs don't necessarily have opcode decoding, and some opcodes are not even decodable. It is still interesting for debugging SSD issues to know what opcode is timing out, what request type it came from, and the data size (if applicable).
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)drivers/nvme/host/pci.c +1
-0@@ -1284,6 +1284,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req)struct request *abort_req;struct nvme_command cmd = { };u32 csts =readl (dev->bar +NVME_REG_CSTS ); +u8 opcode;/* If PCI error recovery process is happening, we cannot reset or * the recovery mechanism will surely fail. @@ -1361,11 +1362,11 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req) cmd.abort.cid =nvme_cid (req); cmd.abort.sqid =cpu_to_le16 (nvmeq->qid);
drivers/nvme/host/pci.c static enum blk_eh_timer_return nvme_timeout(struct request *req) · line 1369
Quoted 10 lines
drivers/nvme/host/pci.c +4
-4+ opcode =nvme_req (req)->cmd->common.opcode,dev_warn (nvmeq->dev->ctrl.device, -"I/O %d (%s) QID %d timeout, aborting\n" , - req->tag, -nvme_get_opcode_str (nvme_req (req)->cmd->common.opcode), - nvmeq->qid); +"I/O %d (%s %d) QID %d timeout, aborting req_op:%u size:%u\n" , + req->tag,nvme_get_opcode_str (opcode), opcode, nvmeq->qid, +req_op (req),blk_rq_bytes (req));
Your additions look good to me, but I do wish that we'd be equally verbose on what the other values are. Ala:
I/O tag %d (opcode: %s %d) ...
would be a lot more useful, for example. I find myself having to dig into the source for a specific kernel sometimes when looking at various nvme errors, which is really annoying.
Signature · 1 line
Jens Axboe
Keith Busch
maintainerQuoted 42 lines · Jens Axboe
Quoted 27 lines · Keith Busch
From: Keith Busch <[email protected]>
Kernel configs don't necessarily have opcode decoding, and some opcodes are not even decodable. It is still interesting for debugging SSD issues to know what opcode is timing out, what request type it came from, and the data size (if applicable).
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)drivers/nvme/host/pci.c +1
-0@@ -1284,6 +1284,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req)struct request *abort_req;struct nvme_command cmd = { };u32 csts =readl (dev->bar +NVME_REG_CSTS ); +u8 opcode;/* If PCI error recovery process is happening, we cannot reset or * the recovery mechanism will surely fail. @@ -1361,11 +1362,11 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req) cmd.abort.cid =nvme_cid (req); cmd.abort.sqid =cpu_to_le16 (nvmeq->qid);drivers/nvme/host/pci.c static enum blk_eh_timer_return nvme_timeout(struct request *req) · line 1369
Quoted 10 lines
drivers/nvme/host/pci.c +4
-4+ opcode =nvme_req (req)->cmd->common.opcode,dev_warn (nvmeq->dev->ctrl.device, -"I/O %d (%s) QID %d timeout, aborting\n" , - req->tag, -nvme_get_opcode_str (nvme_req (req)->cmd->common.opcode), - nvmeq->qid); +"I/O %d (%s %d) QID %d timeout, aborting req_op:%u size:%u\n" , + req->tag,nvme_get_opcode_str (opcode), opcode, nvmeq->qid, +req_op (req),blk_rq_bytes (req));Your additions look good to me, but I do wish that we'd be equally verbose on what the other values are. Ala:
Quoted 5 lines
I/O tag %d (opcode: %s %d) ...would be a lot more useful, for example. I find myself having to dig into the source for a specific kernel sometimes when looking at various nvme errors, which is really annoying.
That is annoying. I'll add your suggestion in.
Quoted 43 lines · Keith Busch
Quoted 42 lines · Jens Axboe
Quoted 27 lines · Keith Busch
From: Keith Busch <[email protected]>
Kernel configs don't necessarily have opcode decoding, and some opcodes are not even decodable. It is still interesting for debugging SSD issues to know what opcode is timing out, what request type it came from, and the data size (if applicable).
Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/pci.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)drivers/nvme/host/pci.c +1
-0@@ -1284,6 +1284,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req)struct request *abort_req;struct nvme_command cmd = { };u32 csts =readl (dev->bar +NVME_REG_CSTS ); +u8 opcode;/* If PCI error recovery process is happening, we cannot reset or * the recovery mechanism will surely fail. @@ -1361,11 +1362,11 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req) cmd.abort.cid =nvme_cid (req); cmd.abort.sqid =cpu_to_le16 (nvmeq->qid);drivers/nvme/host/pci.c static enum blk_eh_timer_return nvme_timeout(struct request *req) · line 1369
Quoted 10 lines
drivers/nvme/host/pci.c +4
-4+ opcode =nvme_req (req)->cmd->common.opcode,dev_warn (nvmeq->dev->ctrl.device, -"I/O %d (%s) QID %d timeout, aborting\n" , - req->tag, -nvme_get_opcode_str (nvme_req (req)->cmd->common.opcode), - nvmeq->qid); +"I/O %d (%s %d) QID %d timeout, aborting req_op:%u size:%u\n" , + req->tag,nvme_get_opcode_str (opcode), opcode, nvmeq->qid, +req_op (req),blk_rq_bytes (req));Your additions look good to me, but I do wish that we'd be equally verbose on what the other values are. Ala:
Quoted 8 lines
Quoted 5 lines
I/O tag %d (opcode: %s %d) ...would be a lot more useful, for example. I find myself having to dig into the source for a specific kernel sometimes when looking at various nvme errors, which is really annoying.
That is annoying. I'll add your suggestion in.
while doing that is it worth printing the blk_op_str(req_op(req)) along with the req_op(req) since we are doing verbose anyway ? if it is not worth disregard my suggestion ...
-ck