nvme: reject completions for requests that are not in flight
PATCH 1/1 nvme: reject completions for requests that are not in flight
nvme_find_rq() resolves a device-supplied command id to a request with blk_mq_tag_to_rq(), which returns whatever request last used that tag - possibly one that is no longer in flight (freed, or never dispatched and thus with a NULL rq->mq_hctx). Commit e7006de6c238 ("nvme: code command_id with a genctr for use-after-free validation") guards against this, but its generation counter is only 4 bits wide and can be matched by a malfunctioning or malicious device replaying command ids. The driver then completes a request that is not outstanding, dereferencing a NULL rq->mq_hctx or double-completing a command:
Oops: general protection fault ... KASAN: null-ptr-deref RIP: blk_mq_complete_request_remote+0xe5/0xa80 block/blk-mq.c:1319 nvme_handle_cqe drivers/nvme/host/pci.c:1418 [inline] nvme_poll_cq drivers/nvme/host/pci.c:1449 nvme_irq drivers/nvme/host/pci.c:1463
Require the request to be in flight before completing it. The check uses the request state, so it also covers controllers with NVME_QUIRK_SKIP_CID_GEN.
Found by FuzzNvme(Syzkaller with FEMU fuzzing framework).
--- drivers/nvme/host/nvme.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/nvme/host/nvme.h +11 -0
@@ -615,6 +615,17 @@ static inline struct request *nvme_find_rq(struct blk_mq_tags *tags, tag);return NULL ; } +/* +* blk_mq_tag_to_rq() returns whatever request last used this tag, which +* may no longer be in flight if the device reports a bogus command id. +* Completing it would deref a NULL rq->mq_hctx or double-complete a +* command; the 4-bit genctr below only narrows the window. +*/ +if (unlikely (blk_mq_rq_state (rq) !=MQ_RQ_IN_FLIGHT )) { +dev_err (nvme_req (rq)->ctrl->device, +"completion for request %#x not in flight\n" , tag); +return NULL ; + }if (unlikely (nvme_genctr_mask (nvme_req (rq)->genctr) != genctr)) { dev_err(nvme_req(rq)->ctrl->device, "request %#x genctr mismatch (got %#x expected %#x)\n",
Signature · 1 line
2.43.0
Hi,
Since posting this I reproduced a more severe manifestation of the same bug and confirmed the patch handles it; sharing as extra justification.
The commit message covers the freed / never-dispatched case (the NULL rq->mq_hctx dereference). When the stale command id instead maps to a tag that has already been *reused*, the driver completes an unrelated, still-in-flight request -- a use-after-free. Under fuzzing (a device that replays and reorders completions) this did not show up as a clean NULL deref but as cross-subsystem memory corruption: general protection faults in mtree_range_walk(), unlink_anon_vmas() and the slub freelist, in unrelated tasks (modprobe, systemd-udevd, ...). The trigger was a stale completion delivered for a request that a concurrent controller reset had just freed.
To confirm the fix addresses this, I rebuilt the kernel with the patch and re-ran the same workload for ~10h. The guard now rejects the offending completion instead of acting on it:
nvme nvme0: resetting controller
nvme nvme0: completion for request 0x1c0 not in flight nvme nvme0: invalid id 448 completed on queue 2
and no use-after-free / corruption recurred over the run.
The code is unchanged; I'm happy to fold this into the commit message as a v2 if you'd prefer it spelled out there.
Thanks,
Chao
Quoted 56 lines · Chao Shi
nvme_find_rq()resolves a device-supplied command id to a request withblk_mq_tag_to_rq(), which returns whatever request last used that tag - possibly one that is no longer in flight (freed, or never dispatched and thus with a NULLrq->mq_hctx). Commite7006de6c238("nvme: codecommand_idwith a genctr for use-after-free validation") guards against this, but its generation counter is only 4 bits wide and can be matched by a malfunctioning or malicious device replaying command ids. The driver then completes a request that is not outstanding, dereferencing a NULLrq->mq_hctxor double-completing a command:Oops: general protection fault ... KASAN: null-ptr-deref RIP: blk_mq_complete_request_remote+0xe5/0xa80 block/blk-mq.c:1319 nvme_handle_cqe drivers/nvme/host/pci.c:1418 [inline] nvme_poll_cq drivers/nvme/host/pci.c:1449 nvme_irq drivers/nvme/host/pci.c:1463Require the request to be in flight before completing it. The check uses the request state, so it also covers controllers with
NVME_QUIRK_SKIP_CID_GEN.Found by FuzzNvme(Syzkaller with FEMU fuzzing framework).
Acked-by Sungwoo Kim <[email protected]>Acked-by Dave Tian <[email protected]>Acked-by Weidong Zhu <[email protected]>Signed-off-by Chao Shi <[email protected]>--- drivers/nvme/host/nvme.h | 11 +++++++++++ 1 file changed, 11 insertions(+)drivers/nvme/host/nvme.h +11
-1@@ -615,6 +615,17 @@ static inline struct request *nvme_find_rq(struct blk_mq_tags *tags, tag);return NULL ; } +/* +* blk_mq_tag_to_rq() returns whatever request last used this tag, which +* may no longer be in flight if the device reports a bogus command id. +* Completing it would deref a NULL rq->mq_hctx or double-complete a +* command; the 4-bit genctr below only narrows the window. +*/ +if (unlikely (blk_mq_rq_state (rq) !=MQ_RQ_IN_FLIGHT )) { +dev_err (nvme_req (rq)->ctrl->device, +"completion for request %#x not in flight\n" , tag); +return NULL ; + }if (unlikely (nvme_genctr_mask (nvme_req (rq)->genctr) != genctr)) { dev_err(nvme_req(rq)->ctrl->device, "request %#x genctr mismatch (got %#x expected %#x)\n", --2.43.0
Christoph Hellwig
maintainerQuoted 9 lines · Chao Shi
nvme_find_rq()resolves a device-supplied command id to a request withblk_mq_tag_to_rq(), which returns whatever request last used that tag - possibly one that is no longer in flight (freed, or never dispatched and thus with a NULLrq->mq_hctx). Commite7006de6c238("nvme: codecommand_idwith a genctr for use-after-free validation") guards against this, but its generation counter is only 4 bits wide and can be matched by a malfunctioning or malicious device replaying command ids. The driver then completes a request that is not outstanding, dereferencing a NULLrq->mq_hctxor double-completing a command:
I don't think an intentionally malicious device is part of the threat model here. This was added to protect against buggy devices.
drivers/nvme/host/nvme.h static inline struct request *nvme_find_rq(struct blk_mq_tags *tags, · line 628
Quoted 11 lines
drivers/nvme/host/nvme.h +11
-0+/* +* blk_mq_tag_to_rq() returns whatever request last used this tag, which +* may no longer be in flight if the device reports a bogus command id. +* Completing it would deref a NULL rq->mq_hctx or double-complete a +* command; the 4-bit genctr below only narrows the window. +*/ +if (unlikely (blk_mq_rq_state (rq) !=MQ_RQ_IN_FLIGHT )) { +dev_err (nvme_req (rq)->ctrl->device, +"completion for request %#x not in flight\n" , tag); +return NULL ; + }
Although this check looks cheap enough that it should not hurt to add it. So I think this should be ok, but maybe respin with your planned commit message update.
Jens Axboe
maintainerQuoted 11 lines · Christoph Hellwig
Quoted 9 lines · Chao Shi
nvme_find_rq()resolves a device-supplied command id to a request withblk_mq_tag_to_rq(), which returns whatever request last used that tag - possibly one that is no longer in flight (freed, or never dispatched and thus with a NULLrq->mq_hctx). Commite7006de6c238("nvme: codecommand_idwith a genctr for use-after-free validation") guards against this, but its generation counter is only 4 bits wide and can be matched by a malfunctioning or malicious device replaying command ids. The driver then completes a request that is not outstanding, dereferencing a NULLrq->mq_hctxor double-completing a command:
Quoted 2 lines
I don't think an intentionally malicious device is part of the threat model here. This was added to protect against buggy devices.
Malicious devices are explicitly NOT part of the linux threat model. If this is a real device, I'd say go talk to whomever made it and get the firmware fixed. If this is a "hardening" effort to protect against the threat of malicious devices, then I don't think we should bother.
Quoted 6 lines
Quoted 6 lines
drivers/nvme/host/nvme.h +6
-0+* blk_mq_tag_to_rq() returns whatever request last used this tag, which +* may no longer be in flight if the device reports a bogus command id. +* Completing it would deref a NULL rq->mq_hctx or double-complete a +* command; the 4-bit genctr below only narrows the window. +*/ +if (unlikely (blk_mq_rq_state (rq) !=MQ_RQ_IN_FLIGHT )) {
Quoted 8 lines
drivers/nvme/host/nvme.h static inline struct request *nvme_find_rq(struct blk_mq_tags *tags, · line 628
Quoted 4 lines
drivers/nvme/host/nvme.h +4
-0+dev_err (nvme_req (rq)->ctrl->device, +"completion for request %#x not in flight\n" , tag); +return NULL ; + }Although this check looks cheap enough that it should not hurt to add it. So I think this should be ok, but maybe respin with your planned commit message update.
Only for the right reasons, imho.
Signature · 1 line
Jens Axboe
Hi Keith, Christoph, Jens,
Apologies for the slow reply. I've had several patches and crash analyses from the same fuzzing effort in flight at once, along with writing the work up against a paper deadline, and this thread slipped. That's on me; I'll be prompt on follow-ups from here.
On the framing -- you're right, and I'll drop the "malicious device" wording. e7006de6c238 was written for a malfunctioning controller; I've reworded it to match, with no threat-model claim.
On whether this belongs in blk-mq rather than nvme -- I've kept the check in nvme_find_rq() for now, since that is where a device-supplied command id becomes a request. It isn't unique to nvme, though; if you'd rather the check live in the blk-mq completion path, I'm happy to write it there.
Sending v2 with the reworded message shortly.
Thanks for the reviews,
Chao
Quoted 40 lines · Jens Axboe
Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Chao Shi
nvme_find_rq()resolves a device-supplied command id to a request withblk_mq_tag_to_rq(), which returns whatever request last used that tag - possibly one that is no longer in flight (freed, or never dispatched and thus with a NULLrq->mq_hctx). Commite7006de6c238("nvme: codecommand_idwith a genctr for use-after-free validation") guards against this, but its generation counter is only 4 bits wide and can be matched by a malfunctioning or malicious device replaying command ids. The driver then completes a request that is not outstanding, dereferencing a NULLrq->mq_hctxor double-completing a command:Quoted 2 lines
I don't think an intentionally malicious device is part of the threat model here. This was added to protect against buggy devices.
Malicious devices are explicitly NOT part of the linux threat model. If this is a real device, I'd say go talk to whomever made it and get the firmware fixed. If this is a "hardening" effort to protect against the threat of malicious devices, then I don't think we should bother.
Quoted 11 lines
drivers/nvme/host/nvme.h static inline struct request *nvme_find_rq(struct blk_mq_tags *tags, · line 628
Quoted 10 lines
drivers/nvme/host/nvme.h +10
-0+* blk_mq_tag_to_rq() returns whatever request last used this tag, which +* may no longer be in flight if the device reports a bogus command id. +* Completing it would deref a NULL rq->mq_hctx or double-complete a +* command; the 4-bit genctr below only narrows the window. +*/ +if (unlikely (blk_mq_rq_state (rq) !=MQ_RQ_IN_FLIGHT )) { +dev_err (nvme_req (rq)->ctrl->device, +"completion for request %#x not in flight\n" , tag); +return NULL ; + }Quoted 3 lines
Although this check looks cheap enough that it should not hurt to add it. So I think this should be ok, but maybe respin with your planned commit message update.
Only for the right reasons, imho.
--
Jens Axboe
Keith Busch
maintainerQuoted 5 lines · Chao Shi
Oops: general protection fault ... KASAN: null-ptr-deref RIP: blk_mq_complete_request_remote+0xe5/0xa80 block/blk-mq.c:1319 nvme_handle_cqe drivers/nvme/host/pci.c:1418 [inline] nvme_poll_cq drivers/nvme/host/pci.c:1449 nvme_irq drivers/nvme/host/pci.c:1463
This scenario doesn't sound specific to nvme. Should blk-mq completion check for the IN_FLIGHT state instead?