Skip to contentSkip to conversation

nvme: check for valid data from from nvme_identify_ns() before using it

PATCH Applied Ewan D. Milne 2 participants 5 messages Nov 27, 2023 – Nov 28, 2023 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 nvme: check for valid data from from nvme_identify_ns() before using it

When scanning namespaces, it is possible to get valid data from the first call to nvme_identify_ns() in nvme_alloc_ns(), but not from the second call in nvme_update_ns_info_block(). In particular, if the NSID becomes inactive between the two commands, a storage device may return a buffer filled with zero as per 4.1.5.1. In this case, we can get a kernel crash due to a divide-by-zero in blk_stack_limits() because ns->lba_shift will be set to zero.

PID: 326 TASK: ffff95fec3cd8000 CPU: 29 COMMAND: "kworker/u98:10"
#0 [ffffad8f8702f9e0] machine_kexec at ffffffff91c76ec7 #1 [ffffad8f8702fa38] __crash_kexec at ffffffff91dea4fa #2 [ffffad8f8702faf8] crash_kexec at ffffffff91deb788 #3 [ffffad8f8702fb00] oops_end at ffffffff91c2e4bb #4 [ffffad8f8702fb20] do_trap at ffffffff91c2a4ce
#5 [ffffad8f8702fb70] do_error_trap at ffffffff91c2a595 #6 [ffffad8f8702fbb0] exc_divide_error at ffffffff928506e6 #7 [ffffad8f8702fbd0] asm_exc_divide_error at ffffffff92a00926

    [exception RIP: blk_stack_limits+434]
    RIP: ffffffff92191872  RSP: ffffad8f8702fc80  RFLAGS: 00010246
    RAX: 0000000000000000  RBX: ffff95efa0c91800  RCX: 0000000000000001
    RDX: 0000000000000000  RSI: 0000000000000001  RDI: 0000000000000001
    RBP: 00000000ffffffff   R8: ffff95fec7df35a8   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000001  R12: 0000000000000000
    R13: 0000000000000000  R14: 0000000000000000  R15: ffff95fed33c09a8
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#8 [ffffad8f8702fce0] nvme_update_ns_info_block at ffffffffc06d3533 [nvme_core] #9 [ffffad8f8702fd18] nvme_scan_ns at ffffffffc06d6fa7 [nvme_core]

This happened when the check for valid data was moved out of nvme_identify_ns() into one of the callers. Fix this by checking in both callers.

v2: call kfree() on nvme_id_ns struct in error path

Fixes 0dd6fff2aad4 ("nvme: bring back auto-removal of deleted namespaces during sequential scan")
---
 drivers/nvme/host/core.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/nvme/host/core.c +9 -0
index af94d067ac14..a55c2a774b9c 100644
@@ -2040,6 +2040,13 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,
 	if (ret)
 		return ret;
 
+	if (id->ncap == 0) {
+		/* namespace not allocated or attached */
+		info->is_removed = true;
+		ret = -ENODEV;
+		goto error;
+	}
+
 	blk_mq_freeze_queue(ns->disk->queue);
 	lbaf = nvme_lbaf_index(id->flbas);
 	ns->lba_shift = id->lbaf[lbaf].ds;
@@ -2101,6 +2108,8 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,
 		set_bit(NVME_NS_READY, &ns->flags);
 		ret = 0;
 	}
+
+error:
 	kfree(id);
 	return ret;
 }
Signature · 1 line
2.20.1
maintainer Applied
#1 lore raw
Quoted 28 lines · Ewan D. Milne

When scanning namespaces, it is possible to get valid data from the first call to nvme_identify_ns() in nvme_alloc_ns(), but not from the second call in nvme_update_ns_info_block(). In particular, if the NSID becomes inactive between the two commands, a storage device may return a buffer filled with zero as per 4.1.5.1. In this case, we can get a kernel crash due to a divide-by-zero in blk_stack_limits() because ns->lba_shift will be set to zero.

PID: 326 TASK: ffff95fec3cd8000 CPU: 29 COMMAND: "kworker/u98:10"
#0 [ffffad8f8702f9e0] machine_kexec at ffffffff91c76ec7 #1 [ffffad8f8702fa38] __crash_kexec at ffffffff91dea4fa #2 [ffffad8f8702faf8] crash_kexec at ffffffff91deb788 #3 [ffffad8f8702fb00] oops_end at ffffffff91c2e4bb #4 [ffffad8f8702fb20] do_trap at ffffffff91c2a4ce
#5 [ffffad8f8702fb70] do_error_trap at ffffffff91c2a595 #6 [ffffad8f8702fbb0] exc_divide_error at ffffffff928506e6 #7 [ffffad8f8702fbd0] asm_exc_divide_error at ffffffff92a00926

    [exception RIP: blk_stack_limits+434]
    RIP: ffffffff92191872  RSP: ffffad8f8702fc80  RFLAGS: 00010246
    RAX: 0000000000000000  RBX: ffff95efa0c91800  RCX: 0000000000000001
    RDX: 0000000000000000  RSI: 0000000000000001  RDI: 0000000000000001
    RBP: 00000000ffffffff   R8: ffff95fec7df35a8   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000001  R12: 0000000000000000
    R13: 0000000000000000  R14: 0000000000000000  R15: ffff95fed33c09a8
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#8 [ffffad8f8702fce0] nvme_update_ns_info_block at ffffffffc06d3533 [nvme_core] #9 [ffffad8f8702fd18] nvme_scan_ns at ffffffffc06d6fa7 [nvme_core]

Quoted 8 lines

This happened when the check for valid data was moved out of nvme_identify_ns() into one of the callers. Fix this by checking in both callers.

v2: call kfree() on nvme_id_ns struct in error path

Fixes 0dd6fff2aad4 ("nvme: bring back auto-removal of deleted namespaces during sequential scan")

Thanks, applied for nvme-6.7.

Interestingly enough, I think this is the same as what was recently reported here:

https://bugzilla.kernel.org/show_bug.cgi?id=218186

author
#2 lore raw
Quoted 39 lines · Keith Busch
Quoted 28 lines · Ewan D. Milne

When scanning namespaces, it is possible to get valid data from the first call to nvme_identify_ns() in nvme_alloc_ns(), but not from the second call in nvme_update_ns_info_block(). In particular, if the NSID becomes inactive between the two commands, a storage device may return a buffer filled with zero as per 4.1.5.1. In this case, we can get a kernel crash due to a divide-by-zero in blk_stack_limits() because ns->lba_shift will be set to zero.

PID: 326 TASK: ffff95fec3cd8000 CPU: 29 COMMAND: "kworker/u98:10"
#0 [ffffad8f8702f9e0] machine_kexec at ffffffff91c76ec7 #1 [ffffad8f8702fa38] __crash_kexec at ffffffff91dea4fa #2 [ffffad8f8702faf8] crash_kexec at ffffffff91deb788 #3 [ffffad8f8702fb00] oops_end at ffffffff91c2e4bb #4 [ffffad8f8702fb20] do_trap at ffffffff91c2a4ce
#5 [ffffad8f8702fb70] do_error_trap at ffffffff91c2a595 #6 [ffffad8f8702fbb0] exc_divide_error at ffffffff928506e6 #7 [ffffad8f8702fbd0] asm_exc_divide_error at ffffffff92a00926

    [exception RIP: blk_stack_limits+434]
    RIP: ffffffff92191872  RSP: ffffad8f8702fc80  RFLAGS: 00010246
    RAX: 0000000000000000  RBX: ffff95efa0c91800  RCX: 0000000000000001
    RDX: 0000000000000000  RSI: 0000000000000001  RDI: 0000000000000001
    RBP: 00000000ffffffff   R8: ffff95fec7df35a8   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000001  R12: 0000000000000000
    R13: 0000000000000000  R14: 0000000000000000  R15: ffff95fed33c09a8
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#8 [ffffad8f8702fce0] nvme_update_ns_info_block at ffffffffc06d3533 [nvme_core] #9 [ffffad8f8702fd18] nvme_scan_ns at ffffffffc06d6fa7 [nvme_core]

Quoted 8 lines

This happened when the check for valid data was moved out of nvme_identify_ns() into one of the callers. Fix this by checking in both callers.

v2: call kfree() on nvme_id_ns struct in error path

Fixes 0dd6fff2aad4 ("nvme: bring back auto-removal of deleted namespaces during sequential scan")
Quoted 7 lines

Thanks, applied for nvme-6.7.

Interestingly enough, I think this is the same as what was recently reported here:

https://bugzilla.kernel.org/show_bug.cgi?id=218186

Yes, from the stack trace and BZ comments it looks like it.

-Ewan

maintainer
#3 lore raw
Quoted 7 lines · Ewan Milne
Quoted 5 lines

Interestingly enough, I think this is the same as what was recently reported here:

https://bugzilla.kernel.org/show_bug.cgi?id=218186

Yes, from the stack trace and BZ comments it looks like it.

Slightly related, there's nothing stopping a device from reporting a bogus LBA shift value. We're only checking for "too big", but neglect to check for too small, and may similiarly shift off to a division by 0. We might need a check like this too:

---

drivers/nvme/host/core.c +1 -1
index a55c2a774b9c4..8bf78f876f7ac 100644
@@ -1903,7 +1903,7 @@ static void nvme_update_disk_info(struct gendisk *disk,
 	 * The block layer can't support LBA sizes larger than the page size
 	 * yet, so catch this early and don't allow block I/O.
 	 */
-	if (ns->lba_shift > PAGE_SHIFT) {
+	if (ns->lba_shift > PAGE_SHIFT || ns->lba_shift < SECTOR_SHIFT) {
 		capacity = 0;
 		bs = (1 << 9);
 	}

--

author
#4 lore raw

Sure, the kernel should not crash regardless of what any device reports, especially an external one.

The _identify_ns() divide by zero happened because the NVMe spec says a page of zeroes can be returned if a namespace isn't configured. I'm not really sure why the spec didn't require a nonzero status code.

-Ewan

Quoted 31 lines · Keith Busch
Quoted 7 lines · Ewan Milne
Quoted 5 lines

Interestingly enough, I think this is the same as what was recently reported here:

https://bugzilla.kernel.org/show_bug.cgi?id=218186

Yes, from the stack trace and BZ comments it looks like it.

Slightly related, there's nothing stopping a device from reporting a bogus LBA shift value. We're only checking for "too big", but neglect to check for too small, and may similiarly shift off to a division by 0. We might need a check like this too:

---

drivers/nvme/host/core.c +1 -2
index a55c2a774b9c4..8bf78f876f7ac 100644
@@ -1903,7 +1903,7 @@ static void nvme_update_disk_info(struct gendisk *disk,
         * The block layer can't support LBA sizes larger than the page size
         * yet, so catch this early and don't allow block I/O.
         */
-       if (ns->lba_shift > PAGE_SHIFT) {
+       if (ns->lba_shift > PAGE_SHIFT || ns->lba_shift < SECTOR_SHIFT) {
                capacity = 0;
                bs = (1 << 9);
        }
--