nvme: check for valid data from from nvme_identify_ns() before using it
PATCH v2 1/1 nvme: check for valid data from from nvme_identify_ns() before using it
When scanning namespaces, it is possible to get valid data from the first call to nvme_identify_ns() in nvme_alloc_ns(), but not from the second call in nvme_update_ns_info_block(). In particular, if the NSID becomes inactive between the two commands, a storage device may return a buffer filled with zero as per 4.1.5.1. In this case, we can get a kernel crash due to a divide-by-zero in blk_stack_limits() because ns->lba_shift will be set to zero.
PID: 326 TASK: ffff95fec3cd8000 CPU: 29 COMMAND: "kworker/u98:10"
#0 [ffffad8f8702f9e0] machine_kexec at ffffffff91c76ec7 #1 [ffffad8f8702fa38] __crash_kexec at ffffffff91dea4fa #2 [ffffad8f8702faf8] crash_kexec at ffffffff91deb788 #3 [ffffad8f8702fb00] oops_end at ffffffff91c2e4bb #4 [ffffad8f8702fb20] do_trap at ffffffff91c2a4ce
#5 [ffffad8f8702fb70] do_error_trap at ffffffff91c2a595 #6 [ffffad8f8702fbb0] exc_divide_error at ffffffff928506e6 #7 [ffffad8f8702fbd0] asm_exc_divide_error at ffffffff92a00926
[exception RIP: blk_stack_limits+434]
RIP: ffffffff92191872 RSP: ffffad8f8702fc80 RFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff95efa0c91800 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001
RBP: 00000000ffffffff R8: ffff95fec7df35a8 R9: 0000000000000000
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: ffff95fed33c09a8
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #8 [ffffad8f8702fce0] nvme_update_ns_info_block at ffffffffc06d3533 [nvme_core] #9 [ffffad8f8702fd18] nvme_scan_ns at ffffffffc06d6fa7 [nvme_core]
This happened when the check for valid data was moved out of nvme_identify_ns() into one of the callers. Fix this by checking in both callers.
v2: call kfree() on nvme_id_ns struct in error path
--- drivers/nvme/host/core.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/nvme/host/core.c +9 -0
@@ -2040,6 +2040,13 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,if (ret)return ret; +if (id->ncap ==0 ) { +/* namespace not allocated or attached */ + info->is_removed = true; + ret = -ENODEV ; + goto error; + } +blk_mq_freeze_queue (ns->disk->queue); lbaf =nvme_lbaf_index (id->flbas); ns->lba_shift = id->lbaf[lbaf].ds;
@@ -2101,6 +2108,8 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,set_bit (NVME_NS_READY , &ns->flags); ret =0 ; } + +error:kfree (id);return ret; }
Signature · 1 line
2.20.1
Quoted 28 lines · Ewan D. Milne
When scanning namespaces, it is possible to get valid data from the first call to
nvme_identify_ns()innvme_alloc_ns(), but not from the second call innvme_update_ns_info_block(). In particular, if the NSID becomes inactive between the two commands, a storage device may return a buffer filled with zero as per 4.1.5.1. In this case, we can get a kernel crash due to a divide-by-zero inblk_stack_limits()becausens->lba_shiftwill be set to zero.PID: 326 TASK:
ffff95fec3cd8000CPU: 29 COMMAND: "kworker/u98:10"
#0 [ffffad8f8702f9e0]machine_kexecatffffffff91c76ec7#1 [ffffad8f8702fa38]__crash_kexecatffffffff91dea4fa#2 [ffffad8f8702faf8]crash_kexecatffffffff91deb788#3 [ffffad8f8702fb00]oops_endatffffffff91c2e4bb#4 [ffffad8f8702fb20]do_trapatffffffff91c2a4ce
#5 [ffffad8f8702fb70]do_error_trapatffffffff91c2a595#6 [ffffad8f8702fbb0]exc_divide_erroratffffffff928506e6#7 [ffffad8f8702fbd0]asm_exc_divide_erroratffffffff92a00926[exception RIP: blk_stack_limits+434] RIP: ffffffff92191872 RSP: ffffad8f8702fc80 RFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff95efa0c91800 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001 RBP: 00000000ffffffff R8: ffff95fec7df35a8 R9: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: ffff95fed33c09a8 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018#8 [
ffffad8f8702fce0]nvme_update_ns_info_blockatffffffffc06d3533[nvme_core] #9 [ffffad8f8702fd18]nvme_scan_nsatffffffffc06d6fa7[nvme_core]
Quoted 8 lines
This happened when the check for valid data was moved out of
nvme_identify_ns()into one of the callers. Fix this by checking in both callers.v2: call
kfree()onnvme_id_nsstruct in error pathFixes 0dd6fff2aad4 ("nvme: bring back auto-removal of deleted namespaces during sequential scan")Signed-off-by Ewan D. Milne <[email protected]>
Thanks, applied for nvme-6.7.
Interestingly enough, I think this is the same as what was recently reported here:
Quoted 39 lines · Keith Busch
Quoted 28 lines · Ewan D. Milne
When scanning namespaces, it is possible to get valid data from the first call to
nvme_identify_ns()innvme_alloc_ns(), but not from the second call innvme_update_ns_info_block(). In particular, if the NSID becomes inactive between the two commands, a storage device may return a buffer filled with zero as per 4.1.5.1. In this case, we can get a kernel crash due to a divide-by-zero inblk_stack_limits()becausens->lba_shiftwill be set to zero.PID: 326 TASK:
ffff95fec3cd8000CPU: 29 COMMAND: "kworker/u98:10"
#0 [ffffad8f8702f9e0]machine_kexecatffffffff91c76ec7#1 [ffffad8f8702fa38]__crash_kexecatffffffff91dea4fa#2 [ffffad8f8702faf8]crash_kexecatffffffff91deb788#3 [ffffad8f8702fb00]oops_endatffffffff91c2e4bb#4 [ffffad8f8702fb20]do_trapatffffffff91c2a4ce
#5 [ffffad8f8702fb70]do_error_trapatffffffff91c2a595#6 [ffffad8f8702fbb0]exc_divide_erroratffffffff928506e6#7 [ffffad8f8702fbd0]asm_exc_divide_erroratffffffff92a00926[exception RIP: blk_stack_limits+434] RIP: ffffffff92191872 RSP: ffffad8f8702fc80 RFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff95efa0c91800 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001 RBP: 00000000ffffffff R8: ffff95fec7df35a8 R9: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: ffff95fed33c09a8 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018#8 [
ffffad8f8702fce0]nvme_update_ns_info_blockatffffffffc06d3533[nvme_core] #9 [ffffad8f8702fd18]nvme_scan_nsatffffffffc06d6fa7[nvme_core]Quoted 8 lines
This happened when the check for valid data was moved out of
nvme_identify_ns()into one of the callers. Fix this by checking in both callers.v2: call
kfree()onnvme_id_nsstruct in error pathFixes 0dd6fff2aad4 ("nvme: bring back auto-removal of deleted namespaces during sequential scan")Signed-off-by Ewan D. Milne <[email protected]>
Quoted 7 lines
Thanks, applied for nvme-6.7.
Interestingly enough, I think this is the same as what was recently reported here:
Yes, from the stack trace and BZ comments it looks like it.
-Ewan
Keith Busch
maintainerQuoted 7 lines · Ewan Milne
Quoted 5 lines
Interestingly enough, I think this is the same as what was recently reported here:
Yes, from the stack trace and BZ comments it looks like it.
Slightly related, there's nothing stopping a device from reporting a bogus LBA shift value. We're only checking for "too big", but neglect to check for too small, and may similiarly shift off to a division by 0. We might need a check like this too:
---
drivers/nvme/host/core.c +1 -1
@@ -1903,7 +1903,7 @@ static void nvme_update_disk_info(struct gendisk *disk,* The block layer can't support LBA sizes larger than the page size * yet, so catch this early and don't allow block I/O. */ -if (ns->lba_shift >PAGE_SHIFT ) { +if (ns->lba_shift >PAGE_SHIFT || ns->lba_shift <SECTOR_SHIFT ) { capacity =0 ; bs = (1 <<9 ); }
--
Sure, the kernel should not crash regardless of what any device reports, especially an external one.
The _identify_ns() divide by zero happened because the NVMe spec says a page of zeroes can be returned if a namespace isn't configured. I'm not really sure why the spec didn't require a nonzero status code.
-Ewan
Quoted 31 lines · Keith Busch
Quoted 7 lines · Ewan Milne
Quoted 5 lines
Interestingly enough, I think this is the same as what was recently reported here:
Yes, from the stack trace and BZ comments it looks like it.
Slightly related, there's nothing stopping a device from reporting a bogus LBA shift value. We're only checking for "too big", but neglect to check for too small, and may similiarly shift off to a division by 0. We might need a check like this too:
---
drivers/nvme/host/core.c +1
-2@@ -1903,7 +1903,7 @@ static void nvme_update_disk_info(struct gendisk *disk,* The block layer can't support LBA sizes larger than the page size * yet, so catch this early and don't allow block I/O. */ -if (ns->lba_shift >PAGE_SHIFT ) { +if (ns->lba_shift >PAGE_SHIFT || ns->lba_shift <SECTOR_SHIFT ) { capacity =0 ; bs = (1 <<9 ); } --