Skip to contentSkip to conversation

block: Fix handling of offline queues in blk_mq_alloc_request_hctx()

PATCH Applied Bart Van Assche 5 participants 6 messages Jun 15, 2022 – Jun 16, 2022 Block layer

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 block: Fix handling of offline queues in blk_mq_alloc_request_hctx()

This patch prevents that test nvme/004 triggers the following:

UBSAN: array-index-out-of-bounds in block/blk-mq.h:135:9
index 512 is out of range for type 'long unsigned int [512]'
Call Trace:
 show_stack+0x52/0x58
 dump_stack_lvl+0x49/0x5e
 dump_stack+0x10/0x12
 ubsan_epilogue+0x9/0x3b
 __ubsan_handle_out_of_bounds.cold+0x44/0x49
 blk_mq_alloc_request_hctx+0x304/0x310
 __nvme_submit_sync_cmd+0x70/0x200 [nvme_core]
 nvmf_connect_io_queue+0x23e/0x2a0 [nvme_fabrics]
 nvme_loop_connect_io_queues+0x8d/0xb0 [nvme_loop]
 nvme_loop_create_ctrl+0x58e/0x7d0 [nvme_loop]
 nvmf_create_ctrl+0x1d7/0x4d0 [nvme_fabrics]
 nvmf_dev_write+0xae/0x111 [nvme_fabrics]
 vfs_write+0x144/0x560
 ksys_write+0xb7/0x140
 __x64_sys_write+0x42/0x50
 do_syscall_64+0x35/0x80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Fixes 20e4d8139319 ("blk-mq: simplify queue mapping & schedule with each possisble CPU")
---
 block/blk-mq.c | 2 ++
 1 file changed, 2 insertions(+)
block/blk-mq.c +2 -0
index 7a5558bbc7f6..1c09c6017ea9 100644
@@ -579,6 +579,8 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	if (!blk_mq_hw_queue_mapped(data.hctx))
 		goto out_queue_exit;
 	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+	if (cpu >= nr_cpu_ids)
+		goto out_queue_exit;
 	data.ctx = __blk_mq_get_ctx(q, cpu);
 
 	if (!q->elevator)
Reviewed-by on v1 1/1
#1 lore raw
Quoted 37 lines · Bart Van Assche

This patch prevents that test nvme/004 triggers the following:

UBSAN: array-index-out-of-bounds in block/blk-mq.h:135:9
index 512 is out of range for type 'long unsigned int [512]'
Call Trace:
 show_stack+0x52/0x58
 dump_stack_lvl+0x49/0x5e
 dump_stack+0x10/0x12
 ubsan_epilogue+0x9/0x3b
 __ubsan_handle_out_of_bounds.cold+0x44/0x49
 blk_mq_alloc_request_hctx+0x304/0x310
 __nvme_submit_sync_cmd+0x70/0x200 [nvme_core]
 nvmf_connect_io_queue+0x23e/0x2a0 [nvme_fabrics]
 nvme_loop_connect_io_queues+0x8d/0xb0 [nvme_loop]
 nvme_loop_create_ctrl+0x58e/0x7d0 [nvme_loop]
 nvmf_create_ctrl+0x1d7/0x4d0 [nvme_fabrics]
 nvmf_dev_write+0xae/0x111 [nvme_fabrics]
 vfs_write+0x144/0x560
 ksys_write+0xb7/0x140
 __x64_sys_write+0x42/0x50
 do_syscall_64+0x35/0x80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Fixes 20e4d8139319 ("blk-mq: simplify queue mapping & schedule with each possisble CPU")
---
 block/blk-mq.c | 2 ++
 1 file changed, 2 insertions(+)
block/blk-mq.c +0 -0
index 7a5558bbc7f6..1c09c6017ea9 100644

block/blk-mq.c struct request *blk_mq_alloc_request_hctx(struct request_queue *q, · line 586

Quoted 8 lines
block/blk-mq.c +2 -0
@@ -579,6 +579,8 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	if (!blk_mq_hw_queue_mapped(data.hctx))
 		goto out_queue_exit;
 	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+	if (cpu >= nr_cpu_ids)
+		goto out_queue_exit;
 	data.ctx = __blk_mq_get_ctx(q, cpu);
 
 	if (!q->elevator)

Thanks,
Ming

#3 lore raw
In reply to Bart Van Assche
Quoted 36 lines · Bart Van Assche

This patch prevents that test nvme/004 triggers the following:

UBSAN: array-index-out-of-bounds in block/blk-mq.h:135:9
index 512 is out of range for type 'long unsigned int [512]'
Call Trace:
  show_stack+0x52/0x58
  dump_stack_lvl+0x49/0x5e
  dump_stack+0x10/0x12
  ubsan_epilogue+0x9/0x3b
  __ubsan_handle_out_of_bounds.cold+0x44/0x49
  blk_mq_alloc_request_hctx+0x304/0x310
  __nvme_submit_sync_cmd+0x70/0x200 [nvme_core]
  nvmf_connect_io_queue+0x23e/0x2a0 [nvme_fabrics]
  nvme_loop_connect_io_queues+0x8d/0xb0 [nvme_loop]
  nvme_loop_create_ctrl+0x58e/0x7d0 [nvme_loop]
  nvmf_create_ctrl+0x1d7/0x4d0 [nvme_fabrics]
  nvmf_dev_write+0xae/0x111 [nvme_fabrics]
  vfs_write+0x144/0x560
  ksys_write+0xb7/0x140
  __x64_sys_write+0x42/0x50
  do_syscall_64+0x35/0x80
  entry_SYSCALL_64_after_hwframe+0x44/0xae
Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Fixes 20e4d8139319 ("blk-mq: simplify queue mapping & schedule with each possisble CPU")
---
  block/blk-mq.c | 2 ++
  1 file changed, 2 insertions(+)
block/blk-mq.c +0 -0
index 7a5558bbc7f6..1c09c6017ea9 100644

block/blk-mq.c struct request *blk_mq_alloc_request_hctx(struct request_queue *q, · line 583

Quoted 5 lines
block/blk-mq.c +2 -0
@@ -579,6 +579,8 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
  	if (!blk_mq_hw_queue_mapped(data.hctx))
  		goto out_queue_exit;
  	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+	if (cpu >= nr_cpu_ids)
+		goto out_queue_exit;

Ming, Did you give up on this:

https://lore.kernel.org/linux-block/[email protected]/

Thanks,
John

Quoted 4 lines
block/blk-mq.c +0 -0
  	data.ctx = __blk_mq_get_ctx(q, cpu);
 
  	if (!q->elevator)

.

#4 lore raw
Quoted 45 lines · John Garry
Quoted 36 lines · Bart Van Assche

This patch prevents that test nvme/004 triggers the following:

UBSAN: array-index-out-of-bounds in block/blk-mq.h:135:9
index 512 is out of range for type 'long unsigned int [512]'
Call Trace:
  show_stack+0x52/0x58
  dump_stack_lvl+0x49/0x5e
  dump_stack+0x10/0x12
  ubsan_epilogue+0x9/0x3b
  __ubsan_handle_out_of_bounds.cold+0x44/0x49
  blk_mq_alloc_request_hctx+0x304/0x310
  __nvme_submit_sync_cmd+0x70/0x200 [nvme_core]
  nvmf_connect_io_queue+0x23e/0x2a0 [nvme_fabrics]
  nvme_loop_connect_io_queues+0x8d/0xb0 [nvme_loop]
  nvme_loop_create_ctrl+0x58e/0x7d0 [nvme_loop]
  nvmf_create_ctrl+0x1d7/0x4d0 [nvme_fabrics]
  nvmf_dev_write+0xae/0x111 [nvme_fabrics]
  vfs_write+0x144/0x560
  ksys_write+0xb7/0x140
  __x64_sys_write+0x42/0x50
  do_syscall_64+0x35/0x80
  entry_SYSCALL_64_after_hwframe+0x44/0xae
Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Fixes 20e4d8139319 ("blk-mq: simplify queue mapping & schedule with each possisble CPU")
---
  block/blk-mq.c | 2 ++
  1 file changed, 2 insertions(+)
block/blk-mq.c +0 -0
index 7a5558bbc7f6..1c09c6017ea9 100644

block/blk-mq.c struct request *blk_mq_alloc_request_hctx(struct request_queue *q, · line 583

Quoted 5 lines
block/blk-mq.c +2 -0
@@ -579,6 +579,8 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
  	if (!blk_mq_hw_queue_mapped(data.hctx))
  		goto out_queue_exit;
  	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
+	if (cpu >= nr_cpu_ids)
+		goto out_queue_exit;

Ming, Did you give up on this:

Quoted 1 line

https://lore.kernel.org/linux-block/[email protected]/

Yeah, this approach has some big problem, see

https://lore.kernel.org/linux-block/[email protected]/#t

Thanks,
Ming

maintainer Applied
#5 lore raw
In reply to Bart Van Assche
Quoted 23 lines · Bart Van Assche

This patch prevents that test nvme/004 triggers the following:

UBSAN: array-index-out-of-bounds in block/blk-mq.h:135:9
index 512 is out of range for type 'long unsigned int [512]'
Call Trace:
 show_stack+0x52/0x58
 dump_stack_lvl+0x49/0x5e
 dump_stack+0x10/0x12
 ubsan_epilogue+0x9/0x3b
 __ubsan_handle_out_of_bounds.cold+0x44/0x49
 blk_mq_alloc_request_hctx+0x304/0x310
 __nvme_submit_sync_cmd+0x70/0x200 [nvme_core]
 nvmf_connect_io_queue+0x23e/0x2a0 [nvme_fabrics]
 nvme_loop_connect_io_queues+0x8d/0xb0 [nvme_loop]
 nvme_loop_create_ctrl+0x58e/0x7d0 [nvme_loop]
 nvmf_create_ctrl+0x1d7/0x4d0 [nvme_fabrics]
 nvmf_dev_write+0xae/0x111 [nvme_fabrics]
 vfs_write+0x144/0x560
 ksys_write+0xb7/0x140
 __x64_sys_write+0x42/0x50
 do_syscall_64+0x35/0x80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
Quoted 1 line

[...]

Applied, thanks!

[1/1] block: Fix handling of offline queues in blk_mq_alloc_request_hctx()

      commit: 14dc7a18abbe4176f5626c13c333670da8e06aa1

Best regards,

Signature · 1 line
Jens Axboe