Skip to contentSkip to conversation

nvme: core: don't hold rcu read lock in nvme_ns_chr_uring_cmd_iopoll

PATCH Applied Ming Lei 4 participants 8 messages Aug 9, 2023 – Aug 11, 2023

Subsystems (2)
lore
Share & tools
lore

Ming Lei

author
#0 lore raw

PATCH 1/1 nvme: core: don't hold rcu read lock in nvme_ns_chr_uring_cmd_iopoll

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Fixes 585079b6e425 ("nvme: wire up async polling for io passthrough commands")
Cc Kanchan Joshi <[email protected]>
Cc Anuj Gupta <[email protected]>
---
 drivers/nvme/host/ioctl.c | 2 --
 1 file changed, 2 deletions(-)
drivers/nvme/host/ioctl.c +0 -2
index 5c3250f36ce7..d39f3219358b 100644
@@ -786,11 +786,9 @@ int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd,
 	if (!(ioucmd->flags & IORING_URING_CMD_POLLED))
 		return 0;
 
-	rcu_read_lock();
 	req = READ_ONCE(ioucmd->cookie);
 	if (req && blk_rq_is_poll(req))
 		ret = blk_rq_poll(req, iob, poll_flags);
-	rcu_read_unlock();
 	return ret;
 }
 #ifdef CONFIG_NVME_MULTIPATH
Signature · 1 line
2.40.1
#1 lore raw
Quoted 5 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Quoted 8 lines

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Fixes 585079b6e425 ("nvme: wire up async polling for io passthrough commands")

Thanks Ming. Looks fine, but any link to this report?
I don't see this breaking in my tests. So I wonder how to reproduce and improve the coverage.

author
#2 lore raw
Quoted 12 lines · Kanchan Joshi
Quoted 11 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Quoted 6 lines
Quoted 2 lines
Fixes 585079b6e425 ("nvme: wire up async polling for io passthrough commands")

Thanks Ming. Looks fine, but any link to this report?
I don't see this breaking in my tests. So I wonder how to reproduce and improve the coverage.

It is reported in RH BZ2227639, and follows the stack trace:

[ 3286.960425] BUG: sleeping function called from invalid context at include/linux/pagemap.h:914
[ 3286.960434] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 530910, name: fio
[ 3286.960440] preempt_count: 1, expected: 0
[ 3286.960443] RCU nest depth: 1, expected: 0
[ 3286.960446] 3 locks held by fio/530910:
[ 3286.960450]  #0: ffff8881108e40b0 (&ctx->uring_lock){+.+.}-{3:3}, at: __do_sys_io_uring_enter+0x535/0x980
[ 3286.960476]  #1: ffffffff9b72a320 (rcu_read_lock){....}-{1:2}, at: nvme_ns_chr_uring_cmd_iopoll+0x5/0x270 [nvme_core]
[ 3286.960530]  #2: ffff88837937b098 (&nvmeq->cq_poll_lock){+.+.}-{2:2}, at: nvme_poll+0x129/0x180 [nvme]
[ 3286.960553] Preemption disabled at:
[ 3286.960555] [<0000000000000000>] 0x0
[ 3286.960691] CPU: 1 PID: 530910 Comm: fio Kdump: loaded Tainted: G        W    L X  -------  ---  5.14.0-345.el9.x86_64+debug #1
[ 3286.960700] Hardware name: Dell Inc. PowerEdge R640/06DKY5, BIOS 2.15.1 06/15/2022
[ 3286.960704] Call Trace:
[ 3286.960707]  <TASK>
[ 3286.960720]  dump_stack_lvl+0x57/0x81
[ 3286.960734]  __might_resched.cold+0x222/0x26b
[ 3286.960756]  set_page_dirty_lock+0x1d/0x130
[ 3286.960773]  __bio_release_pages+0x266/0x470
[ 3286.960811]  blk_rq_unmap_user+0x2a8/0x660
[ 3286.960824]  ? lock_acquire+0x1d8/0x640
[ 3286.960839]  ? sched_clock_cpu+0x15/0x1b0
[ 3286.960850]  ? find_held_lock+0x33/0x120
[ 3286.960870]  ? __pfx_blk_rq_unmap_user+0x10/0x10
[ 3286.960876]  ? __lock_release+0x4c1/0xa00
[ 3286.960894]  ? __pfx___lock_release+0x10/0x10
[ 3286.960908]  ? mark_held_locks+0xa5/0xf0
[ 3286.960938]  nvme_uring_cmd_end_io+0x204/0x300 [nvme_core]
[ 3286.960974]  ? __pfx_nvme_uring_cmd_end_io+0x10/0x10 [nvme_core]
[ 3286.961020]  __blk_mq_end_request+0xf6/0x4c0
[ 3286.961042]  nvme_poll_cq+0x71e/0xe40 [nvme]
[ 3286.961102]  nvme_poll+0x134/0x180 [nvme]
[ 3286.961121]  blk_mq_poll_classic+0x179/0x420
[ 3286.961153]  bio_poll+0x1f5/0x440
[ 3286.961182]  nvme_ns_chr_uring_cmd_iopoll+0x16f/0x270 [nvme_core]

Thanks,
Ming

#3 lore raw
Quoted 13 lines · Ming Lei
Quoted 12 lines · Kanchan Joshi
Quoted 11 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Fixes 585079b6e425 ("nvme: wire up async polling for io passthrough commands")

Thanks Ming. Looks fine, but any link to this report?
I don't see this breaking in my tests. So I wonder how to reproduce and improve the coverage.

It is reported in RH BZ2227639, and follows the stack trace:

Tried to access, but no luck.
Any chance that steps can be posted here?

author
#4 lore raw
Quoted 17 lines · Kanchan Joshi
Quoted 16 lines · Ming Lei
Quoted 15 lines · Kanchan Joshi
Quoted 13 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Fixes 585079b6e425 ("nvme: wire up async polling for io passthrough commands")
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Thanks Ming. Looks fine, but any link to this report?
I don't see this breaking in my tests. So I wonder how to reproduce and improve the coverage.

It is reported in RH BZ2227639, and follows the stack trace:

Tried to access, but no luck.
Any chance that steps can be posted here?

It is reported by Guang Wu, and I think it can be triggered by:

1) enable CONFIG_DEBUG_ATOMIC_SLEEP

2) run some nvme pt read workload, and fio should be fine, but don't pass --fixedbufs

Just run a quick trace on set_page_dirty_lock() in non-debug kernel, which is really called from bio_poll()<-nvme_ns_chr_uring_cmd_iopoll().

Thanks,
Ming

Tested-by
#5 lore raw

Hi Ming,
Don't find any IO block after apply your patch.

Quoted 41 lines · Ming Lei <[email protected]> 于2023年8月10日周四 16:14写道:
Quoted 17 lines · Kanchan Joshi
Quoted 16 lines · Ming Lei
Quoted 15 lines · Kanchan Joshi
Quoted 13 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Fixes 585079b6e425 ("nvme: wire up async polling for io passthrough commands")
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Thanks Ming. Looks fine, but any link to this report?
I don't see this breaking in my tests. So I wonder how to reproduce and improve the coverage.

It is reported in RH BZ2227639, and follows the stack trace:

Tried to access, but no luck.
Any chance that steps can be posted here?

It is reported by Guang Wu, and I think it can be triggered by:

1) enable CONFIG_DEBUG_ATOMIC_SLEEP

2) run some nvme pt read workload, and fio should be fine, but don't pass --fixedbufs

Just run a quick trace on set_page_dirty_lock() in non-debug kernel, which is really called from bio_poll()<-nvme_ns_chr_uring_cmd_iopoll().

Thanks,
Ming

Signature · 2 lines
Guangwu Zhang
Thanks
maintainer Reviewed-by on v1 1/1
#6 lore raw
In reply to Ming Lei
Quoted 10 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

Fix "BUG: sleeping function called from invalid context" because set_page_dirty_lock() from blk_rq_unmap_user() may sleep.

Keith, I'll just apply this directly so it can make this weeks pull request.

Signature · 1 line
Jens Axboe
maintainer Applied
#7 lore raw
In reply to Ming Lei
Quoted 9 lines · Ming Lei

Now nvme_ns_chr_uring_cmd_iopoll() has switched to request based io polling, and the associated NS is guaranteed to be live in case of io polling, so request is guaranteed to be valid because blk-mq uses pre-allocated request pool.

Remove the rcu read lock in nvme_ns_chr_uring_cmd_iopoll(), which isn't needed any more after switching to request based io polling.

[...]

Applied, thanks!

[1/1] nvme: core: don't hold rcu read lock in nvme_ns_chr_uring_cmd_iopoll
(no commit info)

Best regards,

Signature · 1 line
Jens Axboe