Skip to contentSkip to conversation

BUG: using __this_cpu_read() in preemptible code observed with blktests nvme-tcp on rt kernel

DISCUSSION Yi Zhang 3 participants 4 messages Mar 14, 2021 – Mar 15, 2021 1 message not in the archive NVMe

lore
Share & tools
lore

Yi Zhang

author
#1 lore raw
In reply to a message not in the archive

Hi

I reproduced this issue on latest rt kernel, could you help check it, thanks.

[   76.812567] run blktests nvme/003 at 2021-03-14 07:29:39
[   76.880530] loop: module loaded
[   76.893827] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[   76.895766] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[   76.898923] nvmet: creating controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:f1d7a9f1-79ef-42a3-952b-59c7338f3b54.
[   76.899061] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 127.0.0.1:4420
[   76.899063] BUG: using __this_cpu_read() in preemptible [00000000] code: kworker/u64:25/231
[   76.899065] caller is nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899069] CPU: 3 PID: 231 Comm: kworker/u64:25 Tainted: G S        I       5.12.0-rc2-rt1 #4
[   76.899071] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
[   76.899072] Workqueue: nvme-wq nvme_async_event_work [nvme_core]
[   76.899081] Call Trace:
[   76.899083]  dump_stack+0x64/0x7c
[   76.899086]  check_preemption_disabled+0xb6/0xd0
[   76.899090]  nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899093]  nvme_async_event_work+0x5d/0xc0 [nvme_core]
[   76.899098]  process_one_work+0x1c8/0x3f0
[   76.899101]  ? process_one_work+0x3f0/0x3f0
[   76.899102]  worker_thread+0x30/0x370
[   76.899104]  ? process_one_work+0x3f0/0x3f0
[   76.899105]  kthread+0x183/0x1a0
[   76.899108]  ? kthread_park+0x80/0x80
[   76.899110]  ret_from_fork+0x1f/0x30
[   86.912113] nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"

Best Regards,
Yi Zhang

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#2 lore raw
Quoted 21 lines

Hi

I reproduced this issue on latest rt kernel, could you help check it, thanks.

[   76.812567] run blktests nvme/003 at 2021-03-14 07:29:39
[   76.880530] loop: module loaded
[   76.893827] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[   76.895766] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[   76.898923] nvmet: creating controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:f1d7a9f1-79ef-42a3-952b-59c7338f3b54.
[   76.899061] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 127.0.0.1:4420
[   76.899063] BUG: using __this_cpu_read() in preemptible [00000000] code: kworker/u64:25/231
[   76.899065] caller is nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899069] CPU: 3 PID: 231 Comm: kworker/u64:25 Tainted: G S        I       5.12.0-rc2-rt1 #4
[   76.899071] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
[   76.899072] Workqueue: nvme-wq nvme_async_event_work [nvme_core]
[   76.899081] Call Trace:
[   76.899083]  dump_stack+0x64/0x7c
[   76.899086]  check_preemption_disabled+0xb6/0xd0
[   76.899090]  nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899093]  nvme_async_event_work+0x5d/0xc0 [nvme_core]
Quoted 8 lines
[   76.899098]  process_one_work+0x1c8/0x3f0
[   76.899101]  ? process_one_work+0x3f0/0x3f0
[   76.899102]  worker_thread+0x30/0x370
[   76.899104]  ? process_one_work+0x3f0/0x3f0
[   76.899105]  kthread+0x183/0x1a0
[   76.899108]  ? kthread_park+0x80/0x80
[   76.899110]  ret_from_fork+0x1f/0x30
[   86.912113] nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"

Annoying... why can't we just get an advisory hint of the running cpu without disabling preemption ?!

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#3 lore raw
Quoted 30 lines · Sagi Grimberg
Quoted 21 lines

Hi

I reproduced this issue on latest rt kernel, could you help check it, thanks.

[   76.812567] run blktests nvme/003 at 2021-03-14 07:29:39
[   76.880530] loop: module loaded
[   76.893827] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[   76.895766] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[   76.898923] nvmet: creating controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:f1d7a9f1-79ef-42a3-952b-59c7338f3b54.
[   76.899061] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 127.0.0.1:4420
[   76.899063] BUG: using __this_cpu_read() in preemptible [00000000] code: kworker/u64:25/231
[   76.899065] caller is nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899069] CPU: 3 PID: 231 Comm: kworker/u64:25 Tainted: G S        I       5.12.0-rc2-rt1 #4
[   76.899071] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
[   76.899072] Workqueue: nvme-wq nvme_async_event_work [nvme_core]
[   76.899081] Call Trace:
[   76.899083]  dump_stack+0x64/0x7c
[   76.899086]  check_preemption_disabled+0xb6/0xd0
[   76.899090]  nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899093]  nvme_async_event_work+0x5d/0xc0 [nvme_core]
Quoted 8 lines
[   76.899098]  process_one_work+0x1c8/0x3f0
[   76.899101]  ? process_one_work+0x3f0/0x3f0
[   76.899102]  worker_thread+0x30/0x370
[   76.899104]  ? process_one_work+0x3f0/0x3f0
[   76.899105]  kthread+0x183/0x1a0
[   76.899108]  ? kthread_park+0x80/0x80
[   76.899110]  ret_from_fork+0x1f/0x30
[   86.912113] nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
Quoted 2 lines

Annoying... why can't we just get an advisory hint of the running cpu without disabling preemption ?!

You'll have to use raw_smp_processor_id() for that.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#4 lore raw
Quoted 30 lines
Quoted 30 lines
Quoted 29 lines

Hi

I reproduced this issue on latest rt kernel, could you help check it, thanks.

[   76.812567] run blktests nvme/003 at 2021-03-14 07:29:39
[   76.880530] loop: module loaded
[   76.893827] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[   76.895766] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[   76.898923] nvmet: creating controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:f1d7a9f1-79ef-42a3-952b-59c7338f3b54.
[   76.899061] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 127.0.0.1:4420
[   76.899063] BUG: using __this_cpu_read() in preemptible [00000000] code: kworker/u64:25/231
[   76.899065] caller is nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899069] CPU: 3 PID: 231 Comm: kworker/u64:25 Tainted: G S        I       5.12.0-rc2-rt1 #4
[   76.899071] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
[   76.899072] Workqueue: nvme-wq nvme_async_event_work [nvme_core]
[   76.899081] Call Trace:
[   76.899083]  dump_stack+0x64/0x7c
[   76.899086]  check_preemption_disabled+0xb6/0xd0
[   76.899090]  nvme_tcp_submit_async_event+0x128/0x170 [nvme_tcp]
[   76.899093]  nvme_async_event_work+0x5d/0xc0 [nvme_core]
[   76.899098]  process_one_work+0x1c8/0x3f0
[   76.899101]  ? process_one_work+0x3f0/0x3f0
[   76.899102]  worker_thread+0x30/0x370
[   76.899104]  ? process_one_work+0x3f0/0x3f0
[   76.899105]  kthread+0x183/0x1a0
[   76.899108]  ? kthread_park+0x80/0x80
[   76.899110]  ret_from_fork+0x1f/0x30
[   86.912113] nvme nvme0: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery"
Quoted 4 lines
Quoted 2 lines

Annoying... why can't we just get an advisory hint of the running cpu without disabling preemption ?!

You'll have to use raw_smp_processor_id() for that.

:)

I'll do that then...

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme