RCU stall in blk_mq_timeout_work (potentially a regression in 6.19.7 or 6.19.8)
Subsystems (2)
Dear Linux kernel developers,
I experience repeated RCU stalls in blk_mq_timeout_work causing system freezes. The NVMe drive (more info at the end) shows no errors or controller resets in the kernel log: the hardware appears healthy. No I/O timeout messages precede the stalls.
The stall cascades: khugepaged blocks on __lru_add_drain_all waiting for a workqueue flush that cannot complete, and additional kworkers block on a mutex held by the kblockd rescuer thread. The system becomes noticeably unresponsive and then I notice it in the logs.
In case the logs are too condensed, please tell me and I can provide them fully. If you have pointers on how this could be reproduced and a commit range,
I could try to bisect the problem.
I've assembled the following table by grepping my journalctl -t kernel logs over several boots and letting Claude Opus 4.6 analyze it, to detect when the problems could have started.
Analysis of journalctl across all boots since March 09 shows:
Kernel Boots blk_mq_timeout_work stalls
6.18.13 (Debian) 2 0 6.18.13-bisect (self-built) 10 0 6.18.14 (Debian) 2 0 6.18.15 (Debian) 7 0 6.19.6 (Debian) 3 0 6.19.8 (Debian) 5 8 stalls across 2 boots 7.0.0-rc4 (self-built) 5 1 stall on 1 boot
7.0.0-rc5 (self-built) 5 14 stalls across 2 boots
Zero stalls on any 6.18.x kernel (21 boots total) and on 6.19.6 (3 boots). Stalls begin with 6.19.8. The bug is intermittent, not every boot triggers it, but when it does, stalls come in clusters.
== Trace from 6.19.8+deb14-amd64 (March 26) ==
Mar 26 16:25:16 thinkpad-benjamin kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Mar 26 16:25:16 thinkpad-benjamin kernel: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-15): P80
Mar 26 16:25:16 thinkpad-benjamin kernel: rcu: (detected by 5, t=5252 jiffies, g=1882657, q=10144 ncpus=16)
Mar 26 16:25:16 thinkpad-benjamin kernel: task:kworker/5:0H state:R running task stack:0 pid:80 tgid:80 ppid:2 task_flags:0x4208060 flags:0x00080010
Mar 26 16:25:16 thinkpad-benjamin kernel: Workqueue: kblockd blk_mq_timeout_work
Mar 26 16:25:16 thinkpad-benjamin kernel: Call Trace:
Mar 26 16:25:16 thinkpad-benjamin kernel: <IRQ>
Mar 26 16:25:16 thinkpad-benjamin kernel: sched_show_task+0x172/0x1c0
Mar 26 16:25:16 thinkpad-benjamin kernel: rcu_sched_clock_irq.cold+0x4b8/0x5d7
Mar 26 16:25:16 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Mar 26 16:25:16 thinkpad-benjamin kernel: ? __pfx_tick_nohz_handler+0x10/0x10
Mar 26 16:25:16 thinkpad-benjamin kernel: update_process_times+0x70/0xc0
Mar 26 16:25:16 thinkpad-benjamin kernel: tick_nohz_handler+0x8f/0x180
Mar 26 16:25:16 thinkpad-benjamin kernel: __hrtimer_run_queues+0x10b/0x240
Mar 26 16:25:16 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Mar 26 16:25:16 thinkpad-benjamin kernel: hrtimer_interrupt+0xfc/0x230
Mar 26 16:25:16 thinkpad-benjamin kernel: __sysvec_apic_timer_interrupt+0x58/0x100
Mar 26 16:25:16 thinkpad-benjamin kernel: ? __irq_exit_rcu+0x3d/0xe0
Mar 26 16:25:16 thinkpad-benjamin kernel: sysvec_apic_timer_interrupt+0x6c/0x90
Mar 26 16:25:16 thinkpad-benjamin kernel: </IRQ>
Mar 26 16:25:16 thinkpad-benjamin kernel: <TASK>
Mar 26 16:25:16 thinkpad-benjamin kernel: asm_sysvec_apic_timer_interrupt+0x1a/0x20
Mar 26 16:25:16 thinkpad-benjamin kernel: RIP: 0010:finish_task_switch.isra.0+0x9b/0x2c0
Mar 26 16:25:16 thinkpad-benjamin kernel: __schedule+0x492/0xfc0
Mar 26 16:25:16 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60
Mar 26 16:25:16 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40
Mar 26 16:25:16 thinkpad-benjamin kernel: RIP: 0010:blk_mq_timeout_work+0x194/0x1c0
Mar 26 16:25:16 thinkpad-benjamin kernel: process_one_work+0x192/0x350
Mar 26 16:25:16 thinkpad-benjamin kernel: worker_thread+0x196/0x300
Mar 26 16:25:16 thinkpad-benjamin kernel: kthread+0xfc/0x240
Mar 26 16:25:16 thinkpad-benjamin kernel: ret_from_fork+0x24d/0x290
Mar 26 16:25:16 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30
Mar 26 16:25:16 thinkpad-benjamin kernel: </TASK>
Mar 26 16:25:31 thinkpad-benjamin kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P80 } 5334 jiffies s: 4617 root: 0x0/TCascading hung tasks:
Mar 26 16:26:14 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 120 seconds. khugepaged -> __lru_add_drain_all -> __flush_work -> wait_for_completion
== Trace from 7.0.0-rc5 (self-built, March 27) ==
Mar 27 12:38:23 thinkpad-benjamin kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Mar 27 12:38:23 thinkpad-benjamin kernel: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-15): P32704/1:b..l Mar 27 12:38:23 thinkpad-benjamin kernel: rcu: (detected by 5, t=5252 jiffies, g=464621, q=10641 ncpus=16) Mar 27 12:38:23 thinkpad-benjamin kernel: task:kworker/5:2H state:R running task stack:0 pid:32704 tgid:32704 ppid:2 task_flags:0x4208060 flags:0x00080000 Mar 27 12:38:23 thinkpad-benjamin kernel: Workqueue: kblockd blk_mq_timeout_work Mar 27 12:38:23 thinkpad-benjamin kernel: Call Trace: Mar 27 12:38:23 thinkpad-benjamin kernel: <TASK> Mar 27 12:38:23 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Mar 27 12:38:23 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60 Mar 27 12:38:23 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40 Mar 27 12:38:23 thinkpad-benjamin kernel: RIP: 0010:blk_mq_timeout_work+0x4c/0x1c0 Mar 27 12:38:23 thinkpad-benjamin kernel: ? blk_mq_timeout_work+0x45/0x1c0 Mar 27 12:38:23 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Mar 27 12:38:23 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Mar 27 12:38:23 thinkpad-benjamin kernel: kthread+0xe3/0x120 Mar 27 12:38:23 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Mar 27 12:38:23 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Mar 27 12:38:23 thinkpad-benjamin kernel: </TASK>
Cascading hung tasks 3 minutes later:
Mar 27 12:41:33 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 120 seconds. khugepaged -> __lru_add_drain_all -> __flush_work -> wait_for_completion
Mar 27 12:41:33 thinkpad-benjamin kernel: INFO: task kworker/2:0:28992 blocked for more than 120 seconds. kworker/2:0 -> worker_attach_to_pool -> __mutex_lock (blocked on mutex held by kworker/R-kbloc:139)
Mar 27 12:41:33 thinkpad-benjamin kernel: INFO: task kworker/R-kbloc:139 is the mutex owner: rescuer_thread -> worker_attach_to_pool -> set_cpus_allowed_ptr -> affine_move_task -> wake_up_var
The stall recurred 6 times on this boot:
12:38, 12:47, 13:55, 13:59, 14:10, 14:12
== NVMe device info ==
No NVMe errors in dmesg, hardware appears healthy:
nvme 0000:03:00.0: platform quirk: setting simple suspend nvme nvme0: pci function 0000:03:00.0
nvme nvme0: 16/0/0 default/read/poll queues
PCI: 03:00.0 Non-Volatile memory controller: SK hynix Platinum P41/PC801 [1c5c:1959]
Kind regards
Benjamin
[CCing the regressions list and Jens, the Block layer maintainer, who might have an idea what's wrong here and/or how to best debug it]
Quoted 7 lines · Benjamin Eikel
Dear Linux kernel developers,
I experience repeated RCU stalls in
blk_mq_timeout_workcausing system freezes. The NVMe drive (more info at the end) shows no errors or controller resets in the kernel log: the hardware appears healthy. No I/O timeout messages precede the stalls.
Quoted 8 lines
The stall cascades: khugepaged blocks on
__lru_add_drain_allwaiting for a workqueue flush that cannot complete, and additional kworkers block on a mutex held by the kblockd rescuer thread. The system becomes noticeably unresponsive and then I notice it in the logs.In case the logs are too condensed, please tell me and I can provide them fully. If you have pointers on how this could be reproduced and a commit range,
I could try to bisect the problem.
A bisection with mainline would help a lot. Maybe doing it between 6.19.6 and 6.19.8 (which would be quicker) might work out, too (but normally it's best to focus on mainline).
And this is not my area of expertise, but I noticed that some RCU fixes went into -rc6, so might be wise to try that first.
Ciao, Thorsten
Quoted 110 lines
I've assembled the following table by grepping my
journalctl -t kernellogs over several boots and letting Claude Opus 4.6 analyze it, to detect when the problems could have started.Analysis of journalctl across all boots since March 09 shows:
Kernel Boots
blk_mq_timeout_workstalls6.18.13 (Debian) 2 0 6.18.13-bisect (self-built) 10 0 6.18.14 (Debian) 2 0 6.18.15 (Debian) 7 0 6.19.6 (Debian) 3 0 6.19.8 (Debian) 5 8 stalls across 2 boots 7.0.0-rc4 (self-built) 5 1 stall on 1 boot7.0.0-rc5 (self-built) 5 14 stalls across 2 boots
Zero stalls on any 6.18.x kernel (21 boots total) and on 6.19.6 (3 boots). Stalls begin with 6.19.8. The bug is intermittent, not every boot triggers it, but when it does, stalls come in clusters.
== Trace from 6.19.8+deb14-amd64 (March 26) ==
Mar 26 16:25:16 thinkpad-benjamin kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Mar 26 16:25:16 thinkpad-benjamin kernel: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-15): P80 Mar 26 16:25:16 thinkpad-benjamin kernel: rcu: (detected by 5, t=5252 jiffies, g=1882657, q=10144 ncpus=16) Mar 26 16:25:16 thinkpad-benjamin kernel: task:kworker/5:0H state:R running task stack:0 pid:80 tgid:80 ppid:2 task_flags:0x4208060 flags:0x00080010 Mar 26 16:25:16 thinkpad-benjamin kernel: Workqueue: kblockd blk_mq_timeout_work Mar 26 16:25:16 thinkpad-benjamin kernel: Call Trace: Mar 26 16:25:16 thinkpad-benjamin kernel: <IRQ> Mar 26 16:25:16 thinkpad-benjamin kernel: sched_show_task+0x172/0x1c0 Mar 26 16:25:16 thinkpad-benjamin kernel: rcu_sched_clock_irq.cold+0x4b8/0x5d7 Mar 26 16:25:16 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Mar 26 16:25:16 thinkpad-benjamin kernel: ? __pfx_tick_nohz_handler+0x10/0x10 Mar 26 16:25:16 thinkpad-benjamin kernel: update_process_times+0x70/0xc0 Mar 26 16:25:16 thinkpad-benjamin kernel: tick_nohz_handler+0x8f/0x180 Mar 26 16:25:16 thinkpad-benjamin kernel: __hrtimer_run_queues+0x10b/0x240 Mar 26 16:25:16 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Mar 26 16:25:16 thinkpad-benjamin kernel: hrtimer_interrupt+0xfc/0x230 Mar 26 16:25:16 thinkpad-benjamin kernel: __sysvec_apic_timer_interrupt+0x58/0x100 Mar 26 16:25:16 thinkpad-benjamin kernel: ? __irq_exit_rcu+0x3d/0xe0 Mar 26 16:25:16 thinkpad-benjamin kernel: sysvec_apic_timer_interrupt+0x6c/0x90 Mar 26 16:25:16 thinkpad-benjamin kernel: </IRQ> Mar 26 16:25:16 thinkpad-benjamin kernel: <TASK> Mar 26 16:25:16 thinkpad-benjamin kernel: asm_sysvec_apic_timer_interrupt+0x1a/0x20 Mar 26 16:25:16 thinkpad-benjamin kernel: RIP: 0010:finish_task_switch.isra.0+0x9b/0x2c0 Mar 26 16:25:16 thinkpad-benjamin kernel: __schedule+0x492/0xfc0 Mar 26 16:25:16 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60 Mar 26 16:25:16 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40 Mar 26 16:25:16 thinkpad-benjamin kernel: RIP: 0010:blk_mq_timeout_work+0x194/0x1c0 Mar 26 16:25:16 thinkpad-benjamin kernel: process_one_work+0x192/0x350 Mar 26 16:25:16 thinkpad-benjamin kernel: worker_thread+0x196/0x300 Mar 26 16:25:16 thinkpad-benjamin kernel: kthread+0xfc/0x240 Mar 26 16:25:16 thinkpad-benjamin kernel: ret_from_fork+0x24d/0x290 Mar 26 16:25:16 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Mar 26 16:25:16 thinkpad-benjamin kernel: </TASK> Mar 26 16:25:31 thinkpad-benjamin kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P80 } 5334 jiffies s: 4617 root: 0x0/TCascading hung tasks:
Mar 26 16:26:14 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 120 seconds. khugepaged -> __lru_add_drain_all -> __flush_work -> wait_for_completion== Trace from 7.0.0-rc5 (self-built, March 27) ==
Mar 27 12:38:23 thinkpad-benjamin kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Mar 27 12:38:23 thinkpad-benjamin kernel: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-15): P32704/1:b..l Mar 27 12:38:23 thinkpad-benjamin kernel: rcu: (detected by 5, t=5252 jiffies, g=464621, q=10641 ncpus=16) Mar 27 12:38:23 thinkpad-benjamin kernel: task:kworker/5:2H state:R running task stack:0 pid:32704 tgid:32704 ppid:2 task_flags:0x4208060 flags:0x00080000 Mar 27 12:38:23 thinkpad-benjamin kernel: Workqueue: kblockd blk_mq_timeout_work Mar 27 12:38:23 thinkpad-benjamin kernel: Call Trace: Mar 27 12:38:23 thinkpad-benjamin kernel: <TASK> Mar 27 12:38:23 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Mar 27 12:38:23 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60 Mar 27 12:38:23 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40 Mar 27 12:38:23 thinkpad-benjamin kernel: RIP: 0010:blk_mq_timeout_work+0x4c/0x1c0 Mar 27 12:38:23 thinkpad-benjamin kernel: ? blk_mq_timeout_work+0x45/0x1c0 Mar 27 12:38:23 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Mar 27 12:38:23 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Mar 27 12:38:23 thinkpad-benjamin kernel: kthread+0xe3/0x120 Mar 27 12:38:23 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Mar 27 12:38:23 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Mar 27 12:38:23 thinkpad-benjamin kernel: </TASK>Cascading hung tasks 3 minutes later:
Mar 27 12:41:33 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 120 seconds. khugepaged -> __lru_add_drain_all -> __flush_work -> wait_for_completionMar 27 12:41:33 thinkpad-benjamin kernel: INFO: task kworker/2:0:28992 blocked for more than 120 seconds. kworker/2:0 -> worker_attach_to_pool -> __mutex_lock (blocked on mutex held by kworker/R-kbloc:139)Mar 27 12:41:33 thinkpad-benjamin kernel: INFO: task kworker/R-kbloc:139 is the mutex owner: rescuer_thread -> worker_attach_to_pool -> set_cpus_allowed_ptr -> affine_move_task -> wake_up_varThe stall recurred 6 times on this boot:
12:38, 12:47, 13:55, 13:59, 14:10, 14:12== NVMe device info ==
No NVMe errors in dmesg, hardware appears healthy:
nvme 0000:03:00.0: platform quirk: setting simple suspend nvme nvme0: pci function 0000:03:00.0
nvme nvme0: 16/0/0 default/read/poll queuesPCI: 03:00.0 Non-Volatile memory controller: SK hynix Platinum P41/PC801 [1c5c:1959]
Kind regards
Benjamin
Hi Thorsten and others,
Am Dienstag, 31. März 2026, 10:32:30 Mitteleuropäische Sommerzeit schrieb Thorsten Leemhuis:
Quoted 16 lines
[CCing the regressions list and Jens, the Block layer maintainer, who might have an idea what's wrong here and/or how to best debug it]
Quoted 12 lines · Benjamin Eikel
Dear Linux kernel developers,
I experience repeated RCU stalls in
blk_mq_timeout_workcausing system freezes. The NVMe drive (more info at the end) shows no errors or controller resets in the kernel log: the hardware appears healthy. No I/O timeout messages precede the stalls.The stall cascades: khugepaged blocks on
__lru_add_drain_allwaiting for a workqueue flush that cannot complete, and additional kworkers block on a mutex held by the kblockd rescuer thread. The system becomes noticeably unresponsive and then I notice it in the logs.
Quoted 8 lines
Quoted 4 lines
In case the logs are too condensed, please tell me and I can provide them fully. If you have pointers on how this could be reproduced and a commit range,>
I could try to bisect the problem.A bisection with mainline would help a lot. Maybe doing it between 6.19.6 and 6.19.8 (which would be quicker) might work out, too (but normally it's best to focus on mainline).
I have not yet done this because I do not have a reliable way to reproduce it.
Quoted 3 lines
And this is not my area of expertise, but I noticed that some RCU fixes went into -rc6, so might be wise to try that first.
I encountered this issue with -rc6 (see log below, same config). I suspect that this only happens after the notebook has previously been suspended.
Kind regards
Benjamin
== Trace from Linux version 7.0.0-rc6 (1–2 April) ==
Apr 01 20:36:59 thinkpad-benjamin kernel: Linux version 7.0.0-rc6-70-rc6 (benjamin@thinkpad-benjamin) (gcc (Debian 15.2.0-16) 15.2.0, GNU ld (GNU Binutils for Debian) 2.46) #24 SMP PREEMPT_DYNAMIC Tue Mar 31 14:36:43 CEST 2026 Apr 01 20:36:59 thinkpad-benjamin kernel: Command line: BOOT_IMAGE=/vmlinuz-7.0.0-rc6-70-rc6 root=/dev/mapper/thinkpad--benjamin--vg-root ro quiet […] Apr 01 21:06:54 thinkpad-benjamin kernel: PM: suspend entry (s2idle) […] Apr 02 08:44:55 thinkpad-benjamin kernel: PM: suspend exit […] Apr 02 08:50:59 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 120 seconds. Apr 02 08:50:59 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:50:59 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:50:59 thinkpad-benjamin kernel: task:khugepaged state:D stack:0 pid:138 tgid:138 ppid:2 task_flags:0x200040 flags:0x00080000 Apr 02 08:50:59 thinkpad-benjamin kernel: Call Trace: Apr 02 08:50:59 thinkpad-benjamin kernel: <TASK> Apr 02 08:50:59 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:50:59 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:50:59 thinkpad-benjamin kernel: schedule_timeout+0xbd/0x100 Apr 02 08:50:59 thinkpad-benjamin kernel: wait_for_completion+0x8a/0x140 Apr 02 08:50:59 thinkpad-benjamin kernel: __flush_work+0x276/0x3b0 Apr 02 08:50:59 thinkpad-benjamin kernel: ? __pfx_wq_barrier_func+0x10/0x10 Apr 02 08:50:59 thinkpad-benjamin kernel: __lru_add_drain_all+0x198/0x2b0 Apr 02 08:50:59 thinkpad-benjamin kernel: khugepaged+0x65/0x950 Apr 02 08:50:59 thinkpad-benjamin kernel: ? __pfx_autoremove_wake_function+0x10/0x10 Apr 02 08:50:59 thinkpad-benjamin kernel: ? __pfx_khugepaged+0x10/0x10 Apr 02 08:50:59 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:50:59 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:50:59 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:50:59 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:50:59 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:50:59 thinkpad-benjamin kernel: </TASK> Apr 02 08:53:00 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 241 seconds. Apr 02 08:53:00 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:53:00 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:53:00 thinkpad-benjamin kernel: task:khugepaged state:D stack:0 pid:138 tgid:138 ppid:2 task_flags:0x200040 flags:0x00080000 Apr 02 08:53:00 thinkpad-benjamin kernel: Call Trace: Apr 02 08:53:00 thinkpad-benjamin kernel: <TASK> Apr 02 08:53:00 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:53:00 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:53:00 thinkpad-benjamin kernel: schedule_timeout+0xbd/0x100 Apr 02 08:53:00 thinkpad-benjamin kernel: wait_for_completion+0x8a/0x140 Apr 02 08:53:00 thinkpad-benjamin kernel: __flush_work+0x276/0x3b0 Apr 02 08:53:00 thinkpad-benjamin kernel: ? __pfx_wq_barrier_func+0x10/0x10 Apr 02 08:53:00 thinkpad-benjamin kernel: __lru_add_drain_all+0x198/0x2b0 Apr 02 08:53:00 thinkpad-benjamin kernel: khugepaged+0x65/0x950 Apr 02 08:53:00 thinkpad-benjamin kernel: ? __pfx_autoremove_wake_function+0x10/0x10 Apr 02 08:53:00 thinkpad-benjamin kernel: ? __pfx_khugepaged+0x10/0x10 Apr 02 08:53:00 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:53:00 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:53:00 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:53:00 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:53:00 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:53:00 thinkpad-benjamin kernel: </TASK> Apr 02 08:55:00 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 362 seconds. Apr 02 08:55:00 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:55:00 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:55:00 thinkpad-benjamin kernel: task:khugepaged state:D stack:0 pid:138 tgid:138 ppid:2 task_flags:0x200040 flags:0x00080000 Apr 02 08:55:00 thinkpad-benjamin kernel: Call Trace: Apr 02 08:55:00 thinkpad-benjamin kernel: <TASK> Apr 02 08:55:00 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:55:00 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:55:00 thinkpad-benjamin kernel: schedule_timeout+0xbd/0x100 Apr 02 08:55:00 thinkpad-benjamin kernel: wait_for_completion+0x8a/0x140 Apr 02 08:55:00 thinkpad-benjamin kernel: __flush_work+0x276/0x3b0 Apr 02 08:55:00 thinkpad-benjamin kernel: ? __pfx_wq_barrier_func+0x10/0x10 Apr 02 08:55:00 thinkpad-benjamin kernel: __lru_add_drain_all+0x198/0x2b0 Apr 02 08:55:00 thinkpad-benjamin kernel: khugepaged+0x65/0x950 Apr 02 08:55:00 thinkpad-benjamin kernel: ? __pfx_autoremove_wake_function+0x10/0x10 Apr 02 08:55:00 thinkpad-benjamin kernel: ? __pfx_khugepaged+0x10/0x10 Apr 02 08:55:00 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:55:00 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:55:00 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:55:00 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:55:00 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:55:00 thinkpad-benjamin kernel: </TASK> Apr 02 08:57:01 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 483 seconds. Apr 02 08:57:01 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:57:01 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:57:01 thinkpad-benjamin kernel: task:khugepaged state:D stack:0 pid:138 tgid:138 ppid:2 task_flags:0x200040 flags:0x00080000 Apr 02 08:57:01 thinkpad-benjamin kernel: Call Trace: Apr 02 08:57:01 thinkpad-benjamin kernel: <TASK> Apr 02 08:57:01 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:57:01 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:57:01 thinkpad-benjamin kernel: schedule_timeout+0xbd/0x100 Apr 02 08:57:01 thinkpad-benjamin kernel: wait_for_completion+0x8a/0x140 Apr 02 08:57:01 thinkpad-benjamin kernel: __flush_work+0x276/0x3b0 Apr 02 08:57:01 thinkpad-benjamin kernel: ? __pfx_wq_barrier_func+0x10/0x10 Apr 02 08:57:01 thinkpad-benjamin kernel: __lru_add_drain_all+0x198/0x2b0 Apr 02 08:57:01 thinkpad-benjamin kernel: khugepaged+0x65/0x950 Apr 02 08:57:01 thinkpad-benjamin kernel: ? __pfx_autoremove_wake_function+0x10/0x10 Apr 02 08:57:01 thinkpad-benjamin kernel: ? __pfx_khugepaged+0x10/0x10 Apr 02 08:57:01 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:57:01 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:57:01 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:57:01 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:57:01 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:57:01 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:14:126 blocked for more than 120 seconds. Apr 02 08:59:02 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:59:02 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/u64:14 state:D stack:0 pid:126 tgid:126 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Workqueue: events_unbound idle_cull_fn Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule_preempt_disabled+0x15/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: __mutex_lock.constprop.0+0x54b/0xac0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? ttwu_queue_wakelist+0x13c/0x230 Apr 02 08:59:02 thinkpad-benjamin kernel: idle_cull_fn+0x3e/0x170 Apr 02 08:59:02 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_worker_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:14:126 is blocked on a mutex likely owned by task kworker/R-kbloc:139. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/R-kbloc state:R running task stack:0 pid:139 tgid:139 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40 Apr 02 08:59:02 thinkpad-benjamin kernel: RIP: 0010:affine_move_task+0x198/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: Code: 00 49 8b 36 48 8d bb ec 0d 00 00 e8 22 8a d9 00 45 85 ff 44 8b 04 24 0f 84 21 02 00 00 65 ff 0d 4e 73 8b 02 0f 84 6f 03 00 00 <41> 83 e0 04 49 8d 7c 24 08 75 3e e8 18 09 d9 00 b8 ff ff ff ff f0 Apr 02 08:59:02 thinkpad-benjamin kernel: RSP: 0018:ffffcc5f4063fcd8 EFLAGS: 00000246 Apr 02 08:59:02 thinkpad-benjamin kernel: RAX: 0000000000000001 RBX: ffff8af641ab2180 RCX: 0000000000000004 Apr 02 08:59:02 thinkpad-benjamin kernel: RDX: 0000000000000000 RSI: 0000000000000283 RDI: ffff8af640c08dec Apr 02 08:59:02 thinkpad-benjamin kernel: RBP: ffff8b04a1eb2100 R08: 0000000000000000 R09: 0000000000000283 Apr 02 08:59:02 thinkpad-benjamin kernel: R10: 0000000000000002 R11: 0000000000000000 R12: ffffcc5f4063fd28 Apr 02 08:59:02 thinkpad-benjamin kernel: R13: ffffcc5f4063fd28 R14: ffffcc5f4063fde0 R15: 0000000000000000 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x50c/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_next_entity+0xe7/0x270 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_migration_cpu_stop+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x3c8/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: __set_cpus_allowed_ptr+0x66/0xc0 Apr 02 08:59:02 thinkpad-benjamin kernel: set_cpus_allowed_ptr+0x3a/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_attach_to_pool+0x5f/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: rescuer_thread+0x12d/0x5d0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_rescuer_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task khugepaged:138 blocked for more than 604 seconds. Apr 02 08:59:02 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:59:02 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:59:02 thinkpad-benjamin kernel: task:khugepaged state:D stack:0 pid:138 tgid:138 ppid:2 task_flags:0x200040 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule_timeout+0xbd/0x100 Apr 02 08:59:02 thinkpad-benjamin kernel: wait_for_completion+0x8a/0x140 Apr 02 08:59:02 thinkpad-benjamin kernel: __flush_work+0x276/0x3b0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_wq_barrier_func+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: __lru_add_drain_all+0x198/0x2b0 Apr 02 08:59:02 thinkpad-benjamin kernel: khugepaged+0x65/0x950 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_autoremove_wake_function+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_khugepaged+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:16:5369 blocked for more than 120 seconds. Apr 02 08:59:02 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:59:02 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/u64:16 state:D stack:0 pid:5369 tgid:5369 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Workqueue: events_unbound idle_cull_fn Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule_preempt_disabled+0x15/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: __mutex_lock.constprop.0+0x54b/0xac0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? finish_task_switch.isra.0+0x96/0x2c0 Apr 02 08:59:02 thinkpad-benjamin kernel: idle_cull_fn+0x3e/0x170 Apr 02 08:59:02 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_worker_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:16:5369 is blocked on a mutex likely owned by task kworker/R-kbloc:139. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/R-kbloc state:R running task stack:0 pid:139 tgid:139 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: ? __schedule+0x484/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: ? preempt_schedule_irq+0x38/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: ? asm_common_interrupt+0x26/0x40 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x198/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x50c/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_next_entity+0xe7/0x270 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_migration_cpu_stop+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x3c8/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __set_cpus_allowed_ptr+0x66/0xc0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_cpus_allowed_ptr+0x3a/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: ? worker_attach_to_pool+0x5f/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? rescuer_thread+0x12d/0x5d0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_rescuer_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:12:9088 blocked for more than 120 seconds. Apr 02 08:59:02 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:59:02 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/u64:12 state:D stack:0 pid:9088 tgid:9088 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Workqueue: events_unbound idle_cull_fn Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule_preempt_disabled+0x15/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: __mutex_lock.constprop.0+0x54b/0xac0 Apr 02 08:59:02 thinkpad-benjamin kernel: idle_cull_fn+0x3e/0x170 Apr 02 08:59:02 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_worker_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:12:9088 is blocked on a mutex likely owned by task kworker/R-kbloc:139. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/R-kbloc state:R running task stack:0 pid:139 tgid:139 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40 Apr 02 08:59:02 thinkpad-benjamin kernel: RIP: 0010:affine_move_task+0x198/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: Code: 00 49 8b 36 48 8d bb ec 0d 00 00 e8 22 8a d9 00 45 85 ff 44 8b 04 24 0f 84 21 02 00 00 65 ff 0d 4e 73 8b 02 0f 84 6f 03 00 00 <41> 83 e0 04 49 8d 7c 24 08 75 3e e8 18 09 d9 00 b8 ff ff ff ff f0 Apr 02 08:59:02 thinkpad-benjamin kernel: RSP: 0018:ffffcc5f4063fcd8 EFLAGS: 00000246 Apr 02 08:59:02 thinkpad-benjamin kernel: RAX: 0000000000000001 RBX: ffff8af641ab2180 RCX: 0000000000000004 Apr 02 08:59:02 thinkpad-benjamin kernel: RDX: 0000000000000000 RSI: 0000000000000283 RDI: ffff8af640c08dec Apr 02 08:59:02 thinkpad-benjamin kernel: RBP: ffff8b04a1eb2100 R08: 0000000000000000 R09: 0000000000000283 Apr 02 08:59:02 thinkpad-benjamin kernel: R10: 0000000000000002 R11: 0000000000000000 R12: ffffcc5f4063fd28 Apr 02 08:59:02 thinkpad-benjamin kernel: R13: ffffcc5f4063fd28 R14: ffffcc5f4063fde0 R15: 0000000000000000 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x50c/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_next_entity+0xe7/0x270 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_migration_cpu_stop+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x3c8/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: __set_cpus_allowed_ptr+0x66/0xc0 Apr 02 08:59:02 thinkpad-benjamin kernel: set_cpus_allowed_ptr+0x3a/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_attach_to_pool+0x5f/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: rescuer_thread+0x12d/0x5d0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_rescuer_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:22:10249 blocked for more than 120 seconds. Apr 02 08:59:02 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:59:02 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/u64:22 state:D stack:0 pid:10249 tgid:10249 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Workqueue: events_unbound idle_cull_fn Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule_preempt_disabled+0x15/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: __mutex_lock.constprop.0+0x54b/0xac0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? finish_task_switch.isra.0+0x96/0x2c0 Apr 02 08:59:02 thinkpad-benjamin kernel: idle_cull_fn+0x3e/0x170 Apr 02 08:59:02 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_worker_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:22:10249 is blocked on a mutex likely owned by task kworker/R-kbloc:139. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/R-kbloc state:R running task stack:0 pid:139 tgid:139 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: preempt_schedule_irq+0x38/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: asm_common_interrupt+0x26/0x40 Apr 02 08:59:02 thinkpad-benjamin kernel: RIP: 0010:affine_move_task+0x198/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: Code: 00 49 8b 36 48 8d bb ec 0d 00 00 e8 22 8a d9 00 45 85 ff 44 8b 04 24 0f 84 21 02 00 00 65 ff 0d 4e 73 8b 02 0f 84 6f 03 00 00 <41> 83 e0 04 49 8d 7c 24 08 75 3e e8 18 09 d9 00 b8 ff ff ff ff f0 Apr 02 08:59:02 thinkpad-benjamin kernel: RSP: 0018:ffffcc5f4063fcd8 EFLAGS: 00000246 Apr 02 08:59:02 thinkpad-benjamin kernel: RAX: 0000000000000001 RBX: ffff8af641ab2180 RCX: 0000000000000004 Apr 02 08:59:02 thinkpad-benjamin kernel: RDX: 0000000000000000 RSI: 0000000000000283 RDI: ffff8af640c08dec Apr 02 08:59:02 thinkpad-benjamin kernel: RBP: ffff8b04a1eb2100 R08: 0000000000000000 R09: 0000000000000283 Apr 02 08:59:02 thinkpad-benjamin kernel: R10: 0000000000000002 R11: 0000000000000000 R12: ffffcc5f4063fd28 Apr 02 08:59:02 thinkpad-benjamin kernel: R13: ffffcc5f4063fd28 R14: ffffcc5f4063fde0 R15: 0000000000000000 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x50c/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_next_entity+0xe7/0x270 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_migration_cpu_stop+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x3c8/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: __set_cpus_allowed_ptr+0x66/0xc0 Apr 02 08:59:02 thinkpad-benjamin kernel: set_cpus_allowed_ptr+0x3a/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_attach_to_pool+0x5f/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: rescuer_thread+0x12d/0x5d0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_rescuer_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:31:10258 blocked for more than 120 seconds. Apr 02 08:59:02 thinkpad-benjamin kernel: Not tainted 7.0.0-rc6-70-rc6 #24 Apr 02 08:59:02 thinkpad-benjamin kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/u64:31 state:D stack:0 pid:10258 tgid:10258 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Workqueue: events_unbound idle_cull_fn Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: __schedule+0x47c/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule+0x27/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: schedule_preempt_disabled+0x15/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: __mutex_lock.constprop.0+0x54b/0xac0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? ttwu_queue_wakelist+0x13c/0x230 Apr 02 08:59:02 thinkpad-benjamin kernel: idle_cull_fn+0x3e/0x170 Apr 02 08:59:02 thinkpad-benjamin kernel: process_one_work+0x19d/0x3a0 Apr 02 08:59:02 thinkpad-benjamin kernel: worker_thread+0x1af/0x320 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_worker_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: INFO: task kworker/u64:31:10258 is blocked on a mutex likely owned by task kworker/R-kbloc:139. Apr 02 08:59:02 thinkpad-benjamin kernel: task:kworker/R-kbloc state:R running task stack:0 pid:139 tgid:139 ppid:2 task_flags:0x4208060 flags:0x00080000 Apr 02 08:59:02 thinkpad-benjamin kernel: Call Trace: Apr 02 08:59:02 thinkpad-benjamin kernel: <TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: ? __schedule+0x484/0x1000 Apr 02 08:59:02 thinkpad-benjamin kernel: ? preempt_schedule_irq+0x38/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: ? asm_common_interrupt+0x26/0x40 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x198/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x50c/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? srso_alias_return_thunk+0x5/0xfbef5 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_next_entity+0xe7/0x270 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_migration_cpu_stop+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? affine_move_task+0x3c8/0x580 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __set_cpus_allowed_ptr+0x66/0xc0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? set_cpus_allowed_ptr+0x3a/0x60 Apr 02 08:59:02 thinkpad-benjamin kernel: ? worker_attach_to_pool+0x5f/0xd0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? rescuer_thread+0x12d/0x5d0 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_rescuer_thread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? kthread+0xe3/0x120 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? ret_from_fork+0x2c9/0x360 Apr 02 08:59:02 thinkpad-benjamin kernel: ? __pfx_kthread+0x10/0x10 Apr 02 08:59:02 thinkpad-benjamin kernel: ? ret_from_fork_asm+0x1a/0x30 Apr 02 08:59:02 thinkpad-benjamin kernel: </TASK> Apr 02 08:59:02 thinkpad-benjamin kernel: Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings […] Apr 02 09:05:10 thinkpad-benjamin kernel: rcu_tasks_wait_gp: rcu_tasks grace period number 9 (since boot) is 2583 jiffies old. Apr 02 09:05:40 thinkpad-benjamin kernel: rcu_tasks_wait_gp: rcu_tasks grace period number 9 (since boot) is 10136 jiffies old. […] Apr 02 09:07:11 thinkpad-benjamin kernel: rcu_tasks_wait_gp: rcu_tasks grace period number 9 (since boot) is 32832 jiffies old. […] Apr 02 09:08:09 thinkpad-benjamin kernel: sysrq: Emergency Sync Apr 02 09:08:09 thinkpad-benjamin kernel: Emergency Sync complete Apr 02 09:08:09 thinkpad-benjamin kernel: sysrq: Emergency Remount R/O