Skip to contentSkip to conversation

possible deadlock in touch_wq_lockdep_map

syzbot syzbot 3 participants 6 messages Mar 16, 2024 – Jul 26, 2024

Subsystems (3)
USB Bluetooth drivers Bluetooth
lore
Share & tools
lore

syzbot

author syzbot
#0 lore raw

Hello,

syzbot found the following issue on:

HEAD commit:    9187210eee7d Merge tag 'net-next-6.9' of git://git.kernel...
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=122856fa180000
kernel config:  https://syzkaller.appspot.com/x/.config?x=222448ff79dba2ea
dashboard link: https://syzkaller.appspot.com/bug?extid=91dbdfecdd3287734d8e
compiler:       gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/98c1ea7ddb95/disk-9187210e.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/343c5524f68f/vmlinux-9187210e.xz
kernel image: https://storage.googleapis.com/syzbot-assets/6c0c68b0ee90/bzImage-9187210e.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:

Reported-by [email protected]

Bluetooth: hci0: Opcode 0x0c03 failed: -110
============================================
WARNING: possible recursive locking detected
6.8.0-syzkaller-05202-g9187210eee7d #0 Not tainted
--------------------------------------------
kworker/u9:5/13642 is trying to acquire lock:
ffff88803165d148 ((wq_completion)hci0){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x6e/0x120 kernel/workqueue.c:3901

but task is already holding lock:
ffff88803165d148 ((wq_completion)hci0){+.+.}-{0:0}, at: process_one_work+0x1296/0x1a60 kernel/workqueue.c:3229

other info that might help us debug this:

 Possible unsafe locking scenario:
       CPU0
       ----

lock((wq_completion)hci0);
lock((wq_completion)hci0);

*** DEADLOCK ***

May be due to missing lock nesting notation

2 locks held by kworker/u9:5/13642:
#0: ffff88803165d148 ((wq_completion)hci0){+.+.}-{0:0}, at: process_one_work+0x1296/0x1a60 kernel/workqueue.c:3229 #1: ffffc90004d47d80 ((work_completion)(&hdev->error_reset)){+.+.}-{0:0}, at: process_one_work+0x906/0x1a60 kernel/workqueue.c:3230

stack backtrace:
CPU: 0 PID: 13642 Comm: kworker/u9:5 Not tainted 6.8.0-syzkaller-05202-g9187210eee7d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Workqueue: hci0 hci_error_reset
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
 check_deadlock kernel/locking/lockdep.c:3062 [inline]
 validate_chain kernel/locking/lockdep.c:3856 [inline]
 __lock_acquire+0x20e6/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x540 kernel/locking/lockdep.c:5719
 touch_wq_lockdep_map+0x78/0x120 kernel/workqueue.c:3901
 __flush_workqueue+0x129/0x1200 kernel/workqueue.c:3943
 drain_workqueue+0x18f/0x3d0 kernel/workqueue.c:4107
 destroy_workqueue+0xc2/0xaa0 kernel/workqueue.c:5673
 hci_release_dev+0x14e/0x660 net/bluetooth/hci_core.c:2808
 bt_host_release+0x6a/0xb0 net/bluetooth/hci_sysfs.c:94
 device_release+0xa1/0x240 drivers/base/core.c:2499
 kobject_cleanup lib/kobject.c:689 [inline]
 kobject_release lib/kobject.c:720 [inline]
 kref_put include/linux/kref.h:65 [inline]
 kobject_put+0x1fa/0x5b0 lib/kobject.c:737
 put_device+0x1f/0x30 drivers/base/core.c:3747
 process_one_work+0x9a9/0x1a60 kernel/workqueue.c:3254
 process_scheduled_works kernel/workqueue.c:3335 [inline]
 worker_thread+0x6c8/0xf70 kernel/workqueue.c:3416
 kthread+0x2c1/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
 </TASK>

---
This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at [email protected].

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

author syzbot
#1 lore raw
syzbot has found a reproducer for the following issue on:

syzbot has found a reproducer for the following issue on:

HEAD commit:    734610514cb0 Merge tag 'erofs-for-6.10-rc7-fixes' of git:/..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=151ea512980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=de2d4dc103148cd6
dashboard link: https://syzkaller.appspot.com/bug?extid=91dbdfecdd3287734d8e
compiler:       gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1424d281980000

Downloadable assets:
disk image (non-bootable): https://storage.googleapis.com/syzbot-assets/7bc7510fe41f/non_bootable_disk-73461051.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/d390d5c7156f/vmlinux-73461051.xz
kernel image: https://storage.googleapis.com/syzbot-assets/b56ef48ffa7e/bzImage-73461051.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:

Reported-by [email protected]

Bluetooth: hci0: hardware error 0x20
Bluetooth: hci0: Opcode 0x0c03 failed: -110
============================================
WARNING: possible recursive locking detected
6.10.0-rc6-syzkaller-00055-g734610514cb0 #0 Not tainted
--------------------------------------------
kworker/u33:1/4633 is trying to acquire lock:
ffff88802b9cc148 ((wq_completion)hci0){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x6e/0x120 kernel/workqueue.c:3895

but task is already holding lock:
ffff88802b9cc148 ((wq_completion)hci0){+.+.}-{0:0}, at: process_one_work+0x1277/0x1b40 kernel/workqueue.c:3223

other info that might help us debug this:

 Possible unsafe locking scenario:
       CPU0
       ----

lock((wq_completion)hci0);
lock((wq_completion)hci0);

*** DEADLOCK ***

May be due to missing lock nesting notation

2 locks held by kworker/u33:1/4633:
#0: ffff88802b9cc148 ((wq_completion)hci0){+.+.}-{0:0}, at: process_one_work+0x1277/0x1b40 kernel/workqueue.c:3223 #1: ffffc90025177d80 ((work_completion)(&hdev->error_reset)){+.+.}-{0:0}, at: process_one_work+0x921/0x1b40 kernel/workqueue.c:3224

stack backtrace:
CPU: 2 PID: 4633 Comm: kworker/u33:1 Not tainted 6.10.0-rc6-syzkaller-00055-g734610514cb0 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Workqueue: hci0 hci_error_reset
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:114
 check_deadlock kernel/locking/lockdep.c:3062 [inline]
 validate_chain kernel/locking/lockdep.c:3856 [inline]
 __lock_acquire+0x20e6/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
 touch_wq_lockdep_map+0x78/0x120 kernel/workqueue.c:3895
 __flush_workqueue+0x129/0x1200 kernel/workqueue.c:3937
 drain_workqueue+0x18f/0x3d0 kernel/workqueue.c:4101
 destroy_workqueue+0xc2/0xaa0 kernel/workqueue.c:5817
 hci_release_dev+0x14e/0x660 net/bluetooth/hci_core.c:2795
 bt_host_release+0x6a/0xb0 net/bluetooth/hci_sysfs.c:94
 device_release+0xa1/0x240 drivers/base/core.c:2581
 kobject_cleanup lib/kobject.c:689 [inline]
 kobject_release lib/kobject.c:720 [inline]
 kref_put include/linux/kref.h:65 [inline]
 kobject_put+0x1fa/0x5b0 lib/kobject.c:737
 put_device+0x1f/0x30 drivers/base/core.c:3787
 process_one_work+0x9c5/0x1b40 kernel/workqueue.c:3248
 process_scheduled_works kernel/workqueue.c:3329 [inline]
 worker_thread+0x6c8/0xf30 kernel/workqueue.c:3409
 kthread+0x2c1/0x3a0 kernel/kthread.c:389
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

#2 lore raw

On Tue, 02 Jul 2024 18:47:23 -0700

Quoted 5 lines

syzbot has found a reproducer for the following issue on:

HEAD commit:    734610514cb0 Merge tag 'erofs-for-6.10-rc7-fixes' of git:/..
git tree:       upstream
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1424d281980000

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 734610514cb0

net/bluetooth/hci_core.c +5 -1
@@ -1075,6 +1075,7 @@ static void hci_power_off(struct work_st
 static void hci_error_reset(struct work_struct *work)
 {
 	struct hci_dev *hdev = container_of(work, struct hci_dev, error_reset);
+	bool put;
 
 	hci_dev_hold(hdev);
 	BT_DBG("%s", hdev->name);

@@ -1084,10 +1085,13 @@ static void hci_error_reset(struct work_
 	else
 		bt_dev_err(hdev, "hardware error 0x%2.2x", hdev->hw_error_code);
 
+	put = !test_bit(HCI_UP, &hdev->flags);
+
 	if (!hci_dev_do_close(hdev))
 		hci_dev_do_open(hdev);
 
-	hci_dev_put(hdev);
+	if (put)
+		hci_dev_put(hdev);
 }
 
 void hci_uuids_clear(struct hci_dev *hdev)

--

author syzbot
#3 lore raw
syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by [email protected]

Tested on:

commit:         73461051 Merge tag 'erofs-for-6.10-rc7-fixes' of git:/..
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=1627f269980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=de2d4dc103148cd6
dashboard link: https://syzkaller.appspot.com/bug?extid=91dbdfecdd3287734d8e
compiler:       gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
patch:          https://syzkaller.appspot.com/x/patch.diff?x=1323109e980000

Note: testing is done by a robot and is best-effort only.

author syzbot
#4 lore raw
In reply to syzbot
syzbot has found a reproducer for the following issue on:

syzbot has found a reproducer for the following issue on:

HEAD commit:    933069701c1b Merge tag '6.11-rc-smb3-server-fixes' of git:..
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing
console output: https://syzkaller.appspot.com/x/log.txt?x=10fba1f1980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=f828342678294017
dashboard link: https://syzkaller.appspot.com/bug?extid=91dbdfecdd3287734d8e
compiler:       gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14f45af1980000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14e8b645980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/22dd51445d03/disk-93306970.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/f85f111961d5/vmlinux-93306970.xz
kernel image: https://storage.googleapis.com/syzbot-assets/7971b4814e87/bzImage-93306970.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:

Reported-by [email protected]

============================================
WARNING: possible recursive locking detected
6.10.0-syzkaller-g933069701c1b #0 Not tainted
--------------------------------------------
kworker/1:1H/1247 is trying to acquire lock:
ffff888121075948 ((wq_completion)xillyusb){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x6e/0x120 kernel/workqueue.c:3876

but task is already holding lock:
ffff888121075948 ((wq_completion)xillyusb){+.+.}-{0:0}, at: process_one_work+0x1277/0x1b40 kernel/workqueue.c:3206

other info that might help us debug this:

 Possible unsafe locking scenario:
       CPU0
       ----

lock((wq_completion)xillyusb);
lock((wq_completion)xillyusb);

*** DEADLOCK ***

May be due to missing lock nesting notation

2 locks held by kworker/1:1H/1247:
#0: ffff888121075948 ((wq_completion)xillyusb){+.+.}-{0:0}, at: process_one_work+0x1277/0x1b40 kernel/workqueue.c:3206 #1: ffffc900023afd80 ((work_completion)(&xdev->wakeup_workitem)){+.+.}-{0:0}, at: process_one_work+0x921/0x1b40 kernel/workqueue.c:3207

stack backtrace:
CPU: 1 UID: 0 PID: 1247 Comm: kworker/1:1H Not tainted 6.10.0-syzkaller-g933069701c1b #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
Workqueue: xillyusb wakeup_all
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:93 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:119
 check_deadlock kernel/locking/lockdep.c:3061 [inline]
 validate_chain kernel/locking/lockdep.c:3855 [inline]
 __lock_acquire+0x2167/0x3cb0 kernel/locking/lockdep.c:5142
 lock_acquire kernel/locking/lockdep.c:5759 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5724
 touch_wq_lockdep_map+0x78/0x120 kernel/workqueue.c:3876
 __flush_workqueue+0x129/0x1200 kernel/workqueue.c:3918
 drain_workqueue+0x18f/0x3d0 kernel/workqueue.c:4082
 destroy_workqueue+0xc2/0xaa0 kernel/workqueue.c:5781
 cleanup_dev+0xc5/0x150 drivers/char/xillybus/xillyusb.c:558
 kref_put include/linux/kref.h:65 [inline]
 wakeup_all+0x28c/0x300 drivers/char/xillybus/xillyusb.c:612
 process_one_work+0x9c5/0x1b40 kernel/workqueue.c:3231
 process_scheduled_works kernel/workqueue.c:3312 [inline]
 worker_thread+0x6c8/0xf20 kernel/workqueue.c:3390
 kthread+0x2c1/0x3a0 kernel/kthread.c:389
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

#5 lore raw
Quoted 8 lines · syzbot

WARNING: possible recursive locking detected
6.10.0-syzkaller-g933069701c1b #0 Not tainted
--------------------------------------------
kworker/1:1H/1247 is trying to acquire lock:
ffff888121075948 ((wq_completion)xillyusb){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x6e/0x120 kernel/workqueue.c:3876

but task is already holding lock:
ffff888121075948 ((wq_completion)xillyusb){+.+.}-{0:0}, at: process_one_work+0x1277/0x1b40 kernel/workqueue.c:3206

This is caused by xillyusb.c: destroy_workqueue() is (potentially) called from within a work item function, wakeup_all(). So the work item may attempt to destroy the work queue it sits on.

This bug has sneaked through the hardware tests as well as XillyUSB's users for several years, as its scenario is extremely unlikely in real life. It's nevertheless a bug to be fixed -- I'll submit a patch for this.

Thanks,
Eli