possible deadlock in anon_pipe_prefill_and_lock
Subsystems (8)
syzbot
syzbotHello,
syzbot found the following issue on:
HEAD commit: 1fc5a74b108f Merge tag 'kmalloc_obj-v7.3-rc2' of git://git.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=17aeb2f9580000 kernel config: https://syzkaller.appspot.com/x/.config?x=8c5c3949d762a91f dashboard link: https://syzkaller.appspot.com/bug?extid=002fb7b2d0bc53bf7e75 compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/f5473bcef69a/disk-1fc5a74b.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/fc94ec0cede2/vmlinux-1fc5a74b.xz
kernel image: https://storage.googleapis.com/syzbot-assets/1316ecaf7adb/bzImage-1fc5a74b.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Tainted: G L
------------------------------------------------------
syz.3.511/7847 is trying to acquire lock:ffff88802c68cc60 (&pipe->mutex){+.+.}-{4:4}, at: anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165
but task is already holding lock:ffff88807d378138 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_write fs/autofs/waitq.c:55 [inline] ffff88807d378138 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (&sbi->pipe_mutex){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5908 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5865
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
autofs_write fs/autofs/waitq.c:55 [inline]
autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426
autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256
autofs_d_automount+0x490/0x950 fs/autofs/root.c:410
follow_automount fs/namei.c:1565 [inline]
__traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618
traverse_mounts fs/namei.c:1647 [inline]
handle_mounts fs/namei.c:1749 [inline]
step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104
step_into fs/namei.c:2152 [inline]
walk_component fs/namei.c:2288 [inline]
lookup_last fs/namei.c:2789 [inline]
path_lookupat+0x58b/0xc40 fs/namei.c:2813
filename_lookup+0x202/0x590 fs/namei.c:2842
kern_path+0x37/0x50 fs/namei.c:3036
lookup_bdev+0xd8/0x2a0 block/bdev.c:1268
resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278
kobj_attr_store+0x58/0x80 lib/kobject.c:842
sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145
kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345
new_sync_write fs/read_write.c:595 [inline]
vfs_write+0x6af/0x1050 fs/read_write.c:687
ksys_write+0x12a/0x250 fs/read_write.c:739
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #1 (&of->mutex){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5908 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5865
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
kernfs_fop_write_iter+0x2c2/0x5f0 fs/kernfs/file.c:336
iter_file_splice_write+0x830/0x10b0 fs/splice.c:736
do_splice_from fs/splice.c:936 [inline]
do_splice+0x109c/0x1fa0 fs/splice.c:1349
__do_splice+0x33b/0x370 fs/splice.c:1431
__do_sys_splice fs/splice.c:1634 [inline]
__se_sys_splice fs/splice.c:1616 [inline]
__x64_sys_splice+0x187/0x250 fs/splice.c:1616
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #0 (&pipe->mutex){+.+.}-{4:4}:
check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3181
check_prevs_add kernel/locking/lockdep.c:3300 [inline]
validate_chain kernel/locking/lockdep.c:3924 [inline]
__lock_acquire+0x1492/0x1ec0 kernel/locking/lockdep.c:5254
lock_acquire kernel/locking/lockdep.c:5908 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5865
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165
anon_pipe_write+0x164/0x1b50 fs/pipe.c:534
__kernel_write_iter+0x6bb/0x930 fs/read_write.c:621
__kernel_write+0xf6/0x140 fs/read_write.c:641
autofs_write fs/autofs/waitq.c:57 [inline]
autofs_notify_daemon+0x50d/0xd90 fs/autofs/waitq.c:164
autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426
autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256
autofs_d_automount+0x490/0x950 fs/autofs/root.c:410
follow_automount fs/namei.c:1565 [inline]
__traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618
traverse_mounts fs/namei.c:1647 [inline]
handle_mounts fs/namei.c:1749 [inline]
step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104
step_into fs/namei.c:2152 [inline]
walk_component fs/namei.c:2288 [inline]
lookup_last fs/namei.c:2789 [inline]
path_lookupat+0x58b/0xc40 fs/namei.c:2813
filename_lookup+0x202/0x590 fs/namei.c:2842
kern_path+0x37/0x50 fs/namei.c:3036
lookup_bdev+0xd8/0x2a0 block/bdev.c:1268
resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278
kobj_attr_store+0x58/0x80 lib/kobject.c:842
sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145
kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345
new_sync_write fs/read_write.c:595 [inline]
vfs_write+0x6af/0x1050 fs/read_write.c:687
ksys_write+0x12a/0x250 fs/read_write.c:739
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7fother info that might help us debug this:
Chain exists of: &pipe->mutex --> &of->mutex --> &sbi->pipe_mutex Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&sbi->pipe_mutex);
lock(&of->mutex);
lock(&sbi->pipe_mutex); lock(&pipe->mutex);
*** DEADLOCK ***
locks held by syz.3.511/7847: 5, last CPU#0:
#0: ffff88807334f430 (&f->f_pos_lock){+.+.}-{4:4}, at: fdget_pos+0x2aa/0x380 fs/file.c:1259 #1: ffff8880339b4460 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0x12a/0x250 fs/read_write.c:739 #2: ffff888056438c80 (&of->mutex){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x2c2/0x5f0 fs/kernfs/file.c:336 #3: ffff888020ac5f08 (kn->active#71){.+.+}-{0:0}, at: kernfs_get_active_of fs/kernfs/file.c:73 [inline] #3: ffff888020ac5f08 (kn->active#71){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x332/0x5f0 fs/kernfs/file.c:337 #4: ffff88807d378138 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_write fs/autofs/waitq.c:55 [inline] #4: ffff88807d378138 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
stack backtrace: CPU: 0 UID: 0 PID: 7847 Comm: syz.3.511 Tainted: G L syzkaller #0 PREEMPT(full) Tainted: [L]=SOFTLOCKUP Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/05/2026 Call Trace: <TASK> __dump_stack lib/dump_stack.c:94 [inline] dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120 print_circular_bug.cold+0x178/0x1be kernel/locking/lockdep.c:2059 check_noncircular+0x146/0x160 kernel/locking/lockdep.c:2191 check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3181 check_prevs_add kernel/locking/lockdep.c:3300 [inline] validate_chain kernel/locking/lockdep.c:3924 [inline] __lock_acquire+0x1492/0x1ec0 kernel/locking/lockdep.c:5254 lock_acquire kernel/locking/lockdep.c:5908 [inline] lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5865 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821 anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165 anon_pipe_write+0x164/0x1b50 fs/pipe.c:534 __kernel_write_iter+0x6bb/0x930 fs/read_write.c:621 __kernel_write+0xf6/0x140 fs/read_write.c:641 autofs_write fs/autofs/waitq.c:57 [inline] autofs_notify_daemon+0x50d/0xd90 fs/autofs/waitq.c:164 autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426 autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256 autofs_d_automount+0x490/0x950 fs/autofs/root.c:410 follow_automount fs/namei.c:1565 [inline] __traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618 traverse_mounts fs/namei.c:1647 [inline] handle_mounts fs/namei.c:1749 [inline] step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104 step_into fs/namei.c:2152 [inline] walk_component fs/namei.c:2288 [inline] lookup_last fs/namei.c:2789 [inline] path_lookupat+0x58b/0xc40 fs/namei.c:2813 filename_lookup+0x202/0x590 fs/namei.c:2842 kern_path+0x37/0x50 fs/namei.c:3036 lookup_bdev+0xd8/0x2a0 block/bdev.c:1268 resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278 kobj_attr_store+0x58/0x80 lib/kobject.c:842 sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145 kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345 new_sync_write fs/read_write.c:595 [inline] vfs_write+0x6af/0x1050 fs/read_write.c:687 ksys_write+0x12a/0x250 fs/read_write.c:739 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline] do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7eff3b99e159 Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007eff3c78c028 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 00007eff3bc25fa0 RCX: 00007eff3b99e159 RDX: 0000000000000012 RSI: 0000200000000040 RDI: 0000000000000008 RBP: 00007eff3ba35024 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007eff3bc26038 R14: 00007eff3bc25fa0 R15: 00007ffdcf0d0ce8 </TASK>
PM: Image not found (code -6)
---
This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at [email protected].
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
syzbot
syzbotsyzbot has found a reproducer for the following issue on:
syzbot has found a reproducer for the following issue on:
HEAD commit: 4982d3552a3b Merge tag 'sound-7.3-rc4' of git://git.kernel.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=176da915580000 kernel config: https://syzkaller.appspot.com/x/.config?x=84649d3ff8d550cb dashboard link: https://syzkaller.appspot.com/bug?extid=002fb7b2d0bc53bf7e75 compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10eda915580000
Downloadable assets:
disk image (non-bootable): https://storage.googleapis.com/syzbot-assets/d900f083ada3/non_bootable_disk-4982d355.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/d2afaea1fac3/vmlinux-4982d355.xz
kernel image: https://storage.googleapis.com/syzbot-assets/06e82fadea82/bzImage-4982d355.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Not tainted
------------------------------------------------------
syz.0.17/5928 is trying to acquire lock:ffff88802d512c60 (&pipe->mutex){+.+.}-{4:4}, at: anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165
but task is already holding lock:ffff888036200938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_write fs/autofs/waitq.c:55 [inline] ffff888036200938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (&sbi->pipe_mutex){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
autofs_write fs/autofs/waitq.c:55 [inline]
autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426
autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256
autofs_d_automount+0x490/0x950 fs/autofs/root.c:410
follow_automount fs/namei.c:1565 [inline]
__traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618
traverse_mounts fs/namei.c:1647 [inline]
handle_mounts fs/namei.c:1749 [inline]
step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104
step_into fs/namei.c:2152 [inline]
walk_component fs/namei.c:2288 [inline]
lookup_last fs/namei.c:2789 [inline]
path_lookupat+0x58b/0xc40 fs/namei.c:2813
filename_lookup+0x202/0x590 fs/namei.c:2842
kern_path+0x37/0x50 fs/namei.c:3036
lookup_bdev+0xd8/0x2a0 block/bdev.c:1268
resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278
kobj_attr_store+0x58/0x80 lib/kobject.c:842
sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145
kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345
new_sync_write fs/read_write.c:595 [inline]
vfs_write+0x6af/0x1050 fs/read_write.c:687
ksys_write+0x12a/0x250 fs/read_write.c:739
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #2 (&of->mutex){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
kernfs_seq_start+0x4f/0x2a0 fs/kernfs/file.c:165
seq_read_iter+0x2c1/0x1270 fs/seq_file.c:226
kernfs_fop_read_iter+0x46c/0x610 fs/kernfs/file.c:290
copy_splice_read+0x4ba/0xbb0 fs/splice.c:362
do_splice_read fs/splice.c:980 [inline]
do_splice_read+0x285/0x370 fs/splice.c:954
splice_file_to_pipe+0x82/0x120 fs/splice.c:1290
do_sendfile+0x366/0xe20 fs/read_write.c:1377
__do_sys_sendfile64 fs/read_write.c:1432 [inline]
__se_sys_sendfile64 fs/read_write.c:1418 [inline]
__x64_sys_sendfile64+0x1d8/0x220 fs/read_write.c:1418
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #1 (&p->lock){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
seq_read_iter+0xe1/0x1270 fs/seq_file.c:183
kernfs_fop_read_iter+0x46c/0x610 fs/kernfs/file.c:290
copy_splice_read+0x4ba/0xbb0 fs/splice.c:362
do_splice_read fs/splice.c:980 [inline]
do_splice_read+0x285/0x370 fs/splice.c:954
splice_file_to_pipe+0x82/0x120 fs/splice.c:1290
do_sendfile+0x366/0xe20 fs/read_write.c:1377
__do_sys_sendfile64 fs/read_write.c:1432 [inline]
__se_sys_sendfile64 fs/read_write.c:1418 [inline]
__x64_sys_sendfile64+0x1d8/0x220 fs/read_write.c:1418
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #0 (&pipe->mutex){+.+.}-{4:4}:
check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3209
check_prevs_add kernel/locking/lockdep.c:3328 [inline]
validate_chain kernel/locking/lockdep.c:3952 [inline]
__lock_acquire+0x1528/0x1f40 kernel/locking/lockdep.c:5288
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165
anon_pipe_write+0x164/0x1b50 fs/pipe.c:534
__kernel_write_iter+0x6bb/0x930 fs/read_write.c:621
__kernel_write+0xf6/0x140 fs/read_write.c:641
autofs_write fs/autofs/waitq.c:57 [inline]
autofs_notify_daemon+0x50d/0xd90 fs/autofs/waitq.c:164
autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426
autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256
autofs_d_automount+0x490/0x950 fs/autofs/root.c:410
follow_automount fs/namei.c:1565 [inline]
__traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618
traverse_mounts fs/namei.c:1647 [inline]
handle_mounts fs/namei.c:1749 [inline]
step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104
step_into fs/namei.c:2152 [inline]
walk_component fs/namei.c:2288 [inline]
lookup_last fs/namei.c:2789 [inline]
path_lookupat+0x58b/0xc40 fs/namei.c:2813
filename_lookup+0x202/0x590 fs/namei.c:2842
kern_path+0x37/0x50 fs/namei.c:3036
lookup_bdev+0xd8/0x2a0 block/bdev.c:1268
resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278
kobj_attr_store+0x58/0x80 lib/kobject.c:842
sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145
kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345
new_sync_write fs/read_write.c:595 [inline]
vfs_write+0x6af/0x1050 fs/read_write.c:687
ksys_write+0x12a/0x250 fs/read_write.c:739
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7fother info that might help us debug this:
Chain exists of: &pipe->mutex --> &of->mutex --> &sbi->pipe_mutex Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&sbi->pipe_mutex);
lock(&of->mutex);
lock(&sbi->pipe_mutex); lock(&pipe->mutex);
*** DEADLOCK ***
locks held by syz.0.17/5928: 4, last CPU#3:
#0: ffff88802c2fc460 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0x12a/0x250 fs/read_write.c:739 #1: ffff88802be9e080 (&of->mutex){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x2c2/0x5f0 fs/kernfs/file.c:336 #2: ffff88801d344878 (kn->active#67){.+.+}-{0:0}, at: kernfs_get_active_of fs/kernfs/file.c:73 [inline] #2: ffff88801d344878 (kn->active#67){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x332/0x5f0 fs/kernfs/file.c:337 #3: ffff888036200938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_write fs/autofs/waitq.c:55 [inline] #3: ffff888036200938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
stack backtrace: CPU: 3 UID: 0 PID: 5928 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 Call Trace: <TASK> __dump_stack lib/dump_stack.c:94 [inline] dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120 print_circular_bug.cold+0x178/0x1be kernel/locking/lockdep.c:2087 check_noncircular+0x146/0x160 kernel/locking/lockdep.c:2219 check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3209 check_prevs_add kernel/locking/lockdep.c:3328 [inline] validate_chain kernel/locking/lockdep.c:3952 [inline] __lock_acquire+0x1528/0x1f40 kernel/locking/lockdep.c:5288 lock_acquire kernel/locking/lockdep.c:5942 [inline] lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821 anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165 anon_pipe_write+0x164/0x1b50 fs/pipe.c:534 __kernel_write_iter+0x6bb/0x930 fs/read_write.c:621 __kernel_write+0xf6/0x140 fs/read_write.c:641 autofs_write fs/autofs/waitq.c:57 [inline] autofs_notify_daemon+0x50d/0xd90 fs/autofs/waitq.c:164 autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426 autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256 autofs_d_automount+0x490/0x950 fs/autofs/root.c:410 follow_automount fs/namei.c:1565 [inline] __traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618 traverse_mounts fs/namei.c:1647 [inline] handle_mounts fs/namei.c:1749 [inline] step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104 step_into fs/namei.c:2152 [inline] walk_component fs/namei.c:2288 [inline] lookup_last fs/namei.c:2789 [inline] path_lookupat+0x58b/0xc40 fs/namei.c:2813 filename_lookup+0x202/0x590 fs/namei.c:2842 kern_path+0x37/0x50 fs/namei.c:3036 lookup_bdev+0xd8/0x2a0 block/bdev.c:1268 resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278 kobj_attr_store+0x58/0x80 lib/kobject.c:842 sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145 kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345 new_sync_write fs/read_write.c:595 [inline] vfs_write+0x6af/0x1050 fs/read_write.c:687 ksys_write+0x12a/0x250 fs/read_write.c:739 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline] do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7f00df39e159 Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007ffeff202ac8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 00007f00df625fa0 RCX: 00007f00df39e159 RDX: 0000000000000012 RSI: 0000200000000040 RDI: 0000000000000008 RBP: 00007f00df43506b R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007f00df625fac R14: 00007f00df625fa0 R15: 00007f00df625fa0 </TASK>
PM: Image not found (code -6)
---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
syzbot
syzbotsyzbot has found a reproducer for the following issue on:
syzbot has found a reproducer for the following issue on:
HEAD commit: fe2ec83746e5 Merge tag 'hid-for-linus-2026092202' of git:/.. git tree: upstream console+strace: https://syzkaller.appspot.com/x/log.txt?x=16b78d25580000 kernel config: https://syzkaller.appspot.com/x/.config?x=7d012d9c67977ee4 dashboard link: https://syzkaller.appspot.com/bug?extid=002fb7b2d0bc53bf7e75 compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14854405580000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=11c0f4c9580000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/cd25f1c33d2a/disk-fe2ec837.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/2c35e3694ae6/vmlinux-fe2ec837.xz
kernel image: https://storage.googleapis.com/syzbot-assets/1ff8b2a62fde/bzImage-fe2ec837.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Not tainted
------------------------------------------------------
syz-executor105/5614 is trying to acquire lock:ffff888037550c60 (&pipe->mutex){+.+.}-{4:4}, at: anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165
but task is already holding lock:ffff8880247ac938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_write fs/autofs/waitq.c:55 [inline] ffff8880247ac938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (&sbi->pipe_mutex){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
autofs_write fs/autofs/waitq.c:55 [inline]
autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426
autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256
autofs_d_automount+0x490/0x950 fs/autofs/root.c:410
follow_automount fs/namei.c:1565 [inline]
__traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618
traverse_mounts fs/namei.c:1647 [inline]
handle_mounts fs/namei.c:1749 [inline]
step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104
step_into fs/namei.c:2152 [inline]
walk_component fs/namei.c:2288 [inline]
lookup_last fs/namei.c:2789 [inline]
path_lookupat+0x58b/0xc40 fs/namei.c:2813
filename_lookup+0x202/0x590 fs/namei.c:2842
kern_path+0x37/0x50 fs/namei.c:3036
lookup_bdev+0xd8/0x2a0 block/bdev.c:1268
resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278
kobj_attr_store+0x58/0x80 lib/kobject.c:842
sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145
kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345
new_sync_write fs/read_write.c:595 [inline]
vfs_write+0x6af/0x1050 fs/read_write.c:687
ksys_write+0x12a/0x250 fs/read_write.c:739
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #1 (&of->mutex){+.+.}-{4:4}:
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
kernfs_fop_write_iter+0x2c2/0x5f0 fs/kernfs/file.c:336
iter_file_splice_write+0x830/0x10b0 fs/splice.c:736
do_splice_from fs/splice.c:936 [inline]
do_splice+0x109c/0x1fa0 fs/splice.c:1349
__do_splice+0x33b/0x370 fs/splice.c:1431
__do_sys_splice fs/splice.c:1634 [inline]
__se_sys_splice fs/splice.c:1616 [inline]
__x64_sys_splice+0x187/0x250 fs/splice.c:1616
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f-> #0 (&pipe->mutex){+.+.}-{4:4}:
check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3209
check_prevs_add kernel/locking/lockdep.c:3328 [inline]
validate_chain kernel/locking/lockdep.c:3952 [inline]
__lock_acquire+0x1528/0x1f40 kernel/locking/lockdep.c:5288
lock_acquire kernel/locking/lockdep.c:5942 [inline]
lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821
anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165
anon_pipe_write+0x164/0x1b50 fs/pipe.c:534
__kernel_write_iter+0x6bb/0x930 fs/read_write.c:621
__kernel_write+0xf6/0x140 fs/read_write.c:641
autofs_write fs/autofs/waitq.c:57 [inline]
autofs_notify_daemon+0x50d/0xd90 fs/autofs/waitq.c:164
autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426
autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256
autofs_d_automount+0x490/0x950 fs/autofs/root.c:410
follow_automount fs/namei.c:1565 [inline]
__traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618
traverse_mounts fs/namei.c:1647 [inline]
handle_mounts fs/namei.c:1749 [inline]
step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104
step_into fs/namei.c:2152 [inline]
walk_component fs/namei.c:2288 [inline]
lookup_last fs/namei.c:2789 [inline]
path_lookupat+0x58b/0xc40 fs/namei.c:2813
filename_lookup+0x202/0x590 fs/namei.c:2842
kern_path+0x37/0x50 fs/namei.c:3036
lookup_bdev+0xd8/0x2a0 block/bdev.c:1268
resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278
kobj_attr_store+0x58/0x80 lib/kobject.c:842
sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145
kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345
new_sync_write fs/read_write.c:595 [inline]
vfs_write+0x6af/0x1050 fs/read_write.c:687
ksys_write+0x12a/0x250 fs/read_write.c:739
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7fother info that might help us debug this:
Chain exists of: &pipe->mutex --> &of->mutex --> &sbi->pipe_mutex Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&sbi->pipe_mutex);
lock(&of->mutex);
lock(&sbi->pipe_mutex); lock(&pipe->mutex);
*** DEADLOCK ***
locks held by syz-executor105/5614: 4, last CPU#0:
#0: ffff888037ff0460 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0x12a/0x250 fs/read_write.c:739 #1: ffff8880781ea480 (&of->mutex){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x2c2/0x5f0 fs/kernfs/file.c:336 #2: ffff888020adbe18 (kn->active#57){.+.+}-{0:0}, at: kernfs_get_active_of fs/kernfs/file.c:73 [inline] #2: ffff888020adbe18 (kn->active#57){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x332/0x5f0 fs/kernfs/file.c:337 #3: ffff8880247ac938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_write fs/autofs/waitq.c:55 [inline] #3: ffff8880247ac938 (&sbi->pipe_mutex){+.+.}-{4:4}, at: autofs_notify_daemon+0x4f8/0xd90 fs/autofs/waitq.c:164
stack backtrace: CPU: 0 UID: 0 PID: 5614 Comm: syz-executor105 Not tainted syzkaller #0 PREEMPT(full) Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/05/2026 Call Trace: <TASK> __dump_stack lib/dump_stack.c:94 [inline] dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120 print_circular_bug.cold+0x178/0x1be kernel/locking/lockdep.c:2087 check_noncircular+0x146/0x160 kernel/locking/lockdep.c:2219 check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3209 check_prevs_add kernel/locking/lockdep.c:3328 [inline] validate_chain kernel/locking/lockdep.c:3952 [inline] __lock_acquire+0x1528/0x1f40 kernel/locking/lockdep.c:5288 lock_acquire kernel/locking/lockdep.c:5942 [inline] lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821 anon_pipe_prefill_and_lock+0x16c/0x580 fs/pipe.c:165 anon_pipe_write+0x164/0x1b50 fs/pipe.c:534 __kernel_write_iter+0x6bb/0x930 fs/read_write.c:621 __kernel_write+0xf6/0x140 fs/read_write.c:641 autofs_write fs/autofs/waitq.c:57 [inline] autofs_notify_daemon+0x50d/0xd90 fs/autofs/waitq.c:164 autofs_wait+0x10fd/0x1b50 fs/autofs/waitq.c:426 autofs_mount_wait+0x132/0x3b0 fs/autofs/root.c:256 autofs_d_automount+0x490/0x950 fs/autofs/root.c:410 follow_automount fs/namei.c:1565 [inline] __traverse_mounts+0x1b9/0x8a0 fs/namei.c:1618 traverse_mounts fs/namei.c:1647 [inline] handle_mounts fs/namei.c:1749 [inline] step_into_slowpath+0xb7e/0xf90 fs/namei.c:2104 step_into fs/namei.c:2152 [inline] walk_component fs/namei.c:2288 [inline] lookup_last fs/namei.c:2789 [inline] path_lookupat+0x58b/0xc40 fs/namei.c:2813 filename_lookup+0x202/0x590 fs/namei.c:2842 kern_path+0x37/0x50 fs/namei.c:3036 lookup_bdev+0xd8/0x2a0 block/bdev.c:1268 resume_store+0x1d6/0x460 kernel/power/hibernate.c:1278 kobj_attr_store+0x58/0x80 lib/kobject.c:842 sysfs_kf_write+0xf2/0x150 fs/sysfs/file.c:145 kernfs_fop_write_iter+0x3e0/0x5f0 fs/kernfs/file.c:345 new_sync_write fs/read_write.c:595 [inline] vfs_write+0x6af/0x1050 fs/read_write.c:687 ksys_write+0x12a/0x250 fs/read_write.c:739 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline] do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7f02d5f9ea29 Code: c0 79 93 eb d5 48 8d 7c 1d 00 eb 99 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 d8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fffe4dfde98 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f02d5f9ea29 RDX: 0000000000000012 RSI: 0000200000000040 RDI: 000000000000000b RBP: 0000000000000000 R08: 0000000000000009 R09: 0000000000000000 R10: 0000000000000007 R11: 0000000000000246 R12: 00007f02d6006042 R13: 00007fffe4dfded0 R14: 00007fffe4dfdf10 R15: 0000000000000000 </TASK>
PM: Image not found (code -6)
---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
cover kernfs, autofs: fix pipe lock cycles through sysfs stores
In reply to syzbotsyzbot reports a lockdep cycle &pipe->mutex --> &of->mutex --> &sbi->pipe_mutex. splice holds a pipe mutex while it calls the write or read handler of a kernfs file, kernfs takes of->mutex there, and a write to /sys/power/resume looks up a path that can make autofs write a request into its daemon pipe and wait for the daemon.
Patch 1 stops holding of->mutex around the ->store() callback of regular sysfs attributes, which do not get the open file and already run concurrently for separate opens.
Patch 2 handles the case that patch 1 leaves. A store called from splice still runs with the pipe locked. autofs no longer sends or waits for a request in that context and returns -EAGAIN instead. A new mount is refused before it is marked pending, so no other lookup sends the request in its place. This also removes a deadlock in which the spliced pipe is the autofs daemon pipe itself.
Patch 2 changes behavior for one narrow case. A lookup from a splice callback that needs an automount fails with -EAGAIN instead of waiting.
The series was tested in QEMU with lockdep using the original C and syz reproducers and focused splice and autofs cases. The target reports were not reproduced with the series applied, and the paired functional cases matched the unpatched baseline.
Krystian Kaniewski (2):
kernfs: do not hold of->mutex across sysfs ->store()
autofs: do not wait for the daemon while splice holds a pipe
fs/autofs/expire.c | 6 +++++ fs/autofs/root.c | 28 +++++++++++++++++++++-- fs/kernfs/file.c | 52 +++++++++++++++++++++++++++--------------- fs/splice.c | 13 ++++++++++- fs/sysfs/file.c | 18 ++++++++++++++- include/linux/kernfs.h | 11 +++++++++ include/linux/sched.h | 2 ++ kernel/fork.c | 1 + 8 files changed, 109 insertions(+), 22 deletions(-)
base-commit: 93f51579e7df248780214094418f205253383cc5
Signature · 1 line
2.53.0
PATCH 1/2 kernfs: do not hold of->mutex across sysfs ->store()
Lockdep reports a circular dependency &pipe->mutex --> &of->mutex --> &sbi->pipe_mutex. A splice between a pipe and a kernfs file calls ->write_iter() or ->read_iter() of the file with the pipe locked, and kernfs takes of->mutex there. A write to /sys/power/resume runs resume_store() under of->mutex, and its lookup_bdev() can trigger an autofs mount request, whose autofs_write() takes sbi->pipe_mutex and then the mutex of the daemon pipe.
of->mutex only has to keep the callbacks of one open file apart. sysfs_kf_write() does not pass the open file to ->store(), and two opens of the same attribute already run ->store() concurrently. Add a write_unlocked flag to struct kernfs_ops, cache it in struct kernfs_open_file next to atomic_write_len, and take of->mutex around ->write() only when the flag is clear. Set it for the regular sysfs attributes. The preallocated buffer stays protected by of->prealloc_mutex, and the other kernfs users keep of->mutex.
This does not remove every report through the autofs daemon pipe. A pipe spliced straight into such an attribute stays locked while ->store() looks up the path, and a ->store() that takes its own lock around a lookup keeps that dependency.
--- fs/kernfs/file.c | 52 +++++++++++++++++++++++++++--------------- fs/sysfs/file.c | 18 ++++++++++++++- include/linux/kernfs.h | 11 +++++++++ 3 files changed, 62 insertions(+), 19 deletions(-)
fs/kernfs/file.c +34 -18
@@ -291,6 +291,26 @@ static ssize_t kernfs_fop_read_iter(struct kiocb *iocb, struct iov_iter *iter)return kernfs_file_read_iter (iocb, iter); } +/* Call the ->write() operation of @of under an active reference. */ +static ssize_t kernfs_write_active (struct kernfs_open_file *of,char *buf, +size_t len,loff_t pos) +{ +const struct kernfs_ops *ops; +ssize_t ret; + +if (!kernfs_get_active_of (of)) +return -ENODEV ; + + ops =kernfs_ops (of->kn); +if (ops->write) + ret = ops->write (of, buf, len, pos); +else + ret = -EINVAL ; + +kernfs_put_active_of (of); +return ret; +} +/* * Copy data in from userland and pass it to the matching kernfs write * operation.
@@ -305,7 +325,6 @@ static ssize_t kernfs_fop_write_iter(struct kiocb *iocb, struct iov_iter *iter) {struct kernfs_open_file *of =kernfs_of (iocb->ki_filp);ssize_t len =iov_iter_count (iter); -const struct kernfs_ops *ops;char *buf;if (of->atomic_write_len) {
@@ -330,25 +349,18 @@ static ssize_t kernfs_fop_write_iter(struct kiocb *iocb, struct iov_iter *iter) buf[len] ='\0' ;/* guarantee string termination */ /* -* @of->mutex nests outside active ref and is used both to ensure that -* the ops aren't called concurrently for the same open file. +* @of->mutex nests outside active ref and keeps the ops of one open +* file from running concurrently. ->write() of files with +* ->write_unlocked set does not need that and runs without it. */ -mutex_lock (&of->mutex); -if (!kernfs_get_active_of (of)) { +if (of->write_unlocked) { + len =kernfs_write_active (of, buf, len, iocb->ki_pos); + }else { +mutex_lock (&of->mutex); + len =kernfs_write_active (of, buf, len, iocb->ki_pos);mutex_unlock (&of->mutex); - len = -ENODEV ; - goto out_free; } - ops =kernfs_ops (of->kn); -if (ops->write) - len = ops->write (of, buf, len, iocb->ki_pos); -else - len = -EINVAL ; - -kernfs_put_active_of (of); -mutex_unlock (&of->mutex); -if (len >0 ) iocb->ki_pos += len;
@@ -651,6 +663,8 @@ static int kernfs_fop_open(struct inode *inode, struct file *file)* lockdep key, because the writable file /sys/power/resume may call vfs * lookup helpers for arbitrary paths and readonly files can be read by * overlayfs from vfs helpers when sysfs is a lower layer of overalyfs. +* (Note that regular sysfs attributes no longer run ->store() under +* @of->mutex, but other kernfs users still do.) * * All three cases look the same. They're supposed to * look that way and give @of->mutex different static lockdep keys.
@@ -666,10 +680,12 @@ static int kernfs_fop_open(struct inode *inode, struct file *file) of->file = file;/* -* Write path needs to atomic_write_len outside active reference. -* Cache it in open_file. See kernfs_fop_write_iter() for details. +* Write path needs atomic_write_len and write_unlocked outside +* active reference. Cache them in open_file. See +* kernfs_fop_write_iter() for details. */ of->atomic_write_len = ops->atomic_write_len; + of->write_unlocked = ops->write_unlocked; error = -EINVAL ;/*
fs/sysfs/file.c +17 -1
@@ -132,7 +132,19 @@ static ssize_t sysfs_kf_read(struct kernfs_open_file *of, char *buf,return min_t (ssize_t, count, len); } -/* kernfs write callback for regular sysfs files */ +/* +* kernfs write callback for regular sysfs files. +* +* Regular sysfs files set .write_unlocked = true and are called without +* holding of->mutex. Sysfs ->store() maintains no per-open-file state, and +* concurrent opens already execute ->store() concurrently without +* serialization. Calling ->store() without of->mutex avoids circular +* locking dependencies when ->store() performs operations like path +* lookups or automounts while a generic VFS lock is held (such as a pipe +* lock during splice). Preallocated attributes still serialize buffer +* access via of->prealloc_mutex. Existing autofs and callback-owned lock +* dependencies remain outside this narrow change. +*/ static ssize_t sysfs_kf_write (struct kernfs_open_file *of,char *buf,size_t count,loff_t pos) {
@@ -229,11 +241,13 @@ static const struct kernfs_ops sysfs_file_kfops_ro = {static const struct kernfs_ops sysfs_file_kfops_wo = { .write = sysfs_kf_write, + .write_unlocked = true, };static const struct kernfs_ops sysfs_file_kfops_rw = { .seq_show = sysfs_kf_seq_show, .write = sysfs_kf_write, + .write_unlocked = true, };static const struct kernfs_ops sysfs_prealloc_kfops_ro = {
@@ -244,12 +258,14 @@ static const struct kernfs_ops sysfs_prealloc_kfops_ro = {static const struct kernfs_ops sysfs_prealloc_kfops_wo = { .write = sysfs_kf_write, .prealloc = true, + .write_unlocked = true, };static const struct kernfs_ops sysfs_prealloc_kfops_rw = { .read = sysfs_kf_read, .write = sysfs_kf_write, .prealloc = true, + .write_unlocked = true, };static const struct kernfs_ops sysfs_bin_kfops_ro = {
include/linux/kernfs.h +11 -0
@@ -272,6 +272,7 @@ struct kernfs_open_file {char *prealloc_buf;size_t atomic_write_len; +bool write_unlocked; bool mmapped:1; bool released:1;const struct vm_operations_struct *vm_ops;
@@ -320,6 +321,16 @@ struct kernfs_ops {* ->prealloc. Provide ->read and ->write with ->prealloc. */ bool prealloc; +/* +* "write_unlocked" indicates that ->write() does not require +* per-open-file serialization with of->mutex and can be called +* concurrently for the same open file. Callbacks must handle +* concurrent execution. Buffer access for preallocated attributes +* is still protected by of->prealloc_mutex. Lock dependencies from +* callers (such as pipe locks during splice) or within callbacks +* remain outside this scope. +*/ +bool write_unlocked;ssize_t (*write)(struct kernfs_open_file *of,char *buf,size_t bytes,loff_t off);
Signature · 1 line
2.53.0
PATCH 2/2 autofs: do not wait for the daemon while splice holds a pipe
In reply to Krystian Kaniewskisplice holds the mutex of a pipe while it calls ->write_iter() of the target file in iter_file_splice_write() or ->splice_read() of the source file in splice_file_to_pipe(). When such a callback looks up a path in an autofs mount, autofs sends a request through its daemon pipe and waits for the daemon. A write to /sys/power/resume that comes from splice() does this, so lockdep still reports &pipe->mutex --> &sbi->pipe_mutex --> &pipe->mutex after of->mutex is gone from the sysfs store path. When the spliced pipe is the daemon pipe itself, the task can wait for the mutex that it already holds, which SIGKILL does not end, or wait for a daemon that cannot read its pipe.
Mark the task while splice runs such a callback with a pipe really locked (pipe->files set, as in pipe_lock()), and make autofs return -EAGAIN in that context instead of sending a mount request or waiting for a pending mount or expire. A new mount is refused before AUTOFS_INF_PENDING is set, because another lookup that sees the flag would send the request instead. The short wait for expire selection does not depend on the daemon and is kept. RCU walks, lookups that need no request and all other callers keep their behavior, and a child does not inherit the mark.
A lookup from such a callback that needs an automount now fails with -EAGAIN instead of waiting for the daemon. This affects splice() from a pipe into a file whose write handler looks up a path under an autofs mount that is not mounted yet, for example /sys/power/resume. sendfile() into a file goes through the private pipe of the task, which is never locked, and keeps its behavior.
--- fs/autofs/expire.c | 6 ++++++ fs/autofs/root.c | 28 ++++++++++++++++++++++++++-- fs/splice.c | 13 ++++++++++++- include/linux/sched.h | 2 ++ kernel/fork.c | 1 + 5 files changed, 47 insertions(+), 3 deletions(-)
fs/autofs/expire.c +6 -0
@@ -514,6 +514,12 @@ int autofs_expire_wait(const struct path *path, int rcu_walk) }if (state &AUTOFS_INF_EXPIRING ) {spin_unlock (&sbi->fs_lock); +/* +* The expire waits for the daemon, which may need a pipe that +* the splice of this task holds. Refuse instead of waiting. +*/ +if (unlikely (current->in_splice_pipe_callback)) +return -EAGAIN ;pr_debug ("waiting for expire %p name=%pd\n" , dentry, dentry);
fs/autofs/root.c +26 -2
@@ -252,6 +252,8 @@ static int autofs_mount_wait(const struct path *path, bool rcu_walk)if (ino->flags &AUTOFS_INF_PENDING ) {if (rcu_walk)return -ECHILD ; +if (unlikely (current->in_splice_pipe_callback)) +return -EAGAIN ;pr_debug ("waiting for mount name=%pd\n" , path->dentry); status =autofs_wait (sbi, path,NFY_MOUNT );pr_debug ("mount wait done status=%d\n" , status);
@@ -263,10 +265,18 @@ static int autofs_mount_wait(const struct path *path, bool rcu_walk)return status; } +static bool autofs_expire_wait_blocked (int status) +{ +/* A marked caller is refused before a daemon can return a status. */ +return status == -EAGAIN && +unlikely (current->in_splice_pipe_callback); +} +static int do_expire_wait (const struct path *path,bool rcu_walk) {struct dentry *dentry = path->dentry;struct dentry *expiring; +int status; expiring =autofs_lookup_expiring (dentry, rcu_walk);if (IS_ERR (expiring))
@@ -280,7 +290,11 @@ static int do_expire_wait(const struct path *path, bool rcu_walk)* be quite complete, but the directory has been removed * so it must have been successful, just wait for it. */ -autofs_expire_wait (&this,0 ); + status =autofs_expire_wait (&this,0 ); +if (autofs_expire_wait_blocked (status)) { +dput (expiring); +return status; + }autofs_del_expiring (expiring);dput (expiring); }
@@ -359,6 +373,8 @@ static struct vfsmount *autofs_d_automount(struct path *path)* the mount. */ status =do_expire_wait (path,0 ); +if (autofs_expire_wait_blocked (status)) +return ERR_PTR (-EAGAIN );if (status && status != -EAGAIN )return NULL ;
@@ -405,6 +421,11 @@ static struct vfsmount *autofs_d_automount(struct path *path) goto done; } } +/* Do not publish PENDING without issuing its mount request. */ +if (unlikely (current->in_splice_pipe_callback)) { +spin_unlock (&sbi->fs_lock); +return ERR_PTR (-EAGAIN ); + } ino->flags |=AUTOFS_INF_PENDING ;spin_unlock (&sbi->fs_lock); status =autofs_mount_wait (path,0 );
@@ -442,8 +463,11 @@ static int autofs_d_manage(const struct path *path, bool rcu_walk) }/* Wait for pending expires */ -if (do_expire_wait (path, rcu_walk) == -ECHILD ) + status =do_expire_wait (path, rcu_walk); +if (status == -ECHILD )return -ECHILD ; +if (autofs_expire_wait_blocked (status)) +return -EAGAIN ;/* * This dentry may be under construction so wait on mount
fs/splice.c +12 -1
@@ -686,6 +686,7 @@ iter_file_splice_write(struct pipe_inode_info *pipe, struct file *out,struct kiocb kiocb;struct iov_iter from;unsigned int head, tail; +bool was_in_splice_pipe_callback;size_t left;int n;
@@ -733,7 +734,11 @@ iter_file_splice_write(struct pipe_inode_info *pipe, struct file *out,iov_iter_bvec (&from,ITER_SOURCE , array, n, sd.total_len - left);init_sync_kiocb (&kiocb, out); kiocb.ki_pos = sd.pos; + was_in_splice_pipe_callback = current->in_splice_pipe_callback; +if (pipe->files) + current->in_splice_pipe_callback = true; ret = out->f_op->write_iter (&kiocb, &from); + current->in_splice_pipe_callback = was_in_splice_pipe_callback; sd.pos = kiocb.ki_pos;if (ret <=0 )break ;
@@ -1282,12 +1287,18 @@ ssize_t splice_file_to_pipe(struct file *in,loff_t *offset, size_t len, unsigned int flags) { + bool was_in_splice_pipe_callback;ssize_t ret;pipe_lock (opipe); ret =wait_for_space (opipe, flags); -if (!ret) +if (!ret) { + was_in_splice_pipe_callback = current->in_splice_pipe_callback; +if (opipe->files) + current->in_splice_pipe_callback = true; ret =do_splice_read (in, offset, opipe, len, flags); + current->in_splice_pipe_callback = was_in_splice_pipe_callback; + }pipe_unlock (opipe);if (ret >0 )wakeup_pipe_readers (opipe);
include/linux/sched.h +2 -0
@@ -1025,6 +1025,8 @@ struct task_struct {/* Bit to tell TOMOYO we're in execve(): */ unsigned in_execve:1; unsigned in_iowait:1; +/* A file callback is running while splice holds a pipe mutex. */ + unsigned in_splice_pipe_callback:1;#ifndef TIF_RESTORE_SIGMASK unsigned restore_sigmask:1;#endif
kernel/fork.c +1 -0
@@ -2133,6 +2133,7 @@ __latent_entropy struct task_struct *copy_process( p =dup_task_struct (current, node);if (!p) goto fork_out; + p->in_splice_pipe_callback =0 ;/* * Must run before the first fallible op, so error paths never * free the parent's ret_stack.
Signature · 1 line
2.53.0
Tejun Heo
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
In reply to Krystian KaniewskiHello, Krystian.
Quoted 5 lines · Krystian Kaniewski
syzbot reports a lockdep cycle
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutex. splice holds a pipe mutex while it calls the write or read handler of a kernfs file, kernfs takesof->mutexthere, and a write to/sys/power/resumelooks up a path that can make autofs write a request into its daemon pipe and wait for the daemon.
This is an extremely convoluted case that isn't going to happen in any realistic scenario, and it doesn't make sense to change the locking scheme for it. Can we just disallow splice on /sys/power/resume instead?
Thanks.
--
tejun
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Quoted 12 lines · Tejun Heo
Hello, Krystian.
Quoted 8 lines · Krystian Kaniewski
syzbot reports a lockdep cycle
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutex. splice holds a pipe mutex while it calls the write or
read handler of a kernfs file, kernfs takesof->mutexthere, and a write
to/sys/power/resumelooks up a path that can make autofs write a request
into its daemon pipe and wait for the daemon.
Quoted 5 lines
This is an extremely convoluted case that isn't going to happen in any
realistic scenario, and it doesn't make sense to change the locking scheme
for it. Can we just disallow splice on/sys/power/resumeinstead?
I agree but it might not be as simple as disallowing that path.
Not only is the description hard to follow the example is flawed because there should not be a mount callback on anything that's not an empty directory or an autofs trigger mount (and the above should be able to be either (systemd's /proc/sys/fs/binfmt_misc is an example of an autofs trigger mount).
I mostly can't grok the reproducers for things like this so a complicated description makes it so much harder to come up with a sensible solution.
The other possibility I was thinking about when I saw that report is to disallow file systems that don't make sense for automount.
That would take some thought though and isn't entirely clear for kernfs since it's used by other file systems that may occasionally need to automount something at some point.
OTOH, if the problem description could be cleaned up, and simplified, I wouldn't be entirely against the change but still wouldn't much like it. That's purely because I've always been in favor of defensing programming even for cases that are unlikely.
Ian
Re: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Hi Tejun, Ian,
Quoted 3 lines · Ian Kent
Quoted 2 lines · Tejun Heo
it doesn't make sense to change the locking scheme for it. Can we just disallow splice on
/sys/power/resumeinstead?
That would address the direct case, but it would not make this report go away. The lockdep report combines two separate operations.
First, the reproducer splices a pipe into the cgroup v1 devices.deny file. This records:
&pipe->mutex -> &of->mutex
It later uses a normal write() to /sys/power/resume. That records:
&of->mutex -> &sbi->pipe_mutex -> &pipe->mutex
There is no splice into /sys/power/resume in the reported sequence. Disabling splice on that file would prevent some direct deadlock cases, but the first edge of this report would still come from devices.deny.
A complete restriction for this attribute would also have to cover both splice directions. With an O_RDWR open, a splice read takes the output pipe mutex before the same of->mutex. Another thread can write through the same open and reach autofs in the opposite order.
Kernfs splice support is established behavior. Commit f2d6c2708bd8 ("kernfs: wire up ->splice_read and ->splice_write") explicitly restored it in v5.11 after the VFS stopped providing the implicit fallback. The pipe has been locked across ->write_iter() since 2014.
I agree that changing serialization for every regular sysfs store is too broad for this case. Before preparing a v2, I would like to clarify the intended scope.
Quoted 1 line
Not only is the description hard to follow the example is flawed
You are right that the cover letter was hard to follow. I should have described the two operations separately. The automount itself uses the normal empty directory case.
The reproducer creates a detached indirect autofs mount with its own daemon pipe. It changes its current directory to the root of that mount and moves out of the daemon process group. It then writes the relative name "0x0000000004000921" to /sys/power/resume.
resume_store() calls lookup_bdev() on that string before trying to parse it as a device number. The pathname lookup therefore starts in the autofs root. The new negative name triggers the normal mount request for an empty indirect autofs directory.
The dentry whose ->d_automount callback runs belongs to autofs, not to sysfs. There is no autofs mount on top of kernfs in this reproducer. Kernfs is involved only because resume_store() starts the pathname lookup.
The change which made this path possible is 1e8c813b083c ("PM:
hibernate: don't use early_lookup_bdev in resume_store"). It replaced early_lookup_bdev() with the normal pathname lookup in v6.5. That is what made autofs reachable from resume_store().
Quoted 2 lines
The other possibility I was thinking about when I saw that report is to disallow file systems that don't make sense for automount.
I may be misunderstanding this suggestion. In this reproducer, the filesystem which owns the automount dentry is autofs. A restriction based on the filesystem owning that dentry would therefore see autofs, not kernfs.
The lookup from a sysfs store is also not unique to /sys/power/resume. For example, with CONFIG_BCACHE=y, sys/fs/bcache/register also resolves a user supplied pathname.
The exact reproducer records the lockdep edges with different pipes and different open files. It does not deadlock on those mutex instances. After sending the request, it waits killably for a daemon reply that never arrives. Related setups can deadlock when the daemon pipe itself participates in splice or tee.
Given these two separate paths, did you intend to prevent only direct splice operations on /sys/power/resume even though the reported class cycle would remain, or should a replacement also eliminate the lockdep report?
A per-attribute splice restriction alone cannot do both. If this does not justify new kernfs infrastructure, I can leave it alone.
Thanks,
Signature · 1 line
Krystian Kaniewski
Tejun Heo
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Hello, Krystian.
Quoted 4 lines · Krystian Kaniewski
Given these two separate paths, did you intend to prevent only direct splice operations on
/sys/power/resumeeven though the reported class cycle would remain, or should a replacement also eliminate the lockdep report?
The lockdep report should go away too.
What about disabling splice only for the files whose ->store() can look up arbitrary paths, such as /sys/power/resume, and giving their of->mutex a separate lockdep class? Splice is what nests of->mutex inside pipe->mutex, so if those files can't be spliced, pipe->mutex never nests outside the new class and the cycle is gone. Commit 16b52bbee482 ("kernfs: annotate different lockdep class for of->mutex of writable files") did the same kind of split for writable files.
Thanks.
--
tejun
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
In reply to Krystian KaniewskiQuoted 45 lines · Krystian Kaniewski
Hi Tejun, Ian,
Quoted 3 lines · Ian Kent
Quoted 2 lines · Tejun Heo
it doesn't make sense to change the locking scheme for it. Can we just disallow splice on
/sys/power/resumeinstead?That would address the direct case, but it would not make this report go away. The lockdep report combines two separate operations.
First, the reproducer splices a pipe into the cgroup v1 devices.deny file. This records:
&pipe->mutex->&of->mutexIt later uses a normal
write()to/sys/power/resume. That records:
&of->mutex->&sbi->pipe_mutex->&pipe->mutexThere is no splice into
/sys/power/resumein the reported sequence. Disabling splice on that file would prevent some direct deadlock cases, but the first edge of this report would still come from devices.deny.A complete restriction for this attribute would also have to cover both splice directions. With an
O_RDWRopen, a splice read takes the output pipe mutex before the sameof->mutex. Another thread can write through the same open and reach autofs in the opposite order.Kernfs splice support is established behavior. Commit
f2d6c2708bd8("kernfs: wire up ->splice_read and ->splice_write") explicitly restored it in v5.11 after the VFS stopped providing the implicit fallback. The pipe has been locked across ->write_iter() since 2014.I agree that changing serialization for every regular sysfs store is too broad for this case. Before preparing a v2, I would like to clarify the intended scope.
Quoted 1 line
Not only is the description hard to follow the example is flawed
You are right that the cover letter was hard to follow. I should have described the two operations separately. The automount itself uses the
normal empty directory case.
Quoted 4 lines
The reproducer creates a detached indirect autofs mount with its own daemon pipe. It changes its current directory to the root of that mount and moves out of the daemon process group. It then writes the relative name "
0x0000000004000921" to/sys/power/resume.
I still don't quite get the scenario.
Maybe it's just because I (still) don't understand how autofs is implicated.
By definition an indirect mount is an autofs file system that triggers mounts in the root directory of the autofs mount and those entries must be empty directories. So I don't get how accessing /sys/power/resume can result in triggering an automount. At least that's how it reads to me from my admittedly autofs centric POV.
If there is something missing from autofs that could be used to prevent this scenario then I'd be happy to implement that.
The above paragraph reads like the autofs file system needs to check request paths are within the autofs file system but that doesn't make sense either because that that would mean there's an autofs mount on /sys/power. But it is stated below there isn't an autofs mount within kernfs so rejecting autofs file system mounts within kernfs won't help.
Note that an autofs direct mount might be susceptible to this as well.
Quoted 10 lines
resume_store()callslookup_bdev()on that string before trying to parse it as a device number. The pathname lookup therefore starts in the autofs root. The new negative name triggers the normal mount request for an empty indirect autofs directory.The dentry whose ->d_automount callback runs belongs to autofs, not to sysfs. There is no autofs mount on top of kernfs in this reproducer. Kernfs is involved only because
resume_store()starts the pathname lookup.
I'm sorry, there's just not enough path information to work out if there's something going on that should be checked for and rejected.
Quoted 6 lines
The change which made this path possible is
1e8c813b083c("PM:
hibernate: don't useearly_lookup_bdevinresume_store"). It replacedearly_lookup_bdev()with the normal pathname lookup in v6.5. That is what made autofs reachable fromresume_store().
Quoted 8 lines
Quoted 2 lines
The other possibility I was thinking about when I saw that report is to disallow file systems that don't make sense for automount.
I may be misunderstanding this suggestion. In this reproducer, the filesystem which owns the automount dentry is autofs. A restriction based on the filesystem owning that dentry would therefore see autofs,
not kernfs.
As I say, it sounds like, the only valid indirect mount request must be triggered by a sub-directory of /sys/power which is a mountpoint within kernfs (or maybe we are talking sysfs) which could be rejected in the autofs mount code.
But it's not straight forward to be sure about which file systems should not allow auto-mounting but is probably doable. For example, systemd (direct) mounts /proc/sys/fs/binfmt_misc but mostly proc should not allow auto-mounting.
Quoted 10 lines
The lookup from a sysfs store is also not unique to
/sys/power/resume. For example, withCONFIG_BCACHE=y, sys/fs/bcache/register also resolves a user supplied pathname.The exact reproducer records the lockdep edges with different pipes and different open files. It does not deadlock on those mutex instances. After sending the request, it waits killably for a daemon reply that never arrives. Related setups can deadlock when the daemon pipe itself participates in splice or tee.
I think this is what's causing me trouble.
Such a request should never be sent over the pipe, there should be validity checks in autofs to prevent it. Unfortunately I'm not sure about the path info.
Ian
Quoted 10 lines
Given these two separate paths, did you intend to prevent only direct splice operations on
/sys/power/resumeeven though the reported class cycle would remain, or should a replacement also eliminate the lockdep report?A per-attribute splice restriction alone cannot do both. If this does not justify new kernfs infrastructure, I can leave it alone.
Thanks,
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
In reply to Krystian KaniewskiQuoted 29 lines · Krystian Kaniewski
syzbot reports a lockdep cycle
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutex. splice holds a pipe mutex while it calls the write or
read handler of a kernfs file, kernfs takesof->mutexthere, and a write
to/sys/power/resumelooks up a path that can make autofs write a request
into its daemon pipe and wait for the daemon.Patch 1 stops holding
of->mutexaround the ->store() callback of regular
sysfs attributes, which do not get the open file and already run concurrently for separate opens.Patch 2 handles the case that patch 1 leaves. A store called from splice
still runs with the pipe locked. autofs no longer sends or waits for a
request in that context and returns-EAGAINinstead. A new mount is refused before it is marked pending, so no other lookup sends the request
in its place. This also removes a deadlock in which the spliced pipe is
the autofs daemon pipe itself.Patch 2 changes behavior for one narrow case. A lookup from a splice callback that needs an automount fails with
-EAGAINinstead of waiting.
Quoted 7 lines
The series was tested in QEMU with lockdep using the original C and syz
reproducers and focused splice and autofs cases. The target reports were
not reproduced with the series applied, and the paired functional cases
matched the unpatched baseline.
I can't seem to find a reproducer that reproduces the bug or maybe I don't know what to expect from running the syzbot reproducer I did locate.
Can anyone help please.
Ian
Quoted 18 lines
Krystian Kaniewski (2):
kernfs: do not holdof->mutexacross sysfs ->store()
autofs: do not wait for the daemon while splice holds a pipefs/autofs/expire.c | 6 +++++
fs/autofs/root.c | 28 +++++++++++++++++++++-- fs/kernfs/file.c | 52 +++++++++++++++++++++++++++------------- --
fs/splice.c | 13 ++++++++++-
fs/sysfs/file.c | 18 ++++++++++++++-include/linux/kernfs.h| 11 +++++++++ include/linux/sched.h | 2 ++
kernel/fork.c | 1 +8 files changed, 109 insertions(+), 22 deletions(-)base-commit:
93f51579e7df248780214094418f205253383cc5
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Quoted 41 lines · Ian Kent
Quoted 33 lines · Krystian Kaniewski
syzbot reports a lockdep cycle
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutex. splice holds a pipe mutex while it calls the write
or
read handler of a kernfs file, kernfs takesof->mutexthere, and a write
to/sys/power/resumelooks up a path that can make autofs write a request
into its daemon pipe and wait for the daemon.Patch 1 stops holding
of->mutexaround the ->store() callback of regular
sysfs attributes, which do not get the open file and already run concurrently for separate opens.Patch 2 handles the case that patch 1 leaves. A store called from splice
still runs with the pipe locked. autofs no longer sends or waits for
a
request in that context and returns-EAGAINinstead. A new mount is refused before it is marked pending, so no other lookup sends the request
in its place. This also removes a deadlock in which the spliced pipe
is
the autofs daemon pipe itself.Patch 2 changes behavior for one narrow case. A lookup from a splice
callback that needs an automount fails with-EAGAINinstead of waiting.Quoted 7 lines
The series was tested in QEMU with lockdep using the original C and syz
reproducers and focused splice and autofs cases. The target reports were
not reproduced with the series applied, and the paired functional cases
matched the unpatched baseline.
Quoted 3 lines
I can't seem to find a reproducer that reproduces the bug or maybe I don't know what to expect from running the syzbot reproducer I did locate.
I think I have found the appropriate reproducer, certainly looks like it.
Commit 1e8c813b083c ("PM: hibernate: don't use early_lookup_bdev in resume_store") looks like it's been around for a while so I assume the reproducer should work on 7.2.12?
What should I expect, a hang, it doesn't.
Also it looks to me like the reproducer is trying exploit an invalid request similar to what I described earlier but I can't test to see if my simple autofs resolution fixes it if I don't know what to expect from the reproducer.
Ian
Quoted 23 lines
Can anyone help please.
Ian
Quoted 19 lines
Krystian Kaniewski (2):
kernfs: do not holdof->mutexacross sysfs ->store()
autofs: do not wait for the daemon while splice holds a pipefs/autofs/expire.c | 6 +++++
fs/autofs/root.c | 28 +++++++++++++++++++++-- fs/kernfs/file.c | 52 +++++++++++++++++++++++++++----------- --
--
fs/splice.c | 13 ++++++++++-
fs/sysfs/file.c | 18 ++++++++++++++-include/linux/kernfs.h| 11 +++++++++ include/linux/sched.h | 2 ++
kernel/fork.c | 1 +8 files changed, 109 insertions(+), 22 deletions(-)base-commit:
93f51579e7df248780214094418f205253383cc5
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Quoted 57 lines · Ian Kent
Quoted 45 lines · Ian Kent
Quoted 35 lines · Krystian Kaniewski
syzbot reports a lockdep cycle
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutex. splice holds a pipe mutex while it calls the write
or
read handler of a kernfs file, kernfs takesof->mutexthere, and a
write
to/sys/power/resumelooks up a path that can make autofs write a request
into its daemon pipe and wait for the daemon.Patch 1 stops holding
of->mutexaround the ->store() callback of regular
sysfs attributes, which do not get the open file and already run concurrently for separate opens.Patch 2 handles the case that patch 1 leaves. A store called from splice
still runs with the pipe locked. autofs no longer sends or waits for
a
request in that context and returns-EAGAINinstead. A new mount is
refused before it is marked pending, so no other lookup sends the request
in its place. This also removes a deadlock in which the spliced pipe
is
the autofs daemon pipe itself.Patch 2 changes behavior for one narrow case. A lookup from a splice
callback that needs an automount fails with-EAGAINinstead of waiting.Quoted 9 lines
The series was tested in QEMU with lockdep using the original C and
syz
reproducers and focused splice and autofs cases. The target reports
were
not reproduced with the series applied, and the paired functional cases
matched the unpatched baseline.Quoted 3 lines
I can't seem to find a reproducer that reproduces the bug or maybe I don't know what to expect from running the syzbot reproducer I did locate.
I think I have found the appropriate reproducer, certainly looks like it.
Commit
1e8c813b083c("PM: hibernate: don't useearly_lookup_bdevinresume_store") looks like it's been around for a while so I assume the reproducer should work on 7.2.12?
Quoted 6 lines
What should I expect, a hang, it doesn't.
Also it looks to me like the reproducer is trying exploit an invalid request similar to what I described earlier but I can't test to see if my simple autofs resolution fixes it if I don't know what to expect from the reproducer.
Further investigation shows that the reproducer is initiating path walks within the autofs mount it has mounted which is valid in the sense the autofs file system will attempt to send a request over it's pipe. These are not seen as invalid so I can't prevent them in the way I thought I might be able to.
I can't confirm there is a deadlock, how do I do this?
A little more validation and I'll be in a position to review and probably ack the autofs changes.
Ian
Quoted 26 lines
Ian
Quoted 24 lines
Can anyone help please.
Ian
Quoted 20 lines
Krystian Kaniewski (2):
kernfs: do not holdof->mutexacross sysfs ->store()
autofs: do not wait for the daemon while splice holds a pipefs/autofs/expire.c | 6 +++++
fs/autofs/root.c | 28 +++++++++++++++++++++-- fs/kernfs/file.c | 52 +++++++++++++++++++++++++++--------- --
--
--
fs/splice.c | 13 ++++++++++- fs/sysfs/file.c | 18 ++++++++++++++-include/linux/kernfs.h| 11 +++++++++ include/linux/sched.h | 2 ++
kernel/fork.c | 1 +8 files changed, 109 insertions(+), 22 deletions(-)base-commit:
93f51579e7df248780214094418f205253383cc5
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Quoted 81 lines · Ian Kent
Quoted 67 lines · Ian Kent
Quoted 54 lines · Ian Kent
Quoted 43 lines · Krystian Kaniewski
syzbot reports a lockdep cycle
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutex. splice holds a pipe mutex while it calls the write
or
read handler of a kernfs file, kernfs takesof->mutexthere, and
a
write
to/sys/power/resumelooks up a path that can make autofs write a
request
into its daemon pipe and wait for the daemon.Patch 1 stops holding
of->mutexaround the ->store() callback of
regular
sysfs attributes, which do not get the open file and already run
concurrently for separate opens.Patch 2 handles the case that patch 1 leaves. A store called from
splice
still runs with the pipe locked. autofs no longer sends or waits
for
a
request in that context and returns-EAGAINinstead. A new mount
is
refused before it is marked pending, so no other lookup sends the
request
in its place. This also removes a deadlock in which the spliced pipe
is
the autofs daemon pipe itself.Patch 2 changes behavior for one narrow case. A lookup from a splice
callback that needs an automount fails with-EAGAINinstead of waiting.Quoted 10 lines
The series was tested in QEMU with lockdep using the original C and
syz
reproducers and focused splice and autofs cases. The target reports
were
not reproduced with the series applied, and the paired functional
cases
matched the unpatched baseline.Quoted 4 lines
I can't seem to find a reproducer that reproduces the bug or maybe
I don't know what to expect from running the syzbot reproducer I did locate.I think I have found the appropriate reproducer, certainly looks like it.
Commit
1e8c813b083c("PM: hibernate: don't useearly_lookup_bdevinresume_store") looks like it's been around for a while so I assume the reproducer should work on 7.2.12?Quoted 6 lines
What should I expect, a hang, it doesn't.
Also it looks to me like the reproducer is trying exploit an invalid request similar to what I described earlier but I can't test to see if my simple autofs resolution fixes it if I don't know what to expect from the reproducer.
Further investigation shows that the reproducer is initiating path walks within the autofs mount it has mounted which is valid in the sense the autofs file system will attempt to send a request over it's pipe. These are not seen as invalid so I can't prevent them in the way I thought I might be able to.
Quoted 4 lines
I can't confirm there is a deadlock, how do I do this?
A little more validation and I'll be in a position to review and probably ack the autofs changes.
Sorry for taking so long getting back but I'm struggling with a few time critical tasks (including this one).
I haven't looked as far as I wanted to yet but I have looked at the autofs patch again mainly because it occurred to me that the EAGAIN return might be a bit of a problem.
The expiration of automounts has always handled EAGAIN returns even though that handling has changed over the years. So this amounts to little more than deferring an expire which is fine.
But mount requests have never handled EAGAIN and I'm pretty sure this return will result in an unexpected mount fail being returned to the requester in user space. OTOH this deadlock case should not occur very often and neither should the case where an automount has expired (so the mount has gone away and hence a path walk initiated) but the automount daemon has not yet finalised the expire by notifying the kernel.
Also I still think the scenario where this occurs is something that will not happen in normal use and would most likely only only occur for nefarious purposes. For these reasons it's a little bit of an open question whether the autofs changes are OK.
Updating user space is of course possible but significantly difficult due to the number of distributions that would/may need to update the daemon.
Another option is to handle these EAGAIN returns for NFY_MOUNT requests in the autofs fs itself by re-queuing the mount request. In fact I already had a patch series to do something very similar to handle stale request entries after a "kill -9" followed immediately by an autofs startup. Sadly I got side tracked by overthinking what was happening and reworked the patch series considerably and had not tested it at all. I'm pretty sure that a recent contributed autofs patch will fix what lead me to this diversion and it will take a while to recover the original series (or re-write it).
Ian
Quoted 29 lines
Ian
Quoted 27 lines
Ian
Quoted 25 lines
Can anyone help please.
Ian
Quoted 21 lines
Krystian Kaniewski (2):
kernfs: do not holdof->mutexacross sysfs ->store()
autofs: do not wait for the daemon while splice holds a pipefs/autofs/expire.c | 6 +++++
fs/autofs/root.c | 28 +++++++++++++++++++++-- fs/kernfs/file.c | 52 +++++++++++++++++++++++++++------- --
--
--
--
fs/splice.c | 13 ++++++++++- fs/sysfs/file.c | 18 ++++++++++++++-include/linux/kernfs.h| 11 +++++++++ include/linux/sched.h | 2 ++
kernel/fork.c | 1 +8 files changed, 109 insertions(+), 22 deletions(-)base-commit:
93f51579e7df248780214094418f205253383cc5
Re: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
In reply to Ian KentHi Ian,
Quoted 3 lines · Ian Kent
Commit
1e8c813b083c("PM: hibernate: don't useearly_lookup_bdevinresume_store") looks like it's been around for a while so I assume the reproducer should work on 7.2.12?
Yes. That commit entered v6.5-rc1, so 7.2.12 contains the relevant code.
I used the unmodified C reproducer linked from the syzbot report:
https://syzkaller.appspot.com/text?tag=ReproC&x=11c0f4c9580000
I also used the configuration provided with the report:
https://syzkaller.appspot.com/text?tag=KernelConfig&x=7d012d9c67977ee4
Quoted 1 line
What should I expect, a hang, it doesn't.
A permanent hang is not expected. The result to look for is this lockdep warning in dmesg:
WARNING: possible circular locking dependency detected
...
&pipe->mutex --> &of->mutex --> &sbi->pipe_mutexThe reproducer records the first dependency by splicing into the cgroup v1 devices.deny file. It records the opposite dependencies through a normal write to /sys/power/resume. These operations use different lock instances, so the exact reproducer reports a possible cycle rather than deadlocking.
The write waits for an autofs daemon that never replies, but the C wrapper kills the blocked child after five seconds and starts another iteration. In my test the warning appeared in the first iteration.
The test needs lockdep and CONFIG_PROVE_LOCKING, the cgroup v1 devices controller, autofs, and working hibernation support. It must also run as root. Without lockdep, the program can appear to do nothing useful.
Quoted 2 lines
Also it looks to me like the reproducer is trying exploit an invalid request similar to what I described earlier
Whether the string identifies a usable resume device is separate from the autofs request. resume_store() first passes it to lookup_bdev(), which performs a pathname lookup before the numeric fallback.
The reproducer changes its current directory to the root of a detached indirect autofs mount. It then opens /sys/power/resume using an absolute path and writes the relative name "0x0000000004000921".
The resulting lookup is therefore:
<autofs root>/0x0000000004000921
For autofs, this is an ordinary lookup of a new immediate child of the indirect mount root, so it is a normal mount trigger. There is no autofs mount under /sys/power.
In parallel, I am preparing a revised patch following Tejun's suggestion. It disables splice for writable opens of /sys/power/resume and gives its of->mutex a separate lockdep class.
If you send your autofs change, I can run the same reproducer against it and check both the request and the lockdep warning.
Thanks,
Krystian
Ian Kent
maintainerRe: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
Quoted 2 lines · Krystian Kaniewski
Hi Ian,
Thanks for replying, ;)
Quoted 35 lines · Ian Kent
Quoted 3 lines
Commit
1e8c813b083c("PM: hibernate: don't useearly_lookup_bdevinresume_store") looks like it's been around for a while so I assume the reproducer should work on 7.2.12?Yes. That commit entered v6.5-rc1, so 7.2.12 contains the relevant code.
I used the unmodified C reproducer linked from the syzbot report:
https://syzkaller.appspot.com/text?tag=ReproC&x=11c0f4c9580000
I also used the configuration provided with the report:
https://syzkaller.appspot.com/text?tag=KernelConfig&x=7d012d9c67977ee4
Quoted 1 line
What should I expect, a hang, it doesn't.
A permanent hang is not expected. The result to look for is this lockdep warning in dmesg:
WARNING: possible circular locking dependency detected
...
&pipe->mutex-->&of->mutex-->&sbi->pipe_mutexThe reproducer records the first dependency by splicing into the cgroup v1 devices.deny file. It records the opposite dependencies through a normal write to
/sys/power/resume. These operations use different lock instances, so the exact reproducer reports a possible cycle rather than deadlocking.The write waits for an autofs daemon that never replies, but the C wrapper kills the blocked child after five seconds and starts another iteration. In my test the warning appeared in the first iteration.
Quoted 5 lines
The test needs lockdep and
CONFIG_PROVE_LOCKING, the cgroup v1 devices
controller, autofs, and working hibernation support. It must also run as root. Without lockdep, the program can appear to do nothing useful.
Ok, I'll check on those, not sure about hibernation support in my VM ...
Quoted 14 lines
Quoted 3 lines
Also it looks to me like the reproducer is trying exploit an invalid request similar to what I described earlier
Whether the string identifies a usable resume device is separate from the autofs request.
resume_store()first passes it tolookup_bdev(), which performs a pathname lookup before the numeric fallback.The reproducer changes its current directory to the root of a detached
indirect autofs mount. It then opens/sys/power/resumeusing an absolute path and writes the relative name "0x0000000004000921".
Quoted 8 lines
The resulting lookup is therefore:
<autofs root>/0x0000000004000921
For autofs, this is an ordinary lookup of a new immediate child of the
indirect mount root, so it is a normal mount trigger. There is no autofs mount under/sys/power.
Right, enabling the autofs pr-debug logging and looking at the reproducer verified the exection flow for me. I didn't see a lockdep splat but my environment is probably not quite right. I'll check what I can and see if I can reproduce.
Quoted 10 lines
In parallel, I am preparing a revised patch following Tejun's suggestion. It disables splice for writable opens of
/sys/power/resumeand gives itsof->mutexa separate lockdep class.If you send your autofs change, I can run the same reproducer against it and check both the request and the lockdep warning.
Thanks,
Krystian
Re: [PATCH 0/2] kernfs, autofs: fix pipe lock cycles through sysfs stores
In reply to Ian KentOne additional detail I missed in my previous reply. I reproduced the warning on the unmodified kernel at commit:
93f51579e7df248780214094418f205253383cc5
Exact kernel version:
7.3.0-rc4-g93f51579e7df #1
I ran it under QEMU TCG using the configuration supplied by syzbot. Both the unmodified C reproducer and the original syz reproducer reported the same lockdep cycle.
Thanks,
Krystian