Skip to contentSkip to conversation

BUG: unable to handle kernel paging request in delayed_put_task_struct

DISCUSSION Dmitry Vyukov 2 participants 3 messages Dec 19, 2017 – Dec 22, 2017 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 107 lines · syzbot

Hello,

syzkaller hit the following crash on
5bef2980adef8a6032d4f4709aebe9486181052f
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/master
compiler: gcc (GCC) 7.1.1 20170620
.config is attached
Raw console output is attached.

Unfortunately, I don't have any reproducer for this bug yet.

BUG: unable to handle kernel paging request at fffffffffffffff8
IP: delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178
netlink: 6 bytes leftover after parsing attributes in process
`syz-executor0'.
PGD 5e28067 P4D 5e28067 PUD 5e2a067 PMD 0
Oops: 0002 [#1] SMP KASAN
Dumping ftrace buffer:
   (ftrace buffer empty)
Modules linked in:
CPU: 1 PID: 5363 Comm: syz-executor4 Not tainted 4.15.0-rc1-next-20171128+
#54
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
task: ffff8801d8d244c0 task.stack: ffff8801c15d8000
RIP: 0010:delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178
RSP: 0018:ffff8801db5078b0 EFLAGS: 00010206
RAX: ffff8801d8d244c0 RBX: 1ffff1003b6a0f17 RCX: ffffffff81872631
RDX: 0000000000000100 RSI: 000000003189bbb1 RDI: ffff8801d8bbb480
RBP: ffff8801db507980 R08: 0000000000000005 R09: 0000000000000002
R10: 0000000000000000 R11: ffffffff8748cd60 R12: ffff8801d8bbb5d0
R13: ffff8801d8bbb5d8 R14: ffff8801d8bba1c0 R15: ffff8801db507c58
FS:  00007f251e195700(0000) GS:ffff8801db500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: fffffffffffffff8 CR3: 00000001980c2000 CR4: 00000000001426e0
Call Trace:
 <IRQ>
 __rcu_reclaim kernel/rcu/rcu.h:172 [inline]
 rcu_do_batch kernel/rcu/tree.c:2674 [inline]
 invoke_rcu_callbacks kernel/rcu/tree.c:2933 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:2900 [inline]
 rcu_process_callbacks+0xd74/0x17d0 kernel/rcu/tree.c:2917
 __do_softirq+0x29d/0xbb2 kernel/softirq.c:285
 invoke_softirq kernel/softirq.c:365 [inline]
 irq_exit+0x1d3/0x210 kernel/softirq.c:405
 exiting_irq arch/x86/include/asm/apic.h:540 [inline]
 smp_apic_timer_interrupt+0x16b/0x700 arch/x86/kernel/apic/apic.c:1052
 apic_timer_interrupt+0xa9/0xb0 arch/x86/entry/entry_64.S:907
 </IRQ>
RIP: 0010:__wrmsr arch/x86/include/asm/msr.h:105 [inline]
RIP: 0010:native_write_msr+0xa/0x30 arch/x86/include/asm/msr.h:148
RSP: 0018:ffff8801c15de5a0 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff11
RAX: 00000000000000fb RBX: ffff8801db413124 RCX: 0000000000000830
RDX: 0000000000000000 RSI: 00000000000000fb RDI: 0000000000000830
RBP: ffff8801c15de5a0 R08: 1ffff100382bbd00 R09: 0000000000000000
R10: ffff8801c15de880 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff81270180
 paravirt_write_msr arch/x86/include/asm/paravirt.h:116 [inline]
 wrmsrl arch/x86/include/asm/paravirt.h:149 [inline]
 native_x2apic_icr_write arch/x86/include/asm/apic.h:240 [inline]
 __x2apic_send_IPI_dest+0x5c/0x80 arch/x86/kernel/apic/x2apic_phys.c:117
 x2apic_send_IPI+0x6f/0xa0 arch/x86/kernel/apic/x2apic_cluster.c:35
 native_send_call_func_single_ipi+0x55/0x70 arch/x86/kernel/smp.c:136
 arch_send_call_function_single_ipi arch/x86/include/asm/smp.h:121 [inline]
 generic_exec_single+0x185/0x5b0 kernel/smp.c:179
 smp_call_function_single+0x2d2/0x560 kernel/smp.c:299
 smp_call_function_many+0x773/0x930 kernel/smp.c:434
 smp_call_function kernel/smp.c:492 [inline]
 on_each_cpu+0x3d/0x1b0 kernel/smp.c:602
 text_poke_bp+0xbb/0x170 arch/x86/kernel/alternative.c:807
 __jump_label_transform.isra.0+0x6a5/0x8a0 arch/x86/kernel/jump_label.c:102
 arch_jump_label_transform+0x2f/0x40 arch/x86/kernel/jump_label.c:110
 __jump_label_update+0x207/0x2d0 kernel/jump_label.c:368
 jump_label_update+0x22c/0x2b0 kernel/jump_label.c:735
 static_key_slow_dec_cpuslocked+0x176/0x1d0 kernel/jump_label.c:204
 __static_key_slow_dec kernel/jump_label.c:214 [inline]
 static_key_slow_dec+0x56/0x90 kernel/jump_label.c:228
 tracepoint_remove_func kernel/tracepoint.c:252 [inline]
 tracepoint_probe_unregister+0x70d/0x870 kernel/tracepoint.c:323
 trace_event_reg+0xed/0x320 kernel/trace/trace_events.c:309
 perf_trace_event_unreg.isra.2+0xad/0x1f0
kernel/trace/trace_event_perf.c:155
 perf_trace_destroy+0xbc/0x100 kernel/trace/trace_event_perf.c:236
 tp_perf_event_destroy+0x15/0x20 kernel/events/core.c:7940
 _free_event+0x3bd/0x10f0 kernel/events/core.c:4104
 put_event+0x24/0x30 kernel/events/core.c:4187
 perf_event_release_kernel+0x407/0xc10 kernel/events/core.c:4288
 perf_release+0x37/0x50 kernel/events/core.c:4298
 __fput+0x333/0x7f0 fs/file_table.c:210
 ____fput+0x15/0x20 fs/file_table.c:244
 task_work_run+0x199/0x270 kernel/task_work.c:113
 exit_task_work include/linux/task_work.h:22 [inline]
 do_exit+0x9bb/0x1ae0 kernel/exit.c:869
 do_group_exit+0x149/0x400 kernel/exit.c:972
 get_signal+0x73f/0x16c0 kernel/signal.c:2335
 do_signal+0x94/0x1ee0 arch/x86/kernel/signal.c:809
 exit_to_usermode_loop+0x214/0x310 arch/x86/entry/common.c:158
 prepare_exit_to_usermode arch/x86/entry/common.c:195 [inline]
 syscall_return_slowpath+0x490/0x550 arch/x86/entry/common.c:264
 entry_SYSCALL_64_fastpath+0x94/0x96
RIP: 0033:0x4529d9
RSP: 002b:00007f251e194ce8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: fffffffffffffe00 RBX: 0000000000758100 RCX: 00000000004529d9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000758100
RBP: 0000000000758100 R08: 000000000000055b R09: 00000000007580d8
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
Quoted 8 lines
R13: 0000000000a6f7ff R14: 00007f251e1959c0 R15: 0000000000000002
Code: f1 f1 c7 40 04 00 f2 f2 f2 c7 40 08 f2 f2 f2 f2 c7 40 0c 00 f2 f2 f2
c7 40 10 f3 f3 f3 f3 e8 01 7a 2e 00 4c 89 f7 e8 39 75 45 00 <cc> 78 00 00 00
e8 ef 79 2e 00 e8 ea 79 2e 00 65 8b 05 73 80 bf
RIP: delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178 RSP:
ffff8801db5078b0
CR2: fffffffffffffff8
---[ end trace d50259b7d3fcfc0a ]---

#syz dup: BUG: unable to handle kernel paging request in __switch_to

Quoted 25 lines

---
This bug is generated by a dumb bot. It may contain errors. See https://goo.gl/tpsmEJ for details.
Direct all questions to [email protected].
Please credit me with: Reported-by: syzbot <[email protected]>

syzbot will keep track of this bug report.
Once a fix for this bug is committed, please reply to this email with:
#syz fix: exact-commit-title
To mark this as a duplicate of another syzbot report, please reply with:
#syz dup: exact-subject-of-another-report
If it's a one-off invalid bug report, please reply with:
#syz invalid
Note: if the crash happens again, it will cause creation of a new bug report.
Note: all commands must start from beginning of the line in the email body.

--
You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/syzkaller-bugs/f4f5e803e1a086d1cf055f705a68%40google.com.
For more options, visit https://groups.google.com/d/optout.

#2 lore raw
Quoted 5 lines · Dmitry Vyukov
Quoted 2 lines · syzbot

Hello,

Quoted 8 lines
Quoted 8 lines

syzkaller hit the following crash on
5bef2980adef8a6032d4f4709aebe9486181052f
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/master
compiler: gcc (GCC) 7.1.1 20170620
.config is attached
Raw console output is attached.

Unfortunately, I don't have any reproducer for this bug yet.

That does make things more difficult. Nevertheless...

Quoted 19 lines
Quoted 19 lines
BUG: unable to handle kernel paging request at fffffffffffffff8
IP: delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178
netlink: 6 bytes leftover after parsing attributes in process
`syz-executor0'.
PGD 5e28067 P4D 5e28067 PUD 5e2a067 PMD 0
Oops: 0002 [#1] SMP KASAN
Dumping ftrace buffer:
   (ftrace buffer empty)
Modules linked in:
CPU: 1 PID: 5363 Comm: syz-executor4 Not tainted 4.15.0-rc1-next-20171128+
#54
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
task: ffff8801d8d244c0 task.stack: ffff8801c15d8000
RIP: 0010:delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178
RSP: 0018:ffff8801db5078b0 EFLAGS: 00010206
RAX: ffff8801d8d244c0 RBX: 1ffff1003b6a0f17 RCX: ffffffff81872631
RDX: 0000000000000100 RSI: 000000003189bbb1 RDI: ffff8801d8bbb480
RBP: ffff8801db507980 R08: 0000000000000005 R09: 0000000000000002
Quoted 8 lines
Quoted 8 lines
R10: 0000000000000000 R11: ffffffff8748cd60 R12: ffff8801d8bbb5d0
R13: ffff8801d8bbb5d8 R14: ffff8801d8bba1c0 R15: ffff8801db507c58
FS:  00007f251e195700(0000) GS:ffff8801db500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: fffffffffffffff8 CR3: 00000001980c2000 CR4: 00000000001426e0
Call Trace:
 <IRQ>
 __rcu_reclaim kernel/rcu/rcu.h:172 [inline]

If you do find a reproducer, please try reproducing with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y, which checks for the call_rcu() equivalent to double frees.

Thanx, Paul

Quoted 105 lines
Quoted 68 lines
 rcu_do_batch kernel/rcu/tree.c:2674 [inline]
 invoke_rcu_callbacks kernel/rcu/tree.c:2933 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:2900 [inline]
 rcu_process_callbacks+0xd74/0x17d0 kernel/rcu/tree.c:2917
 __do_softirq+0x29d/0xbb2 kernel/softirq.c:285
 invoke_softirq kernel/softirq.c:365 [inline]
 irq_exit+0x1d3/0x210 kernel/softirq.c:405
 exiting_irq arch/x86/include/asm/apic.h:540 [inline]
 smp_apic_timer_interrupt+0x16b/0x700 arch/x86/kernel/apic/apic.c:1052
 apic_timer_interrupt+0xa9/0xb0 arch/x86/entry/entry_64.S:907
 </IRQ>
RIP: 0010:__wrmsr arch/x86/include/asm/msr.h:105 [inline]
RIP: 0010:native_write_msr+0xa/0x30 arch/x86/include/asm/msr.h:148
RSP: 0018:ffff8801c15de5a0 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff11
RAX: 00000000000000fb RBX: ffff8801db413124 RCX: 0000000000000830
RDX: 0000000000000000 RSI: 00000000000000fb RDI: 0000000000000830
RBP: ffff8801c15de5a0 R08: 1ffff100382bbd00 R09: 0000000000000000
R10: ffff8801c15de880 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff81270180
 paravirt_write_msr arch/x86/include/asm/paravirt.h:116 [inline]
 wrmsrl arch/x86/include/asm/paravirt.h:149 [inline]
 native_x2apic_icr_write arch/x86/include/asm/apic.h:240 [inline]
 __x2apic_send_IPI_dest+0x5c/0x80 arch/x86/kernel/apic/x2apic_phys.c:117
 x2apic_send_IPI+0x6f/0xa0 arch/x86/kernel/apic/x2apic_cluster.c:35
 native_send_call_func_single_ipi+0x55/0x70 arch/x86/kernel/smp.c:136
 arch_send_call_function_single_ipi arch/x86/include/asm/smp.h:121 [inline]
 generic_exec_single+0x185/0x5b0 kernel/smp.c:179
 smp_call_function_single+0x2d2/0x560 kernel/smp.c:299
 smp_call_function_many+0x773/0x930 kernel/smp.c:434
 smp_call_function kernel/smp.c:492 [inline]
 on_each_cpu+0x3d/0x1b0 kernel/smp.c:602
 text_poke_bp+0xbb/0x170 arch/x86/kernel/alternative.c:807
 __jump_label_transform.isra.0+0x6a5/0x8a0 arch/x86/kernel/jump_label.c:102
 arch_jump_label_transform+0x2f/0x40 arch/x86/kernel/jump_label.c:110
 __jump_label_update+0x207/0x2d0 kernel/jump_label.c:368
 jump_label_update+0x22c/0x2b0 kernel/jump_label.c:735
 static_key_slow_dec_cpuslocked+0x176/0x1d0 kernel/jump_label.c:204
 __static_key_slow_dec kernel/jump_label.c:214 [inline]
 static_key_slow_dec+0x56/0x90 kernel/jump_label.c:228
 tracepoint_remove_func kernel/tracepoint.c:252 [inline]
 tracepoint_probe_unregister+0x70d/0x870 kernel/tracepoint.c:323
 trace_event_reg+0xed/0x320 kernel/trace/trace_events.c:309
 perf_trace_event_unreg.isra.2+0xad/0x1f0
kernel/trace/trace_event_perf.c:155
 perf_trace_destroy+0xbc/0x100 kernel/trace/trace_event_perf.c:236
 tp_perf_event_destroy+0x15/0x20 kernel/events/core.c:7940
 _free_event+0x3bd/0x10f0 kernel/events/core.c:4104
 put_event+0x24/0x30 kernel/events/core.c:4187
 perf_event_release_kernel+0x407/0xc10 kernel/events/core.c:4288
 perf_release+0x37/0x50 kernel/events/core.c:4298
 __fput+0x333/0x7f0 fs/file_table.c:210
 ____fput+0x15/0x20 fs/file_table.c:244
 task_work_run+0x199/0x270 kernel/task_work.c:113
 exit_task_work include/linux/task_work.h:22 [inline]
 do_exit+0x9bb/0x1ae0 kernel/exit.c:869
 do_group_exit+0x149/0x400 kernel/exit.c:972
 get_signal+0x73f/0x16c0 kernel/signal.c:2335
 do_signal+0x94/0x1ee0 arch/x86/kernel/signal.c:809
 exit_to_usermode_loop+0x214/0x310 arch/x86/entry/common.c:158
 prepare_exit_to_usermode arch/x86/entry/common.c:195 [inline]
 syscall_return_slowpath+0x490/0x550 arch/x86/entry/common.c:264
 entry_SYSCALL_64_fastpath+0x94/0x96
RIP: 0033:0x4529d9
RSP: 002b:00007f251e194ce8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: fffffffffffffe00 RBX: 0000000000758100 RCX: 00000000004529d9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000758100
RBP: 0000000000758100 R08: 000000000000055b R09: 00000000007580d8
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
Quoted 8 lines
R13: 0000000000a6f7ff R14: 00007f251e1959c0 R15: 0000000000000002
Code: f1 f1 c7 40 04 00 f2 f2 f2 c7 40 08 f2 f2 f2 f2 c7 40 0c 00 f2 f2 f2
c7 40 10 f3 f3 f3 f3 e8 01 7a 2e 00 4c 89 f7 e8 39 75 45 00 <cc> 78 00 00 00
e8 ef 79 2e 00 e8 ea 79 2e 00 65 8b 05 73 80 bf
RIP: delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178 RSP:
ffff8801db5078b0
CR2: fffffffffffffff8
---[ end trace d50259b7d3fcfc0a ]---

#syz dup: BUG: unable to handle kernel paging request in __switch_to

Quoted 25 lines

---
This bug is generated by a dumb bot. It may contain errors. See https://goo.gl/tpsmEJ for details.
Direct all questions to [email protected].
Please credit me with: Reported-by: syzbot <[email protected]>

syzbot will keep track of this bug report.
Once a fix for this bug is committed, please reply to this email with:
#syz fix: exact-commit-title
To mark this as a duplicate of another syzbot report, please reply with:
#syz dup: exact-subject-of-another-report
If it's a one-off invalid bug report, please reply with:
#syz invalid
Note: if the crash happens again, it will cause creation of a new bug report.
Note: all commands must start from beginning of the line in the email body.

--
You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/syzkaller-bugs/f4f5e803e1a086d1cf055f705a68%40google.com.
For more options, visit https://groups.google.com/d/optout.

#3 lore raw
Quoted 45 lines · Paul E. McKenney
Quoted 5 lines · Dmitry Vyukov
Quoted 2 lines · syzbot

Hello,

Quoted 8 lines
Quoted 8 lines

syzkaller hit the following crash on
5bef2980adef8a6032d4f4709aebe9486181052f
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/master
compiler: gcc (GCC) 7.1.1 20170620
.config is attached
Raw console output is attached.

Unfortunately, I don't have any reproducer for this bug yet.

That does make things more difficult. Nevertheless...

Quoted 19 lines
Quoted 19 lines
BUG: unable to handle kernel paging request at fffffffffffffff8
IP: delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178
netlink: 6 bytes leftover after parsing attributes in process
`syz-executor0'.
PGD 5e28067 P4D 5e28067 PUD 5e2a067 PMD 0
Oops: 0002 [#1] SMP KASAN
Dumping ftrace buffer:
   (ftrace buffer empty)
Modules linked in:
CPU: 1 PID: 5363 Comm: syz-executor4 Not tainted 4.15.0-rc1-next-20171128+
#54
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
task: ffff8801d8d244c0 task.stack: ffff8801c15d8000
RIP: 0010:delayed_put_task_struct+0x87/0x3d0 kernel/exit.c:178
RSP: 0018:ffff8801db5078b0 EFLAGS: 00010206
RAX: ffff8801d8d244c0 RBX: 1ffff1003b6a0f17 RCX: ffffffff81872631
RDX: 0000000000000100 RSI: 000000003189bbb1 RDI: ffff8801d8bbb480
RBP: ffff8801db507980 R08: 0000000000000005 R09: 0000000000000002
Quoted 8 lines
Quoted 8 lines
R10: 0000000000000000 R11: ffffffff8748cd60 R12: ffff8801d8bbb5d0
R13: ffff8801d8bbb5d8 R14: ffff8801d8bba1c0 R15: ffff8801db507c58
FS:  00007f251e195700(0000) GS:ffff8801db500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: fffffffffffffff8 CR3: 00000001980c2000 CR4: 00000000001426e0
Call Trace:
 <IRQ>
 __rcu_reclaim kernel/rcu/rcu.h:172 [inline]
Quoted 3 lines

If you do find a reproducer, please try reproducing with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y, which checks for the call_rcu() equivalent to double frees.

This looks like a generally useful config for debugging, so I just enabled CONFIG_DEBUG_OBJECTS* in all runs.

But this particular bug was already root caused. You can see that it's marked as a duplicate of "BUG: unable to handle kernel paging request in __switch_to" in the previous email.
All syzbot reported bugs are here:
https://groups.google.com/forum/#!forum/syzkaller-bugs
You can find "BUG: unable to handle kernel paging request in __switch_to" there and see what was the resolution.