Skip to contentSkip to conversation

general protection fault in is_page_fault_stale

syzbot syzbot 2 participants 2 messages Jul 22, 2024

Subsystems (3)
lore
Share & tools
lore

syzbot

author syzbot
#0 lore raw

Hello,

syzbot found the following issue on:

HEAD commit:    2c9b3512402e Merge tag 'for-linus' of git://git.kernel.org..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=17abcfe9980000
kernel config:  https://syzkaller.appspot.com/x/.config?x=e322acd9c27a6b84
dashboard link: https://syzkaller.appspot.com/bug?extid=23786faffb695f17edaa
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
userspace arch: i386

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/c112a5c3b199/disk-2c9b3512.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/d0dae74e61d2/vmlinux-2c9b3512.xz
kernel image: https://storage.googleapis.com/syzbot-assets/54648874d89c/bzImage-2c9b3512.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:

Reported-by [email protected]

RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000

 </TASK>
Oops: general protection fault, probably for non-canonical address 0xe000013ffffffffd: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: maybe wild-memory-access in range [0x000029ffffffffe8-0x000029ffffffffef]
CPU: 0 PID: 11829 Comm: syz.1.1799 Not tainted 6.10.0-syzkaller-11185-g2c9b3512402e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
RIP: 0010:to_shadow_page arch/x86/kvm/mmu/spte.h:245 [inline]
RIP: 0010:spte_to_child_sp arch/x86/kvm/mmu/spte.h:250 [inline]
RIP: 0010:root_to_sp arch/x86/kvm/mmu/spte.h:267 [inline]
RIP: 0010:is_page_fault_stale+0xc4/0x530 arch/x86/kvm/mmu/mmu.c:4517
Code: e9 00 01 00 00 48 b8 ff ff ff ff ff 00 00 00 48 21 c3 48 c1 e3 06 49 bc 28 00 00 00 00 ea ff ff 49 01 dc 4c 89 e0 48 c1 e8 03 <42> 80 3c 28 00 74 08 4c 89 e7 e8 6d b7 d8 00 4d 8b 2c 24 31 ff 4c
RSP: 0018:ffffc9000fc6f6f0 EFLAGS: 00010202
RAX: 0000053ffffffffd RBX: 00003fffffffffc0 RCX: ffff88806a8bda00
RDX: 0000000000000000 RSI: 000fffffffffffff RDI: 00000000000129d3
RBP: 00000000000129d3 R08: ffffffff8120c8e0 R09: 1ffff920005e6c00
R10: dffffc0000000000 R11: fffff520005e6c01 R12: 000029ffffffffe8
R13: dffffc0000000000 R14: ffffc9000fc6f800 R15: ffff88807cbed000
FS:  0000000000000000(0000) GS:ffff8880b9400000(0063) knlGS:00000000f5d46b40
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 00000000576d24c0 CR3: 000000007d930000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 kvm_tdp_mmu_page_fault arch/x86/kvm/mmu/mmu.c:4662 [inline]
 kvm_tdp_page_fault+0x25c/0x320 arch/x86/kvm/mmu/mmu.c:4693
 kvm_mmu_do_page_fault+0x589/0xca0 arch/x86/kvm/mmu/mmu_internal.h:323
 kvm_tdp_map_page arch/x86/kvm/mmu/mmu.c:4715 [inline]
 kvm_arch_vcpu_pre_fault_memory+0x2db/0x5a0 arch/x86/kvm/mmu/mmu.c:4760
 kvm_vcpu_pre_fault_memory+0x24c/0x4b0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:4418
 kvm_vcpu_ioctl+0xa47/0xea0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:4648
 kvm_vcpu_compat_ioctl+0x242/0x450 arch/x86/kvm/../../../virt/kvm/kvm_main.c:4700
 __do_compat_sys_ioctl fs/ioctl.c:1007 [inline]
 __se_compat_sys_ioctl+0x51c/0xca0 fs/ioctl.c:950
 do_syscall_32_irqs_on arch/x86/entry/common.c:165 [inline]
 __do_fast_syscall_32+0xb4/0x110 arch/x86/entry/common.c:386
 do_fast_syscall_32+0x34/0x80 arch/x86/entry/common.c:411
 entry_SYSENTER_compat_after_hwframe+0x84/0x8e
RIP: 0023:0xf7f92579
Code: b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 002b:00000000f5d4656c EFLAGS: 00000206 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00000000c040aed5
RDX: 0000000020000040 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:to_shadow_page arch/x86/kvm/mmu/spte.h:245 [inline]
RIP: 0010:spte_to_child_sp arch/x86/kvm/mmu/spte.h:250 [inline]
RIP: 0010:root_to_sp arch/x86/kvm/mmu/spte.h:267 [inline]
RIP: 0010:is_page_fault_stale+0xc4/0x530 arch/x86/kvm/mmu/mmu.c:4517
Code: e9 00 01 00 00 48 b8 ff ff ff ff ff 00 00 00 48 21 c3 48 c1 e3 06 49 bc 28 00 00 00 00 ea ff ff 49 01 dc 4c 89 e0 48 c1 e8 03 <42> 80 3c 28 00 74 08 4c 89 e7 e8 6d b7 d8 00 4d 8b 2c 24 31 ff 4c
RSP: 0018:ffffc9000fc6f6f0 EFLAGS: 00010202
RAX: 0000053ffffffffd RBX: 00003fffffffffc0 RCX: ffff88806a8bda00
RDX: 0000000000000000 RSI: 000fffffffffffff RDI: 00000000000129d3
RBP: 00000000000129d3 R08: ffffffff8120c8e0 R09: 1ffff920005e6c00
R10: dffffc0000000000 R11: fffff520005e6c01 R12: 000029ffffffffe8
R13: dffffc0000000000 R14: ffffc9000fc6f800 R15: ffff88807cbed000
FS:  0000000000000000(0000) GS:ffff8880b9400000(0063) knlGS:00000000f5d46b40
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 00000000576d24c0 CR3: 000000007d930000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
----------------
Code disassembly (best guess):
   0:	e9 00 01 00 00       	jmp    0x105
   5:	48 b8 ff ff ff ff ff 	movabs $0xffffffffff,%rax
   c:	00 00 00
   f:	48 21 c3             	and    %rax,%rbx
  12:	48 c1 e3 06          	shl    $0x6,%rbx
  16:	49 bc 28 00 00 00 00 	movabs $0xffffea0000000028,%r12
  1d:	ea ff ff
  20:	49 01 dc             	add    %rbx,%r12
  23:	4c 89 e0             	mov    %r12,%rax
  26:	48 c1 e8 03          	shr    $0x3,%rax
* 2a:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1) <-- trapping instruction
  2f:	74 08                	je     0x39
  31:	4c 89 e7             	mov    %r12,%rdi
  34:	e8 6d b7 d8 00       	call   0xd8b7a6
  39:	4d 8b 2c 24          	mov    (%r12),%r13
  3d:	31 ff                	xor    %edi,%edi
  3f:	4c                   	rex.WR

---
This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at [email protected].

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

#1 lore raw
Quoted 28 lines · syzbot
Oops: general protection fault, probably for non-canonical address 0xe000013ffffffffd: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: maybe wild-memory-access in range [0x000029ffffffffe8-0x000029ffffffffef]
CPU: 0 PID: 11829 Comm: syz.1.1799 Not tainted 6.10.0-syzkaller-11185-g2c9b3512402e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
RIP: 0010:to_shadow_page arch/x86/kvm/mmu/spte.h:245 [inline]
RIP: 0010:spte_to_child_sp arch/x86/kvm/mmu/spte.h:250 [inline]
RIP: 0010:root_to_sp arch/x86/kvm/mmu/spte.h:267 [inline]
RIP: 0010:is_page_fault_stale+0xc4/0x530 arch/x86/kvm/mmu/mmu.c:4517
Code: e9 00 01 00 00 48 b8 ff ff ff ff ff 00 00 00 48 21 c3 48 c1 e3 06 49 bc 28 00 00 00 00 ea ff ff 49 01 dc 4c 89 e0 48 c1 e8 03 <42> 80 3c 28 00 74 08 4c 89 e7 e8 6d b7 d8 00 4d 8b 2c 24 31 ff 4c
RSP: 0018:ffffc9000fc6f6f0 EFLAGS: 00010202
RAX: 0000053ffffffffd RBX: 00003fffffffffc0 RCX: ffff88806a8bda00
RDX: 0000000000000000 RSI: 000fffffffffffff RDI: 00000000000129d3
RBP: 00000000000129d3 R08: ffffffff8120c8e0 R09: 1ffff920005e6c00
R10: dffffc0000000000 R11: fffff520005e6c01 R12: 000029ffffffffe8
R13: dffffc0000000000 R14: ffffc9000fc6f800 R15: ffff88807cbed000
FS:  0000000000000000(0000) GS:ffff8880b9400000(0063) knlGS:00000000f5d46b40
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 00000000576d24c0 CR3: 000000007d930000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 kvm_tdp_mmu_page_fault arch/x86/kvm/mmu/mmu.c:4662 [inline]
 kvm_tdp_page_fault+0x25c/0x320 arch/x86/kvm/mmu/mmu.c:4693
 kvm_mmu_do_page_fault+0x589/0xca0 arch/x86/kvm/mmu/mmu_internal.h:323
 kvm_tdp_map_page arch/x86/kvm/mmu/mmu.c:4715 [inline]
 kvm_arch_vcpu_pre_fault_memory+0x2db/0x5a0 arch/x86/kvm/mmu/mmu.c:4760
 kvm_vcpu_pre_fault_memory+0x24c/0x4b0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:4418
Quoted 8 lines
 kvm_vcpu_ioctl+0xa47/0xea0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:4648
 kvm_vcpu_compat_ioctl+0x242/0x450 arch/x86/kvm/../../../virt/kvm/kvm_main.c:4700
 __do_compat_sys_ioctl fs/ioctl.c:1007 [inline]
 __se_compat_sys_ioctl+0x51c/0xca0 fs/ioctl.c:950
 do_syscall_32_irqs_on arch/x86/entry/common.c:165 [inline]
 __do_fast_syscall_32+0xb4/0x110 arch/x86/entry/common.c:386
 do_fast_syscall_32+0x34/0x80 arch/x86/entry/common.c:411
 entry_SYSENTER_compat_after_hwframe+0x84/0x8e

The amount sanitizer code in play makes it difficult to read the assembly, but unless I'm misreading things the explosion happens on

	return (struct kvm_mmu_page *)page_private(page);

which suggests that vcpu->arch.mmu->root.hpa is garbage. Lo and behold! Just before the explosion, there's a malloc() injection during kvm_mmu_load().

Not sure why syzbot can't get a repro, but I'm pretty confident the bug is that kvm_arch_vcpu_pre_fault_memory() doesn't check the result of kvm_mmu_reload().

I'll send this after a bit of testing:

arch/x86/kvm/mmu/mmu.c +3 -1
index 901be9e420a4..ee516baf3a31 100644
@@ -4747,7 +4747,9 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,
         * reload is efficient when called repeatedly, so we can do it on
         * every iteration.
         */
-       kvm_mmu_reload(vcpu);
+       r = kvm_mmu_reload(vcpu);
+       if (r)
+               return r;
 
        if (kvm_arch_has_private_mem(vcpu->kvm) &&
            kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(range->gpa)))
[  363.075965][T11829] FAULT_INJECTION: forcing a failure.
[  363.075965][T11829] name failslab, interval 1, probability 0, space 0, times 0
[  363.089953][   T53] vhci_hcd: release socket
[  363.094422][   T53] vhci_hcd: disconnect device
[  363.117979][T11829] CPU: 0 PID: 11829 Comm: syz.1.1799 Not tainted 6.10.0-syzkaller-11185-g2c9b3512402e #0
[  363.127841][T11829] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
[  363.137917][T11829] Call Trace:
[  363.141207][T11829]  <TASK>
[  363.144137][T11829]  dump_stack_lvl+0x241/0x360
[  363.148907][T11829]  ? __pfx_dump_stack_lvl+0x10/0x10
[  363.154103][T11829]  ? __pfx__printk+0x10/0x10
[  363.158689][T11829]  ? validate_chain+0x11e/0x5900
[  363.163722][T11829]  should_fail_ex+0x3b0/0x4e0
[  363.168410][T11829]  should_failslab+0x9/0x20
[  363.172915][T11829]  __kmalloc_node_noprof+0xdf/0x440
[  363.178111][T11829]  ? __kvmalloc_node_noprof+0x72/0x190
[  363.183574][T11829]  __kvmalloc_node_noprof+0x72/0x190
[  363.188869][T11829]  __kvm_mmu_topup_memory_cache+0x4d9/0x6b0
[  363.194801][T11829]  kvm_mmu_load+0x115/0x26e0
[  363.199416][T11829]  ? __asan_memset+0x23/0x50
[  363.204038][T11829]  ? vmx_vcpu_pi_load+0x13b/0x8c0
[  363.209381][T11829]  ? __pfx_kvm_mmu_load+0x10/0x10
[  363.214447][T11829]  ? __lock_acquire+0x137a/0x2040
[  363.219511][T11829]  kvm_arch_vcpu_pre_fault_memory+0x4c0/0x5a0
[  363.225862][T11829]  ? __pfx_kvm_arch_vcpu_load+0x10/0x10
[  363.231674][T11829]  ? __pfx_kvm_arch_vcpu_pre_fault_memory+0x10/0x10
[  363.238267][T11829]  ? __pfx_lock_release+0x10/0x10
[  363.243305][T11829]  kvm_vcpu_pre_fault_memory+0x24c/0x4b0
[  363.248941][T11829]  ? kvm_vcpu_pre_fault_memory+0x16a/0x4b0
[  363.254748][T11829]  kvm_vcpu_ioctl+0xa47/0xea0
[  363.259422][T11829]  ? __lock_acquire+0x137a/0x2040
[  363.264461][T11829]  ? __pfx_kvm_vcpu_ioctl+0x10/0x10
[  363.269663][T11829]  ? __pfx_tomoyo_path_number_perm+0x10/0x10
[  363.275672][T11829]  kvm_vcpu_compat_ioctl+0x242/0x450
[  363.280962][T11829]  ? __pfx_kvm_vcpu_compat_ioctl+0x10/0x10
[  363.286780][T11829]  ? __fget_files+0x3f6/0x470
[  363.291474][T11829]  ? bpf_lsm_file_ioctl_compat+0x9/0x10
[  363.297023][T11829]  ? security_file_ioctl_compat+0x87/0xb0
[  363.302765][T11829]  __se_compat_sys_ioctl+0x51c/0xca0
[  363.308053][T11829]  ? __pfx___se_compat_sys_ioctl+0x10/0x10
[  363.313859][T11829]  ? lockdep_hardirqs_on_prepare+0x43d/0x780
[  363.319981][T11829]  ? __pfx_lockdep_hardirqs_on_prepare+0x10/0x10
[  363.326322][T11829]  ? syscall_enter_from_user_mode_prepare+0x7f/0xe0
[  363.332919][T11829]  ? lockdep_hardirqs_on+0x99/0x150
[  363.338116][T11829]  __do_fast_syscall_32+0xb4/0x110
[  363.343221][T11829]  ? exc_page_fault+0x590/0x8c0
[  363.348095][T11829]  do_fast_syscall_32+0x34/0x80
[  363.352937][T11829]  entry_SYSENTER_compat_after_hwframe+0x84/0x8e