Skip to contentSkip to conversation

kernel bug when performing heavy IO operations

DISCUSSION dianlujitao 2 participants 2 messages Aug 25, 2023 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Dear developers,

When the IO load is heavy (compiling AOSP in my case), there's a chance to crash the kernel, the only way to recover is to perform a hard reset. Logs look like follows:

8月 25 13:52:23 arch-pc kernel: BUG: Bad page map in process tmux: client  pte:8000000462500025 pmd:b99c98067
8月 25 13:52:23 arch-pc kernel: page:00000000460fa108 refcount:4 mapcount:-256 mapping:00000000612a1864 index:0x16 pfn:0x462500
8月 25 13:52:23 arch-pc kernel: memcg:ffff8a1056ed0000
8月 25 13:52:23 arch-pc kernel: aops:btrfs_aops [btrfs] ino:9c4635 dentry name:"locale-archive"
8月 25 13:52:23 arch-pc kernel: flags: 0x2ffff5800002056(referenced|uptodate|lru|workingset|private|node=0|zone=2|lastcpupid=0xffff)
8月 25 13:52:23 arch-pc kernel: page_type: 0xfffffeff(offline)
8月 25 13:52:23 arch-pc kernel: raw: 02ffff5800002056 ffffe6e210c05248 ffffe6e20e714dc8 ffff8a10472a8c70
8月 25 13:52:23 arch-pc kernel: raw: 0000000000000016 0000000000000001 00000003fffffeff ffff8a1056ed0000
8月 25 13:52:23 arch-pc kernel: page dumped because: bad pte
8月 25 13:52:23 arch-pc kernel: addr:00007f5fc9816000 vm_flags:08000071 anon_vma:0000000000000000 mapping:ffff8a10472a8c70 index:16
8月 25 13:52:23 arch-pc kernel: file:locale-archive fault:filemap_fault mmap:btrfs_file_mmap [btrfs] read_folio:btrfs_read_folio [btrfs] 8月 25 13:52:23 arch-pc kernel: CPU: 40 PID: 2033787 Comm: tmux: client Tainted: G           OE      6.4.11-zen2-1-zen #1 a571467d6effd6120b1e64d2f88f90c58106da17 8月 25 13:52:23 arch-pc kernel: Hardware name: JGINYUE X99-8D3/2.5G Server/X99-8D3/2.5G Server, BIOS 5.11 06/30/2022
8月 25 13:52:23 arch-pc kernel: Call Trace:
8月 25 13:52:23 arch-pc kernel:  <TASK>
8月 25 13:52:23 arch-pc kernel:  dump_stack_lvl+0x47/0x60
8月 25 13:52:23 arch-pc kernel:  print_bad_pte+0x194/0x250
8月 25 13:52:23 arch-pc kernel:  ? page_remove_rmap+0x8d/0x260
8月 25 13:52:23 arch-pc kernel:  unmap_page_range+0xbb1/0x20f0
8月 25 13:52:23 arch-pc kernel:  unmap_vmas+0x142/0x220
8月 25 13:52:23 arch-pc kernel:  exit_mmap+0xe4/0x350
8月 25 13:52:23 arch-pc kernel:  mmput+0x5f/0x140
8月 25 13:52:23 arch-pc kernel:  do_exit+0x31f/0xbc0
8月 25 13:52:23 arch-pc kernel:  do_group_exit+0x31/0x80
8月 25 13:52:23 arch-pc kernel:  __x64_sys_exit_group+0x18/0x20
8月 25 13:52:23 arch-pc kernel:  do_syscall_64+0x60/0x90
8月 25 13:52:23 arch-pc kernel: entry_SYSCALL_64_after_hwframe+0x77/0xe1
8月 25 13:52:23 arch-pc kernel: RIP: 0033:0x7f5fca0da14d
8月 25 13:52:23 arch-pc kernel: Code: Unable to access opcode bytes at 0x7f5fca0da123.
8月 25 13:52:23 arch-pc kernel: RSP: 002b:00007fff54a44358 EFLAGS: 00000206 ORIG_RAX: 00000000000000e7
8月 25 13:52:23 arch-pc kernel: RAX: ffffffffffffffda RBX: 00007f5fca23ffa8 RCX: 00007f5fca0da14d
8月 25 13:52:23 arch-pc kernel: RDX: 00000000000000e7 RSI: fffffffffffffeb8 RDI: 0000000000000000
8月 25 13:52:23 arch-pc kernel: RBP: 0000000000000002 R08: 00007fff54a442f8 R09: 00007fff54a4421f
8月 25 13:52:23 arch-pc kernel: R10: 00007fff54a44130 R11: 0000000000000206 R12: 0000000000000000
8月 25 13:52:23 arch-pc kernel: R13: 0000000000000000 R14: 00007f5fca23e680 R15: 00007f5fca23ffc0
8月 25 13:52:23 arch-pc kernel:  </TASK>
8月 25 13:52:23 arch-pc kernel: Disabling lock debugging due to kernel taint

Full log is available at https://fars.ee/HJw3
Notice that the issue is introduced by linux kernel released in recent months.

I also reported the issue at https://bugzilla.kernel.org/show_bug.cgi?id=217823

#1 lore raw
Quoted 64 lines · dianlujitao

Dear developers,

When the IO load is heavy (compiling AOSP in my case), there's a chance to crash the kernel, the only way to recover is to perform a hard reset. Logs look like follows:

8月 25 13:52:23 arch-pc kernel: BUG: Bad page map in process tmux:
client  pte:8000000462500025 pmd:b99c98067
8月 25 13:52:23 arch-pc kernel: page:00000000460fa108 refcount:4
mapcount:-256 mapping:00000000612a1864 index:0x16 pfn:0x462500
8月 25 13:52:23 arch-pc kernel: memcg:ffff8a1056ed0000
8月 25 13:52:23 arch-pc kernel: aops:btrfs_aops [btrfs] ino:9c4635
dentry name:"locale-archive"
8月 25 13:52:23 arch-pc kernel: flags:
0x2ffff5800002056(referenced|uptodate|lru|workingset|private|node=0|zone=2|lastcpupid=0xffff) 8月 25 13:52:23 arch-pc kernel: page_type: 0xfffffeff(offline)
8月 25 13:52:23 arch-pc kernel: raw: 02ffff5800002056 ffffe6e210c05248 ffffe6e20e714dc8 ffff8a10472a8c70
8月 25 13:52:23 arch-pc kernel: raw: 0000000000000016 0000000000000001 00000003fffffeff ffff8a1056ed0000
8月 25 13:52:23 arch-pc kernel: page dumped because: bad pte
8月 25 13:52:23 arch-pc kernel: addr:00007f5fc9816000 vm_flags:08000071 anon_vma:0000000000000000 mapping:ffff8a10472a8c70 index:16
8月 25 13:52:23 arch-pc kernel: file:locale-archive fault:filemap_fault mmap:btrfs_file_mmap [btrfs] read_folio:btrfs_read_folio [btrfs]
8月 25 13:52:23 arch-pc kernel: CPU: 40 PID: 2033787 Comm: tmux: client

Tainted: G           OE      6.4.11-zen2-1-zen #1

a571467d6effd6120b1e64d2f88f90c58106da17
8月 25 13:52:23 arch-pc kernel: Hardware name: JGINYUE X99-8D3/2.5G Server/X99-8D3/2.5G Server, BIOS 5.11 06/30/2022
8月 25 13:52:23 arch-pc kernel: Call Trace:
8月 25 13:52:23 arch-pc kernel:  <TASK>
8月 25 13:52:23 arch-pc kernel:  dump_stack_lvl+0x47/0x60
8月 25 13:52:23 arch-pc kernel:  print_bad_pte+0x194/0x250
8月 25 13:52:23 arch-pc kernel:  ? page_remove_rmap+0x8d/0x260 8月 25 13:52:23 arch-pc kernel:  unmap_page_range+0xbb1/0x20f0 8月 25 13:52:23 arch-pc kernel:  unmap_vmas+0x142/0x220
8月 25 13:52:23 arch-pc kernel:  exit_mmap+0xe4/0x350
8月 25 13:52:23 arch-pc kernel:  mmput+0x5f/0x140
8月 25 13:52:23 arch-pc kernel:  do_exit+0x31f/0xbc0
8月 25 13:52:23 arch-pc kernel:  do_group_exit+0x31/0x80
8月 25 13:52:23 arch-pc kernel:  __x64_sys_exit_group+0x18/0x20 8月 25 13:52:23 arch-pc kernel:  do_syscall_64+0x60/0x90
8月 25 13:52:23 arch-pc kernel: entry_SYSCALL_64_after_hwframe+0x77/0xe1 8月 25 13:52:23 arch-pc kernel: RIP: 0033:0x7f5fca0da14d
8月 25 13:52:23 arch-pc kernel: Code: Unable to access opcode bytes at 0x7f5fca0da123.
8月 25 13:52:23 arch-pc kernel: RSP: 002b:00007fff54a44358 EFLAGS:
00000206 ORIG_RAX: 00000000000000e7
8月 25 13:52:23 arch-pc kernel: RAX: ffffffffffffffda RBX:
00007f5fca23ffa8 RCX: 00007f5fca0da14d
8月 25 13:52:23 arch-pc kernel: RDX: 00000000000000e7 RSI:
fffffffffffffeb8 RDI: 0000000000000000
8月 25 13:52:23 arch-pc kernel: RBP: 0000000000000002 R08:
00007fff54a442f8 R09: 00007fff54a4421f
8月 25 13:52:23 arch-pc kernel: R10: 00007fff54a44130 R11:
0000000000000206 R12: 0000000000000000
8月 25 13:52:23 arch-pc kernel: R13: 0000000000000000 R14:
00007f5fca23e680 R15: 00007f5fca23ffc0
8月 25 13:52:23 arch-pc kernel:  </TASK>
8月 25 13:52:23 arch-pc kernel: Disabling lock debugging due to kernel taint

Quoted 1 line

Full log is available at https://fars.ee/HJw3

Any log before the carsh?

Unfortunately the backtrace itself doesn't show a strong relationship to btrfs.
The only btrfs related part is the call back of btrfs_read_folio(), considering the BUG itself is about bad page mapping, which may or may not be btrfs' fault.

Another point is, it looks like you're using zen kernel (from Archlinux repo?), zen kernels have quite some out-of-tree patches, which can be another cause of problems.

Have you tried upstream kernels? (Arch official linux package is more than good enough for this case).

Quoted 7 lines

Notice that the issue is introduced by linux kernel released in recent months.

I also reported the issue at
https://bugzilla.kernel.org/show_bug.cgi?id=217823

For zen kernels, I strongly doubt if kernel bugzilla is the correct location to report bugs, as kernel bugzilla/btrfs mailing list are only for upstream kernels.

Thanks,
Qu