2.6.18: Kernel BUG at mm/rmap.c:522
Hi, I can readily reproduce this with 2.6.18 doing 4 simultanous kernel compiles on two disks to load test a P4 3.2 HT with 2GB. I have SMP and SMT scheduling enabled, and the 4GB memory option. Here is output with CONFIG_DEBUG_VM enabled followed by another crash before CONFIG_DEBUG_VM was enabled.
[*ROOT* hen /usr/src/linux-2.6.18 14 ] uname -a
Linux hen.igconcepts.com 2.6.18 #2 SMP Tue Oct 10 11:46:01 CDT 2006 i686 i686 i386 GNU/Linux
[*ROOT* hen /usr/src/linux-2.6.18 15 ] gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux Thread model: posix
gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)
----------------
Oct 11 04:53:35 hen kernel: VM: killing process cc1 Oct 11 04:53:35 hen kernel: swap_free: Unused swap offset entry 00004000 Oct 11 04:53:35 hen kernel: Eeek! page_mapcount(page) went negative! (-1) Oct 11 04:53:35 hen kernel: page->flags = c0080014 Oct 11 04:53:35 hen kernel: page->count = 0 Oct 11 04:53:35 hen kernel: page->mapping = 00000000 Oct 11 04:53:35 hen kernel: ------------[ cut here ]------------ Oct 11 04:53:35 hen kernel: kernel BUG at mm/rmap.c:522! Oct 11 04:53:35 hen kernel: invalid opcode: 0000 [#1] Oct 11 04:53:35 hen kernel: SMP DEBUG_PAGEALLOC Oct 11 04:53:35 hen kernel: CPU: 1 Oct 11 04:53:35 hen kernel: EIP: 0060:[<c01494c2>] Not tainted VLI Oct 11 04:53:35 hen kernel: EFLAGS: 00210286 (2.6.18 #2) Oct 11 04:53:35 hen kernel: EIP is at page_remove_rmap+0x44/0xb9 Oct 11 04:53:35 hen kernel: eax: ffffffff ebx: c1b5cd80 ecx: ffffffff edx: 00200046 Oct 11 04:53:35 hen kernel: esi: 403fc000 edi: c1b5cd80 ebp: c20108a0 esp: d85e1ea8 Oct 11 04:53:35 hen kernel: ds: 007b es: 007b ss: 0068 Oct 11 04:53:35 hen kernel: Process cc1 (pid: 23624, ti=d85e0000 task=e817c030 task.ti=d85e0000) Oct 11 04:53:35 hen kernel: Stack: c0390116 00000000 d80adff0 c0143249 5ae6c067 c13015ac ffffff14 ffffffff Oct 11 04:53:35 hen kernel: eceade40 f3c50b00 40400000 d8929400 40400000 40310000 c014347e 40310000 Oct 11 04:53:35 hen kernel: 40400000 d85e1f3c 00000000 4046ffff d8929400 d8929400 f3c50b00 c20108a0 Oct 11 04:53:35 hen kernel: Call Trace: Oct 11 04:53:35 hen kernel: [<c0143249>] zap_pte_range+0x15b/0x273 Oct 11 04:53:35 hen kernel: [<c014347e>] unmap_page_range+0x11d/0x13f Oct 11 04:53:35 hen kernel: [<c0143569>] unmap_vmas+0xc9/0x1bf Oct 11 04:53:35 hen kernel: [<c014797c>] exit_mmap+0x79/0xf3 Oct 11 04:53:35 hen kernel: [<c0117d41>] mmput+0x33/0x91 Oct 11 04:53:35 hen kernel: [<c011c9e8>] do_exit+0x110/0x3c8 Oct 11 04:53:35 hen kernel: [<c0111a65>] do_page_fault+0x13b/0x54c Oct 11 04:53:35 hen kernel: [<c0147626>] do_munmap+0xea/0x113 Oct 11 04:53:35 hen kernel: [<c011cd47>] do_group_exit+0x87/0xa7 Oct 11 04:53:35 hen kernel: [<c0102ca7>] syscall_call+0x7/0xb Oct 11 04:53:35 hen kernel: Code: 94 00 00 00 8b 43 08 83 c0 01 78 29 8b 43 08 83 c0 01 78 17 31 d2 89 d8 f6 43 10 01 0f 94 c2 8b 5c 24 08 83 c4 0c e9 92 7b ff ff <0f> 0b 0a 02 d4 00 39 c0 eb df 8b 43 08 c7 04 24 e0 0c 39 c0 83 Oct 11 04:53:35 hen kernel: EIP: [<c01494c2>] page_remove_rmap+0x44/0xb9 SS:ESP 0068:d85e1ea8 Oct 11 04:53:35 hen kernel: <1>Fixing recursive fault but reboot is needed! Oct 11 04:54:31 hen kernel: Bad page state in process 'tripwire' Oct 11 04:54:31 hen kernel: page:c1b5cd80 flags:0xc0000014 mapping:00000000 mapcount:-1 count:0 Oct 11 04:54:31 hen kernel: Trying to fix it up, but a reboot is needed Oct 11 04:54:31 hen kernel: Backtrace: Oct 11 04:54:31 hen kernel: [<c013afe4>] bad_page+0x60/0x8f Oct 11 04:54:31 hen kernel: [<c013b5bd>] prep_new_page+0x197/0x1a4 Oct 11 04:54:31 hen kernel: [<c013bb7a>] buffered_rmqueue+0xda/0x152 Oct 11 04:54:31 hen kernel: [<c013bcfc>] get_page_from_freelist+0x6d/0x98 Oct 11 04:54:31 hen kernel: [<c013bd71>] __alloc_pages+0x4a/0x2b0 Oct 11 04:54:31 hen kernel: [<c03664bb>] __wait_on_bit_lock+0x41/0x5e Oct 11 04:54:31 hen kernel: [<c0136e2c>] sync_page+0x0/0x4c Oct 11 04:54:31 hen kernel: [<c013db15>] __do_page_cache_readahead+0x110/0x14f Oct 11 04:54:31 hen kernel: [<c0138861>] filemap_nopage+0x312/0x368 Oct 11 04:54:31 hen kernel: [<c0144cb4>] do_no_page+0x6f/0x295 Oct 11 04:54:31 hen kernel: [<c0145049>] __handle_mm_fault+0xec/0x1f9 Oct 11 04:54:31 hen kernel: [<c0111a65>] do_page_fault+0x13b/0x54c Oct 11 04:54:32 hen kernel: [<c011192a>] do_page_fault+0x0/0x54c Oct 11 04:54:32 hen kernel: [<c010374d>] error_code+0x39/0x40
----------------
Another crash from a day earlier before enabling DEBUG_VM
Oct 10 05:19:43 hen kernel: VM: killing process cc1 Oct 10 05:19:43 hen kernel: swap_free: Unused swap offset entry 00002000 Oct 10 05:19:56 hen kernel: swap_free: Unused swap offset entry 00000400 Oct 10 05:20:13 hen kernel: ------------[ cut here ]------------ Oct 10 05:20:13 hen kernel: kernel BUG at mm/rmap.c:522! Oct 10 05:20:13 hen kernel: invalid opcode: 0000 [#1] Oct 10 05:20:13 hen kernel: SMP Oct 10 05:20:13 hen kernel: CPU: 0 Oct 10 05:20:13 hen kernel: EIP: 0060:[<c0148ffc>] Not tainted VLI Oct 10 05:20:13 hen kernel: EFLAGS: 00210286 (2.6.18 #1) Oct 10 05:20:13 hen kernel: EIP is at page_remove_rmap+0x2a/0x35 Oct 10 05:20:13 hen kernel: eax: ffffffff ebx: d80df18c ecx: c1ffc800 edx: 0007fe40 Oct 10 05:20:13 hen kernel: esi: 08063000 edi: c1ffc800 ebp: c200be20 esp: d8e35dd8 Oct 10 05:20:13 hen kernel: ds: 007b es: 007b ss: 0068 Oct 10 05:20:13 hen kernel: Process sh (pid: 23162, ti=d8e34000 task=db85a030 task.ti=d8e34000) Oct 10 05:20:13 hen kernel: Stack: c0142f1d 7fe40025 c1301bec 00000000 ffffffe4 f264d200 f7a975c0 080dc000 Oct 10 05:20:13 hen kernel: d8b33080 080dc000 08048000 c0143152 08048000 080dc000 d8e35e60 00000000 Oct 10 05:20:13 hen kernel: 080dbfff d8b33080 d8b33080 f7a975c0 c200be20 d8e35e84 f7a975c0 08048000 Oct 10 05:20:13 hen kernel: Call Trace: Oct 10 05:20:13 hen kernel: [<c0142f1d>] zap_pte_range+0x15b/0x273 Oct 10 05:20:13 hen kernel: [<c0143152>] unmap_page_range+0x11d/0x13f Oct 10 05:20:13 hen kernel: [<c014323d>] unmap_vmas+0xc9/0x1bf Oct 10 05:20:13 hen kernel: [<c0147605>] exit_mmap+0x79/0xf3 Oct 10 05:20:13 hen kernel: [<c0117c29>] mmput+0x33/0x91 Oct 10 05:20:13 hen kernel: [<c011c8c8>] do_exit+0x110/0x3c8 Oct 10 05:20:13 hen kernel: [<c011cc27>] do_group_exit+0x87/0xa7 Oct 10 05:20:13 hen kernel: [<c0125e6f>] get_signal_to_deliver+0x200/0x2e5 Oct 10 05:20:13 hen kernel: [<c0102a2f>] do_signal+0x6a/0x180 Oct 10 05:20:13 hen kernel: [<c0124490>] signal_wake_up+0x28/0x36 Oct 10 05:20:13 hen kernel: [<c012498b>] specific_send_sig_info+0xaa/0xc3 Oct 10 05:20:13 hen kernel: [<c0124a15>] force_sig_info+0x71/0x8b Oct 10 05:20:13 hen kernel: [<c010471b>] do_general_protection+0xa7/0x188 Oct 10 05:20:13 hen kernel: [<c0104674>] do_general_protection+0x0/0x188 Oct 10 05:20:13 hen kernel: [<c0102b8a>] do_notify_resume+0x45/0x47 Oct 10 05:20:13 hen kernel: [<c0102d32>] work_notifysig+0x13/0x19 Oct 10 05:20:13 hen kernel: Code: ff 89 c1 b8 ff ff ff ff f0 01 41 08 0f 98 c0 84 c0 74 22 8b 41 08 83 c0 01 78 10 31 d2 89 c8 f6 41 10 01 0f 94 c2 e9 34 7d ff ff <0f> 0b 0a 02 94 ff 38 c0 eb e6 c3 55 57 56 89 d6 53 89 cb 31 c9 Oct 10 05:20:13 hen kernel: EIP: [<c0148ffc>] page_remove_rmap+0x2a/0x35 SS:ESP 0068:d8e35dd8 Oct 10 05:20:13 hen kernel: <1>Fixing recursive fault but reboot is needed!
Quoted 5 lines · On Wed, 11 Oct 2006 11:07:40 -0500 Michael Harris <[email protected]>…
Hi, I can readily reproduce this with 2.6.18 doing 4 simultanous kernel compiles on two disks to load test a P4 3.2 HT with 2GB. I have SMP and SMT scheduling enabled, and the 4GB memory option. Here is output with
CONFIG_DEBUG_VMenabled followed by another crash beforeCONFIG_DEBUG_VMwas enabled.
Repeatable. That's good news.
Quoted 11 lines
[*ROOT* hen
/usr/src/linux-2.6.1814 ] uname -a
Linux hen.igconcepts.com 2.6.18 #2 SMP Tue Oct 10 11:46:01 CDT 2006 i686 i686 i386 GNU/Linux
[*ROOT* hen/usr/src/linux-2.6.1815 ] gcc -v
Reading specs from/usr/lib/gcc-lib/i386-redhat-linux/3.2.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux Thread model: posix
gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)----------------
Quoted 8 lines
Oct 11 04:53:35 hen kernel: VM: killing process cc1 Oct 11 04:53:35 hen kernel: swap_free: Unused swap offset entry 00004000 Oct 11 04:53:35 hen kernel: Eeek! page_mapcount(page) went negative! (-1) Oct 11 04:53:35 hen kernel: page->flags = c0080014 Oct 11 04:53:35 hen kernel: page->count = 0 Oct 11 04:53:35 hen kernel: page->mapping = 00000000 Oct 11 04:53:35 hen kernel: ------------[ cut here ]------------ Oct 11 04:53:35 hen kernel: kernel BUG at mm/rmap.c:522!
Does that machine run any earlier kernels OK? If so, which?
Thanks.
Quoted 11 lines · Andrew Morton
Quoted 9 lines
----------------
Oct 11 04:53:35 hen kernel: VM: killing process cc1 Oct 11 04:53:35 hen kernel: swap_free: Unused swap offset entry 00004000 Oct 11 04:53:35 hen kernel: Eeek! page_mapcount(page) went negative! (-1) Oct 11 04:53:35 hen kernel: page->flags = c0080014 Oct 11 04:53:35 hen kernel: page->count = 0 Oct 11 04:53:35 hen kernel: page->mapping = 00000000 Oct 11 04:53:35 hen kernel: ------------[ cut here ]------------ Oct 11 04:53:35 hen kernel: kernel BUG at mm/rmap.c:522!Does that machine run any earlier kernels OK? If so, which?
FWIW, I've seen a bunch of reports of this being triggered in Fedora bugzilla which have had the nvidia module loaded. Is that the case here ?
Dave
Signature · 1 line
http://www.codemonkey.org.uk
Quoted 7 lines · Michael Harris
Hi, I can readily reproduce this with 2.6.18 doing 4 simultanous kernel compiles on two disks to load test a P4 3.2 HT with 2GB. I have SMP and SMT scheduling enabled, and the 4GB memory option. Here is output with
CONFIG_DEBUG_VMenabled followed by another crash beforeCONFIG_DEBUG_VMwas enabled.Oct 11 04:53:35 hen kernel: swap_free: Unused swap offset entry 00004000 Oct 11 04:53:35 hen kernel: Eeek! page_mapcount(page) went negative! (-1) Oct 11 04:53:35 hen kernel: page->flags = c0080014 Oct 11 04:53:35 hen kernel: page->count = 0 Oct 11 04:53:35 hen kernel: page->mapping = 00000000
Hmm, this is a new one. The page is free and not reserved, wheras we are used to seeing them reserved here.
Quoted 7 lines
Oct 11 04:54:31 hen kernel: Bad page state in process 'tripwire' Oct 11 04:54:31 hen kernel: page:c1b5cd80 flags:0xc0000014 mapping:00000000 mapcount:-1 count:0Another crash from a day earlier before enabling
DEBUG_VMOct 10 05:19:43 hen kernel: VM: killing process cc1 Oct 10 05:19:43 hen kernel: swap_free: Unused swap offset entry 00002000 Oct 10 05:19:56 hen kernel: swap_free: Unused swap offset entry 00000400
These unused swap offset entry messages seem to indicate extensive memory corruption in your page tables. Probably bad RAM, or system overheating when you load it up :(
Can you run a good memory tester like memtest86+ overnight?
Signature · 2 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 25 lines
Quoted 11 lines · Michael Harris
Hi, I can readily reproduce this with 2.6.18 doing 4 simultanous kernel compiles on two disks to load test a P4 3.2 HT with 2GB. I have SMP and SMT scheduling enabled, and the 4GB memory option. Here is output with
CONFIG_DEBUG_VMenabled followed by another crash beforeCONFIG_DEBUG_VMwas enabled.Oct 11 04:53:35 hen kernel: swap_free: Unused swap offset entry 00004000 Oct 11 04:53:35 hen kernel: Eeek! page_mapcount(page) went negative! (-1) Oct 11 04:53:35 hen kernel: page->flags = c0080014 Oct 11 04:53:35 hen kernel: page->count = 0 Oct 11 04:53:35 hen kernel: page->mapping = 00000000Hmm, this is a new one. The page is free and not reserved, wheras we are used to seeing them reserved here.
Quoted 8 lines
Oct 11 04:54:31 hen kernel: Bad page state in process 'tripwire' Oct 11 04:54:31 hen kernel: page:c1b5cd80 flags:0xc0000014mapping:00000000 mapcount:-1 count:0
Another crash from a day earlier before enabling
DEBUG_VMOct 10 05:19:43 hen kernel: VM: killing process cc1 Oct 10 05:19:43 hen kernel: swap_free: Unused swap offset entry 00002000 Oct 10 05:19:56 hen kernel: swap_free: Unused swap offset entry 00000400
Quoted 5 lines
These unused swap offset entry messages seem to indicate extensive memory corruption in your page tables. Probably bad RAM, or system overheating when you load it up :(
Can you run a good memory tester like memtest86+ overnight?
Hi, I think this is the case, a stick of ram gone bad. It had worked testing under 2.4 but coincidentally failed about the time I upgraded to 2.6. memtest86 uncovered it at once. Sorry for the trouble and thanks for the help.
Mike
Quoted 6 lines · Michael Harris
Quoted 6 lines
Michael Harris wrote:
These unused swap offset entry messages seem to indicate extensive memory corruption in your page tables. Probably bad RAM, or system overheating when you load it up :(
Quoted 7 lines
Quoted 1 line
Can you run a good memory tester like memtest86+ overnight?
Hi, I think this is the case, a stick of ram gone bad. It had worked testing under 2.4 but coincidentally failed about the time I upgraded to 2.6. memtest86 uncovered it at once. Sorry for the trouble and thanks
for the help.
No problem, thanks for reporting.
Signature · 2 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com