repeated Oops
Getting lots of these since attempting to upgrade past 3.8.0-rc7. I *don't* think it's a kernel issue at this point, because while older kernels (found an old 3.5.0-rc4 setup from about a year ago in my archives) seem to take longer to reach this point, they'll eventually die exactly the same way. In other words, either my old hardware is starting to go bye-bye, or something critical in userspace (libraries?) is horribly broken since the last round of package updates (Debian unstable for alpha).
(System currently powered-down. Will open the case later and go looking for clogged/bad cooling fans, cat fur, etc.)
The process running at the time will vary, but the commonality I've noticed is "lots of disk I/O". Examples: cpio, applying the 3.9.0-rc1 kernel patch (approx. 40 MB uncompressed), running "git pull" on a local kernel source repository where v3.8 was the most recent tag, the final link of vmlinux on a kernel build, and so forth.
Reminder: this is a DEC Alpha system (PWS 433au).
Unable to handle kernel paging request at virtual address 0000000000000010
cpio(4445): Oops 0
pc = [<fffffc0000315d1c>] ra = [<fffffc000031e2bc>] ps = 0007 Not tainted
pc is at process_mcheck_info+0x4c/0x320
ra is at cia_machine_check+0x9c/0xb0
v0 = 0000000000000004 t0 = 0000000000000630 t1 = 0000000000000630 t2 = 0000000000000001 t3 = fffffc0000000000 t4 = fffffc00425ede38 t5 = fffffc00425ee000 t6 = 0000000000245b15 t7 = fffffc0042dbc000 s0 = 0000000000000000 s1 = fffffc00009ce258 s2 = fffffc00422b2498 s3 = fffffc0042dbfb68 s4 = 0000000000000002 s5 = 0000000000000002 s6 = 0000000000000002 a0 = 0000000000000630 a1 = 0000000000000000 a2 = fffffc00008aeb4c a3 = 0000000000000000 a4 = fffffc0042dbfb68 a5 = fffffc0042dbfb58 t8 = 0000000000000001 t9 = 0000000000245b15 t10= fffffc00422b23b8 t11= 0000000000245b15 pv = fffffc0000315cd0 at = fffffc0042dbf878 gp = fffffc0000a0d0d8 sp = fffffc0042dbf8a0 Disabling lock debugging due to kernel taint Trace: [<fffffc000031e2bc>] cia_machine_check+0x9c/0xb0 [<fffffc000042fb40>] ext3_get_blocks_handle+0xe0/0xd00 [<fffffc0000315c70>] do_entInt+0x180/0x1e0 [<fffffc0000372704>] mempool_alloc_slab+0x24/0x40 [<fffffc0000310dc0>] ret_from_sys_call+0x0/0x10 [<fffffc00003729a0>] mempool_alloc+0x50/0x170 [<fffffc00003f18c4>] do_mpage_readpage+0x344/0x7e0 [<fffffc00004fb220>] __constant_c_memset+0x0/0x50 [<fffffc00004fb288>] loop+0x8/0x10 [<fffffc00003f1ee8>] mpage_readpages+0xf8/0x1c0 [<fffffc0000430760>] ext3_get_block+0x0/0x170 [<fffffc00004f0b3c>] radix_tree_insert+0x1ac/0x2f0 [<fffffc000036f550>] add_to_page_cache_locked+0xb0/0x180 [<fffffc00003f1eb8>] mpage_readpages+0xc8/0x1c0 [<fffffc0000430760>] ext3_get_block+0x0/0x170 [<fffffc000042c19c>] ext3_readpages+0x2c/0x40 [<fffffc00004545b0>] journal_stop+0x160/0x300 [<fffffc00004758d4>] security_file_open+0xa4/0xb0 [<fffffc000037b22c>] __do_page_cache_readahead+0x1fc/0x320 [<fffffc000037b798>] ra_submit+0x38/0x50 [<fffffc000037182c>] generic_file_aio_read+0x51c/0x800 [<fffffc00003adefc>] do_sync_read+0x9c/0x110 [<fffffc00003ae764>] vfs_read+0xb4/0x1c0 [<fffffc00004754e8>] security_file_permission+0xd8/0x110 [<fffffc00003ae204>] rw_verify_area+0x64/0x120 [<fffffc00003ae734>] vfs_read+0x84/0x1c0 [<fffffc00003ae8dc>] SyS_read+0x6c/0xc0 [<fffffc0000310da4>] entSys+0xa4/0xc0 Code: a75e0000 a53e0008 a55e0010 23de0020 6bfa8001 a55d0158 <a2910010> 261dffea
Thanks in advance for an assist in figuring out what's going on here.
--Bob
Quoted 5 lines · Bob Tracy
Getting lots of these since attempting to upgrade past 3.8.0-rc7. I *don't* think it's a kernel issue at this point, because while older kernels (found an old 3.5.0-rc4 setup from about a year ago in my archives) seem to take longer to reach this point, they'll eventually die exactly the same way.
Presumably the older kernel has worked reliably at some stage?
Quoted 3 lines
In other words, either my old hardware is starting to go bye-bye, or something critical in userspace (libraries?) is horribly broken since the last round of package updates (Debian unstable for alpha).
Recently updated Debian unstable has been working fine on my Alphas including a PWS. I have not seen the kernel paging request problems for some time now; I saw them in the past when linking huge libraries that exercised virtual memory when running kernels older than about 3.2.23.
How about the Debian supplied generic kernel? That should work (except that the radeon module may not load leaving the console in a VGA mode). Testing a stable kernel is probably a good idea at this stage.
Quoted 2 lines
(System currently powered-down. Will open the case later and go looking for clogged/bad cooling fans, cat fur, etc.)
Good idea. Check motherboard is well seated into I/O board. Check memory is all well seated. Doing that got me some extra life out of my PWS!
Good luck!
Cheers
Michael.
Quoted 60 lines
The process running at the time will vary, but the commonality I've noticed is "lots of disk I/O". Examples: cpio, applying the 3.9.0-rc1 kernel patch (approx. 40 MB uncompressed), running "git pull" on a local kernel source repository where v3.8 was the most recent tag, the final link of vmlinux on a kernel build, and so forth.
Reminder: this is a DEC Alpha system (PWS 433au).
Unable to handle kernel paging request at virtual address 0000000000000010cpio(4445): Oops 0
pc = [<fffffc0000315d1c>] ra = [<fffffc000031e2bc>] ps = 0007 Not tainted
pc is atprocess_mcheck_info+0x4c/0x320
ra is atcia_machine_check+0x9c/0xb0v0 = 0000000000000004 t0 = 0000000000000630 t1 = 0000000000000630 t2 = 0000000000000001 t3 = fffffc0000000000 t4 = fffffc00425ede38 t5 = fffffc00425ee000 t6 = 0000000000245b15 t7 = fffffc0042dbc000 s0 = 0000000000000000 s1 = fffffc00009ce258 s2 = fffffc00422b2498 s3 = fffffc0042dbfb68 s4 = 0000000000000002 s5 = 0000000000000002 s6 = 0000000000000002 a0 = 0000000000000630 a1 = 0000000000000000 a2 = fffffc00008aeb4c a3 = 0000000000000000 a4 = fffffc0042dbfb68 a5 = fffffc0042dbfb58 t8 = 0000000000000001 t9 = 0000000000245b15 t10= fffffc00422b23b8 t11= 0000000000245b15 pv = fffffc0000315cd0 at = fffffc0042dbf878 gp = fffffc0000a0d0d8 sp = fffffc0042dbf8a0 Disabling lock debugging due to kernel taint Trace: [<fffffc000031e2bc>] cia_machine_check+0x9c/0xb0 [<fffffc000042fb40>] ext3_get_blocks_handle+0xe0/0xd00 [<fffffc0000315c70>] do_entInt+0x180/0x1e0 [<fffffc0000372704>] mempool_alloc_slab+0x24/0x40 [<fffffc0000310dc0>] ret_from_sys_call+0x0/0x10 [<fffffc00003729a0>] mempool_alloc+0x50/0x170 [<fffffc00003f18c4>] do_mpage_readpage+0x344/0x7e0 [<fffffc00004fb220>] __constant_c_memset+0x0/0x50 [<fffffc00004fb288>] loop+0x8/0x10 [<fffffc00003f1ee8>] mpage_readpages+0xf8/0x1c0 [<fffffc0000430760>] ext3_get_block+0x0/0x170 [<fffffc00004f0b3c>] radix_tree_insert+0x1ac/0x2f0 [<fffffc000036f550>] add_to_page_cache_locked+0xb0/0x180 [<fffffc00003f1eb8>] mpage_readpages+0xc8/0x1c0 [<fffffc0000430760>] ext3_get_block+0x0/0x170 [<fffffc000042c19c>] ext3_readpages+0x2c/0x40 [<fffffc00004545b0>] journal_stop+0x160/0x300 [<fffffc00004758d4>] security_file_open+0xa4/0xb0 [<fffffc000037b22c>] __do_page_cache_readahead+0x1fc/0x320 [<fffffc000037b798>] ra_submit+0x38/0x50 [<fffffc000037182c>] generic_file_aio_read+0x51c/0x800 [<fffffc00003adefc>] do_sync_read+0x9c/0x110 [<fffffc00003ae764>] vfs_read+0xb4/0x1c0 [<fffffc00004754e8>] security_file_permission+0xd8/0x110 [<fffffc00003ae204>] rw_verify_area+0x64/0x120 [<fffffc00003ae734>] vfs_read+0x84/0x1c0 [<fffffc00003ae8dc>] SyS_read+0x6c/0xc0 [<fffffc0000310da4>] entSys+0xa4/0xc0 Code: a75e0000 a53e0008 a55e0010 23de0020 6bfa8001 a55d0158 <a2910010> 261dffeaThanks in advance for an assist in figuring out what's going on here.
--Bob
Quoted 8 lines · Michael Cree
Quoted 5 lines · Bob Tracy
Getting lots of these since attempting to upgrade past 3.8.0-rc7. I *don't* think it's a kernel issue at this point, because while older kernels (found an old 3.5.0-rc4 setup from about a year ago in my archives) seem to take longer to reach this point, they'll eventually die exactly the same way.
Presumably the older kernel has worked reliably at some stage?
It was reliable enough to hook up an external USB hard drive and create a back-up, which was a non-trivial undertaking at the time on the PWS :-).
Quoted 5 lines
Quoted 2 lines
(System currently powered-down. Will open the case later and go looking for clogged/bad cooling fans, cat fur, etc.)
Good idea. Check motherboard is well seated into I/O board. Check memory is all well seated. Doing that got me some extra life out of my PWS!
Opened the case last night and found the expected amount of (which is to say, too much) cat, dog, and carpet fur on the air-intake grates in front of the two fans on the front of the machine. Power supply intake vents had *some* crud on them, but not severe. The rest of the machine interior looked good. All fans operational. Frankly, I've seen much worse.
Bottom line: still getting Oopses as below. At least they're consistent. The first one is normally followed immediately by one or two more (usually having to do with "scheduling while atomic, etc.), depending on what is running at the time. If I let the machine set idle after booting (except for stopping kdm, postgresql, apache2, samba, winbind), it will happily behave itself for hours (days? dunno... can't seem to leave it alone that long for some reason :-( ). If I do an "apt-get upgrade", sometimes I get through it ok, and sometimes not. I'm on my fifth reboot trying to get 3.9.0-rc4 built after having to restore my kernel source tree because applying the 3.9.0-rc1 patch (all 40 some odd MB of it) caused an Oops roughly half-way through the
process: that one was particularly nasty -- filesystem corruption got cleaned up ok by fsck on the reboot, but the contents of many blocks allocated to files that got updated hadn't been flushed to disk, so many of the patched files ended up containing fragments of unrelated files and/or nulls. Cleaning up that kind of corruption without a recent backup or access to an appropriate "git" repo is well nigh impossible.
Rarely, the Oops will lock things up bad enough I have to hit the reset switch (the kernel source patch episode above). Most of the time, I can perform an orderly shutdown.
I suppose if I had a bit of a clue, the consistency of what's happening would be enough to say, "*there's* your problem." One would think that the Oopses would be a bit more random if they're being caused by flaky hardware (as opposed to badness that is a bit more "immutable" in nature). You mentioned reseating memory and the I/O board, and I didn't do that when I had the case open: probably should have.
Quoted 60 lines
Quoted 60 lines
The process running at the time will vary, but the commonality I've noticed is "lots of disk I/O". Examples: cpio, applying the 3.9.0-rc1 kernel patch (approx. 40 MB uncompressed), running "git pull" on a local kernel source repository where v3.8 was the most recent tag, the final link of vmlinux on a kernel build, and so forth.
Reminder: this is a DEC Alpha system (PWS 433au).
Unable to handle kernel paging request at virtual address 0000000000000010cpio(4445): Oops 0
pc = [<fffffc0000315d1c>] ra = [<fffffc000031e2bc>] ps = 0007 Not tainted
pc is atprocess_mcheck_info+0x4c/0x320
ra is atcia_machine_check+0x9c/0xb0v0 = 0000000000000004 t0 = 0000000000000630 t1 = 0000000000000630 t2 = 0000000000000001 t3 = fffffc0000000000 t4 = fffffc00425ede38 t5 = fffffc00425ee000 t6 = 0000000000245b15 t7 = fffffc0042dbc000 s0 = 0000000000000000 s1 = fffffc00009ce258 s2 = fffffc00422b2498 s3 = fffffc0042dbfb68 s4 = 0000000000000002 s5 = 0000000000000002 s6 = 0000000000000002 a0 = 0000000000000630 a1 = 0000000000000000 a2 = fffffc00008aeb4c a3 = 0000000000000000 a4 = fffffc0042dbfb68 a5 = fffffc0042dbfb58 t8 = 0000000000000001 t9 = 0000000000245b15 t10= fffffc00422b23b8 t11= 0000000000245b15 pv = fffffc0000315cd0 at = fffffc0042dbf878 gp = fffffc0000a0d0d8 sp = fffffc0042dbf8a0 Disabling lock debugging due to kernel taint Trace: [<fffffc000031e2bc>] cia_machine_check+0x9c/0xb0 [<fffffc000042fb40>] ext3_get_blocks_handle+0xe0/0xd00 [<fffffc0000315c70>] do_entInt+0x180/0x1e0 [<fffffc0000372704>] mempool_alloc_slab+0x24/0x40 [<fffffc0000310dc0>] ret_from_sys_call+0x0/0x10 [<fffffc00003729a0>] mempool_alloc+0x50/0x170 [<fffffc00003f18c4>] do_mpage_readpage+0x344/0x7e0 [<fffffc00004fb220>] __constant_c_memset+0x0/0x50 [<fffffc00004fb288>] loop+0x8/0x10 [<fffffc00003f1ee8>] mpage_readpages+0xf8/0x1c0 [<fffffc0000430760>] ext3_get_block+0x0/0x170 [<fffffc00004f0b3c>] radix_tree_insert+0x1ac/0x2f0 [<fffffc000036f550>] add_to_page_cache_locked+0xb0/0x180 [<fffffc00003f1eb8>] mpage_readpages+0xc8/0x1c0 [<fffffc0000430760>] ext3_get_block+0x0/0x170 [<fffffc000042c19c>] ext3_readpages+0x2c/0x40 [<fffffc00004545b0>] journal_stop+0x160/0x300 [<fffffc00004758d4>] security_file_open+0xa4/0xb0 [<fffffc000037b22c>] __do_page_cache_readahead+0x1fc/0x320 [<fffffc000037b798>] ra_submit+0x38/0x50 [<fffffc000037182c>] generic_file_aio_read+0x51c/0x800 [<fffffc00003adefc>] do_sync_read+0x9c/0x110 [<fffffc00003ae764>] vfs_read+0xb4/0x1c0 [<fffffc00004754e8>] security_file_permission+0xd8/0x110 [<fffffc00003ae204>] rw_verify_area+0x64/0x120 [<fffffc00003ae734>] vfs_read+0x84/0x1c0 [<fffffc00003ae8dc>] SyS_read+0x6c/0xc0 [<fffffc0000310da4>] entSys+0xa4/0xc0 Code: a75e0000 a53e0008 a55e0010 23de0020 6bfa8001 a55d0158 <a2910010> 261dffeaThanks in advance for an assist in figuring out what's going on here.
--Bob