BUG: soft lockup detected on CPU#1!
Hello,
I am running the 2.6.28 based xfs kernel driver on a custom kernel with following kernel config enabled.
CONFIG_PREEMPTCONFIG_DETECT_SOFTLOCKUP
Running the following xfsqa causes a soft lockup. The configuration is a x86 with Hyperthreading, 4GB RAM and a AHCI connected JBOD. Its 100% reproducible.
Any suggestions/inputs on where to start debugging the problem would be much appreciated.
#! /bin/sh
# FS QA Test No. 008
#
# randholes test #
BUG: soft lockup detected on CPU#1! [<4013d525>] softlockup_tick+0x9c/0xaf [<40123246>] update_process_times+0x3d/0x60 [<401100ab>] smp_apic_timer_interrupt+0x52/0x58 [<40103633>] apic_timer_interrupt+0x1f/0x24 [<402a1557>] _spin_lock_irqsave+0x48/0x61 [<f8b8fe30>] xfs_iflush_cluster+0x16d/0x31c [xfs] [<f8b9018b>] xfs_iflush+0x1ac/0x271 [xfs] [<f8ba49a1>] xfs_inode_flush+0xd6/0xfa [xfs] [<f8bb13c8>] xfs_fs_write_inode+0x27/0x40 [xfs] [<401789d9>] __writeback_single_inode+0x1b0/0x2ff [<40101ad5>] __switch_to+0x23/0x1f9 [<40178f87>] sync_sb_inodes+0x196/0x261 [<4017920a>] writeback_inodes+0x67/0xb1 [<401465df>] wb_kupdate+0x7b/0xe0 [<40146bc3>] pdflush+0x0/0x1b5 [<40146ce1>] pdflush+0x11e/0x1b5 [<40146564>] wb_kupdate+0x0/0xe0 [<4012be6d>] kthread+0xc1/0xec [<4012bdac>] kthread+0x0/0xec [<401038b3>] kernel_thread_helper+0x7/0x10 =======================
Thanks,
Rakesh
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 51 lines · On Tue, 10 Feb 2009, [email protected] wrote:
Hello,
I am running the 2.6.28 based xfs kernel driver on a custom kernel with following kernel config enabled.
CONFIG_PREEMPTCONFIG_DETECT_SOFTLOCKUPRunning the following xfsqa causes a soft lockup. The configuration is a x86 with Hyperthreading, 4GB RAM and a AHCI connected JBOD. Its 100% reproducible.
Any suggestions/inputs on where to start debugging the problem would be much appreciated.
#!
/bin/sh
# FS QA Test No. 008
## randholes test #BUG: soft lockup detected on CPU#1! [<4013d525>] softlockup_tick+0x9c/0xaf [<40123246>] update_process_times+0x3d/0x60 [<401100ab>] smp_apic_timer_interrupt+0x52/0x58 [<40103633>] apic_timer_interrupt+0x1f/0x24 [<402a1557>] _spin_lock_irqsave+0x48/0x61 [<f8b8fe30>] xfs_iflush_cluster+0x16d/0x31c [xfs] [<f8b9018b>] xfs_iflush+0x1ac/0x271 [xfs] [<f8ba49a1>] xfs_inode_flush+0xd6/0xfa [xfs] [<f8bb13c8>] xfs_fs_write_inode+0x27/0x40 [xfs] [<401789d9>] __writeback_single_inode+0x1b0/0x2ff [<40101ad5>] __switch_to+0x23/0x1f9 [<40178f87>] sync_sb_inodes+0x196/0x261 [<4017920a>] writeback_inodes+0x67/0xb1 [<401465df>] wb_kupdate+0x7b/0xe0 [<40146bc3>] pdflush+0x0/0x1b5 [<40146ce1>] pdflush+0x11e/0x1b5 [<40146564>] wb_kupdate+0x0/0xe0 [<4012be6d>] kthread+0xc1/0xec [<4012bdac>] kthread+0x0/0xec [<401038b3>] kernel_thread_helper+0x7/0x10 =======================Thanks,
Rakesh
Quoted 5 lines
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
There were some pretty nasty bugs in 2.6.28 for XFS, can you reproduce it on 2.6.28.4?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hi Justin,
Thank you for the pointer. Well as suggested I did and now the xfsqa goes upto test 011
#! /bin/sh
# FS QA Test No. 011
#
# dirstress
but dies with a oops. Any suggestions?
Here is the oops trace -
BUG: unable to handle kernel NULL pointer dereference
at virtual address 00000000
printing eip:f8bd02c2
*pde = e7167067
Oops: 0000 [#1] PREEMPT SMP last sysfs file: /devices/pci0000:00/0000:00:1f.3/i2c-0/0-002e/temp1_input Modules linked in: xfs sg sunrpc m24c02 pca9554 pca9555 mcp23016 lm85 hwmon_vid i2c_i801 i2c_core midplane uhci_hcd sk98lin tg3 e1000 mv_sata sd_mod ahci libata CPU: 1 EIP: 0060:[<f8bd02c2>] Not tainted VLI EFLAGS: 00010286 (2.6.18.rhel5 #2)
EIP is at xfs_iget_core+0x4d6/0x5e9 [xfs]
eax: 00000000 ebx: e8ae7850 ecx: e79b2000 edx:
00000000
esi: e8ae7920 edi: ea44d2d0 ebp: ea44d298 esp:e79b3ce4
ds: 007b es: 007b ss: 0068
Process dirstress (pid: 9927, ti=e79b2000
task=e790faa0 task.ti=e79b2000)
Stack: ea7b0034 e9072c00 ea2a6380 0003b84a e8ae7850
ea2a6380 02c9f600 00000004
e9072c00 f8bd0478 1003b84a 00000000 00000001
00000004 e79b3d5c 00000000
00000000 ea7b0034 e9072c00 ea7b0034 1003b84a
00000000 f8bebbf1 1003b84a
Call Trace:
[<f8bd0478>] xfs_iget+0xa3/0x12f [xfs]
[<f8bebbf1>] xfs_trans_iget+0x1bd/0x249 [xfs]
[<f8bd4ebe>] xfs_ialloc+0xb9/0x5a9 [xfs]
[<f8bdc777>] xlog_grant_push_ail+0x105/0x12c [xfs]
[<f8bec673>] xfs_dir_ialloc+0x7e/0x26b [xfs]
[<f8be8c81>] xfs_trans_reserve+0x15c/0x240 [xfs]
[<f8bf12c8>] xfs_symlink+0x34a/0x862 [xfs]
[<401a62dd>] avc_has_perm_noaudit+0x38f/0x42d
[<401a6e9d>] avc_has_perm+0x3b/0x46
[<f8bf9d8e>] xfs_vn_symlink+0x6a/0xad [xfs]
[<4016a06d>] vfs_symlink+0xb2/0x11a
[<4016a149>] sys_symlinkat+0x74/0xab
[<402a14c2>] _spin_lock+0xd/0x5a
[<402a1593>] _spin_unlock+0xd/0x21
[<4016f416>] dput+0x83/0x11c
[<4015c89d>] __fput+0x152/0x175
[<4016a18f>] sys_symlink+0xf/0x13
[<40102b73>] syscall_call+0x7/0xb
=======================
Code: 8b 40 08 a8 08 74 05 e8 6d ff 6c 47 8b 44 24 0405 38 01 00 00 e8 24 02 6d 47 8b 44 24 04 8b 98 2c 01 00 00 85 db 74 3d 8b 43 04 <39> 18 74 14 b9 14 01 00 00 ba 20 8a c0 f8 b8 d8 8c c0 f8 e8 32
EIP: [<f8bd02c2>] xfs_iget_core+0x4d6/0x5e9 [xfs] SS:ESP 0068:e79b3ce4
<0>Kernel panic - not syncing: Fatal exception
Thanks,
Rakesh
Quoted 65 lines · --- Justin Piszcz
Quoted 4 lines · On Tue, 10 Feb 2009, [email protected] wrote:
Hello,
I am running the 2.6.28 based xfs kernel driver on
a
Quoted 1 line
custom kernel with following kernel config
enabled.
Quoted 5 lines
CONFIG_PREEMPTCONFIG_DETECT_SOFTLOCKUPRunning the following xfsqa causes a soft lockup.
The
Quoted 1 line
configuration is a x86 with Hyperthreading, 4GB
RAM
Quoted 3 lines
and a AHCI connected JBOD. Its 100% reproducible.
Any suggestions/inputs on where to start debugging
the
Quoted 37 lines
problem would be much appreciated.
#!
/bin/sh
# FS QA Test No. 008
## randholes test #BUG: soft lockup detected on CPU#1! [<4013d525>] softlockup_tick+0x9c/0xaf [<40123246>] update_process_times+0x3d/0x60 [<401100ab>] smp_apic_timer_interrupt+0x52/0x58 [<40103633>] apic_timer_interrupt+0x1f/0x24 [<402a1557>] _spin_lock_irqsave+0x48/0x61 [<f8b8fe30>] xfs_iflush_cluster+0x16d/0x31c [xfs] [<f8b9018b>] xfs_iflush+0x1ac/0x271 [xfs] [<f8ba49a1>] xfs_inode_flush+0xd6/0xfa [xfs] [<f8bb13c8>] xfs_fs_write_inode+0x27/0x40 [xfs] [<401789d9>] __writeback_single_inode+0x1b0/0x2ff [<40101ad5>] __switch_to+0x23/0x1f9 [<40178f87>] sync_sb_inodes+0x196/0x261 [<4017920a>] writeback_inodes+0x67/0xb1 [<401465df>] wb_kupdate+0x7b/0xe0 [<40146bc3>] pdflush+0x0/0x1b5 [<40146ce1>] pdflush+0x11e/0x1b5 [<40146564>] wb_kupdate+0x0/0xe0 [<4012be6d>] kthread+0xc1/0xec [<4012bdac>] kthread+0x0/0xec [<401038b3>] kernel_thread_helper+0x7/0x10 =======================Thanks,
RakeshQuoted 5 lines
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines
There were some pretty nasty bugs in 2.6.28 for XFS, can you reproduce it on
2.6.28.4?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
With debug enabled it fails with this -
BUG: unable to handle kernel NULL pointer dereference
at virtual address 00000000
printing eip:f8bd02c2
*pde = d658e067
Assertion failed: atomic_read(&ip->i_pincount) > 0,
file: fs/xfs/xfs_inode.c, line: 2703
------------[ cut here ]------------ Kernel BUG at [verbose debug info unavailable]
invalid opcode: 0000 [#1]
PREEMPT SMP
last sysfs file:
/devices/pci0000:00/0000:00:1f.3/i2c-0/0-002e/temp2_input
Modules linked in: sg xfs sunrpc m24c02 pca9554 pca9555 mcp23016 lm85 hwmon_vid i2c_i801 i2c_core midplane uhci_hcd sk98lin tg3 e1000 mv_sata sd_mod ahci libata CPU: 0 EIP: 0060:[<f8bfde19>] Not tainted VLI EFLAGS: 00010296 (2.6.18.rhel5 #2)
EIP is at assfail+0xd/0x13 [xfs]
eax: 0000005c ebx: e1fb4980 ecx: e4f46000 edx:
00000000
esi: 00000007 edi: e1fb0a58 ebp: 00000007 esp:e4f47eb8
ds: 007b es: 007b ss: 0068
Process xfslogd/0 (pid: 4370, ti=e4f46000 task=75118aa0 task.ti=e4f46000)
Stack: f8c10ccb f8c090ae f8c08d97 00000a8f f8bd0dca
00000526 f8be8e28 00000526
00000007 e9db1d80 e861c008 e9db1da0 e861c000
00000003 e9db1d80 e9db1ca4
e9db1ca0 00000000 f8be8f56 00000000 00000000
e9db1ca4 ea449880 ea449800
Call Trace:
[<f8bd0dca>] xfs_iunpin+0x21/0x49 [xfs]
[<f8be8e28>] xfs_trans_chunk_committed+0xc3/0xe6
[xfs]
[<f8be8f56>] xfs_trans_committed+0x38/0xd1 [xfs]
[<f8bdba5c>] xlog_state_do_callback+0x1b7/0x329 [xfs]
[<f8bf66b7>] xfs_buf_iodone_work+0x41/0x63 [xfs]
[<401294c5>] run_workqueue+0x71/0xae
[<f8bf6676>] xfs_buf_iodone_work+0x0/0x63 [xfs]
[<40129666>] worker_thread+0xd9/0x10a
[<40116ca2>] default_wake_function+0x0/0xc
[<4012958d>] worker_thread+0x0/0x10a
[<4012be6d>] kthread+0xc1/0xec
[<4012bdac>] kthread+0x0/0xec
[<401038b3>] kernel_thread_helper+0x7/0x10
=======================
Code: d2 df 51 47 89 ea b8 94 59 c2 f8 e8 a9 37 6a 4783 c4 0c 85 ff 75 02 0f 0b 5b 5e 5f 5d c3 51 52 50 68 cb 0c c1 f8 e8 ab df 51 47 <0f> 0b 83 c4 10 c3 55 57 56 53 83 ec 14 89 44 24 04 89 d7 89 cd
EIP: [<f8bfde19>] assfail+0xd/0x13 [xfs] SS:ESP 0068:e4f47eb8
<0>Kernel panic - not syncing: Fatal exception
Quoted 65 lines · --- Justin Piszcz
Quoted 4 lines · On Tue, 10 Feb 2009, [email protected] wrote:
Hello,
I am running the 2.6.28 based xfs kernel driver on
a
Quoted 1 line
custom kernel with following kernel config
enabled.
Quoted 5 lines
CONFIG_PREEMPTCONFIG_DETECT_SOFTLOCKUPRunning the following xfsqa causes a soft lockup.
The
Quoted 1 line
configuration is a x86 with Hyperthreading, 4GB
RAM
Quoted 3 lines
and a AHCI connected JBOD. Its 100% reproducible.
Any suggestions/inputs on where to start debugging
the
Quoted 37 lines
problem would be much appreciated.
#!
/bin/sh
# FS QA Test No. 008
## randholes test #BUG: soft lockup detected on CPU#1! [<4013d525>] softlockup_tick+0x9c/0xaf [<40123246>] update_process_times+0x3d/0x60 [<401100ab>] smp_apic_timer_interrupt+0x52/0x58 [<40103633>] apic_timer_interrupt+0x1f/0x24 [<402a1557>] _spin_lock_irqsave+0x48/0x61 [<f8b8fe30>] xfs_iflush_cluster+0x16d/0x31c [xfs] [<f8b9018b>] xfs_iflush+0x1ac/0x271 [xfs] [<f8ba49a1>] xfs_inode_flush+0xd6/0xfa [xfs] [<f8bb13c8>] xfs_fs_write_inode+0x27/0x40 [xfs] [<401789d9>] __writeback_single_inode+0x1b0/0x2ff [<40101ad5>] __switch_to+0x23/0x1f9 [<40178f87>] sync_sb_inodes+0x196/0x261 [<4017920a>] writeback_inodes+0x67/0xb1 [<401465df>] wb_kupdate+0x7b/0xe0 [<40146bc3>] pdflush+0x0/0x1b5 [<40146ce1>] pdflush+0x11e/0x1b5 [<40146564>] wb_kupdate+0x0/0xe0 [<4012be6d>] kthread+0xc1/0xec [<4012bdac>] kthread+0x0/0xec [<401038b3>] kernel_thread_helper+0x7/0x10 =======================Thanks,
RakeshQuoted 5 lines
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines
There were some pretty nasty bugs in 2.6.28 for XFS, can you reproduce it on
2.6.28.4?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 31 lines · On Wed, 11 Feb 2009, [email protected] wrote:
Hi Justin,
Thank you for the pointer. Well as suggested I did and now the xfsqa goes upto test 011
#!
/bin/sh
# FS QA Test No. 011
## dirstressbut dies with a oops. Any suggestions?
Here is the oops trace -
BUG: unable to handle kernel NULL pointer dereferenceat virtual address 00000000
printing eip:f8bd02c2
*pde =e7167067Oops: 0000 [#1] PREEMPT SMP last sysfs file: /devices/pci0000:00/0000:00:1f.3/i2c-0/0-002e/temp1_input Modules linked in: xfs sg sunrpc m24c02 pca9554 pca9555 mcp23016 lm85 hwmon_vid i2c_i801 i2c_core midplane uhci_hcd sk98lin tg3 e1000 mv_sata sd_mod ahci libata CPU: 1 EIP: 0060:[<f8bd02c2>] Not tainted VLI EFLAGS: 00010286 (2.6.18.rhel5 #2)
Quoted 1 line
EIP is at
xfs_iget_core+0x4d6/0x5e9[xfs]
Can you show uname -a output?
EFLAGS: 00010286 (2.6.18.rhel5 #2)
^^^^^^^^^^^^
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hi Justin,
Yes it is a 2.6.18 rhel5 based custom kernel but the XFS driver is a back port from the 2.6.28.4
Thanks,
Rakesh
Quoted 15 lines · --- Justin Piszcz
Quoted 4 lines · On Wed, 11 Feb 2009, [email protected] wrote:
Hi Justin,
Thank you for the pointer. Well as suggested I did
and
Quoted 7 lines
now the xfsqa goes upto test 011
#!
/bin/sh
# FS QA Test No. 011
## dirstress
Quoted 15 lines
Quoted 5 lines
but dies with a oops. Any suggestions?
Here is the oops trace -
BUG: unable to handle kernel NULL pointerdereference
Quoted 8 lines
at virtual address 00000000
printing eip:f8bd02c2
*pde =e7167067Oops: 0000 [#1]PREEMPT SMP
last sysfs file:
/devices/pci0000:00/0000:00:1f.3/i2c-0/0-002e/temp1_input
Quoted 9 lines
Quoted 8 lines
Modules linked in: xfs sg sunrpc m24c02 pca9554 pca9555 mcp23016 lm85 hwmon_vid i2c_i801 i2c_core midplane uhci_hcd sk98lin tg3 e1000 mv_sata sd_mod ahci libata CPU: 1 EIP: 0060:[<f8bd02c2>] Not tainted VLI EFLAGS: 00010286 (2.6.18.rhel5 #2)EIP is at
xfs_iget_core+0x4d6/0x5e9[xfs]
Quoted 5 lines
Can you show uname -a output?
EFLAGS: 00010286 (2.6.18.rhel5 #2)
^^^^^^^^^^^^
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 6 lines · On Tue, Feb 10, 2009 at 11:16:25PM -0800, [email protected] wrote:
Hello,
I am running the 2.6.28 based xfs kernel driver on a custom kernel with following kernel config enabled.
Quoted 6 lines
CONFIG_PREEMPTCONFIG_DETECT_SOFTLOCKUPRunning the following xfsqa causes a soft lockup. The configuration is a x86 with Hyperthreading, 4GB RAM and a AHCI connected JBOD. Its 100% reproducible.
Is the system making progress, or has it hung? i.e. does the test complete?
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · On Thu, Feb 12, 2009 at 01:22:16AM -0800, [email protected] wrote:
Hi Justin,
Yes it is a 2.6.18 rhel5 based custom kernel but the XFS driver is a back port from the 2.6.28.4
Then you get to keep all the broken bits to yourself. If you want to throw random versions of XFS at random versions of kernels then we can't help you - we don't have the time or resources to support random backports of XFS to older kernels (and non-vanilla kernels at that).
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Hi Dave,
No it does not. Just sits there any access to mount point (ll) also blocks forever.
I must point out that I brought the changes from 2.6.28.4 xfs into my private space and that seems to show improvement on test 008 however now test 011 dies with kernel oops.
Please see my reply posts.
Thanks,
Rakesh
Quoted 19 lines · --- Dave Chinner
Quoted 4 lines · On Tue, Feb 10, 2009 at 11:16:25PM -0800, [email protected] wrote:
Hello,
I am running the 2.6.28 based xfs kernel driver on
a
Quoted 1 line
custom kernel with following kernel config
enabled.
Quoted 5 lines
CONFIG_PREEMPTCONFIG_DETECT_SOFTLOCKUPRunning the following xfsqa causes a soft lockup.
The
Quoted 1 line
configuration is a x86 with Hyperthreading, 4GB
RAM
Quoted 1 line
and a AHCI connected JBOD. Its 100% reproducible.
Quoted 11 lines
Is the system making progress, or has it hung? i.e. does the test
complete?Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Well the problem is the older kernel XFS driver is buggy to such a large extent that there is data loss even for data on rest should a power loss occur.
With a newer version back port I can preserve the kernel version change since it becomes far more reaching to the other kernel components and they have to move, to which ..... there is strong reservation.
Hope this gives you the perspective.
Thanks,
Rakesh
Quoted 19 lines · --- Dave Chinner
Quoted 4 lines · On Thu, Feb 12, 2009 at 01:22:16AM -0800, [email protected] wrote:
Hi Justin,
Yes it is a 2.6.18 rhel5 based custom kernel but
the
Quoted 1 line
XFS driver is a back port from the 2.6.28.4
Then you get to keep all the broken bits to yourself. If you want
to throw random versions of XFS at random versions of kernels then
we can't help you - we don't have the time or resources to support
random backports of XFS to older kernels (and non-vanilla kernels
at that).
Quoted 7 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · [email protected] wrote:
Well the problem is the older kernel XFS driver is buggy to such a large extent that there is data loss even for data on rest should a power loss occur.
With a newer version back port I can preserve the kernel version change since it becomes far more reaching to the other kernel components and they have to move, to which ..... there is strong reservation.
Hope this gives you the perspective.
It's totally understandable why you might want to do it.
It's also totally understandable why upstream developers can't spend a lot of time on your custom codebase.
What you need, of course, is a distribution with good support for xfs, so you can make it Someone Else's Problem. :)
-Eric
Quoted 2 lines
Thanks,
Rakesh
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Guys,
Thank you for taking the time to write. Having said where I stand and we are kind of on the same page. Is there something I can expect which would put me on a track of nailing down the problem. It maybe a wild goose chase but something that I can start with would be much appreciated.
Unfortunately there is no distro which gets closer to where mainline lives today. Reading the changelog there are several problems that I have already come across and has convincingly driven me to take on this task.
Thanks,
Rakesh
Quoted 22 lines · --- Eric Sandeen
Quoted 2 lines · [email protected] wrote:
Well the problem is the older kernel XFS driver is buggy to such a large extent that there is data
loss
Quoted 5 lines
even for data on rest should a power loss occur.
With a newer version back port I can preserve the kernel version change since it becomes far more reaching to the other kernel components and they
have
Quoted 1 line
to move, to which ..... there is strong
reservation.
Quoted 2 lines
Hope this gives you the perspective.
It's totally understandable why you might want to do it.
It's also totally understandable why upstream
developers can't spend a
lot of time on your custom codebase.
Quoted 12 lines
What you need, of course, is a distribution with
good support for xfs,
so you can make it Someone Else's Problem. :)-Eric
Quoted 2 lines
Thanks,
Rakesh
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · [email protected] wrote:
Guys,
Thank you for taking the time to write. Having said where I stand and we are kind of on the same page. Is there something I can expect which would put me on a track of nailing down the problem. It maybe a wild goose chase but something that I can start with would be much appreciated.
Just random debugging thoughts...
Try stock 2.6.28.4, to see if you have the same problem. If so, and esp. if you also see it on 2.6.29, then you'll get a lot more attention here. :)
If not, then it's something with your backport most likely. Figure out what you had to backport and see if it's possibly causing the error(s).
If it's locked up, try sysrq-w (echo w > /proc/sysrq-trigger) and look at dmesg to see if other threads are locked against it. Figure out why.
On the oops try memory debugging etc, see if you're referencing freed memory, using corrupt lists, etc.
Look for other errors in the logs prior to this.
See if your filesystem is corrupted.
Bug Red Hat for XFS support, assuming you're actually buying RHEL5 support from them. :)
Quoted 5 lines
Unfortunately there is no distro which gets closer to where mainline lives today. Reading the changelog there are several problems that I have already come across and has convincingly driven me to take on this task.
well certainly there are distros with kernels newer than 2.6.18, but it depends on your needs & goals I guess.
Good luck,
-Eric
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 2 lines · On Donnerstag 12 Februar 2009 [email protected] wrote:
Unfortunately there is no distro which gets closer to where mainline lives today.
If I understand correctly, I would say use openSUSE 11.1, which has kernel 2.6.27.7-9 as of today (and full XFS support).
And you can still download a newer vanilla kernel and use it on your distro. You do not need to use the distro kernel, you just need to install kernel updates on your own then.
mfg zmi
Signature · 10 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import" // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
I think I am getting closer now. Can you suggest what to look for from this oops -
Stack traceback for pid 1920xad12f030 192 11 1 0 R 0xad12f1d0
*xfsdatad/0
esp eip Function (args)0xaff37eec 0x7815007b map_vm_area+0xc30xaff37ef0 0x7814007b find_get_pages_contig+0x3d0xaff37ef8 0x78156c04 free_block+0x410xaff37f20 0x78156d9b cache_flusharray+0x630xaff37f3c 0x78156b4e kmem_cache_free+0x520xaff37f4c 0x7814296f mempool_free_slab+0xb0xaff37f50 0x78142954 mempool_free+0x600xaff37f60 0x781f429f xfs_destroy_ioend+0x4e0xaff37f6c 0x781f43d0 xfs_end_bio_read+0x50xaff37f70 0x78128498 run_workqueue+0x710xaff37f74 0x781f43cb xfs_end_bio_read0xaff37f8c 0x78128634 worker_thread+0xd90xaff37fac 0x781164c0 default_wake_function0xaff37fc8 0x7812855b worker_thread0xaff37fcc 0x7812ad7a kthread+0xc10xaff37fd8 0x7812acb9 kthread0xaff37fe4 0x781036df kernel_thread_helper+0x7
Thanks,
Rakesh
Quoted 59 lines · --- Eric Sandeen
Quoted 3 lines · [email protected] wrote:
Guys,
Thank you for taking the time to write. Having
said
Quoted 1 line
where I stand and we are kind of on the same page.
Is
Quoted 1 line
there something I can expect which would put me on
a
Quoted 2 lines
track of nailing down the problem. It maybe a wild goose chase but something that I can start with
would
Quoted 1 line
be much appreciated.
Just random debugging thoughts...
Try stock 2.6.28.4, to see if you have the same problem. If so, and
esp. if you also see it on 2.6.29, then you'll get a lot more attention
here. :)If not, then it's something with your backport most likely. Figure out
what you had to backport and see if it's possibly causing the error(s).If it's locked up, try sysrq-w (echo w >
/proc/sysrq-trigger) and look
at dmesg to see if other threads are locked against it. Figure out why.On the oops try memory debugging etc, see if you're referencing freed
memory, using corrupt lists, etc.Look for other errors in the logs prior to this.
See if your filesystem is corrupted.
Bug Red Hat for XFS support, assuming you're actually buying RHEL5
support from them. :)Quoted 1 line
Unfortunately there is no distro which gets closer
to
Quoted 2 lines
where mainline lives today. Reading the changelog
there are several problems that I have alreadycome
Quoted 1 line
across and has convincingly driven me to take on
this
Quoted 1 line
task.
well certainly there are distros with kernels newer than 2.6.18, but it
depends on your needs & goals I guess.Good luck,
Review
Quoted 3 lines
(quoted diff) +0
-2-Eric -Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs