performance problems with XFS and Linux kernel 2.6.24.4

Hi,

I am aware of the change regarding barrier/nobarrier in 2.6.17 and above. However, the performance hit I'm seeing is very severe and much worse than what I was seeing in 2.6.17 (bundled with latest updates in Fedora Core 4).

I'm currently running Fedora 8 with 2.6.24.4-64.fc8. If I remove nobarrier mount option, it takes seconds just to write few megabytes and the system becomes very very sluggish. On 2.6.17 (Fedora 4) without the nobarrier option it was still usable.

The XFS file system is about 900GB, if that matters.

Is this a known problem? How can I gather more information to figure out exactly what is going on?

Thanks,
-Bond

Doesn't sound like a barrier problem.

<shrug>

Start with what your hardware is, what your storage config is, output of 'xfs_info <mtpt>', the log of your test (complete output) on working and non-working kernels, etc.

Cheers,

Dave.

Dave Chinner
Principal Engineer
SGI Australian Software Group

Not sure if this is XFS related or not, but this seems to be reoccurring.

The hardware is:

Dell Inspiron 530, Q6600 quad core, 4GB Ram, 2x750GB SATA HDD

I have Fedora 8/64bit installed, using XFS for "/", ext3 for "/boot". The two identical 750GB drives are striped to create a 1.4TB "md1" device, which is mounted as "/". Kernel is from Fedora8, 2.6.24.4-64.fc8.

During the install, everything seemed to work fine. I was able to use the system without any problems for about 2 days. Then I reboot, and the md1 device running XFS shows the following error messages:

I/O error in filesystem ("md1") meta-data dev md1 block 0x15d048574120008 ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
md1: rw=0, want=98239738096517136, limit=2927780864

The above message repeats several times, but the system manages to boot to runlevel 3. I can login, but almost any command triggers that message which has flooded the logs. here is xfs_info on "/":

meta-data=/dev/root isize=256 agcount=32,
agsize=11436608 blks

	   =					sectsz=512	attr=1
data	   =					bsize=4096
blocks=365971456, imaxpct=25
	   =					sunit=64	swidth=128

blks, unwritten=1
naming =version2 bsize=4096
log =internal bsize=4096 blocks=32768,
version=1

	   =					sectsz=512	sunit=0

blks, lazy-count=0
realtime =none extsz=524288 blocks=0, rtextents=0

At first, I thought perhaps something wrong with the disks so I had them replaced. Reinstalled Fedora8 as before, then again, 2 days later (I don't know if the 2 day time period is coincidental or not), I reboot, and the exact same problem occurred. Since I've gone through 4 brand new 750GB HDD now, (Seagate diagnostics show all the disks to be fine in both occurances, but I replaced them anyway) I'm feeling that this is not a hardware issue, but software related.

A search on google seems to show others with possibly similar issues:

http://ubuntuforums.org/archive/index.php/t-568309.html

Is that related? Other google searches show a pattern of this problem only with >1TB XFS partitions, but that's just what I found googling.

I'm considering re-installing again, but using ext3 for "/" to see if the problem reoccurs, in hopes of "isolating" the issue. Not sure if it is related to XFS, the software stripe md1, or something else (but I don't think it's HDD related now).

Thanks for any info or assistance. Though the system isn't really usable, (many commands fail, but does boot to runlevel 3 and permits me to login) I'll leave it as is for today so let me know if there's other info I can try to gather that would be useful.

Any help would be appreciated.
-Bond

You are probably hitting:

https://bugzilla.redhat.com/show_bug.cgi?id=437968
Bugzilla Bug 437968: Corrupt xfs root filesystem with kernel kernel-2.6.24.3-xx

Fix is in 2.6.25-rc9 upstream and kernel-2.6.24.5-85.fc8 has been submitted as an update for Fedora 8 and is now in stable.

If you need to keep the system as-is I'd upgrade the kernel immediately, then try xfs_repair.

A fresh install might be best; do an install, upgrade to that kernel, reboot into it, *then* yum update the rest.

-Eric