kernel 2.6.27.7 problem
Hi,
I have trouble with xfs after upgrade kernel 2.6.24.3 -> 2.6.27.7.
Message on console:
end_request: I/O error, dev sde, sector 351572544
I/O error in filesystem ("sde1") meta-data dev sde1 block 0x14f49201 ("xlog_iodone") error 5 buf count 9728 xfs_force_shutdown(sde1,0x2) called from line 1056 of file fs/xfs/xfs_log.c. Return address = 0xffffffff80373233 Filesystem "sde1": Log I/O Error Detected. Shutting down filesystem:
sde1
Please umount the filesystem, and rectify the problem(s)
Remount is possible, but filesystem hangs up again. After xfs_repair too.
It looks OK now after rebooting to old kernel.
On other similar system (same disk array, other vol) I use 2.6.27.6 kernel without problems.
System:
AMD x64, debian stable (etch)
filesytem is on iSCSI storage
mount options:
rw,noatime,nodiratime,logbsize=256k,logbufs=8,quota
xfs_info:
meta-data=/dev/sde1 isize=256 agcount=16, agsize=5493104
blks
= sectsz=512 attr=0
data = bsize=4096 blocks=87889584,
imaxpct=25
= sunit=16 swidth=112 blks
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=32768, version=2
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=65536 blocks=0, rtextents=0Any ideas.
Thanx
Dusan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · DS
Hi,
I have trouble with xfs after upgrade kernel 2.6.24.3 -> 2.6.27.7.
Quoted 9 lines
Message on console:
end_request: I/O error, dev sde, sector 351572544
I/O error in filesystem ("sde1") meta-data dev sde1 block0x14f49201("xlog_iodone") error 5 buf count 9728xfs_force_shutdown(sde1,0x2)called from line 1056 of filefs/xfs/xfs_log.c. Return address =0xffffffff80373233Filesystem "sde1": Log I/O Error Detected. Shutting down filesystem:
sde1
Please umount the filesystem, and rectify the problem(s)
No messages before that? It's telling you that you had an IO error (under the filesystem, potentially a storage proboblem).
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
No other messages before. No iSCSI/network messages...nothing. Only this few lines and FS is down. After remount it works ....while and hangs again.
Few hours it runs back on 2.6.24.3 and everything is OK.
It's production system, but I test it again in late night. What can I do to identify problem?
Dusan
Quoted 15 lines · Eric Sandeen
Quoted 4 lines · DS
Hi,
I have trouble with xfs after upgrade kernel 2.6.24.3 -> 2.6.27.7.
Quoted 9 lines
Message on console:
end_request: I/O error, dev sde, sector 351572544
I/O error in filesystem ("sde1") meta-data dev sde1 block0x14f49201("xlog_iodone") error 5 buf count 9728xfs_force_shutdown(sde1,0x2)called from line 1056 of filefs/xfs/xfs_log.c. Return address =0xffffffff80373233Filesystem "sde1": Log I/O Error Detected. Shutting down filesystem:
sde1
Please umount the filesystem, and rectify the problem(s)
Quoted 4 lines
No messages before that? It's telling you that you had an IO error (under the filesystem, potentially a storage proboblem).
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · DS
No other messages before. No iSCSI/network messages...nothing. Only this few lines and FS is down. After remount it works ....while and hangs again.
Few hours it runs back on 2.6.24.3 and everything is OK.It's production system, but I test it again in late night. What can I do to identify problem?
Dusan
Run:
smartctl -a /dev/sde
Post output.
Justin.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 12 lines · Justin Piszcz
Quoted 8 lines · DS
No other messages before. No iSCSI/network messages...nothing. Only this few lines and FS is down. After remount it works ....while and hangs again.
Few hours it runs back on 2.6.24.3 and everything is OK.It's production system, but I test it again in late night. What can I do to identify problem?
Quoted 10 lines
Quoted 1 line
Dusan
Run:
smartctl -a/dev/sdePost output.
Justin.
AMD x64, debian stable (etch)
filesytem is on iSCSI storage
^^^^^^^^^^^^^^
Nevermind my comment above:
Is there anyway to test the health of the disks on the system?
Justin.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
iSCSI storage is nexsan SATA boy with 2 controlers, firmware was updated few months ago
other 3 servers has data on this storage without problems (all on XFS) all is going on same switch, all ports are in one storage VLAN
I don't know where "start debug" and it must be safe for data.
Dusan
Quoted 29 lines · Justin Piszcz
Quoted 12 lines · Justin Piszcz
Quoted 8 lines · DS
No other messages before. No iSCSI/network messages...nothing. Only this few lines and FS is down. After remount it works ....while and hangs again.
Few hours it runs back on 2.6.24.3 and everything is OK.It's production system, but I test it again in late night. What can I do to identify problem?
Quoted 10 lines
Quoted 1 line
Dusan
Run:
smartctl -a/dev/sdePost output.
Justin.
AMD x64, debian stable (etch)
filesytem is on iSCSI storage
Quoted 8 lines
^^^^^^^^^^^^^^Nevermind my comment above:
Is there anyway to test the health of the disks on the system?
Justin.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · DS
No other messages before. No iSCSI/network messages...nothing. Only this few lines and FS is down. After remount it works ....while and hangs again.
Few hours it runs back on 2.6.24.3 and everything is OK.It's production system, but I test it again in late night. What can I do to identify problem?
Complain to the iscsi people perhaps...?
__end_that_request_first() in block/block-core.c() is what is issuing your first error,
Quoted 1 line
end_request: I/O error, dev sde, sector 351572544
and the rest is xfs reacting to that.
Maybe the first thing I'd check is whether that sector nr. is sane for your device (I'd assume it is, but maybe worth checking).
If that's all you've got I'm not sure offhand what else to debug....
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs