arcmsr - device going offline under 3.10.x (regression?)
Hi,
we're experiencing strange trouble with new box using areca based RAID and 3.10.x kernel.
When I load disk subsystem (ie with fio benchmark), system seems to be working fine, but after some time (usually next day), device seems to die:
<3>[43567.763088] EXT4-fs (dm-4): previous I/O error to superblock detected <3>[43567.763091] sd 0:0:1:0: rejecting I/O to offline device <2>[43567.763093] EXT4-fs error (device dm-4): __ext4_get_inode_loc:4039: inode #135868: block 524619: comm syslog-ng: unable to read itable block <2>[43567.763094] EXT4-fs error (device dm-4) in ext4_reserve_inode_write:4962: IO failure <3>[43567.763095] EXT4-fs (dm-4): previous I/O error to superblock detected <3>[43567.763097] sd 0:0:1:0: rejecting I/O to offline device <4>[43567.763099] EXT4-fs warning (device dm-4): ext4_evict_inode:258: couldn't mark inode dirty (err -5) <3>[43567.763143] sd 0:0:1:0: rejecting I/O to offline device <3>[43567.763150] sd 0:0:1:0: rejecting I/O to offline device
Hardware is Supermicro X9DRW/X9DRW with 6 core E5-2620, 64GB RAM, running centos6 + vanilla 3.10.x kernel. I tried with two different adapters (1223 and 1882). Since box uses redundant powersupplies, I don't think it could be power issue as I found in some areca FAQ. I'm using westerdn digital raid edition SATA drives.
we've experienced this problem with 3.10.22 and 3.10.25. I wasn't able to reproduce this with 3.0.101 (yet, I'm torturing the box for ~2 days now, so I can't be sure).
Since the problem takes quite long to reproduce (and I'm not sure after how long I can tell the system works fine), I'd like to ask about your oppinions before trying to bisect this..
any ideas?
thanks a lot in advance!
BR
nik
Signature · 13 lines
------------------------------------- Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
CC Nick Cheng
one more (a bit) related question, Nick, I can see Your last commit in git on Jan 6 2011, since then mostly only miscelanous changes to keep arcmsr in sync with rest of kernel.
But on areca webpages, I can see linux driver 1.30.0X.16-131015, ie only few months old. There is neither changelog, nor older releases to be found, only mention of ARC-12x4 cards supported.
Is there really no development in this driver, no bugfixes, anything?
We've got quite a lot of areca boards in production, and it's making me a bit nervous...
If You could also look at this issue, I'd be grateful and happy to provide whatever debug info could help to track this down..
BR
nik
Quoted 52 lines · Nikola Ciprich
Hi,
we're experiencing strange trouble with new box using areca based RAID and 3.10.x kernel.
When I load disk subsystem (ie with fio benchmark), system seems to be working fine, but after some time (usually next day), device seems to die:
<3>[43567.763088] EXT4-fs (dm-4): previous I/O error to superblock detected <3>[43567.763091] sd 0:0:1:0: rejecting I/O to offline device <2>[43567.763093] EXT4-fs error (device dm-4): __ext4_get_inode_loc:4039: inode #135868: block 524619: comm syslog-ng: unable to read itable block <2>[43567.763094] EXT4-fs error (device dm-4) in ext4_reserve_inode_write:4962: IO failure <3>[43567.763095] EXT4-fs (dm-4): previous I/O error to superblock detected <3>[43567.763097] sd 0:0:1:0: rejecting I/O to offline device <4>[43567.763099] EXT4-fs warning (device dm-4): ext4_evict_inode:258: couldn't mark inode dirty (err -5) <3>[43567.763143] sd 0:0:1:0: rejecting I/O to offline device <3>[43567.763150] sd 0:0:1:0: rejecting I/O to offline deviceHardware is Supermicro X9DRW/X9DRW with 6 core E5-2620, 64GB RAM, running centos6 + vanilla 3.10.x kernel. I tried with two different adapters (1223 and 1882). Since box uses redundant powersupplies, I don't think it could be power issue as I found in some areca FAQ. I'm using westerdn digital raid edition SATA drives.
we've experienced this problem with 3.10.22 and 3.10.25. I wasn't able to reproduce this with 3.0.101 (yet, I'm torturing the box for ~2 days now, so I can't be sure).
Since the problem takes quite long to reproduce (and I'm not sure after how long I can tell the system works fine), I'd like to ask about your opinions before trying to bisect this..
any ideas?
thanks a lot in advance!
BR
nik
Signature · 13 lines
------------------------------------- Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
Signature · 13 lines
------------------------------------- Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------