Skip to contentSkip to conversation

arcmsr - device going offline under 3.10.x (regression?)

DISCUSSION Nikola Ciprich 1 participant 2 messages Jan 6, 2014 SCSI

lore
Share & tools
lore
#0 lore raw

Hi,

we're experiencing strange trouble with new box using areca based RAID and 3.10.x kernel.

When I load disk subsystem (ie with fio benchmark), system seems to be working fine, but after some time (usually next day), device seems to die:

<3>[43567.763088] EXT4-fs (dm-4): previous I/O error to superblock detected
<3>[43567.763091] sd 0:0:1:0: rejecting I/O to offline device
<2>[43567.763093] EXT4-fs error (device dm-4): __ext4_get_inode_loc:4039: inode #135868: block 524619: comm syslog-ng: unable to read itable block
<2>[43567.763094] EXT4-fs error (device dm-4) in ext4_reserve_inode_write:4962: IO failure
<3>[43567.763095] EXT4-fs (dm-4): previous I/O error to superblock detected
<3>[43567.763097] sd 0:0:1:0: rejecting I/O to offline device
<4>[43567.763099] EXT4-fs warning (device dm-4): ext4_evict_inode:258: couldn't mark inode dirty (err -5)
<3>[43567.763143] sd 0:0:1:0: rejecting I/O to offline device
<3>[43567.763150] sd 0:0:1:0: rejecting I/O to offline device

Hardware is Supermicro X9DRW/X9DRW with 6 core E5-2620, 64GB RAM, running centos6 + vanilla 3.10.x kernel. I tried with two different adapters (1223 and 1882). Since box uses redundant powersupplies, I don't think it could be power issue as I found in some areca FAQ. I'm using westerdn digital raid edition SATA drives.

we've experienced this problem with 3.10.22 and 3.10.25. I wasn't able to reproduce this with 3.0.101 (yet, I'm torturing the box for ~2 days now, so I can't be sure).

Since the problem takes quite long to reproduce (and I'm not sure after how long I can tell the system works fine), I'd like to ask about your oppinions before trying to bisect this..

any ideas?

thanks a lot in advance!

BR

nik

Signature · 13 lines
-------------------------------------
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28.rijna 168, 709 00 Ostrava

tel.:   +420 591 166 214
fax:    +420 596 621 273
mobil:  +420 777 093 799
www.linuxbox.cz

mobil servis: +420 737 238 656
email servis: [email protected]
-------------------------------------
#1 lore raw

CC Nick Cheng

one more (a bit) related question, Nick, I can see Your last commit in git on Jan 6 2011, since then mostly only miscelanous changes to keep arcmsr in sync with rest of kernel.
But on areca webpages, I can see linux driver 1.30.0X.16-131015, ie only few months old. There is neither changelog, nor older releases to be found, only mention of ARC-12x4 cards supported.

Is there really no development in this driver, no bugfixes, anything?

We've got quite a lot of areca boards in production, and it's making me a bit nervous...

If You could also look at this issue, I'd be grateful and happy to provide whatever debug info could help to track this down..

BR

nik

Quoted 52 lines · Nikola Ciprich

Hi,

we're experiencing strange trouble with new box using areca based RAID and 3.10.x kernel.

When I load disk subsystem (ie with fio benchmark), system seems to be working fine, but after some time (usually next day), device seems to die:

<3>[43567.763088] EXT4-fs (dm-4): previous I/O error to superblock detected
<3>[43567.763091] sd 0:0:1:0: rejecting I/O to offline device
<2>[43567.763093] EXT4-fs error (device dm-4): __ext4_get_inode_loc:4039: inode #135868: block 524619: comm syslog-ng: unable to read itable block
<2>[43567.763094] EXT4-fs error (device dm-4) in ext4_reserve_inode_write:4962: IO failure
<3>[43567.763095] EXT4-fs (dm-4): previous I/O error to superblock detected
<3>[43567.763097] sd 0:0:1:0: rejecting I/O to offline device
<4>[43567.763099] EXT4-fs warning (device dm-4): ext4_evict_inode:258: couldn't mark inode dirty (err -5)
<3>[43567.763143] sd 0:0:1:0: rejecting I/O to offline device
<3>[43567.763150] sd 0:0:1:0: rejecting I/O to offline device

Hardware is Supermicro X9DRW/X9DRW with 6 core E5-2620, 64GB RAM, running centos6 + vanilla 3.10.x kernel. I tried with two different adapters (1223 and 1882). Since box uses redundant powersupplies, I don't think it could be power issue as I found in some areca FAQ. I'm using westerdn digital raid edition SATA drives.

we've experienced this problem with 3.10.22 and 3.10.25. I wasn't able to reproduce this with 3.0.101 (yet, I'm torturing the box for ~2 days now, so I can't be sure).

Since the problem takes quite long to reproduce (and I'm not sure after how long I can tell the system works fine), I'd like to ask about your opinions before trying to bisect this..

any ideas?

thanks a lot in advance!

BR

nik

Signature · 13 lines
-------------------------------------
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28.rijna 168, 709 00 Ostrava

tel.:   +420 591 166 214
fax:    +420 596 621 273
mobil:  +420 777 093 799
www.linuxbox.cz

mobil servis: +420 737 238 656
email servis: [email protected]
-------------------------------------
Signature · 13 lines
-------------------------------------
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28.rijna 168, 709 00 Ostrava

tel.:   +420 591 166 214
fax:    +420 596 621 273
mobil:  +420 777 093 799
www.linuxbox.cz

mobil servis: +420 737 238 656
email servis: [email protected]
-------------------------------------