checking md device parity (forced resync) - is it necessary?
Lately I installed Debian on a Thecus n4100 machine. It's a 600 MHz ARM storage device, and has 4 x 400 GB drives.
I made Linux software RAID on these drives:
- RAID-1 - ~1 GB for system (/)
- RAID-1 - ~1 GB for swap
- RAID-10 - ~798 GB for iSCSI storage
I noticed that each day the device slows down; a quick investigation discovered that Debian runs a "checkarray" script each night at 1 am (via cron). The essence of "checkarray" script is basically this:
echo check > /sys/block/$dev/md/sync_action
Which starts a resync of drives. As one can imagine, resync of 800 GB on a rather slow device (600 MHz ARM) can take 12 hours or so...
So my question is: is this "daily forced resync" necessary?
Perhaps in some cases, yes, because someone wrote that tool which does it daily.
On the other hand, if we consider Linux software RAID stable, such a resync would be only needed in some rare situations.
When can one need to run a "daily forced resync", and in which circumstances?
Signature · 1 line
Tomasz Chmielewski
Quoted 9 lines · On Tuesday September 5, [email protected] wrote:
Lately I installed Debian on a Thecus n4100 machine. It's a 600 MHz ARM storage device, and has 4 x 400 GB drives.
I made Linux software RAID on these drives:
- RAID-1 - ~1 GB for system (/)
- RAID-1 - ~1 GB for swap
- RAID-10 - ~798 GB for iSCSI storage
Quoted 9 lines
I noticed that each day the device slows down; a quick investigation discovered that Debian runs a "checkarray" script each night at 1 am (via cron). The essence of "checkarray" script is basically this:
echo check >
/sys/block/$dev/md/sync_actionWhich starts a resync of drives. As one can imagine, resync of 800 GB on a rather slow device (600 MHz ARM) can take 12 hours or so...
I believe that was intended to be once a month, not once a day. Slight error in crontab.
Quoted 2 lines
So my question is: is this "daily forced resync" necessary?
Daily is probably excessive, certainly on an array that size.
Monthly is good. Weekly might be justified on cheap (i.e. unreliable) drives and very critical data.
With RAID, sleeping bad blocks can be bad. If you hit one while recovering a failed drive, you have to put the piece back together by hand.
A regular check can wake up those sleeping bad blocks.
Quoted 6 lines
Perhaps in some cases, yes, because someone wrote that tool which does it daily.
On the other hand, if we consider Linux software RAID stable, such a resync would be only needed in some rare situations.
It has little to do with the stability of Linux software RAID and a lot to do with stability of modern disk drives.
Quoted 3 lines
When can one need to run a "daily forced resync", and in which circumstances?
As I said, I think the 'daily' is an error. What exactly do you have in crontab??
NeilBrown
Neil Brown wrote:
(...)
Quoted 10 lines
Quoted 2 lines
Which starts a resync of drives. As one can imagine, resync of 800 GB on a rather slow device (600 MHz ARM) can take 12 hours or so...
I believe that was intended to be once a month, not once a day. Slight error in crontab.
Quoted 1 line
So my question is: is this "daily forced resync" necessary?
Daily is probably excessive, certainly on an array that size.
Quoted 7 lines
Monthly is good. Weekly might be justified on cheap (i.e. unreliable) drives and very critical data.
With RAID, sleeping bad blocks can be bad. If you hit one while recovering a failed drive, you have to put the piece back together by hand.
A regular check can wake up those sleeping bad blocks.
Thanks a lot for clarification.
Quoted 4 lines
Quoted 1 line
When can one need to run a "daily forced resync", and in which circumstances?
As I said, I think the 'daily' is an error. What exactly do you have in crontab??
Indeed, the crontab entry is wrong:
# by default, run at 01:06 on the first Sunday of each month. 6 1 1-7 * 7 root [ -x /usr/share/mdadm/checkarray ] && /usr/share/mdadm/checkarray --cron --all --quiet
However, it will run at 01:06, on 1st-7th day of each month, and on Sundays (Debian etch).
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 4 lines · Tomasz Chmielewski
# by default, run at 01:06 on the first Sunday of each month. 6 1 1-7 * 7 root [ -x /usr/share/mdadm/checkarray ] && /usr/share/mdadm/checkarray --cron --all --quietHowever, it will run at 01:06, on 1st-7th day of each month, and on Sundays (Debian etch).
hihihi
monthday and weekday are or-ed in crontab
L.
Signature · 6 lines
Luca Berra -- [email protected] Communication Media & Services S.r.l. /"\ \ / ASCII RIBBON CAMPAIGN X AGAINST HTML MAIL / \
Quoted 2 lines · Tomasz Chmielewski
# by default, run at 01:06 on the first Sunday of each month. 6 1 1-7 * 7 root [ -x /usr/share/mdadm/checkarray ] &&
You have a quite old version of mdadm. This issue has been fixed in mdadm (2.5.2-8) unstable from 27 Jul 2006. Current version is mdadm (2.5.3.git200608202239-2) unstable from 23 Aug 2006.
Since the feature did just appear in mdadm (2.5.2-3) UNRELEASED from 7 Jul 2006 (and even just went to unstable with mdadm (2.5.2-7) unstable from 20 Jul 2006), it seems you were simply in quite bad luck :)
regards
Mario
Signature · 3 lines
It is practically impossible to teach good programming style to students that have had prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. -- Dijkstra
Quoted 11 lines · Mario 'BitKoenig' Holbe
Quoted 2 lines · Tomasz Chmielewski
# by default, run at 01:06 on the first Sunday of each month. 6 1 1-7 * 7 root [ -x /usr/share/mdadm/checkarray ] &&You have a quite old version of mdadm. This issue has been fixed in mdadm (2.5.2-8) unstable from 27 Jul 2006. Current version is mdadm (2.5.3.git200608202239-2) unstable from 23 Aug 2006.
Since the feature did just appear in mdadm (2.5.2-3) UNRELEASED from 7 Jul 2006 (and even just went to unstable with mdadm (2.5.2-7) unstable from 20 Jul 2006), it seems you were simply in quite bad luck :)
Yes, I saw somewhere in the changelogs that the issue is fixed.
Still, I can't find 2.5.2-8 anywhere on the mirrors, perhaps it's just a matter of time...
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Tomasz> Lately I installed Debian on a Thecus n4100 machine. It's a Tomasz> 600 MHz ARM storage device, and has 4 x 400 GB drives.
Interesting box... how quiet is it? I'm thinking of one of these for home use, but I'll probably go with an EPIA box so I can actually setup a NFS/CiFS/backup/http/mysql server...
I've looked at the Irfant and the Buffalo Logic ones as well... all tempting. But backups are the killer. :]
Thanks,
John
Quoted 4 lines · John Stoffel
Tomasz> Lately I installed Debian on a Thecus n4100 machine. It's a Tomasz> 600 MHz ARM storage device, and has 4 x 400 GB drives.
Interesting box... how quiet is it?
I can't tell you if it's quiet or not, it's in the server room right now, and serves as a backup and iSCSI storage for Xen.
But as I still configured it and it was on my desk, it seemed a bit louder than a normal PC. After all, it has 4 drives (which you have to buy separately).
Quoted 3 lines
I'm thinking of one of these for
home use, but I'll probably go with an EPIA box so I can actually setup a NFS/CiFS/backup/http/mysql server...
Although it has two gigabit network cards, you can't fully use them - the IXP3xx ARM CPU (600 MHz) allows you to only send about 25 MB/s of packets (measured on a loopback interface, so iSCSI LAN speeds are a bit lower).
Quoted 2 lines
I've looked at the Irfant and the Buffalo Logic ones as well... all tempting. But backups are the killer. :]
There's also Thecus n5200, 800 MHz mobile Celeron, which you can supply with 5 drives... :)
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org