Skip to contentSkip to conversation

replacing drives

DISCUSSION Roberto Nunnari 9 participants 38 messages Apr 26, 2013 – May 10, 2013 MD RAID

lore
Share & tools
lore
#0 lore raw

Hi all.

I'd like to replace two hd in raid1 with larger ones.

I could just add the new drives in raid1 and mount it on /opt after a dump/restore, but I'd prefer to just have to drives instead of four.. less noise and less power consumption and noise.

The question is: what whould be the best way to go? Tricks and tips? Drawbacks? Common errors?

Any hint/advice welcome.
Thank you. :-)

present HD: two WD caviar green 500GB
new HD: two WD caviar green 2TB

root@host1:~# uname -rms
Linux 2.6.32-46-server x86_64
root@host1:~# mdadm --version
mdadm - v2.6.7.1 - 15th October 2008
root@host1:~# cat /proc/mdstat
Personalities : [linear] [raid1] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda2[0] sdb2[1]
      7812032 blocks [2/2] [UU]

md2 : active raid1 sda3[0] sdb3[1]

      431744960 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]

      48827328 blocks [2/2] [UU]

unused devices: <none>

root@host1:~# parted /dev/sda print
Model: ATA WDC WD5000ABPS-0 (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start   End     Size    Type     File system     Flags
 1      1049kB  50.0GB  50.0GB  primary  ext4            boot, raid
 2      50.0GB  58.0GB  8000MB  primary  linux-swap(v1)  raid
 3      58.0GB  500GB   442GB   primary  ext4            raid
root@host1:~# parted /dev/sdb print
Model: ATA WDC WD5000ABPS-0 (scsi)
Disk /dev/sdb: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start   End     Size    Type     File system     Flags
 1      1049kB  50.0GB  50.0GB  primary  ext4            boot, raid
 2      50.0GB  58.0GB  8000MB  primary  linux-swap(v1)  raid
 3      58.0GB  500GB   442GB   primary  ext4            raid
root@host1:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/md0              48060232   6460476  39158392  15% /
none                   3052888       272   3052616   1% /dev
none                   3057876         0   3057876   0% /dev/shm
none                   3057876        88   3057788   1% /var/run
none                   3057876         0   3057876   0% /var/lock
none                   3057876         0   3057876   0% /lib/init/rw
/dev/md2             424970552 399524484   3858820 100% /opt

Thank you and best regards.
Robi

#1 lore raw
Quoted 2 lines

Any hint/advice welcome.
Thank you. :-)

There's a section on extending an existing array on the raid wiki that I believe covers what you're trying to do. I've never had any problems using the approach that's documented there.

https://raid.wiki.kernel.org/index.php/Growing

Regards,

Tregaron

#2 lore raw
In reply to Roberto Nunnari
Quoted 8 lines · Roberto Nunnari

I'd like to replace two hd in raid1 with larger ones.

I could just add the new drives in raid1 and mount it on /opt after a dump/restore, but I'd prefer to just have to drives instead of four.. less noise and less power consumption and noise.

The question is: what whould be the best way to go?

I've never tried it, but mdadm supports growing arrays. If you replace each drive in your RAID1 and do a rebuild, when the second rebuild is complete, you can use the --grow -z max option to tell md to resize the array to use the new space. Then you will need to resize the filesystem using FS tools like resize2fs or xfs_growfs.

--keith

Signature · 1 line
[email protected]
#3 lore raw
In reply to Roberto Nunnari
Quoted 11 lines · Roberto Nunnari

Hi all.

I'd like to replace two hd in raid1 with larger ones.

I could just add the new drives in raid1 and mount it on /opt after a dump/restore, but I'd prefer to just have to drives instead of four.. less noise and less power consumption and noise.

The question is: what whould be the best way to go? Tricks and tips? Drawbacks? Common errors?

Quoted 7 lines

Any hint/advice welcome.
Thank you. :-)

present HD: two WD caviar green 500GB
new HD: two WD caviar green 2TB

I don't think these have SCTERC configuration options, so you'll need to make sure you increase the timeout in the storage stack to prevent read timeouts from causing drives to be prematurely kicked out of the array.

Quoted 4 lines
root@host1:~# uname -rms
Linux 2.6.32-46-server x86_64

That'll be too old for the hot-replacement functionality, but that doesn't make much difference for RAID1 anyway.

Quoted 6 lines
root@host1:~# cat /proc/mdstat
Personalities : [linear] [raid1] [multipath] [raid0] [raid6] [raid5] 
[raid4] [raid10]
md1 : active raid1 sda2[0] sdb2[1]
       7812032 blocks [2/2] [UU]
Quoted 8 lines

md2 : active raid1 sda3[0] sdb3[1]

       431744960 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]

       48827328 blocks [2/2] [UU]

unused devices: <none>

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Then, if you're keeping the same number of partitions but increasing the size:
- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays
or, if you're adding extra partitions:
- create new arrays on extra partitions
- format and mount

If you have hot-plug bays then you can do all this without any downtime (you could also do one disk at a time and just grow the arrays to 3 members), otherwise you'll need to shut down to install and remove the disks. If you only have two bays then you could fail one of the disks then recover to a new one, but that's definitely a risky option.

That's the outline of the process anyway - if you need any details of the actual commands then do ask.

HTH,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#4 lore raw
In reply to Robin Hill
Quoted 15 lines · Robin Hill
Quoted 13 lines · Roberto Nunnari

Hi all.

I'd like to replace two hd in raid1 with larger ones.

I could just add the new drives in raid1 and mount it on /opt after a dump/restore, but I'd prefer to just have to drives instead of four.. less noise and less power consumption and noise.

The question is: what whould be the best way to go? Tricks and tips? Drawbacks? Common errors?

Any hint/advice welcome.
Thank you. :-)

Quoted 6 lines
Quoted 3 lines

present HD: two WD caviar green 500GB
new HD: two WD caviar green 2TB

I don't think these have SCTERC configuration options, so you'll need to make sure you increase the timeout in the storage stack to prevent read timeouts from causing drives to be prematurely kicked out of the array.

How do I increase that timeout?

Also, the old HD are up and running for over 4 years now, and never got any trouble.. just time to time a few warning on /dev/sdb from smartctl:

Device: /dev/sdb, ATA error count increased from 27 to 28

But I don't believe that's something to worry about..

Quoted 6 lines
Quoted 4 lines
root@host1:~# uname -rms
Linux 2.6.32-46-server x86_64

That'll be too old for the hot-replacement functionality, but that doesn't make much difference for RAID1 anyway.

ok.

Quoted 12 lines
Quoted 12 lines
root@host1:~# cat /proc/mdstat
Personalities : [linear] [raid1] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda2[0] sdb2[1]
       7812032 blocks [2/2] [UU]

md2 : active raid1 sda3[0] sdb3[1]

       431744960 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]

       48827328 blocks [2/2] [UU]
Quoted 7 lines
Quoted 2 lines

unused devices: <none>

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)

got till here..

Quoted 1 line

- grow the arrays to 4 members (this avoids any loss of redundancy)

now the next step.. that's a raid1 array.. is it possible to grow the arrays to 4 members?

Thank you!

Robi

#5 lore raw
Quoted 1 line · Roberto Nunnari

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

Signature · 1 line
Mikael Abrahamsson    email: [email protected]
#20 lore raw
In reply to Roberto Nunnari
Quoted 18 lines · Roberto Nunnari
Quoted 17 lines · Robin Hill
Quoted 15 lines · Roberto Nunnari

Hi all.

I'd like to replace two hd in raid1 with larger ones.

I could just add the new drives in raid1 and mount it on /opt after a dump/restore, but I'd prefer to just have to drives instead of four.. less noise and less power consumption and noise.

The question is: what whould be the best way to go? Tricks and tips? Drawbacks? Common errors?

Any hint/advice welcome.
Thank you. :-)

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

present HD: two WD caviar green 500GB
new HD: two WD caviar green 2TB

I don't think these have SCTERC configuration options, so you'll need to make sure you increase the timeout in the storage stack to prevent read timeouts from causing drives to be prematurely kicked out of the array.

How do I increase that timeout?

Mikael's just answered this one.

Quoted 7 lines

Also, the old HD are up and running for over 4 years now, and never got any trouble.. just time to time a few warning on /dev/sdb from smartctl:

Device: /dev/sdb, ATA error count increased from 27 to 28

But I don't believe that's something to worry about..

Probably not. The only counter that's really significant is the number of reallocated sectors. As for not having had any timeout issues before, it does depend on the setup. It may be that the disk manufacturers have increased timeouts on newer disks (the higher data desnity could well increase the odds of getting failures on the first pass), or it may be down to vibrations in the chassis causing problems, etc. It's safer to make sure that the storage subsystem has longer timeouts than the drives anyway.

Quoted 31 lines
Quoted 6 lines
Quoted 4 lines
root@host1:~# uname -rms
Linux 2.6.32-46-server x86_64

That'll be too old for the hot-replacement functionality, but that doesn't make much difference for RAID1 anyway.

ok.

Quoted 13 lines
Quoted 13 lines
root@host1:~# cat /proc/mdstat
Personalities : [linear] [raid1] [multipath] [raid0] [raid6] [raid5] 
[raid4] [raid10]
md1 : active raid1 sda2[0] sdb2[1]
       7812032 blocks [2/2] [UU]

md2 : active raid1 sda3[0] sdb3[1]

       431744960 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]

       48827328 blocks [2/2] [UU]
Quoted 7 lines
Quoted 2 lines

unused devices: <none>

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)

Quoted 8 lines

got till here..

Quoted 1 line

- grow the arrays to 4 members (this avoids any loss of redundancy)

now the next step.. that's a raid1 array.. is it possible to grow the arrays to 4 members?

Yes, there's no problem with running RAID1 arrays with more than two mirrors (with md anyway) - they're all identical so it doesn't really make any difference how many you have.

Cheers,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#21 lore raw
Quoted 10 lines · Robin Hill
Quoted 3 lines · Roberto Nunnari
Quoted 1 line

- grow the arrays to 4 members (this avoids any loss of redundancy)

now the next step.. that's a raid1 array.. is it possible to grow the arrays to 4 members?

Yes, there's no problem with running RAID1 arrays with more than two mirrors (with md anyway) - they're all identical so it doesn't really make any difference how many you have.

Cheers,
Robin

ok.. it's rebuilding.. I started with md0.. I'll wait it finishes and then do md1(8GB) and after that, md2(almost 2TB).. for now it seems to be going well, isn't it?

# mdadm -D /dev/md0
/dev/md0:
        Version : 00.90
  Creation Time : Fri Apr 22 08:20:49 2011
     Raid Level : raid1
     Array Size : 48827328 (46.57 GiB 50.00 GB)
  Used Dev Size : 48827328 (46.57 GiB 50.00 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0
    Persistence : Superblock is persistent

Update Time : Tue Apr 30 16:01:40 2013
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 4
Failed Devices : 0
Spare Devices : 2

Rebuild Status : 15% complete

UUID : 1158db16:ee1fcafc:b6fab772:d376c644
Events : 0.964

Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 33 1 active sync /dev/sdc1
4 8 49 2 spare rebuilding /dev/sdd1
5 8 17 3 spare rebuilding /dev/sdb1

Robi

#6 lore raw
In reply to Mikael Abrahamsson
Quoted 6 lines · Mikael Abrahamsson
Quoted 1 line · Roberto Nunnari

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

You believe that this is not enough? What's the unit? milliseconds?

# for x in /sys/block/sd[a-z] ; do cat $x/device/timeout ; done
30
30
30
30

Thank you.

Robi

#7 lore raw
Quoted 8 lines · Roberto Nunnari
Quoted 6 lines · Mikael Abrahamsson
Quoted 1 line · Roberto Nunnari

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

Quoted 7 lines

You believe that this is not enough? What's the unit? milliseconds?

# for x in /sys/block/sd[a-z] ; do cat $x/device/timeout ; done
30
30
30
30

The units are seconds. According to WD[1], their drives can take up to 2 minutes to timeout, but other manufacturers may differ. 30 seconds is definitely too short.

[1] http://wdc.custhelp.com/app/answers/detail/a_id/1397/p/227,283/session/L3RpbWUvMTMyMTQzOTc4NS9zaWQvdVhvYmpmSms%3D

Cheers,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#22 lore raw
In reply to Roberto Nunnari
Quoted 38 lines · Roberto Nunnari
Quoted 10 lines · Robin Hill
Quoted 3 lines · Roberto Nunnari
Quoted 1 line

- grow the arrays to 4 members (this avoids any loss of redundancy)

now the next step.. that's a raid1 array.. is it possible to grow the arrays to 4 members?

Yes, there's no problem with running RAID1 arrays with more than two mirrors (with md anyway) - they're all identical so it doesn't really make any difference how many you have.

Cheers,
Robin

ok.. it's rebuilding.. I started with md0.. I'll wait it finishes and then do md1(8GB) and after that, md2(almost 2TB).. for now it seems to be going well, isn't it?

# mdadm -D /dev/md0
/dev/md0:
        Version : 00.90
  Creation Time : Fri Apr 22 08:20:49 2011
     Raid Level : raid1
     Array Size : 48827328 (46.57 GiB 50.00 GB)
  Used Dev Size : 48827328 (46.57 GiB 50.00 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0
    Persistence : Superblock is persistent

Update Time : Tue Apr 30 16:01:40 2013
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 4
Failed Devices : 0
Spare Devices : 2

Rebuild Status : 15% complete

UUID : 1158db16:ee1fcafc:b6fab772:d376c644
Events : 0.964

Quoted 8 lines

Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 33 1 active sync /dev/sdc1
4 8 49 2 spare rebuilding /dev/sdd1
5 8 17 3 spare rebuilding /dev/sdb1

Robi

rebuilt and clean! hehehe
Robi

#10 lore raw
In reply to Mikael Abrahamsson
Quoted 5 lines · Mikael Abrahamsson
Quoted 1 line · Roberto Nunnari

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

what the... these are SECONDS! What on earth could delay a sata attached disk read for 30 (now 180) seconds if not a disk failure? I OS hang? Sorry about that question, but I don't understand.. I have never seen such a problem.

Robi

#8 lore raw
In reply to Roberto Nunnari
Quoted 6 lines · Roberto Nunnari

You believe that this is not enough? What's the unit? milliseconds?

# for x in /sys/block/sd[a-z] ; do cat $x/device/timeout ; done
30
30
30
30

The unit is in seconds, and yes, you want the drive to report an error before the kernel drops it, so 180 seconds is sufficient for most drives to give up and return a read error.

Signature · 1 line
Mikael Abrahamsson    email: [email protected]
#11 lore raw
In reply to Roberto Nunnari
Quoted 1 line · Roberto Nunnari

what the... these are SECONDS! What on earth could delay a sata attached disk read for 30 (now 180) seconds if not a disk failure? I OS hang? Sorry about that question, but I don't understand.. I have never seen such a problem.

A consumer drive will spend considerable time trying to read a sector before giving up. I believe 120 seconds is not uncommon.

The raid drives will typically give up after 7 seconds.

http://en.wikipedia.org/wiki/Time-Limited_Error_Recovery

Signature · 1 line
Mikael Abrahamsson    email: [email protected]
#12 lore raw
Quoted 9 lines · Mikael Abrahamsson
Quoted 1 line · Roberto Nunnari

what the... these are SECONDS! What on earth could delay a sata attached disk read for 30 (now 180) seconds if not a disk failure? I OS hang? Sorry about that question, but I don't understand.. I have never seen such a problem.

A consumer drive will spend considerable time trying to read a sector before giving up. I believe 120 seconds is not uncommon.

The raid drives will typically give up after 7 seconds.

http://en.wikipedia.org/wiki/Time-Limited_Error_Recovery

ok.. thank you for the explanation. Now I got it. :-)

Robi

#13 lore raw
In reply to Roberto Nunnari
Quoted 11 lines · Roberto Nunnari
Quoted 5 lines · Mikael Abrahamsson
Quoted 1 line · Roberto Nunnari

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

what the... these are SECONDS! What on earth could delay a sata attached disk read for 30 (now 180) seconds if not a disk failure? I OS hang? Sorry about that question, but I don't understand.. I have never seen such a problem.

The worst horror stories on this mailing list are directly attributable to this problem. Usually after months or even years of apparently trouble-free operation.

Consumer-grade drives intended for desktop usage are not "out-of-the-box" compatible with RAID. Some of them are configurable after each power-up to behave like an enterprise drive. The rest must be accommodated with extended driver timeouts.

Search the list archives for "scterc", "timeout", and "URE" (or combinations thereof).

Phil

#23 lore raw
In reply to Robin Hill
Quoted 4 lines · Robin Hill

Then, if you're keeping the same number of partitions but increasing the size:
- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

This is the most scaring part.. I'd better take a full backup before doing that.. but 500GB.. don't have LVM.. and can't afford a long service stop..

Robi

#14 lore raw
In reply to Phil Turmel

check disk temperature via smartctl
some sata disks stop running after 55C and wait any time (a day if needed) to get temperature down to +-53C and resume operation

2013/4/30 Phil Turmel <[email protected]>:

Quoted 32 lines · Roberto Nunnari
Quoted 11 lines
Quoted 5 lines · Mikael Abrahamsson
Quoted 1 line · Roberto Nunnari

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

what the... these are SECONDS! What on earth could delay a sata attached disk read for 30 (now 180) seconds if not a disk failure? I OS hang? Sorry about that question, but I don't understand.. I have never seen such a problem.

The worst horror stories on this mailing list are directly attributable to this problem. Usually after months or even years of apparently trouble-free operation.

Consumer-grade drives intended for desktop usage are not "out-of-the-box" compatible with RAID. Some of them are configurable after each power-up to behave like an enterprise drive. The rest must be accommodated with extended driver timeouts.

Search the list archives for "scterc", "timeout", and "URE" (or combinations thereof).

Phil

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 1 line
Roberto Spadim
#15 lore raw
Quoted 4 lines · Roberto Spadim

check disk temperature via smartctl
some sata disks stop running after 55C and wait any time (a day if needed) to get temperature down to +-53C and resume operation

G'day Roberto,

Have you got any data or model numbers to back this one up?
I've seen plenty of disks set the "warranty void" flag if they exceed 55C, but I've never seen a disk stop or pause (and I've run some _very_ hot).

I'd be really interested in finding out a bit more about this.

Regards,
Brad

#16 lore raw
Quoted 13 lines · Brad Campbell
Quoted 4 lines · Roberto Spadim

check disk temperature via smartctl
some sata disks stop running after 55C and wait any time (a day if needed) to get temperature down to +-53C and resume operation

G'day Roberto,

Have you got any data or model numbers to back this one up? I've seen plenty of disks set the "warranty void" flag if they exceed 55C, but I've never seen a disk stop or pause (and I've run some _very_ hot).

Quoted 4 lines

I'd be really interested in finding out a bit more about this.

Regards,
Brad

Hi Brad.

No.. Luckly that never was a problem for me.. at present my old drives are 39 and 44 degrees.

Regards.
Robi

#17 lore raw
In reply to Brad Campbell

2013/4/30 Brad Campbell <[email protected]>:

Quoted 9 lines · Roberto Spadim
Quoted 5 lines

check disk temperature via smartctl
some sata disks stop running after 55C and wait any time (a day if needed) to get temperature down to +-53C and resume operation

G'day Roberto,

Quoted 8 lines

Have you got any data or model numbers to back this one up? I've seen plenty of disks set the "warranty void" flag if they exceed 55C, but I've never seen a disk stop or pause (and I've run some _very_ hot).

I'd be really interested in finding out a bit more about this.

Regards,
Brad

i replaced 4 old disk with this 'new' one:
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-2.6.39-ARCH] (local build) Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===

Model Family:     SAMSUNG SpinPoint F3
Device Model:     SAMSUNG HD502HJ
Serial Number:    S2BWJ60B237288
LU WWN Device Id: 5 0024e9 4009a5bf6
Firmware Version: 1AJ10001
User Capacity:    500.107.862.016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 6
Local Time is:    Wed May  1 15:07:29 2013 BRT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever been run.
Total time to complete Offline
data collection: ( 4740) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection
on/off support.
Suspend Offline collection upon new command.
Offline surface scan supported. Self-test supported.
No Conveyance Self-test supported. Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode. Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine

recommended polling time:        (  79) minutes.
SCT capabilities:              (0x003f) SCT Status supported.

SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported.

SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE
UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 051 Pre-fail
Always - 0
2 Throughput_Performance 0x0026 048 048 000 Old_age
Always - 5091
3 Spin_Up_Time 0x0023 082 081 025 Pre-fail
Always - 5638
4 Start_Stop_Count 0x0032 100 100 000 Old_age
Always - 19
5 Reallocated_Sector_Ct 0x0033 252 252 010 Pre-fail
Always - 0
7 Seek_Error_Rate 0x002e 252 252 051 Old_age
Always - 0
8 Seek_Time_Performance 0x0024 252 252 015 Old_age
Offline - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age
Always - 5921
10 Spin_Retry_Count 0x0032 252 252 051 Old_age
Always - 0
11 Calibration_Retry_Count 0x0032 252 252 000 Old_age
Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age
Always - 23
191 G-Sense_Error_Rate 0x0022 252 252 000 Old_age
Always - 0
192 Power-Off_Retract_Count 0x0022 252 252 000 Old_age
Always - 0
194 Temperature_Celsius 0x0002 064 056 000 Old_age
Always - 27 (Min/Max 15/45)
195 Hardware_ECC_Recovered 0x003a 100 100 000 Old_age
Always - 0
196 Reallocated_Event_Count 0x0032 252 252 000 Old_age
Always - 0
197 Current_Pending_Sector 0x0032 252 252 000 Old_age
Always - 0
198 Offline_Uncorrectable 0x0030 252 252 000 Old_age
Offline - 0
199 UDMA_CRC_Error_Count 0x0036 200 200 000 Old_age
Always - 0
200 Multi_Zone_Error_Rate 0x002a 100 100 000 Old_age
Always - 2
223 Load_Retry_Count 0x0032 252 252 000 Old_age
Always - 0
225 Load_Cycle_Count 0x0032 100 100 000 Old_age
Always - 23

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining
LifeTime(hours) LBA_of_first_error

# 1  Extended offline    Completed without error       00%      4074         -
# 2  Extended offline    Completed without error       00%       383         -

Note: selective self-test log revision number (0) not 1 implies that no selective self-test has ever been run
SMART Selective self-test log data structure revision number 0
Note: revision number not 1 implies that no selective self-test has ever been run

 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Completed [00% left] (0-65535)
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing

Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

this disk model (SAMSUNG HD502HJ) stop and wait temperature get down again (they have about 6months of heavy read, and are not enterprise level), i have about 20 units, i replace every 6months, they are cheap and never lost information with this 6months interval, but i'm not surelly safe without a raid1 and backups, and of course it's a enterprise level system running not enterprise level hardware (ok it's wrong but still working)
after 6months i replace the disk, i replace the first with 6months the second with 7, the third with 8 and the fourth with 9, after 6 months i replace normally each disk (the older have 9 months on the first replace), i had no problem yet, it's a raid1 4 disks setup running mariadb/mysql database with high read and low write, raid0/raid10 don't work in this workload

--
Roberto Spadim

#24 lore raw
In reply to Robin Hill
Quoted 8 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks

Ok.. got here.

Quoted 1 line

- fail and remove the old disk partitions from the arrays

Now.. I may need some guidance for the next steps.. please correct me if I'm wrong..

to fail the old disk partition from the arrays I should:
mdadm -f /dev/md0 /dev/sda1
mdadm -f /dev/md1 /dev/sda2
mdadm -f /dev/md2 /dev/sda3
mdadm -f /dev/md0 /dev/sdc1
mdadm -f /dev/md1 /dev/sdc2
mdadm -f /dev/md2 /dev/sdc3

and to remove the old disk partition from the arrays I should:
mdadm -r /dev/md0 /dev/sda1
mdadm -r /dev/md1 /dev/sda2
mdadm -r /dev/md2 /dev/sda3
mdadm -r /dev/md0 /dev/sdc1
mdadm -r /dev/md1 /dev/sdc2
mdadm -r /dev/md2 /dev/sdc3

correct?

Quoted 1 line

- shrink the arrays back down to 2 members

to shrink the arrays back down to two members:
mdadm --grow --raid-devices=2 /dev/md0 mdadm --grow --raid-devices=2 /dev/md1 mdadm --grow --raid-devices=2 /dev/md2

correct?

Thank you very much for your precious help!

Robi

#25 lore raw
Quoted 34 lines · Roberto Nunnari
Quoted 8 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks

Ok.. got here.

Quoted 1 line

- fail and remove the old disk partitions from the arrays

Now.. I may need some guidance for the next steps.. please correct me if I'm wrong..

to fail the old disk partition from the arrays I should:
mdadm -f /dev/md0 /dev/sda1
mdadm -f /dev/md1 /dev/sda2
mdadm -f /dev/md2 /dev/sda3
mdadm -f /dev/md0 /dev/sdc1
mdadm -f /dev/md1 /dev/sdc2
mdadm -f /dev/md2 /dev/sdc3

and to remove the old disk partition from the arrays I should:
mdadm -r /dev/md0 /dev/sda1
mdadm -r /dev/md1 /dev/sda2
mdadm -r /dev/md2 /dev/sda3
mdadm -r /dev/md0 /dev/sdc1
mdadm -r /dev/md1 /dev/sdc2
mdadm -r /dev/md2 /dev/sdc3

Quoted 2 lines

correct?

Assuming sda & sdc are your old drives, yes (they were sda & sdb in your original mail, but possibly they've been reordered if you've rebooted with the new drives in).

Quoted 10 lines
Quoted 2 lines

- shrink the arrays back down to 2 members

to shrink the arrays back down to two members:
mdadm --grow --raid-devices=2 /dev/md0 mdadm --grow --raid-devices=2 /dev/md1 mdadm --grow --raid-devices=2 /dev/md2

correct?

Yes, that's all correct.

Cheers,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#26 lore raw
Quoted 33 lines · Robin Hill
Quoted 31 lines · Roberto Nunnari
Quoted 8 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks

Ok.. got here.

Quoted 1 line

- fail and remove the old disk partitions from the arrays

Now.. I may need some guidance for the next steps.. please correct me if I'm wrong..

to fail the old disk partition from the arrays I should:
mdadm -f /dev/md0 /dev/sda1
mdadm -f /dev/md1 /dev/sda2
mdadm -f /dev/md2 /dev/sda3
mdadm -f /dev/md0 /dev/sdc1
mdadm -f /dev/md1 /dev/sdc2
mdadm -f /dev/md2 /dev/sdc3

and to remove the old disk partition from the arrays I should:
mdadm -r /dev/md0 /dev/sda1
mdadm -r /dev/md1 /dev/sda2
mdadm -r /dev/md2 /dev/sda3
mdadm -r /dev/md0 /dev/sdc1
mdadm -r /dev/md1 /dev/sdc2
mdadm -r /dev/md2 /dev/sdc3

Quoted 5 lines
Quoted 2 lines

correct?

Assuming sda & sdc are your old drives, yes (they were sda & sdb in your original mail, but possibly they've been reordered if you've rebooted with the new drives in).

yes.. they've been reordered.

Any checks I should run here, or can I relay on error messages evenctually printed on screen when the commands are run?

Thanks.
Robi

Quoted 13 lines
Quoted 9 lines
Quoted 2 lines

- shrink the arrays back down to 2 members

to shrink the arrays back down to two members:
mdadm --grow --raid-devices=2 /dev/md0 mdadm --grow --raid-devices=2 /dev/md1 mdadm --grow --raid-devices=2 /dev/md2

correct?

Yes, that's all correct.

Cheers,
Robin

#19 lore raw
In reply to Mikael Abrahamsson

----- Opprinnelig melding -----

Quoted 5 lines · Roberto Nunnari
Quoted 2 lines

How do I increase that timeout?

for x in /sys/block/sd[a-z] ; do echo 180 > $x/device/timeout ; done

I have this in my /etc/rc.local to attempt to enable scterc and fall back to increasing timeout when it fails (on my wd black drives)

for i in b c d e f g h
do

    dev=sd$i
    smartctl -l scterc,70,70 /dev/$dev || echo 180 > /sys/block/$dev/device/timeout

done

Signature · 15 lines
Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#18 lore raw
In reply to Roberto Spadim
Quoted 4 lines

SCT capabilities: (0x003f) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported.
SCT Data Table supported.

drive supports SCT, so enable it. See the script I just posted (from my rc.local)

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#27 lore raw
In reply to Robin Hill
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

and

resize2fs /dev/md0
resize2fs /dev/md1
resize2fs /dev/md2

Correct?

.. I still have a couple of questions:

1) how do I know if there's a bitmap?

2) at present /dev/md2 usage is 100%.. could that cause any problem?

3) the new drives are 2TG drives.. As around one year ago had trouble on linux (it was a server dated 2006 with CentOS 5) that would not handle drives larger than 2TB.. I wander what happens if one day one drive fails and the drive I'll buy to replace will be sold as 2TB but in reality slightly larger than 2TB.. what will happen? Will linux fail again to use a drive larger than 2TB? At present I'm on ubuntu 10.04, all software from standard distribution.

Pitfalls I should know?

Thank you very much
Robi

#28 lore raw
Quoted 48 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

and

resize2fs /dev/md0
resize2fs /dev/md1
resize2fs /dev/md2

Correct?

.. I still have a couple of questions:

1) how do I know if there's a bitmap?

2) at present /dev/md2 usage is 100%.. could that cause any problem?

Quoted 7 lines

3) the new drives are 2TG drives.. As around one year ago had trouble on linux (it was a server dated 2006 with CentOS 5) that would not handle drives larger than 2TB.. I wander what happens if one day one drive fails and the drive I'll buy to replace will be sold as 2TB but in reality slightly larger than 2TB.. what will happen? Will linux fail again to use a drive larger than 2TB? At present I'm on ubuntu 10.04, all software from standard distribution.

Pitfalls I should know?

Thank you very much
Robi

Anybody on this, please?

I'd really appreciate some guidance, especially for these last steps.

Thank you and best regards.
Robi

#29 lore raw
In reply to Roberto Nunnari
Quoted 21 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Quoted 7 lines

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

Yep, that's right. Make sure they've actually grown to the correct size before you progress though - I have had one occasion where using --size=max actually ended up shrinking the array and I had to manually work out the size to use in order to recover. That was using an older version of mdadm though, and I've not seen it happen since.

Quoted 6 lines

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

You'll need 'fsck -f' here to force it to run.

Quoted 8 lines

and

resize2fs /dev/md0
resize2fs /dev/md1
resize2fs /dev/md2

Correct?

That should be it, yes.

Quoted 5 lines

.. I still have a couple of questions:

1) how do I know if there's a bitmap?

Check /proc/mdstat - it'll report a bitmap - e.g. md6 : active raid6 sdg[0] sdf[6] sde[5] sdi[2] sdh[1]

      11721052272 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5] [UUUUU]
      bitmap: 0/30 pages [0KB], 65536KB chunk
Quoted 2 lines

2) at present /dev/md2 usage is 100%.. could that cause any problem?

It'll slow things down a bit but otherwise shouldn't be an issue.

Quoted 7 lines

3) the new drives are 2TG drives.. As around one year ago had trouble on linux (it was a server dated 2006 with CentOS 5) that would not handle drives larger than 2TB.. I wander what happens if one day one drive fails and the drive I'll buy to replace will be sold as 2TB but in reality slightly larger than 2TB.. what will happen? Will linux fail again to use a drive larger than 2TB?

All 2TB drives are exactly the same size. Since somewhere around the 320G/500G mark, all drive manufacturers have agreed to standardise the drive sizes, so getting mismatches like this is a thing of the past.

Quoted 4 lines

At present I'm on ubuntu 10.04, all software from standard distribution.

Pitfalls I should know?

You'll need to use GPT partitions instead of standard MBR partitions for drives over 2TB, but there shouldn't be any issue with handling them.

Cheers,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#30 lore raw
Quoted 81 lines · Robin Hill
Quoted 21 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Quoted 7 lines

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

Yep, that's right. Make sure they've actually grown to the correct size before you progress though - I have had one occasion where using --size=max actually ended up shrinking the array and I had to manually work out the size to use in order to recover. That was using an older version of mdadm though, and I've not seen it happen since.

Quoted 6 lines

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

You'll need 'fsck -f' here to force it to run.

Quoted 8 lines

and

resize2fs /dev/md0
resize2fs /dev/md1
resize2fs /dev/md2

Correct?

That should be it, yes.

Quoted 5 lines

.. I still have a couple of questions:

1) how do I know if there's a bitmap?

Check /proc/mdstat - it'll report a bitmap - e.g. md6 : active raid6 sdg[0] sdf[6] sde[5] sdi[2] sdh[1]

       11721052272 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5] [UUUUU]
       bitmap: 0/30 pages [0KB], 65536KB chunk
Quoted 2 lines

2) at present /dev/md2 usage is 100%.. could that cause any problem?

It'll slow things down a bit but otherwise shouldn't be an issue.

Quoted 7 lines

3) the new drives are 2TG drives.. As around one year ago had trouble on linux (it was a server dated 2006 with CentOS 5) that would not handle drives larger than 2TB.. I wander what happens if one day one drive fails and the drive I'll buy to replace will be sold as 2TB but in reality slightly larger than 2TB.. what will happen? Will linux fail again to use a drive larger than 2TB?

All 2TB drives are exactly the same size. Since somewhere around the 320G/500G mark, all drive manufacturers have agreed to standardise the drive sizes, so getting mismatches like this is a thing of the past.

Quoted 2 lines

At present I'm on ubuntu 10.04, all software from standard distribution.

Quoted 8 lines
Quoted 2 lines

Pitfalls I should know?

You'll need to use GPT partitions instead of standard MBR partitions for drives over 2TB, but there shouldn't be any issue with handling them.

Cheers,
Robin

Thank you Robin.
Today I'm on holiday, but I will look at it tomorrow. :-) Best regards.
Robi

#31 lore raw
In reply to Robin Hill
Quoted 33 lines · Robin Hill
Quoted 17 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 8 lines
Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

Yep, that's right. Make sure they've actually grown to the correct size before you progress though - I have had one occasion where using --size=max actually ended up shrinking the array and I had to manually work out the size to use in order to recover. That was using an older version of mdadm though, and I've not seen it happen since.

Quoted 7 lines
Quoted 6 lines

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

You'll need 'fsck -f' here to force it to run.

humm.. as /dev/md0 is mounted on / I probably should boot from a cd, and run fsck and resize2fs from there.. maybe using UUIDs, right?

Quoted 10 lines
Quoted 9 lines

and

resize2fs /dev/md0
resize2fs /dev/md1
resize2fs /dev/md2

Correct?

That should be it, yes.

Thank you.
Robi

#32 lore raw
Quoted 37 lines · Roberto Nunnari
Quoted 36 lines · Robin Hill
Quoted 17 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 9 lines
Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

Yep, that's right. Make sure they've actually grown to the correct size before you progress though - I have had one occasion where using --size=max actually ended up shrinking the array and I had to manually work out the size to use in order to recover. That was using an older version of mdadm though, and I've not seen it happen since.

Quoted 2 lines

and after that

Quoted 9 lines
Quoted 5 lines
Quoted 4 lines

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

You'll need 'fsck -f' here to force it to run.

humm.. as /dev/md0 is mounted on / I probably should boot from a cd, and run fsck and resize2fs from there.. maybe using UUIDs, right?

You can just skip the fsck and run resize2fs - it'll work fine on a mounted filesystem. It'll probably be safer to do it offline though.

Cheers,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#33 lore raw
Quoted 42 lines · Robin Hill
Quoted 40 lines · Roberto Nunnari
Quoted 39 lines · Robin Hill
Quoted 17 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 8 lines
Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

Yep, that's right. Make sure they've actually grown to the correct size before you progress though - I have had one occasion where using --size=max actually ended up shrinking the array and I had to manually work out the size to use in order to recover. That was using an older version of mdadm though, and I've not seen it happen since.

Quoted 6 lines

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

You'll need 'fsck -f' here to force it to run.

humm.. as /dev/md0 is mounted on / I probably should boot from a cd, and run fsck and resize2fs from there.. maybe using UUIDs, right?

You can just skip the fsck and run resize2fs - it'll work fine on a mounted filesystem. It'll probably be safer to do it offline though.

Cheers,
Robin

I'd rather stay on the safe side.. how do I assemble the array if I boot from a cd?

something like:

mdadm --scan --assemble --uuid=a26bf396:31389f83:0df1722d:f404fe4c

would to the job and let me with a /dev/mdX I will be able to work with (fsck and resize2fs)?

Thank you.
Robi

#34 lore raw
Quoted 57 lines · Roberto Nunnari
Quoted 44 lines · Robin Hill
Quoted 42 lines · Roberto Nunnari
Quoted 41 lines · Robin Hill
Quoted 17 lines · Roberto Nunnari
Quoted 12 lines · Robin Hill

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Quoted 2 lines

Then, if you're keeping the same number of partitions but increasing the size:

Ok.. got here.

Quoted 9 lines
Quoted 2 lines

- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays

Now the scary part.. so.. here I believe I should give the following commands:

mdadm --grow /dev/md0 --size=max
mdadm --grow /dev/md1 --size=max
mdadm --grow /dev/md2 --size=max

Yep, that's right. Make sure they've actually grown to the correct size before you progress though - I have had one occasion where using --size=max actually ended up shrinking the array and I had to manually work out the size to use in order to recover. That was using an older version of mdadm though, and I've not seen it happen since.

Quoted 6 lines

and after that

fsck /dev/md0
fsck /dev/md1
fsck /dev/md2

You'll need 'fsck -f' here to force it to run.

Quoted 8 lines
Quoted 3 lines

humm.. as /dev/md0 is mounted on / I probably should boot from a cd, and run fsck and resize2fs from there.. maybe using UUIDs, right?

You can just skip the fsck and run resize2fs - it'll work fine on a mounted filesystem. It'll probably be safer to do it offline though.

Cheers,
Robin

I'd rather stay on the safe side.. how do I assemble the array if I boot from a cd?

Quoted 7 lines

something like:

mdadm --scan --assemble --uuid=a26bf396:31389f83:0df1722d:f404fe4c

would to the job and let me with a /dev/mdX I will be able to work with (fsck and resize2fs)?

That should do it, yes. If not, you can always do it explicitly with:

    mdadm -A /dev/md0 /dev/sd[abcd]1

You'd need to double-check what the device names end up as though.

Cheers,
Robin

Signature · 4 lines
     ___        
    ( ' }     |       Robin Hill        <[email protected]> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |
#35 lore raw
Quoted 9 lines · Robin Hill
Quoted 8 lines · Roberto Nunnari

I'd rather stay on the safe side.. how do I assemble the array if I boot from a cd?

something like:

mdadm --scan --assemble --uuid=a26bf396:31389f83:0df1722d:f404fe4c

would to the job and let me with a /dev/mdX I will be able to work with (fsck and resize2fs)?

Quoted 7 lines

That should do it, yes. If not, you can always do it explicitly with:

    mdadm -A /dev/md0 /dev/sd[abcd]1

You'd need to double-check what the device names end up as though.

Cheers,
Robin

Thank you. :-)
Robi

#36 lore raw
In reply to Robin Hill
Quoted 71 lines · Robin Hill
Quoted 11 lines · Roberto Nunnari

Hi all.

I'd like to replace two hd in raid1 with larger ones.

I could just add the new drives in raid1 and mount it on /opt after a dump/restore, but I'd prefer to just have to drives instead of four.. less noise and less power consumption and noise.

The question is: what whould be the best way to go? Tricks and tips? Drawbacks? Common errors?

Quoted 7 lines

Any hint/advice welcome.
Thank you. :-)

present HD: two WD caviar green 500GB
new HD: two WD caviar green 2TB

I don't think these have SCTERC configuration options, so you'll need to make sure you increase the timeout in the storage stack to prevent read timeouts from causing drives to be prematurely kicked out of the array.

Quoted 4 lines
root@host1:~# uname -rms
Linux 2.6.32-46-server x86_64

That'll be too old for the hot-replacement functionality, but that doesn't make much difference for RAID1 anyway.

Quoted 6 lines
root@host1:~# cat /proc/mdstat
Personalities : [linear] [raid1] [multipath] [raid0] [raid6] [raid5]
[raid4] [raid10]
md1 : active raid1 sda2[0] sdb2[1]
        7812032 blocks [2/2] [UU]
Quoted 8 lines

md2 : active raid1 sda3[0] sdb3[1]

        431744960 blocks [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]

        48827328 blocks [2/2] [UU]

unused devices: <none>

The safest option would be:
- add in the new disks
- partition to at least the same size as your existing partitions (they can be larger)
- add the new partitions into the arrays (they'll go in as spares)
- grow the arrays to 4 members (this avoids any loss of redundancy)
- wait for the resync to complete
- install grub/lilo/syslinux to the new disks
- fail and remove the old disk partitions from the arrays
- shrink the arrays back down to 2 members
- remove the old disks

Then, if you're keeping the same number of partitions but increasing the size:
- grow the arrays to fill the partitions
- grow the filesystems to fill the arrays
or, if you're adding extra partitions:
- create new arrays on extra partitions
- format and mount

If you have hot-plug bays then you can do all this without any downtime (you could also do one disk at a time and just grow the arrays to 3 members), otherwise you'll need to shut down to install and remove the disks. If you only have two bays then you could fail one of the disks then recover to a new one, but that's definitely a risky option.

Quoted 6 lines

That's the outline of the process anyway - if you need any details of the actual commands then do ask.

HTH,
Robin

Done!

linux-raid is GREAT!! I was worried.. but it all went very smooth and my system is up and running with very little downtime! mdadm is very flexible and powerful!

Thank you Robin for your support!
Best regards.
Robi