Skip to contentSkip to conversation

how to which to bigger disk (md can do it?)

DISCUSSION Farkas Levente 5 participants 9 messages Nov 16, 2003 – Nov 17, 2003 MD RAID

lore
Share & tools
lore
#0 lore raw

hi,
we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd and software raid-5. the 1TB disk is running out of the space so we decided to switch to 200 GB hard disks. unfortunately there is only one day (saturday) when we can turn off the machine. how we can do it in the fastest way?:
- put the 8 disk to another server and run rsync through two gigabit ethernet card. but it can takes more than a day to sync about 800GB.
- switch the disks one-by-one in the current server and leave linux's software raid to sync the data. in this case we only have to stop the system a few mintues per day and takes 8 days, but continous operation. does it possible? after I put all new hard disk the totel capacity will be (8-1)*200 GB or it will remain (8-1)*120 GB? since during the disk switch the total space is the smallest hd's*(n-1), but what happens when I put the last 200GB disk?
- dd (or somehow imageing) the one 120GB disk content to one 200 GB disk. if they are on the same ide bus (or better on the same 3ware controller) the cloning is much faster. can I do a
dd if=/dev/sda1 of=/dev/sdb1
if they are different size and after the dd the second will be 200 GB raid component?
- rsync between two local disk?
any other or better solution?
thanks in advance.
yours.

Signature · 1 line
  Levente                               "Si vis pacem para bellum!"
#1 lore raw
Quoted 3 lines · Farkas Levente

hi,
we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd and software raid-5. the 1TB disk is running out of the space so we

Are you sure about the Software RAID on this 3ware adapater?? I was under the impression it's a HW Raid adapter?
(I have a preference to HW RAID if available...)

Quoted 2 lines

- put the 8 disk to another server and run rsync through two gigabit ethernet card. but it can takes more than a day to sync about 800GB.

You could do this initially during the week before the swapout, and then just update it before the swap out. Something te note, if it's lots of files, I'd advise to run multiple rsyncs over smaller directories, as it speeds up the rsync processing alot.

Quoted 7 lines

- switch the disks one-by-one in the current server and leave linux's software raid to sync the data. in this case we only have to stop the system a few mintues per day and takes 8 days, but continous operation. does it possible? after I put all new hard disk the totel capacity will be (8-1)*200 GB or it will remain (8-1)*120 GB? since during the disk switch the total space is the smallest hd's*(n-1), but what happens when I put the last 200GB disk?

If it's SW raid, it'll be based on the partitions, so you'll have to create the partitions bigger than the current 120GB partitions. Then after the last disk, the size would be the bigger size, however you'll then need to run a growfs (or similar) for your filesystem to expand it to the bigger size.

Fr HW Raid, it'll depend on the controller.

Hendrik

#2 lore raw
Quoted 9 lines · Hendrik Visage
Quoted 2 lines · Farkas Levente

hi,
we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd and software raid-5. the 1TB disk is running out of the space so we

Are you sure about the Software RAID on this 3ware adapater?? I was under the impression it's a HW Raid adapter?
(I have a preference to HW RAID if available...)

we've got a kernel creash with hardware raid 5 what's more software raid 5 seems to be faster.

Quoted 8 lines
Quoted 2 lines

- put the 8 disk to another server and run rsync through two gigabit ethernet card. but it can takes more than a day to sync about 800GB.

You could do this initially during the week before the swapout, and then just update it before the swap out. Something te note, if it's lots of files, I'd advise to run multiple rsyncs over smaller directories, as it speeds up the rsync processing alot.

can't help since this is the backup server and almost everything changing on it.

Quoted 8 lines
Quoted 2 lines

- switch the disks one-by-one in the current server and leave linux's
software raid to sync the data. in this case we only have to stop the system a few mintues per day and takes 8 days, but continous operation. does it possible? after I put all new hard disk the totel capacity will be (8-1)*200 GB or it will remain (8-1)*120 GB? since during the disk switch the total space is the smallest hd's*(n-1), but what happens when I put the last 200GB disk?

If it's SW raid, it'll be based on the partitions, so you'll have to create the partitions bigger than the current 120GB partitions. Then after the last disk, the size would be the bigger size, however you'll then need to run a growfs (or similar) for your filesystem to expand it to the bigger size.

which partitions? it's not clear to me. we've got :
- 8 pieces of 120 GB Linux raid autodetect (/dev/sda1, /dev/sdb1 etc)
- 1 pieces of 840 GB ext3 (/dev/md2)
so which partition should I have to create and grow?
but what I'd like to prefere the cloning. is there any way to 'rsync' between a 120 GB and 200 GB disk which is Linux raid autodetect (and not eg. ext2)?

another good question anybody use 200 GB maxtor hd with 3ware 7500-8 controller?

thanks.

Signature · 1 line
  Levente                               "Si vis pacem para bellum!"
#3 lore raw
In reply to Hendrik Visage

With regards to your message at 06:31 AM 11/16/03, Hendrik Visage. Where you stated:

Quoted 8 lines · Farkas Levente
Quoted 3 lines

hi,
we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd and software raid-5. the 1TB disk is running out of the space so we

Are you sure about the Software RAID on this 3ware adapater?? I was under the impression it's a HW Raid adapter?
(I have a preference to HW RAID if available...)

The 3Ware is a very good hardware RAID device.
That is irrelevant in this context.
One of the many advantages of software RAID is the ability to use multiple controllers in separate PCI slots ( and in many cases on multiple PCI buses) to accomplish much better throughput and I/O rates. I could perhaps agree with your preference for HWare RAID IF:
1) It could be spanned over multiple controllers
2) It could allow one to create non-RAID boot partitions on the disks used.
3) It could compete for RAID5 performance with software RAID on a modern machine.

With our best regards,

Maurice W. Hilarius Telephone: 01-780-456-9771
Hard Data Ltd. FAX: 01-780-456-9772
11060 - 166 Avenue mailto:[email protected]
Edmonton, AB, Canada http://www.harddata.com/
T5X 1Y3

#4 lore raw
In reply to Farkas Levente
Quoted 3 lines · Farkas Levente

hi,
- put the 8 disk to another server and run rsync through two gigabit ethernet card. but it can takes more than a day to sync about 800GB.

well rsync is incremental, you could do an initial rsync during the working week and do another to upgrade deltas during the weekend.

Quoted 2 lines

- switch the disks one-by-one in the current server and leave linux's software raid to sync the data. in this case we only have to stop the

in this case you will have disks used up to 120Gb and an 80 gb of unused space at the end. you could create another raid5 there, but it does not seem appealing to me.

Really weird idea: please do not do it on real data.
You will probably shoot yourself doing this
- Switch the disks one by one.
- When you are over create a new raid5 over the existing one (keeping the same settings for stripe size and parity algo)
- Resize the filesystem.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#5 lore raw
Quoted 8 lines · On Monday November 17, [email protected] wrote:

Really weird idea: please do not do it on real data.
You will probably shoot yourself doing this
- Switch the disks one by one.
- When you are over create a new raid5 over the existing one (keeping the same settings for stripe size and parity algo)
- Resize the filesystem.

Not so weird - it is exactly what I was going to suggest. Providing you get the chunk size, parity algoithm and device order right, and use "--force" to make sure mdadm doesn't re-arrange the devices on on it should work perfectly.

Somewhere low-down on my todo list for raid is allowing this to be done on-line (mdadm /dev/md0 --size=max). It is right along side allowing the number of live devices in a raid1 to be changed (mdadm /dev/md1 --devices=3) and adding devices to a linear array (mdadm /dev/md2 --add /dev/sdd).

NeilBrown

#6 lore raw
Quoted 11 lines · Neil Brown
Quoted 7 lines · On Monday November 17, [email protected] wrote:

Really weird idea: please do not do it on real data.
You will probably shoot yourself doing this
- Switch the disks one by one.
- When you are over create a new raid5 over the existing one (keeping the same settings for stripe size and parity algo)
- Resize the filesystem.

Quoted 4 lines

Not so weird - it is exactly what I was going to suggest. Providing you get the chunk size, parity algoithm and device order right, and use "--force" to make sure mdadm doesn't re-arrange the devices on on it should work perfectly.

this's one of our production server, so I'd like to be sure. so:
- switch the disk one by one (do these steps 8 times):
- put out one 120 hd (mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1)
- put the new 200 GB one and create a 200 GB partition (Linux raid autodetect)
- add to md0 (mdadm -a /dev/hda1) this step do not change the device

    order???

- resize2fs

when should I use --force??

Quoted 2 lines

Somewhere low-down on my todo list for raid is allowing this to be done on-line (mdadm /dev/md0 --size=max). It is right along side

that's be nice:-)

Quoted 1 line

allowing the number of live devices in a raid1 to be changed (mdadm /dev/md1 --devices=3) and adding devices to a linear array (mdadm /dev/md2 --add /dev/sdd).

that would be useful for raid5 too?
yes I know the reason why this is not possible, BUT it can do (rebuild the whole raid5) the same as the create for raid5 and it can be run in the backgroup. that would be a very useful feature for many people. yours.

Signature · 1 line
  Levente                               "Si vis pacem para bellum!"
#7 lore raw
Quoted 6 lines · Farkas Levente
Quoted 4 lines

Not so weird - it is exactly what I was going to suggest. Providing you get the chunk size, parity algoithm and device order right, and use "--force" to make sure mdadm doesn't re-arrange the devices on on it should work perfectly.

this's one of our production server, so I'd like to be sure. so:

this is the reason i said it is a weird idea, it is not easy to understand and if you fuck up you fuck up badly!

Quoted 6 lines

- switch the disk one by one (do these steps 8 times):
- put out one 120 hd (mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1)
- put the new 200 GB one and create a 200 GB partition (Linux raid autodetect)
- add to md0 (mdadm -a /dev/hda1) this step do not change the device

    order???

at this point you will find yourself with a 120GBx8 raid 5 array now you would have to (but i would be very careful in doing this)

check the exact device order (/proc/mdstat ???)
write it down
check the chunk size
write it down
check the parity algorithm
write it down

umount the fs
stop the damned array

mdadm -C /dev/md<whatever> -c <your chink size> -p <your parity> -n 8 -f <list of all devices in the same exact order>

THIS WILL REWRITE THE PARITY ON ALL DRIVES, SO IF YOU GET IT WRONG YOU WILL DESTROY YOUR DATA

(would using missing as the last drive prevent the parity rebuild and allow to check the expected result ?????)

cross you fingers,
mount your fs back and see what happened
umount it again

Quoted 1 line

- resize2fs

mount it for the last time

Quoted 1 line

when should I use --force??

only when you are really sure of what you are doing.

please test it on a test system before doing it on real data

you might well find that a slower way might be better for you

(i don't want to sound catastrophic, but i am not going to be held responsible if someone reads these instruction and fucks-up badly)

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#8 lore raw
In reply to Farkas Levente
Quoted 3 lines · On Monday November 17, [email protected] wrote:

this's one of our production server, so I'd like to be sure. so:
- switch the disk one by one (do these steps 8 times):
- put out one 120 hd (mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1)

I would do this as two separate steps
mdadm /dev/md0 -f /dev/sda1
mdadm /dev/md0 -r /dev/sda1

as there are some possibly race problems when removing a device the instant that it failes.

Quoted 4 lines

- put the new 200 GB one and create a 200 GB partition (Linux raid autodetect)
- add to md0 (mdadm -a /dev/hda1) this step do not change the device

     order???

No, it shouldnt' but it shouldn't matter.

Quoted 3 lines

- resize2fs

when should I use --force??

When you have the array full of 200GB drives, use
mdadm -D /dev/md0
to find out what it looks like. This will include algorithm, chunk size, etc. Keep a copy of the output.
Then
mdadm -S /dev/md0

It probably would be to use the "missing" directive as Luca suggested. This will create a degraded array and so will not automatically regenerate any data.

Something like:
mdadm -C /dev/md0 --level 5 -n 8 --chunk=64 /dev/sda1 /dev/sdb1 /dev/sdc1

    .... /dev/sdf1 /dev/sdg1 missing

If you do this, it will only over-write the raid superblocks and will not touch the data.

You can then
fsck -n /dev/md0
just to make sure it looks right (you don't have to let this run to completion. Just let it run for a new minutes to make sure nothing obvious is wrong) Then mount the filesystem and have a look around. If everything is OK, unmount and try resize2fs.

When you are happy the the array looks good, add the last drive back in with
mdadm /dev/md0 -a /dev/sdh1

NeilBrown