Skip to contentSkip to conversation

Help needed: Partitioned software raid > 2TB

DISCUSSION Christian Schmidt 4 participants 7 messages Jun 15, 2007 – Jun 16, 2007

lore
Share & tools
lore
#0 lore raw

Hi everyone,

I added a drive to a linux software RAID-5 last night. Now that worked fine... until I changed the partition table.

Disk /dev/md_d5: 2499.9 GB, 2499990978560 bytes
2 heads, 4 sectors/track, 610349360 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d5p1               1      244142      976566   83  Linux
/dev/md_d5p2          244143     5126956    19531256   8e  Linux LVM
/dev/md_d5p3         5126957   488279488  1932610128   8e  Linux LVM

This is how the layout looked before the changes.

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d5p1               1      244142      976566   83  Linux
/dev/md_d5p2          244143     5126956    19531256   8e  Linux LVM
/dev/md_d5p3         5126957    73478448   273405968   8e  Linux LVM

This is how it looks now - the fdisk tools (fdisk/cfdisk) are unable to crate a single partition > 2TB. Of course now LVM complains that the PV is too small:

WARNING: Physical Volume /dev/md_d5p3 is too large for underlying device
PV Size 260.74 GB / not usable 8192.00 EB

I was able to restore a somewhat close partition table now... Close enough for the LVM to work correctly.

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d5p1               1      244142      976566   83  Linux
/dev/md_d5p2          244143     5126956    19531256   8e  Linux LVM
/dev/md_d5p3         5126957   488279495  1932610156   8e  Linux LVM
/dev/md_d5p4       488279496    73478448   488279460   8e  Linux LVM

Where is the inherent limit? The partitioning software, or partitioning all by itself?

Regards,
Christian

#1 lore raw
Quoted 3 lines · Christian Schmidt

Where is the inherent limit? The partitioning software, or partitioning all by itself?

DOS style partitioning don't support more than 2TB. You either need to use EFI partitions (e.g. using parted) or LVM. Since parted's user interface is not good for much more than serving
as a bad example I would recommend LVM.

-Andi

#2 lore raw

Hi Andi,

Quoted 8 lines · Andi Kleen
Quoted 2 lines · Christian Schmidt

Where is the inherent limit? The partitioning software, or partitioning all by itself?

DOS style partitioning don't support more than 2TB. You either need to use EFI partitions (e.g. using parted) or LVM. Since parted's user interface is not good for much more than serving
as a bad example I would recommend LVM.

Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)

Regards,
Christian

#3 lore raw
Quoted 11 lines · Christian Schmidt

Hi Andi,

Quoted 8 lines · Andi Kleen
Quoted 2 lines · Christian Schmidt

Where is the inherent limit? The partitioning software, or partitioning all by itself?

DOS style partitioning don't support more than 2TB. You either need to use EFI partitions (e.g. using parted) or LVM. Since parted's user interface is not good for much more than serving
as a bad example I would recommend LVM.

parted's interface is no worse than that of [sfc]disk. Anyway...

Quoted 6 lines

Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)

Uh, it does. By means of initrd/ramfs image. Blame your distro if it still can't do root-on-LVM, there is at least one who can.

Jan

#4 lore raw
Quoted 11 lines · Jan Engelhardt
Quoted 6 lines · Christian Schmidt

Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)

Uh, it does. By means of initrd/ramfs image. Blame your distro if it still can't do root-on-LVM, there is at least one who can.

AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself) requires LILO, doesn't it? Could you please list a few distributions with an easy method to install LILO (and, of course, root and /boot on LVM on whole disk) from their default installation media? So far, I only know that Debian can do it if you run debootstrap by hand, although they say that such setup is a bug (http://bugs.debian.org/401393).

Signature · 1 line
Alexander E. Patrakov
#5 lore raw
Quoted 13 lines · Alexander E. Patrakov
Quoted 11 lines · Jan Engelhardt
Quoted 6 lines · Christian Schmidt

Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)

Uh, it does. By means of initrd/ramfs image. Blame your distro if it still can't do root-on-LVM, there is at least one who can.

Quoted 6 lines

AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself) requires LILO, doesn't it? Could you please list a few distributions with an easy method to install LILO (and, of course, root and /boot on LVM on whole disk) from their default installation media? So far, I only know that Debian can do it if you run debootstrap by hand, although they say that such setup is a bug (http://bugs.debian.org/401393).

I am not sure (would have to check again), but I believe both opensuse and fedora (the latter of which uses LVM for all partitions by default) have that working, while still using GRUB. But, what's much more amazing, is that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps it's just luck that the needed files are contiguous?

Jan

#6 lore raw
Quoted 3 lines · Jan Engelhardt

I am not sure (would have to check again), but I believe both opensuse and fedora (the latter of which uses LVM for all partitions by default) have that working, while still using GRUB.

Keyword: partitions. I.e., they partition the hard drive (so that the first 31 sectors are available for GRUB) and use LVM on devices such as /dev/hda2. But this is not what was needed. I need to use LVM on /dev/hda, without a partition table.

Quoted 3 lines

But, what's much more amazing, is
that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps it's just luck that the needed files are contiguous?

[wild guess] Maybe they set up a raid1 /boot partition?

Signature · 1 line
Alexander E. Patrakov