Help needed: Partitioned software raid > 2TB
Hi everyone,
I added a drive to a linux software RAID-5 last night. Now that worked fine... until I changed the partition table.
Disk /dev/md_d5: 2499.9 GB, 2499990978560 bytes
2 heads, 4 sectors/track, 610349360 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Device Boot Start End Blocks Id System /dev/md_d5p1 1 244142 976566 83 Linux /dev/md_d5p2 244143 5126956 19531256 8e Linux LVM /dev/md_d5p3 5126957 488279488 1932610128 8e Linux LVM
This is how the layout looked before the changes.
Device Boot Start End Blocks Id System /dev/md_d5p1 1 244142 976566 83 Linux /dev/md_d5p2 244143 5126956 19531256 8e Linux LVM /dev/md_d5p3 5126957 73478448 273405968 8e Linux LVM
This is how it looks now - the fdisk tools (fdisk/cfdisk) are unable to crate a single partition > 2TB. Of course now LVM complains that the PV is too small:
WARNING: Physical Volume /dev/md_d5p3 is too large for underlying device
PV Size 260.74 GB / not usable 8192.00 EB
I was able to restore a somewhat close partition table now... Close enough for the LVM to work correctly.
Device Boot Start End Blocks Id System /dev/md_d5p1 1 244142 976566 83 Linux /dev/md_d5p2 244143 5126956 19531256 8e Linux LVM /dev/md_d5p3 5126957 488279495 1932610156 8e Linux LVM /dev/md_d5p4 488279496 73478448 488279460 8e Linux LVM
Where is the inherent limit? The partitioning software, or partitioning all by itself?
Regards,
Christian
Quoted 3 lines · Christian Schmidt
Where is the inherent limit? The partitioning software, or partitioning all by itself?
DOS style partitioning don't support more than 2TB. You either need to use EFI partitions (e.g. using parted) or LVM. Since parted's user interface is not good for much more than serving
as a bad example I would recommend LVM.
-Andi
Hi Andi,
Quoted 8 lines · Andi Kleen
Quoted 2 lines · Christian Schmidt
Where is the inherent limit? The partitioning software, or partitioning all by itself?
DOS style partitioning don't support more than 2TB. You either need to use EFI partitions (e.g. using parted) or LVM. Since parted's user interface is not good for much more than serving
as a bad example I would recommend LVM.
Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)
Regards,
Christian
Quoted 11 lines · Christian Schmidt
Hi Andi,
Quoted 8 lines · Andi Kleen
Quoted 2 lines · Christian Schmidt
Where is the inherent limit? The partitioning software, or partitioning all by itself?
DOS style partitioning don't support more than 2TB. You either need to use EFI partitions (e.g. using parted) or LVM. Since parted's user interface is not good for much more than serving
as a bad example I would recommend LVM.
parted's interface is no worse than that of [sfc]disk. Anyway...
Quoted 6 lines
Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)
Uh, it does. By means of initrd/ramfs image. Blame your distro if it still can't do root-on-LVM, there is at least one who can.
Jan
Quoted 11 lines · Jan Engelhardt
Quoted 6 lines · Christian Schmidt
Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)
Uh, it does. By means of initrd/ramfs image. Blame your distro if it still can't do root-on-LVM, there is at least one who can.
AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself) requires LILO, doesn't it? Could you please list a few distributions with an easy method to install LILO (and, of course, root and /boot on LVM on whole disk) from their default installation media? So far, I only know that Debian can do it if you run debootstrap by hand, although they say that such setup is a bug (http://bugs.debian.org/401393).
Signature · 1 line
Alexander E. Patrakov
Quoted 13 lines · Alexander E. Patrakov
Quoted 11 lines · Jan Engelhardt
Quoted 6 lines · Christian Schmidt
Thanks for the clarification. I didn't use LVM on the device on purpose, as root on LVM requires initrd (which I strongly dislike as yet-another-point-of-failure). As LVM is on the large partition anyway I'll just add the second partition for now, and change the system setup with the next drive migration. Maybe linux even supports root-on-lvm natively until then ;)
Uh, it does. By means of initrd/ramfs image. Blame your distro if it still can't do root-on-LVM, there is at least one who can.
Quoted 6 lines
AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself) requires LILO, doesn't it? Could you please list a few distributions with an easy method to install LILO (and, of course, root and /boot on LVM on whole disk) from their default installation media? So far, I only know that Debian can do it if you run debootstrap by hand, although they say that such setup is a bug (http://bugs.debian.org/401393).
I am not sure (would have to check again), but I believe both opensuse and fedora (the latter of which uses LVM for all partitions by default) have that working, while still using GRUB. But, what's much more amazing, is that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps it's just luck that the needed files are contiguous?
Jan
Quoted 3 lines · Jan Engelhardt
I am not sure (would have to check again), but I believe both opensuse and fedora (the latter of which uses LVM for all partitions by default) have that working, while still using GRUB.
Keyword: partitions. I.e., they partition the hard drive (so that the first 31 sectors are available for GRUB) and use LVM on devices such as /dev/hda2. But this is not what was needed. I need to use LVM on /dev/hda, without a partition table.
Quoted 3 lines
But, what's much more amazing, is
that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps it's just luck that the needed files are contiguous?
[wild guess] Maybe they set up a raid1 /boot partition?
Signature · 1 line
Alexander E. Patrakov