LVM, partitions, RAID, and the Grand Scheme of Things TM
Greetings, LVM people :-)
I am the GNU Parted maintainer (www.gnu.org/software/parted), and I work for Conectiva (www.conectiva.com).
First, we think it would be useful for the LVM tools to be able to resize logical volumes, along with the file systems on it. Perhaps a --filesystem parameter to lvreduce, lvextend and lvcreate? This functionality should be fairly easy to add with the existing libparted. Although, we'd need to abstract out device I/O a bit more... but this is trivial.
BTW: we are aware of Ted Tso's resizer.
(1) we like our's better :-P
(2) libparted can resize other file systems. We can do ext2, fat[16|32], linux-swap at the moment, and we're planning to do others. libparted provides an abstract interface, so you get new features supported without any work :-)
More importantly:
We are interested in the future of partitions, LVM, RAID and Microsoft's new system, which seems similar to LVM - although we haven't seen it yet. Also, AIX's and HPUX's LVM have some features that Linux LVM doesn't have - you're working in that direction?
It seems, on the face of it, that LVM offers a strict superset of functionality of partition tables and RAID (well, RAID-linear/0, anyway
- RAID-1/4/5 coming soon?).
However, it doesn't look like partition tables will disappear anytime soon. Rather, it looks like LVM will have to cooperate with partition tables. And, perhaps Microsoft's new system as well (?).
Since LVM, partition tables, RAID, and M$'s thing are similar, we think it might make sense to make a single tool (in addition to the LVM tools) that can manipulate them all. And in fact, a library (that wraps libparted, liblvm, etc) to provide this abstracted functionality to other programs.
Arguments for such a move:
* a lot of the complexity, WRT interaction between filesystems and partition tables/LVM/* is similar. Eg: making a partition/LV usually happens at the same time as creating a file system on that partition. Likewise for resizing. This occurs right from the low-level (IO - "which part of which disk do I read/write?") to the high-level (how big can/should I make this partition?)
* may make life easier for boot-loader installers. There has been a bit of discussion between as and GRUB people about making a libbootload. This is still a distant plan. Basic idea: allow communication between partition programs (or whatever) and boot loaders about information like, where to find kernels, etc.
* users will love it :-)
* it's "philosophically" Right.
* may simplify automatic partitioning, since everything is in one place.
It may turn out that squeezing everything into a single interface will complicate things too much, but we don't think this will happen.
Anyway, onto the NEXT part: HOW.
First, I think it's desirable that we be independent of Linux, and assume that there is no support for anything much in the kernel. Is this OK? We can break this assumption, but we'd rather not...
Anyway, we are thinking about some rough correspondence like this:
* volume groups - equivalent to partition tables
* logical volumes - equivalent to partitions, RAID sets and raw devices.
* physical volumes - equivalent to a logical volume on the parent's volume group.
Note: this "parent's volume group" thing. Basically, partition tables are nested. So, LVM is like a partition table, nested inside, perhaps, many MSDOS partition tables.
So, we'd provide some sort of API for manipulating these objects.
Comments...?
Thanks,
Andrew Clausen
Andrew Clausen, you write:
Quoted 3 lines
First, we think it would be useful for the LVM tools to be able to resize logical volumes, along with the file systems on it. Perhaps a --filesystem parameter to lvreduce, lvextend and lvcreate?
The existing LVM user tools ship with something called "e2fsadm" which does this for ext2 filesystems, first extending the LV and then the filesystem. I have updated e2fsadm to also handle online filesystem resizing, but this is not in the official tools yet. Yesterday, a tool called lvmadm was announced which does this in a more generic way, similar to fsck, by calling a fs-specific resizer depending on the fs type.
Quoted 1 line
BTW: we are aware of Ted Tso's resizer. We like our's better :-P
On a tangent, how different has your ext2 resizer become from the ext2resize code? You may want to reply in a separate note.
Quoted 8 lines
We are interested in the future of partitions, LVM, RAID and Microsoft's new system, which seems similar to LVM - although we haven't seen it yet. Also, AIX's and HPUX's LVM have some features that Linux LVM doesn't have - you're working in that direction?
It seems, on the face of it, that LVM offers a strict superset of functionality of partition tables and RAID (well, RAID-linear/0, anyway
- RAID-1/4/5 coming soon?).
I think Heinz's stated direction is that LVM will stay layered on top of MD/RAID rather than incorporating this functionality itself.
Quoted 3 lines
However, it doesn't look like partition tables will disappear anytime soon. Rather, it looks like LVM will have to cooperate with partition tables. And, perhaps Microsoft's new system as well (?).
Personally, I would rather get all knowledge of partition tables out of LVM and have the existing partition table support in the kernel handle everything. Having a library to do all the partition-table parsing and writing would be a boon not only to parted, but also if it was added to fdisk, cfdisk, grub, et-al.
Quoted 7 lines
* a lot of the complexity, WRT interaction between filesystems and partition tables/LVM/* is similar. Eg: making a partition/LV usually happens at the same time as creating a file system on that partition. Likewise for resizing. This occurs right from the low-level (IO - "which part of which disk do I read/write?") to the high-level (how big can/should I make this partition?)
Quoted 5 lines
* users will love it :-)
* it's "philosophically" Right.
* may simplify automatic partitioning, since everything is in one place.
You should read the LVM mailing list archive for June. IBM has already proposed such a system, called LVMS, which would be the glue layer between LVM, partitions, filesystems, RAID, etc. From the sounds of it, they will try and leverage as much of the existing code from the kernel as possible.
Cheers, Andreas
Quoted 10 lines · Andreas Dilger
Andrew Clausen, you write:
Quoted 3 lines
First, we think it would be useful for the LVM tools to be able to resize logical volumes, along with the file systems on it. Perhaps a --filesystem parameter to lvreduce, lvextend and lvcreate?
The existing LVM user tools ship with something called "e2fsadm" which does this for ext2 filesystems, first extending the LV and then the filesystem. I have updated e2fsadm to also handle online filesystem resizing, but this is not in the official tools yet.
I'm not sure I like this idea of file system specific programs doing LVM stuff. What you describe here is better:
Quoted 4 lines
Yesterday, a tool
called lvmadm was announced which does this in a more generic way, similar to fsck, by calling a fs-specific resizer depending on the fs type.
It would be trivial (i.e. a bit of beaurocracy) to write a front end to libparted, to do this (or even, a shell script wrapper of parted)
One thing: there is non-trivial interaction between the file system and partition table handling (not with LVM), when resizing the start of a partition. Some resizers (including ext2resize, and reiserfs's resizer) can't resize the start, and use the partition as the address space. When resizing the start, this doesn't make sense.
So, while it's rather cute to do:
# ext2resize /dev/hda3 [new-size]
It generalizes to:
# ext2resize /dev/hda3 [new-start] [new-end]
Doing this only makes sense if ext2resize (or whatever) updates the partition table (as Parted does).
So, having /sbin/resize.* isn't as elegant as it looks. Were you suggesting have resize.* deal with partition table stuff?
Quoted 2 lines
I think Heinz's stated direction is that LVM will stay layered on top of MD/RAID rather than incorporating this functionality itself.
OK.
Quoted 9 lines
Quoted 3 lines
However, it doesn't look like partition tables will disappear anytime soon. Rather, it looks like LVM will have to cooperate with partition tables. And, perhaps Microsoft's new system as well (?).
Personally, I would rather get all knowledge of partition tables out of LVM and have the existing partition table support in the kernel handle everything. Having a library to do all the partition-table parsing and writing would be a boon not only to parted, but also if it was added to fdisk, cfdisk, grub, et-al.
Yep. OTOH, Parted already has lots of partition table parsing stuff already. It could be separated into a different library.
libparted's partition table API (version 1.3.x) is much cleaner than anything else I've seen. It supports msdos, mac and pc98 so far.
Quoted 4 lines
You should read the LVM mailing list archive for June. IBM has already proposed such a system, called LVMS, which would be the glue layer between LVM, partitions, filesystems, RAID, etc. From the sounds of it, they will try and leverage as much of the existing code from the kernel as possible.
Looks like I have lots of reading to do!
Thanks,
Andrew Clausen
Re: [linux-lvm] LVM, partitions, RAID, and the Grand Scheme of ThingsTM
In reply to Andreas DilgerQuoted 3 lines · Andreas Dilger
When using LVM its not important to be able to move the beginning of the filesystem, since you can always change the physical LE locations without changing the filesystem itself.
True.
Quoted 5 lines
What _is_ interesting, if you live in an LVM-centric world, is being able to move the start of the filesystem a small amount into the partition (after shrinking the end) so that you can turn the partition into an PV/LV. After such a conversion, you could do any normal LVM handling of the LV/filesystem (e.g. shrink, grow, migrate).
Yep :-)
Quoted 8 lines
It may just be easy enough to write a generic tool to shift the partition contents X blocks via copying (after using an fs-specific tool to shrink the end of the fs), and then handle the creation of the PV header, and adding an LV of the appropriate size that covers the whole partition. This is only really useful if you don't have enough space to duplicate the whole partition. Otherwise, you are just as well off creating a new LV of the appropriate size, doing a raw copy of the filesystem to the new LV, and deleting the old partition and/or turning it into a PV.
This is VERY slow (need to read & write the entire file system), and is dangerous (requires journaling to prevent corruption on power failure, or other interruptions). Also: you may copy, at most, X bytes at a time,
where X is the distance you move the start - unless you use an intermediate copy area. Either way (journaling, or intermediate copying),
you're doing LOTS of seeks (if you're working with one disk) - which is where IO gets really expensive.
We can do much better :-)
I'll hopefully implement move-the-start for ext2 when I go to Brazil (late Nov to late Feb)
Quoted 7 lines
Quoted 2 lines
So, having
/sbin/resize.*isn't as elegant as it looks. Were you suggesting have resize.* deal with partition table stuff?Not at all. The resize.* programs are filesystem specific, and only resize from the end of the partition. I suppose that they could be generalized to accept an additional parameter (at the end) to give the new start of the partition, e.g. "resize.ext2
/dev/hda1[size] [start]"
"/dev/hda1" is not as simple as it seems. It provides a linear address space from the start of the old partition, to the end of the old partition. Where partition is defined by the kernel, BTW. So, a resizer
can use that linear address space the kernel provides. If that is a bad idea (which it is for move-the-start, and is somewhat dubious for growth)
- then it must create it's own address space. i.e. it must understand partition tables.
Andrew Clausen
Quoted 5 lines · Andrew Clausen
One thing: there is non-trivial interaction between the file system and partition table handling (not with LVM), when resizing the start of a partition. Some resizers (including ext2resize, and reiserfs's resizer) can't resize the start, and use the partition as the address space.
When using LVM its not important to be able to move the beginning of the filesystem, since you can always change the physical LE locations without changing the filesystem itself.
What _is_ interesting, if you live in an LVM-centric world, is being able to move the start of the filesystem a small amount into the partition (after shrinking the end) so that you can turn the partition into an PV/LV. After such a conversion, you could do any normal LVM handling of the LV/filesystem (e.g. shrink, grow, migrate).
It may just be easy enough to write a generic tool to shift the partition contents X blocks via copying (after using an fs-specific tool to shrink the end of the fs), and then handle the creation of the PV header, and adding an LV of the appropriate size that covers the whole partition. This is only really useful if you don't have enough space to duplicate the whole partition. Otherwise, you are just as well off creating a new LV of the appropriate size, doing a raw copy of the filesystem to the new LV, and deleting the old partition and/or turning it into a PV.
Quoted 2 lines
So, having
/sbin/resize.*isn't as elegant as it looks. Were you suggesting have resize.* deal with partition table stuff?
Not at all. The resize.* programs are filesystem specific, and only resize from the end of the partition. I suppose that they could be generalized to accept an additional parameter (at the end) to give the new start of the partition, e.g. "resize.ext2 /dev/hda1 [size] [start]"
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert