generic wrappers for multi-device FS operations
Subsystems (2)
After seeing some of the feedback and confusion that happened in the fedora community after Josef suggestion that we default to btrfs in an upcoming Fedora release, it became clear to me that many users are incredibly unaware of the common features that we have across file systems today given LVM/device mapper support.
btrfs will make multi-volume/multi-disk operations common place and easy to do, but there is no reason not to do most/all of this today with ext4, xfs, etc on top of lvm.
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?
Thanks!
Ric
Alasdair G Kergon
maintainerQuoted 5 lines · Ric Wheeler
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?
We began this within LVM some time ago with a script we called 'fsadm'.
Alasdair
fsadm: Utility to resize or check the filesystem on a device
fsadm [options] check device
- Check the filesystem on device using fsck
fsadm [options] resize device [new_size[BKMGTPE]]
- Change the size of the filesystem on device to new_size
Options:
-h | --help Show this help message
-v | --verbose Be verbose
-e | --ext-offline unmount filesystem before ext2/ext3/ext4 resize
-f | --force Bypass sanity checks
-n | --dry-run Print commands without running them
-l | --lvresize Resize given device (if it is LVM device)
-y | --yes Answer "yes" at any prompts new_size - Absolute number of filesystem blocks to be in the filesystem,
or an absolute size using a suffix (in powers of 1024). If new_size is not supplied, the whole device is used.
Quoted 7 lines · Ric Wheeler
Quoted 5 lines
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
-- Wendy
Quoted 11 lines · Wendy Cheng
Quoted 6 lines · Ric Wheeler
Quoted 5 lines
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
-- Wendy
I think any "standard" operation would be allowed to fail with "notsupported" effectively. Shrink is definitely a challenge & even when it works, it often has performance implications.
I think that adding support to fsadm might be the easy path forward, but we would need to be able to do at least a handful of more advanced things (add a whole device for example).
Ric
Quoted 13 lines · Ric Wheeler
Quoted 11 lines · Wendy Cheng
Quoted 6 lines · Ric Wheeler
Quoted 5 lines
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
-- Wendy
Quoted 7 lines
I think any "standard" operation would be allowed to fail with "notsupported" effectively. Shrink is definitely a challenge & even when it works, it often has performance implications.
I think that adding support to fsadm might be the easy path forward, but we would need to be able to do at least a handful of more advanced things (add a whole device for example).
There's still the problem of moving the underlying device from being a plain block device to being a dm one ... what happened to the idea of presenting all devices as dm ones to solve this?
James
Quoted 17 lines · James Bottomley
Quoted 16 lines · Ric Wheeler
Quoted 11 lines · Wendy Cheng
Quoted 6 lines · Ric Wheeler
Quoted 5 lines
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
-- Wendy
I think any "standard" operation would be allowed to fail with "notsupported" effectively. Shrink is definitely a challenge& even when it works, it often has performance implications.
Quoted 8 lines
Quoted 3 lines
I think that adding support to fsadm might be the easy path forward, but we would need to be able to do at least a handful of more advanced things (add a whole device for example).
There's still the problem of moving the underlying device from being a plain block device to being a dm one ... what happened to the idea of presenting all devices as dm ones to solve this?
James
I think that would certainly be a nice start, but we could do something short term that did not require that.
Ric
Quoted 3 lines · James Bottomley
There's still the problem of moving the underlying device from being a plain block device to being a dm one ... what happened to the idea of presenting all devices as dm ones to solve this?
Giving remapping abilities to every block device certainly would be extremly useful.
Regards,
Lars
Signature · 8 lines
Architect Storage/HA, OPS Engineering, Novell, Inc. SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) "Experience is the name everyone gives to their mistakes." -- Oscar Wilde -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Wendy Cheng
Quoted 7 lines · Ric Wheeler
Quoted 5 lines
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?
Quoted 4 lines
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
Who wants to make their fs smaller anyway? ;)
Josef
Quoted 7 lines · Josef Bacik
Quoted 6 lines · Wendy Cheng
Quoted 6 lines · Ric Wheeler
Quoted 5 lines
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Good idea? Bad idea?
Quoted 7 lines
Quoted 4 lines
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
Who wants to make their fs smaller anyway? ;)
Josef
I have been surprised more than once by customers who do this on a regular basis :)
Ric
Alasdair G Kergon
maintainerQuoted 3 lines · Wendy Cheng
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
The LVM situation today works both ways around.
You can use 'lvresize' with an option to resize the filesystem too, or use 'fsadm' with an option to resize the LV. We felt that 'fsadm' was a more-natural approach: allow the user to resize their filesystem and automatically adjust the things underneath as necessary.
Long term, we'd like it to be possible to configure a system for resizing entire device stacks both top down (fsadm) and bottom up (triggered by the appropriate Unit Attention).
(The current fsadm script was written as a proof-of-concept under the constraint of just wrapping existing binaries. Of course there are cleaner ways to do it when you allow filesystem-specific code.)
Alasdair
Quoted 15 lines · Alasdair G Kergon
Quoted 3 lines · Wendy Cheng
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
The LVM situation today works both ways around.
You can use 'lvresize' with an option to resize the filesystem too, or use 'fsadm' with an option to resize the LV. We felt that 'fsadm' was a more-natural approach: allow the user to resize their filesystem and automatically adjust the things underneath as necessary.
Long term, we'd like it to be possible to configure a system for resizing entire device stacks both top down (fsadm) and bottom up (triggered by the appropriate Unit Attention).
Quoted 5 lines
(The current fsadm script was written as a proof-of-concept under the constraint of just wrapping existing binaries. Of course there are cleaner ways to do it when you allow filesystem-specific code.)
Alasdair
Is anyone actively looking at taken it beyond proof of concept?
Ric
Quoted 17 lines · Ric Wheeler
Quoted 16 lines · Alasdair G Kergon
Quoted 3 lines · Wendy Cheng
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
The LVM situation today works both ways around.
You can use 'lvresize' with an option to resize the filesystem too, or use 'fsadm' with an option to resize the LV. We felt that 'fsadm' was a more-natural approach: allow the user to resize their filesystem and automatically adjust the things underneath as necessary.
Long term, we'd like it to be possible to configure a system for resizing entire device stacks both top down (fsadm) and bottom up (triggered by the
appropriate Unit Attention).
Quoted 9 lines
Quoted 6 lines
(The current fsadm script was written as a proof-of-concept under the constraint of just wrapping existing binaries. Of course there are cleaner
ways to do it when you allow filesystem-specific code.)Alasdair
Is anyone actively looking at taken it beyond proof of concept?
I'm working on the kernel-side parts, namely pushing unit attention events out to userspace via debugfs.
There'll be daemon reading those and eventually triggering the appropriate action.
We could hold a BoF at LSF or Collab summit, seeing that there is quite some interest in these kind of things.
Cheers,
Hannes
Signature · 8 lines
Dr. Hannes Reinecke zSeries & Storage [email protected] +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 31 lines · Hannes Reinecke
Quoted 17 lines · Ric Wheeler
Quoted 16 lines · Alasdair G Kergon
Quoted 3 lines · Wendy Cheng
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
The LVM situation today works both ways around.
You can use 'lvresize' with an option to resize the filesystem too, or use 'fsadm' with an option to resize the LV. We felt that 'fsadm' was a more-natural approach: allow the user to resize their filesystem and automatically adjust the things underneath as necessary.
Long term, we'd like it to be possible to configure a system for resizing entire device stacks both top down (fsadm) and bottom up (triggered by the
appropriate Unit Attention).Quoted 8 lines
Quoted 6 lines
(The current fsadm script was written as a proof-of-concept under the constraint of just wrapping existing binaries. Of course there are cleaner
ways to do it when you allow filesystem-specific code.)Alasdair
Is anyone actively looking at taken it beyond proof of concept?
I'm working on the kernel-side parts, namely pushing unit attention events out to userspace via debugfs.
There'll be daemon reading those and eventually triggering the appropriate action.
Quoted 6 lines
We could hold a BoF at LSF or Collab summit, seeing that there is quite some interest in these kind of things.
Cheers,
Hannes
I think that the idea of a BOF would be great - probably more room/time at collab given the schedule of LSF at this point,
Ric
Quoted 7 lines · Alasdair G Kergon
Quoted 3 lines · Wendy Cheng
So the "resize" is on the filesystem, not the volume ? The "grow" part is probably easy. Unfortunately, the "shrink" may not be easy for some of the filesystems:
The LVM situation today works both ways around.
Quoted 8 lines
You can use 'lvresize' with an option to resize the filesystem too, or use 'fsadm' with an option to resize the LV. We felt that 'fsadm' was a more-natural approach: allow the user to resize their filesystem and automatically adjust the things underneath as necessary.
Long term, we'd like it to be possible to configure a system for resizing entire device stacks both top down (fsadm) and bottom up (triggered by the appropriate Unit Attention).
I can see how this would be advantageous from a management perspective for growing LVM pools, but I can see quite a few problems with extending that to automatically growing filesystems above LVM. e.g. The LUN resizes, the PV is expanded, the VG is expanded, but which LV in the VG is going to make use the new space?
Similarly, you can't safely shrink a filesystem from the bottom up - the filesystem has to empty the space first before it is safe to shrink it. How would you handle this from a bottom-up driven change?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Alasdair G Kergon
maintainerQuoted 5 lines · Dave Chinner
I can see how this would be advantageous from a management perspective for growing LVM pools, but I can see quite a few problems with extending that to automatically growing filesystems above LVM. e.g. The LUN resizes, the PV is expanded, the VG is expanded, but which LV in the VG is going to make use the new space?
You'd handle that by configuration settings under sysadmin control. There are plenty of situations where there would be an 'obvious' LV to extend.
Quoted 4 lines
Similarly, you can't safely shrink a filesystem from the bottom up - the filesystem has to empty the space first before it is safe to shrink it. How would you handle this from a bottom-up driven change?
Obviously there are limits, and passing a message "Prepare to be made smaller by X" up the stack probably isn't very feasible.
Alasdair
Quoted 8 lines · Alasdair G Kergon
Quoted 5 lines · Dave Chinner
I can see how this would be advantageous from a management perspective for growing LVM pools, but I can see quite a few problems with extending that to automatically growing filesystems above LVM. e.g. The LUN resizes, the PV is expanded, the VG is expanded, but which LV in the VG is going to make use the new space?
You'd handle that by configuration settings under sysadmin control.
So you have to set it up ahead of time, and then remember that you have set it up to automatically expand LV X when PV Y expands because device Z was expanded.
Quoted 2 lines
There are plenty of situations where there would be an 'obvious' LV to extend.
Sure, but that's the easy case. My concern is always the hard cases
- what to do when there isn't an obvious LV to expand, or you want an LV that isn't the obvious or configured LV to expand when the PV is grown, or you just want to grow the PV ready to add new LVs rather than expand an LV....
Quoted 7 lines
Quoted 4 lines
Similarly, you can't safely shrink a filesystem from the bottom up - the filesystem has to empty the space first before it is safe to shrink it. How would you handle this from a bottom-up driven change?
Obviously there are limits, and passing a message "Prepare to be made smaller by X" up the stack probably isn't very feasible.
Agreed, especially as it can take hours of IO before an "ok to be made smaller now " message can be sent or the attempt to shrink the filesystem fails...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 5 lines · Ric Wheeler
After seeing some of the feedback and confusion that happened in the fedora community after Josef suggestion that we default to btrfs in an upcoming Fedora release, it became clear to me that many users are incredibly unaware of the common features that we have across file systems today given LVM/device mapper support.
btrfs will make multi-volume/multi-disk operations common place and easy to do, but there is no reason not to do most/all of this today with ext4, xfs, etc on top of lvm.
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
I definitely think this makes sense. However, taking a quick look at fsadm, I don't think it is the right starting point for this work. It is essentially a single script that is special-casing each filesystem it is touching, which makes it a maintenance nightmare to add in support for different filesystems.
A better structure is the mkfs.* and fsck.* tools that extend the basic mkfs/fsck functionality for each new filesystem. That allows new filesystems to be added without the requirement to modify the upstream fsadm script.
Another tool similar to this that I've been trying to push upstream for some time is the "lvcheck" script, which is essentially a wrapper for online filesystem checking. It is currently structured as an extension to the LVM tools, since it depends on creating a snapshot of an LV and does a check on the snapshot. If the snapshot is clean the original filesystem is marked checked as well, which avoids the "slow ext* check on boot" problem, while still ensuring that periodic filesystem checks will catch latent errors.
It wouldn't be unreasonable to have a new wrapper for online filesystem checking (e.g. ofsck) or just an extension to fsck that does this in a more "plug-in" manner like fsck.* does today. It would naturally progress into real online checking for filesystems that support this (e.g. btrfs, and I think XFS is going in this direction as well).
Cheers, Andreas
Quoted 6 lines · Andreas Dilger
Quoted 5 lines · Ric Wheeler
After seeing some of the feedback and confusion that happened in the fedora community after Josef suggestion that we default to btrfs in an upcoming Fedora release, it became clear to me that many users are incredibly unaware of the common features that we have across file systems today given LVM/device mapper support.
btrfs will make multi-volume/multi-disk operations common place and easy to do, but there is no reason not to do most/all of this today with ext4, xfs, etc on top of lvm.
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Quoted 8 lines
I definitely think this makes sense. However, taking a quick look at fsadm, I don't think it is the right starting point for this work. It is essentially a single script that is special-casing each filesystem it is touching, which makes it a maintenance nightmare to add in support for different filesystems.
A better structure is the mkfs.* and fsck.* tools that extend the basic mkfs/fsck functionality for each new filesystem. That allows new filesystems to be added without the requirement to modify the upstream fsadm script.
I do like the two level scheme that mkfs.* and fsck.* use - same interface largely regardless of the file system and the ability to customize the fs specific command as needed.
Quoted 9 lines
Another tool similar to this that I've been trying to push upstream for some time is the "lvcheck" script, which is essentially a wrapper for online filesystem checking. It is currently structured as an extension to the LVM tools, since it depends on creating a snapshot of an LV and does a check on the snapshot. If the snapshot is clean the original filesystem is marked checked as well, which avoids the "slow ext* check on boot" problem, while still ensuring that periodic filesystem checks will catch latent errors.
Quoted 7 lines
It wouldn't be unreasonable to have a new wrapper for online filesystem checking (e.g. ofsck) or just an extension to fsck that does this in a more "plug-in" manner like fsck.* does today. It would naturally progress into real online checking for filesystems that support this (e.g. btrfs, and I think XFS is going in this direction as well).
Cheers, Andreas
Online fsck would certainly be a win & would make a lot of users happy,
Ric
Dne 8.3.2011 21:58, Ric Wheeler napsal(a):
Quoted 44 lines · Andreas Dilger
Quoted 15 lines · Ric Wheeler
Quoted 6 lines
After seeing some of the feedback and confusion that happened in the fedora community after Josef suggestion that we default to btrfs in an upcoming Fedora release, it became clear to me that many users are incredibly unaware of the common features that we have across file systems today given LVM/device mapper support.
Quoted 8 lines
btrfs will make multi-volume/multi-disk operations common place and easy to do, but there is no reason not to do most/all of this today with ext4, xfs, etc on top of lvm.
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Quoted 8 lines
I definitely think this makes sense. However, taking a quick look at fsadm, I don't think it is the right starting point for this work. It is essentially a single script that is special-casing each filesystem it is touching, which makes it a maintenance nightmare to add in support for different filesystems.
A better structure is the mkfs.* and fsck.* tools that extend the basic mkfs/fsck functionality for each new filesystem. That allows new filesystems to be added without the requirement to modify the upstream fsadm script.
I do like the two level scheme that mkfs.* and fsck.* use - same interface largely regardless of the file system and the ability to customize the fs specific command as needed.
Quoted 15 lines
Another tool similar to this that I've been trying to push upstream for some time is the "lvcheck" script, which is essentially a wrapper for online filesystem checking. It is currently structured as an extension to the LVM tools, since it depends on creating a snapshot of an LV and does a check on the snapshot. If the snapshot is clean the original filesystem is marked checked as well, which avoids the "slow ext* check on boot" problem, while still ensuring that periodic filesystem checks will catch latent errors.
It wouldn't be unreasonable to have a new wrapper for online filesystem checking (e.g. ofsck) or just an extension to fsck that does this in a more "plug-in" manner like fsck.* does today. It would naturally progress into real online checking for filesystems that support this (e.g. btrfs, and I think XFS is going in this direction as well).
Quoted 4 lines
Quoted 1 line
Cheers, Andreas
Online fsck would certainly be a win & would make a lot of users happy,
Can I ask for creating RH bugzillas for missing fsadm functionality. (Support for btrfs (Bug 643907) is already there thought other more prio task are currently resolved).
Zdenek
Quoted 7 lines · Andreas Dilger
Quoted 5 lines · Ric Wheeler
After seeing some of the feedback and confusion that happened in the fedora community after Josef suggestion that we default to btrfs in an upcoming Fedora release, it became clear to me that many users are incredibly unaware of the common features that we have across file systems today given LVM/device mapper support.
btrfs will make multi-volume/multi-disk operations common place and easy to do, but there is no reason not to do most/all of this today with ext4, xfs, etc on top of lvm.
To make this trivial to do for users, I think that it would be really nice to have a two-level wrappers for things like resize, add a volume, shrink, etc. Similar to the way we have mount or fsck invoke file system specific bits.
Quoted 8 lines
I definitely think this makes sense. However, taking a quick look at fsadm, I don't think it is the right starting point for this work. It is essentially a single script that is special-casing each filesystem it is touching, which makes it a maintenance nightmare to add in support for different filesystems.
A better structure is the mkfs.* and fsck.* tools that extend the basic mkfs/fsck functionality for each new filesystem. That allows new filesystems to be added without the requirement to modify the upstream fsadm script.
That seems like a sensible approach to me, however handling the different volumes could be trouble. e.g. the top level app would still need to know about the difference between log devices and data devices for XFS/ext3/ext4, realtime devices for XFS (as they can be grown separately to the data device but as still part of the same filesystem), while for btrfs just uses generic block devices, pools and volumes....
Quoted 7 lines
Another tool similar to this that I've been trying to push upstream for some time is the "lvcheck" script, which is essentially a wrapper for online filesystem checking. It is currently structured as an extension to the LVM tools, since it depends on creating a snapshot of an LV and does a check on the snapshot. If the snapshot is clean the original filesystem is marked checked as well, which avoids the "slow ext* check on boot" problem, while still ensuring that periodic filesystem checks will catch latent errors.
I think this is very different to the well defined operations of growing and shrinking filesystems and block devices.
Checking snapshots isn't really "online" checking at all - it's generating a temporary stable image of the filesystem that is used for an offline check. If anything is found wrong, you've still got to take the fs offline and run the offline repair program.
As it is, dm-snapshot based checking really isn't a solution that can be employed in production environments with performance SLAs or that require sustained high performance because of the performance hit the COW based snapshot mechanism causes while there are active snapshots. And that's before considering the impact of all the IO the check process would issue...
Quoted 5 lines
It wouldn't be unreasonable to have a new wrapper for online filesystem checking (e.g. ofsck) or just an extension to fsck that does this in a more "plug-in" manner like fsck.* does today. It would naturally progress into real online checking for filesystems that support this (e.g. btrfs, and I think XFS is going in this direction as well).
Online filesystem scrubbing and repair is highly filesystem specific, just like offline repair is. It can't be easily separated from the kernel code because of the need to be coherent with current operations.
My current line of thinking is that for XFS it would be an entirely in-kernel operation in combination with a scrubber and additional on-disk metadata structures (such as an rmap btree) to make the operation of the scrubber as efficient as possible. The scrubber would run in the background and trigger repair of problems it encounters, with extra triggers for when normal operation encounter corruption problems. i.e. check and reapir with no specific external userspace control or intervention.
IOWs, until we actually have real online repair implemented in more than one filesystem and can determine similarites in their operation, I think trying to develop a generic interface for them is premature....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]