XFS status update for January 2011
On the 4th of January we saw the release of Linux 2.6.37, which contains a large XFS update:
67 files changed, 1424 insertions(+), 1524 deletions(-)
User visible changes are the new XFS_IOC_ZERO_RANGE ioctl which allows to convert already allocated space into unwritten extents that return zeros on a read, and support for 32-bit wide project IDs. The other large item are various changes to improve metadata scalability even further, by changes to the the buffer cache, inode lookup and other parts of the filesystem driver.
After that the XFS development tree for 2.6.38 was merged into mainline, with an even larger set of changes. Notable items include support for the FITRIM ioctl to discard unused space on SSDs and thinly provisioned storage systems, a buffer LRU scheme to improve hit rates for metadata, an overhaul of the log subsystem locking, dramatically improving scalability in that area, and much smarter handling of preallocations, especially for files closed and reopened frequently, e.g. by the NFS server.
User space development has been very quite, with just a few fixes committed to the xfstests repository, although various additional patches for xfsprogs and xfstests that haven't been committed yet were discussed on the mailing list.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hi Christoph,
Quoted 9 lines · Christoph Hellwig
On the 4th of January we saw the release of Linux 2.6.37, which contains a large XFS update:
67 files changed, 1424 insertions(+), 1524 deletions(-)User visible changes are the new
XFS_IOC_ZERO_RANGEioctl which allows to convert already allocated space into unwritten extents that return zeros on a read,
would you mind describing some scenario that this ioctl can be used. I am just wondering whether ocfs2 can implement it as well.
Regards,
Tao
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Tao Ma
Hi Christoph,
Quoted 3 lines · Christoph Hellwig
On the 4th of January we saw the release of Linux 2.6.37, which contains a large XFS update:
Quoted 7 lines
Quoted 5 lines
67 files changed, 1424 insertions(+), 1524 deletions(-)User visible changes are the new
XFS_IOC_ZERO_RANGEioctl which allows to convert already allocated space into unwritten extents that return zeros on a read,would you mind describing some scenario that this ioctl can be used. I am just wondering whether ocfs2 can implement it as well.
Zeroing a file without doing IO or having to punch out the blocks already allocated to the file.
In this case, we had a couple of different people in cloud storage land asking for such functionality to optimise record deletion be avoiding disruption of their preallocated file layouts as a punch-then-preallocate operation does.
If you you have some kind of use for it in ocfs2, then implementing the XFS ioctl is not the correct thing to do - using the fallocate patch I've had sitting around (since about 15min after creating the XFS ioctl) is most likely the right way to proceed....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Hi Dave,
Quoted 16 lines · Tao Ma
Quoted 6 lines
Hi Christoph,
Quoted 1 line · Christoph Hellwig
On the 4th of January we saw the release of Linux 2.6.37, which
contains a
Quoted 2 lines
large XFS update:
Quoted 8 lines
Quoted 5 lines
67 files changed, 1424 insertions(+), 1524 deletions(-)User visible changes are the new
XFS_IOC_ZERO_RANGEioctl which allows to convert already allocated space into unwritten extents that return zeros on a read,would you mind describing some scenario that this ioctl can be used. I am
just wondering whether ocfs2 can implement it as well.
Quoted 7 lines
Zeroing a file without doing IO or having to punch out the blocks already allocated to the file.
In this case, we had a couple of different people in cloud storage land asking for such functionality to optimise record deletion be avoiding disruption of their preallocated file layouts as a punch-then-preallocate operation does.
Thanks for the info. yeah, ocfs2 is also used to host images in some cloud computing environment. So It looks helpful for us too.
Quoted 5 lines
If you you have some kind of use for it in ocfs2, then implementing the XFS ioctl is not the correct thing to do - using the fallocate patch I've had sitting around (since about 15min after creating the XFS ioctl) is most likely the right way to proceed....
yeah, Josef has added the punching hole in fallocate, so I guess another parameter for keeping the blocks during hole punching should be enough for us. :)
Regards,
Tao
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 20 lines · On Mon, Feb 14, 2011 at 08:20:18AM -0700, [email protected] wrote:
Hi Dave,
Quoted 19 lines · Tao Ma
Quoted 6 lines
Hi Christoph,
Quoted 1 line · Christoph Hellwig
On the 4th of January we saw the release of Linux 2.6.37, which
contains a
Quoted 2 lines
large XFS update:
Quoted 8 lines
Quoted 5 lines
67 files changed, 1424 insertions(+), 1524 deletions(-)User visible changes are the new
XFS_IOC_ZERO_RANGEioctl which allows to convert already allocated space into unwritten extents that return zeros on a read,would you mind describing some scenario that this ioctl can be used. I am
just wondering whether ocfs2 can implement it as well.Zeroing a file without doing IO or having to punch out the blocks already allocated to the file.
Quoted 6 lines
Quoted 4 lines
In this case, we had a couple of different people in cloud storage land asking for such functionality to optimise record deletion be avoiding disruption of their preallocated file layouts as a punch-then-preallocate operation does.
Thanks for the info. yeah, ocfs2 is also used to host images in some cloud computing environment. So It looks helpful for us too.
Just to be clear, this optimisation isn't relevant for hosting VM images in a cloud compute environment - this was added for optimising the back end of distributed storage applications that hold tens of millions of records and tens of TB of data per back end storage host.
Hosting VM images is largely static, especially if you are preallocating them - they never, ever get punched. Even if you are using thin provisioning semantics and punching TRIMmed ranges, you aren't converting the TRIMmed ranges back to preallocated state so you wouldn't be using this interface. Hence I don't see this as something that you would use in such an environment.
The distributed storage applications that this was added for required atomic record deletes from the back end and the fastest and safest way to do that was to turn the record being deleted back into unwritten extents. This allows that operation to be done atomically by the filesystem whilst providing simple recovery semantics to the application. The XFS_IOC_ZERO_RANGE ioctl simply prevents the fragmentation that this punch-then-preallocate operation was causing and allows the back end to scale to much larger record stores...
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 49 lines · Dave Chinner
Quoted 26 lines · On Mon, Feb 14, 2011 at 08:20:18AM -0700, [email protected] wrote:
Hi Dave,
Quoted 24 lines · Tao Ma
Quoted 12 lines
Hi Christoph,
Quoted 2 lines · Christoph Hellwig
On the 4th of January we saw the release of Linux 2.6.37, which
contains a
Quoted 4 lines
large XFS update:
67 files changed, 1424 insertions(+), 1524 deletions(-)Quoted 8 lines
Quoted 4 lines
User visible changes are the new
XFS_IOC_ZERO_RANGEioctl which allows to convert already allocated space into unwritten extents that return zeros on a read,would you mind describing some scenario that this ioctl can be used. I am
just wondering whether ocfs2 can implement it as well.Zeroing a file without doing IO or having to punch out the blocks already allocated to the file.
Quoted 8 lines
Quoted 5 lines
In this case, we had a couple of different people in cloud storage land asking for such functionality to optimise record deletion be avoiding disruption of their preallocated file layouts as a punch-then-preallocate operation does.
Thanks for the info. yeah, ocfs2 is also used to host images in some cloud computing environment. So It looks helpful for us too.
Just to be clear, this optimisation isn't relevant for hosting VM images in a cloud compute environment - this was added for optimising the back end of distributed storage applications that hold tens of millions of records and tens of TB of data per back end storage host.
Hosting VM images is largely static, especially if you are preallocating them - they never, ever get punched. Even if you are using thin provisioning semantics and punching TRIMmed ranges, you aren't converting the TRIMmed ranges back to preallocated state so you wouldn't be using this interface. Hence I don't see this as something that you would use in such an environment.
Quoted 9 lines
The distributed storage applications that this was added for required atomic record deletes from the back end and the fastest and safest way to do that was to turn the record being deleted back into unwritten extents. This allows that operation to be done atomically by the filesystem whilst providing simple recovery semantics to the application. The
XFS_IOC_ZERO_RANGEioctl simply prevents the fragmentation that this punch-then-preallocate operation was causing and allows the back end to scale to much larger record stores...
aha, got it. thanks for the detailed explanation.
Regards,
Tao
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs