xfs: byte range buffer dirty region tracking
RFC PATCH 1/1 xfs: byte range buffer dirty region tracking
From: Dave Chinner <[email protected]>
The biggest problem with large directory block sizes is the CPU overhead in maintaining the buffer log item direty region bitmap. The bit manipulations and buffer region mapping calls are right at the top of the profiles when running tests on 64k directory buffers:
+ 16.65% [kernel] [k] memcpy + 11.99% [kernel] [k] xfs_next_bit + 5.87% [kernel] [k] xfs_buf_item_format + 5.85% [kernel] [k] xfs_buf_item_size_segment.isra.4 + 5.72% [kernel] [k] xfs_buf_offset
The memcpy is the copying of the dirty regions into the log vec array, but almost twice as much CPU time is spent working out what needs to be copied and where it needs to be copied from. As a result, on a production kernel creating 100,000 entries in a 64k directory runs at about 9,000 files/s while on a 4k directory block size it runs at 19,000 files/s - about half the speed.
Switching this to just track the first and last modified bytes in the block and only converting that to a dirty bitmap in the buffer log item at format time, however, gets rid of most of this dirty bitmap overhead without increasing memcpy time at all. the result is that peformance on a 64k directory block size increases to roughly 16,000 files/s with memcpy() overhead only slightly increasing.
The code so far is a rudimentary proof of concept. The point for discussion is not whether the technique works (clearly it does) or whether the code is production quality (clearly it isn't), but how to apply apply it appropriately.
I think that we need to track multiple regions - 3 or 4 is probably sufficient - because the nature of directory operations are that just about every operation modifies a header in the buffer, a tail section in the buffer and then some number of bytes/regions in the middle of the buffer.
Hence if we just track a signle region, it will almost always cover the entire directory buffer - if we only modify a single entry in the buffer, then that's a fairly large cost in terms of log space and CPU overhead for random individual operations. If we decide that we are going to use a single range, then we may as well just use the dirty flag and log the entire buffer every time.
We also have to consider non-directory buffer modification patterns. freespace, inode and extent btrees are the other major types of buffers that get logged, but they also have modification patterns that lend themselves well to a small number of ranges for dirty tracking. That is, each btree block is kept compact, so when we insert or remove a record or pointer we shift then higher records/ptrs up or down as a block, and then log the lot of them. And they also often have a header that is dirtied with each insert/delete, so typically there are usually only one or two dirty ranges in a btree block.
The only metadata type that really seems to benefit from dirty range logging is the inode buffers. Specifically, for v4 sueprblocks the create transaction only dirties the regions of the inode core, so for 256 byte inodes only dirties every alternate bitmap segement. Dirty range tracking will double the required log bandwidth of inode buffers during create (roughly 25% increase on a 4k directory block size filesystem). There isn't any performance differential noticable on typical systems because the log is far from being bandwidth bound.
For v5 filesystems, this isn't an issue because the initialised inode buffers are XFS_BLI_ORDERED buffers and so their contents aren't logged.
The same problem happens with unlinks. Again there is an increase in log bandwidth on both v4 and v5 filesystems, but there isn't any performance differential that occurs because, again, the log isn't bandwidth bound. As it is, there is an existing plan of improvement to the unlinked list logging (moving the unlinked list logging into the inode core transaction) and hence that will avoid any extra overhead here as well.
Overall, I think this is a no-brainer given the performance improvements we get on large directory block size filesystems. There are some downsides, but looking at the medium-term development plans we are planning to mitigate the impact of most of those downsides. Hence I think this is the way forward.
Discuss. :)
--- fs/xfs/xfs_buf.c | 2 + fs/xfs/xfs_buf_item.c | 357 +++++++++++++++++++++++--------------------------- fs/xfs/xfs_buf_item.h | 14 ++ 3 files changed, 179 insertions(+), 194 deletions(-)
fs/xfs/xfs_buf.c +2 -0
@@ -1428,6 +1428,8 @@ xfs_buf_iomove( page = bp->b_pages[page_index]; csize =min_t (size_t,PAGE_SIZE - page_offset,BBTOB (bp->b_io_length) - boff); +if (boff + csize > bend) + csize = bend - boff;ASSERT ((csize + page_offset) <=PAGE_SIZE );
fs/xfs/xfs_buf_item.c +163 -194
446 lines not on this page · raw message
fs/xfs/xfs_buf_item.h +14 -0
@@ -57,6 +57,20 @@ typedef struct xfs_buf_log_item {unsigned int bli_recur;/* lock recursion count */ atomic_t bli_refcount;/* cnt of tp refs */ int bli_format_count;/* count of headers */ + +/* +* logging ranges. Keep a small number of distinct ranges rather than a +* bitmap which is expensive to maintain. 4 is chosen at first to be +* able to keep a header region, a tail region, and a couple separate +* regions in the middle if necessary +*/ +#define XFS_BLI_RANGES 4 +struct xfs_bli_range { +uint16_t first; +uint16_t last; + } bli_range[XFS_BLI_RANGES ]; +int bli_ranges; +struct xfs_buf_log_format *bli_formats;/* array of in-log header ptrs */ struct xfs_buf_log_format __bli_format;/* embedded in-log header */ } xfs_buf_log_item_t;
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Interesting. I had a prototype of this supporting just a single region a while ago, but never managed to get it to pass all recovery tests. That was before I separated the in-core from the on-disk log items, though.
Quoted 6 lines
Hence if we just track a signle region, it will almost always cover the entire directory buffer - if we only modify a single entry in the buffer, then that's a fairly large cost in terms of log space and CPU overhead for random individual operations. If we decide that we are going to use a single range, then we may as well just use the dirty flag and log the entire buffer every time.
Which might not be an all that bad idea given how much log bandwith we have available. Defintively would be interesting to instrument and benchmark it vs the 4 regions version.
Note that we probably should also introduce a log incompat feature to just log the range instead of converting it to the old bitmap for v5 filesystems.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · Christoph Hellwig
Interesting. I had a prototype of this supporting just a single region a while ago, but never managed to get it to pass all recovery tests. That was before I separated the in-core from the on-disk log items, though.
Yeah, that makes it much easier to deal with. It's still a bit messy though, and it still panics every so often under heavy load. IOWs I still don't quite have the range-to-bitmap accounting correct.
Quoted 10 lines
Quoted 6 lines
Hence if we just track a signle region, it will almost always cover the entire directory buffer - if we only modify a single entry in the buffer, then that's a fairly large cost in terms of log space and CPU overhead for random individual operations. If we decide that we are going to use a single range, then we may as well just use the dirty flag and log the entire buffer every time.
Which might not be an all that bad idea given how much log bandwith we have available. Defintively would be interesting to instrument and benchmark it vs the 4 regions version.
Well, for v4 filesystems under create workloads the increase is definitely noticable - I haven't got measurements to hand, though it was somethign in the order 30% or so for 256 byte inodes due to the increase in size of the inode cluster buffers being logged. For larger inodes that's going to be even worse. We don't have to worry on v5 filesystems, though, so only btree blocks are the main concern....
Quoted 3 lines
Note that we probably should also introduce a log incompat feature to just log the range instead of converting it to the old bitmap for v5 filesystems.
Yup, that's easy enough to do, and should make the v5 code even simpler and faster....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Dave Chinner
From: Dave Chinner<[email protected]>
Switching this to just track the first and last modified bytes in the block and only converting that to a dirty bitmap in the buffer log item at format time, however, gets rid of most of this dirty bitmap overhead without increasing memcpy time at all. the result is that peformance on a 64k directory block size increases to roughly 16,000 files/s withmemcpy()overhead only slightly increasing.
Looks interesting. 4 regions seems reasonable to me.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs