xfs: CPU usage optimizations for realtime allocator
v2 cover xfs: CPU usage optimizations for realtime allocator
Hello,
This is version 2 of my XFS realtime allocator opimization patch series.
Changes since v1 [1]:
- Fixed potential overflow in patch 4.
- Changed deprecated typedefs to normal struct names
- Fixed broken indentation
- Used xfs_fileoff_t instead of xfs_fsblock_t where appropriate.
- Added calls to xfs_rtbuf_cache_relse anywhere that the cache is used instead of relying on the buffers being dirtied and thus attached to the transaction.
- Clarified comments and commit messages in a few places.
- Added Darrick's Reviewed-bys.
Cover letter from v1:
Our distributed storage system uses XFS's realtime device support as a way to split an XFS filesystem between an SSD and an HDD -- we configure the HDD as the realtime device so that metadata goes on the SSD and data goes on the HDD.
We've been running this in production for a few years now, so we have some fairly fragmented filesystems. This has exposed various CPU inefficiencies in the realtime allocator. These became even worse when we experimented with using XFS_XFLAG_EXTSIZE to force files to be allocated contiguously.
This series adds several optimizations that don't change the realtime allocator's decisions, but make them happen more efficiently, mainly by avoiding redundant work. We've tested these in production and measured ~10% lower CPU utilization. Furthermore, it made it possible to use XFS_XFLAG_EXTSIZE to force contiguous allocations -- without these patches, our most fragmented systems would become unresponsive due to high CPU usage in the realtime allocator, but with them, CPU utilization is actually ~4-6% lower than before, and disk I/O utilization is 15-20% lower.
Patches 2 and 3 are preparations for later optimizations; the remaining patches are the optimizations themselves.
1: https://lore.kernel.org/linux-xfs/[email protected]/
Omar Sandoval (6):
xfs: cache last bitmap block in realtime allocator
xfs: invert the realtime summary cache
xfs: return maximum free size from xfs_rtany_summary()
xfs: limit maxlen based on available space in
xfs_rtallocate_extent_near()
xfs: don't try redundant allocations in xfs_rtallocate_extent_near()
xfs: don't look for end of extent further than necessary in
xfs_rtallocate_extent_near()
fs/xfs/libxfs/xfs_rtbitmap.c | 185 ++++++++++++++-------------- fs/xfs/xfs_mount.h | 6 +- fs/xfs/xfs_rtalloc.c | 228 ++++++++++++++++------------------- fs/xfs/xfs_rtalloc.h | 30 +++-- 4 files changed, 224 insertions(+), 225 deletions(-)
Signature · 1 line
2.41.0
PATCH v2 1/6 xfs: cache last bitmap block in realtime allocator
From: Omar Sandoval <[email protected]>
Profiling a workload on a highly fragmented realtime device showed a ton of CPU cycles being spent in xfs_trans_read_buf() called by xfs_rtbuf_get(). Further tracing showed that much of that was repeated calls to xfs_rtbuf_get() for the same block of the realtime bitmap. These come from xfs_rtallocate_extent_block(): as it walks through ranges of free bits in the bitmap, each call to xfs_rtcheck_range() and xfs_rtfind_{forw,back}() gets the same bitmap block. If the bitmap block is very fragmented, then this is _a lot_ of buffer lookups.
The realtime allocator already passes around a cache of the last used realtime summary block to avoid repeated reads (the parameters rbpp and rsb). We can do the same for the realtime bitmap.
This replaces rbpp and rsb with a struct xfs_rtbuf_cache, which caches the most recently used block for both the realtime bitmap and summary. xfs_rtbuf_get() now handles the caching instead of the callers, which requires plumbing xfs_rtbuf_cache to more functions but also makes sure we don't miss anything.
--- fs/xfs/libxfs/xfs_rtbitmap.c | 179 ++++++++++++++++++----------------- fs/xfs/xfs_rtalloc.c | 119 +++++++++++------------ fs/xfs/xfs_rtalloc.h | 30 ++++-- 3 files changed, 170 insertions(+), 158 deletions(-)
fs/xfs/libxfs/xfs_rtbitmap.c +93 -86
@@ -46,6 +46,21 @@ const struct xfs_buf_ops xfs_rtbuf_ops = { .verify_write = xfs_rtbuf_verify_write, }; +void +xfs_rtbuf_cache_relse ( +struct xfs_trans *tp,/* transaction pointer */ +struct xfs_rtbuf_cache *cache)/* cached blocks */ +{ +if (cache->bbuf) { +xfs_trans_brelse (tp, cache->bbuf); + cache->bbuf =NULL ; + } +if (cache->sbuf) { +xfs_trans_brelse (tp, cache->sbuf); + cache->sbuf =NULL ; + } +} +/* * Get a buffer for the bitmap or summary file block specified. * The buffer is returned read and locked.
@@ -56,14 +71,35 @@ xfs_rtbuf_get(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t block, /* block number in bitmap or summary */ int issum, /* is summary not bitmap */ + struct xfs_rtbuf_cache *cache, /* in/out: cached blocks */ struct xfs_buf **bpp) /* output: buffer for the block */ { + struct xfs_buf **cbpp; /* cached block buffer */ +xfs_fsblock_t *cbp;/* cached block number */ struct xfs_buf *bp;/* block buffer, result */ xfs_inode_t *ip;/* bitmap or summary inode */ xfs_bmbt_irec_t map;int nmap =1 ;int error;/* error value */ + cbpp = issum ? &cache->bbuf : &cache->sbuf; + cbp = issum ? &cache->bblock : &cache->sblock; +/* +* If we have a cached buffer, and the block number matches, use that. +*/ +if (*cbpp && *cbp == block) { + *bpp = *cbpp; +return 0 ; + } +/* +* Otherwise we have to have to get the buffer. If there was an old +* one, get rid of it first. +*/ +if (*cbpp) { +xfs_trans_brelse (tp, *cbpp); + *cbpp =NULL ; + } + ip = issum ? mp->m_rsumip : mp->m_rbmip; error =xfs_bmapi_read (ip, block,1 , &map, &nmap,0 );
@@ -82,7 +118,8 @@ xfs_rtbuf_get(xfs_trans_buf_set_type (tp, bp, issum ?XFS_BLFT_RTSUMMARY_BUF :XFS_BLFT_RTBITMAP_BUF ); - *bpp = bp; + *cbpp = *bpp = bp; + *cbp = block;return 0 ; }
@@ -96,6 +133,7 @@ xfs_rtfind_back(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t start, /* starting block to look at */ xfs_rtblock_t limit, /* last block to look at */ + struct xfs_rtbuf_cache *rtbufc, /* in/out: cache of realtime blocks */ xfs_rtblock_t *rtblock) /* out: start block found */ { xfs_rtword_t *b; /* current word in buffer */
@@ -116,7 +154,7 @@ xfs_rtfind_back(* Compute and read in starting bitmap block for starting block. */ block =XFS_BITTOBLOCK (mp, start); - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, block,0 , rtbufc, &bp);if (error) {return error; }
@@ -153,7 +191,6 @@ xfs_rtfind_back(/* * Different. Mark where we are and return. */ -xfs_trans_brelse (tp, bp); i = bit -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;
@@ -167,8 +204,7 @@ xfs_rtfind_back(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, --block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, --block,0 , rtbufc, &bp);if (error) {return error; }
@@ -199,7 +235,6 @@ xfs_rtfind_back(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); i +=XFS_NBWORD -1 -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;
@@ -213,8 +248,7 @@ xfs_rtfind_back(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, --block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, --block,0 , rtbufc, &bp);if (error) {return error; }
@@ -246,7 +280,6 @@ xfs_rtfind_back(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); i +=XFS_NBWORD -1 -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;
@@ -256,7 +289,6 @@ xfs_rtfind_back(/* * No match, return that we scanned the whole area. */ -xfs_trans_brelse (tp, bp); *rtblock = start - i +1 ;return 0 ; }
@@ -271,6 +303,7 @@ xfs_rtfind_forw(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t start, /* starting block to look at */ xfs_rtblock_t limit, /* last block to look at */ + struct xfs_rtbuf_cache *rtbufc, /* in/out: cache of realtime blocks */ xfs_rtblock_t *rtblock) /* out: start block found */ { xfs_rtword_t *b; /* current word in buffer */
@@ -291,7 +324,7 @@ xfs_rtfind_forw(* Compute and read in starting bitmap block for starting block. */ block =XFS_BITTOBLOCK (mp, start); - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, block,0 , rtbufc, &bp);if (error) {return error; }
@@ -327,7 +360,6 @@ xfs_rtfind_forw(/* * Different. Mark where we are and return. */ -xfs_trans_brelse (tp, bp); i =XFS_RTLOBIT (wdiff) - bit; *rtblock = start + i -1 ;return 0 ;
@@ -341,8 +373,7 @@ xfs_rtfind_forw(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, ++block,0 , rtbufc, &bp);if (error) {return error; }
@@ -372,7 +403,6 @@ xfs_rtfind_forw(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); i +=XFS_RTLOBIT (wdiff); *rtblock = start + i -1 ;return 0 ;
@@ -386,8 +416,7 @@ xfs_rtfind_forw(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, ++block,0 , rtbufc, &bp);if (error) {return error; }
@@ -416,7 +445,6 @@ xfs_rtfind_forw(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); i +=XFS_RTLOBIT (wdiff); *rtblock = start + i -1 ;return 0 ;
@@ -426,7 +454,6 @@ xfs_rtfind_forw(/* * No match, return that we scanned the whole area. */ -xfs_trans_brelse (tp, bp); *rtblock = start + i -1 ;return 0 ; }
@@ -447,8 +474,7 @@ xfs_rtmodify_summary_int(int log,/* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */ - struct xfs_buf **rbpp, /* in/out: summary block buffer */ - xfs_fsblock_t *rsb, /* in/out: summary block number */ + struct xfs_rtbuf_cache *rtbufc, /* in/out: cache of realtime blocks */ xfs_suminfo_t *sum) /* out: summary info for this block */ { struct xfs_buf *bp; /* buffer for the summary block */
@@ -465,30 +491,9 @@ xfs_rtmodify_summary_int(* Compute the block number in the summary file. */ sb =XFS_SUMOFFSTOBLOCK (mp, so); -/* -* If we have an old buffer, and the block number matches, use that. -*/ -if (*rbpp && *rsb == sb) - bp = *rbpp; -/* -* Otherwise we have to get the buffer. -*/ -else { -/* -* If there was an old one, get rid of it first. -*/ -if (*rbpp) -xfs_trans_brelse (tp, *rbpp); - error =xfs_rtbuf_get (mp, tp, sb,1 , &bp); -if (error) { -return error; - } -/* -* Remember this buffer and block for the next call. -*/ - *rbpp = bp; - *rsb = sb; - } + error =xfs_rtbuf_get (mp, tp, sb,1 , rtbufc, &bp); +if (error) +return error;/* * Point to the summary information, modify/log it, and/or copy it out. */
@@ -517,11 +522,9 @@ xfs_rtmodify_summary(int log,/* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */ - struct xfs_buf **rbpp, /* in/out: summary block buffer */ - xfs_fsblock_t *rsb) /* in/out: summary block number */ + struct xfs_rtbuf_cache *rtbufc) /* in/out: cache of realtime blocks */ { - return xfs_rtmodify_summary_int(mp, tp, log, bbno, -delta ,rbpp ,rsb ,NULL ); + return xfs_rtmodify_summary_int(mp, tp, log, bbno, delta, rtbufc, NULL); }/*
@@ -534,7 +537,8 @@ xfs_rtmodify_range(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t start, /* starting block to modify */ xfs_extlen_t len, /* length of extent to modify */ - int val) /* 1 for free, 0 for allocated */ + int val, /* 1 for free, 0 for allocated */ + struct xfs_rtbuf_cache *rtbufc) /* in/out: cache of realtime blocks */ { xfs_rtword_t *b; /* current word in buffer */int bit;/* bit number in the word */
@@ -555,7 +559,7 @@ xfs_rtmodify_range(/* * Read the bitmap block, and point to its data. */ - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, block,0 , rtbufc, &bp);if (error) {return error; }
@@ -600,7 +604,7 @@ xfs_rtmodify_range(xfs_trans_log_buf (tp, bp, (uint)((char *)first - (char *)bufp), (uint)((char *)b - (char *)bufp)); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, ++block,0 , rtbufc, &bp);if (error) {return error; }
@@ -640,7 +644,7 @@ xfs_rtmodify_range(xfs_trans_log_buf (tp, bp, (uint)((char *)first - (char *)bufp), (uint)((char *)b - (char *)bufp)); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, ++block,0 , rtbufc, &bp);if (error) {return error; }
@@ -690,8 +694,7 @@ xfs_rtfree_range(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t start, /* starting block to free */ xfs_extlen_t len, /* length to free */ - struct xfs_buf **rbpp, /* in/out: summary block buffer */ - xfs_fsblock_t *rsb) /* in/out: summary block number */ + struct xfs_rtbuf_cache *rtbufc) /* in/out: cache of realtime blocks */ { xfs_rtblock_t end; /* end of the freed extent */int error;/* error value */
@@ -702,7 +705,7 @@ xfs_rtfree_range(/* * Modify the bitmap to mark this extent freed. */ - error =xfs_rtmodify_range (mp, tp, start, len,1 ); + error =xfs_rtmodify_range (mp, tp, start, len,1 , rtbufc);if (error) {return error; }
@@ -711,7 +714,7 @@ xfs_rtfree_range(* We need to find the beginning and end of the extent so we can * properly update the summary. */ - error =xfs_rtfind_back (mp, tp, start,0 , &preblock); + error =xfs_rtfind_back (mp, tp, start,0 , rtbufc, &preblock);if (error) {return error; }
@@ -719,7 +722,7 @@ xfs_rtfree_range(* Find the next allocated block (end of allocated extent). */ error =xfs_rtfind_forw (mp, tp, end, mp->m_sb.sb_rextents -1 , - &postblock); + rtbufc, &postblock);if (error)return error;/*
@@ -729,7 +732,7 @@ xfs_rtfree_range(if (preblock < start) { error =xfs_rtmodify_summary (mp, tp,XFS_RTBLOCKLOG (start - preblock), -XFS_BITTOBLOCK (mp, preblock),-1 , rbpp, rsb); +XFS_BITTOBLOCK (mp, preblock),-1 , rtbufc);if (error) {return error; }
@@ -741,7 +744,7 @@ xfs_rtfree_range(if (postblock > end) { error =xfs_rtmodify_summary (mp, tp,XFS_RTBLOCKLOG (postblock - end), -XFS_BITTOBLOCK (mp, end +1 ),-1 , rbpp, rsb); +XFS_BITTOBLOCK (mp, end +1 ),-1 , rtbufc);if (error) {return error; }
@@ -752,7 +755,7 @@ xfs_rtfree_range(*/ error =xfs_rtmodify_summary (mp, tp,XFS_RTBLOCKLOG (postblock +1 - preblock), -XFS_BITTOBLOCK (mp, preblock),1 , rbpp, rsb); +XFS_BITTOBLOCK (mp, preblock),1 , rtbufc);return error; }
@@ -767,6 +770,7 @@ xfs_rtcheck_range(xfs_rtblock_t start,/* starting block number of extent */ xfs_extlen_t len, /* length of extent */ int val, /* 1 for free, 0 for allocated */ + struct xfs_rtbuf_cache *rtbufc, /* in/out: cache of realtime blocks */ xfs_rtblock_t *new, /* out: first block not matching */ int *stat) /* out: 1 for matches, 0 for not */ {
@@ -789,7 +793,7 @@ xfs_rtcheck_range(/* * Read the bitmap block. */ - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, block,0 , rtbufc, &bp);if (error) {return error; }
@@ -824,7 +828,6 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); i =XFS_RTLOBIT (wdiff) - bit; *new = start + i; *stat =0 ;
@@ -839,8 +842,7 @@ xfs_rtcheck_range(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, ++block,0 , rtbufc, &bp);if (error) {return error; }
@@ -870,7 +872,6 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); i +=XFS_RTLOBIT (wdiff); *new = start + i; *stat =0 ;
@@ -885,8 +886,7 @@ xfs_rtcheck_range(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (mp, tp, ++block,0 , rtbufc, &bp);if (error) {return error; }
@@ -915,7 +915,6 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); i +=XFS_RTLOBIT (wdiff); *new = start + i; *stat =0 ;
@@ -926,7 +925,6 @@ xfs_rtcheck_range(/* * Successful, return. */ -xfs_trans_brelse (tp, bp); *new = start + i; *stat =1 ;return 0 ;
@@ -941,20 +939,21 @@ xfs_rtcheck_alloc_range( xfs_mount_t *mp,/* file system mount point */ xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t bno, /* starting block number of extent */ - xfs_extlen_t len) /* length of extent */ + xfs_extlen_t len, /* length of extent */ + struct xfs_rtbuf_cache *rtbufc) /* in/out: cached blocks */ { xfs_rtblock_t new; /* dummy for xfs_rtcheck_range */int stat;int error; - error =xfs_rtcheck_range (mp, tp, bno, len,0 , &new, &stat); + error =xfs_rtcheck_range (mp, tp, bno, len,0 , rtbufc, &new, &stat);if (error)return error;ASSERT (stat);return 0 ; }#else -#define xfs_rtcheck_alloc_range (m,t,b,l) (0) +#define xfs_rtcheck_alloc_range (m,t,b,l,r) (0)#endif /* * Free an extent in the realtime subvolume. Length is expressed in
@@ -968,25 +967,23 @@ xfs_rtfree_extent( {int error;/* error value */ xfs_mount_t *mp;/* file system mount structure */ -xfs_fsblock_t sb;/* summary file block number */ -struct xfs_buf *sumbp =NULL ;/* summary file block buffer */ +struct xfs_rtbuf_cache rtbufc = {};/* cache of realtime blocks */ mp = tp->t_mountp;ASSERT (mp->m_rbmip->i_itemp !=NULL );ASSERT (xfs_isilocked (mp->m_rbmip,XFS_ILOCK_EXCL )); - error =xfs_rtcheck_alloc_range (mp, tp, bno, len); + error =xfs_rtcheck_alloc_range (mp, tp, bno, len, &rtbufc);if (error) -return error; + goto out;/* * Free the range of realtime blocks. */ - error =xfs_rtfree_range (mp, tp, bno, len, &sumbp, &sb); -if (error) { -return error; - } + error =xfs_rtfree_range (mp, tp, bno, len, &rtbufc); +if (error) + goto out;/* * Mark more blocks free in the superblock. */
@@ -1002,7 +999,10 @@ xfs_rtfree_extent( *(uint64_t *)&VFS_I (mp->m_rbmip)->i_atime =0 ;xfs_trans_log_inode (tp, mp->m_rbmip,XFS_ILOG_CORE ); } -return 0 ; + error =0 ; +out: +xfs_rtbuf_cache_relse (tp, &rtbufc); +return error; }/* Find all the free records within a given range. */
@@ -1021,6 +1021,7 @@ xfs_rtalloc_query_range(xfs_rtblock_t high_key;int is_free;int error =0 ; +struct xfs_rtbuf_cache rtbufc = {};if (low_rec->ar_startext > high_rec->ar_startext)return -EINVAL ;
@@ -1034,13 +1035,14 @@ xfs_rtalloc_query_range( rtstart = low_rec->ar_startext;while (rtstart <= high_key) {/* Is the first block free? */ - error =xfs_rtcheck_range (mp, tp, rtstart,1 ,1 , &rtend, - &is_free); + error =xfs_rtcheck_range (mp, tp, rtstart,1 ,1 , &rtbufc, + &rtend, &is_free);if (error)break ;/* How long does the extent go for? */ - error =xfs_rtfind_forw (mp, tp, rtstart, high_key, &rtend); + error =xfs_rtfind_forw (mp, tp, rtstart, high_key, &rtbufc, + &rtend);if (error)break ;
@@ -1056,6 +1058,8 @@ xfs_rtalloc_query_range( rtstart = rtend +1 ; } +xfs_rtbuf_cache_relse (tp, &rtbufc); +return error; }
@@ -1085,11 +1089,14 @@ xfs_rtalloc_extent_is_free(xfs_extlen_t len, bool *is_free) { + struct xfs_rtbuf_cache rtbufc = {};xfs_rtblock_t end;int matches;int error; - error =xfs_rtcheck_range (mp, tp, start, len,1 , &end, &matches); + error =xfs_rtcheck_range (mp, tp, start, len,1 , &rtbufc, &end, + &matches); +xfs_rtbuf_cache_relse (tp, &rtbufc);if (error)return error;
fs/xfs/xfs_rtalloc.c +56 -63
368 lines not on this page · raw message
fs/xfs/xfs_rtalloc.h +21 -9
@@ -101,29 +101,40 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ +struct xfs_rtbuf_cache { +struct xfs_buf *bbuf;/* bitmap block buffer */ +xfs_fileoff_t bblock;/* bitmap block number */ +struct xfs_buf *sbuf;/* summary block buffer */ +xfs_fileoff_t sblock;/* summary block number */ +}; + +void xfs_rtbuf_cache_relse (struct xfs_trans *tp,struct xfs_rtbuf_cache *cache);int xfs_rtbuf_get (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_rtblock_t block,int issum,struct xfs_buf **bpp); +xfs_rtblock_t block,int issum,struct xfs_rtbuf_cache *cache, +struct xfs_buf **bpp);int xfs_rtcheck_range (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_extlen_t len,int val, -xfs_rtblock_t *new,int *stat); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *new, +int *stat);int xfs_rtfind_back (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_rtblock_t limit, -xfs_rtblock_t *rtblock); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *rtblock);int xfs_rtfind_forw (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_rtblock_t limit, -xfs_rtblock_t *rtblock); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *rtblock);int xfs_rtmodify_range (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_rtblock_t start,xfs_extlen_t len,int val); +xfs_rtblock_t start,xfs_extlen_t len,int val, +struct xfs_rtbuf_cache *rtbufc);int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp,int log,xfs_rtblock_t bbno,int delta, -struct xfs_buf **rbpp,xfs_fsblock_t *rsb, +struct xfs_rtbuf_cache *rtbufc,xfs_suminfo_t *sum);int xfs_rtmodify_summary (struct xfs_mount *mp,struct xfs_trans *tp,int log, -xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp, -xfs_fsblock_t *rsb); +xfs_rtblock_t bbno,int delta, +struct xfs_rtbuf_cache *rtbufc);int xfs_rtfree_range (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_extlen_t len, -struct xfs_buf **rbpp,xfs_fsblock_t *rsb); +struct xfs_rtbuf_cache *rtbufc); int xfs_rtalloc_query_range(struct xfs_mount *mp, struct xfs_trans *tp, const struct xfs_rtalloc_rec *low_rec, const struct xfs_rtalloc_rec *high_rec,
@@ -143,6 +154,7 @@ int xfs_rtalloc_reinit_frextents(struct xfs_mount *mp);# define xfs_growfs_rt (mp,in) (ENOSYS)# define xfs_rtalloc_query_range (t,l,h,f,p) (ENOSYS)# define xfs_rtalloc_query_all (m,t,f,p) (ENOSYS) +# define xfs_rtbuf_cache_relse (t,c) (ENOSYS)# define xfs_rtbuf_get (m,t,b,i,p) (ENOSYS)# define xfs_verify_rtbno (m, r) (false)# define xfs_rtalloc_extent_is_free (m,t,s,l,i) (ENOSYS)
Signature · 1 line
2.41.0
Re: [PATCH v2 1/6] xfs: cache last bitmap block in realtime allocator
Quoted 27 lines · Omar Sandoval
From: Omar Sandoval <[email protected]>
Profiling a workload on a highly fragmented realtime device showed a ton of CPU cycles being spent in
xfs_trans_read_buf()called byxfs_rtbuf_get(). Further tracing showed that much of that was repeated calls toxfs_rtbuf_get()for the same block of the realtime bitmap. These come fromxfs_rtallocate_extent_block(): as it walks through ranges of free bits in the bitmap, each call toxfs_rtcheck_range()and xfs_rtfind_{forw,back}() gets the same bitmap block. If the bitmap block is very fragmented, then this is _alot_of buffer lookups.The realtime allocator already passes around a cache of the last used realtime summary block to avoid repeated reads (the parameters rbpp and rsb). We can do the same for the realtime bitmap.
This replaces rbpp and rsb with a struct
xfs_rtbuf_cache, which caches the most recently used block for both the realtime bitmap and summary.xfs_rtbuf_get()now handles the caching instead of the callers, which requires plumbingxfs_rtbuf_cacheto more functions but also makes sure we don't miss anything.Signed-off-by Omar Sandoval <[email protected]>--- fs/xfs/libxfs/xfs_rtbitmap.c | 179 ++++++++++++++++++----------------- fs/xfs/xfs_rtalloc.c | 119 +++++++++++------------ fs/xfs/xfs_rtalloc.h | 30 ++++-- 3 files changed, 170 insertions(+), 158 deletions(-)
....
fs/xfs/xfs_rtalloc.h xfs_growfs_rt( · line 108
Quoted 13 lines
fs/xfs/xfs_rtalloc.h +5
-0@@ -101,29 +101,40 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ +struct xfs_rtbuf_cache { +struct xfs_buf *bbuf;/* bitmap block buffer */ +xfs_fileoff_t bblock;/* bitmap block number */ +struct xfs_buf *sbuf;/* summary block buffer */ +xfs_fileoff_t sblock;/* summary block number */
I don't think the block numbers are file offsets? Most of the code passes the bitmap and summary block numbers around as a xfs_fsblock_t...
Quoted 31 lines
fs/xfs/xfs_rtalloc.h +12
-7+}; + +void xfs_rtbuf_cache_relse (struct xfs_trans *tp,struct xfs_rtbuf_cache *cache);int xfs_rtbuf_get (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_rtblock_t block,int issum,struct xfs_buf **bpp); +xfs_rtblock_t block,int issum,struct xfs_rtbuf_cache *cache, +struct xfs_buf **bpp);int xfs_rtcheck_range (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_extlen_t len,int val, -xfs_rtblock_t *new,int *stat); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *new, +int *stat);int xfs_rtfind_back (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_rtblock_t limit, -xfs_rtblock_t *rtblock); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *rtblock);int xfs_rtfind_forw (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_rtblock_t limit, -xfs_rtblock_t *rtblock); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *rtblock);int xfs_rtmodify_range (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_rtblock_t start,xfs_extlen_t len,int val); +xfs_rtblock_t start,xfs_extlen_t len,int val, +struct xfs_rtbuf_cache *rtbufc);int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp,int log,xfs_rtblock_t bbno,int delta, -struct xfs_buf **rbpp,xfs_fsblock_t *rsb, +struct xfs_rtbuf_cache *rtbufc,xfs_suminfo_t *sum); int xfs_rtmodify_summary(struct xfs_mount *mp, struct xfs_trans *tp, int log, - xfs_rtblock_t bbno, int delta, struct xfs_buf **rbpp,
fs/xfs/xfs_rtalloc.h xfs_growfs_rt( · line 140
Quoted 10 lines
fs/xfs/xfs_rtalloc.h +3
-2- xfs_fsblock_t *rsb); + xfs_rtblock_t bbno, int delta, + struct xfs_rtbuf_cache *rtbufc);int xfs_rtfree_range (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_extlen_t len, -struct xfs_buf **rbpp,xfs_fsblock_t *rsb); +struct xfs_rtbuf_cache *rtbufc); int xfs_rtalloc_query_range(struct xfs_mount *mp, struct xfs_trans *tp, const struct xfs_rtalloc_rec *low_rec, const struct xfs_rtalloc_rec *high_rec,
Given that this is basically plumbing a new structure into almost the entire rt allocator API, I have to wonder if this would be better done as a 'struct xfs_rtalloc_args' similar to how we pass all these args around the btree allocator (i.e. struct xfs_alloc_arg).
That is:
struct xfs_rtalloc_arg {
struct xfs_mount *mp;
struct xfs_trans *tp;
struct xfs_buf *bitmap_buf;
struct xfs_buf *summary_buf;
xfs_fsblock_t bitmap_block;
xfs_fsblock_t simmary_block;
};And now the api changes becomes a one off change like this:
-int xfs_rtcheck_range(struct xfs_mount *mp, struct xfs_trans *tp, +int xfs_rtcheck_range(struct xfs_rtalloc_arg *arg,
and the wrapping code becomes:
struct xfs_rtalloc_arg args = {
.mount = mp,
.trans = tp,
};.....
xfs_rtalloc_done(&args);
rebase this patch on an initial change like the (compile tested only) patch below? Then adding the cached buffer infrastructure becomes a much smaller change
(no api changes needed at all) and the context in which the cached buffers are valid becomes very clear. AFAICT, the rest of the patchset should be unaffected by doing this...
Cheers,
Dave.
Signature · 18 lines
Dave Chinner [email protected] xfs: consolidate realtime allocation arguments From: Dave Chinner <[email protected]> Consolidate the arguments passed around the rt allocator into a struct xfs_rtalloc_arg similar to how the btree allocator arguments are consolidated in a struct xfs_alloc_arg.... Signed-off-by: Dave Chinner <[email protected]> --- fs/xfs/libxfs/xfs_rtbitmap.c | 157 +++++++++++++++++++++++-------------------- fs/xfs/scrub/rtsummary.c | 6 +- fs/xfs/xfs_rtalloc.c | 119 +++++++++++++++++--------------- fs/xfs/xfs_rtalloc.h | 22 +++--- 4 files changed, 166 insertions(+), 138 deletions(-)
fs/xfs/libxfs/xfs_rtbitmap.c +83 -74
@@ -52,12 +52,12 @@ const struct xfs_buf_ops xfs_rtbuf_ops = {*/ int xfs_rtbuf_get ( -xfs_mount_t *mp,/* file system mount structure */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t block,/* block number in bitmap or summary */ int issum,/* is summary not bitmap */ struct xfs_buf **bpp)/* output: buffer for the block */ { +struct xfs_mount *mp = args->mount;struct xfs_buf *bp;/* block buffer, result */ xfs_inode_t *ip;/* bitmap or summary inode */ xfs_bmbt_irec_t map;
@@ -74,13 +74,13 @@ xfs_rtbuf_get(return -EFSCORRUPTED ;ASSERT (map.br_startblock !=NULLFSBLOCK ); - error =xfs_trans_read_buf (mp, tp, mp->m_ddev_targp, + error =xfs_trans_read_buf (mp, args->trans, mp->m_ddev_targp,XFS_FSB_TO_DADDR (mp, map.br_startblock), mp->m_bsize,0 , &bp, &xfs_rtbuf_ops);if (error)return error; -xfs_trans_buf_set_type (tp, bp, issum ?XFS_BLFT_RTSUMMARY_BUF +xfs_trans_buf_set_type (args->trans, bp, issum ?XFS_BLFT_RTSUMMARY_BUF :XFS_BLFT_RTBITMAP_BUF ); *bpp = bp;return 0 ;
@@ -92,12 +92,12 @@ xfs_rtbuf_get(*/ int xfs_rtfind_back ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to look at */ xfs_rtblock_t limit,/* last block to look at */ xfs_rtblock_t *rtblock)/* out: start block found */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */
@@ -116,7 +116,7 @@ xfs_rtfind_back(* Compute and read in starting bitmap block for starting block. */ block =XFS_BITTOBLOCK (mp, start); - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }
@@ -153,7 +153,7 @@ xfs_rtfind_back(/* * Different. Mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i = bit -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;
@@ -167,8 +167,8 @@ xfs_rtfind_back(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, --block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, --block,0 , &bp);if (error) {return error; }
@@ -199,7 +199,7 @@ xfs_rtfind_back(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_NBWORD -1 -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;
@@ -213,8 +213,8 @@ xfs_rtfind_back(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, --block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, --block,0 , &bp);if (error) {return error; }
@@ -246,7 +246,7 @@ xfs_rtfind_back(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_NBWORD -1 -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;
@@ -256,7 +256,7 @@ xfs_rtfind_back(/* * No match, return that we scanned the whole area. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); *rtblock = start - i +1 ;return 0 ; }
@@ -267,12 +267,12 @@ xfs_rtfind_back(*/ int xfs_rtfind_forw ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to look at */ xfs_rtblock_t limit,/* last block to look at */ xfs_rtblock_t *rtblock)/* out: start block found */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */
@@ -291,7 +291,7 @@ xfs_rtfind_forw(* Compute and read in starting bitmap block for starting block. */ block =XFS_BITTOBLOCK (mp, start); - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }
@@ -327,7 +327,7 @@ xfs_rtfind_forw(/* * Different. Mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i =XFS_RTLOBIT (wdiff) - bit; *rtblock = start + i -1 ;return 0 ;
@@ -341,8 +341,8 @@ xfs_rtfind_forw(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }
@@ -372,7 +372,7 @@ xfs_rtfind_forw(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *rtblock = start + i -1 ;return 0 ;
@@ -386,8 +386,8 @@ xfs_rtfind_forw(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }
@@ -416,7 +416,7 @@ xfs_rtfind_forw(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *rtblock = start + i -1 ;return 0 ;
@@ -426,7 +426,7 @@ xfs_rtfind_forw(/* * No match, return that we scanned the whole area. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); *rtblock = start + i -1 ;return 0 ; }
@@ -442,8 +442,7 @@ xfs_rtfind_forw(*/ int xfs_rtmodify_summary_int( - xfs_mount_t *mp, /* file system mount structure */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, int log, /* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */
@@ -451,6 +450,7 @@ xfs_rtmodify_summary_int( xfs_fsblock_t *rsb, /* in/out: summary block number */ xfs_suminfo_t *sum) /* out: summary info for this block */ { +struct xfs_mount *mp = args->mount;struct xfs_buf *bp;/* buffer for the summary block */ int error;/* error value */ xfs_fsblock_t sb;/* summary fsblock */
@@ -478,8 +478,8 @@ xfs_rtmodify_summary_int(* If there was an old one, get rid of it first. */ if (*rbpp) -xfs_trans_brelse (tp, *rbpp); - error =xfs_rtbuf_get (mp, tp, sb,1 , &bp); +xfs_trans_brelse (args->trans, *rbpp); + error =xfs_rtbuf_get (args, sb,1 , &bp);if (error) {return error; }
@@ -503,7 +503,7 @@ xfs_rtmodify_summary_int(if (*sp !=0 && log < mp->m_rsum_cache[bbno]) mp->m_rsum_cache[bbno] = log; } -xfs_trans_log_buf (tp, bp, first, first +sizeof (*sp) -1 ); +xfs_trans_log_buf (args->trans, bp, first, first +sizeof (*sp) -1 ); }if (sum) *sum = *sp;
@@ -512,15 +512,14 @@ xfs_rtmodify_summary_int(int xfs_rtmodify_summary ( -xfs_mount_t *mp,/* file system mount structure */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,int log,/* log2 of extent size */ xfs_rtblock_t bbno,/* bitmap block number */ int delta,/* change to make to summary info */ struct xfs_buf **rbpp,/* in/out: summary block buffer */ xfs_fsblock_t *rsb)/* in/out: summary block number */ { -return xfs_rtmodify_summary_int (mp, tp, log, bbno, +return xfs_rtmodify_summary_int (args, log, bbno, delta, rbpp, rsb,NULL ); }
@@ -530,12 +529,12 @@ xfs_rtmodify_summary(*/ int xfs_rtmodify_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to modify */ xfs_extlen_t len,/* length of extent to modify */ int val)/* 1 for free, 0 for allocated */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */
@@ -555,7 +554,7 @@ xfs_rtmodify_range(/* * Read the bitmap block, and point to its data. */ - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }
@@ -597,10 +596,10 @@ xfs_rtmodify_range(* Log the changed part of this block. * Get the next one. */ -xfs_trans_log_buf (tp, bp, +xfs_trans_log_buf (args->trans, bp, (uint)((char *)first - (char *)bufp), (uint)((char *)b - (char *)bufp)); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }
@@ -637,10 +636,10 @@ xfs_rtmodify_range(* Log the changed part of this block. * Get the next one. */ -xfs_trans_log_buf (tp, bp, +xfs_trans_log_buf (args->trans, bp, (uint)((char *)first - (char *)bufp), (uint)((char *)b - (char *)bufp)); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }
@@ -675,8 +674,9 @@ xfs_rtmodify_range(* Log any remaining changed bytes. */ if (b > first) -xfs_trans_log_buf (tp, bp, (uint)((char *)first - (char *)bufp), - (uint)((char *)b - (char *)bufp -1 )); +xfs_trans_log_buf (args->trans, bp, + (uint)((char *)first - (char *)bufp), + (uint)((char *)b - (char *)bufp -1 ));return 0 ; }
@@ -686,13 +686,13 @@ xfs_rtmodify_range(*/ int xfs_rtfree_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to free */ xfs_extlen_t len,/* length to free */ struct xfs_buf **rbpp,/* in/out: summary block buffer */ xfs_fsblock_t *rsb)/* in/out: summary block number */ { +struct xfs_mount *mp = args->mount;xfs_rtblock_t end;/* end of the freed extent */ int error;/* error value */ xfs_rtblock_t postblock;/* first block freed > end */
@@ -702,7 +702,7 @@ xfs_rtfree_range(/* * Modify the bitmap to mark this extent freed. */ - error =xfs_rtmodify_range (mp, tp, start, len,1 ); + error =xfs_rtmodify_range (args, start, len,1 );if (error) {return error; }
@@ -711,14 +711,14 @@ xfs_rtfree_range(* We need to find the beginning and end of the extent so we can * properly update the summary. */ - error =xfs_rtfind_back (mp, tp, start,0 , &preblock); + error =xfs_rtfind_back (args, start,0 , &preblock);if (error) {return error; }/* * Find the next allocated block (end of allocated extent). */ - error =xfs_rtfind_forw (mp, tp, end, mp->m_sb.sb_rextents -1 , + error =xfs_rtfind_forw (args, end, mp->m_sb.sb_rextents -1 , &postblock);if (error)return error;
@@ -727,7 +727,7 @@ xfs_rtfree_range(* old extent, add summary data for them to be allocated. */ if (preblock < start) { - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (start - preblock),XFS_BITTOBLOCK (mp, preblock),-1 , rbpp, rsb);if (error) {
@@ -739,7 +739,7 @@ xfs_rtfree_range(* old extent, add summary data for them to be allocated. */ if (postblock > end) { - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (postblock - end),XFS_BITTOBLOCK (mp, end +1 ),-1 , rbpp, rsb);if (error) {
@@ -750,7 +750,7 @@ xfs_rtfree_range(* Increment the summary information corresponding to the entire * (new) free extent. */ - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (postblock +1 - preblock),XFS_BITTOBLOCK (mp, preblock),1 , rbpp, rsb);return error;
@@ -762,14 +762,14 @@ xfs_rtfree_range(*/ int xfs_rtcheck_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block number of extent */ xfs_extlen_t len,/* length of extent */ int val,/* 1 for free, 0 for allocated */ xfs_rtblock_t *new,/* out: first block not matching */ int *stat)/* out: 1 for matches, 0 for not */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */
@@ -789,7 +789,7 @@ xfs_rtcheck_range(/* * Read the bitmap block. */ - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }
@@ -824,7 +824,7 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i =XFS_RTLOBIT (wdiff) - bit; *new = start + i; *stat =0 ;
@@ -839,8 +839,8 @@ xfs_rtcheck_range(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }
@@ -870,7 +870,7 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *new = start + i; *stat =0 ;
@@ -885,8 +885,8 @@ xfs_rtcheck_range(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }
@@ -915,7 +915,7 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *new = start + i; *stat =0 ;
@@ -926,7 +926,7 @@ xfs_rtcheck_range(/* * Successful, return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); *new = start + i; *stat =1 ;return 0 ;
@@ -938,8 +938,7 @@ xfs_rtcheck_range(*/ STATIC int /* error */xfs_rtcheck_alloc_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t bno,/* starting block number of extent */ xfs_extlen_t len)/* length of extent */ {
@@ -947,14 +946,14 @@ xfs_rtcheck_alloc_range(int stat;int error; - error =xfs_rtcheck_range (mp, tp, bno, len,0 , &new, &stat); + error =xfs_rtcheck_range (args, bno, len,0 , &new, &stat);if (error)return error;ASSERT (stat); return 0; }#else -#define xfs_rtcheck_alloc_range (m,t,b,l) (0) +#define xfs_rtcheck_alloc_range (a,b,l) (0)#endif /* * Free an extent in the realtime subvolume. Length is expressed in
@@ -966,24 +965,26 @@ xfs_rtfree_extent( xfs_rtblock_t bno, /* starting block number to free */ xfs_extlen_t len) /* length of extent freed */ { +struct xfs_mount *mp = tp->t_mountp; +struct xfs_rtalloc_args args = { + .mount = mp, + .trans = tp, + };int error;/* error value */ -xfs_mount_t *mp;/* file system mount structure */ xfs_fsblock_t sb;/* summary file block number */ struct xfs_buf *sumbp =NULL ;/* summary file block buffer */ - mp = tp->t_mountp; -ASSERT (mp->m_rbmip->i_itemp !=NULL );ASSERT (xfs_isilocked (mp->m_rbmip,XFS_ILOCK_EXCL )); - error =xfs_rtcheck_alloc_range (mp, tp, bno, len); + error =xfs_rtcheck_alloc_range (&args, bno, len);if (error)return error;/* * Free the range of realtime blocks. */ - error =xfs_rtfree_range (mp, tp, bno, len, &sumbp, &sb); + error =xfs_rtfree_range (&args, bno, len, &sumbp, &sb);if (error) {return error; }
@@ -1015,6 +1016,10 @@ xfs_rtalloc_query_range(xfs_rtalloc_query_range_fn fn, void *priv) { + struct xfs_rtalloc_args args = { + .mount = mp, + .trans = tp, + };struct xfs_rtalloc_rec rec;xfs_rtblock_t rtstart;xfs_rtblock_t rtend;
@@ -1034,13 +1039,13 @@ xfs_rtalloc_query_range( rtstart = low_rec->ar_startext;while (rtstart <= high_key) {/* Is the first block free? */ - error =xfs_rtcheck_range (mp, tp, rtstart,1 ,1 , &rtend, + error =xfs_rtcheck_range (&args, rtstart,1 ,1 , &rtend, &is_free);if (error)break ;/* How long does the extent go for? */ - error =xfs_rtfind_forw (mp, tp, rtstart, high_key, &rtend); + error =xfs_rtfind_forw (&args, rtstart, high_key, &rtend);if (error)break ;
@@ -1085,11 +1090,15 @@ xfs_rtalloc_extent_is_free(xfs_extlen_t len, bool *is_free) { + struct xfs_rtalloc_args args = { + .mount = mp, + .trans = tp, + };xfs_rtblock_t end;int matches;int error; - error =xfs_rtcheck_range (mp, tp, start, len,1 , &end, &matches); + error =xfs_rtcheck_range (&args, start, len,1 , &end, &matches);if (error)return error;
fs/xfs/scrub/rtsummary.c +5 -1
@@ -177,6 +177,10 @@ STATIC intxchk_rtsum_compare ( struct xfs_scrub *sc) { +struct xfs_rtalloc_args args = { + .mount = sc->mp, + .trans = sc->tp, + };struct xfs_mount *mp = sc->mp;struct xfs_buf *bp;struct xfs_bmbt_irec map;
@@ -205,7 +209,7 @@ xchk_rtsum_compare( }/* Read a block's worth of ondisk rtsummary file. */ - error =xfs_rtbuf_get (mp, sc->tp, off,1 , &bp); + error =xfs_rtbuf_get (&args, off,1 , &bp);if (!xchk_fblock_process_error (sc,XFS_DATA_FORK , off, &error))return error;
fs/xfs/xfs_rtalloc.c +64 -55
406 lines not on this page · raw message
fs/xfs/xfs_rtalloc.h +14 -8
@@ -11,6 +11,11 @@struct xfs_mount ;struct xfs_trans ; +struct xfs_rtalloc_args { +struct xfs_mount *mount; +struct xfs_trans *trans; +}; +/* * XXX: Most of the realtime allocation functions deal in units of realtime * extents, not realtime blocks. This looks funny when paired with the type
@@ -101,29 +106,30 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ -int xfs_rtbuf_get (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtbuf_get (struct xfs_rtalloc_args *args,xfs_rtblock_t block,int issum,struct xfs_buf **bpp); -int xfs_rtcheck_range (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtcheck_range (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_extlen_t len,int val,xfs_rtblock_t *new,int *stat); -int xfs_rtfind_back (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtfind_back (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_rtblock_t limit,xfs_rtblock_t *rtblock); -int xfs_rtfind_forw (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtfind_forw (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_rtblock_t limit,xfs_rtblock_t *rtblock); -int xfs_rtmodify_range (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtmodify_range (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_extlen_t len,int val); -int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtmodify_summary_int (struct xfs_rtalloc_args *args,int log,xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp,xfs_fsblock_t *rsb,xfs_suminfo_t *sum); -int xfs_rtmodify_summary (struct xfs_mount *mp,struct xfs_trans *tp,int log, +int xfs_rtmodify_summary (struct xfs_rtalloc_args *args,int log,xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp,xfs_fsblock_t *rsb); -int xfs_rtfree_range (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtfree_range (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_extlen_t len,struct xfs_buf **rbpp,xfs_fsblock_t *rsb); + int xfs_rtalloc_query_range(struct xfs_mount *mp, struct xfs_trans *tp, const struct xfs_rtalloc_rec *low_rec, const struct xfs_rtalloc_rec *high_rec,
Re: [PATCH v2 1/6] xfs: cache last bitmap block in realtime allocator
Quoted 40 lines · Dave Chinner
Quoted 27 lines · Omar Sandoval
From: Omar Sandoval <[email protected]>
Profiling a workload on a highly fragmented realtime device showed a ton of CPU cycles being spent in
xfs_trans_read_buf()called byxfs_rtbuf_get(). Further tracing showed that much of that was repeated calls toxfs_rtbuf_get()for the same block of the realtime bitmap. These come fromxfs_rtallocate_extent_block(): as it walks through ranges of free bits in the bitmap, each call toxfs_rtcheck_range()and xfs_rtfind_{forw,back}() gets the same bitmap block. If the bitmap block is very fragmented, then this is _alot_of buffer lookups.The realtime allocator already passes around a cache of the last used realtime summary block to avoid repeated reads (the parameters rbpp and rsb). We can do the same for the realtime bitmap.
This replaces rbpp and rsb with a struct
xfs_rtbuf_cache, which caches the most recently used block for both the realtime bitmap and summary.xfs_rtbuf_get()now handles the caching instead of the callers, which requires plumbingxfs_rtbuf_cacheto more functions but also makes sure we don't miss anything.Signed-off-by Omar Sandoval <[email protected]>--- fs/xfs/libxfs/xfs_rtbitmap.c | 179 ++++++++++++++++++----------------- fs/xfs/xfs_rtalloc.c | 119 +++++++++++------------ fs/xfs/xfs_rtalloc.h | 30 ++++-- 3 files changed, 170 insertions(+), 158 deletions(-)....
Quoted 9 lines
fs/xfs/xfs_rtalloc.h +1
-0@@ -101,29 +101,40 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ +struct xfs_rtbuf_cache {
Quoted 8 lines
fs/xfs/xfs_rtalloc.h xfs_growfs_rt( · line 108
Quoted 4 lines
fs/xfs/xfs_rtalloc.h +4
-0+struct xfs_buf *bbuf;/* bitmap block buffer */ +xfs_fileoff_t bblock;/* bitmap block number */ +struct xfs_buf *sbuf;/* summary block buffer */ +xfs_fileoff_t sblock;/* summary block number */ I don't think the block numbers are file offsets? Most of the code passes the bitmap and summary block numbers around as a
xfs_fsblock_t...
They're fed into xfs_bmapi_read as the xfs_fileoff_t parameter.
I have a whole series cleaning up all of the units abuse in the rt code part of the realtime modernization patchdeluge:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=clean-up-realtime-units
Here's the specific patch that cleans up this one part:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/commit/?h=realtime-groups&id=b344bd4bd655576f8bda193b0e33f471a6295b05
With the unit cleanups in place, I can actually tell what's going on in the rt allocator just by looking at variables types, instead of trying to guess whether a particular xfs_fsblock_t refers to (a) segmented address on the data device, (b) block in rt volume, (c) extent in rt volume, (d) block offset in bitmap or summaryfile, or (e) other random places where someone needed a u64.
Quoted 66 lines
Quoted 31 lines
fs/xfs/xfs_rtalloc.h +12
-7+}; + +void xfs_rtbuf_cache_relse (struct xfs_trans *tp,struct xfs_rtbuf_cache *cache);int xfs_rtbuf_get (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_rtblock_t block,int issum,struct xfs_buf **bpp); +xfs_rtblock_t block,int issum,struct xfs_rtbuf_cache *cache, +struct xfs_buf **bpp);int xfs_rtcheck_range (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_extlen_t len,int val, -xfs_rtblock_t *new,int *stat); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *new, +int *stat);int xfs_rtfind_back (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_rtblock_t limit, -xfs_rtblock_t *rtblock); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *rtblock);int xfs_rtfind_forw (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_rtblock_t limit, -xfs_rtblock_t *rtblock); +struct xfs_rtbuf_cache *rtbufc,xfs_rtblock_t *rtblock);int xfs_rtmodify_range (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_rtblock_t start,xfs_extlen_t len,int val); +xfs_rtblock_t start,xfs_extlen_t len,int val, +struct xfs_rtbuf_cache *rtbufc);int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp,int log,xfs_rtblock_t bbno,int delta, -struct xfs_buf **rbpp,xfs_fsblock_t *rsb, +struct xfs_rtbuf_cache *rtbufc,xfs_suminfo_t *sum); int xfs_rtmodify_summary(struct xfs_mount *mp, struct xfs_trans *tp, int log, - xfs_rtblock_t bbno, int delta, struct xfs_buf **rbpp,fs/xfs/xfs_rtalloc.h xfs_growfs_rt( · line 140
Quoted 10 lines
fs/xfs/xfs_rtalloc.h +3
-2- xfs_fsblock_t *rsb); + xfs_rtblock_t bbno, int delta, + struct xfs_rtbuf_cache *rtbufc);int xfs_rtfree_range (struct xfs_mount *mp,struct xfs_trans *tp,xfs_rtblock_t start,xfs_extlen_t len, -struct xfs_buf **rbpp,xfs_fsblock_t *rsb); +struct xfs_rtbuf_cache *rtbufc); int xfs_rtalloc_query_range(struct xfs_mount *mp, struct xfs_trans *tp, const struct xfs_rtalloc_rec *low_rec, const struct xfs_rtalloc_rec *high_rec,Given that this is basically plumbing a new structure into almost the entire rt allocator API, I have to wonder if this would be better done as a 'struct
xfs_rtalloc_args' similar to how we pass all these args around the btree allocator (i.e. structxfs_alloc_arg).That is:
struct xfs_rtalloc_arg { struct xfs_mount *mp; struct xfs_trans *tp; struct xfs_buf *bitmap_buf; struct xfs_buf *summary_buf; xfs_fsblock_t bitmap_block; xfs_fsblock_t simmary_block; };And now the api changes becomes a one off change like this:
(quoted diff) +1
-1-int xfs_rtcheck_range(struct xfs_mount *mp, struct xfs_trans *tp, +int xfs_rtcheck_range(struct xfs_rtalloc_arg *arg,and the wrapping code becomes:
Quoted 7 lines
struct xfs_rtalloc_arg args = { .mount = mp, .trans = tp, };.....
xfs_rtalloc_done(&args);
Yeah, I was thinking about changing all the APIs to pass a structure around like we do for the bnobt. :)
Quoted 6 lines
rebase this patch on an initial change like the (compile tested only) patch below? Then adding the cached buffer infrastructure becomes a much smaller change
(no api changes needed at all) and the context in which the cached buffers are valid becomes very clear. AFAICT, the rest of the patchset should be unaffected by doing this...
Could someone (Omar?) take a look at my other rt cleanups too? I'd very much like to get them merged out of my dev tree.
--D
Quoted 1,127 lines
Cheers,
Dave.
Signature · 18 lines
Dave Chinner [email protected] xfs: consolidate realtime allocation arguments From: Dave Chinner <[email protected]> Consolidate the arguments passed around the rt allocator into a struct xfs_rtalloc_arg similar to how the btree allocator arguments are consolidated in a struct xfs_alloc_arg.... Signed-off-by: Dave Chinner <[email protected]> --- fs/xfs/libxfs/xfs_rtbitmap.c | 157 +++++++++++++++++++++++-------------------- fs/xfs/scrub/rtsummary.c | 6 +- fs/xfs/xfs_rtalloc.c | 119 +++++++++++++++++--------------- fs/xfs/xfs_rtalloc.h | 22 +++--- 4 files changed, 166 insertions(+), 138 deletions(-)fs/xfs/libxfs/xfs_rtbitmap.c +83
-74@@ -52,12 +52,12 @@ const struct xfs_buf_ops xfs_rtbuf_ops = {*/ int xfs_rtbuf_get ( -xfs_mount_t *mp,/* file system mount structure */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t block,/* block number in bitmap or summary */ int issum,/* is summary not bitmap */ struct xfs_buf **bpp)/* output: buffer for the block */ { +struct xfs_mount *mp = args->mount;struct xfs_buf *bp;/* block buffer, result */ xfs_inode_t *ip;/* bitmap or summary inode */ xfs_bmbt_irec_t map;@@ -74,13 +74,13 @@ xfs_rtbuf_get(return -EFSCORRUPTED ;ASSERT (map.br_startblock !=NULLFSBLOCK ); - error =xfs_trans_read_buf (mp, tp, mp->m_ddev_targp, + error =xfs_trans_read_buf (mp, args->trans, mp->m_ddev_targp,XFS_FSB_TO_DADDR (mp, map.br_startblock), mp->m_bsize,0 , &bp, &xfs_rtbuf_ops);if (error)return error; -xfs_trans_buf_set_type (tp, bp, issum ?XFS_BLFT_RTSUMMARY_BUF +xfs_trans_buf_set_type (args->trans, bp, issum ?XFS_BLFT_RTSUMMARY_BUF :XFS_BLFT_RTBITMAP_BUF ); *bpp = bp;return 0 ;@@ -92,12 +92,12 @@ xfs_rtbuf_get(*/ int xfs_rtfind_back ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to look at */ xfs_rtblock_t limit,/* last block to look at */ xfs_rtblock_t *rtblock)/* out: start block found */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ @@ -116,7 +116,7 @@ xfs_rtfind_back(* Compute and read in starting bitmap block for starting block. */ block =XFS_BITTOBLOCK (mp, start); - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }@@ -153,7 +153,7 @@ xfs_rtfind_back(/* * Different. Mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i = bit -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;@@ -167,8 +167,8 @@ xfs_rtfind_back(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, --block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, --block,0 , &bp);if (error) {return error; }@@ -199,7 +199,7 @@ xfs_rtfind_back(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_NBWORD -1 -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;@@ -213,8 +213,8 @@ xfs_rtfind_back(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, --block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, --block,0 , &bp);if (error) {return error; }@@ -246,7 +246,7 @@ xfs_rtfind_back(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_NBWORD -1 -XFS_RTHIBIT (wdiff); *rtblock = start - i +1 ;return 0 ;@@ -256,7 +256,7 @@ xfs_rtfind_back(/* * No match, return that we scanned the whole area. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); *rtblock = start - i +1 ;return 0 ; }@@ -267,12 +267,12 @@ xfs_rtfind_back(*/ int xfs_rtfind_forw ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to look at */ xfs_rtblock_t limit,/* last block to look at */ xfs_rtblock_t *rtblock)/* out: start block found */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ @@ -291,7 +291,7 @@ xfs_rtfind_forw(* Compute and read in starting bitmap block for starting block. */ block =XFS_BITTOBLOCK (mp, start); - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }@@ -327,7 +327,7 @@ xfs_rtfind_forw(/* * Different. Mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i =XFS_RTLOBIT (wdiff) - bit; *rtblock = start + i -1 ;return 0 ;@@ -341,8 +341,8 @@ xfs_rtfind_forw(/* * If done with this block, get the previous one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }@@ -372,7 +372,7 @@ xfs_rtfind_forw(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *rtblock = start + i -1 ;return 0 ;@@ -386,8 +386,8 @@ xfs_rtfind_forw(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }@@ -416,7 +416,7 @@ xfs_rtfind_forw(/* * Different, mark where we are and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *rtblock = start + i -1 ;return 0 ;@@ -426,7 +426,7 @@ xfs_rtfind_forw(/* * No match, return that we scanned the whole area. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); *rtblock = start + i -1 ;return 0 ; }@@ -442,8 +442,7 @@ xfs_rtfind_forw(*/ int xfs_rtmodify_summary_int( - xfs_mount_t *mp, /* file system mount structure */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, int log, /* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */@@ -451,6 +450,7 @@ xfs_rtmodify_summary_int( xfs_fsblock_t *rsb, /* in/out: summary block number */ xfs_suminfo_t *sum) /* out: summary info for this block */ { +struct xfs_mount *mp = args->mount;struct xfs_buf *bp;/* buffer for the summary block */ int error;/* error value */ xfs_fsblock_t sb;/* summary fsblock */ @@ -478,8 +478,8 @@ xfs_rtmodify_summary_int(* If there was an old one, get rid of it first. */ if (*rbpp) -xfs_trans_brelse (tp, *rbpp); - error =xfs_rtbuf_get (mp, tp, sb,1 , &bp); +xfs_trans_brelse (args->trans, *rbpp); + error =xfs_rtbuf_get (args, sb,1 , &bp);if (error) {return error; }@@ -503,7 +503,7 @@ xfs_rtmodify_summary_int(if (*sp !=0 && log < mp->m_rsum_cache[bbno]) mp->m_rsum_cache[bbno] = log; } -xfs_trans_log_buf (tp, bp, first, first +sizeof (*sp) -1 ); +xfs_trans_log_buf (args->trans, bp, first, first +sizeof (*sp) -1 ); }if (sum) *sum = *sp;@@ -512,15 +512,14 @@ xfs_rtmodify_summary_int(int xfs_rtmodify_summary ( -xfs_mount_t *mp,/* file system mount structure */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,int log,/* log2 of extent size */ xfs_rtblock_t bbno,/* bitmap block number */ int delta,/* change to make to summary info */ struct xfs_buf **rbpp,/* in/out: summary block buffer */ xfs_fsblock_t *rsb)/* in/out: summary block number */ { -return xfs_rtmodify_summary_int (mp, tp, log, bbno, +return xfs_rtmodify_summary_int (args, log, bbno, delta, rbpp, rsb,NULL ); }@@ -530,12 +529,12 @@ xfs_rtmodify_summary(*/ int xfs_rtmodify_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to modify */ xfs_extlen_t len,/* length of extent to modify */ int val)/* 1 for free, 0 for allocated */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ @@ -555,7 +554,7 @@ xfs_rtmodify_range(/* * Read the bitmap block, and point to its data. */ - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }@@ -597,10 +596,10 @@ xfs_rtmodify_range(* Log the changed part of this block. * Get the next one. */ -xfs_trans_log_buf (tp, bp, +xfs_trans_log_buf (args->trans, bp, (uint)((char *)first - (char *)bufp), (uint)((char *)b - (char *)bufp)); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }@@ -637,10 +636,10 @@ xfs_rtmodify_range(* Log the changed part of this block. * Get the next one. */ -xfs_trans_log_buf (tp, bp, +xfs_trans_log_buf (args->trans, bp, (uint)((char *)first - (char *)bufp), (uint)((char *)b - (char *)bufp)); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }@@ -675,8 +674,9 @@ xfs_rtmodify_range(* Log any remaining changed bytes. */ if (b > first) -xfs_trans_log_buf (tp, bp, (uint)((char *)first - (char *)bufp), - (uint)((char *)b - (char *)bufp -1 )); +xfs_trans_log_buf (args->trans, bp, + (uint)((char *)first - (char *)bufp), + (uint)((char *)b - (char *)bufp -1 ));return 0 ; }@@ -686,13 +686,13 @@ xfs_rtmodify_range(*/ int xfs_rtfree_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block to free */ xfs_extlen_t len,/* length to free */ struct xfs_buf **rbpp,/* in/out: summary block buffer */ xfs_fsblock_t *rsb)/* in/out: summary block number */ { +struct xfs_mount *mp = args->mount;xfs_rtblock_t end;/* end of the freed extent */ int error;/* error value */ xfs_rtblock_t postblock;/* first block freed > end */ @@ -702,7 +702,7 @@ xfs_rtfree_range(/* * Modify the bitmap to mark this extent freed. */ - error =xfs_rtmodify_range (mp, tp, start, len,1 ); + error =xfs_rtmodify_range (args, start, len,1 );if (error) {return error; }@@ -711,14 +711,14 @@ xfs_rtfree_range(* We need to find the beginning and end of the extent so we can * properly update the summary. */ - error =xfs_rtfind_back (mp, tp, start,0 , &preblock); + error =xfs_rtfind_back (args, start,0 , &preblock);if (error) {return error; }/* * Find the next allocated block (end of allocated extent). */ - error =xfs_rtfind_forw (mp, tp, end, mp->m_sb.sb_rextents -1 , + error =xfs_rtfind_forw (args, end, mp->m_sb.sb_rextents -1 , &postblock);if (error)return error;@@ -727,7 +727,7 @@ xfs_rtfree_range(* old extent, add summary data for them to be allocated. */ if (preblock < start) { - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (start - preblock),XFS_BITTOBLOCK (mp, preblock),-1 , rbpp, rsb);if (error) {@@ -739,7 +739,7 @@ xfs_rtfree_range(* old extent, add summary data for them to be allocated. */ if (postblock > end) { - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (postblock - end),XFS_BITTOBLOCK (mp, end +1 ),-1 , rbpp, rsb);if (error) {@@ -750,7 +750,7 @@ xfs_rtfree_range(* Increment the summary information corresponding to the entire * (new) free extent. */ - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (postblock +1 - preblock),XFS_BITTOBLOCK (mp, preblock),1 , rbpp, rsb);return error;@@ -762,14 +762,14 @@ xfs_rtfree_range(*/ int xfs_rtcheck_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* starting block number of extent */ xfs_extlen_t len,/* length of extent */ int val,/* 1 for free, 0 for allocated */ xfs_rtblock_t *new,/* out: first block not matching */ int *stat)/* out: 1 for matches, 0 for not */ { +struct xfs_mount *mp = args->mount;xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ @@ -789,7 +789,7 @@ xfs_rtcheck_range(/* * Read the bitmap block. */ - error =xfs_rtbuf_get (mp, tp, block,0 , &bp); + error =xfs_rtbuf_get (args, block,0 , &bp);if (error) {return error; }@@ -824,7 +824,7 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i =XFS_RTLOBIT (wdiff) - bit; *new = start + i; *stat =0 ;@@ -839,8 +839,8 @@ xfs_rtcheck_range(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }@@ -870,7 +870,7 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *new = start + i; *stat =0 ;@@ -885,8 +885,8 @@ xfs_rtcheck_range(/* * If done with this block, get the next one. */ -xfs_trans_brelse (tp, bp); - error =xfs_rtbuf_get (mp, tp, ++block,0 , &bp); +xfs_trans_brelse (args->trans, bp); + error =xfs_rtbuf_get (args, ++block,0 , &bp);if (error) {return error; }@@ -915,7 +915,7 @@ xfs_rtcheck_range(/* * Different, compute first wrong bit and return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); i +=XFS_RTLOBIT (wdiff); *new = start + i; *stat =0 ;@@ -926,7 +926,7 @@ xfs_rtcheck_range(/* * Successful, return. */ -xfs_trans_brelse (tp, bp); +xfs_trans_brelse (args->trans, bp); *new = start + i; *stat =1 ;return 0 ;@@ -938,8 +938,7 @@ xfs_rtcheck_range(*/ STATIC int /* error */xfs_rtcheck_alloc_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t bno,/* starting block number of extent */ xfs_extlen_t len)/* length of extent */ {@@ -947,14 +946,14 @@ xfs_rtcheck_alloc_range(int stat;int error; - error =xfs_rtcheck_range (mp, tp, bno, len,0 , &new, &stat); + error =xfs_rtcheck_range (args, bno, len,0 , &new, &stat);if (error)return error;ASSERT (stat); return 0; }#else -#define xfs_rtcheck_alloc_range (m,t,b,l) (0) +#define xfs_rtcheck_alloc_range (a,b,l) (0)#endif /* * Free an extent in the realtime subvolume. Length is expressed in @@ -966,24 +965,26 @@ xfs_rtfree_extent( xfs_rtblock_t bno, /* starting block number to free */ xfs_extlen_t len) /* length of extent freed */ { +struct xfs_mount *mp = tp->t_mountp; +struct xfs_rtalloc_args args = { + .mount = mp, + .trans = tp, + };int error;/* error value */ -xfs_mount_t *mp;/* file system mount structure */ xfs_fsblock_t sb;/* summary file block number */ struct xfs_buf *sumbp =NULL ;/* summary file block buffer */ - mp = tp->t_mountp; -ASSERT (mp->m_rbmip->i_itemp !=NULL );ASSERT (xfs_isilocked (mp->m_rbmip,XFS_ILOCK_EXCL )); - error =xfs_rtcheck_alloc_range (mp, tp, bno, len); + error =xfs_rtcheck_alloc_range (&args, bno, len);if (error)return error;/* * Free the range of realtime blocks. */ - error =xfs_rtfree_range (mp, tp, bno, len, &sumbp, &sb); + error =xfs_rtfree_range (&args, bno, len, &sumbp, &sb);if (error) {return error; }@@ -1015,6 +1016,10 @@ xfs_rtalloc_query_range(xfs_rtalloc_query_range_fn fn, void *priv) { + struct xfs_rtalloc_args args = { + .mount = mp, + .trans = tp, + };struct xfs_rtalloc_rec rec;xfs_rtblock_t rtstart;xfs_rtblock_t rtend;@@ -1034,13 +1039,13 @@ xfs_rtalloc_query_range( rtstart = low_rec->ar_startext;while (rtstart <= high_key) {/* Is the first block free? */ - error =xfs_rtcheck_range (mp, tp, rtstart,1 ,1 , &rtend, + error =xfs_rtcheck_range (&args, rtstart,1 ,1 , &rtend, &is_free);if (error)break ;/* How long does the extent go for? */ - error =xfs_rtfind_forw (mp, tp, rtstart, high_key, &rtend); + error =xfs_rtfind_forw (&args, rtstart, high_key, &rtend);if (error)break ;@@ -1085,11 +1090,15 @@ xfs_rtalloc_extent_is_free(xfs_extlen_t len, bool *is_free) { + struct xfs_rtalloc_args args = { + .mount = mp, + .trans = tp, + };xfs_rtblock_t end;int matches;int error; - error =xfs_rtcheck_range (mp, tp, start, len,1 , &end, &matches); + error =xfs_rtcheck_range (&args, start, len,1 , &end, &matches);if (error)return error;fs/xfs/scrub/rtsummary.c +5
-1@@ -177,6 +177,10 @@ STATIC intxchk_rtsum_compare ( struct xfs_scrub *sc) { +struct xfs_rtalloc_args args = { + .mount = sc->mp, + .trans = sc->tp, + };struct xfs_mount *mp = sc->mp;struct xfs_buf *bp;struct xfs_bmbt_irec map;@@ -205,7 +209,7 @@ xchk_rtsum_compare( }/* Read a block's worth of ondisk rtsummary file. */ - error =xfs_rtbuf_get (mp, sc->tp, off,1 , &bp); + error =xfs_rtbuf_get (&args, off,1 , &bp);if (!xchk_fblock_process_error (sc,XFS_DATA_FORK , off, &error))return error;fs/xfs/xfs_rtalloc.c +64
-55@@ -28,15 +28,14 @@*/ static int xfs_rtget_summary ( -xfs_mount_t *mp,/* file system mount structure */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,int log,/* log2 of extent size */ xfs_rtblock_t bbno,/* bitmap block number */ struct xfs_buf **rbpp,/* in/out: summary block buffer */ xfs_fsblock_t *rsb,/* in/out: summary block number */ xfs_suminfo_t *sum)/* out: summary info for this block */ { -return xfs_rtmodify_summary_int (mp, tp, log, bbno,0 , rbpp, rsb, sum); +return xfs_rtmodify_summary_int (args, log, bbno,0 , rbpp, rsb, sum); }/* @@ -45,8 +44,7 @@ xfs_rtget_summary(*/ STATIC int /* error */ xfs_rtany_summary( - xfs_mount_t *mp, /* file system mount structure */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, int low, /* low log2 extent size */ int high, /* high log2 extent size */ xfs_rtblock_t bbno, /* bitmap block number */@@ -54,6 +52,7 @@ xfs_rtany_summary( xfs_fsblock_t *rsb, /* in/out: summary block number */ int *stat) /* out: any good extents here? */ { +struct xfs_mount *mp = args->mount;int error;/* error value */ int log;/* loop counter, log2 of ext. size */ xfs_suminfo_t sum;/* summary data */ @@ -69,7 +68,7 @@ xfs_rtany_summary(/* * Get one summary datum. */ - error =xfs_rtget_summary (mp, tp, log, bbno, rbpp, rsb, &sum); + error =xfs_rtget_summary (args, log, bbno, rbpp, rsb, &sum);if (error) {return error; }@@ -99,9 +98,8 @@ xfs_rtany_summary(*/ STATIC int /* error */xfs_rtcopy_summary ( -xfs_mount_t *omp,/* old file system mount point */ -xfs_mount_t *nmp,/* new file system mount point */ -xfs_trans_t *tp)/* transaction pointer */ +struct xfs_rtalloc_args *oargs, +struct xfs_rtalloc_args *nargs) {xfs_rtblock_t bbno;/* bitmap block number */ struct xfs_buf *bp;/* summary buffer */ @@ -111,21 +109,21 @@ xfs_rtcopy_summary(xfs_fsblock_t sumbno;/* summary block number */ bp =NULL ; -for (log = omp->m_rsumlevels -1 ; log >=0 ; log--) { -for (bbno = omp->m_sb.sb_rbmblocks -1 ; +for (log = oargs->mount->m_rsumlevels -1 ; log >=0 ; log--) { +for (bbno = oargs->mount->m_sb.sb_rbmblocks -1 ; (xfs_srtblock_t )bbno >=0 ; bbno--) { - error =xfs_rtget_summary (omp, tp, log, bbno, &bp, + error =xfs_rtget_summary (oargs, log, bbno, &bp, &sumbno, &sum);if (error)return error;if (sum ==0 )continue ; - error =xfs_rtmodify_summary (omp, tp, log, bbno, -sum, + error =xfs_rtmodify_summary (oargs, log, bbno, -sum, &bp, &sumbno);if (error)return error; - error =xfs_rtmodify_summary (nmp, tp, log, bbno, sum, + error =xfs_rtmodify_summary (nargs, log, bbno, sum, &bp, &sumbno);if (error)return error;@@ -140,13 +138,13 @@ xfs_rtcopy_summary(*/ STATIC int /* error */xfs_rtallocate_range ( -xfs_mount_t *mp,/* file system mount point */ -xfs_trans_t *tp,/* transaction pointer */ +struct xfs_rtalloc_args *args,xfs_rtblock_t start,/* start block to allocate */ xfs_extlen_t len,/* length to allocate */ struct xfs_buf **rbpp,/* in/out: summary block buffer */ xfs_fsblock_t *rsb)/* in/out: summary block number */ { +struct xfs_mount *mp = args->mount;xfs_rtblock_t end;/* end of the allocated extent */ int error;/* error value */ xfs_rtblock_t postblock =0 ;/* first block allocated > end */ @@ -158,14 +156,14 @@ xfs_rtallocate_range(* We need to find the beginning and end of the extent so we can * properly update the summary. */ - error =xfs_rtfind_back (mp, tp, start,0 , &preblock); + error =xfs_rtfind_back (args, start,0 , &preblock);if (error) {return error; }/* * Find the next allocated block (end of free extent). */ - error =xfs_rtfind_forw (mp, tp, end, mp->m_sb.sb_rextents -1 , + error =xfs_rtfind_forw (args, end, mp->m_sb.sb_rextents -1 , &postblock);if (error) {return error;@@ -174,7 +172,7 @@ xfs_rtallocate_range(* Decrement the summary information corresponding to the entire * (old) free extent. */ - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (postblock +1 - preblock),XFS_BITTOBLOCK (mp, preblock),-1 , rbpp, rsb);if (error) {@@ -185,7 +183,7 @@ xfs_rtallocate_range(* old extent, add summary data for them to be free. */ if (preblock < start) { - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (start - preblock),XFS_BITTOBLOCK (mp, preblock),1 , rbpp, rsb);if (error) {@@ -197,7 +195,7 @@ xfs_rtallocate_range(* old extent, add summary data for them to be free. */ if (postblock > end) { - error =xfs_rtmodify_summary (mp, tp, + error =xfs_rtmodify_summary (args,XFS_RTBLOCKLOG (postblock - end),XFS_BITTOBLOCK (mp, end +1 ),1 , rbpp, rsb);if (error) {@@ -207,7 +205,7 @@ xfs_rtallocate_range(/* * Modify the bitmap to mark this extent allocated. */ - error =xfs_rtmodify_range (mp, tp, start, len,0 ); + error =xfs_rtmodify_range (args, start, len,0 );return error; }@@ -219,8 +217,7 @@ xfs_rtallocate_range(*/ STATIC int /* error */ xfs_rtallocate_extent_block( - xfs_mount_t *mp, /* file system mount point */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, xfs_rtblock_t bbno, /* bitmap block number */ xfs_extlen_t minlen, /* minimum length to allocate */ xfs_extlen_t maxlen, /* maximum length to allocate */@@ -231,6 +228,7 @@ xfs_rtallocate_extent_block(xfs_extlen_t prod,/* extent product factor */ xfs_rtblock_t *rtblock) /* out: start block allocated */ { + struct xfs_mount *mp = args->mount;xfs_rtblock_t besti;/* best rtblock found so far */ xfs_rtblock_t bestlen;/* best length found so far */ xfs_rtblock_t end;/* last rtblock in chunk */ @@ -254,7 +252,7 @@ xfs_rtallocate_extent_block(* See if there's a free extent of maxlen starting at i. * If it's not so then next will contain the first non-free. */ - error =xfs_rtcheck_range (mp, tp, i, maxlen,1 , &next, &stat); + error =xfs_rtcheck_range (args, i, maxlen,1 , &next, &stat);if (error) {return error; }@@ -262,7 +260,7 @@ xfs_rtallocate_extent_block(/* * i for maxlen is all free, allocate and return that. */ - error =xfs_rtallocate_range (mp, tp, i, maxlen, rbpp, + error =xfs_rtallocate_range (args, i, maxlen, rbpp, rsb);if (error) {return error;@@ -290,7 +288,7 @@ xfs_rtallocate_extent_block(* If not done yet, find the start of the next free space. */ if (next < end) { - error =xfs_rtfind_forw (mp, tp, next, end, &i); + error =xfs_rtfind_forw (args, next, end, &i);if (error) {return error; }@@ -315,7 +313,7 @@ xfs_rtallocate_extent_block(/* * Allocate besti for bestlen & return that. */ - error =xfs_rtallocate_range (mp, tp, besti, bestlen, rbpp, rsb); + error =xfs_rtallocate_range (args, besti, bestlen, rbpp, rsb);if (error) {return error; }@@ -339,8 +337,7 @@ xfs_rtallocate_extent_block(*/ STATIC int /* error */ xfs_rtallocate_extent_exact( - xfs_mount_t *mp, /* file system mount point */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, xfs_rtblock_t bno, /* starting block number to allocate */ xfs_extlen_t minlen, /* minimum length to allocate */ xfs_extlen_t maxlen, /* maximum length to allocate */@@ -359,7 +356,7 @@ xfs_rtallocate_extent_exact(/* * Check if the range in question (for maxlen) is free. */ - error =xfs_rtcheck_range (mp, tp, bno, maxlen,1 , &next, &isfree); + error =xfs_rtcheck_range (args, bno, maxlen,1 , &next, &isfree);if (error) {return error; }@@ -367,7 +364,7 @@ xfs_rtallocate_extent_exact(/* * If it is, allocate it and return success. */ - error =xfs_rtallocate_range (mp, tp, bno, maxlen, rbpp, rsb); + error =xfs_rtallocate_range (args, bno, maxlen, rbpp, rsb);if (error) {return error; }@@ -402,7 +399,7 @@ xfs_rtallocate_extent_exact(/* * Allocate what we can and return it. */ - error =xfs_rtallocate_range (mp, tp, bno, maxlen, rbpp, rsb); + error =xfs_rtallocate_range (args, bno, maxlen, rbpp, rsb);if (error) {return error; }@@ -418,8 +415,7 @@ xfs_rtallocate_extent_exact(*/ STATIC int /* error */ xfs_rtallocate_extent_near( - xfs_mount_t *mp, /* file system mount point */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, xfs_rtblock_t bno, /* starting block number to allocate */ xfs_extlen_t minlen, /* minimum length to allocate */ xfs_extlen_t maxlen, /* maximum length to allocate */@@ -429,6 +425,7 @@ xfs_rtallocate_extent_near(xfs_extlen_t prod,/* extent product factor */ xfs_rtblock_t *rtblock) /* out: start block allocated */ { + struct xfs_mount *mp = args->mount;int any;/* any useful extents from summary */ xfs_rtblock_t bbno;/* bitmap block number */ int error;/* error value */ @@ -456,7 +453,7 @@ xfs_rtallocate_extent_near(/* * Try the exact allocation first. */ - error =xfs_rtallocate_extent_exact (mp, tp, bno, minlen, maxlen, len, + error =xfs_rtallocate_extent_exact (args, bno, minlen, maxlen, len, rbpp, rsb, prod, &r);if (error) {return error;@@ -480,7 +477,7 @@ xfs_rtallocate_extent_near(* Get summary information of extents of all useful levels * starting in this bitmap block. */ - error =xfs_rtany_summary (mp, tp, log2len, mp->m_rsumlevels -1 , + error =xfs_rtany_summary (args, log2len, mp->m_rsumlevels -1 , bbno + i, rbpp, rsb, &any);if (error) {return error;@@ -498,7 +495,7 @@ xfs_rtallocate_extent_near(* Try to allocate an extent starting in * this block. */ - error =xfs_rtallocate_extent_block (mp, tp, + error =xfs_rtallocate_extent_block (args, bbno + i, minlen, maxlen, len, &n, rbpp, rsb, prod, &r);if (error) {@@ -527,7 +524,7 @@ xfs_rtallocate_extent_near(* Grab the summary information for * this bitmap block. */ - error =xfs_rtany_summary (mp, tp, + error =xfs_rtany_summary (args, log2len, mp->m_rsumlevels -1 , bbno + j, rbpp, rsb, &any);if (error) {@@ -543,8 +540,8 @@ xfs_rtallocate_extent_near(*/ if (any)continue ; - error =xfs_rtallocate_extent_block (mp, - tp, bbno + j, minlen, maxlen, + error =xfs_rtallocate_extent_block (args, + bbno + j, minlen, maxlen, len, &n, rbpp, rsb, prod, &r);if (error) {return error;@@ -565,7 +562,7 @@ xfs_rtallocate_extent_near(* Try to allocate from the summary block * that we found. */ - error =xfs_rtallocate_extent_block (mp, tp, + error =xfs_rtallocate_extent_block (args, bbno + i, minlen, maxlen, len, &n, rbpp, rsb, prod, &r);if (error) {@@ -619,10 +616,9 @@ xfs_rtallocate_extent_near(* specified. If we don't get maxlen then use prod to trim * the length, if given. The lengths are all in rtextents. */ -STATIC int /* error */ +static int /* error */ xfs_rtallocate_extent_size( - xfs_mount_t *mp, /* file system mount point */ - xfs_trans_t *tp, /* transaction pointer */ + struct xfs_rtalloc_args *args, xfs_extlen_t minlen, /* minimum length to allocate */ xfs_extlen_t maxlen, /* maximum length to allocate */ xfs_extlen_t *len, /* out: actual length allocated */@@ -631,6 +627,7 @@ xfs_rtallocate_extent_size(xfs_extlen_t prod,/* extent product factor */ xfs_rtblock_t *rtblock) /* out: start block allocated */ { + struct xfs_mount *mp = args->mount;int error;/* error value */ int i;/* bitmap block number */ int l;/* level number (loop control) */ @@ -656,7 +653,7 @@ xfs_rtallocate_extent_size(/* * Get the summary for this level/block. */ - error =xfs_rtget_summary (mp, tp, l, i, rbpp, rsb, + error =xfs_rtget_summary (args, l, i, rbpp, rsb, &sum);if (error) {return error;@@ -669,7 +666,7 @@ xfs_rtallocate_extent_size(/* * Try allocating the extent. */ - error =xfs_rtallocate_extent_block (mp, tp, i, maxlen, + error =xfs_rtallocate_extent_block (args, i, maxlen, maxlen, len, &n, rbpp, rsb, prod, &r);if (error) {return error;@@ -715,7 +712,7 @@ xfs_rtallocate_extent_size(/* * Get the summary information for this level/block. */ - error =xfs_rtget_summary (mp, tp, l, i, rbpp, rsb, + error =xfs_rtget_summary (args, l, i, rbpp, rsb, &sum);if (error) {return error;@@ -730,7 +727,7 @@ xfs_rtallocate_extent_size(* minlen/maxlen are in the possible range for * this summary level. */ - error =xfs_rtallocate_extent_block (mp, tp, i, + error =xfs_rtallocate_extent_block (args, i,XFS_RTMAX (minlen,1 << l),XFS_RTMIN (maxlen, (1 << (l +1 )) -1 ), len, &n, rbpp, rsb, prod, &r);@@ -1023,6 +1020,12 @@ xfs_growfs_rt( ((sbp->sb_rextents & ((1 << mp->m_blkbit_log) -1 )) !=0 ); bmbno < nrbmblocks; bmbno++) { +struct xfs_rtalloc_args args = { + .mount = mp, + }; +struct xfs_rtalloc_args nargs = { + .mount = nmp, + };struct xfs_trans *tp;xfs_rfsblock_t nrblocks_step;@@ -1053,6 +1056,9 @@ xfs_growfs_rt( &tp);if (error)break ; + args.trans = tp; + nargs.trans = tp; +/* * Lock out other callers by grabbing the bitmap inode lock. */ @@ -1086,7 +1092,7 @@ xfs_growfs_rt(*/ if (sbp->sb_rbmblocks != nsbp->sb_rbmblocks || mp->m_rsumlevels != nmp->m_rsumlevels) { - error =xfs_rtcopy_summary (mp, nmp, tp); + error =xfs_rtcopy_summary (&args, &nargs);if (error) goto error_cancel; }@@ -1112,7 +1118,7 @@ xfs_growfs_rt(* Free new extent. */ bp =NULL ; - error =xfs_rtfree_range (nmp, tp, sbp->sb_rextents, + error =xfs_rtfree_range (&nargs, sbp->sb_rextents, nsbp->sb_rextents - sbp->sb_rextents, &bp, &sumbno);if (error) { error_cancel:@@ -1182,13 +1188,16 @@ xfs_rtallocate_extent(xfs_extlen_t prod,/* extent product factor */ xfs_rtblock_t *rtblock) /* out: start block allocated */ { - xfs_mount_t *mp = tp->t_mountp; + struct xfs_rtalloc_args args = { + .mount = tp->t_mountp, + .trans = tp, + };int error;/* error value */ xfs_rtblock_t r;/* result allocated block */ xfs_fsblock_t sb;/* summary file block number */ struct xfs_buf *sumbp;/* summary file block buffer */ -ASSERT (xfs_isilocked (mp->m_rbmip,XFS_ILOCK_EXCL )); +ASSERT (xfs_isilocked (args.mount->m_rbmip,XFS_ILOCK_EXCL ));ASSERT (minlen >0 && minlen <= maxlen);/* @@ -1210,10 +1219,10 @@ xfs_rtallocate_extent( retry: sumbp =NULL ;if (bno ==0 ) { - error =xfs_rtallocate_extent_size (mp, tp, minlen, maxlen, len, + error =xfs_rtallocate_extent_size (&args, minlen, maxlen, len, &sumbp, &sb, prod, &r); }else { - error =xfs_rtallocate_extent_near (mp, tp, bno, minlen, maxlen, + error =xfs_rtallocate_extent_near (&args, bno, minlen, maxlen, len, &sumbp, &sb, prod, &r); }fs/xfs/xfs_rtalloc.h +14
-8@@ -11,6 +11,11 @@struct xfs_mount ;struct xfs_trans ; +struct xfs_rtalloc_args { +struct xfs_mount *mount; +struct xfs_trans *trans; +}; +/* * XXX: Most of the realtime allocation functions deal in units of realtime * extents, not realtime blocks. This looks funny when paired with the type @@ -101,29 +106,30 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ -int xfs_rtbuf_get (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtbuf_get (struct xfs_rtalloc_args *args,xfs_rtblock_t block,int issum,struct xfs_buf **bpp); -int xfs_rtcheck_range (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtcheck_range (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_extlen_t len,int val,xfs_rtblock_t *new,int *stat); -int xfs_rtfind_back (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtfind_back (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_rtblock_t limit,xfs_rtblock_t *rtblock); -int xfs_rtfind_forw (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtfind_forw (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_rtblock_t limit,xfs_rtblock_t *rtblock); -int xfs_rtmodify_range (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtmodify_range (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_extlen_t len,int val); -int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtmodify_summary_int (struct xfs_rtalloc_args *args,int log,xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp,xfs_fsblock_t *rsb,xfs_suminfo_t *sum); -int xfs_rtmodify_summary (struct xfs_mount *mp,struct xfs_trans *tp,int log, +int xfs_rtmodify_summary (struct xfs_rtalloc_args *args,int log,xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp,xfs_fsblock_t *rsb); -int xfs_rtfree_range (struct xfs_mount *mp,struct xfs_trans *tp, +int xfs_rtfree_range (struct xfs_rtalloc_args *args,xfs_rtblock_t start,xfs_extlen_t len,struct xfs_buf **rbpp,xfs_fsblock_t *rsb); + int xfs_rtalloc_query_range(struct xfs_mount *mp, struct xfs_trans *tp, const struct xfs_rtalloc_rec *low_rec, const struct xfs_rtalloc_rec *high_rec,
Re: [PATCH v2 1/6] xfs: cache last bitmap block in realtime allocator
Quoted 58 lines · Darrick J. Wong
Quoted 40 lines · Dave Chinner
Quoted 27 lines · Omar Sandoval
From: Omar Sandoval <[email protected]>
Profiling a workload on a highly fragmented realtime device showed a ton of CPU cycles being spent in
xfs_trans_read_buf()called byxfs_rtbuf_get(). Further tracing showed that much of that was repeated calls toxfs_rtbuf_get()for the same block of the realtime bitmap. These come fromxfs_rtallocate_extent_block(): as it walks through ranges of free bits in the bitmap, each call toxfs_rtcheck_range()and xfs_rtfind_{forw,back}() gets the same bitmap block. If the bitmap block is very fragmented, then this is _alot_of buffer lookups.The realtime allocator already passes around a cache of the last used realtime summary block to avoid repeated reads (the parameters rbpp and rsb). We can do the same for the realtime bitmap.
This replaces rbpp and rsb with a struct
xfs_rtbuf_cache, which caches the most recently used block for both the realtime bitmap and summary.xfs_rtbuf_get()now handles the caching instead of the callers, which requires plumbingxfs_rtbuf_cacheto more functions but also makes sure we don't miss anything.Signed-off-by Omar Sandoval <[email protected]>--- fs/xfs/libxfs/xfs_rtbitmap.c | 179 ++++++++++++++++++----------------- fs/xfs/xfs_rtalloc.c | 119 +++++++++++------------ fs/xfs/xfs_rtalloc.h | 30 ++++-- 3 files changed, 170 insertions(+), 158 deletions(-)....
Quoted 9 lines
fs/xfs/xfs_rtalloc.h +1
-0@@ -101,29 +101,40 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ +struct xfs_rtbuf_cache {Quoted 8 lines
fs/xfs/xfs_rtalloc.h xfs_growfs_rt( · line 108
Quoted 4 lines
fs/xfs/xfs_rtalloc.h +4
-0+struct xfs_buf *bbuf;/* bitmap block buffer */ +xfs_fileoff_t bblock;/* bitmap block number */ +struct xfs_buf *sbuf;/* summary block buffer */ +xfs_fileoff_t sblock;/* summary block number */ I don't think the block numbers are file offsets? Most of the code passes the bitmap and summary block numbers around as a
xfs_fsblock_t...They're fed into
xfs_bmapi_readas thexfs_fileoff_tparameter.I have a whole series cleaning up all of the units abuse in the rt code part of the realtime modernization patchdeluge:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=clean-up-realtime-unitsHere's the specific patch that cleans up this one part:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/commit/?h=realtime-groups&id=b344bd4bd655576f8bda193b0e33f471a6295b05
....
Quoted 2 lines
Could someone (Omar?) take a look at my other rt cleanups too? I'd very much like to get them merged out of my dev tree.
Can you post the series to the list? That makes it much easier to comment on them....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 1/6] xfs: cache last bitmap block in realtime allocator
Quoted 59 lines · Dave Chinner
Quoted 50 lines · Darrick J. Wong
Quoted 40 lines · Dave Chinner
Quoted 27 lines · Omar Sandoval
From: Omar Sandoval <[email protected]>
Profiling a workload on a highly fragmented realtime device showed a ton of CPU cycles being spent in
xfs_trans_read_buf()called byxfs_rtbuf_get(). Further tracing showed that much of that was repeated calls toxfs_rtbuf_get()for the same block of the realtime bitmap. These come fromxfs_rtallocate_extent_block(): as it walks through ranges of free bits in the bitmap, each call toxfs_rtcheck_range()and xfs_rtfind_{forw,back}() gets the same bitmap block. If the bitmap block is very fragmented, then this is _alot_of buffer lookups.The realtime allocator already passes around a cache of the last used realtime summary block to avoid repeated reads (the parameters rbpp and rsb). We can do the same for the realtime bitmap.
This replaces rbpp and rsb with a struct
xfs_rtbuf_cache, which caches the most recently used block for both the realtime bitmap and summary.xfs_rtbuf_get()now handles the caching instead of the callers, which requires plumbingxfs_rtbuf_cacheto more functions but also makes sure we don't miss anything.Signed-off-by Omar Sandoval <[email protected]>--- fs/xfs/libxfs/xfs_rtbitmap.c | 179 ++++++++++++++++++----------------- fs/xfs/xfs_rtalloc.c | 119 +++++++++++------------ fs/xfs/xfs_rtalloc.h | 30 ++++-- 3 files changed, 170 insertions(+), 158 deletions(-)....
Quoted 9 lines
fs/xfs/xfs_rtalloc.h +1
-0@@ -101,29 +101,40 @@ xfs_growfs_rt(/* * From xfs_rtbitmap.c */ +struct xfs_rtbuf_cache {Quoted 8 lines
fs/xfs/xfs_rtalloc.h xfs_growfs_rt( · line 108
Quoted 4 lines
fs/xfs/xfs_rtalloc.h +4
-0+struct xfs_buf *bbuf;/* bitmap block buffer */ +xfs_fileoff_t bblock;/* bitmap block number */ +struct xfs_buf *sbuf;/* summary block buffer */ +xfs_fileoff_t sblock;/* summary block number */ I don't think the block numbers are file offsets? Most of the code passes the bitmap and summary block numbers around as a
xfs_fsblock_t...Quoted 8 lines
They're fed into
xfs_bmapi_readas thexfs_fileoff_tparameter.I have a whole series cleaning up all of the units abuse in the rt code part of the realtime modernization patchdeluge:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=clean-up-realtime-unitsHere's the specific patch that cleans up this one part:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/commit/?h=realtime-groups&id=b344bd4bd655576f8bda193b0e33f471a6295b05
Quoted 7 lines
....
Quoted 2 lines
Could someone (Omar?) take a look at my other rt cleanups too? I'd very much like to get them merged out of my dev tree.
Can you post the series to the list? That makes it much easier to comment on them....
https://lore.kernel.org/linux-xfs/167243865605.709511.15650588946095003543.stgit@magnolia/
--D
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 1/6] xfs: cache last bitmap block in realtime allocator
In reply to Omar SandovalNotwithstanding the whole RT code type confusion, the change here looks good to me:
PATCH v2 2/6 xfs: invert the realtime summary cache
In reply to Omar SandovalFrom: Omar Sandoval <[email protected]>
In commit 355e3532132b ("xfs: cache minimum realtime summary level"), I added a cache of the minimum level of the realtime summary that has any free extents. However, it turns out that the _maximum_ level is more useful for upcoming optimizations, and basically equivalent for the existing usage. So, let's change the meaning of the cache to be the maximum level + 1, or 0 if there are no free extents.
For example, if the cache contains:
{0, 4}
then there are no free extents starting in realtime bitmap block 0, and there are no free extents larger than or equal to 2^4 blocks starting in realtime bitmap block 1. The cache is a loose upper bound, so there may or may not be free extents smaller than 2^4 blocks in realtime bitmap block 1.
--- fs/xfs/libxfs/xfs_rtbitmap.c | 6 +++--- fs/xfs/xfs_mount.h | 6 +++--- fs/xfs/xfs_rtalloc.c | 31 +++++++++++++++++++------------ 3 files changed, 25 insertions(+), 18 deletions(-)
fs/xfs/libxfs/xfs_rtbitmap.c +3 -3
@@ -503,10 +503,10 @@ xfs_rtmodify_summary_int( *sp += delta;if (mp->m_rsum_cache) { -if (*sp ==0 && log == mp->m_rsum_cache[bbno]) - mp->m_rsum_cache[bbno]++; -if (*sp !=0 && log < mp->m_rsum_cache[bbno]) +if (*sp ==0 && log +1 == mp->m_rsum_cache[bbno]) mp->m_rsum_cache[bbno] = log; +if (*sp !=0 && log >= mp->m_rsum_cache[bbno]) + mp->m_rsum_cache[bbno] = log +1 ; }xfs_trans_log_buf (tp, bp, first, first +sizeof (*sp) -1 ); }
fs/xfs/xfs_mount.h +3 -3
@@ -103,9 +103,9 @@ typedef struct xfs_mount {/* * Optional cache of rt summary level per bitmap block with the -* invariant that m_rsum_cache[bbno] <= the minimum i for which -* rsum[i][bbno] != 0. Reads and writes are serialized by the rsumip -* inode lock. +* invariant that m_rsum_cache[bbno] > the maximum i for which +* rsum[i][bbno] != 0, or 0 if rsum[i][bbno] == 0 for all i. +* Reads and writes are serialized by the rsumip inode lock. */ uint8_t *m_rsum_cache;struct xfs_mru_cache *m_filestream;/* per-mount filestream data */
fs/xfs/xfs_rtalloc.c +19 -12
@@ -56,14 +56,19 @@ xfs_rtany_summary(int log;/* loop counter, log2 of ext. size */ xfs_suminfo_t sum;/* summary data */ -/* There are no extents at levels < m_rsum_cache[bbno]. */ -if (mp->m_rsum_cache && low < mp->m_rsum_cache[bbno]) - low = mp->m_rsum_cache[bbno]; +/* There are no extents at levels >= m_rsum_cache[bbno]. */ +if (mp->m_rsum_cache) { + high =min (high, mp->m_rsum_cache[bbno] -1 ); +if (low > high) { + *stat =0 ; +return 0 ; + } + }/* * Loop over logs of extent sizes. */ -for (log = low; log <= high; log++) { +for (log = high; log >= low; log--) {/* * Get one summary datum. */
@@ -84,9 +89,9 @@ xfs_rtany_summary(*/ *stat =0 ; out: -/* There were no extents at levels < log. */ -if (mp->m_rsum_cache && log > mp->m_rsum_cache[bbno]) - mp->m_rsum_cache[bbno] = log; +/* There were no extents at levels > log. */ +if (mp->m_rsum_cache && log +1 < mp->m_rsum_cache[bbno]) + mp->m_rsum_cache[bbno] = log +1 ;return 0 ; }
@@ -881,12 +886,14 @@ xfs_alloc_rsum_cache( xfs_extlen_t rbmblocks) /* number of rt bitmap blocks */ {/* -* The rsum cache is initialized to all zeroes, which is trivially a -* lower bound on the minimum level with any free extents. We can -* continue without the cache if it couldn't be allocated. +* The rsum cache is initialized to the maximum value, which is +* trivially an upper bound on the maximum level with any free extents. +* We can continue without the cache if it couldn't be allocated. */ - mp->m_rsum_cache =kvzalloc (rbmblocks,GFP_KERNEL ); -if (!mp->m_rsum_cache) + mp->m_rsum_cache =kvmalloc (rbmblocks,GFP_KERNEL ); +if (mp->m_rsum_cache) +memset (mp->m_rsum_cache,-1 , rbmblocks); +else xfs_warn (mp,"could not allocate realtime summary cache" ); }
Signature · 1 line
2.41.0
Re: [PATCH v2 2/6] xfs: invert the realtime summary cache
Looks good:
PATCH v2 3/6 xfs: return maximum free size from xfs_rtany_summary()
In reply to Omar SandovalFrom: Omar Sandoval <[email protected]>
Instead of only returning whether there is any free space, return the maximum size, which is fast thanks to the previous commit. This will be used by two upcoming optimizations.
--- fs/xfs/xfs_rtalloc.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
fs/xfs/xfs_rtalloc.c +9 -9
@@ -50,7 +50,7 @@ xfs_rtany_summary(int high,/* high log2 extent size */ xfs_rtblock_t bbno, /* bitmap block number */ struct xfs_rtbuf_cache *rtbufc, /* in/out: cache of realtime blocks */ - int *stat) /* out: any good extents here? */ + int *maxlog) /* out: maximum log2 extent size free */ { int error; /* error value */int log;/* loop counter, log2 of ext. size */
@@ -60,7 +60,7 @@ xfs_rtany_summary(if (mp->m_rsum_cache) { high =min (high, mp->m_rsum_cache[bbno] -1 );if (low > high) { - *stat =0 ; + *maxlog =-1 ;return 0 ; } }
@@ -80,14 +80,14 @@ xfs_rtany_summary(* If there are any, return success. */ if (sum) { - *stat =1 ; + *maxlog = log; goto out; } }/* * Found nothing, return failure. */ - *stat =0 ; + *maxlog =-1 ; out:/* There were no extents at levels > log. */ if (mp->m_rsum_cache && log +1 < mp->m_rsum_cache[bbno])
@@ -430,7 +430,7 @@ xfs_rtallocate_extent_near(xfs_extlen_t prod,/* extent product factor */ xfs_rtblock_t *rtblock) /* out: start block allocated */ { - int any; /* any useful extents from summary */ + int maxlog; /* maximum useful extent from summary */xfs_rtblock_t bbno;/* bitmap block number */ int error;/* error value */ int i;/* bitmap block offset (loop control) */
@@ -482,7 +482,7 @@ xfs_rtallocate_extent_near(* starting in this bitmap block. */ error =xfs_rtany_summary (mp, tp, log2len, mp->m_rsumlevels -1 , - bbno + i, rtbufc, &any); + bbno + i, rtbufc, &maxlog);if (error) {return error; }
@@ -490,7 +490,7 @@ xfs_rtallocate_extent_near(* If there are any useful extents starting here, try * allocating one. */ -if (any) { +if (maxlog >=0 ) {/* * On the positive side of the starting location. */
@@ -530,7 +530,7 @@ xfs_rtallocate_extent_near(*/ error =xfs_rtany_summary (mp, tp, log2len, mp->m_rsumlevels -1 , - bbno + j, rtbufc, &any); + bbno + j, rtbufc, &maxlog);if (error) {return error; }
@@ -542,7 +542,7 @@ xfs_rtallocate_extent_near(* extent given, we've already tried * that allocation, don't do it again. */ -if (any) +if (maxlog >=0 )continue ; error = xfs_rtallocate_extent_block(mp, tp, bbno + j, minlen, maxlen,
Signature · 1 line
2.41.0
Re: [PATCH v2 3/6] xfs: return maximum free size from xfs_rtany_summary()
Looks good:
PATCH v2 4/6 xfs: limit maxlen based on available space in xfs_rtallocate_extent_near()
In reply to Omar SandovalFrom: Omar Sandoval <[email protected]>
xfs_rtallocate_extent_near() calls xfs_rtallocate_extent_block() with the minlen and maxlen that were passed to it. xfs_rtallocate_extent_block() then scans the bitmap block looking for a free range of size maxlen. If there is none, it has to scan the whole bitmap block before returning the largest range of at least size minlen. For a fragmented realtime device and a large allocation request, it's almost certain that this will have to search the whole bitmap block, leading to high CPU usage.
However, the realtime summary tells us the maximum size available in the bitmap block. We can limit the search in xfs_rtallocate_extent_block() to that size and often stop before scanning the whole bitmap block.
--- fs/xfs/xfs_rtalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/xfs_rtalloc.c +6 -3
@@ -491,6 +491,9 @@ xfs_rtallocate_extent_near(* allocating one. */ if (maxlog >=0 ) { +xfs_extlen_t maxavail = +min_t (xfs_rtblock_t, maxlen, + (1ULL << (maxlog +1 )) -1 );/* * On the positive side of the starting location. */
@@ -500,7 +503,7 @@ xfs_rtallocate_extent_near(* this block. */ error =xfs_rtallocate_extent_block (mp, tp, - bbno + i, minlen, maxlen, len, &n, + bbno + i, minlen, maxavail, len, &n, rtbufc, prod, &r);if (error) {return error;
@@ -545,7 +548,7 @@ xfs_rtallocate_extent_near(if (maxlog >=0 )continue ; error =xfs_rtallocate_extent_block (mp, - tp, bbno + j, minlen, maxlen, + tp, bbno + j, minlen, maxavail, len, &n, rtbufc, prod, &r);if (error) {return error;
@@ -567,7 +570,7 @@ xfs_rtallocate_extent_near(* that we found. */ error =xfs_rtallocate_extent_block (mp, tp, - bbno + i, minlen, maxlen, len, &n, + bbno + i, minlen, maxavail, len, &n, rtbufc, prod, &r);if (error) {return error;
Signature · 1 line
2.41.0
Re: [PATCH v2 4/6] xfs: limit maxlen based on available space in xfs_rtallocate_extent_near()
Looks good:
PATCH v2 5/6 xfs: don't try redundant allocations in xfs_rtallocate_extent_near()
In reply to Omar SandovalFrom: Omar Sandoval <[email protected]>
xfs_rtallocate_extent_near() tries to find a free extent as close to a target bitmap block given by bbno as possible, which may be before or after bbno. Searching backwards has a complication: the realtime summary accounts for free space _starting_ in a bitmap block, but not straddling or ending in a bitmap block. So, when the negative search finds a free extent in the realtime summary, in order to end up closer to the target, it looks for the end of the free extent. For example, if bbno - 2 has a free extent, then it will check bbno - 1, then bbno - 2. But then if bbno - 3 has a free extent, it will check bbno - 1 again, then bbno - 2 again, and then bbno - 3. This results in a quadratic loop, which is completely pointless since the repeated checks won't find anything new.
Fix it by remembering where we last checked up to and continue from there. This also obviates the need for a check of the realtime summary.
--- fs/xfs/xfs_rtalloc.c | 50 +++++--------------------------------------- 1 file changed, 5 insertions(+), 45 deletions(-)
fs/xfs/xfs_rtalloc.c +5 -45
@@ -471,6 +471,7 @@ xfs_rtallocate_extent_near( } bbno =XFS_BITTOBLOCK (mp, bno); i =0 ; + j =-1 ;ASSERT (minlen !=0 ); log2len =xfs_highbit32 (minlen);/*
@@ -522,31 +523,11 @@ xfs_rtallocate_extent_near(else {/* i < 0 */ /* * Loop backwards through the bitmap blocks from -* the starting point-1 up to where we are now. -* There should be an extent which ends in this -* bitmap block and is long enough. +* where we last checked down to where we are +* now. There should be an extent which ends in +* this bitmap block and is long enough. */ -for (j =-1 ; j > i; j--) { -/* -* Grab the summary information for -* this bitmap block. -*/ - error =xfs_rtany_summary (mp, tp, - log2len, mp->m_rsumlevels -1 , - bbno + j, rtbufc, &maxlog); -if (error) { -return error; - } -/* -* If there's no extent given in the -* summary that means the extent we -* found must carry over from an -* earlier block. If there is an -* extent given, we've already tried -* that allocation, don't do it again. -*/ -if (maxlog >=0 ) -continue ; +for (; j >= i; j--) { error =xfs_rtallocate_extent_block (mp, tp, bbno + j, minlen, maxavail, len, &n, rtbufc, prod, &r);
@@ -561,27 +542,6 @@ xfs_rtallocate_extent_near(return 0 ; } } -/* -* There weren't intervening bitmap blocks -* with a long enough extent, or the -* allocation didn't work for some reason -* (i.e. it's a little * too short). -* Try to allocate from the summary block -* that we found. -*/ - error =xfs_rtallocate_extent_block (mp, tp, - bbno + i, minlen, maxavail, len, &n, - rtbufc, prod, &r); -if (error) { -return error; - } -/* -* If it works, return the extent. -*/ -if (r !=NULLRTBLOCK ) { - *rtblock = r; -return 0 ; - } } }/*
Signature · 1 line
2.41.0
Re: [PATCH v2 5/6] xfs: don't try redundant allocations in xfs_rtallocate_extent_near()
Looks good:
PATCH v2 6/6 xfs: don't look for end of extent further than necessary in xfs_rtallocate_extent_near()
In reply to Omar SandovalFrom: Omar Sandoval <[email protected]>
As explained in the previous commit, xfs_rtallocate_extent_near() looks for the end of a free extent when searching backwards from the target bitmap block. Since the previous commit, it searches from the last bitmap block it checked to the bitmap block containing the start of the extent.
This may still be more than necessary, since the free extent may not be that long. We know the maximum size of the free extent from the realtime summary. Use that to compute how many bitmap blocks we actually need to check.
--- fs/xfs/xfs_rtalloc.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
fs/xfs/xfs_rtalloc.c +21 -4
@@ -521,12 +521,29 @@ xfs_rtallocate_extent_near(* On the negative side of the starting location. */ else { /* i < 0 */ +int maxblocks; +/* -* Loop backwards through the bitmap blocks from -* where we last checked down to where we are -* now. There should be an extent which ends in -* this bitmap block and is long enough. +* Loop backwards to find the end of the extent +* we found in the realtime summary. +* +* maxblocks is the maximum possible number of +* bitmap blocks from the start of the extent to +* the end of the extent. */ +if (maxlog ==0 ) + maxblocks =0 ; +else if (maxlog < mp->m_blkbit_log) + maxblocks =1 ; +else + maxblocks =2 << (maxlog - mp->m_blkbit_log); +/* +* We need to check bbno + i + maxblocks down to +* bbno + i. We already checked bbno down to +* bbno + j + 1, so we don't need to check those +* again. +*/ + j =min (i + maxblocks, j);for (; j >= i; j--) { error = xfs_rtallocate_extent_block(mp, tp, bbno + j, minlen, maxavail,
Signature · 1 line
2.41.0
Re: [PATCH v2 6/6] xfs: don't look for end of extent further than necessary in xfs_rtallocate_extent_near()
fs/xfs/xfs_rtalloc.c xfs_rtallocate_extent_near( · line 539
Quoted 6 lines · Omar Sandoval
fs/xfs/xfs_rtalloc.c +6
-0+if (maxlog ==0 ) + maxblocks =0 ; +else if (maxlog < mp->m_blkbit_log) + maxblocks =1 ; +else + maxblocks =2 << (maxlog - mp->m_blkbit_log);
Overly long line here (can be fixed when applying I guess).
Otherwise looks good: