xfsprogs: xfs_buf unification and AIO
cover xfsprogs: xfs_buf unification and AIO
Hi folks,
Big rework. Aimed at unifying the user and kernel buffer caches so the first thing to do is port the kernel buffer cache code and hack it to pieces to use userspace AIO.
Overview:
This patchset introduces the kernel buffer caching mechanism to userspace libxfs. THe core of this is per-AG indexing of the buffer cache along with global scope LRU based reclaiming. Like the kernel, it supports both cached and uncached buffers, and the implementation of these APIs are identical between userspace and kernel space.
To allow the buffer cache implementation to be shared between kernel and userspace, it needs to be split into two parts. The first is the domain specific "target" abstraction that implements all the kernel or userspace specific functions (e.g. memory allocation, the per-AG cache indexing, the IO engine, etc). The second part is the shared buffer cache and buffer IO API implementation.
TO use the kernel code, however, we need a bunch of infrastructure userspace doesn't currently provide. These are atomic variables, locking primitives, completions, memory barriers, LRU lists, memory reclaim triggers, and so on.
The simplest, most portable choice for atomic variables and memory barriers is to use the Userspace RCU library, which is provides many of the same semantics and interfaces we use within the kernel for atomic, memory barriers and RCU algorithms. Hence xfsprogs grows a hard dependency on liburcu6, but this is packaged on all major distros these days, so that's not a huge problem.
Locking APIs are fleshed out simply by using pthread locks. Spin locks are not a reliable solution in userspace, so all the kernel locking mechanisms wrap sleeping locks. We can even use a pthread_mutex_t as a kernel semaphore replacement because, by default, pthread_mutex_unlock() does not check the owner matches the unlocking and hence, unlike kernel mutexes, can be used safely in non-owner release situations.
For reclaiming the buffer cache when memory usage goes too high, we make use of the kernel Pressure Stall Information (PSI) monitoring API via /proc/pressure/memory. THis was introduced in kernel v5.2 and provides dynamic notification of the kernel memory allocation stalling to reclaim memory. When we get such a notification, we trigger the buffer cache shrinker to run to free cached buffers, just like the kernel does. Hence we no longer need to care about how much RAM we need to reserve for the buffer cache.
The patch series is structured in a way that:
- guts old tracing and debug interfaces
- adds support infrastructure for atomics, locking, etc.
- creates a new buftarg abstraction
- restructures buftarg implemenation and usage
- moves uncached IO into new buftarg abstraction
- reworks userspace specific buffer read/write interfaces
- implements new functions the buftarg abstraction eeds to provide
- adds the kernel buffer cache implementation
- switches the implementation and removes the old rd/wr and caching code
- converts the buftarg IO engine to use AIO
The AIO engine is not fully functional yet - it does not support discontiguous buffers correctly yet (needs an iocb per buffer map) - but otherwise it largely works just fine.
There are -lots- of loose ends that still need to be worked on, especially w.r.t. xfs_repair. Stuff that comes to mind:
- xfs_buf_mark_dirty() just currently calls xfs_bwrite() and does synchronous IO. This needs to be converted to a buftarg internal delwri buffer list and a flushing mechanism needs to be added.
- AIO engine does not support discontiguous buffers.
- work out best way to handle IOCBs for AIO - is embedding them in the xfs_buf the only way to do this efficiently?
- rationalise the xfs_buf/xfs_buftarg split. Work out where to define the stuff that is different between kernel and userspace (e.g. the struct xfs_buf) vs the stuff that remains the same (e.g. the XBF buffer flags)
- lots of code cleanup
- xfs_repair does not cache between phases right now - it unconditionally purges the AG cache at the end of AG processing. This keeps memory usage way down, and for filesystems that have too much metadata to cache entirely in RAM, it's *much* faster than xfs_repair v5.6.0. On smaller filesystems, however, hitting RAM caches is much more desirable. Hence there is work to be done here determining how to manage the caches effectively.
- async buffer readahead works in userspace now, but unless you have a very high IOP device (think in multiples of 100kiops) the xfs_repair prefetch mechanism that trades off bandwidth for iops is still faster. More investigative work is needed here.
- xfs_db could likely use readahead for btree traversals and other things.
- More memory pressure trigger testing needs to be done to determine if the trigger settings are sufficient to prevent OOM kills on different hardware and filesystem configurations.
- Replace AIO engine with io_uring engine?
- start working on splitting the kernel xfs_buf.[ch] code the same way as the userspace code and moving xfs_buf.[ch] to fs/xfs/libxfs so that they can be updated in sync.
There's plenty of other stuff, too, but this like is a good start....
Cheers,
Dave.
Dave Chinner (27):
xfsprogs: remove unused buffer tracing code
xfsprogs: remove unused IO_DEBUG functionality
libxfs: get rid of b_bcount from xfs_buf
libxfs: rename buftarg->dev to btdev
xfsprogs: get rid of ancient btree tracing fragments
xfsprogs: remove xfs_buf_t typedef
xfsprogs: introduce liburcu support
libxfs: add spinlock_t wrapper
atomic: convert to uatomic
libxfs: add kernel-compatible completion API
libxfs: add wrappers for kernel semaphores
xfsprogs: convert use-once buffer reads to uncached IO
libxfs: introduce userspace buftarg infrastructure
xfs: rename libxfs_buftarg_init to libxfs_open_devices()
libxfs: introduce userspace buftarg infrastructure
libxfs: add a synchronous IO engine to the buftarg
xfsprogs: convert libxfs_readbufr to libxfs_buf_read_uncached
libxfs: convert libxfs_bwrite to buftarg IO
libxfs: add cache infrastructure to buftarg
libxfs: add internal lru to btcache
libxfs: Add kernel list_lru wrapper
libxfs: introduce new buffer cache infrastructure
libxfs: use PSI information to detect memory pressure
libxfs: add a buftarg cache shrinker implementation
libxfs: switch buffer cache implementations
build: set platform_defs.h.in dependency correctly
libxfs: convert sync IO buftarg engine to AIO
Makefile | 2 +- configure.ac | 3 + copy/Makefile | 3 +- copy/xfs_copy.c | 15 +- db/Makefile | 3 +- db/init.c | 4 +- db/io.c | 33 +- db/metadump.c | 2 +- db/sb.c | 2 +- debian/control | 2 +- growfs/Makefile | 3 +- include/Makefile | 6 +- include/atomic.h | 63 +- include/builddefs.in | 6 +- include/cache.h | 133 ---- include/completion.h | 61 ++ include/libxfs.h | 30 +- include/libxlog.h | 6 +- include/list_lru.h | 69 ++ include/platform_defs.h.in | 2 + include/sema.h | 35 + include/spinlock.h | 25 + include/xfs.h | 23 + include/xfs_btree_trace.h | 87 --- include/xfs_inode.h | 3 +- include/xfs_mount.h | 13 + include/xfs_trace.h | 24 + include/xfs_trans.h | 1 + libfrog/linux.c | 14 +- libfrog/workqueue.c | 3 + libxfs/Makefile | 7 +- libxfs/buftarg.c | 953 +++++++++++++++++++++++++ libxfs/cache.c | 724 ------------------- libxfs/init.c | 220 +++--- libxfs/libxfs_api_defs.h | 7 + libxfs/libxfs_io.h | 185 +---- libxfs/libxfs_priv.h | 74 +- libxfs/logitem.c | 10 +- libxfs/rdwr.c | 1234 +------------------------------- libxfs/trans.c | 21 +- libxfs/util.c | 8 +- libxfs/xfs_alloc.c | 16 +- libxfs/xfs_bmap.c | 6 +- libxfs/xfs_btree.c | 10 +- libxfs/xfs_buf.c | 1357 ++++++++++++++++++++++++++++++++++++ libxfs/xfs_buf.h | 242 +++++++ libxfs/xfs_buftarg.h | 176 +++++ libxfs/xfs_ialloc.c | 4 +- libxfs/xfs_rtbitmap.c | 22 +- libxlog/xfs_log_recover.c | 23 +- logprint/Makefile | 3 +- logprint/log_print_all.c | 2 +- logprint/logprint.c | 2 +- m4/Makefile | 1 + m4/package_urcu.m4 | 22 + mdrestore/Makefile | 3 +- mkfs/Makefile | 2 +- mkfs/proto.c | 11 +- mkfs/xfs_mkfs.c | 61 +- repair/Makefile | 2 +- repair/agheader.c | 2 +- repair/attr_repair.c | 14 +- repair/da_util.c | 2 +- repair/da_util.h | 2 +- repair/dino_chunks.c | 14 +- repair/dinode.c | 4 +- repair/incore.h | 2 +- repair/phase3.c | 7 +- repair/phase4.c | 5 +- repair/phase5.c | 2 +- repair/phase6.c | 4 +- repair/prefetch.c | 118 ++-- repair/progress.c | 16 +- repair/progress.h | 4 +- repair/rt.c | 4 +- repair/scan.c | 10 +- repair/xfs_repair.c | 192 +++-- scrub/Makefile | 3 +- scrub/progress.c | 2 + 79 files changed, 3644 insertions(+), 2847 deletions(-) delete mode 100644 include/cache.h create mode 100644 include/completion.h create mode 100644 include/list_lru.h create mode 100644 include/sema.h create mode 100644 include/spinlock.h delete mode 100644 include/xfs_btree_trace.h create mode 100644 libxfs/buftarg.c delete mode 100644 libxfs/cache.c create mode 100644 libxfs/xfs_buf.c create mode 100644 libxfs/xfs_buf.h create mode 100644 libxfs/xfs_buftarg.h create mode 100644 m4/package_urcu.m4
Signature · 1 line
2.28.0
PATCH 1/27 xfsprogs: remove unused buffer tracing code
From: Dave Chinner <[email protected]>
This isn't particularly useful for finding issues, it's rarely used and complicates the conversion to the kernel buffer cache code. THe kernel code also carries it's own trace hooks that could be implemented if tracing is needed, so remove this code to make the conversion simpler.
--- libxfs/libxfs_io.h | 49 --------------- libxfs/rdwr.c | 149 --------------------------------------------- 2 files changed, 198 deletions(-)
libxfs/libxfs_io.h +0 -49
@@ -80,12 +80,6 @@ typedef struct xfs_buf {struct xfs_buf_map __b_map;int b_nmaps;struct list_head b_list; -#ifdef XFS_BUF_TRACING -struct list_head b_lock_list; -const char *b_func; -const char *b_file; -int b_line; -#endif } xfs_buf_t;bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic);
@@ -129,47 +123,6 @@ extern struct cache_operations libxfs_bcache_operations;/* Return the buffer even if the verifiers fail. */ #define LIBXFS_READBUF_SALVAGE (1 << 1) -#ifdef XFS_BUF_TRACING - -#define libxfs_buf_read (dev, daddr, len, flags, bpp, ops) \ - libxfs_trace_readbuf(__FUNCTION__, __FILE__, __LINE__, \ - (dev), (daddr), (len), (flags), (bpp), (ops)) -#define libxfs_buf_read_map (dev, map, nmaps, flags, bpp, ops) \ - libxfs_trace_readbuf_map(__FUNCTION__, __FILE__, __LINE__, \ - (dev), (map), (nmaps), (flags), (bpp), (ops)) -#define libxfs_buf_mark_dirty (buf) \ - libxfs_trace_dirtybuf(__FUNCTION__, __FILE__, __LINE__, \ - (buf)) -#define libxfs_buf_get (dev, daddr, len, bpp) \ - libxfs_trace_getbuf(__FUNCTION__, __FILE__, __LINE__, \ - (dev), (daddr), (len), (bpp)) -#define libxfs_buf_get_map (dev, map, nmaps, flags, bpp) \ - libxfs_trace_getbuf_map(__FUNCTION__, __FILE__, __LINE__, \ - (dev), (map), (nmaps), (flags), (bpp)) -#define libxfs_buf_relse (buf) \ - libxfs_trace_putbuf(__FUNCTION__, __FILE__, __LINE__, (buf)) - -int libxfs_trace_readbuf (const char *func,const char *file,int line, -struct xfs_buftarg *btp,xfs_daddr_t daddr,size_t len, -int flags,const struct xfs_buf_ops *ops, -struct xfs_buf **bpp); -int libxfs_trace_readbuf_map (const char *func,const char *file,int line, -struct xfs_buftarg *btp,struct xfs_buf_map *maps, -int nmaps,int flags,struct xfs_buf **bpp, -const struct xfs_buf_ops *ops); -void libxfs_trace_dirtybuf (const char *func,const char *file,int line, -struct xfs_buf *bp); -int libxfs_trace_getbuf (const char *func,const char *file,int line, -struct xfs_buftarg *btp,xfs_daddr_t daddr, -size_t len,struct xfs_buf **bpp); -int libxfs_trace_getbuf_map (const char *func,const char *file,int line, -struct xfs_buftarg *btp,struct xfs_buf_map *map, -int nmaps,int flags,struct xfs_buf **bpp); -extern void libxfs_trace_putbuf (const char *,const char *,int , -xfs_buf_t *); - -#else -int libxfs_buf_read_map (struct xfs_buftarg *btp,struct xfs_buf_map *maps,int nmaps,int flags,struct xfs_buf **bpp,const struct xfs_buf_ops *ops);
@@ -204,8 +157,6 @@ libxfs_buf_read(return libxfs_buf_read_map (target, &map,1 , flags, bpp, ops); } -#endif /* XFS_BUF_TRACING */ -int libxfs_readbuf_verify (struct xfs_buf *bp,const struct xfs_buf_ops *ops);struct xfs_buf *libxfs_getsb (struct xfs_mount *mp);extern void libxfs_bcache_purge (void );
libxfs/rdwr.c +0 -149
179 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 2/27 xfsprogs: remove unused IO_DEBUG functionality
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Similar to the XFS_BUF_TRACING code, this is largely unused and not hugely helpfule for tracing buffer IO. Remove it to simplify the conversion process to the kernel buffer cache.
--- libxfs/Makefile | 1 - libxfs/rdwr.c | 45 --------------------------------------------- 2 files changed, 46 deletions(-)
libxfs/Makefile +0 -1
@@ -102,7 +102,6 @@ CFILES = cache.c \ # # Tracing flags: -# -DIO_DEBUG reads and writes of buffers # -DMEM_DEBUG all zone memory use # -DLI_DEBUG log item (ino/buf) manipulation # -DXACT_DEBUG transaction state changes
libxfs/rdwr.c +0 -45
93 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH 02/27] xfsprogs: remove unused IO_DEBUG functionality
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Similar to the
XFS_BUF_TRACINGcode, this is largely unused and not hugely helpfule for tracing buffer IO. Remove it to simplify the conversion process to the kernel buffer cache.Signed-off-by Dave Chinner <[email protected]>
Fine by me, I can merge this ahead of other stuff so you don't have to carry it along.
PATCH 3/27 libxfs: get rid of b_bcount from xfs_buf
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
We no longer use it in the kernel - it has been replaced by b_length and it only exists in userspace because we haven't converted it over. Do that now before we introduce a heap of code that doesn't ever set it and so breaks all the progs code.
WHile we are doing this, kill the XFS_BUF_SIZE macro that has also been removed from the kernel, too.
--- db/metadump.c | 2 +- libxfs/libxfs_io.h | 4 +--- libxfs/logitem.c | 4 ++-- libxfs/rdwr.c | 20 ++++++++++---------- libxfs/trans.c | 2 +- libxlog/xfs_log_recover.c | 6 +++--- mkfs/proto.c | 9 ++++++--- repair/attr_repair.c | 4 ++-- repair/dino_chunks.c | 2 +- repair/prefetch.c | 14 ++++++++------ 10 files changed, 35 insertions(+), 32 deletions(-)
db/metadump.c +1 -1
@@ -204,7 +204,7 @@ write_buf(print_warning ("obfuscation corrupted block at %s bno 0x%llx/0x%x" , bp->b_ops->name, - (long long )bp->b_bn, bp->b_bcount); + (long long )bp->b_bn,BBTOB (bp->b_length)); } }
libxfs/libxfs_io.h +1 -3
@@ -61,7 +61,6 @@ typedef struct xfs_buf {struct cache_node b_node;unsigned int b_flags;xfs_daddr_t b_bn; -unsigned b_bcount;unsigned int b_length;struct xfs_buftarg *b_target;#define b_dev b_target->dev
@@ -98,7 +97,6 @@ typedef unsigned int xfs_buf_flags_t;#define xfs_buf_offset (bp, offset) ((bp)->b_addr + (offset))#define XFS_BUF_ADDR (bp) ((bp)->b_bn) -#define XFS_BUF_SIZE (bp) ((bp)->b_bcount)#define XFS_BUF_SET_ADDR (bp,blk) ((bp)->b_bn = (blk))
@@ -191,7 +189,7 @@ static inline int xfs_buf_associate_memory(struct xfs_buf *bp, void *mem, size_t len) { bp->b_addr = mem; - bp->b_bcount = len; + bp->b_length =BTOBB (len);return 0 ; }
libxfs/logitem.c +2 -2
@@ -47,7 +47,7 @@ xfs_trans_buf_item_match(if (blip->bli_item.li_type ==XFS_LI_BUF && blip->bli_buf->b_target->dev == btp->dev &&XFS_BUF_ADDR (blip->bli_buf) == map[0 ].bm_bn && - blip->bli_buf->b_bcount ==BBTOB (len)) { + blip->bli_buf->b_length == len) {ASSERT (blip->bli_buf->b_map_count == nmaps);return blip->bli_buf; }
@@ -105,7 +105,7 @@ xfs_buf_item_init( bip->bli_buf = bp; bip->__bli_format.blf_type =XFS_LI_BUF ; bip->__bli_format.blf_blkno = (int64_t )XFS_BUF_ADDR (bp); - bip->__bli_format.blf_len = (unsigned short )BTOBB (bp->b_bcount); + bip->__bli_format.blf_len = (unsigned short )bp->b_length; bp->b_log_item = bip; }
libxfs/rdwr.c +10 -10
@@ -141,7 +141,7 @@ static char *next(struct xfs_buf *buf = (struct xfs_buf *)private;if (buf && - (buf->b_bcount < (int )(ptr - (char *)buf->b_addr) + offset)) + (BBTOB (buf->b_length) < (int )(ptr - (char *)buf->b_addr) + offset))abort ();return ptr + offset;
@@ -203,7 +203,7 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key)if (bp->b_target->dev == bkey->buftarg->dev && bp->b_bn == bkey->blkno) { -if (bp->b_bcount ==BBTOB (bkey->bblen)) +if (bp->b_length == bkey->bblen)return CACHE_HIT ;#ifdef IO_BCOMPARE_CHECK if (!(libxfs_bcache->c_flags &CACHE_MISCOMPARE_PURGE )) {
@@ -211,7 +211,8 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key)"%lx: Badness in key lookup (length)\n" "bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n" ,pthread_self (), - (unsigned long long )bp->b_bn, (int )bp->b_bcount, + (unsigned long long )bp->b_bn, +BBTOB (bp->b_length), (unsigned long long )bkey->blkno, BBTOB(bkey->bblen)); }
@@ -227,7 +228,6 @@ __initbuf(xfs_buf_t *bp, struct xfs_buftarg *btp, xfs_daddr_t bno, { bp->b_flags =0 ; bp->b_bn = bno; - bp->b_bcount = bytes; bp->b_length =BTOBB (bytes); bp->b_target = btp; bp->b_mount = btp->bt_mount;
@@ -306,7 +306,7 @@ __libxfs_getbufr(int blen)pthread_mutex_lock (&xfs_buf_freelist.cm_mutex);if (!list_empty (&xfs_buf_freelist.cm_list)) {list_for_each_entry (bp, &xfs_buf_freelist.cm_list, b_node.cn_mru) { -if (bp->b_bcount == blen) { +if (bp->b_length ==BTOBB (blen)) {list_del_init (&bp->b_node.cn_mru);break ; }
@@ -581,13 +581,13 @@ libxfs_readbufr(struct xfs_buftarg *btp, xfs_daddr_t blkno, xfs_buf_t *bp,int bytes =BBTOB (len);int error; -ASSERT (BBTOB (len) <= bp->b_bcount); +ASSERT (len <= bp->b_length); error =__read_buf (fd, bp->b_addr, bytes,LIBXFS_BBTOOFF64 (blkno), flags);if (!error && bp->b_target->dev == btp->dev && bp->b_bn == blkno && - bp->b_bcount == bytes) + bp->b_length == len) bp->b_flags |=LIBXFS_B_UPTODATE ; bp->b_error = error;return error;
@@ -824,13 +824,13 @@ libxfs_bwrite(fprintf (stderr,_ ("%s: write verifier failed on %s bno 0x%llx/0x%x\n" ), __func__, bp->b_ops->name, - (long long )bp->b_bn, bp->b_bcount); + (long long )bp->b_bn, bp->b_length);return bp->b_error; } }if (!(bp->b_flags &LIBXFS_B_DISCONTIG )) { - bp->b_error =__write_buf (fd, bp->b_addr, bp->b_bcount, + bp->b_error =__write_buf (fd, bp->b_addr,BBTOB (bp->b_length),LIBXFS_BBTOOFF64 (bp->b_bn), bp->b_flags); }else {int i;
@@ -852,7 +852,7 @@ libxfs_bwrite(fprintf (stderr,_ ("%s: write failed on %s bno 0x%llx/0x%x, err=%d\n" ), __func__, bp->b_ops ? bp->b_ops->name :"(unknown)" , - (long long )bp->b_bn, bp->b_bcount, -bp->b_error); + (long long )bp->b_bn, bp->b_length, -bp->b_error); }else { bp->b_flags |=LIBXFS_B_UPTODATE ; bp->b_flags &= ~(LIBXFS_B_DIRTY |LIBXFS_B_UNCHECKED );
libxfs/trans.c +1 -1
@@ -721,7 +721,7 @@ libxfs_trans_ordered_buf(bool ret; ret =test_bit (XFS_LI_DIRTY , &bip->bli_item.li_flags); -libxfs_trans_log_buf (tp, bp,0 , bp->b_bcount); +libxfs_trans_log_buf (tp, bp,0 ,BBTOB (bp->b_length));return ret; }
libxlog/xfs_log_recover.c +3 -3
@@ -112,10 +112,10 @@ xlog_bread_noalign( nbblks =round_up (nbblks, log->l_sectBBsize);ASSERT (nbblks >0 ); -ASSERT (BBTOB (nbblks) <=XFS_BUF_SIZE (bp)); +ASSERT (nbblks <= bp->b_length);XFS_BUF_SET_ADDR (bp, log->l_logBBstart + blk_no); - bp->b_bcount =BBTOB (nbblks); + bp->b_length = nbblks; bp->b_error =0 ;return libxfs_readbufr (log->l_dev,XFS_BUF_ADDR (bp), bp, nbblks,0 );
@@ -152,7 +152,7 @@ xlog_bread_offset( char *offset) {char *orig_offset = bp->b_addr; -int orig_len = bp->b_bcount; +int orig_len =BBTOB (bp->b_length);int error, error2; error =xfs_buf_associate_memory (bp, offset,BBTOB (nbblks));
mkfs/proto.c +6 -3
@@ -241,6 +241,8 @@ newfile( ip->i_df.if_format =XFS_DINODE_FMT_LOCAL ; flags =XFS_ILOG_DDATA ; }else if (len >0 ) { +int bcount; + nb =XFS_B_TO_FSB (mp, len); nmap =1 ; error = -libxfs_bmapi_write (tp, ip,0 , nb,0 , nb, &map, &nmap);
@@ -269,10 +271,11 @@ newfile(exit (1 ); }memmove (bp->b_addr, buf, len); -if (len < bp->b_bcount) -memset ((char *)bp->b_addr + len,0 , bp->b_bcount - len); + bcount =BBTOB (bp->b_length); +if (len < bcount) +memset ((char *)bp->b_addr + len,0 , bcount - len);if (logit) -libxfs_trans_log_buf (tp, bp,0 , bp->b_bcount -1 ); +libxfs_trans_log_buf (tp, bp,0 , bcount -1 );else {libxfs_buf_mark_dirty (bp);libxfs_buf_relse (bp);
repair/attr_repair.c +2 -2
@@ -424,9 +424,9 @@ rmtval_get(xfs_mount_t *mp, xfs_ino_t ino, blkmap_t *blkmap,break ; } -ASSERT (mp->m_sb.sb_blocksize == bp->b_bcount); +ASSERT (mp->m_sb.sb_blocksize ==BBTOB (bp->b_length)); - length =min (bp->b_bcount - hdrsize, valuelen - amountdone); + length =min (BBTOB (bp->b_length) - hdrsize, valuelen - amountdone);memmove (value, bp->b_addr + hdrsize, length); amountdone += length; value += length;
repair/dino_chunks.c +1 -1
@@ -687,7 +687,7 @@ process_inode_chunk(pftrace ("readbuf %p (%llu, %d) in AG %d" , bplist[bp_index], (long long )XFS_BUF_ADDR (bplist[bp_index]), - bplist[bp_index]->b_bcount, agno); + bplist[bp_index]->b_length, agno); bplist[bp_index]->b_ops = &xfs_inode_buf_ops;
repair/prefetch.c +8 -6
@@ -411,7 +411,9 @@ pf_read_inode_dirs(if (error)return ; -for (icnt =0 ; icnt < (bp->b_bcount >> mp->m_sb.sb_inodelog); icnt++) { +for (icnt =0 ; + icnt < (BBTOB (bp->b_length) >> mp->m_sb.sb_inodelog); + icnt++) { dino =xfs_make_iptr (mp, bp, icnt);/*
@@ -523,21 +525,21 @@ pf_batch_read(*/ first_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[0 ])); last_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[num-1 ])) + -XFS_BUF_SIZE (bplist[num-1 ]); +BBTOB (bplist[num-1 ]->b_length);while (num >1 && last_off - first_off > pf_max_bytes) { num--; last_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[num-1 ])) + -XFS_BUF_SIZE (bplist[num-1 ]); +BBTOB (bplist[num-1 ]->b_length); }if (num < ((last_off - first_off) >> (mp->m_sb.sb_blocklog +3 ))) {/* * not enough blocks for one big read, so determine * the number of blocks that are close enough. */ - last_off = first_off +XFS_BUF_SIZE (bplist[0 ]); + last_off = first_off +BBTOB (bplist[0 ]->b_length);for (i =1 ; i < num; i++) { next_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[i])) + -XFS_BUF_SIZE (bplist[i]); +BBTOB (bplist[i]->b_length);if (next_off - last_off > pf_batch_bytes)break ; last_off = next_off;
@@ -596,7 +598,7 @@ pf_batch_read(for (i =0 ; i < num; i++) { pbuf = ((char *)buf) + (LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[i])) - first_off); - size =XFS_BUF_SIZE (bplist[i]); + size =BBTOB (bplist[i]->b_length);if (len < size)break ;memcpy (bplist[i]->b_addr, pbuf, size);
Signature · 1 line
2.28.0
Re: [PATCH 03/27] libxfs: get rid of b_bcount from xfs_buf
Quoted 11 lines · Dave Chinner
From: Dave Chinner <[email protected]>
We no longer use it in the kernel - it has been replaced by
b_lengthand it only exists in userspace because we haven't converted it over. Do that now before we introduce a heap of code that doesn't ever set it and so breaks all the progs code.WHile we are doing this, kill the
XFS_BUF_SIZEmacro that has also been removed from the kernel, too.Signed-off-by Dave Chinner <[email protected]>
Sorry, I should have picked this up earlier with the couple other "trivial" patches in this series, will do so now.
PATCH 4/27 libxfs: rename buftarg->dev to btdev
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
To prepare for alignment with kernel buftarg code.
--- libxfs/init.c | 14 +++++++------- libxfs/libxfs_io.h | 3 +-- libxfs/logitem.c | 2 +- libxfs/rdwr.c | 20 ++++++++++---------- mkfs/xfs_mkfs.c | 2 +- repair/prefetch.c | 2 +- 6 files changed, 21 insertions(+), 22 deletions(-)
libxfs/init.c +7 -7
@@ -443,7 +443,7 @@ rtmount_init(return -1 ; } -if (mp->m_rtdev_targp->dev ==0 && !(flags &LIBXFS_MOUNT_DEBUGGER )) { +if (mp->m_rtdev_targp->bt_bdev ==0 && !(flags &LIBXFS_MOUNT_DEBUGGER )) {fprintf (stderr,_ ("%s: filesystem has a realtime subvolume\n" ), progname);return -1 ;
@@ -601,7 +601,7 @@ libxfs_buftarg_alloc(exit (1 ); } btp->bt_mount = mp; - btp->dev = dev; + btp->bt_bdev = dev; btp->flags =0 ;return btp;
@@ -616,7 +616,7 @@ libxfs_buftarg_init( {if (mp->m_ddev_targp) {/* should already have all buftargs initialised */ -if (mp->m_ddev_targp->dev != dev || +if (mp->m_ddev_targp->bt_bdev != dev || mp->m_ddev_targp->bt_mount != mp) { fprintf(stderr, _("%s: bad buftarg reinit, ddev\n"),
@@ -630,14 +630,14 @@ libxfs_buftarg_init( progname);exit (1 ); } - }else if (mp->m_logdev_targp->dev != logdev || + }else if (mp->m_logdev_targp->bt_bdev != logdev || mp->m_logdev_targp->bt_mount != mp) {fprintf (stderr,_ ("%s: bad buftarg reinit, logdev\n" ), progname);exit (1 ); } -if (rtdev && (mp->m_rtdev_targp->dev != rtdev || +if (rtdev && (mp->m_rtdev_targp->bt_bdev != rtdev || mp->m_rtdev_targp->bt_mount != mp)) { fprintf(stderr, _("%s: bad buftarg reinit, rtdev\n"),
@@ -760,8 +760,8 @@ libxfs_mount( } elselibxfs_buf_relse (bp ); - if (mp->m_logdev_targp->dev && - mp->m_logdev_targp->dev != mp->m_ddev_targp->dev) { + if (mp->m_logdev_targp->bt_bdev && + mp->m_logdev_targp->bt_bdev != mp->m_ddev_targp->bt_bdev) { d = (xfs_daddr_t) XFS_FSB_TO_BB(mp, mp->m_sb.sb_logblocks); if (XFS_BB_TO_FSB(mp, d) != mp->m_sb.sb_logblocks || libxfs_buf_read(mp->m_logdev_targp,
libxfs/libxfs_io.h +1 -2
@@ -22,7 +22,7 @@ struct xfs_perag;*/ struct xfs_buftarg {struct xfs_mount *bt_mount; -dev_t dev; +dev_t bt_bdev;unsigned int flags; };
@@ -63,7 +63,6 @@ typedef struct xfs_buf {xfs_daddr_t b_bn;unsigned int b_length;struct xfs_buftarg *b_target; -#define b_dev b_target->devpthread_mutex_t b_lock;pthread_t b_holder;unsigned int b_recur;
libxfs/logitem.c +1 -1
@@ -45,7 +45,7 @@ xfs_trans_buf_item_match(list_for_each_entry (lip, &tp->t_items, li_trans) { blip = (struct xfs_buf_log_item *)lip;if (blip->bli_item.li_type ==XFS_LI_BUF && - blip->bli_buf->b_target->dev == btp->dev && + blip->bli_buf->b_target->bt_bdev == btp->bt_bdev &&XFS_BUF_ADDR (blip->bli_buf) == map[0 ].bm_bn && blip->bli_buf->b_length == len) {ASSERT (blip->bli_buf->b_map_count == nmaps);
libxfs/rdwr.c +10 -10
@@ -68,7 +68,7 @@ libxfs_device_zero(struct xfs_buftarg *btp, xfs_daddr_t start, uint len)char *z;int error, fd; - fd =libxfs_device_to_fd (btp->dev); + fd =libxfs_device_to_fd (btp->bt_bdev); start_offset =LIBXFS_BBTOOFF64 (start);/* try to use special zeroing methods, fall back to writes if needed */
@@ -201,7 +201,7 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key) b_node);struct xfs_bufkey *bkey = (struct xfs_bufkey *)key; -if (bp->b_target->dev == bkey->buftarg->dev && +if (bp->b_target->bt_bdev == bkey->buftarg->bt_bdev && bp->b_bn == bkey->blkno) {if (bp->b_length == bkey->bblen)return CACHE_HIT ;
@@ -577,7 +577,7 @@ int libxfs_readbufr(struct xfs_buftarg *btp, xfs_daddr_t blkno, xfs_buf_t *bp, int len, int flags) { - int fd = libxfs_device_to_fd(btp->dev); + int fd = libxfs_device_to_fd(btp->bt_bdev);int bytes =BBTOB (len);int error;
@@ -585,7 +585,7 @@ libxfs_readbufr(struct xfs_buftarg *btp, xfs_daddr_t blkno, xfs_buf_t *bp, error =__read_buf (fd, bp->b_addr, bytes,LIBXFS_BBTOOFF64 (blkno), flags);if (!error && - bp->b_target->dev == btp->dev && + bp->b_target->bt_bdev == btp->bt_bdev && bp->b_bn == blkno && bp->b_length == len) bp->b_flags |=LIBXFS_B_UPTODATE ;
@@ -615,7 +615,7 @@ libxfs_readbufr_map(struct xfs_buftarg *btp, struct xfs_buf *bp, int flags)void *buf;int i; - fd =libxfs_device_to_fd (btp->dev); + fd =libxfs_device_to_fd (btp->bt_bdev); buf = bp->b_addr;for (i =0 ; i < bp->b_nmaps; i++) {off64_t offset =LIBXFS_BBTOOFF64 (bp->b_maps[i].bm_bn);
@@ -799,7 +799,7 @@ intlibxfs_bwrite ( struct xfs_buf *bp) { -int fd =libxfs_device_to_fd (bp->b_target->dev); +int fd =libxfs_device_to_fd (bp->b_target->bt_bdev);/* * we never write buffers that are marked stale. This indicates they
@@ -1126,11 +1126,11 @@ libxfs_blkdev_issue_flush( {int fd, ret; -if (btp->dev ==0 ) +if (btp->bt_bdev ==0 )return 0 ; - fd =libxfs_device_to_fd (btp->dev); - ret =platform_flush_device (fd, btp->dev); + fd =libxfs_device_to_fd (btp->bt_bdev); + ret =platform_flush_device (fd, btp->bt_bdev);return ret ? -errno :0 ; }
@@ -1207,7 +1207,7 @@ libxfs_log_clear(char *ptr;if (((btp && dptr) || (!btp && !dptr)) || - (btp && !btp->dev) || !fs_uuid) + (btp && !btp->bt_bdev) || !fs_uuid)return -EINVAL ;/* first zero the log */
mkfs/xfs_mkfs.c +1 -1
@@ -3524,7 +3524,7 @@ prepare_devices( lsunit, XLOG_FMT, XLOG_INIT_CYCLE, false);/* finally, check we can write the last block in the realtime area */ -if (mp->m_rtdev_targp->dev && cfg->rtblocks >0 ) { +if (mp->m_rtdev_targp->bt_bdev && cfg->rtblocks >0 ) { buf =alloc_write_buf (mp->m_rtdev_targp,XFS_FSB_TO_BB (mp, cfg->rtblocks -1LL ),BTOBB (cfg->blocksize));
repair/prefetch.c +1 -1
@@ -871,7 +871,7 @@ init_prefetch( xfs_mount_t *pmp) { mp = pmp; - mp_fd =libxfs_device_to_fd (mp->m_ddev_targp->dev); + mp_fd =libxfs_device_to_fd (mp->m_ddev_targp->bt_bdev); pf_max_bytes =sysconf (_SC_PAGE_SIZE) <<7 ; pf_max_bbs = pf_max_bytes >>BBSHIFT ; pf_max_fsbs = pf_max_bytes >> mp->m_sb.sb_blocklog;
Signature · 1 line
2.28.0
Re: [PATCH 04/27] libxfs: rename buftarg->dev to btdev
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
To prepare for alignment with kernel buftarg code.
Signed-off-by Dave Chinner <[email protected]>
PATCH 5/27 xfsprogs: get rid of ancient btree tracing fragments
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
If we are going to do any userspace tracing, it will be via the existing libxfs tracepoint hooks, not the ancient Irix tracing macros.
--- include/Makefile | 1 - include/libxfs.h | 1 - include/xfs_btree_trace.h | 87 --------------------------------------- 3 files changed, 89 deletions(-) delete mode 100644 include/xfs_btree_trace.h
include/Makefile +0 -1
@@ -16,7 +16,6 @@ LIBHFILES = libxfs.h \ kmem.h \ list.h \ parent.h \ - xfs_btree_trace.h \ xfs_inode.h \ xfs_log_recover.h \ xfs_metadump.h \
include/libxfs.h +0 -1
@@ -67,7 +67,6 @@ struct iomap;#include "xfs_inode_buf.h" #include "xfs_alloc.h" #include "xfs_btree.h" -#include "xfs_btree_trace.h" #include "xfs_bmap.h" #include "xfs_trace.h" #include "xfs_trans.h"
include/xfs_btree_trace.h deleted +0 -87
87 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH 05/27] xfsprogs: get rid of ancient btree tracing fragments
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
If we are going to do any userspace tracing, it will be via the existing libxfs tracepoint hooks, not the ancient Irix tracing macros.
Signed-off-by Dave Chinner <[email protected]>
PATCH 6/27 xfsprogs: remove xfs_buf_t typedef
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Prepare for kernel xfs_buf alignment by getting rid of the xfs_buf_t typedef from userspace.
--- copy/xfs_copy.c | 2 +- include/libxlog.h | 6 +++--- libxfs/init.c | 2 +- libxfs/libxfs_io.h | 7 ++++--- libxfs/libxfs_priv.h | 4 ++-- libxfs/logitem.c | 4 ++-- libxfs/rdwr.c | 26 +++++++++++++------------- libxfs/trans.c | 18 +++++++++--------- libxfs/util.c | 7 +++---- libxfs/xfs_alloc.c | 16 ++++++++-------- libxfs/xfs_bmap.c | 6 +++--- libxfs/xfs_btree.c | 10 +++++----- libxfs/xfs_ialloc.c | 4 ++-- libxfs/xfs_rtbitmap.c | 22 +++++++++++----------- libxlog/xfs_log_recover.c | 12 ++++++------ logprint/log_print_all.c | 2 +- mkfs/proto.c | 2 +- mkfs/xfs_mkfs.c | 2 +- repair/agheader.c | 2 +- repair/attr_repair.c | 4 ++-- repair/da_util.h | 2 +- repair/dino_chunks.c | 8 ++++---- repair/incore.h | 2 +- repair/phase5.c | 2 +- repair/phase6.c | 4 ++-- repair/prefetch.c | 12 ++++++------ repair/rt.c | 4 ++-- repair/scan.c | 4 ++-- repair/xfs_repair.c | 2 +- 29 files changed, 99 insertions(+), 99 deletions(-)
copy/xfs_copy.c +1 -1
@@ -569,7 +569,7 @@ main(int argc, char **argv)xfs_mount_t *mp;xfs_mount_t mbuf;struct xlog xlog; -xfs_buf_t *sbp; +struct xfs_buf *sbp;xfs_sb_t *sb;xfs_agnumber_t num_ags, agno;xfs_agblock_t bno;
include/libxlog.h +3 -3
@@ -76,12 +76,12 @@ extern int xlog_is_dirty(struct xfs_mount *, struct xlog *, libxfs_init_t *, int);extern struct xfs_buf *xlog_get_bp (struct xlog *,int );extern int xlog_bread (struct xlog *log,xfs_daddr_t blk_no,int nbblks, -xfs_buf_t *bp,char **offset); +struct xfs_buf *bp,char **offset);extern int xlog_bread_noalign (struct xlog *log,xfs_daddr_t blk_no, -int nbblks,xfs_buf_t *bp); +int nbblks,struct xfs_buf *bp);extern int xlog_find_zeroed (struct xlog *log,xfs_daddr_t *blk_no); -extern int xlog_find_cycle_start (struct xlog *log,xfs_buf_t *bp, +extern int xlog_find_cycle_start (struct xlog *log,struct xfs_buf *bp,xfs_daddr_t first_blk,xfs_daddr_t *last_blk,uint cycle); extern int xlog_find_tail(struct xlog *log, xfs_daddr_t *head_blk,
libxfs/init.c +1 -1
@@ -30,7 +30,7 @@ char *progname = "libxfs"; /* default, changed by each tool */struct cache *libxfs_bcache;/* global buffer cache */ int libxfs_bhash_size;/* #buckets in bcache */ -int use_xfs_buf_lock;/* global flag: use xfs_buf_t locks for MT */ +int use_xfs_buf_lock;/* global flag: use struct xfs_buf locks for MT */ /* * dev_map - map open devices to fd.
libxfs/libxfs_io.h +4 -3
@@ -57,7 +57,7 @@ struct xfs_buf_ops {xfs_failaddr_t (*verify_struct)(struct xfs_buf *); }; -typedef struct xfs_buf { +struct xfs_buf {struct cache_node b_node;unsigned int b_flags;xfs_daddr_t b_bn;
@@ -78,7 +78,7 @@ typedef struct xfs_buf {struct xfs_buf_map __b_map;int b_nmaps;struct list_head b_list; -} xfs_buf_t; +};bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic);bool xfs_verify_magic16 (struct xfs_buf *bp,__be16 dmagic);
@@ -163,7 +163,8 @@ extern int libxfs_bcache_overflowed(void);/* Buffer (Raw) Interfaces */ int libxfs_bwrite (struct xfs_buf *bp); -extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t ,xfs_buf_t *,int ,int ); +extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t , +struct xfs_buf *,int ,int );extern int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int );extern int libxfs_device_zero (struct xfs_buftarg *,xfs_daddr_t ,uint );
libxfs/libxfs_priv.h +2 -2
@@ -665,10 +665,10 @@ int xfs_rtmodify_range(struct xfs_mount *mp, struct xfs_trans *tp, xfs_rtblock_t start, xfs_extlen_t len, int val);int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp,int log,xfs_rtblock_t bbno,int delta, -xfs_buf_t **rbpp,xfs_fsblock_t *rsb, +struct xfs_buf **rbpp,xfs_fsblock_t *rsb,xfs_suminfo_t *sum);int xfs_rtmodify_summary (struct xfs_mount *mp,struct xfs_trans *tp,int log, -xfs_rtblock_t bbno,int delta,xfs_buf_t **rbpp, +xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp,xfs_fsblock_t *rsb); int xfs_rtfree_range(struct xfs_mount *mp, struct xfs_trans *tp, xfs_rtblock_t start, xfs_extlen_t len,
libxfs/logitem.c +2 -2
@@ -27,7 +27,7 @@ kmem_zone_t *xfs_ili_zone; /* inode log item zone */* Check to see if a buffer matching the given parameters is already * a part of the given transaction. */ -xfs_buf_t * +struct xfs_buf * xfs_trans_buf_item_match( xfs_trans_t *tp, struct xfs_buftarg *btp,
@@ -68,7 +68,7 @@ xfs_trans_buf_item_match(*/ void xfs_buf_item_init ( -xfs_buf_t *bp, +struct xfs_buf *bp,xfs_mount_t *mp) {xfs_log_item_t *lip;
libxfs/rdwr.c +13 -13
@@ -223,7 +223,7 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key) }static void -__initbuf (xfs_buf_t *bp,struct xfs_buftarg *btp,xfs_daddr_t bno, +__initbuf (struct xfs_buf *bp,struct xfs_buftarg *btp,xfs_daddr_t bno,unsigned int bytes) { bp->b_flags =0 ;
@@ -257,14 +257,14 @@ __initbuf(xfs_buf_t *bp, struct xfs_buftarg *btp, xfs_daddr_t bno, }static void -libxfs_initbuf (xfs_buf_t *bp,struct xfs_buftarg *btp,xfs_daddr_t bno, +libxfs_initbuf (struct xfs_buf *bp,struct xfs_buftarg *btp,xfs_daddr_t bno,unsigned int bytes) {__initbuf (bp, btp, bno, bytes); }static void -libxfs_initbuf_map (xfs_buf_t *bp,struct xfs_buftarg *btp, +libxfs_initbuf_map (struct xfs_buf *bp,struct xfs_buftarg *btp,struct xfs_buf_map *map,int nmaps) {unsigned int bytes =0 ;
@@ -292,10 +292,10 @@ libxfs_initbuf_map(xfs_buf_t *bp, struct xfs_buftarg *btp, bp->b_flags |=LIBXFS_B_DISCONTIG ; } -static xfs_buf_t * +static struct xfs_buf *__libxfs_getbufr (int blen) { -xfs_buf_t *bp; +struct xfs_buf *bp;/* * first look for a buffer that can be used as-is,
@@ -313,7 +313,7 @@ __libxfs_getbufr(int blen) }if (&bp->b_node.cn_mru == &xfs_buf_freelist.cm_list) { bp =list_entry (xfs_buf_freelist.cm_list.next, - xfs_buf_t, b_node.cn_mru); + struct xfs_buf, b_node.cn_mru);list_del_init (&bp->b_node.cn_mru);free (bp->b_addr); bp->b_addr =NULL ;
@@ -331,10 +331,10 @@ __libxfs_getbufr(int blen)return bp; } -static xfs_buf_t * +static struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen) { -xfs_buf_t *bp; +struct xfs_buf *bp;int blen =BBTOB (bblen); bp =__libxfs_getbufr (blen);
@@ -343,11 +343,11 @@ libxfs_getbufr(struct xfs_buftarg *btp, xfs_daddr_t blkno, int bblen)return bp; } -static xfs_buf_t * +static struct xfs_buf *libxfs_getbufr_map (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen,struct xfs_buf_map *map,int nmaps) { -xfs_buf_t *bp; +struct xfs_buf *bp;int blen =BBTOB (bblen);if (!map || !nmaps) {
@@ -574,7 +574,7 @@ __read_buf(int fd, void *buf, int len, off64_t offset, int flags) }int -libxfs_readbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,xfs_buf_t *bp, +libxfs_readbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,struct xfs_buf *bp,int len,int flags) {int fd =libxfs_device_to_fd (btp->bt_bdev);
@@ -915,7 +915,7 @@ libxfs_bulkrelse(struct cache *cache, struct list_head *list) { - xfs_buf_t *bp; + struct xfs_buf *bp;int count =0 ;if (list_empty (list))
@@ -941,7 +941,7 @@ voidlibxfs_bcache_free (void) {struct list_head *cm_list; -xfs_buf_t *bp, *next; +struct xfs_buf *bp, *next; cm_list = &xfs_buf_freelist.cm_list;list_for_each_entry_safe (bp, next, cm_list, b_node.cn_mru) {
libxfs/trans.c +9 -9
@@ -395,7 +395,7 @@ libxfs_trans_bjoin(void libxfs_trans_bhold_release (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;
@@ -461,12 +461,12 @@ libxfs_trans_get_buf_map(return 0 ; } -xfs_buf_t * +struct xfs_buf *libxfs_trans_getsb (xfs_trans_t *tp,struct xfs_mount *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp;struct xfs_buf_log_item *bip;int len =XFS_FSS_TO_BB (mp,1 );DEFINE_SINGLE_BUF_MAP (map,XFS_SB_DADDR , len);
@@ -604,7 +604,7 @@ libxfs_trans_brelse(void libxfs_trans_bhold (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;
@@ -661,7 +661,7 @@ libxfs_trans_log_buf(void libxfs_trans_binval (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;
@@ -695,7 +695,7 @@ libxfs_trans_binval(void libxfs_trans_inode_alloc_buf (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;
@@ -799,7 +799,7 @@ static voidinode_item_done ( struct xfs_inode_log_item *iip) { -xfs_buf_t *bp; +struct xfs_buf *bp;int error;ASSERT (iip->ili_inode !=NULL );
@@ -835,7 +835,7 @@ static voidbuf_item_done ( xfs_buf_log_item_t *bip) { -xfs_buf_t *bp; +struct xfs_buf *bp;int hold;extern kmem_zone_t *xfs_buf_item_zone;
@@ -879,7 +879,7 @@ static voidbuf_item_unlock ( xfs_buf_log_item_t *bip) { -xfs_buf_t *bp = bip->bli_buf; +struct xfs_buf *bp = bip->bli_buf;uint hold;/* Clear the buffer's association with this transaction. */
libxfs/util.c +3 -4
@@ -255,7 +255,7 @@ libxfs_ialloc(xfs_dev_t rdev, struct cred *cr, struct fsxattr *fsx, - xfs_buf_t **ialloc_context, + struct xfs_buf **ialloc_context, xfs_inode_t **ipp) { xfs_ino_t ino;
@@ -358,7 +358,7 @@ libxfs_ialloc(int libxfs_iflush_int (xfs_inode_t *ip, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_inode_log_item *iip;xfs_dinode_t *dip;
@@ -540,11 +540,10 @@ libxfs_inode_alloc(struct fsxattr *fsx, xfs_inode_t **ipp) { - xfs_buf_t *ialloc_context; + struct xfs_buf *ialloc_context = NULL;xfs_inode_t *ip;int error; - ialloc_context = (xfs_buf_t *)0 ; error =libxfs_ialloc (*tp, pip, mode, nlink, rdev, cr, fsx, &ialloc_context, &ip);if (error) {
libxfs/xfs_alloc.c +8 -8
@@ -686,9 +686,9 @@ xfs_alloc_read_agfl( xfs_mount_t *mp, /* mount point structure */ xfs_trans_t *tp, /* transaction pointer */ xfs_agnumber_t agno, /* allocation group number */ - xfs_buf_t **bpp) /* buffer for the ag free block array */ + struct xfs_buf **bpp) /* buffer for the ag free block array */ { -xfs_buf_t *bp;/* return value */ +struct xfs_buf *bp;/* return value */ int error;ASSERT (agno !=NULLAGNUMBER );
@@ -2642,12 +2642,12 @@ out_no_agbp:int /* error */ xfs_alloc_get_freelist (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *agbp,/* buffer containing the agf structure */ +struct xfs_buf *agbp,/* buffer containing the agf structure */ xfs_agblock_t *bnop,/* block address retrieved from freelist */ int btreeblk)/* destination is a AGF btree */ {struct xfs_agf *agf = agbp->b_addr; -xfs_buf_t *agflbp;/* buffer for a.g. freelist structure */ +struct xfs_buf *agflbp;/* buffer for a.g. freelist structure */ xfs_agblock_t bno;/* block number returned */ __be32 *agfl_bno;int error;
@@ -2706,7 +2706,7 @@ xfs_alloc_get_freelist(void xfs_alloc_log_agf (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *bp,/* buffer for a.g. freelist header */ +struct xfs_buf *bp,/* buffer for a.g. freelist header */ int fields)/* mask of fields to be logged (XFS_AGF_...) */ {int first;/* first byte offset */
@@ -2752,7 +2752,7 @@ xfs_alloc_pagf_init( xfs_agnumber_t agno, /* allocation group number */ int flags) /* XFS_ALLOC_FLAGS_... */ { -xfs_buf_t *bp; +struct xfs_buf *bp;int error; error =xfs_alloc_read_agf (mp, tp, agno, flags, &bp);
@@ -2767,8 +2767,8 @@ xfs_alloc_pagf_init(int /* error */ xfs_alloc_put_freelist (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *agbp,/* buffer for a.g. freelist header */ -xfs_buf_t *agflbp,/* buffer for a.g. free block array */ +struct xfs_buf *agbp,/* buffer for a.g. freelist header */ +struct xfs_buf *agflbp,/* buffer for a.g. free block array */ xfs_agblock_t bno,/* block being freed */ int btreeblk)/* block came from a AGF btree */ {
libxfs/xfs_bmap.c +3 -3
@@ -314,7 +314,7 @@ xfs_bmap_check_leaf_extents(struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);struct xfs_btree_block *block;/* current btree block */ xfs_fsblock_t bno;/* block # of "block" */ -xfs_buf_t *bp;/* buffer for "block" */ +struct xfs_buf *bp;/* buffer for "block" */ int error;/* error return value */ xfs_extnum_t i=0 , j;/* index into the extents list */ int level;/* btree level, for checking */
@@ -585,7 +585,7 @@ xfs_bmap_btree_to_extents(struct xfs_btree_block *rblock = ifp->if_broot;struct xfs_btree_block *cblock;/* child btree block */ xfs_fsblock_t cbno;/* child block number */ -xfs_buf_t *cbp;/* child block's buffer */ +struct xfs_buf *cbp;/* child block's buffer */ int error;/* error return value */ __be64 *pp;/* ptr to block address */ struct xfs_owner_info oinfo;
@@ -823,7 +823,7 @@ xfs_bmap_local_to_extents(int flags;/* logging flags returned */ struct xfs_ifork *ifp;/* inode fork pointer */ xfs_alloc_arg_t args;/* allocation arguments */ -xfs_buf_t *bp;/* buffer for extent block */ +struct xfs_buf *bp;/* buffer for extent block */ struct xfs_bmbt_irec rec;struct xfs_iext_cursor icur;
libxfs/xfs_btree.c +5 -5
@@ -394,7 +394,7 @@ xfs_btree_dup_cursor( xfs_btree_cur_t *cur, /* input cursor */ xfs_btree_cur_t **ncur) /* output cursor */ { -xfs_buf_t *bp;/* btree block's buffer pointer */ +struct xfs_buf *bp;/* btree block's buffer pointer */ int error;/* error return value */ int i;/* level number of btree block */ xfs_mount_t *mp;/* mount structure for filesystem */
@@ -698,7 +698,7 @@ xfs_btree_firstrec( int level) /* level to change */ {struct xfs_btree_block *block;/* generic btree block pointer */ -xfs_buf_t *bp;/* buffer containing block */ +struct xfs_buf *bp;/* buffer containing block */ /* * Get the block pointer for this level.
@@ -728,7 +728,7 @@ xfs_btree_lastrec( int level) /* level to change */ {struct xfs_btree_block *block;/* generic btree block pointer */ -xfs_buf_t *bp;/* buffer containing block */ +struct xfs_buf *bp;/* buffer containing block */ /* * Get the block pointer for this level.
@@ -990,7 +990,7 @@ STATIC voidxfs_btree_setbuf ( xfs_btree_cur_t *cur,/* btree cursor */ int lev, /* level in btree */ - xfs_buf_t *bp) /* new buffer to set */ + struct xfs_buf *bp) /* new buffer to set */ {struct xfs_btree_block *b;/* btree block */
@@ -1633,7 +1633,7 @@ xfs_btree_decrement( int *stat) /* success/failure */ {struct xfs_btree_block *block; -xfs_buf_t *bp; +struct xfs_buf *bp;int error;/* error return value */ int lev;union xfs_btree_ptr ptr;
libxfs/xfs_ialloc.c +2 -2
@@ -2448,7 +2448,7 @@ out_map:void xfs_ialloc_log_agi (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *bp,/* allocation group header buffer */ +struct xfs_buf *bp,/* allocation group header buffer */ int fields)/* bitmask of fields to log */ {int first;/* first byte number */
@@ -2668,7 +2668,7 @@ xfs_ialloc_pagi_init( xfs_trans_t *tp, /* transaction pointer */ xfs_agnumber_t agno) /* allocation group number */ { -xfs_buf_t *bp =NULL ; +struct xfs_buf *bp =NULL ;int error; error =xfs_ialloc_read_agi (mp, tp, agno, &bp);
libxfs/xfs_rtbitmap.c +11 -11
@@ -54,9 +54,9 @@ xfs_rtbuf_get(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t block, /* block number in bitmap or summary */ int issum, /* is summary not bitmap */ - xfs_buf_t **bpp) /* output: buffer for the block */ + struct xfs_buf **bpp) /* output: buffer for the block */ { - xfs_buf_t *bp; /* block buffer, result */ + struct xfs_buf *bp; /* block buffer, result */xfs_inode_t *ip;/* bitmap or summary inode */ xfs_bmbt_irec_t map;int nmap =1 ;
@@ -99,7 +99,7 @@ xfs_rtfind_back(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtblock_t firstbit;/* first useful bit in the word */
@@ -274,7 +274,7 @@ xfs_rtfind_forw(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtblock_t i;/* current bit number rel. to start */
@@ -445,11 +445,11 @@ xfs_rtmodify_summary_int(int log,/* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */ - xfs_buf_t **rbpp, /* in/out: summary block buffer */ + struct xfs_buf **rbpp, /* in/out: summary block buffer */ xfs_fsblock_t *rsb, /* in/out: summary block number */ xfs_suminfo_t *sum) /* out: summary info for this block */ { - xfs_buf_t *bp; /* buffer for the summary block */ + struct xfs_buf *bp; /* buffer for the summary block */int error;/* error value */ xfs_fsblock_t sb;/* summary fsblock */ int so;/* index into the summary file */
@@ -515,7 +515,7 @@ xfs_rtmodify_summary( int log, /* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */ - xfs_buf_t **rbpp, /* in/out: summary block buffer */ + struct xfs_buf **rbpp, /* in/out: summary block buffer */ xfs_fsblock_t *rsb) /* in/out: summary block number */ { return xfs_rtmodify_summary_int(mp, tp, log, bbno,
@@ -537,7 +537,7 @@ xfs_rtmodify_range(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtword_t *first;/* first used word in the buffer */
@@ -688,7 +688,7 @@ xfs_rtfree_range(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t start, /* starting block to free */ xfs_extlen_t len, /* length to free */ - xfs_buf_t **rbpp, /* in/out: summary block buffer */ + struct xfs_buf **rbpp, /* in/out: summary block buffer */ xfs_fsblock_t *rsb) /* in/out: summary block number */ { xfs_rtblock_t end; /* end of the freed extent */
@@ -771,7 +771,7 @@ xfs_rtcheck_range(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtblock_t i;/* current bit number rel. to start */
@@ -967,7 +967,7 @@ xfs_rtfree_extent(int error;/* error value */ xfs_mount_t *mp;/* file system mount structure */ xfs_fsblock_t sb;/* summary file block number */ -xfs_buf_t *sumbp =NULL ;/* summary file block buffer */ +struct xfs_buf *sumbp =NULL ;/* summary file block buffer */ mp = tp->t_mountp;
libxlog/xfs_log_recover.c +6 -6
@@ -227,7 +227,7 @@ xlog_find_verify_cycle( {xfs_daddr_t i, j;uint cycle; -xfs_buf_t *bp; +struct xfs_buf *bp;int bufblks;char *buf =NULL ;int error =0 ;
@@ -294,7 +294,7 @@ xlog_find_verify_log_record( int extra_bblks) {xfs_daddr_t i; -xfs_buf_t *bp; +struct xfs_buf *bp;char *offset =NULL ;xlog_rec_header_t *head =NULL ;int error =0 ;
@@ -401,7 +401,7 @@ xlog_find_head(struct xlog *log, xfs_daddr_t *return_head_blk) { - xfs_buf_t *bp; + struct xfs_buf *bp;char *offset;xfs_daddr_t new_blk, first_blk, start_blk, last_blk, head_blk;int num_scan_bblks;
@@ -676,7 +676,7 @@ xlog_find_tail(xlog_rec_header_t *rhead;xlog_op_header_t *op_head;char *offset =NULL ; -xfs_buf_t *bp; +struct xfs_buf *bp;int error, i, found;xfs_daddr_t umount_data_blk;xfs_daddr_t after_umount_blk;
@@ -882,7 +882,7 @@ xlog_find_zeroed(struct xlog *log, xfs_daddr_t *blk_no) { - xfs_buf_t *bp; + struct xfs_buf *bp;char *offset;uint first_cycle, last_cycle;xfs_daddr_t new_blk, last_blk, start_blk;
@@ -1419,7 +1419,7 @@ xlog_do_recovery_pass(xlog_rec_header_t *rhead;xfs_daddr_t blk_no;char *offset; -xfs_buf_t *hbp, *dbp; +struct xfs_buf *hbp, *dbp;int error =0 , h_size;int bblks, split_bblks;int hblks, split_hblks, wrapped_hblks;
logprint/log_print_all.c +1 -1
@@ -16,7 +16,7 @@ xlog_print_find_oldest(struct xlog *log, xfs_daddr_t *last_blk) { - xfs_buf_t *bp; + struct xfs_buf *bp;xfs_daddr_t first_blk;uint first_half_cycle, last_half_cycle;int error =0 ;
mkfs/proto.c +1 -1
@@ -225,7 +225,7 @@ newfile( char *buf, int len) { -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_daddr_t d;int error;int flags;
mkfs/xfs_mkfs.c +1 -1
@@ -3737,7 +3737,7 @@ main( char **argv) {xfs_agnumber_t agno; -xfs_buf_t *buf; +struct xfs_buf *buf;int c;char *dfile =NULL ;char *logfile =NULL ;
repair/agheader.c +1 -1
@@ -467,7 +467,7 @@ secondary_sb_whack(*/ int -verify_set_agheader (xfs_mount_t *mp,xfs_buf_t *sbuf,xfs_sb_t *sb, +verify_set_agheader (xfs_mount_t *mp,struct xfs_buf *sbuf,xfs_sb_t *sb,xfs_agf_t *agf,xfs_agi_t *agi,xfs_agnumber_t i) {int rval =0 ;
repair/attr_repair.c +2 -2
@@ -388,7 +388,7 @@ rmtval_get(xfs_mount_t *mp, xfs_ino_t ino, blkmap_t *blkmap, xfs_dablk_t blocknum, int valuelen, char* value) { xfs_fsblock_t bno; -xfs_buf_t *bp; +struct xfs_buf *bp;int clearit =0 , i =0 , length =0 , amountdone =0 ;int hdrsize =0 ;int error;
@@ -730,7 +730,7 @@ process_leaf_attr_level(xfs_mount_t *mp, {int repair;xfs_attr_leafblock_t *leaf; -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_ino_t ino;xfs_fsblock_t dev_bno;xfs_dablk_t da_bno;
repair/da_util.h +1 -1
@@ -8,7 +8,7 @@#define _XR_DA_UTIL_Hstruct da_level_state { -xfs_buf_t *bp;/* block bp */ +struct xfs_buf *bp;/* block bp */ xfs_dablk_t bno;/* file block number */ xfs_dahash_t hashval;/* last verified hashval */ int index;/* current index in block */
repair/dino_chunks.c +4 -4
@@ -30,7 +30,7 @@ check_aginode_block(xfs_mount_t *mp,xfs_dinode_t *dino_p;int i;int cnt =0 ; -xfs_buf_t *bp; +struct xfs_buf *bp;int error;/*
@@ -597,7 +597,7 @@ process_inode_chunk( {xfs_ino_t parent;ino_tree_node_t *ino_rec; -xfs_buf_t **bplist; +struct xfs_buf **bplist;xfs_dinode_t *dino;int icnt;int status;
@@ -644,10 +644,10 @@ process_inode_chunk( ino_rec = first_irec; irec_offset =0 ; - bplist =malloc (cluster_count *sizeof (xfs_buf_t *)); + bplist =malloc (cluster_count *sizeof (struct xfs_buf *));if (bplist ==NULL )do_error (_ ("failed to allocate %zd bytes of memory\n" ), - cluster_count *sizeof (xfs_buf_t *)); + cluster_count *sizeof (struct xfs_buf *));for (bp_index =0 ; bp_index < cluster_count; bp_index++) {/*
repair/incore.h +1 -1
@@ -600,7 +600,7 @@ typedef struct bm_level_state {/* int level; uint64_t prev_last_key; -xfs_buf_t *bp; +struct xfs_buf *bp; xfs_bmbt_block_t *block; */ } bm_level_state_t;
repair/phase5.c +1 -1
@@ -390,7 +390,7 @@ build_agf_agfl(static void sync_sb (xfs_mount_t *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp; bp =libxfs_getsb (mp);if (!bp)
repair/phase6.c +2 -2
@@ -562,7 +562,7 @@ mk_rbmino(xfs_mount_t *mp)static int fill_rbmino (xfs_mount_t *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_trans_t *tp;xfs_inode_t *ip;xfs_rtword_t *bmp;
@@ -630,7 +630,7 @@ _("can't access block %" PRIu64 " (fsbno %" PRIu64 ") of realtime bitmap inode %static int fill_rsumino (xfs_mount_t *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_trans_t *tp;xfs_inode_t *ip;xfs_suminfo_t *smp;
repair/prefetch.c +6 -6
@@ -34,7 +34,7 @@ static int pf_max_fsbs;static int pf_batch_bytes;static int pf_batch_fsbs; -static void pf_read_inode_dirs (prefetch_args_t *,xfs_buf_t *); +static void pf_read_inode_dirs (prefetch_args_t *,struct xfs_buf *);/* * Buffer priorities for the libxfs cache
@@ -271,7 +271,7 @@ pf_scan_lbtree(int isadir, prefetch_args_t *args)) { - xfs_buf_t *bp; + struct xfs_buf *bp;int rc;int error;
@@ -399,7 +399,7 @@ pf_read_exinode(static void pf_read_inode_dirs (prefetch_args_t *args, -xfs_buf_t *bp) +struct xfs_buf *bp) {xfs_dinode_t *dino;int icnt =0 ;
@@ -473,7 +473,7 @@ pf_batch_read(pf_which_t which, void *buf) { - xfs_buf_t *bplist[MAX_BUFS]; + struct xfs_buf *bplist[MAX_BUFS];unsigned int num;off64_t first_off, last_off, next_off;int len, size;
@@ -592,8 +592,8 @@ pf_batch_read(if (len >0 ) {/* -* go through the xfs_buf_t list copying from the -* read buffer into the xfs_buf_t's and release them. +* go through the struct xfs_buf list copying from the +* read buffer into the struct xfs_buf's and release them. */ for (i =0 ; i < num; i++) {
repair/rt.c +2 -2
@@ -163,7 +163,7 @@ process_rtbitmap(xfs_mount_t *mp,int bmbno;int end_bmbno;xfs_fsblock_t bno; -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_rtblock_t extno;int i;int len;
@@ -243,7 +243,7 @@ process_rtsummary(xfs_mount_t *mp, blkmap_t *blkmap) {xfs_fsblock_t bno; -xfs_buf_t *bp; +struct xfs_buf *bp;char *bytes;int sumbno;
repair/scan.c +2 -2
@@ -152,7 +152,7 @@ scan_lbtree(uint64_t magic, const struct xfs_buf_ops *ops) { - xfs_buf_t *bp; + struct xfs_buf *bp;int err;int dirty =0 ;bool badcrc = false;
@@ -2195,7 +2195,7 @@ scan_freelist(xfs_agf_t *agf, struct aghdr_cnts *agcnts) { - xfs_buf_t *agflbuf; + struct xfs_buf *agflbuf;xfs_agnumber_t agno;struct agfl_state state;int error;
repair/xfs_repair.c +1 -1
@@ -718,7 +718,7 @@ main(int argc, char **argv)xfs_mount_t *temp_mp;xfs_mount_t *mp;xfs_dsb_t *dsb; -xfs_buf_t *sbp; +struct xfs_buf *sbp;xfs_mount_t xfs_m;struct xlog log = {0 };char *msgbuf;
Signature · 1 line
2.28.0
Re: [PATCH 06/27] xfsprogs: remove xfs_buf_t typedef
Quoted 6 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Prepare for kernel
xfs_bufalignment by getting rid of thexfs_buf_ttypedef from userspace.Signed-off-by Dave Chinner <[email protected]>
Quoted 15 lines
--- copy/xfs_copy.c | 2 +- include/libxlog.h | 6 +++--- libxfs/init.c | 2 +- libxfs/libxfs_io.h | 7 ++++--- libxfs/libxfs_priv.h | 4 ++-- libxfs/logitem.c | 4 ++-- libxfs/rdwr.c | 26 +++++++++++++------------- libxfs/trans.c | 18 +++++++++--------- libxfs/util.c | 7 +++---- libxfs/xfs_alloc.c | 16 ++++++++-------- libxfs/xfs_bmap.c | 6 +++--- libxfs/xfs_btree.c | 10 +++++----- libxfs/xfs_ialloc.c | 4 ++-- libxfs/xfs_rtbitmap.c | 22 +++++++++++-----------
Hmmm, do you want me to apply this to the kernel ASAP, since we're approaching the end of the merge window and nobody's rebased to 5.10 yet?
--D
Quoted 951 lines
libxlog/xfs_log_recover.c | 12 ++++++------ logprint/log_print_all.c | 2 +- mkfs/proto.c | 2 +- mkfs/xfs_mkfs.c | 2 +- repair/agheader.c | 2 +- repair/attr_repair.c | 4 ++-- repair/da_util.h | 2 +- repair/dino_chunks.c | 8 ++++---- repair/incore.h | 2 +- repair/phase5.c | 2 +- repair/phase6.c | 4 ++-- repair/prefetch.c | 12 ++++++------ repair/rt.c | 4 ++-- repair/scan.c | 4 ++-- repair/xfs_repair.c | 2 +- 29 files changed, 99 insertions(+), 99 deletions(-)copy/xfs_copy.c +1
-1@@ -569,7 +569,7 @@ main(int argc, char **argv)xfs_mount_t *mp;xfs_mount_t mbuf;struct xlog xlog; -xfs_buf_t *sbp; +struct xfs_buf *sbp;xfs_sb_t *sb;xfs_agnumber_t num_ags, agno;xfs_agblock_t bno;include/libxlog.h +3
-3@@ -76,12 +76,12 @@ extern int xlog_is_dirty(struct xfs_mount *, struct xlog *, libxfs_init_t *, int);extern struct xfs_buf *xlog_get_bp (struct xlog *,int );extern int xlog_bread (struct xlog *log,xfs_daddr_t blk_no,int nbblks, -xfs_buf_t *bp,char **offset); +struct xfs_buf *bp,char **offset);extern int xlog_bread_noalign (struct xlog *log,xfs_daddr_t blk_no, -int nbblks,xfs_buf_t *bp); +int nbblks,struct xfs_buf *bp);extern int xlog_find_zeroed (struct xlog *log,xfs_daddr_t *blk_no); -extern int xlog_find_cycle_start (struct xlog *log,xfs_buf_t *bp, +extern int xlog_find_cycle_start (struct xlog *log,struct xfs_buf *bp,xfs_daddr_t first_blk,xfs_daddr_t *last_blk,uint cycle); extern int xlog_find_tail(struct xlog *log, xfs_daddr_t *head_blk,libxfs/init.c +1
-1@@ -30,7 +30,7 @@ char *progname = "libxfs"; /* default, changed by each tool */struct cache *libxfs_bcache;/* global buffer cache */ int libxfs_bhash_size;/* #buckets in bcache */ -int use_xfs_buf_lock;/* global flag: use xfs_buf_t locks for MT */ +int use_xfs_buf_lock;/* global flag: use struct xfs_buf locks for MT */ /* * dev_map - map open devices to fd. libxfs/libxfs_io.h +4
-3@@ -57,7 +57,7 @@ struct xfs_buf_ops {xfs_failaddr_t (*verify_struct)(struct xfs_buf *); }; -typedef struct xfs_buf { +struct xfs_buf {struct cache_node b_node;unsigned int b_flags;xfs_daddr_t b_bn;@@ -78,7 +78,7 @@ typedef struct xfs_buf {struct xfs_buf_map __b_map;int b_nmaps;struct list_head b_list; -} xfs_buf_t; +};bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic);bool xfs_verify_magic16 (struct xfs_buf *bp,__be16 dmagic);@@ -163,7 +163,8 @@ extern int libxfs_bcache_overflowed(void);/* Buffer (Raw) Interfaces */ int libxfs_bwrite (struct xfs_buf *bp); -extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t ,xfs_buf_t *,int ,int ); +extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t , +struct xfs_buf *,int ,int );extern int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int );extern int libxfs_device_zero (struct xfs_buftarg *,xfs_daddr_t ,uint );libxfs/libxfs_priv.h +2
-2@@ -665,10 +665,10 @@ int xfs_rtmodify_range(struct xfs_mount *mp, struct xfs_trans *tp, xfs_rtblock_t start, xfs_extlen_t len, int val);int xfs_rtmodify_summary_int (struct xfs_mount *mp,struct xfs_trans *tp,int log,xfs_rtblock_t bbno,int delta, -xfs_buf_t **rbpp,xfs_fsblock_t *rsb, +struct xfs_buf **rbpp,xfs_fsblock_t *rsb,xfs_suminfo_t *sum);int xfs_rtmodify_summary (struct xfs_mount *mp,struct xfs_trans *tp,int log, -xfs_rtblock_t bbno,int delta,xfs_buf_t **rbpp, +xfs_rtblock_t bbno,int delta,struct xfs_buf **rbpp,xfs_fsblock_t *rsb); int xfs_rtfree_range(struct xfs_mount *mp, struct xfs_trans *tp, xfs_rtblock_t start, xfs_extlen_t len,libxfs/logitem.c +2
-2@@ -27,7 +27,7 @@ kmem_zone_t *xfs_ili_zone; /* inode log item zone */* Check to see if a buffer matching the given parameters is already * a part of the given transaction. */ -xfs_buf_t * +struct xfs_buf * xfs_trans_buf_item_match( xfs_trans_t *tp, struct xfs_buftarg *btp,@@ -68,7 +68,7 @@ xfs_trans_buf_item_match(*/ void xfs_buf_item_init ( -xfs_buf_t *bp, +struct xfs_buf *bp,xfs_mount_t *mp) {xfs_log_item_t *lip;libxfs/rdwr.c +13
-13@@ -223,7 +223,7 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key) }static void -__initbuf (xfs_buf_t *bp,struct xfs_buftarg *btp,xfs_daddr_t bno, +__initbuf (struct xfs_buf *bp,struct xfs_buftarg *btp,xfs_daddr_t bno,unsigned int bytes) { bp->b_flags =0 ;@@ -257,14 +257,14 @@ __initbuf(xfs_buf_t *bp, struct xfs_buftarg *btp, xfs_daddr_t bno, }static void -libxfs_initbuf (xfs_buf_t *bp,struct xfs_buftarg *btp,xfs_daddr_t bno, +libxfs_initbuf (struct xfs_buf *bp,struct xfs_buftarg *btp,xfs_daddr_t bno,unsigned int bytes) {__initbuf (bp, btp, bno, bytes); }static void -libxfs_initbuf_map (xfs_buf_t *bp,struct xfs_buftarg *btp, +libxfs_initbuf_map (struct xfs_buf *bp,struct xfs_buftarg *btp,struct xfs_buf_map *map,int nmaps) {unsigned int bytes =0 ;@@ -292,10 +292,10 @@ libxfs_initbuf_map(xfs_buf_t *bp, struct xfs_buftarg *btp, bp->b_flags |=LIBXFS_B_DISCONTIG ; } -static xfs_buf_t * +static struct xfs_buf *__libxfs_getbufr (int blen) { -xfs_buf_t *bp; +struct xfs_buf *bp;/* * first look for a buffer that can be used as-is, @@ -313,7 +313,7 @@ __libxfs_getbufr(int blen) }if (&bp->b_node.cn_mru == &xfs_buf_freelist.cm_list) { bp =list_entry (xfs_buf_freelist.cm_list.next, - xfs_buf_t, b_node.cn_mru); + struct xfs_buf, b_node.cn_mru);list_del_init (&bp->b_node.cn_mru);free (bp->b_addr); bp->b_addr =NULL ;@@ -331,10 +331,10 @@ __libxfs_getbufr(int blen)return bp; } -static xfs_buf_t * +static struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen) { -xfs_buf_t *bp; +struct xfs_buf *bp;int blen =BBTOB (bblen); bp =__libxfs_getbufr (blen);@@ -343,11 +343,11 @@ libxfs_getbufr(struct xfs_buftarg *btp, xfs_daddr_t blkno, int bblen)return bp; } -static xfs_buf_t * +static struct xfs_buf *libxfs_getbufr_map (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen,struct xfs_buf_map *map,int nmaps) { -xfs_buf_t *bp; +struct xfs_buf *bp;int blen =BBTOB (bblen);if (!map || !nmaps) {@@ -574,7 +574,7 @@ __read_buf(int fd, void *buf, int len, off64_t offset, int flags) }int -libxfs_readbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,xfs_buf_t *bp, +libxfs_readbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,struct xfs_buf *bp,int len,int flags) {int fd =libxfs_device_to_fd (btp->bt_bdev);@@ -915,7 +915,7 @@ libxfs_bulkrelse(struct cache *cache, struct list_head *list) { - xfs_buf_t *bp; + struct xfs_buf *bp;int count =0 ;if (list_empty (list))@@ -941,7 +941,7 @@ voidlibxfs_bcache_free (void) {struct list_head *cm_list; -xfs_buf_t *bp, *next; +struct xfs_buf *bp, *next; cm_list = &xfs_buf_freelist.cm_list;list_for_each_entry_safe (bp, next, cm_list, b_node.cn_mru) {libxfs/trans.c +9
-9@@ -395,7 +395,7 @@ libxfs_trans_bjoin(void libxfs_trans_bhold_release (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;@@ -461,12 +461,12 @@ libxfs_trans_get_buf_map(return 0 ; } -xfs_buf_t * +struct xfs_buf *libxfs_trans_getsb (xfs_trans_t *tp,struct xfs_mount *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp;struct xfs_buf_log_item *bip;int len =XFS_FSS_TO_BB (mp,1 );DEFINE_SINGLE_BUF_MAP (map,XFS_SB_DADDR , len);@@ -604,7 +604,7 @@ libxfs_trans_brelse(void libxfs_trans_bhold (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;@@ -661,7 +661,7 @@ libxfs_trans_log_buf(void libxfs_trans_binval (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;@@ -695,7 +695,7 @@ libxfs_trans_binval(void libxfs_trans_inode_alloc_buf (xfs_trans_t *tp, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_buf_log_item *bip = bp->b_log_item;@@ -799,7 +799,7 @@ static voidinode_item_done ( struct xfs_inode_log_item *iip) { -xfs_buf_t *bp; +struct xfs_buf *bp;int error;ASSERT (iip->ili_inode !=NULL );@@ -835,7 +835,7 @@ static voidbuf_item_done ( xfs_buf_log_item_t *bip) { -xfs_buf_t *bp; +struct xfs_buf *bp;int hold;extern kmem_zone_t *xfs_buf_item_zone;@@ -879,7 +879,7 @@ static voidbuf_item_unlock ( xfs_buf_log_item_t *bip) { -xfs_buf_t *bp = bip->bli_buf; +struct xfs_buf *bp = bip->bli_buf;uint hold;/* Clear the buffer's association with this transaction. */ libxfs/util.c +3
-4@@ -255,7 +255,7 @@ libxfs_ialloc(xfs_dev_t rdev, struct cred *cr, struct fsxattr *fsx, - xfs_buf_t **ialloc_context, + struct xfs_buf **ialloc_context, xfs_inode_t **ipp) { xfs_ino_t ino;@@ -358,7 +358,7 @@ libxfs_ialloc(int libxfs_iflush_int (xfs_inode_t *ip, -xfs_buf_t *bp) +struct xfs_buf *bp) {struct xfs_inode_log_item *iip;xfs_dinode_t *dip;@@ -540,11 +540,10 @@ libxfs_inode_alloc(struct fsxattr *fsx, xfs_inode_t **ipp) { - xfs_buf_t *ialloc_context; + struct xfs_buf *ialloc_context = NULL;xfs_inode_t *ip;int error; - ialloc_context = (xfs_buf_t *)0 ; error =libxfs_ialloc (*tp, pip, mode, nlink, rdev, cr, fsx, &ialloc_context, &ip);if (error) {libxfs/xfs_alloc.c +8
-8@@ -686,9 +686,9 @@ xfs_alloc_read_agfl( xfs_mount_t *mp, /* mount point structure */ xfs_trans_t *tp, /* transaction pointer */ xfs_agnumber_t agno, /* allocation group number */ - xfs_buf_t **bpp) /* buffer for the ag free block array */ + struct xfs_buf **bpp) /* buffer for the ag free block array */ { -xfs_buf_t *bp;/* return value */ +struct xfs_buf *bp;/* return value */ int error;ASSERT (agno !=NULLAGNUMBER );@@ -2642,12 +2642,12 @@ out_no_agbp:int /* error */ xfs_alloc_get_freelist (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *agbp,/* buffer containing the agf structure */ +struct xfs_buf *agbp,/* buffer containing the agf structure */ xfs_agblock_t *bnop,/* block address retrieved from freelist */ int btreeblk)/* destination is a AGF btree */ {struct xfs_agf *agf = agbp->b_addr; -xfs_buf_t *agflbp;/* buffer for a.g. freelist structure */ +struct xfs_buf *agflbp;/* buffer for a.g. freelist structure */ xfs_agblock_t bno;/* block number returned */ __be32 *agfl_bno;int error;@@ -2706,7 +2706,7 @@ xfs_alloc_get_freelist(void xfs_alloc_log_agf (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *bp,/* buffer for a.g. freelist header */ +struct xfs_buf *bp,/* buffer for a.g. freelist header */ int fields)/* mask of fields to be logged (XFS_AGF_...) */ {int first;/* first byte offset */ @@ -2752,7 +2752,7 @@ xfs_alloc_pagf_init( xfs_agnumber_t agno, /* allocation group number */ int flags) /* XFS_ALLOC_FLAGS_... */ { -xfs_buf_t *bp; +struct xfs_buf *bp;int error; error =xfs_alloc_read_agf (mp, tp, agno, flags, &bp);@@ -2767,8 +2767,8 @@ xfs_alloc_pagf_init(int /* error */ xfs_alloc_put_freelist (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *agbp,/* buffer for a.g. freelist header */ -xfs_buf_t *agflbp,/* buffer for a.g. free block array */ +struct xfs_buf *agbp,/* buffer for a.g. freelist header */ +struct xfs_buf *agflbp,/* buffer for a.g. free block array */ xfs_agblock_t bno,/* block being freed */ int btreeblk)/* block came from a AGF btree */ {libxfs/xfs_bmap.c +3
-3@@ -314,7 +314,7 @@ xfs_bmap_check_leaf_extents(struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);struct xfs_btree_block *block;/* current btree block */ xfs_fsblock_t bno;/* block # of "block" */ -xfs_buf_t *bp;/* buffer for "block" */ +struct xfs_buf *bp;/* buffer for "block" */ int error;/* error return value */ xfs_extnum_t i=0 , j;/* index into the extents list */ int level;/* btree level, for checking */ @@ -585,7 +585,7 @@ xfs_bmap_btree_to_extents(struct xfs_btree_block *rblock = ifp->if_broot;struct xfs_btree_block *cblock;/* child btree block */ xfs_fsblock_t cbno;/* child block number */ -xfs_buf_t *cbp;/* child block's buffer */ +struct xfs_buf *cbp;/* child block's buffer */ int error;/* error return value */ __be64 *pp;/* ptr to block address */ struct xfs_owner_info oinfo;@@ -823,7 +823,7 @@ xfs_bmap_local_to_extents(int flags;/* logging flags returned */ struct xfs_ifork *ifp;/* inode fork pointer */ xfs_alloc_arg_t args;/* allocation arguments */ -xfs_buf_t *bp;/* buffer for extent block */ +struct xfs_buf *bp;/* buffer for extent block */ struct xfs_bmbt_irec rec;struct xfs_iext_cursor icur;libxfs/xfs_btree.c +5
-5@@ -394,7 +394,7 @@ xfs_btree_dup_cursor( xfs_btree_cur_t *cur, /* input cursor */ xfs_btree_cur_t **ncur) /* output cursor */ { -xfs_buf_t *bp;/* btree block's buffer pointer */ +struct xfs_buf *bp;/* btree block's buffer pointer */ int error;/* error return value */ int i;/* level number of btree block */ xfs_mount_t *mp;/* mount structure for filesystem */ @@ -698,7 +698,7 @@ xfs_btree_firstrec( int level) /* level to change */ {struct xfs_btree_block *block;/* generic btree block pointer */ -xfs_buf_t *bp;/* buffer containing block */ +struct xfs_buf *bp;/* buffer containing block */ /* * Get the block pointer for this level. @@ -728,7 +728,7 @@ xfs_btree_lastrec( int level) /* level to change */ {struct xfs_btree_block *block;/* generic btree block pointer */ -xfs_buf_t *bp;/* buffer containing block */ +struct xfs_buf *bp;/* buffer containing block */ /* * Get the block pointer for this level. @@ -990,7 +990,7 @@ STATIC voidxfs_btree_setbuf ( xfs_btree_cur_t *cur,/* btree cursor */ int lev, /* level in btree */ - xfs_buf_t *bp) /* new buffer to set */ + struct xfs_buf *bp) /* new buffer to set */ {struct xfs_btree_block *b;/* btree block */ @@ -1633,7 +1633,7 @@ xfs_btree_decrement( int *stat) /* success/failure */ {struct xfs_btree_block *block; -xfs_buf_t *bp; +struct xfs_buf *bp;int error;/* error return value */ int lev;union xfs_btree_ptr ptr;libxfs/xfs_ialloc.c +2
-2@@ -2448,7 +2448,7 @@ out_map:void xfs_ialloc_log_agi (xfs_trans_t *tp,/* transaction pointer */ -xfs_buf_t *bp,/* allocation group header buffer */ +struct xfs_buf *bp,/* allocation group header buffer */ int fields)/* bitmask of fields to log */ {int first;/* first byte number */ @@ -2668,7 +2668,7 @@ xfs_ialloc_pagi_init( xfs_trans_t *tp, /* transaction pointer */ xfs_agnumber_t agno) /* allocation group number */ { -xfs_buf_t *bp =NULL ; +struct xfs_buf *bp =NULL ;int error; error =xfs_ialloc_read_agi (mp, tp, agno, &bp);libxfs/xfs_rtbitmap.c +11
-11@@ -54,9 +54,9 @@ xfs_rtbuf_get(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t block, /* block number in bitmap or summary */ int issum, /* is summary not bitmap */ - xfs_buf_t **bpp) /* output: buffer for the block */ + struct xfs_buf **bpp) /* output: buffer for the block */ { - xfs_buf_t *bp; /* block buffer, result */ + struct xfs_buf *bp; /* block buffer, result */xfs_inode_t *ip;/* bitmap or summary inode */ xfs_bmbt_irec_t map;int nmap =1 ;@@ -99,7 +99,7 @@ xfs_rtfind_back(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtblock_t firstbit;/* first useful bit in the word */ @@ -274,7 +274,7 @@ xfs_rtfind_forw(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtblock_t i;/* current bit number rel. to start */ @@ -445,11 +445,11 @@ xfs_rtmodify_summary_int(int log,/* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */ - xfs_buf_t **rbpp, /* in/out: summary block buffer */ + struct xfs_buf **rbpp, /* in/out: summary block buffer */ xfs_fsblock_t *rsb, /* in/out: summary block number */ xfs_suminfo_t *sum) /* out: summary info for this block */ { - xfs_buf_t *bp; /* buffer for the summary block */ + struct xfs_buf *bp; /* buffer for the summary block */int error;/* error value */ xfs_fsblock_t sb;/* summary fsblock */ int so;/* index into the summary file */ @@ -515,7 +515,7 @@ xfs_rtmodify_summary( int log, /* log2 of extent size */ xfs_rtblock_t bbno, /* bitmap block number */ int delta, /* change to make to summary info */ - xfs_buf_t **rbpp, /* in/out: summary block buffer */ + struct xfs_buf **rbpp, /* in/out: summary block buffer */ xfs_fsblock_t *rsb) /* in/out: summary block number */ { return xfs_rtmodify_summary_int(mp, tp, log, bbno,@@ -537,7 +537,7 @@ xfs_rtmodify_range(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtword_t *first;/* first used word in the buffer */ @@ -688,7 +688,7 @@ xfs_rtfree_range(xfs_trans_t *tp,/* transaction pointer */ xfs_rtblock_t start, /* starting block to free */ xfs_extlen_t len, /* length to free */ - xfs_buf_t **rbpp, /* in/out: summary block buffer */ + struct xfs_buf **rbpp, /* in/out: summary block buffer */ xfs_fsblock_t *rsb) /* in/out: summary block number */ { xfs_rtblock_t end; /* end of the freed extent */@@ -771,7 +771,7 @@ xfs_rtcheck_range(xfs_rtword_t *b;/* current word in buffer */ int bit;/* bit number in the word */ xfs_rtblock_t block;/* bitmap block number */ -xfs_buf_t *bp;/* buf for the block */ +struct xfs_buf *bp;/* buf for the block */ xfs_rtword_t *bufp;/* starting word in buffer */ int error;/* error value */ xfs_rtblock_t i;/* current bit number rel. to start */ @@ -967,7 +967,7 @@ xfs_rtfree_extent(int error;/* error value */ xfs_mount_t *mp;/* file system mount structure */ xfs_fsblock_t sb;/* summary file block number */ -xfs_buf_t *sumbp =NULL ;/* summary file block buffer */ +struct xfs_buf *sumbp =NULL ;/* summary file block buffer */ mp = tp->t_mountp;libxlog/xfs_log_recover.c +6
-6@@ -227,7 +227,7 @@ xlog_find_verify_cycle( {xfs_daddr_t i, j;uint cycle; -xfs_buf_t *bp; +struct xfs_buf *bp;int bufblks;char *buf =NULL ;int error =0 ;@@ -294,7 +294,7 @@ xlog_find_verify_log_record( int extra_bblks) {xfs_daddr_t i; -xfs_buf_t *bp; +struct xfs_buf *bp;char *offset =NULL ;xlog_rec_header_t *head =NULL ;int error =0 ;@@ -401,7 +401,7 @@ xlog_find_head(struct xlog *log, xfs_daddr_t *return_head_blk) { - xfs_buf_t *bp; + struct xfs_buf *bp;char *offset;xfs_daddr_t new_blk, first_blk, start_blk, last_blk, head_blk;int num_scan_bblks;@@ -676,7 +676,7 @@ xlog_find_tail(xlog_rec_header_t *rhead;xlog_op_header_t *op_head;char *offset =NULL ; -xfs_buf_t *bp; +struct xfs_buf *bp;int error, i, found;xfs_daddr_t umount_data_blk;xfs_daddr_t after_umount_blk;@@ -882,7 +882,7 @@ xlog_find_zeroed(struct xlog *log, xfs_daddr_t *blk_no) { - xfs_buf_t *bp; + struct xfs_buf *bp;char *offset;uint first_cycle, last_cycle;xfs_daddr_t new_blk, last_blk, start_blk;@@ -1419,7 +1419,7 @@ xlog_do_recovery_pass(xlog_rec_header_t *rhead;xfs_daddr_t blk_no;char *offset; -xfs_buf_t *hbp, *dbp; +struct xfs_buf *hbp, *dbp;int error =0 , h_size;int bblks, split_bblks;int hblks, split_hblks, wrapped_hblks;logprint/log_print_all.c +1
-1@@ -16,7 +16,7 @@ xlog_print_find_oldest(struct xlog *log, xfs_daddr_t *last_blk) { - xfs_buf_t *bp; + struct xfs_buf *bp;xfs_daddr_t first_blk;uint first_half_cycle, last_half_cycle;int error =0 ;mkfs/proto.c +1
-1@@ -225,7 +225,7 @@ newfile( char *buf, int len) { -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_daddr_t d;int error;int flags;mkfs/xfs_mkfs.c +1
-1@@ -3737,7 +3737,7 @@ main( char **argv) {xfs_agnumber_t agno; -xfs_buf_t *buf; +struct xfs_buf *buf;int c;char *dfile =NULL ;char *logfile =NULL ;repair/agheader.c +1
-1@@ -467,7 +467,7 @@ secondary_sb_whack(*/ int -verify_set_agheader (xfs_mount_t *mp,xfs_buf_t *sbuf,xfs_sb_t *sb, +verify_set_agheader (xfs_mount_t *mp,struct xfs_buf *sbuf,xfs_sb_t *sb,xfs_agf_t *agf,xfs_agi_t *agi,xfs_agnumber_t i) {int rval =0 ;repair/attr_repair.c +2
-2@@ -388,7 +388,7 @@ rmtval_get(xfs_mount_t *mp, xfs_ino_t ino, blkmap_t *blkmap, xfs_dablk_t blocknum, int valuelen, char* value) { xfs_fsblock_t bno; -xfs_buf_t *bp; +struct xfs_buf *bp;int clearit =0 , i =0 , length =0 , amountdone =0 ;int hdrsize =0 ;int error;@@ -730,7 +730,7 @@ process_leaf_attr_level(xfs_mount_t *mp, {int repair;xfs_attr_leafblock_t *leaf; -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_ino_t ino;xfs_fsblock_t dev_bno;xfs_dablk_t da_bno;repair/da_util.h +1
-1@@ -8,7 +8,7 @@#define _XR_DA_UTIL_Hstruct da_level_state { -xfs_buf_t *bp;/* block bp */ +struct xfs_buf *bp;/* block bp */ xfs_dablk_t bno;/* file block number */ xfs_dahash_t hashval;/* last verified hashval */ int index;/* current index in block */ repair/dino_chunks.c +4
-4@@ -30,7 +30,7 @@ check_aginode_block(xfs_mount_t *mp,xfs_dinode_t *dino_p;int i;int cnt =0 ; -xfs_buf_t *bp; +struct xfs_buf *bp;int error;/* @@ -597,7 +597,7 @@ process_inode_chunk( {xfs_ino_t parent;ino_tree_node_t *ino_rec; -xfs_buf_t **bplist; +struct xfs_buf **bplist;xfs_dinode_t *dino;int icnt;int status;@@ -644,10 +644,10 @@ process_inode_chunk( ino_rec = first_irec; irec_offset =0 ; - bplist =malloc (cluster_count *sizeof (xfs_buf_t *)); + bplist =malloc (cluster_count *sizeof (struct xfs_buf *));if (bplist ==NULL )do_error (_ ("failed to allocate %zd bytes of memory\n" ), - cluster_count *sizeof (xfs_buf_t *)); + cluster_count *sizeof (struct xfs_buf *));for (bp_index =0 ; bp_index < cluster_count; bp_index++) {/* repair/incore.h +1
-1@@ -600,7 +600,7 @@ typedef struct bm_level_state {/* int level; uint64_t prev_last_key; -xfs_buf_t *bp; +struct xfs_buf *bp; xfs_bmbt_block_t *block; */ } bm_level_state_t;repair/phase5.c +1
-1@@ -390,7 +390,7 @@ build_agf_agfl(static void sync_sb (xfs_mount_t *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp; bp =libxfs_getsb (mp);if (!bp)repair/phase6.c +2
-2@@ -562,7 +562,7 @@ mk_rbmino(xfs_mount_t *mp)static int fill_rbmino (xfs_mount_t *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_trans_t *tp;xfs_inode_t *ip;xfs_rtword_t *bmp;@@ -630,7 +630,7 @@ _("can't access block %" PRIu64 " (fsbno %" PRIu64 ") of realtime bitmap inode %static int fill_rsumino (xfs_mount_t *mp) { -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_trans_t *tp;xfs_inode_t *ip;xfs_suminfo_t *smp;repair/prefetch.c +6
-6@@ -34,7 +34,7 @@ static int pf_max_fsbs;static int pf_batch_bytes;static int pf_batch_fsbs; -static void pf_read_inode_dirs (prefetch_args_t *,xfs_buf_t *); +static void pf_read_inode_dirs (prefetch_args_t *,struct xfs_buf *);/* * Buffer priorities for the libxfs cache @@ -271,7 +271,7 @@ pf_scan_lbtree(int isadir, prefetch_args_t *args)) { - xfs_buf_t *bp; + struct xfs_buf *bp;int rc;int error;@@ -399,7 +399,7 @@ pf_read_exinode(static void pf_read_inode_dirs (prefetch_args_t *args, -xfs_buf_t *bp) +struct xfs_buf *bp) {xfs_dinode_t *dino;int icnt =0 ;@@ -473,7 +473,7 @@ pf_batch_read(pf_which_t which, void *buf) { - xfs_buf_t *bplist[MAX_BUFS]; + struct xfs_buf *bplist[MAX_BUFS];unsigned int num;off64_t first_off, last_off, next_off;int len, size;@@ -592,8 +592,8 @@ pf_batch_read(if (len >0 ) {/* -* go through the xfs_buf_t list copying from the -* read buffer into the xfs_buf_t's and release them. +* go through the struct xfs_buf list copying from the +* read buffer into the struct xfs_buf's and release them. */ for (i =0 ; i < num; i++) {repair/rt.c +2
-2@@ -163,7 +163,7 @@ process_rtbitmap(xfs_mount_t *mp,int bmbno;int end_bmbno;xfs_fsblock_t bno; -xfs_buf_t *bp; +struct xfs_buf *bp;xfs_rtblock_t extno;int i;int len;@@ -243,7 +243,7 @@ process_rtsummary(xfs_mount_t *mp, blkmap_t *blkmap) {xfs_fsblock_t bno; -xfs_buf_t *bp; +struct xfs_buf *bp;char *bytes;int sumbno;repair/scan.c +2
-2@@ -152,7 +152,7 @@ scan_lbtree(uint64_t magic, const struct xfs_buf_ops *ops) { - xfs_buf_t *bp; + struct xfs_buf *bp;int err;int dirty =0 ;bool badcrc = false;@@ -2195,7 +2195,7 @@ scan_freelist(xfs_agf_t *agf, struct aghdr_cnts *agcnts) { - xfs_buf_t *agflbuf; + struct xfs_buf *agflbuf;xfs_agnumber_t agno;struct agfl_state state;int error;repair/xfs_repair.c +1
-1@@ -718,7 +718,7 @@ main(int argc, char **argv)xfs_mount_t *temp_mp;xfs_mount_t *mp;xfs_dsb_t *dsb; -xfs_buf_t *sbp; +struct xfs_buf *sbp;xfs_mount_t xfs_m;struct xlog log = {0 };char *msgbuf;Signature · 1 line
2.28.0
Re: [PATCH 06/27] xfsprogs: remove xfs_buf_t typedef
Quoted 23 lines · Darrick J. Wong
Quoted 6 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Prepare for kernel
xfs_bufalignment by getting rid of thexfs_buf_ttypedef from userspace.Signed-off-by Dave Chinner <[email protected]>Quoted 15 lines
--- copy/xfs_copy.c | 2 +- include/libxlog.h | 6 +++--- libxfs/init.c | 2 +- libxfs/libxfs_io.h | 7 ++++--- libxfs/libxfs_priv.h | 4 ++-- libxfs/logitem.c | 4 ++-- libxfs/rdwr.c | 26 +++++++++++++------------- libxfs/trans.c | 18 +++++++++--------- libxfs/util.c | 7 +++---- libxfs/xfs_alloc.c | 16 ++++++++-------- libxfs/xfs_bmap.c | 6 +++--- libxfs/xfs_btree.c | 10 +++++----- libxfs/xfs_ialloc.c | 4 ++-- libxfs/xfs_rtbitmap.c | 22 +++++++++++-----------
Quoted 3 lines
Hmmm, do you want me to apply this to the kernel ASAP, since we're approaching the end of the merge window and nobody's rebased to 5.10 yet?
Sure, makes sense to do this before everyone rebases. It's not a huge change, either.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 7/27 xfsprogs: introduce liburcu support
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The upcoming buffer cache rework/kerenl sync-up requires atomic variables. I could use C++11 atomics build into GCC, but they are a pain to work with and shoe-horn into the kernel atomic variable API.
Much easier is to introduce a dependency on liburcu - the userspace RCU library. This provides atomic variables that very closely match the kernel atomic variable API, and it provides a very similar memory model and memory barrier support to the kernel. And we get RCU support that has an identical interface to the kernel and works the same way.
Hence kernel code written with RCU algorithms and atomic variables will just slot straight into the userspace xfsprogs code without us having to think about whether the lockless algorithms will work in userspace or not. This reduces glue and hoop jumping, and gets us a step closer to having the entire userspace libxfs code MT safe.
--- configure.ac | 3 +++ copy/Makefile | 3 ++- copy/xfs_copy.c | 3 +++ db/Makefile | 3 ++- debian/control | 2 +- growfs/Makefile | 3 ++- include/builddefs.in | 4 +++- include/platform_defs.h.in | 1 + libfrog/workqueue.c | 3 +++ libxfs/init.c | 3 +++ libxfs/libxfs_priv.h | 3 +-- logprint/Makefile | 3 ++- m4/Makefile | 1 + m4/package_urcu.m4 | 22 ++++++++++++++++++++++ mdrestore/Makefile | 3 ++- mkfs/Makefile | 2 +- repair/Makefile | 2 +- repair/prefetch.c | 9 +++++++-- repair/progress.c | 4 +++- scrub/Makefile | 3 ++- scrub/progress.c | 2 ++ 21 files changed, 67 insertions(+), 15 deletions(-) create mode 100644 m4/package_urcu.m4
configure.ac +3 -0
@@ -154,6 +154,9 @@ AC_PACKAGE_NEED_UUIDCOMPARE AC_PACKAGE_NEED_PTHREAD_H AC_PACKAGE_NEED_PTHREADMUTEXINIT +AC_PACKAGE_NEED_URCU_H +AC_PACKAGE_NEED_RCU_INIT + AC_HAVE_FADVISE AC_HAVE_MADVISE AC_HAVE_MINCORE
copy/Makefile +2 -1
@@ -9,7 +9,8 @@ LTCOMMAND = xfs_copy CFILES = xfs_copy.c HFILES = xfs_copy.h -LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBPTHREAD) $(LIBRT) +LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBPTHREAD) $(LIBRT) \ + $(LIBURCU) LTDEPENDENCIES = $(LIBXFS) $(LIBXLOG) $(LIBFROG) LLDFLAGS = -static-libtool-libs
copy/xfs_copy.c +3 -0
@@ -110,6 +110,7 @@ do_message(int flags, int code, const char *fmt, ...)fprintf (stderr,_ ("Aborting XFS copy -- logfile error -- reason: %s\n" ),strerror (errno)); +rcu_unregister_thread ();pthread_exit (NULL ); } }
@@ -224,6 +225,7 @@ begin_reader(void *arg) {thread_args *args = arg; +rcu_register_thread ();for (;;) {pthread_mutex_lock (&args->wait);if (do_write (args,NULL ))
@@ -243,6 +245,7 @@ handle_error:if (--glob_masks.num_working ==0 )pthread_mutex_unlock (&mainwait);pthread_mutex_unlock (&glob_masks.mutex); +rcu_unregister_thread ();pthread_exit (NULL );return NULL ; }
db/Makefile +2 -1
@@ -17,7 +17,8 @@ HFILES = addr.h agf.h agfl.h agi.h attr.h attrshort.h bit.h block.h bmap.h \ CFILES = $(HFILES:.h=.c) btdump.c btheight.c convert.c info.c LSRCFILES = xfs_admin.sh xfs_ncheck.sh xfs_metadump.sh -LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD) +LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD) \ + $(LIBURCU) LTDEPENDENCIES = $(LIBXFS) $(LIBXLOG) $(LIBFROG) LLDFLAGS += -static-libtool-libs
debian/control +1 -1
@@ -3,7 +3,7 @@ Section: admin Priority: optional Maintainer: XFS Development Team <[email protected]> Uploaders: Nathan Scott <[email protected]>, Anibal Monsalve Salazar <[email protected]> -Build-Depends: uuid-dev, dh-autoreconf, debhelper (>= 5), gettext, libtool, libedit-dev, libblkid-dev (>= 2.17), linux-libc-dev, libdevmapper-dev, libattr1-dev, libicu-dev, dh-python, pkg-config +Build-Depends: uuid-dev, dh-autoreconf, debhelper (>= 5), gettext, libtool, libedit-dev, libblkid-dev (>= 2.17), linux-libc-dev, libdevmapper-dev, libattr1-dev, libicu-dev, dh-python, pkg-config, liburcu-dev Standards-Version: 4.0.0 Homepage: https://xfs.wiki.kernel.org/
growfs/Makefile +2 -1
@@ -9,7 +9,8 @@ LTCOMMAND = xfs_growfs CFILES = xfs_growfs.c -LLDLIBS = $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD) +LLDLIBS = $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD) \ + $(LIBURCU) ifeq ($(ENABLE_EDITLINE),yes) LLDLIBS += $(LIBEDITLINE) $(LIBTERMCAP)
include/builddefs.in +3 -1
@@ -22,6 +22,7 @@ LDFLAGS = LIBRT = @librt@ LIBUUID = @libuuid@ +LIBURCU = @liburcu@ LIBPTHREAD = @libpthread@ LIBTERMCAP = @libtermcap@ LIBEDITLINE = @libeditline@
@@ -125,7 +126,8 @@ CROND_DIR = @crond_dir@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall # -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl -PCFLAGS = -D_GNU_SOURCE $(GCCFLAGS) +# _LGPL_SOURCE is for liburcu to work correctly with GPL/LGPL programs +PCFLAGS = -D_LGPL_SOURCE -D_GNU_SOURCE $(GCCFLAGS) ifeq ($(HAVE_UMODE_T),yes) PCFLAGS += -DHAVE_UMODE_T endif
include/platform_defs.h.in +1 -0
@@ -23,6 +23,7 @@ #include <limits.h> #include <stdbool.h> #include <libgen.h> +#include <urcu.h> typedef struct filldir filldir_t;
libfrog/workqueue.c +3 -0
@@ -11,6 +11,7 @@#include <stdbool.h> #include <errno.h> #include <assert.h> +#include <urcu.h> #include "workqueue.h" /* Main processing thread */
@@ -24,6 +25,7 @@ workqueue_thread(void *arg)* Loop pulling work from the passed in work queue. * Check for notification to exit after every chunk of work. */ +rcu_register_thread ();while (1 ) {pthread_mutex_lock (&wq->lock);
@@ -52,6 +54,7 @@ workqueue_thread(void *arg) (wi->function)(wi->queue, wi->index, wi->arg);free (wi); } +rcu_unregister_thread ();return NULL ; }
libxfs/init.c +3 -0
@@ -310,6 +310,8 @@ libxfs_init(libxfs_init_t *a) fd =-1 ; flags = (a->isreadonly | a->isdirect); +rcu_init (); +rcu_register_thread ();radix_tree_init ();if (a->volname) {
@@ -957,6 +959,7 @@ libxfs_destroy(libxfs_bcache_free ();cache_destroy (libxfs_bcache); leaked =destroy_zones (); +rcu_unregister_thread ();if (getenv ("LIBXFS_LEAK_CHECK" ) && leaked)exit (1 ); }
libxfs/libxfs_priv.h +1 -2
@@ -194,8 +194,7 @@ enum ce { CE_DEBUG, CE_CONT, CE_NOTE, CE_WARN, CE_ALERT, CE_PANIC };#define spin_unlock (a) ((void) 0)#define likely (x) (x)#define unlikely (x) (x) -#define rcu_read_lock () ((void) 0) -#define rcu_read_unlock () ((void) 0) +/* Need to be able to handle this bare or in control flow */ static inline bool WARN_ON (bool expr) {return (expr);
logprint/Makefile +2 -1
@@ -12,7 +12,8 @@ CFILES = logprint.c \ log_copy.c log_dump.c log_misc.c \ log_print_all.c log_print_trans.c log_redo.c -LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD) +LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBFROG) $(LIBUUID) $(LIBRT) $(LIBPTHREAD) \ + $(LIBURCU) LTDEPENDENCIES = $(LIBXFS) $(LIBXLOG) $(LIBFROG) LLDFLAGS = -static-libtool-libs
m4/Makefile +1 -0
@@ -24,6 +24,7 @@ LSRCFILES = \ package_services.m4 \ package_types.m4 \ package_icu.m4 \ + package_urcu.m4 \ package_utilies.m4 \ package_uuiddev.m4 \ multilib.m4 \
m4/package_urcu.m4 new file +22 -0
@@ -0,0 +1,22 @@ +AC_DEFUN([AC_PACKAGE_NEED_URCU_H], + [ AC_CHECK_HEADERS(urcu.h) + if test $ac_cv_header_urcu_h = no; then + AC_CHECK_HEADERS(urcu.h,, [ + echo + echo 'FATAL ERROR: could not find a valid urcu header.' + exit 1]) + fi + ]) + +AC_DEFUN([AC_PACKAGE_NEED_RCU_INIT], + [ AC_MSG_CHECKING([for liburcu]) + AC_TRY_COMPILE([ +#define _GNU_SOURCE +#include <urcu.h> + ], [ + rcu_init(); + ], liburcu=-lurcu + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(liburcu) + ])
mdrestore/Makefile +2 -1
@@ -8,7 +8,8 @@ include $(TOPDIR)/include/builddefs LTCOMMAND = xfs_mdrestore CFILES = xfs_mdrestore.c -LLDLIBS = $(LIBXFS) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBUUID) +LLDLIBS = $(LIBXFS) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBUUID) \ + $(LIBURCU) LTDEPENDENCIES = $(LIBXFS) $(LIBFROG) LLDFLAGS = -static
mkfs/Makefile +1 -1
@@ -11,7 +11,7 @@ HFILES = CFILES = proto.c xfs_mkfs.c LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBBLKID) \ - $(LIBUUID) $(LIBINIH) + $(LIBUUID) $(LIBINIH) $(LIBURCU) LTDEPENDENCIES += $(LIBXFS) $(LIBXCMD) $(LIBFROG) LLDFLAGS = -static-libtool-libs
repair/Makefile +1 -1
@@ -72,7 +72,7 @@ CFILES = \ xfs_repair.c LLDLIBS = $(LIBXFS) $(LIBXLOG) $(LIBXCMD) $(LIBFROG) $(LIBUUID) $(LIBRT) \ - $(LIBPTHREAD) $(LIBBLKID) + $(LIBPTHREAD) $(LIBBLKID) $(LIBURCU) LTDEPENDENCIES = $(LIBXFS) $(LIBXLOG) $(LIBXCMD) $(LIBFROG) LLDFLAGS = -static-libtool-libs
repair/prefetch.c +7 -2
@@ -660,6 +660,7 @@ pf_io_worker(if (buf ==NULL )return NULL ; +rcu_register_thread ();pthread_mutex_lock (&args->lock);while (!args->queuing_done || !btree_is_empty (args->io_queue)) {pftrace ("waiting to start prefetch I/O for AG %d" , args->agno);
@@ -682,6 +683,7 @@ pf_io_worker(free (buf);pftrace ("finished prefetch I/O for AG %d" , args->agno); +rcu_unregister_thread ();return NULL ; }
@@ -726,6 +728,8 @@ pf_queuing_worker(struct xfs_ino_geometry *igeo =M_IGEO (mp);unsigned long long cluster_mask; +rcu_register_thread (); + cluster_mask = (1ULL << igeo->inodes_per_cluster) -1 ;for (i =0 ; i <PF_THREAD_COUNT ; i++) {
@@ -739,7 +743,7 @@ pf_queuing_worker( args->io_threads[i] =0 ;if (i ==0 ) {pf_skip_prefetch_thread (args); -return NULL ; + goto out; }/* * since we have at least one I/O thread, use them for
@@ -779,7 +783,6 @@ pf_queuing_worker(* Start processing as well, in case everything so * far was already prefetched and the queue is empty. */ -pf_start_io_workers (args);pf_start_processing (args);sem_wait (&args->ra_count);
@@ -841,6 +844,8 @@ pf_queuing_worker(if (next_args)pf_create_prefetch_thread (next_args); +out: +rcu_unregister_thread ();return NULL ; }
repair/progress.c +3 -1
@@ -182,6 +182,7 @@ progress_rpt_thread (void *p)do_error (_ ("progress_rpt: cannot malloc progress msg buffer\n" )); running =1 ; +rcu_register_thread ();/* * Specify a repeating timer that fires each MSG_INTERVAL seconds.
@@ -286,7 +287,8 @@ progress_rpt_thread (void *p)do_warn (_ ("cannot delete timer\n" ));free (msgbuf); -return (NULL ); +rcu_unregister_thread (); +return NULL ; }int
scrub/Makefile +2 -1
@@ -71,7 +71,8 @@ spacemap.c \ vfs.c \ xfs_scrub.c -LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBPTHREAD) $(LIBICU_LIBS) $(LIBRT) +LLDLIBS += $(LIBHANDLE) $(LIBFROG) $(LIBPTHREAD) $(LIBICU_LIBS) $(LIBRT) \ + $(LIBURCU) LTDEPENDENCIES += $(LIBHANDLE) $(LIBFROG) LLDFLAGS = -static
scrub/progress.c +2 -0
@@ -117,6 +117,7 @@ progress_report_thread(void *arg)struct timespec abstime;int ret; +rcu_register_thread ();pthread_mutex_lock (&pt.lock);while (1 ) {uint64_t progress_val;
@@ -140,6 +141,7 @@ progress_report_thread(void *arg)progress_report (progress_val); }pthread_mutex_unlock (&pt.lock); +rcu_unregister_thread ();return NULL ; }
Signature · 1 line
2.28.0
PATCH 8/27 libxfs: add spinlock_t wrapper
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
These provide the kernel spinlock_t interface, but are *not* spinlocks. Spinlocks cannot be used by general purpose userspace processes due to the fact they cannot control task preemption and scheduling reliability. Hence these are implemented as a pthread_mutex_t, similar to the way the kernel RT build implements spinlock_t as a kernel mutex.
Because the current libxfs spinlock "implementation" just makes spinlocks go away, we have to also add initialisation to spinlocks that libxfs uses that are missing from the userspace implementation.
--- include/Makefile | 1 + include/libxfs.h | 1 + include/spinlock.h | 25 +++++++++++++++++++++++++ include/xfs_inode.h | 1 + include/xfs_mount.h | 2 ++ include/xfs_trans.h | 1 + libxfs/init.c | 4 +++- libxfs/libxfs_priv.h | 4 +--- libxfs/rdwr.c | 2 ++ 9 files changed, 37 insertions(+), 4 deletions(-) create mode 100644 include/spinlock.h
include/Makefile +1 -0
@@ -16,6 +16,7 @@ LIBHFILES = libxfs.h \ kmem.h \ list.h \ parent.h \ + spinlock.h \ xfs_inode.h \ xfs_log_recover.h \ xfs_metadump.h \
include/libxfs.h +1 -0
@@ -18,6 +18,7 @@#include "kmem.h" #include "libfrog/radix-tree.h" #include "atomic.h" +#include "spinlock.h" #include "xfs_types.h" #include "xfs_fs.h"
include/spinlock.h new file +25 -0
@@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2019-20 RedHat, Inc. +* All Rights Reserved. +*/ +#ifndef __LIBXFS_SPINLOCK_H__ +#define __LIBXFS_SPINLOCK_H__ + +/* +* This implements kernel compatible spinlock exclusion semantics. These, +* however, are not spinlocks, as spinlocks cannot be reliably implemented in +* userspace without using realtime scheduling task contexts. Hence this +* interface is implemented with pthread mutexes and so can block, but this is +* no different to the kernel RT build which replaces spinlocks with mutexes. +* Hence we know it works. +*/ + +typedef pthread_mutex_t spinlock_t ; + +#define spin_lock_init (l) pthread_mutex_init(l, NULL) +#define spin_lock (l) pthread_mutex_lock(l) +#define spin_trylock (l) (pthread_mutex_trylock(l) != EBUSY) +#define spin_unlock (l) pthread_mutex_unlock(l) + +#endif /* __LIBXFS_SPINLOCK_H__ */
include/xfs_inode.h +1 -0
@@ -42,6 +42,7 @@ struct inode {struct timespec i_atime;struct timespec i_mtime;struct timespec i_ctime; +spinlock_t i_lock; };static inline uint32_t i_uid_read (struct inode *inode)
include/xfs_mount.h +2 -0
@@ -20,6 +20,7 @@ typedef struct xfs_mount {#define m_icount m_sb.sb_icount#define m_ifree m_sb.sb_ifree#define m_fdblocks m_sb.sb_fdblocks +spinlock_t m_sb_lock;/* * Bitsets of per-fs metadata that have been checked and/or are sick.
@@ -30,6 +31,7 @@ typedef struct xfs_mount {char *m_fsname;/* filesystem name */ int m_bsize;/* fs logical block size */ +spinlock_t m_agirotor_lock;xfs_agnumber_t m_agfrotor;/* last ag where space found */ xfs_agnumber_t m_agirotor;/* last ag dir inode alloced */ xfs_agnumber_t m_maxagi;/* highest inode alloc group */
include/xfs_trans.h +1 -0
@@ -35,6 +35,7 @@ struct xfs_inode_log_item {unsigned int ili_last_fields;/* fields when flushed*/ unsigned int ili_fields;/* fields to be logged */ unsigned int ili_fsync_fields;/* ignored by userspace */ +spinlock_t ili_lock; }; typedef struct xfs_buf_log_item {
libxfs/init.c +3 -1
@@ -683,7 +683,9 @@ libxfs_mount( mp->m_flags = (LIBXFS_MOUNT_32BITINODES |LIBXFS_MOUNT_32BITINOOPT ); mp->m_sb = *sb;INIT_RADIX_TREE (&mp->m_perag_tree,GFP_KERNEL ); - sbp = &(mp->m_sb); + sbp = &mp->m_sb; +spin_lock_init (&mp->m_sb_lock); +spin_lock_init (&mp->m_agirotor_lock);xfs_sb_mount_common (mp, sb);
libxfs/libxfs_priv.h +1 -3
@@ -48,6 +48,7 @@#include "kmem.h" #include "libfrog/radix-tree.h" #include "atomic.h" +#include "spinlock.h" #include "xfs_types.h" #include "xfs_arch.h"
@@ -189,9 +190,6 @@ enum ce { CE_DEBUG, CE_CONT, CE_NOTE, CE_WARN, CE_ALERT, CE_PANIC };#endif /* miscellaneous kernel routines not in user space */ -#define spin_lock_init (a) ((void) 0) -#define spin_lock (a) ((void) 0) -#define spin_unlock (a) ((void) 0)#define likely (x) (x)#define unlikely (x) (x)
libxfs/rdwr.c +2 -0
@@ -1062,6 +1062,8 @@ libxfs_iget( ip->i_ino = ino; ip->i_mount = mp; +spin_lock_init (&VFS_I (ip)->i_lock); + error =xfs_imap (mp, tp, ip->i_ino, &ip->i_imap,0 );if (error) goto out_destroy;
Signature · 1 line
2.28.0
PATCH 9/27 atomic: convert to uatomic
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Now we have liburcu, we can make use of it's atomic variable implementation. It is almost identical to the kernel API - it's just got a "uatomic" prefix. liburcu also provides all the same aomtic variable memory barriers as the kernel, so if we pull memory barrier dependent kernel code across, it will just work with the right barrier wrappers.
This is preparation the addition of more extensive atomic operations the that kernel buffer cache requires to function correctly.
--- include/atomic.h | 60 +++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 52 insertions(+), 8 deletions(-)
include/atomic.h +52 -8
@@ -7,18 +7,62 @@#define __ATOMIC_H__/* -* Warning: These are not really atomic at all. They are wrappers around the -* kernel atomic variable interface. If we do need these variables to be atomic -* (due to multithreading of the code that uses them) we need to add some -* pthreads magic here. +* Atomics are provided by liburcu. +* +* API and guidelines for which operations provide memory barriers is here: +* +* https://github.com/urcu/userspace-rcu/blob/master/doc/uatomic-api.md +* +* Unlike the kernel, the same interface supports 32 and 64 bit atomic integers. */ +#include <urcu/uatomic.h> +#include "spinlock.h" +typedef int32_t atomic_t ;typedef int64_t atomic64_t ; -#define atomic_inc_return (x) (++(*(x))) -#define atomic_dec_return (x) (--(*(x))) +#define atomic_read (a) uatomic_read(a) +#define atomic_set (a, v) uatomic_set(a, v) + +#define atomic_inc_return (a) uatomic_add_return(a, 1) +#define atomic_dec_return (a) uatomic_sub_return(a, 1) + +#define atomic_inc (a) atomic_inc_return(a) +#define atomic_dec (a) atomic_inc_return(a) + +#define atomic_dec_and_test (a) (atomic_dec_return(a) == 0) + +#define cmpxchg (a, o, n) uatomic_cmpxchg(a, o, n); + +static inline bool atomic_add_unless (atomic_t *a,int v,int u) +{ +int r =atomic_read (a); +int n, o; + +do { + o = r; +if (o == u) +break ; + n = o + v; + r =uatomic_cmpxchg (a, o, n); + }while (r != o); + +return o != u; +} + +static inline bool atomic_dec_and_lock (atomic_t *a,spinlock_t *lock) +{ +if (atomic_add_unless (a,-1 ,1 )) +return 0 ; + +spin_lock (lock); +if (atomic_dec_and_test (a)) +return 1 ; +spin_unlock (lock); +return 0 ; +} -#define atomic64_read (x) *(x) -#define atomic64_set (x, v) (*(x) = v) +#define atomic64_read (x) uatomic_read(x) +#define atomic64_set (x, v) uatomic_set(x, v)#endif /* __ATOMIC_H__ */
Signature · 1 line
2.28.0
PATCH 10/27 libxfs: add kernel-compatible completion API
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
This is needed for the kernel buffer cache conversion to be able to wait on IO synchrnously. It is implemented with pthread mutexes and conditional variables.
--- include/Makefile | 1 + include/completion.h | 61 ++++++++++++++++++++++++++++++++++++++++++++ include/libxfs.h | 1 + libxfs/libxfs_priv.h | 1 + 4 files changed, 64 insertions(+) create mode 100644 include/completion.h
include/Makefile +1 -0
@@ -12,6 +12,7 @@ LIBHFILES = libxfs.h \ atomic.h \ bitops.h \ cache.h \ + completion.h \ hlist.h \ kmem.h \ list.h \
include/completion.h new file +61 -0
61 lines not on this page · raw message
include/libxfs.h +1 -0
@@ -19,6 +19,7 @@#include "libfrog/radix-tree.h" #include "atomic.h" #include "spinlock.h" +#include "completion.h" #include "xfs_types.h" #include "xfs_fs.h"
libxfs/libxfs_priv.h +1 -0
@@ -49,6 +49,7 @@#include "libfrog/radix-tree.h" #include "atomic.h" #include "spinlock.h" +#include "completion.h" #include "xfs_types.h" #include "xfs_arch.h"
Signature · 1 line
2.28.0
Re: [PATCH 10/27] libxfs: add kernel-compatible completion API
Quoted 39 lines · Dave Chinner
From: Dave Chinner <[email protected]>
This is needed for the kernel buffer cache conversion to be able to wait on IO synchrnously. It is implemented with pthread mutexes and conditional variables.
Signed-off-by Dave Chinner <[email protected]>--- include/Makefile | 1 + include/completion.h | 61 ++++++++++++++++++++++++++++++++++++++++++++ include/libxfs.h | 1 + libxfs/libxfs_priv.h | 1 + 4 files changed, 64 insertions(+) create mode 100644 include/completion.hinclude/Makefile +1
-0@@ -12,6 +12,7 @@ LIBHFILES = libxfs.h \ atomic.h \ bitops.h \ cache.h \ + completion.h \ hlist.h \ kmem.h \ list.h \include/completion.h new file +6
-0@@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2019 RedHat, Inc. +* All Rights Reserved. +*/ +#ifndef __LIBXFS_COMPLETION_H__
include/completion.h line 16
Quoted 10 lines
include/completion.h new file +10
-0+#define __LIBXFS_COMPLETION_H__ + +/* +* This implements kernel compatible completion semantics. This is slightly +* different to the way pthread conditional variables work in that completions +* can be signalled before the waiter tries to wait on the variable. In the +* pthread case, the completion is ignored and the waiter goes to sleep, whilst +* the kernel will see that the completion has already been completed and so +* will not block. This is handled through the addition of the the @signalled +* flag in the struct completion.
Hmm... do any of the existing pthread_cond_t users need these semantics?
I suspect the ones in scrub/vfs.c might actually be vulnerable to the signal-before-wait race that this completion structure solves.
In any case, seeing as this primitive isn't inherent to the xfs disk format, maybe these new concurrency management things belong in libfrog?
--D
Quoted 72 lines
include/completion.h new file +45
-0+*/ +struct completion { +pthread_mutex_t lock; +pthread_cond_t cond; +bool signalled;/* for kernel completion behaviour */ +int waiters; +}; + +static inline void +init_completion (struct completion *w) +{ +pthread_mutex_init (&w->lock,NULL ); +pthread_cond_init (&w->cond,NULL ); + w->signalled = false; +} + +static inline void +complete (struct completion *w) +{ +pthread_mutex_lock (&w->lock); + w->signalled = true; +pthread_cond_broadcast (&w->cond); +pthread_mutex_unlock (&w->lock); +} + +/* +* Support for mulitple waiters requires that we count the number of waiters +* we have and only clear the signalled variable once all those waiters have +* been woken. +*/ +static inline void +wait_for_completion (struct completion *w) +{ +pthread_mutex_lock (&w->lock); +if (!w->signalled) { + w->waiters++; +pthread_cond_wait (&w->cond, &w->lock); + w->waiters--; + } +if (!w->waiters) + w->signalled = false; +pthread_mutex_unlock (&w->lock); +} + +#endif /* __LIBXFS_COMPLETION_H__ */ include/libxfs.h +1
-0@@ -19,6 +19,7 @@#include "libfrog/radix-tree.h" #include "atomic.h" #include "spinlock.h" +#include "completion.h" #include "xfs_types.h" #include "xfs_fs.h" libxfs/libxfs_priv.h +1
-0@@ -49,6 +49,7 @@#include "libfrog/radix-tree.h" #include "atomic.h" #include "spinlock.h" +#include "completion.h" #include "xfs_types.h" #include "xfs_arch.h" Signature · 1 line
2.28.0
Re: [PATCH 10/27] libxfs: add kernel-compatible completion API
Quoted 44 lines · Darrick J. Wong
Quoted 43 lines · Dave Chinner
From: Dave Chinner <[email protected]>
This is needed for the kernel buffer cache conversion to be able to wait on IO synchrnously. It is implemented with pthread mutexes and conditional variables.
Signed-off-by Dave Chinner <[email protected]>--- include/Makefile | 1 + include/completion.h | 61 ++++++++++++++++++++++++++++++++++++++++++++ include/libxfs.h | 1 + libxfs/libxfs_priv.h | 1 + 4 files changed, 64 insertions(+) create mode 100644 include/completion.hinclude/Makefile +1
-0@@ -12,6 +12,7 @@ LIBHFILES = libxfs.h \ atomic.h \ bitops.h \ cache.h \ + completion.h \ hlist.h \ kmem.h \ list.h \include/completion.h new file +10
-0@@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2019 RedHat, Inc. +* All Rights Reserved. +*/ +#ifndef __LIBXFS_COMPLETION_H__ +#define __LIBXFS_COMPLETION_H__ + +/* +* This implements kernel compatible completion semantics. This is slightly
Quoted 8 lines
include/completion.h line 16
Quoted 6 lines
include/completion.h new file +6
-0+* different to the way pthread conditional variables work in that completions +* can be signalled before the waiter tries to wait on the variable. In the +* pthread case, the completion is ignored and the waiter goes to sleep, whilst +* the kernel will see that the completion has already been completed and so +* will not block. This is handled through the addition of the the @signalled +* flag in the struct completion. Hmm... do any of the existing
pthread_cond_tusers need these semantics?
No idea. I haven't done an audit, and that's outside the scope of what I'm doing here...
Quoted 5 lines
I suspect the ones in
scrub/vfs.cmight actually be vulnerable to the signal-before-wait race that this completion structure solves.In any case, seeing as this primitive isn't inherent to the xfs disk format, maybe these new concurrency management things belong in libfrog?
I've just mirrored all the other kernel wrapper headers by placing them in include/. I don't see much point in moving these to libfrog, because they have no actual built object code that needs to be linked. i.e. they are included globally for everything via libxfs/libxfs_priv.h and include/libxfs.h, and that's all that is needed to use these APIs. If they get more complex and require actual object files to be built, then it's time to reconsider...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 11/27 libxfs: add wrappers for kernel semaphores
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Implemented vi pthread mutexes.
On Linux, fast pthread mutexes don't actaully check which thread owns the lock on unlock, so can be used in situations where the unlock occurs in a different thread to the lock. This is non-portable behaviour, so if other platforms are supported, this may need to be converted to posix semaphores.
--- include/Makefile | 1 + include/libxfs.h | 1 + include/sema.h | 35 +++++++++++++++++++++++++++++++++++ libxfs/libxfs_priv.h | 1 + 4 files changed, 38 insertions(+) create mode 100644 include/sema.h
include/Makefile +1 -0
@@ -17,6 +17,7 @@ LIBHFILES = libxfs.h \ kmem.h \ list.h \ parent.h \ + sema.h \ spinlock.h \ xfs_inode.h \ xfs_log_recover.h \
include/libxfs.h +1 -0
@@ -20,6 +20,7 @@#include "atomic.h" #include "spinlock.h" #include "completion.h" +#include "sema.h" #include "xfs_types.h" #include "xfs_fs.h"
include/sema.h new file +35 -0
@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2019-20 RedHat, Inc. +* All Rights Reserved. +*/ +#ifndef __LIBXFS_SEMA_H__ +#define __LIBXFS_SEMA_H__ + +/* +* This implements kernel compatible semaphore _exclusion_ semantics. It does +* not implement counting semaphore behaviour. +* +* This makes use of the fact that fast pthread mutexes on Linux don't check +* that the unlocker is the same thread that locked the mutex, and hence can be +* unlocked in a different thread safely. +* +* If this needs to be portable or we require counting semaphore behaviour in +* libxfs code, this requires re-implementation based on posix semaphores. +*/ +struct semaphore { +pthread_mutex_t lock; +}; + +#define sema_init (l, nolock) \ +do { \ + pthread_mutex_init(&(l)->lock, NULL); \ + if (!nolock) \ + pthread_mutex_lock(&(l)->lock); \ +} while (0) + +#define down (l) pthread_mutex_lock(&(l)->lock) +#define down_trylock (l) pthread_mutex_trylock(&(l)->lock) +#define up (l) pthread_mutex_unlock(&(l)->lock) + +#endif /* __LIBXFS_SEMA_H__ */
libxfs/libxfs_priv.h +1 -0
@@ -50,6 +50,7 @@#include "atomic.h" #include "spinlock.h" #include "completion.h" +#include "sema.h" #include "xfs_types.h" #include "xfs_arch.h"
Signature · 1 line
2.28.0
PATCH 12/27 xfsprogs: convert use-once buffer reads to uncached IO
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
--- db/init.c | 2 +- libxfs/init.c | 93 ++++++++++++++++++++++++++++++--------------------- 2 files changed, 55 insertions(+), 40 deletions(-)
db/init.c +1 -1
@@ -153,7 +153,7 @@ init(*/ if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) { -int error = -libxfs_initialize_perag_data (mp, sbp->sb_agcount); + error = -libxfs_initialize_perag_data (mp, sbp->sb_agcount);if (error) { fprintf(stderr, _("%s: cannot init perag data (%d). Continuing anyway.\n"),
libxfs/init.c +54 -39
135 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH 12/27] xfsprogs: convert use-once buffer reads to uncached IO
Quoted 13 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Signed-off-by Dave Chinner <[email protected]>--- db/init.c | 2 +- libxfs/init.c | 93 ++++++++++++++++++++++++++++++--------------------- 2 files changed, 55 insertions(+), 40 deletions(-)db/init.c +0
-0
db/init.c init( · line 156
Quoted 5 lines
db/init.c +1
-1@@ -153,7 +153,7 @@ init(*/ if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) { -int error = -libxfs_initialize_perag_data (mp, sbp->sb_agcount); + error = -libxfs_initialize_perag_data (mp, sbp->sb_agcount);
Er... this and the xfs_check_sizes hoisting below don't have anything to do with uncached io conversion...?
Quoted 29 lines
db/init.c +0
-0if (error) { fprintf(stderr, _("%s: cannot init perag data (%d). Continuing anyway.\n"),libxfs/init.c +4
-3@@ -419,7 +419,7 @@ done:*/ static int rtmount_init ( -xfs_mount_t *mp,/* file system mount structure */ +struct xfs_mount *mp,int flags) {struct xfs_buf *bp;/* buffer for last block of subvolume */ @@ -473,8 +473,9 @@ rtmount_init( (unsigned long long) mp->m_sb.sb_rblocks);return -1 ; } - error =libxfs_buf_read (mp->m_rtdev, d -XFS_FSB_TO_BB (mp,1 ), -XFS_FSB_TO_BB (mp,1 ),0 , &bp,NULL ); + error =libxfs_buf_read_uncached (mp->m_rtdev_targp, + d -XFS_FSB_TO_BB (mp,1 ), +XFS_FSB_TO_BB (mp,1 ),0 , &bp,NULL );if (error) {fprintf (stderr,_ ("%s: realtime size check failed\n" ), progname);
libxfs/init.c libxfs_buftarg_init( · line 664
Quoted 7 lines
libxfs/init.c +4
-0@@ -657,6 +658,52 @@ libxfs_buftarg_init( mp->m_rtdev_targp =libxfs_buftarg_alloc (mp, rtdev); } +/* +* Check that the data (and log if separate) is an ok size. +* +* XXX: copied from kernel, needs to be moved to shared code
Ah, because you want to share this function with the kernel.
Hmm... what do you think about putting it in libxfs/xfs_sb.c ?
--D
Quoted 116 lines
libxfs/init.c +46
-36+*/ +STATIC int +xfs_check_sizes ( +struct xfs_mount *mp) +{ +struct xfs_buf *bp; +xfs_daddr_t d; +int error; + + d = (xfs_daddr_t )XFS_FSB_TO_BB (mp, mp->m_sb.sb_dblocks); +if (XFS_BB_TO_FSB (mp, d) != mp->m_sb.sb_dblocks) { +xfs_warn (mp,"filesystem size mismatch detected" ); +return -EFBIG ; + } + error =libxfs_buf_read_uncached (mp->m_ddev_targp, + d -XFS_FSS_TO_BB (mp,1 ), +XFS_FSS_TO_BB (mp,1 ),0 , &bp,NULL ); +if (error) { +xfs_warn (mp,"last sector read failed" ); +return error; + } +libxfs_buf_relse (bp); + +if (mp->m_logdev_targp == mp->m_ddev_targp) +return 0 ; + + d = (xfs_daddr_t )XFS_FSB_TO_BB (mp, mp->m_sb.sb_logblocks); +if (XFS_BB_TO_FSB (mp, d) != mp->m_sb.sb_logblocks) { +xfs_warn (mp,"log size mismatch detected" ); +return -EFBIG ; + } + error =libxfs_buf_read_uncached (mp->m_logdev_targp, + d -XFS_FSB_TO_BB (mp,1 ), +XFS_FSB_TO_BB (mp,1 ),0 , &bp,NULL ); +if (error) { +xfs_warn (mp,"log device read failed" ); +return error; + } +libxfs_buf_relse (bp); +return 0 ; +} +/* * Mount structure initialization, provides a filled-in xfs_mount_t * such that the numerous XFS_* macros can be used. If dev is zero, @@ -673,7 +720,6 @@ libxfs_mount( {struct xfs_buf *bp;struct xfs_sb *sbp; -xfs_daddr_t d;bool debugger = (flags &LIBXFS_MOUNT_DEBUGGER );int error;@@ -704,16 +750,6 @@ libxfs_mount(xfs_rmapbt_compute_maxlevels (mp);xfs_refcountbt_compute_maxlevels (mp); -/* -* Check that the data (and log if separate) are an ok size. -*/ - d = (xfs_daddr_t )XFS_FSB_TO_BB (mp, mp->m_sb.sb_dblocks); -if (XFS_BB_TO_FSB (mp, d) != mp->m_sb.sb_dblocks) { -fprintf (stderr,_ ("%s: size check failed\n" ), progname); -if (!(flags &LIBXFS_MOUNT_DEBUGGER )) -return NULL ; - } -/* * We automatically convert v1 inodes to v2 inodes now, so if * the NLINK bit is not set we can't operate on the filesystem. @@ -755,30 +791,9 @@ libxfs_mount(return mp;/* device size checks must pass unless we're a debugger. */ - error =libxfs_buf_read (mp->m_dev, d -XFS_FSS_TO_BB (mp,1 ), -XFS_FSS_TO_BB (mp,1 ),0 , &bp,NULL ); -if (error) { -fprintf (stderr,_ ("%s: data size check failed\n" ), progname); -if (!debugger) -return NULL ; - }else -libxfs_buf_relse (bp); - -if (mp->m_logdev_targp->bt_bdev && - mp->m_logdev_targp->bt_bdev != mp->m_ddev_targp->bt_bdev) { - d = (xfs_daddr_t )XFS_FSB_TO_BB (mp, mp->m_sb.sb_logblocks); -if (XFS_BB_TO_FSB (mp, d) != mp->m_sb.sb_logblocks || -libxfs_buf_read (mp->m_logdev_targp, - d -XFS_FSB_TO_BB (mp,1 ),XFS_FSB_TO_BB (mp,1 ), -0 , &bp,NULL )) { -fprintf (stderr,_ ("%s: log size checks failed\n" ), - progname); -if (!debugger) -return NULL ; - } -if (bp) -libxfs_buf_relse (bp); - } + error =xfs_check_sizes (mp); +if (error && !debugger) +return NULL ;/* Initialize realtime fields in the mount structure */ if (rtmount_init (mp, flags)) {@@ -795,7 +810,7 @@ libxfs_mount(* read the first one and let the user know to check the geometry. */ if (sbp->sb_agcount >1000000 ) { - error =libxfs_buf_read (mp->m_dev, + error =libxfs_buf_read_uncached (mp->m_ddev_targp,XFS_AG_DADDR (mp, sbp->sb_agcount -1 ,0 ),1 ,0 , &bp,NULL );if (error) {Signature · 1 line
2.28.0
Re: [PATCH 12/27] xfsprogs: convert use-once buffer reads to uncached IO
Quoted 55 lines · Darrick J. Wong
Quoted 13 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Signed-off-by Dave Chinner <[email protected]>--- db/init.c | 2 +- libxfs/init.c | 93 ++++++++++++++++++++++++++++++--------------------- 2 files changed, 55 insertions(+), 40 deletions(-)db/init.c +0
-0db/init.c init( · line 156
Quoted 5 lines
db/init.c +1
-1@@ -153,7 +153,7 @@ init(*/ if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) { -int error = -libxfs_initialize_perag_data (mp, sbp->sb_agcount); + error = -libxfs_initialize_perag_data (mp, sbp->sb_agcount);Er... this and the
xfs_check_sizeshoisting below don't have anything to do with uncached io conversion...?Quoted 32 lines
db/init.c +0
-0if (error) { fprintf(stderr, _("%s: cannot init perag data (%d). Continuing anyway.\n"),libxfs/init.c +4
-3@@ -419,7 +419,7 @@ done:*/ static int rtmount_init ( -xfs_mount_t *mp,/* file system mount structure */ +struct xfs_mount *mp,int flags) {struct xfs_buf *bp;/* buffer for last block of subvolume */ @@ -473,8 +473,9 @@ rtmount_init( (unsigned long long) mp->m_sb.sb_rblocks);return -1 ; } - error =libxfs_buf_read (mp->m_rtdev, d -XFS_FSB_TO_BB (mp,1 ), -XFS_FSB_TO_BB (mp,1 ),0 , &bp,NULL ); + error =libxfs_buf_read_uncached (mp->m_rtdev_targp, + d -XFS_FSB_TO_BB (mp,1 ), +XFS_FSB_TO_BB (mp,1 ),0 , &bp,NULL );if (error) {fprintf (stderr,_ ("%s: realtime size check failed\n" ), progname);@@ -657,6 +658,52 @@ libxfs_buftarg_init( mp->m_rtdev_targp =libxfs_buftarg_alloc (mp, rtdev); }
Quoted 8 lines
libxfs/init.c libxfs_buftarg_init( · line 664
Quoted 4 lines
libxfs/init.c +4
-0+/* +* Check that the data (and log if separate) is an ok size. +* +* XXX: copied from kernel, needs to be moved to shared code Ah, because you want to share this function with the kernel.
Hmm... what do you think about putting it in
libxfs/xfs_sb.c?
It's not really superblock functionality - it's buftarg functionality - but I don't see anywhere else that is even vaguely appropriate. And the buftarg implementation is not intendeed to be shared code, so it doesn't fit there, either. :/
So, yeah, maybe that is the only suitable place it can be put. I'll split it out into it's own patch.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 13/27 libxfs: introduce userspace buftarg infrastructure
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
This mirrors the buftarg interface provided by the kernel for devices. While parts of the interface are the same for supporting xfs_buf.c and the allocation/freeing of buftargs, the implementation in userspace is substantially different and so we are starting with a cut down copy of the kernel xfs_buftarg.h rather than sharing it via libxfs.
The buftarg implementation in this patch will provide most of the mangement infrastructure the kernel side provides. This initial patch provides buftarg setup and teardown routines.
Note that mkfs abuses the mounting code to calculate the log size before we've finished setting up the superblock. Given that mount will now actually open and check device sizes unconditionally, the mkfs code now needs to set up enough of the superblock and pass real devices to the mount code for it to work correctly.
--- include/libxfs.h | 1 + include/xfs_inode.h | 1 - libfrog/linux.c | 14 ++++++- libxfs/Makefile | 4 +- libxfs/buftarg.c | 99 ++++++++++++++++++++++++++++++++++++++++++++ libxfs/init.c | 38 +++++++---------- libxfs/libxfs_io.h | 21 ++-------- libxfs/libxfs_priv.h | 3 ++ libxfs/xfs_buftarg.h | 55 ++++++++++++++++++++++++ mkfs/xfs_mkfs.c | 23 +++++++--- 10 files changed, 209 insertions(+), 50 deletions(-) create mode 100644 libxfs/buftarg.c create mode 100644 libxfs/xfs_buftarg.h
include/libxfs.h +1 -0
@@ -50,6 +50,7 @@ struct iomap;* This mirrors the kernel include for xfs_buf.h - it's implicitly included in * every files via a similar include in the kernel xfs_linux.h. */ +#include "xfs_buftarg.h" #include "libxfs_io.h" #include "xfs_bit.h"
include/xfs_inode.h +0 -1
@@ -67,7 +67,6 @@ typedef struct xfs_inode {struct xfs_mount *i_mount;/* fs mount struct ptr */ xfs_ino_t i_ino;/* inode number (agno/agino) */ struct xfs_imap i_imap;/* location for xfs_imap() */ -struct xfs_buftarg i_dev;/* dev for this inode */ struct xfs_ifork *i_afp;/* attribute fork pointer */ struct xfs_ifork *i_cowfp;/* copy on write extents */ struct xfs_ifork i_df;/* data fork */
libfrog/linux.c +13 -1
@@ -129,7 +129,19 @@ platform_check_iswritable(char *name, char *block, struct stat *s)int platform_set_blocksize (int fd,char *path,dev_t device,int blocksize,int fatal) { -int error =0 ; +struct stat st; +int error =0 ; + +if (fstat (fd, &st) <0 ) { +fprintf (stderr,_ ("%s: " +"cannot stat the device file \"%s\": %s\n" ), + progname, path,strerror (errno)); +exit (1 ); + } + +/* Can't set block sizes on image files. */ +if ((st.st_mode &S_IFMT ) !=S_IFBLK ) +return 0 ;if (major (device) !=RAMDISK_MAJOR ) {if ((error =ioctl (fd,BLKBSZSET , &blocksize)) <0 ) {
libxfs/Makefile +3 -1
@@ -30,6 +30,7 @@ HFILES = \ xfs_bmap_btree.h \ xfs_btree.h \ xfs_btree_staging.h \ + xfs_buftarg.h \ xfs_attr_remote.h \ xfs_cksum.h \ xfs_da_btree.h \
@@ -54,7 +55,8 @@ HFILES = \ libxfs_priv.h \ xfs_dir2_priv.h -CFILES = cache.c \ +CFILES = buftarg.c \ + cache.c \ defer_item.c \ init.c \ kmem.c \
libxfs/buftarg.c new file +99 -0
99 lines not on this page · raw message
libxfs/init.c +15 -23
@@ -590,26 +590,6 @@ out_unwind:return error; } -static struct xfs_buftarg * -libxfs_buftarg_alloc ( -struct xfs_mount *mp, -dev_t dev) -{ -struct xfs_buftarg *btp; - - btp =malloc (sizeof (*btp)); -if (!btp) { -fprintf (stderr,_ ("%s: buftarg init failed\n" ), - progname); -exit (1 ); - } - btp->bt_mount = mp; - btp->bt_bdev = dev; - btp->flags =0 ; - -return btp; -} - void libxfs_buftarg_init( struct xfs_mount *mp,
@@ -650,12 +630,24 @@ libxfs_buftarg_init(return ; } - mp->m_ddev_targp =libxfs_buftarg_alloc (mp, dev); + mp->m_ddev_targp =xfs_buftarg_alloc (mp, dev); +if (!mp->m_ddev_targp) + goto out_fail;if (!logdev || logdev == dev) mp->m_logdev_targp = mp->m_ddev_targp;else - mp->m_logdev_targp =libxfs_buftarg_alloc (mp, logdev); - mp->m_rtdev_targp =libxfs_buftarg_alloc (mp, rtdev); + mp->m_logdev_targp =xfs_buftarg_alloc (mp, logdev); +if (!mp->m_logdev_targp) + goto out_fail; +if (rtdev) { + mp->m_rtdev_targp =xfs_buftarg_alloc (mp, rtdev); +if (!mp->m_rtdev_targp) + goto out_fail; + } +return ; +out_fail: +fprintf (stderr,_ ("%s: Failed to allocate buftarg\n" ), progname); +exit (1 ); }/*
libxfs/libxfs_io.h +3 -18
@@ -14,25 +14,10 @@struct xfs_buf ;struct xfs_mount ;struct xfs_perag ; +struct xfs_buftarg ; -/* -* IO verifier callbacks need the xfs_mount pointer, so we have to behave -* somewhat like the kernel now for userspace IO in terms of having buftarg -* based devices... -*/ -struct xfs_buftarg { -struct xfs_mount *bt_mount; -dev_t bt_bdev; -unsigned int flags; -}; - -/* We purged a dirty buffer and lost a write. */ -#define XFS_BUFTARG_LOST_WRITE (1 << 0) -/* A dirty buffer failed the write verifier. */ -#define XFS_BUFTARG_CORRUPT_WRITE (1 << 1) - -extern void libxfs_buftarg_init (struct xfs_mount *mp,dev_t ddev, -dev_t logdev,dev_t rtdev); +void libxfs_buftarg_init (struct xfs_mount *mp,dev_t ddev, +dev_t logdev,dev_t rtdev);int libxfs_blkdev_issue_flush (struct xfs_buftarg *btp);#define LIBXFS_BBTOOFF64 (bbs) (((xfs_off_t)(bbs)) << BBSHIFT)
libxfs/libxfs_priv.h +3 -0
@@ -85,6 +85,7 @@ struct iomap;* This mirrors the kernel include for xfs_buf.h - it's implicitly included in * every files via a similar include in the kernel xfs_linux.h. */ +#include "xfs_buftarg.h" #include "libxfs_io.h" /* for all the support code that uses progname in error messages */
@@ -201,6 +202,8 @@ static inline bool WARN_ON(bool expr) { }#define WARN_ON_ONCE (e) WARN_ON(e) +#define percpu_counter_init (x,v,gfp) (*x = v) +#define percpu_counter_destroy (x) ((void) 0)#define percpu_counter_read (x) (*x)#define percpu_counter_read_positive (x) ((*x) > 0 ? (*x) : 0)#define percpu_counter_sum (x) (*x)
libxfs/xfs_buftarg.h new file +55 -0
55 lines not on this page · raw message
mkfs/xfs_mkfs.c +17 -6
@@ -2660,6 +2660,16 @@ _("size %lld of data subvolume is too small, minimum %lld blocks\n"), reported by the device (%u).\n"), cfg->sectorsize, xi->dbsize); } + +if (xi->disfile && + xi->dsize * xi->dbsize < cfg->dblocks * cfg->blocksize) { +if (ftruncate (xi->dfd, cfg->dblocks * cfg->blocksize) <0 ) { +fprintf (stderr, +_ ("%s: Growing the data section failed\n" ), + progname); +exit (1 ); + } + } }/*
@@ -3185,6 +3195,7 @@ calculate_log_size(struct cli_params *cli, struct xfs_mount *mp) { + struct libxfs_xinit *xi = cli->xi;struct xfs_sb *sbp = &mp->m_sb;int min_logblocks;struct xfs_mount mount;
@@ -3192,7 +3203,7 @@ calculate_log_size(/* we need a temporary mount to calculate the minimum log size. */ memset (&mount,0 ,sizeof (mount)); mount.m_sb = *sbp; -libxfs_mount (&mount, &mp->m_sb,0 ,0 ,0 ,0 ); +libxfs_mount (&mount, &mp->m_sb, xi->ddev, xi->logdev, xi->rtdev,0 ); min_logblocks =libxfs_log_calc_minimum_size (&mount);libxfs_umount (&mount);
@@ -3352,8 +3363,10 @@ start_superblock_setup( }else sbp->sb_logsunit =0 ; -/* log reservation calculations depend on rt geometry */ +/* log reservation calculations depends on geometry */ + sbp->sb_dblocks = cfg->dblocks; sbp->sb_rblocks = cfg->rtblocks; + sbp->sb_rextents = cfg->rtextents; sbp->sb_rextsize = cfg->rtextblocks; }
@@ -3390,8 +3403,6 @@ finish_superblock_setup(memcpy (sbp->sb_fname, cfg->label, label_len); } - sbp->sb_dblocks = cfg->dblocks; - sbp->sb_rextents = cfg->rtextents;platform_uuid_copy (&sbp->sb_uuid, &cfg->uuid);/* Only in memory; libxfs expects this as if read from disk */ platform_uuid_copy (&sbp->sb_meta_uuid, &cfg->uuid);
@@ -3414,7 +3425,6 @@ finish_superblock_setup( sbp->sb_qflags =0 ; sbp->sb_unit = cfg->dsunit; sbp->sb_width = cfg->dswidth; - }/* Prepare an uncached buffer, ready to write something out. */
@@ -3524,7 +3534,8 @@ prepare_devices( lsunit, XLOG_FMT, XLOG_INIT_CYCLE, false);/* finally, check we can write the last block in the realtime area */ -if (mp->m_rtdev_targp->bt_bdev && cfg->rtblocks >0 ) { +if (mp->m_rtdev_targp && mp->m_rtdev_targp->bt_bdev && + cfg->rtblocks >0 ) { buf =alloc_write_buf (mp->m_rtdev_targp,XFS_FSB_TO_BB (mp, cfg->rtblocks -1LL ),BTOBB (cfg->blocksize));
Signature · 1 line
2.28.0
PATCH 14/27 xfs: rename libxfs_buftarg_init to libxfs_open_devices()
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
This matches the kernel function for allocating the buftargs for each device. The userspace function takes a bunch of devices, so the new name matches what it does much more closely.
--- copy/xfs_copy.c | 2 +- db/init.c | 2 +- db/sb.c | 2 +- libxfs/init.c | 4 ++-- libxfs/libxfs_io.h | 2 +- logprint/logprint.c | 2 +- mkfs/xfs_mkfs.c | 2 +- 7 files changed, 8 insertions(+), 8 deletions(-)
copy/xfs_copy.c +1 -1
@@ -733,7 +733,7 @@ main(int argc, char **argv)memset (&mbuf,0 ,sizeof (xfs_mount_t));/* We don't yet know the sector size, so read maximal size */ -libxfs_buftarg_init (&mbuf, xargs.ddev, xargs.logdev, xargs.rtdev); +libxfs_open_devices (&mbuf, xargs.ddev, xargs.logdev, xargs.rtdev); error = -libxfs_buf_read_uncached (mbuf.m_ddev_targp,XFS_SB_DADDR ,1 << (XFS_MAX_SECTORSIZE_LOG -BBSHIFT ),0 , &sbp,NULL );if (error) {
db/init.c +1 -1
@@ -109,7 +109,7 @@ init(* tool and so need to be able to mount busted filesystems. */ memset (&xmount,0 ,sizeof (struct xfs_mount )); -libxfs_buftarg_init (&xmount, x.ddev, x.logdev, x.rtdev); +libxfs_open_devices (&xmount, x.ddev, x.logdev, x.rtdev); error = -libxfs_buf_read_uncached (xmount.m_ddev_targp,XFS_SB_DADDR ,1 << (XFS_MAX_SECTORSIZE_LOG -BBSHIFT ),0 , &bp,NULL );if (error) {
db/sb.c +1 -1
@@ -233,7 +233,7 @@ sb_logcheck(void) } } -libxfs_buftarg_init (mp, x.ddev, x.logdev, x.rtdev); +libxfs_open_devices (mp, x.ddev, x.logdev, x.rtdev); dirty =xlog_is_dirty (mp, mp->m_log, &x,0 );if (dirty ==-1 ) {
libxfs/init.c +2 -2
@@ -591,7 +591,7 @@ out_unwind: } void -libxfs_buftarg_init( +libxfs_open_devices( struct xfs_mount *mp, dev_t dev, dev_t logdev,
@@ -715,7 +715,7 @@ libxfs_mount(bool debugger = (flags &LIBXFS_MOUNT_DEBUGGER );int error; -libxfs_buftarg_init (mp, dev, logdev, rtdev); +libxfs_open_devices (mp, dev, logdev, rtdev); mp->m_finobt_nores = true; mp->m_flags = (LIBXFS_MOUNT_32BITINODES |LIBXFS_MOUNT_32BITINOOPT );
libxfs/libxfs_io.h +1 -1
@@ -16,7 +16,7 @@ struct xfs_mount;struct xfs_perag ;struct xfs_buftarg ; -void libxfs_buftarg_init (struct xfs_mount *mp,dev_t ddev, +void libxfs_open_devices (struct xfs_mount *mp,dev_t ddev,dev_t logdev,dev_t rtdev);int libxfs_blkdev_issue_flush (struct xfs_buftarg *btp);
logprint/logprint.c +1 -1
@@ -212,7 +212,7 @@ main(int argc, char **argv)exit (1 );logstat (&mount); -libxfs_buftarg_init (&mount, x.ddev, x.logdev, x.rtdev); +libxfs_open_devices (&mount, x.ddev, x.logdev, x.rtdev); logfd = (x.logfd <0 ) ? x.dfd : x.logfd;
mkfs/xfs_mkfs.c +1 -1
@@ -3986,7 +3986,7 @@ main(/* * we need the libxfs buffer cache from here on in. */ -libxfs_buftarg_init (mp, xi.ddev, xi.logdev, xi.rtdev); +libxfs_open_devices (mp, xi.ddev, xi.logdev, xi.rtdev);/* * Before we mount the filesystem we need to make sure the devices have
Signature · 1 line
2.28.0
PATCH 15/27 libxfs: introduce userspace buftarg infrastructure
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Move the uncached buffer IO API into the xfs_buftarg.h and the local buftarg implementation. The uncached buffer IO implementation is different between kernel and userspace, but the API is the same. Hence implement it via the buftarg abstraction.
Pull the "alloc_write_buf()" function from mkfs up into the API as xfs_buf_get_uncached_daddr() so that it can be used in other places that need the same functionality.
The API movement still uses the existing raw buffer allocation and read IO implementation. This requires us to temporarily export the the prototypes for these functions in xfs_buftarg.h. They will go away once the buftarg has it's own buffer allocation and IO engine implementations.
--- libxfs/buftarg.c | 90 ++++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_io.h | 22 +---------- libxfs/rdwr.c | 88 ++++++------------------------------------- libxfs/xfs_buftarg.h | 39 +++++++++++++++++++ mkfs/xfs_mkfs.c | 29 ++++++++++---- 5 files changed, 164 insertions(+), 104 deletions(-)
libxfs/buftarg.c +90 -0
@@ -97,3 +97,93 @@ xfs_buftarg_free(platform_flush_device (btp->bt_fd, btp->bt_bdev);free (btp); } + +/* +* Allocate an uncached buffer that points at daddr. The refcount will be 1, +* and the cache node hash list will be empty to indicate that it's uncached. +*/ +int +xfs_buf_get_uncached_daddr ( +struct xfs_buftarg *target, +xfs_daddr_t daddr, +size_t bblen, +struct xfs_buf **bpp) +{ +struct xfs_buf *bp; + + bp =libxfs_getbufr (target, daddr, bblen); +if (!bp) +return -ENOMEM ; + +INIT_LIST_HEAD (&bp->b_node.cn_hash); + bp->b_node.cn_count =1 ; + bp->b_bn = daddr; + bp->b_maps[0 ].bm_bn = daddr; + *bpp = bp; +return 0 ; +} + +int +xfs_buf_read_uncached ( +struct xfs_buftarg *target, +xfs_daddr_t daddr, +size_t bblen, +int flags, +struct xfs_buf **bpp, +const struct xfs_buf_ops *ops) +{ +struct xfs_buf *bp; +int error; + + + error =xfs_buf_get_uncached (target, bblen, flags, &bp); +if (error) +return error; + + error =libxfs_readbufr (target, daddr, bp, bblen, flags); +if (error) + goto release_buf; + + error =libxfs_readbuf_verify (bp, ops); +if (error) + goto release_buf; + + *bpp = bp; +return 0 ; + +release_buf: +libxfs_buf_relse (bp); +return error; +} + +/* +* Return a buffer associated to external memory via xfs_buf_associate_memory() +* back to it's empty state. +*/ +void +xfs_buf_set_empty ( +struct xfs_buf *bp, +size_t numblks) +{ + bp->b_addr =NULL ; + bp->b_length = numblks; + +ASSERT (bp->b_map_count ==1 ); + bp->b_bn =XFS_BUF_DADDR_NULL ; + bp->b_maps[0 ].bm_bn =XFS_BUF_DADDR_NULL ; + bp->b_maps[0 ].bm_len = bp->b_length; +} + +/* +* Associate external memory with an empty uncached buffer. +*/ +int +xfs_buf_associate_memory ( +struct xfs_buf *bp, +void *mem, +size_t len) +{ + bp->b_addr = mem; + bp->b_length =BTOBB (len); +return 0 ; +}
libxfs/libxfs_io.h +1 -21
@@ -61,7 +61,7 @@ struct xfs_buf {struct xfs_mount *b_mount;struct xfs_buf_map *b_maps;struct xfs_buf_map __b_map; -int b_nmaps; +int b_map_count;struct list_head b_list; };
@@ -77,8 +77,6 @@ bool xfs_verify_magic16(struct xfs_buf *bp, __be16 dmagic);typedef unsigned int xfs_buf_flags_t ; -#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL)) -#define xfs_buf_offset (bp, offset) ((bp)->b_addr + (offset))#define XFS_BUF_ADDR (bp) ((bp)->b_bn)
@@ -148,10 +146,6 @@ extern int libxfs_bcache_overflowed(void);/* Buffer (Raw) Interfaces */ int libxfs_bwrite (struct xfs_buf *bp); -extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t , -struct xfs_buf *,int ,int ); -extern int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int ); -extern int libxfs_device_zero (struct xfs_buftarg *,xfs_daddr_t ,uint );extern int libxfs_bhash_size;
@@ -170,26 +164,12 @@ xfs_buf_update_cksum(struct xfs_buf *bp, unsigned long cksum_offset) cksum_offset); } -static inline int -xfs_buf_associate_memory (struct xfs_buf *bp,void *mem,size_t len) -{ - bp->b_addr = mem; - bp->b_length =BTOBB (len); -return 0 ; -} -static inline void xfs_buf_hold (struct xfs_buf *bp) { bp->b_node.cn_count++; } -int libxfs_buf_get_uncached (struct xfs_buftarg *targ,size_t bblen,int flags, -struct xfs_buf **bpp); -int libxfs_buf_read_uncached (struct xfs_buftarg *targ,xfs_daddr_t daddr, -size_t bblen,int flags,struct xfs_buf **bpp, -const struct xfs_buf_ops *ops); -/* Push a single buffer on a delwri queue. */ static inline bool xfs_buf_delwri_queue (struct xfs_buf *bp,struct list_head *buffer_list)
libxfs/rdwr.c +12 -76
142 lines not on this page · raw message
libxfs/xfs_buftarg.h +39 -0
@@ -11,6 +11,8 @@ struct xfs_mount;struct xfs_buf ;struct xfs_buf_ops ; +#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL)) +/* * The xfs_buftarg contains 2 notions of "sector size" - *
@@ -52,4 +54,41 @@ int xfs_buftarg_setsize(struct xfs_buftarg *target, unsigned int size);#define xfs_getsize_buftarg (buftarg) block_size((buftarg)->bt_bdev) +/* +* Low level buftarg IO routines. +* +* This includes the uncached buffer IO API, as the memory management associated +* with uncached buffers is tightly tied to the kernel buffer implementation. +*/ + +void xfs_buf_set_empty (struct xfs_buf *bp,size_t numblks); +int xfs_buf_associate_memory (struct xfs_buf *bp,void *mem,size_t length); + +int xfs_buf_get_uncached_daddr (struct xfs_buftarg *target,xfs_daddr_t daddr, +size_t bblen,struct xfs_buf **bpp); +static inline int +xfs_buf_get_uncached ( +struct xfs_buftarg *target, +size_t bblen, +int flags, +struct xfs_buf **bpp) +{ +return xfs_buf_get_uncached_daddr (target,XFS_BUF_DADDR_NULL , bblen, bpp); +} + +int xfs_buf_read_uncached (struct xfs_buftarg *target,xfs_daddr_t daddr, +size_t bblen,int flags,struct xfs_buf **bpp, +const struct xfs_buf_ops *ops); + +/* +* Raw buffer access functions. These exist as temporary bridges for uncached IO +* that uses direct access to the buffers to submit IO. These will go away with +* the new buffer cache IO engine. +*/ +struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno, +int bblen); +int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t ,struct xfs_buf *,int , +int ); +int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int ); +#endif /* __XFS_BUFTARG_H */
mkfs/xfs_mkfs.c +22 -7
@@ -3463,6 +3463,7 @@ prepare_devices(struct xfs_buf *buf;int whack_blks =BTOBB (WHACK_SIZE );int lsunit; +int error;/* * If there's an old XFS filesystem on the device with enough intact
@@ -3496,8 +3497,10 @@ prepare_devices(* the end of the device. (MD sb is ~64k from the end, take out a wider * swath to be sure) */ - buf =alloc_write_buf (mp->m_ddev_targp, (xi->dsize - whack_blks), - whack_blks); + error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp, + (xi->dsize - whack_blks), whack_blks, &buf); +if (error) + goto out_error;memset (buf->b_addr,0 ,WHACK_SIZE );libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);
@@ -3508,14 +3511,18 @@ prepare_devices(* swap (somewhere around the page size), jfs (32k), * ext[2,3] and reiserfs (64k) - and hopefully all else. */ - buf =alloc_write_buf (mp->m_ddev_targp,0 , whack_blks); + error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp,0 , whack_blks, &buf); +if (error) + goto out_error;memset (buf->b_addr,0 ,WHACK_SIZE );libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);/* OK, now write the superblock... */ - buf =alloc_write_buf (mp->m_ddev_targp,XFS_SB_DADDR , -XFS_FSS_TO_BB (mp,1 )); + error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp,XFS_SB_DADDR , +XFS_FSS_TO_BB (mp,1 ), &buf); +if (error) + goto out_error; buf->b_ops = &xfs_sb_buf_ops;memset (buf->b_addr,0 , cfg->sectorsize);libxfs_sb_to_disk (buf->b_addr, sbp);
@@ -3536,14 +3543,22 @@ prepare_devices(/* finally, check we can write the last block in the realtime area */ if (mp->m_rtdev_targp && mp->m_rtdev_targp->bt_bdev && cfg->rtblocks >0 ) { - buf =alloc_write_buf (mp->m_rtdev_targp, + error =xfs_buf_get_uncached_daddr (mp->m_rtdev_targp,XFS_FSB_TO_BB (mp, cfg->rtblocks -1LL ), -BTOBB (cfg->blocksize)); +BTOBB (cfg->blocksize), &buf); +if (error) + goto out_error;memset (buf->b_addr,0 , cfg->blocksize);libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf); } +return ; + +out_error: +fprintf (stderr,_ ("Could not get memory for buffer, err=%d\n" ), + error); +exit (1 ); } static void
Signature · 1 line
2.28.0
Re: [PATCH 15/27] libxfs: introduce userspace buftarg infrastructure
Quoted 10 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Move the uncached buffer IO API into the
xfs_buftarg.hand the local buftarg implementation. The uncached buffer IO implementation is different between kernel and userspace, but the API is the same. Hence implement it via the buftarg abstraction.Pull the "
alloc_write_buf()" function from mkfs up into the API asxfs_buf_get_uncached_daddr()so that it can be used in other places that need the same functionality.
But this doesn't actually remove alloc_write_buf from xfs_mkfs.c...
Quoted 35 lines
The API movement still uses the existing raw buffer allocation and read IO implementation. This requires us to temporarily export the the prototypes for these functions in
xfs_buftarg.h. They will go away once the buftarg has it's own buffer allocation and IO engine implementations.Signed-off-by Dave Chinner <[email protected]>--- libxfs/buftarg.c | 90 ++++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_io.h | 22 +---------- libxfs/rdwr.c | 88 ++++++------------------------------------- libxfs/xfs_buftarg.h | 39 +++++++++++++++++++ mkfs/xfs_mkfs.c | 29 ++++++++++---- 5 files changed, 164 insertions(+), 104 deletions(-)libxfs/buftarg.c +12
-0@@ -97,3 +97,93 @@ xfs_buftarg_free(platform_flush_device (btp->bt_fd, btp->bt_bdev);free (btp); } + +/* +* Allocate an uncached buffer that points at daddr. The refcount will be 1, +* and the cache node hash list will be empty to indicate that it's uncached. +*/ +int +xfs_buf_get_uncached_daddr ( +struct xfs_buftarg *target, +xfs_daddr_t daddr, +size_t bblen, +struct xfs_buf **bpp) +{
libxfs/buftarg.c xfs_buftarg_free( · line 121
Quoted 10 lines
libxfs/buftarg.c +10
-0+struct xfs_buf *bp; + + bp =libxfs_getbufr (target, daddr, bblen); +if (!bp) +return -ENOMEM ; + +INIT_LIST_HEAD (&bp->b_node.cn_hash); + bp->b_node.cn_count =1 ; + bp->b_bn = daddr; + bp->b_maps[0 ].bm_bn = daddr;
This function has all kinds of indent weirdness.
The general approach seems ok so far, but WIP indeed. :)
--D
Quoted 413 lines
libxfs/buftarg.c +68
-0+ *bpp = bp; +return 0 ; +} + +int +xfs_buf_read_uncached ( +struct xfs_buftarg *target, +xfs_daddr_t daddr, +size_t bblen, +int flags, +struct xfs_buf **bpp, +const struct xfs_buf_ops *ops) +{ +struct xfs_buf *bp; +int error; + + + error =xfs_buf_get_uncached (target, bblen, flags, &bp); +if (error) +return error; + + error =libxfs_readbufr (target, daddr, bp, bblen, flags); +if (error) + goto release_buf; + + error =libxfs_readbuf_verify (bp, ops); +if (error) + goto release_buf; + + *bpp = bp; +return 0 ; + +release_buf: +libxfs_buf_relse (bp); +return error; +} + +/* +* Return a buffer associated to external memory via xfs_buf_associate_memory() +* back to it's empty state. +*/ +void +xfs_buf_set_empty ( +struct xfs_buf *bp, +size_t numblks) +{ + bp->b_addr =NULL ; + bp->b_length = numblks; + +ASSERT (bp->b_map_count ==1 ); + bp->b_bn =XFS_BUF_DADDR_NULL ; + bp->b_maps[0 ].bm_bn =XFS_BUF_DADDR_NULL ; + bp->b_maps[0 ].bm_len = bp->b_length; +} + +/* +* Associate external memory with an empty uncached buffer. +*/ +int +xfs_buf_associate_memory ( +struct xfs_buf *bp, +void *mem, +size_t len) +{ + bp->b_addr = mem; + bp->b_length =BTOBB (len); +return 0 ; +}libxfs/libxfs_io.h +1
-21@@ -61,7 +61,7 @@ struct xfs_buf {struct xfs_mount *b_mount;struct xfs_buf_map *b_maps;struct xfs_buf_map __b_map; -int b_nmaps; +int b_map_count;struct list_head b_list; };@@ -77,8 +77,6 @@ bool xfs_verify_magic16(struct xfs_buf *bp, __be16 dmagic);typedef unsigned int xfs_buf_flags_t ; -#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL)) -#define xfs_buf_offset (bp, offset) ((bp)->b_addr + (offset))#define XFS_BUF_ADDR (bp) ((bp)->b_bn)@@ -148,10 +146,6 @@ extern int libxfs_bcache_overflowed(void);/* Buffer (Raw) Interfaces */ int libxfs_bwrite (struct xfs_buf *bp); -extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t , -struct xfs_buf *,int ,int ); -extern int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int ); -extern int libxfs_device_zero (struct xfs_buftarg *,xfs_daddr_t ,uint );extern int libxfs_bhash_size;@@ -170,26 +164,12 @@ xfs_buf_update_cksum(struct xfs_buf *bp, unsigned long cksum_offset) cksum_offset); } -static inline int -xfs_buf_associate_memory (struct xfs_buf *bp,void *mem,size_t len) -{ - bp->b_addr = mem; - bp->b_length =BTOBB (len); -return 0 ; -} -static inline void xfs_buf_hold (struct xfs_buf *bp) { bp->b_node.cn_count++; } -int libxfs_buf_get_uncached (struct xfs_buftarg *targ,size_t bblen,int flags, -struct xfs_buf **bpp); -int libxfs_buf_read_uncached (struct xfs_buftarg *targ,xfs_daddr_t daddr, -size_t bblen,int flags,struct xfs_buf **bpp, -const struct xfs_buf_ops *ops); -/* Push a single buffer on a delwri queue. */ static inline bool xfs_buf_delwri_queue (struct xfs_buf *bp,struct list_head *buffer_list)libxfs/rdwr.c +12
-76@@ -249,7 +249,7 @@ __initbuf(struct xfs_buf *bp, struct xfs_buftarg *btp, xfs_daddr_t bno,INIT_LIST_HEAD (&bp->b_li_list);if (!bp->b_maps) { - bp->b_nmaps =1 ; + bp->b_map_count =1 ; bp->b_maps = &bp->__b_map; bp->b_maps[0 ].bm_bn = bp->b_bn; bp->b_maps[0 ].bm_len = bp->b_length;@@ -279,7 +279,7 @@ libxfs_initbuf_map(struct xfs_buf *bp, struct xfs_buftarg *btp, strerror(errno));exit (1 ); } - bp->b_nmaps = nmaps; + bp->b_map_count = nmaps; bytes =0 ;for ( i =0 ; i < nmaps; i++) {@@ -331,7 +331,7 @@ __libxfs_getbufr(int blen)return bp; } -static struct xfs_buf * +struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen) {struct xfs_buf *bp;@@ -617,7 +617,7 @@ libxfs_readbufr_map(struct xfs_buftarg *btp, struct xfs_buf *bp, int flags) fd =libxfs_device_to_fd (btp->bt_bdev); buf = bp->b_addr; -for (i =0 ; i < bp->b_nmaps; i++) { +for (i =0 ; i < bp->b_map_count; i++) {off64_t offset =LIBXFS_BBTOOFF64 (bp->b_maps[i].bm_bn);int len =BBTOB (bp->b_maps[i].bm_len);@@ -707,75 +707,6 @@ err:return error; } -/* Allocate a raw uncached buffer. */ -static inline struct xfs_buf * -libxfs_getbufr_uncached ( -struct xfs_buftarg *targ, -xfs_daddr_t daddr, -size_t bblen) -{ -struct xfs_buf *bp; - - bp =libxfs_getbufr (targ, daddr, bblen); -if (!bp) -return NULL ; - -INIT_LIST_HEAD (&bp->b_node.cn_hash); - bp->b_node.cn_count =1 ; -return bp; -} - -/* -* Allocate an uncached buffer that points nowhere. The refcount will be 1, -* and the cache node hash list will be empty to indicate that it's uncached. -*/ -int -libxfs_buf_get_uncached ( -struct xfs_buftarg *targ, -size_t bblen, -int flags, -struct xfs_buf **bpp) -{ - *bpp =libxfs_getbufr_uncached (targ,XFS_BUF_DADDR_NULL , bblen); -return *bpp !=NULL ?0 : -ENOMEM ; -} - -/* -* Allocate and read an uncached buffer. The refcount will be 1, and the cache -* node hash list will be empty to indicate that it's uncached. -*/ -int -libxfs_buf_read_uncached ( -struct xfs_buftarg *targ, -xfs_daddr_t daddr, -size_t bblen, -int flags, -struct xfs_buf **bpp, -const struct xfs_buf_ops *ops) -{ -struct xfs_buf *bp; -int error; - - *bpp =NULL ; - bp =libxfs_getbufr_uncached (targ, daddr, bblen); -if (!bp) -return -ENOMEM ; - - error =libxfs_readbufr (targ, daddr, bp, bblen, flags); -if (error) - goto err; - - error =libxfs_readbuf_verify (bp, ops); -if (error) - goto err; - - *bpp = bp; -return 0 ; -err: -libxfs_buf_relse (bp); -return error; -} -static int __write_buf (int fd,void *buf,int len,off64_t offset,int flags) {@@ -836,7 +767,7 @@ libxfs_bwrite(int i;void *buf = bp->b_addr; -for (i =0 ; i < bp->b_nmaps; i++) { +for (i =0 ; i < bp->b_map_count; i++) {off64_t offset =LIBXFS_BBTOOFF64 (bp->b_maps[i].bm_bn);int len =BBTOB (bp->b_maps[i].bm_len);@@ -1207,6 +1138,7 @@ libxfs_log_clear(xfs_daddr_t blk;xfs_daddr_t end_blk;char *ptr; +int error;if (((btp && dptr) || (!btp && !dptr)) || (btp && !btp->bt_bdev) || !fs_uuid)@@ -1236,7 +1168,9 @@ libxfs_log_clear(/* write out the first log record */ ptr = dptr;if (btp) { - bp =libxfs_getbufr_uncached (btp, start, len); + error =xfs_buf_get_uncached_daddr (btp, start, len, &bp); +if (error) +return error; ptr = bp->b_addr; } libxfs_log_header(ptr, fs_uuid, version, sunit, fmt, lsn, tail_lsn,@@ -1284,7 +1218,9 @@ libxfs_log_clear( ptr = dptr;if (btp) { - bp =libxfs_getbufr_uncached (btp, blk, len); + error =xfs_buf_get_uncached_daddr (btp, blk, len, &bp); +if (error) +return error; ptr = bp->b_addr; }/* libxfs/xfs_buftarg.h +39
-0@@ -11,6 +11,8 @@ struct xfs_mount;struct xfs_buf ;struct xfs_buf_ops ; +#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL)) +/* * The xfs_buftarg contains 2 notions of "sector size" - * @@ -52,4 +54,41 @@ int xfs_buftarg_setsize(struct xfs_buftarg *target, unsigned int size);#define xfs_getsize_buftarg (buftarg) block_size((buftarg)->bt_bdev) +/* +* Low level buftarg IO routines. +* +* This includes the uncached buffer IO API, as the memory management associated +* with uncached buffers is tightly tied to the kernel buffer implementation. +*/ + +void xfs_buf_set_empty (struct xfs_buf *bp,size_t numblks); +int xfs_buf_associate_memory (struct xfs_buf *bp,void *mem,size_t length); + +int xfs_buf_get_uncached_daddr (struct xfs_buftarg *target,xfs_daddr_t daddr, +size_t bblen,struct xfs_buf **bpp); +static inline int +xfs_buf_get_uncached ( +struct xfs_buftarg *target, +size_t bblen, +int flags, +struct xfs_buf **bpp) +{ +return xfs_buf_get_uncached_daddr (target,XFS_BUF_DADDR_NULL , bblen, bpp); +} + +int xfs_buf_read_uncached (struct xfs_buftarg *target,xfs_daddr_t daddr, +size_t bblen,int flags,struct xfs_buf **bpp, +const struct xfs_buf_ops *ops); + +/* +* Raw buffer access functions. These exist as temporary bridges for uncached IO +* that uses direct access to the buffers to submit IO. These will go away with +* the new buffer cache IO engine. +*/ +struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno, +int bblen); +int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t ,struct xfs_buf *,int , +int ); +int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int ); +#endif /* __XFS_BUFTARG_H */ mkfs/xfs_mkfs.c +22
-7@@ -3463,6 +3463,7 @@ prepare_devices(struct xfs_buf *buf;int whack_blks =BTOBB (WHACK_SIZE );int lsunit; +int error;/* * If there's an old XFS filesystem on the device with enough intact @@ -3496,8 +3497,10 @@ prepare_devices(* the end of the device. (MD sb is ~64k from the end, take out a wider * swath to be sure) */ - buf =alloc_write_buf (mp->m_ddev_targp, (xi->dsize - whack_blks), - whack_blks); + error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp, + (xi->dsize - whack_blks), whack_blks, &buf); +if (error) + goto out_error;memset (buf->b_addr,0 ,WHACK_SIZE );libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);@@ -3508,14 +3511,18 @@ prepare_devices(* swap (somewhere around the page size), jfs (32k), * ext[2,3] and reiserfs (64k) - and hopefully all else. */ - buf =alloc_write_buf (mp->m_ddev_targp,0 , whack_blks); + error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp,0 , whack_blks, &buf); +if (error) + goto out_error;memset (buf->b_addr,0 ,WHACK_SIZE );libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);/* OK, now write the superblock... */ - buf =alloc_write_buf (mp->m_ddev_targp,XFS_SB_DADDR , -XFS_FSS_TO_BB (mp,1 )); + error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp,XFS_SB_DADDR , +XFS_FSS_TO_BB (mp,1 ), &buf); +if (error) + goto out_error; buf->b_ops = &xfs_sb_buf_ops;memset (buf->b_addr,0 , cfg->sectorsize);libxfs_sb_to_disk (buf->b_addr, sbp);@@ -3536,14 +3543,22 @@ prepare_devices(/* finally, check we can write the last block in the realtime area */ if (mp->m_rtdev_targp && mp->m_rtdev_targp->bt_bdev && cfg->rtblocks >0 ) { - buf =alloc_write_buf (mp->m_rtdev_targp, + error =xfs_buf_get_uncached_daddr (mp->m_rtdev_targp,XFS_FSB_TO_BB (mp, cfg->rtblocks -1LL ), -BTOBB (cfg->blocksize)); +BTOBB (cfg->blocksize), &buf); +if (error) + goto out_error;memset (buf->b_addr,0 , cfg->blocksize);libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf); } +return ; + +out_error: +fprintf (stderr,_ ("Could not get memory for buffer, err=%d\n" ), + error); +exit (1 ); } static voidSignature · 1 line
2.28.0
PATCH 16/27 libxfs: add a synchronous IO engine to the buftarg
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Replace the use of the rdwr.c uncached IO routines with a new buftarg based IO engine. This will currently be synchronous so as to match the existing functionality it replaces, but will be easily modified to run AIO in future.
--- libxfs/buftarg.c | 197 ++++++++++++++++++++++++++++++++++++++++--- libxfs/libxfs_io.h | 2 + libxfs/xfs_buftarg.h | 4 +- 3 files changed, 192 insertions(+), 11 deletions(-)
libxfs/buftarg.c +187 -10
@@ -98,6 +98,182 @@ xfs_buftarg_free(free (btp); } +/* +* Low level IO routines +*/ +static void +xfs_buf_ioend ( +struct xfs_buf *bp) +{ +bool read = bp->b_flags &XBF_READ ; + +// printf("endio bn %ld l %d/%d, io err %d err %d f 0x%x\n", bp->b_maps[0].bm_bn, +// bp->b_maps[0].bm_len, BBTOB(bp->b_length), +// bp->b_io_error, bp->b_error, bp->b_flags); + + bp->b_flags &= ~(XBF_READ |XBF_WRITE ); + +/* +* Pull in IO completion errors now. We are guaranteed to be running +* single threaded, so we don't need the lock to read b_io_error. +*/ +if (!bp->b_error && bp->b_io_error) +xfs_buf_ioerror (bp, bp->b_io_error); + +/* Only validate buffers that were read without errors */ +if (read && !bp->b_error && bp->b_ops) { +ASSERT (!bp->b_iodone); + bp->b_ops->verify_read (bp); + } +} + +static void +xfs_buf_complete_io ( +struct xfs_buf *bp, +int status) +{ + +/* +* don't overwrite existing errors - otherwise we can lose errors on +* buffers that require multiple bios to complete. +*/ +if (status) +cmpxchg (&bp->b_io_error,0 , status); + +if (atomic_dec_and_test (&bp->b_io_remaining) ==1 ) +xfs_buf_ioend (bp); +} + +/* +* XXX: this will be replaced by an AIO submission engine in future. In the mean +* time, just complete the IO synchronously so all the machinery still works. +*/ +static int +submit_io ( +struct xfs_buf *bp, +int fd, +void *buf, +xfs_daddr_t blkno, +int size, +bool write) +{ +int ret; + +if (!write) + ret =pread (fd, buf, size,BBTOB (blkno)); +else + ret =pwrite (fd, buf, size,BBTOB (blkno)); +if (ret <0 ) + ret = -errno; +else if (ret != size) + ret = -EIO ; +else + ret =0 ; +xfs_buf_complete_io (bp, ret); +return ret; +} + +static void +xfs_buftarg_submit_io_map ( +struct xfs_buf *bp, +int map, +int *buf_offset, +int *count) +{ +int size; +int offset; +bool rw = (bp->b_flags &XBF_WRITE ); +int error; + + offset = *buf_offset; + +/* +* Limit the IO size to the length of the current vector, and update the +* remaining IO count for the next time around. +*/ + size =min_t (int,BBTOB (bp->b_maps[map].bm_len), *count); + *count -= size; + *buf_offset += size; + +atomic_inc (&bp->b_io_remaining); + + error =submit_io (bp, bp->b_target->bt_fd, bp->b_addr + offset, + bp->b_maps[map].bm_bn, size, rw); +if (error) { +/* +* This is guaranteed not to be the last io reference count +* because the caller (xfs_buf_submit) holds a count itself. +*/ +atomic_dec (&bp->b_io_remaining); +xfs_buf_ioerror (bp, error); + } +} + +void +xfs_buftarg_submit_io ( +struct xfs_buf *bp) +{ +int offset; +int size; +int i; + +/* +* Make sure we capture only current IO errors rather than stale errors +* left over from previous use of the buffer (e.g. failed readahead). +*/ + bp->b_error =0 ; + +if (bp->b_flags &XBF_WRITE ) { +/* +* Run the write verifier callback function if it exists. If +* this function fails it will mark the buffer with an error and +* the IO should not be dispatched. +*/ +if (bp->b_ops) { + bp->b_ops->verify_write (bp); +if (bp->b_error) { +xfs_force_shutdown (bp->b_target->bt_mount, +SHUTDOWN_CORRUPT_INCORE ); +return ; + } + }else if (bp->b_bn !=XFS_BUF_DADDR_NULL ) { +struct xfs_mount *mp = bp->b_target->bt_mount; + +/* +* non-crc filesystems don't attach verifiers during +* log recovery, so don't warn for such filesystems. +*/ +if (xfs_sb_version_hascrc (&mp->m_sb)) { +xfs_warn (mp, +"%s: no buf ops on daddr 0x%llx len %d" , + __func__, bp->b_bn, bp->b_length); +xfs_hex_dump (bp->b_addr, +XFS_CORRUPTION_DUMP_LEN ); + } + } + } + +atomic_set (&bp->b_io_remaining,1 ); + +/* +* Walk all the vectors issuing IO on them. Set up the initial offset +* into the buffer and the desired IO size before we start - +* xfs_buf_ioapply_map() will modify them appropriately for each +* subsequent call. +*/ + offset =0 ; + size =BBTOB (bp->b_length); +for (i =0 ; i < bp->b_map_count; i++) { +xfs_buftarg_submit_io_map (bp, i, &offset, &size); +if (bp->b_error) +break ; +if (size <=0 ) +break ;/* all done */ + } + +xfs_buf_complete_io (bp, bp->b_error); +} +/* * Allocate an uncached buffer that points at daddr. The refcount will be 1, * and the cache node hash list will be empty to indicate that it's uncached.
@@ -140,20 +316,21 @@ xfs_buf_read_uncached(if (error)return error; - error =libxfs_readbufr (target, daddr, bp, bblen, flags); -if (error) - goto release_buf; +/* set up the buffer for a read IO */ +ASSERT (bp->b_map_count ==1 ); + bp->b_maps[0 ].bm_bn = daddr; + bp->b_flags |=XBF_READ ; + bp->b_ops = ops; - error =libxfs_readbuf_verify (bp, ops); -if (error) - goto release_buf; +xfs_buftarg_submit_io (bp); +if (bp->b_error) { + error = bp->b_error; +xfs_buf_relse (bp); +return error; + } *bpp = bp;return 0 ; - -release_buf: -libxfs_buf_relse (bp); -return error; }/*
libxfs/libxfs_io.h +2 -0
@@ -62,6 +62,8 @@ struct xfs_buf {struct xfs_buf_map *b_maps;struct xfs_buf_map __b_map;int b_map_count; +int b_io_remaining; +int b_io_error;struct list_head b_list; };
libxfs/xfs_buftarg.h +3 -1
@@ -60,7 +60,6 @@ int xfs_buftarg_setsize(struct xfs_buftarg *target, unsigned int size);* This includes the uncached buffer IO API, as the memory management associated * with uncached buffers is tightly tied to the kernel buffer implementation. */ -void xfs_buf_set_empty (struct xfs_buf *bp,size_t numblks);int xfs_buf_associate_memory (struct xfs_buf *bp,void *mem,size_t length);
@@ -80,6 +79,9 @@ int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr, size_t bblen, int flags, struct xfs_buf **bpp, const struct xfs_buf_ops *ops); +#define XBF_READ (1 << 0) +#define XBF_WRITE (1 << 1) +/* * Raw buffer access functions. These exist as temporary bridges for uncached IO * that uses direct access to the buffers to submit IO. These will go away with
Signature · 1 line
2.28.0
PATCH 17/27 xfsprogs: convert libxfs_readbufr to libxfs_buf_read_uncached
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
libxfs_readbufr() and libxfs_readbufr_map() are messy ways of reading an existing buffer. We have xfs_bwrite() already, so introduced this function, implement it with the new buftarg based IO engine, and call it xfs_bread().
Note that to make this new code be discontiguous buffer agnostic and still play nice with rdwr.c's LIBXFS_B_DISCONTIG buffoonery, we need to ensure buffers init both bp->b_bn and bp->b_maps[0].bm_bn correctly.
--- db/io.c | 29 ++++------------------ libxfs/buftarg.c | 51 ++++++++++++++++++++++++++++++++------- libxfs/libxfs_api_defs.h | 1 + libxfs/libxfs_io.h | 4 +-- libxfs/libxfs_priv.h | 1 - libxfs/rdwr.c | 24 ++++++++++-------- libxfs/xfs_buftarg.h | 15 ++++++++---- libxlog/xfs_log_recover.c | 7 ++---- repair/prefetch.c | 22 +++++++++++------ 9 files changed, 89 insertions(+), 65 deletions(-)
db/io.c +5 -24
@@ -424,31 +424,15 @@ ring_add(void)static void write_cur_buf (void ) { -int ret; +struct xfs_buf *bp = iocur_top->bp; +int ret; - ret = -libxfs_bwrite (iocur_top->bp); + ret = -libxfs_bwrite (bp);if (ret !=0 )dbprintf (_ ("write error: %s\n" ),strerror (ret));/* re-read buffer from disk */ - ret = -libxfs_readbufr (mp->m_ddev_targp, iocur_top->bb, iocur_top->bp, - iocur_top->blen,0 ); -if (ret !=0 ) -dbprintf (_ ("read error: %s\n" ),strerror (ret)); -} - -static void -write_cur_bbs (void ) -{ -int ret; - - ret = -libxfs_bwrite (iocur_top->bp); -if (ret !=0 ) -dbprintf (_ ("write error: %s\n" ),strerror (ret)); - - -/* re-read buffer from disk */ - ret = -libxfs_readbufr_map (mp->m_ddev_targp, iocur_top->bp,0 ); + ret = -libxfs_bread (bp, bp->b_length);if (ret !=0 )dbprintf (_ ("read error: %s\n" ),strerror (ret)); }
@@ -488,10 +472,7 @@ write_cur(void)else if (iocur_top->dquot_buf)xfs_dquot_set_crc (iocur_top->bp); } -if (iocur_top->bbmap) -write_cur_bbs (); -else -write_cur_buf (); +write_cur_buf ();/* If we didn't write the crc automatically, re-check inode validity */ if (xfs_sb_version_hascrc(&mp->m_sb) &&
libxfs/buftarg.c +42 -9
84 lines not on this page · raw message
libxfs/libxfs_api_defs.h +1 -0
@@ -42,6 +42,7 @@#define xfs_bmbt_maxrecs libxfs_bmbt_maxrecs#define xfs_bmdr_maxrecs libxfs_bmdr_maxrecs +#define xfs_bread libxfs_bread#define xfs_btree_bload libxfs_btree_bload#define xfs_btree_bload_compute_geometry libxfs_btree_bload_compute_geometry#define xfs_btree_del_cursor libxfs_btree_del_cursor
libxfs/libxfs_io.h +1 -3
@@ -80,9 +80,7 @@ bool xfs_verify_magic16(struct xfs_buf *bp, __be16 dmagic);typedef unsigned int xfs_buf_flags_t ;#define xfs_buf_offset (bp, offset) ((bp)->b_addr + (offset)) -#define XFS_BUF_ADDR (bp) ((bp)->b_bn) - -#define XFS_BUF_SET_ADDR (bp,blk) ((bp)->b_bn = (blk)) +#define XFS_BUF_ADDR (bp) ((bp)->b_maps[0].bm_bn)void libxfs_buf_set_priority (struct xfs_buf *bp,int priority);int libxfs_buf_priority (struct xfs_buf *bp);
libxfs/libxfs_priv.h +0 -1
@@ -408,7 +408,6 @@ howmany_64(uint64_t x, uint32_t y)/* buffer management */ #define XBF_TRYLOCK 0#define XBF_UNMAPPED 0 -#define XBF_DONE 0#define xfs_buf_stale (bp) ((bp)->b_flags |= LIBXFS_B_STALE)#define XFS_BUF_UNDELAYWRITE (bp) ((bp)->b_flags &= ~LIBXFS_B_DIRTY)
libxfs/rdwr.c +14 -10
@@ -247,19 +247,17 @@ __initbuf(struct xfs_buf *bp, struct xfs_buftarg *btp, xfs_daddr_t bno, bp->b_recur =0 ; bp->b_ops =NULL ;INIT_LIST_HEAD (&bp->b_li_list); - -if (!bp->b_maps) { - bp->b_map_count =1 ; - bp->b_maps = &bp->__b_map; - bp->b_maps[0 ].bm_bn = bp->b_bn; - bp->b_maps[0 ].bm_len = bp->b_length; - } }static void libxfs_initbuf (struct xfs_buf *bp,struct xfs_buftarg *btp,xfs_daddr_t bno,unsigned int bytes) { + bp->b_map_count =1 ; + bp->b_maps = &bp->__b_map; + bp->b_maps[0 ].bm_bn = bno; + bp->b_maps[0 ].bm_len = bytes; +__initbuf (bp, btp, bno, bytes); }
@@ -270,6 +268,11 @@ libxfs_initbuf_map(struct xfs_buf *bp, struct xfs_buftarg *btp,unsigned int bytes =0 ;int i; +if (nmaps ==1 ) { +libxfs_initbuf (bp, btp, map[0 ].bm_bn, map[0 ].bm_len); +return ; + } + bytes =sizeof (struct xfs_buf_map ) * nmaps; bp->b_maps =malloc (bytes);if (!bp->b_maps) {
@@ -573,7 +576,7 @@ __read_buf(int fd, void *buf, int len, off64_t offset, int flags)return 0 ; } -int +static int libxfs_readbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,struct xfs_buf *bp,int len,int flags) {
@@ -607,7 +610,7 @@ libxfs_readbuf_verify(return bp->b_error; } -int +static int libxfs_readbufr_map (struct xfs_buftarg *btp,struct xfs_buf *bp,int flags) {int fd;
@@ -762,7 +765,8 @@ libxfs_bwrite(if (!(bp->b_flags &LIBXFS_B_DISCONTIG )) { bp->b_error =__write_buf (fd, bp->b_addr,BBTOB (bp->b_length), -LIBXFS_BBTOOFF64 (bp->b_bn), bp->b_flags); +LIBXFS_BBTOOFF64 (bp->b_maps[0 ].bm_bn), + bp->b_flags); }else {int i;void *buf = bp->b_addr;
libxfs/xfs_buftarg.h +10 -5
@@ -79,8 +79,16 @@ int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr, size_t bblen, int flags, struct xfs_buf **bpp, const struct xfs_buf_ops *ops); -#define XBF_READ (1 << 0) -#define XBF_WRITE (1 << 1) +int xfs_bread (struct xfs_buf *bp,size_t bblen); + +/* +* Temporary: these need to be the same as the LIBXFS_B_* flags until we change +* over to the kernel structures. For those that aren't the same or don't yet +* exist, start the numbering from the top down. +*/ +#define XBF_READ (1 << 31) +#define XBF_WRITE (1 << 30) +#define XBF_DONE (1 << 3) // LIBXFS_B_UPTODATE 0x0008/* * Raw buffer access functions. These exist as temporary bridges for uncached IO
@@ -89,8 +97,5 @@ int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr,*/ struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen); -int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t ,struct xfs_buf *,int , -int ); -int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *,int );#endif /* __XFS_BUFTARG_H */
libxlog/xfs_log_recover.c +2 -5
@@ -110,15 +110,12 @@ xlog_bread_noalign( blk_no =round_down (blk_no, log->l_sectBBsize); nbblks =round_up (nbblks, log->l_sectBBsize); -ASSERT (nbblks >0 );ASSERT (nbblks <= bp->b_length); -XFS_BUF_SET_ADDR (bp, log->l_logBBstart + blk_no); - bp->b_length = nbblks; - bp->b_error =0 ; + bp->b_maps[0 ].bm_bn = log->l_logBBstart + blk_no; -return libxfs_readbufr (log->l_dev,XFS_BUF_ADDR (bp), bp, nbblks,0 ); +return libxfs_bread (bp, nbblks); }int
repair/prefetch.c +14 -8
@@ -474,6 +474,7 @@ pf_batch_read( void *buf) {struct xfs_buf *bplist[MAX_BUFS ]; +struct xfs_buf *lbp;unsigned int num;off64_t first_off, last_off, next_off;int len, size;
@@ -518,18 +519,21 @@ pf_batch_read(if (!num)return ; +/* * do a big read if 25% of the potential buffer is useful, * otherwise, find as many close together blocks and * read them in one read */ + lbp = bplist[num -1 ]; first_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[0 ])); - last_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[num-1 ])) + -BBTOB (bplist[num-1 ]->b_length); + last_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (lbp)) + +BBTOB (lbp->b_length);while (num >1 && last_off - first_off > pf_max_bytes) { num--; - last_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (bplist[num-1 ])) + -BBTOB (bplist[num-1 ]->b_length); + lbp = bplist[num -1 ]; + last_off =LIBXFS_BBTOOFF64 (XFS_BUF_ADDR (lbp)) + +BBTOB (lbp->b_length); }if (num < ((last_off - first_off) >> (mp->m_sb.sb_blocklog +3 ))) {/*
@@ -545,6 +549,7 @@ pf_batch_read( last_off = next_off; } num = i; + lbp = bplist[num -1 ]; }for (i =0 ; i < num; i++) {
@@ -583,11 +588,12 @@ pf_batch_read(* guarantees that only the last buffer in the list will be a * discontiguous buffer. */ -if ((bplist[num -1 ]->b_flags &LIBXFS_B_DISCONTIG )) { -libxfs_readbufr_map (mp->m_ddev_targp, bplist[num -1 ],0 ); - bplist[num -1 ]->b_flags |=LIBXFS_B_UNCHECKED ; -libxfs_buf_relse (bplist[num -1 ]); +if (lbp->b_flags &LIBXFS_B_DISCONTIG ) { +libxfs_bread (lbp, lbp->b_length); + lbp->b_flags |=LIBXFS_B_UNCHECKED ; +libxfs_buf_relse (lbp); num--; + lbp = bplist[num -1 ]; }if (len >0 ) {
Signature · 1 line
2.28.0
PATCH 18/27 libxfs: convert libxfs_bwrite to buftarg IO
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Buffers can now be written by the buftarg IO engine, so redirect the API to the new implementation and ensure it twiddles flag state correctly.
--- libxfs/buftarg.c | 30 +++++++++++++++- libxfs/libxfs_io.h | 1 - libxfs/libxfs_priv.h | 2 +- libxfs/rdwr.c | 85 -------------------------------------------- libxfs/xfs_buftarg.h | 5 ++- 5 files changed, 34 insertions(+), 89 deletions(-)
libxfs/buftarg.c +29 -1
@@ -126,8 +126,16 @@ xfs_buf_ioend( bp->b_ops->verify_read (bp); } -if (!bp->b_error) +if (!bp->b_error) { bp->b_flags |=XBF_DONE ; + bp->b_flags &= ~(LIBXFS_B_DIRTY |LIBXFS_B_UNCHECKED ); + }else { +fprintf (stderr, +_ ("%s: IO failed on %s bno 0x%llx/0x%x, err=%d\n" ), + __func__, bp->b_ops ? bp->b_ops->name :"(unknown)" , + (long long )bp->b_maps[0 ].bm_bn, bp->b_length, + -bp->b_error); + } } static void
@@ -227,6 +235,19 @@ xfs_buftarg_submit_io( bp->b_error =0 ;if (bp->b_flags &XBF_WRITE ) { + +/* +* we never write buffers that are marked stale. This indicates +* they contain data that has been invalidated, and even if the +* buffer is dirty it must *never* be written. Verifiers are +* wonderful for finding bugs like this. Make sure the error is +* obvious as to the cause. +*/ +if (bp->b_flags &XBF_STALE ) { + bp->b_error = -ESTALE ; +return ; + } +/* * Run the write verifier callback function if it exists. If * this function fails it will mark the buffer with an error and
@@ -366,6 +387,13 @@ xfs_buf_read_uncached(return 0 ; } +int +xfs_bwrite (struct xfs_buf *bp) +{ +return xfs_buf_uncached_submit (bp->b_target, bp, bp->b_length, +XBF_WRITE ); +} +/* * Return a buffer associated to external memory via xfs_buf_associate_memory() * back to it's empty state.
libxfs/libxfs_io.h +0 -1
@@ -145,7 +145,6 @@ extern void libxfs_bcache_flush(void);extern int libxfs_bcache_overflowed (void );/* Buffer (Raw) Interfaces */ -int libxfs_bwrite (struct xfs_buf *bp);extern int libxfs_device_zero (struct xfs_buftarg *,xfs_daddr_t ,uint );extern int libxfs_bhash_size;
libxfs/libxfs_priv.h +1 -1
@@ -408,7 +408,7 @@ howmany_64(uint64_t x, uint32_t y)/* buffer management */ #define XBF_TRYLOCK 0#define XBF_UNMAPPED 0 -#define xfs_buf_stale (bp) ((bp)->b_flags |= LIBXFS_B_STALE) +#define xfs_buf_stale (bp) ((bp)->b_flags |= XBF_STALE)#define XFS_BUF_UNDELAYWRITE (bp) ((bp)->b_flags &= ~LIBXFS_B_DIRTY)/* buffer type flags for write callbacks */
libxfs/rdwr.c +0 -85
91 lines not on this page · raw message
libxfs/xfs_buftarg.h +4 -1
@@ -81,6 +81,8 @@ int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr,int xfs_bread (struct xfs_buf *bp,size_t bblen); +int xfs_bwrite (struct xfs_buf *bp); +/* * Temporary: these need to be the same as the LIBXFS_B_* flags until we change * over to the kernel structures. For those that aren't the same or don't yet
@@ -88,7 +90,8 @@ int xfs_bread(struct xfs_buf *bp, size_t bblen);*/ #define XBF_READ (1 << 31)#define XBF_WRITE (1 << 30) -#define XBF_DONE (1 << 3) // LIBXFS_B_UPTODATE 0x0008 +#define XBF_DONE (1 << 3) // LIBXFS_B_UPTODATE 0x0008 +#define XBF_STALE (1 << 2) // LIBXFS_B_STALE 0x0004/* * Raw buffer access functions. These exist as temporary bridges for uncached IO
Signature · 1 line
2.28.0
PATCH 19/27 libxfs: add cache infrastructure to buftarg
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Add a hash cache interface to the libxfs buftarg implementation. This is a massively cut down version of the existing cache, just with all the stuff the buftarg will provide chopped out of it.
--- include/xfs_mount.h | 3 + libxfs/buftarg.c | 233 +++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_io.h | 3 +- libxfs/xfs_buftarg.h | 45 +++++++++ 4 files changed, 283 insertions(+), 1 deletion(-)
include/xfs_mount.h +3 -0
@@ -10,6 +10,7 @@struct xfs_inode ;struct xfs_buftarg ;struct xfs_da_geometry ; +struct btcache ;/* * Define a user-level mount structure with all we need
@@ -155,6 +156,8 @@ typedef struct xfs_perag {/* reference count */ uint8_t pagf_refcount_level; + +struct btcache *pag_buf_hash; } xfs_perag_t; static inline struct xfs_ag_resv *
libxfs/buftarg.c +233 -0
236 lines not on this page · raw message
libxfs/libxfs_io.h +2 -1
@@ -44,9 +44,10 @@ struct xfs_buf_ops {struct xfs_buf {struct cache_node b_node; -unsigned int b_flags; +struct list_head b_hash;/* will replace b_node */ xfs_daddr_t b_bn;unsigned int b_length; +unsigned int b_flags;struct xfs_buftarg *b_target;pthread_mutex_t b_lock;pthread_t b_holder;
libxfs/xfs_buftarg.h +45 -0
55 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 20/27 libxfs: add internal lru to btcache
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
THis tracks all the buffers in a given btcache, hence allowing us to purge all the buffers from a cache without having to walk the global buffer cache LRU list.
This will useful for per-AG scan operations, allowing us to purge the cache when we've completed processing on specific AGs and don't need the cache anymore.
--- libxfs/buftarg.c | 63 +++++++++++++++++++++++++++++++++++++++++++- libxfs/libxfs_io.h | 8 ++++++ libxfs/rdwr.c | 4 +-- libxfs/xfs_buftarg.h | 2 ++ 4 files changed, 73 insertions(+), 4 deletions(-)
libxfs/buftarg.c +62 -1
@@ -428,8 +428,16 @@ xfs_buf_associate_memory(/* * Buffer cache hash implementation +* +* Lock orders: +* +* hash->lock cache hash chain lock +* btc->lock cache lock +* +* btc->lock cache lock +* bp->b_lock buffer state lock +* */ -struct btcache *btc_init (unsigned int hashsize)
@@ -456,6 +464,7 @@ btc_init( btc->hashsize = hashsize; btc->hashshift =libxfs_highbit32 (hashsize);pthread_mutex_init (&btc->lock,NULL ); +list_head_init (&btc->lru);for (i =0 ; i < hashsize; i++) {list_head_init (&btc->hash[i].chain);
@@ -475,6 +484,7 @@ btc_destroy(if (!btc)return ; +list_head_destroy (&btc->lru);for (i =0 ; i < btc->hashsize; i++) {list_head_destroy (&btc->hash[i].chain);pthread_mutex_destroy (&btc->hash[i].lock);
@@ -635,6 +645,10 @@ btc_node_insert( head = &hash->chain;pthread_mutex_lock (&hash->lock); +pthread_mutex_lock (&btc->lock); +list_add (&bp->b_btc_list, &btc->lru); +pthread_mutex_unlock (&btc->lock); +list_add (&bp->b_hash, head); hash->count++;atomic_inc (&btc->count);
@@ -653,8 +667,55 @@ btc_node_remove( hash = btc->hash + hashidx;pthread_mutex_lock (&hash->lock); +pthread_mutex_lock (&btc->lock); +list_del (&bp->b_btc_list); +pthread_mutex_unlock (&btc->lock); +list_del (&bp->b_hash); hash->count--;atomic_dec (&btc->count);pthread_mutex_unlock (&hash->lock); } + +/* +* Purge the buffers from the cache list. +* +* This is nasty - it steals the buffer cache LRU reference and drops it, +* using the dispose flag to indicate it's about to go away. +*/ +static void +btc_purge_buffers ( +struct btcache *btc) +{ +struct xfs_buf *bp, *n; +LIST_HEAD (dispose); + +pthread_mutex_lock (&btc->lock); +list_for_each_entry_safe (bp, n, &btc->lru, b_btc_list) { +if (bp->b_state &XFS_BSTATE_DISPOSE ) +continue ; +spin_lock (&bp->b_lock); +atomic_set (&bp->b_lru_ref,0 ); + bp->b_state |=XFS_BSTATE_DISPOSE ; +list_move (&bp->b_btc_list, &dispose); +spin_unlock (&bp->b_lock); + } +pthread_mutex_unlock (&btc->lock); + +while (!list_empty (&dispose)) { + bp = list_first_entry(&dispose, struct xfs_buf, b_btc_list); +list_del_init (&bp->b_btc_list); +libxfs_brelse (&bp->b_node); + } +} + +void +xfs_buftarg_purge_ag ( +struct xfs_buftarg *btp, +xfs_agnumber_t agno) +{ +struct xfs_perag *pag =xfs_perag_get (btp->bt_mount, agno); + +btc_purge_buffers (pag->pag_buf_hash); +xfs_perag_put (pag); +}
libxfs/libxfs_io.h +8 -0
@@ -42,6 +42,8 @@ struct xfs_buf_ops {xfs_failaddr_t (*verify_struct)(struct xfs_buf *); }; +#define XFS_BSTATE_DISPOSE (1 << 0)/* buffer being discarded */ +struct xfs_buf {struct cache_node b_node;struct list_head b_hash;/* will replace b_node */
@@ -66,6 +68,10 @@ struct xfs_buf {int b_io_remaining;int b_io_error;struct list_head b_list; + +struct list_head b_btc_list; +unsigned int b_state; +atomic_t b_lru_ref; };bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic);
@@ -98,6 +104,8 @@ int libxfs_buf_priority(struct xfs_buf *bp);extern struct cache *libxfs_bcache;extern struct cache_operations libxfs_bcache_operations; +void libxfs_brelse (struct cache_node *node); +#define LIBXFS_GETBUF_TRYLOCK (1 << 0)
libxfs/rdwr.c +1 -3
@@ -21,8 +21,6 @@#include "libxfs.h" -static void libxfs_brelse (struct cache_node *node); -/* * Important design/architecture note: *
@@ -740,7 +738,7 @@ libxfs_whine_dirty_buf( bp->b_target->flags |=XFS_BUFTARG_LOST_WRITE ; } -static void +void libxfs_brelse (struct cache_node *node) {
libxfs/xfs_buftarg.h +2 -0
@@ -55,6 +55,7 @@ struct xfs_buftarg *xfs_buftarg_alloc(struct xfs_mount *mp, dev_t bdev);void xfs_buftarg_free (struct xfs_buftarg *target);void xfs_buftarg_wait (struct xfs_buftarg *target);int xfs_buftarg_setsize (struct xfs_buftarg *target,unsigned int size); +void xfs_buftarg_purge_ag (struct xfs_buftarg *btp,xfs_agnumber_t agno);#define xfs_getsize_buftarg (buftarg) block_size((buftarg)->bt_bdev)
@@ -136,6 +137,7 @@ struct btcache {unsigned long long misses;/* cache misses */ unsigned long long hits;/* cache hits */ unsigned int max;/* max nodes ever used */ +struct list_head lru;/* list of all items in cache */ };struct btcache *btc_init (unsigned int hashsize);
Signature · 1 line
2.28.0
PATCH 21/27 libxfs: Add kernel list_lru wrapper
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The buffer cache in the kernel uses the list_lru infrastructure for cache reclaim, so we need to add some wrappers to provide the necessary functionality to userspace to use the same buffer cache and buftarg code as the kernel for managing the global buffer cache LRU.
--- include/Makefile | 1 + include/libxfs.h | 1 + include/list_lru.h | 69 ++++++++++++++++++++++++++++++++++++++++++++ libxfs/buftarg.c | 7 ++++- libxfs/libxfs_io.h | 1 + libxfs/libxfs_priv.h | 1 + libxfs/xfs_buftarg.h | 1 + 7 files changed, 80 insertions(+), 1 deletion(-) create mode 100644 include/list_lru.h
include/Makefile +1 -0
@@ -16,6 +16,7 @@ LIBHFILES = libxfs.h \ hlist.h \ kmem.h \ list.h \ + list_lru.h \ parent.h \ sema.h \ spinlock.h \
include/libxfs.h +1 -0
@@ -21,6 +21,7 @@#include "spinlock.h" #include "completion.h" #include "sema.h" +#include "list_lru.h" #include "xfs_types.h" #include "xfs_fs.h"
include/list_lru.h new file +69 -0
69 lines not on this page · raw message
libxfs/buftarg.c +6 -1
@@ -78,11 +78,16 @@ xfs_buftarg_alloc(if (xfs_buftarg_setsize_early (btp)) goto error_free; -if (percpu_counter_init (&btp->bt_io_count,0 ,GFP_KERNEL )) +if (list_lru_init (&btp->bt_lru)) goto error_free; +if (percpu_counter_init (&btp->bt_io_count,0 ,GFP_KERNEL )) + goto error_lru; +return btp; +error_lru: +list_lru_destroy (&btp->bt_lru); error_free:free (btp);return NULL ;
libxfs/libxfs_io.h +1 -0
@@ -72,6 +72,7 @@ struct xfs_buf {struct list_head b_btc_list;unsigned int b_state;atomic_t b_lru_ref; +struct list_head b_lru; };bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic);
libxfs/libxfs_priv.h +1 -0
@@ -51,6 +51,7 @@#include "spinlock.h" #include "completion.h" #include "sema.h" +#include "list_lru.h" #include "xfs_types.h" #include "xfs_arch.h"
libxfs/xfs_buftarg.h +1 -0
@@ -41,6 +41,7 @@ struct xfs_buftarg {uint32_t bt_io_count;unsigned int flags; +struct list_lru bt_lru; };/* We purged a dirty buffer and lost a write. */
Signature · 1 line
2.28.0
PATCH 22/27 libxfs: introduce new buffer cache infrastructure
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Now we have a separate buftarg infrastructure, we can introduce the kernel equivalent buffer cache infrastructure. This will (eventually) be shared with the kernel implementation, with all the differences being implemented in the private buftarg implementions.
Add the high level buffer lookup, IO and reference counting functions to libxfs/xfs_buf.c and the low level infrastructure this requires to libxfs/xfs_buftarg.c in preparation for switching the entire of xfsprogs over to using the new infrastructure.
The eventual shared kernel/userspace buffer definitions will end up in xfs_buf.h, so we start building that here to enable the code to compile and sort of function. It gets somewhat messy at this point, but patches after this can get on with switching over the cache implementation and cleaning up the mess.
Note: this uses an older xfs_buf.c from the kernel code, so there will be updates needed to bring it up to date once the initial conversion is complete.
--- include/libxfs.h | 1 + include/xfs.h | 23 + include/xfs_mount.h | 1 + include/xfs_trace.h | 24 + libxfs/Makefile | 2 + libxfs/buftarg.c | 54 +- libxfs/libxfs_api_defs.h | 2 + libxfs/libxfs_io.h | 96 +-- libxfs/libxfs_priv.h | 43 +- libxfs/rdwr.c | 4 +- libxfs/xfs_buf.c | 1350 ++++++++++++++++++++++++++++++++++++++ libxfs/xfs_buf.h | 203 ++++++ libxfs/xfs_buftarg.h | 31 +- 13 files changed, 1661 insertions(+), 173 deletions(-) create mode 100644 libxfs/xfs_buf.c create mode 100644 libxfs/xfs_buf.h
include/libxfs.h +1 -0
@@ -52,6 +52,7 @@ struct iomap;* every files via a similar include in the kernel xfs_linux.h. */ #include "xfs_buftarg.h" +#include "xfs_buf.h" #include "libxfs_io.h" #include "xfs_bit.h"
include/xfs.h +23 -0
@@ -38,6 +38,29 @@ extern int xfs_assert_largefile[sizeof(off_t)-8];#define BUILD_BUG_ON (condition) ((void)sizeof(char[1 - 2*!!(condition)]))#endif +/* +* Kernel side compiler address and barrier functionality we use +*/ +#ifdef __GNUC__ +#define __return_address __builtin_return_address(0) + +/* +* Return the address of a label. Use barrier() so that the optimizer +* won't reorder code to refactor the error jumpouts into a single +* return, which throws off the reported address. +*/ +#define __this_address ({ __label__ __here; __here: barrier(); &&__here; }) +/* Optimization barrier */ + +/* The "volatile" is due to gcc bugs */ +#define barrier () __asm__ __volatile__("": : :"memory") +#endif + +/* Optimization barrier */ +#ifndef barrier +# define barrier () __memory_barrier() +#endif +#include <xfs/xfs_types.h> /* Include deprecated/compat pre-vfs xfs-specific symbols */ #include <xfs/xfs_fs_compat.h>
include/xfs_mount.h +1 -0
@@ -157,6 +157,7 @@ typedef struct xfs_perag {/* reference count */ uint8_t pagf_refcount_level; +spinlock_t pag_buf_lock;struct btcache *pag_buf_hash; } xfs_perag_t;
include/xfs_trace.h +24 -0
@@ -312,4 +312,28 @@#define trace_xfs_perag_get_tag (a,b,c,d) ((c) = (c))#define trace_xfs_perag_put (a,b,c,d) ((c) = (c)) +#define trace_xfs_buf_init (...) ((void) 0) +#define trace_xfs_buf_free (...) ((void) 0) +#define trace_xfs_buf_find (...) ((void) 0) +#define trace_xfs_buf_get (...) ((void) 0) +#define trace_xfs_buf_read (...) ((void) 0) +#define trace_xfs_buf_hold (...) ((void) 0) +#define trace_xfs_buf_rele (...) ((void) 0) +#define trace_xfs_buf_trylock (...) ((void) 0) +#define trace_xfs_buf_trylock_fail (...) ((void) 0) +#define trace_xfs_buf_lock (...) ((void) 0) +#define trace_xfs_buf_lock_done (...) ((void) 0) +#define trace_xfs_buf_unlock (...) ((void) 0) +#define trace_xfs_buf_iodone (...) ((void) 0) +#define trace_xfs_buf_ioerror (...) ((void) 0) +#define trace_xfs_buf_iowait (...) ((void) 0) +#define trace_xfs_buf_iowait_done (...) ((void) 0) +#define trace_xfs_buf_submit (...) ((void) 0) +#define trace_xfs_buf_wait_buftarg (...) ((void) 0) +#define trace_xfs_buf_delwri_queued (...) ((void) 0) +#define trace_xfs_buf_delwri_queue (...) ((void) 0) +#define trace_xfs_buf_delwri_split (...) ((void) 0) +#define trace_xfs_buf_delwri_pushbuf (...) ((void) 0) +#define trace_xfs_buf_get_uncached (...) ((void) 0) +#endif /* __TRACE_H__ */
libxfs/Makefile +2 -0
@@ -30,6 +30,7 @@ HFILES = \ xfs_bmap_btree.h \ xfs_btree.h \ xfs_btree_staging.h \ + xfs_buf.h \ xfs_buftarg.h \ xfs_attr_remote.h \ xfs_cksum.h \
@@ -76,6 +77,7 @@ CFILES = buftarg.c \ xfs_bmap_btree.c \ xfs_btree.c \ xfs_btree_staging.c \ + xfs_buf.c \ xfs_da_btree.c \ xfs_defer.c \ xfs_dir2.c \
libxfs/buftarg.c +22 -32
67 lines not on this page · raw message
libxfs/libxfs_api_defs.h +2 -0
@@ -49,8 +49,10 @@#define xfs_btree_init_block libxfs_btree_init_block#define xfs_buf_delwri_submit libxfs_buf_delwri_submit#define xfs_buf_get libxfs_buf_get +#define xfs_buf_get_map libxfs_buf_get_map#define xfs_buf_get_uncached libxfs_buf_get_uncached#define xfs_buf_read libxfs_buf_read +#define xfs_buf_read_map libxfs_buf_read_map#define xfs_buf_read_uncached libxfs_buf_read_uncached#define xfs_buf_relse libxfs_buf_relse#define xfs_bunmapi libxfs_bunmapi
libxfs/libxfs_io.h +1 -95
129 lines not on this page · raw message
libxfs/libxfs_priv.h +1 -42
75 lines not on this page · raw message
libxfs/rdwr.c +2 -2
@@ -498,7 +498,7 @@ libxfs_buf_get_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps, - int flags, + xfs_buf_flags_t flags, struct xfs_buf **bpp) { int error;
@@ -640,7 +640,7 @@ libxfs_buf_read_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps, - int flags, + xfs_buf_flags_t flags, struct xfs_buf **bpp, const struct xfs_buf_ops *ops) {
libxfs/xfs_buf.c new file +1350 -0
1,350 lines not on this page · raw message
libxfs/xfs_buf.h new file +203 -0
203 lines not on this page · raw message
libxfs/xfs_buftarg.h +29 -2
@@ -85,9 +85,13 @@ int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr, size_t bblen, int flags, struct xfs_buf **bpp, const struct xfs_buf_ops *ops); -int xfs_bread (struct xfs_buf *bp,size_t bblen); +void xfs_buftarg_submit_io (struct xfs_buf *bp); -int xfs_bwrite (struct xfs_buf *bp); +/* +* Cached buffer memory manangement +*/ +int xfs_buf_allocate_memory (struct xfs_buf *bp,uint flags); +void xfs_buf_free_memory (struct xfs_buf *bp);/* * Temporary: these need to be the same as the LIBXFS_B_* flags until we change
@@ -99,6 +103,23 @@ int xfs_bwrite(struct xfs_buf *bp);#define XBF_DONE (1 << 3) // LIBXFS_B_UPTODATE 0x0008#define XBF_STALE (1 << 2) // LIBXFS_B_STALE 0x0004 +#define XBF_READ_AHEAD (1 << 30)/* asynchronous read-ahead */ +#define XBF_NO_IOACCT (1 << 29)/* bypass I/O accounting (non-LRU bufs) */ +#define XBF_ASYNC (1 << 28)/* initiator will not wait for completion */ +#define XBF_WRITE_FAIL (0)/* unused in userspace */ + +/* buffer type flags for write callbacks */ +#define _XBF_INODES (0)/* inode buffer */ +#define _XBF_DQUOTS (0)/* dquot buffer */ +#define _XBF_LOGRECOVERY (0)/* log recovery buffer */ + +/* flags used only as arguments to access routines */ +#define XBF_TRYLOCK (1 << 16)/* lock requested, but do not wait */ +#define XBF_UNMAPPED (0)/* unused in userspace */ + +/* flags used only internally */ +#define _XBF_DELWRI_Q (1 << 22)/* buffer on a delwri queue */ +/* * Raw buffer access functions. These exist as temporary bridges for uncached IO * that uses direct access to the buffers to submit IO. These will go away with
@@ -107,6 +128,12 @@ int xfs_bwrite(struct xfs_buf *bp);struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen); +/* temporary, just for compile for the moment */ +#define xfs_buf_ioend_async (bp) xfs_buf_ioend(bp) +#define bdi_read_congested (bdi) (false) +#define xfs_buf_ispinned (bp) (false) +#define xfs_buf_wait_unpin (bp) ((void)0) +/* * Hash cache implementation */
Signature · 1 line
2.28.0
Re: [PATCH 22/27] libxfs: introduce new buffer cache infrastructure
Quoted 23 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Now we have a separate buftarg infrastructure, we can introduce the kernel equivalent buffer cache infrastructure. This will (eventually) be shared with the kernel implementation, with all the differences being implemented in the private buftarg implementions.
Add the high level buffer lookup, IO and reference counting functions to
libxfs/xfs_buf.cand the low level infrastructure this requires tolibxfs/xfs_buftarg.cin preparation for switching the entire of xfsprogs over to using the new infrastructure.The eventual shared kernel/userspace buffer definitions will end up in
xfs_buf.h, so we start building that here to enable the code to compile and sort of function. It gets somewhat messy at this point, but patches after this can get on with switching over the cache implementation and cleaning up the mess.Note: this uses an older
xfs_buf.cfrom the kernel code, so there will be updates needed to bring it up to date once the initial conversion is complete.Signed-off-by Dave Chinner <[email protected]>
Quoted 13 lines
--- include/libxfs.h | 1 + include/xfs.h | 23 + include/xfs_mount.h | 1 + include/xfs_trace.h | 24 + libxfs/Makefile | 2 + libxfs/buftarg.c | 54 +- libxfs/libxfs_api_defs.h | 2 + libxfs/libxfs_io.h | 96 +-- libxfs/libxfs_priv.h | 43 +- libxfs/rdwr.c | 4 +- libxfs/xfs_buf.c | 1350 ++++++++++++++++++++++++++++++++++++++ libxfs/xfs_buf.h | 203 ++++++
Hmm, are you planning to move xfs_buf.c to libxfs in the kernel too?
Quoted 68 lines
libxfs/xfs_buftarg.h | 31 +- 13 files changed, 1661 insertions(+), 173 deletions(-) create mode 100644 libxfs/xfs_buf.c create mode 100644 libxfs/xfs_buf.hinclude/libxfs.h +1
-0@@ -52,6 +52,7 @@ struct iomap;* every files via a similar include in the kernel xfs_linux.h. */ #include "xfs_buftarg.h" +#include "xfs_buf.h" #include "libxfs_io.h" #include "xfs_bit.h" include/xfs.h +23
-0@@ -38,6 +38,29 @@ extern int xfs_assert_largefile[sizeof(off_t)-8];#define BUILD_BUG_ON (condition) ((void)sizeof(char[1 - 2*!!(condition)]))#endif +/* +* Kernel side compiler address and barrier functionality we use +*/ +#ifdef __GNUC__ +#define __return_address __builtin_return_address(0) + +/* +* Return the address of a label. Use barrier() so that the optimizer +* won't reorder code to refactor the error jumpouts into a single +* return, which throws off the reported address. +*/ +#define __this_address ({ __label__ __here; __here: barrier(); &&__here; }) +/* Optimization barrier */ + +/* The "volatile" is due to gcc bugs */ +#define barrier () __asm__ __volatile__("": : :"memory") +#endif + +/* Optimization barrier */ +#ifndef barrier +# define barrier () __memory_barrier() +#endif +#include <xfs/xfs_types.h> /* Include deprecated/compat pre-vfs xfs-specific symbols */ #include <xfs/xfs_fs_compat.h> include/xfs_mount.h +1
-0@@ -157,6 +157,7 @@ typedef struct xfs_perag {/* reference count */ uint8_t pagf_refcount_level; +spinlock_t pag_buf_lock;struct btcache *pag_buf_hash; } xfs_perag_t;include/xfs_trace.h +0
-0
include/xfs_trace.h line 316
Quoted 5 lines
include/xfs_trace.h +2
-0@@ -312,4 +312,28 @@#define trace_xfs_perag_get_tag (a,b,c,d) ((c) = (c))#define trace_xfs_perag_put (a,b,c,d) ((c) = (c)) +#define trace_xfs_buf_init (...) ((void) 0) +#define trace_xfs_buf_free (...) ((void) 0)
I started wondering if there was an easy way to hook all these up to some kind of actual tracepoints, but uprobes still seem to require the user to do /all/ the legwork of finding text offsets, decoding registers, etc. There seems to be some other library called usdt that can create static dtrace tracepoints, but it doesn't exist in Debian so... oh well.
--D
Quoted 1,999 lines
include/xfs_trace.h +22
-0+#define trace_xfs_buf_find (...) ((void) 0) +#define trace_xfs_buf_get (...) ((void) 0) +#define trace_xfs_buf_read (...) ((void) 0) +#define trace_xfs_buf_hold (...) ((void) 0) +#define trace_xfs_buf_rele (...) ((void) 0) +#define trace_xfs_buf_trylock (...) ((void) 0) +#define trace_xfs_buf_trylock_fail (...) ((void) 0) +#define trace_xfs_buf_lock (...) ((void) 0) +#define trace_xfs_buf_lock_done (...) ((void) 0) +#define trace_xfs_buf_unlock (...) ((void) 0) +#define trace_xfs_buf_iodone (...) ((void) 0) +#define trace_xfs_buf_ioerror (...) ((void) 0) +#define trace_xfs_buf_iowait (...) ((void) 0) +#define trace_xfs_buf_iowait_done (...) ((void) 0) +#define trace_xfs_buf_submit (...) ((void) 0) +#define trace_xfs_buf_wait_buftarg (...) ((void) 0) +#define trace_xfs_buf_delwri_queued (...) ((void) 0) +#define trace_xfs_buf_delwri_queue (...) ((void) 0) +#define trace_xfs_buf_delwri_split (...) ((void) 0) +#define trace_xfs_buf_delwri_pushbuf (...) ((void) 0) +#define trace_xfs_buf_get_uncached (...) ((void) 0) +#endif /* __TRACE_H__ */ libxfs/Makefile +2
-0@@ -30,6 +30,7 @@ HFILES = \ xfs_bmap_btree.h \ xfs_btree.h \ xfs_btree_staging.h \ + xfs_buf.h \ xfs_buftarg.h \ xfs_attr_remote.h \ xfs_cksum.h \@@ -76,6 +77,7 @@ CFILES = buftarg.c \ xfs_bmap_btree.c \ xfs_btree.c \ xfs_btree_staging.c \ + xfs_buf.c \ xfs_da_btree.c \ xfs_defer.c \ xfs_dir2.c \libxfs/buftarg.c +22
-32@@ -104,45 +104,35 @@ xfs_buftarg_free( }/* -* Low level IO routines +* Buffer memory buffer allocation routines +* +* Userspace just has a flat memory buffer, so it's quite simple compared +* to the kernel code. */ -static void -xfs_buf_ioend ( +void +xfs_buf_free_memory (struct xfs_buf *bp) { -bool read = bp->b_flags &XBF_READ ; - -// printf("endio bn %ld l %d/%d, io err %d err %d f 0x%x\n", bp->b_maps[0].bm_bn, -// bp->b_maps[0].bm_len, BBTOB(bp->b_length), -// bp->b_io_error, bp->b_error, bp->b_flags); - - bp->b_flags &= ~(XBF_READ |XBF_WRITE ); - -/* -* Pull in IO completion errors now. We are guaranteed to be running -* single threaded, so we don't need the lock to read b_io_error. -*/ -if (!bp->b_error && bp->b_io_error) -xfs_buf_ioerror (bp, bp->b_io_error); +free (bp->b_addr); +} -/* Only validate buffers that were read without errors */ -if (read && !bp->b_error && bp->b_ops) { -ASSERT (!bp->b_iodone); - bp->b_ops->verify_read (bp); - } +int +xfs_buf_allocate_memory ( +struct xfs_buf *bp, +uint flags) +{ +size_t size; -if (!bp->b_error) { - bp->b_flags |=XBF_DONE ; - bp->b_flags &= ~(LIBXFS_B_DIRTY |LIBXFS_B_UNCHECKED ); - }else { -fprintf (stderr, -_ ("%s: IO failed on %s bno 0x%llx/0x%x, err=%d\n" ), - __func__, bp->b_ops ? bp->b_ops->name :"(unknown)" , - (long long )bp->b_maps[0 ].bm_bn, bp->b_length, - -bp->b_error); - } + size =BBTOB (bp->b_length); + bp->b_addr =memalign (bp->b_target->bt_meta_sectorsize, size); +if (!bp->b_addr) +return -ENOMEM ; +return 0 ; } +/* +* Low level IO routines +*/ static void xfs_buf_complete_io( struct xfs_buf *bp,libxfs/libxfs_api_defs.h +2
-0@@ -49,8 +49,10 @@#define xfs_btree_init_block libxfs_btree_init_block#define xfs_buf_delwri_submit libxfs_buf_delwri_submit#define xfs_buf_get libxfs_buf_get +#define xfs_buf_get_map libxfs_buf_get_map#define xfs_buf_get_uncached libxfs_buf_get_uncached#define xfs_buf_read libxfs_buf_read +#define xfs_buf_read_map libxfs_buf_read_map#define xfs_buf_read_uncached libxfs_buf_read_uncached#define xfs_buf_relse libxfs_buf_relse#define xfs_bunmapi libxfs_bunmapilibxfs/libxfs_io.h +1
-95@@ -10,7 +10,6 @@/* * Kernel equivalent buffer based I/O interface */ -struct xfs_buf ;struct xfs_mount ;struct xfs_perag ;@@ -20,60 +19,7 @@ void libxfs_open_devices(struct xfs_mount *mp, dev_t ddev, dev_t logdev, dev_t rtdev);int libxfs_blkdev_issue_flush (struct xfs_buftarg *btp); -#define LIBXFS_BBTOOFF64 (bbs) (((xfs_off_t)(bbs)) << BBSHIFT) - -#define XB_PAGES 2 -struct xfs_buf_map { -xfs_daddr_t bm_bn;/* block number for I/O */ -int bm_len;/* size of I/O */ -}; - -#define DEFINE_SINGLE_BUF_MAP (map, blkno, numblk) \ - struct xfs_buf_map (map) = { .bm_bn = (blkno), .bm_len = (numblk) }; - -struct xfs_buf_ops { -char *name; -union { -__be32 magic[2 ];/* v4 and v5 on disk magic values */ -__be16 magic16[2 ];/* v4 and v5 on disk magic values */ - }; -void (*verify_read)(struct xfs_buf *); -void (*verify_write)(struct xfs_buf *); -xfs_failaddr_t (*verify_struct)(struct xfs_buf *); -}; - -#define XFS_BSTATE_DISPOSE (1 << 0)/* buffer being discarded */ - -struct xfs_buf { -struct cache_node b_node; -struct list_head b_hash;/* will replace b_node */ -xfs_daddr_t b_bn; -unsigned int b_length; -unsigned int b_flags; -struct xfs_buftarg *b_target; -pthread_mutex_t b_lock; -pthread_t b_holder; -unsigned int b_recur; -void *b_log_item; -struct list_head b_li_list;/* Log items list head */ -void *b_transp; -void *b_addr; -int b_error; -const struct xfs_buf_ops *b_ops; -struct xfs_perag *b_pag; -struct xfs_mount *b_mount; -struct xfs_buf_map *b_maps; -struct xfs_buf_map __b_map; -int b_map_count; -int b_io_remaining; -int b_io_error; -struct list_head b_list; - -struct list_head b_btc_list; -unsigned int b_state; -atomic_t b_lru_ref; -struct list_head b_lru; -}; +#define LIBXFS_BBTOOFF64 (bbs) (((xfs_off_t)(bbs)) << BBSHIFT)bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic);bool xfs_verify_magic16 (struct xfs_buf *bp,__be16 dmagic);@@ -93,9 +39,6 @@ typedef unsigned int xfs_buf_flags_t;void libxfs_buf_set_priority (struct xfs_buf *bp,int priority);int libxfs_buf_priority (struct xfs_buf *bp); -#define xfs_buf_set_ref (bp,ref) ((void) 0) -#define xfs_buf_ioerror (bp,err) ((bp)->b_error = (err)) -#define xfs_daddr_to_agno (mp,d) \ ((xfs_agnumber_t)(XFS_BB_TO_FSBT(mp, d) / (mp)->m_sb.sb_agblocks))#define xfs_daddr_to_agbno (mp,d) \@@ -113,40 +56,9 @@ void libxfs_brelse(struct cache_node *node);/* Return the buffer even if the verifiers fail. */ #define LIBXFS_READBUF_SALVAGE (1 << 1) -int libxfs_buf_read_map (struct xfs_buftarg *btp,struct xfs_buf_map *maps, -int nmaps,int flags,struct xfs_buf **bpp, -const struct xfs_buf_ops *ops);void libxfs_buf_mark_dirty (struct xfs_buf *bp); -int libxfs_buf_get_map (struct xfs_buftarg *btp,struct xfs_buf_map *maps, -int nmaps,int flags,struct xfs_buf **bpp);void libxfs_buf_relse (struct xfs_buf *bp); -static inline int -libxfs_buf_get ( -struct xfs_buftarg *target, -xfs_daddr_t blkno, -size_t numblks, -struct xfs_buf **bpp) -{ -DEFINE_SINGLE_BUF_MAP (map, blkno, numblks); - -return libxfs_buf_get_map (target, &map,1 ,0 , bpp); -} - -static inline int -libxfs_buf_read ( -struct xfs_buftarg *target, -xfs_daddr_t blkno, -size_t numblks, -xfs_buf_flags_t flags, -struct xfs_buf **bpp, -const struct xfs_buf_ops *ops) -{ -DEFINE_SINGLE_BUF_MAP (map, blkno, numblks); - -return libxfs_buf_read_map (target, &map,1 , flags, bpp, ops); -} -int libxfs_readbuf_verify (struct xfs_buf *bp,const struct xfs_buf_ops *ops);struct xfs_buf *libxfs_getsb (struct xfs_mount *mp);extern void libxfs_bcache_purge (void );@@ -173,12 +85,6 @@ xfs_buf_update_cksum(struct xfs_buf *bp, unsigned long cksum_offset) cksum_offset); } -static inline void -xfs_buf_hold (struct xfs_buf *bp) -{ - bp->b_node.cn_count++; -} -/* Push a single buffer on a delwri queue. */ static inline bool xfs_buf_delwri_queue (struct xfs_buf *bp,struct list_head *buffer_list)libxfs/libxfs_priv.h +1
-42@@ -87,6 +87,7 @@ struct iomap;* every files via a similar include in the kernel xfs_linux.h. */ #include "xfs_buftarg.h" +#include "xfs_buf.h" #include "libxfs_io.h" /* for all the support code that uses progname in error messages */ @@ -131,8 +132,6 @@ enum ce { CE_DEBUG, CE_CONT, CE_NOTE, CE_WARN, CE_ALERT, CE_PANIC }; #define xfs_err(mp,fmt,args...) cmn_err(CE_ALERT, _(fmt), ## args) #define xfs_alert(mp,fmt,args...) cmn_err(CE_ALERT, _(fmt), ## args) -#define xfs_buf_ioerror_alert (bp,f) ((void) 0); -#define xfs_hex_dump (d,n) ((void) 0)#define xfs_stack_trace () ((void) 0)@@ -173,25 +172,6 @@ enum ce { CE_DEBUG, CE_CONT, CE_NOTE, CE_WARN, CE_ALERT, CE_PANIC };#define XFS_STATS_ADD (mp, count, x) do { (mp) = (mp); } while (0)#define XFS_TEST_ERROR (expr,a,b) ( expr ) -#ifdef __GNUC__ -#define __return_address __builtin_return_address(0) - -/* -* Return the address of a label. Use barrier() so that the optimizer -* won't reorder code to refactor the error jumpouts into a single -* return, which throws off the reported address. -*/ -#define __this_address ({ __label__ __here; __here: barrier(); &&__here; }) -/* Optimization barrier */ - -/* The "volatile" is due to gcc bugs */ -#define barrier () __asm__ __volatile__("": : :"memory") -#endif - -/* Optimization barrier */ -#ifndef barrier -# define barrier () __memory_barrier() -#endif /* miscellaneous kernel routines not in user space */ #define likely (x) (x)@@ -407,22 +387,8 @@ howmany_64(uint64_t x, uint32_t y) }/* buffer management */ -#define XBF_TRYLOCK 0 -#define XBF_UNMAPPED 0 -#define xfs_buf_stale (bp) ((bp)->b_flags |= XBF_STALE)#define XFS_BUF_UNDELAYWRITE (bp) ((bp)->b_flags &= ~LIBXFS_B_DIRTY) -/* buffer type flags for write callbacks */ -#define _XBF_INODES 0/* inode buffer */ -#define _XBF_DQUOTS 0/* dquot buffer */ -#define _XBF_LOGRECOVERY 0/* log recovery buffer */ - -static inline struct xfs_buf *xfs_buf_incore (struct xfs_buftarg *target, -xfs_daddr_t blkno,size_t numblks,xfs_buf_flags_t flags) -{ -return NULL ; -} -#define xfs_buf_oneshot (bp) ((void) 0)#define xfs_buf_zero (bp, off, len) \@@ -454,13 +420,6 @@ void __xfs_buf_mark_corrupt(struct xfs_buf *bp, xfs_failaddr_t fa);#define xfs_trans_buf_copy_type (dbp, sbp) -/* no readahead, need to avoid set-but-unused var warnings. */ -#define xfs_buf_readahead(a,d,c,ops) ({ \ - xfs_daddr_t __d = d; \ - __d = __d; /* no set-but-unused warning */ \ -}) -#define xfs_buf_readahead_map (a,b,c,ops) ((void) 0)/* no readahead */ -#define xfs_sort qsort#define xfs_ilock (ip,mode) ((void) 0)libxfs/rdwr.c +2
-2@@ -498,7 +498,7 @@ libxfs_buf_get_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps, - int flags, + xfs_buf_flags_t flags, struct xfs_buf **bpp) { int error;@@ -640,7 +640,7 @@ libxfs_buf_read_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps, - int flags, + xfs_buf_flags_t flags, struct xfs_buf **bpp, const struct xfs_buf_ops *ops) {libxfs/xfs_buf.c new file +1350
-0@@ -0,0 +1,1350 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2000-2006 Silicon Graphics, Inc. +* Copyright (c) 2019-2020 Red Hat, Inc. +* All Rights Reserved. +*/ +#include "libxfs_priv.h" +#include "xfs_buf.h" // temporary +#include "init.h" +#include "xfs_fs.h" +#include "xfs_format.h" +#include "xfs_log_format.h" +#include "xfs_shared.h" +#include "xfs_trans_resv.h" +#include "xfs_sb.h" +#include "xfs_mount.h" +#include "xfs_trace.h" +#include "xfs_errortag.h" +#include "xfs_errortag.h" + +#include <libaio.h> + +#include "libxfs.h" /* libxfs_device_to_fd */ + +//struct kmem_zone *xfs_buf_zone; + +/* +* Locking orders +* +* xfs_buf_ioacct_inc: +* xfs_buf_ioacct_dec: +* b_sema (caller holds) +* b_lock +* +* xfs_buf_stale: +* b_sema (caller holds) +* b_lock +* lru_lock +* +* xfs_buf_rele: +* b_lock +* pag_buf_lock +* lru_lock +* +* xfs_buftarg_wait_rele +* lru_lock +* b_lock (trylock due to inversion) +* +* xfs_buftarg_isolate +* lru_lock +* b_lock (trylock due to inversion) +*/ + +/* +* Bump the I/O in flight count on the buftarg if we haven't yet done so for +* this buffer. The count is incremented once per buffer (per hold cycle) +* because the corresponding decrement is deferred to buffer release. Buffers +* can undergo I/O multiple times in a hold-release cycle and per buffer I/O +* tracking adds unnecessary overhead. This is used for sychronization purposes +* with unmount (see xfs_wait_buftarg()), so all we really need is a count of +* in-flight buffers. +* +* Buffers that are never released (e.g., superblock, iclog buffers) must set +* the XBF_NO_IOACCT flag before I/O submission. Otherwise, the buftarg count +* never reaches zero and unmount hangs indefinitely. +*/ +static inline void +xfs_buf_ioacct_inc ( +struct xfs_buf *bp) +{ +if (bp->b_flags &XBF_NO_IOACCT ) +return ; + +ASSERT (bp->b_flags &XBF_ASYNC ); +spin_lock (&bp->b_lock); +if (!(bp->b_state &XFS_BSTATE_IN_FLIGHT )) { + bp->b_state |=XFS_BSTATE_IN_FLIGHT ; +atomic_inc (&bp->b_target->bt_io_count); + } +spin_unlock (&bp->b_lock); +} + +/* +* Clear the in-flight state on a buffer about to be released to the LRU or +* freed and unaccount from the buftarg. +*/ +static inline void +__xfs_buf_ioacct_dec ( +struct xfs_buf *bp) +{ +if (bp->b_state &XFS_BSTATE_IN_FLIGHT ) { + bp->b_state &= ~XFS_BSTATE_IN_FLIGHT ; +atomic_dec (&bp->b_target->bt_io_count); + } +} + +static inline void +xfs_buf_ioacct_dec ( +struct xfs_buf *bp) +{ +spin_lock (&bp->b_lock); +__xfs_buf_ioacct_dec (bp); +spin_unlock (&bp->b_lock); +} + +/* +* When we mark a buffer stale, we remove the buffer from the LRU and clear the +* b_lru_ref count so that the buffer is freed immediately when the buffer +* reference count falls to zero. If the buffer is already on the LRU, we need +* to remove the reference that LRU holds on the buffer. +* +* This prevents build-up of stale buffers on the LRU. +*/ +void +xfs_buf_stale ( +struct xfs_buf *bp) +{ +ASSERT (xfs_buf_islocked (bp)); + + bp->b_flags |=XBF_STALE ; + +/* +* Clear the delwri status so that a delwri queue walker will not +* flush this buffer to disk now that it is stale. The delwri queue has +* a reference to the buffer, so this is safe to do. +*/ + bp->b_flags &= ~_XBF_DELWRI_Q; + +/* +* Once the buffer is marked stale and unlocked, a subsequent lookup +* could reset b_flags. There is no guarantee that the buffer is +* unaccounted (released to LRU) before that occurs. Drop in-flight +* status now to preserve accounting consistency. +*/ +spin_lock (&bp->b_lock); +__xfs_buf_ioacct_dec (bp); + +atomic_set (&bp->b_lru_ref,0 ); +if (!(bp->b_state &XFS_BSTATE_DISPOSE ) && + (list_lru_del (&bp->b_target->bt_lru, &bp->b_lru))) +atomic_dec (&bp->b_hold); + +ASSERT (atomic_read (&bp->b_hold) >=1 ); +spin_unlock (&bp->b_lock); +} + +#ifdef NOT_YET +static int +xfs_buf_get_maps ( +struct xfs_buf *bp, +int map_count) +{ +ASSERT (bp->b_maps ==NULL ); + bp->b_map_count = map_count; + +if (map_count ==1 ) { + bp->b_maps = &bp->__b_map; +return 0 ; + } + + bp->b_maps =kmem_zalloc (map_count *sizeof (struct xfs_buf_map ), +KM_NOFS ); +if (!bp->b_maps) +return -ENOMEM ; +return 0 ; +} +#endif /* not yet */ + +static void +xfs_buf_free_maps ( +struct xfs_buf *bp) +{ +if (bp->b_maps != &bp->__b_map) { +kmem_free (bp->b_maps); + bp->b_maps =NULL ; + } +} + +#ifdef NOT_YET +static int +_xfs_buf_alloc ( +struct xfs_buftarg *target, +struct xfs_buf_map *map, +int nmaps, +xfs_buf_flags_t flags, +struct xfs_buf **bpp) +{ +struct xfs_buf *bp; +int error; +int i; + + *bpp =NULL ; + bp =kmem_zone_zalloc (xfs_buf_zone,KM_NOFS ); + +/* +* We don't want certain flags to appear in b_flags unless they are +* specifically set by later operations on the buffer. +*/ + flags &= ~(XBF_UNMAPPED |XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD ); + +atomic_set (&bp->b_hold,1 ); +atomic_set (&bp->b_lru_ref,1 ); +init_completion (&bp->b_iowait); +INIT_LIST_HEAD (&bp->b_lru); +INIT_LIST_HEAD (&bp->b_list); +INIT_LIST_HEAD (&bp->b_li_list); +INIT_LIST_HEAD (&bp->b_btc_list); +sema_init (&bp->b_sema,0 );/* held, no waiters */ +spin_lock_init (&bp->b_lock); + bp->b_target = target; + bp->b_mount = target->bt_mount; + bp->b_flags = flags; + +/* +* Set length and io_length to the same value initially. +* I/O routines should use io_length, which will be the same in +* most cases but may be reset (e.g. XFS recovery). +*/ + error =xfs_buf_get_maps (bp, nmaps); +if (error) { +kmem_cache_free (xfs_buf_zone, bp); +return error; + } + + bp->b_bn = map[0 ].bm_bn; + bp->b_length =0 ; +for (i =0 ; i < nmaps; i++) { + bp->b_maps[i].bm_bn = map[i].bm_bn; + bp->b_maps[i].bm_len = map[i].bm_len; + bp->b_length += map[i].bm_len; + } + +XFS_STATS_INC (bp->b_mount, xb_create); +trace_xfs_buf_init (bp, _RET_IP_); + + *bpp = bp; +return 0 ; +} +#endif /* not yet */ + +/* +* Releases the specified buffer. +* +* The modification state of any associated pages is left unchanged. The buffer +* must not be on any hash - use xfs_buf_rele instead for hashed and refcounted +* buffers +*/ +void +xfs_buf_free ( +struct xfs_buf *bp) +{ +trace_xfs_buf_free (bp, _RET_IP_); + +ASSERT (list_empty (&bp->b_lru)); +xfs_buf_free_memory (bp); +xfs_buf_free_maps (bp); +kmem_cache_free (xfs_buf_zone, bp); +} + +/* +* Look up a buffer in the buffer cache and return it referenced and locked +* in @found_bp. +* +* If @new_bp is supplied and we have a lookup miss, insert @new_bp into the +* cache. +* +* If XBF_TRYLOCK is set in @flags, only try to lock the buffer and return +* -EAGAIN if we fail to lock it. +* +* Return values are: +* -EFSCORRUPTED if have been supplied with an invalid address +* -EAGAIN on trylock failure +* -ENOENT if we fail to find a match and @new_bp was NULL +* 0, with @found_bp: +* - @new_bp if we inserted it into the cache +* - the buffer we found and locked. +*/ +static int +xfs_buf_find ( +struct xfs_buftarg *btp, +struct xfs_buf_map *map, +int nmaps, +xfs_buf_flags_t flags, +struct xfs_buf *new_bp, +struct xfs_buf **found_bp) +{ +struct xfs_perag *pag; +struct xfs_buf *bp; +struct xfs_buf_map cmap = { .bm_bn = map[0 ].bm_bn }; +xfs_daddr_t eofs; +int i; + + *found_bp =NULL ; + +for (i =0 ; i < nmaps; i++) + cmap.bm_len += map[i].bm_len; + +/* Check for IOs smaller than the sector size / not sector aligned */ +ASSERT (!(BBTOB (cmap.bm_len) < btp->bt_meta_sectorsize)); +ASSERT (!(BBTOB (cmap.bm_bn) & (xfs_off_t )btp->bt_meta_sectormask)); + +/* +* Corrupted block numbers can get through to here, unfortunately, so we +* have to check that the buffer falls within the filesystem bounds. +*/ + eofs =XFS_FSB_TO_BB (btp->bt_mount, btp->bt_mount->m_sb.sb_dblocks); +if (cmap.bm_bn <0 || cmap.bm_bn >= eofs) { +xfs_alert (btp->bt_mount, +"%s: daddr 0x%llx out of range, EOFS 0x%llx" , + __func__, cmap.bm_bn, eofs); +WARN_ON (1 ); +return -EFSCORRUPTED ; + } + + pag =xfs_perag_get (btp->bt_mount, +xfs_daddr_to_agno (btp->bt_mount, cmap.bm_bn)); + +spin_lock (&pag->pag_buf_lock); + bp =btc_node_find (pag->pag_buf_hash, &cmap); +if (bp) { +atomic_inc (&bp->b_hold); + goto found; + } + +/* No match found */ +if (!new_bp) { +XFS_STATS_INC (btp->bt_mount, xb_miss_locked); +spin_unlock (&pag->pag_buf_lock); +xfs_perag_put (pag); +return -ENOENT ; + } + +/* the buffer keeps the perag reference until it is freed */ + new_bp->b_pag = pag; +btc_node_insert (pag->pag_buf_hash, new_bp); +spin_unlock (&pag->pag_buf_lock); + *found_bp = new_bp; +return 0 ; + +found: +spin_unlock (&pag->pag_buf_lock); +xfs_perag_put (pag); + +if (!xfs_buf_trylock (bp)) { +if (flags &XBF_TRYLOCK ) { +xfs_buf_rele (bp); +XFS_STATS_INC (btp->bt_mount, xb_busy_locked); +return -EAGAIN ; + } +xfs_buf_lock (bp); +XFS_STATS_INC (btp->bt_mount, xb_get_locked_waited); + } + +/* +* if the buffer is stale, clear all the external state associated with +* it. We need to keep flags such as how we allocated the buffer memory +* intact here. +*/ +if (bp->b_flags &XBF_STALE ) { +ASSERT ((bp->b_flags & _XBF_DELWRI_Q) ==0 ); +ASSERT (bp->b_iodone ==NULL ); + bp->b_ops =NULL ; + } + +trace_xfs_buf_find (bp, flags, _RET_IP_); +XFS_STATS_INC (btp->bt_mount, xb_get_locked); + *found_bp = bp; +return 0 ; +} + +struct xfs_buf * +xfs_buf_incore ( +struct xfs_buftarg *target, +xfs_daddr_t blkno, +size_t numblks, +xfs_buf_flags_t flags) +{ +struct xfs_buf *bp; +int error; +DEFINE_SINGLE_BUF_MAP (map, blkno, numblks); + + error =xfs_buf_find (target, &map,1 , flags,NULL , &bp); +if (error) +return NULL ; +return bp; +} + + +/* +* Assembles a buffer covering the specified range. The code is optimised for +* cache hits, as metadata intensive workloads will see 3 orders of magnitude +* more hits than misses. +*/ +#ifdef NOT_YET +int +xfs_buf_get_map ( +struct xfs_buftarg *target, +struct xfs_buf_map *map, +int nmaps, +xfs_buf_flags_t flags, +struct xfs_buf **bpp) +{ +struct xfs_buf *bp; +struct xfs_buf *new_bp; +int error =0 ; + + *bpp =NULL ; + error =xfs_buf_find (target, map, nmaps, flags,NULL , &bp); +if (!error) + goto found; +if (error != -ENOENT ) +return error; + + error =_xfs_buf_alloc (target, map, nmaps, flags, &new_bp); +if (error) +return error; + + error =xfs_buf_allocate_memory (new_bp, flags); +if (error) { +xfs_buf_free (new_bp); +return error; + } + + error =xfs_buf_find (target, map, nmaps, flags, new_bp, &bp); +if (error) { +xfs_buf_free (new_bp); +return error; + } + +if (bp != new_bp) +xfs_buf_free (new_bp); + +found: +/* +* Clear b_error if this is a lookup from a caller that doesn't expect +* valid data to be found in the buffer. +*/ +if (!(flags &XBF_READ )) +xfs_buf_ioerror (bp,0 ); + +XFS_STATS_INC (target->bt_mount, xb_get); +trace_xfs_buf_get (bp, flags, _RET_IP_); + *bpp = bp; +return 0 ; +} + +STATIC int +_xfs_buf_read ( +struct xfs_buf *bp, +xfs_buf_flags_t flags) +{ +ASSERT (!(flags &XBF_WRITE )); +ASSERT (bp->b_maps[0 ].bm_bn !=XFS_BUF_DADDR_NULL ); + + bp->b_flags &= ~(XBF_WRITE |XBF_ASYNC |XBF_READ_AHEAD ); + bp->b_flags |= flags & (XBF_READ |XBF_ASYNC |XBF_READ_AHEAD ); + +return xfs_buf_submit (bp); +} +#endif /* not yet */ + +/* +* Reverify a buffer found in cache without an attached ->b_ops. +* +* If the caller passed an ops structure and the buffer doesn't have ops +* assigned, set the ops and use it to verify the contents. If verification +* fails, clear XBF_DONE. We assume the buffer has no recorded errors and is +* already in XBF_DONE state on entry. +* +* Under normal operations, every in-core buffer is verified on read I/O +* completion. There are two scenarios that can lead to in-core buffers without +* an assigned ->b_ops. The first is during log recovery of buffers on a V4 +* filesystem, though these buffers are purged at the end of recovery. The +* other is online repair, which intentionally reads with a NULL buffer ops to +* run several verifiers across an in-core buffer in order to establish buffer +* type. If repair can't establish that, the buffer will be left in memory +* with NULL buffer ops. +*/ +int +xfs_buf_reverify ( +struct xfs_buf *bp, +const struct xfs_buf_ops *ops) +{ +ASSERT (bp->b_flags &XBF_DONE ); +ASSERT (bp->b_error ==0 ); + +if (!ops || bp->b_ops) +return 0 ; + + bp->b_ops = ops; + bp->b_ops->verify_read (bp); +if (bp->b_error) + bp->b_flags &= ~XBF_DONE ; +return bp->b_error; +} + +#ifdef NOT_YET +int +xfs_buf_read_map ( +struct xfs_buftarg *target, +struct xfs_buf_map *map, +int nmaps, +xfs_buf_flags_t flags, +struct xfs_buf **bpp, +const struct xfs_buf_ops *ops) +{ +struct xfs_buf *bp; +int error; + + flags |=XBF_READ ; + *bpp =NULL ; + + error =xfs_buf_get_map (target, map, nmaps, flags, &bp); +if (error) +return error; + +trace_xfs_buf_read (bp, flags, _RET_IP_); + +if (!(bp->b_flags &XBF_DONE )) { +/* Initiate the buffer read and wait. */ +XFS_STATS_INC (target->bt_mount, xb_get_read); + bp->b_ops = ops; + error =_xfs_buf_read (bp, flags); + +/* Readahead iodone already dropped the buffer, so exit. */ +if (flags &XBF_ASYNC ) +return 0 ; + }else { +/* Buffer already read; all we need to do is check it. */ + error =xfs_buf_reverify (bp, ops); + +/* Readahead already finished; drop the buffer and exit. */ +if (flags &XBF_ASYNC ) { +xfs_buf_relse (bp); +return 0 ; + } + +/* We do not want read in the flags */ + bp->b_flags &= ~XBF_READ ; +ASSERT (bp->b_ops !=NULL || ops ==NULL ); + } + +/* +* If we've had a read error, then the contents of the buffer are +* invalid and should not be used. To ensure that a followup read tries +* to pull the buffer from disk again, we clear the XBF_DONE flag and +* mark the buffer stale. This ensures that anyone who has a current +* reference to the buffer will interpret it's contents correctly and +* future cache lookups will also treat it as an empty, uninitialised +* buffer. +*/ +if (error) { +if (!XFS_FORCED_SHUTDOWN (target->bt_mount)) +xfs_buf_ioerror_alert (bp, __this_address); + + bp->b_flags &= ~XBF_DONE ; +xfs_buf_stale (bp); +xfs_buf_relse (bp); + +/* bad CRC means corrupted metadata */ +if (error == -EFSBADCRC ) + error = -EFSCORRUPTED ; +return error; + } + + *bpp = bp; +return 0 ; +} +#endif /* not yet */ + +/* +* If we are not low on memory then do the readahead in a deadlock +* safe manner. +*/ +void +xfs_buf_readahead_map ( +struct xfs_buftarg *target, +struct xfs_buf_map *map, +int nmaps, +const struct xfs_buf_ops *ops) +{ +struct xfs_buf *bp; + +if (bdi_read_congested (target->bt_bdev->bd_bdi)) +return ; + +xfs_buf_read_map (target, map, nmaps, +XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD , &bp, ops); +} + +/* +* Increment reference count on buffer, to hold the buffer concurrently +* with another thread which may release (free) the buffer asynchronously. +* Must hold the buffer already to call this function. +*/ +void +xfs_buf_hold ( +struct xfs_buf *bp) +{ +trace_xfs_buf_hold (bp, _RET_IP_); +atomic_inc (&bp->b_hold); + bp->b_node.cn_count++; +} + +/* +* Release a hold on the specified buffer. If the hold count is 1, the buffer is +* placed on LRU or freed (depending on b_lru_ref). +* +* XXX: purging via btc lru is broken in this code. Needs fixing. +*/ +void +xfs_buf_rele ( +struct xfs_buf *bp) +{ +struct xfs_perag *pag = bp->b_pag; +bool release; +bool freebuf = false; + +trace_xfs_buf_rele (bp, _RET_IP_); + +if (!pag) { +ASSERT (list_empty (&bp->b_lru)); +if (atomic_dec_and_test (&bp->b_hold)) { +xfs_buf_ioacct_dec (bp); +xfs_buf_free (bp); + } +return ; + } + +ASSERT (atomic_read (&bp->b_hold) >0 ); + +/* +* We grab the b_lock here first to serialise racing xfs_buf_rele() +* calls. The pag_buf_lock being taken on the last reference only +* serialises against racing lookups in xfs_buf_find(). IOWs, the second +* to last reference we drop here is not serialised against the last +* reference until we take bp->b_lock. Hence if we don't grab b_lock +* first, the last "release" reference can win the race to the lock and +* free the buffer before the second-to-last reference is processed, +* leading to a use-after-free scenario. +*/ +spin_lock (&bp->b_lock); + release =atomic_dec_and_lock (&bp->b_hold, &pag->pag_buf_lock); +if (!release) { +/* +* Drop the in-flight state if the buffer is already on the LRU +* and it holds the only reference. This is racy because we +* haven't acquired the pag lock, but the use of _XBF_IN_FLIGHT +* ensures the decrement occurs only once per-buf. +*/ +if ((atomic_read (&bp->b_hold) ==1 ) && !list_empty (&bp->b_lru)) +__xfs_buf_ioacct_dec (bp); + goto out_unlock; + } + +/* the last reference has been dropped ... */ +__xfs_buf_ioacct_dec (bp); +//if (!(bp->b_flags & XBF_STALE) && atomic_read(&bp->b_lru_ref)) { +if (0 ) { +/* +* If the buffer is added to the LRU take a new reference to the +* buffer for the LRU and clear the (now stale) dispose list +* state flag +*/ +if (list_lru_add (&bp->b_target->bt_lru, &bp->b_lru)) { + bp->b_state &= ~XFS_BSTATE_DISPOSE ; +atomic_inc (&bp->b_hold); + } +spin_unlock (&pag->pag_buf_lock); + }else { +/* +* most of the time buffers will already be removed from the +* LRU, so optimise that case by checking for the +* XFS_BSTATE_DISPOSE flag indicating the last list the buffer +* was on was the disposal list +*/ +if (!(bp->b_state &XFS_BSTATE_DISPOSE )) { +list_lru_del (&bp->b_target->bt_lru, &bp->b_lru); + }else { +ASSERT (list_empty (&bp->b_lru)); + } + +ASSERT (!(bp->b_flags & _XBF_DELWRI_Q)); +btc_node_remove (pag->pag_buf_hash, bp); +spin_unlock (&pag->pag_buf_lock); +xfs_perag_put (pag); + freebuf = true; + } + +out_unlock: +spin_unlock (&bp->b_lock); + +if (freebuf) +xfs_buf_free (bp); +} + + +/* +* Lock a buffer object, if it is not already locked. +* +* If we come across a stale, pinned, locked buffer, we know that we are +* being asked to lock a buffer that has been reallocated. Because it is +* pinned, we know that the log has not been pushed to disk and hence it +* will still be locked. Rather than continuing to have trylock attempts +* fail until someone else pushes the log, push it ourselves before +* returning. This means that the xfsaild will not get stuck trying +* to push on stale inode buffers. +*/ +int +xfs_buf_trylock ( +struct xfs_buf *bp) +{ +int locked; + + locked =down_trylock (&bp->b_sema) ==0 ; +if (locked) +trace_xfs_buf_trylock (bp, _RET_IP_); +else +trace_xfs_buf_trylock_fail (bp, _RET_IP_); +return locked; +} + +/* +* Lock a buffer object. +* +* If we come across a stale, pinned, locked buffer, we know that we +* are being asked to lock a buffer that has been reallocated. Because +* it is pinned, we know that the log has not been pushed to disk and +* hence it will still be locked. Rather than sleeping until someone +* else pushes the log, push it ourselves before trying to get the lock. +*/ +void +xfs_buf_lock ( +struct xfs_buf *bp) +{ +trace_xfs_buf_lock (bp, _RET_IP_); + +down (&bp->b_sema); + +trace_xfs_buf_lock_done (bp, _RET_IP_); +} + +void +xfs_buf_unlock ( +struct xfs_buf *bp) +{ +ASSERT (xfs_buf_islocked (bp)); + +up (&bp->b_sema); +trace_xfs_buf_unlock (bp, _RET_IP_); +} + +/* +* Buffer Utility Routines +*/ + +void +xfs_buf_ioend ( +struct xfs_buf *bp) +{ +bool read = bp->b_flags &XBF_READ ; + +trace_xfs_buf_iodone (bp, _RET_IP_); + +// printf("endio bn %ld l %d/%d, io err %d err %d f 0x%x\n", bp->b_maps[0].bm_bn, +// bp->b_maps[0].bm_len, BBTOB(bp->b_length), +// bp->b_io_error, bp->b_error, bp->b_flags); + + bp->b_flags &= ~(XBF_READ |XBF_WRITE |XBF_READ_AHEAD ); +/* +* Pull in IO completion errors now. We are guaranteed to be running +* single threaded, so we don't need the lock to read b_io_error. +*/ +if (!bp->b_error && bp->b_io_error) +xfs_buf_ioerror (bp, bp->b_io_error); + +/* Only validate buffers that were read without errors */ +if (read && !bp->b_error && bp->b_ops) { +ASSERT (!bp->b_iodone); + bp->b_ops->verify_read (bp); + } + +if (!bp->b_error) { + bp->b_flags |=XBF_DONE ; + bp->b_flags &= ~(LIBXFS_B_DIRTY |LIBXFS_B_UNCHECKED ); + }else { +fprintf (stderr, +_ ("%s: IO failed on %s bno 0x%llx/0x%x, err=%d\n" ), + __func__, bp->b_ops ? bp->b_ops->name :"(unknown)" , + (long long )bp->b_maps[0 ].bm_bn, bp->b_length, + -bp->b_error); + } + +if (bp->b_iodone) + (*(bp->b_iodone))(bp); +else if (bp->b_flags &XBF_ASYNC ) +xfs_buf_relse (bp); +else +complete (&bp->b_iowait); +} + +void +__xfs_buf_ioerror ( +struct xfs_buf *bp, +int error, +xfs_failaddr_t failaddr) +{ +ASSERT (error <=0 && error >=-1000 ); + bp->b_error = error; +trace_xfs_buf_ioerror (bp, error, failaddr); +} + +void +xfs_buf_ioerror_alert ( +struct xfs_buf *bp, +const char *func) +{ +xfs_alert (bp->b_target->bt_mount, +"metadata I/O error in \"%s\" at daddr 0x%llx len %d error %d" , + func, (uint64_t )XFS_BUF_ADDR (bp), bp->b_length, + -bp->b_error); +} + +#ifdef NOT_YET +int +xfs_bread ( +struct xfs_buf *bp, +size_t bblen) +{ +int error; + +ASSERT (xfs_buf_islocked (bp)); + + bp->b_flags |=XBF_READ ; + bp->b_flags &= ~(XBF_ASYNC |XBF_WRITE | _XBF_DELWRI_Q | +XBF_WRITE_FAIL |XBF_DONE ); + + error =xfs_buf_submit (bp); +if (error) { +xfs_force_shutdown (bp->b_target->bt_mount, +SHUTDOWN_META_IO_ERROR ); + } +return error; +} + +int +xfs_bwrite ( +struct xfs_buf *bp) +{ +int error; + +ASSERT (xfs_buf_islocked (bp)); + + bp->b_flags |=XBF_WRITE ; + bp->b_flags &= ~(XBF_ASYNC |XBF_READ | _XBF_DELWRI_Q | +XBF_WRITE_FAIL |XBF_DONE ); + + error =xfs_buf_submit (bp); +if (error) { +xfs_force_shutdown (bp->b_target->bt_mount, +SHUTDOWN_META_IO_ERROR ); + } +return error; +} +#endif /* not yet */ + +/* +* Wait for I/O completion of a sync buffer and return the I/O error code. +*/ +static int +xfs_buf_iowait ( +struct xfs_buf *bp) +{ +ASSERT (!(bp->b_flags &XBF_ASYNC )); + +trace_xfs_buf_iowait (bp, _RET_IP_); +wait_for_completion (&bp->b_iowait); +trace_xfs_buf_iowait_done (bp, _RET_IP_); + +return bp->b_error; +} + +/* +* Buffer I/O submission path, read or write. Asynchronous submission transfers +* the buffer lock ownership and the current reference to the IO. It is not +* safe to reference the buffer after a call to this function unless the caller +* holds an additional reference itself. +*/ +int +__xfs_buf_submit ( +struct xfs_buf *bp, +bool wait) +{ +int error =0 ; + +trace_xfs_buf_submit (bp, _RET_IP_); + +ASSERT (!(bp->b_flags & _XBF_DELWRI_Q)); + +/* on shutdown we stale and complete the buffer immediately */ +if (XFS_FORCED_SHUTDOWN (bp->b_target->bt_mount)) { +xfs_buf_ioerror (bp, -EIO ); + bp->b_flags &= ~XBF_DONE ; +xfs_buf_stale (bp); +xfs_buf_ioend (bp); +return -EIO ; + } + +/* +* Grab a reference so the buffer does not go away underneath us. For +* async buffers, I/O completion drops the callers reference, which +* could occur before submission returns. +*/ +xfs_buf_hold (bp); + +if (bp->b_flags &XBF_WRITE ) +xfs_buf_wait_unpin (bp); + +/* clear the internal error state to avoid spurious errors */ + bp->b_io_error =0 ; + +/* +* Set the count to 1 initially, this will stop an I/O completion +* callout which happens before we have started all the I/O from calling +* xfs_buf_ioend too early. +*/ +atomic_set (&bp->b_io_remaining,1 ); +if (bp->b_flags &XBF_ASYNC ) +xfs_buf_ioacct_inc (bp); + +xfs_buftarg_submit_io (bp); + +/* +* If _xfs_buf_ioapply failed, we can get back here with only the IO +* reference we took above. If we drop it to zero, run completion so +* that we don't return to the caller with completion still pending. +*/ +if (atomic_dec_and_test (&bp->b_io_remaining) ==1 ) { +if (bp->b_error || !(bp->b_flags &XBF_ASYNC )) +xfs_buf_ioend (bp); +else +xfs_buf_ioend_async (bp); + } + +if (wait) + error =xfs_buf_iowait (bp); + +/* +* Release the hold that keeps the buffer referenced for the entire +* I/O. Note that if the buffer is async, it is not safe to reference +* after this release. +*/ +xfs_buf_rele (bp); +return error; +} + +/* +* Cancel a delayed write list. +* +* Remove each buffer from the list, clear the delwri queue flag and drop the +* associated buffer reference. +*/ +#ifdef NOT_YET +void +xfs_buf_delwri_cancel ( +struct list_head *list) +{ +struct xfs_buf *bp; + +while (!list_empty (list)) { + bp = list_first_entry(list, struct xfs_buf, b_list); + +xfs_buf_lock (bp); + bp->b_flags &= ~_XBF_DELWRI_Q; +list_del_init (&bp->b_list); +xfs_buf_relse (bp); + } +} + +/* +* Add a buffer to the delayed write list. +* +* This queues a buffer for writeout if it hasn't already been. Note that +* neither this routine nor the buffer list submission functions perform +* any internal synchronization. It is expected that the lists are thread-local +* to the callers. +* +* Returns true if we queued up the buffer, or false if it already had +* been on the buffer list. +*/ +bool +xfs_buf_delwri_queue ( +struct xfs_buf *bp, +struct list_head *list) +{ +ASSERT (xfs_buf_islocked (bp)); +ASSERT (!(bp->b_flags &XBF_READ )); + +/* +* If the buffer is already marked delwri it already is queued up +* by someone else for imediate writeout. Just ignore it in that +* case. +*/ +if (bp->b_flags & _XBF_DELWRI_Q) { +trace_xfs_buf_delwri_queued (bp, _RET_IP_); +return false; + } + +trace_xfs_buf_delwri_queue (bp, _RET_IP_); + +/* +* If a buffer gets written out synchronously or marked stale while it +* is on a delwri list we lazily remove it. To do this, the other party +* clears the _XBF_DELWRI_Q flag but otherwise leaves the buffer alone. +* It remains referenced and on the list. In a rare corner case it +* might get readded to a delwri list after the synchronous writeout, in +* which case we need just need to re-add the flag here. +*/ + bp->b_flags |= _XBF_DELWRI_Q; +if (list_empty (&bp->b_list)) { +atomic_inc (&bp->b_hold); +list_add_tail (&bp->b_list, list); + } + +return true; +} + +/* +* Compare function is more complex than it needs to be because +* the return value is only 32 bits and we are doing comparisons +* on 64 bit values +*/ +static int +xfs_buf_cmp ( +void *priv, +struct list_head *a, +struct list_head *b) +{ + struct xfs_buf *ap = container_of(a, struct xfs_buf, b_list); + struct xfs_buf *bp = container_of(b, struct xfs_buf, b_list); +xfs_daddr_t diff; + + diff = ap->b_maps[0 ].bm_bn - bp->b_maps[0 ].bm_bn; +if (diff <0 ) +return -1 ; +if (diff >0 ) +return 1 ; +return 0 ; +} + +/* +* Submit buffers for write. If wait_list is specified, the buffers are +* submitted using sync I/O and placed on the wait list such that the caller can +* iowait each buffer. Otherwise async I/O is used and the buffers are released +* at I/O completion time. In either case, buffers remain locked until I/O +* completes and the buffer is released from the queue. +*/ +static int +xfs_buf_delwri_submit_buffers ( +struct list_head *buffer_list, +struct list_head *wait_list) +{ +struct xfs_buf *bp, *n; +int pinned =0 ; + +list_sort (NULL , buffer_list, xfs_buf_cmp); + +list_for_each_entry_safe (bp, n, buffer_list, b_list) { +if (!wait_list) { +if (xfs_buf_ispinned (bp)) { + pinned++; +continue ; + } +if (!xfs_buf_trylock (bp)) +continue ; + }else { +xfs_buf_lock (bp); + } + +/* +* Someone else might have written the buffer synchronously or +* marked it stale in the meantime. In that case only the +* _XBF_DELWRI_Q flag got cleared, and we have to drop the +* reference and remove it from the list here. +*/ +if (!(bp->b_flags & _XBF_DELWRI_Q)) { +list_del_init (&bp->b_list); +xfs_buf_relse (bp); +continue ; + } + +trace_xfs_buf_delwri_split (bp, _RET_IP_); + +/* +* If we have a wait list, each buffer (and associated delwri +* queue reference) transfers to it and is submitted +* synchronously. Otherwise, drop the buffer from the delwri +* queue and submit async. +*/ + bp->b_flags &= ~(_XBF_DELWRI_Q |XBF_WRITE_FAIL ); + bp->b_flags |=XBF_WRITE ; +if (wait_list) { + bp->b_flags &= ~XBF_ASYNC ; +list_move_tail (&bp->b_list, wait_list); + }else { + bp->b_flags |=XBF_ASYNC ; +list_del_init (&bp->b_list); + } +__xfs_buf_submit (bp, false); + } + +return pinned; +} + +/* +* Write out a buffer list asynchronously. +* +* This will take the @buffer_list, write all non-locked and non-pinned buffers +* out and not wait for I/O completion on any of the buffers. This interface +* is only safely useable for callers that can track I/O completion by higher +* level means, e.g. AIL pushing as the @buffer_list is consumed in this +* function. +* +* Note: this function will skip buffers it would block on, and in doing so +* leaves them on @buffer_list so they can be retried on a later pass. As such, +* it is up to the caller to ensure that the buffer list is fully submitted or +* cancelled appropriately when they are finished with the list. Failure to +* cancel or resubmit the list until it is empty will result in leaked buffers +* at unmount time. +*/ +int +xfs_buf_delwri_submit_nowait ( +struct list_head *buffer_list) +{ +return xfs_buf_delwri_submit_buffers (buffer_list,NULL ); +} + +/* +* Write out a buffer list synchronously. +* +* This will take the @buffer_list, write all buffers out and wait for I/O +* completion on all of the buffers. @buffer_list is consumed by the function, +* so callers must have some other way of tracking buffers if they require such +* functionality. +*/ +int +xfs_buf_delwri_submit ( +struct list_head *buffer_list) +{ +LIST_HEAD (wait_list); +int error =0 , error2; +struct xfs_buf *bp; + +xfs_buf_delwri_submit_buffers (buffer_list, &wait_list); + +/* Wait for IO to complete. */ +while (!list_empty (&wait_list)) { + bp = list_first_entry(&wait_list, struct xfs_buf, b_list); + +list_del_init (&bp->b_list); + +/* +* Wait on the locked buffer, check for errors and unlock and +* release the delwri queue reference. +*/ + error2 =xfs_buf_iowait (bp); +xfs_buf_relse (bp); +if (!error) + error = error2; + } + +return error; +} + +/* +* Push a single buffer on a delwri queue. +* +* The purpose of this function is to submit a single buffer of a delwri queue +* and return with the buffer still on the original queue. The waiting delwri +* buffer submission infrastructure guarantees transfer of the delwri queue +* buffer reference to a temporary wait list. We reuse this infrastructure to +* transfer the buffer back to the original queue. +* +* Note the buffer transitions from the queued state, to the submitted and wait +* listed state and back to the queued state during this call. The buffer +* locking and queue management logic between _delwri_pushbuf() and +* _delwri_queue() guarantee that the buffer cannot be queued to another list +* before returning. +*/ +int +xfs_buf_delwri_pushbuf ( +struct xfs_buf *bp, +struct list_head *buffer_list) +{ +LIST_HEAD (submit_list); +int error; + +ASSERT (bp->b_flags & _XBF_DELWRI_Q); + +trace_xfs_buf_delwri_pushbuf (bp, _RET_IP_); + +/* +* Isolate the buffer to a new local list so we can submit it for I/O +* independently from the rest of the original list. +*/ +xfs_buf_lock (bp); +list_move (&bp->b_list, &submit_list); +xfs_buf_unlock (bp); + +/* +* Delwri submission clears the DELWRI_Q buffer flag and returns with +* the buffer on the wait list with the original reference. Rather than +* bounce the buffer from a local wait list back to the original list +* after I/O completion, reuse the original list as the wait list. +*/ +xfs_buf_delwri_submit_buffers (&submit_list, buffer_list); + +/* +* The buffer is now locked, under I/O and wait listed on the original +* delwri queue. Wait for I/O completion, restore the DELWRI_Q flag and +* return with the buffer unlocked and on the original queue. +*/ + error =xfs_buf_iowait (bp); + bp->b_flags |= _XBF_DELWRI_Q; +xfs_buf_unlock (bp); + +return error; +} +#endif /* not yet */ + +void xfs_buf_set_ref (struct xfs_buf *bp,int lru_ref) +{ +/* +* Set the lru reference count to 0 based on the error injection tag. +* This allows userspace to disrupt buffer caching for debug/testing +* purposes. +*/ +if (XFS_TEST_ERROR (false, bp->b_target->bt_mount, +XFS_ERRTAG_BUF_LRU_REF )) + lru_ref =0 ; + +atomic_set (&bp->b_lru_ref, lru_ref); +} + +#ifdef NOT_YET +/* +* Verify an on-disk magic value against the magic value specified in the +* verifier structure. The verifier magic is in disk byte order so the caller is +* expected to pass the value directly from disk. +*/ +bool +xfs_verify_magic ( +struct xfs_buf *bp, +__be32 dmagic) +{ +struct xfs_mount *mp = bp->b_target->bt_mount; +int idx; + + idx =xfs_sb_version_hascrc (&mp->m_sb); +if (unlikely (WARN_ON (!bp->b_ops || !bp->b_ops->magic[idx]))) +return false; +return dmagic == bp->b_ops->magic[idx]; +} + +/* +* Verify an on-disk magic value against the magic value specified in the +* verifier structure. The verifier magic is in disk byte order so the caller is +* expected to pass the value directly from disk. +*/ +bool +xfs_verify_magic16 ( +struct xfs_buf *bp, +__be16 dmagic) +{ +struct xfs_mount *mp = bp->b_target->bt_mount; +int idx; + + idx =xfs_sb_version_hascrc (&mp->m_sb); +if (unlikely (WARN_ON (!bp->b_ops || !bp->b_ops->magic16[idx]))) +return false; +return dmagic == bp->b_ops->magic16[idx]; +} + +/* +* Read an uncached buffer from disk. Allocates and returns a locked +* buffer containing the disk contents or nothing. +*/ +int +xfs_buf_read_uncached ( +struct xfs_buftarg *target, +xfs_daddr_t daddr, +size_t numblks, +int flags, +struct xfs_buf **bpp, +const struct xfs_buf_ops *ops) +{ +struct xfs_buf *bp; + + *bpp =NULL ; + + bp =xfs_buf_get_uncached (target, numblks, flags); +if (!bp) +return -ENOMEM ; + +/* set up the buffer for a read IO */ +ASSERT (bp->b_map_count ==1 ); + bp->b_bn =XFS_BUF_DADDR_NULL ;/* always null for uncached buffers */ + bp->b_maps[0 ].bm_bn = daddr; + bp->b_flags |=XBF_READ ; + bp->b_ops = ops; + +xfs_buf_submit (bp); +if (bp->b_error) { +int error = bp->b_error; +xfs_buf_relse (bp); +return error; + } + + *bpp = bp; +return 0 ; +} + +struct xfs_buf * +xfs_buf_get_uncached ( +struct xfs_buftarg *target, +size_t numblks, +int flags) +{ +int error; +struct xfs_buf *bp; +DEFINE_SINGLE_BUF_MAP (map,XFS_BUF_DADDR_NULL , numblks); + +/* flags might contain irrelevant bits, pass only what we care about */ + bp =_xfs_buf_alloc (target, &map,1 , flags &XBF_NO_IOACCT ); +if (unlikely (bp ==NULL )) + goto fail; + + error =xfs_buf_allocate_memory (bp, flags); +if (error) + goto fail_free_buf; + +trace_xfs_buf_get_uncached (bp, _RET_IP_); +return bp; + + fail_free_buf: +kmem_cache_free (xfs_buf_zone, bp); + fail: +return NULL ; +} +#endif libxfs/xfs_buf.h new file +203
-0@@ -0,0 +1,203 @@ +#ifndef __LIBXFS_XFS_BUF_H_ +#define __LIBXFS_XFS_BUF_H_ + +struct xfs_buf ; +struct xfs_mount ; +struct xfs_perag ; +struct xfs_buftarg ; + +/* +* Base types +*/ +#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL)) + +typedef unsigned int xfs_buf_flags_t ; + +bool xfs_verify_magic (struct xfs_buf *bp,__be32 dmagic); +bool xfs_verify_magic16 (struct xfs_buf *bp,__be16 dmagic); + +/* Finding and Reading Buffers */ +struct xfs_buf_map { +xfs_daddr_t bm_bn;/* block number for I/O */ +int bm_len;/* size of I/O */ +}; + +#define DEFINE_SINGLE_BUF_MAP (map, blkno, numblk) \ + struct xfs_buf_map (map) = { .bm_bn = (blkno), .bm_len = (numblk) }; + +struct xfs_buf_ops { +char *name; +union { +__be32 magic[2 ];/* v4 and v5 on disk magic values */ +__be16 magic16[2 ];/* v4 and v5 on disk magic values */ + }; +void (*verify_read)(struct xfs_buf *); +void (*verify_write)(struct xfs_buf *); +xfs_failaddr_t (*verify_struct)(struct xfs_buf *); +}; + +/* +* Internal state flags. +*/ +#define XFS_BSTATE_DISPOSE (1 << 0)/* buffer being discarded */ +#define XFS_BSTATE_IN_FLIGHT (1 << 1)/* I/O in flight */ + +typedef void (*xfs_buf_iodone_t )(struct xfs_buf *bp); + +/* +* This is a mess of userspace and kernel variables for the moment. It will +* clean up soon and should be identical between kernel and userspace.. +*/ +struct xfs_buf { +struct cache_node b_node; +struct list_head b_hash;/* will replace b_node */ +xfs_daddr_t b_bn; +unsigned int b_length; +unsigned int b_flags; +struct xfs_buftarg *b_target; +pthread_mutex_t b_lock; +pthread_t b_holder; +unsigned int b_recur; +void *b_log_item; +void *b_transp; +void *b_addr; +int b_error; +const struct xfs_buf_ops *b_ops; +struct xfs_perag *b_pag; +struct xfs_mount *b_mount; +struct xfs_buf_map *b_maps; +struct xfs_buf_map __b_map; +int b_map_count; +int b_io_remaining; +int b_io_error; +struct list_head b_list; +struct list_head b_li_list;/* Log items list head */ + +struct list_head b_btc_list; +unsigned int b_state; +atomic_t b_lru_ref; +struct list_head b_lru; +atomic_t b_hold; +struct completion b_iowait; +struct semaphore b_sema; +xfs_buf_iodone_t b_iodone; +}; + +struct xfs_buf *xfs_buf_incore (struct xfs_buftarg *target, +xfs_daddr_t blkno,size_t numblks, +xfs_buf_flags_t flags); + +int xfs_buf_get_map (struct xfs_buftarg *btp,struct xfs_buf_map *maps, +int nmaps,xfs_buf_flags_t flags,struct xfs_buf **bpp); +int xfs_buf_read_map (struct xfs_buftarg *btp,struct xfs_buf_map *maps, +int nmaps,xfs_buf_flags_t flags,struct xfs_buf **bpp, +const struct xfs_buf_ops *ops); +void xfs_buf_readahead_map (struct xfs_buftarg *target, +struct xfs_buf_map *map,int nmaps, +const struct xfs_buf_ops *ops); + +static inline int +xfs_buf_get ( +struct xfs_buftarg *target, +xfs_daddr_t blkno, +size_t numblks, +struct xfs_buf **bpp) +{ +DEFINE_SINGLE_BUF_MAP (map, blkno, numblks); + +return xfs_buf_get_map (target, &map,1 ,0 , bpp); +} + +static inline int +xfs_buf_read ( +struct xfs_buftarg *target, +xfs_daddr_t blkno, +size_t numblks, +xfs_buf_flags_t flags, +struct xfs_buf **bpp, +const struct xfs_buf_ops *ops) +{ +DEFINE_SINGLE_BUF_MAP (map, blkno, numblks); + +return xfs_buf_read_map (target, &map,1 , flags, bpp, ops); +} + +static inline void +xfs_buf_readahead ( +struct xfs_buftarg *target, +xfs_daddr_t blkno, +size_t numblks, +const struct xfs_buf_ops *ops) +{ +DEFINE_SINGLE_BUF_MAP (map, blkno, numblks); +return xfs_buf_readahead_map (target, &map,1 , ops); +} + +int xfs_bwrite (struct xfs_buf *bp); +int xfs_bread (struct xfs_buf *bp,size_t bblen); + +#define xfs_buf_offset (bp, offset) ((bp)->b_addr + (offset)) + +/* Locking and Unlocking Buffers */ +int xfs_buf_trylock (struct xfs_buf *bp); +void xfs_buf_lock (struct xfs_buf *bp); +void xfs_buf_unlock (struct xfs_buf *bp); + +/* Releasing Buffers */ +void xfs_buf_hold (struct xfs_buf *bp); +void xfs_buf_rele (struct xfs_buf *bp); +/* +static inline void xfs_buf_relse(struct xfs_buf *bp) +{ +xfs_buf_unlock(bp); +xfs_buf_rele(bp); +} +*/ +void xfs_buf_free (struct xfs_buf *bp); + + +/* Buffer Utility Routines */ +extern int __xfs_buf_submit (struct xfs_buf *bp,bool ); +static inline int xfs_buf_submit (struct xfs_buf *bp) +{ +bool wait = bp->b_flags &XBF_ASYNC ? false : true; +return __xfs_buf_submit (bp, wait); +} + +void xfs_buf_stale (struct xfs_buf *bp); +void xfs_buf_ioend (struct xfs_buf *bp); +void __xfs_buf_ioerror (struct xfs_buf *bp,int error, +xfs_failaddr_t failaddr); +void xfs_buf_ioerror_alert (struct xfs_buf *,const char *func); + +#define xfs_buf_ioerror (bp, err) __xfs_buf_ioerror((bp), (err), __this_address) + + +/* +* These macros use the IO block map rather than b_bn. b_bn is now really +* just for the buffer cache index for cached buffers. As IO does not use b_bn +* anymore, uncached buffers do not use b_bn at all and hence must modify the IO +* map directly. Uncached buffers are not allowed to be discontiguous, so this +* is safe to do. +* +* In future, uncached buffers will pass the block number directly to the io +* request function and hence these macros will go away at that point. +*/ +#define XFS_BUF_ADDR (bp) ((bp)->b_maps[0].bm_bn) + +void xfs_buf_set_ref (struct xfs_buf *bp,int lru_ref); + +/* +* If the buffer is already on the LRU, do nothing. Otherwise set the buffer +* up with a reference count of 0 so it will be tossed from the cache when +* released. +static inline void xfs_buf_oneshot(struct xfs_buf *bp) +{ +if (!list_empty(&bp->b_lru) || atomic_read(&bp->b_lru_ref) > 1) +return; +atomic_set(&bp->b_lru_ref, 0); +} +*/ + +#endif /* __LIBXFS_IO_H__ */ +libxfs/xfs_buftarg.h +29
-2@@ -85,9 +85,13 @@ int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr, size_t bblen, int flags, struct xfs_buf **bpp, const struct xfs_buf_ops *ops); -int xfs_bread (struct xfs_buf *bp,size_t bblen); +void xfs_buftarg_submit_io (struct xfs_buf *bp); -int xfs_bwrite (struct xfs_buf *bp); +/* +* Cached buffer memory manangement +*/ +int xfs_buf_allocate_memory (struct xfs_buf *bp,uint flags); +void xfs_buf_free_memory (struct xfs_buf *bp);/* * Temporary: these need to be the same as the LIBXFS_B_* flags until we change @@ -99,6 +103,23 @@ int xfs_bwrite(struct xfs_buf *bp);#define XBF_DONE (1 << 3) // LIBXFS_B_UPTODATE 0x0008#define XBF_STALE (1 << 2) // LIBXFS_B_STALE 0x0004 +#define XBF_READ_AHEAD (1 << 30)/* asynchronous read-ahead */ +#define XBF_NO_IOACCT (1 << 29)/* bypass I/O accounting (non-LRU bufs) */ +#define XBF_ASYNC (1 << 28)/* initiator will not wait for completion */ +#define XBF_WRITE_FAIL (0)/* unused in userspace */ + +/* buffer type flags for write callbacks */ +#define _XBF_INODES (0)/* inode buffer */ +#define _XBF_DQUOTS (0)/* dquot buffer */ +#define _XBF_LOGRECOVERY (0)/* log recovery buffer */ + +/* flags used only as arguments to access routines */ +#define XBF_TRYLOCK (1 << 16)/* lock requested, but do not wait */ +#define XBF_UNMAPPED (0)/* unused in userspace */ + +/* flags used only internally */ +#define _XBF_DELWRI_Q (1 << 22)/* buffer on a delwri queue */ +/* * Raw buffer access functions. These exist as temporary bridges for uncached IO * that uses direct access to the buffers to submit IO. These will go away with @@ -107,6 +128,12 @@ int xfs_bwrite(struct xfs_buf *bp);struct xfs_buf *libxfs_getbufr (struct xfs_buftarg *btp,xfs_daddr_t blkno,int bblen); +/* temporary, just for compile for the moment */ +#define xfs_buf_ioend_async (bp) xfs_buf_ioend(bp) +#define bdi_read_congested (bdi) (false) +#define xfs_buf_ispinned (bp) (false) +#define xfs_buf_wait_unpin (bp) ((void)0) +/* * Hash cache implementation */ Signature · 1 line
2.28.0
PATCH 23/27 libxfs: use PSI information to detect memory pressure
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The buffer cache needs to have a reliable trigger for shrinking the cache. Modern kernels track and report memory pressure events to the userspace via the Pressure Stall Interface (PSI). Create a PSI memory pressure monitoring thread to listen for memory pressure events and use that to drive buffer cache shrinking interfaces.
Add the shrinker framework that will allow us to implement LRU reclaim of buffers when memory pressure occues. We also create a low memory detection and reclaim wait mechanism to allow use to throttle back new allocations while we are shrinking the buffer cache.
We also include malloc heap trimming callouts so that once the shrinker frees the memory, we trim the malloc heap to release the freed memory back to the system.
--- libxfs/buftarg.c | 142 ++++++++++++++++++++++++++++++++++++++++++- libxfs/xfs_buftarg.h | 9 +++ 2 files changed, 150 insertions(+), 1 deletion(-)
libxfs/buftarg.c +141 -1
@@ -62,6 +62,128 @@ xfs_buftarg_setsize_early(return xfs_buftarg_setsize (btp, bsize); } +/* +* Scan a chunk of the buffer cache and drop LRU reference counts. If the +* count goes to zero, dispose of the buffer. +*/ +static void +xfs_buftarg_shrink ( +struct xfs_buftarg *btc) +{ +/* +* Make the fact we are in memory reclaim externally visible. This +* allows buffer cache allocation throttling while we are trying to +* free memory. +*/ +atomic_inc_return (&btc->bt_low_mem); + +fprintf (stderr,"Got memory pressure event. Shrinking caches!\n" ); + +/* +* Now we've free a bunch of memory, trim the heap down to release the +* freed memory back to the kernel and reduce the pressure we are +* placing on the system. +*/ +malloc_trim (0 ); + +/* +* Done, wake anyone waiting on memory reclaim to complete. +*/ +atomic_dec_return (&btc->bt_low_mem); +complete (&btc->bt_low_mem_wait); +} + +static void * +xfs_buftarg_shrinker ( +void *args) +{ +struct xfs_buftarg *btp = args; +struct pollfd fds = { + .fd = btp->bt_psi_fd, + .events =POLLPRI , + }; + +rcu_register_thread (); +while (!btp->bt_exiting) { +int n; + + n =poll (&fds,1 ,100 ); +if (n ==0 ) +continue ;/* timeout */ +if (n <0 ) { +perror ("poll(PSI)" ); +break ; + } +if (fds.revents &POLLERR ) { +fprintf (stderr, +"poll(psi) POLLERR: event source dead?\n" ); +break ; + } +if (!(fds.revents &POLLPRI )) { +fprintf (stderr, +"poll(psi): unknown event. Ignoring.\n" ); +continue ; + } + +/* run the shrinker here */ +xfs_buftarg_shrink (btp); + + } +rcu_unregister_thread (); +return NULL ; +} + +/* +* This only picks up on global memory pressure. Maybe in future we can detect +* whether we are running inside a container and use the PSI information for the +* container. +* +* We want relatively early notification of memory pressure stalls because +* xfs_repair will consume lots of memory. Hence set a low trigger threshold for +* reclaim to run - a partial stall of 5ms over a 1s sample period will trigger +* reclaim algorithms. +*/ +static int +xfs_buftarg_mempressue_init ( +struct xfs_buftarg *btp) +{ +const char *fname ="/proc/pressure/memory" ; +const char *trigger ="some 10000 1000000" ; +int error; + + btp->bt_psi_fd =open (fname,O_RDWR |O_NONBLOCK ); +if (btp->bt_psi_fd <0 ) { +perror ("open(PSI)" ); +return -errno; + } +if (write (btp->bt_psi_fd, trigger,strlen (trigger) +1 ) != +strlen (trigger) +1 ) { +perror ("write(PSI)" ); + error = -errno; + goto out_close; + } + +atomic_set (&btp->bt_low_mem,0 ); +init_completion (&btp->bt_low_mem_wait); + +/* +* Now create the monitoring reclaim thread. This will run until the +* buftarg is torn down. +*/ + error =pthread_create (&btp->bt_psi_tid,NULL , + xfs_buftarg_shrinker, btp); +if (error) + goto out_close; + +return 0 ; + +out_close: +close (btp->bt_psi_fd); + btp->bt_psi_fd =-1 ; +return error; +} + +struct xfs_buftarg * xfs_buftarg_alloc( struct xfs_mount *mp,
@@ -74,6 +196,8 @@ xfs_buftarg_alloc( btp->bt_mount = mp; btp->bt_fd =libxfs_device_to_fd (bdev); btp->bt_bdev = bdev; + btp->bt_psi_fd =-1 ; + btp->bt_exiting = false;if (xfs_buftarg_setsize_early (btp)) goto error_free;
@@ -84,8 +208,13 @@ xfs_buftarg_alloc(if (percpu_counter_init (&btp->bt_io_count,0 ,GFP_KERNEL )) goto error_lru; +if (xfs_buftarg_mempressue_init (btp)) + goto error_pcp; +return btp; +error_pcp: +percpu_counter_destroy (&btp->bt_io_count); error_lru:list_lru_destroy (&btp->bt_lru); error_free:
@@ -97,6 +226,12 @@ voidxfs_buftarg_free ( struct xfs_buftarg *btp) { + btp->bt_exiting = true; +if (btp->bt_psi_tid) +pthread_join (btp->bt_psi_tid,NULL ); +if (btp->bt_psi_fd >=0 ) +close (btp->bt_psi_fd); +ASSERT (percpu_counter_sum (&btp->bt_io_count) ==0 );percpu_counter_destroy (&btp->bt_io_count);platform_flush_device (btp->bt_fd, btp->bt_bdev);
@@ -121,10 +256,15 @@ xfs_buf_allocate_memory( struct xfs_buf *bp, uint flags) { +struct xfs_buftarg *btp = bp->b_target;size_t size; +/* Throttle allocation while dealing with low memory events */ +while (atomic_read (&btp->bt_low_mem)) +wait_for_completion (&btp->bt_low_mem_wait); + size =BBTOB (bp->b_length); - bp->b_addr =memalign (bp->b_target->bt_meta_sectorsize, size); + bp->b_addr =memalign (btp->bt_meta_sectorsize, size);if (!bp->b_addr)return -ENOMEM ;return 0 ;
libxfs/xfs_buftarg.h +9 -0
@@ -41,7 +41,16 @@ struct xfs_buftarg {uint32_t bt_io_count;unsigned int flags; + +/* +* Memory pressure (PSI) and cache reclaim infrastructure +*/ struct list_lru bt_lru; +int bt_psi_fd; +pthread_t bt_psi_tid; +bool bt_exiting; +bool bt_low_mem; +struct completion bt_low_mem_wait; };/* We purged a dirty buffer and lost a write. */
Signature · 1 line
2.28.0
Re: [PATCH 23/27] libxfs: use PSI information to detect memory pressure
Quoted 101 lines · Dave Chinner
From: Dave Chinner <[email protected]>
The buffer cache needs to have a reliable trigger for shrinking the cache. Modern kernels track and report memory pressure events to the userspace via the Pressure Stall Interface (PSI). Create a PSI memory pressure monitoring thread to listen for memory pressure events and use that to drive buffer cache shrinking interfaces.
Add the shrinker framework that will allow us to implement LRU reclaim of buffers when memory pressure occues. We also create a low memory detection and reclaim wait mechanism to allow use to throttle back new allocations while we are shrinking the buffer cache.
We also include malloc heap trimming callouts so that once the shrinker frees the memory, we trim the malloc heap to release the freed memory back to the system.
Signed-off-by Dave Chinner <[email protected]>--- libxfs/buftarg.c | 142 ++++++++++++++++++++++++++++++++++++++++++- libxfs/xfs_buftarg.h | 9 +++ 2 files changed, 150 insertions(+), 1 deletion(-)libxfs/buftarg.c +69
-0@@ -62,6 +62,128 @@ xfs_buftarg_setsize_early(return xfs_buftarg_setsize (btp, bsize); } +/* +* Scan a chunk of the buffer cache and drop LRU reference counts. If the +* count goes to zero, dispose of the buffer. +*/ +static void +xfs_buftarg_shrink ( +struct xfs_buftarg *btc) +{ +/* +* Make the fact we are in memory reclaim externally visible. This +* allows buffer cache allocation throttling while we are trying to +* free memory. +*/ +atomic_inc_return (&btc->bt_low_mem); + +fprintf (stderr,"Got memory pressure event. Shrinking caches!\n" ); + +/* +* Now we've free a bunch of memory, trim the heap down to release the +* freed memory back to the kernel and reduce the pressure we are +* placing on the system. +*/ +malloc_trim (0 ); + +/* +* Done, wake anyone waiting on memory reclaim to complete. +*/ +atomic_dec_return (&btc->bt_low_mem); +complete (&btc->bt_low_mem_wait); +} + +static void * +xfs_buftarg_shrinker ( +void *args) +{ +struct xfs_buftarg *btp = args; +struct pollfd fds = { + .fd = btp->bt_psi_fd, + .events =POLLPRI , + }; + +rcu_register_thread (); +while (!btp->bt_exiting) { +int n; + + n =poll (&fds,1 ,100 ); +if (n ==0 ) +continue ;/* timeout */ +if (n <0 ) { +perror ("poll(PSI)" ); +break ; + } +if (fds.revents &POLLERR ) { +fprintf (stderr, +"poll(psi) POLLERR: event source dead?\n" ); +break ; + } +if (!(fds.revents &POLLPRI )) { +fprintf (stderr, +"poll(psi): unknown event. Ignoring.\n" ); +continue ; + } + +/* run the shrinker here */ +xfs_buftarg_shrink (btp); + + } +rcu_unregister_thread (); +return NULL ;
libxfs/buftarg.c xfs_buftarg_setsize_early( · line 143
Quoted 10 lines
libxfs/buftarg.c +10
-0+} + +/* +* This only picks up on global memory pressure. Maybe in future we can detect +* whether we are running inside a container and use the PSI information for the +* container. +* +* We want relatively early notification of memory pressure stalls because +* xfs_repair will consume lots of memory. Hence set a low trigger threshold for +* reclaim to run - a partial stall of 5ms over a 1s sample period will trigger
The trigger string looks like it's configuring for a partial stall of 10ms over a 1s sample period?
libxfs/buftarg.c xfs_buftarg_setsize_early( · line 147
Quoted 4 lines
libxfs/buftarg.c +4
-0+* reclaim algorithms. +*/ +static int +xfs_buftarg_mempressue_init(
xfs_buftarg_mempressure_init() ?
Quoted 52 lines
libxfs/buftarg.c +41
-0+ struct xfs_buftarg *btp) +{ +const char *fname ="/proc/pressure/memory" ; +const char *trigger ="some 10000 1000000" ; +int error; + + btp->bt_psi_fd =open (fname,O_RDWR |O_NONBLOCK ); +if (btp->bt_psi_fd <0 ) { +perror ("open(PSI)" ); +return -errno; + } +if (write (btp->bt_psi_fd, trigger,strlen (trigger) +1 ) != +strlen (trigger) +1 ) { +perror ("write(PSI)" ); + error = -errno; + goto out_close; + } + +atomic_set (&btp->bt_low_mem,0 ); +init_completion (&btp->bt_low_mem_wait); + +/* +* Now create the monitoring reclaim thread. This will run until the +* buftarg is torn down. +*/ + error =pthread_create (&btp->bt_psi_tid,NULL , + xfs_buftarg_shrinker, btp); +if (error) + goto out_close; + +return 0 ; + +out_close: +close (btp->bt_psi_fd); + btp->bt_psi_fd =-1 ; +return error; +} + +struct xfs_buftarg * xfs_buftarg_alloc( struct xfs_mount *mp,@@ -74,6 +196,8 @@ xfs_buftarg_alloc( btp->bt_mount = mp; btp->bt_fd =libxfs_device_to_fd (bdev); btp->bt_bdev = bdev; + btp->bt_psi_fd =-1 ; + btp->bt_exiting = false;if (xfs_buftarg_setsize_early (btp)) goto error_free;
libxfs/buftarg.c xfs_buftarg_alloc( · line 211
Quoted 4 lines
libxfs/buftarg.c +1
-0@@ -84,8 +208,13 @@ xfs_buftarg_alloc(if (percpu_counter_init (&btp->bt_io_count,0 ,GFP_KERNEL )) goto error_lru; +if (xfs_buftarg_mempressue_init (btp))
So what happens if PSI isn't enabled or procfs isn't mounted yet? xfs_repair just ... fails? That seems disappointing, particularly if the admin is trying to fix a dead root fs from the initramfs premount shell and /proc isn't set up yet.
Hmm, looks like Debian actually /does/ set up procfs nowadays. Still, if we're going to add a hard requirement on CONFIG_PSI=y and CONFIG_PSI_DEFAULT_DISABLED=n, we need to advertise this kind of loudly.
(Personally, I thought that if there's no pressure stall information, we'd just fall back to not having a shrinker and daring the system to OOM us like it does now...)
--D
Quoted 63 lines
libxfs/buftarg.c +16
-1+ goto error_pcp; +return btp; +error_pcp: +percpu_counter_destroy (&btp->bt_io_count); error_lru:list_lru_destroy (&btp->bt_lru); error_free:@@ -97,6 +226,12 @@ voidxfs_buftarg_free ( struct xfs_buftarg *btp) { + btp->bt_exiting = true; +if (btp->bt_psi_tid) +pthread_join (btp->bt_psi_tid,NULL ); +if (btp->bt_psi_fd >=0 ) +close (btp->bt_psi_fd); +ASSERT (percpu_counter_sum (&btp->bt_io_count) ==0 );percpu_counter_destroy (&btp->bt_io_count);platform_flush_device (btp->bt_fd, btp->bt_bdev);@@ -121,10 +256,15 @@ xfs_buf_allocate_memory( struct xfs_buf *bp, uint flags) { +struct xfs_buftarg *btp = bp->b_target;size_t size; +/* Throttle allocation while dealing with low memory events */ +while (atomic_read (&btp->bt_low_mem)) +wait_for_completion (&btp->bt_low_mem_wait); + size =BBTOB (bp->b_length); - bp->b_addr =memalign (bp->b_target->bt_meta_sectorsize, size); + bp->b_addr =memalign (btp->bt_meta_sectorsize, size);if (!bp->b_addr)return -ENOMEM ;return 0 ;libxfs/xfs_buftarg.h +9
-0@@ -41,7 +41,16 @@ struct xfs_buftarg {uint32_t bt_io_count;unsigned int flags; + +/* +* Memory pressure (PSI) and cache reclaim infrastructure +*/ struct list_lru bt_lru; +int bt_psi_fd; +pthread_t bt_psi_tid; +bool bt_exiting; +bool bt_low_mem; +struct completion bt_low_mem_wait; };/* We purged a dirty buffer and lost a write. */ Signature · 1 line
2.28.0
Re: [PATCH 23/27] libxfs: use PSI information to detect memory pressure
Quoted 14 lines · Darrick J. Wong
Quoted 13 lines · Dave Chinner
libxfs/buftarg.c +2
-0@@ -74,6 +196,8 @@ xfs_buftarg_alloc( btp->bt_mount = mp; btp->bt_fd =libxfs_device_to_fd (bdev); btp->bt_bdev = bdev; + btp->bt_psi_fd =-1 ; + btp->bt_exiting = false;if (xfs_buftarg_setsize_early (btp)) goto error_free;@@ -84,8 +208,13 @@ xfs_buftarg_alloc(if (percpu_counter_init (&btp->bt_io_count,0 ,GFP_KERNEL )) goto error_lru;
Quoted 6 lines
libxfs/buftarg.c xfs_buftarg_alloc( · line 211
Quoted 1 line
libxfs/buftarg.c +1
-0+if (xfs_buftarg_mempressue_init (btp))So what happens if PSI isn't enabled or procfs isn't mounted yet?
xfs_repairjust ... fails? That seems disappointing, particularly if the admin is trying to fix a dead root fs from the initramfs premount shell and /proc isn't set up yet.
Yes, right now it just fails. I'm more interested right now in getting the new infrastructure working such that the kernel buffer cache "just works" when there's more metadata than RAM to cache it in.
Quoted 7 lines
Hmm, looks like Debian actually /does/ set up procfs nowadays. Still, if we're going to add a hard requirement on
CONFIG_PSI=yandCONFIG_PSI_DEFAULT_DISABLED=n, we need to advertise this kind of loudly.(Personally, I thought that if there's no pressure stall information, we'd just fall back to not having a shrinker and daring the system to OOM us like it does now...)
Well, the existing buffer cache does have a shrinker mechanism - it will shake the cache down when it is full to free up old buffers. That's what all the MRU lists and buffer priority stuff in the repair prefetch code is all about.
repair tries to bound the maximum size of the buffer cache and prevent OOM that way. If it calculates that the memory requirement is larger than RAM, that's when it gets into OOM trouble because we still allow it to use lots of memory and then just hope...
I kind of want to get away from all those messy static heuristics. I'd much prefer that we do dynamic cache growth detection and size calculations in repair and determine if we should purge the cache at the end of each AG or retain it in RAM. i.e. if ((per ag cache size
* no. of AGs) > 75% RAM) then purge the AG cache when the phase scan is done. This way we run with minimal caching (just what is needed for prefetching to be efficient) when it is likely we can't fit all the metadata in RAM, and otherwise we behave like we currently do.
That sort of setup will go a long way to avoiding OOM kill and the need for actual memory shrinkers to activate. This mode could be activated if the PSI infomration is not there, hence might also solve most of the rescue situation problems.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 24/27 libxfs: add a buftarg cache shrinker implementation
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Add a list_lru scanner that runs from the memory pressure detection to free an amount of the buffer cache that will keep the cache from growing when there is memory pressure.
--- libxfs/buftarg.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)
libxfs/buftarg.c +51 -0
@@ -62,6 +62,19 @@ xfs_buftarg_setsize_early(return xfs_buftarg_setsize (btp, bsize); } +static void +dispose_list ( +struct list_head *dispose) +{ +struct xfs_buf *bp; + +while (!list_empty (dispose)) { + bp = list_first_entry(dispose, struct xfs_buf, b_lru); +list_del_init (&bp->b_lru); +xfs_buf_rele (bp); + } +} +/* * Scan a chunk of the buffer cache and drop LRU reference counts. If the * count goes to zero, dispose of the buffer.
@@ -70,6 +83,13 @@ static voidxfs_buftarg_shrink ( struct xfs_buftarg *btc) { +struct list_lru *lru = &btc->bt_lru; +struct xfs_buf *bp; +int count; +int progress =16384 ; +int rotate =0 ; +LIST_HEAD (dispose); +/* * Make the fact we are in memory reclaim externally visible. This * allows buffer cache allocation throttling while we are trying to
@@ -79,6 +99,37 @@ xfs_buftarg_shrink(fprintf (stderr,"Got memory pressure event. Shrinking caches!\n" ); +spin_lock (&lru->l_lock); + count = lru->l_count /50 ;/* 2% */ +fprintf (stderr,"cache size before %ld/%d\n" , lru->l_count, count); +while (count-- >0 && !list_empty (&lru->l_lru)) { + bp = list_first_entry(&lru->l_lru, struct xfs_buf, b_lru); +spin_lock (&bp->b_lock); +if (!atomic_add_unless (&bp->b_lru_ref,-1 ,1 )) { +atomic_set (&bp->b_lru_ref,0 ); + bp->b_state |=XFS_BSTATE_DISPOSE ; +list_move (&bp->b_lru, &dispose); + lru->l_count--; + }else { + rotate++; +list_move_tail (&bp->b_lru, &lru->l_lru); + } + +spin_unlock (&bp->b_lock); +if (--progress ==0 ) { +fprintf (stderr,"Disposing! rotated %d, lru %ld\n" , rotate, lru->l_count); +spin_unlock (&lru->l_lock); +dispose_list (&dispose); +spin_lock (&lru->l_lock); + progress =16384 ; + rotate =0 ; + } + } +spin_unlock (&lru->l_lock); + +dispose_list (&dispose); +fprintf (stderr,"cache size after %ld, count remaining %d\n" , lru->l_count, count); +/* * Now we've free a bunch of memory, trim the heap down to release the * freed memory back to the kernel and reduce the pressure we are
Signature · 1 line
2.28.0
Re: [PATCH 24/27] libxfs: add a buftarg cache shrinker implementation
Quoted 50 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Add a
list_lruscanner that runs from the memory pressure detection to free an amount of the buffer cache that will keep the cache from growing when there is memory pressure.Signed-off-by Dave Chinner <[email protected]>--- libxfs/buftarg.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)libxfs/buftarg.c +20
-0@@ -62,6 +62,19 @@ xfs_buftarg_setsize_early(return xfs_buftarg_setsize (btp, bsize); } +static void +dispose_list ( +struct list_head *dispose) +{ +struct xfs_buf *bp; + +while (!list_empty (dispose)) { + bp = list_first_entry(dispose, struct xfs_buf, b_lru); +list_del_init (&bp->b_lru); +xfs_buf_rele (bp); + } +} +/* * Scan a chunk of the buffer cache and drop LRU reference counts. If the * count goes to zero, dispose of the buffer. @@ -70,6 +83,13 @@ static voidxfs_buftarg_shrink ( struct xfs_buftarg *btc) { +struct list_lru *lru = &btc->bt_lru; +struct xfs_buf *bp; +int count; +int progress =16384 ; +int rotate =0 ; +LIST_HEAD (dispose); +/* * Make the fact we are in memory reclaim externally visible. This * allows buffer cache allocation throttling while we are trying to
libxfs/buftarg.c xfs_buftarg_shrink( · line 103
Quoted 5 lines
libxfs/buftarg.c +2
-0@@ -79,6 +99,37 @@ xfs_buftarg_shrink(fprintf (stderr,"Got memory pressure event. Shrinking caches!\n" ); +spin_lock (&lru->l_lock); + count = lru->l_count /50 ;/* 2% */
If I'm reading this correctly, we react to a memory pressure event by trying to skim 2% of the oldest disposable buffers off the buftarg LRU? And every 16384 loop iterations we'll dispose the list even if we haven't gotten our 2% yet? How did you arrive at 2%?
(Also, I'm assuming that some of these stderr printfs will at some point get turned into tracepoints or dbg_printf or the like?)
--D
Quoted 35 lines
libxfs/buftarg.c +29
-0+fprintf (stderr,"cache size before %ld/%d\n" , lru->l_count, count); +while (count-- >0 && !list_empty (&lru->l_lru)) { + bp = list_first_entry(&lru->l_lru, struct xfs_buf, b_lru); +spin_lock (&bp->b_lock); +if (!atomic_add_unless (&bp->b_lru_ref,-1 ,1 )) { +atomic_set (&bp->b_lru_ref,0 ); + bp->b_state |=XFS_BSTATE_DISPOSE ; +list_move (&bp->b_lru, &dispose); + lru->l_count--; + }else { + rotate++; +list_move_tail (&bp->b_lru, &lru->l_lru); + } + +spin_unlock (&bp->b_lock); +if (--progress ==0 ) { +fprintf (stderr,"Disposing! rotated %d, lru %ld\n" , rotate, lru->l_count); +spin_unlock (&lru->l_lock); +dispose_list (&dispose); +spin_lock (&lru->l_lock); + progress =16384 ; + rotate =0 ; + } + } +spin_unlock (&lru->l_lock); + +dispose_list (&dispose); +fprintf (stderr,"cache size after %ld, count remaining %d\n" , lru->l_count, count); +/* * Now we've free a bunch of memory, trim the heap down to release the * freed memory back to the kernel and reduce the pressure we are Signature · 1 line
2.28.0
Re: [PATCH 24/27] libxfs: add a buftarg cache shrinker implementation
Quoted 54 lines · Darrick J. Wong
Quoted 53 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Add a
list_lruscanner that runs from the memory pressure detection to free an amount of the buffer cache that will keep the cache from growing when there is memory pressure.Signed-off-by Dave Chinner <[email protected]>--- libxfs/buftarg.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)libxfs/buftarg.c +20
-0@@ -62,6 +62,19 @@ xfs_buftarg_setsize_early(return xfs_buftarg_setsize (btp, bsize); } +static void +dispose_list ( +struct list_head *dispose) +{ +struct xfs_buf *bp; + +while (!list_empty (dispose)) { + bp = list_first_entry(dispose, struct xfs_buf, b_lru); +list_del_init (&bp->b_lru); +xfs_buf_rele (bp); + } +} +/* * Scan a chunk of the buffer cache and drop LRU reference counts. If the * count goes to zero, dispose of the buffer. @@ -70,6 +83,13 @@ static voidxfs_buftarg_shrink ( struct xfs_buftarg *btc) { +struct list_lru *lru = &btc->bt_lru; +struct xfs_buf *bp; +int count; +int progress =16384 ; +int rotate =0 ; +LIST_HEAD (dispose); +/* * Make the fact we are in memory reclaim externally visible. This * allows buffer cache allocation throttling while we are trying to @@ -79,6 +99,37 @@ xfs_buftarg_shrink(fprintf (stderr,"Got memory pressure event. Shrinking caches!\n" );
Quoted 7 lines
libxfs/buftarg.c xfs_buftarg_shrink( · line 103
Quoted 2 lines
libxfs/buftarg.c +2
-0+spin_lock (&lru->l_lock); + count = lru->l_count /50 ;/* 2% */ If I'm reading this correctly, we react to a memory pressure event by trying to skim 2% of the oldest disposable buffers off the buftarg LRU? And every 16384 loop iterations we'll dispose the list even if we haven't gotten our 2% yet? How did you arrive at 2%?
Yup, 2% was the number I came up with. It's a trade-off between scanning enough to keep the cache growth in check but not so much as to trash the entire cache as stall events roll in. Also, the PSI monitor will report at most 1 event per second with the current config, so the amount of work the shrinker does doesn't need to consume lots of time.
The system I was testing on ended up OOMing at around 1.2M cached buffers. Hence each invocation was scanning ~25-30k buffers every invocation. This was sufficient to control memory usage, without the PSI stall event tail trashing the cache once repair-triggered memory pressure had been brought under control
The "progress" thing is just a way of batching up the work so that we free memory sooner. THe number of 16384 was from when I was discovering how this behaved and I was trimming up to 50% of the cache in a singel event. I needed some kind of progress indicaction while it stalled for seconds freeing memory. It may stay, it may go. We'll see.
Quoted 2 lines
(Also, I'm assuming that some of these stderr printfs will at some point get turned into tracepoints or
dbg_printfor the like?)
Maybe. I would prefer the tracepoint model over dbg_printf(), but that's not something I'm thinking about right now...
Note that this shrinker does not rotate buffers by default. The kernel rotates buffers once through the LRU before they are reclaimed. If I try to do that with PSI events, then we OOM kill because the system is completely out of memory by the time 5-6 events have been delivered and then we get OOM killed. Hence it reclaims immediately, but that can be tuned for repair by converting the cache priorities for buffers in LRU references...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 25/27 libxfs: switch buffer cache implementations
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Now the kernel buffer cache code is present, switch to using it. This kills off most of the old read/write code, and the cache implementation used to support it.
This requires changes to xfs_repair to handle cache usage reporting changes, along with how it sets up the cache sizes as we are moving from a single global cache to per-ag caches and so needs different logic. Cache size is controlled purely by manual purging - it will not respond to memory pressure or size limits yet.
XXX: xfs_buf_ioerror_alert() causes LTO linking failures in xfs_copy (and only xfs_copy, so a real WTF), so it's single caller in xfs_buf.c is commented out.
--- copy/xfs_copy.c | 8 +- db/io.c | 4 +- include/Makefile | 1 - include/cache.h | 133 ------ include/libxfs.h | 23 +- include/xfs_inode.h | 1 - include/xfs_mount.h | 7 + libxfs/Makefile | 2 - libxfs/buftarg.c | 150 ++----- libxfs/cache.c | 724 -------------------------------- libxfs/init.c | 74 ++-- libxfs/libxfs_api_defs.h | 4 + libxfs/libxfs_priv.h | 13 - libxfs/rdwr.c | 869 +-------------------------------------- libxfs/trans.c | 1 - libxfs/util.c | 1 - libxfs/xfs_buf.c | 97 +++-- libxfs/xfs_buf.h | 51 ++- libxfs/xfs_buftarg.h | 75 ++-- mkfs/xfs_mkfs.c | 19 +- repair/attr_repair.c | 6 +- repair/da_util.c | 2 +- repair/dino_chunks.c | 4 +- repair/dinode.c | 4 +- repair/phase3.c | 7 +- repair/phase4.c | 5 +- repair/prefetch.c | 65 +-- repair/progress.c | 12 +- repair/progress.h | 4 +- repair/scan.c | 6 +- repair/xfs_repair.c | 190 +++++---- 31 files changed, 400 insertions(+), 2162 deletions(-) delete mode 100644 include/cache.h delete mode 100644 libxfs/cache.c
copy/xfs_copy.c +3 -5
@@ -17,7 +17,7 @@#define rounddown (x, y) (((x)/(y))*(y))#define uuid_equal (s,d) (platform_uuid_compare((s),(d)) == 0) -extern int platform_check_ismounted (char *,char *,struct stat *,int ); +//extern int platform_check_ismounted(char *, char *, struct stat *, int); static char *logfile_name;static FILE *logerr;
@@ -49,8 +49,6 @@ static pthread_mutex_t mainwait;#define ACTIVE 1#define INACTIVE 2 -xfs_off_t write_log_trailer (int fd,wbuf *w,xfs_mount_t *mp); -xfs_off_t write_log_header (int fd,wbuf *w,xfs_mount_t *mp);static int format_logs (struct xfs_mount *);/* general purpose message reporting routine */
@@ -1261,7 +1259,7 @@ next_log_chunk(char *p, int offset, void *private)* * Returns the next buffer-length-aligned disk address. */ -xfs_off_t +static xfs_off_t write_log_header (int fd,wbuf *buf,xfs_mount_t *mp) {char *p = buf->data;
@@ -1293,7 +1291,7 @@ write_log_header(int fd, wbuf *buf, xfs_mount_t *mp)* the start of that buffer). Returns the disk address at the * end of last aligned buffer in the log. */ -xfs_off_t +static xfs_off_t write_log_trailer (int fd,wbuf *buf,xfs_mount_t *mp) {xfs_off_t logend;
db/io.c +2 -2
@@ -525,11 +525,11 @@ set_cur(return ;memcpy (iocur_top->bbmap, bbmap,sizeof (struct bbmap )); error = -libxfs_buf_read_map (mp->m_ddev_targp, bbmap->b, - bbmap->nmaps,LIBXFS_READBUF_SALVAGE , &bp, + bbmap->nmaps,XBF_SALVAGE , &bp, ops); }else { error = -libxfs_buf_read (mp->m_ddev_targp, blknum, len, -LIBXFS_READBUF_SALVAGE , &bp, ops); +XBF_SALVAGE , &bp, ops); iocur_top->bbmap =NULL ; }
include/Makefile +0 -1
@@ -11,7 +11,6 @@ LIBHFILES = libxfs.h \ libxcmd.h \ atomic.h \ bitops.h \ - cache.h \ completion.h \ hlist.h \ kmem.h \
include/cache.h deleted +0 -133
133 lines not on this page · raw message
include/libxfs.h +13 -10
@@ -13,7 +13,6 @@#include "list.h" #include "hlist.h" -#include "cache.h" #include "bitops.h" #include "kmem.h" #include "libfrog/radix-tree.h"
@@ -53,7 +52,6 @@ struct iomap;*/ #include "xfs_buftarg.h" #include "xfs_buf.h" -#include "libxfs_io.h" #include "xfs_bit.h" #include "xfs_sb.h"
@@ -138,15 +136,20 @@ typedef struct libxfs_xinit {#define LIBXFS_EXCLUSIVELY 0x0010/* disallow other accesses (O_EXCL) */ #define LIBXFS_DIRECT 0x0020/* can use direct I/O, not buffered */ -extern char *progname; +extern char *progname;extern xfs_lsn_t libxfs_max_lsn; -extern int libxfs_init (libxfs_init_t *); -void libxfs_destroy (struct libxfs_xinit *li); -extern int libxfs_device_to_fd (dev_t ); -extern dev_t libxfs_device_open (char *,int ,int ,int ); -extern void libxfs_device_close (dev_t ); -extern int libxfs_device_alignment (void ); -extern void libxfs_report (FILE *); +extern int libxfs_bhash_size; + +int libxfs_init (libxfs_init_t *); +void libxfs_destroy (struct libxfs_xinit *li); +int libxfs_device_to_fd (dev_t ); +dev_t libxfs_device_open (char *,int ,int ,int ); +void libxfs_open_devices (struct xfs_mount *mp,dev_t ddev,dev_t logdev, +dev_t rtdev); +void libxfs_device_close (dev_t ); +int libxfs_device_alignment (void ); +int libxfs_device_zero (struct xfs_buftarg *btp,xfs_daddr_t start,uint len); +void libxfs_report (FILE *);/* check or write log footer: specify device, log size in blocks & uuid */ typedef char *(libxfs_get_block_t )(char *,int ,void *);
include/xfs_inode.h +0 -1
@@ -63,7 +63,6 @@ static inline void i_gid_write(struct inode *inode, uint32_t gid) } typedef struct xfs_inode { - struct cache_node i_node; struct xfs_mount *i_mount; /* fs mount struct ptr */ xfs_ino_t i_ino; /* inode number (agno/agino) */ struct xfs_imap i_imap; /* location for xfs_imap() */
include/xfs_mount.h +7 -0
@@ -176,6 +176,11 @@ xfs_perag_resv( } } +#define xfs_daddr_to_agno (mp,d) \ + ((xfs_agnumber_t)(XFS_BB_TO_FSBT(mp, d) / (mp)->m_sb.sb_agblocks)) +#define xfs_daddr_to_agbno (mp,d) \ + ((xfs_agblock_t)(XFS_BB_TO_FSBT(mp, d) % (mp)->m_sb.sb_agblocks)) +#define LIBXFS_MOUNT_DEBUGGER 0x0001#define LIBXFS_MOUNT_32BITINODES 0x0002#define LIBXFS_MOUNT_32BITINOOPT 0x0004
@@ -190,4 +195,6 @@ extern xfs_mount_t *libxfs_mount (xfs_mount_t *, xfs_sb_t *,int libxfs_umount (struct xfs_mount *mp);extern void libxfs_rtmount_destroy (xfs_mount_t *); +struct xfs_buf *libxfs_getsb (struct xfs_mount *mp); +#endif /* __XFS_MOUNT_H__ */
libxfs/Makefile +0 -2
@@ -50,14 +50,12 @@ HFILES = \ xfs_shared.h \ xfs_trans_resv.h \ xfs_trans_space.h \ - libxfs_io.h \ libxfs_api_defs.h \ init.h \ libxfs_priv.h \ xfs_dir2_priv.h CFILES = buftarg.c \ - cache.c \ defer_item.c \ init.c \ kmem.c \
libxfs/buftarg.c +28 -122
232 lines not on this page · raw message
libxfs/cache.c deleted +0 -724
724 lines not on this page · raw message
libxfs/init.c +34 -40
166 lines not on this page · raw message
libxfs/libxfs_api_defs.h +4 -0
@@ -47,14 +47,18 @@#define xfs_btree_bload_compute_geometry libxfs_btree_bload_compute_geometry#define xfs_btree_del_cursor libxfs_btree_del_cursor#define xfs_btree_init_block libxfs_btree_init_block +#define xfs_blkdev_issue_flush libxfs_blkdev_issue_flush#define xfs_buf_delwri_submit libxfs_buf_delwri_submit#define xfs_buf_get libxfs_buf_get#define xfs_buf_get_map libxfs_buf_get_map#define xfs_buf_get_uncached libxfs_buf_get_uncached +#define xfs_buf_mark_dirty libxfs_buf_mark_dirty#define xfs_buf_read libxfs_buf_read#define xfs_buf_read_map libxfs_buf_read_map#define xfs_buf_read_uncached libxfs_buf_read_uncached#define xfs_buf_relse libxfs_buf_relse +#define xfs_buf_reverify libxfs_buf_reverify +#define xfs_buftarg_purge_ag libxfs_buftarg_purge_ag#define xfs_bunmapi libxfs_bunmapi#define xfs_bwrite libxfs_bwrite#define xfs_calc_dquots_per_chunk libxfs_calc_dquots_per_chunk
libxfs/libxfs_priv.h +0 -13
@@ -43,7 +43,6 @@#include "list.h" #include "hlist.h" -#include "cache.h" #include "bitops.h" #include "kmem.h" #include "libfrog/radix-tree.h"
@@ -88,7 +87,6 @@ struct iomap;*/ #include "xfs_buftarg.h" #include "xfs_buf.h" -#include "libxfs_io.h" /* for all the support code that uses progname in error messages */ extern char *progname;
@@ -386,17 +384,6 @@ howmany_64(uint64_t x, uint32_t y)return x; } -/* buffer management */ -#define XFS_BUF_UNDELAYWRITE (bp) ((bp)->b_flags &= ~LIBXFS_B_DIRTY) - -#define xfs_buf_oneshot (bp) ((void) 0) - -#define xfs_buf_zero (bp, off, len) \ - memset((bp)->b_addr + off, 0, len); - -void __xfs_buf_mark_corrupt (struct xfs_buf *bp,xfs_failaddr_t fa); -#define xfs_buf_mark_corrupt (bp) __xfs_buf_mark_corrupt((bp), __this_address) -/* mount stuff */ #define XFS_MOUNT_32BITINODES LIBXFS_MOUNT_32BITINODES#define XFS_MOUNT_ATTR2 LIBXFS_MOUNT_ATTR2
libxfs/rdwr.c +10 -859
931 lines not on this page · raw message
libxfs/trans.c +0 -1
@@ -672,7 +672,6 @@ libxfs_trans_binval(if (bip->bli_flags &XFS_BLI_STALE )return ; -XFS_BUF_UNDELAYWRITE (bp);xfs_buf_stale (bp); bip->bli_flags |=XFS_BLI_STALE ;
libxfs/util.c +0 -1
@@ -6,7 +6,6 @@#include "libxfs_priv.h" #include "libxfs.h" -#include "libxfs_io.h" #include "init.h" #include "xfs_fs.h" #include "xfs_shared.h"
libxfs/xfs_buf.c +52 -45
276 lines not on this page · raw message
libxfs/xfs_buf.h +42 -9
102 lines not on this page · raw message
libxfs/xfs_buftarg.h +28 -47
110 lines not on this page · raw message
mkfs/xfs_mkfs.c +10 -9
@@ -3497,10 +3497,10 @@ prepare_devices(* the end of the device. (MD sb is ~64k from the end, take out a wider * swath to be sure) */ - error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp, - (xi->dsize - whack_blks), whack_blks, &buf); + error =xfs_buf_get_uncached (mp->m_ddev_targp, whack_blks,0 , &buf);if (error) goto out_error; + buf->b_maps[0 ].bm_bn = xi->dsize - whack_blks;memset (buf->b_addr,0 ,WHACK_SIZE );libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);
@@ -3511,19 +3511,21 @@ prepare_devices(* swap (somewhere around the page size), jfs (32k), * ext[2,3] and reiserfs (64k) - and hopefully all else. */ - error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp,0 , whack_blks, &buf); + error =xfs_buf_get_uncached (mp->m_ddev_targp, whack_blks,0 , &buf);if (error) goto out_error; + buf->b_maps[0 ].bm_bn =0 ;memset (buf->b_addr,0 ,WHACK_SIZE );libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);/* OK, now write the superblock... */ - error =xfs_buf_get_uncached_daddr (mp->m_ddev_targp,XFS_SB_DADDR , -XFS_FSS_TO_BB (mp,1 ), &buf); + error =xfs_buf_get_uncached (mp->m_ddev_targp,XFS_FSS_TO_BB (mp,1 ),0 , + &buf);if (error) goto out_error; buf->b_ops = &xfs_sb_buf_ops; + buf->b_maps[0 ].bm_bn =XFS_SB_DADDR ;memset (buf->b_addr,0 , cfg->sectorsize);libxfs_sb_to_disk (buf->b_addr, sbp);libxfs_buf_mark_dirty (buf);
@@ -3543,11 +3545,11 @@ prepare_devices(/* finally, check we can write the last block in the realtime area */ if (mp->m_rtdev_targp && mp->m_rtdev_targp->bt_bdev && cfg->rtblocks >0 ) { - error =xfs_buf_get_uncached_daddr (mp->m_rtdev_targp, -XFS_FSB_TO_BB (mp, cfg->rtblocks -1LL ), -BTOBB (cfg->blocksize), &buf); + error =xfs_buf_get_uncached (mp->m_rtdev_targp, +BTOBB (cfg->blocksize),0 , &buf);if (error) goto out_error; + buf->b_maps[0 ].bm_bn =XFS_FSB_TO_BB (mp, cfg->rtblocks -1LL );memset (buf->b_addr,0 , cfg->blocksize);libxfs_buf_mark_dirty (buf);libxfs_buf_relse (buf);
@@ -4070,7 +4072,6 @@ main(* Need to drop references to inodes we still hold, first. */ libxfs_rtmount_destroy (mp); -libxfs_bcache_purge ();/* * Mark the filesystem ok.
repair/attr_repair.c +3 -3
@@ -407,7 +407,7 @@ rmtval_get(xfs_mount_t *mp, xfs_ino_t ino, blkmap_t *blkmap,break ; } error = -libxfs_buf_read (mp->m_dev,XFS_FSB_TO_DADDR (mp, bno), -XFS_FSB_TO_BB (mp,1 ),LIBXFS_READBUF_SALVAGE , +XFS_FSB_TO_BB (mp,1 ),XBF_SALVAGE , &bp, &xfs_attr3_rmt_buf_ops);if (error) { do_warn(
@@ -767,7 +767,7 @@ process_leaf_attr_level(xfs_mount_t *mp, error = -libxfs_buf_read (mp->m_dev,XFS_FSB_TO_DADDR (mp, dev_bno), -XFS_FSB_TO_BB (mp,1 ),LIBXFS_READBUF_SALVAGE , +XFS_FSB_TO_BB (mp,1 ),XBF_SALVAGE , &bp, &xfs_attr3_leaf_buf_ops);if (error) { do_warn(
@@ -1099,7 +1099,7 @@ process_longform_attr( } error = -libxfs_buf_read (mp->m_dev,XFS_FSB_TO_DADDR (mp, bno), -XFS_FSB_TO_BB (mp,1 ),LIBXFS_READBUF_SALVAGE , &bp, +XFS_FSB_TO_BB (mp,1 ),XBF_SALVAGE , &bp, &xfs_da3_node_buf_ops);if (error) { do_warn(
repair/da_util.c +1 -1
@@ -64,7 +64,7 @@ da_read_buf( map[i].bm_bn =XFS_FSB_TO_DADDR (mp, bmp[i].startblock); map[i].bm_len =XFS_FSB_TO_BB (mp, bmp[i].blockcount); } -libxfs_buf_read_map (mp->m_dev, map, nex,LIBXFS_READBUF_SALVAGE , +libxfs_buf_read_map (mp->m_dev, map, nex,XBF_SALVAGE , &bp, ops);if (map != map_array)free (map);
repair/dino_chunks.c +2 -2
@@ -41,7 +41,7 @@ check_aginode_block(xfs_mount_t *mp,* so no one else will overlap them. */ error = -libxfs_buf_read (mp->m_dev,XFS_AGB_TO_DADDR (mp, agno, agbno), -XFS_FSB_TO_BB (mp,1 ),LIBXFS_READBUF_SALVAGE , &bp, +XFS_FSB_TO_BB (mp,1 ),XBF_SALVAGE , &bp,NULL );if (error) { do_warn(_("cannot read agbno (%u/%u), disk block %" PRId64 "\n"),
@@ -669,7 +669,7 @@ process_inode_chunk(XFS_AGB_TO_DADDR (mp, agno, agbno),XFS_FSB_TO_BB (mp,M_IGEO (mp)->blocks_per_cluster), -LIBXFS_READBUF_SALVAGE , &bplist[bp_index], +XBF_SALVAGE , &bplist[bp_index], &xfs_inode_buf_ops);if (error) { do_warn(_("cannot read inode %" PRIu64 ", disk block %" PRId64 ", cnt %d\n"),
repair/dinode.c +2 -2
@@ -1106,7 +1106,7 @@ process_quota_inode( error = -libxfs_buf_read (mp->m_dev,XFS_FSB_TO_DADDR (mp, fsbno), dqchunklen, -LIBXFS_READBUF_SALVAGE , &bp, +XBF_SALVAGE , &bp, &xfs_dquot_buf_ops);if (error) { do_warn(
@@ -1218,7 +1218,7 @@ _("cannot read inode %" PRIu64 ", file block %d, NULL disk block\n"), error = -libxfs_buf_read (mp->m_dev,XFS_FSB_TO_DADDR (mp, fsbno),BTOBB (byte_cnt), -LIBXFS_READBUF_SALVAGE , &bp, +XBF_SALVAGE , &bp, &xfs_symlink_buf_ops);if (error) { do_warn(
repair/phase3.c +5 -2
@@ -31,7 +31,7 @@ process_agi_unlinked( error = -libxfs_buf_read (mp->m_dev,XFS_AG_DADDR (mp, agno,XFS_AGI_DADDR (mp)), - mp->m_sb.sb_sectsize /BBSIZE ,LIBXFS_READBUF_SALVAGE , + mp->m_sb.sb_sectsize /BBSIZE ,XBF_SALVAGE , &bp, &xfs_agi_buf_ops);if (error) do_error(_("cannot read agi block %" PRId64 " for ag %u\n"),
@@ -62,15 +62,18 @@ process_ag_func(xfs_agnumber_t agno, void *arg)
Lines of this file are not shown. View the raw message.
repair/prefetch.c +42 -23
65 lines not on this page · raw message
repair/xfs_repair.c +93 -97
190 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.28.0
Part of the diff is not shown here. View the raw message.
PATCH 26/27 build: set platform_defs.h.in dependency correctly
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
So that changing this file causes configure to be re-run and platform_defs.h to be rebuilt from the new template.
--- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
@@ -130,7 +130,7 @@ configure: configure.ac include/builddefs: configure ./configure $$LOCAL_CONFIGURE_OPTIONS -include/platform_defs.h: include/builddefs +include/platform_defs.h: include/builddefs include/platform_defs.h.in ## Recover from the removal of $@ @if test -f $@; then :; else \ rm -f include/builddefs; \
Signature · 1 line
2.28.0
PATCH 27/27 libxfs: convert sync IO buftarg engine to AIO
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Simple per-ag thread based completion engine. Will have issues with large AG counts.
XXX: should this be combined with the struct btcache?
--- include/atomic.h | 7 +- include/builddefs.in | 2 +- include/platform_defs.h.in | 1 + libxfs/buftarg.c | 202 +++++++++++++++++++++++++++++++------ libxfs/xfs_buf.h | 6 ++ libxfs/xfs_buftarg.h | 7 ++ 6 files changed, 191 insertions(+), 34 deletions(-)
include/atomic.h +5 -2
@@ -27,8 +27,11 @@ typedef int64_t atomic64_t;#define atomic_inc_return (a) uatomic_add_return(a, 1)#define atomic_dec_return (a) uatomic_sub_return(a, 1) -#define atomic_inc (a) atomic_inc_return(a) -#define atomic_dec (a) atomic_inc_return(a) +#define atomic_add (a, v) uatomic_add(a, v) +#define atomic_sub (a, v) uatomic_sub(a, v) + +#define atomic_inc (a) uatomic_inc(a) +#define atomic_dec (a) uatomic_dec(a)#define atomic_dec_and_test (a) (atomic_dec_return(a) == 0)
include/builddefs.in +1 -1
@@ -29,7 +29,7 @@ LIBEDITLINE = @libeditline@ LIBBLKID = @libblkid@ LIBDEVMAPPER = @libdevmapper@ LIBINIH = @libinih@ -LIBXFS = $(TOPDIR)/libxfs/libxfs.la +LIBXFS = $(TOPDIR)/libxfs/libxfs.la -laio LIBFROG = $(TOPDIR)/libfrog/libfrog.la LIBXCMD = $(TOPDIR)/libxcmd/libxcmd.la LIBXLOG = $(TOPDIR)/libxlog/libxlog.la
include/platform_defs.h.in +1 -0
@@ -24,6 +24,7 @@ #include <stdbool.h> #include <libgen.h> #include <urcu.h> +#include <libaio.h> typedef struct filldir filldir_t;
libxfs/buftarg.c +171 -31
248 lines not on this page · raw message
libxfs/xfs_buf.h +6 -0
@@ -81,6 +81,12 @@ struct xfs_buf {struct completion b_iowait;struct semaphore b_sema;xfs_buf_iodone_t b_iodone; + +/* +* XXX: AIO needs as many iocbs as we have maps for discontig +* buffers to work correctly. +*/ +struct iocb b_iocb; }; struct xfs_buf *xfs_buf_incore(struct xfs_buftarg *target,
libxfs/xfs_buftarg.h +7 -0
@@ -16,6 +16,7 @@ struct xfs_buf_ops;struct xfs_buf ;struct xfs_buf_map ;struct xfs_mount ; +struct xfs_btaio ;/* this needs to die */ #define LIBXFS_BBTOOFF64 (bbs) (((xfs_off_t)(bbs)) << BBSHIFT)
@@ -55,6 +56,9 @@ struct xfs_buftarg {bool bt_exiting;bool bt_low_mem;struct completion bt_low_mem_wait; + +/* AIO submission/completion structures */ +struct xfs_btaio *bt_aio; };/* We purged a dirty buffer and lost a write. */
@@ -90,6 +94,9 @@ int xfs_buf_associate_memory(struct xfs_buf *bp, void *mem, size_t length);void xfs_buftarg_submit_io (struct xfs_buf *bp);void xfs_buf_mark_dirty (struct xfs_buf *bp); +int xfs_buftarg_aio_init (struct xfs_btaio **aiop); +void xfs_buftarg_aio_destroy (struct xfs_btaio *aio); +/* * Cached buffer memory manangement */
Signature · 1 line
2.28.0
Re: [PATCH 27/27] libxfs: convert sync IO buftarg engine to AIO
Quoted 4 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Simple per-ag thread based completion engine. Will have issues with large AG counts.
Hm, I missed how any of this is per-AG... all I saw was an aio control context that's attached to the buftarg.
Quoted 17 lines
XXX: should this be combined with the struct btcache?
Signed-off-by Dave Chinner <[email protected]>--- include/atomic.h | 7 +- include/builddefs.in | 2 +- include/platform_defs.h.in | 1 + libxfs/buftarg.c | 202 +++++++++++++++++++++++++++++++------ libxfs/xfs_buf.h | 6 ++ libxfs/xfs_buftarg.h | 7 ++ 6 files changed, 191 insertions(+), 34 deletions(-)include/atomic.h +0
-0
include/atomic.h typedef int64_t atomic64_t; · line 34
Quoted 10 lines
include/atomic.h +5
-2@@ -27,8 +27,11 @@ typedef int64_t atomic64_t;#define atomic_inc_return (a) uatomic_add_return(a, 1)#define atomic_dec_return (a) uatomic_sub_return(a, 1) -#define atomic_inc (a) atomic_inc_return(a) -#define atomic_dec (a) atomic_inc_return(a) +#define atomic_add (a, v) uatomic_add(a, v) +#define atomic_sub (a, v) uatomic_sub(a, v) + +#define atomic_inc (a) uatomic_inc(a) +#define atomic_dec (a) uatomic_dec(a)
Does this belong in the liburcu patch?
include/builddefs.in LIBEDITLINE = @libeditline@ · line 32
Quoted 13 lines
include/atomic.h +0
-0#define atomic_dec_and_test (a) (atomic_dec_return(a) == 0)include/builddefs.in +1
-1@@ -29,7 +29,7 @@ LIBEDITLINE = @libeditline@ LIBBLKID = @libblkid@ LIBDEVMAPPER = @libdevmapper@ LIBINIH = @libinih@ -LIBXFS = $(TOPDIR)/libxfs/libxfs.la +LIBXFS = $(TOPDIR)/libxfs/libxfs.la -laio
This needs all the autoconf magic to complain if libaio isn't installed, right?
Quoted 92 lines
include/builddefs.in +0
-0LIBFROG = $(TOPDIR)/libfrog/libfrog.la LIBXCMD = $(TOPDIR)/libxcmd/libxcmd.la LIBXLOG = $(TOPDIR)/libxlog/libxlog.lainclude/platform_defs.h.in +1
-0@@ -24,6 +24,7 @@ #include <stdbool.h> #include <libgen.h> #include <urcu.h> +#include <libaio.h> typedef struct filldir filldir_t;libxfs/buftarg.c +24
-23@@ -259,11 +259,16 @@ xfs_buftarg_alloc(if (percpu_counter_init (&btp->bt_io_count,0 ,GFP_KERNEL )) goto error_lru; -if (xfs_buftarg_mempressue_init (btp)) +if (xfs_buftarg_aio_init (&btp->bt_aio)) goto error_pcp; +if (xfs_buftarg_mempressue_init (btp)) + goto error_aio; +return btp; +error_aio: +xfs_buftarg_aio_destroy (btp->bt_aio); error_pcp:percpu_counter_destroy (&btp->bt_io_count); error_lru:@@ -286,6 +291,7 @@ xfs_buftarg_free(if (btp->bt_psi_fd >=0 )close (btp->bt_psi_fd); +xfs_buftarg_aio_destroy (btp->bt_aio);ASSERT (percpu_counter_sum (&btp->bt_io_count) ==0 );percpu_counter_destroy (&btp->bt_io_count);platform_flush_device (btp->bt_fd, btp->bt_bdev);@@ -329,39 +335,132 @@ xfs_buf_allocate_memory(*/ /* -* XXX: this will be replaced by an AIO submission engine in future. In the mean -* time, just complete the IO synchronously so all the machinery still works. +* AIO context for dispatch and completion. +* +* Run completion polling in a separate thread, the poll timeout will stop it +* from spinning in tight loops when there is nothing to do. */ -static int -submit_io ( -struct xfs_buf *bp, -int fd, -void *buf, -xfs_daddr_t blkno, -int size, -bool write) +#define MAX_AIO_EVENTS 1024 +struct xfs_btaio { +io_context_t ctxp; +int aio_fd; +int aio_in_flight; +pthread_t completion_tid; +bool done; +}; + +static void +xfs_buf_aio_ioend ( +struct io_event *ev) { -int ret; +struct xfs_buf *bp = (struct xfs_buf *)ev->data; -if (!write) - ret =pread (fd, buf, size,BBTOB (blkno)); -else - ret =pwrite (fd, buf, size,BBTOB (blkno)); -if (ret <0 ) - ret = -errno; -else if (ret != size) - ret = -EIO ; -else - ret =0 ;/* -* This is a bit of a hack until we get AIO that runs completions.
libxfs/buftarg.c xfs_buf_allocate_memory( · line 365
Quoted 10 lines
libxfs/buftarg.c +7
-3-* Success is treated as a completion here, but IO errors are handled as -* a submission error and are handled by the caller. AIO will clean this -* up. +* don't overwrite existing errors - otherwise we can lose errors on +* buffers that require multiple bios to complete. +* +* We check that the returned length was the same as specified for this +* IO. Note that this onyl works for read and write - if we start +* using readv/writev for discontiguous buffers then this needs more +* work.
Interesting RFC, though I gather discontig buffers don't work yet? Or hmm maybe there's something funny going on with xfs_buftarg_submit_io_map? You didn't post a git branch link, so it's hard(er) to just rummage around on my own. :/
This seems like a reasonable restructuring to allow asynchronous io. It's sort of a pity that this isn't modular enough to allow multiple IO engines (synchronous system calls and io_uring come to mine) but that can come later.
Urp, I'll go reply to the cover letter now.
--D
Quoted 223 lines
libxfs/buftarg.c +140
-5*/ -if (!ret) +if (ev->res <0 || ev->res != ev->obj->u.c.nbytes) { +int error = ev->res <0 ? (int )ev->res : -EIO ; + +cmpxchg (&bp->b_io_error,0 , error); + } + +if (atomic_dec_and_test (&bp->b_io_remaining))xfs_buf_ioend (bp); -return ret; +} + +static void +get_io_completions ( +struct xfs_btaio *aio, +int threshold) +{ +struct io_event ioevents[MAX_AIO_EVENTS ]; +struct timespec tout = { + .tv_nsec =100 *1000 *1000 ,/* 100ms */ + }; +int i, r; + +/* gather up some completions */ + r =io_getevents (aio->ctxp,1 ,MAX_AIO_EVENTS , ioevents, &tout); +if (r <0 ) { +fprintf (stderr,"FAIL! io_getevents returned %d\n" , r); +if (r == -EINTR ) +return ; +exit (1 ); + } +if (r ==0 ) { +/* timeout, return to caller to check for what to do next */ +return ; + } + +atomic_sub (&aio->aio_in_flight, r); +for (i =0 ; i < r; ++i) +xfs_buf_aio_ioend (&ioevents[i]); +} + +static void * +aio_completion_thread ( +void *arg) +{ +struct xfs_btaio *aio = arg; + +while (!aio->done) { +get_io_completions (aio,1 ); + } +return NULL ; +} + +static int +submit_aio ( +struct xfs_buf *bp, +void *buf, +xfs_daddr_t blkno, +int size) +{ +struct xfs_btaio *aio = bp->b_target->bt_aio; +int r; + +if (!aio->aio_fd) + aio->aio_fd = bp->b_target->bt_fd; + +/* +* Reserve and bound the number of in flight IOs to keep the number of +* pending IOs overrunning the tail of the event loop. This also serves +* to throttle incoming IOs without burning CPU by spinning. +*/ +while (!atomic_add_unless (&aio->aio_in_flight,1 ,MAX_AIO_EVENTS -1 )) +get_io_completions (aio,1 ); + +if (bp->b_flags &XBF_WRITE ) +io_prep_pwrite (&bp->b_iocb, aio->aio_fd, buf, size,BBTOB (blkno)); +else +io_prep_pread (&bp->b_iocb, aio->aio_fd, buf, size,BBTOB (blkno)); + + bp->b_iocb.data = bp; +do { +struct iocb *iocb; + + iocb = &bp->b_iocb; + r =io_submit (aio->ctxp,1 , &iocb); +if (r ==1 ) +return 0 ;/* successful submission */ +fprintf (stderr,"io_submit returned %d\n" , r); +if (r != -EAGAIN ) +break ; +/* On EAGAIN, reap some completions and try again. */ +get_io_completions (aio,1 ); + }while (1 ); + +if (bp->b_flags &LIBXFS_B_EXIT ) +exit (1 ); + +atomic_dec (&aio->aio_in_flight); +return r; } static void@@ -373,7 +472,6 @@ xfs_buftarg_submit_io_map( {int size;int offset; -bool rw = (bp->b_flags &XBF_WRITE );int error; offset = *buf_offset;@@ -388,8 +486,7 @@ xfs_buftarg_submit_io_map(atomic_inc (&bp->b_io_remaining); - error =submit_io (bp, bp->b_target->bt_fd, bp->b_addr + offset, - bp->b_maps[map].bm_bn, size, rw); + error =submit_aio (bp, bp->b_addr + offset, bp->b_maps[map].bm_bn, size);if (error) {/* * This is guaranteed not to be the last io reference count @@ -474,6 +571,49 @@ xfs_buftarg_submit_io( } } +int +xfs_buftarg_aio_init ( +struct xfs_btaio **aiop) +{ +struct xfs_btaio *aio; +int r; + + aio =calloc (1 ,sizeof (*aio)); +if (!aio) +return -ENOMEM ; + + r =io_setup (MAX_AIO_EVENTS , &aio->ctxp); +if (r) { +printf ("FAIL! io_setup returned %d\n" , r); + goto free_aio; + } + + r =pthread_create (&aio->completion_tid,NULL , aio_completion_thread, + aio); +if (r) { +printf ("FAIL! aio thread create returned %d\n" , r); + goto free_aio; + } + + *aiop = aio; +return 0 ; + +free_aio: +free (aio); +return r; +} + +void +xfs_buftarg_aio_destroy ( +struct xfs_btaio *aio) +{ +if (!aio) +return ; + + aio->done = true; +pthread_join (aio->completion_tid,NULL ); +free (aio); +}/* * Return a buffer associated to external memory via xfs_buf_associate_memory() * back to it's empty state. libxfs/xfs_buf.h +6
-0@@ -81,6 +81,12 @@ struct xfs_buf {struct completion b_iowait;struct semaphore b_sema;xfs_buf_iodone_t b_iodone; + +/* +* XXX: AIO needs as many iocbs as we have maps for discontig +* buffers to work correctly. +*/ +struct iocb b_iocb; }; struct xfs_buf *xfs_buf_incore(struct xfs_buftarg *target,libxfs/xfs_buftarg.h +7
-0@@ -16,6 +16,7 @@ struct xfs_buf_ops;struct xfs_buf ;struct xfs_buf_map ;struct xfs_mount ; +struct xfs_btaio ;/* this needs to die */ #define LIBXFS_BBTOOFF64 (bbs) (((xfs_off_t)(bbs)) << BBSHIFT)@@ -55,6 +56,9 @@ struct xfs_buftarg {bool bt_exiting;bool bt_low_mem;struct completion bt_low_mem_wait; + +/* AIO submission/completion structures */ +struct xfs_btaio *bt_aio; };/* We purged a dirty buffer and lost a write. */ @@ -90,6 +94,9 @@ int xfs_buf_associate_memory(struct xfs_buf *bp, void *mem, size_t length);void xfs_buftarg_submit_io (struct xfs_buf *bp);void xfs_buf_mark_dirty (struct xfs_buf *bp); +int xfs_buftarg_aio_init (struct xfs_btaio **aiop); +void xfs_buftarg_aio_destroy (struct xfs_btaio *aio); +/* * Cached buffer memory manangement */ Signature · 1 line
2.28.0
Re: [PATCH 27/27] libxfs: convert sync IO buftarg engine to AIO
Quoted 8 lines · Darrick J. Wong
Quoted 4 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Simple per-ag thread based completion engine. Will have issues with large AG counts.
Hm, I missed how any of this is per-AG... all I saw was an aio control context that's attached to the buftarg.
Stale - I got rid of the per-ag AIO structures because having 500 completion threads in gdb is a PITA and it provided no better performance. i.e. it had problems with large AG counts.
Quoted 20 lines
Quoted 20 lines
XXX: should this be combined with the struct btcache?
Signed-off-by Dave Chinner <[email protected]>--- include/atomic.h | 7 +- include/builddefs.in | 2 +- include/platform_defs.h.in | 1 + libxfs/buftarg.c | 202 +++++++++++++++++++++++++++++++------ libxfs/xfs_buf.h | 6 ++ libxfs/xfs_buftarg.h | 7 ++ 6 files changed, 191 insertions(+), 34 deletions(-)include/atomic.h +0
-0@@ -27,8 +27,11 @@ typedef int64_t atomic64_t;#define atomic_inc_return (a) uatomic_add_return(a, 1)#define atomic_dec_return (a) uatomic_sub_return(a, 1)
Quoted 9 lines
include/atomic.h typedef int64_t atomic64_t; · line 34
Quoted 7 lines
include/atomic.h +5
-2-#define atomic_inc (a) atomic_inc_return(a) -#define atomic_dec (a) atomic_inc_return(a) +#define atomic_add (a, v) uatomic_add(a, v) +#define atomic_sub (a, v) uatomic_sub(a, v) + +#define atomic_inc (a) uatomic_inc(a) +#define atomic_dec (a) uatomic_dec(a)Does this belong in the liburcu patch?
Probably.
Quoted 4 lines
Quoted 4 lines
include/atomic.h +0
-0#define atomic_dec_and_test (a) (atomic_dec_return(a) == 0)
Quoted 13 lines
include/builddefs.in LIBEDITLINE = @libeditline@ · line 32
Quoted 10 lines
include/builddefs.in +1
-1@@ -29,7 +29,7 @@ LIBEDITLINE = @libeditline@ LIBBLKID = @libblkid@ LIBDEVMAPPER = @libdevmapper@ LIBINIH = @libinih@ -LIBXFS = $(TOPDIR)/libxfs/libxfs.la +LIBXFS = $(TOPDIR)/libxfs/libxfs.la -laioThis needs all the autoconf magic to complain if libaio isn't installed, right?
Yes.
Quoted 6 lines
Quoted 6 lines
include/builddefs.in +1
-4/* - * This is a bit of a hack until we get AIO that runs completions. - * Success is treated as a completion here, but IO errors are handled as - * a submission error and are handled by the caller. AIO will clean this - * up. + * don't overwrite existing errors - otherwise we can lose errors on
Quoted 8 lines
include/builddefs.in
Quoted 6 lines
include/builddefs.in +6
-0+ * buffers that require multiple bios to complete. + * + * We check that the returned length was the same as specified for this + * IO. Note that this onyl works for read and write - if we start + * using readv/writev for discontiguous buffers then this needs more + * work.Interesting RFC, though I gather discontig buffers don't work yet?
Right they don't work yet. I mention that in a couple of places.
Quoted 8 lines
Or hmm maybe there's something funny going on with
xfs_buftarg_submit_io_map? You didn't post a git branch link, so it's hard(er) to just rummage around on my own. :/This seems like a reasonable restructuring to allow asynchronous io. It's sort of a pity that this isn't modular enough to allow multiple IO engines (synchronous system calls and
io_uringcome to mine) but that can come later.
Making it modular is easy enough, but that can be done independently of this buffer cache enabling patchset.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 5 lines · Dave Chinner
Hi folks,
Big rework. Aimed at unifying the user and kernel buffer caches so the first thing to do is port the kernel buffer cache code and hack it to pieces to use userspace AIO.
Just a quick addition - this seems to be smoke testing on fstests just fine at the moment. It looks like there is nothing catastrophically broken in the conversion (apart from discontiguous buffers w/ AIO).
The xfsprogs base is the current for-next tree (5.9.0-rc1, IIRC), and it applies on top of the config file patchset I posted earlier.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 5 lines · Dave Chinner
Hi folks,
Big rework. Aimed at unifying the user and kernel buffer caches so the first thing to do is port the kernel buffer cache code and hack it to pieces to use userspace AIO.
Hurrah!
Quoted 35 lines
Overview:
This patchset introduces the kernel buffer caching mechanism to userspace libxfs. THe core of this is per-AG indexing of the buffer cache along with global scope LRU based reclaiming. Like the kernel, it supports both cached and uncached buffers, and the implementation of these APIs are identical between userspace and kernel space.
To allow the buffer cache implementation to be shared between kernel and userspace, it needs to be split into two parts. The first is the domain specific "target" abstraction that implements all the kernel or userspace specific functions (e.g. memory allocation, the per-AG cache indexing, the IO engine, etc). The second part is the shared buffer cache and buffer IO API implementation.
TO use the kernel code, however, we need a bunch of infrastructure userspace doesn't currently provide. These are atomic variables, locking primitives, completions, memory barriers, LRU lists, memory reclaim triggers, and so on.
The simplest, most portable choice for atomic variables and memory barriers is to use the Userspace RCU library, which is provides many of the same semantics and interfaces we use within the kernel for atomic, memory barriers and RCU algorithms. Hence xfsprogs grows a hard dependency on liburcu6, but this is packaged on all major distros these days, so that's not a huge problem.
Locking APIs are fleshed out simply by using pthread locks. Spin locks are not a reliable solution in userspace, so all the kernel locking mechanisms wrap sleeping locks. We can even use a
pthread_mutex_tas a kernel semaphore replacement because, by default,pthread_mutex_unlock()does not check the owner matches the unlocking and hence, unlike kernel mutexes, can be used safely in non-owner release situations.
Quoted 8 lines
For reclaiming the buffer cache when memory usage goes too high, we make use of the kernel Pressure Stall Information (PSI) monitoring API via
/proc/pressure/memory. THis was introduced in kernel v5.2 and provides dynamic notification of the kernel memory allocation stalling to reclaim memory. When we get such a notification, we trigger the buffer cache shrinker to run to free cached buffers, just like the kernel does. Hence we no longer need to care about how much RAM we need to reserve for the buffer cache.
<nod> I have a few comments about that, but they're all in the PSI patch. :)
Quoted 18 lines
The patch series is structured in a way that:
- guts old tracing and debug interfaces
- adds support infrastructure for atomics, locking, etc.
- creates a new buftarg abstraction
- restructures buftarg implemenation and usage
- moves uncached IO into new buftarg abstraction
- reworks userspace specific buffer read/write interfaces
- implements new functions the buftarg abstraction eeds to provide
- adds the kernel buffer cache implementation
- switches the implementation and removes the old rd/wr and caching code
- converts the buftarg IO engine to use AIOThe AIO engine is not fully functional yet - it does not support discontiguous buffers correctly yet (needs an iocb per buffer map) - but otherwise it largely works just fine.
Quoted 6 lines
There are -lots- of loose ends that still need to be worked on, especially w.r.t.
xfs_repair. Stuff that comes to mind:-
xfs_buf_mark_dirty()just currently callsxfs_bwrite()and does synchronous IO. This needs to be converted to a buftarg internal delwri buffer list and a flushing mechanism needs to be added.
<nod>
Quoted 4 lines
- AIO engine does not support discontiguous buffers.
- work out best way to handle IOCBs for AIO - is embedding them in the
xfs_bufthe only way to do this efficiently?
The only other way I can think of is to embed MAX_AIO_EVENTS iocbs in the buftarg and (I guess) track which ones are free with a bitmap. That would cut down the iocb memory overhead to the IOs that we're actually running at a cost of more bookkeepping and potential starvation issues for the yuuuge buffer that takes a long time to collect all NR iocbs.
Quoted 4 lines
- rationalise the xfs_buf/xfs_buftarg split. Work out where to define the stuff that is different between kernel and userspace (e.g. the struct
xfs_buf) vs the stuff that remains the same (e.g. the XBF buffer flags)
Ow my brain.
Quoted 6 lines
- lots of code cleanup
-
xfs_repairdoes not cache between phases right now - it unconditionally purges the AG cache at the end of AG processing. This keeps memory usage way down, and for filesystems that have too much metadata to cache entirely in RAM, it's *much* faster
Why is it faster to blow out the cache? Is it the internal overhead of digging through a huge buftarg hash table to find the buffer that the caller wants and/or whacking enough of the other stale buffers to free up memory to avoid the buffer cache limits?
Quoted 8 lines
than
xfs_repairv5.6.0. On smaller filesystems, however, hitting RAM caches is much more desirable. Hence there is work to be done here determining how to manage the caches effectively.- async buffer readahead works in userspace now, but unless you have a very high IOP device (think in multiples of 100kiops) the
xfs_repairprefetch mechanism that trades off bandwidth for iops is still faster. More investigative work is needed here.
(No idea what this means.)
Quoted 8 lines
-
xfs_dbcould likely use readahead for btree traversals and other things.- More memory pressure trigger testing needs to be done to determine if the trigger settings are sufficient to prevent OOM kills on different hardware and filesystem configurations.
- Replace AIO engine with
io_uringengine?
Hm, is Al actually going to review io_uring as he's been threatening to do? I'd hold off until that happens, or one of us goes and tests it in anger to watch all the smoke leak out, etc...
Quoted 3 lines
- start working on splitting the kernel xfs_buf.[ch] code the same way as the userspace code and moving xfs_buf.[ch] to
fs/xfs/libxfsso that they can be updated in sync.
Aha, so yes that answers my question in ... whichever patch that was somewhere around #17.
--D
Quoted 133 lines
There's plenty of other stuff, too, but this like is a good start....
Cheers,
Dave.
Dave Chinner (27):
xfsprogs: remove unused buffer tracing code
xfsprogs: remove unusedIO_DEBUGfunctionality
libxfs: get rid ofb_bcountfromxfs_buf
libxfs: renamebuftarg->devto btdev
xfsprogs: get rid of ancient btree tracing fragments
xfsprogs: removexfs_buf_ttypedef
xfsprogs: introduce liburcu support
libxfs: addspinlock_twrapper
atomic: convert to uatomic
libxfs: add kernel-compatible completion API
libxfs: add wrappers for kernel semaphores
xfsprogs: convert use-once buffer reads to uncached IO
libxfs: introduce userspace buftarg infrastructure
xfs: renamelibxfs_buftarg_inittolibxfs_open_devices()
libxfs: introduce userspace buftarg infrastructure
libxfs: add a synchronous IO engine to the buftarg
xfsprogs: convertlibxfs_readbufrtolibxfs_buf_read_uncached
libxfs: convertlibxfs_bwriteto buftarg IO
libxfs: add cache infrastructure to buftarg
libxfs: add internal lru to btcache
libxfs: Add kernellist_lruwrapper
libxfs: introduce new buffer cache infrastructure
libxfs: use PSI information to detect memory pressure
libxfs: add a buftarg cache shrinker implementation
libxfs: switch buffer cache implementations
build: set platform_defs.h.in dependency correctly
libxfs: convert sync IO buftarg engine to AIOMakefile | 2 +- configure.ac | 3 + copy/Makefile | 3 +- copy/xfs_copy.c | 15 +- db/Makefile | 3 +- db/init.c | 4 +- db/io.c | 33 +- db/metadump.c | 2 +- db/sb.c | 2 +- debian/control | 2 +- growfs/Makefile | 3 +- include/Makefile | 6 +- include/atomic.h | 63 +- include/builddefs.in | 6 +- include/cache.h | 133 ---- include/completion.h | 61 ++ include/libxfs.h | 30 +- include/libxlog.h | 6 +- include/list_lru.h | 69 ++ include/platform_defs.h.in | 2 + include/sema.h | 35 + include/spinlock.h | 25 + include/xfs.h | 23 + include/xfs_btree_trace.h | 87 --- include/xfs_inode.h | 3 +- include/xfs_mount.h | 13 + include/xfs_trace.h | 24 + include/xfs_trans.h | 1 + libfrog/linux.c | 14 +- libfrog/workqueue.c | 3 + libxfs/Makefile | 7 +- libxfs/buftarg.c | 953 +++++++++++++++++++++++++ libxfs/cache.c | 724 ------------------- libxfs/init.c | 220 +++--- libxfs/libxfs_api_defs.h | 7 + libxfs/libxfs_io.h | 185 +---- libxfs/libxfs_priv.h | 74 +- libxfs/logitem.c | 10 +- libxfs/rdwr.c | 1234 +------------------------------- libxfs/trans.c | 21 +- libxfs/util.c | 8 +- libxfs/xfs_alloc.c | 16 +- libxfs/xfs_bmap.c | 6 +- libxfs/xfs_btree.c | 10 +- libxfs/xfs_buf.c | 1357 ++++++++++++++++++++++++++++++++++++ libxfs/xfs_buf.h | 242 +++++++ libxfs/xfs_buftarg.h | 176 +++++ libxfs/xfs_ialloc.c | 4 +- libxfs/xfs_rtbitmap.c | 22 +- libxlog/xfs_log_recover.c | 23 +- logprint/Makefile | 3 +- logprint/log_print_all.c | 2 +- logprint/logprint.c | 2 +- m4/Makefile | 1 + m4/package_urcu.m4 | 22 + mdrestore/Makefile | 3 +- mkfs/Makefile | 2 +- mkfs/proto.c | 11 +- mkfs/xfs_mkfs.c | 61 +- repair/Makefile | 2 +- repair/agheader.c | 2 +- repair/attr_repair.c | 14 +- repair/da_util.c | 2 +- repair/da_util.h | 2 +- repair/dino_chunks.c | 14 +- repair/dinode.c | 4 +- repair/incore.h | 2 +- repair/phase3.c | 7 +- repair/phase4.c | 5 +- repair/phase5.c | 2 +- repair/phase6.c | 4 +- repair/prefetch.c | 118 ++-- repair/progress.c | 16 +- repair/progress.h | 4 +- repair/rt.c | 4 +- repair/scan.c | 10 +- repair/xfs_repair.c | 192 +++-- scrub/Makefile | 3 +- scrub/progress.c | 2 + 79 files changed, 3644 insertions(+), 2847 deletions(-) delete mode 100644 include/cache.h create mode 100644 include/completion.h create mode 100644 include/list_lru.h create mode 100644 include/sema.h create mode 100644 include/spinlock.h delete mode 100644 include/xfs_btree_trace.h create mode 100644 libxfs/buftarg.c delete mode 100644 libxfs/cache.c create mode 100644 libxfs/xfs_buf.c create mode 100644 libxfs/xfs_buf.h create mode 100644 libxfs/xfs_buftarg.h create mode 100644 m4/package_urcu.m4Signature · 1 line
2.28.0
Quoted 8 lines · Darrick J. Wong
Quoted 4 lines · Dave Chinner
- AIO engine does not support discontiguous buffers.
- work out best way to handle IOCBs for AIO - is embedding them in the
xfs_bufthe only way to do this efficiently?The only other way I can think of is to embed
MAX_AIO_EVENTSiocbs in the buftarg and (I guess) track which ones are free with a bitmap.
I originally had an array embedded in the buftarg, and when I realised that I had to rtack exactly which ones were still in use I trashed it and moved the iocb to the struct xfs_buf so no tracking is necessary. All I need to do is change the buffer allocation code to allocate an iocb array when it allocates the map array so we have a direct b_maps -> b_iocbs relationship at all times.
Quoted 4 lines
That
would cut down the iocb memory overhead to the IOs that we're actually running at a cost of more bookkeepping and potential starvation issues for the yuuuge buffer that takes a long time to collect all NR iocbs.
We've already got a huge amount of per-buffer state, so adding iocbs to that isn't a huge deal...
Quoted 6 lines
Quoted 4 lines
- rationalise the xfs_buf/xfs_buftarg split. Work out where to define the stuff that is different between kernel and userspace (e.g. the struct
xfs_buf) vs the stuff that remains the same (e.g. the XBF buffer flags)Ow my brain.
Yeah. :(
Quoted 11 lines
Quoted 6 lines
- lots of code cleanup
-
xfs_repairdoes not cache between phases right now - it unconditionally purges the AG cache at the end of AG processing. This keeps memory usage way down, and for filesystems that have too much metadata to cache entirely in RAM, it's *much* fasterWhy is it faster to blow out the cache? Is it the internal overhead of digging through a huge buftarg hash table to find the buffer that the caller wants and/or whacking enough of the other stale buffers to free up memory to avoid the buffer cache limits?
Lots of reasons. The size of the hash table/chain length is largely irrelevant. The biggest issue seems to be memory allocation for the buffers - when we allocate the buffer, malloc does a mprotect() call on the new heap space for some reason and that takes the mmap_sem in write mode. Which serialises all the page faults being done while reading those buffers as they take the mmap_sem shared. Hence while we are allocating new heap space, there's massive numbers of context switches on the mmap_sem contention. COntext switch profile while running at ~150,000 context switches a second:
64.45% 64.45% xfs_repair [kernel.kallsyms] [k] schedule
|
|--41.02%--__memmove_sse2_unaligned_erms
| |
| --40.95%--asm_exc_page_fault
| exc_page_fault
| |
| --40.95%--down_read
| rwsem_down_read_slowpath
| schedule
| schedule
|
|--7.88%--sysmalloc
| |
| --7.87%--asm_exc_page_fault
| exc_page_fault
| |
| --7.86%--down_read
| rwsem_down_read_slowpath
| schedule
| schedule
....
|
--1.68%--__mprotect
|
--1.68%--entry_SYSCALL_64_after_hwframe
do_syscall_64
|
--1.68%--__x64_sys_mprotect
do_mprotect_pkey
|
--1.68%--down_write_killable
rwsem_down_write_slowpath
schedule
scheduleThe __memmove_sse2_unaligned_erms() function is from the memcpy() loop in repair/prefetch.c where it is copying metadata from the large IO buffer into the individual xfs_bufs (i.e. read TLB faults) and AFAICT the mprotect() syscalls are coming from malloc heap expansion. About 7% of the context switches are from pthread_mutex locks, and only 2.5% of the context switches are from the pread() IO that is being done.
IOWs, while memory footprint is growing/changing, repair performance is largely limited by mmap_sem concurrency issues.
So my reading of this is that by bounding the memory footprint of repair by freeing the buffers back to the heap regularly, we largely stay out of the heap grow mprotect path and avoid this mmap_sem contention. That allows the CPU intensive parts of prefetch and metadata verification to run more efficiently....
Quoted 10 lines
Quoted 8 lines
than
xfs_repairv5.6.0. On smaller filesystems, however, hitting RAM caches is much more desirable. Hence there is work to be done here determining how to manage the caches effectively.- async buffer readahead works in userspace now, but unless you have a very high IOP device (think in multiples of 100kiops) the
xfs_repairprefetch mechanism that trades off bandwidth for iops is still faster. More investigative work is needed here.(No idea what this means.)
The SSD I've been testing on peaks at about 70,000kiops for reads. If I drive prefetch by xfs_buf_readahead() only (got patches that do that) then, compared to 5.6.0, prefetch bandwidth drops to ~400MB/s but the device is iops bound and so prefetch is slower than when 5.6.0 uses 2MB IOs, does ~5-10kiops and consumes 900MB/s of bandwidth.
This patchset runs the existing prefetch code at 15-20kiops and 1.8-2.1GB/s using 2MB IOs - we still get better throughput on these SSDs by trading off iops for bandwidth.
The original patchset I wrote (but never published) had similar performance on the above hardware, but I also ran it on faster SSDs. On those, repair could push them to around 200-250kiops with prefetch by xfs_buf_readahead() and that was much faster than the 2-2.5GB/s that the existing prefetch could get before being limited by the pcie 3.0 4x bus the SSD is on. I'll have some numbers from this patchset on my faster hardware next week, but I don't expect them to be much different...
So, essentially, if you've got more small read IOPS capacity than you have "sparse metadata population optimised" large IO bandwidth (or metadata too sparse to trigger the large IO optimisations) then using xfs_buf_readahead() appears to more efficient than using the existing prefetch code.
That said, there's a lot of testing, observation and analysis needed to determine what will be the best general approach here. Signs are pointing towards "existing prefetch for rotational storage and low iops ssds" and miminal caching and on-demand prefetch for high end SSDs, but we'll see how that stands up...
Quoted 7 lines
Quoted 7 lines
-
xfs_dbcould likely use readahead for btree traversals and other things.- More memory pressure trigger testing needs to be done to determine if the trigger settings are sufficient to prevent OOM kills on different hardware and filesystem configurations.
Quoted 5 lines
Quoted 1 line
- Replace AIO engine with
io_uringengine?Hm, is Al actually going to review
io_uringas he's been threatening to do? I'd hold off until that happens, or one of us goes and tests it in anger to watch all the smoke leak out, etc...
Yeah, I'm in no hurry to do this. Just making sure people understand that it's a potential future direction.
Quoted 6 lines
Quoted 3 lines
- start working on splitting the kernel xfs_buf.[ch] code the same way as the userspace code and moving xfs_buf.[ch] to
fs/xfs/libxfsso that they can be updated in sync.Aha, so yes that answers my question in ... whichever patch that was somewhere around #17.
Right. I had to start somewhere, and given that userspace requirements largely define how the split needs to occur, I decided to start by making userspace work and then, once that is done, change the kernel to match the same structure that userspace needed....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]