xfs: freespace defrag for online shrink
Subsystems (3)
PATCH 4/4 xfs: freespace defrag for online shrink
Hi everyone,
This fourth patchriver has two faces -- one way of looking at it is that it is random odds and ends at the tail of my development tree. A second interpretation is that it is necessary pieces for defragmenting free space, which is a precursor for online shrink of XFS filesystems.
The kernel side isn't that exciting -- we export refcounting information for space extents, and add a new fallocate mode for mapping exact portions of free filesystem space into a file.
Userspace is where things get interesting! The free space defragmenter is an iterative algorithm that assigns free space to a dummy file, and then uses the GETFSMAP and GETFSREFCOUNTS information to target file space extents in order of decreasing share counts. Once an extent has been targeted, it uses reflinking to freeze the space, copies it elsewhere, and uses FIDEDUPERANGE to remap existing file data until the dummy file is the sole owner of the targetted space. If metadata are involved, the defrag utility invokes online repair to rebuild the metadata somewhere else.
Since last year, I've also merged in Dave's xfs_spaceman code to move inodes. I've not had much time to play with it otherwise.
When the defragmenter finishes, all the free space has been isolated to the dummy file, which can be unlinked and closed if defragmentation was the goal; or it could be passed to a shrinkfs operation.
--D
PATCH 1/5 xfs: improve post-close eofblocks gc behavior
Hi all,
Here's a few patches mostly from Dave to make XFS more aggressive about keeping post-eof speculative preallocations when closing files.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=reduce-eofblocks-gc-on-close
--- fs/xfs/xfs_bmap_util.c | 9 ++++++--- fs/xfs/xfs_file.c | 14 ++++++++++++-- fs/xfs/xfs_inode.c | 13 ++++++------- fs/xfs/xfs_inode.h | 2 +- 4 files changed, 25 insertions(+), 13 deletions(-)
PATCH 1/3 xfs: only free posteof blocks on first close
From: Darrick J. Wong <[email protected]>
Certain workloads fragment files on XFS very badly, such as a software package that creates a number of threads, each of which repeatedly run the sequence: open a file, perform a synchronous write, and close the file, which defeats the speculative preallocation mechanism. We work around this problem by only deleting posteof blocks the /first/ time a file is closed to preserve the behavior that unpacking a tarball lays out files one after the other with no gaps.
--- fs/xfs/xfs_inode.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
fs/xfs/xfs_inode.c +1 -3
@@ -1207,9 +1207,7 @@ xfs_release(if (error) goto out_unlock; -/* delalloc blocks after truncation means it really is dirty */ -if (ip->i_delayed_blks) -xfs_iflags_set (ip,XFS_IDIRTY_RELEASE ); +xfs_iflags_set (ip,XFS_IDIRTY_RELEASE ); } out_unlock:
PATCH 2/3 xfs: don't free EOF blocks on read close
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
When we have a workload that does open/read/close in parallel with other allocation, the file becomes rapidly fragmented. This is due to close() calling xfs_release() and removing the speculative preallocation beyond EOF.
The existing open/*/close heuristic in xfs_release() does not catch this as a sync writer does not leave delayed allocation blocks allocated on the inode for later writeback that can be detected in xfs_release() and hence XFS_IDIRTY_RELEASE never gets set.
In xfs_file_release(), we know more about the released file context, and so we need to communicate some of the details to xfs_release() so it can do the right thing here and skip EOF block truncation. This defers the EOF block cleanup for synchronous write contexts to the background EOF block cleaner which will clean up within a few minutes.
Before:
Test 1: sync write fragmentation counts
/mnt/scratch/file.0: 919/mnt/scratch/file.1: 916/mnt/scratch/file.2: 919/mnt/scratch/file.3: 920/mnt/scratch/file.4: 920/mnt/scratch/file.5: 921/mnt/scratch/file.6: 916/mnt/scratch/file.7: 918
After:
Test 1: sync write fragmentation counts
/mnt/scratch/file.0: 24/mnt/scratch/file.1: 24/mnt/scratch/file.2: 11/mnt/scratch/file.3: 24/mnt/scratch/file.4: 3/mnt/scratch/file.5: 24/mnt/scratch/file.6: 24/mnt/scratch/file.7: 23
--- fs/xfs/xfs_file.c | 14 ++++++++++++-- fs/xfs/xfs_inode.c | 9 +++++---- fs/xfs/xfs_inode.h | 2 +- 3 files changed, 18 insertions(+), 7 deletions(-)
fs/xfs/xfs_file.c +12 -2
@@ -1367,12 +1367,22 @@ xfs_dir_open(return error; } +/* +* When we release the file, we don't want it to trim EOF blocks if it is a +* readonly context. This avoids open/read/close workloads from removing +* EOF blocks that other writers depend upon to reduce fragmentation. +*/ STATIC intxfs_file_release (struct inode *inode, -struct file *filp) +struct file *file) { -return xfs_release (XFS_I (inode)); +bool free_eof_blocks = true; + +if ((file->f_mode & (FMODE_WRITE |FMODE_READ )) ==FMODE_READ ) + free_eof_blocks = false; + +return xfs_release (XFS_I (inode), free_eof_blocks); }STATIC int
fs/xfs/xfs_inode.c +5 -4
@@ -1137,10 +1137,11 @@ xfs_itruncate_extents_flags(int xfs_release ( -xfs_inode_t *ip) +struct xfs_inode *ip, +bool want_free_eofblocks) { -xfs_mount_t *mp = ip->i_mount; -int error =0 ; +struct xfs_mount *mp = ip->i_mount; +int error =0 ;if (!S_ISREG (VFS_I (ip)->i_mode) || (VFS_I (ip)->i_mode ==0 ))return 0 ;
@@ -1182,7 +1183,7 @@ xfs_release(* another chance to drop them once the last reference to the inode is * dropped, so we'll never leak blocks permanently. */ -if (!xfs_ilock_nowait (ip,XFS_IOLOCK_EXCL )) +if (!want_free_eofblocks || !xfs_ilock_nowait (ip,XFS_IOLOCK_EXCL ))return 0 ;if (xfs_can_free_eofblocks (ip, false)) {
fs/xfs/xfs_inode.h +1 -1
@@ -525,7 +525,7 @@ enum layout_break_reason {#define XFS_INHERIT_GID (pip) \ (xfs_has_grpid((pip)->i_mount) || (VFS_I(pip)->i_mode & S_ISGID)) -int xfs_release (struct xfs_inode *ip); +int xfs_release (struct xfs_inode *ip,bool want_free_eofblocks);int xfs_inactive (struct xfs_inode *ip);int xfs_lookup (struct xfs_inode *dp,const struct xfs_name *name,struct xfs_inode **ipp,struct xfs_name *ci_name);
PATCH 3/3 xfs: Don't free EOF blocks on close when extent size hints are set
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
When we have a workload that does open/write/close on files with extent size hints set in parallel with other allocation, the file becomes rapidly fragmented. This is due to close() calling xfs_release() and removing the preallocated extent beyond EOF. This occurs for both buffered and direct writes that append to files with extent size hints.
The existing open/write/close hueristic in xfs_release() does not catch this as writes to files using extent size hints do not use delayed allocation and hence do not leave delayed allocation blocks allocated on the inode that can be detected in xfs_release(). Hence XFS_IDIRTY_RELEASE never gets set.
In xfs_file_release(), we can tell whether the inode has extent size hints set and skip EOF block truncation. We add this check to xfs_can_free_eofblocks() so that we treat the post-EOF preallocated extent like intentional preallocation and so are persistent unless directly removed by userspace.
Before:
Test 2: Extent size hint fragmentation counts
/mnt/scratch/file.0: 1002/mnt/scratch/file.1: 1002/mnt/scratch/file.2: 1002/mnt/scratch/file.3: 1002/mnt/scratch/file.4: 1002/mnt/scratch/file.5: 1002/mnt/scratch/file.6: 1002/mnt/scratch/file.7: 1002
After:
Test 2: Extent size hint fragmentation counts
/mnt/scratch/file.0: 4/mnt/scratch/file.1: 4/mnt/scratch/file.2: 4/mnt/scratch/file.3: 4/mnt/scratch/file.4: 4/mnt/scratch/file.5: 4/mnt/scratch/file.6: 4/mnt/scratch/file.7: 4
--- fs/xfs/xfs_bmap_util.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/xfs_bmap_util.c +6 -3
@@ -710,12 +710,15 @@ xfs_can_free_eofblocks(return false;/* -* Do not free real preallocated or append-only files unless the file -* has delalloc blocks and we are forced to remove them. +* Do not free extent size hints, real preallocated or append-only files +* unless the file has delalloc blocks and we are forced to remove +* them. */ -if (ip->i_diflags & (XFS_DIFLAG_PREALLOC |XFS_DIFLAG_APPEND )) +if (xfs_get_extsz_hint (ip) || + (ip->i_diflags & (XFS_DIFLAG_PREALLOC |XFS_DIFLAG_APPEND ))) {if (!force || ip->i_delayed_blks ==0 )return false; + }/* * Do not try to free post-EOF blocks if EOF is beyond the end of the
RFC PATCH 2/5 xfs: noalloc allocation groups
In reply to Darrick J. WongHi all,
This series creates a new NOALLOC flag for allocation groups that causes the block and inode allocators to look elsewhere when trying to allocate resources. This is either the first part of a patchset to implement online shrinking (set noalloc on the last AGs, run fsr to move the files and directories) or freeze-free rmapbt rebuilding (set noalloc to prevent creation of new mappings, then hook deletion of old mappings). This is still totally a research project.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=noalloc-ags
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=noalloc-ags
--- fs/xfs/Kconfig | 13 +++++ fs/xfs/libxfs/xfs_ag.c | 114 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_ag.h | 8 +++ fs/xfs/libxfs/xfs_ag_resv.c | 27 +++++++++- fs/xfs/libxfs/xfs_defer.c | 14 +++++ fs/xfs/libxfs/xfs_fs.h | 5 ++ fs/xfs/libxfs/xfs_ialloc.c | 3 + fs/xfs/scrub/btree.c | 88 +++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.c | 107 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 1 fs/xfs/scrub/dabtree.c | 24 +++++++++ fs/xfs/scrub/fscounters.c | 3 + fs/xfs/scrub/inode.c | 4 ++ fs/xfs/scrub/scrub.c | 40 +++++++++++++++ fs/xfs/scrub/trace.c | 22 ++++++++ fs/xfs/scrub/trace.h | 2 + fs/xfs/xfs_fsops.c | 10 +++- fs/xfs/xfs_globals.c | 5 ++ fs/xfs/xfs_ioctl.c | 4 +- fs/xfs/xfs_super.c | 1 fs/xfs/xfs_sysctl.h | 1 fs/xfs/xfs_sysfs.c | 32 ++++++++++++ fs/xfs/xfs_trace.h | 65 +++++++++++++++++++++++++ fs/xfs/xfs_trans.c | 3 + fs/xfs/xfs_trans.h | 7 +++ 25 files changed, 597 insertions(+), 6 deletions(-)
PATCH 1/5 xfs: track deferred ops statistics
From: Darrick J. Wong <[email protected]>
Track some basic statistics on how hard we're pushing the defer ops.
--- fs/xfs/libxfs/xfs_defer.c | 14 ++++++++++++++ fs/xfs/xfs_trace.h | 19 +++++++++++++++++++ fs/xfs/xfs_trans.c | 3 +++ fs/xfs/xfs_trans.h | 7 +++++++ 4 files changed, 43 insertions(+)
fs/xfs/libxfs/xfs_defer.c +14 -0
@@ -617,6 +617,8 @@ xfs_defer_finish_one(/* Done with the dfp, free it. */ list_del (&dfp->dfp_list);kmem_cache_free (xfs_defer_pending_cache, dfp); + tp->t_dfops_nr--; + tp->t_dfops_finished++; out:if (ops->finish_cleanup) ops->finish_cleanup (tp, state, error);
@@ -679,6 +681,9 @@ xfs_defer_finish_noroll(list_splice_init (&(*tp)->t_dfops, &dop_pending); + (*tp)->t_dfops_nr_max =max ((*tp)->t_dfops_nr, + (*tp)->t_dfops_nr_max); +if (has_intents <0 ) { error = has_intents; goto out_shutdown;
@@ -720,6 +725,7 @@ xfs_defer_finish_noroll(xfs_force_shutdown ((*tp)->t_mountp,SHUTDOWN_CORRUPT_INCORE );trace_xfs_defer_finish_error (*tp, error);xfs_defer_cancel_list ((*tp)->t_mountp, &dop_pending); + (*tp)->t_dfops_nr =0 ;xfs_defer_cancel (*tp);return error; }
@@ -767,6 +773,7 @@ xfs_defer_cancel(trace_xfs_defer_cancel (tp, _RET_IP_);xfs_defer_trans_abort (tp, &tp->t_dfops);xfs_defer_cancel_list (mp, &tp->t_dfops); + tp->t_dfops_nr =0 ; }/*
@@ -830,6 +837,7 @@ xfs_defer_alloc( dfp->dfp_ops = ops;INIT_LIST_HEAD (&dfp->dfp_work);list_add_tail (&dfp->dfp_list, &tp->t_dfops); + tp->t_dfops_nr++;return dfp; }
@@ -942,6 +950,12 @@ xfs_defer_move( struct xfs_trans *stp) {list_splice_init (&stp->t_dfops, &dtp->t_dfops); + dtp->t_dfops_nr += stp->t_dfops_nr; + dtp->t_dfops_nr_max = stp->t_dfops_nr_max; + dtp->t_dfops_finished = stp->t_dfops_finished; + stp->t_dfops_nr =0 ; + stp->t_dfops_nr_max =0 ; + stp->t_dfops_finished =0 ;/* * Low free space mode was historically controlled by a dfops field.
fs/xfs/xfs_trace.h +19 -0
@@ -2711,6 +2711,25 @@ TRACE_EVENT(xfs_btree_free_block,/* deferred ops */ struct xfs_defer_pending ; +TRACE_EVENT (xfs_defer_stats, + TP_PROTO(struct xfs_trans *tp), +TP_ARGS (tp), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(unsigned int, max) + __field(unsigned int, finished) + ), +TP_fast_assign ( + __entry->dev = tp->t_mountp->m_super->s_dev; + __entry->max = tp->t_dfops_nr_max; + __entry->finished = tp->t_dfops_finished; + ), +TP_printk ("dev %d:%d max %u finished %u" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->max, + __entry->finished) +) + DECLARE_EVENT_CLASS(xfs_defer_class, TP_PROTO(struct xfs_trans *tp, unsigned long caller_ip), TP_ARGS(tp, caller_ip),
fs/xfs/xfs_trans.c +3 -0
@@ -70,6 +70,9 @@ xfs_trans_free(xfs_extent_busy_sort (&tp->t_busy);xfs_extent_busy_clear (tp->t_mountp, &tp->t_busy, false); +if (tp->t_dfops_finished >0 ) +trace_xfs_defer_stats (tp); +trace_xfs_trans_free (tp, _RET_IP_);xfs_trans_clear_context (tp);if (!(tp->t_flags &XFS_TRANS_NO_WRITECOUNT ))
fs/xfs/xfs_trans.h +7 -0
@@ -151,6 +151,13 @@ typedef struct xfs_trans {struct list_head t_busy;/* list of busy extents */ struct list_head t_dfops;/* deferred operations */ unsigned long t_pflags;/* saved process flags state */ + +/* Count of deferred ops attached to transaction. */ +unsigned int t_dfops_nr; +/* Maximum t_dfops_nr seen in a loop. */ +unsigned int t_dfops_nr_max; +/* Number of dfops finished. */ +unsigned int t_dfops_finished; } xfs_trans_t;/*
PATCH 2/5 xfs: whine to dmesg when we encounter errors
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Forward everything scrub whines about to dmesg.
--- fs/xfs/Kconfig | 13 ++++++ fs/xfs/scrub/btree.c | 88 +++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.c | 107 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 1 fs/xfs/scrub/dabtree.c | 24 +++++++++++ fs/xfs/scrub/inode.c | 4 ++ fs/xfs/scrub/scrub.c | 40 ++++++++++++++++++ fs/xfs/scrub/trace.c | 22 ++++++++++ fs/xfs/scrub/trace.h | 2 + fs/xfs/xfs_globals.c | 5 ++ fs/xfs/xfs_sysctl.h | 1 fs/xfs/xfs_sysfs.c | 32 ++++++++++++++ 12 files changed, 339 insertions(+)
fs/xfs/Kconfig +13 -0
@@ -172,6 +172,19 @@ config XFS_ONLINE_SCRUB_STATS If unsure, say N. +config XFS_ONLINE_SCRUB_WHINE + bool "XFS online metadata verbose logging by default" + default y + depends on XFS_ONLINE_SCRUB + help + If you say Y here, the kernel will by default log the outcomes of all + scrub and repair operations, as well as any corruptions found. This + may slow down scrub due to printk logging overhead timers. + + This value can be changed by editing /sys/fs/xfs/debug/scrub_whine + + If unsure, say N. + choice prompt "XFS hook implementation" depends on XFS_FS && XFS_LIVE_HOOKS && XFS_ONLINE_SCRUB
fs/xfs/scrub/btree.c +88 -0
120 lines not on this page · raw message
fs/xfs/scrub/common.c +107 -0
206 lines not on this page · raw message
fs/xfs/scrub/common.h +1 -0
@@ -188,6 +188,7 @@ bool xchk_ilock_nowait(struct xfs_scrub *sc, unsigned int ilock_flags);void xchk_iunlock (struct xfs_scrub *sc,unsigned int ilock_flags);void xchk_buffer_recheck (struct xfs_scrub *sc,struct xfs_buf *bp); +void xchk_whine (const struct xfs_mount *mp,const char *fmt, ...);/* * Grab the inode at @inum. The caller must have created a scrub transaction
fs/xfs/scrub/dabtree.c +24 -0
@@ -47,9 +47,26 @@ xchk_da_process_error(case -EFSCORRUPTED :/* Note the badness but don't abort. */ sc->sm->sm_flags |=XFS_SCRUB_OFLAG_CORRUPT ; +xchk_whine (sc->mp,"ino 0x%llx fork %d type %s dablk 0x%llx error %d ret_ip %pS" , + sc->ip->i_ino, + ds->dargs.whichfork, +xchk_type_string (sc->sm->sm_type), +xfs_dir2_da_to_db (ds->dargs.geo, + ds->state->path.blk[level].blkno), + *error, + __return_address); *error =0 ; fallthrough;default : +if (*error) +xchk_whine (sc->mp,"ino 0x%llx fork %d type %s dablk 0x%llx error %d ret_ip %pS" , + sc->ip->i_ino, + ds->dargs.whichfork, +xchk_type_string (sc->sm->sm_type), +xfs_dir2_da_to_db (ds->dargs.geo, + ds->state->path.blk[level].blkno), + *error, + __return_address); trace_xchk_file_op_error(sc, ds->dargs.whichfork, xfs_dir2_da_to_db(ds->dargs.geo, ds->state->path.blk[level].blkno),
@@ -72,6 +89,13 @@ xchk_da_set_corrupt( sc->sm->sm_flags |=XFS_SCRUB_OFLAG_CORRUPT ; +xchk_whine (sc->mp,"ino 0x%llx fork %d type %s dablk 0x%llx ret_ip %pS" , + sc->ip->i_ino, + ds->dargs.whichfork, +xchk_type_string (sc->sm->sm_type), +xfs_dir2_da_to_db (ds->dargs.geo, + ds->state->path.blk[level].blkno), + __return_address); trace_xchk_fblock_error(sc, ds->dargs.whichfork, xfs_dir2_da_to_db(ds->dargs.geo, ds->state->path.blk[level].blkno),
fs/xfs/scrub/inode.c +4 -0
@@ -217,6 +217,10 @@ xchk_setup_inode( out_cancel:xchk_trans_cancel (sc); out_error: +xchk_whine (mp,"type %s agno 0x%x agbno 0x%x error %d ret_ip %pS" , +xchk_type_string (sc->sm->sm_type), agno, +XFS_INO_TO_AGBNO (mp, sc->sm->sm_ino), error, + __return_address);trace_xchk_op_error (sc, agno,XFS_INO_TO_AGBNO (mp, sc->sm->sm_ino), error, __return_address);return error;
fs/xfs/scrub/scrub.c +40 -0
@@ -632,6 +632,45 @@ xchk_scrub_create_subord(return sub; } +static inline void +repair_outcomes (struct xfs_scrub *sc,int error) +{ +struct xfs_scrub_metadata *sm = sc->sm; +const char *wut =NULL ; + +if (!xfs_globals.scrub_whine) +return ; + +if (sc->flags &XREP_ALREADY_FIXED ) { + wut ="*** REPAIR SUCCESS" ; + error =0 ; + }else if (error == -EBUSY ) { + wut ="??? FILESYSTEM BUSY" ; + }else if (error == -EAGAIN ) { + wut ="??? REPAIR DEFERRED" ; + }else if (error == -ECANCELED ) { + wut ="??? REPAIR CANCELLED" ; + }else if (error == -EINTR ) { + wut ="??? REPAIR INTERRUPTED" ; + }else if (error != -EOPNOTSUPP && error != -ENOENT ) { + wut ="!!! REPAIR FAILED" ; +xfs_info (sc->mp, +"%s ino 0x%llx type %s agno 0x%x inum 0x%llx gen 0x%x flags 0x%x error %d" , + wut,XFS_I (file_inode (sc->file))->i_ino, +xchk_type_string (sm->sm_type), sm->sm_agno, + sm->sm_ino, sm->sm_gen, sm->sm_flags, error); +return ; + }else { +return ; + } + +xfs_info_ratelimited (sc->mp, +"%s ino 0x%llx type %s agno 0x%x inum 0x%llx gen 0x%x flags 0x%x error %d" , + wut,XFS_I (file_inode (sc->file))->i_ino, +xchk_type_string (sm->sm_type), sm->sm_agno, sm->sm_ino, + sm->sm_gen, sm->sm_flags, error); +} +/* Dispatch metadata scrubbing. */ int xfs_scrub_metadata(
@@ -729,6 +768,7 @@ xfs_scrub_metadata(* already tried to fix it, then attempt a repair. */ error =xrep_attempt (sc, &run); +repair_outcomes (sc, error);if (error == -EAGAIN ) {/* * Either the repair function succeeded or it couldn't
fs/xfs/scrub/trace.c +22 -0
@@ -69,3 +69,25 @@ xfbtree_ino(*/ #define CREATE_TRACE_POINTS #include "scrub/trace.h" + +/* xchk_whine stuff */ +struct xchk_tstr { +unsigned int type; +const char *tag; +}; + +static const struct xchk_tstr xchk_tstr_tags[] = {XFS_SCRUB_TYPE_STRINGS }; + +const char * +xchk_type_string ( +unsigned int type) +{ +unsigned int i; + +for (i =0 ; i <ARRAY_SIZE (xchk_tstr_tags); i++) { +if (xchk_tstr_tags[i].type == type) +return xchk_tstr_tags[i].tag; + } + +return "???" ; +}
fs/xfs/scrub/trace.h +2 -0
@@ -127,6 +127,8 @@ TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RTREFCBT); { XFS_SCRUB_TYPE_RTRMAPBT, "rtrmapbt" }, \ { XFS_SCRUB_TYPE_RTREFCBT, "rtrefcountbt" } +const char *xchk_type_string (unsigned int type); +#define XFS_SCRUB_FLAG_STRINGS \ { XFS_SCRUB_IFLAG_REPAIR, "repair" }, \ { XFS_SCRUB_OFLAG_CORRUPT, "corrupt" }, \
fs/xfs/xfs_globals.c +5 -0
@@ -44,6 +44,11 @@ struct xfs_globals xfs_globals = { .pwork_threads = -1, /* automatic thread detection */ .larp = false, /* log attribute replay */#endif +#ifdef CONFIG_XFS_ONLINE_SCRUB_WHINE + .scrub_whine = true, +#else + .scrub_whine = false, +#endif /* * Leave this many record slots empty when bulk loading btrees. By
fs/xfs/xfs_sysctl.h +1 -0
@@ -91,6 +91,7 @@ struct xfs_globals {int mount_delay;/* mount setup delay (secs) */ bool bug_on_assert;/* BUG() the kernel on assert failure */ bool always_cow;/* use COW fork for all overwrites */ +bool scrub_whine;/* noisier output from scrub */ };extern struct xfs_globals xfs_globals;
fs/xfs/xfs_sysfs.c +32 -0
@@ -262,6 +262,37 @@ larp_show(XFS_SYSFS_ATTR_RW (larp);#endif /* DEBUG */ +/* Logging of the outcomes of everything that scrub does */ +STATIC ssize_t +scrub_whine_store ( +struct kobject *kobject, +const char *buf, +size_t count) +{ +int ret; +int val; + + ret =kstrtoint (buf,0 , &val); +if (ret) +return ret; + +if (val <-1 || val >num_possible_cpus ()) +return -EINVAL ; + + xfs_globals.scrub_whine = val; + +return count; +} + +STATIC ssize_t +scrub_whine_show ( +struct kobject *kobject, +char *buf) +{ +return sysfs_emit (buf,"%d\n" , xfs_globals.scrub_whine); +} +XFS_SYSFS_ATTR_RW (scrub_whine); + STATIC ssize_t bload_leaf_slack_store( struct kobject *kobject,
@@ -323,6 +354,7 @@ static struct attribute *xfs_dbg_attrs[] = {ATTR_LIST (pwork_threads),ATTR_LIST (larp), #endif +ATTR_LIST (scrub_whine),ATTR_LIST (bload_leaf_slack),ATTR_LIST (bload_node_slack), NULL,
PATCH 3/5 xfs: create a noalloc mode for allocation groups
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new noalloc state for the per-AG structure that will disable block allocation in this AG. We accomplish this by subtracting from fdblocks all the free blocks in this AG, hiding those blocks from the allocator, and preventing freed blocks from updating fdblocks until we're ready to lift noalloc mode.
Note that we reduce the free block count of the filesystem so that we can prevent transactions from entering the allocator looking for "free" space that we've turned off incore.
--- fs/xfs/libxfs/xfs_ag.c | 60 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_ag.h | 8 ++++++ fs/xfs/libxfs/xfs_ag_resv.c | 27 +++++++++++++++++-- fs/xfs/scrub/fscounters.c | 3 +- fs/xfs/xfs_fsops.c | 10 ++++++- fs/xfs/xfs_super.c | 1 + fs/xfs/xfs_trace.h | 46 +++++++++++++++++++++++++++++++++ 7 files changed, 150 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_ag.c +60 -0
@@ -1119,3 +1119,63 @@ xfs_ag_get_geometry(xfs_buf_relse (agi_bp);return error; } + +/* How many blocks does this AG contribute to fdblocks? */ +xfs_extlen_t +xfs_ag_fdblocks ( +struct xfs_perag *pag) +{ +xfs_extlen_t ret; + +ASSERT (xfs_perag_initialised_agf (pag)); + + ret = pag->pagf_freeblks + pag->pagf_flcount + pag->pagf_btreeblks; + ret -= pag->pag_meta_resv.ar_reserved; + ret -= pag->pag_rmapbt_resv.ar_orig_reserved; +return ret; +} + +/* +* Hide all the free space in this AG. Caller must hold both the AGI and the +* AGF buffers or have otherwise prevented concurrent access. +*/ +int +xfs_ag_set_noalloc ( +struct xfs_perag *pag) +{ +struct xfs_mount *mp = pag->pag_mount; +int error; + +ASSERT (xfs_perag_initialised_agf (pag)); +ASSERT (xfs_perag_initialised_agi (pag)); + +if (xfs_perag_prohibits_alloc (pag)) +return 0 ; + + error =xfs_mod_fdblocks (mp, -(int64_t )xfs_ag_fdblocks (pag), false); +if (error) +return error; + +trace_xfs_ag_set_noalloc (pag); +set_bit (XFS_AGSTATE_NOALLOC , &pag->pag_opstate); +return 0 ; +} + +/* +* Unhide all the free space in this AG. Caller must hold both the AGI and +* the AGF buffers or have otherwise prevented concurrent access. +*/ +void +xfs_ag_clear_noalloc ( +struct xfs_perag *pag) +{ +struct xfs_mount *mp = pag->pag_mount; + +if (!xfs_perag_prohibits_alloc (pag)) +return ; + +xfs_mod_fdblocks (mp,xfs_ag_fdblocks (pag), false); + +trace_xfs_ag_clear_noalloc (pag); +clear_bit (XFS_AGSTATE_NOALLOC , &pag->pag_opstate); +}
fs/xfs/libxfs/xfs_ag.h +8 -0
@@ -131,6 +131,7 @@ struct xfs_perag {#define XFS_AGSTATE_PREFERS_METADATA 2#define XFS_AGSTATE_ALLOWS_INODES 3#define XFS_AGSTATE_AGFL_NEEDS_RESET 4 +#define XFS_AGSTATE_NOALLOC 5#define __XFS_AG_OPSTATE (name,NAME ) \ static inline bool xfs_perag_ ## name (struct xfs_perag *pag) \
@@ -143,6 +144,7 @@ __XFS_AG_OPSTATE(initialised_agi, AGI_INIT) __XFS_AG_OPSTATE(prefers_metadata, PREFERS_METADATA) __XFS_AG_OPSTATE(allows_inodes, ALLOWS_INODES) __XFS_AG_OPSTATE(agfl_needs_reset, AGFL_NEEDS_RESET) +__XFS_AG_OPSTATE(prohibits_alloc, NOALLOC)int xfs_initialize_perag (struct xfs_mount *mp,xfs_agnumber_t agcount,xfs_rfsblock_t dcount,xfs_agnumber_t *maxagi);
@@ -156,12 +158,18 @@ struct xfs_perag *xfs_perag_get_tag(struct xfs_mount *mp, xfs_agnumber_t agno,struct xfs_perag *xfs_perag_hold (struct xfs_perag *pag);void xfs_perag_put (struct xfs_perag *pag); +/* Active AG references */ struct xfs_perag *xfs_perag_grab (struct xfs_mount *,xfs_agnumber_t );struct xfs_perag *xfs_perag_grab_tag (struct xfs_mount *,xfs_agnumber_t ,int tag);void xfs_perag_rele (struct xfs_perag *pag); +/* Enable or disable allocation from an AG */ +xfs_extlen_t xfs_ag_fdblocks (struct xfs_perag *pag); +int xfs_ag_set_noalloc (struct xfs_perag *pag); +void xfs_ag_clear_noalloc (struct xfs_perag *pag); +/* * Per-ag geometry infomation and validation */
fs/xfs/libxfs/xfs_ag_resv.c +24 -3
@@ -73,6 +73,13 @@ xfs_ag_resv_critical(xfs_extlen_t avail;xfs_extlen_t orig; +/* +* Pretend we're critically low on reservations in this AG to scare +* everyone else away. +*/ +if (xfs_perag_prohibits_alloc (pag)) +return true; +switch (type) {case XFS_AG_RESV_METADATA : avail = pag->pagf_freeblks - pag->pag_rmapbt_resv.ar_reserved;
@@ -115,7 +122,12 @@ xfs_ag_resv_needed(break ;case XFS_AG_RESV_IMETA :case XFS_AG_RESV_NONE : -/* empty */ +/* +* In noalloc mode, we pretend that all the free blocks in this +* AG have been allocated. Make this AG look full. +*/ +if (xfs_perag_prohibits_alloc (pag)) + len +=xfs_ag_fdblocks (pag);break ;default :ASSERT (0 );
@@ -344,6 +356,8 @@ xfs_ag_resv_alloc_extent(xfs_extlen_t len;uint field; +ASSERT (type !=XFS_AG_RESV_NONE || !xfs_perag_prohibits_alloc (pag)); +trace_xfs_ag_resv_alloc_extent (pag, type, args->len);switch (type) {
@@ -401,7 +415,14 @@ xfs_ag_resv_free_extent(ASSERT (0 ); fallthrough;case XFS_AG_RESV_NONE : -xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , (int64_t )len); +/* +* Normally we put freed blocks back into fdblocks. In noalloc +* mode, however, we pretend that there are no fdblocks in the +* AG, so don't put them back. +*/ +if (!xfs_perag_prohibits_alloc (pag)) +xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , + (int64_t )len); fallthrough;case XFS_AG_RESV_IGNORE :return ;
@@ -414,6 +435,6 @@ xfs_ag_resv_free_extent(/* Freeing into the reserved pool only requires on-disk update... */ xfs_trans_mod_sb (tp,XFS_TRANS_SB_RES_FDBLOCKS , len);/* ...but freeing beyond that requires in-core and on-disk update. */ -if (len > leftover) +if (len > leftover && !xfs_perag_prohibits_alloc (pag))xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , len - leftover); }
fs/xfs/scrub/fscounters.c +2 -1
@@ -338,7 +338,8 @@ xchk_fscount_aggregate_agcounts(*/ fsc->fdblocks -= pag->pag_meta_resv.ar_reserved; fsc->fdblocks -= pag->pag_rmapbt_resv.ar_orig_reserved; - +if (xfs_perag_prohibits_alloc (pag)) + fsc->fdblocks -=xfs_ag_fdblocks (pag); }if (pag)xfs_perag_rele (pag);
fs/xfs/xfs_fsops.c +9 -1
@@ -638,6 +638,14 @@ xfs_fs_unreserve_ag_blocks(if (xfs_has_realtime (mp))xfs_rt_resv_free (mp); -for_each_perag (mp, agno, pag) +for_each_perag (mp, agno, pag) { +/* +* Bring the AG back online because our AG hiding only exists +* in-core and we need the superblock to be written out with +* the super fdblocks reflecting the AGF freeblks. Do this +* before adding the per-AG reservations back to fdblocks. +*/ +xfs_ag_clear_noalloc (pag);xfs_ag_resv_free (pag); + } }
fs/xfs/xfs_super.c +1 -0
@@ -324,6 +324,7 @@ xfs_set_inode_alloc( pag =xfs_perag_get (mp, index);if (xfs_set_inode_alloc_perag (pag, ino, max_metadata)) maxagi++; +clear_bit (XFS_AGSTATE_NOALLOC , &pag->pag_opstate);xfs_perag_put (pag); }
fs/xfs/xfs_trace.h +46 -0
52 lines not on this page · raw message
PATCH 4/5 xfs: enable userspace to hide an AG from allocation
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add an administrative interface so that userspace can hide an allocation group from block allocation.
--- fs/xfs/libxfs/xfs_ag.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_fs.h | 5 ++++ fs/xfs/xfs_ioctl.c | 4 +++- 3 files changed, 62 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_ag.c +54 -0
@@ -1075,6 +1075,54 @@ xfs_ag_extend_space(return 0 ; } +/* Compute the AG geometry flags. */ +static inline uint32_t +xfs_ag_calc_geoflags ( +struct xfs_perag *pag) +{ +uint32_t ret =0 ; + +if (xfs_perag_prohibits_alloc (pag)) + ret |=XFS_AG_FLAG_NOALLOC ; + +return ret; +} + +/* +* Compare the current AG geometry flags against the flags in the AG geometry +* structure and update the AG state to reflect any changes, then update the +* struct to reflect the current status. +*/ +static inline int +xfs_ag_update_geoflags ( +struct xfs_perag *pag, +struct xfs_ag_geometry *ageo, +uint32_t new_flags) +{ +uint32_t old_flags =xfs_ag_calc_geoflags (pag); +int error; + +if (!(new_flags &XFS_AG_FLAG_UPDATE )) { + ageo->ag_flags = old_flags; +return 0 ; + } + +if ((old_flags &XFS_AG_FLAG_NOALLOC ) && + !(new_flags &XFS_AG_FLAG_NOALLOC )) { +xfs_ag_clear_noalloc (pag); + } + +if (!(old_flags &XFS_AG_FLAG_NOALLOC ) && + (new_flags &XFS_AG_FLAG_NOALLOC )) { + error =xfs_ag_set_noalloc (pag); +if (error) +return error; + } + + ageo->ag_flags =xfs_ag_calc_geoflags (pag); +return 0 ; +} +/* Retrieve AG geometry. */ int xfs_ag_get_geometry(
@@ -1086,6 +1134,7 @@ xfs_ag_get_geometry(struct xfs_agi *agi;struct xfs_agf *agf;unsigned int freeblks; +uint32_t inflags = ageo->ag_flags;int error;/* Lock the AG headers. */
@@ -1096,6 +1145,10 @@ xfs_ag_get_geometry(if (error) goto out_agi; + error =xfs_ag_update_geoflags (pag, ageo, inflags); +if (error) + goto out; +/* Fill out form. */ memset (ageo,0 ,sizeof (*ageo)); ageo->ag_number = pag->pag_agno;
@@ -1113,6 +1166,7 @@ xfs_ag_get_geometry( ageo->ag_freeblks = freeblks;xfs_ag_geom_health (pag, ageo); +out:/* Release resources. */ xfs_buf_relse (agf_bp); out_agi:
fs/xfs/libxfs/xfs_fs.h +5 -0
@@ -305,6 +305,11 @@ struct xfs_ag_geometry {#define XFS_AG_GEOM_SICK_REFCNTBT (1 << 9)/* reference counts */ #define XFS_AG_GEOM_SICK_INODES (1 << 10)/* bad inodes were seen */ +#define XFS_AG_FLAG_UPDATE (1 << 0)/* update flags */ +#define XFS_AG_FLAG_NOALLOC (1 << 1)/* do not allocate from this AG */ +#define XFS_AG_FLAG_ALL (XFS_AG_FLAG_UPDATE | \ + XFS_AG_FLAG_NOALLOC) +/* * Structures for XFS_IOC_FSGROWFSDATA, XFS_IOC_FSGROWFSLOG & XFS_IOC_FSGROWFSRT */
fs/xfs/xfs_ioctl.c +3 -1
@@ -976,10 +976,12 @@ xfs_ioc_ag_geometry(if (copy_from_user (&ageo, arg,sizeof (ageo)))return -EFAULT ; -if (ageo.ag_flags) +if (ageo.ag_flags & ~XFS_AG_FLAG_ALL )return -EINVAL ;if (memchr_inv (&ageo.ag_reserved,0 ,sizeof (ageo.ag_reserved)))return -EINVAL ; +if ((ageo.ag_flags &XFS_AG_FLAG_UPDATE ) && !capable (CAP_SYS_ADMIN )) +return -EPERM ; pag =xfs_perag_get (mp, ageo.ag_number);if (!pag)
PATCH 5/5 xfs: apply noalloc mode to inode allocations too
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Don't allow inode allocations from this group if it's marked noalloc.
--- fs/xfs/libxfs/xfs_ialloc.c | 3 +++ 1 file changed, 3 insertions(+)
fs/xfs/libxfs/xfs_ialloc.c +3 -0
@@ -1064,6 +1064,7 @@ xfs_dialloc_ag_inobt(ASSERT (xfs_perag_initialised_agi (pag));ASSERT (xfs_perag_allows_inodes (pag)); +ASSERT (!xfs_perag_prohibits_alloc (pag));ASSERT (pag->pagi_freecount >0 ); restart_pagno:
@@ -1692,6 +1693,8 @@ xfs_dialloc_good_ag(return false;if (!xfs_perag_allows_inodes (pag))return false; +if (xfs_perag_prohibits_alloc (pag)) +return false;if (!xfs_perag_initialised_agi (pag)) { error =xfs_ialloc_read_agi (pag, tp,NULL );
PATCH 3/5 xfs: report refcount information to userspace
In reply to Darrick J. WongHi all,
Create a new ioctl to report the number of owners of each disk block so that reflink-aware defraggers can make better decisions about which extents to target.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=report-refcounts
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=report-refcounts
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=report-refcounts
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_fs_staging.h | 82 +++ fs/xfs/xfs_fsrefs.c | 970 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_fsrefs.h | 34 + fs/xfs/xfs_ioctl.c | 135 ++++++ fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 99 ++++ 7 files changed, 1322 insertions(+) create mode 100644 fs/xfs/xfs_fsrefs.c create mode 100644 fs/xfs/xfs_fsrefs.h
PATCH 1/1 xfs: export reference count information to userspace
From: Darrick J. Wong <[email protected]>
Export refcount info to userspace so we can prototype a sharing-aware defrag/fs rearranging tool.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_fs_staging.h | 82 +++ fs/xfs/xfs_fsrefs.c | 970 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_fsrefs.h | 34 + fs/xfs/xfs_ioctl.c | 135 ++++++ fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 99 ++++ 7 files changed, 1322 insertions(+) create mode 100644 fs/xfs/xfs_fsrefs.c create mode 100644 fs/xfs/xfs_fsrefs.h
fs/xfs/Makefile +1 -0
@@ -81,6 +81,7 @@ xfs-y += xfs_aops.o \ xfs_filestream.o \ xfs_fsmap.o \ xfs_fsops.o \ + xfs_fsrefs.o \ xfs_globals.o \ xfs_health.o \ xfs_icache.o \
fs/xfs/libxfs/xfs_fs_staging.h +82 -0
86 lines not on this page · raw message
fs/xfs/xfs_fsrefs.c new file +970 -0
970 lines not on this page · raw message
fs/xfs/xfs_fsrefs.h new file +34 -0
@@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021-2024 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_FSREFS_H__ +#define __XFS_FSREFS_H__ + +struct xfs_getfsrefs ; + +/* internal fsrefs representation */ +struct xfs_fsrefs { +dev_t fcr_device;/* device id */ +uint32_t fcr_flags;/* mapping flags */ +uint64_t fcr_physical;/* device offset of segment */ +uint64_t fcr_owners;/* number of owners */ +xfs_filblks_t fcr_length;/* length of segment, blocks */ +}; + +struct xfs_fsrefs_head { +uint32_t fch_iflags;/* control flags */ +uint32_t fch_oflags;/* output flags */ +unsigned int fch_count;/* # of entries in array incl. input */ +unsigned int fch_entries;/* # of entries filled in (output). */ + +struct xfs_fsrefs fch_keys[2 ];/* low and high keys */ +}; + +void xfs_fsrefs_to_internal (struct xfs_fsrefs *dest,struct xfs_getfsrefs *src); + +int xfs_getfsrefs (struct xfs_mount *mp,struct xfs_fsrefs_head *head, +struct xfs_getfsrefs *out_recs); + +#endif /* __XFS_FSREFS_H__ */
fs/xfs/xfs_ioctl.c +135 -0
153 lines not on this page · raw message
fs/xfs/xfs_trace.c +1 -0
@@ -47,6 +47,7 @@#include "xfs_rtgroup.h" #include "xfs_rmap.h" #include "xfs_refcount.h" +#include "xfs_fsrefs.h" static inline void xfs_rmapbt_crack_agno_opdev(
fs/xfs/xfs_trace.h +99 -0
111 lines not on this page · raw message
PATCH 4/5 xfs: defragment free space
In reply to Darrick J. WongHi all,
These patches contain experimental code to enable userspace to defragment the free space in a filesystem. Two purposes are imagined for this
functionality: clearing space at the end of a filesystem before shrinking it, and clearing free space in anticipation of making a large allocation.
The first patch adds a new fallocate mode that allows userspace to allocate free space from the filesystem into a file. The goal here is to allow the filesystem shrink process to prevent allocation from a certain part of the filesystem while a free space defragmenter evacuates all the files from the doomed part of the filesystem.
The second patch amends the online repair system to allow the sysadmin to forcibly rebuild metadata structures, even if they're not corrupt. Without adding an ioctl to move metadata btree blocks, this is the only way to dislodge metadata.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=defrag-freespace
--- fs/xfs/libxfs/xfs_alloc.c | 88 +++++++++++ fs/xfs/libxfs/xfs_alloc.h | 3 fs/xfs/libxfs/xfs_bmap.c | 150 +++++++++++++++++++ fs/xfs/libxfs/xfs_bmap.h | 3 fs/xfs/libxfs/xfs_fs.h | 1 fs/xfs/libxfs/xfs_fs_staging.h | 15 ++ fs/xfs/xfs_bmap_util.c | 319 +++++++++++++++++++++++++++++++++++++++- fs/xfs/xfs_bmap_util.h | 3 fs/xfs/xfs_file.c | 79 ++++++++++ fs/xfs/xfs_file.h | 2 fs/xfs/xfs_ioctl.c | 68 +++++++++ fs/xfs/xfs_rtalloc.c | 52 +++++++ fs/xfs/xfs_rtalloc.h | 12 +- fs/xfs/xfs_trace.h | 72 ++++++++- 14 files changed, 856 insertions(+), 11 deletions(-)
PATCH 1/2 xfs: capture the offset and length in fallocate tracepoints
From: Darrick J. Wong <[email protected]>
Change the class of the fallocate tracepoints to capture the offset and length of the requested operation.
--- fs/xfs/xfs_bmap_util.c | 8 ++++---- fs/xfs/xfs_file.c | 2 +- fs/xfs/xfs_trace.h | 10 +++++----- 3 files changed, 10 insertions(+), 10 deletions(-)
fs/xfs/xfs_bmap_util.c +4 -4
@@ -826,7 +826,7 @@ xfs_alloc_file_space(xfs_bmbt_irec_t imaps[1 ], *imapp;int error; -trace_xfs_alloc_file_space (ip); +trace_xfs_alloc_file_space (ip, offset, len);if (xfs_is_shutdown (mp))return -EIO ;
@@ -1014,7 +1014,7 @@ xfs_free_file_space(xfs_fileoff_t endoffset_fsb;int done =0 , error; -trace_xfs_free_file_space (ip); +trace_xfs_free_file_space (ip, offset, len); error =xfs_qm_dqattach (ip);if (error)
@@ -1152,7 +1152,7 @@ xfs_collapse_file_space(ASSERT (xfs_isilocked (ip,XFS_IOLOCK_EXCL ));ASSERT (xfs_isilocked (ip,XFS_MMAPLOCK_EXCL )); -trace_xfs_collapse_file_space (ip); +trace_xfs_collapse_file_space (ip, offset, len); error =xfs_free_file_space (ip, offset, len);if (error)
@@ -1222,7 +1222,7 @@ xfs_insert_file_space(ASSERT (xfs_isilocked (ip,XFS_IOLOCK_EXCL ));ASSERT (xfs_isilocked (ip,XFS_MMAPLOCK_EXCL )); -trace_xfs_insert_file_space (ip); +trace_xfs_insert_file_space (ip, offset, len); error =xfs_bmap_can_insert_extents (ip, stop_fsb, shift_fsb);if (error)
fs/xfs/xfs_file.c +1 -1
@@ -1150,7 +1150,7 @@ xfs_file_fallocate(*/ unsigned int blksize =i_blocksize (inode); -trace_xfs_zero_file_space (ip); +trace_xfs_zero_file_space (ip, offset, len);/* Unshare around the region to zero, if needed. */ if (xfs_file_write_needs_cow_around (ip, offset, len)) {
fs/xfs/xfs_trace.h +5 -5
@@ -831,11 +831,6 @@ DEFINE_INODE_EVENT(xfs_getattr);DEFINE_INODE_EVENT (xfs_setattr);DEFINE_INODE_EVENT (xfs_readlink);DEFINE_INODE_EVENT (xfs_inactive_symlink); -DEFINE_INODE_EVENT (xfs_alloc_file_space); -DEFINE_INODE_EVENT (xfs_free_file_space); -DEFINE_INODE_EVENT (xfs_zero_file_space); -DEFINE_INODE_EVENT (xfs_collapse_file_space); -DEFINE_INODE_EVENT (xfs_insert_file_space);DEFINE_INODE_EVENT (xfs_readdir);#ifdef CONFIG_XFS_POSIX_ACL DEFINE_INODE_EVENT (xfs_get_acl);
@@ -1629,6 +1624,11 @@ DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write);DEFINE_SIMPLE_IO_EVENT (xfs_end_io_direct_write_unwritten);DEFINE_SIMPLE_IO_EVENT (xfs_end_io_direct_write_append);DEFINE_SIMPLE_IO_EVENT (xfs_file_splice_read); +DEFINE_SIMPLE_IO_EVENT (xfs_alloc_file_space); +DEFINE_SIMPLE_IO_EVENT (xfs_free_file_space); +DEFINE_SIMPLE_IO_EVENT (xfs_zero_file_space); +DEFINE_SIMPLE_IO_EVENT (xfs_collapse_file_space); +DEFINE_SIMPLE_IO_EVENT (xfs_insert_file_space); DECLARE_EVENT_CLASS(xfs_itrunc_class, TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size),
PATCH 2/2 xfs: add an ioctl to map free space into a file
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new ioctl to map free physical space into a file, at the same file offset as if the file were a sparse image of the physical device backing the filesystem. The intent here is to use this to prototype a free space defragmentation tool.
--- fs/xfs/libxfs/xfs_alloc.c | 88 +++++++++++ fs/xfs/libxfs/xfs_alloc.h | 3 fs/xfs/libxfs/xfs_bmap.c | 150 +++++++++++++++++++ fs/xfs/libxfs/xfs_bmap.h | 3 fs/xfs/libxfs/xfs_fs.h | 1 fs/xfs/libxfs/xfs_fs_staging.h | 15 ++ fs/xfs/xfs_bmap_util.c | 311 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_bmap_util.h | 3 fs/xfs/xfs_file.c | 77 ++++++++++ fs/xfs/xfs_file.h | 2 fs/xfs/xfs_ioctl.c | 68 +++++++++ fs/xfs/xfs_rtalloc.c | 52 +++++++ fs/xfs/xfs_rtalloc.h | 12 +- fs/xfs/xfs_trace.h | 62 ++++++++ 14 files changed, 846 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_alloc.c +88 -0
@@ -4096,3 +4096,91 @@ xfs_extfree_intent_destroy_cache(void)kmem_cache_destroy (xfs_extfree_item_cache); xfs_extfree_item_cache =NULL ; } + +/* +* Find the next chunk of free space in @pag starting at @agbno and going no +* higher than @end_agbno. Set @agbno and @len to whatever free space we find, +* or to @end_agbno if we find no space. +*/ +int +xfs_alloc_find_freesp ( +struct xfs_trans *tp, +struct xfs_perag *pag, +xfs_agblock_t *agbno, +xfs_agblock_t end_agbno, +xfs_extlen_t *len) +{ +struct xfs_mount *mp = pag->pag_mount; +struct xfs_btree_cur *cur; +struct xfs_buf *agf_bp =NULL ; +xfs_agblock_t found_agbno; +xfs_extlen_t found_len; +int found; +int error; + +trace_xfs_alloc_find_freesp (mp, pag->pag_agno, *agbno, + end_agbno - *agbno); + + error =xfs_alloc_read_agf (pag, tp,0 , &agf_bp); +if (error) +return error; + + cur =xfs_allocbt_init_cursor (mp, tp, agf_bp, pag,XFS_BTNUM_BNO ); + +/* Try to find a free extent that starts before here. */ + error =xfs_alloc_lookup_le (cur, *agbno,0 , &found); +if (error) + goto out_cur; +if (found) { + error =xfs_alloc_get_rec (cur, &found_agbno, &found_len, + &found); +if (error) + goto out_cur; +if (XFS_IS_CORRUPT (mp, !found)) { +xfs_btree_mark_sick (cur); + error = -EFSCORRUPTED ; + goto out_cur; + } + +if (found_agbno + found_len > *agbno) + goto found; + } + +/* Examine the next record if free extent not in range. */ + error =xfs_btree_increment (cur,0 , &found); +if (error) + goto out_cur; +if (!found) + goto next_ag; + + error =xfs_alloc_get_rec (cur, &found_agbno, &found_len, &found); +if (error) + goto out_cur; +if (XFS_IS_CORRUPT (mp, !found)) { +xfs_btree_mark_sick (cur); + error = -EFSCORRUPTED ; + goto out_cur; + } + +if (found_agbno >= end_agbno) + goto next_ag; + +found: +/* Found something, so update the mapping. */ +trace_xfs_alloc_find_freesp_done (mp, pag->pag_agno, found_agbno, + found_len); +if (found_agbno < *agbno) { + found_len -= *agbno - found_agbno; + found_agbno = *agbno; + } + *len = found_len; + *agbno = found_agbno; + goto out_cur; +next_ag: +/* Found nothing, so advance the cursor beyond the end of the range. */ + *agbno = end_agbno; + *len =0 ; +out_cur: +xfs_btree_del_cursor (cur, error); +return error; +}
fs/xfs/libxfs/xfs_alloc.h +3 -0
@@ -290,5 +290,8 @@ void xfs_extfree_intent_destroy_cache(void);xfs_failaddr_t xfs_validate_ag_length (struct xfs_buf *bp,uint32_t seqno,uint32_t length); +int xfs_alloc_find_freesp (struct xfs_trans *tp,struct xfs_perag *pag, +xfs_agblock_t *agbno,xfs_agblock_t end_agbno, +xfs_extlen_t *len);#endif /* __XFS_ALLOC_H__ */
fs/xfs/libxfs/xfs_bmap.c +150 -0
159 lines not on this page · raw message
fs/xfs/libxfs/xfs_bmap.h +3 -0
@@ -197,6 +197,9 @@ int xfs_bmapi_read(struct xfs_inode *ip, xfs_fileoff_t bno,int xfs_bmapi_write (struct xfs_trans *tp,struct xfs_inode *ip,xfs_fileoff_t bno,xfs_filblks_t len,uint32_t flags,xfs_extlen_t total,struct xfs_bmbt_irec *mval,int *nmap); +int xfs_bmapi_freesp (struct xfs_trans *tp,struct xfs_inode *ip, +xfs_fsblock_t fsbno,xfs_extlen_t len, +struct xfs_bmbt_irec *mval);int xfs_bunmapi (struct xfs_trans *tp,struct xfs_inode *ip,xfs_fileoff_t bno,xfs_filblks_t len,uint32_t flags,xfs_extnum_t nexts,int *done);
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -864,6 +864,7 @@ struct xfs_scrub_metadata {#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry)/* XFS_IOC_GETPARENTS ---- staging 62 */ /* XFS_IOC_RTGROUP_GEOMETRY - staging 63 */ +/* XFS_IOC_MAP_FREESP ---- staging 64 */ /* * ioctl commands that replace IRIX syssgi()'s
fs/xfs/libxfs/xfs_fs_staging.h +15 -0
@@ -303,4 +303,19 @@ xfs_getfsrefs_advance(/* XXX stealing XFS_IOC_GETBIOSIZE */ #define XFS_IOC_GETFSREFCOUNTS _IOWR('X', 47, struct xfs_getfsrefs_head) +/* map free space to file */ + +struct xfs_map_freesp { +__s64 offset;/* disk address to map, in bytes */ +__s64 len;/* length in bytes */ +__u64 flags;/* must be zero */ +__u64 pad;/* must be zero */ +}; + +/* +* XFS_IOC_MAP_FREESP maps all the free physical space in the filesystem into +* the file at the same offsets. This ioctl requires CAP_SYS_ADMIN. +*/ +#define XFS_IOC_MAP_FREESP _IOWR('X', 64, struct xfs_map_freesp) +#endif /* __XFS_FS_STAGING_H__ */
fs/xfs/xfs_bmap_util.c +311 -0
320 lines not on this page · raw message
fs/xfs/xfs_bmap_util.h +3 -0
@@ -61,6 +61,7 @@ int xfs_collapse_file_space(struct xfs_inode *, xfs_off_t offset, xfs_off_t len);int xfs_insert_file_space (struct xfs_inode *,xfs_off_t offset,xfs_off_t len); +int xfs_map_free_space (struct xfs_inode *ip,xfs_off_t off,xfs_off_t len);/* EOF block manipulation functions */ bool xfs_can_free_eofblocks (struct xfs_inode *ip,bool force);
@@ -79,8 +80,10 @@ int xfs_flush_unmap_range(struct xfs_inode *ip, xfs_off_t offset,#ifdef CONFIG_XFS_RT int xfs_convert_bigalloc_file_space (struct xfs_inode *ip,loff_t pos,uint64_t len); +int xfs_map_free_rt_space (struct xfs_inode *ip,xfs_off_t off,xfs_off_t len);#else # define xfs_convert_bigalloc_file_space (ip, pos, len) (-EOPNOTSUPP) +# define xfs_map_free_rt_space (ip, off, len) (-EOPNOTSUPP)#endif #endif /* __XFS_BMAP_UTIL_H__ */
fs/xfs/xfs_file.c +77 -0
83 lines not on this page · raw message
fs/xfs/xfs_file.h +2 -0
@@ -15,4 +15,6 @@ bool xfs_is_falloc_aligned(struct xfs_inode *ip, loff_t pos,bool xfs_truncate_needs_cow_around (struct xfs_inode *ip,loff_t pos);int xfs_file_unshare_at (struct xfs_inode *ip,loff_t pos); +int xfs_file_map_freesp (struct file *file,const struct xfs_map_freesp *mf); +#endif /* __XFS_FILE_H__ */
fs/xfs/xfs_ioctl.c +68 -0
86 lines not on this page · raw message
fs/xfs/xfs_rtalloc.c +52 -0
55 lines not on this page · raw message
fs/xfs/xfs_rtalloc.h +11 -1
@@ -86,8 +86,17 @@ int xfs_growfs_check_rtgeom(const struct xfs_mount *mp, xfs_rfsblock_t dblocks, xfs_rfsblock_t rblocks, xfs_agblock_t rextsize, xfs_rtblock_t rextents, xfs_extlen_t rbmblocks, uint8_t rextslog); + +int xfs_rtalloc_find_freesp (struct xfs_trans *tp,xfs_rtxnum_t *rtx, +xfs_rtxnum_t end_rtx,xfs_rtxlen_t *len_rtx);#else -# define xfs_rtallocate_extent (t,b,min,max,l,f,p,rb) (-ENOSYS) +static inline int +xfs_rtallocate_extent (struct xfs_trans *tp,xfs_rtxnum_t start, +xfs_rtxlen_t minlen,xfs_rtxlen_t maxlen,xfs_rtxlen_t *len, +int wasdel,xfs_rtxlen_t prod,xfs_rtxnum_t *rtblock) +{ +return -ENOSYS ; +}# define xfs_rtpick_extent (m,t,l,rb) (-ENOSYS)# define xfs_growfs_rt (mp,in) (-ENOSYS)# define xfs_rtalloc_reinit_frextents (m) (0)
@@ -109,6 +118,7 @@ xfs_rtmount_init(# define xfs_rt_resv_init (mp) (0)# define xfs_growfs_check_rtgeom (mp, d, r, rs, rx, rb, rl) (0)# define xfs_rtmount_dqattach (mp) (0) +# define xfs_rtalloc_find_freesp (tp, rtx, end_rtx, len_rtx) (-ENOSYS)#endif /* CONFIG_XFS_RT */ #endif /* __XFS_RTALLOC_H__ */
fs/xfs/xfs_trace.h +62 -0
98 lines not on this page · raw message
PATCH v2 5/5 xfs: aligned file data extent mappings
In reply to Darrick J. WongHi all,
Years ago during the adaptation of XFS to persistent memory, a product development team expressed a desire to guarantee that mmap mappings of an S_DAX file on pmem would always get a PMD mapping to increase TLB hit rates. That means ensuring that all space allocations are aligned to 2MB (on x86) and ensuring that the file range of each file space mapping are aligned to 2MB. Then pmem faded.
NVME/SCSI atomic write unit support has brought this discussion back to the forefront, because untorn writes to conventional storage only work if the space allocations are aligned to the max atomic write unit. To simplify the administrative model a bit, we want to ensure that the mappings are aligned to the same value. IOWs, for a storage device that supports up to 64k atomic write units, we want to ensure that the file range of all mappings are aligned to 64k.
From a file metadata perspective, these are nearly the same use case. The realtime volume already has a realtime extent size field that forces the alignment of both space and mapping, but not everyone wants to adminster a realtime volume. This patchset adds a forcealign superblock feature and inode flag so that we can impose the same alignment restrictions on selected files on the data volume.
IOWs, you can now set an inode flag that causes regular files on the data volume to have rextsize-aligned space mappings. This is an improvement over ext4 bigalloc style things that impose the larger allocation unit size even for things that don't need it.
Note: Currently this patchset reuses sb_rextsize for the forcealign value because that enabled reuse of the COW around code that I wrote for realtime reflink that ensures that remapping operations follow forcealign.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=file-force-align
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=file-force-align
--- fs/xfs/libxfs/xfs_bmap.c | 48 +++++++++++++++++++++++++++++++++++++--- fs/xfs/libxfs/xfs_format.h | 19 +++++++++++++--- fs/xfs/libxfs/xfs_inode_buf.c | 36 ++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_inode_buf.h | 3 +++ fs/xfs/libxfs/xfs_inode_util.c | 14 ++++++++++++ fs/xfs/libxfs/xfs_sb.c | 30 +++++++++++++++++++++++++ fs/xfs/scrub/inode_repair.c | 41 ++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 1 + fs/xfs/xfs_bmap_util.c | 2 -- fs/xfs/xfs_bmap_util.h | 4 ++- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 8 ++++++- fs/xfs/xfs_ioctl.c | 14 ++++++++++++ fs/xfs/xfs_iomap.c | 4 +++ fs/xfs/xfs_mount.h | 2 ++ fs/xfs/xfs_reflink.c | 16 ++++++++++--- fs/xfs/xfs_rtalloc.c | 4 +++ fs/xfs/xfs_super.c | 4 +++ fs/xfs/xfs_trace.h | 4 --- include/uapi/linux/fs.h | 2 ++ 20 files changed, 236 insertions(+), 22 deletions(-)
PATCH 1/4 xfs: create a new inode flag to require extsize alignment of file data space
From: Darrick J. Wong <[email protected]>
Add a new inode flag to require that all file data extent mappings must be aligned (both the file offset range and the allocated space itself) to the extent size hint. Having a separate COW extent size hint is no longer allowed.
The goal here is to enable sysadmins and users to mandate that all space mappings in a file must have a startoff/blockcount that are aligned to (say) a 2MB alignment and that the startblock/blockcount will follow the same alignment.
--- fs/xfs/libxfs/xfs_format.h | 16 ++++++++++++++-- fs/xfs/libxfs/xfs_inode_buf.c | 36 +++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_inode_buf.h | 3 +++ fs/xfs/libxfs/xfs_inode_util.c | 14 ++++++++++++++ fs/xfs/libxfs/xfs_sb.c | 30 +++++++++++++++++++++++++++++ fs/xfs/scrub/inode_repair.c | 41 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 1 + fs/xfs/xfs_inode.h | 5 +++++ fs/xfs/xfs_ioctl.c | 14 ++++++++++++++ fs/xfs/xfs_mount.h | 2 ++ fs/xfs/xfs_rtalloc.c | 4 ++++ fs/xfs/xfs_super.c | 4 ++++ include/uapi/linux/fs.h | 2 ++ 13 files changed, 170 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_format.h +14 -2
@@ -103,7 +103,12 @@ typedef struct xfs_sb {xfs_ino_t sb_rootino;/* root inode number */ xfs_ino_t sb_rbmino;/* bitmap inode for realtime extents */ xfs_ino_t sb_rsumino;/* summary inode for rt bitmap */ -xfs_agblock_t sb_rextsize;/* realtime extent size, blocks */ +/* +* Realtime extent size, blocks. If the FORCEALIGN feature is set, +* the allocation group size must be a multiple of this value, and +* file data allocations will be aligned to this value. +*/ +xfs_agblock_t sb_rextsize;xfs_agblock_t sb_agblocks;/* size of an allocation group */ xfs_agnumber_t sb_agcount;/* number of allocation groups */ xfs_extlen_t sb_rbmblocks;/* number of rt bitmap blocks */
@@ -387,6 +392,8 @@ xfs_sb_has_compat_feature(#define XFS_SB_FEAT_RO_COMPAT_RMAPBT (1 << 1)/* reverse map btree */ #define XFS_SB_FEAT_RO_COMPAT_REFLINK (1 << 2)/* reflinked files */ #define XFS_SB_FEAT_RO_COMPAT_INOBTCNT (1 << 3)/* inobt block counts */ +/* all AGs and data allocations must be aligned to rextsize, even for !rt files */ +#define XFS_SB_FEAT_RO_COMPAT_FORCEALIGN (1 << 30)#define XFS_SB_FEAT_RO_COMPAT_ALL \ (XFS_SB_FEAT_RO_COMPAT_FINOBT | \ XFS_SB_FEAT_RO_COMPAT_RMAPBT | \
@@ -1206,6 +1213,8 @@ static inline void xfs_dinode_put_rdev(struct xfs_dinode *dip, xfs_dev_t rdev)#define XFS_DIFLAG2_COWEXTSIZE_BIT 2/* copy on write extent size hint */ #define XFS_DIFLAG2_BIGTIME_BIT 3/* big timestamps */ #define XFS_DIFLAG2_NREXT64_BIT 4/* large extent counters */ +/* data extent mappings for regular files must be aligned to extent size hint */ +#define XFS_DIFLAG2_FORCEALIGN_BIT 5#define XFS_DIFLAG2_METADIR_BIT 63/* filesystem metadata */ #define XFS_DIFLAG2_DAX (1ULL << XFS_DIFLAG2_DAX_BIT)
@@ -1239,9 +1248,12 @@ static inline void xfs_dinode_put_rdev(struct xfs_dinode *dip, xfs_dev_t rdev)*/ #define XFS_DIFLAG2_METADIR (1ULL << XFS_DIFLAG2_METADIR_BIT) +#define XFS_DIFLAG2_FORCEALIGN (1ULL << XFS_DIFLAG2_FORCEALIGN_BIT) +#define XFS_DIFLAG2_ANY \ (XFS_DIFLAG2_DAX | XFS_DIFLAG2_REFLINK | XFS_DIFLAG2_COWEXTSIZE | \ - XFS_DIFLAG2_BIGTIME | XFS_DIFLAG2_NREXT64 | XFS_DIFLAG2_METADIR) + XFS_DIFLAG2_BIGTIME | XFS_DIFLAG2_NREXT64 | XFS_DIFLAG2_METADIR | \ + XFS_DIFLAG2_FORCEALIGN)static inline bool xfs_dinode_has_bigtime (const struct xfs_dinode *dip) {
fs/xfs/libxfs/xfs_inode_buf.c +36 -0
@@ -726,6 +726,14 @@ xfs_dinode_verify( }else if (nextents + naextents ==0 && nblocks !=0 )return __this_address; +if (flags2 &XFS_DIFLAG2_FORCEALIGN ) { + fa =xfs_inode_validate_forcealign (mp, mode, flags, +be32_to_cpu (dip->di_extsize), +be32_to_cpu (dip->di_cowextsize)); +if (fa) +return fa; + } +return NULL ; }
@@ -900,3 +908,31 @@ xfs_inode_validate_cowextsize(return NULL ; } + +/* Validate the forcealign inode flag */ +xfs_failaddr_t +xfs_inode_validate_forcealign ( +struct xfs_mount *mp, +uint16_t mode, +uint16_t flags, +uint32_t extsize, +uint32_t cowextsize) +{ +/* superblock rocompat feature flag required */ +if (!xfs_has_forcealign (mp)) +return __this_address; + +/* Only regular files and directories */ +if (!S_ISDIR (mode) && !S_ISREG (mode)) +return __this_address; + +/* Requires no extent size hint */ +if (extsize !=0 ) +return __this_address; + +/* Requires no cow extent size hint */ +if (cowextsize !=0 ) +return __this_address; + +return NULL ; +}
fs/xfs/libxfs/xfs_inode_buf.h +3 -0
@@ -36,6 +36,9 @@ xfs_failaddr_t xfs_inode_validate_extsize(struct xfs_mount *mp,xfs_failaddr_t xfs_inode_validate_cowextsize (struct xfs_mount *mp,uint32_t cowextsize,uint16_t mode,uint16_t flags,uint64_t flags2); +xfs_failaddr_t xfs_inode_validate_forcealign (struct xfs_mount *mp, +uint16_t mode,uint16_t flags,uint32_t extsize, +uint32_t cowextsize);static inline uint64_t xfs_inode_encode_bigtime (struct timespec64 tv) {
fs/xfs/libxfs/xfs_inode_util.c +14 -0
@@ -81,6 +81,8 @@ xfs_flags2diflags2( di_flags2 |=XFS_DIFLAG2_DAX ;if (xflags &FS_XFLAG_COWEXTSIZE ) di_flags2 |=XFS_DIFLAG2_COWEXTSIZE ; +if (xflags &FS_XFLAG_FORCEALIGN ) + di_flags2 |=XFS_DIFLAG2_FORCEALIGN ;return di_flags2; }
@@ -127,6 +129,8 @@ xfs_ip2xflags( flags |=FS_XFLAG_DAX ;if (ip->i_diflags2 &XFS_DIFLAG2_COWEXTSIZE ) flags |=FS_XFLAG_COWEXTSIZE ; +if (ip->i_diflags2 &XFS_DIFLAG2_FORCEALIGN ) + flags |=FS_XFLAG_FORCEALIGN ; }if (xfs_inode_has_attr_fork (ip))
@@ -228,6 +232,8 @@ xfs_inode_inherit_flags2( ip->i_diflags2 |=XFS_DIFLAG2_DAX ;if (pip->i_diflags2 &XFS_DIFLAG2_METADIR ) ip->i_diflags2 |=XFS_DIFLAG2_METADIR ; +if (pip->i_diflags2 &XFS_DIFLAG2_FORCEALIGN ) + ip->i_diflags2 |=XFS_DIFLAG2_FORCEALIGN ;/* Don't let invalid cowextsize hints propagate. */ failaddr = xfs_inode_validate_cowextsize(ip->i_mount, ip->i_cowextsize,
@@ -236,6 +242,14 @@ xfs_inode_inherit_flags2( ip->i_diflags2 &= ~XFS_DIFLAG2_COWEXTSIZE ; ip->i_cowextsize =0 ; } + +if (ip->i_diflags2 &XFS_DIFLAG2_FORCEALIGN ) { + failaddr =xfs_inode_validate_forcealign (ip->i_mount, +VFS_I (ip)->i_mode, ip->i_diflags, ip->i_extsize, + ip->i_cowextsize); +if (failaddr) + ip->i_diflags2 &= ~XFS_DIFLAG2_FORCEALIGN ; + } }/* Initialise an inode's attributes. */
fs/xfs/libxfs/xfs_sb.c +30 -0
@@ -167,6 +167,9 @@ xfs_sb_version_to_features( features |=XFS_FEAT_REFLINK ;if (sbp->sb_features_ro_compat &XFS_SB_FEAT_RO_COMPAT_INOBTCNT ) features |=XFS_FEAT_INOBTCNT ; +if (sbp->sb_features_ro_compat &XFS_SB_FEAT_RO_COMPAT_FORCEALIGN ) + features |=XFS_FEAT_FORCEALIGN ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_FTYPE ) features |=XFS_FEAT_FTYPE ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_SPINODES )
@@ -370,6 +373,27 @@ xfs_validate_sb_rtgroups(return 0 ; } +static int +xfs_validate_sb_forcealign ( +struct xfs_mount *mp, +struct xfs_sb *sbp) +{ +if (sbp->sb_rextsize ==0 ) { +xfs_warn (mp, +"Cannot have forced allocation alignment of zero." ); +return -EINVAL ; + } + +if (sbp->sb_agblocks % sbp->sb_rextsize !=0 ) { +xfs_warn (mp, +"Allocation group size %u not aligned to forcealign %u." , + sbp->sb_agblocks, sbp->sb_rextsize); +return -EINVAL ; + } + +return 0 ; +} +/* Check the validity of the SB. */ STATIC int xfs_validate_sb_common(
@@ -437,6 +461,12 @@ xfs_validate_sb_common(if (error)return error; } + +if (sbp->sb_features_ro_compat &XFS_SB_FEAT_RO_COMPAT_FORCEALIGN ) { + error =xfs_validate_sb_forcealign (mp, sbp); +if (error) +return error; + } }else if (sbp->sb_qflags & (XFS_PQUOTA_ENFD |XFS_GQUOTA_ENFD |XFS_PQUOTA_CHKD |XFS_GQUOTA_CHKD )) { xfs_notice(mp,
fs/xfs/scrub/inode_repair.c +41 -0
53 lines not on this page · raw message
fs/xfs/scrub/trace.h +1 -0
@@ -2559,6 +2559,7 @@ DEFINE_REPAIR_DINODE_EVENT(xrep_dinode_zap_forks);DEFINE_REPAIR_DINODE_EVENT (xrep_dinode_zap_dfork);DEFINE_REPAIR_DINODE_EVENT (xrep_dinode_zap_afork);DEFINE_REPAIR_DINODE_EVENT (xrep_dinode_ensure_forkoff); +DEFINE_REPAIR_DINODE_EVENT (xrep_dinode_forcealign); DECLARE_EVENT_CLASS(xrep_inode_class, TP_PROTO(struct xfs_scrub *sc),
fs/xfs/xfs_inode.h +5 -0
@@ -326,6 +326,11 @@ static inline bool xfs_inode_has_large_extent_counts(struct xfs_inode *ip)return ip->i_diflags2 &XFS_DIFLAG2_NREXT64 ; } +static inline bool xfs_inode_force_align (struct xfs_inode *ip) +{ +return ip->i_diflags2 &XFS_DIFLAG2_FORCEALIGN ; +} +static inline bool xfs_inode_has_bigallocunit (struct xfs_inode *ip) {return XFS_IS_REALTIME_INODE (ip) && ip->i_mount->m_sb.sb_rextsize >1 ;
fs/xfs/xfs_ioctl.c +14 -0
@@ -1144,6 +1144,20 @@ xfs_ioctl_setattr_xflags(if (i_flags2 && !xfs_has_v3inodes (mp))return -EINVAL ; +/* +* Force-align requires a zero extent size hint and a zero cow extent +* size hint. +*/ +if (fa->fsx_xflags &FS_XFLAG_FORCEALIGN ) { +if (!xfs_has_forcealign (mp)) +return -EINVAL ; +if (fa->fsx_xflags &FS_XFLAG_COWEXTSIZE ) +return -EINVAL ; +if (fa->fsx_xflags & (FS_XFLAG_EXTSIZE | +FS_XFLAG_EXTSZINHERIT )) +return -EINVAL ; + } + ip->i_diflags =xfs_flags2diflags (ip, fa->fsx_xflags); ip->i_diflags2 = i_flags2;
fs/xfs/xfs_mount.h +2 -0
@@ -310,6 +310,7 @@ typedef struct xfs_mount {#define XFS_FEAT_NREXT64 (1ULL << 26)/* large extent counters */ #define XFS_FEAT_METADIR (1ULL << 27)/* metadata directory tree */ #define XFS_FEAT_RTGROUPS (1ULL << 28)/* realtime groups */ +#define XFS_FEAT_FORCEALIGN (1ULL << 29)/* aligned file data extents */ /* Mount features */ #define XFS_FEAT_NOATTR2 (1ULL << 48)/* disable attr2 creation */
@@ -375,6 +376,7 @@ __XFS_HAS_FEAT(needsrepair, NEEDSREPAIR) __XFS_HAS_FEAT(large_extent_counts, NREXT64) __XFS_HAS_FEAT(metadir, METADIR) __XFS_HAS_FEAT(rtgroups, RTGROUPS) +__XFS_HAS_FEAT(forcealign, FORCEALIGN)static inline bool xfs_has_rtrmapbt (struct xfs_mount *mp) {
fs/xfs/xfs_rtalloc.c +4 -0
@@ -1238,6 +1238,10 @@ xfs_growfs_rt(if (sbp->sb_rblocks >0 && in->extsize != sbp->sb_rextsize)return -EINVAL ; +/* Cannot change rt extent size when forcealign is set. */ +if (xfs_has_forcealign (mp) && in->extsize != sbp->sb_rextsize) +return -EINVAL ; +/* Range check the extent size. */ if (XFS_FSB_TO_B (mp, in->extsize) >XFS_MAX_RTEXTSIZE ||XFS_FSB_TO_B (mp, in->extsize) <XFS_MIN_RTEXTSIZE )
fs/xfs/xfs_super.c +4 -0
@@ -1727,6 +1727,10 @@ xfs_fs_fill_super(xfs_warn (mp,"EXPERIMENTAL realtime allocation group feature in use. Use at your own risk!" ); +if (xfs_has_forcealign (mp)) +xfs_warn (mp, +"EXPERIMENTAL forced data extent alignment feature in use. Use at your own risk!" ); +if (xfs_has_reflink (mp)) {/* * Reflink doesn't support pagecache pages that span multiple
include/uapi/linux/fs.h +2 -0
@@ -140,6 +140,8 @@ struct fsxattr {#define FS_XFLAG_FILESTREAM 0x00004000/* use filestream allocator */ #define FS_XFLAG_DAX 0x00008000/* use DAX for IO */ #define FS_XFLAG_COWEXTSIZE 0x00010000/* CoW extent size allocator hint */ +/* data extent mappings for regular files must be aligned to extent size hint */ +#define FS_XFLAG_FORCEALIGN 0x00020000#define FS_XFLAG_HASATTR 0x80000000/* no DIFLAG for this */ /* the read-only stuff doesn't really belong here, but any other place is
PATCH 2/4 xfs: make file data allocations observe the 'forcealign' flag
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The existing extsize hint code already did the work of expanding file range mapping requests so that the range is aligned to the hint value. Now add the code we need to guarantee that the space allocations are also always aligned.
--- fs/xfs/libxfs/xfs_bmap.c | 48 ++++++++++++++++++++++++++++++++++++++++++---- fs/xfs/xfs_iomap.c | 4 +++- fs/xfs/xfs_reflink.c | 4 ++++ 3 files changed, 51 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +44 -4
@@ -3351,6 +3351,17 @@ xfs_bmap_btalloc_accounting( args->len); } +static inline bool +xfs_bmap_use_forcealign ( +const struct xfs_bmalloca *ap) +{ +if (!xfs_inode_force_align (ap->ip)) +return false; + +return (ap->flags &XFS_BMAPI_COWFORK ) || + (ap->datatype &XFS_ALLOC_USERDATA ); +} + static int xfs_bmap_compute_alignments( struct xfs_bmalloca *ap,
@@ -3366,6 +3377,17 @@ xfs_bmap_compute_alignments(else if (mp->m_dalign) stripe_align = mp->m_dalign; +/* +* File data mappings with forced alignment can use the stripe +* alignment if it's a multiple of the forcealign value. Otherwise, +* use the regular forcealign value. +*/ +if (xfs_bmap_use_forcealign (ap)) { +if (!stripe_align || stripe_align % mp->m_sb.sb_rextsize) + stripe_align = mp->m_sb.sb_rextsize; + args->alignment = stripe_align; + } +if (ap->flags &XFS_BMAPI_COWFORK ) align =xfs_get_cowextsz_hint (ap->ip);else if (ap->datatype &XFS_ALLOC_USERDATA )
@@ -3438,6 +3460,9 @@ xfs_bmap_exact_minlen_extent_alloc(ASSERT (ap->length); +if (xfs_inode_force_align (ap->ip)) +return -ENOSPC ; +if (ap->minlen !=1 ) { ap->blkno =NULLFSBLOCK ; ap->length =0 ;
@@ -3511,6 +3536,7 @@ xfs_bmap_btalloc_at_eof( {struct xfs_mount *mp = args->mp;struct xfs_perag *caller_pag = args->pag; +int orig_alignment = args->alignment;int error;/*
@@ -3585,10 +3611,10 @@ xfs_bmap_btalloc_at_eof(/* * Allocation failed, so turn return the allocation args to their -* original non-aligned state so the caller can proceed on allocation -* failure as if this function was never called. +* original state so the caller can proceed on allocation failure as +* if this function was never called. */ - args->alignment =1 ; + args->alignment = orig_alignment;return 0 ; }
@@ -3611,6 +3637,10 @@ xfs_bmap_btalloc_low_space( {int error; +/* Don't try unaligned last-chance allocations with forcealign */ +if (xfs_inode_force_align (ap->ip)) +return -ENOSPC ; +if (args->minlen > ap->minlen) { args->minlen = ap->minlen; error =xfs_alloc_vextent_start_ag (args, ap->blkno);
@@ -4115,7 +4145,9 @@ xfs_bmap_alloc_userdata(if (bma->offset ==0 ) bma->datatype |=XFS_ALLOC_INITIAL_USER_DATA ; -if (mp->m_dalign && bma->length >= mp->m_dalign) { +/* forcealign mode reuses the stripe unit alignment code */ +if (xfs_inode_force_align (bma->ip) || + (mp->m_dalign && bma->length >= mp->m_dalign)) { error =xfs_bmap_isaeof (bma, whichfork);if (error)return error;
@@ -6381,6 +6413,10 @@ xfs_extlen_txfs_get_extsz_hint ( struct xfs_inode *ip) { +/* forcealign means we align to rextsize */ +if (xfs_inode_force_align (ip)) +return ip->i_mount->m_sb.sb_rextsize; +/* * No point in aligning allocations if we need to COW to actually * write to them.
@@ -6405,6 +6441,10 @@ xfs_get_cowextsz_hint( {xfs_extlen_t a, b; +/* forcealign means we align to rextsize */ +if (xfs_inode_force_align (ip)) +return ip->i_mount->m_sb.sb_rextsize; + a =0 ;if (ip->i_diflags2 &XFS_DIFLAG2_COWEXTSIZE ) a = ip->i_cowextsize;
fs/xfs/xfs_iomap.c +3 -1
@@ -185,7 +185,9 @@ xfs_eof_alignment(* If mounted with the "-o swalloc" option the alignment is * increased from the strip unit size to the stripe width. */ -if (mp->m_swidth &&xfs_has_swalloc (mp)) +if (xfs_inode_force_align (ip)) + align =xfs_get_extsz_hint (ip); +else if (mp->m_swidth &&xfs_has_swalloc (mp)) align = mp->m_swidth;else if (mp->m_dalign) align = mp->m_dalign;
fs/xfs/xfs_reflink.c +4 -0
@@ -1803,6 +1803,10 @@ xfs_reflink_remap_prep(if (IS_DAX (inode_in) !=IS_DAX (inode_out)) goto out_unlock; +/* XXX Can't reflink forcealign files for now */ +if (xfs_inode_force_align (src) ||xfs_inode_force_align (dest)) + goto out_unlock; +/* Check non-power of two alignment issues, if necessary. */ if (XFS_IS_REALTIME_INODE (dest) && !is_power_of_2 (alloc_unit)) { ret = xfs_reflink_remap_check_rtalign(src, pos_in, dest,
Re: [PATCH 2/4] xfs: make file data allocations observe the 'forcealign' flag
Hi Darrick,
As mentioned internally, we have an issue for atomic writes [0] that we get an aligned and but not fully-written extent when we initially write a size less than the forcealign size, like:
#/mkfs.xfs -f -d forcealign=16k /dev/sda
...
# mount /dev/sda mnt # touch mnt/file # /test-pwritev2 -a -d -l 4096 -p 0 /root/mnt/file # direct IO, atomic write, 4096B at pos 0 # filefrag -v mnt/file Filesystem type is: 58465342 File size of mnt/file is 4096 (1 block of 4096 bytes) ext: logical_offset: physical_offset: length: expected: flags: 0: 0.. 0: 24.. 24: 1: last,eof mnt/file: 1 extent found # /test-pwritev2 -a -d -l 16384 -p 0 /root/mnt/file wrote -1 bytes at pos 0 write_size=16384 #
This causes an issue for atomic writes in that the 16K write means 2x mappings and then 2x BIOs, which we cannot tolerate.
So how about this change on top:
fs/xfs/xfs_bmap_util.c +6 -0
@@ -685,6 +685,12 @@ xfs_can_free_eofblocks( end_fsb =XFS_B_TO_FSB (mp, (xfs_ufsize_t )XFS_ISIZE (ip));if (XFS_IS_REALTIME_INODE (ip) && mp->m_sb.sb_rextsize >1 ) end_fsb =xfs_rtb_roundup_rtx (mp, end_fsb); + +/* Don't trim eof blocks */ +if (xfs_inode_force_align (ip)) { + end_fsb =roundup_64 (end_fsb,xfs_get_extsz_hint (ip)); + } + last_fsb =XFS_B_TO_FSB (mp, mp->m_super->s_maxbytes);if (last_fsb <= end_fsb)return false;
fs/xfs/xfs_iomap.c +4 -0
@@ -291,6 +291,10 @@ xfs_iomap_write_direct( }[email protected] } +if (xfs_inode_force_align (ip)) { + bmapi_flags =XFS_BMAPI_ZERO ; + } + error =xfs_trans_alloc_inode (ip, &M_RES (mp)->tr_write, dblocks, rblocks, force, &tp);if (error)
lines 1-38/38 (END)
Which gives:
#/mkfs.xfs -d forcealign=16k /dev/sda
...
# /test-pwritev2 -a -d -l 4096 -p 0 /root/mnt/file wrote 4096 bytes at pos 0 write_size=4096 # filefrag -v mnt/file Filesystem type is: 58465342 File size of mnt/file is 4096 (1 block of 4096 bytes) ext: logical_offset: physical_offset: length: expected: flags: 0: 0.. 3: 24.. 27: 4: last,eof mnt/file: 1 extent found # # /test-pwritev2 -a -d -l 16384 -p 0 /root/mnt/file wrote 16384 bytes at pos 0 write_size=16384 # filefrag -v mnt/file Filesystem type is: 58465342 File size of mnt/file is 16384 (4 blocks of 4096 bytes) ext: logical_offset: physical_offset: length: expected: flags: 0: 0.. 3: 24.. 27: 4: last,eof mnt/file: 1 extent found #
Or maybe make that change under FS_XFLAG_ATOMICWRITES flag. Previously we were pre-zero'ing the complete file to get around this.
Thanks,
John
[0] https://lore.kernel.org/linux-scsi/[email protected]/T/#mbc6824fbe9ce62c9506aa4c3f281173747695d77 (just referencing for others)
PATCH 3/4 xfs: support reflink with force align enabled
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Reuse the "big rt extents" code to do the necessary "COW around" behaviors so that we always reflink entire forcealign units.
--- fs/xfs/xfs_bmap_util.c | 2 -- fs/xfs/xfs_bmap_util.h | 4 ++-- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 3 ++- fs/xfs/xfs_reflink.c | 18 +++++++++++------- fs/xfs/xfs_trace.h | 4 ---- 6 files changed, 16 insertions(+), 17 deletions(-)
fs/xfs/xfs_bmap_util.c +0 -2
@@ -1283,7 +1283,6 @@ xfs_insert_file_space(return error; } -#ifdef CONFIG_XFS_RT /* * Decide if this is an unwritten extent that isn't aligned to an allocation * unit boundary.
@@ -1462,7 +1461,6 @@ xfs_convert_bigalloc_file_space(return 0 ; } -#endif /* CONFIG_XFS_RT */ /* * Reserve space and quota to this transaction to map in as much free space
fs/xfs/xfs_bmap_util.h +2 -2
@@ -77,12 +77,12 @@ int xfs_bmap_count_blocks(struct xfs_trans *tp, struct xfs_inode *ip,int xfs_flush_unmap_range (struct xfs_inode *ip,xfs_off_t offset,xfs_off_t len); -#ifdef CONFIG_XFS_RT int xfs_convert_bigalloc_file_space (struct xfs_inode *ip,loff_t pos,uint64_t len); + +#ifdef CONFIG_XFS_RT int xfs_map_free_rt_space (struct xfs_inode *ip,xfs_off_t off,xfs_off_t len);#else -# define xfs_convert_bigalloc_file_space (ip, pos, len) (-EOPNOTSUPP)# define xfs_map_free_rt_space (ip, off, len) (-EOPNOTSUPP)#endif
fs/xfs/xfs_inode.c +1 -1
@@ -3204,7 +3204,7 @@ xfs_inode_alloc_unitsize( {unsigned int blocks =1 ; -if (XFS_IS_REALTIME_INODE (ip)) +if (XFS_IS_REALTIME_INODE (ip) ||xfs_inode_force_align (ip)) blocks = ip->i_mount->m_sb.sb_rextsize;return XFS_FSB_TO_B (ip->i_mount, blocks);
fs/xfs/xfs_inode.h +2 -1
@@ -333,7 +333,8 @@ static inline bool xfs_inode_force_align(struct xfs_inode *ip)static inline bool xfs_inode_has_bigallocunit (struct xfs_inode *ip) { -return XFS_IS_REALTIME_INODE (ip) && ip->i_mount->m_sb.sb_rextsize >1 ; +return (xfs_inode_force_align (ip) ||XFS_IS_REALTIME_INODE (ip)) && + ip->i_mount->m_sb.sb_rextsize >1 ; }/* Decide if we need to unshare the blocks around a range that we're writing. */
fs/xfs/xfs_reflink.c +11 -7
@@ -1614,7 +1614,6 @@ xfs_reflink_zero_posteof(return xfs_zero_range (ip, isize, pos - isize,NULL ); } -#ifdef CONFIG_XFS_RT /* * Adjust the length of the remap operation to end on an allocation unit (AU) * boundary.
@@ -1661,9 +1660,6 @@ xfs_reflink_adjust_bigalloc_len(trace_xfs_reflink_adjust_bigalloc_len (src, pos_in, *len, dest, pos_out);return 0 ; } -#else -# define xfs_reflink_adjust_bigalloc_len (...) (0) -#endif /* CONFIG_XFS_RT */ /* * Check the alignment of a remap request when the allocation unit size isn't a
@@ -1803,9 +1799,17 @@ xfs_reflink_remap_prep(if (IS_DAX (inode_in) !=IS_DAX (inode_out)) goto out_unlock; -/* XXX Can't reflink forcealign files for now */ -if (xfs_inode_force_align (src) ||xfs_inode_force_align (dest)) - goto out_unlock; +/* Check non-power of two alignment issues, if necessary. */ +if ((xfs_inode_force_align (src) ||xfs_inode_force_align (dest)) && + !is_power_of_2 (alloc_unit)) { + ret =xfs_reflink_remap_check_rtalign (src, pos_in, dest, + pos_out, len, remap_flags); +if (ret) + goto out_unlock; + +/* Do the VFS checks with the regular block alignment. */ + alloc_unit = src->i_mount->m_sb.sb_blocksize; + }/* Check non-power of two alignment issues, if necessary. */ if (XFS_IS_REALTIME_INODE (dest) && !is_power_of_2 (alloc_unit)) {
fs/xfs/xfs_trace.h +0 -4
@@ -3968,9 +3968,7 @@ TRACE_EVENT(xfs_reflink_remap_blocks, __entry->dest_lblk) );DEFINE_DOUBLE_IO_EVENT (xfs_reflink_remap_range); -#ifdef CONFIG_XFS_RT DEFINE_DOUBLE_IO_EVENT (xfs_reflink_adjust_bigalloc_len); -#endif /* CONFIG_XFS_RT */ DEFINE_INODE_ERROR_EVENT (xfs_reflink_remap_range_error);DEFINE_INODE_ERROR_EVENT (xfs_reflink_set_inode_flag_error);DEFINE_INODE_ERROR_EVENT (xfs_reflink_update_inode_size_error);
@@ -4014,9 +4012,7 @@ TRACE_EVENT(xfs_ioctl_clone,DEFINE_SIMPLE_IO_EVENT (xfs_reflink_unshare);DEFINE_SIMPLE_IO_EVENT (xfs_file_cow_around);DEFINE_INODE_ERROR_EVENT (xfs_reflink_unshare_error); -#ifdef CONFIG_XFS_RT DEFINE_SIMPLE_IO_EVENT (xfs_convert_bigalloc_file_space); -#endif /* CONFIG_XFS_RT */ /* copy on write */ DEFINE_INODE_IREC_EVENT (xfs_reflink_trim_around_shared);
PATCH 4/4 xfs: enable file data force-align feature
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Enable this feature.
--- fs/xfs/libxfs/xfs_format.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +2 -1
@@ -398,7 +398,8 @@ xfs_sb_has_compat_feature( (XFS_SB_FEAT_RO_COMPAT_FINOBT | \XFS_SB_FEAT_RO_COMPAT_RMAPBT | \XFS_SB_FEAT_RO_COMPAT_REFLINK | \ - XFS_SB_FEAT_RO_COMPAT_INOBTCNT) +XFS_SB_FEAT_RO_COMPAT_INOBTCNT | \ + XFS_SB_FEAT_RO_COMPAT_FORCEALIGN)#define XFS_SB_FEAT_RO_COMPAT_UNKNOWN ~XFS_SB_FEAT_RO_COMPAT_ALLstatic inline bool xfs_sb_has_ro_compat_feature(
RFC PATCH 1/3 xfsprogs: noalloc allocation groups
In reply to Darrick J. WongHi all,
This series creates a new NOALLOC flag for allocation groups that causes the block and inode allocators to look elsewhere when trying to allocate resources. This is either the first part of a patchset to implement online shrinking (set noalloc on the last AGs, run fsr to move the files and directories) or freeze-free rmapbt rebuilding (set noalloc to prevent creation of new mappings, then hook deletion of old mappings). This is still totally a research project.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=noalloc-ags
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=noalloc-ags
--- include/xfs_trace.h | 2 + include/xfs_trans.h | 4 ++ io/aginfo.c | 45 ++++++++++++++++++-- libxfs/xfs_ag.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_ag.h | 8 ++++ libxfs/xfs_ag_resv.c | 28 +++++++++++- libxfs/xfs_defer.c | 14 ++++++ libxfs/xfs_fs.h | 5 ++ libxfs/xfs_ialloc.c | 3 + man/man8/xfs_io.8 | 6 ++- 10 files changed, 220 insertions(+), 9 deletions(-)
PATCH 1/5 xfs: track deferred ops statistics
From: Darrick J. Wong <[email protected]>
Track some basic statistics on how hard we're pushing the defer ops.
--- include/xfs_trans.h | 4 ++++ libxfs/xfs_defer.c | 14 ++++++++++++++ 2 files changed, 18 insertions(+)
include/xfs_trans.h +4 -0
@@ -82,6 +82,10 @@ typedef struct xfs_trans {long t_frextents_delta;/* superblock freextents chg*/ struct list_head t_items;/* log item descriptors */ struct list_head t_dfops;/* deferred operations */ + +unsigned int t_dfops_nr; +unsigned int t_dfops_nr_max; +unsigned int t_dfops_finished; } xfs_trans_t;void xfs_trans_init (struct xfs_mount *);
libxfs/xfs_defer.c +14 -0
@@ -611,6 +611,8 @@ xfs_defer_finish_one(/* Done with the dfp, free it. */ list_del (&dfp->dfp_list);kmem_cache_free (xfs_defer_pending_cache, dfp); + tp->t_dfops_nr--; + tp->t_dfops_finished++; out:if (ops->finish_cleanup) ops->finish_cleanup (tp, state, error);
@@ -673,6 +675,9 @@ xfs_defer_finish_noroll(list_splice_init (&(*tp)->t_dfops, &dop_pending); + (*tp)->t_dfops_nr_max =max ((*tp)->t_dfops_nr, + (*tp)->t_dfops_nr_max); +if (has_intents <0 ) { error = has_intents; goto out_shutdown;
@@ -714,6 +719,7 @@ xfs_defer_finish_noroll(xfs_force_shutdown ((*tp)->t_mountp,SHUTDOWN_CORRUPT_INCORE );trace_xfs_defer_finish_error (*tp, error);xfs_defer_cancel_list ((*tp)->t_mountp, &dop_pending); + (*tp)->t_dfops_nr =0 ;xfs_defer_cancel (*tp);return error; }
@@ -761,6 +767,7 @@ xfs_defer_cancel(trace_xfs_defer_cancel (tp, _RET_IP_);xfs_defer_trans_abort (tp, &tp->t_dfops);xfs_defer_cancel_list (mp, &tp->t_dfops); + tp->t_dfops_nr =0 ; }/*
@@ -824,6 +831,7 @@ xfs_defer_alloc( dfp->dfp_ops = ops;INIT_LIST_HEAD (&dfp->dfp_work);list_add_tail (&dfp->dfp_list, &tp->t_dfops); + tp->t_dfops_nr++;return dfp; }
@@ -936,6 +944,12 @@ xfs_defer_move( struct xfs_trans *stp) {list_splice_init (&stp->t_dfops, &dtp->t_dfops); + dtp->t_dfops_nr += stp->t_dfops_nr; + dtp->t_dfops_nr_max = stp->t_dfops_nr_max; + dtp->t_dfops_finished = stp->t_dfops_finished; + stp->t_dfops_nr =0 ; + stp->t_dfops_nr_max =0 ; + stp->t_dfops_finished =0 ;/* * Low free space mode was historically controlled by a dfops field.
PATCH 2/5 xfs: create a noalloc mode for allocation groups
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new noalloc state for the per-AG structure that will disable block allocation in this AG. We accomplish this by subtracting from fdblocks all the free blocks in this AG, hiding those blocks from the allocator, and preventing freed blocks from updating fdblocks until we're ready to lift noalloc mode.
Note that we reduce the free block count of the filesystem so that we can prevent transactions from entering the allocator looking for "free" space that we've turned off incore.
--- include/xfs_trace.h | 2 ++ libxfs/xfs_ag.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_ag.h | 8 +++++++ libxfs/xfs_ag_resv.c | 28 +++++++++++++++++++++-- 4 files changed, 95 insertions(+), 3 deletions(-)
include/xfs_trace.h +2 -0
@@ -13,6 +13,8 @@#define trace_xfbtree_trans_cancel_buf (...) ((void) 0)#define trace_xfbtree_trans_commit_buf (...) ((void) 0) +#define trace_xfs_ag_clear_noalloc (a) ((void) 0) +#define trace_xfs_ag_set_noalloc (a) ((void) 0)#define trace_xfs_agfl_free_defer (...) ((void) 0)#define trace_xfs_agfl_reset (a,b,c,d) ((void) 0)#define trace_xfs_alloc_cur_check (a,b,c,d,e,f) ((void) 0)
libxfs/xfs_ag.c +60 -0
63 lines not on this page · raw message
libxfs/xfs_ag.h +8 -0
@@ -131,6 +131,7 @@ struct xfs_perag {#define XFS_AGSTATE_PREFERS_METADATA 2#define XFS_AGSTATE_ALLOWS_INODES 3#define XFS_AGSTATE_AGFL_NEEDS_RESET 4 +#define XFS_AGSTATE_NOALLOC 5#define __XFS_AG_OPSTATE (name,NAME ) \ static inline bool xfs_perag_ ## name (struct xfs_perag *pag) \
@@ -143,6 +144,7 @@ __XFS_AG_OPSTATE(initialised_agi, AGI_INIT) __XFS_AG_OPSTATE(prefers_metadata, PREFERS_METADATA) __XFS_AG_OPSTATE(allows_inodes, ALLOWS_INODES) __XFS_AG_OPSTATE(agfl_needs_reset, AGFL_NEEDS_RESET) +__XFS_AG_OPSTATE(prohibits_alloc, NOALLOC)int xfs_initialize_perag (struct xfs_mount *mp,xfs_agnumber_t agcount,xfs_rfsblock_t dcount,xfs_agnumber_t *maxagi);
@@ -156,12 +158,18 @@ struct xfs_perag *xfs_perag_get_tag(struct xfs_mount *mp, xfs_agnumber_t agno,struct xfs_perag *xfs_perag_hold (struct xfs_perag *pag);void xfs_perag_put (struct xfs_perag *pag); +/* Active AG references */ struct xfs_perag *xfs_perag_grab (struct xfs_mount *,xfs_agnumber_t );struct xfs_perag *xfs_perag_grab_tag (struct xfs_mount *,xfs_agnumber_t ,int tag);void xfs_perag_rele (struct xfs_perag *pag); +/* Enable or disable allocation from an AG */ +xfs_extlen_t xfs_ag_fdblocks (struct xfs_perag *pag); +int xfs_ag_set_noalloc (struct xfs_perag *pag); +void xfs_ag_clear_noalloc (struct xfs_perag *pag); +/* * Per-ag geometry infomation and validation */
libxfs/xfs_ag_resv.c +25 -3
@@ -20,6 +20,7 @@#include "xfs_ialloc_btree.h" #include "xfs_ag.h" #include "xfs_ag_resv.h" +#include "xfs_ag.h" /* * Per-AG Block Reservations
@@ -72,6 +73,13 @@ xfs_ag_resv_critical(xfs_extlen_t avail;xfs_extlen_t orig; +/* +* Pretend we're critically low on reservations in this AG to scare +* everyone else away. +*/ +if (xfs_perag_prohibits_alloc (pag)) +return true; +switch (type) {case XFS_AG_RESV_METADATA : avail = pag->pagf_freeblks - pag->pag_rmapbt_resv.ar_reserved;
@@ -114,7 +122,12 @@ xfs_ag_resv_needed(break ;case XFS_AG_RESV_IMETA :case XFS_AG_RESV_NONE : -/* empty */ +/* +* In noalloc mode, we pretend that all the free blocks in this +* AG have been allocated. Make this AG look full. +*/ +if (xfs_perag_prohibits_alloc (pag)) + len +=xfs_ag_fdblocks (pag);break ;default :ASSERT (0 );
@@ -343,6 +356,8 @@ xfs_ag_resv_alloc_extent(xfs_extlen_t len;uint field; +ASSERT (type !=XFS_AG_RESV_NONE || !xfs_perag_prohibits_alloc (pag)); +trace_xfs_ag_resv_alloc_extent (pag, type, args->len);switch (type) {
@@ -400,7 +415,14 @@ xfs_ag_resv_free_extent(ASSERT (0 ); fallthrough;case XFS_AG_RESV_NONE : -xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , (int64_t )len); +/* +* Normally we put freed blocks back into fdblocks. In noalloc +* mode, however, we pretend that there are no fdblocks in the +* AG, so don't put them back. +*/ +if (!xfs_perag_prohibits_alloc (pag)) +xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , + (int64_t )len); fallthrough;case XFS_AG_RESV_IGNORE :return ;
@@ -413,6 +435,6 @@ xfs_ag_resv_free_extent(/* Freeing into the reserved pool only requires on-disk update... */ xfs_trans_mod_sb (tp,XFS_TRANS_SB_RES_FDBLOCKS , len);/* ...but freeing beyond that requires in-core and on-disk update. */ -if (len > leftover) +if (len > leftover && !xfs_perag_prohibits_alloc (pag))xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , len - leftover); }
PATCH 3/5 xfs: enable userspace to hide an AG from allocation
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add an administrative interface so that userspace can hide an allocation group from block allocation.
--- libxfs/xfs_ag.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_fs.h | 5 +++++ 2 files changed, 59 insertions(+)
libxfs/xfs_ag.c +54 -0
@@ -1073,6 +1073,54 @@ xfs_ag_extend_space(return 0 ; } +/* Compute the AG geometry flags. */ +static inline uint32_t +xfs_ag_calc_geoflags ( +struct xfs_perag *pag) +{ +uint32_t ret =0 ; + +if (xfs_perag_prohibits_alloc (pag)) + ret |=XFS_AG_FLAG_NOALLOC ; + +return ret; +} + +/* +* Compare the current AG geometry flags against the flags in the AG geometry +* structure and update the AG state to reflect any changes, then update the +* struct to reflect the current status. +*/ +static inline int +xfs_ag_update_geoflags ( +struct xfs_perag *pag, +struct xfs_ag_geometry *ageo, +uint32_t new_flags) +{ +uint32_t old_flags =xfs_ag_calc_geoflags (pag); +int error; + +if (!(new_flags &XFS_AG_FLAG_UPDATE )) { + ageo->ag_flags = old_flags; +return 0 ; + } + +if ((old_flags &XFS_AG_FLAG_NOALLOC ) && + !(new_flags &XFS_AG_FLAG_NOALLOC )) { +xfs_ag_clear_noalloc (pag); + } + +if (!(old_flags &XFS_AG_FLAG_NOALLOC ) && + (new_flags &XFS_AG_FLAG_NOALLOC )) { + error =xfs_ag_set_noalloc (pag); +if (error) +return error; + } + + ageo->ag_flags =xfs_ag_calc_geoflags (pag); +return 0 ; +} +/* Retrieve AG geometry. */ int xfs_ag_get_geometry(
@@ -1084,6 +1132,7 @@ xfs_ag_get_geometry(struct xfs_agi *agi;struct xfs_agf *agf;unsigned int freeblks; +uint32_t inflags = ageo->ag_flags;int error;/* Lock the AG headers. */
@@ -1094,6 +1143,10 @@ xfs_ag_get_geometry(if (error) goto out_agi; + error =xfs_ag_update_geoflags (pag, ageo, inflags); +if (error) + goto out; +/* Fill out form. */ memset (ageo,0 ,sizeof (*ageo)); ageo->ag_number = pag->pag_agno;
@@ -1111,6 +1164,7 @@ xfs_ag_get_geometry( ageo->ag_freeblks = freeblks;xfs_ag_geom_health (pag, ageo); +out:/* Release resources. */ xfs_buf_relse (agf_bp); out_agi:
libxfs/xfs_fs.h +5 -0
@@ -305,6 +305,11 @@ struct xfs_ag_geometry {#define XFS_AG_GEOM_SICK_REFCNTBT (1 << 9)/* reference counts */ #define XFS_AG_GEOM_SICK_INODES (1 << 10)/* bad inodes were seen */ +#define XFS_AG_FLAG_UPDATE (1 << 0)/* update flags */ +#define XFS_AG_FLAG_NOALLOC (1 << 1)/* do not allocate from this AG */ +#define XFS_AG_FLAG_ALL (XFS_AG_FLAG_UPDATE | \ + XFS_AG_FLAG_NOALLOC) +/* * Structures for XFS_IOC_FSGROWFSDATA, XFS_IOC_FSGROWFSLOG & XFS_IOC_FSGROWFSRT */
PATCH 4/5 xfs: apply noalloc mode to inode allocations too
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Don't allow inode allocations from this group if it's marked noalloc.
--- libxfs/xfs_ialloc.c | 3 +++ 1 file changed, 3 insertions(+)
libxfs/xfs_ialloc.c +3 -0
@@ -1059,6 +1059,7 @@ xfs_dialloc_ag_inobt(ASSERT (xfs_perag_initialised_agi (pag));ASSERT (xfs_perag_allows_inodes (pag)); +ASSERT (!xfs_perag_prohibits_alloc (pag));ASSERT (pag->pagi_freecount >0 ); restart_pagno:
@@ -1687,6 +1688,8 @@ xfs_dialloc_good_ag(return false;if (!xfs_perag_allows_inodes (pag))return false; +if (xfs_perag_prohibits_alloc (pag)) +return false;if (!xfs_perag_initialised_agi (pag)) { error =xfs_ialloc_read_agi (pag, tp,NULL );
PATCH 5/5 xfs_io: enhance the aginfo command to control the noalloc flag
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Augment the aginfo command to be able to set and clear the noalloc state for an AG.
--- io/aginfo.c | 45 ++++++++++++++++++++++++++++++++++++++++----- man/man8/xfs_io.8 | 6 +++++- 2 files changed, 45 insertions(+), 6 deletions(-)
io/aginfo.c +40 -5
@@ -19,9 +19,11 @@ static cmdinfo_t rginfo_cmd;static int report_aginfo (struct xfs_fd *xfd, -xfs_agnumber_t agno) +xfs_agnumber_t agno, +int oflag) {struct xfs_ag_geometry ageo = {0 }; +bool update = false;int ret; ret = -xfrog_ag_geometry (xfd->fd, agno, &ageo);
@@ -30,6 +32,26 @@ report_aginfo(return 1 ; } +switch (oflag) { +case 0 : + ageo.ag_flags |=XFS_AG_FLAG_UPDATE ; + ageo.ag_flags &= ~XFS_AG_FLAG_NOALLOC ; + update = true; +break ; +case 1 : + ageo.ag_flags |= (XFS_AG_FLAG_UPDATE |XFS_AG_FLAG_NOALLOC ); + update = true; +break ; + } + +if (update) { + ret = -xfrog_ag_geometry (xfd->fd, agno, &ageo); +if (ret) { +xfrog_perror (ret,"aginfo update" ); +return 1 ; + } + } +printf (_ ("AG: %u\n" ), ageo.ag_number);printf (_ ("Blocks: %u\n" ), ageo.ag_length);printf (_ ("Free Blocks: %u\n" ), ageo.ag_freeblks);
@@ -51,6 +73,7 @@ aginfo_f(struct xfs_fd xfd =XFS_FD_INIT (file->fd);unsigned long long x;xfs_agnumber_t agno =NULLAGNUMBER ; +int oflag =-1 ;int c;int ret =0 ;
@@ -61,7 +84,7 @@ aginfo_f(return 1 ; } -while ((c =getopt (argc, argv,"a:" )) !=EOF ) { +while ((c =getopt (argc, argv,"a:o:" )) !=EOF ) {switch (c) {case 'a' : errno =0 ;
@@ -74,16 +97,27 @@ aginfo_f( } agno = x;break ; +case 'o' : + errno =0 ; + x =strtoll (optarg,NULL ,10 ); +if (!errno && x !=0 && x !=1 ) + errno =ERANGE ; +if (errno) { +perror ("aginfo" ); +return 1 ; + } + oflag = x; +break ;default :return command_usage (&aginfo_cmd); } }if (agno !=NULLAGNUMBER ) { - ret =report_aginfo (&xfd, agno); + ret =report_aginfo (&xfd, agno, oflag); }else {for (agno =0 ; !ret && agno < xfd.fsgeom.agcount; agno++) { - ret =report_aginfo (&xfd, agno); + ret =report_aginfo (&xfd, agno, oflag); } }
@@ -98,6 +132,7 @@ aginfo_help(void)"Report allocation group geometry.\n" "\n" " -a agno -- Report on the given allocation group.\n" +" -o state -- Change the NOALLOC state for this allocation group.\n" "\n")); }
@@ -107,7 +142,7 @@ static cmdinfo_t aginfo_cmd = { .cfunc = aginfo_f, .argmin = 0, .argmax = -1, - .args = "[-a agno]", + .args = "[-a agno] [-o state]", .flags = CMD_NOMAP_OK, .help = aginfo_help, };
man/man8/xfs_io.8 +5 -1
@@ -1240,7 +1240,7 @@ for the current memory mapping. .SH FILESYSTEM COMMANDS .TP -.BI "aginfo [ \-a " agno " ]" +.BI "aginfo [ \-a " agno " ] [ \-o " nr " ]" Show information about or update the state of allocation groups. .RE .RS 1.0i
@@ -1248,6 +1248,10 @@ Show information about or update the state of allocation groups. .TP .BI \-a Act only on a specific allocation group. +.TP +.BI \-o +If 0, clear the NOALLOC flag. +If 1, set the NOALLOC flag. .PD .RE
PATCH 2/3 xfsprogs: report refcount information to userspace
In reply to Darrick J. WongHi all,
Create a new ioctl to report the number of owners of each disk block so that reflink-aware defraggers can make better decisions about which extents to target.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=report-refcounts
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=report-refcounts
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=report-refcounts
--- io/Makefile | 2 io/fsrefcounts.c | 477 +++++++++++++++++++++++++++++++++++ io/init.c | 1 io/io.h | 1 libxfs/xfs_fs_staging.h | 82 ++++++ man/man2/ioctl_xfs_getfsrefcounts.2 | 239 ++++++++++++++++++ man/man8/xfs_io.8 | 88 ++++++ 7 files changed, 889 insertions(+), 1 deletion(-) create mode 100644 io/fsrefcounts.c create mode 100644 man/man2/ioctl_xfs_getfsrefcounts.2
PATCH 1/2 xfs: export reference count information to userspace
From: Darrick J. Wong <[email protected]>
Export refcount info to userspace so we can prototype a sharing-aware defrag/fs rearranging tool.
--- libxfs/xfs_fs_staging.h | 82 ++++++++++++ man/man2/ioctl_xfs_getfsrefcounts.2 | 239 +++++++++++++++++++++++++++++++++++ 2 files changed, 321 insertions(+) create mode 100644 man/man2/ioctl_xfs_getfsrefcounts.2
libxfs/xfs_fs_staging.h +82 -0
@@ -221,4 +221,86 @@ struct xfs_rtgroup_geometry {#define XFS_IOC_RTGROUP_GEOMETRY _IOWR('X', 63, struct xfs_rtgroup_geometry) +/* +* Structure for XFS_IOC_GETFSREFCOUNTS. +* +* The memory layout for this call are the scalar values defined in struct +* xfs_getfsrefs_head, followed by two struct xfs_getfsrefs that describe +* the lower and upper bound of mappings to return, followed by an array +* of struct xfs_getfsrefs mappings. +* +* fch_iflags control the output of the call, whereas fch_oflags report +* on the overall record output. fch_count should be set to the length +* of the fch_recs array, and fch_entries will be set to the number of +* entries filled out during each call. If fch_count is zero, the number +* of refcount mappings will be returned in fch_entries, though no +* mappings will be returned. fch_reserved must be set to zero. +* +* The two elements in the fch_keys array are used to constrain the +* output. The first element in the array should represent the lowest +* disk mapping ("low key") that the user wants to learn about. If this +* value is all zeroes, the filesystem will return the first entry it +* knows about. For a subsequent call, the contents of +* fsrefs_head.fch_recs[fsrefs_head.fch_count - 1] should be copied into +* fch_keys[0] to have the kernel start where it left off. +* +* The second element in the fch_keys array should represent the highest +* disk mapping ("high key") that the user wants to learn about. If this +* value is all ones, the filesystem will not stop until it runs out of +* mapping to return or runs out of space in fch_recs. +* +* fcr_device can be either a 32-bit cookie representing a device, or a +* 32-bit dev_t if the FCH_OF_DEV_T flag is set. fcr_physical and +* fcr_length are expressed in units of bytes. fcr_owners is the number +* of owners. +*/ +struct xfs_getfsrefs { +__u32 fcr_device;/* device id */ +__u32 fcr_flags;/* mapping flags */ +__u64 fcr_physical;/* device offset of segment */ +__u64 fcr_owners;/* number of owners */ +__u64 fcr_length;/* length of segment */ +__u64 fcr_reserved[4 ];/* must be zero */ +}; + +struct xfs_getfsrefs_head { +__u32 fch_iflags;/* control flags */ +__u32 fch_oflags;/* output flags */ +__u32 fch_count;/* # of entries in array incl. input */ +__u32 fch_entries;/* # of entries filled in (output). */ +__u64 fch_reserved[6 ];/* must be zero */ + +struct xfs_getfsrefs fch_keys[2 ];/* low and high keys for the mapping search */ +struct xfs_getfsrefs fch_recs[];/* returned records */ +}; + +/* Size of an fsrefs_head with room for nr records. */ +static inline unsigned long long +xfs_getfsrefs_sizeof ( +unsigned int nr) +{ +return sizeof (struct xfs_getfsrefs_head ) + nr *sizeof (struct xfs_getfsrefs ); +} + +/* Start the next fsrefs query at the end of the current query results. */ +static inline void +xfs_getfsrefs_advance ( +struct xfs_getfsrefs_head *head) +{ + head->fch_keys[0 ] = head->fch_recs[head->fch_entries -1 ]; +} + +/* fch_iflags values - set by XFS_IOC_GETFSREFCOUNTS caller in the header. */ +/* no flags defined yet */ +#define FCH_IF_VALID 0 + +/* fch_oflags values - returned in the header segment only. */ +#define FCH_OF_DEV_T (1U << 0)/* fcr_device values will be dev_t */ + +/* fcr_flags values - returned for each non-header segment */ +#define FCR_OF_LAST (1U << 0)/* segment is the last in the dataset */ + +/* XXX stealing XFS_IOC_GETBIOSIZE */ +#define XFS_IOC_GETFSREFCOUNTS _IOWR('X', 47, struct xfs_getfsrefs_head) +#endif /* __XFS_FS_STAGING_H__ */
man/man2/ioctl_xfs_getfsrefcounts.2 new file +239 -0
239 lines not on this page · raw message
PATCH 2/2 xfs_io: dump reference count information
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Dump refcount info from the kernel so we can prototype a sharing-aware defrag/fs rearranging tool.
--- io/Makefile | 2 io/fsrefcounts.c | 477 +++++++++++++++++++++++++++++++++++++++++++++++++++++ io/init.c | 1 io/io.h | 1 man/man8/xfs_io.8 | 88 ++++++++++ 5 files changed, 568 insertions(+), 1 deletion(-) create mode 100644 io/fsrefcounts.c
io/Makefile +1 -1
@@ -13,7 +13,7 @@ CFILES = init.c aginfo.c \ file.c freeze.c fsuuid.c fsync.c getrusage.c imap.c inject.c label.c \ link.c mmap.c open.c parent.c pread.c prealloc.c pwrite.c reflink.c \ resblks.c scrub.c seek.c shutdown.c stat.c swapext.c sync.c \ - truncate.c utimes.c atomicupdate.c + truncate.c utimes.c atomicupdate.c fsrefcounts.c LLDLIBS = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG) $(LIBPTHREAD) $(LIBUUID) LTDEPENDENCIES = $(LIBXCMD) $(LIBHANDLE) $(LIBFROG)
io/fsrefcounts.c new file +477 -0
477 lines not on this page · raw message
io/init.c +1 -0
@@ -59,6 +59,7 @@ init_commands(void)freeze_init ();fsmap_init ();fsuuid_init (); +fsrefcounts_init ();fsync_init ();getrusage_init ();help_init ();
io/io.h +1 -0
@@ -191,3 +191,4 @@ extern void crc32cselftest_init(void);extern void bulkstat_init (void );extern void atomicupdate_init (void );extern void aginfo_init (void ); +extern void fsrefcounts_init (void );
man/man8/xfs_io.8 +88 -0
94 lines not on this page · raw message
PATCH 3/3 xfsprogs: defragment free space
In reply to Darrick J. WongHi all,
These patches contain experimental code to enable userspace to defragment the free space in a filesystem. Two purposes are imagined for this
functionality: clearing space at the end of a filesystem before shrinking it, and clearing free space in anticipation of making a large allocation.
The first patch adds a new fallocate mode that allows userspace to allocate free space from the filesystem into a file. The goal here is to allow the filesystem shrink process to prevent allocation from a certain part of the filesystem while a free space defragmenter evacuates all the files from the doomed part of the filesystem.
The second patch amends the online repair system to allow the sysadmin to forcibly rebuild metadata structures, even if they're not corrupt. Without adding an ioctl to move metadata btree blocks, this is the only way to dislodge metadata.
This patchset also includes a separate inode migration tool as prototyped by Dave Chinner in 2020.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=defrag-freespace
--- Makefile | 2 configure.ac | 1 db/agfl.c | 297 ++++ include/builddefs.in | 1 include/xfs_trace.h | 4 io/prealloc.c | 35 libfrog/Makefile | 5 libfrog/clearspace.c | 3103 +++++++++++++++++++++++++++++++++++++++ libfrog/clearspace.h | 72 + libfrog/fsgeom.h | 29 libfrog/radix-tree.c | 2 libfrog/radix-tree.h | 2 libxfs/libxfs_api_defs.h | 4 libxfs/libxfs_priv.h | 10 libxfs/xfs_alloc.c | 88 + libxfs/xfs_alloc.h | 3 libxfs/xfs_bmap.c | 149 ++ libxfs/xfs_bmap.h | 3 libxfs/xfs_fs.h | 1 libxfs/xfs_fs_staging.h | 15 m4/package_libcdev.m4 | 20 man/man2/ioctl_xfs_map_freesp.2 | 76 + man/man8/xfs_db.8 | 11 man/man8/xfs_io.8 | 8 man/man8/xfs_spaceman.8 | 40 + spaceman/Makefile | 10 spaceman/clearfree.c | 164 ++ spaceman/find_owner.c | 443 ++++++ spaceman/init.c | 7 spaceman/move_inode.c | 663 ++++++++ spaceman/relocation.c | 566 +++++++ spaceman/relocation.h | 53 + spaceman/space.h | 6 33 files changed, 5883 insertions(+), 10 deletions(-) create mode 100644 libfrog/clearspace.c create mode 100644 libfrog/clearspace.h create mode 100644 man/man2/ioctl_xfs_map_freesp.2 create mode 100644 spaceman/clearfree.c create mode 100644 spaceman/find_owner.c create mode 100644 spaceman/move_inode.c create mode 100644 spaceman/relocation.c create mode 100644 spaceman/relocation.h
PATCH 1/10 xfs: add an ioctl to map free space into a file
From: Darrick J. Wong <[email protected]>
Add a new ioctl to map free physical space into a file, at the same file offset as if the file were a sparse image of the physical device backing the filesystem. The intent here is to use this to prototype a free space defragmentation tool.
--- include/xfs_trace.h | 4 + libxfs/libxfs_priv.h | 10 +++ libxfs/xfs_alloc.c | 88 +++++++++++++++++++++++ libxfs/xfs_alloc.h | 3 + libxfs/xfs_bmap.c | 149 +++++++++++++++++++++++++++++++++++++++ libxfs/xfs_bmap.h | 3 + libxfs/xfs_fs.h | 1 libxfs/xfs_fs_staging.h | 15 ++++ man/man2/ioctl_xfs_map_freesp.2 | 76 ++++++++++++++++++++ 9 files changed, 349 insertions(+) create mode 100644 man/man2/ioctl_xfs_map_freesp.2
include/xfs_trace.h +4 -0
@@ -26,6 +26,8 @@#define trace_xfs_alloc_exact_done (a) ((void) 0)#define trace_xfs_alloc_exact_notfound (a) ((void) 0)#define trace_xfs_alloc_exact_error (a) ((void) 0) +#define trace_xfs_alloc_find_freesp (...) ((void) 0) +#define trace_xfs_alloc_find_freesp_done (...) ((void) 0)#define trace_xfs_alloc_near_first (a) ((void) 0)#define trace_xfs_alloc_near_greater (a) ((void) 0)#define trace_xfs_alloc_near_lesser (a) ((void) 0)
@@ -196,6 +198,8 @@#define trace_xfs_bmap_pre_update (a,b,c,d) ((void) 0)#define trace_xfs_bmap_post_update (a,b,c,d) ((void) 0) +#define trace_xfs_bmapi_freesp (...) ((void) 0) +#define trace_xfs_bmapi_freesp_done (...) ((void) 0)#define trace_xfs_bunmap (a,b,c,d,e) ((void) 0)#define trace_xfs_read_extent (a,b,c,d) ((void) 0)
libxfs/libxfs_priv.h +10 -0
@@ -501,6 +501,16 @@ void __xfs_buf_mark_corrupt(struct xfs_buf *bp, xfs_failaddr_t fa);#define xfs_filestream_new_ag (ip,ag) (0)#define xfs_filestream_select_ag (...) (-ENOSYS) +struct xfs_trans ; + +static inline int +xfs_rtallocate_extent (struct xfs_trans *tp,xfs_rtblock_t bno, +xfs_extlen_t minlen,xfs_extlen_t maxlen,xfs_extlen_t *len, +int wasdel,xfs_extlen_t prod,xfs_rtblock_t *rtblock) +{ +return -EOPNOTSUPP ; +} +#define xfs_trans_inode_buf (tp, bp) ((void) 0)/* quota bits */
libxfs/xfs_alloc.c +88 -0
91 lines not on this page · raw message
libxfs/xfs_alloc.h +3 -0
@@ -290,5 +290,8 @@ void xfs_extfree_intent_destroy_cache(void);xfs_failaddr_t xfs_validate_ag_length (struct xfs_buf *bp,uint32_t seqno,uint32_t length); +int xfs_alloc_find_freesp (struct xfs_trans *tp,struct xfs_perag *pag, +xfs_agblock_t *agbno,xfs_agblock_t end_agbno, +xfs_extlen_t *len);#endif /* __XFS_ALLOC_H__ */
libxfs/xfs_bmap.c +149 -0
152 lines not on this page · raw message
libxfs/xfs_bmap.h +3 -0
@@ -197,6 +197,9 @@ int xfs_bmapi_read(struct xfs_inode *ip, xfs_fileoff_t bno,int xfs_bmapi_write (struct xfs_trans *tp,struct xfs_inode *ip,xfs_fileoff_t bno,xfs_filblks_t len,uint32_t flags,xfs_extlen_t total,struct xfs_bmbt_irec *mval,int *nmap); +int xfs_bmapi_freesp (struct xfs_trans *tp,struct xfs_inode *ip, +xfs_fsblock_t fsbno,xfs_extlen_t len, +struct xfs_bmbt_irec *mval);int xfs_bunmapi (struct xfs_trans *tp,struct xfs_inode *ip,xfs_fileoff_t bno,xfs_filblks_t len,uint32_t flags,xfs_extnum_t nexts,int *done);
libxfs/xfs_fs.h +1 -0
@@ -864,6 +864,7 @@ struct xfs_scrub_metadata {#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry)/* XFS_IOC_GETPARENTS ---- staging 62 */ /* XFS_IOC_RTGROUP_GEOMETRY - staging 63 */ +/* XFS_IOC_MAP_FREESP ---- staging 64 */ /* * ioctl commands that replace IRIX syssgi()'s
libxfs/xfs_fs_staging.h +15 -0
@@ -303,4 +303,19 @@ xfs_getfsrefs_advance(/* XXX stealing XFS_IOC_GETBIOSIZE */ #define XFS_IOC_GETFSREFCOUNTS _IOWR('X', 47, struct xfs_getfsrefs_head) +/* map free space to file */ + +struct xfs_map_freesp { +__s64 offset;/* disk address to map, in bytes */ +__s64 len;/* length in bytes */ +__u64 flags;/* must be zero */ +__u64 pad;/* must be zero */ +}; + +/* +* XFS_IOC_MAP_FREESP maps all the free physical space in the filesystem into +* the file at the same offsets. This ioctl requires CAP_SYS_ADMIN. +*/ +#define XFS_IOC_MAP_FREESP _IOWR('X', 64, struct xfs_map_freesp) +#endif /* __XFS_FS_STAGING_H__ */
man/man2/ioctl_xfs_map_freesp.2 new file +76 -0
76 lines not on this page · raw message
PATCH 2/10 xfs_io: support using XFS_IOC_MAP_FREESP to map free space
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a command to call XFS_IOC_MAP_FREESP. This is experimental code to see if we can build a free space defragmenter out of this.
--- io/prealloc.c | 35 +++++++++++++++++++++++++++++++++++ man/man8/xfs_io.8 | 8 +++++++- 2 files changed, 42 insertions(+), 1 deletion(-)
io/prealloc.c +35 -0
@@ -45,6 +45,7 @@ static cmdinfo_t finsert_cmd;static cmdinfo_t fzero_cmd;static cmdinfo_t funshare_cmd;#endif +static cmdinfo_t fmapfree_cmd;static int offset_length(
@@ -383,6 +384,30 @@ funshare_f( }#endif /* HAVE_FALLOCATE */ +static int +fmapfree_f ( +int argc, +char **argv) +{ +struct xfs_flock64 segment; +struct xfs_map_freesp args = { }; + +if (!offset_length (argv[1 ], argv[2 ], &segment)) { + exitcode =1 ; +return 0 ; + } + + args.offset = segment.l_start; + args.len = segment.l_len; + +if (ioctl (file->fd,XFS_IOC_MAP_FREESP , &args)) { +perror ("XFS_IOC_MAP_FREESP" ); + exitcode =1 ; +return 0 ; + } +return 0 ; +} +void prealloc_init (void ) {
@@ -497,4 +522,14 @@ prealloc_init(void)_ ("unshares shared blocks within the range" );add_command (&funshare_cmd);#endif /* HAVE_FALLOCATE */ + + fmapfree_cmd.name ="fmapfree" ; + fmapfree_cmd.cfunc = fmapfree_f; + fmapfree_cmd.argmin =2 ; + fmapfree_cmd.argmax =2 ; + fmapfree_cmd.flags =CMD_NOMAP_OK |CMD_FOREIGN_OK ; + fmapfree_cmd.args =_ ("off len" ); + fmapfree_cmd.oneline = +_ ("maps free space into a file" ); +add_command (&fmapfree_cmd); }
man/man8/xfs_io.8 +7 -1
@@ -513,8 +513,14 @@ Call fallocate with FALLOC_FL_INSERT_RANGE flag as described in the .BR fallocate (2) manual page to create the hole by shifting data blocks. .TP +.BI fmapfree " offset length" +Maps free physical space into the file by calling XFS_IOC_MAP_FREESP as +described in the +.BR XFS_IOC_MAP_FREESP (2) +manual page. +.TP .BI fpunch " offset length" -Punches (de-allocates) blocks in the file by calling fallocate with +Punches (de-allocates) blocks in the file by calling fallocate with the FALLOC_FL_PUNCH_HOLE flag as described in the .BR fallocate (2) manual page.
PATCH 3/10 xfs_db: get and put blocks on the AGFL
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new xfs_db command to let people add and remove blocks from an AGFL. This isn't really related to rmap btree reconstruction, other than enabling debugging code to mess around with the AGFL to exercise various odd scenarios.
--- db/agfl.c | 297 ++++++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_api_defs.h | 4 + man/man8/xfs_db.8 | 11 ++ 3 files changed, 308 insertions(+), 4 deletions(-)
db/agfl.c +294 -3
@@ -15,13 +15,14 @@#include "output.h" #include "init.h" #include "agfl.h" +#include "libfrog/bitmap.h" static int agfl_bno_size (void *obj,int startoff);static int agfl_f (int argc,char **argv);static void agfl_help (void );static const cmdinfo_t agfl_cmd = - {"agfl" ,NULL , agfl_f,0 ,1 ,1 ,N_ ("[agno]" ), + {"agfl" ,NULL , agfl_f,0 ,-1 ,1 ,N_ ("[agno] [-g nr] [-p nr]" ),N_ ("set address to agfl block" ), agfl_help };const field_t agfl_hfld[] = { {
@@ -77,10 +78,280 @@ agfl_help(void)" for each allocation group. This acts as a reserved pool of space\n" " separate from the general filesystem freespace (not used for user data).\n" "\n" +" -g quantity\tRemove this many blocks from the AGFL.\n" +" -p quantity\tAdd this many blocks to the AGFL.\n" +"\n" )); } +struct dump_info { +struct xfs_perag *pag; +bool leak; +}; + +/* Return blocks freed from the AGFL to the free space btrees. */ +static int +free_grabbed ( +uint64_t start, +uint64_t length, +void *data) +{ +struct dump_info *di = data; +struct xfs_perag *pag = di->pag; +struct xfs_mount *mp = pag->pag_mount; +struct xfs_trans *tp; +struct xfs_buf *agf_bp; +int error; + + error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate,0 ,0 ,0 , + &tp); +if (error) +return error; + + error = -libxfs_alloc_read_agf (pag, tp,0 , &agf_bp); +if (error) + goto out_cancel; + + error = -libxfs_free_extent (tp, pag, start, length, &XFS_RMAP_OINFO_AG , +XFS_AG_RESV_AGFL ); +if (error) + goto out_cancel; + +return -libxfs_trans_commit (tp); + +out_cancel: +libxfs_trans_cancel (tp); +return error; +} + +/* Report blocks freed from the AGFL. */ +static int +dump_grabbed ( +uint64_t start, +uint64_t length, +void *data) +{ +struct dump_info *di = data; +const char *fmt; + +if (length ==1 ) + fmt = di->leak ?_ ("agfl %u: leaked agbno %u\n" ) : +_ ("agfl %u: removed agbno %u\n" ); +else + fmt = di->leak ?_ ("agfl %u: leaked agbno %u-%u\n" ) : +_ ("agfl %u: removed agbno %u-%u\n" ); + +printf (fmt, di->pag->pag_agno, (unsigned int )start, + (unsigned int )(start + length -1 )); +return 0 ; +} + +/* Remove blocks from the AGFL. */ +static int +agfl_get ( +struct xfs_perag *pag, +int quantity) +{ +struct dump_info di = { + .pag = pag, + .leak = quantity <0 , + }; +struct xfs_agf *agf; +struct xfs_buf *agf_bp; +struct xfs_trans *tp; +struct bitmap *grabbed; +const unsigned int agfl_size =libxfs_agfl_size (pag->pag_mount); +unsigned int i; +int error; + +if (!quantity) +return 0 ; + +if (di.leak) + quantity = -quantity; + quantity =min (quantity, agfl_size); + + error =bitmap_alloc (&grabbed); +if (error) + goto out; + + error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate, quantity,0 , +0 , &tp); +if (error) + goto out_bitmap; + + error = -libxfs_alloc_read_agf (pag, tp,0 , &agf_bp); +if (error) + goto out_cancel; + + agf = agf_bp->b_addr; + quantity =min (quantity,be32_to_cpu (agf->agf_flcount)); + +for (i =0 ; i < quantity; i++) { +xfs_agblock_t agbno; + + error = -libxfs_alloc_get_freelist (pag, tp, agf_bp, &agbno,0 ); +if (error) + goto out_cancel; + +if (agbno ==NULLAGBLOCK ) { + error =ENOSPC ; + goto out_cancel; + } + + error =bitmap_set (grabbed, agbno,1 ); +if (error) + goto out_cancel; + } + + error = -libxfs_trans_commit (tp); +if (error) + goto out_bitmap; + + error =bitmap_iterate (grabbed, dump_grabbed, &di); +if (error) + goto out_bitmap; + +if (!di.leak) { + error =bitmap_iterate (grabbed, free_grabbed, &di); +if (error) + goto out_bitmap; + } + +bitmap_free (&grabbed); +return 0 ; + +out_cancel: +libxfs_trans_cancel (tp); +out_bitmap: +bitmap_free (&grabbed); +out: +if (error) +printf (_ ("agfl %u: %s\n" ), pag->pag_agno,strerror (error)); +return error; +} + +/* Add blocks to the AGFL. */ +static int +agfl_put ( +struct xfs_perag *pag, +int quantity) +{ +struct xfs_alloc_arg args = { + .mp = pag->pag_mount, + .alignment =1 , + .minlen =1 , + .prod =1 , + .resv =XFS_AG_RESV_AGFL , + .oinfo =XFS_RMAP_OINFO_AG , + }; +struct xfs_buf *agfl_bp; +struct xfs_agf *agf; +struct xfs_trans *tp; +xfs_fsblock_t target; +const unsigned int agfl_size =libxfs_agfl_size (pag->pag_mount); +unsigned int i; +bool eoag = quantity <0 ; +int error; + +if (!quantity) +return 0 ; + +if (eoag) + quantity = -quantity; + quantity =min (quantity, agfl_size); + + error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_itruncate, quantity,0 , +0 , &tp); +if (error) +return error; + args.tp = tp; + + error = -libxfs_alloc_read_agf (pag, tp,0 , &args.agbp); +if (error) + goto out_cancel; + + agf = args.agbp->b_addr; + args.maxlen =min (quantity, agfl_size -be32_to_cpu (agf->agf_flcount)); + +if (eoag) + target =XFS_AGB_TO_FSB (pag->pag_mount, pag->pag_agno, +be32_to_cpu (agf->agf_length) -1 ); +else + target =XFS_AGB_TO_FSB (pag->pag_mount, pag->pag_agno,0 ); + + error = -libxfs_alloc_read_agfl (pag, tp, &agfl_bp); +if (error) + goto out_cancel; + + error = -libxfs_alloc_vextent_near_bno (&args, target); +if (error) + goto out_cancel; + +if (args.agbno ==NULLAGBLOCK ) { + error =ENOSPC ; + goto out_cancel; + } + +for (i =0 ; i < args.len; i++) { + error = -libxfs_alloc_put_freelist (pag, tp, args.agbp, + agfl_bp, args.agbno + i,0 ); +if (error) + goto out_cancel; + } + +if (i ==1 ) +printf (_ ("agfl %u: added agbno %u\n" ), pag->pag_agno, + args.agbno); +else if (i >1 ) +printf (_ ("agfl %u: added agbno %u-%u\n" ), pag->pag_agno, + args.agbno, args.agbno + i -1 ); + + error = -libxfs_trans_commit (tp); +if (error) + goto out; + +return 0 ; + +out_cancel: +libxfs_trans_cancel (tp); +out: +if (error) +printf (_ ("agfl %u: %s\n" ), pag->pag_agno,strerror (error)); +return error; +} + +static void +agfl_adjust ( +struct xfs_mount *mp, +xfs_agnumber_t agno, +int gblocks, +int pblocks) +{ +struct xfs_perag *pag; +int error; + +if (!expert_mode) { +printf (_ ("AGFL get/put only supported in expert mode.\n" )); + exitcode =1 ; +return ; + } + + pag =libxfs_perag_get (mp, agno); + + error =agfl_get (pag, gblocks); +if (error) + goto out_pag; + + error =agfl_put (pag, pblocks); + +out_pag: +libxfs_perag_put (pag); +if (error) + exitcode =1 ; +} + static int agfl_f( int argc,
@@ -88,9 +359,25 @@ agfl_f( {xfs_agnumber_t agno;char *p; +int c; +int gblocks =0 , pblocks =0 ; -if (argc >1 ) { - agno = (xfs_agnumber_t )strtoul (argv[1 ], &p,0 ); +while ((c =getopt (argc, argv,"g:p:" )) !=-1 ) { +switch (c) { +case 'g' : + gblocks =atoi (optarg); +break ; +case 'p' : + pblocks =atoi (optarg); +break ; +default : +agfl_help (); +return 1 ; + } + } + +if (argc > optind) { + agno = (xfs_agnumber_t )strtoul (argv[optind], &p,0 );if (*p !='\0' || agno >= mp->m_sb.sb_agcount) {dbprintf (_ ("bad allocation group number %s\n" ), argv[1 ]);return 0 ;
@@ -98,6 +385,10 @@ agfl_f( cur_agno = agno; }else if (cur_agno ==NULLAGNUMBER ) cur_agno =0 ; + +if (gblocks || pblocks) +agfl_adjust (mp, cur_agno, gblocks, pblocks); +ASSERT (typtab[TYP_AGFL ].typnm ==TYP_AGFL ); set_cur(&typtab[TYP_AGFL], XFS_AG_DADDR(mp, cur_agno, XFS_AGFL_DADDR(mp)),
libxfs/libxfs_api_defs.h +4 -0
@@ -30,8 +30,12 @@#define xfs_allocbt_maxrecs libxfs_allocbt_maxrecs#define xfs_allocbt_stage_cursor libxfs_allocbt_stage_cursor#define xfs_alloc_fix_freelist libxfs_alloc_fix_freelist +#define xfs_alloc_get_freelist libxfs_alloc_get_freelist#define xfs_alloc_min_freelist libxfs_alloc_min_freelist +#define xfs_alloc_put_freelist libxfs_alloc_put_freelist#define xfs_alloc_read_agf libxfs_alloc_read_agf +#define xfs_alloc_read_agfl libxfs_alloc_read_agfl +#define xfs_alloc_vextent_near_bno libxfs_alloc_vextent_near_bno#define xfs_alloc_vextent_start_ag libxfs_alloc_vextent_start_ag#define xfs_ascii_ci_hashname libxfs_ascii_ci_hashname
man/man8/xfs_db.8 +10 -1
@@ -182,10 +182,19 @@ Set current address to the AGF block for allocation group .IR agno . If no argument is given, use the current allocation group. .TP -.BI "agfl [" agno ] +.BI "agfl [" agno "] [\-g " " quantity" "] [\-p " quantity ] Set current address to the AGFL block for allocation group .IR agno . If no argument is given, use the current allocation group. +If the +.B -g +option is specified with a positive quantity, remove that many blocks from the +AGFL and put them in the free space btrees. +If the quantity is negative, remove the blocks and leak them. +If the +.B -p +option is specified, add that many blocks to the AGFL. +If the quantity is negative, the blocks are selected from the end of the AG. .TP .BI "agi [" agno ] Set current address to the AGI block for allocation group
PATCH 4/10 xfs_spaceman: implement clearing free space
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
First attempt at evacuating all the used blocks from part of a filesystem.
--- Makefile | 2 libfrog/Makefile | 5 libfrog/clearspace.c | 3103 +++++++++++++++++++++++++++++++++++++++++++++++ libfrog/clearspace.h | 72 + man/man8/xfs_spaceman.8 | 17 spaceman/Makefile | 2 spaceman/clearfree.c | 164 ++ spaceman/init.c | 1 spaceman/space.h | 2 9 files changed, 3366 insertions(+), 2 deletions(-) create mode 100644 libfrog/clearspace.c create mode 100644 libfrog/clearspace.h create mode 100644 spaceman/clearfree.c
Makefile +1 -1
@@ -97,7 +97,7 @@ quota: libxcmd repair: libxlog libxcmd copy: libxlog mkfs: libxcmd -spaceman: libxcmd +spaceman: libhandle libxcmd scrub: libhandle libxcmd rtcp: libfrog
libfrog/Makefile +5 -0
@@ -64,6 +64,11 @@ ifeq ($(HAVE_GETMNTENT),yes) LCFLAGS += -DHAVE_GETMNTENT endif +ifeq ($(HAVE_GETFSMAP),yes) +CFILES+=clearspace.c +HFILES+=clearspace.h +endif + LDIRT = gen_crc32table crc32table.h default: ltdepend $(LTLIBRARY)
libfrog/clearspace.c new file +3103 -0
2,981 lines not on this page · raw message
libfrog/clearspace.h new file +72 -0
72 lines not on this page · raw message
spaceman/clearfree.c new file +164 -0
164 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
PATCH 5/10 spaceman: physically move a regular inode
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
To be able to shrink a filesystem, we need to be able to physically move an inode and all it's data and metadata from it's current location to a new AG. Add a command to spaceman to allow an inode to be moved to a new AG.
This new command is not intended to be a perfect solution. I am not trying to handle atomic movement of open files - this is intended to be run as a maintenance operation on idle filesystem. If root filesystems are the target, then this should be run via a rescue environment that is not executing directly on the root fs. With those caveats in place, we can do the entire inode move as a set of non-destructive operations finalised by an atomic inode swap without any needing special kernel support.
To ensure we move metadata such as BMBT blocks even if we don't need to move data, we clone the data to a new inode that we've allocated in the destination AG. This will result in new bmbt blocks being allocated in the new location even though the data is not copied. Attributes need to be copied one at a time from the original inode.
If data needs to be moved, then we use fallocate(UNSHARE) to create a private copy of the range of data that needs to be moved in the new inode. This will be allocated in the destination AG by normal allocation policy.
Once the new inode has been finalised, use RENAME_EXCHANGE to swap it into place and unlink the original inode to free up all the resources it still pins.
There are many optimisations still possible to speed this up, but the goal here is "functional" rather than "optimal". Performance can be optimised once all the parts for a "empty the tail of the filesystem before shrink" operation are implemented and solidly tested.
This functionality has been smoke tested by creating a 32MB data file with 4k extents and several hundred attributes:
$ cat test.sh fname=/mnt/scratch/foo xfs_io -f -c "pwrite 0 32m" -c sync $fname for (( i=0; i < 4096 ; i++ )); do xfs_io -c "fpunch $((i * 8))k 4k" $fname done
for (( i=0; i < 100 ; i++ )); do
setfattr -n user.blah.$i.$i.blah -v blah.$i.$i.blah $fname setfattr -n user.foo.$i.$i.foo -v $i.cantbele.$i.ve.$i.tsnotbutter $fname
done
for (( i=0; i < 100 ; i++ )); do
setfattr -n security.baz.$i.$i.baz -v wotchul$i$iookinat $fname
done
xfs_io -c stat -c "bmap -vp" -c "bmap -avp" $fname xfs_spaceman -c "move_inode -a 22" /mnt/scratch/foo xfs_io -c stat -c "bmap -vp" -c "bmap -avp" $fname $
and the output looks something like:
$ sudo ./test.sh .... fd.path = "/mnt/scratch/foo" fd.flags = non-sync,non-direct,read-write stat.ino = 133 /mnt/scratch/foo: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..7]: hole 8 1: [8..15]: 208..215 0 (208..215) 8 000000 2: [16..23]: hole 8 3: [24..31]: 224..231 0 (224..231) 8 000000 .... 8189: [65512..65519]: 65712..65719 0 (65712..65719) 8 000000 8190: [65520..65527]: hole 8 8191: [65528..65535]: 65728..65735 0 (65728..65735) 8 000000 mnt/scratch/foo: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..7]: 392..399 0 (392..399) 8 000000 1: [8..15]: 408..415 0 (408..415) 8 000000 2: [16..23]: 424..431 0 (424..431) 8 000000 3: [24..31]: 456..463 0 (456..463) 8 000000 move mnt /mnt/scratch, path /mnt/scratch/foo, agno 22 fd.path = "/mnt/scratch/foo" fd.flags = non-sync,non-direct,read-write stat.ino = 47244651475 .... /mnt/scratch/foo: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..7]: hole 8 1: [8..15]: 47244763192..47244763199 22 (123112..123119) 8 000000 2: [16..23]: hole 8 3: [24..31]: 47244763208..47244763215 22 (123128..123135) 8 000000 .... 8189: [65512..65519]: 47244828808..47244828815 22 (188728..188735) 8 000000 8190: [65520..65527]: hole 8 8191: [65528..65535]: 47244828824..47244828831 22 (188744..188751) 8 000000 /mnt/scratch/foo: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..7]: 47244763176..47244763183 22 (123096..123103) 8 000000 $
--- man/man8/xfs_spaceman.8 | 4 spaceman/Makefile | 4 spaceman/init.c | 1 spaceman/move_inode.c | 563 +++++++++++++++++++++++++++++++++++++++++++++++ spaceman/space.h | 1 5 files changed, 571 insertions(+), 2 deletions(-) create mode 100644 spaceman/move_inode.c
man/man8/xfs_spaceman.8 +4 -0
@@ -146,6 +146,10 @@ Report on the health of the files at the given path. .TP .BR "help [ " command " ]" Display a brief description of one or all commands. +.TP +.BI "move_inode \-a agno" +Move the currently open file into the specified allocation group. + .TP .BI "prealloc [ \-u id ] [ \-g id ] [ -p id ] [ \-m minlen ] [ \-s ]" Removes speculative preallocation.
spaceman/Makefile +2 -2
@@ -7,10 +7,10 @@ include $(TOPDIR)/include/builddefs LTCOMMAND = xfs_spaceman HFILES = init.h space.h -CFILES = info.c init.c file.c health.c prealloc.c trim.c +CFILES = info.c init.c file.c health.c move_inode.c prealloc.c trim.c LSRCFILES = xfs_info.sh -LLDLIBS = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) +LLDLIBS = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) $(LIBHANDLE) LTDEPENDENCIES = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) LLDFLAGS = -static
spaceman/init.c +1 -0
@@ -36,6 +36,7 @@ init_commands(void)freesp_init ();health_init ();clearfree_init (); +move_inode_init (); } static int
spaceman/move_inode.c new file +563 -0
563 lines not on this page · raw message
spaceman/space.h +1 -0
@@ -38,5 +38,6 @@ extern void clearfree_init(void);#endif extern void info_init (void );extern void health_init (void ); +void move_inode_init (void );#endif /* XFS_SPACEMAN_SPACE_H_ */
PATCH 6/10 spaceman: find owners of space in an AG
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
Before we can move inodes for a shrink operation, we have to find all the inodes that own space in the AG(s) we want to empty.
This implementation uses FS_IOC_GETFSMAP on the assumption that filesystems to be shrunk have reverse mapping enabled as it is the only way to identify inode related metadata that userspace is unable to see or influence (e.g. BMBT blocks) that may be located in the specific AG. We can use GETFSMAP to identify both inodes to be moved (via XFS_FMR_OWN_INODES records) and inodes with just data and/or metadata to be moved.
Once we have identified all the inodes to be moved, we have to map them to paths so that we can use renameat2() to exchange the directory entries pointing at the moved inode atomically. We also need to record inodes with hard links and all of the paths to the inode so that hard links can be recreated appropriately.
This requires a directory tree walk to discover the paths (until parent pointers are a thing). Hence for filesystems that aren't reverse mapping enabled, we can eventually use this pass to discover inodes with visible data and metadata that need to be moved.
As we resolve the paths to the inodes to be moved, output the information to stdout so that it can be acted upon by other utilities. This results in a command that acts similar to find but with a physical location filter rather than an inode metadata filter.
Again, this is not meant to be an optimal implementation. It shouldn't suck, but there is plenty of scope for performance optimisation, especially with a multithreaded and/or async directory traversal/parent pointer path resolution process to hide access latencies.
--- libfrog/fsgeom.h | 19 ++ libfrog/radix-tree.c | 2 libfrog/radix-tree.h | 2 man/man8/xfs_spaceman.8 | 11 + spaceman/Makefile | 2 spaceman/find_owner.c | 482 +++++++++++++++++++++++++++++++++++++++++++++++ spaceman/init.c | 4 spaceman/space.h | 2 8 files changed, 522 insertions(+), 2 deletions(-) create mode 100644 spaceman/find_owner.c
libfrog/fsgeom.h +19 -0
@@ -103,6 +103,25 @@ cvt_ino_to_agino(return ino & ((1ULL << xfd->aginolog) -1 ); } +/* Convert an AG block to an AG inode number. */ +static inline uint32_t +cvt_agbno_to_agino ( +const struct xfs_fd *xfd, +xfs_agblock_t agbno) +{ +return agbno << xfd->inopblog; +} + +/* Calculate the number of inodes in a byte range */ +static inline uint32_t +cvt_b_to_inode_count ( +const struct xfs_fd *xfd, +uint64_t bytes) +{ +return (bytes >> xfd->blocklog) << xfd->inopblog; +} + +/* * Convert a linear fs block offset number into bytes. This is the runtime * equivalent of XFS_FSB_TO_B, which means that it is /not/ for segmented fsbno
libfrog/radix-tree.c +2 -0
@@ -377,6 +377,8 @@ void *radix_tree_tag_set(struct radix_tree_root *root,unsigned int height, shift;struct radix_tree_node *slot; +ASSERT (tag <RADIX_TREE_MAX_TAGS ); + height = root->height;if (index >radix_tree_maxindex (height))return NULL ;
libfrog/radix-tree.h +1 -1
@@ -28,7 +28,7 @@ do { \ } while (0)#ifdef RADIX_TREE_TAGS -#define RADIX_TREE_MAX_TAGS 2 +#define RADIX_TREE_MAX_TAGS 3#endif int radix_tree_insert (struct radix_tree_root *,unsigned long ,void *);
man/man8/xfs_spaceman.8 +11 -0
@@ -41,6 +41,14 @@ option is given, clear the realtime device. If the .B -v option is given, print what's happening every step of the way. +.TP +.BI "find_owner \-a agno" +Create an internal structure to map physical space in the given allocation +group to file paths. +This enables space reorganization on a mounted filesystem by enabling +us to find files. +Unclear why we can't just use FSMAP and BULKSTAT to open by handle. + .TP .BI "freesp [ \-dgrs ] [-a agno]... [ \-b | \-e bsize | \-h bsize | \-m factor ]" With no arguments,
@@ -195,6 +203,9 @@ Wait for removal to complete. .B print Display a list of all open files. .TP +.B resolve_owner +Resolves space in the filesystem to file paths, maybe? +.TP .B quit Exit .BR xfs_spaceman .
spaceman/Makefile +1 -1
@@ -7,7 +7,7 @@ include $(TOPDIR)/include/builddefs LTCOMMAND = xfs_spaceman HFILES = init.h space.h -CFILES = info.c init.c file.c health.c move_inode.c prealloc.c trim.c +CFILES = find_owner.c info.c init.c file.c health.c move_inode.c prealloc.c trim.c LSRCFILES = xfs_info.sh LLDLIBS = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) $(LIBHANDLE)
spaceman/find_owner.c new file +482 -0
482 lines not on this page · raw message
spaceman/init.c +4 -0
@@ -10,6 +10,7 @@#include "input.h" #include "init.h" #include "libfrog/paths.h" +#include "libfrog/radix-tree.h" #include "space.h" char *progname;
@@ -37,6 +38,8 @@ init_commands(void)health_init ();clearfree_init ();move_inode_init (); +find_owner_init (); +resolve_owner_init (); } static int
@@ -71,6 +74,7 @@ init(setlocale (LC_ALL ,"" );bindtextdomain (PACKAGE ,LOCALEDIR );textdomain (PACKAGE ); +radix_tree_init ();fs_table_initialise (0 ,NULL ,0 ,NULL );while ((c =getopt (argc, argv,"c:p:V" )) !=EOF ) {
spaceman/space.h +2 -0
@@ -39,5 +39,7 @@ extern void clearfree_init(void);extern void info_init (void );extern void health_init (void );void move_inode_init (void ); +void find_owner_init (void ); +void resolve_owner_init (void );#endif /* XFS_SPACEMAN_SPACE_H_ */
PATCH 7/10 xfs_spaceman: wrap radix tree accesses in find_owner.c
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Wrap the raw radix tree accesses here so that we can provide an alternate implementation on platforms where radix tree indices cannot store a full 64-bit inode number.
--- spaceman/Makefile | 2 - spaceman/find_owner.c | 76 +++++++++------------------------ spaceman/relocation.c | 114 +++++++++++++++++++++++++++++++++++++++++++++++++ spaceman/relocation.h | 46 ++++++++++++++++++++ 4 files changed, 183 insertions(+), 55 deletions(-) create mode 100644 spaceman/relocation.c create mode 100644 spaceman/relocation.h
spaceman/Makefile +1 -1
@@ -7,7 +7,7 @@ include $(TOPDIR)/include/builddefs LTCOMMAND = xfs_spaceman HFILES = init.h space.h -CFILES = find_owner.c info.c init.c file.c health.c move_inode.c prealloc.c trim.c +CFILES = find_owner.c info.c init.c file.c health.c move_inode.c prealloc.c relocation.c trim.c LSRCFILES = xfs_info.sh LLDLIBS = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) $(LIBHANDLE)
spaceman/find_owner.c +22 -54
171 lines not on this page · raw message
spaceman/relocation.c new file +114 -0
114 lines not on this page · raw message
spaceman/relocation.h new file +46 -0
46 lines not on this page · raw message
PATCH 8/10 xfs_spaceman: port relocation structure to 32-bit systems
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We can't use the radix tree to store relocation information on 32-bit systems because unsigned longs are not large enough to hold 64-bit inodes. Use an avl64 tree instead.
--- configure.ac | 1 include/builddefs.in | 1 m4/package_libcdev.m4 | 20 +++++ spaceman/Makefile | 4 + spaceman/relocation.c | 203 +++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 229 insertions(+)
configure.ac +1 -0
@@ -258,6 +258,7 @@ AC_HAVE_MEMFD_CLOEXEC AC_HAVE_MEMFD_NOEXEC_SEAL AC_HAVE_O_TMPFILE AC_HAVE_MKOSTEMP_CLOEXEC +AC_USE_RADIX_TREE_FOR_INUMS AC_CONFIG_FILES([include/builddefs]) AC_OUTPUT
include/builddefs.in +1 -0
@@ -138,6 +138,7 @@ HAVE_MEMFD_CLOEXEC = @have_memfd_cloexec@ HAVE_MEMFD_NOEXEC_SEAL = @have_memfd_noexec_seal@ HAVE_O_TMPFILE = @have_o_tmpfile@ HAVE_MKOSTEMP_CLOEXEC = @have_mkostemp_cloexec@ +USE_RADIX_TREE_FOR_INUMS = @use_radix_tree_for_inums@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall # -Wbitwise -Wno-transparent-union -Wno-old-initializer -Wno-decl
m4/package_libcdev.m4 +20 -0
@@ -612,3 +612,23 @@ AC_DEFUN([AC_HAVE_MKOSTEMP_CLOEXEC], AC_MSG_RESULT(yes)],[AC_MSG_RESULT(no)]) AC_SUBST(have_mkostemp_cloexec) ]) + +# +# Check if the radix tree index (unsigned long) is large enough to hold a +# 64-bit inode number +# +AC_DEFUN([AC_USE_RADIX_TREE_FOR_INUMS], + [ AC_MSG_CHECKING([if radix tree can store XFS inums]) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[ +#include <sys/param.h> +#include <stdint.h> +#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) + ]], [[ + typedef uint64_t xfs_ino_t; + + BUILD_BUG_ON(sizeof(unsigned long) < sizeof(xfs_ino_t)); + return 0; + ]])],[use_radix_tree_for_inums=yes + AC_MSG_RESULT(yes)],[AC_MSG_RESULT(no)]) + AC_SUBST(use_radix_tree_for_inums) + ])
spaceman/Makefile +4 -0
@@ -22,6 +22,10 @@ ifeq ($(HAVE_GETFSMAP),yes) CFILES += freesp.c clearfree.c endif +ifeq ($(USE_RADIX_TREE_FOR_INUMS),yes) +LCFLAGS += -DUSE_RADIX_TREE_FOR_INUMS +endif + default: depend $(LTCOMMAND) include $(BUILDRULES)
spaceman/relocation.c +203 -0
219 lines not on this page · raw message
PATCH 9/10 spaceman: relocate the contents of an AG
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
Shrinking a filesystem needs to first remove all the active user data and metadata from the AGs that are going to be lopped off the filesystem. Before we can do this, we have to relocate this information to a region of the filesystem that is going to be retained.
We have a function to move an inode and all it's related information to a specific AG, we have functions to find the owners of all the information in an AG and we can find their paths. This gives us all the information we need to relocate all the objects in an AG we are going to remove via shrinking.
Firstly we scan the AG to be emptied to find the inodes that need to be relocated, then we scan the directory structure to find all the paths to those inodes that need to be moved. Then we iterate over all the inodes to be moved attempting to move them to the lowest numbers AGs.
When the destination AG fills up, we'll get ENOSPC from the moving code and this is a trigger to bump the destination AG and retry the move. If we haven't moved all the inodes and their data by the time the destination reaches the source AG, then the entire operation will fail with ENOSPC - there is not enough room in the filesystem to empty the selected AG in preparation for a shrink.
This, once again, is not intended as an optimal or even guaranteed way of emptying an AG for shrink. It simply provides the basic algorithm and mechanisms we need to perform a shrink operation. Improvements and optimisations will come in time, but we can't get to an optimal solution without first having basic functionality in place.
--- libfrog/fsgeom.h | 10 ++ man/man8/xfs_spaceman.8 | 8 ++ spaceman/find_owner.c | 32 +++--- spaceman/init.c | 1 spaceman/move_inode.c | 7 + spaceman/relocation.c | 234 +++++++++++++++++++++++++++++++++++++++++++++++ spaceman/relocation.h | 5 + spaceman/space.h | 1 8 files changed, 280 insertions(+), 18 deletions(-)
libfrog/fsgeom.h +10 -0
@@ -202,6 +202,16 @@ cvt_daddr_to_agno(return cvt_bb_to_off_fsbt (xfd, daddr) / xfd->fsgeom.agblocks; } +/* Convert sparse filesystem block to AG Number */ +static inline uint32_t +cvt_fsb_to_agno ( +struct xfs_fd *xfd, +uint64_t fsbno) +{ +return fsbno >> xfd->agblklog; +} + +/* Convert sector number to AG block number. */ static inline uint32_t cvt_daddr_to_agbno(
man/man8/xfs_spaceman.8 +8 -0
@@ -202,9 +202,17 @@ Wait for removal to complete. .TP .B print Display a list of all open files. +.TP +.BI "relocate \-a agno [ \-h agno ]" +Empty out the given allocation group by moving file data elsewhere. +The +.B -h +option specifies the highest allocation group into which we can move data. + .TP .B resolve_owner Resolves space in the filesystem to file paths, maybe? + .TP .B quit Exit
spaceman/find_owner.c +17 -15
@@ -9,10 +9,10 @@#include <linux/fiemap.h> #include "libfrog/fsgeom.h" #include "libfrog/radix-tree.h" -#include "command.h" -#include "init.h" #include "libfrog/paths.h" #include <linux/fsmap.h> +#include "command.h" +#include "init.h" #include "space.h" #include "input.h" #include "relocation.h"
@@ -65,8 +65,8 @@ track_inode(set_reloc_iflag (owner,MOVE_BLOCKS ); } -static void -scan_ag ( +int +find_relocation_targets (xfs_agnumber_t agno) {struct fsmap_head *fsmap;
@@ -80,8 +80,7 @@ scan_ag( fsmap =malloc (fsmap_sizeof (NR_EXTENTS ));if (!fsmap) {fprintf (stderr,_ ("%s: fsmap malloc failed.\n" ), progname); - exitcode =1 ; -return ; +return -ENOMEM ; }memset (fsmap,0 ,sizeof (*fsmap));
@@ -102,8 +101,7 @@ scan_ag(fprintf (stderr,_ ("%s: FS_IOC_GETFSMAP [\"%s\"]: %s\n" ), progname, file->name,strerror (errno));free (fsmap); - exitcode =1 ; -return ; +return -errno; }/* No more extents to map, exit */
@@ -148,6 +146,7 @@ scan_ag( }free (fsmap); +return 0 ; }/*
@@ -159,6 +158,7 @@ find_owner_f( char **argv) {xfs_agnumber_t agno =-1 ; +int ret;int c;while ((c =getopt (argc, argv,"a:" )) !=EOF ) {
@@ -198,7 +198,9 @@ _("Filesystem at %s does not have reverse mapping enabled. Aborting.\n"),return 0 ; } -scan_ag (agno); + ret =find_relocation_targets (agno); +if (ret) + exitcode =1 ;return 0 ; }
@@ -299,8 +301,8 @@ _("Aborting: Storing path %s for inode 0x%lx failed: %s\n"),* This should be parallelised - pass subdirs off to a work queue, have the * work queue processes subdirs, queueing more subdirs to work on. */ -static int -walk_mount ( +int +resolve_target_paths (const char *mntpt) {int ret;
@@ -361,9 +363,9 @@ list_inode_paths(void)/* * Any inodes remaining in the tree at this point indicate inodes whose -* paths were not found. This will be unlinked but still open inodes or -* lost inodes due to corruptions. Either way, a shrink will not succeed -* until these inodes are removed from the filesystem. +* paths were not found. This will be free inodes or unlinked but still +* open inodes. Either way, a shrink will not succeed until these inodes +* are removed from the filesystem. */ idx =0 ; do {
@@ -400,7 +402,7 @@ _("Inode list has not been populated. No inodes to resolve.\n"));return 0 ; } - ret =walk_mount (file->fs_path.fs_dir); + ret =resolve_target_paths (file->fs_path.fs_dir);if (ret) { fprintf(stderr, _("Failed to resolve all paths from mount point %s: %s\n"),
spaceman/init.c +1 -0
@@ -40,6 +40,7 @@ init_commands(void)move_inode_init ();find_owner_init ();resolve_owner_init (); +relocate_init (); } static int
spaceman/move_inode.c +4 -3
@@ -12,6 +12,7 @@#include "space.h" #include "input.h" #include "handle.h" +#include "relocation.h" #include <linux/fiemap.h> #include <linux/falloc.h>
@@ -404,8 +405,8 @@ exchange_inodes(return 0 ; } -static int -move_file_to_ag( +int +relocate_file_to_ag( const char *mnt, const char *path, struct xfs_fd *xfd,
@@ -511,7 +512,7 @@ _("Destination AG %d does not exist. Filesystem only has %d AGs\n"), }if (S_ISREG (st.st_mode)) { - ret =move_file_to_ag (file->fs_path.fs_dir, file->name, + ret =relocate_file_to_ag (file->fs_path.fs_dir, file->name, &file->xfd, agno); }else { fprintf(stderr, _("Unsupported: %s is not a regular file.\n"),
spaceman/relocation.c +234 -0
237 lines not on this page · raw message
spaceman/relocation.h +5 -0
@@ -43,4 +43,9 @@ struct inode_path {*/ #define UNLINKED_IPATH ((struct inode_path *)1) +int find_relocation_targets (xfs_agnumber_t agno); +int relocate_file_to_ag (const char *mnt,const char *path,struct xfs_fd *xfd, +xfs_agnumber_t agno); +int resolve_target_paths (const char *mntpt); +#endif /* XFS_SPACEMAN_RELOCATION_H_ */
spaceman/space.h +1 -0
@@ -41,5 +41,6 @@ extern void health_init(void);void move_inode_init (void );void find_owner_init (void );void resolve_owner_init (void ); +void relocate_init (void );#endif /* XFS_SPACEMAN_SPACE_H_ */
PATCH 10/10 spaceman: move inodes with hardlinks
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
When a inode to be moved to a different AG has multiple hard links, we need to "move" all the hard links, too. To do this, we need to create temporary hardlinks to the new file, and then use rename exchange to swap all the hardlinks that point to the old inode with new hardlinks that point to the new inode.
We already know that an inode has hard links via the path discovery, and we can check it against the link count that is reported for the inode before we start building the link farm.
--- spaceman/find_owner.c | 13 +---- spaceman/move_inode.c | 119 +++++++++++++++++++++++++++++++++++++++++++++---- spaceman/relocation.c | 35 ++++++++++---- spaceman/relocation.h | 6 ++ 4 files changed, 140 insertions(+), 33 deletions(-)
spaceman/find_owner.c +2 -11
@@ -240,7 +240,6 @@ resolve_owner_cb( struct FTW *data) {struct inode_path *ipath, *slot_ipath; -int pathlen;struct inode_path **slot;/*
@@ -260,17 +259,9 @@ _("Failed to obtain stat(2) information from path %s. Aborting\n"), }/* Allocate a new inode path and record the path in it. */ - pathlen =strlen (path); - ipath =calloc (1 ,sizeof (*ipath) + pathlen +1 ); -if (!ipath) { -fprintf (stderr, -_ ("Aborting: Storing path %s for inode 0x%lx failed: %s\n" ), - path, stat->st_ino,strerror (ENOMEM )); + ipath =ipath_alloc (path, stat); +if (!ipath)return -ENOMEM ; - } -INIT_LIST_HEAD (&ipath->path_list); -memcpy (&ipath->path[0 ], path, pathlen); - ipath->ino = stat->st_ino;/* * If the slot contains the inode number we just looked up, then we
spaceman/move_inode.c +109 -10
203 lines not on this page · raw message
spaceman/relocation.c +25 -10
@@ -318,6 +318,30 @@ forget_reloc_ino(static struct cmdinfo relocate_cmd; +struct inode_path * +ipath_alloc ( +const char *path, +const struct stat *stat) +{ +struct inode_path *ipath; +int pathlen =strlen (path); + +/* Allocate a new inode path and record the path in it. */ + ipath =calloc (1 ,sizeof (*ipath) + pathlen +1 ); +if (!ipath) { +fprintf (stderr, +_ ("Failed to allocate ipath %s for inode 0x%llx failed: %s\n" ), + path, (unsigned long long )stat->st_ino, +strerror (-errno)); +return NULL ; + } +INIT_LIST_HEAD (&ipath->path_list); +memcpy (&ipath->path[0 ], path, pathlen); + ipath->ino = stat->st_ino; + +return ipath; +} + static int relocate_targets_to_ag( const char *mnt,
@@ -336,15 +360,6 @@ relocate_targets_to_ag(if (!ipath)break ; -/* XXX: don't handle hard link cases yet */ -if (ipath->link_count >1 ) { -fprintf (stderr, -"FIXME! Skipping hardlinked inode at path %s\n" , - ipath->path); - goto next; - } - - ret =stat (ipath->path, &st);if (ret) { fprintf(stderr, _("stat(%s) failed: %s\n"),
@@ -367,7 +382,7 @@ relocate_targets_to_ag( }/* move to destination AG */ - ret =relocate_file_to_ag (mnt, ipath->path, &xfd, dst_agno); + ret =relocate_file_to_ag (mnt, ipath, &xfd, dst_agno);xfd_close (&xfd);/*
spaceman/relocation.h +4 -2
@@ -43,9 +43,11 @@ struct inode_path {*/ #define UNLINKED_IPATH ((struct inode_path *)1) +struct inode_path *ipath_alloc (const char *path,const struct stat *st); +int find_relocation_targets (xfs_agnumber_t agno); -int relocate_file_to_ag (const char *mnt,const char *path,struct xfs_fd *xfd, -xfs_agnumber_t agno); +int relocate_file_to_ag (const char *mnt,struct inode_path *ipath, +struct xfs_fd *xfd,xfs_agnumber_t agno);int resolve_target_paths (const char *mntpt);#endif /* XFS_SPACEMAN_RELOCATION_H_ */
PATCH 1/2 fstests: functional test for refcount reporting
In reply to Darrick J. WongHi all,
Add a short functional test for the new GETFSREFCOUNTS ioctl that allows userspace to query reference count information for a given range of physical blocks.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=report-refcounts
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=report-refcounts
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=report-refcounts
--- common/rc | 4 + doc/group-names.txt | 1 tests/xfs/122.out | 2 + tests/xfs/1921 | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1921.out | 4 + 5 files changed, 177 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/1921 create mode 100644 tests/xfs/1921.out
PATCH 1/2 xfs/122: update for the getfsrefs ioctl
From: Darrick J. Wong <[email protected]>
Add this new ioctl here too.
--- tests/xfs/122.out | 2 ++ 1 file changed, 2 insertions(+)
tests/xfs/122.out +2 -0
@@ -101,6 +101,8 @@ sizeof(struct xfs_fsop_ag_resblks) = 64 sizeof(struct xfs_fsop_geom) = 256 sizeof(struct xfs_fsop_geom_v1) = 112 sizeof(struct xfs_fsop_geom_v4) = 112 +sizeof(struct xfs_getfsrefs) = 64 +sizeof(struct xfs_getfsrefs_head) = 192 sizeof(struct xfs_getparents) = 96 sizeof(struct xfs_getparents_rec) = 24 sizeof(struct xfs_icreate_log) = 28
PATCH 2/2 xfs: test output of new FSREFCOUNTS ioctl
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make sure the cursors work properly and that refcounts are correct.
--- common/rc | 4 + doc/group-names.txt | 1 tests/xfs/1921 | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1921.out | 4 + 4 files changed, 175 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/1921 create mode 100644 tests/xfs/1921.out
common/rc +2 -2
@@ -2640,8 +2640,8 @@ _require_xfs_io_command() echo $testio | grep -q "Operation not supported" && \ _notrun "O_TMPFILE is not supported" ;; - "fsmap") - testio=`$XFS_IO_PROG -f -c "fsmap" $testfile 2>&1` + "fsmap"|"fsrefcounts") + testio=`$XFS_IO_PROG -f -c "$command" $testfile 2>&1` echo $testio | grep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;;
doc/group-names.txt +1 -0
@@ -57,6 +57,7 @@ freeze filesystem freeze tests fsck general fsck tests fsmap FS_IOC_GETFSMAP ioctl fsr XFS free space reorganizer +fsrefcounts FS_IOC_GETFSREFCOUNTS ioctl fuzzers filesystem fuzz tests growfs increasing the size of a filesystem hardlink hardlinks
tests/xfs/1921 new file +168 -0
168 lines not on this page · raw message
tests/xfs/1921.out new file +4 -0
@@ -0,0 +1,4 @@ +QA output created by 1921 +Format and mount +Compare fsrefcounts +Compare fsrefcounts to fsmap
PATCH 2/2 fstests: defragment free space
In reply to Darrick J. WongHi all,
These patches contain experimental code to enable userspace to defragment the free space in a filesystem. Two purposes are imagined for this
functionality: clearing space at the end of a filesystem before shrinking it, and clearing free space in anticipation of making a large allocation.
The first patch adds a new fallocate mode that allows userspace to allocate free space from the filesystem into a file. The goal here is to allow the filesystem shrink process to prevent allocation from a certain part of the filesystem while a free space defragmenter evacuates all the files from the doomed part of the filesystem.
The second patch amends the online repair system to allow the sysadmin to forcibly rebuild metadata structures, even if they're not corrupt. Without adding an ioctl to move metadata btree blocks, this is the only way to dislodge metadata.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=defrag-freespace
--- common/rc | 5 +++ tests/xfs/122.out | 1 + tests/xfs/1400 | 57 ++++++++++++++++++++++++++++++++++++ tests/xfs/1400.out | 2 + tests/xfs/1401 | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1401.out | 2 + 6 files changed, 149 insertions(+) create mode 100755 tests/xfs/1400 create mode 100644 tests/xfs/1400.out create mode 100755 tests/xfs/1401 create mode 100644 tests/xfs/1401.out
PATCH 1/2 xfs/122: update for XFS_IOC_MAP_FREESP
From: Darrick J. Wong <[email protected]>
Update this test to include the new free space mapping ioctl.
--- tests/xfs/122.out | 1 + 1 file changed, 1 insertion(+)
tests/xfs/122.out +1 -0
@@ -114,6 +114,7 @@ sizeof(struct xfs_legacy_timestamp) = 8 sizeof(struct xfs_log_dinode) = 176 sizeof(struct xfs_log_legacy_timestamp) = 8 sizeof(struct xfs_map_extent) = 32 +sizeof(struct xfs_map_freesp) = 32 sizeof(struct xfs_parent_name_rec) = 16 sizeof(struct xfs_phys_extent) = 16 sizeof(struct xfs_refcount_key) = 4
PATCH 2/2 xfs: test clearing of free space
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Simple regression test for the spaceman clearspace command, which tries to free all the used space in some part of the filesystem.
--- common/rc | 5 +++ tests/xfs/1400 | 57 ++++++++++++++++++++++++++++++++++++ tests/xfs/1400.out | 2 + tests/xfs/1401 | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1401.out | 2 + 5 files changed, 148 insertions(+) create mode 100755 tests/xfs/1400 create mode 100644 tests/xfs/1400.out create mode 100755 tests/xfs/1401 create mode 100644 tests/xfs/1401.out
common/rc +5 -0
@@ -2615,6 +2615,11 @@ _require_xfs_io_command() -c "fsync" -c "$command $blocksize $((2 * $blocksize))" \ $testfile 2>&1` ;; + "fmapfree") + local blocksize=$(_get_file_block_size $TEST_DIR) + testio=`$XFS_IO_PROG -F -f -c "$command $blocksize $((2 * $blocksize))" \ + $testfile 2>&1` + ;; "fiemap") # If 'ranged' is passed as argument then we check to see if fiemap supports # ranged query params
tests/xfs/1400 new file +57 -0
57 lines not on this page · raw message
tests/xfs/1400.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1400 +mapfree space used is in range
tests/xfs/1401 new file +82 -0
82 lines not on this page · raw message
tests/xfs/1401.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 1401 +Silence is golden