xfs: Parent Pointers
Hi everyone,
This deluge contains all of the additions to the parent pointers patchset that I've been working on for the past month. The kernel and xfsprogs patchsets are based on Allison's v9r2 tag from last week; the fstests patches are merely a part of my development tree. To recap Allison's cover letter:
"The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point."
The kernel branches start with a number of buf fixes that I need to get fstests to pass. I also restructured the kernel implementation of GETPARENTS to cut the memory usage considerably.
For userspace, I cleaned up the xfsprogs patches so that libxfs-diff shows no discrepancies with the kernel and cleaned up the parent pointer usage code that I prototyped in 2017 so that it's less buggy and moldy. I also rewired xfs_scrub to use GETPARENTS to report file paths of corrupt files instead of inode numbers, since that part had bitrotted badly.
With that out of the way, I implemented a prototype of online repairs for directories and parent pointers. This is only a proof of concept, because I had already backported many many patches from part 1 of online repair, and didn't feel like porting the parts needed to commit new structures atomically and reap the old dir/xattr blocks. IOWs, the prototype scans the filesystem to build a parallel directory or xattr structure, and then reports on any discrepancies between the two versions. Obviously this won't fix a corrupt directory tree, but it enables us to test the repair code on a consistent filesystem to demonstrate that it works.
Next, I implemented fully functional parent pointer checking and repair for xfs_repair. This was less hard than I guessed it would be because the current design of phase 6 includes a walk of all directories. From the dirent data, we can build a per-AG index of all the parent pointers for all the inodes in that AG, then walk all the inodes in that AG to compare the lists. As you might guess, this eats a fair amount of memory, even with a rudimentary dirent name deduplication table to cut down on memory usage.
After that, I moved on to solving the major problem that I've been having with the directory repair code, and that is the problem of reconstructing dirents at the offsets specified by the parent pointers. The details of the problem and how I dealt with it are captured in the cover letter for those patches. Suffice to say, we now encode the dirent name in the parent pointer attrname (or a collision resistant hash if it doesn't fit), which makes it possible to commit new directories atomically.
The last part of this patchset reorganizes the XFS_IOC_GETPARENTS ioctl to encode variable length parent pointer records in the caller's buffer. The denser encodings mean that we can extract the parent list with fewer kernel calls.
--D
cover xfs: Parent Pointers
Hi all,
This is the latest parent pointer attributes for xfs.
The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.
This set can be viewed on github here
https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv9_r2
And the corresponding xfsprogs code is here
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v9_r2
This set has been tested with the below parent pointers tests
https://lore.kernel.org/fstests/[email protected]/T/#t
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_attr.c | 71 +++++- fs/xfs/libxfs/xfs_attr.h | 13 + fs/xfs/libxfs/xfs_da_btree.h | 3 fs/xfs/libxfs/xfs_da_format.h | 26 ++ fs/xfs/libxfs/xfs_defer.c | 28 ++ fs/xfs/libxfs/xfs_defer.h | 8 + fs/xfs/libxfs/xfs_dir2.c | 21 +- fs/xfs/libxfs/xfs_dir2.h | 7 - fs/xfs/libxfs/xfs_dir2_block.c | 9 - fs/xfs/libxfs/xfs_dir2_leaf.c | 8 + fs/xfs/libxfs/xfs_dir2_node.c | 8 + fs/xfs/libxfs/xfs_dir2_sf.c | 6 + fs/xfs/libxfs/xfs_format.h | 4 fs/xfs/libxfs/xfs_fs.h | 75 +++++++ fs/xfs/libxfs/xfs_log_format.h | 7 - fs/xfs/libxfs/xfs_log_rlimit.c | 53 +++++ fs/xfs/libxfs/xfs_parent.c | 203 ++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 84 +++++++ fs/xfs/libxfs/xfs_sb.c | 4 fs/xfs/libxfs/xfs_trans_resv.c | 324 ++++++++++++++++++++++++---- fs/xfs/libxfs/xfs_trans_space.h | 8 - fs/xfs/scrub/attr.c | 4 fs/xfs/xfs_attr_item.c | 142 ++++++++++-- fs/xfs/xfs_attr_item.h | 1 fs/xfs/xfs_attr_list.c | 17 + fs/xfs/xfs_dquot.c | 38 +++ fs/xfs/xfs_dquot.h | 1 fs/xfs/xfs_file.c | 1 fs/xfs/xfs_inode.c | 447 +++++++++++++++++++++++++++++++-------- fs/xfs/xfs_inode.h | 3 fs/xfs/xfs_ioctl.c | 148 +++++++++++-- fs/xfs/xfs_ioctl.h | 2 fs/xfs/xfs_iops.c | 3 fs/xfs/xfs_ondisk.h | 4 fs/xfs/xfs_parent_utils.c | 126 +++++++++++ fs/xfs/xfs_parent_utils.h | 11 + fs/xfs/xfs_qm.c | 4 fs/xfs/xfs_qm.h | 2 fs/xfs/xfs_super.c | 14 + fs/xfs/xfs_symlink.c | 58 ++++- fs/xfs/xfs_trans.c | 13 + fs/xfs/xfs_trans_dquot.c | 15 + fs/xfs/xfs_xattr.c | 7 - fs/xfs/xfs_xattr.h | 2 45 files changed, 1782 insertions(+), 253 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
PATCH 1/28 xfs: Add new name to attri/d
From: Allison Henderson <[email protected]>
This patch adds two new fields to the atti/d. They are nname and nnamelen. This will be used for parent pointer updates since a rename operation may cause the parent pointer to update both the name and value. So we need to carry both the new name as well as the target name in the attri/d.
--- fs/xfs/libxfs/xfs_attr.c | 12 +++- fs/xfs/libxfs/xfs_attr.h | 4 + fs/xfs/libxfs/xfs_da_btree.h | 2 + fs/xfs/libxfs/xfs_log_format.h | 6 +- fs/xfs/xfs_attr_item.c | 135 +++++++++++++++++++++++++++++++++------- fs/xfs/xfs_attr_item.h | 1 6 files changed, 133 insertions(+), 27 deletions(-)
fs/xfs/libxfs/xfs_attr.c +11 -1
@@ -423,6 +423,12 @@ xfs_attr_complete_op( args->op_flags &= ~XFS_DA_OP_REPLACE ;if (do_replace) { args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; +if (args->new_namelen >0 ) { + args->name = args->new_name; + args->namelen = args->new_namelen; + args->hashval =xfs_da_hashname (args->name, + args->namelen); + }return replace_state; }return XFS_DAS_DONE ;
@@ -922,9 +928,13 @@ xfs_attr_defer_replace( struct xfs_da_args *args) {struct xfs_attr_intent *new; +int op_flag;int error =0 ; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_REPLACE , &new); + op_flag = args->new_namelen ==0 ?XFS_ATTRI_OP_FLAGS_REPLACE : +XFS_ATTRI_OP_FLAGS_NVREPLACE ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
fs/xfs/libxfs/xfs_attr.h +2 -2
@@ -510,8 +510,8 @@ struct xfs_attr_intent {struct xfs_da_args *xattri_da_args;/* -* Shared buffer containing the attr name and value so that the logging -* code can share large memory buffers between log items. +* Shared buffer containing the attr name, new name, and value so that +* the logging code can share large memory buffers between log items. */ struct xfs_attri_log_nameval *xattri_nameval;
fs/xfs/libxfs/xfs_da_btree.h +2 -0
@@ -55,7 +55,9 @@ enum xfs_dacmp { typedef struct xfs_da_args { struct xfs_da_geometry *geo; /* da block geometry */ const uint8_t *name; /* string (maybe not NULL terminated) */ + const uint8_t *new_name; /* new attr name */ int namelen; /* length of string (maybe no NULL) */ + int new_namelen; /* new attr name len */ uint8_t filetype; /* filetype of inode for directories */ void *value; /* set of bytes (maybe contain NULLs) */ int valuelen; /* length of value */
fs/xfs/libxfs/xfs_log_format.h +4 -2
@@ -117,7 +117,8 @@ struct xfs_unmount_log_format {#define XLOG_REG_TYPE_ATTRD_FORMAT 28#define XLOG_REG_TYPE_ATTR_NAME 29#define XLOG_REG_TYPE_ATTR_VALUE 30 -#define XLOG_REG_TYPE_MAX 30 +#define XLOG_REG_TYPE_ATTR_NNAME 31 +#define XLOG_REG_TYPE_MAX 31/*
@@ -957,6 +958,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_SET 1/* Set the attribute */ #define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ +#define XFS_ATTRI_OP_FLAGS_NVREPLACE 4/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
@@ -974,7 +976,7 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ +uint32_t alfi_nname_len;/* attr new name length */ uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */
fs/xfs/xfs_attr_item.c +113 -22
265 lines not on this page · raw message
fs/xfs/xfs_attr_item.h +1 -0
@@ -13,6 +13,7 @@ struct kmem_zone;struct xfs_attri_log_nameval {struct xfs_log_iovec name; +struct xfs_log_iovec nname;struct xfs_log_iovec value;refcount_t refcount;
PATCH 2/28 xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.
--- fs/xfs/libxfs/xfs_defer.c | 28 ++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_defer.h | 8 +++++++- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 4 files changed, 35 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_defer.c +26 -2
@@ -820,13 +820,37 @@ xfs_defer_ops_continue(struct xfs_trans *tp, struct xfs_defer_resources *dres) { - unsigned int i; + unsigned int i, j; +struct xfs_inode *sips[XFS_DEFER_OPS_NR_INODES ]; +struct xfs_inode *temp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );ASSERT (!(tp->t_flags &XFS_TRANS_DIRTY ));/* Lock the captured resources to the new transaction. */ -if (dfc->dfc_held.dr_inos ==2 ) +if (dfc->dfc_held.dr_inos >2 ) { +/* +* Renames with parent pointer updates can lock up to 5 inodes, +* sorted by their inode number. So we need to make sure they +* are relocked in the same way. +*/ +memset (sips,0 ,sizeof (sips)); +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) + sips[i] = dfc->dfc_held.dr_ip[i]; + +/* Bubble sort of at most 5 inodes */ +for (i =0 ; i < dfc->dfc_held.dr_inos; i++) { +for (j =1 ; j < dfc->dfc_held.dr_inos; j++) { +if (sips[j]->i_ino < sips[j-1 ]->i_ino) { + temp = sips[j]; + sips[j] = sips[j-1 ]; + sips[j-1 ] = temp; + } + } + } + +xfs_lock_inodes (sips, dfc->dfc_held.dr_inos,XFS_ILOCK_EXCL ); + }else if (dfc->dfc_held.dr_inos ==2 )xfs_lock_two_inodes (dfc->dfc_held.dr_ip[0 ],XFS_ILOCK_EXCL , dfc->dfc_held.dr_ip[1 ],XFS_ILOCK_EXCL );else if (dfc->dfc_held.dr_inos ==1 )
fs/xfs/libxfs/xfs_defer.h +7 -1
@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;/* * Deferred operation item relogging limits. */ -#define XFS_DEFER_OPS_NR_INODES 2/* join up to two inodes */ + +/* +* Rename w/ parent pointers can require up to 5 inodes with deferred ops to +* be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip. +* These inodes are locked in sorted order by their inode numbers +*/ +#define XFS_DEFER_OPS_NR_INODES 5#define XFS_DEFER_OPS_NR_BUFS 2/* join up to two buffers */ /* Resources that must be held across a transaction roll. */
fs/xfs/xfs_inode.c +1 -1
@@ -447,7 +447,7 @@ xfs_lock_inumorder(* lock more than one at a time, lockdep will report false positives saying we * have violated locking orders. */ -static void +void xfs_lock_inodes( struct xfs_inode **ips, int inodes,
fs/xfs/xfs_inode.h +1 -0
@@ -574,5 +574,6 @@ void xfs_end_io(struct work_struct *work);int xfs_ilock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2); +void xfs_lock_inodes (struct xfs_inode **ips,int inodes,uint lock_mode);#endif /* __XFS_INODE_H__ */
PATCH 3/28 xfs: Increase XFS_QM_TRANS_MAXDQS to 5
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
With parent pointers enabled, a rename operation can update up to 5
inodes: src_dp, target_dp, src_ip, target_ip and wip. This causes their dquots to a be attached to the transaction chain, so we need to increase XFS_QM_TRANS_MAXDQS. This patch also add a helper function xfs_dqlockn to lock an arbitrary number of dquots.
--- fs/xfs/xfs_dquot.c | 38 ++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_dquot.h | 1 + fs/xfs/xfs_qm.h | 2 +- fs/xfs/xfs_trans_dquot.c | 15 ++++++++++----- 4 files changed, 50 insertions(+), 6 deletions(-)
fs/xfs/xfs_dquot.c +38 -0
@@ -1333,6 +1333,44 @@ xfs_dqlock2( } } +static int +xfs_dqtrx_cmp ( +const void *a, +const void *b) +{ +const struct xfs_dqtrx *qa = a; +const struct xfs_dqtrx *qb = b; + +if (qa->qt_dquot->q_id > qb->qt_dquot->q_id) +return 1 ; +if (qa->qt_dquot->q_id < qb->qt_dquot->q_id) +return -1 ; +return 0 ; +} + +void +xfs_dqlockn ( +struct xfs_dqtrx *q) +{ +unsigned int i; + +/* Sort in order of dquot id, do not allow duplicates */ +for (i =0 ; i <XFS_QM_TRANS_MAXDQS && q[i].qt_dquot !=NULL ; i++) { +unsigned int j; + +for (j =0 ; j < i; j++) +ASSERT (q[i].qt_dquot != q[j].qt_dquot); + } +if (i ==0 ) +return ; + +sort (q, i,sizeof (struct xfs_dqtrx ), xfs_dqtrx_cmp,NULL ); + +mutex_lock (&q[0 ].qt_dquot->q_qlock); +for (i =1 ; i <XFS_QM_TRANS_MAXDQS && q[i].qt_dquot !=NULL ; i++) +mutex_lock_nested (&q[i].qt_dquot->q_qlock,XFS_QLOCK_NESTED ); +} +int __init xfs_qm_init (void ) {
fs/xfs/xfs_dquot.h +1 -0
@@ -223,6 +223,7 @@ int xfs_qm_dqget_uncached(struct xfs_mount *mp,void xfs_qm_dqput (struct xfs_dquot *dqp);void xfs_dqlock2 (struct xfs_dquot *,struct xfs_dquot *); +void xfs_dqlockn (struct xfs_dqtrx *q);void xfs_dquot_set_prealloc_limits (struct xfs_dquot *);
fs/xfs/xfs_qm.h +1 -1
@@ -120,7 +120,7 @@ enum {XFS_QM_TRANS_PRJ , XFS_QM_TRANS_DQTYPES }; -#define XFS_QM_TRANS_MAXDQS 2 +#define XFS_QM_TRANS_MAXDQS 5struct xfs_dquot_acct {struct xfs_dqtrx dqs[XFS_QM_TRANS_DQTYPES ][XFS_QM_TRANS_MAXDQS ]; };
fs/xfs/xfs_trans_dquot.c +10 -5
@@ -268,24 +268,29 @@ xfs_trans_mod_dquot(/* * Given an array of dqtrx structures, lock all the dquots associated and join -* them to the transaction, provided they have been modified. We know that the -* highest number of dquots of one type - usr, grp and prj - involved in a -* transaction is 3 so we don't need to make this very generic. +* them to the transaction, provided they have been modified. */ STATIC voidxfs_trans_dqlockedjoin (struct xfs_trans *tp,struct xfs_dqtrx *q) { +unsigned int i;ASSERT (q[0 ].qt_dquot !=NULL );if (q[1 ].qt_dquot ==NULL ) {xfs_dqlock (q[0 ].qt_dquot);xfs_trans_dqjoin (tp, q[0 ].qt_dquot); - }else { -ASSERT (XFS_QM_TRANS_MAXDQS ==2 ); + }else if (q[2 ].qt_dquot ==NULL ) {xfs_dqlock2 (q[0 ].qt_dquot, q[1 ].qt_dquot);xfs_trans_dqjoin (tp, q[0 ].qt_dquot);xfs_trans_dqjoin (tp, q[1 ].qt_dquot); + }else { +xfs_dqlockn (q); +for (i =0 ; i <XFS_QM_TRANS_MAXDQS ; i++) { +if (q[i].qt_dquot ==NULL ) +break ; +xfs_trans_dqjoin (tp, q[i].qt_dquot); + } } }
PATCH 4/28 xfs: Hold inode locks in xfs_ialloc
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_ialloc to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 8 +++++++- fs/xfs/xfs_qm.c | 4 +++- fs/xfs/xfs_symlink.c | 3 +++ 3 files changed, 13 insertions(+), 2 deletions(-)
fs/xfs/xfs_inode.c +7 -1
@@ -774,6 +774,8 @@ xfs_inode_inherit_flags2(/* * Initialise a newly allocated inode and return the in-core inode to the * caller locked exclusively. +* +* Caller is responsible for unlocking the inode manually upon return */ int xfs_init_new_inode(
@@ -899,7 +901,7 @@ xfs_init_new_inode(/* * Log the new values stuffed into the inode. */ -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, ip,0 );xfs_trans_log_inode (tp, ip, flags);/* now that we have an i_mode we can setup the inode structure */
@@ -1076,6 +1078,7 @@ xfs_create(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1089,6 +1092,7 @@ xfs_create(if (ip) {xfs_finish_inode_setup (ip);xfs_irele (ip); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); } out_release_dquots:xfs_qm_dqrele (udqp);
@@ -1172,6 +1176,7 @@ xfs_create_tmpfile(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -1185,6 +1190,7 @@ xfs_create_tmpfile(if (ip) {xfs_finish_inode_setup (ip);xfs_irele (ip); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); } out_release_dquots:xfs_qm_dqrele (udqp);
fs/xfs/xfs_qm.c +3 -1
@@ -826,8 +826,10 @@ xfs_qm_qino_alloc(ASSERT (xfs_is_shutdown (mp));xfs_alert (mp,"%s failed (error %d)!" , __func__, error); } -if (need_alloc) +if (need_alloc) {xfs_finish_inode_setup (*ipp); +xfs_iunlock (*ipp,XFS_ILOCK_EXCL ); + }return error; }
fs/xfs/xfs_symlink.c +3 -0
@@ -337,6 +337,7 @@ xfs_symlink(xfs_qm_dqrele (pdqp); *ipp = ip; +xfs_iunlock (ip,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:
@@ -358,6 +359,8 @@ xfs_symlink(if (unlock_dp_on_error)xfs_iunlock (dp,XFS_ILOCK_EXCL ); +if (ip) +xfs_iunlock (ip,XFS_ILOCK_EXCL );return error; }
PATCH 5/28 xfs: Hold inode locks in xfs_trans_alloc_dir
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_trans_alloc_dir to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations
--- fs/xfs/xfs_inode.c | 14 ++++++++++++-- fs/xfs/xfs_trans.c | 9 +++++++-- 2 files changed, 19 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +12 -2
@@ -1279,10 +1279,15 @@ xfs_link(if (xfs_has_wsync (mp) ||xfs_has_dirsync (mp))xfs_trans_set_sync (tp); -return xfs_trans_commit (tp); + error =xfs_trans_commit (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); +return error; error_return:xfs_trans_cancel (tp); +xfs_iunlock (tdp,XFS_ILOCK_EXCL ); +xfs_iunlock (sip,XFS_ILOCK_EXCL ); std_return:if (error == -ENOSPC && nospace_error) error = nospace_error;
@@ -2518,15 +2523,20 @@ xfs_remove( error =xfs_trans_commit (tp);if (error) - goto std_return; + goto out_unlock;if (is_dir &&xfs_inode_is_filestream (ip))xfs_filestream_deassociate (ip); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL );return 0 ; out_trans_cancel:xfs_trans_cancel (tp); + out_unlock: +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); std_return:return error; }
fs/xfs/xfs_trans.c +7 -2
@@ -1356,6 +1356,8 @@ xfs_trans_alloc_ichange(* The caller must ensure that the on-disk dquots attached to this inode have * already been allocated and initialized. The ILOCKs will be dropped when the * transaction is committed or cancelled. +* +* Caller is responsible for unlocking the inodes manually upon return */ int xfs_trans_alloc_dir(
@@ -1386,8 +1388,8 @@ xfs_trans_alloc_dir(xfs_lock_two_inodes (dp,XFS_ILOCK_EXCL , ip,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, dp,0 ); +xfs_trans_ijoin (tp, ip,0 ); error =xfs_qm_dqattach_locked (dp, false);if (error) {
@@ -1410,6 +1412,9 @@ xfs_trans_alloc_dir(if (error == -EDQUOT || error == -ENOSPC ) {if (!retried) {xfs_trans_cancel (tp); +xfs_iunlock (dp,XFS_ILOCK_EXCL ); +if (dp != ip) +xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_blockgc_free_quota (dp,0 ); retried = true; goto retry;
PATCH 6/28 xfs: Hold inode locks in xfs_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Modify xfs_rename to hold all inode locks across a rename operation We will need this later when we add parent pointers
--- fs/xfs/xfs_inode.c | 43 ++++++++++++++++++++++++++++++------------- 1 file changed, 30 insertions(+), 13 deletions(-)
fs/xfs/xfs_inode.c +30 -13
@@ -2541,6 +2541,21 @@ xfs_remove(return error; } +static inline void +xfs_iunlock_rename ( +struct xfs_inode **i_tab, +int num_inodes) +{ +int i; + +for (i = num_inodes -1 ; i >=0 ; i--) { +/* Skip duplicate inodes if src and target dps are the same */ +if (!i_tab[i] || (i >0 && i_tab[i] == i_tab[i -1 ])) +continue ; +xfs_iunlock (i_tab[i],XFS_ILOCK_EXCL ); + } +} +/* * Enter all inodes for a rename transaction into a sorted array. */
@@ -2839,18 +2854,16 @@ xfs_rename(xfs_lock_inodes (inodes, num_inodes,XFS_ILOCK_EXCL );/* -* Join all the inodes to the transaction. From this point on, -* we can rely on either trans_commit or trans_cancel to unlock -* them. +* Join all the inodes to the transaction. */ -xfs_trans_ijoin (tp, src_dp,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, src_dp,0 );if (new_parent) -xfs_trans_ijoin (tp, target_dp,XFS_ILOCK_EXCL ); -xfs_trans_ijoin (tp, src_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_dp,0 ); +xfs_trans_ijoin (tp, src_ip,0 );if (target_ip) -xfs_trans_ijoin (tp, target_ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, target_ip,0 );if (wip) -xfs_trans_ijoin (tp, wip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, wip,0 );/* * If we are using project inheritance, we only allow renames
@@ -2864,10 +2877,12 @@ xfs_rename( }/* RENAME_EXCHANGE is unique from here on. */ -if (flags &RENAME_EXCHANGE ) -return xfs_cross_rename (tp, src_dp, src_name, src_ip, +if (flags &RENAME_EXCHANGE ) { + error =xfs_cross_rename (tp, src_dp, src_name, src_ip, target_dp, target_name, target_ip, spaceres); + goto out_unlock; + }/* * Try to reserve quota to handle an expansion of the target directory.
@@ -2881,6 +2896,7 @@ xfs_rename(if (error == -EDQUOT || error == -ENOSPC ) {if (!retried) {xfs_trans_cancel (tp); +xfs_iunlock_rename (inodes, num_inodes);xfs_blockgc_free_quota (target_dp,0 ); retried = true; goto retry;
@@ -3092,12 +3108,13 @@ xfs_rename(xfs_trans_log_inode (tp, target_dp,XFS_ILOG_CORE ); error =xfs_finish_rename (tp); -if (wip) -xfs_irele (wip); -return error; + + goto out_unlock; out_trans_cancel:xfs_trans_cancel (tp); +out_unlock: +xfs_iunlock_rename (inodes, num_inodes); out_release_wip:if (wip)xfs_irele (wip);
PATCH 7/28 xfs: Expose init_xattrs in xfs_create_tmpfile
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Tmp files are used as part of rename operations and will need attr forks initialized for parent pointers. Expose the init_xattrs parameter to the calling function to initialize the fork.
--- fs/xfs/xfs_inode.c | 5 +++-- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 3 ++- 3 files changed, 6 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +3 -2
@@ -1109,6 +1109,7 @@ xfs_create_tmpfile(struct user_namespace *mnt_userns, struct xfs_inode *dp, umode_t mode, + bool init_xattrs, struct xfs_inode **ipp) { struct xfs_mount *mp = dp->i_mount;
@@ -1149,7 +1150,7 @@ xfs_create_tmpfile( error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error =xfs_init_new_inode (mnt_userns, tp, dp, ino, mode, -0 ,0 , prid, false, &ip); +0 ,0 , prid, init_xattrs, &ip);if (error) goto out_trans_cancel;
@@ -2750,7 +2751,7 @@ xfs_rename_alloc_whiteout(int error; error =xfs_create_tmpfile (mnt_userns, dp,S_IFCHR |WHITEOUT_MODE , - &tmpfile); + false, &tmpfile);if (error)return error;
fs/xfs/xfs_inode.h +1 -1
@@ -478,7 +478,7 @@ int xfs_create(struct user_namespace *mnt_userns, umode_t mode, dev_t rdev, bool need_xattr, struct xfs_inode **ipp);int xfs_create_tmpfile (struct user_namespace *mnt_userns, -struct xfs_inode *dp,umode_t mode, +struct xfs_inode *dp,umode_t mode,bool init_xattrs,struct xfs_inode **ipp);int xfs_remove (struct xfs_inode *dp,struct xfs_name *name,struct xfs_inode *ip);
fs/xfs/xfs_iops.c +2 -1
@@ -200,7 +200,8 @@ xfs_generic_create(xfs_create_need_xattr (dir, default_acl, acl), &ip); }else { - error =xfs_create_tmpfile (mnt_userns,XFS_I (dir), mode, &ip); + error =xfs_create_tmpfile (mnt_userns,XFS_I (dir), mode, true, + &ip); }if (unlikely (error)) goto out_free_acl;
PATCH 8/28 xfs: get directory offset when adding directory name
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Return the directory offset information when adding an entry to the directory.
This offset will be used as the parent pointer offset in xfs_create, xfs_symlink, xfs_link and xfs_rename.
--- fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_dir2.c | 9 +++++++-- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_dir2_block.c | 1 + fs/xfs/libxfs/xfs_dir2_leaf.c | 2 ++ fs/xfs/libxfs/xfs_dir2_node.c | 2 ++ fs/xfs/libxfs/xfs_dir2_sf.c | 2 ++ fs/xfs/xfs_inode.c | 6 +++--- fs/xfs/xfs_symlink.c | 3 ++- 9 files changed, 21 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_da_btree.h +1 -0
@@ -81,6 +81,7 @@ typedef struct xfs_da_args {int rmtvaluelen2;/* remote attr value length in bytes */ uint32_t op_flags;/* operation flags */ enum xfs_dacmp cmpresult;/* name compare result for lookups */ +xfs_dir2_dataptr_t offset;/* OUT: offset in directory */ } xfs_da_args_t;/*
fs/xfs/libxfs/xfs_dir2.c +7 -2
@@ -257,7 +257,8 @@ xfs_dir_createname(struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t inum, /* new entry inode number */ - xfs_extlen_t total) /* bmap's total block count */ + xfs_extlen_t total, /* bmap's total block count */ + xfs_dir2_dataptr_t *offset) /* OUT entry's dir offset */ { struct xfs_da_args *args;int rval;
@@ -312,6 +313,10 @@ xfs_dir_createname( rval =xfs_dir2_node_addname (args); out_free: +/* return the location that this entry was place in the parent inode */ +if (offset) + *offset = args->offset; +kmem_free (args);return rval; }
@@ -550,7 +555,7 @@ xfs_dir_canenter(xfs_inode_t *dp, struct xfs_name *name) /* name of entry to add */ { - return xfs_dir_createname(tp, dp, name, 0, 0); + return xfs_dir_createname(tp, dp, name, 0, 0, NULL); }/*
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -40,7 +40,7 @@ extern int xfs_dir_init(struct xfs_trans *tp, struct xfs_inode *dp, struct xfs_inode *pdp);extern int xfs_dir_createname (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t inum, -xfs_extlen_t tot); +xfs_extlen_t tot,xfs_dir2_dataptr_t *offset);extern int xfs_dir_lookup (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t *inum,struct xfs_name *ci_name);
fs/xfs/libxfs/xfs_dir2_block.c +1 -0
@@ -573,6 +573,7 @@ xfs_dir2_block_addname(xfs_dir2_data_put_ftype (dp->i_mount, dep, args->filetype); tagp =xfs_dir2_data_entry_tag_p (dp->i_mount, dep); *tagp =cpu_to_be16 ((char *)dep - (char *)hdr); + args->offset =xfs_dir2_byte_to_dataptr ((char *)dep - (char *)hdr);/* * Clean up the bestfree array and log the header, tail, and entry. */
fs/xfs/libxfs/xfs_dir2_leaf.c +2 -0
@@ -870,6 +870,8 @@ xfs_dir2_leaf_addname(xfs_dir2_data_put_ftype (dp->i_mount, dep, args->filetype); tagp =xfs_dir2_data_entry_tag_p (dp->i_mount, dep); *tagp =cpu_to_be16 ((char *)dep - (char *)hdr); + args->offset =xfs_dir2_db_off_to_dataptr (args->geo, use_block, + (char *)dep - (char *)hdr);/* * Need to scan fix up the bestfree table. */
fs/xfs/libxfs/xfs_dir2_node.c +2 -0
@@ -1974,6 +1974,8 @@ xfs_dir2_node_addname_int(xfs_dir2_data_put_ftype (dp->i_mount, dep, args->filetype); tagp =xfs_dir2_data_entry_tag_p (dp->i_mount, dep); *tagp =cpu_to_be16 ((char *)dep - (char *)hdr); + args->offset =xfs_dir2_db_off_to_dataptr (args->geo, dbno, + (char *)dep - (char *)hdr);xfs_dir2_data_log_entry (args, dbp, dep);/* Rescan the freespace and log the data block if needed. */
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
@@ -485,6 +485,7 @@ xfs_dir2_sf_addname_easy(memcpy (sfep->name, args->name, sfep->namelen);xfs_dir2_sf_put_ino (mp, sfp, sfep, args->inumber);xfs_dir2_sf_put_ftype (mp, sfep, args->filetype); + args->offset =xfs_dir2_byte_to_dataptr (offset);/* * Update the header and inode.
@@ -575,6 +576,7 @@ xfs_dir2_sf_addname_hard(memcpy (sfep->name, args->name, sfep->namelen);xfs_dir2_sf_put_ino (mp, sfp, sfep, args->inumber);xfs_dir2_sf_put_ftype (mp, sfep, args->filetype); + args->offset =xfs_dir2_byte_to_dataptr (offset); sfp->count++;if (args->inumber >XFS_DIR2_MAX_SHORT_INUM && !objchange) sfp->i8count++;
fs/xfs/xfs_inode.c +3 -3
@@ -1038,7 +1038,7 @@ xfs_create( unlock_dp_on_error = false; error =xfs_dir_createname (tp, dp, name, ip->i_ino, - resblks -XFS_IALLOC_SPACE_RES (mp)); + resblks -XFS_IALLOC_SPACE_RES (mp),NULL );if (error) {ASSERT (error != -ENOSPC ); goto out_trans_cancel;
@@ -1264,7 +1264,7 @@ xfs_link( } error =xfs_dir_createname (tp, tdp, target_name, sip->i_ino, - resblks); + resblks,NULL );if (error) goto error_return;xfs_trans_ichgtime (tp, tdp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );
@@ -3001,7 +3001,7 @@ xfs_rename(* to account for the ".." reference from the new entry. */ error =xfs_dir_createname (tp, target_dp, target_name, - src_ip->i_ino, spaceres); + src_ip->i_ino, spaceres,NULL );if (error) goto out_trans_cancel;
fs/xfs/xfs_symlink.c +2 -1
@@ -314,7 +314,8 @@ xfs_symlink(/* * Create the directory entry for the symlink. */ - error =xfs_dir_createname (tp, dp, link_name, ip->i_ino, resblks); + error =xfs_dir_createname (tp, dp, link_name, + ip->i_ino, resblks,NULL );if (error) goto out_trans_cancel;xfs_trans_ichgtime (tp, dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );
PATCH 9/28 xfs: get directory offset when removing directory name
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Return the directory offset information when removing an entry to the directory.
This offset will be used as the parent pointer offset in xfs_remove.
--- fs/xfs/libxfs/xfs_dir2.c | 6 +++++- fs/xfs/libxfs/xfs_dir2.h | 3 ++- fs/xfs/libxfs/xfs_dir2_block.c | 4 ++-- fs/xfs/libxfs/xfs_dir2_leaf.c | 5 +++-- fs/xfs/libxfs/xfs_dir2_node.c | 5 +++-- fs/xfs/libxfs/xfs_dir2_sf.c | 2 ++ fs/xfs/xfs_inode.c | 4 ++-- 7 files changed, 19 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +5 -1
@@ -436,7 +436,8 @@ xfs_dir_removename(struct xfs_inode *dp, struct xfs_name *name, xfs_ino_t ino, - xfs_extlen_t total) /* bmap's total block count */ + xfs_extlen_t total, /* bmap's total block count */ + xfs_dir2_dataptr_t *offset) /* OUT: offset in directory */ { struct xfs_da_args *args;int rval;
@@ -481,6 +482,9 @@ xfs_dir_removename(else rval =xfs_dir2_node_removename (args); out_free: +if (offset) + *offset = args->offset; +kmem_free (args);return rval; }
fs/xfs/libxfs/xfs_dir2.h +2 -1
@@ -46,7 +46,8 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp, struct xfs_name *ci_name);extern int xfs_dir_removename (struct xfs_trans *tp,struct xfs_inode *dp,struct xfs_name *name,xfs_ino_t ino, -xfs_extlen_t tot); +xfs_extlen_t tot, +xfs_dir2_dataptr_t *offset);extern int xfs_dir_replace (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t inum,xfs_extlen_t tot);
fs/xfs/libxfs/xfs_dir2_block.c +2 -2
@@ -810,9 +810,9 @@ xfs_dir2_block_removename(/* * Point to the data entry using the leaf entry. */ + args->offset =be32_to_cpu (blp[ent].address); dep = (xfs_dir2_data_entry_t *)((char *)hdr + -xfs_dir2_dataptr_to_off (args->geo, -be32_to_cpu (blp[ent].address))); +xfs_dir2_dataptr_to_off (args->geo, args->offset));/* * Mark the data entry's space free. */
fs/xfs/libxfs/xfs_dir2_leaf.c +3 -2
@@ -1386,9 +1386,10 @@ xfs_dir2_leaf_removename(* Point to the leaf entry, use that to point to the data entry. */ lep = &leafhdr.ents[index]; - db =xfs_dir2_dataptr_to_db (geo,be32_to_cpu (lep->address)); + args->offset =be32_to_cpu (lep->address); + db =xfs_dir2_dataptr_to_db (args->geo, args->offset); dep = (xfs_dir2_data_entry_t *)((char *)hdr + -xfs_dir2_dataptr_to_off (geo,be32_to_cpu (lep->address))); +xfs_dir2_dataptr_to_off (args->geo, args->offset)); needscan = needlog =0 ; oldbest =be16_to_cpu (bf[0 ].length); ltp =xfs_dir2_leaf_tail_p (geo, leaf);
fs/xfs/libxfs/xfs_dir2_node.c +3 -2
@@ -1296,9 +1296,10 @@ xfs_dir2_leafn_remove(/* * Extract the data block and offset from the entry. */ - db =xfs_dir2_dataptr_to_db (geo,be32_to_cpu (lep->address)); + args->offset =be32_to_cpu (lep->address); + db =xfs_dir2_dataptr_to_db (args->geo, args->offset);ASSERT (dblk->blkno == db); - off =xfs_dir2_dataptr_to_off (geo,be32_to_cpu (lep->address)); + off =xfs_dir2_dataptr_to_off (args->geo, args->offset);ASSERT (dblk->index == off);/*
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
@@ -969,6 +969,8 @@ xfs_dir2_sf_removename( XFS_CMP_EXACT) {ASSERT (xfs_dir2_sf_get_ino (mp, sfp, sfep) == args->inumber); + args->offset =xfs_dir2_byte_to_dataptr ( +xfs_dir2_sf_get_offset (sfep));break ; } }
fs/xfs/xfs_inode.c +2 -2
@@ -2508,7 +2508,7 @@ xfs_remove(if (error) goto out_trans_cancel; - error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks); + error =xfs_dir_removename (tp, dp, name, ip->i_ino, resblks,NULL );if (error) {ASSERT (error != -ENOENT ); goto out_trans_cancel;
@@ -3098,7 +3098,7 @@ xfs_rename( spaceres); else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, - spaceres); + spaceres,NULL );if (error) goto out_trans_cancel;
PATCH 10/28 xfs: get directory offset when replacing a directory name
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Return the directory offset information when replacing an entry to the directory.
This offset will be used as the parent pointer offset in xfs_rename.
--- fs/xfs/libxfs/xfs_dir2.c | 8 ++++++-- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_dir2_block.c | 4 ++-- fs/xfs/libxfs/xfs_dir2_leaf.c | 1 + fs/xfs/libxfs/xfs_dir2_node.c | 1 + fs/xfs/libxfs/xfs_dir2_sf.c | 2 ++ fs/xfs/xfs_inode.c | 16 ++++++++-------- 7 files changed, 21 insertions(+), 13 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +6 -2
@@ -482,7 +482,7 @@ xfs_dir_removename(else rval =xfs_dir2_node_removename (args); out_free: -if (offset) +if (!rval && offset) *offset = args->offset;kmem_free (args);
@@ -498,7 +498,8 @@ xfs_dir_replace(struct xfs_inode *dp, const struct xfs_name *name, /* name of entry to replace */ xfs_ino_t inum, /* new inode number */ - xfs_extlen_t total) /* bmap's total block count */ + xfs_extlen_t total, /* bmap's total block count */ + xfs_dir2_dataptr_t *offset) /* OUT: offset in directory */ { struct xfs_da_args *args;int rval;
@@ -546,6 +547,9 @@ xfs_dir_replace(else rval =xfs_dir2_node_replace (args); out_free: +if (offset) + *offset = args->offset; +kmem_free (args);return rval; }
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -50,7 +50,7 @@ extern int xfs_dir_removename(struct xfs_trans *tp, struct xfs_inode *dp, xfs_dir2_dataptr_t *offset);extern int xfs_dir_replace (struct xfs_trans *tp,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t inum, -xfs_extlen_t tot); +xfs_extlen_t tot,xfs_dir2_dataptr_t *offset);extern int xfs_dir_canenter (struct xfs_trans *tp,struct xfs_inode *dp,struct xfs_name *name);
fs/xfs/libxfs/xfs_dir2_block.c +2 -2
@@ -885,9 +885,9 @@ xfs_dir2_block_replace(/* * Point to the data entry we need to change. */ + args->offset =be32_to_cpu (blp[ent].address); dep = (xfs_dir2_data_entry_t *)((char *)hdr + -xfs_dir2_dataptr_to_off (args->geo, -be32_to_cpu (blp[ent].address))); +xfs_dir2_dataptr_to_off (args->geo, args->offset));ASSERT (be64_to_cpu (dep->inumber) != args->inumber);/* * Change the inode number to the new value.
fs/xfs/libxfs/xfs_dir2_leaf.c +1 -0
@@ -1523,6 +1523,7 @@ xfs_dir2_leaf_replace(/* * Point to the data entry. */ + args->offset =be32_to_cpu (lep->address); dep = (xfs_dir2_data_entry_t *) ((char *)dbp->b_addr +xfs_dir2_dataptr_to_off (args->geo,be32_to_cpu (lep->address)));
fs/xfs/libxfs/xfs_dir2_node.c +1 -0
@@ -2242,6 +2242,7 @@ xfs_dir2_node_replace( hdr = state->extrablk.bp->b_addr;ASSERT (hdr->magic ==cpu_to_be32 (XFS_DIR2_DATA_MAGIC ) || hdr->magic ==cpu_to_be32 (XFS_DIR3_DATA_MAGIC )); + args->offset =be32_to_cpu (leafhdr.ents[blk->index].address); dep = (xfs_dir2_data_entry_t *) ((char *)hdr + xfs_dir2_dataptr_to_off(args->geo,
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
@@ -1107,6 +1107,8 @@ xfs_dir2_sf_replace(xfs_dir2_sf_put_ino (mp, sfp, sfep, args->inumber);xfs_dir2_sf_put_ftype (mp, sfep, args->filetype); + args->offset =xfs_dir2_byte_to_dataptr ( +xfs_dir2_sf_get_offset (sfep));break ; } }
fs/xfs/xfs_inode.c +8 -8
@@ -2489,7 +2489,7 @@ xfs_remove(*/ if (dp->i_ino != tp->t_mountp->m_sb.sb_rootino) { error =xfs_dir_replace (tp, ip, &xfs_name_dotdot, - tp->t_mountp->m_sb.sb_rootino,0 ); + tp->t_mountp->m_sb.sb_rootino,0 ,NULL );if (error) goto out_trans_cancel; }
@@ -2644,12 +2644,12 @@ xfs_cross_rename(int dp2_flags =0 ;/* Swap inode number for dirent in first parent */ - error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres); + error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres,NULL );if (error) goto out_trans_abort;/* Swap inode number for dirent in second parent */ - error =xfs_dir_replace (tp, dp2, name2, ip1->i_ino, spaceres); + error =xfs_dir_replace (tp, dp2, name2, ip1->i_ino, spaceres,NULL );if (error) goto out_trans_abort;
@@ -2663,7 +2663,7 @@ xfs_cross_rename(if (S_ISDIR (VFS_I (ip2)->i_mode)) { error =xfs_dir_replace (tp, ip2, &xfs_name_dotdot, - dp1->i_ino, spaceres); + dp1->i_ino, spaceres,NULL );if (error) goto out_trans_abort;
@@ -2687,7 +2687,7 @@ xfs_cross_rename(if (S_ISDIR (VFS_I (ip1)->i_mode)) { error =xfs_dir_replace (tp, ip1, &xfs_name_dotdot, - dp2->i_ino, spaceres); + dp2->i_ino, spaceres,NULL );if (error) goto out_trans_abort;
@@ -3022,7 +3022,7 @@ xfs_rename(* name at the destination directory, remove it first. */ error =xfs_dir_replace (tp, target_dp, target_name, - src_ip->i_ino, spaceres); + src_ip->i_ino, spaceres,NULL );if (error) goto out_trans_cancel;
@@ -3056,7 +3056,7 @@ xfs_rename(* directory. */ error =xfs_dir_replace (tp, src_ip, &xfs_name_dotdot, - target_dp->i_ino, spaceres); + target_dp->i_ino, spaceres,NULL );ASSERT (error != -EEXIST );if (error) goto out_trans_cancel;
@@ -3095,7 +3095,7 @@ xfs_rename(*/ if (wip) error =xfs_dir_replace (tp, src_dp, src_name, wip->i_ino, - spaceres); + spaceres,NULL );else error =xfs_dir_removename (tp, src_dp, src_name, src_ip->i_ino, spaceres,NULL );
PATCH 11/28 xfs: add parent pointer support to attribute code
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.
--- fs/xfs/libxfs/xfs_attr.c | 4 +++- fs/xfs/libxfs/xfs_da_format.h | 5 ++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/scrub/attr.c | 2 +- 4 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.c +3 -1
@@ -976,11 +976,13 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool rsvd;int error, local;int rmt_blks =0 ;unsigned int total; + rsvd = (args->attr_filter & (XFS_ATTR_ROOT |XFS_ATTR_PARENT )) !=0 ; +if (xfs_is_shutdown (dp->i_mount))return -EIO ;
fs/xfs/libxfs/xfs_da_format.h +4 -1
@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)/* * Alignment for namelist and valuelist entries (since they are mixed
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -967,6 +967,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/*
fs/xfs/scrub/attr.c +1 -1
@@ -441,7 +441,7 @@ xchk_xattr_rec(/* Retrieve the entry and check it. */ hash =be32_to_cpu (ent->hashval); badflags = ~(XFS_ATTR_LOCAL |XFS_ATTR_ROOT |XFS_ATTR_SECURE | -XFS_ATTR_INCOMPLETE ); +XFS_ATTR_INCOMPLETE |XFS_ATTR_PARENT );if ((ent->flags & badflags) !=0 )xchk_da_set_corrupt (ds, level);if (ent->flags &XFS_ATTR_LOCAL ) {
PATCH 12/28 xfs: define parent pointer xattr format
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:
name={parent inode #, parent inode generation, dirent offset}
value={dirent filename}The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls.
By using the dirent offset in the EA name, we have a method of knowing the exact parent pointer EA we need to modify/remove in rename/unlink without an unbound EA name search.
By keeping the dirent name in the value, we have enough information to be able to validate and reconstruct damaged directory trees. While the diroffset of a filename alone is not unique enough to identify the child, the {diroffset,filename,child_inode} tuple is sufficient. That is, if the diroffset gets reused and points to a different filename, we can detect that from the contents of EA. If a link of the same name is created, then we can check whether it points at the same inode as the parent EA we current have.
--- fs/xfs/libxfs/xfs_da_format.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +25 -0
@@ -805,4 +805,29 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* +* Parent pointer attribute format definition +* +* EA name encodes the parent inode number, generation and the offset of +* the dirent that points to the child inode. The EA value contains the +* same name as the dirent in the parent directory. +*/ +struct xfs_parent_name_rec { +__be64 p_ino; +__be32 p_gen; +__be32 p_diroffset; +}; + +/* +* incore version of the above, also contains name pointers so callers +* can pass/obtain all the parent pointer information in a single structure +*/ +struct xfs_parent_name_irec { +xfs_ino_t p_ino; +uint32_t p_gen; +xfs_dir2_dataptr_t p_diroffset; +const char *p_name; +uint8_t p_namelen; +}; +#endif /* __XFS_DA_FORMAT_H__ */
PATCH 13/28 xfs: Add xfs_verify_pptr
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set.
--- fs/xfs/libxfs/xfs_attr.c | 47 ++++++++++++++++++++++++++++++++++++++--- fs/xfs/libxfs/xfs_attr.h | 3 ++- fs/xfs/libxfs/xfs_da_format.h | 8 +++++++ fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_attr_item.c | 11 ++++++---- fs/xfs/xfs_attr_list.c | 17 ++++++++++----- 6 files changed, 74 insertions(+), 14 deletions(-)
fs/xfs/libxfs/xfs_attr.c +44 -3
@@ -1577,9 +1577,33 @@ xfs_attr_node_get(return error; } -/* Returns true if the attribute entry name is valid. */ -bool -xfs_attr_namecheck ( +/* +* Verify parent pointer attribute is valid. +* Return true on success or false on failure +*/ +STATIC bool +xfs_verify_pptr ( +struct xfs_mount *mp, +const struct xfs_parent_name_rec *rec) +{ +xfs_ino_t p_ino; +xfs_dir2_dataptr_t p_diroffset; + + p_ino =be64_to_cpu (rec->p_ino); + p_diroffset =be32_to_cpu (rec->p_diroffset); + +if (!xfs_verify_ino (mp, p_ino)) +return false; + +if (p_diroffset >XFS_DIR2_MAX_DATAPTR ) +return false; + +return true; +} + +/* Returns true if the string attribute entry name is valid. */ +static bool +xfs_str_attr_namecheck (const void *name,size_t length) {
@@ -1594,6 +1618,23 @@ xfs_attr_namecheck(return !memchr (name,0 , length); } +/* Returns true if the attribute entry name is valid. */ +bool +xfs_attr_namecheck ( +struct xfs_mount *mp, +const void *name, +size_t length, +int flags) +{ +if (flags &XFS_ATTR_PARENT ) { +if (length !=sizeof (struct xfs_parent_name_rec )) +return false; +return xfs_verify_pptr (mp, (struct xfs_parent_name_rec *)name); + } + +return xfs_str_attr_namecheck (name, length); +} +int __init xfs_attr_intent_init_cache (void ) {
fs/xfs/libxfs/xfs_attr.h +2 -1
@@ -547,7 +547,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
fs/xfs/libxfs/xfs_da_format.h +8 -0
@@ -731,6 +731,14 @@ xfs_attr3_leaf_name(xfs_attr_leafblock_t *leafp, int idx)return &((char *)leafp)[be16_to_cpu (entries[idx].nameidx)]; } +static inline int +xfs_attr3_leaf_flags (xfs_attr_leafblock_t *leafp,int idx) +{ +struct xfs_attr_leaf_entry *entries =xfs_attr3_leaf_entryp (leafp); + +return entries[idx].flags; +} +static inline xfs_attr_leaf_name_remote_t *xfs_attr3_leaf_name_remote (xfs_attr_leafblock_t *leafp,int idx) {
fs/xfs/scrub/attr.c +1 -1
@@ -129,7 +129,7 @@ xchk_xattr_listent( }/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (sx->sc->mp, name, namelen, flags)) {xchk_fblock_set_corrupt (sx->sc,XFS_ATTR_FORK , args.blkno);return ; }
fs/xfs/xfs_attr_item.c +7 -4
@@ -593,7 +593,8 @@ xfs_attri_item_recover(*/ attrp = &attrip->attri_format;if (!xfs_attri_validate (mp, attrp) || - !xfs_attr_namecheck (nv->name.i_addr, nv->name.i_len)) + !xfs_attr_namecheck (mp, nv->name.i_addr, nv->name.i_len, + attrp->alfi_attr_filter))return -EFSCORRUPTED ; error =xlog_recover_iget (mp, attrp->alfi_ino, &ip);
@@ -804,7 +805,8 @@ xlog_recover_attri_commit_pass2( } attr_name = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_name, attri_formatp->alfi_name_len)) { +if (!xfs_attr_namecheck (mp, attr_name, attri_formatp->alfi_name_len, + attri_formatp->alfi_attr_filter)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[i].i_addr, item->ri_buf[i].i_len);return -EFSCORRUPTED ;
@@ -822,8 +824,9 @@ xlog_recover_attri_commit_pass2( } attr_nname = item->ri_buf[i].i_addr; -if (!xfs_attr_namecheck (attr_nname, - attri_formatp->alfi_nname_len)) { +if (!xfs_attr_namecheck (mp, attr_nname, + attri_formatp->alfi_nname_len, + attri_formatp->alfi_attr_filter)) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, item->ri_buf[i].i_addr, item->ri_buf[i].i_len);
fs/xfs/xfs_attr_list.c +12 -5
@@ -58,9 +58,13 @@ xfs_attr_shortform_list(struct xfs_attr_sf_sort *sbuf, *sbp;struct xfs_attr_shortform *sf;struct xfs_attr_sf_entry *sfe; +struct xfs_mount *mp;int sbsize, nsbuf, count, i;int error =0 ; +ASSERT (context !=NULL ); +ASSERT (dp !=NULL ); + mp = dp->i_mount; sf = (struct xfs_attr_shortform *)dp->i_af.if_u1.if_data;ASSERT (sf !=NULL );if (!sf->hdr.count)
@@ -82,8 +86,9 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->hdr.count * 16) < context->bufsize)) {for (i =0 , sfe = &sf->list[0 ]; i < sf->hdr.count; i++) {if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sfe->nameval, - sfe->namelen))) + !xfs_attr_namecheck (mp, sfe->nameval, + sfe->namelen, + sfe->flags)))return -EFSCORRUPTED ; context->put_listent(context, sfe->flags,
@@ -174,8 +179,9 @@ xfs_attr_shortform_list( cursor->offset =0 ; }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, - sbp->namelen))) { + !xfs_attr_namecheck (mp, sbp->name, + sbp->namelen, + sbp->flags))) { error = -EFSCORRUPTED ; goto out; }
@@ -465,7 +471,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) + !xfs_attr_namecheck (mp, name, namelen, + entry->flags)))return -EFSCORRUPTED ; context->put_listent (context, entry->flags, name, namelen, valuelen);
PATCH 14/28 xfs: extend transaction reservations for parent attributes
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
--- fs/xfs/libxfs/xfs_trans_resv.c | 324 ++++++++++++++++++++++++++++++++++------ 1 file changed, 272 insertions(+), 52 deletions(-)
fs/xfs/libxfs/xfs_trans_resv.c +272 -52
@@ -19,6 +19,9 @@#include "xfs_trans.h" #include "xfs_qm.h" #include "xfs_trans_space.h" +#include "xfs_attr_item.h" +#include "xfs_log.h" +#include "xfs_da_format.h" #define _ALLOC true#define _FREE false
@@ -420,29 +423,108 @@ xfs_calc_itruncate_reservation_minlogsize(return xfs_calc_itruncate_reservation (mp, true); } +static inline unsigned int xfs_calc_pptr_link_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (XATTR_NAME_MAX ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )); +} +static inline unsigned int xfs_calc_pptr_unlink_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )); +} +static inline unsigned int xfs_calc_pptr_replace_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (XATTR_NAME_MAX ) + +xlog_calc_iovec_len (XATTR_NAME_MAX ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )); +} +/* * In renaming a files we can modify: * the five inodes involved: 5 * inode size * the two directory btrees: 2 * (max depth + v2) * dir block size * the two directory bmap btrees: 2 * max depth * block size * And the bmap_finish transaction can free dir and bmap blocks (two sets -* of bmap blocks) giving: +* of bmap blocks) giving (t2): * the agf for the ags in which the blocks live: 3 * sector size * the agfl for the ags in which the blocks live: 3 * sector size * the superblock for the free block count: sector size * the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) * block size +* If parent pointers are enabled (t3), then each transaction in the chain +* must be capable of setting or removing the extended attribute +* containing the parent information. It must also be able to handle +* the three xattr intent items that track the progress of the parent +* pointer update. */ STATIC uintxfs_calc_rename_reservation (struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max ((xfs_calc_inode_res (mp,5 ) + -xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_inode_res (mp,5 ) + +xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), +XFS_FSB_TO_B (mp,1 )); + + t2 =xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { +unsigned int rename_overhead, exchange_overhead; + + t3 =max (resp->tr_attrsetm.tr_logres, + resp->tr_attrrm.tr_logres); + +/* +* For a standard rename, the three xattr intent log items +* are (1) replacing the pptr for the source file; (2) +* removing the pptr on the dest file; and (3) adding a +* pptr for the whiteout file in the src dir. +* +* For an RENAME_EXCHANGE, there are two xattr intent +* items to replace the pptr for both src and dest +* files. Link counts don't change and there is no +* whiteout. +* +* In the worst case we can end up relogging all log +* intent items to allow the log tail to move ahead, so +* they become overhead added to each transaction in a +* processing chain. +*/ + rename_overhead =xfs_calc_pptr_replace_overhead () + +xfs_calc_pptr_unlink_overhead () + +xfs_calc_pptr_link_overhead (); + exchange_overhead =2 *xfs_calc_pptr_replace_overhead (); + + overhead +=max (rename_overhead, exchange_overhead); + } + +return overhead +max3 (t1, t2, t3); +} + +static inline unsigned int +xfs_rename_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +/* One for the rename, one more for freeing blocks */ +unsigned int ret =XFS_RENAME_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to remove or add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret +=max (resp->tr_attrsetm.tr_logcount, + resp->tr_attrrm.tr_logcount); + +return ret; }/*
@@ -459,6 +541,23 @@ xfs_calc_iunlink_remove_reservation(2 *M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_link_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_LINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * For creating a link to an inode: * the parent directory inode: inode size
@@ -475,14 +574,23 @@ STATIC uintxfs_calc_link_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_remove_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_remove_reservation (mp); + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -497,6 +605,23 @@ xfs_calc_iunlink_add_reservation(xfs_mount_t *mp)M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_remove_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_REMOVE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrrm.tr_logcount; + +return ret; +} +/* * For removing a directory entry we can modify: * the parent directory inode: inode size
@@ -513,14 +638,24 @@ STATIC uintxfs_calc_remove_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_add_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_add_reservation (mp); + + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrrm.tr_logres; + overhead +=xfs_calc_pptr_unlink_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -569,12 +704,40 @@ xfs_calc_icreate_resv_alloc(xfs_calc_finobt_res (mp); } +static inline unsigned int +xfs_icreate_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_CREATE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} + STATIC uint -xfs_calc_icreate_reservation (xfs_mount_t *mp) +xfs_calc_icreate_reservation ( +struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max (xfs_calc_icreate_resv_alloc (mp), -xfs_calc_create_resv_modify (mp)); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_icreate_resv_alloc (mp); + t2 =xfs_calc_create_resv_modify (mp); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }STATIC uint
@@ -587,6 +750,23 @@ xfs_calc_create_tmpfile_reservation(return res +xfs_calc_iunlink_add_reservation (mp); } +static inline unsigned int +xfs_mkdir_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_MKDIR_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * Making a new directory is the same as creating a new file. */
@@ -597,6 +777,22 @@ xfs_calc_mkdir_reservation(return xfs_calc_icreate_reservation (mp); } +static inline unsigned int +xfs_symlink_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_SYMLINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +}/* * Making a new symplink is the same as creating a new file, but
@@ -909,6 +1105,52 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute +* modifications occurring from within these transactions. Rather than confuse +* each of these reservation calculations with the conditional attribute +* reservations, add them here in a clear and concise manner. This requires that +* the attribute reservations have already been calculated. +* +* Note that we only include the static attribute reservation here; the runtime +* reservation will have to be modified by the size of the attributes being +* added/removed/modified. See the comments on the attribute reservation +* calculations for more details. +*/ +STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); + + resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); + resp->tr_rename.tr_logcount =xfs_rename_log_count (mp, resp); + resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); + resp->tr_link.tr_logcount =xfs_link_log_count (mp, resp); + resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); + resp->tr_remove.tr_logcount =xfs_remove_log_count (mp, resp); + resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); + resp->tr_symlink.tr_logcount =xfs_symlink_log_count (mp, resp); + resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); + resp->tr_create.tr_logcount =xfs_icreate_log_count (mp, resp); + resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =xfs_mkdir_log_count (mp, resp); + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +} + void xfs_trans_resv_calc( struct xfs_mount *mp,
@@ -928,35 +1170,11 @@ xfs_trans_resv_calc( resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); - resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ; - resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); - resp->tr_link.tr_logcount =XFS_LINK_LOG_COUNT ; - resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); - resp->tr_remove.tr_logcount =XFS_REMOVE_LOG_COUNT ; - resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); - resp->tr_symlink.tr_logcount =XFS_SYMLINK_LOG_COUNT ; - resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); - resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; - resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;
@@ -986,6 +1204,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count.
PATCH 15/28 xfs: parent pointer attribute creation
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add parent pointer attribute during xfs_create, and subroutines to initialize attributes
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_attr.c | 4 + fs/xfs/libxfs/xfs_attr.h | 4 + fs/xfs/libxfs/xfs_da_format.h | 12 ---- fs/xfs/libxfs/xfs_parent.c | 139 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 57 +++++++++++++++++ fs/xfs/xfs_inode.c | 64 ++++++++++++++++--- fs/xfs/xfs_super.c | 10 +++ fs/xfs/xfs_xattr.c | 4 + fs/xfs/xfs_xattr.h | 2 + 10 files changed, 271 insertions(+), 26 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h
fs/xfs/Makefile +1 -0
@@ -40,6 +40,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_fork.o \ xfs_inode_buf.o \ xfs_log_rlimit.o \ + xfs_parent.o \ xfs_ag_resv.o \ xfs_rmap.o \ xfs_rmap_btree.o \
fs/xfs/libxfs/xfs_attr.c +2 -2
@@ -886,7 +886,7 @@ xfs_attr_lookup(return error; } -static int +int xfs_attr_intent_init( struct xfs_da_args *args, unsigned int op_flags, /* op flag (set or remove) */
@@ -904,7 +904,7 @@ xfs_attr_intent_init( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_add (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +3 -1
@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); - +int xfs_attr_intent_init (struct xfs_da_args *args,unsigned int op_flags, +struct xfs_attr_intent **attr);/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list.
fs/xfs/libxfs/xfs_da_format.h +0 -12
@@ -826,16 +826,4 @@ struct xfs_parent_name_rec { __be32 p_diroffset; }; -/* -* incore version of the above, also contains name pointers so callers -* can pass/obtain all the parent pointer information in a single structure -*/ -struct xfs_parent_name_irec { -xfs_ino_t p_ino; -uint32_t p_gen; -xfs_dir2_dataptr_t p_diroffset; -const char *p_name; -uint8_t p_namelen; -}; -#endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_parent.c new file +139 -0
139 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h new file +57 -0
57 lines not on this page · raw message
fs/xfs/xfs_inode.c +55 -9
143 lines not on this page · raw message
fs/xfs/xfs_super.c +10 -0
@@ -41,6 +41,7 @@#include "xfs_attr_item.h" #include "xfs_xattr.h" #include "xfs_iunlink_item.h" +#include "xfs_parent.h" #include <linux/magic.h> #include <linux/fs_context.h>
@@ -2115,8 +2116,16 @@ xfs_init_caches(void)if (!xfs_iunlink_cache) goto out_destroy_attri_cache; + xfs_parent_intent_cache =kmem_cache_create ("xfs_parent_intent" , +sizeof (struct xfs_parent_defer ), +0 ,0 ,NULL ); +if (!xfs_parent_intent_cache) + goto out_destroy_iul_cache; +return 0 ; + out_destroy_iul_cache: +kmem_cache_destroy (xfs_iunlink_cache); out_destroy_attri_cache:kmem_cache_destroy (xfs_attri_cache); out_destroy_attrd_cache:
@@ -2171,6 +2180,7 @@ xfs_destroy_caches(void)* destroy caches. */ rcu_barrier (); +kmem_cache_destroy (xfs_parent_intent_cache);kmem_cache_destroy (xfs_iunlink_cache);kmem_cache_destroy (xfs_attri_cache);kmem_cache_destroy (xfs_attrd_cache);
fs/xfs/xfs_xattr.c +2 -2
@@ -27,7 +27,7 @@* they must release the permission by calling xlog_drop_incompat_feat * when they're done. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {
@@ -61,7 +61,7 @@ xfs_attr_grab_log_assist(return error; } -static inline void +void xfs_attr_rele_log_assist (struct xfs_mount *mp) {
fs/xfs/xfs_xattr.h +2 -0
@@ -7,6 +7,8 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp); +void xfs_attr_rele_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *xfs_xattr_handlers[];
PATCH 16/28 xfs: add parent attributes to link
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_link to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.h | 2 - fs/xfs/xfs_inode.c | 60 ++++++++++++++++++++++++++++++++++----- 2 files changed, 53 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -84,8 +84,6 @@ (2 * (mp)->m_alloc_maxlevels)#define XFS_GROWFSRT_SPACE_RES (mp,b) \ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK)) -#define XFS_LINK_SPACE_RES (mp,nl) \ - XFS_DIRENTER_SPACE_RES(mp,nl)#define XFS_MKDIR_SPACE_RES (mp,nl) \ (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_QM_DQALLOC_SPACE_RES (mp) \
fs/xfs/xfs_inode.c +53 -7
105 lines not on this page · raw message
PATCH 17/28 xfs: add parent attributes to symlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_symlink to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.h | 2 - fs/xfs/xfs_symlink.c | 58 ++++++++++++++++++++++++++++++++------- 2 files changed, 48 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -95,8 +95,6 @@ XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl)) -#define XFS_SYMLINK_SPACE_RES (mp,nl,b) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl) + (b))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
fs/xfs/xfs_symlink.c +48 -10
134 lines not on this page · raw message
PATCH 18/28 xfs: remove parent pointers in unlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the parent pointer attribute during unlink
--- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr.h | 1 + fs/xfs/libxfs/xfs_parent.c | 17 ++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 5 +++++ fs/xfs/libxfs/xfs_trans_space.h | 2 -- fs/xfs/xfs_inode.c | 42 +++++++++++++++++++++++++++++++++------ 6 files changed, 59 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -946,7 +946,7 @@ xfs_attr_defer_replace( }/* Removes an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_remove (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +1 -0
@@ -545,6 +545,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args); +int xfs_attr_defer_remove (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +17 -0
@@ -115,6 +115,23 @@ xfs_parent_defer_add(return xfs_attr_defer_add (args); } +int +xfs_parent_defer_remove ( +struct xfs_trans *tp, +struct xfs_inode *dp, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset, +struct xfs_inode *child) +{ +struct xfs_da_args *args = &parent->args; + +xfs_init_parent_name_rec (&parent->rec, dp, diroffset); + args->trans = tp; + args->dp = child; + args->hashval =xfs_da_hashname (args->name, args->namelen); +return xfs_attr_defer_remove (args); +} + void __xfs_parent_cancel( xfs_mount_t *mp,
fs/xfs/libxfs/xfs_parent.h +5 -0
@@ -40,6 +40,11 @@ xfs_parent_start(int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,struct xfs_name *parent_name,xfs_dir2_dataptr_t diroffset,struct xfs_inode *child); +int xfs_parent_defer_remove (struct xfs_trans *tp,struct xfs_inode *dp, +struct xfs_parent_defer *parent, +xfs_dir2_dataptr_t diroffset, +struct xfs_inode *child); +void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent); static inline void
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_REMOVE_SPACE_RES (mp) \ - XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \
fs/xfs/xfs_inode.c +35 -7
99 lines not on this page · raw message
PATCH 19/28 xfs: Indent xfs_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Indent variables and parameters in xfs_rename in preparation for parent pointer modifications. White space only, no functional changes. This will make reviewing new code easier on reviewers.
--- fs/xfs/xfs_inode.c | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-)
fs/xfs/xfs_inode.c +20 -19
@@ -2902,26 +2902,27 @@ xfs_rename_alloc_whiteout(*/ int xfs_rename ( -struct user_namespace *mnt_userns, -struct xfs_inode *src_dp, -struct xfs_name *src_name, -struct xfs_inode *src_ip, -struct xfs_inode *target_dp, -struct xfs_name *target_name, -struct xfs_inode *target_ip, -unsigned int flags) +struct user_namespace *mnt_userns, +struct xfs_inode *src_dp, +struct xfs_name *src_name, +struct xfs_inode *src_ip, +struct xfs_inode *target_dp, +struct xfs_name *target_name, +struct xfs_inode *target_ip, +unsigned int flags) { -struct xfs_mount *mp = src_dp->i_mount; -struct xfs_trans *tp; -struct xfs_inode *wip =NULL ;/* whiteout inode */ -struct xfs_inode *inodes[__XFS_SORT_INODES]; -int i; -int num_inodes = __XFS_SORT_INODES; -bool new_parent = (src_dp != target_dp); -bool src_is_directory =S_ISDIR (VFS_I (src_ip)->i_mode); -int spaceres; -bool retried = false; -int error, nospace_error =0 ; +struct xfs_mount *mp = src_dp->i_mount; +struct xfs_trans *tp; +struct xfs_inode *wip =NULL ;/* whiteout inode */ +struct xfs_inode *inodes[__XFS_SORT_INODES]; +int i; +int num_inodes = __XFS_SORT_INODES; +bool new_parent = (src_dp != target_dp); +bool src_is_directory = +S_ISDIR (VFS_I (src_ip)->i_mode); +int spaceres; +bool retried = false; +int error, nospace_error =0 ;trace_xfs_rename (src_dp, target_dp, src_name, target_name);
PATCH 20/28 xfs: Add parent pointers to rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer.
--- fs/xfs/libxfs/xfs_attr.c | 2 - fs/xfs/libxfs/xfs_attr.h | 1 fs/xfs/libxfs/xfs_parent.c | 47 ++++++++++++++-- fs/xfs/libxfs/xfs_parent.h | 24 +++++++- fs/xfs/libxfs/xfs_trans_space.h | 2 - fs/xfs/xfs_inode.c | 117 ++++++++++++++++++++++++++++++++++++--- 6 files changed, 174 insertions(+), 19 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -923,7 +923,7 @@ xfs_attr_defer_add( }/* Sets an attribute for an inode as a deferred operation */ -static int +int xfs_attr_defer_replace (struct xfs_da_args *args) {
fs/xfs/libxfs/xfs_attr.h +1 -0
@@ -546,6 +546,7 @@ int xfs_attr_get_ilocked(struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args);int xfs_attr_defer_add (struct xfs_da_args *args);int xfs_attr_defer_remove (struct xfs_da_args *args); +int xfs_attr_defer_replace (struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +42 -5
76 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +22 -2
@@ -14,7 +14,9 @@ extern struct kmem_cache *xfs_parent_intent_cache;*/ struct xfs_parent_defer {struct xfs_parent_name_rec rec; +struct xfs_parent_name_rec old_rec;struct xfs_da_args args; +bool have_log; };/*
@@ -23,7 +25,8 @@ struct xfs_parent_defer {void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec,struct xfs_inode *ip,uint32_t p_diroffset); -int __xfs_parent_init (struct xfs_mount *mp,struct xfs_parent_defer **parentp); +int __xfs_parent_init (struct xfs_mount *mp,bool grab_log, +struct xfs_parent_defer **parentp);static inline int xfs_parent_start(
@@ -33,13 +36,30 @@ xfs_parent_start( *pp =NULL ;if (xfs_has_parent (mp)) -return __xfs_parent_init (mp, pp); +return __xfs_parent_init (mp, true, pp); +return 0 ; +} + +static inline int +xfs_parent_start_locked ( +struct xfs_mount *mp, +struct xfs_parent_defer **pp) +{ + *pp =NULL ; + +if (xfs_has_parent (mp)) +return __xfs_parent_init (mp, false, pp);return 0 ; }int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,struct xfs_name *parent_name,xfs_dir2_dataptr_t diroffset,struct xfs_inode *child); +int xfs_parent_defer_replace (struct xfs_trans *tp, +struct xfs_parent_defer *new_parent,struct xfs_inode *old_dp, +xfs_dir2_dataptr_t old_diroffset,struct xfs_name *parent_name, +struct xfs_inode *new_ip,xfs_dir2_dataptr_t new_diroffset, +struct xfs_inode *child); int xfs_parent_defer_remove(struct xfs_trans *tp, struct xfs_inode *dp, struct xfs_parent_defer *parent, xfs_dir2_dataptr_t diroffset,
fs/xfs/libxfs/xfs_trans_space.h +0 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_RENAME_SPACE_RES (mp,nl) \ - (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
fs/xfs/xfs_inode.c +108 -9
190 lines not on this page · raw message
PATCH 21/28 xfs: Add parent pointers to xfs_cross_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Cross renames are handled separately from standard renames, and need different handling to update the parent attributes correctly.
--- fs/xfs/xfs_inode.c | 49 +++++++++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 16 deletions(-)
fs/xfs/xfs_inode.c +33 -16
@@ -2749,27 +2749,31 @@ xfs_finish_rename(*/ STATIC intxfs_cross_rename ( -struct xfs_trans *tp, -struct xfs_inode *dp1, -struct xfs_name *name1, -struct xfs_inode *ip1, -struct xfs_inode *dp2, -struct xfs_name *name2, -struct xfs_inode *ip2, -int spaceres) +struct xfs_trans *tp, +struct xfs_inode *dp1, +struct xfs_name *name1, +struct xfs_inode *ip1, +struct xfs_parent_defer *ip1_pptr, +struct xfs_inode *dp2, +struct xfs_name *name2, +struct xfs_inode *ip2, +struct xfs_parent_defer *ip2_pptr, +int spaceres) { -int error =0 ; -int ip1_flags =0 ; -int ip2_flags =0 ; -int dp2_flags =0 ; +struct xfs_mount *mp = dp1->i_mount; +int error =0 ; +int ip1_flags =0 ; +int ip2_flags =0 ; +int dp2_flags =0 ; +int new_diroffset, old_diroffset;/* Swap inode number for dirent in first parent */ - error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres,NULL ); + error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres, &old_diroffset);if (error) goto out_trans_abort;/* Swap inode number for dirent in second parent */ - error =xfs_dir_replace (tp, dp2, name2, ip1->i_ino, spaceres,NULL ); + error =xfs_dir_replace (tp, dp2, name2, ip1->i_ino, spaceres, &new_diroffset);if (error) goto out_trans_abort;
@@ -2830,6 +2834,18 @@ xfs_cross_rename( } } +if (xfs_has_parent (mp)) { + error =xfs_parent_defer_replace (tp, ip1_pptr, dp1, + old_diroffset, name2, dp2, new_diroffset, ip1); +if (error) + goto out_trans_abort; + + error =xfs_parent_defer_replace (tp, ip2_pptr, dp2, + new_diroffset, name1, dp1, old_diroffset, ip2); +if (error) + goto out_trans_abort; + } +if (ip1_flags) {xfs_trans_ichgtime (tp, ip1, ip1_flags);xfs_trans_log_inode (tp, ip1,XFS_ILOG_CORE );
@@ -2844,6 +2860,7 @@ xfs_cross_rename( }xfs_trans_ichgtime (tp, dp1,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp1,XFS_ILOG_CORE ); +return xfs_finish_rename (tp); out_trans_abort:
@@ -3060,8 +3077,8 @@ xfs_rename(/* RENAME_EXCHANGE is unique from here on. */ if (flags &RENAME_EXCHANGE ) { error =xfs_cross_rename (tp, src_dp, src_name, src_ip, - target_dp, target_name, target_ip, - spaceres); + src_ip_pptr, target_dp, target_name, target_ip, + tgt_ip_pptr, spaceres); goto out_unlock; }
PATCH 22/28 xfs: Add the parent pointer support to the superblock version 5.
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
--- fs/xfs/libxfs/xfs_format.h | 4 +++- fs/xfs/libxfs/xfs_fs.h | 1 + fs/xfs/libxfs/xfs_sb.c | 4 ++++ fs/xfs/xfs_super.c | 4 ++++ 4 files changed, 12 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +3 -1
@@ -373,13 +373,15 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_BIGTIME (1 << 3)/* large timestamps */ #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ #define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* large extent counters */ +#define XFS_SB_FEAT_INCOMPAT_PARENT (1 << 6)/* parent pointers */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ XFS_SB_FEAT_INCOMPAT_META_UUID| \ XFS_SB_FEAT_INCOMPAT_BIGTIME| \ XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \ - XFS_SB_FEAT_INCOMPAT_NREXT64) + XFS_SB_FEAT_INCOMPAT_NREXT64| \ + XFS_SB_FEAT_INCOMPAT_PARENT)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -237,6 +237,7 @@ typedef struct xfs_fsop_resblks {#define XFS_FSOP_GEOM_FLAGS_BIGTIME (1 << 21)/* 64-bit nsec timestamps */ #define XFS_FSOP_GEOM_FLAGS_INOBTCNT (1 << 22)/* inobt btree counter */ #define XFS_FSOP_GEOM_FLAGS_NREXT64 (1 << 23)/* large extent counters */ +#define XFS_FSOP_GEOM_FLAGS_PARENT (1 << 24)/* parent pointers */ /* * Minimum and maximum sizes need for growth checks.
fs/xfs/libxfs/xfs_sb.c +4 -0
@@ -173,6 +173,8 @@ xfs_sb_version_to_features( features |=XFS_FEAT_NEEDSREPAIR ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NREXT64 ) features |=XFS_FEAT_NREXT64 ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_PARENT ) + features |=XFS_FEAT_PARENT ;return features; }
@@ -1189,6 +1191,8 @@ xfs_fs_geometry( geo->flags |=XFS_FSOP_GEOM_FLAGS_BIGTIME ;if (xfs_has_inobtcounts (mp)) geo->flags |=XFS_FSOP_GEOM_FLAGS_INOBTCNT ; +if (xfs_has_parent (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_PARENT ;if (xfs_has_sector (mp)) { geo->flags |=XFS_FSOP_GEOM_FLAGS_SECTOR ; geo->logsectsize = sbp->sb_logsectsize;
fs/xfs/xfs_super.c +4 -0
@@ -1664,6 +1664,10 @@ xfs_fs_fill_super(xfs_warn (mp,"EXPERIMENTAL Large extent counts feature in use. Use at your own risk!" ); +if (xfs_has_parent (mp)) +xfs_alert (mp, +"EXPERIMENTAL parent pointer feature enabled. Use at your own risk!" ); + error =xfs_mountfs (mp);if (error) goto out_filestream_unmount;
PATCH 23/28 xfs: Add helper function xfs_attr_list_context_init
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch adds a helper function xfs_attr_list_context_init used by xfs_attr_list. This function initializes the xfs_attr_list_context structure passed to xfs_attr_list_int. We will need this later to call xfs_attr_list_int_ilocked when the node is already locked.
--- fs/xfs/xfs_file.c | 1 + fs/xfs/xfs_ioctl.c | 54 +++++++++++++++++++++++++++++++++++++--------------- fs/xfs/xfs_ioctl.h | 2 ++ 3 files changed, 41 insertions(+), 16 deletions(-)
fs/xfs/xfs_file.c +1 -0
@@ -17,6 +17,7 @@#include "xfs_bmap_util.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" #include "xfs_ioctl.h" #include "xfs_trace.h" #include "xfs_log.h"
fs/xfs/xfs_ioctl.c +38 -16
72 lines not on this page · raw message
fs/xfs/xfs_ioctl.h +2 -0
@@ -35,6 +35,8 @@ int xfs_ioc_attrmulti_one(struct file *parfilp, struct inode *inode,int xfs_ioc_attr_list (struct xfs_inode *dp,void __user *ubuf,size_t bufsize,int flags,struct xfs_attrlist_cursor __user *ucursor); +int xfs_ioc_attr_list_context_init (struct xfs_inode *dp,char *buffer, +int bufsize,int flags,struct xfs_attr_list_context *context);extern struct dentry * xfs_handle_to_dentry(
PATCH 24/28 xfs: Filter XFS_ATTR_PARENT for getfattr
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Parent pointers returned to the get_fattr tool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers from xfs_xattr_put_listent.
--- fs/xfs/xfs_xattr.c | 3 +++ 1 file changed, 3 insertions(+)
fs/xfs/xfs_xattr.c +3 -0
@@ -234,6 +234,9 @@ xfs_xattr_put_listent(ASSERT (context->count >=0 ); +if (flags &XFS_ATTR_PARENT ) +return ; +if (flags &XFS_ATTR_ROOT ) {#ifdef CONFIG_XFS_POSIX_ACL if (namelen == SGI_ACL_FILE_SIZE &&
PATCH 25/28 xfs: Add parent pointer ioctl
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch adds a new file ioctl to retrieve the parent pointer of a given inode
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_fs.h | 74 ++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.c | 10 +++ fs/xfs/libxfs/xfs_parent.h | 2 + fs/xfs/xfs_ioctl.c | 94 ++++++++++++++++++++++++++++++++- fs/xfs/xfs_ondisk.h | 4 + fs/xfs/xfs_parent_utils.c | 126 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 11 ++++ 8 files changed, 321 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
fs/xfs/Makefile +1 -0
@@ -86,6 +86,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \
fs/xfs/libxfs/xfs_fs.h +74 -0
86 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.c +10 -0
@@ -29,6 +29,16 @@struct kmem_cache *xfs_parent_intent_cache; +/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */ +void +xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +const struct xfs_parent_name_rec *rec) +{ + xpp->xpp_ino =be64_to_cpu (rec->p_ino); + xpp->xpp_gen =be32_to_cpu (rec->p_gen); + xpp->xpp_diroffset =be32_to_cpu (rec->p_diroffset); +} +/* * Parent pointer attribute handling. *
fs/xfs/libxfs/xfs_parent.h +2 -0
@@ -25,6 +25,8 @@ struct xfs_parent_defer {void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec,struct xfs_inode *ip,uint32_t p_diroffset); +void xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, +const struct xfs_parent_name_rec *rec);int __xfs_parent_init (struct xfs_mount *mp,bool grab_log,struct xfs_parent_defer **parentp);
fs/xfs/xfs_ioctl.c +93 -1
112 lines not on this page · raw message
fs/xfs/xfs_ondisk.h +4 -0
@@ -150,6 +150,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_OFFSET(struct xfs_efi_log_format_32, efi_extents, 16); XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr, 280); + XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info, 104); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5
fs/xfs/xfs_parent_utils.c new file +126 -0
126 lines not on this page · raw message
fs/xfs/xfs_parent_utils.h new file +11 -0
@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022 Oracle, Inc. +* All rights reserved. +*/ +#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +int xfs_attr_get_parent_pointer (struct xfs_inode *ip, +struct xfs_pptr_info *ppi); +#endif /* __XFS_PARENT_UTILS_H__ */
PATCH 26/28 xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.
Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.
We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.
However, the large extent counters feature is still EXPERIMENTAL, so we can gate the correction on that feature (or any features that get added after that) being enabled. Any filesystem with nrext64 or any of the as-yet-undefined feature bits turned on will be rejected by old uncorrected kernels, so this should be safe even in the upgrade case.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 43 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +43 -0
@@ -16,6 +16,39 @@#include "xfs_bmap_btree.h" #include "xfs_trace.h" +/* +* Decide if the filesystem has the parent pointer feature or any feature +* added after that. +*/ +static inline bool +xfs_has_parent_or_newer_feature ( +struct xfs_mount *mp) +{ +if (!xfs_sb_is_v5 (&mp->m_sb)) +return false; + +if (xfs_sb_has_compat_feature (&mp->m_sb, ~0 )) +return true; + +if (xfs_sb_has_ro_compat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_RO_COMPAT_FINOBT | +XFS_SB_FEAT_RO_COMPAT_RMAPBT | +XFS_SB_FEAT_RO_COMPAT_REFLINK | +XFS_SB_FEAT_RO_COMPAT_INOBTCNT ))) +return true; + +if (xfs_sb_has_incompat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_INCOMPAT_FTYPE | +XFS_SB_FEAT_INCOMPAT_SPINODES | +XFS_SB_FEAT_INCOMPAT_META_UUID | +XFS_SB_FEAT_INCOMPAT_BIGTIME | +XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR | +XFS_SB_FEAT_INCOMPAT_NREXT64 ))) +return true; + +return false; +} +/* * Calculate the maximum length in bytes that would be required for a local * attribute value as large attributes out of line are not logged.
@@ -31,6 +64,16 @@ xfs_log_calc_max_attrsetm_res(MAXNAMELEN -1 ; nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); nblks +=XFS_B_TO_FSB (mp, size); + +/* +* Starting with the parent pointer feature, every new fs feature +* corrects a unit conversion error in the xattr transaction +* reservation code that resulted in oversized minimum log size +* computations. +*/ +if (xfs_has_parent_or_newer_feature (mp)) + size =XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); return M_RES(mp)->tr_attrsetm.tr_logres +
PATCH 27/28 xfs: drop compatibility minimum log size computations for reflink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Having established that we can reduce the minimum log size computation for filesystems with parent pointers or any newer feature, we should also drop the compat minlogsize code that we added when we reduced the transaction reservation size for rmap and reflink.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +10 -0
@@ -91,6 +91,16 @@ xfs_log_calc_trans_resv_for_minlogblocks( {unsigned int rmap_maxlevels = mp->m_rmap_maxlevels; +/* +* Starting with the parent pointer feature, every new fs feature +* drops the oversized minimum log size computation introduced by the +* original reflink code. +*/ +if (xfs_has_parent_or_newer_feature (mp)) { +xfs_trans_resv_calc (mp, resv); +return ; + } +/* * In the early days of rmap+reflink, we always set the rmap maxlevels * to 9 even if the AG was small enough that it would never grow to
PATCH 28/28 xfs: add xfs_trans_mod_sb tracing
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Reservationless operations are not allowed with parent pointers because the attr expansion may cause a shutdown if an operation is retried without reservation and succeeds without enough space for the parent pointer. Add tracing to detect if this shutdown occurs.
--- fs/xfs/xfs_trans.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
fs/xfs/xfs_trans.c +3 -1
@@ -375,8 +375,10 @@ xfs_trans_mod_sb(*/ if (delta <0 ) { tp->t_blk_res_used += (uint)-delta; -if (tp->t_blk_res_used > tp->t_blk_res) +if (tp->t_blk_res_used > tp->t_blk_res) { +xfs_err (mp,"URK blkres 0x%x used 0x%x" , tp->t_blk_res, tp->t_blk_res_used);xfs_force_shutdown (mp,SHUTDOWN_CORRUPT_INCORE ); + } }else if (delta >0 && (tp->t_flags &XFS_TRANS_RES_FDBLKS )) {int64_t blkres_delta;
cover xfs: bug fixes for parent pointers
In reply to Darrick J. WongHi all,
This series contains the accumulated bug fixes from Darrick to make fstests pass and online repair work.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-bugfixes
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-bugfixes
--- fs/xfs/libxfs/xfs_attr.c | 61 +++++++++------------------------------- fs/xfs/libxfs/xfs_attr.h | 2 + fs/xfs/libxfs/xfs_attr_leaf.c | 6 +++- fs/xfs/libxfs/xfs_dir2_block.c | 2 + fs/xfs/libxfs/xfs_dir2_leaf.c | 2 + fs/xfs/libxfs/xfs_dir2_node.c | 2 + fs/xfs/libxfs/xfs_dir2_sf.c | 4 +++ fs/xfs/libxfs/xfs_parent.c | 44 +++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 7 +++++ 9 files changed, 79 insertions(+), 51 deletions(-)
PATCH 1/3 xfs: directory lookups should return diroffsets too
From: Darrick J. Wong <[email protected]>
Teach the directory lookup functions to return the dir offset of the dirent that it finds. Online fsck will use this when checking and repairing filesystems.
--- fs/xfs/libxfs/xfs_dir2_block.c | 2 ++ fs/xfs/libxfs/xfs_dir2_leaf.c | 2 ++ fs/xfs/libxfs/xfs_dir2_node.c | 2 ++ fs/xfs/libxfs/xfs_dir2_sf.c | 4 ++++ 4 files changed, 10 insertions(+)
fs/xfs/libxfs/xfs_dir2_block.c +2 -0
@@ -749,6 +749,8 @@ xfs_dir2_block_lookup_int( cmp =xfs_dir2_compname (args, dep->name, dep->namelen);if (cmp !=XFS_CMP_DIFFERENT && cmp != args->cmpresult) { args->cmpresult = cmp; + args->offset =xfs_dir2_byte_to_dataptr ( + (char *)dep - (char *)hdr); *bpp = bp; *entno = mid;if (cmp ==XFS_CMP_EXACT )
fs/xfs/libxfs/xfs_dir2_leaf.c +2 -0
@@ -1300,6 +1300,8 @@ xfs_dir2_leaf_lookup_int( cmp =xfs_dir2_compname (args, dep->name, dep->namelen);if (cmp !=XFS_CMP_DIFFERENT && cmp != args->cmpresult) { args->cmpresult = cmp; + args->offset =xfs_dir2_db_off_to_dataptr (args->geo, + newdb, (char *)dep - (char *)dbp->b_addr); *indexp = index;/* case exact match: return the current buffer. */ if (cmp ==XFS_CMP_EXACT ) {
fs/xfs/libxfs/xfs_dir2_node.c +2 -0
@@ -887,6 +887,8 @@ xfs_dir2_leafn_lookup_for_entry( args->cmpresult = cmp; args->inumber =be64_to_cpu (dep->inumber); args->filetype =xfs_dir2_data_get_ftype (mp, dep); + args->offset =xfs_dir2_db_off_to_dataptr (args->geo, ++ newdb, (char *)dep - (char *)curbp->b_addr); *indexp = index; state->extravalid =1 ; state->extrablk.bp = curbp;
fs/xfs/libxfs/xfs_dir2_sf.c +4 -0
@@ -889,6 +889,7 @@ xfs_dir2_sf_lookup( args->inumber = dp->i_ino; args->cmpresult =XFS_CMP_EXACT ; args->filetype =XFS_DIR3_FT_DIR ; + args->offset =1 ;return -EEXIST ; }/*
@@ -899,6 +900,7 @@ xfs_dir2_sf_lookup( args->inumber =xfs_dir2_sf_get_parent_ino (sfp); args->cmpresult =XFS_CMP_EXACT ; args->filetype =XFS_DIR3_FT_DIR ; + args->offset =2 ;return -EEXIST ; }/*
@@ -917,6 +919,8 @@ xfs_dir2_sf_lookup( args->cmpresult = cmp; args->inumber =xfs_dir2_sf_get_ino (mp, sfp, sfep); args->filetype =xfs_dir2_sf_get_ftype (mp, sfep); + args->offset =xfs_dir2_byte_to_dataptr ( +xfs_dir2_sf_get_offset (sfep));if (cmp ==XFS_CMP_EXACT )return -EEXIST ; ci_sfep = sfep;
PATCH 2/3 xfs: move/add parent pointer validators to xfs_parent
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Move the parent pointer xattr name validator to xfs_parent.c, and add a new function to check the xattr value.
--- fs/xfs/libxfs/xfs_attr.c | 61 +++++++++----------------------------------- fs/xfs/libxfs/xfs_attr.h | 2 + fs/xfs/libxfs/xfs_parent.c | 44 ++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 7 +++++ 4 files changed, 65 insertions(+), 49 deletions(-)
fs/xfs/libxfs/xfs_attr.c +13 -48
@@ -26,6 +26,7 @@#include "xfs_trace.h" #include "xfs_attr_item.h" #include "xfs_xattr.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attr_intent_cache;
@@ -1577,62 +1578,26 @@ xfs_attr_node_get(return error; } -/* -* Verify parent pointer attribute is valid. -* Return true on success or false on failure -*/ -STATIC bool -xfs_verify_pptr ( -struct xfs_mount *mp, -const struct xfs_parent_name_rec *rec) -{ -xfs_ino_t p_ino; -xfs_dir2_dataptr_t p_diroffset; - - p_ino =be64_to_cpu (rec->p_ino); - p_diroffset =be32_to_cpu (rec->p_diroffset); - -if (!xfs_verify_ino (mp, p_ino)) -return false; - -if (p_diroffset >XFS_DIR2_MAX_DATAPTR ) -return false; - -return true; -} - -/* Returns true if the string attribute entry name is valid. */ -static bool -xfs_str_attr_namecheck ( -const void *name, -size_t length) -{ -/* -* MAXNAMELEN includes the trailing null, but (name/length) leave it -* out, so use >= for the length check. -*/ -if (length >=MAXNAMELEN ) -return false; - -/* There shouldn't be any nulls here */ -return !memchr (name,0 , length); -} -/* Returns true if the attribute entry name is valid. */ bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, -int flags) +unsigned int flags) { -if (flags &XFS_ATTR_PARENT ) { -if (length !=sizeof (struct xfs_parent_name_rec )) -return false; -return xfs_verify_pptr (mp, (struct xfs_parent_name_rec *)name); - } +if (flags &XFS_ATTR_PARENT ) +return xfs_parent_namecheck (mp, name, length, flags); -return xfs_str_attr_namecheck (name, length); +/* +* MAXNAMELEN includes the trailing null, but (name/length) leave it +* out, so use >= for the length check. +*/ +if (length >=MAXNAMELEN ) +return false; + +/* There shouldn't be any nulls here */ +return !memchr (name,0 , length); }int __init
fs/xfs/libxfs/xfs_attr.h +1 -1
@@ -551,7 +551,7 @@ int xfs_attr_set(struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, -int flags); +unsigned int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
fs/xfs/libxfs/xfs_parent.c +44 -0
50 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +7 -0
@@ -8,6 +8,13 @@extern struct kmem_cache *xfs_parent_intent_cache; +/* Metadata validators */ +bool xfs_parent_namecheck (struct xfs_mount *mp, +const struct xfs_parent_name_rec *rec,size_t reclen, +unsigned int attr_flags); +bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value, +size_t valuelen); +/* * Dynamically allocd structure used to wrap the needed data to pass around * the defer ops machinery
PATCH 3/3 xfs: don't remove the attr fork when parent pointers are enabled
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When running generic/388, I observed the following .out.bad output:
_check_xfs_filesystem: filesystem on /dev/sda4 is inconsistent (r) *** xfs_repair -n output ***
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
mismatch between format (2) and size (276) in symlink ino 37223730 bad data fork in symlink 37223730
would have cleared inode 37223730
- agno = 2
- agno = 3
mismatch between format (2) and size (276) in symlink ino 102725435 bad data fork in symlink 102725435
would have cleared inode 102725435
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
unknown block state, ag 1, blocks 458655-458655
unknown block state, ag 3, blocks 257772-257772
- check for inodes claiming duplicate blocks...
- agno = 1
- agno = 2
- agno = 3
- agno = 0
mismatch between format (2) and size (276) in symlink ino 102725435 bad data fork in symlink 102725435
would have cleared inode 102725435
mismatch between format (2) and size (276) in symlink ino 37223730 bad data fork in symlink 37223730
would have cleared inode 37223730
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
user quota id 0 has bcount 1140448, expected 1140446 user quota id 0 has icount 39892, expected 39890
No modify flag set, skipping filesystem flush and exiting.
Inode 37223730 is an unlinked remote-format symlink with no xattr fork. According to the inode verifier and xfs_repair, this symlink ought to have a local format data fork, since 276 bytes is small enough to fit in the immediate area.
How did we get here? fsstress removed the symlink, which removed the last parent pointer xattr. There were no other xattrs, so that removal also removed the attr fork. This transaction got flushed to the log, but the system went down before we could inactivate the symlink. Log recovery tried to inactivate this inode (since it is on the unlinked list) but the verifier tripped over the remote value and leaked it.
Hence we ended up with a file in this odd state on a "clean" mount. The "obvious" fix is to prohibit erasure of the attr fork to avoid tripping over the verifiers when pptrs are enabled.
I wonder this could be reproduced with normal xattrs and (say) a directory? Maybe this fix should target /any/ symlink or directory?
--- fs/xfs/libxfs/xfs_attr_leaf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +4 -2
@@ -854,7 +854,8 @@ xfs_attr_sf_removename( totsize -= size;if (totsize ==sizeof (xfs_attr_sf_hdr_t) &&xfs_has_attr2 (mp) && (dp->i_df.if_format !=XFS_DINODE_FMT_BTREE ) && - !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE ))) { + !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE )) && + !xfs_has_parent (mp)) {xfs_attr_fork_remove (dp, args->trans); }else {xfs_idata_realloc (dp, -size,XFS_ATTR_FORK );
@@ -863,7 +864,8 @@ xfs_attr_sf_removename(ASSERT (totsize >sizeof (xfs_attr_sf_hdr_t) || (args->op_flags &XFS_DA_OP_ADDNAME ) || !xfs_has_attr2 (mp) || - dp->i_df.if_format ==XFS_DINODE_FMT_BTREE ); + dp->i_df.if_format ==XFS_DINODE_FMT_BTREE || +xfs_has_parent (mp));xfs_trans_log_inode (args->trans, dp,XFS_ILOG_CORE |XFS_ILOG_ADATA ); }
cover xfs: rework the GETPARENTS ioctl
In reply to Darrick J. WongHi all,
This series fixes a few bugs that I found in the XFS_IOC_GETPARENTS implementation. It also reworks the xfs_attr_list implementations to provide an xattr value pointer when available, and finally it reworks the whole implementation to take advantage of this and use less memory.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-ioctl
--- fs/xfs/libxfs/xfs_attr.h | 5 + fs/xfs/libxfs/xfs_attr_sf.h | 1 fs/xfs/libxfs/xfs_parent.c | 40 +++++++-- fs/xfs/libxfs/xfs_parent.h | 21 ++++- fs/xfs/scrub/attr.c | 8 ++ fs/xfs/xfs_attr_list.c | 8 ++ fs/xfs/xfs_ioctl.c | 40 ++++++--- fs/xfs/xfs_parent_utils.c | 184 ++++++++++++++++++++++++------------------- fs/xfs/xfs_parent_utils.h | 4 - fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 73 +++++++++++++++++ fs/xfs/xfs_xattr.c | 1 12 files changed, 272 insertions(+), 114 deletions(-)
PATCH 1/4 xfs: fix multiple problems when doing getparents by handle
From: Darrick J. Wong <[email protected]>
Fix a few problems in the file handle processing part of GETPARENTS. First, we need to validate that the fsid of the handle matches the filesystem that we're talking to. Second, we can skip the iget if the inode number matches the open file. Third, if we are going to do the iget file, we need to use an UNTRUSTED lookup to guard against crap. Finally, we mustn't leak any inodes that we iget.
--- fs/xfs/xfs_ioctl.c | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-)
fs/xfs/xfs_ioctl.c +21 -9
@@ -1694,8 +1694,9 @@ xfs_ioc_get_parent_pointer( {struct xfs_pptr_info *ppi =NULL ;int error =0 ; -struct xfs_inode *ip =XFS_I (file_inode (filp)); -struct xfs_mount *mp = ip->i_mount; +struct xfs_inode *file_ip =XFS_I (file_inode (filp)); +struct xfs_inode *call_ip = file_ip; +struct xfs_mount *mp = file_ip->i_mount;if (!capable (CAP_SYS_ADMIN ))return -EPERM ;
@@ -1733,23 +1734,32 @@ xfs_ioc_get_parent_pointer(return -ENOMEM ;if (ppi->pi_flags &XFS_PPTR_IFLAG_HANDLE ) { - error =xfs_iget (mp,NULL , ppi->pi_handle.ha_fid.fid_ino, -0 ,0 , &ip); -if (error) +struct xfs_handle *hanp = &ppi->pi_handle; + +if (memcmp (&hanp->ha_fsid, mp->m_fixedfsid, +sizeof (xfs_fsid_t))) { + error = -EINVAL ; goto out; + } -if (VFS_I (ip)->i_generation != ppi->pi_handle.ha_fid.fid_gen) { +if (hanp->ha_fid.fid_ino != file_ip->i_ino) { + error =xfs_iget (mp,NULL , hanp->ha_fid.fid_ino, +XFS_IGET_UNTRUSTED ,0 , &call_ip); +if (error) + goto out; + } + +if (VFS_I (call_ip)->i_generation != hanp->ha_fid.fid_gen) { error = -EINVAL ; goto out; } } -if (ip->i_ino == mp->m_sb.sb_rootino) +if (call_ip->i_ino == mp->m_sb.sb_rootino) ppi->pi_flags |=XFS_PPTR_OFLAG_ROOT ;/* Get the parent pointers */ - error =xfs_attr_get_parent_pointer (ip, ppi); - + error =xfs_attr_get_parent_pointer (call_ip, ppi);if (error) goto out;
@@ -1762,6 +1772,8 @@ xfs_ioc_get_parent_pointer( } out: +if (call_ip != file_ip) +xfs_irele (call_ip);kmem_free (ppi);return error; }
PATCH 2/4 xfs: use kvalloc for the parent pointer info buffer
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
It's possible that userspace could call us with large(ish) 64k buffer. Use kvalloc for this, so that the kernel doesn't have to find a contiguous physical region. Zero the realloc buffer so that we don't leak kernel contents to userspace.
--- fs/xfs/xfs_ioctl.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
fs/xfs/xfs_ioctl.c +5 -3
@@ -1702,7 +1702,7 @@ xfs_ioc_get_parent_pointer(return -EPERM ;/* Allocate an xfs_pptr_info to put the user data */ - ppi =kmalloc (sizeof (struct xfs_pptr_info ),0 ); + ppi =kvmalloc (sizeof (struct xfs_pptr_info ),GFP_KERNEL );if (!ppi)return -ENOMEM ;
@@ -1729,7 +1729,9 @@ xfs_ioc_get_parent_pointer(* Now that we know how big the trailing buffer is, expand * our kernel xfs_pptr_info to be the same size */ - ppi =krealloc (ppi,xfs_pptr_info_sizeof (ppi->pi_ptrs_size),0 ); + ppi =kvrealloc (ppi,sizeof (struct xfs_pptr_info ), +xfs_pptr_info_sizeof (ppi->pi_ptrs_size), +GFP_KERNEL | __GFP_ZERO);if (!ppi)return -ENOMEM ;
@@ -1774,7 +1776,7 @@ xfs_ioc_get_parent_pointer( out:if (call_ip != file_ip)xfs_irele (call_ip); -kmem_free (ppi); +kvfree (ppi);return error; }
PATCH 3/4 xfs: pass the attr value to put_listent when possible
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Pass the attr value to put_listent when we have local xattrs or shortform xattrs.
--- fs/xfs/libxfs/xfs_attr.h | 5 +++-- fs/xfs/libxfs/xfs_attr_sf.h | 1 + fs/xfs/scrub/attr.c | 8 ++++++++ fs/xfs/xfs_attr_list.c | 8 +++++++- fs/xfs/xfs_ioctl.c | 1 + fs/xfs/xfs_xattr.c | 1 + 6 files changed, 21 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.h +3 -2
@@ -47,8 +47,9 @@ struct xfs_attrlist_cursor_kern {/* void; state communicated via *context */ -typedef void (*put_listent_func_t )(struct xfs_attr_list_context *,int , -unsigned char *,int ,int ); +typedef void (*put_listent_func_t )(struct xfs_attr_list_context *context, +int flags,unsigned char *name,int namelen,void *value, +int valuelen);struct xfs_attr_list_context {struct xfs_trans *tp;
fs/xfs/libxfs/xfs_attr_sf.h +1 -0
@@ -24,6 +24,7 @@ typedef struct xfs_attr_sf_sort {uint8_t flags;/* flags bits (see xfs_attr_leaf.h) */ xfs_dahash_t hash;/* this entry's hash value */ unsigned char *name;/* name value, pointer into buffer */ +void *value; } xfs_attr_sf_sort_t;#define XFS_ATTR_SF_ENTSIZE_MAX /* max space for name&value */ \
fs/xfs/scrub/attr.c +8 -0
@@ -109,6 +109,7 @@ xchk_xattr_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { struct xchk_xattr *sx;
@@ -134,6 +135,13 @@ xchk_xattr_listent(return ; } +/* +* Shortform and local attrs don't require external lookups to retrieve +* the value, so there's nothing else to check here. +*/ +if (value) +return ; +/* * Try to allocate enough memory to extrat the attr value. If that * doesn't work, we overload the seen_enough variable to convey
fs/xfs/xfs_attr_list.c +7 -1
@@ -94,6 +94,7 @@ xfs_attr_shortform_list( sfe->flags, sfe->nameval, (int )sfe->namelen, + &sfe->nameval[sfe->namelen], (int)sfe->valuelen);/* * Either search callback finished early or
@@ -139,6 +140,7 @@ xfs_attr_shortform_list( sbp->name = sfe->nameval; sbp->namelen = sfe->namelen;/* These are bytes, and both on-disk, don't endian-flip */ + sbp->value = &sfe->nameval[sfe->namelen], sbp->valuelen = sfe->valuelen; sbp->flags = sfe->flags; sfe =xfs_attr_sf_nextentry (sfe);
@@ -189,6 +191,7 @@ xfs_attr_shortform_list( sbp->flags, sbp->name, sbp->namelen, + sbp->value, sbp->valuelen);if (context->seen_enough)break ;
@@ -443,6 +446,7 @@ xfs_attr3_leaf_list_int(*/ for (; i < ichdr.count; entry++, i++) {char *name; +void *value;int namelen, valuelen;if (be32_to_cpu (entry->hashval) != cursor->hashval) {
@@ -460,6 +464,7 @@ xfs_attr3_leaf_list_int( name_loc =xfs_attr3_leaf_name_local (leaf, i); name = name_loc->nameval; namelen = name_loc->namelen; + value = &name_loc->nameval[name_loc->namelen]; valuelen =be16_to_cpu (name_loc->valuelen); }else {xfs_attr_leaf_name_remote_t *name_rmt;
@@ -467,6 +472,7 @@ xfs_attr3_leaf_list_int( name_rmt =xfs_attr3_leaf_name_remote (leaf, i); name = name_rmt->name; namelen = name_rmt->namelen; + value =NULL ; valuelen =be32_to_cpu (name_rmt->valuelen); }
@@ -475,7 +481,7 @@ xfs_attr3_leaf_list_int( entry->flags)))return -EFSCORRUPTED ; context->put_listent (context, entry->flags, - name, namelen, valuelen); + name, namelen, value, valuelen);if (context->seen_enough)break ; cursor->offset++;
fs/xfs/xfs_ioctl.c +1 -0
@@ -308,6 +308,7 @@ xfs_ioc_attr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { struct xfs_attrlist *alist = context->buffer;
fs/xfs/xfs_xattr.c +1 -0
@@ -227,6 +227,7 @@ xfs_xattr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { char *prefix;
PATCH 4/4 xfs: replace the XFS_IOC_GETPARENTS backend
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that xfs_attr_list can pass local xattr values to the put_listent function, build a new version of the GETPARENTS backend that supplies a custom put_listent function to format parent pointer info directly into the caller's buffer. This uses a lot less memory and obviates the iterate list and then grab the values logic, since parent pointers aren't supposed to have remote values anyway.
--- fs/xfs/libxfs/xfs_parent.c | 40 +++++++--- fs/xfs/libxfs/xfs_parent.h | 21 +++++ fs/xfs/xfs_ioctl.c | 5 - fs/xfs/xfs_parent_utils.c | 184 ++++++++++++++++++++++++-------------------- fs/xfs/xfs_parent_utils.h | 4 - fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 73 +++++++++++++++++ 7 files changed, 227 insertions(+), 101 deletions(-)
fs/xfs/libxfs/xfs_parent.c +30 -10
@@ -29,16 +29,6 @@struct kmem_cache *xfs_parent_intent_cache; -/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */ -void -xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, -const struct xfs_parent_name_rec *rec) -{ - xpp->xpp_ino =be64_to_cpu (rec->p_ino); - xpp->xpp_gen =be32_to_cpu (rec->p_gen); - xpp->xpp_diroffset =be32_to_cpu (rec->p_diroffset); -} -/* * Parent pointer attribute handling. *
@@ -115,6 +105,36 @@ xfs_init_parent_name_rec( rec->p_diroffset =cpu_to_be32 (p_diroffset); } +/* +* Convert an ondisk parent_name xattr to its incore format. If @value is +* NULL, set @irec->p_namelen to zero and leave @irec->p_name untouched. +*/ +void +xfs_parent_irec_from_disk ( +struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec, +const void *value, +int valuelen) +{ + irec->p_ino =be64_to_cpu (rec->p_ino); + irec->p_gen =be32_to_cpu (rec->p_gen); + irec->p_diroffset =be32_to_cpu (rec->p_diroffset); + +if (!value) { + irec->p_namelen =0 ; +return ; + } + +ASSERT (valuelen >0 ); +ASSERT (valuelen <MAXNAMELEN ); + + valuelen =min (valuelen,MAXNAMELEN ); + + irec->p_namelen = valuelen; +memcpy (irec->p_name, value, valuelen); +memset (&irec->p_name[valuelen],0 ,sizeof (irec->p_name) - valuelen); +} + int __xfs_parent_init( struct xfs_mount *mp,
fs/xfs/libxfs/xfs_parent.h +19 -2
@@ -15,6 +15,25 @@ bool xfs_parent_namecheck(struct xfs_mount *mp,bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value,size_t valuelen); +/* +* Incore version of a parent pointer, also contains dirent name so callers +* can pass/obtain all the parent pointer information in a single structure +*/ +struct xfs_parent_name_irec { +/* Key fields for looking up a particular parent pointer. */ +xfs_ino_t p_ino; +uint32_t p_gen; +xfs_dir2_dataptr_t p_diroffset; + +/* Attributes of a parent pointer. */ +uint8_t p_namelen; +unsigned char p_name[MAXNAMELEN ]; +}; + +void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec, +const void *value,int valuelen); +/* * Dynamically allocd structure used to wrap the needed data to pass around * the defer ops machinery
@@ -32,8 +51,6 @@ struct xfs_parent_defer {void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec,struct xfs_inode *ip,uint32_t p_diroffset); -void xfs_init_parent_ptr (struct xfs_parent_ptr *xpp, -const struct xfs_parent_name_rec *rec);int __xfs_parent_init (struct xfs_mount *mp,bool grab_log,struct xfs_parent_defer **parentp);
fs/xfs/xfs_ioctl.c +1 -4
@@ -1758,11 +1758,8 @@ xfs_ioc_get_parent_pointer( } } -if (call_ip->i_ino == mp->m_sb.sb_rootino) - ppi->pi_flags |=XFS_PPTR_OFLAG_ROOT ; -/* Get the parent pointers */ - error =xfs_attr_get_parent_pointer (call_ip, ppi); + error =xfs_getparent_pointers (call_ip, ppi);if (error) goto out;
fs/xfs/xfs_parent_utils.c +101 -83
205 lines not on this page · raw message
fs/xfs/xfs_parent_utils.h +2 -2
@@ -6,6 +6,6 @@#ifndef __XFS_PARENT_UTILS_H__#define __XFS_PARENT_UTILS_H__ -int xfs_attr_get_parent_pointer (struct xfs_inode *ip, -struct xfs_pptr_info *ppi); +int xfs_getparent_pointers (struct xfs_inode *ip,struct xfs_pptr_info *ppi); +#endif /* __XFS_PARENT_UTILS_H__ */
fs/xfs/xfs_trace.c +1 -0
@@ -36,6 +36,7 @@#include "xfs_error.h" #include <linux/iomap.h> #include "xfs_iomap.h" +#include "xfs_parent.h" /* * We include this last to have the helpers above available for the trace
fs/xfs/xfs_trace.h +73 -0
85 lines not on this page · raw message
cover xfs: online fsck support patches
In reply to Darrick J. WongHi all,
These are all the patches that I needed to backport from the online fsck patchset to start writing online fsck for parent pointers and directories.
IOWS, we're blatantly copying things from the online repair part 1 megaseries; this is what online repair part 2 requires.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-fsck-backports
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-fsck-backports
--- fs/xfs/Kconfig | 38 +++ fs/xfs/Makefile | 11 + fs/xfs/libxfs/xfs_dir2.c | 6 fs/xfs/libxfs/xfs_dir2.h | 1 fs/xfs/scrub/agheader_repair.c | 99 +++++--- fs/xfs/scrub/bitmap.c | 389 ++++++++++++++++++++------------ fs/xfs/scrub/bitmap.h | 35 ++- fs/xfs/scrub/bmap.c | 6 fs/xfs/scrub/common.c | 133 ++++++++--- fs/xfs/scrub/common.h | 10 + fs/xfs/scrub/dir.c | 233 ++++++------------- fs/xfs/scrub/inode.c | 6 fs/xfs/scrub/iscan.c | 483 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/iscan.h | 65 +++++ fs/xfs/scrub/listxattr.c | 314 ++++++++++++++++++++++++++ fs/xfs/scrub/listxattr.h | 17 + fs/xfs/scrub/parent.c | 290 ++++++++++-------------- fs/xfs/scrub/quota.c | 9 - fs/xfs/scrub/readdir.c | 375 +++++++++++++++++++++++++++++++ fs/xfs/scrub/readdir.h | 19 ++ fs/xfs/scrub/repair.c | 102 +++++--- fs/xfs/scrub/rtbitmap.c | 11 - fs/xfs/scrub/scrub.c | 23 ++ fs/xfs/scrub/scrub.h | 7 + fs/xfs/scrub/tempfile.c | 243 ++++++++++++++++++++ fs/xfs/scrub/tempfile.h | 29 ++ fs/xfs/scrub/trace.c | 5 fs/xfs/scrub/trace.h | 272 +++++++++++++++++++++++ fs/xfs/scrub/xfarray.c | 394 +++++++++++++++++++++++++++++++++ fs/xfs/scrub/xfarray.h | 60 +++++ fs/xfs/scrub/xfblob.c | 176 +++++++++++++++ fs/xfs/scrub/xfblob.h | 27 ++ fs/xfs/scrub/xfile.c | 329 +++++++++++++++++++++++++++ fs/xfs/scrub/xfile.h | 60 +++++ fs/xfs/xfs_buf.c | 5 fs/xfs/xfs_buf.h | 10 + fs/xfs/xfs_export.c | 2 fs/xfs/xfs_hooks.c | 94 ++++++++ fs/xfs/xfs_hooks.h | 72 ++++++ fs/xfs/xfs_icache.c | 3 fs/xfs/xfs_icache.h | 11 + fs/xfs/xfs_inode.c | 229 +++++++++++++++++++ fs/xfs/xfs_inode.h | 37 +++ fs/xfs/xfs_itable.c | 8 + fs/xfs/xfs_linux.h | 1 fs/xfs/xfs_mount.h | 2 fs/xfs/xfs_super.c | 2 fs/xfs/xfs_symlink.c | 1 48 files changed, 4112 insertions(+), 642 deletions(-) create mode 100644 fs/xfs/scrub/iscan.c create mode 100644 fs/xfs/scrub/iscan.h create mode 100644 fs/xfs/scrub/listxattr.c create mode 100644 fs/xfs/scrub/listxattr.h create mode 100644 fs/xfs/scrub/readdir.c create mode 100644 fs/xfs/scrub/readdir.h create mode 100644 fs/xfs/scrub/tempfile.c create mode 100644 fs/xfs/scrub/tempfile.h create mode 100644 fs/xfs/scrub/xfarray.c create mode 100644 fs/xfs/scrub/xfarray.h create mode 100644 fs/xfs/scrub/xfblob.c create mode 100644 fs/xfs/scrub/xfblob.h create mode 100644 fs/xfs/scrub/xfile.c create mode 100644 fs/xfs/scrub/xfile.h create mode 100644 fs/xfs/xfs_hooks.c create mode 100644 fs/xfs/xfs_hooks.h
PATCH 1/23 xfs: manage inode DONTCACHE status at irele time
From: Darrick J. Wong <[email protected]>
Right now, there are statements scattered all over the online fsck codebase about how we can't use XFS_IGET_DONTCACHE because of concerns about scrub's unusual practice of releasing inodes with transactions held.
However, iget is the wrong place to handle this -- the DONTCACHE state doesn't matter at all until we try to *release* the inode, and here we get things wrong in multiple ways:
First, if we /do/ have a transaction, we must NOT drop the inode, because the inode could have dirty pages, dropping the inode will trigger writeback, and writeback can trigger a nested transaction.
Second, if the inode already had an active reference and the DONTCACHE flag set, the icache hit when scrub grabs another ref will not clear DONTCACHE. This is sort of by design, since DONTCACHE is now used to initiate cache drops so that sysadmins can change a file's access mode between pagecache and DAX.
Third, if we do actually have the last active reference to the inode, we can set DONTCACHE to avoid polluting the cache. This is the /one/ case where we actually want that flag.
Create an xchk_irele helper to encode all that logic and switch the online fsck code to use it. Since this now means that nearly all scrubbers use the same xfs_iget flags, we can wrap them too.
--- fs/xfs/scrub/common.c | 52 +++++++++++++++++++++++++++++++++++++++++++++---- fs/xfs/scrub/common.h | 3 +++ fs/xfs/scrub/dir.c | 2 +- fs/xfs/scrub/parent.c | 9 ++++---- fs/xfs/scrub/scrub.c | 2 +- 5 files changed, 57 insertions(+), 11 deletions(-)
fs/xfs/scrub/common.c +48 -4
@@ -625,6 +625,16 @@ xchk_checkpoint_log(return 0 ; } +/* Verify that an inode is allocated ondisk, then return its cached inode. */ +int +xchk_iget ( +struct xfs_scrub *sc, +xfs_ino_t inum, +struct xfs_inode **ipp) +{ +return xfs_iget (sc->mp, sc->tp, inum,XFS_IGET_UNTRUSTED ,0 , ipp); +} +/* * Given an inode and the scrub control structure, grab either the * inode referenced in the control structure or the inode passed in.
@@ -649,8 +659,7 @@ xchk_get_inode(/* Look up the inode, see if the generation number matches. */ if (xfs_internal_inum (mp, sc->sm->sm_ino))return -ENOENT ; - error =xfs_iget (mp,NULL , sc->sm->sm_ino, -XFS_IGET_UNTRUSTED |XFS_IGET_DONTCACHE ,0 , &ip); + error =xchk_iget (sc, sc->sm->sm_ino, &ip);switch (error) {case -ENOENT :/* Inode doesn't exist, just bail out. */
@@ -672,7 +681,7 @@ xchk_get_inode(* that it no longer exists. */ error =xfs_imap (sc->mp, sc->tp, sc->sm->sm_ino, &imap, -XFS_IGET_UNTRUSTED |XFS_IGET_DONTCACHE ); +XFS_IGET_UNTRUSTED );if (error)return -ENOENT ; error = -EFSCORRUPTED ;
@@ -685,7 +694,7 @@ xchk_get_inode(return error; }if (VFS_I (ip)->i_generation != sc->sm->sm_gen) { -xfs_irele (ip); +xchk_irele (sc, ip);return -ENOENT ; }
@@ -693,6 +702,41 @@ xchk_get_inode(return 0 ; } +/* Release an inode, possibly dropping it in the process. */ +void +xchk_irele ( +struct xfs_scrub *sc, +struct xfs_inode *ip) +{ +if (current->journal_info !=NULL ) { +ASSERT (current->journal_info == sc->tp); + +/* +* If we are in a transaction, we /cannot/ drop the inode +* ourselves, because the VFS will trigger writeback, which +* can require a transaction. Clear DONTCACHE to force the +* inode to the LRU, where someone else can take care of +* dropping it. +* +* Note that when we grabbed our reference to the inode, it +* could have had an active ref and DONTCACHE set if a sysadmin +* is trying to coerce a change in file access mode. icache +* hits do not clear DONTCACHE, so we must do it here. +*/ +spin_lock (&VFS_I (ip)->i_lock); +VFS_I (ip)->i_state &= ~I_DONTCACHE ; +spin_unlock (&VFS_I (ip)->i_lock); + }else if (atomic_read (&VFS_I (ip)->i_count) ==1 ) { +/* +* If this is the last reference to the inode and the caller +* permits it, set DONTCACHE to avoid thrashing. +*/ +d_mark_dontcache (VFS_I (ip)); + } + +xfs_irele (ip); +} +/* Set us up to scrub a file's contents. */ int xchk_setup_inode_contents(
fs/xfs/scrub/common.h +3 -0
@@ -136,6 +136,9 @@ int xchk_get_inode(struct xfs_scrub *sc);int xchk_setup_inode_contents (struct xfs_scrub *sc,unsigned int resblks);void xchk_buffer_recheck (struct xfs_scrub *sc,struct xfs_buf *bp); +int xchk_iget (struct xfs_scrub *sc,xfs_ino_t inum,struct xfs_inode **ipp); +void xchk_irele (struct xfs_scrub *sc,struct xfs_inode *ip); +/* * Don't bother cross-referencing if we already found corruption or cross * referencing discrepancies.
fs/xfs/scrub/dir.c +1 -1
@@ -86,7 +86,7 @@ xchk_dir_check_ftype( xfs_mode_to_ftype(VFS_I(ip)->i_mode));if (ino_dtype != dtype)xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , offset); -xfs_irele (ip); +xchk_irele (sdc->sc, ip); out:return error; }
fs/xfs/scrub/parent.c +4 -5
@@ -131,7 +131,6 @@ xchk_parent_validate(xfs_ino_t dnum, bool *try_again) { - struct xfs_mount *mp = sc->mp; struct xfs_inode *dp = NULL;xfs_nlink_t expected_nlink;xfs_nlink_t nlink;
@@ -168,7 +167,7 @@ xchk_parent_validate(* -EFSCORRUPTED or -EFSBADCRC then the parent is corrupt which is a * cross referencing error. Any other error is an operational error. */ - error =xfs_iget (mp, sc->tp, dnum,XFS_IGET_UNTRUSTED ,0 , &dp); + error =xchk_iget (sc, dnum, &dp);if (error == -EINVAL || error == -ENOENT ) { error = -EFSCORRUPTED ;xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error);
@@ -236,11 +235,11 @@ xchk_parent_validate(/* Drat, parent changed. Try again! */ if (dnum != dp->i_ino) { -xfs_irele (dp); +xchk_irele (sc, dp); *try_again = true;return 0 ; } -xfs_irele (dp); +xchk_irele (sc, dp);/* * '..' didn't change, so check that there was only one entry
@@ -253,7 +252,7 @@ xchk_parent_validate( out_unlock:xfs_iunlock (dp,XFS_IOLOCK_SHARED ); out_rele: -xfs_irele (dp); +xchk_irele (sc, dp); out:return error; }
fs/xfs/scrub/scrub.c +1 -1
@@ -166,7 +166,7 @@ xchk_teardown(xfs_iunlock (sc->ip, sc->ilock_flags);if (sc->ip != ip_in && !xfs_internal_inum (sc->mp, sc->ip->i_ino)) -xfs_irele (sc->ip); +xchk_irele (sc, sc->ip); sc->ip =NULL ; }if (sc->sm->sm_flags &XFS_SCRUB_IFLAG_REPAIR )
PATCH 2/23 xfs: make checking directory dotdot entries more reliable
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The current directory parent scrubbing code could be tighter in its execution -- instead of bailing out to userspace after a couple of seconds of waiting for the (alleged) parent directory's IOLOCK while refusing to release the child directory's IOLOCK, we could just cycle both locks until we get both or the child process absorbs a fatal signal.
Note that because the usual sequence is to take IOLOCKs before grabbing a transaction, we have to use the _nowait variants on both inodes to avoid an ABBA deadlock. Since parent pointer checking is the only place in scrub that needs this kind of functionality, move it to parent.c as a private function.
Furthermore, if the child directory's parent changes during the lock cycling, we know that the new parent has stamped the correct parent into the dotdot entry, so we can conclude that the parent entry is correct.
This eliminates an entire source of -EDEADLOCK-based "retry harder" scrub executions.
--- fs/xfs/scrub/common.c | 22 ----- fs/xfs/scrub/common.h | 1 fs/xfs/scrub/parent.c | 203 +++++++++++++++++++++++-------------------------- 3 files changed, 97 insertions(+), 129 deletions(-)
fs/xfs/scrub/common.c +0 -22
@@ -908,28 +908,6 @@ xchk_metadata_inode_forks(return 0 ; } -/* -* Try to lock an inode in violation of the usual locking order rules. For -* example, trying to get the IOLOCK while in transaction context, or just -* plain breaking AG-order or inode-order inode locking rules. Either way, -* the only way to avoid an ABBA deadlock is to use trylock and back off if -* we can't. -*/ -int -xchk_ilock_inverted ( -struct xfs_inode *ip, -uint lock_mode) -{ -int i; - -for (i =0 ; i <20 ; i++) { -if (xfs_ilock_nowait (ip, lock_mode)) -return 0 ; -delay (1 ); - } -return -EDEADLOCK ; -} -/* Pause background reaping of resources. */ void xchk_stop_reaping(
fs/xfs/scrub/common.h +0 -1
@@ -150,7 +150,6 @@ static inline bool xchk_skip_xref(struct xfs_scrub_metadata *sm) }int xchk_metadata_inode_forks (struct xfs_scrub *sc); -int xchk_ilock_inverted (struct xfs_inode *ip,uint lock_mode);void xchk_stop_reaping (struct xfs_scrub *sc);void xchk_start_reaping (struct xfs_scrub *sc);
fs/xfs/scrub/parent.c +97 -106
291 lines not on this page · raw message
PATCH 3/23 xfs: xfs_iget in the directory scrubber needs to use UNTRUSTED
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
In commit 4b80ac64450f, we tried to strengthen the directory scrubber by using the iget call to detect directory entries that point to unallocated inodes. Unfortunately, that commit neglected to pass XFS_IGET_UNTRUSTED to xfs_iget, so we don't check the inode btree first. If the inode number points to something that isn't even an inode cluster, iget will throw corruption errors and return -EFSCORRUPTED, which means that we fail to mark the directory corrupt.
--- fs/xfs/scrub/dir.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
fs/xfs/scrub/dir.c +3 -7
@@ -59,19 +59,15 @@ xchk_dir_check_ftype( }/* -* Grab the inode pointed to by the dirent. We release the -* inode before we cancel the scrub transaction. Since we're -* don't know a priori that releasing the inode won't trigger -* eofblocks cleanup (which allocates what would be a nested -* transaction), we can't use DONTCACHE here because DONTCACHE -* inodes can trigger immediate inactive cleanup of the inode. +* Grab the inode pointed to by the dirent. Use UNTRUSTED here to +* check the allocation status of the inode in the inode btrees. * * If _iget returns -EINVAL or -ENOENT then the child inode number is * garbage and the directory is corrupt. If the _iget returns * -EFSCORRUPTED or -EFSBADCRC then the child is corrupt which is a * cross referencing error. Any other error is an operational error. */ - error =xfs_iget (mp, sdc->sc->tp, inum,0 ,0 , &ip); + error =xchk_iget (sdc->sc, inum, &ip);if (error == -EINVAL || error == -ENOENT ) { error = -EFSCORRUPTED ;xchk_fblock_process_error (sdc->sc,XFS_DATA_FORK ,0 , &error);
PATCH 4/23 xfs: always check the existence of a dirent's child inode
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When we're scrubbing directory entries, we always need to iget the child inode to make sure that the inode pointer points to a valid inode. The original directory scrub code (commit a5c4) only set us up to do this for ftype=1 filesystems, which is not sufficient; and then commit 4b80 made it worse by exempting the dot and dotdot entries.
Sorta-fixes: a5c46e5e8912 ("xfs: scrub directory metadata")
Sorta-fixes: 4b80ac64450f ("xfs: scrub should mark a directory corrupt if any entries cannot be iget'd")
--- fs/xfs/scrub/dir.c | 75 ++++++++++++++++++++-------------------------------- 1 file changed, 29 insertions(+), 46 deletions(-)
fs/xfs/scrub/dir.c +29 -46
@@ -39,52 +39,28 @@ struct xchk_dir_ctx { };/* Check that an inode's mode matches a given DT_ type. */ -STATIC int +STATIC voidxchk_dir_check_ftype (struct xchk_dir_ctx *sdc,xfs_fileoff_t offset, -xfs_ino_t inum, +struct xfs_inode *ip,int dtype) {struct xfs_mount *mp = sdc->sc->mp; -struct xfs_inode *ip;int ino_dtype; -int error =0 ;if (!xfs_has_ftype (mp)) {if (dtype !=DT_UNKNOWN && dtype !=DT_DIR )xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , offset); - goto out; +return ; } -/* -* Grab the inode pointed to by the dirent. Use UNTRUSTED here to -* check the allocation status of the inode in the inode btrees. -* -* If _iget returns -EINVAL or -ENOENT then the child inode number is -* garbage and the directory is corrupt. If the _iget returns -* -EFSCORRUPTED or -EFSBADCRC then the child is corrupt which is a -* cross referencing error. Any other error is an operational error. -*/ - error =xchk_iget (sdc->sc, inum, &ip); -if (error == -EINVAL || error == -ENOENT ) { - error = -EFSCORRUPTED ; -xchk_fblock_process_error (sdc->sc,XFS_DATA_FORK ,0 , &error); - goto out; - } -if (!xchk_fblock_xref_process_error (sdc->sc,XFS_DATA_FORK , offset, - &error)) - goto out; -/* Convert mode to the DT_* values that dir_emit uses. */ ino_dtype =xfs_dir3_get_dtype (mp,xfs_mode_to_ftype (VFS_I (ip)->i_mode));if (ino_dtype != dtype)xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , offset); -xchk_irele (sdc->sc, ip); -out: -return error; }/*
@@ -105,17 +81,17 @@ xchk_dir_actor( unsigned type) {struct xfs_mount *mp; +struct xfs_inode *dp;struct xfs_inode *ip;struct xchk_dir_ctx *sdc;struct xfs_name xname;xfs_ino_t lookup_ino;xfs_dablk_t offset; -bool checked_ftype = false;int error =0 ; sdc = container_of(dir_iter, struct xchk_dir_ctx, dir_iter); - ip = sdc->sc->ip; - mp = ip->i_mount; + dp = sdc->sc->ip; + mp = dp->i_mount; offset =xfs_dir2_db_to_da (mp->m_dir_geo,xfs_dir2_dataptr_to_db (mp->m_dir_geo, pos));
@@ -136,11 +112,7 @@ xchk_dir_actor(if (!strncmp ("." , name, namelen)) {/* If this is "." then check that the inum matches the dir. */ -if (xfs_has_ftype (mp) && type !=DT_DIR ) -xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , - offset); - checked_ftype = true; -if (ino != ip->i_ino) +if (ino != dp->i_ino)xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , offset); }else if (!strncmp (".." , name, namelen)) {
@@ -148,11 +120,7 @@ xchk_dir_actor(* If this is ".." in the root inode, check that the inum * matches this dir. */ -if (xfs_has_ftype (mp) && type !=DT_DIR ) -xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , - offset); - checked_ftype = true; -if (ip->i_ino == mp->m_sb.sb_rootino && ino != ip->i_ino) +if (dp->i_ino == mp->m_sb.sb_rootino && ino != dp->i_ino)xchk_fblock_set_corrupt (sdc->sc,XFS_DATA_FORK , offset); }
@@ -162,7 +130,7 @@ xchk_dir_actor( xname.len = namelen; xname.type =XFS_DIR3_FT_UNKNOWN ; - error =xfs_dir_lookup (sdc->sc->tp, ip, &xname, &lookup_ino,NULL ); + error =xfs_dir_lookup (sdc->sc->tp, dp, &xname, &lookup_ino,NULL );/* ENOENT means the hash lookup failed and the dir is corrupt */ if (error == -ENOENT ) error = -EFSCORRUPTED ;
@@ -174,12 +142,27 @@ xchk_dir_actor( goto out; } -/* Verify the file type. This function absorbs error codes. */ -if (!checked_ftype) { - error =xchk_dir_check_ftype (sdc, offset, lookup_ino, type); -if (error) - goto out; +/* +* Grab the inode pointed to by the dirent. Use UNTRUSTED here to +* check the allocation status of the inode in the inode btrees. +* +* If _iget returns -EINVAL or -ENOENT then the child inode number is +* garbage and the directory is corrupt. If the _iget returns +* -EFSCORRUPTED or -EFSBADCRC then the child is corrupt which is a +* cross referencing error. Any other error is an operational error. +*/ + error =xchk_iget (sdc->sc, ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { + error = -EFSCORRUPTED ; +xchk_fblock_process_error (sdc->sc,XFS_DATA_FORK ,0 , &error); + goto out; } +if (!xchk_fblock_xref_process_error (sdc->sc,XFS_DATA_FORK , offset, + &error)) + goto out; + +xchk_dir_check_ftype (sdc, offset, ip, type); +xchk_irele (sdc->sc, ip); out:/* * A negative error code returned here is supposed to cause the
PATCH 5/23 xfs: remove the for_each_xbitmap_ helpers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Remove the for_each_xbitmap_ macros in favor of proper iterator functions. We'll soon be switching this data structure over to an interval tree implementation, which means that we can't allow callers to modify the bitmap during iteration without telling us.
--- fs/xfs/scrub/agheader_repair.c | 89 ++++++++++++++++++++--------------- fs/xfs/scrub/bitmap.c | 59 +++++++++++++++++++++++ fs/xfs/scrub/bitmap.h | 22 ++++++--- fs/xfs/scrub/repair.c | 102 ++++++++++++++++++++++------------------ 4 files changed, 179 insertions(+), 93 deletions(-)
fs/xfs/scrub/agheader_repair.c +50 -39
@@ -486,10 +486,11 @@ xrep_agfl_walk_rmap(/* Strike out the blocks that are cross-linked according to the rmapbt. */ STATIC intxrep_agfl_check_extent ( -struct xrep_agfl *ra,uint64_t start, -uint64_t len) +uint64_t len, +void *priv) { +struct xrep_agfl *ra = priv;xfs_agblock_t agbno =XFS_FSB_TO_AGBNO (ra->sc->mp, start);xfs_agblock_t last_agbno = agbno + len -1 ;int error;
@@ -537,7 +538,6 @@ xrep_agfl_collect_blocks(struct xrep_agfl ra;struct xfs_mount *mp = sc->mp;struct xfs_btree_cur *cur; -struct xbitmap_range *br, *n;int error; ra.sc = sc;
@@ -578,11 +578,7 @@ xrep_agfl_collect_blocks(/* Strike out the blocks that are cross-linked. */ ra.rmap_cur =xfs_rmapbt_init_cursor (mp, sc->tp, agf_bp, sc->sa.pag); -for_each_xbitmap_extent (br, n, agfl_extents) { - error =xrep_agfl_check_extent (&ra, br->start, br->len); -if (error) -break ; - } + error =xbitmap_walk (agfl_extents, xrep_agfl_check_extent, &ra);xfs_btree_del_cursor (ra.rmap_cur, error);if (error) goto out_bmp;
@@ -628,6 +624,43 @@ xrep_agfl_update_agf( XFS_AGF_FLFIRST | XFS_AGF_FLLAST | XFS_AGF_FLCOUNT); } +struct xrep_agfl_fill { +struct xbitmap used_extents; +struct xfs_scrub *sc; +__be32 *agfl_bno; +xfs_agblock_t flcount; +unsigned int fl_off; +}; + +/* Fill the AGFL with whatever blocks are in this extent. */ +static int +xrep_agfl_fill ( +uint64_t start, +uint64_t len, +void *priv) +{ +struct xrep_agfl_fill *af = priv; +struct xfs_scrub *sc = af->sc; +xfs_fsblock_t fsbno = start; +int error; + +while (fsbno < start + len && af->fl_off < af->flcount) + af->agfl_bno[af->fl_off++] = +cpu_to_be32 (XFS_FSB_TO_AGBNO (sc->mp, fsbno++)); + +trace_xrep_agfl_insert (sc->mp, sc->sa.pag->pag_agno, +XFS_FSB_TO_AGBNO (sc->mp, start), len); + + error =xbitmap_set (&af->used_extents, start, fsbno -1 ); +if (error) +return error; + +if (af->fl_off == af->flcount) +return -ECANCELED ; + +return 0 ; +} +/* Write out a totally new AGFL. */ STATIC void xrep_agfl_init_header(
@@ -636,13 +669,12 @@ xrep_agfl_init_header( struct xbitmap *agfl_extents, xfs_agblock_t flcount) { +struct xrep_agfl_fill af = { + .sc = sc, + .flcount = flcount, + };struct xfs_mount *mp = sc->mp; -__be32 *agfl_bno; -struct xbitmap_range *br; -struct xbitmap_range *n;struct xfs_agfl *agfl; -xfs_agblock_t agbno; -unsigned int fl_off;ASSERT (flcount <=xfs_agfl_size (mp));
@@ -661,36 +693,15 @@ xrep_agfl_init_header(* blocks than fit in the AGFL, they will be freed in a subsequent * step. */ - fl_off =0 ; - agfl_bno =xfs_buf_to_agfl_bno (agfl_bp); -for_each_xbitmap_extent (br, n, agfl_extents) { - agbno =XFS_FSB_TO_AGBNO (mp, br->start); - -trace_xrep_agfl_insert (mp, sc->sa.pag->pag_agno, agbno, - br->len); - -while (br->len >0 && fl_off < flcount) { - agfl_bno[fl_off] =cpu_to_be32 (agbno); - fl_off++; - agbno++; - -/* -* We've now used br->start by putting it in the AGFL, -* so bump br so that we don't reap the block later. -*/ - br->start++; - br->len--; - } - -if (br->len) -break ; -list_del (&br->list); -kfree (br); - } +xbitmap_init (&af.used_extents); + af.agfl_bno =xfs_buf_to_agfl_bno (agfl_bp), +xbitmap_walk (agfl_extents, xrep_agfl_fill, &af); +xbitmap_disunion (agfl_extents, &af.used_extents);/* Write new AGFL to disk. */ xfs_trans_buf_set_type (sc->tp, agfl_bp,XFS_BLFT_AGFL_BUF );xfs_trans_log_buf (sc->tp, agfl_bp,0 ,BBTOB (agfl_bp->b_length) -1 ); +xbitmap_destroy (&af.used_extents); }/* Repair the AGFL. */
fs/xfs/scrub/bitmap.c +59 -0
68 lines not on this page · raw message
fs/xfs/scrub/bitmap.h +15 -7
@@ -19,13 +19,6 @@ struct xbitmap {void xbitmap_init (struct xbitmap *bitmap);void xbitmap_destroy (struct xbitmap *bitmap); -#define for_each_xbitmap_extent (bex, n, bitmap) \ - list_for_each_entry_safe((bex), (n), &(bitmap)->list, list) - -#define for_each_xbitmap_block (b, bex, n, bitmap) \ - list_for_each_entry_safe((bex), (n), &(bitmap)->list, list) \ - for ((b) = (bex)->start; (b) < (bex)->start + (bex)->len; (b)++) -int xbitmap_set (struct xbitmap *bitmap,uint64_t start,uint64_t len);int xbitmap_disunion (struct xbitmap *bitmap,struct xbitmap *sub); int xbitmap_set_btcur_path(struct xbitmap *bitmap,
@@ -34,4 +27,19 @@ int xbitmap_set_btblocks(struct xbitmap *bitmap, struct xfs_btree_cur *cur);uint64_t xbitmap_hweight (struct xbitmap *bitmap); +/* +* Return codes for the bitmap iterator functions are 0 to continue iterating, +* and non-zero to stop iterating. Any non-zero value will be passed up to the +* iteration caller. The special value -ECANCELED can be used to stop +* iteration, because neither bitmap iterator ever generates that error code on +* its own. Callers must not modify the bitmap while walking it. +*/ +typedef int (*xbitmap_walk_fn )(uint64_t start,uint64_t len,void *priv); +int xbitmap_walk (struct xbitmap *bitmap,xbitmap_walk_fn fn, +void *priv); + +typedef int (*xbitmap_walk_bits_fn )(uint64_t bit,void *priv); +int xbitmap_walk_bits (struct xbitmap *bitmap,xbitmap_walk_bits_fn fn, +void *priv); +#endif /* __XFS_SCRUB_BITMAP_H__ */
fs/xfs/scrub/repair.c +55 -47
162 lines not on this page · raw message
PATCH 6/23 xfs: drop the _safe behavior from the xbitmap foreach macro
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
It's not safe to edit bitmap intervals while we're iterating them with for_each_xbitmap_extent. None of the existing callers actually need that ability anyway, so drop the safe variable.
--- fs/xfs/scrub/bitmap.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-)
fs/xfs/scrub/bitmap.c +8 -9
@@ -13,8 +13,9 @@#include "scrub/scrub.h" #include "scrub/bitmap.h" -#define for_each_xbitmap_extent (bex, n, bitmap) \ - list_for_each_entry_safe((bex), (n), &(bitmap)->list, list) +/* Iterate each interval of a bitmap. Do not change the bitmap. */ +#define for_each_xbitmap_extent (bex, bitmap) \ + list_for_each_entry((bex), &(bitmap)->list, list)/* * Set a range of this bitmap. Caller must ensure the range is not set.
@@ -46,10 +47,9 @@ voidxbitmap_destroy ( struct xbitmap *bitmap) { -struct xbitmap_range *bmr; -struct xbitmap_range *n; +struct xbitmap_range *bmr, *n; -for_each_xbitmap_extent (bmr, n, bitmap) { +list_for_each_entry_safe (bmr, n, &bitmap->list, list) {list_del (&bmr->list);kfree (bmr); }
@@ -308,10 +308,9 @@ xbitmap_hweight( struct xbitmap *bitmap) {struct xbitmap_range *bmr; -struct xbitmap_range *n;uint64_t ret =0 ; -for_each_xbitmap_extent (bmr, n, bitmap) +for_each_xbitmap_extent (bmr, bitmap) ret += bmr->len;return ret;
@@ -324,10 +323,10 @@ xbitmap_walk(xbitmap_walk_fn fn, void *priv) { - struct xbitmap_range *bex, *n; + struct xbitmap_range *bex;int error =0 ; -for_each_xbitmap_extent (bex, n, bitmap) { +for_each_xbitmap_extent (bex, bitmap) { error =fn (bex->start, bex->len, priv);if (error)break ;
PATCH 7/23 xfs: convert xbitmap to interval tree
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Convert the xbitmap code to use interval trees instead of linked lists. This reduces the amount of coding required to handle the disunion operation and in the future will make it easier to set bits in arbitrary order yet later be able to extract maximally sized extents, which we'll need for rebuilding certain structures. We define our own interval tree type so that it can deal with 64-bit indices even on 32-bit machines.
--- fs/xfs/scrub/agheader_repair.c | 12 + fs/xfs/scrub/bitmap.c | 323 ++++++++++++++++++++++------------------ fs/xfs/scrub/bitmap.h | 11 - 3 files changed, 187 insertions(+), 159 deletions(-)
fs/xfs/scrub/agheader_repair.c +9 -3
@@ -662,7 +662,7 @@ xrep_agfl_fill( }/* Write out a totally new AGFL. */ -STATIC void +STATIC int xrep_agfl_init_header( struct xfs_scrub *sc, struct xfs_buf *agfl_bp,
@@ -675,6 +675,7 @@ xrep_agfl_init_header( };struct xfs_mount *mp = sc->mp;struct xfs_agfl *agfl; +int error;ASSERT (flcount <=xfs_agfl_size (mp));
@@ -696,12 +697,15 @@ xrep_agfl_init_header(xbitmap_init (&af.used_extents); af.agfl_bno =xfs_buf_to_agfl_bno (agfl_bp),xbitmap_walk (agfl_extents, xrep_agfl_fill, &af); -xbitmap_disunion (agfl_extents, &af.used_extents); + error =xbitmap_disunion (agfl_extents, &af.used_extents); +if (error) +return error;/* Write new AGFL to disk. */ xfs_trans_buf_set_type (sc->tp, agfl_bp,XFS_BLFT_AGFL_BUF );xfs_trans_log_buf (sc->tp, agfl_bp,0 ,BBTOB (agfl_bp->b_length) -1 );xbitmap_destroy (&af.used_extents); +return 0 ; }/* Repair the AGFL. */
@@ -754,7 +758,9 @@ xrep_agfl(* buffers until we know that part works. */ xrep_agfl_update_agf (sc, agf_bp, flcount); -xrep_agfl_init_header (sc, agfl_bp, &agfl_extents, flcount); + error =xrep_agfl_init_header (sc, agfl_bp, &agfl_extents, flcount); +if (error) + goto err;/* * Ok, the AGFL should be ready to go now. Roll the transaction to
fs/xfs/scrub/bitmap.c +174 -149
389 lines not on this page · raw message
fs/xfs/scrub/bitmap.h +4 -7
@@ -6,19 +6,14 @@#ifndef __XFS_SCRUB_BITMAP_H__#define __XFS_SCRUB_BITMAP_H__ -struct xbitmap_range { -struct list_head list; -uint64_t start; -uint64_t len; -}; -struct xbitmap { -struct list_head list; +struct rb_root_cached xb_root; };void xbitmap_init (struct xbitmap *bitmap);void xbitmap_destroy (struct xbitmap *bitmap); +int xbitmap_clear (struct xbitmap *bitmap,uint64_t start,uint64_t len);int xbitmap_set (struct xbitmap *bitmap,uint64_t start,uint64_t len);int xbitmap_disunion (struct xbitmap *bitmap,struct xbitmap *sub); int xbitmap_set_btcur_path(struct xbitmap *bitmap,
@@ -42,4 +37,6 @@ typedef int (*xbitmap_walk_bits_fn)(uint64_t bit, void *priv);int xbitmap_walk_bits (struct xbitmap *bitmap,xbitmap_walk_bits_fn fn,void *priv); +bool xbitmap_empty (struct xbitmap *bitmap); +#endif /* __XFS_SCRUB_BITMAP_H__ */
PATCH 8/23 xfs: port xbitmap_test
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
--- fs/xfs/Makefile | 2 +- fs/xfs/scrub/bitmap.c | 22 ++++++++++++++++++++++ fs/xfs/scrub/bitmap.h | 2 ++ 3 files changed, 25 insertions(+), 1 deletion(-)
fs/xfs/Makefile +1 -1
@@ -147,6 +147,7 @@ xfs-y += $(addprefix scrub/, \ agheader.o \ alloc.o \ attr.o \ + bitmap.o \ bmap.o \ btree.o \ common.o \
@@ -170,7 +171,6 @@ xfs-$(CONFIG_XFS_QUOTA) += scrub/quota.o ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ - bitmap.o \ repair.o \ ) endif
fs/xfs/scrub/bitmap.c +22 -0
@@ -396,3 +396,25 @@ xbitmap_empty( {return bitmap->xb_root.rb_root.rb_node ==NULL ; } + +/* Is the start of the range set or clear? And for how long? */ +bool +xbitmap_test ( +struct xbitmap *bitmap, +uint64_t start, +uint64_t *len) +{ +struct xbitmap_node *bn; +uint64_t last = start + *len -1 ; + + bn =xbitmap_tree_iter_first (&bitmap->xb_root, start, last); +if (!bn) +return false; +if (bn->bn_start <= start) { +if (bn->bn_last < last) + *len = bn->bn_last - start +1 ; +return true; + } + *len = bn->bn_start - start; +return false; +}
fs/xfs/scrub/bitmap.h +2 -0
@@ -39,4 +39,6 @@ int xbitmap_walk_bits(struct xbitmap *bitmap, xbitmap_walk_bits_fn fn,bool xbitmap_empty (struct xbitmap *bitmap); +bool xbitmap_test (struct xbitmap *bitmap,uint64_t start,uint64_t *len); +#endif /* __XFS_SCRUB_BITMAP_H__ */
PATCH 9/23 xfs: ignore stale buffers when scanning the buffer cache
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
After an online repair, we need to invalidate buffers representing the blocks from the old metadata that we're replacing. It's possible that parts of a tree that were previously cached in memory are no longer accessible due to media failure or other corruption on interior nodes, so repair figures out the old blocks from the reverse mapping data and scans the buffer cache directly.
Unfortunately, the current buffer cache code triggers asserts if the rhashtable lookup finds a non-stale buffer of a different length than the key we searched for. For regular operation this is desirable, but for this repair procedure, we don't care since we're going to forcibly stale the buffer anyway. Add an internal lookup flag to avoid the assert.
--- fs/xfs/xfs_buf.c | 5 ++++- fs/xfs/xfs_buf.h | 10 ++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-)
fs/xfs/xfs_buf.c +4 -1
@@ -482,7 +482,8 @@ _xfs_buf_obj_cmp(* reallocating a busy extent. Skip this buffer and * continue searching for an exact match. */ -ASSERT (bp->b_flags &XBF_STALE ); +if (!(map->bm_flags &XBM_IGNORE_LENGTH_MISMATCH )) +ASSERT (bp->b_flags &XBF_STALE );return 1 ; }return 0 ;
@@ -683,6 +684,8 @@ xfs_buf_get_map(int error;int i; +if (flags &XBF_BCACHE_SCAN ) + cmap.bm_flags |=XBM_IGNORE_LENGTH_MISMATCH ;for (i =0 ; i < nmaps; i++) cmap.bm_len += map[i].bm_len;
fs/xfs/xfs_buf.h +10 -0
@@ -44,6 +44,11 @@ struct xfs_buf;#define _XBF_DELWRI_Q (1u << 22)/* buffer on a delwri queue */ /* flags used only as arguments to access routines */ +/* +* We're scanning the buffer cache; do not warn about lookup mismatches. +* Only online repair should use this. +*/ +#define XBF_BCACHE_SCAN (1u << 28)#define XBF_INCORE (1u << 29)/* lookup only, return if found in cache */ #define XBF_TRYLOCK (1u << 30)/* lock requested, but do not wait */ #define XBF_UNMAPPED (1u << 31)/* do not map the buffer */
@@ -67,6 +72,7 @@ typedef unsigned int xfs_buf_flags_t; { _XBF_KMEM, "KMEM" }, \ { _XBF_DELWRI_Q, "DELWRI_Q" }, \/* The following interface flags should never be set */ \ + { XBF_BCACHE_SCAN, "BCACHE_SCAN" }, \ { XBF_INCORE, "INCORE" }, \ { XBF_TRYLOCK, "TRYLOCK" }, \ { XBF_UNMAPPED, "UNMAPPED" }
@@ -114,8 +120,12 @@ typedef struct xfs_buftarg {struct xfs_buf_map {xfs_daddr_t bm_bn;/* block number for I/O */ int bm_len;/* size of I/O */ +unsigned int bm_flags; }; +/* Don't complain about live buffers with the wrong length during lookup. */ +#define XBM_IGNORE_LENGTH_MISMATCH (1U << 0) +#define DEFINE_SINGLE_BUF_MAP (map, blkno, numblk) \ struct xfs_buf_map (map) = { .bm_bn = (blkno), .bm_len = (numblk) };
PATCH 10/23 xfs: create a big array data structure
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a simple 'big array' data structure for storage of fixed-size metadata records that will be used to reconstruct a btree index. For repair operations, the most important operations are append, iterate, and sort.
Earlier implementations of the big array used linked lists and suffered from severe problems -- pinning all records in kernel memory was not a good idea and frequently lead to OOM situations; random access was very inefficient; and record overhead for the lists was unacceptably high at 40-60%.
Therefore, the big memory array relies on the 'xfile' abstraction, which creates a memfd file and stores the records in page cache pages. Since the memfd is created in tmpfs, the memory pages can be pushed out to disk if necessary and we have a built-in usage limit of 50% of physical memory.
--- fs/xfs/Kconfig | 1 fs/xfs/Makefile | 2 fs/xfs/scrub/trace.c | 4 - fs/xfs/scrub/trace.h | 123 ++++++++++++++++ fs/xfs/scrub/xfarray.c | 370 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/xfarray.h | 58 ++++++++ fs/xfs/scrub/xfile.c | 318 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/xfile.h | 58 ++++++++ 8 files changed, 933 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/scrub/xfarray.c create mode 100644 fs/xfs/scrub/xfarray.h create mode 100644 fs/xfs/scrub/xfile.c create mode 100644 fs/xfs/scrub/xfile.h
fs/xfs/Kconfig +1 -0
@@ -97,6 +97,7 @@ config XFS_ONLINE_SCRUB bool "XFS online metadata check support" default n depends on XFS_FS + depends on TMPFS && SHMEM help If you say Y here you will be able to check metadata on a mounted XFS filesystem. This feature is intended to reduce
fs/xfs/Makefile +2 -0
@@ -162,6 +162,8 @@ xfs-y += $(addprefix scrub/, \ rmap.o \ scrub.o \ symlink.o \ + xfarray.o \ + xfile.o \ ) xfs-$(CONFIG_XFS_RT) += scrub/rtbitmap.o
fs/xfs/scrub/trace.c +3 -1
@@ -12,8 +12,10 @@#include "xfs_mount.h" #include "xfs_inode.h" #include "xfs_btree.h" -#include "scrub/scrub.h" #include "xfs_ag.h" +#include "scrub/scrub.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +123 -0
135 lines not on this page · raw message
fs/xfs/scrub/xfarray.c new file +370 -0
370 lines not on this page · raw message
fs/xfs/scrub/xfarray.h new file +58 -0
58 lines not on this page · raw message
fs/xfs/scrub/xfile.c new file +318 -0
318 lines not on this page · raw message
fs/xfs/scrub/xfile.h new file +58 -0
58 lines not on this page · raw message
PATCH 11/23 xfs: wrap ilock/iunlock operations on sc->ip
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Scrub tracks the resources that it's holding onto in the xfs_scrub structure. This includes the inode being checked (if applicable) and the inode lock state of that inode. Replace the open-coded structure manipulation with a trivial helper to eliminate sources of error.
--- fs/xfs/scrub/bmap.c | 6 ++---- fs/xfs/scrub/common.c | 38 +++++++++++++++++++++++++++++++++----- fs/xfs/scrub/common.h | 4 ++++ fs/xfs/scrub/dir.c | 3 +-- fs/xfs/scrub/inode.c | 6 ++---- fs/xfs/scrub/parent.c | 10 +++------- fs/xfs/scrub/quota.c | 9 +++------ fs/xfs/scrub/rtbitmap.c | 11 +++++------ fs/xfs/scrub/scrub.c | 2 +- 9 files changed, 54 insertions(+), 35 deletions(-)
fs/xfs/scrub/bmap.c +2 -4
@@ -35,8 +35,7 @@ xchk_setup_inode_bmap(if (error) goto out; - sc->ilock_flags =XFS_IOLOCK_EXCL |XFS_MMAPLOCK_EXCL ; -xfs_ilock (sc->ip, sc->ilock_flags); +xchk_ilock (sc,XFS_IOLOCK_EXCL |XFS_MMAPLOCK_EXCL );/* * We don't want any ephemeral data fork updates sitting around
@@ -73,9 +72,8 @@ xchk_setup_inode_bmap( error =xchk_trans_alloc (sc,0 );if (error) goto out; - sc->ilock_flags |=XFS_ILOCK_EXCL ; -xfs_ilock (sc->ip,XFS_ILOCK_EXCL ); +xchk_ilock (sc,XFS_ILOCK_EXCL ); out:/* scrub teardown will unlock and release the inode */ return error;
fs/xfs/scrub/common.c +33 -5
@@ -750,19 +750,47 @@ xchk_setup_inode_contents(return error;/* Got the inode, lock it and we're ready to go. */ - sc->ilock_flags =XFS_IOLOCK_EXCL |XFS_MMAPLOCK_EXCL ; -xfs_ilock (sc->ip, sc->ilock_flags); +xchk_ilock (sc,XFS_IOLOCK_EXCL |XFS_MMAPLOCK_EXCL ); error =xchk_trans_alloc (sc, resblks);if (error) goto out; - sc->ilock_flags |=XFS_ILOCK_EXCL ; -xfs_ilock (sc->ip,XFS_ILOCK_EXCL ); - +xchk_ilock (sc,XFS_ILOCK_EXCL ); out:/* scrub teardown will unlock and release the inode for us */ return error; } +void +xchk_ilock ( +struct xfs_scrub *sc, +unsigned int ilock_flags) +{ + sc->ilock_flags |= ilock_flags; +xfs_ilock (sc->ip, ilock_flags); +} + +bool +xchk_ilock_nowait ( +struct xfs_scrub *sc, +unsigned int ilock_flags) +{ +if (xfs_ilock_nowait (sc->ip, ilock_flags)) { + sc->ilock_flags |= ilock_flags; +return true; + } + +return false; +} + +void +xchk_iunlock ( +struct xfs_scrub *sc, +unsigned int ilock_flags) +{ +xfs_iunlock (sc->ip, ilock_flags); + sc->ilock_flags &= ~ilock_flags; +} +/* * Predicate that decides if we need to evaluate the cross-reference check. * If there was an error accessing the cross-reference btree, just delete
fs/xfs/scrub/common.h +4 -0
@@ -139,6 +139,10 @@ void xchk_buffer_recheck(struct xfs_scrub *sc, struct xfs_buf *bp);int xchk_iget (struct xfs_scrub *sc,xfs_ino_t inum,struct xfs_inode **ipp);void xchk_irele (struct xfs_scrub *sc,struct xfs_inode *ip); +void xchk_ilock (struct xfs_scrub *sc,unsigned int ilock_flags); +bool xchk_ilock_nowait (struct xfs_scrub *sc,unsigned int ilock_flags); +void xchk_iunlock (struct xfs_scrub *sc,unsigned int ilock_flags); +/* * Don't bother cross-referencing if we already found corruption or cross * referencing discrepancies.
fs/xfs/scrub/dir.c +1 -2
@@ -840,8 +840,7 @@ xchk_directory(* _dir_lookup routines, which do their own ILOCK locking. */ oldpos =0 ; - sc->ilock_flags &= ~XFS_ILOCK_EXCL ; -xfs_iunlock (sc->ip,XFS_ILOCK_EXCL ); +xchk_iunlock (sc,XFS_ILOCK_EXCL ); while (true) { error = xfs_readdir(sc->tp, sc->ip, &sdc.dir_iter, bufsize); if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, 0,
fs/xfs/scrub/inode.c +2 -4
@@ -48,13 +48,11 @@ xchk_setup_inode( }/* Got the inode, lock it and we're ready to go. */ - sc->ilock_flags =XFS_IOLOCK_EXCL |XFS_MMAPLOCK_EXCL ; -xfs_ilock (sc->ip, sc->ilock_flags); +xchk_ilock (sc,XFS_IOLOCK_EXCL |XFS_MMAPLOCK_EXCL ); error =xchk_trans_alloc (sc,0 );if (error) goto out; - sc->ilock_flags |=XFS_ILOCK_EXCL ; -xfs_ilock (sc->ip,XFS_ILOCK_EXCL ); +xchk_ilock (sc,XFS_ILOCK_EXCL ); out:/* scrub teardown will unlock and release the inode for us */
fs/xfs/scrub/parent.c +3 -7
@@ -137,8 +137,7 @@ xchk_parent_lock_two_dirs(return -EINVAL ; } -xfs_iunlock (sc->ip, sc->ilock_flags); - sc->ilock_flags =0 ; +xchk_iunlock (sc, sc->ilock_flags);while (true) {if (xchk_should_terminate (sc, &error))return error;
@@ -149,10 +148,8 @@ xchk_parent_lock_two_dirs(* on either IOLOCK. */ if (xfs_ilock_nowait (dp,XFS_IOLOCK_SHARED )) { -if (xfs_ilock_nowait (sc->ip,XFS_IOLOCK_EXCL )) { - sc->ilock_flags =XFS_IOLOCK_EXCL ; +if (xchk_ilock_nowait (sc,XFS_IOLOCK_EXCL ))break ; - }xfs_iunlock (dp,XFS_IOLOCK_SHARED ); }
@@ -299,8 +296,7 @@ xchk_parent(* getting a write lock on i_rwsem. Therefore, it is safe for us * to drop the ILOCK here in order to do directory lookups. */ - sc->ilock_flags &= ~(XFS_ILOCK_EXCL |XFS_MMAPLOCK_EXCL ); -xfs_iunlock (sc->ip,XFS_ILOCK_EXCL |XFS_MMAPLOCK_EXCL ); +xchk_iunlock (sc,XFS_ILOCK_EXCL |XFS_MMAPLOCK_EXCL );/* Look up '..' */ error = xfs_dir_lookup(sc->tp, sc->ip, &xfs_name_dotdot, &parent_ino,
fs/xfs/scrub/quota.c +3 -6
@@ -57,8 +57,7 @@ xchk_setup_quota(if (error)return error; sc->ip =xfs_quota_inode (sc->mp, dqtype); -xfs_ilock (sc->ip,XFS_ILOCK_EXCL ); - sc->ilock_flags =XFS_ILOCK_EXCL ; +xchk_ilock (sc,XFS_ILOCK_EXCL );return 0 ; }
@@ -232,13 +231,11 @@ xchk_quota(* data fork we have to drop ILOCK_EXCL to use the regular dquot * functions. */ -xfs_iunlock (sc->ip, sc->ilock_flags); - sc->ilock_flags =0 ; +xchk_iunlock (sc, sc->ilock_flags); sqi.sc = sc; sqi.last_id =0 ; error =xfs_qm_dqiterate (mp, dqtype, xchk_quota_item, &sqi); - sc->ilock_flags =XFS_ILOCK_EXCL ; -xfs_ilock (sc->ip, sc->ilock_flags); +xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,
fs/xfs/scrub/rtbitmap.c +5 -6
@@ -28,10 +28,9 @@ xchk_setup_rt(if (error)return error; - sc->ilock_flags =XFS_ILOCK_EXCL |XFS_ILOCK_RTBITMAP ; sc->ip = sc->mp->m_rbmip; -xfs_ilock (sc->ip, sc->ilock_flags); - + sc->ilock_flags =0 ; +xchk_ilock (sc,XFS_ILOCK_EXCL |XFS_ILOCK_RTBITMAP );return 0 ; }
@@ -141,8 +140,8 @@ xchk_rtsummary(* flags so that we don't mix up the inode state that @sc tracks. */ sc->ip = rsumip; - sc->ilock_flags =XFS_ILOCK_EXCL |XFS_ILOCK_RTSUM ; -xfs_ilock (sc->ip, sc->ilock_flags); + sc->ilock_flags =0 ; +xchk_ilock (sc,XFS_ILOCK_EXCL |XFS_ILOCK_RTSUM );/* Invoke the fork scrubber. */ error =xchk_metadata_inode_forks (sc);
@@ -153,7 +152,7 @@ xchk_rtsummary(xchk_set_incomplete (sc); out:/* Switch back to the rtbitmap inode and lock flags. */ -xfs_iunlock (sc->ip, sc->ilock_flags); +xchk_iunlock (sc,XFS_ILOCK_EXCL |XFS_ILOCK_RTSUM ); sc->ilock_flags = old_ilock_flags; sc->ip = old_ip;return error;
fs/xfs/scrub/scrub.c +1 -1
@@ -163,7 +163,7 @@ xchk_teardown( }if (sc->ip) {if (sc->ilock_flags) -xfs_iunlock (sc->ip, sc->ilock_flags); +xchk_iunlock (sc, sc->ilock_flags);if (sc->ip != ip_in && !xfs_internal_inum (sc->mp, sc->ip->i_ino))xchk_irele (sc, sc->ip);
PATCH 12/23 xfs: port scrub inode scan from djwong-dev
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Port the inode scanning code from djwong-dev so we can use it here.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/iscan.c | 483 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/iscan.h | 65 +++++++ fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 73 ++++++++ fs/xfs/xfs_icache.c | 3 fs/xfs/xfs_icache.h | 11 + 7 files changed, 632 insertions(+), 5 deletions(-) create mode 100644 fs/xfs/scrub/iscan.c create mode 100644 fs/xfs/scrub/iscan.h
fs/xfs/Makefile +1 -0
@@ -157,6 +157,7 @@ xfs-y += $(addprefix scrub/, \ health.o \ ialloc.o \ inode.o \ + iscan.o \ parent.o \ refcount.o \ rmap.o \
fs/xfs/scrub/iscan.c new file +483 -0
483 lines not on this page · raw message
fs/xfs/scrub/iscan.h new file +65 -0
65 lines not on this page · raw message
fs/xfs/scrub/trace.c +1 -0
@@ -16,6 +16,7 @@#include "scrub/scrub.h" #include "scrub/xfile.h" #include "scrub/xfarray.h" +#include "scrub/iscan.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +73 -0
85 lines not on this page · raw message
fs/xfs/xfs_icache.c +2 -1
@@ -767,7 +767,8 @@ xfs_iget(return 0 ; out_error_or_again: -if (!(flags &XFS_IGET_INCORE ) && error == -EAGAIN ) { +if (!(flags & (XFS_IGET_INCORE |XFS_IGET_NORETRY )) && + error == -EAGAIN ) {delay (1 ); goto again; }
fs/xfs/xfs_icache.h +7 -4
@@ -34,10 +34,13 @@ struct xfs_icwalk {/* * Flags for xfs_iget() */ -#define XFS_IGET_CREATE 0x1 -#define XFS_IGET_UNTRUSTED 0x2 -#define XFS_IGET_DONTCACHE 0x4 -#define XFS_IGET_INCORE 0x8/* don't read from disk or reinit */ +#define XFS_IGET_CREATE (1U << 0) +#define XFS_IGET_UNTRUSTED (1U << 1) +#define XFS_IGET_DONTCACHE (1U << 2) +/* don't read from disk or reinit */ +#define XFS_IGET_INCORE (1U << 3) +/* Return -EAGAIN immediately if the inode is unavailable. */ +#define XFS_IGET_NORETRY (1U << 4)int xfs_iget (struct xfs_mount *mp,struct xfs_trans *tp,xfs_ino_t ino,uint flags,uint lock_flags,xfs_inode_t **ipp);
PATCH 13/23 xfs: allow scrub to hook metadata updates in other writers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Certain types of filesystem metadata can only be checked by scanning every file in the entire filesystem. Specific examples of this include quota counts, file link counts, and reverse mappings of file extents. Directory and parent pointer reconstruction may also fall into this category. File scanning is much trickier than scanning AG metadata because we have to take inode locks in the same order as the rest of [VX]FS, we can't be holding buffer locks when we do that, and scanning the whole filesystem takes time.
Earlier versions of the online repair patchset relied heavily on fsfreeze as a means to quiesce the filesystem so that we could take locks in the proper order without worrying about concurrent updates from other writers. Reviewers of those patches opined that freezing the entire fs to check and repair something was not sufficiently better than unmounting to run fsck offline. I don't agree with that 100%, but the message was clear: find a way to repair things that minimizes the quiet period where nobody can write to the filesystem.
Generally, building btree indexes online can be split into two phases: a collection phase where we compute the records that will be put into the new btree; and a construction phase, where we construct the physical btree blocks and persist them. While it's simple to hold resource locks for the entirety of the two phases to ensure that the new index is consistent with the rest of the system, we don't need to hold resource locks during the collection phase if we have a means to receive live updates of other work going on elsewhere in the system.
The goal of this patch, then, is to enable online fsck to learn about metadata updates going on in other threads while it constructs a shadow copy of the metadata records to verify or correct the real metadata. To minimize the overhead when online fsck isn't running, we use srcu notifiers because they prioritize fast access to the notifier call chain (particularly when the chain is empty) at a cost to configuring notifiers. Online fsck should be relatively infrequent, so this is acceptable.
The intended usage model is fairly simple. Code that modifies a metadata structure of interest should declare a xfs_hook_chain structure in some well defined place, and call xfs_hook_call whenever an update happens. Online fsck code should define a struct notifier_block and use xfs_hook_add to attach the block to the chain, along with a function to be called. This function should synchronize with the fsck scanner to update whatever in-memory data the scanner is collecting. When finished, xfs_hook_del removes the notifier from the list and waits for them all to complete.
On the author's computer, calling an empty srcu notifier chain was observed to have an overhead averaging ~40ns with a maximum of 60ns. Adding a no-op notifier function increased the average to ~58ns and 66ns. When the quotacheck live update notifier is attached, the average increases to ~322ns with a max of 372ns to update scrub's in-memory observation data, assuming no lock contention.
With jump labels enabled, calls to empty srcu notifier chains are elided from the call sites when there are no hooks registered, which means that the overhead is 0.36ns when fsck is not running. For compilers that do not support jump labels (all major architectures do), the overhead of a no-op notifier call is less bad (on a many-cpu system) than the atomic counter ops, so we make the hook switch itself a nop.
Note: This new code is also split out as a separate patch from its initial user so that the author can move patches around his tree with ease.
--- fs/xfs/Kconfig | 6 +++++ fs/xfs/Makefile | 2 ++ fs/xfs/xfs_hooks.c | 53 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_hooks.h | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_linux.h | 1 + 5 files changed, 130 insertions(+) create mode 100644 fs/xfs/xfs_hooks.c create mode 100644 fs/xfs/xfs_hooks.h
fs/xfs/Kconfig +6 -0
@@ -92,12 +92,18 @@ config XFS_RT See the xfs man page in section 5 for additional information. If unsure, say N. + +config XFS_LIVE_HOOKS + bool + select JUMP_LABEL if HAVE_ARCH_JUMP_LABEL config XFS_ONLINE_SCRUB bool "XFS online metadata check support" default n depends on XFS_FS depends on TMPFS && SHMEM + depends on SRCU + select XFS_LIVE_HOOKS help If you say Y here you will be able to check metadata on a mounted XFS filesystem. This feature is intended to reduce
fs/xfs/Makefile +2 -0
@@ -137,6 +137,8 @@ ifeq ($(CONFIG_MEMORY_FAILURE),y) xfs-$(CONFIG_FS_DAX) += xfs_notify_failure.o endif +xfs-$(CONFIG_XFS_LIVE_HOOKS) += xfs_hooks.o + # online scrub/repair ifeq ($(CONFIG_XFS_ONLINE_SCRUB),y)
fs/xfs/xfs_hooks.c new file +53 -0
53 lines not on this page · raw message
fs/xfs/xfs_hooks.h new file +68 -0
68 lines not on this page · raw message
fs/xfs/xfs_linux.h +1 -0
@@ -79,6 +79,7 @@ typedef __u32 xfs_nlink_t;#include "xfs_cksum.h" #include "xfs_buf.h" #include "xfs_message.h" +#include "xfs_hooks.h" #ifdef __BIG_ENDIAN#define XFS_NATIVE_HOST 1
PATCH 14/23 xfs: allow blocking notifier chains with filesystem hooks
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make it so that we can switch between notifier chain implementations for testing purposes. On the author's test system, calling an empty srcu notifier chain cost about 19ns per call, vs. 4ns for a blocking notifier chain. Hm. Might we actually want regular blocking notifiers?
--- fs/xfs/Kconfig | 33 ++++++++++++++++++++++++++++++++- fs/xfs/xfs_hooks.c | 41 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_hooks.h | 6 +++++- 3 files changed, 78 insertions(+), 2 deletions(-)
fs/xfs/Kconfig +32 -1
@@ -102,7 +102,6 @@ config XFS_ONLINE_SCRUB default n depends on XFS_FS depends on TMPFS && SHMEM - depends on SRCU select XFS_LIVE_HOOKS help If you say Y here you will be able to check metadata on a
@@ -117,6 +116,38 @@ config XFS_ONLINE_SCRUB If unsure, say N. +choice + prompt "XFS hook implementation" + depends on XFS_FS && XFS_LIVE_HOOKS && XFS_ONLINE_SCRUB + default XFS_LIVE_HOOKS_BLOCKING if HAVE_ARCH_JUMP_LABEL + default XFS_LIVE_HOOKS_SRCU if !HAVE_ARCH_JUMP_LABEL + help + Pick one + +config XFS_LIVE_HOOKS_SRCU + bool "SRCU notifier chains" + depends on SRCU + help + Use SRCU notifier chains for filesystem hooks. These have very low + overhead for event initiators (the main filesystem) and higher + overhead for chain modifiers (scrub waits for RCU grace). This is + the best option when jump labels are not supported or there are many + CPUs in the system. + + This may cause problems with CPU hotplug invoking reclaim invoking + XFS. + +config XFS_LIVE_HOOKS_BLOCKING + bool "Blocking notifier chains" + help + Use blocking notifier chains for filesystem hooks. These have medium + overhead for event initiators (the main fs) and chain modifiers + (scrub) due to their use of rwsems. This is the best option when + jump labels can be used to eliminate overhead for the filesystem when + scrub is not running. + +endchoice + config XFS_ONLINE_REPAIR bool "XFS online metadata repair support" default n
fs/xfs/xfs_hooks.c +41 -0
50 lines not on this page · raw message
fs/xfs/xfs_hooks.h +5 -1
@@ -6,10 +6,14 @@#ifndef XFS_HOOKS_H_ #define XFS_HOOKS_H_ -#ifdef CONFIG_XFS_LIVE_HOOKS +#if defined(CONFIG_XFS_LIVE_HOOKS_SRCU )struct xfs_hooks {struct srcu_notifier_head head; }; +#elif defined(CONFIG_XFS_LIVE_HOOKS_BLOCKING ) +struct xfs_hooks { +struct blocking_notifier_head head; +};#else struct xfs_hooks {/* empty */ };#endif
PATCH 15/23 xfs: streamline the directory iteration code for scrub
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Currently, online scrub reuses the xfs_readdir code to walk every entry in a directory. This isn't awesome for performance, since we end up cycling the directory ILOCK needlessly and coding around the particular quirks of the VFS dir_context interface.
Create a streamlined version of readdir that keeps the ILOCK (since the walk function isn't going to copy stuff to userspace), skips a whole lot of directory walk cursor checks (since we start at 0 and walk to the end) and has a sane way to return error codes.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/dir.c | 173 +++++++--------------- fs/xfs/scrub/parent.c | 90 +++--------- fs/xfs/scrub/readdir.c | 375 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/readdir.h | 19 ++ 5 files changed, 473 insertions(+), 185 deletions(-) create mode 100644 fs/xfs/scrub/readdir.c create mode 100644 fs/xfs/scrub/readdir.h
fs/xfs/Makefile +1 -0
@@ -161,6 +161,7 @@ xfs-y += $(addprefix scrub/, \ inode.o \ iscan.o \ parent.o \ + readdir.o \ refcount.o \ rmap.o \ scrub.o \
fs/xfs/scrub/dir.c +51 -122
270 lines not on this page · raw message
fs/xfs/scrub/parent.c +27 -63
148 lines not on this page · raw message
fs/xfs/scrub/readdir.c new file +375 -0
375 lines not on this page · raw message
fs/xfs/scrub/readdir.h new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2022 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_SCRUB_READDIR_H__ +#define __XFS_SCRUB_READDIR_H__ + +typedef int (*xchk_dirent_fn )(struct xfs_scrub *sc,struct xfs_inode *dp, +xfs_dir2_dataptr_t dapos,const struct xfs_name *name, +xfs_ino_t ino,void *priv); + +int xchk_dir_walk (struct xfs_scrub *sc,struct xfs_inode *dp, +xchk_dirent_fn dirent_fn,void *priv); + +int xchk_dir_lookup (struct xfs_scrub *sc,struct xfs_inode *dp, +const struct xfs_name *name,xfs_ino_t *ino); + +#endif /* __XFS_SCRUB_READDIR_H__ */
PATCH 16/23 xfs: track file link count updates during live nlinks fsck
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create the necessary hooks in the file create/unlink/rename code so that our live nlink scrub code can stay up to date with the rest of the filesystem. This will be the means to keep our shadow link count information up to date while the scan runs in real time.
--- fs/xfs/libxfs/xfs_dir2.c | 6 + fs/xfs/libxfs/xfs_dir2.h | 1 fs/xfs/scrub/common.c | 20 ++++ fs/xfs/scrub/common.h | 2 fs/xfs/scrub/scrub.c | 17 +++ fs/xfs/scrub/scrub.h | 3 + fs/xfs/scrub/trace.h | 42 +++++++++ fs/xfs/xfs_inode.c | 226 ++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_inode.h | 35 +++++++ fs/xfs/xfs_mount.h | 2 fs/xfs/xfs_super.c | 2 fs/xfs/xfs_symlink.c | 1 12 files changed, 357 insertions(+)
fs/xfs/libxfs/xfs_dir2.c +6 -0
@@ -25,6 +25,12 @@ const struct xfs_name xfs_name_dotdot = { .type = XFS_DIR3_FT_DIR, }; +const struct xfs_name xfs_name_dot = { + .name = (const unsigned char *)"." , + .len =1 , + .type =XFS_DIR3_FT_DIR , +}; +/* * Convert inode mode to directory entry filetype */
fs/xfs/libxfs/xfs_dir2.h +1 -0
@@ -22,6 +22,7 @@ struct xfs_dir3_icfree_hdr;struct xfs_dir3_icleaf_hdr ;extern const struct xfs_name xfs_name_dotdot; +extern const struct xfs_name xfs_name_dot;/* * Convert inode mode to directory entry filetype
fs/xfs/scrub/common.c +20 -0
@@ -961,3 +961,23 @@ xchk_start_reaping( } sc->flags &= ~XCHK_REAPING_DISABLED ; } + +/* +* Enable filesystem hooks (i.e. runtime code patching) before starting a scrub +* operation. Callers must not hold any locks that intersect with the CPU +* hotplug lock (e.g. writeback locks) because code patching must halt the CPUs +* to change kernel code. +*/ +void +xchk_fshooks_enable ( +struct xfs_scrub *sc, +unsigned int scrub_fshooks) +{ +ASSERT (!(scrub_fshooks & ~XCHK_FSHOOKS_ALL )); +ASSERT (!(sc->flags & scrub_fshooks)); + +if (scrub_fshooks &XCHK_FSHOOKS_DIRENTS ) +xfs_dirent_hook_enable (); + + sc->flags |= scrub_fshooks; +}
fs/xfs/scrub/common.h +2 -0
@@ -157,4 +157,6 @@ int xchk_metadata_inode_forks(struct xfs_scrub *sc);void xchk_stop_reaping (struct xfs_scrub *sc);void xchk_start_reaping (struct xfs_scrub *sc); +void xchk_fshooks_enable (struct xfs_scrub *sc,unsigned int scrub_fshooks); +#endif /* __XFS_SCRUB_COMMON_H__ */
fs/xfs/scrub/scrub.c +17 -0
@@ -145,6 +145,21 @@ xchk_probe(/* Scrub setup and teardown */ +static inline void +xchk_fshooks_disable ( +struct xfs_scrub *sc) +{ +if (!(sc->flags &XCHK_FSHOOKS_ALL )) +return ; + +//trace_xchk_fshooks_disable(sc, sc->flags & XCHK_FSHOOKS_ALL); + +if (sc->flags &XCHK_FSHOOKS_DIRENTS ) +xfs_dirent_hook_disable (); + + sc->flags &= ~XCHK_FSHOOKS_ALL ; +} +/* Free all the resources and finish the transactions. */ STATIC int xchk_teardown(
@@ -177,6 +192,8 @@ xchk_teardown(kvfree (sc->buf); sc->buf =NULL ; } + +xchk_fshooks_disable (sc);return error; }
fs/xfs/scrub/scrub.h +3 -0
@@ -97,8 +97,11 @@ struct xfs_scrub {/* XCHK state flags grow up from zero, XREP state flags grown down from 2^31 */ #define XCHK_TRY_HARDER (1 << 0)/* can't get resources, try again */ #define XCHK_REAPING_DISABLED (1 << 2)/* background block reaping paused */ +#define XCHK_FSHOOKS_DIRENTS (1 << 5)/* link count live update enabled */ #define XREP_ALREADY_FIXED (1 << 31)/* checking our repair work */ +#define XCHK_FSHOOKS_ALL (XCHK_FSHOOKS_DIRENTS) +/* Metadata scrubbers */ int xchk_tester (struct xfs_scrub *sc);int xchk_superblock (struct xfs_scrub *sc);
fs/xfs/scrub/trace.h +42 -0
48 lines not on this page · raw message
fs/xfs/xfs_inode.c +226 -0
268 lines not on this page · raw message
fs/xfs/xfs_inode.h +35 -0
@@ -576,4 +576,39 @@ int xfs_ilock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);void xfs_iunlock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);void xfs_lock_inodes (struct xfs_inode **ips,int inodes,uint lock_mode); +/* +* Parameters for tracking bumplink and droplink operations. The hook +* function arg parameter is one of these. +*/ +enum xfs_dirent_update_type { +XFS_DIRENT_CHILD_DELTA ,/* parent pointing to child */ +XFS_DIRENT_BACKREF_DELTA ,/* dotdot entries */ +XFS_DIRENT_SELF_DELTA ,/* dot entries */ +}; + +struct xfs_dirent_update_params { +const struct xfs_inode *dp; +const struct xfs_inode *ip; +const struct xfs_name *name; +int delta; +}; + +#ifdef CONFIG_XFS_LIVE_HOOKS +void xfs_dirent_child_delta (struct xfs_inode *dp,struct xfs_inode *ip, +int delta,struct xfs_name *name); + +struct xfs_dirent_hook { +struct xfs_hook delta_hook; +}; + +void xfs_dirent_hook_disable (void ); +void xfs_dirent_hook_enable (void ); + +int xfs_dirent_hook_add (struct xfs_mount *mp,struct xfs_dirent_hook *hook); +void xfs_dirent_hook_del (struct xfs_mount *mp,struct xfs_dirent_hook *hook); + +#else +# define xfs_dirent_child_delta (dp, ip, delta, name) ((void)0) +#endif /* CONFIG_XFS_LIVE_HOOKS */ +#endif /* __XFS_INODE_H__ */
fs/xfs/xfs_mount.h +2 -0
@@ -242,6 +242,8 @@ typedef struct xfs_mount {unsigned int *m_errortag;struct xfs_kobj m_errortag_kobj;#endif +/* Hook to feed file directory updates to an active online repair. */ +struct xfs_hooks m_dirent_update_hooks; } xfs_mount_t;#define M_IGEO (mp) (&(mp)->m_ino_geo)
fs/xfs/xfs_super.c +2 -0
@@ -1949,6 +1949,8 @@ static int xfs_init_fs_context( mp->m_logbsize =-1 ; mp->m_allocsize_log =16 ;/* 64k */ +xfs_hooks_init (&mp->m_dirent_update_hooks); +/* * Copy binary VFS mount flags we are interested in. */
fs/xfs/xfs_symlink.c +1 -0
@@ -354,6 +354,7 @@ xfs_symlink( goto out_trans_cancel; } +xfs_dirent_child_delta (dp, ip,1 , link_name);/* * If this is a synchronous mount, make sure that the
PATCH 17/23 xfs: connect in-memory btrees to xfiles
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add to our stubbed-out in-memory btrees the ability to connect them with an actual in-memory backing file (aka xfiles) and the necessary pieces to track free space in the xfile and flush dirty xfbtree buffers on demand, which we'll need for online repair.
--- fs/xfs/scrub/trace.h | 1 + fs/xfs/scrub/xfile.c | 11 +++++++++++ fs/xfs/scrub/xfile.h | 2 ++ 3 files changed, 14 insertions(+)
fs/xfs/scrub/trace.h +1 -0
@@ -755,6 +755,7 @@ DEFINE_EVENT(xfile_class, name, \DEFINE_XFILE_EVENT (xfile_pread);DEFINE_XFILE_EVENT (xfile_pwrite);DEFINE_XFILE_EVENT (xfile_seek_data); +DEFINE_XFILE_EVENT (xfile_discard); TRACE_EVENT(xfarray_create, TP_PROTO(struct xfarray *xfa, unsigned long long required_capacity),
fs/xfs/scrub/xfile.c +11 -0
@@ -285,6 +285,17 @@ xfile_pwrite(return error; } +/* Discard pages backing a range of the xfile. */ +void +xfile_discard ( +struct xfile *xf, +loff_t pos, +u64 count) +{ +trace_xfile_discard (xf, pos, count); +shmem_truncate_range (file_inode (xf->file), pos, pos + count -1 ); +} +/* Find the next written area in the xfile data for a given offset. */ loff_t xfile_seek_data(
fs/xfs/scrub/xfile.h +2 -0
@@ -46,6 +46,8 @@ xfile_obj_store(struct xfile *xf, const void *buf, size_t count, loff_t pos)return 0 ; } +void xfile_discard (struct xfile *xf,loff_t pos,u64 count); +int xfile_prealloc (struct xfile *xf,loff_t pos,u64 count);loff_t xfile_seek_data (struct xfile *xf,loff_t pos);struct xfile_stat {
PATCH 18/23 xfs: create temporary files and directories for online repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Teach the online repair code how to create temporary files or directories. These temporary files can be used to stage reconstructed information until we're ready to perform an atomic extent swap to commit the new metadata.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/common.c | 1 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/scrub.h | 4 + fs/xfs/scrub/tempfile.c | 230 +++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/tempfile.h | 27 ++++++ fs/xfs/scrub/trace.h | 33 +++++++ fs/xfs/xfs_inode.c | 3 - fs/xfs/xfs_inode.h | 2 9 files changed, 301 insertions(+), 2 deletions(-) create mode 100644 fs/xfs/scrub/tempfile.c create mode 100644 fs/xfs/scrub/tempfile.h
fs/xfs/Makefile +1 -0
@@ -178,6 +178,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ repair.o \ + tempfile.o \ ) endif endif
fs/xfs/scrub/common.c +1 -0
@@ -33,6 +33,7 @@#include "scrub/trace.h" #include "scrub/repair.h" #include "scrub/health.h" +#include "scrub/tempfile.h" /* Common code for the metadata scrubbers. */
fs/xfs/scrub/scrub.c +2 -0
@@ -22,6 +22,7 @@#include "scrub/trace.h" #include "scrub/repair.h" #include "scrub/health.h" +#include "scrub/tempfile.h" /* * Online Scrub and Repair
@@ -193,6 +194,7 @@ xchk_teardown( sc->buf =NULL ; } +xrep_tempfile_rele (sc);xchk_fshooks_disable (sc);return error; }
fs/xfs/scrub/scrub.h +4 -0
@@ -80,6 +80,10 @@ struct xfs_scrub {void *buf;uint ilock_flags; +/* A temporary file on this filesystem, for staging new metadata. */ +struct xfs_inode *tempip; +uint temp_ilock_flags; +/* See the XCHK/XREP state flags below. */ unsigned int flags;
fs/xfs/scrub/tempfile.c new file +230 -0
230 lines not on this page · raw message
fs/xfs/scrub/tempfile.h new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2022 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_SCRUB_TEMPFILE_H__ +#define __XFS_SCRUB_TEMPFILE_H__ + +#ifdef CONFIG_XFS_ONLINE_REPAIR +int xrep_tempfile_create (struct xfs_scrub *sc,uint16_t mode); +void xrep_tempfile_rele (struct xfs_scrub *sc); + +bool xrep_tempfile_iolock_nowait (struct xfs_scrub *sc); +void xrep_tempfile_iounlock (struct xfs_scrub *sc); + +void xrep_tempfile_ilock (struct xfs_scrub *sc); +bool xrep_tempfile_ilock_nowait (struct xfs_scrub *sc); +void xrep_tempfile_iunlock (struct xfs_scrub *sc); +#else +static inline void xrep_tempfile_iolock_both (struct xfs_scrub *sc) +{ +xchk_ilock (sc,XFS_IOLOCK_EXCL ); +} +# define xrep_tempfile_rele (sc) +#endif /* CONFIG_XFS_ONLINE_REPAIR */ + +#endif /* __XFS_SCRUB_TEMPFILE_H__ */
fs/xfs/scrub/trace.h +33 -0
@@ -1149,6 +1149,39 @@ TRACE_EVENT(xrep_ialloc_insert, __entry->freemask) ) +TRACE_EVENT (xrep_tempfile_create, + TP_PROTO(struct xfs_scrub *sc), +TP_ARGS (sc), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, type) + __field(xfs_agnumber_t, agno) + __field(xfs_ino_t, inum) + __field(unsigned int, gen) + __field(unsigned int, flags) +__field (xfs_ino_t, temp_inum) + ), +TP_fast_assign ( + __entry->dev = sc->mp->m_super->s_dev; + __entry->ino = sc->file ? XFS_I(file_inode(sc->file))->i_ino : 0; + __entry->type = sc->sm->sm_type; + __entry->agno = sc->sm->sm_agno; + __entry->inum = sc->sm->sm_ino; + __entry->gen = sc->sm->sm_gen; + __entry->flags = sc->sm->sm_flags; + __entry->temp_inum = sc->tempip->i_ino; + ), +TP_printk ("dev %d:%d ino 0x%llx type %s inum 0x%llx gen 0x%x flags 0x%x temp_inum 0x%llx" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, +__print_symbolic (__entry->type,XFS_SCRUB_TYPE_STRINGS ), + __entry->inum, + __entry->gen, + __entry->flags, + __entry->temp_inum) +); +#endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */ #endif /* _TRACE_XFS_SCRUB_TRACE_H */
fs/xfs/xfs_inode.c +1 -2
@@ -48,7 +48,6 @@ struct kmem_cache *xfs_inode_cache;*/ #define XFS_ITRUNC_MAX_EXTENTS 2 -STATIC int xfs_iunlink(struct xfs_trans *, struct xfs_inode *); STATIC int xfs_iunlink_remove(struct xfs_trans *tp, struct xfs_perag *pag,struct xfs_inode *);
@@ -2187,7 +2186,7 @@ xfs_iunlink_insert_inode(* We place the on-disk inode on a list in the AGI. It will be pulled from this * list when the inode is freed. */ -STATIC int +int xfs_iunlink (struct xfs_trans *tp,struct xfs_inode *ip)
fs/xfs/xfs_inode.h +2 -0
@@ -570,6 +570,8 @@ extern struct kmem_cache *xfs_inode_cache;bool xfs_inode_needs_inactive (struct xfs_inode *ip); +int xfs_iunlink (struct xfs_trans *tp,struct xfs_inode *ip); +void xfs_end_io (struct work_struct *work);int xfs_ilock2_io_mmap (struct xfs_inode *ip1,struct xfs_inode *ip2);
PATCH 19/23 xfs: hide private inodes from bulkstat and handle functions
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We're about to start adding functionality that uses internal inodes that are private to XFS. What this means is that userspace should never be able to access any information about these files, and should not be able to open these files by handle. Callers are not allowed to link these files into the directory tree, which should suffice to make these private inodes actually private.
--- fs/xfs/xfs_export.c | 2 +- fs/xfs/xfs_itable.c | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-)
fs/xfs/xfs_export.c +1 -1
@@ -146,7 +146,7 @@ xfs_nfs_get_inode(return ERR_PTR (error); } -if (VFS_I (ip)->i_generation != generation) { +if (VFS_I (ip)->i_generation != generation ||IS_PRIVATE (VFS_I (ip))) {xfs_irele (ip);return ERR_PTR (-ESTALE ); }
fs/xfs/xfs_itable.c +8 -0
@@ -86,6 +86,14 @@ xfs_bulkstat_one_int( vfsuid =i_uid_into_vfsuid (mnt_userns, inode); vfsgid =i_gid_into_vfsgid (mnt_userns, inode); +/* If this is a private inode, don't leak its details to userspace. */ +if (IS_PRIVATE (inode)) { +xfs_iunlock (ip,XFS_ILOCK_SHARED ); +xfs_irele (ip); + error = -EINVAL ; + goto out_advance; + } +/* xfs_iget returns the following without needing * further change. */
PATCH 20/23 xfs: create a blob array data structure
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a simple 'blob array' data structure for storage of arbitrarily sized metadata objects that will be used to reconstruct metadata. For the intended usage (temporarily storing extended attribute names and values) we only have to support storing objects and retrieving them. Use the xfile abstraction to store the attribute information in memory that can be swapped out.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/xfblob.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/xfblob.h | 25 ++++++++ 3 files changed, 178 insertions(+) create mode 100644 fs/xfs/scrub/xfblob.c create mode 100644 fs/xfs/scrub/xfblob.h
fs/xfs/Makefile +1 -0
@@ -179,6 +179,7 @@ xfs-y += $(addprefix scrub/, \ agheader_repair.o \ repair.o \ tempfile.o \ + xfblob.o \ ) endif endif
fs/xfs/scrub/xfblob.c new file +152 -0
152 lines not on this page · raw message
fs/xfs/scrub/xfblob.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2022 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_SCRUB_XFBLOB_H__ +#define __XFS_SCRUB_XFBLOB_H__ + +struct xfblob { +struct xfile *xfile; +loff_t last_offset; +}; + +typedef loff_t xfblob_cookie ; + +int xfblob_create (struct xfs_mount *mp,const char *descr, +struct xfblob **blobp); +void xfblob_destroy (struct xfblob *blob); +int xfblob_load (struct xfblob *blob,xfblob_cookie cookie,void *ptr, +uint32_t size); +int xfblob_store (struct xfblob *blob,xfblob_cookie *cookie,const void *ptr, +uint32_t size); +int xfblob_free (struct xfblob *blob,xfblob_cookie cookie); + +#endif /* __XFS_SCRUB_XFBLOB_H__ */
PATCH 21/23 xfs: repair extended attributes
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the extended attributes look bad, try to sift through the rubble to find whatever keys/values we can, stage a new attribute structure in a temporary file and use the atomic extent swapping mechanism to commit the results in bulk.
--- fs/xfs/scrub/xfarray.c | 24 ++++++++++++++++++++++++ fs/xfs/scrub/xfarray.h | 2 ++ fs/xfs/scrub/xfblob.c | 24 ++++++++++++++++++++++++ fs/xfs/scrub/xfblob.h | 2 ++ 4 files changed, 52 insertions(+)
fs/xfs/scrub/xfarray.c +24 -0
@@ -368,3 +368,27 @@ xfarray_load_next( *idx = cur;return 0 ; } + +/* How many bytes is this array consuming? */ +long long +xfarray_bytes ( +struct xfarray *array) +{ +struct xfile_stat statbuf; +int error; + + error =xfile_stat (array->xfile, &statbuf); +if (error) +return error; + +return statbuf.bytes; +} + +/* Empty the entire array. */ +void +xfarray_truncate ( +struct xfarray *array) +{ +xfile_discard (array->xfile,0 ,MAX_LFS_FILESIZE ); + array->nr =0 ; +}
fs/xfs/scrub/xfarray.h +2 -0
@@ -45,6 +45,8 @@ int xfarray_unset(struct xfarray *array, xfarray_idx_t idx);int xfarray_store (struct xfarray *array,xfarray_idx_t idx,const void *ptr);int xfarray_store_anywhere (struct xfarray *array,const void *ptr);bool xfarray_element_is_null (struct xfarray *array,const void *ptr); +void xfarray_truncate (struct xfarray *array); +long long xfarray_bytes (struct xfarray *array);/* Append an element to the array. */ static inline int xfarray_append (struct xfarray *array,const void *ptr)
fs/xfs/scrub/xfblob.c +24 -0
@@ -150,3 +150,27 @@ xfblob_free(xfile_discard (blob->xfile, cookie,sizeof (key) + key.xb_size);return 0 ; } + +/* How many bytes is this blob storage object consuming? */ +long long +xfblob_bytes ( +struct xfblob *blob) +{ +struct xfile_stat statbuf; +int error; + + error =xfile_stat (blob->xfile, &statbuf); +if (error) +return error; + +return statbuf.bytes; +} + +/* Drop all the blobs. */ +void +xfblob_truncate ( +struct xfblob *blob) +{ +xfile_discard (blob->xfile,0 ,MAX_LFS_FILESIZE ); + blob->last_offset =0 ; +}
fs/xfs/scrub/xfblob.h +2 -0
@@ -21,5 +21,7 @@ int xfblob_load(struct xfblob *blob, xfblob_cookie cookie, void *ptr,int xfblob_store (struct xfblob *blob,xfblob_cookie *cookie,const void *ptr,uint32_t size);int xfblob_free (struct xfblob *blob,xfblob_cookie cookie); +long long xfblob_bytes (struct xfblob *blob); +void xfblob_truncate (struct xfblob *blob);#endif /* __XFS_SCRUB_XFBLOB_H__ */
PATCH 22/23 xfs: online repair of directories
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If a directory looks like it's in bad shape, try to sift through the rubble to find whatever directory entries we can, scan the directory tree for the parent (if needed), stage the new directory contents in a temporary file and use the atomic extent swapping mechanism to commit the results in bulk. As a side effect of this patch, directory inactivation will be able to purge any leftover dir blocks.
--- fs/xfs/scrub/tempfile.c | 13 +++++++++++++ fs/xfs/scrub/tempfile.h | 2 ++ 2 files changed, 15 insertions(+)
fs/xfs/scrub/tempfile.c +13 -0
@@ -228,3 +228,16 @@ xrep_tempfile_rele(xchk_irele (sc, sc->tempip); sc->tempip =NULL ; } + +/* Decide if a given XFS inode is a temporary file for a repair. */ +bool +xrep_is_tempfile ( +const struct xfs_inode *ip) +{ +const struct inode *inode = &ip->i_vnode; + +if (IS_PRIVATE (inode) && !(inode->i_opflags &IOP_XATTR )) +return true; + +return false; +}
fs/xfs/scrub/tempfile.h +2 -0
@@ -16,11 +16,13 @@ void xrep_tempfile_iounlock(struct xfs_scrub *sc);void xrep_tempfile_ilock (struct xfs_scrub *sc);bool xrep_tempfile_ilock_nowait (struct xfs_scrub *sc);void xrep_tempfile_iunlock (struct xfs_scrub *sc); +bool xrep_is_tempfile (const struct xfs_inode *ip);#else static inline void xrep_tempfile_iolock_both (struct xfs_scrub *sc) {xchk_ilock (sc,XFS_IOLOCK_EXCL ); } +# define xrep_is_tempfile (ip) (false)# define xrep_tempfile_rele (sc)#endif /* CONFIG_XFS_ONLINE_REPAIR */
PATCH 23/23 xfs: create an xattr iteration function for scrub
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a streamlined function to walk a file's xattrs, without all the cursor management stuff in the regular listxattr.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/listxattr.c | 314 ++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/listxattr.h | 17 ++ 3 files changed, 332 insertions(+) create mode 100644 fs/xfs/scrub/listxattr.c create mode 100644 fs/xfs/scrub/listxattr.h
fs/xfs/Makefile +1 -0
@@ -160,6 +160,7 @@ xfs-y += $(addprefix scrub/, \ ialloc.o \ inode.o \ iscan.o \ + listxattr.o \ parent.o \ readdir.o \ refcount.o \
fs/xfs/scrub/listxattr.c new file +314 -0
314 lines not on this page · raw message
fs/xfs/scrub/listxattr.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2022 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_SCRUB_LISTXATTR_H__ +#define __XFS_SCRUB_LISTXATTR_H__ + +typedef int (*xchk_xattr_fn )(struct xfs_scrub *sc,struct xfs_inode *ip, +unsigned int attr_flags,const unsigned char *name, +unsigned int namelen,const void *value,unsigned int valuelen, +void *priv); + +int xchk_xattr_walk (struct xfs_scrub *sc,struct xfs_inode *ip, +xchk_xattr_fn attr_fn,void *priv); + +#endif /* __XFS_SCRUB_LISTXATTR_H__ */
cover xfs: online repair of directories
In reply to Darrick J. WongHi all,
With this patchset, we implement online reconstruction of directories by scanning the entire filesystem looking for parent pointer data. This mostly works, except for the part where we need to resync the diroffset field of the parent pointers to match the new directory structure.
Fixing that is left as an open research question, with a few possible solutions:
1. As part of committing the new directory, queue a bunch of parent pointer updates to make those changes.
2. Leave them inconsistent and let the parent pointer repair fix it.
3. Change the ondisk format of parent pointers (and xattrs) so that we can encode the full dirent name in the xattr name.
4. Change the ondisk format of parent pointers to encode a sha256 hash of the dirent name in the xattr name. This will work as long as nobody breaks sha256.
Thoughts? Note that the atomic swapext and block reaping code is NOT ported for this PoC, so we do not commit any repairs.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-repair
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-dir-repair
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_da_format.h | 11 fs/xfs/libxfs/xfs_dir2.c | 2 fs/xfs/libxfs/xfs_dir2.h | 2 fs/xfs/libxfs/xfs_parent.c | 47 +- fs/xfs/libxfs/xfs_parent.h | 24 - fs/xfs/scrub/common.c | 15 + fs/xfs/scrub/common.h | 28 + fs/xfs/scrub/dir.c | 11 fs/xfs/scrub/dir_repair.c | 1129 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/readdir.c | 12 fs/xfs/scrub/readdir.h | 3 fs/xfs/scrub/repair.h | 16 + fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/tempfile.c | 42 ++ fs/xfs/scrub/tempfile.h | 2 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 69 +++ fs/xfs/xfs_inode.c | 56 +- fs/xfs/xfs_inode.h | 5 fs/xfs/xfs_symlink.c | 4 21 files changed, 1428 insertions(+), 54 deletions(-) create mode 100644 fs/xfs/scrub/dir_repair.c
PATCH 1/7 xfs: pass directory offsets as part of the dirent hook data
From: Darrick J. Wong <[email protected]>
When we're calling the dirent hooks about a directory entry update, be sure to pass the diroffset associated with the change. We're going to need this in the next patch.
--- fs/xfs/xfs_inode.c | 40 +++++++++++++++++++++++++--------------- fs/xfs/xfs_inode.h | 5 +++-- fs/xfs/xfs_symlink.c | 2 +- 3 files changed, 29 insertions(+), 18 deletions(-)
fs/xfs/xfs_inode.c +25 -15
@@ -1042,7 +1042,8 @@ xfs_dirent_child_delta( struct xfs_inode *dp, struct xfs_inode *ip, int delta, - struct xfs_name *name) + struct xfs_name *name, + unsigned int diroffset) { if (xfs_hooks_switched_on(&xfs_dirents_hooks_switch)) { struct xfs_dirent_update_params p = {
@@ -1050,6 +1051,7 @@ xfs_dirent_child_delta( .ip = ip, .delta = delta, .name = name, + .diroffset = diroffset, };struct xfs_mount *mp = ip->i_mount;
@@ -1210,7 +1212,7 @@ xfs_create(* Create ip with a reference from dp, and add '.' and '..' references * if it's a directory. */ -xfs_dirent_child_delta (dp, ip,1 , name); +xfs_dirent_child_delta (dp, ip,1 , name, diroffset);if (is_dir) {xfs_dirent_self_delta (ip,1 );xfs_dirent_backref_delta (dp, ip,1 );
@@ -1481,7 +1483,7 @@ xfs_link( goto error_return; } -xfs_dirent_child_delta (tdp, sip,1 , target_name); +xfs_dirent_child_delta (tdp, sip,1 , target_name, diroffset);/* * If this is a synchronous mount, make sure that the
@@ -2757,7 +2759,7 @@ xfs_remove(* Drop the link from dp to ip, and if ip was a directory, remove the * '.' and '..' references since we freed the directory. */ -xfs_dirent_child_delta (dp, ip,-1 , name); +xfs_dirent_child_delta (dp, ip,-1 , name, dir_offset);if (S_ISDIR (VFS_I (ip)->i_mode)) {xfs_dirent_backref_delta (dp, ip,-1 );xfs_dirent_self_delta (ip,-1 );
@@ -2873,18 +2875,22 @@ static inline voidxfs_exchange_call_nlink_hooks ( struct xfs_inode *src_dp, struct xfs_name *src_name, + xfs_dir2_dataptr_t src_diroffset, struct xfs_inode *src_ip, struct xfs_inode *target_dp, struct xfs_name *target_name, + xfs_dir2_dataptr_t target_diroffset, struct xfs_inode *target_ip) {/* Exchange files in the source directory. */ - xfs_dirent_child_delta(src_dp, src_ip, -1, src_name); -xfs_dirent_child_delta (src_dp, target_ip,1 , src_name); + xfs_dirent_child_delta(src_dp, src_ip, -1, src_name, src_diroffset); +xfs_dirent_child_delta (src_dp, target_ip,1 , src_name, src_diroffset);/* Exchange files in the target directory. */ -xfs_dirent_child_delta (target_dp, target_ip,-1 , target_name); -xfs_dirent_child_delta (target_dp, src_ip,1 , target_name); +xfs_dirent_child_delta (target_dp, target_ip,-1 , target_name, + target_diroffset); +xfs_dirent_child_delta (target_dp, src_ip,1 , target_name, + target_diroffset);/* If the source file is a dir, update its dotdot entry. */ if (S_ISDIR (VFS_I (src_ip)->i_mode)) {
@@ -2903,9 +2909,11 @@ static inline voidxfs_rename_call_nlink_hooks ( struct xfs_inode *src_dp, struct xfs_name *src_name, + xfs_dir2_dataptr_t src_diroffset, struct xfs_inode *src_ip, struct xfs_inode *target_dp, struct xfs_name *target_name, + xfs_dir2_dataptr_t target_diroffset, struct xfs_inode *target_ip, struct xfs_inode *wip) {
@@ -2914,16 +2922,16 @@ xfs_rename_call_nlink_hooks(* move the source file to the target directory. */ if (target_ip) -xfs_dirent_child_delta (target_dp, target_ip,-1 , target_name); -xfs_dirent_child_delta (target_dp, src_ip,1 , target_name); +xfs_dirent_child_delta (target_dp, target_ip,-1 , target_name, target_diroffset); +xfs_dirent_child_delta (target_dp, src_ip,1 , target_name, target_diroffset);/* * Remove the source file from the source directory, and possibly move * the whiteout file into its place. */ -xfs_dirent_child_delta (src_dp, src_ip,-1 , src_name); +xfs_dirent_child_delta (src_dp, src_ip,-1 , src_name, src_diroffset);if (wip) -xfs_dirent_child_delta (src_dp, wip,1 , src_name); +xfs_dirent_child_delta (src_dp, wip,1 , src_name, src_diroffset);/* If the source file is a dir, update its dotdot entry. */ if (S_ISDIR (VFS_I (src_ip)->i_mode)) {
@@ -3080,7 +3088,8 @@ xfs_cross_rename(xfs_trans_log_inode (tp, dp1,XFS_ILOG_CORE );if (xfs_hooks_switched_on (&xfs_dirents_hooks_switch)) -xfs_exchange_call_nlink_hooks (dp1, name1, ip1, dp2, name2, ip2); +xfs_exchange_call_nlink_hooks (dp1, name1, old_diroffset, ip1, + dp2, name2, new_diroffset, ip2);return xfs_finish_rename (tp);
@@ -3560,8 +3569,9 @@ xfs_rename(xfs_trans_log_inode (tp, target_dp,XFS_ILOG_CORE );if (xfs_hooks_switched_on (&xfs_dirents_hooks_switch)) -xfs_rename_call_nlink_hooks (src_dp, src_name, src_ip, - target_dp, target_name, target_ip, wip); +xfs_rename_call_nlink_hooks (src_dp, src_name, old_diroffset, + src_ip, target_dp, target_name, new_diroffset, + target_ip, wip); error =xfs_finish_rename (tp);
fs/xfs/xfs_inode.h +3 -2
@@ -592,12 +592,13 @@ struct xfs_dirent_update_params {const struct xfs_inode *dp;const struct xfs_inode *ip;const struct xfs_name *name; +unsigned int diroffset;int delta; }; #ifdef CONFIG_XFS_LIVE_HOOKS void xfs_dirent_child_delta(struct xfs_inode *dp, struct xfs_inode *ip, - int delta, struct xfs_name *name); + int delta, struct xfs_name *name, unsigned int diroffset); struct xfs_dirent_hook { struct xfs_hook delta_hook;
@@ -610,7 +611,7 @@ int xfs_dirent_hook_add(struct xfs_mount *mp, struct xfs_dirent_hook *hook);void xfs_dirent_hook_del (struct xfs_mount *mp,struct xfs_dirent_hook *hook);#else -# define xfs_dirent_child_delta (dp, ip, delta, name) ((void)0) +# define xfs_dirent_child_delta (dp, ip, delta, name, doff) ((void)0)#endif /* CONFIG_XFS_LIVE_HOOKS */ #endif /* __XFS_INODE_H__ */
fs/xfs/xfs_symlink.c +1 -1
@@ -354,7 +354,7 @@ xfs_symlink( goto out_trans_cancel; } -xfs_dirent_child_delta (dp, ip,1 , link_name); +xfs_dirent_child_delta (dp, ip,1 , link_name, diroffset);/* * If this is a synchronous mount, make sure that the
PATCH 2/7 xfs: pass diroffset back from xchk_dir_lookup
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Pass directory offsets back from xchk_dir_lookup so that we can compare things in scrub.
--- fs/xfs/scrub/dir.c | 2 +- fs/xfs/scrub/readdir.c | 12 ++++++++++-- fs/xfs/scrub/readdir.h | 3 ++- 3 files changed, 13 insertions(+), 4 deletions(-)
fs/xfs/scrub/dir.c +1 -1
@@ -105,7 +105,7 @@ xchk_dir_actor( }/* Verify that we can look up this name by hash. */ - error =xchk_dir_lookup (sc, dp, name, &lookup_ino); + error =xchk_dir_lookup (sc, dp, name, &lookup_ino,NULL );/* ENOENT means the hash lookup failed and the dir is corrupt */ if (error == -ENOENT ) error = -EFSCORRUPTED ;
fs/xfs/scrub/readdir.c +10 -2
@@ -314,7 +314,8 @@ xchk_dir_lookup( struct xfs_scrub *sc, struct xfs_inode *dp, const struct xfs_name *name, - xfs_ino_t *ino) + xfs_ino_t *ino, + xfs_dir2_dataptr_t *diroffsetp) { struct xfs_da_args args = { .dp = dp,
@@ -326,10 +327,14 @@ xchk_dir_lookup( .hashval = xfs_dir2_hashname(dp->i_mount, name), .whichfork = XFS_DATA_FORK, .op_flags = XFS_DA_OP_OKNOENT, + .offset = XFS_DIR2_NULL_DATAPTR, };bool isblock, isleaf;int error; +if (diroffsetp) + *diroffsetp =XFS_DIR2_NULL_DATAPTR ; +if (xfs_is_shutdown (dp->i_mount))return -EIO ;
@@ -369,7 +374,10 @@ xchk_dir_lookup( out_check_rval:if (error == -EEXIST ) error =0 ; -if (!error) +if (!error) { *ino = args.inumber; +if (diroffsetp) + *diroffsetp = args.offset; + }return error; }
fs/xfs/scrub/readdir.h +2 -1
@@ -14,6 +14,7 @@ int xchk_dir_walk(struct xfs_scrub *sc, struct xfs_inode *dp, xchk_dirent_fn dirent_fn, void *priv);int xchk_dir_lookup (struct xfs_scrub *sc,struct xfs_inode *dp, -const struct xfs_name *name,xfs_ino_t *ino); +const struct xfs_name *name,xfs_ino_t *ino, +xfs_dir2_dataptr_t *diroffsetp);#endif /* __XFS_SCRUB_READDIR_H__ */
PATCH 3/7 xfs: shorten parent pointer function names
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Shorten the function names and add brief comments to each, outlining what they're supposed to be doing.
--- fs/xfs/libxfs/xfs_parent.c | 18 ++++++++++++------ fs/xfs/libxfs/xfs_parent.h | 24 ++++++++++++------------ fs/xfs/xfs_inode.c | 16 ++++++++-------- fs/xfs/xfs_symlink.c | 2 +- 4 files changed, 33 insertions(+), 27 deletions(-)
fs/xfs/libxfs/xfs_parent.c +12 -6
@@ -135,6 +135,10 @@ xfs_parent_irec_from_disk(memset (&irec->p_name[valuelen],0 ,sizeof (irec->p_name) - valuelen); } +/* +* Allocate memory to control a logged parent pointer update as part of a +* dirent operation. +*/ int __xfs_parent_init( struct xfs_mount *mp,
@@ -170,12 +174,13 @@ __xfs_parent_init(return 0 ; } +/* Add a parent pointer to reflect a dirent addition. */ int -xfs_parent_defer_add ( +xfs_parent_add (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp, -struct xfs_name *parent_name, +const struct xfs_name *parent_name,xfs_dir2_dataptr_t diroffset,struct xfs_inode *child) {
@@ -194,8 +199,9 @@ xfs_parent_defer_add(return xfs_attr_defer_add (args); } +/* Remove a parent pointer to reflect a dirent removal. */ int -xfs_parent_defer_remove( +xfs_parent_remove( struct xfs_trans *tp, struct xfs_inode *dp, struct xfs_parent_defer *parent,
@@ -211,14 +217,14 @@ xfs_parent_defer_remove(return xfs_attr_defer_remove (args); } - +/* Replace one parent pointer with another to reflect a rename. */ int -xfs_parent_defer_replace ( +xfs_parent_replace (struct xfs_trans *tp,struct xfs_parent_defer *new_parent,struct xfs_inode *old_dp,xfs_dir2_dataptr_t old_diroffset, -struct xfs_name *parent_name, +const struct xfs_name *parent_name,struct xfs_inode *new_dp,xfs_dir2_dataptr_t new_diroffset,struct xfs_inode *child)
fs/xfs/libxfs/xfs_parent.h +12 -12
@@ -49,8 +49,9 @@ struct xfs_parent_defer {* Parent pointer attribute prototypes */ void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, -struct xfs_inode *ip, -uint32_t p_diroffset); +struct xfs_inode *ip,uint32_t p_diroffset); +void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec, +struct xfs_parent_name_rec *rec);int __xfs_parent_init (struct xfs_mount *mp,bool grab_log,struct xfs_parent_defer **parentp);
@@ -78,18 +79,17 @@ xfs_parent_start_locked(return 0 ; } -int xfs_parent_defer_add (struct xfs_trans *tp,struct xfs_parent_defer *parent, -struct xfs_inode *dp,struct xfs_name *parent_name, -xfs_dir2_dataptr_t diroffset,struct xfs_inode *child); -int xfs_parent_defer_replace (struct xfs_trans *tp, +int xfs_parent_add (struct xfs_trans *tp,struct xfs_parent_defer *parent, +struct xfs_inode *dp,const struct xfs_name *parent_name, +xfs_dir2_dataptr_t diroffset,struct xfs_inode *child); +int xfs_parent_replace (struct xfs_trans *tp,struct xfs_parent_defer *new_parent,struct xfs_inode *old_dp, -xfs_dir2_dataptr_t old_diroffset,struct xfs_name *parent_name, -struct xfs_inode *new_ip,xfs_dir2_dataptr_t new_diroffset, +xfs_dir2_dataptr_t old_diroffset, +const struct xfs_name *parent_name,struct xfs_inode *new_ip, +xfs_dir2_dataptr_t new_diroffset,struct xfs_inode *child); +int xfs_parent_remove (struct xfs_trans *tp,struct xfs_inode *dp, +struct xfs_parent_defer *parent,xfs_dir2_dataptr_t diroffset,struct xfs_inode *child); -int xfs_parent_defer_remove (struct xfs_trans *tp,struct xfs_inode *dp, -struct xfs_parent_defer *parent, -xfs_dir2_dataptr_t diroffset, -struct xfs_inode *child);void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent);
fs/xfs/xfs_inode.c +8 -8
@@ -1202,7 +1202,7 @@ xfs_create(* the parent information now. */ if (parent) { - error =xfs_parent_defer_add (tp, parent, dp, name, diroffset, + error =xfs_parent_add (tp, parent, dp, name, diroffset, ip);if (error) goto out_trans_cancel;
@@ -1477,7 +1477,7 @@ xfs_link(* the parent to the inode. */ if (parent) { - error =xfs_parent_defer_add (tp, parent, tdp, target_name, + error =xfs_parent_add (tp, parent, tdp, target_name, diroffset, sip);if (error) goto error_return;
@@ -2750,7 +2750,7 @@ xfs_remove( }if (parent) { - error =xfs_parent_defer_remove (tp, dp, parent, dir_offset, ip); + error =xfs_parent_remove (tp, dp, parent, dir_offset, ip);if (error) goto out_trans_cancel; }
@@ -3061,12 +3061,12 @@ xfs_cross_rename( }if (xfs_has_parent (mp)) { - error =xfs_parent_defer_replace (tp, ip1_pptr, dp1, + error =xfs_parent_replace (tp, ip1_pptr, dp1, old_diroffset, name2, dp2, new_diroffset, ip1);if (error) goto out_trans_abort; - error =xfs_parent_defer_replace (tp, ip2_pptr, dp2, + error =xfs_parent_replace (tp, ip2_pptr, dp2, new_diroffset, name1, dp1, old_diroffset, ip2);if (error) goto out_trans_abort;
@@ -3540,7 +3540,7 @@ xfs_rename( goto out_trans_cancel;if (wip_pptr) { - error =xfs_parent_defer_add (tp, wip_pptr, + error =xfs_parent_add (tp, wip_pptr, src_dp, src_name, old_diroffset, wip);if (error)
@@ -3548,7 +3548,7 @@ xfs_rename( }if (src_ip_pptr) { - error =xfs_parent_defer_replace (tp, src_ip_pptr, src_dp, + error =xfs_parent_replace (tp, src_ip_pptr, src_dp, old_diroffset, target_name, target_dp, new_diroffset, src_ip);if (error)
@@ -3556,7 +3556,7 @@ xfs_rename( }if (tgt_ip_pptr) { - error =xfs_parent_defer_remove (tp, target_dp, + error =xfs_parent_remove (tp, target_dp, tgt_ip_pptr, new_diroffset, target_ip);if (error)
fs/xfs/xfs_symlink.c +1 -1
@@ -348,7 +348,7 @@ xfs_symlink(xfs_trans_log_inode (tp, dp,XFS_ILOG_CORE );if (parent) { - error =xfs_parent_defer_add (tp, parent, dp, link_name, + error =xfs_parent_add (tp, parent, dp, link_name, diroffset, ip);if (error) goto out_trans_cancel;
PATCH 4/7 xfs: rearrange bits of the parent pointer apis for fsck
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Rearrange parts of this thing in preparation for fsck code.
--- fs/xfs/libxfs/xfs_da_format.h | 11 +++++++++++ fs/xfs/libxfs/xfs_parent.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_parent.h | 6 ++---- 3 files changed, 41 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +11 -0
@@ -159,6 +159,17 @@ struct xfs_da3_intnode {#define XFS_DIR3_FT_MAX 9 +#define XFS_DIR3_FTYPE_STR \ + { XFS_DIR3_FT_UNKNOWN, "unknown" }, \ + { XFS_DIR3_FT_REG_FILE, "file" }, \ + { XFS_DIR3_FT_DIR, "directory" }, \ + { XFS_DIR3_FT_CHRDEV, "char" }, \ + { XFS_DIR3_FT_BLKDEV, "block" }, \ + { XFS_DIR3_FT_FIFO, "fifo" }, \ + { XFS_DIR3_FT_SOCK, "sock" }, \ + { XFS_DIR3_FT_SYMLINK, "symlink" }, \ + { XFS_DIR3_FT_WHT, "whiteout" } +/* * Byte offset in data block and shortform entry. */
fs/xfs/libxfs/xfs_parent.c +28 -1
@@ -91,7 +91,7 @@ xfs_parent_valuecheck( }/* Initializes a xfs_parent_name_rec to be stored as an attribute name */ -void +static inline void xfs_init_parent_name_rec( struct xfs_parent_name_rec *rec, struct xfs_inode *ip,
@@ -135,6 +135,33 @@ xfs_parent_irec_from_disk(memset (&irec->p_name[valuelen],0 ,sizeof (irec->p_name) - valuelen); } +/* +* Convert an incore parent_name record to its ondisk format. If @value or +* @valuelen are NULL, they will not be written to. +*/ +void +xfs_parent_irec_to_disk ( +struct xfs_parent_name_rec *rec, +void *value, +int *valuelen, +const struct xfs_parent_name_irec *irec) +{ + rec->p_ino =cpu_to_be64 (irec->p_ino); + rec->p_gen =cpu_to_be32 (irec->p_gen); + rec->p_diroffset =cpu_to_be32 (irec->p_diroffset); + +if (valuelen) { +ASSERT (*valuelen >0 ); +ASSERT (*valuelen >= irec->p_namelen); +ASSERT (*valuelen <MAXNAMELEN ); + + *valuelen = irec->p_namelen; + } + +if (value) +memcpy (value, irec->p_name, irec->p_namelen); +} +/* * Allocate memory to control a logged parent pointer update as part of a * dirent operation.
fs/xfs/libxfs/xfs_parent.h +2 -4
@@ -33,6 +33,8 @@ struct xfs_parent_name_irec {void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec,const struct xfs_parent_name_rec *rec,const void *value,int valuelen); +void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec,void *value, +int *valuelen,const struct xfs_parent_name_irec *irec);/* * Dynamically allocd structure used to wrap the needed data to pass around
@@ -48,10 +50,6 @@ struct xfs_parent_defer {/* * Parent pointer attribute prototypes */ -void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, -struct xfs_inode *ip,uint32_t p_diroffset); -void xfs_init_parent_name_irec (struct xfs_parent_name_irec *irec, -struct xfs_parent_name_rec *rec);int __xfs_parent_init (struct xfs_mount *mp,bool grab_log,struct xfs_parent_defer **parentp);
PATCH 5/7 xfs: reconstruct directories from parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use the filesystem scanning infrastructure to walk the filesystem looking for parent pointers and child dirents that reference the directory that we're rebuilding.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/common.c | 15 + fs/xfs/scrub/common.h | 28 + fs/xfs/scrub/dir.c | 9 fs/xfs/scrub/dir_repair.c | 940 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 16 + fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/tempfile.c | 42 ++ fs/xfs/scrub/tempfile.h | 2 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 67 +++ 11 files changed, 1122 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/scrub/dir_repair.c
fs/xfs/Makefile +1 -0
@@ -178,6 +178,7 @@ xfs-$(CONFIG_XFS_QUOTA) += scrub/quota.o ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ + dir_repair.o \ repair.o \ tempfile.o \ xfblob.o \
fs/xfs/scrub/common.c +15 -0
@@ -551,6 +551,21 @@ xchk_ag_init(/* Per-scrubber setup functions */ +void +xchk_trans_cancel ( +struct xfs_scrub *sc) +{ +xfs_trans_cancel (sc->tp); + sc->tp =NULL ; +} + +int +xchk_trans_alloc_empty ( +struct xfs_scrub *sc) +{ +return xfs_trans_alloc_empty (sc->mp, &sc->tp); +} +/* * Grab an empty transaction so that we can re-grab locked buffers if * one of our btrees turns out to be cyclic.
fs/xfs/scrub/common.h +28 -0
@@ -31,6 +31,9 @@ xchk_should_terminate(return false; } +void xchk_trans_cancel (struct xfs_scrub *sc); +int xchk_trans_alloc_empty (struct xfs_scrub *sc); +int xchk_trans_alloc (struct xfs_scrub *sc,uint resblks);bool xchk_process_error (struct xfs_scrub *sc,xfs_agnumber_t agno,xfs_agblock_t bno,int *error);
@@ -159,4 +162,29 @@ void xchk_start_reaping(struct xfs_scrub *sc);void xchk_fshooks_enable (struct xfs_scrub *sc,unsigned int scrub_fshooks); +#ifdef CONFIG_XFS_ONLINE_REPAIR +/* Decide if a repair is required. */ +static inline bool xchk_needs_repair (const struct xfs_scrub_metadata *sm) +{ +return sm->sm_flags & (XFS_SCRUB_OFLAG_CORRUPT | +XFS_SCRUB_OFLAG_XCORRUPT | +XFS_SCRUB_OFLAG_PREEN ); +} + +/* +* "Should we prepare for a repair?" +* +* Return true if the caller permits us to repair metadata and we're not +* setting up for a post-repair evaluation. +*/ +static inline bool xchk_could_repair (const struct xfs_scrub *sc) +{ +return (sc->sm->sm_flags &XFS_SCRUB_IFLAG_REPAIR ) && + !(sc->flags &XREP_ALREADY_FIXED ); +} +#else +# define xchk_needs_repair (sc) (false) +# define xchk_could_repair (sc) (false) +#endif /* CONFIG_XFS_ONLINE_REPAIR */ +#endif /* __XFS_SCRUB_COMMON_H__ */
fs/xfs/scrub/dir.c +9 -0
@@ -19,12 +19,21 @@#include "scrub/common.h" #include "scrub/dabtree.h" #include "scrub/readdir.h" +#include "scrub/repair.h" /* Set us up to scrub directories. */ int xchk_setup_directory (struct xfs_scrub *sc) { +int error; + +if (xchk_could_repair (sc)) { + error =xrep_setup_directory (sc); +if (error) +return error; + } +return xchk_setup_inode_contents (sc,0 ); }
fs/xfs/scrub/dir_repair.c new file +940 -0
940 lines not on this page · raw message
fs/xfs/scrub/repair.h +16 -0
@@ -30,6 +30,16 @@ int xrep_init_btblock(struct xfs_scrub *sc, xfs_fsblock_t fsb, struct xfs_buf **bpp, xfs_btnum_t btnum, const struct xfs_buf_ops *ops); +static inline int +xrep_trans_commit ( +struct xfs_scrub *sc) +{ +int error =xfs_trans_commit (sc->tp); + + sc->tp =NULL ; +return error; +} +struct xbitmap ;int xrep_fix_freelist (struct xfs_scrub *sc,bool can_shrink);
@@ -57,6 +67,8 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,void xrep_force_quotacheck (struct xfs_scrub *sc,xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc); +int xrep_setup_directory (struct xfs_scrub *sc); +/* Metadata repairers */ int xrep_probe (struct xfs_scrub *sc);
@@ -64,6 +76,7 @@ int xrep_superblock(struct xfs_scrub *sc);int xrep_agf (struct xfs_scrub *sc);int xrep_agfl (struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc); +int xrep_directory (struct xfs_scrub *sc);#else
@@ -83,11 +96,14 @@ xrep_calc_ag_resblks( return 0; } +#define xrep_setup_directory (sc) (0) +#define xrep_probe xrep_notsupported#define xrep_superblock xrep_notsupported#define xrep_agf xrep_notsupported#define xrep_agfl xrep_notsupported#define xrep_agi xrep_notsupported +#define xrep_directory xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -299,7 +299,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_directory, .scrub = xchk_directory, - .repair = xrep_notsupported, + .repair = xrep_directory, }, [XFS_SCRUB_TYPE_XATTR] = { /* extended attributes */ .type = ST_INODE,
fs/xfs/scrub/tempfile.c +42 -0
66 lines not on this page · raw message
fs/xfs/scrub/tempfile.h +2 -0
@@ -11,10 +11,12 @@ int xrep_tempfile_create(struct xfs_scrub *sc, uint16_t mode);void xrep_tempfile_rele (struct xfs_scrub *sc);bool xrep_tempfile_iolock_nowait (struct xfs_scrub *sc); +int xrep_tempfile_iolock_polled (struct xfs_scrub *sc);void xrep_tempfile_iounlock (struct xfs_scrub *sc);void xrep_tempfile_ilock (struct xfs_scrub *sc);bool xrep_tempfile_ilock_nowait (struct xfs_scrub *sc); +int xrep_tempfile_ilock_polled (struct xfs_scrub *sc);void xrep_tempfile_iunlock (struct xfs_scrub *sc);bool xrep_is_tempfile (const struct xfs_inode *ip);#else
fs/xfs/scrub/trace.c +1 -0
@@ -17,6 +17,7 @@#include "scrub/xfile.h" #include "scrub/xfarray.h" #include "scrub/iscan.h" +#include "xfs_da_format.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +67 -0
73 lines not on this page · raw message
PATCH 6/7 xfs: add hooks to do directory updates
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
While we're scanning the filesystem, we still need to keep the tempdir up to date with whatever changes get made to the you know what.
--- fs/xfs/libxfs/xfs_dir2.c | 2 - fs/xfs/libxfs/xfs_dir2.h | 2 - fs/xfs/scrub/dir_repair.c | 94 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 96 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +1 -1
@@ -440,7 +440,7 @@ intxfs_dir_removename ( struct xfs_trans *tp, struct xfs_inode *dp, - struct xfs_name *name, + const struct xfs_name *name, xfs_ino_t ino, xfs_extlen_t total, /* bmap's total block count */ xfs_dir2_dataptr_t *offset) /* OUT: offset in directory */
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -46,7 +46,7 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t *inum, struct xfs_name *ci_name);extern int xfs_dir_removename (struct xfs_trans *tp,struct xfs_inode *dp, -struct xfs_name *name,xfs_ino_t ino, +const struct xfs_name *name,xfs_ino_t ino,xfs_extlen_t tot,xfs_dir2_dataptr_t *offset); extern int xfs_dir_replace(struct xfs_trans *tp, struct xfs_inode *dp,
fs/xfs/scrub/dir_repair.c +94 -0
132 lines not on this page · raw message
PATCH 7/7 xfs: compare generated and existing dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check our work to make sure we found all the dirents that the original directory had.
--- fs/xfs/scrub/dir_repair.c | 101 ++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 2 + 2 files changed, 100 insertions(+), 3 deletions(-)
fs/xfs/scrub/dir_repair.c +98 -3
@@ -757,7 +757,10 @@ xrep_dir_scan_dirtree(return 0 ; } -/* Dump a dirent from the temporary dir. */ +/* +* Dump a dirent from the temporary dir and check it against the dir we're +* rebuilding. We are not committing any of this. +*/ STATIC int xrep_dir_dump_tempdir( struct xfs_scrub *sc,
@@ -768,7 +771,9 @@ xrep_dir_dump_tempdir( void *priv) {struct xrep_dir *rd = priv; +xfs_ino_t child_ino;bool child = true; +xfs_dir2_dataptr_t child_diroffset =XFS_DIR2_NULL_DATAPTR ;int error;/*
@@ -809,7 +814,88 @@ xrep_dir_dump_tempdir(mutex_lock (&rd->lock); error =xrep_dir_remove_dirent (rd, name, ino, dapos);mutex_unlock (&rd->lock); -return error; +if (error) +return error; + +/* Check that the dir being repaired has the same entry. */ + error =xchk_dir_lookup (sc, sc->ip, name, &child_ino, + &child_diroffset); +if (error == -ENOENT ) { +trace_xrep_dir_checkname (sc->ip, name,NULLFSINO , +XFS_DIR2_NULL_DATAPTR ); +ASSERT (error != -ENOENT ); +return -EFSCORRUPTED ; + } +if (error) +return error; + +if (ino != child_ino) { +trace_xrep_dir_checkname (sc->ip, name, child_ino, + child_diroffset); +ASSERT (ino == child_ino); +return -EFSCORRUPTED ; + } + +if (dapos != child_diroffset) { +trace_xrep_dir_badposname (sc->ip, name, child_ino, + child_diroffset); +/* We have no way to update this, so we just leave it. */ + } + +return 0 ; +} + +/* +* Dump a dirent from the dir we're rebuilding and check it against the +* temporary dir. This assumes that the directory wasn't really corrupt to +* begin with. +*/ +STATIC int +xrep_dir_dump_baddir ( +struct xfs_scrub *sc, +struct xfs_inode *dp, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +xfs_ino_t ino, +void *priv) +{ +xfs_ino_t child_ino; +xfs_dir2_dataptr_t child_diroffset =XFS_DIR2_NULL_DATAPTR ; +int error; + +/* Ignore the directory's dot and dotdot entries. */ +if (xrep_dir_samename (name, &xfs_name_dotdot) || +xrep_dir_samename (name, &xfs_name_dot)) +return 0 ; + +trace_xrep_dir_dumpname (sc->ip, name, ino, dapos); + +/* Check that the tempdir has the same entry. */ + error =xchk_dir_lookup (sc, sc->tempip, name, &child_ino, + &child_diroffset); +if (error == -ENOENT ) { +trace_xrep_dir_checkname (sc->tempip, name,NULLFSINO , +XFS_DIR2_NULL_DATAPTR ); +ASSERT (error != -ENOENT ); +return -EFSCORRUPTED ; + } +if (error) +return error; + +if (ino != child_ino) { +trace_xrep_dir_checkname (sc->tempip, name, child_ino, + child_diroffset); +ASSERT (ino == child_ino); +return -EFSCORRUPTED ; + } + +if (dapos != child_diroffset) { +trace_xrep_dir_badposname (sc->ip, name, child_ino, + child_diroffset); +/* We have no way to update this, so we just leave it. */ + } + +return 0 ; }/*
@@ -876,12 +962,21 @@ xrep_dir_rebuild_tree(trace_xrep_dir_rebuild_tree (sc->ip, rd->parent_ino); -xrep_tempfile_ilock (sc); +xchk_ilock (sc,XFS_ILOCK_EXCL ); + error =xrep_tempfile_ilock_polled (sc); +if (error) +return error; + error =xchk_dir_walk (sc, sc->tempip, xrep_dir_dump_tempdir, rd);if (error)return error; + error =xchk_dir_walk (sc, sc->ip, xrep_dir_dump_baddir, rd); +if (error) +return error; +xrep_tempfile_iunlock (sc); +xchk_iunlock (sc,XFS_ILOCK_EXCL );xchk_trans_cancel (sc);return xrep_dir_replay_updates (rd);
fs/xfs/scrub/trace.h +2 -0
@@ -1224,6 +1224,8 @@ DEFINE_XREP_DIRENT_CLASS(xrep_dir_createname);DEFINE_XREP_DIRENT_CLASS (xrep_dir_removename);DEFINE_XREP_DIRENT_CLASS (xrep_dir_replacename);DEFINE_XREP_DIRENT_CLASS (xrep_dir_dumpname); +DEFINE_XREP_DIRENT_CLASS (xrep_dir_checkname); +DEFINE_XREP_DIRENT_CLASS (xrep_dir_badposname); DECLARE_EVENT_CLASS(xrep_dir_class, TP_PROTO(struct xfs_inode *dp, xfs_ino_t parent_ino),
cover xfs: online checking of parent pointers
In reply to Darrick J. WongHi all,
Update the existing online parent pointer checker to confirm the directory entries that should also exist.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-check
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-parent-check
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_parent.c | 38 +++ fs/xfs/libxfs/xfs_parent.h | 10 + fs/xfs/scrub/parent.c | 529 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 66 +++++ 5 files changed, 644 insertions(+), 1 deletion(-)
PATCH 1/2 xfs: scrub parent pointers
From: Darrick J. Wong <[email protected]>
Actually check parent pointers now.
--- fs/xfs/scrub/parent.c | 291 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 33 ++++++ 2 files changed, 324 insertions(+)
fs/xfs/scrub/parent.c +291 -0
@@ -14,9 +14,13 @@#include "xfs_icache.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/readdir.h" +#include "scrub/listxattr.h" +#include "scrub/trace.h" /* Set us up to scrub parents. */ int
@@ -231,6 +235,8 @@ xchk_parent_validate(return error; } +STATIC int xchk_parent_pptr(struct xfs_scrub *sc); +/* Scrub a parent pointer. */ int xchk_parent(
@@ -240,6 +246,9 @@ xchk_parent(xfs_ino_t parent_ino;int error; +if (xfs_has_parent (mp)) +return xchk_parent_pptr (sc); +/* * If we're a directory, check that the '..' link points up to * a directory that has one entry pointing to us.
@@ -282,3 +291,285 @@ xchk_parent(return xchk_parent_validate (sc, parent_ino); } + +/* +* Checking of Parent Pointers +* =========================== +* +* On filesystems with directory parent pointers, we check the referential +* integrity by visiting each parent pointer of a child file and checking that +* the directory referenced by the pointer actually has a dirent pointing back +* to the child file. +*/ + +struct xchk_pptrs { +struct xfs_scrub *sc; + +/* Scratch buffer for scanning pptr xattrs */ +struct xfs_parent_name_irec pptr; + +/* Parent of this directory. */ +xfs_ino_t parent_ino; +}; + +/* Look up the dotdot entry so that we can check it as we walk the pptrs. */ +STATIC int +xchk_parent_dotdot ( +struct xchk_pptrs *pp) +{ +struct xfs_scrub *sc = pp->sc; +int error; + +if (!S_ISDIR (VFS_I (sc->ip)->i_mode)) { + pp->parent_ino =NULLFSINO ; +return 0 ; + } + +/* Look up '..' */ + error =xchk_dir_lookup (sc, sc->ip, &xfs_name_dotdot, &pp->parent_ino, +NULL ); +if (!xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; +if (!xfs_verify_dir_ino (sc->mp, pp->parent_ino)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +/* Is this the root dir? Then '..' must point to itself. */ +if (sc->ip == sc->mp->m_rootip && sc->ip->i_ino != pp->parent_ino) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); + +return 0 ; +} + +/* +* Try to lock a parent directory for checking dirents. Returns the inode +* flags for the locks we now hold, or zero if we failed. +*/ +STATIC unsigned int +xchk_parent_lock_dir ( +struct xfs_scrub *sc, +struct xfs_inode *dp) +{ +if (!xfs_ilock_nowait (dp,XFS_IOLOCK_SHARED )) +return 0 ; + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_SHARED )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +if (!xfs_need_iread_extents (&dp->i_df)) +return XFS_IOLOCK_SHARED |XFS_ILOCK_SHARED ; + +xfs_iunlock (dp,XFS_ILOCK_SHARED ); + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_EXCL )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +return XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; +} + +/* Check the forward link (dirent) associated with this parent pointer. */ +STATIC int +xchk_parent_dirent ( +struct xchk_pptrs *pp, +struct xfs_inode *dp) +{ +struct xfs_name xname = { + .name = pp->pptr.p_name, + .len = pp->pptr.p_namelen, + }; +struct xfs_scrub *sc = pp->sc; +xfs_ino_t child_ino; +xfs_dir2_dataptr_t child_diroffset; +int error; + +/* +* Use the name attached to this parent pointer to look up the +* directory entry in the alleged parent. +*/ + error =xchk_dir_lookup (sc, dp, &xname, &child_ino, &child_diroffset); +if (error == -ENOENT ) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* Does the inode number match? */ +if (child_ino != sc->ip->i_ino) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + +/* Does the directory offset match? */ +if (pp->pptr.p_diroffset != child_diroffset) { +trace_xchk_parent_bad_dapos (sc->ip, pp->pptr.p_diroffset, + dp->i_ino, child_diroffset, xname.name, + xname.len); +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + +/* +* If we're scanning a directory, we should only ever encounter a +* single parent pointer, and it should match the dotdot entry. We set +* the parent_ino from the dotdot entry before the scan, so compare it +* now. +*/ +if (!S_ISDIR (VFS_I (sc->ip)->i_mode)) +return 0 ; + +if (pp->parent_ino != dp->i_ino) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + + pp->parent_ino =NULLFSINO ; +return 0 ; +} + +/* Try to grab a parent directory. */ +STATIC int +xchk_parent_iget ( +struct xchk_pptrs *pp, +struct xfs_inode **dpp) +{ +struct xfs_scrub *sc = pp->sc; +struct xfs_inode *ip; +int error; + +/* Validate inode number. */ + error =xfs_dir_ino_validate (sc->mp, pp->pptr.p_ino); +if (error) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + + error =xchk_iget (sc, pp->pptr.p_ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* The parent must be a directory. */ +if (!S_ISDIR (VFS_I (ip)->i_mode)) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + +/* Validate generation number. */ +if (VFS_I (ip)->i_generation != pp->pptr.p_gen) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + + *dpp = ip; +return 0 ; +out_rele: +xchk_irele (sc, ip); +return 0 ; +} + +/* +* Walk an xattr of a file. If this xattr is a parent pointer, follow it up +* to a parent directory and check that the parent has a dirent pointing back +* to us. +*/ +STATIC int +xchk_parent_scan_attr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct xchk_pptrs *pp = priv; +struct xfs_inode *dp =NULL ; +const struct xfs_parent_name_rec *rec = (const void *)name; +unsigned int lockmode; +int error; + +/* Ignore incomplete xattrs */ +if (attr_flags &XFS_ATTR_INCOMPLETE ) +return 0 ; + +/* Ignore anything that isn't a parent pointer. */ +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ; + +/* Does the ondisk parent pointer structure make sense? */ +if (!xfs_parent_namecheck (sc->mp, rec, namelen, attr_flags)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +if (!xfs_parent_valuecheck (sc->mp, value, valuelen)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +xfs_parent_irec_from_disk (&pp->pptr, rec, value, valuelen); + + error =xchk_parent_iget (pp, &dp); +if (error) +return error; +if (!dp) +return 0 ; + +/* Try to lock the inode. */ + lockmode =xchk_parent_lock_dir (sc, dp); +if (!lockmode) { +xchk_set_incomplete (sc); + error = -ECANCELED ; + goto out_rele; + } + + error =xchk_parent_dirent (pp, dp); +if (error) + goto out_unlock; + +out_unlock: +xfs_iunlock (dp, lockmode); +out_rele: +xchk_irele (sc, dp); +return error; +} + +/* Check parent pointers of a file. */ +STATIC int +xchk_parent_pptr ( +struct xfs_scrub *sc) +{ +struct xchk_pptrs *pp; +int error; + + pp =kvzalloc (sizeof (struct xchk_pptrs ),XCHK_GFP_FLAGS ); +if (!pp) +return -ENOMEM ; + pp->sc = sc; + + error =xchk_parent_dotdot (pp); +if (error) + goto out_pp; + + error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_attr, pp); +if (error == -ECANCELED ) { + error =0 ; + goto out_pp; + } +if (error) + goto out_pp; + +out_pp: +kvfree (pp); +return error; +}
fs/xfs/scrub/trace.h +33 -0
@@ -896,6 +896,39 @@ TRACE_EVENT(xchk_nlinks_live_update, __get_str(name)) ); +TRACE_EVENT (xchk_parent_bad_dapos, + TP_PROTO(struct xfs_inode *ip, unsigned int p_diroffset, + xfs_ino_t parent_ino, unsigned int dapos, + const char *name, unsigned int namelen), +TP_ARGS (ip, p_diroffset, parent_ino, dapos, name, namelen), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, p_diroffset) + __field(xfs_ino_t, parent_ino) + __field(unsigned int, dapos) + __field(unsigned int, namelen) +__dynamic_array (char, name, namelen) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->p_diroffset = p_diroffset; + __entry->parent_ino = parent_ino; + __entry->dapos = dapos; + __entry->namelen = namelen; + memcpy(__get_str(name), name, namelen); + ), +TP_printk ("dev %d:%d ino 0x%llx p_diroff 0x%x parent_ino 0x%llx parent_diroff 0x%x name '%.*s'" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->p_diroffset, + __entry->parent_ino, + __entry->dapos, + __entry->namelen, +__get_str (name)) +); +/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
PATCH 2/2 xfs: deferred scrub of parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based dirent check fails, retain those parent pointers and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_parent.c | 38 +++++++ fs/xfs/libxfs/xfs_parent.h | 10 ++ fs/xfs/scrub/parent.c | 246 +++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 33 ++++++ 5 files changed, 324 insertions(+), 5 deletions(-)
fs/xfs/Makefile +1 -1
@@ -168,6 +168,7 @@ xfs-y += $(addprefix scrub/, \ scrub.o \ symlink.o \ xfarray.o \ + xfblob.o \ xfile.o \ )
@@ -181,7 +182,6 @@ xfs-y += $(addprefix scrub/, \ dir_repair.o \ repair.o \ tempfile.o \ - xfblob.o \ ) endif endif
fs/xfs/libxfs/xfs_parent.c +38 -0
@@ -298,3 +298,41 @@ xfs_pptr_calc_space_res(XFS_NEXTENTADD_SPACE_RES (mp, namelen,XFS_ATTR_FORK ); } +/* +* Look up the @name associated with the parent pointer (@pptr) of @ip. Caller +* must hold at least ILOCK_SHARED. Returns the length of the dirent name, or +* a negative errno. The scratchpad need not be initialized. +*/ +int +xfs_parent_lookup ( +struct xfs_trans *tp, +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +unsigned char *name, +unsigned int namelen, +struct xfs_parent_scratch *scr) +{ +int error; + +xfs_parent_irec_to_disk (&scr->rec,NULL ,NULL , pptr); + +memset (&scr->args,0 ,sizeof (struct xfs_da_args )); + scr->args.attr_filter =XFS_ATTR_PARENT ; + scr->args.dp = ip; + scr->args.geo = ip->i_mount->m_attr_geo; + scr->args.name = (const unsigned char *)&scr->rec; + scr->args.namelen =sizeof (struct xfs_parent_name_rec ); + scr->args.op_flags =XFS_DA_OP_OKNOENT ; + scr->args.trans = tp; + scr->args.valuelen = namelen; + scr->args.value = name; + scr->args.whichfork =XFS_ATTR_FORK ; + + scr->args.hashval =xfs_da_hashname (scr->args.name, scr->args.namelen); + + error =xfs_attr_get_ilocked (&scr->args); +if (error) +return error; + +return scr->args.valuelen; +}
fs/xfs/libxfs/xfs_parent.h +10 -0
@@ -103,4 +103,14 @@ xfs_parent_finish(unsigned int xfs_pptr_calc_space_res (struct xfs_mount *mp,unsigned int namelen); +/* Scratchpad memory so that raw parent operations don't burn stack space. */ +struct xfs_parent_scratch { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +int xfs_parent_lookup (struct xfs_trans *tp,struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr,unsigned char *name, +unsigned int namelen,struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/parent.c +242 -4
290 lines not on this page · raw message
fs/xfs/scrub/trace.h +33 -0
@@ -929,6 +929,39 @@ TRACE_EVENT(xchk_parent_bad_dapos, __get_str(name)) ); +DECLARE_EVENT_CLASS (xchk_pptr_class, + TP_PROTO(struct xfs_inode *ip, const unsigned char *name, + unsigned int namelen, xfs_ino_t parent_ino), +TP_ARGS (ip, name, namelen, parent_ino), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, namelen) + __dynamic_array(char, name, namelen) +__field (xfs_ino_t, parent_ino) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->namelen = namelen; + memcpy(__get_str(name), name, namelen); + __entry->parent_ino = parent_ino; + ), +TP_printk ("dev %d:%d ino 0x%llx name '%.*s' parent_ino 0x%llx" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->namelen, +__get_str (name), + __entry->parent_ino) +) +#define DEFINE_XCHK_PPTR_CLASS (name) \ +DEFINE_EVENT(xchk_pptr_class, name, \ + TP_PROTO(struct xfs_inode *ip, const unsigned char *name, \ + unsigned int namelen, xfs_ino_t parent_ino), \ + TP_ARGS(ip, name, namelen, parent_ino)) +DEFINE_XCHK_PPTR_CLASS (xchk_parent_defer); +DEFINE_XCHK_PPTR_CLASS (xchk_parent_slowpath); +/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
cover xfs: online checking of parent pointers
In reply to Darrick J. WongHi all,
With this patchset, we implement online repairs for parent pointers. This is structured similarly to the directory repair code in that we scan the entire filesystem looking for dirents and use them to reconstruct the parent pointer information.
Note that the atomic swapext and block reaping code is NOT ported for this PoC, so we do not commit any repairs.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-repair
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-parent-repair
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_parent.c | 56 +++ fs/xfs/libxfs/xfs_parent.h | 8 fs/xfs/scrub/parent.c | 10 + fs/xfs/scrub/parent_repair.c | 739 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 4 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.c | 2 fs/xfs/scrub/trace.h | 80 +++++ fs/xfs/xfs_inode.h | 6 10 files changed, 905 insertions(+), 3 deletions(-) create mode 100644 fs/xfs/scrub/parent_repair.c
PATCH 1/3 xfs: repair parent pointers by scanning directories
From: Darrick J. Wong <[email protected]>
Walk the filesystem to rebuild parent pointer information.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_parent.c | 31 ++ fs/xfs/libxfs/xfs_parent.h | 4 fs/xfs/scrub/parent.c | 10 + fs/xfs/scrub/parent_repair.c | 583 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 4 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.c | 2 fs/xfs/scrub/trace.h | 77 ++++++ fs/xfs/xfs_inode.h | 6 10 files changed, 717 insertions(+), 3 deletions(-) create mode 100644 fs/xfs/scrub/parent_repair.c
fs/xfs/Makefile +1 -0
@@ -180,6 +180,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ dir_repair.o \ + parent_repair.o \ repair.o \ tempfile.o \ )
fs/xfs/libxfs/xfs_parent.c +29 -2
@@ -94,11 +94,11 @@ xfs_parent_valuecheck(static inline void xfs_init_parent_name_rec (struct xfs_parent_name_rec *rec, -struct xfs_inode *ip, +const struct xfs_inode *ip,uint32_t p_diroffset) {xfs_ino_t p_ino = ip->i_ino; -uint32_t p_gen =VFS_I (ip)->i_generation; +uint32_t p_gen =VFS_IC (ip)->i_generation; rec->p_ino =cpu_to_be64 (p_ino); rec->p_gen =cpu_to_be32 (p_gen);
@@ -336,3 +336,30 @@ xfs_parent_lookup(return scr->args.valuelen; } + +/* +* Attach the parent pointer (@pptr -> @name) to @ip immediately. Caller must +* not have a transaction or hold the ILOCK. The update will not use logged +* xattrs. This is for specialized repair functions only. The scratchpad need +* not be initialized. +*/ +int +xfs_parent_set ( +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scr) +{ +xfs_parent_irec_to_disk (&scr->rec,NULL ,NULL , pptr); + +memset (&scr->args,0 ,sizeof (struct xfs_da_args )); + scr->args.attr_filter =XFS_ATTR_PARENT ; + scr->args.dp = ip; + scr->args.geo = ip->i_mount->m_attr_geo; + scr->args.name = (const unsigned char *)&scr->rec; + scr->args.namelen =sizeof (struct xfs_parent_name_rec ); + scr->args.valuelen = pptr->p_namelen; + scr->args.value = (void *)pptr->p_name; + scr->args.whichfork =XFS_ATTR_FORK ; + +return xfs_attr_set (&scr->args); +}
fs/xfs/libxfs/xfs_parent.h +4 -0
@@ -113,4 +113,8 @@ int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip, const struct xfs_parent_name_irec *pptr, unsigned char *name, unsigned int namelen, struct xfs_parent_scratch *scratch); +int xfs_parent_set (struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/parent.c +10 -0
@@ -10,6 +10,7 @@#include "xfs_trans_resv.h" #include "xfs_mount.h" #include "xfs_log_format.h" +#include "xfs_trans.h" #include "xfs_inode.h" #include "xfs_icache.h" #include "xfs_dir2.h"
@@ -24,12 +25,21 @@#include "scrub/xfarray.h" #include "scrub/xfblob.h" #include "scrub/trace.h" +#include "scrub/repair.h" /* Set us up to scrub parents. */ int xchk_setup_parent (struct xfs_scrub *sc) { +int error; + +if (xchk_could_repair (sc)) { + error =xrep_setup_parent (sc); +if (error) +return error; + } +return xchk_setup_inode_contents (sc,0 ); }
fs/xfs/scrub/parent_repair.c new file +583 -0
583 lines not on this page · raw message
fs/xfs/scrub/repair.h +4 -0
@@ -68,6 +68,7 @@ void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc);int xrep_setup_directory (struct xfs_scrub *sc); +int xrep_setup_parent (struct xfs_scrub *sc);/* Metadata repairers */
@@ -77,6 +78,7 @@ int xrep_agf(struct xfs_scrub *sc);int xrep_agfl (struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc);int xrep_directory (struct xfs_scrub *sc); +int xrep_parent (struct xfs_scrub *sc);#else
@@ -97,6 +99,7 @@ xrep_calc_ag_resblks( }#define xrep_setup_directory (sc) (0) +#define xrep_setup_parent (sc) (0)#define xrep_probe xrep_notsupported#define xrep_superblock xrep_notsupported
@@ -104,6 +107,7 @@ xrep_calc_ag_resblks(#define xrep_agfl xrep_notsupported#define xrep_agi xrep_notsupported#define xrep_directory xrep_notsupported +#define xrep_parent xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -317,7 +317,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_parent, .scrub = xchk_parent, - .repair = xrep_notsupported, + .repair = xrep_parent, }, [XFS_SCRUB_TYPE_RTBITMAP] = { /* realtime bitmap */ .type = ST_FS,
fs/xfs/scrub/trace.c +2 -0
@@ -18,6 +18,8 @@#include "scrub/xfarray.h" #include "scrub/iscan.h" #include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_parent.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +77 -0
89 lines not on this page · raw message
fs/xfs/xfs_inode.h +6 -0
@@ -153,6 +153,12 @@ static inline struct inode *VFS_I(struct xfs_inode *ip)return &ip->i_vnode; } +/* convert from const xfs inode to vfs inode */ +static inline const struct inode *VFS_IC (const struct xfs_inode *ip) +{ +return &ip->i_vnode; +} +/* * For regular files we only update the on-disk filesize when actually * writing data back to disk. Until then only the copy in the VFS inode
PATCH 2/3 xfs: repair parent pointers with live scan hooks
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use the nlink hooks to keep our tempfile's parent pointers up to date.
--- fs/xfs/libxfs/xfs_parent.c | 25 ++++++++++ fs/xfs/libxfs/xfs_parent.h | 4 ++ fs/xfs/scrub/parent_repair.c | 110 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 2 + 4 files changed, 141 insertions(+)
fs/xfs/libxfs/xfs_parent.c +25 -0
@@ -363,3 +363,28 @@ xfs_parent_set(return xfs_attr_set (&scr->args); } + +/* +* Remove the parent pointer (@rec -> @name) from @ip immediately. Caller must +* not have a transaction or hold the ILOCK. The update will not use logged +* xattrs. This is for specialized repair functions only. The scratchpad need +* not be initialized. +*/ +int +xfs_parent_unset ( +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scr) +{ +xfs_parent_irec_to_disk (&scr->rec,NULL ,NULL , pptr); + +memset (&scr->args,0 ,sizeof (struct xfs_da_args )); + scr->args.attr_filter =XFS_ATTR_PARENT ; + scr->args.dp = ip; + scr->args.geo = ip->i_mount->m_attr_geo; + scr->args.name = (const unsigned char *)&scr->rec; + scr->args.namelen =sizeof (struct xfs_parent_name_rec ); + scr->args.whichfork =XFS_ATTR_FORK ; + +return xfs_attr_set (&scr->args); +}
fs/xfs/libxfs/xfs_parent.h +4 -0
@@ -117,4 +117,8 @@ int xfs_parent_set(struct xfs_inode *ip,const struct xfs_parent_name_irec *pptr, struct xfs_parent_scratch *scratch); +int xfs_parent_unset (struct xfs_inode *ip, +const struct xfs_parent_name_irec *rec, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/parent_repair.c +110 -0
160 lines not on this page · raw message
fs/xfs/scrub/trace.h +2 -0
@@ -1354,6 +1354,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \ TP_PROTO(struct xfs_inode *ip, const struct xfs_parent_name_irec *pptr), \ TP_ARGS(ip, pptr)) DEFINE_XREP_PPTR_CLASS(xrep_pptr_createname); +DEFINE_XREP_PPTR_CLASS (xrep_pptr_removename);DEFINE_XREP_PPTR_CLASS (xrep_pptr_dumpname); DECLARE_EVENT_CLASS(xrep_pptr_scan_class,
@@ -1393,6 +1394,7 @@ DEFINE_EVENT(xrep_pptr_scan_class, name, \ unsigned int diroffset, const struct xfs_name *name), \ TP_ARGS(ip, dp, diroffset, name)) DEFINE_XREP_PPTR_SCAN_CLASS(xrep_pptr_add_pointer); +DEFINE_XREP_PPTR_SCAN_CLASS (xrep_pptr_remove_pointer);#endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
PATCH 3/3 xfs: compare generated and existing parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check our work to make sure we found all the parent pointers that the original file had.
--- fs/xfs/scrub/parent_repair.c | 52 ++++++++++++++++++++++++++++++++++++++++-- fs/xfs/scrub/trace.h | 1 + 2 files changed, 50 insertions(+), 3 deletions(-)
fs/xfs/scrub/parent_repair.c +49 -3
@@ -127,6 +127,9 @@ struct xrep_pptrs {/* Parent pointer names. */ struct xfblob *pptr_names; + +/* Buffer for validation. */ +unsigned char namebuf[MAXNAMELEN ]; };/* Tear down all the incore stuff we created. */
@@ -490,7 +493,10 @@ xrep_pptr_scan_dirtree(return 0 ; } -/* Dump a parent pointer from the temporary file. */ +/* +* Dump a parent pointer from the temporary file and check it against the file +* we're rebuilding. We are not committing any of this. +*/ STATIC int xrep_pptr_dump_tempptr( struct xfs_scrub *sc,
@@ -504,13 +510,45 @@ xrep_pptr_dump_tempptr( {struct xrep_pptrs *rp = priv;const struct xfs_parent_name_rec *rec = (const void *)name; +struct xfs_inode *other_ip; +int pptr_namelen;if (!(attr_flags &XFS_ATTR_PARENT ))return 0 ; +if (ip == sc->ip) + other_ip = sc->tempip; +else if (ip == sc->tempip) + other_ip = sc->ip; +else +return -EFSCORRUPTED ; +xfs_parent_irec_from_disk (&rp->pptr, rec, value, valuelen);trace_xrep_pptr_dumpname (sc->tempip, &rp->pptr); + + pptr_namelen =xfs_parent_lookup (sc->tp, other_ip, &rp->pptr, + rp->namebuf,MAXNAMELEN , &rp->pptr_scratch); +if (pptr_namelen == -ENOATTR ) { +trace_xrep_pptr_checkname (other_ip, &rp->pptr); +ASSERT (pptr_namelen != -ENOATTR ); +return -EFSCORRUPTED ; + } +if (pptr_namelen <0 ) +return pptr_namelen; + +if (pptr_namelen != rp->pptr.p_namelen) { +trace_xrep_pptr_checkname (other_ip, &rp->pptr); +ASSERT (pptr_namelen == rp->pptr.p_namelen); +return -EFSCORRUPTED ; + } + +if (memcmp (rp->namebuf, rp->pptr.p_name, rp->pptr.p_namelen)) { +trace_xrep_pptr_checkname (other_ip, &rp->pptr); +ASSERT (0 ); +return -EFSCORRUPTED ; + } +return 0 ; }
@@ -566,8 +604,16 @@ xrep_pptr_rebuild_tree(trace_xrep_pptr_rebuild_tree (sc->ip,0 ); -xrep_tempfile_ilock (sc); -return xchk_xattr_walk (sc, sc->tempip, xrep_pptr_dump_tempptr, rp); +xchk_ilock (sc,XFS_ILOCK_EXCL ); + error =xrep_tempfile_ilock_polled (sc); +if (error) +return error; + + error =xchk_xattr_walk (sc, sc->tempip, xrep_pptr_dump_tempptr, rp); +if (error) +return error; + +return xchk_xattr_walk (sc, sc->ip, xrep_pptr_dump_tempptr, rp); }/*
fs/xfs/scrub/trace.h +1 -0
@@ -1356,6 +1356,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \DEFINE_XREP_PPTR_CLASS (xrep_pptr_createname);DEFINE_XREP_PPTR_CLASS (xrep_pptr_removename);DEFINE_XREP_PPTR_CLASS (xrep_pptr_dumpname); +DEFINE_XREP_PPTR_CLASS (xrep_pptr_checkname); DECLARE_EVENT_CLASS(xrep_pptr_scan_class, TP_PROTO(struct xfs_inode *ip, const struct xfs_inode *dp,
cover xfs: online checking of directories
In reply to Darrick J. WongHi all,
Update the existing online directory checker to confirm the parent pointers that should also exist.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-check
--- fs/xfs/scrub/dir.c | 367 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 2 2 files changed, 368 insertions(+), 1 deletion(-)
PATCH 1/2 xfs: check dirents have parent pointers
From: Darrick J. Wong <[email protected]>
If the fs has parent pointers, we need to check that each child dirent points to a file that has a parent pointer pointing back at us.
--- fs/xfs/scrub/dir.c | 134 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 133 insertions(+), 1 deletion(-)
fs/xfs/scrub/dir.c +133 -1
@@ -15,6 +15,8 @@#include "xfs_icache.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/dabtree.h"
@@ -39,6 +41,20 @@ xchk_setup_directory(/* Directories */ +struct xchk_dir { +struct xfs_scrub *sc; + +/* Scratch buffer for scanning pptr xattrs */ +struct xfs_parent_name_irec pptr; + +/* xattr key and da args for parent pointer validation. */ +struct xfs_parent_scratch pptr_scratch; + +/* Name buffer for pptr validation and dirent revalidation. */ +uint8_t namebuf[MAXNAMELEN ]; + +}; +/* Scrub a directory entry. */ /* Check that an inode's mode matches a given XFS_DIR3_FT_* type. */
@@ -61,6 +77,105 @@ xchk_dir_check_ftype(xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); } +/* +* Try to lock a child file for checking parent pointers. Returns the inode +* flags for the locks we now hold, or zero if we failed. +*/ +STATIC unsigned int +xchk_dir_lock_child ( +struct xfs_scrub *sc, +struct xfs_inode *ip) +{ +if (!xfs_ilock_nowait (ip,XFS_IOLOCK_SHARED )) +return 0 ; + +if (!xfs_ilock_nowait (ip,XFS_ILOCK_SHARED )) { +xfs_iunlock (ip,XFS_IOLOCK_SHARED ); +return 0 ; + } + +if (!xfs_inode_has_attr_fork (ip) || !xfs_need_iread_extents (&ip->i_af)) +return XFS_IOLOCK_SHARED |XFS_ILOCK_SHARED ; + +xfs_iunlock (ip,XFS_ILOCK_SHARED ); + +if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) { +xfs_iunlock (ip,XFS_IOLOCK_SHARED ); +return 0 ; + } + +return XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; +} + +/* Check the backwards link (parent pointer) associated with this dirent. */ +STATIC int +xchk_dir_parent_pointer ( +struct xchk_dir *sd, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +struct xfs_inode *ip) +{ +struct xfs_scrub *sc = sd->sc; +int pptr_namelen; + + sd->pptr.p_ino = sc->ip->i_ino; + sd->pptr.p_gen =VFS_I (sc->ip)->i_generation; + sd->pptr.p_diroffset = dapos; + + pptr_namelen =xfs_parent_lookup (sc->tp, ip, &sd->pptr, sd->namebuf, +MAXNAMELEN , &sd->pptr_scratch); +if (pptr_namelen == -ENOATTR ) { +xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } +if (pptr_namelen <0 ) { +xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &pptr_namelen); +return pptr_namelen; + } + +if (pptr_namelen != name->len) { +xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +if (memcmp (sd->namebuf, name->name, name->len)) { +xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +return 0 ; +} + +/* Look for a parent pointer matching this dirent, if the child isn't busy. */ +STATIC int +xchk_dir_check_pptr_fast ( +struct xchk_dir *sd, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +struct xfs_inode *ip) +{ +struct xfs_scrub *sc = sd->sc; +unsigned int lockmode; +int error; + +/* dot and dotdot entries do not have parent pointers */ +if (!strncmp ("." , name->name, name->len) || + !strncmp (".." , name->name, name->len)) +return 0 ; + +/* Try to lock the inode. */ + lockmode =xchk_dir_lock_child (sc, ip); +if (!lockmode) { +xchk_set_incomplete (sc); +return -ECANCELED ; + } + + error =xchk_dir_parent_pointer (sd, dapos, name, ip); +xfs_iunlock (ip, lockmode); +return error; +} +/* * Scrub a single directory entry. *
@@ -78,6 +193,7 @@ xchk_dir_actor( {struct xfs_mount *mp = dp->i_mount;struct xfs_inode *ip; +struct xchk_dir *sd = priv;xfs_ino_t lookup_ino;xfs_dablk_t offset;int error =0 ;
@@ -144,6 +260,14 @@ xchk_dir_actor( goto out;xchk_dir_check_ftype (sc, offset, ip, name->type); + +if (xfs_has_parent (mp)) { + error =xchk_dir_check_pptr_fast (sd, dapos, name, ip); +if (error) + goto out_rele; + } + +out_rele:xchk_irele (sc, ip); out:if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT )
@@ -759,6 +883,7 @@ intxchk_directory ( struct xfs_scrub *sc) { +struct xchk_dir *sd;int error;if (!S_ISDIR (VFS_I (sc->ip)->i_mode))
@@ -786,9 +911,16 @@ xchk_directory(if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT )return 0 ; + sd =kvzalloc (sizeof (struct xchk_dir ),XCHK_GFP_FLAGS ); +if (!sd) +return -ENOMEM ; + sd->sc = sc; +/* Look up every name in this directory by hash. */ - error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor,NULL ); + error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor, sd);if (error == -ECANCELED ) error =0 ; + +kvfree (sd);return error; }
PATCH 2/2 xfs: deferred scrub of dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based parent pointer check fails, retain those dirents and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/scrub/dir.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 2 2 files changed, 237 insertions(+), 2 deletions(-)
fs/xfs/scrub/dir.c +235 -2
@@ -22,6 +22,10 @@#include "scrub/dabtree.h" #include "scrub/readdir.h" #include "scrub/repair.h" +#include "scrub/trace.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" /* Set us up to scrub directories. */ int
@@ -41,6 +45,21 @@ xchk_setup_directory(/* Directories */ +/* Deferred directory entry that we saved for later. */ +struct xchk_dirent { +/* Cookie for retrieval of the dirent name. */ +xfblob_cookie name_cookie; + +/* Child inode number. */ +xfs_ino_t ino; + +/* Directory offset. */ +xfs_dir2_dataptr_t diroffset; + +/* Length of the pptr name. */ +uint8_t namelen; +}; +struct xchk_dir {struct xfs_scrub *sc;
@@ -50,6 +69,15 @@ struct xchk_dir {/* xattr key and da args for parent pointer validation. */ struct xfs_parent_scratch pptr_scratch; +/* Fixed-size array of xchk_dirent structures. */ +struct xfarray *dir_entries; + +/* Blobs containing dirent names. */ +struct xfblob *dir_names; + +/* If we've cycled the ILOCK, we must revalidate deferred dirents. */ +bool need_revalidate; +/* Name buffer for pptr validation and dirent revalidation. */ uint8_t namebuf[MAXNAMELEN ];
@@ -167,8 +195,25 @@ xchk_dir_check_pptr_fast(/* Try to lock the inode. */ lockmode =xchk_dir_lock_child (sc, ip);if (!lockmode) { -xchk_set_incomplete (sc); -return -ECANCELED ; +struct xchk_dirent save_de = { + .namelen = name->len, + .ino = ip->i_ino, + .diroffset = dapos, + }; + +/* Couldn't lock the inode, so save the dirent for later. */ +trace_xchk_dir_defer (sc->ip, name->name, name->len, ip->i_ino); + + error =xfblob_store (sd->dir_names, &save_de.name_cookie, + name->name, name->len); +if (xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; + + error =xfarray_append (sd->dir_entries, &save_de); +if (xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; + +return 0 ; } error =xchk_dir_parent_pointer (sd, dapos, name, ip);
@@ -878,6 +923,164 @@ xchk_directory_blocks(return error; } +/* +* Revalidate a dirent that we collected in the past but couldn't check because +* of lock contention. Returns 0 if the dirent is still valid, -ENOENT if it +* has gone away on us, or a negative errno. +*/ +STATIC int +xchk_dir_revalidate_dirent ( +struct xchk_dir *sd, +const struct xfs_name *xname, +xfs_ino_t ino, +xfs_dir2_dataptr_t diroffset) +{ +struct xfs_scrub *sc = sd->sc; +xfs_ino_t child_ino; +xfs_dir2_dataptr_t child_diroffset =XFS_DIR2_NULL_DATAPTR ; +int error; + + error =xchk_dir_lookup (sc, sc->ip, xname, &child_ino, + &child_diroffset); +if (error == -ENOENT ) { +/* Directory entry went away, nothing to revalidate. */ +return -ENOENT ; + } +if (error) +return error; + +/* The inode number changed, nothing to revalidate. */ +if (ino != child_ino) +return -ENOENT ; + +/* The directory offset changed, nothing to revalidate. */ +if (diroffset != child_diroffset) +return -ENOENT ; + +return 0 ; +} + +/* +* Check a directory entry's parent pointers the slow way, which means we cycle +* locks a bunch and put up with revalidation until we get it done. +*/ +STATIC int +xchk_dir_slow_dirent ( +struct xchk_dir *sd, +struct xchk_dirent *dirent) +{ +struct xfs_name xname = { + .name = sd->namebuf, + .len = dirent->namelen, + }; +struct xfs_scrub *sc = sd->sc; +struct xfs_inode *ip; +unsigned int lockmode; +int error; + +/* Check that the deferred dirent still exists. */ +if (sd->need_revalidate) { + error =xchk_dir_revalidate_dirent (sd, &xname, dirent->ino, + dirent->diroffset); +if (error == -ENOENT ) +return 0 ; +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + } + + error =xchk_iget (sc, dirent->ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* +* If we can grab both IOLOCK and ILOCK of the alleged child, we can +* proceed with the validation. +*/ + lockmode =xchk_dir_lock_child (sc, ip); +if (lockmode) + goto check_pptr; + +/* +* We couldn't lock the child file. Drop all the locks and try to +* get them again, one at a time. +*/ +xchk_iunlock (sc, sc->ilock_flags); + sd->need_revalidate = true; + +trace_xchk_dir_slowpath (sc->ip, xname.name, xname.len, ip->i_ino); + +while (true) { +xchk_ilock (sc,XFS_IOLOCK_EXCL ); +if (xfs_ilock_nowait (ip,XFS_IOLOCK_SHARED )) { +xchk_ilock (sc,XFS_ILOCK_EXCL ); +if (xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) { +break ; + } +xchk_iunlock (sc,XFS_ILOCK_EXCL ); + } +xchk_iunlock (sc,XFS_IOLOCK_EXCL ); + +if (xchk_should_terminate (sc, &error)) + goto out_rele; + +delay (1 ); + } + lockmode =XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; + +/* Revalidate, since we just cycled the locks. */ + error =xchk_dir_revalidate_dirent (sd, &xname, dirent->ino, + dirent->diroffset); +if (error == -ENOENT ) + goto out_unlock; +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error)) + goto out_unlock; + +check_pptr: + error =xchk_dir_parent_pointer (sd, dirent->diroffset, &xname, ip); +out_unlock: +xfs_iunlock (ip, lockmode); +out_rele: +xchk_irele (sc, ip); +return error; +} + +/* Check all the dirents that we deferred the first time around. */ +STATIC int +xchk_dir_finish_slow_dirents ( +struct xchk_dir *sd) +{ +xfarray_idx_t array_cur; +int error; + + foreach_xfarray_idx(sd->dir_entries, array_cur) { +struct xchk_dirent dirent; + +if (sd->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) +return 0 ; + + error =xfarray_load (sd->dir_entries, array_cur, &dirent); +if (error) +return error; + + error =xfblob_load (sd->dir_names, dirent.name_cookie, + sd->namebuf, dirent.namelen); +if (error) +return error; + sd->namebuf[MAXNAMELEN -1 ] =0 ; + + error =xchk_dir_slow_dirent (sd, &dirent); +if (error) +return error; + } + +return 0 ; +} +/* Scrub a whole directory. */ int xchk_directory(
@@ -916,11 +1119,41 @@ xchk_directory(return -ENOMEM ; sd->sc = sc; +if (xfs_has_parent (sc->mp)) { +/* +* Set up some staging memory for dirents that we can't check +* due to locking contention. +*/ + error =xfarray_create (sc->mp,"directory entries" ,0 , +sizeof (struct xchk_dirent ), &sd->dir_entries); +if (error) + goto out_sd; + + error =xfblob_create (sc->mp,"dirent names" , &sd->dir_names); +if (error) + goto out_entries; + } +/* Look up every name in this directory by hash. */ error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor, sd);if (error == -ECANCELED ) error =0 ; +if (error) + goto out_names; +if (xfs_has_parent (sc->mp)) { + error =xchk_dir_finish_slow_dirents (sd); +if (error) + goto out_names; + } + +out_names: +if (sd->dir_names) +xfblob_destroy (sd->dir_names); +out_entries: +if (sd->dir_entries) +xfarray_destroy (sd->dir_entries); +out_sd:kvfree (sd);return error; }
fs/xfs/scrub/trace.h +2 -0
@@ -962,6 +962,8 @@ DEFINE_EVENT(xchk_pptr_class, name, \ TP_ARGS(ip, name, namelen, parent_ino))DEFINE_XCHK_PPTR_CLASS (xchk_parent_defer);DEFINE_XCHK_PPTR_CLASS (xchk_parent_slowpath); +DEFINE_XCHK_PPTR_CLASS (xchk_dir_defer); +DEFINE_XCHK_PPTR_CLASS (xchk_dir_slowpath);/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
cover xfs: encode parent pointer name in xattr key
In reply to Darrick J. WongHi all,
As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:
(parent_ino, parent_gen, diroffset) -> (dirent_name)
If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.
One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.
This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.
Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.
Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:
(parent_ino, dirent_name) -> NULL
The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:
(parent_ino, parent_gen, dirent_name) -> NULL
Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:
(parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)
The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:
For dirent names shorter than 243 bytes:
(parent_ino, parent_gen, dirent_name) -> NULL
For dirent names longer than 243 bytes:
(parent_ino, parent_gen, dirent_name[0:178],
sha512(child_gen, dirent_name)) -> (dirent_name[179:255])The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.
A messier solution to this problem would be to extend the xattr ondisk format to allow parent pointers to have xattr names up to 267 bytes. This would likely involve redefining the ondisk namelen field to omit the size of the parent ino/gen information and might be madness.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key
--- fs/xfs/Kconfig | 1 fs/xfs/libxfs/xfs_da_format.h | 49 +++++++ fs/xfs/libxfs/xfs_fs.h | 4 - fs/xfs/libxfs/xfs_parent.c | 265 ++++++++++++++++++++++++++++++++-------- fs/xfs/libxfs/xfs_parent.h | 48 +++++-- fs/xfs/libxfs/xfs_trans_resv.c | 6 - fs/xfs/scrub/dir.c | 16 ++ fs/xfs/scrub/dir_repair.c | 87 ++++--------- fs/xfs/scrub/parent.c | 51 +++++--- fs/xfs/scrub/parent_repair.c | 29 ++-- fs/xfs/scrub/trace.h | 48 ++----- fs/xfs/xfs_attr_item.c | 4 - fs/xfs/xfs_inode.c | 30 ++--- fs/xfs/xfs_linux.h | 1 fs/xfs/xfs_mount.c | 13 ++ fs/xfs/xfs_mount.h | 3 fs/xfs/xfs_ondisk.h | 6 + fs/xfs/xfs_parent_utils.c | 4 - fs/xfs/xfs_sha512.h | 42 ++++++ fs/xfs/xfs_super.c | 3 fs/xfs/xfs_symlink.c | 3 21 files changed, 481 insertions(+), 232 deletions(-) create mode 100644 fs/xfs/xfs_sha512.h
PATCH 1/5 xfs: load secure hash algorithm for parent pointers
From: Darrick J. Wong <[email protected]>
We're about to start replacing the diroffset field of parent pointers with a collision-resistant hash of the directory entry name. Start by attaching the sha512 crypto implementation if parent pointers are attached.
--- fs/xfs/Kconfig | 1 + fs/xfs/xfs_linux.h | 1 + fs/xfs/xfs_mount.c | 13 +++++++++++++ fs/xfs/xfs_mount.h | 3 +++ fs/xfs/xfs_super.c | 3 +++ 5 files changed, 21 insertions(+)
fs/xfs/Kconfig +1 -0
@@ -5,6 +5,7 @@ config XFS_FS select EXPORTFS select LIBCRC32C select FS_IOMAP + select CRYPTO_SHA512 help XFS is a high performance journaling filesystem which originated on the SGI IRIX platform. It is completely multi-threaded, can
fs/xfs/xfs_linux.h +1 -0
@@ -62,6 +62,7 @@ typedef __u32 xfs_nlink_t;#include <linux/rhashtable.h> #include <linux/xattr.h> #include <linux/mnt_idmapping.h> +#include <crypto/hash.h> #include <asm/page.h> #include <asm/div64.h>
fs/xfs/xfs_mount.c +13 -0
@@ -983,6 +983,19 @@ xfs_mountfs( goto out_agresv; } +if (xfs_has_parent (mp)) { +struct crypto_shash *tfm; + + tfm =crypto_alloc_shash ("sha512" ,0 ,0 ); +if (IS_ERR (tfm)) { + error =PTR_ERR (tfm); + goto out_agresv; + } +xfs_info (mp,"parent pointer hash %s" , +crypto_shash_driver_name (tfm)); + mp->m_sha512 = tfm; + } +return 0 ; out_agresv:
fs/xfs/xfs_mount.h +3 -0
@@ -244,6 +244,9 @@ typedef struct xfs_mount {#endif /* Hook to feed file directory updates to an active online repair. */ struct xfs_hooks m_dirent_update_hooks; + +/* sha512 engine, if needed */ +struct crypto_shash *m_sha512; } xfs_mount_t;#define M_IGEO (mp) (&(mp)->m_ino_geo)
fs/xfs/xfs_super.c +3 -0
@@ -738,6 +738,8 @@ xfs_mount_free( {kfree (mp->m_rtname);kfree (mp->m_logname); +if (mp->m_sha512) +crypto_free_shash (mp->m_sha512);kmem_free (mp); }
@@ -1961,6 +1963,7 @@ static int xfs_init_fs_context(if (fc->sb_flags &SB_SYNCHRONOUS ) mp->m_features |=XFS_FEAT_WSYNC ; + mp->m_sha512 =NULL ; fc->s_fs_info = mp; fc->ops = &xfs_context_ops;
PATCH 2/5 xfs: replace parent pointer diroffset with sha512 hash of name
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Replace the diroffset with the sha512 hash of the dirent name, thereby eliminating the need for directory repair to update all the parent pointers after rebuilding the directory.
--- fs/xfs/libxfs/xfs_da_format.h | 15 +++-- fs/xfs/libxfs/xfs_fs.h | 4 + fs/xfs/libxfs/xfs_parent.c | 125 +++++++++++++++++++++++++++++++---------- fs/xfs/libxfs/xfs_parent.h | 21 ++++--- fs/xfs/scrub/dir.c | 12 +++- fs/xfs/scrub/dir_repair.c | 83 ++++++++------------------- fs/xfs/scrub/parent.c | 43 ++++++++++---- fs/xfs/scrub/parent_repair.c | 27 ++++----- fs/xfs/scrub/trace.h | 48 +++++----------- fs/xfs/xfs_inode.c | 30 ++++------ fs/xfs/xfs_ondisk.h | 4 + fs/xfs/xfs_parent_utils.c | 2 - fs/xfs/xfs_sha512.h | 42 ++++++++++++++ fs/xfs/xfs_symlink.c | 3 - 14 files changed, 271 insertions(+), 188 deletions(-) create mode 100644 fs/xfs/xfs_sha512.h
fs/xfs/libxfs/xfs_da_format.h +10 -5
@@ -824,17 +824,22 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* We use sha512 for the parent pointer name hash. */ +#define XFS_PARENT_NAME_HASH_SIZE (64) +/* * Parent pointer attribute format definition * -* EA name encodes the parent inode number, generation and the offset of -* the dirent that points to the child inode. The EA value contains the -* same name as the dirent in the parent directory. +* The EA name encodes the parent inode number, generation and a collision +* resistant hash computed from the dirent name. The hash is defined to be the +* sha512 of the child inode generation and the dirent name. +* +* The EA value contains the same name as the dirent in the parent directory. */ struct xfs_parent_name_rec {__be64 p_ino;__be32 p_gen; -__be32 p_diroffset; -}; +__u8 p_namehash[XFS_PARENT_NAME_HASH_SIZE ]; +} __attribute__((packed));#endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_fs.h +2 -2
@@ -770,8 +770,8 @@ struct xfs_scrub_metadata {struct xfs_parent_ptr {__u64 xpp_ino;/* Inode */ __u32 xpp_gen;/* Inode generation */ -__u32 xpp_diroffset;/* Directory offset */ -__u64 xpp_rsvd;/* Reserved */ +__u32 xpp_rsvd;/* Reserved */ +__u64 xpp_rsvd2;/* Reserved */ __u8 xpp_name[XFS_PPTR_MAXNAMELEN ];/* File name */ };
fs/xfs/libxfs/xfs_parent.c +96 -29
205 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +13 -8
@@ -23,7 +23,7 @@ struct xfs_parent_name_irec {/* Key fields for looking up a particular parent pointer. */ xfs_ino_t p_ino;uint32_t p_gen; -xfs_dir2_dataptr_t p_diroffset; +uint8_t p_namehash[XFS_PARENT_NAME_HASH_SIZE ];/* Attributes of a parent pointer. */ uint8_t p_namelen;
@@ -79,15 +79,14 @@ xfs_parent_start_locked(int xfs_parent_add (struct xfs_trans *tp,struct xfs_parent_defer *parent,struct xfs_inode *dp,const struct xfs_name *parent_name, -xfs_dir2_dataptr_t diroffset,struct xfs_inode *child); +struct xfs_inode *child);int xfs_parent_replace (struct xfs_trans *tp,struct xfs_parent_defer *new_parent,struct xfs_inode *old_dp, -xfs_dir2_dataptr_t old_diroffset, -const struct xfs_name *parent_name,struct xfs_inode *new_ip, -xfs_dir2_dataptr_t new_diroffset,struct xfs_inode *child); -int xfs_parent_remove (struct xfs_trans *tp,struct xfs_inode *dp, -struct xfs_parent_defer *parent,xfs_dir2_dataptr_t diroffset, -struct xfs_inode *child); +const struct xfs_name *old_name,struct xfs_inode *new_ip, +const struct xfs_name *new_name,struct xfs_inode *child); +int xfs_parent_remove (struct xfs_trans *tp, +struct xfs_parent_defer *parent,struct xfs_inode *dp, +const struct xfs_name *name,struct xfs_inode *child);void __xfs_parent_cancel (struct xfs_mount *mp,struct xfs_parent_defer *parent);
@@ -100,6 +99,12 @@ xfs_parent_finish(__xfs_parent_cancel (mp, p); } +int xfs_parent_namehash (struct xfs_inode *ip,const struct xfs_name *name, +void *namehash,unsigned int namehash_len); + +int xfs_parent_irec_hash (struct xfs_inode *ip, +struct xfs_parent_name_irec *pptr); +unsigned int xfs_pptr_calc_space_res (struct xfs_mount *mp,unsigned int namelen);
fs/xfs/scrub/dir.c +8 -4
@@ -139,16 +139,20 @@ xchk_dir_lock_child( STATIC intxchk_dir_parent_pointer (struct xchk_dir *sd, -xfs_dir2_dataptr_t dapos,const struct xfs_name *name,struct xfs_inode *ip) {struct xfs_scrub *sc = sd->sc;int pptr_namelen; +int error; sd->pptr.p_ino = sc->ip->i_ino; sd->pptr.p_gen =VFS_I (sc->ip)->i_generation; - sd->pptr.p_diroffset = dapos; + + error =xfs_parent_namehash (ip, name, &sd->pptr.p_namehash, +sizeof (sd->pptr.p_namehash)); +if (error) +return error; pptr_namelen =xfs_parent_lookup (sc->tp, ip, &sd->pptr, sd->namebuf,MAXNAMELEN , &sd->pptr_scratch);
@@ -216,7 +220,7 @@ xchk_dir_check_pptr_fast(return 0 ; } - error =xchk_dir_parent_pointer (sd, dapos, name, ip); + error =xchk_dir_parent_pointer (sd, name, ip);xfs_iunlock (ip, lockmode);return error; }
@@ -1041,7 +1045,7 @@ xchk_dir_slow_dirent( goto out_unlock; check_pptr: - error =xchk_dir_parent_pointer (sd, dirent->diroffset, &xname, ip); + error =xchk_dir_parent_pointer (sd, &xname, ip); out_unlock:xfs_iunlock (ip, lockmode); out_rele:
fs/xfs/scrub/dir_repair.c +25 -58
244 lines not on this page · raw message
fs/xfs/scrub/parent.c +29 -14
103 lines not on this page · raw message
fs/xfs/scrub/parent_repair.c +12 -15
@@ -95,7 +95,6 @@ struct xrep_pptr {/* Parent pointer attr key. */ xfs_ino_t p_ino;uint32_t p_gen; -xfs_dir2_dataptr_t p_diroffset;/* Length of the pptr name. */ uint8_t namelen;
@@ -183,12 +182,16 @@ xrep_pptr_replay_update( const struct xrep_pptr *pptr) {struct xfs_scrub *sc = rp->sc; +int error; rp->pptr.p_ino = pptr->p_ino; rp->pptr.p_gen = pptr->p_gen; - rp->pptr.p_diroffset = pptr->p_diroffset; rp->pptr.p_namelen = pptr->namelen; + error =xfs_parent_irec_hash (sc->ip, &rp->pptr); +if (error) +return error; +if (pptr->action ==XREP_PPTR_ADD ) {/* Create parent pointer. */ trace_xrep_pptr_createname (sc->tempip, &rp->pptr);
@@ -261,19 +264,17 @@ STATIC intxrep_pptr_add_pointer ( struct xrep_pptrs *rp, const struct xfs_name *name, - const struct xfs_inode *dp, - xfs_dir2_dataptr_t diroffset) + const struct xfs_inode *dp) { struct xrep_pptr pptr = { .action =XREP_PPTR_ADD , .namelen = name->len, .p_ino = dp->i_ino, .p_gen =VFS_IC (dp)->i_generation, - .p_diroffset = diroffset, };int error; -trace_xrep_pptr_add_pointer (rp->sc->tempip, dp, diroffset, name); +trace_xrep_pptr_add_pointer (rp->sc->tempip, dp, name); error =xfblob_store (rp->pptr_names, &pptr.name_cookie, name->name, name->len);
@@ -291,19 +292,17 @@ STATIC intxrep_pptr_remove_pointer ( struct xrep_pptrs *rp, const struct xfs_name *name, - const struct xfs_inode *dp, - xfs_dir2_dataptr_t diroffset) + const struct xfs_inode *dp) { struct xrep_pptr pptr = { .action =XREP_PPTR_REMOVE , .namelen = name->len, .p_ino = dp->i_ino, .p_gen =VFS_IC (dp)->i_generation, - .p_diroffset = diroffset, };int error; -trace_xrep_pptr_remove_pointer (rp->sc->tempip, dp, diroffset, name); +trace_xrep_pptr_remove_pointer (rp->sc->tempip, dp, name); error =xfblob_store (rp->pptr_names, &pptr.name_cookie, name->name, name->len);
@@ -352,7 +351,7 @@ xrep_pptr_scan_dirent(* addition to the temporary file. */ mutex_lock (&rp->lock); - error =xrep_pptr_add_pointer (rp, name, dp, dapos); + error =xrep_pptr_add_pointer (rp, name, dp);mutex_unlock (&rp->lock);return error; }
@@ -646,11 +645,9 @@ xrep_pptr_live_update( xchk_iscan_want_live_update(&rp->iscan, p->dp->i_ino)) {mutex_lock (&rp->lock);if (p->delta >0 ) - error =xrep_pptr_add_pointer (rp, p->name, p->dp, - p->diroffset); + error =xrep_pptr_add_pointer (rp, p->name, p->dp);else - error =xrep_pptr_remove_pointer (rp, p->name, p->dp, - p->diroffset); + error =xrep_pptr_remove_pointer (rp, p->name, p->dp);mutex_unlock (&rp->lock);if (error) goto out_abort;
fs/xfs/scrub/trace.h +16 -32
147 lines not on this page · raw message
fs/xfs/xfs_inode.c +12 -18
@@ -1202,8 +1202,7 @@ xfs_create(* the parent information now. */ if (parent) { - error =xfs_parent_add (tp, parent, dp, name, diroffset, - ip); + error =xfs_parent_add (tp, parent, dp, name, ip);if (error) goto out_trans_cancel; }
@@ -1477,8 +1476,7 @@ xfs_link(* the parent to the inode. */ if (parent) { - error =xfs_parent_add (tp, parent, tdp, target_name, - diroffset, sip); + error =xfs_parent_add (tp, parent, tdp, target_name, sip);if (error) goto error_return; }
@@ -2750,7 +2748,7 @@ xfs_remove( }if (parent) { - error =xfs_parent_remove (tp, dp, parent, dir_offset, ip); + error =xfs_parent_remove (tp, parent, dp, name, ip);if (error) goto out_trans_cancel; }
@@ -3061,13 +3059,13 @@ xfs_cross_rename( }if (xfs_has_parent (mp)) { - error =xfs_parent_replace (tp, ip1_pptr, dp1, - old_diroffset, name2, dp2, new_diroffset, ip1); + error =xfs_parent_replace (tp, ip1_pptr, dp1, name1, dp2, + name2, ip1);if (error) goto out_trans_abort; - error =xfs_parent_replace (tp, ip2_pptr, dp2, - new_diroffset, name1, dp1, old_diroffset, ip2); + error =xfs_parent_replace (tp, ip2_pptr, dp2, name2, dp1, + name1, ip2);if (error) goto out_trans_abort; }
@@ -3540,25 +3538,21 @@ xfs_rename( goto out_trans_cancel;if (wip_pptr) { - error =xfs_parent_add (tp, wip_pptr, - src_dp, src_name, - old_diroffset, wip); + error =xfs_parent_add (tp, wip_pptr, src_dp, src_name, wip);if (error) goto out_trans_cancel; }if (src_ip_pptr) { - error =xfs_parent_replace (tp, src_ip_pptr, src_dp, - old_diroffset, target_name, target_dp, - new_diroffset, src_ip); + error =xfs_parent_replace (tp, src_ip_pptr, src_dp, src_name, + target_dp, target_name, src_ip);if (error) goto out_trans_cancel; }if (tgt_ip_pptr) { - error =xfs_parent_remove (tp, target_dp, - tgt_ip_pptr, - new_diroffset, target_ip); + error =xfs_parent_remove (tp, tgt_ip_pptr, target_dp, + target_name, target_ip);if (error) goto out_trans_cancel; }
fs/xfs/xfs_ondisk.h +4 -0
@@ -6,6 +6,8 @@#ifndef __XFS_ONDISK_H#define __XFS_ONDISK_H +#include <crypto/sha2.h> +#define XFS_CHECK_STRUCT_SIZE (structname, size) \ BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \ #structname ") is wrong, expected " #size)
@@ -114,6 +116,8 @@ xfs_check_ondisk_structs(void)XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, offset,1 );XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, name,3 );XFS_CHECK_STRUCT_SIZE (xfs_dir2_sf_hdr_t,10 ); + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec, 76); +BUILD_BUG_ON (XFS_PARENT_NAME_HASH_SIZE !=SHA512_DIGEST_SIZE );/* log structures */ XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format, 88);
fs/xfs/xfs_parent_utils.c +1 -1
@@ -83,7 +83,7 @@ xfs_getparent_listent( pptr = &ppi->pi_parents[ppi->pi_ptrs_used++]; pptr->xpp_ino = irec->p_ino; pptr->xpp_gen = irec->p_gen; - pptr->xpp_diroffset = irec->p_diroffset; + pptr->xpp_rsvd2 =0 ; pptr->xpp_rsvd =0 ;memcpy (pptr->xpp_name, irec->p_name, irec->p_namelen);
fs/xfs/xfs_sha512.h new file +42 -0
42 lines not on this page · raw message
fs/xfs/xfs_symlink.c +1 -2
@@ -348,8 +348,7 @@ xfs_symlink(xfs_trans_log_inode (tp, dp,XFS_ILOG_CORE );if (parent) { - error =xfs_parent_add (tp, parent, dp, link_name, - diroffset, ip); + error =xfs_parent_add (tp, parent, dp, link_name, ip);if (error) goto out_trans_cancel; }
PATCH 3/5 xfs: skip the sha512 namehash when possible
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Reduce the size and performance impacts of parent pointer name hashes by using the dirent name as the hash if the dirent name is shorter than a sha512 hash would be. IOWs, we only use sha512 for names longer than 63 bytes.
--- fs/xfs/libxfs/xfs_da_format.h | 21 +++++++++- fs/xfs/libxfs/xfs_parent.c | 85 +++++++++++++++++++++++++++-------------- fs/xfs/libxfs/xfs_parent.h | 8 ++-- fs/xfs/scrub/dir.c | 12 ++++-- fs/xfs/scrub/dir_repair.c | 2 - fs/xfs/scrub/parent.c | 16 +++++--- fs/xfs/scrub/parent_repair.c | 2 - fs/xfs/xfs_parent_utils.c | 2 - 8 files changed, 101 insertions(+), 47 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +19 -2
@@ -831,8 +831,11 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,* Parent pointer attribute format definition * * The EA name encodes the parent inode number, generation and a collision -* resistant hash computed from the dirent name. The hash is defined to be the -* sha512 of the child inode generation and the dirent name. +* resistant hash computed from the dirent name. The hash is defined to be: +* +* - The dirent name if it fits within the EA name. +* +* - The sha512 of the child inode generation and the dirent name. * * The EA value contains the same name as the dirent in the parent directory. */
@@ -842,4 +845,18 @@ struct xfs_parent_name_rec {__u8 p_namehash[XFS_PARENT_NAME_HASH_SIZE ]; } __attribute__((packed)); +static inline unsigned int +xfs_parent_name_rec_sizeof ( +unsigned int hashlen) +{ +return offsetof(struct xfs_parent_name_rec , p_namehash) + hashlen; +} + +static inline unsigned int +xfs_parent_name_hashlen ( +unsigned int rec_sizeof) +{ +return rec_sizeof - offsetof(struct xfs_parent_name_rec , p_namehash); +} +#endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_parent.c +56 -29
203 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +5 -3
@@ -23,6 +23,7 @@ struct xfs_parent_name_irec {/* Key fields for looking up a particular parent pointer. */ xfs_ino_t p_ino;uint32_t p_gen; +uint8_t hashlen;uint8_t p_namehash[XFS_PARENT_NAME_HASH_SIZE ];/* Attributes of a parent pointer. */
@@ -31,10 +32,11 @@ struct xfs_parent_name_irec { };void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec, -const struct xfs_parent_name_rec *rec, +const struct xfs_parent_name_rec *rec,int reclen,const void *value,int valuelen); -void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec,void *value, -int *valuelen,const struct xfs_parent_name_irec *irec); +void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec,int *reclen, +void *value,int *valuelen, +const struct xfs_parent_name_irec *irec);/* * Dynamically allocd structure used to wrap the needed data to pass around
fs/xfs/scrub/dir.c +8 -4
@@ -144,15 +144,19 @@ xchk_dir_parent_pointer( {struct xfs_scrub *sc = sd->sc;int pptr_namelen; -int error; +int hashlen; sd->pptr.p_ino = sc->ip->i_ino; sd->pptr.p_gen =VFS_I (sc->ip)->i_generation; - error =xfs_parent_namehash (ip, name, &sd->pptr.p_namehash, + hashlen =xfs_parent_namehash (ip, name, &sd->pptr.p_namehash,sizeof (sd->pptr.p_namehash)); -if (error) -return error; +if (hashlen <0 ) { +xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &hashlen); +return hashlen; + } + sd->pptr.hashlen = hashlen; pptr_namelen =xfs_parent_lookup (sc->tp, ip, &sd->pptr, sd->namebuf,MAXNAMELEN , &sd->pptr_scratch);
fs/xfs/scrub/dir_repair.c +1 -1
@@ -540,7 +540,7 @@ xrep_dir_scan_parent_pointer( !xfs_parent_valuecheck(sc->mp, value, valuelen))return -EFSCORRUPTED ; -xfs_parent_irec_from_disk (&rd->pptr, rec, value, valuelen); +xfs_parent_irec_from_disk (&rd->pptr, rec, namelen, value, valuelen);/* Ignore parent pointers that point back to a different dir. */ if (rd->pptr.p_ino != sc->ip->i_ino ||
fs/xfs/scrub/parent.c +10 -6
@@ -531,6 +531,7 @@ xchk_parent_scan_attr(struct xfs_inode *dp =NULL ;const struct xfs_parent_name_rec *rec = (const void *)name;unsigned int lockmode; +int hashlen;int error;/* Ignore incomplete xattrs */
@@ -552,7 +553,7 @@ xchk_parent_scan_attr(return -ECANCELED ; } -xfs_parent_irec_from_disk (&pp->pptr, rec, value, valuelen); +xfs_parent_irec_from_disk (&pp->pptr, rec, namelen, value, valuelen); xname.name = pp->pptr.p_name; xname.len = pp->pptr.p_namelen;
@@ -561,13 +562,16 @@ xchk_parent_scan_attr(* Does the namehash in the parent pointer match the actual name? * If not, there's no point in checking further. */ - error =xfs_parent_namehash (sc->ip, &xname, pp->child_namehash, + hashlen =xfs_parent_namehash (sc->ip, &xname, pp->child_namehash,sizeof (pp->child_namehash)); -if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) -return error; +if (hashlen <0 ) { +xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &hashlen); +return hashlen; + } -if (memcmp (pp->pptr.p_namehash, pp->child_namehash, -sizeof (pp->pptr.p_namehash))) { +if (hashlen != pp->pptr.hashlen || +memcmp (pp->pptr.p_namehash, pp->child_namehash, + pp->pptr.hashlen)) {trace_xchk_parent_bad_namehash (sc->ip, pp->pptr.p_ino, xname.name, xname.len);xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 );
fs/xfs/scrub/parent_repair.c +1 -1
@@ -522,7 +522,7 @@ xrep_pptr_dump_tempptr(else return -EFSCORRUPTED ; -xfs_parent_irec_from_disk (&rp->pptr, rec, value, valuelen); +xfs_parent_irec_from_disk (&rp->pptr, rec, namelen, value, valuelen);trace_xrep_pptr_dumpname (sc->tempip, &rp->pptr);
fs/xfs/xfs_parent_utils.c +1 -1
@@ -74,7 +74,7 @@ xfs_getparent_listent(return ; } -xfs_parent_irec_from_disk (&gp->pptr_irec, (void *)name, value, +xfs_parent_irec_from_disk (&gp->pptr_irec, (void *)name, namelen, value, valuelen);trace_xfs_getparent_listent (context->dp, ppi, irec);
PATCH 4/5 xfs: make the ondisk parent pointer record a flex array
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that we can use the filename as the parent pointer name hash, we always write the full 64 bytes into the xattr. In other words, the namehash is really a flex array, so adjust its C definition.
--- fs/xfs/libxfs/xfs_da_format.h | 9 ++++++--- fs/xfs/libxfs/xfs_parent.c | 4 ++-- fs/xfs/libxfs/xfs_parent.h | 15 ++++++++++++--- fs/xfs/libxfs/xfs_trans_resv.c | 6 +++--- fs/xfs/xfs_ondisk.h | 2 +- 5 files changed, 24 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +6 -3
@@ -842,21 +842,24 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,struct xfs_parent_name_rec {__be64 p_ino;__be32 p_gen; -__u8 p_namehash[XFS_PARENT_NAME_HASH_SIZE ]; +__u8 p_namehash[]; } __attribute__((packed)); +#define XFS_PARENT_NAME_MAX_SIZE \ + (sizeof(struct xfs_parent_name_rec) + XFS_PARENT_NAME_HASH_SIZE) +static inline unsigned int xfs_parent_name_rec_sizeof (unsigned int hashlen) { -return offsetof(struct xfs_parent_name_rec , p_namehash) + hashlen; +return sizeof (struct xfs_parent_name_rec ) + hashlen; }static inline unsigned int xfs_parent_name_hashlen (unsigned int rec_sizeof) { -return rec_sizeof - offsetof(struct xfs_parent_name_rec , p_namehash); +return rec_sizeof -sizeof (struct xfs_parent_name_rec ); }#endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_parent.c +2 -2
@@ -98,7 +98,7 @@ xfs_init_parent_name_rec( rec->p_ino =cpu_to_be64 (dp->i_ino); rec->p_gen =cpu_to_be32 (VFS_IC (dp)->i_generation);return xfs_parent_namehash (ip, name, rec->p_namehash, -sizeof (rec->p_namehash)); +XFS_PARENT_NAME_HASH_SIZE ); }/*
@@ -197,7 +197,7 @@ __xfs_parent_init( parent->args.attr_filter =XFS_ATTR_PARENT ; parent->args.op_flags =XFS_DA_OP_OKNOENT |XFS_DA_OP_LOGGED ; parent->args.name = (const uint8_t *)&parent->rec; - parent->args.namelen =sizeof (struct xfs_parent_name_rec ); + parent->args.namelen =0 ; *parentp = parent;return 0 ;
fs/xfs/libxfs/xfs_parent.h +12 -3
@@ -43,8 +43,14 @@ void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, int *reclen,* the defer ops machinery */ struct xfs_parent_defer { -struct xfs_parent_name_rec rec; -struct xfs_parent_name_rec old_rec; +union { +struct xfs_parent_name_rec rec; +__u8 dummy1[XFS_PARENT_NAME_MAX_SIZE ]; + }; +union { +struct xfs_parent_name_rec old_rec; +__u8 dummy2[XFS_PARENT_NAME_MAX_SIZE ]; + };struct xfs_da_args args;bool have_log; };
@@ -112,7 +118,10 @@ unsigned int xfs_pptr_calc_space_res(struct xfs_mount *mp,/* Scratchpad memory so that raw parent operations don't burn stack space. */ struct xfs_parent_scratch { -struct xfs_parent_name_rec rec; +union { +struct xfs_parent_name_rec rec; +__u8 dummy1[XFS_PARENT_NAME_MAX_SIZE ]; + };struct xfs_da_args args; };
fs/xfs/libxfs/xfs_trans_resv.c +3 -3
@@ -427,19 +427,19 @@ static inline unsigned int xfs_calc_pptr_link_overhead(void) {return sizeof (struct xfs_attri_log_format ) +xlog_calc_iovec_len (XATTR_NAME_MAX ) + -xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )); +xlog_calc_iovec_len (XFS_PARENT_NAME_MAX_SIZE ); }static inline unsigned int xfs_calc_pptr_unlink_overhead (void ) {return sizeof (struct xfs_attri_log_format ) + -xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )); +xlog_calc_iovec_len (XFS_PARENT_NAME_MAX_SIZE ); }static inline unsigned int xfs_calc_pptr_replace_overhead (void ) {return sizeof (struct xfs_attri_log_format ) +xlog_calc_iovec_len (XATTR_NAME_MAX ) +xlog_calc_iovec_len (XATTR_NAME_MAX ) + -xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )); +xlog_calc_iovec_len (XFS_PARENT_NAME_MAX_SIZE ); }/*
fs/xfs/xfs_ondisk.h +1 -1
@@ -116,7 +116,7 @@ xfs_check_ondisk_structs(void)XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, offset,1 );XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, name,3 );XFS_CHECK_STRUCT_SIZE (xfs_dir2_sf_hdr_t,10 ); - XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec, 76); + XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec, 12);BUILD_BUG_ON (XFS_PARENT_NAME_HASH_SIZE !=SHA512_DIGEST_SIZE );/* log structures */
PATCH 5/5 xfs: use parent pointer xattr space more efficiently
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Amend the parent pointer xattr format even more. Now we put as much of the dirent name in the namehash as we can. For names that don't fit, the namehash is the truncated dirent name with the sha512 of the entire name at the end of the namehash. The EA value is then truncated to whatever doesn't fit in the namehash.
--- fs/xfs/libxfs/xfs_da_format.h | 26 +++++++--- fs/xfs/libxfs/xfs_parent.c | 111 ++++++++++++++++++++++++++++++++--------- fs/xfs/libxfs/xfs_parent.h | 6 +- fs/xfs/scrub/dir_repair.c | 2 - fs/xfs/scrub/parent.c | 4 + fs/xfs/xfs_attr_item.c | 4 + fs/xfs/xfs_ondisk.h | 4 + 7 files changed, 118 insertions(+), 39 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +20 -6
@@ -825,19 +825,24 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp, struct xfs_da3_blkinfo *hdr3);/* We use sha512 for the parent pointer name hash. */ -#define XFS_PARENT_NAME_HASH_SIZE (64) +#define XFS_PARENT_NAME_SHA512_SIZE (64)/* * Parent pointer attribute format definition * * The EA name encodes the parent inode number, generation and a collision -* resistant hash computed from the dirent name. The hash is defined to be: +* resistant hash computed from the dirent name. The hash is defined to be +* one of the following: * -* - The dirent name if it fits within the EA name. +* - The dirent name, as long as it does not use the last possible byte of the +* EA name space. * -* - The sha512 of the child inode generation and the dirent name. +* - The truncated dirent name, with the sha512 hash of the child inode +* generation number and dirent name. The hash is written at the end of the +* EA name. * -* The EA value contains the same name as the dirent in the parent directory. +* The EA value contains however much of the dirent name that does not fit in +* the EA name. */ struct xfs_parent_name_rec {__be64 p_ino;
@@ -845,8 +850,17 @@ struct xfs_parent_name_rec {__u8 p_namehash[]; } __attribute__((packed)); +/* Maximum size of a parent pointer EA name. */ #define XFS_PARENT_NAME_MAX_SIZE \ - (sizeof(struct xfs_parent_name_rec) + XFS_PARENT_NAME_HASH_SIZE) + (MAXNAMELEN - 1) + +/* Maximum size of a parent pointer name hash. */ +#define XFS_PARENT_NAME_MAX_HASH_SIZE \ + (XFS_PARENT_NAME_MAX_SIZE - sizeof(struct xfs_parent_name_rec)) + +/* Offset of the sha512 hash, if used. */ +#define XFS_PARENT_NAME_SHA512_OFFSET \ + (XFS_PARENT_NAME_MAX_HASH_SIZE - XFS_PARENT_NAME_SHA512_SIZE)static inline unsigned int xfs_parent_name_rec_sizeof(
fs/xfs/libxfs/xfs_parent.c +87 -24
238 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +3 -3
@@ -12,8 +12,8 @@ extern struct kmem_cache *xfs_parent_intent_cache;bool xfs_parent_namecheck (struct xfs_mount *mp,const struct xfs_parent_name_rec *rec,size_t reclen,unsigned int attr_flags); -bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value, -size_t valuelen); +bool xfs_parent_valuecheck (struct xfs_mount *mp,size_t namelen, +const void *value,size_t valuelen);/* * Incore version of a parent pointer, also contains dirent name so callers
@@ -24,7 +24,7 @@ struct xfs_parent_name_irec {xfs_ino_t p_ino;uint32_t p_gen;uint8_t hashlen; -uint8_t p_namehash[XFS_PARENT_NAME_HASH_SIZE ]; +uint8_t p_namehash[XFS_PARENT_NAME_MAX_HASH_SIZE ];/* Attributes of a parent pointer. */ uint8_t p_namelen;
fs/xfs/scrub/dir_repair.c +1 -1
@@ -537,7 +537,7 @@ xrep_dir_scan_parent_pointer(/* Does the ondisk parent pointer structure make sense? */ if (!xfs_parent_namecheck (sc->mp, rec, namelen, attr_flags) || - !xfs_parent_valuecheck (sc->mp, value, valuelen)) + !xfs_parent_valuecheck (sc->mp, namelen, value, valuelen))return -EFSCORRUPTED ;xfs_parent_irec_from_disk (&rd->pptr, rec, namelen, value, valuelen);
fs/xfs/scrub/parent.c +2 -2
@@ -350,7 +350,7 @@ struct xchk_pptrs {struct xfs_parent_scratch pptr_scratch;/* Name hashes */ -uint8_t child_namehash[XFS_PARENT_NAME_HASH_SIZE ]; +uint8_t child_namehash[XFS_PARENT_NAME_MAX_HASH_SIZE ];/* Name buffer for revalidation. */ uint8_t namebuf[MAXNAMELEN ];
@@ -548,7 +548,7 @@ xchk_parent_scan_attr(return -ECANCELED ; } -if (!xfs_parent_valuecheck (sc->mp, value, valuelen)) { +if (!xfs_parent_valuecheck (sc->mp, namelen, value, valuelen)) {xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 );return -ECANCELED ; }
fs/xfs/xfs_attr_item.c +2 -2
@@ -769,7 +769,7 @@ xlog_recover_attri_commit_pass2(switch (op) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : -if (item->ri_total !=3 ) { +if (item->ri_total !=3 && item->ri_total !=2 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
@@ -783,7 +783,7 @@ xlog_recover_attri_commit_pass2( }break ;case XFS_ATTRI_OP_FLAGS_NVREPLACE : -if (item->ri_total !=4 ) { +if (item->ri_total !=3 && item->ri_total !=4 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);return -EFSCORRUPTED ;
fs/xfs/xfs_ondisk.h +3 -1
@@ -117,7 +117,9 @@ xfs_check_ondisk_structs(void)XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, name,3 );XFS_CHECK_STRUCT_SIZE (xfs_dir2_sf_hdr_t,10 ); XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec, 12); -BUILD_BUG_ON (XFS_PARENT_NAME_HASH_SIZE !=SHA512_DIGEST_SIZE ); +BUILD_BUG_ON (XFS_PARENT_NAME_MAX_HASH_SIZE <SHA512_DIGEST_SIZE ); +BUILD_BUG_ON (XFS_PARENT_NAME_MAX_HASH_SIZE !=243 ); +BUILD_BUG_ON (XFS_PARENT_NAME_SHA512_OFFSET !=179 );/* log structures */ XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format, 88);
Re: [PATCHSET v9r2d1 0/5] xfs: encode parent pointer name in xattr key
In reply to Darrick J. WongQuoted 68 lines · Darrick J. Wong
Hi all,
As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:
(parent_ino, parent_gen, diroffset) -> (dirent_name)If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.
One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.
This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.
Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.
Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:
(parent_ino, dirent_name) -> NULLThe xattr code supports xattrs with zero-length values, surprisingly. The
parent_genfield makes it easy to export parent handle information, so it can be retained:(parent_ino, parent_gen, dirent_name) -> NULLMoving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:
(parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:
For dirent names shorter than 243 bytes:
(parent_ino, parent_gen, dirent_name) -> NULL
Quoted 9 lines
For dirent names longer than 243 bytes:
(parent_ino, parent_gen, dirent_name[0:178], sha512(child_gen, dirent_name)) -> (dirent_name[179:255])The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.
Naive question:
Obviously, the spec of stradard xattrs does not allow duplicate keys, but dabtree does allow duplicate keys, does it not?
So if you were to allow duplicate parent pointer records, e.g.:
(parent_ino, parent_gen) -> dirent_name1
(parent_ino, parent_gen) -> dirent_name2
Or to optimize performance for the case of large number of sibling hardlinks of the same parent (if that case is worth optimizing):
(parent_ino, parent_gen, dirent_name[0:178]) -> (dirent_name1[179:255]) (parent_ino, parent_gen, dirent_name[0:178]) -> (dirent_name2[179:255])
Then pptr code should have no problem walking all the matching parent pointer records to find the unique parent->child record that it needs to operate on?
I am sure it would be more complicated than how I depicted it, but having to deal with even remote possibility of hash collisions sounds like a massive headache - having to maintain code that is really hard to test and rarely exercised is not a recipe for peace of mind...
Thanks,
Amir.
Re: [PATCHSET v9r2d1 0/5] xfs: encode parent pointer name in xattr key
Quoted 79 lines · Amir Goldstein
Quoted 68 lines · Darrick J. Wong
Hi all,
As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:
(parent_ino, parent_gen, diroffset) -> (dirent_name)If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.
One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.
This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.
Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.
Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:
(parent_ino, dirent_name) -> NULLThe xattr code supports xattrs with zero-length values, surprisingly. The
parent_genfield makes it easy to export parent handle information, so it can be retained:(parent_ino, parent_gen, dirent_name) -> NULLMoving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:
(parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:
For dirent names shorter than 243 bytes:
(parent_ino, parent_gen, dirent_name) -> NULLQuoted 9 lines
For dirent names longer than 243 bytes:
(parent_ino, parent_gen, dirent_name[0:178], sha512(child_gen, dirent_name)) -> (dirent_name[179:255])The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.
Quoted 4 lines
Naive question:
Obviously, the spec of stradard xattrs does not allow duplicate keys, but dabtree does allow duplicate keys, does it not?
The dabtree allows duplicate hashes for a given name, yes. It doesn't allow for duplicate names, though.
(Also note that small xattr structures skip the dabtree and hashing.)
Quoted 8 lines
So if you were to allow duplicate parent pointer records, e.g.:
(
parent_ino,parent_gen) ->dirent_name1
(parent_ino,parent_gen) ->dirent_name2Or to optimize performance for the case of large number of sibling hardlinks of the same parent (if that case is worth optimizing):
Quoted 6 lines
(
parent_ino,parent_gen, dirent_name[0:178]) -> (dirent_name1[179:255]) (parent_ino,parent_gen, dirent_name[0:178]) -> (dirent_name2[179:255])Then pptr code should have no problem walking all the matching parent pointer records to find the unique
parent->childrecord that it needs to operate on?
Theoretically, yes, the parent pointer code could walk all the xattrs that have the same attr name looking for the one with matching value. But keep in mind that there could be min(2^32, 2^(8+76)) potential matches.
The other difficulty is that the xattr lookup and removal code don't have a means to return the dastate to callers or for callers to provide a dastate to go back into the xattr code. You'd need that to identify the specific parent pointer xattr you want to operate on.
Quoted 4 lines
I am sure it would be more complicated than how I depicted it, but having to deal with even remote possibility of hash collisions sounds like a massive headache - having to maintain code that is really hard to test and rarely exercised is not a recipe for peace of mind...
Yep. Hash functions are definitely finger-crossing headhurting.
--D
Quoted 2 lines
Thanks,
Amir.
Re: [PATCHSET v9r2d1 0/5] xfs: encode parent pointer name in xattr key
In reply to Darrick J. WongQuoted 63 lines · Darrick J. Wong
Hi all,
As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:
(parent_ino, parent_gen, diroffset) -> (dirent_name)If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.
One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.
This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.
Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.
Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:
(parent_ino, dirent_name) -> NULLThe xattr code supports xattrs with zero-length values, surprisingly. The
parent_genfield makes it easy to export parent handle information, so it can be retained:(parent_ino, parent_gen, dirent_name) -> NULLMoving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:
(parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:
Quoted 8 lines
For dirent names shorter than 243 bytes:
(parent_ino, parent_gen, dirent_name) -> NULLFor dirent names longer than 243 bytes:
(parent_ino, parent_gen, dirent_name[0:178], sha512(child_gen, dirent_name)) -> (dirent_name[179:255])
Heh, that should've been dirent_name[179:254].
Quoted 8 lines
The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.
A messier solution to this problem would be to extend the xattr ondisk format to allow parent pointers to have xattr names up to 267 bytes. This would likely involve redefining the ondisk namelen field to omit the size of the parent ino/gen information and might be madness.
Update:
After some subtle prodding from Dave, I realized that there's a simpler solution to this problem: extend the xattr match predicate to check both the xattr name /and/ the xattr value. Parent pointers cannot be remote format because the amount of data are never larger than 3/4 of 1FSB, so the value of an ATTR_PARENT attribute is always immediately available.
The pptr ondisk format becomes one of:
(parent_ino, parent_gen, dirent_name) -> NULL
(parent_ino, parent_gen, dirent_name[0:242]) -> (dirent_name[243:254])Matching on both xattr name and value is only useful* for parent pointers, so I introduced a new XFS_DA_OP_VLOOKUP flag to gate this new mode. No more sha512 in the attr name, no more worrying about collision resistance of sha*, and one less dependency for xfs.
The next challenge was to log the VLOOKUP flag when we're doing xattr operations and to recover that state when replaying an ATTRI log item. This I did by creating new "NV" variants of ATTRI_OP_FLAGS_{SET,REMOVE}. The NVSET and NVREMOVE opcodes require VLOOKUP, and NVREMOVE can log an xattr value buffer. Since parent pointers is an unmerged incompat feature, we don't need a new log-incompat feature to protect them.
The last thing was to update NVREPLACE so that we can handle rename operations. Here too we need VLOOKUPs, but I also needed to log both the old value and the new value, so I changed the attri ondisk format. For NVREPLACE, the old and new name lengths are two u16 overlaid atop the alfi_name_len field; and the new value length is a u32 that replaces the old pad (and what Allison called alfi_nname_len).
I'm going to run QA on this over the weekend, and figure out how to collapse this patchset with the new one. I'll rebase the whole branch on pptrs v10 whenever it comes out.
--D
* Unless someone has a usecase for cmpxchg of extended attributes?
Quoted 38 lines
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-keyxfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-keyfstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key--- fs/xfs/Kconfig | 1 fs/xfs/libxfs/xfs_da_format.h | 49 +++++++ fs/xfs/libxfs/xfs_fs.h | 4 - fs/xfs/libxfs/xfs_parent.c | 265 ++++++++++++++++++++++++++++++++-------- fs/xfs/libxfs/xfs_parent.h | 48 +++++-- fs/xfs/libxfs/xfs_trans_resv.c | 6 - fs/xfs/scrub/dir.c | 16 ++ fs/xfs/scrub/dir_repair.c | 87 ++++--------- fs/xfs/scrub/parent.c | 51 +++++--- fs/xfs/scrub/parent_repair.c | 29 ++-- fs/xfs/scrub/trace.h | 48 ++----- fs/xfs/xfs_attr_item.c | 4 - fs/xfs/xfs_inode.c | 30 ++--- fs/xfs/xfs_linux.h | 1 fs/xfs/xfs_mount.c | 13 ++ fs/xfs/xfs_mount.h | 3 fs/xfs/xfs_ondisk.h | 6 + fs/xfs/xfs_parent_utils.c | 4 - fs/xfs/xfs_sha512.h | 42 ++++++ fs/xfs/xfs_super.c | 3 fs/xfs/xfs_symlink.c | 3 21 files changed, 481 insertions(+), 232 deletions(-) create mode 100644 fs/xfs/xfs_sha512.h
cover xfs: remove parent pointer hashing
In reply to Darrick J. WongHi all,
Dave Chinner pointed out (a bit too subtly) that hashing the dirent name to try to squash it into the parent pointer xattr name is unnecessary because we could simply make the xattr matching predicate compare names. Do that instead and drop the hashing.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-vlookup
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-vlookup
--- fs/xfs/Kconfig | 1 fs/xfs/libxfs/xfs_attr.c | 39 +++-- fs/xfs/libxfs/xfs_attr_leaf.c | 41 +++++ fs/xfs/libxfs/xfs_da_btree.h | 6 + fs/xfs/libxfs/xfs_da_format.h | 34 ++--- fs/xfs/libxfs/xfs_log_format.h | 30 +++- fs/xfs/libxfs/xfs_parent.c | 302 +++++++++++++--------------------------- fs/xfs/libxfs/xfs_parent.h | 16 -- fs/xfs/libxfs/xfs_trans_resv.c | 1 fs/xfs/scrub/dir.c | 38 +---- fs/xfs/scrub/parent.c | 61 +------- fs/xfs/scrub/parent_repair.c | 34 +---- fs/xfs/xfs_attr_item.c | 217 ++++++++++++++++++++--------- fs/xfs/xfs_attr_item.h | 3 fs/xfs/xfs_linux.h | 1 fs/xfs/xfs_mount.c | 13 -- fs/xfs/xfs_mount.h | 3 fs/xfs/xfs_ondisk.h | 3 fs/xfs/xfs_sha512.h | 42 ------ fs/xfs/xfs_super.c | 3 fs/xfs/xfs_xattr.c | 5 + 21 files changed, 383 insertions(+), 510 deletions(-) delete mode 100644 fs/xfs/xfs_sha512.h
PATCH 1/13 xfs: make xfs_attr_set require XFS_DA_OP_REMOVE
From: Darrick J. Wong <[email protected]>
In the next patch we're going to add the ability to look up local/sf xattrs based on the attr name and value matching. As a result, we need callers of xfs_attr_set to declare explicitly that they want to remove an xattr. Passing in NULL value will no longer suffice.
--- fs/xfs/libxfs/xfs_attr.c | 9 +++++---- fs/xfs/libxfs/xfs_parent.c | 1 + fs/xfs/xfs_xattr.c | 5 +++++ 3 files changed, 11 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -4
@@ -977,6 +977,7 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; +bool is_remove = args->op_flags &XFS_DA_OP_REMOVE ;bool rsvd;int error, local;int rmt_blks =0 ;
@@ -1004,7 +1005,7 @@ xfs_attr_set( args->op_flags =XFS_DA_OP_OKNOENT | (args->op_flags &XFS_DA_OP_LOGGED ); -if (args->value) { +if (!is_remove) {XFS_STATS_INC (mp, xs_attr_set); args->total =xfs_attr_calc_size (args, &local);
@@ -1038,7 +1039,7 @@ xfs_attr_set(if (error)return error; -if (args->value ||xfs_inode_hasattr (dp)) { +if (!is_remove ||xfs_inode_hasattr (dp)) { error =xfs_iext_count_may_overflow (dp,XFS_ATTR_FORK ,XFS_IEXT_ATTR_MANIP_CNT (rmt_blks));if (error == -EFBIG )
@@ -1052,7 +1053,7 @@ xfs_attr_set(switch (error) {case -EEXIST :/* if no value, we are performing a remove operation */ -if (!args->value) { +if (is_remove) { error =xfs_attr_defer_remove (args);break ; }
@@ -1064,7 +1065,7 @@ xfs_attr_set(break ;case -ENOATTR :/* Can't remove what isn't there. */ -if (!args->value) +if (is_remove) goto out_trans_cancel;/* Pure replace fails if no existing attr to replace. */
fs/xfs/libxfs/xfs_parent.c +1 -0
@@ -460,6 +460,7 @@ xfs_parent_unset( scr->args.geo = ip->i_mount->m_attr_geo; scr->args.name = (const unsigned char *)&scr->rec; scr->args.namelen = reclen; + scr->args.op_flags =XFS_DA_OP_REMOVE ; scr->args.whichfork =XFS_ATTR_FORK ;return xfs_attr_set (&scr->args);
fs/xfs/xfs_xattr.c +5 -0
@@ -103,6 +103,11 @@ xfs_attr_change( use_logging = true; } +if (args->value) + args->op_flags &= ~XFS_DA_OP_REMOVE ; +else + args->op_flags |=XFS_DA_OP_REMOVE ; + error =xfs_attr_set (args);if (use_logging)
PATCH 2/13 xfs: allow xattr matching on value for local/sf attrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
--- fs/xfs/libxfs/xfs_attr_leaf.c | 41 +++++++++++++++++++++++++++++++++++------ fs/xfs/libxfs/xfs_da_btree.h | 4 +++- 2 files changed, 38 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +35 -6
@@ -473,10 +473,12 @@ xfs_attr3_leaf_read(*/ static bool xfs_attr_match ( -struct xfs_da_args *args, -uint8_t namelen, -unsigned char *name, -int flags) +const struct xfs_da_args *args, +uint8_t namelen, +const unsigned char *name, +unsigned int valuelen, +const void *value, +int flags) {if (args->namelen != namelen)
@@ -484,6 +486,23 @@ xfs_attr_match(if (memcmp (args->name, name, namelen) !=0 )return false; +if (args->op_flags &XFS_DA_OP_VLOOKUP ) { +if (args->valuelen != valuelen) +return false; +if (args->valuelen && !value) { +/* not implemented for remote values */ +ASSERT (0 ); +return false; + } +if (valuelen && !args->value) { +/* caller gave us valuelen > 0 but no value?? */ +ASSERT (0 ); +return false; + } +if (valuelen >0 &&memcmp (args->value, value, valuelen) !=0 ) +return false; + } +/* Recovery ignores the INCOMPLETE flag. */ if ((args->op_flags &XFS_DA_OP_RECOVERY ) && args->attr_filter == (flags &XFS_ATTR_NSP_ONDISK_MASK ))
@@ -502,6 +521,10 @@ xfs_attr_copy_value( unsigned char *value, int valuelen) { +/* vlookups already supplied the attr value; don't copy anything */ +if (args->op_flags &XFS_DA_OP_VLOOKUP ) +return 0 ; +/* * No copy if all we have to do is get the length */
@@ -726,6 +749,7 @@ xfs_attr_sf_findname( base += size, i++) { size =xfs_attr_sf_entsize (sfe);if (!xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))continue ;break ;
@@ -896,6 +920,7 @@ xfs_attr_shortform_lookup(xfs_da_args_t *args)for (i =0 ; i < sf->hdr.count; sfe =xfs_attr_sf_nextentry (sfe), i++) {if (xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))return -EEXIST ; }
@@ -923,6 +948,7 @@ xfs_attr_shortform_getvalue(for (i =0 ; i < sf->hdr.count; sfe =xfs_attr_sf_nextentry (sfe), i++) {if (xfs_attr_match (args, sfe->namelen, sfe->nameval, + sfe->valuelen, &sfe->nameval[sfe->namelen], sfe->flags))return xfs_attr_copy_value (args, &sfe->nameval[args->namelen], sfe->valuelen);
@@ -2484,14 +2510,17 @@ xfs_attr3_leaf_lookup_int(if (entry->flags &XFS_ATTR_LOCAL ) { name_loc =xfs_attr3_leaf_name_local (leaf, probe);if (!xfs_attr_match (args, name_loc->namelen, - name_loc->nameval, entry->flags)) + name_loc->nameval, +be16_to_cpu (name_loc->valuelen), + &name_loc->nameval[name_loc->namelen], + entry->flags))continue ; args->index = probe;return -EEXIST ; }else { name_rmt =xfs_attr3_leaf_name_remote (leaf, probe);if (!xfs_attr_match (args, name_rmt->namelen, - name_rmt->name, entry->flags)) + name_rmt->name,0 ,NULL , entry->flags))continue ; args->index = probe; args->rmtvaluelen =be32_to_cpu (name_rmt->valuelen);
fs/xfs/libxfs/xfs_da_btree.h +3 -1
@@ -96,6 +96,7 @@ typedef struct xfs_da_args {#define XFS_DA_OP_REMOVE (1u << 6)/* this is a remove operation */ #define XFS_DA_OP_RECOVERY (1u << 7)/* Log recovery operation */ #define XFS_DA_OP_LOGGED (1u << 8)/* Use intent items to track op */ +#define XFS_DA_OP_VLOOKUP (1u << 9)/* Compare attr value during lookup */ #define XFS_DA_OP_FLAGS \ { XFS_DA_OP_JUSTCHECK, "JUSTCHECK" }, \
@@ -106,7 +107,8 @@ typedef struct xfs_da_args { { XFS_DA_OP_NOTIME, "NOTIME" }, \ { XFS_DA_OP_REMOVE, "REMOVE" }, \ { XFS_DA_OP_RECOVERY, "RECOVERY" }, \ - { XFS_DA_OP_LOGGED, "LOGGED" } + { XFS_DA_OP_LOGGED, "LOGGED" }, \ + { XFS_DA_OP_VLOOKUP, "VLOOKUP" }/* * Storage for holding state during Btree searches and split/join ops.
PATCH 3/13 xfs: preserve VLOOKUP in xfs_attr_set
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Preserve the attr-value lookup flag when calling xfs_attr_set. Normal xattr users will never use this, but parent pointer fsck will.
--- fs/xfs/libxfs/xfs_attr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +4 -4
@@ -999,11 +999,11 @@ xfs_attr_set(/* * We have no control over the attribute names that userspace passes us * to remove, so we have to allow the name lookup prior to attribute -* removal to fail as well. Preserve the logged flag, since we need -* to pass that through to the logging code. +* removal to fail as well. Preserve the logged and vlookup flags, +* since we need to pass them through to the lower levels. */ - args->op_flags =XFS_DA_OP_OKNOENT | - (args->op_flags &XFS_DA_OP_LOGGED ); + args->op_flags &= (XFS_DA_OP_LOGGED |XFS_DA_OP_VLOOKUP ); + args->op_flags |=XFS_DA_OP_OKNOENT ;if (!is_remove) {XFS_STATS_INC (mp, xs_attr_set);
PATCH 4/13 xfs: log VLOOKUP xattr removal operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr remove operation with the VLOOKUP flag set, we need to push this through the log.
--- fs/xfs/libxfs/xfs_attr.c | 6 +++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 7 +++++++ 3 files changed, 13 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
@@ -953,9 +953,13 @@ xfs_attr_defer_remove( {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_REMOVE ;int error; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_REMOVE , &new); +if (args->op_flags &XFS_DA_OP_VLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREMOVE ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -959,6 +959,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REMOVE 2/* Remove the attribute */ #define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_NVREPLACE 4/* Replace attr name and val */ +#define XFS_ATTRI_OP_FLAGS_NVREMOVE 5/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +7 -0
@@ -544,6 +544,7 @@ xfs_attri_validate(case XFS_ATTRI_OP_FLAGS_REPLACE :case XFS_ATTRI_OP_FLAGS_REMOVE :case XFS_ATTRI_OP_FLAGS_NVREPLACE : +case XFS_ATTRI_OP_FLAGS_NVREMOVE :break ;default :return false;
@@ -643,6 +644,11 @@ xfs_attri_item_recover(else attr->xattri_dela_state =xfs_attr_init_add_state (args);break ; +case XFS_ATTRI_OP_FLAGS_NVREMOVE : + args->op_flags |=XFS_DA_OP_VLOOKUP ; + args->value = nv->value.i_addr; + args->valuelen = nv->value.i_len; + fallthrough;case XFS_ATTRI_OP_FLAGS_REMOVE :if (!xfs_inode_hasattr (args->dp)) goto out;
@@ -769,6 +775,7 @@ xlog_recover_attri_commit_pass2(switch (op) {case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : +case XFS_ATTRI_OP_FLAGS_NVREMOVE :if (item->ri_total !=3 && item->ri_total !=2 ) {XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , mp, attri_formatp, len);
PATCH 5/13 xfs: log VLOOKUP xattr setting operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr set operation with the VLOOKUP flag set, we need to push this through the log.
--- fs/xfs/libxfs/xfs_attr.c | 6 +++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 5 +++++ 3 files changed, 11 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
@@ -910,9 +910,13 @@ xfs_attr_defer_add( struct xfs_da_args *args) {struct xfs_attr_intent *new; +int op_flag =XFS_ATTRI_OP_FLAGS_SET ;int error =0 ; - error =xfs_attr_intent_init (args,XFS_ATTRI_OP_FLAGS_SET , &new); +if (args->op_flags &XFS_DA_OP_VLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVSET ; + + error =xfs_attr_intent_init (args, op_flag, &new);if (error)return error;
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -960,6 +960,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_REPLACE 3/* Replace the attribute */ #define XFS_ATTRI_OP_FLAGS_NVREPLACE 4/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 5/* Remove attr w/ vlookup */ +#define XFS_ATTRI_OP_FLAGS_NVSET 6/* Set attr with w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +5 -0
@@ -545,6 +545,7 @@ xfs_attri_validate(case XFS_ATTRI_OP_FLAGS_REMOVE :case XFS_ATTRI_OP_FLAGS_NVREPLACE :case XFS_ATTRI_OP_FLAGS_NVREMOVE : +case XFS_ATTRI_OP_FLAGS_NVSET :break ;default :return false;
@@ -633,6 +634,9 @@ xfs_attri_item_recover(ASSERT (xfs_sb_version_haslogxattrs (&mp->m_sb));switch (attr->xattri_op_flags) { +case XFS_ATTRI_OP_FLAGS_NVSET : + args->op_flags |=XFS_DA_OP_VLOOKUP ; + fallthrough;case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :case XFS_ATTRI_OP_FLAGS_NVREPLACE :
@@ -773,6 +777,7 @@ xlog_recover_attri_commit_pass2( op = attri_formatp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ;switch (op) { +case XFS_ATTRI_OP_FLAGS_NVSET :case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE :case XFS_ATTRI_OP_FLAGS_NVREMOVE :
PATCH 6/13 xfs: refactor extracting attri ops from alfi_op_flags
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Refactor this very long expression into a helper before we start adding more of them.
--- fs/xfs/xfs_attr_item.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
fs/xfs/xfs_attr_item.c +9 -5
@@ -374,6 +374,12 @@ xfs_xattri_finish_update(return error; } +static inline unsigned int +xfs_attr_log_item_op (const struct xfs_attri_log_format *attrp) +{ +return attrp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; +} +/* Log an attr to the intent item. */ STATIC void xfs_attr_log_item(
@@ -525,8 +531,7 @@ xfs_attri_validate(struct xfs_mount *mp, struct xfs_attri_log_format *attrp) { - unsigned int op = attrp->alfi_op_flags & -XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + unsigned int op = xfs_attr_log_item_op(attrp);if (attrp->alfi_op_flags !=XFS_ATTRI_OP_FLAGS_NVREPLACE && attrp->alfi_nname_len !=0 )
@@ -608,8 +613,7 @@ xfs_attri_item_recover( args = (struct xfs_da_args *)(attr +1 ); attr->xattri_da_args = args; - attr->xattri_op_flags = attrp->alfi_op_flags & -XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + attr->xattri_op_flags =xfs_attr_log_item_op (attrp);/* * We're reconstructing the deferred work state structure from the
@@ -775,7 +779,7 @@ xlog_recover_attri_commit_pass2(return -EFSCORRUPTED ; } - op = attri_formatp->alfi_op_flags &XFS_ATTRI_OP_FLAGS_TYPE_MASK ; + op =xfs_attr_log_item_op (attri_formatp);switch (op) {case XFS_ATTRI_OP_FLAGS_NVSET :case XFS_ATTRI_OP_FLAGS_SET :
PATCH 7/13 xfs: overlay alfi_nname_len atop alfi_name_len for NVREPLACE
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
In preparation for being able to log the old attr value in a NVREPLACE operation, encode the old and new name lengths in the alfi_name_len field. We haven't shipped a kernel with XFS_ATTRI_OP_FLAGS_NVREPLACE, so we can still tweak the ondisk log item format.
--- fs/xfs/libxfs/xfs_log_format.h | 14 ++++++- fs/xfs/xfs_attr_item.c | 81 ++++++++++++++++++++++++---------------- 2 files changed, 60 insertions(+), 35 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +12 -2
@@ -979,11 +979,21 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t alfi_nname_len;/* attr new name length */ +uint32_t __pad;/* pad to 64 bit aligned */ uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */ -uint32_t alfi_name_len;/* attr name length */ +union { +uint32_t alfi_name_len;/* attr name length */ +struct { +/* +* For NVREPLACE, these are the lengths of the old and +* new attr name. +*/ +uint16_t alfi_oldname_len; +uint16_t alfi_newname_len; + }; + };uint32_t alfi_value_len;/* attr value length */ uint32_t alfi_attr_filter;/* attr filter flags */ };
fs/xfs/xfs_attr_item.c +48 -33
169 lines not on this page · raw message
PATCH 8/13 xfs: rename nname to newname
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Burn a couple of extra bytes to make it clear what this does.
--- fs/xfs/libxfs/xfs_log_format.h | 2 +- fs/xfs/xfs_attr_item.c | 48 ++++++++++++++++++++-------------------- fs/xfs/xfs_attr_item.h | 2 +- 3 files changed, 26 insertions(+), 26 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +1 -1
@@ -117,7 +117,7 @@ struct xfs_unmount_log_format {#define XLOG_REG_TYPE_ATTRD_FORMAT 28#define XLOG_REG_TYPE_ATTR_NAME 29#define XLOG_REG_TYPE_ATTR_VALUE 30 -#define XLOG_REG_TYPE_ATTR_NNAME 31 +#define XLOG_REG_TYPE_ATTR_NEWNAME 31#define XLOG_REG_TYPE_MAX 31
fs/xfs/xfs_attr_item.c +24 -24
125 lines not on this page · raw message
fs/xfs/xfs_attr_item.h +1 -1
@@ -13,7 +13,7 @@ struct kmem_zone;struct xfs_attri_log_nameval {struct xfs_log_iovec name; -struct xfs_log_iovec nname; +struct xfs_log_iovec newname;struct xfs_log_iovec value;refcount_t refcount;
PATCH 9/13 xfs: log VLOOKUP xattr nvreplace operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If high level code wants to do a deferred xattr nvreplace operation with the VLOOKUP flag set, we need to push this through the log. To avoid breaking the parent pointer code, we'll temporarily create a new NVREPLACEXXX flag that connects to the VLOOKUP flag.
--- fs/xfs/libxfs/xfs_attr.c | 15 +++++++++++---- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/xfs_attr_item.c | 13 +++++++++++-- 3 files changed, 23 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_attr.c +11 -4
@@ -424,7 +424,12 @@ xfs_attr_complete_op( args->op_flags &= ~XFS_DA_OP_REPLACE ;if (do_replace) { args->attr_filter &= ~XFS_ATTR_INCOMPLETE ; -if (args->new_namelen >0 ) { +if (args->op_flags &XFS_DA_OP_VLOOKUP ) { + args->name = args->new_name; + args->namelen = args->new_namelen; + args->hashval =xfs_da_hashname (args->name, + args->namelen); + }else if (args->new_namelen >0 ) { args->name = args->new_name; args->namelen = args->new_namelen; args->hashval = xfs_da_hashname(args->name,
@@ -933,11 +938,13 @@ xfs_attr_defer_replace( struct xfs_da_args *args) {struct xfs_attr_intent *new; -int op_flag; +int op_flag =XFS_ATTRI_OP_FLAGS_REPLACE ;int error =0 ; - op_flag = args->new_namelen ==0 ?XFS_ATTRI_OP_FLAGS_REPLACE : -XFS_ATTRI_OP_FLAGS_NVREPLACE ; +if (args->op_flags &XFS_DA_OP_VLOOKUP ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREPLACEXXX ; +else if (args->new_namelen >0 ) + op_flag =XFS_ATTRI_OP_FLAGS_NVREPLACE ; error =xfs_attr_intent_init (args, op_flag, &new);if (error)
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -961,6 +961,7 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_NVREPLACE 4/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 5/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_NVSET 6/* Set attr with w/ vlookup */ +#define XFS_ATTRI_OP_FLAGS_NVREPLACEXXX 7/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +11 -2
@@ -403,7 +403,10 @@ xfs_attr_log_item( attrp->alfi_op_flags = attr->xattri_op_flags; attrp->alfi_value_len = attr->xattri_nameval->value.i_len; -if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { +if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACEXXX ) { + attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len; + attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len; + }else if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len; attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len; }else {
@@ -564,6 +567,7 @@ xfs_attri_validate(if (attrp->alfi_value_len >XATTR_SIZE_MAX )return false;break ; +case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX :case XFS_ATTRI_OP_FLAGS_NVREPLACE :if (attrp->alfi_oldname_len ==0 || attrp->alfi_oldname_len >XATTR_NAME_MAX )
@@ -649,6 +653,7 @@ xfs_attri_item_recover(switch (attr->xattri_op_flags) {case XFS_ATTRI_OP_FLAGS_NVSET : +case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX : args->op_flags |=XFS_DA_OP_VLOOKUP ; fallthrough;case XFS_ATTRI_OP_FLAGS_SET :
@@ -747,7 +752,10 @@ xfs_attri_item_relog( new_attrp->alfi_ino = old_attrp->alfi_ino; new_attrp->alfi_op_flags = old_attrp->alfi_op_flags; new_attrp->alfi_value_len = old_attrp->alfi_value_len; -if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { +if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACEXXX ) { + new_attrp->alfi_newname_len = old_attrp->alfi_newname_len; + new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len; + }else if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { new_attrp->alfi_newname_len = old_attrp->alfi_newname_len; new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len; }else {
@@ -815,6 +823,7 @@ xlog_recover_attri_commit_pass2( } name_len = attri_formatp->alfi_name_len;break ; +case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX :case XFS_ATTRI_OP_FLAGS_NVREPLACE :if (item->ri_total !=3 && item->ri_total !=4 ) { XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
PATCH 10/13 xfs: log old xattr values for NVREPLACEXXX operations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
For NVREPLACEXXX operations, make it possible to log the old and new attr values, since this variant does VLOOKUP operations.
--- fs/xfs/libxfs/xfs_attr.c | 2 + fs/xfs/libxfs/xfs_da_btree.h | 2 + fs/xfs/libxfs/xfs_log_format.h | 14 +++++--- fs/xfs/xfs_attr_item.c | 74 +++++++++++++++++++++++++++++++++++----- fs/xfs/xfs_attr_item.h | 3 +- 5 files changed, 80 insertions(+), 15 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -0
@@ -429,6 +429,8 @@ xfs_attr_complete_op( args->namelen = args->new_namelen; args->hashval =xfs_da_hashname (args->name, args->namelen); + args->value = args->new_value; + args->valuelen = args->new_valuelen; }else if (args->new_namelen >0 ) { args->name = args->new_name; args->namelen = args->new_namelen;
fs/xfs/libxfs/xfs_da_btree.h +2 -0
@@ -60,7 +60,9 @@ typedef struct xfs_da_args {int new_namelen;/* new attr name len */ uint8_t filetype;/* filetype of inode for directories */ void *value;/* set of bytes (maybe contain NULLs) */ +void *new_value;/* new xattr value (may contain NULLs) */ int valuelen;/* length of value */ +int new_valuelen;/* length of new value */ unsigned int attr_filter;/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */ unsigned int attr_flags;/* XATTR_{CREATE,REPLACE} */ xfs_dahash_t hashval;/* hash value of name */
fs/xfs/libxfs/xfs_log_format.h +10 -4
@@ -115,11 +115,11 @@ struct xfs_unmount_log_format {#define XLOG_REG_TYPE_BUD_FORMAT 26#define XLOG_REG_TYPE_ATTRI_FORMAT 27#define XLOG_REG_TYPE_ATTRD_FORMAT 28 -#define XLOG_REG_TYPE_ATTR_NAME 29 +#define XLOG_REG_TYPE_ATTR_NAME 29#define XLOG_REG_TYPE_ATTR_VALUE 30#define XLOG_REG_TYPE_ATTR_NEWNAME 31 -#define XLOG_REG_TYPE_MAX 31 - +#define XLOG_REG_TYPE_ATTR_NEWVALUE 32 +#define XLOG_REG_TYPE_MAX 32/* * Flags to log operation header
@@ -980,7 +980,13 @@ struct xfs_icreate_log {struct xfs_attri_log_format {uint16_t alfi_type;/* attri log item type */ uint16_t alfi_size;/* size of this item */ -uint32_t __pad;/* pad to 64 bit aligned */ + +/* +* For NVREPLACE, this is the length of the new xattr value. +* alfi_value_len contains the length of the old xattr value. +*/ +uint32_t alfi_newvalue_len; +uint64_t alfi_id;/* attri identifier */ uint64_t alfi_ino;/* the inode for this attr operation */ uint32_t alfi_op_flags;/* marks the op as a set or remove */
fs/xfs/xfs_attr_item.c +64 -10
203 lines not on this page · raw message
fs/xfs/xfs_attr_item.h +2 -1
@@ -13,8 +13,9 @@ struct kmem_zone;struct xfs_attri_log_nameval {struct xfs_log_iovec name; -struct xfs_log_iovec newname; +struct xfs_log_iovec newname;/* NVREPLACE only */ struct xfs_log_iovec value; +struct xfs_log_iovec newvalue;/* NVREPLACE only */ refcount_t refcount;/* name and value follow the end of this struct */
PATCH 11/13 xfs: use VLOOKUP mode to avoid hashing parent pointer names
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Hashing the parent pointer name is fugly because no hashing function can be collision proof. Since we store as much of the dirent name as we can in the xattr name and spill the rest to the xattr value, use VLOOKUP mode so that we can match on name and value. Then we can get rid of the hashing stuff.
--- fs/xfs/libxfs/xfs_da_format.h | 34 ++-- fs/xfs/libxfs/xfs_parent.c | 303 +++++++++++++--------------------------- fs/xfs/libxfs/xfs_parent.h | 16 -- fs/xfs/libxfs/xfs_trans_resv.c | 1 fs/xfs/scrub/dir.c | 38 +---- fs/xfs/scrub/parent.c | 61 +------- fs/xfs/scrub/parent_repair.c | 34 +--- fs/xfs/xfs_ondisk.h | 3 8 files changed, 136 insertions(+), 354 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +12 -22
@@ -824,22 +824,12 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); -/* We use sha512 for the parent pointer name hash. */ -#define XFS_PARENT_NAME_SHA512_SIZE (64) -/* * Parent pointer attribute format definition * -* The EA name encodes the parent inode number, generation and a collision -* resistant hash computed from the dirent name. The hash is defined to be -* one of the following: -* -* - The dirent name, as long as it does not use the last possible byte of the -* EA name space. -* -* - The truncated dirent name, with the sha512 hash of the child inode -* generation number and dirent name. The hash is written at the end of the -* EA name. +* The EA name encodes the parent inode number, generation and as much of the +* dirent name as fits. In other words, it contains up to 243 bytes of the +* dirent name. * * The EA value contains however much of the dirent name that does not fit in * the EA name.
@@ -847,30 +837,30 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,struct xfs_parent_name_rec {__be64 p_ino;__be32 p_gen; -__u8 p_namehash[]; +__u8 p_dname[]; } __attribute__((packed));/* Maximum size of a parent pointer EA name. */ #define XFS_PARENT_NAME_MAX_SIZE \ (MAXNAMELEN - 1) -/* Maximum size of a parent pointer name hash. */ -#define XFS_PARENT_NAME_MAX_HASH_SIZE \ +/* Maximum number of dirent name bytes stored in p_dname. */ +#define XFS_PARENT_MAX_DNAME_SIZE \ (XFS_PARENT_NAME_MAX_SIZE - sizeof(struct xfs_parent_name_rec)) -/* Offset of the sha512 hash, if used. */ -#define XFS_PARENT_NAME_SHA512_OFFSET \ - (XFS_PARENT_NAME_MAX_HASH_SIZE - XFS_PARENT_NAME_SHA512_SIZE) +/* Maximum number of dirent name bytes stored in the xattr value. */ +#define XFS_PARENT_MAX_DNAME_VALUELEN \ + sizeof(struct xfs_parent_name_rec)static inline unsigned int xfs_parent_name_rec_sizeof ( -unsigned int hashlen) +unsigned int dnamelen) { -return sizeof (struct xfs_parent_name_rec ) + hashlen; +return sizeof (struct xfs_parent_name_rec ) + dnamelen; }static inline unsigned int -xfs_parent_name_hashlen ( +xfs_parent_name_dnamelen (unsigned int rec_sizeof) {return rec_sizeof -sizeof (struct xfs_parent_name_rec );
fs/xfs/libxfs/xfs_parent.c +101 -202
469 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +3 -13
@@ -23,10 +23,6 @@ struct xfs_parent_name_irec {/* Key fields for looking up a particular parent pointer. */ xfs_ino_t p_ino;uint32_t p_gen; -uint8_t hashlen; -uint8_t p_namehash[XFS_PARENT_NAME_MAX_HASH_SIZE ]; - -/* Attributes of a parent pointer. */ uint8_t p_namelen;unsigned char p_name[MAXNAMELEN ]; };
@@ -34,7 +30,7 @@ struct xfs_parent_name_irec {void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec,const struct xfs_parent_name_rec *rec,int reclen,const void *value,int valuelen); -void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec,int *reclen, +int xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec,int *reclen,void *value,int *valuelen,const struct xfs_parent_name_irec *irec);
@@ -107,12 +103,6 @@ xfs_parent_finish(__xfs_parent_cancel (mp, p); } -int xfs_parent_namehash (struct xfs_inode *ip,const struct xfs_name *name, -void *namehash,unsigned int namehash_len); - -int xfs_parent_irec_hash (struct xfs_inode *ip, -struct xfs_parent_name_irec *pptr); -unsigned int xfs_pptr_calc_space_res (struct xfs_mount *mp,unsigned int namelen);
@@ -126,8 +116,8 @@ struct xfs_parent_scratch { };int xfs_parent_lookup (struct xfs_trans *tp,struct xfs_inode *ip, -const struct xfs_parent_name_irec *pptr,unsigned char *name, -unsigned int namelen,struct xfs_parent_scratch *scratch); +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scratch); int xfs_parent_set(struct xfs_inode *ip, const struct xfs_parent_name_irec *pptr,
fs/xfs/libxfs/xfs_trans_resv.c +1 -0
@@ -439,6 +439,7 @@ static inline unsigned int xfs_calc_pptr_replace_overhead(void)return sizeof (struct xfs_attri_log_format ) +xlog_calc_iovec_len (XATTR_NAME_MAX ) +xlog_calc_iovec_len (XATTR_NAME_MAX ) + +xlog_calc_iovec_len (XFS_PARENT_NAME_MAX_SIZE ) +xlog_calc_iovec_len (XFS_PARENT_NAME_MAX_SIZE ); }
fs/xfs/scrub/dir.c +6 -32
@@ -78,7 +78,7 @@ struct xchk_dir {/* If we've cycled the ILOCK, we must revalidate deferred dirents. */ bool need_revalidate; -/* Name buffer for pptr validation and dirent revalidation. */ +/* Name buffer for dirent revalidation. */ uint8_t namebuf[MAXNAMELEN ]; };
@@ -143,42 +143,16 @@ xchk_dir_parent_pointer( struct xfs_inode *ip) {struct xfs_scrub *sc = sd->sc; -int pptr_namelen; -int hashlen; +int error; sd->pptr.p_ino = sc->ip->i_ino; sd->pptr.p_gen =VFS_I (sc->ip)->i_generation; + sd->pptr.p_namelen = name->len; +memcpy (sd->pptr.p_name, name->name, name->len); - hashlen =xfs_parent_namehash (ip, name, &sd->pptr.p_namehash, -sizeof (sd->pptr.p_namehash)); -if (hashlen <0 ) { -xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , - &hashlen); -return hashlen; - } - sd->pptr.hashlen = hashlen; - - pptr_namelen =xfs_parent_lookup (sc->tp, ip, &sd->pptr, sd->namebuf, -MAXNAMELEN , &sd->pptr_scratch); -if (pptr_namelen == -ENOATTR ) { -xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); -return 0 ; - } -if (pptr_namelen <0 ) { -xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , - &pptr_namelen); -return pptr_namelen; - } - -if (pptr_namelen != name->len) { -xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); -return 0 ; - } - -if (memcmp (sd->namebuf, name->name, name->len)) { + error =xfs_parent_lookup (sc->tp, ip, &sd->pptr, &sd->pptr_scratch); +if (error == -ENOATTR )xchk_fblock_xref_set_corrupt (sc,XFS_DATA_FORK ,0 ); -return 0 ; - }return 0 ; }
fs/xfs/scrub/parent.c +7 -54
105 lines not on this page · raw message
fs/xfs/scrub/parent_repair.c +6 -28
@@ -126,9 +126,6 @@ struct xrep_pptrs {/* Parent pointer names. */ struct xfblob *pptr_names; - -/* Buffer for validation. */ -unsigned char namebuf[MAXNAMELEN ]; };/* Tear down all the incore stuff we created. */
@@ -182,16 +179,11 @@ xrep_pptr_replay_update( const struct xrep_pptr *pptr) {struct xfs_scrub *sc = rp->sc; -int error; rp->pptr.p_ino = pptr->p_ino; rp->pptr.p_gen = pptr->p_gen; rp->pptr.p_namelen = pptr->namelen; - error =xfs_parent_irec_hash (sc->ip, &rp->pptr); -if (error) -return error; -if (pptr->action ==XREP_PPTR_ADD ) {/* Create parent pointer. */ trace_xrep_pptr_createname (sc->tempip, &rp->pptr);
@@ -510,7 +502,7 @@ xrep_pptr_dump_tempptr(struct xrep_pptrs *rp = priv;const struct xfs_parent_name_rec *rec = (const void *)name;struct xfs_inode *other_ip; -int pptr_namelen; +int error;if (!(attr_flags &XFS_ATTR_PARENT ))return 0 ;
@@ -526,29 +518,15 @@ xrep_pptr_dump_tempptr(trace_xrep_pptr_dumpname (sc->tempip, &rp->pptr); - pptr_namelen =xfs_parent_lookup (sc->tp, other_ip, &rp->pptr, - rp->namebuf,MAXNAMELEN , &rp->pptr_scratch); -if (pptr_namelen == -ENOATTR ) { + error =xfs_parent_lookup (sc->tp, other_ip, &rp->pptr, + &rp->pptr_scratch); +if (error == -ENOATTR ) {trace_xrep_pptr_checkname (other_ip, &rp->pptr); -ASSERT (pptr_namelen != -ENOATTR ); +ASSERT (error != -ENOATTR );return -EFSCORRUPTED ; } -if (pptr_namelen <0 ) -return pptr_namelen; -if (pptr_namelen != rp->pptr.p_namelen) { -trace_xrep_pptr_checkname (other_ip, &rp->pptr); -ASSERT (pptr_namelen == rp->pptr.p_namelen); -return -EFSCORRUPTED ; - } - -if (memcmp (rp->namebuf, rp->pptr.p_name, rp->pptr.p_namelen)) { -trace_xrep_pptr_checkname (other_ip, &rp->pptr); -ASSERT (0 ); -return -EFSCORRUPTED ; - } - -return 0 ; +return error; }/*
fs/xfs/xfs_ondisk.h +0 -3
@@ -117,9 +117,6 @@ xfs_check_ondisk_structs(void)XFS_CHECK_OFFSET (xfs_dir2_sf_entry_t, name,3 );XFS_CHECK_STRUCT_SIZE (xfs_dir2_sf_hdr_t,10 ); XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec, 12); -BUILD_BUG_ON (XFS_PARENT_NAME_MAX_HASH_SIZE <SHA512_DIGEST_SIZE ); -BUILD_BUG_ON (XFS_PARENT_NAME_MAX_HASH_SIZE !=243 ); -BUILD_BUG_ON (XFS_PARENT_NAME_SHA512_OFFSET !=179 );/* log structures */ XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format, 88);
PATCH 12/13 xfs: turn NVREPLACEXXX into NVREPLACE
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
NVREPLACEXXX is NVREPLACE with VLOOKUP enabled. Nobody uses NVREPLACE now, so get rid of NVREPLACE and make NVREPLACEXXX take its place.
--- fs/xfs/libxfs/xfs_attr.c | 7 ------- fs/xfs/libxfs/xfs_log_format.h | 1 - fs/xfs/xfs_attr_item.c | 15 +++------------ 3 files changed, 3 insertions(+), 20 deletions(-)
fs/xfs/libxfs/xfs_attr.c +0 -7
@@ -431,11 +431,6 @@ xfs_attr_complete_op( args->namelen); args->value = args->new_value; args->valuelen = args->new_valuelen; - }else if (args->new_namelen >0 ) { - args->name = args->new_name; - args->namelen = args->new_namelen; - args->hashval =xfs_da_hashname (args->name, - args->namelen); }return replace_state; }
@@ -944,8 +939,6 @@ xfs_attr_defer_replace(int error =0 ;if (args->op_flags &XFS_DA_OP_VLOOKUP ) - op_flag =XFS_ATTRI_OP_FLAGS_NVREPLACEXXX ; -else if (args->new_namelen >0 ) op_flag =XFS_ATTRI_OP_FLAGS_NVREPLACE ; error =xfs_attr_intent_init (args, op_flag, &new);
fs/xfs/libxfs/xfs_log_format.h +0 -1
@@ -961,7 +961,6 @@ struct xfs_icreate_log {#define XFS_ATTRI_OP_FLAGS_NVREPLACE 4/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_NVREMOVE 5/* Remove attr w/ vlookup */ #define XFS_ATTRI_OP_FLAGS_NVSET 6/* Set attr with w/ vlookup */ -#define XFS_ATTRI_OP_FLAGS_NVREPLACEXXX 7/* Replace attr name and val */ #define XFS_ATTRI_OP_FLAGS_TYPE_MASK 0xFF/* Flags type mask */ /*
fs/xfs/xfs_attr_item.c +3 -12
@@ -428,13 +428,10 @@ xfs_attr_log_item( attrp->alfi_op_flags = attr->xattri_op_flags; attrp->alfi_value_len = attr->xattri_nameval->value.i_len; -if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACEXXX ) { +if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len; attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len; attrp->alfi_newvalue_len = attr->xattri_nameval->newvalue.i_len; - }else if (xfs_attr_log_item_op (attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { - attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len; - attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len; }else { attrp->alfi_name_len = attr->xattri_nameval->name.i_len; }
@@ -598,7 +595,6 @@ xfs_attri_validate(if (attrp->alfi_newvalue_len !=0 )return false;break ; -case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX :case XFS_ATTRI_OP_FLAGS_NVREPLACE :if (attrp->alfi_oldname_len ==0 || attrp->alfi_oldname_len >XATTR_NAME_MAX )
@@ -685,7 +681,7 @@ xfs_attri_item_recover(ASSERT (xfs_sb_version_haslogxattrs (&mp->m_sb));switch (attr->xattri_op_flags) { -case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX : +case XFS_ATTRI_OP_FLAGS_NVREPLACE : args->new_value = nv->newvalue.i_addr; args->new_valuelen = nv->newvalue.i_len; fallthrough;
@@ -694,7 +690,6 @@ xfs_attri_item_recover( fallthrough;case XFS_ATTRI_OP_FLAGS_SET :case XFS_ATTRI_OP_FLAGS_REPLACE : -case XFS_ATTRI_OP_FLAGS_NVREPLACE : args->value = nv->value.i_addr; args->valuelen = nv->value.i_len; args->total =xfs_attr_calc_size (args, &local);
@@ -788,13 +783,10 @@ xfs_attri_item_relog( new_attrp->alfi_ino = old_attrp->alfi_ino; new_attrp->alfi_op_flags = old_attrp->alfi_op_flags; new_attrp->alfi_value_len = old_attrp->alfi_value_len; -if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACEXXX ) { +if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { new_attrp->alfi_newname_len = old_attrp->alfi_newname_len; new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len; new_attrp->alfi_newvalue_len = old_attrp->alfi_newvalue_len; - }else if (xfs_attr_log_item_op (old_attrp) ==XFS_ATTRI_OP_FLAGS_NVREPLACE ) { - new_attrp->alfi_newname_len = old_attrp->alfi_newname_len; - new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len; }else { new_attrp->alfi_name_len = old_attrp->alfi_name_len; }
@@ -864,7 +856,6 @@ xlog_recover_attri_commit_pass2( name_len = attri_formatp->alfi_name_len; value_len = attri_formatp->alfi_value_len;break ; -case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX :case XFS_ATTRI_OP_FLAGS_NVREPLACE :if (item->ri_total <3 || item->ri_total >5 ) { XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
PATCH 13/13 xfs: revert "load secure hash algorithm for parent pointers"
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We don't use this anymore, so get rid of it.
--- fs/xfs/Kconfig | 1 - fs/xfs/xfs_linux.h | 1 - fs/xfs/xfs_mount.c | 13 ------------- fs/xfs/xfs_mount.h | 3 --- fs/xfs/xfs_sha512.h | 42 ------------------------------------------ fs/xfs/xfs_super.c | 3 --- 6 files changed, 63 deletions(-) delete mode 100644 fs/xfs/xfs_sha512.h
fs/xfs/Kconfig +0 -1
@@ -5,7 +5,6 @@ config XFS_FS select EXPORTFS select LIBCRC32C select FS_IOMAP - select CRYPTO_SHA512 help XFS is a high performance journaling filesystem which originated on the SGI IRIX platform. It is completely multi-threaded, can
fs/xfs/xfs_linux.h +0 -1
@@ -62,7 +62,6 @@ typedef __u32 xfs_nlink_t;#include <linux/rhashtable.h> #include <linux/xattr.h> #include <linux/mnt_idmapping.h> -#include <crypto/hash.h> #include <asm/page.h> #include <asm/div64.h>
fs/xfs/xfs_mount.c +0 -13
@@ -983,19 +983,6 @@ xfs_mountfs( goto out_agresv; } -if (xfs_has_parent (mp)) { -struct crypto_shash *tfm; - - tfm =crypto_alloc_shash ("sha512" ,0 ,0 ); -if (IS_ERR (tfm)) { - error =PTR_ERR (tfm); - goto out_agresv; - } -xfs_info (mp,"parent pointer hash %s" , -crypto_shash_driver_name (tfm)); - mp->m_sha512 = tfm; - } -return 0 ; out_agresv:
fs/xfs/xfs_mount.h +0 -3
@@ -244,9 +244,6 @@ typedef struct xfs_mount {#endif /* Hook to feed file directory updates to an active online repair. */ struct xfs_hooks m_dirent_update_hooks; - -/* sha512 engine, if needed */ -struct crypto_shash *m_sha512; } xfs_mount_t;#define M_IGEO (mp) (&(mp)->m_ino_geo)
fs/xfs/xfs_sha512.h deleted +0 -42
42 lines not on this page · raw message
fs/xfs/xfs_super.c +0 -3
@@ -738,8 +738,6 @@ xfs_mount_free( {kfree (mp->m_rtname);kfree (mp->m_logname); -if (mp->m_sha512) -crypto_free_shash (mp->m_sha512);kmem_free (mp); }
@@ -1963,7 +1961,6 @@ static int xfs_init_fs_context(if (fc->sb_flags &SB_SYNCHRONOUS ) mp->m_features |=XFS_FEAT_WSYNC ; - mp->m_sha512 =NULL ; fc->s_fs_info = mp; fc->ops = &xfs_context_ops;
cover xfs: use flex arrays for XFS_IOC_GETPARENTS
In reply to Darrick J. WongHi all,
Change the XFS_IOC_GETPARENTS structure definitions so that we can pack parent pointer information more densely, in a manner similar to the attrlistmulti ioctl. This also reduces the amount of memory that has to be copied back to userspace if the buffer doesn't fill up.
If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-ioctl-flexarray
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-ioctl-flexarray
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-ioctl-flexarray
--- fs/xfs/libxfs/xfs_fs.h | 75 ++++++++++++++++++++------------------------- fs/xfs/xfs_ioctl.c | 67 +++++++++++++++++++++++++++++----------- fs/xfs/xfs_ondisk.h | 4 +- fs/xfs/xfs_parent_utils.c | 57 +++++++++++++++++++++------------- fs/xfs/xfs_parent_utils.h | 11 ++++++- fs/xfs/xfs_trace.h | 30 +++++++++--------- 6 files changed, 143 insertions(+), 101 deletions(-)