xfs: parent pointers
Subsystems (2)
PATCH v13 1/1 xfs: parent pointers
Hi everyone,
Christoph asked me to send all the parent pointers patchset to the list so that we can get a head start on reviewing stuff for 6.10.
After -rc1 comes out I'll rebase my development tree and resend the five or so un-reviewed patches from the end of online repair part 1. Depending on how far hch and I get on this stuff, that might make it in time too.
--D
PATCH 1/2 xfs: Parent Pointers
Hi all,
This is the latest parent pointer attributes for xfs. The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.
At this point, Allison is moving on to other things, so I've merged her patchset into djwong-dev for merging.
Updates since v12 [djwong]:
Rebase on 6.5-rc and update the online fsck design document.
Updates since v11 [djwong]:
Rebase on 6.4-rc and make some tweaks and bugfixes to enable the repair prototypes. Merge with djwong-dev and make online repair actually work.
Updates since v10 [djwong]:
Merge in the ondisk format changes to get rid of the diroffset conflicts with the parent pointer repair code, rebase the entire series with the attr vlookup changes first, and merge all the other random fixes.
Updates since v9:
Reordered patches 2 and 3 to be 6 and 7
xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent
xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc reworked ioctl to use pptr listenty and flex array
NEW
xfs: don't remove the attr fork when parent pointers are enabled
NEW
directory lookups should return diroffsets too
NEW
xfs: move/add parent pointer validators to xfs_parent
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=pptrs
---
Commits in this patchset:
* xfs: Expose init_xattrs in xfs_create_tmpfile
* xfs: add parent pointer support to attribute code
* xfs: define parent pointer ondisk extended attribute format
* xfs: add parent pointer validator functions
* xfs: extend transaction reservations for parent attributes
* xfs: parent pointer attribute creation
* xfs: add parent attributes to link
* xfs: add parent attributes to symlink
* xfs: remove parent pointers in unlink
* xfs: Add parent pointers to rename
* xfs: Add parent pointers to xfs_cross_rename
* xfs: Filter XFS_ATTR_PARENT for getfattr
* xfs: pass the attr value to put_listent when possible
* xfs: add a libxfs header file for staging new ioctls
* xfs: Add parent pointer ioctl
* xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
* xfs: drop compatibility minimum log size computations for reflink
* xfs: don't remove the attr fork when parent pointers are enabled
* xfs: Add the parent pointer support to the superblock version 5.
* xfs: only clear some log incompat bits at unmount
* xfs: allow adding multiple log incompat feature bits
* xfs: make XFS_SB_FEAT_INCOMPAT_LOG_XATTRS sticky for parent pointers
* xfs: make XFS_SB_FEAT_INCOMPAT_LOG_EXCHMAPS sticky for parent pointers
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 2 fs/xfs/Kconfig | 11 + fs/xfs/Makefile | 3 fs/xfs/libxfs/xfs_attr.c | 15 + fs/xfs/libxfs/xfs_attr.h | 10 - fs/xfs/libxfs/xfs_attr_leaf.c | 6 fs/xfs/libxfs/xfs_attr_sf.h | 1 fs/xfs/libxfs/xfs_da_format.h | 33 ++ fs/xfs/libxfs/xfs_format.h | 9 fs/xfs/libxfs/xfs_fs.h | 3 fs/xfs/libxfs/xfs_fs_staging.h | 84 +++++ fs/xfs/libxfs/xfs_log_format.h | 1 fs/xfs/libxfs/xfs_log_rlimit.c | 43 ++ fs/xfs/libxfs/xfs_ondisk.h | 4 fs/xfs/libxfs/xfs_parent.c | 368 ++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 155 ++++++++ fs/xfs/libxfs/xfs_sb.c | 4 fs/xfs/libxfs/xfs_trans_resv.c | 326 +++++++++++++++--- fs/xfs/libxfs/xfs_trans_space.c | 121 +++++++ fs/xfs/libxfs/xfs_trans_space.h | 25 + fs/xfs/scrub/agheader.c | 7 fs/xfs/scrub/agheader_repair.c | 4 fs/xfs/scrub/attr.c | 4 fs/xfs/scrub/dir_repair.c | 2 fs/xfs/scrub/orphanage.c | 5 fs/xfs/scrub/parent_repair.c | 3 fs/xfs/scrub/symlink_repair.c | 2 fs/xfs/scrub/tempfile.c | 2 fs/xfs/xfs_attr_item.c | 42 ++ fs/xfs/xfs_attr_list.c | 22 + fs/xfs/xfs_exchrange.c | 2 fs/xfs/xfs_inode.c | 199 +++++++++-- fs/xfs/xfs_inode.h | 2 fs/xfs/xfs_ioctl.c | 150 ++++++++ fs/xfs/xfs_iops.c | 15 + fs/xfs/xfs_linux.h | 1 fs/xfs/xfs_mount.c | 68 +++- fs/xfs/xfs_mount.h | 5 fs/xfs/xfs_parent_utils.c | 164 +++++++++ fs/xfs/xfs_parent_utils.h | 20 + fs/xfs/xfs_super.c | 17 + fs/xfs/xfs_symlink.c | 28 +- fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 76 ++++ fs/xfs/xfs_xattr.c | 8 fs/xfs/xfs_xattr.h | 2 46 files changed, 1918 insertions(+), 157 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_fs_staging.h create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/libxfs/xfs_trans_space.c create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
PATCH 1/23 xfs: Expose init_xattrs in xfs_create_tmpfile
From: Allison Henderson <[email protected]>
Tmp files are used as part of rename operations and will need attr forks initialized for parent pointers. Expose the init_xattrs parameter to the calling function to initialize the fork.
--- fs/xfs/xfs_inode.c | 5 +++-- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +3 -2
@@ -1206,6 +1206,7 @@ xfs_create_tmpfile(struct mnt_idmap *idmap, struct xfs_inode *dp, umode_t mode, + bool init_xattrs, struct xfs_inode **ipp) { struct xfs_mount *mp = dp->i_mount;
@@ -1246,7 +1247,7 @@ xfs_create_tmpfile( error =xfs_dialloc (&tp, dp->i_ino, mode, &ino);if (!error) error =xfs_init_new_inode (idmap, tp, dp, ino, mode, -0 ,0 , prid, false, &ip); +0 ,0 , prid, init_xattrs, &ip);if (error) goto out_trans_cancel;
@@ -3047,7 +3048,7 @@ xfs_rename_alloc_whiteout(int error; error =xfs_create_tmpfile (idmap, dp,S_IFCHR |WHITEOUT_MODE , - &tmpfile); + false, &tmpfile);if (error)return error;
fs/xfs/xfs_inode.h +1 -1
@@ -522,7 +522,7 @@ int xfs_create(struct mnt_idmap *idmap, umode_t mode, dev_t rdev, bool need_xattr, struct xfs_inode **ipp);int xfs_create_tmpfile (struct mnt_idmap *idmap, -struct xfs_inode *dp,umode_t mode, +struct xfs_inode *dp,umode_t mode,bool init_xattrs,struct xfs_inode **ipp);int xfs_remove (struct xfs_inode *dp,struct xfs_name *name,struct xfs_inode *ip);
fs/xfs/xfs_iops.c +1 -1
@@ -202,7 +202,7 @@ xfs_generic_create(xfs_create_need_xattr (dir, default_acl, acl), &ip); }else { - error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, &ip); + error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, false, &ip); }if (unlikely (error)) goto out_free_acl;
PATCH 2/23 xfs: add parent pointer support to attribute code
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.
--- fs/xfs/libxfs/xfs_attr.c | 3 ++- fs/xfs/libxfs/xfs_da_format.h | 5 ++++- fs/xfs/libxfs/xfs_log_format.h | 1 + fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_trace.h | 3 ++- 5 files changed, 10 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -1
@@ -951,7 +951,8 @@ xfs_attr_set(struct xfs_inode *dp = args->dp;struct xfs_mount *mp = dp->i_mount;struct xfs_trans_res tres; -bool rsvd = (args->attr_filter &XFS_ATTR_ROOT ); +bool rsvd = (args->attr_filter & (XFS_ATTR_ROOT | +XFS_ATTR_PARENT ));bool is_remove = args->op_flags &XFS_DA_OP_REMOVE ;int error, local;int rmt_blks =0 ;
fs/xfs/libxfs/xfs_da_format.h +4 -1
@@ -714,12 +714,15 @@ struct xfs_attr3_leafblock {#define XFS_ATTR_LOCAL_BIT 0/* attr is stored locally */ #define XFS_ATTR_ROOT_BIT 1/* limit access to trusted attrs */ #define XFS_ATTR_SECURE_BIT 2/* limit access to secure attrs */ +#define XFS_ATTR_PARENT_BIT 3/* parent pointer attrs */ #define XFS_ATTR_INCOMPLETE_BIT 7/* attr in middle of create/delete */ #define XFS_ATTR_LOCAL (1u << XFS_ATTR_LOCAL_BIT)#define XFS_ATTR_ROOT (1u << XFS_ATTR_ROOT_BIT)#define XFS_ATTR_SECURE (1u << XFS_ATTR_SECURE_BIT) +#define XFS_ATTR_PARENT (1u << XFS_ATTR_PARENT_BIT)#define XFS_ATTR_INCOMPLETE (1u << XFS_ATTR_INCOMPLETE_BIT) -#define XFS_ATTR_NSP_ONDISK_MASK (XFS_ATTR_ROOT | XFS_ATTR_SECURE) +#define XFS_ATTR_NSP_ONDISK_MASK \ + (XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)#define XFS_ATTR_NAMESPACE_STR \ { XFS_ATTR_LOCAL, "local" }, \
fs/xfs/libxfs/xfs_log_format.h +1 -0
@@ -1037,6 +1037,7 @@ struct xfs_icreate_log {*/ #define XFS_ATTRI_FILTER_MASK (XFS_ATTR_ROOT | \ XFS_ATTR_SECURE | \ + XFS_ATTR_PARENT | \ XFS_ATTR_INCOMPLETE)/*
fs/xfs/scrub/attr.c +1 -1
@@ -513,7 +513,7 @@ xchk_xattr_rec(/* Retrieve the entry and check it. */ hash =be32_to_cpu (ent->hashval); badflags = ~(XFS_ATTR_LOCAL |XFS_ATTR_ROOT |XFS_ATTR_SECURE | -XFS_ATTR_INCOMPLETE ); +XFS_ATTR_INCOMPLETE |XFS_ATTR_PARENT );if ((ent->flags & badflags) !=0 )xchk_da_set_corrupt (ds, level);if (ent->flags &XFS_ATTR_LOCAL ) {
fs/xfs/xfs_trace.h +2 -1
@@ -91,7 +91,8 @@ struct xfs_exchrange;#define XFS_ATTR_FILTER_FLAGS \ { XFS_ATTR_ROOT, "ROOT" }, \ { XFS_ATTR_SECURE, "SECURE" }, \ - { XFS_ATTR_INCOMPLETE, "INCOMPLETE" } + { XFS_ATTR_INCOMPLETE, "INCOMPLETE" }, \ + { XFS_ATTR_PARENT, "PARENT" } DECLARE_EVENT_CLASS(xfs_attr_list_class, TP_PROTO(struct xfs_attr_list_context *ctx),
PATCH 3/23 xfs: define parent pointer ondisk extended attribute format
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:
name={parent inode #, parent inode generation, dirent namehash}
value={dirent name}The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls. Storing the dirent name hash in the key reduces hash collisions if a file is hardlinked multiple times in the same directory.
By using the NVLOOKUP mode in the extended attribute code to match parent pointers using both the xattr name and value, we can identify the exact parent pointer EA we need to modify/remove in rename/unlink operations without searching the entire EA space.
By storing the dirent name, we have enough information to be able to validate and reconstruct damaged directory trees. Earlier iterations of this patchset encoded the directory offset in the parent pointer key, but this format required repair to keep that in sync across directory rebuilds, which is unnecessary complexity.
--- fs/xfs/libxfs/xfs_da_format.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +20 -0
@@ -883,4 +883,24 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)xfs_failaddr_t xfs_da3_blkinfo_verify (struct xfs_buf *bp,struct xfs_da3_blkinfo *hdr3); +/* +* Parent pointer attribute format definition +* +* The xattr name encodes the parent inode number, generation and the crc32c +* hash of the dirent name. +* +* The xattr value contains the dirent name. +*/ +struct xfs_parent_name_rec { +__be64 p_ino; +__be32 p_gen; +__be32 p_namehash; +}; + +/* +* Maximum size of the dirent name that can be stored in a parent pointer. +* This matches the maximum dirent name length. +*/ +#define XFS_PARENT_DIRENT_NAME_MAX_SIZE (MAXNAMELEN - 1) +#endif /* __XFS_DA_FORMAT_H__ */
PATCH 4/23 xfs: add parent pointer validator functions
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Attribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set. At the same time, we need to validate attr values during log recovery if the xattr is really a parent pointer.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_attr.c | 10 +++- fs/xfs/libxfs/xfs_attr.h | 3 + fs/xfs/libxfs/xfs_da_format.h | 8 +++ fs/xfs/libxfs/xfs_parent.c | 113 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 19 +++++++ fs/xfs/scrub/attr.c | 2 - fs/xfs/xfs_attr_item.c | 42 ++++++++++++++- fs/xfs/xfs_attr_list.c | 14 +++-- 9 files changed, 200 insertions(+), 12 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h
fs/xfs/Makefile +1 -0
@@ -42,6 +42,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_inode_buf.o \ xfs_log_rlimit.o \ xfs_ag_resv.o \ + xfs_parent.o \ xfs_rmap.o \ xfs_rmap_btree.o \ xfs_refcount.o \
fs/xfs/libxfs/xfs_attr.c +8 -2
@@ -26,6 +26,7 @@#include "xfs_trace.h" #include "xfs_attr_item.h" #include "xfs_xattr.h" +#include "xfs_parent.h" struct kmem_cache *xfs_attr_intent_cache;
@@ -1541,9 +1542,14 @@ xfs_attr_node_get(/* Returns true if the attribute entry name is valid. */ bool xfs_attr_namecheck ( -const void *name, -size_t length) +struct xfs_mount *mp, +const void *name, +size_t length, +unsigned int flags) { +if (flags &XFS_ATTR_PARENT ) +return xfs_parent_namecheck (mp, name, length, flags); +/* * MAXNAMELEN includes the trailing null, but (name/length) leave it * out, so use >= for the length check.
fs/xfs/libxfs/xfs_attr.h +2 -1
@@ -552,7 +552,8 @@ int xfs_attr_get(struct xfs_da_args *args);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr); -bool xfs_attr_namecheck (const void *name,size_t length); +bool xfs_attr_namecheck (struct xfs_mount *mp,const void *name,size_t length, +unsigned int flags);int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total);
fs/xfs/libxfs/xfs_da_format.h +8 -0
@@ -762,6 +762,14 @@ xfs_attr3_leaf_name(xfs_attr_leafblock_t *leafp, int idx)return &((char *)leafp)[be16_to_cpu (entries[idx].nameidx)]; } +static inline int +xfs_attr3_leaf_flags (xfs_attr_leafblock_t *leafp,int idx) +{ +struct xfs_attr_leaf_entry *entries =xfs_attr3_leaf_entryp (leafp); + +return entries[idx].flags; +} +static inline xfs_attr_leaf_name_remote_t *xfs_attr3_leaf_name_remote (xfs_attr_leafblock_t *leafp,int idx) {
fs/xfs/libxfs/xfs_parent.c new file +113 -0
113 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022-2024 Oracle. +* All Rights Reserved. +*/ +#ifndef __XFS_PARENT_H__ +#define __XFS_PARENT_H__ + +/* Metadata validators */ +bool xfs_parent_namecheck (struct xfs_mount *mp, +const struct xfs_parent_name_rec *rec,size_t reclen, +unsigned int attr_flags); +bool xfs_parent_valuecheck (struct xfs_mount *mp,const void *value, +size_t valuelen); +bool xfs_parent_hashcheck (struct xfs_mount *mp, +const struct xfs_parent_name_rec *rec,const void *value, +size_t valuelen); + +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/attr.c +1 -1
@@ -206,7 +206,7 @@ xchk_xattr_actor( }/* Does this name make sense? */ -if (!xfs_attr_namecheck (name, namelen)) { +if (!xfs_attr_namecheck (sc->mp, name, namelen, attr_flags)) {xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno);return -ECANCELED ; }
fs/xfs/xfs_attr_item.c +39 -3
78 lines not on this page · raw message
fs/xfs/xfs_attr_list.c +9 -5
@@ -59,6 +59,7 @@ xfs_attr_shortform_list(struct xfs_attr_sf_sort *sbuf, *sbp;struct xfs_attr_sf_hdr *sf = dp->i_af.if_data;struct xfs_attr_sf_entry *sfe; +struct xfs_mount *mp = dp->i_mount;int sbsize, nsbuf, count, i;int error =0 ;
@@ -82,8 +83,9 @@ xfs_attr_shortform_list( (dp->i_af.if_bytes + sf->count * 16) < context->bufsize)) {for (i =0 , sfe =xfs_attr_sf_firstentry (sf); i < sf->count; i++) {if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sfe->nameval, - sfe->namelen))) { + !xfs_attr_namecheck (mp, sfe->nameval, + sfe->namelen, + sfe->flags))) {xfs_dirattr_mark_sick (context->dp,XFS_ATTR_FORK );return -EFSCORRUPTED ; }
@@ -177,8 +179,9 @@ xfs_attr_shortform_list( cursor->offset =0 ; }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (sbp->name, - sbp->namelen))) { + !xfs_attr_namecheck (mp, sbp->name, + sbp->namelen, + sbp->flags))) {xfs_dirattr_mark_sick (context->dp,XFS_ATTR_FORK ); error = -EFSCORRUPTED ; goto out;
@@ -502,7 +505,8 @@ xfs_attr3_leaf_list_int( }if (XFS_IS_CORRUPT (context->dp->i_mount, - !xfs_attr_namecheck (name, namelen))) { + !xfs_attr_namecheck (mp, name, namelen, + entry->flags))) {xfs_dirattr_mark_sick (context->dp,XFS_ATTR_FORK );return -EFSCORRUPTED ; }
PATCH 5/23 xfs: extend transaction reservations for parent attributes
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.
--- fs/xfs/libxfs/xfs_trans_resv.c | 326 ++++++++++++++++++++++++++++++++++------ 1 file changed, 274 insertions(+), 52 deletions(-)
fs/xfs/libxfs/xfs_trans_resv.c +274 -52
@@ -20,6 +20,9 @@#include "xfs_qm.h" #include "xfs_trans_space.h" #include "xfs_rtbitmap.h" +#include "xfs_attr_item.h" +#include "xfs_log.h" +#include "xfs_da_format.h" #define _ALLOC true#define _FREE false
@@ -422,29 +425,110 @@ xfs_calc_itruncate_reservation_minlogsize(return xfs_calc_itruncate_reservation (mp, true); } +static inline unsigned int xfs_calc_pptr_link_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ); +} +static inline unsigned int xfs_calc_pptr_unlink_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ); +} +static inline unsigned int xfs_calc_pptr_replace_overhead (void ) +{ +return sizeof (struct xfs_attri_log_format ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ) + +xlog_calc_iovec_len (sizeof (struct xfs_parent_name_rec )) + +xlog_calc_iovec_len (XFS_PARENT_DIRENT_NAME_MAX_SIZE ); +} +/* * In renaming a files we can modify: * the five inodes involved: 5 * inode size * the two directory btrees: 2 * (max depth + v2) * dir block size * the two directory bmap btrees: 2 * max depth * block size * And the bmap_finish transaction can free dir and bmap blocks (two sets -* of bmap blocks) giving: +* of bmap blocks) giving (t2): * the agf for the ags in which the blocks live: 3 * sector size * the agfl for the ags in which the blocks live: 3 * sector size * the superblock for the free block count: sector size * the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) * block size +* If parent pointers are enabled (t3), then each transaction in the chain +* must be capable of setting or removing the extended attribute +* containing the parent information. It must also be able to handle +* the three xattr intent items that track the progress of the parent +* pointer update. */ STATIC uintxfs_calc_rename_reservation (struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max ((xfs_calc_inode_res (mp,5 ) + -xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_inode_res (mp,5 ) + +xfs_calc_buf_res (2 *XFS_DIROP_LOG_COUNT (mp), +XFS_FSB_TO_B (mp,1 )); + + t2 =xfs_calc_buf_res (7 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,3 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { +unsigned int rename_overhead, exchange_overhead; + + t3 =max (resp->tr_attrsetm.tr_logres, + resp->tr_attrrm.tr_logres); + +/* +* For a standard rename, the three xattr intent log items +* are (1) replacing the pptr for the source file; (2) +* removing the pptr on the dest file; and (3) adding a +* pptr for the whiteout file in the src dir. +* +* For an RENAME_EXCHANGE, there are two xattr intent +* items to replace the pptr for both src and dest +* files. Link counts don't change and there is no +* whiteout. +* +* In the worst case we can end up relogging all log +* intent items to allow the log tail to move ahead, so +* they become overhead added to each transaction in a +* processing chain. +*/ + rename_overhead =xfs_calc_pptr_replace_overhead () + +xfs_calc_pptr_unlink_overhead () + +xfs_calc_pptr_link_overhead (); + exchange_overhead =2 *xfs_calc_pptr_replace_overhead (); + + overhead +=max (rename_overhead, exchange_overhead); + } + +return overhead +max3 (t1, t2, t3); +} + +static inline unsigned int +xfs_rename_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +/* One for the rename, one more for freeing blocks */ +unsigned int ret =XFS_RENAME_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to remove or add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret +=max (resp->tr_attrsetm.tr_logcount, + resp->tr_attrrm.tr_logcount); + +return ret; }/*
@@ -461,6 +545,23 @@ xfs_calc_iunlink_remove_reservation(2 *M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_link_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_LINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * For creating a link to an inode: * the parent directory inode: inode size
@@ -477,14 +578,23 @@ STATIC uintxfs_calc_link_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_remove_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_remove_reservation (mp); + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (3 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,1 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -499,6 +609,23 @@ xfs_calc_iunlink_add_reservation(xfs_mount_t *mp)M_IGEO (mp)->inode_cluster_size; } +static inline unsigned int +xfs_remove_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_REMOVE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrrm.tr_logcount; + +return ret; +} +/* * For removing a directory entry we can modify: * the parent directory inode: inode size
@@ -515,14 +642,24 @@ STATIC uintxfs_calc_remove_reservation ( struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -xfs_calc_iunlink_add_reservation (mp) + -max ((xfs_calc_inode_res (mp,2 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), -XFS_FSB_TO_B (mp,1 )))); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int t1, t2, t3 =0 ; + + overhead +=xfs_calc_iunlink_add_reservation (mp); + + t1 =xfs_calc_inode_res (mp,2 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); + t2 =xfs_calc_buf_res (4 , mp->m_sb.sb_sectsize) + +xfs_calc_buf_res (xfs_allocfree_block_count (mp,2 ), +XFS_FSB_TO_B (mp,1 )); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrrm.tr_logres; + overhead +=xfs_calc_pptr_unlink_overhead (); + } + +return overhead +max3 (t1, t2, t3); }/*
@@ -571,12 +708,40 @@ xfs_calc_icreate_resv_alloc(xfs_calc_finobt_res (mp); } +static inline unsigned int +xfs_icreate_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_CREATE_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} + STATIC uint -xfs_calc_icreate_reservation (xfs_mount_t *mp) +xfs_calc_icreate_reservation ( +struct xfs_mount *mp) { -return XFS_DQUOT_LOGRES (mp) + -max (xfs_calc_icreate_resv_alloc (mp), -xfs_calc_create_resv_modify (mp)); +struct xfs_trans_resv *resp =M_RES (mp); +unsigned int overhead =XFS_DQUOT_LOGRES (mp); +unsigned int t1, t2, t3 =0 ; + + t1 =xfs_calc_icreate_resv_alloc (mp); + t2 =xfs_calc_create_resv_modify (mp); + +if (xfs_has_parent (mp)) { + t3 = resp->tr_attrsetm.tr_logres; + overhead +=xfs_calc_pptr_link_overhead (); + } + +return overhead +max3 (t1, t2, t3); }STATIC uint
@@ -589,6 +754,23 @@ xfs_calc_create_tmpfile_reservation(return res +xfs_calc_iunlink_add_reservation (mp); } +static inline unsigned int +xfs_mkdir_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_MKDIR_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +} +/* * Making a new directory is the same as creating a new file. */
@@ -599,6 +781,22 @@ xfs_calc_mkdir_reservation(return xfs_calc_icreate_reservation (mp); } +static inline unsigned int +xfs_symlink_log_count ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +unsigned int ret =XFS_SYMLINK_LOG_COUNT ; + +/* +* Pre-reserve enough log reservation to handle the transaction +* rolling needed to add one parent pointer. +*/ +if (xfs_has_parent (mp)) + ret += resp->tr_attrsetm.tr_logcount; + +return ret; +}/* * Making a new symplink is the same as creating a new file, but
@@ -911,6 +1109,52 @@ xfs_calc_sb_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize); } +/* +* Namespace reservations. +* +* These get tricky when parent pointers are enabled as we have attribute +* modifications occurring from within these transactions. Rather than confuse +* each of these reservation calculations with the conditional attribute +* reservations, add them here in a clear and concise manner. This requires that +* the attribute reservations have already been calculated. +* +* Note that we only include the static attribute reservation here; the runtime +* reservation will have to be modified by the size of the attributes being +* added/removed/modified. See the comments on the attribute reservation +* calculations for more details. +*/ +STATIC void +xfs_calc_namespace_reservations ( +struct xfs_mount *mp, +struct xfs_trans_resv *resp) +{ +ASSERT (resp->tr_attrsetm.tr_logres >0 ); + + resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); + resp->tr_rename.tr_logcount =xfs_rename_log_count (mp, resp); + resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); + resp->tr_link.tr_logcount =xfs_link_log_count (mp, resp); + resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); + resp->tr_remove.tr_logcount =xfs_remove_log_count (mp, resp); + resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); + resp->tr_symlink.tr_logcount =xfs_symlink_log_count (mp, resp); + resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); + resp->tr_create.tr_logcount =xfs_icreate_log_count (mp, resp); + resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; + + resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); + resp->tr_mkdir.tr_logcount =xfs_mkdir_log_count (mp, resp); + resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +} + void xfs_trans_resv_calc( struct xfs_mount *mp,
@@ -930,35 +1174,11 @@ xfs_trans_resv_calc( resp->tr_itruncate.tr_logcount =XFS_ITRUNCATE_LOG_COUNT ; resp->tr_itruncate.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_rename.tr_logres =xfs_calc_rename_reservation (mp); - resp->tr_rename.tr_logcount =XFS_RENAME_LOG_COUNT ; - resp->tr_rename.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_link.tr_logres =xfs_calc_link_reservation (mp); - resp->tr_link.tr_logcount =XFS_LINK_LOG_COUNT ; - resp->tr_link.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_remove.tr_logres =xfs_calc_remove_reservation (mp); - resp->tr_remove.tr_logcount =XFS_REMOVE_LOG_COUNT ; - resp->tr_remove.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_symlink.tr_logres =xfs_calc_symlink_reservation (mp); - resp->tr_symlink.tr_logcount =XFS_SYMLINK_LOG_COUNT ; - resp->tr_symlink.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - - resp->tr_create.tr_logres =xfs_calc_icreate_reservation (mp); - resp->tr_create.tr_logcount =XFS_CREATE_LOG_COUNT ; - resp->tr_create.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_create_tmpfile.tr_logres =xfs_calc_create_tmpfile_reservation (mp); resp->tr_create_tmpfile.tr_logcount =XFS_CREATE_TMPFILE_LOG_COUNT ; resp->tr_create_tmpfile.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_mkdir.tr_logres =xfs_calc_mkdir_reservation (mp); - resp->tr_mkdir.tr_logcount =XFS_MKDIR_LOG_COUNT ; - resp->tr_mkdir.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; - resp->tr_ifree.tr_logres =xfs_calc_ifree_reservation (mp); resp->tr_ifree.tr_logcount =XFS_INACTIVE_LOG_COUNT ; resp->tr_ifree.tr_logflags |=XFS_TRANS_PERM_LOG_RES ;
@@ -988,6 +1208,8 @@ xfs_trans_resv_calc( resp->tr_qm_dqalloc.tr_logcount =XFS_WRITE_LOG_COUNT ; resp->tr_qm_dqalloc.tr_logflags |=XFS_TRANS_PERM_LOG_RES ; +xfs_calc_namespace_reservations (mp, resp); +/* * The following transactions are logged in logical format with * a default log count.
PATCH 6/23 xfs: parent pointer attribute creation
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add parent pointer attribute during xfs_create, and subroutines to initialize attributes
[djwong: shorten names, adjust to new format, set init_xattrs for parent pointers]
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_attr.c | 2 - fs/xfs/libxfs/xfs_attr.h | 2 - fs/xfs/libxfs/xfs_parent.c | 91 +++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 77 +++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.c | 52 ++++++++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 9 ++-- fs/xfs/scrub/tempfile.c | 2 - fs/xfs/xfs_inode.c | 30 ++++++++++--- fs/xfs/xfs_iops.c | 15 ++++++ fs/xfs/xfs_super.c | 10 ++++ fs/xfs/xfs_xattr.c | 2 - fs/xfs/xfs_xattr.h | 2 + 13 files changed, 279 insertions(+), 16 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_trans_space.c
fs/xfs/Makefile +1 -0
@@ -51,6 +51,7 @@ xfs-y += $(addprefix libxfs/, \ xfs_symlink_remote.o \ xfs_trans_inode.o \ xfs_trans_resv.o \ + xfs_trans_space.o \ xfs_types.o \ ) # xfs_rtbitmap is shared with libxfs
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -898,7 +898,7 @@ xfs_attr_lookup(return error; } -static void +void xfs_attr_defer_add (struct xfs_da_args *args,unsigned int op_flags)
fs/xfs/libxfs/xfs_attr.h +1 -1
@@ -549,6 +549,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);bool xfs_attr_is_leaf (struct xfs_inode *ip);int xfs_attr_get_ilocked (struct xfs_da_args *args);int xfs_attr_get (struct xfs_da_args *args); +void xfs_attr_defer_add (struct xfs_da_args *args,unsigned int op_flags);int xfs_attr_set (struct xfs_da_args *args);int xfs_attr_set_iter (struct xfs_attr_intent *attr);int xfs_attr_remove_iter (struct xfs_attr_intent *attr);
@@ -557,7 +558,6 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,int xfs_attr_calc_size (struct xfs_da_args *args,int *local);void xfs_init_attr_trans (struct xfs_da_args *args,struct xfs_trans_res *tres,unsigned int *total); -/* * Check to see if the attr should be upgraded from non-existent or shortform to * single-leaf-block attribute list.
fs/xfs/libxfs/xfs_parent.c +91 -0
100 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +76 -1
80 lines not on this page · raw message
fs/xfs/libxfs/xfs_trans_space.c new file +52 -0
52 lines not on this page · raw message
fs/xfs/libxfs/xfs_trans_space.h +5 -4
@@ -80,8 +80,6 @@/* This macro is not used - see inline code in xfs_attr_set */ #define XFS_ATTRSET_SPACE_RES (mp, v) \ (XFS_DAENTER_SPACE_RES(mp, XFS_ATTR_FORK) + XFS_B_TO_FSB(mp, v)) -#define XFS_CREATE_SPACE_RES (mp,nl) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_DIOSTRAT_SPACE_RES (mp, v) \ (XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK) + (v))#define XFS_GROWFS_SPACE_RES (mp) \
@@ -90,8 +88,6 @@ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK))#define XFS_LINK_SPACE_RES (mp,nl) \ XFS_DIRENTER_SPACE_RES(mp,nl) -#define XFS_MKDIR_SPACE_RES (mp,nl) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_QM_DQALLOC_SPACE_RES (mp) \ (XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK) + \ XFS_DQUOT_CLUSTER_SIZE_FSB)
@@ -106,5 +102,10 @@#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0) +unsigned int xfs_parent_calc_space_res (struct xfs_mount *mp, +unsigned int namelen); + +unsigned int xfs_create_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_mkdir_space_res (struct xfs_mount *mp,unsigned int namelen);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/tempfile.c +1 -1
@@ -71,7 +71,7 @@ xrep_tempfile_create(return error;if (is_dir) { - resblks =XFS_MKDIR_SPACE_RES (mp,0 ); + resblks =xfs_mkdir_space_res (mp,0 ); tres = &M_RES (mp)->tr_mkdir; }else { resblks =XFS_IALLOC_SPACE_RES (mp);
fs/xfs/xfs_inode.c +24 -6
@@ -40,6 +40,8 @@#include "xfs_log_priv.h" #include "xfs_health.h" #include "xfs_pnfs.h" +#include "xfs_parent.h" +#include "xfs_xattr.h" struct kmem_cache *xfs_inode_cache;
@@ -1038,7 +1040,7 @@ xfs_dir_hook_setup( int xfs_create( struct mnt_idmap *idmap, - xfs_inode_t *dp, + struct xfs_inode *dp, struct xfs_name *name, umode_t mode, dev_t rdev,
@@ -1050,7 +1052,7 @@ xfs_create(struct xfs_inode *ip =NULL ;struct xfs_trans *tp =NULL ;int error; -bool unlock_dp_on_error = false; +bool unlock_dp_on_error = false;prid_t prid;struct xfs_dquot *udqp =NULL ;struct xfs_dquot *gdqp =NULL ;
@@ -1058,6 +1060,7 @@ xfs_create(struct xfs_trans_res *tres;uint resblks;xfs_ino_t ino; +struct xfs_parent_args *ppargs;trace_xfs_create (dp, name);
@@ -1079,13 +1082,17 @@ xfs_create(return error;if (is_dir) { - resblks =XFS_MKDIR_SPACE_RES (mp, name->len); + resblks =xfs_mkdir_space_res (mp, name->len); tres = &M_RES (mp)->tr_mkdir; }else { - resblks =XFS_CREATE_SPACE_RES (mp, name->len); + resblks =xfs_create_space_res (mp, name->len); tres = &M_RES (mp)->tr_create; } + error =xfs_parent_start (mp, &ppargs); +if (error) + goto out_release_dquots; +/* * Initially assume that the file does not exist and * reserve the resources for that case. If that is not
@@ -1101,7 +1108,7 @@ xfs_create( resblks, &tp); }if (error) - goto out_release_dquots; + goto out_parent;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;
@@ -1144,6 +1151,14 @@ xfs_create(xfs_bumplink (tp, dp); } +/* +* If we have parent pointers, we need to add the attribute containing +* the parent information now. +*/ + error =xfs_parent_add (tp, ppargs, dp, name, ip); +if (error) + goto out_trans_cancel; +/* * Create ip with a reference from dp, and add '.' and '..' references * if it's a directory.
@@ -1176,6 +1191,7 @@ xfs_create( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, ppargs);return 0 ; out_trans_cancel:
@@ -1191,6 +1207,8 @@ xfs_create(xfs_finish_inode_setup (ip);xfs_irele (ip); } + out_parent: +xfs_parent_finish (mp, ppargs); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);
@@ -3048,7 +3066,7 @@ xfs_rename_alloc_whiteout(int error; error =xfs_create_tmpfile (idmap, dp,S_IFCHR |WHITEOUT_MODE , - false, &tmpfile); +xfs_has_parent (dp->i_mount), &tmpfile);if (error)return error;
fs/xfs/xfs_iops.c +14 -1
@@ -158,6 +158,8 @@ xfs_create_need_xattr(if (dir->i_sb->s_security)return true;#endif +if (xfs_has_parent (XFS_I (dir)->i_mount)) +return true;return false; }
@@ -202,7 +204,18 @@ xfs_generic_create(xfs_create_need_xattr (dir, default_acl, acl), &ip); }else { - error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, false, &ip); +bool init_xattrs = false; + +/* +* If this temporary file will be linkable, set up the file +* with an attr fork to receive a parent pointer. +*/ +if (!(tmpfile->f_flags &O_EXCL ) && +xfs_has_parent (XFS_I (dir)->i_mount)) + init_xattrs = true; + + error =xfs_create_tmpfile (idmap,XFS_I (dir), mode, + init_xattrs, &ip); }if (unlikely (error)) goto out_free_acl;
fs/xfs/xfs_super.c +10 -0
@@ -44,6 +44,7 @@#include "xfs_dahash_test.h" #include "xfs_rtbitmap.h" #include "xfs_exchmaps_item.h" +#include "xfs_parent.h" #include "scrub/stats.h" #include "scrub/rcbag_btree.h"
@@ -2211,8 +2212,16 @@ xfs_init_caches(void)if (!xfs_xmi_cache) goto out_destroy_xmd_cache; + xfs_parent_args_cache =kmem_cache_create ("xfs_parent_args" , +sizeof (struct xfs_parent_args ), +0 ,0 ,NULL ); +if (!xfs_parent_args_cache) + goto out_destroy_xmi_cache; +return 0 ; + out_destroy_xmi_cache: +kmem_cache_destroy (xfs_xmi_cache); out_destroy_xmd_cache:kmem_cache_destroy (xfs_xmd_cache); out_destroy_iul_cache:
@@ -2273,6 +2282,7 @@ xfs_destroy_caches(void)* destroy caches. */ rcu_barrier (); +kmem_cache_destroy (xfs_parent_args_cache);kmem_cache_destroy (xfs_xmd_cache);kmem_cache_destroy (xfs_xmi_cache);kmem_cache_destroy (xfs_iunlink_cache);
fs/xfs/xfs_xattr.c +1 -1
@@ -24,7 +24,7 @@* Get permission to use log-assisted atomic exchange of file extents. * Callers must not be running any transactions or hold any ILOCKs. */ -static inline int +int xfs_attr_grab_log_assist (struct xfs_mount *mp) {
fs/xfs/xfs_xattr.h +2 -0
@@ -7,6 +7,8 @@#define __XFS_XATTR_H__int xfs_attr_change (struct xfs_da_args *args); +int xfs_attr_grab_log_assist (struct xfs_mount *mp); +void xfs_attr_rele_log_assist (struct xfs_mount *mp);extern const struct xattr_handler *const xfs_xattr_handlers[];
PATCH 7/23 xfs: add parent attributes to link
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_link to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.c | 14 +++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 3 +-- fs/xfs/scrub/dir_repair.c | 2 +- fs/xfs/scrub/orphanage.c | 2 +- fs/xfs/xfs_inode.c | 41 +++++++++++++++++++++++++++++++++------ 5 files changed, 52 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_trans_space.c +14 -0
@@ -50,3 +50,17 @@ xfs_mkdir_space_res( {return xfs_create_space_res (mp, namelen); } + +unsigned int +xfs_link_space_res ( +struct xfs_mount *mp, +unsigned int namelen) +{ +unsigned int ret; + + ret =XFS_DIRENTER_SPACE_RES (mp, namelen); +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +1 -2
@@ -86,8 +86,6 @@ (2 * (mp)->m_alloc_maxlevels)#define XFS_GROWFSRT_SPACE_RES (mp,b) \ ((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK)) -#define XFS_LINK_SPACE_RES (mp,nl) \ - XFS_DIRENTER_SPACE_RES(mp,nl)#define XFS_QM_DQALLOC_SPACE_RES (mp) \ (XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK) + \ XFS_DQUOT_CLUSTER_SIZE_FSB)
@@ -107,5 +105,6 @@ unsigned int xfs_parent_calc_space_res(struct xfs_mount *mp,unsigned int xfs_create_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_mkdir_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_link_space_res (struct xfs_mount *mp,unsigned int namelen);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/dir_repair.c +1 -1
@@ -705,7 +705,7 @@ xrep_dir_replay_update(uint resblks;int error; - resblks =XFS_LINK_SPACE_RES (mp, dirent->namelen); + resblks =xfs_link_space_res (mp, dirent->namelen); error =xchk_trans_alloc (rd->sc, resblks);if (error)return error;
fs/xfs/scrub/orphanage.c +1 -1
@@ -326,7 +326,7 @@ xrep_adoption_trans_alloc(/* Compute the worst case space reservation that we need. */ adopt->sc = sc; - adopt->orphanage_blkres =XFS_LINK_SPACE_RES (mp,MAXNAMELEN ); + adopt->orphanage_blkres =xfs_link_space_res (mp,MAXNAMELEN );if (S_ISDIR (VFS_I (sc->ip)->i_mode)) child_blkres =XFS_RENAME_SPACE_RES (mp, xfs_name_dotdot.len); adopt->child_blkres = child_blkres;
fs/xfs/xfs_inode.c +35 -6
78 lines not on this page · raw message
PATCH 8/23 xfs: add parent attributes to symlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch modifies xfs_symlink to add a parent pointer to the inode.
--- fs/xfs/libxfs/xfs_trans_space.c | 17 +++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 4 ++-- fs/xfs/scrub/symlink_repair.c | 2 +- fs/xfs/xfs_symlink.c | 28 +++++++++++++++++++++++----- 4 files changed, 43 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_trans_space.c +17 -0
@@ -64,3 +64,20 @@ xfs_link_space_res(return ret; } + +unsigned int +xfs_symlink_space_res ( +struct xfs_mount *mp, +unsigned int namelen, +unsigned int fsblocks) +{ +unsigned int ret; + + ret =XFS_IALLOC_SPACE_RES (mp) +XFS_DIRENTER_SPACE_RES (mp, namelen) + + fsblocks; + +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +2 -2
@@ -95,8 +95,6 @@ XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl)) -#define XFS_SYMLINK_SPACE_RES (mp,nl,b) \ - (XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl) + (b))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
@@ -106,5 +104,7 @@ unsigned int xfs_parent_calc_space_res(struct xfs_mount *mp,unsigned int xfs_create_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_mkdir_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_link_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_symlink_space_res (struct xfs_mount *mp,unsigned int namelen, +unsigned int fsblocks);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/symlink_repair.c +1 -1
@@ -419,7 +419,7 @@ xrep_symlink_rebuild(* unlikely. */ fs_blocks =xfs_symlink_blocks (sc->mp, target_len); - resblks =XFS_SYMLINK_SPACE_RES (sc->mp, target_len, fs_blocks); + resblks =xfs_symlink_space_res (sc->mp, target_len, fs_blocks); error =xfs_trans_reserve_quota_nblks (sc->tp, sc->tempip, resblks,0 , true);if (error)
fs/xfs/xfs_symlink.c +23 -5
@@ -25,6 +25,8 @@#include "xfs_error.h" #include "xfs_health.h" #include "xfs_symlink_remote.h" +#include "xfs_parent.h" +#include "xfs_defer.h" /* ----- Kernel only functions below ----- */ int
@@ -101,6 +103,7 @@ xfs_symlink(struct xfs_dquot *pdqp =NULL ;uint resblks;xfs_ino_t ino; +struct xfs_parent_args *ppargs; *ipp =NULL ;
@@ -131,18 +134,24 @@ xfs_symlink(/* * The symlink will fit into the inode data fork? -* There can't be any attributes so we get the whole variable part. +* If there are no parent pointers, then there wont't be any attributes. +* So we get the whole variable part, and do not need to reserve extra +* blocks. Otherwise, we need to reserve the blocks. */ -if (pathlen <=XFS_LITINO (mp)) +if (pathlen <=XFS_LITINO (mp) && !xfs_has_parent (mp)) fs_blocks =0 ;else fs_blocks =xfs_symlink_blocks (mp, pathlen); - resblks =XFS_SYMLINK_SPACE_RES (mp, link_name->len, fs_blocks); + resblks =xfs_symlink_space_res (mp, link_name->len, fs_blocks); + + error =xfs_parent_start (mp, &ppargs); +if (error) + goto out_release_dquots; error =xfs_trans_alloc_icreate (mp, &M_RES (mp)->tr_symlink, udqp, gdqp, pdqp, resblks, &tp);if (error) - goto out_release_dquots; + goto out_parent;xfs_ilock (dp,XFS_ILOCK_EXCL |XFS_ILOCK_PARENT ); unlock_dp_on_error = true;
@@ -162,7 +171,7 @@ xfs_symlink(if (!error) error =xfs_init_new_inode (idmap, tp, dp, ino,S_IFLNK | (mode & ~S_IFMT ),1 ,0 , prid, - false, &ip); +xfs_has_parent (mp), &ip);if (error) goto out_trans_cancel;
@@ -196,6 +205,12 @@ xfs_symlink( goto out_trans_cancel;xfs_trans_ichgtime (tp, dp,XFS_ICHGTIME_MOD |XFS_ICHGTIME_CHG );xfs_trans_log_inode (tp, dp,XFS_ILOG_CORE ); + +/* Add parent pointer for the new symlink. */ + error =xfs_parent_add (tp, ppargs, dp, link_name, ip); +if (error) + goto out_trans_cancel; +xfs_dir_update_hook (dp, ip,1 , link_name);/*
@@ -217,6 +232,7 @@ xfs_symlink( *ipp = ip;xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, ppargs);return 0 ; out_trans_cancel:
@@ -232,6 +248,8 @@ xfs_symlink(xfs_finish_inode_setup (ip);xfs_irele (ip); } +out_parent: +xfs_parent_finish (mp, ppargs); out_release_dquots:xfs_qm_dqrele (udqp);xfs_qm_dqrele (gdqp);
PATCH 9/23 xfs: remove parent pointers in unlink
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the parent pointer attribute during unlink
--- fs/xfs/libxfs/xfs_parent.c | 39 +++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 16 ++++++++++++++++ fs/xfs/libxfs/xfs_trans_space.c | 13 +++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 3 +-- fs/xfs/xfs_inode.c | 25 +++++++++++++++++++------ 5 files changed, 88 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_parent.c +39 -0
@@ -27,6 +27,7 @@#include "xfs_xattr.h" #include "xfs_parent.h" #include "xfs_trans_space.h" +#include "xfs_health.h" struct kmem_cache *xfs_parent_args_cache;
@@ -194,6 +195,44 @@ xfs_parent_addname(return 0 ; } +/* Remove a parent pointer to reflect a dirent removal. */ +int +xfs_parent_removename ( +struct xfs_trans *tp, +struct xfs_parent_args *ppargs, +struct xfs_inode *dp, +const struct xfs_name *parent_name, +struct xfs_inode *child) +{ +struct xfs_da_args *args = &ppargs->args; + +if (XFS_IS_CORRUPT (tp->t_mountp, + !xfs_parent_valuecheck (tp->t_mountp, parent_name->name, + parent_name->len))) +return -EFSCORRUPTED ; + +/* +* For regular attrs, removing an attr from a !hasattr inode is a nop. +* For parent pointers, we require that the pointer must exist if the +* caller wants us to remove the pointer. +*/ +if (XFS_IS_CORRUPT (child->i_mount, !xfs_inode_hasattr (child))) { +xfs_inode_mark_sick (child,XFS_SICK_INO_PARENT ); +return -EFSCORRUPTED ; + } + +xfs_init_parent_name_rec (&ppargs->rec, dp, parent_name, child); + args->hashval =xfs_parent_hashname (dp, ppargs); + + args->trans = tp; + args->dp = child; + +xfs_init_parent_davalue (&ppargs->args, parent_name); + +xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_REMOVE ); +return 0 ; +} +/* Free a parent pointer context object. */ void xfs_parent_args_free(
fs/xfs/libxfs/xfs_parent.h +16 -0
@@ -79,6 +79,22 @@ xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_args *ppargs,return 0 ; } +int xfs_parent_removename (struct xfs_trans *tp,struct xfs_parent_args *ppargs, +struct xfs_inode *dp,const struct xfs_name *parent_name, +struct xfs_inode *child); + +/* Schedule a parent pointer removal. */ +static inline int +xfs_parent_remove (struct xfs_trans *tp,struct xfs_parent_args *ppargs, +struct xfs_inode *dp,const struct xfs_name *parent_name, +struct xfs_inode *child) +{ +if (ppargs) +return xfs_parent_removename (tp, ppargs, dp, parent_name, + child); +return 0 ; +} +void xfs_parent_args_free (struct xfs_mount *mp,struct xfs_parent_args *ppargs);/* Finish a parent pointer update by freeing the context object. */
fs/xfs/libxfs/xfs_trans_space.c +13 -0
@@ -81,3 +81,16 @@ xfs_symlink_space_res(return ret; } + +unsigned int +xfs_remove_space_res ( +struct xfs_mount *mp, +unsigned int namelen) +{ +unsigned int ret =XFS_DIRREMOVE_SPACE_RES (mp); + +if (xfs_has_parent (mp)) + ret +=xfs_parent_calc_space_res (mp, namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +1 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_REMOVE_SPACE_RES (mp) \ - XFS_DIRREMOVE_SPACE_RES(mp)#define XFS_RENAME_SPACE_RES (mp,nl) \ (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \
@@ -106,5 +104,6 @@ unsigned int xfs_mkdir_space_res(struct xfs_mount *mp, unsigned int namelen);unsigned int xfs_link_space_res (struct xfs_mount *mp,unsigned int namelen);unsigned int xfs_symlink_space_res (struct xfs_mount *mp,unsigned int namelen,unsigned int fsblocks); +unsigned int xfs_remove_space_res (struct xfs_mount *mp,unsigned int namelen);#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/xfs_inode.c +19 -6
@@ -2727,16 +2727,17 @@ xfs_iunpin_wait(*/ int xfs_remove ( -xfs_inode_t *dp, +struct xfs_inode *dp,struct xfs_name *name, -xfs_inode_t *ip) +struct xfs_inode *ip) { -xfs_mount_t *mp = dp->i_mount; -xfs_trans_t *tp =NULL ; +struct xfs_mount *mp = dp->i_mount; +struct xfs_trans *tp =NULL ;int is_dir =S_ISDIR (VFS_I (ip)->i_mode);int dontcare;int error =0 ;uint resblks; +struct xfs_parent_args *ppargs;trace_xfs_remove (dp, name);
@@ -2753,6 +2754,10 @@ xfs_remove(if (error) goto std_return; + error =xfs_parent_start (mp, &ppargs); +if (error) + goto std_return; +/* * We try to get the real space reservation first, allowing for * directory btree deletion(s) implying possible bmap insert(s). If we
@@ -2764,12 +2769,12 @@ xfs_remove(* the directory code can handle a reservationless update and we don't * want to prevent a user from trying to free space by deleting things. */ - resblks =XFS_REMOVE_SPACE_RES (mp); + resblks =xfs_remove_space_res (mp, name->len); error =xfs_trans_alloc_dir (dp, &M_RES (mp)->tr_remove, ip, &resblks, &tp, &dontcare);if (error) {ASSERT (error != -ENOSPC ); - goto std_return; + goto out_parent; }/*
@@ -2829,6 +2834,11 @@ xfs_remove( goto out_trans_cancel; } +/* Remove parent pointer. */ + error =xfs_parent_remove (tp, ppargs, dp, name, ip); +if (error) + goto out_trans_cancel; +/* * Drop the link from dp to ip, and if ip was a directory, remove the * '.' and '..' references since we freed the directory.
@@ -2852,6 +2862,7 @@ xfs_remove(xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); +xfs_parent_finish (mp, ppargs);return 0 ; out_trans_cancel:
@@ -2859,6 +2870,8 @@ xfs_remove( out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );xfs_iunlock (dp,XFS_ILOCK_EXCL ); + out_parent: +xfs_parent_finish (mp, ppargs); std_return:return error; }
PATCH 10/23 xfs: Add parent pointers to rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer.
--- fs/xfs/libxfs/xfs_parent.c | 63 ++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 20 +++++++++++ fs/xfs/libxfs/xfs_trans_space.c | 25 +++++++++++++ fs/xfs/libxfs/xfs_trans_space.h | 6 ++- fs/xfs/scrub/orphanage.c | 3 +- fs/xfs/scrub/parent_repair.c | 3 +- fs/xfs/xfs_inode.c | 73 +++++++++++++++++++++++++++++++++++---- 7 files changed, 182 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_parent.c +63 -0
@@ -138,6 +138,19 @@ xfs_init_parent_davalue( args->value = (void *)name->name; } +/* +* Point the da args new value fields at the non-key parts of a replacement +* parent pointer. +*/ +static inline void +xfs_init_parent_danewvalue ( +struct xfs_da_args *args, +const struct xfs_name *name) +{ + args->new_valuelen = name->len; + args->new_value = (void *)name->name; +} +/* * Allocate memory to control a logged parent pointer update as part of a * dirent operation.
@@ -233,6 +246,56 @@ xfs_parent_removename(return 0 ; } +/* Replace one parent pointer with another to reflect a rename. */ +int +xfs_parent_replacename ( +struct xfs_trans *tp, +struct xfs_parent_args *ppargs, +struct xfs_inode *old_dp, +const struct xfs_name *old_name, +struct xfs_inode *new_dp, +const struct xfs_name *new_name, +struct xfs_inode *child) +{ +struct xfs_da_args *args = &ppargs->args; + +if (XFS_IS_CORRUPT (tp->t_mountp, + !xfs_parent_valuecheck (tp->t_mountp, old_name->name, + old_name->len))) +return -EFSCORRUPTED ; + +if (XFS_IS_CORRUPT (tp->t_mountp, + !xfs_parent_valuecheck (tp->t_mountp, new_name->name, + new_name->len))) +return -EFSCORRUPTED ; + +/* +* For regular attrs, replacing an attr from a !hasattr inode becomes +* an attr-set operation. For replacing a parent pointer, however, we +* require that the old pointer must exist. +*/ +if (XFS_IS_CORRUPT (child->i_mount, !xfs_inode_hasattr (child))) { +xfs_inode_mark_sick (child,XFS_SICK_INO_PARENT ); +return -EFSCORRUPTED ; + } + +xfs_init_parent_name_rec (&ppargs->rec, old_dp, old_name, child); + args->hashval =xfs_parent_hashname (old_dp, ppargs); + +xfs_init_parent_name_rec (&ppargs->new_rec, new_dp, new_name, child); + args->new_name = (const uint8_t *)&ppargs->new_rec; + args->new_namelen =sizeof (struct xfs_parent_name_rec ); + + args->trans = tp; + args->dp = child; + +xfs_init_parent_davalue (&ppargs->args, old_name); +xfs_init_parent_danewvalue (&ppargs->args, new_name); + +xfs_attr_defer_add (args,XFS_ATTRI_OP_FLAGS_REPLACE ); +return 0 ; +} +/* Free a parent pointer context object. */ void xfs_parent_args_free(
fs/xfs/libxfs/xfs_parent.h +20 -0
@@ -24,6 +24,7 @@ extern struct kmem_cache *xfs_parent_args_cache;*/ struct xfs_parent_args {struct xfs_parent_name_rec rec; +struct xfs_parent_name_rec new_rec;struct xfs_da_args args; };
@@ -95,6 +96,25 @@ xfs_parent_remove(struct xfs_trans *tp, struct xfs_parent_args *ppargs,return 0 ; } +int xfs_parent_replacename (struct xfs_trans *tp, +struct xfs_parent_args *ppargs, +struct xfs_inode *old_dp,const struct xfs_name *old_name, +struct xfs_inode *new_dp,const struct xfs_name *new_name, +struct xfs_inode *child); + +/* Schedule a parent pointer replacement. */ +static inline int +xfs_parent_replace (struct xfs_trans *tp,struct xfs_parent_args *ppargs, +struct xfs_inode *old_dp,const struct xfs_name *old_name, +struct xfs_inode *new_dp,const struct xfs_name *new_name, +struct xfs_inode *child) +{ +if (ppargs) +return xfs_parent_replacename (tp, ppargs, old_dp, old_name, + new_dp, new_name, child); +return 0 ; +} +void xfs_parent_args_free (struct xfs_mount *mp,struct xfs_parent_args *ppargs);/* Finish a parent pointer update by freeing the context object. */
fs/xfs/libxfs/xfs_trans_space.c +25 -0
@@ -94,3 +94,28 @@ xfs_remove_space_res(return ret; } + +unsigned int +xfs_rename_space_res ( +struct xfs_mount *mp, +unsigned int src_namelen, +bool target_exists, +unsigned int target_namelen, +bool has_whiteout) +{ +unsigned int ret; + + ret =XFS_DIRREMOVE_SPACE_RES (mp) + +XFS_DIRENTER_SPACE_RES (mp, target_namelen); + +if (xfs_has_parent (mp)) { +if (has_whiteout) + ret +=xfs_parent_calc_space_res (mp, src_namelen); + ret +=2 *xfs_parent_calc_space_res (mp, target_namelen); + } + +if (target_exists) + ret +=xfs_parent_calc_space_res (mp, target_namelen); + +return ret; +}
fs/xfs/libxfs/xfs_trans_space.h +4 -2
@@ -91,8 +91,6 @@ XFS_DQUOT_CLUSTER_SIZE_FSB)#define XFS_QM_QINOCREATE_SPACE_RES (mp) \ XFS_IALLOC_SPACE_RES(mp) -#define XFS_RENAME_SPACE_RES (mp,nl) \ - (XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))#define XFS_IFREE_SPACE_RES (mp) \ (xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
@@ -106,4 +104,8 @@ unsigned int xfs_symlink_space_res(struct xfs_mount *mp, unsigned int namelen, unsigned int fsblocks);unsigned int xfs_remove_space_res (struct xfs_mount *mp,unsigned int namelen); +unsigned int xfs_rename_space_res (struct xfs_mount *mp, +unsigned int src_namelen,bool target_exists, +unsigned int target_namelen,bool has_whiteout); +#endif /* __XFS_TRANS_SPACE_H__ */
fs/xfs/scrub/orphanage.c +2 -1
@@ -328,7 +328,8 @@ xrep_adoption_trans_alloc( adopt->sc = sc; adopt->orphanage_blkres =xfs_link_space_res (mp,MAXNAMELEN );if (S_ISDIR (VFS_I (sc->ip)->i_mode)) - child_blkres =XFS_RENAME_SPACE_RES (mp, xfs_name_dotdot.len); + child_blkres =xfs_rename_space_res (mp,0 , false, + xfs_name_dotdot.len, false); adopt->child_blkres = child_blkres;/*
fs/xfs/scrub/parent_repair.c +2 -1
@@ -169,7 +169,8 @@ xrep_parent_reset_dotdot(* Reserve more space just in case we have to expand the dir. We're * allowed to exceed quota to repair inconsistent metadata. */ - spaceres =XFS_RENAME_SPACE_RES (sc->mp, xfs_name_dotdot.len); + spaceres =xfs_rename_space_res (sc->mp,0 , false, xfs_name_dotdot.len, + false); error =xfs_trans_reserve_more_inode (sc->tp, sc->ip, spaceres,0 , true);if (error)
fs/xfs/xfs_inode.c +66 -7
121 lines not on this page · raw message
PATCH 11/23 xfs: Add parent pointers to xfs_cross_rename
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Cross renames are handled separately from standard renames, and need different handling to update the parent attributes correctly.
--- fs/xfs/xfs_inode.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-)
fs/xfs/xfs_inode.c +19 -8
@@ -2976,15 +2976,17 @@ xfs_cross_rename(struct xfs_inode *dp1, struct xfs_name *name1, struct xfs_inode *ip1, + struct xfs_parent_args *ip1_ppargs, struct xfs_inode *dp2, struct xfs_name *name2, struct xfs_inode *ip2, + struct xfs_parent_args *ip2_ppargs, int spaceres) { - int error = 0; -int ip1_flags =0 ; -int ip2_flags =0 ; -int dp2_flags =0 ; + int error = 0; +int ip1_flags =0 ; +int ip2_flags =0 ; +int dp2_flags =0 ;/* Swap inode number for dirent in first parent */ error =xfs_dir_replace (tp, dp1, name1, ip2->i_ino, spaceres);
@@ -3053,6 +3055,15 @@ xfs_cross_rename( } } +/* Schedule parent pointer replacements */ + error =xfs_parent_replace (tp, ip1_ppargs, dp1, name1, dp2, name2, ip1); +if (error) + goto out_trans_abort; + + error =xfs_parent_replace (tp, ip2_ppargs, dp2, name2, dp1, name1, ip2); +if (error) + goto out_trans_abort; +if (ip1_flags) {xfs_trans_ichgtime (tp, ip1, ip1_flags);xfs_trans_log_inode (tp, ip1,XFS_ILOG_CORE );
@@ -3269,10 +3280,10 @@ xfs_rename(/* RENAME_EXCHANGE is unique from here on. */ if (flags &RENAME_EXCHANGE ) { error =xfs_cross_rename (tp, src_dp, src_name, src_ip, - target_dp, target_name, target_ip, - spaceres); -xfs_iunlock_rename (inodes, num_inodes); -return error; + src_ppargs, target_dp, target_name, target_ip, + tgt_ppargs, spaceres); + nospace_error =0 ; + goto out_unlock; }/*
PATCH 12/23 xfs: Filter XFS_ATTR_PARENT for getfattr
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Parent pointers returned to the get_fattr tool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers from xfs_xattr_put_listent.
--- fs/xfs/xfs_xattr.c | 3 +++ 1 file changed, 3 insertions(+)
fs/xfs/xfs_xattr.c +3 -0
@@ -220,6 +220,9 @@ xfs_xattr_put_listent(ASSERT (context->count >=0 ); +if (flags &XFS_ATTR_PARENT ) +return ; +if (flags &XFS_ATTR_ROOT ) {#ifdef CONFIG_XFS_POSIX_ACL if (namelen == SGI_ACL_FILE_SIZE &&
PATCH 13/23 xfs: pass the attr value to put_listent when possible
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Pass the attr value to put_listent when we have local xattrs or shortform xattrs.
--- fs/xfs/libxfs/xfs_attr.h | 5 +++-- fs/xfs/libxfs/xfs_attr_sf.h | 1 + fs/xfs/xfs_attr_list.c | 8 +++++++- fs/xfs/xfs_ioctl.c | 1 + fs/xfs/xfs_xattr.c | 1 + 5 files changed, 13 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.h +3 -2
@@ -47,8 +47,9 @@ struct xfs_attrlist_cursor_kern {/* void; state communicated via *context */ -typedef void (*put_listent_func_t )(struct xfs_attr_list_context *,int , -unsigned char *,int ,int ); +typedef void (*put_listent_func_t )(struct xfs_attr_list_context *context, +int flags,unsigned char *name,int namelen,void *value, +int valuelen);struct xfs_attr_list_context {struct xfs_trans *tp;
fs/xfs/libxfs/xfs_attr_sf.h +1 -0
@@ -16,6 +16,7 @@ typedef struct xfs_attr_sf_sort {uint8_t flags;/* flags bits (see xfs_attr_leaf.h) */ xfs_dahash_t hash;/* this entry's hash value */ unsigned char *name;/* name value, pointer into buffer */ +void *value; } xfs_attr_sf_sort_t;#define XFS_ATTR_SF_ENTSIZE_MAX /* max space for name&value */ \
fs/xfs/xfs_attr_list.c +7 -1
@@ -93,6 +93,7 @@ xfs_attr_shortform_list( sfe->flags, sfe->nameval, (int )sfe->namelen, + &sfe->nameval[sfe->namelen], (int)sfe->valuelen);/* * Either search callback finished early or
@@ -139,6 +140,7 @@ xfs_attr_shortform_list( sbp->name = sfe->nameval; sbp->namelen = sfe->namelen;/* These are bytes, and both on-disk, don't endian-flip */ + sbp->value = &sfe->nameval[sfe->namelen], sbp->valuelen = sfe->valuelen; sbp->flags = sfe->flags; sfe =xfs_attr_sf_nextentry (sfe);
@@ -190,6 +192,7 @@ xfs_attr_shortform_list( sbp->flags, sbp->name, sbp->namelen, + sbp->value, sbp->valuelen);if (context->seen_enough)break ;
@@ -477,6 +480,7 @@ xfs_attr3_leaf_list_int(*/ for (; i < ichdr.count; entry++, i++) {char *name; +void *value;int namelen, valuelen;if (be32_to_cpu (entry->hashval) != cursor->hashval) {
@@ -494,6 +498,7 @@ xfs_attr3_leaf_list_int( name_loc =xfs_attr3_leaf_name_local (leaf, i); name = name_loc->nameval; namelen = name_loc->namelen; + value = &name_loc->nameval[name_loc->namelen]; valuelen =be16_to_cpu (name_loc->valuelen); }else {xfs_attr_leaf_name_remote_t *name_rmt;
@@ -501,6 +506,7 @@ xfs_attr3_leaf_list_int( name_rmt =xfs_attr3_leaf_name_remote (leaf, i); name = name_rmt->name; namelen = name_rmt->namelen; + value =NULL ; valuelen =be32_to_cpu (name_rmt->valuelen); }
@@ -511,7 +517,7 @@ xfs_attr3_leaf_list_int(return -EFSCORRUPTED ; } context->put_listent (context, entry->flags, - name, namelen, valuelen); + name, namelen, value, valuelen);if (context->seen_enough)break ; cursor->offset++;
fs/xfs/xfs_ioctl.c +1 -0
@@ -310,6 +310,7 @@ xfs_ioc_attr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { struct xfs_attrlist *alist = context->buffer;
fs/xfs/xfs_xattr.c +1 -0
@@ -213,6 +213,7 @@ xfs_xattr_put_listent(int flags, unsigned char *name, int namelen, + void *value, int valuelen) { char *prefix;
PATCH 14/23 xfs: add a libxfs header file for staging new ioctls
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new xfs_fs_staging.h header where we can land experimental ioctls without committing them to any stable interfaces anywhere.
--- fs/xfs/Kconfig | 11 +++++++++++ fs/xfs/libxfs/xfs_fs_staging.h | 18 ++++++++++++++++++ fs/xfs/xfs_linux.h | 1 + fs/xfs/xfs_super.c | 3 +++ 4 files changed, 33 insertions(+) create mode 100644 fs/xfs/libxfs/xfs_fs_staging.h
fs/xfs/Kconfig +11 -0
@@ -191,6 +191,17 @@ config XFS_ONLINE_REPAIR If unsure, say N. +config XFS_EXPERIMENTAL_IOCTLS + bool "XFS experimental system calls" + default y if XFS_DEBUG + depends on XFS_FS + help + If you say Y here, the kernel will be built with support for new + system calls that are not yet ready to become part of the stable + kernel ABI. This enables testing for early adopters. + + If unsure, say N. + config XFS_WARN bool "XFS Verbose Warnings" depends on XFS_FS && !XFS_DEBUG
fs/xfs/libxfs/xfs_fs_staging.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: LGPL-2.1 */ +/* +* Copyright (c) 2020-2024 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_FS_STAGING_H__ +#define __XFS_FS_STAGING_H__ + +/* +* Experimental system calls, ioctls and data structures supporting them. +* Nothing in here should be considered part of a stable interface of any kind. +* +* If you add an ioctl here, please leave a comment in xfs_fs.h marking it +* reserved. If you promote anything out of this file, please leave a comment +* explaining where it went. +*/ + +#endif /* __XFS_FS_STAGING_H__ */
fs/xfs/xfs_linux.h +1 -0
@@ -73,6 +73,7 @@ typedef __u32 xfs_nlink_t;#include <asm/unaligned.h> #include "xfs_fs.h" +#include "xfs_fs_staging.h" #include "xfs_stats.h" #include "xfs_sysctl.h" #include "xfs_iops.h"
fs/xfs/xfs_super.c +3 -0
@@ -2356,6 +2356,9 @@ init_xfs_fs(void) printk(KERN_INFO XFS_VERSION_STRING " with " XFS_BUILD_OPTIONS " enabled\n"); +#ifdef CONFIG_XFS_EXPERIMENTAL_IOCTLS +xfs_info (NULL ,"EXPERIMENTAL ioctls in use. Use at your own risk!" ); +#endif xfs_dir_startup (); error =xfs_init_caches ();
PATCH 15/23 xfs: Add parent pointer ioctl
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
This patch adds a new file ioctl to retrieve the parent pointer of a given inode
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_fs.h | 1 fs/xfs/libxfs/xfs_fs_staging.h | 66 ++++++++++++++++ fs/xfs/libxfs/xfs_ondisk.h | 4 + fs/xfs/libxfs/xfs_parent.c | 62 +++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 25 ++++++ fs/xfs/xfs_ioctl.c | 149 ++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.c | 164 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_parent_utils.h | 20 +++++ fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 73 ++++++++++++++++++ 11 files changed, 565 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h
fs/xfs/Makefile +1 -0
@@ -90,6 +90,7 @@ xfs-y += xfs_aops.o \ xfs_mount.o \ xfs_mru_cache.o \ xfs_pwork.o \ + xfs_parent_utils.o \ xfs_reflink.o \ xfs_stats.o \ xfs_super.o \
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -875,6 +875,7 @@ struct xfs_commit_range {/* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata)#define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) +/* XFS_IOC_GETPARENTS ---- staging 62 */ /* * ioctl commands that replace IRIX syssgi()'s
fs/xfs/libxfs/xfs_fs_staging.h +66 -0
70 lines not on this page · raw message
fs/xfs/libxfs/xfs_ondisk.h +4 -0
@@ -155,6 +155,10 @@ xfs_check_ondisk_structs(void) XFS_CHECK_OFFSET(struct xfs_efi_log_format_32, efi_extents, 16); XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents, 16); +/* parent pointer ioctls */ + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_rec, 24); + XFS_CHECK_STRUCT_SIZE(struct xfs_getparents, 96); +/* * The v5 superblock format extended several v4 header structures with * additional data. While new fields are only accessible on v5
fs/xfs/libxfs/xfs_parent.c +62 -0
65 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +25 -0
@@ -127,4 +127,29 @@ xfs_parent_finish( xfs_parent_args_free(mp, ppargs); } +/* +* Incore version of a parent pointer, also contains dirent name so callers +* can pass/obtain all the parent pointer information in a single structure +*/ +struct xfs_parent_name_irec { +/* Parent pointer attribute name fields */ +xfs_ino_t p_ino; +uint32_t p_gen; +xfs_dahash_t p_namehash; + +/* Parent pointer attribute value fields */ +uint8_t p_namelen; +unsigned char p_name[MAXNAMELEN ]; +}; + +void xfs_parent_irec_from_disk (struct xfs_parent_name_irec *irec, +const struct xfs_parent_name_rec *rec,const void *value, +unsigned int valuelen); +void xfs_parent_irec_to_disk (struct xfs_parent_name_rec *rec, +const struct xfs_parent_name_irec *irec); +void xfs_parent_irec_hashname (struct xfs_mount *mp, +struct xfs_parent_name_irec *irec); +bool xfs_parent_verify_irec (struct xfs_mount *mp, +const struct xfs_parent_name_irec *irec); +#endif /* __XFS_PARENT_H__ */
fs/xfs/xfs_ioctl.c +148 -1
167 lines not on this page · raw message
fs/xfs/xfs_parent_utils.c new file +164 -0
164 lines not on this page · raw message
fs/xfs/xfs_parent_utils.h new file +20 -0
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2022-2024 Oracle. +* All rights reserved. +*/ +#ifndef __XFS_PARENT_UTILS_H__ +#define __XFS_PARENT_UTILS_H__ + +static inline unsigned int +xfs_getparents_arraytop ( +const struct xfs_getparents *ppi, +unsigned int nr) +{ +return sizeof (struct xfs_getparents ) + + (nr *sizeof (ppi->gp_offsets[0 ])); +} + +int xfs_getparent_pointers (struct xfs_inode *ip,struct xfs_getparents *ppi); + +#endif /* __XFS_PARENT_UTILS_H__ */
fs/xfs/xfs_trace.c +1 -0
@@ -41,6 +41,7 @@#include "xfs_bmap.h" #include "xfs_exchmaps.h" #include "xfs_exchrange.h" +#include "xfs_parent.h" /* * We include this last to have the helpers above available for the trace
fs/xfs/xfs_trace.h +73 -0
85 lines not on this page · raw message
PATCH 16/23 xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.
Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.
We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.
However, the large extent counters feature is still EXPERIMENTAL, so we can gate the correction on that feature (or any features that get added after that) being enabled. Any filesystem with nrext64 or any of the as-yet-undefined feature bits turned on will be rejected by old uncorrected kernels, so this should be safe even in the upgrade case.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +33 -0
@@ -16,6 +16,29 @@#include "xfs_bmap_btree.h" #include "xfs_trace.h" +/* +* Decide if the filesystem has the parent pointer feature or any feature +* added after that. If so, we can improve the accuracy of the transaction +* reservation computations that should lead to more efficient log grant use. +*/ +static inline bool +xfs_has_parent_or_newer_feature ( +struct xfs_mount *mp) +{ +if (!xfs_sb_is_v5 (&mp->m_sb)) +return false; + +if (xfs_sb_has_incompat_feature (&mp->m_sb, + ~(XFS_SB_FEAT_INCOMPAT_FTYPE | +XFS_SB_FEAT_INCOMPAT_SPINODES | +XFS_SB_FEAT_INCOMPAT_META_UUID | +XFS_SB_FEAT_INCOMPAT_BIGTIME | +XFS_SB_FEAT_INCOMPAT_NREXT64 ))) +return true; + +return false; +} +/* * Calculate the maximum length in bytes that would be required for a local * attribute value as large attributes out of line are not logged.
@@ -31,6 +54,16 @@ xfs_log_calc_max_attrsetm_res(MAXNAMELEN -1 ; nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); nblks +=XFS_B_TO_FSB (mp, size); + +/* +* Starting with the parent pointer feature, every new fs feature +* corrects a unit conversion error in the xattr transaction +* reservation code that resulted in oversized minimum log size +* computations. +*/ +if (xfs_has_parent_or_newer_feature (mp)) + size =XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); return M_RES(mp)->tr_attrsetm.tr_logres +
PATCH 17/23 xfs: drop compatibility minimum log size computations for reflink
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Having established that we can reduce the minimum log size computation for filesystems with parent pointers or any newer feature, we should also drop the compat minlogsize code that we added when we reduced the transaction reservation size for rmap and reflink.
--- fs/xfs/libxfs/xfs_log_rlimit.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +10 -0
@@ -81,6 +81,16 @@ xfs_log_calc_trans_resv_for_minlogblocks( {unsigned int rmap_maxlevels = mp->m_rmap_maxlevels; +/* +* Starting with the parent pointer feature, every new fs feature +* drops the oversized minimum log size computation introduced by the +* original reflink code. +*/ +if (xfs_has_parent_or_newer_feature (mp)) { +xfs_trans_resv_calc (mp, resv); +return ; + } +/* * In the early days of rmap+reflink, we always set the rmap maxlevels * to 9 even if the AG was small enough that it would never grow to
PATCH 18/23 xfs: don't remove the attr fork when parent pointers are enabled
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
When an inode is removed, it may also cause the attribute fork to be removed if it is the last attribute. This transaction gets flushed to the log, but if the system goes down before we could inactivate the symlink, the log recovery tries to inactivate this inode (since it is on the unlinked list) but the verifier trips over the remote value and leaks it.
Hence we ended up with a file in this odd state on a "clean" mount. The "obvious" fix is to prohibit erasure of the attr fork to avoid tripping over the verifiers when pptrs are enabled.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +4 -2
@@ -877,7 +877,8 @@ xfs_attr_sf_removename(*/ if (totsize ==sizeof (struct xfs_attr_sf_hdr ) &&xfs_has_attr2 (mp) && (dp->i_df.if_format !=XFS_DINODE_FMT_BTREE ) && - !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE ))) { + !(args->op_flags & (XFS_DA_OP_ADDNAME |XFS_DA_OP_REPLACE )) && + !xfs_has_parent (mp)) {xfs_attr_fork_remove (dp, args->trans); }else {xfs_idata_realloc (dp, -size,XFS_ATTR_FORK );
@@ -886,7 +887,8 @@ xfs_attr_sf_removename(ASSERT (totsize >sizeof (struct xfs_attr_sf_hdr ) || (args->op_flags &XFS_DA_OP_ADDNAME ) || !xfs_has_attr2 (mp) || - dp->i_df.if_format ==XFS_DINODE_FMT_BTREE ); + dp->i_df.if_format ==XFS_DINODE_FMT_BTREE || +xfs_has_parent (mp));xfs_trans_log_inode (args->trans, dp,XFS_ILOG_CORE |XFS_ILOG_ADATA ); }
PATCH 19/23 xfs: Add the parent pointer support to the superblock version 5.
In reply to Darrick J. WongFrom: Allison Henderson <[email protected]>
Add the parent pointer superblock flag so that we can actually mount filesystems with this feature enabled.
--- fs/xfs/libxfs/xfs_format.h | 4 +++- fs/xfs/libxfs/xfs_fs.h | 2 ++ fs/xfs/libxfs/xfs_sb.c | 4 ++++ fs/xfs/xfs_super.c | 4 ++++ 4 files changed, 13 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +3 -1
@@ -373,13 +373,15 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_BIGTIME (1 << 3)/* large timestamps */ #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ #define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* large extent counters */ +#define XFS_SB_FEAT_INCOMPAT_PARENT (1 << 6)/* parent pointers */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ XFS_SB_FEAT_INCOMPAT_META_UUID| \ XFS_SB_FEAT_INCOMPAT_BIGTIME| \ XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \ - XFS_SB_FEAT_INCOMPAT_NREXT64) + XFS_SB_FEAT_INCOMPAT_NREXT64| \ + XFS_SB_FEAT_INCOMPAT_PARENT)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
fs/xfs/libxfs/xfs_fs.h +2 -0
@@ -243,6 +243,8 @@ typedef struct xfs_fsop_resblks {/* file range exchange available to userspace */ #define XFS_FSOP_GEOM_FLAGS_EXCHANGE_RANGE (1 << 24) +#define XFS_FSOP_GEOM_FLAGS_PARENT (1U << 30)/* parent pointers */ +/* * Minimum and maximum sizes need for growth checks. *
fs/xfs/libxfs/xfs_sb.c +4 -0
@@ -176,6 +176,8 @@ xfs_sb_version_to_features( features |=XFS_FEAT_NEEDSREPAIR ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NREXT64 ) features |=XFS_FEAT_NREXT64 ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_PARENT ) + features |=XFS_FEAT_PARENT ;return features; }
@@ -1251,6 +1253,8 @@ xfs_fs_geometry( geo->flags |=XFS_FSOP_GEOM_FLAGS_BIGTIME ;if (xfs_has_inobtcounts (mp)) geo->flags |=XFS_FSOP_GEOM_FLAGS_INOBTCNT ; +if (xfs_has_parent (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_PARENT ;if (xfs_has_sector (mp)) { geo->flags |=XFS_FSOP_GEOM_FLAGS_SECTOR ; geo->logsectsize = sbp->sb_logsectsize;
fs/xfs/xfs_super.c +4 -0
@@ -1741,6 +1741,10 @@ xfs_fs_fill_super( goto out_filestream_unmount; } +if (xfs_has_parent (mp)) +xfs_warn (mp, +"EXPERIMENTAL parent pointer feature enabled. Use at your own risk!" ); + error =xfs_mountfs (mp);if (error) goto out_filestream_unmount;
PATCH 20/23 xfs: only clear some log incompat bits at unmount
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a mask for log incompat feature bits that we don't want to clear at unmount time. This will be used in the next patch to make the LOG_XATTRS feature bit sticky when parent pointers are enabled.
--- fs/xfs/libxfs/xfs_format.h | 5 +++-- fs/xfs/scrub/agheader.c | 7 ++++++- fs/xfs/scrub/agheader_repair.c | 4 +++- fs/xfs/xfs_mount.c | 10 +++++----- fs/xfs/xfs_mount.h | 3 +++ 5 files changed, 20 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_format.h +3 -2
@@ -417,9 +417,10 @@ xfs_sb_has_incompat_log_feature(static inline void xfs_sb_remove_incompat_log_features ( -struct xfs_sb *sbp) +struct xfs_sb *sbp, +unsigned int features) { - sbp->sb_features_log_incompat &= ~XFS_SB_FEAT_INCOMPAT_LOG_ALL ; + sbp->sb_features_log_incompat &= ~features; } static inline void
fs/xfs/scrub/agheader.c +6 -1
@@ -329,8 +329,13 @@ xchk_superblock(* log incompat features protect newer log record types from * older log recovery code. Log recovery doesn't check the * secondary supers, so we can clear these if needed. +* +* Note that mkfs sets the "permanent" bits in the secondary +* supers, so we don't flag the field if those are the only +* bits set. */ -if (sb->sb_features_log_incompat) +if (sb->sb_features_log_incompat != +cpu_to_be32 (mp->m_perm_log_incompat))xchk_block_set_preen (sc, bp);/* Don't care about sb_crc */
fs/xfs/scrub/agheader_repair.c +3 -1
@@ -66,13 +66,15 @@ xrep_superblock(/* * Don't write out a secondary super with NEEDSREPAIR or log incompat * features set, since both are ignored when set on a secondary. +* Always set the permanent log incompat features, just like mkfs. */ if (xfs_has_crc (mp)) {struct xfs_dsb *sb = bp->b_addr; sb->sb_features_incompat &= ~cpu_to_be32 (XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR ); - sb->sb_features_log_incompat =0 ; + sb->sb_features_log_incompat = +cpu_to_be32 (mp->m_perm_log_incompat); }/* Write this to disk. */
fs/xfs/xfs_mount.c +5 -5
@@ -1388,11 +1388,12 @@ boolxfs_clear_incompat_log_features ( struct xfs_mount *mp) { +unsigned int to_clear =XFS_SB_FEAT_INCOMPAT_LOG_ALL & + ~mp->m_perm_log_incompat;bool ret = false;if (!xfs_has_crc (mp) || - !xfs_sb_has_incompat_log_feature (&mp->m_sb, -XFS_SB_FEAT_INCOMPAT_LOG_ALL ) || + !xfs_sb_has_incompat_log_feature (&mp->m_sb, to_clear) ||xfs_is_shutdown (mp) || !xfs_is_done_with_log_incompat (mp))return false;
@@ -1405,9 +1406,8 @@ xfs_clear_incompat_log_features(xfs_buf_lock (mp->m_sb_bp);xfs_buf_hold (mp->m_sb_bp); -if (xfs_sb_has_incompat_log_feature (&mp->m_sb, -XFS_SB_FEAT_INCOMPAT_LOG_ALL )) { -xfs_sb_remove_incompat_log_features (&mp->m_sb); +if (xfs_sb_has_incompat_log_feature (&mp->m_sb, to_clear)) { +xfs_sb_remove_incompat_log_features (&mp->m_sb, to_clear); ret = true; }
fs/xfs/xfs_mount.h +3 -0
@@ -163,6 +163,9 @@ typedef struct xfs_mount {bool m_finobt_nores;/* no per-AG finobt resv. */ bool m_update_sb;/* sb needs update in mount */ +/* never clear these log incompat feature bits */ +unsigned int m_perm_log_incompat; +/* * Bitsets of per-fs metadata that have been checked and/or are sick. * Callers must hold m_sb_lock to access these two fields.
PATCH 21/23 xfs: allow adding multiple log incompat feature bits
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make it so that we can set multiple log incompat feature bits at once.
--- .../filesystems/xfs/xfs-online-fsck-design.rst | 2 +- fs/xfs/xfs_exchrange.c | 2 +- fs/xfs/xfs_mount.c | 13 ++++++------- fs/xfs/xfs_mount.h | 2 +- fs/xfs/xfs_xattr.c | 2 +- 5 files changed, 10 insertions(+), 11 deletions(-)
Documentation/filesystems/xfs/xfs-online-fsck-design.rst +1 -1
@@ -4055,7 +4055,7 @@ series. | clear the bit; if the lock attempt fails, the feature bit remains set. | | The code supporting a log incompat feature should create wrapper | | functions to obtain the log feature and call | -| ``xfs_add_incompat_log_feature`` to set the feature bits in the primary | +| ``xfs_add_incompat_log_features`` to set the feature bits in the primary | | superblock. | | The superblock update is performed transactionally, so the wrapper to | | obtain log assistance must be called just prior to the creation of the |
fs/xfs/xfs_exchrange.c +1 -1
@@ -72,7 +72,7 @@ xfs_exchrange_enable(if (!xfs_exchrange_upgradeable (mp))return -EOPNOTSUPP ; - error =xfs_add_incompat_log_feature (mp, + error =xfs_add_incompat_log_features (mp,XFS_SB_FEAT_INCOMPAT_LOG_EXCHMAPS );if (error)return error;
fs/xfs/xfs_mount.c +6 -7
@@ -1305,15 +1305,14 @@ xfs_can_add_incompat_log_features(* cannot have any other transactions in progress. */ int -xfs_add_incompat_log_feature ( +xfs_add_incompat_log_features (struct xfs_mount *mp, -uint32_t feature) +uint32_t features) {struct xfs_dsb *dsb;int error; -ASSERT (hweight32 (feature) ==1 ); -ASSERT (!(feature &XFS_SB_FEAT_INCOMPAT_LOG_UNKNOWN )); +ASSERT (!(features &XFS_SB_FEAT_INCOMPAT_LOG_UNKNOWN ));/* * Force the log to disk and kick the background AIL thread to reduce
@@ -1338,7 +1337,7 @@ xfs_add_incompat_log_feature( goto rele; } -if (xfs_sb_has_incompat_log_feature (&mp->m_sb, feature)) +if ((mp->m_sb.sb_features_log_incompat & features) == features) goto rele;if (!xfs_can_add_incompat_log_features (mp, true)) {
@@ -1353,7 +1352,7 @@ xfs_add_incompat_log_feature(*/ dsb = mp->m_sb_bp->b_addr;xfs_sb_to_disk (dsb, &mp->m_sb); - dsb->sb_features_log_incompat |=cpu_to_be32 (feature); + dsb->sb_features_log_incompat |=cpu_to_be32 (features); error =xfs_bwrite (mp->m_sb_bp);if (error) goto shutdown;
@@ -1362,7 +1361,7 @@ xfs_add_incompat_log_feature(* Add the feature bits to the incore superblock before we unlock the * buffer. */ -xfs_sb_add_incompat_log_features (&mp->m_sb, feature); +xfs_sb_add_incompat_log_features (&mp->m_sb, features);xfs_buf_relse (mp->m_sb_bp);/* Log the superblock to disk. */
fs/xfs/xfs_mount.h +1 -1
@@ -577,7 +577,7 @@ int xfs_zero_extent(struct xfs_inode *ip, xfs_fsblock_t start_fsb,struct xfs_error_cfg *xfs_error_get_cfg (struct xfs_mount *mp,int error_class,int error);void xfs_force_summary_recalc (struct xfs_mount *mp); -int xfs_add_incompat_log_feature (struct xfs_mount *mp,uint32_t feature); +int xfs_add_incompat_log_features (struct xfs_mount *mp,uint32_t feature);bool xfs_clear_incompat_log_features (struct xfs_mount *mp);void xfs_mod_delalloc (struct xfs_mount *mp,int64_t delta);
fs/xfs/xfs_xattr.c +1 -1
@@ -44,7 +44,7 @@ xfs_attr_grab_log_assist(return -EOPNOTSUPP ;/* Enable log-assisted xattrs. */ - error =xfs_add_incompat_log_feature (mp, + error =xfs_add_incompat_log_features (mp,XFS_SB_FEAT_INCOMPAT_LOG_XATTRS );if (error)return error;
PATCH 22/23 xfs: make XFS_SB_FEAT_INCOMPAT_LOG_XATTRS sticky for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Directory parent pointers require logged extended attribute updates to maintain referential integrity with dirent updates. Make sure that the log incompat bit never goes away.
--- fs/xfs/xfs_mount.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)
fs/xfs/xfs_mount.c +31 -0
@@ -620,6 +620,31 @@ xfs_agbtree_compute_maxlevels( mp->m_agbtree_maxlevels =max (levels, mp->m_refc_maxlevels); } +/* +* Mark certain log incompat feature bits as sticky, and make sure they're +* enabled. +*/ +STATIC int +xfs_mountfs_set_perm_log_features ( +struct xfs_mount *mp) +{ +if (xfs_has_parent (mp)) { +/* +* Directory parent pointers require logged extended attribute +* updates to maintain referential integrity with dirent +* updates. Set the LARP bit. +*/ + mp->m_perm_log_incompat |=XFS_SB_FEAT_INCOMPAT_LOG_XATTRS ; + } + +/* Make sure the permanent bits are set in the ondisk primary super. */ +if ((mp->m_sb.sb_features_log_incompat & mp->m_perm_log_incompat) == + mp->m_perm_log_incompat) +return 0 ; + +return xfs_add_incompat_log_features (mp, mp->m_perm_log_incompat); +} +/* * This function does the following on an initial mount of a file system: * - reads the superblock from disk and init the mount struct
@@ -826,6 +851,12 @@ xfs_mountfs( goto out_inodegc_shrinker; } + error =xfs_mountfs_set_perm_log_features (mp); +if (error) { +xfs_warn (mp,"setting permanent log incompat features failed" ); + goto out_inodegc_shrinker; + } +/* Enable background inode inactivation workers. */ xfs_inodegc_start (mp);xfs_blockgc_start (mp);
PATCH 23/23 xfs: make XFS_SB_FEAT_INCOMPAT_LOG_EXCHMAPS sticky for parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Directory parent pointers make directory repairs practical. However, online repairs of directories (and parent pointers which are embedded in extended attributes) must commit the repairs atomically by building a replacement structure in a temporary file and then exchanging the contents.
Although there's no hard dependency between parent pointers and file mapping exchanges like there is with logged xattr updates, these two features will likely go hand in hand. Make sure we always have the exchmaps bit turned on.
--- fs/xfs/xfs_mount.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/xfs/xfs_mount.c +14 -0
@@ -635,6 +635,20 @@ xfs_mountfs_set_perm_log_features(* updates. Set the LARP bit. */ mp->m_perm_log_incompat |=XFS_SB_FEAT_INCOMPAT_LOG_XATTRS ; + +/* +* Directory parent pointers make directory repairs practical. +* However, online repairs of directories (and parent pointers +* which are embedded in extended attributes) must commit the +* repairs atomically by building a replacement structure in a +* temporary file and then exchanging the contents. +* +* Although there's no hard dependency between parent pointers +* and file mapping exchanges like there is with logged xattr +* updates, these two features will likely go hand in hand. +* Set the exchmaps bit. +*/ + mp->m_perm_log_incompat |=XFS_SB_FEAT_INCOMPAT_LOG_EXCHMAPS ; }/* Make sure the permanent bits are set in the ondisk primary super. */
PATCH 2/2 xfs: fsck for parent pointers
In reply to Darrick J. WongHi all,
This series implements online checking and repair for directory parent pointer metadata. The checking half is fairly straightforward -- for each outgoing directory link (forward or backwards), grab the inode at the other end, and confirm that there's a corresponding link. If we can't grab an inode or lock it, we'll save that link for a slower loop that cycles all the locks, confirms the continued existence of the link, and rechecks the link if it's actually still there.
Repairs are a bit more involved -- for directories, we walk the entire filesystem to rebuild the dirents from parent pointer information. Parent pointer repairs do the same walk but rebuild the pptrs from the dirent information, but with the added twist that it duplicates all the xattrs so that it can use the atomic extent swapping code to commit the repairs atomically.
This introduces an added twist to the xattr repair code -- we use dirent hooks to detect a colliding update to the pptr data while we're not holding the ILOCKs; if one is detected, we restart the xattr salvaging process but this time hold all the ILOCKs until the end of the scan.
For offline repair, the phase6 directory connectivity scan generates an index of all the expected parent pointers in the filesystem. Then it walks each file and compares the parent pointers attached to that file against the index generated, and resyncs the results as necessary.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-fsck
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-fsck
---
Commits in this patchset:
* xfs: check dirents have parent pointers
* xfs: deferred scrub of dirents
* xfs: create a parent pointer walk function for scrubbers
* xfs: scrub parent pointers
* xfs: deferred scrub of parent pointers
* xfs: walk directory parent pointers to determine backref count
* xfs: add raw parent pointer apis to support repair
* xfs: set child file owner in xfs_da_args when changing parent pointers
* xfs: check parent pointer xattrs when scrubbing
* xfs: salvage parent pointers when rebuilding xattr structures
* xfs: replace namebuf with parent pointer in directory repair
* xfs: repair directories by scanning directory parent pointers
* xfs: implement live updates for directory repairs
* xfs: replay unlocked parent pointer updates that accrue during xattr repair
* xfs: replace namebuf with parent pointer in parent pointer repair
* xfs: repair directory parent pointers by scanning for dirents
* xfs: implement live updates for parent pointer repairs
* xfs: remove pointless unlocked assertion
* xfs: split xfs_bmap_add_attrfork into two pieces
* xfs: actually rebuild the parent pointer xattrs
* xfs: adapt the orphanage code to handle parent pointers
* xfs: repair link count of nondirectories after rebuilding parent pointers
* xfs: inode repair should ensure there's an attr fork to store parent pointers
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_attr.c | 39 + fs/xfs/libxfs/xfs_attr.h | 1 fs/xfs/libxfs/xfs_bmap.c | 38 - fs/xfs/libxfs/xfs_bmap.h | 3 fs/xfs/libxfs/xfs_dir2.c | 2 fs/xfs/libxfs/xfs_dir2.h | 2 fs/xfs/libxfs/xfs_parent.c | 107 +++ fs/xfs/libxfs/xfs_parent.h | 18 + fs/xfs/scrub/attr.c | 16 - fs/xfs/scrub/attr_repair.c | 502 ++++++++++++++++ fs/xfs/scrub/attr_repair.h | 4 fs/xfs/scrub/common.h | 1 fs/xfs/scrub/dir.c | 352 +++++++++++ fs/xfs/scrub/dir_repair.c | 568 +++++++++++++++++- fs/xfs/scrub/findparent.c | 12 fs/xfs/scrub/findparent.h | 10 fs/xfs/scrub/inode_repair.c | 41 + fs/xfs/scrub/listxattr.c | 94 +++ fs/xfs/scrub/listxattr.h | 13 fs/xfs/scrub/nlinks.c | 71 ++ fs/xfs/scrub/nlinks.h | 3 fs/xfs/scrub/nlinks_repair.c | 2 fs/xfs/scrub/orphanage.c | 42 + fs/xfs/scrub/orphanage.h | 3 fs/xfs/scrub/parent.c | 695 ++++++++++++++++++++++ fs/xfs/scrub/parent_repair.c | 1317 +++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/readdir.c | 78 ++ fs/xfs/scrub/readdir.h | 3 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 217 +++++++ 32 files changed, 4151 insertions(+), 108 deletions(-)
PATCH 1/23 xfs: check dirents have parent pointers
From: Darrick J. Wong <[email protected]>
If the fs has parent pointers, we need to check that each child dirent points to a file that has a parent pointer pointing back at us.
--- fs/xfs/libxfs/xfs_parent.c | 54 +++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 10 ++++ fs/xfs/scrub/dir.c | 122 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 185 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_parent.c +54 -0
@@ -366,3 +366,57 @@ xfs_parent_irec_hashname( irec->p_namehash =xfs_dir2_hashname (mp, &dname); } + +static inline void +xfs_parent_scratch_init ( +struct xfs_trans *tp, +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scr) +{ +memset (&scr->args,0 ,sizeof (struct xfs_da_args )); + scr->args.attr_filter =XFS_ATTR_PARENT ; + scr->args.dp = ip; + scr->args.geo = ip->i_mount->m_attr_geo; + scr->args.name = (const unsigned char *)&scr->rec; + scr->args.namelen =sizeof (struct xfs_parent_name_rec ); + scr->args.op_flags =XFS_DA_OP_NVLOOKUP ; + scr->args.trans = tp; + scr->args.value = (void *)pptr->p_name; + scr->args.valuelen = pptr->p_namelen; + scr->args.whichfork =XFS_ATTR_FORK ; + scr->args.hashval =xfs_da_hashname ((const void *)&scr->rec, +sizeof (struct xfs_parent_name_rec )); +} + +/* +* Look up the @name associated with the parent pointer (@pptr) of @ip. +* Caller must hold at least ILOCK_SHARED. Returns 0 if the pointer is found, +* -ENOATTR if there is no match, or a negative errno. The scratchpad need not +* be initialized. +*/ +int +xfs_parent_lookup ( +struct xfs_trans *tp, +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scr) +{ +int error; + +/* +* Make sure the attr fork iext tree is loaded in transaction context +* before we start down the rest of the call path. +*/ +if (xfs_inode_hasattr (ip)) { + error =xfs_iread_extents (tp, ip,XFS_ATTR_FORK ); +if (error) +return error; + } + +xfs_parent_irec_to_disk (&scr->rec, pptr); +xfs_parent_scratch_init (tp, ip, pptr, scr); + scr->args.op_flags |=XFS_DA_OP_OKNOENT ; + +return xfs_attr_get_ilocked (&scr->args); +}
fs/xfs/libxfs/xfs_parent.h +10 -0
@@ -152,4 +152,14 @@ void xfs_parent_irec_hashname(struct xfs_mount *mp,bool xfs_parent_verify_irec (struct xfs_mount *mp,const struct xfs_parent_name_irec *irec); +/* Scratchpad memory so that raw parent operations don't burn stack space. */ +struct xfs_parent_scratch { +struct xfs_parent_name_rec rec; +struct xfs_da_args args; +}; + +int xfs_parent_lookup (struct xfs_trans *tp,struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
fs/xfs/scrub/dir.c +121 -1
165 lines not on this page · raw message
PATCH 2/23 xfs: deferred scrub of dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based parent pointer check fails, retain those dirents and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/scrub/dir.c | 234 +++++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/readdir.c | 78 ++++++++++++++++ fs/xfs/scrub/readdir.h | 3 + fs/xfs/scrub/trace.h | 34 +++++++ 4 files changed, 346 insertions(+), 3 deletions(-)
fs/xfs/scrub/dir.c +231 -3
@@ -24,6 +24,10 @@#include "scrub/readdir.h" #include "scrub/health.h" #include "scrub/repair.h" +#include "scrub/trace.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" /* Set us up to scrub directories. */ int
@@ -43,6 +47,18 @@ xchk_setup_directory(/* Directories */ +/* Deferred directory entry that we saved for later. */ +struct xchk_dirent { +/* Cookie for retrieval of the dirent name. */ +xfblob_cookie name_cookie; + +/* Child inode number. */ +xfs_ino_t ino; + +/* Length of the pptr name. */ +uint8_t namelen; +}; +struct xchk_dir {struct xfs_scrub *sc;
@@ -52,6 +68,15 @@ struct xchk_dir {/* xattr key and da args for parent pointer validation. */ struct xfs_parent_scratch pptr_scratch; +/* Fixed-size array of xchk_dirent structures. */ +struct xfarray *dir_entries; + +/* Blobs containing dirent names. */ +struct xfblob *dir_names; + +/* If we've cycled the ILOCK, we must revalidate deferred dirents. */ +bool need_revalidate; +/* Name buffer for dirent revalidation. */ uint8_t namebuf[MAXNAMELEN ];
@@ -158,8 +183,26 @@ xchk_dir_check_pptr_fast(/* Try to lock the inode. */ lockmode =xchk_dir_lock_child (sc, ip);if (!lockmode) { -xchk_set_incomplete (sc); -return -ECANCELED ; +struct xchk_dirent save_de = { + .namelen = name->len, + .ino = ip->i_ino, + }; + +/* Couldn't lock the inode, so save the dirent for later. */ +trace_xchk_dir_defer (sc->ip, name->name, name->len, ip->i_ino); + + error =xfblob_store (sd->dir_names, &save_de.name_cookie, + name->name, name->len); +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + + error =xfarray_append (sd->dir_entries, &save_de); +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + +return 0 ; } error =xchk_dir_parent_pointer (sd, name, ip);
@@ -875,6 +918,147 @@ xchk_directory_blocks(return error; } +/* +* Revalidate a dirent that we collected in the past but couldn't check because +* of lock contention. Returns 0 if the dirent is still valid, -ENOENT if it +* has gone away on us, or a negative errno. +*/ +STATIC int +xchk_dir_revalidate_dirent ( +struct xchk_dir *sd, +const struct xfs_name *xname, +xfs_ino_t ino) +{ +struct xfs_scrub *sc = sd->sc; +xfs_ino_t child_ino; +int error; + +/* +* Look up the directory entry. If we get -ENOENT, the directory entry +* went away and there's nothing to revalidate. Return any other +* error. +*/ + error =xchk_dir_lookup (sc, sc->ip, xname, &child_ino); +if (error) +return error; + +/* The inode number changed, nothing to revalidate. */ +if (ino != child_ino) +return -ENOENT ; + +return 0 ; +} + +/* +* Check a directory entry's parent pointers the slow way, which means we cycle +* locks a bunch and put up with revalidation until we get it done. +*/ +STATIC int +xchk_dir_slow_dirent ( +struct xchk_dir *sd, +struct xchk_dirent *dirent) +{ +struct xfs_name xname = { + .name = sd->namebuf, + .len = dirent->namelen, + }; +struct xfs_scrub *sc = sd->sc; +struct xfs_inode *ip; +unsigned int lockmode; +int error; + +/* Check that the deferred dirent still exists. */ +if (sd->need_revalidate) { + error =xchk_dir_revalidate_dirent (sd, &xname, dirent->ino); +if (error == -ENOENT ) +return 0 ; +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , + &error)) +return error; + } + + error =xchk_iget (sc, dirent->ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; + +/* +* If we can grab both IOLOCK and ILOCK of the alleged child, we can +* proceed with the validation. +*/ + lockmode =xchk_dir_lock_child (sc, ip); +if (lockmode) { +trace_xchk_dir_slowpath (sc->ip, xname.name, xname.len, + ip->i_ino); + goto check_pptr; + } + +/* +* We couldn't lock the child file. Drop all the locks and try to +* get them again, one at a time. +*/ +xchk_iunlock (sc, sc->ilock_flags); + sd->need_revalidate = true; + +trace_xchk_dir_ultraslowpath (sc->ip, xname.name, xname.len, ip->i_ino); + + error =xchk_dir_trylock_for_pptrs (sc, ip, &lockmode); +if (error) + goto out_rele; + +/* Revalidate, since we just cycled the locks. */ + error =xchk_dir_revalidate_dirent (sd, &xname, dirent->ino); +if (error == -ENOENT ) { + error =0 ; + goto out_unlock; + } +if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error)) + goto out_unlock; + +check_pptr: + error =xchk_dir_parent_pointer (sd, &xname, ip); +out_unlock: +xfs_iunlock (ip, lockmode); +out_rele: +xchk_irele (sc, ip); +return error; +} + +/* Check all the dirents that we deferred the first time around. */ +STATIC int +xchk_dir_finish_slow_dirents ( +struct xchk_dir *sd) +{ +xfarray_idx_t array_cur; +int error; + + foreach_xfarray_idx(sd->dir_entries, array_cur) { +struct xchk_dirent dirent; + +if (sd->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) +return 0 ; + + error =xfarray_load (sd->dir_entries, array_cur, &dirent); +if (error) +return error; + + error =xfblob_load (sd->dir_names, dirent.name_cookie, + sd->namebuf, dirent.namelen); +if (error) +return error; + sd->namebuf[MAXNAMELEN -1 ] =0 ; + + error =xchk_dir_slow_dirent (sd, &dirent); +if (error) +return error; + } + +return 0 ; +} +/* Scrub a whole directory. */ int xchk_directory(
@@ -918,10 +1102,54 @@ xchk_directory(return -ENOMEM ; sd->sc = sc; +if (xfs_has_parent (sc->mp)) { +char *descr; + +/* +* Set up some staging memory for dirents that we can't check +* due to locking contention. +*/ + descr =xchk_xfile_ino_descr (sc,"slow directory entries" ); + error =xfarray_create (descr,0 ,sizeof (struct xchk_dirent ), + &sd->dir_entries); +kfree (descr); +if (error) + goto out_sd; + + descr =xchk_xfile_ino_descr (sc,"slow directory entry names" ); + error =xfblob_create (descr, &sd->dir_names); +kfree (descr); +if (error) + goto out_entries; + } +/* Look up every name in this directory by hash. */ error =xchk_dir_walk (sc, sc->ip, xchk_dir_actor, sd); +if (error == -ECANCELED ) + error =0 ; +if (error) + goto out_names; + +if (xfs_has_parent (sc->mp)) { + error =xchk_dir_finish_slow_dirents (sd); +if (error == -ETIMEDOUT ) { +/* Couldn't grab a lock, scrub was marked incomplete */ + error =0 ; + goto out_names; + } +if (error) + goto out_names; + } + +out_names: +if (sd->dir_names) +xfblob_destroy (sd->dir_names); +out_entries: +if (sd->dir_entries) +xfarray_destroy (sd->dir_entries); +out_sd:kvfree (sd); -if (error && error != -ECANCELED ) +if (error)return error;/* If the dir is clean, it is clearly not zapped. */
fs/xfs/scrub/readdir.c +78 -0
87 lines not on this page · raw message
fs/xfs/scrub/readdir.h +3 -0
@@ -16,4 +16,7 @@ int xchk_dir_walk(struct xfs_scrub *sc, struct xfs_inode *dp,int xchk_dir_lookup (struct xfs_scrub *sc,struct xfs_inode *dp,const struct xfs_name *name,xfs_ino_t *ino); +int xchk_dir_trylock_for_pptrs (struct xfs_scrub *sc,struct xfs_inode *ip, +unsigned int *lockmode); +#endif /* __XFS_SCRUB_READDIR_H__ */
fs/xfs/scrub/trace.h +34 -0
@@ -1505,6 +1505,40 @@ DEFINE_EVENT(xchk_nlinks_diff_class, name, \ TP_ARGS(mp, ip, live))DEFINE_SCRUB_NLINKS_DIFF_EVENT (xchk_nlinks_compare_inode); +DECLARE_EVENT_CLASS (xchk_pptr_class, + TP_PROTO(struct xfs_inode *ip, const unsigned char *name, + unsigned int namelen, xfs_ino_t far_ino), +TP_ARGS (ip, name, namelen, far_ino), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(unsigned int, namelen) + __dynamic_array(char, name, namelen) +__field (xfs_ino_t, far_ino) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->namelen = namelen; + memcpy(__get_str(name), name, namelen); + __entry->far_ino = far_ino; + ), +TP_printk ("dev %d:%d ino 0x%llx name '%.*s' far_ino 0x%llx" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->namelen, +__get_str (name), + __entry->far_ino) +) +#define DEFINE_XCHK_PPTR_EVENT (name) \ +DEFINE_EVENT(xchk_pptr_class, name, \ + TP_PROTO(struct xfs_inode *ip, const unsigned char *name, \ + unsigned int namelen, xfs_ino_t far_ino), \ + TP_ARGS(ip, name, namelen, far_ino)) +DEFINE_XCHK_PPTR_EVENT (xchk_dir_defer); +DEFINE_XCHK_PPTR_EVENT (xchk_dir_slowpath); +DEFINE_XCHK_PPTR_EVENT (xchk_dir_ultraslowpath); +/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
PATCH 3/23 xfs: create a parent pointer walk function for scrubbers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Build a parent pointer iteration function off of the existing xattr walking code. This will be used by subsequent patches.
--- fs/xfs/scrub/listxattr.c | 84 +++++++++++++++++++++++++++++++++++++++++++--- fs/xfs/scrub/listxattr.h | 9 +++++ 2 files changed, 88 insertions(+), 5 deletions(-)
fs/xfs/scrub/listxattr.c +79 -5
@@ -17,11 +17,46 @@#include "xfs_attr_leaf.h" #include "xfs_attr_sf.h" #include "xfs_trans.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/bitmap.h" #include "scrub/dab_bitmap.h" #include "scrub/listxattr.h" +struct xchk_pptr_walk { +struct xfs_parent_name_irec *pptr_buf; +xchk_pptr_fn fn; +void *priv; +}; + +/* Call the parent pointer callback if this xattr is a valid parent pointer. */ +STATIC int +xchk_pptr_walk_attr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct xchk_pptr_walk *pw = priv; +const struct xfs_parent_name_rec *rec = (const void *)name; + +/* Ignore anything that isn't a parent pointer. */ +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ; + +if (!xfs_parent_namecheck (sc->mp, rec, namelen, attr_flags)) +return -EFSCORRUPTED ; +if (!xfs_parent_valuecheck (sc->mp, value, valuelen)) +return -EFSCORRUPTED ; + +xfs_parent_irec_from_disk (pw->pptr_buf, rec, value, valuelen); +return pw->fn (sc, ip, pw->pptr_buf, pw->priv); +} +/* Call a function for every entry in a shortform xattr structure. */ STATIC int xchk_xattr_walk_sf(
@@ -37,9 +72,16 @@ xchk_xattr_walk_sf( sfe =xfs_attr_sf_firstentry (hdr);for (i =0 ; i < hdr->count; i++) { - error =attr_fn (sc, ip, sfe->flags, sfe->nameval, sfe->namelen, - &sfe->nameval[sfe->namelen], sfe->valuelen, - priv); +if (attr_fn == xchk_pptr_walk_attr) + error =xchk_pptr_walk_attr (sc, ip, sfe->flags, + sfe->nameval, sfe->namelen, + &sfe->nameval[sfe->namelen], + sfe->valuelen, priv); +else + error =attr_fn (sc, ip, sfe->flags, + sfe->nameval, sfe->namelen, + &sfe->nameval[sfe->namelen], + sfe->valuelen, priv);if (error)return error;
@@ -91,8 +133,12 @@ xchk_xattr_walk_leaf_entries( valuelen =be32_to_cpu (name_rmt->valuelen); } - error =attr_fn (sc, ip, entry->flags, name, namelen, value, - valuelen, priv); +if (attr_fn == xchk_pptr_walk_attr) + error =xchk_pptr_walk_attr (sc, ip, entry->flags, name, + namelen, value, valuelen, priv); +else + error =attr_fn (sc, ip, entry->flags, name, namelen, + value, valuelen, priv);if (error)return error;
@@ -310,3 +356,31 @@ xchk_xattr_walk(return xchk_xattr_walk_node (sc, ip, attr_fn, priv); } + +/* +* Walk every parent pointer of this file. The parent pointer will be +* formatted into the provided @pptr_buf, which is then passed to the callback +* function. +* +* The callback function must decide if an invalid parent_ino or invalid name +* should halt the parent pointer walk; the only validation done here is the +* structure of the xattrs themselves. +*/ +int +xchk_pptr_walk ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +xchk_pptr_fn pptr_fn, +struct xfs_parent_name_irec *pptr_buf, +void *priv) +{ +struct xchk_pptr_walk pw = { + .fn = pptr_fn, + .pptr_buf = pptr_buf, + .priv = priv, + }; + +ASSERT (xfs_has_parent (sc->mp)); + +return xchk_xattr_walk (sc, ip, xchk_pptr_walk_attr, &pw); +}
fs/xfs/scrub/listxattr.h +9 -0
@@ -14,4 +14,13 @@ typedef int (*xchk_xattr_fn)(struct xfs_scrub *sc, struct xfs_inode *ip,int xchk_xattr_walk (struct xfs_scrub *sc,struct xfs_inode *ip,xchk_xattr_fn attr_fn,void *priv); +struct xfs_parent_name_irec ; + +typedef int (*xchk_pptr_fn )(struct xfs_scrub *sc,struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr,void *priv); + +int xchk_pptr_walk (struct xfs_scrub *sc,struct xfs_inode *ip, +xchk_pptr_fn pptr_fn,struct xfs_parent_name_irec *pptr_buf, +void *priv); +#endif /* __XFS_SCRUB_LISTXATTR_H__ */
PATCH 4/23 xfs: scrub parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Actually check parent pointers now.
--- fs/xfs/scrub/parent.c | 385 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 385 insertions(+)
fs/xfs/scrub/parent.c +385 -0
@@ -15,11 +15,15 @@#include "xfs_icache.h" #include "xfs_dir2.h" #include "xfs_dir2_priv.h" +#include "xfs_attr.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/readdir.h" #include "scrub/tempfile.h" #include "scrub/repair.h" +#include "scrub/listxattr.h" +#include "scrub/trace.h" /* Set us up to scrub parents. */ int
@@ -197,6 +201,384 @@ xchk_parent_validate(return error; } +/* +* Checking of Parent Pointers +* =========================== +* +* On filesystems with directory parent pointers, we check the referential +* integrity by visiting each parent pointer of a child file and checking that +* the directory referenced by the pointer actually has a dirent pointing +* forward to the child file. +*/ + +struct xchk_pptrs { +struct xfs_scrub *sc; + +/* Scratch buffer for scanning pptr xattrs */ +struct xfs_parent_name_irec pptr; + +/* How many parent pointers did we find at the end? */ +unsigned long long pptrs_found; + +/* Parent of this directory. */ +xfs_ino_t parent_ino; +}; + +/* Does this parent pointer match the dotdot entry? */ +STATIC int +xchk_parent_scan_dotdot ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +void *priv) +{ +struct xchk_pptrs *pp = priv; + +if (pp->parent_ino == pptr->p_ino && +xfs_parent_verify_irec (sc->mp, pptr)) +return -ECANCELED ; + +return 0 ; +} + +/* Look up the dotdot entry so that we can check it as we walk the pptrs. */ +STATIC int +xchk_parent_pptr_and_dotdot ( +struct xchk_pptrs *pp) +{ +struct xfs_scrub *sc = pp->sc; +int error; + +/* Look up '..' */ + error =xchk_dir_lookup (sc, sc->ip, &xfs_name_dotdot, &pp->parent_ino); +if (!xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) +return error; +if (!xfs_verify_dir_ino (sc->mp, pp->parent_ino)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +/* Is this the root dir? Then '..' must point to itself. */ +if (sc->ip == sc->mp->m_rootip) { +if (sc->ip->i_ino != pp->parent_ino) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK ,0 ); +return 0 ; + } + +/* +* If this is now an unlinked directory, the dotdot value is +* meaningless as long as it points to a valid inode. +*/ +if (VFS_I (sc->ip)->i_nlink ==0 ) +return 0 ; + +if (pp->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) +return 0 ; + +/* Otherwise, walk the pptrs again, and check. */ + error =xchk_pptr_walk (sc, sc->ip, xchk_parent_scan_dotdot, &pp->pptr, + pp); +if (error == -ECANCELED ) { +/* Found a parent pointer that matches dotdot. */ +return 0 ; + } +if (!error || error == -EFSCORRUPTED ) { +/* Found a broken parent pointer or no match. */ +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +return error; +} + +/* +* Try to lock a parent directory for checking dirents. Returns the inode +* flags for the locks we now hold, or zero if we failed. +*/ +STATIC unsigned int +xchk_parent_lock_dir ( +struct xfs_scrub *sc, +struct xfs_inode *dp) +{ +if (!xfs_ilock_nowait (dp,XFS_IOLOCK_SHARED )) +return 0 ; + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_SHARED )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +if (!xfs_need_iread_extents (&dp->i_df)) +return XFS_IOLOCK_SHARED |XFS_ILOCK_SHARED ; + +xfs_iunlock (dp,XFS_ILOCK_SHARED ); + +if (!xfs_ilock_nowait (dp,XFS_ILOCK_EXCL )) { +xfs_iunlock (dp,XFS_IOLOCK_SHARED ); +return 0 ; + } + +return XFS_IOLOCK_SHARED |XFS_ILOCK_EXCL ; +} + +/* Check the forward link (dirent) associated with this parent pointer. */ +STATIC int +xchk_parent_dirent ( +struct xchk_pptrs *pp, +struct xfs_inode *dp) +{ +struct xfs_name xname = { + .name = pp->pptr.p_name, + .len = pp->pptr.p_namelen, + }; +struct xfs_scrub *sc = pp->sc; +xfs_ino_t child_ino; +int error; + +/* +* Use the name attached to this parent pointer to look up the +* directory entry in the alleged parent. +*/ + error =xchk_dir_lookup (sc, dp, &xname, &child_ino); +if (error == -ENOENT ) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* Does the inode number match? */ +if (child_ino != sc->ip->i_ino) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return 0 ; + } + +return 0 ; +} + +/* Try to grab a parent directory. */ +STATIC int +xchk_parent_iget ( +struct xchk_pptrs *pp, +struct xfs_inode **dpp) +{ +struct xfs_scrub *sc = pp->sc; +struct xfs_inode *ip; +int error; + +/* Validate inode number. */ + error =xfs_dir_ino_validate (sc->mp, pp->pptr.p_ino); +if (error) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + + error =xchk_iget (sc, pp->pptr.p_ino, &ip); +if (error == -EINVAL || error == -ENOENT ) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } +if (!xchk_fblock_xref_process_error (sc,XFS_ATTR_FORK ,0 , &error)) +return error; + +/* The parent must be a directory. */ +if (!S_ISDIR (VFS_I (ip)->i_mode)) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + +/* Validate generation number. */ +if (VFS_I (ip)->i_generation != pp->pptr.p_gen) { +xchk_fblock_xref_set_corrupt (sc,XFS_ATTR_FORK ,0 ); + goto out_rele; + } + + *dpp = ip; +return 0 ; +out_rele: +xchk_irele (sc, ip); +return 0 ; +} + +/* +* Walk an xattr of a file. If this xattr is a parent pointer, follow it up +* to a parent directory and check that the parent has a dirent pointing back +* to us. +*/ +STATIC int +xchk_parent_scan_attr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct xfs_name dname = { + .name = value, + .len = valuelen, + }; +struct xchk_pptrs *pp = priv; +struct xfs_inode *dp =NULL ; +const struct xfs_parent_name_rec *rec = (const void *)name; +unsigned int lockmode; +xfs_dahash_t computed_hash; +int error; + +/* Ignore anything that isn't a parent pointer. */ +if (!(attr_flags &XFS_ATTR_PARENT )) +return 0 ; + +/* Does the ondisk parent pointer structure make sense? */ +if (!xfs_parent_namecheck (sc->mp, rec, namelen, attr_flags)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +if (!xfs_parent_valuecheck (sc->mp, value, valuelen)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +xfs_parent_irec_from_disk (&pp->pptr, rec, value, valuelen); + +if (!xfs_parent_verify_irec (sc->mp, &pp->pptr)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +/* No self-referential parent pointers. */ +if (pp->pptr.p_ino == sc->ip->i_ino) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + +/* +* If the namehash of the dirent name encoded in the parent pointer +* attr value doesn't match the namehash in the parent pointer key, +* the parent pointer is corrupt. +*/ + computed_hash =xfs_dir2_hashname (ip->i_mount, &dname); +if (pp->pptr.p_namehash != computed_hash) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK ,0 ); +return -ECANCELED ; + } + pp->pptrs_found++; + + error =xchk_parent_iget (pp, &dp); +if (error) +return error; +if (!dp) +return 0 ; + +/* Try to lock the inode. */ + lockmode =xchk_parent_lock_dir (sc, dp); +if (!lockmode) { +xchk_set_incomplete (sc); + error = -ECANCELED ; + goto out_rele; + } + + error =xchk_parent_dirent (pp, dp); +if (error) + goto out_unlock; + +out_unlock: +xfs_iunlock (dp, lockmode); +out_rele: +xchk_irele (sc, dp); +return error; +} + +/* +* Compare the number of parent pointers to the link count. For +* non-directories these should be the same. For unlinked directories the +* count should be zero; for linked directories, it should be nonzero. +*/ +STATIC int +xchk_parent_count_pptrs ( +struct xchk_pptrs *pp) +{ +struct xfs_scrub *sc = pp->sc; + +if (S_ISDIR (VFS_I (sc->ip)->i_mode)) { +if (sc->ip == sc->mp->m_rootip) + pp->pptrs_found++; + +if (VFS_I (sc->ip)->i_nlink ==0 && pp->pptrs_found >0 ) +xchk_ino_set_corrupt (sc, sc->ip->i_ino); +else if (VFS_I (sc->ip)->i_nlink >0 && + pp->pptrs_found ==0 ) +xchk_ino_set_corrupt (sc, sc->ip->i_ino); + }else { +if (VFS_I (sc->ip)->i_nlink != pp->pptrs_found) +xchk_ino_set_corrupt (sc, sc->ip->i_ino); + } + +return 0 ; +} + +/* Check parent pointers of a file. */ +STATIC int +xchk_parent_pptr ( +struct xfs_scrub *sc) +{ +struct xchk_pptrs *pp; +int error; + + pp =kvzalloc (sizeof (struct xchk_pptrs ),XCHK_GFP_FLAGS ); +if (!pp) +return -ENOMEM ; + pp->sc = sc; + + error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_attr, pp); +if (error == -ECANCELED ) { + error =0 ; + goto out_pp; + } +if (error) + goto out_pp; + +if (pp->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) + goto out_pp; + +/* +* For subdirectories, make sure the dotdot entry references the same +* inode as the parent pointers. +* +* If we're scanning a /consistent/ directory, there should only be +* one parent pointer, and it should point to the same directory as +* the dotdot entry. +* +* However, a corrupt directory tree might feature a subdirectory with +* multiple parents. The directory loop scanner is responsible for +* correcting that kind of problem, so for now we only validate that +* the dotdot entry matches /one/ of the parents. +*/ +if (S_ISDIR (VFS_I (sc->ip)->i_mode)) { + error =xchk_parent_pptr_and_dotdot (pp); +if (error) + goto out_pp; + } + +if (pp->sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) + goto out_pp; + +/* +* Complain if the number of parent pointers doesn't match the link +* count. This could be a sign of missing parent pointers (or an +* incorrect link count). +*/ + error =xchk_parent_count_pptrs (pp); +if (error) + goto out_pp; + +out_pp: +kvfree (pp); +return error; +} +/* Scrub a parent pointer. */ int xchk_parent(
@@ -206,6 +588,9 @@ xchk_parent(xfs_ino_t parent_ino;int error =0 ; +if (xfs_has_parent (mp)) +return xchk_parent_pptr (sc); +/* * If we're a directory, check that the '..' link points up to * a directory that has one entry pointing to us.
PATCH 5/23 xfs: deferred scrub of parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the trylock-based dirent check fails, retain those parent pointers and check them at the end. This may involve dropping the locks on the file being scanned, so yay.
--- fs/xfs/Makefile | 2 fs/xfs/scrub/parent.c | 263 ++++++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 3 + 3 files changed, 260 insertions(+), 8 deletions(-)
fs/xfs/Makefile +1 -1
@@ -178,6 +178,7 @@ xfs-y += $(addprefix scrub/, \ scrub.o \ symlink.o \ xfarray.o \ + xfblob.o \ xfile.o \ )
@@ -219,7 +220,6 @@ xfs-y += $(addprefix scrub/, \ rmap_repair.o \ symlink_repair.o \ tempfile.o \ - xfblob.o \ ) xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \
fs/xfs/scrub/parent.c +256 -7
336 lines not on this page · raw message
fs/xfs/scrub/trace.h +3 -0
@@ -1538,6 +1538,9 @@ DEFINE_EVENT(xchk_pptr_class, name, \DEFINE_XCHK_PPTR_EVENT (xchk_dir_defer);DEFINE_XCHK_PPTR_EVENT (xchk_dir_slowpath);DEFINE_XCHK_PPTR_EVENT (xchk_dir_ultraslowpath); +DEFINE_XCHK_PPTR_EVENT (xchk_parent_defer); +DEFINE_XCHK_PPTR_EVENT (xchk_parent_slowpath); +DEFINE_XCHK_PPTR_EVENT (xchk_parent_ultraslowpath);/* repair tracepoints */ #if IS_ENABLED (CONFIG_XFS_ONLINE_REPAIR )
PATCH 6/23 xfs: walk directory parent pointers to determine backref count
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If the filesystem has parent pointers enabled, walk the parent pointers of subdirectories to determine the true backref count. In theory each subdir should have a single parent reachable via dotdot, but in the case of (corrupt) subdirs with multiple parents, we need to keep the link counts high enough that the directory loop detector will be able to correct the multiple parents problems.
--- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/nlinks.c | 71 +++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/nlinks.h | 3 ++ fs/xfs/scrub/nlinks_repair.c | 2 + fs/xfs/scrub/parent.c | 61 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 27 ++++++++++++++++ 7 files changed, 165 insertions(+), 1 deletion(-)
fs/xfs/scrub/common.h +1 -0
@@ -212,6 +212,7 @@ static inline bool xchk_skip_xref(struct xfs_scrub_metadata *sm) }bool xchk_dir_looks_zapped (struct xfs_inode *dp); +bool xchk_pptr_looks_zapped (struct xfs_inode *ip);#ifdef CONFIG_XFS_ONLINE_REPAIR /* Decide if a repair is required. */
fs/xfs/scrub/nlinks.c +70 -1
106 lines not on this page · raw message
fs/xfs/scrub/nlinks.h +3 -0
@@ -23,6 +23,9 @@ struct xchk_nlink_ctrs {struct xchk_iscan collect_iscan;struct xchk_iscan compare_iscan; +/* Parent pointer for finding backrefs. */ +struct xfs_parent_name_irec pptr; +/* * Hook into directory updates so that we can receive live updates * from other writer threads.
fs/xfs/scrub/nlinks_repair.c +2 -0
@@ -18,6 +18,8 @@#include "xfs_ialloc.h" #include "xfs_sb.h" #include "xfs_ag.h" +#include "xfs_dir2.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/repair.h"
fs/xfs/scrub/parent.c +61 -0
64 lines not on this page · raw message
fs/xfs/scrub/trace.c +1 -0
@@ -19,6 +19,7 @@#include "xfs_da_format.h" #include "xfs_dir2.h" #include "xfs_rmap.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/xfile.h" #include "scrub/xfarray.h"
fs/xfs/scrub/trace.h +27 -0
@@ -26,6 +26,7 @@ struct xchk_iscan;struct xchk_nlink ;struct xchk_fscounters ;struct xfs_rmap_update_params ; +struct xfs_parent_name_irec ;/* * ftrace's __print_symbolic requires that all enum values be wrapped in the
@@ -1357,6 +1358,32 @@ TRACE_EVENT(xchk_nlinks_collect_dirent, __get_str(name)) ); +TRACE_EVENT (xchk_nlinks_collect_pptr, + TP_PROTO(struct xfs_mount *mp, struct xfs_inode *dp, + const struct xfs_parent_name_irec *pptr), +TP_ARGS (mp, dp, pptr), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, dir) + __field(xfs_ino_t, ino) + __field(unsigned int, namelen) +__dynamic_array (char, name, pptr->p_namelen) + ), +TP_fast_assign ( + __entry->dev = mp->m_super->s_dev; + __entry->dir = dp->i_ino; + __entry->ino = pptr->p_ino; + __entry->namelen = pptr->p_namelen; + memcpy(__get_str(name), pptr->p_name, pptr->p_namelen); + ), +TP_printk ("dev %d:%d dir 0x%llx -> ino 0x%llx name '%.*s'" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->dir, + __entry->ino, + __entry->namelen, +__get_str (name)) +); + TRACE_EVENT(xchk_nlinks_collect_metafile, TP_PROTO(struct xfs_mount *mp, xfs_ino_t ino), TP_ARGS(mp, ino),
PATCH 7/23 xfs: add raw parent pointer apis to support repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a couple of utility functions to set or remove parent pointers from a file. These functions will be used by repair code, hence they skip the xattr logging that regular parent pointer updates use.
--- fs/xfs/libxfs/xfs_dir2.c | 2 +- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_parent.c | 46 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 8 ++++++++ 4 files changed, 56 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +1 -1
@@ -440,7 +440,7 @@ intxfs_dir_removename ( struct xfs_trans *tp, struct xfs_inode *dp, - struct xfs_name *name, + const struct xfs_name *name, xfs_ino_t ino, xfs_extlen_t total) /* bmap's total block count */ {
fs/xfs/libxfs/xfs_dir2.h +1 -1
@@ -58,7 +58,7 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t *inum, struct xfs_name *ci_name);extern int xfs_dir_removename (struct xfs_trans *tp,struct xfs_inode *dp, -struct xfs_name *name,xfs_ino_t ino, +const struct xfs_name *name,xfs_ino_t ino,xfs_extlen_t tot); extern int xfs_dir_replace(struct xfs_trans *tp, struct xfs_inode *dp, const struct xfs_name *name, xfs_ino_t inum,
fs/xfs/libxfs/xfs_parent.c +46 -0
49 lines not on this page · raw message
fs/xfs/libxfs/xfs_parent.h +8 -0
@@ -162,4 +162,12 @@ int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip,const struct xfs_parent_name_irec *pptr, struct xfs_parent_scratch *scratch); +int xfs_parent_set (struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +struct xfs_parent_scratch *scratch); + +int xfs_parent_unset (struct xfs_inode *ip, +const struct xfs_parent_name_irec *rec, +struct xfs_parent_scratch *scratch); +#endif /* __XFS_PARENT_H__ */
PATCH 8/23 xfs: set child file owner in xfs_da_args when changing parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that struct xfs_da_args has an explicit file owner field, we must set it when modifying parent pointers.
--- fs/xfs/libxfs/xfs_parent.c | 13 ++++++++++--- fs/xfs/libxfs/xfs_parent.h | 4 ++-- 2 files changed, 12 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_parent.c +10 -3
@@ -201,6 +201,7 @@ xfs_parent_addname( args->trans = tp; args->dp = child; + args->owner = child->i_ino;xfs_init_parent_davalue (&ppargs->args, parent_name);
@@ -239,6 +240,7 @@ xfs_parent_removename( args->trans = tp; args->dp = child; + args->owner = child->i_ino;xfs_init_parent_davalue (&ppargs->args, parent_name);
@@ -288,6 +290,7 @@ xfs_parent_replacename( args->trans = tp; args->dp = child; + args->owner = child->i_ino;xfs_init_parent_davalue (&ppargs->args, old_name);xfs_init_parent_danewvalue (&ppargs->args, new_name);
@@ -371,6 +374,7 @@ static inline voidxfs_parent_scratch_init ( struct xfs_trans *tp, struct xfs_inode *ip, + xfs_ino_t owner, const struct xfs_parent_name_irec *pptr, struct xfs_parent_scratch *scr) {
@@ -387,6 +391,7 @@ xfs_parent_scratch_init( scr->args.whichfork =XFS_ATTR_FORK ; scr->args.hashval =xfs_da_hashname ((const void *)&scr->rec,sizeof (struct xfs_parent_name_rec )); + scr->args.owner = owner; }/*
@@ -415,7 +420,7 @@ xfs_parent_lookup( }xfs_parent_irec_to_disk (&scr->rec, pptr); -xfs_parent_scratch_init (tp, ip, pptr, scr); +xfs_parent_scratch_init (tp, ip, ip->i_ino, pptr, scr); scr->args.op_flags |=XFS_DA_OP_OKNOENT ;return xfs_attr_get_ilocked (&scr->args);
@@ -429,6 +434,7 @@ xfs_parent_lookup(int xfs_parent_set (struct xfs_inode *ip, +xfs_ino_t owner,const struct xfs_parent_name_irec *pptr,struct xfs_parent_scratch *scr) {
@@ -438,7 +444,7 @@ xfs_parent_set( }xfs_parent_irec_to_disk (&scr->rec, pptr); -xfs_parent_scratch_init (NULL , ip, pptr, scr); +xfs_parent_scratch_init (NULL , ip, owner, pptr, scr); scr->args.op_flags |=XFS_DA_OP_LOGGED ;return xfs_attr_set (&scr->args);
@@ -452,6 +458,7 @@ xfs_parent_set(int xfs_parent_unset (struct xfs_inode *ip, +xfs_ino_t owner,const struct xfs_parent_name_irec *pptr,struct xfs_parent_scratch *scr) {
@@ -461,7 +468,7 @@ xfs_parent_unset( }xfs_parent_irec_to_disk (&scr->rec, pptr); -xfs_parent_scratch_init (NULL , ip, pptr, scr); +xfs_parent_scratch_init (NULL , ip, owner, pptr, scr); scr->args.op_flags |=XFS_DA_OP_LOGGED |XFS_DA_OP_REMOVE ;return xfs_attr_set (&scr->args);
fs/xfs/libxfs/xfs_parent.h +2 -2
@@ -162,11 +162,11 @@ int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip,const struct xfs_parent_name_irec *pptr, struct xfs_parent_scratch *scratch); -int xfs_parent_set (struct xfs_inode *ip, +int xfs_parent_set (struct xfs_inode *ip,xfs_ino_t owner,const struct xfs_parent_name_irec *pptr,struct xfs_parent_scratch *scratch); -int xfs_parent_unset (struct xfs_inode *ip, +int xfs_parent_unset (struct xfs_inode *ip,xfs_ino_t owner,const struct xfs_parent_name_irec *rec,struct xfs_parent_scratch *scratch);
PATCH 9/23 xfs: check parent pointer xattrs when scrubbing
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Check parent pointer xattrs as part of scrubbing xattrs.
--- fs/xfs/scrub/attr.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/xfs/scrub/attr.c +14 -0
@@ -17,6 +17,7 @@#include "xfs_attr.h" #include "xfs_attr_leaf.h" #include "xfs_attr_sf.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/dabtree.h"
@@ -205,6 +206,19 @@ xchk_xattr_actor(return -ECANCELED ; } +/* Check parent pointer geometry. */ +if (attr_flags &XFS_ATTR_PARENT ) { +if (!xfs_has_parent (sc->mp)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno); +return -ECANCELED ; + } + +if (!xfs_parent_valuecheck (sc->mp, value, valuelen)) { +xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno); +return -ECANCELED ; + } + } +/* Does this name make sense? */ if (!xfs_attr_namecheck (sc->mp, name, namelen, attr_flags)) {xchk_fblock_set_corrupt (sc,XFS_ATTR_FORK , args.blkno);
PATCH 10/23 xfs: salvage parent pointers when rebuilding xattr structures
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When we're salvaging extended attributes, make sure we validate the ones that claim to be parent pointers before adding them to the salvage pile.
--- fs/xfs/scrub/attr_repair.c | 42 +++++++++++++++++++++++++++++++++--------- fs/xfs/scrub/trace.h | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+), 9 deletions(-)
fs/xfs/scrub/attr_repair.c +33 -9
@@ -28,6 +28,7 @@#include "xfs_exchmaps.h" #include "xfs_exchrange.h" #include "xfs_acl.h" +#include "xfs_parent.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h"
@@ -127,6 +128,14 @@ xrep_xattr_want_salvage(return false;if (hweight32 (attr_flags &XFS_ATTR_NSP_ONDISK_MASK ) >1 )return false; +if (attr_flags &XFS_ATTR_PARENT ) { +if (!xfs_has_parent (rx->sc->mp)) +return false; +if (!xfs_parent_namecheck (rx->sc->mp, name, namelen, + attr_flags)) +return false; +return xfs_parent_valuecheck (rx->sc->mp, value, valuelen); + }return true; }
@@ -154,14 +163,21 @@ xrep_xattr_salvage_key(* Truncate the name to the first character that would trip namecheck. * If we no longer have a name after that, ignore this attribute. */ -while (i < namelen && name[i] !=0 ) - i++; -if (i ==0 ) -return 0 ; - key.namelen = i; +if (flags &XFS_ATTR_PARENT ) { + key.namelen = namelen; -trace_xrep_xattr_salvage_rec (rx->sc->ip, flags, name, key.namelen, - valuelen); +trace_xrep_xattr_salvage_pptr (rx->sc->ip, flags, name, + key.namelen, value, valuelen); + }else { +while (i < namelen && name[i] !=0 ) + i++; +if (i ==0 ) +return 0 ; + key.namelen = i; + +trace_xrep_xattr_salvage_rec (rx->sc->ip, flags, name, + key.namelen, valuelen); + } error =xfblob_store (rx->xattr_blobs, &key.name_cookie, name, key.namelen);
@@ -565,6 +581,9 @@ xrep_xattr_insert_rec(struct xchk_xattr_buf *ab = rx->sc->buf;int error; +if (key->flags &XFS_ATTR_PARENT ) + args.op_flags |=XFS_DA_OP_NVLOOKUP ; +/* * Grab pointers to the scrub buffer so that we can use them to insert * attrs into the temp file.
@@ -598,8 +617,13 @@ xrep_xattr_insert_rec( ab->name[key->namelen] =0 ; -trace_xrep_xattr_insert_rec (rx->sc->tempip, key->flags, ab->name, - key->namelen, key->valuelen); +if (key->flags &XFS_ATTR_PARENT ) +trace_xrep_xattr_insert_pptr (rx->sc->tempip, key->flags, + ab->name, key->namelen, ab->value, + key->valuelen); +else +trace_xrep_xattr_insert_rec (rx->sc->tempip, key->flags, + ab->name, key->namelen, key->valuelen);/* * xfs_attr_set creates and commits its own transaction. If the attr
fs/xfs/scrub/trace.h +40 -0
@@ -2533,6 +2533,46 @@ DEFINE_EVENT(xrep_xattr_salvage_class, name, \DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_xattr_salvage_rec);DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_xattr_insert_rec); +DECLARE_EVENT_CLASS (xrep_pptr_salvage_class, + TP_PROTO(struct xfs_inode *ip, unsigned int flags, const void *name, + unsigned int namelen, const void *value, unsigned int valuelen), +TP_ARGS (ip, flags, name, namelen, value, valuelen), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(xfs_ino_t, parent_ino) + __field(unsigned int, parent_gen) + __field(unsigned int, namelen) +__dynamic_array (char, name, valuelen) + ), +TP_fast_assign ( + struct xfs_parent_name_irec pptr; + + xfs_parent_irec_from_disk(&pptr, name, value, valuelen); + + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->parent_ino = pptr.p_ino; + __entry->parent_gen = pptr.p_gen; + __entry->namelen = pptr.p_namelen; + memcpy(__get_str(name), pptr.p_name, pptr.p_namelen); + ), +TP_printk ("dev %d:%d ino 0x%llx parent_ino 0x%llx parent_gen 0x%x name '%.*s'" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->parent_ino, + __entry->parent_gen, + __entry->namelen, +__get_str (name)) +) +#define DEFINE_XREP_PPTR_SALVAGE_EVENT (name) \ +DEFINE_EVENT(xrep_pptr_salvage_class, name, \ + TP_PROTO(struct xfs_inode *ip, unsigned int flags, const void *name, \ + unsigned int namelen, const void *value, unsigned int valuelen), \ + TP_ARGS(ip, flags, name, namelen, value, valuelen)) +DEFINE_XREP_PPTR_SALVAGE_EVENT (xrep_xattr_salvage_pptr); +DEFINE_XREP_PPTR_SALVAGE_EVENT (xrep_xattr_insert_pptr); + TRACE_EVENT(xrep_xattr_class, TP_PROTO(struct xfs_inode *ip, struct xfs_inode *arg_ip), TP_ARGS(ip, arg_ip),
PATCH 11/23 xfs: replace namebuf with parent pointer in directory repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Replace the dirent name buffer at the end of struct xrep_dir with a xfs_parent_name_irec object. The namebuf and p_name usage do not overlap, so we can save 256 bytes of memory by allowing them to overlap. Doing so makes the code a bit more complex, so this is called out separately.
--- fs/xfs/scrub/dir_repair.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
fs/xfs/scrub/dir_repair.c +13 -6
@@ -28,6 +28,7 @@#include "xfs_exchmaps.h" #include "xfs_exchrange.h" #include "xfs_ag.h" +#include "xfs_parent.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h"
@@ -131,8 +132,14 @@ struct xrep_dir {/* Should we move this directory to the orphanage? */ bool needs_adoption; -/* Directory entry name, plus the trailing null. */ -unsigned char namebuf[MAXNAMELEN ]; +/* +* Scratch buffer for reading parent pointers from child files. The +* p_name field is used to flush stashed dirents into the temporary +* directory in between parent pointers. At the very end of the +* repair, it can also be used to compute the lost+found filename +* if we need to reparent the directory. +*/ +struct xfs_parent_name_irec pptr; };/* Tear down all the incore stuff we created. */
@@ -696,7 +703,7 @@ xrep_dir_replay_update(struct xfs_name name = { .len = dirent->namelen, .type = dirent->ftype, - .name = rd->namebuf, + .name = rd->pptr.p_name, };struct xfs_mount *mp = rd->sc->mp;#ifdef DEBUG
@@ -773,10 +780,10 @@ xrep_dir_replay_updates(/* The dirent name is stored in the in-core buffer. */ error =xfblob_load (rd->dir_names, dirent.name_cookie, - rd->namebuf, dirent.namelen); + rd->pptr.p_name, dirent.namelen);if (error)return error; - rd->namebuf[MAXNAMELEN -1 ] =0 ; + rd->pptr.p_name[MAXNAMELEN -1 ] =0 ; error =xrep_dir_replay_update (rd, &dirent);if (error)
@@ -1421,7 +1428,7 @@ xrep_dir_move_to_orphanage(if (error)return error; - error =xrep_adoption_compute_name (&rd->adoption, rd->namebuf); + error =xrep_adoption_compute_name (&rd->adoption, rd->pptr.p_name);if (error)return error;
PATCH 12/23 xfs: repair directories by scanning directory parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
For filesystems with parent pointers, scan the entire filesystem looking for parent pointers that target the directory we're rebuilding instead of trying to salvage whatever we can from the directory data blocks. This will be more robust than salvaging, but there's more code to come.
--- fs/xfs/scrub/dir_repair.c | 328 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 324 insertions(+), 4 deletions(-)
fs/xfs/scrub/dir_repair.c +324 -4
@@ -44,6 +44,7 @@#include "scrub/reap.h" #include "scrub/findparent.h" #include "scrub/orphanage.h" +#include "scrub/listxattr.h" /* * Directory Repair
@@ -58,6 +59,15 @@* being repaired and the temporary directory, and will later become important * for parent pointer scanning. * +* If parent pointers are enabled on this filesystem, we instead reconstruct +* the directory by visiting each parent pointer of each file in the filesystem +* and translating the relevant parent pointer records into dirents. In this +* case, it is advantageous to stash all directory entries created from parent +* pointers for a single child file before replaying them into the temporary +* directory. To save memory, the live filesystem scan reuses the findparent +* fields. Directory repair chooses either parent pointer scanning or +* directory entry salvaging, but not both. +* * Directory entries added to the temporary directory do not elevate the link * counts of the inodes found. When salvaging completes, the remaining stashed * entries are replayed to the temporary directory. An atomic mapping exchange
@@ -113,7 +123,15 @@ struct xrep_dir {/* * Information used to scan the filesystem to find the inumber of the -* dotdot entry for this directory. +* dotdot entry for this directory. For directory salvaging when +* parent pointers are not enabled, we use the findparent_* functions +* on this object and access only the parent_ino field directly. +* +* When parent pointers are enabled, however, the pptr scanner uses the +* iscan, hooks, lock, and parent_ino fields of this object directly. +* @pscan.lock coordinates access to dir_entries, dir_names, +* parent_ino, subdirs, dirents, and args. This reduces the memory +* requirements of this structure. */ struct xrep_parent_scan_info pscan;
@@ -1004,6 +1022,262 @@ xrep_dir_salvage_entries( } +/* +* Examine a parent pointer of a file. If it leads us back to the directory +* that we're rebuilding, create an incore dirent from the parent pointer and +* stash it. +*/ +STATIC int +xrep_dir_scan_pptr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +void *priv) +{ +struct xfs_name xname; +struct xrep_dir *rd = priv; +int error; + +/* +* Ignore parent pointers that point back to a different dir, list the +* wrong generation number, or are invalid. +*/ +if (pptr->p_ino != sc->ip->i_ino || + pptr->p_gen !=VFS_I (sc->ip)->i_generation || + !xfs_parent_verify_irec (sc->mp, pptr)) +return 0 ; + +/* +* Transform this parent pointer into a dirent and queue it for later +* addition to the temporary directory. +*/ + xname.name = pptr->p_name; + xname.len = pptr->p_namelen; + xname.type =xfs_mode_to_ftype (VFS_I (ip)->i_mode); + +mutex_lock (&rd->pscan.lock); + error =xrep_dir_stash_createname (rd, &xname, ip->i_ino); +mutex_unlock (&rd->pscan.lock); +return error; +} + +/* +* If this child dirent points to the directory being repaired, remember that +* fact so that we can reset the dotdot entry if necessary. +*/ +STATIC int +xrep_dir_scan_dirent ( +struct xfs_scrub *sc, +struct xfs_inode *dp, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +xfs_ino_t ino, +void *priv) +{ +struct xrep_dir *rd = priv; + +/* Dirent doesn't point to this directory. */ +if (ino != rd->sc->ip->i_ino) +return 0 ; + +/* Ignore garbage inum. */ +if (!xfs_verify_dir_ino (rd->sc->mp, ino)) +return 0 ; + +/* No weird looking names. */ +if (name->len >=MAXNAMELEN || name->len <=0 ) +return 0 ; + +/* Don't pick up dot or dotdot entries; we only want child dirents. */ +if (xfs_dir2_samename (name, &xfs_name_dotdot) || +xfs_dir2_samename (name, &xfs_name_dot)) +return 0 ; + +trace_xrep_dir_stash_createname (sc->tempip, &xfs_name_dotdot, + dp->i_ino); + +xrep_findparent_scan_found (&rd->pscan, dp->i_ino); +return 0 ; +} + +/* +* Decide if we want to look for child dirents or parent pointers in this file. +* Skip the dir being repaired and any files being used to stage repairs. +*/ +static inline bool +xrep_dir_want_scan ( +struct xrep_dir *rd, +const struct xfs_inode *ip) +{ +return ip != rd->sc->ip && !xrep_is_tempfile (ip); +} + +/* +* Take ILOCK on a file that we want to scan. +* +* Select ILOCK_EXCL if the file is a directory with an unloaded data bmbt or +* has an unloaded attr bmbt. Otherwise, take ILOCK_SHARED. +*/ +static inline unsigned int +xrep_dir_scan_ilock ( +struct xrep_dir *rd, +struct xfs_inode *ip) +{ +uint lock_mode =XFS_ILOCK_SHARED ; + +/* Need to take the shared ILOCK to advance the iscan cursor. */ +if (!xrep_dir_want_scan (rd, ip)) + goto lock; + +if (S_ISDIR (VFS_I (ip)->i_mode) &&xfs_need_iread_extents (&ip->i_df)) { + lock_mode =XFS_ILOCK_EXCL ; + goto lock; + } + +if (xfs_inode_has_attr_fork (ip) &&xfs_need_iread_extents (&ip->i_af)) + lock_mode =XFS_ILOCK_EXCL ; + +lock: +xfs_ilock (ip, lock_mode); +return lock_mode; +} + +/* +* Scan this file for relevant child dirents or parent pointers that point to +* the directory we're rebuilding. +*/ +STATIC int +xrep_dir_scan_file ( +struct xrep_dir *rd, +struct xfs_inode *ip) +{ +unsigned int lock_mode; +int error =0 ; + + lock_mode =xrep_dir_scan_ilock (rd, ip); + +if (!xrep_dir_want_scan (rd, ip)) + goto scan_done; + +/* +* If the extended attributes look as though they has been zapped by +* the inode record repair code, we cannot scan for parent pointers. +*/ +if (xchk_pptr_looks_zapped (ip)) { + error = -EBUSY ; + goto scan_done; + } + + error =xchk_pptr_walk (rd->sc, ip, xrep_dir_scan_pptr, &rd->pptr, rd); +if (error) + goto scan_done; + +if (S_ISDIR (VFS_I (ip)->i_mode)) { +/* +* If the directory looks as though it has been zapped by the +* inode record repair code, we cannot scan for child dirents. +*/ +if (xchk_dir_looks_zapped (ip)) { + error = -EBUSY ; + goto scan_done; + } + + error =xchk_dir_walk (rd->sc, ip, xrep_dir_scan_dirent, rd); +if (error) + goto scan_done; + } + +scan_done: +xchk_iscan_mark_visited (&rd->pscan.iscan, ip); +xfs_iunlock (ip, lock_mode); +return error; +} + +/* +* Scan all files in the filesystem for parent pointers that we can turn into +* replacement dirents, and a dirent that we can use to set the dotdot pointer. +*/ +STATIC int +xrep_dir_scan_dirtree ( +struct xrep_dir *rd) +{ +struct xfs_scrub *sc = rd->sc; +struct xfs_inode *ip; +int error; + +/* Roots of directory trees are their own parents. */ +if (sc->ip == sc->mp->m_rootip) +xrep_findparent_scan_found (&rd->pscan, sc->ip->i_ino); + +/* +* Filesystem scans are time consuming. Drop the directory ILOCK and +* all other resources for the duration of the scan and hope for the +* best. The live update hooks will keep our scan information up to +* date even though we've dropped the locks. +*/ +xchk_trans_cancel (sc); +if (sc->ilock_flags & (XFS_ILOCK_SHARED |XFS_ILOCK_EXCL )) +xchk_iunlock (sc, sc->ilock_flags & (XFS_ILOCK_SHARED | +XFS_ILOCK_EXCL )); + error =xchk_trans_alloc_empty (sc); +if (error) +return error; + +while ((error =xchk_iscan_iter (&rd->pscan.iscan, &ip)) ==1 ) { +bool flush; + + error =xrep_dir_scan_file (rd, ip); +xchk_irele (sc, ip); +if (error) +break ; + +/* Flush stashed dirent updates to constrain memory usage. */ +mutex_lock (&rd->pscan.lock); + flush =xrep_dir_want_flush_stashed (rd); +mutex_unlock (&rd->pscan.lock); +if (flush) { +xchk_trans_cancel (sc); + + error =xrep_tempfile_iolock_polled (sc); +if (error) +break ; + +mutex_lock (&rd->pscan.lock); + error =xrep_dir_replay_updates (rd); +mutex_unlock (&rd->pscan.lock); +xrep_tempfile_iounlock (sc); +if (error) +break ; + + error =xchk_trans_alloc_empty (sc); +if (error) +break ; + } + +if (xchk_should_terminate (sc, &error)) +break ; + } +xchk_iscan_iter_finish (&rd->pscan.iscan); +if (error) { +/* +* If we couldn't grab an inode that was busy with a state +* change, change the error code so that we exit to userspace +* as quickly as possible. +*/ +if (error == -EBUSY ) +return -ECANCELED ; +return error; + } + +/* +* Cancel the empty transaction so that we can (later) use the atomic +* file mapping exchange functions to lock files and commit the new +* directory. +*/ +xchk_trans_cancel (rd->sc); +return 0 ; +} +/* * Free all the directory blocks and reset the data fork. The caller must * join the inode to the transaction. This function returns with the inode
@@ -1203,6 +1477,45 @@ xrep_dir_set_nlink(return 0 ; } +/* +* Finish replaying stashed dirent updates, allocate a transaction for +* exchanging data fork mappings, and take the ILOCKs of both directories +* before we commit the new directory structure. +*/ +STATIC int +xrep_dir_finalize_tempdir ( +struct xrep_dir *rd) +{ +struct xfs_scrub *sc = rd->sc; +int error; + +if (!xfs_has_parent (sc->mp)) +return xrep_tempexch_trans_alloc (sc,XFS_DATA_FORK , &rd->tx); + +/* +* Repair relies on the ILOCK to quiesce all possible dirent updates. +* Replay all queued dirent updates into the tempdir before exchanging +* the contents, even if that means dropping the ILOCKs and the +* transaction. +*/ +do { + error =xrep_dir_replay_updates (rd); +if (error) +return error; + + error =xrep_tempexch_trans_alloc (sc,XFS_DATA_FORK , &rd->tx); +if (error) +return error; + +if (xfarray_length (rd->dir_entries) ==0 ) +break ; + +xchk_trans_cancel (sc); +xrep_tempfile_iunlock_both (sc); + }while (!xchk_should_terminate (sc, &error)); +return error; +} +/* Exchange the temporary directory's data fork with the one being repaired. */ STATIC int xrep_dir_swap(
@@ -1305,8 +1618,12 @@ xrep_dir_rebuild_tree(if (error)return error; -/* Allocate transaction and ILOCK the scrub file and the temp file. */ - error =xrep_tempexch_trans_alloc (sc,XFS_DATA_FORK , &rd->tx); +/* +* Allocate transaction, lock inodes, and make sure that we've replayed +* all the stashed dirent updates to the tempdir. After this point, +* we're ready to exchange data fork mappings. +*/ + error =xrep_dir_finalize_tempdir (rd);if (error)return error;
@@ -1491,7 +1808,10 @@ xrep_directory(if (error)return error; - error =xrep_dir_salvage_entries (rd); +if (xfs_has_parent (sc->mp)) + error =xrep_dir_scan_dirtree (rd); +else + error =xrep_dir_salvage_entries (rd);if (error) goto out_teardown;
PATCH 13/23 xfs: implement live updates for directory repairs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
While we're scanning the filesystem for parent pointers that we can turn into dirents, we cannot hold the IOLOCK or ILOCK of the directory being repaired. Therefore, we need to set up a dirent hook so that we can keep the temporary directory up to date with the rest of the filesystem. Hence we add the ability to *remove* entries from the temporary dir.
--- fs/xfs/scrub/dir_repair.c | 221 +++++++++++++++++++++++++++++++++++++++++---- fs/xfs/scrub/findparent.c | 10 +- fs/xfs/scrub/findparent.h | 10 ++ fs/xfs/scrub/trace.h | 2 4 files changed, 220 insertions(+), 23 deletions(-)
fs/xfs/scrub/dir_repair.c +203 -18
@@ -85,6 +85,12 @@* other threads. */ +/* Create a dirent in the tempdir. */ +#define XREP_DIRENT_ADD (1) + +/* Remove a dirent from the tempdir. */ +#define XREP_DIRENT_REMOVE (2) +/* Directory entry to be restored in the new directory. */ struct xrep_dirent {/* Cookie for retrieval of the dirent name. */
@@ -98,6 +104,9 @@ struct xrep_dirent {/* File type of the dirent. */ uint8_t ftype; + +/* XREP_DIRENT_{ADD,REMOVE} */ +uint8_t action; };/*
@@ -341,6 +350,7 @@ xrep_dir_stash_createname( xfs_ino_t ino) {struct xrep_dirent dirent = { + .action =XREP_DIRENT_ADD , .ino = ino, .namelen = name->len, .ftype = name->type,
@@ -357,6 +367,34 @@ xrep_dir_stash_createname(return xfarray_append (rd->dir_entries, &dirent); } +/* +* Remember that we want to remove a dirent from the tempdir. These stashed +* actions will be replayed later. +*/ +STATIC int +xrep_dir_stash_removename ( +struct xrep_dir *rd, +const struct xfs_name *name, +xfs_ino_t ino) +{ +struct xrep_dirent dirent = { + .action =XREP_DIRENT_REMOVE , + .ino = ino, + .namelen = name->len, + .ftype = name->type, + }; +int error; + +trace_xrep_dir_stash_removename (rd->sc->tempip, name, ino); + + error =xfblob_store (rd->dir_names, &dirent.name_cookie, name->name, + name->len); +if (error) +return error; + +return xfarray_append (rd->dir_entries, &dirent); +} +/* Allocate an in-core record to hold entries while we rebuild the dir data. */ STATIC int xrep_dir_salvage_entry(
@@ -708,6 +746,43 @@ xrep_dir_replay_createname(return xfs_dir2_node_addname (&rd->args); } +/* Replay a stashed removename onto the temporary directory. */ +STATIC int +xrep_dir_replay_removename ( +struct xrep_dir *rd, +const struct xfs_name *name, +xfs_extlen_t total) +{ +struct xfs_inode *dp = rd->args.dp; +bool is_block, is_leaf; +int error; + +ASSERT (S_ISDIR (VFS_I (dp)->i_mode)); + +xrep_dir_init_args (rd, dp, name); + rd->args.op_flags =0 ; + rd->args.total = total; + +trace_xrep_dir_replay_removename (dp, name,0 ); + +if (dp->i_df.if_format ==XFS_DINODE_FMT_LOCAL ) +return xfs_dir2_sf_removename (&rd->args); + + error =xfs_dir2_isblock (&rd->args, &is_block); +if (error) +return error; +if (is_block) +return xfs_dir2_block_removename (&rd->args); + + error =xfs_dir2_isleaf (&rd->args, &is_leaf); +if (error) +return error; +if (is_leaf) +return xfs_dir2_leaf_removename (&rd->args); + +return xfs_dir2_node_removename (&rd->args); +} +/* * Add this stashed incore directory entry to the temporary directory. * The caller must hold the tempdir's IOLOCK, must not hold any ILOCKs, and
@@ -739,26 +814,64 @@ xrep_dir_replay_update(xrep_tempfile_ilock (rd->sc);xfs_trans_ijoin (rd->sc->tp, rd->sc->tempip,0 ); -/* -* Create a replacement dirent in the temporary directory. Note that -* _createname doesn't check for existing entries. There shouldn't be -* any in the temporary dir, but we'll verify this in debug mode. -*/ +switch (dirent->action) { +case XREP_DIRENT_ADD : +/* +* Create a replacement dirent in the temporary directory. +* Note that _createname doesn't check for existing entries. +* There shouldn't be any in the temporary dir, but we'll +* verify this in debug mode. +*/ #ifdef DEBUG - error =xchk_dir_lookup (rd->sc, rd->sc->tempip, &name, &ino); -if (error != -ENOENT ) { -ASSERT (error != -ENOENT ); + error =xchk_dir_lookup (rd->sc, rd->sc->tempip, &name, &ino); +if (error != -ENOENT ) { +ASSERT (error != -ENOENT ); + goto out_cancel; + } +#endif + + error =xrep_dir_replay_createname (rd, &name, dirent->ino, + resblks); +if (error) + goto out_cancel; + +if (name.type ==XFS_DIR3_FT_DIR ) + rd->subdirs++; + rd->dirents++; +break ; +case XREP_DIRENT_REMOVE : +/* +* Remove a dirent from the temporary directory. Note that +* _removename doesn't check the inode target of the exist +* entry. There should be a perfect match in the temporary +* dir, but we'll verify this in debug mode. +*/ +#ifdef DEBUG + error =xchk_dir_lookup (rd->sc, rd->sc->tempip, &name, &ino); +if (error) { +ASSERT (error !=0 ); + goto out_cancel; + } +if (ino != dirent->ino) { +ASSERT (ino == dirent->ino); + error = -EIO ; + goto out_cancel; + } +#endif + + error =xrep_dir_replay_removename (rd, &name, resblks); +if (error) + goto out_cancel; + +if (name.type ==XFS_DIR3_FT_DIR ) + rd->subdirs--; + rd->dirents--; +break ; +default : +ASSERT (0 ); + error = -EIO ; goto out_cancel; } -#endif - - error =xrep_dir_replay_createname (rd, &name, dirent->ino, resblks); -if (error) - goto out_cancel; - -if (name.type ==XFS_DIR3_FT_DIR ) - rd->subdirs++; - rd->dirents++;/* Commit and unlock. */ error =xrep_trans_commit (rd->sc);
@@ -1278,6 +1391,71 @@ xrep_dir_scan_dirtree(return 0 ; } +/* +* Capture dirent updates being made by other threads which are relevant to the +* directory being repaired. +*/ +STATIC int +xrep_dir_live_update ( +struct notifier_block *nb, +unsigned long action, +void *data) +{ +struct xfs_dir_update_params *p = data; +struct xrep_dir *rd; +struct xfs_scrub *sc; +int error =0 ; + + rd = container_of(nb, struct xrep_dir, pscan.dhook.dirent_hook.nb); + sc = rd->sc; + +/* +* This thread updated a child dirent in the directory that we're +* rebuilding. Stash the update for replay against the temporary +* directory. +*/ +if (p->dp->i_ino == sc->ip->i_ino && +xchk_iscan_want_live_update (&rd->pscan.iscan, p->ip->i_ino)) { +mutex_lock (&rd->pscan.lock); +if (p->delta >0 ) + error =xrep_dir_stash_createname (rd, p->name, + p->ip->i_ino); +else + error =xrep_dir_stash_removename (rd, p->name, + p->ip->i_ino); +mutex_unlock (&rd->pscan.lock); +if (error) + goto out_abort; + } + +/* +* This thread updated another directory's child dirent that points to +* the directory that we're rebuilding, so remember the new dotdot +* target. +*/ +if (p->ip->i_ino == sc->ip->i_ino && +xchk_iscan_want_live_update (&rd->pscan.iscan, p->dp->i_ino)) { +if (p->delta >0 ) { +trace_xrep_dir_stash_createname (sc->tempip, + &xfs_name_dotdot, + p->dp->i_ino); + +xrep_findparent_scan_found (&rd->pscan, p->dp->i_ino); + }else { +trace_xrep_dir_stash_removename (sc->tempip, + &xfs_name_dotdot, + rd->pscan.parent_ino); + +xrep_findparent_scan_found (&rd->pscan,NULLFSINO ); + } + } + +return NOTIFY_DONE ; +out_abort: +xchk_iscan_abort (&rd->pscan.iscan); +return NOTIFY_DONE ; +} +/* * Free all the directory blocks and reset the data fork. The caller must * join the inode to the transaction. This function returns with the inode
@@ -1627,6 +1805,9 @@ xrep_dir_rebuild_tree(if (error)return error; +if (xchk_iscan_aborted (&rd->pscan.iscan)) +return -ECANCELED ; +/* * Exchange the tempdir's data fork with the file being repaired. This * recreates the transaction and re-takes the ILOCK in the scrub
@@ -1682,7 +1863,11 @@ xrep_dir_setup_scan(if (error) goto out_xfarray; - error =xrep_findparent_scan_start (sc, &rd->pscan); +if (xfs_has_parent (sc->mp)) + error =__xrep_findparent_scan_start (sc, &rd->pscan, + xrep_dir_live_update); +else + error =xrep_findparent_scan_start (sc, &rd->pscan);if (error) goto out_xfblob;
fs/xfs/scrub/findparent.c +7 -3
@@ -238,9 +238,10 @@ xrep_findparent_live_update(* will be called when there is a dotdot update for the inode being repaired. */ int -xrep_findparent_scan_start ( +__xrep_findparent_scan_start (struct xfs_scrub *sc, -struct xrep_parent_scan_info *pscan) +struct xrep_parent_scan_info *pscan, +notifier_fn_t custom_fn) {int error;
@@ -262,7 +263,10 @@ xrep_findparent_scan_start(* ILOCK, which means that any in-progress inode updates will finish * before we can scan the inode. */ -xfs_dir_hook_setup (&pscan->dhook, xrep_findparent_live_update); +if (custom_fn) +xfs_dir_hook_setup (&pscan->dhook, custom_fn); +else +xfs_dir_hook_setup (&pscan->dhook, xrep_findparent_live_update); error =xfs_dir_hook_add (sc->mp, &pscan->dhook);if (error) goto out_iscan;
fs/xfs/scrub/findparent.h +8 -2
@@ -24,8 +24,14 @@ struct xrep_parent_scan_info {bool lookup_parent; }; -int xrep_findparent_scan_start (struct xfs_scrub *sc, -struct xrep_parent_scan_info *pscan); +int __xrep_findparent_scan_start (struct xfs_scrub *sc, +struct xrep_parent_scan_info *pscan, +notifier_fn_t custom_fn); +static inline int xrep_findparent_scan_start (struct xfs_scrub *sc, +struct xrep_parent_scan_info *pscan) +{ +return __xrep_findparent_scan_start (sc, pscan,NULL ); +}int xrep_findparent_scan (struct xrep_parent_scan_info *pscan);void xrep_findparent_scan_teardown (struct xrep_parent_scan_info *pscan);
fs/xfs/scrub/trace.h +2 -0
@@ -2687,6 +2687,8 @@ DEFINE_XREP_DIRENT_EVENT(xrep_dir_salvage_entry);DEFINE_XREP_DIRENT_EVENT (xrep_dir_stash_createname);DEFINE_XREP_DIRENT_EVENT (xrep_dir_replay_createname);DEFINE_XREP_DIRENT_EVENT (xrep_adoption_reparent); +DEFINE_XREP_DIRENT_EVENT (xrep_dir_stash_removename); +DEFINE_XREP_DIRENT_EVENT (xrep_dir_replay_removename); DECLARE_EVENT_CLASS(xrep_adoption_class, TP_PROTO(struct xfs_inode *dp, struct xfs_inode *ip, bool moved),
PATCH 14/23 xfs: replay unlocked parent pointer updates that accrue during xattr repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
There are a few places where the extended attribute repair code drops the ILOCK to apply stashed xattrs to the temporary file. Although setxattr and removexattr are still locked out because we retain our hold on the IOLOCK, this doesn't prevent renames from updating parent pointers, because the VFS doesn't take i_rwsem on children that are being moved.
Therefore, set up a dirent hook to capture parent pointer updates for this file, and replay(?) the updates.
--- fs/xfs/scrub/attr_repair.c | 456 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 71 +++++++ 2 files changed, 525 insertions(+), 2 deletions(-)
fs/xfs/scrub/attr_repair.c +454 -2
@@ -96,6 +96,56 @@ struct xrep_xattr {/* Number of attributes that we are salvaging. */ unsigned long long attrs_found; + +/* Can we flush stashed attrs to the tempfile? */ +bool can_flush; + +/* Did the live update fail, and hence the repair is now out of date? */ +bool live_update_aborted; + +/* Lock protecting parent pointer updates */ +struct mutex lock; + +/* Fixed-size array of xrep_xattr_pptr structures. */ +struct xfarray *pptr_recs; + +/* Blobs containing parent pointer names. */ +struct xfblob *pptr_names; + +/* Hook to capture parent pointer updates. */ +struct xfs_dir_hook dhook; + +/* xattr key and da args for parent pointer replay. */ +struct xfs_parent_scratch pptr_scratch; + +/* +* Scratch buffer for scanning dirents to create pptr xattrs. At the +* very end of the repair, it can also be used to compute the +* lost+found filename if we need to reparent the file. +*/ +struct xfs_parent_name_irec pptr; +}; + +/* Create a parent pointer in the tempfile. */ +#define XREP_XATTR_PPTR_ADD (1) + +/* Remove a parent pointer from the tempfile. */ +#define XREP_XATTR_PPTR_REMOVE (2) + +/* A stashed parent pointer update. */ +struct xrep_xattr_pptr { +/* Cookie for retrieval of the pptr name. */ +xfblob_cookie name_cookie; + +/* Parent pointer attr key. */ +xfs_ino_t p_ino; +uint32_t p_gen; + +/* Length of the pptr name. */ +uint8_t namelen; + +/* XREP_XATTR_PPTR_{ADD,REMOVE} */ +uint8_t action; };/* Set up to recreate the extended attributes. */
@@ -103,6 +153,9 @@ intxrep_setup_xattr ( struct xfs_scrub *sc) { +if (xfs_has_parent (sc->mp)) +xchk_fsgates_enable (sc,XCHK_FSGATES_DIRENTS ); +return xrep_tempfile_create (sc,S_IFREG ); }
@@ -716,11 +769,122 @@ xrep_xattr_want_flush_stashed( {unsigned long long bytes; +if (!rx->can_flush) +return false; + bytes =xfarray_bytes (rx->xattr_records) +xfblob_bytes (rx->xattr_blobs);return bytes >XREP_XATTR_MAX_STASH_BYTES ; } +/* +* Did we observe rename changing parent pointer xattrs while we were flushing +* salvaged attrs? +*/ +static inline bool +xrep_xattr_saw_pptr_conflict ( +struct xrep_xattr *rx) +{ +bool ret; + +ASSERT (rx->can_flush); + +if (!xfs_has_parent (rx->sc->mp)) +return false; + +ASSERT (xfs_isilocked (rx->sc->ip,XFS_ILOCK_EXCL )); + +mutex_lock (&rx->lock); + ret =xfarray_bytes (rx->pptr_recs) >0 ; +mutex_unlock (&rx->lock); + +return ret; +} + +/* +* Reset the entire repair state back to initial conditions, now that we've +* detected a parent pointer update to the attr structure while we were +* flushing salvaged attrs. See the locking notes in dir_repair.c for more +* information on why this is all necessary. +*/ +STATIC int +xrep_xattr_full_reset ( +struct xrep_xattr *rx) +{ +struct xfs_scrub *sc = rx->sc; +struct xfs_attr_sf_hdr *hdr; +struct xfs_ifork *ifp = &sc->tempip->i_af; +int error; + +trace_xrep_xattr_full_reset (sc->ip, sc->tempip); + +/* The temporary file's data fork had better not be in btree format. */ +if (sc->tempip->i_df.if_format ==XFS_DINODE_FMT_BTREE ) { +ASSERT (0 ); +return -EIO ; + } + +/* +* We begin in transaction context with sc->ip ILOCKed but not joined +* to the transaction. To reset to the initial state, we must hold +* sc->ip's ILOCK to prevent rename from updating parent pointer +* information and the tempfile's ILOCK to clear its contents. +*/ +xchk_iunlock (rx->sc,XFS_ILOCK_EXCL ); +xrep_tempfile_ilock_both (sc); +xfs_trans_ijoin (sc->tp, sc->ip,0 ); +xfs_trans_ijoin (sc->tp, sc->tempip,0 ); + +/* +* Free all the blocks of the attr fork of the temp file, and reset +* it back to local format. +*/ +if (xfs_ifork_has_extents (&sc->tempip->i_af)) { + error =xrep_reap_ifork (sc, sc->tempip,XFS_ATTR_FORK ); +if (error) +return error; + +ASSERT (ifp->if_bytes ==0 ); + ifp->if_format =XFS_DINODE_FMT_LOCAL ; +xfs_idata_realloc (sc->tempip,sizeof (*hdr),XFS_ATTR_FORK ); + } + +/* Reinitialize the attr fork to an empty shortform structure. */ + hdr = ifp->if_data; +memset (hdr,0 ,sizeof (*hdr)); + hdr->totsize =cpu_to_be16 (sizeof (*hdr)); +xfs_trans_log_inode (sc->tp, sc->tempip,XFS_ILOG_CORE |XFS_ILOG_ADATA ); + +/* +* Roll this transaction to commit our reset ondisk. The tempfile +* should no longer be joined to the transaction, so we drop its ILOCK. +* This should leave us in transaction context with sc->ip ILOCKed but +* not joined to the transaction. +*/ + error =xrep_roll_trans (sc); +if (error) +return error; +xrep_tempfile_iunlock (sc); + +/* +* Erase any accumulated parent pointer updates now that we've erased +* the tempfile's attr fork. We're resetting the entire repair state +* back to where we were initially, except now we won't flush salvaged +* xattrs until the very end. +*/ +mutex_lock (&rx->lock); +xfarray_truncate (rx->pptr_recs); +xfblob_truncate (rx->pptr_names); +mutex_unlock (&rx->lock); + + rx->can_flush = false; + rx->attrs_found =0 ; + +ASSERT (xfarray_bytes (rx->xattr_records) ==0 ); +ASSERT (xfblob_bytes (rx->xattr_blobs) ==0 ); +return 0 ; +} +/* Extract as many attribute keys and values as we can. */ STATIC int xrep_xattr_recover(
@@ -735,6 +899,7 @@ xrep_xattr_recover(int nmap;int error; +restart:/* * Iterate each xattr leaf block in the attr fork to scan them for any * attributes that we might salvage.
@@ -773,6 +938,14 @@ xrep_xattr_recover( error =xrep_xattr_flush_stashed (rx);if (error)return error; + +if (xrep_xattr_saw_pptr_conflict (rx)) { + error =xrep_xattr_full_reset (rx); +if (error) +return error; + + goto restart; + } } } }
@@ -932,6 +1105,195 @@ xrep_xattr_salvage_attributes(return xrep_xattr_flush_stashed (rx); } +/* +* Add this stashed incore parent pointer to the temporary file. The caller +* must hold the tempdir's IOLOCK, must not hold any ILOCKs, and must not be in +* transaction context. +*/ +STATIC int +xrep_xattr_replay_pptr_update ( +struct xrep_xattr *rx, +const struct xrep_xattr_pptr *pptr) +{ +struct xfs_scrub *sc = rx->sc; +int error; + + rx->pptr.p_ino = pptr->p_ino; + rx->pptr.p_gen = pptr->p_gen; + rx->pptr.p_namelen = pptr->namelen; +xfs_parent_irec_hashname (sc->mp, &rx->pptr); + +switch (pptr->action) { +case XREP_XATTR_PPTR_ADD : +/* Create parent pointer. */ +trace_xrep_xattr_replay_parentadd (sc->tempip, &rx->pptr); + + error =xfs_parent_set (sc->tempip, sc->ip->i_ino, &rx->pptr, + &rx->pptr_scratch); +if (error) { +ASSERT (error != -EEXIST ); +return error; + } +break ; +case XREP_XATTR_PPTR_REMOVE : +/* Remove parent pointer. */ +trace_xrep_xattr_replay_parentremove (sc->tempip, &rx->pptr); + + error =xfs_parent_unset (sc->tempip, sc->ip->i_ino, &rx->pptr, + &rx->pptr_scratch); +if (error) { +ASSERT (error != -ENOATTR ); +return error; + } +break ; +default : +ASSERT (0 ); +return -EIO ; + } + +return 0 ; +} + +/* +* Flush stashed parent pointer updates that have been recorded by the scanner. +* This is done to reduce the memory requirements of the parent pointer +* rebuild, since files can have a lot of hardlinks and the fs can be busy. +* +* Caller must not hold transactions or ILOCKs. Caller must hold the tempfile +* IOLOCK. +*/ +STATIC int +xrep_xattr_replay_pptr_updates ( +struct xrep_xattr *rx) +{ +xfarray_idx_t array_cur; +int error; + +mutex_lock (&rx->lock); + foreach_xfarray_idx(rx->pptr_recs, array_cur) { +struct xrep_xattr_pptr pptr; + + error =xfarray_load (rx->pptr_recs, array_cur, &pptr); +if (error) + goto out_unlock; + + error =xfblob_load (rx->pptr_names, pptr.name_cookie, + rx->pptr.p_name, pptr.namelen); +if (error) + goto out_unlock; + rx->pptr.p_name[MAXNAMELEN -1 ] =0 ; +mutex_unlock (&rx->lock); + + error =xrep_xattr_replay_pptr_update (rx, &pptr); +if (error) +return error; + +mutex_lock (&rx->lock); + } + +/* Empty out both arrays now that we've added the entries. */ +xfarray_truncate (rx->pptr_recs); +xfblob_truncate (rx->pptr_names); +mutex_unlock (&rx->lock); +return 0 ; +out_unlock: +mutex_unlock (&rx->lock); +return error; +} + +/* +* Remember that we want to create a parent pointer in the tempfile. These +* stashed actions will be replayed later. +*/ +STATIC int +xrep_xattr_stash_parentadd ( +struct xrep_xattr *rx, +const struct xfs_name *name, +const struct xfs_inode *dp) +{ +struct xrep_xattr_pptr pptr = { + .action =XREP_XATTR_PPTR_ADD , + .namelen = name->len, + .p_ino = dp->i_ino, + .p_gen =VFS_IC (dp)->i_generation, + }; +int error; + +trace_xrep_xattr_stash_parentadd (rx->sc->tempip, dp, name); + + error =xfblob_store (rx->pptr_names, &pptr.name_cookie, name->name, + name->len); +if (error) +return error; + +return xfarray_append (rx->pptr_recs, &pptr); +} + +/* +* Remember that we want to remove a parent pointer from the tempfile. These +* stashed actions will be replayed later. +*/ +STATIC int +xrep_xattr_stash_parentremove ( +struct xrep_xattr *rx, +const struct xfs_name *name, +const struct xfs_inode *dp) +{ +struct xrep_xattr_pptr pptr = { + .action =XREP_XATTR_PPTR_REMOVE , + .namelen = name->len, + .p_ino = dp->i_ino, + .p_gen =VFS_IC (dp)->i_generation, + }; +int error; + +trace_xrep_xattr_stash_parentremove (rx->sc->tempip, dp, name); + + error =xfblob_store (rx->pptr_names, &pptr.name_cookie, name->name, + name->len); +if (error) +return error; + +return xfarray_append (rx->pptr_recs, &pptr); +} + +/* +* Capture dirent updates being made by other threads. We will have to replay +* the parent pointer updates before exchanging attr forks. +*/ +STATIC int +xrep_xattr_live_dirent_update ( +struct notifier_block *nb, +unsigned long action, +void *data) +{ +struct xfs_dir_update_params *p = data; +struct xrep_xattr *rx; +struct xfs_scrub *sc; +int error; + + rx = container_of(nb, struct xrep_xattr, dhook.dirent_hook.nb); + sc = rx->sc; + +/* +* This thread updated a dirent that points to the file that we're +* repairing, so stash the update for replay against the temporary +* file. +*/ +if (p->ip->i_ino != sc->ip->i_ino) +return NOTIFY_DONE ; + +mutex_lock (&rx->lock); +if (p->delta >0 ) + error =xrep_xattr_stash_parentadd (rx, p->name, p->dp); +else + error =xrep_xattr_stash_parentremove (rx, p->name, p->dp); +if (error) + rx->live_update_aborted = true; +mutex_unlock (&rx->lock); +return NOTIFY_DONE ; +} +/* * Prepare both inodes' attribute forks for an exchange. Promote the tempfile * from short format to leaf format, and if the file being repaired has a short
@@ -1035,6 +1397,45 @@ xrep_xattr_swap(return xrep_tempexch_contents (sc, tx); } +/* +* Finish replaying stashed parent pointer updates, allocate a transaction for +* exchanging extent mappings, and take the ILOCKs of both files before we +* commit the new extended attribute structure. +*/ +STATIC int +xrep_xattr_finalize_tempfile ( +struct xrep_xattr *rx) +{ +struct xfs_scrub *sc = rx->sc; +int error; + +if (!xfs_has_parent (sc->mp)) +return xrep_tempexch_trans_alloc (sc,XFS_ATTR_FORK , &rx->tx); + +/* +* Repair relies on the ILOCK to quiesce all possible xattr updates. +* Replay all queued parent pointer updates into the tempfile before +* exchanging the contents, even if that means dropping the ILOCKs and +* the transaction. +*/ +do { + error =xrep_xattr_replay_pptr_updates (rx); +if (error) +return error; + + error =xrep_tempexch_trans_alloc (sc,XFS_ATTR_FORK , &rx->tx); +if (error) +return error; + +if (xfarray_length (rx->pptr_recs) ==0 ) +break ; + +xchk_trans_cancel (sc); +xrep_tempfile_iunlock_both (sc); + }while (!xchk_should_terminate (sc, &error)); +return error; +} +/* * Exchange the new extended attribute data (which we created in the tempfile) * with the file being repaired.
@@ -1087,8 +1488,12 @@ xrep_xattr_rebuild_tree(if (error)return error; -/* Allocate exchange transaction and lock both inodes. */ - error =xrep_tempexch_trans_alloc (rx->sc,XFS_ATTR_FORK , &rx->tx); +/* +* Allocate transaction, lock inodes, and make sure that we've replayed +* all the stashed parent pointer updates to the temp file. After this +* point, we're ready to exchange attr fork mappings. +*/ + error =xrep_xattr_finalize_tempfile (rx);if (error)return error;
@@ -1129,8 +1534,15 @@ STATIC voidxrep_xattr_teardown ( struct xrep_xattr *rx) { +if (xfs_has_parent (rx->sc->mp)) +xfs_dir_hook_del (rx->sc->mp, &rx->dhook); +if (rx->pptr_names) +xfblob_destroy (rx->pptr_names); +if (rx->pptr_recs) +xfarray_destroy (rx->pptr_recs);xfblob_destroy (rx->xattr_blobs);xfarray_destroy (rx->xattr_records); +mutex_destroy (&rx->lock);kfree (rx); }
@@ -1149,6 +1561,9 @@ xrep_xattr_setup_scan(if (!rx)return -ENOMEM ; rx->sc = sc; + rx->can_flush = true; + +mutex_init (&rx->lock);/* * Allocate enough memory to handle loading local attr values from the
@@ -1176,11 +1591,43 @@ xrep_xattr_setup_scan(if (error) goto out_keys; +if (xfs_has_parent (sc->mp)) { +ASSERT (sc->flags &XCHK_FSGATES_DIRENTS ); + + descr =xchk_xfile_ino_descr (sc, +"xattr retained parent pointer entries" ); + error =xfarray_create (descr,0 , +sizeof (struct xrep_xattr_pptr ), + &rx->pptr_recs); +kfree (descr); +if (error) + goto out_values; + + descr =xchk_xfile_ino_descr (sc, +"xattr retained parent pointer names" ); + error =xfblob_create (descr, &rx->pptr_names); +kfree (descr); +if (error) + goto out_pprecs; + +xfs_dir_hook_setup (&rx->dhook, xrep_xattr_live_dirent_update); + error =xfs_dir_hook_add (sc->mp, &rx->dhook); +if (error) + goto out_ppnames; + } + *rxp = rx;return 0 ; +out_ppnames: +xfblob_destroy (rx->pptr_names); +out_pprecs: +xfarray_destroy (rx->pptr_recs); +out_values: +xfblob_destroy (rx->xattr_blobs); out_keys:xfarray_destroy (rx->xattr_records); out_rx: +mutex_destroy (&rx->lock);kfree (rx);return error; }
@@ -1217,6 +1664,11 @@ xrep_xattr(if (error) goto out_scan; +if (rx->live_update_aborted) { + error = -EIO ; + goto out_scan; + } +/* Last chance to abort before we start committing fixes. */ if (xchk_should_terminate (sc, &error)) goto out_scan;
fs/xfs/scrub/trace.h +71 -0
83 lines not on this page · raw message
PATCH 15/23 xfs: replace namebuf with parent pointer in parent pointer repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Replace the dirent name buffer at the end of struct xrep_parent with a xfs_parent_name_irec object. The namebuf and p_name usage do not overlap, so we can save 256 bytes of memory by allowing them to overlap. Doing so makes the code a bit more complex, so this is called out separately.
--- fs/xfs/scrub/parent_repair.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
fs/xfs/scrub/parent_repair.c +8 -3
@@ -24,6 +24,7 @@#include "xfs_trans_space.h" #include "xfs_health.h" #include "xfs_exchmaps.h" +#include "xfs_parent.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h"
@@ -63,8 +64,12 @@ struct xrep_parent {/* Orphanage reparenting request. */ struct xrep_adoption adoption; -/* Directory entry name, plus the trailing null. */ -unsigned char namebuf[MAXNAMELEN ]; +/* +* Scratch buffer for scanning dirents to create pptr xattrs. At the +* very end of the repair, it can also be used to compute the +* lost+found filename if we need to reparent the file. +*/ +struct xfs_parent_name_irec pptr; };/* Tear down all the incore stuff we created. */
@@ -236,7 +241,7 @@ xrep_parent_move_to_orphanage(if (error)return error; - error =xrep_adoption_compute_name (&rp->adoption, rp->namebuf); + error =xrep_adoption_compute_name (&rp->adoption, rp->pptr.p_name);if (error)return error;
PATCH 16/23 xfs: repair directory parent pointers by scanning for dirents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If parent pointers are enabled on the filesystem, we can repair the entire dataset by walking the directories of the filesystem looking for dirents that we can turn into parent pointers. Once we have a full incore dataset, we'll figure out what to do with it, but that's for a subsequent patch.
--- fs/xfs/scrub/parent_repair.c | 425 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 36 ++++ 2 files changed, 458 insertions(+), 3 deletions(-)
fs/xfs/scrub/parent_repair.c +422 -3
@@ -35,6 +35,9 @@#include "scrub/readdir.h" #include "scrub/tempfile.h" #include "scrub/orphanage.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" /* * Repairing The Directory Parent Pointer
@@ -50,20 +53,71 @@* See the section on locking issues in dir_repair.c for more information about * conflicts with the VFS. The findparent code wll keep our incore parent * inode up to date. +* +* If parent pointers are enabled, we instead reconstruct the parent pointer +* information by visiting every directory entry of every directory in the +* system and translating the relevant dirents into parent pointers. In this +* case, it is advantageous to stash all parent pointers created from dirents +* from a single parent file before replaying them into the temporary file. To +* save memory, the live filesystem scan reuses the findparent object. Parent +* pointer repair chooses either directory scanning or findparent, but not +* both. +* +* When salvaging completes, the remaining stashed entries are replayed to the +* temporary file. All non-parent pointer extended attributes are copied to +* the temporary file's extended attributes. An atomic extent swap is used to +* commit the new directory blocks to the directory being repaired. This will +* disrupt attrmulti cursors. */ +/* A stashed parent pointer update. */ +struct xrep_pptr { +/* Cookie for retrieval of the pptr name. */ +xfblob_cookie name_cookie; + +/* Parent pointer attr key. */ +xfs_ino_t p_ino; +uint32_t p_gen; + +/* Length of the pptr name. */ +uint8_t namelen; +}; + +/* +* Stash up to 8 pages of recovered parent pointers in pptr_recs and +* pptr_names before we write them to the temp file. +*/ +#define XREP_PARENT_MAX_STASH_BYTES (PAGE_SIZE * 8) +struct xrep_parent {struct xfs_scrub *sc; +/* Fixed-size array of xrep_pptr structures. */ +struct xfarray *pptr_recs; + +/* Blobs containing parent pointer names. */ +struct xfblob *pptr_names; +/* * Information used to scan the filesystem to find the inumber of the -* dotdot entry for this directory. +* dotdot entry for this directory. On filesystems without parent +* pointers, we use the findparent_* functions on this object and +* access only the parent_ino field directly. +* +* When parent pointers are enabled, the directory entry scanner uses +* the iscan, hooks, and lock fields of this object directly. +* @pscan.lock coordinates access to pptr_recs, pptr_names, pptr, and +* pptr_scratch. This reduces the memory requirements of this +* structure. */ struct xrep_parent_scan_info pscan;/* Orphanage reparenting request. */ struct xrep_adoption adoption; +/* xattr key and da args for parent pointer replay. */ +struct xfs_parent_scratch pptr_scratch; +/* * Scratch buffer for scanning dirents to create pptr xattrs. At the * very end of the repair, it can also be used to compute the
@@ -78,6 +132,12 @@ xrep_parent_teardown( struct xrep_parent *rp) {xrep_findparent_scan_teardown (&rp->pscan); +if (rp->pptr_names) +xfblob_destroy (rp->pptr_names); + rp->pptr_names =NULL ; +if (rp->pptr_recs) +xfarray_destroy (rp->pptr_recs); + rp->pptr_recs =NULL ; }/* Set up for a parent repair. */
@@ -86,6 +146,7 @@ xrep_setup_parent( struct xfs_scrub *sc) {struct xrep_parent *rp; +int error;xchk_fsgates_enable (sc,XCHK_FSGATES_DIRENTS );
@@ -95,6 +156,10 @@ xrep_setup_parent( rp->sc = sc; sc->buf = rp; + error =xrep_tempfile_create (sc,S_IFREG ); +if (error) +return error; +return xrep_orphanage_try_create (sc); }
@@ -150,6 +215,318 @@ xrep_parent_find_dotdot(return error; } +/* +* Add this stashed incore parent pointer to the temporary file. +* The caller must hold the tempdir's IOLOCK, must not hold any ILOCKs, and +* must not be in transaction context. +*/ +STATIC int +xrep_parent_replay_update ( +struct xrep_parent *rp, +const struct xrep_pptr *pptr) +{ +struct xfs_scrub *sc = rp->sc; +int error; + + rp->pptr.p_ino = pptr->p_ino; + rp->pptr.p_gen = pptr->p_gen; + rp->pptr.p_namelen = pptr->namelen; +xfs_parent_irec_hashname (sc->mp, &rp->pptr); + +/* Create parent pointer. */ +trace_xrep_parent_replay_parentadd (sc->tempip, &rp->pptr); + + error =xfs_parent_set (sc->tempip, sc->ip->i_ino, &rp->pptr, + &rp->pptr_scratch); +if (error) +return error; + +return 0 ; +} + +/* +* Flush stashed parent pointer updates that have been recorded by the scanner. +* This is done to reduce the memory requirements of the parent pointer +* rebuild, since files can have a lot of hardlinks and the fs can be busy. +* +* Caller must not hold transactions or ILOCKs. Caller must hold the tempfile +* IOLOCK. +*/ +STATIC int +xrep_parent_replay_updates ( +struct xrep_parent *rp) +{ +xfarray_idx_t array_cur; +int error; + +mutex_lock (&rp->pscan.lock); + foreach_xfarray_idx(rp->pptr_recs, array_cur) { +struct xrep_pptr pptr; + + error =xfarray_load (rp->pptr_recs, array_cur, &pptr); +if (error) + goto out_unlock; + + error =xfblob_load (rp->pptr_names, pptr.name_cookie, + rp->pptr.p_name, pptr.namelen); +if (error) + goto out_unlock; + rp->pptr.p_name[MAXNAMELEN -1 ] =0 ; +mutex_unlock (&rp->pscan.lock); + + error =xrep_parent_replay_update (rp, &pptr); +if (error) +return error; + +mutex_lock (&rp->pscan.lock); + } + +/* Empty out both arrays now that we've added the entries. */ +xfarray_truncate (rp->pptr_recs); +xfblob_truncate (rp->pptr_names); +mutex_unlock (&rp->pscan.lock); +return 0 ; +out_unlock: +mutex_unlock (&rp->pscan.lock); +return error; +} + +/* +* Remember that we want to create a parent pointer in the tempfile. These +* stashed actions will be replayed later. +*/ +STATIC int +xrep_parent_stash_parentadd ( +struct xrep_parent *rp, +const struct xfs_name *name, +const struct xfs_inode *dp) +{ +struct xrep_pptr pptr = { + .namelen = name->len, + .p_ino = dp->i_ino, + .p_gen =VFS_IC (dp)->i_generation, + }; +int error; + +trace_xrep_parent_stash_parentadd (rp->sc->tempip, dp, name); + + error =xfblob_store (rp->pptr_names, &pptr.name_cookie, name->name, + name->len); +if (error) +return error; + +return xfarray_append (rp->pptr_recs, &pptr); +} + +/* +* Examine an entry of a directory. If this dirent leads us back to the file +* whose parent pointers we're rebuilding, add a pptr to the temporary +* directory. +*/ +STATIC int +xrep_parent_scan_dirent ( +struct xfs_scrub *sc, +struct xfs_inode *dp, +xfs_dir2_dataptr_t dapos, +const struct xfs_name *name, +xfs_ino_t ino, +void *priv) +{ +struct xrep_parent *rp = priv; +int error; + +/* Dirent doesn't point to this directory. */ +if (ino != rp->sc->ip->i_ino) +return 0 ; + +/* No weird looking names. */ +if (name->len ==0 || !xfs_dir2_namecheck (name->name, name->len)) +return -EFSCORRUPTED ; + +/* No mismatching ftypes. */ +if (name->type !=xfs_mode_to_ftype (VFS_I (sc->ip)->i_mode)) +return -EFSCORRUPTED ; + +/* Don't pick up dot or dotdot entries; we only want child dirents. */ +if (xfs_dir2_samename (name, &xfs_name_dotdot) || +xfs_dir2_samename (name, &xfs_name_dot)) +return 0 ; + +/* +* Transform this dirent into a parent pointer and queue it for later +* addition to the temporary file. +*/ +mutex_lock (&rp->pscan.lock); + error =xrep_parent_stash_parentadd (rp, name, dp); +mutex_unlock (&rp->pscan.lock); +return error; +} + +/* +* Decide if we want to look for dirents in this directory. Skip the file +* being repaired and any files being used to stage repairs. +*/ +static inline bool +xrep_parent_want_scan ( +struct xrep_parent *rp, +const struct xfs_inode *ip) +{ +return ip != rp->sc->ip && !xrep_is_tempfile (ip); +} + +/* +* Take ILOCK on a file that we want to scan. +* +* Select ILOCK_EXCL if the file is a directory with an unloaded data bmbt. +* Otherwise, take ILOCK_SHARED. +*/ +static inline unsigned int +xrep_parent_scan_ilock ( +struct xrep_parent *rp, +struct xfs_inode *ip) +{ +uint lock_mode =XFS_ILOCK_SHARED ; + +/* Still need to take the shared ILOCK to advance the iscan cursor. */ +if (!xrep_parent_want_scan (rp, ip)) + goto lock; + +if (S_ISDIR (VFS_I (ip)->i_mode) &&xfs_need_iread_extents (&ip->i_df)) { + lock_mode =XFS_ILOCK_EXCL ; + goto lock; + } + +lock: +xfs_ilock (ip, lock_mode); +return lock_mode; +} + +/* +* Scan this file for relevant child dirents that point to the file whose +* parent pointers we're rebuilding. +*/ +STATIC int +xrep_parent_scan_file ( +struct xrep_parent *rp, +struct xfs_inode *ip) +{ +unsigned int lock_mode; +int error =0 ; + + lock_mode =xrep_parent_scan_ilock (rp, ip); + +if (!xrep_parent_want_scan (rp, ip)) + goto scan_done; + +if (S_ISDIR (VFS_I (ip)->i_mode)) { +/* +* If the directory looks as though it has been zapped by the +* inode record repair code, we cannot scan for child dirents. +*/ +if (xchk_dir_looks_zapped (ip)) { + error = -EBUSY ; + goto scan_done; + } + + error =xchk_dir_walk (rp->sc, ip, xrep_parent_scan_dirent, rp); +if (error) + goto scan_done; + } + +scan_done: +xchk_iscan_mark_visited (&rp->pscan.iscan, ip); +xfs_iunlock (ip, lock_mode); +return error; +} + +/* Decide if we've stashed too much pptr data in memory. */ +static inline bool +xrep_parent_want_flush_stashed ( +struct xrep_parent *rp) +{ +unsigned long long bytes; + + bytes =xfarray_bytes (rp->pptr_recs) +xfblob_bytes (rp->pptr_names); +return bytes >XREP_PARENT_MAX_STASH_BYTES ; +} + +/* +* Scan all directories in the filesystem to look for dirents that we can turn +* into parent pointers. +*/ +STATIC int +xrep_parent_scan_dirtree ( +struct xrep_parent *rp) +{ +struct xfs_scrub *sc = rp->sc; +struct xfs_inode *ip; +int error; + +/* +* Filesystem scans are time consuming. Drop the file ILOCK and all +* other resources for the duration of the scan and hope for the best. +* The live update hooks will keep our scan information up to date. +*/ +xchk_trans_cancel (sc); +if (sc->ilock_flags & (XFS_ILOCK_SHARED |XFS_ILOCK_EXCL )) +xchk_iunlock (sc, sc->ilock_flags & (XFS_ILOCK_SHARED | +XFS_ILOCK_EXCL )); + error =xchk_trans_alloc_empty (sc); +if (error) +return error; + +while ((error =xchk_iscan_iter (&rp->pscan.iscan, &ip)) ==1 ) { +bool flush; + + error =xrep_parent_scan_file (rp, ip); +xchk_irele (sc, ip); +if (error) +break ; + +/* Flush stashed pptr updates to constrain memory usage. */ +mutex_lock (&rp->pscan.lock); + flush =xrep_parent_want_flush_stashed (rp); +mutex_unlock (&rp->pscan.lock); +if (flush) { +xchk_trans_cancel (sc); + + error =xrep_tempfile_iolock_polled (sc); +if (error) +break ; + + error =xrep_parent_replay_updates (rp); +xrep_tempfile_iounlock (sc); +if (error) +break ; + + error =xchk_trans_alloc_empty (sc); +if (error) +break ; + } + +if (xchk_should_terminate (sc, &error)) +break ; + } +xchk_iscan_iter_finish (&rp->pscan.iscan); +if (error) { +/* +* If we couldn't grab an inode that was busy with a state +* change, change the error code so that we exit to userspace +* as quickly as possible. +*/ +if (error == -EBUSY ) +return -ECANCELED ; +return error; + } + +/* +* Cancel the empty transaction so that we can (later) use the atomic +* extent swap helpers to lock files and commit the new directory. +*/ +xchk_trans_cancel (rp->sc); +return 0 ; +} +/* Reset a directory's dotdot entry, if needed. */ STATIC int xrep_parent_reset_dotdot(
@@ -301,8 +678,39 @@ xrep_parent_setup_scan( struct xrep_parent *rp) {struct xfs_scrub *sc = rp->sc; +char *descr; +int error; -return xrep_findparent_scan_start (sc, &rp->pscan); +if (!xfs_has_parent (sc->mp)) +return xrep_findparent_scan_start (sc, &rp->pscan); + +/* Set up some staging memory for logging parent pointer updates. */ + descr =xchk_xfile_ino_descr (sc,"parent pointer entries" ); + error =xfarray_create (descr,0 ,sizeof (struct xrep_pptr ), + &rp->pptr_recs); +kfree (descr); +if (error) +return error; + + descr =xchk_xfile_ino_descr (sc,"parent pointer names" ); + error =xfblob_create (descr, &rp->pptr_names); +kfree (descr); +if (error) + goto out_recs; + + error =xrep_findparent_scan_start (sc, &rp->pscan); +if (error) + goto out_names; + +return 0 ; + +out_names: +xfblob_destroy (rp->pptr_names); + rp->pptr_names =NULL ; +out_recs: +xfarray_destroy (rp->pptr_recs); + rp->pptr_recs =NULL ; +return error; }int
@@ -312,11 +720,22 @@ xrep_parent(struct xrep_parent *rp = sc->buf;int error; +/* +* When the parent pointers feature is enabled, repairs are committed +* by atomically committing a new xattr structure and reaping the old +* attr fork. Reaping requires rmap to be enabled. +*/ +if (xfs_has_parent (sc->mp) && !xfs_has_rmapbt (sc->mp)) +return -EOPNOTSUPP ; + error =xrep_parent_setup_scan (rp);if (error)return error; - error =xrep_parent_find_dotdot (rp); +if (xfs_has_parent (sc->mp)) + error =xrep_parent_scan_dirtree (rp); +else + error =xrep_parent_find_dotdot (rp);if (error) goto out_teardown;
fs/xfs/scrub/trace.h +36 -0
@@ -2813,6 +2813,42 @@ DEFINE_EVENT(xrep_pptr_class, name, \ TP_ARGS(ip, pptr))DEFINE_XREP_PPTR_EVENT (xrep_xattr_replay_parentadd);DEFINE_XREP_PPTR_EVENT (xrep_xattr_replay_parentremove); +DEFINE_XREP_PPTR_EVENT (xrep_parent_replay_parentadd); + +DECLARE_EVENT_CLASS (xrep_pptr_scan_class, + TP_PROTO(struct xfs_inode *ip, const struct xfs_inode *dp, + const struct xfs_name *name), +TP_ARGS (ip, dp, name), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(xfs_ino_t, parent_ino) + __field(unsigned int, parent_gen) + __field(unsigned int, namelen) +__dynamic_array (char, name, name->len) + ), +TP_fast_assign ( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->parent_ino = dp->i_ino; + __entry->parent_gen = VFS_IC(dp)->i_generation; + __entry->namelen = name->len; + memcpy(__get_str(name), name->name, name->len); + ), +TP_printk ("dev %d:%d ino 0x%llx parent_ino 0x%llx parent_gen 0x%x name '%.*s'" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, + __entry->parent_ino, + __entry->parent_gen, + __entry->namelen, +__get_str (name)) +) +#define DEFINE_XREP_PPTR_SCAN_EVENT (name) \ +DEFINE_EVENT(xrep_pptr_scan_class, name, \ + TP_PROTO(struct xfs_inode *ip, const struct xfs_inode *dp, \ + const struct xfs_name *name), \ + TP_ARGS(ip, dp, name)) +DEFINE_XREP_PPTR_SCAN_EVENT (xrep_parent_stash_parentadd); TRACE_EVENT(xrep_nlinks_set_record, TP_PROTO(struct xfs_mount *mp, xfs_ino_t ino,
PATCH 17/23 xfs: implement live updates for parent pointer repairs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
While we're scanning the filesystem for dirents that we can turn into parent pointers, we cannot hold the IOLOCK or ILOCK of the file being repaired. Therefore, we need to set up a dirent hook so that we can keep the temporary file's parent pionters up to date with the rest of the filesystem. Hence we add the ability to *remove* pptrs from the temporary file.
--- fs/xfs/scrub/parent_repair.c | 111 +++++++++++++++++++++++++++++++++++++++--- fs/xfs/scrub/trace.h | 2 + 2 files changed, 106 insertions(+), 7 deletions(-)
fs/xfs/scrub/parent_repair.c +104 -7
@@ -70,6 +70,12 @@* disrupt attrmulti cursors. */ +/* Create a parent pointer in the tempfile. */ +#define XREP_PPTR_ADD (1) + +/* Remove a parent pointer from the tempfile. */ +#define XREP_PPTR_REMOVE (2) +/* A stashed parent pointer update. */ struct xrep_pptr {/* Cookie for retrieval of the pptr name. */
@@ -81,6 +87,9 @@ struct xrep_pptr {/* Length of the pptr name. */ uint8_t namelen; + +/* XREP_PPTR_{ADD,REMOVE} */ +uint8_t action; };/*
@@ -233,13 +242,29 @@ xrep_parent_replay_update( rp->pptr.p_namelen = pptr->namelen;xfs_parent_irec_hashname (sc->mp, &rp->pptr); -/* Create parent pointer. */ -trace_xrep_parent_replay_parentadd (sc->tempip, &rp->pptr); +switch (pptr->action) { +case XREP_PPTR_ADD : +/* Create parent pointer. */ +trace_xrep_parent_replay_parentadd (sc->tempip, &rp->pptr); - error =xfs_parent_set (sc->tempip, sc->ip->i_ino, &rp->pptr, - &rp->pptr_scratch); -if (error) -return error; + error =xfs_parent_set (sc->tempip, sc->ip->i_ino, &rp->pptr, + &rp->pptr_scratch); +if (error) +return error; +break ; +case XREP_PPTR_REMOVE : +/* Remove parent pointer. */ +trace_xrep_parent_replay_parentremove (sc->tempip, &rp->pptr); + + error =xfs_parent_unset (sc->tempip, sc->ip->i_ino, &rp->pptr, + &rp->pptr_scratch); +if (error) +return error; +break ; +default : +ASSERT (0 ); +return -EIO ; + }return 0 ; }
@@ -302,6 +327,7 @@ xrep_parent_stash_parentadd( const struct xfs_inode *dp) {struct xrep_pptr pptr = { + .action =XREP_PPTR_ADD , .namelen = name->len, .p_ino = dp->i_ino, .p_gen =VFS_IC (dp)->i_generation,
@@ -318,6 +344,34 @@ xrep_parent_stash_parentadd(return xfarray_append (rp->pptr_recs, &pptr); } +/* +* Remember that we want to remove a parent pointer from the tempfile. These +* stashed actions will be replayed later. +*/ +STATIC int +xrep_parent_stash_parentremove ( +struct xrep_parent *rp, +const struct xfs_name *name, +const struct xfs_inode *dp) +{ +struct xrep_pptr pptr = { + .action =XREP_PPTR_REMOVE , + .namelen = name->len, + .p_ino = dp->i_ino, + .p_gen =VFS_IC (dp)->i_generation, + }; +int error; + +trace_xrep_parent_stash_parentremove (rp->sc->tempip, dp, name); + + error =xfblob_store (rp->pptr_names, &pptr.name_cookie, name->name, + name->len); +if (error) +return error; + +return xfarray_append (rp->pptr_recs, &pptr); +} +/* * Examine an entry of a directory. If this dirent leads us back to the file * whose parent pointers we're rebuilding, add a pptr to the temporary
@@ -527,6 +581,48 @@ xrep_parent_scan_dirtree(return 0 ; } +/* +* Capture dirent updates being made by other threads which are relevant to the +* file being repaired. +*/ +STATIC int +xrep_parent_live_update ( +struct notifier_block *nb, +unsigned long action, +void *data) +{ +struct xfs_dir_update_params *p = data; +struct xrep_parent *rp; +struct xfs_scrub *sc; +int error; + + rp = container_of(nb, struct xrep_parent, pscan.dhook.dirent_hook.nb); + sc = rp->sc; + +/* +* This thread updated a dirent that points to the file that we're +* repairing, so stash the update for replay against the temporary +* file. +*/ +if (p->ip->i_ino == sc->ip->i_ino && +xchk_iscan_want_live_update (&rp->pscan.iscan, p->dp->i_ino)) { +mutex_lock (&rp->pscan.lock); +if (p->delta >0 ) + error =xrep_parent_stash_parentadd (rp, p->name, p->dp); +else + error =xrep_parent_stash_parentremove (rp, p->name, + p->dp); +mutex_unlock (&rp->pscan.lock); +if (error) + goto out_abort; + } + +return NOTIFY_DONE ; +out_abort: +xchk_iscan_abort (&rp->pscan.iscan); +return NOTIFY_DONE ; +} +/* Reset a directory's dotdot entry, if needed. */ STATIC int xrep_parent_reset_dotdot(
@@ -698,7 +794,8 @@ xrep_parent_setup_scan(if (error) goto out_recs; - error =xrep_findparent_scan_start (sc, &rp->pscan); + error =__xrep_findparent_scan_start (sc, &rp->pscan, + xrep_parent_live_update);if (error) goto out_names;
fs/xfs/scrub/trace.h +2 -0
@@ -2814,6 +2814,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \DEFINE_XREP_PPTR_EVENT (xrep_xattr_replay_parentadd);DEFINE_XREP_PPTR_EVENT (xrep_xattr_replay_parentremove);DEFINE_XREP_PPTR_EVENT (xrep_parent_replay_parentadd); +DEFINE_XREP_PPTR_EVENT (xrep_parent_replay_parentremove); DECLARE_EVENT_CLASS(xrep_pptr_scan_class, TP_PROTO(struct xfs_inode *ip, const struct xfs_inode *dp,
@@ -2849,6 +2850,7 @@ DEFINE_EVENT(xrep_pptr_scan_class, name, \ const struct xfs_name *name), \ TP_ARGS(ip, dp, name)) DEFINE_XREP_PPTR_SCAN_EVENT(xrep_parent_stash_parentadd); +DEFINE_XREP_PPTR_SCAN_EVENT (xrep_parent_stash_parentremove); TRACE_EVENT(xrep_nlinks_set_record, TP_PROTO(struct xfs_mount *mp, xfs_ino_t ino,
PATCH 18/23 xfs: remove pointless unlocked assertion
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Remove this assertion about the inode not having an attr fork from xfs_bmap_add_attrfork because the function handles that case just fine. Weirder still, the function actually /requires/ the caller not to hold the ILOCK, which means that its accesses are not stabilized.
--- fs/xfs/libxfs/xfs_bmap.c | 2 -- 1 file changed, 2 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +0 -2
@@ -1041,8 +1041,6 @@ xfs_bmap_add_attrfork(int logflags;/* logging flags */ int error;/* error return value */ -ASSERT (xfs_inode_has_attr_fork (ip) ==0 ); - mp = ip->i_mount;ASSERT (!XFS_NOT_DQATTACHED (mp, ip));
PATCH 19/23 xfs: split xfs_bmap_add_attrfork into two pieces
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Split this function into two pieces -- one to make the actual changes to the inode core to add the attr fork, and another one to deal with getting the transaction and locking the inodes.
The next couple of patches will need this to be split into two. One patch implements committing new parent pointer recordsets to damaged files. If one file has an attr fork and the other does not, we have to create the missing attr fork before the atomic swap transaction, and can use the behavior encoded in the current xfs_bmap_add_attrfork.
The second patch adapts /lost+found adoptions to handle parent pointers correctly. The adoption process will add a parent pointer to a child that is being moved to /lost+found, but this requires that the attr fork already exists. We don't know if we're actually going to commit the adoption until we've already reserved a transaction and taken the ILOCKs, which means that we must have a way to bypass the start of the current xfs_bmap_add_attrfork.
Therefore, create xfs_attr_add_fork as the helper that creates a transaction and takes locks; and make xfs_bmap_add_attrfork the function that updates the inode core and allocates the incore attr fork.
--- fs/xfs/libxfs/xfs_attr.c | 39 ++++++++++++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_bmap.c | 36 ++++++++++-------------------------- fs/xfs/libxfs/xfs_bmap.h | 3 ++- 3 files changed, 50 insertions(+), 28 deletions(-)
fs/xfs/libxfs/xfs_attr.c +38 -1
@@ -941,6 +941,43 @@ xfs_attr_defer_add(trace_xfs_attr_defer_add (new->xattri_dela_state, args->dp); } +STATIC int +xfs_attr_add_fork ( +struct xfs_inode *ip,/* incore inode pointer */ +int size,/* space new attribute needs */ +int rsvd)/* xact may use reserved blks */ +{ +struct xfs_mount *mp = ip->i_mount; +struct xfs_trans *tp;/* transaction pointer */ +unsigned int blks;/* space reservation */ +int error;/* error return value */ + +ASSERT (!XFS_NOT_DQATTACHED (mp, ip)); + + blks =XFS_ADDAFORK_SPACE_RES (mp); + + error =xfs_trans_alloc_inode (ip, &M_RES (mp)->tr_addafork, blks,0 , + rsvd, &tp); +if (error) +return error; + +if (xfs_inode_has_attr_fork (ip)) + goto trans_cancel; + + error =xfs_bmap_add_attrfork (tp, ip, size, rsvd); +if (error) + goto trans_cancel; + + error =xfs_trans_commit (tp); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +return error; + +trans_cancel: +xfs_trans_cancel (tp); +xfs_iunlock (ip,XFS_ILOCK_EXCL ); +return error; +} +/* * Note: If args->value is NULL the attribute will be removed, just like the * Linux ->setattr API.
@@ -992,7 +1029,7 @@ xfs_attr_set(xfs_attr_sf_entsize_byname (args->namelen, args->valuelen); - error =xfs_bmap_add_attrfork (dp, sf_size, rsvd); + error =xfs_attr_add_fork (dp, sf_size, rsvd);if (error)return error; }
fs/xfs/libxfs/xfs_bmap.c +10 -26
@@ -1025,38 +1025,29 @@ xfs_bmap_set_attrforkoff( }/* -* Convert inode from non-attributed to attributed. -* Must not be in a transaction, ip must not be locked. +* Convert inode from non-attributed to attributed. Caller must hold the +* ILOCK_EXCL and the file cannot have an attr fork. */ int /* error code */ xfs_bmap_add_attrfork ( -xfs_inode_t *ip,/* incore inode pointer */ +struct xfs_trans *tp, +struct xfs_inode *ip,/* incore inode pointer */ int size,/* space new attribute needs */ int rsvd)/* xact may use reserved blks */ { -xfs_mount_t *mp;/* mount structure */ -xfs_trans_t *tp;/* transaction pointer */ -int blks;/* space reservation */ +struct xfs_mount *mp = tp->t_mountp;int version =1 ;/* superblock attr version */ int logflags;/* logging flags */ int error;/* error return value */ - mp = ip->i_mount; +ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL ));ASSERT (!XFS_NOT_DQATTACHED (mp, ip)); - - blks =XFS_ADDAFORK_SPACE_RES (mp); - - error =xfs_trans_alloc_inode (ip, &M_RES (mp)->tr_addafork, blks,0 , - rsvd, &tp); -if (error) -return error; -if (xfs_inode_has_attr_fork (ip)) - goto trans_cancel; +ASSERT (!xfs_inode_has_attr_fork (ip));xfs_trans_log_inode (tp, ip,XFS_ILOG_CORE ); error =xfs_bmap_set_attrforkoff (ip, size, &version);if (error) - goto trans_cancel; +return error;xfs_ifork_init_attr (ip,XFS_DINODE_FMT_EXTENTS ,0 ); logflags =0 ;
@@ -1077,7 +1068,7 @@ xfs_bmap_add_attrfork(if (logflags)xfs_trans_log_inode (tp, ip, logflags);if (error) - goto trans_cancel; +return error;if (!xfs_has_attr (mp) || (!xfs_has_attr2 (mp) && version ==2 )) {bool log_sb = false;
@@ -1096,14 +1087,7 @@ xfs_bmap_add_attrfork(xfs_log_sb (tp); } - error =xfs_trans_commit (tp); -xfs_iunlock (ip,XFS_ILOCK_EXCL ); -return error; - -trans_cancel: -xfs_trans_cancel (tp); -xfs_iunlock (ip,XFS_ILOCK_EXCL ); -return error; +return 0 ; }/*
fs/xfs/libxfs/xfs_bmap.h +2 -1
@@ -176,7 +176,8 @@ int xfs_bmap_longest_free_extent(struct xfs_perag *pag,void xfs_trim_extent (struct xfs_bmbt_irec *irec,xfs_fileoff_t bno,xfs_filblks_t len);unsigned int xfs_bmap_compute_attr_offset (struct xfs_mount *mp); -int xfs_bmap_add_attrfork (struct xfs_inode *ip,int size,int rsvd); +int xfs_bmap_add_attrfork (struct xfs_trans *tp,struct xfs_inode *ip, +int size,int rsvd);void xfs_bmap_local_to_extents_empty (struct xfs_trans *tp,struct xfs_inode *ip,int whichfork); int xfs_bmap_local_to_extents(struct xfs_trans *tp, struct xfs_inode *ip,
PATCH 20/23 xfs: actually rebuild the parent pointer xattrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Once we've assembled all the parent pointers for a file, we need to commit the new dataset atomically to that file. Parent pointer records are embedded in the xattr structure, which means that we must write a new extended attribute structure, again, atomically. Therefore, we must copy the non-parent-pointer attributes from the file being repaired into the temporary file's extended attributes and then call the atomic extent swap mechanism to exchange the blocks.
--- fs/xfs/libxfs/xfs_attr.c | 2 fs/xfs/libxfs/xfs_attr.h | 1 fs/xfs/scrub/attr.c | 2 fs/xfs/scrub/attr_repair.c | 4 fs/xfs/scrub/attr_repair.h | 4 fs/xfs/scrub/findparent.c | 2 fs/xfs/scrub/listxattr.c | 12 + fs/xfs/scrub/listxattr.h | 4 fs/xfs/scrub/parent.c | 2 fs/xfs/scrub/parent_repair.c | 703 +++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/trace.h | 2 11 files changed, 710 insertions(+), 28 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
@@ -941,7 +941,7 @@ xfs_attr_defer_add(trace_xfs_attr_defer_add (new->xattri_dela_state, args->dp); } -STATIC int +int xfs_attr_add_fork( struct xfs_inode *ip, /* incore inode pointer */ int size, /* space new attribute needs */
fs/xfs/libxfs/xfs_attr.h +1 -0
@@ -626,5 +626,6 @@ int __init xfs_attr_intent_init_cache(void);void xfs_attr_intent_destroy_cache (void );int xfs_attr_sf_totsize (struct xfs_inode *dp); +int xfs_attr_add_fork (struct xfs_inode *ip,int size,int rsvd);#endif /* __XFS_ATTR_H__ */
fs/xfs/scrub/attr.c +1 -1
@@ -656,7 +656,7 @@ xchk_xattr(* iteration, which doesn't really follow the usual buffer * locking order. */ - error =xchk_xattr_walk (sc, sc->ip, xchk_xattr_actor,NULL ); + error =xchk_xattr_walk (sc, sc->ip, xchk_xattr_actor,NULL ,NULL );if (!xchk_fblock_process_error (sc,XFS_ATTR_FORK ,0 , &error))return error;
fs/xfs/scrub/attr_repair.c +2 -2
@@ -1039,7 +1039,7 @@ xrep_xattr_reset_fork(* fork. The caller must ILOCK the tempfile and join it to the transaction. * This function returns with the inode joined to a clean scrub transaction. */ -STATIC int +int xrep_xattr_reset_tempfile_fork (struct xfs_scrub *sc) {
@@ -1360,7 +1360,7 @@ xrep_xattr_swap_prep( }/* Exchange the temporary file's attribute fork with the one being repaired. */ -STATIC int +int xrep_xattr_swap (struct xfs_scrub *sc,struct xrep_tempexch *tx)
fs/xfs/scrub/attr_repair.h +4 -0
@@ -6,6 +6,10 @@#ifndef __XFS_SCRUB_ATTR_REPAIR_H__#define __XFS_SCRUB_ATTR_REPAIR_H__ +struct xrep_tempexch ; + +int xrep_xattr_swap (struct xfs_scrub *sc,struct xrep_tempexch *tx);int xrep_xattr_reset_fork (struct xfs_scrub *sc); +int xrep_xattr_reset_tempfile_fork (struct xfs_scrub *sc);#endif /* __XFS_SCRUB_ATTR_REPAIR_H__ */
fs/xfs/scrub/findparent.c +2 -0
@@ -24,6 +24,7 @@#include "xfs_trans_space.h" #include "xfs_health.h" #include "xfs_exchmaps.h" +#include "xfs_parent.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h"
@@ -33,6 +34,7 @@#include "scrub/findparent.h" #include "scrub/readdir.h" #include "scrub/tempfile.h" +#include "scrub/listxattr.h" /* * Finding the Parent of a Directory
fs/xfs/scrub/listxattr.c +10 -2
@@ -267,6 +267,7 @@ xchk_xattr_walk_node(struct xfs_scrub *sc, struct xfs_inode *ip, xchk_xattr_fn attr_fn, + xchk_xattrleaf_fn leaf_fn, void *priv) { struct xfs_attr3_icleaf_hdr leafhdr;
@@ -298,6 +299,12 @@ xchk_xattr_walk_node(xfs_trans_brelse (sc->tp, leaf_bp); +if (leaf_fn) { + error =leaf_fn (sc, priv); +if (error) + goto out_bitmap; + } +/* Make sure we haven't seen this new leaf already. */ len =1 ;if (xdab_bitmap_test (&seen_dablks, leafhdr.forw, &len)) {
@@ -334,6 +341,7 @@ xchk_xattr_walk(struct xfs_scrub *sc, struct xfs_inode *ip, xchk_xattr_fn attr_fn, + xchk_xattrleaf_fn leaf_fn, void *priv) { int error;
@@ -354,7 +362,7 @@ xchk_xattr_walk(if (xfs_attr_is_leaf (ip))return xchk_xattr_walk_leaf (sc, ip, attr_fn, priv); -return xchk_xattr_walk_node (sc, ip, attr_fn, priv); +return xchk_xattr_walk_node (sc, ip, attr_fn, leaf_fn, priv); }/*
@@ -382,5 +390,5 @@ xchk_pptr_walk(ASSERT (xfs_has_parent (sc->mp)); -return xchk_xattr_walk (sc, ip, xchk_pptr_walk_attr, &pw); +return xchk_xattr_walk (sc, ip, xchk_pptr_walk_attr,NULL , &pw); }
fs/xfs/scrub/listxattr.h +3 -1
@@ -11,8 +11,10 @@ typedef int (*xchk_xattr_fn)(struct xfs_scrub *sc, struct xfs_inode *ip, unsigned int namelen, const void *value, unsigned int valuelen, void *priv); +typedef int (*xchk_xattrleaf_fn )(struct xfs_scrub *sc,void *priv); +int xchk_xattr_walk (struct xfs_scrub *sc,struct xfs_inode *ip, -xchk_xattr_fn attr_fn,void *priv); +xchk_xattr_fn attr_fn,xchk_xattrleaf_fn leaf_fn,void *priv);struct xfs_parent_name_irec ;
fs/xfs/scrub/parent.c +1 -1
@@ -768,7 +768,7 @@ xchk_parent_pptr(if (error) goto out_entries; - error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_attr, pp); + error =xchk_xattr_walk (sc, sc->ip, xchk_parent_scan_attr,NULL , pp);if (error == -ECANCELED ) { error =0 ; goto out_names;
fs/xfs/scrub/parent_repair.c +683 -20
846 lines not on this page · raw message
fs/xfs/scrub/trace.h +2 -0
@@ -2532,6 +2532,8 @@ DEFINE_EVENT(xrep_xattr_salvage_class, name, \ TP_ARGS(ip, flags, name, namelen, valuelen))DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_xattr_salvage_rec);DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_xattr_insert_rec); +DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_parent_stash_xattr); +DEFINE_XREP_XATTR_SALVAGE_EVENT (xrep_parent_insert_xattr); DECLARE_EVENT_CLASS(xrep_pptr_salvage_class, TP_PROTO(struct xfs_inode *ip, unsigned int flags, const void *name,
PATCH 21/23 xfs: adapt the orphanage code to handle parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Adapt the orphanage's adoption code to update the child file's parent pointers as part of the reparenting process. Also ensure that the child has an attr fork to receive the parent pointer update, since the runtime code assumes one exists.
--- fs/xfs/scrub/orphanage.c | 42 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/orphanage.h | 3 +++ fs/xfs/scrub/scrub.c | 2 ++ 3 files changed, 47 insertions(+)
fs/xfs/scrub/orphanage.c +42 -0
@@ -19,6 +19,8 @@#include "xfs_icache.h" #include "xfs_bmap.h" #include "xfs_bmap_btree.h" +#include "xfs_parent.h" +#include "xfs_attr_sf.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/repair.h"
@@ -330,8 +332,12 @@ xrep_adoption_trans_alloc(if (S_ISDIR (VFS_I (sc->ip)->i_mode)) child_blkres =xfs_rename_space_res (mp,0 , false, xfs_name_dotdot.len, false); +if (xfs_has_parent (mp)) + child_blkres +=XFS_ADDAFORK_SPACE_RES (mp); adopt->child_blkres = child_blkres; +xfs_parent_args_init (mp, &adopt->ppargs); +/* * Allocate a transaction to link the child into the parent, along with * enough disk space to handle expansion of both the orphanage and the
@@ -500,6 +506,21 @@ xrep_adoption_zap_dcache(dput (d_orphanage); } +/* +* If we have to add an attr fork ahead of a parent pointer update, how much +* space should we ask for? +*/ +static inline int +xrep_adoption_attr_sizeof ( +const struct xrep_adoption *adopt) +{ +size_t res =sizeof (struct xfs_attr_sf_hdr ); + + res +=xfs_attr_sf_entsize_byname (sizeof (struct xfs_parent_name_rec ), + adopt->xname.len); +return res; +} +/* * Move the current file to the orphanage under the computed name. *
@@ -522,6 +543,19 @@ xrep_adoption_move(if (error)return error; +/* +* If this filesystem has parent pointers, ensure that the file being +* moved to the orphanage has an attribute fork. This is required +* because the parent pointer code does not itself add attr forks. +*/ +if (!xfs_inode_has_attr_fork (sc->ip) &&xfs_has_parent (sc->mp)) { +int sf_size =xrep_adoption_attr_sizeof (adopt); + + error =xfs_bmap_add_attrfork (sc->tp, sc->ip, sf_size, true); +if (error) +return error; + } +/* Create the new name in the orphanage. */ error =xfs_dir_createname (sc->tp, sc->orphanage, xname, sc->ip->i_ino, adopt->orphanage_blkres);
@@ -546,6 +580,14 @@ xrep_adoption_move(return error; } +/* Add a parent pointer from the file back to the lost+found. */ +if (xfs_has_parent (sc->mp)) { + error =xfs_parent_addname (sc->tp, &adopt->ppargs, + sc->orphanage, xname, sc->ip); +if (error) +return error; + } +/* * Notify dirent hooks that we moved the file to /lost+found, and * finish all the deferred work so that we know the adoption is fully
fs/xfs/scrub/orphanage.h +3 -0
@@ -54,6 +54,9 @@ struct xrep_adoption {struct xfs_scrub *sc; +/* Parent pointer context tracking */ +struct xfs_parent_args ppargs; +/* Block reservations for orphanage and child (if directory). */ unsigned int orphanage_blkres;unsigned int child_blkres;
fs/xfs/scrub/scrub.c +2 -0
@@ -19,6 +19,8 @@#include "xfs_rmap.h" #include "xfs_exchrange.h" #include "xfs_exchmaps.h" +#include "xfs_dir2.h" +#include "xfs_parent.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/trace.h"
PATCH 22/23 xfs: repair link count of nondirectories after rebuilding parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Since the parent pointer scrubber does not exhaustively search the filesystem for missing parent pointers, it doesn't have a good way to determine that there are pointers missing from an otherwise uncorrupt xattr structure. Instead, for nondirectories it employs a heuristic of comparing the file link count to the number of parent pointers found.
However, we don't want this heuristic flagging a false corruption after a repair has actually scanned the entire filesystem to rebuild the parent pointers. Therefore, reset the file link count in this one case because we actually know the correct link count.
--- fs/xfs/scrub/parent_repair.c | 97 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+)
fs/xfs/scrub/parent_repair.c +97 -0
@@ -27,6 +27,7 @@#include "xfs_parent.h" #include "xfs_attr.h" #include "xfs_bmap.h" +#include "xfs_ag.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h"
@@ -160,6 +161,9 @@ struct xrep_parent {* lost+found filename if we need to reparent the file. */ struct xfs_parent_name_irec pptr; + +/* Number of parents we found after all other repairs */ +unsigned long long parents; };struct xrep_parent_xattr {
@@ -1382,6 +1386,92 @@ xrep_parent_rebuild_tree(return 0 ; } +/* Count the number of parent pointers. */ +STATIC int +xrep_parent_count_pptr ( +struct xfs_scrub *sc, +struct xfs_inode *ip, +const struct xfs_parent_name_irec *pptr, +void *priv) +{ +struct xrep_parent *rp = priv; + +if (!xfs_parent_verify_irec (sc->mp, pptr)) +return -EFSCORRUPTED ; + + rp->parents++; +return 0 ; +} + +/* +* After all parent pointer rebuilding and adoption activity completes, reset +* the link count of this nondirectory, having scanned the fs to rebuild all +* parent pointers. +*/ +STATIC int +xrep_parent_set_nondir_nlink ( +struct xrep_parent *rp) +{ +struct xfs_scrub *sc = rp->sc; +struct xfs_inode *ip = sc->ip; +struct xfs_perag *pag; +bool joined = false; +int error; + +/* Count parent pointers so we can reset the file link count. */ + rp->parents =0 ; + error =xchk_pptr_walk (sc, ip, xrep_parent_count_pptr, &rp->pptr, rp); +if (error) +return error; + +if (rp->parents >0 &&xfs_inode_on_unlinked_list (ip)) { +xfs_trans_ijoin (sc->tp, sc->ip,0 ); + joined = true; + +/* +* The file is on the unlinked list but we found parents. +* Remove the file from the unlinked list. +*/ + pag =xfs_perag_get (sc->mp,XFS_INO_TO_AGNO (sc->mp, ip->i_ino)); +if (!pag) { +ASSERT (0 ); +return -EFSCORRUPTED ; + } + + error =xfs_iunlink_remove (sc->tp, pag, ip); +xfs_perag_put (pag); +if (error) +return error; + }else if (rp->parents ==0 && !xfs_inode_on_unlinked_list (ip)) { +xfs_trans_ijoin (sc->tp, sc->ip,0 ); + joined = true; + +/* +* The file is not on the unlinked list but we found no +* parents. Add the file to the unlinked list. +*/ + error =xfs_iunlink (sc->tp, ip); +if (error) +return error; + } + +/* Set the correct link count. */ +if (VFS_I (ip)->i_nlink != rp->parents) { +if (!joined) { +xfs_trans_ijoin (sc->tp, sc->ip,0 ); + joined = true; + } + + set_nlink(VFS_I(ip), min_t(unsigned long long, rp->parents, +XFS_NLINK_PINNED )); + } + +/* Log the inode to keep it moving forward if we dirtied anything. */ +if (joined) +xfs_trans_log_inode (sc->tp, ip,XFS_ILOG_CORE ); +return 0 ; +} +/* Set up the filesystem scan so we can look for parents. */ STATIC int xrep_parent_setup_scan(
@@ -1506,6 +1596,13 @@ xrep_parent( error =xrep_parent_rebuild_tree (rp);if (error) goto out_teardown; +if (xfs_has_parent (sc->mp) && !S_ISDIR (VFS_I (sc->ip)->i_mode)) { + error =xrep_parent_set_nondir_nlink (rp); +if (error) + goto out_teardown; + } + + error =xrep_defer_finish (sc); out_teardown:xrep_parent_teardown (rp);
PATCH 23/23 xfs: inode repair should ensure there's an attr fork to store parent pointers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The runtime parent pointer update code expects that any file being moved around the directory tree already has an attr fork. However, if we had to rebuild an inode core record, there's a chance that we zeroed forkoff as part of the inode to pass the iget verifiers.
Therefore, if we performed any repairs on an inode core, ensure that the inode has a nonzero forkoff before unlocking the inode.
--- fs/xfs/scrub/inode_repair.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
fs/xfs/scrub/inode_repair.c +41 -0
@@ -1688,6 +1688,44 @@ xrep_inode_extsize( } } +/* Ensure this file has an attr fork if it needs to hold a parent pointer. */ +STATIC int +xrep_inode_pptr ( +struct xfs_scrub *sc) +{ +struct xfs_mount *mp = sc->mp; +struct xfs_inode *ip = sc->ip; +struct inode *inode =VFS_I (ip); + +if (!xfs_has_parent (mp)) +return 0 ; + +/* +* Unlinked inodes that cannot be added to the directory tree will not +* have a parent pointer. +*/ +if (inode->i_nlink ==0 && !(inode->i_state &I_LINKABLE )) +return 0 ; + +/* The root directory doesn't have a parent pointer. */ +if (ip == mp->m_rootip) +return 0 ; + +/* +* Metadata inodes are rooted in the superblock and do not have any +* parents. +*/ +if (xfs_is_metadata_inode (ip)) +return 0 ; + +/* Inode already has an attr fork; no further work possible here. */ +if (xfs_inode_has_attr_fork (ip)) +return 0 ; + +return xfs_bmap_add_attrfork (sc->tp, ip, +sizeof (struct xfs_attr_sf_hdr ), true); +} +/* Fix any irregularities in an inode that the verifiers don't catch. */ STATIC int xrep_inode_problems(
@@ -1696,6 +1734,9 @@ xrep_inode_problems(int error; error =xrep_inode_blockcounts (sc); +if (error) +return error; + error =xrep_inode_pptr (sc);if (error)return error;xrep_inode_timestamps (sc->ip);