Skip to contentSkip to conversation

xfs: Parent Pointers

RFC New Darrick J. Wong 3 participants 227 messages Feb 16, 2023 – Mar 14, 2023 XFS filesystem Page 1 of 3

lore
Share & tools
lore
#0 lore raw

Hi everyone,

This deluge contains all of the additions to the parent pointers patchset that I've been working on for the past month. The kernel and xfsprogs patchsets are based on Allison's v9r2 tag from last week; the fstests patches are merely a part of my development tree. To recap Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point."

The kernel branches start with a number of buf fixes that I need to get fstests to pass. I also restructured the kernel implementation of GETPARENTS to cut the memory usage considerably.

For userspace, I cleaned up the xfsprogs patches so that libxfs-diff shows no discrepancies with the kernel and cleaned up the parent pointer usage code that I prototyped in 2017 so that it's less buggy and moldy. I also rewired xfs_scrub to use GETPARENTS to report file paths of corrupt files instead of inode numbers, since that part had bitrotted badly.

With that out of the way, I implemented a prototype of online repairs for directories and parent pointers. This is only a proof of concept, because I had already backported many many patches from part 1 of online repair, and didn't feel like porting the parts needed to commit new structures atomically and reap the old dir/xattr blocks. IOWs, the prototype scans the filesystem to build a parallel directory or xattr structure, and then reports on any discrepancies between the two versions. Obviously this won't fix a corrupt directory tree, but it enables us to test the repair code on a consistent filesystem to demonstrate that it works.

Next, I implemented fully functional parent pointer checking and repair for xfs_repair. This was less hard than I guessed it would be because the current design of phase 6 includes a walk of all directories. From the dirent data, we can build a per-AG index of all the parent pointers for all the inodes in that AG, then walk all the inodes in that AG to compare the lists. As you might guess, this eats a fair amount of memory, even with a rudimentary dirent name deduplication table to cut down on memory usage.

After that, I moved on to solving the major problem that I've been having with the directory repair code, and that is the problem of reconstructing dirents at the offsets specified by the parent pointers. The details of the problem and how I dealt with it are captured in the cover letter for those patches. Suffice to say, we now encode the dirent name in the parent pointer attrname (or a collision resistant hash if it doesn't fit), which makes it possible to commit new directories atomically.

The last part of this patchset reorganizes the XFS_IOC_GETPARENTS ioctl to encode variable length parent pointer records in the caller's buffer. The denser encodings mean that we can extract the parent list with fewer kernel calls.

--D

#1 lore raw

cover xfs: Parent Pointers

Hi all,

This is the latest parent pointer attributes for xfs.
The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.

This set can be viewed on github here
https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv9_r2

And the corresponding xfsprogs code is here
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v9_r2

This set has been tested with the below parent pointers tests
https://lore.kernel.org/fstests/[email protected]/T/#t

Updates since v8:

xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec

xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas

xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates

xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free

xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail
Move parent frees to out_release_wip

xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename

Questions comments and feedback appreciated!

Thanks all!
Allison

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs

---
 fs/xfs/Makefile                 |    2 
 fs/xfs/libxfs/xfs_attr.c        |   71 +++++-
 fs/xfs/libxfs/xfs_attr.h        |   13 +
 fs/xfs/libxfs/xfs_da_btree.h    |    3 
 fs/xfs/libxfs/xfs_da_format.h   |   26 ++
 fs/xfs/libxfs/xfs_defer.c       |   28 ++
 fs/xfs/libxfs/xfs_defer.h       |    8 +
 fs/xfs/libxfs/xfs_dir2.c        |   21 +-
 fs/xfs/libxfs/xfs_dir2.h        |    7 -
 fs/xfs/libxfs/xfs_dir2_block.c  |    9 -
 fs/xfs/libxfs/xfs_dir2_leaf.c   |    8 +
 fs/xfs/libxfs/xfs_dir2_node.c   |    8 +
 fs/xfs/libxfs/xfs_dir2_sf.c     |    6 +
 fs/xfs/libxfs/xfs_format.h      |    4 
 fs/xfs/libxfs/xfs_fs.h          |   75 +++++++
 fs/xfs/libxfs/xfs_log_format.h  |    7 -
 fs/xfs/libxfs/xfs_log_rlimit.c  |   53 +++++
 fs/xfs/libxfs/xfs_parent.c      |  203 ++++++++++++++++++
 fs/xfs/libxfs/xfs_parent.h      |   84 +++++++
 fs/xfs/libxfs/xfs_sb.c          |    4 
 fs/xfs/libxfs/xfs_trans_resv.c  |  324 ++++++++++++++++++++++++----
 fs/xfs/libxfs/xfs_trans_space.h |    8 -
 fs/xfs/scrub/attr.c             |    4 
 fs/xfs/xfs_attr_item.c          |  142 ++++++++++--
 fs/xfs/xfs_attr_item.h          |    1 
 fs/xfs/xfs_attr_list.c          |   17 +
 fs/xfs/xfs_dquot.c              |   38 +++
 fs/xfs/xfs_dquot.h              |    1 
 fs/xfs/xfs_file.c               |    1 
 fs/xfs/xfs_inode.c              |  447 +++++++++++++++++++++++++++++++--------
 fs/xfs/xfs_inode.h              |    3 
 fs/xfs/xfs_ioctl.c              |  148 +++++++++++--
 fs/xfs/xfs_ioctl.h              |    2 
 fs/xfs/xfs_iops.c               |    3 
 fs/xfs/xfs_ondisk.h             |    4 
 fs/xfs/xfs_parent_utils.c       |  126 +++++++++++
 fs/xfs/xfs_parent_utils.h       |   11 +
 fs/xfs/xfs_qm.c                 |    4 
 fs/xfs/xfs_qm.h                 |    2 
 fs/xfs/xfs_super.c              |   14 +
 fs/xfs/xfs_symlink.c            |   58 ++++-
 fs/xfs/xfs_trans.c              |   13 +
 fs/xfs/xfs_trans_dquot.c        |   15 +
 fs/xfs/xfs_xattr.c              |    7 -
 fs/xfs/xfs_xattr.h              |    2 
 45 files changed, 1782 insertions(+), 253 deletions(-)
 create mode 100644 fs/xfs/libxfs/xfs_parent.c
 create mode 100644 fs/xfs/libxfs/xfs_parent.h
 create mode 100644 fs/xfs/xfs_parent_utils.c
 create mode 100644 fs/xfs/xfs_parent_utils.h
#2 lore raw

PATCH 1/28 xfs: Add new name to attri/d

From: Allison Henderson <[email protected]>

This patch adds two new fields to the atti/d. They are nname and nnamelen. This will be used for parent pointer updates since a rename operation may cause the parent pointer to update both the name and value. So we need to carry both the new name as well as the target name in the attri/d.

---
 fs/xfs/libxfs/xfs_attr.c       |   12 +++-
 fs/xfs/libxfs/xfs_attr.h       |    4 +
 fs/xfs/libxfs/xfs_da_btree.h   |    2 +
 fs/xfs/libxfs/xfs_log_format.h |    6 +-
 fs/xfs/xfs_attr_item.c         |  135 +++++++++++++++++++++++++++++++++-------
 fs/xfs/xfs_attr_item.h         |    1 
 6 files changed, 133 insertions(+), 27 deletions(-)
fs/xfs/libxfs/xfs_attr.c +11 -1
index e28d93d232de..b1dbed7655e8 100644
@@ -423,6 +423,12 @@ xfs_attr_complete_op(
 	args->op_flags &= ~XFS_DA_OP_REPLACE;
 	if (do_replace) {
 		args->attr_filter &= ~XFS_ATTR_INCOMPLETE;
+		if (args->new_namelen > 0) {
+			args->name = args->new_name;
+			args->namelen = args->new_namelen;
+			args->hashval = xfs_da_hashname(args->name,
+							args->namelen);
+		}
 		return replace_state;
 	}
 	return XFS_DAS_DONE;
@@ -922,9 +928,13 @@ xfs_attr_defer_replace(
 	struct xfs_da_args	*args)
 {
 	struct xfs_attr_intent	*new;
+	int			op_flag;
 	int			error = 0;
 
-	error = xfs_attr_intent_init(args, XFS_ATTRI_OP_FLAGS_REPLACE, &new);
+	op_flag = args->new_namelen == 0 ? XFS_ATTRI_OP_FLAGS_REPLACE :
+		  XFS_ATTRI_OP_FLAGS_NVREPLACE;
+
+	error = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
 		return error;
 
fs/xfs/libxfs/xfs_attr.h +2 -2
index 81be9b3e4004..3e81f3f48560 100644
@@ -510,8 +510,8 @@ struct xfs_attr_intent {
 	struct xfs_da_args		*xattri_da_args;
 
 	/*
-	 * Shared buffer containing the attr name and value so that the logging
-	 * code can share large memory buffers between log items.
+	 * Shared buffer containing the attr name, new name, and value so that
+	 * the logging code can share large memory buffers between log items.
 	 */
 	struct xfs_attri_log_nameval	*xattri_nameval;
 
fs/xfs/libxfs/xfs_da_btree.h +2 -0
index ffa3df5b2893..a4b29827603f 100644
@@ -55,7 +55,9 @@ enum xfs_dacmp {
 typedef struct xfs_da_args {
 	struct xfs_da_geometry *geo;	/* da block geometry */
 	const uint8_t		*name;		/* string (maybe not NULL terminated) */
+	const uint8_t	*new_name;	/* new attr name */
 	int		namelen;	/* length of string (maybe no NULL) */
+	int		new_namelen;	/* new attr name len */
 	uint8_t		filetype;	/* filetype of inode for directories */
 	void		*value;		/* set of bytes (maybe contain NULLs) */
 	int		valuelen;	/* length of value */
fs/xfs/libxfs/xfs_log_format.h +4 -2
index f13e0809dc63..ae9c99762a24 100644
@@ -117,7 +117,8 @@ struct xfs_unmount_log_format {
 #define XLOG_REG_TYPE_ATTRD_FORMAT	28
 #define XLOG_REG_TYPE_ATTR_NAME	29
 #define XLOG_REG_TYPE_ATTR_VALUE	30
-#define XLOG_REG_TYPE_MAX		30
+#define XLOG_REG_TYPE_ATTR_NNAME	31
+#define XLOG_REG_TYPE_MAX		31
 
 
 /*
@@ -957,6 +958,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_SET		1	/* Set the attribute */
 #define XFS_ATTRI_OP_FLAGS_REMOVE	2	/* Remove the attribute */
 #define XFS_ATTRI_OP_FLAGS_REPLACE	3	/* Replace the attribute */
+#define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
@@ -974,7 +976,7 @@ struct xfs_icreate_log {
 struct xfs_attri_log_format {
 	uint16_t	alfi_type;	/* attri log item type */
 	uint16_t	alfi_size;	/* size of this item */
-	uint32_t	__pad;		/* pad to 64 bit aligned */
+	uint32_t	alfi_nname_len;	/* attr new name length */
 	uint64_t	alfi_id;	/* attri identifier */
 	uint64_t	alfi_ino;	/* the inode for this attr operation */
 	uint32_t	alfi_op_flags;	/* marks the op as a set or remove */
fs/xfs/xfs_attr_item.c +113 -22

265 lines not on this page · raw message

fs/xfs/xfs_attr_item.h +1 -0
index 3280a7930287..24d4968dd6cc 100644
@@ -13,6 +13,7 @@ struct kmem_zone;
 
 struct xfs_attri_log_nameval {
 	struct xfs_log_iovec	name;
+	struct xfs_log_iovec	nname;
 	struct xfs_log_iovec	value;
 	refcount_t		refcount;
 
#3 lore raw

PATCH 2/28 xfs: Increase XFS_DEFER_OPS_NR_INODES to 5

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Renames that generate parent pointer updates can join up to 5 inodes locked in sorted order. So we need to increase the number of defer ops inodes and relock them in the same way.

---
 fs/xfs/libxfs/xfs_defer.c |   28 ++++++++++++++++++++++++++--
 fs/xfs/libxfs/xfs_defer.h |    8 +++++++-
 fs/xfs/xfs_inode.c        |    2 +-
 fs/xfs/xfs_inode.h        |    1 +
 4 files changed, 35 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_defer.c +26 -2
index 5a321b783398..c0279b57e51d 100644
@@ -820,13 +820,37 @@ xfs_defer_ops_continue(
 	struct xfs_trans		*tp,
 	struct xfs_defer_resources	*dres)
 {
-	unsigned int			i;
+	unsigned int			i, j;
+	struct xfs_inode		*sips[XFS_DEFER_OPS_NR_INODES];
+	struct xfs_inode		*temp;
 
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 	ASSERT(!(tp->t_flags & XFS_TRANS_DIRTY));
 
 	/* Lock the captured resources to the new transaction. */
-	if (dfc->dfc_held.dr_inos == 2)
+	if (dfc->dfc_held.dr_inos > 2) {
+		/*
+		 * Renames with parent pointer updates can lock up to 5 inodes,
+		 * sorted by their inode number.  So we need to make sure they
+		 * are relocked in the same way.
+		 */
+		memset(sips, 0, sizeof(sips));
+		for (i = 0; i < dfc->dfc_held.dr_inos; i++)
+			sips[i] = dfc->dfc_held.dr_ip[i];
+
+		/* Bubble sort of at most 5 inodes */
+		for (i = 0; i < dfc->dfc_held.dr_inos; i++) {
+			for (j = 1; j < dfc->dfc_held.dr_inos; j++) {
+				if (sips[j]->i_ino < sips[j-1]->i_ino) {
+					temp = sips[j];
+					sips[j] = sips[j-1];
+					sips[j-1] = temp;
+				}
+			}
+		}
+
+		xfs_lock_inodes(sips, dfc->dfc_held.dr_inos, XFS_ILOCK_EXCL);
+	} else if (dfc->dfc_held.dr_inos == 2)
 		xfs_lock_two_inodes(dfc->dfc_held.dr_ip[0], XFS_ILOCK_EXCL,
 				    dfc->dfc_held.dr_ip[1], XFS_ILOCK_EXCL);
 	else if (dfc->dfc_held.dr_inos == 1)
fs/xfs/libxfs/xfs_defer.h +7 -1
index 114a3a4930a3..fdf6941f8f4d 100644
@@ -70,7 +70,13 @@ extern const struct xfs_defer_op_type xfs_attr_defer_type;
 /*
  * Deferred operation item relogging limits.
  */
-#define XFS_DEFER_OPS_NR_INODES	2	/* join up to two inodes */
+
+/*
+ * Rename w/ parent pointers can require up to 5 inodes with deferred ops to
+ * be joined to the transaction: src_dp, target_dp, src_ip, target_ip, and wip.
+ * These inodes are locked in sorted order by their inode numbers
+ */
+#define XFS_DEFER_OPS_NR_INODES	5
 #define XFS_DEFER_OPS_NR_BUFS	2	/* join up to two buffers */
 
 /* Resources that must be held across a transaction roll. */
fs/xfs/xfs_inode.c +1 -1
index d354ea2b74f9..27532053a67b 100644
@@ -447,7 +447,7 @@ xfs_lock_inumorder(
  * lock more than one at a time, lockdep will report false positives saying we
  * have violated locking orders.
  */
-static void
+void
 xfs_lock_inodes(
 	struct xfs_inode	**ips,
 	int			inodes,
fs/xfs/xfs_inode.h +1 -0
index fa780f08dc89..2eaed98af814 100644
@@ -574,5 +574,6 @@ void xfs_end_io(struct work_struct *work);
 
 int xfs_ilock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);
 void xfs_iunlock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);
+void xfs_lock_inodes(struct xfs_inode **ips, int inodes, uint lock_mode);
 
 #endif	/* __XFS_INODE_H__ */
#4 lore raw

PATCH 3/28 xfs: Increase XFS_QM_TRANS_MAXDQS to 5

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

With parent pointers enabled, a rename operation can update up to 5
inodes: src_dp, target_dp, src_ip, target_ip and wip. This causes their dquots to a be attached to the transaction chain, so we need to increase XFS_QM_TRANS_MAXDQS. This patch also add a helper function xfs_dqlockn to lock an arbitrary number of dquots.

---
 fs/xfs/xfs_dquot.c       |   38 ++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_dquot.h       |    1 +
 fs/xfs/xfs_qm.h          |    2 +-
 fs/xfs/xfs_trans_dquot.c |   15 ++++++++++-----
 4 files changed, 50 insertions(+), 6 deletions(-)
fs/xfs/xfs_dquot.c +38 -0
index 8fb90da89787..9f311729c4c8 100644
@@ -1333,6 +1333,44 @@ xfs_dqlock2(
 	}
 }
 
+static int
+xfs_dqtrx_cmp(
+	const void		*a,
+	const void		*b)
+{
+	const struct xfs_dqtrx	*qa = a;
+	const struct xfs_dqtrx	*qb = b;
+
+	if (qa->qt_dquot->q_id > qb->qt_dquot->q_id)
+		return 1;
+	if (qa->qt_dquot->q_id < qb->qt_dquot->q_id)
+		return -1;
+	return 0;
+}
+
+void
+xfs_dqlockn(
+	struct xfs_dqtrx	*q)
+{
+	unsigned int		i;
+
+	/* Sort in order of dquot id, do not allow duplicates */
+	for (i = 0; i < XFS_QM_TRANS_MAXDQS && q[i].qt_dquot != NULL; i++) {
+		unsigned int	j;
+
+		for (j = 0; j < i; j++)
+			ASSERT(q[i].qt_dquot != q[j].qt_dquot);
+	}
+	if (i == 0)
+		return;
+
+	sort(q, i, sizeof(struct xfs_dqtrx), xfs_dqtrx_cmp, NULL);
+
+	mutex_lock(&q[0].qt_dquot->q_qlock);
+	for (i = 1; i < XFS_QM_TRANS_MAXDQS && q[i].qt_dquot != NULL; i++)
+		mutex_lock_nested(&q[i].qt_dquot->q_qlock, XFS_QLOCK_NESTED);
+}
+
 int __init
 xfs_qm_init(void)
 {
fs/xfs/xfs_dquot.h +1 -0
index 80c8f851a2f3..dc7d0226242b 100644
@@ -223,6 +223,7 @@ int		xfs_qm_dqget_uncached(struct xfs_mount *mp,
 void		xfs_qm_dqput(struct xfs_dquot *dqp);
 
 void		xfs_dqlock2(struct xfs_dquot *, struct xfs_dquot *);
+void		xfs_dqlockn(struct xfs_dqtrx *q);
 
 void		xfs_dquot_set_prealloc_limits(struct xfs_dquot *);
 
fs/xfs/xfs_qm.h +1 -1
index 9683f0457d19..c6ec88779356 100644
@@ -120,7 +120,7 @@ enum {
 	XFS_QM_TRANS_PRJ,
 	XFS_QM_TRANS_DQTYPES
 };
-#define XFS_QM_TRANS_MAXDQS		2
+#define XFS_QM_TRANS_MAXDQS		5
 struct xfs_dquot_acct {
 	struct xfs_dqtrx	dqs[XFS_QM_TRANS_DQTYPES][XFS_QM_TRANS_MAXDQS];
 };
fs/xfs/xfs_trans_dquot.c +10 -5
index aa00cf67ad72..8a48175ea3a7 100644
@@ -268,24 +268,29 @@ xfs_trans_mod_dquot(
 
 /*
  * Given an array of dqtrx structures, lock all the dquots associated and join
- * them to the transaction, provided they have been modified.  We know that the
- * highest number of dquots of one type - usr, grp and prj - involved in a
- * transaction is 3 so we don't need to make this very generic.
+ * them to the transaction, provided they have been modified.
  */
 STATIC void
 xfs_trans_dqlockedjoin(
 	struct xfs_trans	*tp,
 	struct xfs_dqtrx	*q)
 {
+	unsigned int		i;
 	ASSERT(q[0].qt_dquot != NULL);
 	if (q[1].qt_dquot == NULL) {
 		xfs_dqlock(q[0].qt_dquot);
 		xfs_trans_dqjoin(tp, q[0].qt_dquot);
-	} else {
-		ASSERT(XFS_QM_TRANS_MAXDQS == 2);
+	} else if (q[2].qt_dquot == NULL) {
 		xfs_dqlock2(q[0].qt_dquot, q[1].qt_dquot);
 		xfs_trans_dqjoin(tp, q[0].qt_dquot);
 		xfs_trans_dqjoin(tp, q[1].qt_dquot);
+	} else {
+		xfs_dqlockn(q);
+		for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
+			if (q[i].qt_dquot == NULL)
+				break;
+			xfs_trans_dqjoin(tp, q[i].qt_dquot);
+		}
 	}
 }
 
#5 lore raw

PATCH 4/28 xfs: Hold inode locks in xfs_ialloc

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Modify xfs_ialloc to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations

---
 fs/xfs/xfs_inode.c   |    8 +++++++-
 fs/xfs/xfs_qm.c      |    4 +++-
 fs/xfs/xfs_symlink.c |    3 +++
 3 files changed, 13 insertions(+), 2 deletions(-)
fs/xfs/xfs_inode.c +7 -1
index 27532053a67b..772e3f105b7b 100644
@@ -774,6 +774,8 @@ xfs_inode_inherit_flags2(
 /*
  * Initialise a newly allocated inode and return the in-core inode to the
  * caller locked exclusively.
+ *
+ * Caller is responsible for unlocking the inode manually upon return
  */
 int
 xfs_init_new_inode(
@@ -899,7 +901,7 @@ xfs_init_new_inode(
 	/*
 	 * Log the new values stuffed into the inode.
 	 */
-	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
+	xfs_trans_ijoin(tp, ip, 0);
 	xfs_trans_log_inode(tp, ip, flags);
 
 	/* now that we have an i_mode we can setup the inode structure */
@@ -1076,6 +1078,7 @@ xfs_create(
 	xfs_qm_dqrele(pdqp);
 
 	*ipp = ip;
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	return 0;
 
  out_trans_cancel:
@@ -1089,6 +1092,7 @@ xfs_create(
 	if (ip) {
 		xfs_finish_inode_setup(ip);
 		xfs_irele(ip);
+		xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	}
  out_release_dquots:
 	xfs_qm_dqrele(udqp);
@@ -1172,6 +1176,7 @@ xfs_create_tmpfile(
 	xfs_qm_dqrele(pdqp);
 
 	*ipp = ip;
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	return 0;
 
  out_trans_cancel:
@@ -1185,6 +1190,7 @@ xfs_create_tmpfile(
 	if (ip) {
 		xfs_finish_inode_setup(ip);
 		xfs_irele(ip);
+		xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	}
  out_release_dquots:
 	xfs_qm_dqrele(udqp);
fs/xfs/xfs_qm.c +3 -1
index e2c542f6dcd4..fbecf54d3b44 100644
@@ -826,8 +826,10 @@ xfs_qm_qino_alloc(
 		ASSERT(xfs_is_shutdown(mp));
 		xfs_alert(mp, "%s failed (error %d)!", __func__, error);
 	}
-	if (need_alloc)
+	if (need_alloc) {
 		xfs_finish_inode_setup(*ipp);
+		xfs_iunlock(*ipp, XFS_ILOCK_EXCL);
+	}
 	return error;
 }
 
fs/xfs/xfs_symlink.c +3 -0
index 8389f3ef88ef..d8e120913036 100644
@@ -337,6 +337,7 @@ xfs_symlink(
 	xfs_qm_dqrele(pdqp);
 
 	*ipp = ip;
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	return 0;
 
 out_trans_cancel:
@@ -358,6 +359,8 @@ xfs_symlink(
 
 	if (unlock_dp_on_error)
 		xfs_iunlock(dp, XFS_ILOCK_EXCL);
+	if (ip)
+		xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	return error;
 }
 
#6 lore raw

PATCH 5/28 xfs: Hold inode locks in xfs_trans_alloc_dir

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Modify xfs_trans_alloc_dir to hold locks after return. Caller will be responsible for manual unlock. We will need this later to hold locks across parent pointer operations

---
 fs/xfs/xfs_inode.c |   14 ++++++++++++--
 fs/xfs/xfs_trans.c |    9 +++++++--
 2 files changed, 19 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +12 -2
index 772e3f105b7b..e292688ee608 100644
@@ -1279,10 +1279,15 @@ xfs_link(
 	if (xfs_has_wsync(mp) || xfs_has_dirsync(mp))
 		xfs_trans_set_sync(tp);
 
-	return xfs_trans_commit(tp);
+	error = xfs_trans_commit(tp);
+	xfs_iunlock(tdp, XFS_ILOCK_EXCL);
+	xfs_iunlock(sip, XFS_ILOCK_EXCL);
+	return error;
 
  error_return:
 	xfs_trans_cancel(tp);
+	xfs_iunlock(tdp, XFS_ILOCK_EXCL);
+	xfs_iunlock(sip, XFS_ILOCK_EXCL);
  std_return:
 	if (error == -ENOSPC && nospace_error)
 		error = nospace_error;
@@ -2518,15 +2523,20 @@ xfs_remove(
 
 	error = xfs_trans_commit(tp);
 	if (error)
-		goto std_return;
+		goto out_unlock;
 
 	if (is_dir && xfs_inode_is_filestream(ip))
 		xfs_filestream_deassociate(ip);
 
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
+	xfs_iunlock(dp, XFS_ILOCK_EXCL);
 	return 0;
 
  out_trans_cancel:
 	xfs_trans_cancel(tp);
+ out_unlock:
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
+	xfs_iunlock(dp, XFS_ILOCK_EXCL);
  std_return:
 	return error;
 }
fs/xfs/xfs_trans.c +7 -2
index 7bd16fbff534..43f4b0943f49 100644
@@ -1356,6 +1356,8 @@ xfs_trans_alloc_ichange(
  * The caller must ensure that the on-disk dquots attached to this inode have
  * already been allocated and initialized.  The ILOCKs will be dropped when the
  * transaction is committed or cancelled.
+ *
+ * Caller is responsible for unlocking the inodes manually upon return
  */
 int
 xfs_trans_alloc_dir(
@@ -1386,8 +1388,8 @@ xfs_trans_alloc_dir(
 
 	xfs_lock_two_inodes(dp, XFS_ILOCK_EXCL, ip, XFS_ILOCK_EXCL);
 
-	xfs_trans_ijoin(tp, dp, XFS_ILOCK_EXCL);
-	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
+	xfs_trans_ijoin(tp, dp, 0);
+	xfs_trans_ijoin(tp, ip, 0);
 
 	error = xfs_qm_dqattach_locked(dp, false);
 	if (error) {
@@ -1410,6 +1412,9 @@ xfs_trans_alloc_dir(
 	if (error == -EDQUOT || error == -ENOSPC) {
 		if (!retried) {
 			xfs_trans_cancel(tp);
+			xfs_iunlock(dp, XFS_ILOCK_EXCL);
+			if (dp != ip)
+				xfs_iunlock(ip, XFS_ILOCK_EXCL);
 			xfs_blockgc_free_quota(dp, 0);
 			retried = true;
 			goto retry;
#7 lore raw

PATCH 6/28 xfs: Hold inode locks in xfs_rename

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Modify xfs_rename to hold all inode locks across a rename operation We will need this later when we add parent pointers

---
 fs/xfs/xfs_inode.c |   43 ++++++++++++++++++++++++++++++-------------
 1 file changed, 30 insertions(+), 13 deletions(-)
fs/xfs/xfs_inode.c +30 -13
index e292688ee608..131abf84ea87 100644
@@ -2541,6 +2541,21 @@ xfs_remove(
 	return error;
 }
 
+static inline void
+xfs_iunlock_rename(
+	struct xfs_inode	**i_tab,
+	int			num_inodes)
+{
+	int			i;
+
+	for (i = num_inodes - 1; i >= 0; i--) {
+		/* Skip duplicate inodes if src and target dps are the same */
+		if (!i_tab[i] || (i > 0 && i_tab[i] == i_tab[i - 1]))
+			continue;
+		xfs_iunlock(i_tab[i], XFS_ILOCK_EXCL);
+	}
+}
+
 /*
  * Enter all inodes for a rename transaction into a sorted array.
  */
@@ -2839,18 +2854,16 @@ xfs_rename(
 	xfs_lock_inodes(inodes, num_inodes, XFS_ILOCK_EXCL);
 
 	/*
-	 * Join all the inodes to the transaction. From this point on,
-	 * we can rely on either trans_commit or trans_cancel to unlock
-	 * them.
+	 * Join all the inodes to the transaction.
 	 */
-	xfs_trans_ijoin(tp, src_dp, XFS_ILOCK_EXCL);
+	xfs_trans_ijoin(tp, src_dp, 0);
 	if (new_parent)
-		xfs_trans_ijoin(tp, target_dp, XFS_ILOCK_EXCL);
-	xfs_trans_ijoin(tp, src_ip, XFS_ILOCK_EXCL);
+		xfs_trans_ijoin(tp, target_dp, 0);
+	xfs_trans_ijoin(tp, src_ip, 0);
 	if (target_ip)
-		xfs_trans_ijoin(tp, target_ip, XFS_ILOCK_EXCL);
+		xfs_trans_ijoin(tp, target_ip, 0);
 	if (wip)
-		xfs_trans_ijoin(tp, wip, XFS_ILOCK_EXCL);
+		xfs_trans_ijoin(tp, wip, 0);
 
 	/*
 	 * If we are using project inheritance, we only allow renames
@@ -2864,10 +2877,12 @@ xfs_rename(
 	}
 
 	/* RENAME_EXCHANGE is unique from here on. */
-	if (flags & RENAME_EXCHANGE)
-		return xfs_cross_rename(tp, src_dp, src_name, src_ip,
+	if (flags & RENAME_EXCHANGE) {
+		error = xfs_cross_rename(tp, src_dp, src_name, src_ip,
 					target_dp, target_name, target_ip,
 					spaceres);
+		goto out_unlock;
+	}
 
 	/*
 	 * Try to reserve quota to handle an expansion of the target directory.
@@ -2881,6 +2896,7 @@ xfs_rename(
 		if (error == -EDQUOT || error == -ENOSPC) {
 			if (!retried) {
 				xfs_trans_cancel(tp);
+				xfs_iunlock_rename(inodes, num_inodes);
 				xfs_blockgc_free_quota(target_dp, 0);
 				retried = true;
 				goto retry;
@@ -3092,12 +3108,13 @@ xfs_rename(
 		xfs_trans_log_inode(tp, target_dp, XFS_ILOG_CORE);
 
 	error = xfs_finish_rename(tp);
-	if (wip)
-		xfs_irele(wip);
-	return error;
+
+	goto out_unlock;
 
 out_trans_cancel:
 	xfs_trans_cancel(tp);
+out_unlock:
+	xfs_iunlock_rename(inodes, num_inodes);
 out_release_wip:
 	if (wip)
 		xfs_irele(wip);
#8 lore raw

PATCH 7/28 xfs: Expose init_xattrs in xfs_create_tmpfile

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Tmp files are used as part of rename operations and will need attr forks initialized for parent pointers. Expose the init_xattrs parameter to the calling function to initialize the fork.

---
 fs/xfs/xfs_inode.c |    5 +++--
 fs/xfs/xfs_inode.h |    2 +-
 fs/xfs/xfs_iops.c  |    3 ++-
 3 files changed, 6 insertions(+), 4 deletions(-)
fs/xfs/xfs_inode.c +3 -2
index 131abf84ea87..267d629a33d9 100644
@@ -1109,6 +1109,7 @@ xfs_create_tmpfile(
 	struct user_namespace	*mnt_userns,
 	struct xfs_inode	*dp,
 	umode_t			mode,
+	bool			init_xattrs,
 	struct xfs_inode	**ipp)
 {
 	struct xfs_mount	*mp = dp->i_mount;
@@ -1149,7 +1150,7 @@ xfs_create_tmpfile(
 	error = xfs_dialloc(&tp, dp->i_ino, mode, &ino);
 	if (!error)
 		error = xfs_init_new_inode(mnt_userns, tp, dp, ino, mode,
-				0, 0, prid, false, &ip);
+				0, 0, prid, init_xattrs, &ip);
 	if (error)
 		goto out_trans_cancel;
 
@@ -2750,7 +2751,7 @@ xfs_rename_alloc_whiteout(
 	int			error;
 
 	error = xfs_create_tmpfile(mnt_userns, dp, S_IFCHR | WHITEOUT_MODE,
-				   &tmpfile);
+				   false, &tmpfile);
 	if (error)
 		return error;
 
fs/xfs/xfs_inode.h +1 -1
index 2eaed98af814..5735de32beeb 100644
@@ -478,7 +478,7 @@ int		xfs_create(struct user_namespace *mnt_userns,
 			   umode_t mode, dev_t rdev, bool need_xattr,
 			   struct xfs_inode **ipp);
 int		xfs_create_tmpfile(struct user_namespace *mnt_userns,
-			   struct xfs_inode *dp, umode_t mode,
+			   struct xfs_inode *dp, umode_t mode, bool init_xattrs,
 			   struct xfs_inode **ipp);
 int		xfs_remove(struct xfs_inode *dp, struct xfs_name *name,
 			   struct xfs_inode *ip);
fs/xfs/xfs_iops.c +2 -1
index 515318dfbc38..45e66c961829 100644
@@ -200,7 +200,8 @@ xfs_generic_create(
 				xfs_create_need_xattr(dir, default_acl, acl),
 				&ip);
 	} else {
-		error = xfs_create_tmpfile(mnt_userns, XFS_I(dir), mode, &ip);
+		error = xfs_create_tmpfile(mnt_userns, XFS_I(dir), mode, true,
+					   &ip);
 	}
 	if (unlikely(error))
 		goto out_free_acl;
#9 lore raw

PATCH 8/28 xfs: get directory offset when adding directory name

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Return the directory offset information when adding an entry to the directory.

This offset will be used as the parent pointer offset in xfs_create, xfs_symlink, xfs_link and xfs_rename.

---
 fs/xfs/libxfs/xfs_da_btree.h   |    1 +
 fs/xfs/libxfs/xfs_dir2.c       |    9 +++++++--
 fs/xfs/libxfs/xfs_dir2.h       |    2 +-
 fs/xfs/libxfs/xfs_dir2_block.c |    1 +
 fs/xfs/libxfs/xfs_dir2_leaf.c  |    2 ++
 fs/xfs/libxfs/xfs_dir2_node.c  |    2 ++
 fs/xfs/libxfs/xfs_dir2_sf.c    |    2 ++
 fs/xfs/xfs_inode.c             |    6 +++---
 fs/xfs/xfs_symlink.c           |    3 ++-
 9 files changed, 21 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_da_btree.h +1 -0
index a4b29827603f..90b86d00258f 100644
@@ -81,6 +81,7 @@ typedef struct xfs_da_args {
 	int		rmtvaluelen2;	/* remote attr value length in bytes */
 	uint32_t	op_flags;	/* operation flags */
 	enum xfs_dacmp	cmpresult;	/* name compare result for lookups */
+	xfs_dir2_dataptr_t offset;	/* OUT: offset in directory */
 } xfs_da_args_t;
 
 /*
fs/xfs/libxfs/xfs_dir2.c +7 -2
index 92bac3373f1f..69a6561c22cc 100644
@@ -257,7 +257,8 @@ xfs_dir_createname(
 	struct xfs_inode	*dp,
 	const struct xfs_name	*name,
 	xfs_ino_t		inum,		/* new entry inode number */
-	xfs_extlen_t		total)		/* bmap's total block count */
+	xfs_extlen_t		total,		/* bmap's total block count */
+	xfs_dir2_dataptr_t	*offset)	/* OUT entry's dir offset */
 {
 	struct xfs_da_args	*args;
 	int			rval;
@@ -312,6 +313,10 @@ xfs_dir_createname(
 		rval = xfs_dir2_node_addname(args);
 
 out_free:
+	/* return the location that this entry was place in the parent inode */
+	if (offset)
+		*offset = args->offset;
+
 	kmem_free(args);
 	return rval;
 }
@@ -550,7 +555,7 @@ xfs_dir_canenter(
 	xfs_inode_t	*dp,
 	struct xfs_name	*name)		/* name of entry to add */
 {
-	return xfs_dir_createname(tp, dp, name, 0, 0);
+	return xfs_dir_createname(tp, dp, name, 0, 0, NULL);
 }
 
 /*
fs/xfs/libxfs/xfs_dir2.h +1 -1
index dd39f17dd9a9..d96954478696 100644
@@ -40,7 +40,7 @@ extern int xfs_dir_init(struct xfs_trans *tp, struct xfs_inode *dp,
 				struct xfs_inode *pdp);
 extern int xfs_dir_createname(struct xfs_trans *tp, struct xfs_inode *dp,
 				const struct xfs_name *name, xfs_ino_t inum,
-				xfs_extlen_t tot);
+				xfs_extlen_t tot, xfs_dir2_dataptr_t *offset);
 extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp,
 				const struct xfs_name *name, xfs_ino_t *inum,
 				struct xfs_name *ci_name);
fs/xfs/libxfs/xfs_dir2_block.c +1 -0
index 00f960a703b2..70aeab9d2a12 100644
@@ -573,6 +573,7 @@ xfs_dir2_block_addname(
 	xfs_dir2_data_put_ftype(dp->i_mount, dep, args->filetype);
 	tagp = xfs_dir2_data_entry_tag_p(dp->i_mount, dep);
 	*tagp = cpu_to_be16((char *)dep - (char *)hdr);
+	args->offset = xfs_dir2_byte_to_dataptr((char *)dep - (char *)hdr);
 	/*
 	 * Clean up the bestfree array and log the header, tail, and entry.
 	 */
fs/xfs/libxfs/xfs_dir2_leaf.c +2 -0
index cb9e950a911d..9ab520b66547 100644
@@ -870,6 +870,8 @@ xfs_dir2_leaf_addname(
 	xfs_dir2_data_put_ftype(dp->i_mount, dep, args->filetype);
 	tagp = xfs_dir2_data_entry_tag_p(dp->i_mount, dep);
 	*tagp = cpu_to_be16((char *)dep - (char *)hdr);
+	args->offset = xfs_dir2_db_off_to_dataptr(args->geo, use_block,
+						(char *)dep - (char *)hdr);
 	/*
 	 * Need to scan fix up the bestfree table.
 	 */
fs/xfs/libxfs/xfs_dir2_node.c +2 -0
index 7a03aeb9f4c9..5a9513c036b8 100644
@@ -1974,6 +1974,8 @@ xfs_dir2_node_addname_int(
 	xfs_dir2_data_put_ftype(dp->i_mount, dep, args->filetype);
 	tagp = xfs_dir2_data_entry_tag_p(dp->i_mount, dep);
 	*tagp = cpu_to_be16((char *)dep - (char *)hdr);
+	args->offset = xfs_dir2_db_off_to_dataptr(args->geo, dbno,
+						  (char *)dep - (char *)hdr);
 	xfs_dir2_data_log_entry(args, dbp, dep);
 
 	/* Rescan the freespace and log the data block if needed. */
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
index 8cd37e6e9d38..44bc4ba3da8a 100644
@@ -485,6 +485,7 @@ xfs_dir2_sf_addname_easy(
 	memcpy(sfep->name, args->name, sfep->namelen);
 	xfs_dir2_sf_put_ino(mp, sfp, sfep, args->inumber);
 	xfs_dir2_sf_put_ftype(mp, sfep, args->filetype);
+	args->offset = xfs_dir2_byte_to_dataptr(offset);
 
 	/*
 	 * Update the header and inode.
@@ -575,6 +576,7 @@ xfs_dir2_sf_addname_hard(
 	memcpy(sfep->name, args->name, sfep->namelen);
 	xfs_dir2_sf_put_ino(mp, sfp, sfep, args->inumber);
 	xfs_dir2_sf_put_ftype(mp, sfep, args->filetype);
+	args->offset = xfs_dir2_byte_to_dataptr(offset);
 	sfp->count++;
 	if (args->inumber > XFS_DIR2_MAX_SHORT_INUM && !objchange)
 		sfp->i8count++;
fs/xfs/xfs_inode.c +3 -3
index 267d629a33d9..143de4202cf4 100644
@@ -1038,7 +1038,7 @@ xfs_create(
 	unlock_dp_on_error = false;
 
 	error = xfs_dir_createname(tp, dp, name, ip->i_ino,
-					resblks - XFS_IALLOC_SPACE_RES(mp));
+				   resblks - XFS_IALLOC_SPACE_RES(mp), NULL);
 	if (error) {
 		ASSERT(error != -ENOSPC);
 		goto out_trans_cancel;
@@ -1264,7 +1264,7 @@ xfs_link(
 	}
 
 	error = xfs_dir_createname(tp, tdp, target_name, sip->i_ino,
-				   resblks);
+				   resblks, NULL);
 	if (error)
 		goto error_return;
 	xfs_trans_ichgtime(tp, tdp, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
@@ -3001,7 +3001,7 @@ xfs_rename(
 		 * to account for the ".." reference from the new entry.
 		 */
 		error = xfs_dir_createname(tp, target_dp, target_name,
-					   src_ip->i_ino, spaceres);
+					   src_ip->i_ino, spaceres, NULL);
 		if (error)
 			goto out_trans_cancel;
 
fs/xfs/xfs_symlink.c +2 -1
index d8e120913036..27a7d7c57015 100644
@@ -314,7 +314,8 @@ xfs_symlink(
 	/*
 	 * Create the directory entry for the symlink.
 	 */
-	error = xfs_dir_createname(tp, dp, link_name, ip->i_ino, resblks);
+	error = xfs_dir_createname(tp, dp, link_name,
+			ip->i_ino, resblks, NULL);
 	if (error)
 		goto out_trans_cancel;
 	xfs_trans_ichgtime(tp, dp, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
#10 lore raw

PATCH 9/28 xfs: get directory offset when removing directory name

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Return the directory offset information when removing an entry to the directory.

This offset will be used as the parent pointer offset in xfs_remove.

---
 fs/xfs/libxfs/xfs_dir2.c       |    6 +++++-
 fs/xfs/libxfs/xfs_dir2.h       |    3 ++-
 fs/xfs/libxfs/xfs_dir2_block.c |    4 ++--
 fs/xfs/libxfs/xfs_dir2_leaf.c  |    5 +++--
 fs/xfs/libxfs/xfs_dir2_node.c  |    5 +++--
 fs/xfs/libxfs/xfs_dir2_sf.c    |    2 ++
 fs/xfs/xfs_inode.c             |    4 ++--
 7 files changed, 19 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +5 -1
index 69a6561c22cc..891c1f701f53 100644
@@ -436,7 +436,8 @@ xfs_dir_removename(
 	struct xfs_inode	*dp,
 	struct xfs_name		*name,
 	xfs_ino_t		ino,
-	xfs_extlen_t		total)		/* bmap's total block count */
+	xfs_extlen_t		total,		/* bmap's total block count */
+	xfs_dir2_dataptr_t	*offset)	/* OUT: offset in directory */
 {
 	struct xfs_da_args	*args;
 	int			rval;
@@ -481,6 +482,9 @@ xfs_dir_removename(
 	else
 		rval = xfs_dir2_node_removename(args);
 out_free:
+	if (offset)
+		*offset = args->offset;
+
 	kmem_free(args);
 	return rval;
 }
fs/xfs/libxfs/xfs_dir2.h +2 -1
index d96954478696..0c2d7c0af78f 100644
@@ -46,7 +46,8 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp,
 				struct xfs_name *ci_name);
 extern int xfs_dir_removename(struct xfs_trans *tp, struct xfs_inode *dp,
 				struct xfs_name *name, xfs_ino_t ino,
-				xfs_extlen_t tot);
+				xfs_extlen_t tot,
+				xfs_dir2_dataptr_t *offset);
 extern int xfs_dir_replace(struct xfs_trans *tp, struct xfs_inode *dp,
 				const struct xfs_name *name, xfs_ino_t inum,
 				xfs_extlen_t tot);
fs/xfs/libxfs/xfs_dir2_block.c +2 -2
index 70aeab9d2a12..d36f3f1491da 100644
@@ -810,9 +810,9 @@ xfs_dir2_block_removename(
 	/*
 	 * Point to the data entry using the leaf entry.
 	 */
+	args->offset = be32_to_cpu(blp[ent].address);
 	dep = (xfs_dir2_data_entry_t *)((char *)hdr +
-			xfs_dir2_dataptr_to_off(args->geo,
-						be32_to_cpu(blp[ent].address)));
+			xfs_dir2_dataptr_to_off(args->geo, args->offset));
 	/*
 	 * Mark the data entry's space free.
 	 */
fs/xfs/libxfs/xfs_dir2_leaf.c +3 -2
index 9ab520b66547..b4a066259d97 100644
@@ -1386,9 +1386,10 @@ xfs_dir2_leaf_removename(
 	 * Point to the leaf entry, use that to point to the data entry.
 	 */
 	lep = &leafhdr.ents[index];
-	db = xfs_dir2_dataptr_to_db(geo, be32_to_cpu(lep->address));
+	args->offset = be32_to_cpu(lep->address);
+	db = xfs_dir2_dataptr_to_db(args->geo, args->offset);
 	dep = (xfs_dir2_data_entry_t *)((char *)hdr +
-		xfs_dir2_dataptr_to_off(geo, be32_to_cpu(lep->address)));
+		xfs_dir2_dataptr_to_off(args->geo, args->offset));
 	needscan = needlog = 0;
 	oldbest = be16_to_cpu(bf[0].length);
 	ltp = xfs_dir2_leaf_tail_p(geo, leaf);
fs/xfs/libxfs/xfs_dir2_node.c +3 -2
index 5a9513c036b8..39cbdeafa0f6 100644
@@ -1296,9 +1296,10 @@ xfs_dir2_leafn_remove(
 	/*
 	 * Extract the data block and offset from the entry.
 	 */
-	db = xfs_dir2_dataptr_to_db(geo, be32_to_cpu(lep->address));
+	args->offset = be32_to_cpu(lep->address);
+	db = xfs_dir2_dataptr_to_db(args->geo, args->offset);
 	ASSERT(dblk->blkno == db);
-	off = xfs_dir2_dataptr_to_off(geo, be32_to_cpu(lep->address));
+	off = xfs_dir2_dataptr_to_off(args->geo, args->offset);
 	ASSERT(dblk->index == off);
 
 	/*
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
index 44bc4ba3da8a..b49578a547b3 100644
@@ -969,6 +969,8 @@ xfs_dir2_sf_removename(
 								XFS_CMP_EXACT) {
 			ASSERT(xfs_dir2_sf_get_ino(mp, sfp, sfep) ==
 			       args->inumber);
+			args->offset = xfs_dir2_byte_to_dataptr(
+						xfs_dir2_sf_get_offset(sfep));
 			break;
 		}
 	}
fs/xfs/xfs_inode.c +2 -2
index 143de4202cf4..e5ed8bdef9fe 100644
@@ -2508,7 +2508,7 @@ xfs_remove(
 	if (error)
 		goto out_trans_cancel;
 
-	error = xfs_dir_removename(tp, dp, name, ip->i_ino, resblks);
+	error = xfs_dir_removename(tp, dp, name, ip->i_ino, resblks, NULL);
 	if (error) {
 		ASSERT(error != -ENOENT);
 		goto out_trans_cancel;
@@ -3098,7 +3098,7 @@ xfs_rename(
 					spaceres);
 	else
 		error = xfs_dir_removename(tp, src_dp, src_name, src_ip->i_ino,
-					   spaceres);
+					   spaceres, NULL);
 
 	if (error)
 		goto out_trans_cancel;
#11 lore raw

PATCH 10/28 xfs: get directory offset when replacing a directory name

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Return the directory offset information when replacing an entry to the directory.

This offset will be used as the parent pointer offset in xfs_rename.

---
 fs/xfs/libxfs/xfs_dir2.c       |    8 ++++++--
 fs/xfs/libxfs/xfs_dir2.h       |    2 +-
 fs/xfs/libxfs/xfs_dir2_block.c |    4 ++--
 fs/xfs/libxfs/xfs_dir2_leaf.c  |    1 +
 fs/xfs/libxfs/xfs_dir2_node.c  |    1 +
 fs/xfs/libxfs/xfs_dir2_sf.c    |    2 ++
 fs/xfs/xfs_inode.c             |   16 ++++++++--------
 7 files changed, 21 insertions(+), 13 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +6 -2
index 891c1f701f53..c1a9394d7478 100644
@@ -482,7 +482,7 @@ xfs_dir_removename(
 	else
 		rval = xfs_dir2_node_removename(args);
 out_free:
-	if (offset)
+	if (!rval && offset)
 		*offset = args->offset;
 
 	kmem_free(args);
@@ -498,7 +498,8 @@ xfs_dir_replace(
 	struct xfs_inode	*dp,
 	const struct xfs_name	*name,		/* name of entry to replace */
 	xfs_ino_t		inum,		/* new inode number */
-	xfs_extlen_t		total)		/* bmap's total block count */
+	xfs_extlen_t		total,		/* bmap's total block count */
+	xfs_dir2_dataptr_t	*offset)	/* OUT: offset in directory */
 {
 	struct xfs_da_args	*args;
 	int			rval;
@@ -546,6 +547,9 @@ xfs_dir_replace(
 	else
 		rval = xfs_dir2_node_replace(args);
 out_free:
+	if (offset)
+		*offset = args->offset;
+
 	kmem_free(args);
 	return rval;
 }
fs/xfs/libxfs/xfs_dir2.h +1 -1
index 0c2d7c0af78f..ff59f009d1fd 100644
@@ -50,7 +50,7 @@ extern int xfs_dir_removename(struct xfs_trans *tp, struct xfs_inode *dp,
 				xfs_dir2_dataptr_t *offset);
 extern int xfs_dir_replace(struct xfs_trans *tp, struct xfs_inode *dp,
 				const struct xfs_name *name, xfs_ino_t inum,
-				xfs_extlen_t tot);
+				xfs_extlen_t tot, xfs_dir2_dataptr_t *offset);
 extern int xfs_dir_canenter(struct xfs_trans *tp, struct xfs_inode *dp,
 				struct xfs_name *name);
 
fs/xfs/libxfs/xfs_dir2_block.c +2 -2
index d36f3f1491da..0f3a03e87278 100644
@@ -885,9 +885,9 @@ xfs_dir2_block_replace(
 	/*
 	 * Point to the data entry we need to change.
 	 */
+	args->offset = be32_to_cpu(blp[ent].address);
 	dep = (xfs_dir2_data_entry_t *)((char *)hdr +
-			xfs_dir2_dataptr_to_off(args->geo,
-						be32_to_cpu(blp[ent].address)));
+			xfs_dir2_dataptr_to_off(args->geo, args->offset));
 	ASSERT(be64_to_cpu(dep->inumber) != args->inumber);
 	/*
 	 * Change the inode number to the new value.
fs/xfs/libxfs/xfs_dir2_leaf.c +1 -0
index b4a066259d97..fe75ffadace9 100644
@@ -1523,6 +1523,7 @@ xfs_dir2_leaf_replace(
 	/*
 	 * Point to the data entry.
 	 */
+	args->offset = be32_to_cpu(lep->address);
 	dep = (xfs_dir2_data_entry_t *)
 	      ((char *)dbp->b_addr +
 	       xfs_dir2_dataptr_to_off(args->geo, be32_to_cpu(lep->address)));
fs/xfs/libxfs/xfs_dir2_node.c +1 -0
index 39cbdeafa0f6..53cd0d5d94f7 100644
@@ -2242,6 +2242,7 @@ xfs_dir2_node_replace(
 		hdr = state->extrablk.bp->b_addr;
 		ASSERT(hdr->magic == cpu_to_be32(XFS_DIR2_DATA_MAGIC) ||
 		       hdr->magic == cpu_to_be32(XFS_DIR3_DATA_MAGIC));
+		args->offset = be32_to_cpu(leafhdr.ents[blk->index].address);
 		dep = (xfs_dir2_data_entry_t *)
 		      ((char *)hdr +
 		       xfs_dir2_dataptr_to_off(args->geo,
fs/xfs/libxfs/xfs_dir2_sf.c +2 -0
index b49578a547b3..032c65804610 100644
@@ -1107,6 +1107,8 @@ xfs_dir2_sf_replace(
 				xfs_dir2_sf_put_ino(mp, sfp, sfep,
 						args->inumber);
 				xfs_dir2_sf_put_ftype(mp, sfep, args->filetype);
+				args->offset = xfs_dir2_byte_to_dataptr(
+						  xfs_dir2_sf_get_offset(sfep));
 				break;
 			}
 		}
fs/xfs/xfs_inode.c +8 -8
index e5ed8bdef9fe..a896ee4c9680 100644
@@ -2489,7 +2489,7 @@ xfs_remove(
 		 */
 		if (dp->i_ino != tp->t_mountp->m_sb.sb_rootino) {
 			error = xfs_dir_replace(tp, ip, &xfs_name_dotdot,
-					tp->t_mountp->m_sb.sb_rootino, 0);
+					tp->t_mountp->m_sb.sb_rootino, 0, NULL);
 			if (error)
 				goto out_trans_cancel;
 		}
@@ -2644,12 +2644,12 @@ xfs_cross_rename(
 	int		dp2_flags = 0;
 
 	/* Swap inode number for dirent in first parent */
-	error = xfs_dir_replace(tp, dp1, name1, ip2->i_ino, spaceres);
+	error = xfs_dir_replace(tp, dp1, name1, ip2->i_ino, spaceres, NULL);
 	if (error)
 		goto out_trans_abort;
 
 	/* Swap inode number for dirent in second parent */
-	error = xfs_dir_replace(tp, dp2, name2, ip1->i_ino, spaceres);
+	error = xfs_dir_replace(tp, dp2, name2, ip1->i_ino, spaceres, NULL);
 	if (error)
 		goto out_trans_abort;
 
@@ -2663,7 +2663,7 @@ xfs_cross_rename(
 
 		if (S_ISDIR(VFS_I(ip2)->i_mode)) {
 			error = xfs_dir_replace(tp, ip2, &xfs_name_dotdot,
-						dp1->i_ino, spaceres);
+						dp1->i_ino, spaceres, NULL);
 			if (error)
 				goto out_trans_abort;
 
@@ -2687,7 +2687,7 @@ xfs_cross_rename(
 
 		if (S_ISDIR(VFS_I(ip1)->i_mode)) {
 			error = xfs_dir_replace(tp, ip1, &xfs_name_dotdot,
-						dp2->i_ino, spaceres);
+						dp2->i_ino, spaceres, NULL);
 			if (error)
 				goto out_trans_abort;
 
@@ -3022,7 +3022,7 @@ xfs_rename(
 		 * name at the destination directory, remove it first.
 		 */
 		error = xfs_dir_replace(tp, target_dp, target_name,
-					src_ip->i_ino, spaceres);
+					src_ip->i_ino, spaceres, NULL);
 		if (error)
 			goto out_trans_cancel;
 
@@ -3056,7 +3056,7 @@ xfs_rename(
 		 * directory.
 		 */
 		error = xfs_dir_replace(tp, src_ip, &xfs_name_dotdot,
-					target_dp->i_ino, spaceres);
+					target_dp->i_ino, spaceres, NULL);
 		ASSERT(error != -EEXIST);
 		if (error)
 			goto out_trans_cancel;
@@ -3095,7 +3095,7 @@ xfs_rename(
 	 */
 	if (wip)
 		error = xfs_dir_replace(tp, src_dp, src_name, wip->i_ino,
-					spaceres);
+					spaceres, NULL);
 	else
 		error = xfs_dir_removename(tp, src_dp, src_name, src_ip->i_ino,
 					   spaceres, NULL);
#12 lore raw

PATCH 11/28 xfs: add parent pointer support to attribute code

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer entries; it uses reserved blocks like XFS_ATTR_ROOT.

---
 fs/xfs/libxfs/xfs_attr.c       |    4 +++-
 fs/xfs/libxfs/xfs_da_format.h  |    5 ++++-
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/scrub/attr.c            |    2 +-
 4 files changed, 9 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.c +3 -1
index b1dbed7655e8..101823772bf9 100644
@@ -976,11 +976,13 @@ xfs_attr_set(
 	struct xfs_inode	*dp = args->dp;
 	struct xfs_mount	*mp = dp->i_mount;
 	struct xfs_trans_res	tres;
-	bool			rsvd = (args->attr_filter & XFS_ATTR_ROOT);
+	bool			rsvd;
 	int			error, local;
 	int			rmt_blks = 0;
 	unsigned int		total;
 
+	rsvd = (args->attr_filter & (XFS_ATTR_ROOT | XFS_ATTR_PARENT)) != 0;
+
 	if (xfs_is_shutdown(dp->i_mount))
 		return -EIO;
 
fs/xfs/libxfs/xfs_da_format.h +4 -1
index 25e2841084e1..3dc03968bba6 100644
@@ -688,12 +688,15 @@ struct xfs_attr3_leafblock {
 #define	XFS_ATTR_LOCAL_BIT	0	/* attr is stored locally */
 #define	XFS_ATTR_ROOT_BIT	1	/* limit access to trusted attrs */
 #define	XFS_ATTR_SECURE_BIT	2	/* limit access to secure attrs */
+#define	XFS_ATTR_PARENT_BIT	3	/* parent pointer attrs */
 #define	XFS_ATTR_INCOMPLETE_BIT	7	/* attr in middle of create/delete */
 #define XFS_ATTR_LOCAL		(1u << XFS_ATTR_LOCAL_BIT)
 #define XFS_ATTR_ROOT		(1u << XFS_ATTR_ROOT_BIT)
 #define XFS_ATTR_SECURE		(1u << XFS_ATTR_SECURE_BIT)
+#define XFS_ATTR_PARENT		(1u << XFS_ATTR_PARENT_BIT)
 #define XFS_ATTR_INCOMPLETE	(1u << XFS_ATTR_INCOMPLETE_BIT)
-#define XFS_ATTR_NSP_ONDISK_MASK	(XFS_ATTR_ROOT | XFS_ATTR_SECURE)
+#define XFS_ATTR_NSP_ONDISK_MASK \
+			(XFS_ATTR_ROOT | XFS_ATTR_SECURE | XFS_ATTR_PARENT)
 
 /*
  * Alignment for namelist and valuelist entries (since they are mixed
fs/xfs/libxfs/xfs_log_format.h +1 -0
index ae9c99762a24..727b5a858028 100644
@@ -967,6 +967,7 @@ struct xfs_icreate_log {
  */
 #define XFS_ATTRI_FILTER_MASK		(XFS_ATTR_ROOT | \
 					 XFS_ATTR_SECURE | \
+					 XFS_ATTR_PARENT | \
 					 XFS_ATTR_INCOMPLETE)
 
 /*
fs/xfs/scrub/attr.c +1 -1
index 31529b9bf389..9d2e33743ecd 100644
@@ -441,7 +441,7 @@ xchk_xattr_rec(
 	/* Retrieve the entry and check it. */
 	hash = be32_to_cpu(ent->hashval);
 	badflags = ~(XFS_ATTR_LOCAL | XFS_ATTR_ROOT | XFS_ATTR_SECURE |
-			XFS_ATTR_INCOMPLETE);
+			XFS_ATTR_INCOMPLETE | XFS_ATTR_PARENT);
 	if ((ent->flags & badflags) != 0)
 		xchk_da_set_corrupt(ds, level);
 	if (ent->flags & XFS_ATTR_LOCAL) {
#13 lore raw

PATCH 12/28 xfs: define parent pointer xattr format

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

We need to define the parent pointer attribute format before we start adding support for it into all the code that needs to use it. The EA format we will use encodes the following information:

        name={parent inode #, parent inode generation, dirent offset}
        value={dirent filename}

The inode/gen gives all the information we need to reliably identify the parent without requiring child->parent lock ordering, and allows userspace to do pathname component level reconstruction without the kernel ever needing to verify the parent itself as part of ioctl calls.

By using the dirent offset in the EA name, we have a method of knowing the exact parent pointer EA we need to modify/remove in rename/unlink without an unbound EA name search.

By keeping the dirent name in the value, we have enough information to be able to validate and reconstruct damaged directory trees. While the diroffset of a filename alone is not unique enough to identify the child, the {diroffset,filename,child_inode} tuple is sufficient. That is, if the diroffset gets reused and points to a different filename, we can detect that from the contents of EA. If a link of the same name is created, then we can check whether it points at the same inode as the parent EA we current have.

---
 fs/xfs/libxfs/xfs_da_format.h |   25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
fs/xfs/libxfs/xfs_da_format.h +25 -0
index 3dc03968bba6..b02b67f1999e 100644
@@ -805,4 +805,29 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)
 xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 				      struct xfs_da3_blkinfo *hdr3);
 
+/*
+ * Parent pointer attribute format definition
+ *
+ * EA name encodes the parent inode number, generation and the offset of
+ * the dirent that points to the child inode. The EA value contains the
+ * same name as the dirent in the parent directory.
+ */
+struct xfs_parent_name_rec {
+	__be64  p_ino;
+	__be32  p_gen;
+	__be32  p_diroffset;
+};
+
+/*
+ * incore version of the above, also contains name pointers so callers
+ * can pass/obtain all the parent pointer information in a single structure
+ */
+struct xfs_parent_name_irec {
+	xfs_ino_t		p_ino;
+	uint32_t		p_gen;
+	xfs_dir2_dataptr_t	p_diroffset;
+	const char		*p_name;
+	uint8_t			p_namelen;
+};
+
 #endif /* __XFS_DA_FORMAT_H__ */
#14 lore raw

PATCH 13/28 xfs: Add xfs_verify_pptr

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Attribute names of parent pointers are not strings. So we need to modify attr_namecheck to verify parent pointer records when the XFS_ATTR_PARENT flag is set.

---
 fs/xfs/libxfs/xfs_attr.c      |   47 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/libxfs/xfs_attr.h      |    3 ++-
 fs/xfs/libxfs/xfs_da_format.h |    8 +++++++
 fs/xfs/scrub/attr.c           |    2 +-
 fs/xfs/xfs_attr_item.c        |   11 ++++++----
 fs/xfs/xfs_attr_list.c        |   17 ++++++++++-----
 6 files changed, 74 insertions(+), 14 deletions(-)
fs/xfs/libxfs/xfs_attr.c +44 -3
index 101823772bf9..711022742e34 100644
@@ -1577,9 +1577,33 @@ xfs_attr_node_get(
 	return error;
 }
 
-/* Returns true if the attribute entry name is valid. */
-bool
-xfs_attr_namecheck(
+/*
+ * Verify parent pointer attribute is valid.
+ * Return true on success or false on failure
+ */
+STATIC bool
+xfs_verify_pptr(
+	struct xfs_mount			*mp,
+	const struct xfs_parent_name_rec	*rec)
+{
+	xfs_ino_t				p_ino;
+	xfs_dir2_dataptr_t			p_diroffset;
+
+	p_ino = be64_to_cpu(rec->p_ino);
+	p_diroffset = be32_to_cpu(rec->p_diroffset);
+
+	if (!xfs_verify_ino(mp, p_ino))
+		return false;
+
+	if (p_diroffset > XFS_DIR2_MAX_DATAPTR)
+		return false;
+
+	return true;
+}
+
+/* Returns true if the string attribute entry name is valid. */
+static bool
+xfs_str_attr_namecheck(
 	const void	*name,
 	size_t		length)
 {
@@ -1594,6 +1618,23 @@ xfs_attr_namecheck(
 	return !memchr(name, 0, length);
 }
 
+/* Returns true if the attribute entry name is valid. */
+bool
+xfs_attr_namecheck(
+	struct xfs_mount	*mp,
+	const void		*name,
+	size_t			length,
+	int			flags)
+{
+	if (flags & XFS_ATTR_PARENT) {
+		if (length != sizeof(struct xfs_parent_name_rec))
+			return false;
+		return xfs_verify_pptr(mp, (struct xfs_parent_name_rec *)name);
+	}
+
+	return xfs_str_attr_namecheck(name, length);
+}
+
 int __init
 xfs_attr_intent_init_cache(void)
 {
fs/xfs/libxfs/xfs_attr.h +2 -1
index 3e81f3f48560..b79dae788cfb 100644
@@ -547,7 +547,8 @@ int xfs_attr_get(struct xfs_da_args *args);
 int xfs_attr_set(struct xfs_da_args *args);
 int xfs_attr_set_iter(struct xfs_attr_intent *attr);
 int xfs_attr_remove_iter(struct xfs_attr_intent *attr);
-bool xfs_attr_namecheck(const void *name, size_t length);
+bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,
+			int flags);
 int xfs_attr_calc_size(struct xfs_da_args *args, int *local);
 void xfs_init_attr_trans(struct xfs_da_args *args, struct xfs_trans_res *tres,
 			 unsigned int *total);
fs/xfs/libxfs/xfs_da_format.h +8 -0
index b02b67f1999e..75b13807145d 100644
@@ -731,6 +731,14 @@ xfs_attr3_leaf_name(xfs_attr_leafblock_t *leafp, int idx)
 	return &((char *)leafp)[be16_to_cpu(entries[idx].nameidx)];
 }
 
+static inline int
+xfs_attr3_leaf_flags(xfs_attr_leafblock_t *leafp, int idx)
+{
+	struct xfs_attr_leaf_entry *entries = xfs_attr3_leaf_entryp(leafp);
+
+	return entries[idx].flags;
+}
+
 static inline xfs_attr_leaf_name_remote_t *
 xfs_attr3_leaf_name_remote(xfs_attr_leafblock_t *leafp, int idx)
 {
fs/xfs/scrub/attr.c +1 -1
index 9d2e33743ecd..2a79a13cb600 100644
@@ -129,7 +129,7 @@ xchk_xattr_listent(
 	}
 
 	/* Does this name make sense? */
-	if (!xfs_attr_namecheck(name, namelen)) {
+	if (!xfs_attr_namecheck(sx->sc->mp, name, namelen, flags)) {
 		xchk_fblock_set_corrupt(sx->sc, XFS_ATTR_FORK, args.blkno);
 		return;
 	}
fs/xfs/xfs_attr_item.c +7 -4
index 95e9ecbb4a67..da807f286a09 100644
@@ -593,7 +593,8 @@ xfs_attri_item_recover(
 	 */
 	attrp = &attrip->attri_format;
 	if (!xfs_attri_validate(mp, attrp) ||
-	    !xfs_attr_namecheck(nv->name.i_addr, nv->name.i_len))
+	    !xfs_attr_namecheck(mp, nv->name.i_addr, nv->name.i_len,
+				attrp->alfi_attr_filter))
 		return -EFSCORRUPTED;
 
 	error = xlog_recover_iget(mp,  attrp->alfi_ino, &ip);
@@ -804,7 +805,8 @@ xlog_recover_attri_commit_pass2(
 	}
 
 	attr_name = item->ri_buf[i].i_addr;
-	if (!xfs_attr_namecheck(attr_name, attri_formatp->alfi_name_len)) {
+	if (!xfs_attr_namecheck(mp, attr_name, attri_formatp->alfi_name_len,
+				attri_formatp->alfi_attr_filter)) {
 		XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 				item->ri_buf[i].i_addr, item->ri_buf[i].i_len);
 		return -EFSCORRUPTED;
@@ -822,8 +824,9 @@ xlog_recover_attri_commit_pass2(
 		}
 
 		attr_nname = item->ri_buf[i].i_addr;
-		if (!xfs_attr_namecheck(attr_nname,
-				attri_formatp->alfi_nname_len)) {
+		if (!xfs_attr_namecheck(mp, attr_nname,
+				attri_formatp->alfi_nname_len,
+				attri_formatp->alfi_attr_filter)) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					item->ri_buf[i].i_addr,
 					item->ri_buf[i].i_len);
fs/xfs/xfs_attr_list.c +12 -5
index 99bbbe1a0e44..a51f7f13a352 100644
@@ -58,9 +58,13 @@ xfs_attr_shortform_list(
 	struct xfs_attr_sf_sort		*sbuf, *sbp;
 	struct xfs_attr_shortform	*sf;
 	struct xfs_attr_sf_entry	*sfe;
+	struct xfs_mount		*mp;
 	int				sbsize, nsbuf, count, i;
 	int				error = 0;
 
+	ASSERT(context != NULL);
+	ASSERT(dp != NULL);
+	mp = dp->i_mount;
 	sf = (struct xfs_attr_shortform *)dp->i_af.if_u1.if_data;
 	ASSERT(sf != NULL);
 	if (!sf->hdr.count)
@@ -82,8 +86,9 @@ xfs_attr_shortform_list(
 	     (dp->i_af.if_bytes + sf->hdr.count * 16) < context->bufsize)) {
 		for (i = 0, sfe = &sf->list[0]; i < sf->hdr.count; i++) {
 			if (XFS_IS_CORRUPT(context->dp->i_mount,
-					   !xfs_attr_namecheck(sfe->nameval,
-							       sfe->namelen)))
+					   !xfs_attr_namecheck(mp, sfe->nameval,
+							       sfe->namelen,
+							       sfe->flags)))
 				return -EFSCORRUPTED;
 			context->put_listent(context,
 					     sfe->flags,
@@ -174,8 +179,9 @@ xfs_attr_shortform_list(
 			cursor->offset = 0;
 		}
 		if (XFS_IS_CORRUPT(context->dp->i_mount,
-				   !xfs_attr_namecheck(sbp->name,
-						       sbp->namelen))) {
+				   !xfs_attr_namecheck(mp, sbp->name,
+						       sbp->namelen,
+						       sbp->flags))) {
 			error = -EFSCORRUPTED;
 			goto out;
 		}
@@ -465,7 +471,8 @@ xfs_attr3_leaf_list_int(
 		}
 
 		if (XFS_IS_CORRUPT(context->dp->i_mount,
-				   !xfs_attr_namecheck(name, namelen)))
+				   !xfs_attr_namecheck(mp, name, namelen,
+						       entry->flags)))
 			return -EFSCORRUPTED;
 		context->put_listent(context, entry->flags,
 					      name, namelen, valuelen);
#15 lore raw

PATCH 14/28 xfs: extend transaction reservations for parent attributes

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

We need to add, remove or modify parent pointer attributes during create/link/unlink/rename operations atomically with the dirents in the parent directories being modified. This means they need to be modified in the same transaction as the parent directories, and so we need to add the required space for the attribute modifications to the transaction reservations.

---
 fs/xfs/libxfs/xfs_trans_resv.c |  324 ++++++++++++++++++++++++++++++++++------
 1 file changed, 272 insertions(+), 52 deletions(-)
fs/xfs/libxfs/xfs_trans_resv.c +272 -52
index 5b2f27cbdb80..93419956b9e5 100644
@@ -19,6 +19,9 @@
 #include "xfs_trans.h"
 #include "xfs_qm.h"
 #include "xfs_trans_space.h"
+#include "xfs_attr_item.h"
+#include "xfs_log.h"
+#include "xfs_da_format.h"
 
 #define _ALLOC	true
 #define _FREE	false
@@ -420,29 +423,108 @@ xfs_calc_itruncate_reservation_minlogsize(
 	return xfs_calc_itruncate_reservation(mp, true);
 }
 
+static inline unsigned int xfs_calc_pptr_link_overhead(void)
+{
+	return sizeof(struct xfs_attri_log_format) +
+			xlog_calc_iovec_len(XATTR_NAME_MAX) +
+			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+}
+static inline unsigned int xfs_calc_pptr_unlink_overhead(void)
+{
+	return sizeof(struct xfs_attri_log_format) +
+			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+}
+static inline unsigned int xfs_calc_pptr_replace_overhead(void)
+{
+	return sizeof(struct xfs_attri_log_format) +
+			xlog_calc_iovec_len(XATTR_NAME_MAX) +
+			xlog_calc_iovec_len(XATTR_NAME_MAX) +
+			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+}
+
 /*
  * In renaming a files we can modify:
  *    the five inodes involved: 5 * inode size
  *    the two directory btrees: 2 * (max depth + v2) * dir block size
  *    the two directory bmap btrees: 2 * max depth * block size
  * And the bmap_finish transaction can free dir and bmap blocks (two sets
- *	of bmap blocks) giving:
+ *	of bmap blocks) giving (t2):
  *    the agf for the ags in which the blocks live: 3 * sector size
  *    the agfl for the ags in which the blocks live: 3 * sector size
  *    the superblock for the free block count: sector size
  *    the allocation btrees: 3 exts * 2 trees * (2 * max depth - 1) * block size
+ * If parent pointers are enabled (t3), then each transaction in the chain
+ *    must be capable of setting or removing the extended attribute
+ *    containing the parent information.  It must also be able to handle
+ *    the three xattr intent items that track the progress of the parent
+ *    pointer update.
  */
 STATIC uint
 xfs_calc_rename_reservation(
 	struct xfs_mount	*mp)
 {
-	return XFS_DQUOT_LOGRES(mp) +
-		max((xfs_calc_inode_res(mp, 5) +
-		     xfs_calc_buf_res(2 * XFS_DIROP_LOG_COUNT(mp),
-				      XFS_FSB_TO_B(mp, 1))),
-		    (xfs_calc_buf_res(7, mp->m_sb.sb_sectsize) +
-		     xfs_calc_buf_res(xfs_allocfree_block_count(mp, 3),
-				      XFS_FSB_TO_B(mp, 1))));
+	unsigned int		overhead = XFS_DQUOT_LOGRES(mp);
+	struct xfs_trans_resv	*resp = M_RES(mp);
+	unsigned int		t1, t2, t3 = 0;
+
+	t1 = xfs_calc_inode_res(mp, 5) +
+	     xfs_calc_buf_res(2 * XFS_DIROP_LOG_COUNT(mp),
+			XFS_FSB_TO_B(mp, 1));
+
+	t2 = xfs_calc_buf_res(7, mp->m_sb.sb_sectsize) +
+	     xfs_calc_buf_res(xfs_allocfree_block_count(mp, 3),
+			XFS_FSB_TO_B(mp, 1));
+
+	if (xfs_has_parent(mp)) {
+		unsigned int	rename_overhead, exchange_overhead;
+
+		t3 = max(resp->tr_attrsetm.tr_logres,
+			 resp->tr_attrrm.tr_logres);
+
+		/*
+		 * For a standard rename, the three xattr intent log items
+		 * are (1) replacing the pptr for the source file; (2)
+		 * removing the pptr on the dest file; and (3) adding a
+		 * pptr for the whiteout file in the src dir.
+		 *
+		 * For an RENAME_EXCHANGE, there are two xattr intent
+		 * items to replace the pptr for both src and dest
+		 * files.  Link counts don't change and there is no
+		 * whiteout.
+		 *
+		 * In the worst case we can end up relogging all log
+		 * intent items to allow the log tail to move ahead, so
+		 * they become overhead added to each transaction in a
+		 * processing chain.
+		 */
+		rename_overhead = xfs_calc_pptr_replace_overhead() +
+				  xfs_calc_pptr_unlink_overhead() +
+				  xfs_calc_pptr_link_overhead();
+		exchange_overhead = 2 * xfs_calc_pptr_replace_overhead();
+
+		overhead += max(rename_overhead, exchange_overhead);
+	}
+
+	return overhead + max3(t1, t2, t3);
+}
+
+static inline unsigned int
+xfs_rename_log_count(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	/* One for the rename, one more for freeing blocks */
+	unsigned int		ret = XFS_RENAME_LOG_COUNT;
+
+	/*
+	 * Pre-reserve enough log reservation to handle the transaction
+	 * rolling needed to remove or add one parent pointer.
+	 */
+	if (xfs_has_parent(mp))
+		ret += max(resp->tr_attrsetm.tr_logcount,
+			   resp->tr_attrrm.tr_logcount);
+
+	return ret;
 }
 
 /*
@@ -459,6 +541,23 @@ xfs_calc_iunlink_remove_reservation(
 	       2 * M_IGEO(mp)->inode_cluster_size;
 }
 
+static inline unsigned int
+xfs_link_log_count(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	unsigned int		ret = XFS_LINK_LOG_COUNT;
+
+	/*
+	 * Pre-reserve enough log reservation to handle the transaction
+	 * rolling needed to add one parent pointer.
+	 */
+	if (xfs_has_parent(mp))
+		ret += resp->tr_attrsetm.tr_logcount;
+
+	return ret;
+}
+
 /*
  * For creating a link to an inode:
  *    the parent directory inode: inode size
@@ -475,14 +574,23 @@ STATIC uint
 xfs_calc_link_reservation(
 	struct xfs_mount	*mp)
 {
-	return XFS_DQUOT_LOGRES(mp) +
-		xfs_calc_iunlink_remove_reservation(mp) +
-		max((xfs_calc_inode_res(mp, 2) +
-		     xfs_calc_buf_res(XFS_DIROP_LOG_COUNT(mp),
-				      XFS_FSB_TO_B(mp, 1))),
-		    (xfs_calc_buf_res(3, mp->m_sb.sb_sectsize) +
-		     xfs_calc_buf_res(xfs_allocfree_block_count(mp, 1),
-				      XFS_FSB_TO_B(mp, 1))));
+	unsigned int            overhead = XFS_DQUOT_LOGRES(mp);
+	struct xfs_trans_resv   *resp = M_RES(mp);
+	unsigned int            t1, t2, t3 = 0;
+
+	overhead += xfs_calc_iunlink_remove_reservation(mp);
+	t1 = xfs_calc_inode_res(mp, 2) +
+	       xfs_calc_buf_res(XFS_DIROP_LOG_COUNT(mp), XFS_FSB_TO_B(mp, 1));
+	t2 = xfs_calc_buf_res(3, mp->m_sb.sb_sectsize) +
+	     xfs_calc_buf_res(xfs_allocfree_block_count(mp, 1),
+			      XFS_FSB_TO_B(mp, 1));
+
+	if (xfs_has_parent(mp)) {
+		t3 = resp->tr_attrsetm.tr_logres;
+		overhead += xfs_calc_pptr_link_overhead();
+	}
+
+	return overhead + max3(t1, t2, t3);
 }
 
 /*
@@ -497,6 +605,23 @@ xfs_calc_iunlink_add_reservation(xfs_mount_t *mp)
 			M_IGEO(mp)->inode_cluster_size;
 }
 
+static inline unsigned int
+xfs_remove_log_count(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	unsigned int		ret = XFS_REMOVE_LOG_COUNT;
+
+	/*
+	 * Pre-reserve enough log reservation to handle the transaction
+	 * rolling needed to add one parent pointer.
+	 */
+	if (xfs_has_parent(mp))
+		ret += resp->tr_attrrm.tr_logcount;
+
+	return ret;
+}
+
 /*
  * For removing a directory entry we can modify:
  *    the parent directory inode: inode size
@@ -513,14 +638,24 @@ STATIC uint
 xfs_calc_remove_reservation(
 	struct xfs_mount	*mp)
 {
-	return XFS_DQUOT_LOGRES(mp) +
-		xfs_calc_iunlink_add_reservation(mp) +
-		max((xfs_calc_inode_res(mp, 2) +
-		     xfs_calc_buf_res(XFS_DIROP_LOG_COUNT(mp),
-				      XFS_FSB_TO_B(mp, 1))),
-		    (xfs_calc_buf_res(4, mp->m_sb.sb_sectsize) +
-		     xfs_calc_buf_res(xfs_allocfree_block_count(mp, 2),
-				      XFS_FSB_TO_B(mp, 1))));
+	unsigned int            overhead = XFS_DQUOT_LOGRES(mp);
+	struct xfs_trans_resv   *resp = M_RES(mp);
+	unsigned int            t1, t2, t3 = 0;
+
+	overhead += xfs_calc_iunlink_add_reservation(mp);
+
+	t1 = xfs_calc_inode_res(mp, 2) +
+	     xfs_calc_buf_res(XFS_DIROP_LOG_COUNT(mp), XFS_FSB_TO_B(mp, 1));
+	t2 = xfs_calc_buf_res(4, mp->m_sb.sb_sectsize) +
+	     xfs_calc_buf_res(xfs_allocfree_block_count(mp, 2),
+			      XFS_FSB_TO_B(mp, 1));
+
+	if (xfs_has_parent(mp)) {
+		t3 = resp->tr_attrrm.tr_logres;
+		overhead += xfs_calc_pptr_unlink_overhead();
+	}
+
+	return overhead + max3(t1, t2, t3);
 }
 
 /*
@@ -569,12 +704,40 @@ xfs_calc_icreate_resv_alloc(
 		xfs_calc_finobt_res(mp);
 }
 
+static inline unsigned int
+xfs_icreate_log_count(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	unsigned int		ret = XFS_CREATE_LOG_COUNT;
+
+	/*
+	 * Pre-reserve enough log reservation to handle the transaction
+	 * rolling needed to add one parent pointer.
+	 */
+	if (xfs_has_parent(mp))
+		ret += resp->tr_attrsetm.tr_logcount;
+
+	return ret;
+}
+
 STATIC uint
-xfs_calc_icreate_reservation(xfs_mount_t *mp)
+xfs_calc_icreate_reservation(
+	struct xfs_mount	*mp)
 {
-	return XFS_DQUOT_LOGRES(mp) +
-		max(xfs_calc_icreate_resv_alloc(mp),
-		    xfs_calc_create_resv_modify(mp));
+	struct xfs_trans_resv   *resp = M_RES(mp);
+	unsigned int		overhead = XFS_DQUOT_LOGRES(mp);
+	unsigned int		t1, t2, t3 = 0;
+
+	t1 = xfs_calc_icreate_resv_alloc(mp);
+	t2 = xfs_calc_create_resv_modify(mp);
+
+	if (xfs_has_parent(mp)) {
+		t3 = resp->tr_attrsetm.tr_logres;
+		overhead += xfs_calc_pptr_link_overhead();
+	}
+
+	return overhead + max3(t1, t2, t3);
 }
 
 STATIC uint
@@ -587,6 +750,23 @@ xfs_calc_create_tmpfile_reservation(
 	return res + xfs_calc_iunlink_add_reservation(mp);
 }
 
+static inline unsigned int
+xfs_mkdir_log_count(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	unsigned int		ret = XFS_MKDIR_LOG_COUNT;
+
+	/*
+	 * Pre-reserve enough log reservation to handle the transaction
+	 * rolling needed to add one parent pointer.
+	 */
+	if (xfs_has_parent(mp))
+		ret += resp->tr_attrsetm.tr_logcount;
+
+	return ret;
+}
+
 /*
  * Making a new directory is the same as creating a new file.
  */
@@ -597,6 +777,22 @@ xfs_calc_mkdir_reservation(
 	return xfs_calc_icreate_reservation(mp);
 }
 
+static inline unsigned int
+xfs_symlink_log_count(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	unsigned int		ret = XFS_SYMLINK_LOG_COUNT;
+
+	/*
+	 * Pre-reserve enough log reservation to handle the transaction
+	 * rolling needed to add one parent pointer.
+	 */
+	if (xfs_has_parent(mp))
+		ret += resp->tr_attrsetm.tr_logcount;
+
+	return ret;
+}
 
 /*
  * Making a new symplink is the same as creating a new file, but
@@ -909,6 +1105,52 @@ xfs_calc_sb_reservation(
 	return xfs_calc_buf_res(1, mp->m_sb.sb_sectsize);
 }
 
+/*
+ * Namespace reservations.
+ *
+ * These get tricky when parent pointers are enabled as we have attribute
+ * modifications occurring from within these transactions. Rather than confuse
+ * each of these reservation calculations with the conditional attribute
+ * reservations, add them here in a clear and concise manner. This requires that
+ * the attribute reservations have already been calculated.
+ *
+ * Note that we only include the static attribute reservation here; the runtime
+ * reservation will have to be modified by the size of the attributes being
+ * added/removed/modified. See the comments on the attribute reservation
+ * calculations for more details.
+ */
+STATIC void
+xfs_calc_namespace_reservations(
+	struct xfs_mount	*mp,
+	struct xfs_trans_resv	*resp)
+{
+	ASSERT(resp->tr_attrsetm.tr_logres > 0);
+
+	resp->tr_rename.tr_logres = xfs_calc_rename_reservation(mp);
+	resp->tr_rename.tr_logcount = xfs_rename_log_count(mp, resp);
+	resp->tr_rename.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
+
+	resp->tr_link.tr_logres = xfs_calc_link_reservation(mp);
+	resp->tr_link.tr_logcount = xfs_link_log_count(mp, resp);
+	resp->tr_link.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
+
+	resp->tr_remove.tr_logres = xfs_calc_remove_reservation(mp);
+	resp->tr_remove.tr_logcount = xfs_remove_log_count(mp, resp);
+	resp->tr_remove.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
+
+	resp->tr_symlink.tr_logres = xfs_calc_symlink_reservation(mp);
+	resp->tr_symlink.tr_logcount = xfs_symlink_log_count(mp, resp);
+	resp->tr_symlink.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
+
+	resp->tr_create.tr_logres = xfs_calc_icreate_reservation(mp);
+	resp->tr_create.tr_logcount = xfs_icreate_log_count(mp, resp);
+	resp->tr_create.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
+
+	resp->tr_mkdir.tr_logres = xfs_calc_mkdir_reservation(mp);
+	resp->tr_mkdir.tr_logcount = xfs_mkdir_log_count(mp, resp);
+	resp->tr_mkdir.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
+}
+
 void
 xfs_trans_resv_calc(
 	struct xfs_mount	*mp,
@@ -928,35 +1170,11 @@ xfs_trans_resv_calc(
 	resp->tr_itruncate.tr_logcount = XFS_ITRUNCATE_LOG_COUNT;
 	resp->tr_itruncate.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
 
-	resp->tr_rename.tr_logres = xfs_calc_rename_reservation(mp);
-	resp->tr_rename.tr_logcount = XFS_RENAME_LOG_COUNT;
-	resp->tr_rename.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
-
-	resp->tr_link.tr_logres = xfs_calc_link_reservation(mp);
-	resp->tr_link.tr_logcount = XFS_LINK_LOG_COUNT;
-	resp->tr_link.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
-
-	resp->tr_remove.tr_logres = xfs_calc_remove_reservation(mp);
-	resp->tr_remove.tr_logcount = XFS_REMOVE_LOG_COUNT;
-	resp->tr_remove.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
-
-	resp->tr_symlink.tr_logres = xfs_calc_symlink_reservation(mp);
-	resp->tr_symlink.tr_logcount = XFS_SYMLINK_LOG_COUNT;
-	resp->tr_symlink.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
-
-	resp->tr_create.tr_logres = xfs_calc_icreate_reservation(mp);
-	resp->tr_create.tr_logcount = XFS_CREATE_LOG_COUNT;
-	resp->tr_create.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
-
 	resp->tr_create_tmpfile.tr_logres =
 			xfs_calc_create_tmpfile_reservation(mp);
 	resp->tr_create_tmpfile.tr_logcount = XFS_CREATE_TMPFILE_LOG_COUNT;
 	resp->tr_create_tmpfile.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
 
-	resp->tr_mkdir.tr_logres = xfs_calc_mkdir_reservation(mp);
-	resp->tr_mkdir.tr_logcount = XFS_MKDIR_LOG_COUNT;
-	resp->tr_mkdir.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
-
 	resp->tr_ifree.tr_logres = xfs_calc_ifree_reservation(mp);
 	resp->tr_ifree.tr_logcount = XFS_INACTIVE_LOG_COUNT;
 	resp->tr_ifree.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
@@ -986,6 +1204,8 @@ xfs_trans_resv_calc(
 	resp->tr_qm_dqalloc.tr_logcount = XFS_WRITE_LOG_COUNT;
 	resp->tr_qm_dqalloc.tr_logflags |= XFS_TRANS_PERM_LOG_RES;
 
+	xfs_calc_namespace_reservations(mp, resp);
+
 	/*
 	 * The following transactions are logged in logical format with
 	 * a default log count.
#16 lore raw

PATCH 15/28 xfs: parent pointer attribute creation

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Add parent pointer attribute during xfs_create, and subroutines to initialize attributes

---
 fs/xfs/Makefile               |    1 
 fs/xfs/libxfs/xfs_attr.c      |    4 +
 fs/xfs/libxfs/xfs_attr.h      |    4 +
 fs/xfs/libxfs/xfs_da_format.h |   12 ----
 fs/xfs/libxfs/xfs_parent.c    |  139 +++++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_parent.h    |   57 +++++++++++++++++
 fs/xfs/xfs_inode.c            |   64 ++++++++++++++++---
 fs/xfs/xfs_super.c            |   10 +++
 fs/xfs/xfs_xattr.c            |    4 +
 fs/xfs/xfs_xattr.h            |    2 +
 10 files changed, 271 insertions(+), 26 deletions(-)
 create mode 100644 fs/xfs/libxfs/xfs_parent.c
 create mode 100644 fs/xfs/libxfs/xfs_parent.h
fs/xfs/Makefile +1 -0
index 03135a1c31b6..e2b2cf50ffcf 100644
@@ -40,6 +40,7 @@ xfs-y				+= $(addprefix libxfs/, \
 				   xfs_inode_fork.o \
 				   xfs_inode_buf.o \
 				   xfs_log_rlimit.o \
+				   xfs_parent.o \
 				   xfs_ag_resv.o \
 				   xfs_rmap.o \
 				   xfs_rmap_btree.o \
fs/xfs/libxfs/xfs_attr.c +2 -2
index 711022742e34..f68d41f0f998 100644
@@ -886,7 +886,7 @@ xfs_attr_lookup(
 	return error;
 }
 
-static int
+int
 xfs_attr_intent_init(
 	struct xfs_da_args	*args,
 	unsigned int		op_flags,	/* op flag (set or remove) */
@@ -904,7 +904,7 @@ xfs_attr_intent_init(
 }
 
 /* Sets an attribute for an inode as a deferred operation */
-static int
+int
 xfs_attr_defer_add(
 	struct xfs_da_args	*args)
 {
fs/xfs/libxfs/xfs_attr.h +3 -1
index b79dae788cfb..0cf23f5117ad 100644
@@ -544,6 +544,7 @@ int xfs_inode_hasattr(struct xfs_inode *ip);
 bool xfs_attr_is_leaf(struct xfs_inode *ip);
 int xfs_attr_get_ilocked(struct xfs_da_args *args);
 int xfs_attr_get(struct xfs_da_args *args);
+int xfs_attr_defer_add(struct xfs_da_args *args);
 int xfs_attr_set(struct xfs_da_args *args);
 int xfs_attr_set_iter(struct xfs_attr_intent *attr);
 int xfs_attr_remove_iter(struct xfs_attr_intent *attr);
@@ -552,7 +553,8 @@ bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,
 int xfs_attr_calc_size(struct xfs_da_args *args, int *local);
 void xfs_init_attr_trans(struct xfs_da_args *args, struct xfs_trans_res *tres,
 			 unsigned int *total);
-
+int xfs_attr_intent_init(struct xfs_da_args *args, unsigned int op_flags,
+			 struct xfs_attr_intent  **attr);
 /*
  * Check to see if the attr should be upgraded from non-existent or shortform to
  * single-leaf-block attribute list.
fs/xfs/libxfs/xfs_da_format.h +0 -12
index 75b13807145d..2db1cf97b2c8 100644
@@ -826,16 +826,4 @@ struct xfs_parent_name_rec {
 	__be32  p_diroffset;
 };
 
-/*
- * incore version of the above, also contains name pointers so callers
- * can pass/obtain all the parent pointer information in a single structure
- */
-struct xfs_parent_name_irec {
-	xfs_ino_t		p_ino;
-	uint32_t		p_gen;
-	xfs_dir2_dataptr_t	p_diroffset;
-	const char		*p_name;
-	uint8_t			p_namelen;
-};
-
 #endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_parent.c new file +139 -0

139 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h new file +57 -0

57 lines not on this page · raw message

fs/xfs/xfs_inode.c +55 -9

143 lines not on this page · raw message

fs/xfs/xfs_super.c +10 -0
index 0c4b73e9b29d..6795761c31e0 100644
@@ -41,6 +41,7 @@
 #include "xfs_attr_item.h"
 #include "xfs_xattr.h"
 #include "xfs_iunlink_item.h"
+#include "xfs_parent.h"
 
 #include <linux/magic.h>
 #include <linux/fs_context.h>
@@ -2115,8 +2116,16 @@ xfs_init_caches(void)
 	if (!xfs_iunlink_cache)
 		goto out_destroy_attri_cache;
 
+	xfs_parent_intent_cache = kmem_cache_create("xfs_parent_intent",
+					     sizeof(struct xfs_parent_defer),
+					     0, 0, NULL);
+	if (!xfs_parent_intent_cache)
+		goto out_destroy_iul_cache;
+
 	return 0;
 
+ out_destroy_iul_cache:
+	kmem_cache_destroy(xfs_iunlink_cache);
  out_destroy_attri_cache:
 	kmem_cache_destroy(xfs_attri_cache);
  out_destroy_attrd_cache:
@@ -2171,6 +2180,7 @@ xfs_destroy_caches(void)
 	 * destroy caches.
 	 */
 	rcu_barrier();
+	kmem_cache_destroy(xfs_parent_intent_cache);
 	kmem_cache_destroy(xfs_iunlink_cache);
 	kmem_cache_destroy(xfs_attri_cache);
 	kmem_cache_destroy(xfs_attrd_cache);
fs/xfs/xfs_xattr.c +2 -2
index 10aa1fd39d2b..8bb5f53a31fe 100644
@@ -27,7 +27,7 @@
  * they must release the permission by calling xlog_drop_incompat_feat
  * when they're done.
  */
-static inline int
+int
 xfs_attr_grab_log_assist(
 	struct xfs_mount	*mp)
 {
@@ -61,7 +61,7 @@ xfs_attr_grab_log_assist(
 	return error;
 }
 
-static inline void
+void
 xfs_attr_rele_log_assist(
 	struct xfs_mount	*mp)
 {
fs/xfs/xfs_xattr.h +2 -0
index 2b09133b1b9b..7e0a2f3bb7f8 100644
@@ -7,6 +7,8 @@
 #define __XFS_XATTR_H__
 
 int xfs_attr_change(struct xfs_da_args *args);
+int xfs_attr_grab_log_assist(struct xfs_mount *mp);
+void xfs_attr_rele_log_assist(struct xfs_mount *mp);
 
 extern const struct xattr_handler *xfs_xattr_handlers[];
 
#17 lore raw

PATCH 16/28 xfs: add parent attributes to link

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

This patch modifies xfs_link to add a parent pointer to the inode.

---
 fs/xfs/libxfs/xfs_trans_space.h |    2 -
 fs/xfs/xfs_inode.c              |   60 ++++++++++++++++++++++++++++++++++-----
 2 files changed, 53 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_trans_space.h +0 -2
index 87b31c69a773..f72207923ec2 100644
@@ -84,8 +84,6 @@
 	(2 * (mp)->m_alloc_maxlevels)
 #define	XFS_GROWFSRT_SPACE_RES(mp,b)	\
 	((b) + XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK))
-#define	XFS_LINK_SPACE_RES(mp,nl)	\
-	XFS_DIRENTER_SPACE_RES(mp,nl)
 #define	XFS_MKDIR_SPACE_RES(mp,nl)	\
 	(XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))
 #define	XFS_QM_DQALLOC_SPACE_RES(mp)	\
fs/xfs/xfs_inode.c +53 -7

105 lines not on this page · raw message

#18 lore raw

PATCH 17/28 xfs: add parent attributes to symlink

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

This patch modifies xfs_symlink to add a parent pointer to the inode.

---
 fs/xfs/libxfs/xfs_trans_space.h |    2 -
 fs/xfs/xfs_symlink.c            |   58 ++++++++++++++++++++++++++++++++-------
 2 files changed, 48 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_trans_space.h +0 -2
index f72207923ec2..25a55650baf4 100644
@@ -95,8 +95,6 @@
 	XFS_DIRREMOVE_SPACE_RES(mp)
 #define	XFS_RENAME_SPACE_RES(mp,nl)	\
 	(XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))
-#define	XFS_SYMLINK_SPACE_RES(mp,nl,b)	\
-	(XFS_IALLOC_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl) + (b))
 #define XFS_IFREE_SPACE_RES(mp)		\
 	(xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
 
fs/xfs/xfs_symlink.c +48 -10

134 lines not on this page · raw message

#19 lore raw

PATCH 18/28 xfs: remove parent pointers in unlink

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

This patch removes the parent pointer attribute during unlink

---
 fs/xfs/libxfs/xfs_attr.c        |    2 +-
 fs/xfs/libxfs/xfs_attr.h        |    1 +
 fs/xfs/libxfs/xfs_parent.c      |   17 ++++++++++++++++
 fs/xfs/libxfs/xfs_parent.h      |    5 +++++
 fs/xfs/libxfs/xfs_trans_space.h |    2 --
 fs/xfs/xfs_inode.c              |   42 +++++++++++++++++++++++++++++++++------
 6 files changed, 59 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
index f68d41f0f998..a8db44728b11 100644
@@ -946,7 +946,7 @@ xfs_attr_defer_replace(
 }
 
 /* Removes an attribute for an inode as a deferred operation */
-static int
+int
 xfs_attr_defer_remove(
 	struct xfs_da_args	*args)
 {
fs/xfs/libxfs/xfs_attr.h +1 -0
index 0cf23f5117ad..033005542b9e 100644
@@ -545,6 +545,7 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip);
 int xfs_attr_get_ilocked(struct xfs_da_args *args);
 int xfs_attr_get(struct xfs_da_args *args);
 int xfs_attr_defer_add(struct xfs_da_args *args);
+int xfs_attr_defer_remove(struct xfs_da_args *args);
 int xfs_attr_set(struct xfs_da_args *args);
 int xfs_attr_set_iter(struct xfs_attr_intent *attr);
 int xfs_attr_remove_iter(struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +17 -0
index 6b6d415319e6..245855a5f969 100644
@@ -115,6 +115,23 @@ xfs_parent_defer_add(
 	return xfs_attr_defer_add(args);
 }
 
+int
+xfs_parent_defer_remove(
+	struct xfs_trans	*tp,
+	struct xfs_inode	*dp,
+	struct xfs_parent_defer	*parent,
+	xfs_dir2_dataptr_t	diroffset,
+	struct xfs_inode	*child)
+{
+	struct xfs_da_args	*args = &parent->args;
+
+	xfs_init_parent_name_rec(&parent->rec, dp, diroffset);
+	args->trans = tp;
+	args->dp = child;
+	args->hashval = xfs_da_hashname(args->name, args->namelen);
+	return xfs_attr_defer_remove(args);
+}
+
 void
 __xfs_parent_cancel(
 	xfs_mount_t		*mp,
fs/xfs/libxfs/xfs_parent.h +5 -0
index d5a8c8e52cb5..0f39d033d84e 100644
@@ -40,6 +40,11 @@ xfs_parent_start(
 int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
 			 struct xfs_inode *dp, struct xfs_name *parent_name,
 			 xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
+int xfs_parent_defer_remove(struct xfs_trans *tp, struct xfs_inode *dp,
+			    struct xfs_parent_defer *parent,
+			    xfs_dir2_dataptr_t diroffset,
+			    struct xfs_inode *child);
+
 void __xfs_parent_cancel(struct xfs_mount *mp, struct xfs_parent_defer *parent);
 
 static inline void
fs/xfs/libxfs/xfs_trans_space.h +0 -2
index 25a55650baf4..b5ab6701e7fb 100644
@@ -91,8 +91,6 @@
 	 XFS_DQUOT_CLUSTER_SIZE_FSB)
 #define	XFS_QM_QINOCREATE_SPACE_RES(mp)	\
 	XFS_IALLOC_SPACE_RES(mp)
-#define	XFS_REMOVE_SPACE_RES(mp)	\
-	XFS_DIRREMOVE_SPACE_RES(mp)
 #define	XFS_RENAME_SPACE_RES(mp,nl)	\
 	(XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))
 #define XFS_IFREE_SPACE_RES(mp)		\
fs/xfs/xfs_inode.c +35 -7

99 lines not on this page · raw message

#20 lore raw

PATCH 19/28 xfs: Indent xfs_rename

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Indent variables and parameters in xfs_rename in preparation for parent pointer modifications. White space only, no functional changes. This will make reviewing new code easier on reviewers.

---
 fs/xfs/xfs_inode.c |   39 ++++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 19 deletions(-)
fs/xfs/xfs_inode.c +20 -19
index 7b34ca2de569..2d8f225cb57d 100644
@@ -2902,26 +2902,27 @@ xfs_rename_alloc_whiteout(
  */
 int
 xfs_rename(
-	struct user_namespace	*mnt_userns,
-	struct xfs_inode	*src_dp,
-	struct xfs_name		*src_name,
-	struct xfs_inode	*src_ip,
-	struct xfs_inode	*target_dp,
-	struct xfs_name		*target_name,
-	struct xfs_inode	*target_ip,
-	unsigned int		flags)
+	struct user_namespace		*mnt_userns,
+	struct xfs_inode		*src_dp,
+	struct xfs_name			*src_name,
+	struct xfs_inode		*src_ip,
+	struct xfs_inode		*target_dp,
+	struct xfs_name			*target_name,
+	struct xfs_inode		*target_ip,
+	unsigned int			flags)
 {
-	struct xfs_mount	*mp = src_dp->i_mount;
-	struct xfs_trans	*tp;
-	struct xfs_inode	*wip = NULL;		/* whiteout inode */
-	struct xfs_inode	*inodes[__XFS_SORT_INODES];
-	int			i;
-	int			num_inodes = __XFS_SORT_INODES;
-	bool			new_parent = (src_dp != target_dp);
-	bool			src_is_directory = S_ISDIR(VFS_I(src_ip)->i_mode);
-	int			spaceres;
-	bool			retried = false;
-	int			error, nospace_error = 0;
+	struct xfs_mount		*mp = src_dp->i_mount;
+	struct xfs_trans		*tp;
+	struct xfs_inode		*wip = NULL;	/* whiteout inode */
+	struct xfs_inode		*inodes[__XFS_SORT_INODES];
+	int				i;
+	int				num_inodes = __XFS_SORT_INODES;
+	bool				new_parent = (src_dp != target_dp);
+	bool				src_is_directory =
+						S_ISDIR(VFS_I(src_ip)->i_mode);
+	int				spaceres;
+	bool				retried = false;
+	int				error, nospace_error = 0;
 
 	trace_xfs_rename(src_dp, target_dp, src_name, target_name);
 
#21 lore raw

PATCH 20/28 xfs: Add parent pointers to rename

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

This patch removes the old parent pointer attribute during the rename operation, and re-adds the updated parent pointer.

---
 fs/xfs/libxfs/xfs_attr.c        |    2 -
 fs/xfs/libxfs/xfs_attr.h        |    1 
 fs/xfs/libxfs/xfs_parent.c      |   47 ++++++++++++++--
 fs/xfs/libxfs/xfs_parent.h      |   24 +++++++-
 fs/xfs/libxfs/xfs_trans_space.h |    2 -
 fs/xfs/xfs_inode.c              |  117 ++++++++++++++++++++++++++++++++++++---
 6 files changed, 174 insertions(+), 19 deletions(-)
fs/xfs/libxfs/xfs_attr.c +1 -1
index a8db44728b11..57080ea4c869 100644
@@ -923,7 +923,7 @@ xfs_attr_defer_add(
 }
 
 /* Sets an attribute for an inode as a deferred operation */
-static int
+int
 xfs_attr_defer_replace(
 	struct xfs_da_args	*args)
 {
fs/xfs/libxfs/xfs_attr.h +1 -0
index 033005542b9e..985761264d1f 100644
@@ -546,6 +546,7 @@ int xfs_attr_get_ilocked(struct xfs_da_args *args);
 int xfs_attr_get(struct xfs_da_args *args);
 int xfs_attr_defer_add(struct xfs_da_args *args);
 int xfs_attr_defer_remove(struct xfs_da_args *args);
+int xfs_attr_defer_replace(struct xfs_da_args *args);
 int xfs_attr_set(struct xfs_da_args *args);
 int xfs_attr_set_iter(struct xfs_attr_intent *attr);
 int xfs_attr_remove_iter(struct xfs_attr_intent *attr);
fs/xfs/libxfs/xfs_parent.c +42 -5

76 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +22 -2
index 0f39d033d84e..039005883bb6 100644
@@ -14,7 +14,9 @@ extern struct kmem_cache	*xfs_parent_intent_cache;
  */
 struct xfs_parent_defer {
 	struct xfs_parent_name_rec	rec;
+	struct xfs_parent_name_rec	old_rec;
 	struct xfs_da_args		args;
+	bool				have_log;
 };
 
 /*
@@ -23,7 +25,8 @@ struct xfs_parent_defer {
 void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
 			      struct xfs_inode *ip,
 			      uint32_t p_diroffset);
-int __xfs_parent_init(struct xfs_mount *mp, struct xfs_parent_defer **parentp);
+int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
+		struct xfs_parent_defer **parentp);
 
 static inline int
 xfs_parent_start(
@@ -33,13 +36,30 @@ xfs_parent_start(
 	*pp = NULL;
 
 	if (xfs_has_parent(mp))
-		return __xfs_parent_init(mp, pp);
+		return __xfs_parent_init(mp, true, pp);
+	return 0;
+}
+
+static inline int
+xfs_parent_start_locked(
+	struct xfs_mount	*mp,
+	struct xfs_parent_defer	**pp)
+{
+	*pp = NULL;
+
+	if (xfs_has_parent(mp))
+		return __xfs_parent_init(mp, false, pp);
 	return 0;
 }
 
 int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
 			 struct xfs_inode *dp, struct xfs_name *parent_name,
 			 xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
+int xfs_parent_defer_replace(struct xfs_trans *tp,
+		struct xfs_parent_defer *new_parent, struct xfs_inode *old_dp,
+		xfs_dir2_dataptr_t old_diroffset, struct xfs_name *parent_name,
+		struct xfs_inode *new_ip, xfs_dir2_dataptr_t new_diroffset,
+		struct xfs_inode *child);
 int xfs_parent_defer_remove(struct xfs_trans *tp, struct xfs_inode *dp,
 			    struct xfs_parent_defer *parent,
 			    xfs_dir2_dataptr_t diroffset,
fs/xfs/libxfs/xfs_trans_space.h +0 -2
index b5ab6701e7fb..810610a14c4d 100644
@@ -91,8 +91,6 @@
 	 XFS_DQUOT_CLUSTER_SIZE_FSB)
 #define	XFS_QM_QINOCREATE_SPACE_RES(mp)	\
 	XFS_IALLOC_SPACE_RES(mp)
-#define	XFS_RENAME_SPACE_RES(mp,nl)	\
-	(XFS_DIRREMOVE_SPACE_RES(mp) + XFS_DIRENTER_SPACE_RES(mp,nl))
 #define XFS_IFREE_SPACE_RES(mp)		\
 	(xfs_has_finobt(mp) ? M_IGEO(mp)->inobt_maxlevels : 0)
 
fs/xfs/xfs_inode.c +108 -9

190 lines not on this page · raw message

#22 lore raw

PATCH 21/28 xfs: Add parent pointers to xfs_cross_rename

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Cross renames are handled separately from standard renames, and need different handling to update the parent attributes correctly.

---
 fs/xfs/xfs_inode.c |   49 +++++++++++++++++++++++++++++++++----------------
 1 file changed, 33 insertions(+), 16 deletions(-)
fs/xfs/xfs_inode.c +33 -16
index cdbd7df64ff0..6626aa7486f1 100644
@@ -2749,27 +2749,31 @@ xfs_finish_rename(
  */
 STATIC int
 xfs_cross_rename(
-	struct xfs_trans	*tp,
-	struct xfs_inode	*dp1,
-	struct xfs_name		*name1,
-	struct xfs_inode	*ip1,
-	struct xfs_inode	*dp2,
-	struct xfs_name		*name2,
-	struct xfs_inode	*ip2,
-	int			spaceres)
+	struct xfs_trans		*tp,
+	struct xfs_inode		*dp1,
+	struct xfs_name			*name1,
+	struct xfs_inode		*ip1,
+	struct xfs_parent_defer		*ip1_pptr,
+	struct xfs_inode		*dp2,
+	struct xfs_name			*name2,
+	struct xfs_inode		*ip2,
+	struct xfs_parent_defer		*ip2_pptr,
+	int				spaceres)
 {
-	int		error = 0;
-	int		ip1_flags = 0;
-	int		ip2_flags = 0;
-	int		dp2_flags = 0;
+	struct xfs_mount		*mp = dp1->i_mount;
+	int				error = 0;
+	int				ip1_flags = 0;
+	int				ip2_flags = 0;
+	int				dp2_flags = 0;
+	int				new_diroffset, old_diroffset;
 
 	/* Swap inode number for dirent in first parent */
-	error = xfs_dir_replace(tp, dp1, name1, ip2->i_ino, spaceres, NULL);
+	error = xfs_dir_replace(tp, dp1, name1, ip2->i_ino, spaceres, &old_diroffset);
 	if (error)
 		goto out_trans_abort;
 
 	/* Swap inode number for dirent in second parent */
-	error = xfs_dir_replace(tp, dp2, name2, ip1->i_ino, spaceres, NULL);
+	error = xfs_dir_replace(tp, dp2, name2, ip1->i_ino, spaceres, &new_diroffset);
 	if (error)
 		goto out_trans_abort;
 
@@ -2830,6 +2834,18 @@ xfs_cross_rename(
 		}
 	}
 
+	if (xfs_has_parent(mp)) {
+		error = xfs_parent_defer_replace(tp, ip1_pptr, dp1,
+				old_diroffset, name2, dp2, new_diroffset, ip1);
+		if (error)
+			goto out_trans_abort;
+
+		error = xfs_parent_defer_replace(tp, ip2_pptr, dp2,
+				new_diroffset, name1, dp1, old_diroffset, ip2);
+		if (error)
+			goto out_trans_abort;
+	}
+
 	if (ip1_flags) {
 		xfs_trans_ichgtime(tp, ip1, ip1_flags);
 		xfs_trans_log_inode(tp, ip1, XFS_ILOG_CORE);
@@ -2844,6 +2860,7 @@ xfs_cross_rename(
 	}
 	xfs_trans_ichgtime(tp, dp1, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
 	xfs_trans_log_inode(tp, dp1, XFS_ILOG_CORE);
+
 	return xfs_finish_rename(tp);
 
 out_trans_abort:
@@ -3060,8 +3077,8 @@ xfs_rename(
 	/* RENAME_EXCHANGE is unique from here on. */
 	if (flags & RENAME_EXCHANGE) {
 		error = xfs_cross_rename(tp, src_dp, src_name, src_ip,
-					target_dp, target_name, target_ip,
-					spaceres);
+				src_ip_pptr, target_dp, target_name, target_ip,
+				tgt_ip_pptr, spaceres);
 		goto out_unlock;
 	}
 
#23 lore raw

PATCH 22/28 xfs: Add the parent pointer support to the superblock version 5.

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

---
 fs/xfs/libxfs/xfs_format.h |    4 +++-
 fs/xfs/libxfs/xfs_fs.h     |    1 +
 fs/xfs/libxfs/xfs_sb.c     |    4 ++++
 fs/xfs/xfs_super.c         |    4 ++++
 4 files changed, 12 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +3 -1
index 371dc07233e0..f413819b2a8a 100644
@@ -373,13 +373,15 @@ xfs_sb_has_ro_compat_feature(
 #define XFS_SB_FEAT_INCOMPAT_BIGTIME	(1 << 3)	/* large timestamps */
 #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)	/* needs xfs_repair */
 #define XFS_SB_FEAT_INCOMPAT_NREXT64	(1 << 5)	/* large extent counters */
+#define XFS_SB_FEAT_INCOMPAT_PARENT	(1 << 6)	/* parent pointers */
 #define XFS_SB_FEAT_INCOMPAT_ALL \
 		(XFS_SB_FEAT_INCOMPAT_FTYPE|	\
 		 XFS_SB_FEAT_INCOMPAT_SPINODES|	\
 		 XFS_SB_FEAT_INCOMPAT_META_UUID| \
 		 XFS_SB_FEAT_INCOMPAT_BIGTIME| \
 		 XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \
-		 XFS_SB_FEAT_INCOMPAT_NREXT64)
+		 XFS_SB_FEAT_INCOMPAT_NREXT64| \
+		 XFS_SB_FEAT_INCOMPAT_PARENT)
 
 #define XFS_SB_FEAT_INCOMPAT_UNKNOWN	~XFS_SB_FEAT_INCOMPAT_ALL
 static inline bool
fs/xfs/libxfs/xfs_fs.h +1 -0
index 1cfd5bc6520a..b0b4d7a3aa15 100644
@@ -237,6 +237,7 @@ typedef struct xfs_fsop_resblks {
 #define XFS_FSOP_GEOM_FLAGS_BIGTIME	(1 << 21) /* 64-bit nsec timestamps */
 #define XFS_FSOP_GEOM_FLAGS_INOBTCNT	(1 << 22) /* inobt btree counter */
 #define XFS_FSOP_GEOM_FLAGS_NREXT64	(1 << 23) /* large extent counters */
+#define XFS_FSOP_GEOM_FLAGS_PARENT	(1 << 24) /* parent pointers 	    */
 
 /*
  * Minimum and maximum sizes need for growth checks.
fs/xfs/libxfs/xfs_sb.c +4 -0
index 1eeecf2eb2a7..a59bf09495b1 100644
@@ -173,6 +173,8 @@ xfs_sb_version_to_features(
 		features |= XFS_FEAT_NEEDSREPAIR;
 	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_NREXT64)
 		features |= XFS_FEAT_NREXT64;
+	if (sbp->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_PARENT)
+		features |= XFS_FEAT_PARENT;
 
 	return features;
 }
@@ -1189,6 +1191,8 @@ xfs_fs_geometry(
 		geo->flags |= XFS_FSOP_GEOM_FLAGS_BIGTIME;
 	if (xfs_has_inobtcounts(mp))
 		geo->flags |= XFS_FSOP_GEOM_FLAGS_INOBTCNT;
+	if (xfs_has_parent(mp))
+		geo->flags |= XFS_FSOP_GEOM_FLAGS_PARENT;
 	if (xfs_has_sector(mp)) {
 		geo->flags |= XFS_FSOP_GEOM_FLAGS_SECTOR;
 		geo->logsectsize = sbp->sb_logsectsize;
fs/xfs/xfs_super.c +4 -0
index 6795761c31e0..0ac55d191f1f 100644
@@ -1664,6 +1664,10 @@ xfs_fs_fill_super(
 		xfs_warn(mp,
 	"EXPERIMENTAL Large extent counts feature in use. Use at your own risk!");
 
+	if (xfs_has_parent(mp))
+		xfs_alert(mp,
+	"EXPERIMENTAL parent pointer feature enabled. Use at your own risk!");
+
 	error = xfs_mountfs(mp);
 	if (error)
 		goto out_filestream_unmount;
#24 lore raw

PATCH 23/28 xfs: Add helper function xfs_attr_list_context_init

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

This patch adds a helper function xfs_attr_list_context_init used by xfs_attr_list. This function initializes the xfs_attr_list_context structure passed to xfs_attr_list_int. We will need this later to call xfs_attr_list_int_ilocked when the node is already locked.

---
 fs/xfs/xfs_file.c  |    1 +
 fs/xfs/xfs_ioctl.c |   54 +++++++++++++++++++++++++++++++++++++---------------
 fs/xfs/xfs_ioctl.h |    2 ++
 3 files changed, 41 insertions(+), 16 deletions(-)
fs/xfs/xfs_file.c +1 -0
index 595a5bcf46b9..9c09d32a6c9e 100644
@@ -17,6 +17,7 @@
 #include "xfs_bmap_util.h"
 #include "xfs_dir2.h"
 #include "xfs_dir2_priv.h"
+#include "xfs_attr.h"
 #include "xfs_ioctl.h"
 #include "xfs_trace.h"
 #include "xfs_log.h"
fs/xfs/xfs_ioctl.c +38 -16

72 lines not on this page · raw message

fs/xfs/xfs_ioctl.h +2 -0
index d4abba2c13c1..ca60e1c427a3 100644
@@ -35,6 +35,8 @@ int xfs_ioc_attrmulti_one(struct file *parfilp, struct inode *inode,
 int xfs_ioc_attr_list(struct xfs_inode *dp, void __user *ubuf,
 		      size_t bufsize, int flags,
 		      struct xfs_attrlist_cursor __user *ucursor);
+int xfs_ioc_attr_list_context_init(struct xfs_inode *dp, char *buffer,
+		int bufsize, int flags, struct xfs_attr_list_context *context);
 
 extern struct dentry *
 xfs_handle_to_dentry(
#25 lore raw

PATCH 24/28 xfs: Filter XFS_ATTR_PARENT for getfattr

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Parent pointers returned to the get_fattr tool cause errors since the tool cannot parse parent pointers. Fix this by filtering parent parent pointers from xfs_xattr_put_listent.

---
 fs/xfs/xfs_xattr.c |    3 +++
 1 file changed, 3 insertions(+)
fs/xfs/xfs_xattr.c +3 -0
index 8bb5f53a31fe..ddc2db5d6f73 100644
@@ -234,6 +234,9 @@ xfs_xattr_put_listent(
 
 	ASSERT(context->count >= 0);
 
+	if (flags & XFS_ATTR_PARENT)
+		return;
+
 	if (flags & XFS_ATTR_ROOT) {
 #ifdef CONFIG_XFS_POSIX_ACL
 		if (namelen == SGI_ACL_FILE_SIZE &&
#26 lore raw

PATCH 25/28 xfs: Add parent pointer ioctl

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

This patch adds a new file ioctl to retrieve the parent pointer of a given inode

---
 fs/xfs/Makefile            |    1 
 fs/xfs/libxfs/xfs_fs.h     |   74 ++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_parent.c |   10 +++
 fs/xfs/libxfs/xfs_parent.h |    2 +
 fs/xfs/xfs_ioctl.c         |   94 ++++++++++++++++++++++++++++++++-
 fs/xfs/xfs_ondisk.h        |    4 +
 fs/xfs/xfs_parent_utils.c  |  126 ++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_parent_utils.h  |   11 ++++
 8 files changed, 321 insertions(+), 1 deletion(-)
 create mode 100644 fs/xfs/xfs_parent_utils.c
 create mode 100644 fs/xfs/xfs_parent_utils.h
fs/xfs/Makefile +1 -0
index e2b2cf50ffcf..42d0496fdad7 100644
@@ -86,6 +86,7 @@ xfs-y				+= xfs_aops.o \
 				   xfs_mount.o \
 				   xfs_mru_cache.o \
 				   xfs_pwork.o \
+				   xfs_parent_utils.o \
 				   xfs_reflink.o \
 				   xfs_stats.o \
 				   xfs_super.o \
fs/xfs/libxfs/xfs_fs.h +74 -0

86 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.c +10 -0
index 629762701952..9176adfaa9e8 100644
@@ -29,6 +29,16 @@
 
 struct kmem_cache		*xfs_parent_intent_cache;
 
+/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */
+void
+xfs_init_parent_ptr(struct xfs_parent_ptr		*xpp,
+		    const struct xfs_parent_name_rec	*rec)
+{
+	xpp->xpp_ino = be64_to_cpu(rec->p_ino);
+	xpp->xpp_gen = be32_to_cpu(rec->p_gen);
+	xpp->xpp_diroffset = be32_to_cpu(rec->p_diroffset);
+}
+
 /*
  * Parent pointer attribute handling.
  *
fs/xfs/libxfs/xfs_parent.h +2 -0
index 039005883bb6..13040b9d8b08 100644
@@ -25,6 +25,8 @@ struct xfs_parent_defer {
 void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
 			      struct xfs_inode *ip,
 			      uint32_t p_diroffset);
+void xfs_init_parent_ptr(struct xfs_parent_ptr *xpp,
+			 const struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
fs/xfs/xfs_ioctl.c +93 -1

112 lines not on this page · raw message

fs/xfs/xfs_ondisk.h +4 -0
index 9737b5a9f405..6a6bd05c2a68 100644
@@ -150,6 +150,10 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(struct xfs_efi_log_format_32, efi_extents,	16);
 	XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents,	16);
 
+	/* parent pointer ioctls */
+	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr,            280);
+	XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info,             104);
+
 	/*
 	 * The v5 superblock format extended several v4 header structures with
 	 * additional data. While new fields are only accessible on v5
fs/xfs/xfs_parent_utils.c new file +126 -0

126 lines not on this page · raw message

fs/xfs/xfs_parent_utils.h new file +11 -0
new file mode 100644
index 000000000000..ad60baee8b2a
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 Oracle, Inc.
+ * All rights reserved.
+ */
+#ifndef	__XFS_PARENT_UTILS_H__
+#define	__XFS_PARENT_UTILS_H__
+
+int xfs_attr_get_parent_pointer(struct xfs_inode *ip,
+				struct xfs_pptr_info *ppi);
+#endif	/* __XFS_PARENT_UTILS_H__ */
#27 lore raw

PATCH 26/28 xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Dave and I were discussing some recent test regressions as a result of me turning on nrext64=1 on realtime filesystems, when we noticed that the minimum log size of a 32M filesystem jumped from 954 blocks to 4287 blocks.

Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size contains the maximum estimated amount of space needed for a local format xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES, which requires units of blocks. This has resulted in an overestimation of the minimum log size over the years.

We should nominally correct this, but there's a backwards compatibility problem -- if we enable it now, the minimum log size will decrease. If a corrected mkfs formats a filesystem with this new smaller log size, a user will encounter mount failures on an uncorrected kernel due to the larger minimum log size computations there.

However, the large extent counters feature is still EXPERIMENTAL, so we can gate the correction on that feature (or any features that get added after that) being enabled. Any filesystem with nrext64 or any of the as-yet-undefined feature bits turned on will be rejected by old uncorrected kernels, so this should be safe even in the upgrade case.

---
 fs/xfs/libxfs/xfs_log_rlimit.c |   43 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +43 -0
index 9975b93a7412..e5c606fb7a6a 100644
@@ -16,6 +16,39 @@
 #include "xfs_bmap_btree.h"
 #include "xfs_trace.h"
 
+/*
+ * Decide if the filesystem has the parent pointer feature or any feature
+ * added after that.
+ */
+static inline bool
+xfs_has_parent_or_newer_feature(
+	struct xfs_mount	*mp)
+{
+	if (!xfs_sb_is_v5(&mp->m_sb))
+		return false;
+
+	if (xfs_sb_has_compat_feature(&mp->m_sb, ~0))
+		return true;
+
+	if (xfs_sb_has_ro_compat_feature(&mp->m_sb,
+				~(XFS_SB_FEAT_RO_COMPAT_FINOBT |
+				 XFS_SB_FEAT_RO_COMPAT_RMAPBT |
+				 XFS_SB_FEAT_RO_COMPAT_REFLINK |
+				 XFS_SB_FEAT_RO_COMPAT_INOBTCNT)))
+		return true;
+
+	if (xfs_sb_has_incompat_feature(&mp->m_sb,
+				~(XFS_SB_FEAT_INCOMPAT_FTYPE |
+				 XFS_SB_FEAT_INCOMPAT_SPINODES |
+				 XFS_SB_FEAT_INCOMPAT_META_UUID |
+				 XFS_SB_FEAT_INCOMPAT_BIGTIME |
+				 XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR |
+				 XFS_SB_FEAT_INCOMPAT_NREXT64)))
+		return true;
+
+	return false;
+}
+
 /*
  * Calculate the maximum length in bytes that would be required for a local
  * attribute value as large attributes out of line are not logged.
@@ -31,6 +64,16 @@ xfs_log_calc_max_attrsetm_res(
 	       MAXNAMELEN - 1;
 	nblks = XFS_DAENTER_SPACE_RES(mp, XFS_ATTR_FORK);
 	nblks += XFS_B_TO_FSB(mp, size);
+
+	/*
+	 * Starting with the parent pointer feature, every new fs feature
+	 * corrects a unit conversion error in the xattr transaction
+	 * reservation code that resulted in oversized minimum log size
+	 * computations.
+	 */
+	if (xfs_has_parent_or_newer_feature(mp))
+		size = XFS_B_TO_FSB(mp, size);
+
 	nblks += XFS_NEXTENTADD_SPACE_RES(mp, size, XFS_ATTR_FORK);
 
 	return  M_RES(mp)->tr_attrsetm.tr_logres +
#28 lore raw

PATCH 27/28 xfs: drop compatibility minimum log size computations for reflink

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Having established that we can reduce the minimum log size computation for filesystems with parent pointers or any newer feature, we should also drop the compat minlogsize code that we added when we reduced the transaction reservation size for rmap and reflink.

---
 fs/xfs/libxfs/xfs_log_rlimit.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
fs/xfs/libxfs/xfs_log_rlimit.c +10 -0
index e5c606fb7a6a..74821c7fd0cc 100644
@@ -91,6 +91,16 @@ xfs_log_calc_trans_resv_for_minlogblocks(
 {
 	unsigned int		rmap_maxlevels = mp->m_rmap_maxlevels;
 
+	/*
+	 * Starting with the parent pointer feature, every new fs feature
+	 * drops the oversized minimum log size computation introduced by the
+	 * original reflink code.
+	 */
+	if (xfs_has_parent_or_newer_feature(mp)) {
+		xfs_trans_resv_calc(mp, resv);
+		return;
+	}
+
 	/*
 	 * In the early days of rmap+reflink, we always set the rmap maxlevels
 	 * to 9 even if the AG was small enough that it would never grow to
#29 lore raw

PATCH 28/28 xfs: add xfs_trans_mod_sb tracing

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Reservationless operations are not allowed with parent pointers because the attr expansion may cause a shutdown if an operation is retried without reservation and succeeds without enough space for the parent pointer. Add tracing to detect if this shutdown occurs.

---
 fs/xfs/xfs_trans.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
fs/xfs/xfs_trans.c +3 -1
index 43f4b0943f49..bfb7e87e7794 100644
@@ -375,8 +375,10 @@ xfs_trans_mod_sb(
 		 */
 		if (delta < 0) {
 			tp->t_blk_res_used += (uint)-delta;
-			if (tp->t_blk_res_used > tp->t_blk_res)
+			if (tp->t_blk_res_used > tp->t_blk_res) {
+				xfs_err(mp, "URK blkres 0x%x used 0x%x", tp->t_blk_res, tp->t_blk_res_used);
 				xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE);
+			}
 		} else if (delta > 0 && (tp->t_flags & XFS_TRANS_RES_FDBLKS)) {
 			int64_t	blkres_delta;
 
#30 lore raw

cover xfs: bug fixes for parent pointers

In reply to Darrick J. Wong

Hi all,

This series contains the accumulated bug fixes from Darrick to make fstests pass and online repair work.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-bugfixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-bugfixes

---
 fs/xfs/libxfs/xfs_attr.c       |   61 +++++++++-------------------------------
 fs/xfs/libxfs/xfs_attr.h       |    2 +
 fs/xfs/libxfs/xfs_attr_leaf.c  |    6 +++-
 fs/xfs/libxfs/xfs_dir2_block.c |    2 +
 fs/xfs/libxfs/xfs_dir2_leaf.c  |    2 +
 fs/xfs/libxfs/xfs_dir2_node.c  |    2 +
 fs/xfs/libxfs/xfs_dir2_sf.c    |    4 +++
 fs/xfs/libxfs/xfs_parent.c     |   44 +++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_parent.h     |    7 +++++
 9 files changed, 79 insertions(+), 51 deletions(-)
#31 lore raw

PATCH 1/3 xfs: directory lookups should return diroffsets too

From: Darrick J. Wong <[email protected]>

Teach the directory lookup functions to return the dir offset of the dirent that it finds. Online fsck will use this when checking and repairing filesystems.

---
 fs/xfs/libxfs/xfs_dir2_block.c |    2 ++
 fs/xfs/libxfs/xfs_dir2_leaf.c  |    2 ++
 fs/xfs/libxfs/xfs_dir2_node.c  |    2 ++
 fs/xfs/libxfs/xfs_dir2_sf.c    |    4 ++++
 4 files changed, 10 insertions(+)
fs/xfs/libxfs/xfs_dir2_block.c +2 -0
index 0f3a03e87278..24467e1a0d6f 100644
@@ -749,6 +749,8 @@ xfs_dir2_block_lookup_int(
 		cmp = xfs_dir2_compname(args, dep->name, dep->namelen);
 		if (cmp != XFS_CMP_DIFFERENT && cmp != args->cmpresult) {
 			args->cmpresult = cmp;
+			args->offset = xfs_dir2_byte_to_dataptr(
+					(char *)dep - (char *)hdr);
 			*bpp = bp;
 			*entno = mid;
 			if (cmp == XFS_CMP_EXACT)
fs/xfs/libxfs/xfs_dir2_leaf.c +2 -0
index fe75ffadace9..b7ea73b4f592 100644
@@ -1300,6 +1300,8 @@ xfs_dir2_leaf_lookup_int(
 		cmp = xfs_dir2_compname(args, dep->name, dep->namelen);
 		if (cmp != XFS_CMP_DIFFERENT && cmp != args->cmpresult) {
 			args->cmpresult = cmp;
+			args->offset = xfs_dir2_db_off_to_dataptr(args->geo,
+					newdb, (char *)dep - (char *)dbp->b_addr);
 			*indexp = index;
 			/* case exact match: return the current buffer. */
 			if (cmp == XFS_CMP_EXACT) {
fs/xfs/libxfs/xfs_dir2_node.c +2 -0
index 53cd0d5d94f7..f8c01e8d885c 100644
@@ -887,6 +887,8 @@ xfs_dir2_leafn_lookup_for_entry(
 			args->cmpresult = cmp;
 			args->inumber = be64_to_cpu(dep->inumber);
 			args->filetype = xfs_dir2_data_get_ftype(mp, dep);
+			args->offset = xfs_dir2_db_off_to_dataptr(args->geo,
++					newdb, (char *)dep - (char *)curbp->b_addr);
 			*indexp = index;
 			state->extravalid = 1;
 			state->extrablk.bp = curbp;
fs/xfs/libxfs/xfs_dir2_sf.c +4 -0
index 032c65804610..f8670c56c7a6 100644
@@ -889,6 +889,7 @@ xfs_dir2_sf_lookup(
 		args->inumber = dp->i_ino;
 		args->cmpresult = XFS_CMP_EXACT;
 		args->filetype = XFS_DIR3_FT_DIR;
+		args->offset = 1;
 		return -EEXIST;
 	}
 	/*
@@ -899,6 +900,7 @@ xfs_dir2_sf_lookup(
 		args->inumber = xfs_dir2_sf_get_parent_ino(sfp);
 		args->cmpresult = XFS_CMP_EXACT;
 		args->filetype = XFS_DIR3_FT_DIR;
+		args->offset = 2;
 		return -EEXIST;
 	}
 	/*
@@ -917,6 +919,8 @@ xfs_dir2_sf_lookup(
 			args->cmpresult = cmp;
 			args->inumber = xfs_dir2_sf_get_ino(mp, sfp, sfep);
 			args->filetype = xfs_dir2_sf_get_ftype(mp, sfep);
+			args->offset = xfs_dir2_byte_to_dataptr(
+						xfs_dir2_sf_get_offset(sfep));
 			if (cmp == XFS_CMP_EXACT)
 				return -EEXIST;
 			ci_sfep = sfep;
#32 lore raw

PATCH 2/3 xfs: move/add parent pointer validators to xfs_parent

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Move the parent pointer xattr name validator to xfs_parent.c, and add a new function to check the xattr value.

---
 fs/xfs/libxfs/xfs_attr.c   |   61 +++++++++-----------------------------------
 fs/xfs/libxfs/xfs_attr.h   |    2 +
 fs/xfs/libxfs/xfs_parent.c |   44 ++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_parent.h |    7 +++++
 4 files changed, 65 insertions(+), 49 deletions(-)
fs/xfs/libxfs/xfs_attr.c +13 -48
index 57080ea4c869..3065dd622102 100644
@@ -26,6 +26,7 @@
 #include "xfs_trace.h"
 #include "xfs_attr_item.h"
 #include "xfs_xattr.h"
+#include "xfs_parent.h"
 
 struct kmem_cache		*xfs_attr_intent_cache;
 
@@ -1577,62 +1578,26 @@ xfs_attr_node_get(
 	return error;
 }
 
-/*
- * Verify parent pointer attribute is valid.
- * Return true on success or false on failure
- */
-STATIC bool
-xfs_verify_pptr(
-	struct xfs_mount			*mp,
-	const struct xfs_parent_name_rec	*rec)
-{
-	xfs_ino_t				p_ino;
-	xfs_dir2_dataptr_t			p_diroffset;
-
-	p_ino = be64_to_cpu(rec->p_ino);
-	p_diroffset = be32_to_cpu(rec->p_diroffset);
-
-	if (!xfs_verify_ino(mp, p_ino))
-		return false;
-
-	if (p_diroffset > XFS_DIR2_MAX_DATAPTR)
-		return false;
-
-	return true;
-}
-
-/* Returns true if the string attribute entry name is valid. */
-static bool
-xfs_str_attr_namecheck(
-	const void	*name,
-	size_t		length)
-{
-	/*
-	 * MAXNAMELEN includes the trailing null, but (name/length) leave it
-	 * out, so use >= for the length check.
-	 */
-	if (length >= MAXNAMELEN)
-		return false;
-
-	/* There shouldn't be any nulls here */
-	return !memchr(name, 0, length);
-}
-
 /* Returns true if the attribute entry name is valid. */
 bool
 xfs_attr_namecheck(
 	struct xfs_mount	*mp,
 	const void		*name,
 	size_t			length,
-	int			flags)
+	unsigned int		flags)
 {
-	if (flags & XFS_ATTR_PARENT) {
-		if (length != sizeof(struct xfs_parent_name_rec))
-			return false;
-		return xfs_verify_pptr(mp, (struct xfs_parent_name_rec *)name);
-	}
+	if (flags & XFS_ATTR_PARENT)
+		return xfs_parent_namecheck(mp, name, length, flags);
 
-	return xfs_str_attr_namecheck(name, length);
+	/*
+	 * MAXNAMELEN includes the trailing null, but (name/length) leave it
+	 * out, so use >= for the length check.
+	 */
+	if (length >= MAXNAMELEN)
+		return false;
+
+	/* There shouldn't be any nulls here */
+	return !memchr(name, 0, length);
 }
 
 int __init
fs/xfs/libxfs/xfs_attr.h +1 -1
index 985761264d1f..d6d23cf19ade 100644
@@ -551,7 +551,7 @@ int xfs_attr_set(struct xfs_da_args *args);
 int xfs_attr_set_iter(struct xfs_attr_intent *attr);
 int xfs_attr_remove_iter(struct xfs_attr_intent *attr);
 bool xfs_attr_namecheck(struct xfs_mount *mp, const void *name, size_t length,
-			int flags);
+		unsigned int flags);
 int xfs_attr_calc_size(struct xfs_da_args *args, int *local);
 void xfs_init_attr_trans(struct xfs_da_args *args, struct xfs_trans_res *tres,
 			 unsigned int *total);
fs/xfs/libxfs/xfs_parent.c +44 -0

50 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +7 -0
index 13040b9d8b08..4ffcb81d399c 100644
@@ -8,6 +8,13 @@
 
 extern struct kmem_cache	*xfs_parent_intent_cache;
 
+/* Metadata validators */
+bool xfs_parent_namecheck(struct xfs_mount *mp,
+		const struct xfs_parent_name_rec *rec, size_t reclen,
+		unsigned int attr_flags);
+bool xfs_parent_valuecheck(struct xfs_mount *mp, const void *value,
+		size_t valuelen);
+
 /*
  * Dynamically allocd structure used to wrap the needed data to pass around
  * the defer ops machinery
#33 lore raw

PATCH 3/3 xfs: don't remove the attr fork when parent pointers are enabled

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When running generic/388, I observed the following .out.bad output:

_check_xfs_filesystem: filesystem on /dev/sda4 is inconsistent (r) *** xfs_repair -n output ***
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
mismatch between format (2) and size (276) in symlink ino 37223730 bad data fork in symlink 37223730
would have cleared inode 37223730
- agno = 2
- agno = 3
mismatch between format (2) and size (276) in symlink ino 102725435 bad data fork in symlink 102725435
would have cleared inode 102725435
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
unknown block state, ag 1, blocks 458655-458655
unknown block state, ag 3, blocks 257772-257772
- check for inodes claiming duplicate blocks...
- agno = 1
- agno = 2
- agno = 3
- agno = 0
mismatch between format (2) and size (276) in symlink ino 102725435 bad data fork in symlink 102725435
would have cleared inode 102725435
mismatch between format (2) and size (276) in symlink ino 37223730 bad data fork in symlink 37223730
would have cleared inode 37223730
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
user quota id 0 has bcount 1140448, expected 1140446 user quota id 0 has icount 39892, expected 39890
No modify flag set, skipping filesystem flush and exiting.

Inode 37223730 is an unlinked remote-format symlink with no xattr fork. According to the inode verifier and xfs_repair, this symlink ought to have a local format data fork, since 276 bytes is small enough to fit in the immediate area.

How did we get here? fsstress removed the symlink, which removed the last parent pointer xattr. There were no other xattrs, so that removal also removed the attr fork. This transaction got flushed to the log, but the system went down before we could inactivate the symlink. Log recovery tried to inactivate this inode (since it is on the unlinked list) but the verifier tripped over the remote value and leaked it.

Hence we ended up with a file in this odd state on a "clean" mount. The "obvious" fix is to prohibit erasure of the attr fork to avoid tripping over the verifiers when pptrs are enabled.

I wonder this could be reproduced with normal xattrs and (say) a directory? Maybe this fix should target /any/ symlink or directory?

---
 fs/xfs/libxfs/xfs_attr_leaf.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +4 -2
index beee51ad75ce..e6c4c8b52a55 100644
@@ -854,7 +854,8 @@ xfs_attr_sf_removename(
 	totsize -= size;
 	if (totsize == sizeof(xfs_attr_sf_hdr_t) && xfs_has_attr2(mp) &&
 	    (dp->i_df.if_format != XFS_DINODE_FMT_BTREE) &&
-	    !(args->op_flags & (XFS_DA_OP_ADDNAME | XFS_DA_OP_REPLACE))) {
+	    !(args->op_flags & (XFS_DA_OP_ADDNAME | XFS_DA_OP_REPLACE)) &&
+	    !xfs_has_parent(mp)) {
 		xfs_attr_fork_remove(dp, args->trans);
 	} else {
 		xfs_idata_realloc(dp, -size, XFS_ATTR_FORK);
@@ -863,7 +864,8 @@ xfs_attr_sf_removename(
 		ASSERT(totsize > sizeof(xfs_attr_sf_hdr_t) ||
 				(args->op_flags & XFS_DA_OP_ADDNAME) ||
 				!xfs_has_attr2(mp) ||
-				dp->i_df.if_format == XFS_DINODE_FMT_BTREE);
+				dp->i_df.if_format == XFS_DINODE_FMT_BTREE ||
+				xfs_has_parent(mp));
 		xfs_trans_log_inode(args->trans, dp,
 					XFS_ILOG_CORE | XFS_ILOG_ADATA);
 	}
#34 lore raw

cover xfs: rework the GETPARENTS ioctl

In reply to Darrick J. Wong

Hi all,

This series fixes a few bugs that I found in the XFS_IOC_GETPARENTS implementation. It also reworks the xfs_attr_list implementations to provide an xattr value pointer when available, and finally it reworks the whole implementation to take advantage of this and use less memory.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-ioctl

---
 fs/xfs/libxfs/xfs_attr.h    |    5 +
 fs/xfs/libxfs/xfs_attr_sf.h |    1 
 fs/xfs/libxfs/xfs_parent.c  |   40 +++++++--
 fs/xfs/libxfs/xfs_parent.h  |   21 ++++-
 fs/xfs/scrub/attr.c         |    8 ++
 fs/xfs/xfs_attr_list.c      |    8 ++
 fs/xfs/xfs_ioctl.c          |   40 ++++++---
 fs/xfs/xfs_parent_utils.c   |  184 ++++++++++++++++++++++++-------------------
 fs/xfs/xfs_parent_utils.h   |    4 -
 fs/xfs/xfs_trace.c          |    1 
 fs/xfs/xfs_trace.h          |   73 +++++++++++++++++
 fs/xfs/xfs_xattr.c          |    1 
 12 files changed, 272 insertions(+), 114 deletions(-)
#35 lore raw

PATCH 1/4 xfs: fix multiple problems when doing getparents by handle

From: Darrick J. Wong <[email protected]>

Fix a few problems in the file handle processing part of GETPARENTS. First, we need to validate that the fsid of the handle matches the filesystem that we're talking to. Second, we can skip the iget if the inode number matches the open file. Third, if we are going to do the iget file, we need to use an UNTRUSTED lookup to guard against crap. Finally, we mustn't leak any inodes that we iget.

---
 fs/xfs/xfs_ioctl.c |   30 +++++++++++++++++++++---------
 1 file changed, 21 insertions(+), 9 deletions(-)
fs/xfs/xfs_ioctl.c +21 -9
index df5a45b97f8f..a1929b08c539 100644
@@ -1694,8 +1694,9 @@ xfs_ioc_get_parent_pointer(
 {
 	struct xfs_pptr_info		*ppi = NULL;
 	int				error = 0;
-	struct xfs_inode		*ip = XFS_I(file_inode(filp));
-	struct xfs_mount		*mp = ip->i_mount;
+	struct xfs_inode		*file_ip = XFS_I(file_inode(filp));
+	struct xfs_inode		*call_ip = file_ip;
+	struct xfs_mount		*mp = file_ip->i_mount;
 
 	if (!capable(CAP_SYS_ADMIN))
 		return -EPERM;
@@ -1733,23 +1734,32 @@ xfs_ioc_get_parent_pointer(
 		return -ENOMEM;
 
 	if (ppi->pi_flags & XFS_PPTR_IFLAG_HANDLE) {
-		error = xfs_iget(mp, NULL, ppi->pi_handle.ha_fid.fid_ino,
-				0, 0, &ip);
-		if (error)
+		struct xfs_handle	*hanp = &ppi->pi_handle;
+
+		if (memcmp(&hanp->ha_fsid, mp->m_fixedfsid,
+							sizeof(xfs_fsid_t))) {
+			error = -EINVAL;
 			goto out;
+		}
 
-		if (VFS_I(ip)->i_generation != ppi->pi_handle.ha_fid.fid_gen) {
+		if (hanp->ha_fid.fid_ino != file_ip->i_ino) {
+			error = xfs_iget(mp, NULL, hanp->ha_fid.fid_ino,
+					XFS_IGET_UNTRUSTED, 0, &call_ip);
+			if (error)
+				goto out;
+		}
+
+		if (VFS_I(call_ip)->i_generation != hanp->ha_fid.fid_gen) {
 			error = -EINVAL;
 			goto out;
 		}
 	}
 
-	if (ip->i_ino == mp->m_sb.sb_rootino)
+	if (call_ip->i_ino == mp->m_sb.sb_rootino)
 		ppi->pi_flags |= XFS_PPTR_OFLAG_ROOT;
 
 	/* Get the parent pointers */
-	error = xfs_attr_get_parent_pointer(ip, ppi);
-
+	error = xfs_attr_get_parent_pointer(call_ip, ppi);
 	if (error)
 		goto out;
 
@@ -1762,6 +1772,8 @@ xfs_ioc_get_parent_pointer(
 	}
 
 out:
+	if (call_ip != file_ip)
+		xfs_irele(call_ip);
 	kmem_free(ppi);
 	return error;
 }
#36 lore raw

PATCH 2/4 xfs: use kvalloc for the parent pointer info buffer

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

It's possible that userspace could call us with large(ish) 64k buffer. Use kvalloc for this, so that the kernel doesn't have to find a contiguous physical region. Zero the realloc buffer so that we don't leak kernel contents to userspace.

---
 fs/xfs/xfs_ioctl.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
fs/xfs/xfs_ioctl.c +5 -3
index a1929b08c539..19f71d6eb561 100644
@@ -1702,7 +1702,7 @@ xfs_ioc_get_parent_pointer(
 		return -EPERM;
 
 	/* Allocate an xfs_pptr_info to put the user data */
-	ppi = kmalloc(sizeof(struct xfs_pptr_info), 0);
+	ppi = kvmalloc(sizeof(struct xfs_pptr_info), GFP_KERNEL);
 	if (!ppi)
 		return -ENOMEM;
 
@@ -1729,7 +1729,9 @@ xfs_ioc_get_parent_pointer(
 	 * Now that we know how big the trailing buffer is, expand
 	 * our kernel xfs_pptr_info to be the same size
 	 */
-	ppi = krealloc(ppi, xfs_pptr_info_sizeof(ppi->pi_ptrs_size), 0);
+	ppi = kvrealloc(ppi, sizeof(struct xfs_pptr_info),
+			xfs_pptr_info_sizeof(ppi->pi_ptrs_size),
+			GFP_KERNEL | __GFP_ZERO);
 	if (!ppi)
 		return -ENOMEM;
 
@@ -1774,7 +1776,7 @@ xfs_ioc_get_parent_pointer(
 out:
 	if (call_ip != file_ip)
 		xfs_irele(call_ip);
-	kmem_free(ppi);
+	kvfree(ppi);
 	return error;
 }
 
#37 lore raw

PATCH 3/4 xfs: pass the attr value to put_listent when possible

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Pass the attr value to put_listent when we have local xattrs or shortform xattrs.

---
 fs/xfs/libxfs/xfs_attr.h    |    5 +++--
 fs/xfs/libxfs/xfs_attr_sf.h |    1 +
 fs/xfs/scrub/attr.c         |    8 ++++++++
 fs/xfs/xfs_attr_list.c      |    8 +++++++-
 fs/xfs/xfs_ioctl.c          |    1 +
 fs/xfs/xfs_xattr.c          |    1 +
 6 files changed, 21 insertions(+), 3 deletions(-)
fs/xfs/libxfs/xfs_attr.h +3 -2
index d6d23cf19ade..02a20b948c8f 100644
@@ -47,8 +47,9 @@ struct xfs_attrlist_cursor_kern {
 
 
 /* void; state communicated via *context */
-typedef void (*put_listent_func_t)(struct xfs_attr_list_context *, int,
-			      unsigned char *, int, int);
+typedef void (*put_listent_func_t)(struct xfs_attr_list_context *context,
+		int flags, unsigned char *name, int namelen, void *value,
+		int valuelen);
 
 struct xfs_attr_list_context {
 	struct xfs_trans	*tp;
fs/xfs/libxfs/xfs_attr_sf.h +1 -0
index 37578b369d9b..c6e259791bc3 100644
@@ -24,6 +24,7 @@ typedef struct xfs_attr_sf_sort {
 	uint8_t		flags;		/* flags bits (see xfs_attr_leaf.h) */
 	xfs_dahash_t	hash;		/* this entry's hash value */
 	unsigned char	*name;		/* name value, pointer into buffer */
+	void		*value;
 } xfs_attr_sf_sort_t;
 
 #define XFS_ATTR_SF_ENTSIZE_MAX			/* max space for name&value */ \
fs/xfs/scrub/attr.c +8 -0
index 2a79a13cb600..00682006d0d3 100644
@@ -109,6 +109,7 @@ xchk_xattr_listent(
 	int				flags,
 	unsigned char			*name,
 	int				namelen,
+	void				*value,
 	int				valuelen)
 {
 	struct xchk_xattr		*sx;
@@ -134,6 +135,13 @@ xchk_xattr_listent(
 		return;
 	}
 
+	/*
+	 * Shortform and local attrs don't require external lookups to retrieve
+	 * the value, so there's nothing else to check here.
+	 */
+	if (value)
+		return;
+
 	/*
 	 * Try to allocate enough memory to extrat the attr value.  If that
 	 * doesn't work, we overload the seen_enough variable to convey
fs/xfs/xfs_attr_list.c +7 -1
index a51f7f13a352..8e3891b96736 100644
@@ -94,6 +94,7 @@ xfs_attr_shortform_list(
 					     sfe->flags,
 					     sfe->nameval,
 					     (int)sfe->namelen,
+					     &sfe->nameval[sfe->namelen],
 					     (int)sfe->valuelen);
 			/*
 			 * Either search callback finished early or
@@ -139,6 +140,7 @@ xfs_attr_shortform_list(
 		sbp->name = sfe->nameval;
 		sbp->namelen = sfe->namelen;
 		/* These are bytes, and both on-disk, don't endian-flip */
+		sbp->value = &sfe->nameval[sfe->namelen],
 		sbp->valuelen = sfe->valuelen;
 		sbp->flags = sfe->flags;
 		sfe = xfs_attr_sf_nextentry(sfe);
@@ -189,6 +191,7 @@ xfs_attr_shortform_list(
 				     sbp->flags,
 				     sbp->name,
 				     sbp->namelen,
+				     sbp->value,
 				     sbp->valuelen);
 		if (context->seen_enough)
 			break;
@@ -443,6 +446,7 @@ xfs_attr3_leaf_list_int(
 	 */
 	for (; i < ichdr.count; entry++, i++) {
 		char *name;
+		void *value;
 		int namelen, valuelen;
 
 		if (be32_to_cpu(entry->hashval) != cursor->hashval) {
@@ -460,6 +464,7 @@ xfs_attr3_leaf_list_int(
 			name_loc = xfs_attr3_leaf_name_local(leaf, i);
 			name = name_loc->nameval;
 			namelen = name_loc->namelen;
+			value = &name_loc->nameval[name_loc->namelen];
 			valuelen = be16_to_cpu(name_loc->valuelen);
 		} else {
 			xfs_attr_leaf_name_remote_t *name_rmt;
@@ -467,6 +472,7 @@ xfs_attr3_leaf_list_int(
 			name_rmt = xfs_attr3_leaf_name_remote(leaf, i);
 			name = name_rmt->name;
 			namelen = name_rmt->namelen;
+			value = NULL;
 			valuelen = be32_to_cpu(name_rmt->valuelen);
 		}
 
@@ -475,7 +481,7 @@ xfs_attr3_leaf_list_int(
 						       entry->flags)))
 			return -EFSCORRUPTED;
 		context->put_listent(context, entry->flags,
-					      name, namelen, valuelen);
+					      name, namelen, value, valuelen);
 		if (context->seen_enough)
 			break;
 		cursor->offset++;
fs/xfs/xfs_ioctl.c +1 -0
index 19f71d6eb561..e6d1e69c6d4a 100644
@@ -308,6 +308,7 @@ xfs_ioc_attr_put_listent(
 	int			flags,
 	unsigned char		*name,
 	int			namelen,
+	void			*value,
 	int			valuelen)
 {
 	struct xfs_attrlist	*alist = context->buffer;
fs/xfs/xfs_xattr.c +1 -0
index ddc2db5d6f73..85edd7e05fde 100644
@@ -227,6 +227,7 @@ xfs_xattr_put_listent(
 	int		flags,
 	unsigned char	*name,
 	int		namelen,
+	void		*value,
 	int		valuelen)
 {
 	char *prefix;
#38 lore raw

PATCH 4/4 xfs: replace the XFS_IOC_GETPARENTS backend

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that xfs_attr_list can pass local xattr values to the put_listent function, build a new version of the GETPARENTS backend that supplies a custom put_listent function to format parent pointer info directly into the caller's buffer. This uses a lot less memory and obviates the iterate list and then grab the values logic, since parent pointers aren't supposed to have remote values anyway.

---
 fs/xfs/libxfs/xfs_parent.c |   40 +++++++---
 fs/xfs/libxfs/xfs_parent.h |   21 +++++
 fs/xfs/xfs_ioctl.c         |    5 -
 fs/xfs/xfs_parent_utils.c  |  184 ++++++++++++++++++++++++--------------------
 fs/xfs/xfs_parent_utils.h  |    4 -
 fs/xfs/xfs_trace.c         |    1 
 fs/xfs/xfs_trace.h         |   73 +++++++++++++++++
 7 files changed, 227 insertions(+), 101 deletions(-)
fs/xfs/libxfs/xfs_parent.c +30 -10
index 8cc264baf6c7..179b9bebaf25 100644
@@ -29,16 +29,6 @@
 
 struct kmem_cache		*xfs_parent_intent_cache;
 
-/* Initializes a xfs_parent_ptr from an xfs_parent_name_rec */
-void
-xfs_init_parent_ptr(struct xfs_parent_ptr		*xpp,
-		    const struct xfs_parent_name_rec	*rec)
-{
-	xpp->xpp_ino = be64_to_cpu(rec->p_ino);
-	xpp->xpp_gen = be32_to_cpu(rec->p_gen);
-	xpp->xpp_diroffset = be32_to_cpu(rec->p_diroffset);
-}
-
 /*
  * Parent pointer attribute handling.
  *
@@ -115,6 +105,36 @@ xfs_init_parent_name_rec(
 	rec->p_diroffset = cpu_to_be32(p_diroffset);
 }
 
+/*
+ * Convert an ondisk parent_name xattr to its incore format.  If @value is
+ * NULL, set @irec->p_namelen to zero and leave @irec->p_name untouched.
+ */
+void
+xfs_parent_irec_from_disk(
+	struct xfs_parent_name_irec	*irec,
+	const struct xfs_parent_name_rec *rec,
+	const void			*value,
+	int				valuelen)
+{
+	irec->p_ino = be64_to_cpu(rec->p_ino);
+	irec->p_gen = be32_to_cpu(rec->p_gen);
+	irec->p_diroffset = be32_to_cpu(rec->p_diroffset);
+
+	if (!value) {
+		irec->p_namelen = 0;
+		return;
+	}
+
+	ASSERT(valuelen > 0);
+	ASSERT(valuelen < MAXNAMELEN);
+
+	valuelen = min(valuelen, MAXNAMELEN);
+
+	irec->p_namelen = valuelen;
+	memcpy(irec->p_name, value, valuelen);
+	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
+}
+
 int
 __xfs_parent_init(
 	struct xfs_mount		*mp,
fs/xfs/libxfs/xfs_parent.h +19 -2
index 4ffcb81d399c..f4f5887d1133 100644
@@ -15,6 +15,25 @@ bool xfs_parent_namecheck(struct xfs_mount *mp,
 bool xfs_parent_valuecheck(struct xfs_mount *mp, const void *value,
 		size_t valuelen);
 
+/*
+ * Incore version of a parent pointer, also contains dirent name so callers
+ * can pass/obtain all the parent pointer information in a single structure
+ */
+struct xfs_parent_name_irec {
+	/* Key fields for looking up a particular parent pointer. */
+	xfs_ino_t		p_ino;
+	uint32_t		p_gen;
+	xfs_dir2_dataptr_t	p_diroffset;
+
+	/* Attributes of a parent pointer. */
+	uint8_t			p_namelen;
+	unsigned char		p_name[MAXNAMELEN];
+};
+
+void xfs_parent_irec_from_disk(struct xfs_parent_name_irec *irec,
+		const struct xfs_parent_name_rec *rec,
+		const void *value, int valuelen);
+
 /*
  * Dynamically allocd structure used to wrap the needed data to pass around
  * the defer ops machinery
@@ -32,8 +51,6 @@ struct xfs_parent_defer {
 void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
 			      struct xfs_inode *ip,
 			      uint32_t p_diroffset);
-void xfs_init_parent_ptr(struct xfs_parent_ptr *xpp,
-			 const struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
fs/xfs/xfs_ioctl.c +1 -4
index e6d1e69c6d4a..4c36ddd19dbd 100644
@@ -1758,11 +1758,8 @@ xfs_ioc_get_parent_pointer(
 		}
 	}
 
-	if (call_ip->i_ino == mp->m_sb.sb_rootino)
-		ppi->pi_flags |= XFS_PPTR_OFLAG_ROOT;
-
 	/* Get the parent pointers */
-	error = xfs_attr_get_parent_pointer(call_ip, ppi);
+	error = xfs_getparent_pointers(call_ip, ppi);
 	if (error)
 		goto out;
 
fs/xfs/xfs_parent_utils.c +101 -83

205 lines not on this page · raw message

fs/xfs/xfs_parent_utils.h +2 -2
index ad60baee8b2a..9936c74e6f96 100644
@@ -6,6 +6,6 @@
 #ifndef	__XFS_PARENT_UTILS_H__
 #define	__XFS_PARENT_UTILS_H__
 
-int xfs_attr_get_parent_pointer(struct xfs_inode *ip,
-				struct xfs_pptr_info *ppi);
+int xfs_getparent_pointers(struct xfs_inode *ip, struct xfs_pptr_info *ppi);
+
 #endif	/* __XFS_PARENT_UTILS_H__ */
fs/xfs/xfs_trace.c +1 -0
index 8a5dc1538aa8..c1f339481697 100644
@@ -36,6 +36,7 @@
 #include "xfs_error.h"
 #include <linux/iomap.h>
 #include "xfs_iomap.h"
+#include "xfs_parent.h"
 
 /*
  * We include this last to have the helpers above available for the trace
fs/xfs/xfs_trace.h +73 -0

85 lines not on this page · raw message

#39 lore raw

cover xfs: online fsck support patches

In reply to Darrick J. Wong

Hi all,

These are all the patches that I needed to backport from the online fsck patchset to start writing online fsck for parent pointers and directories.

IOWS, we're blatantly copying things from the online repair part 1 megaseries; this is what online repair part 2 requires.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-fsck-backports

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-fsck-backports

---
 fs/xfs/Kconfig                 |   38 +++
 fs/xfs/Makefile                |   11 +
 fs/xfs/libxfs/xfs_dir2.c       |    6 
 fs/xfs/libxfs/xfs_dir2.h       |    1 
 fs/xfs/scrub/agheader_repair.c |   99 +++++---
 fs/xfs/scrub/bitmap.c          |  389 ++++++++++++++++++++------------
 fs/xfs/scrub/bitmap.h          |   35 ++-
 fs/xfs/scrub/bmap.c            |    6 
 fs/xfs/scrub/common.c          |  133 ++++++++---
 fs/xfs/scrub/common.h          |   10 +
 fs/xfs/scrub/dir.c             |  233 ++++++-------------
 fs/xfs/scrub/inode.c           |    6 
 fs/xfs/scrub/iscan.c           |  483 ++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/iscan.h           |   65 +++++
 fs/xfs/scrub/listxattr.c       |  314 ++++++++++++++++++++++++++
 fs/xfs/scrub/listxattr.h       |   17 +
 fs/xfs/scrub/parent.c          |  290 ++++++++++--------------
 fs/xfs/scrub/quota.c           |    9 -
 fs/xfs/scrub/readdir.c         |  375 +++++++++++++++++++++++++++++++
 fs/xfs/scrub/readdir.h         |   19 ++
 fs/xfs/scrub/repair.c          |  102 +++++---
 fs/xfs/scrub/rtbitmap.c        |   11 -
 fs/xfs/scrub/scrub.c           |   23 ++
 fs/xfs/scrub/scrub.h           |    7 +
 fs/xfs/scrub/tempfile.c        |  243 ++++++++++++++++++++
 fs/xfs/scrub/tempfile.h        |   29 ++
 fs/xfs/scrub/trace.c           |    5 
 fs/xfs/scrub/trace.h           |  272 +++++++++++++++++++++++
 fs/xfs/scrub/xfarray.c         |  394 +++++++++++++++++++++++++++++++++
 fs/xfs/scrub/xfarray.h         |   60 +++++
 fs/xfs/scrub/xfblob.c          |  176 +++++++++++++++
 fs/xfs/scrub/xfblob.h          |   27 ++
 fs/xfs/scrub/xfile.c           |  329 +++++++++++++++++++++++++++
 fs/xfs/scrub/xfile.h           |   60 +++++
 fs/xfs/xfs_buf.c               |    5 
 fs/xfs/xfs_buf.h               |   10 +
 fs/xfs/xfs_export.c            |    2 
 fs/xfs/xfs_hooks.c             |   94 ++++++++
 fs/xfs/xfs_hooks.h             |   72 ++++++
 fs/xfs/xfs_icache.c            |    3 
 fs/xfs/xfs_icache.h            |   11 +
 fs/xfs/xfs_inode.c             |  229 +++++++++++++++++++
 fs/xfs/xfs_inode.h             |   37 +++
 fs/xfs/xfs_itable.c            |    8 +
 fs/xfs/xfs_linux.h             |    1 
 fs/xfs/xfs_mount.h             |    2 
 fs/xfs/xfs_super.c             |    2 
 fs/xfs/xfs_symlink.c           |    1 
 48 files changed, 4112 insertions(+), 642 deletions(-)
 create mode 100644 fs/xfs/scrub/iscan.c
 create mode 100644 fs/xfs/scrub/iscan.h
 create mode 100644 fs/xfs/scrub/listxattr.c
 create mode 100644 fs/xfs/scrub/listxattr.h
 create mode 100644 fs/xfs/scrub/readdir.c
 create mode 100644 fs/xfs/scrub/readdir.h
 create mode 100644 fs/xfs/scrub/tempfile.c
 create mode 100644 fs/xfs/scrub/tempfile.h
 create mode 100644 fs/xfs/scrub/xfarray.c
 create mode 100644 fs/xfs/scrub/xfarray.h
 create mode 100644 fs/xfs/scrub/xfblob.c
 create mode 100644 fs/xfs/scrub/xfblob.h
 create mode 100644 fs/xfs/scrub/xfile.c
 create mode 100644 fs/xfs/scrub/xfile.h
 create mode 100644 fs/xfs/xfs_hooks.c
 create mode 100644 fs/xfs/xfs_hooks.h
#40 lore raw

PATCH 1/23 xfs: manage inode DONTCACHE status at irele time

From: Darrick J. Wong <[email protected]>

Right now, there are statements scattered all over the online fsck codebase about how we can't use XFS_IGET_DONTCACHE because of concerns about scrub's unusual practice of releasing inodes with transactions held.

However, iget is the wrong place to handle this -- the DONTCACHE state doesn't matter at all until we try to *release* the inode, and here we get things wrong in multiple ways:

First, if we /do/ have a transaction, we must NOT drop the inode, because the inode could have dirty pages, dropping the inode will trigger writeback, and writeback can trigger a nested transaction.

Second, if the inode already had an active reference and the DONTCACHE flag set, the icache hit when scrub grabs another ref will not clear DONTCACHE. This is sort of by design, since DONTCACHE is now used to initiate cache drops so that sysadmins can change a file's access mode between pagecache and DAX.

Third, if we do actually have the last active reference to the inode, we can set DONTCACHE to avoid polluting the cache. This is the /one/ case where we actually want that flag.

Create an xchk_irele helper to encode all that logic and switch the online fsck code to use it. Since this now means that nearly all scrubbers use the same xfs_iget flags, we can wrap them too.

---
 fs/xfs/scrub/common.c |   52 +++++++++++++++++++++++++++++++++++++++++++++----
 fs/xfs/scrub/common.h |    3 +++
 fs/xfs/scrub/dir.c    |    2 +-
 fs/xfs/scrub/parent.c |    9 ++++----
 fs/xfs/scrub/scrub.c  |    2 +-
 5 files changed, 57 insertions(+), 11 deletions(-)
fs/xfs/scrub/common.c +48 -4
index 613260b04a3d..03039d4ade16 100644
@@ -625,6 +625,16 @@ xchk_checkpoint_log(
 	return 0;
 }
 
+/* Verify that an inode is allocated ondisk, then return its cached inode. */
+int
+xchk_iget(
+	struct xfs_scrub	*sc,
+	xfs_ino_t		inum,
+	struct xfs_inode	**ipp)
+{
+	return xfs_iget(sc->mp, sc->tp, inum, XFS_IGET_UNTRUSTED, 0, ipp);
+}
+
 /*
  * Given an inode and the scrub control structure, grab either the
  * inode referenced in the control structure or the inode passed in.
@@ -649,8 +659,7 @@ xchk_get_inode(
 	/* Look up the inode, see if the generation number matches. */
 	if (xfs_internal_inum(mp, sc->sm->sm_ino))
 		return -ENOENT;
-	error = xfs_iget(mp, NULL, sc->sm->sm_ino,
-			XFS_IGET_UNTRUSTED | XFS_IGET_DONTCACHE, 0, &ip);
+	error = xchk_iget(sc, sc->sm->sm_ino, &ip);
 	switch (error) {
 	case -ENOENT:
 		/* Inode doesn't exist, just bail out. */
@@ -672,7 +681,7 @@ xchk_get_inode(
 		 * that it no longer exists.
 		 */
 		error = xfs_imap(sc->mp, sc->tp, sc->sm->sm_ino, &imap,
-				XFS_IGET_UNTRUSTED | XFS_IGET_DONTCACHE);
+				XFS_IGET_UNTRUSTED);
 		if (error)
 			return -ENOENT;
 		error = -EFSCORRUPTED;
@@ -685,7 +694,7 @@ xchk_get_inode(
 		return error;
 	}
 	if (VFS_I(ip)->i_generation != sc->sm->sm_gen) {
-		xfs_irele(ip);
+		xchk_irele(sc, ip);
 		return -ENOENT;
 	}
 
@@ -693,6 +702,41 @@ xchk_get_inode(
 	return 0;
 }
 
+/* Release an inode, possibly dropping it in the process. */
+void
+xchk_irele(
+	struct xfs_scrub	*sc,
+	struct xfs_inode	*ip)
+{
+	if (current->journal_info != NULL) {
+		ASSERT(current->journal_info == sc->tp);
+
+		/*
+		 * If we are in a transaction, we /cannot/ drop the inode
+		 * ourselves, because the VFS will trigger writeback, which
+		 * can require a transaction.  Clear DONTCACHE to force the
+		 * inode to the LRU, where someone else can take care of
+		 * dropping it.
+		 *
+		 * Note that when we grabbed our reference to the inode, it
+		 * could have had an active ref and DONTCACHE set if a sysadmin
+		 * is trying to coerce a change in file access mode.  icache
+		 * hits do not clear DONTCACHE, so we must do it here.
+		 */
+		spin_lock(&VFS_I(ip)->i_lock);
+		VFS_I(ip)->i_state &= ~I_DONTCACHE;
+		spin_unlock(&VFS_I(ip)->i_lock);
+	} else if (atomic_read(&VFS_I(ip)->i_count) == 1) {
+		/*
+		 * If this is the last reference to the inode and the caller
+		 * permits it, set DONTCACHE to avoid thrashing.
+		 */
+		d_mark_dontcache(VFS_I(ip));
+	}
+
+	xfs_irele(ip);
+}
+
 /* Set us up to scrub a file's contents. */
 int
 xchk_setup_inode_contents(
fs/xfs/scrub/common.h +3 -0
index b73648d81d23..1c4525d97939 100644
@@ -136,6 +136,9 @@ int xchk_get_inode(struct xfs_scrub *sc);
 int xchk_setup_inode_contents(struct xfs_scrub *sc, unsigned int resblks);
 void xchk_buffer_recheck(struct xfs_scrub *sc, struct xfs_buf *bp);
 
+int xchk_iget(struct xfs_scrub *sc, xfs_ino_t inum, struct xfs_inode **ipp);
+void xchk_irele(struct xfs_scrub *sc, struct xfs_inode *ip);
+
 /*
  * Don't bother cross-referencing if we already found corruption or cross
  * referencing discrepancies.
fs/xfs/scrub/dir.c +1 -1
index d1b0f23c2c59..677b21c3c865 100644
@@ -86,7 +86,7 @@ xchk_dir_check_ftype(
 			xfs_mode_to_ftype(VFS_I(ip)->i_mode));
 	if (ino_dtype != dtype)
 		xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK, offset);
-	xfs_irele(ip);
+	xchk_irele(sdc->sc, ip);
 out:
 	return error;
 }
fs/xfs/scrub/parent.c +4 -5
index d8dff3fd8053..2696bb49324a 100644
@@ -131,7 +131,6 @@ xchk_parent_validate(
 	xfs_ino_t		dnum,
 	bool			*try_again)
 {
-	struct xfs_mount	*mp = sc->mp;
 	struct xfs_inode	*dp = NULL;
 	xfs_nlink_t		expected_nlink;
 	xfs_nlink_t		nlink;
@@ -168,7 +167,7 @@ xchk_parent_validate(
 	 * -EFSCORRUPTED or -EFSBADCRC then the parent is corrupt which is a
 	 *  cross referencing error.  Any other error is an operational error.
 	 */
-	error = xfs_iget(mp, sc->tp, dnum, XFS_IGET_UNTRUSTED, 0, &dp);
+	error = xchk_iget(sc, dnum, &dp);
 	if (error == -EINVAL || error == -ENOENT) {
 		error = -EFSCORRUPTED;
 		xchk_fblock_process_error(sc, XFS_DATA_FORK, 0, &error);
@@ -236,11 +235,11 @@ xchk_parent_validate(
 
 	/* Drat, parent changed.  Try again! */
 	if (dnum != dp->i_ino) {
-		xfs_irele(dp);
+		xchk_irele(sc, dp);
 		*try_again = true;
 		return 0;
 	}
-	xfs_irele(dp);
+	xchk_irele(sc, dp);
 
 	/*
 	 * '..' didn't change, so check that there was only one entry
@@ -253,7 +252,7 @@ xchk_parent_validate(
 out_unlock:
 	xfs_iunlock(dp, XFS_IOLOCK_SHARED);
 out_rele:
-	xfs_irele(dp);
+	xchk_irele(sc, dp);
 out:
 	return error;
 }
fs/xfs/scrub/scrub.c +1 -1
index 07a7a75f987f..752cb4fbd26f 100644
@@ -166,7 +166,7 @@ xchk_teardown(
 			xfs_iunlock(sc->ip, sc->ilock_flags);
 		if (sc->ip != ip_in &&
 		    !xfs_internal_inum(sc->mp, sc->ip->i_ino))
-			xfs_irele(sc->ip);
+			xchk_irele(sc, sc->ip);
 		sc->ip = NULL;
 	}
 	if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR)
#41 lore raw

PATCH 2/23 xfs: make checking directory dotdot entries more reliable

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

The current directory parent scrubbing code could be tighter in its execution -- instead of bailing out to userspace after a couple of seconds of waiting for the (alleged) parent directory's IOLOCK while refusing to release the child directory's IOLOCK, we could just cycle both locks until we get both or the child process absorbs a fatal signal.

Note that because the usual sequence is to take IOLOCKs before grabbing a transaction, we have to use the _nowait variants on both inodes to avoid an ABBA deadlock. Since parent pointer checking is the only place in scrub that needs this kind of functionality, move it to parent.c as a private function.

Furthermore, if the child directory's parent changes during the lock cycling, we know that the new parent has stamped the correct parent into the dotdot entry, so we can conclude that the parent entry is correct.

This eliminates an entire source of -EDEADLOCK-based "retry harder" scrub executions.

---
 fs/xfs/scrub/common.c |   22 -----
 fs/xfs/scrub/common.h |    1 
 fs/xfs/scrub/parent.c |  203 +++++++++++++++++++++++--------------------------
 3 files changed, 97 insertions(+), 129 deletions(-)
fs/xfs/scrub/common.c +0 -22
index 03039d4ade16..d523cbb2c90b 100644
@@ -908,28 +908,6 @@ xchk_metadata_inode_forks(
 	return 0;
 }
 
-/*
- * Try to lock an inode in violation of the usual locking order rules.  For
- * example, trying to get the IOLOCK while in transaction context, or just
- * plain breaking AG-order or inode-order inode locking rules.  Either way,
- * the only way to avoid an ABBA deadlock is to use trylock and back off if
- * we can't.
- */
-int
-xchk_ilock_inverted(
-	struct xfs_inode	*ip,
-	uint			lock_mode)
-{
-	int			i;
-
-	for (i = 0; i < 20; i++) {
-		if (xfs_ilock_nowait(ip, lock_mode))
-			return 0;
-		delay(1);
-	}
-	return -EDEADLOCK;
-}
-
 /* Pause background reaping of resources. */
 void
 xchk_stop_reaping(
fs/xfs/scrub/common.h +0 -1
index 1c4525d97939..367f754c5cef 100644
@@ -150,7 +150,6 @@ static inline bool xchk_skip_xref(struct xfs_scrub_metadata *sm)
 }
 
 int xchk_metadata_inode_forks(struct xfs_scrub *sc);
-int xchk_ilock_inverted(struct xfs_inode *ip, uint lock_mode);
 void xchk_stop_reaping(struct xfs_scrub *sc);
 void xchk_start_reaping(struct xfs_scrub *sc);
 
fs/xfs/scrub/parent.c +97 -106

291 lines not on this page · raw message

#42 lore raw

PATCH 3/23 xfs: xfs_iget in the directory scrubber needs to use UNTRUSTED

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

In commit 4b80ac64450f, we tried to strengthen the directory scrubber by using the iget call to detect directory entries that point to unallocated inodes. Unfortunately, that commit neglected to pass XFS_IGET_UNTRUSTED to xfs_iget, so we don't check the inode btree first. If the inode number points to something that isn't even an inode cluster, iget will throw corruption errors and return -EFSCORRUPTED, which means that we fail to mark the directory corrupt.

Fixes 4b80ac64450f ("xfs: scrub should mark a directory corrupt if any entries cannot be iget'd")
---
 fs/xfs/scrub/dir.c |   10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
fs/xfs/scrub/dir.c +3 -7
index 677b21c3c865..ec0c73e0eb0c 100644
@@ -59,19 +59,15 @@ xchk_dir_check_ftype(
 	}
 
 	/*
-	 * Grab the inode pointed to by the dirent.  We release the
-	 * inode before we cancel the scrub transaction.  Since we're
-	 * don't know a priori that releasing the inode won't trigger
-	 * eofblocks cleanup (which allocates what would be a nested
-	 * transaction), we can't use DONTCACHE here because DONTCACHE
-	 * inodes can trigger immediate inactive cleanup of the inode.
+	 * Grab the inode pointed to by the dirent.  Use UNTRUSTED here to
+	 * check the allocation status of the inode in the inode btrees.
 	 *
 	 * If _iget returns -EINVAL or -ENOENT then the child inode number is
 	 * garbage and the directory is corrupt.  If the _iget returns
 	 * -EFSCORRUPTED or -EFSBADCRC then the child is corrupt which is a
 	 *  cross referencing error.  Any other error is an operational error.
 	 */
-	error = xfs_iget(mp, sdc->sc->tp, inum, 0, 0, &ip);
+	error = xchk_iget(sdc->sc, inum, &ip);
 	if (error == -EINVAL || error == -ENOENT) {
 		error = -EFSCORRUPTED;
 		xchk_fblock_process_error(sdc->sc, XFS_DATA_FORK, 0, &error);
#43 lore raw

PATCH 4/23 xfs: always check the existence of a dirent's child inode

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When we're scrubbing directory entries, we always need to iget the child inode to make sure that the inode pointer points to a valid inode. The original directory scrub code (commit a5c4) only set us up to do this for ftype=1 filesystems, which is not sufficient; and then commit 4b80 made it worse by exempting the dot and dotdot entries.

Sorta-fixes: a5c46e5e8912 ("xfs: scrub directory metadata")
Sorta-fixes: 4b80ac64450f ("xfs: scrub should mark a directory corrupt if any entries cannot be iget'd")

---
 fs/xfs/scrub/dir.c |   75 ++++++++++++++++++++--------------------------------
 1 file changed, 29 insertions(+), 46 deletions(-)
fs/xfs/scrub/dir.c +29 -46
index ec0c73e0eb0c..8076e7620734 100644
@@ -39,52 +39,28 @@ struct xchk_dir_ctx {
 };
 
 /* Check that an inode's mode matches a given DT_ type. */
-STATIC int
+STATIC void
 xchk_dir_check_ftype(
 	struct xchk_dir_ctx	*sdc,
 	xfs_fileoff_t		offset,
-	xfs_ino_t		inum,
+	struct xfs_inode	*ip,
 	int			dtype)
 {
 	struct xfs_mount	*mp = sdc->sc->mp;
-	struct xfs_inode	*ip;
 	int			ino_dtype;
-	int			error = 0;
 
 	if (!xfs_has_ftype(mp)) {
 		if (dtype != DT_UNKNOWN && dtype != DT_DIR)
 			xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK,
 					offset);
-		goto out;
+		return;
 	}
 
-	/*
-	 * Grab the inode pointed to by the dirent.  Use UNTRUSTED here to
-	 * check the allocation status of the inode in the inode btrees.
-	 *
-	 * If _iget returns -EINVAL or -ENOENT then the child inode number is
-	 * garbage and the directory is corrupt.  If the _iget returns
-	 * -EFSCORRUPTED or -EFSBADCRC then the child is corrupt which is a
-	 *  cross referencing error.  Any other error is an operational error.
-	 */
-	error = xchk_iget(sdc->sc, inum, &ip);
-	if (error == -EINVAL || error == -ENOENT) {
-		error = -EFSCORRUPTED;
-		xchk_fblock_process_error(sdc->sc, XFS_DATA_FORK, 0, &error);
-		goto out;
-	}
-	if (!xchk_fblock_xref_process_error(sdc->sc, XFS_DATA_FORK, offset,
-			&error))
-		goto out;
-
 	/* Convert mode to the DT_* values that dir_emit uses. */
 	ino_dtype = xfs_dir3_get_dtype(mp,
 			xfs_mode_to_ftype(VFS_I(ip)->i_mode));
 	if (ino_dtype != dtype)
 		xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK, offset);
-	xchk_irele(sdc->sc, ip);
-out:
-	return error;
 }
 
 /*
@@ -105,17 +81,17 @@ xchk_dir_actor(
 	unsigned		type)
 {
 	struct xfs_mount	*mp;
+	struct xfs_inode	*dp;
 	struct xfs_inode	*ip;
 	struct xchk_dir_ctx	*sdc;
 	struct xfs_name		xname;
 	xfs_ino_t		lookup_ino;
 	xfs_dablk_t		offset;
-	bool			checked_ftype = false;
 	int			error = 0;
 
 	sdc = container_of(dir_iter, struct xchk_dir_ctx, dir_iter);
-	ip = sdc->sc->ip;
-	mp = ip->i_mount;
+	dp = sdc->sc->ip;
+	mp = dp->i_mount;
 	offset = xfs_dir2_db_to_da(mp->m_dir_geo,
 			xfs_dir2_dataptr_to_db(mp->m_dir_geo, pos));
 
@@ -136,11 +112,7 @@ xchk_dir_actor(
 
 	if (!strncmp(".", name, namelen)) {
 		/* If this is "." then check that the inum matches the dir. */
-		if (xfs_has_ftype(mp) && type != DT_DIR)
-			xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK,
-					offset);
-		checked_ftype = true;
-		if (ino != ip->i_ino)
+		if (ino != dp->i_ino)
 			xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK,
 					offset);
 	} else if (!strncmp("..", name, namelen)) {
@@ -148,11 +120,7 @@ xchk_dir_actor(
 		 * If this is ".." in the root inode, check that the inum
 		 * matches this dir.
 		 */
-		if (xfs_has_ftype(mp) && type != DT_DIR)
-			xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK,
-					offset);
-		checked_ftype = true;
-		if (ip->i_ino == mp->m_sb.sb_rootino && ino != ip->i_ino)
+		if (dp->i_ino == mp->m_sb.sb_rootino && ino != dp->i_ino)
 			xchk_fblock_set_corrupt(sdc->sc, XFS_DATA_FORK,
 					offset);
 	}
@@ -162,7 +130,7 @@ xchk_dir_actor(
 	xname.len = namelen;
 	xname.type = XFS_DIR3_FT_UNKNOWN;
 
-	error = xfs_dir_lookup(sdc->sc->tp, ip, &xname, &lookup_ino, NULL);
+	error = xfs_dir_lookup(sdc->sc->tp, dp, &xname, &lookup_ino, NULL);
 	/* ENOENT means the hash lookup failed and the dir is corrupt */
 	if (error == -ENOENT)
 		error = -EFSCORRUPTED;
@@ -174,12 +142,27 @@ xchk_dir_actor(
 		goto out;
 	}
 
-	/* Verify the file type.  This function absorbs error codes. */
-	if (!checked_ftype) {
-		error = xchk_dir_check_ftype(sdc, offset, lookup_ino, type);
-		if (error)
-			goto out;
+	/*
+	 * Grab the inode pointed to by the dirent.  Use UNTRUSTED here to
+	 * check the allocation status of the inode in the inode btrees.
+	 *
+	 * If _iget returns -EINVAL or -ENOENT then the child inode number is
+	 * garbage and the directory is corrupt.  If the _iget returns
+	 * -EFSCORRUPTED or -EFSBADCRC then the child is corrupt which is a
+	 *  cross referencing error.  Any other error is an operational error.
+	 */
+	error = xchk_iget(sdc->sc, ino, &ip);
+	if (error == -EINVAL || error == -ENOENT) {
+		error = -EFSCORRUPTED;
+		xchk_fblock_process_error(sdc->sc, XFS_DATA_FORK, 0, &error);
+		goto out;
 	}
+	if (!xchk_fblock_xref_process_error(sdc->sc, XFS_DATA_FORK, offset,
+			&error))
+		goto out;
+
+	xchk_dir_check_ftype(sdc, offset, ip, type);
+	xchk_irele(sdc->sc, ip);
 out:
 	/*
 	 * A negative error code returned here is supposed to cause the
#44 lore raw

PATCH 5/23 xfs: remove the for_each_xbitmap_ helpers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Remove the for_each_xbitmap_ macros in favor of proper iterator functions. We'll soon be switching this data structure over to an interval tree implementation, which means that we can't allow callers to modify the bitmap during iteration without telling us.

---
 fs/xfs/scrub/agheader_repair.c |   89 ++++++++++++++++++++---------------
 fs/xfs/scrub/bitmap.c          |   59 +++++++++++++++++++++++
 fs/xfs/scrub/bitmap.h          |   22 ++++++---
 fs/xfs/scrub/repair.c          |  102 ++++++++++++++++++++++------------------
 4 files changed, 179 insertions(+), 93 deletions(-)
fs/xfs/scrub/agheader_repair.c +50 -39
index d75d82151eeb..26bce2f12b09 100644
@@ -486,10 +486,11 @@ xrep_agfl_walk_rmap(
 /* Strike out the blocks that are cross-linked according to the rmapbt. */
 STATIC int
 xrep_agfl_check_extent(
-	struct xrep_agfl	*ra,
 	uint64_t		start,
-	uint64_t		len)
+	uint64_t		len,
+	void			*priv)
 {
+	struct xrep_agfl	*ra = priv;
 	xfs_agblock_t		agbno = XFS_FSB_TO_AGBNO(ra->sc->mp, start);
 	xfs_agblock_t		last_agbno = agbno + len - 1;
 	int			error;
@@ -537,7 +538,6 @@ xrep_agfl_collect_blocks(
 	struct xrep_agfl	ra;
 	struct xfs_mount	*mp = sc->mp;
 	struct xfs_btree_cur	*cur;
-	struct xbitmap_range	*br, *n;
 	int			error;
 
 	ra.sc = sc;
@@ -578,11 +578,7 @@ xrep_agfl_collect_blocks(
 
 	/* Strike out the blocks that are cross-linked. */
 	ra.rmap_cur = xfs_rmapbt_init_cursor(mp, sc->tp, agf_bp, sc->sa.pag);
-	for_each_xbitmap_extent(br, n, agfl_extents) {
-		error = xrep_agfl_check_extent(&ra, br->start, br->len);
-		if (error)
-			break;
-	}
+	error = xbitmap_walk(agfl_extents, xrep_agfl_check_extent, &ra);
 	xfs_btree_del_cursor(ra.rmap_cur, error);
 	if (error)
 		goto out_bmp;
@@ -628,6 +624,43 @@ xrep_agfl_update_agf(
 			XFS_AGF_FLFIRST | XFS_AGF_FLLAST | XFS_AGF_FLCOUNT);
 }
 
+struct xrep_agfl_fill {
+	struct xbitmap		used_extents;
+	struct xfs_scrub	*sc;
+	__be32			*agfl_bno;
+	xfs_agblock_t		flcount;
+	unsigned int		fl_off;
+};
+
+/* Fill the AGFL with whatever blocks are in this extent. */
+static int
+xrep_agfl_fill(
+	uint64_t		start,
+	uint64_t		len,
+	void			*priv)
+{
+	struct xrep_agfl_fill	*af = priv;
+	struct xfs_scrub	*sc = af->sc;
+	xfs_fsblock_t		fsbno = start;
+	int			error;
+
+	while (fsbno < start + len && af->fl_off < af->flcount)
+		af->agfl_bno[af->fl_off++] =
+				cpu_to_be32(XFS_FSB_TO_AGBNO(sc->mp, fsbno++));
+
+	trace_xrep_agfl_insert(sc->mp, sc->sa.pag->pag_agno,
+			XFS_FSB_TO_AGBNO(sc->mp, start), len);
+
+	error = xbitmap_set(&af->used_extents, start, fsbno - 1);
+	if (error)
+		return error;
+
+	if (af->fl_off == af->flcount)
+		return -ECANCELED;
+
+	return 0;
+}
+
 /* Write out a totally new AGFL. */
 STATIC void
 xrep_agfl_init_header(
@@ -636,13 +669,12 @@ xrep_agfl_init_header(
 	struct xbitmap		*agfl_extents,
 	xfs_agblock_t		flcount)
 {
+	struct xrep_agfl_fill	af = {
+		.sc		= sc,
+		.flcount	= flcount,
+	};
 	struct xfs_mount	*mp = sc->mp;
-	__be32			*agfl_bno;
-	struct xbitmap_range	*br;
-	struct xbitmap_range	*n;
 	struct xfs_agfl		*agfl;
-	xfs_agblock_t		agbno;
-	unsigned int		fl_off;
 
 	ASSERT(flcount <= xfs_agfl_size(mp));
 
@@ -661,36 +693,15 @@ xrep_agfl_init_header(
 	 * blocks than fit in the AGFL, they will be freed in a subsequent
 	 * step.
 	 */
-	fl_off = 0;
-	agfl_bno = xfs_buf_to_agfl_bno(agfl_bp);
-	for_each_xbitmap_extent(br, n, agfl_extents) {
-		agbno = XFS_FSB_TO_AGBNO(mp, br->start);
-
-		trace_xrep_agfl_insert(mp, sc->sa.pag->pag_agno, agbno,
-				br->len);
-
-		while (br->len > 0 && fl_off < flcount) {
-			agfl_bno[fl_off] = cpu_to_be32(agbno);
-			fl_off++;
-			agbno++;
-
-			/*
-			 * We've now used br->start by putting it in the AGFL,
-			 * so bump br so that we don't reap the block later.
-			 */
-			br->start++;
-			br->len--;
-		}
-
-		if (br->len)
-			break;
-		list_del(&br->list);
-		kfree(br);
-	}
+	xbitmap_init(&af.used_extents);
+	af.agfl_bno = xfs_buf_to_agfl_bno(agfl_bp),
+	xbitmap_walk(agfl_extents, xrep_agfl_fill, &af);
+	xbitmap_disunion(agfl_extents, &af.used_extents);
 
 	/* Write new AGFL to disk. */
 	xfs_trans_buf_set_type(sc->tp, agfl_bp, XFS_BLFT_AGFL_BUF);
 	xfs_trans_log_buf(sc->tp, agfl_bp, 0, BBTOB(agfl_bp->b_length) - 1);
+	xbitmap_destroy(&af.used_extents);
 }
 
 /* Repair the AGFL. */
fs/xfs/scrub/bitmap.c +59 -0

68 lines not on this page · raw message

fs/xfs/scrub/bitmap.h +15 -7
index 900646b72de1..53601d281ffb 100644
@@ -19,13 +19,6 @@ struct xbitmap {
 void xbitmap_init(struct xbitmap *bitmap);
 void xbitmap_destroy(struct xbitmap *bitmap);
 
-#define for_each_xbitmap_extent(bex, n, bitmap) \
-	list_for_each_entry_safe((bex), (n), &(bitmap)->list, list)
-
-#define for_each_xbitmap_block(b, bex, n, bitmap) \
-	list_for_each_entry_safe((bex), (n), &(bitmap)->list, list) \
-		for ((b) = (bex)->start; (b) < (bex)->start + (bex)->len; (b)++)
-
 int xbitmap_set(struct xbitmap *bitmap, uint64_t start, uint64_t len);
 int xbitmap_disunion(struct xbitmap *bitmap, struct xbitmap *sub);
 int xbitmap_set_btcur_path(struct xbitmap *bitmap,
@@ -34,4 +27,19 @@ int xbitmap_set_btblocks(struct xbitmap *bitmap,
 		struct xfs_btree_cur *cur);
 uint64_t xbitmap_hweight(struct xbitmap *bitmap);
 
+/*
+ * Return codes for the bitmap iterator functions are 0 to continue iterating,
+ * and non-zero to stop iterating.  Any non-zero value will be passed up to the
+ * iteration caller.  The special value -ECANCELED can be used to stop
+ * iteration, because neither bitmap iterator ever generates that error code on
+ * its own.  Callers must not modify the bitmap while walking it.
+ */
+typedef int (*xbitmap_walk_fn)(uint64_t start, uint64_t len, void *priv);
+int xbitmap_walk(struct xbitmap *bitmap, xbitmap_walk_fn fn,
+		void *priv);
+
+typedef int (*xbitmap_walk_bits_fn)(uint64_t bit, void *priv);
+int xbitmap_walk_bits(struct xbitmap *bitmap, xbitmap_walk_bits_fn fn,
+		void *priv);
+
 #endif	/* __XFS_SCRUB_BITMAP_H__ */
fs/xfs/scrub/repair.c +55 -47

162 lines not on this page · raw message

#45 lore raw

PATCH 6/23 xfs: drop the _safe behavior from the xbitmap foreach macro

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

It's not safe to edit bitmap intervals while we're iterating them with for_each_xbitmap_extent. None of the existing callers actually need that ability anyway, so drop the safe variable.

---
 fs/xfs/scrub/bitmap.c |   17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
fs/xfs/scrub/bitmap.c +8 -9
index d32ded56da90..f8ebc4d61462 100644
@@ -13,8 +13,9 @@
 #include "scrub/scrub.h"
 #include "scrub/bitmap.h"
 
-#define for_each_xbitmap_extent(bex, n, bitmap) \
-	list_for_each_entry_safe((bex), (n), &(bitmap)->list, list)
+/* Iterate each interval of a bitmap.  Do not change the bitmap. */
+#define for_each_xbitmap_extent(bex, bitmap) \
+	list_for_each_entry((bex), &(bitmap)->list, list)
 
 /*
  * Set a range of this bitmap.  Caller must ensure the range is not set.
@@ -46,10 +47,9 @@ void
 xbitmap_destroy(
 	struct xbitmap		*bitmap)
 {
-	struct xbitmap_range	*bmr;
-	struct xbitmap_range	*n;
+	struct xbitmap_range	*bmr, *n;
 
-	for_each_xbitmap_extent(bmr, n, bitmap) {
+	list_for_each_entry_safe(bmr, n, &bitmap->list, list) {
 		list_del(&bmr->list);
 		kfree(bmr);
 	}
@@ -308,10 +308,9 @@ xbitmap_hweight(
 	struct xbitmap		*bitmap)
 {
 	struct xbitmap_range	*bmr;
-	struct xbitmap_range	*n;
 	uint64_t		ret = 0;
 
-	for_each_xbitmap_extent(bmr, n, bitmap)
+	for_each_xbitmap_extent(bmr, bitmap)
 		ret += bmr->len;
 
 	return ret;
@@ -324,10 +323,10 @@ xbitmap_walk(
 	xbitmap_walk_fn	fn,
 	void			*priv)
 {
-	struct xbitmap_range	*bex, *n;
+	struct xbitmap_range	*bex;
 	int			error = 0;
 
-	for_each_xbitmap_extent(bex, n, bitmap) {
+	for_each_xbitmap_extent(bex, bitmap) {
 		error = fn(bex->start, bex->len, priv);
 		if (error)
 			break;
#46 lore raw

PATCH 7/23 xfs: convert xbitmap to interval tree

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Convert the xbitmap code to use interval trees instead of linked lists. This reduces the amount of coding required to handle the disunion operation and in the future will make it easier to set bits in arbitrary order yet later be able to extract maximally sized extents, which we'll need for rebuilding certain structures. We define our own interval tree type so that it can deal with 64-bit indices even on 32-bit machines.

---
 fs/xfs/scrub/agheader_repair.c |   12 +
 fs/xfs/scrub/bitmap.c          |  323 ++++++++++++++++++++++------------------
 fs/xfs/scrub/bitmap.h          |   11 -
 3 files changed, 187 insertions(+), 159 deletions(-)
fs/xfs/scrub/agheader_repair.c +9 -3
index 26bce2f12b09..c22dc71fdd82 100644
@@ -662,7 +662,7 @@ xrep_agfl_fill(
 }
 
 /* Write out a totally new AGFL. */
-STATIC void
+STATIC int
 xrep_agfl_init_header(
 	struct xfs_scrub	*sc,
 	struct xfs_buf		*agfl_bp,
@@ -675,6 +675,7 @@ xrep_agfl_init_header(
 	};
 	struct xfs_mount	*mp = sc->mp;
 	struct xfs_agfl		*agfl;
+	int			error;
 
 	ASSERT(flcount <= xfs_agfl_size(mp));
 
@@ -696,12 +697,15 @@ xrep_agfl_init_header(
 	xbitmap_init(&af.used_extents);
 	af.agfl_bno = xfs_buf_to_agfl_bno(agfl_bp),
 	xbitmap_walk(agfl_extents, xrep_agfl_fill, &af);
-	xbitmap_disunion(agfl_extents, &af.used_extents);
+	error = xbitmap_disunion(agfl_extents, &af.used_extents);
+	if (error)
+		return error;
 
 	/* Write new AGFL to disk. */
 	xfs_trans_buf_set_type(sc->tp, agfl_bp, XFS_BLFT_AGFL_BUF);
 	xfs_trans_log_buf(sc->tp, agfl_bp, 0, BBTOB(agfl_bp->b_length) - 1);
 	xbitmap_destroy(&af.used_extents);
+	return 0;
 }
 
 /* Repair the AGFL. */
@@ -754,7 +758,9 @@ xrep_agfl(
 	 * buffers until we know that part works.
 	 */
 	xrep_agfl_update_agf(sc, agf_bp, flcount);
-	xrep_agfl_init_header(sc, agfl_bp, &agfl_extents, flcount);
+	error = xrep_agfl_init_header(sc, agfl_bp, &agfl_extents, flcount);
+	if (error)
+		goto err;
 
 	/*
 	 * Ok, the AGFL should be ready to go now.  Roll the transaction to
fs/xfs/scrub/bitmap.c +174 -149

389 lines not on this page · raw message

fs/xfs/scrub/bitmap.h +4 -7
index 53601d281ffb..7afd64a318d1 100644
@@ -6,19 +6,14 @@
 #ifndef __XFS_SCRUB_BITMAP_H__
 #define __XFS_SCRUB_BITMAP_H__
 
-struct xbitmap_range {
-	struct list_head	list;
-	uint64_t		start;
-	uint64_t		len;
-};
-
 struct xbitmap {
-	struct list_head	list;
+	struct rb_root_cached	xb_root;
 };
 
 void xbitmap_init(struct xbitmap *bitmap);
 void xbitmap_destroy(struct xbitmap *bitmap);
 
+int xbitmap_clear(struct xbitmap *bitmap, uint64_t start, uint64_t len);
 int xbitmap_set(struct xbitmap *bitmap, uint64_t start, uint64_t len);
 int xbitmap_disunion(struct xbitmap *bitmap, struct xbitmap *sub);
 int xbitmap_set_btcur_path(struct xbitmap *bitmap,
@@ -42,4 +37,6 @@ typedef int (*xbitmap_walk_bits_fn)(uint64_t bit, void *priv);
 int xbitmap_walk_bits(struct xbitmap *bitmap, xbitmap_walk_bits_fn fn,
 		void *priv);
 
+bool xbitmap_empty(struct xbitmap *bitmap);
+
 #endif	/* __XFS_SCRUB_BITMAP_H__ */
#47 lore raw

PATCH 8/23 xfs: port xbitmap_test

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

---
 fs/xfs/Makefile       |    2 +-
 fs/xfs/scrub/bitmap.c |   22 ++++++++++++++++++++++
 fs/xfs/scrub/bitmap.h |    2 ++
 3 files changed, 25 insertions(+), 1 deletion(-)
fs/xfs/Makefile +1 -1
index 42d0496fdad7..2de5a71a2fa3 100644
@@ -147,6 +147,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   agheader.o \
 				   alloc.o \
 				   attr.o \
+				   bitmap.o \
 				   bmap.o \
 				   btree.o \
 				   common.o \
@@ -170,7 +171,6 @@ xfs-$(CONFIG_XFS_QUOTA)		+= scrub/quota.o
 ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
-				   bitmap.o \
 				   repair.o \
 				   )
 endif
fs/xfs/scrub/bitmap.c +22 -0
index 1b04d2ce020a..14caff0a28ce 100644
@@ -396,3 +396,25 @@ xbitmap_empty(
 {
 	return bitmap->xb_root.rb_root.rb_node == NULL;
 }
+
+/* Is the start of the range set or clear?  And for how long? */
+bool
+xbitmap_test(
+	struct xbitmap		*bitmap,
+	uint64_t		start,
+	uint64_t		*len)
+{
+	struct xbitmap_node	*bn;
+	uint64_t		last = start + *len - 1;
+
+	bn = xbitmap_tree_iter_first(&bitmap->xb_root, start, last);
+	if (!bn)
+		return false;
+	if (bn->bn_start <= start) {
+		if (bn->bn_last < last)
+			*len = bn->bn_last - start + 1;
+		return true;
+	}
+	*len = bn->bn_start - start;
+	return false;
+}
fs/xfs/scrub/bitmap.h +2 -0
index 7afd64a318d1..dd492798b7af 100644
@@ -39,4 +39,6 @@ int xbitmap_walk_bits(struct xbitmap *bitmap, xbitmap_walk_bits_fn fn,
 
 bool xbitmap_empty(struct xbitmap *bitmap);
 
+bool xbitmap_test(struct xbitmap *bitmap, uint64_t start, uint64_t *len);
+
 #endif	/* __XFS_SCRUB_BITMAP_H__ */
#48 lore raw

PATCH 9/23 xfs: ignore stale buffers when scanning the buffer cache

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

After an online repair, we need to invalidate buffers representing the blocks from the old metadata that we're replacing. It's possible that parts of a tree that were previously cached in memory are no longer accessible due to media failure or other corruption on interior nodes, so repair figures out the old blocks from the reverse mapping data and scans the buffer cache directly.

Unfortunately, the current buffer cache code triggers asserts if the rhashtable lookup finds a non-stale buffer of a different length than the key we searched for. For regular operation this is desirable, but for this repair procedure, we don't care since we're going to forcibly stale the buffer anyway. Add an internal lookup flag to avoid the assert.

---
 fs/xfs/xfs_buf.c |    5 ++++-
 fs/xfs/xfs_buf.h |   10 ++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)
fs/xfs/xfs_buf.c +4 -1
index 54c774af6e1c..a538501b652b 100644
@@ -482,7 +482,8 @@ _xfs_buf_obj_cmp(
 		 * reallocating a busy extent. Skip this buffer and
 		 * continue searching for an exact match.
 		 */
-		ASSERT(bp->b_flags & XBF_STALE);
+		if (!(map->bm_flags & XBM_IGNORE_LENGTH_MISMATCH))
+			ASSERT(bp->b_flags & XBF_STALE);
 		return 1;
 	}
 	return 0;
@@ -683,6 +684,8 @@ xfs_buf_get_map(
 	int			error;
 	int			i;
 
+	if (flags & XBF_BCACHE_SCAN)
+		cmap.bm_flags |= XBM_IGNORE_LENGTH_MISMATCH;
 	for (i = 0; i < nmaps; i++)
 		cmap.bm_len += map[i].bm_len;
 
fs/xfs/xfs_buf.h +10 -0
index 549c60942208..d6e8c3bab9f6 100644
@@ -44,6 +44,11 @@ struct xfs_buf;
 #define _XBF_DELWRI_Q	 (1u << 22)/* buffer on a delwri queue */
 
 /* flags used only as arguments to access routines */
+/*
+ * We're scanning the buffer cache; do not warn about lookup mismatches.
+ * Only online repair should use this.
+ */
+#define XBF_BCACHE_SCAN	 (1u << 28)
 #define XBF_INCORE	 (1u << 29)/* lookup only, return if found in cache */
 #define XBF_TRYLOCK	 (1u << 30)/* lock requested, but do not wait */
 #define XBF_UNMAPPED	 (1u << 31)/* do not map the buffer */
@@ -67,6 +72,7 @@ typedef unsigned int xfs_buf_flags_t;
 	{ _XBF_KMEM,		"KMEM" }, \
 	{ _XBF_DELWRI_Q,	"DELWRI_Q" }, \
 	/* The following interface flags should never be set */ \
+	{ XBF_BCACHE_SCAN,	"BCACHE_SCAN" }, \
 	{ XBF_INCORE,		"INCORE" }, \
 	{ XBF_TRYLOCK,		"TRYLOCK" }, \
 	{ XBF_UNMAPPED,		"UNMAPPED" }
@@ -114,8 +120,12 @@ typedef struct xfs_buftarg {
 struct xfs_buf_map {
 	xfs_daddr_t		bm_bn;	/* block number for I/O */
 	int			bm_len;	/* size of I/O */
+	unsigned int		bm_flags;
 };
 
+/* Don't complain about live buffers with the wrong length during lookup. */
+#define XBM_IGNORE_LENGTH_MISMATCH	(1U << 0)
+
 #define DEFINE_SINGLE_BUF_MAP(map, blkno, numblk) \
 	struct xfs_buf_map (map) = { .bm_bn = (blkno), .bm_len = (numblk) };
 
#49 lore raw

PATCH 10/23 xfs: create a big array data structure

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a simple 'big array' data structure for storage of fixed-size metadata records that will be used to reconstruct a btree index. For repair operations, the most important operations are append, iterate, and sort.

Earlier implementations of the big array used linked lists and suffered from severe problems -- pinning all records in kernel memory was not a good idea and frequently lead to OOM situations; random access was very inefficient; and record overhead for the lists was unacceptably high at 40-60%.

Therefore, the big memory array relies on the 'xfile' abstraction, which creates a memfd file and stores the records in page cache pages. Since the memfd is created in tmpfs, the memory pages can be pushed out to disk if necessary and we have a built-in usage limit of 50% of physical memory.

---
 fs/xfs/Kconfig         |    1 
 fs/xfs/Makefile        |    2 
 fs/xfs/scrub/trace.c   |    4 -
 fs/xfs/scrub/trace.h   |  123 ++++++++++++++++
 fs/xfs/scrub/xfarray.c |  370 ++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/xfarray.h |   58 ++++++++
 fs/xfs/scrub/xfile.c   |  318 +++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/xfile.h   |   58 ++++++++
 8 files changed, 933 insertions(+), 1 deletion(-)
 create mode 100644 fs/xfs/scrub/xfarray.c
 create mode 100644 fs/xfs/scrub/xfarray.h
 create mode 100644 fs/xfs/scrub/xfile.c
 create mode 100644 fs/xfs/scrub/xfile.h
fs/xfs/Kconfig +1 -0
index 9fac5ea8d0e4..7f12b40146b3 100644
@@ -97,6 +97,7 @@ config XFS_ONLINE_SCRUB
 	bool "XFS online metadata check support"
 	default n
 	depends on XFS_FS
+	depends on TMPFS && SHMEM
 	help
 	  If you say Y here you will be able to check metadata on a
 	  mounted XFS filesystem.  This feature is intended to reduce
fs/xfs/Makefile +2 -0
index 2de5a71a2fa3..b55b8ece7627 100644
@@ -162,6 +162,8 @@ xfs-y				+= $(addprefix scrub/, \
 				   rmap.o \
 				   scrub.o \
 				   symlink.o \
+				   xfarray.o \
+				   xfile.o \
 				   )
 
 xfs-$(CONFIG_XFS_RT)		+= scrub/rtbitmap.o
fs/xfs/scrub/trace.c +3 -1
index b5f94676c37c..4a0385c97ea6 100644
@@ -12,8 +12,10 @@
 #include "xfs_mount.h"
 #include "xfs_inode.h"
 #include "xfs_btree.h"
-#include "scrub/scrub.h"
 #include "xfs_ag.h"
+#include "scrub/scrub.h"
+#include "scrub/xfile.h"
+#include "scrub/xfarray.h"
 
 /* Figure out which block the btree cursor was pointing to. */
 static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +123 -0

135 lines not on this page · raw message

fs/xfs/scrub/xfarray.c new file +370 -0

370 lines not on this page · raw message

fs/xfs/scrub/xfarray.h new file +58 -0

58 lines not on this page · raw message

fs/xfs/scrub/xfile.c new file +318 -0

318 lines not on this page · raw message

fs/xfs/scrub/xfile.h new file +58 -0

58 lines not on this page · raw message

#50 lore raw

PATCH 11/23 xfs: wrap ilock/iunlock operations on sc->ip

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Scrub tracks the resources that it's holding onto in the xfs_scrub structure. This includes the inode being checked (if applicable) and the inode lock state of that inode. Replace the open-coded structure manipulation with a trivial helper to eliminate sources of error.

---
 fs/xfs/scrub/bmap.c     |    6 ++----
 fs/xfs/scrub/common.c   |   38 +++++++++++++++++++++++++++++++++-----
 fs/xfs/scrub/common.h   |    4 ++++
 fs/xfs/scrub/dir.c      |    3 +--
 fs/xfs/scrub/inode.c    |    6 ++----
 fs/xfs/scrub/parent.c   |   10 +++-------
 fs/xfs/scrub/quota.c    |    9 +++------
 fs/xfs/scrub/rtbitmap.c |   11 +++++------
 fs/xfs/scrub/scrub.c    |    2 +-
 9 files changed, 54 insertions(+), 35 deletions(-)
fs/xfs/scrub/bmap.c +2 -4
index d50d0eab196a..bab3c5db437b 100644
@@ -35,8 +35,7 @@ xchk_setup_inode_bmap(
 	if (error)
 		goto out;
 
-	sc->ilock_flags = XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL;
-	xfs_ilock(sc->ip, sc->ilock_flags);
+	xchk_ilock(sc, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL);
 
 	/*
 	 * We don't want any ephemeral data fork updates sitting around
@@ -73,9 +72,8 @@ xchk_setup_inode_bmap(
 	error = xchk_trans_alloc(sc, 0);
 	if (error)
 		goto out;
-	sc->ilock_flags |= XFS_ILOCK_EXCL;
-	xfs_ilock(sc->ip, XFS_ILOCK_EXCL);
 
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
 out:
 	/* scrub teardown will unlock and release the inode */
 	return error;
fs/xfs/scrub/common.c +33 -5
index d523cbb2c90b..dc78e28a9447 100644
@@ -750,19 +750,47 @@ xchk_setup_inode_contents(
 		return error;
 
 	/* Got the inode, lock it and we're ready to go. */
-	sc->ilock_flags = XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL;
-	xfs_ilock(sc->ip, sc->ilock_flags);
+	xchk_ilock(sc, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL);
 	error = xchk_trans_alloc(sc, resblks);
 	if (error)
 		goto out;
-	sc->ilock_flags |= XFS_ILOCK_EXCL;
-	xfs_ilock(sc->ip, XFS_ILOCK_EXCL);
-
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
 out:
 	/* scrub teardown will unlock and release the inode for us */
 	return error;
 }
 
+void
+xchk_ilock(
+	struct xfs_scrub	*sc,
+	unsigned int		ilock_flags)
+{
+	sc->ilock_flags |= ilock_flags;
+	xfs_ilock(sc->ip, ilock_flags);
+}
+
+bool
+xchk_ilock_nowait(
+	struct xfs_scrub	*sc,
+	unsigned int		ilock_flags)
+{
+	if (xfs_ilock_nowait(sc->ip, ilock_flags)) {
+		sc->ilock_flags |= ilock_flags;
+		return true;
+	}
+
+	return false;
+}
+
+void
+xchk_iunlock(
+	struct xfs_scrub	*sc,
+	unsigned int		ilock_flags)
+{
+	xfs_iunlock(sc->ip, ilock_flags);
+	sc->ilock_flags &= ~ilock_flags;
+}
+
 /*
  * Predicate that decides if we need to evaluate the cross-reference check.
  * If there was an error accessing the cross-reference btree, just delete
fs/xfs/scrub/common.h +4 -0
index 367f754c5cef..5286c263ff60 100644
@@ -139,6 +139,10 @@ void xchk_buffer_recheck(struct xfs_scrub *sc, struct xfs_buf *bp);
 int xchk_iget(struct xfs_scrub *sc, xfs_ino_t inum, struct xfs_inode **ipp);
 void xchk_irele(struct xfs_scrub *sc, struct xfs_inode *ip);
 
+void xchk_ilock(struct xfs_scrub *sc, unsigned int ilock_flags);
+bool xchk_ilock_nowait(struct xfs_scrub *sc, unsigned int ilock_flags);
+void xchk_iunlock(struct xfs_scrub *sc, unsigned int ilock_flags);
+
 /*
  * Don't bother cross-referencing if we already found corruption or cross
  * referencing discrepancies.
fs/xfs/scrub/dir.c +1 -2
index 8076e7620734..2a3107cc8ccb 100644
@@ -840,8 +840,7 @@ xchk_directory(
 	 * _dir_lookup routines, which do their own ILOCK locking.
 	 */
 	oldpos = 0;
-	sc->ilock_flags &= ~XFS_ILOCK_EXCL;
-	xfs_iunlock(sc->ip, XFS_ILOCK_EXCL);
+	xchk_iunlock(sc, XFS_ILOCK_EXCL);
 	while (true) {
 		error = xfs_readdir(sc->tp, sc->ip, &sdc.dir_iter, bufsize);
 		if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, 0,
fs/xfs/scrub/inode.c +2 -4
index 7a2f38e5202c..a0ee3ce35ed9 100644
@@ -48,13 +48,11 @@ xchk_setup_inode(
 	}
 
 	/* Got the inode, lock it and we're ready to go. */
-	sc->ilock_flags = XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL;
-	xfs_ilock(sc->ip, sc->ilock_flags);
+	xchk_ilock(sc, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL);
 	error = xchk_trans_alloc(sc, 0);
 	if (error)
 		goto out;
-	sc->ilock_flags |= XFS_ILOCK_EXCL;
-	xfs_ilock(sc->ip, XFS_ILOCK_EXCL);
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
 
 out:
 	/* scrub teardown will unlock and release the inode for us */
fs/xfs/scrub/parent.c +3 -7
index 0c23fd49716b..8581a21bfbfd 100644
@@ -137,8 +137,7 @@ xchk_parent_lock_two_dirs(
 		return -EINVAL;
 	}
 
-	xfs_iunlock(sc->ip, sc->ilock_flags);
-	sc->ilock_flags = 0;
+	xchk_iunlock(sc, sc->ilock_flags);
 	while (true) {
 		if (xchk_should_terminate(sc, &error))
 			return error;
@@ -149,10 +148,8 @@ xchk_parent_lock_two_dirs(
 		 * on either IOLOCK.
 		 */
 		if (xfs_ilock_nowait(dp, XFS_IOLOCK_SHARED)) {
-			if (xfs_ilock_nowait(sc->ip, XFS_IOLOCK_EXCL)) {
-				sc->ilock_flags = XFS_IOLOCK_EXCL;
+			if (xchk_ilock_nowait(sc, XFS_IOLOCK_EXCL))
 				break;
-			}
 			xfs_iunlock(dp, XFS_IOLOCK_SHARED);
 		}
 
@@ -299,8 +296,7 @@ xchk_parent(
 	 * getting a write lock on i_rwsem.  Therefore, it is safe for us
 	 * to drop the ILOCK here in order to do directory lookups.
 	 */
-	sc->ilock_flags &= ~(XFS_ILOCK_EXCL | XFS_MMAPLOCK_EXCL);
-	xfs_iunlock(sc->ip, XFS_ILOCK_EXCL | XFS_MMAPLOCK_EXCL);
+	xchk_iunlock(sc, XFS_ILOCK_EXCL | XFS_MMAPLOCK_EXCL);
 
 	/* Look up '..' */
 	error = xfs_dir_lookup(sc->tp, sc->ip, &xfs_name_dotdot, &parent_ino,
fs/xfs/scrub/quota.c +3 -6
index 9eeac8565394..d3db47a3dafb 100644
@@ -57,8 +57,7 @@ xchk_setup_quota(
 	if (error)
 		return error;
 	sc->ip = xfs_quota_inode(sc->mp, dqtype);
-	xfs_ilock(sc->ip, XFS_ILOCK_EXCL);
-	sc->ilock_flags = XFS_ILOCK_EXCL;
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	return 0;
 }
 
@@ -232,13 +231,11 @@ xchk_quota(
 	 * data fork we have to drop ILOCK_EXCL to use the regular dquot
 	 * functions.
 	 */
-	xfs_iunlock(sc->ip, sc->ilock_flags);
-	sc->ilock_flags = 0;
+	xchk_iunlock(sc, sc->ilock_flags);
 	sqi.sc = sc;
 	sqi.last_id = 0;
 	error = xfs_qm_dqiterate(mp, dqtype, xchk_quota_item, &sqi);
-	sc->ilock_flags = XFS_ILOCK_EXCL;
-	xfs_ilock(sc->ip, sc->ilock_flags);
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,
fs/xfs/scrub/rtbitmap.c +5 -6
index 0a3bde64c675..340964a35963 100644
@@ -28,10 +28,9 @@ xchk_setup_rt(
 	if (error)
 		return error;
 
-	sc->ilock_flags = XFS_ILOCK_EXCL | XFS_ILOCK_RTBITMAP;
 	sc->ip = sc->mp->m_rbmip;
-	xfs_ilock(sc->ip, sc->ilock_flags);
-
+	sc->ilock_flags = 0;
+	xchk_ilock(sc, XFS_ILOCK_EXCL | XFS_ILOCK_RTBITMAP);
 	return 0;
 }
 
@@ -141,8 +140,8 @@ xchk_rtsummary(
 	 * flags so that we don't mix up the inode state that @sc tracks.
 	 */
 	sc->ip = rsumip;
-	sc->ilock_flags = XFS_ILOCK_EXCL | XFS_ILOCK_RTSUM;
-	xfs_ilock(sc->ip, sc->ilock_flags);
+	sc->ilock_flags = 0;
+	xchk_ilock(sc, XFS_ILOCK_EXCL | XFS_ILOCK_RTSUM);
 
 	/* Invoke the fork scrubber. */
 	error = xchk_metadata_inode_forks(sc);
@@ -153,7 +152,7 @@ xchk_rtsummary(
 	xchk_set_incomplete(sc);
 out:
 	/* Switch back to the rtbitmap inode and lock flags. */
-	xfs_iunlock(sc->ip, sc->ilock_flags);
+	xchk_iunlock(sc, XFS_ILOCK_EXCL | XFS_ILOCK_RTSUM);
 	sc->ilock_flags = old_ilock_flags;
 	sc->ip = old_ip;
 	return error;
fs/xfs/scrub/scrub.c +1 -1
index 752cb4fbd26f..6aedce9b67fc 100644
@@ -163,7 +163,7 @@ xchk_teardown(
 	}
 	if (sc->ip) {
 		if (sc->ilock_flags)
-			xfs_iunlock(sc->ip, sc->ilock_flags);
+			xchk_iunlock(sc, sc->ilock_flags);
 		if (sc->ip != ip_in &&
 		    !xfs_internal_inum(sc->mp, sc->ip->i_ino))
 			xchk_irele(sc, sc->ip);
#51 lore raw

PATCH 12/23 xfs: port scrub inode scan from djwong-dev

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Port the inode scanning code from djwong-dev so we can use it here.

---
 fs/xfs/Makefile      |    1 
 fs/xfs/scrub/iscan.c |  483 ++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/iscan.h |   65 +++++++
 fs/xfs/scrub/trace.c |    1 
 fs/xfs/scrub/trace.h |   73 ++++++++
 fs/xfs/xfs_icache.c  |    3 
 fs/xfs/xfs_icache.h  |   11 +
 7 files changed, 632 insertions(+), 5 deletions(-)
 create mode 100644 fs/xfs/scrub/iscan.c
 create mode 100644 fs/xfs/scrub/iscan.h
fs/xfs/Makefile +1 -0
index b55b8ece7627..69805b4ad79f 100644
@@ -157,6 +157,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   health.o \
 				   ialloc.o \
 				   inode.o \
+				   iscan.o \
 				   parent.o \
 				   refcount.o \
 				   rmap.o \
fs/xfs/scrub/iscan.c new file +483 -0

483 lines not on this page · raw message

fs/xfs/scrub/iscan.h new file +65 -0

65 lines not on this page · raw message

fs/xfs/scrub/trace.c +1 -0
index 4a0385c97ea6..83e8a64c95d4 100644
@@ -16,6 +16,7 @@
 #include "scrub/scrub.h"
 #include "scrub/xfile.h"
 #include "scrub/xfarray.h"
+#include "scrub/iscan.h"
 
 /* Figure out which block the btree cursor was pointing to. */
 static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +73 -0

85 lines not on this page · raw message

fs/xfs/xfs_icache.c +2 -1
index ddeaccc04aec..0d58d7b0d8ac 100644
@@ -767,7 +767,8 @@ xfs_iget(
 	return 0;
 
 out_error_or_again:
-	if (!(flags & XFS_IGET_INCORE) && error == -EAGAIN) {
+	if (!(flags & (XFS_IGET_INCORE | XFS_IGET_NORETRY)) &&
+	    error == -EAGAIN) {
 		delay(1);
 		goto again;
 	}
fs/xfs/xfs_icache.h +7 -4
index 6cd180721659..87910191a9dd 100644
@@ -34,10 +34,13 @@ struct xfs_icwalk {
 /*
  * Flags for xfs_iget()
  */
-#define XFS_IGET_CREATE		0x1
-#define XFS_IGET_UNTRUSTED	0x2
-#define XFS_IGET_DONTCACHE	0x4
-#define XFS_IGET_INCORE		0x8	/* don't read from disk or reinit */
+#define XFS_IGET_CREATE		(1U << 0)
+#define XFS_IGET_UNTRUSTED	(1U << 1)
+#define XFS_IGET_DONTCACHE	(1U << 2)
+/* don't read from disk or reinit */
+#define XFS_IGET_INCORE		(1U << 3)
+/* Return -EAGAIN immediately if the inode is unavailable. */
+#define XFS_IGET_NORETRY	(1U << 4)
 
 int xfs_iget(struct xfs_mount *mp, struct xfs_trans *tp, xfs_ino_t ino,
 	     uint flags, uint lock_flags, xfs_inode_t **ipp);
#52 lore raw

PATCH 13/23 xfs: allow scrub to hook metadata updates in other writers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Certain types of filesystem metadata can only be checked by scanning every file in the entire filesystem. Specific examples of this include quota counts, file link counts, and reverse mappings of file extents. Directory and parent pointer reconstruction may also fall into this category. File scanning is much trickier than scanning AG metadata because we have to take inode locks in the same order as the rest of [VX]FS, we can't be holding buffer locks when we do that, and scanning the whole filesystem takes time.

Earlier versions of the online repair patchset relied heavily on fsfreeze as a means to quiesce the filesystem so that we could take locks in the proper order without worrying about concurrent updates from other writers. Reviewers of those patches opined that freezing the entire fs to check and repair something was not sufficiently better than unmounting to run fsck offline. I don't agree with that 100%, but the message was clear: find a way to repair things that minimizes the quiet period where nobody can write to the filesystem.

Generally, building btree indexes online can be split into two phases: a collection phase where we compute the records that will be put into the new btree; and a construction phase, where we construct the physical btree blocks and persist them. While it's simple to hold resource locks for the entirety of the two phases to ensure that the new index is consistent with the rest of the system, we don't need to hold resource locks during the collection phase if we have a means to receive live updates of other work going on elsewhere in the system.

The goal of this patch, then, is to enable online fsck to learn about metadata updates going on in other threads while it constructs a shadow copy of the metadata records to verify or correct the real metadata. To minimize the overhead when online fsck isn't running, we use srcu notifiers because they prioritize fast access to the notifier call chain (particularly when the chain is empty) at a cost to configuring notifiers. Online fsck should be relatively infrequent, so this is acceptable.

The intended usage model is fairly simple. Code that modifies a metadata structure of interest should declare a xfs_hook_chain structure in some well defined place, and call xfs_hook_call whenever an update happens. Online fsck code should define a struct notifier_block and use xfs_hook_add to attach the block to the chain, along with a function to be called. This function should synchronize with the fsck scanner to update whatever in-memory data the scanner is collecting. When finished, xfs_hook_del removes the notifier from the list and waits for them all to complete.

On the author's computer, calling an empty srcu notifier chain was observed to have an overhead averaging ~40ns with a maximum of 60ns. Adding a no-op notifier function increased the average to ~58ns and 66ns. When the quotacheck live update notifier is attached, the average increases to ~322ns with a max of 372ns to update scrub's in-memory observation data, assuming no lock contention.

With jump labels enabled, calls to empty srcu notifier chains are elided from the call sites when there are no hooks registered, which means that the overhead is 0.36ns when fsck is not running. For compilers that do not support jump labels (all major architectures do), the overhead of a no-op notifier call is less bad (on a many-cpu system) than the atomic counter ops, so we make the hook switch itself a nop.

Note: This new code is also split out as a separate patch from its initial user so that the author can move patches around his tree with ease.

---
 fs/xfs/Kconfig     |    6 +++++
 fs/xfs/Makefile    |    2 ++
 fs/xfs/xfs_hooks.c |   53 +++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_hooks.h |   68 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_linux.h |    1 +
 5 files changed, 130 insertions(+)
 create mode 100644 fs/xfs/xfs_hooks.c
 create mode 100644 fs/xfs/xfs_hooks.h
fs/xfs/Kconfig +6 -0
index 7f12b40146b3..e99821c4c337 100644
@@ -92,12 +92,18 @@ config XFS_RT
 	  See the xfs man page in section 5 for additional information.
 
 	  If unsure, say N.
+ 
+config XFS_LIVE_HOOKS
+	bool
+	select JUMP_LABEL if HAVE_ARCH_JUMP_LABEL
 
 config XFS_ONLINE_SCRUB
 	bool "XFS online metadata check support"
 	default n
 	depends on XFS_FS
 	depends on TMPFS && SHMEM
+	depends on SRCU
+	select XFS_LIVE_HOOKS
 	help
 	  If you say Y here you will be able to check metadata on a
 	  mounted XFS filesystem.  This feature is intended to reduce
fs/xfs/Makefile +2 -0
index 69805b4ad79f..64a3cc396e16 100644
@@ -137,6 +137,8 @@ ifeq ($(CONFIG_MEMORY_FAILURE),y)
 xfs-$(CONFIG_FS_DAX)		+= xfs_notify_failure.o
 endif
 
+xfs-$(CONFIG_XFS_LIVE_HOOKS)	+= xfs_hooks.o
+
 # online scrub/repair
 ifeq ($(CONFIG_XFS_ONLINE_SCRUB),y)
 
fs/xfs/xfs_hooks.c new file +53 -0

53 lines not on this page · raw message

fs/xfs/xfs_hooks.h new file +68 -0

68 lines not on this page · raw message

fs/xfs/xfs_linux.h +1 -0
index f9878021e7d0..c05f7e309c3e 100644
@@ -79,6 +79,7 @@ typedef __u32			xfs_nlink_t;
 #include "xfs_cksum.h"
 #include "xfs_buf.h"
 #include "xfs_message.h"
+#include "xfs_hooks.h"
 
 #ifdef __BIG_ENDIAN
 #define XFS_NATIVE_HOST 1
#53 lore raw

PATCH 14/23 xfs: allow blocking notifier chains with filesystem hooks

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make it so that we can switch between notifier chain implementations for testing purposes. On the author's test system, calling an empty srcu notifier chain cost about 19ns per call, vs. 4ns for a blocking notifier chain. Hm. Might we actually want regular blocking notifiers?

---
 fs/xfs/Kconfig     |   33 ++++++++++++++++++++++++++++++++-
 fs/xfs/xfs_hooks.c |   41 +++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_hooks.h |    6 +++++-
 3 files changed, 78 insertions(+), 2 deletions(-)
fs/xfs/Kconfig +32 -1
index e99821c4c337..4798a147fd9e 100644
@@ -102,7 +102,6 @@ config XFS_ONLINE_SCRUB
 	default n
 	depends on XFS_FS
 	depends on TMPFS && SHMEM
-	depends on SRCU
 	select XFS_LIVE_HOOKS
 	help
 	  If you say Y here you will be able to check metadata on a
@@ -117,6 +116,38 @@ config XFS_ONLINE_SCRUB
 
 	  If unsure, say N.
 
+choice
+	prompt "XFS hook implementation"
+	depends on XFS_FS && XFS_LIVE_HOOKS && XFS_ONLINE_SCRUB
+	default XFS_LIVE_HOOKS_BLOCKING if HAVE_ARCH_JUMP_LABEL
+	default XFS_LIVE_HOOKS_SRCU if !HAVE_ARCH_JUMP_LABEL
+	help
+	  Pick one
+
+config XFS_LIVE_HOOKS_SRCU
+	bool "SRCU notifier chains"
+	depends on SRCU
+	help
+	  Use SRCU notifier chains for filesystem hooks.  These have very low
+	  overhead for event initiators (the main filesystem) and higher
+	  overhead for chain modifiers (scrub waits for RCU grace).  This is
+	  the best option when jump labels are not supported or there are many
+	  CPUs in the system.
+
+	  This may cause problems with CPU hotplug invoking reclaim invoking
+	  XFS.
+
+config XFS_LIVE_HOOKS_BLOCKING
+	bool "Blocking notifier chains"
+	help
+	  Use blocking notifier chains for filesystem hooks.  These have medium
+	  overhead for event initiators (the main fs) and chain modifiers
+	  (scrub) due to their use of rwsems.  This is the best option when
+	  jump labels can be used to eliminate overhead for the filesystem when
+	  scrub is not running.
+
+endchoice
+
 config XFS_ONLINE_REPAIR
 	bool "XFS online metadata repair support"
 	default n
fs/xfs/xfs_hooks.c +41 -0

50 lines not on this page · raw message

fs/xfs/xfs_hooks.h +5 -1
index 8dd5baffb84a..a63f2276f614 100644
@@ -6,10 +6,14 @@
 #ifndef XFS_HOOKS_H_
 #define XFS_HOOKS_H_
 
-#ifdef CONFIG_XFS_LIVE_HOOKS
+#if defined(CONFIG_XFS_LIVE_HOOKS_SRCU)
 struct xfs_hooks {
 	struct srcu_notifier_head	head;
 };
+#elif defined(CONFIG_XFS_LIVE_HOOKS_BLOCKING)
+struct xfs_hooks {
+	struct blocking_notifier_head	head;
+};
 #else
 struct xfs_hooks { /* empty */ };
 #endif
#54 lore raw

PATCH 15/23 xfs: streamline the directory iteration code for scrub

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Currently, online scrub reuses the xfs_readdir code to walk every entry in a directory. This isn't awesome for performance, since we end up cycling the directory ILOCK needlessly and coding around the particular quirks of the VFS dir_context interface.

Create a streamlined version of readdir that keeps the ILOCK (since the walk function isn't going to copy stuff to userspace), skips a whole lot of directory walk cursor checks (since we start at 0 and walk to the end) and has a sane way to return error codes.

---
 fs/xfs/Makefile        |    1 
 fs/xfs/scrub/dir.c     |  173 +++++++---------------
 fs/xfs/scrub/parent.c  |   90 +++---------
 fs/xfs/scrub/readdir.c |  375 ++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/readdir.h |   19 ++
 5 files changed, 473 insertions(+), 185 deletions(-)
 create mode 100644 fs/xfs/scrub/readdir.c
 create mode 100644 fs/xfs/scrub/readdir.h
fs/xfs/Makefile +1 -0
index 64a3cc396e16..9a6ef1f7c27b 100644
@@ -161,6 +161,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   inode.o \
 				   iscan.o \
 				   parent.o \
+				   readdir.o \
 				   refcount.o \
 				   rmap.o \
 				   scrub.o \
fs/xfs/scrub/dir.c +51 -122

270 lines not on this page · raw message

fs/xfs/scrub/parent.c +27 -63

148 lines not on this page · raw message

fs/xfs/scrub/readdir.c new file +375 -0

375 lines not on this page · raw message

fs/xfs/scrub/readdir.h new file +19 -0
new file mode 100644
index 000000000000..7272f3bd28b4
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2022 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_SCRUB_READDIR_H__
+#define __XFS_SCRUB_READDIR_H__
+
+typedef int (*xchk_dirent_fn)(struct xfs_scrub *sc, struct xfs_inode *dp,
+		xfs_dir2_dataptr_t dapos, const struct xfs_name *name,
+		xfs_ino_t ino, void *priv);
+
+int xchk_dir_walk(struct xfs_scrub *sc, struct xfs_inode *dp,
+		xchk_dirent_fn dirent_fn, void *priv);
+
+int xchk_dir_lookup(struct xfs_scrub *sc, struct xfs_inode *dp,
+		const struct xfs_name *name, xfs_ino_t *ino);
+
+#endif /* __XFS_SCRUB_READDIR_H__ */
#55 lore raw

PATCH 16/23 xfs: track file link count updates during live nlinks fsck

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create the necessary hooks in the file create/unlink/rename code so that our live nlink scrub code can stay up to date with the rest of the filesystem. This will be the means to keep our shadow link count information up to date while the scan runs in real time.

---
 fs/xfs/libxfs/xfs_dir2.c |    6 +
 fs/xfs/libxfs/xfs_dir2.h |    1 
 fs/xfs/scrub/common.c    |   20 ++++
 fs/xfs/scrub/common.h    |    2 
 fs/xfs/scrub/scrub.c     |   17 +++
 fs/xfs/scrub/scrub.h     |    3 +
 fs/xfs/scrub/trace.h     |   42 +++++++++
 fs/xfs/xfs_inode.c       |  226 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_inode.h       |   35 +++++++
 fs/xfs/xfs_mount.h       |    2 
 fs/xfs/xfs_super.c       |    2 
 fs/xfs/xfs_symlink.c     |    1 
 12 files changed, 357 insertions(+)
fs/xfs/libxfs/xfs_dir2.c +6 -0
index c1a9394d7478..27e408d20d18 100644
@@ -25,6 +25,12 @@ const struct xfs_name xfs_name_dotdot = {
 	.type	= XFS_DIR3_FT_DIR,
 };
 
+const struct xfs_name xfs_name_dot = {
+	.name	= (const unsigned char *)".",
+	.len	= 1,
+	.type	= XFS_DIR3_FT_DIR,
+};
+
 /*
  * Convert inode mode to directory entry filetype
  */
fs/xfs/libxfs/xfs_dir2.h +1 -0
index ff59f009d1fd..ac360c0b2fe7 100644
@@ -22,6 +22,7 @@ struct xfs_dir3_icfree_hdr;
 struct xfs_dir3_icleaf_hdr;
 
 extern const struct xfs_name	xfs_name_dotdot;
+extern const struct xfs_name	xfs_name_dot;
 
 /*
  * Convert inode mode to directory entry filetype
fs/xfs/scrub/common.c +20 -0
index dc78e28a9447..a4cfe5653880 100644
@@ -961,3 +961,23 @@ xchk_start_reaping(
 	}
 	sc->flags &= ~XCHK_REAPING_DISABLED;
 }
+
+/*
+ * Enable filesystem hooks (i.e. runtime code patching) before starting a scrub
+ * operation.  Callers must not hold any locks that intersect with the CPU
+ * hotplug lock (e.g. writeback locks) because code patching must halt the CPUs
+ * to change kernel code.
+ */
+void
+xchk_fshooks_enable(
+	struct xfs_scrub	*sc,
+	unsigned int		scrub_fshooks)
+{
+	ASSERT(!(scrub_fshooks & ~XCHK_FSHOOKS_ALL));
+	ASSERT(!(sc->flags & scrub_fshooks));
+
+	if (scrub_fshooks & XCHK_FSHOOKS_DIRENTS)
+		xfs_dirent_hook_enable();
+
+	sc->flags |= scrub_fshooks;
+}
fs/xfs/scrub/common.h +2 -0
index 5286c263ff60..423a98c39fb6 100644
@@ -157,4 +157,6 @@ int xchk_metadata_inode_forks(struct xfs_scrub *sc);
 void xchk_stop_reaping(struct xfs_scrub *sc);
 void xchk_start_reaping(struct xfs_scrub *sc);
 
+void xchk_fshooks_enable(struct xfs_scrub *sc, unsigned int scrub_fshooks);
+
 #endif	/* __XFS_SCRUB_COMMON_H__ */
fs/xfs/scrub/scrub.c +17 -0
index 6aedce9b67fc..871a72e22a8a 100644
@@ -145,6 +145,21 @@ xchk_probe(
 
 /* Scrub setup and teardown */
 
+static inline void
+xchk_fshooks_disable(
+	struct xfs_scrub	*sc)
+{
+	if (!(sc->flags & XCHK_FSHOOKS_ALL))
+		return;
+
+	//trace_xchk_fshooks_disable(sc, sc->flags & XCHK_FSHOOKS_ALL);
+
+	if (sc->flags & XCHK_FSHOOKS_DIRENTS)
+		xfs_dirent_hook_disable();
+
+	sc->flags &= ~XCHK_FSHOOKS_ALL;
+}
+
 /* Free all the resources and finish the transactions. */
 STATIC int
 xchk_teardown(
@@ -177,6 +192,8 @@ xchk_teardown(
 		kvfree(sc->buf);
 		sc->buf = NULL;
 	}
+
+	xchk_fshooks_disable(sc);
 	return error;
 }
 
fs/xfs/scrub/scrub.h +3 -0
index b4d391b4c938..484e5fb7fe7a 100644
@@ -97,8 +97,11 @@ struct xfs_scrub {
 /* XCHK state flags grow up from zero, XREP state flags grown down from 2^31 */
 #define XCHK_TRY_HARDER		(1 << 0)  /* can't get resources, try again */
 #define XCHK_REAPING_DISABLED	(1 << 2)  /* background block reaping paused */
+#define XCHK_FSHOOKS_DIRENTS	(1 << 5)  /* link count live update enabled */
 #define XREP_ALREADY_FIXED	(1 << 31) /* checking our repair work */
 
+#define XCHK_FSHOOKS_ALL	(XCHK_FSHOOKS_DIRENTS)
+
 /* Metadata scrubbers */
 int xchk_tester(struct xfs_scrub *sc);
 int xchk_superblock(struct xfs_scrub *sc);
fs/xfs/scrub/trace.h +42 -0

48 lines not on this page · raw message

fs/xfs/xfs_inode.c +226 -0

268 lines not on this page · raw message

fs/xfs/xfs_inode.h +35 -0
index 5735de32beeb..b7a16642a8c3 100644
@@ -576,4 +576,39 @@ int xfs_ilock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);
 void xfs_iunlock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);
 void xfs_lock_inodes(struct xfs_inode **ips, int inodes, uint lock_mode);
 
+/*
+ * Parameters for tracking bumplink and droplink operations.  The hook
+ * function arg parameter is one of these.
+ */
+enum xfs_dirent_update_type {
+	XFS_DIRENT_CHILD_DELTA,		/* parent pointing to child */
+	XFS_DIRENT_BACKREF_DELTA,		/* dotdot entries */
+	XFS_DIRENT_SELF_DELTA,		/* dot entries */
+};
+
+struct xfs_dirent_update_params {
+	const struct xfs_inode	*dp;
+	const struct xfs_inode	*ip;
+	const struct xfs_name	*name;
+	int			delta;
+};
+
+#ifdef CONFIG_XFS_LIVE_HOOKS
+void xfs_dirent_child_delta(struct xfs_inode *dp, struct xfs_inode *ip,
+		int delta, struct xfs_name *name);
+
+struct xfs_dirent_hook {
+	struct xfs_hook		delta_hook;
+};
+
+void xfs_dirent_hook_disable(void);
+void xfs_dirent_hook_enable(void);
+
+int xfs_dirent_hook_add(struct xfs_mount *mp, struct xfs_dirent_hook *hook);
+void xfs_dirent_hook_del(struct xfs_mount *mp, struct xfs_dirent_hook *hook);
+
+#else
+# define xfs_dirent_child_delta(dp, ip, delta, name)	((void)0)
+#endif /* CONFIG_XFS_LIVE_HOOKS */
+
 #endif	/* __XFS_INODE_H__ */
fs/xfs/xfs_mount.h +2 -0
index 8aca2cc173ac..c08f55cc4f36 100644
@@ -242,6 +242,8 @@ typedef struct xfs_mount {
 	unsigned int		*m_errortag;
 	struct xfs_kobj		m_errortag_kobj;
 #endif
+	/* Hook to feed file directory updates to an active online repair. */
+	struct xfs_hooks	m_dirent_update_hooks;
 } xfs_mount_t;
 
 #define M_IGEO(mp)		(&(mp)->m_ino_geo)
fs/xfs/xfs_super.c +2 -0
index 0ac55d191f1f..0432a4a096e8 100644
@@ -1949,6 +1949,8 @@ static int xfs_init_fs_context(
 	mp->m_logbsize = -1;
 	mp->m_allocsize_log = 16; /* 64k */
 
+	xfs_hooks_init(&mp->m_dirent_update_hooks);
+
 	/*
 	 * Copy binary VFS mount flags we are interested in.
 	 */
fs/xfs/xfs_symlink.c +1 -0
index f305226109f0..77427a50a760 100644
@@ -354,6 +354,7 @@ xfs_symlink(
 			goto out_trans_cancel;
 	}
 
+	xfs_dirent_child_delta(dp, ip, 1, link_name);
 
 	/*
 	 * If this is a synchronous mount, make sure that the
#56 lore raw

PATCH 17/23 xfs: connect in-memory btrees to xfiles

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add to our stubbed-out in-memory btrees the ability to connect them with an actual in-memory backing file (aka xfiles) and the necessary pieces to track free space in the xfile and flush dirty xfbtree buffers on demand, which we'll need for online repair.

---
 fs/xfs/scrub/trace.h |    1 +
 fs/xfs/scrub/xfile.c |   11 +++++++++++
 fs/xfs/scrub/xfile.h |    2 ++
 3 files changed, 14 insertions(+)
fs/xfs/scrub/trace.h +1 -0
index 979ee2789668..4a6f0f1b0881 100644
@@ -755,6 +755,7 @@ DEFINE_EVENT(xfile_class, name, \
 DEFINE_XFILE_EVENT(xfile_pread);
 DEFINE_XFILE_EVENT(xfile_pwrite);
 DEFINE_XFILE_EVENT(xfile_seek_data);
+DEFINE_XFILE_EVENT(xfile_discard);
 
 TRACE_EVENT(xfarray_create,
 	TP_PROTO(struct xfarray *xfa, unsigned long long required_capacity),
fs/xfs/scrub/xfile.c +11 -0
index 43455aa78243..f9888b6dd728 100644
@@ -285,6 +285,17 @@ xfile_pwrite(
 	return error;
 }
 
+/* Discard pages backing a range of the xfile. */
+void
+xfile_discard(
+	struct xfile		*xf,
+	loff_t			pos,
+	u64			count)
+{
+	trace_xfile_discard(xf, pos, count);
+	shmem_truncate_range(file_inode(xf->file), pos, pos + count - 1);
+}
+
 /* Find the next written area in the xfile data for a given offset. */
 loff_t
 xfile_seek_data(
fs/xfs/scrub/xfile.h +2 -0
index b37dba1961d8..973c8fc37707 100644
@@ -46,6 +46,8 @@ xfile_obj_store(struct xfile *xf, const void *buf, size_t count, loff_t pos)
 	return 0;
 }
 
+void xfile_discard(struct xfile *xf, loff_t pos, u64 count);
+int xfile_prealloc(struct xfile *xf, loff_t pos, u64 count);
 loff_t xfile_seek_data(struct xfile *xf, loff_t pos);
 
 struct xfile_stat {
#57 lore raw

PATCH 18/23 xfs: create temporary files and directories for online repair

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Teach the online repair code how to create temporary files or directories. These temporary files can be used to stage reconstructed information until we're ready to perform an atomic extent swap to commit the new metadata.

---
 fs/xfs/Makefile         |    1 
 fs/xfs/scrub/common.c   |    1 
 fs/xfs/scrub/scrub.c    |    2 
 fs/xfs/scrub/scrub.h    |    4 +
 fs/xfs/scrub/tempfile.c |  230 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/tempfile.h |   27 ++++++
 fs/xfs/scrub/trace.h    |   33 +++++++
 fs/xfs/xfs_inode.c      |    3 -
 fs/xfs/xfs_inode.h      |    2 
 9 files changed, 301 insertions(+), 2 deletions(-)
 create mode 100644 fs/xfs/scrub/tempfile.c
 create mode 100644 fs/xfs/scrub/tempfile.h
fs/xfs/Makefile +1 -0
index 9a6ef1f7c27b..2562c852db7f 100644
@@ -178,6 +178,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   repair.o \
+				   tempfile.o \
 				   )
 endif
 endif
fs/xfs/scrub/common.c +1 -0
index a4cfe5653880..2874da088e8d 100644
@@ -33,6 +33,7 @@
 #include "scrub/trace.h"
 #include "scrub/repair.h"
 #include "scrub/health.h"
+#include "scrub/tempfile.h"
 
 /* Common code for the metadata scrubbers. */
 
fs/xfs/scrub/scrub.c +2 -0
index 871a72e22a8a..a19ea7fdd510 100644
@@ -22,6 +22,7 @@
 #include "scrub/trace.h"
 #include "scrub/repair.h"
 #include "scrub/health.h"
+#include "scrub/tempfile.h"
 
 /*
  * Online Scrub and Repair
@@ -193,6 +194,7 @@ xchk_teardown(
 		sc->buf = NULL;
 	}
 
+	xrep_tempfile_rele(sc);
 	xchk_fshooks_disable(sc);
 	return error;
 }
fs/xfs/scrub/scrub.h +4 -0
index 484e5fb7fe7a..20e814a53850 100644
@@ -80,6 +80,10 @@ struct xfs_scrub {
 	void				*buf;
 	uint				ilock_flags;
 
+	/* A temporary file on this filesystem, for staging new metadata. */
+	struct xfs_inode		*tempip;
+	uint				temp_ilock_flags;
+
 	/* See the XCHK/XREP state flags below. */
 	unsigned int			flags;
 
fs/xfs/scrub/tempfile.c new file +230 -0

230 lines not on this page · raw message

fs/xfs/scrub/tempfile.h new file +27 -0
new file mode 100644
index 000000000000..f00a9ce43a32
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2022 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_SCRUB_TEMPFILE_H__
+#define __XFS_SCRUB_TEMPFILE_H__
+
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+int xrep_tempfile_create(struct xfs_scrub *sc, uint16_t mode);
+void xrep_tempfile_rele(struct xfs_scrub *sc);
+
+bool xrep_tempfile_iolock_nowait(struct xfs_scrub *sc);
+void xrep_tempfile_iounlock(struct xfs_scrub *sc);
+
+void xrep_tempfile_ilock(struct xfs_scrub *sc);
+bool xrep_tempfile_ilock_nowait(struct xfs_scrub *sc);
+void xrep_tempfile_iunlock(struct xfs_scrub *sc);
+#else
+static inline void xrep_tempfile_iolock_both(struct xfs_scrub *sc)
+{
+	xchk_ilock(sc, XFS_IOLOCK_EXCL);
+}
+# define xrep_tempfile_rele(sc)
+#endif /* CONFIG_XFS_ONLINE_REPAIR */
+
+#endif /* __XFS_SCRUB_TEMPFILE_H__ */
fs/xfs/scrub/trace.h +33 -0
index 4a6f0f1b0881..0c27eb197f83 100644
@@ -1149,6 +1149,39 @@ TRACE_EVENT(xrep_ialloc_insert,
 		  __entry->freemask)
 )
 
+TRACE_EVENT(xrep_tempfile_create,
+	TP_PROTO(struct xfs_scrub *sc),
+	TP_ARGS(sc),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_ino_t, ino)
+		__field(unsigned int, type)
+		__field(xfs_agnumber_t, agno)
+		__field(xfs_ino_t, inum)
+		__field(unsigned int, gen)
+		__field(unsigned int, flags)
+		__field(xfs_ino_t, temp_inum)
+	),
+	TP_fast_assign(
+		__entry->dev = sc->mp->m_super->s_dev;
+		__entry->ino = sc->file ? XFS_I(file_inode(sc->file))->i_ino : 0;
+		__entry->type = sc->sm->sm_type;
+		__entry->agno = sc->sm->sm_agno;
+		__entry->inum = sc->sm->sm_ino;
+		__entry->gen = sc->sm->sm_gen;
+		__entry->flags = sc->sm->sm_flags;
+		__entry->temp_inum = sc->tempip->i_ino;
+	),
+	TP_printk("dev %d:%d ino 0x%llx type %s inum 0x%llx gen 0x%x flags 0x%x temp_inum 0x%llx",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino,
+		  __print_symbolic(__entry->type, XFS_SCRUB_TYPE_STRINGS),
+		  __entry->inum,
+		  __entry->gen,
+		  __entry->flags,
+		  __entry->temp_inum)
+);
+
 #endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
 
 #endif /* _TRACE_XFS_SCRUB_TRACE_H */
fs/xfs/xfs_inode.c +1 -2
index b17e4ba3622b..8ad646beee75 100644
@@ -48,7 +48,6 @@ struct kmem_cache *xfs_inode_cache;
  */
 #define	XFS_ITRUNC_MAX_EXTENTS	2
 
-STATIC int xfs_iunlink(struct xfs_trans *, struct xfs_inode *);
 STATIC int xfs_iunlink_remove(struct xfs_trans *tp, struct xfs_perag *pag,
 	struct xfs_inode *);
 
@@ -2187,7 +2186,7 @@ xfs_iunlink_insert_inode(
  * We place the on-disk inode on a list in the AGI.  It will be pulled from this
  * list when the inode is freed.
  */
-STATIC int
+int
 xfs_iunlink(
 	struct xfs_trans	*tp,
 	struct xfs_inode	*ip)
fs/xfs/xfs_inode.h +2 -0
index b7a16642a8c3..94a1490fb7b0 100644
@@ -570,6 +570,8 @@ extern struct kmem_cache	*xfs_inode_cache;
 
 bool xfs_inode_needs_inactive(struct xfs_inode *ip);
 
+int xfs_iunlink(struct xfs_trans *tp, struct xfs_inode *ip);
+
 void xfs_end_io(struct work_struct *work);
 
 int xfs_ilock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2);
#58 lore raw

PATCH 19/23 xfs: hide private inodes from bulkstat and handle functions

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

We're about to start adding functionality that uses internal inodes that are private to XFS. What this means is that userspace should never be able to access any information about these files, and should not be able to open these files by handle. Callers are not allowed to link these files into the directory tree, which should suffice to make these private inodes actually private.

---
 fs/xfs/xfs_export.c |    2 +-
 fs/xfs/xfs_itable.c |    8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)
fs/xfs/xfs_export.c +1 -1
index 1064c2342876..b6ba96e0dd75 100644
@@ -146,7 +146,7 @@ xfs_nfs_get_inode(
 		return ERR_PTR(error);
 	}
 
-	if (VFS_I(ip)->i_generation != generation) {
+	if (VFS_I(ip)->i_generation != generation || IS_PRIVATE(VFS_I(ip))) {
 		xfs_irele(ip);
 		return ERR_PTR(-ESTALE);
 	}
fs/xfs/xfs_itable.c +8 -0
index a1c2bcf65d37..7a967cc78010 100644
@@ -86,6 +86,14 @@ xfs_bulkstat_one_int(
 	vfsuid = i_uid_into_vfsuid(mnt_userns, inode);
 	vfsgid = i_gid_into_vfsgid(mnt_userns, inode);
 
+	/* If this is a private inode, don't leak its details to userspace. */
+	if (IS_PRIVATE(inode)) {
+		xfs_iunlock(ip, XFS_ILOCK_SHARED);
+		xfs_irele(ip);
+		error = -EINVAL;
+		goto out_advance;
+	}
+
 	/* xfs_iget returns the following without needing
 	 * further change.
 	 */
#59 lore raw

PATCH 20/23 xfs: create a blob array data structure

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a simple 'blob array' data structure for storage of arbitrarily sized metadata objects that will be used to reconstruct metadata. For the intended usage (temporarily storing extended attribute names and values) we only have to support storing objects and retrieving them. Use the xfile abstraction to store the attribute information in memory that can be swapped out.

---
 fs/xfs/Makefile       |    1 
 fs/xfs/scrub/xfblob.c |  152 +++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/xfblob.h |   25 ++++++++
 3 files changed, 178 insertions(+)
 create mode 100644 fs/xfs/scrub/xfblob.c
 create mode 100644 fs/xfs/scrub/xfblob.h
fs/xfs/Makefile +1 -0
index 2562c852db7f..f2f3ab589c04 100644
@@ -179,6 +179,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   repair.o \
 				   tempfile.o \
+				   xfblob.o \
 				   )
 endif
 endif
fs/xfs/scrub/xfblob.c new file +152 -0

152 lines not on this page · raw message

fs/xfs/scrub/xfblob.h new file +25 -0
new file mode 100644
index 000000000000..d1282810bb1d
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2022 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_SCRUB_XFBLOB_H__
+#define __XFS_SCRUB_XFBLOB_H__
+
+struct xfblob {
+	struct xfile	*xfile;
+	loff_t		last_offset;
+};
+
+typedef loff_t		xfblob_cookie;
+
+int xfblob_create(struct xfs_mount *mp, const char *descr,
+		struct xfblob **blobp);
+void xfblob_destroy(struct xfblob *blob);
+int xfblob_load(struct xfblob *blob, xfblob_cookie cookie, void *ptr,
+		uint32_t size);
+int xfblob_store(struct xfblob *blob, xfblob_cookie *cookie, const void *ptr,
+		uint32_t size);
+int xfblob_free(struct xfblob *blob, xfblob_cookie cookie);
+
+#endif /* __XFS_SCRUB_XFBLOB_H__ */
#60 lore raw

PATCH 21/23 xfs: repair extended attributes

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the extended attributes look bad, try to sift through the rubble to find whatever keys/values we can, stage a new attribute structure in a temporary file and use the atomic extent swapping mechanism to commit the results in bulk.

---
 fs/xfs/scrub/xfarray.c |   24 ++++++++++++++++++++++++
 fs/xfs/scrub/xfarray.h |    2 ++
 fs/xfs/scrub/xfblob.c  |   24 ++++++++++++++++++++++++
 fs/xfs/scrub/xfblob.h  |    2 ++
 4 files changed, 52 insertions(+)
fs/xfs/scrub/xfarray.c +24 -0
index 8fdd7dd40193..fccb3a3d9199 100644
@@ -368,3 +368,27 @@ xfarray_load_next(
 	*idx = cur;
 	return 0;
 }
+
+/* How many bytes is this array consuming? */
+long long
+xfarray_bytes(
+	struct xfarray		*array)
+{
+	struct xfile_stat	statbuf;
+	int			error;
+
+	error = xfile_stat(array->xfile, &statbuf);
+	if (error)
+		return error;
+
+	return statbuf.bytes;
+}
+
+/* Empty the entire array. */
+void
+xfarray_truncate(
+	struct xfarray	*array)
+{
+	xfile_discard(array->xfile, 0, MAX_LFS_FILESIZE);
+	array->nr = 0;
+}
fs/xfs/scrub/xfarray.h +2 -0
index 26e2b594f121..8a3af0cecc3e 100644
@@ -45,6 +45,8 @@ int xfarray_unset(struct xfarray *array, xfarray_idx_t idx);
 int xfarray_store(struct xfarray *array, xfarray_idx_t idx, const void *ptr);
 int xfarray_store_anywhere(struct xfarray *array, const void *ptr);
 bool xfarray_element_is_null(struct xfarray *array, const void *ptr);
+void xfarray_truncate(struct xfarray *array);
+long long xfarray_bytes(struct xfarray *array);
 
 /* Append an element to the array. */
 static inline int xfarray_append(struct xfarray *array, const void *ptr)
fs/xfs/scrub/xfblob.c +24 -0
index 1f89d7d13c59..2f89617a2db8 100644
@@ -150,3 +150,27 @@ xfblob_free(
 	xfile_discard(blob->xfile, cookie, sizeof(key) + key.xb_size);
 	return 0;
 }
+
+/* How many bytes is this blob storage object consuming? */
+long long
+xfblob_bytes(
+	struct xfblob		*blob)
+{
+	struct xfile_stat	statbuf;
+	int			error;
+
+	error = xfile_stat(blob->xfile, &statbuf);
+	if (error)
+		return error;
+
+	return statbuf.bytes;
+}
+
+/* Drop all the blobs. */
+void
+xfblob_truncate(
+	struct xfblob	*blob)
+{
+	xfile_discard(blob->xfile, 0, MAX_LFS_FILESIZE);
+	blob->last_offset = 0;
+}
fs/xfs/scrub/xfblob.h +2 -0
index d1282810bb1d..8a5738e1d568 100644
@@ -21,5 +21,7 @@ int xfblob_load(struct xfblob *blob, xfblob_cookie cookie, void *ptr,
 int xfblob_store(struct xfblob *blob, xfblob_cookie *cookie, const void *ptr,
 		uint32_t size);
 int xfblob_free(struct xfblob *blob, xfblob_cookie cookie);
+long long xfblob_bytes(struct xfblob *blob);
+void xfblob_truncate(struct xfblob *blob);
 
 #endif /* __XFS_SCRUB_XFBLOB_H__ */
#61 lore raw

PATCH 22/23 xfs: online repair of directories

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If a directory looks like it's in bad shape, try to sift through the rubble to find whatever directory entries we can, scan the directory tree for the parent (if needed), stage the new directory contents in a temporary file and use the atomic extent swapping mechanism to commit the results in bulk. As a side effect of this patch, directory inactivation will be able to purge any leftover dir blocks.

---
 fs/xfs/scrub/tempfile.c |   13 +++++++++++++
 fs/xfs/scrub/tempfile.h |    2 ++
 2 files changed, 15 insertions(+)
fs/xfs/scrub/tempfile.c +13 -0
index 8f80f1c2555c..91875d4bb67f 100644
@@ -228,3 +228,16 @@ xrep_tempfile_rele(
 	xchk_irele(sc, sc->tempip);
 	sc->tempip = NULL;
 }
+
+/* Decide if a given XFS inode is a temporary file for a repair. */
+bool
+xrep_is_tempfile(
+	const struct xfs_inode	*ip)
+{
+	const struct inode	*inode = &ip->i_vnode;
+
+	if (IS_PRIVATE(inode) && !(inode->i_opflags & IOP_XATTR))
+		return true;
+
+	return false;
+}
fs/xfs/scrub/tempfile.h +2 -0
index f00a9ce43a32..e2f493b5d3d9 100644
@@ -16,11 +16,13 @@ void xrep_tempfile_iounlock(struct xfs_scrub *sc);
 void xrep_tempfile_ilock(struct xfs_scrub *sc);
 bool xrep_tempfile_ilock_nowait(struct xfs_scrub *sc);
 void xrep_tempfile_iunlock(struct xfs_scrub *sc);
+bool xrep_is_tempfile(const struct xfs_inode *ip);
 #else
 static inline void xrep_tempfile_iolock_both(struct xfs_scrub *sc)
 {
 	xchk_ilock(sc, XFS_IOLOCK_EXCL);
 }
+# define xrep_is_tempfile(ip)		(false)
 # define xrep_tempfile_rele(sc)
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
#62 lore raw

PATCH 23/23 xfs: create an xattr iteration function for scrub

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a streamlined function to walk a file's xattrs, without all the cursor management stuff in the regular listxattr.

---
 fs/xfs/Makefile          |    1 
 fs/xfs/scrub/listxattr.c |  314 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/listxattr.h |   17 ++
 3 files changed, 332 insertions(+)
 create mode 100644 fs/xfs/scrub/listxattr.c
 create mode 100644 fs/xfs/scrub/listxattr.h
fs/xfs/Makefile +1 -0
index f2f3ab589c04..6a30b145491d 100644
@@ -160,6 +160,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   ialloc.o \
 				   inode.o \
 				   iscan.o \
+				   listxattr.o \
 				   parent.o \
 				   readdir.o \
 				   refcount.o \
fs/xfs/scrub/listxattr.c new file +314 -0

314 lines not on this page · raw message

fs/xfs/scrub/listxattr.h new file +17 -0
new file mode 100644
index 000000000000..97af8ca23324
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2022 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_SCRUB_LISTXATTR_H__
+#define __XFS_SCRUB_LISTXATTR_H__
+
+typedef int (*xchk_xattr_fn)(struct xfs_scrub *sc, struct xfs_inode *ip,
+		unsigned int attr_flags, const unsigned char *name,
+		unsigned int namelen, const void *value, unsigned int valuelen,
+		void *priv);
+
+int xchk_xattr_walk(struct xfs_scrub *sc, struct xfs_inode *ip,
+		xchk_xattr_fn attr_fn, void *priv);
+
+#endif /* __XFS_SCRUB_LISTXATTR_H__ */
#63 lore raw

cover xfs: online repair of directories

In reply to Darrick J. Wong

Hi all,

With this patchset, we implement online reconstruction of directories by scanning the entire filesystem looking for parent pointer data. This mostly works, except for the part where we need to resync the diroffset field of the parent pointers to match the new directory structure.

Fixing that is left as an open research question, with a few possible solutions:

1. As part of committing the new directory, queue a bunch of parent pointer updates to make those changes.

2. Leave them inconsistent and let the parent pointer repair fix it.

3. Change the ondisk format of parent pointers (and xattrs) so that we can encode the full dirent name in the xattr name.

4. Change the ondisk format of parent pointers to encode a sha256 hash of the dirent name in the xattr name. This will work as long as nobody breaks sha256.

Thoughts? Note that the atomic swapext and block reaping code is NOT ported for this PoC, so we do not commit any repairs.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-repair

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-dir-repair

---
 fs/xfs/Makefile               |    1 
 fs/xfs/libxfs/xfs_da_format.h |   11 
 fs/xfs/libxfs/xfs_dir2.c      |    2 
 fs/xfs/libxfs/xfs_dir2.h      |    2 
 fs/xfs/libxfs/xfs_parent.c    |   47 +-
 fs/xfs/libxfs/xfs_parent.h    |   24 -
 fs/xfs/scrub/common.c         |   15 +
 fs/xfs/scrub/common.h         |   28 +
 fs/xfs/scrub/dir.c            |   11 
 fs/xfs/scrub/dir_repair.c     | 1129 +++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/readdir.c        |   12 
 fs/xfs/scrub/readdir.h        |    3 
 fs/xfs/scrub/repair.h         |   16 +
 fs/xfs/scrub/scrub.c          |    2 
 fs/xfs/scrub/tempfile.c       |   42 ++
 fs/xfs/scrub/tempfile.h       |    2 
 fs/xfs/scrub/trace.c          |    1 
 fs/xfs/scrub/trace.h          |   69 +++
 fs/xfs/xfs_inode.c            |   56 +-
 fs/xfs/xfs_inode.h            |    5 
 fs/xfs/xfs_symlink.c          |    4 
 21 files changed, 1428 insertions(+), 54 deletions(-)
 create mode 100644 fs/xfs/scrub/dir_repair.c
#64 lore raw

PATCH 1/7 xfs: pass directory offsets as part of the dirent hook data

From: Darrick J. Wong <[email protected]>

When we're calling the dirent hooks about a directory entry update, be sure to pass the diroffset associated with the change. We're going to need this in the next patch.

---
 fs/xfs/xfs_inode.c   |   40 +++++++++++++++++++++++++---------------
 fs/xfs/xfs_inode.h   |    5 +++--
 fs/xfs/xfs_symlink.c |    2 +-
 3 files changed, 29 insertions(+), 18 deletions(-)
fs/xfs/xfs_inode.c +25 -15
index 8ad646beee75..ce1f6d03c3a9 100644
@@ -1042,7 +1042,8 @@ xfs_dirent_child_delta(
 	struct xfs_inode		*dp,
 	struct xfs_inode		*ip,
 	int				delta,
-	struct xfs_name			*name)
+	struct xfs_name			*name,
+	unsigned int			diroffset)
 {
 	if (xfs_hooks_switched_on(&xfs_dirents_hooks_switch)) {
 		struct xfs_dirent_update_params	p = {
@@ -1050,6 +1051,7 @@ xfs_dirent_child_delta(
 			.ip		= ip,
 			.delta		= delta,
 			.name		= name,
+			.diroffset	= diroffset,
 		};
 		struct xfs_mount	*mp = ip->i_mount;
 
@@ -1210,7 +1212,7 @@ xfs_create(
 	 * Create ip with a reference from dp, and add '.' and '..' references
 	 * if it's a directory.
 	 */
-	xfs_dirent_child_delta(dp, ip, 1, name);
+	xfs_dirent_child_delta(dp, ip, 1, name, diroffset);
 	if (is_dir) {
 		xfs_dirent_self_delta(ip, 1);
 		xfs_dirent_backref_delta(dp, ip, 1);
@@ -1481,7 +1483,7 @@ xfs_link(
 			goto error_return;
 	}
 
-	xfs_dirent_child_delta(tdp, sip, 1, target_name);
+	xfs_dirent_child_delta(tdp, sip, 1, target_name, diroffset);
 
 	/*
 	 * If this is a synchronous mount, make sure that the
@@ -2757,7 +2759,7 @@ xfs_remove(
 	 * Drop the link from dp to ip, and if ip was a directory, remove the
 	 * '.' and '..' references since we freed the directory.
 	 */
-	xfs_dirent_child_delta(dp, ip, -1, name);
+	xfs_dirent_child_delta(dp, ip, -1, name, dir_offset);
 	if (S_ISDIR(VFS_I(ip)->i_mode)) {
 		xfs_dirent_backref_delta(dp, ip, -1);
 		xfs_dirent_self_delta(ip, -1);
@@ -2873,18 +2875,22 @@ static inline void
 xfs_exchange_call_nlink_hooks(
 	struct xfs_inode	*src_dp,
 	struct xfs_name		*src_name,
+	xfs_dir2_dataptr_t	src_diroffset,
 	struct xfs_inode	*src_ip,
 	struct xfs_inode	*target_dp,
 	struct xfs_name		*target_name,
+	xfs_dir2_dataptr_t	target_diroffset,
 	struct xfs_inode	*target_ip)
 {
 	/* Exchange files in the source directory. */
-	xfs_dirent_child_delta(src_dp, src_ip, -1, src_name);
-	xfs_dirent_child_delta(src_dp, target_ip, 1, src_name);
+	xfs_dirent_child_delta(src_dp, src_ip, -1, src_name, src_diroffset);
+	xfs_dirent_child_delta(src_dp, target_ip, 1, src_name, src_diroffset);
 
 	/* Exchange files in the target directory. */
-	xfs_dirent_child_delta(target_dp, target_ip, -1, target_name);
-	xfs_dirent_child_delta(target_dp, src_ip, 1, target_name);
+	xfs_dirent_child_delta(target_dp, target_ip, -1, target_name,
+			target_diroffset);
+	xfs_dirent_child_delta(target_dp, src_ip, 1, target_name,
+			target_diroffset);
 
 	/* If the source file is a dir, update its dotdot entry. */
 	if (S_ISDIR(VFS_I(src_ip)->i_mode)) {
@@ -2903,9 +2909,11 @@ static inline void
 xfs_rename_call_nlink_hooks(
 	struct xfs_inode	*src_dp,
 	struct xfs_name		*src_name,
+	xfs_dir2_dataptr_t	src_diroffset,
 	struct xfs_inode	*src_ip,
 	struct xfs_inode	*target_dp,
 	struct xfs_name		*target_name,
+	xfs_dir2_dataptr_t	target_diroffset,
 	struct xfs_inode	*target_ip,
 	struct xfs_inode	*wip)
 {
@@ -2914,16 +2922,16 @@ xfs_rename_call_nlink_hooks(
 	 * move the source file to the target directory.
 	 */
 	if (target_ip)
-		xfs_dirent_child_delta(target_dp, target_ip, -1, target_name);
-	xfs_dirent_child_delta(target_dp, src_ip, 1, target_name);
+		xfs_dirent_child_delta(target_dp, target_ip, -1, target_name, target_diroffset);
+	xfs_dirent_child_delta(target_dp, src_ip, 1, target_name, target_diroffset);
 
 	/*
 	 * Remove the source file from the source directory, and possibly move
 	 * the whiteout file into its place.
 	 */
-	xfs_dirent_child_delta(src_dp, src_ip, -1, src_name);
+	xfs_dirent_child_delta(src_dp, src_ip, -1, src_name, src_diroffset);
 	if (wip)
-		xfs_dirent_child_delta(src_dp, wip, 1, src_name);
+		xfs_dirent_child_delta(src_dp, wip, 1, src_name, src_diroffset);
 
 	/* If the source file is a dir, update its dotdot entry. */
 	if (S_ISDIR(VFS_I(src_ip)->i_mode)) {
@@ -3080,7 +3088,8 @@ xfs_cross_rename(
 	xfs_trans_log_inode(tp, dp1, XFS_ILOG_CORE);
 
 	if (xfs_hooks_switched_on(&xfs_dirents_hooks_switch))
-		xfs_exchange_call_nlink_hooks(dp1, name1, ip1, dp2, name2, ip2);
+		xfs_exchange_call_nlink_hooks(dp1, name1, old_diroffset, ip1,
+				dp2, name2, new_diroffset, ip2);
 
 	return xfs_finish_rename(tp);
 
@@ -3560,8 +3569,9 @@ xfs_rename(
 		xfs_trans_log_inode(tp, target_dp, XFS_ILOG_CORE);
 
 	if (xfs_hooks_switched_on(&xfs_dirents_hooks_switch))
-		xfs_rename_call_nlink_hooks(src_dp, src_name, src_ip,
-				target_dp, target_name, target_ip, wip);
+		xfs_rename_call_nlink_hooks(src_dp, src_name, old_diroffset,
+				src_ip, target_dp, target_name, new_diroffset,
+				target_ip, wip);
 
 	error = xfs_finish_rename(tp);
 
fs/xfs/xfs_inode.h +3 -2
index 94a1490fb7b0..403b0f4cb5c0 100644
@@ -592,12 +592,13 @@ struct xfs_dirent_update_params {
 	const struct xfs_inode	*dp;
 	const struct xfs_inode	*ip;
 	const struct xfs_name	*name;
+	unsigned int		diroffset;
 	int			delta;
 };
 
 #ifdef CONFIG_XFS_LIVE_HOOKS
 void xfs_dirent_child_delta(struct xfs_inode *dp, struct xfs_inode *ip,
-		int delta, struct xfs_name *name);
+		int delta, struct xfs_name *name, unsigned int diroffset);
 
 struct xfs_dirent_hook {
 	struct xfs_hook		delta_hook;
@@ -610,7 +611,7 @@ int xfs_dirent_hook_add(struct xfs_mount *mp, struct xfs_dirent_hook *hook);
 void xfs_dirent_hook_del(struct xfs_mount *mp, struct xfs_dirent_hook *hook);
 
 #else
-# define xfs_dirent_child_delta(dp, ip, delta, name)	((void)0)
+# define xfs_dirent_child_delta(dp, ip, delta, name, doff)	((void)0)
 #endif /* CONFIG_XFS_LIVE_HOOKS */
 
 #endif	/* __XFS_INODE_H__ */
fs/xfs/xfs_symlink.c +1 -1
index 77427a50a760..fdfaab466f5d 100644
@@ -354,7 +354,7 @@ xfs_symlink(
 			goto out_trans_cancel;
 	}
 
-	xfs_dirent_child_delta(dp, ip, 1, link_name);
+	xfs_dirent_child_delta(dp, ip, 1, link_name, diroffset);
 
 	/*
 	 * If this is a synchronous mount, make sure that the
#65 lore raw

PATCH 2/7 xfs: pass diroffset back from xchk_dir_lookup

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Pass directory offsets back from xchk_dir_lookup so that we can compare things in scrub.

---
 fs/xfs/scrub/dir.c     |    2 +-
 fs/xfs/scrub/readdir.c |   12 ++++++++++--
 fs/xfs/scrub/readdir.h |    3 ++-
 3 files changed, 13 insertions(+), 4 deletions(-)
fs/xfs/scrub/dir.c +1 -1
index 46080134b408..06783e4b95ad 100644
@@ -105,7 +105,7 @@ xchk_dir_actor(
 	}
 
 	/* Verify that we can look up this name by hash. */
-	error = xchk_dir_lookup(sc, dp, name, &lookup_ino);
+	error = xchk_dir_lookup(sc, dp, name, &lookup_ino, NULL);
 	/* ENOENT means the hash lookup failed and the dir is corrupt */
 	if (error == -ENOENT)
 		error = -EFSCORRUPTED;
fs/xfs/scrub/readdir.c +10 -2
index 7d1695e98cc6..0a53438975c3 100644
@@ -314,7 +314,8 @@ xchk_dir_lookup(
 	struct xfs_scrub	*sc,
 	struct xfs_inode	*dp,
 	const struct xfs_name	*name,
-	xfs_ino_t		*ino)
+	xfs_ino_t		*ino,
+	xfs_dir2_dataptr_t	*diroffsetp)
 {
 	struct xfs_da_args	args = {
 		.dp		= dp,
@@ -326,10 +327,14 @@ xchk_dir_lookup(
 		.hashval	= xfs_dir2_hashname(dp->i_mount, name),
 		.whichfork	= XFS_DATA_FORK,
 		.op_flags	= XFS_DA_OP_OKNOENT,
+		.offset		= XFS_DIR2_NULL_DATAPTR,
 	};
 	bool			isblock, isleaf;
 	int			error;
 
+	if (diroffsetp)
+		*diroffsetp = XFS_DIR2_NULL_DATAPTR;
+
 	if (xfs_is_shutdown(dp->i_mount))
 		return -EIO;
 
@@ -369,7 +374,10 @@ xchk_dir_lookup(
 out_check_rval:
 	if (error == -EEXIST)
 		error = 0;
-	if (!error)
+	if (!error) {
 		*ino = args.inumber;
+		if (diroffsetp)
+			*diroffsetp = args.offset;
+	}
 	return error;
 }
fs/xfs/scrub/readdir.h +2 -1
index 7272f3bd28b4..1a18bb59adb2 100644
@@ -14,6 +14,7 @@ int xchk_dir_walk(struct xfs_scrub *sc, struct xfs_inode *dp,
 		xchk_dirent_fn dirent_fn, void *priv);
 
 int xchk_dir_lookup(struct xfs_scrub *sc, struct xfs_inode *dp,
-		const struct xfs_name *name, xfs_ino_t *ino);
+		const struct xfs_name *name, xfs_ino_t *ino,
+		xfs_dir2_dataptr_t *diroffsetp);
 
 #endif /* __XFS_SCRUB_READDIR_H__ */
#66 lore raw

PATCH 3/7 xfs: shorten parent pointer function names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Shorten the function names and add brief comments to each, outlining what they're supposed to be doing.

---
 fs/xfs/libxfs/xfs_parent.c |   18 ++++++++++++------
 fs/xfs/libxfs/xfs_parent.h |   24 ++++++++++++------------
 fs/xfs/xfs_inode.c         |   16 ++++++++--------
 fs/xfs/xfs_symlink.c       |    2 +-
 4 files changed, 33 insertions(+), 27 deletions(-)
fs/xfs/libxfs/xfs_parent.c +12 -6
index 179b9bebaf25..ec2bff195773 100644
@@ -135,6 +135,10 @@ xfs_parent_irec_from_disk(
 	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
 }
 
+/*
+ * Allocate memory to control a logged parent pointer update as part of a
+ * dirent operation.
+ */
 int
 __xfs_parent_init(
 	struct xfs_mount		*mp,
@@ -170,12 +174,13 @@ __xfs_parent_init(
 	return 0;
 }
 
+/* Add a parent pointer to reflect a dirent addition. */
 int
-xfs_parent_defer_add(
+xfs_parent_add(
 	struct xfs_trans	*tp,
 	struct xfs_parent_defer	*parent,
 	struct xfs_inode	*dp,
-	struct xfs_name		*parent_name,
+	const struct xfs_name	*parent_name,
 	xfs_dir2_dataptr_t	diroffset,
 	struct xfs_inode	*child)
 {
@@ -194,8 +199,9 @@ xfs_parent_defer_add(
 	return xfs_attr_defer_add(args);
 }
 
+/* Remove a parent pointer to reflect a dirent removal. */
 int
-xfs_parent_defer_remove(
+xfs_parent_remove(
 	struct xfs_trans	*tp,
 	struct xfs_inode	*dp,
 	struct xfs_parent_defer	*parent,
@@ -211,14 +217,14 @@ xfs_parent_defer_remove(
 	return xfs_attr_defer_remove(args);
 }
 
-
+/* Replace one parent pointer with another to reflect a rename. */
 int
-xfs_parent_defer_replace(
+xfs_parent_replace(
 	struct xfs_trans	*tp,
 	struct xfs_parent_defer	*new_parent,
 	struct xfs_inode	*old_dp,
 	xfs_dir2_dataptr_t	old_diroffset,
-	struct xfs_name		*parent_name,
+	const struct xfs_name	*parent_name,
 	struct xfs_inode	*new_dp,
 	xfs_dir2_dataptr_t	new_diroffset,
 	struct xfs_inode	*child)
fs/xfs/libxfs/xfs_parent.h +12 -12
index f4f5887d1133..35854e968f1d 100644
@@ -49,8 +49,9 @@ struct xfs_parent_defer {
  * Parent pointer attribute prototypes
  */
 void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
-			      struct xfs_inode *ip,
-			      uint32_t p_diroffset);
+		struct xfs_inode *ip, uint32_t p_diroffset);
+void xfs_init_parent_name_irec(struct xfs_parent_name_irec *irec,
+			       struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
@@ -78,18 +79,17 @@ xfs_parent_start_locked(
 	return 0;
 }
 
-int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
-			 struct xfs_inode *dp, struct xfs_name *parent_name,
-			 xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
-int xfs_parent_defer_replace(struct xfs_trans *tp,
+int xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
+		struct xfs_inode *dp, const struct xfs_name *parent_name,
+		xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
+int xfs_parent_replace(struct xfs_trans *tp,
 		struct xfs_parent_defer *new_parent, struct xfs_inode *old_dp,
-		xfs_dir2_dataptr_t old_diroffset, struct xfs_name *parent_name,
-		struct xfs_inode *new_ip, xfs_dir2_dataptr_t new_diroffset,
+		xfs_dir2_dataptr_t old_diroffset,
+		const struct xfs_name *parent_name, struct xfs_inode *new_ip,
+		xfs_dir2_dataptr_t new_diroffset, struct xfs_inode *child);
+int xfs_parent_remove(struct xfs_trans *tp, struct xfs_inode *dp,
+		struct xfs_parent_defer *parent, xfs_dir2_dataptr_t diroffset,
 		struct xfs_inode *child);
-int xfs_parent_defer_remove(struct xfs_trans *tp, struct xfs_inode *dp,
-			    struct xfs_parent_defer *parent,
-			    xfs_dir2_dataptr_t diroffset,
-			    struct xfs_inode *child);
 
 void __xfs_parent_cancel(struct xfs_mount *mp, struct xfs_parent_defer *parent);
 
fs/xfs/xfs_inode.c +8 -8
index ce1f6d03c3a9..09b0ac6b99cb 100644
@@ -1202,7 +1202,7 @@ xfs_create(
 	 * the parent information now.
 	 */
 	if (parent) {
-		error = xfs_parent_defer_add(tp, parent, dp, name, diroffset,
+		error = xfs_parent_add(tp, parent, dp, name, diroffset,
 					     ip);
 		if (error)
 			goto out_trans_cancel;
@@ -1477,7 +1477,7 @@ xfs_link(
 	 * the parent to the inode.
 	 */
 	if (parent) {
-		error = xfs_parent_defer_add(tp, parent, tdp, target_name,
+		error = xfs_parent_add(tp, parent, tdp, target_name,
 					     diroffset, sip);
 		if (error)
 			goto error_return;
@@ -2750,7 +2750,7 @@ xfs_remove(
 	}
 
 	if (parent) {
-		error = xfs_parent_defer_remove(tp, dp, parent, dir_offset, ip);
+		error = xfs_parent_remove(tp, dp, parent, dir_offset, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
@@ -3061,12 +3061,12 @@ xfs_cross_rename(
 	}
 
 	if (xfs_has_parent(mp)) {
-		error = xfs_parent_defer_replace(tp, ip1_pptr, dp1,
+		error = xfs_parent_replace(tp, ip1_pptr, dp1,
 				old_diroffset, name2, dp2, new_diroffset, ip1);
 		if (error)
 			goto out_trans_abort;
 
-		error = xfs_parent_defer_replace(tp, ip2_pptr, dp2,
+		error = xfs_parent_replace(tp, ip2_pptr, dp2,
 				new_diroffset, name1, dp1, old_diroffset, ip2);
 		if (error)
 			goto out_trans_abort;
@@ -3540,7 +3540,7 @@ xfs_rename(
 		goto out_trans_cancel;
 
 	if (wip_pptr) {
-		error = xfs_parent_defer_add(tp, wip_pptr,
+		error = xfs_parent_add(tp, wip_pptr,
 					     src_dp, src_name,
 					     old_diroffset, wip);
 		if (error)
@@ -3548,7 +3548,7 @@ xfs_rename(
 	}
 
 	if (src_ip_pptr) {
-		error = xfs_parent_defer_replace(tp, src_ip_pptr, src_dp,
+		error = xfs_parent_replace(tp, src_ip_pptr, src_dp,
 				old_diroffset, target_name, target_dp,
 				new_diroffset, src_ip);
 		if (error)
@@ -3556,7 +3556,7 @@ xfs_rename(
 	}
 
 	if (tgt_ip_pptr) {
-		error = xfs_parent_defer_remove(tp, target_dp,
+		error = xfs_parent_remove(tp, target_dp,
 						tgt_ip_pptr,
 						new_diroffset, target_ip);
 		if (error)
fs/xfs/xfs_symlink.c +1 -1
index fdfaab466f5d..63e68e832551 100644
@@ -348,7 +348,7 @@ xfs_symlink(
 	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	if (parent) {
-		error = xfs_parent_defer_add(tp, parent, dp, link_name,
+		error = xfs_parent_add(tp, parent, dp, link_name,
 					     diroffset, ip);
 		if (error)
 			goto out_trans_cancel;
#67 lore raw

PATCH 4/7 xfs: rearrange bits of the parent pointer apis for fsck

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rearrange parts of this thing in preparation for fsck code.

---
 fs/xfs/libxfs/xfs_da_format.h |   11 +++++++++++
 fs/xfs/libxfs/xfs_parent.c    |   29 ++++++++++++++++++++++++++++-
 fs/xfs/libxfs/xfs_parent.h    |    6 ++----
 3 files changed, 41 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +11 -0
index 2db1cf97b2c8..c07b8166e8ff 100644
@@ -159,6 +159,17 @@ struct xfs_da3_intnode {
 
 #define XFS_DIR3_FT_MAX			9
 
+#define XFS_DIR3_FTYPE_STR \
+	{ XFS_DIR3_FT_UNKNOWN,	"unknown" }, \
+	{ XFS_DIR3_FT_REG_FILE,	"file" }, \
+	{ XFS_DIR3_FT_DIR,	"directory" }, \
+	{ XFS_DIR3_FT_CHRDEV,	"char" }, \
+	{ XFS_DIR3_FT_BLKDEV,	"block" }, \
+	{ XFS_DIR3_FT_FIFO,	"fifo" }, \
+	{ XFS_DIR3_FT_SOCK,	"sock" }, \
+	{ XFS_DIR3_FT_SYMLINK,	"symlink" }, \
+	{ XFS_DIR3_FT_WHT,	"whiteout" }
+
 /*
  * Byte offset in data block and shortform entry.
  */
fs/xfs/libxfs/xfs_parent.c +28 -1
index ec2bff195773..fe6d4d1a7d57 100644
@@ -91,7 +91,7 @@ xfs_parent_valuecheck(
 }
 
 /* Initializes a xfs_parent_name_rec to be stored as an attribute name */
-void
+static inline void
 xfs_init_parent_name_rec(
 	struct xfs_parent_name_rec	*rec,
 	struct xfs_inode		*ip,
@@ -135,6 +135,33 @@ xfs_parent_irec_from_disk(
 	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
 }
 
+/*
+ * Convert an incore parent_name record to its ondisk format.  If @value or
+ * @valuelen are NULL, they will not be written to.
+ */
+void
+xfs_parent_irec_to_disk(
+	struct xfs_parent_name_rec	*rec,
+	void				*value,
+	int				*valuelen,
+	const struct xfs_parent_name_irec *irec)
+{
+	rec->p_ino = cpu_to_be64(irec->p_ino);
+	rec->p_gen = cpu_to_be32(irec->p_gen);
+	rec->p_diroffset = cpu_to_be32(irec->p_diroffset);
+
+	if (valuelen) {
+		ASSERT(*valuelen > 0);
+		ASSERT(*valuelen >= irec->p_namelen);
+		ASSERT(*valuelen < MAXNAMELEN);
+
+		*valuelen = irec->p_namelen;
+	}
+
+	if (value)
+		memcpy(value, irec->p_name, irec->p_namelen);
+}
+
 /*
  * Allocate memory to control a logged parent pointer update as part of a
  * dirent operation.
fs/xfs/libxfs/xfs_parent.h +2 -4
index 35854e968f1d..4eb92fb4b11b 100644
@@ -33,6 +33,8 @@ struct xfs_parent_name_irec {
 void xfs_parent_irec_from_disk(struct xfs_parent_name_irec *irec,
 		const struct xfs_parent_name_rec *rec,
 		const void *value, int valuelen);
+void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, void *value,
+		int *valuelen, const struct xfs_parent_name_irec *irec);
 
 /*
  * Dynamically allocd structure used to wrap the needed data to pass around
@@ -48,10 +50,6 @@ struct xfs_parent_defer {
 /*
  * Parent pointer attribute prototypes
  */
-void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
-		struct xfs_inode *ip, uint32_t p_diroffset);
-void xfs_init_parent_name_irec(struct xfs_parent_name_irec *irec,
-			       struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
#68 lore raw

PATCH 5/7 xfs: reconstruct directories from parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use the filesystem scanning infrastructure to walk the filesystem looking for parent pointers and child dirents that reference the directory that we're rebuilding.

---
 fs/xfs/Makefile           |    1 
 fs/xfs/scrub/common.c     |   15 +
 fs/xfs/scrub/common.h     |   28 +
 fs/xfs/scrub/dir.c        |    9 
 fs/xfs/scrub/dir_repair.c |  940 +++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h     |   16 +
 fs/xfs/scrub/scrub.c      |    2 
 fs/xfs/scrub/tempfile.c   |   42 ++
 fs/xfs/scrub/tempfile.h   |    2 
 fs/xfs/scrub/trace.c      |    1 
 fs/xfs/scrub/trace.h      |   67 +++
 11 files changed, 1122 insertions(+), 1 deletion(-)
 create mode 100644 fs/xfs/scrub/dir_repair.c
fs/xfs/Makefile +1 -0
index 6a30b145491d..a32f6da27a86 100644
@@ -178,6 +178,7 @@ xfs-$(CONFIG_XFS_QUOTA)		+= scrub/quota.o
 ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
+				   dir_repair.o \
 				   repair.o \
 				   tempfile.o \
 				   xfblob.o \
fs/xfs/scrub/common.c +15 -0
index 2874da088e8d..17a9bc610a76 100644
@@ -551,6 +551,21 @@ xchk_ag_init(
 
 /* Per-scrubber setup functions */
 
+void
+xchk_trans_cancel(
+	struct xfs_scrub	*sc)
+{
+	xfs_trans_cancel(sc->tp);
+	sc->tp = NULL;
+}
+
+int
+xchk_trans_alloc_empty(
+	struct xfs_scrub	*sc)
+{
+	return xfs_trans_alloc_empty(sc->mp, &sc->tp);
+}
+
 /*
  * Grab an empty transaction so that we can re-grab locked buffers if
  * one of our btrees turns out to be cyclic.
fs/xfs/scrub/common.h +28 -0
index 423a98c39fb6..7720982adfc6 100644
@@ -31,6 +31,9 @@ xchk_should_terminate(
 	return false;
 }
 
+void xchk_trans_cancel(struct xfs_scrub *sc);
+int xchk_trans_alloc_empty(struct xfs_scrub *sc);
+
 int xchk_trans_alloc(struct xfs_scrub *sc, uint resblks);
 bool xchk_process_error(struct xfs_scrub *sc, xfs_agnumber_t agno,
 		xfs_agblock_t bno, int *error);
@@ -159,4 +162,29 @@ void xchk_start_reaping(struct xfs_scrub *sc);
 
 void xchk_fshooks_enable(struct xfs_scrub *sc, unsigned int scrub_fshooks);
 
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+/* Decide if a repair is required. */
+static inline bool xchk_needs_repair(const struct xfs_scrub_metadata *sm)
+{
+	return sm->sm_flags & (XFS_SCRUB_OFLAG_CORRUPT |
+			       XFS_SCRUB_OFLAG_XCORRUPT |
+			       XFS_SCRUB_OFLAG_PREEN);
+}
+
+/*
+ * "Should we prepare for a repair?"
+ *
+ * Return true if the caller permits us to repair metadata and we're not
+ * setting up for a post-repair evaluation.
+ */
+static inline bool xchk_could_repair(const struct xfs_scrub *sc)
+{
+	return (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR) &&
+		!(sc->flags & XREP_ALREADY_FIXED);
+}
+#else
+# define xchk_needs_repair(sc)		(false)
+# define xchk_could_repair(sc)		(false)
+#endif /* CONFIG_XFS_ONLINE_REPAIR */
+
 #endif	/* __XFS_SCRUB_COMMON_H__ */
fs/xfs/scrub/dir.c +9 -0
index 06783e4b95ad..d720f1e143dd 100644
@@ -19,12 +19,21 @@
 #include "scrub/common.h"
 #include "scrub/dabtree.h"
 #include "scrub/readdir.h"
+#include "scrub/repair.h"
 
 /* Set us up to scrub directories. */
 int
 xchk_setup_directory(
 	struct xfs_scrub	*sc)
 {
+	int			error;
+
+	if (xchk_could_repair(sc)) {
+		error = xrep_setup_directory(sc);
+		if (error)
+			return error;
+	}
+
 	return xchk_setup_inode_contents(sc, 0);
 }
 
fs/xfs/scrub/dir_repair.c new file +940 -0

940 lines not on this page · raw message

fs/xfs/scrub/repair.h +16 -0
index 840f74ec431c..ff254ff9b86d 100644
@@ -30,6 +30,16 @@ int xrep_init_btblock(struct xfs_scrub *sc, xfs_fsblock_t fsb,
 		struct xfs_buf **bpp, xfs_btnum_t btnum,
 		const struct xfs_buf_ops *ops);
 
+static inline int
+xrep_trans_commit(
+	struct xfs_scrub	*sc)
+{
+	int			error = xfs_trans_commit(sc->tp);
+
+	sc->tp = NULL;
+	return error;
+}
+ 
 struct xbitmap;
 
 int xrep_fix_freelist(struct xfs_scrub *sc, bool can_shrink);
@@ -57,6 +67,8 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,
 void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);
 int xrep_ino_dqattach(struct xfs_scrub *sc);
 
+int xrep_setup_directory(struct xfs_scrub *sc);
+
 /* Metadata repairers */
 
 int xrep_probe(struct xfs_scrub *sc);
@@ -64,6 +76,7 @@ int xrep_superblock(struct xfs_scrub *sc);
 int xrep_agf(struct xfs_scrub *sc);
 int xrep_agfl(struct xfs_scrub *sc);
 int xrep_agi(struct xfs_scrub *sc);
+int xrep_directory(struct xfs_scrub *sc);
 
 #else
 
@@ -83,11 +96,14 @@ xrep_calc_ag_resblks(
 	return 0;
 }
 
+#define xrep_setup_directory(sc)	(0)
+
 #define xrep_probe			xrep_notsupported
 #define xrep_superblock			xrep_notsupported
 #define xrep_agf			xrep_notsupported
 #define xrep_agfl			xrep_notsupported
 #define xrep_agi			xrep_notsupported
+#define xrep_directory			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +1 -1
index a19ea7fdd510..b2a8de449d11 100644
@@ -299,7 +299,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_INODE,
 		.setup	= xchk_setup_directory,
 		.scrub	= xchk_directory,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_directory,
 	},
 	[XFS_SCRUB_TYPE_XATTR] = {	/* extended attributes */
 		.type	= ST_INODE,
fs/xfs/scrub/tempfile.c +42 -0

66 lines not on this page · raw message

fs/xfs/scrub/tempfile.h +2 -0
index e2f493b5d3d9..1e61d8e1ddce 100644
@@ -11,10 +11,12 @@ int xrep_tempfile_create(struct xfs_scrub *sc, uint16_t mode);
 void xrep_tempfile_rele(struct xfs_scrub *sc);
 
 bool xrep_tempfile_iolock_nowait(struct xfs_scrub *sc);
+int xrep_tempfile_iolock_polled(struct xfs_scrub *sc);
 void xrep_tempfile_iounlock(struct xfs_scrub *sc);
 
 void xrep_tempfile_ilock(struct xfs_scrub *sc);
 bool xrep_tempfile_ilock_nowait(struct xfs_scrub *sc);
+int xrep_tempfile_ilock_polled(struct xfs_scrub *sc);
 void xrep_tempfile_iunlock(struct xfs_scrub *sc);
 bool xrep_is_tempfile(const struct xfs_inode *ip);
 #else
fs/xfs/scrub/trace.c +1 -0
index 83e8a64c95d4..61b51617fbb4 100644
@@ -17,6 +17,7 @@
 #include "scrub/xfile.h"
 #include "scrub/xfarray.h"
 #include "scrub/iscan.h"
+#include "xfs_da_format.h"
 
 /* Figure out which block the btree cursor was pointing to. */
 static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +67 -0

73 lines not on this page · raw message

#69 lore raw

PATCH 6/7 xfs: add hooks to do directory updates

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

While we're scanning the filesystem, we still need to keep the tempdir up to date with whatever changes get made to the you know what.

---
 fs/xfs/libxfs/xfs_dir2.c  |    2 -
 fs/xfs/libxfs/xfs_dir2.h  |    2 -
 fs/xfs/scrub/dir_repair.c |   94 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 96 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_dir2.c +1 -1
index 27e408d20d18..8bed71a5e9cc 100644
@@ -440,7 +440,7 @@ int
 xfs_dir_removename(
 	struct xfs_trans	*tp,
 	struct xfs_inode	*dp,
-	struct xfs_name		*name,
+	const struct xfs_name	*name,
 	xfs_ino_t		ino,
 	xfs_extlen_t		total,		/* bmap's total block count */
 	xfs_dir2_dataptr_t	*offset)	/* OUT: offset in directory */
fs/xfs/libxfs/xfs_dir2.h +1 -1
index ac360c0b2fe7..6ed86b7bd13c 100644
@@ -46,7 +46,7 @@ extern int xfs_dir_lookup(struct xfs_trans *tp, struct xfs_inode *dp,
 				const struct xfs_name *name, xfs_ino_t *inum,
 				struct xfs_name *ci_name);
 extern int xfs_dir_removename(struct xfs_trans *tp, struct xfs_inode *dp,
-				struct xfs_name *name, xfs_ino_t ino,
+				const struct xfs_name *name, xfs_ino_t ino,
 				xfs_extlen_t tot,
 				xfs_dir2_dataptr_t *offset);
 extern int xfs_dir_replace(struct xfs_trans *tp, struct xfs_inode *dp,
fs/xfs/scrub/dir_repair.c +94 -0

132 lines not on this page · raw message

#70 lore raw

PATCH 7/7 xfs: compare generated and existing dirents

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Check our work to make sure we found all the dirents that the original directory had.

---
 fs/xfs/scrub/dir_repair.c |  101 ++++++++++++++++++++++++++++++++++++++++++++-
 fs/xfs/scrub/trace.h      |    2 +
 2 files changed, 100 insertions(+), 3 deletions(-)
fs/xfs/scrub/dir_repair.c +98 -3
index 25af002df1da..ec48b3268809 100644
@@ -757,7 +757,10 @@ xrep_dir_scan_dirtree(
 	return 0;
 }
 
-/* Dump a dirent from the temporary dir. */
+/*
+ * Dump a dirent from the temporary dir and check it against the dir we're
+ * rebuilding.  We are not committing any of this.
+ */
 STATIC int
 xrep_dir_dump_tempdir(
 	struct xfs_scrub	*sc,
@@ -768,7 +771,9 @@ xrep_dir_dump_tempdir(
 	void			*priv)
 {
 	struct xrep_dir		*rd = priv;
+	xfs_ino_t		child_ino;
 	bool			child = true;
+	xfs_dir2_dataptr_t	child_diroffset = XFS_DIR2_NULL_DATAPTR;
 	int			error;
 
 	/*
@@ -809,7 +814,88 @@ xrep_dir_dump_tempdir(
 	mutex_lock(&rd->lock);
 	error = xrep_dir_remove_dirent(rd, name, ino, dapos);
 	mutex_unlock(&rd->lock);
-	return error;
+	if (error)
+		return error;
+
+	/* Check that the dir being repaired has the same entry. */
+	error = xchk_dir_lookup(sc, sc->ip, name, &child_ino,
+			&child_diroffset);
+	if (error == -ENOENT) {
+		trace_xrep_dir_checkname(sc->ip, name, NULLFSINO,
+				XFS_DIR2_NULL_DATAPTR);
+		ASSERT(error != -ENOENT);
+		return -EFSCORRUPTED;
+	}
+	if (error)
+		return error;
+
+	if (ino != child_ino) {
+		trace_xrep_dir_checkname(sc->ip, name, child_ino,
+				child_diroffset);
+		ASSERT(ino == child_ino);
+		return -EFSCORRUPTED;
+	}
+
+	if (dapos != child_diroffset) {
+		trace_xrep_dir_badposname(sc->ip, name, child_ino,
+				child_diroffset);
+		/* We have no way to update this, so we just leave it. */
+	}
+
+	return 0;
+}
+
+/*
+ * Dump a dirent from the dir we're rebuilding and check it against the
+ * temporary dir.  This assumes that the directory wasn't really corrupt to
+ * begin with.
+ */
+STATIC int
+xrep_dir_dump_baddir(
+	struct xfs_scrub	*sc,
+	struct xfs_inode	*dp,
+	xfs_dir2_dataptr_t	dapos,
+	const struct xfs_name	*name,
+	xfs_ino_t		ino,
+	void			*priv)
+{
+	xfs_ino_t		child_ino;
+	xfs_dir2_dataptr_t	child_diroffset = XFS_DIR2_NULL_DATAPTR;
+	int			error;
+
+	/* Ignore the directory's dot and dotdot entries. */
+	if (xrep_dir_samename(name, &xfs_name_dotdot) ||
+	    xrep_dir_samename(name, &xfs_name_dot))
+		return 0;
+
+	trace_xrep_dir_dumpname(sc->ip, name, ino, dapos);
+
+	/* Check that the tempdir has the same entry. */
+	error = xchk_dir_lookup(sc, sc->tempip, name, &child_ino,
+			&child_diroffset);
+	if (error == -ENOENT) {
+		trace_xrep_dir_checkname(sc->tempip, name, NULLFSINO,
+				XFS_DIR2_NULL_DATAPTR);
+		ASSERT(error != -ENOENT);
+		return -EFSCORRUPTED;
+	}
+	if (error)
+		return error;
+
+	if (ino != child_ino) {
+		trace_xrep_dir_checkname(sc->tempip, name, child_ino,
+				child_diroffset);
+		ASSERT(ino == child_ino);
+		return -EFSCORRUPTED;
+	}
+
+	if (dapos != child_diroffset) {
+		trace_xrep_dir_badposname(sc->ip, name, child_ino,
+				child_diroffset);
+		/* We have no way to update this, so we just leave it. */
+	}
+
+	return 0;
 }
 
 /*
@@ -876,12 +962,21 @@ xrep_dir_rebuild_tree(
 
 	trace_xrep_dir_rebuild_tree(sc->ip, rd->parent_ino);
 
-	xrep_tempfile_ilock(sc);
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
+	error = xrep_tempfile_ilock_polled(sc);
+	if (error)
+		return error;
+
 	error = xchk_dir_walk(sc, sc->tempip, xrep_dir_dump_tempdir, rd);
 	if (error)
 		return error;
 
+	error = xchk_dir_walk(sc, sc->ip, xrep_dir_dump_baddir, rd);
+	if (error)
+		return error;
+
 	xrep_tempfile_iunlock(sc);
+	xchk_iunlock(sc, XFS_ILOCK_EXCL);
 	xchk_trans_cancel(sc);
 
 	return xrep_dir_replay_updates(rd);
fs/xfs/scrub/trace.h +2 -0
index cbf914bce6db..81d26be0ef3b 100644
@@ -1224,6 +1224,8 @@ DEFINE_XREP_DIRENT_CLASS(xrep_dir_createname);
 DEFINE_XREP_DIRENT_CLASS(xrep_dir_removename);
 DEFINE_XREP_DIRENT_CLASS(xrep_dir_replacename);
 DEFINE_XREP_DIRENT_CLASS(xrep_dir_dumpname);
+DEFINE_XREP_DIRENT_CLASS(xrep_dir_checkname);
+DEFINE_XREP_DIRENT_CLASS(xrep_dir_badposname);
 
 DECLARE_EVENT_CLASS(xrep_dir_class,
 	TP_PROTO(struct xfs_inode *dp, xfs_ino_t parent_ino),
#71 lore raw

cover xfs: online checking of parent pointers

In reply to Darrick J. Wong

Hi all,

Update the existing online parent pointer checker to confirm the directory entries that should also exist.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-check

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-parent-check

---
 fs/xfs/Makefile            |    2 
 fs/xfs/libxfs/xfs_parent.c |   38 +++
 fs/xfs/libxfs/xfs_parent.h |   10 +
 fs/xfs/scrub/parent.c      |  529 ++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/trace.h       |   66 +++++
 5 files changed, 644 insertions(+), 1 deletion(-)
#72 lore raw

PATCH 1/2 xfs: scrub parent pointers

From: Darrick J. Wong <[email protected]>

Actually check parent pointers now.

---
 fs/xfs/scrub/parent.c |  291 +++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/trace.h  |   33 ++++++
 2 files changed, 324 insertions(+)
fs/xfs/scrub/parent.c +291 -0
index d59184a59671..1bb196f2c1b2 100644
@@ -14,9 +14,13 @@
 #include "xfs_icache.h"
 #include "xfs_dir2.h"
 #include "xfs_dir2_priv.h"
+#include "xfs_attr.h"
+#include "xfs_parent.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 #include "scrub/readdir.h"
+#include "scrub/listxattr.h"
+#include "scrub/trace.h"
 
 /* Set us up to scrub parents. */
 int
@@ -231,6 +235,8 @@ xchk_parent_validate(
 	return error;
 }
 
+STATIC int xchk_parent_pptr(struct xfs_scrub *sc);
+
 /* Scrub a parent pointer. */
 int
 xchk_parent(
@@ -240,6 +246,9 @@ xchk_parent(
 	xfs_ino_t		parent_ino;
 	int			error;
 
+	if (xfs_has_parent(mp))
+		return xchk_parent_pptr(sc);
+
 	/*
 	 * If we're a directory, check that the '..' link points up to
 	 * a directory that has one entry pointing to us.
@@ -282,3 +291,285 @@ xchk_parent(
 
 	return xchk_parent_validate(sc, parent_ino);
 }
+
+/*
+ * Checking of Parent Pointers
+ * ===========================
+ *
+ * On filesystems with directory parent pointers, we check the referential
+ * integrity by visiting each parent pointer of a child file and checking that
+ * the directory referenced by the pointer actually has a dirent pointing back
+ * to the child file.
+ */
+
+struct xchk_pptrs {
+	struct xfs_scrub	*sc;
+
+	/* Scratch buffer for scanning pptr xattrs */
+	struct xfs_parent_name_irec pptr;
+
+	/* Parent of this directory. */
+	xfs_ino_t		parent_ino;
+};
+
+/* Look up the dotdot entry so that we can check it as we walk the pptrs. */
+STATIC int
+xchk_parent_dotdot(
+	struct xchk_pptrs	*pp)
+{
+	struct xfs_scrub	*sc = pp->sc;
+	int			error;
+
+	if (!S_ISDIR(VFS_I(sc->ip)->i_mode)) {
+		pp->parent_ino = NULLFSINO;
+		return 0;
+	}
+
+	/* Look up '..' */
+	error = xchk_dir_lookup(sc, sc->ip, &xfs_name_dotdot, &pp->parent_ino,
+			NULL);
+	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, 0, &error))
+		return error;
+	if (!xfs_verify_dir_ino(sc->mp, pp->parent_ino)) {
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, 0);
+		return 0;
+	}
+
+	/* Is this the root dir?  Then '..' must point to itself. */
+	if (sc->ip == sc->mp->m_rootip && sc->ip->i_ino != pp->parent_ino)
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, 0);
+
+	return 0;
+}
+
+/*
+ * Try to lock a parent directory for checking dirents.  Returns the inode
+ * flags for the locks we now hold, or zero if we failed.
+ */
+STATIC unsigned int
+xchk_parent_lock_dir(
+	struct xfs_scrub	*sc,
+	struct xfs_inode	*dp)
+{
+	if (!xfs_ilock_nowait(dp, XFS_IOLOCK_SHARED))
+		return 0;
+
+	if (!xfs_ilock_nowait(dp, XFS_ILOCK_SHARED)) {
+		xfs_iunlock(dp, XFS_IOLOCK_SHARED);
+		return 0;
+	}
+
+	if (!xfs_need_iread_extents(&dp->i_df))
+		return XFS_IOLOCK_SHARED | XFS_ILOCK_SHARED;
+
+	xfs_iunlock(dp, XFS_ILOCK_SHARED);
+
+	if (!xfs_ilock_nowait(dp, XFS_ILOCK_EXCL)) {
+		xfs_iunlock(dp, XFS_IOLOCK_SHARED);
+		return 0;
+	}
+
+	return XFS_IOLOCK_SHARED | XFS_ILOCK_EXCL;
+}
+
+/* Check the forward link (dirent) associated with this parent pointer. */
+STATIC int
+xchk_parent_dirent(
+	struct xchk_pptrs	*pp,
+	struct xfs_inode	*dp)
+{
+	struct xfs_name		xname = {
+		.name		= pp->pptr.p_name,
+		.len		= pp->pptr.p_namelen,
+	};
+	struct xfs_scrub	*sc = pp->sc;
+	xfs_ino_t		child_ino;
+	xfs_dir2_dataptr_t	child_diroffset;
+	int			error;
+
+	/*
+	 * Use the name attached to this parent pointer to look up the
+	 * directory entry in the alleged parent.
+	 */
+	error = xchk_dir_lookup(sc, dp, &xname, &child_ino, &child_diroffset);
+	if (error == -ENOENT) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return 0;
+	}
+	if (!xchk_fblock_xref_process_error(sc, XFS_ATTR_FORK, 0, &error))
+		return error;
+
+	/* Does the inode number match? */
+	if (child_ino != sc->ip->i_ino) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return 0;
+	}
+
+	/* Does the directory offset match? */
+	if (pp->pptr.p_diroffset != child_diroffset) {
+		trace_xchk_parent_bad_dapos(sc->ip, pp->pptr.p_diroffset,
+				dp->i_ino, child_diroffset, xname.name,
+				xname.len);
+		xchk_fblock_xref_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return 0;
+	}
+
+	/*
+	 * If we're scanning a directory, we should only ever encounter a
+	 * single parent pointer, and it should match the dotdot entry.  We set
+	 * the parent_ino from the dotdot entry before the scan, so compare it
+	 * now.
+	 */
+	if (!S_ISDIR(VFS_I(sc->ip)->i_mode))
+		return 0;
+
+	if (pp->parent_ino != dp->i_ino) {
+		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return 0;
+	}
+
+	pp->parent_ino = NULLFSINO;
+	return 0;
+}
+
+/* Try to grab a parent directory. */
+STATIC int
+xchk_parent_iget(
+	struct xchk_pptrs		*pp,
+	struct xfs_inode		**dpp)
+{
+	struct xfs_scrub		*sc = pp->sc;
+	struct xfs_inode		*ip;
+	int				error;
+
+	/* Validate inode number. */
+	error = xfs_dir_ino_validate(sc->mp, pp->pptr.p_ino);
+	if (error) {
+		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return -ECANCELED;
+	}
+
+	error = xchk_iget(sc, pp->pptr.p_ino, &ip);
+	if (error == -EINVAL || error == -ENOENT) {
+		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return -ECANCELED;
+	}
+	if (!xchk_fblock_xref_process_error(sc, XFS_ATTR_FORK, 0, &error))
+		return error;
+
+	/* The parent must be a directory. */
+	if (!S_ISDIR(VFS_I(ip)->i_mode)) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		goto out_rele;
+	}
+
+	/* Validate generation number. */
+	if (VFS_I(ip)->i_generation != pp->pptr.p_gen) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		goto out_rele;
+	}
+
+	*dpp = ip;
+	return 0;
+out_rele:
+	xchk_irele(sc, ip);
+	return 0;
+}
+
+/*
+ * Walk an xattr of a file.  If this xattr is a parent pointer, follow it up
+ * to a parent directory and check that the parent has a dirent pointing back
+ * to us.
+ */
+STATIC int
+xchk_parent_scan_attr(
+	struct xfs_scrub	*sc,
+	struct xfs_inode	*ip,
+	unsigned int		attr_flags,
+	const unsigned char	*name,
+	unsigned int		namelen,
+	const void		*value,
+	unsigned int		valuelen,
+	void			*priv)
+{
+	struct xchk_pptrs	*pp = priv;
+	struct xfs_inode	*dp = NULL;
+	const struct xfs_parent_name_rec *rec = (const void *)name;
+	unsigned int		lockmode;
+	int			error;
+
+	/* Ignore incomplete xattrs */
+	if (attr_flags & XFS_ATTR_INCOMPLETE)
+		return 0;
+
+	/* Ignore anything that isn't a parent pointer. */
+	if (!(attr_flags & XFS_ATTR_PARENT))
+		return 0;
+
+	/* Does the ondisk parent pointer structure make sense? */
+	if (!xfs_parent_namecheck(sc->mp, rec, namelen, attr_flags)) {
+		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return -ECANCELED;
+	}
+
+	if (!xfs_parent_valuecheck(sc->mp, value, valuelen)) {
+		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return -ECANCELED;
+	}
+
+	xfs_parent_irec_from_disk(&pp->pptr, rec, value, valuelen);
+
+	error = xchk_parent_iget(pp, &dp);
+	if (error)
+		return error;
+	if (!dp)
+		return 0;
+
+	/* Try to lock the inode. */
+	lockmode = xchk_parent_lock_dir(sc, dp);
+	if (!lockmode) {
+		xchk_set_incomplete(sc);
+		error = -ECANCELED;
+		goto out_rele;
+	}
+
+	error = xchk_parent_dirent(pp, dp);
+	if (error)
+		goto out_unlock;
+
+out_unlock:
+	xfs_iunlock(dp, lockmode);
+out_rele:
+	xchk_irele(sc, dp);
+	return error;
+}
+
+/* Check parent pointers of a file. */
+STATIC int
+xchk_parent_pptr(
+	struct xfs_scrub	*sc)
+{
+	struct xchk_pptrs	*pp;
+	int			error;
+
+	pp = kvzalloc(sizeof(struct xchk_pptrs), XCHK_GFP_FLAGS);
+	if (!pp)
+		return -ENOMEM;
+	pp->sc = sc;
+
+	error = xchk_parent_dotdot(pp);
+	if (error)
+		goto out_pp;
+
+	error = xchk_xattr_walk(sc, sc->ip, xchk_parent_scan_attr, pp);
+	if (error == -ECANCELED) {
+		error = 0;
+		goto out_pp;
+	}
+	if (error)
+		goto out_pp;
+
+out_pp:
+	kvfree(pp);
+	return error;
+}
fs/xfs/scrub/trace.h +33 -0
index 81d26be0ef3b..ac21759fc3e1 100644
@@ -896,6 +896,39 @@ TRACE_EVENT(xchk_nlinks_live_update,
 		  __get_str(name))
 );
 
+TRACE_EVENT(xchk_parent_bad_dapos,
+	TP_PROTO(struct xfs_inode *ip, unsigned int p_diroffset,
+		 xfs_ino_t parent_ino, unsigned int dapos,
+		 const char *name, unsigned int namelen),
+	TP_ARGS(ip, p_diroffset, parent_ino, dapos, name, namelen),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_ino_t, ino)
+		__field(unsigned int, p_diroffset)
+		__field(xfs_ino_t, parent_ino)
+		__field(unsigned int, dapos)
+		__field(unsigned int, namelen)
+		__dynamic_array(char, name, namelen)
+	),
+	TP_fast_assign(
+		__entry->dev = ip->i_mount->m_super->s_dev;
+		__entry->ino = ip->i_ino;
+		__entry->p_diroffset = p_diroffset;
+		__entry->parent_ino = parent_ino;
+		__entry->dapos = dapos;
+		__entry->namelen = namelen;
+		memcpy(__get_str(name), name, namelen);
+	),
+	TP_printk("dev %d:%d ino 0x%llx p_diroff 0x%x parent_ino 0x%llx parent_diroff 0x%x name '%.*s'",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino,
+		  __entry->p_diroffset,
+		  __entry->parent_ino,
+		  __entry->dapos,
+		  __entry->namelen,
+		  __get_str(name))
+);
+
 /* repair tracepoints */
 #if IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR)
 
#73 lore raw

PATCH 2/2 xfs: deferred scrub of parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the trylock-based dirent check fails, retain those parent pointers and check them at the end. This may involve dropping the locks on the file being scanned, so yay.

---
 fs/xfs/Makefile            |    2 
 fs/xfs/libxfs/xfs_parent.c |   38 +++++++
 fs/xfs/libxfs/xfs_parent.h |   10 ++
 fs/xfs/scrub/parent.c      |  246 +++++++++++++++++++++++++++++++++++++++++++-
 fs/xfs/scrub/trace.h       |   33 ++++++
 5 files changed, 324 insertions(+), 5 deletions(-)
fs/xfs/Makefile +1 -1
index a32f6da27a86..0a908382d033 100644
@@ -168,6 +168,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   scrub.o \
 				   symlink.o \
 				   xfarray.o \
+				   xfblob.o \
 				   xfile.o \
 				   )
 
@@ -181,7 +182,6 @@ xfs-y				+= $(addprefix scrub/, \
 				   dir_repair.o \
 				   repair.o \
 				   tempfile.o \
-				   xfblob.o \
 				   )
 endif
 endif
fs/xfs/libxfs/xfs_parent.c +38 -0
index fe6d4d1a7d57..36e1968337d5 100644
@@ -298,3 +298,41 @@ xfs_pptr_calc_space_res(
 	       XFS_NEXTENTADD_SPACE_RES(mp, namelen, XFS_ATTR_FORK);
 }
 
+/*
+ * Look up the @name associated with the parent pointer (@pptr) of @ip.  Caller
+ * must hold at least ILOCK_SHARED.  Returns the length of the dirent name, or
+ * a negative errno.  The scratchpad need not be initialized.
+ */
+int
+xfs_parent_lookup(
+	struct xfs_trans		*tp,
+	struct xfs_inode		*ip,
+	const struct xfs_parent_name_irec *pptr,
+	unsigned char			*name,
+	unsigned int			namelen,
+	struct xfs_parent_scratch	*scr)
+{
+	int				error;
+
+	xfs_parent_irec_to_disk(&scr->rec, NULL, NULL, pptr);
+
+	memset(&scr->args, 0, sizeof(struct xfs_da_args));
+	scr->args.attr_filter	= XFS_ATTR_PARENT;
+	scr->args.dp		= ip;
+	scr->args.geo		= ip->i_mount->m_attr_geo;
+	scr->args.name		= (const unsigned char *)&scr->rec;
+	scr->args.namelen	= sizeof(struct xfs_parent_name_rec);
+	scr->args.op_flags	= XFS_DA_OP_OKNOENT;
+	scr->args.trans		= tp;
+	scr->args.valuelen	= namelen;
+	scr->args.value		= name;
+	scr->args.whichfork	= XFS_ATTR_FORK;
+
+	scr->args.hashval = xfs_da_hashname(scr->args.name, scr->args.namelen);
+
+	error = xfs_attr_get_ilocked(&scr->args);
+	if (error)
+		return error;
+
+	return scr->args.valuelen;
+}
fs/xfs/libxfs/xfs_parent.h +10 -0
index 4eb92fb4b11b..cd1b135195a2 100644
@@ -103,4 +103,14 @@ xfs_parent_finish(
 unsigned int xfs_pptr_calc_space_res(struct xfs_mount *mp,
 				     unsigned int namelen);
 
+/* Scratchpad memory so that raw parent operations don't burn stack space. */
+struct xfs_parent_scratch {
+	struct xfs_parent_name_rec	rec;
+	struct xfs_da_args		args;
+};
+
+int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip,
+		const struct xfs_parent_name_irec *pptr, unsigned char *name,
+		unsigned int namelen, struct xfs_parent_scratch *scratch);
+
 #endif	/* __XFS_PARENT_H__ */
fs/xfs/scrub/parent.c +242 -4

290 lines not on this page · raw message

fs/xfs/scrub/trace.h +33 -0
index ac21759fc3e1..61f18632cb6f 100644
@@ -929,6 +929,39 @@ TRACE_EVENT(xchk_parent_bad_dapos,
 		  __get_str(name))
 );
 
+DECLARE_EVENT_CLASS(xchk_pptr_class,
+	TP_PROTO(struct xfs_inode *ip, const unsigned char *name,
+		 unsigned int namelen, xfs_ino_t parent_ino),
+	TP_ARGS(ip, name, namelen, parent_ino),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_ino_t, ino)
+		__field(unsigned int, namelen)
+		__dynamic_array(char, name, namelen)
+		__field(xfs_ino_t, parent_ino)
+	),
+	TP_fast_assign(
+		__entry->dev = ip->i_mount->m_super->s_dev;
+		__entry->ino = ip->i_ino;
+		__entry->namelen = namelen;
+		memcpy(__get_str(name), name, namelen);
+		__entry->parent_ino = parent_ino;
+	),
+	TP_printk("dev %d:%d ino 0x%llx name '%.*s' parent_ino 0x%llx",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino,
+		  __entry->namelen,
+		  __get_str(name),
+		  __entry->parent_ino)
+)
+#define DEFINE_XCHK_PPTR_CLASS(name) \
+DEFINE_EVENT(xchk_pptr_class, name, \
+	TP_PROTO(struct xfs_inode *ip, const unsigned char *name, \
+		 unsigned int namelen, xfs_ino_t parent_ino), \
+	TP_ARGS(ip, name, namelen, parent_ino))
+DEFINE_XCHK_PPTR_CLASS(xchk_parent_defer);
+DEFINE_XCHK_PPTR_CLASS(xchk_parent_slowpath);
+
 /* repair tracepoints */
 #if IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR)
 
#74 lore raw

cover xfs: online checking of parent pointers

In reply to Darrick J. Wong

Hi all,

With this patchset, we implement online repairs for parent pointers. This is structured similarly to the directory repair code in that we scan the entire filesystem looking for dirents and use them to reconstruct the parent pointer information.

Note that the atomic swapext and block reaping code is NOT ported for this PoC, so we do not commit any repairs.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-repair

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-online-parent-repair

---
 fs/xfs/Makefile              |    1 
 fs/xfs/libxfs/xfs_parent.c   |   56 +++
 fs/xfs/libxfs/xfs_parent.h   |    8 
 fs/xfs/scrub/parent.c        |   10 +
 fs/xfs/scrub/parent_repair.c |  739 ++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h        |    4 
 fs/xfs/scrub/scrub.c         |    2 
 fs/xfs/scrub/trace.c         |    2 
 fs/xfs/scrub/trace.h         |   80 +++++
 fs/xfs/xfs_inode.h           |    6 
 10 files changed, 905 insertions(+), 3 deletions(-)
 create mode 100644 fs/xfs/scrub/parent_repair.c
#75 lore raw

PATCH 1/3 xfs: repair parent pointers by scanning directories

From: Darrick J. Wong <[email protected]>

Walk the filesystem to rebuild parent pointer information.

---
 fs/xfs/Makefile              |    1 
 fs/xfs/libxfs/xfs_parent.c   |   31 ++
 fs/xfs/libxfs/xfs_parent.h   |    4 
 fs/xfs/scrub/parent.c        |   10 +
 fs/xfs/scrub/parent_repair.c |  583 ++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h        |    4 
 fs/xfs/scrub/scrub.c         |    2 
 fs/xfs/scrub/trace.c         |    2 
 fs/xfs/scrub/trace.h         |   77 ++++++
 fs/xfs/xfs_inode.h           |    6 
 10 files changed, 717 insertions(+), 3 deletions(-)
 create mode 100644 fs/xfs/scrub/parent_repair.c
fs/xfs/Makefile +1 -0
index 0a908382d033..f0d5a517ca00 100644
@@ -180,6 +180,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   dir_repair.o \
+				   parent_repair.o \
 				   repair.o \
 				   tempfile.o \
 				   )
fs/xfs/libxfs/xfs_parent.c +29 -2
index 36e1968337d5..5f07bd3debee 100644
@@ -94,11 +94,11 @@ xfs_parent_valuecheck(
 static inline void
 xfs_init_parent_name_rec(
 	struct xfs_parent_name_rec	*rec,
-	struct xfs_inode		*ip,
+	const struct xfs_inode		*ip,
 	uint32_t			p_diroffset)
 {
 	xfs_ino_t			p_ino = ip->i_ino;
-	uint32_t			p_gen = VFS_I(ip)->i_generation;
+	uint32_t			p_gen = VFS_IC(ip)->i_generation;
 
 	rec->p_ino = cpu_to_be64(p_ino);
 	rec->p_gen = cpu_to_be32(p_gen);
@@ -336,3 +336,30 @@ xfs_parent_lookup(
 
 	return scr->args.valuelen;
 }
+
+/*
+ * Attach the parent pointer (@pptr -> @name) to @ip immediately.  Caller must
+ * not have a transaction or hold the ILOCK.  The update will not use logged
+ * xattrs.  This is for specialized repair functions only.  The scratchpad need
+ * not be initialized.
+ */
+int
+xfs_parent_set(
+	struct xfs_inode		*ip,
+	const struct xfs_parent_name_irec *pptr,
+	struct xfs_parent_scratch	*scr)
+{
+	xfs_parent_irec_to_disk(&scr->rec, NULL, NULL, pptr);
+
+	memset(&scr->args, 0, sizeof(struct xfs_da_args));
+	scr->args.attr_filter	= XFS_ATTR_PARENT;
+	scr->args.dp		= ip;
+	scr->args.geo		= ip->i_mount->m_attr_geo;
+	scr->args.name		= (const unsigned char *)&scr->rec;
+	scr->args.namelen	= sizeof(struct xfs_parent_name_rec);
+	scr->args.valuelen	= pptr->p_namelen;
+	scr->args.value		= (void *)pptr->p_name;
+	scr->args.whichfork	= XFS_ATTR_FORK;
+
+	return xfs_attr_set(&scr->args);
+}
fs/xfs/libxfs/xfs_parent.h +4 -0
index cd1b135195a2..effbccdf6b0e 100644
@@ -113,4 +113,8 @@ int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip,
 		const struct xfs_parent_name_irec *pptr, unsigned char *name,
 		unsigned int namelen, struct xfs_parent_scratch *scratch);
 
+int xfs_parent_set(struct xfs_inode *ip,
+		const struct xfs_parent_name_irec *pptr,
+		struct xfs_parent_scratch *scratch);
+
 #endif	/* __XFS_PARENT_H__ */
fs/xfs/scrub/parent.c +10 -0
index 056e11337cec..14f16fefd1b0 100644
@@ -10,6 +10,7 @@
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
 #include "xfs_log_format.h"
+#include "xfs_trans.h"
 #include "xfs_inode.h"
 #include "xfs_icache.h"
 #include "xfs_dir2.h"
@@ -24,12 +25,21 @@
 #include "scrub/xfarray.h"
 #include "scrub/xfblob.h"
 #include "scrub/trace.h"
+#include "scrub/repair.h"
 
 /* Set us up to scrub parents. */
 int
 xchk_setup_parent(
 	struct xfs_scrub	*sc)
 {
+	int			error;
+
+	if (xchk_could_repair(sc)) {
+		error = xrep_setup_parent(sc);
+		if (error)
+			return error;
+	}
+
 	return xchk_setup_inode_contents(sc, 0);
 }
 
fs/xfs/scrub/parent_repair.c new file +583 -0

583 lines not on this page · raw message

fs/xfs/scrub/repair.h +4 -0
index ff254ff9b86d..cc42cf65ac92 100644
@@ -68,6 +68,7 @@ void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);
 int xrep_ino_dqattach(struct xfs_scrub *sc);
 
 int xrep_setup_directory(struct xfs_scrub *sc);
+int xrep_setup_parent(struct xfs_scrub *sc);
 
 /* Metadata repairers */
 
@@ -77,6 +78,7 @@ int xrep_agf(struct xfs_scrub *sc);
 int xrep_agfl(struct xfs_scrub *sc);
 int xrep_agi(struct xfs_scrub *sc);
 int xrep_directory(struct xfs_scrub *sc);
+int xrep_parent(struct xfs_scrub *sc);
 
 #else
 
@@ -97,6 +99,7 @@ xrep_calc_ag_resblks(
 }
 
 #define xrep_setup_directory(sc)	(0)
+#define xrep_setup_parent(sc)		(0)
 
 #define xrep_probe			xrep_notsupported
 #define xrep_superblock			xrep_notsupported
@@ -104,6 +107,7 @@ xrep_calc_ag_resblks(
 #define xrep_agfl			xrep_notsupported
 #define xrep_agi			xrep_notsupported
 #define xrep_directory			xrep_notsupported
+#define xrep_parent			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +1 -1
index b2a8de449d11..5ddb4dcff978 100644
@@ -317,7 +317,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_INODE,
 		.setup	= xchk_setup_parent,
 		.scrub	= xchk_parent,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_parent,
 	},
 	[XFS_SCRUB_TYPE_RTBITMAP] = {	/* realtime bitmap */
 		.type	= ST_FS,
fs/xfs/scrub/trace.c +2 -0
index 61b51617fbb4..30946b6a16dd 100644
@@ -18,6 +18,8 @@
 #include "scrub/xfarray.h"
 #include "scrub/iscan.h"
 #include "xfs_da_format.h"
+#include "xfs_da_btree.h"
+#include "xfs_parent.h"
 
 /* Figure out which block the btree cursor was pointing to. */
 static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +77 -0

89 lines not on this page · raw message

fs/xfs/xfs_inode.h +6 -0
index 403b0f4cb5c0..32efd4cb6507 100644
@@ -153,6 +153,12 @@ static inline struct inode *VFS_I(struct xfs_inode *ip)
 	return &ip->i_vnode;
 }
 
+/* convert from const xfs inode to vfs inode */
+static inline const struct inode *VFS_IC(const struct xfs_inode *ip)
+{
+	return &ip->i_vnode;
+}
+
 /*
  * For regular files we only update the on-disk filesize when actually
  * writing data back to disk.  Until then only the copy in the VFS inode
#76 lore raw

PATCH 2/3 xfs: repair parent pointers with live scan hooks

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use the nlink hooks to keep our tempfile's parent pointers up to date.

---
 fs/xfs/libxfs/xfs_parent.c   |   25 ++++++++++
 fs/xfs/libxfs/xfs_parent.h   |    4 ++
 fs/xfs/scrub/parent_repair.c |  110 ++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/trace.h         |    2 +
 4 files changed, 141 insertions(+)
fs/xfs/libxfs/xfs_parent.c +25 -0
index 5f07bd3debee..a2575bf44c89 100644
@@ -363,3 +363,28 @@ xfs_parent_set(
 
 	return xfs_attr_set(&scr->args);
 }
+
+/*
+ * Remove the parent pointer (@rec -> @name) from @ip immediately.  Caller must
+ * not have a transaction or hold the ILOCK.  The update will not use logged
+ * xattrs.  This is for specialized repair functions only.  The scratchpad need
+ * not be initialized.
+ */
+int
+xfs_parent_unset(
+	struct xfs_inode		*ip,
+	const struct xfs_parent_name_irec *pptr,
+	struct xfs_parent_scratch	*scr)
+{
+	xfs_parent_irec_to_disk(&scr->rec, NULL, NULL, pptr);
+
+	memset(&scr->args, 0, sizeof(struct xfs_da_args));
+	scr->args.attr_filter	= XFS_ATTR_PARENT;
+	scr->args.dp		= ip;
+	scr->args.geo		= ip->i_mount->m_attr_geo;
+	scr->args.name		= (const unsigned char *)&scr->rec;
+	scr->args.namelen	= sizeof(struct xfs_parent_name_rec);
+	scr->args.whichfork	= XFS_ATTR_FORK;
+
+	return xfs_attr_set(&scr->args);
+}
fs/xfs/libxfs/xfs_parent.h +4 -0
index effbccdf6b0e..a7fc621b82c4 100644
@@ -117,4 +117,8 @@ int xfs_parent_set(struct xfs_inode *ip,
 		const struct xfs_parent_name_irec *pptr,
 		struct xfs_parent_scratch *scratch);
 
+int xfs_parent_unset(struct xfs_inode *ip,
+		const struct xfs_parent_name_irec *rec,
+		struct xfs_parent_scratch *scratch);
+
 #endif	/* __XFS_PARENT_H__ */
fs/xfs/scrub/parent_repair.c +110 -0

160 lines not on this page · raw message

fs/xfs/scrub/trace.h +2 -0
index 1fb9832a113a..283a1cedf368 100644
@@ -1354,6 +1354,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \
 	TP_PROTO(struct xfs_inode *ip, const struct xfs_parent_name_irec *pptr), \
 	TP_ARGS(ip, pptr))
 DEFINE_XREP_PPTR_CLASS(xrep_pptr_createname);
+DEFINE_XREP_PPTR_CLASS(xrep_pptr_removename);
 DEFINE_XREP_PPTR_CLASS(xrep_pptr_dumpname);
 
 DECLARE_EVENT_CLASS(xrep_pptr_scan_class,
@@ -1393,6 +1394,7 @@ DEFINE_EVENT(xrep_pptr_scan_class, name, \
 		 unsigned int diroffset, const struct xfs_name *name), \
 	TP_ARGS(ip, dp, diroffset, name))
 DEFINE_XREP_PPTR_SCAN_CLASS(xrep_pptr_add_pointer);
+DEFINE_XREP_PPTR_SCAN_CLASS(xrep_pptr_remove_pointer);
 
 #endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
 
#77 lore raw

PATCH 3/3 xfs: compare generated and existing parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Check our work to make sure we found all the parent pointers that the original file had.

---
 fs/xfs/scrub/parent_repair.c |   52 ++++++++++++++++++++++++++++++++++++++++--
 fs/xfs/scrub/trace.h         |    1 +
 2 files changed, 50 insertions(+), 3 deletions(-)
fs/xfs/scrub/parent_repair.c +49 -3
index 4aec32081c6d..56b47bf2807b 100644
@@ -127,6 +127,9 @@ struct xrep_pptrs {
 
 	/* Parent pointer names. */
 	struct xfblob		*pptr_names;
+
+	/* Buffer for validation. */
+	unsigned char		namebuf[MAXNAMELEN];
 };
 
 /* Tear down all the incore stuff we created. */
@@ -490,7 +493,10 @@ xrep_pptr_scan_dirtree(
 	return 0;
 }
 
-/* Dump a parent pointer from the temporary file. */
+/*
+ * Dump a parent pointer from the temporary file and check it against the file
+ * we're rebuilding.  We are not committing any of this.
+ */
 STATIC int
 xrep_pptr_dump_tempptr(
 	struct xfs_scrub	*sc,
@@ -504,13 +510,45 @@ xrep_pptr_dump_tempptr(
 {
 	struct xrep_pptrs	*rp = priv;
 	const struct xfs_parent_name_rec *rec = (const void *)name;
+	struct xfs_inode	*other_ip;
+	int			pptr_namelen;
 
 	if (!(attr_flags & XFS_ATTR_PARENT))
 		return 0;
 
+	if (ip == sc->ip)
+		other_ip = sc->tempip;
+	else if (ip == sc->tempip)
+		other_ip = sc->ip;
+	else
+		return -EFSCORRUPTED;
+
 	xfs_parent_irec_from_disk(&rp->pptr, rec, value, valuelen);
 
 	trace_xrep_pptr_dumpname(sc->tempip, &rp->pptr);
+
+	pptr_namelen = xfs_parent_lookup(sc->tp, other_ip, &rp->pptr,
+			rp->namebuf, MAXNAMELEN, &rp->pptr_scratch);
+	if (pptr_namelen == -ENOATTR) {
+		trace_xrep_pptr_checkname(other_ip, &rp->pptr);
+		ASSERT(pptr_namelen != -ENOATTR);
+		return -EFSCORRUPTED;
+	}
+	if (pptr_namelen < 0)
+		return pptr_namelen;
+
+	if (pptr_namelen != rp->pptr.p_namelen) {
+		trace_xrep_pptr_checkname(other_ip, &rp->pptr);
+		ASSERT(pptr_namelen == rp->pptr.p_namelen);
+		return -EFSCORRUPTED;
+	}
+
+	if (memcmp(rp->namebuf, rp->pptr.p_name, rp->pptr.p_namelen)) {
+		trace_xrep_pptr_checkname(other_ip, &rp->pptr);
+		ASSERT(0);
+		return -EFSCORRUPTED;
+	}
+
 	return 0;
 }
 
@@ -566,8 +604,16 @@ xrep_pptr_rebuild_tree(
 
 	trace_xrep_pptr_rebuild_tree(sc->ip, 0);
 
-	xrep_tempfile_ilock(sc);
-	return xchk_xattr_walk(sc, sc->tempip, xrep_pptr_dump_tempptr, rp);
+	xchk_ilock(sc, XFS_ILOCK_EXCL);
+	error = xrep_tempfile_ilock_polled(sc);
+	if (error)
+		return error;
+
+	error = xchk_xattr_walk(sc, sc->tempip, xrep_pptr_dump_tempptr, rp);
+	if (error)
+		return error;
+
+	return xchk_xattr_walk(sc, sc->ip, xrep_pptr_dump_tempptr, rp);
 }
 
 /*
fs/xfs/scrub/trace.h +1 -0
index 283a1cedf368..e536d070f9c7 100644
@@ -1356,6 +1356,7 @@ DEFINE_EVENT(xrep_pptr_class, name, \
 DEFINE_XREP_PPTR_CLASS(xrep_pptr_createname);
 DEFINE_XREP_PPTR_CLASS(xrep_pptr_removename);
 DEFINE_XREP_PPTR_CLASS(xrep_pptr_dumpname);
+DEFINE_XREP_PPTR_CLASS(xrep_pptr_checkname);
 
 DECLARE_EVENT_CLASS(xrep_pptr_scan_class,
 	TP_PROTO(struct xfs_inode *ip, const struct xfs_inode *dp,
#78 lore raw

cover xfs: online checking of directories

In reply to Darrick J. Wong

Hi all,

Update the existing online directory checker to confirm the parent pointers that should also exist.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-check

---
 fs/xfs/scrub/dir.c   |  367 ++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/trace.h |    2 
 2 files changed, 368 insertions(+), 1 deletion(-)
#79 lore raw

PATCH 1/2 xfs: check dirents have parent pointers

From: Darrick J. Wong <[email protected]>

If the fs has parent pointers, we need to check that each child dirent points to a file that has a parent pointer pointing back at us.

---
 fs/xfs/scrub/dir.c |  134 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 133 insertions(+), 1 deletion(-)
fs/xfs/scrub/dir.c +133 -1
index d720f1e143dd..39ae59eb4f40 100644
@@ -15,6 +15,8 @@
 #include "xfs_icache.h"
 #include "xfs_dir2.h"
 #include "xfs_dir2_priv.h"
+#include "xfs_attr.h"
+#include "xfs_parent.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 #include "scrub/dabtree.h"
@@ -39,6 +41,20 @@ xchk_setup_directory(
 
 /* Directories */
 
+struct xchk_dir {
+	struct xfs_scrub	*sc;
+
+	/* Scratch buffer for scanning pptr xattrs */
+	struct xfs_parent_name_irec pptr;
+
+	/* xattr key and da args for parent pointer validation. */
+	struct xfs_parent_scratch pptr_scratch;
+
+	/* Name buffer for pptr validation and dirent revalidation. */
+	uint8_t			namebuf[MAXNAMELEN];
+
+};
+
 /* Scrub a directory entry. */
 
 /* Check that an inode's mode matches a given XFS_DIR3_FT_* type. */
@@ -61,6 +77,105 @@ xchk_dir_check_ftype(
 		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
 }
 
+/*
+ * Try to lock a child file for checking parent pointers.  Returns the inode
+ * flags for the locks we now hold, or zero if we failed.
+ */
+STATIC unsigned int
+xchk_dir_lock_child(
+	struct xfs_scrub	*sc,
+	struct xfs_inode	*ip)
+{
+	if (!xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED))
+		return 0;
+
+	if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) {
+		xfs_iunlock(ip, XFS_IOLOCK_SHARED);
+		return 0;
+	}
+
+	if (!xfs_inode_has_attr_fork(ip) || !xfs_need_iread_extents(&ip->i_af))
+		return XFS_IOLOCK_SHARED | XFS_ILOCK_SHARED;
+
+	xfs_iunlock(ip, XFS_ILOCK_SHARED);
+
+	if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL)) {
+		xfs_iunlock(ip, XFS_IOLOCK_SHARED);
+		return 0;
+	}
+
+	return XFS_IOLOCK_SHARED | XFS_ILOCK_EXCL;
+}
+
+/* Check the backwards link (parent pointer) associated with this dirent. */
+STATIC int
+xchk_dir_parent_pointer(
+	struct xchk_dir		*sd,
+	xfs_dir2_dataptr_t	dapos,
+	const struct xfs_name	*name,
+	struct xfs_inode	*ip)
+{
+	struct xfs_scrub	*sc = sd->sc;
+	int			pptr_namelen;
+
+	sd->pptr.p_ino = sc->ip->i_ino;
+	sd->pptr.p_gen = VFS_I(sc->ip)->i_generation;
+	sd->pptr.p_diroffset = dapos;
+
+	pptr_namelen = xfs_parent_lookup(sc->tp, ip, &sd->pptr, sd->namebuf,
+			MAXNAMELEN, &sd->pptr_scratch);
+	if (pptr_namelen == -ENOATTR) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
+		return 0;
+	}
+	if (pptr_namelen < 0) {
+		xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0,
+				&pptr_namelen);
+		return pptr_namelen;
+	}
+
+	if (pptr_namelen != name->len) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
+		return 0;
+	}
+
+	if (memcmp(sd->namebuf, name->name, name->len)) {
+		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
+		return 0;
+	}
+
+	return 0;
+}
+
+/* Look for a parent pointer matching this dirent, if the child isn't busy. */
+STATIC int
+xchk_dir_check_pptr_fast(
+	struct xchk_dir		*sd,
+	xfs_dir2_dataptr_t	dapos,
+	const struct xfs_name	*name,
+	struct xfs_inode	*ip)
+{
+	struct xfs_scrub	*sc = sd->sc;
+	unsigned int		lockmode;
+	int			error;
+
+	/* dot and dotdot entries do not have parent pointers */
+	if (!strncmp(".", name->name, name->len) ||
+	    !strncmp("..", name->name, name->len))
+		return 0;
+
+	/* Try to lock the inode. */
+	lockmode = xchk_dir_lock_child(sc, ip);
+	if (!lockmode) {
+		xchk_set_incomplete(sc);
+		return -ECANCELED;
+	}
+
+	error = xchk_dir_parent_pointer(sd, dapos, name, ip);
+	xfs_iunlock(ip, lockmode);
+	return error;
+}
+
 /*
  * Scrub a single directory entry.
  *
@@ -78,6 +193,7 @@ xchk_dir_actor(
 {
 	struct xfs_mount	*mp = dp->i_mount;
 	struct xfs_inode	*ip;
+	struct xchk_dir		*sd = priv;
 	xfs_ino_t		lookup_ino;
 	xfs_dablk_t		offset;
 	int			error = 0;
@@ -144,6 +260,14 @@ xchk_dir_actor(
 		goto out;
 
 	xchk_dir_check_ftype(sc, offset, ip, name->type);
+
+	if (xfs_has_parent(mp)) {
+		error = xchk_dir_check_pptr_fast(sd, dapos, name, ip);
+		if (error)
+			goto out_rele;
+	}
+
+out_rele:
 	xchk_irele(sc, ip);
 out:
 	if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
@@ -759,6 +883,7 @@ int
 xchk_directory(
 	struct xfs_scrub	*sc)
 {
+	struct xchk_dir		*sd;
 	int			error;
 
 	if (!S_ISDIR(VFS_I(sc->ip)->i_mode))
@@ -786,9 +911,16 @@ xchk_directory(
 	if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
 		return 0;
 
+	sd = kvzalloc(sizeof(struct xchk_dir), XCHK_GFP_FLAGS);
+	if (!sd)
+		return -ENOMEM;
+	sd->sc = sc;
+
 	/* Look up every name in this directory by hash. */
-	error = xchk_dir_walk(sc, sc->ip, xchk_dir_actor, NULL);
+	error = xchk_dir_walk(sc, sc->ip, xchk_dir_actor, sd);
 	if (error == -ECANCELED)
 		error = 0;
+
+	kvfree(sd);
 	return error;
 }
#80 lore raw

PATCH 2/2 xfs: deferred scrub of dirents

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If the trylock-based parent pointer check fails, retain those dirents and check them at the end. This may involve dropping the locks on the file being scanned, so yay.

---
 fs/xfs/scrub/dir.c   |  237 ++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/trace.h |    2 
 2 files changed, 237 insertions(+), 2 deletions(-)
fs/xfs/scrub/dir.c +235 -2
index 39ae59eb4f40..3f3223e563ae 100644
@@ -22,6 +22,10 @@
 #include "scrub/dabtree.h"
 #include "scrub/readdir.h"
 #include "scrub/repair.h"
+#include "scrub/trace.h"
+#include "scrub/xfile.h"
+#include "scrub/xfarray.h"
+#include "scrub/xfblob.h"
 
 /* Set us up to scrub directories. */
 int
@@ -41,6 +45,21 @@ xchk_setup_directory(
 
 /* Directories */
 
+/* Deferred directory entry that we saved for later. */
+struct xchk_dirent {
+	/* Cookie for retrieval of the dirent name. */
+	xfblob_cookie			name_cookie;
+
+	/* Child inode number. */
+	xfs_ino_t			ino;
+
+	/* Directory offset. */
+	xfs_dir2_dataptr_t		diroffset;
+
+	/* Length of the pptr name. */
+	uint8_t				namelen;
+};
+
 struct xchk_dir {
 	struct xfs_scrub	*sc;
 
@@ -50,6 +69,15 @@ struct xchk_dir {
 	/* xattr key and da args for parent pointer validation. */
 	struct xfs_parent_scratch pptr_scratch;
 
+	/* Fixed-size array of xchk_dirent structures. */
+	struct xfarray		*dir_entries;
+
+	/* Blobs containing dirent names. */
+	struct xfblob		*dir_names;
+
+	/* If we've cycled the ILOCK, we must revalidate deferred dirents. */
+	bool			need_revalidate;
+
 	/* Name buffer for pptr validation and dirent revalidation. */
 	uint8_t			namebuf[MAXNAMELEN];
 
@@ -167,8 +195,25 @@ xchk_dir_check_pptr_fast(
 	/* Try to lock the inode. */
 	lockmode = xchk_dir_lock_child(sc, ip);
 	if (!lockmode) {
-		xchk_set_incomplete(sc);
-		return -ECANCELED;
+		struct xchk_dirent	save_de = {
+			.namelen	= name->len,
+			.ino		= ip->i_ino,
+			.diroffset	= dapos,
+		};
+
+		/* Couldn't lock the inode, so save the dirent for later. */
+		trace_xchk_dir_defer(sc->ip, name->name, name->len, ip->i_ino);
+
+		error = xfblob_store(sd->dir_names, &save_de.name_cookie,
+				name->name, name->len);
+		if (xchk_fblock_process_error(sc, XFS_DATA_FORK, 0, &error))
+			return error;
+
+		error = xfarray_append(sd->dir_entries, &save_de);
+		if (xchk_fblock_process_error(sc, XFS_DATA_FORK, 0, &error))
+			return error;
+
+		return 0;
 	}
 
 	error = xchk_dir_parent_pointer(sd, dapos, name, ip);
@@ -878,6 +923,164 @@ xchk_directory_blocks(
 	return error;
 }
 
+/*
+ * Revalidate a dirent that we collected in the past but couldn't check because
+ * of lock contention.  Returns 0 if the dirent is still valid, -ENOENT if it
+ * has gone away on us, or a negative errno.
+ */
+STATIC int
+xchk_dir_revalidate_dirent(
+	struct xchk_dir		*sd,
+	const struct xfs_name	*xname,
+	xfs_ino_t		ino,
+	xfs_dir2_dataptr_t	diroffset)
+{
+	struct xfs_scrub	*sc = sd->sc;
+	xfs_ino_t		child_ino;
+	xfs_dir2_dataptr_t	child_diroffset = XFS_DIR2_NULL_DATAPTR;
+	int			error;
+
+	error = xchk_dir_lookup(sc, sc->ip, xname, &child_ino,
+			&child_diroffset);
+	if (error == -ENOENT) {
+		/* Directory entry went away, nothing to revalidate. */
+		return -ENOENT;
+	}
+	if (error)
+		return error;
+
+	/* The inode number changed, nothing to revalidate. */
+	if (ino != child_ino)
+		return -ENOENT;
+
+	/* The directory offset changed, nothing to revalidate. */
+	if (diroffset != child_diroffset)
+		return -ENOENT;
+
+	return 0;
+}
+
+/*
+ * Check a directory entry's parent pointers the slow way, which means we cycle
+ * locks a bunch and put up with revalidation until we get it done.
+ */
+STATIC int
+xchk_dir_slow_dirent(
+	struct xchk_dir		*sd,
+	struct xchk_dirent	*dirent)
+{
+	struct xfs_name		xname = {
+		.name		= sd->namebuf,
+		.len		= dirent->namelen,
+	};
+	struct xfs_scrub	*sc = sd->sc;
+	struct xfs_inode	*ip;
+	unsigned int		lockmode;
+	int			error;
+
+	/* Check that the deferred dirent still exists. */
+	if (sd->need_revalidate) {
+		error = xchk_dir_revalidate_dirent(sd, &xname, dirent->ino,
+				dirent->diroffset);
+		if (error == -ENOENT)
+			return 0;
+		if (!xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0,
+					&error))
+			return error;
+	}
+
+	error = xchk_iget(sc, dirent->ino, &ip);
+	if (error == -EINVAL || error == -ENOENT) {
+		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
+		return 0;
+	}
+	if (!xchk_fblock_xref_process_error(sc, XFS_ATTR_FORK, 0, &error))
+		return error;
+
+	/*
+	 * If we can grab both IOLOCK and ILOCK of the alleged child, we can
+	 * proceed with the validation.
+	 */
+	lockmode = xchk_dir_lock_child(sc, ip);
+	if (lockmode)
+		goto check_pptr;
+
+	/*
+	 * We couldn't lock the child file.  Drop all the locks and try to
+	 * get them again, one at a time.
+	 */
+	xchk_iunlock(sc, sc->ilock_flags);
+	sd->need_revalidate = true;
+
+	trace_xchk_dir_slowpath(sc->ip, xname.name, xname.len, ip->i_ino);
+
+	while (true) {
+		xchk_ilock(sc, XFS_IOLOCK_EXCL);
+		if (xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED)) {
+			xchk_ilock(sc, XFS_ILOCK_EXCL);
+			if (xfs_ilock_nowait(ip, XFS_ILOCK_EXCL)) {
+				break;
+			}
+			xchk_iunlock(sc, XFS_ILOCK_EXCL);
+		}
+		xchk_iunlock(sc, XFS_IOLOCK_EXCL);
+
+		if (xchk_should_terminate(sc, &error))
+			goto out_rele;
+
+		delay(1);
+	}
+	lockmode = XFS_IOLOCK_SHARED | XFS_ILOCK_EXCL;
+
+	/* Revalidate, since we just cycled the locks. */
+	error = xchk_dir_revalidate_dirent(sd, &xname, dirent->ino,
+			dirent->diroffset);
+	if (error == -ENOENT)
+		goto out_unlock;
+	if (!xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0, &error))
+		goto out_unlock;
+
+check_pptr:
+	error = xchk_dir_parent_pointer(sd, dirent->diroffset, &xname, ip);
+out_unlock:
+	xfs_iunlock(ip, lockmode);
+out_rele:
+	xchk_irele(sc, ip);
+	return error;
+}
+
+/* Check all the dirents that we deferred the first time around. */
+STATIC int
+xchk_dir_finish_slow_dirents(
+	struct xchk_dir		*sd)
+{
+	xfarray_idx_t		array_cur;
+	int			error;
+
+	foreach_xfarray_idx(sd->dir_entries, array_cur) {
+		struct xchk_dirent	dirent;
+
+		if (sd->sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
+			return 0;
+
+		error = xfarray_load(sd->dir_entries, array_cur, &dirent);
+		if (error)
+			return error;
+
+		error = xfblob_load(sd->dir_names, dirent.name_cookie,
+				sd->namebuf, dirent.namelen);
+		if (error)
+			return error;
+		sd->namebuf[MAXNAMELEN - 1] = 0;
+
+		error = xchk_dir_slow_dirent(sd, &dirent);
+		if (error)
+			return error;
+	}
+
+	return 0;
+}
+
 /* Scrub a whole directory. */
 int
 xchk_directory(
@@ -916,11 +1119,41 @@ xchk_directory(
 		return -ENOMEM;
 	sd->sc = sc;
 
+	if (xfs_has_parent(sc->mp)) {
+		/*
+		 * Set up some staging memory for dirents that we can't check
+		 * due to locking contention.
+		 */
+		error = xfarray_create(sc->mp, "directory entries", 0,
+				sizeof(struct xchk_dirent), &sd->dir_entries);
+		if (error)
+			goto out_sd;
+
+		error = xfblob_create(sc->mp, "dirent names", &sd->dir_names);
+		if (error)
+			goto out_entries;
+	}
+
 	/* Look up every name in this directory by hash. */
 	error = xchk_dir_walk(sc, sc->ip, xchk_dir_actor, sd);
 	if (error == -ECANCELED)
 		error = 0;
+	if (error)
+		goto out_names;
 
+	if (xfs_has_parent(sc->mp)) {
+		error = xchk_dir_finish_slow_dirents(sd);
+		if (error)
+			goto out_names;
+	}
+
+out_names:
+	if (sd->dir_names)
+		xfblob_destroy(sd->dir_names);
+out_entries:
+	if (sd->dir_entries)
+		xfarray_destroy(sd->dir_entries);
+out_sd:
 	kvfree(sd);
 	return error;
 }
fs/xfs/scrub/trace.h +2 -0
index e536d070f9c7..911d947db787 100644
@@ -962,6 +962,8 @@ DEFINE_EVENT(xchk_pptr_class, name, \
 	TP_ARGS(ip, name, namelen, parent_ino))
 DEFINE_XCHK_PPTR_CLASS(xchk_parent_defer);
 DEFINE_XCHK_PPTR_CLASS(xchk_parent_slowpath);
+DEFINE_XCHK_PPTR_CLASS(xchk_dir_defer);
+DEFINE_XCHK_PPTR_CLASS(xchk_dir_slowpath);
 
 /* repair tracepoints */
 #if IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR)
#81 lore raw

cover xfs: encode parent pointer name in xattr key

In reply to Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:

    (parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)

The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:

For dirent names shorter than 243 bytes:

    (parent_ino, parent_gen, dirent_name) -> NULL

For dirent names longer than 243 bytes:

    (parent_ino, parent_gen, dirent_name[0:178],
     sha512(child_gen, dirent_name)) -> (dirent_name[179:255])

The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.

A messier solution to this problem would be to extend the xattr ondisk format to allow parent pointers to have xattr names up to 267 bytes. This would likely involve redefining the ondisk namelen field to omit the size of the parent ino/gen information and might be madness.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key

---
 fs/xfs/Kconfig                 |    1 
 fs/xfs/libxfs/xfs_da_format.h  |   49 +++++++
 fs/xfs/libxfs/xfs_fs.h         |    4 -
 fs/xfs/libxfs/xfs_parent.c     |  265 ++++++++++++++++++++++++++++++++--------
 fs/xfs/libxfs/xfs_parent.h     |   48 +++++--
 fs/xfs/libxfs/xfs_trans_resv.c |    6 -
 fs/xfs/scrub/dir.c             |   16 ++
 fs/xfs/scrub/dir_repair.c      |   87 ++++---------
 fs/xfs/scrub/parent.c          |   51 +++++---
 fs/xfs/scrub/parent_repair.c   |   29 ++--
 fs/xfs/scrub/trace.h           |   48 ++-----
 fs/xfs/xfs_attr_item.c         |    4 -
 fs/xfs/xfs_inode.c             |   30 ++---
 fs/xfs/xfs_linux.h             |    1 
 fs/xfs/xfs_mount.c             |   13 ++
 fs/xfs/xfs_mount.h             |    3 
 fs/xfs/xfs_ondisk.h            |    6 +
 fs/xfs/xfs_parent_utils.c      |    4 -
 fs/xfs/xfs_sha512.h            |   42 ++++++
 fs/xfs/xfs_super.c             |    3 
 fs/xfs/xfs_symlink.c           |    3 
 21 files changed, 481 insertions(+), 232 deletions(-)
 create mode 100644 fs/xfs/xfs_sha512.h
#82 lore raw

PATCH 1/5 xfs: load secure hash algorithm for parent pointers

From: Darrick J. Wong <[email protected]>

We're about to start replacing the diroffset field of parent pointers with a collision-resistant hash of the directory entry name. Start by attaching the sha512 crypto implementation if parent pointers are attached.

---
 fs/xfs/Kconfig     |    1 +
 fs/xfs/xfs_linux.h |    1 +
 fs/xfs/xfs_mount.c |   13 +++++++++++++
 fs/xfs/xfs_mount.h |    3 +++
 fs/xfs/xfs_super.c |    3 +++
 5 files changed, 21 insertions(+)
fs/xfs/Kconfig +1 -0
index 4798a147fd9e..6422daaf8914 100644
@@ -5,6 +5,7 @@ config XFS_FS
 	select EXPORTFS
 	select LIBCRC32C
 	select FS_IOMAP
+	select CRYPTO_SHA512
 	help
 	  XFS is a high performance journaling filesystem which originated
 	  on the SGI IRIX platform.  It is completely multi-threaded, can
fs/xfs/xfs_linux.h +1 -0
index c05f7e309c3e..3f93a742b896 100644
@@ -62,6 +62,7 @@ typedef __u32			xfs_nlink_t;
 #include <linux/rhashtable.h>
 #include <linux/xattr.h>
 #include <linux/mnt_idmapping.h>
+#include <crypto/hash.h>
 
 #include <asm/page.h>
 #include <asm/div64.h>
fs/xfs/xfs_mount.c +13 -0
index fb87ffb48f7f..a5f3dce658e9 100644
@@ -983,6 +983,19 @@ xfs_mountfs(
 			goto out_agresv;
 	}
 
+	if (xfs_has_parent(mp)) {
+		struct crypto_shash	*tfm;
+
+		tfm = crypto_alloc_shash("sha512", 0, 0);
+		if (IS_ERR(tfm)) {
+			error = PTR_ERR(tfm);
+			goto out_agresv;
+		}
+		xfs_info(mp, "parent pointer hash %s",
+				crypto_shash_driver_name(tfm));
+		mp->m_sha512 = tfm;
+	}
+
 	return 0;
 
  out_agresv:
fs/xfs/xfs_mount.h +3 -0
index c08f55cc4f36..7c8e15e84cd6 100644
@@ -244,6 +244,9 @@ typedef struct xfs_mount {
 #endif
 	/* Hook to feed file directory updates to an active online repair. */
 	struct xfs_hooks	m_dirent_update_hooks;
+
+	/* sha512 engine, if needed */
+	struct crypto_shash	*m_sha512;
 } xfs_mount_t;
 
 #define M_IGEO(mp)		(&(mp)->m_ino_geo)
fs/xfs/xfs_super.c +3 -0
index 0432a4a096e8..610d72353f39 100644
@@ -738,6 +738,8 @@ xfs_mount_free(
 {
 	kfree(mp->m_rtname);
 	kfree(mp->m_logname);
+	if (mp->m_sha512)
+		crypto_free_shash(mp->m_sha512);
 	kmem_free(mp);
 }
 
@@ -1961,6 +1963,7 @@ static int xfs_init_fs_context(
 	if (fc->sb_flags & SB_SYNCHRONOUS)
 		mp->m_features |= XFS_FEAT_WSYNC;
 
+	mp->m_sha512 = NULL;
 	fc->s_fs_info = mp;
 	fc->ops = &xfs_context_ops;
 
#83 lore raw

PATCH 2/5 xfs: replace parent pointer diroffset with sha512 hash of name

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Replace the diroffset with the sha512 hash of the dirent name, thereby eliminating the need for directory repair to update all the parent pointers after rebuilding the directory.

---
 fs/xfs/libxfs/xfs_da_format.h |   15 +++--
 fs/xfs/libxfs/xfs_fs.h        |    4 +
 fs/xfs/libxfs/xfs_parent.c    |  125 +++++++++++++++++++++++++++++++----------
 fs/xfs/libxfs/xfs_parent.h    |   21 ++++---
 fs/xfs/scrub/dir.c            |   12 +++-
 fs/xfs/scrub/dir_repair.c     |   83 ++++++++-------------------
 fs/xfs/scrub/parent.c         |   43 ++++++++++----
 fs/xfs/scrub/parent_repair.c  |   27 ++++-----
 fs/xfs/scrub/trace.h          |   48 +++++-----------
 fs/xfs/xfs_inode.c            |   30 ++++------
 fs/xfs/xfs_ondisk.h           |    4 +
 fs/xfs/xfs_parent_utils.c     |    2 -
 fs/xfs/xfs_sha512.h           |   42 ++++++++++++++
 fs/xfs/xfs_symlink.c          |    3 -
 14 files changed, 271 insertions(+), 188 deletions(-)
 create mode 100644 fs/xfs/xfs_sha512.h
fs/xfs/libxfs/xfs_da_format.h +10 -5
index c07b8166e8ff..386f63b262d5 100644
@@ -824,17 +824,22 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)
 xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 				      struct xfs_da3_blkinfo *hdr3);
 
+/* We use sha512 for the parent pointer name hash. */
+#define XFS_PARENT_NAME_HASH_SIZE	(64)
+
 /*
  * Parent pointer attribute format definition
  *
- * EA name encodes the parent inode number, generation and the offset of
- * the dirent that points to the child inode. The EA value contains the
- * same name as the dirent in the parent directory.
+ * The EA name encodes the parent inode number, generation and a collision
+ * resistant hash computed from the dirent name.  The hash is defined to be the
+ * sha512 of the child inode generation and the dirent name.
+ *
+ * The EA value contains the same name as the dirent in the parent directory.
  */
 struct xfs_parent_name_rec {
 	__be64  p_ino;
 	__be32  p_gen;
-	__be32  p_diroffset;
-};
+	__u8	p_namehash[XFS_PARENT_NAME_HASH_SIZE];
+} __attribute__((packed));
 
 #endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_fs.h +2 -2
index 9e59a1fdfb0c..c65345d2ba7a 100644
@@ -770,8 +770,8 @@ struct xfs_scrub_metadata {
 struct xfs_parent_ptr {
 	__u64		xpp_ino;			/* Inode */
 	__u32		xpp_gen;			/* Inode generation */
-	__u32		xpp_diroffset;			/* Directory offset */
-	__u64		xpp_rsvd;			/* Reserved */
+	__u32		xpp_rsvd;			/* Reserved */
+	__u64		xpp_rsvd2;			/* Reserved */
 	__u8		xpp_name[XFS_PPTR_MAXNAMELEN];	/* File name */
 };
 
fs/xfs/libxfs/xfs_parent.c +96 -29

205 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +13 -8
index a7fc621b82c4..d3f2841e0f6e 100644
@@ -23,7 +23,7 @@ struct xfs_parent_name_irec {
 	/* Key fields for looking up a particular parent pointer. */
 	xfs_ino_t		p_ino;
 	uint32_t		p_gen;
-	xfs_dir2_dataptr_t	p_diroffset;
+	uint8_t			p_namehash[XFS_PARENT_NAME_HASH_SIZE];
 
 	/* Attributes of a parent pointer. */
 	uint8_t			p_namelen;
@@ -79,15 +79,14 @@ xfs_parent_start_locked(
 
 int xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
 		struct xfs_inode *dp, const struct xfs_name *parent_name,
-		xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
+		struct xfs_inode *child);
 int xfs_parent_replace(struct xfs_trans *tp,
 		struct xfs_parent_defer *new_parent, struct xfs_inode *old_dp,
-		xfs_dir2_dataptr_t old_diroffset,
-		const struct xfs_name *parent_name, struct xfs_inode *new_ip,
-		xfs_dir2_dataptr_t new_diroffset, struct xfs_inode *child);
-int xfs_parent_remove(struct xfs_trans *tp, struct xfs_inode *dp,
-		struct xfs_parent_defer *parent, xfs_dir2_dataptr_t diroffset,
-		struct xfs_inode *child);
+		const struct xfs_name *old_name, struct xfs_inode *new_ip,
+		const struct xfs_name *new_name, struct xfs_inode *child);
+int xfs_parent_remove(struct xfs_trans *tp,
+		struct xfs_parent_defer *parent, struct xfs_inode *dp,
+		const struct xfs_name *name, struct xfs_inode *child);
 
 void __xfs_parent_cancel(struct xfs_mount *mp, struct xfs_parent_defer *parent);
 
@@ -100,6 +99,12 @@ xfs_parent_finish(
 		__xfs_parent_cancel(mp, p);
 }
 
+int xfs_parent_namehash(struct xfs_inode *ip, const struct xfs_name *name,
+		void *namehash, unsigned int namehash_len);
+
+int xfs_parent_irec_hash(struct xfs_inode *ip,
+		struct xfs_parent_name_irec *pptr);
+
 unsigned int xfs_pptr_calc_space_res(struct xfs_mount *mp,
 				     unsigned int namelen);
 
fs/xfs/scrub/dir.c +8 -4
index 3f3223e563ae..2494947a0c93 100644
@@ -139,16 +139,20 @@ xchk_dir_lock_child(
 STATIC int
 xchk_dir_parent_pointer(
 	struct xchk_dir		*sd,
-	xfs_dir2_dataptr_t	dapos,
 	const struct xfs_name	*name,
 	struct xfs_inode	*ip)
 {
 	struct xfs_scrub	*sc = sd->sc;
 	int			pptr_namelen;
+	int			error;
 
 	sd->pptr.p_ino = sc->ip->i_ino;
 	sd->pptr.p_gen = VFS_I(sc->ip)->i_generation;
-	sd->pptr.p_diroffset = dapos;
+
+	error = xfs_parent_namehash(ip, name, &sd->pptr.p_namehash,
+			sizeof(sd->pptr.p_namehash));
+	if (error)
+		return error;
 
 	pptr_namelen = xfs_parent_lookup(sc->tp, ip, &sd->pptr, sd->namebuf,
 			MAXNAMELEN, &sd->pptr_scratch);
@@ -216,7 +220,7 @@ xchk_dir_check_pptr_fast(
 		return 0;
 	}
 
-	error = xchk_dir_parent_pointer(sd, dapos, name, ip);
+	error = xchk_dir_parent_pointer(sd, name, ip);
 	xfs_iunlock(ip, lockmode);
 	return error;
 }
@@ -1041,7 +1045,7 @@ xchk_dir_slow_dirent(
 		goto out_unlock;
 
 check_pptr:
-	error = xchk_dir_parent_pointer(sd, dirent->diroffset, &xname, ip);
+	error = xchk_dir_parent_pointer(sd, &xname, ip);
 out_unlock:
 	xfs_iunlock(ip, lockmode);
 out_rele:
fs/xfs/scrub/dir_repair.c +25 -58

244 lines not on this page · raw message

fs/xfs/scrub/parent.c +29 -14

103 lines not on this page · raw message

fs/xfs/scrub/parent_repair.c +12 -15
index 56b47bf2807b..51432ab61c94 100644
@@ -95,7 +95,6 @@ struct xrep_pptr {
 	/* Parent pointer attr key. */
 	xfs_ino_t			p_ino;
 	uint32_t			p_gen;
-	xfs_dir2_dataptr_t		p_diroffset;
 
 	/* Length of the pptr name. */
 	uint8_t				namelen;
@@ -183,12 +182,16 @@ xrep_pptr_replay_update(
 	const struct xrep_pptr	*pptr)
 {
 	struct xfs_scrub	*sc = rp->sc;
+	int			error;
 
 	rp->pptr.p_ino = pptr->p_ino;
 	rp->pptr.p_gen = pptr->p_gen;
-	rp->pptr.p_diroffset = pptr->p_diroffset;
 	rp->pptr.p_namelen = pptr->namelen;
 
+	error = xfs_parent_irec_hash(sc->ip, &rp->pptr);
+	if (error)
+		return error;
+
 	if (pptr->action == XREP_PPTR_ADD) {
 		/* Create parent pointer. */
 		trace_xrep_pptr_createname(sc->tempip, &rp->pptr);
@@ -261,19 +264,17 @@ STATIC int
 xrep_pptr_add_pointer(
 	struct xrep_pptrs	*rp,
 	const struct xfs_name	*name,
-	const struct xfs_inode	*dp,
-	xfs_dir2_dataptr_t	diroffset)
+	const struct xfs_inode	*dp)
 {
 	struct xrep_pptr	pptr = {
 		.action		= XREP_PPTR_ADD,
 		.namelen	= name->len,
 		.p_ino		= dp->i_ino,
 		.p_gen		= VFS_IC(dp)->i_generation,
-		.p_diroffset	= diroffset,
 	};
 	int			error;
 
-	trace_xrep_pptr_add_pointer(rp->sc->tempip, dp, diroffset, name);
+	trace_xrep_pptr_add_pointer(rp->sc->tempip, dp, name);
 
 	error = xfblob_store(rp->pptr_names, &pptr.name_cookie, name->name,
 			name->len);
@@ -291,19 +292,17 @@ STATIC int
 xrep_pptr_remove_pointer(
 	struct xrep_pptrs	*rp,
 	const struct xfs_name	*name,
-	const struct xfs_inode	*dp,
-	xfs_dir2_dataptr_t	diroffset)
+	const struct xfs_inode	*dp)
 {
 	struct xrep_pptr	pptr = {
 		.action		= XREP_PPTR_REMOVE,
 		.namelen	= name->len,
 		.p_ino		= dp->i_ino,
 		.p_gen		= VFS_IC(dp)->i_generation,
-		.p_diroffset	= diroffset,
 	};
 	int			error;
 
-	trace_xrep_pptr_remove_pointer(rp->sc->tempip, dp, diroffset, name);
+	trace_xrep_pptr_remove_pointer(rp->sc->tempip, dp, name);
 
 	error = xfblob_store(rp->pptr_names, &pptr.name_cookie, name->name,
 			name->len);
@@ -352,7 +351,7 @@ xrep_pptr_scan_dirent(
 	 * addition to the temporary file.
 	 */
 	mutex_lock(&rp->lock);
-	error = xrep_pptr_add_pointer(rp, name, dp, dapos);
+	error = xrep_pptr_add_pointer(rp, name, dp);
 	mutex_unlock(&rp->lock);
 	return error;
 }
@@ -646,11 +645,9 @@ xrep_pptr_live_update(
 	    xchk_iscan_want_live_update(&rp->iscan, p->dp->i_ino)) {
 		mutex_lock(&rp->lock);
 		if (p->delta > 0)
-			error = xrep_pptr_add_pointer(rp, p->name, p->dp,
-					p->diroffset);
+			error = xrep_pptr_add_pointer(rp, p->name, p->dp);
 		else
-			error = xrep_pptr_remove_pointer(rp, p->name, p->dp,
-					p->diroffset);
+			error = xrep_pptr_remove_pointer(rp, p->name, p->dp);
 		mutex_unlock(&rp->lock);
 		if (error)
 			goto out_abort;
fs/xfs/scrub/trace.h +16 -32

147 lines not on this page · raw message

fs/xfs/xfs_inode.c +12 -18
index 09b0ac6b99cb..4cd9a4fea5e0 100644
@@ -1202,8 +1202,7 @@ xfs_create(
 	 * the parent information now.
 	 */
 	if (parent) {
-		error = xfs_parent_add(tp, parent, dp, name, diroffset,
-					     ip);
+		error = xfs_parent_add(tp, parent, dp, name, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
@@ -1477,8 +1476,7 @@ xfs_link(
 	 * the parent to the inode.
 	 */
 	if (parent) {
-		error = xfs_parent_add(tp, parent, tdp, target_name,
-					     diroffset, sip);
+		error = xfs_parent_add(tp, parent, tdp, target_name, sip);
 		if (error)
 			goto error_return;
 	}
@@ -2750,7 +2748,7 @@ xfs_remove(
 	}
 
 	if (parent) {
-		error = xfs_parent_remove(tp, dp, parent, dir_offset, ip);
+		error = xfs_parent_remove(tp, parent, dp, name, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
@@ -3061,13 +3059,13 @@ xfs_cross_rename(
 	}
 
 	if (xfs_has_parent(mp)) {
-		error = xfs_parent_replace(tp, ip1_pptr, dp1,
-				old_diroffset, name2, dp2, new_diroffset, ip1);
+		error = xfs_parent_replace(tp, ip1_pptr, dp1, name1, dp2,
+				name2, ip1);
 		if (error)
 			goto out_trans_abort;
 
-		error = xfs_parent_replace(tp, ip2_pptr, dp2,
-				new_diroffset, name1, dp1, old_diroffset, ip2);
+		error = xfs_parent_replace(tp, ip2_pptr, dp2, name2, dp1,
+				name1, ip2);
 		if (error)
 			goto out_trans_abort;
 	}
@@ -3540,25 +3538,21 @@ xfs_rename(
 		goto out_trans_cancel;
 
 	if (wip_pptr) {
-		error = xfs_parent_add(tp, wip_pptr,
-					     src_dp, src_name,
-					     old_diroffset, wip);
+		error = xfs_parent_add(tp, wip_pptr, src_dp, src_name, wip);
 		if (error)
 			goto out_trans_cancel;
 	}
 
 	if (src_ip_pptr) {
-		error = xfs_parent_replace(tp, src_ip_pptr, src_dp,
-				old_diroffset, target_name, target_dp,
-				new_diroffset, src_ip);
+		error = xfs_parent_replace(tp, src_ip_pptr, src_dp, src_name,
+				target_dp, target_name, src_ip);
 		if (error)
 			goto out_trans_cancel;
 	}
 
 	if (tgt_ip_pptr) {
-		error = xfs_parent_remove(tp, target_dp,
-						tgt_ip_pptr,
-						new_diroffset, target_ip);
+		error = xfs_parent_remove(tp, tgt_ip_pptr, target_dp,
+				target_name, target_ip);
 		if (error)
 			goto out_trans_cancel;
 	}
fs/xfs/xfs_ondisk.h +4 -0
index 6a6bd05c2a68..2dc1eef63d96 100644
@@ -6,6 +6,8 @@
 #ifndef __XFS_ONDISK_H
 #define __XFS_ONDISK_H
 
+#include <crypto/sha2.h>
+
 #define XFS_CHECK_STRUCT_SIZE(structname, size) \
 	BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \
 		#structname ") is wrong, expected " #size)
@@ -114,6 +116,8 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, offset,		1);
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, name,		3);
 	XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_hdr_t,		10);
+	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec,	76);
+	BUILD_BUG_ON(XFS_PARENT_NAME_HASH_SIZE != SHA512_DIGEST_SIZE);
 
 	/* log structures */
 	XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format,	88);
fs/xfs/xfs_parent_utils.c +1 -1
index 5ff7d38bc375..65bec3875308 100644
@@ -83,7 +83,7 @@ xfs_getparent_listent(
 	pptr = &ppi->pi_parents[ppi->pi_ptrs_used++];
 	pptr->xpp_ino = irec->p_ino;
 	pptr->xpp_gen = irec->p_gen;
-	pptr->xpp_diroffset = irec->p_diroffset;
+	pptr->xpp_rsvd2 = 0;
 	pptr->xpp_rsvd = 0;
 
 	memcpy(pptr->xpp_name, irec->p_name, irec->p_namelen);
fs/xfs/xfs_sha512.h new file +42 -0

42 lines not on this page · raw message

fs/xfs/xfs_symlink.c +1 -2
index 63e68e832551..327c805815dc 100644
@@ -348,8 +348,7 @@ xfs_symlink(
 	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	if (parent) {
-		error = xfs_parent_add(tp, parent, dp, link_name,
-					     diroffset, ip);
+		error = xfs_parent_add(tp, parent, dp, link_name, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
#84 lore raw

PATCH 3/5 xfs: skip the sha512 namehash when possible

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Reduce the size and performance impacts of parent pointer name hashes by using the dirent name as the hash if the dirent name is shorter than a sha512 hash would be. IOWs, we only use sha512 for names longer than 63 bytes.

---
 fs/xfs/libxfs/xfs_da_format.h |   21 +++++++++-
 fs/xfs/libxfs/xfs_parent.c    |   85 +++++++++++++++++++++++++++--------------
 fs/xfs/libxfs/xfs_parent.h    |    8 ++--
 fs/xfs/scrub/dir.c            |   12 ++++--
 fs/xfs/scrub/dir_repair.c     |    2 -
 fs/xfs/scrub/parent.c         |   16 +++++---
 fs/xfs/scrub/parent_repair.c  |    2 -
 fs/xfs/xfs_parent_utils.c     |    2 -
 8 files changed, 101 insertions(+), 47 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +19 -2
index 386f63b262d5..275357506394 100644
@@ -831,8 +831,11 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
  * Parent pointer attribute format definition
  *
  * The EA name encodes the parent inode number, generation and a collision
- * resistant hash computed from the dirent name.  The hash is defined to be the
- * sha512 of the child inode generation and the dirent name.
+ * resistant hash computed from the dirent name.  The hash is defined to be:
+ *
+ * - The dirent name if it fits within the EA name.
+ *
+ * - The sha512 of the child inode generation and the dirent name.
  *
  * The EA value contains the same name as the dirent in the parent directory.
  */
@@ -842,4 +845,18 @@ struct xfs_parent_name_rec {
 	__u8	p_namehash[XFS_PARENT_NAME_HASH_SIZE];
 } __attribute__((packed));
 
+static inline unsigned int
+xfs_parent_name_rec_sizeof(
+	unsigned int		hashlen)
+{
+	return offsetof(struct xfs_parent_name_rec, p_namehash) + hashlen;
+}
+
+static inline unsigned int
+xfs_parent_name_hashlen(
+	unsigned int		rec_sizeof)
+{
+	return rec_sizeof - offsetof(struct xfs_parent_name_rec, p_namehash);
+}
+
 #endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_parent.c +56 -29

203 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +5 -3
index d3f2841e0f6e..4c3100760bba 100644
@@ -23,6 +23,7 @@ struct xfs_parent_name_irec {
 	/* Key fields for looking up a particular parent pointer. */
 	xfs_ino_t		p_ino;
 	uint32_t		p_gen;
+	uint8_t			hashlen;
 	uint8_t			p_namehash[XFS_PARENT_NAME_HASH_SIZE];
 
 	/* Attributes of a parent pointer. */
@@ -31,10 +32,11 @@ struct xfs_parent_name_irec {
 };
 
 void xfs_parent_irec_from_disk(struct xfs_parent_name_irec *irec,
-		const struct xfs_parent_name_rec *rec,
+		const struct xfs_parent_name_rec *rec, int reclen,
 		const void *value, int valuelen);
-void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, void *value,
-		int *valuelen, const struct xfs_parent_name_irec *irec);
+void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, int *reclen,
+		void *value, int *valuelen,
+		const struct xfs_parent_name_irec *irec);
 
 /*
  * Dynamically allocd structure used to wrap the needed data to pass around
fs/xfs/scrub/dir.c +8 -4
index 2494947a0c93..87cff40b15f1 100644
@@ -144,15 +144,19 @@ xchk_dir_parent_pointer(
 {
 	struct xfs_scrub	*sc = sd->sc;
 	int			pptr_namelen;
-	int			error;
+	int			hashlen;
 
 	sd->pptr.p_ino = sc->ip->i_ino;
 	sd->pptr.p_gen = VFS_I(sc->ip)->i_generation;
 
-	error = xfs_parent_namehash(ip, name, &sd->pptr.p_namehash,
+	hashlen = xfs_parent_namehash(ip, name, &sd->pptr.p_namehash,
 			sizeof(sd->pptr.p_namehash));
-	if (error)
-		return error;
+	if (hashlen < 0) {
+		xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0,
+				&hashlen);
+		return hashlen;
+	}
+	sd->pptr.hashlen = hashlen;
 
 	pptr_namelen = xfs_parent_lookup(sc->tp, ip, &sd->pptr, sd->namebuf,
 			MAXNAMELEN, &sd->pptr_scratch);
fs/xfs/scrub/dir_repair.c +1 -1
index c0b2b78da277..b12548787321 100644
@@ -540,7 +540,7 @@ xrep_dir_scan_parent_pointer(
 	    !xfs_parent_valuecheck(sc->mp, value, valuelen))
 		return -EFSCORRUPTED;
 
-	xfs_parent_irec_from_disk(&rd->pptr, rec, value, valuelen);
+	xfs_parent_irec_from_disk(&rd->pptr, rec, namelen, value, valuelen);
 
 	/* Ignore parent pointers that point back to a different dir. */
 	if (rd->pptr.p_ino != sc->ip->i_ino ||
fs/xfs/scrub/parent.c +10 -6
index 53872a7be942..b47f0bcef690 100644
@@ -531,6 +531,7 @@ xchk_parent_scan_attr(
 	struct xfs_inode	*dp = NULL;
 	const struct xfs_parent_name_rec *rec = (const void *)name;
 	unsigned int		lockmode;
+	int			hashlen;
 	int			error;
 
 	/* Ignore incomplete xattrs */
@@ -552,7 +553,7 @@ xchk_parent_scan_attr(
 		return -ECANCELED;
 	}
 
-	xfs_parent_irec_from_disk(&pp->pptr, rec, value, valuelen);
+	xfs_parent_irec_from_disk(&pp->pptr, rec, namelen, value, valuelen);
 
 	xname.name = pp->pptr.p_name;
 	xname.len = pp->pptr.p_namelen;
@@ -561,13 +562,16 @@ xchk_parent_scan_attr(
 	 * Does the namehash in the parent pointer match the actual name?
 	 * If not, there's no point in checking further.
 	 */
-	error = xfs_parent_namehash(sc->ip, &xname, pp->child_namehash,
+	hashlen = xfs_parent_namehash(sc->ip, &xname, pp->child_namehash,
 			sizeof(pp->child_namehash));
-	if (!xchk_fblock_xref_process_error(sc, XFS_ATTR_FORK, 0, &error))
-		return error;
+	if (hashlen < 0) {
+		xchk_fblock_xref_process_error(sc, XFS_ATTR_FORK, 0, &hashlen);
+		return hashlen;
+	}
 
-	if (memcmp(pp->pptr.p_namehash, pp->child_namehash,
-				sizeof(pp->pptr.p_namehash))) {
+	if (hashlen != pp->pptr.hashlen ||
+	    memcmp(pp->pptr.p_namehash, pp->child_namehash,
+				pp->pptr.hashlen)) {
 		trace_xchk_parent_bad_namehash(sc->ip, pp->pptr.p_ino,
 				xname.name, xname.len);
 		xchk_fblock_xref_set_corrupt(sc, XFS_ATTR_FORK, 0);
fs/xfs/scrub/parent_repair.c +1 -1
index 51432ab61c94..7d3b9c82bd05 100644
@@ -522,7 +522,7 @@ xrep_pptr_dump_tempptr(
 	else
 		return -EFSCORRUPTED;
 
-	xfs_parent_irec_from_disk(&rp->pptr, rec, value, valuelen);
+	xfs_parent_irec_from_disk(&rp->pptr, rec, namelen, value, valuelen);
 
 	trace_xrep_pptr_dumpname(sc->tempip, &rp->pptr);
 
fs/xfs/xfs_parent_utils.c +1 -1
index 65bec3875308..284ca3c14a0f 100644
@@ -74,7 +74,7 @@ xfs_getparent_listent(
 		return;
 	}
 
-	xfs_parent_irec_from_disk(&gp->pptr_irec, (void *)name, value,
+	xfs_parent_irec_from_disk(&gp->pptr_irec, (void *)name, namelen, value,
 			valuelen);
 
 	trace_xfs_getparent_listent(context->dp, ppi, irec);
#85 lore raw

PATCH 4/5 xfs: make the ondisk parent pointer record a flex array

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Now that we can use the filename as the parent pointer name hash, we always write the full 64 bytes into the xattr. In other words, the namehash is really a flex array, so adjust its C definition.

---
 fs/xfs/libxfs/xfs_da_format.h  |    9 ++++++---
 fs/xfs/libxfs/xfs_parent.c     |    4 ++--
 fs/xfs/libxfs/xfs_parent.h     |   15 ++++++++++++---
 fs/xfs/libxfs/xfs_trans_resv.c |    6 +++---
 fs/xfs/xfs_ondisk.h            |    2 +-
 5 files changed, 24 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +6 -3
index 275357506394..4d85830785ae 100644
@@ -842,21 +842,24 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 struct xfs_parent_name_rec {
 	__be64  p_ino;
 	__be32  p_gen;
-	__u8	p_namehash[XFS_PARENT_NAME_HASH_SIZE];
+	__u8	p_namehash[];
 } __attribute__((packed));
 
+#define XFS_PARENT_NAME_MAX_SIZE \
+	(sizeof(struct xfs_parent_name_rec) + XFS_PARENT_NAME_HASH_SIZE)
+
 static inline unsigned int
 xfs_parent_name_rec_sizeof(
 	unsigned int		hashlen)
 {
-	return offsetof(struct xfs_parent_name_rec, p_namehash) + hashlen;
+	return sizeof(struct xfs_parent_name_rec) + hashlen;
 }
 
 static inline unsigned int
 xfs_parent_name_hashlen(
 	unsigned int		rec_sizeof)
 {
-	return rec_sizeof - offsetof(struct xfs_parent_name_rec, p_namehash);
+	return rec_sizeof - sizeof(struct xfs_parent_name_rec);
 }
 
 #endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_parent.c +2 -2
index 32235a0e9e0d..6520e35178a0 100644
@@ -98,7 +98,7 @@ xfs_init_parent_name_rec(
 	rec->p_ino = cpu_to_be64(dp->i_ino);
 	rec->p_gen = cpu_to_be32(VFS_IC(dp)->i_generation);
 	return xfs_parent_namehash(ip, name, rec->p_namehash,
-			sizeof(rec->p_namehash));
+			XFS_PARENT_NAME_HASH_SIZE);
 }
 
 /*
@@ -197,7 +197,7 @@ __xfs_parent_init(
 	parent->args.attr_filter = XFS_ATTR_PARENT;
 	parent->args.op_flags = XFS_DA_OP_OKNOENT | XFS_DA_OP_LOGGED;
 	parent->args.name = (const uint8_t *)&parent->rec;
-	parent->args.namelen = sizeof(struct xfs_parent_name_rec);
+	parent->args.namelen = 0;
 
 	*parentp = parent;
 	return 0;
fs/xfs/libxfs/xfs_parent.h +12 -3
index 4c3100760bba..3431aac75e92 100644
@@ -43,8 +43,14 @@ void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, int *reclen,
  * the defer ops machinery
  */
 struct xfs_parent_defer {
-	struct xfs_parent_name_rec	rec;
-	struct xfs_parent_name_rec	old_rec;
+	union {
+		struct xfs_parent_name_rec	rec;
+		__u8			dummy1[XFS_PARENT_NAME_MAX_SIZE];
+	};
+	union {
+		struct xfs_parent_name_rec	old_rec;
+		__u8			dummy2[XFS_PARENT_NAME_MAX_SIZE];
+	};
 	struct xfs_da_args		args;
 	bool				have_log;
 };
@@ -112,7 +118,10 @@ unsigned int xfs_pptr_calc_space_res(struct xfs_mount *mp,
 
 /* Scratchpad memory so that raw parent operations don't burn stack space. */
 struct xfs_parent_scratch {
-	struct xfs_parent_name_rec	rec;
+	union {
+		struct xfs_parent_name_rec	rec;
+		__u8			dummy1[XFS_PARENT_NAME_MAX_SIZE];
+	};
 	struct xfs_da_args		args;
 };
 
fs/xfs/libxfs/xfs_trans_resv.c +3 -3
index 93419956b9e5..0e625c6b0153 100644
@@ -427,19 +427,19 @@ static inline unsigned int xfs_calc_pptr_link_overhead(void)
 {
 	return sizeof(struct xfs_attri_log_format) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
-			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 static inline unsigned int xfs_calc_pptr_unlink_overhead(void)
 {
 	return sizeof(struct xfs_attri_log_format) +
-			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 static inline unsigned int xfs_calc_pptr_replace_overhead(void)
 {
 	return sizeof(struct xfs_attri_log_format) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
-			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 
 /*
fs/xfs/xfs_ondisk.h +1 -1
index 2dc1eef63d96..24361ae0fd48 100644
@@ -116,7 +116,7 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, offset,		1);
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, name,		3);
 	XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_hdr_t,		10);
-	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec,	76);
+	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec,	12);
 	BUILD_BUG_ON(XFS_PARENT_NAME_HASH_SIZE != SHA512_DIGEST_SIZE);
 
 	/* log structures */
#86 lore raw

PATCH 5/5 xfs: use parent pointer xattr space more efficiently

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Amend the parent pointer xattr format even more. Now we put as much of the dirent name in the namehash as we can. For names that don't fit, the namehash is the truncated dirent name with the sha512 of the entire name at the end of the namehash. The EA value is then truncated to whatever doesn't fit in the namehash.

---
 fs/xfs/libxfs/xfs_da_format.h |   26 +++++++---
 fs/xfs/libxfs/xfs_parent.c    |  111 ++++++++++++++++++++++++++++++++---------
 fs/xfs/libxfs/xfs_parent.h    |    6 +-
 fs/xfs/scrub/dir_repair.c     |    2 -
 fs/xfs/scrub/parent.c         |    4 +
 fs/xfs/xfs_attr_item.c        |    4 +
 fs/xfs/xfs_ondisk.h           |    4 +
 7 files changed, 118 insertions(+), 39 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +20 -6
index 4d85830785ae..55f510f82e8d 100644
@@ -825,19 +825,24 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 				      struct xfs_da3_blkinfo *hdr3);
 
 /* We use sha512 for the parent pointer name hash. */
-#define XFS_PARENT_NAME_HASH_SIZE	(64)
+#define XFS_PARENT_NAME_SHA512_SIZE	(64)
 
 /*
  * Parent pointer attribute format definition
  *
  * The EA name encodes the parent inode number, generation and a collision
- * resistant hash computed from the dirent name.  The hash is defined to be:
+ * resistant hash computed from the dirent name.  The hash is defined to be
+ * one of the following:
  *
- * - The dirent name if it fits within the EA name.
+ * - The dirent name, as long as it does not use the last possible byte of the
+ *   EA name space.
  *
- * - The sha512 of the child inode generation and the dirent name.
+ * - The truncated dirent name, with the sha512 hash of the child inode
+ *   generation number and dirent name.  The hash is written at the end of the
+ *   EA name.
  *
- * The EA value contains the same name as the dirent in the parent directory.
+ * The EA value contains however much of the dirent name that does not fit in
+ * the EA name.
  */
 struct xfs_parent_name_rec {
 	__be64  p_ino;
@@ -845,8 +850,17 @@ struct xfs_parent_name_rec {
 	__u8	p_namehash[];
 } __attribute__((packed));
 
+/* Maximum size of a parent pointer EA name. */
 #define XFS_PARENT_NAME_MAX_SIZE \
-	(sizeof(struct xfs_parent_name_rec) + XFS_PARENT_NAME_HASH_SIZE)
+	(MAXNAMELEN - 1)
+
+/* Maximum size of a parent pointer name hash. */
+#define XFS_PARENT_NAME_MAX_HASH_SIZE \
+	(XFS_PARENT_NAME_MAX_SIZE - sizeof(struct xfs_parent_name_rec))
+
+/* Offset of the sha512 hash, if used. */
+#define XFS_PARENT_NAME_SHA512_OFFSET \
+	(XFS_PARENT_NAME_MAX_HASH_SIZE - XFS_PARENT_NAME_SHA512_SIZE)
 
 static inline unsigned int
 xfs_parent_name_rec_sizeof(
fs/xfs/libxfs/xfs_parent.c +87 -24

238 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +3 -3
index 3431aac75e92..6f6136165efe 100644
@@ -12,8 +12,8 @@ extern struct kmem_cache	*xfs_parent_intent_cache;
 bool xfs_parent_namecheck(struct xfs_mount *mp,
 		const struct xfs_parent_name_rec *rec, size_t reclen,
 		unsigned int attr_flags);
-bool xfs_parent_valuecheck(struct xfs_mount *mp, const void *value,
-		size_t valuelen);
+bool xfs_parent_valuecheck(struct xfs_mount *mp, size_t namelen,
+		const void *value, size_t valuelen);
 
 /*
  * Incore version of a parent pointer, also contains dirent name so callers
@@ -24,7 +24,7 @@ struct xfs_parent_name_irec {
 	xfs_ino_t		p_ino;
 	uint32_t		p_gen;
 	uint8_t			hashlen;
-	uint8_t			p_namehash[XFS_PARENT_NAME_HASH_SIZE];
+	uint8_t			p_namehash[XFS_PARENT_NAME_MAX_HASH_SIZE];
 
 	/* Attributes of a parent pointer. */
 	uint8_t			p_namelen;
fs/xfs/scrub/dir_repair.c +1 -1
index b12548787321..76953575f0b2 100644
@@ -537,7 +537,7 @@ xrep_dir_scan_parent_pointer(
 
 	/* Does the ondisk parent pointer structure make sense? */
 	if (!xfs_parent_namecheck(sc->mp, rec, namelen, attr_flags) ||
-	    !xfs_parent_valuecheck(sc->mp, value, valuelen))
+	    !xfs_parent_valuecheck(sc->mp, namelen, value, valuelen))
 		return -EFSCORRUPTED;
 
 	xfs_parent_irec_from_disk(&rd->pptr, rec, namelen, value, valuelen);
fs/xfs/scrub/parent.c +2 -2
index b47f0bcef690..f3b1d7cbe415 100644
@@ -350,7 +350,7 @@ struct xchk_pptrs {
 	struct xfs_parent_scratch pptr_scratch;
 
 	/* Name hashes */
-	uint8_t			child_namehash[XFS_PARENT_NAME_HASH_SIZE];
+	uint8_t			child_namehash[XFS_PARENT_NAME_MAX_HASH_SIZE];
 
 	/* Name buffer for revalidation. */
 	uint8_t			namebuf[MAXNAMELEN];
@@ -548,7 +548,7 @@ xchk_parent_scan_attr(
 		return -ECANCELED;
 	}
 
-	if (!xfs_parent_valuecheck(sc->mp, value, valuelen)) {
+	if (!xfs_parent_valuecheck(sc->mp, namelen, value, valuelen)) {
 		xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0);
 		return -ECANCELED;
 	}
fs/xfs/xfs_attr_item.c +2 -2
index da807f286a09..792c01a49749 100644
@@ -769,7 +769,7 @@ xlog_recover_attri_commit_pass2(
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
-		if (item->ri_total != 3) {
+		if (item->ri_total != 3 && item->ri_total != 2) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
 			return -EFSCORRUPTED;
@@ -783,7 +783,7 @@ xlog_recover_attri_commit_pass2(
 		}
 		break;
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
-		if (item->ri_total != 4) {
+		if (item->ri_total != 3 && item->ri_total != 4) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
 			return -EFSCORRUPTED;
fs/xfs/xfs_ondisk.h +3 -1
index 24361ae0fd48..5f32dea26221 100644
@@ -117,7 +117,9 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, name,		3);
 	XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_hdr_t,		10);
 	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec,	12);
-	BUILD_BUG_ON(XFS_PARENT_NAME_HASH_SIZE != SHA512_DIGEST_SIZE);
+	BUILD_BUG_ON(XFS_PARENT_NAME_MAX_HASH_SIZE < SHA512_DIGEST_SIZE);
+	BUILD_BUG_ON(XFS_PARENT_NAME_MAX_HASH_SIZE !=           243);
+	BUILD_BUG_ON(XFS_PARENT_NAME_SHA512_OFFSET !=           179);
 
 	/* log structures */
 	XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format,	88);
#87 lore raw

Re: [PATCHSET v9r2d1 0/5] xfs: encode parent pointer name in xattr key

In reply to Darrick J. Wong
Quoted 68 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:

    (parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)

The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:

For dirent names shorter than 243 bytes:

    (parent_ino, parent_gen, dirent_name) -> NULL
Quoted 9 lines

For dirent names longer than 243 bytes:

    (parent_ino, parent_gen, dirent_name[0:178],
     sha512(child_gen, dirent_name)) -> (dirent_name[179:255])

The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.

Naive question:

Obviously, the spec of stradard xattrs does not allow duplicate keys, but dabtree does allow duplicate keys, does it not?

So if you were to allow duplicate parent pointer records, e.g.:

(parent_ino, parent_gen) -> dirent_name1
(parent_ino, parent_gen) -> dirent_name2

Or to optimize performance for the case of large number of sibling hardlinks of the same parent (if that case is worth optimizing):

(parent_ino, parent_gen, dirent_name[0:178]) -> (dirent_name1[179:255]) (parent_ino, parent_gen, dirent_name[0:178]) -> (dirent_name2[179:255])

Then pptr code should have no problem walking all the matching parent pointer records to find the unique parent->child record that it needs to operate on?

I am sure it would be more complicated than how I depicted it, but having to deal with even remote possibility of hash collisions sounds like a massive headache - having to maintain code that is really hard to test and rarely exercised is not a recipe for peace of mind...

Thanks,
Amir.

#88 lore raw

Re: [PATCHSET v9r2d1 0/5] xfs: encode parent pointer name in xattr key

Quoted 79 lines · Amir Goldstein
Quoted 68 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:

    (parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)

The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:

For dirent names shorter than 243 bytes:

    (parent_ino, parent_gen, dirent_name) -> NULL
Quoted 9 lines

For dirent names longer than 243 bytes:

    (parent_ino, parent_gen, dirent_name[0:178],
     sha512(child_gen, dirent_name)) -> (dirent_name[179:255])

The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.

Quoted 4 lines

Naive question:

Obviously, the spec of stradard xattrs does not allow duplicate keys, but dabtree does allow duplicate keys, does it not?

The dabtree allows duplicate hashes for a given name, yes. It doesn't allow for duplicate names, though.

(Also note that small xattr structures skip the dabtree and hashing.)

Quoted 8 lines

So if you were to allow duplicate parent pointer records, e.g.:

(parent_ino, parent_gen) -> dirent_name1
(parent_ino, parent_gen) -> dirent_name2

Or to optimize performance for the case of large number of sibling hardlinks of the same parent (if that case is worth optimizing):

Quoted 6 lines

(parent_ino, parent_gen, dirent_name[0:178]) -> (dirent_name1[179:255]) (parent_ino, parent_gen, dirent_name[0:178]) -> (dirent_name2[179:255])

Then pptr code should have no problem walking all the matching parent pointer records to find the unique parent->child record that it needs to operate on?

Theoretically, yes, the parent pointer code could walk all the xattrs that have the same attr name looking for the one with matching value. But keep in mind that there could be min(2^32, 2^(8+76)) potential matches.

The other difficulty is that the xattr lookup and removal code don't have a means to return the dastate to callers or for callers to provide a dastate to go back into the xattr code. You'd need that to identify the specific parent pointer xattr you want to operate on.

Quoted 4 lines

I am sure it would be more complicated than how I depicted it, but having to deal with even remote possibility of hash collisions sounds like a massive headache - having to maintain code that is really hard to test and rarely exercised is not a recipe for peace of mind...

Yep. Hash functions are definitely finger-crossing headhurting.

--D

Quoted 2 lines

Thanks,
Amir.

#89 lore raw

Re: [PATCHSET v9r2d1 0/5] xfs: encode parent pointer name in xattr key

In reply to Darrick J. Wong
Quoted 63 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:

    (parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)

The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:

Quoted 8 lines

For dirent names shorter than 243 bytes:

    (parent_ino, parent_gen, dirent_name) -> NULL

For dirent names longer than 243 bytes:

    (parent_ino, parent_gen, dirent_name[0:178],
     sha512(child_gen, dirent_name)) -> (dirent_name[179:255])

Heh, that should've been dirent_name[179:254].

Quoted 8 lines

The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.

A messier solution to this problem would be to extend the xattr ondisk format to allow parent pointers to have xattr names up to 267 bytes. This would likely involve redefining the ondisk namelen field to omit the size of the parent ino/gen information and might be madness.

Update:

After some subtle prodding from Dave, I realized that there's a simpler solution to this problem: extend the xattr match predicate to check both the xattr name /and/ the xattr value. Parent pointers cannot be remote format because the amount of data are never larger than 3/4 of 1FSB, so the value of an ATTR_PARENT attribute is always immediately available.

The pptr ondisk format becomes one of:

    (parent_ino, parent_gen, dirent_name) -> NULL
    (parent_ino, parent_gen, dirent_name[0:242]) -> (dirent_name[243:254])

Matching on both xattr name and value is only useful* for parent pointers, so I introduced a new XFS_DA_OP_VLOOKUP flag to gate this new mode. No more sha512 in the attr name, no more worrying about collision resistance of sha*, and one less dependency for xfs.

The next challenge was to log the VLOOKUP flag when we're doing xattr operations and to recover that state when replaying an ATTRI log item. This I did by creating new "NV" variants of ATTRI_OP_FLAGS_{SET,REMOVE}. The NVSET and NVREMOVE opcodes require VLOOKUP, and NVREMOVE can log an xattr value buffer. Since parent pointers is an unmerged incompat feature, we don't need a new log-incompat feature to protect them.

The last thing was to update NVREPLACE so that we can handle rename operations. Here too we need VLOOKUPs, but I also needed to log both the old value and the new value, so I changed the attri ondisk format. For NVREPLACE, the old and new name lengths are two u16 overlaid atop the alfi_name_len field; and the new value length is a u32 that replaces the old pad (and what Allison called alfi_nname_len).

I'm going to run QA on this over the weekend, and figure out how to collapse this patchset with the new one. I'll rebase the whole branch on pptrs v10 whenever it comes out.

--D

* Unless someone has a usecase for cmpxchg of extended attributes?

Quoted 38 lines

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key

---
 fs/xfs/Kconfig                 |    1 
 fs/xfs/libxfs/xfs_da_format.h  |   49 +++++++
 fs/xfs/libxfs/xfs_fs.h         |    4 -
 fs/xfs/libxfs/xfs_parent.c     |  265 ++++++++++++++++++++++++++++++++--------
 fs/xfs/libxfs/xfs_parent.h     |   48 +++++--
 fs/xfs/libxfs/xfs_trans_resv.c |    6 -
 fs/xfs/scrub/dir.c             |   16 ++
 fs/xfs/scrub/dir_repair.c      |   87 ++++---------
 fs/xfs/scrub/parent.c          |   51 +++++---
 fs/xfs/scrub/parent_repair.c   |   29 ++--
 fs/xfs/scrub/trace.h           |   48 ++-----
 fs/xfs/xfs_attr_item.c         |    4 -
 fs/xfs/xfs_inode.c             |   30 ++---
 fs/xfs/xfs_linux.h             |    1 
 fs/xfs/xfs_mount.c             |   13 ++
 fs/xfs/xfs_mount.h             |    3 
 fs/xfs/xfs_ondisk.h            |    6 +
 fs/xfs/xfs_parent_utils.c      |    4 -
 fs/xfs/xfs_sha512.h            |   42 ++++++
 fs/xfs/xfs_super.c             |    3 
 fs/xfs/xfs_symlink.c           |    3 
 21 files changed, 481 insertions(+), 232 deletions(-)
 create mode 100644 fs/xfs/xfs_sha512.h
#90 lore raw

cover xfs: remove parent pointer hashing

In reply to Darrick J. Wong

Hi all,

Dave Chinner pointed out (a bit too subtly) that hashing the dirent name to try to squash it into the parent pointer xattr name is unnecessary because we could simply make the xattr matching predicate compare names. Do that instead and drop the hashing.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-vlookup

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-vlookup

---
 fs/xfs/Kconfig                 |    1 
 fs/xfs/libxfs/xfs_attr.c       |   39 +++--
 fs/xfs/libxfs/xfs_attr_leaf.c  |   41 +++++
 fs/xfs/libxfs/xfs_da_btree.h   |    6 +
 fs/xfs/libxfs/xfs_da_format.h  |   34 ++---
 fs/xfs/libxfs/xfs_log_format.h |   30 +++-
 fs/xfs/libxfs/xfs_parent.c     |  302 +++++++++++++---------------------------
 fs/xfs/libxfs/xfs_parent.h     |   16 --
 fs/xfs/libxfs/xfs_trans_resv.c |    1 
 fs/xfs/scrub/dir.c             |   38 +----
 fs/xfs/scrub/parent.c          |   61 +-------
 fs/xfs/scrub/parent_repair.c   |   34 +----
 fs/xfs/xfs_attr_item.c         |  217 ++++++++++++++++++++---------
 fs/xfs/xfs_attr_item.h         |    3 
 fs/xfs/xfs_linux.h             |    1 
 fs/xfs/xfs_mount.c             |   13 --
 fs/xfs/xfs_mount.h             |    3 
 fs/xfs/xfs_ondisk.h            |    3 
 fs/xfs/xfs_sha512.h            |   42 ------
 fs/xfs/xfs_super.c             |    3 
 fs/xfs/xfs_xattr.c             |    5 +
 21 files changed, 383 insertions(+), 510 deletions(-)
 delete mode 100644 fs/xfs/xfs_sha512.h
#91 lore raw

PATCH 1/13 xfs: make xfs_attr_set require XFS_DA_OP_REMOVE

From: Darrick J. Wong <[email protected]>

In the next patch we're going to add the ability to look up local/sf xattrs based on the attr name and value matching. As a result, we need callers of xfs_attr_set to declare explicitly that they want to remove an xattr. Passing in NULL value will no longer suffice.

---
 fs/xfs/libxfs/xfs_attr.c   |    9 +++++----
 fs/xfs/libxfs/xfs_parent.c |    1 +
 fs/xfs/xfs_xattr.c         |    5 +++++
 3 files changed, 11 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -4
index 3065dd622102..756d93526075 100644
@@ -977,6 +977,7 @@ xfs_attr_set(
 	struct xfs_inode	*dp = args->dp;
 	struct xfs_mount	*mp = dp->i_mount;
 	struct xfs_trans_res	tres;
+	bool			is_remove = args->op_flags & XFS_DA_OP_REMOVE;
 	bool			rsvd;
 	int			error, local;
 	int			rmt_blks = 0;
@@ -1004,7 +1005,7 @@ xfs_attr_set(
 	args->op_flags = XFS_DA_OP_OKNOENT |
 					(args->op_flags & XFS_DA_OP_LOGGED);
 
-	if (args->value) {
+	if (!is_remove) {
 		XFS_STATS_INC(mp, xs_attr_set);
 		args->total = xfs_attr_calc_size(args, &local);
 
@@ -1038,7 +1039,7 @@ xfs_attr_set(
 	if (error)
 		return error;
 
-	if (args->value || xfs_inode_hasattr(dp)) {
+	if (!is_remove || xfs_inode_hasattr(dp)) {
 		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
 				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
 		if (error == -EFBIG)
@@ -1052,7 +1053,7 @@ xfs_attr_set(
 	switch (error) {
 	case -EEXIST:
 		/* if no value, we are performing a remove operation */
-		if (!args->value) {
+		if (is_remove) {
 			error = xfs_attr_defer_remove(args);
 			break;
 		}
@@ -1064,7 +1065,7 @@ xfs_attr_set(
 		break;
 	case -ENOATTR:
 		/* Can't remove what isn't there. */
-		if (!args->value)
+		if (is_remove)
 			goto out_trans_cancel;
 
 		/* Pure replace fails if no existing attr to replace. */
fs/xfs/libxfs/xfs_parent.c +1 -0
index f7fecee93894..387f3c65287f 100644
@@ -460,6 +460,7 @@ xfs_parent_unset(
 	scr->args.geo		= ip->i_mount->m_attr_geo;
 	scr->args.name		= (const unsigned char *)&scr->rec;
 	scr->args.namelen	= reclen;
+	scr->args.op_flags	= XFS_DA_OP_REMOVE;
 	scr->args.whichfork	= XFS_ATTR_FORK;
 
 	return xfs_attr_set(&scr->args);
fs/xfs/xfs_xattr.c +5 -0
index 85edd7e05fde..8f8aa13bf7eb 100644
@@ -103,6 +103,11 @@ xfs_attr_change(
 		use_logging = true;
 	}
 
+	if (args->value)
+		args->op_flags &= ~XFS_DA_OP_REMOVE;
+	else
+		args->op_flags |= XFS_DA_OP_REMOVE;
+
 	error = xfs_attr_set(args);
 
 	if (use_logging)
#92 lore raw

PATCH 2/13 xfs: allow xattr matching on value for local/sf attrs

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

---
 fs/xfs/libxfs/xfs_attr_leaf.c |   41 +++++++++++++++++++++++++++++++++++------
 fs/xfs/libxfs/xfs_da_btree.h  |    4 +++-
 2 files changed, 38 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +35 -6
index e6c4c8b52a55..d05f2c5cc0cc 100644
@@ -473,10 +473,12 @@ xfs_attr3_leaf_read(
  */
 static bool
 xfs_attr_match(
-	struct xfs_da_args	*args,
-	uint8_t			namelen,
-	unsigned char		*name,
-	int			flags)
+	const struct xfs_da_args	*args,
+	uint8_t				namelen,
+	const unsigned char		*name,
+	unsigned int			valuelen,
+	const void			*value,
+	int				flags)
 {
 
 	if (args->namelen != namelen)
@@ -484,6 +486,23 @@ xfs_attr_match(
 	if (memcmp(args->name, name, namelen) != 0)
 		return false;
 
+	if (args->op_flags & XFS_DA_OP_VLOOKUP) {
+		if (args->valuelen != valuelen)
+			return false;
+		if (args->valuelen && !value) {
+			/* not implemented for remote values */
+			ASSERT(0);
+			return false;
+		}
+		if (valuelen && !args->value) {
+			/* caller gave us valuelen > 0 but no value?? */
+			ASSERT(0);
+			return false;
+		}
+		if (valuelen > 0 && memcmp(args->value, value, valuelen) != 0)
+			return false;
+	}
+
 	/* Recovery ignores the INCOMPLETE flag. */
 	if ((args->op_flags & XFS_DA_OP_RECOVERY) &&
 	    args->attr_filter == (flags & XFS_ATTR_NSP_ONDISK_MASK))
@@ -502,6 +521,10 @@ xfs_attr_copy_value(
 	unsigned char		*value,
 	int			valuelen)
 {
+	/* vlookups already supplied the attr value; don't copy anything */
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		return 0;
+
 	/*
 	 * No copy if all we have to do is get the length
 	 */
@@ -726,6 +749,7 @@ xfs_attr_sf_findname(
 			     base += size, i++) {
 		size = xfs_attr_sf_entsize(sfe);
 		if (!xfs_attr_match(args, sfe->namelen, sfe->nameval,
+				    sfe->valuelen, &sfe->nameval[sfe->namelen],
 				    sfe->flags))
 			continue;
 		break;
@@ -896,6 +920,7 @@ xfs_attr_shortform_lookup(xfs_da_args_t *args)
 	for (i = 0; i < sf->hdr.count;
 				sfe = xfs_attr_sf_nextentry(sfe), i++) {
 		if (xfs_attr_match(args, sfe->namelen, sfe->nameval,
+				sfe->valuelen, &sfe->nameval[sfe->namelen],
 				sfe->flags))
 			return -EEXIST;
 	}
@@ -923,6 +948,7 @@ xfs_attr_shortform_getvalue(
 	for (i = 0; i < sf->hdr.count;
 				sfe = xfs_attr_sf_nextentry(sfe), i++) {
 		if (xfs_attr_match(args, sfe->namelen, sfe->nameval,
+				sfe->valuelen, &sfe->nameval[sfe->namelen],
 				sfe->flags))
 			return xfs_attr_copy_value(args,
 				&sfe->nameval[args->namelen], sfe->valuelen);
@@ -2484,14 +2510,17 @@ xfs_attr3_leaf_lookup_int(
 		if (entry->flags & XFS_ATTR_LOCAL) {
 			name_loc = xfs_attr3_leaf_name_local(leaf, probe);
 			if (!xfs_attr_match(args, name_loc->namelen,
-					name_loc->nameval, entry->flags))
+					name_loc->nameval,
+					be16_to_cpu(name_loc->valuelen),
+					&name_loc->nameval[name_loc->namelen],
+					entry->flags))
 				continue;
 			args->index = probe;
 			return -EEXIST;
 		} else {
 			name_rmt = xfs_attr3_leaf_name_remote(leaf, probe);
 			if (!xfs_attr_match(args, name_rmt->namelen,
-					name_rmt->name, entry->flags))
+					name_rmt->name, 0, NULL, entry->flags))
 				continue;
 			args->index = probe;
 			args->rmtvaluelen = be32_to_cpu(name_rmt->valuelen);
fs/xfs/libxfs/xfs_da_btree.h +3 -1
index 90b86d00258f..0ef32f629e1b 100644
@@ -96,6 +96,7 @@ typedef struct xfs_da_args {
 #define XFS_DA_OP_REMOVE	(1u << 6) /* this is a remove operation */
 #define XFS_DA_OP_RECOVERY	(1u << 7) /* Log recovery operation */
 #define XFS_DA_OP_LOGGED	(1u << 8) /* Use intent items to track op */
+#define XFS_DA_OP_VLOOKUP	(1u << 9) /* Compare attr value during lookup */
 
 #define XFS_DA_OP_FLAGS \
 	{ XFS_DA_OP_JUSTCHECK,	"JUSTCHECK" }, \
@@ -106,7 +107,8 @@ typedef struct xfs_da_args {
 	{ XFS_DA_OP_NOTIME,	"NOTIME" }, \
 	{ XFS_DA_OP_REMOVE,	"REMOVE" }, \
 	{ XFS_DA_OP_RECOVERY,	"RECOVERY" }, \
-	{ XFS_DA_OP_LOGGED,	"LOGGED" }
+	{ XFS_DA_OP_LOGGED,	"LOGGED" }, \
+	{ XFS_DA_OP_VLOOKUP,	"VLOOKUP" }
 
 /*
  * Storage for holding state during Btree searches and split/join ops.
#93 lore raw

PATCH 3/13 xfs: preserve VLOOKUP in xfs_attr_set

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Preserve the attr-value lookup flag when calling xfs_attr_set. Normal xattr users will never use this, but parent pointer fsck will.

---
 fs/xfs/libxfs/xfs_attr.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +4 -4
index 756d93526075..86672061c99e 100644
@@ -999,11 +999,11 @@ xfs_attr_set(
 	/*
 	 * We have no control over the attribute names that userspace passes us
 	 * to remove, so we have to allow the name lookup prior to attribute
-	 * removal to fail as well.  Preserve the logged flag, since we need
-	 * to pass that through to the logging code.
+	 * removal to fail as well.  Preserve the logged and vlookup flags,
+	 * since we need to pass them through to the lower levels.
 	 */
-	args->op_flags = XFS_DA_OP_OKNOENT |
-					(args->op_flags & XFS_DA_OP_LOGGED);
+	args->op_flags &= (XFS_DA_OP_LOGGED | XFS_DA_OP_VLOOKUP);
+	args->op_flags |= XFS_DA_OP_OKNOENT;
 
 	if (!is_remove) {
 		XFS_STATS_INC(mp, xs_attr_set);
#94 lore raw

PATCH 4/13 xfs: log VLOOKUP xattr removal operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If high level code wants to do a deferred xattr remove operation with the VLOOKUP flag set, we need to push this through the log.

---
 fs/xfs/libxfs/xfs_attr.c       |    6 +++++-
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/xfs_attr_item.c         |    7 +++++++
 3 files changed, 13 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
index 86672061c99e..6468286d2d71 100644
@@ -953,9 +953,13 @@ xfs_attr_defer_remove(
 {
 
 	struct xfs_attr_intent	*new;
+	int			op_flag = XFS_ATTRI_OP_FLAGS_REMOVE;
 	int			error;
 
-	error  = xfs_attr_intent_init(args, XFS_ATTRI_OP_FLAGS_REMOVE, &new);
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVREMOVE;
+
+	error  = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
 		return error;
 
fs/xfs/libxfs/xfs_log_format.h +1 -0
index 727b5a858028..a3d95a3d8476 100644
@@ -959,6 +959,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_REMOVE	2	/* Remove the attribute */
 #define XFS_ATTRI_OP_FLAGS_REPLACE	3	/* Replace the attribute */
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
+#define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +7 -0
index 792c01a49749..08cb26d6b37b 100644
@@ -544,6 +544,7 @@ xfs_attri_validate(
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
 		break;
 	default:
 		return false;
@@ -643,6 +644,11 @@ xfs_attri_item_recover(
 		else
 			attr->xattri_dela_state = xfs_attr_init_add_state(args);
 		break;
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
+		args->op_flags |= XFS_DA_OP_VLOOKUP;
+		args->value = nv->value.i_addr;
+		args->valuelen = nv->value.i_len;
+		fallthrough;
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 		if (!xfs_inode_hasattr(args->dp))
 			goto out;
@@ -769,6 +775,7 @@ xlog_recover_attri_commit_pass2(
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
 		if (item->ri_total != 3 && item->ri_total != 2) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
#95 lore raw

PATCH 5/13 xfs: log VLOOKUP xattr setting operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If high level code wants to do a deferred xattr set operation with the VLOOKUP flag set, we need to push this through the log.

---
 fs/xfs/libxfs/xfs_attr.c       |    6 +++++-
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/xfs_attr_item.c         |    5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
index 6468286d2d71..ba8ad232b306 100644
@@ -910,9 +910,13 @@ xfs_attr_defer_add(
 	struct xfs_da_args	*args)
 {
 	struct xfs_attr_intent	*new;
+	int			op_flag = XFS_ATTRI_OP_FLAGS_SET;
 	int			error = 0;
 
-	error = xfs_attr_intent_init(args, XFS_ATTRI_OP_FLAGS_SET, &new);
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVSET;
+
+	error = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
 		return error;
 
fs/xfs/libxfs/xfs_log_format.h +1 -0
index a3d95a3d8476..1fe9f7394812 100644
@@ -960,6 +960,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_REPLACE	3	/* Replace the attribute */
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
+#define XFS_ATTRI_OP_FLAGS_NVSET	6	/* Set attr with w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +5 -0
index 08cb26d6b37b..79a459e8d51a 100644
@@ -545,6 +545,7 @@ xfs_attri_validate(
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
+	case XFS_ATTRI_OP_FLAGS_NVSET:
 		break;
 	default:
 		return false;
@@ -633,6 +634,9 @@ xfs_attri_item_recover(
 	ASSERT(xfs_sb_version_haslogxattrs(&mp->m_sb));
 
 	switch (attr->xattri_op_flags) {
+	case XFS_ATTRI_OP_FLAGS_NVSET:
+		args->op_flags |= XFS_DA_OP_VLOOKUP;
+		fallthrough;
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
@@ -773,6 +777,7 @@ xlog_recover_attri_commit_pass2(
 
 	op = attri_formatp->alfi_op_flags & XFS_ATTRI_OP_FLAGS_TYPE_MASK;
 	switch (op) {
+	case XFS_ATTRI_OP_FLAGS_NVSET:
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
#96 lore raw

PATCH 6/13 xfs: refactor extracting attri ops from alfi_op_flags

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Refactor this very long expression into a helper before we start adding more of them.

---
 fs/xfs/xfs_attr_item.c |   14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
fs/xfs/xfs_attr_item.c +9 -5
index 79a459e8d51a..6dce2110a871 100644
@@ -374,6 +374,12 @@ xfs_xattri_finish_update(
 	return error;
 }
 
+static inline unsigned int
+xfs_attr_log_item_op(const struct xfs_attri_log_format *attrp)
+{
+	return attrp->alfi_op_flags & XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+}
+
 /* Log an attr to the intent item. */
 STATIC void
 xfs_attr_log_item(
@@ -525,8 +531,7 @@ xfs_attri_validate(
 	struct xfs_mount		*mp,
 	struct xfs_attri_log_format	*attrp)
 {
-	unsigned int			op = attrp->alfi_op_flags &
-					     XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+	unsigned int			op = xfs_attr_log_item_op(attrp);
 
 	if (attrp->alfi_op_flags != XFS_ATTRI_OP_FLAGS_NVREPLACE &&
 	    attrp->alfi_nname_len != 0)
@@ -608,8 +613,7 @@ xfs_attri_item_recover(
 	args = (struct xfs_da_args *)(attr + 1);
 
 	attr->xattri_da_args = args;
-	attr->xattri_op_flags = attrp->alfi_op_flags &
-						XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+	attr->xattri_op_flags = xfs_attr_log_item_op(attrp);
 
 	/*
 	 * We're reconstructing the deferred work state structure from the
@@ -775,7 +779,7 @@ xlog_recover_attri_commit_pass2(
 		return -EFSCORRUPTED;
 	}
 
-	op = attri_formatp->alfi_op_flags & XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+	op = xfs_attr_log_item_op(attri_formatp);
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_NVSET:
 	case XFS_ATTRI_OP_FLAGS_SET:
#97 lore raw

PATCH 7/13 xfs: overlay alfi_nname_len atop alfi_name_len for NVREPLACE

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

In preparation for being able to log the old attr value in a NVREPLACE operation, encode the old and new name lengths in the alfi_name_len field. We haven't shipped a kernel with XFS_ATTRI_OP_FLAGS_NVREPLACE, so we can still tweak the ondisk log item format.

---
 fs/xfs/libxfs/xfs_log_format.h |   14 ++++++-
 fs/xfs/xfs_attr_item.c         |   81 ++++++++++++++++++++++++----------------
 2 files changed, 60 insertions(+), 35 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +12 -2
index 1fe9f7394812..32035786135b 100644
@@ -979,11 +979,21 @@ struct xfs_icreate_log {
 struct xfs_attri_log_format {
 	uint16_t	alfi_type;	/* attri log item type */
 	uint16_t	alfi_size;	/* size of this item */
-	uint32_t	alfi_nname_len;	/* attr new name length */
+	uint32_t	__pad;		/* pad to 64 bit aligned */
 	uint64_t	alfi_id;	/* attri identifier */
 	uint64_t	alfi_ino;	/* the inode for this attr operation */
 	uint32_t	alfi_op_flags;	/* marks the op as a set or remove */
-	uint32_t	alfi_name_len;	/* attr name length */
+	union {
+		uint32_t	alfi_name_len;	/* attr name length */
+		struct {
+			/*
+			 * For NVREPLACE, these are the lengths of the old and
+			 * new attr name.
+			 */
+			uint16_t	alfi_oldname_len;
+			uint16_t	alfi_newname_len;
+		};
+	};
 	uint32_t	alfi_value_len;	/* attr value length */
 	uint32_t	alfi_attr_filter;/* attr filter flags */
 };
fs/xfs/xfs_attr_item.c +48 -33

169 lines not on this page · raw message

#98 lore raw

PATCH 8/13 xfs: rename nname to newname

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Burn a couple of extra bytes to make it clear what this does.

---
 fs/xfs/libxfs/xfs_log_format.h |    2 +-
 fs/xfs/xfs_attr_item.c         |   48 ++++++++++++++++++++--------------------
 fs/xfs/xfs_attr_item.h         |    2 +-
 3 files changed, 26 insertions(+), 26 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +1 -1
index 32035786135b..8b16ae27c2fd 100644
@@ -117,7 +117,7 @@ struct xfs_unmount_log_format {
 #define XLOG_REG_TYPE_ATTRD_FORMAT	28
 #define XLOG_REG_TYPE_ATTR_NAME	29
 #define XLOG_REG_TYPE_ATTR_VALUE	30
-#define XLOG_REG_TYPE_ATTR_NNAME	31
+#define XLOG_REG_TYPE_ATTR_NEWNAME	31
 #define XLOG_REG_TYPE_MAX		31
 
 
fs/xfs/xfs_attr_item.c +24 -24

125 lines not on this page · raw message

fs/xfs/xfs_attr_item.h +1 -1
index 24d4968dd6cc..e374712ba06b 100644
@@ -13,7 +13,7 @@ struct kmem_zone;
 
 struct xfs_attri_log_nameval {
 	struct xfs_log_iovec	name;
-	struct xfs_log_iovec	nname;
+	struct xfs_log_iovec	newname;
 	struct xfs_log_iovec	value;
 	refcount_t		refcount;
 
#99 lore raw

PATCH 9/13 xfs: log VLOOKUP xattr nvreplace operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If high level code wants to do a deferred xattr nvreplace operation with the VLOOKUP flag set, we need to push this through the log. To avoid breaking the parent pointer code, we'll temporarily create a new NVREPLACEXXX flag that connects to the VLOOKUP flag.

---
 fs/xfs/libxfs/xfs_attr.c       |   15 +++++++++++----
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/xfs_attr_item.c         |   13 +++++++++++--
 3 files changed, 23 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_attr.c +11 -4
index ba8ad232b306..b9178c4efdeb 100644
@@ -424,7 +424,12 @@ xfs_attr_complete_op(
 	args->op_flags &= ~XFS_DA_OP_REPLACE;
 	if (do_replace) {
 		args->attr_filter &= ~XFS_ATTR_INCOMPLETE;
-		if (args->new_namelen > 0) {
+		if (args->op_flags & XFS_DA_OP_VLOOKUP) {
+			args->name = args->new_name;
+			args->namelen = args->new_namelen;
+			args->hashval = xfs_da_hashname(args->name,
+							args->namelen);
+		} else if (args->new_namelen > 0) {
 			args->name = args->new_name;
 			args->namelen = args->new_namelen;
 			args->hashval = xfs_da_hashname(args->name,
@@ -933,11 +938,13 @@ xfs_attr_defer_replace(
 	struct xfs_da_args	*args)
 {
 	struct xfs_attr_intent	*new;
-	int			op_flag;
+	int			op_flag = XFS_ATTRI_OP_FLAGS_REPLACE;
 	int			error = 0;
 
-	op_flag = args->new_namelen == 0 ? XFS_ATTRI_OP_FLAGS_REPLACE :
-		  XFS_ATTRI_OP_FLAGS_NVREPLACE;
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVREPLACEXXX;
+	else if (args->new_namelen > 0)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVREPLACE;
 
 	error = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
fs/xfs/libxfs/xfs_log_format.h +1 -0
index 8b16ae27c2fd..a1581dc6f131 100644
@@ -961,6 +961,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_NVSET	6	/* Set attr with w/ vlookup */
+#define XFS_ATTRI_OP_FLAGS_NVREPLACEXXX	7	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +11 -2
index 83e83aa05f94..0dd49c5f235a 100644
@@ -403,7 +403,10 @@ xfs_attr_log_item(
 	attrp->alfi_op_flags = attr->xattri_op_flags;
 	attrp->alfi_value_len = attr->xattri_nameval->value.i_len;
 
-	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
+	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACEXXX) {
+		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
+		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
+	} else if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
 		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
 	} else {
@@ -564,6 +567,7 @@ xfs_attri_validate(
 		if (attrp->alfi_value_len > XATTR_SIZE_MAX)
 			return false;
 		break;
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		if (attrp->alfi_oldname_len == 0 ||
 		    attrp->alfi_oldname_len > XATTR_NAME_MAX)
@@ -649,6 +653,7 @@ xfs_attri_item_recover(
 
 	switch (attr->xattri_op_flags) {
 	case XFS_ATTRI_OP_FLAGS_NVSET:
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 		args->op_flags |= XFS_DA_OP_VLOOKUP;
 		fallthrough;
 	case XFS_ATTRI_OP_FLAGS_SET:
@@ -747,7 +752,10 @@ xfs_attri_item_relog(
 	new_attrp->alfi_ino = old_attrp->alfi_ino;
 	new_attrp->alfi_op_flags = old_attrp->alfi_op_flags;
 	new_attrp->alfi_value_len = old_attrp->alfi_value_len;
-	if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
+	if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACEXXX) {
+		new_attrp->alfi_newname_len = old_attrp->alfi_newname_len;
+		new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len;
+	} else if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		new_attrp->alfi_newname_len = old_attrp->alfi_newname_len;
 		new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len;
 	} else {
@@ -815,6 +823,7 @@ xlog_recover_attri_commit_pass2(
 		}
 		name_len = attri_formatp->alfi_name_len;
 		break;
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		if (item->ri_total != 3 && item->ri_total != 4) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
#100 lore raw

PATCH 10/13 xfs: log old xattr values for NVREPLACEXXX operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

For NVREPLACEXXX operations, make it possible to log the old and new attr values, since this variant does VLOOKUP operations.

---
 fs/xfs/libxfs/xfs_attr.c       |    2 +
 fs/xfs/libxfs/xfs_da_btree.h   |    2 +
 fs/xfs/libxfs/xfs_log_format.h |   14 +++++---
 fs/xfs/xfs_attr_item.c         |   74 +++++++++++++++++++++++++++++++++++-----
 fs/xfs/xfs_attr_item.h         |    3 +-
 5 files changed, 80 insertions(+), 15 deletions(-)
fs/xfs/libxfs/xfs_attr.c +2 -0
index b9178c4efdeb..d807692b259c 100644
@@ -429,6 +429,8 @@ xfs_attr_complete_op(
 			args->namelen = args->new_namelen;
 			args->hashval = xfs_da_hashname(args->name,
 							args->namelen);
+			args->value = args->new_value;
+			args->valuelen = args->new_valuelen;
 		} else if (args->new_namelen > 0) {
 			args->name = args->new_name;
 			args->namelen = args->new_namelen;
fs/xfs/libxfs/xfs_da_btree.h +2 -0
index 0ef32f629e1b..cbea5233159c 100644
@@ -60,7 +60,9 @@ typedef struct xfs_da_args {
 	int		new_namelen;	/* new attr name len */
 	uint8_t		filetype;	/* filetype of inode for directories */
 	void		*value;		/* set of bytes (maybe contain NULLs) */
+	void		*new_value;	/* new xattr value (may contain NULLs) */
 	int		valuelen;	/* length of value */
+	int		new_valuelen;	/* length of new value */
 	unsigned int	attr_filter;	/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */
 	unsigned int	attr_flags;	/* XATTR_{CREATE,REPLACE} */
 	xfs_dahash_t	hashval;	/* hash value of name */
fs/xfs/libxfs/xfs_log_format.h +10 -4
index a1581dc6f131..ed406738847d 100644
@@ -115,11 +115,11 @@ struct xfs_unmount_log_format {
 #define XLOG_REG_TYPE_BUD_FORMAT	26
 #define XLOG_REG_TYPE_ATTRI_FORMAT	27
 #define XLOG_REG_TYPE_ATTRD_FORMAT	28
-#define XLOG_REG_TYPE_ATTR_NAME	29
+#define XLOG_REG_TYPE_ATTR_NAME		29
 #define XLOG_REG_TYPE_ATTR_VALUE	30
 #define XLOG_REG_TYPE_ATTR_NEWNAME	31
-#define XLOG_REG_TYPE_MAX		31
-
+#define XLOG_REG_TYPE_ATTR_NEWVALUE	32
+#define XLOG_REG_TYPE_MAX		32
 
 /*
  * Flags to log operation header
@@ -980,7 +980,13 @@ struct xfs_icreate_log {
 struct xfs_attri_log_format {
 	uint16_t	alfi_type;	/* attri log item type */
 	uint16_t	alfi_size;	/* size of this item */
-	uint32_t	__pad;		/* pad to 64 bit aligned */
+
+	/*
+	 * For NVREPLACE, this is the length of the new xattr value.
+	 * alfi_value_len contains the length of the old xattr value.
+	 */
+	uint32_t	alfi_newvalue_len;
+
 	uint64_t	alfi_id;	/* attri identifier */
 	uint64_t	alfi_ino;	/* the inode for this attr operation */
 	uint32_t	alfi_op_flags;	/* marks the op as a set or remove */
fs/xfs/xfs_attr_item.c +64 -10

203 lines not on this page · raw message

fs/xfs/xfs_attr_item.h +2 -1
index e374712ba06b..d15fe4b1ce28 100644
@@ -13,8 +13,9 @@ struct kmem_zone;
 
 struct xfs_attri_log_nameval {
 	struct xfs_log_iovec	name;
-	struct xfs_log_iovec	newname;
+	struct xfs_log_iovec	newname;	/* NVREPLACE only */
 	struct xfs_log_iovec	value;
+	struct xfs_log_iovec	newvalue;	/* NVREPLACE only */
 	refcount_t		refcount;
 
 	/* name and value follow the end of this struct */
#101 lore raw

PATCH 11/13 xfs: use VLOOKUP mode to avoid hashing parent pointer names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Hashing the parent pointer name is fugly because no hashing function can be collision proof. Since we store as much of the dirent name as we can in the xattr name and spill the rest to the xattr value, use VLOOKUP mode so that we can match on name and value. Then we can get rid of the hashing stuff.

---
 fs/xfs/libxfs/xfs_da_format.h  |   34 ++--
 fs/xfs/libxfs/xfs_parent.c     |  303 +++++++++++++---------------------------
 fs/xfs/libxfs/xfs_parent.h     |   16 --
 fs/xfs/libxfs/xfs_trans_resv.c |    1 
 fs/xfs/scrub/dir.c             |   38 +----
 fs/xfs/scrub/parent.c          |   61 +-------
 fs/xfs/scrub/parent_repair.c   |   34 +---
 fs/xfs/xfs_ondisk.h            |    3 
 8 files changed, 136 insertions(+), 354 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +12 -22
index 55f510f82e8d..dd569286b3be 100644
@@ -824,22 +824,12 @@ static inline unsigned int xfs_dir2_dirblock_bytes(struct xfs_sb *sbp)
 xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 				      struct xfs_da3_blkinfo *hdr3);
 
-/* We use sha512 for the parent pointer name hash. */
-#define XFS_PARENT_NAME_SHA512_SIZE	(64)
-
 /*
  * Parent pointer attribute format definition
  *
- * The EA name encodes the parent inode number, generation and a collision
- * resistant hash computed from the dirent name.  The hash is defined to be
- * one of the following:
- *
- * - The dirent name, as long as it does not use the last possible byte of the
- *   EA name space.
- *
- * - The truncated dirent name, with the sha512 hash of the child inode
- *   generation number and dirent name.  The hash is written at the end of the
- *   EA name.
+ * The EA name encodes the parent inode number, generation and as much of the
+ * dirent name as fits.  In other words, it contains up to 243 bytes of the
+ * dirent name.
  *
  * The EA value contains however much of the dirent name that does not fit in
  * the EA name.
@@ -847,30 +837,30 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 struct xfs_parent_name_rec {
 	__be64  p_ino;
 	__be32  p_gen;
-	__u8	p_namehash[];
+	__u8	p_dname[];
 } __attribute__((packed));
 
 /* Maximum size of a parent pointer EA name. */
 #define XFS_PARENT_NAME_MAX_SIZE \
 	(MAXNAMELEN - 1)
 
-/* Maximum size of a parent pointer name hash. */
-#define XFS_PARENT_NAME_MAX_HASH_SIZE \
+/* Maximum number of dirent name bytes stored in p_dname. */
+#define XFS_PARENT_MAX_DNAME_SIZE \
 	(XFS_PARENT_NAME_MAX_SIZE - sizeof(struct xfs_parent_name_rec))
 
-/* Offset of the sha512 hash, if used. */
-#define XFS_PARENT_NAME_SHA512_OFFSET \
-	(XFS_PARENT_NAME_MAX_HASH_SIZE - XFS_PARENT_NAME_SHA512_SIZE)
+/* Maximum number of dirent name bytes stored in the xattr value. */
+#define XFS_PARENT_MAX_DNAME_VALUELEN \
+	sizeof(struct xfs_parent_name_rec)
 
 static inline unsigned int
 xfs_parent_name_rec_sizeof(
-	unsigned int		hashlen)
+	unsigned int		dnamelen)
 {
-	return sizeof(struct xfs_parent_name_rec) + hashlen;
+	return sizeof(struct xfs_parent_name_rec) + dnamelen;
 }
 
 static inline unsigned int
-xfs_parent_name_hashlen(
+xfs_parent_name_dnamelen(
 	unsigned int		rec_sizeof)
 {
 	return rec_sizeof - sizeof(struct xfs_parent_name_rec);
fs/xfs/libxfs/xfs_parent.c +101 -202

469 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +3 -13
index 6f6136165efe..0b3e0b94d6cb 100644
@@ -23,10 +23,6 @@ struct xfs_parent_name_irec {
 	/* Key fields for looking up a particular parent pointer. */
 	xfs_ino_t		p_ino;
 	uint32_t		p_gen;
-	uint8_t			hashlen;
-	uint8_t			p_namehash[XFS_PARENT_NAME_MAX_HASH_SIZE];
-
-	/* Attributes of a parent pointer. */
 	uint8_t			p_namelen;
 	unsigned char		p_name[MAXNAMELEN];
 };
@@ -34,7 +30,7 @@ struct xfs_parent_name_irec {
 void xfs_parent_irec_from_disk(struct xfs_parent_name_irec *irec,
 		const struct xfs_parent_name_rec *rec, int reclen,
 		const void *value, int valuelen);
-void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, int *reclen,
+int xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, int *reclen,
 		void *value, int *valuelen,
 		const struct xfs_parent_name_irec *irec);
 
@@ -107,12 +103,6 @@ xfs_parent_finish(
 		__xfs_parent_cancel(mp, p);
 }
 
-int xfs_parent_namehash(struct xfs_inode *ip, const struct xfs_name *name,
-		void *namehash, unsigned int namehash_len);
-
-int xfs_parent_irec_hash(struct xfs_inode *ip,
-		struct xfs_parent_name_irec *pptr);
-
 unsigned int xfs_pptr_calc_space_res(struct xfs_mount *mp,
 				     unsigned int namelen);
 
@@ -126,8 +116,8 @@ struct xfs_parent_scratch {
 };
 
 int xfs_parent_lookup(struct xfs_trans *tp, struct xfs_inode *ip,
-		const struct xfs_parent_name_irec *pptr, unsigned char *name,
-		unsigned int namelen, struct xfs_parent_scratch *scratch);
+		const struct xfs_parent_name_irec *pptr,
+		struct xfs_parent_scratch *scratch);
 
 int xfs_parent_set(struct xfs_inode *ip,
 		const struct xfs_parent_name_irec *pptr,
fs/xfs/libxfs/xfs_trans_resv.c +1 -0
index 0e625c6b0153..a8afe2333194 100644
@@ -439,6 +439,7 @@ static inline unsigned int xfs_calc_pptr_replace_overhead(void)
 	return sizeof(struct xfs_attri_log_format) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE) +
 			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 
fs/xfs/scrub/dir.c +6 -32
index 87cff40b15f1..23cb7519c8f0 100644
@@ -78,7 +78,7 @@ struct xchk_dir {
 	/* If we've cycled the ILOCK, we must revalidate deferred dirents. */
 	bool			need_revalidate;
 
-	/* Name buffer for pptr validation and dirent revalidation. */
+	/* Name buffer for dirent revalidation. */
 	uint8_t			namebuf[MAXNAMELEN];
 
 };
@@ -143,42 +143,16 @@ xchk_dir_parent_pointer(
 	struct xfs_inode	*ip)
 {
 	struct xfs_scrub	*sc = sd->sc;
-	int			pptr_namelen;
-	int			hashlen;
+	int			error;
 
 	sd->pptr.p_ino = sc->ip->i_ino;
 	sd->pptr.p_gen = VFS_I(sc->ip)->i_generation;
+	sd->pptr.p_namelen = name->len;
+	memcpy(sd->pptr.p_name, name->name, name->len);
 
-	hashlen = xfs_parent_namehash(ip, name, &sd->pptr.p_namehash,
-			sizeof(sd->pptr.p_namehash));
-	if (hashlen < 0) {
-		xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0,
-				&hashlen);
-		return hashlen;
-	}
-	sd->pptr.hashlen = hashlen;
-
-	pptr_namelen = xfs_parent_lookup(sc->tp, ip, &sd->pptr, sd->namebuf,
-			MAXNAMELEN, &sd->pptr_scratch);
-	if (pptr_namelen == -ENOATTR) {
-		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
-		return 0;
-	}
-	if (pptr_namelen < 0) {
-		xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0,
-				&pptr_namelen);
-		return pptr_namelen;
-	}
-
-	if (pptr_namelen != name->len) {
-		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
-		return 0;
-	}
-
-	if (memcmp(sd->namebuf, name->name, name->len)) {
+	error = xfs_parent_lookup(sc->tp, ip, &sd->pptr, &sd->pptr_scratch);
+	if (error == -ENOATTR)
 		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
-		return 0;
-	}
 
 	return 0;
 }
fs/xfs/scrub/parent.c +7 -54

105 lines not on this page · raw message

fs/xfs/scrub/parent_repair.c +6 -28
index 14647e3da8c1..b55ef1506dd2 100644
@@ -126,9 +126,6 @@ struct xrep_pptrs {
 
 	/* Parent pointer names. */
 	struct xfblob		*pptr_names;
-
-	/* Buffer for validation. */
-	unsigned char		namebuf[MAXNAMELEN];
 };
 
 /* Tear down all the incore stuff we created. */
@@ -182,16 +179,11 @@ xrep_pptr_replay_update(
 	const struct xrep_pptr	*pptr)
 {
 	struct xfs_scrub	*sc = rp->sc;
-	int			error;
 
 	rp->pptr.p_ino = pptr->p_ino;
 	rp->pptr.p_gen = pptr->p_gen;
 	rp->pptr.p_namelen = pptr->namelen;
 
-	error = xfs_parent_irec_hash(sc->ip, &rp->pptr);
-	if (error)
-		return error;
-
 	if (pptr->action == XREP_PPTR_ADD) {
 		/* Create parent pointer. */
 		trace_xrep_pptr_createname(sc->tempip, &rp->pptr);
@@ -510,7 +502,7 @@ xrep_pptr_dump_tempptr(
 	struct xrep_pptrs	*rp = priv;
 	const struct xfs_parent_name_rec *rec = (const void *)name;
 	struct xfs_inode	*other_ip;
-	int			pptr_namelen;
+	int			error;
 
 	if (!(attr_flags & XFS_ATTR_PARENT))
 		return 0;
@@ -526,29 +518,15 @@ xrep_pptr_dump_tempptr(
 
 	trace_xrep_pptr_dumpname(sc->tempip, &rp->pptr);
 
-	pptr_namelen = xfs_parent_lookup(sc->tp, other_ip, &rp->pptr,
-			rp->namebuf, MAXNAMELEN, &rp->pptr_scratch);
-	if (pptr_namelen == -ENOATTR) {
+	error = xfs_parent_lookup(sc->tp, other_ip, &rp->pptr,
+			&rp->pptr_scratch);
+	if (error == -ENOATTR) {
 		trace_xrep_pptr_checkname(other_ip, &rp->pptr);
-		ASSERT(pptr_namelen != -ENOATTR);
+		ASSERT(error != -ENOATTR);
 		return -EFSCORRUPTED;
 	}
-	if (pptr_namelen < 0)
-		return pptr_namelen;
 
-	if (pptr_namelen != rp->pptr.p_namelen) {
-		trace_xrep_pptr_checkname(other_ip, &rp->pptr);
-		ASSERT(pptr_namelen == rp->pptr.p_namelen);
-		return -EFSCORRUPTED;
-	}
-
-	if (memcmp(rp->namebuf, rp->pptr.p_name, rp->pptr.p_namelen)) {
-		trace_xrep_pptr_checkname(other_ip, &rp->pptr);
-		ASSERT(0);
-		return -EFSCORRUPTED;
-	}
-
-	return 0;
+	return error;
 }
 
 /*
fs/xfs/xfs_ondisk.h +0 -3
index b7f29b4acac3..a78a3077b41a 100644
@@ -117,9 +117,6 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, name,		3);
 	XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_hdr_t,		10);
 	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec,	12);
-	BUILD_BUG_ON(XFS_PARENT_NAME_MAX_HASH_SIZE < SHA512_DIGEST_SIZE);
-	BUILD_BUG_ON(XFS_PARENT_NAME_MAX_HASH_SIZE !=           243);
-	BUILD_BUG_ON(XFS_PARENT_NAME_SHA512_OFFSET !=           179);
 
 	/* log structures */
 	XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format,	88);
#102 lore raw

PATCH 12/13 xfs: turn NVREPLACEXXX into NVREPLACE

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

NVREPLACEXXX is NVREPLACE with VLOOKUP enabled. Nobody uses NVREPLACE now, so get rid of NVREPLACE and make NVREPLACEXXX take its place.

---
 fs/xfs/libxfs/xfs_attr.c       |    7 -------
 fs/xfs/libxfs/xfs_log_format.h |    1 -
 fs/xfs/xfs_attr_item.c         |   15 +++------------
 3 files changed, 3 insertions(+), 20 deletions(-)
fs/xfs/libxfs/xfs_attr.c +0 -7
index d807692b259c..c6621aba161d 100644
@@ -431,11 +431,6 @@ xfs_attr_complete_op(
 							args->namelen);
 			args->value = args->new_value;
 			args->valuelen = args->new_valuelen;
-		} else if (args->new_namelen > 0) {
-			args->name = args->new_name;
-			args->namelen = args->new_namelen;
-			args->hashval = xfs_da_hashname(args->name,
-							args->namelen);
 		}
 		return replace_state;
 	}
@@ -944,8 +939,6 @@ xfs_attr_defer_replace(
 	int			error = 0;
 
 	if (args->op_flags & XFS_DA_OP_VLOOKUP)
-		op_flag = XFS_ATTRI_OP_FLAGS_NVREPLACEXXX;
-	else if (args->new_namelen > 0)
 		op_flag = XFS_ATTRI_OP_FLAGS_NVREPLACE;
 
 	error = xfs_attr_intent_init(args, op_flag, &new);
fs/xfs/libxfs/xfs_log_format.h +0 -1
index ed406738847d..ec85af39ed91 100644
@@ -961,7 +961,6 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_NVSET	6	/* Set attr with w/ vlookup */
-#define XFS_ATTRI_OP_FLAGS_NVREPLACEXXX	7	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +3 -12
index 57cc426b1e22..70d56bab4e21 100644
@@ -428,13 +428,10 @@ xfs_attr_log_item(
 	attrp->alfi_op_flags = attr->xattri_op_flags;
 	attrp->alfi_value_len = attr->xattri_nameval->value.i_len;
 
-	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACEXXX) {
+	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
 		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
 		attrp->alfi_newvalue_len = attr->xattri_nameval->newvalue.i_len;
-	} else if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
-		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
-		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
 	} else {
 		attrp->alfi_name_len = attr->xattri_nameval->name.i_len;
 	}
@@ -598,7 +595,6 @@ xfs_attri_validate(
 		if (attrp->alfi_newvalue_len != 0)
 			return false;
 		break;
-	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		if (attrp->alfi_oldname_len == 0 ||
 		    attrp->alfi_oldname_len > XATTR_NAME_MAX)
@@ -685,7 +681,7 @@ xfs_attri_item_recover(
 	ASSERT(xfs_sb_version_haslogxattrs(&mp->m_sb));
 
 	switch (attr->xattri_op_flags) {
-	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
+	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		args->new_value = nv->newvalue.i_addr;
 		args->new_valuelen = nv->newvalue.i_len;
 		fallthrough;
@@ -694,7 +690,6 @@ xfs_attri_item_recover(
 		fallthrough;
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
-	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		args->value = nv->value.i_addr;
 		args->valuelen = nv->value.i_len;
 		args->total = xfs_attr_calc_size(args, &local);
@@ -788,13 +783,10 @@ xfs_attri_item_relog(
 	new_attrp->alfi_ino = old_attrp->alfi_ino;
 	new_attrp->alfi_op_flags = old_attrp->alfi_op_flags;
 	new_attrp->alfi_value_len = old_attrp->alfi_value_len;
-	if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACEXXX) {
+	if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		new_attrp->alfi_newname_len = old_attrp->alfi_newname_len;
 		new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len;
 		new_attrp->alfi_newvalue_len = old_attrp->alfi_newvalue_len;
-	} else if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
-		new_attrp->alfi_newname_len = old_attrp->alfi_newname_len;
-		new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len;
 	} else {
 		new_attrp->alfi_name_len = old_attrp->alfi_name_len;
 	}
@@ -864,7 +856,6 @@ xlog_recover_attri_commit_pass2(
 		name_len = attri_formatp->alfi_name_len;
 		value_len = attri_formatp->alfi_value_len;
 		break;
-	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		if (item->ri_total < 3 || item->ri_total > 5) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
#103 lore raw

PATCH 13/13 xfs: revert "load secure hash algorithm for parent pointers"

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

We don't use this anymore, so get rid of it.

---
 fs/xfs/Kconfig      |    1 -
 fs/xfs/xfs_linux.h  |    1 -
 fs/xfs/xfs_mount.c  |   13 -------------
 fs/xfs/xfs_mount.h  |    3 ---
 fs/xfs/xfs_sha512.h |   42 ------------------------------------------
 fs/xfs/xfs_super.c  |    3 ---
 6 files changed, 63 deletions(-)
 delete mode 100644 fs/xfs/xfs_sha512.h
fs/xfs/Kconfig +0 -1
index 6422daaf8914..4798a147fd9e 100644
@@ -5,7 +5,6 @@ config XFS_FS
 	select EXPORTFS
 	select LIBCRC32C
 	select FS_IOMAP
-	select CRYPTO_SHA512
 	help
 	  XFS is a high performance journaling filesystem which originated
 	  on the SGI IRIX platform.  It is completely multi-threaded, can
fs/xfs/xfs_linux.h +0 -1
index 3f93a742b896..c05f7e309c3e 100644
@@ -62,7 +62,6 @@ typedef __u32			xfs_nlink_t;
 #include <linux/rhashtable.h>
 #include <linux/xattr.h>
 #include <linux/mnt_idmapping.h>
-#include <crypto/hash.h>
 
 #include <asm/page.h>
 #include <asm/div64.h>
fs/xfs/xfs_mount.c +0 -13
index a5f3dce658e9..fb87ffb48f7f 100644
@@ -983,19 +983,6 @@ xfs_mountfs(
 			goto out_agresv;
 	}
 
-	if (xfs_has_parent(mp)) {
-		struct crypto_shash	*tfm;
-
-		tfm = crypto_alloc_shash("sha512", 0, 0);
-		if (IS_ERR(tfm)) {
-			error = PTR_ERR(tfm);
-			goto out_agresv;
-		}
-		xfs_info(mp, "parent pointer hash %s",
-				crypto_shash_driver_name(tfm));
-		mp->m_sha512 = tfm;
-	}
-
 	return 0;
 
  out_agresv:
fs/xfs/xfs_mount.h +0 -3
index 7c8e15e84cd6..c08f55cc4f36 100644
@@ -244,9 +244,6 @@ typedef struct xfs_mount {
 #endif
 	/* Hook to feed file directory updates to an active online repair. */
 	struct xfs_hooks	m_dirent_update_hooks;
-
-	/* sha512 engine, if needed */
-	struct crypto_shash	*m_sha512;
 } xfs_mount_t;
 
 #define M_IGEO(mp)		(&(mp)->m_ino_geo)
fs/xfs/xfs_sha512.h deleted +0 -42

42 lines not on this page · raw message

fs/xfs/xfs_super.c +0 -3
index 610d72353f39..0432a4a096e8 100644
@@ -738,8 +738,6 @@ xfs_mount_free(
 {
 	kfree(mp->m_rtname);
 	kfree(mp->m_logname);
-	if (mp->m_sha512)
-		crypto_free_shash(mp->m_sha512);
 	kmem_free(mp);
 }
 
@@ -1963,7 +1961,6 @@ static int xfs_init_fs_context(
 	if (fc->sb_flags & SB_SYNCHRONOUS)
 		mp->m_features |= XFS_FEAT_WSYNC;
 
-	mp->m_sha512 = NULL;
 	fc->s_fs_info = mp;
 	fc->ops = &xfs_context_ops;
 
#104 lore raw

cover xfs: use flex arrays for XFS_IOC_GETPARENTS

In reply to Darrick J. Wong

Hi all,

Change the XFS_IOC_GETPARENTS structure definitions so that we can pack parent pointer information more densely, in a manner similar to the attrlistmulti ioctl. This also reduces the amount of memory that has to be copied back to userspace if the buffer doesn't fill up.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-ioctl-flexarray

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-ioctl-flexarray

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-ioctl-flexarray

---
 fs/xfs/libxfs/xfs_fs.h    |   75 ++++++++++++++++++++-------------------------
 fs/xfs/xfs_ioctl.c        |   67 +++++++++++++++++++++++++++++-----------
 fs/xfs/xfs_ondisk.h       |    4 +-
 fs/xfs/xfs_parent_utils.c |   57 +++++++++++++++++++++-------------
 fs/xfs/xfs_parent_utils.h |   11 ++++++-
 fs/xfs/xfs_trace.h        |   30 +++++++++---------
 6 files changed, 143 insertions(+), 101 deletions(-)