Skip to contentSkip to conversation

xfs: Parent Pointers

RFC New Darrick J. Wong 4 participants 94 messages Mar 16, 2023 – Mar 30, 2023 XFS filesystem

lore
Share & tools
lore
#0 lore raw

Hi everyone,

This deluge contains all of the additions to the parent pointers patchset that I've been working since last month's deluge. The kernel and xfsprogs patchsets are based on Allison's v10 tag from last week; the fstests patches are merely a part of my development tree. To recap Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point."

v10r1d2 rebases everything against 6.3-rc2. I still want to remove the diroffset from the ondisk parent pointer, but for v10 I've replaced the sha512 hashing code with modifications to the xattr code to support lookups based on name *and* value. With that working, we can encode parent pointers like this:

	(parent_ino, parent_gen, name[])

xattr lookups still work correctly, and repair doesn't have to deal with keeping the diroffsets in sync if the directory gets rebuilt. With this change applied, I'm ready to weave my new changes into Allison's v10 and call parent pointers done. :)

The online directory and parent pointer code are exactly the same as the v9r2d1 release, so I'm eliding that and everything that was in Allison's recent v10 patchset. IOWs, this deluge includes only the bug fixes I've made to parent pointers, the updates I've made to the ondisk format, and the necessary changes to fstests to get everything to pass.

If you want to pull the whole thing, use these links:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-drop-unnecessary
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key

Allison: Could you please resync libxfs in the following patches under
https://github.com/allisonhenderson/xfsprogs/commits/xfsprogs_new_pptrs_v10
please?

xfsprogs: add parent pointer support to attribute code
xfsprogs: extend transaction reservations for parent attributes
xfsprogs: parent pointer attribute creation
xfsprogs: remove parent pointers in unlink
xfsprogs: Add parent pointers to rename
xfsprogs: move/add parent pointer validators to xfs_parent

There are discrepancies between the two, which makes ./tools/libxfs-diff unhappy. Or, if you want me to merge my ondisk format changes into my branches, I'll put out v11 with everything taken care of.

--D

#1 lore raw

cover xfs: bug fixes for parent pointers

Hi all,

This series contains the accumulated bug fixes from Darrick to make fstests pass and online repair work.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-bugfixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-bugfixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-bugfixes

---
 fs/xfs/libxfs/xfs_da_format.h |   11 ++++++
 fs/xfs/libxfs/xfs_fs.h        |   69 +++++++++++++++++--------------------
 fs/xfs/libxfs/xfs_parent.c    |   47 ++++++++++++++++++++++---
 fs/xfs/libxfs/xfs_parent.h    |   24 ++++++-------
 fs/xfs/xfs_inode.c            |   16 ++++-----
 fs/xfs/xfs_ioctl.c            |   47 ++++++++++++-------------
 fs/xfs/xfs_ondisk.h           |    4 +-
 fs/xfs/xfs_parent_utils.c     |   61 +++++++++++++++++----------------
 fs/xfs/xfs_parent_utils.h     |    8 ++--
 fs/xfs/xfs_symlink.c          |    2 +
 fs/xfs/xfs_trace.c            |    1 +
 fs/xfs/xfs_trace.h            |   76 ++++++++++++++++++++++++++++++++++++++++-
 12 files changed, 238 insertions(+), 128 deletions(-)
#2 lore raw

PATCH 1/7 xfs: validate parent pointer xattrs in getparent

From: Darrick J. Wong <[email protected]>

Actually validate the parent pointer xattr before we try to export it to userspace.

---
 fs/xfs/xfs_parent_utils.c |   14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
fs/xfs/xfs_parent_utils.c +6 -8
index 9c1c866346eb..f3cf8b33605d 100644
@@ -49,6 +49,7 @@ xfs_getparent_listent(
 	struct xfs_getparent_ctx	*gp;
 	struct xfs_pptr_info		*ppi;
 	struct xfs_parent_ptr		*pptr;
+	struct xfs_parent_name_rec	*rec = (void *)name;
 	struct xfs_parent_name_irec	*irec;
 	struct xfs_mount		*mp = context->dp->i_mount;
 	int				arraytop;
@@ -62,19 +63,16 @@ xfs_getparent_listent(
 		return;
 
 	/*
-	 * Report corruption for xattrs with any other flag set, or for a
-	 * parent pointer that has a remote value.  The attr list functions
-	 * filtered any INCOMPLETE attrs for us.
+	 * Report corruption for anything that doesn't look like a parent
+	 * pointer.  The attr list functions filtered out INCOMPLETE attrs.
 	 */
-	if (XFS_IS_CORRUPT(mp,
-			   hweight32(flags & XFS_ATTR_NSP_ONDISK_MASK) > 1) ||
-	    XFS_IS_CORRUPT(mp, value == NULL)) {
+	if (XFS_IS_CORRUPT(mp, !xfs_parent_namecheck(mp, rec, namelen, flags)) ||
+	    XFS_IS_CORRUPT(mp, !xfs_parent_valuecheck(mp, value, valuelen))) {
 		context->seen_enough = -EFSCORRUPTED;
 		return;
 	}
 
-	xfs_parent_irec_from_disk(&gp->pptr_irec, (void *)name, value,
-			valuelen);
+	xfs_parent_irec_from_disk(&gp->pptr_irec, rec, value, valuelen);
 
 	/*
 	 * We found a parent pointer, but we've filled up the buffer.  Signal
#3 lore raw

PATCH 2/7 xfs: rename xfs_pptr_info to xfs_getparents

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rename the head structure of the parent pointer ioctl to match the name of the ioctl (XFS_IOC_GETPARENTS).

---
 fs/xfs/libxfs/xfs_fs.h    |   45 +++++++++++++++++++++++---------------------
 fs/xfs/xfs_ioctl.c        |   46 +++++++++++++++++++++++----------------------
 fs/xfs/xfs_ondisk.h       |    2 +-
 fs/xfs/xfs_parent_utils.c |   26 +++++++++++++------------
 fs/xfs/xfs_parent_utils.h |    8 ++++----
 5 files changed, 64 insertions(+), 63 deletions(-)
fs/xfs/libxfs/xfs_fs.h +23 -22
index 0db0c8fc5359..c34303a39157 100644
@@ -752,19 +752,20 @@ struct xfs_scrub_metadata {
 				 XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED)
 #define XFS_SCRUB_FLAGS_ALL	(XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT)
 
-#define XFS_PPTR_MAXNAMELEN				256
+#define XFS_GETPARENTS_MAXNAMELEN	256
 
 /* return parents of the handle, not the open fd */
-#define XFS_PPTR_IFLAG_HANDLE  (1U << 0)
+#define XFS_GETPARENTS_IFLAG_HANDLE	(1U << 0)
 
 /* target was the root directory */
-#define XFS_PPTR_OFLAG_ROOT    (1U << 1)
+#define XFS_GETPARENTS_OFLAG_ROOT	(1U << 1)
 
 /* Cursor is done iterating pptrs */
-#define XFS_PPTR_OFLAG_DONE    (1U << 2)
+#define XFS_GETPARENTS_OFLAG_DONE	(1U << 2)
 
- #define XFS_PPTR_FLAG_ALL     (XFS_PPTR_IFLAG_HANDLE | XFS_PPTR_OFLAG_ROOT | \
-				XFS_PPTR_OFLAG_DONE)
+#define XFS_GETPARENTS_FLAG_ALL		(XFS_GETPARENTS_IFLAG_HANDLE | \
+					 XFS_GETPARENTS_OFLAG_ROOT | \
+					 XFS_GETPARENTS_OFLAG_DONE)
 
 /* Get an inode parent pointer through ioctl */
 struct xfs_parent_ptr {
@@ -776,49 +777,49 @@ struct xfs_parent_ptr {
 };
 
 /* Iterate through an inodes parent pointers */
-struct xfs_pptr_info {
-	/* File handle, if XFS_PPTR_IFLAG_HANDLE is set */
-	struct xfs_handle		pi_handle;
+struct xfs_getparents {
+	/* File handle, if XFS_GETPARENTS_IFLAG_HANDLE is set */
+	struct xfs_handle		gp_handle;
 
 	/*
 	 * Structure to track progress in iterating the parent pointers.
 	 * Must be initialized to zeroes before the first ioctl call, and
 	 * not touched by callers after that.
 	 */
-	struct xfs_attrlist_cursor	pi_cursor;
+	struct xfs_attrlist_cursor	gp_cursor;
 
-	/* Operational flags: XFS_PPTR_*FLAG* */
-	__u32				pi_flags;
+	/* Operational flags: XFS_GETPARENTS_*FLAG* */
+	__u32				gp_flags;
 
 	/* Must be set to zero */
-	__u32				pi_reserved;
+	__u32				gp_reserved;
 
 	/* size of the trailing buffer in bytes */
-	__u32				pi_ptrs_size;
+	__u32				gp_ptrs_size;
 
 	/* # of entries filled in (output) */
-	__u32				pi_count;
+	__u32				gp_count;
 
 	/* Must be set to zero */
-	__u64				pi_reserved2[5];
+	__u64				gp_reserved2[5];
 
 	/* Byte offset of each record within the buffer */
-	__u32				pi_offsets[];
+	__u32				gp_offsets[];
 };
 
 static inline size_t
-xfs_pptr_info_sizeof(int nr_ptrs)
+xfs_getparents_sizeof(int nr_ptrs)
 {
-	return sizeof(struct xfs_pptr_info) +
+	return sizeof(struct xfs_getparents) +
 	       (nr_ptrs * sizeof(struct xfs_parent_ptr));
 }
 
 static inline struct xfs_parent_ptr*
-xfs_ppinfo_to_pp(
-	struct xfs_pptr_info	*info,
+xfs_getparents_rec(
+	struct xfs_getparents	*info,
 	int			idx)
 {
-	return (struct xfs_parent_ptr *)((char *)info + info->pi_offsets[idx]);
+	return (struct xfs_parent_ptr *)((char *)info + info->gp_offsets[idx]);
 }
 
 /*
fs/xfs/xfs_ioctl.c +23 -23

114 lines not on this page · raw message

fs/xfs/xfs_ondisk.h +1 -1
index 829bee58fc63..ba68c3270e07 100644
@@ -152,7 +152,7 @@ xfs_check_ondisk_structs(void)
 
 	/* parent pointer ioctls */
 	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr,            24);
-	XFS_CHECK_STRUCT_SIZE(struct xfs_pptr_info,             96);
+	XFS_CHECK_STRUCT_SIZE(struct xfs_getparents,		96);
 
 	/*
 	 * The v5 superblock format extended several v4 header structures with
fs/xfs/xfs_parent_utils.c +13 -13
index f3cf8b33605d..d74cb2081cd2 100644
@@ -26,7 +26,7 @@
 struct xfs_getparent_ctx {
 	struct xfs_attr_list_context	context;
 	struct xfs_parent_name_irec	pptr_irec;
-	struct xfs_pptr_info		*ppi;
+	struct xfs_getparents		*ppi;
 };
 
 static inline unsigned int
@@ -47,7 +47,7 @@ xfs_getparent_listent(
 	int				valuelen)
 {
 	struct xfs_getparent_ctx	*gp;
-	struct xfs_pptr_info		*ppi;
+	struct xfs_getparents		*ppi;
 	struct xfs_parent_ptr		*pptr;
 	struct xfs_parent_name_rec	*rec = (void *)name;
 	struct xfs_parent_name_irec	*irec;
@@ -79,7 +79,7 @@ xfs_getparent_listent(
 	 * to the caller that we did /not/ reach the end of the parent pointer
 	 * recordset.
 	 */
-	arraytop = xfs_getparents_arraytop(ppi, ppi->pi_count + 1);
+	arraytop = xfs_getparents_arraytop(ppi, ppi->gp_count + 1);
 	context->firstu -= xfs_getparents_rec_sizeof(irec);
 	if (context->firstu < arraytop) {
 		context->seen_enough = 1;
@@ -87,8 +87,8 @@ xfs_getparent_listent(
 	}
 
 	/* Format the parent pointer directly into the caller buffer. */
-	ppi->pi_offsets[ppi->pi_count] = context->firstu;
-	pptr = xfs_ppinfo_to_pp(ppi, ppi->pi_count);
+	ppi->gp_offsets[ppi->gp_count] = context->firstu;
+	pptr = xfs_getparents_rec(ppi, ppi->gp_count);
 	pptr->xpp_ino = irec->p_ino;
 	pptr->xpp_gen = irec->p_gen;
 	pptr->xpp_diroffset = irec->p_diroffset;
@@ -96,14 +96,14 @@ xfs_getparent_listent(
 
 	memcpy(pptr->xpp_name, irec->p_name, irec->p_namelen);
 	pptr->xpp_name[irec->p_namelen] = 0;
-	ppi->pi_count++;
+	ppi->gp_count++;
 }
 
 /* Retrieve the parent pointers for a given inode. */
 int
 xfs_getparent_pointers(
 	struct xfs_inode		*ip,
-	struct xfs_pptr_info		*ppi)
+	struct xfs_getparents		*ppi)
 {
 	struct xfs_getparent_ctx	*gp;
 	int				error;
@@ -115,13 +115,13 @@ xfs_getparent_pointers(
 	gp->context.dp = ip;
 	gp->context.resynch = 1;
 	gp->context.put_listent = xfs_getparent_listent;
-	gp->context.bufsize = round_down(ppi->pi_ptrs_size, sizeof(uint32_t));
+	gp->context.bufsize = round_down(ppi->gp_ptrs_size, sizeof(uint32_t));
 	gp->context.firstu = gp->context.bufsize;
 
 	/* Copy the cursor provided by caller */
-	memcpy(&gp->context.cursor, &ppi->pi_cursor,
+	memcpy(&gp->context.cursor, &ppi->gp_cursor,
 			sizeof(struct xfs_attrlist_cursor));
-	ppi->pi_count = 0;
+	ppi->gp_count = 0;
 
 	error = xfs_attr_list(&gp->context);
 	if (error)
@@ -133,17 +133,17 @@ xfs_getparent_pointers(
 
 	/* Is this the root directory? */
 	if (ip->i_ino == ip->i_mount->m_sb.sb_rootino)
-		ppi->pi_flags |= XFS_PPTR_OFLAG_ROOT;
+		ppi->gp_flags |= XFS_GETPARENTS_OFLAG_ROOT;
 
 	/*
 	 * If we did not run out of buffer space, then we reached the end of
 	 * the pptr recordset, so set the DONE flag.
 	 */
 	if (gp->context.seen_enough == 0)
-		ppi->pi_flags |= XFS_PPTR_OFLAG_DONE;
+		ppi->gp_flags |= XFS_GETPARENTS_OFLAG_DONE;
 
 	/* Update the caller with the current cursor position */
-	memcpy(&ppi->pi_cursor, &gp->context.cursor,
+	memcpy(&ppi->gp_cursor, &gp->context.cursor,
 			sizeof(struct xfs_attrlist_cursor));
 out_free:
 	kfree(gp);
fs/xfs/xfs_parent_utils.h +4 -4
index d79197f23c40..48de5b700f9c 100644
@@ -8,13 +8,13 @@
 
 static inline unsigned int
 xfs_getparents_arraytop(
-	const struct xfs_pptr_info	*ppi,
+	const struct xfs_getparents	*ppi,
 	unsigned int			nr)
 {
-	return sizeof(struct xfs_pptr_info) +
-			(nr * sizeof(ppi->pi_offsets[0]));
+	return sizeof(struct xfs_getparents) +
+			(nr * sizeof(ppi->gp_offsets[0]));
 }
 
-int xfs_getparent_pointers(struct xfs_inode *ip, struct xfs_pptr_info *ppi);
+int xfs_getparent_pointers(struct xfs_inode *ip, struct xfs_getparents *ppi);
 
 #endif	/* __XFS_PARENT_UTILS_H__ */
#4 lore raw

PATCH 3/7 xfs: rename xfs_parent_ptr

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Change the name to xfs_getparents_rec so that the name matches the head structure.

---
 fs/xfs/libxfs/xfs_fs.h    |   20 ++++++++++----------
 fs/xfs/xfs_ioctl.c        |    6 +++---
 fs/xfs/xfs_ondisk.h       |    2 +-
 fs/xfs/xfs_parent_utils.c |   16 ++++++++--------
 4 files changed, 22 insertions(+), 22 deletions(-)
fs/xfs/libxfs/xfs_fs.h +10 -10
index c34303a39157..c8edc7c099e8 100644
@@ -768,12 +768,12 @@ struct xfs_scrub_metadata {
 					 XFS_GETPARENTS_OFLAG_DONE)
 
 /* Get an inode parent pointer through ioctl */
-struct xfs_parent_ptr {
-	__u64		xpp_ino;			/* Inode */
-	__u32		xpp_gen;			/* Inode generation */
-	__u32		xpp_diroffset;			/* Directory offset */
-	__u64		xpp_rsvd;			/* Reserved */
-	__u8		xpp_name[];			/* File name */
+struct xfs_getparents_rec {
+	__u64		gpr_ino;			/* Inode */
+	__u32		gpr_gen;			/* Inode generation */
+	__u32		gpr_diroffset;			/* Directory offset */
+	__u64		gpr_rsvd;			/* Reserved */
+	__u8		gpr_name[];			/* File name */
 };
 
 /* Iterate through an inodes parent pointers */
@@ -811,15 +811,15 @@ static inline size_t
 xfs_getparents_sizeof(int nr_ptrs)
 {
 	return sizeof(struct xfs_getparents) +
-	       (nr_ptrs * sizeof(struct xfs_parent_ptr));
+	       (nr_ptrs * sizeof(struct xfs_getparents_rec));
 }
 
-static inline struct xfs_parent_ptr*
+static inline struct xfs_getparents_rec*
 xfs_getparents_rec(
 	struct xfs_getparents	*info,
 	int			idx)
 {
-	return (struct xfs_parent_ptr *)((char *)info + info->gp_offsets[idx]);
+	return (struct xfs_getparents_rec *)((char *)info + info->gp_offsets[idx]);
 }
 
 /*
@@ -867,7 +867,7 @@ xfs_getparents_rec(
 /*	XFS_IOC_GETFSMAP ------ hoisted 59         */
 #define XFS_IOC_SCRUB_METADATA	_IOWR('X', 60, struct xfs_scrub_metadata)
 #define XFS_IOC_AG_GEOMETRY	_IOWR('X', 61, struct xfs_ag_geometry)
-#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_parent_ptr)
+#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_getparents_rec)
 
 /*
  * ioctl commands that replace IRIX syssgi()'s
fs/xfs/xfs_ioctl.c +3 -3
index bc3fe5704eaa..04123ab41684 100644
@@ -1681,10 +1681,10 @@ xfs_ioc_scrub_metadata(
  * IOCTL routine to get the parent pointers of an inode and return it to user
  * space.  Caller must pass a buffer space containing a struct xfs_getparents,
  * followed by a region large enough to contain an array of struct
- * xfs_parent_ptr of a size specified in gp_ptrs_size.  If the inode contains
+ * xfs_getparents_rec of a size specified in gp_ptrs_size.  If the inode contains
  * more parent pointers than can fit in the buffer space, caller may re-call
  * the function using the returned gp_cursor to resume iteration.  The
- * number of xfs_parent_ptr returned will be stored in gp_ptrs_count.
+ * number of xfs_getparents_rec returned will be stored in gp_ptrs_count.
  *
  * Returns 0 on success or non-zero on failure
  */
@@ -1699,7 +1699,7 @@ xfs_ioc_get_parent_pointer(
 	struct xfs_inode		*call_ip = file_ip;
 	struct xfs_mount		*mp = file_ip->i_mount;
 	void				__user *o_pptr;
-	struct xfs_parent_ptr		*i_pptr;
+	struct xfs_getparents_rec		*i_pptr;
 	unsigned int			bytes;
 
 	if (!capable(CAP_SYS_ADMIN))
fs/xfs/xfs_ondisk.h +1 -1
index ba68c3270e07..88f9ec393c3d 100644
@@ -151,7 +151,7 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(struct xfs_efi_log_format_64, efi_extents,	16);
 
 	/* parent pointer ioctls */
-	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_ptr,            24);
+	XFS_CHECK_STRUCT_SIZE(struct xfs_getparents_rec,	24);
 	XFS_CHECK_STRUCT_SIZE(struct xfs_getparents,		96);
 
 	/*
fs/xfs/xfs_parent_utils.c +8 -8
index d74cb2081cd2..8aff31ed9082 100644
@@ -33,7 +33,7 @@ static inline unsigned int
 xfs_getparents_rec_sizeof(
 	const struct xfs_parent_name_irec	*irec)
 {
-	return round_up(sizeof(struct xfs_parent_ptr) + irec->p_namelen + 1,
+	return round_up(sizeof(struct xfs_getparents_rec) + irec->p_namelen + 1,
 			sizeof(uint32_t));
 }
 
@@ -48,7 +48,7 @@ xfs_getparent_listent(
 {
 	struct xfs_getparent_ctx	*gp;
 	struct xfs_getparents		*ppi;
-	struct xfs_parent_ptr		*pptr;
+	struct xfs_getparents_rec	*pptr;
 	struct xfs_parent_name_rec	*rec = (void *)name;
 	struct xfs_parent_name_irec	*irec;
 	struct xfs_mount		*mp = context->dp->i_mount;
@@ -89,13 +89,13 @@ xfs_getparent_listent(
 	/* Format the parent pointer directly into the caller buffer. */
 	ppi->gp_offsets[ppi->gp_count] = context->firstu;
 	pptr = xfs_getparents_rec(ppi, ppi->gp_count);
-	pptr->xpp_ino = irec->p_ino;
-	pptr->xpp_gen = irec->p_gen;
-	pptr->xpp_diroffset = irec->p_diroffset;
-	pptr->xpp_rsvd = 0;
+	pptr->gpr_ino = irec->p_ino;
+	pptr->gpr_gen = irec->p_gen;
+	pptr->gpr_diroffset = irec->p_diroffset;
+	pptr->gpr_rsvd = 0;
 
-	memcpy(pptr->xpp_name, irec->p_name, irec->p_namelen);
-	pptr->xpp_name[irec->p_namelen] = 0;
+	memcpy(pptr->gpr_name, irec->p_name, irec->p_namelen);
+	pptr->gpr_name[irec->p_namelen] = 0;
 	ppi->gp_count++;
 }
 
#5 lore raw

PATCH 4/7 xfs: fix GETPARENTS ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix a few remaining issues with this ioctl:

The ioctl encodes the size of the parent rec, not the parent head.

The parent rec should say that it returns a null terminated filename.

The parent head encodes the buffer size, not the size of the parent record array, but the field name and documentation doesn't make this clear.

The getparents sizeof function is pointless and wrong.

Get rid of the last vestiges of the non-flex-array definitions.

The rec address should take an unsigned argument

Whitespace damage

---
 fs/xfs/libxfs/xfs_fs.h    |   30 +++++++++++-------------------
 fs/xfs/xfs_ioctl.c        |   15 +++++++--------
 fs/xfs/xfs_parent_utils.c |    3 +--
 3 files changed, 19 insertions(+), 29 deletions(-)
fs/xfs/libxfs/xfs_fs.h +11 -19
index c8edc7c099e8..d7e061089e75 100644
@@ -752,8 +752,6 @@ struct xfs_scrub_metadata {
 				 XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED)
 #define XFS_SCRUB_FLAGS_ALL	(XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT)
 
-#define XFS_GETPARENTS_MAXNAMELEN	256
-
 /* return parents of the handle, not the open fd */
 #define XFS_GETPARENTS_IFLAG_HANDLE	(1U << 0)
 
@@ -769,11 +767,11 @@ struct xfs_scrub_metadata {
 
 /* Get an inode parent pointer through ioctl */
 struct xfs_getparents_rec {
-	__u64		gpr_ino;			/* Inode */
-	__u32		gpr_gen;			/* Inode generation */
-	__u32		gpr_diroffset;			/* Directory offset */
-	__u64		gpr_rsvd;			/* Reserved */
-	__u8		gpr_name[];			/* File name */
+	__u64		gpr_ino;	/* Inode number */
+	__u32		gpr_gen;	/* Inode generation */
+	__u32		gpr_diroffset;	/* Directory offset */
+	__u64		gpr_rsvd;	/* Reserved */
+	__u8		gpr_name[];	/* File name and null terminator */
 };
 
 /* Iterate through an inodes parent pointers */
@@ -794,8 +792,8 @@ struct xfs_getparents {
 	/* Must be set to zero */
 	__u32				gp_reserved;
 
-	/* size of the trailing buffer in bytes */
-	__u32				gp_ptrs_size;
+	/* Size of the buffer in bytes, including this header */
+	__u32				gp_bufsize;
 
 	/* # of entries filled in (output) */
 	__u32				gp_count;
@@ -807,19 +805,13 @@ struct xfs_getparents {
 	__u32				gp_offsets[];
 };
 
-static inline size_t
-xfs_getparents_sizeof(int nr_ptrs)
-{
-	return sizeof(struct xfs_getparents) +
-	       (nr_ptrs * sizeof(struct xfs_getparents_rec));
-}
-
 static inline struct xfs_getparents_rec*
 xfs_getparents_rec(
 	struct xfs_getparents	*info,
-	int			idx)
+	unsigned int		idx)
 {
-	return (struct xfs_getparents_rec *)((char *)info + info->gp_offsets[idx]);
+	return (struct xfs_getparents_rec *)((char *)info +
+					     info->gp_offsets[idx]);
 }
 
 /*
@@ -867,7 +859,7 @@ xfs_getparents_rec(
 /*	XFS_IOC_GETFSMAP ------ hoisted 59         */
 #define XFS_IOC_SCRUB_METADATA	_IOWR('X', 60, struct xfs_scrub_metadata)
 #define XFS_IOC_AG_GEOMETRY	_IOWR('X', 61, struct xfs_ag_geometry)
-#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_getparents_rec)
+#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_getparents)
 
 /*
  * ioctl commands that replace IRIX syssgi()'s
fs/xfs/xfs_ioctl.c +7 -8
index 04123ab41684..f265e28d0611 100644
@@ -1681,10 +1681,10 @@ xfs_ioc_scrub_metadata(
  * IOCTL routine to get the parent pointers of an inode and return it to user
  * space.  Caller must pass a buffer space containing a struct xfs_getparents,
  * followed by a region large enough to contain an array of struct
- * xfs_getparents_rec of a size specified in gp_ptrs_size.  If the inode contains
+ * xfs_getparents_rec of a size specified in gp_bufsize.  If the inode contains
  * more parent pointers than can fit in the buffer space, caller may re-call
  * the function using the returned gp_cursor to resume iteration.  The
- * number of xfs_getparents_rec returned will be stored in gp_ptrs_count.
+ * number of xfs_getparents_rec returned will be stored in gp_count.
  *
  * Returns 0 on success or non-zero on failure
  */
@@ -1699,7 +1699,7 @@ xfs_ioc_get_parent_pointer(
 	struct xfs_inode		*call_ip = file_ip;
 	struct xfs_mount		*mp = file_ip->i_mount;
 	void				__user *o_pptr;
-	struct xfs_getparents_rec		*i_pptr;
+	struct xfs_getparents_rec	*i_pptr;
 	unsigned int			bytes;
 
 	if (!capable(CAP_SYS_ADMIN))
@@ -1718,11 +1718,11 @@ xfs_ioc_get_parent_pointer(
 	}
 
 	/* Check size of buffer requested by user */
-	if (ppi->gp_ptrs_size > XFS_XATTR_LIST_MAX) {
+	if (ppi->gp_bufsize > XFS_XATTR_LIST_MAX) {
 		error = -ENOMEM;
 		goto out;
 	}
-	if (ppi->gp_ptrs_size < sizeof(struct xfs_getparents)) {
+	if (ppi->gp_bufsize < sizeof(struct xfs_getparents)) {
 		error = -EINVAL;
 		goto out;
 	}
@@ -1737,8 +1737,7 @@ xfs_ioc_get_parent_pointer(
 	 * Now that we know how big the trailing buffer is, expand
 	 * our kernel xfs_getparents to be the same size
 	 */
-	ppi = kvrealloc(ppi, sizeof(struct xfs_getparents),
-			xfs_getparents_sizeof(ppi->gp_ptrs_size),
+	ppi = kvrealloc(ppi, sizeof(struct xfs_getparents), ppi->gp_bufsize,
 			GFP_KERNEL | __GFP_ZERO);
 	if (!ppi)
 		return -ENOMEM;
@@ -1794,7 +1793,7 @@ xfs_ioc_get_parent_pointer(
 	/* Copy the parent pointer records back to the user. */
 	o_pptr = (__user char*)arg + ppi->gp_offsets[ppi->gp_count - 1];
 	i_pptr = xfs_getparents_rec(ppi, ppi->gp_count - 1);
-	bytes = ((char *)ppi + ppi->gp_ptrs_size) - (char *)i_pptr;
+	bytes = ((char *)ppi + ppi->gp_bufsize) - (char *)i_pptr;
 	error = copy_to_user(o_pptr, i_pptr, bytes);
 	if (error) {
 		error = -EFAULT;
fs/xfs/xfs_parent_utils.c +1 -2
index 8aff31ed9082..059454c43934 100644
@@ -115,7 +115,7 @@ xfs_getparent_pointers(
 	gp->context.dp = ip;
 	gp->context.resynch = 1;
 	gp->context.put_listent = xfs_getparent_listent;
-	gp->context.bufsize = round_down(ppi->gp_ptrs_size, sizeof(uint32_t));
+	gp->context.bufsize = round_down(ppi->gp_bufsize, sizeof(uint32_t));
 	gp->context.firstu = gp->context.bufsize;
 
 	/* Copy the cursor provided by caller */
@@ -149,4 +149,3 @@ xfs_getparent_pointers(
 	kfree(gp);
 	return error;
 }
-
#6 lore raw

PATCH 5/7 xfs: add tracing to the GETPARENTS ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some tracing to the getparents ioctl for easier debugging.

---
 fs/xfs/xfs_parent_utils.c |    4 ++
 fs/xfs/xfs_trace.c        |    1 +
 fs/xfs/xfs_trace.h        |   76 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 80 insertions(+), 1 deletion(-)
fs/xfs/xfs_parent_utils.c +4 -0
index 059454c43934..5a4f72cd5711 100644
@@ -86,6 +86,8 @@ xfs_getparent_listent(
 		return;
 	}
 
+	trace_xfs_getparent_listent(context->dp, ppi, irec);
+
 	/* Format the parent pointer directly into the caller buffer. */
 	ppi->gp_offsets[ppi->gp_count] = context->firstu;
 	pptr = xfs_getparents_rec(ppi, ppi->gp_count);
@@ -123,6 +125,8 @@ xfs_getparent_pointers(
 			sizeof(struct xfs_attrlist_cursor));
 	ppi->gp_count = 0;
 
+	trace_xfs_getparent_pointers(ip, ppi, &gp->context.cursor);
+
 	error = xfs_attr_list(&gp->context);
 	if (error)
 		goto out_free;
fs/xfs/xfs_trace.c +1 -0
index 8a5dc1538aa8..c1f339481697 100644
@@ -36,6 +36,7 @@
 #include "xfs_error.h"
 #include <linux/iomap.h>
 #include "xfs_iomap.h"
+#include "xfs_parent.h"
 
 /*
  * We include this last to have the helpers above available for the trace
fs/xfs/xfs_trace.h +75 -1

92 lines not on this page · raw message

#7 lore raw

PATCH 6/7 xfs: shorten parent pointer function names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Shorten the function names and add brief comments to each, outlining what they're supposed to be doing.

---
 fs/xfs/libxfs/xfs_parent.c |   18 ++++++++++++------
 fs/xfs/libxfs/xfs_parent.h |   24 ++++++++++++------------
 fs/xfs/xfs_inode.c         |   16 ++++++++--------
 fs/xfs/xfs_symlink.c       |    2 +-
 4 files changed, 33 insertions(+), 27 deletions(-)
fs/xfs/libxfs/xfs_parent.c +12 -6
index 179b9bebaf25..ec2bff195773 100644
@@ -135,6 +135,10 @@ xfs_parent_irec_from_disk(
 	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
 }
 
+/*
+ * Allocate memory to control a logged parent pointer update as part of a
+ * dirent operation.
+ */
 int
 __xfs_parent_init(
 	struct xfs_mount		*mp,
@@ -170,12 +174,13 @@ __xfs_parent_init(
 	return 0;
 }
 
+/* Add a parent pointer to reflect a dirent addition. */
 int
-xfs_parent_defer_add(
+xfs_parent_add(
 	struct xfs_trans	*tp,
 	struct xfs_parent_defer	*parent,
 	struct xfs_inode	*dp,
-	struct xfs_name		*parent_name,
+	const struct xfs_name	*parent_name,
 	xfs_dir2_dataptr_t	diroffset,
 	struct xfs_inode	*child)
 {
@@ -194,8 +199,9 @@ xfs_parent_defer_add(
 	return xfs_attr_defer_add(args);
 }
 
+/* Remove a parent pointer to reflect a dirent removal. */
 int
-xfs_parent_defer_remove(
+xfs_parent_remove(
 	struct xfs_trans	*tp,
 	struct xfs_inode	*dp,
 	struct xfs_parent_defer	*parent,
@@ -211,14 +217,14 @@ xfs_parent_defer_remove(
 	return xfs_attr_defer_remove(args);
 }
 
-
+/* Replace one parent pointer with another to reflect a rename. */
 int
-xfs_parent_defer_replace(
+xfs_parent_replace(
 	struct xfs_trans	*tp,
 	struct xfs_parent_defer	*new_parent,
 	struct xfs_inode	*old_dp,
 	xfs_dir2_dataptr_t	old_diroffset,
-	struct xfs_name		*parent_name,
+	const struct xfs_name	*parent_name,
 	struct xfs_inode	*new_dp,
 	xfs_dir2_dataptr_t	new_diroffset,
 	struct xfs_inode	*child)
fs/xfs/libxfs/xfs_parent.h +12 -12
index f4f5887d1133..35854e968f1d 100644
@@ -49,8 +49,9 @@ struct xfs_parent_defer {
  * Parent pointer attribute prototypes
  */
 void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
-			      struct xfs_inode *ip,
-			      uint32_t p_diroffset);
+		struct xfs_inode *ip, uint32_t p_diroffset);
+void xfs_init_parent_name_irec(struct xfs_parent_name_irec *irec,
+			       struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
@@ -78,18 +79,17 @@ xfs_parent_start_locked(
 	return 0;
 }
 
-int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
-			 struct xfs_inode *dp, struct xfs_name *parent_name,
-			 xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
-int xfs_parent_defer_replace(struct xfs_trans *tp,
+int xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
+		struct xfs_inode *dp, const struct xfs_name *parent_name,
+		xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
+int xfs_parent_replace(struct xfs_trans *tp,
 		struct xfs_parent_defer *new_parent, struct xfs_inode *old_dp,
-		xfs_dir2_dataptr_t old_diroffset, struct xfs_name *parent_name,
-		struct xfs_inode *new_ip, xfs_dir2_dataptr_t new_diroffset,
+		xfs_dir2_dataptr_t old_diroffset,
+		const struct xfs_name *parent_name, struct xfs_inode *new_ip,
+		xfs_dir2_dataptr_t new_diroffset, struct xfs_inode *child);
+int xfs_parent_remove(struct xfs_trans *tp, struct xfs_inode *dp,
+		struct xfs_parent_defer *parent, xfs_dir2_dataptr_t diroffset,
 		struct xfs_inode *child);
-int xfs_parent_defer_remove(struct xfs_trans *tp, struct xfs_inode *dp,
-			    struct xfs_parent_defer *parent,
-			    xfs_dir2_dataptr_t diroffset,
-			    struct xfs_inode *child);
 
 void __xfs_parent_cancel(struct xfs_mount *mp, struct xfs_parent_defer *parent);
 
fs/xfs/xfs_inode.c +8 -8
index 20193ebd3a99..d73f9283f399 100644
@@ -1090,7 +1090,7 @@ xfs_create(
 	 * the parent information now.
 	 */
 	if (parent) {
-		error = xfs_parent_defer_add(tp, parent, dp, name, diroffset,
+		error = xfs_parent_add(tp, parent, dp, name, diroffset,
 					     ip);
 		if (error)
 			goto out_trans_cancel;
@@ -1355,7 +1355,7 @@ xfs_link(
 	 * the parent to the inode.
 	 */
 	if (parent) {
-		error = xfs_parent_defer_add(tp, parent, tdp, target_name,
+		error = xfs_parent_add(tp, parent, tdp, target_name,
 					     diroffset, sip);
 		if (error)
 			goto error_return;
@@ -2626,7 +2626,7 @@ xfs_remove(
 	}
 
 	if (parent) {
-		error = xfs_parent_defer_remove(tp, dp, parent, dir_offset, ip);
+		error = xfs_parent_remove(tp, dp, parent, dir_offset, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
@@ -2835,12 +2835,12 @@ xfs_cross_rename(
 	}
 
 	if (xfs_has_parent(mp)) {
-		error = xfs_parent_defer_replace(tp, ip1_pptr, dp1,
+		error = xfs_parent_replace(tp, ip1_pptr, dp1,
 				old_diroffset, name2, dp2, new_diroffset, ip1);
 		if (error)
 			goto out_trans_abort;
 
-		error = xfs_parent_defer_replace(tp, ip2_pptr, dp2,
+		error = xfs_parent_replace(tp, ip2_pptr, dp2,
 				new_diroffset, name1, dp1, old_diroffset, ip2);
 		if (error)
 			goto out_trans_abort;
@@ -3310,7 +3310,7 @@ xfs_rename(
 		goto out_trans_cancel;
 
 	if (wip_pptr) {
-		error = xfs_parent_defer_add(tp, wip_pptr,
+		error = xfs_parent_add(tp, wip_pptr,
 					     src_dp, src_name,
 					     old_diroffset, wip);
 		if (error)
@@ -3318,7 +3318,7 @@ xfs_rename(
 	}
 
 	if (src_ip_pptr) {
-		error = xfs_parent_defer_replace(tp, src_ip_pptr, src_dp,
+		error = xfs_parent_replace(tp, src_ip_pptr, src_dp,
 				old_diroffset, target_name, target_dp,
 				new_diroffset, src_ip);
 		if (error)
@@ -3326,7 +3326,7 @@ xfs_rename(
 	}
 
 	if (tgt_ip_pptr) {
-		error = xfs_parent_defer_remove(tp, target_dp,
+		error = xfs_parent_remove(tp, target_dp,
 						tgt_ip_pptr,
 						new_diroffset, target_ip);
 		if (error)
fs/xfs/xfs_symlink.c +1 -1
index 7872397b8b5c..3f4eebf0b260 100644
@@ -348,7 +348,7 @@ xfs_symlink(
 	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	if (parent) {
-		error = xfs_parent_defer_add(tp, parent, dp, link_name,
+		error = xfs_parent_add(tp, parent, dp, link_name,
 					     diroffset, ip);
 		if (error)
 			goto out_trans_cancel;
#8 lore raw

PATCH 7/7 xfs: rearrange bits of the parent pointer apis for fsck

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rearrange parts of this thing in preparation for fsck code.

---
 fs/xfs/libxfs/xfs_da_format.h |   11 +++++++++++
 fs/xfs/libxfs/xfs_parent.c    |   29 ++++++++++++++++++++++++++++-
 fs/xfs/libxfs/xfs_parent.h    |    6 ++----
 3 files changed, 41 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +11 -0
index 2db1cf97b2c8..c07b8166e8ff 100644
@@ -159,6 +159,17 @@ struct xfs_da3_intnode {
 
 #define XFS_DIR3_FT_MAX			9
 
+#define XFS_DIR3_FTYPE_STR \
+	{ XFS_DIR3_FT_UNKNOWN,	"unknown" }, \
+	{ XFS_DIR3_FT_REG_FILE,	"file" }, \
+	{ XFS_DIR3_FT_DIR,	"directory" }, \
+	{ XFS_DIR3_FT_CHRDEV,	"char" }, \
+	{ XFS_DIR3_FT_BLKDEV,	"block" }, \
+	{ XFS_DIR3_FT_FIFO,	"fifo" }, \
+	{ XFS_DIR3_FT_SOCK,	"sock" }, \
+	{ XFS_DIR3_FT_SYMLINK,	"symlink" }, \
+	{ XFS_DIR3_FT_WHT,	"whiteout" }
+
 /*
  * Byte offset in data block and shortform entry.
  */
fs/xfs/libxfs/xfs_parent.c +28 -1
index ec2bff195773..fe6d4d1a7d57 100644
@@ -91,7 +91,7 @@ xfs_parent_valuecheck(
 }
 
 /* Initializes a xfs_parent_name_rec to be stored as an attribute name */
-void
+static inline void
 xfs_init_parent_name_rec(
 	struct xfs_parent_name_rec	*rec,
 	struct xfs_inode		*ip,
@@ -135,6 +135,33 @@ xfs_parent_irec_from_disk(
 	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
 }
 
+/*
+ * Convert an incore parent_name record to its ondisk format.  If @value or
+ * @valuelen are NULL, they will not be written to.
+ */
+void
+xfs_parent_irec_to_disk(
+	struct xfs_parent_name_rec	*rec,
+	void				*value,
+	int				*valuelen,
+	const struct xfs_parent_name_irec *irec)
+{
+	rec->p_ino = cpu_to_be64(irec->p_ino);
+	rec->p_gen = cpu_to_be32(irec->p_gen);
+	rec->p_diroffset = cpu_to_be32(irec->p_diroffset);
+
+	if (valuelen) {
+		ASSERT(*valuelen > 0);
+		ASSERT(*valuelen >= irec->p_namelen);
+		ASSERT(*valuelen < MAXNAMELEN);
+
+		*valuelen = irec->p_namelen;
+	}
+
+	if (value)
+		memcpy(value, irec->p_name, irec->p_namelen);
+}
+
 /*
  * Allocate memory to control a logged parent pointer update as part of a
  * dirent operation.
fs/xfs/libxfs/xfs_parent.h +2 -4
index 35854e968f1d..4eb92fb4b11b 100644
@@ -33,6 +33,8 @@ struct xfs_parent_name_irec {
 void xfs_parent_irec_from_disk(struct xfs_parent_name_irec *irec,
 		const struct xfs_parent_name_rec *rec,
 		const void *value, int valuelen);
+void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, void *value,
+		int *valuelen, const struct xfs_parent_name_irec *irec);
 
 /*
  * Dynamically allocd structure used to wrap the needed data to pass around
@@ -48,10 +50,6 @@ struct xfs_parent_defer {
 /*
  * Parent pointer attribute prototypes
  */
-void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
-		struct xfs_inode *ip, uint32_t p_diroffset);
-void xfs_init_parent_name_irec(struct xfs_parent_name_irec *irec,
-			       struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
#9 lore raw

cover xfs: encode parent pointer name in xattr key

In reply to Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we create a special VLOOKUP mode for extended attributes that allows parent pointers to require matching on both the name and the value.

The ondisk format of a parent pointer can then become:

    (parent_ino, parent_gen, dirent_name[0:242]) -> (dirent_name[243:255])

Because we can always look up a specific parent pointer. Most of the patches in this patchset prepare the high level xattr code and the lower level logging code to do this correctly, and the last patch switches the ondisk format.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key

---
 fs/xfs/libxfs/xfs_attr.c       |   66 +++++---
 fs/xfs/libxfs/xfs_attr.h       |    5 +
 fs/xfs/libxfs/xfs_attr_leaf.c  |   41 ++++-
 fs/xfs/libxfs/xfs_da_btree.h   |    6 +
 fs/xfs/libxfs/xfs_da_format.h  |   39 ++++-
 fs/xfs/libxfs/xfs_fs.h         |    2 
 fs/xfs/libxfs/xfs_log_format.h |   31 +++-
 fs/xfs/libxfs/xfs_parent.c     |  215 +++++++++++++++++----------
 fs/xfs/libxfs/xfs_parent.h     |   46 +++---
 fs/xfs/libxfs/xfs_trans_resv.c |    7 +
 fs/xfs/scrub/dir.c             |   34 +---
 fs/xfs/scrub/dir_repair.c      |   87 +++--------
 fs/xfs/scrub/parent.c          |   48 +-----
 fs/xfs/scrub/parent_repair.c   |   55 ++-----
 fs/xfs/scrub/trace.h           |   65 +-------
 fs/xfs/xfs_attr_item.c         |  318 +++++++++++++++++++++++++++++++---------
 fs/xfs/xfs_attr_item.h         |    3 
 fs/xfs/xfs_inode.c             |   30 ++--
 fs/xfs/xfs_ondisk.h            |    1 
 fs/xfs/xfs_parent_utils.c      |    8 +
 fs/xfs/xfs_symlink.c           |    3 
 fs/xfs/xfs_xattr.c             |    5 +
 22 files changed, 660 insertions(+), 455 deletions(-)
#10 lore raw

PATCH 1/17 xfs: document the ri_total validation in xlog_recover_attri_commit_pass2

From: Darrick J. Wong <[email protected]>

Leave comments explaining why log recovery wants the log item buffer counts to be what they are.

---
 fs/xfs/xfs_attr_item.c |    3 +++
 1 file changed, 3 insertions(+)
fs/xfs/xfs_attr_item.c +3 -0
index da807f286a09..dac5d9cfc804 100644
@@ -769,6 +769,7 @@ xlog_recover_attri_commit_pass2(
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
+		/* Log item, attr name, attr value */
 		if (item->ri_total != 3) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
@@ -776,6 +777,7 @@ xlog_recover_attri_commit_pass2(
 		}
 		break;
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
+		/* Log item, attr name */
 		if (item->ri_total != 2) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
@@ -783,6 +785,7 @@ xlog_recover_attri_commit_pass2(
 		}
 		break;
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
+		/* Log item, attr name, new attr name, attr value */
 		if (item->ri_total != 4) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
#11 lore raw

PATCH 2/17 xfs: make xfs_attr_set require XFS_DA_OP_REMOVE

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Quite a few patches from now, we're going to change the parent pointer xattr format to encode as much of the dirent name in the xattr name as fits, and spill the rest of it to the xattr value. To make this work correctly, we'll be adding the ability to look up xattrs based on name /and/ value.

Internally, the xattr data structure supports attributes with a zero length value, which is how we're going to store parent pointers for short dirent names. The parent pointer repair code uses xfs_attr_set to add missing and remove dangling parent pointers, so that interface must be capable of setting an xattr with args->value == NULL.

The userspace API doesn't support this, so xfs_attr_set currently treats a NULL args->value as a request to remove an attr. However, that's a quirk of the existing callers and the interface. Make the callers of xfs_attr_set to declare explicitly that they want to remove an xattr.

---
 fs/xfs/libxfs/xfs_attr.c   |    9 +++++----
 fs/xfs/libxfs/xfs_parent.c |    1 +
 fs/xfs/xfs_xattr.c         |    5 +++++
 3 files changed, 11 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -4
index 3065dd622102..756d93526075 100644
@@ -977,6 +977,7 @@ xfs_attr_set(
 	struct xfs_inode	*dp = args->dp;
 	struct xfs_mount	*mp = dp->i_mount;
 	struct xfs_trans_res	tres;
+	bool			is_remove = args->op_flags & XFS_DA_OP_REMOVE;
 	bool			rsvd;
 	int			error, local;
 	int			rmt_blks = 0;
@@ -1004,7 +1005,7 @@ xfs_attr_set(
 	args->op_flags = XFS_DA_OP_OKNOENT |
 					(args->op_flags & XFS_DA_OP_LOGGED);
 
-	if (args->value) {
+	if (!is_remove) {
 		XFS_STATS_INC(mp, xs_attr_set);
 		args->total = xfs_attr_calc_size(args, &local);
 
@@ -1038,7 +1039,7 @@ xfs_attr_set(
 	if (error)
 		return error;
 
-	if (args->value || xfs_inode_hasattr(dp)) {
+	if (!is_remove || xfs_inode_hasattr(dp)) {
 		error = xfs_iext_count_may_overflow(dp, XFS_ATTR_FORK,
 				XFS_IEXT_ATTR_MANIP_CNT(rmt_blks));
 		if (error == -EFBIG)
@@ -1052,7 +1053,7 @@ xfs_attr_set(
 	switch (error) {
 	case -EEXIST:
 		/* if no value, we are performing a remove operation */
-		if (!args->value) {
+		if (is_remove) {
 			error = xfs_attr_defer_remove(args);
 			break;
 		}
@@ -1064,7 +1065,7 @@ xfs_attr_set(
 		break;
 	case -ENOATTR:
 		/* Can't remove what isn't there. */
-		if (!args->value)
+		if (is_remove)
 			goto out_trans_cancel;
 
 		/* Pure replace fails if no existing attr to replace. */
fs/xfs/libxfs/xfs_parent.c +1 -0
index a2575bf44c89..96ce5de8508a 100644
@@ -384,6 +384,7 @@ xfs_parent_unset(
 	scr->args.geo		= ip->i_mount->m_attr_geo;
 	scr->args.name		= (const unsigned char *)&scr->rec;
 	scr->args.namelen	= sizeof(struct xfs_parent_name_rec);
+	scr->args.op_flags	= XFS_DA_OP_REMOVE;
 	scr->args.whichfork	= XFS_ATTR_FORK;
 
 	return xfs_attr_set(&scr->args);
fs/xfs/xfs_xattr.c +5 -0
index b92fc38bd550..46de0e2bfc46 100644
@@ -103,6 +103,11 @@ xfs_attr_change(
 		use_logging = true;
 	}
 
+	if (args->value)
+		args->op_flags &= ~XFS_DA_OP_REMOVE;
+	else
+		args->op_flags |= XFS_DA_OP_REMOVE;
+
 	error = xfs_attr_set(args);
 
 	if (use_logging)
#12 lore raw

PATCH 3/17 xfs: allow xattr matching on value for local/sf attrs

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a new VLOOKUP flag to signal that the caller wants to look up an extended attribute by name and value. This only works with shortform and local attributes. Only parent pointers need this functionality, parent pointers cannot be remote xattrs, hence this limitation is ok.

---
 fs/xfs/libxfs/xfs_attr_leaf.c |   41 +++++++++++++++++++++++++++++++++++------
 fs/xfs/libxfs/xfs_da_btree.h  |    4 +++-
 2 files changed, 38 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +35 -6
index e6c4c8b52a55..d05f2c5cc0cc 100644
@@ -473,10 +473,12 @@ xfs_attr3_leaf_read(
  */
 static bool
 xfs_attr_match(
-	struct xfs_da_args	*args,
-	uint8_t			namelen,
-	unsigned char		*name,
-	int			flags)
+	const struct xfs_da_args	*args,
+	uint8_t				namelen,
+	const unsigned char		*name,
+	unsigned int			valuelen,
+	const void			*value,
+	int				flags)
 {
 
 	if (args->namelen != namelen)
@@ -484,6 +486,23 @@ xfs_attr_match(
 	if (memcmp(args->name, name, namelen) != 0)
 		return false;
 
+	if (args->op_flags & XFS_DA_OP_VLOOKUP) {
+		if (args->valuelen != valuelen)
+			return false;
+		if (args->valuelen && !value) {
+			/* not implemented for remote values */
+			ASSERT(0);
+			return false;
+		}
+		if (valuelen && !args->value) {
+			/* caller gave us valuelen > 0 but no value?? */
+			ASSERT(0);
+			return false;
+		}
+		if (valuelen > 0 && memcmp(args->value, value, valuelen) != 0)
+			return false;
+	}
+
 	/* Recovery ignores the INCOMPLETE flag. */
 	if ((args->op_flags & XFS_DA_OP_RECOVERY) &&
 	    args->attr_filter == (flags & XFS_ATTR_NSP_ONDISK_MASK))
@@ -502,6 +521,10 @@ xfs_attr_copy_value(
 	unsigned char		*value,
 	int			valuelen)
 {
+	/* vlookups already supplied the attr value; don't copy anything */
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		return 0;
+
 	/*
 	 * No copy if all we have to do is get the length
 	 */
@@ -726,6 +749,7 @@ xfs_attr_sf_findname(
 			     base += size, i++) {
 		size = xfs_attr_sf_entsize(sfe);
 		if (!xfs_attr_match(args, sfe->namelen, sfe->nameval,
+				    sfe->valuelen, &sfe->nameval[sfe->namelen],
 				    sfe->flags))
 			continue;
 		break;
@@ -896,6 +920,7 @@ xfs_attr_shortform_lookup(xfs_da_args_t *args)
 	for (i = 0; i < sf->hdr.count;
 				sfe = xfs_attr_sf_nextentry(sfe), i++) {
 		if (xfs_attr_match(args, sfe->namelen, sfe->nameval,
+				sfe->valuelen, &sfe->nameval[sfe->namelen],
 				sfe->flags))
 			return -EEXIST;
 	}
@@ -923,6 +948,7 @@ xfs_attr_shortform_getvalue(
 	for (i = 0; i < sf->hdr.count;
 				sfe = xfs_attr_sf_nextentry(sfe), i++) {
 		if (xfs_attr_match(args, sfe->namelen, sfe->nameval,
+				sfe->valuelen, &sfe->nameval[sfe->namelen],
 				sfe->flags))
 			return xfs_attr_copy_value(args,
 				&sfe->nameval[args->namelen], sfe->valuelen);
@@ -2484,14 +2510,17 @@ xfs_attr3_leaf_lookup_int(
 		if (entry->flags & XFS_ATTR_LOCAL) {
 			name_loc = xfs_attr3_leaf_name_local(leaf, probe);
 			if (!xfs_attr_match(args, name_loc->namelen,
-					name_loc->nameval, entry->flags))
+					name_loc->nameval,
+					be16_to_cpu(name_loc->valuelen),
+					&name_loc->nameval[name_loc->namelen],
+					entry->flags))
 				continue;
 			args->index = probe;
 			return -EEXIST;
 		} else {
 			name_rmt = xfs_attr3_leaf_name_remote(leaf, probe);
 			if (!xfs_attr_match(args, name_rmt->namelen,
-					name_rmt->name, entry->flags))
+					name_rmt->name, 0, NULL, entry->flags))
 				continue;
 			args->index = probe;
 			args->rmtvaluelen = be32_to_cpu(name_rmt->valuelen);
fs/xfs/libxfs/xfs_da_btree.h +3 -1
index 90b86d00258f..6132787ad1f4 100644
@@ -96,6 +96,7 @@ typedef struct xfs_da_args {
 #define XFS_DA_OP_REMOVE	(1u << 6) /* this is a remove operation */
 #define XFS_DA_OP_RECOVERY	(1u << 7) /* Log recovery operation */
 #define XFS_DA_OP_LOGGED	(1u << 8) /* Use intent items to track op */
+#define XFS_DA_OP_VLOOKUP	(1u << 9) /* Compare local attr value for lookup */
 
 #define XFS_DA_OP_FLAGS \
 	{ XFS_DA_OP_JUSTCHECK,	"JUSTCHECK" }, \
@@ -106,7 +107,8 @@ typedef struct xfs_da_args {
 	{ XFS_DA_OP_NOTIME,	"NOTIME" }, \
 	{ XFS_DA_OP_REMOVE,	"REMOVE" }, \
 	{ XFS_DA_OP_RECOVERY,	"RECOVERY" }, \
-	{ XFS_DA_OP_LOGGED,	"LOGGED" }
+	{ XFS_DA_OP_LOGGED,	"LOGGED" }, \
+	{ XFS_DA_OP_VLOOKUP,	"VLOOKUP" }
 
 /*
  * Storage for holding state during Btree searches and split/join ops.
#13 lore raw

PATCH 4/17 xfs: preserve VLOOKUP in xfs_attr_set

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Preserve the attr-value lookup flag when calling xfs_attr_set. Normal xattr users will never use this, but parent pointer fsck will.

---
 fs/xfs/libxfs/xfs_attr.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
fs/xfs/libxfs/xfs_attr.c +4 -4
index 756d93526075..86672061c99e 100644
@@ -999,11 +999,11 @@ xfs_attr_set(
 	/*
 	 * We have no control over the attribute names that userspace passes us
 	 * to remove, so we have to allow the name lookup prior to attribute
-	 * removal to fail as well.  Preserve the logged flag, since we need
-	 * to pass that through to the logging code.
+	 * removal to fail as well.  Preserve the logged and vlookup flags,
+	 * since we need to pass them through to the lower levels.
 	 */
-	args->op_flags = XFS_DA_OP_OKNOENT |
-					(args->op_flags & XFS_DA_OP_LOGGED);
+	args->op_flags &= (XFS_DA_OP_LOGGED | XFS_DA_OP_VLOOKUP);
+	args->op_flags |= XFS_DA_OP_OKNOENT;
 
 	if (!is_remove) {
 		XFS_STATS_INC(mp, xs_attr_set);
#14 lore raw

PATCH 5/17 xfs: restructure xfs_attr_complete_op a bit

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Reduce the indentation in this function by flattening the nested if statements. We're going to add more code later to this function later, hence the early cleanup. No functional changes.

---
 fs/xfs/libxfs/xfs_attr.c |   20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_attr.c +10 -10
index 86672061c99e..996ef24482e1 100644
@@ -422,17 +422,17 @@ xfs_attr_complete_op(
 	bool			do_replace = args->op_flags & XFS_DA_OP_REPLACE;
 
 	args->op_flags &= ~XFS_DA_OP_REPLACE;
-	if (do_replace) {
-		args->attr_filter &= ~XFS_ATTR_INCOMPLETE;
-		if (args->new_namelen > 0) {
-			args->name = args->new_name;
-			args->namelen = args->new_namelen;
-			args->hashval = xfs_da_hashname(args->name,
-							args->namelen);
-		}
+	if (!do_replace)
+		return XFS_DAS_DONE;
+
+	args->attr_filter &= ~XFS_ATTR_INCOMPLETE;
+	if (args->new_namelen == 0)
 		return replace_state;
-	}
-	return XFS_DAS_DONE;
+
+	args->name = args->new_name;
+	args->namelen = args->new_namelen;
+	args->hashval = xfs_da_hashname(args->name, args->namelen);
+	return replace_state;
 }
 
 static int
#15 lore raw

PATCH 6/17 xfs: use helpers to extract xattr op from opflags

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create helper functions to extract the xattr op from the ondisk xattri log item and the incore attr intent item. These will get more use in the patches that follow.

---
 fs/xfs/libxfs/xfs_attr.h |    5 +++++
 fs/xfs/xfs_attr_item.c   |   18 +++++++++++-------
 2 files changed, 16 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr.h +5 -0
index 02a20b948c8f..0185d29d5109 100644
@@ -530,6 +530,11 @@ struct xfs_attr_intent {
 	struct xfs_bmbt_irec		xattri_map;
 };
 
+static inline unsigned int
+xfs_attr_intent_op(const struct xfs_attr_intent *attr)
+{
+	return attr->xattri_op_flags & XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+}
 
 /*========================================================================
  * Function prototypes for the kernel.
fs/xfs/xfs_attr_item.c +11 -7
index dac5d9cfc804..f4b5777445e1 100644
@@ -374,6 +374,12 @@ xfs_xattri_finish_update(
 	return error;
 }
 
+static inline unsigned int
+xfs_attr_log_item_op(const struct xfs_attri_log_format *attrp)
+{
+	return attrp->alfi_op_flags & XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+}
+
 /* Log an attr to the intent item. */
 STATIC void
 xfs_attr_log_item(
@@ -525,10 +531,9 @@ xfs_attri_validate(
 	struct xfs_mount		*mp,
 	struct xfs_attri_log_format	*attrp)
 {
-	unsigned int			op = attrp->alfi_op_flags &
-					     XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+	unsigned int			op = xfs_attr_log_item_op(attrp);
 
-	if (attrp->alfi_op_flags != XFS_ATTRI_OP_FLAGS_NVREPLACE &&
+	if (op != XFS_ATTRI_OP_FLAGS_NVREPLACE &&
 	    attrp->alfi_nname_len != 0)
 		return false;
 
@@ -606,8 +611,7 @@ xfs_attri_item_recover(
 	args = (struct xfs_da_args *)(attr + 1);
 
 	attr->xattri_da_args = args;
-	attr->xattri_op_flags = attrp->alfi_op_flags &
-						XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+	attr->xattri_op_flags = xfs_attr_log_item_op(attrp);
 
 	/*
 	 * We're reconstructing the deferred work state structure from the
@@ -631,7 +635,7 @@ xfs_attri_item_recover(
 
 	ASSERT(xfs_sb_version_haslogxattrs(&mp->m_sb));
 
-	switch (attr->xattri_op_flags) {
+	switch (xfs_attr_intent_op(attr)) {
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
@@ -765,7 +769,7 @@ xlog_recover_attri_commit_pass2(
 		return -EFSCORRUPTED;
 	}
 
-	op = attri_formatp->alfi_op_flags & XFS_ATTRI_OP_FLAGS_TYPE_MASK;
+	op = xfs_attr_log_item_op(attri_formatp);
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
#16 lore raw

PATCH 7/17 xfs: validate recovered name buffers when recovering xattr items

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Strengthen the xattri log item recovery code by checking that we actually have the required name and newname buffers for whatever operation we're replaying.

---
 fs/xfs/xfs_attr_item.c |   62 ++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 59 insertions(+), 3 deletions(-)
fs/xfs/xfs_attr_item.c +59 -3
index f4b5777445e1..054237177446 100644
@@ -801,8 +801,8 @@ xlog_recover_attri_commit_pass2(
 				     attri_formatp, len);
 		return -EFSCORRUPTED;
 	}
-
 	i++;
+
 	/* Validate the attr name */
 	if (item->ri_buf[i].i_len !=
 			xlog_calc_iovec_len(attri_formatp->alfi_name_len)) {
@@ -818,10 +818,10 @@ xlog_recover_attri_commit_pass2(
 				item->ri_buf[i].i_addr, item->ri_buf[i].i_len);
 		return -EFSCORRUPTED;
 	}
-
 	i++;
+
+	/* Validate the attr name. */
 	if (attri_formatp->alfi_nname_len) {
-		/* Validate the attr nname */
 		if (item->ri_buf[i].i_len !=
 		    xlog_calc_iovec_len(attri_formatp->alfi_nname_len)) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
@@ -852,6 +852,62 @@ xlog_recover_attri_commit_pass2(
 		}
 
 		attr_value = item->ri_buf[i].i_addr;
+		i++;
+	}
+
+	/*
+	 * Make sure we got the correct number of buffers for the operation
+	 * that we just loaded.
+	 */
+	if (i != item->ri_total) {
+		XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+				attri_formatp, len);
+		return -EFSCORRUPTED;
+	}
+
+	switch (op) {
+	case XFS_ATTRI_OP_FLAGS_REMOVE:
+		/* Regular remove operations operate only on names. */
+		if (attr_value != NULL || attri_formatp->alfi_value_len != 0) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		fallthrough;
+	case XFS_ATTRI_OP_FLAGS_SET:
+	case XFS_ATTRI_OP_FLAGS_REPLACE:
+		/*
+		 * Regular xattr set/remove/replace operations require a name
+		 * and do not take a newname.  Values are optional for set and
+		 * replace.
+		 */
+		if (attr_name == NULL || attri_formatp->alfi_name_len == 0) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		if (attr_nname != NULL || attri_formatp->alfi_nname_len != 0) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		break;
+	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
+		/*
+		 * Name-value replace operations require the caller to specify
+		 * the old and new name explicitly.  Values are optional.
+		 */
+		if (attr_name == NULL || attri_formatp->alfi_name_len == 0) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		if (attr_nname == NULL || attri_formatp->alfi_nname_len == 0) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		break;
 	}
 
 	/*
#17 lore raw

PATCH 8/17 xfs: always set args->value in xfs_attri_item_recover

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Always set args->value to the recovered value buffer. This reduces the amount of code in the switch statement, and hence the amount of thinking that I have to do. We validated the recovered buffers, supposedly.

---
 fs/xfs/xfs_attr_item.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/xfs_attr_item.c +2 -2
index 054237177446..66b4c167588d 100644
@@ -629,6 +629,8 @@ xfs_attri_item_recover(
 	args->new_name = nv->nname.i_addr;
 	args->new_namelen = nv->nname.i_len;
 	args->hashval = xfs_da_hashname(args->name, args->namelen);
+	args->value = nv->value.i_addr;
+	args->valuelen = nv->value.i_len;
 	args->attr_filter = attrp->alfi_attr_filter & XFS_ATTRI_FILTER_MASK;
 	args->op_flags = XFS_DA_OP_RECOVERY | XFS_DA_OP_OKNOENT |
 			 XFS_DA_OP_LOGGED;
@@ -639,8 +641,6 @@ xfs_attri_item_recover(
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
-		args->value = nv->value.i_addr;
-		args->valuelen = nv->value.i_len;
 		args->total = xfs_attr_calc_size(args, &local);
 		if (xfs_inode_hasattr(args->dp))
 			attr->xattri_dela_state = xfs_attr_init_replace_state(args);
#18 lore raw

PATCH 9/17 xfs: flip nvreplace detection in xfs_attr_complete_op

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Gate the NVREPLACE code on the op flags directly, instead of inferring it through args->new_namelen.

---
 fs/xfs/libxfs/xfs_attr.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +7 -1
index 996ef24482e1..1a047099e9c7 100644
@@ -426,9 +426,15 @@ xfs_attr_complete_op(
 		return XFS_DAS_DONE;
 
 	args->attr_filter &= ~XFS_ATTR_INCOMPLETE;
-	if (args->new_namelen == 0)
+	if (xfs_attr_intent_op(attr) != XFS_ATTRI_OP_FLAGS_NVREPLACE)
 		return replace_state;
 
+	/*
+	 * NVREPLACE operations require the caller to set the old and new names
+	 * explicitly.
+	 */
+	ASSERT(args->new_namelen > 0);
+
 	args->name = args->new_name;
 	args->namelen = args->new_namelen;
 	args->hashval = xfs_da_hashname(args->name, args->namelen);
#19 lore raw

PATCH 10/17 xfs: log VLOOKUP xattr removal operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If high level code wants to do a deferred xattr remove operation with the VLOOKUP flag set, we need to push this through the log.

---
 fs/xfs/libxfs/xfs_attr.c       |    6 +++++-
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/xfs_attr_item.c         |   16 ++++++++++++++++
 3 files changed, 22 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
index 1a047099e9c7..3425b94ca4ab 100644
@@ -959,9 +959,13 @@ xfs_attr_defer_remove(
 {
 
 	struct xfs_attr_intent	*new;
+	int			op_flag = XFS_ATTRI_OP_FLAGS_REMOVE;
 	int			error;
 
-	error  = xfs_attr_intent_init(args, XFS_ATTRI_OP_FLAGS_REMOVE, &new);
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVREMOVE;
+
+	error  = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
 		return error;
 
fs/xfs/libxfs/xfs_log_format.h +1 -0
index 727b5a858028..a3d95a3d8476 100644
@@ -959,6 +959,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_REMOVE	2	/* Remove the attribute */
 #define XFS_ATTRI_OP_FLAGS_REPLACE	3	/* Replace the attribute */
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
+#define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +16 -0
index 66b4c167588d..a72368acda3d 100644
@@ -549,6 +549,7 @@ xfs_attri_validate(
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
 		break;
 	default:
 		return false;
@@ -647,6 +648,9 @@ xfs_attri_item_recover(
 		else
 			attr->xattri_dela_state = xfs_attr_init_add_state(args);
 		break;
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
+		args->op_flags |= XFS_DA_OP_VLOOKUP;
+		fallthrough;
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 		if (!xfs_inode_hasattr(args->dp))
 			goto out;
@@ -771,6 +775,14 @@ xlog_recover_attri_commit_pass2(
 
 	op = xfs_attr_log_item_op(attri_formatp);
 	switch (op) {
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
+		/* Log item, attr name, optional attr value */
+		if (item->ri_total != 3 && item->ri_total != 2) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		break;
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 		/* Log item, attr name, attr value */
@@ -874,12 +886,16 @@ xlog_recover_attri_commit_pass2(
 			return -EFSCORRUPTED;
 		}
 		fallthrough;
+	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 		/*
 		 * Regular xattr set/remove/replace operations require a name
 		 * and do not take a newname.  Values are optional for set and
 		 * replace.
+		 *
+		 * Name-value remove operations must have a name, do not
+		 * take a newname, and can take a value.
 		 */
 		if (attr_name == NULL || attri_formatp->alfi_name_len == 0) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
#20 lore raw

PATCH 11/17 xfs: log VLOOKUP xattr setting operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If high level code wants to do a deferred xattr set operation with the VLOOKUP flag set, we need to push this through the log.

---
 fs/xfs/libxfs/xfs_attr.c       |    6 +++++-
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/xfs_attr_item.c         |    8 +++++++-
 3 files changed, 13 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_attr.c +5 -1
index 3425b94ca4ab..313593b36e65 100644
@@ -916,9 +916,13 @@ xfs_attr_defer_add(
 	struct xfs_da_args	*args)
 {
 	struct xfs_attr_intent	*new;
+	int			op_flag = XFS_ATTRI_OP_FLAGS_SET;
 	int			error = 0;
 
-	error = xfs_attr_intent_init(args, XFS_ATTRI_OP_FLAGS_SET, &new);
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVSET;
+
+	error = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
 		return error;
 
fs/xfs/libxfs/xfs_log_format.h +1 -0
index a3d95a3d8476..1fe9f7394812 100644
@@ -960,6 +960,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_REPLACE	3	/* Replace the attribute */
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
+#define XFS_ATTRI_OP_FLAGS_NVSET	6	/* Set attr with w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +7 -1
index a72368acda3d..4472250f461c 100644
@@ -550,6 +550,7 @@ xfs_attri_validate(
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
+	case XFS_ATTRI_OP_FLAGS_NVSET:
 		break;
 	default:
 		return false;
@@ -639,6 +640,9 @@ xfs_attri_item_recover(
 	ASSERT(xfs_sb_version_haslogxattrs(&mp->m_sb));
 
 	switch (xfs_attr_intent_op(attr)) {
+	case XFS_ATTRI_OP_FLAGS_NVSET:
+		args->op_flags |= XFS_DA_OP_VLOOKUP;
+		fallthrough;
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
@@ -775,6 +779,7 @@ xlog_recover_attri_commit_pass2(
 
 	op = xfs_attr_log_item_op(attri_formatp);
 	switch (op) {
+	case XFS_ATTRI_OP_FLAGS_NVSET:
 	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
 		/* Log item, attr name, optional attr value */
 		if (item->ri_total != 3 && item->ri_total != 2) {
@@ -886,6 +891,7 @@ xlog_recover_attri_commit_pass2(
 			return -EFSCORRUPTED;
 		}
 		fallthrough;
+	case XFS_ATTRI_OP_FLAGS_NVSET:
 	case XFS_ATTRI_OP_FLAGS_NVREMOVE:
 	case XFS_ATTRI_OP_FLAGS_SET:
 	case XFS_ATTRI_OP_FLAGS_REPLACE:
@@ -894,7 +900,7 @@ xlog_recover_attri_commit_pass2(
 		 * and do not take a newname.  Values are optional for set and
 		 * replace.
 		 *
-		 * Name-value remove operations must have a name, do not
+		 * Name-value set/remove operations must have a name, do not
 		 * take a newname, and can take a value.
 		 */
 		if (attr_name == NULL || attri_formatp->alfi_name_len == 0) {
#21 lore raw

PATCH 12/17 xfs: overlay alfi_nname_len atop alfi_name_len for NVREPLACE

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

In preparation for being able to log the old attr value in a NVREPLACE operation, encode the old and new name lengths in the alfi_name_len field. We haven't shipped a kernel with XFS_ATTRI_OP_FLAGS_NVREPLACE, so we can still tweak the ondisk log item format.

---
 fs/xfs/libxfs/xfs_log_format.h |   14 +++++-
 fs/xfs/xfs_attr_item.c         |   99 ++++++++++++++++++++++------------------
 2 files changed, 67 insertions(+), 46 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +12 -2
index 1fe9f7394812..32035786135b 100644
@@ -979,11 +979,21 @@ struct xfs_icreate_log {
 struct xfs_attri_log_format {
 	uint16_t	alfi_type;	/* attri log item type */
 	uint16_t	alfi_size;	/* size of this item */
-	uint32_t	alfi_nname_len;	/* attr new name length */
+	uint32_t	__pad;		/* pad to 64 bit aligned */
 	uint64_t	alfi_id;	/* attri identifier */
 	uint64_t	alfi_ino;	/* the inode for this attr operation */
 	uint32_t	alfi_op_flags;	/* marks the op as a set or remove */
-	uint32_t	alfi_name_len;	/* attr name length */
+	union {
+		uint32_t	alfi_name_len;	/* attr name length */
+		struct {
+			/*
+			 * For NVREPLACE, these are the lengths of the old and
+			 * new attr name.
+			 */
+			uint16_t	alfi_oldname_len;
+			uint16_t	alfi_newname_len;
+		};
+	};
 	uint32_t	alfi_value_len;	/* attr value length */
 	uint32_t	alfi_attr_filter;/* attr filter flags */
 };
fs/xfs/xfs_attr_item.c +55 -44

208 lines not on this page · raw message

#22 lore raw

PATCH 13/17 xfs: refactor value length in xlog_recover_attri_commit_pass2

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use a shorter convenience variable for the attr value length in this function, since we've just done that for attr name lengths.

---
 fs/xfs/xfs_attr_item.c |   11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
fs/xfs/xfs_attr_item.c +6 -5
index bb21d4e6dcf2..fba6a472805d 100644
@@ -775,6 +775,7 @@ xlog_recover_attri_commit_pass2(
 	size_t				len;
 	const void			*attr_newname = NULL;
 	unsigned int			name_len = 0, newname_len = 0;
+	unsigned int			value_len;
 	int				op, i = 0;
 
 	/* Validate xfs_attri_log_format before the large memory allocation */
@@ -792,6 +793,7 @@ xlog_recover_attri_commit_pass2(
 		return -EFSCORRUPTED;
 	}
 
+	value_len = attri_formatp->alfi_value_len;
 	op = xfs_attr_log_item_op(attri_formatp);
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_NVSET:
@@ -878,8 +880,8 @@ xlog_recover_attri_commit_pass2(
 
 
 	/* Validate the attr value, if present */
-	if (attri_formatp->alfi_value_len != 0) {
-		if (item->ri_buf[i].i_len != xlog_calc_iovec_len(attri_formatp->alfi_value_len)) {
+	if (value_len != 0) {
+		if (item->ri_buf[i].i_len != xlog_calc_iovec_len(value_len)) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					attri_formatp, len);
 			return -EFSCORRUPTED;
@@ -902,7 +904,7 @@ xlog_recover_attri_commit_pass2(
 	switch (op) {
 	case XFS_ATTRI_OP_FLAGS_REMOVE:
 		/* Regular remove operations operate only on names. */
-		if (attr_value != NULL || attri_formatp->alfi_value_len != 0) {
+		if (attr_value != NULL || value_len != 0) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
 					     attri_formatp, len);
 			return -EFSCORRUPTED;
@@ -950,8 +952,7 @@ xlog_recover_attri_commit_pass2(
 	 * reference.
 	 */
 	nv = xfs_attri_log_nameval_alloc(attr_name, name_len, attr_newname,
-			newname_len, attr_value,
-			attri_formatp->alfi_value_len);
+			newname_len, attr_value, value_len);
 
 	attrip = xfs_attri_init(mp, nv);
 	memcpy(&attrip->attri_format, attri_formatp, len);
#23 lore raw

PATCH 14/17 xfs: rename nname to newname

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Burn a couple of extra source code bytes to make it clear what this does.

---
 fs/xfs/libxfs/xfs_log_format.h |    2 +-
 fs/xfs/xfs_attr_item.c         |   40 ++++++++++++++++++++--------------------
 fs/xfs/xfs_attr_item.h         |    2 +-
 3 files changed, 22 insertions(+), 22 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +1 -1
index 32035786135b..8b16ae27c2fd 100644
@@ -117,7 +117,7 @@ struct xfs_unmount_log_format {
 #define XLOG_REG_TYPE_ATTRD_FORMAT	28
 #define XLOG_REG_TYPE_ATTR_NAME	29
 #define XLOG_REG_TYPE_ATTR_VALUE	30
-#define XLOG_REG_TYPE_ATTR_NNAME	31
+#define XLOG_REG_TYPE_ATTR_NEWNAME	31
 #define XLOG_REG_TYPE_MAX		31
 
 
fs/xfs/xfs_attr_item.c +20 -20
index fba6a472805d..b730f8e7a785 100644
@@ -75,8 +75,8 @@ static inline struct xfs_attri_log_nameval *
 xfs_attri_log_nameval_alloc(
 	const void			*name,
 	unsigned int			name_len,
-	const void			*nname,
-	unsigned int			nname_len,
+	const void			*newname,
+	unsigned int			newname_len,
 	const void			*value,
 	unsigned int			value_len)
 {
@@ -87,25 +87,25 @@ xfs_attri_log_nameval_alloc(
 	 * this. But kvmalloc() utterly sucks, so we use our own version.
 	 */
 	nv = xlog_kvmalloc(sizeof(struct xfs_attri_log_nameval) +
-					name_len + nname_len + value_len);
+					name_len + newname_len + value_len);
 
 	nv->name.i_addr = nv + 1;
 	nv->name.i_len = name_len;
 	nv->name.i_type = XLOG_REG_TYPE_ATTR_NAME;
 	memcpy(nv->name.i_addr, name, name_len);
 
-	if (nname_len) {
-		nv->nname.i_addr = nv->name.i_addr + name_len;
-		nv->nname.i_len = nname_len;
-		memcpy(nv->nname.i_addr, nname, nname_len);
+	if (newname_len) {
+		nv->newname.i_addr = nv->name.i_addr + name_len;
+		nv->newname.i_len = newname_len;
+		memcpy(nv->newname.i_addr, newname, newname_len);
 	} else {
-		nv->nname.i_addr = NULL;
-		nv->nname.i_len = 0;
+		nv->newname.i_addr = NULL;
+		nv->newname.i_len = 0;
 	}
-	nv->nname.i_type = XLOG_REG_TYPE_ATTR_NNAME;
+	nv->newname.i_type = XLOG_REG_TYPE_ATTR_NEWNAME;
 
 	if (value_len) {
-		nv->value.i_addr = nv->name.i_addr + nname_len + name_len;
+		nv->value.i_addr = nv->name.i_addr + newname_len + name_len;
 		nv->value.i_len = value_len;
 		memcpy(nv->value.i_addr, value, value_len);
 	} else {
@@ -159,9 +159,9 @@ xfs_attri_item_size(
 	*nbytes += sizeof(struct xfs_attri_log_format) +
 			xlog_calc_iovec_len(nv->name.i_len);
 
-	if (nv->nname.i_len) {
+	if (nv->newname.i_len) {
 		*nvecs += 1;
-		*nbytes += xlog_calc_iovec_len(nv->nname.i_len);
+		*nbytes += xlog_calc_iovec_len(nv->newname.i_len);
 	}
 
 	if (nv->value.i_len) {
@@ -197,7 +197,7 @@ xfs_attri_item_format(
 	ASSERT(nv->name.i_len > 0);
 	attrip->attri_format.alfi_size++;
 
-	if (nv->nname.i_len > 0)
+	if (nv->newname.i_len > 0)
 		attrip->attri_format.alfi_size++;
 
 	if (nv->value.i_len > 0)
@@ -208,8 +208,8 @@ xfs_attri_item_format(
 			sizeof(struct xfs_attri_log_format));
 	xlog_copy_from_iovec(lv, &vecp, &nv->name);
 
-	if (nv->nname.i_len > 0)
-		xlog_copy_from_iovec(lv, &vecp, &nv->nname);
+	if (nv->newname.i_len > 0)
+		xlog_copy_from_iovec(lv, &vecp, &nv->newname);
 
 	if (nv->value.i_len > 0)
 		xlog_copy_from_iovec(lv, &vecp, &nv->value);
@@ -405,7 +405,7 @@ xfs_attr_log_item(
 
 	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
-		attrp->alfi_newname_len = attr->xattri_nameval->nname.i_len;
+		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
 	} else {
 		attrp->alfi_name_len = attr->xattri_nameval->name.i_len;
 	}
@@ -638,8 +638,8 @@ xfs_attri_item_recover(
 	args->whichfork = XFS_ATTR_FORK;
 	args->name = nv->name.i_addr;
 	args->namelen = nv->name.i_len;
-	args->new_name = nv->nname.i_addr;
-	args->new_namelen = nv->nname.i_len;
+	args->new_name = nv->newname.i_addr;
+	args->new_namelen = nv->newname.i_len;
 	args->hashval = xfs_da_hashname(args->name, args->namelen);
 	args->value = nv->value.i_addr;
 	args->valuelen = nv->value.i_len;
@@ -858,7 +858,7 @@ xlog_recover_attri_commit_pass2(
 	}
 	i++;
 
-	/* Validate the attr nname */
+	/* Validate the new attr name */
 	if (newname_len > 0) {
 		if (item->ri_buf[i].i_len != xlog_calc_iovec_len(newname_len)) {
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
fs/xfs/xfs_attr_item.h +1 -1
index 24d4968dd6cc..e374712ba06b 100644
@@ -13,7 +13,7 @@ struct kmem_zone;
 
 struct xfs_attri_log_nameval {
 	struct xfs_log_iovec	name;
-	struct xfs_log_iovec	nname;
+	struct xfs_log_iovec	newname;
 	struct xfs_log_iovec	value;
 	refcount_t		refcount;
 
#24 lore raw

PATCH 15/17 xfs: log VLOOKUP xattr nvreplace operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If high level code wants to do a deferred xattr nvreplace operation with the VLOOKUP flag set, we need to push this through the log. To avoid breaking bisection in parent pointer code, we'll temporarily create a new NVREPLACEXXX flag that connects to the VLOOKUP flag.

---
 fs/xfs/libxfs/xfs_attr.c       |   11 +++++++----
 fs/xfs/libxfs/xfs_log_format.h |    1 +
 fs/xfs/xfs_attr_item.c         |   23 +++++++++++++++++++++--
 3 files changed, 29 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_attr.c +7 -4
index 313593b36e65..35d94608c621 100644
@@ -426,7 +426,8 @@ xfs_attr_complete_op(
 		return XFS_DAS_DONE;
 
 	args->attr_filter &= ~XFS_ATTR_INCOMPLETE;
-	if (xfs_attr_intent_op(attr) != XFS_ATTRI_OP_FLAGS_NVREPLACE)
+	if (xfs_attr_intent_op(attr) != XFS_ATTRI_OP_FLAGS_NVREPLACE &&
+	    xfs_attr_intent_op(attr) != XFS_ATTRI_OP_FLAGS_NVREPLACEXXX)
 		return replace_state;
 
 	/*
@@ -939,11 +940,13 @@ xfs_attr_defer_replace(
 	struct xfs_da_args	*args)
 {
 	struct xfs_attr_intent	*new;
-	int			op_flag;
+	int			op_flag = XFS_ATTRI_OP_FLAGS_REPLACE;
 	int			error = 0;
 
-	op_flag = args->new_namelen == 0 ? XFS_ATTRI_OP_FLAGS_REPLACE :
-		  XFS_ATTRI_OP_FLAGS_NVREPLACE;
+	if (args->op_flags & XFS_DA_OP_VLOOKUP)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVREPLACEXXX;
+	else if (args->new_namelen > 0)
+		op_flag = XFS_ATTRI_OP_FLAGS_NVREPLACE;
 
 	error = xfs_attr_intent_init(args, op_flag, &new);
 	if (error)
fs/xfs/libxfs/xfs_log_format.h +1 -0
index 8b16ae27c2fd..a1581dc6f131 100644
@@ -961,6 +961,7 @@ struct xfs_icreate_log {
 #define XFS_ATTRI_OP_FLAGS_NVREPLACE	4	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_NVREMOVE	5	/* Remove attr w/ vlookup */
 #define XFS_ATTRI_OP_FLAGS_NVSET	6	/* Set attr with w/ vlookup */
+#define XFS_ATTRI_OP_FLAGS_NVREPLACEXXX	7	/* Replace attr name and val */
 #define XFS_ATTRI_OP_FLAGS_TYPE_MASK	0xFF	/* Flags type mask */
 
 /*
fs/xfs/xfs_attr_item.c +21 -2
index b730f8e7a785..9eebc9db2280 100644
@@ -403,7 +403,10 @@ xfs_attr_log_item(
 	attrp->alfi_op_flags = attr->xattri_op_flags;
 	attrp->alfi_value_len = attr->xattri_nameval->value.i_len;
 
-	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
+	if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACEXXX) {
+		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
+		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
+	} else if (xfs_attr_log_item_op(attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		attrp->alfi_oldname_len = attr->xattri_nameval->name.i_len;
 		attrp->alfi_newname_len = attr->xattri_nameval->newname.i_len;
 	} else {
@@ -564,6 +567,7 @@ xfs_attri_validate(
 		if (attrp->alfi_value_len > XATTR_SIZE_MAX)
 			return false;
 		break;
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		if (attrp->alfi_oldname_len == 0 ||
 		    attrp->alfi_oldname_len > XATTR_NAME_MAX)
@@ -650,6 +654,7 @@ xfs_attri_item_recover(
 	ASSERT(xfs_sb_version_haslogxattrs(&mp->m_sb));
 
 	switch (xfs_attr_intent_op(attr)) {
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 	case XFS_ATTRI_OP_FLAGS_NVSET:
 		args->op_flags |= XFS_DA_OP_VLOOKUP;
 		fallthrough;
@@ -745,7 +750,10 @@ xfs_attri_item_relog(
 	new_attrp->alfi_ino = old_attrp->alfi_ino;
 	new_attrp->alfi_op_flags = old_attrp->alfi_op_flags;
 	new_attrp->alfi_value_len = old_attrp->alfi_value_len;
-	if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
+	if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACEXXX) {
+		new_attrp->alfi_newname_len = old_attrp->alfi_newname_len;
+		new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len;
+	} else if (xfs_attr_log_item_op(old_attrp) == XFS_ATTRI_OP_FLAGS_NVREPLACE) {
 		new_attrp->alfi_newname_len = old_attrp->alfi_newname_len;
 		new_attrp->alfi_oldname_len = old_attrp->alfi_oldname_len;
 	} else {
@@ -825,6 +833,16 @@ xlog_recover_attri_commit_pass2(
 		}
 		name_len = attri_formatp->alfi_name_len;
 		break;
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
+		/* Log item, attr name, new attr name, optional attr value */
+		if (item->ri_total < 3 || item->ri_total > 4) {
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp,
+					     attri_formatp, len);
+			return -EFSCORRUPTED;
+		}
+		name_len = attri_formatp->alfi_oldname_len;
+		newname_len = attri_formatp->alfi_newname_len;
+		break;
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
 		/* Log item, attr name, new attr name, attr value */
 		if (item->ri_total != 4) {
@@ -929,6 +947,7 @@ xlog_recover_attri_commit_pass2(
 		}
 		break;
 	case XFS_ATTRI_OP_FLAGS_NVREPLACE:
+	case XFS_ATTRI_OP_FLAGS_NVREPLACEXXX:
 		/*
 		 * Name-value replace operations require the caller to specify
 		 * the old and new name explicitly.  Values are optional.
#25 lore raw

PATCH 16/17 xfs: log new xattr values for NVREPLACEXXX operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

For NVREPLACEXXX operations, make it possible to log the old and new attr values, since this variant does VLOOKUP operations and we actually need both old and new values to be represented.

---
 fs/xfs/libxfs/xfs_attr.c       |    4 +-
 fs/xfs/libxfs/xfs_da_btree.h   |    2 +
 fs/xfs/libxfs/xfs_log_format.h |   14 +++++-
 fs/xfs/xfs_attr_item.c         |   88 +++++++++++++++++++++++++++++++++++-----
 fs/xfs/xfs_attr_item.h         |    3 +
 5 files changed, 93 insertions(+), 18 deletions(-)
fs/xfs/libxfs/xfs_attr.c +3 -1
index 35d94608c621..8022e1e1887f 100644
@@ -432,13 +432,15 @@ xfs_attr_complete_op(
 
 	/*
 	 * NVREPLACE operations require the caller to set the old and new names
-	 * explicitly.
+	 * and values explicitly.
 	 */
 	ASSERT(args->new_namelen > 0);
 
 	args->name = args->new_name;
 	args->namelen = args->new_namelen;
 	args->hashval = xfs_da_hashname(args->name, args->namelen);
+	args->value = args->new_value;
+	args->valuelen = args->new_valuelen;
 	return replace_state;
 }
 
fs/xfs/libxfs/xfs_da_btree.h +2 -0
index 6132787ad1f4..3f3313ace903 100644
@@ -60,7 +60,9 @@ typedef struct xfs_da_args {
 	int		new_namelen;	/* new attr name len */
 	uint8_t		filetype;	/* filetype of inode for directories */
 	void		*value;		/* set of bytes (maybe contain NULLs) */
+	void		*new_value;	/* new xattr value (may contain NULLs) */
 	int		valuelen;	/* length of value */
+	int		new_valuelen;	/* length of new value */
 	unsigned int	attr_filter;	/* XFS_ATTR_{ROOT,SECURE,INCOMPLETE} */
 	unsigned int	attr_flags;	/* XATTR_{CREATE,REPLACE} */
 	xfs_dahash_t	hashval;	/* hash value of name */
fs/xfs/libxfs/xfs_log_format.h +10 -4
index a1581dc6f131..711654fca49a 100644
@@ -115,11 +115,11 @@ struct xfs_unmount_log_format {
 #define XLOG_REG_TYPE_BUD_FORMAT	26
 #define XLOG_REG_TYPE_ATTRI_FORMAT	27
 #define XLOG_REG_TYPE_ATTRD_FORMAT	28
-#define XLOG_REG_TYPE_ATTR_NAME	29
+#define XLOG_REG_TYPE_ATTR_NAME		29
 #define XLOG_REG_TYPE_ATTR_VALUE	30
 #define XLOG_REG_TYPE_ATTR_NEWNAME	31
-#define XLOG_REG_TYPE_MAX		31
-
+#define XLOG_REG_TYPE_ATTR_NEWVALUE	32
+#define XLOG_REG_TYPE_MAX		32
 
 /*
  * Flags to log operation header
@@ -980,7 +980,13 @@ struct xfs_icreate_log {
 struct xfs_attri_log_format {
 	uint16_t	alfi_type;	/* attri log item type */
 	uint16_t	alfi_size;	/* size of this item */
-	uint32_t	__pad;		/* pad to 64 bit aligned */
+
+	/*
+	 * For NVREPLACEXXX, this is the length of the new xattr value.
+	 * alfi_value_len contains the length of the old xattr value.
+	 */
+	uint32_t	alfi_newvalue_len;
+
 	uint64_t	alfi_id;	/* attri identifier */
 	uint64_t	alfi_ino;	/* the inode for this attr operation */
 	uint32_t	alfi_op_flags;	/* marks the op as a set or remove */
fs/xfs/xfs_attr_item.c +76 -12

239 lines not on this page · raw message

fs/xfs/xfs_attr_item.h +2 -1
index e374712ba06b..098f7f3aae78 100644
@@ -13,8 +13,9 @@ struct kmem_zone;
 
 struct xfs_attri_log_nameval {
 	struct xfs_log_iovec	name;
-	struct xfs_log_iovec	newname;
+	struct xfs_log_iovec	newname;	/* NVREPLACEXXX only */
 	struct xfs_log_iovec	value;
+	struct xfs_log_iovec	newvalue;	/* NVREPLACEXXX only */
 	refcount_t		refcount;
 
 	/* name and value follow the end of this struct */
#26 lore raw

PATCH 17/17 xfs: replace parent pointer diroffset with full dirent name

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. This is difficult to guarantee in the process of performing a repair and in the long terms is totally pointless because the the parent pointer code doesn't use the diroffset, nor does the online fsck code. IOWs, this design decision increases the complexity of repair solely to work around the xattr code being unable to store a full dirent name in the xattr name or match on xattr values.

However, xattrs can now perform lookups on both the name and value of an xattr. This means we can redefine the parent pointer format like this:

For dirent names shorter than 243 bytes:

    (parent_ino, parent_gen, dirent_name) -> NULL

For dirent names longer than 243 bytes:

    (parent_ino, parent_gen, dirent_name[0:242]) -> (dirent_name[243:255])

and this works because a parent pointer lookup uses NVLOOKUP, which will read those last twelve bytes of the dirent name if need be.

Replace the diroffset with this new format, thereby eliminating the need for directory repair to update all the parent pointers after rebuilding the directory. We haven't merged parent pointers, so changes to the ondisk format are easy.

---
 fs/xfs/libxfs/xfs_da_format.h  |   39 ++++++-
 fs/xfs/libxfs/xfs_fs.h         |    2 
 fs/xfs/libxfs/xfs_parent.c     |  216 +++++++++++++++++++++++++---------------
 fs/xfs/libxfs/xfs_parent.h     |   46 +++++----
 fs/xfs/libxfs/xfs_trans_resv.c |    7 +
 fs/xfs/scrub/dir.c             |   34 +-----
 fs/xfs/scrub/dir_repair.c      |   87 +++++-----------
 fs/xfs/scrub/parent.c          |   48 ++-------
 fs/xfs/scrub/parent_repair.c   |   55 +++-------
 fs/xfs/scrub/trace.h           |   65 ++----------
 fs/xfs/xfs_inode.c             |   30 ++----
 fs/xfs/xfs_ondisk.h            |    1 
 fs/xfs/xfs_parent_utils.c      |    8 +
 fs/xfs/xfs_symlink.c           |    3 -
 14 files changed, 291 insertions(+), 350 deletions(-)
fs/xfs/libxfs/xfs_da_format.h +34 -5
index c07b8166e8ff..dd569286b3be 100644
@@ -827,14 +827,43 @@ xfs_failaddr_t xfs_da3_blkinfo_verify(struct xfs_buf *bp,
 /*
  * Parent pointer attribute format definition
  *
- * EA name encodes the parent inode number, generation and the offset of
- * the dirent that points to the child inode. The EA value contains the
- * same name as the dirent in the parent directory.
+ * The EA name encodes the parent inode number, generation and as much of the
+ * dirent name as fits.  In other words, it contains up to 243 bytes of the
+ * dirent name.
+ *
+ * The EA value contains however much of the dirent name that does not fit in
+ * the EA name.
  */
 struct xfs_parent_name_rec {
 	__be64  p_ino;
 	__be32  p_gen;
-	__be32  p_diroffset;
-};
+	__u8	p_dname[];
+} __attribute__((packed));
+
+/* Maximum size of a parent pointer EA name. */
+#define XFS_PARENT_NAME_MAX_SIZE \
+	(MAXNAMELEN - 1)
+
+/* Maximum number of dirent name bytes stored in p_dname. */
+#define XFS_PARENT_MAX_DNAME_SIZE \
+	(XFS_PARENT_NAME_MAX_SIZE - sizeof(struct xfs_parent_name_rec))
+
+/* Maximum number of dirent name bytes stored in the xattr value. */
+#define XFS_PARENT_MAX_DNAME_VALUELEN \
+	sizeof(struct xfs_parent_name_rec)
+
+static inline unsigned int
+xfs_parent_name_rec_sizeof(
+	unsigned int		dnamelen)
+{
+	return sizeof(struct xfs_parent_name_rec) + dnamelen;
+}
+
+static inline unsigned int
+xfs_parent_name_dnamelen(
+	unsigned int		rec_sizeof)
+{
+	return rec_sizeof - sizeof(struct xfs_parent_name_rec);
+}
 
 #endif /* __XFS_DA_FORMAT_H__ */
fs/xfs/libxfs/xfs_fs.h +1 -1
index d7e061089e75..d8619479ef4a 100644
@@ -769,7 +769,7 @@ struct xfs_scrub_metadata {
 struct xfs_getparents_rec {
 	__u64		gpr_ino;	/* Inode number */
 	__u32		gpr_gen;	/* Inode generation */
-	__u32		gpr_diroffset;	/* Directory offset */
+	__u32		gpr_rsvd2;	/* Reserved */
 	__u64		gpr_rsvd;	/* Reserved */
 	__u8		gpr_name[];	/* File name and null terminator */
 };
fs/xfs/libxfs/xfs_parent.c +136 -80

385 lines not on this page · raw message

fs/xfs/libxfs/xfs_parent.h +26 -20

88 lines not on this page · raw message

fs/xfs/libxfs/xfs_trans_resv.c +4 -3
index f5d5cb48f78a..3ad1bc57a107 100644
@@ -427,19 +427,20 @@ static inline unsigned int xfs_calc_pptr_link_overhead(void)
 {
 	return sizeof(struct xfs_attri_log_format) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
-			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 static inline unsigned int xfs_calc_pptr_unlink_overhead(void)
 {
 	return sizeof(struct xfs_attri_log_format) +
-			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 static inline unsigned int xfs_calc_pptr_replace_overhead(void)
 {
 	return sizeof(struct xfs_attri_log_format) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
 			xlog_calc_iovec_len(XATTR_NAME_MAX) +
-			xlog_calc_iovec_len(sizeof(struct xfs_parent_name_rec));
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE) +
+			xlog_calc_iovec_len(XFS_PARENT_NAME_MAX_SIZE);
 }
 
 /*
fs/xfs/scrub/dir.c +8 -26
index 3f3223e563ae..23cb7519c8f0 100644
@@ -78,7 +78,7 @@ struct xchk_dir {
 	/* If we've cycled the ILOCK, we must revalidate deferred dirents. */
 	bool			need_revalidate;
 
-	/* Name buffer for pptr validation and dirent revalidation. */
+	/* Name buffer for dirent revalidation. */
 	uint8_t			namebuf[MAXNAMELEN];
 
 };
@@ -139,38 +139,20 @@ xchk_dir_lock_child(
 STATIC int
 xchk_dir_parent_pointer(
 	struct xchk_dir		*sd,
-	xfs_dir2_dataptr_t	dapos,
 	const struct xfs_name	*name,
 	struct xfs_inode	*ip)
 {
 	struct xfs_scrub	*sc = sd->sc;
-	int			pptr_namelen;
+	int			error;
 
 	sd->pptr.p_ino = sc->ip->i_ino;
 	sd->pptr.p_gen = VFS_I(sc->ip)->i_generation;
-	sd->pptr.p_diroffset = dapos;
+	sd->pptr.p_namelen = name->len;
+	memcpy(sd->pptr.p_name, name->name, name->len);
 
-	pptr_namelen = xfs_parent_lookup(sc->tp, ip, &sd->pptr, sd->namebuf,
-			MAXNAMELEN, &sd->pptr_scratch);
-	if (pptr_namelen == -ENOATTR) {
+	error = xfs_parent_lookup(sc->tp, ip, &sd->pptr, &sd->pptr_scratch);
+	if (error == -ENOATTR)
 		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
-		return 0;
-	}
-	if (pptr_namelen < 0) {
-		xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0,
-				&pptr_namelen);
-		return pptr_namelen;
-	}
-
-	if (pptr_namelen != name->len) {
-		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
-		return 0;
-	}
-
-	if (memcmp(sd->namebuf, name->name, name->len)) {
-		xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
-		return 0;
-	}
 
 	return 0;
 }
@@ -216,7 +198,7 @@ xchk_dir_check_pptr_fast(
 		return 0;
 	}
 
-	error = xchk_dir_parent_pointer(sd, dapos, name, ip);
+	error = xchk_dir_parent_pointer(sd, name, ip);
 	xfs_iunlock(ip, lockmode);
 	return error;
 }
@@ -1041,7 +1023,7 @@ xchk_dir_slow_dirent(
 		goto out_unlock;
 
 check_pptr:
-	error = xchk_dir_parent_pointer(sd, dirent->diroffset, &xname, ip);
+	error = xchk_dir_parent_pointer(sd, &xname, ip);
 out_unlock:
 	xfs_iunlock(ip, lockmode);
 out_rele:
fs/xfs/scrub/dir_repair.c +27 -60

255 lines not on this page · raw message

fs/xfs/scrub/parent.c +10 -38

116 lines not on this page · raw message

fs/xfs/scrub/parent_repair.c +15 -40

140 lines not on this page · raw message

fs/xfs/scrub/trace.h +11 -54

153 lines not on this page · raw message

fs/xfs/xfs_inode.c +12 -18
index c90b0a67c2b3..144955377d99 100644
@@ -1202,8 +1202,7 @@ xfs_create(
 	 * the parent information now.
 	 */
 	if (parent) {
-		error = xfs_parent_add(tp, parent, dp, name, diroffset,
-					     ip);
+		error = xfs_parent_add(tp, parent, dp, name, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
@@ -1477,8 +1476,7 @@ xfs_link(
 	 * the parent to the inode.
 	 */
 	if (parent) {
-		error = xfs_parent_add(tp, parent, tdp, target_name,
-					     diroffset, sip);
+		error = xfs_parent_add(tp, parent, tdp, target_name, sip);
 		if (error)
 			goto error_return;
 	}
@@ -2750,7 +2748,7 @@ xfs_remove(
 	}
 
 	if (parent) {
-		error = xfs_parent_remove(tp, dp, parent, dir_offset, ip);
+		error = xfs_parent_remove(tp, parent, dp, name, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
@@ -3061,13 +3059,13 @@ xfs_cross_rename(
 	}
 
 	if (xfs_has_parent(mp)) {
-		error = xfs_parent_replace(tp, ip1_pptr, dp1,
-				old_diroffset, name2, dp2, new_diroffset, ip1);
+		error = xfs_parent_replace(tp, ip1_pptr, dp1, name1, dp2,
+				name2, ip1);
 		if (error)
 			goto out_trans_abort;
 
-		error = xfs_parent_replace(tp, ip2_pptr, dp2,
-				new_diroffset, name1, dp1, old_diroffset, ip2);
+		error = xfs_parent_replace(tp, ip2_pptr, dp2, name2, dp1,
+				name1, ip2);
 		if (error)
 			goto out_trans_abort;
 	}
@@ -3540,25 +3538,21 @@ xfs_rename(
 		goto out_trans_cancel;
 
 	if (wip_pptr) {
-		error = xfs_parent_add(tp, wip_pptr,
-					     src_dp, src_name,
-					     old_diroffset, wip);
+		error = xfs_parent_add(tp, wip_pptr, src_dp, src_name, wip);
 		if (error)
 			goto out_trans_cancel;
 	}
 
 	if (src_ip_pptr) {
-		error = xfs_parent_replace(tp, src_ip_pptr, src_dp,
-				old_diroffset, target_name, target_dp,
-				new_diroffset, src_ip);
+		error = xfs_parent_replace(tp, src_ip_pptr, src_dp, src_name,
+				target_dp, target_name, src_ip);
 		if (error)
 			goto out_trans_cancel;
 	}
 
 	if (tgt_ip_pptr) {
-		error = xfs_parent_remove(tp, target_dp,
-						tgt_ip_pptr,
-						new_diroffset, target_ip);
+		error = xfs_parent_remove(tp, tgt_ip_pptr, target_dp,
+				target_name, target_ip);
 		if (error)
 			goto out_trans_cancel;
 	}
fs/xfs/xfs_ondisk.h +1 -0
index 88f9ec393c3d..aa07858a6a6d 100644
@@ -114,6 +114,7 @@ xfs_check_ondisk_structs(void)
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, offset,		1);
 	XFS_CHECK_OFFSET(xfs_dir2_sf_entry_t, name,		3);
 	XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_hdr_t,		10);
+	XFS_CHECK_STRUCT_SIZE(struct xfs_parent_name_rec,	12);
 
 	/* log structures */
 	XFS_CHECK_STRUCT_SIZE(struct xfs_buf_log_format,	88);
fs/xfs/xfs_parent_utils.c +5 -3
index 5a4f72cd5711..0c1dcd726472 100644
@@ -67,12 +67,14 @@ xfs_getparent_listent(
 	 * pointer.  The attr list functions filtered out INCOMPLETE attrs.
 	 */
 	if (XFS_IS_CORRUPT(mp, !xfs_parent_namecheck(mp, rec, namelen, flags)) ||
-	    XFS_IS_CORRUPT(mp, !xfs_parent_valuecheck(mp, value, valuelen))) {
+	    XFS_IS_CORRUPT(mp, !xfs_parent_valuecheck(mp, namelen, value,
+						      valuelen))) {
 		context->seen_enough = -EFSCORRUPTED;
 		return;
 	}
 
-	xfs_parent_irec_from_disk(&gp->pptr_irec, rec, value, valuelen);
+	xfs_parent_irec_from_disk(&gp->pptr_irec, rec, namelen, value,
+			valuelen);
 
 	/*
 	 * We found a parent pointer, but we've filled up the buffer.  Signal
@@ -93,7 +95,7 @@ xfs_getparent_listent(
 	pptr = xfs_getparents_rec(ppi, ppi->gp_count);
 	pptr->gpr_ino = irec->p_ino;
 	pptr->gpr_gen = irec->p_gen;
-	pptr->gpr_diroffset = irec->p_diroffset;
+	pptr->gpr_rsvd2 = 0;
 	pptr->gpr_rsvd = 0;
 
 	memcpy(pptr->gpr_name, irec->p_name, irec->p_namelen);
fs/xfs/xfs_symlink.c +1 -2
index 7fc58bee231d..e6995a44cce2 100644
@@ -348,8 +348,7 @@ xfs_symlink(
 	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	if (parent) {
-		error = xfs_parent_add(tp, parent, dp, link_name,
-					     diroffset, ip);
+		error = xfs_parent_add(tp, parent, dp, link_name, ip);
 		if (error)
 			goto out_trans_cancel;
 	}
#27 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

In reply to Darrick J. Wong
Quoted 84 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair.  This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent.  There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset.  This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary.  This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure.  If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging.  This breaks repair's policy that repairs should commit atomically.  It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code.  Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code.  Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations.  We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

The ondisk format of a parent pointer can then become:

    (parent_ino, parent_gen, dirent_name[0:242]) -> (dirent_name[243:255])

Because we can always look up a specific parent pointer.  Most of the patches in this patchset prepare the high level xattr code and the lower
level logging code to do this correctly, and the last patch switches the
ondisk format.

Quoted 6 lines

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy! Comments and questions are, as always, welcome. kernel git tree:

So I gave this set a look over and for the most part I think it's ok as long as folks agree on the new format? It's a lot different from what folks originally articulated that they wanted, but for the most part the format change doesnt affect parent pointer mechanics as much as it will affect other features that may use it. I havnt seen much complaints in response, so if it's all the same to everyone else, I am fine with moving forward with it? Thanks all!

Allison

Quoted 36 lines

https://urldefense.com/v3/__https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!NDopQXFVRjDfqgJvGMQRzS3fXyyd1SC4HfGl4dxSMWxNPsNKScecjp7bZYX-dDhL5tatk_xxJFdQwCj9YNGq$

xfsprogs git tree:
https://urldefense.com/v3/__https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!NDopQXFVRjDfqgJvGMQRzS3fXyyd1SC4HfGl4dxSMWxNPsNKScecjp7bZYX-dDhL5tatk_xxJFdQwILBe7jj$

fstests git tree:
https://urldefense.com/v3/__https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!NDopQXFVRjDfqgJvGMQRzS3fXyyd1SC4HfGl4dxSMWxNPsNKScecjp7bZYX-dDhL5tatk_xxJFdQwI5iKKgk$

---
 fs/xfs/libxfs/xfs_attr.c       |   66 +++++--- fs/xfs/libxfs/xfs_attr.h       |    5 + fs/xfs/libxfs/xfs_attr_leaf.c  |   41 ++++- fs/xfs/libxfs/xfs_da_btree.h   |    6 + fs/xfs/libxfs/xfs_da_format.h  |   39 ++++- fs/xfs/libxfs/xfs_fs.h         |    2 fs/xfs/libxfs/xfs_log_format.h |   31 +++- fs/xfs/libxfs/xfs_parent.c     |  215 +++++++++++++++++---------- fs/xfs/libxfs/xfs_parent.h     |   46 +++--- fs/xfs/libxfs/xfs_trans_resv.c |    7 + fs/xfs/scrub/dir.c             |   34 +--- fs/xfs/scrub/dir_repair.c      |   87 +++-------- fs/xfs/scrub/parent.c          |   48 +----- fs/xfs/scrub/parent_repair.c   |   55 ++----- fs/xfs/scrub/trace.h           |   65 +------- fs/xfs/xfs_attr_item.c         |  318
+++++++++++++++++++++++++++++++---------
 fs/xfs/xfs_attr_item.h         |    3 fs/xfs/xfs_inode.c             |   30 ++-- fs/xfs/xfs_ondisk.h            |    1 fs/xfs/xfs_parent_utils.c      |    8 + fs/xfs/xfs_symlink.c           |    3 fs/xfs/xfs_xattr.c             |    5 +

 22 files changed, 660 insertions(+), 455 deletions(-)
#28 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 74 lines · Allison Henderson
Quoted 72 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset. This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure. If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code. Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations. We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

Quoted 4 lines
Quoted 4 lines

The ondisk format of a parent pointer can then become:

    (parent_ino, parent_gen, dirent_name[0:242]) ->

(dirent_name[243:255])

With VLOOKUP in place, why is this better than
(parent_ino, parent_gen) -> (dirent_name)

Is it because the dabtree hash is calculated only on the key and changing that would be way more intrusive?

Does that mean that user can create up to 2^12 parent pointers with the same hash and we are fine with it?

I don't think it is a problem, just wanted to understand and ask that the reason for this part in the format be explained.

Thanks,
Amir.

#29 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 78 lines · Amir Goldstein
Quoted 76 lines · Allison Henderson
Quoted 74 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset. This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure. If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code. Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations. We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

The ondisk format of a parent pointer can then become:

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
    (parent_ino, parent_gen, dirent_name[0:242]) ->

(dirent_name[243:255])

With VLOOKUP in place, why is this better than
(parent_ino, parent_gen) -> (dirent_name)

Is it because the dabtree hash is calculated only on the key and changing that would be way more intrusive?

Yes. The dabtree hash is calculated on the full attr name, so this is an attempt to reduce hash collisions during parent pointer lookups by stuffing as many bytes in the attr name as possible.

It would be very easy to change the xfs_da_hashname calls inside xfs_parent.c to use either the full dirent name (i.e. pptr hash matches dirent hash) or use the full parent pointer and not just the attr key, but that would be a major break from the tradition that the da hash is computed against the attr name only.

(I'm not opposed to doing that too, but one breaking change at a time. ;))

Quoted 2 lines

Does that mean that user can create up to 2^12 parent pointers with the same hash and we are fine with it?

Yes. The dabtree can handle an xattr structure where every name hashes to the same value, though performance will be slow as it scans every attr to find the one it wants.

The number of parent pointers with the same hash can be much higher than 2^12 -- I wrote a dumb C program that creates an arbitrary number of attr names with identical hashes. It's not fast when you get past 100,000. :P

Quoted 2 lines

I don't think it is a problem, just wanted to understand and ask that the reason for this part in the format be explained.

NP. Thanks for asking!

--D

Quoted 2 lines

Thanks,
Amir.

#30 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 105 lines · Darrick J. Wong
Quoted 78 lines · Amir Goldstein
Quoted 76 lines · Allison Henderson
Quoted 74 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset. This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure. If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code. Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations. We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

The ondisk format of a parent pointer can then become:

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
    (parent_ino, parent_gen, dirent_name[0:242]) ->

(dirent_name[243:255])

With VLOOKUP in place, why is this better than
(parent_ino, parent_gen) -> (dirent_name)

Is it because the dabtree hash is calculated only on the key and changing that would be way more intrusive?

Yes. The dabtree hash is calculated on the full attr name, so this is an attempt to reduce hash collisions during parent pointer lookups by stuffing as many bytes in the attr name as possible.

It would be very easy to change the xfs_da_hashname calls inside xfs_parent.c to use either the full dirent name (i.e. pptr hash matches dirent hash) or use the full parent pointer and not just the attr key, but that would be a major break from the tradition that the da hash is computed against the attr name only.

(I'm not opposed to doing that too, but one breaking change at a time. ;))

Quoted 2 lines

Does that mean that user can create up to 2^12 parent pointers with the same hash and we are fine with it?

Quoted 9 lines

Yes. The dabtree can handle an xattr structure where every name hashes to the same value, though performance will be slow as it scans every attr to find the one it wants.

The number of parent pointers with the same hash can be much higher than 2^12 -- I wrote a dumb C program that creates an arbitrary number of attr names with identical hashes. It's not fast when you get past 100,000. :P

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

This is not a breaking change and you won't need to do another breaking change later.

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

Thanks,
Amir.

#31 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 120 lines · Amir Goldstein
Quoted 105 lines · Darrick J. Wong
Quoted 78 lines · Amir Goldstein
Quoted 76 lines · Allison Henderson
Quoted 74 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset. This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure. If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code. Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations. We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

The ondisk format of a parent pointer can then become:

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
    (parent_ino, parent_gen, dirent_name[0:242]) ->

(dirent_name[243:255])

With VLOOKUP in place, why is this better than
(parent_ino, parent_gen) -> (dirent_name)

Is it because the dabtree hash is calculated only on the key and changing that would be way more intrusive?

Yes. The dabtree hash is calculated on the full attr name, so this is an attempt to reduce hash collisions during parent pointer lookups by stuffing as many bytes in the attr name as possible.

It would be very easy to change the xfs_da_hashname calls inside xfs_parent.c to use either the full dirent name (i.e. pptr hash matches dirent hash) or use the full parent pointer and not just the attr key, but that would be a major break from the tradition that the da hash is computed against the attr name only.

(I'm not opposed to doing that too, but one breaking change at a time. ;))

Quoted 2 lines

Does that mean that user can create up to 2^12 parent pointers with the same hash and we are fine with it?

Quoted 9 lines

Yes. The dabtree can handle an xattr structure where every name hashes to the same value, though performance will be slow as it scans every attr to find the one it wants.

The number of parent pointers with the same hash can be much higher than 2^12 -- I wrote a dumb C program that creates an arbitrary number of attr names with identical hashes. It's not fast when you get past 100,000. :P

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

Quoted 5 lines

This is not a breaking change and you won't need to do another breaking change later.

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

Hopefully the fact that the attr name starts with 12 bytes (4 of which aren't known to unprivileged userspace) and omits the last 12 bytes of the dirent name will make it harder to generate double-collisions. Granted there's probably someone who's more math-smart than me who can figure this out.

--D

Quoted 2 lines

Thanks,
Amir.

#32 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 140 lines · Darrick J. Wong
Quoted 120 lines · Amir Goldstein
Quoted 105 lines · Darrick J. Wong
Quoted 78 lines · Amir Goldstein
Quoted 76 lines · Allison Henderson
Quoted 74 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset. This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure. If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code. Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations. We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

The ondisk format of a parent pointer can then become:

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
    (parent_ino, parent_gen, dirent_name[0:242]) ->

(dirent_name[243:255])

With VLOOKUP in place, why is this better than
(parent_ino, parent_gen) -> (dirent_name)

Is it because the dabtree hash is calculated only on the key and changing that would be way more intrusive?

Yes. The dabtree hash is calculated on the full attr name, so this is an attempt to reduce hash collisions during parent pointer lookups by stuffing as many bytes in the attr name as possible.

It would be very easy to change the xfs_da_hashname calls inside xfs_parent.c to use either the full dirent name (i.e. pptr hash matches dirent hash) or use the full parent pointer and not just the attr key, but that would be a major break from the tradition that the da hash is computed against the attr name only.

(I'm not opposed to doing that too, but one breaking change at a time. ;))

Quoted 2 lines

Does that mean that user can create up to 2^12 parent pointers with the same hash and we are fine with it?

Quoted 9 lines

Yes. The dabtree can handle an xattr structure where every name hashes to the same value, though performance will be slow as it scans every attr to find the one it wants.

The number of parent pointers with the same hash can be much higher than 2^12 -- I wrote a dumb C program that creates an arbitrary number of attr names with identical hashes. It's not fast when you get past 100,000. :P

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

Quoted 5 lines

This is not a breaking change and you won't need to do another breaking change later.

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

Quoted 7 lines

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

OK, but this attack is applicable to the dabtree hash itself only with the proposed dirent_name[0:242] key, is it not?

Quoted 5 lines

Hopefully the fact that the attr name starts with 12 bytes (4 of which aren't known to unprivileged userspace) and omits the last 12 bytes of the dirent name will make it harder to generate double-collisions. Granted there's probably someone who's more math-smart than me who can figure this out.

My concern is that generating 7^12 collisions using all the possible printable suffix dirent_name[243:255] is so simple that even a non-malicious but rather unlucky script would end up hitting those collisions:
Report_$ORGANIZATION_$DEPARTEMT_$PRODUCT_$PROJECT_... $GROUP_$OWNER_$DATE_$TIME_$ID.pdf
You get what I mean...

So my thinking was that the unlucky should not suffer the same penalty as the malicious.

Thanks,
Amir.

#33 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

In reply to Darrick J. Wong
Quoted 12 lines · Darrick J. Wong
Quoted 11 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Amir Goldstein

On Fri, Mar 24, 2023 at 8:19 PM Allison Henderson
<[email protected]> wrote:

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

This is not a breaking change and you won't need to do another breaking change later.

Quoted 7 lines
Quoted 2 lines

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

Right, that's because hash collision resolution is very dumb. It's just a linear walk from start to end comparing the names of each coliding entry.

Quoted 6 lines

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

That's worst case behaviour, not real-world production behaviour. It's also a dabtree optimisation problem, not a parent pointer problem. i.e. if this is a real world issue, we need to fix the dabtree indexing, not work around it in the parent pointer format.

IOWs, optimising the parent pointer structure for the *rare* corner case of an intentionally crafted dahash btree collision farm is, to me, the wrong way to design a structure.

Optimising it for the common fast path (single parent xattrs, followed by small numbers of parents) is what we should be doing, because that situation will occur an uncountable number of times more frequently than this theorectical worst case behaviour. i.e. the worst case will, in all likelihood, never happen in production environments - behaviour in this circumstance is only a consideration for mitigating malicious attacks.

And, quite frankly, nobody is going to bother attacking a filesystem this way - there is literally nothing that can be gained from slowing down access to a single directory in this manner. It's a DOS at best, but there are *much* easier ways of doing that and it doesn't require parent pointers or dahash collisions at all.

Quoted 7 lines

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

So why not change the dahash for parent pointer enabled filesystems to use crc32c everywhere?

I'd much prefer we have a parent pointer index key that is fixed length, has constant hash time and a dabtree hash that is far more resilient to collision farming than to have to perform encoding/decoding of varaible length values into the key to work around a collision-rich hashing algorithm.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#34 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 64 lines · Dave Chinner
Quoted 12 lines · Darrick J. Wong
Quoted 11 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Amir Goldstein

On Fri, Mar 24, 2023 at 8:19 PM Allison Henderson
<[email protected]> wrote:

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

This is not a breaking change and you won't need to do another breaking change later.

Quoted 7 lines
Quoted 2 lines

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

Right, that's because hash collision resolution is very dumb. It's just a linear walk from start to end comparing the names of each coliding entry.

Quoted 6 lines

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

That's worst case behaviour, not real-world production behaviour. It's also a dabtree optimisation problem, not a parent pointer problem. i.e. if this is a real world issue, we need to fix the dabtree indexing, not work around it in the parent pointer format.

IOWs, optimising the parent pointer structure for the *rare* corner case of an intentionally crafted dahash btree collision farm is, to me, the wrong way to design a structure.

Optimising it for the common fast path (single parent xattrs, followed by small numbers of parents) is what we should be doing, because that situation will occur an uncountable number of times more frequently than this theorectical worst case behaviour. i.e. the worst case will, in all likelihood, never happen in production environments - behaviour in this circumstance is only a consideration for mitigating malicious attacks.

And, quite frankly, nobody is going to bother attacking a filesystem this way - there is literally nothing that can be gained from slowing down access to a single directory in this manner. It's a DOS at best, but there are *much* easier ways of doing that and it doesn't require parent pointers or dahash collisions at all.

Quoted 7 lines

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

Quoted 2 lines

So why not change the dahash for parent pointer enabled filesystems to use crc32c everywhere?

That's not difficult to do, but it'll break name obscuring in metadump unless you know of a quick way to derive B from A and maintain crc32c(A) == crc32c(B).

Granted I had mostly written off name obscuring in metadump on parent pointer filesystems anyway.

Quoted 5 lines

I'd much prefer we have a parent pointer index key that is fixed length, has constant hash time and a dabtree hash that is far more resilient to collision farming than to have to perform encoding/decoding of varaible length values into the key to work around a collision-rich hashing algorithm.

Wellllll I was 10 minutes away from sending v11 with all of my changes integrated, but I guess now I'll go rework the dabtree to use crc32c, encode the crc32c in the parent pointer attr name, and get rid of all the variable length value decoding. Now that I just got rid of all the intermediate patches with hash-in-attr-name.

Let's see if I can hammer that out in the next 15 minutes.

--D

Quoted 6 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#35 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 65 lines · Darrick J. Wong
Quoted 64 lines · Dave Chinner
Quoted 12 lines · Darrick J. Wong
Quoted 11 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Amir Goldstein

On Fri, Mar 24, 2023 at 8:19 PM Allison Henderson
<[email protected]> wrote:

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

This is not a breaking change and you won't need to do another breaking change later.

Quoted 7 lines
Quoted 2 lines

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

Right, that's because hash collision resolution is very dumb. It's just a linear walk from start to end comparing the names of each coliding entry.

Quoted 6 lines

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

That's worst case behaviour, not real-world production behaviour. It's also a dabtree optimisation problem, not a parent pointer problem. i.e. if this is a real world issue, we need to fix the dabtree indexing, not work around it in the parent pointer format.

IOWs, optimising the parent pointer structure for the *rare* corner case of an intentionally crafted dahash btree collision farm is, to me, the wrong way to design a structure.

Optimising it for the common fast path (single parent xattrs, followed by small numbers of parents) is what we should be doing, because that situation will occur an uncountable number of times more frequently than this theorectical worst case behaviour. i.e. the worst case will, in all likelihood, never happen in production environments - behaviour in this circumstance is only a consideration for mitigating malicious attacks.

And, quite frankly, nobody is going to bother attacking a filesystem this way - there is literally nothing that can be gained from slowing down access to a single directory in this manner. It's a DOS at best, but there are *much* easier ways of doing that and it doesn't require parent pointers or dahash collisions at all.

Quoted 7 lines

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

Quoted 6 lines
Quoted 2 lines

So why not change the dahash for parent pointer enabled filesystems to use crc32c everywhere?

That's not difficult to do, but it'll break name obscuring in metadump unless you know of a quick way to derive B from A and maintain crc32c(A) == crc32c(B).

Not easily, but....

Quoted 2 lines

Granted I had mostly written off name obscuring in metadump on parent pointer filesystems anyway.

... so had I. The name obfuscation needs a completely different approach for PP enabled filesystems because of the name also being embedded in non-user visible structures and potentially changing attr tree dahash-based indexing.

Quoted 11 lines
Quoted 5 lines

I'd much prefer we have a parent pointer index key that is fixed length, has constant hash time and a dabtree hash that is far more resilient to collision farming than to have to perform encoding/decoding of varaible length values into the key to work around a collision-rich hashing algorithm.

Wellllll I was 10 minutes away from sending v11 with all of my changes integrated, but I guess now I'll go rework the dabtree to use crc32c, encode the crc32c in the parent pointer attr name, and get rid of all the variable length value decoding. Now that I just got rid of all the intermediate patches with hash-in-attr-name.

I apologise for not being able to comment on this earlier. What time I've had available over the past coule of weeks has been focussed on the parts of the repair patchset that are ready for merge and I've had little time to follow the progress of the parts you are currently developing.

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#36 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 75 lines · Dave Chinner
Quoted 65 lines · Darrick J. Wong
Quoted 64 lines · Dave Chinner
Quoted 12 lines · Darrick J. Wong
Quoted 11 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Amir Goldstein

On Fri, Mar 24, 2023 at 8:19 PM Allison Henderson
<[email protected]> wrote:

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

This is not a breaking change and you won't need to do another breaking change later.

Quoted 7 lines
Quoted 2 lines

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

Right, that's because hash collision resolution is very dumb. It's just a linear walk from start to end comparing the names of each coliding entry.

Quoted 6 lines

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

That's worst case behaviour, not real-world production behaviour. It's also a dabtree optimisation problem, not a parent pointer problem. i.e. if this is a real world issue, we need to fix the dabtree indexing, not work around it in the parent pointer format.

IOWs, optimising the parent pointer structure for the *rare* corner case of an intentionally crafted dahash btree collision farm is, to me, the wrong way to design a structure.

Optimising it for the common fast path (single parent xattrs, followed by small numbers of parents) is what we should be doing, because that situation will occur an uncountable number of times more frequently than this theorectical worst case behaviour. i.e. the worst case will, in all likelihood, never happen in production environments - behaviour in this circumstance is only a consideration for mitigating malicious attacks.

And, quite frankly, nobody is going to bother attacking a filesystem this way - there is literally nothing that can be gained from slowing down access to a single directory in this manner. It's a DOS at best, but there are *much* easier ways of doing that and it doesn't require parent pointers or dahash collisions at all.

Quoted 7 lines

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

Quoted 6 lines
Quoted 2 lines

So why not change the dahash for parent pointer enabled filesystems to use crc32c everywhere?

That's not difficult to do, but it'll break name obscuring in metadump unless you know of a quick way to derive B from A and maintain crc32c(A) == crc32c(B).

Not easily, but....

Quoted 7 lines
Quoted 2 lines

Granted I had mostly written off name obscuring in metadump on parent pointer filesystems anyway.

... so had I. The name obfuscation needs a completely different approach for PP enabled filesystems because of the name also being embedded in non-user visible structures and potentially changing attr tree dahash-based indexing.

<nod> I've wondered if we just have to record (dir_ino/name) -> (newname) and use that to update the parent pointer and any dabtree entries we find. That's going to be a bit of a mess to iron out since we can't change the "mounted" filesystem.

Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

I'd much prefer we have a parent pointer index key that is fixed length, has constant hash time and a dabtree hash that is far more resilient to collision farming than to have to perform encoding/decoding of varaible length values into the key to work around a collision-rich hashing algorithm.

Wellllll I was 10 minutes away from sending v11 with all of my changes integrated, but I guess now I'll go rework the dabtree to use crc32c, encode the crc32c in the parent pointer attr name, and get rid of all the variable length value decoding. Now that I just got rid of all the intermediate patches with hash-in-attr-name.

Quoted 5 lines

I apologise for not being able to comment on this earlier. What time I've had available over the past coule of weeks has been focussed on the parts of the repair patchset that are ready for merge and I've had little time to follow the progress of the parts you are currently developing.

<nod> Sorry, that was my kneejerk reaction to "let's go reopen this thing and go back to where you were a week ago". I appreciate yuor help getting the online repair patchset ready for merge.

In the end it took about half an hour to put everything back, so I'll go test that overnight and let's see where we end up tomorrow.

--D

Quoted 4 lines

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#37 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

Quoted 67 lines · Darrick J. Wong
Quoted 66 lines · Dave Chinner
Quoted 65 lines · Darrick J. Wong
Quoted 64 lines · Dave Chinner
Quoted 12 lines · Darrick J. Wong
Quoted 11 lines · Amir Goldstein
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Amir Goldstein

On Fri, Mar 24, 2023 at 8:19 PM Allison Henderson
<[email protected]> wrote:

Right.
So how about
(parent_ino, parent_gen, dirent_hash) -> (dirent_name)

This is not a breaking change and you won't need to do another breaking change later.

Quoted 7 lines
Quoted 2 lines

This could also be internal to VLOOKUP: appended vhash to attr name and limit VLOOKUP name size to 255 - vhashsize.

The original "put the hash in the xattr name" patches did that, but I discarded that approach because it increases the size of each parent pointer by 4 bytes, and was really easy to make a verrrry slow filesystem:

Right, that's because hash collision resolution is very dumb. It's just a linear walk from start to end comparing the names of each coliding entry.

Quoted 6 lines

I wrote an xfs_io command to compute lists of names with the same dirhash value. If I created a giant directory with the same file hardlinked millions of times where all those dirent names all hash to the same value, lookups in the directory gets really slow because the dabtree code has to walk (on average) half a million dirents to find the matching one.

That's worst case behaviour, not real-world production behaviour. It's also a dabtree optimisation problem, not a parent pointer problem. i.e. if this is a real world issue, we need to fix the dabtree indexing, not work around it in the parent pointer format.

IOWs, optimising the parent pointer structure for the *rare* corner case of an intentionally crafted dahash btree collision farm is, to me, the wrong way to design a structure.

Optimising it for the common fast path (single parent xattrs, followed by small numbers of parents) is what we should be doing, because that situation will occur an uncountable number of times more frequently than this theorectical worst case behaviour. i.e. the worst case will, in all likelihood, never happen in production environments - behaviour in this circumstance is only a consideration for mitigating malicious attacks.

And, quite frankly, nobody is going to bother attacking a filesystem this way - there is literally nothing that can be gained from slowing down access to a single directory in this manner. It's a DOS at best, but there are *much* easier ways of doing that and it doesn't require parent pointers or dahash collisions at all.

Quoted 7 lines

There were also millions of parent pointer xattrs, all with the same attr name and hence the same hash value here too. Doing that made the performance totally awful. Changing the hash to crc32c and then sha512 made it much harder to induce collision slowdowns on both ends like that, though sha512 added a noticeable performance hit for what it was preventing.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

So why not change the dahash for parent pointer enabled filesystems to use crc32c everywhere?

That's not difficult to do, but it'll break name obscuring in metadump unless you know of a quick way to derive B from A and maintain crc32c(A) == crc32c(B).

Yeah. So I've sort of figured out how to spoof crc32c in much the same way that obfuscate_name() does for the dahash -- fill a buffer with replacement characters, figure out which bits to flip to end up with the same crc at the end. I quickly found Mark Adler's spoof program which seems to work more reliably.

But after a bit of performance comparison, I realized that constructing namehash collisions with same-length strings is about as computationally easy with crc32c as it is with the dahash. This isn't any better than what we have now. I ran a dumb benchmark of the kernel crc32c implementation vs. dahash and noticed that for the sizes we care about (i.e. 255 bytes or less) the old dahash is only slightly slower than hardware accelerated crc32c.

I then noticed that obfuscate_name() totally does the wrong thing if ascii-ci is enabled, and indeed, trying to metadump such a filesystem causes metadump to throw errors all over the place.

In short: doing this doesn't buy us anything, but now someone has to implement a robust obfuscate_name() for crc32c and we have more software to maintain.

Quoted 10 lines
Quoted 9 lines

Not easily, but....

Quoted 2 lines

Granted I had mostly written off name obscuring in metadump on parent pointer filesystems anyway.

... so had I. The name obfuscation needs a completely different approach for PP enabled filesystems because of the name also being embedded in non-user visible structures and potentially changing attr tree dahash-based indexing.

Quoted 4 lines

<nod> I've wondered if we just have to record (dir_ino/name) -> (newname) and use that to update the parent pointer and any dabtree entries we find. That's going to be a bit of a mess to iron out since we can't change the "mounted" filesystem.

For now I've fixed this by implementing a gigantic name remapping table so that we remember (dir_ino, old_name, new_name) tuples. When we encounter the equivalent parent pointer, we can use the exact same obfuscated name, which means that xfs_repair on a restored obscured metadump works again.

While I was at it, I also implemented a 'parent' command for xfs_db that dumps the parents of any inode, similar to the existing xfs_io command.

Quoted 18 lines
Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

I'd much prefer we have a parent pointer index key that is fixed length, has constant hash time and a dabtree hash that is far more resilient to collision farming than to have to perform encoding/decoding of varaible length values into the key to work around a collision-rich hashing algorithm.

Wellllll I was 10 minutes away from sending v11 with all of my changes integrated, but I guess now I'll go rework the dabtree to use crc32c, encode the crc32c in the parent pointer attr name, and get rid of all the variable length value decoding. Now that I just got rid of all the intermediate patches with hash-in-attr-name.

Quoted 5 lines

I apologise for not being able to comment on this earlier. What time I've had available over the past coule of weeks has been focussed on the parts of the repair patchset that are ready for merge and I've had little time to follow the progress of the parts you are currently developing.

Quoted 6 lines

<nod> Sorry, that was my kneejerk reaction to "let's go reopen this thing and go back to where you were a week ago". I appreciate yuor help getting the online repair patchset ready for merge.

In the end it took about half an hour to put everything back, so I'll go test that overnight and let's see where we end up tomorrow.

Now that I've had a day to go sift through all of these changes, I've realized that you're right -- putting the ondisk format back to:

(parent_ino, parent_gen, namehash) -> (dirent name)

Simplifes the parent pointer ondisk conversion and runtime code quite a bit. It also makes it easy to implement obfuscated metadumps. So I'll go run v11 through fstestscloud with all of today's changes integrated, and we'll see about getting something out in a few days.

Again, sorry for being a jerk.

--D

Quoted 6 lines

--D

Quoted 4 lines

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#38 lore raw

Re: [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key

In reply to Allison Henderson
Quoted 91 lines · Allison Henderson
Quoted 84 lines · Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the
proposed ondisk parent pointer format presents a major difficulty for online directory repair.  This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent.  There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take
a diroffset and try to create the new entry at that offset.  This will
not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping).
Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates
necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary.  This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new
directory structure.  If there are a large number of diroffset updates,
then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair
will have no choice but to roll to a fresh transaction to continue logging.  This breaks repair's policy that repairs should commit atomically.  It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes.
This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code.  Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information,
so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair
code.  Unfortunately, there is one hitch: xattr names cannot exceed 255
bytes due to ondisk format limitations.  We don't want to constrain the
length of dirent names, so instead we create a special VLOOKUP mode for
extended attributes that allows parent pointers to require matching on
both the name and the value.

The ondisk format of a parent pointer can then become:

    (parent_ino, parent_gen, dirent_name[0:242]) -> (dirent_name[243:255])

Because we can always look up a specific parent pointer.  Most of the patches in this patchset prepare the high level xattr code and the lower
level logging code to do this correctly, and the last patch switches the
ondisk format.

Quoted 6 lines

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy! Comments and questions are, as always, welcome. kernel git tree:

Quoted 7 lines

So I gave this set a look over and for the most part I think it's ok as long as folks agree on the new format? It's a lot different from what folks originally articulated that they wanted, but for the most part the format change doesnt affect parent pointer mechanics as much as it will affect other features that may use it. I havnt seen much complaints in response, so if it's all the same to everyone else, I am fine with moving forward with it? Thanks all!

Yes, let's move forward. As I mentioned on IRC last night I'll merge
my changes into parent pointers v11 and put that out next weekish.

--D

Quoted 39 lines

Allison

Quoted 36 lines

https://urldefense.com/v3/__https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!NDopQXFVRjDfqgJvGMQRzS3fXyyd1SC4HfGl4dxSMWxNPsNKScecjp7bZYX-dDhL5tatk_xxJFdQwCj9YNGq$

xfsprogs git tree:
https://urldefense.com/v3/__https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!NDopQXFVRjDfqgJvGMQRzS3fXyyd1SC4HfGl4dxSMWxNPsNKScecjp7bZYX-dDhL5tatk_xxJFdQwILBe7jj$

fstests git tree:
https://urldefense.com/v3/__https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!NDopQXFVRjDfqgJvGMQRzS3fXyyd1SC4HfGl4dxSMWxNPsNKScecjp7bZYX-dDhL5tatk_xxJFdQwI5iKKgk$

---
 fs/xfs/libxfs/xfs_attr.c       |   66 +++++--- fs/xfs/libxfs/xfs_attr.h       |    5 + fs/xfs/libxfs/xfs_attr_leaf.c  |   41 ++++- fs/xfs/libxfs/xfs_da_btree.h   |    6 + fs/xfs/libxfs/xfs_da_format.h  |   39 ++++- fs/xfs/libxfs/xfs_fs.h         |    2 fs/xfs/libxfs/xfs_log_format.h |   31 +++- fs/xfs/libxfs/xfs_parent.c     |  215 +++++++++++++++++---------- fs/xfs/libxfs/xfs_parent.h     |   46 +++--- fs/xfs/libxfs/xfs_trans_resv.c |    7 + fs/xfs/scrub/dir.c             |   34 +--- fs/xfs/scrub/dir_repair.c      |   87 +++-------- fs/xfs/scrub/parent.c          |   48 +----- fs/xfs/scrub/parent_repair.c   |   55 ++----- fs/xfs/scrub/trace.h           |   65 +------- fs/xfs/xfs_attr_item.c         |  318
+++++++++++++++++++++++++++++++---------
 fs/xfs/xfs_attr_item.h         |    3 fs/xfs/xfs_inode.c             |   30 ++-- fs/xfs/xfs_ondisk.h            |    1 fs/xfs/xfs_parent_utils.c      |    8 + fs/xfs/xfs_symlink.c           |    3 fs/xfs/xfs_xattr.c             |    5 +

 22 files changed, 660 insertions(+), 455 deletions(-)
#39 lore raw

cover xfsprogs: tool fixes for parent pointers

In reply to Darrick J. Wong

Hi all,

Here are a bunch of tooling changes for the parent pointers code. The only new feature here is to decode the parent pointer xattr name in xfs_db so that we can interpret (and someday fuzz) them. Everything else are bug fixes.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-toolfixes

---
 db/attr.c                |   31 ++++++++++++++++++++++
 db/attrshort.c           |   25 ++++++++++++++++++
 db/metadump.c            |   34 +++++++++++++++++-------
 include/libxfs.h         |    1 +
 io/parent.c              |    5 ++--
 libxfs/init.c            |    7 +++++
 libxfs/libxfs_api_defs.h |    4 +++
 libxfs/util.c            |   14 ++++++++++
 mkfs/proto.c             |   65 +++++++++++++++++++++++++++++++++-------------
 repair/da_util.c         |    2 +
 10 files changed, 157 insertions(+), 31 deletions(-)
#40 lore raw

PATCH 1/9 libxfs: initialize the slab cache for parent defer items

From: Darrick J. Wong <[email protected]>

Initialize the slab cache for parent defer items. We'll need this in an upcoming patch.

---
 include/libxfs.h |    1 +
 libxfs/init.c    |    3 +++
 2 files changed, 4 insertions(+)
include/libxfs.h +1 -0
index b28781d19..cc57e8887 100644
@@ -78,6 +78,7 @@ struct iomap;
 #include "xfs_refcount_btree.h"
 #include "xfs_refcount.h"
 #include "xfs_btree_staging.h"
+#include "xfs_parent.h"
 
 #ifndef ARRAY_SIZE
 #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
libxfs/init.c +3 -0
index fda36ba0f..59cd547d6 100644
@@ -258,6 +258,8 @@ init_caches(void)
 			"xfs_extfree_item");
 	xfs_trans_cache = kmem_cache_init(
 			sizeof(struct xfs_trans), "xfs_trans");
+	xfs_parent_intent_cache = kmem_cache_init(
+			sizeof(struct xfs_parent_defer), "xfs_parent_defer");
 }
 
 static int
@@ -275,6 +277,7 @@ destroy_caches(void)
 	xfs_btree_destroy_cur_caches();
 	leaked += kmem_cache_destroy(xfs_extfree_item_cache);
 	leaked += kmem_cache_destroy(xfs_trans_cache);
+	leaked += kmem_cache_destroy(xfs_parent_intent_cache);
 
 	return leaked;
 }
#41 lore raw

PATCH 2/9 mkfs: fix libxfs api misuse

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix libxfs usage problems as pointed out by xfs/437.

---
 libxfs/libxfs_api_defs.h |    1 +
 mkfs/proto.c             |    4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
libxfs/libxfs_api_defs.h +1 -0
index f8efcce77..e44b0b29e 100644
@@ -124,6 +124,7 @@
 #define xfs_initialize_perag		libxfs_initialize_perag
 #define xfs_initialize_perag_data	libxfs_initialize_perag_data
 #define xfs_init_local_fork		libxfs_init_local_fork
+#define xfs_init_parent_name_rec	libxfs_init_parent_name_rec
 
 #define xfs_inobt_maxrecs		libxfs_inobt_maxrecs
 #define xfs_inobt_stage_cursor		libxfs_inobt_stage_cursor
mkfs/proto.c +2 -2
index 36d8cde21..ac7ffbe9d 100644
@@ -613,8 +613,8 @@ parseproto(
 			.value = (void *)xname.name,
 			.valuelen = xname.len,
 		};
-		xfs_init_parent_name_rec(&rec, pip, offset);
-		error = xfs_attr_set(&args);
+		libxfs_init_parent_name_rec(&rec, pip, offset);
+		error = -libxfs_attr_set(&args);
 		if (error)
 			fail(_("Error creating parent pointer"), error);
 	}
#42 lore raw

PATCH 3/9 libxfs: create new files with attr forks if necessary

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create new files with attr forks if they're going to have parent pointers. In the next patch we'll fix mkfs to use the same parent creation functions as the kernel, so we're going to need this.

---
 libxfs/init.c |    4 ++++
 libxfs/util.c |   14 ++++++++++++++
 2 files changed, 18 insertions(+)
libxfs/init.c +4 -0
index 59cd547d6..101b77602 100644
@@ -746,14 +746,18 @@ void
 libxfs_compute_all_maxlevels(
 	struct xfs_mount	*mp)
 {
+	struct xfs_ino_geometry *igeo = M_IGEO(mp);
+
 	xfs_alloc_compute_maxlevels(mp);
 	xfs_bmap_compute_maxlevels(mp, XFS_DATA_FORK);
 	xfs_bmap_compute_maxlevels(mp, XFS_ATTR_FORK);
+	igeo->attr_fork_offset = xfs_bmap_compute_attr_offset(mp);
 	xfs_ialloc_setup_geometry(mp);
 	xfs_rmapbt_compute_maxlevels(mp);
 	xfs_refcountbt_compute_maxlevels(mp);
 
 	xfs_agbtree_compute_maxlevels(mp);
+
 }
 
 /*
libxfs/util.c +14 -0
index 6525f63de..bea5f1c71 100644
@@ -322,6 +322,20 @@ libxfs_init_new_inode(
 		ASSERT(0);
 	}
 
+	/*
+	 * If we need to create attributes immediately after allocating the
+	 * inode, initialise an empty attribute fork right now. We use the
+	 * default fork offset for attributes here as we don't know exactly what
+	 * size or how many attributes we might be adding. We can do this
+	 * safely here because we know the data fork is completely empty and
+	 * this saves us from needing to run a separate transaction to set the
+	 * fork offset in the immediate future.
+	 */
+	if (xfs_has_parent(tp->t_mountp) && xfs_has_attr(tp->t_mountp)) {
+		ip->i_forkoff = xfs_default_attroffset(ip) >> 3;
+		xfs_ifork_init_attr(ip, XFS_DINODE_FMT_EXTENTS, 0);
+	}
+
 	/*
 	 * Log the new values stuffed into the inode.
 	 */
#43 lore raw

PATCH 4/9 mkfs: fix subdir parent pointer creation

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rework the protofile code so that it uses the same deferred parent pointer ops that the kernel uses to create parent pointers. While we're at it, make it so that subdirs of the root directory and reserved files also get parent pointers. Found by xfs/019.

---
 libxfs/libxfs_api_defs.h |    3 ++
 mkfs/proto.c             |   65 +++++++++++++++++++++++++++++++++-------------
 2 files changed, 50 insertions(+), 18 deletions(-)
libxfs/libxfs_api_defs.h +3 -0
index e44b0b29e..055d2862a 100644
@@ -140,6 +140,9 @@
 #define xfs_log_get_max_trans_res	libxfs_log_get_max_trans_res
 #define xfs_log_sb			libxfs_log_sb
 #define xfs_mode_to_ftype		libxfs_mode_to_ftype
+#define xfs_parent_defer_add		libxfs_parent_defer_add
+#define xfs_parent_finish		libxfs_parent_finish
+#define xfs_parent_start		libxfs_parent_start
 #define xfs_perag_get			libxfs_perag_get
 #define xfs_perag_put			libxfs_perag_put
 #define xfs_prealloc_blocks		libxfs_prealloc_blocks
mkfs/proto.c +47 -18

155 lines not on this page · raw message

#44 lore raw

PATCH 5/9 xfs_db: report parent pointer keys

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Decode the parent pointer inode, generation, and diroffset fields.

---
 db/attr.c      |   31 +++++++++++++++++++++++++++++++
 db/attrshort.c |   25 +++++++++++++++++++++++++
 2 files changed, 56 insertions(+)
db/attr.c +31 -0
index f29e4a544..db7cf54b5 100644
@@ -19,6 +19,7 @@ static int	attr_leaf_entries_count(void *obj, int startoff);
 static int	attr_leaf_hdr_count(void *obj, int startoff);
 static int	attr_leaf_name_local_count(void *obj, int startoff);
 static int	attr_leaf_name_local_name_count(void *obj, int startoff);
+static int	attr_leaf_name_pptr_count(void *obj, int startoff);
 static int	attr_leaf_name_local_value_count(void *obj, int startoff);
 static int	attr_leaf_name_local_value_offset(void *obj, int startoff,
 						  int idx);
@@ -111,6 +112,8 @@ const field_t	attr_leaf_map_flds[] = {
 
 #define	LNOFF(f)	bitize(offsetof(xfs_attr_leaf_name_local_t, f))
 #define	LVOFF(f)	bitize(offsetof(xfs_attr_leaf_name_remote_t, f))
+#define	PPOFF(f)	bitize(offsetof(xfs_attr_leaf_name_local_t, nameval) + \
+			       offsetof(struct xfs_parent_name_rec, f))
 const field_t	attr_leaf_name_flds[] = {
 	{ "valuelen", FLDT_UINT16D, OI(LNOFF(valuelen)),
 	  attr_leaf_name_local_count, FLD_COUNT, TYP_NONE },
@@ -118,6 +121,12 @@ const field_t	attr_leaf_name_flds[] = {
 	  attr_leaf_name_local_count, FLD_COUNT, TYP_NONE },
 	{ "name", FLDT_CHARNS, OI(LNOFF(nameval)),
 	  attr_leaf_name_local_name_count, FLD_COUNT, TYP_NONE },
+	{ "parent_ino", FLDT_INO, OI(PPOFF(p_ino)),
+	  attr_leaf_name_pptr_count, FLD_COUNT, TYP_INODE },
+	{ "parent_gen", FLDT_UINT32D, OI(PPOFF(p_gen)),
+	  attr_leaf_name_pptr_count, FLD_COUNT, TYP_NONE },
+	{ "parent_diroffset", FLDT_UINT32D, OI(PPOFF(p_diroffset)),
+	  attr_leaf_name_pptr_count, FLD_COUNT, TYP_NONE },
 	{ "value", FLDT_CHARNS, attr_leaf_name_local_value_offset,
 	  attr_leaf_name_local_value_count, FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "valueblk", FLDT_UINT32X, OI(LVOFF(valueblk)),
@@ -273,6 +282,26 @@ attr_leaf_name_local_count(
 				    __attr_leaf_name_local_count);
 }
 
+static int
+__attr_leaf_name_pptr_count(
+	struct xfs_attr_leafblock	*leaf,
+	struct xfs_attr_leaf_entry      *e,
+	int				i)
+{
+	if (e->flags & XFS_ATTR_PARENT)
+		return 1;
+	return 0;
+}
+
+static int
+attr_leaf_name_pptr_count(
+	void				*obj,
+	int				startoff)
+{
+	return attr_leaf_entry_walk(obj, startoff,
+			__attr_leaf_name_pptr_count);
+}
+
 static int
 __attr_leaf_name_local_name_count(
 	struct xfs_attr_leafblock	*leaf,
@@ -283,6 +312,8 @@ __attr_leaf_name_local_name_count(
 
 	if (!(e->flags & XFS_ATTR_LOCAL))
 		return 0;
+	if (e->flags & XFS_ATTR_PARENT)
+		return 0;
 
 	l = xfs_attr3_leaf_name_local(leaf, i);
 	return l->namelen;
db/attrshort.c +25 -0
index 872d771d5..7c8ac485d 100644
@@ -13,6 +13,7 @@
 #include "attrshort.h"
 
 static int	attr_sf_entry_name_count(void *obj, int startoff);
+static int	attr_sf_entry_pptr_count(void *obj, int startoff);
 static int	attr_sf_entry_value_count(void *obj, int startoff);
 static int	attr_sf_entry_value_offset(void *obj, int startoff, int idx);
 static int	attr_shortform_list_count(void *obj, int startoff);
@@ -34,6 +35,8 @@ const field_t	attr_sf_hdr_flds[] = {
 };
 
 #define	EOFF(f)	bitize(offsetof(struct xfs_attr_sf_entry, f))
+#define	PPOFF(f) bitize(offsetof(struct xfs_attr_sf_entry, nameval) + \
+			offsetof(struct xfs_parent_name_rec, f))
 const field_t	attr_sf_entry_flds[] = {
 	{ "namelen", FLDT_UINT8D, OI(EOFF(namelen)), C1, 0, TYP_NONE },
 	{ "valuelen", FLDT_UINT8D, OI(EOFF(valuelen)), C1, 0, TYP_NONE },
@@ -49,11 +52,31 @@ const field_t	attr_sf_entry_flds[] = {
 	  TYP_NONE },
 	{ "name", FLDT_CHARNS, OI(EOFF(nameval)), attr_sf_entry_name_count,
 	  FLD_COUNT, TYP_NONE },
+	{ "parent_ino", FLDT_INO, OI(PPOFF(p_ino)), attr_sf_entry_pptr_count,
+	  FLD_COUNT, TYP_INODE },
+	{ "parent_gen", FLDT_UINT32D, OI(PPOFF(p_gen)), attr_sf_entry_pptr_count,
+	  FLD_COUNT, TYP_NONE },
+	{ "parent_diroffset", FLDT_UINT32D, OI(PPOFF(p_diroffset)),
+	   attr_sf_entry_pptr_count, FLD_COUNT, TYP_NONE },
 	{ "value", FLDT_CHARNS, attr_sf_entry_value_offset,
 	  attr_sf_entry_value_count, FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ NULL }
 };
 
+static int
+attr_sf_entry_pptr_count(
+	void				*obj,
+	int				startoff)
+{
+	struct xfs_attr_sf_entry	*e;
+
+	ASSERT(bitoffs(startoff) == 0);
+	e = (struct xfs_attr_sf_entry *)((char *)obj + byteize(startoff));
+	if (e->flags & XFS_ATTR_PARENT)
+		return 1;
+	return 0;
+}
+
 static int
 attr_sf_entry_name_count(
 	void				*obj,
@@ -63,6 +86,8 @@ attr_sf_entry_name_count(
 
 	ASSERT(bitoffs(startoff) == 0);
 	e = (struct xfs_attr_sf_entry *)((char *)obj + byteize(startoff));
+	if (e->flags & XFS_ATTR_PARENT)
+		return 0;
 	return e->namelen;
 }
 
#45 lore raw

PATCH 6/9 xfs_db: obfuscate dirent and pptr names consistently

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When someone wants to perform an obfuscated metadump of a filesystem where parent pointers are enabled, we have to use the *exact* same obfuscated name for both the directory entry and the parent pointer. Instead of using an RNG to influence the obfuscated name, use the dirent inode number to start the obfuscated name. This makes them consistent, though the resulting names aren't quite so full of control characters.

---
 db/metadump.c |   34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)
db/metadump.c +24 -10
index 27d1df432..bb441fbbb 100644
@@ -740,12 +740,14 @@ nametable_add(xfs_dahash_t hash, int namelen, unsigned char *name)
 #define rol32(x,y)		(((x) << (y)) | ((x) >> (32 - (y))))
 
 static inline unsigned char
-random_filename_char(void)
+random_filename_char(xfs_ino_t	ino)
 {
 	static unsigned char filename_alphabet[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
 						"abcdefghijklmnopqrstuvwxyz"
 						"0123456789-_";
 
+	if (ino)
+		return filename_alphabet[ino % (sizeof filename_alphabet - 1)];
 	return filename_alphabet[random() % (sizeof filename_alphabet - 1)];
 }
 
@@ -815,6 +817,7 @@ in_lost_found(
  */
 static void
 obfuscate_name(
+	xfs_ino_t	ino,
 	xfs_dahash_t	hash,
 	size_t		name_len,
 	unsigned char	*name)
@@ -842,7 +845,7 @@ obfuscate_name(
 	 * Accumulate its new hash value as we go.
 	 */
 	for (i = 0; i < name_len - 5; i++) {
-		*newp = random_filename_char();
+		*newp = random_filename_char(ino);
 		new_hash = *newp ^ rol32(new_hash, 7);
 		newp++;
 	}
@@ -1207,7 +1210,10 @@ generate_obfuscated_name(
 	/* Obfuscate the name (if possible) */
 
 	hash = libxfs_da_hashname(name, namelen);
-	obfuscate_name(hash, namelen, name);
+	if (xfs_has_parent(mp))
+		obfuscate_name(ino, hash, namelen, name);
+	else
+		obfuscate_name(0, hash, namelen, name);
 
 	/*
 	 * Make sure the name is not something already seen.  If we
@@ -1320,7 +1326,7 @@ obfuscate_path_components(
 			/* last (or single) component */
 			namelen = strnlen((char *)comp, len);
 			hash = libxfs_da_hashname(comp, namelen);
-			obfuscate_name(hash, namelen, comp);
+			obfuscate_name(0, hash, namelen, comp);
 			break;
 		}
 		namelen = slash - (char *)comp;
@@ -1331,7 +1337,7 @@ obfuscate_path_components(
 			continue;
 		}
 		hash = libxfs_da_hashname(comp, namelen);
-		obfuscate_name(hash, namelen, comp);
+		obfuscate_name(0, hash, namelen, comp);
 		comp += namelen + 1;
 		len -= namelen + 1;
 	}
@@ -1407,10 +1413,15 @@ process_sf_attr(
 		}
 
 		if (obfuscate) {
-			generate_obfuscated_name(0, asfep->namelen,
-						 &asfep->nameval[0]);
-			memset(&asfep->nameval[asfep->namelen], 'v',
-			       asfep->valuelen);
+			if (asfep->flags & XFS_ATTR_PARENT) {
+				generate_obfuscated_name(cur_ino, asfep->valuelen,
+					 &asfep->nameval[asfep->namelen]);
+			} else {
+				generate_obfuscated_name(0, asfep->namelen,
+							 &asfep->nameval[0]);
+				memset(&asfep->nameval[asfep->namelen], 'v',
+				       asfep->valuelen);
+			}
 		}
 
 		asfep = (struct xfs_attr_sf_entry *)((char *)asfep +
@@ -1785,7 +1796,7 @@ process_attr_block(
 						(long long)cur_ino);
 				break;
 			}
-			if (obfuscate) {
+			if (obfuscate && !(entry->flags & XFS_ATTR_PARENT)) {
 				generate_obfuscated_name(0, local->namelen,
 					&local->nameval[0]);
 				memset(&local->nameval[local->namelen], 'v',
@@ -1797,6 +1808,9 @@ process_attr_block(
 			zlen = xfs_attr_leaf_entsize_local(nlen, vlen) -
 				(sizeof(xfs_attr_leaf_name_local_t) - 1 +
 				 nlen + vlen);
+			if (obfuscate && (entry->flags & XFS_ATTR_PARENT))
+				generate_obfuscated_name(cur_ino, vlen,
+						&local->nameval[nlen]);
 			if (zero_stale_data)
 				memset(&local->nameval[nlen + vlen], 0, zlen);
 		} else {
#46 lore raw

PATCH 7/9 xfs_io: print path in path_print

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Actually print the path string once we've bothered to construct it into a string buffer.

---
 io/parent.c |    2 ++
 1 file changed, 2 insertions(+)
io/parent.c +2 -0
index a6f3fa0ca..b18e02c4b 100644
@@ -87,6 +87,8 @@ path_print(
 	ret = path_list_to_string(path, buf + ret, len - ret);
 	if (ret < 0)
 		return ret;
+
+	printf("%s\n", buf);
 	return 0;
 }
 
#47 lore raw

PATCH 8/9 xfs_io: parent command is not experts-only

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This command isn't dangerous, so don't make it experts-only.

---
 io/parent.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
io/parent.c +1 -2
index b18e02c4b..6af342820 100644
@@ -250,6 +250,5 @@ parent_init(void)
 	parent_cmd.oneline = _("print parent inodes");
 	parent_cmd.help = parent_help;
 
-	if (expert)
-		add_command(&parent_cmd);
+	add_command(&parent_cmd);
 }
#48 lore raw

PATCH 9/9 xfs_repair: fix incorrect dabtree hashval comparison

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If an xattr structure contains enough names with the same hash value to fill multiple xattr leaf blocks with names all hashing to the same value, then the dabtree nodes will contain consecutive entries with the same hash value.

This causes false corruption reports in xfs_repair because it's not expecting such a huge same-hashing structure. Fix that.

---
 repair/da_util.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
repair/da_util.c +1 -1
index 7239c2e2c..b229422c8 100644
@@ -330,7 +330,7 @@ _("%s block used/count inconsistency - %d/%hu\n"),
 	/*
 	 * hash values monotonically increasing ???
 	 */
-	if (cursor->level[this_level].hashval >=
+	if (cursor->level[this_level].hashval >
 				be32_to_cpu(nodehdr.btree[entry].hashval)) {
 		do_warn(
 _("%s block hashvalue inconsistency, expected > %u / saw %u\n"),
#49 lore raw

cover xfsprogs: actually use getparent ioctl

In reply to Darrick J. Wong

Hi all,

Yikes. The userspace parent pointers support code dates from 2017 and is very very moldy. This patchset moves the xfs_io filtering stuff back to xfs_io. It also moves the parent pointer support code to libfrog because we don't want to expose things via libhandle until we're absolutely sure that we want to do that.

(We probably want to do that some day.)

Finally, adapt xfs_scrub to use parent pointer information whenever it has something to say about a file handle that it has open.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-use-getparents

---
 scrub/common.c |   21 +++++++++++++++++++++
 scrub/inodes.c |   26 --------------------------
 scrub/inodes.h |    2 --
 3 files changed, 21 insertions(+), 28 deletions(-)
#50 lore raw

PATCH 1/2 xfs_scrub: revert unnecessary code from "implement the upper half of parent pointers"

From: Darrick J. Wong <[email protected]>

Revert this piece which is no longer necessary.

---
 scrub/inodes.c |   26 --------------------------
 scrub/inodes.h |    2 --
 2 files changed, 28 deletions(-)
scrub/inodes.c +0 -26
index 245dd713d..78f0914b8 100644
@@ -19,7 +19,6 @@
 #include "descr.h"
 #include "libfrog/fsgeom.h"
 #include "libfrog/bulkstat.h"
-#include "parent.h"
 
 /*
  * Iterate a range of inodes.
@@ -450,28 +449,3 @@ scrub_open_handle(
 	return open_by_fshandle(handle, sizeof(*handle),
 			O_RDONLY | O_NOATIME | O_NOFOLLOW | O_NOCTTY);
 }
-
-/* Construct a description for an inode. */
-void
-xfs_scrub_ino_descr(
-	struct scrub_ctx	*ctx,
-	struct xfs_handle	*handle,
-	char			*buf,
-	size_t			buflen)
-{
-	uint64_t		ino;
-	xfs_agnumber_t		agno;
-	xfs_agino_t		agino;
-	int			ret;
-
-	ret = handle_to_path(handle, sizeof(struct xfs_handle), buf, buflen);
-	if (ret >= 0)
-		return;
-
-	ino = handle->ha_fid.fid_ino;
-	agno = ino / (1ULL << (ctx->mnt.inopblog + ctx->mnt.agblklog));
-	agino = ino % (1ULL << (ctx->mnt.inopblog + ctx->mnt.agblklog));
-	snprintf(buf, buflen, _("inode %"PRIu64" (%u/%u)"), ino, agno,
-			agino);
-}
-
scrub/inodes.h +0 -2
index 189fa282d..f03180458 100644
@@ -21,7 +21,5 @@ int scrub_scan_all_inodes(struct scrub_ctx *ctx, scrub_inode_iter_fn fn,
 		void *arg);
 
 int scrub_open_handle(struct xfs_handle *handle);
-void xfs_scrub_ino_descr(struct scrub_ctx *ctx, struct xfs_handle *handle,
-		char *buf, size_t buflen);
 
 #endif /* XFS_SCRUB_INODES_H_ */
#51 lore raw

PATCH 2/2 xfs_scrub: use parent pointers when possible to report file operations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If parent pointers are available, use them to supply file paths when doing things to files, instead of merely printing the inode number.

---
 scrub/common.c |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
scrub/common.c +21 -0
index 49a87f412..9f3cde9bc 100644
@@ -12,6 +12,7 @@
 #include "xfs_scrub.h"
 #include "common.h"
 #include "progress.h"
+#include "libfrog/pptrs.h"
 
 extern char		*progname;
 
@@ -407,6 +408,26 @@ scrub_render_ino_descr(
 	uint32_t		agino;
 	int			ret;
 
+	if (ctx->mnt.fsgeom.flags & XFS_FSOP_GEOM_FLAGS_PARENT) {
+		struct xfs_handle handle;
+
+		memcpy(&handle.ha_fsid, ctx->fshandle, sizeof(handle.ha_fsid));
+		handle.ha_fid.fid_len = sizeof(xfs_fid_t) -
+				sizeof(handle.ha_fid.fid_len);
+		handle.ha_fid.fid_pad = 0;
+		handle.ha_fid.fid_ino = ino;
+		handle.ha_fid.fid_gen = gen;
+
+		ret = handle_to_path(&handle, sizeof(struct xfs_handle), buf,
+				buflen);
+		/*
+		 * If successful, return any positive integer to use the
+		 * formatted error string.
+		 */
+		if (ret == 0)
+			return 1;
+	}
+
 	agno = cvt_ino_to_agno(&ctx->mnt, ino);
 	agino = cvt_ino_to_agino(&ctx->mnt, ino);
 	ret = snprintf(buf, buflen, _("inode %"PRIu64" (%"PRIu32"/%"PRIu32")%s"),
#52 lore raw

cover libfrog: fix parent pointer library code

In reply to Darrick J. Wong

Hi all,

This series moves the parent pointer library code out of libhandle and into libfrog. This is necessary because libhandle is exported as a userspace library, and we don't want to export the parent pointer stuff until we're absolutely ready to do that. So that move is made in the first patch.

The rest of the patchset fixes various bugs and inconsistencies and bitrot that have cropped up since I wrote this code in 2017.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-fix-libfrog-code

---
 include/parent.h   |   25 ------
 io/parent.c        |  117 +++++++++++++++++-----------
 libfrog/Makefile   |    2 
 libfrog/paths.c    |   46 +++++++++--
 libfrog/paths.h    |    8 +-
 libfrog/pptrs.c    |  217 ++++++++++++++++++++++------------------------------
 libfrog/pptrs.h    |   25 ++++++
 libhandle/Makefile |    2 
 8 files changed, 237 insertions(+), 205 deletions(-)
 rename libhandle/parent.c => libfrog/pptrs.c (50%)
 create mode 100644 libfrog/pptrs.h
#53 lore raw

PATCH 1/7 xfs_io: move parent pointer filtering and formatting flags out of libhandle

From: Darrick J. Wong <[email protected]>

All this filtering and presentation stuff originates in xfs_io and should stay there. The added arguments seriously complicate the basic iterator interface and there are no other users.

While we're at it, fix a bug in path_print where it doesn't actually print the path.

---
 include/parent.h   |   17 +++------
 io/parent.c        |   99 ++++++++++++++++++++++++++++++++--------------------
 libfrog/paths.c    |   28 ++++++++++++++-
 libfrog/paths.h    |    8 +++-
 libhandle/parent.c |   77 ++++++++++++----------------------------
 5 files changed, 119 insertions(+), 110 deletions(-)
include/parent.h +5 -12
index 2e136724b..fb9000419 100644
@@ -17,27 +17,20 @@ typedef struct parent_cursor {
 	__u32	opaque[4];      /* an opaque cookie */
 } parent_cursor_t;
 
-/* Print parent pointer option flags */
-#define XFS_PPPTR_OFLAG_SHORT  (1<<0)	/* Print in short format */
-
 struct path_list;
 
 typedef int (*walk_pptr_fn)(struct xfs_pptr_info *pi, struct xfs_parent_ptr *pptr,
-		void *arg, int flags);
+		void *arg);
 typedef int (*walk_ppath_fn)(const char *mntpt, struct path_list *path,
 		void *arg);
 
 #define WALK_PPTRS_ABORT	1
-int fd_walk_pptrs(int fd, uint64_t pino, char *pname, walk_pptr_fn fn,
-		void *arg, int flags);
-int handle_walk_pptrs(void *hanp, size_t hanlen, uint64_t pino, char *pname,
-		walk_pptr_fn fn, void *arg, int flags);
+int fd_walk_pptrs(int fd, walk_pptr_fn fn, void *arg);
+int handle_walk_pptrs(void *hanp, size_t hanlen, walk_pptr_fn fn, void *arg);
 
 #define WALK_PPATHS_ABORT	1
-int fd_walk_ppaths(int fd, uint64_t pino, char *pname, walk_ppath_fn fn,
-		void *arg, int flags);
-int handle_walk_ppaths(void *hanp, size_t hanlen, uint64_t pino, char *pname,
-		walk_ppath_fn fn, void *arg, int flags);
+int fd_walk_ppaths(int fd, walk_ppath_fn fn, void *arg);
+int handle_walk_ppaths(void *hanp, size_t hanlen, walk_ppath_fn fn, void *arg);
 
 int fd_to_path(int fd, char *path, size_t pathlen);
 int handle_to_path(void *hanp, size_t hlen, char *path, size_t pathlen);
io/parent.c +60 -39

200 lines not on this page · raw message

libfrog/paths.c +26 -2
index a86ae07c1..e541e2007 100644
@@ -574,13 +574,15 @@ struct path_list {
 
 struct path_component {
 	struct list_head	pc_list;
+	uint64_t		pc_ino;
 	char			*pc_fname;
 };
 
 /* Initialize a path component with a given name. */
 struct path_component *
 path_component_init(
-	const char		*name)
+	const char		*name,
+	uint64_t		ino)
 {
 	struct path_component	*pc;
 
@@ -593,6 +595,7 @@ path_component_init(
 		free(pc);
 		return NULL;
 	}
+	pc->pc_ino = ino;
 	return pc;
 }
 
@@ -610,7 +613,8 @@ int
 path_component_change(
 	struct path_component	*pc,
 	void			*name,
-	size_t			namelen)
+	size_t			namelen,
+	uint64_t		ino)
 {
 	void			*p;
 
@@ -620,6 +624,7 @@ path_component_change(
 	pc->pc_fname = p;
 	memcpy(pc->pc_fname, name, namelen);
 	pc->pc_fname[namelen] = 0;
+	pc->pc_ino = ino;
 	return 0;
 }
 
@@ -699,3 +704,22 @@ path_list_to_string(
 	}
 	return bytes;
 }
+
+/* Walk each component of a path. */
+int
+path_walk_components(
+	struct path_list	*path,
+	path_walk_fn_t		fn,
+	void			*arg)
+{
+	struct path_component	*pos;
+	int			ret;
+
+	list_for_each_entry(pos, &path->p_head, pc_list) {
+		ret = fn(pos->pc_fname, pos->pc_ino, arg);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
libfrog/paths.h +6 -2
index 52538fb56..eb66df0c1 100644
@@ -63,10 +63,10 @@ extern fs_path_t *fs_cursor_next_entry(fs_cursor_t *__cp);
 struct path_list;
 struct path_component;
 
-struct path_component *path_component_init(const char *name);
+struct path_component *path_component_init(const char *name, uint64_t ino);
 void path_component_free(struct path_component *pc);
 int path_component_change(struct path_component *pc, void *name,
-		size_t namelen);
+		size_t namelen, uint64_t ino);
 
 struct path_list *path_list_init(void);
 void path_list_free(struct path_list *path);
@@ -77,4 +77,8 @@ void path_list_del_component(struct path_list *path, struct path_component *pc);
 
 ssize_t path_list_to_string(struct path_list *path, char *buf, size_t buflen);
 
+typedef int (*path_walk_fn_t)(const char *name, uint64_t ino, void *arg);
+
+int path_walk_components(struct path_list *path, path_walk_fn_t fn, void *arg);
+
 #endif	/* __PATH_H__ */
libhandle/parent.c +22 -55

196 lines not on this page · raw message

#54 lore raw

PATCH 2/7 libfrog: remove all the parent pointer code from libhandle

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Move this code out of libhandle and into libfrog. We don't want to expose this stuff to a userspace library until customers actually demand it. While we're here, fix the copyright statements and licensing tags.

---
 include/parent.h   |   18 ------------------
 io/parent.c        |    1 +
 libfrog/Makefile   |    2 ++
 libfrog/pptrs.c    |   22 ++++------------------
 libfrog/pptrs.h    |   27 +++++++++++++++++++++++++++
 libhandle/Makefile |    2 +-
 6 files changed, 35 insertions(+), 37 deletions(-)
 rename libhandle/parent.c => libfrog/pptrs.c (87%)
 create mode 100644 libfrog/pptrs.h
include/parent.h +0 -18
index fb9000419..4d3ad51b4 100644
@@ -17,22 +17,4 @@ typedef struct parent_cursor {
 	__u32	opaque[4];      /* an opaque cookie */
 } parent_cursor_t;
 
-struct path_list;
-
-typedef int (*walk_pptr_fn)(struct xfs_pptr_info *pi, struct xfs_parent_ptr *pptr,
-		void *arg);
-typedef int (*walk_ppath_fn)(const char *mntpt, struct path_list *path,
-		void *arg);
-
-#define WALK_PPTRS_ABORT	1
-int fd_walk_pptrs(int fd, walk_pptr_fn fn, void *arg);
-int handle_walk_pptrs(void *hanp, size_t hanlen, walk_pptr_fn fn, void *arg);
-
-#define WALK_PPATHS_ABORT	1
-int fd_walk_ppaths(int fd, walk_ppath_fn fn, void *arg);
-int handle_walk_ppaths(void *hanp, size_t hanlen, walk_ppath_fn fn, void *arg);
-
-int fd_to_path(int fd, char *path, size_t pathlen);
-int handle_to_path(void *hanp, size_t hlen, char *path, size_t pathlen);
-
 #endif
io/parent.c +1 -0
index 8c822cdeb..f5ebb40e4 100644
@@ -8,6 +8,7 @@
 #include "input.h"
 #include "libfrog/paths.h"
 #include "parent.h"
+#include "libfrog/pptrs.h"
 #include "handle.h"
 #include "init.h"
 #include "io.h"
libfrog/Makefile +2 -0
index 011070823..5622ab9b0 100644
@@ -23,6 +23,7 @@ list_sort.c \
 linux.c \
 logging.c \
 paths.c \
+pptrs.c \
 projects.c \
 ptvar.c \
 radix-tree.c \
@@ -42,6 +43,7 @@ crc32table.h \
 fsgeom.h \
 logging.h \
 paths.h \
+pptrs.h \
 projects.h \
 ptvar.h \
 radix-tree.h \
libfrog/pptrs.c renamed from libhandle/parent.c +4 -18
similarity index 87%
rename from libhandle/parent.c
rename to libfrog/pptrs.c
index b511f5b64..96de47b89 100644
@@ -1,21 +1,7 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
 /*
- * Copyright (C) 2017 Oracle.  All Rights Reserved.
- *
- * Author: Darrick J. Wong <[email protected]>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version 2
- * of the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA.
+ * Copyright (C) 2017-2023 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
  */
 #include "platform_defs.h"
 #include "xfs.h"
@@ -23,7 +9,7 @@
 #include "list.h"
 #include "libfrog/paths.h"
 #include "handle.h"
-#include "parent.h"
+#include "libfrog/pptrs.h"
 
 /* Allocate a buffer large enough for some parent pointer records. */
 static inline struct xfs_pptr_info *
libfrog/pptrs.h new file +27 -0
new file mode 100644
index 000000000..d174aa2a5
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2023 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __LIBFROG_PPTRS_H_
+#define	__LIBFROG_PPTRS_H_
+
+struct path_list;
+
+typedef int (*walk_pptr_fn)(struct xfs_pptr_info *pi, struct xfs_parent_ptr *pptr,
+		void *arg);
+typedef int (*walk_ppath_fn)(const char *mntpt, struct path_list *path,
+		void *arg);
+
+#define WALK_PPTRS_ABORT	1
+int fd_walk_pptrs(int fd, walk_pptr_fn fn, void *arg);
+int handle_walk_pptrs(void *hanp, size_t hanlen, walk_pptr_fn fn, void *arg);
+
+#define WALK_PPATHS_ABORT	1
+int fd_walk_ppaths(int fd, walk_ppath_fn fn, void *arg);
+int handle_walk_ppaths(void *hanp, size_t hanlen, walk_ppath_fn fn, void *arg);
+
+int fd_to_path(int fd, char *path, size_t pathlen);
+int handle_to_path(void *hanp, size_t hlen, char *path, size_t pathlen);
+
+#endif /* __LIBFROG_PPTRS_H_ */
libhandle/Makefile +1 -1
index cf7df67c8..f297a59e4 100644
@@ -12,7 +12,7 @@ LT_AGE = 0
 
 LTLDFLAGS += -Wl,--version-script,libhandle.sym
 
-CFILES = handle.c jdm.c parent.c
+CFILES = handle.c jdm.c
 LSRCFILES = libhandle.sym
 
 default: ltdepend $(LTLIBRARY)
#55 lore raw

PATCH 3/7 libfrog: fix indenting errors in xfss_pptr_alloc

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix some indenting problems, and get rid of the xfs_ prefix.

---
 libfrog/pptrs.c |   18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
libfrog/pptrs.c +9 -9
index 96de47b89..6a4f60cf6 100644
@@ -13,16 +13,16 @@
 
 /* Allocate a buffer large enough for some parent pointer records. */
 static inline struct xfs_pptr_info *
-xfs_pptr_alloc(
-      size_t                  bufsize)
+alloc_pptr_buf(
+	size_t			bufsize)
 {
-      struct xfs_pptr_info    *pi;
+	struct xfs_pptr_info	*pi;
 
-      pi = calloc(bufsize, 1);
-      if (!pi)
-              return NULL;
-      pi->pi_ptrs_size = bufsize;
-      return pi;
+	pi = calloc(bufsize, 1);
+	if (!pi)
+		return NULL;
+	pi->pi_ptrs_size = bufsize;
+	return pi;
 }
 
 /* Walk all parents of the given file handle. */
@@ -38,7 +38,7 @@ handle_walk_parents(
 	unsigned int		i;
 	ssize_t			ret = -1;
 
-	pi = xfs_pptr_alloc(XFS_XATTR_LIST_MAX);
+	pi = alloc_pptr_buf(XFS_XATTR_LIST_MAX);
 	if (!pi)
 		return -1;
 
#56 lore raw

PATCH 4/7 libfrog: return positive errno in pptrs.c

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make all the functions in here return 0 for success or positive errno.

---
 io/parent.c     |   12 +++-----
 libfrog/paths.c |   11 +++++--
 libfrog/pptrs.c |   81 +++++++++++++++++++++++++++++++++----------------------
 libfrog/pptrs.h |    6 +---
 4 files changed, 62 insertions(+), 48 deletions(-)
io/parent.c +5 -7
index f5ebb40e4..6c0d81c12 100644
@@ -68,7 +68,7 @@ print_parents(
 	else
 		ret = fd_walk_pptrs(file->fd, pptr_print, args);
 	if (ret)
-		perror(file->name);
+		fprintf(stderr, "%s: %s\n", file->name, strerror(ret));
 
 	return 0;
 }
@@ -106,14 +106,12 @@ path_print(
 	}
 
 	ret = snprintf(buf, len, "%s", mntpt);
-	if (ret != strlen(mntpt)) {
-		errno = ENOMEM;
-		return -1;
-	}
+	if (ret != strlen(mntpt))
+		return ENAMETOOLONG;
 
 	ret = path_list_to_string(path, buf + ret, len - ret);
 	if (ret < 0)
-		return ret;
+		return ENAMETOOLONG;
 
 	printf("%s\n", buf);
 	return 0;
@@ -132,7 +130,7 @@ print_paths(
  	else
 		ret = fd_walk_ppaths(file->fd, path_print, args);
 	if (ret)
-		perror(file->name);
+		fprintf(stderr, "%s: %s\n", file->name, strerror(ret));
 	return 0;
 }
 
libfrog/paths.c +7 -4
index e541e2007..cc43b02c4 100644
@@ -608,7 +608,7 @@ path_component_free(
 	free(pc);
 }
 
-/* Change a path component's filename. */
+/* Change a path component's filename or returns positive errno. */
 int
 path_component_change(
 	struct path_component	*pc,
@@ -620,7 +620,7 @@ path_component_change(
 
 	p = realloc(pc->pc_fname, namelen + 1);
 	if (!p)
-		return -1;
+		return errno;
 	pc->pc_fname = p;
 	memcpy(pc->pc_fname, name, namelen);
 	pc->pc_fname[namelen] = 0;
@@ -628,7 +628,7 @@ path_component_change(
 	return 0;
 }
 
-/* Initialize a pathname. */
+/* Initialize a pathname or returns positive errno. */
 struct path_list *
 path_list_init(void)
 {
@@ -683,7 +683,10 @@ path_list_del_component(
 	list_del_init(&pc->pc_list);
 }
 
-/* Convert a pathname into a string. */
+/*
+ * Convert a pathname into a string or returns -1 if the buffer isn't long
+ * enough.
+ */
 ssize_t
 path_list_to_string(
 	struct path_list	*path,
libfrog/pptrs.c +48 -33

204 lines not on this page · raw message

libfrog/pptrs.h +2 -4
index d174aa2a5..1666de060 100644
@@ -8,16 +8,14 @@
 
 struct path_list;
 
-typedef int (*walk_pptr_fn)(struct xfs_pptr_info *pi, struct xfs_parent_ptr *pptr,
-		void *arg);
+typedef int (*walk_pptr_fn)(struct xfs_pptr_info *pi,
+		struct xfs_parent_ptr *pptr, void *arg);
 typedef int (*walk_ppath_fn)(const char *mntpt, struct path_list *path,
 		void *arg);
 
-#define WALK_PPTRS_ABORT	1
 int fd_walk_pptrs(int fd, walk_pptr_fn fn, void *arg);
 int handle_walk_pptrs(void *hanp, size_t hanlen, walk_pptr_fn fn, void *arg);
 
-#define WALK_PPATHS_ABORT	1
 int fd_walk_ppaths(int fd, walk_ppath_fn fn, void *arg);
 int handle_walk_ppaths(void *hanp, size_t hanlen, walk_ppath_fn fn, void *arg);
 
#57 lore raw

PATCH 5/7 libfrog: only walk one parent pointer at a time in handle_walk_parent_path_ptr

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

handle_walk_parents already walks each returned parent pointer record, so we don't need a loop in handle_walk_parent_path_ptr.

---
 libfrog/pptrs.c |   24 ++++++++++--------------
 1 file changed, 10 insertions(+), 14 deletions(-)
libfrog/pptrs.c +10 -14
index ac7fcea28..b61a4e005 100644
@@ -133,26 +133,22 @@ handle_walk_parent_path_ptr(
 {
 	struct walk_ppath_level_info	*wpli = arg;
 	struct walk_ppaths_info		*wpi = wpli->wpi;
-	unsigned int			i;
 	int				ret = 0;
 
 	if (pi->pi_flags & XFS_PPTR_OFLAG_ROOT)
 		return wpi->fn(wpi->mntpt, wpi->path, wpi->arg);
 
-	for (i = 0; i < pi->pi_count; i++) {
-		p = xfs_ppinfo_to_pp(pi, i);
-		ret = path_component_change(wpli->pc, p->xpp_name,
+	ret = path_component_change(wpli->pc, p->xpp_name,
 				strlen((char *)p->xpp_name), p->xpp_ino);
-		if (ret)
-			break;
-		wpli->newhandle.ha_fid.fid_ino = p->xpp_ino;
-		wpli->newhandle.ha_fid.fid_gen = p->xpp_gen;
-		path_list_add_parent_component(wpi->path, wpli->pc);
-		ret = handle_walk_parent_paths(wpi, &wpli->newhandle);
-		path_list_del_component(wpi->path, wpli->pc);
-		if (ret)
-			break;
-	}
+	if (ret)
+		return ret;
+
+	wpli->newhandle.ha_fid.fid_ino = p->xpp_ino;
+	wpli->newhandle.ha_fid.fid_gen = p->xpp_gen;
+
+	path_list_add_parent_component(wpi->path, wpli->pc);
+	ret = handle_walk_parent_paths(wpi, &wpli->newhandle);
+	path_list_del_component(wpi->path, wpli->pc);
 
 	return ret;
 }
#58 lore raw

PATCH 6/7 libfrog: trim trailing slashes when printing pptr paths

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Trim the trailing slashes in the mountpoint string when we're printing parent pointer paths.

---
 io/parent.c     |    9 +++++++--
 libfrog/pptrs.c |    9 +++++++--
 2 files changed, 14 insertions(+), 4 deletions(-)
io/parent.c +7 -2
index 6c0d81c12..36522f262 100644
@@ -97,6 +97,7 @@ path_print(
 	struct pptr_args	*args = arg;
 	char			buf[PATH_MAX];
 	size_t			len = PATH_MAX;
+	int			mntpt_len = strlen(mntpt);
 	int			ret;
 
 	if (args->filter_ino || args->filter_name) {
@@ -105,8 +106,12 @@ path_print(
 			return 0;
 	}
 
-	ret = snprintf(buf, len, "%s", mntpt);
-	if (ret != strlen(mntpt))
+	/* Trim trailing slashes from the mountpoint */
+	while (mntpt_len > 0 && mntpt[mntpt_len - 1] == '/')
+		mntpt_len--;
+
+	ret = snprintf(buf, len, "%.*s", mntpt_len, mntpt);
+	if (ret != mntpt_len)
 		return ENAMETOOLONG;
 
 	ret = path_list_to_string(path, buf + ret, len - ret);
libfrog/pptrs.c +7 -2
index b61a4e005..488682738 100644
@@ -267,10 +267,15 @@ handle_to_path_walk(
 	void			*arg)
 {
 	struct path_walk_info	*pwi = arg;
+	int			mntpt_len = strlen(mntpt);
 	int			ret;
 
-	ret = snprintf(pwi->buf, pwi->len, "%s", mntpt);
-	if (ret != strlen(mntpt))
+	/* Trim trailing slashes from the mountpoint */
+	while (mntpt_len > 0 && mntpt[mntpt_len - 1] == '/')
+		mntpt_len--;
+
+	ret = snprintf(pwi->buf, pwi->len, "%.*s", mntpt_len, mntpt);
+	if (ret != mntpt_len)
 		return ENAMETOOLONG;
 
 	ret = path_list_to_string(path, pwi->buf + ret, pwi->len - ret);
#59 lore raw

PATCH 7/7 libfrog: fix a buffer overrun in path_list_to_string

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix a buffer overrun when converting a path list to a string.

---
 libfrog/paths.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
libfrog/paths.c +6 -1
index cc43b02c4..42e000295 100644
@@ -694,13 +694,18 @@ path_list_to_string(
 	size_t			buflen)
 {
 	struct path_component	*pos;
+	char			*buf_end = buf + buflen;
 	ssize_t			bytes = 0;
 	int			ret;
 
 	list_for_each_entry(pos, &path->p_head, pc_list) {
+		if (buf >= buf_end)
+			return -1;
+
 		ret = snprintf(buf, buflen, "/%s", pos->pc_fname);
-		if (ret != 1 + strlen(pos->pc_fname))
+		if (ret < 0 || ret >= buflen)
 			return -1;
+
 		bytes += ret;
 		buf += ret;
 		buflen -= ret;
#60 lore raw

cover xfsprogs: bug fixes for parent pointers

In reply to Darrick J. Wong

Hi all,

This series contains the accumulated bug fixes from Darrick to make fstests pass and online repair work.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-bugfixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-bugfixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-bugfixes

---
 io/parent.c              |   22 +++++++--------
 libfrog/pptrs.c          |   36 ++++++++++++------------
 libfrog/pptrs.h          |    4 +--
 libxfs/libxfs_api_defs.h |    3 +-
 libxfs/xfs_da_format.h   |   11 +++++++
 libxfs/xfs_fs.h          |   69 +++++++++++++++++++++-------------------------
 libxfs/xfs_parent.c      |   47 +++++++++++++++++++++++++++----
 libxfs/xfs_parent.h      |   24 +++++++---------
 mkfs/proto.c             |   12 ++++----
 9 files changed, 131 insertions(+), 97 deletions(-)
#61 lore raw

PATCH 1/5 xfs: rename xfs_pptr_info to xfs_getparents

From: Darrick J. Wong <[email protected]>

Rename the head structure of the parent pointer ioctl to match the name of the ioctl (XFS_IOC_GETPARENTS).

---
 io/parent.c     |    4 ++--
 libfrog/pptrs.c |   24 ++++++++++++------------
 libfrog/pptrs.h |    2 +-
 libxfs/xfs_fs.h |   45 +++++++++++++++++++++++----------------------
 4 files changed, 38 insertions(+), 37 deletions(-)
io/parent.c +2 -2
index 36522f262..1c1453f2c 100644
@@ -24,14 +24,14 @@ struct pptr_args {
 
 static int
 pptr_print(
-	struct xfs_pptr_info	*pi,
+	struct xfs_getparents	*pi,
 	struct xfs_parent_ptr	*pptr,
 	void			*arg)
 {
 	struct pptr_args	*args = arg;
 	unsigned int		namelen;
 
-	if (pi->pi_flags & XFS_PPTR_OFLAG_ROOT) {
+	if (pi->gp_flags & XFS_GETPARENTS_OFLAG_ROOT) {
 		printf(_("Root directory.\n"));
 		return 0;
 	}
libfrog/pptrs.c +12 -12
index 488682738..e292ced19 100644
@@ -12,16 +12,16 @@
 #include "libfrog/pptrs.h"
 
 /* Allocate a buffer large enough for some parent pointer records. */
-static inline struct xfs_pptr_info *
+static inline struct xfs_getparents *
 alloc_pptr_buf(
 	size_t			bufsize)
 {
-	struct xfs_pptr_info	*pi;
+	struct xfs_getparents	*pi;
 
 	pi = calloc(bufsize, 1);
 	if (!pi)
 		return NULL;
-	pi->pi_ptrs_size = bufsize;
+	pi->gp_ptrs_size = bufsize;
 	return pi;
 }
 
@@ -36,7 +36,7 @@ handle_walk_parents(
 	walk_pptr_fn		fn,
 	void			*arg)
 {
-	struct xfs_pptr_info	*pi;
+	struct xfs_getparents	*pi;
 	struct xfs_parent_ptr	*p;
 	unsigned int		i;
 	ssize_t			ret = -1;
@@ -46,25 +46,25 @@ handle_walk_parents(
 		return errno;
 
 	if (handle) {
-		memcpy(&pi->pi_handle, handle, sizeof(struct xfs_handle));
-		pi->pi_flags = XFS_PPTR_IFLAG_HANDLE;
+		memcpy(&pi->gp_handle, handle, sizeof(struct xfs_handle));
+		pi->gp_flags = XFS_GETPARENTS_IFLAG_HANDLE;
 	}
 
 	ret = ioctl(fd, XFS_IOC_GETPARENTS, pi);
 	while (!ret) {
-		if (pi->pi_flags & XFS_PPTR_OFLAG_ROOT) {
+		if (pi->gp_flags & XFS_GETPARENTS_OFLAG_ROOT) {
 			ret = fn(pi, NULL, arg);
 			goto out_pi;
 		}
 
-		for (i = 0; i < pi->pi_count; i++) {
-			p = xfs_ppinfo_to_pp(pi, i);
+		for (i = 0; i < pi->gp_count; i++) {
+			p = xfs_getparents_rec(pi, i);
 			ret = fn(pi, p, arg);
 			if (ret)
 				goto out_pi;
 		}
 
-		if (pi->pi_flags & XFS_PPTR_OFLAG_DONE)
+		if (pi->gp_flags & XFS_GETPARENTS_OFLAG_DONE)
 			break;
 
 		ret = ioctl(fd, XFS_IOC_GETPARENTS, pi);
@@ -127,7 +127,7 @@ static int handle_walk_parent_paths(struct walk_ppaths_info *wpi,
 
 static int
 handle_walk_parent_path_ptr(
-	struct xfs_pptr_info		*pi,
+	struct xfs_getparents		*pi,
 	struct xfs_parent_ptr		*p,
 	void				*arg)
 {
@@ -135,7 +135,7 @@ handle_walk_parent_path_ptr(
 	struct walk_ppaths_info		*wpi = wpli->wpi;
 	int				ret = 0;
 
-	if (pi->pi_flags & XFS_PPTR_OFLAG_ROOT)
+	if (pi->gp_flags & XFS_GETPARENTS_OFLAG_ROOT)
 		return wpi->fn(wpi->mntpt, wpi->path, wpi->arg);
 
 	ret = path_component_change(wpli->pc, p->xpp_name,
libfrog/pptrs.h +1 -1
index 1666de060..ab1d0f2fa 100644
@@ -8,7 +8,7 @@
 
 struct path_list;
 
-typedef int (*walk_pptr_fn)(struct xfs_pptr_info *pi,
+typedef int (*walk_pptr_fn)(struct xfs_getparents *pi,
 		struct xfs_parent_ptr *pptr, void *arg);
 typedef int (*walk_ppath_fn)(const char *mntpt, struct path_list *path,
 		void *arg);
libxfs/xfs_fs.h +23 -22

91 lines not on this page · raw message

#62 lore raw

PATCH 2/5 xfs: rename xfs_parent_ptr

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Change the name to xfs_getparents_rec so that the name matches the head structure.

---
 io/parent.c     |   18 +++++++++---------
 libfrog/pptrs.c |   12 ++++++------
 libfrog/pptrs.h |    2 +-
 libxfs/xfs_fs.h |   20 ++++++++++----------
 4 files changed, 26 insertions(+), 26 deletions(-)
io/parent.c +9 -9
index 1c1453f2c..7b9eed8b8 100644
@@ -25,7 +25,7 @@ struct pptr_args {
 static int
 pptr_print(
 	struct xfs_getparents	*pi,
-	struct xfs_parent_ptr	*pptr,
+	struct xfs_getparents_rec	*pptr,
 	void			*arg)
 {
 	struct pptr_args	*args = arg;
@@ -36,21 +36,21 @@ pptr_print(
 		return 0;
 	}
 
-	if (args->filter_ino && pptr->xpp_ino != args->filter_ino)
+	if (args->filter_ino && pptr->gpr_ino != args->filter_ino)
 		return 0;
-	if (args->filter_name && strcmp(args->filter_name, pptr->xpp_name))
+	if (args->filter_name && strcmp(args->filter_name, pptr->gpr_name))
 		return 0;
 
-	namelen = strlen(pptr->xpp_name);
+	namelen = strlen(pptr->gpr_name);
 	if (args->shortformat) {
 		printf("%llu/%u/%u/%s\n",
-			(unsigned long long)pptr->xpp_ino,
-			(unsigned int)pptr->xpp_gen, namelen, pptr->xpp_name);
+			(unsigned long long)pptr->gpr_ino,
+			(unsigned int)pptr->gpr_gen, namelen, pptr->gpr_name);
 	} else {
-		printf(_("p_ino    = %llu\n"), (unsigned long long)pptr->xpp_ino);
-		printf(_("p_gen    = %u\n"), (unsigned int)pptr->xpp_gen);
+		printf(_("p_ino    = %llu\n"), (unsigned long long)pptr->gpr_ino);
+		printf(_("p_gen    = %u\n"), (unsigned int)pptr->gpr_gen);
 		printf(_("p_reclen = %u\n"), namelen);
-		printf(_("p_name   = \"%s\"\n\n"), pptr->xpp_name);
+		printf(_("p_name   = \"%s\"\n\n"), pptr->gpr_name);
 	}
 	return 0;
 }
libfrog/pptrs.c +6 -6
index e292ced19..eff994df8 100644
@@ -37,7 +37,7 @@ handle_walk_parents(
 	void			*arg)
 {
 	struct xfs_getparents	*pi;
-	struct xfs_parent_ptr	*p;
+	struct xfs_getparents_rec	*p;
 	unsigned int		i;
 	ssize_t			ret = -1;
 
@@ -128,7 +128,7 @@ static int handle_walk_parent_paths(struct walk_ppaths_info *wpi,
 static int
 handle_walk_parent_path_ptr(
 	struct xfs_getparents		*pi,
-	struct xfs_parent_ptr		*p,
+	struct xfs_getparents_rec	*p,
 	void				*arg)
 {
 	struct walk_ppath_level_info	*wpli = arg;
@@ -138,13 +138,13 @@ handle_walk_parent_path_ptr(
 	if (pi->gp_flags & XFS_GETPARENTS_OFLAG_ROOT)
 		return wpi->fn(wpi->mntpt, wpi->path, wpi->arg);
 
-	ret = path_component_change(wpli->pc, p->xpp_name,
-				strlen((char *)p->xpp_name), p->xpp_ino);
+	ret = path_component_change(wpli->pc, p->gpr_name,
+				strlen((char *)p->gpr_name), p->gpr_ino);
 	if (ret)
 		return ret;
 
-	wpli->newhandle.ha_fid.fid_ino = p->xpp_ino;
-	wpli->newhandle.ha_fid.fid_gen = p->xpp_gen;
+	wpli->newhandle.ha_fid.fid_ino = p->gpr_ino;
+	wpli->newhandle.ha_fid.fid_gen = p->gpr_gen;
 
 	path_list_add_parent_component(wpi->path, wpli->pc);
 	ret = handle_walk_parent_paths(wpi, &wpli->newhandle);
libfrog/pptrs.h +1 -1
index ab1d0f2fa..05aaea60b 100644
@@ -9,7 +9,7 @@
 struct path_list;
 
 typedef int (*walk_pptr_fn)(struct xfs_getparents *pi,
-		struct xfs_parent_ptr *pptr, void *arg);
+		struct xfs_getparents_rec *pptr, void *arg);
 typedef int (*walk_ppath_fn)(const char *mntpt, struct path_list *path,
 		void *arg);
 
libxfs/xfs_fs.h +10 -10
index c34303a39..c8edc7c09 100644
@@ -768,12 +768,12 @@ struct xfs_scrub_metadata {
 					 XFS_GETPARENTS_OFLAG_DONE)
 
 /* Get an inode parent pointer through ioctl */
-struct xfs_parent_ptr {
-	__u64		xpp_ino;			/* Inode */
-	__u32		xpp_gen;			/* Inode generation */
-	__u32		xpp_diroffset;			/* Directory offset */
-	__u64		xpp_rsvd;			/* Reserved */
-	__u8		xpp_name[];			/* File name */
+struct xfs_getparents_rec {
+	__u64		gpr_ino;			/* Inode */
+	__u32		gpr_gen;			/* Inode generation */
+	__u32		gpr_diroffset;			/* Directory offset */
+	__u64		gpr_rsvd;			/* Reserved */
+	__u8		gpr_name[];			/* File name */
 };
 
 /* Iterate through an inodes parent pointers */
@@ -811,15 +811,15 @@ static inline size_t
 xfs_getparents_sizeof(int nr_ptrs)
 {
 	return sizeof(struct xfs_getparents) +
-	       (nr_ptrs * sizeof(struct xfs_parent_ptr));
+	       (nr_ptrs * sizeof(struct xfs_getparents_rec));
 }
 
-static inline struct xfs_parent_ptr*
+static inline struct xfs_getparents_rec*
 xfs_getparents_rec(
 	struct xfs_getparents	*info,
 	int			idx)
 {
-	return (struct xfs_parent_ptr *)((char *)info + info->gp_offsets[idx]);
+	return (struct xfs_getparents_rec *)((char *)info + info->gp_offsets[idx]);
 }
 
 /*
@@ -867,7 +867,7 @@ xfs_getparents_rec(
 /*	XFS_IOC_GETFSMAP ------ hoisted 59         */
 #define XFS_IOC_SCRUB_METADATA	_IOWR('X', 60, struct xfs_scrub_metadata)
 #define XFS_IOC_AG_GEOMETRY	_IOWR('X', 61, struct xfs_ag_geometry)
-#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_parent_ptr)
+#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_getparents_rec)
 
 /*
  * ioctl commands that replace IRIX syssgi()'s
#63 lore raw

PATCH 3/5 xfs: fix GETPARENTS ioctl

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix a few remaining issues with this ioctl:

The ioctl encodes the size of the parent rec, not the parent head.

The parent rec should say that it returns a null terminated filename.

The parent head encodes the buffer size, not the size of the parent record array, but the field name and documentation doesn't make this clear.

The getparents sizeof function is pointless and wrong.

Get rid of the last vestiges of the non-flex-array definitions.

The rec address should take an unsigned argument

Whitespace damage

---
 libfrog/pptrs.c |    2 +-
 libxfs/xfs_fs.h |   30 +++++++++++-------------------
 2 files changed, 12 insertions(+), 20 deletions(-)
libfrog/pptrs.c +1 -1
index eff994df8..f3465941d 100644
@@ -21,7 +21,7 @@ alloc_pptr_buf(
 	pi = calloc(bufsize, 1);
 	if (!pi)
 		return NULL;
-	pi->gp_ptrs_size = bufsize;
+	pi->gp_bufsize = bufsize;
 	return pi;
 }
 
libxfs/xfs_fs.h +11 -19
index c8edc7c09..d7e061089 100644
@@ -752,8 +752,6 @@ struct xfs_scrub_metadata {
 				 XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED)
 #define XFS_SCRUB_FLAGS_ALL	(XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT)
 
-#define XFS_GETPARENTS_MAXNAMELEN	256
-
 /* return parents of the handle, not the open fd */
 #define XFS_GETPARENTS_IFLAG_HANDLE	(1U << 0)
 
@@ -769,11 +767,11 @@ struct xfs_scrub_metadata {
 
 /* Get an inode parent pointer through ioctl */
 struct xfs_getparents_rec {
-	__u64		gpr_ino;			/* Inode */
-	__u32		gpr_gen;			/* Inode generation */
-	__u32		gpr_diroffset;			/* Directory offset */
-	__u64		gpr_rsvd;			/* Reserved */
-	__u8		gpr_name[];			/* File name */
+	__u64		gpr_ino;	/* Inode number */
+	__u32		gpr_gen;	/* Inode generation */
+	__u32		gpr_diroffset;	/* Directory offset */
+	__u64		gpr_rsvd;	/* Reserved */
+	__u8		gpr_name[];	/* File name and null terminator */
 };
 
 /* Iterate through an inodes parent pointers */
@@ -794,8 +792,8 @@ struct xfs_getparents {
 	/* Must be set to zero */
 	__u32				gp_reserved;
 
-	/* size of the trailing buffer in bytes */
-	__u32				gp_ptrs_size;
+	/* Size of the buffer in bytes, including this header */
+	__u32				gp_bufsize;
 
 	/* # of entries filled in (output) */
 	__u32				gp_count;
@@ -807,19 +805,13 @@ struct xfs_getparents {
 	__u32				gp_offsets[];
 };
 
-static inline size_t
-xfs_getparents_sizeof(int nr_ptrs)
-{
-	return sizeof(struct xfs_getparents) +
-	       (nr_ptrs * sizeof(struct xfs_getparents_rec));
-}
-
 static inline struct xfs_getparents_rec*
 xfs_getparents_rec(
 	struct xfs_getparents	*info,
-	int			idx)
+	unsigned int		idx)
 {
-	return (struct xfs_getparents_rec *)((char *)info + info->gp_offsets[idx]);
+	return (struct xfs_getparents_rec *)((char *)info +
+					     info->gp_offsets[idx]);
 }
 
 /*
@@ -867,7 +859,7 @@ xfs_getparents_rec(
 /*	XFS_IOC_GETFSMAP ------ hoisted 59         */
 #define XFS_IOC_SCRUB_METADATA	_IOWR('X', 60, struct xfs_scrub_metadata)
 #define XFS_IOC_AG_GEOMETRY	_IOWR('X', 61, struct xfs_ag_geometry)
-#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_getparents_rec)
+#define XFS_IOC_GETPARENTS	_IOWR('X', 62, struct xfs_getparents)
 
 /*
  * ioctl commands that replace IRIX syssgi()'s
#64 lore raw

PATCH 4/5 xfs: shorten parent pointer function names

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Shorten the function names and add brief comments to each, outlining what they're supposed to be doing.

---
 libxfs/libxfs_api_defs.h |    2 +-
 libxfs/xfs_parent.c      |   18 ++++++++++++------
 libxfs/xfs_parent.h      |   24 ++++++++++++------------
 mkfs/proto.c             |   12 ++++++------
 4 files changed, 31 insertions(+), 25 deletions(-)
libxfs/libxfs_api_defs.h +1 -1
index 055d2862a..132e2aca0 100644
@@ -140,7 +140,7 @@
 #define xfs_log_get_max_trans_res	libxfs_log_get_max_trans_res
 #define xfs_log_sb			libxfs_log_sb
 #define xfs_mode_to_ftype		libxfs_mode_to_ftype
-#define xfs_parent_defer_add		libxfs_parent_defer_add
+#define xfs_parent_add			libxfs_parent_add
 #define xfs_parent_finish		libxfs_parent_finish
 #define xfs_parent_start		libxfs_parent_start
 #define xfs_perag_get			libxfs_perag_get
libxfs/xfs_parent.c +12 -6
index 74c7f1f7d..89eb531ff 100644
@@ -136,6 +136,10 @@ xfs_parent_irec_from_disk(
 	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
 }
 
+/*
+ * Allocate memory to control a logged parent pointer update as part of a
+ * dirent operation.
+ */
 int
 __xfs_parent_init(
 	struct xfs_mount		*mp,
@@ -171,12 +175,13 @@ __xfs_parent_init(
 	return 0;
 }
 
+/* Add a parent pointer to reflect a dirent addition. */
 int
-xfs_parent_defer_add(
+xfs_parent_add(
 	struct xfs_trans	*tp,
 	struct xfs_parent_defer	*parent,
 	struct xfs_inode	*dp,
-	struct xfs_name		*parent_name,
+	const struct xfs_name	*parent_name,
 	xfs_dir2_dataptr_t	diroffset,
 	struct xfs_inode	*child)
 {
@@ -195,8 +200,9 @@ xfs_parent_defer_add(
 	return xfs_attr_defer_add(args);
 }
 
+/* Remove a parent pointer to reflect a dirent removal. */
 int
-xfs_parent_defer_remove(
+xfs_parent_remove(
 	struct xfs_trans	*tp,
 	struct xfs_inode	*dp,
 	struct xfs_parent_defer	*parent,
@@ -212,14 +218,14 @@ xfs_parent_defer_remove(
 	return xfs_attr_defer_remove(args);
 }
 
-
+/* Replace one parent pointer with another to reflect a rename. */
 int
-xfs_parent_defer_replace(
+xfs_parent_replace(
 	struct xfs_trans	*tp,
 	struct xfs_parent_defer	*new_parent,
 	struct xfs_inode	*old_dp,
 	xfs_dir2_dataptr_t	old_diroffset,
-	struct xfs_name		*parent_name,
+	const struct xfs_name	*parent_name,
 	struct xfs_inode	*new_dp,
 	xfs_dir2_dataptr_t	new_diroffset,
 	struct xfs_inode	*child)
libxfs/xfs_parent.h +12 -12
index f4f5887d1..35854e968 100644
@@ -49,8 +49,9 @@ struct xfs_parent_defer {
  * Parent pointer attribute prototypes
  */
 void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
-			      struct xfs_inode *ip,
-			      uint32_t p_diroffset);
+		struct xfs_inode *ip, uint32_t p_diroffset);
+void xfs_init_parent_name_irec(struct xfs_parent_name_irec *irec,
+			       struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
@@ -78,18 +79,17 @@ xfs_parent_start_locked(
 	return 0;
 }
 
-int xfs_parent_defer_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
-			 struct xfs_inode *dp, struct xfs_name *parent_name,
-			 xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
-int xfs_parent_defer_replace(struct xfs_trans *tp,
+int xfs_parent_add(struct xfs_trans *tp, struct xfs_parent_defer *parent,
+		struct xfs_inode *dp, const struct xfs_name *parent_name,
+		xfs_dir2_dataptr_t diroffset, struct xfs_inode *child);
+int xfs_parent_replace(struct xfs_trans *tp,
 		struct xfs_parent_defer *new_parent, struct xfs_inode *old_dp,
-		xfs_dir2_dataptr_t old_diroffset, struct xfs_name *parent_name,
-		struct xfs_inode *new_ip, xfs_dir2_dataptr_t new_diroffset,
+		xfs_dir2_dataptr_t old_diroffset,
+		const struct xfs_name *parent_name, struct xfs_inode *new_ip,
+		xfs_dir2_dataptr_t new_diroffset, struct xfs_inode *child);
+int xfs_parent_remove(struct xfs_trans *tp, struct xfs_inode *dp,
+		struct xfs_parent_defer *parent, xfs_dir2_dataptr_t diroffset,
 		struct xfs_inode *child);
-int xfs_parent_defer_remove(struct xfs_trans *tp, struct xfs_inode *dp,
-			    struct xfs_parent_defer *parent,
-			    xfs_dir2_dataptr_t diroffset,
-			    struct xfs_inode *child);
 
 void __xfs_parent_cancel(struct xfs_mount *mp, struct xfs_parent_defer *parent);
 
mkfs/proto.c +6 -6
index e0131df50..b8d7ac968 100644
@@ -508,8 +508,8 @@ parseproto(
 		newdirent(mp, tp, pip, &xname, ip->i_ino, &offset);
 		libxfs_trans_log_inode(tp, ip, flags);
 		if (parent) {
-			error = -libxfs_parent_defer_add(tp, parent, pip,
-					&xname, offset, ip);
+			error = -libxfs_parent_add(tp, parent, pip, &xname,
+					offset, ip);
 			if (error)
 				fail(_("committing parent pointers failed."),
 						error);
@@ -601,8 +601,8 @@ parseproto(
 		newdirectory(mp, tp, ip, pip);
 		libxfs_trans_log_inode(tp, ip, flags);
 		if (parent) {
-			error = -libxfs_parent_defer_add(tp, parent, pip,
-					&xname, offset, ip);
+			error = -libxfs_parent_add(tp, parent, pip, &xname,
+					offset, ip);
 			if (error)
 				fail(_("committing parent pointers failed."),
 						error);
@@ -636,8 +636,8 @@ parseproto(
 	}
 	libxfs_trans_log_inode(tp, ip, flags);
 	if (parent) {
-		error = -libxfs_parent_defer_add(tp, parent, pip, &xname,
-				offset, ip);
+		error = -libxfs_parent_add(tp, parent, pip, &xname, offset,
+				ip);
 		if (error)
 			fail(_("committing parent pointers failed."), error);
 	}
#65 lore raw

PATCH 5/5 xfs: rearrange bits of the parent pointer apis for fsck

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rearrange parts of this thing in preparation for fsck code.

---
 libxfs/libxfs_api_defs.h |    1 -
 libxfs/xfs_da_format.h   |   11 +++++++++++
 libxfs/xfs_parent.c      |   29 ++++++++++++++++++++++++++++-
 libxfs/xfs_parent.h      |    6 ++----
 4 files changed, 41 insertions(+), 6 deletions(-)
libxfs/libxfs_api_defs.h +0 -1
index 132e2aca0..60d3339a7 100644
@@ -124,7 +124,6 @@
 #define xfs_initialize_perag		libxfs_initialize_perag
 #define xfs_initialize_perag_data	libxfs_initialize_perag_data
 #define xfs_init_local_fork		libxfs_init_local_fork
-#define xfs_init_parent_name_rec	libxfs_init_parent_name_rec
 
 #define xfs_inobt_maxrecs		libxfs_inobt_maxrecs
 #define xfs_inobt_stage_cursor		libxfs_inobt_stage_cursor
libxfs/xfs_da_format.h +11 -0
index 2db1cf97b..c07b8166e 100644
@@ -159,6 +159,17 @@ struct xfs_da3_intnode {
 
 #define XFS_DIR3_FT_MAX			9
 
+#define XFS_DIR3_FTYPE_STR \
+	{ XFS_DIR3_FT_UNKNOWN,	"unknown" }, \
+	{ XFS_DIR3_FT_REG_FILE,	"file" }, \
+	{ XFS_DIR3_FT_DIR,	"directory" }, \
+	{ XFS_DIR3_FT_CHRDEV,	"char" }, \
+	{ XFS_DIR3_FT_BLKDEV,	"block" }, \
+	{ XFS_DIR3_FT_FIFO,	"fifo" }, \
+	{ XFS_DIR3_FT_SOCK,	"sock" }, \
+	{ XFS_DIR3_FT_SYMLINK,	"symlink" }, \
+	{ XFS_DIR3_FT_WHT,	"whiteout" }
+
 /*
  * Byte offset in data block and shortform entry.
  */
libxfs/xfs_parent.c +28 -1
index 89eb531ff..980f0b829 100644
@@ -92,7 +92,7 @@ xfs_parent_valuecheck(
 }
 
 /* Initializes a xfs_parent_name_rec to be stored as an attribute name */
-void
+static inline void
 xfs_init_parent_name_rec(
 	struct xfs_parent_name_rec	*rec,
 	struct xfs_inode		*ip,
@@ -136,6 +136,33 @@ xfs_parent_irec_from_disk(
 	memset(&irec->p_name[valuelen], 0, sizeof(irec->p_name) - valuelen);
 }
 
+/*
+ * Convert an incore parent_name record to its ondisk format.  If @value or
+ * @valuelen are NULL, they will not be written to.
+ */
+void
+xfs_parent_irec_to_disk(
+	struct xfs_parent_name_rec	*rec,
+	void				*value,
+	int				*valuelen,
+	const struct xfs_parent_name_irec *irec)
+{
+	rec->p_ino = cpu_to_be64(irec->p_ino);
+	rec->p_gen = cpu_to_be32(irec->p_gen);
+	rec->p_diroffset = cpu_to_be32(irec->p_diroffset);
+
+	if (valuelen) {
+		ASSERT(*valuelen > 0);
+		ASSERT(*valuelen >= irec->p_namelen);
+		ASSERT(*valuelen < MAXNAMELEN);
+
+		*valuelen = irec->p_namelen;
+	}
+
+	if (value)
+		memcpy(value, irec->p_name, irec->p_namelen);
+}
+
 /*
  * Allocate memory to control a logged parent pointer update as part of a
  * dirent operation.
libxfs/xfs_parent.h +2 -4
index 35854e968..4eb92fb4b 100644
@@ -33,6 +33,8 @@ struct xfs_parent_name_irec {
 void xfs_parent_irec_from_disk(struct xfs_parent_name_irec *irec,
 		const struct xfs_parent_name_rec *rec,
 		const void *value, int valuelen);
+void xfs_parent_irec_to_disk(struct xfs_parent_name_rec *rec, void *value,
+		int *valuelen, const struct xfs_parent_name_irec *irec);
 
 /*
  * Dynamically allocd structure used to wrap the needed data to pass around
@@ -48,10 +50,6 @@ struct xfs_parent_defer {
 /*
  * Parent pointer attribute prototypes
  */
-void xfs_init_parent_name_rec(struct xfs_parent_name_rec *rec,
-		struct xfs_inode *ip, uint32_t p_diroffset);
-void xfs_init_parent_name_irec(struct xfs_parent_name_irec *irec,
-			       struct xfs_parent_name_rec *rec);
 int __xfs_parent_init(struct xfs_mount *mp, bool grab_log,
 		struct xfs_parent_defer **parentp);
 
#66 lore raw

cover xfs_logprint: clean up attri/pptr dumping

In reply to Darrick J. Wong

Hi all,

This series starts by reverting all the recent changes to log_redo.c because there's a lot of duplicated code and overly complicated pointer handling. Next, we address a few missing pieces in the attri log item decoding, and finish the series by adding proper decoding of parent pointers embedded in an xattri log intent item.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-logprint-cleanups

---
 libxfs/libxfs_api_defs.h |    1 
 logprint/log_redo.c      |  266 +++++++++++++++++-----------------------------
 logprint/logprint.h      |    5 -
 3 files changed, 102 insertions(+), 170 deletions(-)
#67 lore raw

PATCH 1/4 xfs: revert "xfsprogs: Print pptrs in ATTRI items"

From: Darrick J. Wong <[email protected]>

Parent pointers are stored inside extended attributes, which means that they're a higher level data structure. logprint shouldn't be reporting parent pointers as "ATTRI" items, so let's replace all that with proper helpers that will decode a parent pointer xattri log item.

But first, get rid of all this code that adds a bunch of repetitive logic and unnecessary heap allocations.

---
 logprint/log_redo.c |  193 ++++++---------------------------------------------
 logprint/logprint.h |    5 +
 2 files changed, 26 insertions(+), 172 deletions(-)
logprint/log_redo.c +24 -169
index f7e9c9ad9..b596af02c 100644
@@ -674,31 +674,6 @@ xfs_attri_copy_log_format(
 	return 1;
 }
 
-/* iovec length must be 32-bit aligned */
-static inline size_t ATTR_NVEC_SIZE(size_t size)
-{
-	return round_up(size, sizeof(int32_t));
-}
-
-static int
-xfs_attri_copy_name_format(
-	char                            *buf,
-	uint                            len,
-	struct xfs_parent_name_rec     *dst_attri_fmt)
-{
-	uint dst_len = ATTR_NVEC_SIZE(sizeof(struct xfs_parent_name_rec));
-
-	if (len == dst_len) {
-		memcpy((char *)dst_attri_fmt, buf, len);
-		return 0;
-	}
-
-	fprintf(stderr, _("%s: bad size of attri name format: %u; expected %u\n"),
-		progname, len, dst_len);
-
-	return 1;
-}
-
 int
 xlog_print_trans_attri(
 	char				**ptr,
@@ -739,8 +714,7 @@ xlog_print_trans_attri(
 		(*i)++;
 		head = (xlog_op_header_t *)*ptr;
 		xlog_print_op_header(head, *i, ptr);
-		error = xlog_print_trans_attri_name(ptr, be32_to_cpu(head->oh_len),
-						    src_f->alfi_attr_filter);
+		error = xlog_print_trans_attri_name(ptr, be32_to_cpu(head->oh_len));
 		if (error)
 			goto error;
 	}
@@ -750,8 +724,7 @@ xlog_print_trans_attri(
 		(*i)++;
 		head = (xlog_op_header_t *)*ptr;
 		xlog_print_op_header(head, *i, ptr);
-		error = xlog_print_trans_attri_name(ptr, be32_to_cpu(head->oh_len),
-						    src_f->alfi_attr_filter);
+		error = xlog_print_trans_attri_name(ptr, be32_to_cpu(head->oh_len));
 		if (error)
 			goto error;
 	}
@@ -762,7 +735,7 @@ xlog_print_trans_attri(
 		head = (xlog_op_header_t *)*ptr;
 		xlog_print_op_header(head, *i, ptr);
 		error = xlog_print_trans_attri_value(ptr, be32_to_cpu(head->oh_len),
-				src_f->alfi_value_len, src_f->alfi_attr_filter);
+				src_f->alfi_value_len);
 	}
 error:
 	free(src_f);
@@ -773,45 +746,13 @@ xlog_print_trans_attri(
 int
 xlog_print_trans_attri_name(
 	char				**ptr,
-	uint				src_len,
-	uint				attr_flags)
+	uint				src_len)
 {
-	struct xfs_parent_name_rec	*src_f = NULL;
-	uint				dst_len;
+	printf(_("ATTRI:  name len:%u\n"), src_len);
+	print_or_dump(*ptr, src_len);
 
-	/*
-	 * If this is not a parent pointer, just do a bin dump
-	 */
-	if (!(attr_flags & XFS_ATTR_PARENT)) {
-		printf(_("ATTRI:  name len:%u\n"), src_len);
-		print_or_dump(*ptr, src_len);
-		goto out;
-	}
-
-	dst_len	= ATTR_NVEC_SIZE(sizeof(struct xfs_parent_name_rec));
-	if (dst_len != src_len) {
-		fprintf(stderr, _("%s: bad size of attri name format: %u; expected %u\n"),
-			progname, src_len, dst_len);
-		return 1;
-	}
-
-	/*
-	 * memmove to ensure 8-byte alignment for the long longs in
-	 * xfs_parent_name_rec structure
-	 */
-	if ((src_f = (struct xfs_parent_name_rec *)malloc(src_len)) == NULL) {
-		fprintf(stderr, _("%s: xlog_print_trans_attri_name: malloc failed\n"), progname);
-		exit(1);
-	}
-	memmove((char*)src_f, *ptr, src_len);
-
-	printf(_("ATTRI:  #p_ino: %llu	p_gen: %u, p_diroffset: %u\n"),
-		be64_to_cpu(src_f->p_ino), be32_to_cpu(src_f->p_gen),
-				be32_to_cpu(src_f->p_diroffset));
-
-	free(src_f);
-out:
 	*ptr += src_len;
+
 	return 0;
 }	/* xlog_print_trans_attri */
 
@@ -819,32 +760,15 @@ int
 xlog_print_trans_attri_value(
 	char				**ptr,
 	uint				src_len,
-	int				value_len,
-	uint				attr_flags)
+	int				value_len)
 {
 	int len = min(value_len, src_len);
-	char				*f = NULL;
 
-	/*
-	 * If this is not a parent pointer, just do a bin dump
-	 */
-	if (!(attr_flags & XFS_ATTR_PARENT)) {
-		printf(_("ATTRI:  value len:%u\n"), value_len);
-		print_or_dump(*ptr, min(len, MAX_ATTR_VAL_PRINT));
-		goto out;
-	}
+	printf(_("ATTRI:  value len:%u\n"), value_len);
+	print_or_dump(*ptr, len);
 
-	if ((f = (char *)malloc(src_len)) == NULL) {
-		fprintf(stderr, _("%s: xlog_print_trans_attri: malloc failed\n"), progname);
-		exit(1);
-	}
-
-	memcpy(f, *ptr, value_len);
-	printf(_("ATTRI:  value: %.*s\n"), value_len, f);
-
-	free(f);
-out:
 	*ptr += src_len;
+
 	return 0;
 }	/* xlog_print_trans_attri_value */
 
@@ -855,9 +779,6 @@ xlog_recover_print_attri(
 	struct xfs_attri_log_format	*f, *src_f = NULL;
 	uint				src_len, dst_len;
 
-	struct xfs_parent_name_rec 	*rec, *src_rec = NULL;
-	char				*value, *src_value = NULL;
-
 	int				region = 0;
 
 	src_f = (struct xfs_attri_log_format *)item->ri_buf[0].i_addr;
@@ -882,93 +803,27 @@ xlog_recover_print_attri(
 
 	if (f->alfi_name_len > 0) {
 		region++;
-
-		if (f->alfi_attr_filter & XFS_ATTR_PARENT) {
-			src_rec = (struct xfs_parent_name_rec *)item->ri_buf[region].i_addr;
-			src_len = item->ri_buf[region].i_len;
-
-			dst_len = ATTR_NVEC_SIZE(sizeof(struct xfs_parent_name_rec));
-
-			if ((rec = ((struct xfs_parent_name_rec *)malloc(dst_len))) == NULL) {
-				fprintf(stderr, _("%s: xlog_recover_print_attri: malloc failed\n"),
-					progname);
-				exit(1);
-			}
-			if (xfs_attri_copy_name_format((char *)src_rec, src_len, rec)) {
-				goto out;
-			}
-
-			printf(_("ATTRI:  #inode: %llu     gen: %u, offset: %u\n"),
-				be64_to_cpu(rec->p_ino), be32_to_cpu(rec->p_gen),
-				be32_to_cpu(rec->p_diroffset));
-
-			free(rec);
-		}
-		else {
-			printf(_("ATTRI:  name len:%u\n"), f->alfi_name_len);
-			print_or_dump((char *)item->ri_buf[region].i_addr,
-					f->alfi_name_len);
-		}
+		printf(_("ATTRI:  name len:%u\n"), f->alfi_name_len);
+		print_or_dump((char *)item->ri_buf[region].i_addr,
+			       f->alfi_name_len);
 	}
 
 	if (f->alfi_nname_len > 0) {
 		region++;
-
-		if (f->alfi_attr_filter & XFS_ATTR_PARENT) {
-			src_rec = (struct xfs_parent_name_rec *)item->ri_buf[region].i_addr;
-			src_len = item->ri_buf[region].i_len;
-
-			dst_len = ATTR_NVEC_SIZE(sizeof(struct xfs_parent_name_rec));
-
-			if ((rec = ((struct xfs_parent_name_rec *)malloc(dst_len))) == NULL) {
-				fprintf(stderr, _("%s: xlog_recover_print_attri: malloc failed\n"),
-					progname);
-				exit(1);
-			}
-			if (xfs_attri_copy_name_format((char *)src_rec, src_len, rec)) {
-				goto out;
-			}
-
-			printf(_("ATTRI:  new #inode: %llu     gen: %u, offset: %u\n"),
-				be64_to_cpu(rec->p_ino), be32_to_cpu(rec->p_gen),
-				be32_to_cpu(rec->p_diroffset));
-
-			free(rec);
-		}
-		else {
-			printf(_("ATTRI:  nname len:%u\n"), f->alfi_nname_len);
-			print_or_dump((char *)item->ri_buf[region].i_addr,
-				       f->alfi_nname_len);
-		}
+		printf(_("ATTRI:  nname len:%u\n"), f->alfi_nname_len);
+		print_or_dump((char *)item->ri_buf[region].i_addr,
+			       f->alfi_nname_len);
 	}
 
 	if (f->alfi_value_len > 0) {
+		int len = f->alfi_value_len;
+
+		if (len > MAX_ATTR_VAL_PRINT)
+			len = MAX_ATTR_VAL_PRINT;
+
 		region++;
-
-		if (f->alfi_attr_filter & XFS_ATTR_PARENT) {
-			src_value = (char *)item->ri_buf[region].i_addr;
-
-			if ((value = ((char *)malloc(f->alfi_value_len))) == NULL) {
-				fprintf(stderr, _("%s: xlog_recover_print_attri: malloc failed\n"),
-					progname);
-				exit(1);
-			}
-
-			memcpy((char *)value, (char *)src_value, f->alfi_value_len);
-			printf("ATTRI:  value: %.*s\n", f->alfi_value_len, value);
-
-			free(value);
-		}
-		else {
-			int len = f->alfi_value_len;
-
-			if (len > MAX_ATTR_VAL_PRINT)
-				len = MAX_ATTR_VAL_PRINT;
-
-			printf(_("ATTRI:  value len:%u\n"), f->alfi_value_len);
-			print_or_dump((char *)item->ri_buf[region].i_addr,
-					len);
-		}
+		printf(_("ATTRI:  value len:%u\n"), f->alfi_value_len);
+		print_or_dump((char *)item->ri_buf[region].i_addr, len);
 	}
 
 out:
logprint/logprint.h +2 -3
index b8e1c9328..b4479c240 100644
@@ -59,9 +59,8 @@ extern void xlog_recover_print_bud(struct xlog_recover_item *item);
 #define MAX_ATTR_VAL_PRINT	128
 
 extern int xlog_print_trans_attri(char **ptr, uint src_len, int *i);
-extern int xlog_print_trans_attri_name(char **ptr, uint src_len, uint attr_flags);
-extern int xlog_print_trans_attri_value(char **ptr, uint src_len, int value_len,
-					uint attr_flags);
+extern int xlog_print_trans_attri_name(char **ptr, uint src_len);
+extern int xlog_print_trans_attri_value(char **ptr, uint src_len, int value_len);
 extern void xlog_recover_print_attri(struct xlog_recover_item *item);
 extern int xlog_print_trans_attrd(char **ptr, uint len);
 extern void xlog_recover_print_attrd(struct xlog_recover_item *item);
#68 lore raw

PATCH 2/4 xfs_logprint: print missing attri header fields

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Not sure why logprint doesn't print the op flags, inode, or attr filter fields. Make it do that.

---
 logprint/log_redo.c |   23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
logprint/log_redo.c +18 -5
index b596af02c..a90abc2d8 100644
@@ -705,9 +705,15 @@ xlog_print_trans_attri(
 	memmove((char*)src_f, *ptr, src_len);
 	*ptr += src_len;
 
-	printf(_("ATTRI:  #regs: %d	name_len: %d, nname_len: %d value_len: %d  id: 0x%llx\n"),
-		src_f->alfi_size, src_f->alfi_name_len, src_f->alfi_nname_len,
-		src_f->alfi_value_len, (unsigned long long)src_f->alfi_id);
+	printf(_("ATTRI:  #regs: %d	f: 0x%x, ino: 0x%llx, attr_filter: 0x%x, name_len: %d, newname_len: %d, value_len: %d  id: 0x%llx\n"),
+			src_f->alfi_size,
+			src_f->alfi_op_flags,
+			(unsigned long long)src_f->alfi_ino,
+			src_f->alfi_attr_filter,
+			src_f->alfi_name_len,
+			src_f->alfi_nname_len,
+			src_f->alfi_value_len,
+			(unsigned long long)src_f->alfi_id);
 
 	if (src_f->alfi_name_len > 0) {
 		printf(_("\n"));
@@ -798,8 +804,15 @@ xlog_recover_print_attri(
 	if (xfs_attri_copy_log_format((char*)src_f, src_len, f))
 		goto out;
 
-	printf(_("ATTRI:  #regs: %d	name_len: %d, nname_len:%d, value_len: %d  id: 0x%llx\n"),
-		f->alfi_size, f->alfi_name_len, f->alfi_nname_len, f->alfi_value_len, (unsigned long long)f->alfi_id);
+	printf(_("ATTRI:  #regs: %d	f: 0x%x, ino: 0x%llx, attr_filter: 0x%x, name_len: %d, newname_len:%d, value_len: %d  id: 0x%llx\n"),
+			f->alfi_size,
+			f->alfi_op_flags,
+			(unsigned long long)f->alfi_ino,
+			f->alfi_attr_filter,
+			f->alfi_name_len,
+			f->alfi_nname_len,
+			f->alfi_value_len,
+			(unsigned long long)f->alfi_id);
 
 	if (f->alfi_name_len > 0) {
 		region++;
#69 lore raw

PATCH 3/4 xfs: make logprint note attr names and newnames consistently

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix logprint to print "ATTRI: name" and "ATTRI: newname" consistently.

---
 logprint/log_redo.c |   15 +++++++++------
 logprint/logprint.h |    2 +-
 2 files changed, 10 insertions(+), 7 deletions(-)
logprint/log_redo.c +9 -6
index a90abc2d8..f0d64ae9f 100644
@@ -720,7 +720,8 @@ xlog_print_trans_attri(
 		(*i)++;
 		head = (xlog_op_header_t *)*ptr;
 		xlog_print_op_header(head, *i, ptr);
-		error = xlog_print_trans_attri_name(ptr, be32_to_cpu(head->oh_len));
+		error = xlog_print_trans_attri_name(ptr,
+				be32_to_cpu(head->oh_len), "name");
 		if (error)
 			goto error;
 	}
@@ -730,7 +731,8 @@ xlog_print_trans_attri(
 		(*i)++;
 		head = (xlog_op_header_t *)*ptr;
 		xlog_print_op_header(head, *i, ptr);
-		error = xlog_print_trans_attri_name(ptr, be32_to_cpu(head->oh_len));
+		error = xlog_print_trans_attri_name(ptr,
+				be32_to_cpu(head->oh_len), "newname");
 		if (error)
 			goto error;
 	}
@@ -752,15 +754,16 @@ xlog_print_trans_attri(
 int
 xlog_print_trans_attri_name(
 	char				**ptr,
-	uint				src_len)
+	uint				src_len,
+	const char			*tag)
 {
-	printf(_("ATTRI:  name len:%u\n"), src_len);
+	printf(_("ATTRI:  %s len:%u\n"), tag, src_len);
 	print_or_dump(*ptr, src_len);
 
 	*ptr += src_len;
 
 	return 0;
-}	/* xlog_print_trans_attri */
+}
 
 int
 xlog_print_trans_attri_value(
@@ -823,7 +826,7 @@ xlog_recover_print_attri(
 
 	if (f->alfi_nname_len > 0) {
 		region++;
-		printf(_("ATTRI:  nname len:%u\n"), f->alfi_nname_len);
+		printf(_("ATTRI:  newname len:%u\n"), f->alfi_nname_len);
 		print_or_dump((char *)item->ri_buf[region].i_addr,
 			       f->alfi_nname_len);
 	}
logprint/logprint.h +1 -1
index b4479c240..067226ffb 100644
@@ -59,7 +59,7 @@ extern void xlog_recover_print_bud(struct xlog_recover_item *item);
 #define MAX_ATTR_VAL_PRINT	128
 
 extern int xlog_print_trans_attri(char **ptr, uint src_len, int *i);
-extern int xlog_print_trans_attri_name(char **ptr, uint src_len);
+extern int xlog_print_trans_attri_name(char **ptr, uint src_len, const char *tag);
 extern int xlog_print_trans_attri_value(char **ptr, uint src_len, int value_len);
 extern void xlog_recover_print_attri(struct xlog_recover_item *item);
 extern int xlog_print_trans_attrd(char **ptr, uint len);
#70 lore raw

PATCH 4/4 xfs_logprint: decode parent pointers fully

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Decode logged parent pointers fully when dumping log contents. Between the existing ATTRI: printouts and the new ones introduced here, we can figure out what was stored in each log iovec, as well as the higher level parent pointer that was logged.

---
 libxfs/libxfs_api_defs.h |    1 +
 logprint/log_redo.c      |   63 +++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 63 insertions(+), 1 deletion(-)
libxfs/libxfs_api_defs.h +1 -0
index 60d3339a7..092934935 100644
@@ -141,6 +141,7 @@
 #define xfs_mode_to_ftype		libxfs_mode_to_ftype
 #define xfs_parent_add			libxfs_parent_add
 #define xfs_parent_finish		libxfs_parent_finish
+#define xfs_parent_irec_from_disk	libxfs_parent_irec_from_disk
 #define xfs_parent_start		libxfs_parent_start
 #define xfs_perag_get			libxfs_perag_get
 #define xfs_perag_put			libxfs_perag_put
logprint/log_redo.c +62 -1

122 lines not on this page · raw message

#71 lore raw

cover fstests: adjust tests for xfs parent pointers

In reply to Darrick J. Wong

Hi all,

Adjust fstests as necessary to test the new xfs parent pointers feature. At some point this section needs to grow some specific functionality tests for repair and dumping.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs

---
 common/parent                        |  209 +++++++
 common/populate                      |   38 +
 common/rc                            |    7 
 common/xfs                           |   12 
 doc/group-names.txt                  |    1 
 src/popdir.pl                        |   11 
 tests/generic/050                    |   10 
 tests/generic/050.cfg                |    1 
 tests/generic/050.out.xfsquotaparent |   23 +
 tests/xfs/018                        |    7 
 tests/xfs/021                        |   15 -
 tests/xfs/021.cfg                    |    1 
 tests/xfs/021.out.default            |    0 
 tests/xfs/021.out.parent             |   64 ++
 tests/xfs/122.out                    |    3 
 tests/xfs/191                        |    7 
 tests/xfs/206                        |    3 
 tests/xfs/288                        |    7 
 tests/xfs/306                        |    9 
 tests/xfs/851                        |  116 ++++
 tests/xfs/851.out                    |   69 ++
 tests/xfs/852                        |   69 ++
 tests/xfs/852.out                    | 1002 ++++++++++++++++++++++++++++++++++
 tests/xfs/853                        |   85 +++
 tests/xfs/853.out                    |   14 
 25 files changed, 1772 insertions(+), 11 deletions(-)
 create mode 100644 common/parent
 create mode 100644 tests/generic/050.out.xfsquotaparent
 create mode 100644 tests/xfs/021.cfg
 rename tests/xfs/{021.out => 021.out.default} (100%)
 create mode 100644 tests/xfs/021.out.parent
 create mode 100755 tests/xfs/851
 create mode 100644 tests/xfs/851.out
 create mode 100755 tests/xfs/852
 create mode 100644 tests/xfs/852.out
 create mode 100755 tests/xfs/853
 create mode 100644 tests/xfs/853.out
#72 lore raw

PATCH 1/14 xfs/122: update for parent pointers

From: Darrick J. Wong <[email protected]>

Update test for parent pointers.

---
 tests/xfs/122.out |    3 +++
 tests/xfs/206     |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)
tests/xfs/122.out +3 -0
index 43461e875c..0877f14bf2 100644
@@ -109,7 +109,10 @@ sizeof(struct xfs_legacy_timestamp) = 8
 sizeof(struct xfs_log_dinode) = 176
 sizeof(struct xfs_log_legacy_timestamp) = 8
 sizeof(struct xfs_map_extent) = 32
+sizeof(struct xfs_parent_name_rec) = 16
+sizeof(struct xfs_parent_ptr) = 280
 sizeof(struct xfs_phys_extent) = 16
+sizeof(struct xfs_pptr_info) = 104
 sizeof(struct xfs_refcount_key) = 4
 sizeof(struct xfs_refcount_rec) = 12
 sizeof(struct xfs_rmap_key) = 20
tests/xfs/206 +2 -1
index 904d53deb0..b29edeadf0 100755
@@ -66,7 +66,8 @@ mkfs_filter()
 	    -e "/.*crc=/d" \
 	    -e "/^Default configuration/d" \
 	    -e "/metadir=.*/d" \
-	    -e '/rgcount=/d'
+	    -e '/rgcount=/d' \
+	    -e '/parent=/d'
 }
 
 # mkfs slightly smaller than that, small log for speed.
#73 lore raw

PATCH 2/14 populate: create hardlinks for parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create some hardlinked files so that we can exercise parent pointers.

---
 common/populate |   38 ++++++++++++++++++++++++++++++++++++++
 src/popdir.pl   |   11 +++++++++++
 2 files changed, 49 insertions(+)
common/populate +38 -0
index 389a762329..d52167964c 100644
@@ -376,6 +376,7 @@ _scratch_xfs_populate() {
 	is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")"
 	is_rmapbt="$(_xfs_has_feature "$SCRATCH_MNT" rmapbt -v)"
 	is_reflink="$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)"
+	is_pptr="$(_xfs_has_feature "$SCRATCH_MNT" parent -v)"
 
 	# Reverse-mapping btree
 	if [ $is_rmapbt -gt 0 ]; then
@@ -412,6 +413,43 @@ _scratch_xfs_populate() {
 		cp --reflink=always "${SCRATCH_MNT}/REFCOUNTBT" "${SCRATCH_MNT}/REFCOUNTBT2"
 	fi
 
+	# Parent pointers
+	if [ $is_pptr -gt 0 ]; then
+		echo "+ parent pointers"
+
+		# Create a couple of parent pointers
+		__populate_create_dir "${SCRATCH_MNT}/PPTRS" 1 '' --hardlink --format "two_%d"
+
+		# Create one xattr leaf block of parent pointers
+		nr="$((blksz * 2 / 16))"
+		__populate_create_dir "${SCRATCH_MNT}/PPTRS" ${nr} '' --hardlink --format "many%04d"
+
+		# Create multiple xattr leaf blocks of large parent pointers
+		nr="$((blksz * 16 / 16))"
+		__populate_create_dir "${SCRATCH_MNT}/PPTRS" ${nr} '' --hardlink --format "y%0254d"
+
+		# Create multiple paths to a file
+		local moof="${SCRATCH_MNT}/PPTRS/moofile"
+		touch "${moof}"
+		for ((i = 0; i < 4; i++)); do
+			mkdir -p "${SCRATCH_MNT}/PPTRS/SUB${i}"
+			ln "${moof}" "${SCRATCH_MNT}/PPTRS/SUB${i}/moofile"
+		done
+
+		# Create parent pointers of various lengths
+		touch "${SCRATCH_MNT}/PPTRS/vlength"
+		local len_len
+		local tst
+		local fname
+		ln "${SCRATCH_MNT}/PPTRS/vlength" "${SCRATCH_MNT}/PPTRS/b"
+		for len in 32 64 96 128 160 192 224 250 255; do
+			len_len="${#len}"
+			tst="$(perl -e "print \"b\" x (${len} - (${len_len} + 1))")"
+			fname="v${tst}${len}"
+			ln "${SCRATCH_MNT}/PPTRS/vlength" "${SCRATCH_MNT}/PPTRS/${fname}"
+		done
+	fi
+
 	# Copy some real files (xfs tests, I guess...)
 	echo "+ real files"
 	test $fill -ne 0 && __populate_fill_fs "${SCRATCH_MNT}" 5
src/popdir.pl +11 -0
index dc0c046b7d..950503c621 100755
@@ -17,6 +17,7 @@ GetOptions("start=i" => \$start,
 	   "dir=s" => \$dir,
 	   "remove!" => \$remove,
 	   "help!" => \$help,
+	   "hardlink!" => \$hardlink,
 	   "verbose!" => \$verbose);
 
 
@@ -36,6 +37,7 @@ Options:
   --format=str      printf formatting string for file name ("%08d")
   --verbose         verbose output
   --help            this help screen
+  --hardlink        hardlink subsequent files to the first one created
 EOF
   exit(1) unless defined $help;
   # otherwise...
@@ -51,12 +53,21 @@ $file_mult = 20 if (!defined $file_mult);
 $format = "%08d" if (!defined $format);
 $incr = 1 if (!defined $incr);
 
+if ($hardlink) {
+	$file_mult = 0;
+	$link_fname = sprintf($format, $start);
+}
+
 for ($i = $start; $i <= $end; $i += $incr) {
 	$fname = sprintf($format, $i);
 
 	if ($remove) {
 		$verbose && print "rm $fname\n";
 		unlink($fname) or rmdir($fname) or die("unlink $fname");
+	} elsif ($hardlink && $i > $start) {
+		# hardlink the first file
+		$verbose && print "ln $link_fname $fname\n";
+		link $link_fname, $fname;
 	} elsif ($file_mult == 0 or ($i % $file_mult) == 0) {
 		# create a file
 		$verbose && print "touch $fname\n";
#74 lore raw

PATCH 3/14 xfs/021: adapt golden output files for parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Parent pointers change the xattr structure dramatically, so fix this test to handle them.

---
 common/rc                 |    4 +++
 tests/xfs/021             |   15 +++++++++--
 tests/xfs/021.cfg         |    1 +
 tests/xfs/021.out.default |    0 
 tests/xfs/021.out.parent  |   64 +++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 82 insertions(+), 2 deletions(-)
 create mode 100644 tests/xfs/021.cfg
 rename tests/xfs/{021.out => 021.out.default} (100%)
 create mode 100644 tests/xfs/021.out.parent
common/rc +4 -0
index 2d870d435a..d8e309a5f3 100644
@@ -3312,6 +3312,8 @@ _get_os_name()
 
 _link_out_file_named()
 {
+	test -n "$seqfull" || _fail "need to set seqfull"
+
 	local features=$2
 	local suffix=$(FEATURES="$features" perl -e '
 		my %feathash;
@@ -3347,6 +3349,8 @@ _link_out_file()
 {
 	local features
 
+	test -n "$seqfull" || _fail "need to set seqfull"
+
 	if [ $# -eq 0 ]; then
 		features="$(_get_os_name),$FSTYP"
 		if [ -n "$MOUNT_OPTIONS" ]; then
tests/xfs/021 +13 -2
index 9432e2acb0..ef307fc064 100755
@@ -67,6 +67,13 @@ _scratch_mkfs_xfs >/dev/null \
 echo "*** mount FS"
 _scratch_mount
 
+seqfull=$0
+if _xfs_has_feature $SCRATCH_MNT parent; then
+	_link_out_file "parent"
+else
+	_link_out_file ""
+fi
+
 testfile=$SCRATCH_MNT/testfile
 echo "*** make test file 1"
 
@@ -108,7 +115,10 @@ _scratch_unmount >>$seqres.full 2>&1 \
 echo "*** dump attributes (1)"
 
 _scratch_xfs_db -r -c "inode $inum_1" -c "print a.sfattr"  | \
-	sed -e '/secure = /d' | sed -e '/parent = /d'
+	perl -ne '
+/\.secure/ && next;
+/\.parent/ && next;
+	print unless /^\d+:\[.*/;'
 
 echo "*** dump attributes (2)"
 
@@ -124,10 +134,11 @@ s/info.hdr/info/;
 /hdr.info.uuid/ && next;
 /hdr.info.lsn/ && next;
 /hdr.info.owner/ && next;
+/\.parent/ && next;
 s/^(hdr.info.magic =) 0x3bee/\1 0xfbee/;
 s/^(hdr.firstused =) (\d+)/\1 FIRSTUSED/;
 s/^(hdr.freemap\[0-2] = \[base,size]).*/\1 [FREEMAP..]/;
-s/^(entries\[0-2] = \[hashval,nameidx,incomplete,root,local]).*/\1 [ENTRIES..]/;
+s/^(entries\[0-[23]] = \[hashval,nameidx,incomplete,root,local]).*/\1 [ENTRIES..]/;
 	print unless /^\d+:\[.*/;'
 
 echo "*** done"
tests/xfs/021.cfg new file +1 -0
new file mode 100644
index 0000000000..73b127260c
@@ -0,0 +1 @@
+parent: parent
tests/xfs/021.out.default renamed from tests/xfs/021.out +0 -0
similarity index 100%
rename from tests/xfs/021.out
rename to tests/xfs/021.out.default
tests/xfs/021.out.parent new file +64 -0

64 lines not on this page · raw message

#75 lore raw

PATCH 4/14 generic/050: adapt for parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix this test when quotas and parent pointers are enabled.

---
 tests/generic/050                    |   10 ++++++++++
 tests/generic/050.cfg                |    1 +
 tests/generic/050.out.xfsquotaparent |   23 +++++++++++++++++++++++
 3 files changed, 34 insertions(+)
 create mode 100644 tests/generic/050.out.xfsquotaparent
tests/generic/050 +10 -0
index 0664f8c0e4..b38401b9fc 100755
@@ -36,6 +36,16 @@ elif [ "$FSTYP" = "xfs" ] && echo "$MOUNT_OPTIONS" | grep -q quota ; then
 	# Mounting with quota on XFS requires a writable fs, which means
 	# we expect to fail the ro blockdev test with with EPERM.
 	features="xfsquota"
+
+	if _xfs_has_feature $SCRATCH_DEV parent && \
+	   ! _xfs_has_feature $SCRATCH_DEV realtime; then
+		# If we have quotas and parent pointers enabled, the primary
+		# superblock will be written out with the quota flags set when
+		# the logged xattrs log_incompat feature is set.  Hence the
+		# norecovery mount won't fail due to quota rejecting the
+		# mismatch between the mount qflags and the ondisk ones.
+		features="xfsquotaparent"
+	fi
 fi
 _link_out_file "$features"
 
tests/generic/050.cfg +1 -0
index 1d9d60bc69..85924d117d 100644
@@ -1,2 +1,3 @@
 nojournal: nojournal
 xfsquota: xfsquota
+xfsquotaparent: xfsquotaparent
tests/generic/050.out.xfsquotaparent new file +23 -0
new file mode 100644
index 0000000000..b341aca5be
@@ -0,0 +1,23 @@
+QA output created by 050
+setting device read-only
+mounting read-only block device:
+mount: SCRATCH_MNT: permission denied
+unmounting read-only filesystem
+umount: SCRATCH_DEV: not mounted
+setting device read-write
+mounting read-write block device:
+touch files
+going down:
+unmounting shutdown filesystem:
+setting device read-only
+mounting filesystem that needs recovery on a read-only device:
+mount: device write-protected, mounting read-only
+mount: cannot mount device read-only
+unmounting read-only filesystem
+umount: SCRATCH_DEV: not mounted
+mounting filesystem with -o norecovery on a read-only device:
+mount: device write-protected, mounting read-only
+unmounting read-only filesystem
+setting device read-write
+mounting filesystem that needs recovery with -o ro:
+*** done
#76 lore raw

PATCH 5/14 xfs/018: disable parent pointers for this test

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

This test depends heavily on the xattr formats created for new files. Parent pointers break those assumptions, so force parent pointers off.

---
 tests/xfs/018 |    7 ++++++-
 tests/xfs/191 |    7 ++++++-
 tests/xfs/288 |    7 ++++++-
 3 files changed, 18 insertions(+), 3 deletions(-)
tests/xfs/018 +6 -1
index 1ef51a2e61..34b6e91579 100755
@@ -100,7 +100,12 @@ attr32l="X$attr32k"
 attr64k="$attr32k$attr32k"
 
 echo "*** mkfs"
-_scratch_mkfs >/dev/null
+
+# Parent pointers change the xattr formats sufficiently to break this test.
+# Disable parent pointers if mkfs supports it.
+mkfs_args=()
+$MKFS_XFS_PROG 2>&1 | grep -q parent=0 && mkfs_args+=(-n parent=0)
+_scratch_mkfs "${mkfs_args[@]}" >/dev/null
 
 blk_sz=$(_scratch_xfs_get_sb_field blocksize)
 err_inj_attr_sz=$(( blk_sz / 3 - 50 ))
tests/xfs/191 +6 -1
index 7a02f1be21..0a6c20dad7 100755
@@ -33,7 +33,12 @@ _fixed_by_kernel_commit 7be3bd8856fb "xfs: empty xattr leaf header blocks are no
 _fixed_by_kernel_commit e87021a2bc10 "xfs: use larger in-core attr firstused field and detect overflow"
 _fixed_by_git_commit xfsprogs f50d3462c654 "xfs_repair: ignore empty xattr leaf blocks"
 
-_scratch_mkfs_xfs | _filter_mkfs >$seqres.full 2>$tmp.mkfs
+# Parent pointers change the xattr formats sufficiently to break this test.
+# Disable parent pointers if mkfs supports it.
+mkfs_args=()
+$MKFS_XFS_PROG 2>&1 | grep -q parent=0 && mkfs_args+=(-n parent=0)
+
+_scratch_mkfs_xfs "${mkfs_args[@]}" | _filter_mkfs >$seqres.full 2>$tmp.mkfs
 cat $tmp.mkfs >> $seqres.full
 source $tmp.mkfs
 _scratch_mount
tests/xfs/288 +6 -1
index aa664a266e..6bfc9ac0c8 100755
@@ -19,8 +19,13 @@ _supported_fs xfs
 _require_scratch
 _require_attrs
 
+# Parent pointers change the xattr formats sufficiently to break this test.
+# Disable parent pointers if mkfs supports it.
+mkfs_args=()
+$MKFS_XFS_PROG 2>&1 | grep -q parent=0 && mkfs_args+=(-n parent=0)
+
 # get block size ($dbsize) from the mkfs output
-_scratch_mkfs_xfs 2>/dev/null | _filter_mkfs 2>$tmp.mkfs >/dev/null
+_scratch_mkfs_xfs "${mkfs_args[@]}" 2>/dev/null | _filter_mkfs 2>$tmp.mkfs >/dev/null
 . $tmp.mkfs
 
 _scratch_mount
#77 lore raw

PATCH 6/14 xfs/306: fix formatting failures with parent pointers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

The parent pointers feature isn't supported on tiny 20MB filesystems because the larger directory transactions result in larger minimum log sizes, particularly with nrext64 enabled:

** mkfs failed with extra mkfs options added to " -m rmapbt=0, -i nrext64=1, -n parent=1," by test 306 ** ** attempting to mkfs using only test 306 options: -d size=20m -n size=64k **
max log size 5108 smaller than min log size 5310, filesystem is too small

We don't support 20M filesystems anymore, so bump the filesystem size up to 100M and skip this test if we can't actually format the filesystem. Convert the open-coded punch-alternating logic into a call to that program to reduce execve overhead, which more than makes up having to write 5x as much data to fragment the free space.

---
 tests/xfs/306 |    9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
tests/xfs/306 +4 -5
index b57bf4c0a9..152971cfc3 100755
@@ -23,6 +23,7 @@ _supported_fs xfs
 _require_scratch_nocheck	# check complains about single AG fs
 _require_xfs_io_command "fpunch"
 _require_command $UUIDGEN_PROG uuidgen
+_require_test_program "punch-alternating"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
@@ -30,7 +31,8 @@ unset SCRATCH_RTDEV
 
 # Create a small fs with a large directory block size. We want to fill up the fs
 # quickly and then create multi-fsb dirblocks over fragmented free space.
-_scratch_mkfs_xfs -d size=20m -n size=64k >> $seqres.full 2>&1
+_scratch_mkfs_xfs -d size=100m -n size=64k >> $seqres.full 2>&1 || \
+	_notrun 'could not format tiny scratch fs'
 _scratch_mount
 
 # Fill a source directory with many largish-named files. 1k uuid-named entries
@@ -49,10 +51,7 @@ done
 $XFS_IO_PROG -xc "resblks 16" $SCRATCH_MNT >> $seqres.full 2>&1
 dd if=/dev/zero of=$SCRATCH_MNT/file bs=4k >> $seqres.full 2>&1
 $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/file >> $seqres.full 2>&1
-size=`_get_filesize $SCRATCH_MNT/file`
-for i in $(seq 0 8192 $size); do
-	$XFS_IO_PROG -c "fpunch $i 4k" $SCRATCH_MNT/file >> $seqres.full 2>&1
-done
+$here/src/punch-alternating $SCRATCH_MNT/file
 
 # Replicate the src dir several times into fragmented free space. After one or
 # two dirs, we should have nothing but non-contiguous directory blocks.
#78 lore raw

PATCH 7/14 common: add helpers for parent pointer tests

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Add helper functions in common/parent to parse and verify parent pointers. Also add functions to check that mkfs, kernel, and xfs_io support parent pointers.

---
 common/parent |  198 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 common/rc     |    3 +
 common/xfs    |   12 +++
 3 files changed, 213 insertions(+)
 create mode 100644 common/parent
common/parent new file +198 -0
new file mode 100644
index 0000000000..a0ba7d927a
@@ -0,0 +1,198 @@
+#
+# Parent pointer common functions
+#
+
+#
+# parse_parent_pointer parents parent_inode parent_pointer_name
+#
+# Given a list of parent pointers, find the record that matches
+# the given inode and filename
+#
+# inputs:
+# parents	: A list of parent pointers in the format of:
+#		  inode/generation/name_length/name
+# parent_inode	: The parent inode to search for
+# parent_name	: The parent name to search for
+#
+# outputs:
+# PPINO         : Parent pointer inode
+# PPGEN         : Parent pointer generation
+# PPNAME        : Parent pointer name
+# PPNAME_LEN    : Parent pointer name length
+#
+_parse_parent_pointer()
+{
+	local parents=$1
+	local pino=$2
+	local parent_pointer_name=$3
+
+	local found=0
+
+	# Find the entry that has the same inode as the parent
+	# and parse out the entry info
+	while IFS=\/ read PPINO PPGEN PPNAME_LEN PPNAME; do
+		if [ "$PPINO" != "$pino" ]; then
+			continue
+		fi
+
+		if [ "$PPNAME" != "$parent_pointer_name" ]; then
+			continue
+		fi
+
+		found=1
+		break
+	done <<< $(echo "$parents")
+
+	# Check to see if we found anything
+	# We do not fail the test because we also use this
+	# routine to verify when parent pointers should
+	# be removed or updated  (ie a rename or a move
+	# operation changes your parent pointer)
+	if [ $found -eq "0" ]; then
+		return 1
+	fi
+
+	# Verify the parent pointer name length is correct
+	if [ "$PPNAME_LEN" -ne "${#parent_pointer_name}" ]
+	then
+		echo "*** Bad parent pointer:"\
+			"name:$PPNAME, namelen:$PPNAME_LEN"
+	fi
+
+	#return sucess
+	return 0
+}
+
+#
+# _verify_parent parent_path parent_pointer_name child_path
+#
+# Verify that the given child path lists the given parent as a parent pointer
+# and that the parent pointer name matches the given name
+#
+# Examples:
+#
+# #simple example
+# mkdir testfolder1
+# touch testfolder1/file1
+# verify_parent testfolder1 file1 testfolder1/file1
+#
+# # In this above example, we want to verify that "testfolder1"
+# # appears as a parent pointer of "testfolder1/file1".  Additionally
+# # we verify that the name record of the parent pointer is "file1"
+#
+#
+# #hardlink example
+# mkdir testfolder1
+# mkdir testfolder2
+# touch testfolder1/file1
+# ln testfolder1/file1 testfolder2/file1_ln
+# verify_parent testfolder2 file1_ln testfolder1/file1
+#
+# # In this above example, we want to verify that "testfolder2"
+# # appears as a parent pointer of "testfolder1/file1".  Additionally
+# # we verify that the name record of the parent pointer is "file1_ln"
+#
+_verify_parent()
+{
+	local parent_path=$1
+	local parent_pointer_name=$2
+	local child_path=$3
+
+	local parent_ppath="$parent_path/$parent_pointer_name"
+
+	# Verify parent exists
+	if [ ! -d $SCRATCH_MNT/$parent_path ]; then
+		_fail "$SCRATCH_MNT/$parent_path not found"
+	else
+		echo "*** $parent_path OK"
+	fi
+
+	# Verify child exists
+	if [ ! -f $SCRATCH_MNT/$child_path ]; then
+		_fail "$SCRATCH_MNT/$child_path not found"
+	else
+		echo "*** $child_path OK"
+	fi
+
+	# Verify the parent pointer name exists as a child of the parent
+	if [ ! -f $SCRATCH_MNT/$parent_ppath ]; then
+		_fail "$SCRATCH_MNT/$parent_ppath not found"
+	else
+		echo "*** $parent_ppath OK"
+	fi
+
+	# Get the inodes of both parent and child
+	pino="$(stat -c '%i' $SCRATCH_MNT/$parent_path)"
+	cino="$(stat -c '%i' $SCRATCH_MNT/$child_path)"
+
+	# Get all the parent pointers of the child
+	parents=($($XFS_IO_PROG -x -c \
+	 "parent -f -i $pino -n $parent_pointer_name" $SCRATCH_MNT/$child_path))
+	if [[ $? != 0 ]]; then
+		 _fail "No parent pointers found for $child_path"
+	fi
+
+	# Parse parent pointer output.
+	# This sets PPINO PPGEN PPNAME PPNAME_LEN
+	_parse_parent_pointer $parents $pino $parent_pointer_name
+
+	# If we didnt find one, bail out
+	if [ $? -ne 0 ]; then
+		_fail "No parent pointer record found for $parent_path"\
+			"in $child_path"
+	fi
+
+	# Verify the inode generated by the parent pointer name is
+	# the same as the child inode
+	pppino="$(stat -c '%i' $SCRATCH_MNT/$parent_ppath)"
+	if [ $cino -ne $pppino ]
+	then
+		_fail "Bad parent pointer name value for $child_path."\
+			"$SCRATCH_MNT/$parent_ppath belongs to inode $PPPINO,"\
+			"but should be $cino"
+	fi
+
+	echo "*** Verified parent pointer:"\
+			"name:$PPNAME, namelen:$PPNAME_LEN"
+	echo "*** Parent pointer OK for child $child_path"
+}
+
+#
+# _verify_parent parent_pointer_name pino child_path
+#
+# Verify that the given child path contains no parent pointer entry
+# for the given inode and file name
+#
+_verify_no_parent()
+{
+	local parent_pname=$1
+	local pino=$2
+	local child_path=$3
+
+	# Verify child exists
+	if [ ! -f $SCRATCH_MNT/$child_path ]; then
+		_fail "$SCRATCH_MNT/$child_path not found"
+	else
+		echo "*** $child_path OK"
+	fi
+
+	# Get all the parent pointers of the child
+	local parents=($($XFS_IO_PROG -x -c \
+	 "parent -f -i $pino -n $parent_pname" $SCRATCH_MNT/$child_path))
+	if [[ $? != 0 ]]; then
+		return 0
+	fi
+
+	# Parse parent pointer output.
+	# This sets PPINO PPGEN PPNAME PPNAME_LEN
+	_parse_parent_pointer $parents $pino $parent_pname
+
+	# If we didnt find one, return sucess
+	if [ $? -ne 0 ]; then
+		return 0
+	fi
+
+	_fail "Parent pointer entry found where none should:"\
+			"inode:$PPINO, gen:$PPGEN,"
+			"name:$PPNAME, namelen:$PPNAME_LEN"
+}
common/rc +3 -0
index d8e309a5f3..7cb543098d 100644
@@ -2570,6 +2570,9 @@ _require_xfs_io_command()
 		echo $testio | grep -q "invalid option" && \
 			_notrun "xfs_io $command support is missing"
 		;;
+	"parent")
+		testio=`$XFS_IO_PROG -x -c "parent" $TEST_DIR 2>&1`
+		;;
 	"pwrite")
 		# -N (RWF_NOWAIT) only works with direct vectored I/O writes
 		local pwrite_opts=" "
common/xfs +12 -0
index fec13baec9..68c6dd66f9 100644
@@ -2091,3 +2091,15 @@ _scratch_find_rt_metadir_entry() {
 		grep "${sfkey}.inumber" | awk '{print $1}'
 	return 0
 }
+
+# this test requires the xfs parent pointers feature
+#
+_require_xfs_parent()
+{
+	_scratch_mkfs_xfs_supported -n parent > /dev/null 2>&1 \
+		|| _notrun "mkfs.xfs does not support parent pointers"
+	_scratch_mkfs_xfs -n parent > /dev/null 2>&1
+	_try_scratch_mount >/dev/null 2>&1 \
+		|| _notrun "kernel does not support parent pointers"
+	_scratch_unmount
+}
#79 lore raw

PATCH 8/14 xfs: add parent pointer test

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Add a test to verify basic parent pointers operations (create, move, link, unlink, rename, overwrite).

---
 doc/group-names.txt |    1 +
 tests/xfs/851       |  101 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/851.out   |   59 ++++++++++++++++++++++++++++++
 3 files changed, 161 insertions(+)
 create mode 100755 tests/xfs/851
 create mode 100644 tests/xfs/851.out
doc/group-names.txt +1 -0
index 819039e311..87e0933103 100644
@@ -83,6 +83,7 @@ nfs4_acl		NFSv4 access control lists
 nonsamefs		overlayfs layers on different filesystems
 online_repair		online repair functionality tests
 other			dumping ground, do not add more tests to this group
+parent			Parent pointer tests
 pattern			specific IO pattern tests
 perms			access control and permission checking
 pipe			pipe functionality
tests/xfs/851 new file +101 -0

101 lines not on this page · raw message

tests/xfs/851.out new file +59 -0

59 lines not on this page · raw message

#80 lore raw

PATCH 9/14 xfs: add multi link parent pointer test

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Add a test to verify parent pointers while multiple links to a file are created and removed.

---
 tests/xfs/852     |   69 ++++
 tests/xfs/852.out | 1002 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 1071 insertions(+)
 create mode 100755 tests/xfs/852
 create mode 100644 tests/xfs/852.out
tests/xfs/852 new file +69 -0
new file mode 100755
index 0000000000..4d1be0e945
@@ -0,0 +1,69 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test 852
+#
+# multi link parent pointer test
+#
+. ./common/preamble
+_begin_fstest auto quick parent
+
+# get standard environment, filters and checks
+. ./common/parent
+
+# Modify as appropriate
+_supported_fs xfs
+_require_scratch
+_require_xfs_sysfs debug/larp
+_require_xfs_parent
+_require_xfs_io_command "parent"
+
+# real QA test starts here
+
+# Create a directory tree using a protofile and
+# make sure all inodes created have parent pointers
+
+protofile=$tmp.proto
+
+cat >$protofile <<EOF
+DUMMY1
+0 0
+: root directory
+d--777 3 1
+: a directory
+testfolder1 d--755 3 1
+file1 ---755 3 1 /dev/null
+: done
+$
+EOF
+
+_scratch_mkfs -f -n parent=1 -p $protofile >>$seqresres.full 2>&1 \
+	|| _fail "mkfs failed"
+_check_scratch_fs
+
+_scratch_mount >>$seqres.full 2>&1 \
+	|| _fail "mount failed"
+
+testfolder1="testfolder1"
+file1="file1"
+file1_ln="file1_link"
+
+echo ""
+# Multi link parent pointer test
+NLINKS=100
+for (( j=0; j<$NLINKS; j++ )); do
+	ln $SCRATCH_MNT/$testfolder1/$file1 $SCRATCH_MNT/$testfolder1/$file1_ln.$j
+	_verify_parent "$testfolder1" "$file1_ln.$j" "$testfolder1/$file1"
+	_verify_parent "$testfolder1" "$file1" "$testfolder1/$file1_ln.$j"
+done
+# Multi unlink parent pointer test
+for (( j=$NLINKS-1; j<=0; j-- )); do
+	ino="$(stat -c '%i' $SCRATCH_MNT/$testfolder1/$file1_ln.$j)"
+	rm $SCRATCH_MNT/$testfolder1/$file1_ln.$j
+	_verify_no_parent "$file1_ln.$j" "$ino" "$testfolder1/$file1"
+done
+
+# success, all done
+status=0
+exit
tests/xfs/852.out new file +1002 -0

1,002 lines not on this page · raw message

#81 lore raw

PATCH 10/14 xfs: add parent pointer inject test

In reply to Darrick J. Wong

From: Allison Henderson <[email protected]>

Add a test to verify parent pointers after an error injection and log replay.

---
 tests/xfs/853     |   85 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/853.out |   14 +++++++++
 2 files changed, 99 insertions(+)
 create mode 100755 tests/xfs/853
 create mode 100644 tests/xfs/853.out
tests/xfs/853 new file +85 -0
new file mode 100755
index 0000000000..f17f4b7e9e
@@ -0,0 +1,85 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test 853
+#
+# parent pointer inject test
+#
+. ./common/preamble
+_begin_fstest auto quick parent
+
+# get standard environment, filters and checks
+. ./common/filter
+. ./common/inject
+. ./common/parent
+
+# Modify as appropriate
+_supported_fs xfs
+_require_scratch
+_require_xfs_sysfs debug/larp
+_require_xfs_io_error_injection "larp"
+_require_xfs_parent
+_require_xfs_io_command "parent"
+
+# real QA test starts here
+
+# Create a directory tree using a protofile and
+# make sure all inodes created have parent pointers
+
+protofile=$tmp.proto
+
+cat >$protofile <<EOF
+DUMMY1
+0 0
+: root directory
+d--777 3 1
+: a directory
+testfolder1 d--755 3 1
+$
+: back in the root
+testfolder2 d--755 3 1
+: done
+$
+EOF
+
+_scratch_mkfs -f -n parent=1 -p $protofile >>$seqres.full 2>&1 \
+	|| _fail "mkfs failed"
+_check_scratch_fs
+
+_scratch_mount >>$seqres.full 2>&1 \
+	|| _fail "mount failed"
+
+testfolder1="testfolder1"
+testfolder2="testfolder2"
+file4="file4"
+file5="file5"
+
+echo ""
+
+# Create files
+touch $SCRATCH_MNT/$testfolder1/$file4
+_verify_parent "$testfolder1" "$file4" "$testfolder1/$file4"
+
+# Inject error
+_scratch_inject_error "larp"
+
+# Move files
+mv $SCRATCH_MNT/$testfolder1/$file4 $SCRATCH_MNT/$testfolder2/$file5 2>&1 \
+	| _filter_scratch
+
+# FS should be shut down, touch will fail
+touch $SCRATCH_MNT/$testfolder2/$file5 2>&1 | _filter_scratch
+
+# Remount to replay log
+_scratch_remount_dump_log >> $seqres.full
+
+# FS should be online, touch should succeed
+touch $SCRATCH_MNT/$testfolder2/$file5
+
+# Check files again
+_verify_parent "$testfolder2" "$file5" "$testfolder2/$file5"
+
+# success, all done
+status=0
+exit
tests/xfs/853.out new file +14 -0
new file mode 100644
index 0000000000..56247c1434
@@ -0,0 +1,14 @@
+QA output created by 853
+
+*** testfolder1 OK
+*** testfolder1/file4 OK
+*** testfolder1/file4 OK
+*** Verified parent pointer: name:file4, namelen:5
+*** Parent pointer OK for child testfolder1/file4
+mv: cannot stat 'SCRATCH_MNT/testfolder1/file4': Input/output error
+touch: cannot touch 'SCRATCH_MNT/testfolder2/file5': Input/output error
+*** testfolder2 OK
+*** testfolder2/file5 OK
+*** testfolder2/file5 OK
+*** Verified parent pointer: name:file5, namelen:5
+*** Parent pointer OK for child testfolder2/file5
#82 lore raw

PATCH 11/14 common/parent: add license and copyright

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add the necessary licensing and copyright tags to the new file.

---
 common/parent |    3 +++
 1 file changed, 3 insertions(+)
common/parent +3 -0
index a0ba7d927a..2d52404c39 100644
@@ -1,3 +1,6 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022-2023, Oracle and/or its affiliates.  All Rights Reserved.
 #
 # Parent pointer common functions
 #
#83 lore raw

PATCH 12/14 common/parent: don't _fail on missing parent pointer components

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use echo instead of _fail here so that we run as much of the test as possible. There's no need to stop the test immediately even if the pptr code isn't working.

---
 common/parent |   16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
common/parent +8 -8
index 2d52404c39..8d007bd9ad 100644
@@ -105,21 +105,21 @@ _verify_parent()
 
 	# Verify parent exists
 	if [ ! -d $SCRATCH_MNT/$parent_path ]; then
-		_fail "$SCRATCH_MNT/$parent_path not found"
+		echo "$SCRATCH_MNT/$parent_path not found"
 	else
 		echo "*** $parent_path OK"
 	fi
 
 	# Verify child exists
 	if [ ! -f $SCRATCH_MNT/$child_path ]; then
-		_fail "$SCRATCH_MNT/$child_path not found"
+		echo "$SCRATCH_MNT/$child_path not found"
 	else
 		echo "*** $child_path OK"
 	fi
 
 	# Verify the parent pointer name exists as a child of the parent
 	if [ ! -f $SCRATCH_MNT/$parent_ppath ]; then
-		_fail "$SCRATCH_MNT/$parent_ppath not found"
+		echo "$SCRATCH_MNT/$parent_ppath not found"
 	else
 		echo "*** $parent_ppath OK"
 	fi
@@ -132,7 +132,7 @@ _verify_parent()
 	parents=($($XFS_IO_PROG -x -c \
 	 "parent -f -i $pino -n $parent_pointer_name" $SCRATCH_MNT/$child_path))
 	if [[ $? != 0 ]]; then
-		 _fail "No parent pointers found for $child_path"
+		 echo "No parent pointers found for $child_path"
 	fi
 
 	# Parse parent pointer output.
@@ -141,7 +141,7 @@ _verify_parent()
 
 	# If we didnt find one, bail out
 	if [ $? -ne 0 ]; then
-		_fail "No parent pointer record found for $parent_path"\
+		echo "No parent pointer record found for $parent_path"\
 			"in $child_path"
 	fi
 
@@ -150,7 +150,7 @@ _verify_parent()
 	pppino="$(stat -c '%i' $SCRATCH_MNT/$parent_ppath)"
 	if [ $cino -ne $pppino ]
 	then
-		_fail "Bad parent pointer name value for $child_path."\
+		echo "Bad parent pointer name value for $child_path."\
 			"$SCRATCH_MNT/$parent_ppath belongs to inode $PPPINO,"\
 			"but should be $cino"
 	fi
@@ -174,7 +174,7 @@ _verify_no_parent()
 
 	# Verify child exists
 	if [ ! -f $SCRATCH_MNT/$child_path ]; then
-		_fail "$SCRATCH_MNT/$child_path not found"
+		echo "$SCRATCH_MNT/$child_path not found"
 	else
 		echo "*** $child_path OK"
 	fi
@@ -195,7 +195,7 @@ _verify_no_parent()
 		return 0
 	fi
 
-	_fail "Parent pointer entry found where none should:"\
+	echo "Parent pointer entry found where none should:"\
 			"inode:$PPINO, gen:$PPGEN,"
 			"name:$PPNAME, namelen:$PPNAME_LEN"
 }
#84 lore raw

PATCH 13/14 common/parent: check xfs_io parent command paths

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Make sure that the paths returned by the xfs_io parent command actually point to the same file.

---
 common/parent |    8 ++++++++
 1 file changed, 8 insertions(+)
common/parent +8 -0
index 8d007bd9ad..f849e4b27c 100644
@@ -155,6 +155,14 @@ _verify_parent()
 			"but should be $cino"
 	fi
 
+	# Make sure path printing works by checking that the paths returned
+	# all point to the same inode.
+	local tgt="$SCRATCH_MNT/$child_path"
+	$XFS_IO_PROG -x -c 'parent -p' "$tgt" | while read pptr_path; do
+		test "$tgt" -ef "$pptr_path" || \
+			echo "$tgt parent pointer $pptr_path should be the same file"
+	done
+
 	echo "*** Verified parent pointer:"\
 			"name:$PPNAME, namelen:$PPNAME_LEN"
 	echo "*** Parent pointer OK for child $child_path"
#85 lore raw

PATCH 14/14 xfs/851: test xfs_io parent -p too

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Test the -p argument to the xfs_io parent command too.

---
 tests/xfs/851     |   15 +++++++++++++++
 tests/xfs/851.out |   10 ++++++++++
 2 files changed, 25 insertions(+)
tests/xfs/851 +15 -0
index 27870ec05a..8233c1563c 100755
@@ -12,6 +12,7 @@ _begin_fstest auto quick parent
 
 # get standard environment, filters and checks
 . ./common/parent
+. ./common/filter
 
 # Modify as appropriate
 _supported_fs xfs
@@ -96,6 +97,20 @@ ino="$(stat -c '%i' $SCRATCH_MNT/$testfolder2/$file3)"
 mv -f $SCRATCH_MNT/$testfolder2/$file3 $SCRATCH_MNT/$testfolder1/$file2
 _verify_parent "$testfolder1" "$file2" "$testfolder1/$file2"
 
+# Make sure that parent -p filtering works
+mkdir -p $SCRATCH_MNT/dira/ $SCRATCH_MNT/dirb/
+dira_inum=$(stat -c '%i' $SCRATCH_MNT/dira)
+dirb_inum=$(stat -c '%i' $SCRATCH_MNT/dirb)
+touch $SCRATCH_MNT/gorn
+ln $SCRATCH_MNT/gorn $SCRATCH_MNT/dira/file1
+ln $SCRATCH_MNT/gorn $SCRATCH_MNT/dirb/file1
+echo look for both
+$XFS_IO_PROG -c 'parent -p' $SCRATCH_MNT/gorn | _filter_scratch
+echo look for dira
+$XFS_IO_PROG -c 'parent -p -n dira' -c "parent -p -i $dira_inum" $SCRATCH_MNT/gorn | _filter_scratch
+echo look for dirb
+$XFS_IO_PROG -c 'parent -p -n dirb' -c "parent -p -i $dirb_inum" $SCRATCH_MNT/gorn | _filter_scratch
+
 # success, all done
 status=0
 exit
tests/xfs/851.out +10 -0
index c375ba5f00..f44d3e5d4f 100644
@@ -57,3 +57,13 @@ QA output created by 851
 *** testfolder1/file2 OK
 *** Verified parent pointer: name:file2, namelen:5
 *** Parent pointer OK for child testfolder1/file2
+look for both
+SCRATCH_MNT/gorn
+SCRATCH_MNT/dira/file1
+SCRATCH_MNT/dirb/file1
+look for dira
+SCRATCH_MNT/dira/file1
+SCRATCH_MNT/dira/file1
+look for dirb
+SCRATCH_MNT/dirb/file1
+SCRATCH_MNT/dirb/file1
#86 lore raw

cover xfs: bug fixes for parent pointers

In reply to Darrick J. Wong

Hi all,

This series contains the accumulated bug fixes from Darrick to make fstests pass and online repair work.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-bugfixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-bugfixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-bugfixes

---
 tests/xfs/122.out |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
#87 lore raw

PATCH 1/1 xfs/122: fix parent pointer ioctl structure sizes

From: Darrick J. Wong <[email protected]>

We've renamed (and resized) the structures for the GETPARENTS ioctl, so adjust the golden output here to reflect that.

---
 tests/xfs/122.out |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tests/xfs/122.out +2 -2
index 0877f14bf2..da2da2089c 100644
@@ -100,6 +100,8 @@ sizeof(struct xfs_fsop_ag_resblks) = 64
 sizeof(struct xfs_fsop_geom) = 256
 sizeof(struct xfs_fsop_geom_v1) = 112
 sizeof(struct xfs_fsop_geom_v4) = 112
+sizeof(struct xfs_getparents) = 96
+sizeof(struct xfs_getparents_rec) = 24
 sizeof(struct xfs_icreate_log) = 28
 sizeof(struct xfs_inode_log_format) = 56
 sizeof(struct xfs_inode_log_format_32) = 52
@@ -110,9 +112,7 @@ sizeof(struct xfs_log_dinode) = 176
 sizeof(struct xfs_log_legacy_timestamp) = 8
 sizeof(struct xfs_map_extent) = 32
 sizeof(struct xfs_parent_name_rec) = 16
-sizeof(struct xfs_parent_ptr) = 280
 sizeof(struct xfs_phys_extent) = 16
-sizeof(struct xfs_pptr_info) = 104
 sizeof(struct xfs_refcount_key) = 4
 sizeof(struct xfs_refcount_rec) = 12
 sizeof(struct xfs_rmap_key) = 20
#88 lore raw

cover fstests: encode parent pointer name in xattr key

In reply to Darrick J. Wong

Hi all,

As I've mentioned in past comments on the parent pointers patchset, the proposed ondisk parent pointer format presents a major difficulty for online directory repair. This difficulty derives from encoding the directory offset of the dirent that the parent pointer is mirroring. Recall that parent pointers are stored in extended attributes:

    (parent_ino, parent_gen, diroffset) -> (dirent_name)

If the directory is rebuilt, the offsets of the new directory entries must match the diroffset encoded in the parent pointer, or the filesystem becomes inconsistent. There are a few ways to solve this problem.

One approach would be to augment the directory addname function to take a diroffset and try to create the new entry at that offset. This will not work if the original directory became corrupt and the parent pointers were written out with impossible diroffsets (e.g. overlapping). Requiring matching diroffsets also prevents reorganization and compaction of directories.

This could be remedied by recording the parent pointer diroffset updates necessary to retain consistency, and using the logged parent pointer replace function to rewrite parent pointers as necessary. This is a poor choice from a performance perspective because the logged xattr updates must be committed in the same transaction that commits the new directory structure. If there are a large number of diroffset updates, then the directory commit could take an even longer time.

Worse yet, if the logged xattr updates fill up the transaction, repair will have no choice but to roll to a fresh transaction to continue logging. This breaks repair's policy that repairs should commit atomically. It may break the filesystem as well, since all files involved are pinned until the delayed pptr xattr processing completes. This is a completely bad engineering choice.

Note that the diroffset information is not used anywhere in the directory lookup code. Observe that the only information that we require for a parent pointer is the inverse of an pre-ftype dirent, since this is all we need to reconstruct a directory entry:

    (parent_ino, dirent_name) -> NULL

The xattr code supports xattrs with zero-length values, surprisingly. The parent_gen field makes it easy to export parent handle information, so it can be retained:

    (parent_ino, parent_gen, dirent_name) -> NULL

Moving the ondisk format to this format is very advantageous for repair code. Unfortunately, there is one hitch: xattr names cannot exceed 255 bytes due to ondisk format limitations. We don't want to constrain the length of dirent names, so instead we could use collision resistant hashes to handle dirents with very long names:

    (parent_ino, parent_gen, sha512(dirent_name)) -> (dirent_name)

The first two patches implement this schema. However, this encoding is not maximally efficient, since many directory names are shorter than the length of a sha512 hash. The last three patches in the series bifurcate the parent pointer ondisk format depending on context:

For dirent names shorter than 243 bytes:

    (parent_ino, parent_gen, dirent_name) -> NULL

For dirent names longer than 243 bytes:

    (parent_ino, parent_gen, dirent_name[0:178],
     sha512(child_gen, dirent_name)) -> (dirent_name[179:255])

The child file's generation number is mixed into the sha512 computation to make it a little more difficult for unprivileged userspace to attempt collisions.

A messier solution to this problem would be to extend the xattr ondisk format to allow parent pointers to have xattr names up to 267 bytes. This would likely involve redefining the ondisk namelen field to omit the size of the parent ino/gen information and might be madness.

If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-name-in-attr-key

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-name-in-attr-key

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key

---
 tests/xfs/021.out.parent |   22 ++++++++++------------
 tests/xfs/122.out        |    2 +-
 2 files changed, 11 insertions(+), 13 deletions(-)
#89 lore raw

PATCH 1/1 xfs/{021,122}: adjust parent pointer encoding format

From: Darrick J. Wong <[email protected]>

Adjust these tests to reflect the new parent pointer ondisk format.

---
 tests/xfs/021.out.parent |   22 ++++++++++------------
 tests/xfs/122.out        |    2 +-
 2 files changed, 11 insertions(+), 13 deletions(-)
tests/xfs/021.out.parent +10 -12
index 661d130239..c43dd15900 100644
@@ -19,12 +19,11 @@ size of attr value = 65536
 
 *** unmount FS
 *** dump attributes (1)
-a.sfattr.hdr.totsize = 53
+a.sfattr.hdr.totsize = 49
 a.sfattr.hdr.count = 3
-a.sfattr.list[0].namelen = 16
-a.sfattr.list[0].valuelen = 10
+a.sfattr.list[0].namelen = 22
+a.sfattr.list[0].valuelen = 0
 a.sfattr.list[0].root = 0
-a.sfattr.list[0].value = "testfile.1"
 a.sfattr.list[1].namelen = 2
 a.sfattr.list[1].valuelen = 3
 a.sfattr.list[1].root = 0
@@ -40,7 +39,7 @@ hdr.info.forw = 0
 hdr.info.back = 0
 hdr.info.magic = 0xfbee
 hdr.count = 4
-hdr.usedbytes = 84
+hdr.usedbytes = 80
 hdr.firstused = FIRSTUSED
 hdr.holes = 0
 hdr.freemap[0-2] = [base,size] [FREEMAP..]
@@ -53,12 +52,11 @@ nvlist[1].valueblk = 0x1
 nvlist[1].valuelen = 65535
 nvlist[1].namelen = 2
 nvlist[1].name = "a3"
-nvlist[2].valuelen = 10
-nvlist[2].namelen = 16
-nvlist[2].value = "testfile.2"
-nvlist[3].valuelen = 8
-nvlist[3].namelen = 7
-nvlist[3].name = "a2-----"
-nvlist[3].value = "value_2\d"
+nvlist[2].valuelen = 8
+nvlist[2].namelen = 7
+nvlist[2].name = "a2-----"
+nvlist[2].value = "value_2\d"
+nvlist[3].valuelen = 0
+nvlist[3].namelen = 22
 *** done
 *** unmount
tests/xfs/122.out +1 -1
index da2da2089c..4b7660fac0 100644
@@ -111,7 +111,7 @@ sizeof(struct xfs_legacy_timestamp) = 8
 sizeof(struct xfs_log_dinode) = 176
 sizeof(struct xfs_log_legacy_timestamp) = 8
 sizeof(struct xfs_map_extent) = 32
-sizeof(struct xfs_parent_name_rec) = 16
+sizeof(struct xfs_parent_name_rec) = 12
 sizeof(struct xfs_phys_extent) = 16
 sizeof(struct xfs_refcount_key) = 4
 sizeof(struct xfs_refcount_rec) = 12
#90 lore raw
In reply to Darrick J. Wong
Quoted 70 lines · Darrick J. Wong

Hi everyone,

This deluge contains all of the additions to the parent pointers patchset that I've been working since last month's deluge.  The kernel
and xfsprogs patchsets are based on Allison's v10 tag from last week; the fstests patches are merely a part of my development tree.  To recap
Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to each
inode.  The attribute name containing the parent inode, generation, and
directory offset, while the  attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."

v10r1d2 rebases everything against 6.3-rc2.  I still want to remove the
diroffset from the ondisk parent pointer, but for v10 I've replaced the
sha512 hashing code with modifications to the xattr code to support lookups based on name *and* value.  With that working, we can encode parent pointers like this:

        (parent_ino, parent_gen, name[])

xattr lookups still work correctly, and repair doesn't have to deal with
keeping the diroffsets in sync if the directory gets rebuilt.  With this
change applied, I'm ready to weave my new changes into Allison's v10 and
call parent pointers done. :)

The online directory and parent pointer code are exactly the same as the
v9r2d1 release, so I'm eliding that and everything that was in Allison's
recent v10 patchset.  IOWs, this deluge includes only the bug fixes I've
made to parent pointers, the updates I've made to the ondisk format, and
the necessary changes to fstests to get everything to pass.

If you want to pull the whole thing, use these links:
https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSrGWwDFp$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSmjOh6X7$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSlulOhuJ$

Allison: Could you please resync libxfs in the following patches under
https://urldefense.com/v3/__https://github.com/allisonhenderson/xfsprogs/commits/xfsprogs_new_pptrs_v10__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSqiXa3xN$

please?

xfsprogs: add parent pointer support to attribute code
xfsprogs: extend transaction reservations for parent attributes
xfsprogs: parent pointer attribute creation
xfsprogs: remove parent pointers in unlink
xfsprogs: Add parent pointers to rename
xfsprogs: move/add parent pointer validators to xfs_parent

Quoted 5 lines

There are discrepancies between the two, which makes ./tools/libxfs- diff
unhappy.  Or, if you want me to merge my ondisk format changes into my
branches, I'll put out v11 with everything taken care of.

Sure, will resync, as I recall some of them had to deviate a little bit because the corresponding code appears in different places, or needed special handling.

Originally my intent was just to get the kernel side of things settled and landed first, and then grind through the other spaces since user space is mostly a port. I was trying to avoid sending out giant deluges since people seemed to get hung up enough in just kernel space reviews.

Thanks for all the help tho.

Allison

Quoted 2 lines

--D

#91 lore raw
Quoted 71 lines · Allison Henderson
Quoted 70 lines · Darrick J. Wong

Hi everyone,

This deluge contains all of the additions to the parent pointers patchset that I've been working since last month's deluge.  The kernel
and xfsprogs patchsets are based on Allison's v10 tag from last week; the fstests patches are merely a part of my development tree.  To recap
Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to each
inode.  The attribute name containing the parent inode, generation, and
directory offset, while the  attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."

v10r1d2 rebases everything against 6.3-rc2.  I still want to remove the
diroffset from the ondisk parent pointer, but for v10 I've replaced the
sha512 hashing code with modifications to the xattr code to support lookups based on name *and* value.  With that working, we can encode parent pointers like this:

        (parent_ino, parent_gen, name[])

xattr lookups still work correctly, and repair doesn't have to deal with
keeping the diroffsets in sync if the directory gets rebuilt.  With this
change applied, I'm ready to weave my new changes into Allison's v10 and
call parent pointers done. :)

The online directory and parent pointer code are exactly the same as the
v9r2d1 release, so I'm eliding that and everything that was in Allison's
recent v10 patchset.  IOWs, this deluge includes only the bug fixes I've
made to parent pointers, the updates I've made to the ondisk format, and
the necessary changes to fstests to get everything to pass.

If you want to pull the whole thing, use these links:
https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSrGWwDFp$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSmjOh6X7$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSlulOhuJ$

Allison: Could you please resync libxfs in the following patches under
https://urldefense.com/v3/__https://github.com/allisonhenderson/xfsprogs/commits/xfsprogs_new_pptrs_v10__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSqiXa3xN$

please?

xfsprogs: add parent pointer support to attribute code
xfsprogs: extend transaction reservations for parent attributes
xfsprogs: parent pointer attribute creation
xfsprogs: remove parent pointers in unlink
xfsprogs: Add parent pointers to rename
xfsprogs: move/add parent pointer validators to xfs_parent

Quoted 8 lines
Quoted 5 lines

There are discrepancies between the two, which makes ./tools/libxfs- diff
unhappy.  Or, if you want me to merge my ondisk format changes into my
branches, I'll put out v11 with everything taken care of.

Sure, will resync, as I recall some of them had to deviate a little bit because the corresponding code appears in different places, or needed special handling.

Ok, thank you. It's easier to develop xfsprogs code when libxfs can be kept in sync easily.

Quoted 5 lines

Originally my intent was just to get the kernel side of things settled and landed first, and then grind through the other spaces since user space is mostly a port. I was trying to avoid sending out giant deluges since people seemed to get hung up enough in just kernel space reviews.

Yeah, 'tis true that sending xfsprogs libxfs patches prematurely is just noise on the list. :/

That said, the closer a patchset gets to final review, the tidier the xfsprogs part ought to be. I've added support in repair and the debugger, which means this is /very/ close to its final review.

--D

Quoted 7 lines

Thanks for all the help tho.

Allison

Quoted 2 lines

--D

#92 lore raw
Quoted 99 lines · Darrick J. Wong
Quoted 92 lines · Allison Henderson
Quoted 84 lines · Darrick J. Wong

Hi everyone,

This deluge contains all of the additions to the parent pointers patchset that I've been working since last month's deluge.  The kernel
and xfsprogs patchsets are based on Allison's v10 tag from last week;
the fstests patches are merely a part of my development tree.  To recap
Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to
each
inode.  The attribute name containing the parent inode, generation,
and
directory offset, while the  attribute value contains the file name.
This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."

v10r1d2 rebases everything against 6.3-rc2.  I still want to remove
the
diroffset from the ondisk parent pointer, but for v10 I've replaced
the
sha512 hashing code with modifications to the xattr code to support
lookups based on name *and* value.  With that working, we can encode
parent pointers like this:

        (parent_ino, parent_gen, name[])

xattr lookups still work correctly, and repair doesn't have to deal
with
keeping the diroffsets in sync if the directory gets rebuilt. With
this
change applied, I'm ready to weave my new changes into Allison's v10
and
call parent pointers done. :)

The online directory and parent pointer code are exactly the same as
the
v9r2d1 release, so I'm eliding that and everything that was in Allison's
recent v10 patchset.  IOWs, this deluge includes only the bug fixes
I've
made to parent pointers, the updates I've made to the ondisk format,
and
the necessary changes to fstests to get everything to pass.

If you want to pull the whole thing, use these links:
https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSrGWwDFp$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSmjOh6X7$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSlulOhuJ$

Allison: Could you please resync libxfs in the following patches under
https://urldefense.com/v3/__https://github.com/allisonhenderson/xfsprogs/commits/xfsprogs_new_pptrs_v10__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSqiXa3xN$

please?

xfsprogs: add parent pointer support to attribute code
xfsprogs: extend transaction reservations for parent attributes
xfsprogs: parent pointer attribute creation
xfsprogs: remove parent pointers in unlink
xfsprogs: Add parent pointers to rename
xfsprogs: move/add parent pointer validators to xfs_parent

Quoted 7 lines

There are discrepancies between the two, which makes ./tools/libxfs-
diff
unhappy.  Or, if you want me to merge my ondisk format changes into
my
branches, I'll put out v11 with everything taken care of.

Quoted 5 lines

Sure, will resync, as I recall some of them had to deviate a little bit
because the corresponding code appears in different places, or needed
special handling.

Quoted 3 lines

Ok, thank you.  It's easier to develop xfsprogs code when libxfs can be
kept in sync easily.

Here is a resync, some of them still needed hand porting, but it looks like it sync'd up a lot of white space so let me know if this clears up things for you
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v10r1

Allison

Quoted 27 lines
Quoted 9 lines

Originally my intent was just to get the kernel side of things settled
and landed first, and then grind through the other spaces since user
space is mostly a port.  I was trying to avoid sending out giant deluges since people seemed to get hung up enough in just kernel space
reviews.

Yeah, 'tis true that sending xfsprogs libxfs patches prematurely is just
noise on the list. :/

That said, the closer a patchset gets to final review, the tidier the xfsprogs part ought to be.  I've added support in repair and the debugger, which means this is /very/ close to its final review.

--D

Quoted 7 lines

Thanks for all the help tho.

Allison

Quoted 2 lines

--D

#93 lore raw
Quoted 100 lines · Allison Henderson
Quoted 99 lines · Darrick J. Wong
Quoted 97 lines · Allison Henderson
Quoted 84 lines · Darrick J. Wong

Hi everyone,

This deluge contains all of the additions to the parent pointers patchset that I've been working since last month's deluge.  The kernel
and xfsprogs patchsets are based on Allison's v10 tag from last week;
the fstests patches are merely a part of my development tree.  To recap
Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to
each
inode.  The attribute name containing the parent inode, generation,
and
directory offset, while the  attribute value contains the file name.
This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."

v10r1d2 rebases everything against 6.3-rc2.  I still want to remove
the
diroffset from the ondisk parent pointer, but for v10 I've replaced
the
sha512 hashing code with modifications to the xattr code to support
lookups based on name *and* value.  With that working, we can encode
parent pointers like this:

        (parent_ino, parent_gen, name[])

xattr lookups still work correctly, and repair doesn't have to deal
with
keeping the diroffsets in sync if the directory gets rebuilt. With
this
change applied, I'm ready to weave my new changes into Allison's v10
and
call parent pointers done. :)

The online directory and parent pointer code are exactly the same as
the
v9r2d1 release, so I'm eliding that and everything that was in Allison's
recent v10 patchset.  IOWs, this deluge includes only the bug fixes
I've
made to parent pointers, the updates I've made to the ondisk format,
and
the necessary changes to fstests to get everything to pass.

If you want to pull the whole thing, use these links:
https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSrGWwDFp$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSmjOh6X7$

https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSlulOhuJ$

Allison: Could you please resync libxfs in the following patches under
https://urldefense.com/v3/__https://github.com/allisonhenderson/xfsprogs/commits/xfsprogs_new_pptrs_v10__;!!ACWV5N9M2RV99hQ!MnkBbyDKEdgQiXLfmXZ87uT_j_TAtQHHA1UraPf01op6wNpRZkk2tg5CXru4eL6-pzJyUl-uJAZlSqiXa3xN$

please?

xfsprogs: add parent pointer support to attribute code
xfsprogs: extend transaction reservations for parent attributes
xfsprogs: parent pointer attribute creation
xfsprogs: remove parent pointers in unlink
xfsprogs: Add parent pointers to rename
xfsprogs: move/add parent pointer validators to xfs_parent

Quoted 7 lines

There are discrepancies between the two, which makes ./tools/libxfs-
diff
unhappy.  Or, if you want me to merge my ondisk format changes into
my
branches, I'll put out v11 with everything taken care of.

Sure, will resync, as I recall some of them had to deviate a little bit
because the corresponding code appears in different places, or needed
special handling.

Quoted 8 lines
Quoted 3 lines

Ok, thank you.  It's easier to develop xfsprogs code when libxfs can be
kept in sync easily.

Here is a resync, some of them still needed hand porting, but it looks like it sync'd up a lot of white space so let me know if this clears up things for you
https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v10r1

Yep, thanks. I'll incorporate that into v11. :)

--D

Quoted 30 lines

Allison

Quoted 27 lines
Quoted 9 lines

Originally my intent was just to get the kernel side of things settled
and landed first, and then grind through the other spaces since user
space is mostly a port.  I was trying to avoid sending out giant deluges since people seemed to get hung up enough in just kernel space
reviews.

Yeah, 'tis true that sending xfsprogs libxfs patches prematurely is just
noise on the list. :/

That said, the closer a patchset gets to final review, the tidier the xfsprogs part ought to be.  I've added support in repair and the debugger, which means this is /very/ close to its final review.

--D

Quoted 7 lines

Thanks for all the help tho.

Allison

Quoted 2 lines

--D