Skip to contentSkip to conversation

xfs: fix sf to block inode fork logging

PATCH In review Brian Foster 3 participants 10 messages Oct 7, 2019 – Oct 8, 2019 XFS filesystem

lore
Share & tools
lore
#0 lore raw

v2 cover xfs: fix sf to block inode fork logging

Hi all,

Here's v2 of the directory inode shortform to block logging fixup. This also addresses the similar attr fork conversion function. I put the other cleanups into separate patches because it wasn't totally clear to me if we wanted to add the log call to the conversion function given the other callers log the inode outside of that function. IOW, we can either keep or drop patch 3. I have no strong preference either way.

Brian

v2:
- Also fix up attr fork conversion.
- Add patches 2 and 3 for follow up cleanups.
v1: https://lore.kernel.org/linux-xfs/[email protected]/

Brian Foster (3):
xfs: log the inode on directory sf to block format change
xfs: remove broken error handling on failed attr sf to leaf change
xfs: move local to extent inode logging into bmap helper

 fs/xfs/libxfs/xfs_attr_leaf.c  | 21 +++------------------
 fs/xfs/libxfs/xfs_bmap.c       |  6 ++++--
 fs/xfs/libxfs/xfs_bmap.h       |  3 ++-
 fs/xfs/libxfs/xfs_dir2_block.c |  2 +-
 4 files changed, 10 insertions(+), 22 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 1/3 xfs: log the inode on directory sf to block format change

When a directory changes from shortform (sf) to block format, the sf format is copied to a temporary buffer, the inode format is modified and the updated format filled with the dentries from the temporary buffer. If the inode format is modified and attempt to grow the inode fails (due to I/O error, for example), it is possible to return an error while leaving the directory in an inconsistent state and with an otherwise clean transaction. This results in corruption of the associated directory and leads to xfs_dabuf_map() errors as subsequent lookups cannot accurately determine the format of the directory. This problem is reproduced occasionally by generic/475.

The fundamental problem is that xfs_dir2_sf_to_block() changes the on-disk inode format without logging the inode. The inode is eventually logged by the bmapi layer in the common case, but error checking introduces the possibility of failing the high level request before this happens.

Update both of the dir2 and attr callers of xfs_bmap_local_to_extents_empty() to log the inode core as consistent with the bmap local to extent format change codepath. This ensures that any subsequent errors after the format has changed cause the transaction to abort.

---
 fs/xfs/libxfs/xfs_attr_leaf.c  | 1 +
 fs/xfs/libxfs/xfs_dir2_block.c | 1 +
 2 files changed, 2 insertions(+)
fs/xfs/libxfs/xfs_attr_leaf.c +1 -0
index b9f019603d0b..36c0a32cefcf 100644
@@ -827,6 +827,7 @@ xfs_attr_shortform_to_leaf(
 
 	xfs_idata_realloc(dp, -size, XFS_ATTR_FORK);
 	xfs_bmap_local_to_extents_empty(dp, XFS_ATTR_FORK);
+	xfs_trans_log_inode(args->trans, dp, XFS_ILOG_CORE);
 
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
fs/xfs/libxfs/xfs_dir2_block.c +1 -0
index 9595ced393dc..3d1e5f6d64fd 100644
@@ -1098,6 +1098,7 @@ xfs_dir2_sf_to_block(
 	xfs_idata_realloc(dp, -ifp->if_bytes, XFS_DATA_FORK);
 	xfs_bmap_local_to_extents_empty(dp, XFS_DATA_FORK);
 	dp->i_d.di_size = 0;
+	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	/*
 	 * Add block 0 to the inode.
Signature · 1 line
2.20.1
#4 lore raw

PATCH v2 2/3 xfs: remove broken error handling on failed attr sf to leaf change

In reply to Brian Foster

xfs_attr_shortform_to_leaf() attempts to put the shortform fork back together after a failed attempt to convert from shortform to leaf format. While this code reallocates and copies back the shortform attr fork data, it never resets the inode format field back to local format. Further, now that the inode is properly logged after the initial switch from local format, any error that triggers the recovery code will eventually abort the transaction and shutdown the fs. Therefore, remove the broken and unnecessary error handling code.

---
 fs/xfs/libxfs/xfs_attr_leaf.c | 19 ++-----------------
 1 file changed, 2 insertions(+), 17 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +2 -17
index 36c0a32cefcf..1b956c313b6b 100644
@@ -831,28 +831,13 @@ xfs_attr_shortform_to_leaf(
 
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
-	if (error) {
-		/*
-		 * If we hit an IO error middle of the transaction inside
-		 * grow_inode(), we may have inconsistent data. Bail out.
-		 */
-		if (error == -EIO)
-			goto out;
-		xfs_idata_realloc(dp, size, XFS_ATTR_FORK);	/* try to put */
-		memcpy(ifp->if_u1.if_data, tmpbuffer, size);	/* it back */
+	if (error)
 		goto out;
-	}
 
 	ASSERT(blkno == 0);
 	error = xfs_attr3_leaf_create(args, blkno, &bp);
-	if (error) {
-		/* xfs_attr3_leaf_create may not have instantiated a block */
-		if (bp && (xfs_da_shrink_inode(args, 0, bp) != 0))
-			goto out;
-		xfs_idata_realloc(dp, size, XFS_ATTR_FORK);	/* try to put */
-		memcpy(ifp->if_u1.if_data, tmpbuffer, size);	/* it back */
+	if (error)
 		goto out;
-	}
 
 	memset((char *)&nargs, 0, sizeof(nargs));
 	nargs.dp = dp;
Signature · 1 line
2.20.1
#7 lore raw

PATCH v2 3/3 xfs: move local to extent inode logging into bmap helper

In reply to Brian Foster

The callers of xfs_bmap_local_to_extents_empty() log the inode external to the function, yet this function is where the on-disk format value is updated. Push the inode logging down into the function itself to help prevent future mistakes.

Note that internal bmap callers track the inode logging flags independently and thus may log the inode core twice due to this change. This is harmless, so leave this code around for consistency with the other attr fork conversion functions.

---
 fs/xfs/libxfs/xfs_attr_leaf.c  | 3 +--
 fs/xfs/libxfs/xfs_bmap.c       | 6 ++++--
 fs/xfs/libxfs/xfs_bmap.h       | 3 ++-
 fs/xfs/libxfs/xfs_dir2_block.c | 3 +--
 4 files changed, 8 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +1 -2
index 1b956c313b6b..f0089e862216 100644
@@ -826,8 +826,7 @@ xfs_attr_shortform_to_leaf(
 	sf = (xfs_attr_shortform_t *)tmpbuffer;
 
 	xfs_idata_realloc(dp, -size, XFS_ATTR_FORK);
-	xfs_bmap_local_to_extents_empty(dp, XFS_ATTR_FORK);
-	xfs_trans_log_inode(args->trans, dp, XFS_ILOG_CORE);
+	xfs_bmap_local_to_extents_empty(args->trans, dp, XFS_ATTR_FORK);
 
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
fs/xfs/libxfs/xfs_bmap.c +4 -2
index 4edc25a2ba80..02469d59c787 100644
@@ -792,6 +792,7 @@ xfs_bmap_extents_to_btree(
  */
 void
 xfs_bmap_local_to_extents_empty(
+	struct xfs_trans	*tp,
 	struct xfs_inode	*ip,
 	int			whichfork)
 {
@@ -808,6 +809,7 @@ xfs_bmap_local_to_extents_empty(
 	ifp->if_u1.if_root = NULL;
 	ifp->if_height = 0;
 	XFS_IFORK_FMT_SET(ip, whichfork, XFS_DINODE_FMT_EXTENTS);
+	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
 }
 
 
@@ -840,7 +842,7 @@ xfs_bmap_local_to_extents(
 	ASSERT(XFS_IFORK_FORMAT(ip, whichfork) == XFS_DINODE_FMT_LOCAL);
 
 	if (!ifp->if_bytes) {
-		xfs_bmap_local_to_extents_empty(ip, whichfork);
+		xfs_bmap_local_to_extents_empty(tp, ip, whichfork);
 		flags = XFS_ILOG_CORE;
 		goto done;
 	}
@@ -887,7 +889,7 @@ xfs_bmap_local_to_extents(
 
 	/* account for the change in fork size */
 	xfs_idata_realloc(ip, -ifp->if_bytes, whichfork);
-	xfs_bmap_local_to_extents_empty(ip, whichfork);
+	xfs_bmap_local_to_extents_empty(tp, ip, whichfork);
 	flags |= XFS_ILOG_CORE;
 
 	ifp->if_u1.if_root = NULL;
fs/xfs/libxfs/xfs_bmap.h +2 -1
index 5bb446d80542..e2798c6f3a5f 100644
@@ -182,7 +182,8 @@ void	xfs_trim_extent(struct xfs_bmbt_irec *irec, xfs_fileoff_t bno,
 		xfs_filblks_t len);
 int	xfs_bmap_add_attrfork(struct xfs_inode *ip, int size, int rsvd);
 int	xfs_bmap_set_attrforkoff(struct xfs_inode *ip, int size, int *version);
-void	xfs_bmap_local_to_extents_empty(struct xfs_inode *ip, int whichfork);
+void	xfs_bmap_local_to_extents_empty(struct xfs_trans *tp,
+		struct xfs_inode *ip, int whichfork);
 void	__xfs_bmap_add_free(struct xfs_trans *tp, xfs_fsblock_t bno,
 		xfs_filblks_t len, const struct xfs_owner_info *oinfo,
 		bool skip_discard);
fs/xfs/libxfs/xfs_dir2_block.c +1 -2
index 3d1e5f6d64fd..49e4bc39e7bb 100644
@@ -1096,9 +1096,8 @@ xfs_dir2_sf_to_block(
 	memcpy(sfp, oldsfp, ifp->if_bytes);
 
 	xfs_idata_realloc(dp, -ifp->if_bytes, XFS_DATA_FORK);
-	xfs_bmap_local_to_extents_empty(dp, XFS_DATA_FORK);
+	xfs_bmap_local_to_extents_empty(tp, dp, XFS_DATA_FORK);
 	dp->i_d.di_size = 0;
-	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	/*
 	 * Add block 0 to the inode.
Signature · 1 line
2.20.1
Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 1/3] xfs: log the inode on directory sf to block format change

In reply to Brian Foster
Quoted 17 lines · Brian Foster

When a directory changes from shortform (sf) to block format, the sf format is copied to a temporary buffer, the inode format is modified and the updated format filled with the dentries from the temporary buffer. If the inode format is modified and attempt to grow the inode fails (due to I/O error, for example), it is possible to return an error while leaving the directory in an inconsistent state and with an otherwise clean transaction. This results in corruption of the associated directory and leads to xfs_dabuf_map() errors as subsequent lookups cannot accurately determine the format of the directory. This problem is reproduced occasionally by generic/475.

The fundamental problem is that xfs_dir2_sf_to_block() changes the on-disk inode format without logging the inode. The inode is eventually logged by the bmapi layer in the common case, but error checking introduces the possibility of failing the high level request before this happens.

Quoted 7 lines

Update both of the dir2 and attr callers of xfs_bmap_local_to_extents_empty() to log the inode core as consistent with the bmap local to extent format change codepath. This ensures that any subsequent errors after the format has changed cause the transaction to abort.

Looks good,

Reviewed-by on v2 2/3
#5 lore raw

Re: [PATCH v2 2/3] xfs: remove broken error handling on failed attr sf to leaf change

In reply to Brian Foster
Quoted 11 lines · Brian Foster

xfs_attr_shortform_to_leaf() attempts to put the shortform fork back together after a failed attempt to convert from shortform to leaf format. While this code reallocates and copies back the shortform attr fork data, it never resets the inode format field back to local format. Further, now that the inode is properly logged after the initial switch from local format, any error that triggers the recovery code will eventually abort the transaction and shutdown the fs. Therefore, remove the broken and unnecessary error handling code.

Looks good,

Reviewed-by on v2 3/3
#8 lore raw

Re: [PATCH v2 3/3] xfs: move local to extent inode logging into bmap helper

In reply to Brian Foster
Quoted 11 lines · Brian Foster

The callers of xfs_bmap_local_to_extents_empty() log the inode external to the function, yet this function is where the on-disk format value is updated. Push the inode logging down into the function itself to help prevent future mistakes.

Note that internal bmap callers track the inode logging flags independently and thus may log the inode core twice due to this change. This is harmless, so leave this code around for consistency with the other attr fork conversion functions.

Looks good,

Reviewed-by on v2 1/3
#3 lore raw

Re: [PATCH v2 1/3] xfs: log the inode on directory sf to block format change

In reply to Brian Foster
Quoted 17 lines · Brian Foster

When a directory changes from shortform (sf) to block format, the sf format is copied to a temporary buffer, the inode format is modified and the updated format filled with the dentries from the temporary buffer. If the inode format is modified and attempt to grow the inode fails (due to I/O error, for example), it is possible to return an error while leaving the directory in an inconsistent state and with an otherwise clean transaction. This results in corruption of the associated directory and leads to xfs_dabuf_map() errors as subsequent lookups cannot accurately determine the format of the directory. This problem is reproduced occasionally by generic/475.

The fundamental problem is that xfs_dir2_sf_to_block() changes the on-disk inode format without logging the inode. The inode is eventually logged by the bmapi layer in the common case, but error checking introduces the possibility of failing the high level request before this happens.

Quoted 7 lines

Update both of the dir2 and attr callers of xfs_bmap_local_to_extents_empty() to log the inode core as consistent with the bmap local to extent format change codepath. This ensures that any subsequent errors after the format has changed cause the transaction to abort.

Looks ok,

--D

Quoted 32 lines
---
 fs/xfs/libxfs/xfs_attr_leaf.c  | 1 +
 fs/xfs/libxfs/xfs_dir2_block.c | 1 +
 2 files changed, 2 insertions(+)
fs/xfs/libxfs/xfs_attr_leaf.c +1 -0
index b9f019603d0b..36c0a32cefcf 100644
@@ -827,6 +827,7 @@ xfs_attr_shortform_to_leaf(
 
 	xfs_idata_realloc(dp, -size, XFS_ATTR_FORK);
 	xfs_bmap_local_to_extents_empty(dp, XFS_ATTR_FORK);
+	xfs_trans_log_inode(args->trans, dp, XFS_ILOG_CORE);
 
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
fs/xfs/libxfs/xfs_dir2_block.c +1 -0
index 9595ced393dc..3d1e5f6d64fd 100644
@@ -1098,6 +1098,7 @@ xfs_dir2_sf_to_block(
 	xfs_idata_realloc(dp, -ifp->if_bytes, XFS_DATA_FORK);
 	xfs_bmap_local_to_extents_empty(dp, XFS_DATA_FORK);
 	dp->i_d.di_size = 0;
+	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	/*
 	 * Add block 0 to the inode.
Signature · 1 line
2.20.1
Reviewed-by on v2 2/3
#6 lore raw

Re: [PATCH v2 2/3] xfs: remove broken error handling on failed attr sf to leaf change

In reply to Brian Foster
Quoted 11 lines · Brian Foster

xfs_attr_shortform_to_leaf() attempts to put the shortform fork back together after a failed attempt to convert from shortform to leaf format. While this code reallocates and copies back the shortform attr fork data, it never resets the inode format field back to local format. Further, now that the inode is properly logged after the initial switch from local format, any error that triggers the recovery code will eventually abort the transaction and shutdown the fs. Therefore, remove the broken and unnecessary error handling code.

Looks ok,

--D

Quoted 42 lines
---
 fs/xfs/libxfs/xfs_attr_leaf.c | 19 ++-----------------
 1 file changed, 2 insertions(+), 17 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +2 -17
index 36c0a32cefcf..1b956c313b6b 100644
@@ -831,28 +831,13 @@ xfs_attr_shortform_to_leaf(
 
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
-	if (error) {
-		/*
-		 * If we hit an IO error middle of the transaction inside
-		 * grow_inode(), we may have inconsistent data. Bail out.
-		 */
-		if (error == -EIO)
-			goto out;
-		xfs_idata_realloc(dp, size, XFS_ATTR_FORK);	/* try to put */
-		memcpy(ifp->if_u1.if_data, tmpbuffer, size);	/* it back */
+	if (error)
 		goto out;
-	}
 
 	ASSERT(blkno == 0);
 	error = xfs_attr3_leaf_create(args, blkno, &bp);
-	if (error) {
-		/* xfs_attr3_leaf_create may not have instantiated a block */
-		if (bp && (xfs_da_shrink_inode(args, 0, bp) != 0))
-			goto out;
-		xfs_idata_realloc(dp, size, XFS_ATTR_FORK);	/* try to put */
-		memcpy(ifp->if_u1.if_data, tmpbuffer, size);	/* it back */
+	if (error)
 		goto out;
-	}
 
 	memset((char *)&nargs, 0, sizeof(nargs));
 	nargs.dp = dp;
Signature · 1 line
2.20.1
Reviewed-by on v2 3/3
#9 lore raw

Re: [PATCH v2 3/3] xfs: move local to extent inode logging into bmap helper

In reply to Brian Foster
Quoted 11 lines · Brian Foster

The callers of xfs_bmap_local_to_extents_empty() log the inode external to the function, yet this function is where the on-disk format value is updated. Push the inode logging down into the function itself to help prevent future mistakes.

Note that internal bmap callers track the inode logging flags independently and thus may log the inode core twice due to this change. This is harmless, so leave this code around for consistency with the other attr fork conversion functions.

Looks ok,

--D

Quoted 91 lines
---
 fs/xfs/libxfs/xfs_attr_leaf.c  | 3 +--
 fs/xfs/libxfs/xfs_bmap.c       | 6 ++++--
 fs/xfs/libxfs/xfs_bmap.h       | 3 ++-
 fs/xfs/libxfs/xfs_dir2_block.c | 3 +--
 4 files changed, 8 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +1 -2
index 1b956c313b6b..f0089e862216 100644
@@ -826,8 +826,7 @@ xfs_attr_shortform_to_leaf(
 	sf = (xfs_attr_shortform_t *)tmpbuffer;
 
 	xfs_idata_realloc(dp, -size, XFS_ATTR_FORK);
-	xfs_bmap_local_to_extents_empty(dp, XFS_ATTR_FORK);
-	xfs_trans_log_inode(args->trans, dp, XFS_ILOG_CORE);
+	xfs_bmap_local_to_extents_empty(args->trans, dp, XFS_ATTR_FORK);
 
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
fs/xfs/libxfs/xfs_bmap.c +4 -2
index 4edc25a2ba80..02469d59c787 100644
@@ -792,6 +792,7 @@ xfs_bmap_extents_to_btree(
  */
 void
 xfs_bmap_local_to_extents_empty(
+	struct xfs_trans	*tp,
 	struct xfs_inode	*ip,
 	int			whichfork)
 {
@@ -808,6 +809,7 @@ xfs_bmap_local_to_extents_empty(
 	ifp->if_u1.if_root = NULL;
 	ifp->if_height = 0;
 	XFS_IFORK_FMT_SET(ip, whichfork, XFS_DINODE_FMT_EXTENTS);
+	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
 }
 
 
@@ -840,7 +842,7 @@ xfs_bmap_local_to_extents(
 	ASSERT(XFS_IFORK_FORMAT(ip, whichfork) == XFS_DINODE_FMT_LOCAL);
 
 	if (!ifp->if_bytes) {
-		xfs_bmap_local_to_extents_empty(ip, whichfork);
+		xfs_bmap_local_to_extents_empty(tp, ip, whichfork);
 		flags = XFS_ILOG_CORE;
 		goto done;
 	}
@@ -887,7 +889,7 @@ xfs_bmap_local_to_extents(
 
 	/* account for the change in fork size */
 	xfs_idata_realloc(ip, -ifp->if_bytes, whichfork);
-	xfs_bmap_local_to_extents_empty(ip, whichfork);
+	xfs_bmap_local_to_extents_empty(tp, ip, whichfork);
 	flags |= XFS_ILOG_CORE;
 
 	ifp->if_u1.if_root = NULL;
fs/xfs/libxfs/xfs_bmap.h +2 -1
index 5bb446d80542..e2798c6f3a5f 100644
@@ -182,7 +182,8 @@ void	xfs_trim_extent(struct xfs_bmbt_irec *irec, xfs_fileoff_t bno,
 		xfs_filblks_t len);
 int	xfs_bmap_add_attrfork(struct xfs_inode *ip, int size, int rsvd);
 int	xfs_bmap_set_attrforkoff(struct xfs_inode *ip, int size, int *version);
-void	xfs_bmap_local_to_extents_empty(struct xfs_inode *ip, int whichfork);
+void	xfs_bmap_local_to_extents_empty(struct xfs_trans *tp,
+		struct xfs_inode *ip, int whichfork);
 void	__xfs_bmap_add_free(struct xfs_trans *tp, xfs_fsblock_t bno,
 		xfs_filblks_t len, const struct xfs_owner_info *oinfo,
 		bool skip_discard);
fs/xfs/libxfs/xfs_dir2_block.c +1 -2
index 3d1e5f6d64fd..49e4bc39e7bb 100644
@@ -1096,9 +1096,8 @@ xfs_dir2_sf_to_block(
 	memcpy(sfp, oldsfp, ifp->if_bytes);
 
 	xfs_idata_realloc(dp, -ifp->if_bytes, XFS_DATA_FORK);
-	xfs_bmap_local_to_extents_empty(dp, XFS_DATA_FORK);
+	xfs_bmap_local_to_extents_empty(tp, dp, XFS_DATA_FORK);
 	dp->i_d.di_size = 0;
-	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	/*
 	 * Add block 0 to the inode.
Signature · 1 line
2.20.1