Skip to contentSkip to conversation

xfs: online repair, second part of part 1

PATCH New Darrick J. Wong 2 participants 151 messages Nov 24, 2023 – Dec 6, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v28 1/1 xfs: online repair, second part of part 1

Hi everyone,

[this time not as a reply to v27]

[[this time really not as a reply to v27]]

[[[***** email, this way is insane]]]

I've rebased the online fsck development branches atop 6.7, applied the changes requested during the review of v27, and reworked the automatic space reaping code to avoid open-coding EFI log item handling, and cleaned up a few other things.

In other words, I'm formally submitting part 1 for inclusion in 6.8.

Just like the last several submissions, I would like people to focus the following:

- Are the major subsystems sufficiently documented that you could figure out what the code does?

- Do you see any problems that are severe enough to cause long term support hassles? (e.g. bad API design, writing weird metadata to disk)

- Can you spot mis-interactions between the subsystems?

- What were my blind spots in devising this feature?

- Are there missing pieces that you'd like to help build?

- Can I just merge all of this?

The one thing that is /not/ in scope for this review are requests for more refactoring of existing subsystems. I'm still running QA round the clock. To spare vger, I'm only sending a few patchsets this time. I will of course stress test the new mailing infrastructure on 31 Dec with a full posting, like I always do.

--D

#1 lore raw

cover xfs: prevent livelocks in xchk_iget

Hi all,

Prevent scrub from live locking in xchk_iget if there's a cycle in the inobt by allocating an empty transaction.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-livelock-prevention

---
 fs/xfs/scrub/common.c |    6 ++++--
 fs/xfs/scrub/common.h |   19 +++++++++++++++++++
 fs/xfs/scrub/inode.c  |    4 ++--
 3 files changed, 25 insertions(+), 4 deletions(-)
#2 lore raw

PATCH 1/1 xfs: make xchk_iget safer in the presence of corrupt inode btrees

From: Darrick J. Wong <[email protected]>

When scrub is trying to iget an inode, ensure that it won't end up deadlocked on a cycle in the inode btree by using an empty transaction to store all the buffers.

---
 fs/xfs/scrub/common.c |    6 ++++--
 fs/xfs/scrub/common.h |   19 +++++++++++++++++++
 fs/xfs/scrub/inode.c  |    4 ++--
 3 files changed, 25 insertions(+), 4 deletions(-)
fs/xfs/scrub/common.c +4 -2
index de24532fe0830..23944fcc1a6ca 100644
@@ -733,6 +733,8 @@ xchk_iget(
 	xfs_ino_t		inum,
 	struct xfs_inode	**ipp)
 {
+	ASSERT(sc->tp != NULL);
+
 	return xfs_iget(sc->mp, sc->tp, inum, XFS_IGET_UNTRUSTED, 0, ipp);
 }
 
@@ -882,8 +884,8 @@ xchk_iget_for_scrubbing(
 	if (!xfs_verify_ino(sc->mp, sc->sm->sm_ino))
 		return -ENOENT;
 
-	/* Try a regular untrusted iget. */
-	error = xchk_iget(sc, sc->sm->sm_ino, &ip);
+	/* Try a safe untrusted iget. */
+	error = xchk_iget_safe(sc, sc->sm->sm_ino, &ip);
 	if (!error)
 		return xchk_install_handle_inode(sc, ip);
 	if (error == -ENOENT)
fs/xfs/scrub/common.h +19 -0
index cabdc0e16838c..a39dbe6be1e59 100644
@@ -157,6 +157,25 @@ int xchk_iget_agi(struct xfs_scrub *sc, xfs_ino_t inum,
 void xchk_irele(struct xfs_scrub *sc, struct xfs_inode *ip);
 int xchk_install_handle_inode(struct xfs_scrub *sc, struct xfs_inode *ip);
 
+/*
+ * Safe version of (untrusted) xchk_iget that uses an empty transaction to
+ * avoid deadlocking on loops in the inobt.
+ */
+static inline int
+xchk_iget_safe(struct xfs_scrub *sc, xfs_ino_t inum, struct xfs_inode **ipp)
+{
+	int	error;
+
+	ASSERT(sc->tp == NULL);
+
+	error = xchk_trans_alloc(sc, 0);
+	if (error)
+		return error;
+	error = xchk_iget(sc, inum, ipp);
+	xchk_trans_cancel(sc);
+	return error;
+}
+
 /*
  * Don't bother cross-referencing if we already found corruption or cross
  * referencing discrepancies.
fs/xfs/scrub/inode.c +2 -2
index 889f556bc98f6..b7a93380a1ab0 100644
@@ -95,8 +95,8 @@ xchk_setup_inode(
 	if (!xfs_verify_ino(sc->mp, sc->sm->sm_ino))
 		return -ENOENT;
 
-	/* Try a regular untrusted iget. */
-	error = xchk_iget(sc, sc->sm->sm_ino, &ip);
+	/* Try a safe untrusted iget. */
+	error = xchk_iget_safe(sc, sc->sm->sm_ino, &ip);
 	if (!error)
 		return xchk_install_handle_iscrub(sc, ip);
 	if (error == -ENOENT)
Reviewed-by on v1 1/1
#3 lore raw

Re: [PATCH 1/1] xfs: make xchk_iget safer in the presence of corrupt inode btrees

Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When scrub is trying to iget an inode, ensure that it won't end up deadlocked on a cycle in the inode btree by using an empty transaction to store all the buffers.

My only concern here is how I'm suppsed to know when to use the _safe version or not.

Otherwise looks good:

#4 lore raw

Re: [PATCH 1/1] xfs: make xchk_iget safer in the presence of corrupt inode btrees

Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When scrub is trying to iget an inode, ensure that it won't end up deadlocked on a cycle in the inode btree by using an empty transaction to store all the buffers.

My only concern here is how I'm suppsed to know when to use the _safe version or not.

For xchk_iget_safe, I'll amend the comment to read:

/*
* Safe version of (untrusted) xchk_iget that uses an empty transaction to
* avoid deadlocking on loops in the inobt. This should only be used in a
* scrub or repair setup routine, and only prior to grabbing a transaction. */

and add a comment for xchk_iget that reads:

/*
* Grab the inode at @inum. The caller must have created a scrub transaction
* so that we can confirm the inumber by walking the inobt and not deadlock on
* a loop in the inobt.
*/

Quoted 3 lines

Otherwise looks good:

Thanks!

--D

#5 lore raw

cover xfs: reserve disk space for online repairs

In reply to Darrick J. Wong

Hi all,

Online repair fixes metadata structures by writing a new copy out to disk and atomically committing the new structure into the filesystem. For this to work, we need to reserve all the space we're going to need ahead of time so that the atomic commit transaction is as small as possible. We also require the reserved space to be freed if the system goes down, or if we decide not to commit the repair, or if we reserve too much space.

To keep the atomic commit transaction as small as possible, we would like to allocate some space and simultaneously schedule automatic reaping of the reserved space, even on log recovery. EFIs are the mechanism to get us there, but we need to use them in a novel manner. Once we allocate the space, we want to hold on to the EFI (relogging as necessary) until we can commit or cancel the repair. EFIs for written committed blocks need to go away, but unwritten or uncommitted blocks can be freed like normal.

Earlier versions of this patchset directly manipulated the log items, but Dave thought that to be a layering violation. For v27, I've modified the defer ops handling code to be capable of pausing a deferred work item. Log intent items are created as they always have been, but paused items are pushed onto a side list when finishing deferred work items, and pushed back onto the transaction after that. Log intent done item are not created for paused work.

The second part adds a "stale" flag to the EFI so that the repair reservation code can dispose of an EFI the normal way, but without the space actually being freed.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-auto-reap-space-reservations

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-auto-reap-space-reservations

---
 fs/xfs/Makefile                    |    1 
 fs/xfs/libxfs/xfs_ag.c             |    2 
 fs/xfs/libxfs/xfs_alloc.c          |  104 +++++++
 fs/xfs/libxfs/xfs_alloc.h          |   22 +-
 fs/xfs/libxfs/xfs_bmap.c           |    4 
 fs/xfs/libxfs/xfs_bmap_btree.c     |    2 
 fs/xfs/libxfs/xfs_btree_staging.h  |    7 
 fs/xfs/libxfs/xfs_defer.c          |  229 ++++++++++++++--
 fs/xfs/libxfs/xfs_defer.h          |   20 +
 fs/xfs/libxfs/xfs_ialloc.c         |    5 
 fs/xfs/libxfs/xfs_ialloc_btree.c   |    2 
 fs/xfs/libxfs/xfs_refcount.c       |    6 
 fs/xfs/libxfs/xfs_refcount_btree.c |    2 
 fs/xfs/scrub/agheader_repair.c     |    1 
 fs/xfs/scrub/common.c              |    1 
 fs/xfs/scrub/newbt.c               |  510 ++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/newbt.h               |   65 +++++
 fs/xfs/scrub/reap.c                |    7 
 fs/xfs/scrub/scrub.c               |    2 
 fs/xfs/scrub/trace.h               |   37 +++
 fs/xfs/xfs_extfree_item.c          |   12 -
 fs/xfs/xfs_reflink.c               |    2 
 fs/xfs/xfs_trace.h                 |   13 +
 23 files changed, 990 insertions(+), 66 deletions(-)
 create mode 100644 fs/xfs/scrub/newbt.c
 create mode 100644 fs/xfs/scrub/newbt.h
#6 lore raw

PATCH 1/7 xfs: don't append work items to logged xfs_defer_pending objects

From: Darrick J. Wong <[email protected]>

When someone tries to add a deferred work item to xfs_defer_add, it will try to attach the work item to the most recently added xfs_defer_pending object attached to the transaction. However, it doesn't check if the pending object has a log intent item attached to it. This is incorrect behavior because we cannot add more work to an object that has already been committed to the ondisk log.

Therefore, change the behavior not to append to pending items with a non null dfp_intent. In practice this has not been an issue because the only way xfs_defer_add gets called after log intent items have been committed is from the defer ops ->finish_item functions themselves, and the @dop_pending isolation in xfs_defer_finish_noroll protects the pending items that have already been logged.

However, the next patch will add the ability to pause a deferred extent free object during online btree rebuilding, and any new extfree work items need to have their own pending event.

While we're at it, hoist the predicate to its own static inline function for readability.

---
 fs/xfs/libxfs/xfs_defer.c |   48 ++++++++++++++++++++++++++++++++++-----------
 1 file changed, 36 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_defer.c +36 -12
index f71679ce23b95..6c283b30ea054 100644
@@ -624,6 +624,40 @@ xfs_defer_cancel(
 	xfs_defer_cancel_list(mp, &tp->t_dfops);
 }
 
+/*
+ * Decide if we can add a deferred work item to the last dfops item attached
+ * to the transaction.
+ */
+static inline struct xfs_defer_pending *
+xfs_defer_try_append(
+	struct xfs_trans		*tp,
+	enum xfs_defer_ops_type		type,
+	const struct xfs_defer_op_type	*ops)
+{
+	struct xfs_defer_pending	*dfp = NULL;
+
+	/* No dfops at all? */
+	if (list_empty(&tp->t_dfops))
+		return NULL;
+
+	dfp = list_last_entry(&tp->t_dfops, struct xfs_defer_pending,
+			dfp_list);
+
+	/* Wrong type? */
+	if (dfp->dfp_type != type)
+		return NULL;
+
+	/* Already logged? */
+	if (dfp->dfp_intent)
+		return NULL;
+
+	/* Already full? */
+	if (ops->max_items && dfp->dfp_count >= ops->max_items)
+		return NULL;
+
+	return dfp;
+}
+
 /* Add an item for later deferred processing. */
 void
 xfs_defer_add(
@@ -637,19 +671,9 @@ xfs_defer_add(
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 	BUILD_BUG_ON(ARRAY_SIZE(defer_op_types) != XFS_DEFER_OPS_TYPE_MAX);
 
-	/*
-	 * Add the item to a pending item at the end of the intake list.
-	 * If the last pending item has the same type, reuse it.  Else,
-	 * create a new pending item at the end of the intake list.
-	 */
-	if (!list_empty(&tp->t_dfops)) {
-		dfp = list_last_entry(&tp->t_dfops,
-				struct xfs_defer_pending, dfp_list);
-		if (dfp->dfp_type != type ||
-		    (ops->max_items && dfp->dfp_count >= ops->max_items))
-			dfp = NULL;
-	}
+	dfp = xfs_defer_try_append(tp, type, ops);
 	if (!dfp) {
+		/* Create a new pending item at the end of the intake list. */
 		dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
 				GFP_NOFS | __GFP_NOFAIL);
 		dfp->dfp_type = type;
Reviewed-by on v1 1/7
#7 lore raw

Re: [PATCH 1/7] xfs: don't append work items to logged xfs_defer_pending objects

This looks good:

To make the code nicer for the later addition of the barrier defer ops I'd fold the hunk belw to split xfs_defer_try_append, but we could also do that later:

index 6c283b30ea054a..7be2f9063e0ded 100644

fs/xfs/libxfs/xfs_defer.c +18 -14
@@ -624,17 +624,12 @@ xfs_defer_cancel(
 	xfs_defer_cancel_list(mp, &tp->t_dfops);
 }
 
-/*
- * Decide if we can add a deferred work item to the last dfops item attached
- * to the transaction.
- */
 static inline struct xfs_defer_pending *
-xfs_defer_try_append(
+xfs_defer_find(
 	struct xfs_trans		*tp,
-	enum xfs_defer_ops_type		type,
-	const struct xfs_defer_op_type	*ops)
+	enum xfs_defer_ops_type		type)
 {
-	struct xfs_defer_pending	*dfp = NULL;
+	struct xfs_defer_pending	*dfp;
 
 	/* No dfops at all? */
 	if (list_empty(&tp->t_dfops))
@@ -646,16 +641,25 @@ xfs_defer_try_append(
 	/* Wrong type? */
 	if (dfp->dfp_type != type)
 		return NULL;
+	return dfp;
+}
 
+/*
+ * Decide if we can add a deferred work item to the last dfops item attached
+ * to the transaction.
+ */
+static inline bool
+xfs_defer_can_append(
+	struct xfs_defer_pending	*dfp,
+	const struct xfs_defer_op_type	*ops)
+{
 	/* Already logged? */
 	if (dfp->dfp_intent)
-		return NULL;
-
+		return false;
 	/* Already full? */
 	if (ops->max_items && dfp->dfp_count >= ops->max_items)
 		return NULL;
-
-	return dfp;
+	return true;
 }
 
 /* Add an item for later deferred processing. */
@@ -671,8 +675,8 @@ xfs_defer_add(
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 	BUILD_BUG_ON(ARRAY_SIZE(defer_op_types) != XFS_DEFER_OPS_TYPE_MAX);
 
-	dfp = xfs_defer_try_append(tp, type, ops);
-	if (!dfp) {
+	dfp = xfs_defer_find(tp, type);
+	if (!dfp || !xfs_defer_can_append(dfp, ops)) {
 		/* Create a new pending item at the end of the intake list. */
 		dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
 				GFP_NOFS | __GFP_NOFAIL);
#8 lore raw

PATCH 2/7 xfs: allow pausing of pending deferred work items

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Traditionally, all pending deferred work attached to a transaction is finished when one of the xfs_defer_finish* functions is called. However, online repair wants to be able to allocate space for a new data structure, format a new metadata structure into the allocated space, and commit that into the filesystem.

As a hedge against system crashes during repairs, we also want to log some EFI items for the allocated space speculatively, and cancel them if we elect to commit the new data structure.

Therefore, introduce the idea of pausing a pending deferred work item. Log intent items are still created for paused items and relogged as necessary. However, paused items are pushed onto a side list before we start calling ->finish_item, and the whole list is reattach to the transaction afterwards. New work items are never attached to paused pending items.

Modify xfs_defer_cancel to clean up pending deferred work items holding a log intent item but not a log intent done item, since that is now possible.

---
 fs/xfs/libxfs/xfs_defer.c |   98 +++++++++++++++++++++++++++++++++++++++------
 fs/xfs/libxfs/xfs_defer.h |   17 +++++++-
 fs/xfs/xfs_trace.h        |   13 +++++-
 3 files changed, 112 insertions(+), 16 deletions(-)
fs/xfs/libxfs/xfs_defer.c +86 -12
index 6c283b30ea054..6604eb50058ba 100644
@@ -417,7 +417,7 @@ xfs_defer_cancel_list(
  * done item to release the intent item; and then log a new intent item.
  * The caller should provide a fresh transaction and roll it after we're done.
  */
-static int
+static void
 xfs_defer_relog(
 	struct xfs_trans		**tpp,
 	struct list_head		*dfops)
@@ -458,10 +458,6 @@ xfs_defer_relog(
 		XFS_STATS_INC((*tpp)->t_mountp, defer_relog);
 		dfp->dfp_intent = xfs_trans_item_relog(dfp->dfp_intent, *tpp);
 	}
-
-	if ((*tpp)->t_flags & XFS_TRANS_DIRTY)
-		return xfs_defer_trans_roll(tpp);
-	return 0;
 }
 
 /*
@@ -517,6 +513,24 @@ xfs_defer_finish_one(
 	return error;
 }
 
+/* Move all paused deferred work from @tp to @paused_list. */
+static void
+xfs_defer_isolate_paused(
+	struct xfs_trans		*tp,
+	struct list_head		*paused_list)
+{
+	struct xfs_defer_pending	*dfp;
+	struct xfs_defer_pending	*pli;
+
+	list_for_each_entry_safe(dfp, pli, &tp->t_dfops, dfp_list) {
+		if (!(dfp->dfp_flags & XFS_DEFER_PAUSED))
+			continue;
+
+		list_move_tail(&dfp->dfp_list, paused_list);
+		trace_xfs_defer_isolate_paused(tp->t_mountp, dfp);
+	}
+}
+
 /*
  * Finish all the pending work.  This involves logging intent items for
  * any work items that wandered in since the last transaction roll (if
@@ -532,6 +546,7 @@ xfs_defer_finish_noroll(
 	struct xfs_defer_pending	*dfp = NULL;
 	int				error = 0;
 	LIST_HEAD(dop_pending);
+	LIST_HEAD(dop_paused);
 
 	ASSERT((*tp)->t_flags & XFS_TRANS_PERM_LOG_RES);
 
@@ -550,6 +565,8 @@ xfs_defer_finish_noroll(
 		 */
 		int has_intents = xfs_defer_create_intents(*tp);
 
+		xfs_defer_isolate_paused(*tp, &dop_paused);
+
 		list_splice_init(&(*tp)->t_dfops, &dop_pending);
 
 		if (has_intents < 0) {
@@ -562,22 +579,33 @@ xfs_defer_finish_noroll(
 				goto out_shutdown;
 
 			/* Relog intent items to keep the log moving. */
-			error = xfs_defer_relog(tp, &dop_pending);
-			if (error)
-				goto out_shutdown;
+			xfs_defer_relog(tp, &dop_pending);
+			xfs_defer_relog(tp, &dop_paused);
+
+			if ((*tp)->t_flags & XFS_TRANS_DIRTY) {
+				error = xfs_defer_trans_roll(tp);
+				if (error)
+					goto out_shutdown;
+			}
 		}
 
-		dfp = list_first_entry(&dop_pending, struct xfs_defer_pending,
-				       dfp_list);
+		dfp = list_first_entry_or_null(&dop_pending,
+				struct xfs_defer_pending, dfp_list);
+		if (!dfp)
+			break;
 		error = xfs_defer_finish_one(*tp, dfp);
 		if (error && error != -EAGAIN)
 			goto out_shutdown;
 	}
 
+	/* Requeue the paused items in the outgoing transaction. */
+	list_splice_tail_init(&dop_paused, &(*tp)->t_dfops);
+
 	trace_xfs_defer_finish_done(*tp, _RET_IP_);
 	return 0;
 
 out_shutdown:
+	list_splice_tail_init(&dop_paused, &dop_pending);
 	xfs_defer_trans_abort(*tp, &dop_pending);
 	xfs_force_shutdown((*tp)->t_mountp, SHUTDOWN_CORRUPT_INCORE);
 	trace_xfs_defer_finish_error(*tp, error);
@@ -590,6 +618,9 @@ int
 xfs_defer_finish(
 	struct xfs_trans	**tp)
 {
+#ifdef DEBUG
+	struct xfs_defer_pending *dfp;
+#endif
 	int			error;
 
 	/*
@@ -609,7 +640,10 @@ xfs_defer_finish(
 	}
 
 	/* Reset LOWMODE now that we've finished all the dfops. */
-	ASSERT(list_empty(&(*tp)->t_dfops));
+#ifdef DEBUG
+	list_for_each_entry(dfp, &(*tp)->t_dfops, dfp_list)
+		ASSERT(dfp->dfp_flags & XFS_DEFER_PAUSED);
+#endif
 	(*tp)->t_flags &= ~XFS_TRANS_LOWMODE;
 	return 0;
 }
@@ -621,6 +655,7 @@ xfs_defer_cancel(
 	struct xfs_mount	*mp = tp->t_mountp;
 
 	trace_xfs_defer_cancel(tp, _RET_IP_);
+	xfs_defer_trans_abort(tp, &tp->t_dfops);
 	xfs_defer_cancel_list(mp, &tp->t_dfops);
 }
 
@@ -651,6 +686,10 @@ xfs_defer_try_append(
 	if (dfp->dfp_intent)
 		return NULL;
 
+	/* Paused items cannot absorb more work */
+	if (dfp->dfp_flags & XFS_DEFER_PAUSED)
+		return NULL;
+
 	/* Already full? */
 	if (ops->max_items && dfp->dfp_count >= ops->max_items)
 		return NULL;
@@ -659,7 +698,7 @@ xfs_defer_try_append(
 }
 
 /* Add an item for later deferred processing. */
-void
+struct xfs_defer_pending *
 xfs_defer_add(
 	struct xfs_trans		*tp,
 	enum xfs_defer_ops_type		type,
@@ -687,6 +726,8 @@ xfs_defer_add(
 	list_add_tail(li, &dfp->dfp_work);
 	trace_xfs_defer_add_item(tp->t_mountp, dfp, li);
 	dfp->dfp_count++;
+
+	return dfp;
 }
 
 /*
@@ -962,3 +1003,36 @@ xfs_defer_destroy_item_caches(void)
 	xfs_rmap_intent_destroy_cache();
 	xfs_defer_destroy_cache();
 }
+
+/*
+ * Mark a deferred work item so that it will be requeued indefinitely without
+ * being finished.  Caller must ensure there are no data dependencies on this
+ * work item in the meantime.
+ */
+void
+xfs_defer_item_pause(
+	struct xfs_trans		*tp,
+	struct xfs_defer_pending	*dfp)
+{
+	ASSERT(!(dfp->dfp_flags & XFS_DEFER_PAUSED));
+
+	dfp->dfp_flags |= XFS_DEFER_PAUSED;
+
+	trace_xfs_defer_item_pause(tp->t_mountp, dfp);
+}
+
+/*
+ * Release a paused deferred work item so that it will be finished during the
+ * next transaction roll.
+ */
+void
+xfs_defer_item_unpause(
+	struct xfs_trans		*tp,
+	struct xfs_defer_pending	*dfp)
+{
+	ASSERT(dfp->dfp_flags & XFS_DEFER_PAUSED);
+
+	dfp->dfp_flags &= ~XFS_DEFER_PAUSED;
+
+	trace_xfs_defer_item_unpause(tp->t_mountp, dfp);
+}
fs/xfs/libxfs/xfs_defer.h +15 -2
index 8788ad5f6a731..094ff9062b251 100644
@@ -34,11 +34,24 @@ struct xfs_defer_pending {
 	struct xfs_log_item		*dfp_intent;	/* log intent item */
 	struct xfs_log_item		*dfp_done;	/* log done item */
 	unsigned int			dfp_count;	/* # extent items */
+	unsigned int			dfp_flags;
 	enum xfs_defer_ops_type		dfp_type;
 };
 
-void xfs_defer_add(struct xfs_trans *tp, enum xfs_defer_ops_type type,
-		struct list_head *h);
+/*
+ * Create a log intent item for this deferred item, but don't actually finish
+ * the work.  Caller must clear this before the final transaction commit.
+ */
+#define XFS_DEFER_PAUSED	(1U << 0)
+
+#define XFS_DEFER_PENDING_STRINGS \
+	{ XFS_DEFER_PAUSED,	"paused" }
+
+void xfs_defer_item_pause(struct xfs_trans *tp, struct xfs_defer_pending *dfp);
+void xfs_defer_item_unpause(struct xfs_trans *tp, struct xfs_defer_pending *dfp);
+
+struct xfs_defer_pending *xfs_defer_add(struct xfs_trans *tp,
+		enum xfs_defer_ops_type type, struct list_head *h);
 int xfs_defer_finish_noroll(struct xfs_trans **tp);
 int xfs_defer_finish(struct xfs_trans **tp);
 void xfs_defer_cancel(struct xfs_trans *);
fs/xfs/xfs_trace.h +11 -2
index 3926cf7f2a6ed..514095b6ba2bd 100644
@@ -2551,6 +2551,7 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_class,
 		__field(dev_t, dev)
 		__field(int, type)
 		__field(void *, intent)
+		__field(unsigned int, flags)
 		__field(char, committed)
 		__field(int, nr)
 	),
@@ -2558,13 +2559,15 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_class,
 		__entry->dev = mp ? mp->m_super->s_dev : 0;
 		__entry->type = dfp->dfp_type;
 		__entry->intent = dfp->dfp_intent;
+		__entry->flags = dfp->dfp_flags;
 		__entry->committed = dfp->dfp_done != NULL;
 		__entry->nr = dfp->dfp_count;
 	),
-	TP_printk("dev %d:%d optype %d intent %p committed %d nr %d",
+	TP_printk("dev %d:%d optype %d intent %p flags %s committed %d nr %d",
 		  MAJOR(__entry->dev), MINOR(__entry->dev),
 		  __entry->type,
 		  __entry->intent,
+		  __print_flags(__entry->flags, "|", XFS_DEFER_PENDING_STRINGS),
 		  __entry->committed,
 		  __entry->nr)
 )
@@ -2675,6 +2678,9 @@ DEFINE_DEFER_PENDING_EVENT(xfs_defer_cancel_list);
 DEFINE_DEFER_PENDING_EVENT(xfs_defer_pending_finish);
 DEFINE_DEFER_PENDING_EVENT(xfs_defer_pending_abort);
 DEFINE_DEFER_PENDING_EVENT(xfs_defer_relog_intent);
+DEFINE_DEFER_PENDING_EVENT(xfs_defer_isolate_paused);
+DEFINE_DEFER_PENDING_EVENT(xfs_defer_item_pause);
+DEFINE_DEFER_PENDING_EVENT(xfs_defer_item_unpause);
 
 #define DEFINE_BMAP_FREE_DEFERRED_EVENT DEFINE_PHYS_EXTENT_DEFERRED_EVENT
 DEFINE_BMAP_FREE_DEFERRED_EVENT(xfs_bmap_free_defer);
@@ -2692,6 +2698,7 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_item_class,
 		__field(void *, intent)
 		__field(void *, item)
 		__field(char, committed)
+		__field(unsigned int, flags)
 		__field(int, nr)
 	),
 	TP_fast_assign(
@@ -2700,13 +2707,15 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_item_class,
 		__entry->intent = dfp->dfp_intent;
 		__entry->item = item;
 		__entry->committed = dfp->dfp_done != NULL;
+		__entry->flags = dfp->dfp_flags;
 		__entry->nr = dfp->dfp_count;
 	),
-	TP_printk("dev %d:%d optype %d intent %p item %p committed %d nr %d",
+	TP_printk("dev %d:%d optype %d intent %p item %p flags %s committed %d nr %d",
 		  MAJOR(__entry->dev), MINOR(__entry->dev),
 		  __entry->type,
 		  __entry->intent,
 		  __entry->item,
+		  __print_flags(__entry->flags, "|", XFS_DEFER_PENDING_STRINGS),
 		  __entry->committed,
 		  __entry->nr)
 )
#10 lore raw

PATCH 3/7 xfs: remove __xfs_free_extent_later

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

xfs_free_extent_later is a trivial helper, so remove it to reduce the amount of thinking required to understand the deferred freeing interface. This will make it easier to introduce automatic reaping of speculative allocations in the next patch.

---
 fs/xfs/libxfs/xfs_ag.c             |    2 +-
 fs/xfs/libxfs/xfs_alloc.c          |    2 +-
 fs/xfs/libxfs/xfs_alloc.h          |   14 +-------------
 fs/xfs/libxfs/xfs_bmap.c           |    4 ++--
 fs/xfs/libxfs/xfs_bmap_btree.c     |    2 +-
 fs/xfs/libxfs/xfs_ialloc.c         |    5 +++--
 fs/xfs/libxfs/xfs_ialloc_btree.c   |    2 +-
 fs/xfs/libxfs/xfs_refcount.c       |    6 +++---
 fs/xfs/libxfs/xfs_refcount_btree.c |    2 +-
 fs/xfs/scrub/reap.c                |    2 +-
 fs/xfs/xfs_extfree_item.c          |    2 +-
 fs/xfs/xfs_reflink.c               |    2 +-
 12 files changed, 17 insertions(+), 28 deletions(-)
fs/xfs/libxfs/xfs_ag.c +1 -1
index f9f4d694640d0..f62ff125a50ac 100644
@@ -984,7 +984,7 @@ xfs_ag_shrink_space(
 		if (err2 != -ENOSPC)
 			goto resv_err;
 
-		err2 = __xfs_free_extent_later(*tpp, args.fsbno, delta, NULL,
+		err2 = xfs_free_extent_later(*tpp, args.fsbno, delta, NULL,
 				XFS_AG_RESV_NONE, true);
 		if (err2)
 			goto resv_err;
fs/xfs/libxfs/xfs_alloc.c +1 -1
index 100ab5931b313..c35224ad9428a 100644
@@ -2523,7 +2523,7 @@ xfs_defer_agfl_block(
  * The list is maintained sorted (by block number).
  */
 int
-__xfs_free_extent_later(
+xfs_free_extent_later(
 	struct xfs_trans		*tp,
 	xfs_fsblock_t			bno,
 	xfs_filblks_t			len,
fs/xfs/libxfs/xfs_alloc.h +1 -13
index 6bb8d295c321d..6b95d1d8a8537 100644
@@ -231,7 +231,7 @@ xfs_buf_to_agfl_bno(
 	return bp->b_addr;
 }
 
-int __xfs_free_extent_later(struct xfs_trans *tp, xfs_fsblock_t bno,
+int xfs_free_extent_later(struct xfs_trans *tp, xfs_fsblock_t bno,
 		xfs_filblks_t len, const struct xfs_owner_info *oinfo,
 		enum xfs_ag_resv_type type, bool skip_discard);
 
@@ -256,18 +256,6 @@ void xfs_extent_free_get_group(struct xfs_mount *mp,
 #define XFS_EFI_ATTR_FORK	(1U << 1) /* freeing attr fork block */
 #define XFS_EFI_BMBT_BLOCK	(1U << 2) /* freeing bmap btree block */
 
-static inline int
-xfs_free_extent_later(
-	struct xfs_trans		*tp,
-	xfs_fsblock_t			bno,
-	xfs_filblks_t			len,
-	const struct xfs_owner_info	*oinfo,
-	enum xfs_ag_resv_type		type)
-{
-	return __xfs_free_extent_later(tp, bno, len, oinfo, type, false);
-}
-
-
 extern struct kmem_cache	*xfs_extfree_item_cache;
 
 int __init xfs_extfree_intent_init_cache(void);
fs/xfs/libxfs/xfs_bmap.c +2 -2
index be62acffad6cc..68be1dd4f0f26 100644
@@ -575,7 +575,7 @@ xfs_bmap_btree_to_extents(
 
 	xfs_rmap_ino_bmbt_owner(&oinfo, ip->i_ino, whichfork);
 	error = xfs_free_extent_later(cur->bc_tp, cbno, 1, &oinfo,
-			XFS_AG_RESV_NONE);
+			XFS_AG_RESV_NONE, false);
 	if (error)
 		return error;
 
@@ -5218,7 +5218,7 @@ xfs_bmap_del_extent_real(
 		if (xfs_is_reflink_inode(ip) && whichfork == XFS_DATA_FORK) {
 			xfs_refcount_decrease_extent(tp, del);
 		} else {
-			error = __xfs_free_extent_later(tp, del->br_startblock,
+			error = xfs_free_extent_later(tp, del->br_startblock,
 					del->br_blockcount, NULL,
 					XFS_AG_RESV_NONE,
 					((bflags & XFS_BMAPI_NODISCARD) ||
fs/xfs/libxfs/xfs_bmap_btree.c +1 -1
index bf3f1b36fdd23..8360256cff168 100644
@@ -272,7 +272,7 @@ xfs_bmbt_free_block(
 
 	xfs_rmap_ino_bmbt_owner(&oinfo, ip->i_ino, cur->bc_ino.whichfork);
 	error = xfs_free_extent_later(cur->bc_tp, fsbno, 1, &oinfo,
-			XFS_AG_RESV_NONE);
+			XFS_AG_RESV_NONE, false);
 	if (error)
 		return error;
 
fs/xfs/libxfs/xfs_ialloc.c +3 -2
index b83e54c709069..d61d03e5b853b 100644
@@ -1854,7 +1854,7 @@ xfs_difree_inode_chunk(
 		return xfs_free_extent_later(tp,
 				XFS_AGB_TO_FSB(mp, agno, sagbno),
 				M_IGEO(mp)->ialloc_blks, &XFS_RMAP_OINFO_INODES,
-				XFS_AG_RESV_NONE);
+				XFS_AG_RESV_NONE, false);
 	}
 
 	/* holemask is only 16-bits (fits in an unsigned long) */
@@ -1900,7 +1900,8 @@ xfs_difree_inode_chunk(
 		ASSERT(contigblk % mp->m_sb.sb_spino_align == 0);
 		error = xfs_free_extent_later(tp,
 				XFS_AGB_TO_FSB(mp, agno, agbno), contigblk,
-				&XFS_RMAP_OINFO_INODES, XFS_AG_RESV_NONE);
+				&XFS_RMAP_OINFO_INODES, XFS_AG_RESV_NONE,
+				false);
 		if (error)
 			return error;
 
fs/xfs/libxfs/xfs_ialloc_btree.c +1 -1
index 9258f01c0015e..42a5e1f227a05 100644
@@ -161,7 +161,7 @@ __xfs_inobt_free_block(
 	xfs_inobt_mod_blockcount(cur, -1);
 	fsbno = XFS_DADDR_TO_FSB(cur->bc_mp, xfs_buf_daddr(bp));
 	return xfs_free_extent_later(cur->bc_tp, fsbno, 1,
-			&XFS_RMAP_OINFO_INOBT, resv);
+			&XFS_RMAP_OINFO_INOBT, resv, false);
 }
 
 STATIC int
fs/xfs/libxfs/xfs_refcount.c +3 -3
index 646b3fa362ad0..3702b4a071100 100644
@@ -1153,7 +1153,7 @@ xfs_refcount_adjust_extents(
 						tmp.rc_startblock);
 				error = xfs_free_extent_later(cur->bc_tp, fsbno,
 						  tmp.rc_blockcount, NULL,
-						  XFS_AG_RESV_NONE);
+						  XFS_AG_RESV_NONE, false);
 				if (error)
 					goto out_error;
 			}
@@ -1215,7 +1215,7 @@ xfs_refcount_adjust_extents(
 					ext.rc_startblock);
 			error = xfs_free_extent_later(cur->bc_tp, fsbno,
 					ext.rc_blockcount, NULL,
-					XFS_AG_RESV_NONE);
+					XFS_AG_RESV_NONE, false);
 			if (error)
 				goto out_error;
 		}
@@ -1985,7 +1985,7 @@ xfs_refcount_recover_cow_leftovers(
 		/* Free the block. */
 		error = xfs_free_extent_later(tp, fsb,
 				rr->rr_rrec.rc_blockcount, NULL,
-				XFS_AG_RESV_NONE);
+				XFS_AG_RESV_NONE, false);
 		if (error)
 			goto out_trans;
 
fs/xfs/libxfs/xfs_refcount_btree.c +1 -1
index 5c3987d8dc242..3fa795e2488dd 100644
@@ -112,7 +112,7 @@ xfs_refcountbt_free_block(
 	be32_add_cpu(&agf->agf_refcount_blocks, -1);
 	xfs_alloc_log_agf(cur->bc_tp, agbp, XFS_AGF_REFCOUNT_BLOCKS);
 	return xfs_free_extent_later(cur->bc_tp, fsbno, 1,
-			&XFS_RMAP_OINFO_REFC, XFS_AG_RESV_METADATA);
+			&XFS_RMAP_OINFO_REFC, XFS_AG_RESV_METADATA, false);
 }
 
 STATIC int
fs/xfs/scrub/reap.c +1 -1
index 86a62420e02c6..78c9f2085db46 100644
@@ -410,7 +410,7 @@ xreap_agextent_iter(
 	 * Use deferred frees to get rid of the old btree blocks to try to
 	 * minimize the window in which we could crash and lose the old blocks.
 	 */
-	error = __xfs_free_extent_later(sc->tp, fsbno, *aglenp, rs->oinfo,
+	error = xfs_free_extent_later(sc->tp, fsbno, *aglenp, rs->oinfo,
 			rs->resv, true);
 	if (error)
 		return error;
fs/xfs/xfs_extfree_item.c +1 -1
index 3fa8789820ad9..9e7b58f3566c0 100644
@@ -717,7 +717,7 @@ xfs_efi_item_recover(
 			error = xfs_free_extent_later(tp, fake.xefi_startblock,
 					fake.xefi_blockcount,
 					&XFS_RMAP_OINFO_ANY_OWNER,
-					fake.xefi_agresv);
+					fake.xefi_agresv, false);
 			if (!error) {
 				requeue_only = true;
 				continue;
fs/xfs/xfs_reflink.c +1 -1
index e5b62dc284664..d5ca8bcae65b6 100644
@@ -618,7 +618,7 @@ xfs_reflink_cancel_cow_blocks(
 
 			error = xfs_free_extent_later(*tpp, del.br_startblock,
 					del.br_blockcount, NULL,
-					XFS_AG_RESV_NONE);
+					XFS_AG_RESV_NONE, false);
 			if (error)
 				break;
 
#12 lore raw

PATCH 4/7 xfs: automatic freeing of freshly allocated unwritten space

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

As mentioned in the previous commit, online repair wants to allocate space to write out a new metadata structure, and it also wants to hedge against system crashes during repairs by logging (and later cancelling) EFIs to free the space if we crash before committing the new data structure.

Therefore, create a trio of functions to schedule automatic reaping of freshly allocated unwritten space. xfs_alloc_schedule_autoreap creates a paused EFI representing the space we just allocated. Once the allocations are made and the autoreaps scheduled, we can start writing to disk.

If the writes succeed, xfs_alloc_cancel_autoreap marks the EFI work items as stale and unpauses the pending deferred work item. Assuming that's done in the same transaction that commits the new structure into the filesystem, we guarantee that either the new object is fully visible, or that all the space gets reclaimed.

If the writes succeed but only part of an extent was used, repair must call the same _cancel_autoreap function to kill the first EFI and then log a new EFI to free the unused space. The first EFI is already committed, so it cannot be changed.

For full extents that aren't used, xfs_alloc_commit_autoreap will unpause the EFI, which results in the space being freed during the next _defer_finish cycle.

---
 fs/xfs/libxfs/xfs_alloc.c |  104 +++++++++++++++++++++++++++++++++++++++++++--
 fs/xfs/libxfs/xfs_alloc.h |   12 +++++
 fs/xfs/xfs_extfree_item.c |   10 +++-
 3 files changed, 118 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +100 -4
index c35224ad9428a..4940f9377f21a 100644
@@ -2522,14 +2522,15 @@ xfs_defer_agfl_block(
  * Add the extent to the list of extents to be free at transaction end.
  * The list is maintained sorted (by block number).
  */
-int
-xfs_free_extent_later(
+static int
+xfs_defer_extent_free(
 	struct xfs_trans		*tp,
 	xfs_fsblock_t			bno,
 	xfs_filblks_t			len,
 	const struct xfs_owner_info	*oinfo,
 	enum xfs_ag_resv_type		type,
-	bool				skip_discard)
+	bool				skip_discard,
+	struct xfs_defer_pending	**dfpp)
 {
 	struct xfs_extent_free_item	*xefi;
 	struct xfs_mount		*mp = tp->t_mountp;
@@ -2577,10 +2578,105 @@ xfs_free_extent_later(
 			XFS_FSB_TO_AGBNO(tp->t_mountp, bno), len);
 
 	xfs_extent_free_get_group(mp, xefi);
-	xfs_defer_add(tp, XFS_DEFER_OPS_TYPE_FREE, &xefi->xefi_list);
+	*dfpp = xfs_defer_add(tp, XFS_DEFER_OPS_TYPE_FREE, &xefi->xefi_list);
 	return 0;
 }
 
+int
+xfs_free_extent_later(
+	struct xfs_trans		*tp,
+	xfs_fsblock_t			bno,
+	xfs_filblks_t			len,
+	const struct xfs_owner_info	*oinfo,
+	enum xfs_ag_resv_type		type,
+	bool				skip_discard)
+{
+	struct xfs_defer_pending	*dontcare = NULL;
+
+	return xfs_defer_extent_free(tp, bno, len, oinfo, type, skip_discard,
+			&dontcare);
+}
+
+/*
+ * Set up automatic freeing of unwritten space in the filesystem.
+ *
+ * This function attached a paused deferred extent free item to the
+ * transaction.  Pausing means that the EFI will be logged in the next
+ * transaction commit, but the pending EFI will not be finished until the
+ * pending item is unpaused.
+ *
+ * If the system goes down after the EFI has been persisted to the log but
+ * before the pending item is unpaused, log recovery will find the EFI, fail to
+ * find the EFD, and free the space.
+ *
+ * If the pending item is unpaused, the next transaction commit will log an EFD
+ * without freeing the space.
+ *
+ * Caller must ensure that the tp, fsbno, len, oinfo, and resv flags of the
+ * @args structure are set to the relevant values.
+ */
+int
+xfs_alloc_schedule_autoreap(
+	const struct xfs_alloc_arg	*args,
+	bool				skip_discard,
+	struct xfs_alloc_autoreap	*aarp)
+{
+	int				error;
+
+	error = xfs_defer_extent_free(args->tp, args->fsbno, args->len,
+			&args->oinfo, args->resv, skip_discard, &aarp->dfp);
+	if (error)
+		return error;
+
+	xfs_defer_item_pause(args->tp, aarp->dfp);
+	return 0;
+}
+
+/*
+ * Cancel automatic freeing of unwritten space in the filesystem.
+ *
+ * Earlier, we created a paused deferred extent free item and attached it to
+ * this transaction so that we could automatically roll back a new space
+ * allocation if the system went down.  Now we want to cancel the paused work
+ * item by marking the EFI stale so we don't actually free the space, unpausing
+ * the pending item and logging an EFD.
+ *
+ * The caller generally should have already mapped the space into the ondisk
+ * filesystem.  If the reserved space was partially used, the caller must call
+ * xfs_free_extent_later to create a new EFI to free the unused space.
+ */
+void
+xfs_alloc_cancel_autoreap(
+	struct xfs_trans		*tp,
+	struct xfs_alloc_autoreap	*aarp)
+{
+	struct xfs_defer_pending	*dfp = aarp->dfp;
+	struct xfs_extent_free_item	*xefi;
+
+	if (!dfp)
+		return;
+
+	list_for_each_entry(xefi, &dfp->dfp_work, xefi_list)
+		xefi->xefi_flags |= XFS_EFI_CANCELLED;
+
+	xfs_defer_item_unpause(tp, dfp);
+}
+
+/*
+ * Commit automatic freeing of unwritten space in the filesystem.
+ *
+ * This unpauses an earlier _schedule_autoreap and commits to freeing the
+ * allocated space.  Call this if none of the reserved space was used.
+ */
+void
+xfs_alloc_commit_autoreap(
+	struct xfs_trans		*tp,
+	struct xfs_alloc_autoreap	*aarp)
+{
+	if (aarp->dfp)
+		xfs_defer_item_unpause(tp, aarp->dfp);
+}
+
 #ifdef DEBUG
 /*
  * Check if an AGF has a free extent record whose length is equal to
fs/xfs/libxfs/xfs_alloc.h +12 -0
index 6b95d1d8a8537..851cafbd64494 100644
@@ -255,6 +255,18 @@ void xfs_extent_free_get_group(struct xfs_mount *mp,
 #define XFS_EFI_SKIP_DISCARD	(1U << 0) /* don't issue discard */
 #define XFS_EFI_ATTR_FORK	(1U << 1) /* freeing attr fork block */
 #define XFS_EFI_BMBT_BLOCK	(1U << 2) /* freeing bmap btree block */
+#define XFS_EFI_CANCELLED	(1U << 3) /* dont actually free the space */
+
+struct xfs_alloc_autoreap {
+	struct xfs_defer_pending	*dfp;
+};
+
+int xfs_alloc_schedule_autoreap(const struct xfs_alloc_arg *args,
+		bool skip_discard, struct xfs_alloc_autoreap *aarp);
+void xfs_alloc_cancel_autoreap(struct xfs_trans *tp,
+		struct xfs_alloc_autoreap *aarp);
+void xfs_alloc_commit_autoreap(struct xfs_trans *tp,
+		struct xfs_alloc_autoreap *aarp);
 
 extern struct kmem_cache	*xfs_extfree_item_cache;
 
fs/xfs/xfs_extfree_item.c +6 -4
index 9e7b58f3566c0..4af5b3b338b19 100644
@@ -381,7 +381,7 @@ xfs_trans_free_extent(
 	uint				next_extent;
 	xfs_agblock_t			agbno = XFS_FSB_TO_AGBNO(mp,
 							xefi->xefi_startblock);
-	int				error;
+	int				error = 0;
 
 	oinfo.oi_owner = xefi->xefi_owner;
 	if (xefi->xefi_flags & XFS_EFI_ATTR_FORK)
@@ -392,9 +392,11 @@ xfs_trans_free_extent(
 	trace_xfs_bmap_free_deferred(tp->t_mountp, xefi->xefi_pag->pag_agno, 0,
 			agbno, xefi->xefi_blockcount);
 
-	error = __xfs_free_extent(tp, xefi->xefi_pag, agbno,
-			xefi->xefi_blockcount, &oinfo, xefi->xefi_agresv,
-			xefi->xefi_flags & XFS_EFI_SKIP_DISCARD);
+	if (!(xefi->xefi_flags & XFS_EFI_CANCELLED))
+		error = __xfs_free_extent(tp, xefi->xefi_pag, agbno,
+				xefi->xefi_blockcount, &oinfo,
+				xefi->xefi_agresv,
+				xefi->xefi_flags & XFS_EFI_SKIP_DISCARD);
 
 	/*
 	 * Mark the transaction dirty, even on error. This ensures the
#14 lore raw

PATCH 5/7 xfs: implement block reservation accounting for btrees we're staging

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a new xrep_newbt structure to encapsulate a fake root for creating a staged btree cursor as well as to track all the blocks that we need to reserve in order to build that btree.

---
 fs/xfs/Makefile                   |    1 
 fs/xfs/libxfs/xfs_btree_staging.h |    7 -
 fs/xfs/scrub/agheader_repair.c    |    1 
 fs/xfs/scrub/common.c             |    1 
 fs/xfs/scrub/newbt.c              |  492 +++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/newbt.h              |   62 +++++
 fs/xfs/scrub/scrub.c              |    2 
 fs/xfs/scrub/trace.h              |   37 +++
 8 files changed, 598 insertions(+), 5 deletions(-)
 create mode 100644 fs/xfs/scrub/newbt.c
 create mode 100644 fs/xfs/scrub/newbt.h
fs/xfs/Makefile +1 -0
index 7762c01a85cfb..1537d66e5ab01 100644
@@ -181,6 +181,7 @@ xfs-$(CONFIG_XFS_QUOTA)		+= scrub/quota.o
 ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
+				   newbt.o \
 				   reap.o \
 				   repair.o \
 				   )
fs/xfs/libxfs/xfs_btree_staging.h +2 -5
index f0d2976050aea..d6dea3f0088c6 100644
@@ -38,11 +38,8 @@ struct xbtree_ifakeroot {
 	/* Number of bytes available for this fork in the inode. */
 	unsigned int		if_fork_size;
 
-	/* Fork format. */
-	unsigned int		if_format;
-
-	/* Number of records. */
-	unsigned int		if_extents;
+	/* Which fork is this btree being built for? */
+	int			if_whichfork;
 };
 
 /* Cursor interactions with fake roots for inode-rooted btrees. */
fs/xfs/scrub/agheader_repair.c +1 -0
index 876a2f41b0637..36c511f96b004 100644
@@ -10,6 +10,7 @@
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
 #include "xfs_btree.h"
+#include "xfs_btree_staging.h"
 #include "xfs_log_format.h"
 #include "xfs_trans.h"
 #include "xfs_sb.h"
fs/xfs/scrub/common.c +1 -0
index 23944fcc1a6ca..4bba3c49f8c59 100644
@@ -10,6 +10,7 @@
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
 #include "xfs_btree.h"
+#include "xfs_btree_staging.h"
 #include "xfs_log_format.h"
 #include "xfs_trans.h"
 #include "xfs_inode.h"
fs/xfs/scrub/newbt.c new file +492 -0

492 lines not on this page · raw message

fs/xfs/scrub/newbt.h new file +62 -0

62 lines not on this page · raw message

fs/xfs/scrub/scrub.c +2 -0
index 4849efcaa33ae..474f4c4a9cd3b 100644
@@ -17,6 +17,8 @@
 #include "xfs_errortag.h"
 #include "xfs_error.h"
 #include "xfs_scrub.h"
+#include "xfs_btree.h"
+#include "xfs_btree_staging.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 #include "scrub/trace.h"
fs/xfs/scrub/trace.h +37 -0
index 4a8bc6f3c8f2e..aa76830753196 100644
@@ -1332,6 +1332,43 @@ TRACE_EVENT(xrep_ialloc_insert,
 		  __entry->freemask)
 )
 
+DECLARE_EVENT_CLASS(xrep_newbt_extent_class,
+	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
+		 xfs_agblock_t agbno, xfs_extlen_t len,
+		 int64_t owner),
+	TP_ARGS(mp, agno, agbno, len, owner),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_agnumber_t, agno)
+		__field(xfs_agblock_t, agbno)
+		__field(xfs_extlen_t, len)
+		__field(int64_t, owner)
+	),
+	TP_fast_assign(
+		__entry->dev = mp->m_super->s_dev;
+		__entry->agno = agno;
+		__entry->agbno = agbno;
+		__entry->len = len;
+		__entry->owner = owner;
+	),
+	TP_printk("dev %d:%d agno 0x%x agbno 0x%x fsbcount 0x%x owner 0x%llx",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->agno,
+		  __entry->agbno,
+		  __entry->len,
+		  __entry->owner)
+);
+#define DEFINE_NEWBT_EXTENT_EVENT(name) \
+DEFINE_EVENT(xrep_newbt_extent_class, name, \
+	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, \
+		 xfs_agblock_t agbno, xfs_extlen_t len, \
+		 int64_t owner), \
+	TP_ARGS(mp, agno, agbno, len, owner))
+DEFINE_NEWBT_EXTENT_EVENT(xrep_newbt_alloc_ag_blocks);
+DEFINE_NEWBT_EXTENT_EVENT(xrep_newbt_alloc_file_blocks);
+DEFINE_NEWBT_EXTENT_EVENT(xrep_newbt_free_blocks);
+DEFINE_NEWBT_EXTENT_EVENT(xrep_newbt_claim_block);
+
 #endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
 
 #endif /* _TRACE_XFS_SCRUB_TRACE_H */
#15 lore raw

Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging

The data structure and support code looks fine to me, but I do have some nitpicky comments and questions:

fs/xfs/libxfs/xfs_btree_staging.h struct xbtree_ifakeroot { · line 42

Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.h +2 -5
-	/* Fork format. */
-	unsigned int		if_format;
-
-	/* Number of records. */
-	unsigned int		if_extents;
+	/* Which fork is this btree being built for? */
+	int			if_whichfork;

The two removed fields seems to be unused even before this patch. Should they have been in a separate removal patch?

fs/xfs/scrub/agheader_repair.c line 16

Quoted 12 lines
fs/xfs/scrub/agheader_repair.c +1 -0
index 876a2f41b0637..36c511f96b004 100644
@@ -10,6 +10,7 @@
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
 #include "xfs_btree.h"
+#include "xfs_btree_staging.h"
 #include "xfs_log_format.h"
 #include "xfs_trans.h"
 #include "xfs_sb.h"

I also don't think all the #include churn belongs into this patch, as the only existing header touched by it is xfs_btree_staging.h, which means that anything that didn't need it before still won't need it with the changes.

Quoted 10 lines

+/*
+ * Estimate proper slack values for a btree that's being reloaded.
+ *
+ * Under most circumstances, we'll take whatever default loading value the
+ * btree bulk loading code calculates for us. However, there are some
+ * exceptions to this rule:
+ *
+ * (1) If someone turned one of the debug knobs.
+ * (2) If this is a per-AG btree and the AG has less than ~9% space free.
+ * (3) If this is an inode btree and the FS has less than ~9% space free.

Where does this ~9% number come from? Obviously it is a low-space condition of some sort, but I wonder what are the criteria. It would be nice to document that here, even if the answer is answer is "out of thin air".

fs/xfs/scrub/agheader_repair.c

Quoted 7 lines

+ * Note that we actually use 3/32 for the comparison to avoid division.
+ */

fs/xfs/scrub/agheader_repair.c +4 -0
+static void
 
+	/* No further changes if there's more than 3/32ths space left. */
+	if (free >= ((sz * 3) >> 5))
+		return;

Is this code really in the critical path that a division (or relying on the compiler to do the right thing) is out of question? Because these shits by magic numbers are really annyoing to read (unlike say normal SECTOR_SHIFT or PAGE_SHIFT ones that are fairly easy to read).

#16 lore raw

Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging

Quoted 13 lines · Christoph Hellwig

The data structure and support code looks fine to me, but I do have some nitpicky comments and questions:

fs/xfs/libxfs/xfs_btree_staging.h struct xbtree_ifakeroot { · line 42

Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.h +2 -5
-	/* Fork format. */
-	unsigned int		if_format;
-
-	/* Number of records. */
-	unsigned int		if_extents;
+	/* Which fork is this btree being built for? */
+	int			if_whichfork;

The two removed fields seems to be unused even before this patch. Should they have been in a separate removal patch?

They should have been in the patch that moved if_{format,extents} into xfs_inode_fork:

daf83964a3681 ("xfs: move the per-fork nextents fields into struct xfs_ifork") f7e67b20ecbbc ("xfs: move the fork format fields into struct xfs_ifork")

but I think it just got lost in the review of all that back in May 2020. Since then the design has changed enough that I don't even think the if_whichfork field is in use anywhere:

$ git grep if_whichfork
fs/xfs/libxfs/xfs_bmap_btree.c:664:     cur = xfs_bmbt_init_common(mp, NULL, ip, ifake->if_whichfork);
fs/xfs/libxfs/xfs_btree_staging.h:42:   int                     if_whichfork;
fs/xfs/scrub/newbt.c:117:       xnr->ifake.if_whichfork = whichfork;
fs/xfs/scrub/newbt.c:156:       xnr->ifake.if_whichfork = XFS_DATA_FORK;
$ cd ../xfsprogs/
$ git grep if_whichfork
db/bmap_inflate.c:367:  ifake.if_whichfork = XFS_DATA_FORK;
db/bmap_inflate.c:421:  ifake.if_whichfork = XFS_DATA_FORK;
libxfs/xfs_bmap_btree.c:662:    cur = xfs_bmbt_init_common(mp, NULL, ip, ifake->if_whichfork);
libxfs/xfs_btree_staging.h:42:  int                     if_whichfork;
repair/bulkload.c:38:   bkl->ifake.if_whichfork = whichfork;

So that can all go away.

Quoted 9 lines
Quoted 9 lines
fs/xfs/scrub/agheader_repair.c +1 -0
index 876a2f41b0637..36c511f96b004 100644
@@ -10,6 +10,7 @@
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
 #include "xfs_btree.h"
+#include "xfs_btree_staging.h"
Quoted 8 lines

fs/xfs/scrub/agheader_repair.c line 16

Quoted 3 lines
fs/xfs/scrub/agheader_repair.c +0 -0
 #include "xfs_log_format.h"
 #include "xfs_trans.h"
 #include "xfs_sb.h"

I also don't think all the #include churn belongs into this patch, as the only existing header touched by it is xfs_btree_staging.h, which means that anything that didn't need it before still won't need it with the changes.

Hmm yeah. Not sure when this detritus started accumulating here. :(

Quoted 11 lines
Quoted 10 lines

+/*
+ * Estimate proper slack values for a btree that's being reloaded.
+ *
+ * Under most circumstances, we'll take whatever default loading value the
+ * btree bulk loading code calculates for us. However, there are some
+ * exceptions to this rule:
+ *
+ * (1) If someone turned one of the debug knobs.
+ * (2) If this is a per-AG btree and the AG has less than ~9% space free.
+ * (3) If this is an inode btree and the FS has less than ~9% space free.

Quoted 4 lines

Where does this ~9% number come from? Obviously it is a low-space condition of some sort, but I wonder what are the criteria. It would be nice to document that here, even if the answer is answer is "out of thin air".

It comes from xfs_repair:
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/repair/bulkload.c?h=for-next#n114

Before xfs_btree_staging.[ch] came along, it was open coded in repair/phase5.c in a most unglorious fashion:
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/repair/phase5.c?h=v4.19.0#n1349

At that point the slack factors were arbitrary quantities per btree. The rmapbt automatically left 10 slots free; everything else left zero.

That had a noticeable effect on performance straight after mounting because touching /any/ btree would result in splits. IIRC Dave and I decided that repair should generate btree blocks that were 75% full unless space was tight. We defined tight as ~10% free to avoid repair failures and settled on 3/32 to avoid div64.

IOWs, we mostly pulled it out of thin air. ;)

OFC the other weird thing is that originally I thought that online repair would land sooner than would the retroport of xfs_repair to the new btree bulk loading code.

Quoted 4 lines
Quoted 3 lines

+ * Note that we actually use 3/32 for the comparison to avoid division.
+ */
+static void

Quoted 9 lines

fs/xfs/scrub/agheader_repair.c

Quoted 3 lines
fs/xfs/scrub/agheader_repair.c +3 -0
+	/* No further changes if there's more than 3/32ths space left. */
+	if (free >= ((sz * 3) >> 5))
+		return;

Is this code really in the critical path that a division (or relying on the compiler to do the right thing) is out of question? Because these shits by magic numbers are really annyoing to read (unlike say normal SECTOR_SHIFT or PAGE_SHIFT ones that are fairly easy to read).

Nah, it's not performance critical. Collecting records and formatting blocks is a much bigger strain on the system than a few divisions. I'll change it to div_u64(sz, 10).

Hmm now that I look at it, I've also noticed that even the lowspace btree rebuild in userspace will leave two open slots per block.

--D

#17 lore raw

Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging

Quoted 7 lines · Darrick J. Wong

That had a noticeable effect on performance straight after mounting because touching /any/ btree would result in splits. IIRC Dave and I decided that repair should generate btree blocks that were 75% full unless space was tight. We defined tight as ~10% free to avoid repair failures and settled on 3/32 to avoid div64.

IOWs, we mostly pulled it out of thin air. ;)

Maybe throw a little comment about this in.

#18 lore raw

Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging

Quoted 10 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong

That had a noticeable effect on performance straight after mounting because touching /any/ btree would result in splits. IIRC Dave and I decided that repair should generate btree blocks that were 75% full unless space was tight. We defined tight as ~10% free to avoid repair failures and settled on 3/32 to avoid div64.

IOWs, we mostly pulled it out of thin air. ;)

Maybe throw a little comment about this in.

Yeah, I'll paste the sordid history into the commit message when I change the code to use div_u64.

--D

#19 lore raw

PATCH 6/7 xfs: log EFIs for all btree blocks being used to stage a btree

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

We need to log EFIs for every extent that we allocate for the purpose of staging a new btree so that if we fail then the blocks will be freed during log recovery. Use the autoreaping mechanism provided by the previous patch to attach paused freeing work to the scrub transaction. We can then mark the EFIs stale if we decide to commit the new btree, or we can unpause the EFIs if we decide to abort the repair.

---
 fs/xfs/scrub/newbt.c |   34 ++++++++++++++++++++++++++--------
 fs/xfs/scrub/newbt.h |    3 +++
 2 files changed, 29 insertions(+), 8 deletions(-)
fs/xfs/scrub/newbt.c +26 -8
index 4e8d6637426e4..2932fd317ab23 100644
@@ -136,6 +136,7 @@ xrep_newbt_add_blocks(
 {
 	struct xfs_mount		*mp = xnr->sc->mp;
 	struct xrep_newbt_resv		*resv;
+	int				error;
 
 	resv = kmalloc(sizeof(struct xrep_newbt_resv), XCHK_GFP_FLAGS);
 	if (!resv)
@@ -147,8 +148,18 @@ xrep_newbt_add_blocks(
 	resv->used = 0;
 	resv->pag = xfs_perag_hold(pag);
 
+	ASSERT(xnr->oinfo.oi_offset == 0);
+
+	error = xfs_alloc_schedule_autoreap(args, true, &resv->autoreap);
+	if (error)
+		goto out_pag;
+
 	list_add_tail(&resv->list, &xnr->resv_list);
 	return 0;
+out_pag:
+	xfs_perag_put(resv->pag);
+	kfree(resv);
+	return error;
 }
 
 /* Don't let our allocation hint take us beyond this AG */
@@ -327,16 +338,21 @@ xrep_newbt_free_extent(
 	if (!btree_committed || resv->used == 0) {
 		/*
 		 * If we're not committing a new btree or we didn't use the
-		 * space reservation, free the entire space extent.
+		 * space reservation, let the existing EFI free the entire
+		 * space extent.
 		 */
-		goto free;
+		trace_xrep_newbt_free_blocks(sc->mp, resv->pag->pag_agno,
+				free_agbno, free_aglen, xnr->oinfo.oi_owner);
+		xfs_alloc_commit_autoreap(sc->tp, &resv->autoreap);
+		return 1;
 	}
 
 	/*
-	 * We used space and committed the btree.  Remove the written blocks
-	 * from the reservation and possibly log a new EFI to free any unused
-	 * reservation space.
+	 * We used space and committed the btree.  Cancel the autoreap, remove
+	 * the written blocks from the reservation, and possibly log a new EFI
+	 * to free any unused reservation space.
 	 */
+	xfs_alloc_cancel_autoreap(sc->tp, &resv->autoreap);
 	free_agbno += resv->used;
 	free_aglen -= resv->used;
 
@@ -348,7 +364,6 @@ xrep_newbt_free_extent(
 
 	ASSERT(xnr->resv != XFS_AG_RESV_AGFL);
 
-free:
 	/*
 	 * Use EFIs to free the reservations.  This reduces the chance
 	 * that we leak blocks if the system goes down.
@@ -408,9 +423,10 @@ xrep_newbt_free(
 	/*
 	 * If we still have reservations attached to @newbt, cleanup must have
 	 * failed and the filesystem is about to go down.  Clean up the incore
-	 * reservations.
+	 * reservations and try to commit to freeing the space we used.
 	 */
 	list_for_each_entry_safe(resv, n, &xnr->resv_list, list) {
+		xfs_alloc_commit_autoreap(sc->tp, &resv->autoreap);
 		list_del(&resv->list);
 		xfs_perag_put(resv->pag);
 		kfree(resv);
@@ -488,5 +504,7 @@ xrep_newbt_claim_block(
 								agbno));
 	else
 		ptr->s = cpu_to_be32(agbno);
-	return 0;
+
+	/* Relog all the EFIs. */
+	return xrep_defer_finish(xnr->sc);
 }
fs/xfs/scrub/newbt.h +3 -0
index ca53271f3a4c6..d2baffa17b1ae 100644
@@ -12,6 +12,9 @@ struct xrep_newbt_resv {
 
 	struct xfs_perag	*pag;
 
+	/* Auto-freeing this reservation if we don't commit. */
+	struct xfs_alloc_autoreap autoreap;
+
 	/* AG block of the extent we reserved. */
 	xfs_agblock_t		agbno;
 
#21 lore raw

PATCH 7/7 xfs: force small EFIs for reaping btree extents

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Introduce the concept of a defer ops barrier to separate consecutively queued pending work items of the same type. With a barrier in place, the two work items will be tracked separately, and receive separate log intent items. The goal here is to prevent reaping of old metadata blocks from creating unnecessarily huge EFIs that could then run the risk of overflowing the scrub transaction.

---
 fs/xfs/libxfs/xfs_defer.c |   83 +++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_defer.h |    3 ++
 fs/xfs/scrub/reap.c       |    5 +++
 3 files changed, 91 insertions(+)
fs/xfs/libxfs/xfs_defer.c +83 -0
index 6604eb50058ba..6b0d4c2e844b0 100644
@@ -181,6 +181,58 @@ static struct kmem_cache	*xfs_defer_pending_cache;
  * Note that the continuation requested between t2 and t3 is likely to
  * reoccur.
  */
+STATIC struct xfs_log_item *
+xfs_defer_barrier_create_intent(
+	struct xfs_trans		*tp,
+	struct list_head		*items,
+	unsigned int			count,
+	bool				sort)
+{
+	return NULL;
+}
+
+STATIC void
+xfs_defer_barrier_abort_intent(
+	struct xfs_log_item		*intent)
+{
+	/* empty */
+}
+
+STATIC struct xfs_log_item *
+xfs_defer_barrier_create_done(
+	struct xfs_trans		*tp,
+	struct xfs_log_item		*intent,
+	unsigned int			count)
+{
+	return NULL;
+}
+
+STATIC int
+xfs_defer_barrier_finish_item(
+	struct xfs_trans		*tp,
+	struct xfs_log_item		*done,
+	struct list_head		*item,
+	struct xfs_btree_cur		**state)
+{
+	ASSERT(0);
+	return -EFSCORRUPTED;
+}
+
+STATIC void
+xfs_defer_barrier_cancel_item(
+	struct list_head		*item)
+{
+	ASSERT(0);
+}
+
+static const struct xfs_defer_op_type xfs_barrier_defer_type = {
+	.max_items	= 1,
+	.create_intent	= xfs_defer_barrier_create_intent,
+	.abort_intent	= xfs_defer_barrier_abort_intent,
+	.create_done	= xfs_defer_barrier_create_done,
+	.finish_item	= xfs_defer_barrier_finish_item,
+	.cancel_item	= xfs_defer_barrier_cancel_item,
+};
 
 static const struct xfs_defer_op_type *defer_op_types[] = {
 	[XFS_DEFER_OPS_TYPE_BMAP]	= &xfs_bmap_update_defer_type,
@@ -189,6 +241,7 @@ static const struct xfs_defer_op_type *defer_op_types[] = {
 	[XFS_DEFER_OPS_TYPE_FREE]	= &xfs_extent_free_defer_type,
 	[XFS_DEFER_OPS_TYPE_AGFL_FREE]	= &xfs_agfl_free_defer_type,
 	[XFS_DEFER_OPS_TYPE_ATTR]	= &xfs_attr_defer_type,
+	[XFS_DEFER_OPS_TYPE_BARRIER]	= &xfs_barrier_defer_type,
 };
 
 /*
@@ -1036,3 +1089,33 @@ xfs_defer_item_unpause(
 
 	trace_xfs_defer_item_unpause(tp->t_mountp, dfp);
 }
+
+/*
+ * Add a defer ops barrier to force two otherwise adjacent deferred work items
+ * to be tracked separately and have separate log items.
+ */
+void
+xfs_defer_add_barrier(
+	struct xfs_trans		*tp)
+{
+	struct xfs_defer_pending	*dfp;
+
+	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
+
+	/* If the last defer op added was a barrier, we're done. */
+	if (!list_empty(&tp->t_dfops)) {
+		dfp = list_last_entry(&tp->t_dfops,
+				struct xfs_defer_pending, dfp_list);
+		if (dfp->dfp_type == XFS_DEFER_OPS_TYPE_BARRIER)
+			return;
+	}
+
+	dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
+			GFP_NOFS | __GFP_NOFAIL);
+	dfp->dfp_type = XFS_DEFER_OPS_TYPE_BARRIER;
+	INIT_LIST_HEAD(&dfp->dfp_work);
+	list_add_tail(&dfp->dfp_list, &tp->t_dfops);
+
+	trace_xfs_defer_add_item(tp->t_mountp, dfp, NULL);
+	dfp->dfp_count++;
+}
fs/xfs/libxfs/xfs_defer.h +3 -0
index 094ff9062b251..0112678a8856b 100644
@@ -20,6 +20,7 @@ enum xfs_defer_ops_type {
 	XFS_DEFER_OPS_TYPE_FREE,
 	XFS_DEFER_OPS_TYPE_AGFL_FREE,
 	XFS_DEFER_OPS_TYPE_ATTR,
+	XFS_DEFER_OPS_TYPE_BARRIER,
 	XFS_DEFER_OPS_TYPE_MAX,
 };
 
@@ -141,4 +142,6 @@ void xfs_defer_resources_rele(struct xfs_defer_resources *dres);
 int __init xfs_defer_init_item_caches(void);
 void xfs_defer_destroy_item_caches(void);
 
+void xfs_defer_add_barrier(struct xfs_trans *tp);
+
 #endif /* __XFS_DEFER_H__ */
fs/xfs/scrub/reap.c +5 -0
index 78c9f2085db46..ee26fcb500b78 100644
@@ -31,6 +31,7 @@
 #include "xfs_da_btree.h"
 #include "xfs_attr.h"
 #include "xfs_attr_remote.h"
+#include "xfs_defer.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 #include "scrub/trace.h"
@@ -409,6 +410,8 @@ xreap_agextent_iter(
 	/*
 	 * Use deferred frees to get rid of the old btree blocks to try to
 	 * minimize the window in which we could crash and lose the old blocks.
+	 * Add a defer ops barrier every other extent to avoid stressing the
+	 * system with large EFIs.
 	 */
 	error = xfs_free_extent_later(sc->tp, fsbno, *aglenp, rs->oinfo,
 			rs->resv, true);
@@ -416,6 +419,8 @@ xreap_agextent_iter(
 		return error;
 
 	rs->deferred++;
+	if (rs->deferred % 2 == 0)
+		xfs_defer_add_barrier(sc->tp);
 	return 0;
 }
 
Reviewed-by on v1 7/7
#22 lore raw

Re: [PATCH 7/7] xfs: force small EFIs for reaping btree extents

This generally looks good:

But as mentioned earlier I think it would be nice to share some code between the normal xfs_defer_add and xfs_defer_add_barrier.

This would be the fold for this patch on top of the one previously posted and a light merge fix for the pausing between:

fs/xfs/libxfs/xfs_defer.c +23 -27
index e70dc347074dc5..5d621e445e8ab9 100644
@@ -753,6 +753,22 @@ xfs_defer_can_append(
 	return true;
 }
 
+/* Create a new pending item at the end of the intake list. */
+static struct xfs_defer_pending *
+xfs_defer_alloc(
+	struct xfs_trans		*tp,
+	enum xfs_defer_ops_type		type)
+{
+	struct xfs_defer_pending	*dfp;
+
+	dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
+			GFP_NOFS | __GFP_NOFAIL);
+	dfp->dfp_type = type;
+	INIT_LIST_HEAD(&dfp->dfp_work);
+	list_add_tail(&dfp->dfp_list, &tp->t_dfops);
+	return dfp;
+};
+
 /* Add an item for later deferred processing. */
 struct xfs_defer_pending *
 xfs_defer_add(
@@ -760,29 +776,18 @@ xfs_defer_add(
 	enum xfs_defer_ops_type		type,
 	struct list_head		*li)
 {
-	struct xfs_defer_pending	*dfp = NULL;
 	const struct xfs_defer_op_type	*ops = defer_op_types[type];
+	struct xfs_defer_pending	*dfp;
 
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 	BUILD_BUG_ON(ARRAY_SIZE(defer_op_types) != XFS_DEFER_OPS_TYPE_MAX);
 
 	dfp = xfs_defer_find(tp, type);
-	if (!dfp || !xfs_defer_can_append(dfp, ops)) {
-		/* Create a new pending item at the end of the intake list. */
-		dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
-				GFP_NOFS | __GFP_NOFAIL);
-		dfp->dfp_type = type;
-		dfp->dfp_intent = NULL;
-		dfp->dfp_done = NULL;
-		dfp->dfp_count = 0;
-		INIT_LIST_HEAD(&dfp->dfp_work);
-		list_add_tail(&dfp->dfp_list, &tp->t_dfops);
-	}
-
+	if (!dfp || !xfs_defer_can_append(dfp, ops))
+		dfp = xfs_defer_alloc(tp, type);
 	list_add_tail(li, &dfp->dfp_work);
 	trace_xfs_defer_add_item(tp->t_mountp, dfp, li);
 	dfp->dfp_count++;
-
 	return dfp;
 }
 
@@ -1106,19 +1111,10 @@ xfs_defer_add_barrier(
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 
 	/* If the last defer op added was a barrier, we're done. */
-	if (!list_empty(&tp->t_dfops)) {
-		dfp = list_last_entry(&tp->t_dfops,
-				struct xfs_defer_pending, dfp_list);
-		if (dfp->dfp_type == XFS_DEFER_OPS_TYPE_BARRIER)
-			return;
-	}
-
-	dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
-			GFP_NOFS | __GFP_NOFAIL);
-	dfp->dfp_type = XFS_DEFER_OPS_TYPE_BARRIER;
-	INIT_LIST_HEAD(&dfp->dfp_work);
-	list_add_tail(&dfp->dfp_list, &tp->t_dfops);
-
+	dfp = xfs_defer_find(tp, XFS_DEFER_OPS_TYPE_BARRIER);
+	if (dfp)
+		return;
+	dfp = xfs_defer_alloc(tp, XFS_DEFER_OPS_TYPE_BARRIER);
 	trace_xfs_defer_add_item(tp->t_mountp, dfp, NULL);
 	dfp->dfp_count++;
 }
author Applied
#23 lore raw

Re: [PATCH 7/7] xfs: force small EFIs for reaping btree extents

Quoted 9 lines · Christoph Hellwig

This generally looks good:

But as mentioned earlier I think it would be nice to share some code between the normal xfs_defer_add and xfs_defer_add_barrier.

This would be the fold for this patch on top of the one previously posted and a light merge fix for the pausing between:

Applied. Thanks for the quick cleanup!

--D

Quoted 85 lines
fs/xfs/libxfs/xfs_defer.c +23 -27
index e70dc347074dc5..5d621e445e8ab9 100644
@@ -753,6 +753,22 @@ xfs_defer_can_append(
 	return true;
 }
 
+/* Create a new pending item at the end of the intake list. */
+static struct xfs_defer_pending *
+xfs_defer_alloc(
+	struct xfs_trans		*tp,
+	enum xfs_defer_ops_type		type)
+{
+	struct xfs_defer_pending	*dfp;
+
+	dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
+			GFP_NOFS | __GFP_NOFAIL);
+	dfp->dfp_type = type;
+	INIT_LIST_HEAD(&dfp->dfp_work);
+	list_add_tail(&dfp->dfp_list, &tp->t_dfops);
+	return dfp;
+};
+
 /* Add an item for later deferred processing. */
 struct xfs_defer_pending *
 xfs_defer_add(
@@ -760,29 +776,18 @@ xfs_defer_add(
 	enum xfs_defer_ops_type		type,
 	struct list_head		*li)
 {
-	struct xfs_defer_pending	*dfp = NULL;
 	const struct xfs_defer_op_type	*ops = defer_op_types[type];
+	struct xfs_defer_pending	*dfp;
 
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 	BUILD_BUG_ON(ARRAY_SIZE(defer_op_types) != XFS_DEFER_OPS_TYPE_MAX);
 
 	dfp = xfs_defer_find(tp, type);
-	if (!dfp || !xfs_defer_can_append(dfp, ops)) {
-		/* Create a new pending item at the end of the intake list. */
-		dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
-				GFP_NOFS | __GFP_NOFAIL);
-		dfp->dfp_type = type;
-		dfp->dfp_intent = NULL;
-		dfp->dfp_done = NULL;
-		dfp->dfp_count = 0;
-		INIT_LIST_HEAD(&dfp->dfp_work);
-		list_add_tail(&dfp->dfp_list, &tp->t_dfops);
-	}
-
+	if (!dfp || !xfs_defer_can_append(dfp, ops))
+		dfp = xfs_defer_alloc(tp, type);
 	list_add_tail(li, &dfp->dfp_work);
 	trace_xfs_defer_add_item(tp->t_mountp, dfp, li);
 	dfp->dfp_count++;
-
 	return dfp;
 }
 
@@ -1106,19 +1111,10 @@ xfs_defer_add_barrier(
 	ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES);
 
 	/* If the last defer op added was a barrier, we're done. */
-	if (!list_empty(&tp->t_dfops)) {
-		dfp = list_last_entry(&tp->t_dfops,
-				struct xfs_defer_pending, dfp_list);
-		if (dfp->dfp_type == XFS_DEFER_OPS_TYPE_BARRIER)
-			return;
-	}
-
-	dfp = kmem_cache_zalloc(xfs_defer_pending_cache,
-			GFP_NOFS | __GFP_NOFAIL);
-	dfp->dfp_type = XFS_DEFER_OPS_TYPE_BARRIER;
-	INIT_LIST_HEAD(&dfp->dfp_work);
-	list_add_tail(&dfp->dfp_list, &tp->t_dfops);
-
+	dfp = xfs_defer_find(tp, XFS_DEFER_OPS_TYPE_BARRIER);
+	if (dfp)
+		return;
+	dfp = xfs_defer_alloc(tp, XFS_DEFER_OPS_TYPE_BARRIER);
 	trace_xfs_defer_add_item(tp->t_mountp, dfp, NULL);
 	dfp->dfp_count++;
 }
#24 lore raw

cover xfs: prepare repair for bulk loading

In reply to Darrick J. Wong

Hi all,

Before we start merging the online repair functions, let's improve the bulk loading code a bit. First, we need to fix a misinteraction between the AIL and the btree bulkloader wherein the delwri at the end of the bulk load fails to queue a buffer for writeback if it happens to be on the AIL list.

Second, we introduce a defer ops barrier object so that the process of reaping blocks after a repair cannot queue more than two extents per EFI log item. This increases our exposure to leaking blocks if the system goes down during a reap, but also should prevent transaction overflows, which result in the system going down.

Third, we change the bulkloader itself to copy multiple records into a block if possible, and add some debugging knobs so that developers can control the slack factors, just like they can do for xfs_repair.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-prep-for-bulk-loading

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-prep-for-bulk-loading

---
 fs/xfs/libxfs/xfs_btree.c         |    2 +
 fs/xfs/libxfs/xfs_btree.h         |    3 ++
 fs/xfs/libxfs/xfs_btree_staging.c |   67 +++++++++++++++++++++++++------------
 fs/xfs/libxfs/xfs_btree_staging.h |   25 +++++++++++---
 fs/xfs/scrub/newbt.c              |   11 ++++--
 fs/xfs/xfs_buf.c                  |   47 ++++++++++++++++++++++++--
 fs/xfs/xfs_buf.h                  |    1 +
 fs/xfs/xfs_globals.c              |   12 +++++++
 fs/xfs/xfs_sysctl.h               |    2 +
 fs/xfs/xfs_sysfs.c                |   54 ++++++++++++++++++++++++++++++
 10 files changed, 189 insertions(+), 35 deletions(-)
#25 lore raw

PATCH 1/4 xfs: force all buffers to be written during btree bulk load

From: Darrick J. Wong <[email protected]>

While stress-testing online repair of btrees, I noticed periodic assertion failures from the buffer cache about buffer readers encountering buffers with DELWRI_Q set, even though the btree bulk load had already committed and the buffer itself wasn't on any delwri list.

I traced this to a misunderstanding of how the delwri lists work, particularly with regards to the AIL's buffer list. If a buffer is logged and committed, the buffer can end up on that AIL buffer list. If btree repairs are run twice in rapid succession, it's possible that the first repair will invalidate the buffer and free it before the next time the AIL wakes up. This clears DELWRI_Q from the buffer state.

If the second repair allocates the same block, it will then recycle the buffer to start writing the new btree block. Meanwhile, if the AIL wakes up and walks the buffer list, it will ignore the buffer because it can't lock it, and go back to sleep.

When the second repair calls delwri_queue to put the buffer on the list of buffers to write before committing the new btree, it will set DELWRI_Q again, but since the buffer hasn't been removed from the AIL's buffer list, it won't add it to the bulkload buffer's list.

This is incorrect, because the bulkload caller relies on delwri_submit to ensure that all the buffers have been sent to disk /before/ committing the new btree root pointer. This ordering requirement is required for data consistency.

Worse, the AIL won't clear DELWRI_Q from the buffer when it does finally drop it, so the next thread to walk through the btree will trip over a debug assertion on that flag.

To fix this, create a new function that waits for the buffer to be removed from any other delwri lists before adding the buffer to the caller's delwri list. By waiting for the buffer to clear both the delwri list and any potential delwri wait list, we can be sure that repair will initiate writes of all buffers and report all write errors back to userspace instead of committing the new structure.

---
 fs/xfs/libxfs/xfs_btree_staging.c |    4 +--
 fs/xfs/xfs_buf.c                  |   47 ++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_buf.h                  |    1 +
 3 files changed, 45 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_btree_staging.c +1 -3
index dd75e208b543e..29e3f8ccb1852 100644
@@ -342,9 +342,7 @@ xfs_btree_bload_drop_buf(
 	if (*bpp == NULL)
 		return;
 
-	if (!xfs_buf_delwri_queue(*bpp, buffers_list))
-		ASSERT(0);
-
+	xfs_buf_delwri_queue_here(*bpp, buffers_list);
 	xfs_buf_relse(*bpp);
 	*bpp = NULL;
 }
fs/xfs/xfs_buf.c +43 -4

83 lines not on this page · raw message

fs/xfs/xfs_buf.h +1 -0
index c86e164196568..b470de08a46ca 100644
@@ -319,6 +319,7 @@ extern void xfs_buf_stale(struct xfs_buf *bp);
 /* Delayed Write Buffer Routines */
 extern void xfs_buf_delwri_cancel(struct list_head *);
 extern bool xfs_buf_delwri_queue(struct xfs_buf *, struct list_head *);
+void xfs_buf_delwri_queue_here(struct xfs_buf *bp, struct list_head *bl);
 extern int xfs_buf_delwri_submit(struct list_head *);
 extern int xfs_buf_delwri_submit_nowait(struct list_head *);
 extern int xfs_buf_delwri_pushbuf(struct xfs_buf *, struct list_head *);
#26 lore raw

Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load

The code makes me feel we're fixing the wrong thing, but I have to admin I don't fully understand it. Let me go step by step.

Quoted 4 lines

While stress-testing online repair of btrees, I noticed periodic assertion failures from the buffer cache about buffer readers encountering buffers with DELWRI_Q set, even though the btree bulk load had already committed and the buffer itself wasn't on any delwri list.

What assert do these buffer reader hit? The two I can spot that are in the read path in the broader sense are:

1) in xfs_buf_find_lock for stale buffers.
2) in __xfs_buf_submit just before I/O submission

Quoted 6 lines

I traced this to a misunderstanding of how the delwri lists work, particularly with regards to the AIL's buffer list. If a buffer is logged and committed, the buffer can end up on that AIL buffer list. If btree repairs are run twice in rapid succession, it's possible that the first repair will invalidate the buffer and free it before the next time the AIL wakes up. This clears DELWRI_Q from the buffer state.

Where "this clears" is xfs_buf_stale called from xfs_btree_free_block via xfs_trans_binval?

Quoted 2 lines

If the second repair allocates the same block, it will then recycle the buffer to start writing the new btree block.

If my above theory is correct: how do we end up reusing a stale buffer? If not, what am I misunderstanding above?

Quoted 3 lines

Meanwhile, if the AIL
wakes up and walks the buffer list, it will ignore the buffer because it can't lock it, and go back to sleep.

And I think this is where the trouble starts - we have a buffer that is left on some delwri list, but with the _XBF_DELWRI_Q flag cleared, it is stale and we then reuse it. I don't think we just need to kick it off the delwri list just for btree staging, but in general.

Quoted 6 lines

When the second repair calls delwri_queue to put the buffer on the list of buffers to write before committing the new btree, it will set DELWRI_Q again, but since the buffer hasn't been removed from the AIL's buffer list, it won't add it to the bulkload buffer's list.

Quoted 8 lines

This is incorrect, because the bulkload caller relies on delwri_submit to ensure that all the buffers have been sent to disk /before/ committing the new btree root pointer. This ordering requirement is required for data consistency.

Worse, the AIL won't clear DELWRI_Q from the buffer when it does finally drop it, so the next thread to walk through the btree will trip over a debug assertion on that flag.

Where do it finally drop it?

Quoted 6 lines

To fix this, create a new function that waits for the buffer to be removed from any other delwri lists before adding the buffer to the caller's delwri list. By waiting for the buffer to clear both the delwri list and any potential delwri wait list, we can be sure that repair will initiate writes of all buffers and report all write errors back to userspace instead of committing the new structure.

If my understanding above is correct this just papers over the bug that a buffer that is marked stale and can be reused for something else is left on a delwri list. I've entirely thought about all the consequence, but here is what I'd try:

- if xfs_buf_find_lock finds a stale buffer with _XBF_DELWRI_Q call your new wait code instead of asserting (probably only for the !trylock case)
- make sure we don't leak DELWRI_Q

#27 lore raw

Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load

Quoted 8 lines · Christoph Hellwig

The code makes me feel we're fixing the wrong thing, but I have to admin I don't fully understand it. Let me go step by step.

Quoted 4 lines

While stress-testing online repair of btrees, I noticed periodic assertion failures from the buffer cache about buffer readers encountering buffers with DELWRI_Q set, even though the btree bulk load had already committed and the buffer itself wasn't on any delwri list.

Quoted 5 lines

What assert do these buffer reader hit? The two I can spot that are in the read path in the broader sense are:

1) in xfs_buf_find_lock for stale buffers.
2) in __xfs_buf_submit just before I/O submission

The second assert:

AIL: Repair0: Repair1:

pin buffer X
delwri_queue:
set DELWRI_Q
add to delwri list

stale buf X:
clear DELWRI_Q
does not clear b_list
free space X
commit

delwri_submit # oops

Though there's more to this.

Quoted 9 lines
Quoted 6 lines

I traced this to a misunderstanding of how the delwri lists work, particularly with regards to the AIL's buffer list. If a buffer is logged and committed, the buffer can end up on that AIL buffer list. If btree repairs are run twice in rapid succession, it's possible that the first repair will invalidate the buffer and free it before the next time the AIL wakes up. This clears DELWRI_Q from the buffer state.

Where "this clears" is xfs_buf_stale called from xfs_btree_free_block via xfs_trans_binval?

Yes. I'll rework the last sentence to read: "Marking the buffer stale clears DELWRI_Q from the buffer state without removing the buffer from its delwri list."

Quoted 5 lines
Quoted 2 lines

If the second repair allocates the same block, it will then recycle the buffer to start writing the new btree block.

If my above theory is correct: how do we end up reusing a stale buffer? If not, what am I misunderstanding above?

If we free a metadata buffer, we'll mark it stale, update the bnobt, and add an entry to the extent busy list. If a later repair finds:

1. The same free space in the bnobt;
2. The free space exactly coincides with one extent busy list entry;
3. The entry isn't in the middle of discarding the block;

Then the allocator will remove the extent busy list entry and let us have the space. At that point we could have a stale buffer that's also on one of the AIL's lists:

AIL: Repair0: Repair1:

pin buffer X
delwri_queue:
set DELWRI_Q
add to delwri list

stale buf X:
clear DELWRI_Q
does not clear b_list
free space X
commit

find free space X
get buffer
rewrite buffer delwri_queue:
set DELWRI_Q
already on a list, do not add commit

BAD: committed tree root before all blocks written

delwri_submit # too late now

I could demonstrate this by injecting dmerror while delwri(ting) the new btree blocks, and observing that some of the EIOs would not trigger the rollback but instead would trip IO errors in the AIL.

Quoted 7 lines
Quoted 2 lines

wakes up and walks the buffer list, it will ignore the buffer because it can't lock it, and go back to sleep.

And I think this is where the trouble starts - we have a buffer that is left on some delwri list, but with the _XBF_DELWRI_Q flag cleared, it is stale and we then reuse it. I don't think we just need to kick it off the delwri list just for btree staging, but in general.

...but how to do that? We don't know whose delwri list the buffer's on, let alone how to lock the list so that we can remove the buffer from that list.

(Oh, you have some suggestions below.)

Quoted 10 lines
Quoted 10 lines

When the second repair calls delwri_queue to put the buffer on the list of buffers to write before committing the new btree, it will set DELWRI_Q again, but since the buffer hasn't been removed from the AIL's buffer list, it won't add it to the bulkload buffer's list.

This is incorrect, because the bulkload caller relies on delwri_submit to ensure that all the buffers have been sent to disk /before/ committing the new btree root pointer. This ordering requirement is required for data consistency.

Quoted 5 lines
Quoted 3 lines

Worse, the AIL won't clear DELWRI_Q from the buffer when it does finally drop it, so the next thread to walk through the btree will trip over a debug assertion on that flag.

Where do it finally drop it?

TBH, it's been so long that I can't remember anymore. :(

Quoted 10 lines
Quoted 6 lines

To fix this, create a new function that waits for the buffer to be removed from any other delwri lists before adding the buffer to the caller's delwri list. By waiting for the buffer to clear both the delwri list and any potential delwri wait list, we can be sure that repair will initiate writes of all buffers and report all write errors back to userspace instead of committing the new structure.

If my understanding above is correct this just papers over the bug that a buffer that is marked stale and can be reused for something else is left on a delwri list.

I'm not sure it's a bug for *most* code that encounters it. Regular transactions don't directly use the delwri lists, so it will never see that at all. If the buffer gets rewritten and logged, then it'll just end up on the AIL's delwri buffer list again.

The other delwri_submit users don't seem to care if the buffer gets written directly by delwri_submit or by the AIL. In the first case, the caller will get the EIO and force a shutdown; in the second case, the AIL will shut down the log.

Weirdly, xfs_bwrite clears DELWRI_Q but doesn't necessary shut down the fs if the write fails.

Every time I revisit this patch I wonder if DELWRI_Q is misnamed -- does it mean "b_list is active"? Or does it merely mean "another thread will write this buffer via b_list if nobody gets there first"? I think it's the second, since it's easy enough to list_empty().

It's only repair that requires this new behavior that all new buffers have to be written through the delwri list, and only because it actually /can/ cancel the transaction by finishing the autoreap EFIs.

Quoted 7 lines

I've entirely thought about all the
consequence, but here is what I'd try:

- if xfs_buf_find_lock finds a stale buffer with _XBF_DELWRI_Q call your new wait code instead of asserting (probably only for the !trylock case)
- make sure we don't leak DELWRI_Q

Way back when I first discovered this, my first impulse was to make xfs_buf_stale wait for DELWRI_Q to clear. That IIRC didn't work because a transaction could hold an inode that the AIL will need to lock. I think xfs_buf_find_lock would have the same problem.

Seeing as repair is the only user with the requirement that it alone can issue writes for the delwri list buffers, that's why I went with what's in this patch.

Thank you for persevering so far! :D

--D

#28 lore raw

Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load

Quoted 8 lines
Quoted 3 lines

If my understanding above is correct this just papers over the bug that a buffer that is marked stale and can be reused for something else is left on a delwri list.

I'm not sure it's a bug for *most* code that encounters it. Regular transactions don't directly use the delwri lists, so it will never see that at all. If the buffer gets rewritten and logged, then it'll just end up on the AIL's delwri buffer list again.

Where "regular transactions" means data I/O, yes. All normal metadata I/O eventually ends up on a delwri list. And we want it on that delwri list after actually updating the contents.

Quoted 4 lines

Every time I revisit this patch I wonder if DELWRI_Q is misnamed -- does it mean "b_list is active"? Or does it merely mean "another thread will write this buffer via b_list if nobody gets there first"? I think it's the second, since it's easy enough to list_empty().

I think it is misnamed and not clearly defined, and we should probably fix that. Note that least the current list_empty() usage isn't quite correct either without a lock held by the delwri list owner. We at least need a list_empty_careful for a racey but still save check.

Thinking out loud I wonder if we can just kill XBF_DELWRI_Q entirely. Except for various asserts we mostly use it in reverse polarity, that is to cancel delwri writeout for stale buffers. How about just skipping XBF_STALE buffers on the delwri list directly and not bother with DELWRI_Q? With that we can then support multiple delwri lists that don't get into each others way using say an allocating xarray instead of the invase list and avoid this whole mess.

Let me try to prototype this..

Quoted 3 lines

It's only repair that requires this new behavior that all new buffers have to be written through the delwri list, and only because it actually /can/ cancel the transaction by finishing the autoreap EFIs.

Right now yes. But I think the delwri behavior is a land mine, and this just happens to be the first user to trigger it. Edit: while looking through the DELWRI_Q usage I noticed xfs_qm_flush_one, which seems to deal with what is at least a somewhat related issue based on the comments there.

Quoted 4 lines

Way back when I first discovered this, my first impulse was to make xfs_buf_stale wait for DELWRI_Q to clear. That IIRC didn't work because a transaction could hold an inode that the AIL will need to lock. I think xfs_buf_find_lock would have the same problem.

Yes, that makes sense. Would be great to document such details in the commit message..

#29 lore raw

Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load

Quoted 18 lines · Christoph Hellwig
Quoted 4 lines

Every time I revisit this patch I wonder if DELWRI_Q is misnamed -- does it mean "b_list is active"? Or does it merely mean "another thread will write this buffer via b_list if nobody gets there first"? I think it's the second, since it's easy enough to list_empty().

I think it is misnamed and not clearly defined, and we should probably fix that. Note that least the current list_empty() usage isn't quite correct either without a lock held by the delwri list owner. We at least need a list_empty_careful for a racey but still save check.

Thinking out loud I wonder if we can just kill XBF_DELWRI_Q entirely. Except for various asserts we mostly use it in reverse polarity, that is to cancel delwri writeout for stale buffers. How about just skipping XBF_STALE buffers on the delwri list directly and not bother with DELWRI_Q? With that we can then support multiple delwri lists that don't get into each others way using say an allocating xarray instead of the invase list and avoid this whole mess.

Quoted 1 line

Let me try to prototype this..

Ok, I spent half the day prototyping this and it passes basic sanity checks:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/xfs-multiple-delwri-lists

My conclusions from that is:

1) it works
2) I think it is the right to do
3) it needs a lot more work
4) we can't block the online repair work on it

so I guess we'll need to go with the approach in this patch for now, maybe with a better commit log, and I'll look into finishing this work some time in the future.

#30 lore raw

Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load

Quoted 28 lines · Christoph Hellwig
Quoted 18 lines · Christoph Hellwig
Quoted 4 lines

Every time I revisit this patch I wonder if DELWRI_Q is misnamed -- does it mean "b_list is active"? Or does it merely mean "another thread will write this buffer via b_list if nobody gets there first"? I think it's the second, since it's easy enough to list_empty().

I think it is misnamed and not clearly defined, and we should probably fix that. Note that least the current list_empty() usage isn't quite correct either without a lock held by the delwri list owner. We at least need a list_empty_careful for a racey but still save check.

Thinking out loud I wonder if we can just kill XBF_DELWRI_Q entirely. Except for various asserts we mostly use it in reverse polarity, that is to cancel delwri writeout for stale buffers. How about just skipping XBF_STALE buffers on the delwri list directly and not bother with DELWRI_Q? With that we can then support multiple delwri lists that don't get into each others way using say an allocating xarray instead of the invase list and avoid this whole mess.

Quoted 1 line

Let me try to prototype this..

Ok, I spent half the day prototyping this and it passes basic sanity checks:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/xfs-multiple-delwri-lists

My conclusions from that is:

Quoted 8 lines

1) it works
2) I think it is the right to do
3) it needs a lot more work
4) we can't block the online repair work on it

so I guess we'll need to go with the approach in this patch for now, maybe with a better commit log, and I'll look into finishing this work some time in the future.

<nod> I think an xarray version of this would be less clunky than xfs_delwri_buf with three pointers.

Also note Dave's comments on the v25 posting of this series:
https://lore.kernel.org/linux-xfs/ZJJa7Cnni0mb%[email protected]/

--D

#31 lore raw

Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load

Quoted 6 lines · Darrick J. Wong
Quoted 3 lines

so I guess we'll need to go with the approach in this patch for now, maybe with a better commit log, and I'll look into finishing this work some time in the future.

<nod> I think an xarray version of this would be less clunky than xfs_delwri_buf with three pointers.

It would, but xarray indices are unsigned longs, so we couldn't simply use the block number for it. So unless we decided to give up on building XFS on 32-bit kernels (which would nice for other reasons) it would need something more complicated.

#32 lore raw

PATCH 2/4 xfs: add debug knobs to control btree bulk load slack factors

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some debug knobs so that we can control the leaf and node block slack when rebuilding btrees.

---
 fs/xfs/scrub/newbt.c |   10 ++++++---
 fs/xfs/xfs_globals.c |   12 +++++++++++
 fs/xfs/xfs_sysctl.h  |    2 ++
 fs/xfs/xfs_sysfs.c   |   54 ++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 75 insertions(+), 3 deletions(-)
fs/xfs/scrub/newbt.c +7 -3
index 2932fd317ab23..2c388c647d37f 100644
@@ -47,9 +47,13 @@ xrep_newbt_estimate_slack(
 	uint64_t		free;
 	uint64_t		sz;
 
-	/* Let the btree code compute the default slack values. */
-	bload->leaf_slack = -1;
-	bload->node_slack = -1;
+	/*
+	 * The xfs_globals values are set to -1 (i.e. take the bload defaults)
+	 * unless someone has set them otherwise, so we just pull the values
+	 * here.
+	 */
+	bload->leaf_slack = xfs_globals.bload_leaf_slack;
+	bload->node_slack = xfs_globals.bload_node_slack;
 
 	if (sc->ops->type == ST_PERAG) {
 		free = sc->sa.pag->pagf_freeblks;
fs/xfs/xfs_globals.c +12 -0
index 9edc1f2bc9399..f18fec0adf666 100644
@@ -44,4 +44,16 @@ struct xfs_globals xfs_globals = {
 	.pwork_threads		=	-1,	/* automatic thread detection */
 	.larp			=	false,	/* log attribute replay */
 #endif
+
+	/*
+	 * Leave this many record slots empty when bulk loading btrees.  By
+	 * default we load new btree leaf blocks 75% full.
+	 */
+	.bload_leaf_slack	=	-1,
+
+	/*
+	 * Leave this many key/ptr slots empty when bulk loading btrees.  By
+	 * default we load new btree node blocks 75% full.
+	 */
+	.bload_node_slack	=	-1,
 };
fs/xfs/xfs_sysctl.h +2 -0
index f78ad6b10ea58..276696a07040c 100644
@@ -85,6 +85,8 @@ struct xfs_globals {
 	int	pwork_threads;		/* parallel workqueue threads */
 	bool	larp;			/* log attribute replay */
 #endif
+	int	bload_leaf_slack;	/* btree bulk load leaf slack */
+	int	bload_node_slack;	/* btree bulk load node slack */
 	int	log_recovery_delay;	/* log recovery delay (secs) */
 	int	mount_delay;		/* mount setup delay (secs) */
 	bool	bug_on_assert;		/* BUG() the kernel on assert failure */
fs/xfs/xfs_sysfs.c +54 -0

66 lines not on this page · raw message

#33 lore raw

Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors

Quoted 4 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some debug knobs so that we can control the leaf and node block slack when rebuilding btrees.

Maybe expand a bit on what these "debug knows" are useful for and how they work?

#34 lore raw

Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors

Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add some debug knobs so that we can control the leaf and node block slack when rebuilding btrees.

Maybe expand a bit on what these "debug knows" are useful for and how they work?

They're not generally useful for users, obviously. For developers, it might be useful to construct btrees of various heights by crafting a filesystem with a certain number of records and then using repair+knobs to rebuild the index with a certain shape.

Practically speaking, you'd only ever do that for extreme stress testing.

--D

#35 lore raw

Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors

Quoted 7 lines · Darrick J. Wong

They're not generally useful for users, obviously. For developers, it might be useful to construct btrees of various heights by crafting a filesystem with a certain number of records and then using repair+knobs to rebuild the index with a certain shape.

Practically speaking, you'd only ever do that for extreme stress testing.

Please add this to the commit message.

#36 lore raw

Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors

Quoted 10 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong

They're not generally useful for users, obviously. For developers, it might be useful to construct btrees of various heights by crafting a filesystem with a certain number of records and then using repair+knobs to rebuild the index with a certain shape.

Practically speaking, you'd only ever do that for extreme stress testing.

Please add this to the commit message.

Done.

--D

Quoted 2 lines
#37 lore raw

PATCH 3/4 xfs: move btree bulkload record initialization to ->get_record implementations

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When we're performing a bulk load of a btree, move the code that actually stores the btree record in the new btree block out of the generic code and into the individual ->get_record implementations. This is preparation for being able to store multiple records with a single indirect call.

---
 fs/xfs/libxfs/xfs_btree_staging.c |   17 +++++++----------
 fs/xfs/libxfs/xfs_btree_staging.h |   15 ++++++++++-----
 2 files changed, 17 insertions(+), 15 deletions(-)
fs/xfs/libxfs/xfs_btree_staging.c +7 -10
index 29e3f8ccb1852..369965cacc8c5 100644
@@ -434,22 +434,19 @@ STATIC int
 xfs_btree_bload_leaf(
 	struct xfs_btree_cur		*cur,
 	unsigned int			recs_this_block,
-	xfs_btree_bload_get_record_fn	get_record,
+	xfs_btree_bload_get_records_fn	get_records,
 	struct xfs_btree_block		*block,
 	void				*priv)
 {
-	unsigned int			j;
+	unsigned int			j = 1;
 	int				ret;
 
 	/* Fill the leaf block with records. */
-	for (j = 1; j <= recs_this_block; j++) {
-		union xfs_btree_rec	*block_rec;
-
-		ret = get_record(cur, priv);
-		if (ret)
+	while (j <= recs_this_block) {
+		ret = get_records(cur, j, block, recs_this_block - j + 1, priv);
+		if (ret < 0)
 			return ret;
-		block_rec = xfs_btree_rec_addr(cur, j, block);
-		cur->bc_ops->init_rec_from_cur(cur, block_rec);
+		j += ret;
 	}
 
 	return 0;
@@ -787,7 +784,7 @@ xfs_btree_bload(
 		trace_xfs_btree_bload_block(cur, level, i, blocks, &ptr,
 				nr_this_block);
 
-		ret = xfs_btree_bload_leaf(cur, nr_this_block, bbl->get_record,
+		ret = xfs_btree_bload_leaf(cur, nr_this_block, bbl->get_records,
 				block, priv);
 		if (ret)
 			goto out;
fs/xfs/libxfs/xfs_btree_staging.h +10 -5
index d6dea3f0088c6..82a3a8ef0f125 100644
@@ -50,7 +50,9 @@ void xfs_btree_commit_ifakeroot(struct xfs_btree_cur *cur, struct xfs_trans *tp,
 		int whichfork, const struct xfs_btree_ops *ops);
 
 /* Bulk loading of staged btrees. */
-typedef int (*xfs_btree_bload_get_record_fn)(struct xfs_btree_cur *cur, void *priv);
+typedef int (*xfs_btree_bload_get_records_fn)(struct xfs_btree_cur *cur,
+		unsigned int idx, struct xfs_btree_block *block,
+		unsigned int nr_wanted, void *priv);
 typedef int (*xfs_btree_bload_claim_block_fn)(struct xfs_btree_cur *cur,
 		union xfs_btree_ptr *ptr, void *priv);
 typedef size_t (*xfs_btree_bload_iroot_size_fn)(struct xfs_btree_cur *cur,
@@ -58,11 +60,14 @@ typedef size_t (*xfs_btree_bload_iroot_size_fn)(struct xfs_btree_cur *cur,
 
 struct xfs_btree_bload {
 	/*
-	 * This function will be called nr_records times to load records into
-	 * the btree.  The function does this by setting the cursor's bc_rec
-	 * field in in-core format.  Records must be returned in sort order.
+	 * This function will be called to load @nr_wanted records into the
+	 * btree.  The implementation does this by setting the cursor's bc_rec
+	 * field in in-core format and using init_rec_from_cur to set the
+	 * records in the btree block.  Records must be returned in sort order.
+	 * The function must return the number of records loaded or the usual
+	 * negative errno.
 	 */
-	xfs_btree_bload_get_record_fn	get_record;
+	xfs_btree_bload_get_records_fn	get_records;
 
 	/*
 	 * This function will be called nr_blocks times to obtain a pointer
#39 lore raw

PATCH 4/4 xfs: constrain dirty buffers while formatting a staged btree

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Constrain the number of dirty buffers that are locked by the btree staging code at any given time by establishing a threshold at which we put them all on the delwri queue and push them to disk. This limits memory consumption while writing out new btrees.

---
 fs/xfs/libxfs/xfs_btree.c         |    2 +
 fs/xfs/libxfs/xfs_btree.h         |    3 ++
 fs/xfs/libxfs/xfs_btree_staging.c |   50 +++++++++++++++++++++++++++++--------
 fs/xfs/libxfs/xfs_btree_staging.h |   10 +++++++
 fs/xfs/scrub/newbt.c              |    1 +
 5 files changed, 54 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_btree.c +1 -1
index 6a6503ab0cd76..c100e92140be1 100644
@@ -1330,7 +1330,7 @@ xfs_btree_get_buf_block(
  * Read in the buffer at the given ptr and return the buffer and
  * the block pointer within the buffer.
  */
-STATIC int
+int
 xfs_btree_read_buf_block(
 	struct xfs_btree_cur		*cur,
 	const union xfs_btree_ptr	*ptr,
fs/xfs/libxfs/xfs_btree.h +3 -0
index 4d68a58be160c..e0875cec49392 100644
@@ -700,6 +700,9 @@ void xfs_btree_set_ptr_null(struct xfs_btree_cur *cur,
 int xfs_btree_get_buf_block(struct xfs_btree_cur *cur,
 		const union xfs_btree_ptr *ptr, struct xfs_btree_block **block,
 		struct xfs_buf **bpp);
+int xfs_btree_read_buf_block(struct xfs_btree_cur *cur,
+		const union xfs_btree_ptr *ptr, int flags,
+		struct xfs_btree_block **block, struct xfs_buf **bpp);
 void xfs_btree_set_sibling(struct xfs_btree_cur *cur,
 		struct xfs_btree_block *block, const union xfs_btree_ptr *ptr,
 		int lr);
fs/xfs/libxfs/xfs_btree_staging.c +39 -11

91 lines not on this page · raw message

fs/xfs/libxfs/xfs_btree_staging.h +10 -0
index 82a3a8ef0f125..d2eaf4fdc6032 100644
@@ -115,6 +115,16 @@ struct xfs_btree_bload {
 	 * height of the new btree.
 	 */
 	unsigned int			btree_height;
+
+	/*
+	 * Flush the new btree block buffer list to disk after this many blocks
+	 * have been formatted.  Zero prohibits writing any buffers until all
+	 * blocks have been formatted.
+	 */
+	uint16_t			max_dirty;
+
+	/* Number of dirty buffers. */
+	uint16_t			nr_dirty;
 };
 
 int xfs_btree_bload_compute_geometry(struct xfs_btree_cur *cur,
fs/xfs/scrub/newbt.c +1 -0
index 2c388c647d37f..73e21a9e5e929 100644
@@ -89,6 +89,7 @@ xrep_newbt_init_ag(
 	xnr->alloc_hint = alloc_hint;
 	xnr->resv = resv;
 	INIT_LIST_HEAD(&xnr->resv_list);
+	xnr->bload.max_dirty = XFS_B_TO_FSBT(sc->mp, 256U << 10); /* 256K */
 	xrep_newbt_estimate_slack(xnr);
 }
 
#40 lore raw

Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree

fs/xfs/libxfs/xfs_btree_staging.c xfs_btree_bload_node( · line 504

Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.c +1 -1
@@ -480,7 +500,7 @@ xfs_btree_bload_node(
 
 		ASSERT(!xfs_btree_ptr_is_null(cur, child_ptr));
 
-		ret = xfs_btree_get_buf_block(cur, child_ptr, &child_block,
+		ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
 				&child_bp);

How is this (and making xfs_btree_read_buf_block outside of xfs_buf.c) related to the dirty limit?

#41 lore raw

Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree

Quoted 10 lines · Christoph Hellwig

fs/xfs/libxfs/xfs_btree_staging.c xfs_btree_bload_node( · line 504

Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.c +1 -1
@@ -480,7 +500,7 @@ xfs_btree_bload_node(
 
 		ASSERT(!xfs_btree_ptr_is_null(cur, child_ptr));
 
-		ret = xfs_btree_get_buf_block(cur, child_ptr, &child_block,
+		ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
 				&child_bp);

How is this (and making xfs_btree_read_buf_block outside of xfs_buf.c) related to the dirty limit?

Oh! Looking through my notes, I wanted the /new/ btree block buffers have the same lru reference count the old ones did. I probably should have exported xfs_btree_set_refs instead of reading whatever is on disk into a buffer, only to blow away the contents anyway.

--D

#42 lore raw

Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree

Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig

fs/xfs/libxfs/xfs_btree_staging.c xfs_btree_bload_node( · line 504

Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.c +1 -1
@@ -480,7 +500,7 @@ xfs_btree_bload_node(
 
 		ASSERT(!xfs_btree_ptr_is_null(cur, child_ptr));
 
-		ret = xfs_btree_get_buf_block(cur, child_ptr, &child_block,
+		ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
 				&child_bp);

How is this (and making xfs_btree_read_buf_block outside of xfs_buf.c) related to the dirty limit?

Quoted 4 lines

Oh! Looking through my notes, I wanted the /new/ btree block buffers have the same lru reference count the old ones did. I probably should have exported xfs_btree_set_refs instead of reading whatever is on disk into a buffer, only to blow away the contents anyway.

And now that I've dug further through my notes, I've realized that there's a better reason for this unexplained _get_buf -> _read_buf transition and the setting of XBF_DONE in _delwri_queue_here.

This patch introduces the behavior that we flush the delwri list to disk every 256k. Flushing the buffers releases them, which means that reclaim could free the buffer before xfs_btree_bload_node needs it again to build the next level up. If that's the case, then _get_buf will get us a !DONE buffer with zeroes instead of reading the (freshly written) buffer back in from disk. We'll then end up formatting garbage keys into the node block, which is bad.

/*
* Read the lower-level block in case the buffer for it has
* been reclaimed. LRU refs will be set on the block, which is
* desirable if the new btree commits.
*/
ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
&child_bp);

The behavior of setting XBF_DONE in xfs_buf_delwri_queue_here is an optimization if _delwri_submit releases the buffer and it is /not/ reclaimed. In that case, xfs_btree_read_buf_block will find the buffer without the DONE flag set and reread the contents from disk, which is unnecessary.

I'll split those changes into separate patches with fuller explanations of what's going on.

--D

Quoted 2 lines

--D

#43 lore raw

Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree

Quoted 6 lines · Darrick J. Wong

And now that I've dug further through my notes, I've realized that there's a better reason for this unexplained _get_buf -> _read_buf transition and the setting of XBF_DONE in _delwri_queue_here.

This patch introduces the behavior that we flush the delwri list to disk every 256k.

Where "the delwri list" is the one used for writing stage btrees I think.

Quoted 3 lines

Flushing the buffers releases them, which means that
reclaim could free the buffer before xfs_btree_bload_node needs it again to build the next level up.

Oh, indeed.

Quoted 4 lines

If that's the case, then _get_buf will get
us a !DONE buffer with zeroes instead of reading the (freshly written) buffer back in from disk. We'll then end up formatting garbage keys into the node block, which is bad.

Yeah.

Quoted 8 lines

/*
* Read the lower-level block in case the buffer for it has
* been reclaimed. LRU refs will be set on the block, which is
* desirable if the new btree commits.
*/
ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
&child_bp);

Quoted 5 lines

The behavior of setting XBF_DONE in xfs_buf_delwri_queue_here is an optimization if _delwri_submit releases the buffer and it is /not/ reclaimed. In that case, xfs_btree_read_buf_block will find the buffer without the DONE flag set and reread the contents from disk, which is unnecessary.

Yeah. I still find it weird to set it in the delwri_submit_here helper, but maybe that's a discussion for the other thread.

#44 lore raw

Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree

Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong

And now that I've dug further through my notes, I've realized that there's a better reason for this unexplained _get_buf -> _read_buf transition and the setting of XBF_DONE in _delwri_queue_here.

This patch introduces the behavior that we flush the delwri list to disk every 256k.

Where "the delwri list" is the one used for writing stage btrees I think.

Correct.

Quoted 21 lines
Quoted 3 lines

Flushing the buffers releases them, which means that
reclaim could free the buffer before xfs_btree_bload_node needs it again to build the next level up.

Oh, indeed.

Quoted 4 lines

If that's the case, then _get_buf will get
us a !DONE buffer with zeroes instead of reading the (freshly written) buffer back in from disk. We'll then end up formatting garbage keys into the node block, which is bad.

Yeah.

Quoted 8 lines

/*
* Read the lower-level block in case the buffer for it has
* been reclaimed. LRU refs will be set on the block, which is
* desirable if the new btree commits.
*/
ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
&child_bp);

Quoted 8 lines
Quoted 5 lines

The behavior of setting XBF_DONE in xfs_buf_delwri_queue_here is an optimization if _delwri_submit releases the buffer and it is /not/ reclaimed. In that case, xfs_btree_read_buf_block will find the buffer without the DONE flag set and reread the contents from disk, which is unnecessary.

Yeah. I still find it weird to set it in the delwri_submit_here helper, but maybe that's a discussion for the other thread.

D'Oh! XBF_DONE exists in userspace too, because libxfs uses it. Well then, the proper place for it is at the top of xfs_btree_bload_drop_buf. I'll go change that tomorrow.

--D

#45 lore raw

cover xfs: online repair of AG btrees

In reply to Darrick J. Wong

Hi all,

Now that we've spent a lot of time reworking common code in online fsck, we're ready to start rebuilding the AG space btrees. This series implements repair functions for the free space, inode, and refcount btrees. Rebuilding the reverse mapping btree is much more intense and is left for a subsequent patchset. The fstests counterpart of this patchset implements stress testing of repair.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-ag-btrees

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-ag-btrees

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-ag-btrees

---
 fs/xfs/Makefile                    |    3 
 fs/xfs/libxfs/xfs_ag.h             |   10 
 fs/xfs/libxfs/xfs_ag_resv.c        |    2 
 fs/xfs/libxfs/xfs_alloc.c          |   18 -
 fs/xfs/libxfs/xfs_alloc.h          |    2 
 fs/xfs/libxfs/xfs_alloc_btree.c    |   13 -
 fs/xfs/libxfs/xfs_btree.c          |   26 +
 fs/xfs/libxfs/xfs_btree.h          |    2 
 fs/xfs/libxfs/xfs_ialloc.c         |   41 +-
 fs/xfs/libxfs/xfs_ialloc.h         |    3 
 fs/xfs/libxfs/xfs_refcount.c       |   18 -
 fs/xfs/libxfs/xfs_refcount.h       |    2 
 fs/xfs/libxfs/xfs_refcount_btree.c |   13 -
 fs/xfs/libxfs/xfs_types.h          |    7 
 fs/xfs/scrub/agheader_repair.c     |   18 -
 fs/xfs/scrub/alloc.c               |   16 +
 fs/xfs/scrub/alloc_repair.c        |  914 ++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/bitmap.c              |  404 +++++++++++++---
 fs/xfs/scrub/bitmap.h              |   70 ++-
 fs/xfs/scrub/common.c              |    1 
 fs/xfs/scrub/common.h              |   19 +
 fs/xfs/scrub/ialloc_repair.c       |  874 ++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/newbt.c               |   48 ++
 fs/xfs/scrub/newbt.h               |    6 
 fs/xfs/scrub/reap.c                |    5 
 fs/xfs/scrub/refcount_repair.c     |  793 +++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.c              |  128 +++++
 fs/xfs/scrub/repair.h              |   43 ++
 fs/xfs/scrub/scrub.c               |   22 +
 fs/xfs/scrub/scrub.h               |    9 
 fs/xfs/scrub/trace.h               |  112 +++-
 fs/xfs/scrub/xfarray.h             |   22 +
 fs/xfs/xfs_extent_busy.c           |   13 +
 fs/xfs/xfs_extent_busy.h           |    2 
 34 files changed, 3493 insertions(+), 186 deletions(-)
 create mode 100644 fs/xfs/scrub/alloc_repair.c
 create mode 100644 fs/xfs/scrub/ialloc_repair.c
 create mode 100644 fs/xfs/scrub/refcount_repair.c
#46 lore raw

PATCH 1/5 xfs: create separate structures and code for u32 bitmaps

From: Darrick J. Wong <[email protected]>

Create a version of the xbitmap that handles 32-bit integer intervals and adapt the xfs_agblock_t bitmap to use it. This reduces the size of the interval tree nodes from 48 to 36 bytes and enables us to use a more efficient slab (:0000040 instead of :0000048) which allows us to pack more nodes into a single slab page (102 vs 85).

As a side effect, the users of these bitmaps no longer have to convert between u32 and u64 quantities just to use the bitmap; and the hairy overflow checking code in xagb_bitmap_test goes away.

Later in this patchset we're going to add bitmaps for xfs_agino_t, xfs_rgblock_t, and xfs_dablk_t, so the increase in code size (5622 vs. 9959 bytes) seems worth it.

---
 fs/xfs/scrub/agheader_repair.c |    9 -
 fs/xfs/scrub/bitmap.c          |  404 ++++++++++++++++++++++++++++++++++------
 fs/xfs/scrub/bitmap.h          |   70 ++++---
 fs/xfs/scrub/reap.c            |    5 
 4 files changed, 390 insertions(+), 98 deletions(-)
fs/xfs/scrub/agheader_repair.c +4 -5
index 36c511f96b004..9d541d2087da8 100644
@@ -495,12 +495,11 @@ xrep_agfl_walk_rmap(
 /* Strike out the blocks that are cross-linked according to the rmapbt. */
 STATIC int
 xrep_agfl_check_extent(
-	uint64_t		start,
-	uint64_t		len,
+	uint32_t		agbno,
+	uint32_t		len,
 	void			*priv)
 {
 	struct xrep_agfl	*ra = priv;
-	xfs_agblock_t		agbno = start;
 	xfs_agblock_t		last_agbno = agbno + len - 1;
 	int			error;
 
@@ -648,8 +647,8 @@ struct xrep_agfl_fill {
 /* Fill the AGFL with whatever blocks are in this extent. */
 static int
 xrep_agfl_fill(
-	uint64_t		start,
-	uint64_t		len,
+	uint32_t		start,
+	uint32_t		len,
 	void			*priv)
 {
 	struct xrep_agfl_fill	*af = priv;
fs/xfs/scrub/bitmap.c +341 -63

419 lines not on this page · raw message

fs/xfs/scrub/bitmap.h +43 -27

121 lines not on this page · raw message

fs/xfs/scrub/reap.c +2 -3
index ee26fcb500b78..c8c8e3f9bc7a4 100644
@@ -430,13 +430,12 @@ xreap_agextent_iter(
  */
 STATIC int
 xreap_agmeta_extent(
-	uint64_t		fsbno,
-	uint64_t		len,
+	uint32_t		agbno,
+	uint32_t		len,
 	void			*priv)
 {
 	struct xreap_state	*rs = priv;
 	struct xfs_scrub	*sc = rs->sc;
-	xfs_agblock_t		agbno = fsbno;
 	xfs_agblock_t		agbno_next = agbno + len;
 	int			error = 0;
 
Reviewed-by on v1 1/5
#47 lore raw

Re: [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps

Quoted 5 lines · Darrick J. Wong

Create a version of the xbitmap that handles 32-bit integer intervals and adapt the xfs_agblock_t bitmap to use it. This reduces the size of the interval tree nodes from 48 to 36 bytes and enables us to use a more efficient slab (:0000040 instead of :0000048) which allows us to pack more nodes into a single slab page (102 vs 85).

The changes themsleves looks good:

Q: should we rename the existing xbitmap to xbitmap64 for consistency?

Also why are the agb_bitmap* wrappers in bitmap.h? Following our usual code organization I'd expect bitmap.[ch] to just be the library code and have users outside of that. Maybe for later..

#48 lore raw

Re: [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps

Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

Create a version of the xbitmap that handles 32-bit integer intervals and adapt the xfs_agblock_t bitmap to use it. This reduces the size of the interval tree nodes from 48 to 36 bytes and enables us to use a more efficient slab (:0000040 instead of :0000048) which allows us to pack more nodes into a single slab page (102 vs 85).

Quoted 5 lines

The changes themsleves looks good:

Q: should we rename the existing xbitmap to xbitmap64 for consistency?

Yes. Done.

Quoted 3 lines

Also why are the agb_bitmap* wrappers in bitmap.h? Following our usual code organization I'd expect bitmap.[ch] to just be the library code and have users outside of that. Maybe for later..

Those wrappers are trivial except for the enhanced typechecking, so I didn't think it was a big deal to cram them into bitmap.h.

--D

#49 lore raw

Re: [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps

Quoted 6 lines · Darrick J. Wong
Quoted 3 lines

Also why are the agb_bitmap* wrappers in bitmap.h? Following our usual code organization I'd expect bitmap.[ch] to just be the library code and have users outside of that. Maybe for later..

Those wrappers are trivial except for the enhanced typechecking, so I didn't think it was a big deal to cram them into bitmap.h.

I find that kind of code structure a bit confusion to be honest. If you prefer it I can live with it (at least fow now :)) of course.

#50 lore raw

PATCH 2/5 xfs: roll the scrub transaction after completing a repair

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

When we've finished repairing an AG header, roll the scrub transaction. This ensure that any failures caused by defer ops failing are captured by the xrep_done tracepoint and that any stacktraces that occur will point to the repair code that caused it, instead of xchk_teardown.

Going forward, repair functions should commit the transaction if they're going to return success. Usually the space reaping functions that run after a successful atomic commit of the new metadata will take care of that for us.

---
 fs/xfs/scrub/agheader_repair.c |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/scrub/agheader_repair.c +6 -3
index 9d541d2087da8..5dbf82705eab7 100644
@@ -73,7 +73,7 @@ xrep_superblock(
 	/* Write this to disk. */
 	xfs_trans_buf_set_type(sc->tp, bp, XFS_BLFT_SB_BUF);
 	xfs_trans_log_buf(sc->tp, bp, 0, BBTOB(bp->b_length) - 1);
-	return error;
+	return 0;
 }
 
 /* AGF */
@@ -342,7 +342,7 @@ xrep_agf_commit_new(
 	pag->pagf_refcount_level = be32_to_cpu(agf->agf_refcount_level);
 	set_bit(XFS_AGSTATE_AGF_INIT, &pag->pag_opstate);
 
-	return 0;
+	return xrep_roll_ag_trans(sc);
 }
 
 /* Repair the AGF. v5 filesystems only. */
@@ -789,6 +789,9 @@ xrep_agfl(
 	/* Dump any AGFL overflow. */
 	error = xrep_reap_agblocks(sc, &agfl_extents, &XFS_RMAP_OINFO_AG,
 			XFS_AG_RESV_AGFL);
+	if (error)
+		goto err;
+
 err:
 	xagb_bitmap_destroy(&agfl_extents);
 	return error;
@@ -962,7 +965,7 @@ xrep_agi_commit_new(
 	pag->pagi_freecount = be32_to_cpu(agi->agi_freecount);
 	set_bit(XFS_AGSTATE_AGI_INIT, &pag->pag_opstate);
 
-	return 0;
+	return xrep_roll_ag_trans(sc);
 }
 
 /* Repair the AGI. */
Reviewed-by on v1 2/5
#51 lore raw

Re: [PATCH 2/5] xfs: roll the scrub transaction after completing a repair

Quoted 4 lines · Darrick J. Wong

Going forward, repair functions should commit the transaction if they're going to return success. Usually the space reaping functions that run after a successful atomic commit of the new metadata will take care of that for us.

Generally looks good:

A random comment on a pre-existing function from reading the code, and a nitpick on the patch itself below:

fs/xfs/scrub/agheader_repair.c xrep_superblock( · line 76

Quoted 7 lines

+++ b/fs/xfs/scrub/agheader_repair.c

fs/xfs/scrub/agheader_repair.c +1 -1
@@ -73,7 +73,7 @@ xrep_superblock(
 	/* Write this to disk. */
 	xfs_trans_buf_set_type(sc->tp, bp, XFS_BLFT_SB_BUF);
 	xfs_trans_log_buf(sc->tp, bp, 0, BBTOB(bp->b_length) - 1);
-	return error;
+	return 0;

After looking through the code this is obviously fine, error must be 0 here because the last patch touching it is xchk_should_terminate, which only sets the error if it returns true.

But the calling conventions for xchk_should_terminate really make me scratch my head as they are so hard to reason about. I did quick look over must caller and most of them get there with error always set to 0. So just making xchk_should_terminate return the error would seem a lot better to me - any caller with a previous error would need a second error2, but that seems better than what we have there right now.

fs/xfs/scrub/agheader_repair.c xrep_agfl( · line 795

Quoted 9 lines
fs/xfs/scrub/agheader_repair.c +3 -0
 /* Repair the AGF. v5 filesystems only. */
@@ -789,6 +789,9 @@ xrep_agfl(
 	/* Dump any AGFL overflow. */
 	error = xrep_reap_agblocks(sc, &agfl_extents, &XFS_RMAP_OINFO_AG,
 			XFS_AG_RESV_AGFL);
+	if (error)
+		goto err;
+
 err:

This seems rather pointless and doesn't change anything..

#52 lore raw

Re: [PATCH 2/5] xfs: roll the scrub transaction after completing a repair

Quoted 13 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong

Going forward, repair functions should commit the transaction if they're going to return success. Usually the space reaping functions that run after a successful atomic commit of the new metadata will take care of that for us.

Generally looks good:

A random comment on a pre-existing function from reading the code, and a nitpick on the patch itself below:

Quoted 11 lines

fs/xfs/scrub/agheader_repair.c xrep_superblock( · line 76

Quoted 7 lines

+++ b/fs/xfs/scrub/agheader_repair.c

fs/xfs/scrub/agheader_repair.c +1 -1
@@ -73,7 +73,7 @@ xrep_superblock(
 	/* Write this to disk. */
 	xfs_trans_buf_set_type(sc->tp, bp, XFS_BLFT_SB_BUF);
 	xfs_trans_log_buf(sc->tp, bp, 0, BBTOB(bp->b_length) - 1);
-	return error;
+	return 0;

After looking through the code this is obviously fine, error must be 0 here because the last patch touching it is xchk_should_terminate, which only sets the error if it returns true.

<nod>

Quoted 7 lines

But the calling conventions for xchk_should_terminate really make me scratch my head as they are so hard to reason about. I did quick look over must caller and most of them get there with error always set to 0. So just making xchk_should_terminate return the error would seem a lot better to me - any caller with a previous error would need a second error2, but that seems better than what we have there right now.

Agreed, the callsites would be a bit more obvious if they looked like:

	error = xchk_should_terminate(sc);
	if (error)
		break;

Though I'm working on some tweaks of that function, since it was pointed out to me that cond_resched() and fatal_signal_pending() aren't entirely free. What I've been testing out the last three weeks is:

	unsigned long now = jiffies;

	if (time_after(sc->next_poke, now)) {
		sc->next_poke = now + (HZ / 10);

		cond_resched();

		if (fatal_signal_pending(current))
			return -EINTR;
	}
	return 0;

So far I haven't seen much improvement, but the callsite change is something that I think I could promote to the end of online repair part 2.

Quoted 11 lines

fs/xfs/scrub/agheader_repair.c xrep_agfl( · line 795

Quoted 9 lines
fs/xfs/scrub/agheader_repair.c +3 -0
 /* Repair the AGF. v5 filesystems only. */
@@ -789,6 +789,9 @@ xrep_agfl(
 	/* Dump any AGFL overflow. */
 	error = xrep_reap_agblocks(sc, &agfl_extents, &XFS_RMAP_OINFO_AG,
 			XFS_AG_RESV_AGFL);
+	if (error)
+		goto err;
+
 err:

This seems rather pointless and doesn't change anything..

Oops, lemme get rid of that dead code...

--D

Quoted 1 line
#53 lore raw

PATCH 3/5 xfs: repair free space btrees

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rebuild the free space btrees from the gaps in the rmap btree.

---
 fs/xfs/Makefile                 |    1 
 fs/xfs/libxfs/xfs_ag.h          |    9 
 fs/xfs/libxfs/xfs_ag_resv.c     |    2 
 fs/xfs/libxfs/xfs_alloc.c       |   18 +
 fs/xfs/libxfs/xfs_alloc.h       |    2 
 fs/xfs/libxfs/xfs_alloc_btree.c |   13 +
 fs/xfs/libxfs/xfs_types.h       |    7 
 fs/xfs/scrub/alloc.c            |   16 +
 fs/xfs/scrub/alloc_repair.c     |  914 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/common.h           |   19 +
 fs/xfs/scrub/newbt.c            |   48 ++
 fs/xfs/scrub/newbt.h            |    6 
 fs/xfs/scrub/repair.c           |   69 +++
 fs/xfs/scrub/repair.h           |   24 +
 fs/xfs/scrub/scrub.c            |   14 -
 fs/xfs/scrub/scrub.h            |    8 
 fs/xfs/scrub/trace.h            |   24 +
 fs/xfs/scrub/xfarray.h          |   22 +
 fs/xfs/xfs_extent_busy.c        |   13 +
 fs/xfs/xfs_extent_busy.h        |    2 
 20 files changed, 1212 insertions(+), 19 deletions(-)
 create mode 100644 fs/xfs/scrub/alloc_repair.c
fs/xfs/Makefile +1 -0
index 1537d66e5ab01..026591681937d 100644
@@ -181,6 +181,7 @@ xfs-$(CONFIG_XFS_QUOTA)		+= scrub/quota.o
 ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
+				   alloc_repair.o \
 				   newbt.o \
 				   reap.o \
 				   repair.o \
fs/xfs/libxfs/xfs_ag.h +9 -0
index 2e0aef87d633e..686f4eadd5743 100644
@@ -80,6 +80,15 @@ struct xfs_perag {
 	 */
 	uint16_t	pag_checked;
 	uint16_t	pag_sick;
+
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+	/*
+	 * Alternate btree heights so that online repair won't trip the write
+	 * verifiers while rebuilding the AG btrees.
+	 */
+	uint8_t		pagf_alt_levels[XFS_BTNUM_AGF];
+#endif
+
 	spinlock_t	pag_state_lock;
 
 	spinlock_t	pagb_lock;	/* lock for pagb_tree */
fs/xfs/libxfs/xfs_ag_resv.c +2 -0
index 7fd1fea95552f..da1057bd0e606 100644
@@ -411,6 +411,8 @@ xfs_ag_resv_free_extent(
 		fallthrough;
 	case XFS_AG_RESV_NONE:
 		xfs_trans_mod_sb(tp, XFS_TRANS_SB_FDBLOCKS, (int64_t)len);
+		fallthrough;
+	case XFS_AG_RESV_IGNORE:
 		return;
 	}
 
fs/xfs/libxfs/xfs_alloc.c +12 -6
index 4940f9377f21a..f33914455bf83 100644
@@ -243,14 +243,11 @@ xfs_alloc_btrec_to_irec(
 	irec->ar_blockcount = be32_to_cpu(rec->alloc.ar_blockcount);
 }
 
-/* Simple checks for free space records. */
-xfs_failaddr_t
-xfs_alloc_check_irec(
-	struct xfs_btree_cur		*cur,
+inline xfs_failaddr_t
+xfs_alloc_check_perag_irec(
+	struct xfs_perag		*pag,
 	const struct xfs_alloc_rec_incore *irec)
 {
-	struct xfs_perag		*pag = cur->bc_ag.pag;
-
 	if (irec->ar_blockcount == 0)
 		return __this_address;
 
@@ -261,6 +258,15 @@ xfs_alloc_check_irec(
 	return NULL;
 }
 
+/* Simple checks for free space records. */
+xfs_failaddr_t
+xfs_alloc_check_irec(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_alloc_rec_incore *irec)
+{
+	return xfs_alloc_check_perag_irec(cur->bc_ag.pag, irec);
+}
+
 static inline int
 xfs_alloc_complain_bad_rec(
 	struct xfs_btree_cur		*cur,
fs/xfs/libxfs/xfs_alloc.h +2 -0
index 851cafbd64494..68ffe058a420f 100644
@@ -185,6 +185,8 @@ xfs_alloc_get_rec(
 union xfs_btree_rec;
 void xfs_alloc_btrec_to_irec(const union xfs_btree_rec *rec,
 		struct xfs_alloc_rec_incore *irec);
+xfs_failaddr_t xfs_alloc_check_perag_irec(struct xfs_perag *pag,
+		const struct xfs_alloc_rec_incore *irec);
 xfs_failaddr_t xfs_alloc_check_irec(struct xfs_btree_cur *cur,
 		const struct xfs_alloc_rec_incore *irec);
 
fs/xfs/libxfs/xfs_alloc_btree.c +12 -1
index c65228efed4ae..90c7cb8c54ab0 100644
@@ -323,7 +323,18 @@ xfs_allocbt_verify(
 	if (bp->b_ops->magic[0] == cpu_to_be32(XFS_ABTC_MAGIC))
 		btnum = XFS_BTNUM_CNTi;
 	if (pag && xfs_perag_initialised_agf(pag)) {
-		if (level >= pag->pagf_levels[btnum])
+		unsigned int	maxlevel = pag->pagf_levels[btnum];
+
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+		/*
+		 * Online repair could be rewriting the free space btrees, so
+		 * we'll validate against the larger of either tree while this
+		 * is going on.
+		 */
+		maxlevel = max_t(unsigned int, maxlevel,
+				 pag->pagf_alt_levels[btnum]);
+#endif
+		if (level >= maxlevel)
 			return __this_address;
 	} else if (level >= mp->m_alloc_maxlevels)
 		return __this_address;
fs/xfs/libxfs/xfs_types.h +7 -0
index 533200c4ccc25..035bf703d719a 100644
@@ -208,6 +208,13 @@ enum xfs_ag_resv_type {
 	XFS_AG_RESV_AGFL,
 	XFS_AG_RESV_METADATA,
 	XFS_AG_RESV_RMAPBT,
+
+	/*
+	 * Don't increase fdblocks when freeing extent.  This is a pony for
+	 * the bnobt repair functions to re-free the free space without
+	 * altering fdblocks.  If you think you need this you're wrong.
+	 */
+	XFS_AG_RESV_IGNORE,
 };
 
 /* Results of scanning a btree keyspace to check occupancy. */
fs/xfs/scrub/alloc.c +14 -2
index 279af72b1671d..964089e24ca6d 100644
@@ -9,13 +9,16 @@
 #include "xfs_format.h"
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
+#include "xfs_log_format.h"
+#include "xfs_trans.h"
 #include "xfs_btree.h"
 #include "xfs_alloc.h"
 #include "xfs_rmap.h"
+#include "xfs_ag.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 #include "scrub/btree.h"
-#include "xfs_ag.h"
+#include "scrub/repair.h"
 
 /*
  * Set us up to scrub free space btrees.
@@ -24,10 +27,19 @@ int
 xchk_setup_ag_allocbt(
 	struct xfs_scrub	*sc)
 {
+	int			error;
+
 	if (xchk_need_intent_drain(sc))
 		xchk_fsgates_enable(sc, XCHK_FSGATES_DRAIN);
 
-	return xchk_setup_ag_btree(sc, false);
+	error = xchk_setup_ag_btree(sc, false);
+	if (error)
+		return error;
+
+	if (xchk_could_repair(sc))
+		return xrep_setup_ag_allocbt(sc);
+
+	return 0;
 }
 
 /* Free space btree scrubber. */
fs/xfs/scrub/alloc_repair.c new file +914 -0

914 lines not on this page · raw message

fs/xfs/scrub/common.h +19 -0
index a39dbe6be1e59..4b666f254d700 100644
@@ -194,8 +194,21 @@ static inline bool xchk_needs_repair(const struct xfs_scrub_metadata *sm)
 			       XFS_SCRUB_OFLAG_XCORRUPT |
 			       XFS_SCRUB_OFLAG_PREEN);
 }
+
+/*
+ * "Should we prepare for a repair?"
+ *
+ * Return true if the caller permits us to repair metadata and we're not
+ * setting up for a post-repair evaluation.
+ */
+static inline bool xchk_could_repair(const struct xfs_scrub *sc)
+{
+	return (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR) &&
+		!(sc->flags & XREP_ALREADY_FIXED);
+}
 #else
 # define xchk_needs_repair(sc)		(false)
+# define xchk_could_repair(sc)		(false)
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
 int xchk_metadata_inode_forks(struct xfs_scrub *sc);
@@ -207,6 +220,12 @@ int xchk_metadata_inode_forks(struct xfs_scrub *sc);
 #define xchk_xfile_descr(sc, fmt, ...) \
 	kasprintf(XCHK_GFP_FLAGS, "XFS (%s): " fmt, \
 			(sc)->mp->m_super->s_id, ##__VA_ARGS__)
+#define xchk_xfile_ag_descr(sc, fmt, ...) \
+	kasprintf(XCHK_GFP_FLAGS, "XFS (%s): AG 0x%x " fmt, \
+			(sc)->mp->m_super->s_id, \
+			(sc)->sa.pag ? (sc)->sa.pag->pag_agno : (sc)->sm->sm_agno, \
+			##__VA_ARGS__)
+
 
 /*
  * Setting up a hook to wait for intents to drain is costly -- we have to take
fs/xfs/scrub/newbt.c +44 -4

70 lines not on this page · raw message

fs/xfs/scrub/newbt.h +6 -0
index d2baffa17b1ae..c158d92562787 100644
@@ -50,6 +50,9 @@ struct xrep_newbt {
 	enum xfs_ag_resv_type	resv;
 };
 
+#define for_each_xrep_newbt_reservation(xnr, resv, n)	\
+	list_for_each_entry_safe((resv), (n), &(xnr)->resv_list, list)
+
 void xrep_newbt_init_bare(struct xrep_newbt *xnr, struct xfs_scrub *sc);
 void xrep_newbt_init_ag(struct xrep_newbt *xnr, struct xfs_scrub *sc,
 		const struct xfs_owner_info *oinfo, xfs_fsblock_t alloc_hint,
@@ -57,9 +60,12 @@ void xrep_newbt_init_ag(struct xrep_newbt *xnr, struct xfs_scrub *sc,
 int xrep_newbt_init_inode(struct xrep_newbt *xnr, struct xfs_scrub *sc,
 		int whichfork, const struct xfs_owner_info *oinfo);
 int xrep_newbt_alloc_blocks(struct xrep_newbt *xnr, uint64_t nr_blocks);
+int xrep_newbt_add_extent(struct xrep_newbt *xnr, struct xfs_perag *pag,
+		xfs_agblock_t agbno, xfs_extlen_t len);
 void xrep_newbt_cancel(struct xrep_newbt *xnr);
 int xrep_newbt_commit(struct xrep_newbt *xnr);
 int xrep_newbt_claim_block(struct xfs_btree_cur *cur, struct xrep_newbt *xnr,
 		union xfs_btree_ptr *ptr);
+unsigned int xrep_newbt_unused_blocks(struct xrep_newbt *xnr);
 
 #endif /* __XFS_SCRUB_NEWBT_H__ */
fs/xfs/scrub/repair.c +69 -0

72 lines not on this page · raw message

fs/xfs/scrub/repair.h +24 -0
index 60d2a9ae5f2ec..bc3353ecae8a1 100644
@@ -60,6 +60,15 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,
 void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);
 int xrep_ino_dqattach(struct xfs_scrub *sc);
 
+/* Repair setup functions */
+int xrep_setup_ag_allocbt(struct xfs_scrub *sc);
+
+void xrep_ag_btcur_init(struct xfs_scrub *sc, struct xchk_ag *sa);
+
+/* Metadata revalidators */
+
+int xrep_revalidate_allocbt(struct xfs_scrub *sc);
+
 /* Metadata repairers */
 
 int xrep_probe(struct xfs_scrub *sc);
@@ -67,6 +76,9 @@ int xrep_superblock(struct xfs_scrub *sc);
 int xrep_agf(struct xfs_scrub *sc);
 int xrep_agfl(struct xfs_scrub *sc);
 int xrep_agi(struct xfs_scrub *sc);
+int xrep_allocbt(struct xfs_scrub *sc);
+
+int xrep_reinit_pagf(struct xfs_scrub *sc);
 
 #else
 
@@ -87,11 +99,23 @@ xrep_calc_ag_resblks(
 	return 0;
 }
 
+/* repair setup functions for no-repair */
+static inline int
+xrep_setup_nothing(
+	struct xfs_scrub	*sc)
+{
+	return 0;
+}
+#define xrep_setup_ag_allocbt		xrep_setup_nothing
+
+#define xrep_revalidate_allocbt		(NULL)
+
 #define xrep_probe			xrep_notsupported
 #define xrep_superblock			xrep_notsupported
 #define xrep_agf			xrep_notsupported
 #define xrep_agfl			xrep_notsupported
 #define xrep_agi			xrep_notsupported
+#define xrep_allocbt			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +9 -5
index 474f4c4a9cd3b..ebfaeb3793154 100644
@@ -241,13 +241,15 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_PERAG,
 		.setup	= xchk_setup_ag_allocbt,
 		.scrub	= xchk_bnobt,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_allocbt,
+		.repair_eval = xrep_revalidate_allocbt,
 	},
 	[XFS_SCRUB_TYPE_CNTBT] = {	/* cntbt */
 		.type	= ST_PERAG,
 		.setup	= xchk_setup_ag_allocbt,
 		.scrub	= xchk_cntbt,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_allocbt,
+		.repair_eval = xrep_revalidate_allocbt,
 	},
 	[XFS_SCRUB_TYPE_INOBT] = {	/* inobt */
 		.type	= ST_PERAG,
@@ -533,7 +535,10 @@ xfs_scrub_metadata(
 
 	/* Scrub for errors. */
 	check_start = xchk_stats_now();
-	error = sc->ops->scrub(sc);
+	if ((sc->flags & XREP_ALREADY_FIXED) && sc->ops->repair_eval != NULL)
+		error = sc->ops->repair_eval(sc);
+	else
+		error = sc->ops->scrub(sc);
 	run.scrub_ns += xchk_stats_elapsed_ns(check_start);
 	if (error == -EDEADLOCK && !(sc->flags & XCHK_TRY_HARDER))
 		goto try_harder;
@@ -544,8 +549,7 @@ xfs_scrub_metadata(
 
 	xchk_update_health(sc);
 
-	if ((sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR) &&
-	    !(sc->flags & XREP_ALREADY_FIXED)) {
+	if (xchk_could_repair(sc)) {
 		bool needs_fix = xchk_needs_repair(sc->sm);
 
 		/* Userspace asked us to rebuild the structure regardless. */
fs/xfs/scrub/scrub.h +8 -0
index 1ef9c6b4842a1..cb6294e629836 100644
@@ -35,6 +35,14 @@ struct xchk_meta_ops {
 	/* Repair or optimize the metadata. */
 	int		(*repair)(struct xfs_scrub *);
 
+	/*
+	 * Re-scrub the metadata we repaired, in case there's extra work that
+	 * we need to do to check our repair work.  If this is NULL, we'll use
+	 * the ->scrub function pointer, assuming that the regular scrub is
+	 * sufficient.
+	 */
+	int		(*repair_eval)(struct xfs_scrub *sc);
+
 	/* Decide if we even have this piece of metadata. */
 	bool		(*has)(struct xfs_mount *);
 
fs/xfs/scrub/trace.h +23 -1
index aa76830753196..ea518712efa81 100644
@@ -1172,11 +1172,33 @@ DEFINE_EVENT(xrep_rmap_class, name, \
 		 xfs_agblock_t agbno, xfs_extlen_t len, \
 		 uint64_t owner, uint64_t offset, unsigned int flags), \
 	TP_ARGS(mp, agno, agbno, len, owner, offset, flags))
-DEFINE_REPAIR_RMAP_EVENT(xrep_alloc_extent_fn);
 DEFINE_REPAIR_RMAP_EVENT(xrep_ialloc_extent_fn);
 DEFINE_REPAIR_RMAP_EVENT(xrep_rmap_extent_fn);
 DEFINE_REPAIR_RMAP_EVENT(xrep_bmap_extent_fn);
 
+TRACE_EVENT(xrep_abt_found,
+	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
+		 const struct xfs_alloc_rec_incore *rec),
+	TP_ARGS(mp, agno, rec),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_agnumber_t, agno)
+		__field(xfs_agblock_t, startblock)
+		__field(xfs_extlen_t, blockcount)
+	),
+	TP_fast_assign(
+		__entry->dev = mp->m_super->s_dev;
+		__entry->agno = agno;
+		__entry->startblock = rec->ar_startblock;
+		__entry->blockcount = rec->ar_blockcount;
+	),
+	TP_printk("dev %d:%d agno 0x%x agbno 0x%x fsbcount 0x%x",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->agno,
+		  __entry->startblock,
+		  __entry->blockcount)
+)
+
 TRACE_EVENT(xrep_refcount_extent_fn,
 	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
 		 struct xfs_refcount_irec *irec),
fs/xfs/scrub/xfarray.h +22 -0
index 4ecac01363d9f..62b9c506fdd1b 100644
@@ -54,6 +54,28 @@ static inline int xfarray_append(struct xfarray *array, const void *ptr)
 uint64_t xfarray_length(struct xfarray *array);
 int xfarray_load_next(struct xfarray *array, xfarray_idx_t *idx, void *rec);
 
+/*
+ * Iterate the non-null elements in a sparse xfarray.  Callers should
+ * initialize *idx to XFARRAY_CURSOR_INIT before the first call; on return, it
+ * will be set to one more than the index of the record that was retrieved.
+ * Returns 1 if a record was retrieved, 0 if there weren't any more records, or
+ * a negative errno.
+ */
+static inline int
+xfarray_iter(
+	struct xfarray	*array,
+	xfarray_idx_t	*idx,
+	void		*rec)
+{
+	int ret = xfarray_load_next(array, idx, rec);
+
+	if (ret == -ENODATA)
+		return 0;
+	if (ret == 0)
+		return 1;
+	return ret;
+}
+
 /* Declarations for xfile array sort functionality. */
 
 typedef cmp_func_t xfarray_cmp_fn;
fs/xfs/xfs_extent_busy.c +13 -0
index 9ecfdcdc752f7..2ccde32c9a9e9 100644
@@ -678,3 +678,16 @@ xfs_extent_busy_ag_cmp(
 		diff = b1->bno - b2->bno;
 	return diff;
 }
+
+/* Are there any busy extents in this AG? */
+bool
+xfs_extent_busy_list_empty(
+	struct xfs_perag	*pag)
+{
+	bool			res;
+
+	spin_lock(&pag->pagb_lock);
+	res = RB_EMPTY_ROOT(&pag->pagb_tree);
+	spin_unlock(&pag->pagb_lock);
+	return res;
+}
fs/xfs/xfs_extent_busy.h +2 -0
index 0639aab336f3f..470032de31391 100644
@@ -85,4 +85,6 @@ static inline void xfs_extent_busy_sort(struct list_head *list)
 	list_sort(NULL, list, xfs_extent_busy_ag_cmp);
 }
 
+bool xfs_extent_busy_list_empty(struct xfs_perag *pag);
+
 #endif /* __XFS_EXTENT_BUSY_H__ */
#54 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rebuild the free space btrees from the gaps in the rmap btree.

This commit message feels a bit sparse for the amount of code added, although I can't really offer a good idea of what to add.

Otherwise just two comments on the interaction with the rest of the xfs code, I'll try to digest the new repair code a bit more in the meantime.

fs/xfs/libxfs/xfs_ag.h struct xfs_perag { · line 90

Quoted 7 lines
fs/xfs/libxfs/xfs_ag.h +7 -0
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+	/*
+	 * Alternate btree heights so that online repair won't trip the write
+	 * verifiers while rebuilding the AG btrees.
+	 */
+	uint8_t		pagf_alt_levels[XFS_BTNUM_AGF];
+#endif

Alternate and the alt_ prefix doesn't feel very descriptive. As far as I can tell these are about an ongoign repair, so as a at lest somewhat better choice call it "pagf_repair_levels"?

fs/xfs/libxfs/xfs_alloc.c xfs_alloc_check_irec( · line 268

Quoted 7 lines
fs/xfs/libxfs/xfs_alloc.c +7 -0
+xfs_failaddr_t
+xfs_alloc_check_irec(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_alloc_rec_incore *irec)
+{
+	return xfs_alloc_check_perag_irec(cur->bc_ag.pag, irec);
+}

Is there much of a point in even keeping this wrapper vs just switching xfs_alloc_check_irec to pass the pag instead of the cursor?

#55 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Rebuild the free space btrees from the gaps in the rmap btree.

This commit message feels a bit sparse for the amount of code added, although I can't really offer a good idea of what to add.

"Refer to the design documentation for more details:

?

Quoted 4 lines

Otherwise just two comments on the interaction with the rest of the xfs code, I'll try to digest the new repair code a bit more in the meantime.

Quoted 11 lines

fs/xfs/libxfs/xfs_ag.h struct xfs_perag { · line 90

Quoted 7 lines
fs/xfs/libxfs/xfs_ag.h +7 -0
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+	/*
+	 * Alternate btree heights so that online repair won't trip the write
+	 * verifiers while rebuilding the AG btrees.
+	 */
+	uint8_t		pagf_alt_levels[XFS_BTNUM_AGF];
+#endif

Alternate and the alt_ prefix doesn't feel very descriptive. As far as I can tell these are about an ongoign repair, so as a at lest somewhat better choice call it "pagf_repair_levels"?

Done.

Quoted 11 lines

fs/xfs/libxfs/xfs_alloc.c xfs_alloc_check_irec( · line 268

Quoted 7 lines
fs/xfs/libxfs/xfs_alloc.c +7 -0
+xfs_failaddr_t
+xfs_alloc_check_irec(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_alloc_rec_incore *irec)
+{
+	return xfs_alloc_check_perag_irec(cur->bc_ag.pag, irec);
+}

Is there much of a point in even keeping this wrapper vs just switching xfs_alloc_check_irec to pass the pag instead of the cursor?

Not really. I'll remove this from the next spin.

--D

#56 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

In reply to Darrick J. Wong

A highlevel question on how blocks are (re)used here.

- xrep_abt_find_freespace accounts the old allocbt blocks as freespace per the comment, although as far as I understand the code in xrep_abt_walk_rmap the allocbt blocks aren't actually added to the free_records xfarray, but recorded into the old_allocbt_blocks bitmap (btw, why are we using different data structures for them?)
- xrep_abt_reserve_space seems to allocate space for the new alloc btrees by popping the first entry of ->free_records until it has enough space.
- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?

I've also got a whole bunch of nitpicks that mostly don't require an immediate action and/or about existing code that just grows more users here:

fs/xfs/scrub/alloc_repair.c line 838

Quoted 12 lines
fs/xfs/scrub/alloc_repair.c new file +12 -0
+int
+xrep_allocbt(
+	struct xfs_scrub	*sc)
+{
+	struct xrep_abt		*ra;
+	struct xfs_mount	*mp = sc->mp;
+	char			*descr;
+	int			error;
+
+	/* We require the rmapbt to rebuild anything. */
+	if (!xfs_has_rmapbt(mp))
+		return -EOPNOTSUPP;

Shoudn't this be checked by the .has callback in struct xchk_meta_ops?

fs/xfs/scrub/alloc_repair.c line 863

Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6 -0
+	/* Set up enough storage to handle maximally fragmented free space. */
+	descr = xchk_xfile_ag_descr(sc, "free space records");
+	error = xfarray_create(descr, mp->m_sb.sb_agblocks / 2,
+			sizeof(struct xfs_alloc_rec_incore),
+			&ra->free_records);
+	kfree(descr);

Commenting on a new user of old code here, but why doesn't xfarray_create simply take a format string so that we don't need the separate allocatiom and kasprintf here?

fs/xfs/scrub/alloc_repair.c line 905

Quoted 7 lines
fs/xfs/scrub/alloc_repair.c new file +7 -0
+	/*
+	 * We must update sm_type temporarily so that the tree-to-tree cross
+	 * reference checks will work in the correct direction, and also so
+	 * that tracing will report correctly if there are more errors.
+	 */
+	sc->sm->sm_type = XFS_SCRUB_TYPE_BNOBT;
+	error = xchk_bnobt(sc);

So xchk_bnobt is a tiny wrapper around xchk_allocbt, which is a small wrapper around xchk_btree that basіally de-multiplex the argument pass in by xchk_bnobt again. This is existing code not newly added, but the call chain looks a bit odd to me.

Quoted 13 lines
fs/xfs/scrub/newbt.c +13 -0
+/*
+ * Add an extent to the new btree reservation pool.  Callers are required to
+ * reap this reservation manually if the repair is cancelled.  @pag must be a
+ * passive reference.
+ */
+int
+xrep_newbt_add_extent(
+	struct xrep_newbt	*xnr,
+	struct xfs_perag	*pag,
+	xfs_agblock_t		agbno,
+	xfs_extlen_t		len)
+{
+	struct xfs_mount	*mp = xnr->sc->mp;

fs/xfs/scrub/newbt.c xrep_newbt_add_blocks( · line 194

Quoted 10 lines
fs/xfs/scrub/newbt.c +10 -0
+	struct xfs_alloc_arg	args = {
+		.tp		= NULL, /* no autoreap */
+		.oinfo		= xnr->oinfo,
+		.fsbno		= XFS_AGB_TO_FSB(mp, pag->pag_agno, agbno),
+		.len		= len,
+		.resv		= xnr->resv,
+	};
+
+	return xrep_newbt_add_blocks(xnr, pag, &args);
+}

I don't quite understand what this helper adds, and the _blocks vs _extent naming is a bit confusing.

fs/xfs/scrub/newbt.h struct xrep_newbt { · line 54

Quoted 2 lines
fs/xfs/scrub/newbt.h +2 -0
+#define for_each_xrep_newbt_reservation(xnr, resv, n)	\
+	list_for_each_entry_safe((resv), (n), &(xnr)->resv_list, list)

I have to admit that I find the open code list_for_each_entry_safe easier to follow than such wrappers.

fs/xfs/scrub/repair.c xrep_ino_dqattach( · line 743

Quoted 6 lines
fs/xfs/scrub/repair.c +6 -0
+/* Initialize all the btree cursors for an AG repair. */
+void
+xrep_ag_btcur_init(
+	struct xfs_scrub	*sc,
+	struct xchk_ag		*sa)
+{

As far as I can tell this basically sets up cursors for all the btrees except the one that we want to repair, and the one that goes along with for the alloc and ialloc pairs? Maybe just spell that out for clarity.

#57 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 7 lines · Christoph Hellwig

A highlevel question on how blocks are (re)used here.

- xrep_abt_find_freespace accounts the old allocbt blocks as freespace per the comment, although as far as I understand the code in xrep_abt_walk_rmap the allocbt blocks aren't actually added to the free_records xfarray, but recorded into the old_allocbt_blocks bitmap (btw, why are we using different data structures for them?)

The old free space btree blocks are tracked via old_allocbt_blocks so that we can reap the space after committing the new btree roots. Reaping cross-references the set regions in the bitmap against the rmapbt records so that we don't free crosslinked blocks.

This is a big difference from xfs_repair, which constructs a free space map by walking the directory / bmbt trees and builds completely new indexes in the gaps. It gets away with that because it's building all the space metadata in the AG, not just the free space btrees.

The next question you might have is why there's old_allocbt_blocks and not_allocbt_blocks -- this is due to us using the AGFL to supply the bnobt, cntbt, and rmapbt's alloc_block routines. Hence the blocks tracked by all four data structures are all RMAP_OWN_AG, and we have to do a bit of bitmap work to subtract the rmapbt and AGFL blocks from all the OWN_AG records to end up with the blocks that we think are owned by the free space btrees.

Quoted 3 lines

- xrep_abt_reserve_space seems to allocate space for the new alloc btrees by popping the first entry of ->free_records until it has enough space.

Correct.

Quoted 3 lines

- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?

xrep_abt_reserve_space returns -ENOSPC, so we throw away all the incore records and throw the errno out to userspace. Across all the btree rebuilding code, the block reservation step happens as the very last thing before we start writing btree blocks, so it's still possible to back out cleanly.

Quoted 3 lines

I've also got a whole bunch of nitpicks that mostly don't require an immediate action and/or about existing code that just grows more users here:

Heh.

Quoted 6 lines
Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6 -0
+int
+xrep_allocbt(
+	struct xfs_scrub	*sc)
+{
+	struct xrep_abt		*ra;
+	struct xfs_mount	*mp = sc->mp;
Quoted 8 lines

fs/xfs/scrub/alloc_repair.c line 838

Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6 -0
+	char			*descr;
+	int			error;
+
+	/* We require the rmapbt to rebuild anything. */
+	if (!xfs_has_rmapbt(mp))
+		return -EOPNOTSUPP;

Shoudn't this be checked by the .has callback in struct xchk_meta_ops?

No. Checking doesn't require the rmapbt because all we do is walk the bnobt/cntbt records and cross-reference them with whatever metadata we can find.

A stronger check would scan the AG to build a second recordset of the free space and compare that against what's on disk. However, that would be much slower, and Dave wanted scans to be fast because corruptions are supposed to be the edge case. :)

The weaker checking also means we can scrub old filesystems, even if we still require xfs_repair to fix them.

Quoted 10 lines

fs/xfs/scrub/alloc_repair.c line 863

Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6 -0
+	/* Set up enough storage to handle maximally fragmented free space. */
+	descr = xchk_xfile_ag_descr(sc, "free space records");
+	error = xfarray_create(descr, mp->m_sb.sb_agblocks / 2,
+			sizeof(struct xfs_alloc_rec_incore),
+			&ra->free_records);
+	kfree(descr);

Commenting on a new user of old code here, but why doesn't xfarray_create simply take a format string so that we don't need the separate allocatiom and kasprintf here?

I didn't want to spend the brainpower to figure out how to make the macro and va_args crud work to support pushing both from xrep_allocbt -> xfarray_create -> xfile_create. I don't know how to make that stuff nest short of adding a kas- variant of xfile_create.

Seeing as we don't install xfiles into the file descriptor table anyway, the labels are only visible via ftrace, and not procfs. I decided that cleanliness here wasn't a high enough priority.

Quoted 4 lines
Quoted 4 lines
fs/xfs/scrub/alloc_repair.c new file +4 -0
+	/*
+	 * We must update sm_type temporarily so that the tree-to-tree cross
+	 * reference checks will work in the correct direction, and also so
+	 * that tracing will report correctly if there are more errors.
Quoted 8 lines

fs/xfs/scrub/alloc_repair.c line 905

Quoted 3 lines
fs/xfs/scrub/alloc_repair.c new file +3 -0
+	 */
+	sc->sm->sm_type = XFS_SCRUB_TYPE_BNOBT;
+	error = xchk_bnobt(sc);

So xchk_bnobt is a tiny wrapper around xchk_allocbt, which is a small wrapper around xchk_btree that basіally de-multiplex the argument pass in by xchk_bnobt again. This is existing code not newly added, but the call chain looks a bit odd to me.

Yeah. I suppose one way to clean that up would be to export xchk_allocbt to the dispatch table in scrub.c instead of xchk_{bno,cnt}bt and figure out which btree we want from sm_type.

Way back when I was designing scrub I thought that repair would be separate for each btree type, but that turned out not to be the case. Hence the awkwardness in the call chains.

Quoted 18 lines
Quoted 18 lines
fs/xfs/scrub/newbt.c +18 -0
+/*
+ * Add an extent to the new btree reservation pool.  Callers are required to
+ * reap this reservation manually if the repair is cancelled.  @pag must be a
+ * passive reference.
+ */
+int
+xrep_newbt_add_extent(
+	struct xrep_newbt	*xnr,
+	struct xfs_perag	*pag,
+	xfs_agblock_t		agbno,
+	xfs_extlen_t		len)
+{
+	struct xfs_mount	*mp = xnr->sc->mp;
+	struct xfs_alloc_arg	args = {
+		.tp		= NULL, /* no autoreap */
+		.oinfo		= xnr->oinfo,
+		.fsbno		= XFS_AGB_TO_FSB(mp, pag->pag_agno, agbno),
+		.len		= len,
Quoted 8 lines

fs/xfs/scrub/newbt.c xrep_newbt_add_blocks( · line 194

Quoted 5 lines
fs/xfs/scrub/newbt.c +5 -0
+		.resv		= xnr->resv,
+	};
+
+	return xrep_newbt_add_blocks(xnr, pag, &args);
+}

I don't quite understand what this helper adds, and the _blocks vs _extent naming is a bit confusing.

This wrapper simplifes the interface to xrep_newbt_add_blocks so that external callers don't have to know which magic values of xfs_alloc_arg are actually used by xrep_newbt_add_blocks and therefore need to be set.

For all the other repair functions, they have to allocate space from the free space btree, so xrep_newbt_add_blocks is passed the full _alloc_args as returned by the allocator to xrep_newbt_alloc_ag_blocks.

Quoted 5 lines

fs/xfs/scrub/newbt.h struct xrep_newbt { · line 54

Quoted 2 lines
fs/xfs/scrub/newbt.h +2 -0
+#define for_each_xrep_newbt_reservation(xnr, resv, n)	\
+	list_for_each_entry_safe((resv), (n), &(xnr)->resv_list, list)

I have to admit that I find the open code list_for_each_entry_safe easier to follow than such wrappers.

The funny part is that I don't even use it in newbt.c. Maybe it's time to get rid of it.

$ git grep for_each_xrep_newbt_reservation fs
fs/xfs/scrub/alloc_repair.c:568:        for_each_xrep_newbt_reservation(&ra->new_bnobt, resv, n) {
fs/xfs/scrub/alloc_repair.c:575:        for_each_xrep_newbt_reservation(&ra->new_bnobt, resv, n) {
fs/xfs/scrub/newbt.h:60:#define for_each_xrep_newbt_reservation(xnr, resv, n)   \
fs/xfs/scrub/rmap_repair.c:1126:        for_each_xrep_newbt_reservation(&rr->new_btree, resv, n) {
Quoted 11 lines

fs/xfs/scrub/repair.c xrep_ino_dqattach( · line 743

Quoted 6 lines
fs/xfs/scrub/repair.c +6 -0
+/* Initialize all the btree cursors for an AG repair. */
+void
+xrep_ag_btcur_init(
+	struct xfs_scrub	*sc,
+	struct xchk_ag		*sa)
+{

As far as I can tell this basically sets up cursors for all the btrees except the one that we want to repair, and the one that goes along with for the alloc and ialloc pairs? Maybe just spell that out for clarity.

Done.

--D

#58 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 8 lines · Darrick J. Wong
Quoted 5 lines

- xrep_abt_find_freespace accounts the old allocbt blocks as freespace per the comment, although as far as I understand the code in xrep_abt_walk_rmap the allocbt blocks aren't actually added to the free_records xfarray, but recorded into the old_allocbt_blocks bitmap (btw, why are we using different data structures for them?)

The old free space btree blocks are tracked via old_allocbt_blocks so that we can reap the space after committing the new btree roots.

Yes, I got that from the code. But I'm a bit confused about the comment treating the old allocbt blocks as free space, while the code doesn't. Or I guess the confusion is that we really have two slightly different notions of "free space":

1) the space we try to build the trees for
2) the space used as free space to us for the trees

The old allocbt blocks are part of 1 but not of 2.

Quoted 9 lines
Quoted 3 lines

- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?

xrep_abt_reserve_space returns -ENOSPC, so we throw away all the incore records and throw the errno out to userspace. Across all the btree rebuilding code, the block reservation step happens as the very last thing before we start writing btree blocks, so it's still possible to back out cleanly.

But that means we can't repair the allocation btrees for this case.

Quoted 9 lines
Quoted 5 lines

fs/xfs/scrub/alloc_repair.c line 838

Quoted 3 lines
fs/xfs/scrub/alloc_repair.c new file +3 -0
+	/* We require the rmapbt to rebuild anything. */
+	if (!xfs_has_rmapbt(mp))
+		return -EOPNOTSUPP;

Shoudn't this be checked by the .has callback in struct xchk_meta_ops?

No. Checking doesn't require the rmapbt because all we do is walk the bnobt/cntbt records and cross-reference them with whatever metadata we can find.

Oh, and .has applies to both checking and repairing. Gt it.

Quoted 7 lines

This wrapper simplifes the interface to xrep_newbt_add_blocks so that external callers don't have to know which magic values of xfs_alloc_arg are actually used by xrep_newbt_add_blocks and therefore need to be set.

For all the other repair functions, they have to allocate space from the free space btree, so xrep_newbt_add_blocks is passed the full _alloc_args as returned by the allocator to xrep_newbt_alloc_ag_blocks.

Ok. Maybe throw in a comment that it just is a convenience wrapper?

#59 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 15 lines · Christoph Hellwig
Quoted 8 lines · Darrick J. Wong
Quoted 5 lines

- xrep_abt_find_freespace accounts the old allocbt blocks as freespace per the comment, although as far as I understand the code in xrep_abt_walk_rmap the allocbt blocks aren't actually added to the free_records xfarray, but recorded into the old_allocbt_blocks bitmap (btw, why are we using different data structures for them?)

The old free space btree blocks are tracked via old_allocbt_blocks so that we can reap the space after committing the new btree roots.

Yes, I got that from the code. But I'm a bit confused about the comment treating the old allocbt blocks as free space, while the code doesn't. Or I guess the confusion is that we really have two slightly different notions of "free space":

Quoted 4 lines

1) the space we try to build the trees for
2) the space used as free space to us for the trees

The old allocbt blocks are part of 1 but not of 2.

Yeah. The blocks for #2 only come from the gaps in the rmapbt records that we find during the scan.

Part of the confusion here might be the naming -- at the end of the rmapbt scan, old_allocbt_blocks is set to all the blocks that the rmapbt says are OWN_AG. Each rmapbt block encountered during the scan is added to not_allocbt_blocks, and afterwards each block on the AGFL is also added to not_allocbt_blocks.

After that is where the definitions of old_allocbt_blocks shifts. This expression will help us to identify possible former bnobt/cntbt blocks:

	(OWN_AG blocks) & ~(rmapbt blocks | agfl blocks);

Substituting from above definitions, that becomes:

old_allocbt_blocks & ~not_allocbt_blocks

The OWN_AG bitmap itself isn't needed after this point, so what we really do instead is:

	old_allocbt_blocks &= ~not_allocbt_blocks;

IOWs, after this point, "old_allocbt_blocks" is a bitmap of alleged former bnobt/cntbt blocks. The xagb_bitmap_disunion operation modifies its first parameter in place to avoid copying records around.

The gaps (aka the new freespace records) are stashed in the free_records xfarray. Next, some of the @free_records are diverted to the newbt reservation and used to format new btree blocks.

I hope that makes things clearer?

Quoted 11 lines
Quoted 9 lines
Quoted 3 lines

- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?

xrep_abt_reserve_space returns -ENOSPC, so we throw away all the incore records and throw the errno out to userspace. Across all the btree rebuilding code, the block reservation step happens as the very last thing before we start writing btree blocks, so it's still possible to back out cleanly.

But that means we can't repair the allocation btrees for this case.

Yep. At least we revert cleanly here -- when that happens to xfs_repair it just blows up and leaves a dead fs. :/

Practically speaking, the rmapbt reservations are large enough that we can rebuild the trees if we have to, even if that just means stealing from the per-AG space reservations later.

Though you could create the fs from hell by using reflink and cow to fragmenting the rmapbt until it swells up and consumes the entire AG. That would require billions of fragments; even with my bmap inflation xfs_db commands that still takes about 2 days to get the rmapbt to the point of overflowing the regular reservation.

I tried pushing it to eat an entire AG but then the ILOM exploded and no idea what happened to that machine and its 512G of RAM. The lab people kind of hate me now.

Quoted 11 lines
Quoted 9 lines
Quoted 5 lines

fs/xfs/scrub/alloc_repair.c line 838

Quoted 3 lines
fs/xfs/scrub/alloc_repair.c new file +3 -0
+	/* We require the rmapbt to rebuild anything. */
+	if (!xfs_has_rmapbt(mp))
+		return -EOPNOTSUPP;

Shoudn't this be checked by the .has callback in struct xchk_meta_ops?

No. Checking doesn't require the rmapbt because all we do is walk the bnobt/cntbt records and cross-reference them with whatever metadata we can find.

Oh, and .has applies to both checking and repairing. Gt it.

Correct.

Quoted 9 lines
Quoted 7 lines

This wrapper simplifes the interface to xrep_newbt_add_blocks so that external callers don't have to know which magic values of xfs_alloc_arg are actually used by xrep_newbt_add_blocks and therefore need to be set.

For all the other repair functions, they have to allocate space from the free space btree, so xrep_newbt_add_blocks is passed the full _alloc_args as returned by the allocator to xrep_newbt_alloc_ag_blocks.

Ok. Maybe throw in a comment that it just is a convenience wrapper?

Will do.

--D

#60 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 5 lines · Darrick J. Wong

The gaps (aka the new freespace records) are stashed in the free_records xfarray. Next, some of the @free_records are diverted to the newbt reservation and used to format new btree blocks.

I hope that makes things clearer?

Yes. Can you add this to the code as a comment?

#61 lore raw

Re: [PATCH 3/5] xfs: repair free space btrees

Quoted 8 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

The gaps (aka the new freespace records) are stashed in the free_records xfarray. Next, some of the @free_records are diverted to the newbt reservation and used to format new btree blocks.

I hope that makes things clearer?

Yes. Can you add this to the code as a comment?

Will do.

--D

#62 lore raw

PATCH 4/5 xfs: repair inode btrees

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use the rmapbt to find inode chunks, query the chunks to compute hole and free masks, and with that information rebuild the inobt and finobt.

---
 fs/xfs/Makefile              |    1 
 fs/xfs/libxfs/xfs_ialloc.c   |   41 +-
 fs/xfs/libxfs/xfs_ialloc.h   |    3 
 fs/xfs/scrub/common.c        |    1 
 fs/xfs/scrub/ialloc_repair.c |  874 ++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.c        |   59 +++
 fs/xfs/scrub/repair.h        |   17 +
 fs/xfs/scrub/scrub.c         |    6 
 fs/xfs/scrub/scrub.h         |    1 
 fs/xfs/scrub/trace.h         |   68 ++-
 10 files changed, 1022 insertions(+), 49 deletions(-)
 create mode 100644 fs/xfs/scrub/ialloc_repair.c
fs/xfs/Makefile +1 -0
index 026591681937d..7fed0e706cfa0 100644
@@ -182,6 +182,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   alloc_repair.o \
+				   ialloc_repair.o \
 				   newbt.o \
 				   reap.o \
 				   repair.o \
fs/xfs/libxfs/xfs_ialloc.c +28 -13

63 lines not on this page · raw message

fs/xfs/libxfs/xfs_ialloc.h +3 -0
index fe824bb04a091..012aac5671bad 100644
@@ -79,6 +79,7 @@ int xfs_inobt_lookup(struct xfs_btree_cur *cur, xfs_agino_t ino,
  */
 int xfs_inobt_get_rec(struct xfs_btree_cur *cur,
 		xfs_inobt_rec_incore_t *rec, int *stat);
+uint8_t xfs_inobt_rec_freecount(const struct xfs_inobt_rec_incore *irec);
 
 /*
  * Inode chunk initialisation routine
@@ -93,6 +94,8 @@ union xfs_btree_rec;
 void xfs_inobt_btrec_to_irec(struct xfs_mount *mp,
 		const union xfs_btree_rec *rec,
 		struct xfs_inobt_rec_incore *irec);
+xfs_failaddr_t xfs_inobt_check_perag_irec(struct xfs_perag *pag,
+		const struct xfs_inobt_rec_incore *irec);
 xfs_failaddr_t xfs_inobt_check_irec(struct xfs_btree_cur *cur,
 		const struct xfs_inobt_rec_incore *irec);
 int xfs_ialloc_has_inodes_at_extent(struct xfs_btree_cur *cur,
fs/xfs/scrub/common.c +1 -0
index 4bba3c49f8c59..b6725b05fb417 100644
@@ -605,6 +605,7 @@ xchk_ag_free(
 	struct xchk_ag		*sa)
 {
 	xchk_ag_btcur_free(sa);
+	xrep_reset_perag_resv(sc);
 	if (sa->agf_bp) {
 		xfs_trans_brelse(sc->tp, sa->agf_bp);
 		sa->agf_bp = NULL;
fs/xfs/scrub/ialloc_repair.c new file +874 -0

874 lines not on this page · raw message

fs/xfs/scrub/repair.c +59 -0

62 lines not on this page · raw message

fs/xfs/scrub/repair.h +17 -0
index bc3353ecae8a1..05bd55430e6eb 100644
@@ -59,6 +59,7 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,
 		struct xrep_find_ag_btree *btree_info, struct xfs_buf *agfl_bp);
 void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);
 int xrep_ino_dqattach(struct xfs_scrub *sc);
+int xrep_reset_perag_resv(struct xfs_scrub *sc);
 
 /* Repair setup functions */
 int xrep_setup_ag_allocbt(struct xfs_scrub *sc);
@@ -68,6 +69,7 @@ void xrep_ag_btcur_init(struct xfs_scrub *sc, struct xchk_ag *sa);
 /* Metadata revalidators */
 
 int xrep_revalidate_allocbt(struct xfs_scrub *sc);
+int xrep_revalidate_iallocbt(struct xfs_scrub *sc);
 
 /* Metadata repairers */
 
@@ -77,8 +79,10 @@ int xrep_agf(struct xfs_scrub *sc);
 int xrep_agfl(struct xfs_scrub *sc);
 int xrep_agi(struct xfs_scrub *sc);
 int xrep_allocbt(struct xfs_scrub *sc);
+int xrep_iallocbt(struct xfs_scrub *sc);
 
 int xrep_reinit_pagf(struct xfs_scrub *sc);
+int xrep_reinit_pagi(struct xfs_scrub *sc);
 
 #else
 
@@ -99,6 +103,17 @@ xrep_calc_ag_resblks(
 	return 0;
 }
 
+static inline int
+xrep_reset_perag_resv(
+	struct xfs_scrub	*sc)
+{
+	if (!(sc->flags & XREP_RESET_PERAG_RESV))
+		return 0;
+
+	ASSERT(0);
+	return -EOPNOTSUPP;
+}
+
 /* repair setup functions for no-repair */
 static inline int
 xrep_setup_nothing(
@@ -109,6 +124,7 @@ xrep_setup_nothing(
 #define xrep_setup_ag_allocbt		xrep_setup_nothing
 
 #define xrep_revalidate_allocbt		(NULL)
+#define xrep_revalidate_iallocbt	(NULL)
 
 #define xrep_probe			xrep_notsupported
 #define xrep_superblock			xrep_notsupported
@@ -116,6 +132,7 @@ xrep_setup_nothing(
 #define xrep_agfl			xrep_notsupported
 #define xrep_agi			xrep_notsupported
 #define xrep_allocbt			xrep_notsupported
+#define xrep_iallocbt			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +4 -2
index ebfaeb3793154..dfa949666e291 100644
@@ -255,14 +255,16 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_PERAG,
 		.setup	= xchk_setup_ag_iallocbt,
 		.scrub	= xchk_inobt,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_iallocbt,
+		.repair_eval = xrep_revalidate_iallocbt,
 	},
 	[XFS_SCRUB_TYPE_FINOBT] = {	/* finobt */
 		.type	= ST_PERAG,
 		.setup	= xchk_setup_ag_iallocbt,
 		.scrub	= xchk_finobt,
 		.has	= xfs_has_finobt,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_iallocbt,
+		.repair_eval = xrep_revalidate_iallocbt,
 	},
 	[XFS_SCRUB_TYPE_RMAPBT] = {	/* rmapbt */
 		.type	= ST_PERAG,
fs/xfs/scrub/scrub.h +1 -0
index cb6294e629836..4e6edff57b857 100644
@@ -121,6 +121,7 @@ struct xfs_scrub {
 #define XCHK_HAVE_FREEZE_PROT	(1U << 1)  /* do we have freeze protection? */
 #define XCHK_FSGATES_DRAIN	(1U << 2)  /* defer ops draining enabled */
 #define XCHK_NEED_DRAIN		(1U << 3)  /* scrub needs to drain defer ops */
+#define XREP_RESET_PERAG_RESV	(1U << 30) /* must reset AG space reservation */
 #define XREP_ALREADY_FIXED	(1U << 31) /* checking our repair work */
 
 /*
fs/xfs/scrub/trace.h +34 -34

92 lines not on this page · raw message

#63 lore raw

Re: [PATCH 4/5] xfs: repair inode btrees

fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_check_irec( · line 142

Quoted 8 lines
fs/xfs/libxfs/xfs_ialloc.c +8 -0
+/* Simple checks for inode records. */
+xfs_failaddr_t
+xfs_inobt_check_irec(
+	struct xfs_btree_cur			*cur,
+	const struct xfs_inobt_rec_incore	*irec)
+{
+	return xfs_inobt_check_perag_irec(cur->bc_ag.pag, irec);
+}

Same comment about just dropping the wrapper. Otherwise I'll need more digestion time for the new code.

#64 lore raw

Re: [PATCH 4/5] xfs: repair inode btrees

Quoted 11 lines · Christoph Hellwig

fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_check_irec( · line 142

Quoted 8 lines
fs/xfs/libxfs/xfs_ialloc.c +8 -0
+/* Simple checks for inode records. */
+xfs_failaddr_t
+xfs_inobt_check_irec(
+	struct xfs_btree_cur			*cur,
+	const struct xfs_inobt_rec_incore	*irec)
+{
+	return xfs_inobt_check_perag_irec(cur->bc_ag.pag, irec);
+}

Same comment about just dropping the wrapper. Otherwise I'll need more digestion time for the new code.

Done.

--D

#65 lore raw

Re: [PATCH 4/5] xfs: repair inode btrees

In reply to Darrick J. Wong

This generally looks good to me.

A bunch of my superficial comments to the previous patch apply here as well, but I'm not going to repeat them, but I have a bunch of new just as nitpicky ones:

fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_btrec_to_irec( · line 108

Quoted 6 lines
fs/xfs/libxfs/xfs_ialloc.c +5 -0
+	uint64_t				realfree;
 
+	if (!xfs_inobt_issparse(irec->ir_holemask))
+		realfree = irec->ir_free;
+	else
+		realfree = irec->ir_free & xfs_inobt_irec_to_allocmask(irec);

Nit:

I'd write this as:

	uint64_t				realfree = irec->ir_free;

	if (xfs_inobt_issparse(irec->ir_holemask))
		realfree &= xfs_inobt_irec_to_allocmask(irec);
	return hweight64(realfree);

to simplify the logic a bit (and yes, I see the sniplet was just copied out of an existing function).

Quoted 12 lines
fs/xfs/scrub/ialloc_repair.c new file +12 -0
+/* Record extents that belong to inode btrees. */
+STATIC int
+xrep_ibt_walk_rmap(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_rmap_irec	*rec,
+	void				*priv)
+{
+	struct xrep_ibt			*ri = priv;
+	struct xfs_mount		*mp = cur->bc_mp;
+	struct xfs_ino_geometry		*igeo = M_IGEO(mp);
+	xfs_agblock_t			cluster_base;
+	int				error = 0;

fs/xfs/scrub/ialloc_repair.c line 430

Quoted 10 lines
fs/xfs/scrub/ialloc_repair.c new file +10 -0
+
+	if (xchk_should_terminate(ri->sc, &error))
+		return error;
+
+	if (rec->rm_owner == XFS_RMAP_OWN_INOBT)
+		return xrep_ibt_record_old_btree_blocks(ri, rec);
+
+	/* Skip extents which are not owned by this inode and fork. */
+	if (rec->rm_owner != XFS_RMAP_OWN_INODES)
+		return 0;

The "Skip extents.." comment is clearly wrong and looks like it got here by accident. And may ocaml-trained ind screams for a switch statement and another helper for the rest of the functin body here:

	switch (rec->rm_owner) {
	case XFS_RMAP_OWN_INOBT:
		return xrep_ibt_record_old_btree_blocks(ri, rec);
	case XFS_RMAP_OWN_INODES:
		return xrep_ibt_record_inode_blocks(mp, ri, rec);
	default:
		return 0;

fs/xfs/scrub/ialloc_repair.c line 484

Quoted 6 lines
fs/xfs/scrub/ialloc_repair.c new file +6 -0
+	/* If we have a record ready to go, add it to the array. */
+	if (ri->rie.ir_startino == NULLAGINO)
+		return 0;
+
+	return xrep_ibt_stash(ri);
+}

Superficial, but having the logic inverted from the comment makes my brain a little dizzy. Anything again:

	if (ri->rie.ir_startino != NULLAGINO)
		error = xrep_ibt_stash(ri);

	return error;

?

fs/xfs/scrub/ialloc_repair.c line 594

Quoted 3 lines
fs/xfs/scrub/ialloc_repair.c new file +3 -0
+/* Make sure the records do not overlap in inumber address space. */
+STATIC int
+xrep_ibt_check_startino(

Would xrep_ibt_check_overlap be a better name here?

#66 lore raw

Re: [PATCH 4/5] xfs: repair inode btrees

Quoted 5 lines · Christoph Hellwig

This generally looks good to me.

A bunch of my superficial comments to the previous patch apply here as well, but I'm not going to repeat them, but I have a bunch of new just as nitpicky ones:

I already fixed the nitpicks from yesterday. :)

Quoted 12 lines

fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_btrec_to_irec( · line 108

Quoted 6 lines
fs/xfs/libxfs/xfs_ialloc.c +5 -0
+	uint64_t				realfree;
 
+	if (!xfs_inobt_issparse(irec->ir_holemask))
+		realfree = irec->ir_free;
+	else
+		realfree = irec->ir_free & xfs_inobt_irec_to_allocmask(irec);

Nit:

I'd write this as:

Quoted 8 lines
	uint64_t				realfree = irec->ir_free;

	if (xfs_inobt_issparse(irec->ir_holemask))
		realfree &= xfs_inobt_irec_to_allocmask(irec);
	return hweight64(realfree);

to simplify the logic a bit (and yes, I see the sniplet was just copied out of an existing function).

Ok.

Quoted 17 lines
Quoted 17 lines
fs/xfs/scrub/ialloc_repair.c new file +17 -0
+/* Record extents that belong to inode btrees. */
+STATIC int
+xrep_ibt_walk_rmap(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_rmap_irec	*rec,
+	void				*priv)
+{
+	struct xrep_ibt			*ri = priv;
+	struct xfs_mount		*mp = cur->bc_mp;
+	struct xfs_ino_geometry		*igeo = M_IGEO(mp);
+	xfs_agblock_t			cluster_base;
+	int				error = 0;
+
+	if (xchk_should_terminate(ri->sc, &error))
+		return error;
+
+	if (rec->rm_owner == XFS_RMAP_OWN_INOBT)
Quoted 8 lines

fs/xfs/scrub/ialloc_repair.c line 430

Quoted 5 lines
fs/xfs/scrub/ialloc_repair.c new file +5 -0
+		return xrep_ibt_record_old_btree_blocks(ri, rec);
+
+	/* Skip extents which are not owned by this inode and fork. */
+	if (rec->rm_owner != XFS_RMAP_OWN_INODES)
+		return 0;

The "Skip extents.." comment is clearly wrong and looks like it got here by accident.

Yep. "Skip mappings that are not inode records.", sorry about that.

Quoted 10 lines

And may ocaml-trained ind screams for a switch
statement and another helper for the rest of the functin body here:

	switch (rec->rm_owner) {
	case XFS_RMAP_OWN_INOBT:
		return xrep_ibt_record_old_btree_blocks(ri, rec);
	case XFS_RMAP_OWN_INODES:
		return xrep_ibt_record_inode_blocks(mp, ri, rec);
	default:
		return 0;

Sounds good to me.

Quoted 10 lines

fs/xfs/scrub/ialloc_repair.c line 484

Quoted 6 lines
fs/xfs/scrub/ialloc_repair.c new file +6 -0
+	/* If we have a record ready to go, add it to the array. */
+	if (ri->rie.ir_startino == NULLAGINO)
+		return 0;
+
+	return xrep_ibt_stash(ri);
+}

Superficial, but having the logic inverted from the comment makes my brain a little dizzy. Anything again:

Quoted 6 lines
	if (ri->rie.ir_startino != NULLAGINO)
		error = xrep_ibt_stash(ri);

	return error;

?

Done.

Quoted 5 lines

fs/xfs/scrub/ialloc_repair.c line 594

Quoted 3 lines
fs/xfs/scrub/ialloc_repair.c new file +3 -0
+/* Make sure the records do not overlap in inumber address space. */
+STATIC int
+xrep_ibt_check_startino(

Would xrep_ibt_check_overlap be a better name here?

Yes! Thank you for the suggestion.

--D

Quoted 2 lines
#67 lore raw

PATCH 5/5 xfs: repair refcount btrees

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Reconstruct the refcount data from the rmap btree.

---
 fs/xfs/Makefile                    |    1 
 fs/xfs/libxfs/xfs_ag.h             |    1 
 fs/xfs/libxfs/xfs_btree.c          |   26 +
 fs/xfs/libxfs/xfs_btree.h          |    2 
 fs/xfs/libxfs/xfs_refcount.c       |   18 +
 fs/xfs/libxfs/xfs_refcount.h       |    2 
 fs/xfs/libxfs/xfs_refcount_btree.c |   13 +
 fs/xfs/scrub/refcount_repair.c     |  793 ++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h              |    2 
 fs/xfs/scrub/scrub.c               |    2 
 fs/xfs/scrub/trace.h               |   22 +
 11 files changed, 864 insertions(+), 18 deletions(-)
 create mode 100644 fs/xfs/scrub/refcount_repair.c
fs/xfs/Makefile +1 -0
index 7fed0e706cfa0..a6f708dc56cc2 100644
@@ -185,6 +185,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   ialloc_repair.o \
 				   newbt.o \
 				   reap.o \
+				   refcount_repair.o \
 				   repair.o \
 				   )
 endif
fs/xfs/libxfs/xfs_ag.h +1 -0
index 686f4eadd5743..616812911a23f 100644
@@ -87,6 +87,7 @@ struct xfs_perag {
 	 * verifiers while rebuilding the AG btrees.
 	 */
 	uint8_t		pagf_alt_levels[XFS_BTNUM_AGF];
+	uint8_t		pagf_alt_refcount_level;
 #endif
 
 	spinlock_t	pag_state_lock;
fs/xfs/libxfs/xfs_btree.c +26 -0
index c100e92140be1..ea8d3659df208 100644
@@ -5212,3 +5212,29 @@ xfs_btree_destroy_cur_caches(void)
 	xfs_rmapbt_destroy_cur_cache();
 	xfs_refcountbt_destroy_cur_cache();
 }
+
+/* Move the btree cursor before the first record. */
+int
+xfs_btree_goto_left_edge(
+	struct xfs_btree_cur	*cur)
+{
+	int			stat = 0;
+	int			error;
+
+	memset(&cur->bc_rec, 0, sizeof(cur->bc_rec));
+	error = xfs_btree_lookup(cur, XFS_LOOKUP_LE, &stat);
+	if (error)
+		return error;
+	if (!stat)
+		return 0;
+
+	error = xfs_btree_decrement(cur, 0, &stat);
+	if (error)
+		return error;
+	if (stat != 0) {
+		ASSERT(0);
+		return -EFSCORRUPTED;
+	}
+
+	return 0;
+}
fs/xfs/libxfs/xfs_btree.h +2 -0
index e0875cec49392..d906324e25c86 100644
@@ -738,4 +738,6 @@ xfs_btree_alloc_cursor(
 int __init xfs_btree_init_cur_caches(void);
 void xfs_btree_destroy_cur_caches(void);
 
+int xfs_btree_goto_left_edge(struct xfs_btree_cur *cur);
+
 #endif	/* __XFS_BTREE_H__ */
fs/xfs/libxfs/xfs_refcount.c +12 -6
index 3702b4a071100..5fa1a6f32c17d 100644
@@ -120,14 +120,11 @@ xfs_refcount_btrec_to_irec(
 	irec->rc_refcount = be32_to_cpu(rec->refc.rc_refcount);
 }
 
-/* Simple checks for refcount records. */
-xfs_failaddr_t
-xfs_refcount_check_irec(
-	struct xfs_btree_cur		*cur,
+inline xfs_failaddr_t
+xfs_refcount_check_perag_irec(
+	struct xfs_perag		*pag,
 	const struct xfs_refcount_irec	*irec)
 {
-	struct xfs_perag		*pag = cur->bc_ag.pag;
-
 	if (irec->rc_blockcount == 0 || irec->rc_blockcount > MAXREFCEXTLEN)
 		return __this_address;
 
@@ -144,6 +141,15 @@ xfs_refcount_check_irec(
 	return NULL;
 }
 
+/* Simple checks for refcount records. */
+xfs_failaddr_t
+xfs_refcount_check_irec(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_refcount_irec	*irec)
+{
+	return xfs_refcount_check_perag_irec(cur->bc_ag.pag, irec);
+}
+
 static inline int
 xfs_refcount_complain_bad_rec(
 	struct xfs_btree_cur		*cur,
fs/xfs/libxfs/xfs_refcount.h +2 -0
index 783cd89ca1951..2d6fecb258bb1 100644
@@ -117,6 +117,8 @@ extern int xfs_refcount_has_records(struct xfs_btree_cur *cur,
 union xfs_btree_rec;
 extern void xfs_refcount_btrec_to_irec(const union xfs_btree_rec *rec,
 		struct xfs_refcount_irec *irec);
+xfs_failaddr_t xfs_refcount_check_perag_irec(struct xfs_perag *pag,
+		const struct xfs_refcount_irec *irec);
 xfs_failaddr_t xfs_refcount_check_irec(struct xfs_btree_cur *cur,
 		const struct xfs_refcount_irec *irec);
 extern int xfs_refcount_insert(struct xfs_btree_cur *cur,
fs/xfs/libxfs/xfs_refcount_btree.c +12 -1
index 3fa795e2488dd..dc5d6d4d6316c 100644
@@ -226,7 +226,18 @@ xfs_refcountbt_verify(
 
 	level = be16_to_cpu(block->bb_level);
 	if (pag && xfs_perag_initialised_agf(pag)) {
-		if (level >= pag->pagf_refcount_level)
+		unsigned int	maxlevel = pag->pagf_refcount_level;
+
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+		/*
+		 * Online repair could be rewriting the refcount btree, so
+		 * we'll validate against the larger of either tree while this
+		 * is going on.
+		 */
+		maxlevel = max_t(unsigned int, maxlevel,
+				pag->pagf_alt_refcount_level);
+#endif
+		if (level >= maxlevel)
 			return __this_address;
 	} else if (level >= mp->m_refc_maxlevels)
 		return __this_address;
fs/xfs/scrub/refcount_repair.c new file +793 -0

793 lines not on this page · raw message

fs/xfs/scrub/repair.h +2 -0
index 05bd55430e6eb..cc7ea39427296 100644
@@ -80,6 +80,7 @@ int xrep_agfl(struct xfs_scrub *sc);
 int xrep_agi(struct xfs_scrub *sc);
 int xrep_allocbt(struct xfs_scrub *sc);
 int xrep_iallocbt(struct xfs_scrub *sc);
+int xrep_refcountbt(struct xfs_scrub *sc);
 
 int xrep_reinit_pagf(struct xfs_scrub *sc);
 int xrep_reinit_pagi(struct xfs_scrub *sc);
@@ -133,6 +134,7 @@ xrep_setup_nothing(
 #define xrep_agi			xrep_notsupported
 #define xrep_allocbt			xrep_notsupported
 #define xrep_iallocbt			xrep_notsupported
+#define xrep_refcountbt			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +1 -1
index dfa949666e291..d0d6b2b41219e 100644
@@ -278,7 +278,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.setup	= xchk_setup_ag_refcountbt,
 		.scrub	= xchk_refcountbt,
 		.has	= xfs_has_reflink,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_refcountbt,
 	},
 	[XFS_SCRUB_TYPE_INODE] = {	/* inode record */
 		.type	= ST_INODE,
fs/xfs/scrub/trace.h +12 -10
index c60f76231f0c7..3f7af44309515 100644
@@ -1232,27 +1232,29 @@ TRACE_EVENT(xrep_ibt_found,
 		  __entry->freemask)
 )
 
-TRACE_EVENT(xrep_refcount_extent_fn,
-	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
-		 struct xfs_refcount_irec *irec),
-	TP_ARGS(mp, agno, irec),
+TRACE_EVENT(xrep_refc_found,
+	TP_PROTO(struct xfs_perag *pag, const struct xfs_refcount_irec *rec),
+	TP_ARGS(pag, rec),
 	TP_STRUCT__entry(
 		__field(dev_t, dev)
 		__field(xfs_agnumber_t, agno)
+		__field(enum xfs_refc_domain, domain)
 		__field(xfs_agblock_t, startblock)
 		__field(xfs_extlen_t, blockcount)
 		__field(xfs_nlink_t, refcount)
 	),
 	TP_fast_assign(
-		__entry->dev = mp->m_super->s_dev;
-		__entry->agno = agno;
-		__entry->startblock = irec->rc_startblock;
-		__entry->blockcount = irec->rc_blockcount;
-		__entry->refcount = irec->rc_refcount;
+		__entry->dev = pag->pag_mount->m_super->s_dev;
+		__entry->agno = pag->pag_agno;
+		__entry->domain = rec->rc_domain;
+		__entry->startblock = rec->rc_startblock;
+		__entry->blockcount = rec->rc_blockcount;
+		__entry->refcount = rec->rc_refcount;
 	),
-	TP_printk("dev %d:%d agno 0x%x agbno 0x%x fsbcount 0x%x refcount %u",
+	TP_printk("dev %d:%d agno 0x%x dom %s agbno 0x%x fsbcount 0x%x refcount %u",
 		  MAJOR(__entry->dev), MINOR(__entry->dev),
 		  __entry->agno,
+		  __print_symbolic(__entry->domain, XFS_REFC_DOMAIN_STRINGS),
 		  __entry->startblock,
 		  __entry->blockcount,
 		  __entry->refcount)
#69 lore raw

cover xfs: online repair of inodes and forks

In reply to Darrick J. Wong

Hi all,

In this series, online repair gains the ability to repair inode records. To do this, we must repair the ondisk inode and fork information enough to pass the iget verifiers and hence make the inode igettable again. Once that's done, we can perform higher level repairs on the incore inode. The fstests counterpart of this patchset implements stress testing of repair.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-inodes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-inodes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-inodes

---
 fs/xfs/Makefile                    |    1 
 fs/xfs/libxfs/xfs_attr_leaf.c      |   32 -
 fs/xfs/libxfs/xfs_attr_leaf.h      |    2 
 fs/xfs/libxfs/xfs_bmap.c           |   22 
 fs/xfs/libxfs/xfs_bmap.h           |    2 
 fs/xfs/libxfs/xfs_dir2_priv.h      |    2 
 fs/xfs/libxfs/xfs_dir2_sf.c        |   29 -
 fs/xfs/libxfs/xfs_format.h         |    3 
 fs/xfs/libxfs/xfs_shared.h         |    1 
 fs/xfs/libxfs/xfs_symlink_remote.c |   21 
 fs/xfs/scrub/bmap.c                |   48 +
 fs/xfs/scrub/common.c              |   26 +
 fs/xfs/scrub/common.h              |    8 
 fs/xfs/scrub/dir.c                 |   21 
 fs/xfs/scrub/inode.c               |   14 
 fs/xfs/scrub/inode_repair.c        | 1659 ++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/parent.c              |   17 
 fs/xfs/scrub/repair.c              |   57 +
 fs/xfs/scrub/repair.h              |   29 +
 fs/xfs/scrub/rtbitmap.c            |    4 
 fs/xfs/scrub/rtsummary.c           |    4 
 fs/xfs/scrub/scrub.c               |    2 
 fs/xfs/scrub/trace.h               |  174 ++++
 23 files changed, 2128 insertions(+), 50 deletions(-)
 create mode 100644 fs/xfs/scrub/inode_repair.c
#70 lore raw

PATCH 1/7 xfs: disable online repair quota helpers when quota not enabled

From: Darrick J. Wong <[email protected]>

Don't compile the quota helper functions if quota isn't being built into the XFS module.

---
 fs/xfs/scrub/repair.c |    2 ++
 fs/xfs/scrub/repair.h |    9 +++++++++
 2 files changed, 11 insertions(+)
fs/xfs/scrub/repair.c +2 -0
index ad1df212ec4c1..18f8d54948f26 100644
@@ -673,6 +673,7 @@ xrep_find_ag_btree_roots(
 	return error;
 }
 
+#ifdef CONFIG_XFS_QUOTA
 /* Force a quotacheck the next time we mount. */
 void
 xrep_force_quotacheck(
@@ -734,6 +735,7 @@ xrep_ino_dqattach(
 
 	return error;
 }
+#endif /* CONFIG_XFS_QUOTA */
 
 /* Initialize all the btree cursors for an AG repair. */
 void
fs/xfs/scrub/repair.h +9 -0
index cc7ea39427296..93814acc678a8 100644
@@ -57,8 +57,15 @@ struct xrep_find_ag_btree {
 
 int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,
 		struct xrep_find_ag_btree *btree_info, struct xfs_buf *agfl_bp);
+
+#ifdef CONFIG_XFS_QUOTA
 void xrep_force_quotacheck(struct xfs_scrub *sc, xfs_dqtype_t type);
 int xrep_ino_dqattach(struct xfs_scrub *sc);
+#else
+# define xrep_force_quotacheck(sc, type)	((void)0)
+# define xrep_ino_dqattach(sc)			(0)
+#endif /* CONFIG_XFS_QUOTA */
+
 int xrep_reset_perag_resv(struct xfs_scrub *sc);
 
 /* Repair setup functions */
@@ -87,6 +94,8 @@ int xrep_reinit_pagi(struct xfs_scrub *sc);
 
 #else
 
+#define xrep_ino_dqattach(sc)	(0)
+
 static inline int
 xrep_attempt(
 	struct xfs_scrub	*sc,
#72 lore raw

PATCH 2/7 xfs: try to attach dquots to files before repairing them

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Inode resource usage is tracked in the quota metadata. Repairing a file might change the resources used by that file, which means that we need to attach dquots to the file that we're examining before accessing anything in the file protected by the ILOCK.

However, there's a twist: a dquot cache miss requires the dquot to be read in from the quota file, during which we drop the ILOCK on the file being examined. This means that we *must* try to attach the dquots before taking the ILOCK.

Therefore, dquots must be attached to files in the scrub setup function. If doing so yields corruption errors (or unknown dquot errors), we instead clear the quotachecked status, which will cause a quotacheck on next mount. A future series will make this trigger live quotacheck.

While we're here, change the xrep_ino_dqattach function to use the unlocked dqattach functions so that we avoid cycling the ILOCK if the inode already has dquots attached. This makes the naming and locking requirements consistent with the rest of the filesystem.

---
 fs/xfs/scrub/bmap.c      |    4 ++++
 fs/xfs/scrub/common.c    |   25 +++++++++++++++++++++++++
 fs/xfs/scrub/common.h    |    6 ++++++
 fs/xfs/scrub/inode.c     |    4 ++++
 fs/xfs/scrub/repair.c    |   13 ++++++++-----
 fs/xfs/scrub/rtbitmap.c  |    4 ++++
 fs/xfs/scrub/rtsummary.c |    4 ++++
 7 files changed, 55 insertions(+), 5 deletions(-)
fs/xfs/scrub/bmap.c +4 -0
index 06d8c1996a338..f74bd2a97c7f7 100644
@@ -78,6 +78,10 @@ xchk_setup_inode_bmap(
 	if (error)
 		goto out;
 
+	error = xchk_ino_dqattach(sc);
+	if (error)
+		goto out;
+
 	xchk_ilock(sc, XFS_ILOCK_EXCL);
 out:
 	/* scrub teardown will unlock and release the inode */
fs/xfs/scrub/common.c +25 -0
index b6725b05fb417..9b7d7010495b9 100644
@@ -820,6 +820,26 @@ xchk_iget_agi(
 	return 0;
 }
 
+#ifdef CONFIG_XFS_QUOTA
+/*
+ * Try to attach dquots to this inode if we think we might want to repair it.
+ * Callers must not hold any ILOCKs.  If the dquots are broken and cannot be
+ * attached, a quotacheck will be scheduled.
+ */
+int
+xchk_ino_dqattach(
+	struct xfs_scrub	*sc)
+{
+	ASSERT(sc->tp != NULL);
+	ASSERT(sc->ip != NULL);
+
+	if (!xchk_could_repair(sc))
+		return 0;
+
+	return xrep_ino_dqattach(sc);
+}
+#endif
+
 /* Install an inode that we opened by handle for scrubbing. */
 int
 xchk_install_handle_inode(
@@ -1031,6 +1051,11 @@ xchk_setup_inode_contents(
 	error = xchk_trans_alloc(sc, resblks);
 	if (error)
 		goto out;
+
+	error = xchk_ino_dqattach(sc);
+	if (error)
+		goto out;
+
 	xchk_ilock(sc, XFS_ILOCK_EXCL);
 out:
 	/* scrub teardown will unlock and release the inode for us */
fs/xfs/scrub/common.h +6 -0
index 4b666f254d700..895918565df26 100644
@@ -103,9 +103,15 @@ xchk_setup_rtsummary(struct xfs_scrub *sc)
 }
 #endif
 #ifdef CONFIG_XFS_QUOTA
+int xchk_ino_dqattach(struct xfs_scrub *sc);
 int xchk_setup_quota(struct xfs_scrub *sc);
 #else
 static inline int
+xchk_ino_dqattach(struct xfs_scrub *sc)
+{
+	return 0;
+}
+static inline int
 xchk_setup_quota(struct xfs_scrub *sc)
 {
 	return -ENOENT;
fs/xfs/scrub/inode.c +4 -0
index b7a93380a1ab0..7e97db8255c63 100644
@@ -39,6 +39,10 @@ xchk_prepare_iscrub(
 	if (error)
 		return error;
 
+	error = xchk_ino_dqattach(sc);
+	if (error)
+		return error;
+
 	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	return 0;
 }
fs/xfs/scrub/repair.c +8 -5
index 18f8d54948f26..2e82dace10cc2 100644
@@ -700,10 +700,10 @@ xrep_force_quotacheck(
  *
  * This function ensures that the appropriate dquots are attached to an inode.
  * We cannot allow the dquot code to allocate an on-disk dquot block here
- * because we're already in transaction context with the inode locked.  The
- * on-disk dquot should already exist anyway.  If the quota code signals
- * corruption or missing quota information, schedule quotacheck, which will
- * repair corruptions in the quota metadata.
+ * because we're already in transaction context.  The on-disk dquot should
+ * already exist anyway.  If the quota code signals corruption or missing quota
+ * information, schedule quotacheck, which will repair corruptions in the quota
+ * metadata.
  */
 int
 xrep_ino_dqattach(
@@ -711,7 +711,10 @@ xrep_ino_dqattach(
 {
 	int			error;
 
-	error = xfs_qm_dqattach_locked(sc->ip, false);
+	ASSERT(sc->tp != NULL);
+	ASSERT(sc->ip != NULL);
+
+	error = xfs_qm_dqattach(sc->ip);
 	switch (error) {
 	case -EFSBADCRC:
 	case -EFSCORRUPTED:
fs/xfs/scrub/rtbitmap.c +4 -0
index 41a1d89ae8e6c..d509a08d3fc3e 100644
@@ -32,6 +32,10 @@ xchk_setup_rtbitmap(
 	if (error)
 		return error;
 
+	error = xchk_ino_dqattach(sc);
+	if (error)
+		return error;
+
 	xchk_ilock(sc, XFS_ILOCK_EXCL | XFS_ILOCK_RTBITMAP);
 	return 0;
 }
fs/xfs/scrub/rtsummary.c +4 -0
index 8b15c47408d03..f94800a029f35 100644
@@ -63,6 +63,10 @@ xchk_setup_rtsummary(
 	if (error)
 		return error;
 
+	error = xchk_ino_dqattach(sc);
+	if (error)
+		return error;
+
 	/*
 	 * Locking order requires us to take the rtbitmap first.  We must be
 	 * careful to unlock it ourselves when we are done with the rtbitmap
#74 lore raw

PATCH 3/7 xfs: repair inode records

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

If an inode is so badly damaged that it cannot be loaded into the cache, fix the ondisk metadata and try again. If there /is/ a cached inode, fix any problems and apply any optimizations that can be solved incore.

---
 fs/xfs/Makefile             |    1 
 fs/xfs/libxfs/xfs_format.h  |    3 
 fs/xfs/scrub/inode.c        |   10 -
 fs/xfs/scrub/inode_repair.c |  804 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.c       |   42 ++
 fs/xfs/scrub/repair.h       |   20 +
 fs/xfs/scrub/scrub.c        |    2 
 fs/xfs/scrub/trace.h        |  129 +++++++
 8 files changed, 1008 insertions(+), 3 deletions(-)
 create mode 100644 fs/xfs/scrub/inode_repair.c
fs/xfs/Makefile +1 -0
index a6f708dc56cc2..0d86d75422f60 100644
@@ -183,6 +183,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   alloc_repair.o \
 				   ialloc_repair.o \
+				   inode_repair.o \
 				   newbt.o \
 				   reap.o \
 				   refcount_repair.o \
fs/xfs/libxfs/xfs_format.h +2 -1
index 9a88aba1589f8..9dd3b21434314 100644
@@ -1012,7 +1012,8 @@ enum xfs_dinode_fmt {
 #define XFS_DFORK_APTR(dip)	\
 	(XFS_DFORK_DPTR(dip) + XFS_DFORK_BOFF(dip))
 #define XFS_DFORK_PTR(dip,w)	\
-	((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : XFS_DFORK_APTR(dip))
+	((void *)((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : \
+					 XFS_DFORK_APTR(dip)))
 
 #define XFS_DFORK_FORMAT(dip,w) \
 	((w) == XFS_DATA_FORK ? \
fs/xfs/scrub/inode.c +9 -1
index 7e97db8255c63..8656dd0d95560 100644
@@ -25,6 +25,7 @@
 #include "scrub/common.h"
 #include "scrub/btree.h"
 #include "scrub/trace.h"
+#include "scrub/repair.h"
 
 /* Prepare the attached inode for scrubbing. */
 static inline int
@@ -185,8 +186,11 @@ xchk_setup_inode(
 	 * saying the inode is allocated and the icache being unable to load
 	 * the inode until we can flag the corruption in xchk_inode.  The
 	 * scrub function has to note the corruption, since we're not really
-	 * supposed to do that from the setup function.
+	 * supposed to do that from the setup function.  Save the mapping to
+	 * make repairs to the ondisk inode buffer.
 	 */
+	if (xchk_could_repair(sc))
+		xrep_setup_inode(sc, &imap);
 	return 0;
 
 out_cancel:
@@ -342,6 +346,10 @@ xchk_inode_flags2(
 	if (xfs_dinode_has_bigtime(dip) && !xfs_has_bigtime(mp))
 		goto bad;
 
+	/* no large extent counts without the filesystem feature */
+	if ((flags2 & XFS_DIFLAG2_NREXT64) && !xfs_has_large_extent_counts(mp))
+		goto bad;
+
 	return;
 bad:
 	xchk_ino_set_corrupt(sc, ino);
fs/xfs/scrub/inode_repair.c new file +804 -0

804 lines not on this page · raw message

fs/xfs/scrub/repair.c +42 -0

54 lines not on this page · raw message

fs/xfs/scrub/repair.h +20 -0
index 93814acc678a8..70a6b18e5ad3c 100644
@@ -30,11 +30,22 @@ static inline int xrep_notsupported(struct xfs_scrub *sc)
 int xrep_attempt(struct xfs_scrub *sc, struct xchk_stats_run *run);
 void xrep_failure(struct xfs_mount *mp);
 int xrep_roll_ag_trans(struct xfs_scrub *sc);
+int xrep_roll_trans(struct xfs_scrub *sc);
 int xrep_defer_finish(struct xfs_scrub *sc);
 bool xrep_ag_has_space(struct xfs_perag *pag, xfs_extlen_t nr_blocks,
 		enum xfs_ag_resv_type type);
 xfs_extlen_t xrep_calc_ag_resblks(struct xfs_scrub *sc);
 
+static inline int
+xrep_trans_commit(
+	struct xfs_scrub	*sc)
+{
+	int error = xfs_trans_commit(sc->tp);
+
+	sc->tp = NULL;
+	return error;
+}
+
 struct xbitmap;
 struct xagb_bitmap;
 
@@ -66,11 +77,16 @@ int xrep_ino_dqattach(struct xfs_scrub *sc);
 # define xrep_ino_dqattach(sc)			(0)
 #endif /* CONFIG_XFS_QUOTA */
 
+int xrep_ino_ensure_extent_count(struct xfs_scrub *sc, int whichfork,
+		xfs_extnum_t nextents);
 int xrep_reset_perag_resv(struct xfs_scrub *sc);
 
 /* Repair setup functions */
 int xrep_setup_ag_allocbt(struct xfs_scrub *sc);
 
+struct xfs_imap;
+int xrep_setup_inode(struct xfs_scrub *sc, struct xfs_imap *imap);
+
 void xrep_ag_btcur_init(struct xfs_scrub *sc, struct xchk_ag *sa);
 
 /* Metadata revalidators */
@@ -88,6 +104,7 @@ int xrep_agi(struct xfs_scrub *sc);
 int xrep_allocbt(struct xfs_scrub *sc);
 int xrep_iallocbt(struct xfs_scrub *sc);
 int xrep_refcountbt(struct xfs_scrub *sc);
+int xrep_inode(struct xfs_scrub *sc);
 
 int xrep_reinit_pagf(struct xfs_scrub *sc);
 int xrep_reinit_pagi(struct xfs_scrub *sc);
@@ -133,6 +150,8 @@ xrep_setup_nothing(
 }
 #define xrep_setup_ag_allocbt		xrep_setup_nothing
 
+#define xrep_setup_inode(sc, imap)	((void)0)
+
 #define xrep_revalidate_allocbt		(NULL)
 #define xrep_revalidate_iallocbt	(NULL)
 
@@ -144,6 +163,7 @@ xrep_setup_nothing(
 #define xrep_allocbt			xrep_notsupported
 #define xrep_iallocbt			xrep_notsupported
 #define xrep_refcountbt			xrep_notsupported
+#define xrep_inode			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +1 -1
index d0d6b2b41219e..b9edda17ab64b 100644
@@ -284,7 +284,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_INODE,
 		.setup	= xchk_setup_inode,
 		.scrub	= xchk_inode,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_inode,
 	},
 	[XFS_SCRUB_TYPE_BMBTD] = {	/* inode data fork */
 		.type	= ST_INODE,
fs/xfs/scrub/trace.h +129 -0

135 lines not on this page · raw message

#75 lore raw

Re: [PATCH 3/7] xfs: repair inode records

fs/xfs/libxfs/xfs_format.h enum xfs_dinode_fmt { · line 1016

Quoted 7 lines
fs/xfs/libxfs/xfs_format.h +2 -1
@@ -1012,7 +1012,8 @@ enum xfs_dinode_fmt {
 #define XFS_DFORK_APTR(dip)	\
 	(XFS_DFORK_DPTR(dip) + XFS_DFORK_BOFF(dip))
 #define XFS_DFORK_PTR(dip,w)	\
-	((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : XFS_DFORK_APTR(dip))
+	((void *)((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : \
+					 XFS_DFORK_APTR(dip)))

Not requiring a cast when using XFS_DFORK_PTR is a good thing, but I think this is the wrong way to do it. Instead of adding another cast here we can just change the char * cast in XFS_DFORK_DPTR to a void * one and rely on the widely used void pointer arithmetics extension in gcc (and clang). That'll also need a fixup to use a void instead of char * cast in xchk_dinode.

And in the long run many of these helpers relly should become inline functions..

fs/xfs/scrub/inode.c xchk_inode_flags2( · line 351

Quoted 3 lines
fs/xfs/scrub/inode.c +3 -0
+	/* no large extent counts without the filesystem feature */
+	if ((flags2 & XFS_DIFLAG2_NREXT64) && !xfs_has_large_extent_counts(mp))
+		goto bad;

This is just a missing check and not really related to repair, is it?

fs/xfs/scrub/inode_repair.c line 103

Quoted 9 lines
fs/xfs/scrub/inode_repair.c new file +9 -0
+	/*
+	 * The only information that needs to be passed between inode scrub and
+	 * repair is the location of the ondisk metadata if iget fails.  The
+	 * rest of struct xrep_inode is context data that we need to massage
+	 * the ondisk inode to the point that iget will work, which means that
+	 * we don't allocate anything at all if the incore inode is loaded.
+	 */
+	if (!imap)
+		return 0;

I don't really understand why this comment is here, and how it relates to the imap NULL check. But as the only caller passes the address of an on-stack imap I also don't understand why the check is here to start with.

fs/xfs/scrub/inode_repair.c line 142

Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6 -0
+	for (i = 0; i < ni; i++) {
+		ioff = i << mp->m_sb.sb_inodelog;
+		dip = xfs_buf_offset(bp, ioff);
+		agino = be32_to_cpu(dip->di_next_unlinked);
+
+		unlinked_ok = magic_ok = crc_ok = false;

I'd split the body of this loop into a separate helper and keep a lot of the variables local to it.

fs/xfs/scrub/inode_repair.c line 181

Quoted 11 lines
fs/xfs/scrub/inode_repair.c new file +11 -0
+/* Reinitialize things that never change in an inode. */
+STATIC void
+xrep_dinode_header(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
+	trace_xrep_dinode_header(sc, dip);
+
+	dip->di_magic = cpu_to_be16(XFS_DINODE_MAGIC);
+	if (!xfs_dinode_good_version(sc->mp, dip->di_version))
+		dip->di_version = 3;

Can we ever end up here for v4 file systems? Because in that case the sane default inode version would be 2.

fs/xfs/scrub/inode_repair.c line 198

Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13 -0
+
+/* Turn di_mode into /something/ recognizable. */
+STATIC void
+xrep_dinode_mode(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
+	uint16_t		mode;
+
+	trace_xrep_dinode_mode(sc, dip);
+
+	mode = be16_to_cpu(dip->di_mode);
+	if (mode == 0 || xfs_mode_to_ftype(mode) != XFS_DIR3_FT_UNKNOWN)

This is a somewhat odd way to check for a valid mode, but it works, so..

fs/xfs/scrub/inode_repair.c line 226

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	if (xfs_has_reflink(mp) && S_ISREG(mode))
+		flags2 |= XFS_DIFLAG2_REFLINK;

We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.

fs/xfs/scrub/inode_repair.c line 259

Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13 -0
+STATIC void
+xrep_dinode_zap_symlink(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
+	char			*p;
+
+	trace_xrep_dinode_zap_symlink(sc, dip);
+
+	dip->di_format = XFS_DINODE_FMT_LOCAL;
+	dip->di_size = cpu_to_be64(1);
+	p = XFS_DFORK_PTR(dip, XFS_DATA_FORK);
+	*p = '.';

Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.

fs/xfs/scrub/inode_repair.c line 267

Quoted 8 lines
fs/xfs/scrub/inode_repair.c new file +8 -0
+}
+
+/*
+ * Blow out dir, make it point to the root.  In the future repair will
+ * reconstruct this directory for us.  Note that there's no in-core directory
+ * inode because the sf verifier tripped, so we don't have to worry about the
+ * dentry cache.
+ */

"make it point to root" isn't what I read in the code below. I parents it in root I think.

fs/xfs/scrub/inode_repair.c line 300

Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13 -0
+/* Make sure we don't have a garbage file size. */
+STATIC void
+xrep_dinode_size(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
+	uint64_t		size;
+	uint16_t		mode;
+
+	trace_xrep_dinode_size(sc, dip);
+
+	mode = be16_to_cpu(dip->di_mode);
+	size = be64_to_cpu(dip->di_size);

Any reason to not simplify initialize the variables at declaration time? (Same for a while bunch of other functions / variables)

fs/xfs/scrub/inode_repair.c line 490

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	if (xfs_has_reflink(sc->mp)) {
+		; /* data fork blockcount can exceed physical storage */

... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?

Still, shouldn't we limit the condition to xfs_is_reflink_inode?

fs/xfs/scrub/inode_repair.c line 534

Quoted 10 lines
fs/xfs/scrub/inode_repair.c new file +10 -0
+/* Check for invalid uid/gid/prid. */
+STATIC void
+xrep_inode_ids(
+	struct xfs_scrub	*sc)
+{
+	bool			dirty = false;
+
+	trace_xrep_inode_ids(sc);
+
+	if (i_uid_read(VFS_I(sc->ip)) == -1U) {

What is invalid about all-F uid/gid/projid?

fs/xfs/scrub/inode_repair.c line 579

Quoted 3 lines
fs/xfs/scrub/inode_repair.c new file +3 -0
+	tstamp = inode_get_atime(inode);
+	xrep_clamp_timestamp(ip, &tstamp);
+	inode_set_atime_to_ts(inode, tstamp);

Meh, I hate these new VFS timestamp access helper..

fs/xfs/scrub/inode_repair.c line 669

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	/* Find the last block before 32G; this is the dir size. */
+	error = xfs_iread_extents(sc->tp, sc->ip, XFS_DATA_FORK);

I think that comments needs to go down to the off asignment and xfs_iext_lookup_extent_before call.

Quoted 7 lines
fs/xfs/scrub/inode_repair.c new file +7 -0
+/*
+ * Fix any irregularities in an inode's size now that we can iterate extent
+ * maps and access other regular inode data.
+ */
+STATIC void
+xrep_inode_size(
+	struct xfs_scrub	*sc)

fs/xfs/scrub/inode_repair.c line 714

Quoted 10 lines
fs/xfs/scrub/inode_repair.c new file +10 -0
+{
+	trace_xrep_inode_size(sc);
+
+	/*
+	 * Currently we only support fixing size on extents or btree format
+	 * directories.  Files can be any size and sizes for the other inode
+	 * special types are fixed by xrep_dinode_size.
+	 */
+	if (!S_ISDIR(VFS_I(sc->ip)->i_mode))
+		return;

I think moving this check to the caller and renaming the function would be a bit nicer, especially if we grow more file type specific checks in the future.

Otherwise this looks reasonable to me.

#76 lore raw

Re: [PATCH 3/7] xfs: repair inode records

Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
fs/xfs/libxfs/xfs_format.h +0 -1
@@ -1012,7 +1012,8 @@ enum xfs_dinode_fmt {
 #define XFS_DFORK_APTR(dip)	\
 	(XFS_DFORK_DPTR(dip) + XFS_DFORK_BOFF(dip))
 #define XFS_DFORK_PTR(dip,w)	\
-	((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : XFS_DFORK_APTR(dip))
Quoted 8 lines

fs/xfs/libxfs/xfs_format.h enum xfs_dinode_fmt { · line 1016

Quoted 2 lines
fs/xfs/libxfs/xfs_format.h +2 -0
+	((void *)((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : \
+					 XFS_DFORK_APTR(dip)))

Not requiring a cast when using XFS_DFORK_PTR is a good thing, but I think this is the wrong way to do it. Instead of adding another cast here we can just change the char * cast in XFS_DFORK_DPTR to a void * one and rely on the widely used void pointer arithmetics extension in gcc (and clang).

Ok.

Quoted 2 lines

That'll also need a fixup to use a void instead of char * cast in xchk_dinode.

I'll change the conditional to:

	if (XFS_DFORK_BOFF(dip) >= mp->m_sb.sb_inodesize)
Quoted 8 lines

And in the long run many of these helpers relly should become inline functions..

fs/xfs/scrub/inode.c xchk_inode_flags2( · line 351

Quoted 3 lines
fs/xfs/scrub/inode.c +3 -0
+	/* no large extent counts without the filesystem feature */
+	if ((flags2 & XFS_DIFLAG2_NREXT64) && !xfs_has_large_extent_counts(mp))
+		goto bad;

This is just a missing check and not really related to repair, is it?

Yep. I guess I'll pull that out into a separate patch.

Quoted 6 lines
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6 -0
+	/*
+	 * The only information that needs to be passed between inode scrub and
+	 * repair is the location of the ondisk metadata if iget fails.  The
+	 * rest of struct xrep_inode is context data that we need to massage
+	 * the ondisk inode to the point that iget will work, which means that
+	 * we don't allocate anything at all if the incore inode is loaded.
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 103

Quoted 3 lines
fs/xfs/scrub/inode_repair.c new file +3 -0
+	 */
+	if (!imap)
+		return 0;

I don't really understand why this comment is here, and how it relates to the imap NULL check. But as the only caller passes the address of an on-stack imap I also don't understand why the check is here to start with.

Hmm. I think I've been through too many iterations of this code -- at one point I remember the null check was actually useful for something. But now it's not, so it can go.

Quoted 10 lines

fs/xfs/scrub/inode_repair.c line 142

Quoted 7 lines
fs/xfs/scrub/inode_repair.c new file +6 -0
+	for (i = 0; i < ni; i++) {
+		ioff = i << mp->m_sb.sb_inodelog;
+		dip = xfs_buf_offset(bp, ioff);
+		agino = be32_to_cpu(dip->di_next_unlinked);
+
+		unlinked_ok = magic_ok = crc_ok = false;

I'd split the body of this loop into a separate helper and keep a lot of the variables local to it.

Ok.

Quoted 6 lines
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6 -0
+/* Reinitialize things that never change in an inode. */
+STATIC void
+xrep_dinode_header(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 181

Quoted 5 lines
fs/xfs/scrub/inode_repair.c new file +5 -0
+	trace_xrep_dinode_header(sc, dip);
+
+	dip->di_magic = cpu_to_be16(XFS_DINODE_MAGIC);
+	if (!xfs_dinode_good_version(sc->mp, dip->di_version))
+		dip->di_version = 3;

Can we ever end up here for v4 file systems? Because in that case the sane default inode version would be 2.

No. xchk_validate_inputs will reject IFLAG_REPAIR on a V4 fs. Those are deprecated, there's no point in going back.

Quoted 7 lines
Quoted 7 lines
fs/xfs/scrub/inode_repair.c new file +7 -0
+
+/* Turn di_mode into /something/ recognizable. */
+STATIC void
+xrep_dinode_mode(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 198

Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6 -0
+	uint16_t		mode;
+
+	trace_xrep_dinode_mode(sc, dip);
+
+	mode = be16_to_cpu(dip->di_mode);
+	if (mode == 0 || xfs_mode_to_ftype(mode) != XFS_DIR3_FT_UNKNOWN)

This is a somewhat odd way to check for a valid mode, but it works, so..

:)

Quoted 6 lines

fs/xfs/scrub/inode_repair.c line 226

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	if (xfs_has_reflink(mp) && S_ISREG(mode))
+		flags2 |= XFS_DIFLAG2_REFLINK;

We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.

/*
* For regular files on a reflink filesystem, set the REFLINK flag to
* protect shared extents. A later stage will actually check those
* extents and clear the flag if possible.
*/

Quoted 10 lines
Quoted 10 lines
fs/xfs/scrub/inode_repair.c new file +9 -0
+STATIC void
+xrep_dinode_zap_symlink(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
+	char			*p;
+
+	trace_xrep_dinode_zap_symlink(sc, dip);
+
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 259

Quoted 4 lines
fs/xfs/scrub/inode_repair.c new file +4 -0
+	dip->di_format = XFS_DINODE_FMT_LOCAL;
+	dip->di_size = cpu_to_be64(1);
+	p = XFS_DFORK_PTR(dip, XFS_DATA_FORK);
+	*p = '.';

Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.

How about '🤷'? That's only four bytes.

Or maybe a question mark.

Quoted 11 lines

fs/xfs/scrub/inode_repair.c line 267

Quoted 8 lines
fs/xfs/scrub/inode_repair.c new file +8 -0
+}
+
+/*
+ * Blow out dir, make it point to the root.  In the future repair will
+ * reconstruct this directory for us.  Note that there's no in-core directory
+ * inode because the sf verifier tripped, so we don't have to worry about the
+ * dentry cache.
+ */

"make it point to root" isn't what I read in the code below. I parents it in root I think.

Yes. Changed to "Blow out dir, make the parent point to the root."

Quoted 8 lines
Quoted 8 lines
fs/xfs/scrub/inode_repair.c new file +8 -0
+/* Make sure we don't have a garbage file size. */
+STATIC void
+xrep_dinode_size(
+	struct xfs_scrub	*sc,
+	struct xfs_dinode	*dip)
+{
+	uint64_t		size;
+	uint16_t		mode;
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 300

Quoted 5 lines
fs/xfs/scrub/inode_repair.c new file +5 -0
+
+	trace_xrep_dinode_size(sc, dip);
+
+	mode = be16_to_cpu(dip->di_mode);
+	size = be64_to_cpu(dip->di_size);

Any reason to not simplify initialize the variables at declaration time? (Same for a while bunch of other functions / variables)

No, not really. Will fix.

Quoted 5 lines

fs/xfs/scrub/inode_repair.c line 490

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	if (xfs_has_reflink(sc->mp)) {
+		; /* data fork blockcount can exceed physical storage */

... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?

Yes. That's not a terribly functional file, but users can do such things if they want to pay for the cpu/metadata.

Quoted 1 line

Still, shouldn't we limit the condition to xfs_is_reflink_inode?

Yep.

Quoted 4 lines
Quoted 4 lines
fs/xfs/scrub/inode_repair.c new file +4 -0
+/* Check for invalid uid/gid/prid. */
+STATIC void
+xrep_inode_ids(
+	struct xfs_scrub	*sc)
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 534

Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6 -0
+{
+	bool			dirty = false;
+
+	trace_xrep_inode_ids(sc);
+
+	if (i_uid_read(VFS_I(sc->ip)) == -1U) {

What is invalid about all-F uid/gid/projid?

I thought those were invalid, though apparently they're not now? uidgid.h says:

static inline bool uid_valid(kuid_t uid)

{
	return __kuid_val(uid) != (uid_t) -1;
}

Which is why I thought that it's not possible to have a uid of -1 on a file. Trying to set that uid on a file causes the kernel to reject the value, but OTOH I can apparently create inodes with a -1 UID via idmapping shenanigans.

<shrug>

Quoted 5 lines

fs/xfs/scrub/inode_repair.c line 579

Quoted 3 lines
fs/xfs/scrub/inode_repair.c new file +3 -0
+	tstamp = inode_get_atime(inode);
+	xrep_clamp_timestamp(ip, &tstamp);
+	inode_set_atime_to_ts(inode, tstamp);

Meh, I hate these new VFS timestamp access helper..

They're very clunky.

Quoted 5 lines

fs/xfs/scrub/inode_repair.c line 669

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	/* Find the last block before 32G; this is the dir size. */
+	error = xfs_iread_extents(sc->tp, sc->ip, XFS_DATA_FORK);

I think that comments needs to go down to the off asignment and xfs_iext_lookup_extent_before call.

Done.

Quoted 13 lines
Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13 -0
+/*
+ * Fix any irregularities in an inode's size now that we can iterate extent
+ * maps and access other regular inode data.
+ */
+STATIC void
+xrep_inode_size(
+	struct xfs_scrub	*sc)
+{
+	trace_xrep_inode_size(sc);
+
+	/*
+	 * Currently we only support fixing size on extents or btree format
+	 * directories.  Files can be any size and sizes for the other inode
Quoted 8 lines

fs/xfs/scrub/inode_repair.c line 714

Quoted 4 lines
fs/xfs/scrub/inode_repair.c new file +4 -0
+	 * special types are fixed by xrep_dinode_size.
+	 */
+	if (!S_ISDIR(VFS_I(sc->ip)->i_mode))
+		return;

I think moving this check to the caller and renaming the function would be a bit nicer, especially if we grow more file type specific checks in the future.

That's the only one, so I'll rename it to xrep_inode_dir_size and hoist this check to the caller.

Quoted 1 line

Otherwise this looks reasonable to me.

Woo, thanks for reading through all this. :)

--D

#77 lore raw

Re: [PATCH 3/7] xfs: repair inode records

Quoted 10 lines · Darrick J. Wong
Quoted 4 lines

We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.

/*
* For regular files on a reflink filesystem, set the REFLINK flag to
* protect shared extents. A later stage will actually check those
* extents and clear the flag if possible.
*/

Sounds good.

Quoted 7 lines
Quoted 3 lines

Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.

How about '🤷'? That's only four bytes.

Or maybe a question mark.

Heh. I guess question marks seems a bit better, but the general idea of having new names / different pointing locations show up in the name space scare me a little. I wonder if something like the classic lost+found directory might be the right thing for anything we're not sure about as people know about it.

Quoted 8 lines
Quoted 5 lines

fs/xfs/scrub/inode_repair.c line 490

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	if (xfs_has_reflink(sc->mp)) {
+		; /* data fork blockcount can exceed physical storage */

... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?

Yes. That's not a terribly functional file, but users can do such things if they want to pay for the cpu/metadata.

Yeah. But maybe expand the comment a bit - having spent a fair amout of time with the reflink code this was obvious to me, but for someone new the above might be a bit too cryptic.

Quoted 9 lines
Quoted 4 lines

fs/xfs/scrub/inode_repair.c line 534

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+
+	if (i_uid_read(VFS_I(sc->ip)) == -1U) {

What is invalid about all-F uid/gid/projid?

I thought those were invalid, though apparently they're not now? uidgid.h says:

static inline bool uid_valid(kuid_t uid)

Quoted 8 lines
{
	return __kuid_val(uid) != (uid_t) -1;
}

Which is why I thought that it's not possible to have a uid of -1 on a file. Trying to set that uid on a file causes the kernel to reject the value, but OTOH I can apparently create inodes with a -1 UID via idmapping shenanigans.

Heh. Just wanted an explanation for the check. So a commnt is fine, or finding a way to use the uid_valid and co helpers to make it self-explanatory.

#78 lore raw

Re: [PATCH 3/7] xfs: repair inode records

Quoted 20 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 4 lines

We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.

/*
* For regular files on a reflink filesystem, set the REFLINK flag to
* protect shared extents. A later stage will actually check those
* extents and clear the flag if possible.
*/

Sounds good.

Quoted 6 lines
Quoted 3 lines

Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.

How about '🤷'? That's only four bytes.

Quoted 7 lines
Quoted 1 line

Or maybe a question mark.

Heh. I guess question marks seems a bit better, but the general idea of having new names / different pointing locations show up in the name space scare me a little. I wonder if something like the classic lost+found directory might be the right thing for anything we're not sure about as people know about it.

Hmm. I suppose a problem with "?" is that question-mark is a valid filename, which means that our zapped symlink could now suddenly point to a different file that a user created. "/lost+found" isn't different in that respect, but societal convention might at least provide for raised eyebrows. That said, mkfs.xfs doesn't create one for us like mke2fs does, so maybe a broken symlink to the orphanage is... well, now I'm bikeshedding my own creation.

May I try to make a case for "🚽"? ;)

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

fs/xfs/scrub/inode_repair.c line 490

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+	if (xfs_has_reflink(sc->mp)) {
+		; /* data fork blockcount can exceed physical storage */

... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?

Quoted 6 lines
Quoted 2 lines

Yes. That's not a terribly functional file, but users can do such things if they want to pay for the cpu/metadata.

Yeah. But maybe expand the comment a bit - having spent a fair amout of time with the reflink code this was obvious to me, but for someone new the above might be a bit too cryptic.

	/*
	 * data fork blockcount can exceed physical storage if a
	 * user reflinks the same block over and over again.
	 */
Quoted 13 lines
Quoted 13 lines
Quoted 4 lines

fs/xfs/scrub/inode_repair.c line 534

Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2 -0
+
+	if (i_uid_read(VFS_I(sc->ip)) == -1U) {

What is invalid about all-F uid/gid/projid?

I thought those were invalid, though apparently they're not now? uidgid.h says:

static inline bool uid_valid(kuid_t uid)

{
	return __kuid_val(uid) != (uid_t) -1;
}
Quoted 8 lines
Quoted 4 lines

Which is why I thought that it's not possible to have a uid of -1 on a file. Trying to set that uid on a file causes the kernel to reject the value, but OTOH I can apparently create inodes with a -1 UID via idmapping shenanigans.

Heh. Just wanted an explanation for the check. So a commnt is fine, or finding a way to use the uid_valid and co helpers to make it self-explanatory.

Yeah, I think this converts easily to:

	if (!uid_valid(VFS_I(sc->ip)->i_uid)) {
		/* zap it */
	}

--D

#79 lore raw

Re: [PATCH 3/7] xfs: repair inode records

Quoted 9 lines · Darrick J. Wong

Hmm. I suppose a problem with "?" is that question-mark is a valid filename, which means that our zapped symlink could now suddenly point to a different file that a user created. "/lost+found" isn't different in that respect, but societal convention might at least provide for raised eyebrows. That said, mkfs.xfs doesn't create one for us like mke2fs does, so maybe a broken symlink to the orphanage is... well, now I'm bikeshedding my own creation.

May I try to make a case for "🚽"? ;)

Haha.. I suspect not allowing to follow the link at all if is marked sick is the best idea, i.e. the concept we've talked about for regular files. Make that consistent for all file times, and then we need to look into an expedited on-disk flag for that to make it persistent.

Quoted 5 lines
	/*
	 * data fork blockcount can exceed physical storage if a
	 * user reflinks the same block over and over again.
	 */

Yup.

Quoted 3 lines
	if (!uid_valid(VFS_I(sc->ip)->i_uid)) {
		/* zap it */
	}

Perfect.

#80 lore raw

PATCH 4/7 xfs: zap broken inode forks

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Determine if inode fork damage is responsible for the inode being unable to pass the ifork verifiers in xfs_iget and zap the fork contents if this is true. Once this is done the fork will be empty but we'll be able to construct an in-core inode, and a subsequent call to the inode fork repair ioctl will search the rmapbt to rebuild the records that were in the fork.

---
 fs/xfs/libxfs/xfs_attr_leaf.c      |   32 +-
 fs/xfs/libxfs/xfs_attr_leaf.h      |    2 
 fs/xfs/libxfs/xfs_bmap.c           |   22 +
 fs/xfs/libxfs/xfs_bmap.h           |    2 
 fs/xfs/libxfs/xfs_dir2_priv.h      |    2 
 fs/xfs/libxfs/xfs_dir2_sf.c        |   29 +-
 fs/xfs/libxfs/xfs_shared.h         |    1 
 fs/xfs/libxfs/xfs_symlink_remote.c |   21 +
 fs/xfs/scrub/inode_repair.c        |  696 ++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/trace.h               |   42 ++
 10 files changed, 812 insertions(+), 37 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +21 -11
index 2580ae47209a6..24d266c98bc97 100644
@@ -1040,23 +1040,16 @@ xfs_attr_shortform_allfit(
 	return xfs_attr_shortform_bytesfit(dp, bytes);
 }
 
-/* Verify the consistency of an inline attribute fork. */
+/* Verify the consistency of a raw inline attribute fork. */
 xfs_failaddr_t
-xfs_attr_shortform_verify(
-	struct xfs_inode		*ip)
+xfs_attr_shortform_verify_struct(
+	struct xfs_attr_shortform	*sfp,
+	size_t				size)
 {
-	struct xfs_attr_shortform	*sfp;
 	struct xfs_attr_sf_entry	*sfep;
 	struct xfs_attr_sf_entry	*next_sfep;
 	char				*endp;
-	struct xfs_ifork		*ifp;
 	int				i;
-	int64_t				size;
-
-	ASSERT(ip->i_af.if_format == XFS_DINODE_FMT_LOCAL);
-	ifp = xfs_ifork_ptr(ip, XFS_ATTR_FORK);
-	sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data;
-	size = ifp->if_bytes;
 
 	/*
 	 * Give up if the attribute is way too short.
@@ -1116,6 +1109,23 @@ xfs_attr_shortform_verify(
 	return NULL;
 }
 
+/* Verify the consistency of an inline attribute fork. */
+xfs_failaddr_t
+xfs_attr_shortform_verify(
+	struct xfs_inode		*ip)
+{
+	struct xfs_attr_shortform	*sfp;
+	struct xfs_ifork		*ifp;
+	int64_t				size;
+
+	ASSERT(ip->i_af.if_format == XFS_DINODE_FMT_LOCAL);
+	ifp = xfs_ifork_ptr(ip, XFS_ATTR_FORK);
+	sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data;
+	size = ifp->if_bytes;
+
+	return xfs_attr_shortform_verify_struct(sfp, size);
+}
+
 /*
  * Convert a leaf attribute list to shortform attribute list
  */
fs/xfs/libxfs/xfs_attr_leaf.h +2 -0
index 368f4d9fa1d59..0711a448f64ce 100644
@@ -56,6 +56,8 @@ int	xfs_attr_sf_findname(struct xfs_da_args *args,
 			     unsigned int *basep);
 int	xfs_attr_shortform_allfit(struct xfs_buf *bp, struct xfs_inode *dp);
 int	xfs_attr_shortform_bytesfit(struct xfs_inode *dp, int bytes);
+xfs_failaddr_t xfs_attr_shortform_verify_struct(struct xfs_attr_shortform *sfp,
+		size_t size);
 xfs_failaddr_t xfs_attr_shortform_verify(struct xfs_inode *ip);
 void	xfs_attr_fork_remove(struct xfs_inode *ip, struct xfs_trans *tp);
 
fs/xfs/libxfs/xfs_bmap.c +16 -6
index 68be1dd4f0f26..9968a3a6e6d8d 100644
@@ -6179,19 +6179,18 @@ xfs_bmap_finish_one(
 	return error;
 }
 
-/* Check that an inode's extent does not have invalid flags or bad ranges. */
+/* Check that an extent does not have invalid flags or bad ranges. */
 xfs_failaddr_t
-xfs_bmap_validate_extent(
-	struct xfs_inode	*ip,
+xfs_bmap_validate_extent_raw(
+	struct xfs_mount	*mp,
+	bool			rtfile,
 	int			whichfork,
 	struct xfs_bmbt_irec	*irec)
 {
-	struct xfs_mount	*mp = ip->i_mount;
-
 	if (!xfs_verify_fileext(mp, irec->br_startoff, irec->br_blockcount))
 		return __this_address;
 
-	if (XFS_IS_REALTIME_INODE(ip) && whichfork == XFS_DATA_FORK) {
+	if (rtfile && whichfork == XFS_DATA_FORK) {
 		if (!xfs_verify_rtbext(mp, irec->br_startblock,
 					   irec->br_blockcount))
 			return __this_address;
@@ -6221,3 +6220,14 @@ xfs_bmap_intent_destroy_cache(void)
 	kmem_cache_destroy(xfs_bmap_intent_cache);
 	xfs_bmap_intent_cache = NULL;
 }
+
+/* Check that an inode's extent does not have invalid flags or bad ranges. */
+xfs_failaddr_t
+xfs_bmap_validate_extent(
+	struct xfs_inode	*ip,
+	int			whichfork,
+	struct xfs_bmbt_irec	*irec)
+{
+	return xfs_bmap_validate_extent_raw(ip->i_mount,
+			XFS_IS_REALTIME_INODE(ip), whichfork, irec);
+}
fs/xfs/libxfs/xfs_bmap.h +2 -0
index e33470e39728d..8518324db2855 100644
@@ -263,6 +263,8 @@ static inline uint32_t xfs_bmap_fork_to_state(int whichfork)
 	}
 }
 
+xfs_failaddr_t xfs_bmap_validate_extent_raw(struct xfs_mount *mp, bool rtfile,
+		int whichfork, struct xfs_bmbt_irec *irec);
 xfs_failaddr_t xfs_bmap_validate_extent(struct xfs_inode *ip, int whichfork,
 		struct xfs_bmbt_irec *irec);
 int xfs_bmap_complain_bad_rec(struct xfs_inode *ip, int whichfork,
fs/xfs/libxfs/xfs_dir2_priv.h +2 -0
index 7404a9ff1a929..b10859a43776d 100644
@@ -175,6 +175,8 @@ extern int xfs_dir2_sf_create(struct xfs_da_args *args, xfs_ino_t pino);
 extern int xfs_dir2_sf_lookup(struct xfs_da_args *args);
 extern int xfs_dir2_sf_removename(struct xfs_da_args *args);
 extern int xfs_dir2_sf_replace(struct xfs_da_args *args);
+extern xfs_failaddr_t xfs_dir2_sf_verify_struct(struct xfs_mount *mp,
+		struct xfs_dir2_sf_hdr *sfp, int64_t size);
 extern xfs_failaddr_t xfs_dir2_sf_verify(struct xfs_inode *ip);
 int xfs_dir2_sf_entsize(struct xfs_mount *mp,
 		struct xfs_dir2_sf_hdr *hdr, int len);
fs/xfs/libxfs/xfs_dir2_sf.c +18 -11
index 8cd37e6e9d387..0089046585247 100644
@@ -706,12 +706,11 @@ xfs_dir2_sf_check(
 
 /* Verify the consistency of an inline directory. */
 xfs_failaddr_t
-xfs_dir2_sf_verify(
-	struct xfs_inode		*ip)
+xfs_dir2_sf_verify_struct(
+	struct xfs_mount		*mp,
+	struct xfs_dir2_sf_hdr		*sfp,
+	int64_t				size)
 {
-	struct xfs_mount		*mp = ip->i_mount;
-	struct xfs_ifork		*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
-	struct xfs_dir2_sf_hdr		*sfp;
 	struct xfs_dir2_sf_entry	*sfep;
 	struct xfs_dir2_sf_entry	*next_sfep;
 	char				*endp;
@@ -719,15 +718,9 @@ xfs_dir2_sf_verify(
 	int				i;
 	int				i8count;
 	int				offset;
-	int64_t				size;
 	int				error;
 	uint8_t				filetype;
 
-	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
-
-	sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data;
-	size = ifp->if_bytes;
-
 	/*
 	 * Give up if the directory is way too short.
 	 */
@@ -803,6 +796,20 @@ xfs_dir2_sf_verify(
 	return NULL;
 }
 
+xfs_failaddr_t
+xfs_dir2_sf_verify(
+	struct xfs_inode		*ip)
+{
+	struct xfs_mount		*mp = ip->i_mount;
+	struct xfs_ifork		*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
+	struct xfs_dir2_sf_hdr		*sfp;
+
+	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
+
+	sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data;
+	return xfs_dir2_sf_verify_struct(mp, sfp, ifp->if_bytes);
+}
+
 /*
  * Create a new (shortform) directory.
  */
fs/xfs/libxfs/xfs_shared.h +1 -0
index c4381388c0c1a..57a52fa76a496 100644
@@ -139,6 +139,7 @@ bool xfs_symlink_hdr_ok(xfs_ino_t ino, uint32_t offset,
 			uint32_t size, struct xfs_buf *bp);
 void xfs_symlink_local_to_remote(struct xfs_trans *tp, struct xfs_buf *bp,
 				 struct xfs_inode *ip, struct xfs_ifork *ifp);
+xfs_failaddr_t xfs_symlink_sf_verify_struct(void *sfp, int64_t size);
 xfs_failaddr_t xfs_symlink_shortform_verify(struct xfs_inode *ip);
 
 /* Computed inode geometry for the filesystem. */
fs/xfs/libxfs/xfs_symlink_remote.c +14 -7
index bdc777b9ec4a6..7660a95b1ea97 100644
@@ -201,16 +201,12 @@ xfs_symlink_local_to_remote(
  * does not do on-disk format checks.
  */
 xfs_failaddr_t
-xfs_symlink_shortform_verify(
-	struct xfs_inode	*ip)
+xfs_symlink_sf_verify_struct(
+	void			*sfp,
+	int64_t			size)
 {
-	struct xfs_ifork	*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
-	char			*sfp = (char *)ifp->if_u1.if_data;
-	int			size = ifp->if_bytes;
 	char			*endp = sfp + size;
 
-	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
-
 	/*
 	 * Zero length symlinks should never occur in memory as they are
 	 * never allowed to exist on disk.
@@ -231,3 +227,14 @@ xfs_symlink_shortform_verify(
 		return __this_address;
 	return NULL;
 }
+
+xfs_failaddr_t
+xfs_symlink_shortform_verify(
+	struct xfs_inode	*ip)
+{
+	struct xfs_ifork	*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
+
+	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
+
+	return xfs_symlink_sf_verify_struct(ifp->if_u1.if_data, ifp->if_bytes);
+}
fs/xfs/scrub/inode_repair.c +694 -2

754 lines not on this page · raw message

fs/xfs/scrub/trace.h +42 -0

54 lines not on this page · raw message

#81 lore raw

Re: [PATCH 4/7] xfs: zap broken inode forks

Quoted 5 lines
fs/xfs/libxfs/xfs_attr_leaf.c +5 -0
+/* Verify the consistency of an inline attribute fork. */
+xfs_failaddr_t
+xfs_attr_shortform_verify(
+	struct xfs_inode		*ip)
+{

fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr_shortform_verify( · line 1126

Quoted 10 lines
fs/xfs/libxfs/xfs_attr_leaf.c +10 -0
+	struct xfs_attr_shortform	*sfp;
+	struct xfs_ifork		*ifp;
+	int64_t				size;
+
+	ASSERT(ip->i_af.if_format == XFS_DINODE_FMT_LOCAL);
+	ifp = xfs_ifork_ptr(ip, XFS_ATTR_FORK);
+	sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data;
+	size = ifp->if_bytes;
+
+	return xfs_attr_shortform_verify_struct(sfp, size);

Given that xfs_attr_shortform_verify only has a single caller in the kernel and no extra n in xfsprogs I'd just change the calling convention to pass the xfs_attr_shortform structure and size there and not bother with the wrapper.

fs/xfs/libxfs/xfs_bmap.c xfs_bmap_intent_destroy_cache(void) · line 6233

Quoted 10 lines
fs/xfs/libxfs/xfs_bmap.c +10 -0
+/* Check that an inode's extent does not have invalid flags or bad ranges. */
+xfs_failaddr_t
+xfs_bmap_validate_extent(
+	struct xfs_inode	*ip,
+	int			whichfork,
+	struct xfs_bmbt_irec	*irec)
+{
+	return xfs_bmap_validate_extent_raw(ip->i_mount,
+			XFS_IS_REALTIME_INODE(ip), whichfork, irec);
+}

.. while this one has a bunch of caller so I expect it's actually somewhat useful.

fs/xfs/libxfs/xfs_dir2_priv.h extern int xfs_dir2_sf_create(struct xfs_da_args *args, xfs_ino_t pino); · line 179

Quoted 2 lines
fs/xfs/libxfs/xfs_dir2_priv.h +2 -0
+extern xfs_failaddr_t xfs_dir2_sf_verify_struct(struct xfs_mount *mp,
+		struct xfs_dir2_sf_hdr *sfp, int64_t size);

It would be nice if we didn't add more pointless externs to function declarations in heders.

fs/xfs/libxfs/xfs_dir2_sf.c xfs_dir2_sf_verify( · line 811

Quoted 13 lines
fs/xfs/libxfs/xfs_dir2_sf.c +13 -0
+xfs_failaddr_t
+xfs_dir2_sf_verify(
+	struct xfs_inode		*ip)
+{
+	struct xfs_mount		*mp = ip->i_mount;
+	struct xfs_ifork		*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
+	struct xfs_dir2_sf_hdr		*sfp;
+
+	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
+
+	sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data;
+	return xfs_dir2_sf_verify_struct(mp, sfp, ifp->if_bytes);
+}

This one also only has a single caller in the kernel and user space combined, so I wou;dn't bother with the wrapper.

fs/xfs/libxfs/xfs_symlink_remote.c xfs_symlink_shortform_verify( · line 240

Quoted 10 lines
fs/xfs/libxfs/xfs_symlink_remote.c +10 -0
+xfs_failaddr_t
+xfs_symlink_shortform_verify(
+	struct xfs_inode	*ip)
+{
+	struct xfs_ifork	*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
+
+	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
+
+	return xfs_symlink_sf_verify_struct(ifp->if_u1.if_data, ifp->if_bytes);
+}

Same here.

Once past thes nitpicks the zapping functionality looks fine to me, but leaves me with a very high level question:

As far as I can tell the inodes with the zapped fork(s) remains in it's normal place, and normaly accessible, and I think any read will return zeroes because i_size isn't reset. Which would change the data seen by an application using it. Don't we need to block access to it until it is fully repaired?

#82 lore raw

Re: [PATCH 4/7] xfs: zap broken inode forks

Quoted 12 lines · Christoph Hellwig
Quoted 12 lines
fs/xfs/libxfs/xfs_attr_leaf.c +12 -0
+/* Verify the consistency of an inline attribute fork. */
+xfs_failaddr_t
+xfs_attr_shortform_verify(
+	struct xfs_inode		*ip)
+{
+	struct xfs_attr_shortform	*sfp;
+	struct xfs_ifork		*ifp;
+	int64_t				size;
+
+	ASSERT(ip->i_af.if_format == XFS_DINODE_FMT_LOCAL);
+	ifp = xfs_ifork_ptr(ip, XFS_ATTR_FORK);
+	sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data;
Quoted 8 lines

fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr_shortform_verify( · line 1126

Quoted 3 lines
fs/xfs/libxfs/xfs_attr_leaf.c +3 -0
+	size = ifp->if_bytes;
+
+	return xfs_attr_shortform_verify_struct(sfp, size);

Given that xfs_attr_shortform_verify only has a single caller in the kernel and no extra n in xfsprogs I'd just change the calling convention to pass the xfs_attr_shortform structure and size there and not bother with the wrapper.

Ok.

Quoted 5 lines
Quoted 5 lines
fs/xfs/libxfs/xfs_bmap.c +5 -0
+/* Check that an inode's extent does not have invalid flags or bad ranges. */
+xfs_failaddr_t
+xfs_bmap_validate_extent(
+	struct xfs_inode	*ip,
+	int			whichfork,
Quoted 8 lines

fs/xfs/libxfs/xfs_bmap.c xfs_bmap_intent_destroy_cache(void) · line 6233

Quoted 5 lines
fs/xfs/libxfs/xfs_bmap.c +5 -0
+	struct xfs_bmbt_irec	*irec)
+{
+	return xfs_bmap_validate_extent_raw(ip->i_mount,
+			XFS_IS_REALTIME_INODE(ip), whichfork, irec);
+}

.. while this one has a bunch of caller so I expect it's actually somewhat useful.

Yep. :)

Quoted 5 lines

fs/xfs/libxfs/xfs_dir2_priv.h extern int xfs_dir2_sf_create(struct xfs_da_args *args, xfs_ino_t pino); · line 179

Quoted 2 lines
fs/xfs/libxfs/xfs_dir2_priv.h +2 -0
+extern xfs_failaddr_t xfs_dir2_sf_verify_struct(struct xfs_mount *mp,
+		struct xfs_dir2_sf_hdr *sfp, int64_t size);

It would be nice if we didn't add more pointless externs to function declarations in heders.

I'll get rid of the extern.

Quoted 8 lines
Quoted 8 lines
fs/xfs/libxfs/xfs_dir2_sf.c +8 -0
+xfs_failaddr_t
+xfs_dir2_sf_verify(
+	struct xfs_inode		*ip)
+{
+	struct xfs_mount		*mp = ip->i_mount;
+	struct xfs_ifork		*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
+	struct xfs_dir2_sf_hdr		*sfp;
+
Quoted 8 lines

fs/xfs/libxfs/xfs_dir2_sf.c xfs_dir2_sf_verify( · line 811

Quoted 5 lines
fs/xfs/libxfs/xfs_dir2_sf.c +5 -0
+	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
+
+	sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data;
+	return xfs_dir2_sf_verify_struct(mp, sfp, ifp->if_bytes);
+}

This one also only has a single caller in the kernel and user space combined, so I wou;dn't bother with the wrapper.

<nod>

Quoted 4 lines
Quoted 4 lines
fs/xfs/libxfs/xfs_symlink_remote.c +4 -0
+xfs_failaddr_t
+xfs_symlink_shortform_verify(
+	struct xfs_inode	*ip)
+{
Quoted 8 lines

fs/xfs/libxfs/xfs_symlink_remote.c xfs_symlink_shortform_verify( · line 240

Quoted 6 lines
fs/xfs/libxfs/xfs_symlink_remote.c +6 -0
+	struct xfs_ifork	*ifp = xfs_ifork_ptr(ip, XFS_DATA_FORK);
+
+	ASSERT(ifp->if_format == XFS_DINODE_FMT_LOCAL);
+
+	return xfs_symlink_sf_verify_struct(ifp->if_u1.if_data, ifp->if_bytes);
+}

Same here.

Fixed.

Quoted 8 lines

Once past thes nitpicks the zapping functionality looks fine to me, but leaves me with a very high level question:

As far as I can tell the inodes with the zapped fork(s) remains in it's normal place, and normaly accessible, and I think any read will return zeroes because i_size isn't reset. Which would change the data seen by an application using it. Don't we need to block access to it until it is fully repaired?

Ideally, yes, we ought to do that. It's tricky to do this, however, because i_rwsem doesn't exist until iget succeeds, and we're doing surgery on the dinode buffer to get it into good enough shape that iget will work.

Unfortunately for me, the usual locking order is i_rwsem -> tx freeze protection -> ILOCK. Lockdep will not be happy if I try to grab i_rwsem from withina transaction. Hence the current repair code commits the dinode cleaning function before it tries to iget the inode.

But. trylock exists.

Looking at that code again, the inode scrubber sets us up with the AGI buffer if it can't iget the inode. Repairs to the dinode core acquires the inode cluster buffer, which means that nobody else can be calling iget.

So I think we can grab the inode in the same transaction as the inode core repairs. Nobody else should even be able to see that inode, so it should be safe to grab i_rwsem before committing the transaction. Even if I have to use trylock in a loop to make lockdep happy.

I'll try that out and get back to you.

--D

#83 lore raw

Re: [PATCH 4/7] xfs: zap broken inode forks

Quoted 4 lines · Darrick J. Wong

So I think we can grab the inode in the same transaction as the inode core repairs. Nobody else should even be able to see that inode, so it should be safe to grab i_rwsem before committing the transaction. Even if I have to use trylock in a loop to make lockdep happy.

Hmm, I though more of an inode flag that makes access to the inode outside of the scrubbe return -EIO. I can also warm up to the idea of having all inodes that are broken in some way in lost+found..

#84 lore raw

Re: [PATCH 4/7] xfs: zap broken inode forks

Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong

So I think we can grab the inode in the same transaction as the inode core repairs. Nobody else should even be able to see that inode, so it should be safe to grab i_rwsem before committing the transaction. Even if I have to use trylock in a loop to make lockdep happy.

Hmm, I though more of an inode flag that makes access to the inode outside of the scrubbe return -EIO. I can also warm up to the idea of having all inodes that are broken in some way in lost+found..

Moving things around in the directory tree might be worse, since we'd now have to read the parent pointer(s) from the file to remove those directory connections and add the new ones to lost+found.

I /think/ scouring around in a zapped data fork for a directory access will return EFSCORRUPTED anyway, though that might occur at a late enough stage in the process that the fs goes down, which isn't desirable.

However, once xrep_inode massages the ondisk inode into good enough shape that iget starts working again, I could set XFS_SICK_INO_BMBTD (and XFS_SICK_INO_DIR as appropriate) after zapping the data fork so that the directory accesses would return EFSCORRUPTED instead of scouring around in the zapped fork.

Once we start persisting the sick flags, the prevention will last until scrub or someone came along to fix the inode, instead of being a purely incore flag. But, babysteps for now. I'll fix this patch to set the XFS_SICK_INO_* flags after zapping things, and the predicates to pick them up.

--D

#85 lore raw

Re: [PATCH 4/7] xfs: zap broken inode forks

Quoted 3 lines · Darrick J. Wong

Moving things around in the directory tree might be worse, since we'd now have to read the parent pointer(s) from the file to remove those directory connections and add the new ones to lost+found.

True.

Quoted 11 lines

I /think/ scouring around in a zapped data fork for a directory access will return EFSCORRUPTED anyway, though that might occur at a late enough stage in the process that the fs goes down, which isn't desirable.

However, once xrep_inode massages the ondisk inode into good enough shape that iget starts working again, I could set XFS_SICK_INO_BMBTD (and XFS_SICK_INO_DIR as appropriate) after zapping the data fork so that the directory accesses would return EFSCORRUPTED instead of scouring around in the zapped fork.

Quoted 5 lines

Once we start persisting the sick flags, the prevention will last until scrub or someone came along to fix the inode, instead of being a purely incore flag. But, babysteps for now. I'll fix this patch to set the XFS_SICK_INO_* flags after zapping things, and the predicates to pick them up.

Sounds good.

#86 lore raw

PATCH 5/7 xfs: abort directory parent scrub scans if we encounter a zapped directory

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

In the previous patch, we added some code to perform sufficient repairs to an ondisk inode record such that the inode cache would be willing to load the inode. If the broken inode was a shortform directory, it will reset the directory to something plausible, which is to say an empty subdirectory of the root. The telltale signs that something is seriously wrong is the broken link count.

Such directories look clean, but they shouldn't participate in a filesystem scan to find or confirm a directory parent pointer. Create a predicate that identifies such directories and abort the scrub.

Found by fuzzing xfs/1554 with multithreaded xfs_scrub enabled and u3.bmx[0].startblock = zeroes.

---
 fs/xfs/scrub/common.c |    1 +
 fs/xfs/scrub/common.h |    2 ++
 fs/xfs/scrub/dir.c    |   21 +++++++++++++++++++++
 fs/xfs/scrub/parent.c |   17 +++++++++++++++++
 4 files changed, 41 insertions(+)
fs/xfs/scrub/common.c +1 -0
index 9b7d7010495b9..67ed4c55a27e3 100644
@@ -26,6 +26,7 @@
 #include "xfs_trans_priv.h"
 #include "xfs_da_format.h"
 #include "xfs_da_btree.h"
+#include "xfs_dir2_priv.h"
 #include "xfs_attr.h"
 #include "xfs_reflink.h"
 #include "xfs_ag.h"
fs/xfs/scrub/common.h +2 -0
index 895918565df26..506b808b9fbb3 100644
@@ -192,6 +192,8 @@ static inline bool xchk_skip_xref(struct xfs_scrub_metadata *sm)
 			       XFS_SCRUB_OFLAG_XCORRUPT);
 }
 
+bool xchk_dir_looks_zapped(struct xfs_inode *dp);
+
 #ifdef CONFIG_XFS_ONLINE_REPAIR
 /* Decide if a repair is required. */
 static inline bool xchk_needs_repair(const struct xfs_scrub_metadata *sm)
fs/xfs/scrub/dir.c +21 -0
index 0b491784b7594..acae43d20f387 100644
@@ -788,3 +788,24 @@ xchk_directory(
 		error = 0;
 	return error;
 }
+
+/*
+ * Decide if this directory has been zapped to satisfy the inode and ifork
+ * verifiers.  Checking and repairing should be postponed until the directory
+ * is fixed.
+ */
+bool
+xchk_dir_looks_zapped(
+	struct xfs_inode	*dp)
+{
+	/*
+	 * If the dinode repair found a bad data fork, it will reset the fork
+	 * to extents format with zero records and wait for the bmapbtd
+	 * scrubber to reconstruct the block mappings.  Directories always
+	 * contain some content, so this is a clear sign of a zapped directory.
+	 */
+	if (dp->i_df.if_format == XFS_DINODE_FMT_EXTENTS)
+		return dp->i_df.if_nextents == 0;
+
+	return false;
+}
fs/xfs/scrub/parent.c +17 -0
index e6155d86f7916..7db8736721461 100644
@@ -156,6 +156,16 @@ xchk_parent_validate(
 		goto out_rele;
 	}
 
+	/*
+	 * We cannot yet validate this parent pointer if the directory looks as
+	 * though it has been zapped by the inode record repair code.
+	 */
+	if (xchk_dir_looks_zapped(dp)) {
+		error = -EBUSY;
+		xchk_set_incomplete(sc);
+		goto out_unlock;
+	}
+
 	/* Look for a directory entry in the parent pointing to the child. */
 	error = xchk_dir_walk(sc, dp, xchk_parent_actor, &spc);
 	if (!xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0, &error))
@@ -217,6 +227,13 @@ xchk_parent(
 		 */
 		error = xchk_parent_validate(sc, parent_ino);
 	} while (error == -EAGAIN);
+	if (error == -EBUSY) {
+		/*
+		 * We could not scan a directory, so we marked the check
+		 * incomplete.  No further error return is necessary.
+		 */
+		return 0;
+	}
 
 	return error;
 }
#87 lore raw

Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory

Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

In the previous patch, we added some code to perform sufficient repairs to an ondisk inode record such that the inode cache would be willing to load the inode.

This is now a few commits back. My adjust this to be less specific.

Quoted 11 lines

If the broken inode was a shortform directory, it will reset the directory to something plausible, which is to say an empty subdirectory of the root. The telltale signs that something is seriously wrong is the broken link count.

Such directories look clean, but they shouldn't participate in a filesystem scan to find or confirm a directory parent pointer. Create a predicate that identifies such directories and abort the scrub.

Found by fuzzing xfs/1554 with multithreaded xfs_scrub enabled and u3.bmx[0].startblock = zeroes.

This kind of ties into my comment on the previous comment, but needing heuristics to find zapped inodes or inode forks just seems to be asking for trouble. I suspect we'll need proper on-disk flags to notice the corrupted / half-rebuilt state.

#88 lore raw

Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory

Quoted 18 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

In the previous patch, we added some code to perform sufficient repairs to an ondisk inode record such that the inode cache would be willing to load the inode.

This is now a few commits back. My adjust this to be less specific.

Quoted 9 lines

If the broken inode was a shortform directory, it will reset the directory to something plausible, which is to say an empty subdirectory of the root. The telltale signs that something is seriously wrong is the broken link count.

Such directories look clean, but they shouldn't participate in a filesystem scan to find or confirm a directory parent pointer. Create a predicate that identifies such directories and abort the scrub.

Quoted 7 lines
Quoted 2 lines

Found by fuzzing xfs/1554 with multithreaded xfs_scrub enabled and u3.bmx[0].startblock = zeroes.

This kind of ties into my comment on the previous comment, but needing heuristics to find zapped inodes or inode forks just seems to be asking for trouble. I suspect we'll need proper on-disk flags to notice the corrupted / half-rebuilt state.

Hmm. A single "zapped" bit would be a good way to signal to xchk_dir_looks_zapped and xchk_bmap_want_check_rmaps that a file is probably broken. Clearing that bit would be harder though -- userspace would have to call back into the kernel after checking all the metadata.

A simpler way might be to persist the entire per-inode sick state (both forks and the contents within, for three bits). That would be more to track, but each scrubber could clear its corresponding sick-state bit. A bit further on in this series is a big patchset to set the sick state every time the hot paths encounter an EFSCORRUPTED.

IO operations could check the sick state bit and fail out to userspace, which would solve the problem of keeping programs away from a partially fixed file.

The ondisk state tracking like an entire project on its own. Thoughts?

--D

#89 lore raw

Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory

Quoted 4 lines · Darrick J. Wong

Hmm. A single "zapped" bit would be a good way to signal to xchk_dir_looks_zapped and xchk_bmap_want_check_rmaps that a file is probably broken. Clearing that bit would be harder though -- userspace would have to call back into the kernel after checking all the metadata.

Doesn't sound too horrible to have a special scrub call just for that.

Quoted 5 lines

A simpler way might be to persist the entire per-inode sick state (both forks and the contents within, for three bits). That would be more to track, but each scrubber could clear its corresponding sick-state bit. A bit further on in this series is a big patchset to set the sick state every time the hot paths encounter an EFSCORRUPTED.

That does sound even better.

Quoted 5 lines

IO operations could check the sick state bit and fail out to userspace, which would solve the problem of keeping programs away from a partially fixed file.

The ondisk state tracking like an entire project on its own. Thoughts?

Incore for now sounds fine to me.

#90 lore raw

Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory

Quoted 16 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong

Hmm. A single "zapped" bit would be a good way to signal to xchk_dir_looks_zapped and xchk_bmap_want_check_rmaps that a file is probably broken. Clearing that bit would be harder though -- userspace would have to call back into the kernel after checking all the metadata.

Doesn't sound too horrible to have a special scrub call just for that.

Quoted 5 lines

A simpler way might be to persist the entire per-inode sick state (both forks and the contents within, for three bits). That would be more to track, but each scrubber could clear its corresponding sick-state bit. A bit further on in this series is a big patchset to set the sick state every time the hot paths encounter an EFSCORRUPTED.

That does sound even better.

Quoted 7 lines
Quoted 5 lines

IO operations could check the sick state bit and fail out to userspace, which would solve the problem of keeping programs away from a partially fixed file.

The ondisk state tracking like an entire project on its own. Thoughts?

Incore for now sounds fine to me.

Excellent! I'll go work on that for v28.2 or v29 or whatever the next version number is.

--D

#91 lore raw

PATCH 6/7 xfs: skip the rmapbt search on an empty attr fork unless we know it was zapped

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

The attribute fork scrubber can optionally scan the reverse mapping records of the filesystem to determine if the fork is missing mappings that it should have. However, this is a very expensive operation, so we only want to do this if we suspect that the fork is missing records. For attribute forks the criteria for suspicion is that the attr fork is in EXTENTS format and has zero extents.

However, there are several ways that a file can end up in this state through regular filesystem usage. For example, an LSM can set a s_security hook but then decide not to set an ACL; or an attr set can create the attr fork but then the actual set operation fails with ENOSPC; or we can delete all the attrs on a file whose data fork is in btree format, in which case we do not delete the attr fork. We don't want to run the expensive check for any case that can be arrived at through regular operations.

However.

When online inode repair decides to zap an attribute fork, it cannot determine if it is zapping ACL information. As a precaution it removes all the discretionary access control permissions and sets the user and group ids to zero. Check these three additional conditions to decide if we want to scan the rmap records.

---
 fs/xfs/scrub/bmap.c |   44 +++++++++++++++++++++++++++++++++++++-------
 1 file changed, 37 insertions(+), 7 deletions(-)
fs/xfs/scrub/bmap.c +37 -7
index f74bd2a97c7f7..c12ccc9141163 100644
@@ -662,16 +662,46 @@ xchk_bmap_want_check_rmaps(
 	 * The inode repair code zaps broken inode forks by resetting them back
 	 * to EXTENTS format and zero extent records.  If we encounter a fork
 	 * in this state along with evidence that the fork isn't supposed to be
-	 * empty, we need to scan the reverse mappings to decide if we're going
-	 * to rebuild the fork.  Data forks with nonzero file size are scanned.
-	 * xattr forks are never empty of content, so they are always scanned.
+	 * empty, we might want scan the reverse mappings to decide if we're
+	 * going to rebuild the fork.
 	 */
 	ifp = xfs_ifork_ptr(sc->ip, info->whichfork);
 	if (ifp->if_format == XFS_DINODE_FMT_EXTENTS && ifp->if_nextents == 0) {
-		if (info->whichfork == XFS_DATA_FORK &&
-		    i_size_read(VFS_I(sc->ip)) == 0)
-			return false;
-
+		switch (info->whichfork) {
+		case XFS_DATA_FORK:
+			/*
+			 * Data forks with zero file size are presumed not to
+			 * have any written data blocks.  Skip the scan.
+			 */
+			if (i_size_read(VFS_I(sc->ip)) == 0)
+				return false;
+			break;
+		case XFS_ATTR_FORK:
+			/*
+			 * Files can have an attr fork in EXTENTS format with
+			 * zero records for several reasons:
+			 *
+			 * a) an attr set created a fork but ran out of space
+			 * b) attr replace deleted an old attr but failed
+			 *    during the set step
+			 * c) the data fork was in btree format when all attrs
+			 *    were deleted, so the fork was left in place
+			 * d) the inode repair code zapped the fork
+			 *
+			 * Only in case (d) do we want to scan the rmapbt to
+			 * see if we need to rebuild the attr fork.  The fork
+			 * zap code clears all DAC permission bits and zeroes
+			 * the uid and gid, so avoid the scan if any of those
+			 * three conditions are not met.
+			 */
+			if ((VFS_I(sc->ip)->i_mode & 0777) != 0)
+				return false;
+			if (!uid_eq(VFS_I(sc->ip)->i_uid, GLOBAL_ROOT_UID))
+				return false;
+			if (!gid_eq(VFS_I(sc->ip)->i_gid, GLOBAL_ROOT_GID))
+				return false;
+			break;
+		}
 		return true;
 	}
 
#92 lore raw

PATCH 7/7 xfs: repair obviously broken inode modes

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Building off the rmap scanner that we added in the previous patch, we can now find block 0 and try to use the information contained inside of it to guess the mode of an inode if it's totally improper.

---
 fs/xfs/scrub/inode_repair.c |  181 +++++++++++++++++++++++++++++++++++++++++--
 fs/xfs/scrub/trace.h        |   11 ++-
 2 files changed, 179 insertions(+), 13 deletions(-)
fs/xfs/scrub/inode_repair.c +172 -9
index a73205702ffa5..96114ed889707 100644
@@ -56,10 +56,13 @@
  * fix things on live incore inodes.  The inode repair functions make decisions
  * with security and usability implications when reviving a file:
  *
- * - Files with zero di_mode or a garbage di_mode are converted to regular file
- *   that only root can read.  This file may not actually contain user data,
- *   if the file was not previously a regular file.  Setuid and setgid bits
- *   are cleared.
+ * - Files with zero di_mode or a garbage di_mode are converted to a file
+ *   that only root can read.  If the immediate data fork area or block 0 of
+ *   the data fork look like a directory, the file type will be set to a
+ *   directory.  If the immediate data fork area has no nulls, it will be
+ *   turned into a symbolic link.  Otherwise, it is turned into a regular file.
+ *   This file may not actually contain user data, if the file was not
+ *   previously a regular file.  Setuid and setgid bits are cleared.
  *
  * - Zero-size directories can be truncated to look empty.  It is necessary to
  *   run the bmapbtd and directory repair functions to fully rebuild the
@@ -107,6 +110,9 @@ struct xrep_inode {
 	/* Blocks in use by the attr fork. */
 	xfs_rfsblock_t		attr_blocks;
 
+	/* Physical block containing data block 0. */
+	xfs_fsblock_t		block0;
+
 	/* Number of data device extents for the data fork. */
 	xfs_extnum_t		data_extents;
 
@@ -146,6 +152,7 @@ xrep_setup_inode(
 	ri = sc->buf;
 	memcpy(&ri->imap, imap, sizeof(struct xfs_imap));
 	ri->sc = sc;
+	ri->block0 = NULLFSBLOCK;
 	return 0;
 }
 
@@ -221,12 +228,159 @@ xrep_dinode_header(
 	dip->di_gen = cpu_to_be32(sc->sm->sm_gen);
 }
 
+/* Parse enough of the directory block header to guess if this is a dir. */
+static inline bool
+xrep_dinode_is_dir(
+	xfs_ino_t			ino,
+	xfs_daddr_t			daddr,
+	struct xfs_buf			*bp)
+{
+	struct xfs_dir3_blk_hdr		*hdr3 = bp->b_addr;
+	struct xfs_dir2_data_free	*bf;
+	struct xfs_mount		*mp = bp->b_mount;
+	xfs_lsn_t			lsn = be64_to_cpu(hdr3->lsn);
+
+	/* Does the dir3 header match the filesystem? */
+	if (hdr3->magic != cpu_to_be32(XFS_DIR3_BLOCK_MAGIC) &&
+	    hdr3->magic != cpu_to_be32(XFS_DIR3_DATA_MAGIC))
+		return false;
+
+	if (be64_to_cpu(hdr3->owner) != ino)
+		return false;
+
+	if (!uuid_equal(&hdr3->uuid, &mp->m_sb.sb_meta_uuid))
+		return false;
+
+	if (be64_to_cpu(hdr3->blkno) != daddr)
+		return false;
+
+	/* Directory blocks are always logged and must have a valid LSN. */
+	if (lsn == NULLCOMMITLSN)
+		return false;
+	if (!xlog_valid_lsn(mp->m_log, lsn))
+		return false;
+
+	/*
+	 * bestfree information lives immediately after the end of the header,
+	 * so we won't run off the end of the buffer.
+	 */
+	bf = xfs_dir2_data_bestfree_p(mp, bp->b_addr);
+	if (!bf[0].length && bf[0].offset)
+		return false;
+	if (!bf[1].length && bf[1].offset)
+		return false;
+	if (!bf[2].length && bf[2].offset)
+		return false;
+
+	if (be16_to_cpu(bf[0].length) < be16_to_cpu(bf[1].length))
+		return false;
+	if (be16_to_cpu(bf[1].length) < be16_to_cpu(bf[2].length))
+		return false;
+
+	return true;
+}
+
+/* Guess the mode of this file from the contents. */
+STATIC uint16_t
+xrep_dinode_guess_mode(
+	struct xrep_inode	*ri,
+	struct xfs_dinode	*dip)
+{
+	struct xfs_buf		*bp;
+	struct xfs_mount	*mp = ri->sc->mp;
+	xfs_daddr_t		daddr;
+	uint64_t		fsize = be64_to_cpu(dip->di_size);
+	unsigned int		dfork_sz = XFS_DFORK_DSIZE(dip, mp);
+	uint16_t		mode = S_IFREG;
+	int			error;
+
+	switch (dip->di_format) {
+	case XFS_DINODE_FMT_LOCAL:
+		/*
+		 * If the data fork is local format, the size of the data area
+		 * is reasonable and is big enough to contain the entire file,
+		 * we can guess the file type from the local data.
+		 *
+		 * If there are no nulls, guess this is a symbolic link.
+		 * Otherwise, this is probably a shortform directory.
+		 */
+		if (dfork_sz <= XFS_LITINO(mp) && dfork_sz >= fsize) {
+			if (!memchr(XFS_DFORK_DPTR(dip), 0, fsize))
+				return S_IFLNK;
+			return S_IFDIR;
+		}
+
+		/* By default, we guess regular file. */
+		return S_IFREG;
+	case XFS_DINODE_FMT_DEV:
+		/*
+		 * If the data fork is dev format, the size of the data area is
+		 * reasonable and large enough to store a dev_t, and the file
+		 * size is zero, this could be a blockdev, a chardev, a fifo,
+		 * or a socket.  There is no solid way to distinguish between
+		 * those choices, so we guess blockdev if the device number is
+		 * nonzero and chardev if it's zero (aka whiteout).
+		 */
+		if (dfork_sz <= XFS_LITINO(mp) &&
+		    dfork_sz >= sizeof(__be32) && fsize == 0) {
+			xfs_dev_t	dev = xfs_dinode_get_rdev(dip);
+
+			return dev != 0 ? S_IFBLK : S_IFCHR;
+		}
+
+		/* By default, we guess regular file. */
+		return S_IFREG;
+	case XFS_DINODE_FMT_EXTENTS:
+	case XFS_DINODE_FMT_BTREE:
+		/* There are data blocks to examine below. */
+		break;
+	default:
+		/* Everything else is considered a regular file. */
+		return S_IFREG;
+	}
+
+	/* There are no zero-length directories. */
+	if (fsize == 0)
+		return S_IFREG;
+
+	/*
+	 * If we didn't find a written mapping for file block zero, we'll guess
+	 * that it's a sparse regular file.
+	 */
+	if (ri->block0 == NULLFSBLOCK)
+		return S_IFREG;
+
+	/* Directories can't have rt extents. */
+	if (ri->rt_extents > 0)
+		return S_IFREG;
+
+	/*
+	 * Read the first block of the file.  Since we have no idea what kind
+	 * of file geometry (e.g. dirblock size) we might be reading into, use
+	 * an uncached buffer so that we don't pollute the buffer cache.  We
+	 * can't do uncached mapped buffers, so the best we can do is guess
+	 * from the directory header.
+	 */
+	daddr = XFS_FSB_TO_DADDR(mp, ri->block0);
+	error = xfs_buf_read_uncached(mp->m_ddev_targp, daddr,
+			XFS_FSS_TO_BB(mp, 1), 0, &bp, NULL);
+	if (error)
+		return S_IFREG;
+
+	if (xrep_dinode_is_dir(ri->sc->sm->sm_ino, daddr, bp))
+		mode = S_IFDIR;
+
+	xfs_buf_relse(bp);
+	return mode;
+}
+
 /* Turn di_mode into /something/ recognizable. */
 STATIC void
 xrep_dinode_mode(
-	struct xfs_scrub	*sc,
+	struct xrep_inode	*ri,
 	struct xfs_dinode	*dip)
 {
+	struct xfs_scrub	*sc = ri->sc;
 	uint16_t		mode;
 
 	trace_xrep_dinode_mode(sc, dip);
@@ -236,7 +390,7 @@ xrep_dinode_mode(
 		return;
 
 	/* bad mode, so we set it to a file that only root can read */
-	mode = S_IFREG;
+	mode = xrep_dinode_guess_mode(ri, dip);
 	dip->di_mode = cpu_to_be16(mode);
 	dip->di_uid = 0;
 	dip->di_gid = 0;
@@ -443,9 +597,17 @@ xrep_dinode_walk_rmap(
 	}
 
 	ri->data_blocks += rec->rm_blockcount;
-	if (!(rec->rm_flags & XFS_RMAP_BMBT_BLOCK))
+	if (!(rec->rm_flags & XFS_RMAP_BMBT_BLOCK)) {
 		ri->data_extents++;
 
+		if (rec->rm_offset == 0 &&
+		    !(rec->rm_flags & XFS_RMAP_UNWRITTEN)) {
+			if (ri->block0 != NULLFSBLOCK)
+				return -EFSCORRUPTED;
+			ri->block0 = rec->rm_startblock;
+		}
+	}
+
 	return 0;
 }
 
@@ -496,7 +658,8 @@ xrep_dinode_count_rmaps(
 
 	trace_xrep_dinode_count_rmaps(ri->sc,
 			ri->data_blocks, ri->rt_blocks, ri->attr_blocks,
-			ri->data_extents, ri->rt_extents, ri->attr_extents);
+			ri->data_extents, ri->rt_extents, ri->attr_extents,
+			ri->block0);
 	return 0;
 }
 
@@ -1090,7 +1253,7 @@ xrep_dinode_core(
 	/* Fix everything the verifier will complain about. */
 	dip = xfs_buf_offset(bp, ri->imap.im_boffset);
 	xrep_dinode_header(sc, dip);
-	xrep_dinode_mode(sc, dip);
+	xrep_dinode_mode(ri, dip);
 	xrep_dinode_flags(sc, dip, ri->rt_extents > 0);
 	xrep_dinode_size(sc, dip);
 	xrep_dinode_extsize_hints(sc, dip);
fs/xfs/scrub/trace.h +7 -4
index 75f0d57088b29..6cd5d04c0410c 100644
@@ -1530,9 +1530,9 @@ TRACE_EVENT(xrep_dinode_count_rmaps,
 	TP_PROTO(struct xfs_scrub *sc, xfs_rfsblock_t data_blocks,
 		xfs_rfsblock_t rt_blocks, xfs_rfsblock_t attr_blocks,
 		xfs_extnum_t data_extents, xfs_extnum_t rt_extents,
-		xfs_aextnum_t attr_extents),
+		xfs_aextnum_t attr_extents, xfs_fsblock_t block0),
 	TP_ARGS(sc, data_blocks, rt_blocks, attr_blocks, data_extents,
-		rt_extents, attr_extents),
+		rt_extents, attr_extents, block0),
 	TP_STRUCT__entry(
 		__field(dev_t, dev)
 		__field(xfs_ino_t, ino)
@@ -1542,6 +1542,7 @@ TRACE_EVENT(xrep_dinode_count_rmaps,
 		__field(xfs_extnum_t, data_extents)
 		__field(xfs_extnum_t, rt_extents)
 		__field(xfs_aextnum_t, attr_extents)
+		__field(xfs_fsblock_t, block0)
 	),
 	TP_fast_assign(
 		__entry->dev = sc->mp->m_super->s_dev;
@@ -1552,8 +1553,9 @@ TRACE_EVENT(xrep_dinode_count_rmaps,
 		__entry->data_extents = data_extents;
 		__entry->rt_extents = rt_extents;
 		__entry->attr_extents = attr_extents;
+		__entry->block0 = block0;
 	),
-	TP_printk("dev %d:%d ino 0x%llx dblocks 0x%llx rtblocks 0x%llx ablocks 0x%llx dextents %llu rtextents %llu aextents %u",
+	TP_printk("dev %d:%d ino 0x%llx dblocks 0x%llx rtblocks 0x%llx ablocks 0x%llx dextents %llu rtextents %llu aextents %u startblock0 0x%llx",
 		  MAJOR(__entry->dev), MINOR(__entry->dev),
 		  __entry->ino,
 		  __entry->data_blocks,
@@ -1561,7 +1563,8 @@ TRACE_EVENT(xrep_dinode_count_rmaps,
 		  __entry->attr_blocks,
 		  __entry->data_extents,
 		  __entry->rt_extents,
-		  __entry->attr_extents)
+		  __entry->attr_extents,
+		  __entry->block0)
 );
 
 #endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
#93 lore raw

Re: [PATCH 7/7] xfs: repair obviously broken inode modes

Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Building off the rmap scanner that we added in the previous patch, we can now find block 0 and try to use the information contained inside of it to guess the mode of an inode if it's totally improper.

Maybe I'm missing something important, but I don't see why a normal user couldn't construct a file that looks like an XFS directory, and that's a perfectly fine thing to do?

#94 lore raw

Re: [PATCH 7/7] xfs: repair obviously broken inode modes

Quoted 10 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Building off the rmap scanner that we added in the previous patch, we can now find block 0 and try to use the information contained inside of it to guess the mode of an inode if it's totally improper.

Maybe I'm missing something important, but I don't see why a normal user couldn't construct a file that looks like an XFS directory, and that's a perfectly fine thing to do?

They could very well do that, and it might confuse the scanner. However, I'd like to draw your attention to xrep_dinode_mode, which will set the user/group to root with 0000 access mode. That at least will keep unprivileged users from seeing the potentially weird file until the higher level repair functions can deal with it (or the sysadmin deletes it).

Hmm. That code really ought to zap the attr fork because there could be ACLs attached to the file. Let me go do that.

--D

#95 lore raw

Re: [PATCH 7/7] xfs: repair obviously broken inode modes

Quoted 10 lines · Darrick J. Wong
Quoted 3 lines

Maybe I'm missing something important, but I don't see why a normal user couldn't construct a file that looks like an XFS directory, and that's a perfectly fine thing to do?

They could very well do that, and it might confuse the scanner. However, I'd like to draw your attention to xrep_dinode_mode, which will set the user/group to root with 0000 access mode. That at least will keep unprivileged users from seeing the potentially weird file until the higher level repair functions can deal with it (or the sysadmin deletes it).

Having a perfectly valid (but weird) file cause repair action just seems like a really bad idea.

#96 lore raw

cover xfs: online repair of file fork mappings

In reply to Darrick J. Wong

Hi all,

In this series, online repair gains the ability to rebuild data and attr fork mappings from the reverse mapping information. It is at this point where we reintroduce the ability to reap file extents.

Repair of CoW forks is a little different -- on disk, CoW staging extents are owned by the refcount btree and cannot be mapped back to individual files. Hence we can only detect staging extents that don't quite look right (missing reverse mappings, shared staging extents) and replace them with fresh allocations.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-file-mappings

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-file-mappings

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-file-mappings

---
 fs/xfs/Makefile                   |    2 
 fs/xfs/libxfs/xfs_bmap_btree.c    |  119 ++++-
 fs/xfs/libxfs/xfs_bmap_btree.h    |    5 
 fs/xfs/libxfs/xfs_btree_staging.c |   11 
 fs/xfs/libxfs/xfs_btree_staging.h |    2 
 fs/xfs/libxfs/xfs_iext_tree.c     |   23 +
 fs/xfs/libxfs/xfs_inode_fork.c    |    1 
 fs/xfs/libxfs/xfs_inode_fork.h    |    3 
 fs/xfs/libxfs/xfs_refcount.c      |   41 ++
 fs/xfs/libxfs/xfs_refcount.h      |   10 
 fs/xfs/scrub/bitmap.h             |   56 ++
 fs/xfs/scrub/bmap.c               |   18 +
 fs/xfs/scrub/bmap_repair.c        |  846 +++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/common.h             |    6 
 fs/xfs/scrub/cow_repair.c         |  612 +++++++++++++++++++++++++++
 fs/xfs/scrub/reap.c               |  152 ++++++-
 fs/xfs/scrub/reap.h               |    2 
 fs/xfs/scrub/repair.c             |   50 ++
 fs/xfs/scrub/repair.h             |   11 
 fs/xfs/scrub/scrub.c              |   20 -
 fs/xfs/scrub/trace.h              |  118 +++++
 fs/xfs/xfs_trans.c                |   95 ++++
 fs/xfs/xfs_trans.h                |    4 
 23 files changed, 2155 insertions(+), 52 deletions(-)
 create mode 100644 fs/xfs/scrub/bmap_repair.c
 create mode 100644 fs/xfs/scrub/cow_repair.c
#97 lore raw

PATCH 1/5 xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents

From: Darrick J. Wong <[email protected]>

Reintroduce to xrep_reap_extents the ability to reap extents from any AG. We dropped this before because it was buggy, but in the next patch we will gain the ability to reap old bmap btrees, which can have blocks in any AG. To do this, we require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent.

---
 fs/xfs/scrub/bitmap.h |   28 +++++++++++
 fs/xfs/scrub/reap.c   |  120 +++++++++++++++++++++++++++++++++++++++++++++++--
 fs/xfs/scrub/reap.h   |    2 +
 fs/xfs/scrub/repair.h |    1 
 4 files changed, 147 insertions(+), 4 deletions(-)
fs/xfs/scrub/bitmap.h +28 -0
index 9cdc41b6cb02d..1356a76710ede 100644
@@ -121,4 +121,32 @@ int xagb_bitmap_set_btblocks(struct xagb_bitmap *bitmap,
 int xagb_bitmap_set_btcur_path(struct xagb_bitmap *bitmap,
 		struct xfs_btree_cur *cur);
 
+/* Bitmaps, but for type-checked for xfs_fsblock_t */
+
+struct xfsb_bitmap {
+	struct xbitmap	fsbitmap;
+};
+
+static inline void xfsb_bitmap_init(struct xfsb_bitmap *bitmap)
+{
+	xbitmap_init(&bitmap->fsbitmap);
+}
+
+static inline void xfsb_bitmap_destroy(struct xfsb_bitmap *bitmap)
+{
+	xbitmap_destroy(&bitmap->fsbitmap);
+}
+
+static inline int xfsb_bitmap_set(struct xfsb_bitmap *bitmap,
+		xfs_fsblock_t start, xfs_filblks_t len)
+{
+	return xbitmap_set(&bitmap->fsbitmap, start, len);
+}
+
+static inline int xfsb_bitmap_walk(struct xfsb_bitmap *bitmap,
+		xbitmap_walk_fn fn, void *priv)
+{
+	return xbitmap_walk(&bitmap->fsbitmap, fn, priv);
+}
+
 #endif	/* __XFS_SCRUB_BITMAP_H__ */
fs/xfs/scrub/reap.c +116 -4

129 lines not on this page · raw message

fs/xfs/scrub/reap.h +2 -0
index fe24626af1649..5e710be44b4b1 100644
@@ -8,5 +8,7 @@
 
 int xrep_reap_agblocks(struct xfs_scrub *sc, struct xagb_bitmap *bitmap,
 		const struct xfs_owner_info *oinfo, enum xfs_ag_resv_type type);
+int xrep_reap_fsblocks(struct xfs_scrub *sc, struct xfsb_bitmap *bitmap,
+		const struct xfs_owner_info *oinfo);
 
 #endif /* __XFS_SCRUB_REAP_H__ */
fs/xfs/scrub/repair.h +1 -0
index 70a6b18e5ad3c..46bf841524f8f 100644
@@ -48,6 +48,7 @@ xrep_trans_commit(
 
 struct xbitmap;
 struct xagb_bitmap;
+struct xfsb_bitmap;
 
 int xrep_fix_freelist(struct xfs_scrub *sc, bool can_shrink);
 
Reviewed-by on v1 1/5
#98 lore raw

Re: [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents

Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Reintroduce to xrep_reap_extents the ability to reap extents from any AG. We dropped this before because it was buggy, but in the next patch we will gain the ability to reap old bmap btrees, which can have blocks in any AG. To do this, we require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent.

Can you expand a bit on why it was buggy, in what commit is was dropped and what we're doing better this time around?

fs/xfs/scrub/repair.h xrep_trans_commit( · line 51

Quoted 11 lines

#endif /* __XFS_SCRUB_REAP_H__ */

fs/xfs/scrub/repair.h +1 -0
index 70a6b18e5ad3c..46bf841524f8f 100644
@@ -48,6 +48,7 @@ xrep_trans_commit(
 
 struct xbitmap;
 struct xagb_bitmap;
+struct xfsb_bitmap;

Your might need the forward declaration in reap.h, but definitively not here :)

Otherwise looks good:

#99 lore raw

Re: [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents

Quoted 11 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Reintroduce to xrep_reap_extents the ability to reap extents from any AG. We dropped this before because it was buggy, but in the next patch we will gain the ability to reap old bmap btrees, which can have blocks in any AG. To do this, we require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent.

Can you expand a bit on why it was buggy, in what commit is was dropped and what we're doing better this time around?

Oh! We merged that one! Let me change the commit message:

"Back in commit a55e07308831b ("xfs: only allow reaping of per-AG blocks in xrep_reap_extents"), we removed from the reaping code the ability to handle bmbt blocks. At the time, the reaping code only walked single blocks, didn't correctly detect crosslinked blocks, and the special casing made the function hard to understand. It was easier to remove unneeded functionality prior to fixing all the bugs.

"Now that we've fixed the problems, we want again the ability to reap file metadata blocks. Reintroduce the per-file reaping functionality atop the current implementation. We require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent."

Quoted 16 lines

fs/xfs/scrub/repair.h xrep_trans_commit( · line 51

Quoted 12 lines

#endif /* __XFS_SCRUB_REAP_H__ */

fs/xfs/scrub/repair.h +1 -0
index 70a6b18e5ad3c..46bf841524f8f 100644
@@ -48,6 +48,7 @@ xrep_trans_commit(
 
 struct xbitmap;
 struct xagb_bitmap;
+struct xfsb_bitmap;

Your might need the forward declaration in reap.h, but definitively not here :)

Quoted 3 lines

Otherwise looks good:

Thanks!

--D

#100 lore raw

Re: [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents

Quoted 12 lines · Darrick J. Wong
Quoted 2 lines

Can you expand a bit on why it was buggy, in what commit is was dropped and what we're doing better this time around?

Oh! We merged that one! Let me change the commit message:

"Back in commit a55e07308831b ("xfs: only allow reaping of per-AG blocks in xrep_reap_extents"), we removed from the reaping code the ability to handle bmbt blocks. At the time, the reaping code only walked single blocks, didn't correctly detect crosslinked blocks, and the special casing made the function hard to understand. It was easier to remove unneeded functionality prior to fixing all the bugs.

Quoted 5 lines

"Now that we've fixed the problems, we want again the ability to reap file metadata blocks. Reintroduce the per-file reaping functionality atop the current implementation. We require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent."

That looks much better, thanks!

#101 lore raw

PATCH 2/5 xfs: repair inode fork block mapping data structures

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Use the reverse-mapping btree information to rebuild an inode block map. Update the btree bulk loading code as necessary to support inode rooted btrees and fix some bitrot problems.

---
 fs/xfs/Makefile                   |    1 
 fs/xfs/libxfs/xfs_bmap_btree.c    |  119 ++++-
 fs/xfs/libxfs/xfs_bmap_btree.h    |    5 
 fs/xfs/libxfs/xfs_btree_staging.c |   11 
 fs/xfs/libxfs/xfs_btree_staging.h |    2 
 fs/xfs/libxfs/xfs_iext_tree.c     |   23 +
 fs/xfs/libxfs/xfs_inode_fork.c    |    1 
 fs/xfs/libxfs/xfs_inode_fork.h    |    3 
 fs/xfs/scrub/bmap.c               |   18 +
 fs/xfs/scrub/bmap_repair.c        |  846 +++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/common.h             |    6 
 fs/xfs/scrub/repair.c             |   28 +
 fs/xfs/scrub/repair.h             |    6 
 fs/xfs/scrub/scrub.c              |    4 
 fs/xfs/scrub/trace.h              |   34 +
 fs/xfs/xfs_trans.c                |   95 ++++
 fs/xfs/xfs_trans.h                |    4 
 17 files changed, 1172 insertions(+), 34 deletions(-)
 create mode 100644 fs/xfs/scrub/bmap_repair.c
fs/xfs/Makefile +1 -0
index 0d86d75422f60..f62351d63b147 100644
@@ -182,6 +182,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
 xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   alloc_repair.o \
+				   bmap_repair.o \
 				   ialloc_repair.o \
 				   inode_repair.o \
 				   newbt.o \
fs/xfs/libxfs/xfs_bmap_btree.c +99 -20

164 lines not on this page · raw message

fs/xfs/libxfs/xfs_bmap_btree.h +5 -0
index 3e7a40a83835c..151b8491f60ee 100644
@@ -11,6 +11,7 @@ struct xfs_btree_block;
 struct xfs_mount;
 struct xfs_inode;
 struct xfs_trans;
+struct xbtree_ifakeroot;
 
 /*
  * Btree block header size depends on a superblock flag.
@@ -106,6 +107,10 @@ extern int xfs_bmbt_change_owner(struct xfs_trans *tp, struct xfs_inode *ip,
 
 extern struct xfs_btree_cur *xfs_bmbt_init_cursor(struct xfs_mount *,
 		struct xfs_trans *, struct xfs_inode *, int);
+struct xfs_btree_cur *xfs_bmbt_stage_cursor(struct xfs_mount *mp,
+		struct xfs_inode *ip, struct xbtree_ifakeroot *ifake);
+void xfs_bmbt_commit_staged_btree(struct xfs_btree_cur *cur,
+		struct xfs_trans *tp, int whichfork);
 
 extern unsigned long long xfs_bmbt_calc_size(struct xfs_mount *mp,
 		unsigned long long len);
fs/xfs/libxfs/xfs_btree_staging.c +9 -2
index 6fd6ea8e6fbd7..4cdf7976b7bf5 100644
@@ -399,7 +399,7 @@ xfs_btree_bload_prep_block(
 		ASSERT(*bpp == NULL);
 
 		/* Allocate a new incore btree root block. */
-		new_size = bbl->iroot_size(cur, nr_this_block, priv);
+		new_size = bbl->iroot_size(cur, level, nr_this_block, priv);
 		ifp->if_broot = kmem_zalloc(new_size, 0);
 		ifp->if_broot_bytes = (int)new_size;
 
@@ -585,7 +585,14 @@ xfs_btree_bload_level_geometry(
 	unsigned int		desired_npb;
 	unsigned int		maxnr;
 
-	maxnr = cur->bc_ops->get_maxrecs(cur, level);
+	/*
+	 * Compute the absolute maximum number of records that we can store in
+	 * the ondisk block or inode root.
+	 */
+	if (cur->bc_ops->get_dmaxrecs)
+		maxnr = cur->bc_ops->get_dmaxrecs(cur, level);
+	else
+		maxnr = cur->bc_ops->get_maxrecs(cur, level);
 
 	/*
 	 * Compute the number of blocks we need to fill each block with the
fs/xfs/libxfs/xfs_btree_staging.h +1 -1
index d2eaf4fdc6032..439d3490c878a 100644
@@ -56,7 +56,7 @@ typedef int (*xfs_btree_bload_get_records_fn)(struct xfs_btree_cur *cur,
 typedef int (*xfs_btree_bload_claim_block_fn)(struct xfs_btree_cur *cur,
 		union xfs_btree_ptr *ptr, void *priv);
 typedef size_t (*xfs_btree_bload_iroot_size_fn)(struct xfs_btree_cur *cur,
-		unsigned int nr_this_level, void *priv);
+		unsigned int level, unsigned int nr_this_level, void *priv);
 
 struct xfs_btree_bload {
 	/*
fs/xfs/libxfs/xfs_iext_tree.c +16 -7
index 773cf43494286..d062794cc7957 100644
@@ -622,13 +622,11 @@ static inline void xfs_iext_inc_seq(struct xfs_ifork *ifp)
 }
 
 void
-xfs_iext_insert(
-	struct xfs_inode	*ip,
+xfs_iext_insert_raw(
+	struct xfs_ifork	*ifp,
 	struct xfs_iext_cursor	*cur,
-	struct xfs_bmbt_irec	*irec,
-	int			state)
+	struct xfs_bmbt_irec	*irec)
 {
-	struct xfs_ifork	*ifp = xfs_iext_state_to_fork(ip, state);
 	xfs_fileoff_t		offset = irec->br_startoff;
 	struct xfs_iext_leaf	*new = NULL;
 	int			nr_entries, i;
@@ -662,12 +660,23 @@ xfs_iext_insert(
 	xfs_iext_set(cur_rec(cur), irec);
 	ifp->if_bytes += sizeof(struct xfs_iext_rec);
 
-	trace_xfs_iext_insert(ip, cur, state, _RET_IP_);
-
 	if (new)
 		xfs_iext_insert_node(ifp, xfs_iext_leaf_key(new, 0), new, 2);
 }
 
+void
+xfs_iext_insert(
+	struct xfs_inode	*ip,
+	struct xfs_iext_cursor	*cur,
+	struct xfs_bmbt_irec	*irec,
+	int			state)
+{
+	struct xfs_ifork	*ifp = xfs_iext_state_to_fork(ip, state);
+
+	xfs_iext_insert_raw(ifp, cur, irec);
+	trace_xfs_iext_insert(ip, cur, state, _RET_IP_);
+}
+
 static struct xfs_iext_node *
 xfs_iext_rebalance_node(
 	struct xfs_iext_node	*parent,
fs/xfs/libxfs/xfs_inode_fork.c +1 -0
index 5a2e7ddfa76d6..2390884e0075b 100644
@@ -520,6 +520,7 @@ xfs_idata_realloc(
 	ifp->if_bytes = new_size;
 }
 
+/* Free all memory and reset a fork back to its initial state. */
 void
 xfs_idestroy_fork(
 	struct xfs_ifork	*ifp)
fs/xfs/libxfs/xfs_inode_fork.h +3 -0
index 96d307784c85b..535be5c036899 100644
@@ -180,6 +180,9 @@ void		xfs_init_local_fork(struct xfs_inode *ip, int whichfork,
 				const void *data, int64_t size);
 
 xfs_extnum_t	xfs_iext_count(struct xfs_ifork *ifp);
+void		xfs_iext_insert_raw(struct xfs_ifork *ifp,
+			struct xfs_iext_cursor *cur,
+			struct xfs_bmbt_irec *irec);
 void		xfs_iext_insert(struct xfs_inode *, struct xfs_iext_cursor *cur,
 			struct xfs_bmbt_irec *, int);
 void		xfs_iext_remove(struct xfs_inode *, struct xfs_iext_cursor *,
fs/xfs/scrub/bmap.c +18 -0
index c12ccc9141163..737ab982a2d7a 100644
@@ -48,9 +48,18 @@ xchk_setup_inode_bmap(
 	if (S_ISREG(VFS_I(sc->ip)->i_mode) &&
 	    sc->sm->sm_type != XFS_SCRUB_TYPE_BMBTA) {
 		struct address_space	*mapping = VFS_I(sc->ip)->i_mapping;
+		bool			is_repair = xchk_could_repair(sc);
 
 		xchk_ilock(sc, XFS_MMAPLOCK_EXCL);
 
+		/* Break all our leases, we're going to mess with things. */
+		if (is_repair) {
+			error = xfs_break_layouts(VFS_I(sc->ip),
+					&sc->ilock_flags, BREAK_WRITE);
+			if (error)
+				goto out;
+		}
+
 		inode_dio_wait(VFS_I(sc->ip));
 
 		/*
@@ -71,6 +80,15 @@ xchk_setup_inode_bmap(
 			error = filemap_fdatawait_keep_errors(mapping);
 		if (error && (error != -ENOSPC && error != -EIO))
 			goto out;
+
+		/* Drop the page cache if we're repairing block mappings. */
+		if (is_repair) {
+			error = invalidate_inode_pages2(
+					VFS_I(sc->ip)->i_mapping);
+			if (error)
+				goto out;
+		}
+
 	}
 
 	/* Got the inode, lock it and we're ready to go. */
fs/xfs/scrub/bmap_repair.c new file +846 -0

846 lines not on this page · raw message

fs/xfs/scrub/common.h +5 -1
index 506b808b9fbb3..1033d8172be62 100644
@@ -233,7 +233,11 @@ int xchk_metadata_inode_forks(struct xfs_scrub *sc);
 			(sc)->mp->m_super->s_id, \
 			(sc)->sa.pag ? (sc)->sa.pag->pag_agno : (sc)->sm->sm_agno, \
 			##__VA_ARGS__)
-
+#define xchk_xfile_ino_descr(sc, fmt, ...) \
+	kasprintf(XCHK_GFP_FLAGS, "XFS (%s): inode 0x%llx " fmt, \
+			(sc)->mp->m_super->s_id, \
+			(sc)->ip ? (sc)->ip->i_ino : (sc)->sm->sm_ino, \
+			##__VA_ARGS__)
 
 /*
  * Setting up a hook to wait for intents to drain is costly -- we have to take
fs/xfs/scrub/repair.c +28 -0
index 82c9760776248..4d5bfb2e4cf08 100644
@@ -880,6 +880,34 @@ xrep_reinit_pagi(
 	return 0;
 }
 
+/*
+ * Given an active reference to a perag structure, load AG headers and cursors.
+ * This should only be called to scan an AG while repairing file-based metadata.
+ */
+int
+xrep_ag_init(
+	struct xfs_scrub	*sc,
+	struct xfs_perag	*pag,
+	struct xchk_ag		*sa)
+{
+	int			error;
+
+	ASSERT(!sa->pag);
+
+	error = xfs_ialloc_read_agi(pag, sc->tp, &sa->agi_bp);
+	if (error)
+		return error;
+
+	error = xfs_alloc_read_agf(pag, sc->tp, 0, &sa->agf_bp);
+	if (error)
+		return error;
+
+	/* Grab our own passive reference from the caller's ref. */
+	sa->pag = xfs_perag_hold(pag);
+	xrep_ag_btcur_init(sc, sa);
+	return 0;
+}
+
 /* Reinitialize the per-AG block reservation for the AG we just fixed. */
 int
 xrep_reset_perag_resv(
fs/xfs/scrub/repair.h +6 -0
index 46bf841524f8f..9f0c77b38ae28 100644
@@ -89,6 +89,8 @@ struct xfs_imap;
 int xrep_setup_inode(struct xfs_scrub *sc, struct xfs_imap *imap);
 
 void xrep_ag_btcur_init(struct xfs_scrub *sc, struct xchk_ag *sa);
+int xrep_ag_init(struct xfs_scrub *sc, struct xfs_perag *pag,
+		struct xchk_ag *sa);
 
 /* Metadata revalidators */
 
@@ -106,6 +108,8 @@ int xrep_allocbt(struct xfs_scrub *sc);
 int xrep_iallocbt(struct xfs_scrub *sc);
 int xrep_refcountbt(struct xfs_scrub *sc);
 int xrep_inode(struct xfs_scrub *sc);
+int xrep_bmap_data(struct xfs_scrub *sc);
+int xrep_bmap_attr(struct xfs_scrub *sc);
 
 int xrep_reinit_pagf(struct xfs_scrub *sc);
 int xrep_reinit_pagi(struct xfs_scrub *sc);
@@ -165,6 +169,8 @@ xrep_setup_nothing(
 #define xrep_iallocbt			xrep_notsupported
 #define xrep_refcountbt			xrep_notsupported
 #define xrep_inode			xrep_notsupported
+#define xrep_bmap_data			xrep_notsupported
+#define xrep_bmap_attr			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +2 -2
index b9edda17ab64b..52a09e0652693 100644
@@ -290,13 +290,13 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_INODE,
 		.setup	= xchk_setup_inode_bmap,
 		.scrub	= xchk_bmap_data,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_bmap_data,
 	},
 	[XFS_SCRUB_TYPE_BMBTA] = {	/* inode attr fork */
 		.type	= ST_INODE,
 		.setup	= xchk_setup_inode_bmap,
 		.scrub	= xchk_bmap_attr,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_bmap_attr,
 	},
 	[XFS_SCRUB_TYPE_BMBTC] = {	/* inode CoW fork */
 		.type	= ST_INODE,
fs/xfs/scrub/trace.h +33 -1
index 6cd5d04c0410c..3d55f65c00835 100644
@@ -1175,7 +1175,7 @@ DEFINE_EVENT(xrep_rmap_class, name, \
 	TP_ARGS(mp, agno, agbno, len, owner, offset, flags))
 DEFINE_REPAIR_RMAP_EVENT(xrep_ibt_walk_rmap);
 DEFINE_REPAIR_RMAP_EVENT(xrep_rmap_extent_fn);
-DEFINE_REPAIR_RMAP_EVENT(xrep_bmap_extent_fn);
+DEFINE_REPAIR_RMAP_EVENT(xrep_bmap_walk_rmap);
 
 TRACE_EVENT(xrep_abt_found,
 	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
@@ -1260,6 +1260,38 @@ TRACE_EVENT(xrep_refc_found,
 		  __entry->refcount)
 )
 
+TRACE_EVENT(xrep_bmap_found,
+	TP_PROTO(struct xfs_inode *ip, int whichfork,
+		 struct xfs_bmbt_irec *irec),
+	TP_ARGS(ip, whichfork, irec),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_ino_t, ino)
+		__field(int, whichfork)
+		__field(xfs_fileoff_t, lblk)
+		__field(xfs_filblks_t, len)
+		__field(xfs_fsblock_t, pblk)
+		__field(int, state)
+	),
+	TP_fast_assign(
+		__entry->dev = VFS_I(ip)->i_sb->s_dev;
+		__entry->ino = ip->i_ino;
+		__entry->whichfork = whichfork;
+		__entry->lblk = irec->br_startoff;
+		__entry->len = irec->br_blockcount;
+		__entry->pblk = irec->br_startblock;
+		__entry->state = irec->br_state;
+	),
+	TP_printk("dev %d:%d ino 0x%llx whichfork %s fileoff 0x%llx fsbcount 0x%llx startblock 0x%llx state %d",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino,
+		  __print_symbolic(__entry->whichfork, XFS_WHICHFORK_STRINGS),
+		  __entry->lblk,
+		  __entry->len,
+		  __entry->pblk,
+		  __entry->state)
+);
+
 TRACE_EVENT(xrep_findroot_block,
 	TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, xfs_agblock_t agbno,
 		 uint32_t magic, uint16_t level),
fs/xfs/xfs_trans.c +95 -0

107 lines not on this page · raw message

fs/xfs/xfs_trans.h +4 -0
index 6e3646d524ceb..d32abdd1e0149 100644
@@ -168,6 +168,8 @@ typedef struct xfs_trans {
 int		xfs_trans_alloc(struct xfs_mount *mp, struct xfs_trans_res *resp,
 			uint blocks, uint rtextents, uint flags,
 			struct xfs_trans **tpp);
+int		xfs_trans_reserve_more(struct xfs_trans *tp,
+			unsigned int blocks, unsigned int rtextents);
 int		xfs_trans_alloc_empty(struct xfs_mount *mp,
 			struct xfs_trans **tpp);
 void		xfs_trans_mod_sb(xfs_trans_t *, uint, int64_t);
@@ -260,6 +262,8 @@ struct xfs_dquot;
 int xfs_trans_alloc_inode(struct xfs_inode *ip, struct xfs_trans_res *resv,
 		unsigned int dblocks, unsigned int rblocks, bool force,
 		struct xfs_trans **tpp);
+int xfs_trans_reserve_more_inode(struct xfs_trans *tp, struct xfs_inode *ip,
+		unsigned int dblocks, unsigned int rblocks, bool force_quota);
 int xfs_trans_alloc_icreate(struct xfs_mount *mp, struct xfs_trans_res *resv,
 		struct xfs_dquot *udqp, struct xfs_dquot *gdqp,
 		struct xfs_dquot *pdqp, unsigned int dblocks,
#102 lore raw

Re: [PATCH 2/5] xfs: repair inode fork block mapping data structures

fs/xfs/scrub/bmap_repair.c line 302

Quoted 10 lines · Darrick J. Wong
fs/xfs/scrub/bmap_repair.c new file +10 -0
+static int
+xrep_bmap_extent_cmp(
+	const void			*a,
+	const void			*b)
+{
+	xfs_fileoff_t			ao;
+	xfs_fileoff_t			bo;
+
+	ao = xfs_bmbt_disk_get_startoff((struct xfs_bmbt_rec *)a);
+	bo = xfs_bmbt_disk_get_startoff((struct xfs_bmbt_rec *)b);

It would be nice if we could just have local variables for the xfs_bmbt_recs and not need casts. I guess for that xfs_bmbt_disk_get_startoff would need to take a const argument?

Probably something for later.

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (whichfork == XFS_ATTR_FORK)
+		return 0;

Nit: I'd probably just split the data fork specific validation into a separate helper to keep things nicely organized.

fs/xfs/scrub/bmap_repair.c line 793

Quoted 6 lines
fs/xfs/scrub/bmap_repair.c new file +6 -0
+	/*
+	 * No need to waste time scanning for shared extents if the inode is
+	 * already marked.
+	 */
+	if (whichfork == XFS_DATA_FORK && xfs_is_reflink_inode(sc->ip))
+		rb->shared_extents = true;

The comment doesn't seem to match the code.

Quoted 4 lines
fs/xfs/xfs_trans.c +4 -0
+/*
+ * Try to reserve more blocks for a transaction.
+ *
+ * This is for callers that need to attach resources to a transaction, scan

fs/xfs/xfs_trans.c xfs_trans_dup( · line 148

Quoted 10 lines
fs/xfs/xfs_trans.c +10 -0
+ * those resources to determine the space reservation requirements, and then
+ * modify the attached resources.  In other words, online repair.  This can
+ * fail due to ENOSPC, so the caller must be able to cancel the transaction
+ * without shutting down the fs.
+ */
+int
+xfs_trans_reserve_more(
+	struct xfs_trans	*tp,
+	unsigned int		blocks,
+	unsigned int		rtextents)

This basically seems to duplicate xfs_trans_reserve except that it skips the log reservation. What about just allowing to pass a NULL resp agument to xfs_trans_reserve to skip the log reservation and reuse the code?

Otherwise this looks good to me.

#103 lore raw

Re: [PATCH 2/5] xfs: repair inode fork block mapping data structures

Quoted 16 lines · Christoph Hellwig

fs/xfs/scrub/bmap_repair.c line 302

Quoted 10 lines · Darrick J. Wong
fs/xfs/scrub/bmap_repair.c new file +10 -0
+static int
+xrep_bmap_extent_cmp(
+	const void			*a,
+	const void			*b)
+{
+	xfs_fileoff_t			ao;
+	xfs_fileoff_t			bo;
+
+	ao = xfs_bmbt_disk_get_startoff((struct xfs_bmbt_rec *)a);
+	bo = xfs_bmbt_disk_get_startoff((struct xfs_bmbt_rec *)b);

It would be nice if we could just have local variables for the xfs_bmbt_recs and not need casts. I guess for that xfs_bmbt_disk_get_startoff would need to take a const argument?

Quoted 1 line

Probably something for later.

Oh! Apparently xfs_bmbt_disk_get_startoff does take a const struct pointer now.

Quoted 6 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (whichfork == XFS_ATTR_FORK)
+		return 0;

Nit: I'd probably just split the data fork specific validation into a separate helper to keep things nicely organized.

Quoted 8 lines

fs/xfs/scrub/bmap_repair.c line 793

Quoted 6 lines
fs/xfs/scrub/bmap_repair.c new file +6 -0
+	/*
+	 * No need to waste time scanning for shared extents if the inode is
+	 * already marked.
+	 */
+	if (whichfork == XFS_DATA_FORK && xfs_is_reflink_inode(sc->ip))
+		rb->shared_extents = true;

The comment doesn't seem to match the code.

Ooof, that state handling needs tightening up. There are three states, really -- "irrelevant to this repair", "set the iflag", and "no idea, do a scan". That first state is open-coded in _discover_shared, which is wasteful because that is decidable in xrep_bmap.

Quoted 12 lines
Quoted 12 lines
fs/xfs/xfs_trans.c +11 -0
+/*
+ * Try to reserve more blocks for a transaction.
+ *
+ * This is for callers that need to attach resources to a transaction, scan
+ * those resources to determine the space reservation requirements, and then
+ * modify the attached resources.  In other words, online repair.  This can
+ * fail due to ENOSPC, so the caller must be able to cancel the transaction
+ * without shutting down the fs.
+ */
+int
+xfs_trans_reserve_more(
Quoted 8 lines

fs/xfs/xfs_trans.c xfs_trans_dup( · line 148

Quoted 3 lines
fs/xfs/xfs_trans.c +3 -0
+	struct xfs_trans	*tp,
+	unsigned int		blocks,
+	unsigned int		rtextents)

This basically seems to duplicate xfs_trans_reserve except that it skips the log reservation. What about just allowing to pass a NULL resp agument to xfs_trans_reserve to skip the log reservation and reuse the code?

Hmm. Maybe not a NULL resp, but an empty one looks like it would work fine with less code duplication.

Quoted 1 line

Otherwise this looks good to me.

Cool! Thanks for reviewing!

--D

#104 lore raw

PATCH 3/5 xfs: refactor repair forcing tests into a repair.c helper

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

There are a couple of conditions that userspace can set to force repairs of metadata. These really belong in the repair code and not open-coded into the check code, so refactor them into a helper.

---
 fs/xfs/scrub/repair.c |   22 ++++++++++++++++++++++
 fs/xfs/scrub/repair.h |    2 ++
 fs/xfs/scrub/scrub.c  |   14 +-------------
 3 files changed, 25 insertions(+), 13 deletions(-)
fs/xfs/scrub/repair.c +22 -0
index 4d5bfb2e4cf08..0f8dc25ef998b 100644
@@ -27,6 +27,8 @@
 #include "xfs_quota.h"
 #include "xfs_qm.h"
 #include "xfs_defer.h"
+#include "xfs_errortag.h"
+#include "xfs_error.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 #include "scrub/trace.h"
@@ -937,3 +939,23 @@ xrep_reset_perag_resv(
 out:
 	return error;
 }
+
+/* Decide if we are going to call the repair function for a scrub type. */
+bool
+xrep_will_attempt(
+	struct xfs_scrub	*sc)
+{
+	/* Userspace asked us to rebuild the structure regardless. */
+	if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_FORCE_REBUILD)
+		return true;
+
+	/* Let debug users force us into the repair routines. */
+	if (XFS_TEST_ERROR(false, sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR))
+		return true;
+
+	/* Metadata is corrupt or failed cross-referencing. */
+	if (xchk_needs_repair(sc->sm))
+		return true;
+
+	return false;
+}
fs/xfs/scrub/repair.h +2 -0
index 9f0c77b38ae28..73ac3eca1a781 100644
@@ -28,6 +28,7 @@ static inline int xrep_notsupported(struct xfs_scrub *sc)
 /* Repair helpers */
 
 int xrep_attempt(struct xfs_scrub *sc, struct xchk_stats_run *run);
+bool xrep_will_attempt(struct xfs_scrub *sc);
 void xrep_failure(struct xfs_mount *mp);
 int xrep_roll_ag_trans(struct xfs_scrub *sc);
 int xrep_roll_trans(struct xfs_scrub *sc);
@@ -117,6 +118,7 @@ int xrep_reinit_pagi(struct xfs_scrub *sc);
 #else
 
 #define xrep_ino_dqattach(sc)	(0)
+#define xrep_will_attempt(sc)	(false)
 
 static inline int
 xrep_attempt(
fs/xfs/scrub/scrub.c +1 -13
index 52a09e0652693..8397d1dce25fa 100644
@@ -14,8 +14,6 @@
 #include "xfs_inode.h"
 #include "xfs_quota.h"
 #include "xfs_qm.h"
-#include "xfs_errortag.h"
-#include "xfs_error.h"
 #include "xfs_scrub.h"
 #include "xfs_btree.h"
 #include "xfs_btree_staging.h"
@@ -552,21 +550,11 @@ xfs_scrub_metadata(
 	xchk_update_health(sc);
 
 	if (xchk_could_repair(sc)) {
-		bool needs_fix = xchk_needs_repair(sc->sm);
-
-		/* Userspace asked us to rebuild the structure regardless. */
-		if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_FORCE_REBUILD)
-			needs_fix = true;
-
-		/* Let debug users force us into the repair routines. */
-		if (XFS_TEST_ERROR(needs_fix, mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR))
-			needs_fix = true;
-
 		/*
 		 * If userspace asked for a repair but it wasn't necessary,
 		 * report that back to userspace.
 		 */
-		if (!needs_fix) {
+		if (!xrep_will_attempt(sc)) {
 			sc->sm->sm_flags |= XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED;
 			goto out_nofix;
 		}
#105 lore raw

Re: [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper

Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

There are a couple of conditions that userspace can set to force repairs of metadata. These really belong in the repair code and not open-coded into the check code, so refactor them into a helper.

Just ramblings from someone who is trying to get into the scrub and repair code:

I find this code organization where the check helpers are in foo.c, repair helpers in foo_repair.c and then both are used in scrub.c to fill out ops really annoying to follow. My normal taste would expect a single file that has all the methods, and which then registers the ops vector. But it's probably too late for that now..

#106 lore raw

Re: [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper

Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

There are a couple of conditions that userspace can set to force repairs of metadata. These really belong in the repair code and not open-coded into the check code, so refactor them into a helper.

Quoted 8 lines

Just ramblings from someone who is trying to get into the scrub and repair code:

I find this code organization where the check helpers are in foo.c, repair helpers in foo_repair.c and then both are used in scrub.c to fill out ops really annoying to follow. My normal taste would expect a single file that has all the methods, and which then registers the ops vector. But it's probably too late for that now..

Not really, in theory I could respin the whole series to move FOO_repair.c into FOO.c surrounded by a giant #ifdef CONFIG_XFS_ONLINE_REPAIR block; and change agheader_repair.c.

OTOH I thought it was cleaner to elide the repair code via Makefiles instead of preprocessor directives that we could get lost in.

Longer term I've struggled with whether or not (for example) the alloc.c/alloc_repair.c declarations should go in alloc.h. That's cleaner IMHO but explodes the number of files for usually not that much gain.

--D

#107 lore raw

Re: [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper

Quoted 10 lines · Darrick J. Wong
Quoted 5 lines

I find this code organization where the check helpers are in foo.c, repair helpers in foo_repair.c and then both are used in scrub.c to fill out ops really annoying to follow. My normal taste would expect a single file that has all the methods, and which then registers the ops vector. But it's probably too late for that now..

Not really, in theory I could respin the whole series to move FOO_repair.c into FOO.c surrounded by a giant #ifdef CONFIG_XFS_ONLINE_REPAIR block; and change agheader_repair.c.

Quoted 7 lines

OTOH I thought it was cleaner to elide the repair code via Makefiles instead of preprocessor directives that we could get lost in.

Longer term I've struggled with whether or not (for example) the alloc.c/alloc_repair.c declarations should go in alloc.h. That's cleaner IMHO but explodes the number of files for usually not that much gain.

Heh, and I wondered if the check/repair code should just live with the code implenenting the functionality it is checking/repairing so things can be kept nicely static. I really do not have a good answer here, I just noticed that it requires quite a lot of cycling through files to understand the repair code.

#108 lore raw

PATCH 4/5 xfs: create a ranged query function for refcount btrees

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Implement ranged queries for refcount records. The next patch will use this to scan refcount data.

---
 fs/xfs/libxfs/xfs_refcount.c |   41 +++++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_refcount.h |   10 ++++++++++
 2 files changed, 51 insertions(+)
fs/xfs/libxfs/xfs_refcount.c +41 -0
index 5fa1a6f32c17d..b7311122bd489 100644
@@ -2039,6 +2039,47 @@ xfs_refcount_has_records(
 	return xfs_btree_has_records(cur, &low, &high, NULL, outcome);
 }
 
+struct xfs_refcount_query_range_info {
+	xfs_refcount_query_range_fn	fn;
+	void				*priv;
+};
+
+/* Format btree record and pass to our callback. */
+STATIC int
+xfs_refcount_query_range_helper(
+	struct xfs_btree_cur		*cur,
+	const union xfs_btree_rec	*rec,
+	void				*priv)
+{
+	struct xfs_refcount_query_range_info	*query = priv;
+	struct xfs_refcount_irec	irec;
+	xfs_failaddr_t			fa;
+
+	xfs_refcount_btrec_to_irec(rec, &irec);
+	fa = xfs_refcount_check_irec(cur, &irec);
+	if (fa)
+		return xfs_refcount_complain_bad_rec(cur, fa, &irec);
+
+	return query->fn(cur, &irec, query->priv);
+}
+
+/* Find all refcount records between two keys. */
+int
+xfs_refcount_query_range(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_refcount_irec	*low_rec,
+	const struct xfs_refcount_irec	*high_rec,
+	xfs_refcount_query_range_fn	fn,
+	void				*priv)
+{
+	union xfs_btree_irec		low_brec = { .rc = *low_rec };
+	union xfs_btree_irec		high_brec = { .rc = *high_rec };
+	struct xfs_refcount_query_range_info query = { .priv = priv, .fn = fn };
+
+	return xfs_btree_query_range(cur, &low_brec, &high_brec,
+			xfs_refcount_query_range_helper, &query);
+}
+
 int __init
 xfs_refcount_intent_init_cache(void)
 {
fs/xfs/libxfs/xfs_refcount.h +10 -0
index 2d6fecb258bb1..9563eb91be172 100644
@@ -129,4 +129,14 @@ extern struct kmem_cache	*xfs_refcount_intent_cache;
 int __init xfs_refcount_intent_init_cache(void);
 void xfs_refcount_intent_destroy_cache(void);
 
+typedef int (*xfs_refcount_query_range_fn)(
+	struct xfs_btree_cur		*cur,
+	const struct xfs_refcount_irec	*rec,
+	void				*priv);
+
+int xfs_refcount_query_range(struct xfs_btree_cur *cur,
+		const struct xfs_refcount_irec *low_rec,
+		const struct xfs_refcount_irec *high_rec,
+		xfs_refcount_query_range_fn fn, void *priv);
+
 #endif	/* __XFS_REFCOUNT_H__ */
#110 lore raw

PATCH 5/5 xfs: repair problems in CoW forks

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Try to repair errors that we see in file CoW forks so that we don't do stupid things like remap garbage into a file. There's not a lot we can do with the COW fork -- the ondisk metadata record only that the COW staging extents are owned by the refcount btree, which effectively means that we can't reconstruct this incore structure from scratch.

Actually, this is even worse -- we can't touch written extents, because those map space that are actively under writeback, and there's not much to do with delalloc reservations. Hence we can only detect crosslinked unwritten extents and fix them by punching out the problematic parts and replacing them with delalloc extents.

---
 fs/xfs/Makefile           |    1 
 fs/xfs/scrub/bitmap.h     |   28 ++
 fs/xfs/scrub/cow_repair.c |  612 +++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/reap.c       |   32 ++
 fs/xfs/scrub/repair.h     |    2 
 fs/xfs/scrub/scrub.c      |    2 
 fs/xfs/scrub/trace.h      |   84 ++++++
 7 files changed, 760 insertions(+), 1 deletion(-)
 create mode 100644 fs/xfs/scrub/cow_repair.c
fs/xfs/Makefile +1 -0
index f62351d63b147..71a76f8ac5e47 100644
@@ -183,6 +183,7 @@ xfs-y				+= $(addprefix scrub/, \
 				   agheader_repair.o \
 				   alloc_repair.o \
 				   bmap_repair.o \
+				   cow_repair.o \
 				   ialloc_repair.o \
 				   inode_repair.o \
 				   newbt.o \
fs/xfs/scrub/bitmap.h +28 -0
index 1356a76710ede..e470215232ef0 100644
@@ -149,4 +149,32 @@ static inline int xfsb_bitmap_walk(struct xfsb_bitmap *bitmap,
 	return xbitmap_walk(&bitmap->fsbitmap, fn, priv);
 }
 
+/* Bitmaps, but for type-checked for xfs_fileoff_t */
+
+struct xoff_bitmap {
+	struct xbitmap	offbitmap;
+};
+
+static inline void xoff_bitmap_init(struct xoff_bitmap *bitmap)
+{
+	xbitmap_init(&bitmap->offbitmap);
+}
+
+static inline void xoff_bitmap_destroy(struct xoff_bitmap *bitmap)
+{
+	xbitmap_destroy(&bitmap->offbitmap);
+}
+
+static inline int xoff_bitmap_set(struct xoff_bitmap *bitmap,
+		xfs_fileoff_t off, xfs_filblks_t len)
+{
+	return xbitmap_set(&bitmap->offbitmap, off, len);
+}
+
+static inline int xoff_bitmap_walk(struct xoff_bitmap *bitmap,
+		xbitmap_walk_fn fn, void *priv)
+{
+	return xbitmap_walk(&bitmap->offbitmap, fn, priv);
+}
+
 #endif	/* __XFS_SCRUB_BITMAP_H__ */
fs/xfs/scrub/cow_repair.c new file +612 -0

612 lines not on this page · raw message

fs/xfs/scrub/reap.c +32 -0
index 35794df952bbe..1305e82e3df13 100644
@@ -20,6 +20,7 @@
 #include "xfs_ialloc_btree.h"
 #include "xfs_rmap.h"
 #include "xfs_rmap_btree.h"
+#include "xfs_refcount.h"
 #include "xfs_refcount_btree.h"
 #include "xfs_extent_busy.h"
 #include "xfs_ag.h"
@@ -378,6 +379,17 @@ xreap_agextent_iter(
 		trace_xreap_dispose_unmap_extent(sc->sa.pag, agbno, *aglenp);
 
 		rs->force_roll = true;
+
+		if (rs->oinfo == &XFS_RMAP_OINFO_COW) {
+			/*
+			 * If we're unmapping CoW staging extents, remove the
+			 * records from the refcountbt, which will remove the
+			 * rmap record as well.
+			 */
+			xfs_refcount_free_cow_extent(sc->tp, fsbno, *aglenp);
+			return 0;
+		}
+
 		return xfs_rmap_free(sc->tp, sc->sa.agf_bp, sc->sa.pag, agbno,
 				*aglenp, rs->oinfo);
 	}
@@ -396,6 +408,26 @@ xreap_agextent_iter(
 		return 0;
 	}
 
+	/*
+	 * If we're getting rid of CoW staging extents, use deferred work items
+	 * to remove the refcountbt records (which removes the rmap records)
+	 * and free the extent.  We're not worried about the system going down
+	 * here because log recovery walks the refcount btree to clean out the
+	 * CoW staging extents.
+	 */
+	if (rs->oinfo == &XFS_RMAP_OINFO_COW) {
+		ASSERT(rs->resv == XFS_AG_RESV_NONE);
+
+		xfs_refcount_free_cow_extent(sc->tp, fsbno, *aglenp);
+		error = xfs_free_extent_later(sc->tp, fsbno, *aglenp, NULL,
+				rs->resv, true);
+		if (error)
+			return error;
+
+		rs->force_roll = true;
+		return 0;
+	}
+
 	/* Put blocks back on the AGFL one at a time. */
 	if (rs->resv == XFS_AG_RESV_AGFL) {
 		ASSERT(*aglenp == 1);
fs/xfs/scrub/repair.h +2 -0
index 73ac3eca1a781..be3585b8f4364 100644
@@ -111,6 +111,7 @@ int xrep_refcountbt(struct xfs_scrub *sc);
 int xrep_inode(struct xfs_scrub *sc);
 int xrep_bmap_data(struct xfs_scrub *sc);
 int xrep_bmap_attr(struct xfs_scrub *sc);
+int xrep_bmap_cow(struct xfs_scrub *sc);
 
 int xrep_reinit_pagf(struct xfs_scrub *sc);
 int xrep_reinit_pagi(struct xfs_scrub *sc);
@@ -173,6 +174,7 @@ xrep_setup_nothing(
 #define xrep_inode			xrep_notsupported
 #define xrep_bmap_data			xrep_notsupported
 #define xrep_bmap_attr			xrep_notsupported
+#define xrep_bmap_cow			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +1 -1
index 8397d1dce25fa..bc70a91f8b1bf 100644
@@ -300,7 +300,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_INODE,
 		.setup	= xchk_setup_inode_bmap,
 		.scrub	= xchk_bmap_cow,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_bmap_cow,
 	},
 	[XFS_SCRUB_TYPE_DIR] = {	/* directory */
 		.type	= ST_INODE,
fs/xfs/scrub/trace.h +84 -0

90 lines not on this page · raw message

#112 lore raw

cover xfs: online repair of rt bitmap file

In reply to Darrick J. Wong

Hi all,

Add in the necessary infrastructure to check the inode and data forks of metadata files, then apply that to the realtime bitmap file. We won't be able to reconstruct the contents of the rtbitmap file until rmapbt is added for realtime volumes, but we can at least get the basics started.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-rtbitmap

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-rtbitmap

---
 fs/xfs/Makefile                |    4 +
 fs/xfs/libxfs/xfs_bmap.c       |   39 ++++++++
 fs/xfs/libxfs/xfs_bmap.h       |    2 
 fs/xfs/scrub/bmap_repair.c     |   17 +++
 fs/xfs/scrub/repair.c          |  151 ++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h          |    9 ++
 fs/xfs/scrub/rtbitmap.c        |  101 +++++++++++++++++---
 fs/xfs/scrub/rtbitmap.h        |   22 ++++
 fs/xfs/scrub/rtbitmap_repair.c |  202 ++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/rtsummary.c       |  132 +++++++++++++++++++++-----
 fs/xfs/scrub/scrub.c           |    4 -
 fs/xfs/xfs_inode.c             |   24 +----
 12 files changed, 637 insertions(+), 70 deletions(-)
 create mode 100644 fs/xfs/scrub/rtbitmap.h
 create mode 100644 fs/xfs/scrub/rtbitmap_repair.c
#113 lore raw

PATCH 1/6 xfs: check rt bitmap file geometry more thoroughly

From: Darrick J. Wong <[email protected]>

I forgot that the superblock tracks the number of blocks that are in the realtime bitmap, and that the rt bitmap file can have more blocks mapped to the data fork than sb_rbmblocks if growfsrt fails.

So. Add to the rtbitmap scrubber an explicit check that sb_rextents and sb_rbmblocks are correct, then adjust the rtbitmap i_size checks to allow for the growfsrt failure case. Finally, flag post-eof blocks in the rtbitmap.

---
 fs/xfs/scrub/rtbitmap.c |   97 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 82 insertions(+), 15 deletions(-)
fs/xfs/scrub/rtbitmap.c +82 -15
index d509a08d3fc3e..3b5b62fbf4e0a 100644
@@ -14,16 +14,30 @@
 #include "xfs_rtbitmap.h"
 #include "xfs_inode.h"
 #include "xfs_bmap.h"
+#include "xfs_bit.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
 
+struct xchk_rtbitmap {
+	uint64_t		rextents;
+	uint64_t		rbmblocks;
+	unsigned int		rextslog;
+};
+
 /* Set us up with the realtime metadata locked. */
 int
 xchk_setup_rtbitmap(
 	struct xfs_scrub	*sc)
 {
+	struct xfs_mount	*mp = sc->mp;
+	struct xchk_rtbitmap	*rtb;
 	int			error;
 
+	rtb = kzalloc(sizeof(struct xchk_rtbitmap), XCHK_GFP_FLAGS);
+	if (!rtb)
+		return -ENOMEM;
+	sc->buf = rtb;
+
 	error = xchk_trans_alloc(sc, 0);
 	if (error)
 		return error;
@@ -37,6 +51,15 @@ xchk_setup_rtbitmap(
 		return error;
 
 	xchk_ilock(sc, XFS_ILOCK_EXCL | XFS_ILOCK_RTBITMAP);
+
+	/*
+	 * Now that we've locked the rtbitmap, we can't race with growfsrt
+	 * trying to expand the bitmap or change the size of the rt volume.
+	 * Hence it is safe to compute and check the geometry values.
+	 */
+	rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rtb->rextslog = rtb->rextents ? xfs_highbit32(rtb->rextents) : 0;
+	rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents);
 	return 0;
 }
 
@@ -67,21 +90,30 @@ STATIC int
 xchk_rtbitmap_check_extents(
 	struct xfs_scrub	*sc)
 {
-	struct xfs_mount	*mp = sc->mp;
 	struct xfs_bmbt_irec	map;
-	xfs_rtblock_t		off;
-	int			nmap;
+	struct xfs_iext_cursor	icur;
+	struct xfs_mount	*mp = sc->mp;
+	struct xfs_inode	*ip = sc->ip;
+	xfs_fileoff_t		off = 0;
+	xfs_fileoff_t		endoff;
 	int			error = 0;
 
-	for (off = 0; off < mp->m_sb.sb_rbmblocks;) {
+	/* Mappings may not cross or lie beyond EOF. */
+	endoff = XFS_B_TO_FSB(mp, ip->i_disk_size);
+	if (xfs_iext_lookup_extent(ip, &ip->i_df, endoff, &icur, &map)) {
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, endoff);
+		return 0;
+	}
+
+	while (off < endoff) {
+		int		nmap = 1;
+
 		if (xchk_should_terminate(sc, &error) ||
 		    (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT))
 			break;
 
 		/* Make sure we have a written extent. */
-		nmap = 1;
-		error = xfs_bmapi_read(mp->m_rbmip, off,
-				mp->m_sb.sb_rbmblocks - off, &map, &nmap,
+		error = xfs_bmapi_read(ip, off, endoff - off, &map, &nmap,
 				XFS_DATA_FORK);
 		if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, off, &error))
 			break;
@@ -102,12 +134,48 @@ int
 xchk_rtbitmap(
 	struct xfs_scrub	*sc)
 {
+	struct xfs_mount	*mp = sc->mp;
+	struct xchk_rtbitmap	*rtb = sc->buf;
 	int			error;
 
-	/* Is the size of the rtbitmap correct? */
-	if (sc->mp->m_rbmip->i_disk_size !=
-	    XFS_FSB_TO_B(sc->mp, sc->mp->m_sb.sb_rbmblocks)) {
-		xchk_ino_set_corrupt(sc, sc->mp->m_rbmip->i_ino);
+	/* Is sb_rextents correct? */
+	if (mp->m_sb.sb_rextents != rtb->rextents) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
+		return 0;
+	}
+
+	/* Is sb_rextslog correct? */
+	if (mp->m_sb.sb_rextslog != rtb->rextslog) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
+		return 0;
+	}
+
+	/*
+	 * Is sb_rbmblocks large enough to handle the current rt volume?  In no
+	 * case can we exceed 4bn bitmap blocks since the super field is a u32.
+	 */
+	if (rtb->rbmblocks > U32_MAX) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
+		return 0;
+	}
+	if (mp->m_sb.sb_rbmblocks != rtb->rbmblocks) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
+		return 0;
+	}
+
+	/* The bitmap file length must be aligned to an fsblock. */
+	if (mp->m_rbmip->i_disk_size & mp->m_blockmask) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
+		return 0;
+	}
+
+	/*
+	 * Is the bitmap file itself large enough to handle the rt volume?
+	 * growfsrt expands the bitmap file before updating sb_rextents, so the
+	 * file can be larger than sb_rbmblocks.
+	 */
+	if (mp->m_rbmip->i_disk_size < XFS_FSB_TO_B(mp, rtb->rbmblocks)) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
 		return 0;
 	}
 
@@ -120,12 +188,11 @@ xchk_rtbitmap(
 	if (error || (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT))
 		return error;
 
-	error = xfs_rtalloc_query_all(sc->mp, sc->tp, xchk_rtbitmap_rec, sc);
+	error = xfs_rtalloc_query_all(mp, sc->tp, xchk_rtbitmap_rec, sc);
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, 0, &error))
-		goto out;
+		return error;
 
-out:
-	return error;
+	return 0;
 }
 
 /* xref check that the extent is not free in the rtbitmap */
Reviewed-by on v1 1/6
#114 lore raw

Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly

fs/xfs/scrub/rtbitmap.c xchk_setup_rtbitmap( · line 62

Quoted 8 lines
fs/xfs/scrub/rtbitmap.c +8 -0
+	/*
+	 * Now that we've locked the rtbitmap, we can't race with growfsrt
+	 * trying to expand the bitmap or change the size of the rt volume.
+	 * Hence it is safe to compute and check the geometry values.
+	 */
+	rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rtb->rextslog = rtb->rextents ? xfs_highbit32(rtb->rextents) : 0;
+	rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents);

All these will be 0 if mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:

	if (mp->m_sb.sb_rblocks) {
		rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
		rtb->rextslog = xfs_highbit32(rtb->rextents);
		rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents);
	}

?

Otherwise looks good:

#115 lore raw

Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly

Quoted 12 lines · Christoph Hellwig

fs/xfs/scrub/rtbitmap.c xchk_setup_rtbitmap( · line 62

Quoted 8 lines
fs/xfs/scrub/rtbitmap.c +8 -0
+	/*
+	 * Now that we've locked the rtbitmap, we can't race with growfsrt
+	 * trying to expand the bitmap or change the size of the rt volume.
+	 * Hence it is safe to compute and check the geometry values.
+	 */
+	rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rtb->rextslog = rtb->rextents ? xfs_highbit32(rtb->rextents) : 0;
+	rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents);

All these will be 0 if mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:

Quoted 3 lines
	if (mp->m_sb.sb_rblocks) {
		rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
		rtb->rextslog = xfs_highbit32(rtb->rextents);

Well... xfs_highbit32 returns -1 if its argument is zero, which is possible for the nasty edge case of (say) a 64k block device and a realtime extent size of 1MB, which results in rblocks > 0 and rextents == 0.

So I'll still have to do:

		if (rtb->rextents)
			rtb->rextslog = xfs_highbit32()

but otherwise this is fine.

Quoted 8 lines
		rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents);
	}

?

Otherwise looks good:

Thank you!

--D

#116 lore raw

Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly

Quoted 11 lines · Darrick J. Wong
Quoted 6 lines

All these will be 0 if mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:

	if (mp->m_sb.sb_rblocks) {
		rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
		rtb->rextslog = xfs_highbit32(rtb->rextents);

Well... xfs_highbit32 returns -1 if its argument is zero, which is possible for the nasty edge case of (say) a 64k block device and a realtime extent size of 1MB, which results in rblocks > 0 and rextents == 0.

Eww. How do we even allow creating a mounting that? Such a configuration doesn't make any sense.

#117 lore raw

Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly

Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines

All these will be 0 if mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:

	if (mp->m_sb.sb_rblocks) {
		rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
		rtb->rextslog = xfs_highbit32(rtb->rextents);
Quoted 7 lines
Quoted 4 lines

Well... xfs_highbit32 returns -1 if its argument is zero, which is possible for the nasty edge case of (say) a 64k block device and a realtime extent size of 1MB, which results in rblocks > 0 and rextents == 0.

Eww. How do we even allow creating a mounting that? Such a configuration doesn't make any sense.

$ truncate -s 64k /tmp/realtime
$ truncate -s 1g /tmp/data
$ mkfs.xfs -f /tmp/data -r rtdev=/tmp/realtime,extsize=1m

Pre 4.19 mkfs.xfs would actually write out the fs and pre-4.19 kernels would mount it (and ENOSPC). Since then, due to buggy sb validation code on my part now it just fails verifiers and crashes/doesn't mount.

--D

#118 lore raw

PATCH 2/6 xfs: check rt summary file geometry more thoroughly

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

I forgot that the xfs_mount tracks the size and number of levels in the realtime summary file, and that the rt summary file can have more blocks mapped to the data fork than m_rsumsize implies if growfsrt fails.

So. Add to the rtsummary scrubber an explicit check that all the summary geometry values are correct, then adjust the rtsummary i_size checks to allow for the growfsrt failure case. Finally, flag post-eof blocks in the summary file.

While we're at it, split the extent map checking so that we only call xfs_bmapi_read once per extent instead of once per rtsummary block.

---
 fs/xfs/scrub/rtsummary.c |  132 +++++++++++++++++++++++++++++++++++++---------
 1 file changed, 105 insertions(+), 27 deletions(-)
fs/xfs/scrub/rtsummary.c +105 -27
index f94800a029f35..41f64158c8626 100644
@@ -31,6 +31,18 @@
  * (potentially large) amount of data in pageable memory.
  */
 
+struct xchk_rtsummary {
+	struct xfs_rtalloc_args	args;
+
+	uint64_t		rextents;
+	uint64_t		rbmblocks;
+	uint64_t		rsumsize;
+	unsigned int		rsumlevels;
+
+	/* Memory buffer for the summary comparison. */
+	union xfs_suminfo_raw	words[];
+};
+
 /* Set us up to check the rtsummary file. */
 int
 xchk_setup_rtsummary(
@@ -38,8 +50,16 @@ xchk_setup_rtsummary(
 {
 	struct xfs_mount	*mp = sc->mp;
 	char			*descr;
+	struct xchk_rtsummary	*rts;
+	xfs_filblks_t		rsumblocks;
 	int			error;
 
+	rts = kvzalloc(struct_size(rts, words, mp->m_blockwsize),
+			XCHK_GFP_FLAGS);
+	if (!rts)
+		return -ENOMEM;
+	sc->buf = rts;
+
 	/*
 	 * Create an xfile to construct a new rtsummary file.  The xfile allows
 	 * us to avoid pinning kernel memory for this purpose.
@@ -54,11 +74,6 @@ xchk_setup_rtsummary(
 	if (error)
 		return error;
 
-	/* Allocate a memory buffer for the summary comparison. */
-	sc->buf = kvmalloc(mp->m_sb.sb_blocksize, XCHK_GFP_FLAGS);
-	if (!sc->buf)
-		return -ENOMEM;
-
 	error = xchk_install_live_inode(sc, mp->m_rsumip);
 	if (error)
 		return error;
@@ -75,13 +90,23 @@ xchk_setup_rtsummary(
 	 */
 	xfs_ilock(mp->m_rbmip, XFS_ILOCK_SHARED | XFS_ILOCK_RTBITMAP);
 	xchk_ilock(sc, XFS_ILOCK_EXCL | XFS_ILOCK_RTSUM);
+
+	/*
+	 * Now that we've locked the rtbitmap and rtsummary, we can't race with
+	 * growfsrt trying to expand the summary or change the size of the rt
+	 * volume.  Hence it is safe to compute and check the geometry values.
+	 */
+	rts->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rts->rbmblocks = xfs_rtbitmap_blockcount(mp, rts->rextents);
+	rts->rsumlevels = rts->rextents ? xfs_highbit32(rts->rextents) + 1 : 0;
+	rsumblocks = xfs_rtsummary_blockcount(mp, rts->rsumlevels,
+			rts->rbmblocks);
+	rts->rsumsize = XFS_FSB_TO_B(mp, rsumblocks);
 	return 0;
 }
 
 /* Helper functions to record suminfo words in an xfile. */
 
-typedef unsigned int xchk_rtsumoff_t;
-
 static inline int
 xfsum_load(
 	struct xfs_scrub	*sc,
@@ -192,19 +217,29 @@ STATIC int
 xchk_rtsum_compare(
 	struct xfs_scrub	*sc)
 {
-	struct xfs_rtalloc_args args = {
-		.mp		= sc->mp,
-		.tp		= sc->tp,
-	};
-	struct xfs_mount	*mp = sc->mp;
 	struct xfs_bmbt_irec	map;
-	xfs_fileoff_t		off;
-	xchk_rtsumoff_t		sumoff = 0;
-	int			nmap;
+	struct xfs_iext_cursor	icur;
 
-	for (off = 0; off < XFS_B_TO_FSB(mp, mp->m_rsumsize); off++) {
-		union xfs_suminfo_raw *ondisk_info;
-		int		error = 0;
+	struct xfs_mount	*mp = sc->mp;
+	struct xfs_inode	*ip = sc->ip;
+	struct xchk_rtsummary	*rts = sc->buf;
+	xfs_fileoff_t		off = 0;
+	xfs_fileoff_t		endoff;
+	xfs_rtsumoff_t		sumoff = 0;
+	int			error = 0;
+
+	rts->args.mp = sc->mp;
+	rts->args.tp = sc->tp;
+
+	/* Mappings may not cross or lie beyond EOF. */
+	endoff = XFS_B_TO_FSB(mp, ip->i_disk_size);
+	if (xfs_iext_lookup_extent(ip, &ip->i_df, endoff, &icur, &map)) {
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, endoff);
+		return 0;
+	}
+
+	while (off < endoff) {
+		int		nmap = 1;
 
 		if (xchk_should_terminate(sc, &error))
 			return error;
@@ -212,8 +247,7 @@ xchk_rtsum_compare(
 			return 0;
 
 		/* Make sure we have a written extent. */
-		nmap = 1;
-		error = xfs_bmapi_read(mp->m_rsumip, off, 1, &map, &nmap,
+		error = xfs_bmapi_read(ip, off, endoff - off, &map, &nmap,
 				XFS_DATA_FORK);
 		if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, off, &error))
 			return error;
@@ -223,24 +257,33 @@ xchk_rtsum_compare(
 			return 0;
 		}
 
+		off += map.br_blockcount;
+	}
+
+	for (off = 0; off < endoff; off++) {
+		union xfs_suminfo_raw	*ondisk_info;
+
 		/* Read a block's worth of ondisk rtsummary file. */
-		error = xfs_rtsummary_read_buf(&args, off);
+		error = xfs_rtsummary_read_buf(&rts->args, off);
 		if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, off, &error))
 			return error;
 
 		/* Read a block's worth of computed rtsummary file. */
-		error = xfsum_copyout(sc, sumoff, sc->buf, mp->m_blockwsize);
+		error = xfsum_copyout(sc, sumoff, rts->words, mp->m_blockwsize);
 		if (error) {
-			xfs_rtbuf_cache_relse(&args);
+			xfs_rtbuf_cache_relse(&rts->args);
 			return error;
 		}
 
-		ondisk_info = xfs_rsumblock_infoptr(&args, 0);
-		if (memcmp(ondisk_info, sc->buf,
-					mp->m_blockwsize << XFS_WORDLOG) != 0)
+		ondisk_info = xfs_rsumblock_infoptr(&rts->args, 0);
+		if (memcmp(ondisk_info, rts->words,
+					mp->m_blockwsize << XFS_WORDLOG) != 0) {
 			xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, off);
+			xfs_rtbuf_cache_relse(&rts->args);
+			return error;
+		}
 
-		xfs_rtbuf_cache_relse(&args);
+		xfs_rtbuf_cache_relse(&rts->args);
 		sumoff += mp->m_blockwsize;
 	}
 
@@ -253,8 +296,43 @@ xchk_rtsummary(
 	struct xfs_scrub	*sc)
 {
 	struct xfs_mount	*mp = sc->mp;
+	struct xchk_rtsummary	*rts = sc->buf;
 	int			error = 0;
 
+	/* Is sb_rextents correct? */
+	if (mp->m_sb.sb_rextents != rts->rextents) {
+		xchk_ino_set_corrupt(sc, mp->m_rbmip->i_ino);
+		goto out_rbm;
+	}
+
+	/* Is m_rsumlevels correct? */
+	if (mp->m_rsumlevels != rts->rsumlevels) {
+		xchk_ino_set_corrupt(sc, mp->m_rsumip->i_ino);
+		goto out_rbm;
+	}
+
+	/* Is m_rsumsize correct? */
+	if (mp->m_rsumsize != rts->rsumsize) {
+		xchk_ino_set_corrupt(sc, mp->m_rsumip->i_ino);
+		goto out_rbm;
+	}
+
+	/* The summary file length must be aligned to an fsblock. */
+	if (mp->m_rsumip->i_disk_size & mp->m_blockmask) {
+		xchk_ino_set_corrupt(sc, mp->m_rsumip->i_ino);
+		goto out_rbm;
+	}
+
+	/*
+	 * Is the summary file itself large enough to handle the rt volume?
+	 * growfsrt expands the summary file before updating sb_rextents, so
+	 * the file can be larger than rsumsize.
+	 */
+	if (mp->m_rsumip->i_disk_size < rts->rsumsize) {
+		xchk_ino_set_corrupt(sc, mp->m_rsumip->i_ino);
+		goto out_rbm;
+	}
+
 	/* Invoke the fork scrubber. */
 	error = xchk_metadata_inode_forks(sc);
 	if (error || (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT))
Reviewed-by on v1 2/6
#119 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

fs/xfs/scrub/rtsummary.c xchk_setup_rtsummary( · line 104

Quoted 11 lines
fs/xfs/scrub/rtsummary.c +11 -0
+	/*
+	 * Now that we've locked the rtbitmap and rtsummary, we can't race with
+	 * growfsrt trying to expand the summary or change the size of the rt
+	 * volume.  Hence it is safe to compute and check the geometry values.
+	 */
+	rts->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rts->rbmblocks = xfs_rtbitmap_blockcount(mp, rts->rextents);
+	rts->rsumlevels = rts->rextents ? xfs_highbit32(rts->rextents) + 1 : 0;
+	rsumblocks = xfs_rtsummary_blockcount(mp, rts->rsumlevels,
+			rts->rbmblocks);
+	rts->rsumsize = XFS_FSB_TO_B(mp, rsumblocks);

Same nitpick as for the last patch.

Otherwise looks good:

#120 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
fs/xfs/scrub/rtsummary.c +5 -0
+	/*
+	 * Now that we've locked the rtbitmap and rtsummary, we can't race with
+	 * growfsrt trying to expand the summary or change the size of the rt
+	 * volume.  Hence it is safe to compute and check the geometry values.
+	 */
Quoted 8 lines

fs/xfs/scrub/rtsummary.c xchk_setup_rtsummary( · line 104

Quoted 6 lines
fs/xfs/scrub/rtsummary.c +6 -0
+	rts->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rts->rbmblocks = xfs_rtbitmap_blockcount(mp, rts->rextents);
+	rts->rsumlevels = rts->rextents ? xfs_highbit32(rts->rextents) + 1 : 0;
+	rsumblocks = xfs_rtsummary_blockcount(mp, rts->rsumlevels,
+			rts->rbmblocks);
+	rts->rsumsize = XFS_FSB_TO_B(mp, rsumblocks);

Same nitpick as for the last patch.

LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...

Quoted 3 lines

Otherwise looks good:

Thanks!

--D

#121 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

Quoted 15 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
fs/xfs/scrub/rtsummary.c +5 -0
+	/*
+	 * Now that we've locked the rtbitmap and rtsummary, we can't race with
+	 * growfsrt trying to expand the summary or change the size of the rt
+	 * volume.  Hence it is safe to compute and check the geometry values.
+	 */
Quoted 8 lines

fs/xfs/scrub/rtsummary.c xchk_setup_rtsummary( · line 104

Quoted 6 lines
fs/xfs/scrub/rtsummary.c +6 -0
+	rts->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
+	rts->rbmblocks = xfs_rtbitmap_blockcount(mp, rts->rextents);
+	rts->rsumlevels = rts->rextents ? xfs_highbit32(rts->rextents) + 1 : 0;
+	rsumblocks = xfs_rtsummary_blockcount(mp, rts->rsumlevels,
+			rts->rbmblocks);
+	rts->rsumsize = XFS_FSB_TO_B(mp, rsumblocks);

Same nitpick as for the last patch.

Quoted 2 lines

LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...

Oh good, XFS has been broken since the beginning of git history for the stupid corner case where the rtblocks < rextsize. In this case, mkfs will set sb_rextents and sb_rextslog both to zero:

	sbp->sb_rextslog =
		(uint8_t)(rtextents ?
			libxfs_highbit32((unsigned int)rtextents) : 0);

However, that's not the check that xfs_repair uses for nonzero rtblocks:

	if (sb->sb_rextslog !=
			libxfs_highbit32((unsigned int)sb->sb_rextents))

The difference here is that xfs_highbit32 returns -1 if its argument is zero, which means that for a runt rt volume, repair thinks the "correct" value of rextslog is -1, even though mkfs wrote it as 0 and flags a freshly formatted filesystem as corrupt. Because mkfs has been writing ondisk artifacts like this for decades, we have to accept that as "correct". TBH, zero rextslog for zero rtextents makes more sense to me anyway.

Regrettably, the superblock verifier checks created in commit copied xfs_repair even though mkfs has been writing out such filesystems for ages. In testing /that/ fix, I discovered that the logic above is wrong -- rsumlevels is always rextslog + 1 when rblocks > 0, even if rextents == 0.

IOWs, this logic needs to be:

	/*
	 * Now that we've locked the rtbitmap and rtsummary, we can't race with
	 * growfsrt trying to expand the summary or change the size of the rt
	 * volume.  Hence it is safe to compute and check the geometry values.
	 */
	if (mp->m_sb.sb_rblocks) {
		xfs_filblks_t	rsumblocks;
		int		rextslog = 0;

		rts->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
		if (rts->rextents)
			rextslog = xfs_highbit32(rts->rextents);
		rts->rsumlevels = rextslog + 1;
		rts->rbmblocks = xfs_rtbitmap_blockcount(mp, rts->rextents);
		rsumblocks = xfs_rtsummary_blockcount(mp, rts->rsumlevels,
				rts->rbmblocks);
		rts->rsumsize = XFS_FSB_TO_B(mp, rsumblocks);
	}

Yay winning.

--D

Quoted 8 lines
Quoted 3 lines

Otherwise looks good:

Thanks!

--D

#122 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

In reply to Darrick J. Wong
Quoted 2 lines · Darrick J. Wong

LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...

I think we should just reject rt device size < rtextsize configs in the kernel and all tools.

#123 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong

LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...

I think we should just reject rt device size < rtextsize configs in the kernel and all tools.

"But that could break old weirdass customer filesystems."

The design of rtgroups prohibits that, so we're ok going forward.

--D

#124 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

Quoted 11 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong

LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...

I think we should just reject rt device size < rtextsize configs in the kernel and all tools.

"But that could break old weirdass customer filesystems."

The design of rtgroups prohibits that, so we're ok going forward.

Well, as you just said it hasn't mounted for a long time, and really this is a corner case that just doesn't make any sense. I'd really prefer to cleanly reject it, and if someone really complains with a good reason we can revisit the decisions. But I strongly doubt it's ever going to happen.

#125 lore raw

Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly

Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong

LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...

I think we should just reject rt device size < rtextsize configs in the kernel and all tools.

"But that could break old weirdass customer filesystems."

Quoted 7 lines
Quoted 1 line

The design of rtgroups prohibits that, so we're ok going forward.

Well, as you just said it hasn't mounted for a long time, and really this is a corner case that just doesn't make any sense. I'd really prefer to cleanly reject it, and if someone really complains with a good reason we can revisit the decisions. But I strongly doubt it's ever going to happen.

Oh, even better, Dave and I noticed today that if you format a 17G realtime volume (> 2^32 rt extents) then mkfs fails because there's an integer overflow:

https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/mkfs/xfs_mkfs.c#n3739

Based on your observation that rt free space never exceeds the group length with rtgroups turned on, I'll tweak the sb_rextslog computation so that it's computed with (rgblocks / rextsize) instead of (rblocks / rextsize) which will fix that problem for future filesystems.

--D

#126 lore raw

PATCH 3/6 xfs: always check the rtbitmap and rtsummary files

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

XFS filesystems always have a realtime bitmap and summary file, even if there has never been a realtime volume attached. Always check them.

---
 fs/xfs/scrub/scrub.c |    2 --
 1 file changed, 2 deletions(-)
fs/xfs/scrub/scrub.c +0 -2
index bc70a91f8b1bf..89ce6d2f9ad14 100644
@@ -330,14 +330,12 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_FS,
 		.setup	= xchk_setup_rtbitmap,
 		.scrub	= xchk_rtbitmap,
-		.has	= xfs_has_realtime,
 		.repair	= xrep_notsupported,
 	},
 	[XFS_SCRUB_TYPE_RTSUM] = {	/* realtime summary */
 		.type	= ST_FS,
 		.setup	= xchk_setup_rtsummary,
 		.scrub	= xchk_rtsummary,
-		.has	= xfs_has_realtime,
 		.repair	= xrep_notsupported,
 	},
 	[XFS_SCRUB_TYPE_UQUOTA] = {	/* user quota */
#128 lore raw

PATCH 4/6 xfs: repair the inode core and forks of a metadata inode

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Add a helper function to repair the core and forks of a metadata inode, so that we can get move onto the task of repairing higher level metadata that lives in an inode.

---
 fs/xfs/scrub/bmap_repair.c |   17 ++++-
 fs/xfs/scrub/repair.c      |  151 ++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h      |    2 +
 3 files changed, 166 insertions(+), 4 deletions(-)
fs/xfs/scrub/bmap_repair.c +13 -4
index 2c593eebb1fc4..e55f3e52f36fc 100644
@@ -78,6 +78,9 @@ struct xrep_bmap {
 
 	/* Are there shared extents? */
 	bool			shared_extents;
+
+	/* Do we allow unwritten extents? */
+	bool			allow_unwritten;
 };
 
 /* Is this space extent shared?  Flag the inode if it is. */
@@ -272,6 +275,10 @@ xrep_bmap_walk_rmap(
 	    !(rec->rm_flags & XFS_RMAP_ATTR_FORK))
 		return 0;
 
+	/* Reject unwritten extents if we don't allow those. */
+	if ((rec->rm_flags & XFS_RMAP_UNWRITTEN) && !rb->allow_unwritten)
+		return -EFSCORRUPTED;
+
 	fsbno = XFS_AGB_TO_FSB(mp, cur->bc_ag.pag->pag_agno,
 			rec->rm_startblock);
 
@@ -762,10 +769,11 @@ xrep_bmap_check_inputs(
 }
 
 /* Repair an inode fork. */
-STATIC int
+int
 xrep_bmap(
 	struct xfs_scrub	*sc,
-	int			whichfork)
+	int			whichfork,
+	bool			allow_unwritten)
 {
 	struct xrep_bmap	*rb;
 	char			*descr;
@@ -784,6 +792,7 @@ xrep_bmap(
 		return -ENOMEM;
 	rb->sc = sc;
 	rb->whichfork = whichfork;
+	rb->allow_unwritten = allow_unwritten;
 
 	/*
 	 * No need to waste time scanning for shared extents if the inode is
@@ -834,7 +843,7 @@ int
 xrep_bmap_data(
 	struct xfs_scrub	*sc)
 {
-	return xrep_bmap(sc, XFS_DATA_FORK);
+	return xrep_bmap(sc, XFS_DATA_FORK, true);
 }
 
 /* Repair an inode's attr fork. */
@@ -842,5 +851,5 @@ int
 xrep_bmap_attr(
 	struct xfs_scrub	*sc)
 {
-	return xrep_bmap(sc, XFS_ATTR_FORK);
+	return xrep_bmap(sc, XFS_ATTR_FORK, false);
 }
fs/xfs/scrub/repair.c +151 -0

160 lines not on this page · raw message

fs/xfs/scrub/repair.h +2 -0
index be3585b8f4364..b7ddd35e753eb 100644
@@ -82,6 +82,8 @@ int xrep_ino_dqattach(struct xfs_scrub *sc);
 int xrep_ino_ensure_extent_count(struct xfs_scrub *sc, int whichfork,
 		xfs_extnum_t nextents);
 int xrep_reset_perag_resv(struct xfs_scrub *sc);
+int xrep_bmap(struct xfs_scrub *sc, int whichfork, bool allow_unwritten);
+int xrep_metadata_inode_forks(struct xfs_scrub *sc);
 
 /* Repair setup functions */
 int xrep_setup_ag_allocbt(struct xfs_scrub *sc);
#130 lore raw

PATCH 5/6 xfs: create a new inode fork block unmap helper

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a new helper to unmap blocks from an inode's fork.

---
 fs/xfs/libxfs/xfs_bmap.c |   39 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_bmap.h |    2 ++
 fs/xfs/xfs_inode.c       |   24 ++++--------------------
 3 files changed, 45 insertions(+), 20 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +39 -0
index 9968a3a6e6d8d..cebf490daf43e 100644
@@ -6231,3 +6231,42 @@ xfs_bmap_validate_extent(
 	return xfs_bmap_validate_extent_raw(ip->i_mount,
 			XFS_IS_REALTIME_INODE(ip), whichfork, irec);
 }
+
+/*
+ * Used in xfs_itruncate_extents().  This is the maximum number of extents
+ * freed from a file in a single transaction.
+ */
+#define	XFS_ITRUNC_MAX_EXTENTS	2
+
+/*
+ * Unmap every extent in part of an inode's fork.  We don't do any higher level
+ * invalidation work at all.
+ */
+int
+xfs_bunmapi_range(
+	struct xfs_trans	**tpp,
+	struct xfs_inode	*ip,
+	uint32_t		flags,
+	xfs_fileoff_t		startoff,
+	xfs_fileoff_t		endoff)
+{
+	xfs_filblks_t		unmap_len = endoff - startoff + 1;
+	int			error = 0;
+
+	ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+
+	while (unmap_len > 0) {
+		ASSERT((*tpp)->t_highest_agno == NULLAGNUMBER);
+		error = __xfs_bunmapi(*tpp, ip, startoff, &unmap_len, flags,
+				XFS_ITRUNC_MAX_EXTENTS);
+		if (error)
+			goto out;
+
+		/* free the just unmapped extents */
+		error = xfs_defer_finish(tpp);
+		if (error)
+			goto out;
+	}
+out:
+	return error;
+}
fs/xfs/libxfs/xfs_bmap.h +2 -0
index 8518324db2855..9bc78c717ecf7 100644
@@ -273,6 +273,8 @@ int xfs_bmap_complain_bad_rec(struct xfs_inode *ip, int whichfork,
 int	xfs_bmapi_remap(struct xfs_trans *tp, struct xfs_inode *ip,
 		xfs_fileoff_t bno, xfs_filblks_t len, xfs_fsblock_t startblock,
 		uint32_t flags);
+int	xfs_bunmapi_range(struct xfs_trans **tpp, struct xfs_inode *ip,
+		uint32_t flags, xfs_fileoff_t startoff, xfs_fileoff_t endoff);
 
 extern struct kmem_cache	*xfs_bmap_intent_cache;
 
fs/xfs/xfs_inode.c +4 -20
index c0f1c89786c2a..424b03628b7cf 100644
@@ -40,12 +40,6 @@
 
 struct kmem_cache *xfs_inode_cache;
 
-/*
- * Used in xfs_itruncate_extents().  This is the maximum number of extents
- * freed from a file in a single transaction.
- */
-#define	XFS_ITRUNC_MAX_EXTENTS	2
-
 STATIC int xfs_iunlink(struct xfs_trans *, struct xfs_inode *);
 STATIC int xfs_iunlink_remove(struct xfs_trans *tp, struct xfs_perag *pag,
 	struct xfs_inode *);
@@ -1339,7 +1333,6 @@ xfs_itruncate_extents_flags(
 	struct xfs_mount	*mp = ip->i_mount;
 	struct xfs_trans	*tp = *tpp;
 	xfs_fileoff_t		first_unmap_block;
-	xfs_filblks_t		unmap_len;
 	int			error = 0;
 
 	ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
@@ -1371,19 +1364,10 @@ xfs_itruncate_extents_flags(
 		return 0;
 	}
 
-	unmap_len = XFS_MAX_FILEOFF - first_unmap_block + 1;
-	while (unmap_len > 0) {
-		ASSERT(tp->t_highest_agno == NULLAGNUMBER);
-		error = __xfs_bunmapi(tp, ip, first_unmap_block, &unmap_len,
-				flags, XFS_ITRUNC_MAX_EXTENTS);
-		if (error)
-			goto out;
-
-		/* free the just unmapped extents */
-		error = xfs_defer_finish(&tp);
-		if (error)
-			goto out;
-	}
+	error = xfs_bunmapi_range(&tp, ip, flags, first_unmap_block,
+			XFS_MAX_FILEOFF);
+	if (error)
+		goto out;
 
 	if (whichfork == XFS_DATA_FORK) {
 		/* Remove all pending CoW reservations. */
Reviewed-by on v1 5/6
#131 lore raw

Re: [PATCH 5/6] xfs: create a new inode fork block unmap helper

Quoted 3 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Create a new helper to unmap blocks from an inode's fork.

Look good:

With this __xfs_bunmapi an be marked static in xfs_bmap.c, as there is no user outside of it (and neither in xfsprogs).

#132 lore raw

PATCH 6/6 xfs: online repair of realtime bitmaps

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix all the file metadata surrounding the realtime bitmap file, which includes the rt geometry, file size, forks, and space mappings. The bitmap contents themselves cannot be fixed without rt rmap, so that will come later.

---
 fs/xfs/Makefile                |    4 +
 fs/xfs/scrub/repair.h          |    7 +
 fs/xfs/scrub/rtbitmap.c        |   16 ++-
 fs/xfs/scrub/rtbitmap.h        |   22 ++++
 fs/xfs/scrub/rtbitmap_repair.c |  202 ++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/scrub.c           |    2 
 6 files changed, 245 insertions(+), 8 deletions(-)
 create mode 100644 fs/xfs/scrub/rtbitmap.h
 create mode 100644 fs/xfs/scrub/rtbitmap_repair.c
fs/xfs/Makefile +4 -0
index 71a76f8ac5e47..36e7bc7d147e2 100644
@@ -191,5 +191,9 @@ xfs-y				+= $(addprefix scrub/, \
 				   refcount_repair.o \
 				   repair.o \
 				   )
+
+xfs-$(CONFIG_XFS_RT)		+= $(addprefix scrub/, \
+				   rtbitmap_repair.o \
+				   )
 endif
 endif
fs/xfs/scrub/repair.h +7 -0
index b7ddd35e753eb..f54dff9268bcc 100644
@@ -115,6 +115,12 @@ int xrep_bmap_data(struct xfs_scrub *sc);
 int xrep_bmap_attr(struct xfs_scrub *sc);
 int xrep_bmap_cow(struct xfs_scrub *sc);
 
+#ifdef CONFIG_XFS_RT
+int xrep_rtbitmap(struct xfs_scrub *sc);
+#else
+# define xrep_rtbitmap			xrep_notsupported
+#endif /* CONFIG_XFS_RT */
+
 int xrep_reinit_pagf(struct xfs_scrub *sc);
 int xrep_reinit_pagi(struct xfs_scrub *sc);
 
@@ -177,6 +183,7 @@ xrep_setup_nothing(
 #define xrep_bmap_data			xrep_notsupported
 #define xrep_bmap_attr			xrep_notsupported
 #define xrep_bmap_cow			xrep_notsupported
+#define xrep_rtbitmap			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/rtbitmap.c +9 -7
index 3b5b62fbf4e0a..92058eb545344 100644
@@ -17,12 +17,8 @@
 #include "xfs_bit.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
-
-struct xchk_rtbitmap {
-	uint64_t		rextents;
-	uint64_t		rbmblocks;
-	unsigned int		rextslog;
-};
+#include "scrub/repair.h"
+#include "scrub/rtbitmap.h"
 
 /* Set us up with the realtime metadata locked. */
 int
@@ -38,7 +34,13 @@ xchk_setup_rtbitmap(
 		return -ENOMEM;
 	sc->buf = rtb;
 
-	error = xchk_trans_alloc(sc, 0);
+	if (xchk_could_repair(sc)) {
+		error = xrep_setup_rtbitmap(sc, rtb);
+		if (error)
+			return error;
+	}
+
+	error = xchk_trans_alloc(sc, rtb->resblks);
 	if (error)
 		return error;
 
fs/xfs/scrub/rtbitmap.h new file +22 -0
new file mode 100644
index 0000000000000..85304ff019e1d
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2023 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_SCRUB_RTBITMAP_H__
+#define __XFS_SCRUB_RTBITMAP_H__
+
+struct xchk_rtbitmap {
+	uint64_t		rextents;
+	uint64_t		rbmblocks;
+	unsigned int		rextslog;
+	unsigned int		resblks;
+};
+
+#ifdef CONFIG_XFS_ONLINE_REPAIR
+int xrep_setup_rtbitmap(struct xfs_scrub *sc, struct xchk_rtbitmap *rtb);
+#else
+# define xrep_setup_rtbitmap(sc, rtb)	(0)
+#endif /* CONFIG_XFS_ONLINE_REPAIR */
+
+#endif /* __XFS_SCRUB_RTBITMAP_H__ */
fs/xfs/scrub/rtbitmap_repair.c new file +202 -0

202 lines not on this page · raw message

fs/xfs/scrub/scrub.c +1 -1
index 89ce6d2f9ad14..9982b626bfc33 100644
@@ -330,7 +330,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_FS,
 		.setup	= xchk_setup_rtbitmap,
 		.scrub	= xchk_rtbitmap,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_rtbitmap,
 	},
 	[XFS_SCRUB_TYPE_RTSUM] = {	/* realtime summary */
 		.type	= ST_FS,
#134 lore raw

cover xfs: online repair of quota and rt metadata files

In reply to Darrick J. Wong

Hi all,

XFS stores quota records and free space bitmap information in files. Add the necessary infrastructure to enable repairing metadata inodes and their forks, and then make it so that we can repair the file metadata for the rtbitmap. Repairing the bitmap contents (and the summary file) is left for subsequent patchsets.

We also add the ability to repair file metadata the quota files. As part of these repairs, we also reinitialize the ondisk dquot records as necessary to get the incore dquots working. We can also correct obviously bad dquot record attributes, but we leave checking the resource usage counts for the next patchsets.

If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-quota

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-quota

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-quota

---
 fs/xfs/Makefile             |    9 +
 fs/xfs/libxfs/xfs_format.h  |    3 
 fs/xfs/scrub/dqiterate.c    |  211 ++++++++++++++++
 fs/xfs/scrub/quota.c        |  107 +++++++-
 fs/xfs/scrub/quota.h        |   36 +++
 fs/xfs/scrub/quota_repair.c |  575 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h       |    7 +
 fs/xfs/scrub/scrub.c        |    6 
 fs/xfs/scrub/trace.c        |    3 
 fs/xfs/scrub/trace.h        |   78 ++++++
 fs/xfs/xfs_dquot.c          |   37 ---
 fs/xfs/xfs_dquot.h          |    8 -
 12 files changed, 1026 insertions(+), 54 deletions(-)
 create mode 100644 fs/xfs/scrub/dqiterate.c
 create mode 100644 fs/xfs/scrub/quota.h
 create mode 100644 fs/xfs/scrub/quota_repair.c
#135 lore raw

PATCH 1/5 xfs: check the ondisk space mapping behind a dquot

From: Darrick J. Wong <[email protected]>

Each xfs_dquot object caches the file offset and daddr of the ondisk block that backs the dquot. Make sure these cached values are the same as the bmapi data, and that the block state is written.

---
 fs/xfs/scrub/quota.c |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)
fs/xfs/scrub/quota.c +58 -0
index 5671c81534335..59350cd7a325b 100644
@@ -6,6 +6,7 @@
 #include "xfs.h"
 #include "xfs_fs.h"
 #include "xfs_shared.h"
+#include "xfs_bit.h"
 #include "xfs_format.h"
 #include "xfs_trans_resv.h"
 #include "xfs_mount.h"
@@ -75,6 +76,47 @@ struct xchk_quota_info {
 	xfs_dqid_t		last_id;
 };
 
+/* There's a written block backing this dquot, right? */
+STATIC int
+xchk_quota_item_bmap(
+	struct xfs_scrub	*sc,
+	struct xfs_dquot	*dq,
+	xfs_fileoff_t		offset)
+{
+	struct xfs_bmbt_irec	irec;
+	struct xfs_mount	*mp = sc->mp;
+	int			nmaps = 1;
+	int			error;
+
+	if (!xfs_verify_fileoff(mp, offset)) {
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+		return 0;
+	}
+
+	if (dq->q_fileoffset != offset) {
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+		return 0;
+	}
+
+	error = xfs_bmapi_read(sc->ip, offset, 1, &irec, &nmaps, 0);
+	if (error)
+		return error;
+
+	if (nmaps != 1) {
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+		return 0;
+	}
+
+	if (!xfs_verify_fsbno(mp, irec.br_startblock))
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+	if (XFS_FSB_TO_DADDR(mp, irec.br_startblock) != dq->q_blkno)
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+	if (!xfs_bmap_is_written_extent(&irec))
+		xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+
+	return 0;
+}
+
 /* Scrub the fields in an individual quota item. */
 STATIC int
 xchk_quota_item(
@@ -93,6 +135,17 @@ xchk_quota_item(
 	if (xchk_should_terminate(sc, &error))
 		return error;
 
+	/*
+	 * We want to validate the bmap record for the storage backing this
+	 * dquot, so we need to lock the dquot and the quota file.  For quota
+	 * operations, the locking order is first the ILOCK and then the dquot.
+	 * However, dqiterate gave us a locked dquot, so drop the dquot lock to
+	 * get the ILOCK.
+	 */
+	xfs_dqunlock(dq);
+	xchk_ilock(sc, XFS_ILOCK_SHARED);
+	xfs_dqlock(dq);
+
 	/*
 	 * Except for the root dquot, the actual dquot we got must either have
 	 * the same or higher id as we saw before.
@@ -103,6 +156,11 @@ xchk_quota_item(
 
 	sqi->last_id = dq->q_id;
 
+	error = xchk_quota_item_bmap(sc, dq, offset);
+	xchk_iunlock(sc, XFS_ILOCK_SHARED);
+	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, offset, &error))
+		return error;
+
 	/*
 	 * Warn if the hard limits are larger than the fs.
 	 * Administrators can do this, though in production this seems
#137 lore raw

PATCH 2/5 xfs: check dquot resource timers

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

For each dquot resource, ensure either (a) the resource usage is over the soft limit and there is a nonzero timer; or (b) usage is at or under the soft limit and the timer is unset. (a) is redundant with the dquot buffer verifier, but (b) isn't checked anywhere.

Found by fuzzing xfs/426 and noticing that diskdq.btimer = add didn't trip any kind of warning for having a timer set even with no limits.

---
 fs/xfs/scrub/quota.c |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
fs/xfs/scrub/quota.c +21 -0
index 59350cd7a325b..49835d2840b4a 100644
@@ -117,6 +117,23 @@ xchk_quota_item_bmap(
 	return 0;
 }
 
+/* Complain if a quota timer is incorrectly set. */
+static inline void
+xchk_quota_item_timer(
+	struct xfs_scrub		*sc,
+	xfs_fileoff_t			offset,
+	const struct xfs_dquot_res	*res)
+{
+	if ((res->softlimit && res->count > res->softlimit) ||
+	    (res->hardlimit && res->count > res->hardlimit)) {
+		if (!res->timer)
+			xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+	} else {
+		if (res->timer)
+			xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, offset);
+	}
+}
+
 /* Scrub the fields in an individual quota item. */
 STATIC int
 xchk_quota_item(
@@ -224,6 +241,10 @@ xchk_quota_item(
 	    dq->q_rtb.count > dq->q_rtb.hardlimit)
 		xchk_fblock_set_warning(sc, XFS_DATA_FORK, offset);
 
+	xchk_quota_item_timer(sc, offset, &dq->q_blk);
+	xchk_quota_item_timer(sc, offset, &dq->q_ino);
+	xchk_quota_item_timer(sc, offset, &dq->q_rtb);
+
 out:
 	if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
 		return -ECANCELED;
#139 lore raw

PATCH 3/5 xfs: pull xfs_qm_dqiterate back into scrub

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

There aren't any other users of this code outside of online fsck, so pull it back in there.

---
 fs/xfs/Makefile          |    5 ++++
 fs/xfs/scrub/dqiterate.c |   52 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/quota.c     |    3 ++-
 fs/xfs/scrub/quota.h     |   14 ++++++++++++
 fs/xfs/xfs_dquot.c       |   31 ---------------------------
 fs/xfs/xfs_dquot.h       |    5 ----
 6 files changed, 72 insertions(+), 38 deletions(-)
 create mode 100644 fs/xfs/scrub/dqiterate.c
 create mode 100644 fs/xfs/scrub/quota.h
fs/xfs/Makefile +4 -1
index 36e7bc7d147e2..91008db406fb2 100644
@@ -175,7 +175,10 @@ xfs-$(CONFIG_XFS_RT)		+= $(addprefix scrub/, \
 				   rtsummary.o \
 				   )
 
-xfs-$(CONFIG_XFS_QUOTA)		+= scrub/quota.o
+xfs-$(CONFIG_XFS_QUOTA)		+= $(addprefix scrub/, \
+				   dqiterate.o \
+				   quota.o \
+				   )
 
 # online repair
 ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
fs/xfs/scrub/dqiterate.c new file +52 -0

52 lines not on this page · raw message

fs/xfs/scrub/quota.c +2 -1
index 49835d2840b4a..f142ca6646061 100644
@@ -18,6 +18,7 @@
 #include "xfs_bmap.h"
 #include "scrub/scrub.h"
 #include "scrub/common.h"
+#include "scrub/quota.h"
 
 /* Convert a scrub type code to a DQ flag, or return 0 if error. */
 static inline xfs_dqtype_t
@@ -320,7 +321,7 @@ xchk_quota(
 	xchk_iunlock(sc, sc->ilock_flags);
 	sqi.sc = sc;
 	sqi.last_id = 0;
-	error = xfs_qm_dqiterate(mp, dqtype, xchk_quota_item, &sqi);
+	error = xchk_dqiterate(mp, dqtype, xchk_quota_item, &sqi);
 	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
fs/xfs/scrub/quota.h new file +14 -0
new file mode 100644
index 0000000000000..0d7b3b01436e6
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2018-2023 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_SCRUB_QUOTA_H__
+#define __XFS_SCRUB_QUOTA_H__
+
+typedef int (*xchk_dqiterate_fn)(struct xfs_dquot *dq,
+		xfs_dqtype_t type, void *priv);
+int xchk_dqiterate(struct xfs_mount *mp, xfs_dqtype_t type,
+		xchk_dqiterate_fn iter_fn, void *priv);
+
+#endif /* __XFS_SCRUB_QUOTA_H__ */
fs/xfs/xfs_dquot.c +0 -31
index a013b87ab8d5e..60ec401e26ffd 100644
@@ -1362,34 +1362,3 @@ xfs_qm_exit(void)
 	kmem_cache_destroy(xfs_dqtrx_cache);
 	kmem_cache_destroy(xfs_dquot_cache);
 }
-
-/*
- * Iterate every dquot of a particular type.  The caller must ensure that the
- * particular quota type is active.  iter_fn can return negative error codes,
- * or -ECANCELED to indicate that it wants to stop iterating.
- */
-int
-xfs_qm_dqiterate(
-	struct xfs_mount	*mp,
-	xfs_dqtype_t		type,
-	xfs_qm_dqiterate_fn	iter_fn,
-	void			*priv)
-{
-	struct xfs_dquot	*dq;
-	xfs_dqid_t		id = 0;
-	int			error;
-
-	do {
-		error = xfs_qm_dqget_next(mp, id, type, &dq);
-		if (error == -ENOENT)
-			return 0;
-		if (error)
-			return error;
-
-		error = iter_fn(dq, type, priv);
-		id = dq->q_id + 1;
-		xfs_qm_dqput(dq);
-	} while (error == 0 && id != 0);
-
-	return error;
-}
fs/xfs/xfs_dquot.h +0 -5
index 80c8f851a2f3b..8d9d4b0d979d0 100644
@@ -234,11 +234,6 @@ static inline struct xfs_dquot *xfs_qm_dqhold(struct xfs_dquot *dqp)
 	return dqp;
 }
 
-typedef int (*xfs_qm_dqiterate_fn)(struct xfs_dquot *dq,
-		xfs_dqtype_t type, void *priv);
-int xfs_qm_dqiterate(struct xfs_mount *mp, xfs_dqtype_t type,
-		xfs_qm_dqiterate_fn iter_fn, void *priv);
-
 time64_t xfs_dquot_set_timeout(struct xfs_mount *mp, time64_t timeout);
 time64_t xfs_dquot_set_grace_period(time64_t grace);
 
#140 lore raw

Re: [PATCH 3/5] xfs: pull xfs_qm_dqiterate back into scrub

Quoted 4 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

There aren't any other users of this code outside of online fsck, so pull it back in there.

The move itself looks fine, but what about just open coding it and getting rid of the indirection functional?

#141 lore raw

PATCH 4/5 xfs: improve dquot iteration for scrub

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Upon a closer inspection of the quota record scrubber, I noticed that dqiterate wasn't actually walking all possible dquots for the mapped blocks in the quota file. This is due to xfs_qm_dqget_next skipping all XFS_IS_DQUOT_UNINITIALIZED dquots.

For a fsck program, we really want to look at all the dquots, even if all counters and limits in the dquot record are zero. Rewrite the implementation to do this, as well as switching to an iterator paradigm to reduce the number of indirect calls.

---
 fs/xfs/libxfs/xfs_format.h |    3 +
 fs/xfs/scrub/dqiterate.c   |  195 ++++++++++++++++++++++++++++++++++++++++----
 fs/xfs/scrub/quota.c       |   24 +++--
 fs/xfs/scrub/quota.h       |   28 +++++-
 fs/xfs/scrub/trace.c       |    2 
 fs/xfs/scrub/trace.h       |   49 +++++++++++
 6 files changed, 270 insertions(+), 31 deletions(-)
fs/xfs/libxfs/xfs_format.h +3 -0
index 9dd3b21434314..1b2becaac0b7f 100644
@@ -1273,6 +1273,9 @@ static inline time64_t xfs_dq_bigtime_to_unix(uint32_t ondisk_seconds)
 #define XFS_DQ_GRACE_MIN		((int64_t)0)
 #define XFS_DQ_GRACE_MAX		((int64_t)U32_MAX)
 
+/* Maximum id value for a quota record */
+#define XFS_DQ_ID_MAX			(U32_MAX)
+
 /*
  * This is the main portion of the on-disk representation of quota information
  * for a user.  We pad this with some more expansion room to construct the on
fs/xfs/scrub/dqiterate.c +177 -18

211 lines not on this page · raw message

fs/xfs/scrub/quota.c +15 -9
index f142ca6646061..1a65a75025276 100644
@@ -138,11 +138,9 @@ xchk_quota_item_timer(
 /* Scrub the fields in an individual quota item. */
 STATIC int
 xchk_quota_item(
-	struct xfs_dquot	*dq,
-	xfs_dqtype_t		dqtype,
-	void			*priv)
+	struct xchk_quota_info	*sqi,
+	struct xfs_dquot	*dq)
 {
-	struct xchk_quota_info	*sqi = priv;
 	struct xfs_scrub	*sc = sqi->sc;
 	struct xfs_mount	*mp = sc->mp;
 	struct xfs_quotainfo	*qi = mp->m_quotainfo;
@@ -271,7 +269,7 @@ xchk_quota_data_fork(
 		return error;
 
 	/* Check for data fork problems that apply only to quota files. */
-	max_dqid_off = ((xfs_dqid_t)-1) / qi->qi_dqperchunk;
+	max_dqid_off = XFS_DQ_ID_MAX / qi->qi_dqperchunk;
 	ifp = xfs_ifork_ptr(sc->ip, XFS_DATA_FORK);
 	for_each_xfs_iext(ifp, &icur, &irec) {
 		if (xchk_should_terminate(sc, &error))
@@ -298,9 +296,11 @@ int
 xchk_quota(
 	struct xfs_scrub	*sc)
 {
-	struct xchk_quota_info	sqi;
+	struct xchk_dqiter	cursor = { };
+	struct xchk_quota_info	sqi = { .sc = sc };
 	struct xfs_mount	*mp = sc->mp;
 	struct xfs_quotainfo	*qi = mp->m_quotainfo;
+	struct xfs_dquot	*dq;
 	xfs_dqtype_t		dqtype;
 	int			error = 0;
 
@@ -319,9 +319,15 @@ xchk_quota(
 	 * functions.
 	 */
 	xchk_iunlock(sc, sc->ilock_flags);
-	sqi.sc = sc;
-	sqi.last_id = 0;
-	error = xchk_dqiterate(mp, dqtype, xchk_quota_item, &sqi);
+
+	/* Now look for things that the quota verifiers won't complain about. */
+	xchk_dqiter_init(&cursor, sc, dqtype);
+	while ((error = xchk_dquot_iter(&cursor, &dq)) == 1) {
+		error = xchk_quota_item(&sqi, dq);
+		xfs_qm_dqput(dq);
+		if (error)
+			break;
+	}
 	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
fs/xfs/scrub/quota.h +24 -4
index 0d7b3b01436e6..5056b7766c4a2 100644
@@ -6,9 +6,29 @@
 #ifndef __XFS_SCRUB_QUOTA_H__
 #define __XFS_SCRUB_QUOTA_H__
 
-typedef int (*xchk_dqiterate_fn)(struct xfs_dquot *dq,
-		xfs_dqtype_t type, void *priv);
-int xchk_dqiterate(struct xfs_mount *mp, xfs_dqtype_t type,
-		xchk_dqiterate_fn iter_fn, void *priv);
+/* dquot iteration code */
+
+struct xchk_dqiter {
+	struct xfs_scrub	*sc;
+
+	/* Quota file that we're walking. */
+	struct xfs_inode	*quota_ip;
+
+	/* Cached data fork mapping for the dquot. */
+	struct xfs_bmbt_irec	bmap;
+
+	/* The next dquot to scan. */
+	uint64_t		id;
+
+	/* Quota type (user/group/project). */
+	xfs_dqtype_t		dqtype;
+
+	/* Data fork sequence number to detect stale mappings. */
+	unsigned int		if_seq;
+};
+
+void xchk_dqiter_init(struct xchk_dqiter *cursor, struct xfs_scrub *sc,
+		xfs_dqtype_t dqtype);
+int xchk_dquot_iter(struct xchk_dqiter *cursor, struct xfs_dquot **dqpp);
 
 #endif /* __XFS_SCRUB_QUOTA_H__ */
fs/xfs/scrub/trace.c +2 -0
index 29afa48512355..4641522fd9070 100644
@@ -14,9 +14,11 @@
 #include "xfs_btree.h"
 #include "xfs_ag.h"
 #include "xfs_rtbitmap.h"
+#include "xfs_quota.h"
 #include "scrub/scrub.h"
 #include "scrub/xfile.h"
 #include "scrub/xfarray.h"
+#include "scrub/quota.h"
 
 /* Figure out which block the btree cursor was pointing to. */
 static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +49 -0

61 lines not on this page · raw message

Reviewed-by on v1 4/5
#142 lore raw

Re: [PATCH 4/5] xfs: improve dquot iteration for scrub

Quoted 11 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Upon a closer inspection of the quota record scrubber, I noticed that dqiterate wasn't actually walking all possible dquots for the mapped blocks in the quota file. This is due to xfs_qm_dqget_next skipping all XFS_IS_DQUOT_UNINITIALIZED dquots.

For a fsck program, we really want to look at all the dquots, even if all counters and limits in the dquot record are zero. Rewrite the implementation to do this, as well as switching to an iterator paradigm to reduce the number of indirect calls.

Heh, this basically ends up doing what I suggested in the last patch (and a lot more).

I'd just fold the previous patch into this one, as there's no point in moving the function just to remove it immediately.

Otherwise looks good:

#143 lore raw

PATCH 5/5 xfs: repair quotas

In reply to Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Fix anything that causes the quota verifiers to fail.

---
 fs/xfs/Makefile             |    4 
 fs/xfs/scrub/quota.c        |    3 
 fs/xfs/scrub/quota.h        |    2 
 fs/xfs/scrub/quota_repair.c |  575 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/repair.h       |    7 +
 fs/xfs/scrub/scrub.c        |    6 
 fs/xfs/scrub/trace.c        |    1 
 fs/xfs/scrub/trace.h        |   29 ++
 fs/xfs/xfs_dquot.c          |    6 
 fs/xfs/xfs_dquot.h          |    3 
 10 files changed, 628 insertions(+), 8 deletions(-)
 create mode 100644 fs/xfs/scrub/quota_repair.c
fs/xfs/Makefile +4 -0
index 91008db406fb2..8d10fe02054db 100644
@@ -198,5 +198,9 @@ xfs-y				+= $(addprefix scrub/, \
 xfs-$(CONFIG_XFS_RT)		+= $(addprefix scrub/, \
 				   rtbitmap_repair.o \
 				   )
+
+xfs-$(CONFIG_XFS_QUOTA)		+= $(addprefix scrub/, \
+				   quota_repair.o \
+				   )
 endif
 endif
fs/xfs/scrub/quota.c +1 -2
index 1a65a75025276..183d531875eae 100644
@@ -21,7 +21,7 @@
 #include "scrub/quota.h"
 
 /* Convert a scrub type code to a DQ flag, or return 0 if error. */
-static inline xfs_dqtype_t
+xfs_dqtype_t
 xchk_quota_to_dqtype(
 	struct xfs_scrub	*sc)
 {
@@ -328,7 +328,6 @@ xchk_quota(
 		if (error)
 			break;
 	}
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,
fs/xfs/scrub/quota.h +2 -0
index 5056b7766c4a2..6c7134ce2385e 100644
@@ -6,6 +6,8 @@
 #ifndef __XFS_SCRUB_QUOTA_H__
 #define __XFS_SCRUB_QUOTA_H__
 
+xfs_dqtype_t xchk_quota_to_dqtype(struct xfs_scrub *sc);
+
 /* dquot iteration code */
 
 struct xchk_dqiter {
fs/xfs/scrub/quota_repair.c new file +575 -0

575 lines not on this page · raw message

fs/xfs/scrub/repair.h +7 -0
index f54dff9268bcc..da0866f9b4525 100644
@@ -121,6 +121,12 @@ int xrep_rtbitmap(struct xfs_scrub *sc);
 # define xrep_rtbitmap			xrep_notsupported
 #endif /* CONFIG_XFS_RT */
 
+#ifdef CONFIG_XFS_QUOTA
+int xrep_quota(struct xfs_scrub *sc);
+#else
+# define xrep_quota			xrep_notsupported
+#endif /* CONFIG_XFS_QUOTA */
+
 int xrep_reinit_pagf(struct xfs_scrub *sc);
 int xrep_reinit_pagi(struct xfs_scrub *sc);
 
@@ -184,6 +190,7 @@ xrep_setup_nothing(
 #define xrep_bmap_attr			xrep_notsupported
 #define xrep_bmap_cow			xrep_notsupported
 #define xrep_rtbitmap			xrep_notsupported
+#define xrep_quota			xrep_notsupported
 
 #endif /* CONFIG_XFS_ONLINE_REPAIR */
 
fs/xfs/scrub/scrub.c +3 -3
index 9982b626bfc33..0fbfed522d656 100644
@@ -342,19 +342,19 @@ static const struct xchk_meta_ops meta_scrub_ops[] = {
 		.type	= ST_FS,
 		.setup	= xchk_setup_quota,
 		.scrub	= xchk_quota,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_quota,
 	},
 	[XFS_SCRUB_TYPE_GQUOTA] = {	/* group quota */
 		.type	= ST_FS,
 		.setup	= xchk_setup_quota,
 		.scrub	= xchk_quota,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_quota,
 	},
 	[XFS_SCRUB_TYPE_PQUOTA] = {	/* project quota */
 		.type	= ST_FS,
 		.setup	= xchk_setup_quota,
 		.scrub	= xchk_quota,
-		.repair	= xrep_notsupported,
+		.repair	= xrep_quota,
 	},
 	[XFS_SCRUB_TYPE_FSCOUNTERS] = {	/* fs summary counters */
 		.type	= ST_FS,
fs/xfs/scrub/trace.c +1 -0
index 4641522fd9070..d0e24ffaf7547 100644
@@ -15,6 +15,7 @@
 #include "xfs_ag.h"
 #include "xfs_rtbitmap.h"
 #include "xfs_quota.h"
+#include "xfs_quota_defs.h"
 #include "scrub/scrub.h"
 #include "scrub/xfile.h"
 #include "scrub/xfarray.h"
fs/xfs/scrub/trace.h +29 -0
index 3bfd53b4e8d0b..f8e316357f56f 100644
@@ -1732,6 +1732,35 @@ TRACE_EVENT(xrep_cow_free_staging,
 		  __entry->blockcount)
 );
 
+#ifdef CONFIG_XFS_QUOTA
+DECLARE_EVENT_CLASS(xrep_dquot_class,
+	TP_PROTO(struct xfs_mount *mp, uint8_t type, uint32_t id),
+	TP_ARGS(mp, type, id),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(uint8_t, type)
+		__field(uint32_t, id)
+	),
+	TP_fast_assign(
+		__entry->dev = mp->m_super->s_dev;
+		__entry->id = id;
+		__entry->type = type;
+	),
+	TP_printk("dev %d:%d type %s id 0x%x",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __print_flags(__entry->type, "|", XFS_DQTYPE_STRINGS),
+		  __entry->id)
+);
+
+#define DEFINE_XREP_DQUOT_EVENT(name) \
+DEFINE_EVENT(xrep_dquot_class, name, \
+	TP_PROTO(struct xfs_mount *mp, uint8_t type, uint32_t id), \
+	TP_ARGS(mp, type, id))
+DEFINE_XREP_DQUOT_EVENT(xrep_dquot_item);
+DEFINE_XREP_DQUOT_EVENT(xrep_disk_dquot);
+DEFINE_XREP_DQUOT_EVENT(xrep_dquot_item_fill_bmap_hole);
+#endif /* CONFIG_XFS_QUOTA */
+
 #endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
 
 #endif /* _TRACE_XFS_SCRUB_TRACE_H */
fs/xfs/xfs_dquot.c +3 -3
index 60ec401e26ffd..a93ad76f23c56 100644
@@ -172,14 +172,14 @@ xfs_qm_adjust_dqtimers(
 /*
  * initialize a buffer full of dquots and log the whole thing
  */
-STATIC void
+void
 xfs_qm_init_dquot_blk(
 	struct xfs_trans	*tp,
-	struct xfs_mount	*mp,
 	xfs_dqid_t		id,
 	xfs_dqtype_t		type,
 	struct xfs_buf		*bp)
 {
+	struct xfs_mount	*mp = tp->t_mountp;
 	struct xfs_quotainfo	*q = mp->m_quotainfo;
 	struct xfs_dqblk	*d;
 	xfs_dqid_t		curid;
@@ -353,7 +353,7 @@ xfs_dquot_disk_alloc(
 	 * Make a chunk of dquots out of this buffer and log
 	 * the entire thing.
 	 */
-	xfs_qm_init_dquot_blk(tp, mp, dqp->q_id, qtype, bp);
+	xfs_qm_init_dquot_blk(tp, dqp->q_id, qtype, bp);
 	xfs_buf_set_ref(bp, XFS_DQUOT_REF);
 
 	/*
fs/xfs/xfs_dquot.h +3 -0
index 8d9d4b0d979d0..956272d9b302f 100644
@@ -237,4 +237,7 @@ static inline struct xfs_dquot *xfs_qm_dqhold(struct xfs_dquot *dqp)
 time64_t xfs_dquot_set_timeout(struct xfs_mount *mp, time64_t timeout);
 time64_t xfs_dquot_set_grace_period(time64_t grace);
 
+void xfs_qm_init_dquot_blk(struct xfs_trans *tp, xfs_dqid_t id, xfs_dqtype_t
+		type, struct xfs_buf *bp);
+
 #endif /* __XFS_DQUOT_H__ */
#144 lore raw

Re: [PATCH 5/5] xfs: repair quotas

fs/xfs/scrub/quota.c xchk_quota( · line 333

Quoted 8 lines
fs/xfs/scrub/quota.c +0 -1
@@ -328,7 +328,6 @@ xchk_quota(
 		if (error)
 			break;
 	}
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,

What is the replacement for this lock? The call in xrep_quota_item?

I'm a little confused on how locking works - about all flags in sc->ilock_flags are released, and here we used just lock the exclusive ilock directly without tracking it.

#145 lore raw

Re: [PATCH 5/5] xfs: repair quotas

Quoted 10 lines · Christoph Hellwig

fs/xfs/scrub/quota.c xchk_quota( · line 333

Quoted 8 lines
fs/xfs/scrub/quota.c +0 -1
@@ -328,7 +328,6 @@ xchk_quota(
 		if (error)
 			break;
 	}
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,

What is the replacement for this lock? The call in xrep_quota_item?

The replacement is the conditional re-lock at the start of xrep_quota. I could have left this alone, though for the scrub-only case it reduces lock cycling by one.

Quoted 3 lines

I'm a little confused on how locking works - about all flags in sc->ilock_flags are released, and here we used just lock the exclusive ilock directly without tracking it.

Fixing quota files is a bit of a locking mess because dqget needs to take the ILOCK to read the data fork if the dquot isn't already in memory. As a result, the scrub functions have to take the lock to inspect the data fork for mapping problems only to drop it before iterating the incore dquots.

For each dquot, we then have to take the ILOCK *again* to check that its q_fileoffset and q_blkno fields actually match the bmap.

Similarly, repair has to retake the lock to fix any problems that were found with the mapping before dropping once again to walk the incore dquots. Then we cycle ILOCK for every dquot to fix the mapping.

Not sure what you meant about "we used just lock the exclusive lock directly without tracking it" -- both files call xchk_{ilock,iunlock}. The telemetry data I've collected shows that quota file checking is sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.

--D

#146 lore raw

Re: [PATCH 5/5] xfs: repair quotas

Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig

fs/xfs/scrub/quota.c xchk_quota( · line 333

Quoted 8 lines
fs/xfs/scrub/quota.c +0 -1
@@ -328,7 +328,6 @@ xchk_quota(
 		if (error)
 			break;
 	}
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,

What is the replacement for this lock? The call in xrep_quota_item?

Quoted 1 line

The replacement is the conditional re-lock at the start of xrep_quota.

Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the xchk_iunlock call in xchk_teardown would have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..

Quoted 6 lines

Not sure what you meant about "we used just lock the exclusive lock directly without tracking it" -- both files call xchk_{ilock,iunlock}. The telemetry data I've collected shows that quota file checking is sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.

My confusions was more about checking/using sc->ilock_flags in the callers, while it is maintained by the locking helpers. Probably not *THAT* unusual, but I might have simply been too tired to understand it.

#147 lore raw

Re: [PATCH 5/5] xfs: repair quotas

Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig

fs/xfs/scrub/quota.c xchk_quota( · line 333

Quoted 8 lines
fs/xfs/scrub/quota.c +0 -1
@@ -328,7 +328,6 @@ xchk_quota(
 		if (error)
 			break;
 	}
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,

What is the replacement for this lock? The call in xrep_quota_item?

Quoted 7 lines
Quoted 1 line

The replacement is the conditional re-lock at the start of xrep_quota.

Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the xchk_iunlock call in xchk_teardown would have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..

How about I not remove the xchk_ilock call, then? Repair is already smart enough to take the lock if it doesn't have it, so it's not strictly necessary for correct operation.

Quoted 10 lines
Quoted 6 lines

Not sure what you meant about "we used just lock the exclusive lock directly without tracking it" -- both files call xchk_{ilock,iunlock}. The telemetry data I've collected shows that quota file checking is sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.

My confusions was more about checking/using sc->ilock_flags in the callers, while it is maintained by the locking helpers. Probably not *THAT* unusual, but I might have simply been too tired to understand it.

Ah, got it. I'll ponder a no-alloc dqget in the meantime.

--D

#148 lore raw

Re: [PATCH 5/5] xfs: repair quotas

Quoted 15 lines · Darrick J. Wong
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

fs/xfs/scrub/quota.c xchk_quota( · line 333

Quoted 4 lines
fs/xfs/scrub/quota.c +0 -1
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,

What is the replacement for this lock? The call in xrep_quota_item?

Quoted 7 lines
Quoted 1 line

The replacement is the conditional re-lock at the start of xrep_quota.

Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the xchk_iunlock call in xchk_teardown would have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..

Quoted 3 lines

How about I not remove the xchk_ilock call, then? Repair is already smart enough to take the lock if it doesn't have it, so it's not strictly necessary for correct operation.

No, please keep this hunk. As I said I would have preferred to have it in a separate hunk to understand it, but it understand it now, and it does seems useful.

#149 lore raw

Re: [PATCH 5/5] xfs: repair quotas

Quoted 16 lines · Christoph Hellwig
Quoted 15 lines · Darrick J. Wong
Quoted 14 lines
Quoted 8 lines
Quoted 6 lines

fs/xfs/scrub/quota.c xchk_quota( · line 333

Quoted 4 lines
fs/xfs/scrub/quota.c +0 -1
-	xchk_ilock(sc, XFS_ILOCK_EXCL);
 	if (error == -ECANCELED)
 		error = 0;
 	if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,

What is the replacement for this lock? The call in xrep_quota_item?

The replacement is the conditional re-lock at the start of xrep_quota.

Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the xchk_iunlock call in xchk_teardown would have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..

Quoted 7 lines
Quoted 3 lines

How about I not remove the xchk_ilock call, then? Repair is already smart enough to take the lock if it doesn't have it, so it's not strictly necessary for correct operation.

No, please keep this hunk. As I said I would have preferred to have it in a separate hunk to understand it, but it understand it now, and it does seems useful.

Ok, I'll keep it then.

--D

#150 lore raw

Re: [PATCH 5/5] xfs: repair quotas

In reply to Darrick J. Wong
Quoted 3 lines · Darrick J. Wong

sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.

That beeing said, a no-alloc dqget does sound like a sensible idea in general.