xfs: online repair, second part of part 1
Subsystems (2)
PATCH v28 1/1 xfs: online repair, second part of part 1
Hi everyone,
[this time not as a reply to v27]
[[this time really not as a reply to v27]]
[[[***** email, this way is insane]]]
I've rebased the online fsck development branches atop 6.7, applied the changes requested during the review of v27, and reworked the automatic space reaping code to avoid open-coding EFI log item handling, and cleaned up a few other things.
In other words, I'm formally submitting part 1 for inclusion in 6.8.
Just like the last several submissions, I would like people to focus the following:
- Are the major subsystems sufficiently documented that you could figure out what the code does?
- Do you see any problems that are severe enough to cause long term support hassles? (e.g. bad API design, writing weird metadata to disk)
- Can you spot mis-interactions between the subsystems?
- What were my blind spots in devising this feature?
- Are there missing pieces that you'd like to help build?
- Can I just merge all of this?
The one thing that is /not/ in scope for this review are requests for more refactoring of existing subsystems. I'm still running QA round the clock. To spare vger, I'm only sending a few patchsets this time. I will of course stress test the new mailing infrastructure on 31 Dec with a full posting, like I always do.
--D
cover xfs: prevent livelocks in xchk_iget
Hi all,
Prevent scrub from live locking in xchk_iget if there's a cycle in the inobt by allocating an empty transaction.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-livelock-prevention
--- fs/xfs/scrub/common.c | 6 ++++-- fs/xfs/scrub/common.h | 19 +++++++++++++++++++ fs/xfs/scrub/inode.c | 4 ++-- 3 files changed, 25 insertions(+), 4 deletions(-)
PATCH 1/1 xfs: make xchk_iget safer in the presence of corrupt inode btrees
From: Darrick J. Wong <[email protected]>
When scrub is trying to iget an inode, ensure that it won't end up deadlocked on a cycle in the inode btree by using an empty transaction to store all the buffers.
--- fs/xfs/scrub/common.c | 6 ++++-- fs/xfs/scrub/common.h | 19 +++++++++++++++++++ fs/xfs/scrub/inode.c | 4 ++-- 3 files changed, 25 insertions(+), 4 deletions(-)
fs/xfs/scrub/common.c +4 -2
@@ -733,6 +733,8 @@ xchk_iget(xfs_ino_t inum, struct xfs_inode **ipp) { + ASSERT(sc->tp != NULL); +return xfs_iget (sc->mp, sc->tp, inum,XFS_IGET_UNTRUSTED ,0 , ipp); }
@@ -882,8 +884,8 @@ xchk_iget_for_scrubbing(if (!xfs_verify_ino (sc->mp, sc->sm->sm_ino))return -ENOENT ; -/* Try a regular untrusted iget. */ - error =xchk_iget (sc, sc->sm->sm_ino, &ip); +/* Try a safe untrusted iget. */ + error =xchk_iget_safe (sc, sc->sm->sm_ino, &ip);if (!error)return xchk_install_handle_inode (sc, ip);if (error == -ENOENT )
fs/xfs/scrub/common.h +19 -0
@@ -157,6 +157,25 @@ int xchk_iget_agi(struct xfs_scrub *sc, xfs_ino_t inum,void xchk_irele (struct xfs_scrub *sc,struct xfs_inode *ip);int xchk_install_handle_inode (struct xfs_scrub *sc,struct xfs_inode *ip); +/* +* Safe version of (untrusted) xchk_iget that uses an empty transaction to +* avoid deadlocking on loops in the inobt. +*/ +static inline int +xchk_iget_safe (struct xfs_scrub *sc,xfs_ino_t inum,struct xfs_inode **ipp) +{ +int error; + +ASSERT (sc->tp ==NULL ); + + error =xchk_trans_alloc (sc,0 ); +if (error) +return error; + error =xchk_iget (sc, inum, ipp); +xchk_trans_cancel (sc); +return error; +} +/* * Don't bother cross-referencing if we already found corruption or cross * referencing discrepancies.
fs/xfs/scrub/inode.c +2 -2
@@ -95,8 +95,8 @@ xchk_setup_inode(if (!xfs_verify_ino (sc->mp, sc->sm->sm_ino))return -ENOENT ; -/* Try a regular untrusted iget. */ - error =xchk_iget (sc, sc->sm->sm_ino, &ip); +/* Try a safe untrusted iget. */ + error =xchk_iget_safe (sc, sc->sm->sm_ino, &ip);if (!error)return xchk_install_handle_iscrub (sc, ip);if (error == -ENOENT )
Re: [PATCH 1/1] xfs: make xchk_iget safer in the presence of corrupt inode btrees
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
When scrub is trying to iget an inode, ensure that it won't end up deadlocked on a cycle in the inode btree by using an empty transaction to store all the buffers.
My only concern here is how I'm suppsed to know when to use the _safe version or not.
Otherwise looks good:
Re: [PATCH 1/1] xfs: make xchk_iget safer in the presence of corrupt inode btrees
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
When scrub is trying to iget an inode, ensure that it won't end up deadlocked on a cycle in the inode btree by using an empty transaction to store all the buffers.
My only concern here is how I'm suppsed to know when to use the
_safeversion or not.
For xchk_iget_safe, I'll amend the comment to read:
/*
* Safe version of (untrusted) xchk_iget that uses an empty transaction to
* avoid deadlocking on loops in the inobt. This should only be used in a
* scrub or repair setup routine, and only prior to grabbing a transaction. */
and add a comment for xchk_iget that reads:
/*
* Grab the inode at @inum. The caller must have created a scrub transaction
* so that we can confirm the inumber by walking the inobt and not deadlock on
* a loop in the inobt.
*/
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
cover xfs: reserve disk space for online repairs
In reply to Darrick J. WongHi all,
Online repair fixes metadata structures by writing a new copy out to disk and atomically committing the new structure into the filesystem. For this to work, we need to reserve all the space we're going to need ahead of time so that the atomic commit transaction is as small as possible. We also require the reserved space to be freed if the system goes down, or if we decide not to commit the repair, or if we reserve too much space.
To keep the atomic commit transaction as small as possible, we would like to allocate some space and simultaneously schedule automatic reaping of the reserved space, even on log recovery. EFIs are the mechanism to get us there, but we need to use them in a novel manner. Once we allocate the space, we want to hold on to the EFI (relogging as necessary) until we can commit or cancel the repair. EFIs for written committed blocks need to go away, but unwritten or uncommitted blocks can be freed like normal.
Earlier versions of this patchset directly manipulated the log items, but Dave thought that to be a layering violation. For v27, I've modified the defer ops handling code to be capable of pausing a deferred work item. Log intent items are created as they always have been, but paused items are pushed onto a side list when finishing deferred work items, and pushed back onto the transaction after that. Log intent done item are not created for paused work.
The second part adds a "stale" flag to the EFI so that the repair reservation code can dispose of an EFI the normal way, but without the space actually being freed.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-auto-reap-space-reservations
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-auto-reap-space-reservations
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_ag.c | 2 fs/xfs/libxfs/xfs_alloc.c | 104 +++++++ fs/xfs/libxfs/xfs_alloc.h | 22 +- fs/xfs/libxfs/xfs_bmap.c | 4 fs/xfs/libxfs/xfs_bmap_btree.c | 2 fs/xfs/libxfs/xfs_btree_staging.h | 7 fs/xfs/libxfs/xfs_defer.c | 229 ++++++++++++++-- fs/xfs/libxfs/xfs_defer.h | 20 + fs/xfs/libxfs/xfs_ialloc.c | 5 fs/xfs/libxfs/xfs_ialloc_btree.c | 2 fs/xfs/libxfs/xfs_refcount.c | 6 fs/xfs/libxfs/xfs_refcount_btree.c | 2 fs/xfs/scrub/agheader_repair.c | 1 fs/xfs/scrub/common.c | 1 fs/xfs/scrub/newbt.c | 510 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/newbt.h | 65 +++++ fs/xfs/scrub/reap.c | 7 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 37 +++ fs/xfs/xfs_extfree_item.c | 12 - fs/xfs/xfs_reflink.c | 2 fs/xfs/xfs_trace.h | 13 + 23 files changed, 990 insertions(+), 66 deletions(-) create mode 100644 fs/xfs/scrub/newbt.c create mode 100644 fs/xfs/scrub/newbt.h
PATCH 1/7 xfs: don't append work items to logged xfs_defer_pending objects
From: Darrick J. Wong <[email protected]>
When someone tries to add a deferred work item to xfs_defer_add, it will try to attach the work item to the most recently added xfs_defer_pending object attached to the transaction. However, it doesn't check if the pending object has a log intent item attached to it. This is incorrect behavior because we cannot add more work to an object that has already been committed to the ondisk log.
Therefore, change the behavior not to append to pending items with a non null dfp_intent. In practice this has not been an issue because the only way xfs_defer_add gets called after log intent items have been committed is from the defer ops ->finish_item functions themselves, and the @dop_pending isolation in xfs_defer_finish_noroll protects the pending items that have already been logged.
However, the next patch will add the ability to pause a deferred extent free object during online btree rebuilding, and any new extfree work items need to have their own pending event.
While we're at it, hoist the predicate to its own static inline function for readability.
--- fs/xfs/libxfs/xfs_defer.c | 48 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_defer.c +36 -12
@@ -624,6 +624,40 @@ xfs_defer_cancel(xfs_defer_cancel_list (mp, &tp->t_dfops); } +/* +* Decide if we can add a deferred work item to the last dfops item attached +* to the transaction. +*/ +static inline struct xfs_defer_pending * +xfs_defer_try_append ( +struct xfs_trans *tp, +enum xfs_defer_ops_type type, +const struct xfs_defer_op_type *ops) +{ +struct xfs_defer_pending *dfp =NULL ; + +/* No dfops at all? */ +if (list_empty (&tp->t_dfops)) +return NULL ; + + dfp = list_last_entry(&tp->t_dfops, struct xfs_defer_pending, + dfp_list); + +/* Wrong type? */ +if (dfp->dfp_type != type) +return NULL ; + +/* Already logged? */ +if (dfp->dfp_intent) +return NULL ; + +/* Already full? */ +if (ops->max_items && dfp->dfp_count >= ops->max_items) +return NULL ; + +return dfp; +} +/* Add an item for later deferred processing. */ void xfs_defer_add(
@@ -637,19 +671,9 @@ xfs_defer_add(ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );BUILD_BUG_ON (ARRAY_SIZE (defer_op_types) !=XFS_DEFER_OPS_TYPE_MAX ); -/* -* Add the item to a pending item at the end of the intake list. -* If the last pending item has the same type, reuse it. Else, -* create a new pending item at the end of the intake list. -*/ -if (!list_empty (&tp->t_dfops)) { - dfp =list_last_entry (&tp->t_dfops, - struct xfs_defer_pending, dfp_list); -if (dfp->dfp_type != type || - (ops->max_items && dfp->dfp_count >= ops->max_items)) - dfp =NULL ; - } + dfp =xfs_defer_try_append (tp, type, ops);if (!dfp) { +/* Create a new pending item at the end of the intake list. */ dfp =kmem_cache_zalloc (xfs_defer_pending_cache,GFP_NOFS | __GFP_NOFAIL); dfp->dfp_type = type;
Re: [PATCH 1/7] xfs: don't append work items to logged xfs_defer_pending objects
This looks good:
To make the code nicer for the later addition of the barrier defer ops I'd fold the hunk belw to split xfs_defer_try_append, but we could also do that later:
index 6c283b30ea054a..7be2f9063e0ded 100644
fs/xfs/libxfs/xfs_defer.c +18 -14
@@ -624,17 +624,12 @@ xfs_defer_cancel(xfs_defer_cancel_list (mp, &tp->t_dfops); } -/* -* Decide if we can add a deferred work item to the last dfops item attached -* to the transaction. -*/ static inline struct xfs_defer_pending * -xfs_defer_try_append ( +xfs_defer_find (struct xfs_trans *tp, -enum xfs_defer_ops_type type, -const struct xfs_defer_op_type *ops) +enum xfs_defer_ops_type type) { -struct xfs_defer_pending *dfp =NULL ; +struct xfs_defer_pending *dfp;/* No dfops at all? */ if (list_empty (&tp->t_dfops))
@@ -646,16 +641,25 @@ xfs_defer_try_append(/* Wrong type? */ if (dfp->dfp_type != type)return NULL ; +return dfp; +} +/* +* Decide if we can add a deferred work item to the last dfops item attached +* to the transaction. +*/ +static inline bool +xfs_defer_can_append ( +struct xfs_defer_pending *dfp, +const struct xfs_defer_op_type *ops) +{/* Already logged? */ if (dfp->dfp_intent) -return NULL ; - +return false;/* Already full? */ if (ops->max_items && dfp->dfp_count >= ops->max_items)return NULL ; - -return dfp; +return true; }/* Add an item for later deferred processing. */
@@ -671,8 +675,8 @@ xfs_defer_add(ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );BUILD_BUG_ON (ARRAY_SIZE (defer_op_types) !=XFS_DEFER_OPS_TYPE_MAX ); - dfp =xfs_defer_try_append (tp, type, ops); -if (!dfp) { + dfp =xfs_defer_find (tp, type); +if (!dfp || !xfs_defer_can_append (dfp, ops)) {/* Create a new pending item at the end of the intake list. */ dfp =kmem_cache_zalloc (xfs_defer_pending_cache,GFP_NOFS | __GFP_NOFAIL);
PATCH 2/7 xfs: allow pausing of pending deferred work items
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Traditionally, all pending deferred work attached to a transaction is finished when one of the xfs_defer_finish* functions is called. However, online repair wants to be able to allocate space for a new data structure, format a new metadata structure into the allocated space, and commit that into the filesystem.
As a hedge against system crashes during repairs, we also want to log some EFI items for the allocated space speculatively, and cancel them if we elect to commit the new data structure.
Therefore, introduce the idea of pausing a pending deferred work item. Log intent items are still created for paused items and relogged as necessary. However, paused items are pushed onto a side list before we start calling ->finish_item, and the whole list is reattach to the transaction afterwards. New work items are never attached to paused pending items.
Modify xfs_defer_cancel to clean up pending deferred work items holding a log intent item but not a log intent done item, since that is now possible.
--- fs/xfs/libxfs/xfs_defer.c | 98 +++++++++++++++++++++++++++++++++++++++------ fs/xfs/libxfs/xfs_defer.h | 17 +++++++- fs/xfs/xfs_trace.h | 13 +++++- 3 files changed, 112 insertions(+), 16 deletions(-)
fs/xfs/libxfs/xfs_defer.c +86 -12
@@ -417,7 +417,7 @@ xfs_defer_cancel_list(* done item to release the intent item; and then log a new intent item. * The caller should provide a fresh transaction and roll it after we're done. */ -static int +static void xfs_defer_relog (struct xfs_trans **tpp,struct list_head *dfops)
@@ -458,10 +458,6 @@ xfs_defer_relog(XFS_STATS_INC ((*tpp)->t_mountp, defer_relog); dfp->dfp_intent =xfs_trans_item_relog (dfp->dfp_intent, *tpp); } - -if ((*tpp)->t_flags &XFS_TRANS_DIRTY ) -return xfs_defer_trans_roll (tpp); -return 0 ; }/*
@@ -517,6 +513,24 @@ xfs_defer_finish_one(return error; } +/* Move all paused deferred work from @tp to @paused_list. */ +static void +xfs_defer_isolate_paused ( +struct xfs_trans *tp, +struct list_head *paused_list) +{ +struct xfs_defer_pending *dfp; +struct xfs_defer_pending *pli; + +list_for_each_entry_safe (dfp, pli, &tp->t_dfops, dfp_list) { +if (!(dfp->dfp_flags &XFS_DEFER_PAUSED )) +continue ; + +list_move_tail (&dfp->dfp_list, paused_list); +trace_xfs_defer_isolate_paused (tp->t_mountp, dfp); + } +} +/* * Finish all the pending work. This involves logging intent items for * any work items that wandered in since the last transaction roll (if
@@ -532,6 +546,7 @@ xfs_defer_finish_noroll(struct xfs_defer_pending *dfp =NULL ;int error =0 ;LIST_HEAD (dop_pending); +LIST_HEAD (dop_paused);ASSERT ((*tp)->t_flags &XFS_TRANS_PERM_LOG_RES );
@@ -550,6 +565,8 @@ xfs_defer_finish_noroll(*/ int has_intents =xfs_defer_create_intents (*tp); +xfs_defer_isolate_paused (*tp, &dop_paused); +list_splice_init (&(*tp)->t_dfops, &dop_pending);if (has_intents <0 ) {
@@ -562,22 +579,33 @@ xfs_defer_finish_noroll( goto out_shutdown;/* Relog intent items to keep the log moving. */ - error =xfs_defer_relog (tp, &dop_pending); -if (error) - goto out_shutdown; +xfs_defer_relog (tp, &dop_pending); +xfs_defer_relog (tp, &dop_paused); + +if ((*tp)->t_flags &XFS_TRANS_DIRTY ) { + error =xfs_defer_trans_roll (tp); +if (error) + goto out_shutdown; + } } - dfp = list_first_entry(&dop_pending, struct xfs_defer_pending, - dfp_list); + dfp =list_first_entry_or_null (&dop_pending, + struct xfs_defer_pending, dfp_list); +if (!dfp) +break ; error =xfs_defer_finish_one (*tp, dfp);if (error && error != -EAGAIN ) goto out_shutdown; } +/* Requeue the paused items in the outgoing transaction. */ +list_splice_tail_init (&dop_paused, &(*tp)->t_dfops); +trace_xfs_defer_finish_done (*tp, _RET_IP_);return 0 ; out_shutdown: +list_splice_tail_init (&dop_paused, &dop_pending);xfs_defer_trans_abort (*tp, &dop_pending);xfs_force_shutdown ((*tp)->t_mountp,SHUTDOWN_CORRUPT_INCORE );trace_xfs_defer_finish_error (*tp, error);
@@ -590,6 +618,9 @@ intxfs_defer_finish ( struct xfs_trans **tp) { +#ifdef DEBUG +struct xfs_defer_pending *dfp; +#endif int error;/*
@@ -609,7 +640,10 @@ xfs_defer_finish( }/* Reset LOWMODE now that we've finished all the dfops. */ -ASSERT (list_empty (&(*tp)->t_dfops)); +#ifdef DEBUG +list_for_each_entry (dfp, &(*tp)->t_dfops, dfp_list) +ASSERT (dfp->dfp_flags &XFS_DEFER_PAUSED ); +#endif (*tp)->t_flags &= ~XFS_TRANS_LOWMODE ;return 0 ; }
@@ -621,6 +655,7 @@ xfs_defer_cancel(struct xfs_mount *mp = tp->t_mountp;trace_xfs_defer_cancel (tp, _RET_IP_); +xfs_defer_trans_abort (tp, &tp->t_dfops);xfs_defer_cancel_list (mp, &tp->t_dfops); }
@@ -651,6 +686,10 @@ xfs_defer_try_append(if (dfp->dfp_intent)return NULL ; +/* Paused items cannot absorb more work */ +if (dfp->dfp_flags &XFS_DEFER_PAUSED ) +return NULL ; +/* Already full? */ if (ops->max_items && dfp->dfp_count >= ops->max_items)return NULL ;
@@ -659,7 +698,7 @@ xfs_defer_try_append( }/* Add an item for later deferred processing. */ -void +struct xfs_defer_pending * xfs_defer_add( struct xfs_trans *tp, enum xfs_defer_ops_type type,
@@ -687,6 +726,8 @@ xfs_defer_add(list_add_tail (li, &dfp->dfp_work);trace_xfs_defer_add_item (tp->t_mountp, dfp, li); dfp->dfp_count++; + +return dfp; }/*
@@ -962,3 +1003,36 @@ xfs_defer_destroy_item_caches(void)xfs_rmap_intent_destroy_cache ();xfs_defer_destroy_cache (); } + +/* +* Mark a deferred work item so that it will be requeued indefinitely without +* being finished. Caller must ensure there are no data dependencies on this +* work item in the meantime. +*/ +void +xfs_defer_item_pause ( +struct xfs_trans *tp, +struct xfs_defer_pending *dfp) +{ +ASSERT (!(dfp->dfp_flags &XFS_DEFER_PAUSED )); + + dfp->dfp_flags |=XFS_DEFER_PAUSED ; + +trace_xfs_defer_item_pause (tp->t_mountp, dfp); +} + +/* +* Release a paused deferred work item so that it will be finished during the +* next transaction roll. +*/ +void +xfs_defer_item_unpause ( +struct xfs_trans *tp, +struct xfs_defer_pending *dfp) +{ +ASSERT (dfp->dfp_flags &XFS_DEFER_PAUSED ); + + dfp->dfp_flags &= ~XFS_DEFER_PAUSED ; + +trace_xfs_defer_item_unpause (tp->t_mountp, dfp); +}
fs/xfs/libxfs/xfs_defer.h +15 -2
@@ -34,11 +34,24 @@ struct xfs_defer_pending {struct xfs_log_item *dfp_intent;/* log intent item */ struct xfs_log_item *dfp_done;/* log done item */ unsigned int dfp_count;/* # extent items */ +unsigned int dfp_flags;enum xfs_defer_ops_type dfp_type; }; -void xfs_defer_add (struct xfs_trans *tp,enum xfs_defer_ops_type type, -struct list_head *h); +/* +* Create a log intent item for this deferred item, but don't actually finish +* the work. Caller must clear this before the final transaction commit. +*/ +#define XFS_DEFER_PAUSED (1U << 0) + +#define XFS_DEFER_PENDING_STRINGS \ + { XFS_DEFER_PAUSED, "paused" } + +void xfs_defer_item_pause (struct xfs_trans *tp,struct xfs_defer_pending *dfp); +void xfs_defer_item_unpause (struct xfs_trans *tp,struct xfs_defer_pending *dfp); + +struct xfs_defer_pending *xfs_defer_add (struct xfs_trans *tp, +enum xfs_defer_ops_type type,struct list_head *h);int xfs_defer_finish_noroll (struct xfs_trans **tp);int xfs_defer_finish (struct xfs_trans **tp);void xfs_defer_cancel (struct xfs_trans *);
fs/xfs/xfs_trace.h +11 -2
@@ -2551,6 +2551,7 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_class, __field(dev_t, dev) __field(int, type) __field(void *, intent) + __field(unsigned int, flags) __field(char, committed) __field(int, nr) ),
@@ -2558,13 +2559,15 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_class, __entry->dev = mp ? mp->m_super->s_dev :0 ; __entry->type = dfp->dfp_type; __entry->intent = dfp->dfp_intent; + __entry->flags = dfp->dfp_flags; __entry->committed = dfp->dfp_done !=NULL ; __entry->nr = dfp->dfp_count; ), -TP_printk ("dev %d:%d optype %d intent %p committed %d nr %d" , +TP_printk ("dev %d:%d optype %d intent %p flags %s committed %d nr %d" ,MAJOR (__entry->dev),MINOR (__entry->dev), __entry->type, __entry->intent, +__print_flags (__entry->flags,"|" ,XFS_DEFER_PENDING_STRINGS ), __entry->committed, __entry->nr) )
@@ -2675,6 +2678,9 @@ DEFINE_DEFER_PENDING_EVENT(xfs_defer_cancel_list);DEFINE_DEFER_PENDING_EVENT (xfs_defer_pending_finish);DEFINE_DEFER_PENDING_EVENT (xfs_defer_pending_abort);DEFINE_DEFER_PENDING_EVENT (xfs_defer_relog_intent); +DEFINE_DEFER_PENDING_EVENT (xfs_defer_isolate_paused); +DEFINE_DEFER_PENDING_EVENT (xfs_defer_item_pause); +DEFINE_DEFER_PENDING_EVENT (xfs_defer_item_unpause);#define DEFINE_BMAP_FREE_DEFERRED_EVENT DEFINE_PHYS_EXTENT_DEFERRED_EVENTDEFINE_BMAP_FREE_DEFERRED_EVENT (xfs_bmap_free_defer);
@@ -2692,6 +2698,7 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_item_class, __field(void *, intent) __field(void *, item) __field(char, committed) + __field(unsigned int, flags) __field(int, nr) ), TP_fast_assign(
@@ -2700,13 +2707,15 @@ DECLARE_EVENT_CLASS(xfs_defer_pending_item_class, __entry->intent = dfp->dfp_intent; __entry->item = item; __entry->committed = dfp->dfp_done !=NULL ; + __entry->flags = dfp->dfp_flags; __entry->nr = dfp->dfp_count; ), -TP_printk ("dev %d:%d optype %d intent %p item %p committed %d nr %d" , +TP_printk ("dev %d:%d optype %d intent %p item %p flags %s committed %d nr %d" ,MAJOR (__entry->dev),MINOR (__entry->dev), __entry->type, __entry->intent, __entry->item, +__print_flags (__entry->flags,"|" ,XFS_DEFER_PENDING_STRINGS ), __entry->committed, __entry->nr) )
Re: [PATCH 2/7] xfs: allow pausing of pending deferred work items
Looks good:
PATCH 3/7 xfs: remove __xfs_free_extent_later
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
xfs_free_extent_later is a trivial helper, so remove it to reduce the amount of thinking required to understand the deferred freeing interface. This will make it easier to introduce automatic reaping of speculative allocations in the next patch.
--- fs/xfs/libxfs/xfs_ag.c | 2 +- fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_alloc.h | 14 +------------- fs/xfs/libxfs/xfs_bmap.c | 4 ++-- fs/xfs/libxfs/xfs_bmap_btree.c | 2 +- fs/xfs/libxfs/xfs_ialloc.c | 5 +++-- fs/xfs/libxfs/xfs_ialloc_btree.c | 2 +- fs/xfs/libxfs/xfs_refcount.c | 6 +++--- fs/xfs/libxfs/xfs_refcount_btree.c | 2 +- fs/xfs/scrub/reap.c | 2 +- fs/xfs/xfs_extfree_item.c | 2 +- fs/xfs/xfs_reflink.c | 2 +- 12 files changed, 17 insertions(+), 28 deletions(-)
fs/xfs/libxfs/xfs_ag.c +1 -1
@@ -984,7 +984,7 @@ xfs_ag_shrink_space(if (err2 != -ENOSPC ) goto resv_err; - err2 =__xfs_free_extent_later (*tpp, args.fsbno, delta,NULL , + err2 =xfs_free_extent_later (*tpp, args.fsbno, delta,NULL ,XFS_AG_RESV_NONE , true);if (err2) goto resv_err;
fs/xfs/libxfs/xfs_alloc.c +1 -1
@@ -2523,7 +2523,7 @@ xfs_defer_agfl_block(* The list is maintained sorted (by block number). */ int -__xfs_free_extent_later( +xfs_free_extent_later( struct xfs_trans *tp, xfs_fsblock_t bno, xfs_filblks_t len,
fs/xfs/libxfs/xfs_alloc.h +1 -13
@@ -231,7 +231,7 @@ xfs_buf_to_agfl_bno(return bp->b_addr; } -int __xfs_free_extent_later (struct xfs_trans *tp,xfs_fsblock_t bno, +int xfs_free_extent_later (struct xfs_trans *tp,xfs_fsblock_t bno,xfs_filblks_t len,const struct xfs_owner_info *oinfo,enum xfs_ag_resv_type type,bool skip_discard);
@@ -256,18 +256,6 @@ void xfs_extent_free_get_group(struct xfs_mount *mp,#define XFS_EFI_ATTR_FORK (1U << 1)/* freeing attr fork block */ #define XFS_EFI_BMBT_BLOCK (1U << 2)/* freeing bmap btree block */ -static inline int -xfs_free_extent_later ( -struct xfs_trans *tp, -xfs_fsblock_t bno, -xfs_filblks_t len, -const struct xfs_owner_info *oinfo, -enum xfs_ag_resv_type type) -{ -return __xfs_free_extent_later (tp, bno, len, oinfo, type, false); -} - -extern struct kmem_cache *xfs_extfree_item_cache;int __init xfs_extfree_intent_init_cache (void );
fs/xfs/libxfs/xfs_bmap.c +2 -2
@@ -575,7 +575,7 @@ xfs_bmap_btree_to_extents(xfs_rmap_ino_bmbt_owner (&oinfo, ip->i_ino, whichfork); error =xfs_free_extent_later (cur->bc_tp, cbno,1 , &oinfo, -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false);if (error)return error;
@@ -5218,7 +5218,7 @@ xfs_bmap_del_extent_real(if (xfs_is_reflink_inode (ip) && whichfork ==XFS_DATA_FORK ) {xfs_refcount_decrease_extent (tp, del); }else { - error = __xfs_free_extent_later(tp, del->br_startblock, + error = xfs_free_extent_later(tp, del->br_startblock, del->br_blockcount, NULL, XFS_AG_RESV_NONE, ((bflags & XFS_BMAPI_NODISCARD) ||
fs/xfs/libxfs/xfs_bmap_btree.c +1 -1
@@ -272,7 +272,7 @@ xfs_bmbt_free_block(xfs_rmap_ino_bmbt_owner (&oinfo, ip->i_ino, cur->bc_ino.whichfork); error =xfs_free_extent_later (cur->bc_tp, fsbno,1 , &oinfo, -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false);if (error)return error;
fs/xfs/libxfs/xfs_ialloc.c +3 -2
@@ -1854,7 +1854,7 @@ xfs_difree_inode_chunk(return xfs_free_extent_later (tp,XFS_AGB_TO_FSB (mp, agno, sagbno),M_IGEO (mp)->ialloc_blks, &XFS_RMAP_OINFO_INODES , -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false); }/* holemask is only 16-bits (fits in an unsigned long) */
@@ -1900,7 +1900,8 @@ xfs_difree_inode_chunk(ASSERT (contigblk % mp->m_sb.sb_spino_align ==0 ); error =xfs_free_extent_later (tp,XFS_AGB_TO_FSB (mp, agno, agbno), contigblk, - &XFS_RMAP_OINFO_INODES ,XFS_AG_RESV_NONE ); + &XFS_RMAP_OINFO_INODES ,XFS_AG_RESV_NONE , + false);if (error)return error;
fs/xfs/libxfs/xfs_ialloc_btree.c +1 -1
@@ -161,7 +161,7 @@ __xfs_inobt_free_block(xfs_inobt_mod_blockcount (cur,-1 ); fsbno =XFS_DADDR_TO_FSB (cur->bc_mp,xfs_buf_daddr (bp));return xfs_free_extent_later (cur->bc_tp, fsbno,1 , - &XFS_RMAP_OINFO_INOBT , resv); + &XFS_RMAP_OINFO_INOBT , resv, false); }STATIC int
fs/xfs/libxfs/xfs_refcount.c +3 -3
@@ -1153,7 +1153,7 @@ xfs_refcount_adjust_extents( tmp.rc_startblock); error =xfs_free_extent_later (cur->bc_tp, fsbno, tmp.rc_blockcount,NULL , -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false);if (error) goto out_error; }
@@ -1215,7 +1215,7 @@ xfs_refcount_adjust_extents( ext.rc_startblock); error =xfs_free_extent_later (cur->bc_tp, fsbno, ext.rc_blockcount,NULL , -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false);if (error) goto out_error; }
@@ -1985,7 +1985,7 @@ xfs_refcount_recover_cow_leftovers(/* Free the block. */ error =xfs_free_extent_later (tp, fsb, rr->rr_rrec.rc_blockcount,NULL , -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false);if (error) goto out_trans;
fs/xfs/libxfs/xfs_refcount_btree.c +1 -1
@@ -112,7 +112,7 @@ xfs_refcountbt_free_block(be32_add_cpu (&agf->agf_refcount_blocks,-1 );xfs_alloc_log_agf (cur->bc_tp, agbp,XFS_AGF_REFCOUNT_BLOCKS );return xfs_free_extent_later (cur->bc_tp, fsbno,1 , - &XFS_RMAP_OINFO_REFC ,XFS_AG_RESV_METADATA ); + &XFS_RMAP_OINFO_REFC ,XFS_AG_RESV_METADATA , false); }STATIC int
fs/xfs/scrub/reap.c +1 -1
@@ -410,7 +410,7 @@ xreap_agextent_iter(* Use deferred frees to get rid of the old btree blocks to try to * minimize the window in which we could crash and lose the old blocks. */ - error =__xfs_free_extent_later (sc->tp, fsbno, *aglenp, rs->oinfo, + error =xfs_free_extent_later (sc->tp, fsbno, *aglenp, rs->oinfo, rs->resv, true);if (error)return error;
fs/xfs/xfs_extfree_item.c +1 -1
@@ -717,7 +717,7 @@ xfs_efi_item_recover( error =xfs_free_extent_later (tp, fake.xefi_startblock, fake.xefi_blockcount, &XFS_RMAP_OINFO_ANY_OWNER , - fake.xefi_agresv); + fake.xefi_agresv, false);if (!error) { requeue_only = true;continue ;
fs/xfs/xfs_reflink.c +1 -1
@@ -618,7 +618,7 @@ xfs_reflink_cancel_cow_blocks( error =xfs_free_extent_later (*tpp, del.br_startblock, del.br_blockcount,NULL , -XFS_AG_RESV_NONE ); +XFS_AG_RESV_NONE , false);if (error)break ;
Re: [PATCH 3/7] xfs: remove __xfs_free_extent_later
Looks good:
PATCH 4/7 xfs: automatic freeing of freshly allocated unwritten space
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
As mentioned in the previous commit, online repair wants to allocate space to write out a new metadata structure, and it also wants to hedge against system crashes during repairs by logging (and later cancelling) EFIs to free the space if we crash before committing the new data structure.
Therefore, create a trio of functions to schedule automatic reaping of freshly allocated unwritten space. xfs_alloc_schedule_autoreap creates a paused EFI representing the space we just allocated. Once the allocations are made and the autoreaps scheduled, we can start writing to disk.
If the writes succeed, xfs_alloc_cancel_autoreap marks the EFI work items as stale and unpauses the pending deferred work item. Assuming that's done in the same transaction that commits the new structure into the filesystem, we guarantee that either the new object is fully visible, or that all the space gets reclaimed.
If the writes succeed but only part of an extent was used, repair must call the same _cancel_autoreap function to kill the first EFI and then log a new EFI to free the unused space. The first EFI is already committed, so it cannot be changed.
For full extents that aren't used, xfs_alloc_commit_autoreap will unpause the EFI, which results in the space being freed during the next _defer_finish cycle.
--- fs/xfs/libxfs/xfs_alloc.c | 104 +++++++++++++++++++++++++++++++++++++++++++-- fs/xfs/libxfs/xfs_alloc.h | 12 +++++ fs/xfs/xfs_extfree_item.c | 10 +++- 3 files changed, 118 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +100 -4
@@ -2522,14 +2522,15 @@ xfs_defer_agfl_block(* Add the extent to the list of extents to be free at transaction end. * The list is maintained sorted (by block number). */ -int -xfs_free_extent_later ( +static int +xfs_defer_extent_free (struct xfs_trans *tp,xfs_fsblock_t bno,xfs_filblks_t len,const struct xfs_owner_info *oinfo,enum xfs_ag_resv_type type, -bool skip_discard) +bool skip_discard, +struct xfs_defer_pending **dfpp) {struct xfs_extent_free_item *xefi;struct xfs_mount *mp = tp->t_mountp;
@@ -2577,10 +2578,105 @@ xfs_free_extent_later( XFS_FSB_TO_AGBNO(tp->t_mountp, bno), len);xfs_extent_free_get_group (mp, xefi); -xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_FREE , &xefi->xefi_list); + *dfpp =xfs_defer_add (tp,XFS_DEFER_OPS_TYPE_FREE , &xefi->xefi_list);return 0 ; } +int +xfs_free_extent_later ( +struct xfs_trans *tp, +xfs_fsblock_t bno, +xfs_filblks_t len, +const struct xfs_owner_info *oinfo, +enum xfs_ag_resv_type type, +bool skip_discard) +{ +struct xfs_defer_pending *dontcare =NULL ; + +return xfs_defer_extent_free (tp, bno, len, oinfo, type, skip_discard, + &dontcare); +} + +/* +* Set up automatic freeing of unwritten space in the filesystem. +* +* This function attached a paused deferred extent free item to the +* transaction. Pausing means that the EFI will be logged in the next +* transaction commit, but the pending EFI will not be finished until the +* pending item is unpaused. +* +* If the system goes down after the EFI has been persisted to the log but +* before the pending item is unpaused, log recovery will find the EFI, fail to +* find the EFD, and free the space. +* +* If the pending item is unpaused, the next transaction commit will log an EFD +* without freeing the space. +* +* Caller must ensure that the tp, fsbno, len, oinfo, and resv flags of the +* @args structure are set to the relevant values. +*/ +int +xfs_alloc_schedule_autoreap ( +const struct xfs_alloc_arg *args, +bool skip_discard, +struct xfs_alloc_autoreap *aarp) +{ +int error; + + error =xfs_defer_extent_free (args->tp, args->fsbno, args->len, + &args->oinfo, args->resv, skip_discard, &aarp->dfp); +if (error) +return error; + +xfs_defer_item_pause (args->tp, aarp->dfp); +return 0 ; +} + +/* +* Cancel automatic freeing of unwritten space in the filesystem. +* +* Earlier, we created a paused deferred extent free item and attached it to +* this transaction so that we could automatically roll back a new space +* allocation if the system went down. Now we want to cancel the paused work +* item by marking the EFI stale so we don't actually free the space, unpausing +* the pending item and logging an EFD. +* +* The caller generally should have already mapped the space into the ondisk +* filesystem. If the reserved space was partially used, the caller must call +* xfs_free_extent_later to create a new EFI to free the unused space. +*/ +void +xfs_alloc_cancel_autoreap ( +struct xfs_trans *tp, +struct xfs_alloc_autoreap *aarp) +{ +struct xfs_defer_pending *dfp = aarp->dfp; +struct xfs_extent_free_item *xefi; + +if (!dfp) +return ; + +list_for_each_entry (xefi, &dfp->dfp_work, xefi_list) + xefi->xefi_flags |=XFS_EFI_CANCELLED ; + +xfs_defer_item_unpause (tp, dfp); +} + +/* +* Commit automatic freeing of unwritten space in the filesystem. +* +* This unpauses an earlier _schedule_autoreap and commits to freeing the +* allocated space. Call this if none of the reserved space was used. +*/ +void +xfs_alloc_commit_autoreap ( +struct xfs_trans *tp, +struct xfs_alloc_autoreap *aarp) +{ +if (aarp->dfp) +xfs_defer_item_unpause (tp, aarp->dfp); +} +#ifdef DEBUG /* * Check if an AGF has a free extent record whose length is equal to
fs/xfs/libxfs/xfs_alloc.h +12 -0
@@ -255,6 +255,18 @@ void xfs_extent_free_get_group(struct xfs_mount *mp,#define XFS_EFI_SKIP_DISCARD (1U << 0)/* don't issue discard */ #define XFS_EFI_ATTR_FORK (1U << 1)/* freeing attr fork block */ #define XFS_EFI_BMBT_BLOCK (1U << 2)/* freeing bmap btree block */ +#define XFS_EFI_CANCELLED (1U << 3)/* dont actually free the space */ + +struct xfs_alloc_autoreap { +struct xfs_defer_pending *dfp; +}; + +int xfs_alloc_schedule_autoreap (const struct xfs_alloc_arg *args, +bool skip_discard,struct xfs_alloc_autoreap *aarp); +void xfs_alloc_cancel_autoreap (struct xfs_trans *tp, +struct xfs_alloc_autoreap *aarp); +void xfs_alloc_commit_autoreap (struct xfs_trans *tp, +struct xfs_alloc_autoreap *aarp);extern struct kmem_cache *xfs_extfree_item_cache;
fs/xfs/xfs_extfree_item.c +6 -4
@@ -381,7 +381,7 @@ xfs_trans_free_extent(uint next_extent;xfs_agblock_t agbno =XFS_FSB_TO_AGBNO (mp, xefi->xefi_startblock); -int error; +int error =0 ; oinfo.oi_owner = xefi->xefi_owner;if (xefi->xefi_flags &XFS_EFI_ATTR_FORK )
@@ -392,9 +392,11 @@ xfs_trans_free_extent(trace_xfs_bmap_free_deferred (tp->t_mountp, xefi->xefi_pag->pag_agno,0 , agbno, xefi->xefi_blockcount); - error =__xfs_free_extent (tp, xefi->xefi_pag, agbno, - xefi->xefi_blockcount, &oinfo, xefi->xefi_agresv, - xefi->xefi_flags &XFS_EFI_SKIP_DISCARD ); +if (!(xefi->xefi_flags &XFS_EFI_CANCELLED )) + error =__xfs_free_extent (tp, xefi->xefi_pag, agbno, + xefi->xefi_blockcount, &oinfo, + xefi->xefi_agresv, + xefi->xefi_flags &XFS_EFI_SKIP_DISCARD );/* * Mark the transaction dirty, even on error. This ensures the
Re: [PATCH 4/7] xfs: automatic freeing of freshly allocated unwritten space
Looks good:
PATCH 5/7 xfs: implement block reservation accounting for btrees we're staging
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new xrep_newbt structure to encapsulate a fake root for creating a staged btree cursor as well as to track all the blocks that we need to reserve in order to build that btree.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_btree_staging.h | 7 - fs/xfs/scrub/agheader_repair.c | 1 fs/xfs/scrub/common.c | 1 fs/xfs/scrub/newbt.c | 492 +++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/newbt.h | 62 +++++ fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 37 +++ 8 files changed, 598 insertions(+), 5 deletions(-) create mode 100644 fs/xfs/scrub/newbt.c create mode 100644 fs/xfs/scrub/newbt.h
fs/xfs/Makefile +1 -0
@@ -181,6 +181,7 @@ xfs-$(CONFIG_XFS_QUOTA) += scrub/quota.o ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ + newbt.o \ reap.o \ repair.o \ )
fs/xfs/libxfs/xfs_btree_staging.h +2 -5
@@ -38,11 +38,8 @@ struct xbtree_ifakeroot {/* Number of bytes available for this fork in the inode. */ unsigned int if_fork_size; -/* Fork format. */ -unsigned int if_format; - -/* Number of records. */ -unsigned int if_extents; +/* Which fork is this btree being built for? */ +int if_whichfork; };/* Cursor interactions with fake roots for inode-rooted btrees. */
fs/xfs/scrub/agheader_repair.c +1 -0
@@ -10,6 +10,7 @@#include "xfs_trans_resv.h" #include "xfs_mount.h" #include "xfs_btree.h" +#include "xfs_btree_staging.h" #include "xfs_log_format.h" #include "xfs_trans.h" #include "xfs_sb.h"
fs/xfs/scrub/common.c +1 -0
@@ -10,6 +10,7 @@#include "xfs_trans_resv.h" #include "xfs_mount.h" #include "xfs_btree.h" +#include "xfs_btree_staging.h" #include "xfs_log_format.h" #include "xfs_trans.h" #include "xfs_inode.h"
fs/xfs/scrub/newbt.c new file +492 -0
492 lines not on this page · raw message
fs/xfs/scrub/newbt.h new file +62 -0
62 lines not on this page · raw message
fs/xfs/scrub/scrub.c +2 -0
@@ -17,6 +17,8 @@#include "xfs_errortag.h" #include "xfs_error.h" #include "xfs_scrub.h" +#include "xfs_btree.h" +#include "xfs_btree_staging.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/trace.h"
fs/xfs/scrub/trace.h +37 -0
@@ -1332,6 +1332,43 @@ TRACE_EVENT(xrep_ialloc_insert, __entry->freemask) ) +DECLARE_EVENT_CLASS (xrep_newbt_extent_class, + TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, + xfs_agblock_t agbno, xfs_extlen_t len, + int64_t owner), +TP_ARGS (mp, agno, agbno, len, owner), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_agnumber_t, agno) + __field(xfs_agblock_t, agbno) + __field(xfs_extlen_t, len) +__field (int64_t, owner) + ), +TP_fast_assign ( + __entry->dev = mp->m_super->s_dev; + __entry->agno = agno; + __entry->agbno = agbno; + __entry->len = len; + __entry->owner = owner; + ), +TP_printk ("dev %d:%d agno 0x%x agbno 0x%x fsbcount 0x%x owner 0x%llx" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->agno, + __entry->agbno, + __entry->len, + __entry->owner) +); +#define DEFINE_NEWBT_EXTENT_EVENT (name) \ +DEFINE_EVENT(xrep_newbt_extent_class, name, \ + TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, \ + xfs_agblock_t agbno, xfs_extlen_t len, \ + int64_t owner), \ + TP_ARGS(mp, agno, agbno, len, owner)) +DEFINE_NEWBT_EXTENT_EVENT (xrep_newbt_alloc_ag_blocks); +DEFINE_NEWBT_EXTENT_EVENT (xrep_newbt_alloc_file_blocks); +DEFINE_NEWBT_EXTENT_EVENT (xrep_newbt_free_blocks); +DEFINE_NEWBT_EXTENT_EVENT (xrep_newbt_claim_block); +#endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */ #endif /* _TRACE_XFS_SCRUB_TRACE_H */
Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging
The data structure and support code looks fine to me, but I do have some nitpicky comments and questions:
fs/xfs/libxfs/xfs_btree_staging.h struct xbtree_ifakeroot { · line 42
Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.h +2
-5-/* Fork format. */ -unsigned int if_format; - -/* Number of records. */ -unsigned int if_extents; +/* Which fork is this btree being built for? */ +int if_whichfork;
The two removed fields seems to be unused even before this patch. Should they have been in a separate removal patch?
fs/xfs/scrub/agheader_repair.c line 16
Quoted 12 lines
fs/xfs/scrub/agheader_repair.c +1
-0@@ -10,6 +10,7 @@#include "xfs_trans_resv.h" #include "xfs_mount.h" #include "xfs_btree.h" +#include "xfs_btree_staging.h" #include "xfs_log_format.h" #include "xfs_trans.h" #include "xfs_sb.h"
I also don't think all the #include churn belongs into this patch, as the only existing header touched by it is xfs_btree_staging.h, which means that anything that didn't need it before still won't need it with the changes.
Quoted 10 lines
+/*
+ * Estimate proper slack values for a btree that's being reloaded.
+ *
+ * Under most circumstances, we'll take whatever default loading value the
+ * btree bulk loading code calculates for us. However, there are some
+ * exceptions to this rule:
+ *
+ * (1) If someone turned one of the debug knobs.
+ * (2) If this is a per-AG btree and the AG has less than ~9% space free.
+ * (3) If this is an inode btree and the FS has less than ~9% space free.
Where does this ~9% number come from? Obviously it is a low-space condition of some sort, but I wonder what are the criteria. It would be nice to document that here, even if the answer is answer is "out of thin air".
fs/xfs/scrub/agheader_repair.c
Quoted 7 lines
+ * Note that we actually use 3/32 for the comparison to avoid division.
+ */fs/xfs/scrub/agheader_repair.c +4
-0+static void + /* No further changes if there's more than 3/32ths space left. */ + if (free >= ((sz * 3) >> 5)) + return;
Is this code really in the critical path that a division (or relying on the compiler to do the right thing) is out of question? Because these shits by magic numbers are really annyoing to read (unlike say normal SECTOR_SHIFT or PAGE_SHIFT ones that are fairly easy to read).
Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging
Quoted 13 lines · Christoph Hellwig
The data structure and support code looks fine to me, but I do have some nitpicky comments and questions:
fs/xfs/libxfs/xfs_btree_staging.h struct xbtree_ifakeroot { · line 42
Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.h +2
-5-/* Fork format. */ -unsigned int if_format; - -/* Number of records. */ -unsigned int if_extents; +/* Which fork is this btree being built for? */ +int if_whichfork;The two removed fields seems to be unused even before this patch. Should they have been in a separate removal patch?
They should have been in the patch that moved if_{format,extents} into xfs_inode_fork:
daf83964a3681 ("xfs: move the per-fork nextents fields into struct xfs_ifork") f7e67b20ecbbc ("xfs: move the fork format fields into struct xfs_ifork")
but I think it just got lost in the review of all that back in May 2020. Since then the design has changed enough that I don't even think the if_whichfork field is in use anywhere:
$ git grep if_whichfork fs/xfs/libxfs/xfs_bmap_btree.c:664: cur = xfs_bmbt_init_common(mp, NULL, ip, ifake->if_whichfork); fs/xfs/libxfs/xfs_btree_staging.h:42: int if_whichfork; fs/xfs/scrub/newbt.c:117: xnr->ifake.if_whichfork = whichfork; fs/xfs/scrub/newbt.c:156: xnr->ifake.if_whichfork = XFS_DATA_FORK;
$ cd ../xfsprogs/ $ git grep if_whichfork db/bmap_inflate.c:367: ifake.if_whichfork = XFS_DATA_FORK; db/bmap_inflate.c:421: ifake.if_whichfork = XFS_DATA_FORK; libxfs/xfs_bmap_btree.c:662: cur = xfs_bmbt_init_common(mp, NULL, ip, ifake->if_whichfork); libxfs/xfs_btree_staging.h:42: int if_whichfork; repair/bulkload.c:38: bkl->ifake.if_whichfork = whichfork;
So that can all go away.
Quoted 9 lines
Quoted 9 lines
fs/xfs/scrub/agheader_repair.c +1
-0@@ -10,6 +10,7 @@#include "xfs_trans_resv.h" #include "xfs_mount.h" #include "xfs_btree.h" +#include "xfs_btree_staging.h"
Quoted 8 lines
fs/xfs/scrub/agheader_repair.c line 16
Quoted 3 lines
fs/xfs/scrub/agheader_repair.c +0
-0#include "xfs_log_format.h" #include "xfs_trans.h" #include "xfs_sb.h" I also don't think all the #include churn belongs into this patch, as the only existing header touched by it is
xfs_btree_staging.h, which means that anything that didn't need it before still won't need it with the changes.
Hmm yeah. Not sure when this detritus started accumulating here. :(
Quoted 11 lines
Quoted 10 lines
+/*
+ * Estimate proper slack values for a btree that's being reloaded.
+ *
+ * Under most circumstances, we'll take whatever default loading value the
+ * btree bulk loading code calculates for us. However, there are some
+ * exceptions to this rule:
+ *
+ * (1) If someone turned one of the debug knobs.
+ * (2) If this is a per-AG btree and the AG has less than ~9% space free.
+ * (3) If this is an inode btree and the FS has less than ~9% space free.
Quoted 4 lines
Where does this ~9% number come from? Obviously it is a low-space condition of some sort, but I wonder what are the criteria. It would be nice to document that here, even if the answer is answer is "out of thin air".
It comes from xfs_repair:
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/repair/bulkload.c?h=for-next#n114
Before xfs_btree_staging.[ch] came along, it was open coded in repair/phase5.c in a most unglorious fashion:
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/repair/phase5.c?h=v4.19.0#n1349
At that point the slack factors were arbitrary quantities per btree. The rmapbt automatically left 10 slots free; everything else left zero.
That had a noticeable effect on performance straight after mounting because touching /any/ btree would result in splits. IIRC Dave and I decided that repair should generate btree blocks that were 75% full unless space was tight. We defined tight as ~10% free to avoid repair failures and settled on 3/32 to avoid div64.
IOWs, we mostly pulled it out of thin air. ;)
OFC the other weird thing is that originally I thought that online repair would land sooner than would the retroport of xfs_repair to the new btree bulk loading code.
Quoted 4 lines
Quoted 3 lines
+ * Note that we actually use 3/32 for the comparison to avoid division.
+ */
+static void
Quoted 9 lines
fs/xfs/scrub/agheader_repair.c
Quoted 3 lines
fs/xfs/scrub/agheader_repair.c +3
-0+/* No further changes if there's more than 3/32ths space left. */ +if (free >= ((sz *3 ) >>5 )) +return ;Is this code really in the critical path that a division (or relying on the compiler to do the right thing) is out of question? Because these shits by magic numbers are really annyoing to read (unlike say normal
SECTOR_SHIFTorPAGE_SHIFTones that are fairly easy to read).
Nah, it's not performance critical. Collecting records and formatting blocks is a much bigger strain on the system than a few divisions. I'll change it to div_u64(sz, 10).
Hmm now that I look at it, I've also noticed that even the lowspace btree rebuild in userspace will leave two open slots per block.
--D
Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging
Quoted 7 lines · Darrick J. Wong
That had a noticeable effect on performance straight after mounting because touching /any/ btree would result in splits. IIRC Dave and I decided that repair should generate btree blocks that were 75% full unless space was tight. We defined tight as ~10% free to avoid repair failures and settled on 3/32 to avoid div64.
IOWs, we mostly pulled it out of thin air. ;)
Maybe throw a little comment about this in.
Re: [PATCH 5/7] xfs: implement block reservation accounting for btrees we're staging
Quoted 10 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
That had a noticeable effect on performance straight after mounting because touching /any/ btree would result in splits. IIRC Dave and I decided that repair should generate btree blocks that were 75% full unless space was tight. We defined tight as ~10% free to avoid repair failures and settled on 3/32 to avoid div64.
IOWs, we mostly pulled it out of thin air. ;)
Maybe throw a little comment about this in.
Yeah, I'll paste the sordid history into the commit message when I change the code to use div_u64.
--D
PATCH 6/7 xfs: log EFIs for all btree blocks being used to stage a btree
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
We need to log EFIs for every extent that we allocate for the purpose of staging a new btree so that if we fail then the blocks will be freed during log recovery. Use the autoreaping mechanism provided by the previous patch to attach paused freeing work to the scrub transaction. We can then mark the EFIs stale if we decide to commit the new btree, or we can unpause the EFIs if we decide to abort the repair.
--- fs/xfs/scrub/newbt.c | 34 ++++++++++++++++++++++++++-------- fs/xfs/scrub/newbt.h | 3 +++ 2 files changed, 29 insertions(+), 8 deletions(-)
fs/xfs/scrub/newbt.c +26 -8
@@ -136,6 +136,7 @@ xrep_newbt_add_blocks( {struct xfs_mount *mp = xnr->sc->mp;struct xrep_newbt_resv *resv; +int error; resv =kmalloc (sizeof (struct xrep_newbt_resv ),XCHK_GFP_FLAGS );if (!resv)
@@ -147,8 +148,18 @@ xrep_newbt_add_blocks( resv->used =0 ; resv->pag =xfs_perag_hold (pag); +ASSERT (xnr->oinfo.oi_offset ==0 ); + + error =xfs_alloc_schedule_autoreap (args, true, &resv->autoreap); +if (error) + goto out_pag; +list_add_tail (&resv->list, &xnr->resv_list);return 0 ; +out_pag: +xfs_perag_put (resv->pag); +kfree (resv); +return error; }/* Don't let our allocation hint take us beyond this AG */
@@ -327,16 +338,21 @@ xrep_newbt_free_extent(if (!btree_committed || resv->used ==0 ) {/* * If we're not committing a new btree or we didn't use the -* space reservation, free the entire space extent. +* space reservation, let the existing EFI free the entire +* space extent. */ - goto free; +trace_xrep_newbt_free_blocks (sc->mp, resv->pag->pag_agno, + free_agbno, free_aglen, xnr->oinfo.oi_owner); +xfs_alloc_commit_autoreap (sc->tp, &resv->autoreap); +return 1 ; }/* -* We used space and committed the btree. Remove the written blocks -* from the reservation and possibly log a new EFI to free any unused -* reservation space. +* We used space and committed the btree. Cancel the autoreap, remove +* the written blocks from the reservation, and possibly log a new EFI +* to free any unused reservation space. */ +xfs_alloc_cancel_autoreap (sc->tp, &resv->autoreap); free_agbno += resv->used; free_aglen -= resv->used;
@@ -348,7 +364,6 @@ xrep_newbt_free_extent(ASSERT (xnr->resv !=XFS_AG_RESV_AGFL ); -free:/* * Use EFIs to free the reservations. This reduces the chance * that we leak blocks if the system goes down.
@@ -408,9 +423,10 @@ xrep_newbt_free(/* * If we still have reservations attached to @newbt, cleanup must have * failed and the filesystem is about to go down. Clean up the incore -* reservations. +* reservations and try to commit to freeing the space we used. */ list_for_each_entry_safe (resv, n, &xnr->resv_list, list) { +xfs_alloc_commit_autoreap (sc->tp, &resv->autoreap);list_del (&resv->list);xfs_perag_put (resv->pag);kfree (resv);
@@ -488,5 +504,7 @@ xrep_newbt_claim_block( agbno)); else ptr->s = cpu_to_be32(agbno); -return 0 ; + +/* Relog all the EFIs. */ +return xrep_defer_finish (xnr->sc); }
fs/xfs/scrub/newbt.h +3 -0
@@ -12,6 +12,9 @@ struct xrep_newbt_resv {struct xfs_perag *pag; +/* Auto-freeing this reservation if we don't commit. */ +struct xfs_alloc_autoreap autoreap; +/* AG block of the extent we reserved. */ xfs_agblock_t agbno;
Re: [PATCH 6/7] xfs: log EFIs for all btree blocks being used to stage a btree
Looks good:
PATCH 7/7 xfs: force small EFIs for reaping btree extents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Introduce the concept of a defer ops barrier to separate consecutively queued pending work items of the same type. With a barrier in place, the two work items will be tracked separately, and receive separate log intent items. The goal here is to prevent reaping of old metadata blocks from creating unnecessarily huge EFIs that could then run the risk of overflowing the scrub transaction.
--- fs/xfs/libxfs/xfs_defer.c | 83 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_defer.h | 3 ++ fs/xfs/scrub/reap.c | 5 +++ 3 files changed, 91 insertions(+)
fs/xfs/libxfs/xfs_defer.c +83 -0
@@ -181,6 +181,58 @@ static struct kmem_cache *xfs_defer_pending_cache;* Note that the continuation requested between t2 and t3 is likely to * reoccur. */ +STATIC struct xfs_log_item * +xfs_defer_barrier_create_intent ( +struct xfs_trans *tp, +struct list_head *items, +unsigned int count, +bool sort) +{ +return NULL ; +} + +STATIC void +xfs_defer_barrier_abort_intent ( +struct xfs_log_item *intent) +{ +/* empty */ +} + +STATIC struct xfs_log_item * +xfs_defer_barrier_create_done ( +struct xfs_trans *tp, +struct xfs_log_item *intent, +unsigned int count) +{ +return NULL ; +} + +STATIC int +xfs_defer_barrier_finish_item ( +struct xfs_trans *tp, +struct xfs_log_item *done, +struct list_head *item, +struct xfs_btree_cur **state) +{ +ASSERT (0 ); +return -EFSCORRUPTED ; +} + +STATIC void +xfs_defer_barrier_cancel_item ( +struct list_head *item) +{ +ASSERT (0 ); +} + +static const struct xfs_defer_op_type xfs_barrier_defer_type = { + .max_items =1 , + .create_intent = xfs_defer_barrier_create_intent, + .abort_intent = xfs_defer_barrier_abort_intent, + .create_done = xfs_defer_barrier_create_done, + .finish_item = xfs_defer_barrier_finish_item, + .cancel_item = xfs_defer_barrier_cancel_item, +};static const struct xfs_defer_op_type *defer_op_types[] = { [XFS_DEFER_OPS_TYPE_BMAP ] = &xfs_bmap_update_defer_type,
@@ -189,6 +241,7 @@ static const struct xfs_defer_op_type *defer_op_types[] = { [XFS_DEFER_OPS_TYPE_FREE] = &xfs_extent_free_defer_type, [XFS_DEFER_OPS_TYPE_AGFL_FREE] = &xfs_agfl_free_defer_type, [XFS_DEFER_OPS_TYPE_ATTR] = &xfs_attr_defer_type, + [XFS_DEFER_OPS_TYPE_BARRIER] = &xfs_barrier_defer_type, };/*
@@ -1036,3 +1089,33 @@ xfs_defer_item_unpause(trace_xfs_defer_item_unpause (tp->t_mountp, dfp); } + +/* +* Add a defer ops barrier to force two otherwise adjacent deferred work items +* to be tracked separately and have separate log items. +*/ +void +xfs_defer_add_barrier ( +struct xfs_trans *tp) +{ +struct xfs_defer_pending *dfp; + +ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES ); + +/* If the last defer op added was a barrier, we're done. */ +if (!list_empty (&tp->t_dfops)) { + dfp =list_last_entry (&tp->t_dfops, + struct xfs_defer_pending, dfp_list); +if (dfp->dfp_type ==XFS_DEFER_OPS_TYPE_BARRIER ) +return ; + } + + dfp =kmem_cache_zalloc (xfs_defer_pending_cache, +GFP_NOFS | __GFP_NOFAIL); + dfp->dfp_type =XFS_DEFER_OPS_TYPE_BARRIER ; +INIT_LIST_HEAD (&dfp->dfp_work); +list_add_tail (&dfp->dfp_list, &tp->t_dfops); + +trace_xfs_defer_add_item (tp->t_mountp, dfp,NULL ); + dfp->dfp_count++; +}
fs/xfs/libxfs/xfs_defer.h +3 -0
@@ -20,6 +20,7 @@ enum xfs_defer_ops_type {XFS_DEFER_OPS_TYPE_FREE ,XFS_DEFER_OPS_TYPE_AGFL_FREE ,XFS_DEFER_OPS_TYPE_ATTR , +XFS_DEFER_OPS_TYPE_BARRIER , XFS_DEFER_OPS_TYPE_MAX, };
@@ -141,4 +142,6 @@ void xfs_defer_resources_rele(struct xfs_defer_resources *dres);int __init xfs_defer_init_item_caches (void );void xfs_defer_destroy_item_caches (void ); +void xfs_defer_add_barrier (struct xfs_trans *tp); +#endif /* __XFS_DEFER_H__ */
fs/xfs/scrub/reap.c +5 -0
@@ -31,6 +31,7 @@#include "xfs_da_btree.h" #include "xfs_attr.h" #include "xfs_attr_remote.h" +#include "xfs_defer.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/trace.h"
@@ -409,6 +410,8 @@ xreap_agextent_iter(/* * Use deferred frees to get rid of the old btree blocks to try to * minimize the window in which we could crash and lose the old blocks. +* Add a defer ops barrier every other extent to avoid stressing the +* system with large EFIs. */ error =xfs_free_extent_later (sc->tp, fsbno, *aglenp, rs->oinfo, rs->resv, true);
@@ -416,6 +419,8 @@ xreap_agextent_iter(return error; rs->deferred++; +if (rs->deferred %2 ==0 ) +xfs_defer_add_barrier (sc->tp);return 0 ; }
Re: [PATCH 7/7] xfs: force small EFIs for reaping btree extents
This generally looks good:
But as mentioned earlier I think it would be nice to share some code between the normal xfs_defer_add and xfs_defer_add_barrier.
This would be the fold for this patch on top of the one previously posted and a light merge fix for the pausing between:
fs/xfs/libxfs/xfs_defer.c +23 -27
@@ -753,6 +753,22 @@ xfs_defer_can_append(return true; } +/* Create a new pending item at the end of the intake list. */ +static struct xfs_defer_pending * +xfs_defer_alloc ( +struct xfs_trans *tp, +enum xfs_defer_ops_type type) +{ +struct xfs_defer_pending *dfp; + + dfp =kmem_cache_zalloc (xfs_defer_pending_cache, +GFP_NOFS | __GFP_NOFAIL); + dfp->dfp_type = type; +INIT_LIST_HEAD (&dfp->dfp_work); +list_add_tail (&dfp->dfp_list, &tp->t_dfops); +return dfp; +}; +/* Add an item for later deferred processing. */ struct xfs_defer_pending * xfs_defer_add(
@@ -760,29 +776,18 @@ xfs_defer_add(enum xfs_defer_ops_type type, struct list_head *li) { - struct xfs_defer_pending *dfp = NULL; const struct xfs_defer_op_type *ops = defer_op_types[type]; +struct xfs_defer_pending *dfp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );BUILD_BUG_ON (ARRAY_SIZE (defer_op_types) !=XFS_DEFER_OPS_TYPE_MAX ); dfp =xfs_defer_find (tp, type); -if (!dfp || !xfs_defer_can_append (dfp, ops)) { -/* Create a new pending item at the end of the intake list. */ - dfp =kmem_cache_zalloc (xfs_defer_pending_cache, -GFP_NOFS | __GFP_NOFAIL); - dfp->dfp_type = type; - dfp->dfp_intent =NULL ; - dfp->dfp_done =NULL ; - dfp->dfp_count =0 ; -INIT_LIST_HEAD (&dfp->dfp_work); -list_add_tail (&dfp->dfp_list, &tp->t_dfops); - } - +if (!dfp || !xfs_defer_can_append (dfp, ops)) + dfp =xfs_defer_alloc (tp, type);list_add_tail (li, &dfp->dfp_work);trace_xfs_defer_add_item (tp->t_mountp, dfp, li); dfp->dfp_count++; -return dfp; }
@@ -1106,19 +1111,10 @@ xfs_defer_add_barrier(ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );/* If the last defer op added was a barrier, we're done. */ -if (!list_empty (&tp->t_dfops)) { - dfp =list_last_entry (&tp->t_dfops, - struct xfs_defer_pending, dfp_list); -if (dfp->dfp_type ==XFS_DEFER_OPS_TYPE_BARRIER ) -return ; - } - - dfp =kmem_cache_zalloc (xfs_defer_pending_cache, -GFP_NOFS | __GFP_NOFAIL); - dfp->dfp_type =XFS_DEFER_OPS_TYPE_BARRIER ; -INIT_LIST_HEAD (&dfp->dfp_work); -list_add_tail (&dfp->dfp_list, &tp->t_dfops); - + dfp =xfs_defer_find (tp,XFS_DEFER_OPS_TYPE_BARRIER ); +if (dfp) +return ; + dfp =xfs_defer_alloc (tp,XFS_DEFER_OPS_TYPE_BARRIER );trace_xfs_defer_add_item (tp->t_mountp, dfp,NULL ); dfp->dfp_count++; }
Darrick J. Wong
AppliedRe: [PATCH 7/7] xfs: force small EFIs for reaping btree extents
Quoted 9 lines · Christoph Hellwig
This generally looks good:
Reviewed-by Christoph Hellwig <[email protected]>But as mentioned earlier I think it would be nice to share some code between the normal
xfs_defer_addandxfs_defer_add_barrier.This would be the fold for this patch on top of the one previously posted and a light merge fix for the pausing between:
Applied. Thanks for the quick cleanup!
--D
Quoted 85 lines
fs/xfs/libxfs/xfs_defer.c +23
-27@@ -753,6 +753,22 @@ xfs_defer_can_append(return true; } +/* Create a new pending item at the end of the intake list. */ +static struct xfs_defer_pending * +xfs_defer_alloc ( +struct xfs_trans *tp, +enum xfs_defer_ops_type type) +{ +struct xfs_defer_pending *dfp; + + dfp =kmem_cache_zalloc (xfs_defer_pending_cache, +GFP_NOFS | __GFP_NOFAIL); + dfp->dfp_type = type; +INIT_LIST_HEAD (&dfp->dfp_work); +list_add_tail (&dfp->dfp_list, &tp->t_dfops); +return dfp; +}; +/* Add an item for later deferred processing. */ struct xfs_defer_pending * xfs_defer_add(@@ -760,29 +776,18 @@ xfs_defer_add(enum xfs_defer_ops_type type, struct list_head *li) { - struct xfs_defer_pending *dfp = NULL; const struct xfs_defer_op_type *ops = defer_op_types[type]; +struct xfs_defer_pending *dfp;ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );BUILD_BUG_ON (ARRAY_SIZE (defer_op_types) !=XFS_DEFER_OPS_TYPE_MAX ); dfp =xfs_defer_find (tp, type); -if (!dfp || !xfs_defer_can_append (dfp, ops)) { -/* Create a new pending item at the end of the intake list. */ - dfp =kmem_cache_zalloc (xfs_defer_pending_cache, -GFP_NOFS | __GFP_NOFAIL); - dfp->dfp_type = type; - dfp->dfp_intent =NULL ; - dfp->dfp_done =NULL ; - dfp->dfp_count =0 ; -INIT_LIST_HEAD (&dfp->dfp_work); -list_add_tail (&dfp->dfp_list, &tp->t_dfops); - } - +if (!dfp || !xfs_defer_can_append (dfp, ops)) + dfp =xfs_defer_alloc (tp, type);list_add_tail (li, &dfp->dfp_work);trace_xfs_defer_add_item (tp->t_mountp, dfp, li); dfp->dfp_count++; -return dfp; }@@ -1106,19 +1111,10 @@ xfs_defer_add_barrier(ASSERT (tp->t_flags &XFS_TRANS_PERM_LOG_RES );/* If the last defer op added was a barrier, we're done. */ -if (!list_empty (&tp->t_dfops)) { - dfp =list_last_entry (&tp->t_dfops, - struct xfs_defer_pending, dfp_list); -if (dfp->dfp_type ==XFS_DEFER_OPS_TYPE_BARRIER ) -return ; - } - - dfp =kmem_cache_zalloc (xfs_defer_pending_cache, -GFP_NOFS | __GFP_NOFAIL); - dfp->dfp_type =XFS_DEFER_OPS_TYPE_BARRIER ; -INIT_LIST_HEAD (&dfp->dfp_work); -list_add_tail (&dfp->dfp_list, &tp->t_dfops); - + dfp =xfs_defer_find (tp,XFS_DEFER_OPS_TYPE_BARRIER ); +if (dfp) +return ; + dfp =xfs_defer_alloc (tp,XFS_DEFER_OPS_TYPE_BARRIER );trace_xfs_defer_add_item (tp->t_mountp, dfp,NULL ); dfp->dfp_count++; }
cover xfs: prepare repair for bulk loading
In reply to Darrick J. WongHi all,
Before we start merging the online repair functions, let's improve the bulk loading code a bit. First, we need to fix a misinteraction between the AIL and the btree bulkloader wherein the delwri at the end of the bulk load fails to queue a buffer for writeback if it happens to be on the AIL list.
Second, we introduce a defer ops barrier object so that the process of reaping blocks after a repair cannot queue more than two extents per EFI log item. This increases our exposure to leaking blocks if the system goes down during a reap, but also should prevent transaction overflows, which result in the system going down.
Third, we change the bulkloader itself to copy multiple records into a block if possible, and add some debugging knobs so that developers can control the slack factors, just like they can do for xfs_repair.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-prep-for-bulk-loading
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-prep-for-bulk-loading
--- fs/xfs/libxfs/xfs_btree.c | 2 + fs/xfs/libxfs/xfs_btree.h | 3 ++ fs/xfs/libxfs/xfs_btree_staging.c | 67 +++++++++++++++++++++++++------------ fs/xfs/libxfs/xfs_btree_staging.h | 25 +++++++++++--- fs/xfs/scrub/newbt.c | 11 ++++-- fs/xfs/xfs_buf.c | 47 ++++++++++++++++++++++++-- fs/xfs/xfs_buf.h | 1 + fs/xfs/xfs_globals.c | 12 +++++++ fs/xfs/xfs_sysctl.h | 2 + fs/xfs/xfs_sysfs.c | 54 ++++++++++++++++++++++++++++++ 10 files changed, 189 insertions(+), 35 deletions(-)
PATCH 1/4 xfs: force all buffers to be written during btree bulk load
From: Darrick J. Wong <[email protected]>
While stress-testing online repair of btrees, I noticed periodic assertion failures from the buffer cache about buffer readers encountering buffers with DELWRI_Q set, even though the btree bulk load had already committed and the buffer itself wasn't on any delwri list.
I traced this to a misunderstanding of how the delwri lists work, particularly with regards to the AIL's buffer list. If a buffer is logged and committed, the buffer can end up on that AIL buffer list. If btree repairs are run twice in rapid succession, it's possible that the first repair will invalidate the buffer and free it before the next time the AIL wakes up. This clears DELWRI_Q from the buffer state.
If the second repair allocates the same block, it will then recycle the buffer to start writing the new btree block. Meanwhile, if the AIL wakes up and walks the buffer list, it will ignore the buffer because it can't lock it, and go back to sleep.
When the second repair calls delwri_queue to put the buffer on the list of buffers to write before committing the new btree, it will set DELWRI_Q again, but since the buffer hasn't been removed from the AIL's buffer list, it won't add it to the bulkload buffer's list.
This is incorrect, because the bulkload caller relies on delwri_submit to ensure that all the buffers have been sent to disk /before/ committing the new btree root pointer. This ordering requirement is required for data consistency.
Worse, the AIL won't clear DELWRI_Q from the buffer when it does finally drop it, so the next thread to walk through the btree will trip over a debug assertion on that flag.
To fix this, create a new function that waits for the buffer to be removed from any other delwri lists before adding the buffer to the caller's delwri list. By waiting for the buffer to clear both the delwri list and any potential delwri wait list, we can be sure that repair will initiate writes of all buffers and report all write errors back to userspace instead of committing the new structure.
--- fs/xfs/libxfs/xfs_btree_staging.c | 4 +-- fs/xfs/xfs_buf.c | 47 ++++++++++++++++++++++++++++++++++--- fs/xfs/xfs_buf.h | 1 + 3 files changed, 45 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_btree_staging.c +1 -3
@@ -342,9 +342,7 @@ xfs_btree_bload_drop_buf(if (*bpp ==NULL )return ; -if (!xfs_buf_delwri_queue (*bpp, buffers_list)) -ASSERT (0 ); - +xfs_buf_delwri_queue_here (*bpp, buffers_list);xfs_buf_relse (*bpp); *bpp =NULL ; }
fs/xfs/xfs_buf.c +43 -4
83 lines not on this page · raw message
fs/xfs/xfs_buf.h +1 -0
@@ -319,6 +319,7 @@ extern void xfs_buf_stale(struct xfs_buf *bp);/* Delayed Write Buffer Routines */ extern void xfs_buf_delwri_cancel (struct list_head *);extern bool xfs_buf_delwri_queue (struct xfs_buf *,struct list_head *); +void xfs_buf_delwri_queue_here (struct xfs_buf *bp,struct list_head *bl);extern int xfs_buf_delwri_submit (struct list_head *);extern int xfs_buf_delwri_submit_nowait (struct list_head *);extern int xfs_buf_delwri_pushbuf (struct xfs_buf *,struct list_head *);
Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load
The code makes me feel we're fixing the wrong thing, but I have to admin I don't fully understand it. Let me go step by step.
Quoted 4 lines
While stress-testing online repair of btrees, I noticed periodic assertion failures from the buffer cache about buffer readers encountering buffers with
DELWRI_Qset, even though the btree bulk load had already committed and the buffer itself wasn't on any delwri list.
What assert do these buffer reader hit? The two I can spot that are in the read path in the broader sense are:
1) in xfs_buf_find_lock for stale buffers.
2) in __xfs_buf_submit just before I/O submission
Quoted 6 lines
I traced this to a misunderstanding of how the delwri lists work, particularly with regards to the AIL's buffer list. If a buffer is logged and committed, the buffer can end up on that AIL buffer list. If btree repairs are run twice in rapid succession, it's possible that the first repair will invalidate the buffer and free it before the next time the AIL wakes up. This clears
DELWRI_Qfrom the buffer state.
Where "this clears" is xfs_buf_stale called from xfs_btree_free_block via xfs_trans_binval?
Quoted 2 lines
If the second repair allocates the same block, it will then recycle the buffer to start writing the new btree block.
If my above theory is correct: how do we end up reusing a stale buffer? If not, what am I misunderstanding above?
Quoted 3 lines
Meanwhile, if the AIL
wakes up and walks the buffer list, it will ignore the buffer because it can't lock it, and go back to sleep.
And I think this is where the trouble starts - we have a buffer that is left on some delwri list, but with the _XBF_DELWRI_Q flag cleared, it is stale and we then reuse it. I don't think we just need to kick it off the delwri list just for btree staging, but in general.
Quoted 6 lines
When the second repair calls
delwri_queueto put the buffer on the list of buffers to write before committing the new btree, it will setDELWRI_Qagain, but since the buffer hasn't been removed from the AIL's buffer list, it won't add it to the bulkload buffer's list.
Quoted 8 lines
This is incorrect, because the bulkload caller relies on
delwri_submitto ensure that all the buffers have been sent to disk /before/ committing the new btree root pointer. This ordering requirement is required for data consistency.Worse, the AIL won't clear
DELWRI_Qfrom the buffer when it does finally drop it, so the next thread to walk through the btree will trip over a debug assertion on that flag.
Where do it finally drop it?
Quoted 6 lines
To fix this, create a new function that waits for the buffer to be removed from any other delwri lists before adding the buffer to the caller's delwri list. By waiting for the buffer to clear both the delwri list and any potential delwri wait list, we can be sure that repair will initiate writes of all buffers and report all write errors back to userspace instead of committing the new structure.
If my understanding above is correct this just papers over the bug that a buffer that is marked stale and can be reused for something else is left on a delwri list. I've entirely thought about all the consequence, but here is what I'd try:
- if xfs_buf_find_lock finds a stale buffer with _XBF_DELWRI_Q call your new wait code instead of asserting (probably only for the !trylock case)
- make sure we don't leak DELWRI_Q
Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load
Quoted 8 lines · Christoph Hellwig
The code makes me feel we're fixing the wrong thing, but I have to admin I don't fully understand it. Let me go step by step.
Quoted 4 lines
While stress-testing online repair of btrees, I noticed periodic assertion failures from the buffer cache about buffer readers encountering buffers with
DELWRI_Qset, even though the btree bulk load had already committed and the buffer itself wasn't on any delwri list.
Quoted 5 lines
What assert do these buffer reader hit? The two I can spot that are in the read path in the broader sense are:
1) in
xfs_buf_find_lockfor stale buffers.
2) in__xfs_buf_submitjust before I/O submission
The second assert:
AIL: Repair0: Repair1:
pin buffer Xdelwri_queue:
set DELWRI_Q
add to delwri list
stale buf X:
clear DELWRI_Q
does not clear b_list
free space X
commit
delwri_submit # oops
Though there's more to this.
Quoted 9 lines
Quoted 6 lines
I traced this to a misunderstanding of how the delwri lists work, particularly with regards to the AIL's buffer list. If a buffer is logged and committed, the buffer can end up on that AIL buffer list. If btree repairs are run twice in rapid succession, it's possible that the first repair will invalidate the buffer and free it before the next time the AIL wakes up. This clears
DELWRI_Qfrom the buffer state.Where "this clears" is
xfs_buf_stalecalled fromxfs_btree_free_blockviaxfs_trans_binval?
Yes. I'll rework the last sentence to read: "Marking the buffer stale clears DELWRI_Q from the buffer state without removing the buffer from its delwri list."
Quoted 5 lines
Quoted 2 lines
If the second repair allocates the same block, it will then recycle the buffer to start writing the new btree block.
If my above theory is correct: how do we end up reusing a stale buffer? If not, what am I misunderstanding above?
If we free a metadata buffer, we'll mark it stale, update the bnobt, and add an entry to the extent busy list. If a later repair finds:
1. The same free space in the bnobt;
2. The free space exactly coincides with one extent busy list entry;
3. The entry isn't in the middle of discarding the block;
Then the allocator will remove the extent busy list entry and let us have the space. At that point we could have a stale buffer that's also on one of the AIL's lists:
AIL: Repair0: Repair1:
pin buffer Xdelwri_queue:
set DELWRI_Q
add to delwri list
stale buf X:
clear DELWRI_Q
does not clear b_list
free space X
commit
find free space X
get buffer
rewrite buffer delwri_queue:
set DELWRI_Q
already on a list, do not add commit
BAD: committed tree root before all blocks written
delwri_submit # too late now
I could demonstrate this by injecting dmerror while delwri(ting) the new btree blocks, and observing that some of the EIOs would not trigger the rollback but instead would trip IO errors in the AIL.
Quoted 7 lines
Quoted 2 lines
wakes up and walks the buffer list, it will ignore the buffer because it can't lock it, and go back to sleep.
And I think this is where the trouble starts - we have a buffer that is left on some delwri list, but with the
_XBF_DELWRI_Qflag cleared, it is stale and we then reuse it. I don't think we just need to kick it off the delwri list just for btree staging, but in general.
...but how to do that? We don't know whose delwri list the buffer's on, let alone how to lock the list so that we can remove the buffer from that list.
(Oh, you have some suggestions below.)
Quoted 10 lines
Quoted 10 lines
When the second repair calls
delwri_queueto put the buffer on the list of buffers to write before committing the new btree, it will setDELWRI_Qagain, but since the buffer hasn't been removed from the AIL's buffer list, it won't add it to the bulkload buffer's list.This is incorrect, because the bulkload caller relies on
delwri_submitto ensure that all the buffers have been sent to disk /before/ committing the new btree root pointer. This ordering requirement is required for data consistency.
Quoted 5 lines
Quoted 3 lines
Worse, the AIL won't clear
DELWRI_Qfrom the buffer when it does finally drop it, so the next thread to walk through the btree will trip over a debug assertion on that flag.Where do it finally drop it?
TBH, it's been so long that I can't remember anymore. :(
Quoted 10 lines
Quoted 6 lines
To fix this, create a new function that waits for the buffer to be removed from any other delwri lists before adding the buffer to the caller's delwri list. By waiting for the buffer to clear both the delwri list and any potential delwri wait list, we can be sure that repair will initiate writes of all buffers and report all write errors back to userspace instead of committing the new structure.
If my understanding above is correct this just papers over the bug that a buffer that is marked stale and can be reused for something else is left on a delwri list.
I'm not sure it's a bug for *most* code that encounters it. Regular transactions don't directly use the delwri lists, so it will never see that at all. If the buffer gets rewritten and logged, then it'll just end up on the AIL's delwri buffer list again.
The other delwri_submit users don't seem to care if the buffer gets written directly by delwri_submit or by the AIL. In the first case, the caller will get the EIO and force a shutdown; in the second case, the AIL will shut down the log.
Weirdly, xfs_bwrite clears DELWRI_Q but doesn't necessary shut down the fs if the write fails.
Every time I revisit this patch I wonder if DELWRI_Q is misnamed -- does it mean "b_list is active"? Or does it merely mean "another thread will write this buffer via b_list if nobody gets there first"? I think it's the second, since it's easy enough to list_empty().
It's only repair that requires this new behavior that all new buffers have to be written through the delwri list, and only because it actually /can/ cancel the transaction by finishing the autoreap EFIs.
Quoted 7 lines
I've entirely thought about all the
consequence, but here is what I'd try:- if
xfs_buf_find_lockfinds a stale buffer with_XBF_DELWRI_Qcall your new wait code instead of asserting (probably only for the !trylock case)
- make sure we don't leakDELWRI_Q
Way back when I first discovered this, my first impulse was to make xfs_buf_stale wait for DELWRI_Q to clear. That IIRC didn't work because a transaction could hold an inode that the AIL will need to lock. I think xfs_buf_find_lock would have the same problem.
Seeing as repair is the only user with the requirement that it alone can issue writes for the delwri list buffers, that's why I went with what's in this patch.
Thank you for persevering so far! :D
--D
Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load
Quoted 8 lines
Quoted 3 lines
If my understanding above is correct this just papers over the bug that a buffer that is marked stale and can be reused for something else is left on a delwri list.
I'm not sure it's a bug for *most* code that encounters it. Regular transactions don't directly use the delwri lists, so it will never see that at all. If the buffer gets rewritten and logged, then it'll just end up on the AIL's delwri buffer list again.
Where "regular transactions" means data I/O, yes. All normal metadata I/O eventually ends up on a delwri list. And we want it on that delwri list after actually updating the contents.
Quoted 4 lines
Every time I revisit this patch I wonder if
DELWRI_Qis misnamed -- does it mean "b_listis active"? Or does it merely mean "another thread will write this buffer viab_listif nobody gets there first"? I think it's the second, since it's easy enough tolist_empty().
I think it is misnamed and not clearly defined, and we should probably fix that. Note that least the current list_empty() usage isn't quite correct either without a lock held by the delwri list owner. We at least need a list_empty_careful for a racey but still save check.
Thinking out loud I wonder if we can just kill XBF_DELWRI_Q entirely. Except for various asserts we mostly use it in reverse polarity, that is to cancel delwri writeout for stale buffers. How about just skipping XBF_STALE buffers on the delwri list directly and not bother with DELWRI_Q? With that we can then support multiple delwri lists that don't get into each others way using say an allocating xarray instead of the invase list and avoid this whole mess.
Let me try to prototype this..
Quoted 3 lines
It's only repair that requires this new behavior that all new buffers have to be written through the delwri list, and only because it actually /can/ cancel the transaction by finishing the autoreap EFIs.
Right now yes. But I think the delwri behavior is a land mine, and this just happens to be the first user to trigger it. Edit: while looking through the DELWRI_Q usage I noticed xfs_qm_flush_one, which seems to deal with what is at least a somewhat related issue based on the comments there.
Quoted 4 lines
Way back when I first discovered this, my first impulse was to make
xfs_buf_stalewait forDELWRI_Qto clear. That IIRC didn't work because a transaction could hold an inode that the AIL will need to lock. I thinkxfs_buf_find_lockwould have the same problem.
Yes, that makes sense. Would be great to document such details in the commit message..
Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load
Quoted 18 lines · Christoph Hellwig
Quoted 4 lines
Every time I revisit this patch I wonder if
DELWRI_Qis misnamed -- does it mean "b_listis active"? Or does it merely mean "another thread will write this buffer viab_listif nobody gets there first"? I think it's the second, since it's easy enough tolist_empty().I think it is misnamed and not clearly defined, and we should probably fix that. Note that least the current
list_empty()usage isn't quite correct either without a lock held by the delwri list owner. We at least need alist_empty_carefulfor a racey but still save check.Thinking out loud I wonder if we can just kill
XBF_DELWRI_Qentirely. Except for various asserts we mostly use it in reverse polarity, that is to cancel delwri writeout for stale buffers. How about just skippingXBF_STALEbuffers on the delwri list directly and not bother withDELWRI_Q? With that we can then support multiple delwri lists that don't get into each others way using say an allocating xarray instead of the invase list and avoid this whole mess.
Quoted 1 line
Let me try to prototype this..
Ok, I spent half the day prototyping this and it passes basic sanity checks:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/xfs-multiple-delwri-lists
My conclusions from that is:
1) it works
2) I think it is the right to do
3) it needs a lot more work
4) we can't block the online repair work on it
so I guess we'll need to go with the approach in this patch for now, maybe with a better commit log, and I'll look into finishing this work some time in the future.
Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load
Quoted 28 lines · Christoph Hellwig
Quoted 18 lines · Christoph Hellwig
Quoted 4 lines
Every time I revisit this patch I wonder if
DELWRI_Qis misnamed -- does it mean "b_listis active"? Or does it merely mean "another thread will write this buffer viab_listif nobody gets there first"? I think it's the second, since it's easy enough tolist_empty().I think it is misnamed and not clearly defined, and we should probably fix that. Note that least the current
list_empty()usage isn't quite correct either without a lock held by the delwri list owner. We at least need alist_empty_carefulfor a racey but still save check.Thinking out loud I wonder if we can just kill
XBF_DELWRI_Qentirely. Except for various asserts we mostly use it in reverse polarity, that is to cancel delwri writeout for stale buffers. How about just skippingXBF_STALEbuffers on the delwri list directly and not bother withDELWRI_Q? With that we can then support multiple delwri lists that don't get into each others way using say an allocating xarray instead of the invase list and avoid this whole mess.Quoted 1 line
Let me try to prototype this..
Ok, I spent half the day prototyping this and it passes basic sanity checks:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/xfs-multiple-delwri-lists
My conclusions from that is:
Quoted 8 lines
1) it works
2) I think it is the right to do
3) it needs a lot more work
4) we can't block the online repair work on itso I guess we'll need to go with the approach in this patch for now, maybe with a better commit log, and I'll look into finishing this work some time in the future.
<nod> I think an xarray version of this would be less clunky than xfs_delwri_buf with three pointers.
Also note Dave's comments on the v25 posting of this series:
https://lore.kernel.org/linux-xfs/ZJJa7Cnni0mb%[email protected]/
--D
Re: [PATCH 1/4] xfs: force all buffers to be written during btree bulk load
Quoted 6 lines · Darrick J. Wong
Quoted 3 lines
so I guess we'll need to go with the approach in this patch for now, maybe with a better commit log, and I'll look into finishing this work some time in the future.
<nod> I think an xarray version of this would be less clunky than
xfs_delwri_bufwith three pointers.
It would, but xarray indices are unsigned longs, so we couldn't simply use the block number for it. So unless we decided to give up on building XFS on 32-bit kernels (which would nice for other reasons) it would need something more complicated.
PATCH 2/4 xfs: add debug knobs to control btree bulk load slack factors
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some debug knobs so that we can control the leaf and node block slack when rebuilding btrees.
--- fs/xfs/scrub/newbt.c | 10 ++++++--- fs/xfs/xfs_globals.c | 12 +++++++++++ fs/xfs/xfs_sysctl.h | 2 ++ fs/xfs/xfs_sysfs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 3 deletions(-)
fs/xfs/scrub/newbt.c +7 -3
@@ -47,9 +47,13 @@ xrep_newbt_estimate_slack(uint64_t free;uint64_t sz; -/* Let the btree code compute the default slack values. */ - bload->leaf_slack =-1 ; - bload->node_slack =-1 ; +/* +* The xfs_globals values are set to -1 (i.e. take the bload defaults) +* unless someone has set them otherwise, so we just pull the values +* here. +*/ + bload->leaf_slack = xfs_globals.bload_leaf_slack; + bload->node_slack = xfs_globals.bload_node_slack;if (sc->ops->type ==ST_PERAG ) { free = sc->sa.pag->pagf_freeblks;
fs/xfs/xfs_globals.c +12 -0
@@ -44,4 +44,16 @@ struct xfs_globals xfs_globals = { .pwork_threads = -1, /* automatic thread detection */ .larp = false, /* log attribute replay */#endif + +/* +* Leave this many record slots empty when bulk loading btrees. By +* default we load new btree leaf blocks 75% full. +*/ + .bload_leaf_slack = -1, + +/* +* Leave this many key/ptr slots empty when bulk loading btrees. By +* default we load new btree node blocks 75% full. +*/ + .bload_node_slack = -1, };
fs/xfs/xfs_sysctl.h +2 -0
@@ -85,6 +85,8 @@ struct xfs_globals {int pwork_threads;/* parallel workqueue threads */ bool larp;/* log attribute replay */ #endif +int bload_leaf_slack;/* btree bulk load leaf slack */ +int bload_node_slack;/* btree bulk load node slack */ int log_recovery_delay;/* log recovery delay (secs) */ int mount_delay;/* mount setup delay (secs) */ bool bug_on_assert;/* BUG() the kernel on assert failure */
fs/xfs/xfs_sysfs.c +54 -0
66 lines not on this page · raw message
Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add some debug knobs so that we can control the leaf and node block slack when rebuilding btrees.
Maybe expand a bit on what these "debug knows" are useful for and how they work?
Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors
Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add some debug knobs so that we can control the leaf and node block slack when rebuilding btrees.
Maybe expand a bit on what these "debug knows" are useful for and how they work?
They're not generally useful for users, obviously. For developers, it might be useful to construct btrees of various heights by crafting a filesystem with a certain number of records and then using repair+knobs to rebuild the index with a certain shape.
Practically speaking, you'd only ever do that for extreme stress testing.
--D
Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors
Quoted 7 lines · Darrick J. Wong
They're not generally useful for users, obviously. For developers, it might be useful to construct btrees of various heights by crafting a filesystem with a certain number of records and then using repair+knobs to rebuild the index with a certain shape.
Practically speaking, you'd only ever do that for extreme stress testing.
Please add this to the commit message.
Re: [PATCH 2/4] xfs: add debug knobs to control btree bulk load slack factors
Quoted 10 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
They're not generally useful for users, obviously. For developers, it might be useful to construct btrees of various heights by crafting a filesystem with a certain number of records and then using repair+knobs to rebuild the index with a certain shape.
Practically speaking, you'd only ever do that for extreme stress testing.
Please add this to the commit message.
Done.
--D
Quoted 2 lines
PATCH 3/4 xfs: move btree bulkload record initialization to ->get_record implementations
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When we're performing a bulk load of a btree, move the code that actually stores the btree record in the new btree block out of the generic code and into the individual ->get_record implementations. This is preparation for being able to store multiple records with a single indirect call.
--- fs/xfs/libxfs/xfs_btree_staging.c | 17 +++++++---------- fs/xfs/libxfs/xfs_btree_staging.h | 15 ++++++++++----- 2 files changed, 17 insertions(+), 15 deletions(-)
fs/xfs/libxfs/xfs_btree_staging.c +7 -10
@@ -434,22 +434,19 @@ STATIC intxfs_btree_bload_leaf ( struct xfs_btree_cur *cur, unsigned int recs_this_block, - xfs_btree_bload_get_record_fn get_record, + xfs_btree_bload_get_records_fn get_records, struct xfs_btree_block *block, void *priv) { - unsigned int j; + unsigned int j = 1;int ret;/* Fill the leaf block with records. */ -for (j =1 ; j <= recs_this_block; j++) { -union xfs_btree_rec *block_rec; - - ret =get_record (cur, priv); -if (ret) +while (j <= recs_this_block) { + ret =get_records (cur, j, block, recs_this_block - j +1 , priv); +if (ret <0 )return ret; - block_rec =xfs_btree_rec_addr (cur, j, block); - cur->bc_ops->init_rec_from_cur (cur, block_rec); + j += ret; }return 0 ;
@@ -787,7 +784,7 @@ xfs_btree_bload(trace_xfs_btree_bload_block (cur, level, i, blocks, &ptr, nr_this_block); - ret =xfs_btree_bload_leaf (cur, nr_this_block, bbl->get_record, + ret =xfs_btree_bload_leaf (cur, nr_this_block, bbl->get_records, block, priv);if (ret) goto out;
fs/xfs/libxfs/xfs_btree_staging.h +10 -5
@@ -50,7 +50,9 @@ void xfs_btree_commit_ifakeroot(struct xfs_btree_cur *cur, struct xfs_trans *tp, int whichfork, const struct xfs_btree_ops *ops);/* Bulk loading of staged btrees. */ -typedef int (*xfs_btree_bload_get_record_fn )(struct xfs_btree_cur *cur,void *priv); +typedef int (*xfs_btree_bload_get_records_fn )(struct xfs_btree_cur *cur, +unsigned int idx,struct xfs_btree_block *block, +unsigned int nr_wanted,void *priv);typedef int (*xfs_btree_bload_claim_block_fn )(struct xfs_btree_cur *cur,union xfs_btree_ptr *ptr,void *priv); typedef size_t (*xfs_btree_bload_iroot_size_fn)(struct xfs_btree_cur *cur,
@@ -58,11 +60,14 @@ typedef size_t (*xfs_btree_bload_iroot_size_fn)(struct xfs_btree_cur *cur,struct xfs_btree_bload {/* -* This function will be called nr_records times to load records into -* the btree. The function does this by setting the cursor's bc_rec -* field in in-core format. Records must be returned in sort order. +* This function will be called to load @nr_wanted records into the +* btree. The implementation does this by setting the cursor's bc_rec +* field in in-core format and using init_rec_from_cur to set the +* records in the btree block. Records must be returned in sort order. +* The function must return the number of records loaded or the usual +* negative errno. */ -xfs_btree_bload_get_record_fn get_record; +xfs_btree_bload_get_records_fn get_records;/* * This function will be called nr_blocks times to obtain a pointer
Re: [PATCH 3/4] xfs: move btree bulkload record initialization to ->get_record implementations
Looks good:
PATCH 4/4 xfs: constrain dirty buffers while formatting a staged btree
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Constrain the number of dirty buffers that are locked by the btree staging code at any given time by establishing a threshold at which we put them all on the delwri queue and push them to disk. This limits memory consumption while writing out new btrees.
--- fs/xfs/libxfs/xfs_btree.c | 2 + fs/xfs/libxfs/xfs_btree.h | 3 ++ fs/xfs/libxfs/xfs_btree_staging.c | 50 +++++++++++++++++++++++++++++-------- fs/xfs/libxfs/xfs_btree_staging.h | 10 +++++++ fs/xfs/scrub/newbt.c | 1 + 5 files changed, 54 insertions(+), 12 deletions(-)
fs/xfs/libxfs/xfs_btree.c +1 -1
@@ -1330,7 +1330,7 @@ xfs_btree_get_buf_block(* Read in the buffer at the given ptr and return the buffer and * the block pointer within the buffer. */ -STATIC int +int xfs_btree_read_buf_block( struct xfs_btree_cur *cur, const union xfs_btree_ptr *ptr,
fs/xfs/libxfs/xfs_btree.h +3 -0
@@ -700,6 +700,9 @@ void xfs_btree_set_ptr_null(struct xfs_btree_cur *cur,int xfs_btree_get_buf_block (struct xfs_btree_cur *cur,const union xfs_btree_ptr *ptr,struct xfs_btree_block **block,struct xfs_buf **bpp); +int xfs_btree_read_buf_block (struct xfs_btree_cur *cur, +const union xfs_btree_ptr *ptr,int flags, +struct xfs_btree_block **block,struct xfs_buf **bpp);void xfs_btree_set_sibling (struct xfs_btree_cur *cur,struct xfs_btree_block *block,const union xfs_btree_ptr *ptr,int lr);
fs/xfs/libxfs/xfs_btree_staging.c +39 -11
91 lines not on this page · raw message
fs/xfs/libxfs/xfs_btree_staging.h +10 -0
@@ -115,6 +115,16 @@ struct xfs_btree_bload {* height of the new btree. */ unsigned int btree_height; + +/* +* Flush the new btree block buffer list to disk after this many blocks +* have been formatted. Zero prohibits writing any buffers until all +* blocks have been formatted. +*/ +uint16_t max_dirty; + +/* Number of dirty buffers. */ +uint16_t nr_dirty; }; int xfs_btree_bload_compute_geometry(struct xfs_btree_cur *cur,
fs/xfs/scrub/newbt.c +1 -0
@@ -89,6 +89,7 @@ xrep_newbt_init_ag( xnr->alloc_hint = alloc_hint; xnr->resv = resv;INIT_LIST_HEAD (&xnr->resv_list); + xnr->bload.max_dirty =XFS_B_TO_FSBT (sc->mp,256U <<10 );/* 256K */ xrep_newbt_estimate_slack (xnr); }
Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree
fs/xfs/libxfs/xfs_btree_staging.c xfs_btree_bload_node( · line 504
Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.c +1
-1@@ -480,7 +500,7 @@ xfs_btree_bload_node(ASSERT (!xfs_btree_ptr_is_null (cur, child_ptr)); - ret =xfs_btree_get_buf_block (cur, child_ptr, &child_block, + ret =xfs_btree_read_buf_block (cur, child_ptr,0 , &child_block, &child_bp);
How is this (and making xfs_btree_read_buf_block outside of xfs_buf.c) related to the dirty limit?
Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree
Quoted 10 lines · Christoph Hellwig
fs/xfs/libxfs/xfs_btree_staging.c xfs_btree_bload_node( · line 504
Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.c +1
-1@@ -480,7 +500,7 @@ xfs_btree_bload_node(ASSERT (!xfs_btree_ptr_is_null (cur, child_ptr)); - ret =xfs_btree_get_buf_block (cur, child_ptr, &child_block, + ret =xfs_btree_read_buf_block (cur, child_ptr,0 , &child_block, &child_bp);How is this (and making
xfs_btree_read_buf_blockoutside ofxfs_buf.c) related to the dirty limit?
Oh! Looking through my notes, I wanted the /new/ btree block buffers have the same lru reference count the old ones did. I probably should have exported xfs_btree_set_refs instead of reading whatever is on disk into a buffer, only to blow away the contents anyway.
--D
Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree
Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig
fs/xfs/libxfs/xfs_btree_staging.c xfs_btree_bload_node( · line 504
Quoted 7 lines
fs/xfs/libxfs/xfs_btree_staging.c +1
-1@@ -480,7 +500,7 @@ xfs_btree_bload_node(ASSERT (!xfs_btree_ptr_is_null (cur, child_ptr)); - ret =xfs_btree_get_buf_block (cur, child_ptr, &child_block, + ret =xfs_btree_read_buf_block (cur, child_ptr,0 , &child_block, &child_bp);How is this (and making
xfs_btree_read_buf_blockoutside ofxfs_buf.c) related to the dirty limit?
Quoted 4 lines
Oh! Looking through my notes, I wanted the /new/ btree block buffers have the same lru reference count the old ones did. I probably should have exported
xfs_btree_set_refsinstead of reading whatever is on disk into a buffer, only to blow away the contents anyway.
And now that I've dug further through my notes, I've realized that there's a better reason for this unexplained _get_buf -> _read_buf transition and the setting of XBF_DONE in _delwri_queue_here.
This patch introduces the behavior that we flush the delwri list to disk every 256k. Flushing the buffers releases them, which means that reclaim could free the buffer before xfs_btree_bload_node needs it again to build the next level up. If that's the case, then _get_buf will get us a !DONE buffer with zeroes instead of reading the (freshly written) buffer back in from disk. We'll then end up formatting garbage keys into the node block, which is bad.
/*
* Read the lower-level block in case the buffer for it has
* been reclaimed. LRU refs will be set on the block, which is
* desirable if the new btree commits.
*/
ret = xfs_btree_read_buf_block(cur, child_ptr, 0, &child_block,
&child_bp);
The behavior of setting XBF_DONE in xfs_buf_delwri_queue_here is an optimization if _delwri_submit releases the buffer and it is /not/ reclaimed. In that case, xfs_btree_read_buf_block will find the buffer without the DONE flag set and reread the contents from disk, which is unnecessary.
I'll split those changes into separate patches with fuller explanations of what's going on.
--D
Quoted 2 lines
--D
Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree
Quoted 6 lines · Darrick J. Wong
And now that I've dug further through my notes, I've realized that there's a better reason for this unexplained
_get_buf->_read_buftransition and the setting ofXBF_DONEin_delwri_queue_here.This patch introduces the behavior that we flush the delwri list to disk every 256k.
Where "the delwri list" is the one used for writing stage btrees I think.
Quoted 3 lines
Flushing the buffers releases them, which means that
reclaim could free the buffer beforexfs_btree_bload_nodeneeds it again to build the next level up.
Oh, indeed.
Quoted 4 lines
If that's the case, then
_get_bufwill get
us a !DONE buffer with zeroes instead of reading the (freshly written) buffer back in from disk. We'll then end up formatting garbage keys into the node block, which is bad.
Yeah.
Quoted 8 lines
/*
* Read the lower-level block in case the buffer for it has
* been reclaimed. LRU refs will be set on the block, which is
* desirable if the new btree commits.
*/
ret = xfs_btree_read_buf_block(cur,child_ptr, 0,&child_block,
&child_bp);
Quoted 5 lines
The behavior of setting
XBF_DONEinxfs_buf_delwri_queue_hereis an optimization if_delwri_submitreleases the buffer and it is /not/ reclaimed. In that case,xfs_btree_read_buf_blockwill find the buffer without the DONE flag set and reread the contents from disk, which is unnecessary.
Yeah. I still find it weird to set it in the delwri_submit_here helper, but maybe that's a discussion for the other thread.
Re: [PATCH 4/4] xfs: constrain dirty buffers while formatting a staged btree
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
And now that I've dug further through my notes, I've realized that there's a better reason for this unexplained
_get_buf->_read_buftransition and the setting ofXBF_DONEin_delwri_queue_here.This patch introduces the behavior that we flush the delwri list to disk every 256k.
Where "the delwri list" is the one used for writing stage btrees I think.
Correct.
Quoted 21 lines
Quoted 3 lines
Flushing the buffers releases them, which means that
reclaim could free the buffer beforexfs_btree_bload_nodeneeds it again to build the next level up.Oh, indeed.
Quoted 4 lines
If that's the case, then
_get_bufwill get
us a !DONE buffer with zeroes instead of reading the (freshly written) buffer back in from disk. We'll then end up formatting garbage keys into the node block, which is bad.Yeah.
Quoted 8 lines
/*
* Read the lower-level block in case the buffer for it has
* been reclaimed. LRU refs will be set on the block, which is
* desirable if the new btree commits.
*/
ret = xfs_btree_read_buf_block(cur,child_ptr, 0,&child_block,
&child_bp);
Quoted 8 lines
Quoted 5 lines
The behavior of setting
XBF_DONEinxfs_buf_delwri_queue_hereis an optimization if_delwri_submitreleases the buffer and it is /not/ reclaimed. In that case,xfs_btree_read_buf_blockwill find the buffer without the DONE flag set and reread the contents from disk, which is unnecessary.Yeah. I still find it weird to set it in the
delwri_submit_herehelper, but maybe that's a discussion for the other thread.
D'Oh! XBF_DONE exists in userspace too, because libxfs uses it. Well then, the proper place for it is at the top of xfs_btree_bload_drop_buf. I'll go change that tomorrow.
--D
cover xfs: online repair of AG btrees
In reply to Darrick J. WongHi all,
Now that we've spent a lot of time reworking common code in online fsck, we're ready to start rebuilding the AG space btrees. This series implements repair functions for the free space, inode, and refcount btrees. Rebuilding the reverse mapping btree is much more intense and is left for a subsequent patchset. The fstests counterpart of this patchset implements stress testing of repair.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-ag-btrees
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-ag-btrees
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-ag-btrees
--- fs/xfs/Makefile | 3 fs/xfs/libxfs/xfs_ag.h | 10 fs/xfs/libxfs/xfs_ag_resv.c | 2 fs/xfs/libxfs/xfs_alloc.c | 18 - fs/xfs/libxfs/xfs_alloc.h | 2 fs/xfs/libxfs/xfs_alloc_btree.c | 13 - fs/xfs/libxfs/xfs_btree.c | 26 + fs/xfs/libxfs/xfs_btree.h | 2 fs/xfs/libxfs/xfs_ialloc.c | 41 +- fs/xfs/libxfs/xfs_ialloc.h | 3 fs/xfs/libxfs/xfs_refcount.c | 18 - fs/xfs/libxfs/xfs_refcount.h | 2 fs/xfs/libxfs/xfs_refcount_btree.c | 13 - fs/xfs/libxfs/xfs_types.h | 7 fs/xfs/scrub/agheader_repair.c | 18 - fs/xfs/scrub/alloc.c | 16 + fs/xfs/scrub/alloc_repair.c | 914 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/bitmap.c | 404 +++++++++++++--- fs/xfs/scrub/bitmap.h | 70 ++- fs/xfs/scrub/common.c | 1 fs/xfs/scrub/common.h | 19 + fs/xfs/scrub/ialloc_repair.c | 874 ++++++++++++++++++++++++++++++++++ fs/xfs/scrub/newbt.c | 48 ++ fs/xfs/scrub/newbt.h | 6 fs/xfs/scrub/reap.c | 5 fs/xfs/scrub/refcount_repair.c | 793 +++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.c | 128 +++++ fs/xfs/scrub/repair.h | 43 ++ fs/xfs/scrub/scrub.c | 22 + fs/xfs/scrub/scrub.h | 9 fs/xfs/scrub/trace.h | 112 +++- fs/xfs/scrub/xfarray.h | 22 + fs/xfs/xfs_extent_busy.c | 13 + fs/xfs/xfs_extent_busy.h | 2 34 files changed, 3493 insertions(+), 186 deletions(-) create mode 100644 fs/xfs/scrub/alloc_repair.c create mode 100644 fs/xfs/scrub/ialloc_repair.c create mode 100644 fs/xfs/scrub/refcount_repair.c
PATCH 1/5 xfs: create separate structures and code for u32 bitmaps
From: Darrick J. Wong <[email protected]>
Create a version of the xbitmap that handles 32-bit integer intervals and adapt the xfs_agblock_t bitmap to use it. This reduces the size of the interval tree nodes from 48 to 36 bytes and enables us to use a more efficient slab (:0000040 instead of :0000048) which allows us to pack more nodes into a single slab page (102 vs 85).
As a side effect, the users of these bitmaps no longer have to convert between u32 and u64 quantities just to use the bitmap; and the hairy overflow checking code in xagb_bitmap_test goes away.
Later in this patchset we're going to add bitmaps for xfs_agino_t, xfs_rgblock_t, and xfs_dablk_t, so the increase in code size (5622 vs. 9959 bytes) seems worth it.
--- fs/xfs/scrub/agheader_repair.c | 9 - fs/xfs/scrub/bitmap.c | 404 ++++++++++++++++++++++++++++++++++------ fs/xfs/scrub/bitmap.h | 70 ++++--- fs/xfs/scrub/reap.c | 5 4 files changed, 390 insertions(+), 98 deletions(-)
fs/xfs/scrub/agheader_repair.c +4 -5
@@ -495,12 +495,11 @@ xrep_agfl_walk_rmap(/* Strike out the blocks that are cross-linked according to the rmapbt. */ STATIC intxrep_agfl_check_extent ( -uint64_t start, -uint64_t len, +uint32_t agbno, +uint32_t len,void *priv) {struct xrep_agfl *ra = priv; -xfs_agblock_t agbno = start;xfs_agblock_t last_agbno = agbno + len -1 ;int error;
@@ -648,8 +647,8 @@ struct xrep_agfl_fill {/* Fill the AGFL with whatever blocks are in this extent. */ static int xrep_agfl_fill ( -uint64_t start, -uint64_t len, +uint32_t start, +uint32_t len,void *priv) {struct xrep_agfl_fill *af = priv;
fs/xfs/scrub/bitmap.c +341 -63
419 lines not on this page · raw message
fs/xfs/scrub/bitmap.h +43 -27
121 lines not on this page · raw message
fs/xfs/scrub/reap.c +2 -3
@@ -430,13 +430,12 @@ xreap_agextent_iter(*/ STATIC intxreap_agmeta_extent ( -uint64_t fsbno, -uint64_t len, +uint32_t agbno, +uint32_t len,void *priv) {struct xreap_state *rs = priv;struct xfs_scrub *sc = rs->sc; -xfs_agblock_t agbno = fsbno;xfs_agblock_t agbno_next = agbno + len;int error =0 ;
Re: [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps
Quoted 5 lines · Darrick J. Wong
Create a version of the xbitmap that handles 32-bit integer intervals and adapt the
xfs_agblock_tbitmap to use it. This reduces the size of the interval tree nodes from 48 to 36 bytes and enables us to use a more efficient slab (:0000040 instead of :0000048) which allows us to pack more nodes into a single slab page (102 vs 85).
The changes themsleves looks good:
Q: should we rename the existing xbitmap to xbitmap64 for consistency?
Also why are the agb_bitmap* wrappers in bitmap.h? Following our usual code organization I'd expect bitmap.[ch] to just be the library code and have users outside of that. Maybe for later..
Re: [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
Create a version of the xbitmap that handles 32-bit integer intervals and adapt the
xfs_agblock_tbitmap to use it. This reduces the size of the interval tree nodes from 48 to 36 bytes and enables us to use a more efficient slab (:0000040 instead of :0000048) which allows us to pack more nodes into a single slab page (102 vs 85).
Quoted 5 lines
The changes themsleves looks good:
Reviewed-by Christoph Hellwig <[email protected]>Q: should we rename the existing xbitmap to xbitmap64 for consistency?
Yes. Done.
Quoted 3 lines
Also why are the agb_bitmap* wrappers in
bitmap.h? Following our usual code organization I'd expect bitmap.[ch] to just be the library code and have users outside of that. Maybe for later..
Those wrappers are trivial except for the enhanced typechecking, so I didn't think it was a big deal to cram them into bitmap.h.
--D
Re: [PATCH 1/5] xfs: create separate structures and code for u32 bitmaps
Quoted 6 lines · Darrick J. Wong
Quoted 3 lines
Also why are the agb_bitmap* wrappers in
bitmap.h? Following our usual code organization I'd expect bitmap.[ch] to just be the library code and have users outside of that. Maybe for later..Those wrappers are trivial except for the enhanced typechecking, so I didn't think it was a big deal to cram them into
bitmap.h.
I find that kind of code structure a bit confusion to be honest. If you prefer it I can live with it (at least fow now :)) of course.
PATCH 2/5 xfs: roll the scrub transaction after completing a repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When we've finished repairing an AG header, roll the scrub transaction. This ensure that any failures caused by defer ops failing are captured by the xrep_done tracepoint and that any stacktraces that occur will point to the repair code that caused it, instead of xchk_teardown.
Going forward, repair functions should commit the transaction if they're going to return success. Usually the space reaping functions that run after a successful atomic commit of the new metadata will take care of that for us.
--- fs/xfs/scrub/agheader_repair.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/scrub/agheader_repair.c +6 -3
@@ -73,7 +73,7 @@ xrep_superblock(/* Write this to disk. */ xfs_trans_buf_set_type (sc->tp, bp,XFS_BLFT_SB_BUF );xfs_trans_log_buf (sc->tp, bp,0 ,BBTOB (bp->b_length) -1 ); -return error; +return 0 ; }/* AGF */
@@ -342,7 +342,7 @@ xrep_agf_commit_new( pag->pagf_refcount_level =be32_to_cpu (agf->agf_refcount_level);set_bit (XFS_AGSTATE_AGF_INIT , &pag->pag_opstate); -return 0 ; +return xrep_roll_ag_trans (sc); }/* Repair the AGF. v5 filesystems only. */
@@ -789,6 +789,9 @@ xrep_agfl(/* Dump any AGFL overflow. */ error =xrep_reap_agblocks (sc, &agfl_extents, &XFS_RMAP_OINFO_AG ,XFS_AG_RESV_AGFL ); +if (error) + goto err; + err:xagb_bitmap_destroy (&agfl_extents);return error;
@@ -962,7 +965,7 @@ xrep_agi_commit_new( pag->pagi_freecount =be32_to_cpu (agi->agi_freecount);set_bit (XFS_AGSTATE_AGI_INIT , &pag->pag_opstate); -return 0 ; +return xrep_roll_ag_trans (sc); }/* Repair the AGI. */
Re: [PATCH 2/5] xfs: roll the scrub transaction after completing a repair
Quoted 4 lines · Darrick J. Wong
Going forward, repair functions should commit the transaction if they're going to return success. Usually the space reaping functions that run after a successful atomic commit of the new metadata will take care of that for us.
Generally looks good:
A random comment on a pre-existing function from reading the code, and a nitpick on the patch itself below:
fs/xfs/scrub/agheader_repair.c xrep_superblock( · line 76
Quoted 7 lines
+++
b/fs/xfs/scrub/agheader_repair.cfs/xfs/scrub/agheader_repair.c +1
-1@@ -73,7 +73,7 @@ xrep_superblock(/* Write this to disk. */ xfs_trans_buf_set_type (sc->tp, bp,XFS_BLFT_SB_BUF );xfs_trans_log_buf (sc->tp, bp,0 ,BBTOB (bp->b_length) -1 ); -return error; +return 0 ;
After looking through the code this is obviously fine, error must be 0 here because the last patch touching it is xchk_should_terminate, which only sets the error if it returns true.
But the calling conventions for xchk_should_terminate really make me scratch my head as they are so hard to reason about. I did quick look over must caller and most of them get there with error always set to 0. So just making xchk_should_terminate return the error would seem a lot better to me - any caller with a previous error would need a second error2, but that seems better than what we have there right now.
fs/xfs/scrub/agheader_repair.c xrep_agfl( · line 795
Quoted 9 lines
fs/xfs/scrub/agheader_repair.c +3
-0/* Repair the AGF. v5 filesystems only. */ @@ -789,6 +789,9 @@ xrep_agfl(/* Dump any AGFL overflow. */ error =xrep_reap_agblocks (sc, &agfl_extents, &XFS_RMAP_OINFO_AG ,XFS_AG_RESV_AGFL ); +if (error) + goto err; + err:
This seems rather pointless and doesn't change anything..
Re: [PATCH 2/5] xfs: roll the scrub transaction after completing a repair
Quoted 13 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
Going forward, repair functions should commit the transaction if they're going to return success. Usually the space reaping functions that run after a successful atomic commit of the new metadata will take care of that for us.
Generally looks good:
Reviewed-by Christoph Hellwig <[email protected]>A random comment on a pre-existing function from reading the code, and a nitpick on the patch itself below:
Quoted 11 lines
fs/xfs/scrub/agheader_repair.c xrep_superblock( · line 76
Quoted 7 lines
+++
b/fs/xfs/scrub/agheader_repair.cfs/xfs/scrub/agheader_repair.c +1
-1@@ -73,7 +73,7 @@ xrep_superblock(/* Write this to disk. */ xfs_trans_buf_set_type (sc->tp, bp,XFS_BLFT_SB_BUF );xfs_trans_log_buf (sc->tp, bp,0 ,BBTOB (bp->b_length) -1 ); -return error; +return 0 ;After looking through the code this is obviously fine, error must be 0 here because the last patch touching it is
xchk_should_terminate, which only sets the error if it returns true.
<nod>
Quoted 7 lines
But the calling conventions for
xchk_should_terminatereally make me scratch my head as they are so hard to reason about. I did quick look over must caller and most of them get there with error always set to 0. So just makingxchk_should_terminatereturn the error would seem a lot better to me - any caller with a previous error would need a second error2, but that seems better than what we have there right now.
Agreed, the callsites would be a bit more obvious if they looked like:
error = xchk_should_terminate(sc); if (error) break;
Though I'm working on some tweaks of that function, since it was pointed out to me that cond_resched() and fatal_signal_pending() aren't entirely free. What I've been testing out the last three weeks is:
unsigned long now = jiffies;
if (time_after(sc->next_poke, now)) {
sc->next_poke = now + (HZ / 10);
cond_resched();
if (fatal_signal_pending(current))
return -EINTR;
}
return 0;So far I haven't seen much improvement, but the callsite change is something that I think I could promote to the end of online repair part 2.
Quoted 11 lines
fs/xfs/scrub/agheader_repair.c xrep_agfl( · line 795
Quoted 9 lines
fs/xfs/scrub/agheader_repair.c +3
-0/* Repair the AGF. v5 filesystems only. */ @@ -789,6 +789,9 @@ xrep_agfl(/* Dump any AGFL overflow. */ error =xrep_reap_agblocks (sc, &agfl_extents, &XFS_RMAP_OINFO_AG ,XFS_AG_RESV_AGFL ); +if (error) + goto err; + err:This seems rather pointless and doesn't change anything..
Oops, lemme get rid of that dead code...
--D
Quoted 1 line
PATCH 3/5 xfs: repair free space btrees
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Rebuild the free space btrees from the gaps in the rmap btree.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_ag.h | 9 fs/xfs/libxfs/xfs_ag_resv.c | 2 fs/xfs/libxfs/xfs_alloc.c | 18 + fs/xfs/libxfs/xfs_alloc.h | 2 fs/xfs/libxfs/xfs_alloc_btree.c | 13 + fs/xfs/libxfs/xfs_types.h | 7 fs/xfs/scrub/alloc.c | 16 + fs/xfs/scrub/alloc_repair.c | 914 +++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 19 + fs/xfs/scrub/newbt.c | 48 ++ fs/xfs/scrub/newbt.h | 6 fs/xfs/scrub/repair.c | 69 +++ fs/xfs/scrub/repair.h | 24 + fs/xfs/scrub/scrub.c | 14 - fs/xfs/scrub/scrub.h | 8 fs/xfs/scrub/trace.h | 24 + fs/xfs/scrub/xfarray.h | 22 + fs/xfs/xfs_extent_busy.c | 13 + fs/xfs/xfs_extent_busy.h | 2 20 files changed, 1212 insertions(+), 19 deletions(-) create mode 100644 fs/xfs/scrub/alloc_repair.c
fs/xfs/Makefile +1 -0
@@ -181,6 +181,7 @@ xfs-$(CONFIG_XFS_QUOTA) += scrub/quota.o ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ + alloc_repair.o \ newbt.o \ reap.o \ repair.o \
fs/xfs/libxfs/xfs_ag.h +9 -0
@@ -80,6 +80,15 @@ struct xfs_perag {*/ uint16_t pag_checked;uint16_t pag_sick; + +#ifdef CONFIG_XFS_ONLINE_REPAIR +/* +* Alternate btree heights so that online repair won't trip the write +* verifiers while rebuilding the AG btrees. +*/ +uint8_t pagf_alt_levels[XFS_BTNUM_AGF ]; +#endif +spinlock_t pag_state_lock;spinlock_t pagb_lock;/* lock for pagb_tree */
fs/xfs/libxfs/xfs_ag_resv.c +2 -0
@@ -411,6 +411,8 @@ xfs_ag_resv_free_extent( fallthrough;case XFS_AG_RESV_NONE :xfs_trans_mod_sb (tp,XFS_TRANS_SB_FDBLOCKS , (int64_t )len); + fallthrough; +case XFS_AG_RESV_IGNORE :return ; }
fs/xfs/libxfs/xfs_alloc.c +12 -6
@@ -243,14 +243,11 @@ xfs_alloc_btrec_to_irec( irec->ar_blockcount =be32_to_cpu (rec->alloc.ar_blockcount); } -/* Simple checks for free space records. */ -xfs_failaddr_t -xfs_alloc_check_irec ( -struct xfs_btree_cur *cur, +inline xfs_failaddr_t +xfs_alloc_check_perag_irec ( +struct xfs_perag *pag,const struct xfs_alloc_rec_incore *irec) { -struct xfs_perag *pag = cur->bc_ag.pag; -if (irec->ar_blockcount ==0 )return __this_address;
@@ -261,6 +258,15 @@ xfs_alloc_check_irec(return NULL ; } +/* Simple checks for free space records. */ +xfs_failaddr_t +xfs_alloc_check_irec ( +struct xfs_btree_cur *cur, +const struct xfs_alloc_rec_incore *irec) +{ +return xfs_alloc_check_perag_irec (cur->bc_ag.pag, irec); +} + static inline int xfs_alloc_complain_bad_rec( struct xfs_btree_cur *cur,
fs/xfs/libxfs/xfs_alloc.h +2 -0
@@ -185,6 +185,8 @@ xfs_alloc_get_rec(union xfs_btree_rec ;void xfs_alloc_btrec_to_irec (const union xfs_btree_rec *rec,struct xfs_alloc_rec_incore *irec); +xfs_failaddr_t xfs_alloc_check_perag_irec (struct xfs_perag *pag, +const struct xfs_alloc_rec_incore *irec);xfs_failaddr_t xfs_alloc_check_irec (struct xfs_btree_cur *cur,const struct xfs_alloc_rec_incore *irec);
fs/xfs/libxfs/xfs_alloc_btree.c +12 -1
@@ -323,7 +323,18 @@ xfs_allocbt_verify(if (bp->b_ops->magic[0 ] ==cpu_to_be32 (XFS_ABTC_MAGIC )) btnum = XFS_BTNUM_CNTi;if (pag &&xfs_perag_initialised_agf (pag)) { -if (level >= pag->pagf_levels[btnum]) +unsigned int maxlevel = pag->pagf_levels[btnum]; + +#ifdef CONFIG_XFS_ONLINE_REPAIR +/* +* Online repair could be rewriting the free space btrees, so +* we'll validate against the larger of either tree while this +* is going on. +*/ + maxlevel = max_t(unsigned int, maxlevel, + pag->pagf_alt_levels[btnum]); +#endif +if (level >= maxlevel)return __this_address; }else if (level >= mp->m_alloc_maxlevels)return __this_address;
fs/xfs/libxfs/xfs_types.h +7 -0
@@ -208,6 +208,13 @@ enum xfs_ag_resv_type {XFS_AG_RESV_AGFL ,XFS_AG_RESV_METADATA ,XFS_AG_RESV_RMAPBT , + +/* +* Don't increase fdblocks when freeing extent. This is a pony for +* the bnobt repair functions to re-free the free space without +* altering fdblocks. If you think you need this you're wrong. +*/ + XFS_AG_RESV_IGNORE, };/* Results of scanning a btree keyspace to check occupancy. */
fs/xfs/scrub/alloc.c +14 -2
@@ -9,13 +9,16 @@#include "xfs_format.h" #include "xfs_trans_resv.h" #include "xfs_mount.h" +#include "xfs_log_format.h" +#include "xfs_trans.h" #include "xfs_btree.h" #include "xfs_alloc.h" #include "xfs_rmap.h" +#include "xfs_ag.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/btree.h" -#include "xfs_ag.h" +#include "scrub/repair.h" /* * Set us up to scrub free space btrees.
@@ -24,10 +27,19 @@ intxchk_setup_ag_allocbt ( struct xfs_scrub *sc) { +int error; +if (xchk_need_intent_drain (sc))xchk_fsgates_enable (sc,XCHK_FSGATES_DRAIN ); -return xchk_setup_ag_btree (sc, false); + error =xchk_setup_ag_btree (sc, false); +if (error) +return error; + +if (xchk_could_repair (sc)) +return xrep_setup_ag_allocbt (sc); + +return 0 ; }/* Free space btree scrubber. */
fs/xfs/scrub/alloc_repair.c new file +914 -0
914 lines not on this page · raw message
fs/xfs/scrub/common.h +19 -0
@@ -194,8 +194,21 @@ static inline bool xchk_needs_repair(const struct xfs_scrub_metadata *sm)XFS_SCRUB_OFLAG_XCORRUPT | XFS_SCRUB_OFLAG_PREEN); } + +/* +* "Should we prepare for a repair?" +* +* Return true if the caller permits us to repair metadata and we're not +* setting up for a post-repair evaluation. +*/ +static inline bool xchk_could_repair (const struct xfs_scrub *sc) +{ +return (sc->sm->sm_flags &XFS_SCRUB_IFLAG_REPAIR ) && + !(sc->flags &XREP_ALREADY_FIXED ); +}#else # define xchk_needs_repair (sc) (false) +# define xchk_could_repair (sc) (false)#endif /* CONFIG_XFS_ONLINE_REPAIR */ int xchk_metadata_inode_forks (struct xfs_scrub *sc);
@@ -207,6 +220,12 @@ int xchk_metadata_inode_forks(struct xfs_scrub *sc);#define xchk_xfile_descr (sc, fmt, ...) \ kasprintf(XCHK_GFP_FLAGS, "XFS (%s): " fmt, \ (sc)->mp->m_super->s_id, ##__VA_ARGS__) +#define xchk_xfile_ag_descr (sc, fmt, ...) \ + kasprintf(XCHK_GFP_FLAGS, "XFS (%s): AG 0x%x " fmt, \ + (sc)->mp->m_super->s_id, \ + (sc)->sa.pag ? (sc)->sa.pag->pag_agno : (sc)->sm->sm_agno, \ + ##__VA_ARGS__) +/* * Setting up a hook to wait for intents to drain is costly -- we have to take
fs/xfs/scrub/newbt.c +44 -4
70 lines not on this page · raw message
fs/xfs/scrub/newbt.h +6 -0
@@ -50,6 +50,9 @@ struct xrep_newbt {enum xfs_ag_resv_type resv; }; +#define for_each_xrep_newbt_reservation (xnr, resv, n) \ + list_for_each_entry_safe((resv), (n), &(xnr)->resv_list, list) +void xrep_newbt_init_bare (struct xrep_newbt *xnr,struct xfs_scrub *sc); void xrep_newbt_init_ag(struct xrep_newbt *xnr, struct xfs_scrub *sc, const struct xfs_owner_info *oinfo, xfs_fsblock_t alloc_hint,
@@ -57,9 +60,12 @@ void xrep_newbt_init_ag(struct xrep_newbt *xnr, struct xfs_scrub *sc,int xrep_newbt_init_inode (struct xrep_newbt *xnr,struct xfs_scrub *sc,int whichfork,const struct xfs_owner_info *oinfo);int xrep_newbt_alloc_blocks (struct xrep_newbt *xnr,uint64_t nr_blocks); +int xrep_newbt_add_extent (struct xrep_newbt *xnr,struct xfs_perag *pag, +xfs_agblock_t agbno,xfs_extlen_t len);void xrep_newbt_cancel (struct xrep_newbt *xnr);int xrep_newbt_commit (struct xrep_newbt *xnr);int xrep_newbt_claim_block (struct xfs_btree_cur *cur,struct xrep_newbt *xnr,union xfs_btree_ptr *ptr); +unsigned int xrep_newbt_unused_blocks (struct xrep_newbt *xnr);#endif /* __XFS_SCRUB_NEWBT_H__ */
fs/xfs/scrub/repair.c +69 -0
72 lines not on this page · raw message
fs/xfs/scrub/repair.h +24 -0
@@ -60,6 +60,15 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp,void xrep_force_quotacheck (struct xfs_scrub *sc,xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc); +/* Repair setup functions */ +int xrep_setup_ag_allocbt (struct xfs_scrub *sc); + +void xrep_ag_btcur_init (struct xfs_scrub *sc,struct xchk_ag *sa); + +/* Metadata revalidators */ + +int xrep_revalidate_allocbt (struct xfs_scrub *sc); +/* Metadata repairers */ int xrep_probe (struct xfs_scrub *sc);
@@ -67,6 +76,9 @@ int xrep_superblock(struct xfs_scrub *sc);int xrep_agf (struct xfs_scrub *sc);int xrep_agfl (struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc); +int xrep_allocbt (struct xfs_scrub *sc); + +int xrep_reinit_pagf (struct xfs_scrub *sc);#else
@@ -87,11 +99,23 @@ xrep_calc_ag_resblks( return 0; } +/* repair setup functions for no-repair */ +static inline int +xrep_setup_nothing ( +struct xfs_scrub *sc) +{ +return 0 ; +} +#define xrep_setup_ag_allocbt xrep_setup_nothing + +#define xrep_revalidate_allocbt (NULL) +#define xrep_probe xrep_notsupported#define xrep_superblock xrep_notsupported#define xrep_agf xrep_notsupported#define xrep_agfl xrep_notsupported#define xrep_agi xrep_notsupported +#define xrep_allocbt xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +9 -5
@@ -241,13 +241,15 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_PERAG, .setup = xchk_setup_ag_allocbt, .scrub = xchk_bnobt, - .repair = xrep_notsupported, + .repair = xrep_allocbt, + .repair_eval = xrep_revalidate_allocbt, }, [XFS_SCRUB_TYPE_CNTBT] = { /* cntbt */ .type = ST_PERAG, .setup = xchk_setup_ag_allocbt, .scrub = xchk_cntbt, - .repair = xrep_notsupported, + .repair = xrep_allocbt, + .repair_eval = xrep_revalidate_allocbt, }, [XFS_SCRUB_TYPE_INOBT] = { /* inobt */ .type = ST_PERAG,
@@ -533,7 +535,10 @@ xfs_scrub_metadata(/* Scrub for errors. */ check_start =xchk_stats_now (); - error = sc->ops->scrub (sc); +if ((sc->flags &XREP_ALREADY_FIXED ) && sc->ops->repair_eval !=NULL ) + error = sc->ops->repair_eval (sc); +else + error = sc->ops->scrub (sc); run.scrub_ns +=xchk_stats_elapsed_ns (check_start);if (error == -EDEADLOCK && !(sc->flags &XCHK_TRY_HARDER )) goto try_harder;
@@ -544,8 +549,7 @@ xfs_scrub_metadata(xchk_update_health (sc); -if ((sc->sm->sm_flags &XFS_SCRUB_IFLAG_REPAIR ) && - !(sc->flags &XREP_ALREADY_FIXED )) { +if (xchk_could_repair (sc)) {bool needs_fix =xchk_needs_repair (sc->sm);/* Userspace asked us to rebuild the structure regardless. */
fs/xfs/scrub/scrub.h +8 -0
@@ -35,6 +35,14 @@ struct xchk_meta_ops {/* Repair or optimize the metadata. */ int (*repair)(struct xfs_scrub *); +/* +* Re-scrub the metadata we repaired, in case there's extra work that +* we need to do to check our repair work. If this is NULL, we'll use +* the ->scrub function pointer, assuming that the regular scrub is +* sufficient. +*/ +int (*repair_eval)(struct xfs_scrub *sc); +/* Decide if we even have this piece of metadata. */ bool (*has)(struct xfs_mount *);
fs/xfs/scrub/trace.h +23 -1
@@ -1172,11 +1172,33 @@ DEFINE_EVENT(xrep_rmap_class, name, \ xfs_agblock_t agbno, xfs_extlen_t len, \ uint64_t owner, uint64_t offset, unsigned int flags), \ TP_ARGS(mp, agno, agbno, len, owner, offset, flags)) -DEFINE_REPAIR_RMAP_EVENT(xrep_alloc_extent_fn); DEFINE_REPAIR_RMAP_EVENT(xrep_ialloc_extent_fn);DEFINE_REPAIR_RMAP_EVENT (xrep_rmap_extent_fn);DEFINE_REPAIR_RMAP_EVENT (xrep_bmap_extent_fn); +TRACE_EVENT (xrep_abt_found, + TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, + const struct xfs_alloc_rec_incore *rec), +TP_ARGS (mp, agno, rec), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_agnumber_t, agno) + __field(xfs_agblock_t, startblock) +__field (xfs_extlen_t, blockcount) + ), +TP_fast_assign ( + __entry->dev = mp->m_super->s_dev; + __entry->agno = agno; + __entry->startblock = rec->ar_startblock; + __entry->blockcount = rec->ar_blockcount; + ), +TP_printk ("dev %d:%d agno 0x%x agbno 0x%x fsbcount 0x%x" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->agno, + __entry->startblock, + __entry->blockcount) +) + TRACE_EVENT(xrep_refcount_extent_fn, TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, struct xfs_refcount_irec *irec),
fs/xfs/scrub/xfarray.h +22 -0
@@ -54,6 +54,28 @@ static inline int xfarray_append(struct xfarray *array, const void *ptr)uint64_t xfarray_length (struct xfarray *array);int xfarray_load_next (struct xfarray *array,xfarray_idx_t *idx,void *rec); +/* +* Iterate the non-null elements in a sparse xfarray. Callers should +* initialize *idx to XFARRAY_CURSOR_INIT before the first call; on return, it +* will be set to one more than the index of the record that was retrieved. +* Returns 1 if a record was retrieved, 0 if there weren't any more records, or +* a negative errno. +*/ +static inline int +xfarray_iter ( +struct xfarray *array, +xfarray_idx_t *idx, +void *rec) +{ +int ret =xfarray_load_next (array, idx, rec); + +if (ret == -ENODATA ) +return 0 ; +if (ret ==0 ) +return 1 ; +return ret; +} +/* Declarations for xfile array sort functionality. */ typedef cmp_func_t xfarray_cmp_fn ;
fs/xfs/xfs_extent_busy.c +13 -0
@@ -678,3 +678,16 @@ xfs_extent_busy_ag_cmp( diff = b1->bno - b2->bno;return diff; } + +/* Are there any busy extents in this AG? */ +bool +xfs_extent_busy_list_empty ( +struct xfs_perag *pag) +{ +bool res; + +spin_lock (&pag->pagb_lock); + res =RB_EMPTY_ROOT (&pag->pagb_tree); +spin_unlock (&pag->pagb_lock); +return res; +}
fs/xfs/xfs_extent_busy.h +2 -0
@@ -85,4 +85,6 @@ static inline void xfs_extent_busy_sort(struct list_head *list) list_sort(NULL, list, xfs_extent_busy_ag_cmp); } +bool xfs_extent_busy_list_empty (struct xfs_perag *pag); +#endif /* __XFS_EXTENT_BUSY_H__ */
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Rebuild the free space btrees from the gaps in the rmap btree.
This commit message feels a bit sparse for the amount of code added, although I can't really offer a good idea of what to add.
Otherwise just two comments on the interaction with the rest of the xfs code, I'll try to digest the new repair code a bit more in the meantime.
fs/xfs/libxfs/xfs_ag.h struct xfs_perag { · line 90
Quoted 7 lines
fs/xfs/libxfs/xfs_ag.h +7
-0+#ifdef CONFIG_XFS_ONLINE_REPAIR +/* +* Alternate btree heights so that online repair won't trip the write +* verifiers while rebuilding the AG btrees. +*/ +uint8_t pagf_alt_levels[XFS_BTNUM_AGF ]; +#endif
Alternate and the alt_ prefix doesn't feel very descriptive. As far as I can tell these are about an ongoign repair, so as a at lest somewhat better choice call it "pagf_repair_levels"?
fs/xfs/libxfs/xfs_alloc.c xfs_alloc_check_irec( · line 268
Quoted 7 lines
fs/xfs/libxfs/xfs_alloc.c +7
-0+xfs_failaddr_t +xfs_alloc_check_irec ( +struct xfs_btree_cur *cur, +const struct xfs_alloc_rec_incore *irec) +{ +return xfs_alloc_check_perag_irec (cur->bc_ag.pag, irec); +}
Is there much of a point in even keeping this wrapper vs just switching xfs_alloc_check_irec to pass the pag instead of the cursor?
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Rebuild the free space btrees from the gaps in the rmap btree.
This commit message feels a bit sparse for the amount of code added, although I can't really offer a good idea of what to add.
"Refer to the design documentation for more details:
?
Quoted 4 lines
Otherwise just two comments on the interaction with the rest of the xfs code, I'll try to digest the new repair code a bit more in the meantime.
Quoted 11 lines
fs/xfs/libxfs/xfs_ag.h struct xfs_perag { · line 90
Quoted 7 lines
fs/xfs/libxfs/xfs_ag.h +7
-0+#ifdef CONFIG_XFS_ONLINE_REPAIR +/* +* Alternate btree heights so that online repair won't trip the write +* verifiers while rebuilding the AG btrees. +*/ +uint8_t pagf_alt_levels[XFS_BTNUM_AGF ]; +#endif Alternate and the
alt_prefix doesn't feel very descriptive. As far as I can tell these are about an ongoign repair, so as a at lest somewhat better choice call it "pagf_repair_levels"?
Done.
Quoted 11 lines
fs/xfs/libxfs/xfs_alloc.c xfs_alloc_check_irec( · line 268
Quoted 7 lines
fs/xfs/libxfs/xfs_alloc.c +7
-0+xfs_failaddr_t +xfs_alloc_check_irec ( +struct xfs_btree_cur *cur, +const struct xfs_alloc_rec_incore *irec) +{ +return xfs_alloc_check_perag_irec (cur->bc_ag.pag, irec); +}Is there much of a point in even keeping this wrapper vs just switching
xfs_alloc_check_irecto pass the pag instead of the cursor?
Not really. I'll remove this from the next spin.
--D
Re: [PATCH 3/5] xfs: repair free space btrees
In reply to Darrick J. WongA highlevel question on how blocks are (re)used here.
- xrep_abt_find_freespace accounts the old allocbt blocks as freespace per the comment, although as far as I understand the code in xrep_abt_walk_rmap the allocbt blocks aren't actually added to the free_records xfarray, but recorded into the old_allocbt_blocks bitmap (btw, why are we using different data structures for them?)
- xrep_abt_reserve_space seems to allocate space for the new alloc btrees by popping the first entry of ->free_records until it has enough space.
- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?
I've also got a whole bunch of nitpicks that mostly don't require an immediate action and/or about existing code that just grows more users here:
fs/xfs/scrub/alloc_repair.c line 838
Quoted 12 lines
fs/xfs/scrub/alloc_repair.c new file +12
-0+int +xrep_allocbt ( +struct xfs_scrub *sc) +{ +struct xrep_abt *ra; +struct xfs_mount *mp = sc->mp; +char *descr; +int error; + +/* We require the rmapbt to rebuild anything. */ +if (!xfs_has_rmapbt (mp)) +return -EOPNOTSUPP ;
Shoudn't this be checked by the .has callback in struct xchk_meta_ops?
fs/xfs/scrub/alloc_repair.c line 863
Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6
-0+/* Set up enough storage to handle maximally fragmented free space. */ + descr =xchk_xfile_ag_descr (sc,"free space records" ); + error =xfarray_create (descr, mp->m_sb.sb_agblocks /2 , +sizeof (struct xfs_alloc_rec_incore ), + &ra->free_records); +kfree (descr);
Commenting on a new user of old code here, but why doesn't xfarray_create simply take a format string so that we don't need the separate allocatiom and kasprintf here?
fs/xfs/scrub/alloc_repair.c line 905
Quoted 7 lines
fs/xfs/scrub/alloc_repair.c new file +7
-0+/* +* We must update sm_type temporarily so that the tree-to-tree cross +* reference checks will work in the correct direction, and also so +* that tracing will report correctly if there are more errors. +*/ + sc->sm->sm_type =XFS_SCRUB_TYPE_BNOBT ; + error =xchk_bnobt (sc);
So xchk_bnobt is a tiny wrapper around xchk_allocbt, which is a small wrapper around xchk_btree that basіally de-multiplex the argument pass in by xchk_bnobt again. This is existing code not newly added, but the call chain looks a bit odd to me.
Quoted 13 lines
fs/xfs/scrub/newbt.c +13
-0+/* +* Add an extent to the new btree reservation pool. Callers are required to +* reap this reservation manually if the repair is cancelled. @pag must be a +* passive reference. +*/ +int +xrep_newbt_add_extent ( +struct xrep_newbt *xnr, +struct xfs_perag *pag, +xfs_agblock_t agbno, +xfs_extlen_t len) +{ +struct xfs_mount *mp = xnr->sc->mp;
fs/xfs/scrub/newbt.c xrep_newbt_add_blocks( · line 194
Quoted 10 lines
fs/xfs/scrub/newbt.c +10
-0+struct xfs_alloc_arg args = { + .tp =NULL ,/* no autoreap */ + .oinfo = xnr->oinfo, + .fsbno =XFS_AGB_TO_FSB (mp, pag->pag_agno, agbno), + .len = len, + .resv = xnr->resv, + }; + +return xrep_newbt_add_blocks (xnr, pag, &args); +}
I don't quite understand what this helper adds, and the _blocks vs _extent naming is a bit confusing.
fs/xfs/scrub/newbt.h struct xrep_newbt { · line 54
Quoted 2 lines
fs/xfs/scrub/newbt.h +2
-0+#define for_each_xrep_newbt_reservation (xnr, resv, n) \ + list_for_each_entry_safe((resv), (n), &(xnr)->resv_list, list)
I have to admit that I find the open code list_for_each_entry_safe easier to follow than such wrappers.
fs/xfs/scrub/repair.c xrep_ino_dqattach( · line 743
Quoted 6 lines
fs/xfs/scrub/repair.c +6
-0+/* Initialize all the btree cursors for an AG repair. */ +void +xrep_ag_btcur_init ( +struct xfs_scrub *sc, +struct xchk_ag *sa) +{
As far as I can tell this basically sets up cursors for all the btrees except the one that we want to repair, and the one that goes along with for the alloc and ialloc pairs? Maybe just spell that out for clarity.
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 7 lines · Christoph Hellwig
A highlevel question on how blocks are (re)used here.
-
xrep_abt_find_freespaceaccounts the old allocbt blocks as freespace per the comment, although as far as I understand the code inxrep_abt_walk_rmapthe allocbt blocks aren't actually added to thefree_recordsxfarray, but recorded into theold_allocbt_blocksbitmap (btw, why are we using different data structures for them?)
The old free space btree blocks are tracked via old_allocbt_blocks so that we can reap the space after committing the new btree roots. Reaping cross-references the set regions in the bitmap against the rmapbt records so that we don't free crosslinked blocks.
This is a big difference from xfs_repair, which constructs a free space map by walking the directory / bmbt trees and builds completely new indexes in the gaps. It gets away with that because it's building all the space metadata in the AG, not just the free space btrees.
The next question you might have is why there's old_allocbt_blocks and not_allocbt_blocks -- this is due to us using the AGFL to supply the bnobt, cntbt, and rmapbt's alloc_block routines. Hence the blocks tracked by all four data structures are all RMAP_OWN_AG, and we have to do a bit of bitmap work to subtract the rmapbt and AGFL blocks from all the OWN_AG records to end up with the blocks that we think are owned by the free space btrees.
Quoted 3 lines
-
xrep_abt_reserve_spaceseems to allocate space for the new alloc btrees by popping the first entry of ->free_records until it has enough space.
Correct.
Quoted 3 lines
- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?
xrep_abt_reserve_space returns -ENOSPC, so we throw away all the incore records and throw the errno out to userspace. Across all the btree rebuilding code, the block reservation step happens as the very last thing before we start writing btree blocks, so it's still possible to back out cleanly.
Quoted 3 lines
I've also got a whole bunch of nitpicks that mostly don't require an immediate action and/or about existing code that just grows more users here:
Heh.
Quoted 6 lines
Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6
-0+int +xrep_allocbt ( +struct xfs_scrub *sc) +{ +struct xrep_abt *ra; +struct xfs_mount *mp = sc->mp;
Quoted 8 lines
fs/xfs/scrub/alloc_repair.c line 838
Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6
-0+char *descr; +int error; + +/* We require the rmapbt to rebuild anything. */ +if (!xfs_has_rmapbt (mp)) +return -EOPNOTSUPP ;Shoudn't this be checked by the .has callback in struct
xchk_meta_ops?
No. Checking doesn't require the rmapbt because all we do is walk the bnobt/cntbt records and cross-reference them with whatever metadata we can find.
A stronger check would scan the AG to build a second recordset of the free space and compare that against what's on disk. However, that would be much slower, and Dave wanted scans to be fast because corruptions are supposed to be the edge case. :)
The weaker checking also means we can scrub old filesystems, even if we still require xfs_repair to fix them.
Quoted 10 lines
fs/xfs/scrub/alloc_repair.c line 863
Quoted 6 lines
fs/xfs/scrub/alloc_repair.c new file +6
-0+/* Set up enough storage to handle maximally fragmented free space. */ + descr =xchk_xfile_ag_descr (sc,"free space records" ); + error =xfarray_create (descr, mp->m_sb.sb_agblocks /2 , +sizeof (struct xfs_alloc_rec_incore ), + &ra->free_records); +kfree (descr);Commenting on a new user of old code here, but why doesn't
xfarray_createsimply take a format string so that we don't need the separate allocatiom and kasprintf here?
I didn't want to spend the brainpower to figure out how to make the macro and va_args crud work to support pushing both from xrep_allocbt -> xfarray_create -> xfile_create. I don't know how to make that stuff nest short of adding a kas- variant of xfile_create.
Seeing as we don't install xfiles into the file descriptor table anyway, the labels are only visible via ftrace, and not procfs. I decided that cleanliness here wasn't a high enough priority.
Quoted 4 lines
Quoted 4 lines
fs/xfs/scrub/alloc_repair.c new file +4
-0+/* +* We must update sm_type temporarily so that the tree-to-tree cross +* reference checks will work in the correct direction, and also so +* that tracing will report correctly if there are more errors.
Quoted 8 lines
fs/xfs/scrub/alloc_repair.c line 905
Quoted 3 lines
fs/xfs/scrub/alloc_repair.c new file +3
-0+*/ + sc->sm->sm_type =XFS_SCRUB_TYPE_BNOBT ; + error =xchk_bnobt (sc);So
xchk_bnobtis a tiny wrapper aroundxchk_allocbt, which is a small wrapper aroundxchk_btreethat basіally de-multiplex the argument pass in byxchk_bnobtagain. This is existing code not newly added, but the call chain looks a bit odd to me.
Yeah. I suppose one way to clean that up would be to export xchk_allocbt to the dispatch table in scrub.c instead of xchk_{bno,cnt}bt and figure out which btree we want from sm_type.
Way back when I was designing scrub I thought that repair would be separate for each btree type, but that turned out not to be the case. Hence the awkwardness in the call chains.
Quoted 18 lines
Quoted 18 lines
fs/xfs/scrub/newbt.c +18
-0+/* +* Add an extent to the new btree reservation pool. Callers are required to +* reap this reservation manually if the repair is cancelled. @pag must be a +* passive reference. +*/ +int +xrep_newbt_add_extent ( +struct xrep_newbt *xnr, +struct xfs_perag *pag, +xfs_agblock_t agbno, +xfs_extlen_t len) +{ +struct xfs_mount *mp = xnr->sc->mp; +struct xfs_alloc_arg args = { + .tp =NULL ,/* no autoreap */ + .oinfo = xnr->oinfo, + .fsbno =XFS_AGB_TO_FSB (mp, pag->pag_agno, agbno), + .len = len,
Quoted 8 lines
fs/xfs/scrub/newbt.c xrep_newbt_add_blocks( · line 194
Quoted 5 lines
fs/xfs/scrub/newbt.c +5
-0+ .resv = xnr->resv, + }; + +return xrep_newbt_add_blocks (xnr, pag, &args); +}I don't quite understand what this helper adds, and the
_blocksvs_extentnaming is a bit confusing.
This wrapper simplifes the interface to xrep_newbt_add_blocks so that external callers don't have to know which magic values of xfs_alloc_arg are actually used by xrep_newbt_add_blocks and therefore need to be set.
For all the other repair functions, they have to allocate space from the free space btree, so xrep_newbt_add_blocks is passed the full _alloc_args as returned by the allocator to xrep_newbt_alloc_ag_blocks.
Quoted 5 lines
fs/xfs/scrub/newbt.h struct xrep_newbt { · line 54
Quoted 2 lines
fs/xfs/scrub/newbt.h +2
-0+#define for_each_xrep_newbt_reservation (xnr, resv, n) \ + list_for_each_entry_safe((resv), (n), &(xnr)->resv_list, list)I have to admit that I find the open code
list_for_each_entry_safeeasier to follow than such wrappers.
The funny part is that I don't even use it in newbt.c. Maybe it's time to get rid of it.
$ git grep for_each_xrep_newbt_reservation fs
fs/xfs/scrub/alloc_repair.c:568: for_each_xrep_newbt_reservation(&ra->new_bnobt, resv, n) {
fs/xfs/scrub/alloc_repair.c:575: for_each_xrep_newbt_reservation(&ra->new_bnobt, resv, n) {
fs/xfs/scrub/newbt.h:60:#define for_each_xrep_newbt_reservation(xnr, resv, n) \
fs/xfs/scrub/rmap_repair.c:1126: for_each_xrep_newbt_reservation(&rr->new_btree, resv, n) {Quoted 11 lines
fs/xfs/scrub/repair.c xrep_ino_dqattach( · line 743
Quoted 6 lines
fs/xfs/scrub/repair.c +6
-0+/* Initialize all the btree cursors for an AG repair. */ +void +xrep_ag_btcur_init ( +struct xfs_scrub *sc, +struct xchk_ag *sa) +{As far as I can tell this basically sets up cursors for all the btrees except the one that we want to repair, and the one that goes along with for the alloc and ialloc pairs? Maybe just spell that out for clarity.
Done.
--D
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 8 lines · Darrick J. Wong
Quoted 5 lines
-
xrep_abt_find_freespaceaccounts the old allocbt blocks as freespace per the comment, although as far as I understand the code inxrep_abt_walk_rmapthe allocbt blocks aren't actually added to thefree_recordsxfarray, but recorded into theold_allocbt_blocksbitmap (btw, why are we using different data structures for them?)The old free space btree blocks are tracked via
old_allocbt_blocksso that we can reap the space after committing the new btree roots.
Yes, I got that from the code. But I'm a bit confused about the comment treating the old allocbt blocks as free space, while the code doesn't. Or I guess the confusion is that we really have two slightly different notions of "free space":
1) the space we try to build the trees for
2) the space used as free space to us for the trees
The old allocbt blocks are part of 1 but not of 2.
Quoted 9 lines
Quoted 3 lines
- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?
xrep_abt_reserve_spacereturns-ENOSPC, so we throw away all the incore records and throw the errno out to userspace. Across all the btree rebuilding code, the block reservation step happens as the very last thing before we start writing btree blocks, so it's still possible to back out cleanly.
But that means we can't repair the allocation btrees for this case.
Quoted 9 lines
Quoted 5 lines
fs/xfs/scrub/alloc_repair.c line 838
Quoted 3 lines
fs/xfs/scrub/alloc_repair.c new file +3
-0+/* We require the rmapbt to rebuild anything. */ +if (!xfs_has_rmapbt (mp)) +return -EOPNOTSUPP ;Shoudn't this be checked by the .has callback in struct
xchk_meta_ops?No. Checking doesn't require the rmapbt because all we do is walk the bnobt/cntbt records and cross-reference them with whatever metadata we can find.
Oh, and .has applies to both checking and repairing. Gt it.
Quoted 7 lines
This wrapper simplifes the interface to
xrep_newbt_add_blocksso that external callers don't have to know which magic values ofxfs_alloc_argare actually used byxrep_newbt_add_blocksand therefore need to be set.For all the other repair functions, they have to allocate space from the free space btree, so
xrep_newbt_add_blocksis passed the full_alloc_argsas returned by the allocator toxrep_newbt_alloc_ag_blocks.
Ok. Maybe throw in a comment that it just is a convenience wrapper?
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 15 lines · Christoph Hellwig
Quoted 8 lines · Darrick J. Wong
Quoted 5 lines
-
xrep_abt_find_freespaceaccounts the old allocbt blocks as freespace per the comment, although as far as I understand the code inxrep_abt_walk_rmapthe allocbt blocks aren't actually added to thefree_recordsxfarray, but recorded into theold_allocbt_blocksbitmap (btw, why are we using different data structures for them?)The old free space btree blocks are tracked via
old_allocbt_blocksso that we can reap the space after committing the new btree roots.Yes, I got that from the code. But I'm a bit confused about the comment treating the old allocbt blocks as free space, while the code doesn't. Or I guess the confusion is that we really have two slightly different notions of "free space":
Quoted 4 lines
1) the space we try to build the trees for
2) the space used as free space to us for the treesThe old allocbt blocks are part of 1 but not of 2.
Yeah. The blocks for #2 only come from the gaps in the rmapbt records that we find during the scan.
Part of the confusion here might be the naming -- at the end of the rmapbt scan, old_allocbt_blocks is set to all the blocks that the rmapbt says are OWN_AG. Each rmapbt block encountered during the scan is added to not_allocbt_blocks, and afterwards each block on the AGFL is also added to not_allocbt_blocks.
After that is where the definitions of old_allocbt_blocks shifts. This expression will help us to identify possible former bnobt/cntbt blocks:
(OWN_AG blocks) & ~(rmapbt blocks | agfl blocks);
Substituting from above definitions, that becomes:
old_allocbt_blocks & ~not_allocbt_blocks
The OWN_AG bitmap itself isn't needed after this point, so what we really do instead is:
old_allocbt_blocks &= ~not_allocbt_blocks;
IOWs, after this point, "old_allocbt_blocks" is a bitmap of alleged former bnobt/cntbt blocks. The xagb_bitmap_disunion operation modifies its first parameter in place to avoid copying records around.
The gaps (aka the new freespace records) are stashed in the free_records xfarray. Next, some of the @free_records are diverted to the newbt reservation and used to format new btree blocks.
I hope that makes things clearer?
Quoted 11 lines
Quoted 9 lines
Quoted 3 lines
- what happens if the AG is so full and fragmented that we do not have space to create a second set of allocbts without tearing down the old ones first?
xrep_abt_reserve_spacereturns-ENOSPC, so we throw away all the incore records and throw the errno out to userspace. Across all the btree rebuilding code, the block reservation step happens as the very last thing before we start writing btree blocks, so it's still possible to back out cleanly.But that means we can't repair the allocation btrees for this case.
Yep. At least we revert cleanly here -- when that happens to xfs_repair it just blows up and leaves a dead fs. :/
Practically speaking, the rmapbt reservations are large enough that we can rebuild the trees if we have to, even if that just means stealing from the per-AG space reservations later.
Though you could create the fs from hell by using reflink and cow to fragmenting the rmapbt until it swells up and consumes the entire AG. That would require billions of fragments; even with my bmap inflation xfs_db commands that still takes about 2 days to get the rmapbt to the point of overflowing the regular reservation.
I tried pushing it to eat an entire AG but then the ILOM exploded and no idea what happened to that machine and its 512G of RAM. The lab people kind of hate me now.
Quoted 11 lines
Quoted 9 lines
Quoted 5 lines
fs/xfs/scrub/alloc_repair.c line 838
Quoted 3 lines
fs/xfs/scrub/alloc_repair.c new file +3
-0+/* We require the rmapbt to rebuild anything. */ +if (!xfs_has_rmapbt (mp)) +return -EOPNOTSUPP ;Shoudn't this be checked by the .has callback in struct
xchk_meta_ops?No. Checking doesn't require the rmapbt because all we do is walk the bnobt/cntbt records and cross-reference them with whatever metadata we can find.
Oh, and .has applies to both checking and repairing. Gt it.
Correct.
Quoted 9 lines
Quoted 7 lines
This wrapper simplifes the interface to
xrep_newbt_add_blocksso that external callers don't have to know which magic values ofxfs_alloc_argare actually used byxrep_newbt_add_blocksand therefore need to be set.For all the other repair functions, they have to allocate space from the free space btree, so
xrep_newbt_add_blocksis passed the full_alloc_argsas returned by the allocator toxrep_newbt_alloc_ag_blocks.Ok. Maybe throw in a comment that it just is a convenience wrapper?
Will do.
--D
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 5 lines · Darrick J. Wong
The gaps (aka the new freespace records) are stashed in the
free_recordsxfarray. Next, some of the @free_records are diverted to the newbt reservation and used to format new btree blocks.I hope that makes things clearer?
Yes. Can you add this to the code as a comment?
Re: [PATCH 3/5] xfs: repair free space btrees
Quoted 8 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
The gaps (aka the new freespace records) are stashed in the
free_recordsxfarray. Next, some of the @free_records are diverted to the newbt reservation and used to format new btree blocks.I hope that makes things clearer?
Yes. Can you add this to the code as a comment?
Will do.
--D
PATCH 4/5 xfs: repair inode btrees
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use the rmapbt to find inode chunks, query the chunks to compute hole and free masks, and with that information rebuild the inobt and finobt.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_ialloc.c | 41 +- fs/xfs/libxfs/xfs_ialloc.h | 3 fs/xfs/scrub/common.c | 1 fs/xfs/scrub/ialloc_repair.c | 874 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.c | 59 +++ fs/xfs/scrub/repair.h | 17 + fs/xfs/scrub/scrub.c | 6 fs/xfs/scrub/scrub.h | 1 fs/xfs/scrub/trace.h | 68 ++- 10 files changed, 1022 insertions(+), 49 deletions(-) create mode 100644 fs/xfs/scrub/ialloc_repair.c
fs/xfs/Makefile +1 -0
@@ -182,6 +182,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ alloc_repair.o \ + ialloc_repair.o \ newbt.o \ reap.o \ repair.o \
fs/xfs/libxfs/xfs_ialloc.c +28 -13
63 lines not on this page · raw message
fs/xfs/libxfs/xfs_ialloc.h +3 -0
@@ -79,6 +79,7 @@ int xfs_inobt_lookup(struct xfs_btree_cur *cur, xfs_agino_t ino,*/ int xfs_inobt_get_rec (struct xfs_btree_cur *cur,xfs_inobt_rec_incore_t *rec,int *stat); +uint8_t xfs_inobt_rec_freecount (const struct xfs_inobt_rec_incore *irec);/* * Inode chunk initialisation routine
@@ -93,6 +94,8 @@ union xfs_btree_rec;void xfs_inobt_btrec_to_irec (struct xfs_mount *mp,const union xfs_btree_rec *rec,struct xfs_inobt_rec_incore *irec); +xfs_failaddr_t xfs_inobt_check_perag_irec (struct xfs_perag *pag, +const struct xfs_inobt_rec_incore *irec);xfs_failaddr_t xfs_inobt_check_irec (struct xfs_btree_cur *cur,const struct xfs_inobt_rec_incore *irec); int xfs_ialloc_has_inodes_at_extent(struct xfs_btree_cur *cur,
fs/xfs/scrub/common.c +1 -0
@@ -605,6 +605,7 @@ xchk_ag_free( struct xchk_ag *sa) {xchk_ag_btcur_free (sa); +xrep_reset_perag_resv (sc);if (sa->agf_bp) {xfs_trans_brelse (sc->tp, sa->agf_bp); sa->agf_bp =NULL ;
fs/xfs/scrub/ialloc_repair.c new file +874 -0
874 lines not on this page · raw message
fs/xfs/scrub/repair.c +59 -0
62 lines not on this page · raw message
fs/xfs/scrub/repair.h +17 -0
@@ -59,6 +59,7 @@ int xrep_find_ag_btree_roots(struct xfs_scrub *sc, struct xfs_buf *agf_bp, struct xrep_find_ag_btree *btree_info, struct xfs_buf *agfl_bp);void xrep_force_quotacheck (struct xfs_scrub *sc,xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc); +int xrep_reset_perag_resv (struct xfs_scrub *sc);/* Repair setup functions */ int xrep_setup_ag_allocbt (struct xfs_scrub *sc);
@@ -68,6 +69,7 @@ void xrep_ag_btcur_init(struct xfs_scrub *sc, struct xchk_ag *sa);/* Metadata revalidators */ int xrep_revalidate_allocbt (struct xfs_scrub *sc); +int xrep_revalidate_iallocbt (struct xfs_scrub *sc);/* Metadata repairers */
@@ -77,8 +79,10 @@ int xrep_agf(struct xfs_scrub *sc);int xrep_agfl (struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc);int xrep_allocbt (struct xfs_scrub *sc); +int xrep_iallocbt (struct xfs_scrub *sc);int xrep_reinit_pagf (struct xfs_scrub *sc); +int xrep_reinit_pagi (struct xfs_scrub *sc);#else
@@ -99,6 +103,17 @@ xrep_calc_ag_resblks(return 0 ; } +static inline int +xrep_reset_perag_resv ( +struct xfs_scrub *sc) +{ +if (!(sc->flags &XREP_RESET_PERAG_RESV )) +return 0 ; + +ASSERT (0 ); +return -EOPNOTSUPP ; +} +/* repair setup functions for no-repair */ static inline int xrep_setup_nothing(
@@ -109,6 +124,7 @@ xrep_setup_nothing(#define xrep_setup_ag_allocbt xrep_setup_nothing#define xrep_revalidate_allocbt (NULL) +#define xrep_revalidate_iallocbt (NULL)#define xrep_probe xrep_notsupported#define xrep_superblock xrep_notsupported
@@ -116,6 +132,7 @@ xrep_setup_nothing(#define xrep_agfl xrep_notsupported#define xrep_agi xrep_notsupported#define xrep_allocbt xrep_notsupported +#define xrep_iallocbt xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +4 -2
@@ -255,14 +255,16 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_PERAG, .setup = xchk_setup_ag_iallocbt, .scrub = xchk_inobt, - .repair = xrep_notsupported, + .repair = xrep_iallocbt, + .repair_eval = xrep_revalidate_iallocbt, }, [XFS_SCRUB_TYPE_FINOBT] = { /* finobt */ .type = ST_PERAG, .setup = xchk_setup_ag_iallocbt, .scrub = xchk_finobt, .has = xfs_has_finobt, - .repair = xrep_notsupported, + .repair = xrep_iallocbt, + .repair_eval = xrep_revalidate_iallocbt, }, [XFS_SCRUB_TYPE_RMAPBT] = { /* rmapbt */ .type = ST_PERAG,
fs/xfs/scrub/scrub.h +1 -0
@@ -121,6 +121,7 @@ struct xfs_scrub {#define XCHK_HAVE_FREEZE_PROT (1U << 1)/* do we have freeze protection? */ #define XCHK_FSGATES_DRAIN (1U << 2)/* defer ops draining enabled */ #define XCHK_NEED_DRAIN (1U << 3)/* scrub needs to drain defer ops */ +#define XREP_RESET_PERAG_RESV (1U << 30)/* must reset AG space reservation */ #define XREP_ALREADY_FIXED (1U << 31)/* checking our repair work */ /*
fs/xfs/scrub/trace.h +34 -34
92 lines not on this page · raw message
Re: [PATCH 4/5] xfs: repair inode btrees
fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_check_irec( · line 142
Quoted 8 lines
fs/xfs/libxfs/xfs_ialloc.c +8
-0+/* Simple checks for inode records. */ +xfs_failaddr_t +xfs_inobt_check_irec ( +struct xfs_btree_cur *cur, +const struct xfs_inobt_rec_incore *irec) +{ +return xfs_inobt_check_perag_irec (cur->bc_ag.pag, irec); +}
Same comment about just dropping the wrapper. Otherwise I'll need more digestion time for the new code.
Re: [PATCH 4/5] xfs: repair inode btrees
Quoted 11 lines · Christoph Hellwig
fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_check_irec( · line 142
Quoted 8 lines
fs/xfs/libxfs/xfs_ialloc.c +8
-0+/* Simple checks for inode records. */ +xfs_failaddr_t +xfs_inobt_check_irec ( +struct xfs_btree_cur *cur, +const struct xfs_inobt_rec_incore *irec) +{ +return xfs_inobt_check_perag_irec (cur->bc_ag.pag, irec); +}Same comment about just dropping the wrapper. Otherwise I'll need more digestion time for the new code.
Done.
--D
Re: [PATCH 4/5] xfs: repair inode btrees
In reply to Darrick J. WongThis generally looks good to me.
A bunch of my superficial comments to the previous patch apply here as well, but I'm not going to repeat them, but I have a bunch of new just as nitpicky ones:
fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_btrec_to_irec( · line 108
Quoted 6 lines
fs/xfs/libxfs/xfs_ialloc.c +5
-0+uint64_t realfree; +if (!xfs_inobt_issparse (irec->ir_holemask)) + realfree = irec->ir_free; +else + realfree = irec->ir_free &xfs_inobt_irec_to_allocmask (irec);
Nit:
I'd write this as:
uint64_t realfree = irec->ir_free; if (xfs_inobt_issparse(irec->ir_holemask)) realfree &= xfs_inobt_irec_to_allocmask(irec); return hweight64(realfree);
to simplify the logic a bit (and yes, I see the sniplet was just copied out of an existing function).
Quoted 12 lines
fs/xfs/scrub/ialloc_repair.c new file +12
-0+/* Record extents that belong to inode btrees. */ +STATIC int +xrep_ibt_walk_rmap ( +struct xfs_btree_cur *cur, +const struct xfs_rmap_irec *rec, +void *priv) +{ +struct xrep_ibt *ri = priv; +struct xfs_mount *mp = cur->bc_mp; +struct xfs_ino_geometry *igeo =M_IGEO (mp); +xfs_agblock_t cluster_base; +int error =0 ;
fs/xfs/scrub/ialloc_repair.c line 430
Quoted 10 lines
fs/xfs/scrub/ialloc_repair.c new file +10
-0+ +if (xchk_should_terminate (ri->sc, &error)) +return error; + +if (rec->rm_owner ==XFS_RMAP_OWN_INOBT ) +return xrep_ibt_record_old_btree_blocks (ri, rec); + +/* Skip extents which are not owned by this inode and fork. */ +if (rec->rm_owner !=XFS_RMAP_OWN_INODES ) +return 0 ;
The "Skip extents.." comment is clearly wrong and looks like it got here by accident. And may ocaml-trained ind screams for a switch statement and another helper for the rest of the functin body here:
switch (rec->rm_owner) {
case XFS_RMAP_OWN_INOBT:
return xrep_ibt_record_old_btree_blocks(ri, rec);
case XFS_RMAP_OWN_INODES:
return xrep_ibt_record_inode_blocks(mp, ri, rec);
default:
return 0;fs/xfs/scrub/ialloc_repair.c line 484
Quoted 6 lines
fs/xfs/scrub/ialloc_repair.c new file +6
-0+/* If we have a record ready to go, add it to the array. */ +if (ri->rie.ir_startino ==NULLAGINO ) +return 0 ; + +return xrep_ibt_stash (ri); +}
Superficial, but having the logic inverted from the comment makes my brain a little dizzy. Anything again:
if (ri->rie.ir_startino != NULLAGINO) error = xrep_ibt_stash(ri); return error;
?
fs/xfs/scrub/ialloc_repair.c line 594
Quoted 3 lines
fs/xfs/scrub/ialloc_repair.c new file +3
-0+/* Make sure the records do not overlap in inumber address space. */ +STATIC int +xrep_ibt_check_startino(
Would xrep_ibt_check_overlap be a better name here?
Re: [PATCH 4/5] xfs: repair inode btrees
Quoted 5 lines · Christoph Hellwig
This generally looks good to me.
A bunch of my superficial comments to the previous patch apply here as well, but I'm not going to repeat them, but I have a bunch of new just as nitpicky ones:
I already fixed the nitpicks from yesterday. :)
Quoted 12 lines
fs/xfs/libxfs/xfs_ialloc.c xfs_inobt_btrec_to_irec( · line 108
Quoted 6 lines
fs/xfs/libxfs/xfs_ialloc.c +5
-0+uint64_t realfree; +if (!xfs_inobt_issparse (irec->ir_holemask)) + realfree = irec->ir_free; +else + realfree = irec->ir_free &xfs_inobt_irec_to_allocmask (irec);Nit:
I'd write this as:
Quoted 8 lines
uint64_t realfree = irec->ir_free; if (xfs_inobt_issparse(irec->ir_holemask)) realfree &= xfs_inobt_irec_to_allocmask(irec); return hweight64(realfree);to simplify the logic a bit (and yes, I see the sniplet was just copied out of an existing function).
Ok.
Quoted 17 lines
Quoted 17 lines
fs/xfs/scrub/ialloc_repair.c new file +17
-0+/* Record extents that belong to inode btrees. */ +STATIC int +xrep_ibt_walk_rmap ( +struct xfs_btree_cur *cur, +const struct xfs_rmap_irec *rec, +void *priv) +{ +struct xrep_ibt *ri = priv; +struct xfs_mount *mp = cur->bc_mp; +struct xfs_ino_geometry *igeo =M_IGEO (mp); +xfs_agblock_t cluster_base; +int error =0 ; + +if (xchk_should_terminate (ri->sc, &error)) +return error; + +if (rec->rm_owner ==XFS_RMAP_OWN_INOBT )
Quoted 8 lines
fs/xfs/scrub/ialloc_repair.c line 430
Quoted 5 lines
fs/xfs/scrub/ialloc_repair.c new file +5
-0+return xrep_ibt_record_old_btree_blocks (ri, rec); + +/* Skip extents which are not owned by this inode and fork. */ +if (rec->rm_owner !=XFS_RMAP_OWN_INODES ) +return 0 ;The "Skip extents.." comment is clearly wrong and looks like it got here by accident.
Yep. "Skip mappings that are not inode records.", sorry about that.
Quoted 10 lines
And may ocaml-trained ind screams for a switch
statement and another helper for the rest of the functin body here:switch (rec->rm_owner) { case XFS_RMAP_OWN_INOBT: return xrep_ibt_record_old_btree_blocks(ri, rec); case XFS_RMAP_OWN_INODES: return xrep_ibt_record_inode_blocks(mp, ri, rec); default: return 0;
Sounds good to me.
Quoted 10 lines
fs/xfs/scrub/ialloc_repair.c line 484
Quoted 6 lines
fs/xfs/scrub/ialloc_repair.c new file +6
-0+/* If we have a record ready to go, add it to the array. */ +if (ri->rie.ir_startino ==NULLAGINO ) +return 0 ; + +return xrep_ibt_stash (ri); +}Superficial, but having the logic inverted from the comment makes my brain a little dizzy. Anything again:
Quoted 6 lines
if (ri->rie.ir_startino != NULLAGINO) error = xrep_ibt_stash(ri); return error;?
Done.
Quoted 5 lines
fs/xfs/scrub/ialloc_repair.c line 594
Quoted 3 lines
fs/xfs/scrub/ialloc_repair.c new file +3
-0+/* Make sure the records do not overlap in inumber address space. */ +STATIC int +xrep_ibt_check_startino(Would
xrep_ibt_check_overlapbe a better name here?
Yes! Thank you for the suggestion.
--D
Quoted 2 lines
PATCH 5/5 xfs: repair refcount btrees
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Reconstruct the refcount data from the rmap btree.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_ag.h | 1 fs/xfs/libxfs/xfs_btree.c | 26 + fs/xfs/libxfs/xfs_btree.h | 2 fs/xfs/libxfs/xfs_refcount.c | 18 + fs/xfs/libxfs/xfs_refcount.h | 2 fs/xfs/libxfs/xfs_refcount_btree.c | 13 + fs/xfs/scrub/refcount_repair.c | 793 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 2 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 22 + 11 files changed, 864 insertions(+), 18 deletions(-) create mode 100644 fs/xfs/scrub/refcount_repair.c
fs/xfs/Makefile +1 -0
@@ -185,6 +185,7 @@ xfs-y += $(addprefix scrub/, \ ialloc_repair.o \ newbt.o \ reap.o \ + refcount_repair.o \ repair.o \ ) endif
fs/xfs/libxfs/xfs_ag.h +1 -0
@@ -87,6 +87,7 @@ struct xfs_perag {* verifiers while rebuilding the AG btrees. */ uint8_t pagf_alt_levels[XFS_BTNUM_AGF ]; +uint8_t pagf_alt_refcount_level;#endif spinlock_t pag_state_lock;
fs/xfs/libxfs/xfs_btree.c +26 -0
@@ -5212,3 +5212,29 @@ xfs_btree_destroy_cur_caches(void)xfs_rmapbt_destroy_cur_cache ();xfs_refcountbt_destroy_cur_cache (); } + +/* Move the btree cursor before the first record. */ +int +xfs_btree_goto_left_edge ( +struct xfs_btree_cur *cur) +{ +int stat =0 ; +int error; + +memset (&cur->bc_rec,0 ,sizeof (cur->bc_rec)); + error =xfs_btree_lookup (cur,XFS_LOOKUP_LE , &stat); +if (error) +return error; +if (!stat) +return 0 ; + + error =xfs_btree_decrement (cur,0 , &stat); +if (error) +return error; +if (stat !=0 ) { +ASSERT (0 ); +return -EFSCORRUPTED ; + } + +return 0 ; +}
fs/xfs/libxfs/xfs_btree.h +2 -0
@@ -738,4 +738,6 @@ xfs_btree_alloc_cursor(int __init xfs_btree_init_cur_caches (void );void xfs_btree_destroy_cur_caches (void ); +int xfs_btree_goto_left_edge (struct xfs_btree_cur *cur); +#endif /* __XFS_BTREE_H__ */
fs/xfs/libxfs/xfs_refcount.c +12 -6
@@ -120,14 +120,11 @@ xfs_refcount_btrec_to_irec( irec->rc_refcount =be32_to_cpu (rec->refc.rc_refcount); } -/* Simple checks for refcount records. */ -xfs_failaddr_t -xfs_refcount_check_irec ( -struct xfs_btree_cur *cur, +inline xfs_failaddr_t +xfs_refcount_check_perag_irec ( +struct xfs_perag *pag,const struct xfs_refcount_irec *irec) { -struct xfs_perag *pag = cur->bc_ag.pag; -if (irec->rc_blockcount ==0 || irec->rc_blockcount >MAXREFCEXTLEN )return __this_address;
@@ -144,6 +141,15 @@ xfs_refcount_check_irec(return NULL ; } +/* Simple checks for refcount records. */ +xfs_failaddr_t +xfs_refcount_check_irec ( +struct xfs_btree_cur *cur, +const struct xfs_refcount_irec *irec) +{ +return xfs_refcount_check_perag_irec (cur->bc_ag.pag, irec); +} + static inline int xfs_refcount_complain_bad_rec( struct xfs_btree_cur *cur,
fs/xfs/libxfs/xfs_refcount.h +2 -0
@@ -117,6 +117,8 @@ extern int xfs_refcount_has_records(struct xfs_btree_cur *cur,union xfs_btree_rec ;extern void xfs_refcount_btrec_to_irec (const union xfs_btree_rec *rec,struct xfs_refcount_irec *irec); +xfs_failaddr_t xfs_refcount_check_perag_irec (struct xfs_perag *pag, +const struct xfs_refcount_irec *irec);xfs_failaddr_t xfs_refcount_check_irec (struct xfs_btree_cur *cur,const struct xfs_refcount_irec *irec); extern int xfs_refcount_insert(struct xfs_btree_cur *cur,
fs/xfs/libxfs/xfs_refcount_btree.c +12 -1
@@ -226,7 +226,18 @@ xfs_refcountbt_verify( level =be16_to_cpu (block->bb_level);if (pag &&xfs_perag_initialised_agf (pag)) { -if (level >= pag->pagf_refcount_level) +unsigned int maxlevel = pag->pagf_refcount_level; + +#ifdef CONFIG_XFS_ONLINE_REPAIR +/* +* Online repair could be rewriting the refcount btree, so +* we'll validate against the larger of either tree while this +* is going on. +*/ + maxlevel = max_t(unsigned int, maxlevel, + pag->pagf_alt_refcount_level); +#endif +if (level >= maxlevel)return __this_address; }else if (level >= mp->m_refc_maxlevels)return __this_address;
fs/xfs/scrub/refcount_repair.c new file +793 -0
793 lines not on this page · raw message
fs/xfs/scrub/repair.h +2 -0
@@ -80,6 +80,7 @@ int xrep_agfl(struct xfs_scrub *sc);int xrep_agi (struct xfs_scrub *sc);int xrep_allocbt (struct xfs_scrub *sc);int xrep_iallocbt (struct xfs_scrub *sc); +int xrep_refcountbt (struct xfs_scrub *sc);int xrep_reinit_pagf (struct xfs_scrub *sc);int xrep_reinit_pagi (struct xfs_scrub *sc);
@@ -133,6 +134,7 @@ xrep_setup_nothing(#define xrep_agi xrep_notsupported#define xrep_allocbt xrep_notsupported#define xrep_iallocbt xrep_notsupported +#define xrep_refcountbt xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -278,7 +278,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .setup = xchk_setup_ag_refcountbt, .scrub = xchk_refcountbt, .has = xfs_has_reflink, - .repair = xrep_notsupported, + .repair = xrep_refcountbt, }, [XFS_SCRUB_TYPE_INODE] = { /* inode record */ .type = ST_INODE,
fs/xfs/scrub/trace.h +12 -10
@@ -1232,27 +1232,29 @@ TRACE_EVENT(xrep_ibt_found, __entry->freemask) ) -TRACE_EVENT (xrep_refcount_extent_fn, - TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, - struct xfs_refcount_irec *irec), -TP_ARGS (mp, agno, irec), +TRACE_EVENT (xrep_refc_found, + TP_PROTO(struct xfs_perag *pag, const struct xfs_refcount_irec *rec), +TP_ARGS (pag, rec),TP_STRUCT__entry ( __field(dev_t, dev) __field(xfs_agnumber_t, agno) + __field(enum xfs_refc_domain, domain) __field(xfs_agblock_t, startblock) __field(xfs_extlen_t, blockcount)__field (xfs_nlink_t, refcount) ),TP_fast_assign ( - __entry->dev = mp->m_super->s_dev; - __entry->agno = agno; - __entry->startblock = irec->rc_startblock; - __entry->blockcount = irec->rc_blockcount; - __entry->refcount = irec->rc_refcount; + __entry->dev = pag->pag_mount->m_super->s_dev; + __entry->agno = pag->pag_agno; + __entry->domain = rec->rc_domain; + __entry->startblock = rec->rc_startblock; + __entry->blockcount = rec->rc_blockcount; + __entry->refcount = rec->rc_refcount; ), -TP_printk ("dev %d:%d agno 0x%x agbno 0x%x fsbcount 0x%x refcount %u" , +TP_printk ("dev %d:%d agno 0x%x dom %s agbno 0x%x fsbcount 0x%x refcount %u" ,MAJOR (__entry->dev),MINOR (__entry->dev), __entry->agno, +__print_symbolic (__entry->domain,XFS_REFC_DOMAIN_STRINGS ), __entry->startblock, __entry->blockcount, __entry->refcount)
Re: [PATCH 5/5] xfs: repair refcount btrees
Besides all the nitpicks that are the same as for the previous two patches, this looks good to me:
cover xfs: online repair of inodes and forks
In reply to Darrick J. WongHi all,
In this series, online repair gains the ability to repair inode records. To do this, we must repair the ondisk inode and fork information enough to pass the iget verifiers and hence make the inode igettable again. Once that's done, we can perform higher level repairs on the incore inode. The fstests counterpart of this patchset implements stress testing of repair.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-inodes
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-inodes
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-inodes
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_attr_leaf.c | 32 - fs/xfs/libxfs/xfs_attr_leaf.h | 2 fs/xfs/libxfs/xfs_bmap.c | 22 fs/xfs/libxfs/xfs_bmap.h | 2 fs/xfs/libxfs/xfs_dir2_priv.h | 2 fs/xfs/libxfs/xfs_dir2_sf.c | 29 - fs/xfs/libxfs/xfs_format.h | 3 fs/xfs/libxfs/xfs_shared.h | 1 fs/xfs/libxfs/xfs_symlink_remote.c | 21 fs/xfs/scrub/bmap.c | 48 + fs/xfs/scrub/common.c | 26 + fs/xfs/scrub/common.h | 8 fs/xfs/scrub/dir.c | 21 fs/xfs/scrub/inode.c | 14 fs/xfs/scrub/inode_repair.c | 1659 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/parent.c | 17 fs/xfs/scrub/repair.c | 57 + fs/xfs/scrub/repair.h | 29 + fs/xfs/scrub/rtbitmap.c | 4 fs/xfs/scrub/rtsummary.c | 4 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 174 ++++ 23 files changed, 2128 insertions(+), 50 deletions(-) create mode 100644 fs/xfs/scrub/inode_repair.c
PATCH 1/7 xfs: disable online repair quota helpers when quota not enabled
From: Darrick J. Wong <[email protected]>
Don't compile the quota helper functions if quota isn't being built into the XFS module.
--- fs/xfs/scrub/repair.c | 2 ++ fs/xfs/scrub/repair.h | 9 +++++++++ 2 files changed, 11 insertions(+)
fs/xfs/scrub/repair.c +2 -0
@@ -673,6 +673,7 @@ xrep_find_ag_btree_roots(return error; } +#ifdef CONFIG_XFS_QUOTA /* Force a quotacheck the next time we mount. */ void xrep_force_quotacheck(
@@ -734,6 +735,7 @@ xrep_ino_dqattach( return error; } +#endif /* CONFIG_XFS_QUOTA */ /* Initialize all the btree cursors for an AG repair. */ void
fs/xfs/scrub/repair.h +9 -0
@@ -57,8 +57,15 @@ struct xrep_find_ag_btree {int xrep_find_ag_btree_roots (struct xfs_scrub *sc,struct xfs_buf *agf_bp,struct xrep_find_ag_btree *btree_info,struct xfs_buf *agfl_bp); + +#ifdef CONFIG_XFS_QUOTA void xrep_force_quotacheck (struct xfs_scrub *sc,xfs_dqtype_t type);int xrep_ino_dqattach (struct xfs_scrub *sc); +#else +# define xrep_force_quotacheck (sc, type) ((void)0) +# define xrep_ino_dqattach (sc) (0) +#endif /* CONFIG_XFS_QUOTA */ +int xrep_reset_perag_resv (struct xfs_scrub *sc);/* Repair setup functions */
@@ -87,6 +94,8 @@ int xrep_reinit_pagi(struct xfs_scrub *sc);#else +#define xrep_ino_dqattach (sc) (0) + static inline int xrep_attempt( struct xfs_scrub *sc,
Re: [PATCH 1/7] xfs: disable online repair quota helpers when quota not enabled
Looks good:
PATCH 2/7 xfs: try to attach dquots to files before repairing them
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Inode resource usage is tracked in the quota metadata. Repairing a file might change the resources used by that file, which means that we need to attach dquots to the file that we're examining before accessing anything in the file protected by the ILOCK.
However, there's a twist: a dquot cache miss requires the dquot to be read in from the quota file, during which we drop the ILOCK on the file being examined. This means that we *must* try to attach the dquots before taking the ILOCK.
Therefore, dquots must be attached to files in the scrub setup function. If doing so yields corruption errors (or unknown dquot errors), we instead clear the quotachecked status, which will cause a quotacheck on next mount. A future series will make this trigger live quotacheck.
While we're here, change the xrep_ino_dqattach function to use the unlocked dqattach functions so that we avoid cycling the ILOCK if the inode already has dquots attached. This makes the naming and locking requirements consistent with the rest of the filesystem.
--- fs/xfs/scrub/bmap.c | 4 ++++ fs/xfs/scrub/common.c | 25 +++++++++++++++++++++++++ fs/xfs/scrub/common.h | 6 ++++++ fs/xfs/scrub/inode.c | 4 ++++ fs/xfs/scrub/repair.c | 13 ++++++++----- fs/xfs/scrub/rtbitmap.c | 4 ++++ fs/xfs/scrub/rtsummary.c | 4 ++++ 7 files changed, 55 insertions(+), 5 deletions(-)
fs/xfs/scrub/bmap.c +4 -0
@@ -78,6 +78,10 @@ xchk_setup_inode_bmap(if (error) goto out; + error =xchk_ino_dqattach (sc); +if (error) + goto out; +xchk_ilock (sc,XFS_ILOCK_EXCL ); out:/* scrub teardown will unlock and release the inode */
fs/xfs/scrub/common.c +25 -0
@@ -820,6 +820,26 @@ xchk_iget_agi(return 0 ; } +#ifdef CONFIG_XFS_QUOTA +/* +* Try to attach dquots to this inode if we think we might want to repair it. +* Callers must not hold any ILOCKs. If the dquots are broken and cannot be +* attached, a quotacheck will be scheduled. +*/ +int +xchk_ino_dqattach ( +struct xfs_scrub *sc) +{ +ASSERT (sc->tp !=NULL ); +ASSERT (sc->ip !=NULL ); + +if (!xchk_could_repair (sc)) +return 0 ; + +return xrep_ino_dqattach (sc); +} +#endif +/* Install an inode that we opened by handle for scrubbing. */ int xchk_install_handle_inode(
@@ -1031,6 +1051,11 @@ xchk_setup_inode_contents( error =xchk_trans_alloc (sc, resblks);if (error) goto out; + + error =xchk_ino_dqattach (sc); +if (error) + goto out; +xchk_ilock (sc,XFS_ILOCK_EXCL ); out:/* scrub teardown will unlock and release the inode for us */
fs/xfs/scrub/common.h +6 -0
@@ -103,9 +103,15 @@ xchk_setup_rtsummary(struct xfs_scrub *sc) }#endif #ifdef CONFIG_XFS_QUOTA +int xchk_ino_dqattach (struct xfs_scrub *sc);int xchk_setup_quota (struct xfs_scrub *sc);#else static inline int +xchk_ino_dqattach (struct xfs_scrub *sc) +{ +return 0 ; +} +static inline int xchk_setup_quota (struct xfs_scrub *sc) {return -ENOENT ;
fs/xfs/scrub/inode.c +4 -0
@@ -39,6 +39,10 @@ xchk_prepare_iscrub(if (error)return error; + error =xchk_ino_dqattach (sc); +if (error) +return error; +xchk_ilock (sc,XFS_ILOCK_EXCL );return 0 ; }
fs/xfs/scrub/repair.c +8 -5
@@ -700,10 +700,10 @@ xrep_force_quotacheck(* * This function ensures that the appropriate dquots are attached to an inode. * We cannot allow the dquot code to allocate an on-disk dquot block here -* because we're already in transaction context with the inode locked. The -* on-disk dquot should already exist anyway. If the quota code signals -* corruption or missing quota information, schedule quotacheck, which will -* repair corruptions in the quota metadata. +* because we're already in transaction context. The on-disk dquot should +* already exist anyway. If the quota code signals corruption or missing quota +* information, schedule quotacheck, which will repair corruptions in the quota +* metadata. */ int xrep_ino_dqattach(
@@ -711,7 +711,10 @@ xrep_ino_dqattach( {int error; - error =xfs_qm_dqattach_locked (sc->ip, false); +ASSERT (sc->tp !=NULL ); +ASSERT (sc->ip !=NULL ); + + error =xfs_qm_dqattach (sc->ip);switch (error) {case -EFSBADCRC :case -EFSCORRUPTED :
fs/xfs/scrub/rtbitmap.c +4 -0
@@ -32,6 +32,10 @@ xchk_setup_rtbitmap(if (error)return error; + error =xchk_ino_dqattach (sc); +if (error) +return error; +xchk_ilock (sc,XFS_ILOCK_EXCL |XFS_ILOCK_RTBITMAP );return 0 ; }
fs/xfs/scrub/rtsummary.c +4 -0
@@ -63,6 +63,10 @@ xchk_setup_rtsummary(if (error)return error; + error =xchk_ino_dqattach (sc); +if (error) +return error; +/* * Locking order requires us to take the rtbitmap first. We must be * careful to unlock it ourselves when we are done with the rtbitmap
Re: [PATCH 2/7] xfs: try to attach dquots to files before repairing them
Looks good:
PATCH 3/7 xfs: repair inode records
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
If an inode is so badly damaged that it cannot be loaded into the cache, fix the ondisk metadata and try again. If there /is/ a cached inode, fix any problems and apply any optimizations that can be solved incore.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_format.h | 3 fs/xfs/scrub/inode.c | 10 - fs/xfs/scrub/inode_repair.c | 804 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.c | 42 ++ fs/xfs/scrub/repair.h | 20 + fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 129 +++++++ 8 files changed, 1008 insertions(+), 3 deletions(-) create mode 100644 fs/xfs/scrub/inode_repair.c
fs/xfs/Makefile +1 -0
@@ -183,6 +183,7 @@ xfs-y += $(addprefix scrub/, \ agheader_repair.o \ alloc_repair.o \ ialloc_repair.o \ + inode_repair.o \ newbt.o \ reap.o \ refcount_repair.o \
fs/xfs/libxfs/xfs_format.h +2 -1
@@ -1012,7 +1012,8 @@ enum xfs_dinode_fmt {#define XFS_DFORK_APTR (dip) \ (XFS_DFORK_DPTR(dip) + XFS_DFORK_BOFF(dip))#define XFS_DFORK_PTR (dip,w) \ - ((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : XFS_DFORK_APTR(dip)) + ((void *)((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : \ + XFS_DFORK_APTR(dip)))#define XFS_DFORK_FORMAT (dip,w) \ ((w) == XFS_DATA_FORK ? \
fs/xfs/scrub/inode.c +9 -1
@@ -25,6 +25,7 @@#include "scrub/common.h" #include "scrub/btree.h" #include "scrub/trace.h" +#include "scrub/repair.h" /* Prepare the attached inode for scrubbing. */ static inline int
@@ -185,8 +186,11 @@ xchk_setup_inode(* saying the inode is allocated and the icache being unable to load * the inode until we can flag the corruption in xchk_inode. The * scrub function has to note the corruption, since we're not really -* supposed to do that from the setup function. +* supposed to do that from the setup function. Save the mapping to +* make repairs to the ondisk inode buffer. */ +if (xchk_could_repair (sc)) +xrep_setup_inode (sc, &imap);return 0 ; out_cancel:
@@ -342,6 +346,10 @@ xchk_inode_flags2(if (xfs_dinode_has_bigtime (dip) && !xfs_has_bigtime (mp)) goto bad; +/* no large extent counts without the filesystem feature */ +if ((flags2 &XFS_DIFLAG2_NREXT64 ) && !xfs_has_large_extent_counts (mp)) + goto bad; +return ; bad:xchk_ino_set_corrupt (sc, ino);
fs/xfs/scrub/inode_repair.c new file +804 -0
804 lines not on this page · raw message
fs/xfs/scrub/repair.c +42 -0
54 lines not on this page · raw message
fs/xfs/scrub/repair.h +20 -0
@@ -30,11 +30,22 @@ static inline int xrep_notsupported(struct xfs_scrub *sc)int xrep_attempt (struct xfs_scrub *sc,struct xchk_stats_run *run);void xrep_failure (struct xfs_mount *mp);int xrep_roll_ag_trans (struct xfs_scrub *sc); +int xrep_roll_trans (struct xfs_scrub *sc);int xrep_defer_finish (struct xfs_scrub *sc);bool xrep_ag_has_space (struct xfs_perag *pag,xfs_extlen_t nr_blocks,enum xfs_ag_resv_type type);xfs_extlen_t xrep_calc_ag_resblks (struct xfs_scrub *sc); +static inline int +xrep_trans_commit ( +struct xfs_scrub *sc) +{ +int error =xfs_trans_commit (sc->tp); + + sc->tp =NULL ; +return error; +} +struct xbitmap ;struct xagb_bitmap ;
@@ -66,11 +77,16 @@ int xrep_ino_dqattach(struct xfs_scrub *sc);# define xrep_ino_dqattach (sc) (0)#endif /* CONFIG_XFS_QUOTA */ +int xrep_ino_ensure_extent_count (struct xfs_scrub *sc,int whichfork, +xfs_extnum_t nextents);int xrep_reset_perag_resv (struct xfs_scrub *sc);/* Repair setup functions */ int xrep_setup_ag_allocbt (struct xfs_scrub *sc); +struct xfs_imap ; +int xrep_setup_inode (struct xfs_scrub *sc,struct xfs_imap *imap); +void xrep_ag_btcur_init (struct xfs_scrub *sc,struct xchk_ag *sa);/* Metadata revalidators */
@@ -88,6 +104,7 @@ int xrep_agi(struct xfs_scrub *sc);int xrep_allocbt (struct xfs_scrub *sc);int xrep_iallocbt (struct xfs_scrub *sc);int xrep_refcountbt (struct xfs_scrub *sc); +int xrep_inode (struct xfs_scrub *sc);int xrep_reinit_pagf (struct xfs_scrub *sc);int xrep_reinit_pagi (struct xfs_scrub *sc);
@@ -133,6 +150,8 @@ xrep_setup_nothing( }#define xrep_setup_ag_allocbt xrep_setup_nothing +#define xrep_setup_inode (sc, imap) ((void)0) +#define xrep_revalidate_allocbt (NULL)#define xrep_revalidate_iallocbt (NULL)
@@ -144,6 +163,7 @@ xrep_setup_nothing(#define xrep_allocbt xrep_notsupported#define xrep_iallocbt xrep_notsupported#define xrep_refcountbt xrep_notsupported +#define xrep_inode xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -284,7 +284,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_inode, .scrub = xchk_inode, - .repair = xrep_notsupported, + .repair = xrep_inode, }, [XFS_SCRUB_TYPE_BMBTD] = { /* inode data fork */ .type = ST_INODE,
fs/xfs/scrub/trace.h +129 -0
135 lines not on this page · raw message
Re: [PATCH 3/7] xfs: repair inode records
fs/xfs/libxfs/xfs_format.h enum xfs_dinode_fmt { · line 1016
Quoted 7 lines
fs/xfs/libxfs/xfs_format.h +2
-1@@ -1012,7 +1012,8 @@ enum xfs_dinode_fmt {#define XFS_DFORK_APTR (dip) \ (XFS_DFORK_DPTR(dip) + XFS_DFORK_BOFF(dip))#define XFS_DFORK_PTR (dip,w) \ - ((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : XFS_DFORK_APTR(dip)) + ((void *)((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : \ + XFS_DFORK_APTR(dip)))
Not requiring a cast when using XFS_DFORK_PTR is a good thing, but I think this is the wrong way to do it. Instead of adding another cast here we can just change the char * cast in XFS_DFORK_DPTR to a void * one and rely on the widely used void pointer arithmetics extension in gcc (and clang). That'll also need a fixup to use a void instead of char * cast in xchk_dinode.
And in the long run many of these helpers relly should become inline functions..
fs/xfs/scrub/inode.c xchk_inode_flags2( · line 351
Quoted 3 lines
fs/xfs/scrub/inode.c +3
-0+/* no large extent counts without the filesystem feature */ +if ((flags2 &XFS_DIFLAG2_NREXT64 ) && !xfs_has_large_extent_counts (mp)) + goto bad;
This is just a missing check and not really related to repair, is it?
fs/xfs/scrub/inode_repair.c line 103
Quoted 9 lines
fs/xfs/scrub/inode_repair.c new file +9
-0+/* +* The only information that needs to be passed between inode scrub and +* repair is the location of the ondisk metadata if iget fails. The +* rest of struct xrep_inode is context data that we need to massage +* the ondisk inode to the point that iget will work, which means that +* we don't allocate anything at all if the incore inode is loaded. +*/ +if (!imap) +return 0 ;
I don't really understand why this comment is here, and how it relates to the imap NULL check. But as the only caller passes the address of an on-stack imap I also don't understand why the check is here to start with.
fs/xfs/scrub/inode_repair.c line 142
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6
-0+for (i =0 ; i < ni; i++) { + ioff = i << mp->m_sb.sb_inodelog; + dip =xfs_buf_offset (bp, ioff); + agino =be32_to_cpu (dip->di_next_unlinked); + + unlinked_ok = magic_ok = crc_ok = false;
I'd split the body of this loop into a separate helper and keep a lot of the variables local to it.
fs/xfs/scrub/inode_repair.c line 181
Quoted 11 lines
fs/xfs/scrub/inode_repair.c new file +11
-0+/* Reinitialize things that never change in an inode. */ +STATIC void +xrep_dinode_header ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{ +trace_xrep_dinode_header (sc, dip); + + dip->di_magic =cpu_to_be16 (XFS_DINODE_MAGIC ); +if (!xfs_dinode_good_version (sc->mp, dip->di_version)) + dip->di_version =3 ;
Can we ever end up here for v4 file systems? Because in that case the sane default inode version would be 2.
fs/xfs/scrub/inode_repair.c line 198
Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13
-0+ +/* Turn di_mode into /something/ recognizable. */ +STATIC void +xrep_dinode_mode ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{ +uint16_t mode; + +trace_xrep_dinode_mode (sc, dip); + + mode =be16_to_cpu (dip->di_mode); +if (mode ==0 ||xfs_mode_to_ftype (mode) !=XFS_DIR3_FT_UNKNOWN )
This is a somewhat odd way to check for a valid mode, but it works, so..
fs/xfs/scrub/inode_repair.c line 226
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+if (xfs_has_reflink (mp) &&S_ISREG (mode)) + flags2 |=XFS_DIFLAG2_REFLINK ;
We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.
fs/xfs/scrub/inode_repair.c line 259
Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13
-0+STATIC void +xrep_dinode_zap_symlink ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{ +char *p; + +trace_xrep_dinode_zap_symlink (sc, dip); + + dip->di_format =XFS_DINODE_FMT_LOCAL ; + dip->di_size =cpu_to_be64 (1 ); + p =XFS_DFORK_PTR (dip,XFS_DATA_FORK ); + *p ='.' ;
Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.
fs/xfs/scrub/inode_repair.c line 267
Quoted 8 lines
fs/xfs/scrub/inode_repair.c new file +8
-0+} + +/* +* Blow out dir, make it point to the root. In the future repair will +* reconstruct this directory for us. Note that there's no in-core directory +* inode because the sf verifier tripped, so we don't have to worry about the +* dentry cache. +*/
"make it point to root" isn't what I read in the code below. I parents it in root I think.
fs/xfs/scrub/inode_repair.c line 300
Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13
-0+/* Make sure we don't have a garbage file size. */ +STATIC void +xrep_dinode_size ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{ +uint64_t size; +uint16_t mode; + +trace_xrep_dinode_size (sc, dip); + + mode =be16_to_cpu (dip->di_mode); + size =be64_to_cpu (dip->di_size);
Any reason to not simplify initialize the variables at declaration time? (Same for a while bunch of other functions / variables)
fs/xfs/scrub/inode_repair.c line 490
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+if (xfs_has_reflink (sc->mp)) { + ;/* data fork blockcount can exceed physical storage */
... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?
Still, shouldn't we limit the condition to xfs_is_reflink_inode?
fs/xfs/scrub/inode_repair.c line 534
Quoted 10 lines
fs/xfs/scrub/inode_repair.c new file +10
-0+/* Check for invalid uid/gid/prid. */ +STATIC void +xrep_inode_ids ( +struct xfs_scrub *sc) +{ +bool dirty = false; + +trace_xrep_inode_ids (sc); + +if (i_uid_read (VFS_I (sc->ip)) ==-1U ) {
What is invalid about all-F uid/gid/projid?
fs/xfs/scrub/inode_repair.c line 579
Quoted 3 lines
fs/xfs/scrub/inode_repair.c new file +3
-0+ tstamp =inode_get_atime (inode); +xrep_clamp_timestamp (ip, &tstamp); +inode_set_atime_to_ts (inode, tstamp);
Meh, I hate these new VFS timestamp access helper..
fs/xfs/scrub/inode_repair.c line 669
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+/* Find the last block before 32G; this is the dir size. */ + error =xfs_iread_extents (sc->tp, sc->ip,XFS_DATA_FORK );
I think that comments needs to go down to the off asignment and xfs_iext_lookup_extent_before call.
Quoted 7 lines
fs/xfs/scrub/inode_repair.c new file +7
-0+/* +* Fix any irregularities in an inode's size now that we can iterate extent +* maps and access other regular inode data. +*/ +STATIC void +xrep_inode_size ( +struct xfs_scrub *sc)
fs/xfs/scrub/inode_repair.c line 714
Quoted 10 lines
fs/xfs/scrub/inode_repair.c new file +10
-0+{ +trace_xrep_inode_size (sc); + +/* +* Currently we only support fixing size on extents or btree format +* directories. Files can be any size and sizes for the other inode +* special types are fixed by xrep_dinode_size. +*/ +if (!S_ISDIR (VFS_I (sc->ip)->i_mode)) +return ;
I think moving this check to the caller and renaming the function would be a bit nicer, especially if we grow more file type specific checks in the future.
Otherwise this looks reasonable to me.
Re: [PATCH 3/7] xfs: repair inode records
Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
fs/xfs/libxfs/xfs_format.h +0
-1@@ -1012,7 +1012,8 @@ enum xfs_dinode_fmt {#define XFS_DFORK_APTR (dip) \ (XFS_DFORK_DPTR(dip) + XFS_DFORK_BOFF(dip))#define XFS_DFORK_PTR (dip,w) \ - ((w) == XFS_DATA_FORK ? XFS_DFORK_DPTR(dip) : XFS_DFORK_APTR(dip))
Quoted 8 lines
fs/xfs/libxfs/xfs_format.h enum xfs_dinode_fmt { · line 1016
Quoted 2 lines
fs/xfs/libxfs/xfs_format.h +2
-0+ ((void *)((w) ==XFS_DATA_FORK ?XFS_DFORK_DPTR (dip) : \ +XFS_DFORK_APTR (dip)))Not requiring a cast when using
XFS_DFORK_PTRis a good thing, but I think this is the wrong way to do it. Instead of adding another cast here we can just change the char * cast inXFS_DFORK_DPTRto a void * one and rely on the widely used void pointer arithmetics extension in gcc (and clang).
Ok.
Quoted 2 lines
That'll also need a fixup to use a void instead of char * cast in
xchk_dinode.
I'll change the conditional to:
if (XFS_DFORK_BOFF(dip) >= mp->m_sb.sb_inodesize)
Quoted 8 lines
And in the long run many of these helpers relly should become inline functions..
fs/xfs/scrub/inode.c xchk_inode_flags2( · line 351
Quoted 3 lines
fs/xfs/scrub/inode.c +3
-0+/* no large extent counts without the filesystem feature */ +if ((flags2 &XFS_DIFLAG2_NREXT64 ) && !xfs_has_large_extent_counts (mp)) + goto bad;This is just a missing check and not really related to repair, is it?
Yep. I guess I'll pull that out into a separate patch.
Quoted 6 lines
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6
-0+/* +* The only information that needs to be passed between inode scrub and +* repair is the location of the ondisk metadata if iget fails. The +* rest of struct xrep_inode is context data that we need to massage +* the ondisk inode to the point that iget will work, which means that +* we don't allocate anything at all if the incore inode is loaded.
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 103
Quoted 3 lines
fs/xfs/scrub/inode_repair.c new file +3
-0+*/ +if (!imap) +return 0 ;I don't really understand why this comment is here, and how it relates to the imap NULL check. But as the only caller passes the address of an on-stack imap I also don't understand why the check is here to start with.
Hmm. I think I've been through too many iterations of this code -- at one point I remember the null check was actually useful for something. But now it's not, so it can go.
Quoted 10 lines
fs/xfs/scrub/inode_repair.c line 142
Quoted 7 lines
fs/xfs/scrub/inode_repair.c new file +6
-0+for (i =0 ; i < ni; i++) { + ioff = i << mp->m_sb.sb_inodelog; + dip =xfs_buf_offset (bp, ioff); + agino =be32_to_cpu (dip->di_next_unlinked); + + unlinked_ok = magic_ok = crc_ok = false;I'd split the body of this loop into a separate helper and keep a lot of the variables local to it.
Ok.
Quoted 6 lines
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6
-0+/* Reinitialize things that never change in an inode. */ +STATIC void +xrep_dinode_header ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 181
Quoted 5 lines
fs/xfs/scrub/inode_repair.c new file +5
-0+trace_xrep_dinode_header (sc, dip); + + dip->di_magic =cpu_to_be16 (XFS_DINODE_MAGIC ); +if (!xfs_dinode_good_version (sc->mp, dip->di_version)) + dip->di_version =3 ;Can we ever end up here for v4 file systems? Because in that case the sane default inode version would be 2.
No. xchk_validate_inputs will reject IFLAG_REPAIR on a V4 fs. Those are deprecated, there's no point in going back.
Quoted 7 lines
Quoted 7 lines
fs/xfs/scrub/inode_repair.c new file +7
-0+ +/* Turn di_mode into /something/ recognizable. */ +STATIC void +xrep_dinode_mode ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 198
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6
-0+uint16_t mode; + +trace_xrep_dinode_mode (sc, dip); + + mode =be16_to_cpu (dip->di_mode); +if (mode ==0 ||xfs_mode_to_ftype (mode) !=XFS_DIR3_FT_UNKNOWN )This is a somewhat odd way to check for a valid mode, but it works, so..
:)
Quoted 6 lines
fs/xfs/scrub/inode_repair.c line 226
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+if (xfs_has_reflink (mp) &&S_ISREG (mode)) + flags2 |=XFS_DIFLAG2_REFLINK ;We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.
/*
* For regular files on a reflink filesystem, set the REFLINK flag to
* protect shared extents. A later stage will actually check those
* extents and clear the flag if possible.
*/
Quoted 10 lines
Quoted 10 lines
fs/xfs/scrub/inode_repair.c new file +9
-0+STATIC void +xrep_dinode_zap_symlink ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{ +char *p; + +trace_xrep_dinode_zap_symlink (sc, dip); +
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 259
Quoted 4 lines
fs/xfs/scrub/inode_repair.c new file +4
-0+ dip->di_format =XFS_DINODE_FMT_LOCAL ; + dip->di_size =cpu_to_be64 (1 ); + p =XFS_DFORK_PTR (dip,XFS_DATA_FORK ); + *p ='.' ;Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.
How about '🤷'? That's only four bytes.
Or maybe a question mark.
Quoted 11 lines
fs/xfs/scrub/inode_repair.c line 267
Quoted 8 lines
fs/xfs/scrub/inode_repair.c new file +8
-0+} + +/* +* Blow out dir, make it point to the root. In the future repair will +* reconstruct this directory for us. Note that there's no in-core directory +* inode because the sf verifier tripped, so we don't have to worry about the +* dentry cache. +*/ "make it point to root" isn't what I read in the code below. I parents it in root I think.
Yes. Changed to "Blow out dir, make the parent point to the root."
Quoted 8 lines
Quoted 8 lines
fs/xfs/scrub/inode_repair.c new file +8
-0+/* Make sure we don't have a garbage file size. */ +STATIC void +xrep_dinode_size ( +struct xfs_scrub *sc, +struct xfs_dinode *dip) +{ +uint64_t size; +uint16_t mode;
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 300
Quoted 5 lines
fs/xfs/scrub/inode_repair.c new file +5
-0+ +trace_xrep_dinode_size (sc, dip); + + mode =be16_to_cpu (dip->di_mode); + size =be64_to_cpu (dip->di_size);Any reason to not simplify initialize the variables at declaration time? (Same for a while bunch of other functions / variables)
No, not really. Will fix.
Quoted 5 lines
fs/xfs/scrub/inode_repair.c line 490
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+if (xfs_has_reflink (sc->mp)) { + ;/* data fork blockcount can exceed physical storage */ ... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?
Yes. That's not a terribly functional file, but users can do such things if they want to pay for the cpu/metadata.
Quoted 1 line
Still, shouldn't we limit the condition to
xfs_is_reflink_inode?
Yep.
Quoted 4 lines
Quoted 4 lines
fs/xfs/scrub/inode_repair.c new file +4
-0+/* Check for invalid uid/gid/prid. */ +STATIC void +xrep_inode_ids ( +struct xfs_scrub *sc)
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 534
Quoted 6 lines
fs/xfs/scrub/inode_repair.c new file +6
-0+{ +bool dirty = false; + +trace_xrep_inode_ids (sc); + +if (i_uid_read (VFS_I (sc->ip)) ==-1U ) {What is invalid about all-F uid/gid/projid?
I thought those were invalid, though apparently they're not now? uidgid.h says:
static inline bool uid_valid(kuid_t uid)
{
return __kuid_val(uid) != (uid_t) -1;
}Which is why I thought that it's not possible to have a uid of -1 on a file. Trying to set that uid on a file causes the kernel to reject the value, but OTOH I can apparently create inodes with a -1 UID via idmapping shenanigans.
<shrug>
Quoted 5 lines
fs/xfs/scrub/inode_repair.c line 579
Quoted 3 lines
fs/xfs/scrub/inode_repair.c new file +3
-0+ tstamp =inode_get_atime (inode); +xrep_clamp_timestamp (ip, &tstamp); +inode_set_atime_to_ts (inode, tstamp);Meh, I hate these new VFS timestamp access helper..
They're very clunky.
Quoted 5 lines
fs/xfs/scrub/inode_repair.c line 669
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+/* Find the last block before 32G; this is the dir size. */ + error =xfs_iread_extents (sc->tp, sc->ip,XFS_DATA_FORK );I think that comments needs to go down to the off asignment and
xfs_iext_lookup_extent_beforecall.
Done.
Quoted 13 lines
Quoted 13 lines
fs/xfs/scrub/inode_repair.c new file +13
-0+/* +* Fix any irregularities in an inode's size now that we can iterate extent +* maps and access other regular inode data. +*/ +STATIC void +xrep_inode_size ( +struct xfs_scrub *sc) +{ +trace_xrep_inode_size (sc); + +/* +* Currently we only support fixing size on extents or btree format +* directories. Files can be any size and sizes for the other inode
Quoted 8 lines
fs/xfs/scrub/inode_repair.c line 714
Quoted 4 lines
fs/xfs/scrub/inode_repair.c new file +4
-0+* special types are fixed by xrep_dinode_size. +*/ +if (!S_ISDIR (VFS_I (sc->ip)->i_mode)) +return ;I think moving this check to the caller and renaming the function would be a bit nicer, especially if we grow more file type specific checks in the future.
That's the only one, so I'll rename it to xrep_inode_dir_size and hoist this check to the caller.
Quoted 1 line
Otherwise this looks reasonable to me.
Woo, thanks for reading through all this. :)
--D
Re: [PATCH 3/7] xfs: repair inode records
Quoted 10 lines · Darrick J. Wong
Quoted 4 lines
We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.
/*
* For regular files on a reflink filesystem, set the REFLINK flag to
* protect shared extents. A later stage will actually check those
* extents and clear the flag if possible.
*/
Sounds good.
Quoted 7 lines
Quoted 3 lines
Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.
How about '🤷'? That's only four bytes.
Or maybe a question mark.
Heh. I guess question marks seems a bit better, but the general idea of having new names / different pointing locations show up in the name space scare me a little. I wonder if something like the classic lost+found directory might be the right thing for anything we're not sure about as people know about it.
Quoted 8 lines
Quoted 5 lines
fs/xfs/scrub/inode_repair.c line 490
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+if (xfs_has_reflink (sc->mp)) { + ;/* data fork blockcount can exceed physical storage */ ... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?
Yes. That's not a terribly functional file, but users can do such things if they want to pay for the cpu/metadata.
Yeah. But maybe expand the comment a bit - having spent a fair amout of time with the reflink code this was obvious to me, but for someone new the above might be a bit too cryptic.
Quoted 9 lines
Quoted 4 lines
fs/xfs/scrub/inode_repair.c line 534
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+ +if (i_uid_read (VFS_I (sc->ip)) ==-1U ) {What is invalid about all-F uid/gid/projid?
I thought those were invalid, though apparently they're not now?
uidgid.hsays:static inline bool
uid_valid(kuid_t uid)
Quoted 8 lines
{ return __kuid_val(uid) != (uid_t) -1; }Which is why I thought that it's not possible to have a uid of -1 on a file. Trying to set that uid on a file causes the kernel to reject the value, but OTOH I can apparently create inodes with a -1 UID via idmapping shenanigans.
Heh. Just wanted an explanation for the check. So a commnt is fine, or finding a way to use the uid_valid and co helpers to make it self-explanatory.
Re: [PATCH 3/7] xfs: repair inode records
Quoted 20 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 4 lines
We set the reflink flag by default, because a later stage will clear it if there aren't any shared blocks, right? Maybe add a comment to avoid any future confusion.
/*
* For regular files on a reflink filesystem, set the REFLINK flag to
* protect shared extents. A later stage will actually check those
* extents and clear the flag if possible.
*/Sounds good.
Quoted 6 lines
Quoted 3 lines
Hmm, changing a symlink to actually point somewhere seems very surprising, but making it point to the current directory almost begs for userspace code to run in loops.
How about '🤷'? That's only four bytes.
Quoted 7 lines
Quoted 1 line
Or maybe a question mark.
Heh. I guess question marks seems a bit better, but the general idea of having new names / different pointing locations show up in the name space scare me a little. I wonder if something like the classic lost+found directory might be the right thing for anything we're not sure about as people know about it.
Hmm. I suppose a problem with "?" is that question-mark is a valid filename, which means that our zapped symlink could now suddenly point to a different file that a user created. "/lost+found" isn't different in that respect, but societal convention might at least provide for raised eyebrows. That said, mkfs.xfs doesn't create one for us like mke2fs does, so maybe a broken symlink to the orphanage is... well, now I'm bikeshedding my own creation.
May I try to make a case for "🚽"? ;)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
fs/xfs/scrub/inode_repair.c line 490
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+if (xfs_has_reflink (sc->mp)) { + ;/* data fork blockcount can exceed physical storage */ ... because we would be reflinking the same blocks into the same inode at different offsets over and over again ... ?
Quoted 6 lines
Quoted 2 lines
Yes. That's not a terribly functional file, but users can do such things if they want to pay for the cpu/metadata.
Yeah. But maybe expand the comment a bit - having spent a fair amout of time with the reflink code this was obvious to me, but for someone new the above might be a bit too cryptic.
/* * data fork blockcount can exceed physical storage if a * user reflinks the same block over and over again. */
Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
fs/xfs/scrub/inode_repair.c line 534
Quoted 2 lines
fs/xfs/scrub/inode_repair.c new file +2
-0+ +if (i_uid_read (VFS_I (sc->ip)) ==-1U ) {What is invalid about all-F uid/gid/projid?
I thought those were invalid, though apparently they're not now?
uidgid.hsays:static inline bool
uid_valid(kuid_t uid){ return __kuid_val(uid) != (uid_t) -1; }
Quoted 8 lines
Quoted 4 lines
Which is why I thought that it's not possible to have a uid of -1 on a file. Trying to set that uid on a file causes the kernel to reject the value, but OTOH I can apparently create inodes with a -1 UID via idmapping shenanigans.
Heh. Just wanted an explanation for the check. So a commnt is fine, or finding a way to use the
uid_validand co helpers to make it self-explanatory.
Yeah, I think this converts easily to:
if (!uid_valid(VFS_I(sc->ip)->i_uid)) {
/* zap it */
}--D
Re: [PATCH 3/7] xfs: repair inode records
Quoted 9 lines · Darrick J. Wong
Hmm. I suppose a problem with "?" is that question-mark is a valid filename, which means that our zapped symlink could now suddenly point to a different file that a user created. "/lost+found" isn't different in that respect, but societal convention might at least provide for raised eyebrows. That said, mkfs.xfs doesn't create one for us like mke2fs does, so maybe a broken symlink to the orphanage is... well, now I'm bikeshedding my own creation.
May I try to make a case for "🚽"? ;)
Haha.. I suspect not allowing to follow the link at all if is marked sick is the best idea, i.e. the concept we've talked about for regular files. Make that consistent for all file times, and then we need to look into an expedited on-disk flag for that to make it persistent.
Quoted 5 lines
/* * data fork blockcount can exceed physical storage if a * user reflinks the same block over and over again. */
Yup.
Quoted 3 lines
if (!uid_valid(VFS_I(sc->ip)->i_uid)) {
/* zap it */
}Perfect.
PATCH 4/7 xfs: zap broken inode forks
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Determine if inode fork damage is responsible for the inode being unable to pass the ifork verifiers in xfs_iget and zap the fork contents if this is true. Once this is done the fork will be empty but we'll be able to construct an in-core inode, and a subsequent call to the inode fork repair ioctl will search the rmapbt to rebuild the records that were in the fork.
--- fs/xfs/libxfs/xfs_attr_leaf.c | 32 +- fs/xfs/libxfs/xfs_attr_leaf.h | 2 fs/xfs/libxfs/xfs_bmap.c | 22 + fs/xfs/libxfs/xfs_bmap.h | 2 fs/xfs/libxfs/xfs_dir2_priv.h | 2 fs/xfs/libxfs/xfs_dir2_sf.c | 29 +- fs/xfs/libxfs/xfs_shared.h | 1 fs/xfs/libxfs/xfs_symlink_remote.c | 21 + fs/xfs/scrub/inode_repair.c | 696 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 42 ++ 10 files changed, 812 insertions(+), 37 deletions(-)
fs/xfs/libxfs/xfs_attr_leaf.c +21 -11
@@ -1040,23 +1040,16 @@ xfs_attr_shortform_allfit(return xfs_attr_shortform_bytesfit (dp, bytes); } -/* Verify the consistency of an inline attribute fork. */ +/* Verify the consistency of a raw inline attribute fork. */ xfs_failaddr_t -xfs_attr_shortform_verify ( -struct xfs_inode *ip) +xfs_attr_shortform_verify_struct ( +struct xfs_attr_shortform *sfp, +size_t size) { -struct xfs_attr_shortform *sfp;struct xfs_attr_sf_entry *sfep;struct xfs_attr_sf_entry *next_sfep;char *endp; -struct xfs_ifork *ifp;int i; -int64_t size; - -ASSERT (ip->i_af.if_format ==XFS_DINODE_FMT_LOCAL ); - ifp =xfs_ifork_ptr (ip,XFS_ATTR_FORK ); - sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data; - size = ifp->if_bytes;/* * Give up if the attribute is way too short.
@@ -1116,6 +1109,23 @@ xfs_attr_shortform_verify(return NULL ; } +/* Verify the consistency of an inline attribute fork. */ +xfs_failaddr_t +xfs_attr_shortform_verify ( +struct xfs_inode *ip) +{ +struct xfs_attr_shortform *sfp; +struct xfs_ifork *ifp; +int64_t size; + +ASSERT (ip->i_af.if_format ==XFS_DINODE_FMT_LOCAL ); + ifp =xfs_ifork_ptr (ip,XFS_ATTR_FORK ); + sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data; + size = ifp->if_bytes; + +return xfs_attr_shortform_verify_struct (sfp, size); +} +/* * Convert a leaf attribute list to shortform attribute list */
fs/xfs/libxfs/xfs_attr_leaf.h +2 -0
@@ -56,6 +56,8 @@ int xfs_attr_sf_findname(struct xfs_da_args *args, unsigned int *basep);int xfs_attr_shortform_allfit (struct xfs_buf *bp,struct xfs_inode *dp);int xfs_attr_shortform_bytesfit (struct xfs_inode *dp,int bytes); +xfs_failaddr_t xfs_attr_shortform_verify_struct (struct xfs_attr_shortform *sfp, +size_t size);xfs_failaddr_t xfs_attr_shortform_verify (struct xfs_inode *ip);void xfs_attr_fork_remove (struct xfs_inode *ip,struct xfs_trans *tp);
fs/xfs/libxfs/xfs_bmap.c +16 -6
@@ -6179,19 +6179,18 @@ xfs_bmap_finish_one(return error; } -/* Check that an inode's extent does not have invalid flags or bad ranges. */ +/* Check that an extent does not have invalid flags or bad ranges. */ xfs_failaddr_t -xfs_bmap_validate_extent ( -struct xfs_inode *ip, +xfs_bmap_validate_extent_raw ( +struct xfs_mount *mp, +bool rtfile,int whichfork,struct xfs_bmbt_irec *irec) { -struct xfs_mount *mp = ip->i_mount; -if (!xfs_verify_fileext (mp, irec->br_startoff, irec->br_blockcount))return __this_address; -if (XFS_IS_REALTIME_INODE (ip) && whichfork ==XFS_DATA_FORK ) { +if (rtfile && whichfork ==XFS_DATA_FORK ) {if (!xfs_verify_rtbext (mp, irec->br_startblock, irec->br_blockcount))return __this_address;
@@ -6221,3 +6220,14 @@ xfs_bmap_intent_destroy_cache(void)kmem_cache_destroy (xfs_bmap_intent_cache); xfs_bmap_intent_cache =NULL ; } + +/* Check that an inode's extent does not have invalid flags or bad ranges. */ +xfs_failaddr_t +xfs_bmap_validate_extent ( +struct xfs_inode *ip, +int whichfork, +struct xfs_bmbt_irec *irec) +{ +return xfs_bmap_validate_extent_raw (ip->i_mount, +XFS_IS_REALTIME_INODE (ip), whichfork, irec); +}
fs/xfs/libxfs/xfs_bmap.h +2 -0
@@ -263,6 +263,8 @@ static inline uint32_t xfs_bmap_fork_to_state(int whichfork) } } +xfs_failaddr_t xfs_bmap_validate_extent_raw (struct xfs_mount *mp,bool rtfile, +int whichfork,struct xfs_bmbt_irec *irec);xfs_failaddr_t xfs_bmap_validate_extent (struct xfs_inode *ip,int whichfork,struct xfs_bmbt_irec *irec); int xfs_bmap_complain_bad_rec(struct xfs_inode *ip, int whichfork,
fs/xfs/libxfs/xfs_dir2_priv.h +2 -0
@@ -175,6 +175,8 @@ extern int xfs_dir2_sf_create(struct xfs_da_args *args, xfs_ino_t pino);extern int xfs_dir2_sf_lookup (struct xfs_da_args *args);extern int xfs_dir2_sf_removename (struct xfs_da_args *args);extern int xfs_dir2_sf_replace (struct xfs_da_args *args); +extern xfs_failaddr_t xfs_dir2_sf_verify_struct (struct xfs_mount *mp, +struct xfs_dir2_sf_hdr *sfp,int64_t size);extern xfs_failaddr_t xfs_dir2_sf_verify (struct xfs_inode *ip);int xfs_dir2_sf_entsize (struct xfs_mount *mp,struct xfs_dir2_sf_hdr *hdr,int len);
fs/xfs/libxfs/xfs_dir2_sf.c +18 -11
@@ -706,12 +706,11 @@ xfs_dir2_sf_check(/* Verify the consistency of an inline directory. */ xfs_failaddr_t -xfs_dir2_sf_verify ( -struct xfs_inode *ip) +xfs_dir2_sf_verify_struct ( +struct xfs_mount *mp, +struct xfs_dir2_sf_hdr *sfp, +int64_t size) { -struct xfs_mount *mp = ip->i_mount; -struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); -struct xfs_dir2_sf_hdr *sfp;struct xfs_dir2_sf_entry *sfep;struct xfs_dir2_sf_entry *next_sfep;char *endp;
@@ -719,15 +718,9 @@ xfs_dir2_sf_verify(int i;int i8count;int offset; -int64_t size;int error;uint8_t filetype; -ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); - - sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data; - size = ifp->if_bytes; -/* * Give up if the directory is way too short. */
@@ -803,6 +796,20 @@ xfs_dir2_sf_verify(return NULL ; } +xfs_failaddr_t +xfs_dir2_sf_verify ( +struct xfs_inode *ip) +{ +struct xfs_mount *mp = ip->i_mount; +struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); +struct xfs_dir2_sf_hdr *sfp; + +ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); + + sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data; +return xfs_dir2_sf_verify_struct (mp, sfp, ifp->if_bytes); +} +/* * Create a new (shortform) directory. */
fs/xfs/libxfs/xfs_shared.h +1 -0
@@ -139,6 +139,7 @@ bool xfs_symlink_hdr_ok(xfs_ino_t ino, uint32_t offset, uint32_t size, struct xfs_buf *bp);void xfs_symlink_local_to_remote (struct xfs_trans *tp,struct xfs_buf *bp,struct xfs_inode *ip,struct xfs_ifork *ifp); +xfs_failaddr_t xfs_symlink_sf_verify_struct (void *sfp,int64_t size);xfs_failaddr_t xfs_symlink_shortform_verify (struct xfs_inode *ip);/* Computed inode geometry for the filesystem. */
fs/xfs/libxfs/xfs_symlink_remote.c +14 -7
@@ -201,16 +201,12 @@ xfs_symlink_local_to_remote(* does not do on-disk format checks. */ xfs_failaddr_t -xfs_symlink_shortform_verify ( -struct xfs_inode *ip) +xfs_symlink_sf_verify_struct ( +void *sfp, +int64_t size) { -struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); -char *sfp = (char *)ifp->if_u1.if_data; -int size = ifp->if_bytes;char *endp = sfp + size; -ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); -/* * Zero length symlinks should never occur in memory as they are * never allowed to exist on disk.
@@ -231,3 +227,14 @@ xfs_symlink_shortform_verify(return __this_address;return NULL ; } + +xfs_failaddr_t +xfs_symlink_shortform_verify ( +struct xfs_inode *ip) +{ +struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); + +ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); + +return xfs_symlink_sf_verify_struct (ifp->if_u1.if_data, ifp->if_bytes); +}
fs/xfs/scrub/inode_repair.c +694 -2
754 lines not on this page · raw message
fs/xfs/scrub/trace.h +42 -0
54 lines not on this page · raw message
Re: [PATCH 4/7] xfs: zap broken inode forks
Quoted 5 lines
fs/xfs/libxfs/xfs_attr_leaf.c +5
-0+/* Verify the consistency of an inline attribute fork. */ +xfs_failaddr_t +xfs_attr_shortform_verify ( +struct xfs_inode *ip) +{
fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr_shortform_verify( · line 1126
Quoted 10 lines
fs/xfs/libxfs/xfs_attr_leaf.c +10
-0+struct xfs_attr_shortform *sfp; +struct xfs_ifork *ifp; +int64_t size; + +ASSERT (ip->i_af.if_format ==XFS_DINODE_FMT_LOCAL ); + ifp =xfs_ifork_ptr (ip,XFS_ATTR_FORK ); + sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data; + size = ifp->if_bytes; + +return xfs_attr_shortform_verify_struct (sfp, size);
Given that xfs_attr_shortform_verify only has a single caller in the kernel and no extra n in xfsprogs I'd just change the calling convention to pass the xfs_attr_shortform structure and size there and not bother with the wrapper.
fs/xfs/libxfs/xfs_bmap.c xfs_bmap_intent_destroy_cache(void) · line 6233
Quoted 10 lines
fs/xfs/libxfs/xfs_bmap.c +10
-0+/* Check that an inode's extent does not have invalid flags or bad ranges. */ +xfs_failaddr_t +xfs_bmap_validate_extent ( +struct xfs_inode *ip, +int whichfork, +struct xfs_bmbt_irec *irec) +{ +return xfs_bmap_validate_extent_raw (ip->i_mount, +XFS_IS_REALTIME_INODE (ip), whichfork, irec); +}
.. while this one has a bunch of caller so I expect it's actually somewhat useful.
fs/xfs/libxfs/xfs_dir2_priv.h extern int xfs_dir2_sf_create(struct xfs_da_args *args, xfs_ino_t pino); · line 179
Quoted 2 lines
fs/xfs/libxfs/xfs_dir2_priv.h +2
-0+extern xfs_failaddr_t xfs_dir2_sf_verify_struct (struct xfs_mount *mp, +struct xfs_dir2_sf_hdr *sfp,int64_t size);
It would be nice if we didn't add more pointless externs to function declarations in heders.
fs/xfs/libxfs/xfs_dir2_sf.c xfs_dir2_sf_verify( · line 811
Quoted 13 lines
fs/xfs/libxfs/xfs_dir2_sf.c +13
-0+xfs_failaddr_t +xfs_dir2_sf_verify ( +struct xfs_inode *ip) +{ +struct xfs_mount *mp = ip->i_mount; +struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); +struct xfs_dir2_sf_hdr *sfp; + +ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); + + sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data; +return xfs_dir2_sf_verify_struct (mp, sfp, ifp->if_bytes); +}
This one also only has a single caller in the kernel and user space combined, so I wou;dn't bother with the wrapper.
fs/xfs/libxfs/xfs_symlink_remote.c xfs_symlink_shortform_verify( · line 240
Quoted 10 lines
fs/xfs/libxfs/xfs_symlink_remote.c +10
-0+xfs_failaddr_t +xfs_symlink_shortform_verify ( +struct xfs_inode *ip) +{ +struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); + +ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); + +return xfs_symlink_sf_verify_struct (ifp->if_u1.if_data, ifp->if_bytes); +}
Same here.
Once past thes nitpicks the zapping functionality looks fine to me, but leaves me with a very high level question:
As far as I can tell the inodes with the zapped fork(s) remains in it's normal place, and normaly accessible, and I think any read will return zeroes because i_size isn't reset. Which would change the data seen by an application using it. Don't we need to block access to it until it is fully repaired?
Re: [PATCH 4/7] xfs: zap broken inode forks
Quoted 12 lines · Christoph Hellwig
Quoted 12 lines
fs/xfs/libxfs/xfs_attr_leaf.c +12
-0+/* Verify the consistency of an inline attribute fork. */ +xfs_failaddr_t +xfs_attr_shortform_verify ( +struct xfs_inode *ip) +{ +struct xfs_attr_shortform *sfp; +struct xfs_ifork *ifp; +int64_t size; + +ASSERT (ip->i_af.if_format ==XFS_DINODE_FMT_LOCAL ); + ifp =xfs_ifork_ptr (ip,XFS_ATTR_FORK ); + sfp = (struct xfs_attr_shortform *)ifp->if_u1.if_data;
Quoted 8 lines
fs/xfs/libxfs/xfs_attr_leaf.c xfs_attr_shortform_verify( · line 1126
Quoted 3 lines
fs/xfs/libxfs/xfs_attr_leaf.c +3
-0+ size = ifp->if_bytes; + +return xfs_attr_shortform_verify_struct (sfp, size);Given that
xfs_attr_shortform_verifyonly has a single caller in the kernel and no extra n in xfsprogs I'd just change the calling convention to pass thexfs_attr_shortformstructure and size there and not bother with the wrapper.
Ok.
Quoted 5 lines
Quoted 5 lines
fs/xfs/libxfs/xfs_bmap.c +5
-0+/* Check that an inode's extent does not have invalid flags or bad ranges. */ +xfs_failaddr_t +xfs_bmap_validate_extent( + struct xfs_inode *ip, + int whichfork,
Quoted 8 lines
fs/xfs/libxfs/xfs_bmap.c xfs_bmap_intent_destroy_cache(void) · line 6233
Quoted 5 lines
fs/xfs/libxfs/xfs_bmap.c +5
-0+ struct xfs_bmbt_irec *irec) +{ +return xfs_bmap_validate_extent_raw (ip->i_mount, +XFS_IS_REALTIME_INODE (ip), whichfork, irec); +}.. while this one has a bunch of caller so I expect it's actually somewhat useful.
Yep. :)
Quoted 5 lines
fs/xfs/libxfs/xfs_dir2_priv.h extern int xfs_dir2_sf_create(struct xfs_da_args *args, xfs_ino_t pino); · line 179
Quoted 2 lines
fs/xfs/libxfs/xfs_dir2_priv.h +2
-0+extern xfs_failaddr_t xfs_dir2_sf_verify_struct (struct xfs_mount *mp, +struct xfs_dir2_sf_hdr *sfp,int64_t size);It would be nice if we didn't add more pointless externs to function declarations in heders.
I'll get rid of the extern.
Quoted 8 lines
Quoted 8 lines
fs/xfs/libxfs/xfs_dir2_sf.c +8
-0+xfs_failaddr_t +xfs_dir2_sf_verify ( +struct xfs_inode *ip) +{ +struct xfs_mount *mp = ip->i_mount; +struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); +struct xfs_dir2_sf_hdr *sfp; +
Quoted 8 lines
fs/xfs/libxfs/xfs_dir2_sf.c xfs_dir2_sf_verify( · line 811
Quoted 5 lines
fs/xfs/libxfs/xfs_dir2_sf.c +5
-0+ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); + + sfp = (struct xfs_dir2_sf_hdr *)ifp->if_u1.if_data; +return xfs_dir2_sf_verify_struct (mp, sfp, ifp->if_bytes); +}This one also only has a single caller in the kernel and user space combined, so I wou;dn't bother with the wrapper.
<nod>
Quoted 4 lines
Quoted 4 lines
fs/xfs/libxfs/xfs_symlink_remote.c +4
-0+xfs_failaddr_t +xfs_symlink_shortform_verify ( +struct xfs_inode *ip) +{
Quoted 8 lines
fs/xfs/libxfs/xfs_symlink_remote.c xfs_symlink_shortform_verify( · line 240
Quoted 6 lines
fs/xfs/libxfs/xfs_symlink_remote.c +6
-0+struct xfs_ifork *ifp =xfs_ifork_ptr (ip,XFS_DATA_FORK ); + +ASSERT (ifp->if_format ==XFS_DINODE_FMT_LOCAL ); + +return xfs_symlink_sf_verify_struct (ifp->if_u1.if_data, ifp->if_bytes); +}Same here.
Fixed.
Quoted 8 lines
Once past thes nitpicks the zapping functionality looks fine to me, but leaves me with a very high level question:
As far as I can tell the inodes with the zapped fork(s) remains in it's normal place, and normaly accessible, and I think any read will return zeroes because
i_sizeisn't reset. Which would change the data seen by an application using it. Don't we need to block access to it until it is fully repaired?
Ideally, yes, we ought to do that. It's tricky to do this, however, because i_rwsem doesn't exist until iget succeeds, and we're doing surgery on the dinode buffer to get it into good enough shape that iget will work.
Unfortunately for me, the usual locking order is i_rwsem -> tx freeze protection -> ILOCK. Lockdep will not be happy if I try to grab i_rwsem from withina transaction. Hence the current repair code commits the dinode cleaning function before it tries to iget the inode.
But. trylock exists.
Looking at that code again, the inode scrubber sets us up with the AGI buffer if it can't iget the inode. Repairs to the dinode core acquires the inode cluster buffer, which means that nobody else can be calling iget.
So I think we can grab the inode in the same transaction as the inode core repairs. Nobody else should even be able to see that inode, so it should be safe to grab i_rwsem before committing the transaction. Even if I have to use trylock in a loop to make lockdep happy.
I'll try that out and get back to you.
--D
Re: [PATCH 4/7] xfs: zap broken inode forks
Quoted 4 lines · Darrick J. Wong
So I think we can grab the inode in the same transaction as the inode core repairs. Nobody else should even be able to see that inode, so it should be safe to grab
i_rwsembefore committing the transaction. Even if I have to use trylock in a loop to make lockdep happy.
Hmm, I though more of an inode flag that makes access to the inode outside of the scrubbe return -EIO. I can also warm up to the idea of having all inodes that are broken in some way in lost+found..
Re: [PATCH 4/7] xfs: zap broken inode forks
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
So I think we can grab the inode in the same transaction as the inode core repairs. Nobody else should even be able to see that inode, so it should be safe to grab
i_rwsembefore committing the transaction. Even if I have to use trylock in a loop to make lockdep happy.Hmm, I though more of an inode flag that makes access to the inode outside of the scrubbe return
-EIO. I can also warm up to the idea of having all inodes that are broken in some way in lost+found..
Moving things around in the directory tree might be worse, since we'd now have to read the parent pointer(s) from the file to remove those directory connections and add the new ones to lost+found.
I /think/ scouring around in a zapped data fork for a directory access will return EFSCORRUPTED anyway, though that might occur at a late enough stage in the process that the fs goes down, which isn't desirable.
However, once xrep_inode massages the ondisk inode into good enough shape that iget starts working again, I could set XFS_SICK_INO_BMBTD (and XFS_SICK_INO_DIR as appropriate) after zapping the data fork so that the directory accesses would return EFSCORRUPTED instead of scouring around in the zapped fork.
Once we start persisting the sick flags, the prevention will last until scrub or someone came along to fix the inode, instead of being a purely incore flag. But, babysteps for now. I'll fix this patch to set the XFS_SICK_INO_* flags after zapping things, and the predicates to pick them up.
--D
Re: [PATCH 4/7] xfs: zap broken inode forks
Quoted 3 lines · Darrick J. Wong
Moving things around in the directory tree might be worse, since we'd now have to read the parent pointer(s) from the file to remove those directory connections and add the new ones to lost+found.
True.
Quoted 11 lines
I /think/ scouring around in a zapped data fork for a directory access will return EFSCORRUPTED anyway, though that might occur at a late enough stage in the process that the fs goes down, which isn't desirable.
However, once
xrep_inodemassages the ondisk inode into good enough shape that iget starts working again, I could setXFS_SICK_INO_BMBTD(andXFS_SICK_INO_DIRas appropriate) after zapping the data fork so that the directory accesses would return EFSCORRUPTED instead of scouring around in the zapped fork.
Quoted 5 lines
Once we start persisting the sick flags, the prevention will last until scrub or someone came along to fix the inode, instead of being a purely incore flag. But, babysteps for now. I'll fix this patch to set the XFS_SICK_INO_* flags after zapping things, and the predicates to pick them up.
Sounds good.
PATCH 5/7 xfs: abort directory parent scrub scans if we encounter a zapped directory
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
In the previous patch, we added some code to perform sufficient repairs to an ondisk inode record such that the inode cache would be willing to load the inode. If the broken inode was a shortform directory, it will reset the directory to something plausible, which is to say an empty subdirectory of the root. The telltale signs that something is seriously wrong is the broken link count.
Such directories look clean, but they shouldn't participate in a filesystem scan to find or confirm a directory parent pointer. Create a predicate that identifies such directories and abort the scrub.
Found by fuzzing xfs/1554 with multithreaded xfs_scrub enabled and u3.bmx[0].startblock = zeroes.
--- fs/xfs/scrub/common.c | 1 + fs/xfs/scrub/common.h | 2 ++ fs/xfs/scrub/dir.c | 21 +++++++++++++++++++++ fs/xfs/scrub/parent.c | 17 +++++++++++++++++ 4 files changed, 41 insertions(+)
fs/xfs/scrub/common.c +1 -0
@@ -26,6 +26,7 @@#include "xfs_trans_priv.h" #include "xfs_da_format.h" #include "xfs_da_btree.h" +#include "xfs_dir2_priv.h" #include "xfs_attr.h" #include "xfs_reflink.h" #include "xfs_ag.h"
fs/xfs/scrub/common.h +2 -0
@@ -192,6 +192,8 @@ static inline bool xchk_skip_xref(struct xfs_scrub_metadata *sm) XFS_SCRUB_OFLAG_XCORRUPT); } +bool xchk_dir_looks_zapped (struct xfs_inode *dp); +#ifdef CONFIG_XFS_ONLINE_REPAIR /* Decide if a repair is required. */ static inline bool xchk_needs_repair (const struct xfs_scrub_metadata *sm)
fs/xfs/scrub/dir.c +21 -0
@@ -788,3 +788,24 @@ xchk_directory( error =0 ;return error; } + +/* +* Decide if this directory has been zapped to satisfy the inode and ifork +* verifiers. Checking and repairing should be postponed until the directory +* is fixed. +*/ +bool +xchk_dir_looks_zapped ( +struct xfs_inode *dp) +{ +/* +* If the dinode repair found a bad data fork, it will reset the fork +* to extents format with zero records and wait for the bmapbtd +* scrubber to reconstruct the block mappings. Directories always +* contain some content, so this is a clear sign of a zapped directory. +*/ +if (dp->i_df.if_format ==XFS_DINODE_FMT_EXTENTS ) +return dp->i_df.if_nextents ==0 ; + +return false; +}
fs/xfs/scrub/parent.c +17 -0
@@ -156,6 +156,16 @@ xchk_parent_validate( goto out_rele; } +/* +* We cannot yet validate this parent pointer if the directory looks as +* though it has been zapped by the inode record repair code. +*/ +if (xchk_dir_looks_zapped (dp)) { + error = -EBUSY ; +xchk_set_incomplete (sc); + goto out_unlock; + } +/* Look for a directory entry in the parent pointing to the child. */ error =xchk_dir_walk (sc, dp, xchk_parent_actor, &spc);if (!xchk_fblock_xref_process_error (sc,XFS_DATA_FORK ,0 , &error))
@@ -217,6 +227,13 @@ xchk_parent(*/ error =xchk_parent_validate (sc, parent_ino); }while (error == -EAGAIN ); +if (error == -EBUSY ) { +/* +* We could not scan a directory, so we marked the check +* incomplete. No further error return is necessary. +*/ +return 0 ; + }return error; }
Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
In the previous patch, we added some code to perform sufficient repairs to an ondisk inode record such that the inode cache would be willing to load the inode.
This is now a few commits back. My adjust this to be less specific.
Quoted 11 lines
If the broken inode was a shortform directory, it will reset the directory to something plausible, which is to say an empty subdirectory of the root. The telltale signs that something is seriously wrong is the broken link count.
Such directories look clean, but they shouldn't participate in a filesystem scan to find or confirm a directory parent pointer. Create a predicate that identifies such directories and abort the scrub.
Found by fuzzing xfs/1554 with multithreaded
xfs_scrubenabled and u3.bmx[0].startblock = zeroes.
This kind of ties into my comment on the previous comment, but needing heuristics to find zapped inodes or inode forks just seems to be asking for trouble. I suspect we'll need proper on-disk flags to notice the corrupted / half-rebuilt state.
Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory
Quoted 18 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
In the previous patch, we added some code to perform sufficient repairs to an ondisk inode record such that the inode cache would be willing to load the inode.
This is now a few commits back. My adjust this to be less specific.
Quoted 9 lines
If the broken inode was a shortform directory, it will reset the directory to something plausible, which is to say an empty subdirectory of the root. The telltale signs that something is seriously wrong is the broken link count.
Such directories look clean, but they shouldn't participate in a filesystem scan to find or confirm a directory parent pointer. Create a predicate that identifies such directories and abort the scrub.
Quoted 7 lines
Quoted 2 lines
Found by fuzzing xfs/1554 with multithreaded
xfs_scrubenabled and u3.bmx[0].startblock = zeroes.This kind of ties into my comment on the previous comment, but needing heuristics to find zapped inodes or inode forks just seems to be asking for trouble. I suspect we'll need proper on-disk flags to notice the corrupted / half-rebuilt state.
Hmm. A single "zapped" bit would be a good way to signal to xchk_dir_looks_zapped and xchk_bmap_want_check_rmaps that a file is probably broken. Clearing that bit would be harder though -- userspace would have to call back into the kernel after checking all the metadata.
A simpler way might be to persist the entire per-inode sick state (both forks and the contents within, for three bits). That would be more to track, but each scrubber could clear its corresponding sick-state bit. A bit further on in this series is a big patchset to set the sick state every time the hot paths encounter an EFSCORRUPTED.
IO operations could check the sick state bit and fail out to userspace, which would solve the problem of keeping programs away from a partially fixed file.
The ondisk state tracking like an entire project on its own. Thoughts?
--D
Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory
Quoted 4 lines · Darrick J. Wong
Hmm. A single "zapped" bit would be a good way to signal to
xchk_dir_looks_zappedandxchk_bmap_want_check_rmapsthat a file is probably broken. Clearing that bit would be harder though -- userspace would have to call back into the kernel after checking all the metadata.
Doesn't sound too horrible to have a special scrub call just for that.
Quoted 5 lines
A simpler way might be to persist the entire per-inode sick state (both forks and the contents within, for three bits). That would be more to track, but each scrubber could clear its corresponding sick-state bit. A bit further on in this series is a big patchset to set the sick state every time the hot paths encounter an EFSCORRUPTED.
That does sound even better.
Quoted 5 lines
IO operations could check the sick state bit and fail out to userspace, which would solve the problem of keeping programs away from a partially fixed file.
The ondisk state tracking like an entire project on its own. Thoughts?
Incore for now sounds fine to me.
Re: [PATCH 5/7] xfs: abort directory parent scrub scans if we encounter a zapped directory
Quoted 16 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong
Hmm. A single "zapped" bit would be a good way to signal to
xchk_dir_looks_zappedandxchk_bmap_want_check_rmapsthat a file is probably broken. Clearing that bit would be harder though -- userspace would have to call back into the kernel after checking all the metadata.Doesn't sound too horrible to have a special scrub call just for that.
Quoted 5 lines
A simpler way might be to persist the entire per-inode sick state (both forks and the contents within, for three bits). That would be more to track, but each scrubber could clear its corresponding sick-state bit. A bit further on in this series is a big patchset to set the sick state every time the hot paths encounter an EFSCORRUPTED.
That does sound even better.
Quoted 7 lines
Quoted 5 lines
IO operations could check the sick state bit and fail out to userspace, which would solve the problem of keeping programs away from a partially fixed file.
The ondisk state tracking like an entire project on its own. Thoughts?
Incore for now sounds fine to me.
Excellent! I'll go work on that for v28.2 or v29 or whatever the next version number is.
--D
PATCH 6/7 xfs: skip the rmapbt search on an empty attr fork unless we know it was zapped
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
The attribute fork scrubber can optionally scan the reverse mapping records of the filesystem to determine if the fork is missing mappings that it should have. However, this is a very expensive operation, so we only want to do this if we suspect that the fork is missing records. For attribute forks the criteria for suspicion is that the attr fork is in EXTENTS format and has zero extents.
However, there are several ways that a file can end up in this state through regular filesystem usage. For example, an LSM can set a s_security hook but then decide not to set an ACL; or an attr set can create the attr fork but then the actual set operation fails with ENOSPC; or we can delete all the attrs on a file whose data fork is in btree format, in which case we do not delete the attr fork. We don't want to run the expensive check for any case that can be arrived at through regular operations.
However.
When online inode repair decides to zap an attribute fork, it cannot determine if it is zapping ACL information. As a precaution it removes all the discretionary access control permissions and sets the user and group ids to zero. Check these three additional conditions to decide if we want to scan the rmap records.
--- fs/xfs/scrub/bmap.c | 44 +++++++++++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 7 deletions(-)
fs/xfs/scrub/bmap.c +37 -7
@@ -662,16 +662,46 @@ xchk_bmap_want_check_rmaps(* The inode repair code zaps broken inode forks by resetting them back * to EXTENTS format and zero extent records. If we encounter a fork * in this state along with evidence that the fork isn't supposed to be -* empty, we need to scan the reverse mappings to decide if we're going -* to rebuild the fork. Data forks with nonzero file size are scanned. -* xattr forks are never empty of content, so they are always scanned. +* empty, we might want scan the reverse mappings to decide if we're +* going to rebuild the fork. */ ifp =xfs_ifork_ptr (sc->ip, info->whichfork);if (ifp->if_format ==XFS_DINODE_FMT_EXTENTS && ifp->if_nextents ==0 ) { -if (info->whichfork ==XFS_DATA_FORK && -i_size_read (VFS_I (sc->ip)) ==0 ) -return false; - +switch (info->whichfork) { +case XFS_DATA_FORK : +/* +* Data forks with zero file size are presumed not to +* have any written data blocks. Skip the scan. +*/ +if (i_size_read (VFS_I (sc->ip)) ==0 ) +return false; +break ; +case XFS_ATTR_FORK : +/* +* Files can have an attr fork in EXTENTS format with +* zero records for several reasons: +* +* a) an attr set created a fork but ran out of space +* b) attr replace deleted an old attr but failed +* during the set step +* c) the data fork was in btree format when all attrs +* were deleted, so the fork was left in place +* d) the inode repair code zapped the fork +* +* Only in case (d) do we want to scan the rmapbt to +* see if we need to rebuild the attr fork. The fork +* zap code clears all DAC permission bits and zeroes +* the uid and gid, so avoid the scan if any of those +* three conditions are not met. +*/ +if ((VFS_I (sc->ip)->i_mode &0777 ) !=0 ) +return false; +if (!uid_eq (VFS_I (sc->ip)->i_uid,GLOBAL_ROOT_UID )) +return false; +if (!gid_eq (VFS_I (sc->ip)->i_gid,GLOBAL_ROOT_GID )) +return false; +break ; + }return true; }
PATCH 7/7 xfs: repair obviously broken inode modes
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Building off the rmap scanner that we added in the previous patch, we can now find block 0 and try to use the information contained inside of it to guess the mode of an inode if it's totally improper.
--- fs/xfs/scrub/inode_repair.c | 181 +++++++++++++++++++++++++++++++++++++++++-- fs/xfs/scrub/trace.h | 11 ++- 2 files changed, 179 insertions(+), 13 deletions(-)
fs/xfs/scrub/inode_repair.c +172 -9
@@ -56,10 +56,13 @@* fix things on live incore inodes. The inode repair functions make decisions * with security and usability implications when reviving a file: * -* - Files with zero di_mode or a garbage di_mode are converted to regular file -* that only root can read. This file may not actually contain user data, -* if the file was not previously a regular file. Setuid and setgid bits -* are cleared. +* - Files with zero di_mode or a garbage di_mode are converted to a file +* that only root can read. If the immediate data fork area or block 0 of +* the data fork look like a directory, the file type will be set to a +* directory. If the immediate data fork area has no nulls, it will be +* turned into a symbolic link. Otherwise, it is turned into a regular file. +* This file may not actually contain user data, if the file was not +* previously a regular file. Setuid and setgid bits are cleared. * * - Zero-size directories can be truncated to look empty. It is necessary to * run the bmapbtd and directory repair functions to fully rebuild the
@@ -107,6 +110,9 @@ struct xrep_inode {/* Blocks in use by the attr fork. */ xfs_rfsblock_t attr_blocks; +/* Physical block containing data block 0. */ +xfs_fsblock_t block0; +/* Number of data device extents for the data fork. */ xfs_extnum_t data_extents;
@@ -146,6 +152,7 @@ xrep_setup_inode( ri = sc->buf;memcpy (&ri->imap, imap,sizeof (struct xfs_imap )); ri->sc = sc; + ri->block0 =NULLFSBLOCK ;return 0 ; }
@@ -221,12 +228,159 @@ xrep_dinode_header( dip->di_gen =cpu_to_be32 (sc->sm->sm_gen); } +/* Parse enough of the directory block header to guess if this is a dir. */ +static inline bool +xrep_dinode_is_dir ( +xfs_ino_t ino, +xfs_daddr_t daddr, +struct xfs_buf *bp) +{ +struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; +struct xfs_dir2_data_free *bf; +struct xfs_mount *mp = bp->b_mount; +xfs_lsn_t lsn =be64_to_cpu (hdr3->lsn); + +/* Does the dir3 header match the filesystem? */ +if (hdr3->magic !=cpu_to_be32 (XFS_DIR3_BLOCK_MAGIC ) && + hdr3->magic !=cpu_to_be32 (XFS_DIR3_DATA_MAGIC )) +return false; + +if (be64_to_cpu (hdr3->owner) != ino) +return false; + +if (!uuid_equal (&hdr3->uuid, &mp->m_sb.sb_meta_uuid)) +return false; + +if (be64_to_cpu (hdr3->blkno) != daddr) +return false; + +/* Directory blocks are always logged and must have a valid LSN. */ +if (lsn ==NULLCOMMITLSN ) +return false; +if (!xlog_valid_lsn (mp->m_log, lsn)) +return false; + +/* +* bestfree information lives immediately after the end of the header, +* so we won't run off the end of the buffer. +*/ + bf =xfs_dir2_data_bestfree_p (mp, bp->b_addr); +if (!bf[0 ].length && bf[0 ].offset) +return false; +if (!bf[1 ].length && bf[1 ].offset) +return false; +if (!bf[2 ].length && bf[2 ].offset) +return false; + +if (be16_to_cpu (bf[0 ].length) <be16_to_cpu (bf[1 ].length)) +return false; +if (be16_to_cpu (bf[1 ].length) <be16_to_cpu (bf[2 ].length)) +return false; + +return true; +} + +/* Guess the mode of this file from the contents. */ +STATIC uint16_t +xrep_dinode_guess_mode ( +struct xrep_inode *ri, +struct xfs_dinode *dip) +{ +struct xfs_buf *bp; +struct xfs_mount *mp = ri->sc->mp; +xfs_daddr_t daddr; +uint64_t fsize =be64_to_cpu (dip->di_size); +unsigned int dfork_sz =XFS_DFORK_DSIZE (dip, mp); +uint16_t mode =S_IFREG ; +int error; + +switch (dip->di_format) { +case XFS_DINODE_FMT_LOCAL : +/* +* If the data fork is local format, the size of the data area +* is reasonable and is big enough to contain the entire file, +* we can guess the file type from the local data. +* +* If there are no nulls, guess this is a symbolic link. +* Otherwise, this is probably a shortform directory. +*/ +if (dfork_sz <=XFS_LITINO (mp) && dfork_sz >= fsize) { +if (!memchr (XFS_DFORK_DPTR (dip),0 , fsize)) +return S_IFLNK ; +return S_IFDIR ; + } + +/* By default, we guess regular file. */ +return S_IFREG ; +case XFS_DINODE_FMT_DEV : +/* +* If the data fork is dev format, the size of the data area is +* reasonable and large enough to store a dev_t, and the file +* size is zero, this could be a blockdev, a chardev, a fifo, +* or a socket. There is no solid way to distinguish between +* those choices, so we guess blockdev if the device number is +* nonzero and chardev if it's zero (aka whiteout). +*/ +if (dfork_sz <=XFS_LITINO (mp) && + dfork_sz >=sizeof (__be32) && fsize ==0 ) { +xfs_dev_t dev =xfs_dinode_get_rdev (dip); + +return dev !=0 ?S_IFBLK :S_IFCHR ; + } + +/* By default, we guess regular file. */ +return S_IFREG ; +case XFS_DINODE_FMT_EXTENTS : +case XFS_DINODE_FMT_BTREE : +/* There are data blocks to examine below. */ +break ; +default : +/* Everything else is considered a regular file. */ +return S_IFREG ; + } + +/* There are no zero-length directories. */ +if (fsize ==0 ) +return S_IFREG ; + +/* +* If we didn't find a written mapping for file block zero, we'll guess +* that it's a sparse regular file. +*/ +if (ri->block0 ==NULLFSBLOCK ) +return S_IFREG ; + +/* Directories can't have rt extents. */ +if (ri->rt_extents >0 ) +return S_IFREG ; + +/* +* Read the first block of the file. Since we have no idea what kind +* of file geometry (e.g. dirblock size) we might be reading into, use +* an uncached buffer so that we don't pollute the buffer cache. We +* can't do uncached mapped buffers, so the best we can do is guess +* from the directory header. +*/ + daddr =XFS_FSB_TO_DADDR (mp, ri->block0); + error =xfs_buf_read_uncached (mp->m_ddev_targp, daddr, +XFS_FSS_TO_BB (mp,1 ),0 , &bp,NULL ); +if (error) +return S_IFREG ; + +if (xrep_dinode_is_dir (ri->sc->sm->sm_ino, daddr, bp)) + mode =S_IFDIR ; + +xfs_buf_relse (bp); +return mode; +} +/* Turn di_mode into /something/ recognizable. */ STATIC voidxrep_dinode_mode ( -struct xfs_scrub *sc, +struct xrep_inode *ri,struct xfs_dinode *dip) { +struct xfs_scrub *sc = ri->sc;uint16_t mode;trace_xrep_dinode_mode (sc, dip);
@@ -236,7 +390,7 @@ xrep_dinode_mode(return ;/* bad mode, so we set it to a file that only root can read */ - mode =S_IFREG ; + mode =xrep_dinode_guess_mode (ri, dip); dip->di_mode =cpu_to_be16 (mode); dip->di_uid =0 ; dip->di_gid =0 ;
@@ -443,9 +597,17 @@ xrep_dinode_walk_rmap( } ri->data_blocks += rec->rm_blockcount; -if (!(rec->rm_flags &XFS_RMAP_BMBT_BLOCK )) +if (!(rec->rm_flags &XFS_RMAP_BMBT_BLOCK )) { ri->data_extents++; +if (rec->rm_offset ==0 && + !(rec->rm_flags &XFS_RMAP_UNWRITTEN )) { +if (ri->block0 !=NULLFSBLOCK ) +return -EFSCORRUPTED ; + ri->block0 = rec->rm_startblock; + } + } +return 0 ; }
@@ -496,7 +658,8 @@ xrep_dinode_count_rmaps(trace_xrep_dinode_count_rmaps (ri->sc, ri->data_blocks, ri->rt_blocks, ri->attr_blocks, - ri->data_extents, ri->rt_extents, ri->attr_extents); + ri->data_extents, ri->rt_extents, ri->attr_extents, + ri->block0);return 0 ; }
@@ -1090,7 +1253,7 @@ xrep_dinode_core(/* Fix everything the verifier will complain about. */ dip =xfs_buf_offset (bp, ri->imap.im_boffset);xrep_dinode_header (sc, dip); -xrep_dinode_mode (sc, dip); +xrep_dinode_mode (ri, dip);xrep_dinode_flags (sc, dip, ri->rt_extents >0 );xrep_dinode_size (sc, dip);xrep_dinode_extsize_hints (sc, dip);
fs/xfs/scrub/trace.h +7 -4
@@ -1530,9 +1530,9 @@ TRACE_EVENT(xrep_dinode_count_rmaps, TP_PROTO(struct xfs_scrub *sc, xfs_rfsblock_t data_blocks, xfs_rfsblock_t rt_blocks, xfs_rfsblock_t attr_blocks, xfs_extnum_t data_extents, xfs_extnum_t rt_extents, - xfs_aextnum_t attr_extents), + xfs_aextnum_t attr_extents, xfs_fsblock_t block0),TP_ARGS (sc, data_blocks, rt_blocks, attr_blocks, data_extents, - rt_extents, attr_extents), + rt_extents, attr_extents, block0), TP_STRUCT__entry( __field(dev_t, dev) __field(xfs_ino_t, ino)
@@ -1542,6 +1542,7 @@ TRACE_EVENT(xrep_dinode_count_rmaps, __field(xfs_extnum_t, data_extents) __field(xfs_extnum_t, rt_extents) __field(xfs_aextnum_t, attr_extents) +__field (xfs_fsblock_t, block0) ),TP_fast_assign ( __entry->dev = sc->mp->m_super->s_dev;
@@ -1552,8 +1553,9 @@ TRACE_EVENT(xrep_dinode_count_rmaps, __entry->data_extents = data_extents; __entry->rt_extents = rt_extents; __entry->attr_extents = attr_extents; + __entry->block0 = block0; ), - TP_printk("dev %d:%d ino 0x%llx dblocks 0x%llx rtblocks 0x%llx ablocks 0x%llx dextents %llu rtextents %llu aextents %u", + TP_printk("dev %d:%d ino 0x%llx dblocks 0x%llx rtblocks 0x%llx ablocks 0x%llx dextents %llu rtextents %llu aextents %u startblock0 0x%llx", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->data_blocks,
@@ -1561,7 +1563,8 @@ TRACE_EVENT(xrep_dinode_count_rmaps, __entry->attr_blocks, __entry->data_extents, __entry->rt_extents, - __entry->attr_extents) + __entry->attr_extents, + __entry->block0) );#endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */
Re: [PATCH 7/7] xfs: repair obviously broken inode modes
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Building off the rmap scanner that we added in the previous patch, we can now find block 0 and try to use the information contained inside of it to guess the mode of an inode if it's totally improper.
Maybe I'm missing something important, but I don't see why a normal user couldn't construct a file that looks like an XFS directory, and that's a perfectly fine thing to do?
Re: [PATCH 7/7] xfs: repair obviously broken inode modes
Quoted 10 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Building off the rmap scanner that we added in the previous patch, we can now find block 0 and try to use the information contained inside of it to guess the mode of an inode if it's totally improper.
Maybe I'm missing something important, but I don't see why a normal user couldn't construct a file that looks like an XFS directory, and that's a perfectly fine thing to do?
They could very well do that, and it might confuse the scanner. However, I'd like to draw your attention to xrep_dinode_mode, which will set the user/group to root with 0000 access mode. That at least will keep unprivileged users from seeing the potentially weird file until the higher level repair functions can deal with it (or the sysadmin deletes it).
Hmm. That code really ought to zap the attr fork because there could be ACLs attached to the file. Let me go do that.
--D
Re: [PATCH 7/7] xfs: repair obviously broken inode modes
Quoted 10 lines · Darrick J. Wong
Quoted 3 lines
Maybe I'm missing something important, but I don't see why a normal user couldn't construct a file that looks like an XFS directory, and that's a perfectly fine thing to do?
They could very well do that, and it might confuse the scanner. However, I'd like to draw your attention to
xrep_dinode_mode, which will set the user/group to root with 0000 access mode. That at least will keep unprivileged users from seeing the potentially weird file until the higher level repair functions can deal with it (or the sysadmin deletes it).
Having a perfectly valid (but weird) file cause repair action just seems like a really bad idea.
cover xfs: online repair of file fork mappings
In reply to Darrick J. WongHi all,
In this series, online repair gains the ability to rebuild data and attr fork mappings from the reverse mapping information. It is at this point where we reintroduce the ability to reap file extents.
Repair of CoW forks is a little different -- on disk, CoW staging extents are owned by the refcount btree and cannot be mapped back to individual files. Hence we can only detect staging extents that don't quite look right (missing reverse mappings, shared staging extents) and replace them with fresh allocations.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-file-mappings
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-file-mappings
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-file-mappings
--- fs/xfs/Makefile | 2 fs/xfs/libxfs/xfs_bmap_btree.c | 119 ++++- fs/xfs/libxfs/xfs_bmap_btree.h | 5 fs/xfs/libxfs/xfs_btree_staging.c | 11 fs/xfs/libxfs/xfs_btree_staging.h | 2 fs/xfs/libxfs/xfs_iext_tree.c | 23 + fs/xfs/libxfs/xfs_inode_fork.c | 1 fs/xfs/libxfs/xfs_inode_fork.h | 3 fs/xfs/libxfs/xfs_refcount.c | 41 ++ fs/xfs/libxfs/xfs_refcount.h | 10 fs/xfs/scrub/bitmap.h | 56 ++ fs/xfs/scrub/bmap.c | 18 + fs/xfs/scrub/bmap_repair.c | 846 +++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 6 fs/xfs/scrub/cow_repair.c | 612 +++++++++++++++++++++++++++ fs/xfs/scrub/reap.c | 152 ++++++- fs/xfs/scrub/reap.h | 2 fs/xfs/scrub/repair.c | 50 ++ fs/xfs/scrub/repair.h | 11 fs/xfs/scrub/scrub.c | 20 - fs/xfs/scrub/trace.h | 118 +++++ fs/xfs/xfs_trans.c | 95 ++++ fs/xfs/xfs_trans.h | 4 23 files changed, 2155 insertions(+), 52 deletions(-) create mode 100644 fs/xfs/scrub/bmap_repair.c create mode 100644 fs/xfs/scrub/cow_repair.c
PATCH 1/5 xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents
From: Darrick J. Wong <[email protected]>
Reintroduce to xrep_reap_extents the ability to reap extents from any AG. We dropped this before because it was buggy, but in the next patch we will gain the ability to reap old bmap btrees, which can have blocks in any AG. To do this, we require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent.
--- fs/xfs/scrub/bitmap.h | 28 +++++++++++ fs/xfs/scrub/reap.c | 120 +++++++++++++++++++++++++++++++++++++++++++++++-- fs/xfs/scrub/reap.h | 2 + fs/xfs/scrub/repair.h | 1 4 files changed, 147 insertions(+), 4 deletions(-)
fs/xfs/scrub/bitmap.h +28 -0
@@ -121,4 +121,32 @@ int xagb_bitmap_set_btblocks(struct xagb_bitmap *bitmap,int xagb_bitmap_set_btcur_path (struct xagb_bitmap *bitmap,struct xfs_btree_cur *cur); +/* Bitmaps, but for type-checked for xfs_fsblock_t */ + +struct xfsb_bitmap { +struct xbitmap fsbitmap; +}; + +static inline void xfsb_bitmap_init (struct xfsb_bitmap *bitmap) +{ +xbitmap_init (&bitmap->fsbitmap); +} + +static inline void xfsb_bitmap_destroy (struct xfsb_bitmap *bitmap) +{ +xbitmap_destroy (&bitmap->fsbitmap); +} + +static inline int xfsb_bitmap_set (struct xfsb_bitmap *bitmap, +xfs_fsblock_t start,xfs_filblks_t len) +{ +return xbitmap_set (&bitmap->fsbitmap, start, len); +} + +static inline int xfsb_bitmap_walk (struct xfsb_bitmap *bitmap, +xbitmap_walk_fn fn,void *priv) +{ +return xbitmap_walk (&bitmap->fsbitmap, fn, priv); +} +#endif /* __XFS_SCRUB_BITMAP_H__ */
fs/xfs/scrub/reap.c +116 -4
129 lines not on this page · raw message
fs/xfs/scrub/reap.h +2 -0
@@ -8,5 +8,7 @@int xrep_reap_agblocks (struct xfs_scrub *sc,struct xagb_bitmap *bitmap,const struct xfs_owner_info *oinfo,enum xfs_ag_resv_type type); +int xrep_reap_fsblocks (struct xfs_scrub *sc,struct xfsb_bitmap *bitmap, +const struct xfs_owner_info *oinfo);#endif /* __XFS_SCRUB_REAP_H__ */
fs/xfs/scrub/repair.h +1 -0
@@ -48,6 +48,7 @@ xrep_trans_commit(struct xbitmap ;struct xagb_bitmap ; +struct xfsb_bitmap ;int xrep_fix_freelist (struct xfs_scrub *sc,bool can_shrink);
Re: [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Reintroduce to
xrep_reap_extentsthe ability to reap extents from any AG. We dropped this before because it was buggy, but in the next patch we will gain the ability to reap old bmap btrees, which can have blocks in any AG. To do this, we require thatsc->sais uninitialized, so that we can use it to hold all the per-AG context for a given extent.
Can you expand a bit on why it was buggy, in what commit is was dropped and what we're doing better this time around?
fs/xfs/scrub/repair.h xrep_trans_commit( · line 51
Quoted 11 lines
#endif /*
__XFS_SCRUB_REAP_H__*/fs/xfs/scrub/repair.h +1
-0@@ -48,6 +48,7 @@ xrep_trans_commit(struct xbitmap ;struct xagb_bitmap ; +struct xfsb_bitmap ;
Your might need the forward declaration in reap.h, but definitively not here :)
Otherwise looks good:
Re: [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents
Quoted 11 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Reintroduce to
xrep_reap_extentsthe ability to reap extents from any AG. We dropped this before because it was buggy, but in the next patch we will gain the ability to reap old bmap btrees, which can have blocks in any AG. To do this, we require thatsc->sais uninitialized, so that we can use it to hold all the per-AG context for a given extent.Can you expand a bit on why it was buggy, in what commit is was dropped and what we're doing better this time around?
Oh! We merged that one! Let me change the commit message:
"Back in commit a55e07308831b ("xfs: only allow reaping of per-AG blocks in xrep_reap_extents"), we removed from the reaping code the ability to handle bmbt blocks. At the time, the reaping code only walked single blocks, didn't correctly detect crosslinked blocks, and the special casing made the function hard to understand. It was easier to remove unneeded functionality prior to fixing all the bugs.
"Now that we've fixed the problems, we want again the ability to reap file metadata blocks. Reintroduce the per-file reaping functionality atop the current implementation. We require that sc->sa is uninitialized, so that we can use it to hold all the per-AG context for a given extent."
Quoted 16 lines
fs/xfs/scrub/repair.h xrep_trans_commit( · line 51
Quoted 12 lines
#endif /*
__XFS_SCRUB_REAP_H__*/fs/xfs/scrub/repair.h +1
-0@@ -48,6 +48,7 @@ xrep_trans_commit(struct xbitmap ;struct xagb_bitmap ; +struct xfsb_bitmap ;Your might need the forward declaration in
reap.h, but definitively not here :)
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
Re: [PATCH 1/5] xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents
Quoted 12 lines · Darrick J. Wong
Quoted 2 lines
Can you expand a bit on why it was buggy, in what commit is was dropped and what we're doing better this time around?
Oh! We merged that one! Let me change the commit message:
"Back in commit
a55e07308831b("xfs: only allow reaping of per-AG blocks inxrep_reap_extents"), we removed from the reaping code the ability to handle bmbt blocks. At the time, the reaping code only walked single blocks, didn't correctly detect crosslinked blocks, and the special casing made the function hard to understand. It was easier to remove unneeded functionality prior to fixing all the bugs.
Quoted 5 lines
"Now that we've fixed the problems, we want again the ability to reap file metadata blocks. Reintroduce the per-file reaping functionality atop the current implementation. We require that
sc->sais uninitialized, so that we can use it to hold all the per-AG context for a given extent."
That looks much better, thanks!
PATCH 2/5 xfs: repair inode fork block mapping data structures
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Use the reverse-mapping btree information to rebuild an inode block map. Update the btree bulk loading code as necessary to support inode rooted btrees and fix some bitrot problems.
--- fs/xfs/Makefile | 1 fs/xfs/libxfs/xfs_bmap_btree.c | 119 ++++- fs/xfs/libxfs/xfs_bmap_btree.h | 5 fs/xfs/libxfs/xfs_btree_staging.c | 11 fs/xfs/libxfs/xfs_btree_staging.h | 2 fs/xfs/libxfs/xfs_iext_tree.c | 23 + fs/xfs/libxfs/xfs_inode_fork.c | 1 fs/xfs/libxfs/xfs_inode_fork.h | 3 fs/xfs/scrub/bmap.c | 18 + fs/xfs/scrub/bmap_repair.c | 846 +++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 6 fs/xfs/scrub/repair.c | 28 + fs/xfs/scrub/repair.h | 6 fs/xfs/scrub/scrub.c | 4 fs/xfs/scrub/trace.h | 34 + fs/xfs/xfs_trans.c | 95 ++++ fs/xfs/xfs_trans.h | 4 17 files changed, 1172 insertions(+), 34 deletions(-) create mode 100644 fs/xfs/scrub/bmap_repair.c
fs/xfs/Makefile +1 -0
@@ -182,6 +182,7 @@ ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y) xfs-y += $(addprefix scrub/, \ agheader_repair.o \ alloc_repair.o \ + bmap_repair.o \ ialloc_repair.o \ inode_repair.o \ newbt.o \
fs/xfs/libxfs/xfs_bmap_btree.c +99 -20
164 lines not on this page · raw message
fs/xfs/libxfs/xfs_bmap_btree.h +5 -0
@@ -11,6 +11,7 @@ struct xfs_btree_block;struct xfs_mount ;struct xfs_inode ;struct xfs_trans ; +struct xbtree_ifakeroot ;/* * Btree block header size depends on a superblock flag.
@@ -106,6 +107,10 @@ extern int xfs_bmbt_change_owner(struct xfs_trans *tp, struct xfs_inode *ip,extern struct xfs_btree_cur *xfs_bmbt_init_cursor (struct xfs_mount *,struct xfs_trans *,struct xfs_inode *,int ); +struct xfs_btree_cur *xfs_bmbt_stage_cursor (struct xfs_mount *mp, +struct xfs_inode *ip,struct xbtree_ifakeroot *ifake); +void xfs_bmbt_commit_staged_btree (struct xfs_btree_cur *cur, +struct xfs_trans *tp,int whichfork);extern unsigned long long xfs_bmbt_calc_size (struct xfs_mount *mp,unsigned long long len);
fs/xfs/libxfs/xfs_btree_staging.c +9 -2
@@ -399,7 +399,7 @@ xfs_btree_bload_prep_block(ASSERT (*bpp ==NULL );/* Allocate a new incore btree root block. */ - new_size = bbl->iroot_size (cur, nr_this_block, priv); + new_size = bbl->iroot_size (cur, level, nr_this_block, priv); ifp->if_broot =kmem_zalloc (new_size,0 ); ifp->if_broot_bytes = (int )new_size;
@@ -585,7 +585,14 @@ xfs_btree_bload_level_geometry(unsigned int desired_npb;unsigned int maxnr; - maxnr = cur->bc_ops->get_maxrecs (cur, level); +/* +* Compute the absolute maximum number of records that we can store in +* the ondisk block or inode root. +*/ +if (cur->bc_ops->get_dmaxrecs) + maxnr = cur->bc_ops->get_dmaxrecs (cur, level); +else + maxnr = cur->bc_ops->get_maxrecs (cur, level);/* * Compute the number of blocks we need to fill each block with the
fs/xfs/libxfs/xfs_btree_staging.h +1 -1
@@ -56,7 +56,7 @@ typedef int (*xfs_btree_bload_get_records_fn)(struct xfs_btree_cur *cur,typedef int (*xfs_btree_bload_claim_block_fn )(struct xfs_btree_cur *cur,union xfs_btree_ptr *ptr,void *priv);typedef size_t (*xfs_btree_bload_iroot_size_fn )(struct xfs_btree_cur *cur, -unsigned int nr_this_level,void *priv); +unsigned int level,unsigned int nr_this_level,void *priv);struct xfs_btree_bload {/*
fs/xfs/libxfs/xfs_iext_tree.c +16 -7
@@ -622,13 +622,11 @@ static inline void xfs_iext_inc_seq(struct xfs_ifork *ifp) }void -xfs_iext_insert ( -struct xfs_inode *ip, +xfs_iext_insert_raw ( +struct xfs_ifork *ifp,struct xfs_iext_cursor *cur, -struct xfs_bmbt_irec *irec, -int state) +struct xfs_bmbt_irec *irec) { -struct xfs_ifork *ifp =xfs_iext_state_to_fork (ip, state);xfs_fileoff_t offset = irec->br_startoff;struct xfs_iext_leaf *new =NULL ;int nr_entries, i;
@@ -662,12 +660,23 @@ xfs_iext_insert(xfs_iext_set (cur_rec (cur), irec); ifp->if_bytes +=sizeof (struct xfs_iext_rec ); -trace_xfs_iext_insert (ip, cur, state, _RET_IP_); -if (new)xfs_iext_insert_node (ifp,xfs_iext_leaf_key (new,0 ), new,2 ); } +void +xfs_iext_insert ( +struct xfs_inode *ip, +struct xfs_iext_cursor *cur, +struct xfs_bmbt_irec *irec, +int state) +{ +struct xfs_ifork *ifp =xfs_iext_state_to_fork (ip, state); + +xfs_iext_insert_raw (ifp, cur, irec); +trace_xfs_iext_insert (ip, cur, state, _RET_IP_); +} +static struct xfs_iext_node * xfs_iext_rebalance_node( struct xfs_iext_node *parent,
fs/xfs/libxfs/xfs_inode_fork.c +1 -0
@@ -520,6 +520,7 @@ xfs_idata_realloc( ifp->if_bytes = new_size; } +/* Free all memory and reset a fork back to its initial state. */ void xfs_idestroy_fork (struct xfs_ifork *ifp)
fs/xfs/libxfs/xfs_inode_fork.h +3 -0
@@ -180,6 +180,9 @@ void xfs_init_local_fork(struct xfs_inode *ip, int whichfork, const void *data, int64_t size);xfs_extnum_t xfs_iext_count (struct xfs_ifork *ifp); +void xfs_iext_insert_raw (struct xfs_ifork *ifp, +struct xfs_iext_cursor *cur, +struct xfs_bmbt_irec *irec);void xfs_iext_insert (struct xfs_inode *,struct xfs_iext_cursor *cur,struct xfs_bmbt_irec *,int ); void xfs_iext_remove(struct xfs_inode *, struct xfs_iext_cursor *,
fs/xfs/scrub/bmap.c +18 -0
@@ -48,9 +48,18 @@ xchk_setup_inode_bmap(if (S_ISREG (VFS_I (sc->ip)->i_mode) && sc->sm->sm_type !=XFS_SCRUB_TYPE_BMBTA ) {struct address_space *mapping =VFS_I (sc->ip)->i_mapping; +bool is_repair =xchk_could_repair (sc);xchk_ilock (sc,XFS_MMAPLOCK_EXCL ); +/* Break all our leases, we're going to mess with things. */ +if (is_repair) { + error =xfs_break_layouts (VFS_I (sc->ip), + &sc->ilock_flags,BREAK_WRITE ); +if (error) + goto out; + } +inode_dio_wait (VFS_I (sc->ip));/*
@@ -71,6 +80,15 @@ xchk_setup_inode_bmap( error =filemap_fdatawait_keep_errors (mapping);if (error && (error != -ENOSPC && error != -EIO )) goto out; + +/* Drop the page cache if we're repairing block mappings. */ +if (is_repair) { + error =invalidate_inode_pages2 ( +VFS_I (sc->ip)->i_mapping); +if (error) + goto out; + } + }/* Got the inode, lock it and we're ready to go. */
fs/xfs/scrub/bmap_repair.c new file +846 -0
846 lines not on this page · raw message
fs/xfs/scrub/common.h +5 -1
@@ -233,7 +233,11 @@ int xchk_metadata_inode_forks(struct xfs_scrub *sc); (sc)->mp->m_super->s_id, \ (sc)->sa.pag ? (sc)->sa.pag->pag_agno : (sc)->sm->sm_agno, \ ##__VA_ARGS__) - +#define xchk_xfile_ino_descr (sc, fmt, ...) \ + kasprintf(XCHK_GFP_FLAGS, "XFS (%s): inode 0x%llx " fmt, \ + (sc)->mp->m_super->s_id, \ + (sc)->ip ? (sc)->ip->i_ino : (sc)->sm->sm_ino, \ + ##__VA_ARGS__)/* * Setting up a hook to wait for intents to drain is costly -- we have to take
fs/xfs/scrub/repair.c +28 -0
@@ -880,6 +880,34 @@ xrep_reinit_pagi(return 0 ; } +/* +* Given an active reference to a perag structure, load AG headers and cursors. +* This should only be called to scan an AG while repairing file-based metadata. +*/ +int +xrep_ag_init ( +struct xfs_scrub *sc, +struct xfs_perag *pag, +struct xchk_ag *sa) +{ +int error; + +ASSERT (!sa->pag); + + error =xfs_ialloc_read_agi (pag, sc->tp, &sa->agi_bp); +if (error) +return error; + + error =xfs_alloc_read_agf (pag, sc->tp,0 , &sa->agf_bp); +if (error) +return error; + +/* Grab our own passive reference from the caller's ref. */ + sa->pag =xfs_perag_hold (pag); +xrep_ag_btcur_init (sc, sa); +return 0 ; +} +/* Reinitialize the per-AG block reservation for the AG we just fixed. */ int xrep_reset_perag_resv(
fs/xfs/scrub/repair.h +6 -0
@@ -89,6 +89,8 @@ struct xfs_imap;int xrep_setup_inode (struct xfs_scrub *sc,struct xfs_imap *imap);void xrep_ag_btcur_init (struct xfs_scrub *sc,struct xchk_ag *sa); +int xrep_ag_init (struct xfs_scrub *sc,struct xfs_perag *pag, +struct xchk_ag *sa);/* Metadata revalidators */
@@ -106,6 +108,8 @@ int xrep_allocbt(struct xfs_scrub *sc);int xrep_iallocbt (struct xfs_scrub *sc);int xrep_refcountbt (struct xfs_scrub *sc);int xrep_inode (struct xfs_scrub *sc); +int xrep_bmap_data (struct xfs_scrub *sc); +int xrep_bmap_attr (struct xfs_scrub *sc);int xrep_reinit_pagf (struct xfs_scrub *sc);int xrep_reinit_pagi (struct xfs_scrub *sc);
@@ -165,6 +169,8 @@ xrep_setup_nothing(#define xrep_iallocbt xrep_notsupported#define xrep_refcountbt xrep_notsupported#define xrep_inode xrep_notsupported +#define xrep_bmap_data xrep_notsupported +#define xrep_bmap_attr xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +2 -2
@@ -290,13 +290,13 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_inode_bmap, .scrub = xchk_bmap_data, - .repair = xrep_notsupported, + .repair = xrep_bmap_data, }, [XFS_SCRUB_TYPE_BMBTA] = { /* inode attr fork */ .type = ST_INODE, .setup = xchk_setup_inode_bmap, .scrub = xchk_bmap_attr, - .repair = xrep_notsupported, + .repair = xrep_bmap_attr, }, [XFS_SCRUB_TYPE_BMBTC] = { /* inode CoW fork */ .type = ST_INODE,
fs/xfs/scrub/trace.h +33 -1
@@ -1175,7 +1175,7 @@ DEFINE_EVENT(xrep_rmap_class, name, \ TP_ARGS(mp, agno, agbno, len, owner, offset, flags))DEFINE_REPAIR_RMAP_EVENT (xrep_ibt_walk_rmap);DEFINE_REPAIR_RMAP_EVENT (xrep_rmap_extent_fn); -DEFINE_REPAIR_RMAP_EVENT (xrep_bmap_extent_fn); +DEFINE_REPAIR_RMAP_EVENT (xrep_bmap_walk_rmap); TRACE_EVENT(xrep_abt_found, TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
@@ -1260,6 +1260,38 @@ TRACE_EVENT(xrep_refc_found, __entry->refcount) ) +TRACE_EVENT (xrep_bmap_found, + TP_PROTO(struct xfs_inode *ip, int whichfork, + struct xfs_bmbt_irec *irec), +TP_ARGS (ip, whichfork, irec), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(int, whichfork) + __field(xfs_fileoff_t, lblk) + __field(xfs_filblks_t, len) + __field(xfs_fsblock_t, pblk) +__field (int, state) + ), +TP_fast_assign ( + __entry->dev = VFS_I(ip)->i_sb->s_dev; + __entry->ino = ip->i_ino; + __entry->whichfork = whichfork; + __entry->lblk = irec->br_startoff; + __entry->len = irec->br_blockcount; + __entry->pblk = irec->br_startblock; + __entry->state = irec->br_state; + ), +TP_printk ("dev %d:%d ino 0x%llx whichfork %s fileoff 0x%llx fsbcount 0x%llx startblock 0x%llx state %d" , +MAJOR (__entry->dev),MINOR (__entry->dev), + __entry->ino, +__print_symbolic (__entry->whichfork,XFS_WHICHFORK_STRINGS ), + __entry->lblk, + __entry->len, + __entry->pblk, + __entry->state) +); + TRACE_EVENT(xrep_findroot_block, TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, xfs_agblock_t agbno, uint32_t magic, uint16_t level),
fs/xfs/xfs_trans.c +95 -0
107 lines not on this page · raw message
fs/xfs/xfs_trans.h +4 -0
@@ -168,6 +168,8 @@ typedef struct xfs_trans {int xfs_trans_alloc (struct xfs_mount *mp,struct xfs_trans_res *resp,uint blocks,uint rtextents,uint flags,struct xfs_trans **tpp); +int xfs_trans_reserve_more (struct xfs_trans *tp, +unsigned int blocks,unsigned int rtextents);int xfs_trans_alloc_empty (struct xfs_mount *mp,struct xfs_trans **tpp);void xfs_trans_mod_sb (xfs_trans_t *,uint ,int64_t );
@@ -260,6 +262,8 @@ struct xfs_dquot;int xfs_trans_alloc_inode (struct xfs_inode *ip,struct xfs_trans_res *resv,unsigned int dblocks,unsigned int rblocks,bool force,struct xfs_trans **tpp); +int xfs_trans_reserve_more_inode (struct xfs_trans *tp,struct xfs_inode *ip, +unsigned int dblocks,unsigned int rblocks,bool force_quota); int xfs_trans_alloc_icreate(struct xfs_mount *mp, struct xfs_trans_res *resv, struct xfs_dquot *udqp, struct xfs_dquot *gdqp, struct xfs_dquot *pdqp, unsigned int dblocks,
Re: [PATCH 2/5] xfs: repair inode fork block mapping data structures
fs/xfs/scrub/bmap_repair.c line 302
Quoted 10 lines · Darrick J. Wong
fs/xfs/scrub/bmap_repair.c new file +10
-0+static int +xrep_bmap_extent_cmp ( +const void *a, +const void *b) +{ +xfs_fileoff_t ao; +xfs_fileoff_t bo; + + ao =xfs_bmbt_disk_get_startoff ((struct xfs_bmbt_rec *)a); + bo =xfs_bmbt_disk_get_startoff ((struct xfs_bmbt_rec *)b);
It would be nice if we could just have local variables for the xfs_bmbt_recs and not need casts. I guess for that xfs_bmbt_disk_get_startoff would need to take a const argument?
Probably something for later.
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (whichfork == XFS_ATTR_FORK) + return 0;
Nit: I'd probably just split the data fork specific validation into a separate helper to keep things nicely organized.
fs/xfs/scrub/bmap_repair.c line 793
Quoted 6 lines
fs/xfs/scrub/bmap_repair.c new file +6
-0+/* +* No need to waste time scanning for shared extents if the inode is +* already marked. +*/ +if (whichfork ==XFS_DATA_FORK &&xfs_is_reflink_inode (sc->ip)) + rb->shared_extents = true;
The comment doesn't seem to match the code.
Quoted 4 lines
fs/xfs/xfs_trans.c +4
-0+/* +* Try to reserve more blocks for a transaction. +* +* This is for callers that need to attach resources to a transaction, scan
fs/xfs/xfs_trans.c xfs_trans_dup( · line 148
Quoted 10 lines
fs/xfs/xfs_trans.c +10
-0+* those resources to determine the space reservation requirements, and then +* modify the attached resources. In other words, online repair. This can +* fail due to ENOSPC, so the caller must be able to cancel the transaction +* without shutting down the fs. +*/ +int +xfs_trans_reserve_more ( +struct xfs_trans *tp, +unsigned int blocks, +unsigned int rtextents)
This basically seems to duplicate xfs_trans_reserve except that it skips the log reservation. What about just allowing to pass a NULL resp agument to xfs_trans_reserve to skip the log reservation and reuse the code?
Otherwise this looks good to me.
Re: [PATCH 2/5] xfs: repair inode fork block mapping data structures
Quoted 16 lines · Christoph Hellwig
fs/xfs/scrub/bmap_repair.c line 302
Quoted 10 lines · Darrick J. Wong
fs/xfs/scrub/bmap_repair.c new file +10
-0+static int +xrep_bmap_extent_cmp ( +const void *a, +const void *b) +{ +xfs_fileoff_t ao; +xfs_fileoff_t bo; + + ao =xfs_bmbt_disk_get_startoff ((struct xfs_bmbt_rec *)a); + bo =xfs_bmbt_disk_get_startoff ((struct xfs_bmbt_rec *)b);It would be nice if we could just have local variables for the
xfs_bmbt_recsand not need casts. I guess for thatxfs_bmbt_disk_get_startoffwould need to take a const argument?
Quoted 1 line
Probably something for later.
Oh! Apparently xfs_bmbt_disk_get_startoff does take a const struct pointer now.
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (whichfork == XFS_ATTR_FORK) + return 0;Nit: I'd probably just split the data fork specific validation into a separate helper to keep things nicely organized.
Quoted 8 lines
fs/xfs/scrub/bmap_repair.c line 793
Quoted 6 lines
fs/xfs/scrub/bmap_repair.c new file +6
-0+/* +* No need to waste time scanning for shared extents if the inode is +* already marked. +*/ +if (whichfork ==XFS_DATA_FORK &&xfs_is_reflink_inode (sc->ip)) + rb->shared_extents = true;The comment doesn't seem to match the code.
Ooof, that state handling needs tightening up. There are three states, really -- "irrelevant to this repair", "set the iflag", and "no idea, do a scan". That first state is open-coded in _discover_shared, which is wasteful because that is decidable in xrep_bmap.
Quoted 12 lines
Quoted 12 lines
fs/xfs/xfs_trans.c +11
-0+/* +* Try to reserve more blocks for a transaction. +* +* This is for callers that need to attach resources to a transaction, scan +* those resources to determine the space reservation requirements, and then +* modify the attached resources. In other words, online repair. This can +* fail due to ENOSPC, so the caller must be able to cancel the transaction +* without shutting down the fs. +*/ +int +xfs_trans_reserve_more(
Quoted 8 lines
fs/xfs/xfs_trans.c xfs_trans_dup( · line 148
Quoted 3 lines
fs/xfs/xfs_trans.c +3
-0+struct xfs_trans *tp, + unsigned int blocks, + unsigned int rtextents)This basically seems to duplicate
xfs_trans_reserveexcept that it skips the log reservation. What about just allowing to pass a NULL resp agument toxfs_trans_reserveto skip the log reservation and reuse the code?
Hmm. Maybe not a NULL resp, but an empty one looks like it would work fine with less code duplication.
Quoted 1 line
Otherwise this looks good to me.
Cool! Thanks for reviewing!
--D
PATCH 3/5 xfs: refactor repair forcing tests into a repair.c helper
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
There are a couple of conditions that userspace can set to force repairs of metadata. These really belong in the repair code and not open-coded into the check code, so refactor them into a helper.
--- fs/xfs/scrub/repair.c | 22 ++++++++++++++++++++++ fs/xfs/scrub/repair.h | 2 ++ fs/xfs/scrub/scrub.c | 14 +------------- 3 files changed, 25 insertions(+), 13 deletions(-)
fs/xfs/scrub/repair.c +22 -0
@@ -27,6 +27,8 @@#include "xfs_quota.h" #include "xfs_qm.h" #include "xfs_defer.h" +#include "xfs_errortag.h" +#include "xfs_error.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/trace.h"
@@ -937,3 +939,23 @@ xrep_reset_perag_resv( out:return error; } + +/* Decide if we are going to call the repair function for a scrub type. */ +bool +xrep_will_attempt ( +struct xfs_scrub *sc) +{ +/* Userspace asked us to rebuild the structure regardless. */ +if (sc->sm->sm_flags &XFS_SCRUB_IFLAG_FORCE_REBUILD ) +return true; + +/* Let debug users force us into the repair routines. */ +if (XFS_TEST_ERROR (false, sc->mp,XFS_ERRTAG_FORCE_SCRUB_REPAIR )) +return true; + +/* Metadata is corrupt or failed cross-referencing. */ +if (xchk_needs_repair (sc->sm)) +return true; + +return false; +}
fs/xfs/scrub/repair.h +2 -0
@@ -28,6 +28,7 @@ static inline int xrep_notsupported(struct xfs_scrub *sc)/* Repair helpers */ int xrep_attempt (struct xfs_scrub *sc,struct xchk_stats_run *run); +bool xrep_will_attempt (struct xfs_scrub *sc);void xrep_failure (struct xfs_mount *mp);int xrep_roll_ag_trans (struct xfs_scrub *sc);int xrep_roll_trans (struct xfs_scrub *sc);
@@ -117,6 +118,7 @@ int xrep_reinit_pagi(struct xfs_scrub *sc);#else #define xrep_ino_dqattach (sc) (0) +#define xrep_will_attempt (sc) (false)static inline int xrep_attempt(
fs/xfs/scrub/scrub.c +1 -13
@@ -14,8 +14,6 @@#include "xfs_inode.h" #include "xfs_quota.h" #include "xfs_qm.h" -#include "xfs_errortag.h" -#include "xfs_error.h" #include "xfs_scrub.h" #include "xfs_btree.h" #include "xfs_btree_staging.h"
@@ -552,21 +550,11 @@ xfs_scrub_metadata(xchk_update_health (sc);if (xchk_could_repair (sc)) { -bool needs_fix =xchk_needs_repair (sc->sm); - -/* Userspace asked us to rebuild the structure regardless. */ -if (sc->sm->sm_flags &XFS_SCRUB_IFLAG_FORCE_REBUILD ) - needs_fix = true; - -/* Let debug users force us into the repair routines. */ -if (XFS_TEST_ERROR (needs_fix, mp,XFS_ERRTAG_FORCE_SCRUB_REPAIR )) - needs_fix = true; -/* * If userspace asked for a repair but it wasn't necessary, * report that back to userspace. */ -if (!needs_fix) { +if (!xrep_will_attempt (sc)) { sc->sm->sm_flags |=XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED ; goto out_nofix; }
Re: [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
There are a couple of conditions that userspace can set to force repairs of metadata. These really belong in the repair code and not open-coded into the check code, so refactor them into a helper.
Just ramblings from someone who is trying to get into the scrub and repair code:
I find this code organization where the check helpers are in foo.c, repair helpers in foo_repair.c and then both are used in scrub.c to fill out ops really annoying to follow. My normal taste would expect a single file that has all the methods, and which then registers the ops vector. But it's probably too late for that now..
Re: [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
There are a couple of conditions that userspace can set to force repairs of metadata. These really belong in the repair code and not open-coded into the check code, so refactor them into a helper.
Quoted 8 lines
Just ramblings from someone who is trying to get into the scrub and repair code:
I find this code organization where the check helpers are in
foo.c, repair helpers infoo_repair.cand then both are used inscrub.cto fill out ops really annoying to follow. My normal taste would expect a single file that has all the methods, and which then registers the ops vector. But it's probably too late for that now..
Not really, in theory I could respin the whole series to move FOO_repair.c into FOO.c surrounded by a giant #ifdef CONFIG_XFS_ONLINE_REPAIR block; and change agheader_repair.c.
OTOH I thought it was cleaner to elide the repair code via Makefiles instead of preprocessor directives that we could get lost in.
Longer term I've struggled with whether or not (for example) the alloc.c/alloc_repair.c declarations should go in alloc.h. That's cleaner IMHO but explodes the number of files for usually not that much gain.
--D
Re: [PATCH 3/5] xfs: refactor repair forcing tests into a repair.c helper
Quoted 10 lines · Darrick J. Wong
Quoted 5 lines
I find this code organization where the check helpers are in
foo.c, repair helpers infoo_repair.cand then both are used inscrub.cto fill out ops really annoying to follow. My normal taste would expect a single file that has all the methods, and which then registers the ops vector. But it's probably too late for that now..Not really, in theory I could respin the whole series to move
FOO_repair.cintoFOO.csurrounded by a giant #ifdefCONFIG_XFS_ONLINE_REPAIRblock; and changeagheader_repair.c.
Quoted 7 lines
OTOH I thought it was cleaner to elide the repair code via Makefiles instead of preprocessor directives that we could get lost in.
Longer term I've struggled with whether or not (for example) the
alloc.c/alloc_repair.cdeclarations should go inalloc.h. That's cleaner IMHO but explodes the number of files for usually not that much gain.
Heh, and I wondered if the check/repair code should just live with the code implenenting the functionality it is checking/repairing so things can be kept nicely static. I really do not have a good answer here, I just noticed that it requires quite a lot of cycling through files to understand the repair code.
PATCH 4/5 xfs: create a ranged query function for refcount btrees
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Implement ranged queries for refcount records. The next patch will use this to scan refcount data.
--- fs/xfs/libxfs/xfs_refcount.c | 41 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_refcount.h | 10 ++++++++++ 2 files changed, 51 insertions(+)
fs/xfs/libxfs/xfs_refcount.c +41 -0
@@ -2039,6 +2039,47 @@ xfs_refcount_has_records(return xfs_btree_has_records (cur, &low, &high,NULL , outcome); } +struct xfs_refcount_query_range_info { +xfs_refcount_query_range_fn fn; +void *priv; +}; + +/* Format btree record and pass to our callback. */ +STATIC int +xfs_refcount_query_range_helper ( +struct xfs_btree_cur *cur, +const union xfs_btree_rec *rec, +void *priv) +{ +struct xfs_refcount_query_range_info *query = priv; +struct xfs_refcount_irec irec; +xfs_failaddr_t fa; + +xfs_refcount_btrec_to_irec (rec, &irec); + fa =xfs_refcount_check_irec (cur, &irec); +if (fa) +return xfs_refcount_complain_bad_rec (cur, fa, &irec); + +return query->fn (cur, &irec, query->priv); +} + +/* Find all refcount records between two keys. */ +int +xfs_refcount_query_range ( +struct xfs_btree_cur *cur, +const struct xfs_refcount_irec *low_rec, +const struct xfs_refcount_irec *high_rec, +xfs_refcount_query_range_fn fn, +void *priv) +{ +union xfs_btree_irec low_brec = { .rc = *low_rec }; +union xfs_btree_irec high_brec = { .rc = *high_rec }; +struct xfs_refcount_query_range_info query = { .priv = priv, .fn = fn }; + +return xfs_btree_query_range (cur, &low_brec, &high_brec, + xfs_refcount_query_range_helper, &query); +} +int __init xfs_refcount_intent_init_cache (void ) {
fs/xfs/libxfs/xfs_refcount.h +10 -0
@@ -129,4 +129,14 @@ extern struct kmem_cache *xfs_refcount_intent_cache;int __init xfs_refcount_intent_init_cache (void );void xfs_refcount_intent_destroy_cache (void ); +typedef int (*xfs_refcount_query_range_fn )( +struct xfs_btree_cur *cur, +const struct xfs_refcount_irec *rec, +void *priv); + +int xfs_refcount_query_range (struct xfs_btree_cur *cur, +const struct xfs_refcount_irec *low_rec, +const struct xfs_refcount_irec *high_rec, +xfs_refcount_query_range_fn fn,void *priv); +#endif /* __XFS_REFCOUNT_H__ */
Re: [PATCH 4/5] xfs: create a ranged query function for refcount btrees
Looks good:
PATCH 5/5 xfs: repair problems in CoW forks
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Try to repair errors that we see in file CoW forks so that we don't do stupid things like remap garbage into a file. There's not a lot we can do with the COW fork -- the ondisk metadata record only that the COW staging extents are owned by the refcount btree, which effectively means that we can't reconstruct this incore structure from scratch.
Actually, this is even worse -- we can't touch written extents, because those map space that are actively under writeback, and there's not much to do with delalloc reservations. Hence we can only detect crosslinked unwritten extents and fix them by punching out the problematic parts and replacing them with delalloc extents.
--- fs/xfs/Makefile | 1 fs/xfs/scrub/bitmap.h | 28 ++ fs/xfs/scrub/cow_repair.c | 612 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/reap.c | 32 ++ fs/xfs/scrub/repair.h | 2 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/trace.h | 84 ++++++ 7 files changed, 760 insertions(+), 1 deletion(-) create mode 100644 fs/xfs/scrub/cow_repair.c
fs/xfs/Makefile +1 -0
@@ -183,6 +183,7 @@ xfs-y += $(addprefix scrub/, \ agheader_repair.o \ alloc_repair.o \ bmap_repair.o \ + cow_repair.o \ ialloc_repair.o \ inode_repair.o \ newbt.o \
fs/xfs/scrub/bitmap.h +28 -0
@@ -149,4 +149,32 @@ static inline int xfsb_bitmap_walk(struct xfsb_bitmap *bitmap, return xbitmap_walk(&bitmap->fsbitmap, fn, priv); } +/* Bitmaps, but for type-checked for xfs_fileoff_t */ + +struct xoff_bitmap { +struct xbitmap offbitmap; +}; + +static inline void xoff_bitmap_init (struct xoff_bitmap *bitmap) +{ +xbitmap_init (&bitmap->offbitmap); +} + +static inline void xoff_bitmap_destroy (struct xoff_bitmap *bitmap) +{ +xbitmap_destroy (&bitmap->offbitmap); +} + +static inline int xoff_bitmap_set (struct xoff_bitmap *bitmap, +xfs_fileoff_t off,xfs_filblks_t len) +{ +return xbitmap_set (&bitmap->offbitmap, off, len); +} + +static inline int xoff_bitmap_walk (struct xoff_bitmap *bitmap, +xbitmap_walk_fn fn,void *priv) +{ +return xbitmap_walk (&bitmap->offbitmap, fn, priv); +} +#endif /* __XFS_SCRUB_BITMAP_H__ */
fs/xfs/scrub/cow_repair.c new file +612 -0
612 lines not on this page · raw message
fs/xfs/scrub/reap.c +32 -0
@@ -20,6 +20,7 @@#include "xfs_ialloc_btree.h" #include "xfs_rmap.h" #include "xfs_rmap_btree.h" +#include "xfs_refcount.h" #include "xfs_refcount_btree.h" #include "xfs_extent_busy.h" #include "xfs_ag.h"
@@ -378,6 +379,17 @@ xreap_agextent_iter(trace_xreap_dispose_unmap_extent (sc->sa.pag, agbno, *aglenp); rs->force_roll = true; + +if (rs->oinfo == &XFS_RMAP_OINFO_COW ) { +/* +* If we're unmapping CoW staging extents, remove the +* records from the refcountbt, which will remove the +* rmap record as well. +*/ +xfs_refcount_free_cow_extent (sc->tp, fsbno, *aglenp); +return 0 ; + } +return xfs_rmap_free (sc->tp, sc->sa.agf_bp, sc->sa.pag, agbno, *aglenp, rs->oinfo); }
@@ -396,6 +408,26 @@ xreap_agextent_iter(return 0 ; } +/* +* If we're getting rid of CoW staging extents, use deferred work items +* to remove the refcountbt records (which removes the rmap records) +* and free the extent. We're not worried about the system going down +* here because log recovery walks the refcount btree to clean out the +* CoW staging extents. +*/ +if (rs->oinfo == &XFS_RMAP_OINFO_COW ) { +ASSERT (rs->resv ==XFS_AG_RESV_NONE ); + +xfs_refcount_free_cow_extent (sc->tp, fsbno, *aglenp); + error =xfs_free_extent_later (sc->tp, fsbno, *aglenp,NULL , + rs->resv, true); +if (error) +return error; + + rs->force_roll = true; +return 0 ; + } +/* Put blocks back on the AGFL one at a time. */ if (rs->resv ==XFS_AG_RESV_AGFL ) {ASSERT (*aglenp ==1 );
fs/xfs/scrub/repair.h +2 -0
@@ -111,6 +111,7 @@ int xrep_refcountbt(struct xfs_scrub *sc);int xrep_inode (struct xfs_scrub *sc);int xrep_bmap_data (struct xfs_scrub *sc);int xrep_bmap_attr (struct xfs_scrub *sc); +int xrep_bmap_cow (struct xfs_scrub *sc);int xrep_reinit_pagf (struct xfs_scrub *sc);int xrep_reinit_pagi (struct xfs_scrub *sc);
@@ -173,6 +174,7 @@ xrep_setup_nothing(#define xrep_inode xrep_notsupported#define xrep_bmap_data xrep_notsupported#define xrep_bmap_attr xrep_notsupported +#define xrep_bmap_cow xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +1 -1
@@ -300,7 +300,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_inode_bmap, .scrub = xchk_bmap_cow, - .repair = xrep_notsupported, + .repair = xrep_bmap_cow, }, [XFS_SCRUB_TYPE_DIR] = { /* directory */ .type = ST_INODE,
fs/xfs/scrub/trace.h +84 -0
90 lines not on this page · raw message
Re: [PATCH 5/5] xfs: repair problems in CoW forks
Looks good:
cover xfs: online repair of rt bitmap file
In reply to Darrick J. WongHi all,
Add in the necessary infrastructure to check the inode and data forks of metadata files, then apply that to the realtime bitmap file. We won't be able to reconstruct the contents of the rtbitmap file until rmapbt is added for realtime volumes, but we can at least get the basics started.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-rtbitmap
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-rtbitmap
--- fs/xfs/Makefile | 4 + fs/xfs/libxfs/xfs_bmap.c | 39 ++++++++ fs/xfs/libxfs/xfs_bmap.h | 2 fs/xfs/scrub/bmap_repair.c | 17 +++ fs/xfs/scrub/repair.c | 151 ++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 9 ++ fs/xfs/scrub/rtbitmap.c | 101 +++++++++++++++++--- fs/xfs/scrub/rtbitmap.h | 22 ++++ fs/xfs/scrub/rtbitmap_repair.c | 202 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/rtsummary.c | 132 +++++++++++++++++++++----- fs/xfs/scrub/scrub.c | 4 - fs/xfs/xfs_inode.c | 24 +---- 12 files changed, 637 insertions(+), 70 deletions(-) create mode 100644 fs/xfs/scrub/rtbitmap.h create mode 100644 fs/xfs/scrub/rtbitmap_repair.c
PATCH 1/6 xfs: check rt bitmap file geometry more thoroughly
From: Darrick J. Wong <[email protected]>
I forgot that the superblock tracks the number of blocks that are in the realtime bitmap, and that the rt bitmap file can have more blocks mapped to the data fork than sb_rbmblocks if growfsrt fails.
So. Add to the rtbitmap scrubber an explicit check that sb_rextents and sb_rbmblocks are correct, then adjust the rtbitmap i_size checks to allow for the growfsrt failure case. Finally, flag post-eof blocks in the rtbitmap.
--- fs/xfs/scrub/rtbitmap.c | 97 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 82 insertions(+), 15 deletions(-)
fs/xfs/scrub/rtbitmap.c +82 -15
@@ -14,16 +14,30 @@#include "xfs_rtbitmap.h" #include "xfs_inode.h" #include "xfs_bmap.h" +#include "xfs_bit.h" #include "scrub/scrub.h" #include "scrub/common.h" +struct xchk_rtbitmap { +uint64_t rextents; +uint64_t rbmblocks; +unsigned int rextslog; +}; +/* Set us up with the realtime metadata locked. */ int xchk_setup_rtbitmap (struct xfs_scrub *sc) { +struct xfs_mount *mp = sc->mp; +struct xchk_rtbitmap *rtb;int error; + rtb =kzalloc (sizeof (struct xchk_rtbitmap ),XCHK_GFP_FLAGS ); +if (!rtb) +return -ENOMEM ; + sc->buf = rtb; + error =xchk_trans_alloc (sc,0 );if (error)return error;
@@ -37,6 +51,15 @@ xchk_setup_rtbitmap(return error;xchk_ilock (sc,XFS_ILOCK_EXCL |XFS_ILOCK_RTBITMAP ); + +/* +* Now that we've locked the rtbitmap, we can't race with growfsrt +* trying to expand the bitmap or change the size of the rt volume. +* Hence it is safe to compute and check the geometry values. +*/ + rtb->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rtb->rextslog = rtb->rextents ?xfs_highbit32 (rtb->rextents) :0 ; + rtb->rbmblocks =xfs_rtbitmap_blockcount (mp, rtb->rextents);return 0 ; }
@@ -67,21 +90,30 @@ STATIC intxchk_rtbitmap_check_extents ( struct xfs_scrub *sc) { -struct xfs_mount *mp = sc->mp;struct xfs_bmbt_irec map; -xfs_rtblock_t off; -int nmap; +struct xfs_iext_cursor icur; +struct xfs_mount *mp = sc->mp; +struct xfs_inode *ip = sc->ip; +xfs_fileoff_t off =0 ; +xfs_fileoff_t endoff;int error =0 ; -for (off =0 ; off < mp->m_sb.sb_rbmblocks;) { +/* Mappings may not cross or lie beyond EOF. */ + endoff =XFS_B_TO_FSB (mp, ip->i_disk_size); +if (xfs_iext_lookup_extent (ip, &ip->i_df, endoff, &icur, &map)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , endoff); +return 0 ; + } + +while (off < endoff) { +int nmap =1 ; +if (xchk_should_terminate (sc, &error) || (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ))break ;/* Make sure we have a written extent. */ - nmap =1 ; - error =xfs_bmapi_read (mp->m_rbmip, off, - mp->m_sb.sb_rbmblocks - off, &map, &nmap, + error =xfs_bmapi_read (ip, off, endoff - off, &map, &nmap,XFS_DATA_FORK );if (!xchk_fblock_process_error (sc,XFS_DATA_FORK , off, &error))break ;
@@ -102,12 +134,48 @@ intxchk_rtbitmap ( struct xfs_scrub *sc) { +struct xfs_mount *mp = sc->mp; +struct xchk_rtbitmap *rtb = sc->buf;int error; -/* Is the size of the rtbitmap correct? */ -if (sc->mp->m_rbmip->i_disk_size != -XFS_FSB_TO_B (sc->mp, sc->mp->m_sb.sb_rbmblocks)) { -xchk_ino_set_corrupt (sc, sc->mp->m_rbmip->i_ino); +/* Is sb_rextents correct? */ +if (mp->m_sb.sb_rextents != rtb->rextents) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino); +return 0 ; + } + +/* Is sb_rextslog correct? */ +if (mp->m_sb.sb_rextslog != rtb->rextslog) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino); +return 0 ; + } + +/* +* Is sb_rbmblocks large enough to handle the current rt volume? In no +* case can we exceed 4bn bitmap blocks since the super field is a u32. +*/ +if (rtb->rbmblocks >U32_MAX ) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino); +return 0 ; + } +if (mp->m_sb.sb_rbmblocks != rtb->rbmblocks) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino); +return 0 ; + } + +/* The bitmap file length must be aligned to an fsblock. */ +if (mp->m_rbmip->i_disk_size & mp->m_blockmask) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino); +return 0 ; + } + +/* +* Is the bitmap file itself large enough to handle the rt volume? +* growfsrt expands the bitmap file before updating sb_rextents, so the +* file can be larger than sb_rbmblocks. +*/ +if (mp->m_rbmip->i_disk_size <XFS_FSB_TO_B (mp, rtb->rbmblocks)) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino);return 0 ; }
@@ -120,12 +188,11 @@ xchk_rtbitmap(if (error || (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ))return error; - error =xfs_rtalloc_query_all (sc->mp, sc->tp, xchk_rtbitmap_rec, sc); + error =xfs_rtalloc_query_all (mp, sc->tp, xchk_rtbitmap_rec, sc);if (!xchk_fblock_process_error (sc,XFS_DATA_FORK ,0 , &error)) - goto out; +return error; -out: -return error; +return 0 ; }/* xref check that the extent is not free in the rtbitmap */
Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly
fs/xfs/scrub/rtbitmap.c xchk_setup_rtbitmap( · line 62
Quoted 8 lines
fs/xfs/scrub/rtbitmap.c +8
-0+/* +* Now that we've locked the rtbitmap, we can't race with growfsrt +* trying to expand the bitmap or change the size of the rt volume. +* Hence it is safe to compute and check the geometry values. +*/ + rtb->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rtb->rextslog = rtb->rextents ?xfs_highbit32 (rtb->rextents) :0 ; + rtb->rbmblocks =xfs_rtbitmap_blockcount (mp, rtb->rextents);
All these will be 0 if mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:
if (mp->m_sb.sb_rblocks) {
rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
rtb->rextslog = xfs_highbit32(rtb->rextents);
rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents);
}?
Otherwise looks good:
Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly
Quoted 12 lines · Christoph Hellwig
fs/xfs/scrub/rtbitmap.c xchk_setup_rtbitmap( · line 62
Quoted 8 lines
fs/xfs/scrub/rtbitmap.c +8
-0+/* +* Now that we've locked the rtbitmap, we can't race with growfsrt +* trying to expand the bitmap or change the size of the rt volume. +* Hence it is safe to compute and check the geometry values. +*/ + rtb->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rtb->rextslog = rtb->rextents ?xfs_highbit32 (rtb->rextents) :0 ; + rtb->rbmblocks =xfs_rtbitmap_blockcount (mp, rtb->rextents);All these will be 0 if
mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:
Quoted 3 lines
if (mp->m_sb.sb_rblocks) {
rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
rtb->rextslog = xfs_highbit32(rtb->rextents);Well... xfs_highbit32 returns -1 if its argument is zero, which is possible for the nasty edge case of (say) a 64k block device and a realtime extent size of 1MB, which results in rblocks > 0 and rextents == 0.
So I'll still have to do:
if (rtb->rextents) rtb->rextslog = xfs_highbit32()
but otherwise this is fine.
Quoted 8 lines
rtb->rbmblocks = xfs_rtbitmap_blockcount(mp, rtb->rextents); }?
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thank you!
--D
Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly
Quoted 11 lines · Darrick J. Wong
Quoted 6 lines
All these will be 0 if
mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:if (mp->m_sb.sb_rblocks) { rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks); rtb->rextslog = xfs_highbit32(rtb->rextents);Well...
xfs_highbit32returns -1 if its argument is zero, which is possible for the nasty edge case of (say) a 64k block device and a realtime extent size of 1MB, which results in rblocks > 0 and rextents == 0.
Eww. How do we even allow creating a mounting that? Such a configuration doesn't make any sense.
Re: [PATCH 1/6] xfs: check rt bitmap file geometry more thoroughly
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines
All these will be 0 if
mp->m_sb.sb_rblocks, and rtb is zeroed allocation right above, so calculating the values seems a bit odd. Why not simply:if (mp->m_sb.sb_rblocks) { rtb->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks); rtb->rextslog = xfs_highbit32(rtb->rextents);
Quoted 7 lines
Quoted 4 lines
Well...
xfs_highbit32returns -1 if its argument is zero, which is possible for the nasty edge case of (say) a 64k block device and a realtime extent size of 1MB, which results in rblocks > 0 and rextents == 0.Eww. How do we even allow creating a mounting that? Such a configuration doesn't make any sense.
$ truncate -s 64k /tmp/realtime $ truncate -s 1g /tmp/data $ mkfs.xfs -f /tmp/data -r rtdev=/tmp/realtime,extsize=1m
Pre 4.19 mkfs.xfs would actually write out the fs and pre-4.19 kernels would mount it (and ENOSPC). Since then, due to buggy sb validation code on my part now it just fails verifiers and crashes/doesn't mount.
--D
PATCH 2/6 xfs: check rt summary file geometry more thoroughly
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
I forgot that the xfs_mount tracks the size and number of levels in the realtime summary file, and that the rt summary file can have more blocks mapped to the data fork than m_rsumsize implies if growfsrt fails.
So. Add to the rtsummary scrubber an explicit check that all the summary geometry values are correct, then adjust the rtsummary i_size checks to allow for the growfsrt failure case. Finally, flag post-eof blocks in the summary file.
While we're at it, split the extent map checking so that we only call xfs_bmapi_read once per extent instead of once per rtsummary block.
--- fs/xfs/scrub/rtsummary.c | 132 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 105 insertions(+), 27 deletions(-)
fs/xfs/scrub/rtsummary.c +105 -27
@@ -31,6 +31,18 @@* (potentially large) amount of data in pageable memory. */ +struct xchk_rtsummary { +struct xfs_rtalloc_args args; + +uint64_t rextents; +uint64_t rbmblocks; +uint64_t rsumsize; +unsigned int rsumlevels; + +/* Memory buffer for the summary comparison. */ +union xfs_suminfo_raw words[]; +}; +/* Set us up to check the rtsummary file. */ int xchk_setup_rtsummary(
@@ -38,8 +50,16 @@ xchk_setup_rtsummary( {struct xfs_mount *mp = sc->mp;char *descr; +struct xchk_rtsummary *rts; +xfs_filblks_t rsumblocks;int error; + rts =kvzalloc (struct_size (rts, words, mp->m_blockwsize), +XCHK_GFP_FLAGS ); +if (!rts) +return -ENOMEM ; + sc->buf = rts; +/* * Create an xfile to construct a new rtsummary file. The xfile allows * us to avoid pinning kernel memory for this purpose.
@@ -54,11 +74,6 @@ xchk_setup_rtsummary(if (error)return error; -/* Allocate a memory buffer for the summary comparison. */ - sc->buf =kvmalloc (mp->m_sb.sb_blocksize,XCHK_GFP_FLAGS ); -if (!sc->buf) -return -ENOMEM ; - error =xchk_install_live_inode (sc, mp->m_rsumip);if (error)return error;
@@ -75,13 +90,23 @@ xchk_setup_rtsummary(*/ xfs_ilock (mp->m_rbmip,XFS_ILOCK_SHARED |XFS_ILOCK_RTBITMAP );xchk_ilock (sc,XFS_ILOCK_EXCL |XFS_ILOCK_RTSUM ); + +/* +* Now that we've locked the rtbitmap and rtsummary, we can't race with +* growfsrt trying to expand the summary or change the size of the rt +* volume. Hence it is safe to compute and check the geometry values. +*/ + rts->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rts->rbmblocks =xfs_rtbitmap_blockcount (mp, rts->rextents); + rts->rsumlevels = rts->rextents ?xfs_highbit32 (rts->rextents) +1 :0 ; + rsumblocks =xfs_rtsummary_blockcount (mp, rts->rsumlevels, + rts->rbmblocks); + rts->rsumsize =XFS_FSB_TO_B (mp, rsumblocks);return 0 ; }/* Helper functions to record suminfo words in an xfile. */ -typedef unsigned int xchk_rtsumoff_t ; - static inline int xfsum_load( struct xfs_scrub *sc,
@@ -192,19 +217,29 @@ STATIC intxchk_rtsum_compare ( struct xfs_scrub *sc) { -struct xfs_rtalloc_args args = { - .mp = sc->mp, - .tp = sc->tp, - }; -struct xfs_mount *mp = sc->mp;struct xfs_bmbt_irec map; -xfs_fileoff_t off; -xchk_rtsumoff_t sumoff =0 ; -int nmap; +struct xfs_iext_cursor icur; -for (off =0 ; off <XFS_B_TO_FSB (mp, mp->m_rsumsize); off++) { -union xfs_suminfo_raw *ondisk_info; -int error =0 ; +struct xfs_mount *mp = sc->mp; +struct xfs_inode *ip = sc->ip; +struct xchk_rtsummary *rts = sc->buf; +xfs_fileoff_t off =0 ; +xfs_fileoff_t endoff; +xfs_rtsumoff_t sumoff =0 ; +int error =0 ; + + rts->args.mp = sc->mp; + rts->args.tp = sc->tp; + +/* Mappings may not cross or lie beyond EOF. */ + endoff =XFS_B_TO_FSB (mp, ip->i_disk_size); +if (xfs_iext_lookup_extent (ip, &ip->i_df, endoff, &icur, &map)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , endoff); +return 0 ; + } + +while (off < endoff) { +int nmap =1 ;if (xchk_should_terminate (sc, &error))return error;
@@ -212,8 +247,7 @@ xchk_rtsum_compare(return 0 ;/* Make sure we have a written extent. */ - nmap =1 ; - error =xfs_bmapi_read (mp->m_rsumip, off,1 , &map, &nmap, + error =xfs_bmapi_read (ip, off, endoff - off, &map, &nmap,XFS_DATA_FORK );if (!xchk_fblock_process_error (sc,XFS_DATA_FORK , off, &error))return error;
@@ -223,24 +257,33 @@ xchk_rtsum_compare(return 0 ; } + off += map.br_blockcount; + } + +for (off =0 ; off < endoff; off++) { +union xfs_suminfo_raw *ondisk_info; +/* Read a block's worth of ondisk rtsummary file. */ - error =xfs_rtsummary_read_buf (&args, off); + error =xfs_rtsummary_read_buf (&rts->args, off);if (!xchk_fblock_process_error (sc,XFS_DATA_FORK , off, &error))return error;/* Read a block's worth of computed rtsummary file. */ - error =xfsum_copyout (sc, sumoff, sc->buf, mp->m_blockwsize); + error =xfsum_copyout (sc, sumoff, rts->words, mp->m_blockwsize);if (error) { -xfs_rtbuf_cache_relse (&args); +xfs_rtbuf_cache_relse (&rts->args);return error; } - ondisk_info =xfs_rsumblock_infoptr (&args,0 ); -if (memcmp (ondisk_info, sc->buf, - mp->m_blockwsize <<XFS_WORDLOG ) !=0 ) + ondisk_info =xfs_rsumblock_infoptr (&rts->args,0 ); +if (memcmp (ondisk_info, rts->words, + mp->m_blockwsize <<XFS_WORDLOG ) !=0 ) {xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , off); +xfs_rtbuf_cache_relse (&rts->args); +return error; + } -xfs_rtbuf_cache_relse (&args); +xfs_rtbuf_cache_relse (&rts->args); sumoff += mp->m_blockwsize; }
@@ -253,8 +296,43 @@ xchk_rtsummary( struct xfs_scrub *sc) {struct xfs_mount *mp = sc->mp; +struct xchk_rtsummary *rts = sc->buf;int error =0 ; +/* Is sb_rextents correct? */ +if (mp->m_sb.sb_rextents != rts->rextents) { +xchk_ino_set_corrupt (sc, mp->m_rbmip->i_ino); + goto out_rbm; + } + +/* Is m_rsumlevels correct? */ +if (mp->m_rsumlevels != rts->rsumlevels) { +xchk_ino_set_corrupt (sc, mp->m_rsumip->i_ino); + goto out_rbm; + } + +/* Is m_rsumsize correct? */ +if (mp->m_rsumsize != rts->rsumsize) { +xchk_ino_set_corrupt (sc, mp->m_rsumip->i_ino); + goto out_rbm; + } + +/* The summary file length must be aligned to an fsblock. */ +if (mp->m_rsumip->i_disk_size & mp->m_blockmask) { +xchk_ino_set_corrupt (sc, mp->m_rsumip->i_ino); + goto out_rbm; + } + +/* +* Is the summary file itself large enough to handle the rt volume? +* growfsrt expands the summary file before updating sb_rextents, so +* the file can be larger than rsumsize. +*/ +if (mp->m_rsumip->i_disk_size < rts->rsumsize) { +xchk_ino_set_corrupt (sc, mp->m_rsumip->i_ino); + goto out_rbm; + } +/* Invoke the fork scrubber. */ error =xchk_metadata_inode_forks (sc);if (error || (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ))
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
fs/xfs/scrub/rtsummary.c xchk_setup_rtsummary( · line 104
Quoted 11 lines
fs/xfs/scrub/rtsummary.c +11
-0+/* +* Now that we've locked the rtbitmap and rtsummary, we can't race with +* growfsrt trying to expand the summary or change the size of the rt +* volume. Hence it is safe to compute and check the geometry values. +*/ + rts->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rts->rbmblocks =xfs_rtbitmap_blockcount (mp, rts->rextents); + rts->rsumlevels = rts->rextents ?xfs_highbit32 (rts->rextents) +1 :0 ; + rsumblocks =xfs_rtsummary_blockcount (mp, rts->rsumlevels, + rts->rbmblocks); + rts->rsumsize =XFS_FSB_TO_B (mp, rsumblocks);
Same nitpick as for the last patch.
Otherwise looks good:
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
fs/xfs/scrub/rtsummary.c +5
-0+/* +* Now that we've locked the rtbitmap and rtsummary, we can't race with +* growfsrt trying to expand the summary or change the size of the rt +* volume. Hence it is safe to compute and check the geometry values. +*/
Quoted 8 lines
fs/xfs/scrub/rtsummary.c xchk_setup_rtsummary( · line 104
Quoted 6 lines
fs/xfs/scrub/rtsummary.c +6
-0+ rts->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rts->rbmblocks =xfs_rtbitmap_blockcount (mp, rts->rextents); + rts->rsumlevels = rts->rextents ?xfs_highbit32 (rts->rextents) +1 :0 ; + rsumblocks =xfs_rtsummary_blockcount (mp, rts->rsumlevels, + rts->rbmblocks); + rts->rsumsize =XFS_FSB_TO_B (mp, rsumblocks);Same nitpick as for the last patch.
LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Thanks!
--D
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
Quoted 15 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
Quoted 5 lines
fs/xfs/scrub/rtsummary.c +5
-0+/* +* Now that we've locked the rtbitmap and rtsummary, we can't race with +* growfsrt trying to expand the summary or change the size of the rt +* volume. Hence it is safe to compute and check the geometry values. +*/ Quoted 8 lines
fs/xfs/scrub/rtsummary.c xchk_setup_rtsummary( · line 104
Quoted 6 lines
fs/xfs/scrub/rtsummary.c +6
-0+ rts->rextents =xfs_rtb_to_rtx (mp, mp->m_sb.sb_rblocks); + rts->rbmblocks =xfs_rtbitmap_blockcount (mp, rts->rextents); + rts->rsumlevels = rts->rextents ?xfs_highbit32 (rts->rextents) +1 :0 ; + rsumblocks =xfs_rtsummary_blockcount (mp, rts->rsumlevels, + rts->rbmblocks); + rts->rsumsize =XFS_FSB_TO_B (mp, rsumblocks);Same nitpick as for the last patch.
Quoted 2 lines
LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...
Oh good, XFS has been broken since the beginning of git history for the stupid corner case where the rtblocks < rextsize. In this case, mkfs will set sb_rextents and sb_rextslog both to zero:
sbp->sb_rextslog = (uint8_t)(rtextents ? libxfs_highbit32((unsigned int)rtextents) : 0);
However, that's not the check that xfs_repair uses for nonzero rtblocks:
if (sb->sb_rextslog != libxfs_highbit32((unsigned int)sb->sb_rextents))
The difference here is that xfs_highbit32 returns -1 if its argument is zero, which means that for a runt rt volume, repair thinks the "correct" value of rextslog is -1, even though mkfs wrote it as 0 and flags a freshly formatted filesystem as corrupt. Because mkfs has been writing ondisk artifacts like this for decades, we have to accept that as "correct". TBH, zero rextslog for zero rtextents makes more sense to me anyway.
Regrettably, the superblock verifier checks created in commit copied xfs_repair even though mkfs has been writing out such filesystems for ages. In testing /that/ fix, I discovered that the logic above is wrong -- rsumlevels is always rextslog + 1 when rblocks > 0, even if rextents == 0.
IOWs, this logic needs to be:
/*
* Now that we've locked the rtbitmap and rtsummary, we can't race with
* growfsrt trying to expand the summary or change the size of the rt
* volume. Hence it is safe to compute and check the geometry values.
*/
if (mp->m_sb.sb_rblocks) {
xfs_filblks_t rsumblocks;
int rextslog = 0;
rts->rextents = xfs_rtb_to_rtx(mp, mp->m_sb.sb_rblocks);
if (rts->rextents)
rextslog = xfs_highbit32(rts->rextents);
rts->rsumlevels = rextslog + 1;
rts->rbmblocks = xfs_rtbitmap_blockcount(mp, rts->rextents);
rsumblocks = xfs_rtsummary_blockcount(mp, rts->rsumlevels,
rts->rbmblocks);
rts->rsumsize = XFS_FSB_TO_B(mp, rsumblocks);
}Yay winning.
--D
Quoted 8 lines
Quoted 3 lines
Otherwise looks good:
Reviewed-by Christoph Hellwig <[email protected]>Thanks!
--D
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
In reply to Darrick J. WongQuoted 2 lines · Darrick J. Wong
LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...
I think we should just reject rt device size < rtextsize configs in the kernel and all tools.
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong
LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...
I think we should just reject rt device size < rtextsize configs in the kernel and all tools.
"But that could break old weirdass customer filesystems."
The design of rtgroups prohibits that, so we're ok going forward.
--D
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
Quoted 11 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong
LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...
I think we should just reject rt device size < rtextsize configs in the kernel and all tools.
"But that could break old weirdass customer filesystems."
The design of rtgroups prohibits that, so we're ok going forward.
Well, as you just said it hasn't mounted for a long time, and really this is a corner case that just doesn't make any sense. I'd really prefer to cleanly reject it, and if someone really complains with a good reason we can revisit the decisions. But I strongly doubt it's ever going to happen.
Re: [PATCH 2/6] xfs: check rt summary file geometry more thoroughly
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Darrick J. Wong
LOL so I just tried a 64k rt volume with a 1M rextsize and mkfs crashed. I guess I'll go sort out what's going on there...
I think we should just reject rt device size < rtextsize configs in the kernel and all tools.
"But that could break old weirdass customer filesystems."
Quoted 7 lines
Quoted 1 line
The design of rtgroups prohibits that, so we're ok going forward.
Well, as you just said it hasn't mounted for a long time, and really this is a corner case that just doesn't make any sense. I'd really prefer to cleanly reject it, and if someone really complains with a good reason we can revisit the decisions. But I strongly doubt it's ever going to happen.
Oh, even better, Dave and I noticed today that if you format a 17G realtime volume (> 2^32 rt extents) then mkfs fails because there's an integer overflow:
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/tree/mkfs/xfs_mkfs.c#n3739
Based on your observation that rt free space never exceeds the group length with rtgroups turned on, I'll tweak the sb_rextslog computation so that it's computed with (rgblocks / rextsize) instead of (rblocks / rextsize) which will fix that problem for future filesystems.
--D
PATCH 3/6 xfs: always check the rtbitmap and rtsummary files
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
XFS filesystems always have a realtime bitmap and summary file, even if there has never been a realtime volume attached. Always check them.
--- fs/xfs/scrub/scrub.c | 2 -- 1 file changed, 2 deletions(-)
fs/xfs/scrub/scrub.c +0 -2
@@ -330,14 +330,12 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_FS, .setup = xchk_setup_rtbitmap, .scrub = xchk_rtbitmap, - .has = xfs_has_realtime, .repair = xrep_notsupported, }, [XFS_SCRUB_TYPE_RTSUM] = { /* realtime summary */ .type = ST_FS, .setup = xchk_setup_rtsummary, .scrub = xchk_rtsummary, - .has = xfs_has_realtime, .repair = xrep_notsupported, }, [XFS_SCRUB_TYPE_UQUOTA] = { /* user quota */
Re: [PATCH 3/6] xfs: always check the rtbitmap and rtsummary files
Looks good:
PATCH 4/6 xfs: repair the inode core and forks of a metadata inode
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a helper function to repair the core and forks of a metadata inode, so that we can get move onto the task of repairing higher level metadata that lives in an inode.
--- fs/xfs/scrub/bmap_repair.c | 17 ++++- fs/xfs/scrub/repair.c | 151 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 2 + 3 files changed, 166 insertions(+), 4 deletions(-)
fs/xfs/scrub/bmap_repair.c +13 -4
@@ -78,6 +78,9 @@ struct xrep_bmap {/* Are there shared extents? */ bool shared_extents; + +/* Do we allow unwritten extents? */ +bool allow_unwritten; };/* Is this space extent shared? Flag the inode if it is. */
@@ -272,6 +275,10 @@ xrep_bmap_walk_rmap( !(rec->rm_flags & XFS_RMAP_ATTR_FORK))return 0 ; +/* Reject unwritten extents if we don't allow those. */ +if ((rec->rm_flags &XFS_RMAP_UNWRITTEN ) && !rb->allow_unwritten) +return -EFSCORRUPTED ; + fsbno =XFS_AGB_TO_FSB (mp, cur->bc_ag.pag->pag_agno, rec->rm_startblock);
@@ -762,10 +769,11 @@ xrep_bmap_check_inputs( }/* Repair an inode fork. */ -STATIC int +int xrep_bmap (struct xfs_scrub *sc, -int whichfork) +int whichfork, +bool allow_unwritten) {struct xrep_bmap *rb;char *descr;
@@ -784,6 +792,7 @@ xrep_bmap(return -ENOMEM ; rb->sc = sc; rb->whichfork = whichfork; + rb->allow_unwritten = allow_unwritten;/* * No need to waste time scanning for shared extents if the inode is
@@ -834,7 +843,7 @@ intxrep_bmap_data ( struct xfs_scrub *sc) { -return xrep_bmap (sc,XFS_DATA_FORK ); +return xrep_bmap (sc,XFS_DATA_FORK , true); }/* Repair an inode's attr fork. */
@@ -842,5 +851,5 @@ intxrep_bmap_attr ( struct xfs_scrub *sc) { -return xrep_bmap (sc,XFS_ATTR_FORK ); +return xrep_bmap (sc,XFS_ATTR_FORK , false); }
fs/xfs/scrub/repair.c +151 -0
160 lines not on this page · raw message
fs/xfs/scrub/repair.h +2 -0
@@ -82,6 +82,8 @@ int xrep_ino_dqattach(struct xfs_scrub *sc);int xrep_ino_ensure_extent_count (struct xfs_scrub *sc,int whichfork,xfs_extnum_t nextents);int xrep_reset_perag_resv (struct xfs_scrub *sc); +int xrep_bmap (struct xfs_scrub *sc,int whichfork,bool allow_unwritten); +int xrep_metadata_inode_forks (struct xfs_scrub *sc);/* Repair setup functions */ int xrep_setup_ag_allocbt (struct xfs_scrub *sc);
Re: [PATCH 4/6] xfs: repair the inode core and forks of a metadata inode
Looks good:
PATCH 5/6 xfs: create a new inode fork block unmap helper
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a new helper to unmap blocks from an inode's fork.
--- fs/xfs/libxfs/xfs_bmap.c | 39 +++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_bmap.h | 2 ++ fs/xfs/xfs_inode.c | 24 ++++-------------------- 3 files changed, 45 insertions(+), 20 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +39 -0
@@ -6231,3 +6231,42 @@ xfs_bmap_validate_extent(return xfs_bmap_validate_extent_raw (ip->i_mount,XFS_IS_REALTIME_INODE (ip), whichfork, irec); } + +/* +* Used in xfs_itruncate_extents(). This is the maximum number of extents +* freed from a file in a single transaction. +*/ +#define XFS_ITRUNC_MAX_EXTENTS 2 + +/* +* Unmap every extent in part of an inode's fork. We don't do any higher level +* invalidation work at all. +*/ +int +xfs_bunmapi_range ( +struct xfs_trans **tpp, +struct xfs_inode *ip, +uint32_t flags, +xfs_fileoff_t startoff, +xfs_fileoff_t endoff) +{ +xfs_filblks_t unmap_len = endoff - startoff +1 ; +int error =0 ; + +ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL )); + +while (unmap_len >0 ) { +ASSERT ((*tpp)->t_highest_agno ==NULLAGNUMBER ); + error =__xfs_bunmapi (*tpp, ip, startoff, &unmap_len, flags, +XFS_ITRUNC_MAX_EXTENTS ); +if (error) + goto out; + +/* free the just unmapped extents */ + error =xfs_defer_finish (tpp); +if (error) + goto out; + } +out: +return error; +}
fs/xfs/libxfs/xfs_bmap.h +2 -0
@@ -273,6 +273,8 @@ int xfs_bmap_complain_bad_rec(struct xfs_inode *ip, int whichfork,int xfs_bmapi_remap (struct xfs_trans *tp,struct xfs_inode *ip,xfs_fileoff_t bno,xfs_filblks_t len,xfs_fsblock_t startblock,uint32_t flags); +int xfs_bunmapi_range (struct xfs_trans **tpp,struct xfs_inode *ip, +uint32_t flags,xfs_fileoff_t startoff,xfs_fileoff_t endoff);extern struct kmem_cache *xfs_bmap_intent_cache;
fs/xfs/xfs_inode.c +4 -20
@@ -40,12 +40,6 @@struct kmem_cache *xfs_inode_cache; -/* -* Used in xfs_itruncate_extents(). This is the maximum number of extents -* freed from a file in a single transaction. -*/ -#define XFS_ITRUNC_MAX_EXTENTS 2 - STATIC int xfs_iunlink(struct xfs_trans *, struct xfs_inode *); STATIC int xfs_iunlink_remove(struct xfs_trans *tp, struct xfs_perag *pag,struct xfs_inode *);
@@ -1339,7 +1333,6 @@ xfs_itruncate_extents_flags(struct xfs_mount *mp = ip->i_mount;struct xfs_trans *tp = *tpp;xfs_fileoff_t first_unmap_block; -xfs_filblks_t unmap_len;int error =0 ;ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL ));
@@ -1371,19 +1364,10 @@ xfs_itruncate_extents_flags(return 0 ; } - unmap_len =XFS_MAX_FILEOFF - first_unmap_block +1 ; -while (unmap_len >0 ) { -ASSERT (tp->t_highest_agno ==NULLAGNUMBER ); - error =__xfs_bunmapi (tp, ip, first_unmap_block, &unmap_len, - flags,XFS_ITRUNC_MAX_EXTENTS ); -if (error) - goto out; - -/* free the just unmapped extents */ - error =xfs_defer_finish (&tp); -if (error) - goto out; - } + error =xfs_bunmapi_range (&tp, ip, flags, first_unmap_block, +XFS_MAX_FILEOFF ); +if (error) + goto out;if (whichfork ==XFS_DATA_FORK ) {/* Remove all pending CoW reservations. */
Re: [PATCH 5/6] xfs: create a new inode fork block unmap helper
Quoted 3 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Create a new helper to unmap blocks from an inode's fork.
Look good:
With this __xfs_bunmapi an be marked static in xfs_bmap.c, as there is no user outside of it (and neither in xfsprogs).
PATCH 6/6 xfs: online repair of realtime bitmaps
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Fix all the file metadata surrounding the realtime bitmap file, which includes the rt geometry, file size, forks, and space mappings. The bitmap contents themselves cannot be fixed without rt rmap, so that will come later.
--- fs/xfs/Makefile | 4 + fs/xfs/scrub/repair.h | 7 + fs/xfs/scrub/rtbitmap.c | 16 ++- fs/xfs/scrub/rtbitmap.h | 22 ++++ fs/xfs/scrub/rtbitmap_repair.c | 202 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/scrub.c | 2 6 files changed, 245 insertions(+), 8 deletions(-) create mode 100644 fs/xfs/scrub/rtbitmap.h create mode 100644 fs/xfs/scrub/rtbitmap_repair.c
fs/xfs/Makefile +4 -0
@@ -191,5 +191,9 @@ xfs-y += $(addprefix scrub/, \ refcount_repair.o \ repair.o \ ) + +xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \ + rtbitmap_repair.o \ + ) endif endif
fs/xfs/scrub/repair.h +7 -0
@@ -115,6 +115,12 @@ int xrep_bmap_data(struct xfs_scrub *sc);int xrep_bmap_attr (struct xfs_scrub *sc);int xrep_bmap_cow (struct xfs_scrub *sc); +#ifdef CONFIG_XFS_RT +int xrep_rtbitmap (struct xfs_scrub *sc); +#else +# define xrep_rtbitmap xrep_notsupported +#endif /* CONFIG_XFS_RT */ +int xrep_reinit_pagf (struct xfs_scrub *sc);int xrep_reinit_pagi (struct xfs_scrub *sc);
@@ -177,6 +183,7 @@ xrep_setup_nothing(#define xrep_bmap_data xrep_notsupported#define xrep_bmap_attr xrep_notsupported#define xrep_bmap_cow xrep_notsupported +#define xrep_rtbitmap xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/rtbitmap.c +9 -7
@@ -17,12 +17,8 @@#include "xfs_bit.h" #include "scrub/scrub.h" #include "scrub/common.h" - -struct xchk_rtbitmap { -uint64_t rextents; -uint64_t rbmblocks; -unsigned int rextslog; -}; +#include "scrub/repair.h" +#include "scrub/rtbitmap.h" /* Set us up with the realtime metadata locked. */ int
@@ -38,7 +34,13 @@ xchk_setup_rtbitmap(return -ENOMEM ; sc->buf = rtb; - error =xchk_trans_alloc (sc,0 ); +if (xchk_could_repair (sc)) { + error =xrep_setup_rtbitmap (sc, rtb); +if (error) +return error; + } + + error =xchk_trans_alloc (sc, rtb->resblks);if (error)return error;
fs/xfs/scrub/rtbitmap.h new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_SCRUB_RTBITMAP_H__ +#define __XFS_SCRUB_RTBITMAP_H__ + +struct xchk_rtbitmap { +uint64_t rextents; +uint64_t rbmblocks; +unsigned int rextslog; +unsigned int resblks; +}; + +#ifdef CONFIG_XFS_ONLINE_REPAIR +int xrep_setup_rtbitmap (struct xfs_scrub *sc,struct xchk_rtbitmap *rtb); +#else +# define xrep_setup_rtbitmap (sc, rtb) (0) +#endif /* CONFIG_XFS_ONLINE_REPAIR */ + +#endif /* __XFS_SCRUB_RTBITMAP_H__ */
fs/xfs/scrub/rtbitmap_repair.c new file +202 -0
202 lines not on this page · raw message
fs/xfs/scrub/scrub.c +1 -1
@@ -330,7 +330,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_FS, .setup = xchk_setup_rtbitmap, .scrub = xchk_rtbitmap, - .repair = xrep_notsupported, + .repair = xrep_rtbitmap, }, [XFS_SCRUB_TYPE_RTSUM] = { /* realtime summary */ .type = ST_FS,
Re: [PATCH 6/6] xfs: online repair of realtime bitmaps
Looks good:
cover xfs: online repair of quota and rt metadata files
In reply to Darrick J. WongHi all,
XFS stores quota records and free space bitmap information in files. Add the necessary infrastructure to enable repairing metadata inodes and their forks, and then make it so that we can repair the file metadata for the rtbitmap. Repairing the bitmap contents (and the summary file) is left for subsequent patchsets.
We also add the ability to repair file metadata the quota files. As part of these repairs, we also reinitialize the ondisk dquot records as necessary to get the incore dquots working. We can also correct obviously bad dquot record attributes, but we leave checking the resource usage counts for the next patchsets.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-quota
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-quota
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-quota
--- fs/xfs/Makefile | 9 + fs/xfs/libxfs/xfs_format.h | 3 fs/xfs/scrub/dqiterate.c | 211 ++++++++++++++++ fs/xfs/scrub/quota.c | 107 +++++++- fs/xfs/scrub/quota.h | 36 +++ fs/xfs/scrub/quota_repair.c | 575 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 7 + fs/xfs/scrub/scrub.c | 6 fs/xfs/scrub/trace.c | 3 fs/xfs/scrub/trace.h | 78 ++++++ fs/xfs/xfs_dquot.c | 37 --- fs/xfs/xfs_dquot.h | 8 - 12 files changed, 1026 insertions(+), 54 deletions(-) create mode 100644 fs/xfs/scrub/dqiterate.c create mode 100644 fs/xfs/scrub/quota.h create mode 100644 fs/xfs/scrub/quota_repair.c
PATCH 1/5 xfs: check the ondisk space mapping behind a dquot
From: Darrick J. Wong <[email protected]>
Each xfs_dquot object caches the file offset and daddr of the ondisk block that backs the dquot. Make sure these cached values are the same as the bmapi data, and that the block state is written.
--- fs/xfs/scrub/quota.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+)
fs/xfs/scrub/quota.c +58 -0
@@ -6,6 +6,7 @@#include "xfs.h" #include "xfs_fs.h" #include "xfs_shared.h" +#include "xfs_bit.h" #include "xfs_format.h" #include "xfs_trans_resv.h" #include "xfs_mount.h"
@@ -75,6 +76,47 @@ struct xchk_quota_info {xfs_dqid_t last_id; }; +/* There's a written block backing this dquot, right? */ +STATIC int +xchk_quota_item_bmap ( +struct xfs_scrub *sc, +struct xfs_dquot *dq, +xfs_fileoff_t offset) +{ +struct xfs_bmbt_irec irec; +struct xfs_mount *mp = sc->mp; +int nmaps =1 ; +int error; + +if (!xfs_verify_fileoff (mp, offset)) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); +return 0 ; + } + +if (dq->q_fileoffset != offset) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); +return 0 ; + } + + error =xfs_bmapi_read (sc->ip, offset,1 , &irec, &nmaps,0 ); +if (error) +return error; + +if (nmaps !=1 ) { +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); +return 0 ; + } + +if (!xfs_verify_fsbno (mp, irec.br_startblock)) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); +if (XFS_FSB_TO_DADDR (mp, irec.br_startblock) != dq->q_blkno) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); +if (!xfs_bmap_is_written_extent (&irec)) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); + +return 0 ; +} +/* Scrub the fields in an individual quota item. */ STATIC int xchk_quota_item(
@@ -93,6 +135,17 @@ xchk_quota_item(if (xchk_should_terminate (sc, &error))return error; +/* +* We want to validate the bmap record for the storage backing this +* dquot, so we need to lock the dquot and the quota file. For quota +* operations, the locking order is first the ILOCK and then the dquot. +* However, dqiterate gave us a locked dquot, so drop the dquot lock to +* get the ILOCK. +*/ +xfs_dqunlock (dq); +xchk_ilock (sc,XFS_ILOCK_SHARED ); +xfs_dqlock (dq); +/* * Except for the root dquot, the actual dquot we got must either have * the same or higher id as we saw before.
@@ -103,6 +156,11 @@ xchk_quota_item( sqi->last_id = dq->q_id; + error =xchk_quota_item_bmap (sc, dq, offset); +xchk_iunlock (sc,XFS_ILOCK_SHARED ); +if (!xchk_fblock_process_error (sc,XFS_DATA_FORK , offset, &error)) +return error; +/* * Warn if the hard limits are larger than the fs. * Administrators can do this, though in production this seems
Re: [PATCH 1/5] xfs: check the ondisk space mapping behind a dquot
Looks good:
PATCH 2/5 xfs: check dquot resource timers
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
For each dquot resource, ensure either (a) the resource usage is over the soft limit and there is a nonzero timer; or (b) usage is at or under the soft limit and the timer is unset. (a) is redundant with the dquot buffer verifier, but (b) isn't checked anywhere.
Found by fuzzing xfs/426 and noticing that diskdq.btimer = add didn't trip any kind of warning for having a timer set even with no limits.
--- fs/xfs/scrub/quota.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
fs/xfs/scrub/quota.c +21 -0
@@ -117,6 +117,23 @@ xchk_quota_item_bmap(return 0 ; } +/* Complain if a quota timer is incorrectly set. */ +static inline void +xchk_quota_item_timer ( +struct xfs_scrub *sc, +xfs_fileoff_t offset, +const struct xfs_dquot_res *res) +{ +if ((res->softlimit && res->count > res->softlimit) || + (res->hardlimit && res->count > res->hardlimit)) { +if (!res->timer) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); + }else { +if (res->timer) +xchk_fblock_set_corrupt (sc,XFS_DATA_FORK , offset); + } +} +/* Scrub the fields in an individual quota item. */ STATIC int xchk_quota_item(
@@ -224,6 +241,10 @@ xchk_quota_item( dq->q_rtb.count > dq->q_rtb.hardlimit)xchk_fblock_set_warning (sc,XFS_DATA_FORK , offset); +xchk_quota_item_timer (sc, offset, &dq->q_blk); +xchk_quota_item_timer (sc, offset, &dq->q_ino); +xchk_quota_item_timer (sc, offset, &dq->q_rtb); + out:if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT )return -ECANCELED ;
Re: [PATCH 2/5] xfs: check dquot resource timers
Looks good:
PATCH 3/5 xfs: pull xfs_qm_dqiterate back into scrub
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
There aren't any other users of this code outside of online fsck, so pull it back in there.
--- fs/xfs/Makefile | 5 ++++ fs/xfs/scrub/dqiterate.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/quota.c | 3 ++- fs/xfs/scrub/quota.h | 14 ++++++++++++ fs/xfs/xfs_dquot.c | 31 --------------------------- fs/xfs/xfs_dquot.h | 5 ---- 6 files changed, 72 insertions(+), 38 deletions(-) create mode 100644 fs/xfs/scrub/dqiterate.c create mode 100644 fs/xfs/scrub/quota.h
fs/xfs/Makefile +4 -1
@@ -175,7 +175,10 @@ xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \ rtsummary.o \ ) -xfs-$(CONFIG_XFS_QUOTA) += scrub/quota.o +xfs-$(CONFIG_XFS_QUOTA) += $(addprefix scrub/, \ + dqiterate.o \ + quota.o \ + ) # online repair ifeq ($(CONFIG_XFS_ONLINE_REPAIR),y)
fs/xfs/scrub/dqiterate.c new file +52 -0
52 lines not on this page · raw message
fs/xfs/scrub/quota.c +2 -1
@@ -18,6 +18,7 @@#include "xfs_bmap.h" #include "scrub/scrub.h" #include "scrub/common.h" +#include "scrub/quota.h" /* Convert a scrub type code to a DQ flag, or return 0 if error. */ static inline xfs_dqtype_t
@@ -320,7 +321,7 @@ xchk_quota(xchk_iunlock (sc, sc->ilock_flags); sqi.sc = sc; sqi.last_id =0 ; - error =xfs_qm_dqiterate (mp, dqtype, xchk_quota_item, &sqi); + error =xchk_dqiterate (mp, dqtype, xchk_quota_item, &sqi);xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ;
fs/xfs/scrub/quota.h new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2018-2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_SCRUB_QUOTA_H__ +#define __XFS_SCRUB_QUOTA_H__ + +typedef int (*xchk_dqiterate_fn )(struct xfs_dquot *dq, +xfs_dqtype_t type,void *priv); +int xchk_dqiterate (struct xfs_mount *mp,xfs_dqtype_t type, +xchk_dqiterate_fn iter_fn,void *priv); + +#endif /* __XFS_SCRUB_QUOTA_H__ */
fs/xfs/xfs_dquot.c +0 -31
@@ -1362,34 +1362,3 @@ xfs_qm_exit(void)kmem_cache_destroy (xfs_dqtrx_cache);kmem_cache_destroy (xfs_dquot_cache); } - -/* -* Iterate every dquot of a particular type. The caller must ensure that the -* particular quota type is active. iter_fn can return negative error codes, -* or -ECANCELED to indicate that it wants to stop iterating. -*/ -int -xfs_qm_dqiterate ( -struct xfs_mount *mp, -xfs_dqtype_t type, -xfs_qm_dqiterate_fn iter_fn, -void *priv) -{ -struct xfs_dquot *dq; -xfs_dqid_t id =0 ; -int error; - -do { - error =xfs_qm_dqget_next (mp, id, type, &dq); -if (error == -ENOENT ) -return 0 ; -if (error) -return error; - - error =iter_fn (dq, type, priv); - id = dq->q_id +1 ; -xfs_qm_dqput (dq); - }while (error ==0 && id !=0 ); - -return error; -}
fs/xfs/xfs_dquot.h +0 -5
@@ -234,11 +234,6 @@ static inline struct xfs_dquot *xfs_qm_dqhold(struct xfs_dquot *dqp)return dqp; } -typedef int (*xfs_qm_dqiterate_fn )(struct xfs_dquot *dq, -xfs_dqtype_t type,void *priv); -int xfs_qm_dqiterate (struct xfs_mount *mp,xfs_dqtype_t type, -xfs_qm_dqiterate_fn iter_fn,void *priv); -time64_t xfs_dquot_set_timeout (struct xfs_mount *mp,time64_t timeout);time64_t xfs_dquot_set_grace_period (time64_t grace);
Re: [PATCH 3/5] xfs: pull xfs_qm_dqiterate back into scrub
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
There aren't any other users of this code outside of online fsck, so pull it back in there.
The move itself looks fine, but what about just open coding it and getting rid of the indirection functional?
PATCH 4/5 xfs: improve dquot iteration for scrub
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Upon a closer inspection of the quota record scrubber, I noticed that dqiterate wasn't actually walking all possible dquots for the mapped blocks in the quota file. This is due to xfs_qm_dqget_next skipping all XFS_IS_DQUOT_UNINITIALIZED dquots.
For a fsck program, we really want to look at all the dquots, even if all counters and limits in the dquot record are zero. Rewrite the implementation to do this, as well as switching to an iterator paradigm to reduce the number of indirect calls.
--- fs/xfs/libxfs/xfs_format.h | 3 + fs/xfs/scrub/dqiterate.c | 195 ++++++++++++++++++++++++++++++++++++++++---- fs/xfs/scrub/quota.c | 24 +++-- fs/xfs/scrub/quota.h | 28 +++++- fs/xfs/scrub/trace.c | 2 fs/xfs/scrub/trace.h | 49 +++++++++++ 6 files changed, 270 insertions(+), 31 deletions(-)
fs/xfs/libxfs/xfs_format.h +3 -0
@@ -1273,6 +1273,9 @@ static inline time64_t xfs_dq_bigtime_to_unix(uint32_t ondisk_seconds)#define XFS_DQ_GRACE_MIN ((int64_t)0)#define XFS_DQ_GRACE_MAX ((int64_t)U32_MAX) +/* Maximum id value for a quota record */ +#define XFS_DQ_ID_MAX (U32_MAX) +/* * This is the main portion of the on-disk representation of quota information * for a user. We pad this with some more expansion room to construct the on
fs/xfs/scrub/dqiterate.c +177 -18
211 lines not on this page · raw message
fs/xfs/scrub/quota.c +15 -9
@@ -138,11 +138,9 @@ xchk_quota_item_timer(/* Scrub the fields in an individual quota item. */ STATIC intxchk_quota_item ( -struct xfs_dquot *dq, -xfs_dqtype_t dqtype, -void *priv) +struct xchk_quota_info *sqi, +struct xfs_dquot *dq) { -struct xchk_quota_info *sqi = priv;struct xfs_scrub *sc = sqi->sc;struct xfs_mount *mp = sc->mp;struct xfs_quotainfo *qi = mp->m_quotainfo;
@@ -271,7 +269,7 @@ xchk_quota_data_fork(return error;/* Check for data fork problems that apply only to quota files. */ - max_dqid_off = ((xfs_dqid_t)-1 ) / qi->qi_dqperchunk; + max_dqid_off =XFS_DQ_ID_MAX / qi->qi_dqperchunk; ifp =xfs_ifork_ptr (sc->ip,XFS_DATA_FORK );for_each_xfs_iext (ifp, &icur, &irec) {if (xchk_should_terminate (sc, &error))
@@ -298,9 +296,11 @@ intxchk_quota ( struct xfs_scrub *sc) { -struct xchk_quota_info sqi; +struct xchk_dqiter cursor = { }; +struct xchk_quota_info sqi = { .sc = sc };struct xfs_mount *mp = sc->mp;struct xfs_quotainfo *qi = mp->m_quotainfo; +struct xfs_dquot *dq;xfs_dqtype_t dqtype;int error =0 ;
@@ -319,9 +319,15 @@ xchk_quota(* functions. */ xchk_iunlock (sc, sc->ilock_flags); - sqi.sc = sc; - sqi.last_id =0 ; - error =xchk_dqiterate (mp, dqtype, xchk_quota_item, &sqi); + +/* Now look for things that the quota verifiers won't complain about. */ +xchk_dqiter_init (&cursor, sc, dqtype); +while ((error =xchk_dquot_iter (&cursor, &dq)) ==1 ) { + error =xchk_quota_item (&sqi, dq); +xfs_qm_dqput (dq); +if (error) +break ; + }xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ;
fs/xfs/scrub/quota.h +24 -4
@@ -6,9 +6,29 @@#ifndef __XFS_SCRUB_QUOTA_H__#define __XFS_SCRUB_QUOTA_H__ -typedef int (*xchk_dqiterate_fn )(struct xfs_dquot *dq, -xfs_dqtype_t type,void *priv); -int xchk_dqiterate (struct xfs_mount *mp,xfs_dqtype_t type, -xchk_dqiterate_fn iter_fn,void *priv); +/* dquot iteration code */ + +struct xchk_dqiter { +struct xfs_scrub *sc; + +/* Quota file that we're walking. */ +struct xfs_inode *quota_ip; + +/* Cached data fork mapping for the dquot. */ +struct xfs_bmbt_irec bmap; + +/* The next dquot to scan. */ +uint64_t id; + +/* Quota type (user/group/project). */ +xfs_dqtype_t dqtype; + +/* Data fork sequence number to detect stale mappings. */ +unsigned int if_seq; +}; + +void xchk_dqiter_init (struct xchk_dqiter *cursor,struct xfs_scrub *sc, +xfs_dqtype_t dqtype); +int xchk_dquot_iter (struct xchk_dqiter *cursor,struct xfs_dquot **dqpp);#endif /* __XFS_SCRUB_QUOTA_H__ */
fs/xfs/scrub/trace.c +2 -0
@@ -14,9 +14,11 @@#include "xfs_btree.h" #include "xfs_ag.h" #include "xfs_rtbitmap.h" +#include "xfs_quota.h" #include "scrub/scrub.h" #include "scrub/xfile.h" #include "scrub/xfarray.h" +#include "scrub/quota.h" /* Figure out which block the btree cursor was pointing to. */ static inline xfs_fsblock_t
fs/xfs/scrub/trace.h +49 -0
61 lines not on this page · raw message
Re: [PATCH 4/5] xfs: improve dquot iteration for scrub
Quoted 11 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Upon a closer inspection of the quota record scrubber, I noticed that dqiterate wasn't actually walking all possible dquots for the mapped blocks in the quota file. This is due to
xfs_qm_dqget_nextskipping allXFS_IS_DQUOT_UNINITIALIZEDdquots.For a fsck program, we really want to look at all the dquots, even if all counters and limits in the dquot record are zero. Rewrite the implementation to do this, as well as switching to an iterator paradigm to reduce the number of indirect calls.
Heh, this basically ends up doing what I suggested in the last patch (and a lot more).
I'd just fold the previous patch into this one, as there's no point in moving the function just to remove it immediately.
Otherwise looks good:
PATCH 5/5 xfs: repair quotas
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Fix anything that causes the quota verifiers to fail.
--- fs/xfs/Makefile | 4 fs/xfs/scrub/quota.c | 3 fs/xfs/scrub/quota.h | 2 fs/xfs/scrub/quota_repair.c | 575 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 7 + fs/xfs/scrub/scrub.c | 6 fs/xfs/scrub/trace.c | 1 fs/xfs/scrub/trace.h | 29 ++ fs/xfs/xfs_dquot.c | 6 fs/xfs/xfs_dquot.h | 3 10 files changed, 628 insertions(+), 8 deletions(-) create mode 100644 fs/xfs/scrub/quota_repair.c
fs/xfs/Makefile +4 -0
@@ -198,5 +198,9 @@ xfs-y += $(addprefix scrub/, \ xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \ rtbitmap_repair.o \ ) + +xfs-$(CONFIG_XFS_QUOTA) += $(addprefix scrub/, \ + quota_repair.o \ + ) endif endif
fs/xfs/scrub/quota.c +1 -2
@@ -21,7 +21,7 @@#include "scrub/quota.h" /* Convert a scrub type code to a DQ flag, or return 0 if error. */ -static inline xfs_dqtype_t +xfs_dqtype_t xchk_quota_to_dqtype (struct xfs_scrub *sc) {
@@ -328,7 +328,6 @@ xchk_quota(if (error)break ; } -xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,
fs/xfs/scrub/quota.h +2 -0
@@ -6,6 +6,8 @@#ifndef __XFS_SCRUB_QUOTA_H__#define __XFS_SCRUB_QUOTA_H__ +xfs_dqtype_t xchk_quota_to_dqtype (struct xfs_scrub *sc); +/* dquot iteration code */ struct xchk_dqiter {
fs/xfs/scrub/quota_repair.c new file +575 -0
575 lines not on this page · raw message
fs/xfs/scrub/repair.h +7 -0
@@ -121,6 +121,12 @@ int xrep_rtbitmap(struct xfs_scrub *sc);# define xrep_rtbitmap xrep_notsupported#endif /* CONFIG_XFS_RT */ +#ifdef CONFIG_XFS_QUOTA +int xrep_quota (struct xfs_scrub *sc); +#else +# define xrep_quota xrep_notsupported +#endif /* CONFIG_XFS_QUOTA */ +int xrep_reinit_pagf (struct xfs_scrub *sc);int xrep_reinit_pagi (struct xfs_scrub *sc);
@@ -184,6 +190,7 @@ xrep_setup_nothing(#define xrep_bmap_attr xrep_notsupported#define xrep_bmap_cow xrep_notsupported#define xrep_rtbitmap xrep_notsupported +#define xrep_quota xrep_notsupported#endif /* CONFIG_XFS_ONLINE_REPAIR */
fs/xfs/scrub/scrub.c +3 -3
@@ -342,19 +342,19 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_FS, .setup = xchk_setup_quota, .scrub = xchk_quota, - .repair = xrep_notsupported, + .repair = xrep_quota, }, [XFS_SCRUB_TYPE_GQUOTA] = { /* group quota */ .type = ST_FS, .setup = xchk_setup_quota, .scrub = xchk_quota, - .repair = xrep_notsupported, + .repair = xrep_quota, }, [XFS_SCRUB_TYPE_PQUOTA] = { /* project quota */ .type = ST_FS, .setup = xchk_setup_quota, .scrub = xchk_quota, - .repair = xrep_notsupported, + .repair = xrep_quota, }, [XFS_SCRUB_TYPE_FSCOUNTERS] = { /* fs summary counters */ .type = ST_FS,
fs/xfs/scrub/trace.c +1 -0
@@ -15,6 +15,7 @@#include "xfs_ag.h" #include "xfs_rtbitmap.h" #include "xfs_quota.h" +#include "xfs_quota_defs.h" #include "scrub/scrub.h" #include "scrub/xfile.h" #include "scrub/xfarray.h"
fs/xfs/scrub/trace.h +29 -0
@@ -1732,6 +1732,35 @@ TRACE_EVENT(xrep_cow_free_staging, __entry->blockcount) ); +#ifdef CONFIG_XFS_QUOTA +DECLARE_EVENT_CLASS (xrep_dquot_class, + TP_PROTO(struct xfs_mount *mp, uint8_t type, uint32_t id), +TP_ARGS (mp, type, id), +TP_STRUCT__entry ( + __field(dev_t, dev) + __field(uint8_t, type) +__field (uint32_t, id) + ), +TP_fast_assign ( + __entry->dev = mp->m_super->s_dev; + __entry->id = id; + __entry->type = type; + ), +TP_printk ("dev %d:%d type %s id 0x%x" , +MAJOR (__entry->dev),MINOR (__entry->dev), +__print_flags (__entry->type,"|" ,XFS_DQTYPE_STRINGS ), + __entry->id) +); + +#define DEFINE_XREP_DQUOT_EVENT (name) \ +DEFINE_EVENT(xrep_dquot_class, name, \ + TP_PROTO(struct xfs_mount *mp, uint8_t type, uint32_t id), \ + TP_ARGS(mp, type, id)) +DEFINE_XREP_DQUOT_EVENT (xrep_dquot_item); +DEFINE_XREP_DQUOT_EVENT (xrep_disk_dquot); +DEFINE_XREP_DQUOT_EVENT (xrep_dquot_item_fill_bmap_hole); +#endif /* CONFIG_XFS_QUOTA */ +#endif /* IS_ENABLED(CONFIG_XFS_ONLINE_REPAIR) */ #endif /* _TRACE_XFS_SCRUB_TRACE_H */
fs/xfs/xfs_dquot.c +3 -3
@@ -172,14 +172,14 @@ xfs_qm_adjust_dqtimers(/* * initialize a buffer full of dquots and log the whole thing */ -STATIC void +void xfs_qm_init_dquot_blk (struct xfs_trans *tp, -struct xfs_mount *mp,xfs_dqid_t id,xfs_dqtype_t type,struct xfs_buf *bp) { +struct xfs_mount *mp = tp->t_mountp;struct xfs_quotainfo *q = mp->m_quotainfo;struct xfs_dqblk *d;xfs_dqid_t curid;
@@ -353,7 +353,7 @@ xfs_dquot_disk_alloc(* Make a chunk of dquots out of this buffer and log * the entire thing. */ -xfs_qm_init_dquot_blk (tp, mp, dqp->q_id, qtype, bp); +xfs_qm_init_dquot_blk (tp, dqp->q_id, qtype, bp);xfs_buf_set_ref (bp,XFS_DQUOT_REF );/*
fs/xfs/xfs_dquot.h +3 -0
@@ -237,4 +237,7 @@ static inline struct xfs_dquot *xfs_qm_dqhold(struct xfs_dquot *dqp)time64_t xfs_dquot_set_timeout (struct xfs_mount *mp,time64_t timeout);time64_t xfs_dquot_set_grace_period (time64_t grace); +void xfs_qm_init_dquot_blk (struct xfs_trans *tp,xfs_dqid_t id,xfs_dqtype_t + type,struct xfs_buf *bp); +#endif /* __XFS_DQUOT_H__ */
Re: [PATCH 5/5] xfs: repair quotas
fs/xfs/scrub/quota.c xchk_quota( · line 333
Quoted 8 lines
fs/xfs/scrub/quota.c +0
-1@@ -328,7 +328,6 @@ xchk_quota(if (error)break ; } -xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,
What is the replacement for this lock? The call in xrep_quota_item?
I'm a little confused on how locking works - about all flags in sc->ilock_flags are released, and here we used just lock the exclusive ilock directly without tracking it.
Re: [PATCH 5/5] xfs: repair quotas
Quoted 10 lines · Christoph Hellwig
fs/xfs/scrub/quota.c xchk_quota( · line 333
Quoted 8 lines
fs/xfs/scrub/quota.c +0
-1@@ -328,7 +328,6 @@ xchk_quota(if (error)break ; } -xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,What is the replacement for this lock? The call in
xrep_quota_item?
The replacement is the conditional re-lock at the start of xrep_quota. I could have left this alone, though for the scrub-only case it reduces lock cycling by one.
Quoted 3 lines
I'm a little confused on how locking works - about all flags in
sc->ilock_flagsare released, and here we used just lock the exclusive ilock directly without tracking it.
Fixing quota files is a bit of a locking mess because dqget needs to take the ILOCK to read the data fork if the dquot isn't already in memory. As a result, the scrub functions have to take the lock to inspect the data fork for mapping problems only to drop it before iterating the incore dquots.
For each dquot, we then have to take the ILOCK *again* to check that its q_fileoffset and q_blkno fields actually match the bmap.
Similarly, repair has to retake the lock to fix any problems that were found with the mapping before dropping once again to walk the incore dquots. Then we cycle ILOCK for every dquot to fix the mapping.
Not sure what you meant about "we used just lock the exclusive lock directly without tracking it" -- both files call xchk_{ilock,iunlock}. The telemetry data I've collected shows that quota file checking is sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.
--D
Re: [PATCH 5/5] xfs: repair quotas
Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig
fs/xfs/scrub/quota.c xchk_quota( · line 333
Quoted 8 lines
fs/xfs/scrub/quota.c +0
-1@@ -328,7 +328,6 @@ xchk_quota(if (error)break ; } -xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,What is the replacement for this lock? The call in
xrep_quota_item?
Quoted 1 line
The replacement is the conditional re-lock at the start of
xrep_quota.
Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the xchk_iunlock call in xchk_teardown would have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..
Quoted 6 lines
Not sure what you meant about "we used just lock the exclusive lock directly without tracking it" -- both files call xchk_{ilock,iunlock}. The telemetry data I've collected shows that quota file checking is sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.
My confusions was more about checking/using sc->ilock_flags in the callers, while it is maintained by the locking helpers. Probably not *THAT* unusual, but I might have simply been too tired to understand it.
Re: [PATCH 5/5] xfs: repair quotas
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig
fs/xfs/scrub/quota.c xchk_quota( · line 333
Quoted 8 lines
fs/xfs/scrub/quota.c +0
-1@@ -328,7 +328,6 @@ xchk_quota(if (error)break ; } -xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,What is the replacement for this lock? The call in
xrep_quota_item?
Quoted 7 lines
Quoted 1 line
The replacement is the conditional re-lock at the start of
xrep_quota.Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the
xchk_iunlockcall inxchk_teardownwould have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..
How about I not remove the xchk_ilock call, then? Repair is already smart enough to take the lock if it doesn't have it, so it's not strictly necessary for correct operation.
Quoted 10 lines
Quoted 6 lines
Not sure what you meant about "we used just lock the exclusive lock directly without tracking it" -- both files call xchk_{ilock,iunlock}. The telemetry data I've collected shows that quota file checking is sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.
My confusions was more about checking/using
sc->ilock_flagsin the callers, while it is maintained by the locking helpers. Probably not *THAT* unusual, but I might have simply been too tired to understand it.
Ah, got it. I'll ponder a no-alloc dqget in the meantime.
--D
Re: [PATCH 5/5] xfs: repair quotas
Quoted 15 lines · Darrick J. Wong
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
fs/xfs/scrub/quota.c xchk_quota( · line 333
Quoted 4 lines
fs/xfs/scrub/quota.c +0
-1-xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,What is the replacement for this lock? The call in
xrep_quota_item?Quoted 7 lines
Quoted 1 line
The replacement is the conditional re-lock at the start of
xrep_quota.Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the
xchk_iunlockcall inxchk_teardownwould have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..
Quoted 3 lines
How about I not remove the
xchk_ilockcall, then? Repair is already smart enough to take the lock if it doesn't have it, so it's not strictly necessary for correct operation.
No, please keep this hunk. As I said I would have preferred to have it in a separate hunk to understand it, but it understand it now, and it does seems useful.
Re: [PATCH 5/5] xfs: repair quotas
Quoted 16 lines · Christoph Hellwig
Quoted 15 lines · Darrick J. Wong
Quoted 14 lines
Quoted 8 lines
Quoted 6 lines
fs/xfs/scrub/quota.c xchk_quota( · line 333
Quoted 4 lines
fs/xfs/scrub/quota.c +0
-1-xchk_ilock (sc,XFS_ILOCK_EXCL );if (error == -ECANCELED ) error =0 ; if (!xchk_fblock_process_error(sc, XFS_DATA_FORK,What is the replacement for this lock? The call in
xrep_quota_item?The replacement is the conditional re-lock at the start of
xrep_quota.Hmm. but not all scrub calls do even end up in the repair callbacks, do they? Ok, I guess the
xchk_iunlockcall inxchk_teardownwould have just released it a bit later and we skip the cycle. Would have been a lot easier to understand if this was in a well-explained self-contained patch..
Quoted 7 lines
Quoted 3 lines
How about I not remove the
xchk_ilockcall, then? Repair is already smart enough to take the lock if it doesn't have it, so it's not strictly necessary for correct operation.No, please keep this hunk. As I said I would have preferred to have it in a separate hunk to understand it, but it understand it now, and it does seems useful.
Ok, I'll keep it then.
--D
Re: [PATCH 5/5] xfs: repair quotas
In reply to Darrick J. WongQuoted 3 lines · Darrick J. Wong
sorta slow, so perhaps it would be justified to create a special no-alloc dqget function where the caller is allowed to pre-acquire the ILOCK.
That beeing said, a no-alloc dqget does sound like a sensible idea in general.