xfsprogs: add xfs_repair -R --- log replay
cover xfsprogs: add xfs_repair -R --- log replay
Adds xfs_repair -R: replay the log then repair, without mounting and without discarding it as -L does. Patches 1-2 are standalone fixes; 3-9 implement -R, importing log recovery from v7.1 fs/xfs into libxlog/.
Chris Wedgwood (9):
libxfs: make XBF_DONE actually mark a buffer uptodate
libxfs: don't corrupt a delwri list when a buffer is queued twice
libxfs: record a failed buffer write when it fails
libxlog: rename xfs_log_recover.c to logscan.c
libxlog: import the kernel's log recovery, log items and AIL
libxfs-diff: also compare libxlog against the kernel
libxlog: build the imported kernel code
xfs_repair: add -R to replay a dirty log before repairing
xfs_repair: finish deletions the crash interrupted
db/logformat.c | 6 +- include/Makefile | 1 + include/atomic.h | 2 + include/builddefs.in | 1 + include/hlist.h | 39 +- include/kernel_compat.h | 231 +++ include/kmem.h | 19 + include/libxfs.h | 2 +- include/libxlog.h | 39 +- include/list.h | 7 + include/platform_defs.h | 11 +- include/spinlock.h | 18 +- include/xfs_inode.h | 3 + include/xfs_mount.h | 9 +- include/xfs_trace.h | 30 + include/xfs_trans.h | 118 +- libfrog/bitmask.h | 17 +- libfrog/div64.h | 7 + libxfs/Makefile | 20 +- libxfs/defer_item.c | 995 --------- libxfs/defer_item.h | 56 - libxfs/inactive.c | 181 ++ libxfs/init.c | 73 + libxfs/inode.c | 43 + libxfs/libxfs_io.h | 15 + libxfs/logitem.c | 204 +- libxfs/rdwr.c | 183 +- libxfs/trans.c | 34 +- libxfs/util.c | 8 +- libxfs/xfs_alloc.c | 2 +- libxfs/xfs_attr.c | 2 +- libxfs/xfs_bmap.c | 2 +- libxfs/xfs_defer.c | 4 + libxfs/xfs_exchmaps.c | 2 +- libxfs/xfs_ialloc.c | 3 + libxfs/xfs_parent.c | 2 +- libxfs/xfs_platform.h | 21 +- libxfs/xfs_refcount.c | 2 +- libxfs/xfs_rmap.c | 2 +- libxfs/xfs_trans_resv.c | 7 +- libxlog/Makefile | 14 +- libxlog/logscan.c | 1650 +++++++++++++++ libxlog/util.c | 34 +- libxlog/xfs_attr_item.c | 1206 +++++++++++ libxlog/xfs_attr_item.h | 64 + libxlog/xfs_bmap_item.c | 718 +++++++ libxlog/xfs_bmap_item.h | 78 + libxlog/xfs_buf_item.h | 71 + libxlog/xfs_buf_item_recover.c | 1215 +++++++++++ libxlog/xfs_dquot_item_recover.c | 214 ++ libxlog/xfs_exchmaps_item.c | 608 ++++++ libxlog/xfs_exchmaps_item.h | 64 + libxlog/xfs_extfree_item.c | 1026 +++++++++ libxlog/xfs_extfree_item.h | 100 + libxlog/xfs_icreate_item.c | 260 +++ libxlog/xfs_icreate_item.h | 22 + libxlog/xfs_inode_item.h | 62 + libxlog/xfs_inode_item_recover.c | 602 ++++++ libxlog/xfs_log.h | 147 ++ libxlog/xfs_log_priv.h | 746 +++++++ libxlog/xfs_log_recover.c | 3308 +++++++++++++++++++++++------- libxlog/xfs_refcount_item.c | 861 ++++++++ libxlog/xfs_refcount_item.h | 82 + libxlog/xfs_rmap_item.c | 890 ++++++++ libxlog/xfs_rmap_item.h | 81 + libxlog/xfs_trans_ail.c | 978 +++++++++ libxlog/xfs_trans_priv.h | 170 ++ logprint/log_misc.c | 4 +- logprint/log_print_all.c | 2 +- logprint/logprint.c | 2 +- logprint/logprint.h | 2 +- man/man8/xfs_repair.8 | 68 +- repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 246 ++- repair/xfs_repair.c | 24 +- tools/libxfs-diff | 11 + 77 files changed, 16157 insertions(+), 1896 deletions(-) create mode 100644 include/kernel_compat.h delete mode 100644 libxfs/defer_item.c delete mode 100644 libxfs/defer_item.h create mode 100644 libxfs/inactive.c create mode 100644 libxlog/logscan.c create mode 100644 libxlog/xfs_attr_item.c create mode 100644 libxlog/xfs_attr_item.h create mode 100644 libxlog/xfs_bmap_item.c create mode 100644 libxlog/xfs_bmap_item.h create mode 100644 libxlog/xfs_buf_item.h create mode 100644 libxlog/xfs_buf_item_recover.c create mode 100644 libxlog/xfs_dquot_item_recover.c create mode 100644 libxlog/xfs_exchmaps_item.c create mode 100644 libxlog/xfs_exchmaps_item.h create mode 100644 libxlog/xfs_extfree_item.c create mode 100644 libxlog/xfs_extfree_item.h create mode 100644 libxlog/xfs_icreate_item.c create mode 100644 libxlog/xfs_icreate_item.h create mode 100644 libxlog/xfs_inode_item.h create mode 100644 libxlog/xfs_inode_item_recover.c create mode 100644 libxlog/xfs_log.h create mode 100644 libxlog/xfs_log_priv.h create mode 100644 libxlog/xfs_refcount_item.c create mode 100644 libxlog/xfs_refcount_item.h create mode 100644 libxlog/xfs_rmap_item.c create mode 100644 libxlog/xfs_rmap_item.h create mode 100644 libxlog/xfs_trans_ail.c create mode 100644 libxlog/xfs_trans_priv.h
base-commit: d6d4236027e31e9a54fd3916ff130b1d41e78027
Signature · 1 line
2.47.3
PATCH 1/9 libxfs: make XBF_DONE actually mark a buffer uptodate
XBF_DONE is defined to 0 in the userspace platform header, so every "this buffer's contents are valid, don't re-read it" annotation that libxfs inherits from the kernel is silently a no-op:
bp->b_flags |= XBF_DONE; /* expands to |= 0 */
The userspace equivalent of the kernel's XBF_DONE is LIBXFS_B_UPTODATE, which is what libxfs_buf_read_map() actually tests before deciding whether a physical read can be skipped:
if (bp->b_flags & (LIBXFS_B_UPTODATE | LIBXFS_B_DIRTY)) goto ok; /* no read */
so a buffer that has been marked "done" is nevertheless re-read from disk, discarding whatever the caller had just constructed in memory.
There are three consequences today:
- xfs_btree_staging.c documents the intent it does not get:
"Mark this buffer XBF_DONE (i.e. uptodate) so that a subsequent
xfs_buf_read will not pointlessly reread the contents from the disk."
This is currently masked because xfs_buf_delwri_queue_here() also marks the buffer dirty, and LIBXFS_B_DIRTY happens to suppress the re-read too.
- The readahead verifiers in xfs_inode_buf.c and xfs_dquot_buf.c clear XBF_DONE to force a subsequent real read through the full verifier, as their comments describe. Those clears are no-ops as well.
- __xfs_buf_mark_corrupt() asserts on a condition that can never be true:
ASSERT(bp->b_flags & XBF_DONE);
(Inert in practice only because libxfs/Makefile builds with -DNDEBUG unconditionally.)
The path that motivated this, xfs_ialloc_inode_init() marking a freshly initialised inode chunk uptodate, is only reachable from the kernel's xfs_icreate_item.c, which libxfs does not carry yet; it becomes live as soon as log recovery is brought into libxfs.
Define XBF_DONE as LIBXFS_B_UPTODATE so the flag means what its name and all of its users say it means.
No functional change was observed: xfs_repair -n output and post-repair superblock counters are unchanged across an 8-image corpus, and mkfs.xfs superblock geometry is identical.
--- libxfs/xfs_platform.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/xfs_platform.h +1 -1
@@ -347,7 +347,7 @@ static inline unsigned long long mask64_if_power2(unsigned long b)/* buffer management */ #define XBF_TRYLOCK 0 -#define XBF_DONE 0 +#define XBF_DONE LIBXFS_B_UPTODATE#define xfs_buf_stale (bp) ((bp)->b_flags |= LIBXFS_B_STALE)#define XFS_BUF_UNDELAYWRITE (bp) ((bp)->b_flags &= ~LIBXFS_B_DIRTY)
Signature · 1 line
2.47.3
Re: [PATCH 1/9] libxfs: make XBF_DONE actually mark a buffer uptodate
Note that the current libxfs 7.2 sync (hopefully about to be merged ASAP) actually kills XBF_DONE, so this will need a bit of a redo.
PATCH 2/9 libxfs: don't corrupt a delwri list when a buffer is queued twice
In reply to Chris Wedgwoodxfs_buf_delwri_queue() unconditionally adds the buffer to the list:
xfs_buf_hold(bp); list_add_tail(&bp->b_list, buffer_list);
list_add_tail() on a node that is already linked does not move the node, it relinks it: the entries between its old position and the tail are spliced out of the list and are never submitted. Queueing A, B and C and then queueing A again leaves a list containing just A; B and C are silently dropped and their contents are never written. Each redundant queue also leaks a reference.
The kernel avoids this with the _XBF_DELWRI_Q flag. There is no such flag in userspace, but list membership is an accurate substitute here because both xfs_buf_delwri_submit() and xfs_buf_delwri_cancel() remove entries with list_del_init(), so a buffer that is not on a list always has an empty b_list.
That is only true once b_list is actually initialised. Buffers are allocated with kmem_cache_zalloc() and __initbuf() initialises b_li_list but not b_list, leaving b_list.next NULL, for which list_empty() is false. Initialise b_list alongside b_li_list so the new test means what it says. This also makes the existing
ASSERT(list_empty(&bp->b_list));
in xfs_buf_delwri_queue_here() true for a freshly allocated buffer; it would have fired on every such buffer had assertions been enabled (libxfs/Makefile builds with -DNDEBUG unconditionally).
No caller in the tree queues a buffer twice today, so there is no user-visible change: xfs_repair -n output and post-repair superblock counters are unchanged across an 8-image corpus and mkfs.xfs superblock geometry is identical. Log recovery, which queues a buffer once per log item that touches it, does rely on this.
--- libxfs/libxfs_io.h | 12 ++++++++++++ libxfs/rdwr.c | 1 + 2 files changed, 13 insertions(+)
libxfs/libxfs_io.h +12 -0
@@ -264,6 +264,18 @@ int libxfs_buf_read_uncached(struct xfs_buftarg *targ, xfs_daddr_t daddr,static inline bool xfs_buf_delwri_queue (struct xfs_buf *bp,struct list_head *buffer_list) { +/* +* If the buffer is already on a delwri list, leave it where it is. +* list_add_tail() on an already-linked node relinks it at the tail +* and splices every entry between its old position and the tail out +* of the list, silently dropping those buffers from the submission. +* The kernel guards this with _XBF_DELWRI_Q; here list membership is +* an accurate predicate because both xfs_buf_delwri_submit() and +* xfs_buf_delwri_cancel() remove entries with list_del_init(). +*/ +if (!list_empty (&bp->b_list)) +return false; +xfs_buf_hold (bp);list_add_tail (&bp->b_list, buffer_list);return true;
libxfs/rdwr.c +1 -0
@@ -261,6 +261,7 @@ __initbuf(struct xfs_buf *bp, struct xfs_buftarg *btp, xfs_daddr_t bno, bp->b_recur =0 ; bp->b_ops =NULL ;INIT_LIST_HEAD (&bp->b_li_list); +INIT_LIST_HEAD (&bp->b_list);if (!bp->b_maps) bp->b_maps = &bp->__b_map;
Signature · 1 line
2.47.3
Re: [PATCH 2/9] libxfs: don't corrupt a delwri list when a buffer is queued twice
Quoted 5 lines · Chris Wedgwood
The kernel avoids this with the
_XBF_DELWRI_Qflag. There is no such flag in userspace, but list membership is an accurate substitute here because bothxfs_buf_delwri_submit()andxfs_buf_delwri_cancel()remove entries withlist_del_init(), so a buffer that is not on a list always has an emptyb_list.
We could add the flag and make the code closer to the kernel, though. Did you consider that, and if so what speaks against it?
PATCH 3/9 libxfs: record a failed buffer write when it fails
In reply to Chris Wedgwoodlibxfs_flush_mount() decides whether metadata reached the disk by looking at XFS_BUFTARG_LOST_WRITE and XFS_BUFTARG_CORRUPT_WRITE, and its comment says a buffer that cannot be written sets them. It does not. The flags are only set from libxfs_buf_prepare_mru(), that is, when a still-dirty buffer is later released to the free list. A write that fails during a cache flush leaves the buffer dirty and errored in the cache, and cache_flush() discards what libxfs_bflush() returns, so nothing records the failure and libxfs_flush_mount() returns success.
No existing caller can tell: mkfs.xfs and xfs_repair both detect a failing write through their own error paths first, so today this is a latent contract violation rather than a visible bug.
It stops being latent with log replay. Replay flushes the metadata it has applied and then retires the log, and the flush result is what says the retirement is safe. Believing a flush that did not happen destroys the only remaining copy of that metadata.
Set the flags where the failure is detected. Both failure exits, the I/O error and the write verifier, go through one helper so the promise holds however the write failed. The stale-buffer exit is left alone: it reports a caller bug rather than lost data, and has always done so.
Measured with an LD_PRELOAD shim that fails pwrite() after a chosen number of calls, replaying a log whose recovery needs 558 writes. Failing from write 181, which lands in the flush after replay:
before: no error reported, log retired, reads CLEAN afterwards
after: "Failed to flush replayed metadata to disk", log left DIRTY
Sweeping the failure point across the whole run shows no case where a write fails and xfs_repair still reports success.
--- libxfs/rdwr.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-)
libxfs/rdwr.c +23 -1
@@ -832,6 +832,26 @@ __write_buf(int fd, void *buf, int len, off_t offset, int flags)return 0 ; } +/* +* Record that a buffer's dirty contents did not reach the disk. +* +* libxfs_flush_mount() decides whether metadata is durable from these flags +* rather than from what libxfs_bwrite() returns, because cache_flush() +* discards the return value. Every failure exit that leaves dirty data +* unwritten therefore has to come through here, or the caller is told the +* flush succeeded. xfs_repair -R retires the log on the strength of that +* answer. +*/ +static int +libxfs_bwrite_failed ( +struct xfs_buf *bp) +{ +if (bp->b_error == -EFSCORRUPTED ) + bp->b_target->flags |=XFS_BUFTARG_CORRUPT_WRITE ; + bp->b_target->flags |=XFS_BUFTARG_LOST_WRITE ; +return bp->b_error; +} +int libxfs_bwrite (struct xfs_buf *bp)
@@ -867,7 +887,7 @@ libxfs_bwrite( __func__, bp->b_ops->name, (unsigned long long )xfs_buf_daddr (bp), bp->b_length); -return bp->b_error; +return libxfs_bwrite_failed (bp); } }
@@ -899,6 +919,8 @@ libxfs_bwrite( __func__, bp->b_ops ? bp->b_ops->name :"(unknown)" , (unsigned long long )xfs_buf_daddr (bp), bp->b_length, -bp->b_error); + +libxfs_bwrite_failed (bp); }else { bp->b_flags |=LIBXFS_B_UPTODATE ; bp->b_flags &= ~(LIBXFS_B_DIRTY |LIBXFS_B_UNCHECKED );
Signature · 1 line
2.47.3
Re: [PATCH 3/9] libxfs: record a failed buffer write when it fails
Quoted 18 lines · Chris Wedgwood
libxfs_flush_mount()decides whether metadata reached the disk by looking atXFS_BUFTARG_LOST_WRITEandXFS_BUFTARG_CORRUPT_WRITE, and its comment says a buffer that cannot be written sets them. It does not. The flags are only set fromlibxfs_buf_prepare_mru(), that is, when a still-dirty buffer is later released to the free list. A write that fails during a cache flush leaves the buffer dirty and errored in the cache, andcache_flush()discards whatlibxfs_bflush()returns, so nothing records the failure andlibxfs_flush_mount()returns success.No existing caller can tell: mkfs.xfs and
xfs_repairboth detect a failing write through their own error paths first, so today this is a latent contract violation rather than a visible bug.It stops being latent with log replay. Replay flushes the metadata it has applied and then retires the log, and the flush result is what says the retirement is safe. Believing a flush that did not happen destroys the only remaining copy of that metadata.
Quoted 8 lines
Set the flags where the failure is detected. Both failure exits, the I/O error and the write verifier, go through one helper so the promise holds however the write failed. The stale-buffer exit is left alone: it reports a caller bug rather than lost data, and has always done so.
Measured with an
LD_PRELOADshim that failspwrite()after a chosen number of calls, replaying a log whose recovery needs 558 writes. Failing from write 181, which lands in the flush after replay:
Can you wrire this up in xfstests? Maybe using an environment variable instead of LD_PRELOAD like some of the other error injection we do in userspace if that is easier to maintain.
PATCH 4/9 libxlog: rename xfs_log_recover.c to logscan.c
In reply to Chris WedgwoodThis file is xfs_logprint's log scanner and the code repair uses to find and zero a log: xlog_get_bp(), xlog_bread(), xlog_find_tail() and friends. It is userspace code with no kernel counterpart, and it does not perform recovery.
The kernel has its own fs/xfs/xfs_log_recover.c which does, and which a later patch brings into this directory. Give the userspace scanner a name that says what it is, so the kernel file can keep the name it has upstream.
Pure rename; no code change.
---
libxlog/Makefile | 2 +-
libxlog/{xfs_log_recover.c => logscan.c} | 0
2 files changed, 1 insertion(+), 1 deletion(-)
rename libxlog/{xfs_log_recover.c => logscan.c} (100%)libxlog/Makefile +1 -1
@@ -12,7 +12,7 @@ LT_AGE = 0 # we need a static build even if --disable-static is specified LTLDFLAGS += -static -CFILES = xfs_log_recover.c util.c +CFILES = logscan.c util.c # don't want to link xfs_repair with a debug libxlog. DEBUG = -DNDEBUG
libxlog/logscan.c renamed from libxlog/xfs_log_recover.c +0 -0
Signature · 1 line
2.47.3
Re: [PATCH 4/9] libxlog: rename xfs_log_recover.c to logscan.c
Quoted 9 lines · Chris Wedgwood
This file is xfs_logprint's log scanner and the code repair uses to find and zero a log:
xlog_get_bp(),xlog_bread(),xlog_find_tail()and friends. It is userspace code with no kernel counterpart, and it does not perform recovery.The kernel has its own
fs/xfs/xfs_log_recover.cwhich does, and which a later patch brings into this directory. Give the userspace scanner a name that says what it is, so the kernel file can keep the name it has upstream.
Some of the code is obviously copied from the kernel xfs_log_recover.c. Did you write this commit log yourself? Some of this reads rather AI generated, although probably by something less insane than Claude..
PATCH 5/9 libxlog: import the kernel's log recovery, log items and AIL
In reply to Chris Wedgwoodxfs_repair cannot replay a dirty log. The code that can is in the kernel, and xfsprogs already carries kernel code rather than reimplementing it, so import it.
libxfs/ mirrors the kernel's fs/xfs/libxfs/ and these files are not from there; they are from the top level of fs/xfs. They go in libxlog/, which is where this tree keeps log code, so that each directory has one kernel counterpart and tools/libxfs-diff can check both.
The files are copied from Linux v7.1 and are byte-identical to their kernel counterparts apart from their #include lists, which is the adaptation libxfs has always used. v7.1 is the kernel this tree's libxfs/ is currently in sync with: comparing xfsprogs libxfs/ against v7.1 reports no difference in any of 102 files, while against v7.2 it reports 33. Importing from a newer kernel than libxfs/ is synced to would mix two kernel versions in one tree.
Nothing references these files yet and they are not in the build. The next patch wires them up. Splitting it this way keeps the reviewable change separate from fourteen thousand lines that should be read as "the kernel's code, unchanged".
--- include/kernel_compat.h | 231 ++ libxlog/xfs_attr_item.c | 1206 ++++++++++ libxlog/xfs_attr_item.h | 64 + libxlog/xfs_bmap_item.c | 718 ++++++ libxlog/xfs_bmap_item.h | 78 + libxlog/xfs_buf_item.h | 71 + libxlog/xfs_buf_item_recover.c | 1215 ++++++++++ libxlog/xfs_dquot_item_recover.c | 214 ++ libxlog/xfs_exchmaps_item.c | 608 +++++ libxlog/xfs_exchmaps_item.h | 64 + libxlog/xfs_extfree_item.c | 1026 +++++++++ libxlog/xfs_extfree_item.h | 100 + libxlog/xfs_icreate_item.c | 260 +++ libxlog/xfs_icreate_item.h | 22 + libxlog/xfs_inode_item.h | 62 + libxlog/xfs_inode_item_recover.c | 602 +++++ libxlog/xfs_log.h | 147 ++ libxlog/xfs_log_priv.h | 746 +++++++ libxlog/xfs_log_recover.c | 3575 ++++++++++++++++++++++++++++++ libxlog/xfs_refcount_item.c | 861 +++++++ libxlog/xfs_refcount_item.h | 82 + libxlog/xfs_rmap_item.c | 890 ++++++++ libxlog/xfs_rmap_item.h | 81 + libxlog/xfs_trans_ail.c | 978 ++++++++ libxlog/xfs_trans_priv.h | 170 ++ 25 files changed, 14071 insertions(+) create mode 100644 include/kernel_compat.h create mode 100644 libxlog/xfs_attr_item.c create mode 100644 libxlog/xfs_attr_item.h create mode 100644 libxlog/xfs_bmap_item.c create mode 100644 libxlog/xfs_bmap_item.h create mode 100644 libxlog/xfs_buf_item.h create mode 100644 libxlog/xfs_buf_item_recover.c create mode 100644 libxlog/xfs_dquot_item_recover.c create mode 100644 libxlog/xfs_exchmaps_item.c create mode 100644 libxlog/xfs_exchmaps_item.h create mode 100644 libxlog/xfs_extfree_item.c create mode 100644 libxlog/xfs_extfree_item.h create mode 100644 libxlog/xfs_icreate_item.c create mode 100644 libxlog/xfs_icreate_item.h create mode 100644 libxlog/xfs_inode_item.h create mode 100644 libxlog/xfs_inode_item_recover.c create mode 100644 libxlog/xfs_log.h create mode 100644 libxlog/xfs_log_priv.h create mode 100644 libxlog/xfs_log_recover.c create mode 100644 libxlog/xfs_refcount_item.c create mode 100644 libxlog/xfs_refcount_item.h create mode 100644 libxlog/xfs_rmap_item.c create mode 100644 libxlog/xfs_rmap_item.h create mode 100644 libxlog/xfs_trans_ail.c create mode 100644 libxlog/xfs_trans_priv.h
include/kernel_compat.h new file +231 -0
@@ -0,0 +1,231 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Kernel infrastructure that xfs kernel sources refer to but which has no +* userspace equivalent. +* +* libxfs adapts kernel sources by rewriting their include list and nothing +* else, so the annotations, types and helpers those sources use have to be +* supplied from outside them. Concurrency primitives collapse to no-ops +* because libxfs is single-threaded with respect to the log, and the +* deferred-work and I/O types only ever appear as struct members that +* userspace never schedules or submits. +* +* This header is included both by libxfs internals (via xfs_platform.h) and +* by the tools (via libxfs.h), because struct xlog embeds several of these +* types and the tools inspect struct xlog directly. +*/ +#ifndef __LIBXFS_KERNEL_COMPAT_H__ +#define __LIBXFS_KERNEL_COMPAT_H__ + +#ifndef ____cacheline_aligned_in_smp +#define ____cacheline_aligned_in_smp +#endif + +/* sparse annotations */ +#define __releases (x) +#define __percpu + +typedef uint32_t u32 ; + +struct work_struct {void *unused; }; +struct delayed_work {void *unused; }; + +/* +* Types used only by the log write path (iclogs and the CIL). libxfs +* replays a log but never writes one, so these only ever need to occupy +* space in a struct; nothing dereferences them. +*/ +struct semaphore {void *unused; }; +struct rw_semaphore {void *unused; }; +struct bio {void *unused; }; +struct bio_vec {void *unused; }; +struct cpumask {void *unused; }; +struct xfs_busy_extents {void *unused; }; +struct xfs_kobj {void *unused; }; + +/* +* xlog_wait() is on the log write path, which userspace never takes. The +* task argument of DECLARE_WAITQUEUE() is unused here, so "current" itself +* is deliberately not defined - it is far too common an identifier to +* introduce as a macro into every tool that includes this header. +*/ +#define DECLARE_WAITQUEUE (name, task) int name __attribute__((unused)) +#define add_wait_queue_exclusive (wq, w) ((void)0) +#define remove_wait_queue (wq, w) ((void)0) +#define __set_current_state (state) ((void)0) +#define set_current_state (state) ((void)0) +#define TASK_UNINTERRUPTIBLE 0 + +/* +* Function-like, so it only expands where it is actually called; unlike +* an object-like "current" this cannot capture unrelated identifiers. +*/ +#define schedule () ((void)0) + +#define wait_var_event (var, cond) ((void)0) + +/* +* The AIL's pusher (xfsaild) is a kernel thread. libxfs walks and +* updates the AIL directly and never starts a pusher, so the thread, +* freezer, scheduling and waitqueue primitives it uses are defined away +* here; the functions that call them compile but are not reachable from +* libxfs or the tools. +* +* NOTE: xfs_ail_push_all_sync() spins until the AIL drains, waking a +* pusher that does not exist in userspace, so it would not terminate if +* it were ever called. Nothing in log recovery calls it - the callers +* are all in xfs_icache.c, xfs_log.c and xfs_mount.c, none of which +* libxfs carries - but do not introduce a caller without giving the AIL +* a synchronous push first. +*/ +#define kthread_run (fn, data, fmt, ...) ((void *)NULL) +#define kthread_should_stop () (1) +#define kthread_stop (tsk) ((void)0) +#define set_freezable () ((void)0) +#define try_to_freeze () ((void)0) +#define memalloc_noreclaim_save () (0) +#define memalloc_noreclaim_restore (f) ((void)(f)) +#define msecs_to_jiffies (m) (m) +#define schedule_timeout (t) ((void)(t)) +#define init_waitqueue_head (wq) ((void)0) +#define waitqueue_active (wq) (0) +#define wake_up_all (wq) ((void)0) +#define wake_up_process (tsk) ((void)0) +#define DEFINE_WAIT (name) int name __attribute__((unused)) +#define prepare_to_wait (wq, w, state) ((void)0) +#define finish_wait (wq, w) ((void)0) + +#define assert_spin_locked (l) ((void)0) +#define lockdep_assert_held (l) ((void)0) +#define list_empty_careful (l) list_empty(l) + +/* xfs_alert_tag() drops the tag; libxfs has no per-tag panic mask. */ +#define XFS_PTAG_AILDELETE 0 +#define xfs_alert_tag (mp, tag, fmt, ...) xfs_alert(mp, fmt, ##__VA_ARGS__) + +/* libxfs submits delwri lists synchronously; there is no async variant. */ +#define xfs_buf_delwri_submit_nowait (list) xfs_buf_delwri_submit(list) + +/* +* libxfs holds no buffer locks across threads the way the kernel does, so +* a trylock always succeeds. +* +* xfs_log_space_wake() and xlog_force_shutdown() are declared by the +* kernel's xfs_log.h and implemented for userspace in libxfs/logitem.c. +*/ +#define xfs_buf_trylock (bp) (xfs_buf_lock(bp), true) + + +/* +* The kernel's refcount_t is a saturating wrapper around atomic_t with +* use-after-free detection. libxfs is single-threaded with respect to +* these objects, so map it onto the atomic API it wraps. +*/ +typedef atomic_t refcount_t ; + +#define refcount_set (r, n) atomic_set((r), (n)) +#define refcount_inc_not_zero (r) atomic_inc_not_zero(r) +#define refcount_dec_and_test (r) atomic_dec_and_test(r) + +/* Zoned realtime devices and logged xattrs are kernel-only features. */ +#define xfs_zone_free_blocks (tp, rtg, rtbno, len) (-EOPNOTSUPP) +/* +* The kernel tracks this as a mount opstate set when logged xattrs are +* enabled. libxfs has no mount options, and a log that contains ATTRI +* items is itself proof that the filesystem was using them, so replay must +* accept them. Deliberately true rather than accidentally so. +*/ +#define xfs_is_using_logged_xattrs (mp) (true) + + +/* NOFS allocation scoping and sleeping have no userspace meaning. */ +#define memalloc_nofs_save () (0) +#define memalloc_nofs_restore (f) ((void)(f)) +#define msleep (ms) ((void)0) + + +/* Block layer operation type, for the raw device I/O log recovery does. */ +typedef uint64_t sector_t ; + +enum req_op { +REQ_OP_READ , +REQ_OP_WRITE , +}; + +int xfs_rw_bdev (dev_t bdev,sector_t sector,unsigned int count,char *data, +enum req_op op); +void libxfs_bdev_register (dev_t dev,int fd); + +/* +* Userspace has no inode cache to flush, no read-only device checks and no +* COW recovery, and never leaves the log dirty behind it. +*/ +#define xfs_inodegc_flush (mp) (0) +#define xfs_dev_is_read_only (mp, msg) (0) +#define xfs_readonly_buftarg (btp) (false) +#define xfs_reflink_recover_cow (mp) (0) +#define xfs_set_clean (mp) ((void)0) +#define xfs_notice_once (mp, fmt, ...) xfs_notice(mp, fmt, ##__VA_ARGS__) +#define xfs_buf_rele (bp) libxfs_buf_rele(bp) + + +/* +* Odds and ends the recovery sources reach for. +* +* m_logname and m_sb_bp are VFS-side mount state; libxfs keeps neither, so +* the log device has no name to print and the superblock buffer is read on +* demand. m_logbsize is the in-core log buffer size, which userspace +* derives from the on-disk geometry. +*/ +/* +* libxfs splits the read from the verification that the kernel does in one +* step, so do both. The only caller is xlog_do_recover(), which re-reads the +* superblock after replay and whose own comment says it reverifies it; without +* the verifier a superblock that replay corrupted would be copied straight +* into mp->m_sb and repair would run against it. +*/ +#define _xfs_buf_read (bp) \ +({ \ + int __err = libxfs_readbufr((bp)->b_target, \ + xfs_buf_daddr(bp), (bp), \ + (bp)->b_length, 0); \ + __err ? __err : libxfs_readbuf_verify((bp), (bp)->b_ops); \ +}) + +#define xfs_iflags_clear (ip, flags) ((void)0) +#define XFS_IRECOVERY 0 + +#define SHUTDOWN_LOG_IO_ERROR 0x2 +#define SHUTDOWN_CORRUPT_INCORE 0x8 + +/* Error injection knobs; userspace has none. */ +static const struct { +int log_recovery_delay; +int mount_delay; +int always_cow; +int pwork_threads; +int larp; +} xfs_globals; + + +/* libxfs_iget() has no lock_flags argument. */ +#define xfs_iget (mp, tp, ino, flags, lock_flags, ipp) \ + libxfs_iget((mp), (tp), (ino), (flags), (ipp)) + +/* libuuid's uuid_is_null() takes the array, not a pointer to it. */ +#define uuid_is_null (u) platform_uuid_is_null(u) + +/* +* libxfs takes no inode locks, but the kernel's xfs_exchrange_ilock() also +* joins the inodes to the transaction, which is load bearing. Keep that +* half and drop only the locking. A real function rather than a macro so +* the callers' locals do not look conditionally used. +*/ +struct xfs_trans ; +struct xfs_inode ; +void xfs_exchrange_ilock (struct xfs_trans *tp,struct xfs_inode *ip1, +struct xfs_inode *ip2); + +#define xfs_exchrange_iunlock (ip1, ip2) ((void)0) + +#endif /* __LIBXFS_KERNEL_COMPAT_H__ */
libxlog/xfs_attr_item.c new file +1206 -0
1,206 lines not on this page · raw message
libxlog/xfs_attr_item.h new file +64 -0
64 lines not on this page · raw message
libxlog/xfs_bmap_item.c new file +718 -0
718 lines not on this page · raw message
libxlog/xfs_bmap_item.h new file +78 -0
78 lines not on this page · raw message
libxlog/xfs_buf_item.h new file +71 -0
71 lines not on this page · raw message
libxlog/xfs_buf_item_recover.c new file +1215 -0
632 lines not on this page · raw message
libxlog/xfs_dquot_item_recover.c new file +214 -0
214 lines not on this page · raw message
libxlog/xfs_exchmaps_item.c new file +608 -0
608 lines not on this page · raw message
libxlog/xfs_exchmaps_item.h new file +64 -0
64 lines not on this page · raw message
libxlog/xfs_extfree_item.c new file +1026 -0
1,026 lines not on this page · raw message
libxlog/xfs_extfree_item.h new file +100 -0
100 lines not on this page · raw message
libxlog/xfs_icreate_item.c new file +260 -0
260 lines not on this page · raw message
libxlog/xfs_inode_item.h new file +62 -0
62 lines not on this page · raw message
libxlog/xfs_inode_item_recover.c new file +602 -0
602 lines not on this page · raw message
libxlog/xfs_log.h new file +147 -0
147 lines not on this page · raw message
libxlog/xfs_log_priv.h new file +746 -0
746 lines not on this page · raw message
libxlog/xfs_log_recover.c new file +3575 -0
3,575 lines not on this page · raw message
libxlog/xfs_refcount_item.c new file +861 -0
861 lines not on this page · raw message
libxlog/xfs_refcount_item.h new file +82 -0
82 lines not on this page · raw message
libxlog/xfs_rmap_item.c new file +890 -0
890 lines not on this page · raw message
libxlog/xfs_rmap_item.h new file +81 -0
81 lines not on this page · raw message
libxlog/xfs_trans_ail.c new file +978 -0
978 lines not on this page · raw message
libxlog/xfs_trans_priv.h new file +170 -0
170 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.47.3
Part of the diff is not shown here. View the raw message.
Re: [PATCH 5/9] libxlog: import the kernel's log recovery, log items and AIL
Quoted 9 lines · Chris Wedgwood
xfs_repaircannot replay a dirty log. The code that can is in the kernel, and xfsprogs already carries kernel code rather than reimplementing it, so import it.libxfs/ mirrors the kernel's
fs/xfs/libxfs/and these files are not from there; they are from the top level of fs/xfs. They go in libxlog/, which is where this tree keeps log code, so that each directory has one kernel counterpart andtools/libxfs-diffcan check both.
Quoted 5 lines
The files are copied from Linux v7.1 and are byte-identical to their kernel counterparts apart from their #include lists, which is the adaptation libxfs has always used. v7.1 is the kernel this tree's libxfs/ is currently in sync with: comparing xfsprogs libxfs/ against v7.1 reports no difference in any of 102 files, while against v7.2 it
As in changes except for the include file mess ignored by libxfs-diff? Because otherwise we should have a lot more..
Quoted 2 lines
reports 33. Importing from a newer kernel than libxfs/ is synced to would mix two kernel versions in one tree.
We're about to merge the 7.2 merge. Either way we'll need to coordinate them.
Quoted 11 lines
include/kernel_compat.h new file +5
-0@@ -0,0 +1,231 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Kernel infrastructure that xfs kernel sources refer to but which has no +* userspace equivalent. +*
include/kernel_compat.h line 15
Quoted 10 lines
include/kernel_compat.h new file +10
-0+* libxfs adapts kernel sources by rewriting their include list and nothing +* else, so the annotations, types and helpers those sources use have to be +* supplied from outside them. Concurrency primitives collapse to no-ops +* because libxfs is single-threaded with respect to the log, and the +* deferred-work and I/O types only ever appear as struct members that +* userspace never schedules or submits. +* +* This header is included both by libxfs internals (via xfs_platform.h) and +* by the tools (via libxfs.h), because struct xlog embeds several of these +* types and the tools inspect struct xlog directly.
Please split this out into a separate patch, and make sure all the other kernel compat bits are consistent with this.
Also usually kernel files need some amount of changes
Re: [PATCH 5/9] libxlog: import the kernel's log recovery, log items and AIL
In reply to Chris WedgwoodQuoted 9 lines · Chris Wedgwood
xfs_repaircannot replay a dirty log. The code that can is in the kernel, and xfsprogs already carries kernel code rather than reimplementing it, so import it.libxfs/ mirrors the kernel's
fs/xfs/libxfs/and these files are not from there; they are from the top level of fs/xfs. They go in libxlog/, which is where this tree keeps log code, so that each directory has one kernel counterpart andtools/libxfs-diffcan check both.
Quoted 7 lines
The files are copied from Linux v7.1 and are byte-identical to their kernel counterparts apart from their #include lists, which is the adaptation libxfs has always used. v7.1 is the kernel this tree's libxfs/ is currently in sync with: comparing xfsprogs libxfs/ against v7.1 reports no difference in any of 102 files, while against v7.2 it reports 33. Importing from a newer kernel than libxfs/ is synced to would mix two kernel versions in one tree.
This is not a direct code import - it mixed modification with code copying.
Call it intuition, but the first thing I looked for was xlog_write()
- the function that writes new log records to disk. I immediately notices that the kernel compat header neuters all the xlog_wait() meaning iclogs do not work. Hence I wondered how intent replay is writing to the journal....
Yup, as I suspected, the iclog code has been removed from the journal IO path. i.e. there's a heap of custom code buffer writing code that is most definitely not the same as the kernel code. This path is critical for correctness, and I have little confidence a massive rewrite like this gets it right the first go.
Hence this whole patchset needs to seperate out the "lift to userspace" file copies from the "modify for userspace" code changes so that we can sanely review the actual code changes that matter.
This means -a lot more work for you-; these patches need to be broken down into much smaller chunks that we can actually review; a 14000 line patch that mixes kernel code with custom modifications is not reviewable by anyone, not even a frontier LLM.
I'm not going to look at this in any more detail other than the cursory scan I've already simply because it is impossible to find all the changes that need careful review in this massive code dump....
If the next posting of this series isn't at least 50+ patches, then it probably still isn't fine grained enough to review effectively....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 6/9 libxfs-diff: also compare libxlog against the kernel
In reply to Chris Wedgwoodlibxfs/ mirrors the kernel's fs/xfs/libxfs/, and the tool checks that. libxlog/ carries files taken from the top level of fs/xfs, and nothing checked those, so a divergence there was invisible.
State the second mapping explicitly rather than searching for a matching name, so each directory has one declared kernel counterpart.
--- tools/libxfs-diff | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/libxfs-diff +11 -0
@@ -20,7 +20,18 @@ fi dir="$(readlink -m "${dir}/..")" +# libxfs/ mirrors fs/xfs/libxfs/; libxlog/ mirrors the top level of fs/xfs. for i in libxfs/xfs*.[ch]; do kfile="${dir}/$i" diff -Naurpw --label "$i" <(sed -e '/#include/d' "$i") --label "${kfile}" <(sed -e '/#include/d' "${kfile}") done + +for i in libxlog/xfs*.[ch]; do + test -e "$i" || continue + kfile="${dir}/$(basename "$i")" + if ! test -e "${kfile}"; then + echo "warning: no kernel source for $i" >&2 + continue + fi + diff -Naurpw --label "$i" <(sed -e '/#include/d' "$i") --label "${kfile}" <(sed -e '/#include/d' "${kfile}") +done
Signature · 1 line
2.47.3
Re: [PATCH 6/9] libxfs-diff: also compare libxlog against the kernel
Quoted 3 lines · Chris Wedgwood
libxfs/ mirrors the kernel's
fs/xfs/libxfs/, and the tool checks that. libxlog/ carries files taken from the top level of fs/xfs, and nothing checked those, so a divergence there was invisible.
Yeah, I recently ran into that as well recently.
Quoted 2 lines
State the second mapping explicitly rather than searching for a matching name, so each directory has one declared kernel counterpart.
What I wonder then is why we don't actually have this code in libxfs. I didn't get to investigate if there is a good reason for that, but if we could move the files to libxfs life would be much easier.
Re: [PATCH 6/9] libxfs-diff: also compare libxlog against the kernel
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Chris Wedgwood
libxfs/ mirrors the kernel's
fs/xfs/libxfs/, and the tool checks that. libxlog/ carries files taken from the top level of fs/xfs, and nothing checked those, so a divergence there was invisible.
Quoted 8 lines
Yeah, I recently ran into that as well recently.
Quoted 2 lines
State the second mapping explicitly rather than searching for a matching name, so each directory has one declared kernel counterpart.
What I wonder then is why we don't actually have this code in libxfs. I didn't get to investigate if there is a good reason for that, but if we could move the files to libxfs life would be much easier.
I've long wondered if the kernel logging code should all move to fs/xfs/libxlog/ and the userspace logging code to libxlog/ to make this easier?
--D
Re: [PATCH 6/9] libxfs-diff: also compare libxlog against the kernel
Quoted 11 lines · Darrick J. Wong
Quoted 10 lines · Christoph Hellwig
Quoted 3 lines · Chris Wedgwood
libxfs/ mirrors the kernel's
fs/xfs/libxfs/, and the tool checks that. libxlog/ carries files taken from the top level of fs/xfs, and nothing checked those, so a divergence there was invisible.Yeah, I recently ran into that as well recently.
Quoted 2 lines
State the second mapping explicitly rather than searching for a matching name, so each directory has one declared kernel counterpart.
Quoted 7 lines
Quoted 3 lines
What I wonder then is why we don't actually have this code in libxfs. I didn't get to investigate if there is a good reason for that, but if we could move the files to libxfs life would be much easier.
I've long wondered if the kernel logging code should all move to
fs/xfs/libxlog/and the userspace logging code to libxlog/ to make this easier?
That was the intention I had over a decade ago when I created the shared libxfs/ codebase infrastructure. It never happened because the only thing that used libxlog (logprint) really never needed to be kept in sync with the kernel code as libxlog could (and still can) resolve the head and tail as well as log and ophdr structures perfectly fine wihtout needing to be sync'd to the kernel.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 7/9 libxlog: build the imported kernel code
In reply to Chris WedgwoodWire the imported sources into the build and give them the userspace they expect.
The kernel facilities they refer to - atomics, spinlocks, completions, allocation wrappers, hlists, trace points - are provided rather than edited out, so the sources stay verbatim and future resyncs stay mechanical. They live in a new kernel_compat.h, with libxfs's platform header extended where a shim has to sit beside existing userspace definitions.
The imported sources are compiled into libxfs rather than into libxlog, via vpath, even though they live in libxlog/. They have to be: libxfs code refers to them - xfs_defer.c needs the deferred operation types, xfs_attr.c needs xfs_attr_defer_add(), xfs_trans_resv.c needs the log space helpers - while they in turn refer to libxfs. In the kernel both halves are one module and the question does not arise. Here, two archives with references in both directions cannot be linked in any order, and libtool offers no way to express a link group. One archive is the only arrangement that works.
libxfs's own log item layer is replaced by the kernel's. That removes defer_item.c, whose intent handling the imported items now do properly, and aligns the log item and mount structures with the kernel so the imported code sees the fields it expects. libxlog adopts the kernel's struct xlog for the same reason; xfs_repair and the logprint and db callers follow it.
No behavioural change: xfs_repair still refuses a dirty log.
--- db/logformat.c | 6 +- include/Makefile | 1 + include/atomic.h | 2 + include/builddefs.in | 1 + include/hlist.h | 39 +- include/kmem.h | 19 + include/libxfs.h | 2 +- include/libxlog.h | 39 +- include/list.h | 7 + include/platform_defs.h | 11 +- include/spinlock.h | 18 +- include/xfs_inode.h | 3 + include/xfs_mount.h | 8 +- include/xfs_trace.h | 30 ++ include/xfs_trans.h | 118 +++-- libfrog/bitmask.h | 17 +- libfrog/div64.h | 7 + libxfs/Makefile | 19 +- libxfs/defer_item.c | 995 --------------------------------------- libxfs/defer_item.h | 56 --- libxfs/init.c | 73 +++ libxfs/inode.c | 43 ++ libxfs/libxfs_io.h | 3 + libxfs/logitem.c | 204 +++++++- libxfs/rdwr.c | 158 ++++++- libxfs/trans.c | 34 +- libxfs/util.c | 8 +- libxfs/xfs_alloc.c | 2 +- libxfs/xfs_attr.c | 2 +- libxfs/xfs_bmap.c | 2 +- libxfs/xfs_defer.c | 4 + libxfs/xfs_exchmaps.c | 2 +- libxfs/xfs_ialloc.c | 3 + libxfs/xfs_parent.c | 2 +- libxfs/xfs_platform.h | 19 +- libxfs/xfs_refcount.c | 2 +- libxfs/xfs_rmap.c | 2 +- libxfs/xfs_trans_resv.c | 7 +- libxlog/Makefile | 12 + libxlog/logscan.c | 31 +- libxlog/util.c | 34 +- logprint/log_misc.c | 4 +- logprint/log_print_all.c | 2 +- logprint/logprint.c | 2 +- logprint/logprint.h | 2 +- repair/phase2.c | 4 +- repair/xfs_repair.c | 2 +- 47 files changed, 857 insertions(+), 1204 deletions(-) delete mode 100644 libxfs/defer_item.c delete mode 100644 libxfs/defer_item.h
db/logformat.c +5 -1
@@ -9,6 +9,10 @@#include "init.h" #include "output.h" #include "libxlog.h" +#include "xfs_extfree_item.h" +#include "xfs_rmap_item.h" +#include "xfs_refcount_item.h" +#include "xfs_bmap_item.h" #include "logformat.h" #define MAX_LSUNIT 256 * 1024/* max log buf. size */
@@ -60,7 +64,7 @@ logformat_f(int argc, char **argv)*/ memset (mp->m_log,0 ,sizeof (struct xlog )); mp->m_log->l_mp = mp; - mp->m_log->l_dev = mp->m_logdev_targp; + mp->m_log->l_targ = mp->m_logdev_targp; mp->m_log->l_logBBsize =XFS_FSB_TO_BB (mp, mp->m_sb.sb_logblocks); mp->m_log->l_logBBstart =XFS_FSB_TO_DADDR (mp, mp->m_sb.sb_logstart); mp->m_log->l_sectBBsize =BBSIZE ;
include/Makefile +1 -0
@@ -13,6 +13,7 @@ LIBHFILES = libxfs.h \ bitops.h \ cache.h \ hlist.h \ + kernel_compat.h \ kmem.h \ list.h \ parent.h \
include/atomic.h +2 -0
@@ -119,6 +119,8 @@ atomic64_set(atomic64_t *a, int64_t v)#endif /* HAVE_URCU_ATOMIC64 */ #define __smp_mb () cmm_smp_mb() +#define smp_rmb () cmm_smp_rmb() +#define smp_wmb () cmm_smp_wmb()/* from compiler_types.h */ /*
include/builddefs.in +1 -0
@@ -180,6 +180,7 @@ endif GCFLAGS = $(DEBUG) \ -DVERSION=\"$(PKG_VERSION)\" -DLOCALEDIR=\"$(PKG_LOCALE_DIR)\" \ -DPACKAGE=\"$(PKG_NAME)\" -I$(TOPDIR)/include -I$(TOPDIR)/libxfs \ + -I$(TOPDIR)/libxlog \ -I$(TOPDIR) ifeq ($(ENABLE_GETTEXT),yes)
include/hlist.h +35 -4
@@ -53,10 +53,41 @@ static inline void hlist_del(struct hlist_node *n)#define hlist_for_each (pos, head) \ for (pos = (head)->first; pos; pos = pos->next) -#define hlist_for_each_entry (tpos, pos, head, member) \ - for (pos = (head)->first; \ - pos && ({ tpos = hlist_entry(pos, typeof(*tpos), member); 1;}); \ - pos = pos->next) +/* +* The iteration macros below are the kernel's current (3 argument) form, +* which the xfs sources libxfs carries expect. +*/ +#define hlist_entry_safe (ptr, type, member) \ + ({ typeof(ptr) ____ptr = (ptr); \ + ____ptr ? hlist_entry(____ptr, type, member) : NULL; \ + }) + +#define hlist_for_each_entry (pos, head, member) \ + for (pos = hlist_entry_safe((head)->first, typeof(*(pos)), member);\ + pos; \ + pos = hlist_entry_safe((pos)->member.next, typeof(*(pos)), member)) + +#define hlist_for_each_entry_safe (pos, n, head, member) \ + for (pos = hlist_entry_safe((head)->first, typeof(*pos), member);\ + pos && ({ n = pos->member.next; 1; }); \ + pos = hlist_entry_safe(n, typeof(*pos), member)) + +static inline void INIT_HLIST_HEAD (struct hlist_head *h) +{ + h->first =NULL ; +} + +static inline int hlist_unhashed (const struct hlist_node *h) +{ +return !h->pprev; +} +static inline void hlist_del_init (struct hlist_node *n) +{ +if (!hlist_unhashed (n)) { +__hlist_del (n); +INIT_HLIST_NODE (n); + } +}#endif /* __LIST_H__ */
include/kmem.h +19 -0
@@ -26,6 +26,9 @@ typedef unsigned int __bitwise gfp_t;#define __GFP_NOFAIL ((__force gfp_t)0)#define __GFP_NOLOCKDEP ((__force gfp_t)0)#define __GFP_RETRY_MAYFAIL ((__force gfp_t)0) +#define __GFP_DIRECT_RECLAIM ((__force gfp_t)0) +#define __GFP_NOWARN ((__force gfp_t)0) +#define __GFP_NORETRY ((__force gfp_t)0)#define __GFP_ZERO ((__force gfp_t)1)
@@ -53,11 +56,21 @@ kmem_cache_free(struct kmem_cache *cache, void *ptr)extern void *kvmalloc (size_t ,gfp_t );extern void *krealloc (void *,size_t ,int ); +static inline void *kvrealloc (void *ptr,size_t size,gfp_t flags) +{ +return krealloc (ptr, size, flags); +} +static inline void *kmalloc (size_t size,gfp_t flags) {return kvmalloc (size, flags); } +static inline void *vmalloc (size_t size) +{ +return kvmalloc (size,0 ); +} +#define kzalloc (size, gfp) kvmalloc((size), (gfp) | __GFP_ZERO)#define kvzalloc (size, gfp) kzalloc((size), (gfp))
@@ -111,6 +124,12 @@ static inline size_t __must_check size_mul(size_t factor1, size_t factor2)#define kmalloc_obj (VAR_OR_TYPE , ...) \ __alloc_objs(kmalloc, default_gfp(__VA_ARGS__), typeof(VAR_OR_TYPE), 1) +/* ...and the array forms, where the count is given explicitly. */ +#define kzalloc_objs (P ,COUNT , ...) \ + __alloc_objs(kzalloc, default_gfp(__VA_ARGS__), typeof(P), COUNT) +#define kmalloc_objs (VAR_OR_TYPE ,COUNT , ...) \ + __alloc_objs(kmalloc, default_gfp(__VA_ARGS__), typeof(VAR_OR_TYPE), COUNT) +static inline void kfree (const void *ptr) {free ((void *)ptr);
include/libxfs.h +1 -1
@@ -29,6 +29,7 @@#include "libfrog/util.h" #include "atomic.h" #include "spinlock.h" +#include "kernel_compat.h" #include "xfs_types.h" #include "xfs_fs.h"
@@ -103,7 +104,6 @@ struct iomap;#include "xfs_rtbitmap.h" #include "xfs_rtrmap_btree.h" #include "xfs_ag_resv.h" -#include "defer_item.h" #ifndef ARRAY_SIZE #define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0]))
include/libxlog.h +14 -25
@@ -6,31 +6,12 @@#define LIBXLOG_H /* -* define the userlevel xlog_t to be the subset of the kernel's -* xlog_t that we actually need to get our work done, avoiding -* the need to define any exotic kernel types in userland. +* struct xlog is the kernel's, imported into libxfs, so that the log +* recovery sources libxfs carries and the tools that inspect a log agree +* on its layout. It used to be a hand-written subset here, which is why +* a few of the fields below are referred to by their old names in places. */ -struct xlog { -atomic64_t l_tail_lsn;/* lsn of 1st LR w/ unflush buffers */ -atomic64_t l_last_sync_lsn;/* lsn of last LR on disk */ -xfs_mount_t *l_mp;/* mount point */ -struct xfs_buftarg *l_dev;/* dev_t of log */ -xfs_daddr_t l_logBBstart;/* start block of log */ -int l_logBBsize;/* size of log in 512 byte chunks */ -int l_curr_cycle;/* Cycle number of log writes */ -int l_prev_cycle;/* Cycle # b4 last block increment */ -int l_curr_block;/* current logical block of log */ -int l_prev_block;/* previous logical block of log */ -int l_iclog_size;/* size of log in bytes */ -int l_iclog_size_log;/* log power size of log */ -int l_iclog_bufs;/* number of iclog buffers */ -atomic64_t l_grant_reserve_head; -atomic64_t l_grant_write_head; -uint l_sectbb_log;/* log2 of sector size in bbs */ -uint l_sectbb_mask;/* sector size (in BBs) -* alignment mask */ -int l_sectBBsize;/* size of log sector in 512 byte chunks */ -}; +#include "xfs_log_priv.h" #include "xfs_log_recover.h"
@@ -71,6 +52,15 @@ extern int print_record_header;void xlog_init (struct xfs_mount *mp,struct xlog *log);int xlog_is_dirty (struct xfs_mount *mp,struct xlog *log); +/* +* The kernel used to keep the LSN of the last log record on disk in +* struct xlog as l_last_sync_lsn, and dropped it once the AIL became the +* authority on that. xfs_repair still wants it, to seed the maximum +* metadata LSN it will accept, so libxlog tracks it here rather than +* carrying a field the kernel no longer has. +*/ +extern xfs_lsn_t xlog_last_sync_lsn; +extern struct xfs_buf *xlog_get_bp (struct xlog *,int );extern int xlog_bread (struct xlog *log,xfs_daddr_t blk_no,int nbblks,struct xfs_buf *bp,char **offset);
@@ -84,7 +74,6 @@ extern int xlog_find_cycle_start(struct xlog *log, struct xfs_buf *bp,extern int xlog_find_tail (struct xlog *log,xfs_daddr_t *head_blk,xfs_daddr_t *tail_blk); -extern int xlog_recover (struct xlog *log,int readonly);extern void xlog_recover_print_data (char *p,int len);extern void xlog_recover_print_logitem (struct xlog_recover_item *item);extern void xlog_recover_print_trans_head (struct xlog_recover *tr);
include/list.h +7 -0
@@ -177,6 +177,13 @@ void list_sort(void *priv, struct list_head *head, list_cmp_func_t cmp);#define ARRAY_SIZE (arr) (sizeof(arr) / sizeof((arr)[0])) +static inline void list_splice_tail (struct list_head *list, +struct list_head *head) +{ +if (!list_empty (list)) +__list_splice (list, head->prev, head); +} +/** * list_splice_tail_init - join two lists and reinitialise the emptied list * @list: the new list to add.
include/platform_defs.h +9 -2
@@ -26,8 +26,15 @@#include <urcu.h> #include <linux/blkzoned.h> -/* long and pointer must be either 32 bit or 64 bit */ -#define BITS_PER_LONG (sizeof(long) * CHAR_BIT) +/* +* long and pointer must be either 32 bit or 64 bit. +* +* Spelled with the compiler's predefined width macros rather than +* sizeof(), because kernel headers test this in the preprocessor +* (xfs_trans_priv.h has "#if BITS_PER_LONG != 64") and sizeof() is not +* a preprocessor-evaluable constant. +*/ +#define BITS_PER_LONG (__SIZEOF_LONG__ * __CHAR_BIT__)typedef unsigned short umode_t ;
include/spinlock.h +13 -5
@@ -15,12 +15,20 @@* Hence we know it works. */ -typedef pthread_mutex_t spinlock_t ; +/* +* In the kernel spinlock_t is struct spinlock, and xfs code refers to both +* spellings. Keep the same shape here so kernel sources build unmodified. +*/ +struct spinlock { +pthread_mutex_t lock; +}; + +typedef struct spinlock spinlock_t ; -#define spin_lock_init (l) pthread_mutex_init(l, NULL) -#define spin_lock (l) pthread_mutex_lock(l) -#define spin_trylock (l) (pthread_mutex_trylock(l) != EBUSY) -#define spin_unlock (l) pthread_mutex_unlock(l) +#define spin_lock_init (l) pthread_mutex_init(&(l)->lock, NULL) +#define spin_lock (l) pthread_mutex_lock(&(l)->lock) +#define spin_trylock (l) (pthread_mutex_trylock(&(l)->lock) != EBUSY) +#define spin_unlock (l) pthread_mutex_unlock(&(l)->lock)#define mutex_init (l) pthread_mutex_init(l, NULL)#define mutex_lock (l) pthread_mutex_lock(l)
include/xfs_inode.h +3 -0
@@ -431,6 +431,9 @@ int libxfs_icreate(struct xfs_trans *tp, xfs_ino_t ino, const struct xfs_icreate_args *args, struct xfs_inode **ipp);/* Inode Cache Interfaces */ +struct xfs_inode *xfs_inode_alloc (struct xfs_mount *mp,xfs_ino_t ino); +void xfs_inode_free (struct xfs_inode *ip); +extern int libxfs_iget (struct xfs_mount *,struct xfs_trans *,xfs_ino_t ,uint ,struct xfs_inode **);extern void libxfs_irele (struct xfs_inode *ip);
include/xfs_mount.h +7 -1
@@ -171,6 +171,11 @@ typedef struct xfs_mount {* if warranted. */ struct xlog *m_log;/* log specific stuff */ +struct xfs_ail *m_ail;/* fs active log item list */ +struct xfs_buf *m_rtsb_bp;/* realtime superblock */ +struct xfs_buf *m_sb_bp;/* superblock buffer */ +char *m_logname;/* external log device name */ +int m_logbsize;/* size of each log buffer */ /* * Global count of allocation btree blocks in use across all AGs. Only
@@ -323,6 +328,7 @@ __XFS_UNSUPP_FEAT(grpid)#define XFS_OPSTATE_REPORT_CORRUPTION 2/* report buffer corruption? */ #define XFS_OPSTATE_PERAG_DATA_LOADED 3/* per-AG data initialized? */ #define XFS_OPSTATE_RTGROUP_DATA_LOADED 4/* rtgroup data initialized? */ +#define XFS_OPSTATE_SHUTDOWN 5/* stop accepting metadata writes */ #define __XFS_IS_OPSTATE (name,NAME ) \ static inline bool xfs_is_ ## name (struct xfs_mount *mp) \
@@ -349,6 +355,7 @@ __XFS_IS_OPSTATE(debugger, DEBUGGER) __XFS_IS_OPSTATE(reporting_corruption, REPORT_CORRUPTION) __XFS_IS_OPSTATE(perag_data_loaded, PERAG_DATA_LOADED) __XFS_IS_OPSTATE(rtgroup_data_loaded, RTGROUP_DATA_LOADED) +__XFS_IS_OPSTATE(shutdown, SHUTDOWN)#define __XFS_UNSUPP_OPSTATE (name) \ static inline bool xfs_is_ ## name (struct xfs_mount *mp) \
@@ -356,7 +363,6 @@ static inline bool xfs_is_ ## name (struct xfs_mount *mp) \return false; \ }__XFS_UNSUPP_OPSTATE (readonly) -__XFS_UNSUPP_OPSTATE(shutdown) static inline int64_t xfs_sum_freecounter(struct xfs_mount *mp,enum xfs_free_counter ctr)
include/xfs_trace.h +30 -0
@@ -13,6 +13,36 @@#define trace_xfbtree_trans_cancel_buf (...) ((void) 0)#define trace_xfbtree_trans_commit_buf (...) ((void) 0) +#define trace_xfs_exchmaps_recover (...) ((void) 0) +#define trace_xfs_log_recover_buf_cancel (...) ((void) 0) +#define trace_xfs_log_recover_buf_cancel_add (...) ((void) 0) +#define trace_xfs_log_recover_buf_cancel_ref_inc (...) ((void) 0) +#define trace_xfs_log_recover_buf_dquot_buf (...) ((void) 0) +#define trace_xfs_log_recover_buf_inode_buf (...) ((void) 0) +#define trace_xfs_log_recover_buf_not_cancel (...) ((void) 0) +#define trace_xfs_log_recover_buf_recover (...) ((void) 0) +#define trace_xfs_log_recover_buf_reg_buf (...) ((void) 0) +#define trace_xfs_log_recover_buf_skip (...) ((void) 0) +#define trace_xfs_log_recover_inode_cancel (...) ((void) 0) +#define trace_xfs_log_recover_inode_recover (...) ((void) 0) +#define trace_xfs_log_recover_inode_skip (...) ((void) 0) +#define trace_xfs_log_recover_item_recover (...) ((void) 0) +#define trace_xfs_log_recover_item_reorder_head (...) ((void) 0) +#define trace_xfs_log_recover_item_reorder_tail (...) ((void) 0) +#define trace_xfs_log_recover_record (...) ((void) 0) +#define trace_xfs_agfl_free_deferred (...) ((void) 0) +#define trace_xfs_extent_free_deferred (...) ((void) 0) +#define trace_xfs_log_recover_icreate_cancel (...) ((void) 0) +#define trace_xfs_log_recover_icreate_recover (...) ((void) 0) +#define trace_xfs_log_recover (...) ((void) 0) +#define trace_xfs_ail_delete (...) ((void) 0) +#define trace_xfs_ail_flushing (a,b,c,d) ((void)(a),(void)(b),(void)(c),(void)(d)) +#define trace_xfs_ail_insert (...) ((void) 0) +#define trace_xfs_ail_locked (a,b,c,d) ((void)(a),(void)(b),(void)(c),(void)(d)) +#define trace_xfs_ail_move (...) ((void) 0) +#define trace_xfs_ail_pinned (a,b,c,d) ((void)(a),(void)(b),(void)(c),(void)(d)) +#define trace_xfs_ail_push (...) ((void) 0) +#define trace_xfs_log_assign_tail_lsn (...) ((void) 0)#define trace_xfs_agfl_reset (a,b,c,d) ((void) 0)#define trace_xfs_agfl_free_defer (...) ((void) 0)#define trace_xfs_alloc_cur_check (...) ((void) 0)
include/xfs_trans.h +69 -49
150 lines not on this page · raw message
libfrog/bitmask.h +13 -4
@@ -9,7 +9,7 @@#define BIT_MASK (nr) (1UL << ((nr) % BITS_PER_LONG))#define BIT_WORD (nr) ((nr) / BITS_PER_LONG) -static inline void set_bit (int nr,volatile unsigned long *addr) +static inline void set_bit (int nr,volatile void *addr) {unsigned long mask =BIT_MASK (nr);unsigned long *p = ((unsigned long *)addr) +BIT_WORD (nr);
@@ -17,7 +17,7 @@ static inline void set_bit(int nr, volatile unsigned long *addr) *p |= mask; } -static inline void clear_bit (int nr,volatile unsigned long *addr) +static inline void clear_bit (int nr,volatile void *addr) {unsigned long mask =BIT_MASK (nr);unsigned long *p = ((unsigned long *)addr) +BIT_WORD (nr);
@@ -25,7 +25,7 @@ static inline void clear_bit(int nr, volatile unsigned long *addr) *p &= ~mask; } -static inline int test_bit (int nr,const volatile unsigned long *addr) +static inline int test_bit (int nr,const volatile void *addr) {unsigned long mask =BIT_MASK (nr);unsigned long *p = ((unsigned long *)addr) +BIT_WORD (nr);
@@ -34,7 +34,7 @@ static inline int test_bit(int nr, const volatile unsigned long *addr) }/* Sets and returns original value of the bit */ -static inline int test_and_set_bit (int nr,volatile unsigned long *addr) +static inline int test_and_set_bit (int nr,volatile void *addr) {if (test_bit (nr, addr))return 1 ;
@@ -42,6 +42,15 @@ static inline int test_and_set_bit(int nr, volatile unsigned long *addr)return 0 ; } +/* Clears and returns original value of the bit */ +static inline int test_and_clear_bit (int nr,volatile void *addr) +{ +if (!test_bit (nr, addr)) +return 0 ; +clear_bit (nr, addr); +return 1 ; +} +/* Get high bit set out of 64-bit argument, -1 if none set */ static inline int xfrog_highbit64 (uint64_t v) {
libfrog/div64.h +7 -0
@@ -29,6 +29,13 @@ static inline int __do_div(unsigned long long *n, unsigned base)* This is commonly provided by 32bit archs to provide an optimized 64bit * divide. */ +static inline int64_t +div_s64_rem (int64_t dividend,int32_t divisor,int32_t *remainder) +{ + *remainder = dividend % divisor; +return dividend / divisor; +} +static inline uint64_t div_u64_rem (uint64_t dividend,uint32_t divisor,uint32_t *remainder) {
libxfs/Makefile +17 -2
@@ -20,7 +20,6 @@ PKGHFILES = xfs_fs.h \ xfs_log_format.h HFILES = \ - defer_item.h \ libxfs_io.h \ libxfs_api_defs.h \ listxattr.h \
@@ -74,9 +73,25 @@ HFILES = \ xfs_dir2_priv.h \ xfs_zones.h +# The kernel keeps log recovery, the log items and the AIL at the top level +# of fs/xfs; this tree keeps them in libxlog/. libxfs code refers to them, so +# they are compiled into this library rather than a separate one. +vpath %.c $(TOPDIR)/libxlog + CFILES = buf_mem.c \ + xfs_bmap_item.c \ + xfs_buf_item_recover.c \ + xfs_attr_item.c \ + xfs_dquot_item_recover.c \ + xfs_exchmaps_item.c \ + xfs_extfree_item.c \ + xfs_log_recover.c \ + xfs_inode_item_recover.c \ + xfs_icreate_item.c \ + xfs_refcount_item.c \ + xfs_rmap_item.c \ + xfs_trans_ail.c \ cache.c \ - defer_item.c \ init.c \ inode.c \ iunlink.c \
libxfs/defer_item.c deleted +0 -995
995 lines not on this page · raw message
libxfs/defer_item.h deleted +0 -56
56 lines not on this page · raw message
libxfs/init.c +73 -0
109 lines not on this page · raw message
libxfs/inode.c +43 -0
46 lines not on this page · raw message
libxfs/libxfs_io.h +3 -0
@@ -27,6 +27,8 @@ struct xfs_buftarg {unsigned long writes_left;dev_t bt_bdev;int bt_bdev_fd; +xfs_daddr_t bt_nr_sectors;/* device size, kept in +* step by log recovery */ struct xfile *bt_xfile;unsigned int flags;struct cache *bcache;/* buffer cache */
@@ -179,6 +181,7 @@ void libxfs_buf_mark_dirty(struct xfs_buf *bp);int libxfs_buf_get_map (struct xfs_buftarg *btp,struct xfs_buf_map *maps,int nmaps,int flags,struct xfs_buf **bpp);void libxfs_buf_relse (struct xfs_buf *bp); +void libxfs_buf_rele (struct xfs_buf *bp);static inline int libxfs_buf_get(
libxfs/logitem.c +201 -3
237 lines not on this page · raw message
libxfs/rdwr.c +157 -1
186 lines not on this page · raw message
libxfs/trans.c +31 -3
@@ -16,6 +16,7 @@#include "xfs_inode_fork.h" #include "xfs_inode.h" #include "xfs_trans.h" +#include "xfs_log_priv.h" #include "xfs_sb.h" #include "xfs_defer.h" #include "xfs_trace.h"
@@ -52,7 +53,7 @@ libxfs_trans_add_item(struct xfs_trans *tp, struct xfs_log_item *lip) { - ASSERT(lip->li_mountp == tp->t_mountp); + ASSERT(lip->li_log == tp->t_mountp->m_log);ASSERT (lip->li_ailp == tp->t_mountp->m_ail);ASSERT (list_empty (&lip->li_trans));ASSERT (!test_bit (XFS_LI_DIRTY , &lip->li_flags));
@@ -347,6 +348,23 @@ libxfs_trans_cancel( }if (dirty) { +/* +* The kernel shuts the filesystem down here rather than +* aborting, because a dirty cancel is not necessarily a +* programming error: replaying a corrupt log item reaches +* this path legitimately. Do the same while the log is +* being recovered so the failure is reported instead of +* dumping core, and keep the loud abort everywhere else, +* where it still means xfs_repair has a bug. +*/ +if (tp->t_mountp && tp->t_mountp->m_log && +xlog_recovery_needed (tp->t_mountp->m_log)) { +xlog_force_shutdown (tp->t_mountp->m_log, +SHUTDOWN_CORRUPT_INCORE ); +xfs_trans_free_items (tp); +xfs_trans_free (tp); +return ; + }fprintf (stderr,_ ("Cancelling dirty transaction!\n" ));abort (); }
@@ -355,7 +373,7 @@ libxfs_trans_cancel(xfs_trans_free (tp); } -static void +void xfs_buf_item_put (struct xfs_buf_log_item *bip) {
@@ -963,6 +981,12 @@ buf_item_done(libxfs_buf_relse (bp); } +/* +* Dispose of the items a transaction accumulated. Buffers and inodes have +* always had their own userspace handling; everything else - the intent and +* intent-done items that deferred operations now really do create - is +* disposed of the way the kernel does it, through the item ops vector. +*/ static void trans_committed (xfs_trans_t *tp)
@@ -976,6 +1000,8 @@ trans_committed(buf_item_done ((xfs_buf_log_item_t *)lip); else if (lip->li_type == XFS_LI_INODE) inode_item_done((struct xfs_inode_log_item *)lip); + else if (lip->li_ops->iop_release) + lip->li_ops->iop_release(lip); else { fprintf(stderr, _("%s: unrecognised log item type\n"), progname);
@@ -1021,6 +1047,8 @@ xfs_trans_free_items(buf_item_unlock ((xfs_buf_log_item_t *)lip); else if (lip->li_type == XFS_LI_INODE) inode_item_unlock((struct xfs_inode_log_item *)lip); + else if (lip->li_ops->iop_release) + lip->li_ops->iop_release(lip); else { fprintf(stderr, _("%s: unrecognised log item type\n"), progname);
@@ -1109,7 +1137,7 @@ xfs_trans_run_precommits( } }if (error) -xfs_force_shutdown (mp,SHUTDOWN_CORRUPT_INCORE ); +xfs_force_shutdown (tp->t_mountp,SHUTDOWN_CORRUPT_INCORE );return error; }
libxfs/util.c +6 -2
@@ -408,12 +408,16 @@ xfs_log_item_init(int type, const struct xfs_item_ops *ops) { - item->li_mountp = mp; + item->li_log = mp->m_log; + item->li_ailp = mp->m_ail; item->li_type = type; item->li_ops = ops; + item->li_lv =NULL ; -INIT_LIST_HEAD (&item->li_trans); +INIT_LIST_HEAD (&item->li_ail); +INIT_LIST_HEAD (&item->li_cil);INIT_LIST_HEAD (&item->li_bio_list); +INIT_LIST_HEAD (&item->li_trans); } static struct xfs_buftarg *
libxfs/xfs_alloc.c +1 -1
@@ -23,7 +23,7 @@#include "xfs_ag_resv.h" #include "xfs_bmap.h" #include "xfs_health.h" -#include "defer_item.h" +#include "xfs_extfree_item.h" struct kmem_cache *xfs_extfree_item_cache;
libxfs/xfs_attr.c +1 -1
@@ -24,7 +24,7 @@#include "xfs_quota_defs.h" #include "xfs_trans_space.h" #include "xfs_trace.h" -#include "defer_item.h" +#include "xfs_attr_item.h" #include "xfs_parent.h" struct kmem_cache *xfs_attr_intent_cache;
libxfs/xfs_bmap.c +1 -1
@@ -31,7 +31,7 @@#include "xfs_refcount.h" #include "xfs_rtbitmap.h" #include "xfs_health.h" -#include "defer_item.h" +#include "xfs_bmap_item.h" #include "xfs_symlink_remote.h" #include "xfs_inode_util.h" #include "xfs_rtgroup.h"
libxfs/xfs_defer.c +4 -0
@@ -12,8 +12,12 @@#include "xfs_mount.h" #include "xfs_defer.h" #include "xfs_trans.h" +#include "xfs_trans_priv.h" +#include "xfs_buf_item.h" #include "xfs_inode.h" +#include "xfs_inode_item.h" #include "xfs_trace.h" +#include "xfs_log_priv.h" #include "xfs_rmap.h" #include "xfs_refcount.h" #include "xfs_bmap.h"
libxfs/xfs_exchmaps.c +1 -1
@@ -20,7 +20,7 @@#include "xfs_trans_space.h" #include "xfs_errortag.h" #include "xfs_health.h" -#include "defer_item.h" +#include "xfs_exchmaps_item.h" #include "xfs_da_format.h" #include "xfs_da_btree.h" #include "xfs_attr_leaf.h"
libxfs/xfs_ialloc.c +3 -0
@@ -19,7 +19,10 @@#include "xfs_errortag.h" #include "xfs_bmap.h" #include "xfs_trans.h" +#include "xfs_buf_item.h" +#include "xfs_icreate_item.h" #include "xfs_trace.h" +#include "xfs_log.h" #include "xfs_rmap.h" #include "xfs_ag.h" #include "xfs_health.h"
libxfs/xfs_parent.c +1 -1
@@ -24,7 +24,7 @@#include "xfs_defer.h" #include "xfs_parent.h" #include "xfs_trans_space.h" -#include "defer_item.h" +#include "xfs_attr_item.h" #include "xfs_health.h" #include "xfs_attr_leaf.h"
libxfs/xfs_platform.h +10 -9
@@ -56,6 +56,7 @@#include "libfrog/util.h" #include "atomic.h" #include "spinlock.h" +#include "kernel_compat.h" #include "linux-err.h" #include "xfs_types.h"
@@ -143,7 +144,13 @@ extern char *progname;#define xfs_stack_trace () ((void) 0) -#define xfs_force_shutdown (d,n) ((void) 0) +/* +* The kernel shuts the mount and the log down together. Doing only half of it +* leaves the log looking healthy to xlog_is_shutdown(), which is what phase 2 +* consults before it declares the log clean and retires it. +*/ +#define xfs_force_shutdown (mp,f) ((void)xfs_set_shutdown(mp), \ + xlog_force_shutdown((mp)->m_log, (f)))#define xfs_mod_delalloc (a,b,c) ((void) 0)#define xfs_mod_sb_delalloc (sb, d) ((void) 0)
@@ -155,12 +162,12 @@ extern char *progname; } while (0)#define XFS_CORRUPTION_ERROR (e, lvl, mp, buf, bufsize) do { \ - (mp) = (mp); \ + (void)(mp); \ cmn_err(CE_ALERT, "%s: XFS_CORRUPTION_ERROR", (e)); \ } while (0)#define XFS_ERROR_REPORT (e,l,mp) do { \ - (mp) = (mp); \ + (void)(mp); \ cmn_err(CE_ALERT, "%s: XFS_ERROR_REPORT", (e)); \ } while (0)
@@ -454,9 +461,7 @@ static inline int retzero(void) { return 0; }#define uuid_copy (s,d) platform_uuid_copy((s),(d))#define uuid_equal (s,d) (platform_uuid_compare((s),(d)) == 0) -#define xfs_icreate_log (tp, agno, agbno, cnt, isize, len, gen) ((void) 0)#define xfs_sb_validate_fsb_count (sbp, nblks) (0) -#define xlog_calc_iovec_len (len) roundup(len, sizeof(uint32_t))#define xfs_zoned_add_available (mp, rtxnum) do { } while (0)
@@ -499,10 +504,6 @@ void xfs_trans_del_item(struct xfs_log_item *);/* xfs_inode_item.c */ void xfs_inode_item_init (struct xfs_inode *,struct xfs_mount *); -/* xfs_buf_item.c */ -void xfs_buf_item_init (struct xfs_buf *,struct xfs_mount *); -void xfs_buf_item_log (struct xfs_buf_log_item *,uint ,uint ); -/* xfs_trans_buf.c */ struct xfs_buf *xfs_trans_buf_item_match (struct xfs_trans *,struct xfs_buftarg *,struct xfs_buf_map *,int );
libxfs/xfs_refcount.c +1 -1
@@ -23,7 +23,7 @@#include "xfs_rmap.h" #include "xfs_ag.h" #include "xfs_health.h" -#include "defer_item.h" +#include "xfs_refcount_item.h" #include "xfs_rtgroup.h" #include "xfs_rtrefcount_btree.h"
libxfs/xfs_rmap.c +1 -1
@@ -23,7 +23,7 @@#include "xfs_inode.h" #include "xfs_ag.h" #include "xfs_health.h" -#include "defer_item.h" +#include "xfs_rmap_item.h" #include "xfs_rtgroup.h" #include "xfs_rtrmap_btree.h"
libxfs/xfs_trans_resv.c +6 -1
@@ -20,7 +20,12 @@#include "xfs_quota_defs.h" #include "xfs_rtbitmap.h" #include "xfs_trace.h" -#include "defer_item.h" +#include "xfs_attr_item.h" +#include "xfs_defer.h" +#include "xfs_bmap_item.h" +#include "xfs_extfree_item.h" +#include "xfs_rmap_item.h" +#include "xfs_refcount_item.h" #define _ALLOC true#define _FREE false
libxlog/Makefile +12 -0
@@ -6,6 +6,18 @@ TOPDIR = .. include $(TOPDIR)/include/builddefs LTLIBRARY = libxlog.la +HFILES = xfs_bmap_item.h \ + xfs_buf_item.h \ + xfs_attr_item.h \ + xfs_exchmaps_item.h \ + xfs_extfree_item.h \ + xfs_icreate_item.h \ + xfs_inode_item.h \ + xfs_refcount_item.h \ + xfs_rmap_item.h \ + xfs_log.h \ + xfs_log_priv.h \ + xfs_trans_priv.h LT_CURRENT = 0 LT_REVISION = 0 LT_AGE = 0
libxlog/logscan.c +11 -20
@@ -6,7 +6,6 @@#include "libxfs.h" #include "libxlog.h" -#define xfs_readonly_buftarg (buftarg) (0)/* avoid set-but-unused var warning. gcc is not very bright. */ #define xlog_clear_stale_blocks (log, taillsn) ({ \
@@ -69,7 +68,7 @@ xlog_get_bp( nbblks += log->l_sectBBsize; nbblks =round_up (nbblks, log->l_sectBBsize); -libxfs_buf_get_uncached (log->l_dev, nbblks, &bp); +libxfs_buf_get_uncached (log->l_targ, nbblks, &bp);return bp; }
@@ -118,7 +117,7 @@ xlog_bread_noalign( bp->b_length = nbblks; bp->b_error =0 ; -return libxfs_readbufr (log->l_dev,xfs_buf_daddr (bp), bp, nbblks,0 ); +return libxfs_readbufr (log->l_targ,xfs_buf_daddr (bp), bp, nbblks,0 ); }int
@@ -766,7 +765,7 @@ xlog_find_tail(if (found ==2 ) log->l_curr_cycle++;atomic64_set (&log->l_tail_lsn,be64_to_cpu (rhead->h_tail_lsn)); -atomic64_set (&log->l_last_sync_lsn,be64_to_cpu (rhead->h_lsn)); + xlog_last_sync_lsn =be64_to_cpu (rhead->h_lsn);xlog_assign_grant_head (&log->l_reserve_head.grant, log->l_curr_cycle,BBTOB (log->l_curr_block)); xlog_assign_grant_head(&log->l_write_head.grant, log->l_curr_cycle,
@@ -814,11 +813,16 @@ xlog_find_tail(* Set tail and last sync so that newly written * log records will point recovery to after the * current unmount record. +* +* Note that xlog_assign_atomic_lsn() is a no-op in +* userspace, so neither of these actually takes +* effect; xlog_last_sync_lsn keeps the value the +* record header scan gave it. Preserved as-is to +* avoid changing what xfs_repair concludes about +* the maximum metadata LSN. */ xlog_assign_atomic_lsn (&log->l_tail_lsn, log->l_curr_cycle, after_umount_blk); -xlog_assign_atomic_lsn (&log->l_last_sync_lsn, - log->l_curr_cycle, after_umount_blk); *tail_blk = after_umount_blk; } }
@@ -974,9 +978,8 @@ xlog_recover_find_tid( xlog_tid_t tid) {struct xlog_recover *trans; -struct hlist_node *n; -hlist_for_each_entry (trans, n, head, r_list) { +hlist_for_each_entry (trans, head, r_list) {if (trans->r_log_tid == tid)return trans; }
@@ -1329,18 +1332,6 @@ xlog_unpack_data_crc(* added for v2 logs. Addressing for the cycles array there is off by one, * because the first batch of cycles is in the original header. */ -static inline __be32 *xlog_cycle_data (struct xlog_rec_header *rhead,unsigned i) -{ -if (i >=XLOG_CYCLE_DATA_SIZE ) { -unsigned j = i /XLOG_CYCLE_DATA_SIZE ; -unsigned k = i %XLOG_CYCLE_DATA_SIZE ; - -return &rhead->h_ext[j -1 ].xh_cycle_data[k]; - } - -return &rhead->h_cycle_data[i]; -} - STATIC int xlog_unpack_data( struct xlog_rec_header *rhead,
libxlog/util.c +29 -5
@@ -11,6 +11,8 @@ int print_exit;int print_skip_uuid;int print_record_header; +xfs_lsn_t xlog_last_sync_lsn; + void xlog_init( struct xfs_mount *mp,
@@ -20,23 +22,45 @@ xlog_init(memset (log,0 ,sizeof (*log)); - log->l_dev = mp->m_logdev_targp; + log->l_targ = mp->m_logdev_targp; log->l_logBBsize =XFS_FSB_TO_BB (mp, mp->m_sb.sb_logblocks); log->l_logBBstart =XFS_FSB_TO_DADDR (mp, mp->m_sb.sb_logstart); + log->l_logsize =BBTOB (log->l_logBBsize);if (xfs_has_sector (mp)) log_sect_size <<= (mp->m_sb.sb_logsectlog -BBSHIFT ); log->l_sectBBsize =BTOBB (log_sect_size); log->l_mp = mp; + +/* +* The rest of what xlog_alloc_log() sets up that matters to a log +* that is only ever read and recovered, never written: +* r_dfops collects the intent items recovery finds, and +* XLOG_ACTIVE_RECOVERY marks the log as being recovered right now. +* That is a different bit from XLOG_RECOVERY_NEEDED, which records +* that a log was recovered and is set around the replay window by +* xfs_repair. +*/ +INIT_LIST_HEAD (&log->r_dfops); +set_bit (XLOG_ACTIVE_RECOVERY , &log->l_opstate); + log->l_covered_state =XLOG_STATE_COVER_IDLE ; + log->l_prev_block =-1 ; +xlog_assign_atomic_lsn (&log->l_tail_lsn,1 ,0 ); + log->l_curr_cycle =1 ;/* 0 is bad since this is initial value */ + +if (xfs_has_logv2 (mp) && mp->m_sb.sb_logsunit >1 ) + log->l_iclog_roundoff = mp->m_sb.sb_logsunit; +else if (mp->m_sb.sb_logsectsize >0 ) + log->l_iclog_roundoff = mp->m_sb.sb_logsectsize; +else + log->l_iclog_roundoff =BBSIZE ; +if (xfs_has_sector (mp)) { - log->l_sectbb_log = mp->m_sb.sb_logsectlog -BBSHIFT ; -ASSERT (log->l_sectbb_log <= mp->m_sectbb_log);/* for larger sector sizes, must have v2 or external log */ -ASSERT (log->l_sectbb_log ==0 || +ASSERT (log->l_sectBBsize ==1 || log->l_logBBstart ==0 ||xfs_has_logv2 (mp));ASSERT (mp->m_sb.sb_logsectlog >=BBSHIFT ); } - log->l_sectbb_mask = (1 << log->l_sectbb_log) -1 ; }/*
logprint/log_misc.c +2 -2
@@ -532,7 +532,7 @@ xlog_print_trans_qoff(* (which can have different field alignments) to the native 64 bit version */ struct xfs_inode_log_format * -xfs_inode_item_format_convert ( +logprint_inode_item_format_convert (char *src_buf,uint len,struct xfs_inode_log_format *in_f)
@@ -652,7 +652,7 @@ xlog_print_trans_inode(return f->ilf_size; } - f =xfs_inode_item_format_convert ((char *)&src_lbuf, len, &dst_lbuf); + f =logprint_inode_item_format_convert ((char *)&src_lbuf, len, &dst_lbuf);printf (_ ("INODE: " )); printf(_("#regs: %d ino: 0x%llx flags: 0x%x dsize: %d\n"), f->ilf_size,
logprint/log_print_all.c +1 -1
@@ -291,7 +291,7 @@ xlog_recover_print_inode(ASSERT (item->ri_buf[0 ].iov_len ==sizeof (struct xfs_inode_log_format_32 ) || item->ri_buf[0 ].iov_len ==sizeof (struct xfs_inode_log_format )); - f =xfs_inode_item_format_convert (item->ri_buf[0 ].iov_base, + f =logprint_inode_item_format_convert (item->ri_buf[0 ].iov_base, item->ri_buf[0 ].iov_len, &f_buf); printf(_(" INODE: #regs:%d ino:0x%llx flags:0x%x dsize:%d\n"),
logprint/logprint.c +1 -1
@@ -103,7 +103,7 @@ logstat( log->l_logBBsize = s.st_size >>9 ; log->l_logBBstart =0 ; log->l_sectBBsize =BTOBB (BBSIZE ); - log->l_dev = mp->m_logdev_targp; + log->l_targ = mp->m_logdev_targp; log->l_mp = mp; }
logprint/logprint.h +1 -1
@@ -35,7 +35,7 @@ extern bool is_printable(char* ptr, int len);extern void print_or_dump (char * ptr,int len);extern struct xfs_inode_log_format * -xfs_inode_item_format_convert (char *,uint ,struct xfs_inode_log_format *); +logprint_inode_item_format_convert (char *,uint ,struct xfs_inode_log_format *);extern int xlog_print_trans_efi (char **ptr,uint src_len,int continued);extern void xlog_recover_print_efi (struct xlog_recover_item *item);
repair/phase2.c +2 -2
@@ -89,7 +89,7 @@ zero_log(* filesystems. */ if (!no_modify && zap_log) { - libxfs_log_clear(log->l_dev, NULL, + libxfs_log_clear(log->l_targ, NULL, XFS_FSB_TO_DADDR(mp, mp->m_sb.sb_logstart), (xfs_extlen_t)XFS_FSB_TO_BB(mp, mp->m_sb.sb_logblocks), &mp->m_sb.sb_uuid,
@@ -110,7 +110,7 @@ zero_log(* is a v5 filesystem. */ if (xfs_has_crc (mp)) - libxfs_max_lsn =atomic64_read (&log->l_last_sync_lsn); + libxfs_max_lsn = xlog_last_sync_lsn; } static bool
repair/xfs_repair.c +1 -1
@@ -794,7 +794,7 @@ format_log_max_lsn( }do_warn (_ ("Format log to cycle %d.\n" ), new_cycle); -libxfs_log_clear (log->l_dev,NULL , logstart, logblocks, +libxfs_log_clear (log->l_targ,NULL , logstart, logblocks, &mp->m_sb.sb_uuid, logversion, mp->m_sb.sb_logsunit,XLOG_FMT , new_cycle, true); }
Signature · 1 line
2.47.3
Re: [PATCH 7/9] libxlog: build the imported kernel code
Please try to pre-load the changes to the rest of the code into properly document (as in hand written) patches that do one thing at a time. The wiring up should be mostly mechanical.
PATCH 8/9 xfs_repair: add -R to replay a dirty log before repairing
In reply to Chris Wedgwoodxfs_repair refuses to run on a filesystem with a dirty log, and the only way past that has been -L, which destroys the log and every metadata update it describes. The advice is to mount and unmount the filesystem first, which is not available when the filesystem cannot be mounted, or when the machine that crashed is gone.
-R replays the log first, using the kernel's own recovery code, and then repairs the result. It is defined to be equivalent to mounting the filesystem, unmounting it cleanly, and repairing that - so it retires the log afterwards, and refuses to run alongside -L or -n.
Replay is validated against the kernel: for each image in a corpus of crashed filesystems, the result is compared with the same image recovered by Linux and unmounted cleanly.
Inodes left on the unlinked list are not yet dealt with here; a later patch finishes those deletions, and until then repair moves them to lost+found. Replay is refused on zoned filesystems, where freeing a realtime extent needs the zone allocator rather than a metadata update.
--- man/man8/xfs_repair.8 | 79 +++++++++++++-- repair/globals.c | 1 + repair/globals.h | 1 + repair/phase2.c | 225 +++++++++++++++++++++++++++++++++++++++++- repair/xfs_repair.c | 22 ++++- 5 files changed, 315 insertions(+), 13 deletions(-)
man/man8/xfs_repair.8 +72 -7
@@ -8,6 +8,8 @@ xfs_repair \- repair an XFS filesystem ] [ .BR \-n " | " -e ] [ +.B \-R +] [ .B \-m .I maxmem ] [
@@ -71,6 +73,29 @@ and can cause the loss of user files and/or data. See the .B "DIRTY LOGS" section for more information. .TP +.B \-R +Replay the log. +Instructs +.B xfs_repair +to replay a dirty log before it starts repairing, rather than refusing to +run. This performs the same recovery the kernel performs when it mounts a +filesystem that was not unmounted cleanly, so the metadata updates that were +in progress at the time of the crash are preserved instead of discarded. +.IP +Replay requires the same CPU architecture as the machine that wrote the log, +and it writes to the filesystem, so +.B \-R +cannot be combined with +.B \-n +or +.BR \-L . +If replay fails, +.B xfs_repair +exits without repairing anything and leaves the log dirty. Whether it is then +safe to replay again depends on how far replay got; see the +.B "DIRTY LOGS" +section. +.TP .BI \-l " logdev" Specifies the device special file where the filesystem's external log resides. Only for those filesystems which use an external log.
@@ -543,15 +568,55 @@ Due to the design of the XFS log, a dirty log can only be replayed by the kernel, on a machine having the same CPU architecture as the machine which was writing to the log. .B xfs_repair -cannot replay a dirty log and will exit with a status code of 2 -when it detects a dirty log. +will by default exit with a status code of 2 when it detects a dirty log, +because replaying it is not part of repair. +.PP +There are three ways to deal with this, in decreasing order of preference. +.PP +The log can be replayed by mounting and immediately unmounting the +filesystem on the same class of machine that crashed. Please make sure that +the machine's hardware is reliable before replaying to avoid compounding the +problems. This is the best option because the kernel is the reference +implementation of log recovery. +.PP +If the filesystem cannot be mounted, but the log itself is intact, +.B xfs_repair \-R +will replay it in userspace and then repair the result. This preserves the +metadata updates recorded in the log. +.PP +Replay can still fail on a log that is itself damaged. Replaying a log is not +an atomic operation, so metadata that replay had already written stays written. +.B xfs_repair +does not go on to repair a filesystem it could not replay, and it does not +retire the log, so nothing is discarded. +.PP +Whether it is safe to try again depends on how far replay got, and +.B xfs_repair +says which case it hit. Replaying buffers is idempotent, so a failure reported +as +.B "Log replay failed" +can be retried. Completing intents is not idempotent - unlike the kernel, +userspace records no done items - so after a failure reported as +.B "Log recovery completion failed" +the log still describes intents that have already been applied, and replaying +it again, by mount or by +.BR \-R , +would apply them twice. In that case use +.B \-L +to discard the log and repair without it. .PP -In this situation, the log can be replayed by mounting and immediately -unmounting the filesystem on the same class of machine that crashed. -Please make sure that the machine's hardware is reliable before -replaying to avoid compounding the problems. +There is a difference between +.B \-R +and kernel recovery. A file that was deleted, but whose deletion had not +finished when the system crashed, is left on the filesystem's unlinked list. +The kernel finishes the deletion; replay in userspace does not, so repair +finds the file unreferenced and moves it to +.IR lost+found . +The filesystem is consistent either way, but a +.B \-R +run can leave files there that a mount and unmount would have removed. .PP -If mounting fails, the log can be erased by running +As a last resort the log can be erased by running .B xfs_repair with the -L option. All metadata updates in progress at the time of the crash will be lost,
repair/globals.c +1 -0
@@ -39,6 +39,7 @@ int no_modify;int dangerously;/* live dangerously ... fix ro mount */ int isa_file;int zap_log; +int replay_log;int dumpcore;/* abort, not exit on fatal errs */ int force_geo;/* can set geo on low confidence info */ int assume_xfs;/* assume we have an xfs fs */
repair/globals.h +1 -0
@@ -80,6 +80,7 @@ extern int no_modify;extern int dangerously;/* live dangerously ... fix ro mount */ extern int isa_file;extern int zap_log; +extern int replay_log;extern int dumpcore;/* abort, not exit on fatal errs */ extern int force_geo;/* can set geo on low confidence info */ extern int assume_xfs;/* assume we have an xfs fs */
repair/phase2.c +220 -5
253 lines not on this page · raw message
repair/xfs_repair.c +21 -1
@@ -99,6 +99,8 @@ usage(void)"Options:\n" " -f The device is a file\n" " -L Force log zeroing. Do this as a last resort.\n" +" -R Replay the log before repairing, as mounting and\n" +" unmounting the filesystem would.\n" " -l logdev Specifies the device where the external log resides.\n" " -m maxmem Maximum amount of memory to be used in megabytes.\n" " -n No modify mode, just checks the filesystem for damage.\n"
@@ -211,6 +213,7 @@ process_args(int argc, char **argv) dangerously =0 ; isa_file =0 ; zap_log =0 ; + replay_log =0 ; dumpcore =0 ; full_ino_ex_data =0 ; force_geo =0 ;
@@ -228,7 +231,7 @@ process_args(int argc, char **argv)* XXX have to add suboption processing here * attributes, quotas, nlinks, aligned_inos, sb_fbits */ -while ((c =getopt (argc, argv,"c:o:fl:m:r:LnDvVdPet:" )) !=EOF ) { +while ((c =getopt (argc, argv,"c:o:fl:m:r:LRnDvVdPet:" )) !=EOF ) {switch (c) {case 'D' : dumpcore =1 ;
@@ -403,6 +406,9 @@ process_args(int argc, char **argv)case 'L' : zap_log =1 ;break ; +case 'R' : + replay_log =1 ; +break ;case 'n' : no_modify =1 ;break ;
@@ -442,6 +448,20 @@ process_args(int argc, char **argv)if (report_corrected && no_modify)usage (); +/* +* -L destroys the log and -R replays it. Asking for both is a +* contradiction, and silently picking the destructive one is the worst +* possible answer. -n cannot replay either, since replay writes. +*/ +if (replay_log && zap_log) { +do_warn (_ ("%s: -L and -R are mutually exclusive.\n" ), progname); +usage (); + } +if (replay_log && no_modify) { +do_warn (_ ("%s: -R cannot be used with -n.\n" ), progname); +usage (); + } + p =getenv ("XFS_REPAIR_FAIL_AFTER_PHASE" );if (p) { errno =0 ;
Signature · 1 line
2.47.3
Re: [PATCH 8/9] xfs_repair: add -R to replay a dirty log before repairing
Quoted 2 lines · Chris Wedgwood
lost+found. Replay is refused on zoned filesystems, where freeing a realtime extent needs the zone allocator rather than a metadata update.
Huh? This sentence makes grammatical sense, but already fails at the semantic level. Deleting zoned RT extents goes through exactly the same metadata updates for the bmap and rmap tree, but instead of updating the allocation btrees it just updates a counter in the rmap inodes. So it requires a very similar, but smaller metadata update.
Re: [PATCH 8/9] xfs_repair: add -R to replay a dirty log before repairing
In reply to Chris WedgwoodQuoted 10 lines · Chris Wedgwood
xfs_repairrefuses to run on a filesystem with a dirty log, and the only way past that has been -L, which destroys the log and every metadata update it describes. The advice is to mount and unmount the filesystem first, which is not available when the filesystem cannot be mounted, or when the machine that crashed is gone.-R replays the log first, using the kernel's own recovery code, and then repairs the result. It is defined to be equivalent to mounting the filesystem, unmounting it cleanly, and repairing that - so it retires the log afterwards, and refuses to run alongside -L or -n.
....
Quoted 5 lines
repair/phase2.c +5
-0+/* +* From here on intents are being completed for real. Userspace writes +* no done items, so an intent finished before a failure is still +* described by the log we did not retire, and replaying that log again +* - here or by a kernel mount - would apply it a second time. Say so,
repair/phase2.c int xlog_recover_do_trans(struct xlog *log, struct xlog_recover *t, int p) · line 157
Quoted 10 lines
repair/phase2.c +10
-0+* because the obvious response to a failure is to try again. +*/ + error = -xlog_recover_finish (log); +if (error) +do_error ( +_ ("Log recovery completion failed: %s\n" +"Some intents may already have been applied. Do not replay this log\n" +"again, by mount or by -R, as that would apply them twice. Use -L to\n" +"discard the log and repair instead.\n" ), +strerror (error));
Urk. That's a red flag. Log recovery should -always- be retriable, even if there are failures replaying intents.
The progress that intent processing makes writes new records to the journal (intents and modified objects), and so running log recovery a second time will continue where the intent replay failed last time. i.e. it will recover all the changes up to the last failure, then attempt to replay the remaining intents that are pending in the journal.
If the userspace log recovery cannot be run repeatedly on recovery failure without bad things happening, then the code is buggy. Only once the recovery gets to the point that no new objects can be recovered because of persistent failures should the user need to resort to clearing the log....
Also, why are you copy/pasting and subtly modifying all the kernel log recovery code here instead of running the kernel code in libxlog directly?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 9/9 xfs_repair: finish deletions the crash interrupted
In reply to Chris WedgwoodA file unlinked while still open goes on the AGI unlinked list, and the deletion completes when the last reference is dropped. A crash in between leaves it there: no name, no links, but its blocks are still allocated and its inode record still in use.
The kernel finishes those deletions during recovery. -R did not, so repair found the inodes unreferenced afterwards and moved them to lost+found - which is not merely untidy, it resurrects files the user had deleted, readable by anyone who can read the filesystem.
Do the work. The kernel drives it through xfs_irele() and the inodegc workers, ending in xfs_inactive() in fs/xfs/xfs_inode.c, which is written against the VFS inode lifecycle and does not build here. What it does for an unlinked inode is small, though, and both halves already exist in libxfs: xfs_bunmapi_range() releases the blocks and xfs_inode_uninit() frees the inode and takes it off the list. So walk the unlinked lists explicitly once recovery has finished.
Explicit rather than hooked into xfs_irele(), because inodes are released during recovery for reasons that have nothing to do with deletion, and this way the work happens at a point where the filesystem is consistent and the log has not yet been retired.
The four-image corpus now matches kernel recovery exactly, where before two of the four differed:
unlinked: icount 64 ifree 22 fdblocks 245419 (kernel: identical)
intents2: matches
Before this patch that filesystem reported ifree 9 and moved twelve inodes to lost+found.
--- include/xfs_mount.h | 1 + libxfs/Makefile | 1 + libxfs/inactive.c | 181 ++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_repair.8 | 11 --- repair/phase2.c | 17 ++++ 5 files changed, 200 insertions(+), 11 deletions(-) create mode 100644 libxfs/inactive.c
include/xfs_mount.h +1 -0
@@ -405,6 +405,7 @@ void libxfs_compute_all_maxlevels(struct xfs_mount *mp);struct xfs_mount *libxfs_mount (struct xfs_mount *mp,struct xfs_sb *sb,struct libxfs_init *xi,unsigned int flags);int libxfs_flush_mount (struct xfs_mount *mp); +int libxfs_inactive_unlinked (struct xfs_mount *mp,unsigned int *freed);int libxfs_umount (struct xfs_mount *mp);extern void libxfs_rtmount_destroy (xfs_mount_t *);
libxfs/Makefile +1 -0
@@ -79,6 +79,7 @@ HFILES = \ vpath %.c $(TOPDIR)/libxlog CFILES = buf_mem.c \ + inactive.c \ xfs_bmap_item.c \ xfs_buf_item_recover.c \ xfs_attr_item.c \
libxfs/inactive.c new file +181 -0
181 lines not on this page · raw message
man/man8/xfs_repair.8 +0 -11
@@ -605,17 +605,6 @@ would apply them twice. In that case use .B \-L to discard the log and repair without it. .PP -There is a difference between -.B \-R -and kernel recovery. A file that was deleted, but whose deletion had not -finished when the system crashed, is left on the filesystem's unlinked list. -The kernel finishes the deletion; replay in userspace does not, so repair -finds the file unreferenced and moves it to -.IR lost+found . -The filesystem is consistent either way, but a -.B \-R -run can leave files there that a mount and unmount would have removed. -.PP As a last resort the log can be erased by running .B xfs_repair with the -L option.
repair/phase2.c +17 -0
@@ -46,6 +46,7 @@ replay_dirty_log( {int error;uint32_t new_cycle; +unsigned int inactivated;/* * libxlog's header_check_uuid() does not refuse a log whose UUID does
@@ -165,6 +166,22 @@ replay_dirty_log(do_error (_ ("Log recovery failed and the log was shut down.\n" )); +/* +* A crash can leave inodes that were unlinked but not yet deleted on +* the AGI unlinked lists. The kernel finishes those deletions as part +* of recovery; do the same, or repair finds them unreferenced later and +* moves them to lost+found, resurrecting files the user deleted. +*/ + error = -libxfs_inactive_unlinked (mp, &inactivated); +if (error) +do_error ( +_ ("Failed to delete inodes left unlinked by the crash: %s\n" ), +strerror (error)); +if (inactivated) +do_log ( +_ (" - completed deletion of %u unlinked inode%s\n" ), + inactivated, inactivated ==1 ?"" :"s" ); +libxfs_buf_lock (mp->m_sb_bp);libxfs_buf_relse (mp->m_sb_bp); mp->m_sb_bp =NULL ;
Signature · 1 line
2.47.3
Re: [PATCH 9/9] xfs_repair: finish deletions the crash interrupted
Quoted 1 line · Chris Wedgwood
Do the work.
Not a really useful sentence. Really. Just to emualte the kind of AI bot speach used a bit too much in these commit logs, however they ended up here.
Quoted 2 lines
The kernel drives it through
xfs_irele()and the inodegc workers, ending inxfs_inactive()infs/xfs/xfs_inode.c, which is
As is mentioned file names. Please write your own commit log, instead of generating this kind of rambling. And this also makes me a bit worried how the code came to life.
Quoted 2 lines
Before this patch that filesystem reported ifree 9 and moved twelve inodes to lost+found.
Please create test cases for unlinked inode recovery.
Quoted 3 lines · Chris Wedgwood
Adds
xfs_repair-R: replay the log then repair, without mounting and without discarding it as -L does. Patches 1-2 are standalone fixes; 3-9 implement -R, importing log recovery from v7.1 fs/xfs into libxlog/.
Pretty sure intro for a pretty big and long needed feature. What's the motivation to get this done now? How was it tested? I'd kinda expect an xfstests series to go along with that did something like generic/388 and generic/475 but with log recovery performed by repair at least, plus specific tests for all the pitfalls you ran into when implementing it.