xfs: directory parent pointers
Hi Chandan,
Christoph and I have finished reviewing parent pointers and think it's ready to be merged, so please accept these pull requests against today's for-next branch.
For anyone following along at home, the xfsprogs patches are here:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub-6.10_2024-04-23
--D
[GIT PULL 1/9] xfs: shrink struct xfs_da_args
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit 6a94b1acda7e7262418e23f906c12a2b08b69d12:
xfs: reinstate delalloc for RT inodes (if sb_rextsize == 1) (2024-04-22 18:00:50 +0530)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/shrink-dirattr-args-6.10_2024-04-23
for you to fetch changes up to cda60317ac57add7a0a2865aa29afbc6caad3e9a:
xfs: rearrange xfs_da_args a bit to use less space (2024-04-23 07:46:51 -0700)
----------------------------------------------------------------
xfs: shrink struct xfs_da_args [v13.4 1/9]
Let's clean out some unused flags and fields from struct xfs_da_args.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (5):
xfs: remove XFS_DA_OP_REMOVE
xfs: remove XFS_DA_OP_NOTIME
xfs: remove xfs_da_args.attr_flags
xfs: make attr removal an explicit operation
xfs: rearrange xfs_da_args a bit to use less space
fs/xfs/libxfs/xfs_attr.c | 31 ++++++++++++++++--------------- fs/xfs/libxfs/xfs_attr.h | 11 +++++++++-- fs/xfs/libxfs/xfs_da_btree.h | 29 +++++++++++++---------------- fs/xfs/scrub/attr.c | 1 - fs/xfs/scrub/attr_repair.c | 3 +-- fs/xfs/xfs_acl.c | 17 +++++++++-------- fs/xfs/xfs_ioctl.c | 19 ++++++++++--------- fs/xfs/xfs_iops.c | 2 +- fs/xfs/xfs_trace.h | 7 +------ fs/xfs/xfs_xattr.c | 22 ++++++++++++++++++---- fs/xfs/xfs_xattr.h | 3 ++- 11 files changed, 80 insertions(+), 65 deletions(-)
[GIT PULL 2/9] xfs: improve extended attribute validation
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit cda60317ac57add7a0a2865aa29afbc6caad3e9a:
xfs: rearrange xfs_da_args a bit to use less space (2024-04-23 07:46:51 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/improve-attr-validation-6.10_2024-04-23
for you to fetch changes up to ea0b3e814741fb64e7785b564ea619578058e0b0:
xfs: enforce one namespace per attribute (2024-04-23 07:46:54 -0700)
----------------------------------------------------------------
xfs: improve extended attribute validation [v13.4 2/9]
Prior to introducing parent pointer extended attributes, let's spend some time cleaning up the attr code and strengthening the validation that it performs on attrs coming in from the disk.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (14):
xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery
xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available
xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
xfs: fix missing check for invalid attr flags
xfs: check shortform attr entry flags specifically
xfs: restructure xfs_attr_complete_op a bit
xfs: use helpers to extract xattr op from opflags
xfs: validate recovered name buffers when recovering xattr items
xfs: always set args->value in xfs_attri_item_recover
xfs: use local variables for name and value length in _attri_commit_pass2
xfs: refactor name/length checks in xfs_attri_validate
xfs: refactor name/value iovec validation in xlog_recover_attri_commit_pass2
xfs: enforce one namespace per attribute
fs/xfs/libxfs/xfs_attr.c | 37 ++++++- fs/xfs/libxfs/xfs_attr.h | 9 +- fs/xfs/libxfs/xfs_attr_leaf.c | 7 +- fs/xfs/libxfs/xfs_da_format.h | 5 + fs/xfs/scrub/attr.c | 34 ++++-- fs/xfs/scrub/attr_repair.c | 4 +- fs/xfs/xfs_attr_item.c | 242 +++++++++++++++++++++++++++++++++--------- fs/xfs/xfs_attr_list.c | 18 +++- fs/xfs/xfs_mount.c | 16 +++ fs/xfs/xfs_mount.h | 6 +- fs/xfs/xfs_xattr.c | 3 +- 11 files changed, 304 insertions(+), 77 deletions(-)
[GIT PULL 3/9] xfs: Parent Pointers
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit ea0b3e814741fb64e7785b564ea619578058e0b0:
xfs: enforce one namespace per attribute (2024-04-23 07:46:54 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/pptrs-6.10_2024-04-23
for you to fetch changes up to 67ac7091e35bd34b75c0ec77331b53ca052e0cb3:
xfs: enable parent pointers (2024-04-23 07:47:01 -0700)
----------------------------------------------------------------
xfs: Parent Pointers [v13.4 3/9]
This is the latest parent pointer attributes for xfs. The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point.
Directory parent pointers are stored as namespaced extended attributes of a file. Because parent pointers are an indivisible tuple of (dirent_name, parent_ino, parent_gen) we cannot use the usual attr name lookup functions to find a parent pointer. This is solvable by introducing a new lookup mode that checks both the name and the value of the xattr.
Therefore, introduce this new name-value lookup mode that's gated on the XFS_ATTR_PARENT namespace. This requires the introduction of new opcodes for the extended attribute update log intent items, which actually means that parent pointers (itself an INCOMPAT feature) does not depend on the LOGGED_XATTRS log incompat feature bit.
To reduce collisions on the dirent names of parent pointers, introduce a new attr hash mode that is the dir2 namehash of the dirent name xor'd with the parent inode number.
At this point, Allison has moved on to other things, so I've merged her patchset into djwong-dev for merging.
Updates since v12 [djwong]:
Rebase on 6.9-rc and update the online fsck design document. Redesign the ondisk format to use the name-value lookups to get us back to the point where the attr is (dirent_name -> parent_ino/gen).
Updates since v11 [djwong]:
Rebase on 6.4-rc and make some tweaks and bugfixes to enable the repair prototypes. Merge with djwong-dev and make online repair actually work.
Updates since v10 [djwong]:
Merge in the ondisk format changes to get rid of the diroffset conflicts with the parent pointer repair code, rebase the entire series with the attr vlookup changes first, and merge all the other random fixes.
Updates since v9:
Reordered patches 2 and 3 to be 6 and 7
xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent
xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc reworked ioctl to use pptr listenty and flex array
NEW
xfs: don't remove the attr fork when parent pointers are enabled
NEW
directory lookups should return diroffsets too
NEW
xfs: move/add parent pointer validators to xfs_parent
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once
Fix xfs_rename to disallow reservationless quotas
Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Allison Henderson (15):
xfs: add parent pointer support to attribute code
xfs: define parent pointer ondisk extended attribute format
xfs: Expose init_xattrs in xfs_create_tmpfile
xfs: add parent pointer validator functions
xfs: extend transaction reservations for parent attributes
xfs: parent pointer attribute creation
xfs: add parent attributes to link
xfs: add parent attributes to symlink
xfs: remove parent pointers in unlink
xfs: Add parent pointers to rename
xfs: Add parent pointers to xfs_cross_rename
xfs: don't return XFS_ATTR_PARENT attributes via listxattr
xfs: pass the attr value to put_listent when possible
xfs: don't remove the attr fork when parent pointers are enabled
xfs: add a incompat feature bit for parent pointers
Christoph Hellwig (1):
xfs: check the flags earlier in xfs_attr_match
Darrick J. Wong (14):
xfs: rearrange xfs_attr_match parameters
xfs: move xfs_attr_defer_add to xfs_attr_item.c
xfs: create a separate hashname function for extended attributes
xfs: allow xattr matching on name and value for parent pointers
xfs: refactor xfs_is_using_logged_xattrs checks in attr item recovery
xfs: create attr log item opcodes and formats for parent pointers
xfs: record inode generation in xattr update log intent items
xfs: create a hashname function for parent pointers
xfs: move handle ioctl code to xfs_handle.c
xfs: split out handle management helpers a bit
xfs: add parent pointer ioctls
xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
xfs: drop compatibility minimum log size computations for reflink
xfs: enable parent pointers
fs/xfs/Makefile | 3 + fs/xfs/libxfs/xfs_attr.c | 92 ++-- fs/xfs/libxfs/xfs_attr.h | 23 +- fs/xfs/libxfs/xfs_attr_leaf.c | 87 +++- fs/xfs/libxfs/xfs_attr_sf.h | 1 + fs/xfs/libxfs/xfs_da_btree.h | 4 + fs/xfs/libxfs/xfs_da_format.h | 25 +- fs/xfs/libxfs/xfs_format.h | 4 +- fs/xfs/libxfs/xfs_fs.h | 79 +++- fs/xfs/libxfs/xfs_log_format.h | 25 +- fs/xfs/libxfs/xfs_log_rlimit.c | 46 ++ fs/xfs/libxfs/xfs_ondisk.h | 6 + fs/xfs/libxfs/xfs_parent.c | 293 +++++++++++++ fs/xfs/libxfs/xfs_parent.h | 99 +++++ fs/xfs/libxfs/xfs_sb.c | 4 + fs/xfs/libxfs/xfs_trans_resv.c | 326 +++++++++++--- fs/xfs/libxfs/xfs_trans_space.c | 121 +++++ fs/xfs/libxfs/xfs_trans_space.h | 25 +- fs/xfs/scrub/attr.c | 15 +- fs/xfs/scrub/dir_repair.c | 2 +- fs/xfs/scrub/orphanage.c | 5 +- fs/xfs/scrub/parent_repair.c | 3 +- fs/xfs/scrub/symlink_repair.c | 2 +- fs/xfs/scrub/tempfile.c | 2 +- fs/xfs/xfs_attr_item.c | 331 ++++++++++++-- fs/xfs/xfs_attr_item.h | 10 + fs/xfs/xfs_attr_list.c | 13 +- fs/xfs/xfs_export.c | 2 +- fs/xfs/xfs_export.h | 2 + fs/xfs/xfs_handle.c | 952 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_handle.h | 33 ++ fs/xfs/xfs_inode.c | 218 +++++++-- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_ioctl.c | 595 +------------------------ fs/xfs/xfs_ioctl.h | 28 -- fs/xfs/xfs_ioctl32.c | 1 + fs/xfs/xfs_iops.c | 15 +- fs/xfs/xfs_super.c | 14 + fs/xfs/xfs_symlink.c | 30 +- fs/xfs/xfs_trace.c | 1 + fs/xfs/xfs_trace.h | 95 +++- fs/xfs/xfs_xattr.c | 5 + 42 files changed, 2807 insertions(+), 832 deletions(-)
create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/libxfs/xfs_trans_space.c create mode 100644 fs/xfs/xfs_handle.c create mode 100644 fs/xfs/xfs_handle.h
[GIT PULL 4/9] xfs: scrubbing for parent pointers
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit 67ac7091e35bd34b75c0ec77331b53ca052e0cb3:
xfs: enable parent pointers (2024-04-23 07:47:01 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-pptrs-6.10_2024-04-23
for you to fetch changes up to 59a2af9086f0d60fc8de7346da67db7d764c7221:
xfs: check parent pointer xattrs when scrubbing (2024-04-23 07:47:03 -0700)
----------------------------------------------------------------
xfs: scrubbing for parent pointers [v13.4 4/9]
Teach online fsck to use parent pointers to assist in checking directories, parent pointers, extended attributes, and link counts.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (7):
xfs: revert commit 44af6c7e59b12
xfs: check dirents have parent pointers
xfs: deferred scrub of dirents
xfs: scrub parent pointers
xfs: deferred scrub of parent pointers
xfs: walk directory parent pointers to determine backref count
xfs: check parent pointer xattrs when scrubbing
fs/xfs/Makefile | 2 +- fs/xfs/libxfs/xfs_parent.c | 22 ++ fs/xfs/libxfs/xfs_parent.h | 5 + fs/xfs/scrub/attr.c | 27 +- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/dir.c | 342 ++++++++++++++++++++- fs/xfs/scrub/nlinks.c | 85 +++++- fs/xfs/scrub/nlinks_repair.c | 2 + fs/xfs/scrub/parent.c | 685 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/readdir.c | 78 +++++ fs/xfs/scrub/readdir.h | 3 + fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 65 ++++ 13 files changed, 1307 insertions(+), 11 deletions(-)
[GIT PULL 5/9] xfs: online repair for parent pointers
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit 59a2af9086f0d60fc8de7346da67db7d764c7221:
xfs: check parent pointer xattrs when scrubbing (2024-04-23 07:47:03 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-pptrs-6.10_2024-04-23
for you to fetch changes up to 327ed702d84034879572812f580cb769848af7ae:
xfs: inode repair should ensure there's an attr fork to store parent pointers (2024-04-23 16:55:16 -0700)
----------------------------------------------------------------
xfs: online repair for parent pointers [v13.4 5/9]
This series implements online repair for directory parent pointer metadata. The checking half is fairly straightforward -- for each outgoing directory link (forward or backwards), grab the inode at the other end, and confirm that there's a corresponding link. If we can't grab an inode or lock it, we'll save that link for a slower loop that cycles all the locks, confirms the continued existence of the link, and rechecks the link if it's actually still there.
Repairs are a bit more involved -- for directories, we walk the entire filesystem to rebuild the dirents from parent pointer information. Parent pointer repairs do the same walk but rebuild the pptrs from the dirent information, but with the added twist that it duplicates all the xattrs so that it can use the atomic extent swapping code to commit the repairs atomically.
This introduces an added twist to the xattr repair code -- we use dirent hooks to detect a colliding update to the pptr data while we're not holding the ILOCKs; if one is detected, we restart the xattr salvaging process but this time hold all the ILOCKs until the end of the scan.
For offline repair, the phase6 directory connectivity scan generates an index of all the expected parent pointers in the filesystem. Then it walks each file and compares the parent pointers attached to that file against the index generated, and resyncs the results as necessary.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (16):
xfs: remove some boilerplate from xfs_attr_set
xfs: make the reserved block permission flag explicit in xfs_attr_set
xfs: salvage parent pointers when rebuilding xattr structures
xfs: add raw parent pointer apis to support repair
xfs: repair directories by scanning directory parent pointers
xfs: implement live updates for directory repairs
xfs: replay unlocked parent pointer updates that accrue during xattr repair
xfs: repair directory parent pointers by scanning for dirents
xfs: implement live updates for parent pointer repairs
xfs: remove pointless unlocked assertion
xfs: split xfs_bmap_add_attrfork into two pieces
xfs: add a per-leaf block callback to xchk_xattr_walk
xfs: actually rebuild the parent pointer xattrs
xfs: adapt the orphanage code to handle parent pointers
xfs: repair link count of nondirectories after rebuilding parent pointers
xfs: inode repair should ensure there's an attr fork to store parent pointers
fs/xfs/libxfs/xfs_attr.c | 76 ++- fs/xfs/libxfs/xfs_attr.h | 3 +- fs/xfs/libxfs/xfs_bmap.c | 38 +- fs/xfs/libxfs/xfs_bmap.h | 3 +- fs/xfs/libxfs/xfs_dir2.c | 2 +- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_parent.c | 64 +++ fs/xfs/libxfs/xfs_parent.h | 6 + fs/xfs/scrub/attr.c | 2 +- fs/xfs/scrub/attr_repair.c | 484 +++++++++++++++- fs/xfs/scrub/attr_repair.h | 4 + fs/xfs/scrub/dir_repair.c | 567 +++++++++++++++++- fs/xfs/scrub/findparent.c | 12 +- fs/xfs/scrub/findparent.h | 10 +- fs/xfs/scrub/inode_repair.c | 41 ++ fs/xfs/scrub/listxattr.c | 10 +- fs/xfs/scrub/listxattr.h | 4 +- fs/xfs/scrub/nlinks.c | 3 +- fs/xfs/scrub/orphanage.c | 38 ++ fs/xfs/scrub/orphanage.h | 3 + fs/xfs/scrub/parent.c | 7 +- fs/xfs/scrub/parent_repair.c | 1307 +++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/scrub.c | 2 + fs/xfs/scrub/trace.h | 153 +++++ fs/xfs/xfs_xattr.c | 26 +- 25 files changed, 2743 insertions(+), 124 deletions(-)
[GIT PULL 6/9] xfs: detect and correct directory tree problems
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit 327ed702d84034879572812f580cb769848af7ae:
xfs: inode repair should ensure there's an attr fork to store parent pointers (2024-04-23 16:55:16 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-directory-tree-6.10_2024-04-23
for you to fetch changes up to 3f31406aef493b3f19020909d29974e28253f91c:
xfs: fix corruptions in the directory tree (2024-04-23 16:55:17 -0700)
----------------------------------------------------------------
xfs: detect and correct directory tree problems [v13.4 6/9]
Historically, checking the tree-ness of the directory tree structure has not been complete. Cycles of subdirectories break the tree properties, as do subdirectories with multiple parents. It's easy enough for DFS to detect problems as long as one of the participants is reachable from the root, but this technique cannot find unconnected cycles.
Directory parent pointers change that, because we can discover all of these problems from a simple walk from a subdirectory towards the root. For each child we start with, if the walk terminates without reaching the root, we know the path is disconnected and ought to be attached to the lost and found. If we find ourselves, we know this is a cycle and can delete an incoming edge. If we find multiple paths to the root, we know to delete an incoming edge.
Even better, once we've finished walking paths, we've identified the good ones and know which other path(s) to remove.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (4):
xfs: teach online scrub to find directory tree structure problems
xfs: invalidate dirloop scrub path data when concurrent updates happen
xfs: report directory tree corruption in the health information
xfs: fix corruptions in the directory tree
fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_fs.h | 4 +- fs/xfs/libxfs/xfs_health.h | 4 +- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/dirtree.c | 985 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/dirtree.h | 178 ++++++++ fs/xfs/scrub/dirtree_repair.c | 821 +++++++++++++++++++++++++++++++++++ fs/xfs/scrub/health.c | 1 + fs/xfs/scrub/ino_bitmap.h | 37 ++ fs/xfs/scrub/orphanage.c | 6 + fs/xfs/scrub/orphanage.h | 8 + fs/xfs/scrub/repair.h | 4 + fs/xfs/scrub/scrub.c | 7 + fs/xfs/scrub/scrub.h | 1 + fs/xfs/scrub/stats.c | 1 + fs/xfs/scrub/trace.c | 4 + fs/xfs/scrub/trace.h | 272 +++++++++++- fs/xfs/scrub/xfarray.h | 1 + fs/xfs/xfs_health.c | 1 + fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 21 files changed, 2337 insertions(+), 4 deletions(-)
create mode 100644 fs/xfs/scrub/dirtree.c
create mode 100644 fs/xfs/scrub/dirtree.h
create mode 100644 fs/xfs/scrub/dirtree_repair.c
create mode 100644 fs/xfs/scrub/ino_bitmap.h
[GIT PULL 7/9] xfs: vectorize scrub kernel calls
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit 3f31406aef493b3f19020909d29974e28253f91c:
xfs: fix corruptions in the directory tree (2024-04-23 16:55:17 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/vectorized-scrub-6.10_2024-04-23
for you to fetch changes up to c77b37584c2d1054452853e47e42c7350b8fe687:
xfs: introduce vectored scrub mode (2024-04-23 16:55:18 -0700)
----------------------------------------------------------------
xfs: vectorize scrub kernel calls [v13.4 7/9]
Create a vectorized version of the metadata scrub and repair ioctl, and adapt xfs_scrub to use that. This mitigates the impact of system call overhead on xfs_scrub runtime.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (3):
xfs: reduce the rate of cond_resched calls inside scrub
xfs: move xfs_ioc_scrub_metadata to scrub.c
xfs: introduce vectored scrub mode
fs/xfs/libxfs/xfs_fs.h | 33 +++++++++ fs/xfs/scrub/common.h | 25 ------- fs/xfs/scrub/scrub.c | 177 ++++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/scrub.h | 64 +++++++++++++++++ fs/xfs/scrub/trace.h | 79 ++++++++++++++++++++- fs/xfs/scrub/xfarray.c | 10 +-- fs/xfs/scrub/xfarray.h | 3 + fs/xfs/scrub/xfile.c | 2 +- fs/xfs/scrub/xfs_scrub.h | 6 +- fs/xfs/xfs_ioctl.c | 26 +------ 10 files changed, 366 insertions(+), 59 deletions(-)
[GIT PULL 8/9] xfs: reduce iget overhead in scrub
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit c77b37584c2d1054452853e47e42c7350b8fe687:
xfs: introduce vectored scrub mode (2024-04-23 16:55:18 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/reduce-scrub-iget-overhead-6.10_2024-04-23
for you to fetch changes up to 4ad350ac58627bfe81f71f43f6738e36b4eb75c6:
xfs: only iget the file once when doing vectored scrub-by-handle (2024-04-23 16:55:18 -0700)
----------------------------------------------------------------
xfs: reduce iget overhead in scrub [v13.4 8/9]
This patchset looks to reduce iget overhead in two ways: First, a previous patch conditionally set DONTCACHE on inodes during xchk_irele on the grounds that we knew better at irele time if an inode should be dropped. Unfortunately, over time that patch morphed into a call to d_mark_dontcache, which resulted in inodes being dropped even if they were referenced by the dcache. This actually caused *more* recycle overhead than if we'd simply called xfs_iget to set DONTCACHE only on misses.
The second patch reduces the cost of untrusted iget for a vectored scrub call by having the scrubv code maintain a separate refcount to the inode so that the cache will always hit.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (2):
xfs: use dontcache for grabbing inodes during scrub
xfs: only iget the file once when doing vectored scrub-by-handle
fs/xfs/scrub/common.c | 12 +++--------- fs/xfs/scrub/iscan.c | 13 +++++++++++-- fs/xfs/scrub/scrub.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/scrub.h | 7 +++++++ 4 files changed, 66 insertions(+), 11 deletions(-)
[GIT PULL 9/9] xfs: minor fixes to online repair
In reply to Darrick J. WongHi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems.
--D
The following changes since commit 4ad350ac58627bfe81f71f43f6738e36b4eb75c6:
xfs: only iget the file once when doing vectored scrub-by-handle (2024-04-23 16:55:18 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-fixes-6.10_2024-04-23
for you to fetch changes up to 5e1c7d0b29f7e05b01e448d2579a469cf3a0d350:
xfs: invalidate dentries for a file before moving it to the orphanage (2024-04-23 16:55:19 -0700)
----------------------------------------------------------------
xfs: minor fixes to online repair [v13.4 9/9]
Here are some miscellaneous bug fixes for the online repair code.
This has been running on the djcloud for months with no problems. Enjoy!
----------------------------------------------------------------
Darrick J. Wong (4):
xfs: drop the scrub file's iolock when transaction allocation fails
xfs: fix iunlock calls in xrep_adoption_trans_alloc
xfs: exchange-range for repairs is no longer dynamic
xfs: invalidate dentries for a file before moving it to the orphanage
fs/xfs/scrub/attr_repair.c | 3 +++ fs/xfs/scrub/dir_repair.c | 3 +++ fs/xfs/scrub/nlinks_repair.c | 4 +++- fs/xfs/scrub/orphanage.c | 49 ++++++++++++++++++----------------------- fs/xfs/scrub/parent_repair.c | 10 ++++++--- fs/xfs/scrub/rtsummary_repair.c | 10 ++++----- fs/xfs/scrub/scrub.c | 8 +++---- fs/xfs/scrub/scrub.h | 7 ------ fs/xfs/scrub/symlink_repair.c | 3 +++ fs/xfs/scrub/tempexch.h | 1 - fs/xfs/scrub/tempfile.c | 24 ++------------------ fs/xfs/scrub/trace.h | 3 --- 12 files changed, 49 insertions(+), 76 deletions(-)