xfsprogs: everything I have for 6.12
PATCH v3 1/1 xfsprogs: everything I have for 6.12
Hi Andrey,
All of the patches that I've submitted for xfsprogs 6.12 have completed review, and I'm about to send you pull requests for all of those branches. However, I'm first resending all those patches to the list so that they are archived in their final form.
With these patches pulled, you will be in a position to say that xfsprogs is (at last!) in sync with the kernel. This should make it /much/ easier for authors of large patchsets to run QA on matching kernel and userspace.
--D
PATCH 1/7 libxfs: new code for 6.12
Hi all,
New code for 6.12.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=libxfs-sync-6.12
---
Commits in this patchset:
* libxfs: require -std=gnu11 for compilation by default
* libxfs: test compiling public headers with a C++ compiler
* libxfs: port IS_ENABLED from the kernel
* libfrog: add xarray emulation
* xfs: introduce new file range commit ioctls
* xfs: pass the icreate args object to xfs_dialloc
* xfs: remove xfs_validate_rtextents
* xfs: factor out a xfs_validate_rt_geometry helper
* xfs: remove the limit argument to xfs_rtfind_back
* xfs: assert a valid limit in xfs_rtfind_forw
* xfs: add bounds checking to xfs_rt{bitmap,summary}_read_buf
* xfs: factor out rtbitmap/summary initialization helpers
* xfs: push transaction join out of xfs_rtbitmap_lock and xfs_rtgroup_lock
* xfs: ensure rtx mask/shift are correct after growfs
* xfs: remove xfs_rtb_to_rtxrem
* xfs: simplify xfs_rtalloc_query_range
* xfs: clean up the ISVALID macro in xfs_bmap_adjacent
* xfs: remove xfs_{rtbitmap,rtsummary}_wordcount
* xfs: replace m_rsumsize with m_rsumblocks
* xfs: fix a sloppy memory handling bug in xfs_iroot_realloc
* xfs: replace shouty XFS_BM{BT,DR} macros
* xfs: standardize the btree maxrecs function parameters
* xfs: use kvmalloc for xattr buffers
* xfs: remove unnecessary check
* xfs: use kfree_rcu_mightsleep to free the perag structures
* xfs: move the tagged perag lookup helpers to xfs_icache.c
* xfs: convert perag lookup to xarray
* xfs: ensure st_blocks never goes to zero during COW writes
* xfs: enable block size larger than page size support
* xfs: merge xfs_attr_leaf_try_add into xfs_attr_leaf_addname
* xfs: return bool from xfs_attr3_leaf_add
* xfs: distinguish extra split from real ENOSPC from xfs_attr3_leaf_split
* xfs: distinguish extra split from real ENOSPC from xfs_attr_node_try_addname
* xfs: fold xfs_bmap_alloc_userdata into xfs_bmapi_allocate
* xfs: don't ifdef around the exact minlen allocations
* xfs: call xfs_bmap_exact_minlen_extent_alloc from xfs_bmap_btalloc
* xfs: support lowmode allocations in xfs_bmap_exact_minlen_extent_alloc
* xfs: pass the exact range to initialize to xfs_initialize_perag
* xfs: merge the perag freeing helpers
* xfs: don't use __GFP_RETRY_MAYFAIL in xfs_initialize_perag
* xfs: update the pag for the last AG at recovery time
--- configure.ac | 13 ++ db/bmap.c | 10 +- db/bmap_inflate.c | 2 db/bmroot.c | 8 + db/btheight.c | 18 +-- db/check.c | 11 +- db/frag.c | 8 + db/iunlink.c | 2 db/metadump.c | 16 +-- include/builddefs.in | 8 + include/kmem.h | 11 ++ include/libxfs.h | 6 + include/platform_defs.h | 63 ++++++++++ include/xfs_mount.h | 4 - libfrog/radix-tree.h | 35 +++++ libxfs/Makefile | 31 +++++ libxfs/defer_item.c | 14 ++ libxfs/init.c | 17 ++- libxfs/ioctl_c_dummy.c | 11 ++ libxfs/ioctl_cxx_dummy.cpp | 13 ++ libxfs/libxfs_api_defs.h | 2 libxfs/libxfs_priv.h | 6 + libxfs/xfs_ag.c | 165 +++++--------------------- libxfs/xfs_ag.h | 25 +--- libxfs/xfs_alloc.c | 7 - libxfs/xfs_alloc.h | 4 - libxfs/xfs_alloc_btree.c | 6 - libxfs/xfs_alloc_btree.h | 3 libxfs/xfs_attr.c | 190 +++++++++++++----------------- libxfs/xfs_attr_leaf.c | 63 +++++----- libxfs/xfs_attr_leaf.h | 2 libxfs/xfs_bmap.c | 243 ++++++++++++++++---------------------- libxfs/xfs_bmap_btree.c | 24 ++-- libxfs/xfs_bmap_btree.h | 207 ++++++++++++++++++++++---------- libxfs/xfs_da_btree.c | 5 - libxfs/xfs_fs.h | 26 ++++ libxfs/xfs_ialloc.c | 14 ++ libxfs/xfs_ialloc.h | 4 - libxfs/xfs_ialloc_btree.c | 6 - libxfs/xfs_ialloc_btree.h | 3 libxfs/xfs_inode_fork.c | 40 +++--- libxfs/xfs_inode_util.c | 2 libxfs/xfs_refcount_btree.c | 5 - libxfs/xfs_refcount_btree.h | 3 libxfs/xfs_rmap_btree.c | 7 + libxfs/xfs_rmap_btree.h | 3 libxfs/xfs_rtbitmap.c | 274 +++++++++++++++++++++++++++++++------------ libxfs/xfs_rtbitmap.h | 61 ++-------- libxfs/xfs_sb.c | 92 ++++++++------ libxfs/xfs_sb.h | 3 libxfs/xfs_shared.h | 3 libxfs/xfs_trans_resv.c | 4 - libxfs/xfs_types.h | 12 -- m4/package_utilies.m4 | 5 + mkfs/proto.c | 17 +-- repair/bmap_repair.c | 2 repair/dinode.c | 17 +-- repair/phase5.c | 16 +-- repair/phase6.c | 18 +-- repair/prefetch.c | 8 + repair/rt.c | 7 - repair/scan.c | 6 - 62 files changed, 1062 insertions(+), 849 deletions(-) create mode 100644 libxfs/ioctl_c_dummy.c create mode 100644 libxfs/ioctl_cxx_dummy.cpp
PATCH 1/41 libxfs: require -std=gnu11 for compilation by default
From: Darrick J. Wong <[email protected]>
The kernel now builds with -std=gnu11, so let's make xfsprogs do that by default too. Distributions can still override the parameters by passing CFLAGS= and BUILD_CFLAGS= to configure, just as they always have.
--- configure.ac | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
configure.ac +6 -1
@@ -5,6 +5,11 @@ AC_CONFIG_MACRO_DIR([m4]) AC_CONFIG_SRCDIR([include/libxfs.h]) AC_PREFIX_DEFAULT(/usr) +# Default CFLAGS if nobody specifies anything else +if test "${CFLAGS+set}" != "set"; then + CFLAGS="-g -O2 -std=gnu11" +fi + AC_PROG_INSTALL LT_INIT
@@ -22,7 +27,7 @@ if test "${BUILD_CFLAGS+set}" != "set"; then if test $cross_compiling = no; then BUILD_CFLAGS="$CFLAGS" else - BUILD_CFLAGS="-g -O2" + BUILD_CFLAGS="-g -O2 -std=gnu11" fi fi
PATCH 2/41 libxfs: test compiling public headers with a C++ compiler
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Apparently C++ compilers don't like the implicit void* casts that go on in the system headers. Compile a dummy program with the C++ compiler to make sure this works.
--- configure.ac | 6 ++++++ include/builddefs.in | 8 ++++++++ libxfs/Makefile | 31 ++++++++++++++++++++++++++++++- libxfs/ioctl_c_dummy.c | 11 +++++++++++ libxfs/ioctl_cxx_dummy.cpp | 13 +++++++++++++ m4/package_utilies.m4 | 5 +++++ 6 files changed, 73 insertions(+), 1 deletion(-) create mode 100644 libxfs/ioctl_c_dummy.c create mode 100644 libxfs/ioctl_cxx_dummy.cpp
configure.ac +6 -0
@@ -9,6 +9,9 @@ AC_PREFIX_DEFAULT(/usr) if test "${CFLAGS+set}" != "set"; then CFLAGS="-g -O2 -std=gnu11" fi +if test "${CXXFLAGS+set}" != "set"; then + CXXFLAGS="-g -O2 -std=gnu++11" +fi AC_PROG_INSTALL LT_INIT
@@ -31,6 +34,9 @@ if test "${BUILD_CFLAGS+set}" != "set"; then fi fi +AC_PROG_CXX +# no C++ build tools yet + AC_ARG_ENABLE(shared, [ --enable-shared=[yes/no] Enable use of shared libraries [default=yes]],, enable_shared=yes)
include/builddefs.in +8 -0
@@ -14,6 +14,7 @@ MALLOCLIB = @malloc_lib@ LOADERFLAGS = @LDFLAGS@ LTLDFLAGS = @LDFLAGS@ CFLAGS = @CFLAGS@ -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wno-address-of-packed-member +CXXFLAGS = @CXXFLAGS@ -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wno-address-of-packed-member BUILD_CFLAGS = @BUILD_CFLAGS@ -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 # make sure we don't pick up whacky LDFLAGS from the make environment and
@@ -63,6 +64,7 @@ XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_STAMP=$(PKG_STATE_DIR)/xfs_scrub_all_media.stamp CC = @cc@ BUILD_CC = @BUILD_CC@ +CXX = @cxx@ AWK = @awk@ SED = @sed@ TAR = @tar@
@@ -161,9 +163,15 @@ ifeq ($(ENABLE_GETTEXT),yes) GCFLAGS += -DENABLE_GETTEXT endif +# Override these if C++ needs other options +SANITIZER_CXXFLAGS = $(SANITIZER_CFLAGS) +GCXXFLAGS = $(GCFLAGS) +PCXXFLAGS = $(PCFLAGS) + BUILD_CFLAGS += $(GCFLAGS) $(PCFLAGS) # First, Sanitizer, Global, Platform, Local CFLAGS CFLAGS += $(FCFLAGS) $(SANITIZER_CFLAGS) $(OPTIMIZER) $(GCFLAGS) $(PCFLAGS) $(LCFLAGS) +CXXFLAGS += $(FCXXFLAGS) $(SANITIZER_CXXFLAGS) $(OPTIMIZER) $(GCXXFLAGS) $(PCXXFLAGS) $(LCXXFLAGS) include $(TOPDIR)/include/buildmacros
libxfs/Makefile +30 -1
@@ -125,6 +125,18 @@ CFILES = buf_mem.c \ xfs_trans_space.c \ xfs_types.c +EXTRA_CFILES=\ + ioctl_c_dummy.c + +EXTRA_CXXFILES=\ + ioctl_cxx_dummy.cpp + +EXTRA_OBJECTS=\ + $(patsubst %.c,%.o,$(EXTRA_CFILES)) \ + $(patsubst %.cpp,%.o,$(EXTRA_CXXFILES)) + +LDIRT += $(EXTRA_OBJECTS) + # # Tracing flags: # -DMEM_DEBUG all zone memory use
@@ -148,7 +160,23 @@ LTLIBS = $(LIBPTHREAD) $(LIBRT) # don't try linking xfs_repair with a debug libxfs. DEBUG = -DNDEBUG -default: ltdepend $(LTLIBRARY) +default: ltdepend $(LTLIBRARY) $(EXTRA_OBJECTS) + +%dummy.o: %dummy.cpp + @echo " [CXXD] $@" + $(Q)$(CXX) $(CXXFLAGS) -c $< + +%dummy.o: %dummy.c + @echo " [CCD] $@" + $(Q)$(CC) $(CFLAGS) -c $< + +MAKECXXDEP := $(MAKEDEPEND) $(CXXFLAGS) + +.PHONY: .extradep + +.extradep: $(EXTRA_CFILES) $(EXTRA_CXXFILES) $(HFILES) + $(Q)$(MAKEDEP) $(EXTRA_CFILES) > .extradep + $(Q)$(MAKECXXDEP) $(EXTRA_CXXFILES) >> .extradep # set up include/xfs header directory include $(BUILDRULES)
@@ -172,4 +200,5 @@ install-dev: install # running the install-headers target. ifndef NODEP -include .ltdep +-include .extradep endif
libxfs/ioctl_c_dummy.c new file +11 -0
@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2024 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ + +/* Dummy program to test C compilation of user-exported xfs headers */ + +#include "include/xfs.h" +#include "include/handle.h" +#include "include/jdm.h"
libxfs/ioctl_cxx_dummy.cpp new file +13 -0
@@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2024 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <[email protected]> + */ + +/* Dummy program to test C++ compilation of user-exported xfs headers */ + +extern "C" { +#include "include/xfs.h" +#include "include/handle.h" +#include "include/jdm.h" +};
m4/package_utilies.m4 +5 -0
@@ -42,6 +42,11 @@ AC_DEFUN([AC_PACKAGE_UTILITIES], AC_SUBST(cc) AC_PACKAGE_NEED_UTILITY($1, "$cc", cc, [C compiler]) + AC_PROG_CXX + cxx="$CXX" + AC_SUBST(cxx) + AC_PACKAGE_NEED_UTILITY($1, "$cxx", cxx, [C++ compiler]) + if test -z "$MAKE"; then AC_PATH_PROG(MAKE, gmake,, $PATH) fi
PATCH 3/41 libxfs: port IS_ENABLED from the kernel
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Port the IS_ENABLED macro from the kernel so that it can be used in libxfs. This requires a bit of hygiene on our part -- any CONFIG_XFS_* define in userspace that have counterparts in the kernel must be defined to 1 (and not simply define'd) so that the macro works, because the kernel translates CONFIG_FOO=y in .config to #define CONFIG_FOO 1.
--- include/libxfs.h | 6 +++- include/platform_defs.h | 63 +++++++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_priv.h | 5 ++-- 3 files changed, 70 insertions(+), 4 deletions(-)
include/libxfs.h +4 -2
@@ -7,10 +7,12 @@#ifndef __LIBXFS_H__#define __LIBXFS_H__ +/* CONFIG_XFS_* must be defined to 1 to work with IS_ENABLED() */ +/* For userspace XFS_RT is always defined */ -#define CONFIG_XFS_RT +#define CONFIG_XFS_RT 1/* Ditto in-memory btrees */ -#define CONFIG_XFS_BTREE_IN_MEM +#define CONFIG_XFS_BTREE_IN_MEM 1#include "libxfs_api_defs.h" #include "platform_defs.h"
include/platform_defs.h +63 -0
67 lines not on this page · raw message
libxfs/libxfs_priv.h +3 -2
@@ -37,8 +37,9 @@#ifndef __LIBXFS_INTERNAL_XFS_H__#define __LIBXFS_INTERNAL_XFS_H__ -#define CONFIG_XFS_RT -#define CONFIG_XFS_BTREE_IN_MEM +/* CONFIG_XFS_* must be defined to 1 to work with IS_ENABLED() */ +#define CONFIG_XFS_RT 1 +#define CONFIG_XFS_BTREE_IN_MEM 1#include "libxfs_api_defs.h" #include "platform_defs.h"
PATCH 4/41 libfrog: add xarray emulation
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Implement the simple parts of the kernel xarray API on-top of the libfrog radix-tree.
--- libfrog/radix-tree.h | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
libfrog/radix-tree.h +35 -0
@@ -63,4 +63,39 @@ int radix_tree_tagged(struct radix_tree_root *root, unsigned int tag);static inline int radix_tree_preload (int gfp_mask) {return 0 ; }static inline void radix_tree_preload_end (void ) { } +/* +* Emulation of the kernel xarray API. Note that unlike the kernel +* xarray, there is no internal locking so code using this should not +* allow concurrent operations in userspace. +*/ +struct xarray { +struct radix_tree_root r; +}; + +static inline void xa_init (struct xarray *xa) +{ +INIT_RADIX_TREE (&xa->r,GFP_KERNEL ); +} + +static inline void *xa_load (struct xarray *xa,unsigned long index) +{ +return radix_tree_lookup (&xa->r, index); +} + +static inline void *xa_erase (struct xarray *xa,unsigned long index) +{ +return radix_tree_delete (&xa->r, index); +} + +static inline int xa_insert (struct xarray *xa,unsigned long index,void *entry, +unsigned int gfp) +{ +int error; + + error =radix_tree_insert (&xa->r, index, entry); +if (error == -EEXIST ) +return -EBUSY ; +return error; +} +#endif /* __LIBFROG_RADIX_TREE_H__ */
PATCH 5/41 xfs: introduce new file range commit ioctls
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Source kernel commit: 398597c3ef7fb1d8fa31491c8f4f3996cff45701
This patch introduces two more new ioctls to manage atomic updates to file contents -- XFS_IOC_START_COMMIT and XFS_IOC_COMMIT_RANGE. The does, but with the additional requirement that file2 cannot have changed since some sampling point. The start-commit ioctl performs the sampling of file attributes.
Note: This patch currently samples i_ctime during START_COMMIT and checks that it hasn't changed during COMMIT_RANGE. This isn't entirely safe in kernels prior to 6.12 because ctime only had coarse grained granularity and very fast updates could collide with a COMMIT_RANGE. With the multi-granularity ctime introduced by Jeff Layton, it's now possible to update ctime such that this does not happen.
It is critical, then, that this patch must not be backported to any kernel that does not support fine-grained file change timestamps.
--- libxfs/xfs_fs.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
libxfs/xfs_fs.h +26 -0
@@ -826,6 +826,30 @@ struct xfs_exchange_range {__u64 flags;/* see XFS_EXCHANGE_RANGE_* below */ }; +/* +* Using the same definition of file2 as struct xfs_exchange_range, commit the +* contents of file1 into file2 if file2 has the same inode number, mtime, and +* ctime as the arguments provided to the call. The old contents of file2 will +* be moved to file1. +* +* Returns -EBUSY if there isn't an exact match for the file2 fields. +* +* Filesystems must be able to restart and complete the operation even after +* the system goes down. +*/ +struct xfs_commit_range { +__s32 file1_fd; +__u32 pad;/* must be zeroes */ +__u64 file1_offset;/* file1 offset, bytes */ +__u64 file2_offset;/* file2 offset, bytes */ +__u64 length;/* bytes to exchange */ + +__u64 flags;/* see XFS_EXCHANGE_RANGE_* below */ + +/* opaque file2 metadata for freshness checks */ +__u64 file2_freshness[6 ]; +}; +/* * Exchange file data all the way to the ends of both files, and then exchange * the file sizes. This flag can be used to replace a file's contents with a
@@ -998,6 +1022,8 @@ struct xfs_getparents_by_handle {#define XFS_IOC_BULKSTAT _IOR ('X', 127, struct xfs_bulkstat_req)#define XFS_IOC_INUMBERS _IOR ('X', 128, struct xfs_inumbers_req)#define XFS_IOC_EXCHANGE_RANGE _IOW ('X', 129, struct xfs_exchange_range) +#define XFS_IOC_START_COMMIT _IOR ('X', 130, struct xfs_commit_range) +#define XFS_IOC_COMMIT_RANGE _IOW ('X', 131, struct xfs_commit_range)/* XFS_IOC_GETFSUUID ---------- deprecated 140 */
PATCH 6/41 xfs: pass the icreate args object to xfs_dialloc
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Source kernel commit: 390b4775d6787706b1846f15623a68e576ec900c
Pass the xfs_icreate_args object to xfs_dialloc since we can extract the relevant mode (really just the file type) and parent inumber from there. This simplifies the calling convention in preparation for the next patch.
--- db/iunlink.c | 2 +- libxfs/xfs_ialloc.c | 5 +++-- libxfs/xfs_ialloc.h | 4 +++- mkfs/proto.c | 5 ++--- repair/phase6.c | 2 +- 5 files changed, 10 insertions(+), 8 deletions(-)
db/iunlink.c +1 -1
@@ -223,7 +223,7 @@ create_unlinked(return error; } - error = -libxfs_dialloc (&tp,0 , args.mode, &ino); + error = -libxfs_dialloc (&tp, &args, &ino);if (error) {dbprintf (_ ("alloc inode: %s\n" ),strerror (error)); goto out_cancel;
libxfs/xfs_ialloc.c +3 -2
@@ -1850,11 +1850,12 @@ xfs_dialloc_try_ag(int xfs_dialloc (struct xfs_trans **tpp, -xfs_ino_t parent, -umode_t mode, +const struct xfs_icreate_args *args,xfs_ino_t *new_ino) {struct xfs_mount *mp = (*tpp)->t_mountp; +xfs_ino_t parent = args->pip ? args->pip->i_ino :0 ; +umode_t mode = args->mode &S_IFMT ;xfs_agnumber_t agno;int error =0 ;xfs_agnumber_t start_agno;
libxfs/xfs_ialloc.h +3 -1
@@ -33,11 +33,13 @@ xfs_make_iptr(struct xfs_mount *mp, struct xfs_buf *b, int o)return xfs_buf_offset (b, o << (mp)->m_sb.sb_inodelog); } +struct xfs_icreate_args ; +/* * Allocate an inode on disk. Mode is used to tell whether the new inode will * need space, and whether it is a directory. */ -int xfs_dialloc (struct xfs_trans **tpp,xfs_ino_t parent,umode_t mode, +int xfs_dialloc (struct xfs_trans **tpp,const struct xfs_icreate_args *args,xfs_ino_t *new_ino); int xfs_difree(struct xfs_trans *tp, struct xfs_perag *pag,
mkfs/proto.c +2 -3
@@ -428,7 +428,6 @@ creatproto( };struct xfs_inode *ip;struct inode *inode; -xfs_ino_t parent_ino = dp ? dp->i_ino :0 ;xfs_ino_t ino;int error;
@@ -440,7 +439,7 @@ creatproto(* Call the space management code to pick the on-disk inode to be * allocated. */ - error = -libxfs_dialloc (tpp, parent_ino, mode, &ino); + error = -libxfs_dialloc (tpp, &args, &ino);if (error)return error;
@@ -769,7 +768,7 @@ create_sb_metadata_file(if (error)res_failed (error); - error = -libxfs_dialloc (&tp,0 , args.mode, &ino); + error = -libxfs_dialloc (&tp, &args, &ino);if (error) goto fail;
repair/phase6.c +1 -1
@@ -873,7 +873,7 @@ mk_orphanage(if (i)res_failed (i); - error = -libxfs_dialloc (&tp, mp->m_sb.sb_rootino, args.mode, &ino); + error = -libxfs_dialloc (&tp, &args, &ino);if (error)do_error (_ ("%s inode allocation failed %d\n" ),ORPHANAGE , error);
PATCH 7/41 xfs: remove xfs_validate_rtextents
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 021d9c107e29a598e51fb66a54b22e5416125408
Replace xfs_validate_rtextents with an open coded check for 0 rtextents. The name for the function implies it does a lot more than a zero check, which is more obvious when open coded.
--- libxfs/xfs_sb.c | 2 +- libxfs/xfs_types.h | 12 ------------ 2 files changed, 1 insertion(+), 13 deletions(-)
libxfs/xfs_sb.c +1 -1
@@ -511,7 +511,7 @@ xfs_validate_sb_common( rbmblocks =howmany_64 (sbp->sb_rextents,NBBY * sbp->sb_blocksize); -if (!xfs_validate_rtextents (rexts) || +if (sbp->sb_rextents ==0 || sbp->sb_rextents != rexts || sbp->sb_rextslog !=xfs_compute_rextslog (rexts) || sbp->sb_rbmblocks != rbmblocks) {
libxfs/xfs_types.h +0 -12
@@ -235,16 +235,4 @@ bool xfs_verify_fileoff(struct xfs_mount *mp, xfs_fileoff_t off);bool xfs_verify_fileext (struct xfs_mount *mp,xfs_fileoff_t off,xfs_fileoff_t len); -/* Do we support an rt volume having this number of rtextents? */ -static inline bool -xfs_validate_rtextents ( -xfs_rtbxlen_t rtextents) -{ -/* No runt rt volumes */ -if (rtextents ==0 ) -return false; - -return true; -} -#endif /* __XFS_TYPES_H__ */
PATCH 8/41 xfs: factor out a xfs_validate_rt_geometry helper
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 6529eef810e2ded0e540162273ee31a41314ec4e
Split the RT geometry validation in the early mount code into a helper than can be reused by repair (from which this code was apparently originally stolen anyway).
--- libxfs/xfs_sb.c | 64 ++++++++++++++++++++++++++++++------------------------- libxfs/xfs_sb.h | 1 + 2 files changed, 36 insertions(+), 29 deletions(-)
libxfs/xfs_sb.c +35 -29
@@ -229,6 +229,38 @@ xfs_validate_sb_read(return 0 ; } +static uint64_t +xfs_sb_calc_rbmblocks ( +struct xfs_sb *sbp) +{ +return howmany_64 (sbp->sb_rextents,NBBY * sbp->sb_blocksize); +} + +/* Validate the realtime geometry */ +bool +xfs_validate_rt_geometry ( +struct xfs_sb *sbp) +{ +if (sbp->sb_rextsize * sbp->sb_blocksize >XFS_MAX_RTEXTSIZE || + sbp->sb_rextsize * sbp->sb_blocksize <XFS_MIN_RTEXTSIZE ) +return false; + +if (sbp->sb_rblocks ==0 ) { +if (sbp->sb_rextents !=0 || sbp->sb_rbmblocks !=0 || + sbp->sb_rextslog !=0 || sbp->sb_frextents !=0 ) +return false; +return true; + } + +if (sbp->sb_rextents ==0 || + sbp->sb_rextents !=div_u64 (sbp->sb_rblocks, sbp->sb_rextsize) || + sbp->sb_rextslog !=xfs_compute_rextslog (sbp->sb_rextents) || + sbp->sb_rbmblocks !=xfs_sb_calc_rbmblocks (sbp)) +return false; + +return true; +} +/* Check all the superblock fields we care about when writing one out. */ STATIC int xfs_validate_sb_write(
@@ -488,39 +520,13 @@ xfs_validate_sb_common( } } -/* Validate the realtime geometry; stolen from xfs_repair */ -if (sbp->sb_rextsize * sbp->sb_blocksize >XFS_MAX_RTEXTSIZE || - sbp->sb_rextsize * sbp->sb_blocksize <XFS_MIN_RTEXTSIZE ) { +if (!xfs_validate_rt_geometry (sbp)) {xfs_notice (mp, -"realtime extent sanity check failed" ); +"realtime %sgeometry check failed" , + sbp->sb_rblocks ?"" :"zeroed " );return -EFSCORRUPTED ; } -if (sbp->sb_rblocks ==0 ) { -if (sbp->sb_rextents !=0 || sbp->sb_rbmblocks !=0 || - sbp->sb_rextslog !=0 || sbp->sb_frextents !=0 ) { -xfs_notice (mp, -"realtime zeroed geometry check failed" ); -return -EFSCORRUPTED ; - } - }else { -uint64_t rexts; -uint64_t rbmblocks; - - rexts =div_u64 (sbp->sb_rblocks, sbp->sb_rextsize); - rbmblocks =howmany_64 (sbp->sb_rextents, -NBBY * sbp->sb_blocksize); - -if (sbp->sb_rextents ==0 || - sbp->sb_rextents != rexts || - sbp->sb_rextslog !=xfs_compute_rextslog (rexts) || - sbp->sb_rbmblocks != rbmblocks) { -xfs_notice (mp, -"realtime geometry sanity check failed" ); -return -EFSCORRUPTED ; - } - } -/* * Either (sb_unit and !hasdalign) or (!sb_unit and hasdalign) * would imply the image is corrupted.
libxfs/xfs_sb.h +1 -0
@@ -38,6 +38,7 @@ extern int xfs_sb_get_secondary(struct xfs_mount *mp,bool xfs_validate_stripe_geometry (struct xfs_mount *mp,__s64 sunit,__s64 swidth,int sectorsize,bool may_repair,bool silent); +bool xfs_validate_rt_geometry (struct xfs_sb *sbp);uint8_t xfs_compute_rextslog (xfs_rtbxlen_t rtextents);
PATCH 9/41 xfs: remove the limit argument to xfs_rtfind_back
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 119c65e56bc131b466a7cd958a4089e286ce3c4b
All callers pass a 0 limit to xfs_rtfind_back, so remove the argument and hard code it.
--- libxfs/xfs_rtbitmap.c | 9 ++++----- libxfs/xfs_rtbitmap.h | 2 +- 2 files changed, 5 insertions(+), 6 deletions(-)
libxfs/xfs_rtbitmap.c +4 -5
@@ -137,14 +137,13 @@ xfs_rtbuf_get( }/* -* Searching backward from start to limit, find the first block whose -* allocated/free state is different from start's. +* Searching backward from start find the first block whose allocated/free state +* is different from start's. */ int xfs_rtfind_back (struct xfs_rtalloc_args *args,xfs_rtxnum_t start,/* starting rtext to look at */ -xfs_rtxnum_t limit,/* last rtext to look at */ xfs_rtxnum_t *rtx)/* out: start rtext found */ {struct xfs_mount *mp = args->mp;
@@ -173,7 +172,7 @@ xfs_rtfind_back(*/ word =xfs_rtx_to_rbmword (mp, start); bit = (int )(start & (XFS_NBWORD -1 )); - len = start - limit +1 ; + len = start +1 ;/* * Compute match value, based on the bit at start: if 1 (free) * then all-ones, else all-zeroes.
@@ -696,7 +695,7 @@ xfs_rtfree_range(* We need to find the beginning and end of the extent so we can * properly update the summary. */ - error =xfs_rtfind_back (args, start,0 , &preblock); + error =xfs_rtfind_back (args, start, &preblock);if (error) {return error; }
libxfs/xfs_rtbitmap.h +1 -1
@@ -316,7 +316,7 @@ xfs_rtsummary_read_buf(int xfs_rtcheck_range (struct xfs_rtalloc_args *args,xfs_rtxnum_t start,xfs_rtxlen_t len,int val,xfs_rtxnum_t *new,int *stat);int xfs_rtfind_back (struct xfs_rtalloc_args *args,xfs_rtxnum_t start, -xfs_rtxnum_t limit,xfs_rtxnum_t *rtblock); +xfs_rtxnum_t *rtblock);int xfs_rtfind_forw (struct xfs_rtalloc_args *args,xfs_rtxnum_t start,xfs_rtxnum_t limit,xfs_rtxnum_t *rtblock); int xfs_rtmodify_range(struct xfs_rtalloc_args *args, xfs_rtxnum_t start,
PATCH 10/41 xfs: assert a valid limit in xfs_rtfind_forw
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 6d2db12d56a389b3e8efa236976f8dc3a8ae00f0
Protect against developers passing stupid limits when refactoring the RT code once again.
--- libxfs/xfs_rtbitmap.c | 2 ++ 1 file changed, 2 insertions(+)
libxfs/xfs_rtbitmap.c +2 -0
@@ -313,6 +313,8 @@ xfs_rtfind_forw(xfs_rtword_t incore;unsigned int word;/* word number in the buffer */ +ASSERT (start <= limit); +/* * Compute and read in starting bitmap block for starting block. */
PATCH 11/41 xfs: add bounds checking to xfs_rt{bitmap,summary}_read_buf
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: b4781eea6872431840e53ffebb95a5614e6944b4
Add a corruption check for passing an invalid block number, which is a lot easier to understand than the xfs_bmapi_read failure later on.
--- libxfs/xfs_rtbitmap.c | 31 ++++++++++++++++++++++++++++++- libxfs/xfs_rtbitmap.h | 22 ++-------------------- 2 files changed, 32 insertions(+), 21 deletions(-)
libxfs/xfs_rtbitmap.c +30 -1
@@ -67,7 +67,7 @@ xfs_rtbuf_cache_relse(* Get a buffer for the bitmap or summary file block specified. * The buffer is returned read and locked. */ -int +static int xfs_rtbuf_get( struct xfs_rtalloc_args *args, xfs_fileoff_t block, /* block number in bitmap or summary */
@@ -136,6 +136,35 @@ xfs_rtbuf_get(return 0 ; } +int +xfs_rtbitmap_read_buf ( +struct xfs_rtalloc_args *args, +xfs_fileoff_t block) +{ +struct xfs_mount *mp = args->mp; + +if (XFS_IS_CORRUPT (mp, block >= mp->m_sb.sb_rbmblocks)) { +xfs_rt_mark_sick (mp,XFS_SICK_RT_BITMAP ); +return -EFSCORRUPTED ; + } + +return xfs_rtbuf_get (args, block,0 ); +} + +int +xfs_rtsummary_read_buf ( +struct xfs_rtalloc_args *args, +xfs_fileoff_t block) +{ +struct xfs_mount *mp = args->mp; + +if (XFS_IS_CORRUPT (mp, block >=XFS_B_TO_FSB (mp, mp->m_rsumsize))) { +xfs_rt_mark_sick (args->mp,XFS_SICK_RT_SUMMARY ); +return -EFSCORRUPTED ; + } +return xfs_rtbuf_get (args, block,1 ); +} +/* * Searching backward from start find the first block whose allocated/free state * is different from start's.
libxfs/xfs_rtbitmap.h +2 -20
@@ -293,26 +293,8 @@ typedef int (*xfs_rtalloc_query_range_fn)(#ifdef CONFIG_XFS_RT void xfs_rtbuf_cache_relse (struct xfs_rtalloc_args *args); - -int xfs_rtbuf_get (struct xfs_rtalloc_args *args,xfs_fileoff_t block, -int issum); - -static inline int -xfs_rtbitmap_read_buf ( -struct xfs_rtalloc_args *args, -xfs_fileoff_t block) -{ -return xfs_rtbuf_get (args, block,0 ); -} - -static inline int -xfs_rtsummary_read_buf ( -struct xfs_rtalloc_args *args, -xfs_fileoff_t block) -{ -return xfs_rtbuf_get (args, block,1 ); -} - +int xfs_rtbitmap_read_buf (struct xfs_rtalloc_args *args,xfs_fileoff_t block); +int xfs_rtsummary_read_buf (struct xfs_rtalloc_args *args,xfs_fileoff_t block);int xfs_rtcheck_range (struct xfs_rtalloc_args *args,xfs_rtxnum_t start,xfs_rtxlen_t len,int val,xfs_rtxnum_t *new,int *stat); int xfs_rtfind_back(struct xfs_rtalloc_args *args, xfs_rtxnum_t start,
PATCH 12/41 xfs: factor out rtbitmap/summary initialization helpers
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 2a95ffc44b610643c9d5d2665600d3fbefa5ec4f
Add helpers to libxfs that can be shared by growfs and mkfs for initializing the rtbitmap and summary, and by passing the optional data pointer also by repair for rebuilding them. This will become even more useful when the rtgroups feature adds a metadata header to each block, which means even more shared code.
--- libxfs/xfs_rtbitmap.c | 126 +++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_rtbitmap.h | 3 + 2 files changed, 129 insertions(+)
libxfs/xfs_rtbitmap.c +126 -0
@@ -13,6 +13,8 @@#include "xfs_mount.h" #include "xfs_inode.h" #include "xfs_bmap.h" +#include "xfs_bmap_btree.h" +#include "xfs_trans_space.h" #include "xfs_trans.h" #include "xfs_rtbitmap.h" #include "xfs_health.h"
@@ -1253,3 +1255,127 @@ xfs_rtbitmap_unlock_shared(if (rbmlock_flags &XFS_RBMLOCK_BITMAP )xfs_iunlock (mp->m_rbmip,XFS_ILOCK_SHARED |XFS_ILOCK_RTBITMAP ); } + +static int +xfs_rtfile_alloc_blocks ( +struct xfs_inode *ip, +xfs_fileoff_t offset_fsb, +xfs_filblks_t count_fsb, +struct xfs_bmbt_irec *map) +{ +struct xfs_mount *mp = ip->i_mount; +struct xfs_trans *tp; +int nmap =1 ; +int error; + + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_growrtalloc, +XFS_GROWFSRT_SPACE_RES (mp, count_fsb),0 ,0 , &tp); +if (error) +return error; + +xfs_ilock (ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); + + error =xfs_iext_count_extend (tp, ip,XFS_DATA_FORK , +XFS_IEXT_ADD_NOSPLIT_CNT ); +if (error) + goto out_trans_cancel; + + error =xfs_bmapi_write (tp, ip, offset_fsb, count_fsb, +XFS_BMAPI_METADATA ,0 , map, &nmap); +if (error) + goto out_trans_cancel; + +return xfs_trans_commit (tp); + +out_trans_cancel: +xfs_trans_cancel (tp); +return error; +} + +/* Get a buffer for the block. */ +static int +xfs_rtfile_initialize_block ( +struct xfs_inode *ip, +xfs_fsblock_t fsbno, +void *data) +{ +struct xfs_mount *mp = ip->i_mount; +struct xfs_trans *tp; +struct xfs_buf *bp; +const size_t copylen = mp->m_blockwsize <<XFS_WORDLOG ; +enum xfs_blft buf_type; +int error; + +if (ip == mp->m_rsumip) + buf_type =XFS_BLFT_RTSUMMARY_BUF ; +else + buf_type =XFS_BLFT_RTBITMAP_BUF ; + + error =xfs_trans_alloc (mp, &M_RES (mp)->tr_growrtzero,0 ,0 ,0 , &tp); +if (error) +return error; +xfs_ilock (ip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, ip,XFS_ILOCK_EXCL ); + + error =xfs_trans_get_buf (tp, mp->m_ddev_targp, +XFS_FSB_TO_DADDR (mp, fsbno), mp->m_bsize,0 , &bp); +if (error) { +xfs_trans_cancel (tp); +return error; + } + +xfs_trans_buf_set_type (tp, bp, buf_type); + bp->b_ops = &xfs_rtbuf_ops; +if (data) +memcpy (bp->b_addr, data, copylen); +else +memset (bp->b_addr,0 , copylen); +xfs_trans_log_buf (tp, bp,0 , mp->m_sb.sb_blocksize -1 ); +return xfs_trans_commit (tp); +} + +/* +* Allocate space to the bitmap or summary file, and zero it, for growfs. +* @data must be a contiguous buffer large enough to fill all blocks in the +* file; or NULL to initialize the contents to zeroes. +*/ +int +xfs_rtfile_initialize_blocks ( +struct xfs_inode *ip,/* inode (bitmap/summary) */ +xfs_fileoff_t offset_fsb,/* offset to start from */ +xfs_fileoff_t end_fsb,/* offset to allocate to */ +void *data)/* data to fill the blocks */ +{ +struct xfs_mount *mp = ip->i_mount; +const size_t copylen = mp->m_blockwsize <<XFS_WORDLOG ; + +while (offset_fsb < end_fsb) { +struct xfs_bmbt_irec map; +xfs_filblks_t i; +int error; + + error =xfs_rtfile_alloc_blocks (ip, offset_fsb, + end_fsb - offset_fsb, &map); +if (error) +return error; + +/* +* Now we need to clear the allocated blocks. +* +* Do this one block per transaction, to keep it simple. +*/ +for (i =0 ; i < map.br_blockcount; i++) { + error =xfs_rtfile_initialize_block (ip, + map.br_startblock + i, data); +if (error) +return error; +if (data) + data += copylen; + } + + offset_fsb = map.br_startoff + map.br_blockcount; + } + +return 0 ; +}
libxfs/xfs_rtbitmap.h +3 -0
@@ -343,6 +343,9 @@ xfs_filblks_t xfs_rtsummary_blockcount(struct xfs_mount *mp,unsigned long long xfs_rtsummary_wordcount (struct xfs_mount *mp,unsigned int rsumlevels,xfs_extlen_t rbmblocks); +int xfs_rtfile_initialize_blocks (struct xfs_inode *ip, +xfs_fileoff_t offset_fsb,xfs_fileoff_t end_fsb,void *data); +void xfs_rtbitmap_lock (struct xfs_trans *tp,struct xfs_mount *mp);void xfs_rtbitmap_unlock (struct xfs_mount *mp);
PATCH 13/41 xfs: push transaction join out of xfs_rtbitmap_lock and xfs_rtgroup_lock
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 0a59e4f3e1670bc49d60e1bd1a9b19ca156ae9cb
To prepare for being able to join an already locked rtbitmap inode to a transaction split out separate helpers for joining the transaction from the locking helpers.
--- libxfs/xfs_bmap.c | 3 ++- libxfs/xfs_rtbitmap.c | 24 +++++++++++++----------- libxfs/xfs_rtbitmap.h | 6 ++++-- 3 files changed, 19 insertions(+), 14 deletions(-)
libxfs/xfs_bmap.c +2 -1
@@ -5370,7 +5370,8 @@ xfs_bmap_del_extent_real(*/ if (!(tp->t_flags &XFS_TRANS_RTBITMAP_LOCKED )) { tp->t_flags |=XFS_TRANS_RTBITMAP_LOCKED ; -xfs_rtbitmap_lock (tp, mp); +xfs_rtbitmap_lock (mp); +xfs_rtbitmap_trans_join (tp); } error =xfs_rtfree_blocks (tp, del->br_startblock, del->br_blockcount);
libxfs/xfs_rtbitmap.c +13 -11
@@ -1199,23 +1199,25 @@ xfs_rtsummary_wordcount(return XFS_FSB_TO_B (mp, blocks) >>XFS_WORDLOG ; } -/* -* Lock both realtime free space metadata inodes for a freespace update. If a -* transaction is given, the inodes will be joined to the transaction and the -* ILOCKs will be released on transaction commit. -*/ +/* Lock both realtime free space metadata inodes for a freespace update. */ void xfs_rtbitmap_lock ( -struct xfs_trans *tp,struct xfs_mount *mp) {xfs_ilock (mp->m_rbmip,XFS_ILOCK_EXCL |XFS_ILOCK_RTBITMAP ); -if (tp) -xfs_trans_ijoin (tp, mp->m_rbmip,XFS_ILOCK_EXCL ); -xfs_ilock (mp->m_rsumip,XFS_ILOCK_EXCL |XFS_ILOCK_RTSUM ); -if (tp) -xfs_trans_ijoin (tp, mp->m_rsumip,XFS_ILOCK_EXCL ); +} + +/* +* Join both realtime free space metadata inodes to the transaction. The +* ILOCKs will be released on transaction commit. +*/ +void +xfs_rtbitmap_trans_join ( +struct xfs_trans *tp) +{ +xfs_trans_ijoin (tp, tp->t_mountp->m_rbmip,XFS_ILOCK_EXCL ); +xfs_trans_ijoin (tp, tp->t_mountp->m_rsumip,XFS_ILOCK_EXCL ); }/* Unlock both realtime free space metadata inodes after a freespace update. */
libxfs/xfs_rtbitmap.h +4 -2
@@ -346,8 +346,9 @@ unsigned long long xfs_rtsummary_wordcount(struct xfs_mount *mp,int xfs_rtfile_initialize_blocks (struct xfs_inode *ip,xfs_fileoff_t offset_fsb,xfs_fileoff_t end_fsb,void *data); -void xfs_rtbitmap_lock (struct xfs_trans *tp,struct xfs_mount *mp); +void xfs_rtbitmap_lock (struct xfs_mount *mp);void xfs_rtbitmap_unlock (struct xfs_mount *mp); +void xfs_rtbitmap_trans_join (struct xfs_trans *tp);/* Lock the rt bitmap inode in shared mode */ #define XFS_RBMLOCK_BITMAP (1U << 0)
@@ -376,7 +377,8 @@ xfs_rtbitmap_blockcount(struct xfs_mount *mp, xfs_rtbxlen_t rtextents)# define xfs_rtbitmap_wordcount (mp, r) (0)# define xfs_rtsummary_blockcount (mp, l, b) (0)# define xfs_rtsummary_wordcount (mp, l, b) (0) -# define xfs_rtbitmap_lock (tp, mp) do { } while (0) +# define xfs_rtbitmap_lock (mp) do { } while (0) +# define xfs_rtbitmap_trans_join (tp) do { } while (0)# define xfs_rtbitmap_unlock (mp) do { } while (0)# define xfs_rtbitmap_lock_shared (mp, lf) do { } while (0)# define xfs_rtbitmap_unlock_shared (mp, lf) do { } while (0)
PATCH 14/41 xfs: ensure rtx mask/shift are correct after growfs
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 86a0264ef26e90214a5bd74c72fb6e3455403bcf
When growfs sets an extent size, it doesn't updated the m_rtxblklog and m_rtxblkmask values, which could lead to incorrect usage of them if they were set before and can't be used for the new extent size.
Add a xfs_mount_sb_set_rextsize helper that updates the two fields, and also use it when calculating the new RT geometry instead of disabling the optimization there.
--- libxfs/xfs_sb.c | 12 ++++++++++-- libxfs/xfs_sb.h | 2 ++ 2 files changed, 12 insertions(+), 2 deletions(-)
libxfs/xfs_sb.c +10 -2
@@ -962,6 +962,15 @@ const struct xfs_buf_ops xfs_sb_quiet_buf_ops = { .verify_write = xfs_sb_write_verify, }; +void +xfs_mount_sb_set_rextsize ( +struct xfs_mount *mp, +struct xfs_sb *sbp) +{ + mp->m_rtxblklog =log2_if_power2 (sbp->sb_rextsize); + mp->m_rtxblkmask =mask64_if_power2 (sbp->sb_rextsize); +} +/* * xfs_mount_common *
@@ -986,8 +995,7 @@ xfs_sb_mount_common( mp->m_blockmask = sbp->sb_blocksize -1 ; mp->m_blockwsize = sbp->sb_blocksize >>XFS_WORDLOG ; mp->m_blockwmask = mp->m_blockwsize -1 ; - mp->m_rtxblklog =log2_if_power2 (sbp->sb_rextsize); - mp->m_rtxblkmask =mask64_if_power2 (sbp->sb_rextsize); +xfs_mount_sb_set_rextsize (mp, sbp); mp->m_alloc_mxr[0 ] =xfs_allocbt_maxrecs (mp, sbp->sb_blocksize,1 ); mp->m_alloc_mxr[1 ] =xfs_allocbt_maxrecs (mp, sbp->sb_blocksize,0 );
libxfs/xfs_sb.h +2 -0
@@ -17,6 +17,8 @@ extern void xfs_log_sb(struct xfs_trans *tp);extern int xfs_sync_sb (struct xfs_mount *mp,bool wait);extern int xfs_sync_sb_buf (struct xfs_mount *mp);extern void xfs_sb_mount_common (struct xfs_mount *mp,struct xfs_sb *sbp); +void xfs_mount_sb_set_rextsize (struct xfs_mount *mp, +struct xfs_sb *sbp);extern void xfs_sb_from_disk (struct xfs_sb *to,struct xfs_dsb *from);extern void xfs_sb_to_disk (struct xfs_dsb *to,struct xfs_sb *from);extern void xfs_sb_quota_from_disk (struct xfs_sb *sbp);
PATCH 15/41 xfs: remove xfs_rtb_to_rtxrem
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: fa0fc38b255cc88aef31ff13b5593e27622204e1
Simplify the number of block number conversion helpers by removing xfs_rtb_to_rtxrem. Any recent compiler is smart enough to eliminate the double divisions if using separate xfs_rtb_to_rtx and xfs_rtb_to_rtxoff calls.
--- libxfs/xfs_rtbitmap.c | 9 ++++----- libxfs/xfs_rtbitmap.h | 18 ------------------ 2 files changed, 4 insertions(+), 23 deletions(-)
libxfs/xfs_rtbitmap.c +4 -5
@@ -1020,25 +1020,24 @@ xfs_rtfree_blocks( xfs_filblks_t rtlen) {struct xfs_mount *mp = tp->t_mountp; -xfs_rtxnum_t start; -xfs_filblks_t len;xfs_extlen_t mod;ASSERT (rtlen <=XFS_MAX_BMBT_EXTLEN ); - len =xfs_rtb_to_rtxrem (mp, rtlen, &mod); + mod =xfs_rtb_to_rtxoff (mp, rtlen);if (mod) {ASSERT (mod ==0 );return -EIO ; } - start =xfs_rtb_to_rtxrem (mp, rtbno, &mod); + mod =xfs_rtb_to_rtxoff (mp, rtbno);if (mod) {ASSERT (mod ==0 );return -EIO ; } -return xfs_rtfree_extent (tp, start, len); +return xfs_rtfree_extent (tp,xfs_rtb_to_rtx (mp, rtbno), +xfs_rtb_to_rtx (mp, rtlen)); }/* Find all the free records within a given range. */
libxfs/xfs_rtbitmap.h +0 -18
@@ -86,24 +86,6 @@ xfs_rtb_to_rtxoff(return do_div (rtbno, mp->m_sb.sb_rextsize); } -/* -* Crack an rt block number into an rt extent number and an offset within that -* rt extent. Returns the rt extent number directly and the offset in @off. -*/ -static inline xfs_rtxnum_t -xfs_rtb_to_rtxrem ( -struct xfs_mount *mp, -xfs_rtblock_t rtbno, -xfs_extlen_t *off) -{ -if (likely (mp->m_rtxblklog >=0 )) { - *off = rtbno & mp->m_rtxblkmask; -return rtbno >> mp->m_rtxblklog; - } - -return div_u64_rem (rtbno, mp->m_sb.sb_rextsize, off); -} -/* * Convert an rt block number into an rt extent number, rounding up to the next * rt extent if the rt block is not aligned to an rt extent boundary.
PATCH 16/41 xfs: simplify xfs_rtalloc_query_range
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: df8b181f1551581e96076a653cdca43468093c0f
There isn't much of a good reason to pass the xfs_rtalloc_rec structures that describe extents to xfs_rtalloc_query_range as we really just want a lower and upper bound xfs_rtxnum_t. Pass the rtxnum directly and simply the interface.
--- libxfs/xfs_rtbitmap.c | 42 +++++++++++++++++------------------------- libxfs/xfs_rtbitmap.h | 3 +-- 2 files changed, 18 insertions(+), 27 deletions(-)
libxfs/xfs_rtbitmap.c +17 -25
@@ -1045,8 +1045,8 @@ intxfs_rtalloc_query_range ( struct xfs_mount *mp, struct xfs_trans *tp, - const struct xfs_rtalloc_rec *low_rec, - const struct xfs_rtalloc_rec *high_rec, + xfs_rtxnum_t start, + xfs_rtxnum_t end, xfs_rtalloc_query_range_fn fn, void *priv) {
@@ -1054,45 +1054,42 @@ xfs_rtalloc_query_range( .mp = mp, .tp = tp, }; -struct xfs_rtalloc_rec rec; -xfs_rtxnum_t rtstart; -xfs_rtxnum_t rtend; -xfs_rtxnum_t high_key; -int is_free;int error =0 ; -if (low_rec->ar_startext > high_rec->ar_startext) +if (start > end)return -EINVAL ; -if (low_rec->ar_startext >= mp->m_sb.sb_rextents || - low_rec->ar_startext == high_rec->ar_startext) +if (start == end || start >= mp->m_sb.sb_rextents)return 0 ; - high_key =min (high_rec->ar_startext, mp->m_sb.sb_rextents -1 ); + end =min (end, mp->m_sb.sb_rextents -1 );/* Iterate the bitmap, looking for discrepancies. */ - rtstart = low_rec->ar_startext; -while (rtstart <= high_key) { +while (start <= end) { +struct xfs_rtalloc_rec rec; +int is_free; +xfs_rtxnum_t rtend; +/* Is the first block free? */ - error =xfs_rtcheck_range (&args, rtstart,1 ,1 , &rtend, + error =xfs_rtcheck_range (&args, start,1 ,1 , &rtend, &is_free);if (error)break ;/* How long does the extent go for? */ - error =xfs_rtfind_forw (&args, rtstart, high_key, &rtend); + error =xfs_rtfind_forw (&args, start, end, &rtend);if (error)break ;if (is_free) { - rec.ar_startext = rtstart; - rec.ar_extcount = rtend - rtstart +1 ; + rec.ar_startext = start; + rec.ar_extcount = rtend - start +1 ; error =fn (mp, tp, &rec, priv);if (error)break ; } - rtstart = rtend +1 ; + start = rtend +1 ; }xfs_rtbuf_cache_relse (&args);
@@ -1107,13 +1104,8 @@ xfs_rtalloc_query_all(xfs_rtalloc_query_range_fn fn, void *priv) { - struct xfs_rtalloc_rec keys[2]; - - keys[0 ].ar_startext =0 ; - keys[1 ].ar_startext = mp->m_sb.sb_rextents -1 ; - keys[0 ].ar_extcount = keys[1 ].ar_extcount =0 ; - -return xfs_rtalloc_query_range (mp, tp, &keys[0 ], &keys[1 ], fn, priv); + return xfs_rtalloc_query_range(mp, tp, 0, mp->m_sb.sb_rextents - 1, fn, +priv ); }/* Is the given extent all free? */
libxfs/xfs_rtbitmap.h +1 -2
@@ -292,8 +292,7 @@ int xfs_rtmodify_summary(struct xfs_rtalloc_args *args, int log,int xfs_rtfree_range (struct xfs_rtalloc_args *args,xfs_rtxnum_t start,xfs_rtxlen_t len);int xfs_rtalloc_query_range (struct xfs_mount *mp,struct xfs_trans *tp, -const struct xfs_rtalloc_rec *low_rec, -const struct xfs_rtalloc_rec *high_rec, +xfs_rtxnum_t start,xfs_rtxnum_t end,xfs_rtalloc_query_range_fn fn,void *priv); int xfs_rtalloc_query_all(struct xfs_mount *mp, struct xfs_trans *tp, xfs_rtalloc_query_range_fn fn,
PATCH 17/41 xfs: clean up the ISVALID macro in xfs_bmap_adjacent
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 1e21d1897f935815618d419c94e88452070ec8e5
Turn the ISVALID macro defined and used inside in xfs_bmap_adjacent that relies on implict context into a proper inline function.
--- libxfs/xfs_bmap.c | 55 +++++++++++++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 23 deletions(-)
libxfs/xfs_bmap.c +32 -23
@@ -3106,6 +3106,23 @@ xfs_bmap_extsize_align(return 0 ; } +static inline bool +xfs_bmap_adjacent_valid ( +struct xfs_bmalloca *ap, +xfs_fsblock_t x, +xfs_fsblock_t y) +{ +struct xfs_mount *mp = ap->ip->i_mount; + +if (XFS_IS_REALTIME_INODE (ap->ip) && + (ap->datatype &XFS_ALLOC_USERDATA )) +return x < mp->m_sb.sb_rblocks; + +return XFS_FSB_TO_AGNO (mp, x) ==XFS_FSB_TO_AGNO (mp, y) && +XFS_FSB_TO_AGNO (mp, x) < mp->m_sb.sb_agcount && +XFS_FSB_TO_AGBNO (mp, x) < mp->m_sb.sb_agblocks; +} +#define XFS_ALLOC_GAP_UNITS 4/* returns true if ap->blkno was modified */
@@ -3113,36 +3130,25 @@ boolxfs_bmap_adjacent ( struct xfs_bmalloca *ap) /* bmap alloc argument struct */ { -xfs_fsblock_t adjust;/* adjustment to block numbers */ -xfs_mount_t *mp;/* mount point structure */ -int rt;/* true if inode is realtime */ +xfs_fsblock_t adjust;/* adjustment to block numbers */ -#define ISVALID (x,y) \ - (rt ? \ - (x) < mp->m_sb.sb_rblocks : \ - XFS_FSB_TO_AGNO(mp, x) == XFS_FSB_TO_AGNO(mp, y) && \ - XFS_FSB_TO_AGNO(mp, x) < mp->m_sb.sb_agcount && \ - XFS_FSB_TO_AGBNO(mp, x) < mp->m_sb.sb_agblocks) - - mp = ap->ip->i_mount; - rt =XFS_IS_REALTIME_INODE (ap->ip) && - (ap->datatype &XFS_ALLOC_USERDATA );/* * If allocating at eof, and there's a previous real block, * try to use its last block as our starting point. */ if (ap->eof && ap->prev.br_startoff !=NULLFILEOFF && !isnullstartblock (ap->prev.br_startblock) && -ISVALID (ap->prev.br_startblock + ap->prev.br_blockcount, - ap->prev.br_startblock)) { +xfs_bmap_adjacent_valid (ap, + ap->prev.br_startblock + ap->prev.br_blockcount, + ap->prev.br_startblock)) { ap->blkno = ap->prev.br_startblock + ap->prev.br_blockcount;/* * Adjust for the gap between prevp and us. */ adjust = ap->offset - (ap->prev.br_startoff + ap->prev.br_blockcount); -if (adjust && -ISVALID (ap->blkno + adjust, ap->prev.br_startblock)) +if (adjust &&xfs_bmap_adjacent_valid (ap, ap->blkno + adjust, + ap->prev.br_startblock)) ap->blkno += adjust;return true; }
@@ -3165,7 +3171,8 @@ xfs_bmap_adjacent( !isnullstartblock (ap->prev.br_startblock) && (prevbno = ap->prev.br_startblock + ap->prev.br_blockcount) && - ISVALID(prevbno, ap->prev.br_startblock)) { +xfs_bmap_adjacent_valid (ap, prevbno, + ap->prev.br_startblock)) {/* * Calculate gap to end of previous block. */
@@ -3181,8 +3188,8 @@ xfs_bmap_adjacent(* number, then just use the end of the previous block. */ if (prevdiff <=XFS_ALLOC_GAP_UNITS * ap->length && -ISVALID (prevbno + prevdiff, - ap->prev.br_startblock)) +xfs_bmap_adjacent_valid (ap, prevbno + prevdiff, + ap->prev.br_startblock)) prevbno += adjust;else prevdiff += adjust;
@@ -3214,9 +3221,11 @@ xfs_bmap_adjacent(* offset by our length. */ if (gotdiff <=XFS_ALLOC_GAP_UNITS * ap->length && -ISVALID (gotbno - gotdiff, gotbno)) +xfs_bmap_adjacent_valid (ap, gotbno - gotdiff, + gotbno)) gotbno -= adjust; -else if (ISVALID (gotbno - ap->length, gotbno)) { +else if (xfs_bmap_adjacent_valid (ap, gotbno - ap->length, + gotbno)) { gotbno -= ap->length; gotdiff += adjust - ap->length; }else
@@ -3244,7 +3253,7 @@ xfs_bmap_adjacent(return true; } } -#undef ISVALID +return false; }
PATCH 18/41 xfs: remove xfs_{rtbitmap,rtsummary}_wordcount
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 1fc51cf11dd8b26856ae1c4111e402caec73019c
xfs_rtbitmap_wordcount and xfs_rtsummary_wordcount are currently unused, so remove them to simplify refactoring other rtbitmap helpers. They can be added back or simply open coded when actually needed.
--- db/check.c | 3 +-- libxfs/xfs_rtbitmap.c | 31 ------------------------------- libxfs/xfs_rtbitmap.h | 7 ------- repair/rt.c | 5 ++--- 4 files changed, 3 insertions(+), 43 deletions(-)
db/check.c +1 -2
@@ -1958,8 +1958,7 @@ init( dbmap[c] =xcalloc (mp->m_sb.sb_rblocks,sizeof (**dbmap)); inomap[c] =xcalloc (mp->m_sb.sb_rblocks,sizeof (**inomap)); - words =libxfs_rtsummary_wordcount (mp, mp->m_rsumlevels, - mp->m_sb.sb_rbmblocks); + words = mp->m_rsumsize >>XFS_WORDLOG ; sumfile =xcalloc (words,sizeof (union xfs_suminfo_raw )); sumcompute =xcalloc (words,sizeof (union xfs_suminfo_raw )); }
libxfs/xfs_rtbitmap.c +0 -31
@@ -1146,21 +1146,6 @@ xfs_rtbitmap_blockcount(return howmany_64 (rtextents,NBBY * mp->m_sb.sb_blocksize); } -/* -* Compute the number of rtbitmap words needed to populate every block of a -* bitmap that is large enough to track the given number of rt extents. -*/ -unsigned long long -xfs_rtbitmap_wordcount ( -struct xfs_mount *mp, -xfs_rtbxlen_t rtextents) -{ -xfs_filblks_t blocks; - - blocks =xfs_rtbitmap_blockcount (mp, rtextents); -return XFS_FSB_TO_B (mp, blocks) >>XFS_WORDLOG ; -} -/* Compute the number of rtsummary blocks needed to track the given rt space. */ xfs_filblks_t xfs_rtsummary_blockcount(
@@ -1174,22 +1159,6 @@ xfs_rtsummary_blockcount(return XFS_B_TO_FSB (mp, rsumwords <<XFS_WORDLOG ); } -/* -* Compute the number of rtsummary info words needed to populate every block of -* a summary file that is large enough to track the given rt space. -*/ -unsigned long long -xfs_rtsummary_wordcount ( -struct xfs_mount *mp, -unsigned int rsumlevels, -xfs_extlen_t rbmblocks) -{ -xfs_filblks_t blocks; - - blocks =xfs_rtsummary_blockcount (mp, rsumlevels, rbmblocks); -return XFS_FSB_TO_B (mp, blocks) >>XFS_WORDLOG ; -} -/* Lock both realtime free space metadata inodes for a freespace update. */ void xfs_rtbitmap_lock(
libxfs/xfs_rtbitmap.h +0 -7
@@ -316,13 +316,8 @@ int xfs_rtfree_blocks(struct xfs_trans *tp, xfs_fsblock_t rtbno,xfs_filblks_t xfs_rtbitmap_blockcount (struct xfs_mount *mp,xfs_rtbxlen_t rtextents); -unsigned long long xfs_rtbitmap_wordcount (struct xfs_mount *mp, -xfs_rtbxlen_t rtextents); -xfs_filblks_t xfs_rtsummary_blockcount (struct xfs_mount *mp,unsigned int rsumlevels,xfs_extlen_t rbmblocks); -unsigned long long xfs_rtsummary_wordcount (struct xfs_mount *mp, -unsigned int rsumlevels,xfs_extlen_t rbmblocks);int xfs_rtfile_initialize_blocks (struct xfs_inode *ip,xfs_fileoff_t offset_fsb,xfs_fileoff_t end_fsb,void *data);
@@ -355,9 +350,7 @@ xfs_rtbitmap_blockcount(struct xfs_mount *mp, xfs_rtbxlen_t rtextents)/* shut up gcc */ return 0 ; } -# define xfs_rtbitmap_wordcount (mp, r) (0)# define xfs_rtsummary_blockcount (mp, l, b) (0) -# define xfs_rtsummary_wordcount (mp, l, b) (0)# define xfs_rtbitmap_lock (mp) do { } while (0)# define xfs_rtbitmap_trans_join (tp) do { } while (0)# define xfs_rtbitmap_unlock (mp) do { } while (0)
repair/rt.c +2 -3
@@ -27,14 +27,13 @@ rtinit(xfs_mount_t *mp)* information. The rtbitmap buffer must be large enough to compare * against any unused bytes in the last block of the file. */ - wordcnt =libxfs_rtbitmap_wordcount (mp, mp->m_sb.sb_rextents); + wordcnt =XFS_FSB_TO_B (mp, mp->m_sb.sb_rbmblocks) >>XFS_WORDLOG ; btmcompute =calloc (wordcnt,sizeof (union xfs_rtword_raw ));if (!btmcompute)do_error (_ ("couldn't allocate memory for incore realtime bitmap.\n" )); - wordcnt =libxfs_rtsummary_wordcount (mp, mp->m_rsumlevels, - mp->m_sb.sb_rbmblocks); + wordcnt = mp->m_rsumsize >>XFS_WORDLOG ; sumcompute =calloc (wordcnt,sizeof (union xfs_suminfo_raw ));if (!sumcompute) do_error(
PATCH 19/41 xfs: replace m_rsumsize with m_rsumblocks
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 33912286cb1956920712aba8cb6f38e434824357
Track the RT summary file size in blocks, just like the RT bitmap file. While we have users of both units, blocks are used slightly more often and this matches the bitmap file for consistency.
--- db/check.c | 2 +- include/xfs_mount.h | 2 +- libxfs/init.c | 4 +--- libxfs/xfs_rtbitmap.c | 2 +- libxfs/xfs_trans_resv.c | 2 +- mkfs/proto.c | 12 +++++------- repair/dinode.c | 7 ++++--- repair/phase6.c | 16 ++++++---------- repair/rt.c | 4 ++-- 9 files changed, 22 insertions(+), 29 deletions(-)
db/check.c +1 -1
@@ -1958,7 +1958,7 @@ init( dbmap[c] =xcalloc (mp->m_sb.sb_rblocks,sizeof (**dbmap)); inomap[c] =xcalloc (mp->m_sb.sb_rblocks,sizeof (**inomap)); - words = mp->m_rsumsize >>XFS_WORDLOG ; + words =XFS_FSB_TO_B (mp, mp->m_rsumblocks) >>XFS_WORDLOG ; sumfile =xcalloc (words,sizeof (union xfs_suminfo_raw )); sumcompute =xcalloc (words,sizeof (union xfs_suminfo_raw )); }
include/xfs_mount.h +1 -1
@@ -50,7 +50,7 @@ typedef struct xfs_mount {xfs_agnumber_t m_maxagi;/* highest inode alloc group */ struct xfs_ino_geometry m_ino_geo;/* inode geometry */ uint m_rsumlevels;/* rt summary levels */ -uint m_rsumsize;/* size of rt summary, bytes */ +xfs_filblks_t m_rsumblocks;/* size of rt summary, FSBs */ /* * Optional cache of rt summary level per bitmap block with the * invariant that m_rsum_cache[bbno] <= the minimum i for which
libxfs/init.c +1 -3
@@ -293,7 +293,6 @@ rtmount_init( {struct xfs_buf *bp;/* buffer for last block of subvolume */ xfs_daddr_t d;/* address of last block of subvolume */ -unsigned int rsumblocks;int error;if (mp->m_sb.sb_rblocks ==0 )
@@ -319,9 +318,8 @@ rtmount_init(return -1 ; } mp->m_rsumlevels = mp->m_sb.sb_rextslog +1 ; - rsumblocks =xfs_rtsummary_blockcount (mp, mp->m_rsumlevels, + mp->m_rsumblocks =xfs_rtsummary_blockcount (mp, mp->m_rsumlevels, mp->m_sb.sb_rbmblocks); - mp->m_rsumsize =XFS_FSB_TO_B (mp, rsumblocks); mp->m_rbmip = mp->m_rsumip =NULL ;/*
libxfs/xfs_rtbitmap.c +1 -1
@@ -160,7 +160,7 @@ xfs_rtsummary_read_buf( {struct xfs_mount *mp = args->mp; -if (XFS_IS_CORRUPT (mp, block >=XFS_B_TO_FSB (mp, mp->m_rsumsize))) { +if (XFS_IS_CORRUPT (mp, block >= mp->m_rsumblocks)) {xfs_rt_mark_sick (args->mp,XFS_SICK_RT_SUMMARY );return -EFSCORRUPTED ; }
libxfs/xfs_trans_resv.c +1 -1
@@ -915,7 +915,7 @@ xfs_calc_growrtfree_reservation(return xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize) +xfs_calc_inode_res (mp,2 ) +xfs_calc_buf_res (1 , mp->m_sb.sb_blocksize) + -xfs_calc_buf_res (1 , mp->m_rsumsize); +xfs_calc_buf_res (1 ,XFS_FSB_TO_B (mp, mp->m_rsumblocks)); }/*
mkfs/proto.c +5 -7
@@ -813,7 +813,7 @@ rtsummary_create( {struct xfs_mount *mp = ip->i_mount; - ip->i_disk_size = mp->m_rsumsize; + ip->i_disk_size = mp->m_rsumblocks * mp->m_sb.sb_blocksize; mp->m_sb.sb_rsumino = ip->i_ino; mp->m_rsumip = ip;
@@ -874,25 +874,23 @@ rtsummary_init(struct xfs_trans *tp;struct xfs_bmbt_irec *ep;xfs_fileoff_t bno; -xfs_extlen_t nsumblocks;uint blocks;int i;int nmap;int error; - nsumblocks = mp->m_rsumsize >> mp->m_sb.sb_blocklog; - blocks = nsumblocks +XFS_BM_MAXLEVELS (mp,XFS_DATA_FORK ) -1 ; + blocks = mp->m_rsumblocks +XFS_BM_MAXLEVELS (mp,XFS_DATA_FORK ) -1 ; error = -libxfs_trans_alloc_rollable (mp, blocks, &tp);if (error)res_failed (error);libxfs_trans_ijoin (tp, mp->m_rsumip,0 ); bno =0 ; -while (bno < nsumblocks) { +while (bno < mp->m_rsumblocks) { nmap =XFS_BMAP_MAX_NMAP ; error = -libxfs_bmapi_write (tp, mp->m_rsumip, bno, - (xfs_extlen_t)(nsumblocks - bno), -0 , nsumblocks, map, &nmap); + (xfs_extlen_t)(mp->m_rsumblocks - bno), +0 , mp->m_rsumblocks, map, &nmap);if (error)fail (_ ("Allocation of the realtime summary failed" ), error);
repair/dinode.c +4 -3
@@ -1736,10 +1736,11 @@ _("realtime bitmap inode %" PRIu64 " has bad size %" PRId64 " (should be %" PRIubreak ;case XR_INO_RTSUM : -if (size != mp->m_rsumsize) { +if (size !=XFS_FSB_TO_B (mp, mp->m_rsumblocks)) {do_warn ( -_ ("realtime summary inode %" PRIu64" has bad size %" PRId64" (should be %d)\n" ), - lino, size, mp->m_rsumsize); +_ ("realtime summary inode %" PRIu64" has bad size %" PRIu64" (should be %" PRIu64")\n" ), + lino, size, +XFS_FSB_TO_B (mp, mp->m_rsumblocks));return 1 ; }break ;
repair/phase6.c +6 -10
@@ -633,12 +633,10 @@ fill_rsumino(xfs_mount_t *mp)int nmap;int error;xfs_fileoff_t bno; -xfs_fileoff_t end_bno;xfs_bmbt_irec_t map; smp = sumcompute; bno =0 ; - end_bno = mp->m_rsumsize >> mp->m_sb.sb_blocklog; error = -libxfs_trans_alloc_rollable (mp,10 , &tp);if (error)
@@ -651,7 +649,7 @@ fill_rsumino(xfs_mount_t *mp) error); } -while (bno < end_bno) { +while (bno < mp->m_rsumblocks) {struct xfs_rtalloc_args args = { .mp = mp, .tp = tp,
@@ -711,7 +709,6 @@ mk_rsumino(xfs_mount_t *mp)int i;int nmap;int error; -int nsumblocks;xfs_fileoff_t bno;xfs_bmbt_irec_t map[XFS_BMAP_MAX_NMAP ];uint blocks;
@@ -732,7 +729,7 @@ mk_rsumino(xfs_mount_t *mp)/* Reset the rt summary inode. */ reset_sbroot_ino (tp,S_IFREG , ip); - ip->i_disk_size = mp->m_rsumsize; + ip->i_disk_size = mp->m_rsumblocks * mp->m_sb.sb_blocksize;libxfs_trans_log_inode (tp, ip,XFS_ILOG_CORE ); error = -libxfs_trans_commit (tp);if (error)
@@ -742,19 +739,18 @@ mk_rsumino(xfs_mount_t *mp)* then allocate blocks for file and fill with zeroes (stolen * from mkfs) */ - nsumblocks = mp->m_rsumsize >> mp->m_sb.sb_blocklog; - blocks = nsumblocks +XFS_BM_MAXLEVELS (mp,XFS_DATA_FORK ) -1 ; + blocks = mp->m_rsumblocks +XFS_BM_MAXLEVELS (mp,XFS_DATA_FORK ) -1 ; error = -libxfs_trans_alloc_rollable (mp, blocks, &tp);if (error)res_failed (error);libxfs_trans_ijoin (tp, ip,0 ); bno =0 ; -while (bno < nsumblocks) { +while (bno < mp->m_rsumblocks) { nmap =XFS_BMAP_MAX_NMAP ; error = -libxfs_bmapi_write (tp, ip, bno, - (xfs_extlen_t)(nsumblocks - bno), -0 , nsumblocks, map, &nmap); + (xfs_extlen_t)(mp->m_rsumblocks - bno), +0 , mp->m_rsumblocks, map, &nmap);if (error) { do_error( _("couldn't allocate realtime summary inode, error = %d\n"),
repair/rt.c +2 -2
@@ -33,7 +33,7 @@ rtinit(xfs_mount_t *mp)do_error (_ ("couldn't allocate memory for incore realtime bitmap.\n" )); - wordcnt = mp->m_rsumsize >>XFS_WORDLOG ; + wordcnt =XFS_FSB_TO_B (mp, mp->m_rsumblocks) >>XFS_WORDLOG ; sumcompute =calloc (wordcnt,sizeof (union xfs_suminfo_raw ));if (!sumcompute) do_error(
@@ -228,5 +228,5 @@ check_rtsummary(return ;check_rtfile_contents (mp,"rtsummary" , mp->m_sb.sb_rsumino, sumcompute, -XFS_B_TO_FSB (mp, mp->m_rsumsize)); + mp->m_rsumblocks); }
PATCH 20/41 xfs: fix a sloppy memory handling bug in xfs_iroot_realloc
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Source kernel commit: de55149b6639e903c4d06eb0474ab2c05060e61d
While refactoring code, I noticed that when xfs_iroot_realloc tries to shrink a bmbt root block, it allocates a smaller new block and then copies "records" and pointers to the new block. However, bmbt root blocks cannot ever be leaves, which means that it's not technically correct to copy records. We /should/ be copying keys.
Note that this has never resulted in actual memory corruption because sizeof(bmbt_rec) == (sizeof(bmbt_key) + sizeof(bmbt_ptr)). However, this will no longer be true when we start adding realtime rmap stuff, so fix this now.
--- libxfs/xfs_inode_fork.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
libxfs/xfs_inode_fork.c +5 -5
@@ -461,15 +461,15 @@ xfs_iroot_realloc( }/* -* Only copy the records and pointers if there are any. +* Only copy the keys and pointers if there are any. */ if (new_max >0 ) {/* -* First copy the records. +* First copy the keys. */ - op = (char *)XFS_BMBT_REC_ADDR (mp, ifp->if_broot,1 ); - np = (char *)XFS_BMBT_REC_ADDR (mp, new_broot,1 ); -memcpy (np, op, new_max * (uint )sizeof (xfs_bmbt_rec_t)); + op = (char *)XFS_BMBT_KEY_ADDR (mp, ifp->if_broot,1 ); + np = (char *)XFS_BMBT_KEY_ADDR (mp, new_broot,1 ); +memcpy (np, op, new_max * (uint )sizeof (xfs_bmbt_key_t));/* * Then copy the pointers.
PATCH 21/41 xfs: replace shouty XFS_BM{BT,DR} macros
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Source kernel commit: 79124b3740063573312de4b225407ebdae219275
Replace all the shouty bmap btree and bmap disk root macros with actual functions.
sed \
-e 's/XFS_BMBT_BLOCK_LEN/xfs_bmbt_block_len/g' \
-e 's/XFS_BMBT_REC_ADDR/xfs_bmbt_rec_addr/g' \
-e 's/XFS_BMBT_KEY_ADDR/xfs_bmbt_key_addr/g' \
-e 's/XFS_BMBT_PTR_ADDR/xfs_bmbt_ptr_addr/g' \
-e 's/XFS_BMDR_REC_ADDR/xfs_bmdr_rec_addr/g' \
-e 's/XFS_BMDR_KEY_ADDR/xfs_bmdr_key_addr/g' \
-e 's/XFS_BMDR_PTR_ADDR/xfs_bmdr_ptr_addr/g' \
-e 's/XFS_BMAP_BROOT_PTR_ADDR/xfs_bmap_broot_ptr_addr/g' \
-e 's/XFS_BMAP_BROOT_SPACE_CALC/xfs_bmap_broot_space_calc/g' \
-e 's/XFS_BMAP_BROOT_SPACE/xfs_bmap_broot_space/g' \
-e 's/XFS_BMDR_SPACE_CALC/xfs_bmdr_space_calc/g' \
-e 's/XFS_BMAP_BMDR_SPACE/xfs_bmap_bmdr_space/g' \
-i $(git ls-files fs/xfs/*.[ch] fs/xfs/libxfs/*.[ch] fs/xfs/scrub/*.[ch])
--- db/bmap.c | 10 +- db/bmap_inflate.c | 2 db/bmroot.c | 8 +- db/check.c | 8 +- db/frag.c | 8 +- db/metadump.c | 16 ++-- libxfs/xfs_attr_leaf.c | 8 +- libxfs/xfs_bmap.c | 40 +++++---- libxfs/xfs_bmap_btree.c | 18 ++-- libxfs/xfs_bmap_btree.h | 204 ++++++++++++++++++++++++++++++++--------------- libxfs/xfs_inode_fork.c | 30 +++---- libxfs/xfs_trans_resv.c | 2 repair/bmap_repair.c | 2 repair/dinode.c | 10 +- repair/prefetch.c | 8 +- repair/scan.c | 6 + 16 files changed, 228 insertions(+), 152 deletions(-)
db/bmap.c +5 -5
@@ -78,8 +78,8 @@ bmap(push_cur (); rblock = (xfs_bmdr_block_t *)XFS_DFORK_PTR (dip, whichfork); fsize =XFS_DFORK_SIZE (dip, mp, whichfork); - pp =XFS_BMDR_PTR_ADDR (rblock,1 ,libxfs_bmdr_maxrecs (fsize,0 )); - kp =XFS_BMDR_KEY_ADDR (rblock,1 ); + pp =xfs_bmdr_ptr_addr (rblock,1 ,libxfs_bmdr_maxrecs (fsize,0 )); + kp =xfs_bmdr_key_addr (rblock,1 ); bno =select_child (curoffset, kp, pp,be16_to_cpu (rblock->bb_numrecs));for (;;) {
@@ -88,9 +88,9 @@ bmap( block = (struct xfs_btree_block *)iocur_top->data;if (be16_to_cpu (block->bb_level) ==0 )break ; - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , + pp =xfs_bmbt_ptr_addr (mp, block,1 ,libxfs_bmbt_maxrecs (mp, mp->m_sb.sb_blocksize,0 )); - kp =XFS_BMBT_KEY_ADDR (mp, block,1 ); + kp =xfs_bmbt_key_addr (mp, block,1 ); bno =select_child (curoffset, kp, pp,be16_to_cpu (block->bb_numrecs)); }
@@ -98,7 +98,7 @@ bmap( nextbno =be64_to_cpu (block->bb_u.l.bb_rightsib); nextents =be16_to_cpu (block->bb_numrecs); xp = (xfs_bmbt_rec_t *) -XFS_BMBT_REC_ADDR (mp, block,1 ); +xfs_bmbt_rec_addr (mp, block,1 );for (ep = xp; ep < &xp[nextents] && n < nex; ep++) {if (!bmap_one_extent (ep, &curoffset, eoffset, &n, bep)) {
db/bmap_inflate.c +1 -1
@@ -269,7 +269,7 @@ iroot_size(unsigned int nr_this_level, void *priv) { - return XFS_BMAP_BROOT_SPACE_CALC(cur->bc_mp, nr_this_level); + return xfs_bmap_broot_space_calc(cur->bc_mp, nr_this_level); } static int
db/bmroot.c +4 -4
@@ -89,7 +89,7 @@ bmroota_key_offset( block = (xfs_bmdr_block_t *)((char *)obj +byteize (startoff));ASSERT (dip->di_forkoff !=0 && (char *)block ==XFS_DFORK_APTR (dip));ASSERT (be16_to_cpu (block->bb_level) >0 ); - kp =XFS_BMDR_KEY_ADDR (block, idx); + kp =xfs_bmdr_key_addr (block, idx);return bitize ((int )((char *)kp - (char *)block)); }
@@ -127,7 +127,7 @@ bmroota_ptr_offset( block = (xfs_bmdr_block_t *)((char *)obj +byteize (startoff));ASSERT (dip->di_forkoff !=0 && (char *)block ==XFS_DFORK_APTR (dip));ASSERT (be16_to_cpu (block->bb_level) >0 ); - pp =XFS_BMDR_PTR_ADDR (block, idx, + pp =xfs_bmdr_ptr_addr (block, idx,libxfs_bmdr_maxrecs (XFS_DFORK_ASIZE (dip, mp),0 ));return bitize ((int )((char *)pp - (char *)block)); }
@@ -185,7 +185,7 @@ bmrootd_key_offset(ASSERT (obj == iocur_top->data); block = (xfs_bmdr_block_t *)((char *)obj +byteize (startoff));ASSERT (be16_to_cpu (block->bb_level) >0 ); - kp =XFS_BMDR_KEY_ADDR (block, idx); + kp =xfs_bmdr_key_addr (block, idx);return bitize ((int )((char *)kp - (char *)block)); }
@@ -222,7 +222,7 @@ bmrootd_ptr_offset( dip = obj; block = (xfs_bmdr_block_t *)((char *)obj +byteize (startoff));ASSERT (be16_to_cpu (block->bb_level) >0 ); - pp =XFS_BMDR_PTR_ADDR (block, idx, + pp =xfs_bmdr_ptr_addr (block, idx,libxfs_bmdr_maxrecs (XFS_DFORK_DSIZE (dip, mp),0 ));return bitize ((int )((char *)pp - (char *)block)); }
db/check.c +4 -4
@@ -2321,13 +2321,13 @@ process_btinode(return ; }if (be16_to_cpu (dib->bb_level) ==0 ) { -xfs_bmbt_rec_t *rp =XFS_BMDR_REC_ADDR (dib,1 ); +xfs_bmbt_rec_t *rp =xfs_bmdr_rec_addr (dib,1 );process_bmbt_reclist (rp,be16_to_cpu (dib->bb_numrecs), type, id, totd, blkmapp); *nex +=be16_to_cpu (dib->bb_numrecs);return ; }else { - pp =XFS_BMDR_PTR_ADDR (dib,1 ,libxfs_bmdr_maxrecs ( + pp =xfs_bmdr_ptr_addr (dib,1 ,libxfs_bmdr_maxrecs (XFS_DFORK_SIZE (dip, mp, whichfork),0 ));for (i =0 ; i <be16_to_cpu (dib->bb_numrecs); i++) scan_lbtree(get_unaligned_be64(&pp[i]),
@@ -4357,7 +4357,7 @@ scanfunc_bmap( error++;return ; } - rp =XFS_BMBT_REC_ADDR (mp, block,1 ); + rp =xfs_bmbt_rec_addr (mp, block,1 ); *nex +=be16_to_cpu (block->bb_numrecs);process_bmbt_reclist (rp,be16_to_cpu (block->bb_numrecs), type, id, totd, blkmapp);
@@ -4373,7 +4373,7 @@ scanfunc_bmap( error++;return ; } - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[0 ]); + pp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[0 ]);for (i =0 ; i <be16_to_cpu (block->bb_numrecs); i++)scan_lbtree (be64_to_cpu (pp[i]), level, scanfunc_bmap, type, id, totd, toti, nex, blkmapp,0 , btype);
db/frag.c +4 -4
@@ -243,11 +243,11 @@ process_btinode( dib = (xfs_bmdr_block_t *)XFS_DFORK_PTR (dip, whichfork);if (be16_to_cpu (dib->bb_level) ==0 ) { -xfs_bmbt_rec_t *rp =XFS_BMDR_REC_ADDR (dib,1 ); +xfs_bmbt_rec_t *rp =xfs_bmdr_rec_addr (dib,1 );process_bmbt_reclist (rp,be16_to_cpu (dib->bb_numrecs), extmapp);return ; } - pp =XFS_BMDR_PTR_ADDR (dib,1 , + pp =xfs_bmdr_ptr_addr (dib,1 ,libxfs_bmdr_maxrecs (XFS_DFORK_SIZE (dip, mp, whichfork),0 ));for (i =0 ; i <be16_to_cpu (dib->bb_numrecs); i++) scan_lbtree(get_unaligned_be64(&pp[i]),
@@ -437,7 +437,7 @@ scanfunc_bmap( nrecs, typtab[btype].name);return ; } - rp =XFS_BMBT_REC_ADDR (mp, block,1 ); + rp =xfs_bmbt_rec_addr (mp, block,1 );process_bmbt_reclist (rp, nrecs, extmapp);return ; }
@@ -447,7 +447,7 @@ scanfunc_bmap( nrecs, typtab[btype].name);return ; } - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[0 ]); + pp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[0 ]);for (i =0 ; i < nrecs; i++)scan_lbtree (be64_to_cpu (pp[i]), level, scanfunc_bmap, extmapp, btype);
db/metadump.c +8 -8
@@ -249,8 +249,8 @@ zero_btree_node(if (nrecs > mp->m_bmap_dmxr[1 ])return ; - bkp =XFS_BMBT_KEY_ADDR (mp, block,1 ); - bpp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[1 ]); + bkp =xfs_bmbt_key_addr (mp, block,1 ); + bpp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[1 ]); zp1 = (char *)&bkp[nrecs]; zp2 = (char *)&bpp[nrecs]; key_end = (char *)bpp;
@@ -315,7 +315,7 @@ zero_btree_leaf(if (nrecs > mp->m_bmap_dmxr[0 ])return ; - brp =XFS_BMBT_REC_ADDR (mp, block,1 ); + brp =xfs_bmbt_rec_addr (mp, block,1 ); zp = (char *)&brp[nrecs];break ;case TYP_INOBT :
@@ -2113,7 +2113,7 @@ scanfunc_bmap( typtab[btype].name, agno, agbno);return 1 ; } -return process_bmbt_reclist (XFS_BMBT_REC_ADDR (mp, block,1 ), +return process_bmbt_reclist (xfs_bmbt_rec_addr (mp, block,1 ), nrecs, *(typnm_t *)arg); }
@@ -2123,7 +2123,7 @@ scanfunc_bmap( nrecs, typtab[btype].name, agno, agbno);return 1 ; } - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[1 ]); + pp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[1 ]);for (i =0 ; i < nrecs; i++) {xfs_agnumber_t ag;xfs_agblock_t bno;
@@ -2176,7 +2176,7 @@ process_btinode( }if (level ==0 ) { -return process_bmbt_reclist (XFS_BMDR_REC_ADDR (dib,1 ), +return process_bmbt_reclist (xfs_bmdr_rec_addr (dib,1 ), nrecs, itype); }
@@ -2189,13 +2189,13 @@ process_btinode(return 1 ; } - pp =XFS_BMDR_PTR_ADDR (dib,1 , maxrecs); + pp =xfs_bmdr_ptr_addr (dib,1 , maxrecs);if (metadump.zero_stale_data) {char *top;/* Unused btree key space */ - top = (char *)XFS_BMDR_KEY_ADDR (dib, nrecs +1 ); + top = (char *)xfs_bmdr_key_addr (dib, nrecs +1 );memset (top,0 , (char *)pp - top);/* Unused btree ptr space */
libxfs/xfs_attr_leaf.c +4 -4
@@ -683,7 +683,7 @@ xfs_attr_shortform_bytesfit(*/ if (!dp->i_forkoff && dp->i_df.if_bytes >xfs_default_attroffset (dp)) - dsize =XFS_BMDR_SPACE_CALC (MINDBTPTRS ); + dsize =xfs_bmdr_space_calc (MINDBTPTRS );break ;case XFS_DINODE_FMT_BTREE :/*
@@ -697,7 +697,7 @@ xfs_attr_shortform_bytesfit(return 0 ;return dp->i_forkoff; } - dsize =XFS_BMAP_BROOT_SPACE (mp, dp->i_df.if_broot); + dsize =xfs_bmap_bmdr_space (dp->i_df.if_broot);break ; }
@@ -705,11 +705,11 @@ xfs_attr_shortform_bytesfit(* A data fork btree root must have space for at least * MINDBTPTRS key/ptr pairs if the data fork is small or empty. */ - minforkoff =max_t (int64_t, dsize,XFS_BMDR_SPACE_CALC (MINDBTPTRS )); + minforkoff =max_t (int64_t, dsize,xfs_bmdr_space_calc (MINDBTPTRS )); minforkoff =roundup (minforkoff,8 ) >>3 ;/* attr fork btree root can have at least this many key/ptr pairs */ - maxforkoff =XFS_LITINO (mp) -XFS_BMDR_SPACE_CALC (MINABTPTRS ); + maxforkoff =XFS_LITINO (mp) -xfs_bmdr_space_calc (MINABTPTRS ); maxforkoff = maxforkoff >>3 ;/* rounded down */ if (offset >= maxforkoff)
libxfs/xfs_bmap.c +20 -20
@@ -73,9 +73,9 @@ xfs_bmap_compute_maxlevels( maxleafents =xfs_iext_max_nextents (xfs_has_large_extent_counts (mp), whichfork);if (whichfork ==XFS_DATA_FORK ) - sz =XFS_BMDR_SPACE_CALC (MINDBTPTRS ); + sz =xfs_bmdr_space_calc (MINDBTPTRS );else - sz =XFS_BMDR_SPACE_CALC (MINABTPTRS ); + sz =xfs_bmdr_space_calc (MINABTPTRS ); maxrootrecs =xfs_bmdr_maxrecs (sz,0 ); minleafrecs = mp->m_bmap_dmnr[0 ];
@@ -96,8 +96,8 @@ xfs_bmap_compute_attr_offset( struct xfs_mount *mp) {if (mp->m_sb.sb_inodesize ==256 ) -return XFS_LITINO (mp) -XFS_BMDR_SPACE_CALC (MINABTPTRS ); -return XFS_BMDR_SPACE_CALC (6 *MINABTPTRS ); +return XFS_LITINO (mp) -xfs_bmdr_space_calc (MINABTPTRS ); +return xfs_bmdr_space_calc (6 *MINABTPTRS ); }STATIC int/* error */
@@ -292,7 +292,7 @@ xfs_check_block( prevp =NULL ;for ( i =1 ; i <=xfs_btree_get_numrecs (block); i++) { dmxr = mp->m_bmap_dmxr[0 ]; - keyp =XFS_BMBT_KEY_ADDR (mp, block, i); + keyp =xfs_bmbt_key_addr (mp, block, i);if (prevp) { ASSERT(be64_to_cpu(prevp->br_startoff) <
@@ -304,15 +304,15 @@ xfs_check_block(* Compare the block numbers to see if there are dups. */ if (root) - pp =XFS_BMAP_BROOT_PTR_ADDR (mp, block, i, sz); + pp =xfs_bmap_broot_ptr_addr (mp, block, i, sz);else - pp =XFS_BMBT_PTR_ADDR (mp, block, i, dmxr); + pp =xfs_bmbt_ptr_addr (mp, block, i, dmxr);for (j = i+1 ; j <=be16_to_cpu (block->bb_numrecs); j++) {if (root) - thispa =XFS_BMAP_BROOT_PTR_ADDR (mp, block, j, sz); + thispa =xfs_bmap_broot_ptr_addr (mp, block, j, sz);else - thispa =XFS_BMBT_PTR_ADDR (mp, block, j, dmxr); + thispa =xfs_bmbt_ptr_addr (mp, block, j, dmxr);if (*thispa == *pp) { xfs_warn(mp, "%s: thispa(%d) == pp(%d) %lld", __func__, j, i,
@@ -367,7 +367,7 @@ xfs_bmap_check_leaf_extents( level =be16_to_cpu (block->bb_level);ASSERT (level >0 );xfs_check_block (block, mp,1 , ifp->if_broot_bytes); - pp =XFS_BMAP_BROOT_PTR_ADDR (mp, block,1 , ifp->if_broot_bytes); + pp =xfs_bmap_broot_ptr_addr (mp, block,1 , ifp->if_broot_bytes); bno =be64_to_cpu (*pp);ASSERT (bno !=NULLFSBLOCK );
@@ -400,7 +400,7 @@ xfs_bmap_check_leaf_extents(*/ xfs_check_block (block, mp,0 ,0 ); - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[1 ]); + pp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[1 ]); bno =be64_to_cpu (*pp);if (XFS_IS_CORRUPT (mp, !xfs_verify_fsbno (mp, bno))) {xfs_btree_mark_sick (cur);
@@ -440,14 +440,14 @@ xfs_bmap_check_leaf_extents(* conform with the first entry in this one. */ - ep =XFS_BMBT_REC_ADDR (mp, block,1 ); + ep =xfs_bmbt_rec_addr (mp, block,1 );if (i) {ASSERT (xfs_bmbt_disk_get_startoff (&last) +xfs_bmbt_disk_get_blockcount (&last) <=xfs_bmbt_disk_get_startoff (ep)); }for (j =1 ; j < num_recs; j++) { - nextp =XFS_BMBT_REC_ADDR (mp, block, j +1 ); + nextp =xfs_bmbt_rec_addr (mp, block, j +1 );ASSERT (xfs_bmbt_disk_get_startoff (ep) +xfs_bmbt_disk_get_blockcount (ep) <=xfs_bmbt_disk_get_startoff (nextp));
@@ -580,7 +580,7 @@ xfs_bmap_btree_to_extents(ASSERT (be16_to_cpu (rblock->bb_numrecs) ==1 );ASSERT (xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes,0 ) ==1 ); - pp =XFS_BMAP_BROOT_PTR_ADDR (mp, rblock,1 , ifp->if_broot_bytes); + pp =xfs_bmap_broot_ptr_addr (mp, rblock,1 , ifp->if_broot_bytes); cbno =be64_to_cpu (*pp);#ifdef DEBUG if (XFS_IS_CORRUPT (cur->bc_mp, !xfs_verify_fsbno (mp, cbno))) {
@@ -708,7 +708,7 @@ xfs_bmap_extents_to_btree(for_each_xfs_iext (ifp, &icur, &rec) {if (isnullstartblock (rec.br_startblock))continue ; - arp =XFS_BMBT_REC_ADDR (mp, ablock,1 + cnt); + arp =xfs_bmbt_rec_addr (mp, ablock,1 + cnt);xfs_bmbt_disk_set_all (arp, &rec); cnt++; }
@@ -718,10 +718,10 @@ xfs_bmap_extents_to_btree(/* * Fill in the root key and pointer. */ - kp =XFS_BMBT_KEY_ADDR (mp, block,1 ); - arp =XFS_BMBT_REC_ADDR (mp, ablock,1 ); + kp =xfs_bmbt_key_addr (mp, block,1 ); + arp =xfs_bmbt_rec_addr (mp, ablock,1 ); kp->br_startoff =cpu_to_be64 (xfs_bmbt_disk_get_startoff (arp)); - pp =XFS_BMBT_PTR_ADDR (mp, block,1 ,xfs_bmbt_get_maxrecs (cur, + pp =xfs_bmbt_ptr_addr (mp, block,1 ,xfs_bmbt_get_maxrecs (cur,be16_to_cpu (block->bb_level))); *pp =cpu_to_be64 (args.fsbno);
@@ -890,7 +890,7 @@ xfs_bmap_add_attrfork_btree( mp = ip->i_mount; -if (XFS_BMAP_BMDR_SPACE (block) <=xfs_inode_data_fork_size (ip)) +if (xfs_bmap_bmdr_space (block) <=xfs_inode_data_fork_size (ip)) *flags |=XFS_ILOG_DBROOT ;else { cur =xfs_bmbt_init_cursor (mp, tp, ip,XFS_DATA_FORK );
@@ -1154,7 +1154,7 @@ xfs_iread_bmbt_block( }/* Copy records into the incore cache. */ - frp =XFS_BMBT_REC_ADDR (mp, block,1 ); + frp =xfs_bmbt_rec_addr (mp, block,1 );for (j =0 ; j < num_recs; j++, frp++, ir->loaded++) {struct xfs_bmbt_irec new;xfs_failaddr_t fa;
libxfs/xfs_bmap_btree.c +9 -9
@@ -64,10 +64,10 @@ xfs_bmdr_to_bmbt(ASSERT (be16_to_cpu (rblock->bb_level) >0 ); rblock->bb_numrecs = dblock->bb_numrecs; dmxr =xfs_bmdr_maxrecs (dblocklen,0 ); - fkp =XFS_BMDR_KEY_ADDR (dblock,1 ); - tkp =XFS_BMBT_KEY_ADDR (mp, rblock,1 ); - fpp =XFS_BMDR_PTR_ADDR (dblock,1 , dmxr); - tpp =XFS_BMAP_BROOT_PTR_ADDR (mp, rblock,1 , rblocklen); + fkp =xfs_bmdr_key_addr (dblock,1 ); + tkp =xfs_bmbt_key_addr (mp, rblock,1 ); + fpp =xfs_bmdr_ptr_addr (dblock,1 , dmxr); + tpp =xfs_bmap_broot_ptr_addr (mp, rblock,1 , rblocklen); dmxr =be16_to_cpu (dblock->bb_numrecs);memcpy (tkp, fkp,sizeof (*fkp) * dmxr);memcpy (tpp, fpp,sizeof (*fpp) * dmxr);
@@ -167,10 +167,10 @@ xfs_bmbt_to_bmdr( dblock->bb_level = rblock->bb_level; dblock->bb_numrecs = rblock->bb_numrecs; dmxr =xfs_bmdr_maxrecs (dblocklen,0 ); - fkp =XFS_BMBT_KEY_ADDR (mp, rblock,1 ); - tkp =XFS_BMDR_KEY_ADDR (dblock,1 ); - fpp =XFS_BMAP_BROOT_PTR_ADDR (mp, rblock,1 , rblocklen); - tpp =XFS_BMDR_PTR_ADDR (dblock,1 , dmxr); + fkp =xfs_bmbt_key_addr (mp, rblock,1 ); + tkp =xfs_bmdr_key_addr (dblock,1 ); + fpp =xfs_bmap_broot_ptr_addr (mp, rblock,1 , rblocklen); + tpp =xfs_bmdr_ptr_addr (dblock,1 , dmxr); dmxr =be16_to_cpu (dblock->bb_numrecs);memcpy (tkp, fkp,sizeof (*fkp) * dmxr);memcpy (tpp, fpp,sizeof (*fpp) * dmxr);
@@ -650,7 +650,7 @@ xfs_bmbt_maxrecs( int blocklen, int leaf) { - blocklen -=XFS_BMBT_BLOCK_LEN (mp); + blocklen -=xfs_bmbt_block_len (mp);return xfs_bmbt_block_maxrecs (blocklen, leaf); }
libxfs/xfs_bmap_btree.h +140 -64
214 lines not on this page · raw message
libxfs/xfs_inode_fork.c +15 -15
@@ -183,7 +183,7 @@ xfs_iformat_btree( ifp =xfs_ifork_ptr (ip, whichfork); dfp = (xfs_bmdr_block_t *)XFS_DFORK_PTR (dip, whichfork); - size =XFS_BMAP_BROOT_SPACE (mp, dfp); + size =xfs_bmap_broot_space (mp, dfp); nrecs =be16_to_cpu (dfp->bb_numrecs); level =be16_to_cpu (dfp->bb_level);
@@ -196,7 +196,7 @@ xfs_iformat_btree(*/ if (unlikely (ifp->if_nextents <=XFS_IFORK_MAXEXT (ip, whichfork) || nrecs ==0 || -XFS_BMDR_SPACE_CALC (nrecs) > +xfs_bmdr_space_calc (nrecs) >XFS_DFORK_SIZE (dip, mp, whichfork) || ifp->if_nextents > ip->i_nblocks) || level ==0 || level >XFS_BM_MAXLEVELS (mp, whichfork)) {
@@ -407,7 +407,7 @@ xfs_iroot_realloc(* allocate it now and get out. */ if (ifp->if_broot_bytes ==0 ) { - new_size =XFS_BMAP_BROOT_SPACE_CALC (mp, rec_diff); + new_size =xfs_bmap_broot_space_calc (mp, rec_diff); ifp->if_broot =kmalloc (new_size,GFP_KERNEL | __GFP_NOFAIL); ifp->if_broot_bytes = (int )new_size;
@@ -422,15 +422,15 @@ xfs_iroot_realloc(*/ cur_max =xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes,0 ); new_max = cur_max + rec_diff; - new_size =XFS_BMAP_BROOT_SPACE_CALC (mp, new_max); + new_size =xfs_bmap_broot_space_calc (mp, new_max); ifp->if_broot =krealloc (ifp->if_broot, new_size,GFP_KERNEL | __GFP_NOFAIL); - op = (char *)XFS_BMAP_BROOT_PTR_ADDR (mp, ifp->if_broot,1 , + op = (char *)xfs_bmap_broot_ptr_addr (mp, ifp->if_broot,1 , ifp->if_broot_bytes); - np = (char *)XFS_BMAP_BROOT_PTR_ADDR (mp, ifp->if_broot,1 , + np = (char *)xfs_bmap_broot_ptr_addr (mp, ifp->if_broot,1 , (int )new_size); ifp->if_broot_bytes = (int )new_size; -ASSERT (XFS_BMAP_BMDR_SPACE (ifp->if_broot) <= +ASSERT (xfs_bmap_bmdr_space (ifp->if_broot) <=xfs_inode_fork_size (ip, whichfork));memmove (np, op, cur_max * (uint )sizeof (xfs_fsblock_t));return ;
@@ -446,7 +446,7 @@ xfs_iroot_realloc( new_max = cur_max + rec_diff;ASSERT (new_max >=0 );if (new_max >0 ) - new_size =XFS_BMAP_BROOT_SPACE_CALC (mp, new_max); + new_size =xfs_bmap_broot_space_calc (mp, new_max);else new_size =0 ;if (new_size >0 ) {
@@ -455,7 +455,7 @@ xfs_iroot_realloc(* First copy over the btree block header. */ memcpy (new_broot, ifp->if_broot, -XFS_BMBT_BLOCK_LEN (ip->i_mount)); +xfs_bmbt_block_len (ip->i_mount)); }else { new_broot =NULL ; }
@@ -467,16 +467,16 @@ xfs_iroot_realloc(/* * First copy the keys. */ - op = (char *)XFS_BMBT_KEY_ADDR (mp, ifp->if_broot,1 ); - np = (char *)XFS_BMBT_KEY_ADDR (mp, new_broot,1 ); + op = (char *)xfs_bmbt_key_addr (mp, ifp->if_broot,1 ); + np = (char *)xfs_bmbt_key_addr (mp, new_broot,1 );memcpy (np, op, new_max * (uint )sizeof (xfs_bmbt_key_t));/* * Then copy the pointers. */ - op = (char *)XFS_BMAP_BROOT_PTR_ADDR (mp, ifp->if_broot,1 , + op = (char *)xfs_bmap_broot_ptr_addr (mp, ifp->if_broot,1 , ifp->if_broot_bytes); - np = (char *)XFS_BMAP_BROOT_PTR_ADDR (mp, new_broot,1 , + np = (char *)xfs_bmap_broot_ptr_addr (mp, new_broot,1 , (int )new_size);memcpy (np, op, new_max * (uint )sizeof (xfs_fsblock_t)); }
@@ -484,7 +484,7 @@ xfs_iroot_realloc( ifp->if_broot = new_broot; ifp->if_broot_bytes = (int )new_size;if (ifp->if_broot) -ASSERT (XFS_BMAP_BMDR_SPACE (ifp->if_broot) <= +ASSERT (xfs_bmap_bmdr_space (ifp->if_broot) <=xfs_inode_fork_size (ip, whichfork));return ; }
@@ -653,7 +653,7 @@ xfs_iflush_fork(if ((iip->ili_fields & brootflag[whichfork]) && (ifp->if_broot_bytes >0 )) {ASSERT (ifp->if_broot !=NULL ); -ASSERT (XFS_BMAP_BMDR_SPACE (ifp->if_broot) <= +ASSERT (xfs_bmap_bmdr_space (ifp->if_broot) <=xfs_inode_fork_size (ip, whichfork)); xfs_bmbt_to_bmdr(mp, ifp->if_broot, ifp->if_broot_bytes, (xfs_bmdr_block_t *)cp,
libxfs/xfs_trans_resv.c +1 -1
@@ -127,7 +127,7 @@ xfs_calc_inode_res( (4 *sizeof (struct xlog_op_header ) +sizeof (struct xfs_inode_log_format ) + mp->m_sb.sb_inodesize + -2 *XFS_BMBT_BLOCK_LEN (mp)); +2 *xfs_bmbt_block_len (mp)); }/*
repair/bmap_repair.c +1 -1
@@ -284,7 +284,7 @@ xrep_bmap_iroot_size( {ASSERT (level >0 ); -return XFS_BMAP_BROOT_SPACE_CALC (cur->bc_mp, nr_this_level); +return xfs_bmap_broot_space_calc (cur->bc_mp, nr_this_level); }/* Update the inode counters. */
repair/dinode.c +5 -5
@@ -809,19 +809,19 @@ _("bad numrecs 0 in inode %" PRIu64 " bmap btree root block\n"),/* * use bmdr/dfork_dsize since the root block is in the data fork */ -if (XFS_BMDR_SPACE_CALC (numrecs) >XFS_DFORK_SIZE (dip, mp, whichfork)) { +if (xfs_bmdr_space_calc (numrecs) >XFS_DFORK_SIZE (dip, mp, whichfork)) {do_warn ( -_ ("indicated size of %s btree root (%d bytes) greater than space in " +_ ("indicated size of %s btree root (%zu bytes) greater than space in " "inode %" PRIu64" %s fork\n" ), - forkname,XFS_BMDR_SPACE_CALC (numrecs), lino, forkname); + forkname,xfs_bmdr_space_calc (numrecs), lino, forkname);return (1 ); }init_bm_cursor (&cursor, level +1 ); - pp =XFS_BMDR_PTR_ADDR (dib,1 , + pp =xfs_bmdr_ptr_addr (dib,1 ,libxfs_bmdr_maxrecs (XFS_DFORK_SIZE (dip, mp, whichfork),0 )); - pkey =XFS_BMDR_KEY_ADDR (dib,1 ); + pkey =xfs_bmdr_key_addr (dib,1 ); last_key =NULLFILEOFF ;for (i =0 ; i < numrecs; i++) {
repair/prefetch.c +4 -4
@@ -328,13 +328,13 @@ pf_scanfunc_bmap(if (numrecs > mp->m_bmap_dmxr[0 ] || !isadir)return 0 ;return pf_read_bmbt_reclist (args, -XFS_BMBT_REC_ADDR (mp, block,1 ), numrecs); +xfs_bmbt_rec_addr (mp, block,1 ), numrecs); }if (numrecs > mp->m_bmap_dmxr[1 ])return 0 ; - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[1 ]); + pp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[1 ]);for (i =0 ; i < numrecs; i++) { dbno =get_unaligned_be64 (&pp[i]);
@@ -372,11 +372,11 @@ pf_read_btinode(/* * use bmdr/dfork_dsize since the root block is in the data fork */ -if (XFS_BMDR_SPACE_CALC (numrecs) >XFS_DFORK_DSIZE (dino, mp)) +if (xfs_bmdr_space_calc (numrecs) >XFS_DFORK_DSIZE (dino, mp))return ; dsize =XFS_DFORK_DSIZE (dino, mp); - pp =XFS_BMDR_PTR_ADDR (dib,1 ,libxfs_bmdr_maxrecs (dsize,0 )); + pp =xfs_bmdr_ptr_addr (dib,1 ,libxfs_bmdr_maxrecs (dsize,0 ));for (i =0 ; i < numrecs; i++) { dbno =get_unaligned_be64 (&pp[i]);
repair/scan.c +3 -3
@@ -416,7 +416,7 @@ _("inode %" PRIu64 " bad # of bmap records (%" PRIu64 ", min - %u, max - %u)\n") mp->m_bmap_dmxr[0]);return (1 ); } - rp =XFS_BMBT_REC_ADDR (mp, block,1 ); + rp =xfs_bmbt_rec_addr (mp, block,1 ); *nex += numrecs;/* * XXX - if we were going to fix up the btree record,
@@ -466,8 +466,8 @@ _("inode %" PRIu64 " bad # of bmap records (%" PRIu64 ", min - %u, max - %u)\n") ino, numrecs, mp->m_bmap_dmnr[1], mp->m_bmap_dmxr[1]);return (1 ); } - pp =XFS_BMBT_PTR_ADDR (mp, block,1 , mp->m_bmap_dmxr[1 ]); - pkey =XFS_BMBT_KEY_ADDR (mp, block,1 ); + pp =xfs_bmbt_ptr_addr (mp, block,1 , mp->m_bmap_dmxr[1 ]); + pkey =xfs_bmbt_key_addr (mp, block,1 ); last_key =NULLFILEOFF ;
PATCH 22/41 xfs: standardize the btree maxrecs function parameters
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Source kernel commit: 411a71256de6f5a0015a28929cfbe6bc36c503dc
Standardize the parameters in xfs_{alloc,bm,ino,rmap,refcount}bt_maxrecs so that we have consistent calling conventions. This doesn't affect the kernel that much, but enables us to clean up userspace a bit.
--- db/btheight.c | 18 ++++-------------- libxfs/xfs_alloc_btree.c | 6 +++--- libxfs/xfs_alloc_btree.h | 3 ++- libxfs/xfs_bmap.c | 2 +- libxfs/xfs_bmap_btree.c | 6 +++--- libxfs/xfs_bmap_btree.h | 5 +++-- libxfs/xfs_ialloc.c | 4 ++-- libxfs/xfs_ialloc_btree.c | 6 +++--- libxfs/xfs_ialloc_btree.h | 3 ++- libxfs/xfs_inode_fork.c | 4 ++-- libxfs/xfs_refcount_btree.c | 5 +++-- libxfs/xfs_refcount_btree.h | 3 ++- libxfs/xfs_rmap_btree.c | 7 ++++--- libxfs/xfs_rmap_btree.h | 3 ++- libxfs/xfs_sb.c | 16 ++++++++-------- repair/phase5.c | 16 ++++++++-------- 16 files changed, 52 insertions(+), 55 deletions(-)
db/btheight.c +4 -14
@@ -12,21 +12,11 @@#include "input.h" #include "libfrog/convert.h" -static int refc_maxrecs (struct xfs_mount *mp,int blocklen,int leaf) -{ -return libxfs_refcountbt_maxrecs (blocklen, leaf !=0 ); -} - -static int rmap_maxrecs (struct xfs_mount *mp,int blocklen,int leaf) -{ -return libxfs_rmapbt_maxrecs (blocklen, leaf); -} -struct btmap {const char *tag;unsigned int (*maxlevels)(void ); -int (*maxrecs)(struct xfs_mount *mp,int blocklen, -int leaf); +unsigned int (*maxrecs)(struct xfs_mount *mp,unsigned int blocklen, +bool leaf); } maps[] = { { .tag ="bnobt" ,
@@ -56,12 +46,12 @@ struct btmap { { .tag = "refcountbt", .maxlevels = libxfs_refcountbt_maxlevels_ondisk, - .maxrecs = refc_maxrecs, + .maxrecs = libxfs_refcountbt_maxrecs, }, { .tag = "rmapbt", .maxlevels = libxfs_rmapbt_maxlevels_ondisk, - .maxrecs = rmap_maxrecs, + .maxrecs = libxfs_rmapbt_maxrecs, }, };
libxfs/xfs_alloc_btree.c +3 -3
@@ -567,11 +567,11 @@ xfs_allocbt_block_maxrecs(/* * Calculate number of records in an alloc btree block. */ -int +unsigned int xfs_allocbt_maxrecs (struct xfs_mount *mp, -int blocklen, -int leaf) +unsigned int blocklen, +bool leaf) { blocklen -=XFS_ALLOC_BLOCK_LEN (mp);return xfs_allocbt_block_maxrecs (blocklen, leaf);
libxfs/xfs_alloc_btree.h +2 -1
@@ -53,7 +53,8 @@ struct xfs_btree_cur *xfs_bnobt_init_cursor(struct xfs_mount *mp,struct xfs_btree_cur *xfs_cntbt_init_cursor (struct xfs_mount *mp,struct xfs_trans *tp,struct xfs_buf *bp,struct xfs_perag *pag); -extern int xfs_allocbt_maxrecs (struct xfs_mount *,int ,int ); +unsigned int xfs_allocbt_maxrecs (struct xfs_mount *mp,unsigned int blocklen, +bool leaf);extern xfs_extlen_t xfs_allocbt_calc_size (struct xfs_mount *mp,unsigned long long len);
libxfs/xfs_bmap.c +1 -1
@@ -578,7 +578,7 @@ xfs_bmap_btree_to_extents(ASSERT (ifp->if_format ==XFS_DINODE_FMT_BTREE );ASSERT (be16_to_cpu (rblock->bb_level) ==1 );ASSERT (be16_to_cpu (rblock->bb_numrecs) ==1 ); -ASSERT (xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes,0 ) ==1 ); +ASSERT (xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes, false) ==1 ); pp =xfs_bmap_broot_ptr_addr (mp, rblock,1 , ifp->if_broot_bytes); cbno =be64_to_cpu (*pp);
libxfs/xfs_bmap_btree.c +3 -3
@@ -644,11 +644,11 @@ xfs_bmbt_commit_staged_btree(/* * Calculate number of records in a bmap btree block. */ -int +unsigned int xfs_bmbt_maxrecs (struct xfs_mount *mp, -int blocklen, -int leaf) +unsigned int blocklen, +bool leaf) { blocklen -=xfs_bmbt_block_len (mp);return xfs_bmbt_block_maxrecs (blocklen, leaf);
libxfs/xfs_bmap_btree.h +3 -2
@@ -35,7 +35,8 @@ extern void xfs_bmbt_to_bmdr(struct xfs_mount *, struct xfs_btree_block *, int,extern int xfs_bmbt_get_maxrecs (struct xfs_btree_cur *,int level);extern int xfs_bmdr_maxrecs (int blocklen,int leaf); -extern int xfs_bmbt_maxrecs (struct xfs_mount *,int blocklen,int leaf); +unsigned int xfs_bmbt_maxrecs (struct xfs_mount *mp,unsigned int blocklen, +bool leaf); extern int xfs_bmbt_change_owner(struct xfs_trans *tp, struct xfs_inode *ip, int whichfork, xfs_ino_t new_owner,
@@ -151,7 +152,7 @@ xfs_bmap_broot_ptr_addr(unsigned int i, unsigned int sz) { - return xfs_bmbt_ptr_addr(mp, bb, i, xfs_bmbt_maxrecs(mp, sz, 0)); + return xfs_bmbt_ptr_addr(mp, bb, i, xfs_bmbt_maxrecs(mp, sz, false)); }/*
libxfs/xfs_ialloc.c +2 -2
@@ -2943,8 +2943,8 @@ xfs_ialloc_setup_geometry(/* Compute inode btree geometry. */ igeo->agino_log = sbp->sb_inopblog + sbp->sb_agblklog; - igeo->inobt_mxr[0 ] =xfs_inobt_maxrecs (mp, sbp->sb_blocksize,1 ); - igeo->inobt_mxr[1 ] =xfs_inobt_maxrecs (mp, sbp->sb_blocksize,0 ); + igeo->inobt_mxr[0 ] =xfs_inobt_maxrecs (mp, sbp->sb_blocksize, true); + igeo->inobt_mxr[1 ] =xfs_inobt_maxrecs (mp, sbp->sb_blocksize, false); igeo->inobt_mnr[0 ] = igeo->inobt_mxr[0 ] /2 ; igeo->inobt_mnr[1 ] = igeo->inobt_mxr[1 ] /2 ;
libxfs/xfs_ialloc_btree.c +3 -3
@@ -571,11 +571,11 @@ xfs_inobt_block_maxrecs(/* * Calculate number of records in an inobt btree block. */ -int +unsigned int xfs_inobt_maxrecs (struct xfs_mount *mp, -int blocklen, -int leaf) +unsigned int blocklen, +bool leaf) { blocklen -=XFS_INOBT_BLOCK_LEN (mp);return xfs_inobt_block_maxrecs (blocklen, leaf);
libxfs/xfs_ialloc_btree.h +2 -1
@@ -50,7 +50,8 @@ struct xfs_btree_cur *xfs_inobt_init_cursor(struct xfs_perag *pag, struct xfs_trans *tp, struct xfs_buf *agbp);struct xfs_btree_cur *xfs_finobt_init_cursor (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agbp); -extern int xfs_inobt_maxrecs (struct xfs_mount *,int ,int ); +unsigned int xfs_inobt_maxrecs (struct xfs_mount *mp,unsigned int blocklen, +bool leaf);/* ir_holemask to inode allocation bitmap conversion */ uint64_t xfs_inobt_irec_to_allocmask (const struct xfs_inobt_rec_incore *irec);
libxfs/xfs_inode_fork.c +2 -2
@@ -420,7 +420,7 @@ xfs_iroot_realloc(* location. The records don't change location because * they are kept butted up against the btree block header. */ - cur_max =xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes,0 ); + cur_max =xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes, false); new_max = cur_max + rec_diff; new_size =xfs_bmap_broot_space_calc (mp, new_max); ifp->if_broot = krealloc(ifp->if_broot, new_size,
@@ -442,7 +442,7 @@ xfs_iroot_realloc(* records, just get rid of the root and clear the status bit. */ ASSERT ((ifp->if_broot !=NULL ) && (ifp->if_broot_bytes >0 )); - cur_max =xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes,0 ); + cur_max =xfs_bmbt_maxrecs (mp, ifp->if_broot_bytes, false); new_max = cur_max + rec_diff;ASSERT (new_max >=0 );if (new_max >0 )
libxfs/xfs_refcount_btree.c +3 -2
@@ -416,9 +416,10 @@ xfs_refcountbt_block_maxrecs(/* * Calculate the number of records in a refcount btree block. */ -int +unsigned int xfs_refcountbt_maxrecs ( -int blocklen, +struct xfs_mount *mp, +unsigned int blocklen,bool leaf) { blocklen -=XFS_REFCOUNT_BLOCK_LEN ;
libxfs/xfs_refcount_btree.h +2 -1
@@ -48,7 +48,8 @@ struct xbtree_afakeroot;extern struct xfs_btree_cur *xfs_refcountbt_init_cursor (struct xfs_mount *mp,struct xfs_trans *tp,struct xfs_buf *agbp,struct xfs_perag *pag); -extern int xfs_refcountbt_maxrecs (int blocklen,bool leaf); +unsigned int xfs_refcountbt_maxrecs (struct xfs_mount *mp,unsigned int blocklen, +bool leaf);extern void xfs_refcountbt_compute_maxlevels (struct xfs_mount *mp); extern xfs_extlen_t xfs_refcountbt_calc_size(struct xfs_mount *mp,
libxfs/xfs_rmap_btree.c +4 -3
@@ -730,10 +730,11 @@ xfs_rmapbt_block_maxrecs(/* * Calculate number of records in an rmap btree block. */ -int +unsigned int xfs_rmapbt_maxrecs ( -int blocklen, -int leaf) +struct xfs_mount *mp, +unsigned int blocklen, +bool leaf) { blocklen -=XFS_RMAP_BLOCK_LEN ;return xfs_rmapbt_block_maxrecs (blocklen, leaf);
libxfs/xfs_rmap_btree.h +2 -1
@@ -47,7 +47,8 @@ struct xfs_btree_cur *xfs_rmapbt_init_cursor(struct xfs_mount *mp, struct xfs_perag *pag);void xfs_rmapbt_commit_staged_btree (struct xfs_btree_cur *cur,struct xfs_trans *tp,struct xfs_buf *agbp); -int xfs_rmapbt_maxrecs (int blocklen,int leaf); +unsigned int xfs_rmapbt_maxrecs (struct xfs_mount *mp,unsigned int blocklen, +bool leaf);extern void xfs_rmapbt_compute_maxlevels (struct xfs_mount *mp); extern xfs_extlen_t xfs_rmapbt_calc_size(struct xfs_mount *mp,
libxfs/xfs_sb.c +8 -8
@@ -997,23 +997,23 @@ xfs_sb_mount_common( mp->m_blockwmask = mp->m_blockwsize -1 ;xfs_mount_sb_set_rextsize (mp, sbp); - mp->m_alloc_mxr[0 ] =xfs_allocbt_maxrecs (mp, sbp->sb_blocksize,1 ); - mp->m_alloc_mxr[1 ] =xfs_allocbt_maxrecs (mp, sbp->sb_blocksize,0 ); + mp->m_alloc_mxr[0 ] =xfs_allocbt_maxrecs (mp, sbp->sb_blocksize, true); + mp->m_alloc_mxr[1 ] =xfs_allocbt_maxrecs (mp, sbp->sb_blocksize, false); mp->m_alloc_mnr[0 ] = mp->m_alloc_mxr[0 ] /2 ; mp->m_alloc_mnr[1 ] = mp->m_alloc_mxr[1 ] /2 ; - mp->m_bmap_dmxr[0 ] =xfs_bmbt_maxrecs (mp, sbp->sb_blocksize,1 ); - mp->m_bmap_dmxr[1 ] =xfs_bmbt_maxrecs (mp, sbp->sb_blocksize,0 ); + mp->m_bmap_dmxr[0 ] =xfs_bmbt_maxrecs (mp, sbp->sb_blocksize, true); + mp->m_bmap_dmxr[1 ] =xfs_bmbt_maxrecs (mp, sbp->sb_blocksize, false); mp->m_bmap_dmnr[0 ] = mp->m_bmap_dmxr[0 ] /2 ; mp->m_bmap_dmnr[1 ] = mp->m_bmap_dmxr[1 ] /2 ; - mp->m_rmap_mxr[0 ] =xfs_rmapbt_maxrecs (sbp->sb_blocksize,1 ); - mp->m_rmap_mxr[1 ] =xfs_rmapbt_maxrecs (sbp->sb_blocksize,0 ); + mp->m_rmap_mxr[0 ] =xfs_rmapbt_maxrecs (mp, sbp->sb_blocksize, true); + mp->m_rmap_mxr[1 ] =xfs_rmapbt_maxrecs (mp, sbp->sb_blocksize, false); mp->m_rmap_mnr[0 ] = mp->m_rmap_mxr[0 ] /2 ; mp->m_rmap_mnr[1 ] = mp->m_rmap_mxr[1 ] /2 ; - mp->m_refc_mxr[0 ] =xfs_refcountbt_maxrecs (sbp->sb_blocksize, true); - mp->m_refc_mxr[1 ] =xfs_refcountbt_maxrecs (sbp->sb_blocksize, false); + mp->m_refc_mxr[0 ] =xfs_refcountbt_maxrecs (mp, sbp->sb_blocksize, true); + mp->m_refc_mxr[1 ] =xfs_refcountbt_maxrecs (mp, sbp->sb_blocksize, false); mp->m_refc_mnr[0 ] = mp->m_refc_mxr[0 ] /2 ; mp->m_refc_mnr[1 ] = mp->m_refc_mxr[1 ] /2 ;
repair/phase5.c +8 -8
@@ -641,21 +641,21 @@ phase5(xfs_mount_t *mp)#ifdef XR_BLD_FREE_TRACE fprintf (stderr,"inobt level 1, maxrec = %d, minrec = %d\n" , -libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize,0 ), -libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize,0 ) /2 ); +libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize, false), +libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize, false) /2 );fprintf (stderr,"inobt level 0 (leaf), maxrec = %d, minrec = %d\n" , -libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize,1 ), -libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize,1 ) /2 ); +libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize, true), +libxfs_inobt_maxrecs (mp, mp->m_sb.sb_blocksize, true) /2 );fprintf (stderr,"xr inobt level 0 (leaf), maxrec = %d\n" ,XR_INOBT_BLOCK_MAXRECS (mp,0 ));fprintf (stderr,"xr inobt level 1 (int), maxrec = %d\n" ,XR_INOBT_BLOCK_MAXRECS (mp,1 ));fprintf (stderr,"bnobt level 1, maxrec = %d, minrec = %d\n" , -libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize,0 ), -libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize,0 ) /2 ); +libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize, false), +libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize, false) /2 );fprintf (stderr,"bnobt level 0 (leaf), maxrec = %d, minrec = %d\n" , -libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize,1 ), -libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize,1 ) /2 ); +libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize, true), +libxfs_allocbt_maxrecs (mp, mp->m_sb.sb_blocksize, true) /2 );#endif /* * make sure the root and realtime inodes show up allocated
PATCH 23/41 xfs: use kvmalloc for xattr buffers
In reply to Darrick J. WongFrom: Dave Chinner <[email protected]>
Source kernel commit: de631e1a8b71017b8a12b57d07db82e4052555af
Pankaj Raghav reported that when filesystem block size is larger than page size, the xattr code can use kmalloc() for high order allocations. This triggers a useless warning in the allocator as it is a __GFP_NOFAIL allocation here:
static inline
struct page *rmqueue(struct zone *preferred_zone,
struct zone *zone, unsigned int order,gfp_t gfp_flags, unsigned int alloc_flags,
int migratetype)
{
struct page *page;
/*
* We most definitely don't want callers attempting to
* allocate greater than order-1 page units with __GFP_NOFAIL. */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
WARN_ON_ONCE((gfp_flags & __GFP_NOFAIL) && (order > 1));
...
Fix this by changing all these call sites to use kvmalloc(), which will strip the NOFAIL from the kmalloc attempt and if that fails will do a __GFP_NOFAIL vmalloc().
This is not an issue that productions systems will see as filesystems with block size > page size cannot be mounted by the kernel; Pankaj is developing this functionality right now.
--- include/kmem.h | 6 ++++++ libxfs/xfs_attr_leaf.c | 15 ++++++--------- 2 files changed, 12 insertions(+), 9 deletions(-)
include/kmem.h +6 -0
@@ -59,12 +59,18 @@ static inline void *kmalloc(size_t size, gfp_t flags) }#define kzalloc (size, gfp) kvmalloc((size), (gfp) | __GFP_ZERO) +#define kvzalloc (size, gfp) kzalloc((size), (gfp))static inline void kfree (const void *ptr) {free ((void *)ptr); } +static inline void kvfree (const void *ptr) +{ +kfree (ptr); +} + __attribute__((format (printf,2 ,3 )))char *kasprintf (gfp_t gfp,const char *fmt, ...);
libxfs/xfs_attr_leaf.c +6 -9
@@ -1135,10 +1135,7 @@ xfs_attr3_leaf_to_shortform(trace_xfs_attr_leaf_to_sf (args); - tmpbuffer =kmalloc (args->geo->blksize,GFP_KERNEL | __GFP_NOFAIL); -if (!tmpbuffer) -return -ENOMEM ; - + tmpbuffer =kvmalloc (args->geo->blksize,GFP_KERNEL | __GFP_NOFAIL);memcpy (tmpbuffer, bp->b_addr, args->geo->blksize); leaf = (xfs_attr_leafblock_t *)tmpbuffer;
@@ -1202,7 +1199,7 @@ xfs_attr3_leaf_to_shortform( error =0 ; out: -kfree (tmpbuffer); +kvfree (tmpbuffer);return error; }
@@ -1610,7 +1607,7 @@ xfs_attr3_leaf_compact(trace_xfs_attr_leaf_compact (args); - tmpbuffer =kmalloc (args->geo->blksize,GFP_KERNEL | __GFP_NOFAIL); + tmpbuffer =kvmalloc (args->geo->blksize,GFP_KERNEL | __GFP_NOFAIL);memcpy (tmpbuffer, bp->b_addr, args->geo->blksize);memset (bp->b_addr,0 , args->geo->blksize); leaf_src = (xfs_attr_leafblock_t *)tmpbuffer;
@@ -1648,7 +1645,7 @@ xfs_attr3_leaf_compact(*/ xfs_trans_log_buf (trans, bp,0 , args->geo->blksize -1 ); -kfree (tmpbuffer); +kvfree (tmpbuffer); }/*
@@ -2327,7 +2324,7 @@ xfs_attr3_leaf_unbalance(struct xfs_attr_leafblock *tmp_leaf;struct xfs_attr3_icleaf_hdr tmphdr; - tmp_leaf =kzalloc (state->args->geo->blksize, + tmp_leaf =kvzalloc (state->args->geo->blksize,GFP_KERNEL | __GFP_NOFAIL);/*
@@ -2368,7 +2365,7 @@ xfs_attr3_leaf_unbalance( }memcpy (save_leaf, tmp_leaf, state->args->geo->blksize); savehdr = tmphdr;/* struct copy */ -kfree (tmp_leaf); +kvfree (tmp_leaf); }xfs_attr3_leaf_hdr_to_disk (state->args->geo, save_leaf, &savehdr);
PATCH 24/41 xfs: remove unnecessary check
In reply to Darrick J. WongFrom: Dan Carpenter <[email protected]>
Source kernel commit: fb8b941c75bd70ddfb0a8a3bb9bb770ed1d648f8
We checked that "pip" is non-NULL at the start of the if else statement so there is no need to check again here. Delete the check.
--- libxfs/xfs_inode_util.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/xfs_inode_util.c +1 -1
@@ -305,7 +305,7 @@ xfs_inode_init( !vfsgid_in_group_p(i_gid_into_vfsgid(args->idmap, inode))) inode->i_mode &= ~S_ISGID; - ip->i_projid = pip ?xfs_get_initial_prid (pip) :0 ; + ip->i_projid =xfs_get_initial_prid (pip); } ip->i_disk_size =0 ;
PATCH 25/41 xfs: use kfree_rcu_mightsleep to free the perag structures
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 4ef7c6d39dc72dae983b836c8b2b5de7128c0da3
Using the kfree_rcu_mightsleep is simpler and removes the need for a rcu_head in the perag structure.
--- include/kmem.h | 5 +++++ libxfs/xfs_ag.c | 12 +----------- libxfs/xfs_ag.h | 3 --- 3 files changed, 6 insertions(+), 14 deletions(-)
include/kmem.h +5 -0
@@ -71,6 +71,11 @@ static inline void kvfree(const void *ptr)kfree (ptr); } +static inline void kfree_rcu_mightsleep (const void *ptr) +{ +kfree (ptr); +} + __attribute__((format (printf,2 ,3 )))char *kasprintf (gfp_t gfp,const char *fmt, ...);
libxfs/xfs_ag.c +1 -11
@@ -233,16 +233,6 @@ xfs_initialize_perag_data(return error; } -STATIC void -__xfs_free_perag ( -struct rcu_head *head) -{ - struct xfs_perag *pag = container_of(head, struct xfs_perag, rcu_head); - -ASSERT (!delayed_work_pending (&pag->pag_blockgc_work)); -kfree (pag); -} -/* * Free up the per-ag resources associated with the mount structure. */
@@ -268,7 +258,7 @@ xfs_free_perag(xfs_perag_rele (pag);XFS_IS_CORRUPT (pag->pag_mount,atomic_read (&pag->pag_active_ref) !=0 ); -call_rcu (&pag->rcu_head, __xfs_free_perag); +kfree_rcu_mightsleep (pag); } }
libxfs/xfs_ag.h +0 -3
@@ -63,9 +63,6 @@ struct xfs_perag {/* Blocks reserved for the reverse mapping btree. */ struct xfs_ag_resv pag_rmapbt_resv; -/* for rcu-safe freeing */ -struct rcu_head rcu_head; -/* Precalculated geometry info */ xfs_agblock_t block_count;xfs_agblock_t min_block;
PATCH 26/41 xfs: move the tagged perag lookup helpers to xfs_icache.c
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: f48f0a8e00b67028d4492e7656b346fa0d806570
The tagged perag helpers are only used in xfs_icache.c in the kernel code and not at all in xfsprogs. Move them to xfs_icache.c in preparation for switching to an xarray, for which I have no plan to implement the tagged lookup functions for userspace.
--- libxfs/xfs_ag.c | 51 --------------------------------------------------- libxfs/xfs_ag.h | 11 ----------- 2 files changed, 62 deletions(-)
libxfs/xfs_ag.c +0 -51
@@ -54,31 +54,6 @@ xfs_perag_get(return pag; } -/* -* search from @first to find the next perag with the given tag set. -*/ -struct xfs_perag * -xfs_perag_get_tag ( -struct xfs_mount *mp, -xfs_agnumber_t first, -unsigned int tag) -{ -struct xfs_perag *pag; -int found; - -rcu_read_lock (); - found =radix_tree_gang_lookup_tag (&mp->m_perag_tree, - (void **)&pag, first,1 , tag); -if (found <=0 ) { -rcu_read_unlock (); -return NULL ; - } -trace_xfs_perag_get_tag (pag, _RET_IP_); -atomic_inc (&pag->pag_ref); -rcu_read_unlock (); -return pag; -} -/* Get a passive reference to the given perag. */ struct xfs_perag * xfs_perag_hold(
@@ -125,32 +100,6 @@ xfs_perag_grab(return pag; } -/* -* search from @first to find the next perag with the given tag set. -*/ -struct xfs_perag * -xfs_perag_grab_tag ( -struct xfs_mount *mp, -xfs_agnumber_t first, -int tag) -{ -struct xfs_perag *pag; -int found; - -rcu_read_lock (); - found =radix_tree_gang_lookup_tag (&mp->m_perag_tree, - (void **)&pag, first,1 , tag); -if (found <=0 ) { -rcu_read_unlock (); -return NULL ; - } -trace_xfs_perag_grab_tag (pag, _RET_IP_); -if (!atomic_inc_not_zero (&pag->pag_active_ref)) - pag =NULL ; -rcu_read_unlock (); -return pag; -} -void xfs_perag_rele (struct xfs_perag *pag)
libxfs/xfs_ag.h +0 -11
@@ -153,15 +153,11 @@ void xfs_free_perag(struct xfs_mount *mp);/* Passive AG references */ struct xfs_perag *xfs_perag_get (struct xfs_mount *mp,xfs_agnumber_t agno); -struct xfs_perag *xfs_perag_get_tag (struct xfs_mount *mp,xfs_agnumber_t agno, -unsigned int tag);struct xfs_perag *xfs_perag_hold (struct xfs_perag *pag);void xfs_perag_put (struct xfs_perag *pag);/* Active AG references */ struct xfs_perag *xfs_perag_grab (struct xfs_mount *,xfs_agnumber_t ); -struct xfs_perag *xfs_perag_grab_tag (struct xfs_mount *,xfs_agnumber_t , -int tag);void xfs_perag_rele (struct xfs_perag *pag);/*
@@ -263,13 +259,6 @@ xfs_perag_next( (agno) =0 ; \ for_each_perag_from((mp), (agno), (pag)) -#define for_each_perag_tag (mp, agno, pag, tag) \ - for ((agno) = 0, (pag) = xfs_perag_grab_tag((mp), 0, (tag)); \ - (pag) != NULL; \ - (agno) = (pag)->pag_agno + 1, \ - xfs_perag_rele(pag), \ - (pag) = xfs_perag_grab_tag((mp), (agno), (tag))) -static inline struct xfs_perag * xfs_perag_next_wrap( struct xfs_perag *pag,
PATCH 27/41 xfs: convert perag lookup to xarray
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 32fa4059fe6776d7db1e9058f360e06b36c9f2ce
Convert the perag lookup from the legacy radix tree to the xarray, which allows for much nicer iteration and bulk lookup semantics.
--- include/xfs_mount.h | 2 +- libxfs/init.c | 2 +- libxfs/xfs_ag.c | 31 ++++++++----------------------- 3 files changed, 10 insertions(+), 25 deletions(-)
include/xfs_mount.h +1 -1
@@ -91,7 +91,7 @@ typedef struct xfs_mount {xfs_extlen_t m_ag_prealloc_blocks;/* reserved ag blocks */ uint m_alloc_set_aside;/* space we can't use */ uint m_ag_max_usable;/* max space per AG */ -struct radix_tree_root m_perag_tree; +struct xarray m_perags;uint64_t m_features;/* active filesystem features */ uint64_t m_low_space[XFS_LOWSP_MAX ];uint64_t m_rtxblkmask;/* rt extent block mask */
libxfs/init.c +1 -1
@@ -662,7 +662,7 @@ libxfs_mount( mp->m_finobt_nores = true;xfs_set_inode32 (mp); mp->m_sb = *sb; -INIT_RADIX_TREE (&mp->m_perag_tree,GFP_KERNEL ); +xa_init (&mp->m_perags); sbp = &mp->m_sb;spin_lock_init (&mp->m_sb_lock);spin_lock_init (&mp->m_agirotor_lock);
libxfs/xfs_ag.c +8 -23
@@ -44,7 +44,7 @@ xfs_perag_get(struct xfs_perag *pag;rcu_read_lock (); - pag =radix_tree_lookup (&mp->m_perag_tree, agno); + pag =xa_load (&mp->m_perags, agno);if (pag) {trace_xfs_perag_get (pag, _RET_IP_);ASSERT (atomic_read (&pag->pag_ref) >=0 );
@@ -90,7 +90,7 @@ xfs_perag_grab(struct xfs_perag *pag;rcu_read_lock (); - pag =radix_tree_lookup (&mp->m_perag_tree, agno); + pag =xa_load (&mp->m_perags, agno);if (pag) {trace_xfs_perag_grab (pag, _RET_IP_);if (!atomic_inc_not_zero (&pag->pag_active_ref))
@@ -193,9 +193,7 @@ xfs_free_perag(xfs_agnumber_t agno;for (agno =0 ; agno < mp->m_sb.sb_agcount; agno++) { -spin_lock (&mp->m_perag_lock); - pag =radix_tree_delete (&mp->m_perag_tree, agno); -spin_unlock (&mp->m_perag_lock); + pag =xa_erase (&mp->m_perags, agno);ASSERT (pag);XFS_IS_CORRUPT (pag->pag_mount,atomic_read (&pag->pag_ref) !=0 );xfs_defer_drain_free (&pag->pag_intents_drain);
@@ -284,9 +282,7 @@ xfs_free_unused_perag_range(xfs_agnumber_t index;for (index = agstart; index < agend; index++) { -spin_lock (&mp->m_perag_lock); - pag =radix_tree_delete (&mp->m_perag_tree, index); -spin_unlock (&mp->m_perag_lock); + pag =xa_erase (&mp->m_perags, index);if (!pag)break ;xfs_buf_cache_destroy (&pag->pag_bcache);
@@ -327,20 +323,11 @@ xfs_initialize_perag( pag->pag_agno = index; pag->pag_mount = mp; - error =radix_tree_preload (GFP_KERNEL | __GFP_RETRY_MAYFAIL); -if (error) - goto out_free_pag; - -spin_lock (&mp->m_perag_lock); -if (radix_tree_insert (&mp->m_perag_tree, index, pag)) { -WARN_ON_ONCE (1 ); -spin_unlock (&mp->m_perag_lock); -radix_tree_preload_end (); - error = -EEXIST ; + error =xa_insert (&mp->m_perags, index, pag,GFP_KERNEL ); +if (error) { +WARN_ON_ONCE (error == -EBUSY ); goto out_free_pag; } -spin_unlock (&mp->m_perag_lock); -radix_tree_preload_end ();#ifdef __KERNEL__/* Place kernel structure only init below this point. */
@@ -388,9 +375,7 @@ xfs_initialize_perag( out_remove_pag:xfs_defer_drain_free (&pag->pag_intents_drain); -spin_lock (&mp->m_perag_lock); -radix_tree_delete (&mp->m_perag_tree, index); -spin_unlock (&mp->m_perag_lock); + pag =xa_erase (&mp->m_perags, index); out_free_pag:kfree (pag); out_unwind_new_pags:
PATCH 28/41 xfs: ensure st_blocks never goes to zero during COW writes
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 90fa22da6d6b41dc17435aff7b800f9ca3c00401
COW writes remove the amount overwritten either directly for delalloc reservations, or in earlier deferred transactions than adding the new amount back in the bmap map transaction. This means st_blocks on an inode where all data is overwritten using the COW path can temporarily show a 0 st_blocks. This can easily be reproduced with the pending zoned device support where all writes use this path and trips the check in generic/615, but could also happen on a reflink file without that.
Fix this by temporarily add the pending blocks to be mapped to i_delayed_blks while the item is queued.
--- libxfs/defer_item.c | 14 ++++++++++++++ libxfs/xfs_bmap.c | 1 + 2 files changed, 15 insertions(+)
libxfs/defer_item.c +14 -0
@@ -520,6 +520,17 @@ xfs_bmap_defer_add(trace_xfs_bmap_defer (bi);xfs_bmap_update_get_group (tp->t_mountp, bi); + +/* +* Ensure the deferred mapping is pre-recorded in i_delayed_blks. +* +* Otherwise stat can report zero blocks for an inode that actually has +* data when the entire mapping is in the process of being overwritten +* using the out of place write path. This is undone in xfs_bmapi_remap +* after it has incremented di_nblocks for a successful operation. +*/ +if (bi->bi_type ==XFS_BMAP_MAP ) + bi->bi_owner->i_delayed_blks += bi->bi_bmap.br_blockcount;xfs_defer_add (tp, &bi->bi_list, &xfs_bmap_update_defer_type); }
@@ -541,6 +552,9 @@ xfs_bmap_update_cancel_item( {struct xfs_bmap_intent *bi =bi_entry (item); +if (bi->bi_type ==XFS_BMAP_MAP ) + bi->bi_owner->i_delayed_blks -= bi->bi_bmap.br_blockcount; +xfs_bmap_update_put_group (bi);kmem_cache_free (xfs_bmap_intent_cache, bi); }
libxfs/xfs_bmap.c +1 -0
@@ -4850,6 +4850,7 @@ xfs_bmapi_remap( } ip->i_nblocks += len; + ip->i_delayed_blks -= len;/* see xfs_bmap_defer_add */ xfs_trans_log_inode (tp, ip,XFS_ILOG_CORE );if (ifp->if_format ==XFS_DINODE_FMT_BTREE )
PATCH 29/41 xfs: enable block size larger than page size support
In reply to Darrick J. WongFrom: Pankaj Raghav <[email protected]>
Source kernel commit: 7df7c204c678e24cd32d33360538670b7b90e330
Page cache now has the ability to have a minimum order when allocating a folio which is a prerequisite to add support for block size > page size.
--- libxfs/init.c | 5 +++++ libxfs/libxfs_priv.h | 1 + libxfs/xfs_ialloc.c | 5 +++++ libxfs/xfs_shared.h | 3 +++ 4 files changed, 14 insertions(+)
libxfs/init.c +5 -0
@@ -22,6 +22,7 @@#include "xfs_rmap_btree.h" #include "xfs_refcount_btree.h" #include "libfrog/platform.h" +#include "libfrog/util.h" #include "libxfs/xfile.h" #include "libxfs/buf_mem.h"
@@ -44,6 +45,8 @@ int use_xfs_buf_lock; /* global flag: use xfs_buf locks for MT */static int nextfakedev =-1 ;/* device number to give to next fake device */ +unsigned int PAGE_SHIFT ; +/* * Checks whether a given device has a mounted, writable * filesystem, returns 1 if it does & fatal (just warns
@@ -257,6 +260,8 @@ libxfs_close_devices(int libxfs_init (struct libxfs_init *a) { +if (!PAGE_SHIFT ) +PAGE_SHIFT =log2_roundup (PAGE_SIZE );xfs_check_ondisk_structs ();xmbuf_libinit ();rcu_init ();
libxfs/libxfs_priv.h +1 -0
@@ -224,6 +224,7 @@ uint32_t get_random_u32(void);#endif #define PAGE_SIZE getpagesize() +extern unsigned int PAGE_SHIFT ;#define inode_peek_iversion (inode) (inode)->i_version#define inode_set_iversion_queried (inode, version) do { \
libxfs/xfs_ialloc.c +5 -0
@@ -3029,6 +3029,11 @@ xfs_ialloc_setup_geometry( igeo->ialloc_align = mp->m_dalign; else igeo->ialloc_align = 0; + +if (mp->m_sb.sb_blocksize >PAGE_SIZE ) + igeo->min_folio_order = mp->m_sb.sb_blocklog -PAGE_SHIFT ; +else + igeo->min_folio_order =0 ; }/* Compute the location of the root directory inode that is laid out by mkfs. */
libxfs/xfs_shared.h +3 -0
@@ -224,6 +224,9 @@ struct xfs_ino_geometry {/* precomputed value for di_flags2 */ uint64_t new_diflags2; +/* minimum folio order of a page cache allocation */ + unsigned int min_folio_order; + };#endif /* __XFS_SHARED_H__ */
PATCH 30/41 xfs: merge xfs_attr_leaf_try_add into xfs_attr_leaf_addname
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: b1c649da15c2e4c86344c8e5af69c8afa215efec
xfs_attr_leaf_try_add is only called by xfs_attr_leaf_addname, and merging the two will simplify a following error handling fix.
To facilitate this move the remote block state save/restore helpers up in the file so that they don't need forward declarations now.
--- libxfs/xfs_attr.c | 176 ++++++++++++++++++++++------------------------------- 1 file changed, 74 insertions(+), 102 deletions(-)
libxfs/xfs_attr.c +74 -102
@@ -50,7 +50,6 @@ STATIC int xfs_attr_shortform_addname(xfs_da_args_t *args); STATIC int xfs_attr_leaf_get(xfs_da_args_t *args); STATIC int xfs_attr_leaf_removename(xfs_da_args_t *args); STATIC int xfs_attr_leaf_hasname(struct xfs_da_args *args, struct xfs_buf **bp); -STATIC int xfs_attr_leaf_try_add(struct xfs_da_args *args);/* * Internal routines when attribute list is more than one block.
@@ -436,6 +435,33 @@ xfs_attr_hashval(return xfs_attr_hashname (name, namelen); } +/* Save the current remote block info and clear the current pointers. */ +static void +xfs_attr_save_rmt_blk ( +struct xfs_da_args *args) +{ + args->blkno2 = args->blkno; + args->index2 = args->index; + args->rmtblkno2 = args->rmtblkno; + args->rmtblkcnt2 = args->rmtblkcnt; + args->rmtvaluelen2 = args->rmtvaluelen; + args->rmtblkno =0 ; + args->rmtblkcnt =0 ; + args->rmtvaluelen =0 ; +} + +/* Set stored info about a remote block */ +static void +xfs_attr_restore_rmt_blk ( +struct xfs_da_args *args) +{ + args->blkno = args->blkno2; + args->index = args->index2; + args->rmtblkno = args->rmtblkno2; + args->rmtblkcnt = args->rmtblkcnt2; + args->rmtvaluelen = args->rmtvaluelen2; +} +/* * PPTR_REPLACE operations require the caller to set the old and new names and * values explicitly. Update the canonical fields to the new name and value
@@ -481,49 +507,77 @@ xfs_attr_complete_op(return replace_state; } +/* +* Try to add an attribute to an inode in leaf form. +*/ static int xfs_attr_leaf_addname (struct xfs_attr_intent *attr) {struct xfs_da_args *args = attr->xattri_da_args; +struct xfs_buf *bp;int error;ASSERT (xfs_attr_is_leaf (args->dp)); + error =xfs_attr3_leaf_read (args->trans, args->dp, args->owner,0 , &bp); +if (error) +return error; +/* -* Use the leaf buffer we may already hold locked as a result of -* a sf-to-leaf conversion. +* Look up the xattr name to set the insertion point for the new xattr. */ - error =xfs_attr_leaf_try_add (args); - -if (error == -ENOSPC ) { - error =xfs_attr3_leaf_to_node (args); -if (error) -return error; + error =xfs_attr3_leaf_lookup_int (bp, args); +switch (error) { +case -ENOATTR : +if (args->op_flags &XFS_DA_OP_REPLACE ) + goto out_brelse; +break ; +case -EEXIST : +if (!(args->op_flags &XFS_DA_OP_REPLACE )) + goto out_brelse; +trace_xfs_attr_leaf_replace (args);/* -* We're not in leaf format anymore, so roll the transaction and -* retry the add to the newly allocated node block. +* Save the existing remote attr state so that the current +* values reflect the state of the new attribute we are about to +* add, not the attribute we just found and will remove later. */ - attr->xattri_dela_state =XFS_DAS_NODE_ADD ; - goto out; +xfs_attr_save_rmt_blk (args); +break ; +case 0 : +break ; +default : + goto out_brelse; } -if (error) -return error;/* * We need to commit and roll if we need to allocate remote xattr blocks * or perform more xattr manipulations. Otherwise there is nothing more * to do and we can return success. */ -if (args->rmtblkno) + error =xfs_attr3_leaf_add (bp, args); +if (error) { +if (error != -ENOSPC ) +return error; + error =xfs_attr3_leaf_to_node (args); +if (error) +return error; + + attr->xattri_dela_state =XFS_DAS_NODE_ADD ; + }else if (args->rmtblkno) { attr->xattri_dela_state =XFS_DAS_LEAF_SET_RMT ; -else - attr->xattri_dela_state =xfs_attr_complete_op (attr, -XFS_DAS_LEAF_REPLACE ); -out: + }else { + attr->xattri_dela_state = +xfs_attr_complete_op (attr,XFS_DAS_LEAF_REPLACE ); + } +trace_xfs_attr_leaf_addname_return (attr->xattri_dela_state, args->dp);return error; + +out_brelse: +xfs_trans_brelse (args->trans, bp); +return error; }/*
@@ -1169,88 +1223,6 @@ xfs_attr_shortform_addname(* External routines when attribute list is one block *========================================================================*/ -/* Save the current remote block info and clear the current pointers. */ -static void -xfs_attr_save_rmt_blk ( -struct xfs_da_args *args) -{ - args->blkno2 = args->blkno; - args->index2 = args->index; - args->rmtblkno2 = args->rmtblkno; - args->rmtblkcnt2 = args->rmtblkcnt; - args->rmtvaluelen2 = args->rmtvaluelen; - args->rmtblkno =0 ; - args->rmtblkcnt =0 ; - args->rmtvaluelen =0 ; -} - -/* Set stored info about a remote block */ -static void -xfs_attr_restore_rmt_blk ( -struct xfs_da_args *args) -{ - args->blkno = args->blkno2; - args->index = args->index2; - args->rmtblkno = args->rmtblkno2; - args->rmtblkcnt = args->rmtblkcnt2; - args->rmtvaluelen = args->rmtvaluelen2; -} - -/* -* Tries to add an attribute to an inode in leaf form -* -* This function is meant to execute as part of a delayed operation and leaves -* the transaction handling to the caller. On success the attribute is added -* and the inode and transaction are left dirty. If there is not enough space, -* the attr data is converted to node format and -ENOSPC is returned. Caller is -* responsible for handling the dirty inode and transaction or adding the attr -* in node format. -*/ -STATIC int -xfs_attr_leaf_try_add ( -struct xfs_da_args *args) -{ -struct xfs_buf *bp; -int error; - - error =xfs_attr3_leaf_read (args->trans, args->dp, args->owner,0 , &bp); -if (error) -return error; - -/* -* Look up the xattr name to set the insertion point for the new xattr. -*/ - error =xfs_attr3_leaf_lookup_int (bp, args); -switch (error) { -case -ENOATTR : -if (args->op_flags &XFS_DA_OP_REPLACE ) - goto out_brelse; -break ; -case -EEXIST : -if (!(args->op_flags &XFS_DA_OP_REPLACE )) - goto out_brelse; - -trace_xfs_attr_leaf_replace (args); -/* -* Save the existing remote attr state so that the current -* values reflect the state of the new attribute we are about to -* add, not the attribute we just found and will remove later. -*/ -xfs_attr_save_rmt_blk (args); -break ; -case 0 : -break ; -default : - goto out_brelse; - } - -return xfs_attr3_leaf_add (bp, args); - -out_brelse: -xfs_trans_brelse (args->trans, bp); -return error; -} -/* * Return EEXIST if attr is found, or ENOATTR if not */
PATCH 31/41 xfs: return bool from xfs_attr3_leaf_add
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 346c1d46d4c631c0c88592d371f585214d714da4
xfs_attr3_leaf_add only has two potential return values, indicating if the entry could be added or not. Replace the errno return with a bool so that ENOSPC from it can't easily be confused with a real ENOSPC.
Remove the return value from the xfs_attr3_leaf_add_work helper entirely, as it always return 0.
--- libxfs/xfs_attr.c | 13 +++++-------- libxfs/xfs_attr_leaf.c | 37 +++++++++++++++++++------------------ libxfs/xfs_attr_leaf.h | 2 +- 3 files changed, 25 insertions(+), 27 deletions(-)
libxfs/xfs_attr.c +5 -8
@@ -556,10 +556,7 @@ xfs_attr_leaf_addname(* or perform more xattr manipulations. Otherwise there is nothing more * to do and we can return success. */ - error =xfs_attr3_leaf_add (bp, args); -if (error) { -if (error != -ENOSPC ) -return error; +if (!xfs_attr3_leaf_add (bp, args)) { error =xfs_attr3_leaf_to_node (args);if (error)return error;
@@ -573,7 +570,7 @@ xfs_attr_leaf_addname( }trace_xfs_attr_leaf_addname_return (attr->xattri_dela_state, args->dp); -return error; +return 0 ; out_brelse:xfs_trans_brelse (args->trans, bp);
@@ -1398,21 +1395,21 @@ xfs_attr_node_try_addname( {struct xfs_da_state *state = attr->xattri_da_state;struct xfs_da_state_blk *blk; -int error; +int error =0 ;trace_xfs_attr_node_addname (state->args); blk = &state->path.blk[state->path.active-1 ];ASSERT (blk->magic ==XFS_ATTR_LEAF_MAGIC ); - error =xfs_attr3_leaf_add (blk->bp, state->args); -if (error == -ENOSPC ) { +if (!xfs_attr3_leaf_add (blk->bp, state->args)) {if (state->path.active ==1 ) {/* * Its really a single leaf node, but it had * out-of-line values so it looked like it *might* * have been a b-tree. Let the caller deal with this. */ + error = -ENOSPC ; goto out; }
libxfs/xfs_attr_leaf.c +19 -18
@@ -44,7 +44,7 @@*/ STATIC int xfs_attr3_leaf_create(struct xfs_da_args *args,xfs_dablk_t which_block,struct xfs_buf **bpp); -STATIC int xfs_attr3_leaf_add_work(struct xfs_buf *leaf_buffer, +STATIC void xfs_attr3_leaf_add_work(struct xfs_buf *leaf_buffer,struct xfs_attr3_icleaf_hdr *ichdr,struct xfs_da_args *args,int freemap_index); STATIC void xfs_attr3_leaf_compact(struct xfs_da_args *args,
@@ -992,10 +992,8 @@ xfs_attr_shortform_to_leaf(xfs_attr_sethash (&nargs); error =xfs_attr3_leaf_lookup_int (bp, &nargs);/* set a->index */ ASSERT (error == -ENOATTR ); - error =xfs_attr3_leaf_add (bp, &nargs); -ASSERT (error != -ENOSPC ); -if (error) - goto out; +if (!xfs_attr3_leaf_add (bp, &nargs)) +ASSERT (0 ); sfe =xfs_attr_sf_nextentry (sfe); } error =0 ;
@@ -1337,8 +1335,9 @@ xfs_attr3_leaf_split(struct xfs_da_state_blk *oldblk, struct xfs_da_state_blk *newblk) { - xfs_dablk_t blkno; -int error; + bool added; +xfs_dablk_t blkno; +int error;trace_xfs_attr_leaf_split (state->args);
@@ -1373,10 +1372,10 @@ xfs_attr3_leaf_split(*/ if (state->inleaf) {trace_xfs_attr_leaf_add_old (state->args); - error =xfs_attr3_leaf_add (oldblk->bp, state->args); + added =xfs_attr3_leaf_add (oldblk->bp, state->args); }else {trace_xfs_attr_leaf_add_new (state->args); - error =xfs_attr3_leaf_add (newblk->bp, state->args); + added =xfs_attr3_leaf_add (newblk->bp, state->args); }/*
@@ -1384,13 +1383,15 @@ xfs_attr3_leaf_split(*/ oldblk->hashval =xfs_attr_leaf_lasthash (oldblk->bp,NULL ); newblk->hashval =xfs_attr_leaf_lasthash (newblk->bp,NULL ); -return error; +if (!added) +return -ENOSPC ; +return 0 ; }/* * Add a name to the leaf attribute list structure. */ -int +bool xfs_attr3_leaf_add (struct xfs_buf *bp,struct xfs_da_args *args)
@@ -1399,6 +1400,7 @@ xfs_attr3_leaf_add(struct xfs_attr3_icleaf_hdr ichdr;int tablesize;int entsize; +bool added = true;int sum;int tmp;int i;
@@ -1427,7 +1429,7 @@ xfs_attr3_leaf_add(if (ichdr.freemap[i].base < ichdr.firstused) tmp +=sizeof (xfs_attr_leaf_entry_t);if (ichdr.freemap[i].size >= tmp) { - tmp =xfs_attr3_leaf_add_work (bp, &ichdr, args, i); +xfs_attr3_leaf_add_work (bp, &ichdr, args, i); goto out_log_hdr; } sum += ichdr.freemap[i].size;
@@ -1439,7 +1441,7 @@ xfs_attr3_leaf_add(* no good and we should just give up. */ if (!ichdr.holes && sum < entsize) -return -ENOSPC ; +return false;/* * Compact the entries to coalesce free space.
@@ -1452,24 +1454,24 @@ xfs_attr3_leaf_add(* free region, in freemap[0]. If it is not big enough, give up. */ if (ichdr.freemap[0 ].size < (entsize +sizeof (xfs_attr_leaf_entry_t))) { - tmp = -ENOSPC ; + added = false; goto out_log_hdr; } - tmp =xfs_attr3_leaf_add_work (bp, &ichdr, args,0 ); +xfs_attr3_leaf_add_work (bp, &ichdr, args,0 ); out_log_hdr:xfs_attr3_leaf_hdr_to_disk (args->geo, leaf, &ichdr);xfs_trans_log_buf (args->trans, bp,XFS_DA_LOGRANGE (leaf, &leaf->hdr,xfs_attr3_leaf_hdr_size (leaf))); -return tmp; +return added; }/* * Add a name to a leaf attribute list structure. */ -STATIC int +STATIC void xfs_attr3_leaf_add_work( struct xfs_buf *bp, struct xfs_attr3_icleaf_hdr *ichdr,
@@ -1587,7 +1589,6 @@ xfs_attr3_leaf_add_work( } } ichdr->usedbytes +=xfs_attr_leaf_entsize (leaf, args->index); -return 0 ; }/*
libxfs/xfs_attr_leaf.h +1 -1
@@ -76,7 +76,7 @@ int xfs_attr3_leaf_split(struct xfs_da_state *state,int xfs_attr3_leaf_lookup_int (struct xfs_buf *leaf,struct xfs_da_args *args);int xfs_attr3_leaf_getvalue (struct xfs_buf *bp,struct xfs_da_args *args); -int xfs_attr3_leaf_add (struct xfs_buf *leaf_buffer, +bool xfs_attr3_leaf_add (struct xfs_buf *leaf_buffer,struct xfs_da_args *args);int xfs_attr3_leaf_remove (struct xfs_buf *leaf_buffer,struct xfs_da_args *args);
PATCH 32/41 xfs: distinguish extra split from real ENOSPC from xfs_attr3_leaf_split
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: a5f73342abe1f796140f6585e43e2aa7bc1b7975
xfs_attr3_leaf_split propagates the need for an extra btree split as -ENOSPC to it's only caller, but the same return value can also be returned from xfs_da_grow_inode when it fails to find free space.
Distinguish the two cases by returning 1 for the extra split case instead of overloading -ENOSPC.
This can be triggered relatively easily with the pending realtime group support and a file system with a lot of small zones that use metadata space on the main device. In this case every about 5-10th run of xfs/538 runs into the following assert:
ASSERT(oldblk->magic == XFS_ATTR_LEAF_MAGIC);
in xfs_attr3_leaf_split caused by an allocation failure. Note that the allocation failure is caused by another bug that will be fixed subsequently, but this commit at least sorts out the error handling.
--- libxfs/xfs_attr_leaf.c | 5 ++++- libxfs/xfs_da_btree.c | 5 +++-- 2 files changed, 7 insertions(+), 3 deletions(-)
libxfs/xfs_attr_leaf.c +4 -1
@@ -1328,6 +1328,9 @@ xfs_attr3_leaf_create(/* * Split the leaf node, rebalance, then add the new entry. +* +* Returns 0 if the entry was added, 1 if a further split is needed or a +* negative error number otherwise. */ int xfs_attr3_leaf_split(
@@ -1384,7 +1387,7 @@ xfs_attr3_leaf_split( oldblk->hashval =xfs_attr_leaf_lasthash (oldblk->bp,NULL ); newblk->hashval =xfs_attr_leaf_lasthash (newblk->bp,NULL );if (!added) -return -ENOSPC ; +return 1 ;return 0 ; }
libxfs/xfs_da_btree.c +3 -2
@@ -589,9 +589,8 @@ xfs_da3_split(switch (oldblk->magic) {case XFS_ATTR_LEAF_MAGIC : error =xfs_attr3_leaf_split (state, oldblk, newblk); -if ((error !=0 ) && (error != -ENOSPC )) { +if (error <0 )return error;/* GROT: attr is inconsistent */ - }if (!error) { addblk = newblk;break ;
@@ -613,6 +612,8 @@ xfs_da3_split( error =xfs_attr3_leaf_split (state, newblk, &state->extrablk); } +if (error ==1 ) +return -ENOSPC ;if (error)return error;/* GROT: attr inconsistent */ addblk = newblk;
PATCH 33/41 xfs: distinguish extra split from real ENOSPC from xfs_attr_node_try_addname
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: b3f4e84e2f438a119b7ca8684a25452b3e57c0f0
Just like xfs_attr3_leaf_split, xfs_attr_node_try_addname can return -ENOSPC both for an actual failure to allocate a disk block, but also to signal the caller to convert the format of the attr fork. Use magic 1 to ask for the conversion here as well.
Note that unlike the similar issue in xfs_attr3_leaf_split, this one was only found by code review.
--- libxfs/xfs_attr.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
libxfs/xfs_attr.c +8 -5
@@ -596,7 +596,7 @@ xfs_attr_node_addname(return error; error =xfs_attr_node_try_addname (attr); -if (error == -ENOSPC ) { +if (error ==1 ) { error =xfs_attr3_leaf_to_node (args);if (error)return error;
@@ -1385,9 +1385,12 @@ xfs_attr_node_addname_find_attr(/* * Add a name to a Btree-format attribute list. * -* This will involve walking down the Btree, and may involve splitting -* leaf nodes and even splitting intermediate nodes up to and including -* the root node (a special case of an intermediate node). +* This will involve walking down the Btree, and may involve splitting leaf +* nodes and even splitting intermediate nodes up to and including the root +* node (a special case of an intermediate node). +* +* If the tree was still in single leaf format and needs to converted to +* real node format return 1 and let the caller handle that. */ static int xfs_attr_node_try_addname(
@@ -1409,7 +1412,7 @@ xfs_attr_node_try_addname(* out-of-line values so it looked like it *might* * have been a b-tree. Let the caller deal with this. */ - error = -ENOSPC ; + error =1 ; goto out; }
PATCH 34/41 xfs: fold xfs_bmap_alloc_userdata into xfs_bmapi_allocate
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 865469cd41bce2b04bef9539cbf70676878bc8df
Userdata and metadata allocations end up in the same allocation helpers. Remove the separate xfs_bmap_alloc_userdata function to make this more clear.
--- libxfs/xfs_bmap.c | 73 ++++++++++++++++++++--------------------------------- 1 file changed, 28 insertions(+), 45 deletions(-)
libxfs/xfs_bmap.c +28 -45
@@ -4170,43 +4170,6 @@ xfs_bmapi_reserve_delalloc(return error; } -static int -xfs_bmap_alloc_userdata ( -struct xfs_bmalloca *bma) -{ -struct xfs_mount *mp = bma->ip->i_mount; -int whichfork =xfs_bmapi_whichfork (bma->flags); -int error; - -/* -* Set the data type being allocated. For the data fork, the first data -* in the file is treated differently to all other allocations. For the -* attribute fork, we only need to ensure the allocated range is not on -* the busy list. -*/ - bma->datatype =XFS_ALLOC_NOBUSY ; -if (whichfork ==XFS_DATA_FORK || whichfork ==XFS_COW_FORK ) { - bma->datatype |=XFS_ALLOC_USERDATA ; -if (bma->offset ==0 ) - bma->datatype |=XFS_ALLOC_INITIAL_USER_DATA ; - -if (mp->m_dalign && bma->length >= mp->m_dalign) { - error =xfs_bmap_isaeof (bma, whichfork); -if (error) -return error; - } - -if (XFS_IS_REALTIME_INODE (bma->ip)) -return xfs_bmap_rtalloc (bma); - } - -if (unlikely (XFS_TEST_ERROR (false, mp, -XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT ))) -return xfs_bmap_exact_minlen_extent_alloc (bma); - -return xfs_bmap_btalloc (bma); -} -static int xfs_bmapi_allocate (struct xfs_bmalloca *bma)
@@ -4224,15 +4187,35 @@ xfs_bmapi_allocate(else bma->minlen =1 ; -if (bma->flags &XFS_BMAPI_METADATA ) { -if (unlikely (XFS_TEST_ERROR (false, mp, -XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT ))) - error =xfs_bmap_exact_minlen_extent_alloc (bma); -else - error =xfs_bmap_btalloc (bma); - }else { - error =xfs_bmap_alloc_userdata (bma); +if (!(bma->flags &XFS_BMAPI_METADATA )) { +/* +* For the data and COW fork, the first data in the file is +* treated differently to all other allocations. For the +* attribute fork, we only need to ensure the allocated range +* is not on the busy list. +*/ + bma->datatype =XFS_ALLOC_NOBUSY ; +if (whichfork ==XFS_DATA_FORK || whichfork ==XFS_COW_FORK ) { + bma->datatype |=XFS_ALLOC_USERDATA ; +if (bma->offset ==0 ) + bma->datatype |=XFS_ALLOC_INITIAL_USER_DATA ; + +if (mp->m_dalign && bma->length >= mp->m_dalign) { + error =xfs_bmap_isaeof (bma, whichfork); +if (error) +return error; + } + } } + +if ((bma->datatype &XFS_ALLOC_USERDATA ) && +XFS_IS_REALTIME_INODE (bma->ip)) + error =xfs_bmap_rtalloc (bma); +else if (unlikely (XFS_TEST_ERROR (false, mp, +XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT ))) + error =xfs_bmap_exact_minlen_extent_alloc (bma); +else + error =xfs_bmap_btalloc (bma);if (error)return error;if (bma->blkno ==NULLFSBLOCK )
PATCH 35/41 xfs: don't ifdef around the exact minlen allocations
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: b611fddc0435738e64453bbf1dadd4b12a801858
Exact minlen allocations only exist as an error injection tool for debug builds. Currently this is implemented using ifdefs, which means the code isn't even compiled for non-XFS_DEBUG builds. Enhance the compile test coverage by always building the code and use the compilers' dead code elimination to remove it from the generated binary instead.
The only downside is that the alloc_minlen_only field is unconditionally added to struct xfs_alloc_args now, but by moving it around and packing it tightly this doesn't actually increase the size of the structure.
--- libxfs/xfs_alloc.c | 7 ++----- libxfs/xfs_alloc.h | 4 +--- libxfs/xfs_bmap.c | 6 ------ 3 files changed, 3 insertions(+), 14 deletions(-)
libxfs/xfs_alloc.c +2 -5
@@ -2762,7 +2762,6 @@ xfs_alloc_commit_autoreap(xfs_defer_item_unpause (tp, aarp->dfp); } -#ifdef DEBUG /* * Check if an AGF has a free extent record whose length is equal to * args->minlen.
@@ -2802,7 +2801,6 @@ xfs_exact_minlen_extent_available(return error; } -#endif /* * Decide whether to use this allocation group for this allocation.
@@ -2876,15 +2874,14 @@ xfs_alloc_fix_freelist(if (!xfs_alloc_space_available (args, need, alloc_flags)) goto out_agbp_relse; -#ifdef DEBUG -if (args->alloc_minlen_only) { +if (IS_ENABLED (CONFIG_XFS_DEBUG ) && args->alloc_minlen_only) {int stat; error =xfs_exact_minlen_extent_available (args, agbp, &stat);if (error || !stat) goto out_agbp_relse; } -#endif +/* * Make the freelist shorter if it's too long. *
libxfs/xfs_alloc.h +1 -3
@@ -53,11 +53,9 @@ typedef struct xfs_alloc_arg {int datatype;/* mask defining data type treatment */ char wasdel;/* set if allocation was prev delayed */ char wasfromfl;/* set if allocation is from freelist */ +bool alloc_minlen_only;/* allocate exact minlen extent */ struct xfs_owner_info oinfo;/* owner of blocks being allocated */ enum xfs_ag_resv_type resv;/* block reservation to use */ -#ifdef DEBUG -bool alloc_minlen_only;/* allocate exact minlen extent */ -#endif } xfs_alloc_arg_t;/*
libxfs/xfs_bmap.c +0 -6
@@ -3471,7 +3471,6 @@ xfs_bmap_process_allocated_extent(xfs_bmap_alloc_account (ap); } -#ifdef DEBUG static int xfs_bmap_exact_minlen_extent_alloc (struct xfs_bmalloca *ap)
@@ -3533,11 +3532,6 @@ xfs_bmap_exact_minlen_extent_alloc(return 0 ; } -#else - -#define xfs_bmap_exact_minlen_extent_alloc (bma) (-EFSCORRUPTED) - -#endif /* * If we are not low on available data blocks and we are allocating at
PATCH 36/41 xfs: call xfs_bmap_exact_minlen_extent_alloc from xfs_bmap_btalloc
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 405ee87c6938f67e6ab62a3f8f85b3c60a093886
xfs_bmap_exact_minlen_extent_alloc duplicates the args setup in xfs_bmap_btalloc. Switch to call it from xfs_bmap_btalloc after doing the basic setup.
--- libxfs/xfs_bmap.c | 61 +++++++++++------------------------------------------ 1 file changed, 13 insertions(+), 48 deletions(-)
libxfs/xfs_bmap.c +13 -48
@@ -3473,28 +3473,17 @@ xfs_bmap_process_allocated_extent(static int xfs_bmap_exact_minlen_extent_alloc ( -struct xfs_bmalloca *ap) +struct xfs_bmalloca *ap, +struct xfs_alloc_arg *args) { -struct xfs_mount *mp = ap->ip->i_mount; -struct xfs_alloc_arg args = { .tp = ap->tp, .mp = mp }; -xfs_fileoff_t orig_offset; -xfs_extlen_t orig_length; -int error; - -ASSERT (ap->length); -if (ap->minlen !=1 ) { - ap->blkno =NULLFSBLOCK ; - ap->length =0 ; + args->fsbno =NULLFSBLOCK ;return 0 ; } - orig_offset = ap->offset; - orig_length = ap->length; - - args.alloc_minlen_only =1 ; - -xfs_bmap_compute_alignments (ap, &args); + args->alloc_minlen_only =1 ; + args->minlen = args->maxlen = ap->minlen; + args->total = ap->total;/* * Unlike the longest extent available in an AG, we don't track
@@ -3504,33 +3493,9 @@ xfs_bmap_exact_minlen_extent_alloc(* we need not be concerned about a drop in performance in * "debug only" code paths. */ - ap->blkno =XFS_AGB_TO_FSB (mp,0 ,0 ); + ap->blkno =XFS_AGB_TO_FSB (ap->ip->i_mount,0 ,0 ); - args.oinfo =XFS_RMAP_OINFO_SKIP_UPDATE ; - args.minlen = args.maxlen = ap->minlen; - args.total = ap->total; - - args.alignment =1 ; - args.minalignslop =0 ; - - args.minleft = ap->minleft; - args.wasdel = ap->wasdel; - args.resv =XFS_AG_RESV_NONE ; - args.datatype = ap->datatype; - - error =xfs_alloc_vextent_first_ag (&args, ap->blkno); -if (error) -return error; - -if (args.fsbno !=NULLFSBLOCK ) { -xfs_bmap_process_allocated_extent (ap, &args, orig_offset, - orig_length); - }else { - ap->blkno =NULLFSBLOCK ; - ap->length =0 ; - } - -return 0 ; +return xfs_alloc_vextent_first_ag (args, ap->blkno); }/*
@@ -3789,8 +3754,11 @@ xfs_bmap_btalloc(/* Trim the allocation back to the maximum an AG can fit. */ args.maxlen =min (ap->length, mp->m_ag_max_usable); -if ((ap->datatype &XFS_ALLOC_USERDATA ) && -xfs_inode_is_filestream (ap->ip)) +if (unlikely (XFS_TEST_ERROR (false, mp, +XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT ))) + error =xfs_bmap_exact_minlen_extent_alloc (ap, &args); +else if ((ap->datatype &XFS_ALLOC_USERDATA ) && +xfs_inode_is_filestream (ap->ip)) error =xfs_bmap_btalloc_filestreams (ap, &args, stripe_align);else error =xfs_bmap_btalloc_best_length (ap, &args, stripe_align);
@@ -4205,9 +4173,6 @@ xfs_bmapi_allocate(if ((bma->datatype &XFS_ALLOC_USERDATA ) &&XFS_IS_REALTIME_INODE (bma->ip)) error =xfs_bmap_rtalloc (bma); -else if (unlikely (XFS_TEST_ERROR (false, mp, -XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT ))) - error =xfs_bmap_exact_minlen_extent_alloc (bma);else error =xfs_bmap_btalloc (bma);if (error)
PATCH 37/41 xfs: support lowmode allocations in xfs_bmap_exact_minlen_extent_alloc
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 6aac77059881e4419df499392c995bf02fb9630b
Currently the debug-only xfs_bmap_exact_minlen_extent_alloc allocation variant fails to drop into the lowmode last resort allocator, and thus can sometimes fail allocations for which the caller has a transaction block reservation.
Fix this by using xfs_bmap_btalloc_low_space to do the actual allocation.
--- libxfs/xfs_bmap.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
libxfs/xfs_bmap.c +7 -1
@@ -3495,7 +3495,13 @@ xfs_bmap_exact_minlen_extent_alloc(*/ ap->blkno =XFS_AGB_TO_FSB (ap->ip->i_mount,0 ,0 ); -return xfs_alloc_vextent_first_ag (args, ap->blkno); +/* +* Call xfs_bmap_btalloc_low_space here as it first does a "normal" AG +* iteration and then drops args->total to args->minlen, which might be +* required to find an allocation for the transaction reservation when +* the file system is very full. +*/ +return xfs_bmap_btalloc_low_space (ap, args); }/*
PATCH 38/41 xfs: pass the exact range to initialize to xfs_initialize_perag
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 82742f8c3f1a93787a05a00aca50c2a565231f84
Currently only the new agcount is passed to xfs_initialize_perag, which requires lookups of existing AGs to skip them and complicates error handling. Also pass the previous agcount so that the range that xfs_initialize_perag operates on is exactly defined. That way the extra lookups can be avoided, and error handling can clean up the exact range from the old count to the last added perag structure.
--- libxfs/xfs_ag.c | 28 ++++++---------------------- libxfs/xfs_ag.h | 5 +++-- 2 files changed, 9 insertions(+), 24 deletions(-)
libxfs/xfs_ag.c +6 -22
@@ -294,27 +294,16 @@ xfs_free_unused_perag_range(int xfs_initialize_perag (struct xfs_mount *mp, -xfs_agnumber_t agcount, +xfs_agnumber_t old_agcount, +xfs_agnumber_t new_agcount,xfs_rfsblock_t dblocks,xfs_agnumber_t *maxagi) {struct xfs_perag *pag;xfs_agnumber_t index; -xfs_agnumber_t first_initialised =NULLAGNUMBER ;int error; -/* -* Walk the current per-ag tree so we don't try to initialise AGs -* that already exist (growfs case). Allocate and insert all the -* AGs we don't find ready for initialisation. -*/ -for (index =0 ; index < agcount; index++) { - pag =xfs_perag_get (mp, index); -if (pag) { -xfs_perag_put (pag); -continue ; - } - +for (index = old_agcount; index < new_agcount; index++) { pag =kzalloc (sizeof (*pag),GFP_KERNEL | __GFP_RETRY_MAYFAIL);if (!pag) { error = -ENOMEM ;
@@ -351,21 +340,17 @@ xfs_initialize_perag(/* Active ref owned by mount indicates AG is online. */ atomic_set (&pag->pag_active_ref,1 ); -/* first new pag is fully initialized */ -if (first_initialised ==NULLAGNUMBER ) - first_initialised = index; -/* * Pre-calculated geometry */ - pag->block_count =__xfs_ag_block_count (mp, index, agcount, + pag->block_count =__xfs_ag_block_count (mp, index, new_agcount, dblocks); pag->min_block =XFS_AGFL_BLOCK (mp);__xfs_agino_range (mp, pag->block_count, &pag->agino_min, &pag->agino_max); } - index =xfs_set_inode_alloc (mp, agcount); + index =xfs_set_inode_alloc (mp, new_agcount);if (maxagi) *maxagi = index;
@@ -379,8 +364,7 @@ xfs_initialize_perag( out_free_pag:kfree (pag); out_unwind_new_pags: -/* unwind any prior newly initialized pags */ -xfs_free_unused_perag_range (mp, first_initialised, agcount); +xfs_free_unused_perag_range (mp, old_agcount, index);return error; }
libxfs/xfs_ag.h +3 -2
@@ -146,8 +146,9 @@ __XFS_AG_OPSTATE(agfl_needs_reset, AGFL_NEEDS_RESET)void xfs_free_unused_perag_range (struct xfs_mount *mp,xfs_agnumber_t agstart,xfs_agnumber_t agend); -int xfs_initialize_perag (struct xfs_mount *mp,xfs_agnumber_t agcount, -xfs_rfsblock_t dcount,xfs_agnumber_t *maxagi); +int xfs_initialize_perag (struct xfs_mount *mp,xfs_agnumber_t old_agcount, +xfs_agnumber_t agcount,xfs_rfsblock_t dcount, +xfs_agnumber_t *maxagi);int xfs_initialize_perag_data (struct xfs_mount *mp,xfs_agnumber_t agno);void xfs_free_perag (struct xfs_mount *mp);
PATCH 39/41 xfs: merge the perag freeing helpers
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: aa67ec6a25617e36eba4fb28a88159f500a6cac6
There is no good reason to have two different routines for freeing perag structures for the unmount and error cases. Add two arguments to specify the range of AGs to free to xfs_free_perag, and use that to replace xfs_free_unused_perag_range.
The addition RCU grace period for the error case is harmless, and the extra check for the AG to actually exist is not required now that the callers pass the exact known allocated range.
--- libxfs/init.c | 6 +++--- libxfs/libxfs_api_defs.h | 2 +- libxfs/xfs_ag.c | 40 ++++++++++------------------------------ libxfs/xfs_ag.h | 5 ++--- 4 files changed, 16 insertions(+), 37 deletions(-)
libxfs/init.c +3 -3
@@ -789,8 +789,8 @@ libxfs_mount(libxfs_buf_relse (bp); } - error =libxfs_initialize_perag (mp, sbp->sb_agcount, sbp->sb_dblocks, - &mp->m_maxagi); + error =libxfs_initialize_perag (mp,0 , sbp->sb_agcount, + sbp->sb_dblocks, &mp->m_maxagi);if (error) {fprintf (stderr,_ ("%s: perag init failed\n" ), progname);
@@ -930,7 +930,7 @@ libxfs_umount(* first place. */ if (xfs_is_perag_data_loaded (mp)) -libxfs_free_perag (mp); +libxfs_free_perag_range (mp,0 , mp->m_sb.sb_agcount);xfs_da_unmount (mp);
libxfs/libxfs_api_defs.h +1 -1
@@ -166,7 +166,7 @@#define xfs_finobt_init_cursor libxfs_finobt_init_cursor#define xfs_free_extent libxfs_free_extent#define xfs_free_extent_later libxfs_free_extent_later -#define xfs_free_perag libxfs_free_perag +#define xfs_free_perag_range libxfs_free_perag_range#define xfs_fs_geometry libxfs_fs_geometry#define xfs_get_initial_prid libxfs_get_initial_prid#define xfs_highbit32 libxfs_highbit32
libxfs/xfs_ag.c +10 -30
@@ -183,17 +183,20 @@ xfs_initialize_perag_data( }/* -* Free up the per-ag resources associated with the mount structure. +* Free up the per-ag resources within the specified AG range. */ void -xfs_free_perag ( -struct xfs_mount *mp) +xfs_free_perag_range ( +struct xfs_mount *mp, +xfs_agnumber_t first_agno, +xfs_agnumber_t end_agno) + { -struct xfs_perag *pag;xfs_agnumber_t agno; -for (agno =0 ; agno < mp->m_sb.sb_agcount; agno++) { - pag =xa_erase (&mp->m_perags, agno); +for (agno = first_agno; agno < end_agno; agno++) { +struct xfs_perag *pag =xa_erase (&mp->m_perags, agno); +ASSERT (pag);XFS_IS_CORRUPT (pag->pag_mount,atomic_read (&pag->pag_ref) !=0 );xfs_defer_drain_free (&pag->pag_intents_drain);
@@ -268,29 +271,6 @@ xfs_agino_range(return __xfs_agino_range (mp,xfs_ag_block_count (mp, agno), first, last); } -/* -* Free perag within the specified AG range, it is only used to free unused -* perags under the error handling path. -*/ -void -xfs_free_unused_perag_range ( -struct xfs_mount *mp, -xfs_agnumber_t agstart, -xfs_agnumber_t agend) -{ -struct xfs_perag *pag; -xfs_agnumber_t index; - -for (index = agstart; index < agend; index++) { - pag =xa_erase (&mp->m_perags, index); -if (!pag) -break ; -xfs_buf_cache_destroy (&pag->pag_bcache); -xfs_defer_drain_free (&pag->pag_intents_drain); -kfree (pag); - } -} - int xfs_initialize_perag( struct xfs_mount *mp,
@@ -364,7 +344,7 @@ xfs_initialize_perag( out_free_pag:kfree (pag); out_unwind_new_pags: -xfs_free_unused_perag_range (mp, old_agcount, index); +xfs_free_perag_range (mp, old_agcount, index);return error; }
libxfs/xfs_ag.h +2 -3
@@ -144,13 +144,12 @@ __XFS_AG_OPSTATE(prefers_metadata, PREFERS_METADATA) __XFS_AG_OPSTATE(allows_inodes, ALLOWS_INODES) __XFS_AG_OPSTATE(agfl_needs_reset, AGFL_NEEDS_RESET) -void xfs_free_unused_perag_range (struct xfs_mount *mp,xfs_agnumber_t agstart, -xfs_agnumber_t agend);int xfs_initialize_perag (struct xfs_mount *mp,xfs_agnumber_t old_agcount,xfs_agnumber_t agcount,xfs_rfsblock_t dcount,xfs_agnumber_t *maxagi); +void xfs_free_perag_range (struct xfs_mount *mp,xfs_agnumber_t first_agno, +xfs_agnumber_t end_agno);int xfs_initialize_perag_data (struct xfs_mount *mp,xfs_agnumber_t agno); -void xfs_free_perag (struct xfs_mount *mp);/* Passive AG references */ struct xfs_perag *xfs_perag_get (struct xfs_mount *mp,xfs_agnumber_t agno);
PATCH 40/41 xfs: don't use __GFP_RETRY_MAYFAIL in xfs_initialize_perag
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 069cf5e32b700f94c6ac60f6171662bdfb04f325
__GFP_RETRY_MAYFAIL increases the likelyhood of allocations to fail, which isn't really helpful during log recovery. Remove the flag and stick to the default GFP_KERNEL policies.
--- libxfs/xfs_ag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/xfs_ag.c +1 -1
@@ -284,7 +284,7 @@ xfs_initialize_perag(int error;for (index = old_agcount; index < new_agcount; index++) { - pag =kzalloc (sizeof (*pag),GFP_KERNEL | __GFP_RETRY_MAYFAIL); + pag =kzalloc (sizeof (*pag),GFP_KERNEL );if (!pag) { error = -ENOMEM ; goto out_unwind_new_pags;
PATCH 41/41 xfs: update the pag for the last AG at recovery time
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Source kernel commit: 4a201dcfa1ff0dcfe4348c40f3ad8bd68b97eb6c
Currently log recovery never updates the in-core perag values for the last allocation group when they were grown by growfs. This leads to btree record validation failures for the alloc, ialloc or finotbt trees if a transaction references this new space.
Found by Brian's new growfs recovery stress test.
--- libxfs/xfs_ag.c | 17 +++++++++++++++++ libxfs/xfs_ag.h | 1 + 2 files changed, 18 insertions(+)
libxfs/xfs_ag.c +17 -0
@@ -271,6 +271,23 @@ xfs_agino_range(return __xfs_agino_range (mp,xfs_ag_block_count (mp, agno), first, last); } +int +xfs_update_last_ag_size ( +struct xfs_mount *mp, +xfs_agnumber_t prev_agcount) +{ +struct xfs_perag *pag =xfs_perag_grab (mp, prev_agcount -1 ); + +if (!pag) +return -EFSCORRUPTED ; + pag->block_count =__xfs_ag_block_count (mp, prev_agcount -1 , + mp->m_sb.sb_agcount, mp->m_sb.sb_dblocks); +__xfs_agino_range (mp, pag->block_count, &pag->agino_min, + &pag->agino_max); +xfs_perag_rele (pag); +return 0 ; +} + int xfs_initialize_perag( struct xfs_mount *mp,
libxfs/xfs_ag.h +1 -0
@@ -150,6 +150,7 @@ int xfs_initialize_perag(struct xfs_mount *mp, xfs_agnumber_t old_agcount,void xfs_free_perag_range (struct xfs_mount *mp,xfs_agnumber_t first_agno,xfs_agnumber_t end_agno);int xfs_initialize_perag_data (struct xfs_mount *mp,xfs_agnumber_t agno); +int xfs_update_last_ag_size (struct xfs_mount *mp,xfs_agnumber_t prev_agcount);/* Passive AG references */ struct xfs_perag *xfs_perag_get (struct xfs_mount *mp,xfs_agnumber_t agno);
PATCH 2/7 xfsprogs: atomic file content commits
In reply to Darrick J. WongHi all,
This series creates XFS_IOC_START_COMMIT and XFS_IOC_COMMIT_RANGE ioctls to perform the exchange only if the target file has not been changed since a given sampling point.
This new functionality uses the mechanism underlying EXCHANGE_RANGE to stage and commit file updates such that reader programs will see either the old contents or the new contents in their entirety, with no chance of torn writes. A successful call completion guarantees that the new contents will be seen even if the system fails. The pair of ioctls allows userspace to perform what amounts to a compare and exchange operation on entire file contents.
Note that there are ongoing arguments in the community about how best to implement some sort of file data write counter that nfsd could also use to signal invalidations to clients. Until such a thing is implemented, this patch will rely on ctime/mtime updates.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=atomic-file-commits-6.12
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=atomic-file-commits-6.12
---
Commits in this patchset:
* man: document file range commit ioctls
* libfrog: add support for commit range ioctl family
* libxfs: remove unused xfs_inode fields
* libxfs: validate inumber in xfs_iget
* xfs_fsr: port to new file exchange library function
* xfs_io: add a commitrange option to the exchangerange command
* xfs_io: add atomic file update commands to exercise file commit range
--- fsr/xfs_fsr.c | 74 +++---- include/xfs_inode.h | 4 io/exchrange.c | 390 +++++++++++++++++++++++++++++++++++++ io/io.h | 4 io/open.c | 27 ++- libfrog/file_exchange.c | 194 ++++++++++++++++++ libfrog/file_exchange.h | 10 + libxfs/inode.c | 2 man/man2/ioctl_xfs_commit_range.2 | 296 ++++++++++++++++++++++++++++ man/man2/ioctl_xfs_fsgeometry.2 | 2 man/man2/ioctl_xfs_start_commit.2 | 1 man/man8/xfs_io.8 | 35 +++ 12 files changed, 983 insertions(+), 56 deletions(-) create mode 100644 man/man2/ioctl_xfs_commit_range.2 create mode 100644 man/man2/ioctl_xfs_start_commit.2
PATCH 1/7 man: document file range commit ioctls
From: Darrick J. Wong <[email protected]>
Document the two new ioctls to support committing arbitrary dirty data ranges of two files.
--- man/man2/ioctl_xfs_commit_range.2 | 296 +++++++++++++++++++++++++++++++++++++ man/man2/ioctl_xfs_fsgeometry.2 | 2 man/man2/ioctl_xfs_start_commit.2 | 1 3 files changed, 298 insertions(+), 1 deletion(-) create mode 100644 man/man2/ioctl_xfs_commit_range.2 create mode 100644 man/man2/ioctl_xfs_start_commit.2
man/man2/ioctl_xfs_commit_range.2 new file +296 -0
@@ -0,0 +1,296 @@ +.\" Copyright (c) 2020-2024 Oracle. All rights reserved. +.\" +.\" %%%LICENSE_START(GPLv2+_DOC_FULL) +.\" This is free documentation; you can redistribute it and/or +.\" modify it under the terms of the GNU General Public License as +.\" published by the Free Software Foundation; either version 2 of +.\" the License, or (at your option) any later version. +.\" +.\" The GNU General Public License's references to "object code" +.\" and "executables" are to be interpreted as the output of any +.\" document formatting or typesetting system, including +.\" intermediate and printed output. +.\" +.\" This manual is distributed in the hope that it will be useful, +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +.\" GNU General Public License for more details. +.\" +.\" You should have received a copy of the GNU General Public +.\" License along with this manual; if not, see +.\" <http://www.gnu.org/licenses/>. +.\" %%%LICENSE_END +.TH IOCTL-XFS-COMMIT-RANGE 2 2024-02-18 "XFS" +.SH NAME +ioctl_xfs_start_commit \- prepare to exchange the contents of two files +ioctl_xfs_commit_range \- conditionally exchange the contents of parts of two files +.SH SYNOPSIS +.br +.B #include <sys/ioctl.h> +.br +.B #include <xfs/xfs_fs.h> +.PP +.BI "int ioctl(int " file2_fd ", XFS_IOC_START_COMMIT, struct xfs_commit_range *" arg ); +.PP +.BI "int ioctl(int " file2_fd ", XFS_IOC_COMMIT_RANGE, struct xfs_commit_range *" arg ); +.SH DESCRIPTION +Given a range of bytes in a first file +.B file1_fd +and a second range of bytes in a second file +.BR file2_fd , +this +.BR ioctl (2) +exchanges the contents of the two ranges if +.B file2_fd +passes certain freshness criteria. + +Before exchanging the contents, the program must call the +.B XFS_IOC_START_COMMIT +ioctl to sample freshness data for +.BR file2_fd . +If the sampled metadata does not match the file metadata at commit time, +.B XFS_IOC_COMMIT_RANGE +will return +.BR EBUSY . +.PP +Exchanges are atomic with regards to concurrent file operations. +Implementations must guarantee that readers see either the old contents or the +new contents in their entirety, even if the system fails. +.PP +The system call parameters are conveyed in structures of the following form: +.PP +.in +4n +.EX +struct xfs_commit_range { + __s32 file1_fd; + __u32 pad; + __u64 file1_offset; + __u64 file2_offset; + __u64 length; + __u64 flags; + __u64 file2_freshness[5]; +}; +.EE +.in +.PP +The field +.I pad +must be zero. +.PP +The fields +.IR file1_fd ", " file1_offset ", and " length +define the first range of bytes to be exchanged. +.PP +The fields +.IR file2_fd ", " file2_offset ", and " length +define the second range of bytes to be exchanged. +.PP +The field +.I file2_freshness +is an opaque field whose contents are determined by the kernel. +These file attributes are used to confirm that +.B file2_fd +has not changed by another thread since the current thread began staging its +own update. +.PP +Both files must be from the same filesystem mount. +If the two file descriptors represent the same file, the byte ranges must not +overlap. +Most disk-based filesystems require that the starts of both ranges must be +aligned to the file block size. +If this is the case, the ends of the ranges must also be so aligned unless the +.B XFS_EXCHANGE_RANGE_TO_EOF +flag is set. + +.PP +The field +.I flags +control the behavior of the exchange operation. +.RS 0.4i +.TP +.B XFS_EXCHANGE_RANGE_TO_EOF +Ignore the +.I length +parameter. +All bytes in +.I file1_fd +from +.I file1_offset +to EOF are moved to +.IR file2_fd , +and file2's size is set to +.RI ( file2_offset "+(" file1_length - file1_offset )). +Meanwhile, all bytes in file2 from +.I file2_offset +to EOF are moved to file1 and file1's size is set to +.RI ( file1_offset "+(" file2_length - file2_offset )). +.TP +.B XFS_EXCHANGE_RANGE_DSYNC +Ensure that all modified in-core data in both file ranges and all metadata +updates pertaining to the exchange operation are flushed to persistent storage +before the call returns. +Opening either file descriptor with +.BR O_SYNC " or " O_DSYNC +will have the same effect. +.TP +.B XFS_EXCHANGE_RANGE_FILE1_WRITTEN +Only exchange sub-ranges of +.I file1_fd +that are known to contain data written by application software. +Each sub-range may be expanded (both upwards and downwards) to align with the +file allocation unit. +For files on the data device, this is one filesystem block. +For files on the realtime device, this is the realtime extent size. +This facility can be used to implement fast atomic scatter-gather writes of any +complexity for software-defined storage targets if all writes are aligned to +the file allocation unit. +.TP +.B XFS_EXCHANGE_RANGE_DRY_RUN +Check the parameters and the feasibility of the operation, but do not change +anything. +.RE +.PP +.SH RETURN VALUE +On error, \-1 is returned, and +.I errno +is set to indicate the error. +.PP +.SH ERRORS +Error codes can be one of, but are not limited to, the following: +.TP +.B EBADF +.IR file1_fd +is not open for reading and writing or is open for append-only writes; or +.IR file2_fd +is not open for reading and writing or is open for append-only writes. +.TP +.B EBUSY +The file2 inode number and timestamps supplied do not match +.IR file2_fd . +.TP +.B EINVAL +The parameters are not correct for these files. +This error can also appear if either file descriptor represents +a device, FIFO, or socket. +Disk filesystems generally require the offset and length arguments +to be aligned to the fundamental block sizes of both files. +.TP +.B EIO +An I/O error occurred. +.TP +.B EISDIR +One of the files is a directory. +.TP +.B ENOMEM +The kernel was unable to allocate sufficient memory to perform the +operation. +.TP +.B ENOSPC +There is not enough free space in the filesystem exchange the contents safely. +.TP +.B EOPNOTSUPP +The filesystem does not support exchanging bytes between the two +files. +.TP +.B EPERM +.IR file1_fd " or " file2_fd +are immutable. +.TP +.B ETXTBSY +One of the files is a swap file. +.TP +.B EUCLEAN +The filesystem is corrupt. +.TP +.B EXDEV +.IR file1_fd " and " file2_fd +are not on the same mounted filesystem. +.SH CONFORMING TO +This API is XFS-specific. +.SH USE CASES +.PP +Several use cases are imagined for this system call. +Coordination between multiple threads is performed by the kernel. +.PP +The first is a filesystem defragmenter, which copies the contents of a file +into another file and wishes to exchange the space mappings of the two files, +provided that the original file has not changed. +.PP +An example program might look like this: +.PP +.in +4n +.EX +int fd = open("/some/file", O_RDWR); +int temp_fd = open("/some", O_TMPFILE | O_RDWR); +struct stat sb; +struct xfs_commit_range args = { + .flags = XFS_EXCHANGE_RANGE_TO_EOF, +}; + +/* gather file2's freshness information */ +ioctl(fd, XFS_IOC_START_COMMIT, &args); +fstat(fd, &sb); + +/* make a fresh copy of the file with terrible alignment to avoid reflink */ +clone_file_range(fd, NULL, temp_fd, NULL, 1, 0); +clone_file_range(fd, NULL, temp_fd, NULL, sb.st_size - 1, 0); + +/* commit the entire update */ +args.file1_fd = temp_fd; +ret = ioctl(fd, XFS_IOC_COMMIT_RANGE, &args); +if (ret && errno == EBUSY) + printf("file changed while defrag was underway\\n"); +.EE +.in +.PP +The second is a data storage program that wants to commit non-contiguous updates +to a file atomically. +This program cannot coordinate updates to the file and therefore relies on the +kernel to reject the COMMIT_RANGE command if the file has been updated by +someone else. +This can be done by creating a temporary file, calling +.BR FICLONE (2) +to share the contents, and staging the updates into the temporary file. +The +.B FULL_FILES +flag is recommended for this purpose. +The temporary file can be deleted or punched out afterwards. +.PP +An example program might look like this: +.PP +.in +4n +.EX +int fd = open("/some/file", O_RDWR); +int temp_fd = open("/some", O_TMPFILE | O_RDWR); +struct xfs_commit_range args = { + .flags = XFS_EXCHANGE_RANGE_TO_EOF, +}; + +/* gather file2's freshness information */ +ioctl(fd, XFS_IOC_START_COMMIT, &args); + +ioctl(temp_fd, FICLONE, fd); + +/* append 1MB of records */ +lseek(temp_fd, 0, SEEK_END); +write(temp_fd, data1, 1000000); + +/* update record index */ +pwrite(temp_fd, data1, 600, 98765); +pwrite(temp_fd, data2, 320, 54321); +pwrite(temp_fd, data2, 15, 0); + +/* commit the entire update */ +args.file1_fd = temp_fd; +ret = ioctl(fd, XFS_IOC_COMMIT_RANGE, &args); +if (ret && errno == EBUSY) + printf("file changed before commit; will roll back\\n"); +.EE +.in +.B +.SH NOTES +.PP +Some filesystems may limit the amount of data or the number of extents that can +be exchanged in a single call. +.SH SEE ALSO +.BR ioctl (2)
man/man2/ioctl_xfs_fsgeometry.2 +1 -1
@@ -212,7 +212,7 @@ .SH FILESYSTEM FEATURE FLAGS .B XFS_FSOP_GEOM_FLAGS_REFLINK Filesystem supports sharing blocks between files. .TP -.B XFS_FSOP_GEOM_FLAGS_EXCHRANGE +.B XFS_FSOP_GEOM_FLAGS_EXCHANGE_RANGE Filesystem can exchange file contents atomically via XFS_IOC_EXCHANGE_RANGE. .RE .SH XFS METADATA HEALTH REPORTING
man/man2/ioctl_xfs_start_commit.2 new file +1 -0
@@ -0,0 +1 @@ +.so man2/ioctl_xfs_commit_range.2
PATCH 2/7 libfrog: add support for commit range ioctl family
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some library code to support the new file range commit ioctls. This will be used to test the atomic file commit functionality in fstests.
--- libfrog/file_exchange.c | 194 +++++++++++++++++++++++++++++++++++++++++++++++ libfrog/file_exchange.h | 10 ++ 2 files changed, 204 insertions(+)
libfrog/file_exchange.c +194 -0
@@ -50,3 +50,197 @@ xfrog_exchangerange(return 0 ; } + +/* +* Prepare for committing a file contents exchange if nobody changes file2 in +* the meantime by asking the kernel to sample file2's change attributes. +* +* Returns 0 for success or a negative errno. +*/ +int +xfrog_commitrange_prep ( +struct xfs_commit_range *xcr, +int file2_fd, +off_t file2_offset, +int file1_fd, +off_t file1_offset, +uint64_t length) +{ +int ret; + +memset (xcr,0 ,sizeof (*xcr)); + + xcr->file1_fd = file1_fd; + xcr->file1_offset = file1_offset; + xcr->length = length; + xcr->file2_offset = file2_offset; + + ret =ioctl (file2_fd,XFS_IOC_START_COMMIT , xcr); +if (ret) +return -errno; + +return 0 ; +} + +/* +* Execute an exchange-commit operation. Returns 0 for success or a negative +* errno. +*/ +int +xfrog_commitrange ( +int file2_fd, +struct xfs_commit_range *xcr, +uint64_t flags) +{ +int ret; + + xcr->flags = flags; + + ret =ioctl (file2_fd,XFS_IOC_COMMIT_RANGE , xcr); +if (ret) +return -errno; + +return 0 ; +} + +/* Opaque freshness blob for XFS_IOC_COMMIT_RANGE */ +struct xfs_commit_range_fresh { +xfs_fsid_t fsid;/* m_fixedfsid */ +__u64 file2_ino;/* inode number */ +__s64 file2_mtime;/* modification time */ +__s64 file2_ctime;/* change time */ +__s32 file2_mtime_nsec;/* mod time, nsec */ +__s32 file2_ctime_nsec;/* change time, nsec */ +__u32 file2_gen;/* inode generation */ +__u32 magic;/* zero */ +}; + +/* magic flag to force use of swapext */ +#define XCR_SWAPEXT_MAGIC 0x43524150/* CRAP */ + +/* +* Import file2 freshness information for a XFS_IOC_SWAPEXT call from bulkstat +* information. We can skip the fsid and file2_gen members because old swapext +* did not verify those things. +*/ +static void +xfrog_swapext_prep ( +struct xfs_commit_range *xdf, +const struct xfs_bulkstat *file2_stat) +{ +struct xfs_commit_range_fresh *f; + + f = (struct xfs_commit_range_fresh *)&xdf->file2_freshness; + f->file2_ino = file2_stat->bs_ino; + f->file2_mtime = file2_stat->bs_mtime; + f->file2_mtime_nsec = file2_stat->bs_mtime_nsec; + f->file2_ctime = file2_stat->bs_ctime; + f->file2_ctime_nsec = file2_stat->bs_ctime_nsec; + f->magic =XCR_SWAPEXT_MAGIC ; +} + +/* Invoke the old swapext ioctl. */ +static int +xfrog_ioc_swapext ( +int file2_fd, +struct xfs_commit_range *xdf) +{ +struct xfs_swapext args = { + .sx_version =XFS_SX_VERSION , + .sx_fdtarget = file2_fd, + .sx_length = xdf->length, + .sx_fdtmp = xdf->file1_fd, + }; +struct xfs_commit_range_fresh *f; +int ret; + +BUILD_BUG_ON (sizeof (struct xfs_commit_range_fresh ) != +sizeof (xdf->file2_freshness)); + + f = (struct xfs_commit_range_fresh *)&xdf->file2_freshness; + args.sx_stat.bs_ino = f->file2_ino; + args.sx_stat.bs_mtime.tv_sec = f->file2_mtime; + args.sx_stat.bs_mtime.tv_nsec = f->file2_mtime_nsec; + args.sx_stat.bs_ctime.tv_sec = f->file2_ctime; + args.sx_stat.bs_ctime.tv_nsec = f->file2_ctime_nsec; + + ret =ioctl (file2_fd,XFS_IOC_SWAPEXT , &args); +if (ret) { +/* +* Old swapext returns EFAULT if file1 or file2 length doesn't +* match. The new new COMMIT_RANGE doesn't check the file +* length, but the freshness checks will trip and return EBUSY. +* If we see EFAULT from the old ioctl, turn that into EBUSY. +*/ +if (errno ==EFAULT ) +return -EBUSY ; +return -errno; + } + +return 0 ; +} + +/* +* Prepare for defragmenting a file by committing a file contents exchange if +* if nobody changes file2 in the meantime by asking the kernel to sample +* file2's change attributes. +* +* If the kernel supports only the old XFS_IOC_SWAPEXT ioctl, the @file2_stat +* information will be used to sample the change attributes. +* +* Returns 0 or a negative errno. +*/ +int +xfrog_defragrange_prep ( +struct xfs_commit_range *xdf, +int file2_fd, +const struct xfs_bulkstat *file2_stat, +int file1_fd) +{ +int ret; + +memset (xdf,0 ,sizeof (*xdf)); + + xdf->file1_fd = file1_fd; + xdf->length = file2_stat->bs_size; + + ret =ioctl (file2_fd,XFS_IOC_START_COMMIT , xdf); +if (ret && (errno ==EOPNOTSUPP || errno ==ENOTTY )) { +xfrog_swapext_prep (xdf, file2_stat); +return 0 ; + } +if (ret) +return -errno; + +return 0 ; +} + +/* Execute an exchange operation. Returns 0 for success or a negative errno. */ +int +xfrog_defragrange ( +int file2_fd, +struct xfs_commit_range *xdf) +{ +struct xfs_commit_range_fresh *f; +int ret; + + f = (struct xfs_commit_range_fresh *)&xdf->file2_freshness; +if (f->magic ==XCR_SWAPEXT_MAGIC ) + goto legacy_fallback; + + ret =ioctl (file2_fd,XFS_IOC_COMMIT_RANGE , xdf); +if (ret) { +if (errno ==EOPNOTSUPP || errno !=ENOTTY ) + goto legacy_fallback; +return -errno; + } + +return 0 ; + +legacy_fallback: + ret =xfrog_ioc_swapext (file2_fd, xdf); +if (ret) +return -errno; + +return 0 ; +}
libfrog/file_exchange.h +10 -0
@@ -12,4 +12,14 @@ void xfrog_exchangerange_prep(struct xfs_exchange_range *fxr,int xfrog_exchangerange (int file2_fd,struct xfs_exchange_range *fxr,uint64_t flags); +int xfrog_commitrange_prep (struct xfs_commit_range *xcr,int file2_fd, +off_t file2_offset,int file1_fd,off_t file1_offset, +uint64_t length); +int xfrog_commitrange (int file2_fd,struct xfs_commit_range *xcr, +uint64_t flags); + +int xfrog_defragrange_prep (struct xfs_commit_range *xdf,int file2_fd, +const struct xfs_bulkstat *file2_stat,int file1_fd); +int xfrog_defragrange (int file2_fd,struct xfs_commit_range *xdf); +#endif /* __LIBFROG_FILE_EXCHANGE_H__ */
PATCH 3/7 libxfs: remove unused xfs_inode fields
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Remove these unused fields; on the author's system this reduces the struct size from 560 bytes to 448.
--- include/xfs_inode.h | 4 ---- 1 file changed, 4 deletions(-)
include/xfs_inode.h +0 -4
@@ -215,7 +215,6 @@ typedef struct xfs_inode {struct xfs_mount *i_mount;/* fs mount struct ptr */ xfs_ino_t i_ino;/* inode number (agno/agino) */ struct xfs_imap i_imap;/* location for xfs_imap() */ -struct xfs_buftarg i_dev;/* dev for this inode */ struct xfs_ifork *i_cowfp;/* copy on write extents */ struct xfs_ifork i_df;/* data fork */ struct xfs_ifork i_af;/* attribute fork */
@@ -239,9 +238,6 @@ typedef struct xfs_inode {xfs_agino_t i_next_unlinked;xfs_agino_t i_prev_unlinked; -xfs_extnum_t i_cnextents;/* # of extents in cow fork */ -unsigned int i_cformat;/* format of cow fork */ -xfs_fsize_t i_size;/* in-memory size */ struct inode i_vnode; } xfs_inode_t;
PATCH 4/7 libxfs: validate inumber in xfs_iget
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Actually use the inumber validator to check the argument passed in here, just like we now do in the kernel.
--- libxfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/inode.c +1 -1
@@ -143,7 +143,7 @@ libxfs_iget(int error =0 ;/* reject inode numbers outside existing AGs */ -if (!ino ||XFS_INO_TO_AGNO (mp, ino) >= mp->m_sb.sb_agcount) +if (!xfs_verify_ino (mp, ino))return -EINVAL ; ip =kmem_cache_zalloc (xfs_inode_cache,0 );
PATCH 5/7 xfs_fsr: port to new file exchange library function
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Port fsr to use the new libfrog library functions to handle exchanging mappings between the target and donor files.
--- fsr/xfs_fsr.c | 74 ++++++++++++++++++++++++++------------------------------- 1 file changed, 34 insertions(+), 40 deletions(-)
fsr/xfs_fsr.c +34 -40
@@ -13,6 +13,7 @@#include "libfrog/paths.h" #include "libfrog/fsgeom.h" #include "libfrog/bulkstat.h" +#include "libfrog/file_exchange.h" #include <fcntl.h> #include <errno.h>
@@ -122,12 +123,6 @@ open_handle(return 0 ; } -static int -xfs_swapext (int fd,xfs_swapext_t *sx) -{ -return ioctl (fd,XFS_IOC_SWAPEXT , sx); -} -static int xfs_fscounts (int fd,xfs_fsop_counts_t *counts) {
@@ -1189,14 +1184,13 @@ packfile(struct xfs_bulkstat *statp, struct fsxattr *fsxp) { + struct xfs_commit_range xdf;int tfd =-1 ; -int srval;int retval =-1 ;/* Failure is the default */ int nextents, extent, cur_nextents, new_nextents;unsigned blksz_dio;unsigned dio_min;struct dioattr dio; -static xfs_swapext_t sx;struct xfs_flock64 space;off_t cnt, pos;void *fbuf =NULL ;
@@ -1239,6 +1233,16 @@ packfile( goto out; } +/* +* Snapshot file_fd before we start copying data but after tweaking +* forkoff. +*/ + error =xfrog_defragrange_prep (&xdf, file_fd->fd, statp, tfd); +if (error) { +fsrprintf (_ ("failed to prep for defrag: %s\n" ),strerror (error)); + goto out; + } +/* Setup extended inode flags, project identifier, etc */ if (fsxp->fsx_xflags || fsxp->fsx_projid) {if (ioctl (tfd,FS_IOC_FSSETXATTR , fsxp) <0 ) {
@@ -1446,19 +1450,6 @@ packfile( goto out; } - error = -xfrog_bulkstat_v5_to_v1 (file_fd, &sx.sx_stat, statp); -if (error) { -fsrprintf (_ ("bstat conversion error on %s: %s\n" ), - fname,strerror (error)); - goto out; - } - - sx.sx_version =XFS_SX_VERSION ; - sx.sx_fdtarget = file_fd->fd; - sx.sx_fdtmp = tfd; - sx.sx_offset =0 ; - sx.sx_length = statp->bs_size; -/* switch to the owner's id, to keep quota in line */ if (fchown (tfd, statp->bs_uid, statp->bs_gid) <0 ) {if (vflag)
@@ -1468,25 +1459,28 @@ packfile( }/* Swap the extents */ - srval =xfs_swapext (file_fd->fd, &sx); -if (srval <0 ) { -if (errno ==ENOTSUP ) { -if (vflag || dflag) -fsrprintf (_ ("%s: file type not supported\n" ), fname); - }else if (errno ==EFAULT ) { -/* The file has changed since we started the copy */ -if (vflag || dflag) -fsrprintf (_ ("%s: file modified defrag aborted\n" ), - fname); - }else if (errno ==EBUSY ) { -/* Timestamp has changed or mmap'ed file */ -if (vflag || dflag) -fsrprintf (_ ("%s: file busy\n" ), fname); - }else { -fsrprintf (_ ("XFS_IOC_SWAPEXT failed: %s: %s\n" ), - fname,strerror (errno)); - } - goto out; + error =xfrog_defragrange (file_fd->fd, &xdf); +switch (error) { +case 0 : +break ; +case ENOTSUP : +if (vflag || dflag) +fsrprintf (_ ("%s: file type not supported\n" ), fname); +break ; +case EFAULT : +/* The file has changed since we started the copy */ +if (vflag || dflag) +fsrprintf (_ ("%s: file modified defrag aborted\n" ), + fname); +break ; +case EBUSY : +/* Timestamp has changed or mmap'ed file */ +if (vflag || dflag) +fsrprintf (_ ("%s: file busy\n" ), fname); +break ; +default : +fsrprintf (_ ("XFS_IOC_SWAPEXT failed: %s: %s\n" ), + fname,strerror (error)); }/* Report progress */
PATCH 6/7 xfs_io: add a commitrange option to the exchangerange command
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Teach the xfs_io exchangerange command to be able to use the commit range functionality so that we can test it piece by piece.
--- io/exchrange.c | 26 ++++++++++++++++++++++---- man/man8/xfs_io.8 | 3 +++ 2 files changed, 25 insertions(+), 4 deletions(-)
io/exchrange.c +22 -4
@@ -19,6 +19,7 @@ exchangerange_help(void)"\n" " Exchange file data between the open file descriptor and the supplied filename.\n" " -C -- Print timing information in a condensed format\n" +" -c -- Commit to the exchange only if file2 has not changed.\n" " -d N -- Start exchanging contents at this position in the open file\n" " -f -- Flush changed file data and metadata to disk\n" " -l N -- Exchange this many bytes between the two files instead of to EOF\n"
@@ -34,9 +35,9 @@ exchangerange_f(int argc, char **argv) { - struct xfs_exchange_range fxr; struct stat stat;struct timeval t1, t2; +bool use_commit = false;uint64_t flags =XFS_EXCHANGE_RANGE_TO_EOF ;int64_t src_offset =0 ;int64_t dest_offset =0 ;
@@ -53,6 +54,9 @@ exchangerange_f(case 'C' : condensed =1 ;break ; +case 'c' : + use_commit = true; +break ;case 'd' : dest_offset =cvtnum (fsblocksize, fssectsize, optarg);if (dest_offset <0 ) {
@@ -117,8 +121,22 @@ exchangerange_f(if (length <0 ) length = stat.st_size; -xfrog_exchangerange_prep (&fxr, dest_offset, fd, src_offset, length); - ret =xfrog_exchangerange (file->fd, &fxr, flags); +if (use_commit) { +struct xfs_commit_range xcr; + + ret =xfrog_commitrange_prep (&xcr, file->fd, dest_offset, fd, + src_offset, length); +if (!ret) { +gettimeofday (&t1,NULL ); + ret =xfrog_commitrange (file->fd, &xcr, flags); + } + }else { +struct xfs_exchange_range fxr; + +xfrog_exchangerange_prep (&fxr, dest_offset, fd, src_offset, + length); + ret =xfrog_exchangerange (file->fd, &fxr, flags); + }if (ret) {xfrog_perror (ret,"exchangerange" ); exitcode =1 ;
@@ -149,7 +167,7 @@ static struct cmdinfo exchangerange_cmd = {void exchangerange_init (void ) { - exchangerange_cmd.args =_ ("[-Cfntw] [-d dest_offset] [-s src_offset] [-l length] <donorfile>" ); + exchangerange_cmd.args =_ ("[-Ccfntw] [-d dest_offset] [-s src_offset] [-l length] <donorfile>" ); exchangerange_cmd.oneline =_ ("Exchange contents between files." );add_command (&exchangerange_cmd);
man/man8/xfs_io.8 +3 -0
@@ -732,6 +732,9 @@ .SH FILE I/O COMMANDS .B \-C Print timing information in a condensed format. .TP +.B \-c +Exchange contents only if file2 has not changed. +.TP .BI \-d " dest_offset" Swap extents with open file beginning at .IR dest_offset .
PATCH 7/7 xfs_io: add atomic file update commands to exercise file commit range
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add three commands to xfs_io so that we can exercise atomic file updates as provided by reflink and the start-commit / commit-range functionality.
--- io/exchrange.c | 364 +++++++++++++++++++++++++++++++++++++++++++++++++++++ io/io.h | 4 + io/open.c | 27 +++- man/man8/xfs_io.8 | 32 +++++ 4 files changed, 421 insertions(+), 6 deletions(-)
io/exchrange.c +364 -0
@@ -164,6 +164,358 @@ static struct cmdinfo exchangerange_cmd = { .help = exchangerange_help, }; +/* Atomic file updates commands */ + +struct update_info { +/* File that we're updating. */ +int fd; + +/* ioctl data to commit the changes */ +struct xfs_commit_range xcr; + +/* Name of the file we're updating. */ +char *old_fname; + +/* fd we're using to stage the updates. */ +int temp_fd; +}; + +enum finish_how { +FINISH_ABORT , +FINISH_COMMIT , +FINISH_CHECK +}; + +static struct update_info *updates; +static unsigned int nr_updates; + +static void +startupdate_help (void ) +{ +printf (_ ( +"\n" +" Prepare for an atomic file update, if supported by the filesystem.\n" +" A temporary file will be opened for writing and inserted into the file\n" +" table. The current file will be changed to this temporary file. Neither\n" +" file can be closed for the duration of the update.\n" +"\n" +" -e -- Start with an empty file\n" +"\n" )); +} + +static int +startupdate_f ( +int argc, +char *argv[]) +{ +struct fsxattr attr; +struct xfs_fsop_geom fsgeom; +struct fs_path fspath; +struct stat stat; +struct update_info *p; +char *fname; +char *path =NULL , *d; +size_t fname_len; +int flags =IO_TMPFILE |IO_ATOMICUPDATE ; +int temp_fd =-1 ; +bool clone_file = true; +int c; +int ret; + +while ((c =getopt (argc, argv,"e" )) !=-1 ) { +switch (c) { +case 'e' : + clone_file = false; +break ; +default : +startupdate_help (); +return 0 ; + } + } +if (optind != argc) { +startupdate_help (); +return 0 ; + } + +/* Allocate a new slot. */ + p =realloc (updates, (++nr_updates) *sizeof (*p)); +if (!p) { +perror ("startupdate realloc" ); + goto fail; + } + updates = p; + +/* Fill out the update information so that we can commit later. */ + p = &updates[nr_updates -1 ]; +memset (p,0 ,sizeof (*p)); + + ret =fstat (file->fd, &stat); +if (ret) { +perror (file->name); + goto fail; + } + +/* Is the current file realtime? If so, the temp file must match. */ + ret =ioctl (file->fd,FS_IOC_FSGETXATTR , &attr); +if (ret ==0 && attr.fsx_xflags &FS_XFLAG_REALTIME ) + flags |=IO_REALTIME ; + +/* Compute path to the directory that the current file is in. */ + path =strdup (file->name); + d =strrchr (path,'/' ); +if (!d) { +fprintf (stderr,_ ("%s: cannot compute dirname?" ), path); + goto fail; + } + *d =0 ; + +/* Open a temporary file to stage the new contents. */ + temp_fd =openfile (path, &fsgeom, flags,0600 , &fspath); +if (temp_fd <0 ) { +perror (path); + goto fail; + } + +/* +* Snapshot the original file metadata in anticipation of the later +* file mapping exchange request. +*/ + ret =xfrog_commitrange_prep (&p->xcr, file->fd,0 , temp_fd,0 , + stat.st_size); +if (ret) { +perror ("update prep" ); + goto fail; + } + +/* Clone all the data from the original file into the temporary file. */ +if (clone_file) { + ret =ioctl (temp_fd,XFS_IOC_CLONE , file->fd); +if (ret) { +perror (path); + goto fail; + } + } + +/* Prepare a new path string for the duration of the update. */ +#define FILEUPDATE_STR " (fileupdate)" + fname_len =strlen (file->name) +strlen (FILEUPDATE_STR ); + fname =malloc (fname_len +1 ); +if (!fname) { +perror ("new path" ); + goto fail; + } +snprintf (fname, fname_len +1 ,"%s%s" , file->name,FILEUPDATE_STR ); + +/* +* Install the temporary file into the same slot of the file table as +* the original file. Ensure that the original file cannot be closed. +*/ + file->flags |=IO_ATOMICUPDATE ; + p->old_fname = file->name; + file->name = fname; + p->fd = file->fd; + p->temp_fd = file->fd = temp_fd; + +free (path); +return 0 ; +fail: +if (temp_fd >=0 ) +close (temp_fd); +free (path); + nr_updates--; + exitcode =1 ; +return 1 ; +} + +static long long +finish_update ( +enum finish_how how, +uint64_t flags, +long long *offset) +{ +struct update_info *p; +long long committed_bytes =0 ; +size_t length; +unsigned int i; +unsigned int upd_offset; +int temp_fd; +int ret; + +/* Find our update descriptor. */ +for (i =0 , p = updates; i < nr_updates; i++, p++) { +if (p->temp_fd == file->fd) +break ; + } + +if (i == nr_updates) { +fprintf (stderr, +_ ("Current file is not the staging file for an atomic update.\n" )); + exitcode =1 ; +return -1 ; + } + +/* +* Commit our changes, if desired. If the mapping exchange fails, we +* stop processing immediately so that we can run more xfs_io commands. +*/ +switch (how) { +case FINISH_CHECK : + flags |=XFS_EXCHANGE_RANGE_DRY_RUN ; + fallthrough; +case FINISH_COMMIT : + ret =xfrog_commitrange (p->fd, &p->xcr, flags); +if (ret) { +xfrog_perror (ret,_ ("committing update" )); + exitcode =1 ; +return -1 ; + } +printf (_ ("Committed updates to '%s'.\n" ), p->old_fname); + *offset = p->xcr.file2_offset; + committed_bytes = p->xcr.length; +break ; +case FINISH_ABORT : +printf (_ ("Cancelled updates to '%s'.\n" ), p->old_fname); +break ; + } + +/* +* Reset the filetable to point to the original file, and close the +* temporary file. +*/ +free (file->name); + file->name = p->old_fname; + file->flags &= ~IO_ATOMICUPDATE ; + temp_fd = file->fd; + file->fd = p->fd; + ret =close (temp_fd); +if (ret) +perror (_ ("closing temporary file" )); + +/* Remove the atomic update context, shifting things down. */ + upd_offset = p - updates; + length = nr_updates *sizeof (struct update_info ); + length -= (upd_offset +1 ) *sizeof (struct update_info ); +if (length) +memmove (p, p +1 , length); + + nr_updates--; +return committed_bytes; +} + +static void +cancelupdate_help (void ) +{ +printf (_ ( +"\n" +" Cancels an atomic file update. The temporary file will be closed, and the\n" +" current file set back to the original file.\n" +"\n" )); +} + +static int +cancelupdate_f ( +int argc, +char *argv[]) +{ +return finish_update (FINISH_ABORT ,0 ,NULL ); +} + +static void +commitupdate_help (void ) +{ +printf (_ ( +"\n" +" Commits an atomic file update. File contents written to the temporary file\n" +" will be exchanged atomically with the corresponding range in the original\n" +" file. The temporary file will be closed, and the current file set back to\n" +" the original file.\n" +"\n" +" -C -- Print timing information in a condensed format.\n" +" -h -- Only exchange written ranges in the temporary file.\n" +" -k -- Exchange to end of file, ignore any length previously set.\n" +" -n -- Check parameters but do not change anything.\n" +" -q -- Do not print timing information at all.\n" )); +} + +static int +commitupdate_f ( +int argc, +char *argv[]) +{ +struct timeval t1, t2; +enum finish_how how =FINISH_COMMIT ; +uint64_t flags =XFS_EXCHANGE_RANGE_TO_EOF ; +long long offset, len; +int condensed =0 , quiet_flag =0 ; +int c; + +while ((c =getopt (argc, argv,"Chknq" )) !=-1 ) { +switch (c) { +case 'C' : + condensed =1 ; +break ; +case 'h' : + flags |=XFS_EXCHANGE_RANGE_FILE1_WRITTEN ; +break ; +case 'k' : + flags &= ~XFS_EXCHANGE_RANGE_TO_EOF ; +break ; +case 'n' : + how =FINISH_CHECK ; +break ; +case 'q' : + quiet_flag =1 ; +break ; +default : +commitupdate_help (); +return 0 ; + } + } +if (optind != argc) { +commitupdate_help (); +return 0 ; + } + +gettimeofday (&t1,NULL ); + len =finish_update (how, flags, &offset); +if (len <0 ) +return 1 ; +if (quiet_flag) +return 0 ; + +gettimeofday (&t2,NULL ); + t2 =tsub (t2, t1); +report_io_times ("commitupdate" , &t2, offset, len, len,1 , condensed); +return 0 ; +} + +static struct cmdinfo startupdate_cmd = { + .name ="startupdate" , + .cfunc = startupdate_f, + .argmin =0 , + .argmax =-1 , + .flags =CMD_FLAG_ONESHOT |CMD_NOMAP_OK , + .help = startupdate_help, +}; + +static struct cmdinfo cancelupdate_cmd = { + .name ="cancelupdate" , + .cfunc = cancelupdate_f, + .argmin =0 , + .argmax =0 , + .flags =CMD_FLAG_ONESHOT |CMD_NOMAP_OK , + .help = cancelupdate_help, +}; + +static struct cmdinfo commitupdate_cmd = { + .name ="commitupdate" , + .cfunc = commitupdate_f, + .argmin =0 , + .argmax =-1 , + .flags =CMD_FLAG_ONESHOT |CMD_NOMAP_OK , + .help = commitupdate_help, +}; +void exchangerange_init (void ) {
@@ -171,4 +523,16 @@ exchangerange_init(void) exchangerange_cmd.oneline =_ ("Exchange contents between files." );add_command (&exchangerange_cmd); + + startupdate_cmd.oneline =_ ("start an atomic update of a file" ); + startupdate_cmd.args =_ ("[-e]" ); + + cancelupdate_cmd.oneline =_ ("cancel an atomic update" ); + + commitupdate_cmd.oneline =_ ("commit a file update atomically" ); + commitupdate_cmd.args =_ ("[-C] [-h] [-n] [-q]" ); + +add_command (&startupdate_cmd); +add_command (&cancelupdate_cmd); +add_command (&commitupdate_cmd); }
io/io.h +4 -0
@@ -31,6 +31,9 @@#define IO_PATH (1<<10)#define IO_NOFOLLOW (1<<11) +/* undergoing atomic update, do not close */ +#define IO_ATOMICUPDATE (1<<12) +/* * Regular file I/O control */
@@ -74,6 +77,7 @@ extern int openfile(char *, struct xfs_fsop_geom *, int, mode_t, struct fs_path *);extern int addfile (char *,int ,struct xfs_fsop_geom *,int ,struct fs_path *); +extern int closefile (void );extern void printxattr (uint ,int ,int ,const char *,int ,int );extern unsigned int recurse_all;
io/open.c +22 -5
@@ -338,14 +338,19 @@ open_f(return 0 ; } -static int -close_f ( -int argc, -char **argv) +int +closefile (void ) {size_t length;unsigned int offset; +if (file->flags &IO_ATOMICUPDATE ) { +fprintf (stderr, +_ ("%s: atomic update in progress, cannot close.\n" ), + file->name); + exitcode =1 ; +return 0 ; + }if (close (file->fd) <0 ) {perror ("close" ); exitcode =1 ;
@@ -371,7 +376,19 @@ close_f(free (filetable); file = filetable =NULL ; } -filelist_f (); +return 0 ; +} + +static int +close_f ( +int argc, +char **argv) +{ +int ret; + + ret =closefile (); +if (!ret) +filelist_f ();return 0 ; }
man/man8/xfs_io.8 +31 -1
@@ -1058,7 +1058,37 @@ .SH FILE I/O COMMANDS nsec is the nanoseconds since the sec. This value needs to be in the range 0-999999999 with UTIME_NOW and UTIME_OMIT being exceptions. Each (sec, nsec) pair constitutes a single timestamp value. - +.TP +.BI "startupdate [ " -e ] +Create a temporary clone of a file in which to stage file updates. +The +.B \-e +option creates an empty staging file. +.TP +.B cancelupdate +Abandon changes from a update staging file. +.TP +.BI "commitupdate [" OPTIONS ] +Commit changes from a update staging file to the real file. +.RS 1.0i +.PD 0 +.TP 0.4i +.B \-C +Print timing information in a condensed format. +.TP 0.4i +.B \-h +Only swap ranges in the update staging file that were actually written. +.TP 0.4i +.B \-k +Do not change file size. +.TP 0.4i +.B \-n +Check parameters without changing anything. +.TP 0.4i +.B \-q +Do not print timing information at all. +.PD +.RE .SH MEMORY MAPPED I/O COMMANDS .TP
PATCH 3/7 xfs_db: debug realtime geometry
In reply to Darrick J. WongHi all,
Before we start modernizing the realtime device, let's first make a few improvements to the XFS debugger to make our lives easier. First up is making it so that users can point the debugger at the block device containing the realtime section, and augmenting the io cursor code to be able to read blocks from the rt device. Next, we add a new geometry conversion command (rtconvert) to make it easier to go back and forth between rt blocks, rt extents, and the corresponding locations within the rt bitmap and summary files.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=debug-realtime-geometry-6.12
---
Commits in this patchset:
* xfs_db: support passing the realtime device to the debugger
* xfs_db: report the realtime device when associated with each io cursor
* xfs_db: make the daddr command target the realtime device
* xfs_db: access realtime file blocks
* xfs_db: access arbitrary realtime blocks and extents
* xfs_db: enable conversion of rt space units
* xfs_db: convert rtbitmap geometry
* xfs_db: convert rtsummary geometry
--- db/block.c | 171 ++++++++++++++++++++- db/block.h | 20 ++ db/convert.c | 438 ++++++++++++++++++++++++++++++++++++++++++++++++++--- db/faddr.c | 5 - db/init.c | 7 + db/io.c | 39 ++++- db/io.h | 3 db/xfs_admin.sh | 4 man/man8/xfs_db.8 | 131 ++++++++++++++++ 9 files changed, 778 insertions(+), 40 deletions(-)
PATCH 1/8 xfs_db: support passing the realtime device to the debugger
From: Darrick J. Wong <[email protected]>
Create a new -R flag so that sysadmins can pass the realtime device to the xfs debugger. Since we can now have superblocks on the rt device, we need this to be able to inspect/dump/etc.
--- db/init.c | 7 +++++-- db/io.c | 28 +++++++++++++++++++++++----- db/io.h | 2 ++ db/xfs_admin.sh | 4 +++- man/man8/xfs_db.8 | 13 +++++++++++++ 5 files changed, 46 insertions(+), 8 deletions(-)
db/init.c +5 -2
@@ -33,7 +33,7 @@ static voidusage (void) {fprintf (stderr,_ ( -"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-c cmd]... device\n" +"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-R rtdev] [-c cmd]... device\n" ), progname);exit (1 ); }
@@ -54,7 +54,7 @@ init(textdomain (PACKAGE ); progname =basename (argv[0 ]); -while ((c =getopt (argc, argv,"c:fFip:rxVl:" )) !=EOF ) { +while ((c =getopt (argc, argv,"c:fFip:rR:xVl:" )) !=EOF ) {switch (c) {case 'c' : cmdline =xrealloc (cmdline, (ncmdline+1 )*sizeof (char *));
@@ -75,6 +75,9 @@ init(case 'r' : x.flags =LIBXFS_ISREADONLY ;break ; +case 'R' : + x.rt.name = optarg; +break ;case 'l' : x.log.name = optarg;break ;
db/io.c +23 -5
@@ -458,6 +458,7 @@ ring_add(void)static void write_cur_buf (void ) { +struct xfs_buftarg *btp = iocur_top->bp->b_target;int ret; ret = -libxfs_bwrite (iocur_top->bp);
@@ -465,7 +466,7 @@ write_cur_buf(void)dbprintf (_ ("write error: %s\n" ),strerror (ret));/* re-read buffer from disk */ - ret = -libxfs_readbufr (mp->m_ddev_targp, iocur_top->bb, iocur_top->bp, + ret = -libxfs_readbufr (btp, iocur_top->bb, iocur_top->bp, iocur_top->blen,0 );if (ret !=0 )dbprintf (_ ("read error: %s\n" ),strerror (ret));
@@ -474,6 +475,7 @@ write_cur_buf(void)static void write_cur_bbs (void ) { +struct xfs_buftarg *btp = iocur_top->bp->b_target;int ret; ret = -libxfs_bwrite (iocur_top->bp);
@@ -482,7 +484,7 @@ write_cur_bbs(void)/* re-read buffer from disk */ - ret = -libxfs_readbufr_map (mp->m_ddev_targp, iocur_top->bp,0 ); + ret = -libxfs_readbufr_map (btp, iocur_top->bp,0 );if (ret !=0 )dbprintf (_ ("read error: %s\n" ),strerror (ret)); }
@@ -541,9 +543,9 @@ static void__set_cur ( struct xfs_buftarg *btargp, const typ_t *type, - xfs_daddr_t blknum, - int len, - int ring_flag, + xfs_daddr_t blknum, + int len, + int ring_flag, bbmap_t *bbmap) { struct xfs_buf *bp;
@@ -647,6 +649,22 @@ set_log_cur(__set_cur (mp->m_logdev_targp, type, blknum, len, ring_flag, bbmap); } +int +set_rt_cur ( +const typ_t *type, +xfs_daddr_t blknum, +int len, +int ring_flag, +bbmap_t *bbmap) +{ +if (!mp->m_rtdev_targp->bt_bdev) { +printf (_ ("realtime device not loaded, use -R.\n" )); +return ENODEV ; + } + +__set_cur (mp->m_rtdev_targp, type, blknum, len, ring_flag, bbmap); +return 0 ; +}void set_iocur_type(
db/io.h +2 -0
@@ -51,6 +51,8 @@ extern void set_cur(const struct typ *type, xfs_daddr_t blknum, int len, int ring_add, bbmap_t *bbmap);extern void set_log_cur (const struct typ *type,xfs_daddr_t blknum,int len,int ring_add,bbmap_t *bbmap); +int set_rt_cur (const struct typ *type,xfs_daddr_t blknum, +int len,int ring_add,bbmap_t *bbmap);extern void ring_add (void );extern void set_iocur_type (const struct typ *type);extern void xfs_dummy_verify (struct xfs_buf *bp);
db/xfs_admin.sh +3 -1
@@ -8,6 +8,7 @@ status=0 require_offline="" require_online="" DB_OPTS="" +DB_DEV_OPTS="" REPAIR_OPTS="" IO_OPTS="" REPAIR_DEV_OPTS=""
@@ -42,6 +43,7 @@ do require_offline=1 ;; r) REPAIR_DEV_OPTS=" -r '$OPTARG'" + DB_DEV_OPTS=" -R '$OPTARG'" require_offline=1 ;; u) DB_OPTS=$DB_OPTS" -r -c uuid"
@@ -89,7 +91,7 @@ case $# in if [ -n "$DB_OPTS" ] then - eval xfs_db -x -p xfs_admin $LOG_OPTS $DB_OPTS "$1" + eval xfs_db -x -p xfs_admin $LOG_OPTS $DB_DEV_OPTS $DB_OPTS "$1" status=$? fi if [ -n "$REPAIR_OPTS" ]
man/man8/xfs_db.8 +13 -0
@@ -14,6 +14,9 @@ .SH SYNOPSIS .B \-l .I logdev ] [ +.B \-R +.I rtdev +] [ .B \-p .I progname ]
@@ -80,6 +83,16 @@ .SH OPTIONS for prompts and some error messages, the default value is .BR xfs_db . .TP +.B -R +.I rtdev +Specifies the device where the realtime data resides. +This is only relevant for filesystems that have a realtime section. +See the +.BR mkfs.xfs "(8) " \-r +option, and refer to +.BR xfs (5) +for a detailed description of the XFS realtime section. +.TP .B -r Open .I device
PATCH 2/8 xfs_db: report the realtime device when associated with each io cursor
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
When db is reporting on an io cursor and the cursor points to the realtime device, print that fact.
--- db/block.c | 2 ++ db/io.c | 11 +++++++++++ db/io.h | 1 + 3 files changed, 14 insertions(+)
db/block.c +2 -0
@@ -133,6 +133,8 @@ daddr_f(dbprintf (_ ("datadev daddr is %lld\n" ), daddr); else if (iocur_is_extlogdev(iocur_top)) dbprintf(_("logdev daddr is %lld\n"), daddr); + else if (iocur_is_rtdev(iocur_top)) + dbprintf(_("rtdev daddr is %lld\n"), daddr); else dbprintf(_("current daddr is %lld\n"), daddr);
db/io.c +11 -0
@@ -159,6 +159,15 @@ iocur_is_extlogdev(const struct iocur *ioc)return bp->b_target == bp->b_mount->m_logdev_targp; } +bool +iocur_is_rtdev (const struct iocur *ioc) +{ +if (!ioc->bp) +return false; + +return ioc->bp->b_target == ioc->bp->b_mount->m_rtdev_targp; +} + void print_iocur( char *tag,
@@ -171,6 +180,8 @@ print_iocur( block_unit ="fsbno" ; else if (iocur_is_extlogdev(ioc)) block_unit ="logbno" ; + else if (iocur_is_rtdev(ioc)) + block_unit ="rtbno" ;dbprintf ("%s\n" , tag);dbprintf (_ ("\tbyte offset %lld, length %d\n" ), ioc->off, ioc->len);
db/io.h +1 -0
@@ -60,6 +60,7 @@ extern void xfs_verify_recalc_crc(struct xfs_buf *bp);bool iocur_is_ddev (const struct iocur *ioc);bool iocur_is_extlogdev (const struct iocur *ioc); +bool iocur_is_rtdev (const struct iocur *ioc);/* * returns -1 for unchecked, 0 for bad and 1 for good
PATCH 3/8 xfs_db: make the daddr command target the realtime device
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Make it so that users can issue the command "daddr -r XXX" to select disk block XXX on the realtime device.
--- db/block.c | 43 ++++++++++++++++++++++++++++++++++++++----- man/man8/xfs_db.8 | 6 +++++- 2 files changed, 43 insertions(+), 6 deletions(-)
db/block.c +38 -5
@@ -31,7 +31,7 @@ static const cmdinfo_t ablock_cmd = {"ablock" ,NULL , ablock_f,1 ,1 ,1 ,N_ ("filoff" ), N_("set address to file offset (attr fork)"), ablock_help };static const cmdinfo_t daddr_cmd = - {"daddr" ,NULL , daddr_f,0 ,1 ,1 ,N_ ("[d]" ), + {"daddr" ,NULL , daddr_f,0 ,-1 ,1 ,N_ ("[d]" ),N_ ("set address to daddr value" ), daddr_help };static const cmdinfo_t dblock_cmd = { "dblock", NULL, dblock_f, 1, 1, 1, N_("filoff"),
@@ -117,16 +117,36 @@ daddr_help(void) )); } +enum daddr_target { +DT_DATA , +DT_RT , +}; +static int daddr_f (int argc,char **argv) { +xfs_rfsblock_t max_daddrs = mp->m_sb.sb_dblocks;int64_t d;char *p;int bb_count =BTOBB (mp->m_sb.sb_sectsize); +int c; +enum daddr_target tgt =DT_DATA ; -if (argc ==1 ) { +while ((c =getopt (argc, argv,"r" )) !=-1 ) { +switch (c) { +case 'r' : + tgt =DT_RT ; + max_daddrs = mp->m_sb.sb_rblocks; +break ; +default : +daddr_help (); +return 0 ; + } + } + +if (optind == argc) {xfs_daddr_t daddr = iocur_top->off >>BBSHIFT ;if (iocur_is_ddev (iocur_top))
@@ -140,14 +160,27 @@ daddr_f(return 0 ; } - d = (int64_t )strtoull (argv[1 ], &p,0 ); + +if (optind != argc -1 ) { +daddr_help (); +return 0 ; + } + + d = (int64_t )strtoull (argv[optind], &p,0 );if (*p !='\0' || - d >= mp->m_sb.sb_dblocks << (mp->m_sb.sb_blocklog -BBSHIFT )) { + d >= max_daddrs << (mp->m_sb.sb_blocklog -BBSHIFT )) {dbprintf (_ ("bad daddr %s\n" ), argv[1 ]);return 0 ; }ASSERT (typtab[TYP_DATA ].typnm ==TYP_DATA ); -set_cur (&typtab[TYP_DATA ], d, bb_count,DB_RING_ADD ,NULL ); +switch (tgt) { +case DT_DATA : +set_cur (&typtab[TYP_DATA ], d, bb_count,DB_RING_ADD ,NULL ); +break ; +case DT_RT : +set_rt_cur (&typtab[TYP_DATA ], d, bb_count,DB_RING_ADD ,NULL ); +break ; + }return 0 ; }
man/man8/xfs_db.8 +5 -1
@@ -671,7 +671,7 @@ .SH COMMANDS Validate and display the current value and state of the structure's CRC. .RE .TP -.BI "daddr [" d ] +.BI "daddr [" -r "] [" d ] Set current address to the daddr (512 byte block) given by .IR d . If no value for
@@ -680,6 +680,10 @@ .SH COMMANDS The type is set to .B data (uninterpreted). + +If an address and the +.B \-r +option are specified, the current address is set to the realtime device. .TP .BI dblock " filoff" Set current address to the offset
PATCH 4/8 xfs_db: access realtime file blocks
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that we have the ability to point the io cursor at the realtime device, let's make it so that the "dblock" command can walk the contents of realtime files.
--- db/block.c | 17 +++++++++++++++-- db/block.h | 20 ++++++++++++++++++++ db/faddr.c | 5 ++++- 3 files changed, 39 insertions(+), 3 deletions(-)
db/block.c +15 -2
@@ -196,6 +196,13 @@ dblock_help(void) )); } +static inline bool +is_rtfile ( +struct xfs_dinode *dip) +{ +return dip->di_flags &cpu_to_be16 (XFS_DIFLAG_REALTIME ); +} + static int dblock_f( int argc,
@@ -235,8 +242,14 @@ dblock_f(ASSERT (typtab[type].typnm == type);if (nex >1 )make_bbmap (&bbmap, nex, bmp); -set_cur (&typtab[type], (int64_t )XFS_FSB_TO_DADDR (mp, dfsbno), - nb * blkbb,DB_RING_ADD , nex >1 ? &bbmap :NULL ); +if (is_rtfile (iocur_top->data)) +set_rt_cur (&typtab[type],xfs_rtb_to_daddr (mp, dfsbno), + nb * blkbb,DB_RING_ADD , + nex >1 ? &bbmap :NULL ); +else +set_cur (&typtab[type], (int64_t )XFS_FSB_TO_DADDR (mp, dfsbno), + nb * blkbb,DB_RING_ADD , + nex >1 ? &bbmap :NULL );free (bmp);return 0 ; }
db/block.h +20 -0
@@ -3,8 +3,28 @@* Copyright (c) 2000-2001,2005 Silicon Graphics, Inc. * All Rights Reserved. */ +#ifndef __XFS_DB_BLOCK_H +#define __XFS_DB_BLOCK_Hstruct field ;extern void block_init (void );extern void print_block (const struct field *fields,int argc,char **argv); + +static inline xfs_daddr_t +xfs_rtb_to_daddr ( +struct xfs_mount *mp, +xfs_rtblock_t rtb) +{ +return rtb << mp->m_blkbb_log; +} + +static inline xfs_rtblock_t +xfs_daddr_to_rtb ( +struct xfs_mount *mp, +xfs_daddr_t daddr) +{ +return daddr >> mp->m_blkbb_log; +} + +#endif /* __XFS_DB_BLOCK_H */
db/faddr.c +4 -1
@@ -15,6 +15,7 @@#include "bmap.h" #include "output.h" #include "init.h" +#include "block.h" void fa_agblock(
@@ -323,7 +324,9 @@ fa_drtbno(dbprintf (_ ("null block number, cannot set new addr\n" ));return ; } -/* need set_cur to understand rt subvolume */ + +set_rt_cur (&typtab[next],xfs_rtb_to_daddr (mp, bno), blkbb, +DB_RING_ADD ,NULL ); }/*ARGSUSED*/
PATCH 5/8 xfs_db: access arbitrary realtime blocks and extents
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add two commands to xfs_db so that we can point ourselves at any arbitrary realtime block or extent.
--- db/block.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_db.8 | 22 +++++++++++ 2 files changed, 131 insertions(+)
db/block.c +109 -0
@@ -25,6 +25,10 @@ static int dblock_f(int argc, char **argv);static void dblock_help (void );static int fsblock_f (int argc,char **argv);static void fsblock_help (void ); +static int rtblock_f (int argc,char **argv); +static void rtblock_help (void ); +static int rtextent_f (int argc,char **argv); +static void rtextent_help (void );static void print_rawdata (void *data,int len); static const cmdinfo_t ablock_cmd =
@@ -39,6 +43,12 @@ static const cmdinfo_t dblock_cmd =static const cmdinfo_t fsblock_cmd = {"fsblock" ,"fsb" , fsblock_f,0 ,1 ,1 ,N_ ("[fsb]" ),N_ ("set address to fsblock value" ), fsblock_help }; +static const cmdinfo_t rtblock_cmd = + {"rtblock" ,"rtbno" , rtblock_f,0 ,1 ,1 ,N_ ("[rtbno]" ), +N_ ("set address to rtblock value" ), rtblock_help }; +static const cmdinfo_t rtextent_cmd = + {"rtextent" ,"rtx" , rtextent_f,0 ,1 ,1 ,N_ ("[rtxno]" ), +N_ ("set address to rtextent value" ), rtextent_help };static void ablock_help (void )
@@ -104,6 +114,8 @@ block_init(void)add_command (&daddr_cmd);add_command (&dblock_cmd);add_command (&fsblock_cmd); +add_command (&rtblock_cmd); +add_command (&rtextent_cmd); } static void
@@ -302,6 +314,103 @@ fsblock_f(return 0 ; } +static void +rtblock_help (void ) +{ +dbprintf (_ ( +"\n Example:\n" +"\n" +" 'rtblock 1023' - sets the file position to the 1023rd block on the realtime\n" +" volume. The filesystem block size is specified in the superblock and set\n" +" during mkfs time.\n\n" +)); +} + +static int +rtblock_f ( +int argc, +char **argv) +{ +xfs_rtblock_t rtbno; +char *p; + +if (argc ==1 ) { +if (!iocur_is_rtdev (iocur_top)) { +dbprintf (_ ("cursor does not point to rt device\n" )); +return 0 ; + } +dbprintf (_ ("current rtblock is %lld\n" ), +xfs_daddr_to_rtb (mp, iocur_top->off >>BBSHIFT )); +return 0 ; + } + rtbno =strtoull (argv[1 ], &p,0 ); +if (*p !='\0' ) { +dbprintf (_ ("bad rtblock %s\n" ), argv[1 ]); +return 0 ; + } +if (rtbno >= mp->m_sb.sb_rblocks) { +dbprintf (_ ("bad rtblock %s\n" ), argv[1 ]); +return 0 ; + } +ASSERT (typtab[TYP_DATA ].typnm ==TYP_DATA ); +set_rt_cur (&typtab[TYP_DATA ],xfs_rtb_to_daddr (mp, rtbno), blkbb, +DB_RING_ADD ,NULL ); +return 0 ; +} + +static void +rtextent_help (void ) +{ +dbprintf (_ ( +"\n Example:\n" +"\n" +" 'rtextent 10' - sets the file position to the 10th extent on the realtime\n" +" volume. The realtime extent size is specified in the superblock and set\n" +" during mkfs or growfs time.\n\n" +)); +} + +/* +* Move the cursor to a specific location on the realtime block device given +* a linear address in units of realtime extents. +*/ +static int +rtextent_f ( +int argc, +char **argv) +{ +xfs_rtblock_t rtbno; +xfs_rtxnum_t rtx; +char *p; + +if (argc ==1 ) { +if (!iocur_is_rtdev (iocur_top)) { +dbprintf (_ ("cursor does not point to rt device\n" )); +return 0 ; + } + + rtbno =xfs_daddr_to_rtb (mp, iocur_top->off >>BBSHIFT ); +dbprintf (_ ("current rtextent is %lld\n" ), +xfs_rtb_to_rtx (mp, rtbno)); +return 0 ; + } + rtx =strtoull (argv[1 ], &p,0 ); +if (*p !='\0' ) { +dbprintf (_ ("bad rtextent %s\n" ), argv[1 ]); +return 0 ; + } +if (rtx >= mp->m_sb.sb_rextents) { +dbprintf (_ ("bad rtextent %s\n" ), argv[1 ]); +return 0 ; + } + + rtbno =xfs_rtx_to_rtb (mp, rtx); +ASSERT (typtab[TYP_DATA ].typnm ==TYP_DATA ); +set_rt_cur (&typtab[TYP_DATA ],xfs_rtb_to_daddr (mp, rtbno), + mp->m_sb.sb_rextsize * blkbb,DB_RING_ADD ,NULL ); +return 0 ; +} + void print_block( const field_t *fields,
man/man8/xfs_db.8 +22 -0
@@ -1099,6 +1099,28 @@ .SH COMMANDS Exit .BR xfs_db . .TP +.BI "rtblock [" rtbno ] +Set current address to the location on the realtime device given by +.IR rtbno . +This value must be a realtime block number. +If no value for +.I rtbno +is given the current address is printed, expressed as an rtbno. +The type is set to +.B data +(uninterpreted). +.TP +.BI "rtextent [" rtxno ] +Set current address to the location on the realtime device given by +.IR rtextent . +This value must be a linear address in units of realtime extents. +If no value for +.I rtextent +is given the current address is printed, expressed as an rtextent. +The type is set to +.B data +(uninterpreted). +.TP .BI "ring [" index ] Show position ring (if no .I index
PATCH 6/8 xfs_db: enable conversion of rt space units
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Teach the xfs_db convert function about rt extents, rt block numbers, and how to compute offsets within the rt bitmap and summary files.
--- db/convert.c | 253 ++++++++++++++++++++++++++++++++++++++++++++++++----- man/man8/xfs_db.8 | 51 +++++++++++ 2 files changed, 281 insertions(+), 23 deletions(-)
db/convert.c +230 -23
@@ -8,6 +8,7 @@#include "command.h" #include "output.h" #include "init.h" +#include "block.h" #define M (A ) (1 << CT_ ## A)#define agblock_to_bytes (x) \
@@ -26,6 +27,10 @@ agino_to_bytes(XFS_INO_TO_AGINO(mp, (x))))#define inoidx_to_bytes (x) \ ((uint64_t)(x) << mp->m_sb.sb_inodelog) +#define rtblock_to_bytes (x) \ + ((uint64_t)(x) << mp->m_sb.sb_blocklog) +#define rtx_to_rtblock (x) \ + ((uint64_t)(x) * mp->m_sb.sb_rextsize) typedef enum { CT_NONE = -1,
@@ -40,11 +45,12 @@ typedef enum {CT_INO ,/* xfs_ino_t */ CT_INOIDX ,/* index of inode in fsblock */ CT_INOOFF ,/* byte offset in inode */ +CT_RTBLOCK ,/* realtime block */ +CT_RTX ,/* realtime extent */ NCTS } ctype_t;typedef struct ctydesc { -ctype_t ctype;int allowed;const char **names; }ctydesc_t ;
@@ -61,12 +67,16 @@ typedef union {xfs_ino_t ino;int inoidx;int inooff; +xfs_rtblock_t rtblock; +xfs_rtblock_t rtx; } cval_t;static uint64_t bytevalue (ctype_t ctype,cval_t *val); +static int rtconvert_f (int argc,char **argv);static int convert_f (int argc,char **argv);static int getvalue (char *s,ctype_t ctype,cval_t *val); -static ctype_t lookupcty (char *ctyname); +static ctype_t lookupcty (const struct ctydesc *descs, +const char *ctyname);static const char *agblock_names[] = {"agblock" ,"agbno" ,NULL };static const char *agino_names[] = {"agino" ,"aginode" ,NULL };
@@ -74,6 +84,8 @@ static const char *agnumber_names[] = { "agnumber", "agno", NULL };static const char *bboff_names[] = {"bboff" ,"daddroff" ,NULL };static const char *blkoff_names[] = {"blkoff" ,"fsboff" ,"agboff" ,NULL }; +static const char *rtblkoff_names[] = {"blkoff" ,"rtboff" , +NULL };static const char *byte_names[] = {"byte" ,"fsbyte" ,NULL };static const char *daddr_names[] = {"daddr" ,"bb" ,NULL };static const char *fsblock_names[] = {"fsblock" ,"fsb" ,"fsbno" ,NULL };
@@ -81,30 +93,119 @@ static const char *ino_names[] = { "ino", "inode", NULL };static const char *inoidx_names[] = {"inoidx" ,"offset" ,NULL };static const char *inooff_names[] = {"inooff" ,"inodeoff" ,NULL }; +static const char *rtblock_names[] = {"rtblock" ,"rtb" ,"rtbno" ,NULL }; +static const char *rtx_names[] = {"rtx" ,"rtextent" ,NULL }; +static const ctydesc_t ctydescs[NCTS ] = { - {CT_AGBLOCK ,M (AGNUMBER )|M (BBOFF )|M (BLKOFF )|M (INOIDX )|M (INOOFF ), - agblock_names }, - {CT_AGINO ,M (AGNUMBER )|M (INOOFF ), agino_names }, - {CT_AGNUMBER , -M (AGBLOCK )|M (AGINO )|M (BBOFF )|M (BLKOFF )|M (INOIDX )|M (INOOFF ), - agnumber_names }, - {CT_BBOFF ,M (AGBLOCK )|M (AGNUMBER )|M (DADDR )|M (FSBLOCK ), bboff_names }, - {CT_BLKOFF ,M (AGBLOCK )|M (AGNUMBER )|M (FSBLOCK ), blkoff_names }, - {CT_BYTE ,0 , byte_names }, - {CT_DADDR ,M (BBOFF ), daddr_names }, - {CT_FSBLOCK ,M (BBOFF )|M (BLKOFF )|M (INOIDX ), fsblock_names }, - {CT_INO ,M (INOOFF ), ino_names }, - {CT_INOIDX ,M (AGBLOCK )|M (AGNUMBER )|M (FSBLOCK )|M (INOOFF ), - inoidx_names }, - {CT_INOOFF , -M (AGBLOCK )|M (AGINO )|M (AGNUMBER )|M (FSBLOCK )|M (INO )|M (INOIDX ), - inooff_names }, + [CT_AGBLOCK ] = { + .allowed =M (AGNUMBER ) | +M (BBOFF ) | +M (BLKOFF ) | +M (INOIDX ) | +M (INOOFF ), + .names = agblock_names, + }, + [CT_AGINO ] = { + .allowed =M (AGNUMBER ) | +M (INOOFF ), + .names = agino_names, + }, + [CT_AGNUMBER ] = { + .allowed =M (AGBLOCK ) | +M (AGINO ) | +M (BBOFF ) | +M (BLKOFF ) | +M (INOIDX ) | +M (INOOFF ), + .names = agnumber_names, + }, + [CT_BBOFF ] = { + .allowed =M (AGBLOCK ) | +M (AGNUMBER ) | +M (DADDR ) | +M (FSBLOCK ), + .names = bboff_names, + }, + [CT_BLKOFF ] = { + .allowed =M (AGBLOCK ) | +M (AGNUMBER ) | +M (FSBLOCK ), + .names = blkoff_names, + }, + [CT_BYTE ] = { + .allowed =0 , + .names = byte_names, + }, + [CT_DADDR ] = { + .allowed =M (BBOFF ), + .names = daddr_names, + }, + [CT_FSBLOCK ] = { + .allowed =M (BBOFF ) | +M (BLKOFF ) | +M (INOIDX ), + .names = fsblock_names, + }, + [CT_INO ] = { + .allowed =M (INOOFF ), + .names = ino_names, + }, + [CT_INOIDX ] = { + .allowed =M (AGBLOCK ) | +M (AGNUMBER ) | +M (FSBLOCK ) | +M (INOOFF ), + .names = inoidx_names, + }, + [CT_INOOFF ] = { + .allowed =M (AGBLOCK ) | +M (AGINO ) | +M (AGNUMBER ) | +M (FSBLOCK ) | +M (INO ) | +M (INOIDX ), + .names = inooff_names, + }, +}; + +static const ctydesc_t ctydescs_rt[NCTS ] = { + [CT_BBOFF ] = { + .allowed =M (DADDR ) | +M (RTBLOCK ), + .names = bboff_names, + }, + [CT_BLKOFF ] = { + .allowed =M (RTBLOCK ), + .names = rtblkoff_names, + }, + [CT_BYTE ] = { + .allowed =0 , + .names = byte_names, + }, + [CT_DADDR ] = { + .allowed =M (BBOFF ), + .names = daddr_names, + }, + [CT_RTBLOCK ] = { + .allowed =M (BBOFF ) | +M (BLKOFF ), + .names = rtblock_names, + }, + [CT_RTX ] = { + .allowed =M (BBOFF ) | +M (BLKOFF ), + .names = rtx_names, + }, };static const cmdinfo_t convert_cmd = {"convert" ,NULL , convert_f,3 ,9 ,0 ,"type num [type num]... type" ,"convert from one address form to another" ,NULL }; +static const cmdinfo_t rtconvert_cmd = + {"rtconvert" ,NULL , rtconvert_f,3 ,9 ,0 ,"type num [type num]... type" , +"convert from one realtime address form to another" ,NULL }; +static uint64_t bytevalue (ctype_t ctype,cval_t *val) {
@@ -131,6 +232,10 @@ bytevalue(ctype_t ctype, cval_t *val)return inoidx_to_bytes (val->inoidx);case CT_INOOFF :return (uint64_t )val->inooff; +case CT_RTBLOCK : +return rtblock_to_bytes (val->rtblock); +case CT_RTX : +return rtblock_to_bytes (rtx_to_rtblock (val->rtx));case CT_NONE :case NCTS :break ;
@@ -159,13 +264,13 @@ convert_f(int argc, char **argv) "arguments\n"), argc);return 0 ; } -if ((wtype =lookupcty (argv[argc -1 ])) ==CT_NONE ) { +if ((wtype =lookupcty (ctydescs, argv[argc -1 ])) ==CT_NONE ) {dbprintf (_ ("unknown conversion type %s\n" ), argv[argc -1 ]);return 0 ; }for (i = mask = conmask =0 ; i < (argc -1 ) /2 ; i++) { - c =lookupcty (argv[i *2 ]); + c =lookupcty (ctydescs, argv[i *2 ]);if (c ==CT_NONE ) {dbprintf (_ ("unknown conversion type %s\n" ), argv[i *2 ]);return 0 ;
@@ -230,6 +335,99 @@ convert_f(int argc, char **argv)case CT_INOOFF : v &= mp->m_sb.sb_inodesize -1 ;break ; +case CT_RTBLOCK : +case CT_RTX : +/* shouldn't get here */ +ASSERT (0 ); +break ; +case CT_NONE : +case NCTS : +/* NOTREACHED */ +break ; + } +dbprintf ("0x%llx (%llu)\n" , v, v); +return 0 ; +} + +static int +rtconvert_f (int argc,char **argv) +{ +ctype_t c; +int conmask; +cval_t cvals[NCTS ] = {}; +int i; +int mask; +uint64_t v; +ctype_t wtype; + +/* move past the "rtconvert" command */ + argc--; + argv++; + +if ((argc %2 ) !=1 ) { +dbprintf (_ ("bad argument count %d to rtconvert, expected 3,5,7,9 " +"arguments\n" ), argc); +return 0 ; + } +if ((wtype =lookupcty (ctydescs_rt, argv[argc -1 ])) ==CT_NONE ) { +dbprintf (_ ("unknown conversion type %s\n" ), argv[argc -1 ]); +return 0 ; + } + +for (i = mask = conmask =0 ; i < (argc -1 ) /2 ; i++) { + c =lookupcty (ctydescs_rt, argv[i *2 ]); +if (c ==CT_NONE ) { +dbprintf (_ ("unknown conversion type %s\n" ), argv[i *2 ]); +return 0 ; + } +if (c == wtype) { +dbprintf (_ ("result type same as argument\n" )); +return 0 ; + } +if (conmask & (1 << c)) { +dbprintf (_ ("conflicting conversion type %s\n" ), + argv[i *2 ]); +return 0 ; + } +if (!getvalue (argv[i *2 +1 ], c, &cvals[c])) +return 0 ; + mask |=1 << c; + conmask |= ~ctydescs_rt[c].allowed; + } + v =0 ; +for (c = (ctype_t )0 ; c <NCTS ; c++) { +if (!(mask & (1 << c))) +continue ; + v +=bytevalue (c, &cvals[c]); + } +switch (wtype) { +case CT_BBOFF : + v &=BBMASK ; +break ; +case CT_BLKOFF : + v &= mp->m_blockmask; +break ; +case CT_BYTE : +break ; +case CT_DADDR : + v >>=BBSHIFT ; +break ; +case CT_RTBLOCK : + v =xfs_daddr_to_rtb (mp, v >>BBSHIFT ); +break ; +case CT_RTX : + v =xfs_daddr_to_rtb (mp, v >>BBSHIFT ) / mp->m_sb.sb_rextsize; +break ; +case CT_AGBLOCK : +case CT_AGINO : +case CT_AGNUMBER : +case CT_FSBLOCK : +case CT_INO : +case CT_INOIDX : +case CT_INOOFF : +/* shouldn't get here */ +ASSERT (0 ); +break ;case CT_NONE :case NCTS :/* NOTREACHED */
@@ -243,6 +441,7 @@ voidconvert_init (void) {add_command (&convert_cmd); +add_command (&rtconvert_cmd); } static int
@@ -290,6 +489,12 @@ getvalue(char *s, ctype_t ctype, cval_t *val)case CT_INOOFF : val->inooff = (int )v;break ; +case CT_RTBLOCK : + val->rtblock = (xfs_rtblock_t )v; +break ; +case CT_RTX : + val->rtx = (xfs_rtblock_t )v; +break ;case CT_NONE :case NCTS :/* NOTREACHED */
@@ -299,13 +504,15 @@ getvalue(char *s, ctype_t ctype, cval_t *val) }static ctype_t -lookupcty (char *ctyname) +lookupcty ( +const struct ctydesc *descs, +const char *ctyname) {ctype_t cty;const char **name;for (cty = (ctype_t )0 ; cty <NCTS ; cty++) { -for (name = ctydescs[cty].names; *name; name++) { +for (name = descs[cty].names; name && *name; name++) {if (strcmp (ctyname, *name) ==0 )return cty; }
man/man8/xfs_db.8 +51 -0
57 lines not on this page · raw message
PATCH 7/8 xfs_db: convert rtbitmap geometry
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Teach the rtconvert command to be able to convert realtime blocks and extents to locations within the rt bitmap.
--- db/convert.c | 40 ++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_db.8 | 10 ++++++++++ 2 files changed, 50 insertions(+)
db/convert.c +40 -0
@@ -31,6 +31,10 @@ ((uint64_t)(x) << mp->m_sb.sb_blocklog)#define rtx_to_rtblock (x) \ ((uint64_t)(x) * mp->m_sb.sb_rextsize) +#define rbmblock_to_bytes (x) \ + rtblock_to_bytes(rtx_to_rtblock(xfs_rbmblock_to_rtx(mp, (uint64_t)x))) +#define rbmword_to_bytes (x) \ + rtblock_to_bytes(rtx_to_rtblock((uint64_t)(x) << XFS_NBWORDLOG)) typedef enum { CT_NONE = -1,
@@ -47,6 +51,8 @@ typedef enum {CT_INOOFF ,/* byte offset in inode */ CT_RTBLOCK ,/* realtime block */ CT_RTX ,/* realtime extent */ +CT_RBMBLOCK ,/* block within rt bitmap */ +CT_RBMWORD ,/* word within rt bitmap */ NCTS } ctype_t;
@@ -69,6 +75,8 @@ typedef union {int inooff;xfs_rtblock_t rtblock;xfs_rtblock_t rtx; +xfs_fileoff_t rbmblock; +unsigned int rbmword; } cval_t;static uint64_t bytevalue (ctype_t ctype,cval_t *val);
@@ -95,6 +103,8 @@ static const char *inooff_names[] = { "inooff", "inodeoff", NULL };static const char *rtblock_names[] = {"rtblock" ,"rtb" ,"rtbno" ,NULL };static const char *rtx_names[] = {"rtx" ,"rtextent" ,NULL }; +static const char *rbmblock_names[] = {"rbmblock" ,"rbmb" ,NULL }; +static const char *rbmword_names[] = {"rbmword" ,"rbmw" ,NULL };static const ctydesc_t ctydescs[NCTS ] = { [CT_AGBLOCK ] = {
@@ -196,6 +206,14 @@ static const ctydesc_t ctydescs_rt[NCTS] = {M (BLKOFF ), .names = rtx_names, }, + [CT_RBMBLOCK] = { + .allowed =M (RBMWORD ), + .names = rbmblock_names, + }, + [CT_RBMWORD ] = { + .allowed =M (RBMBLOCK ), + .names = rbmword_names, + }, }; static const cmdinfo_t convert_cmd =
@@ -236,6 +254,10 @@ bytevalue(ctype_t ctype, cval_t *val)return rtblock_to_bytes (val->rtblock);case CT_RTX :return rtblock_to_bytes (rtx_to_rtblock (val->rtx)); +case CT_RBMBLOCK : +return rbmblock_to_bytes (val->rbmblock); +case CT_RBMWORD : +return rbmword_to_bytes (val->rbmword);case CT_NONE :case NCTS :break ;
@@ -337,6 +359,8 @@ convert_f(int argc, char **argv)break ;case CT_RTBLOCK :case CT_RTX : +case CT_RBMBLOCK : +case CT_RBMWORD :/* shouldn't get here */ ASSERT (0 );break ;
@@ -418,6 +442,16 @@ rtconvert_f(int argc, char **argv)case CT_RTX : v =xfs_daddr_to_rtb (mp, v >>BBSHIFT ) / mp->m_sb.sb_rextsize;break ; +case CT_RBMBLOCK : + v =xfs_rtx_to_rbmblock (mp, +xfs_rtb_to_rtx (mp, +xfs_daddr_to_rtb (mp, v >>BBSHIFT ))); +break ; +case CT_RBMWORD : + v =xfs_rtx_to_rbmword (mp, +xfs_rtb_to_rtx (mp, +xfs_daddr_to_rtb (mp, v >>BBSHIFT ))); +break ;case CT_AGBLOCK :case CT_AGINO :case CT_AGNUMBER :
@@ -495,6 +529,12 @@ getvalue(char *s, ctype_t ctype, cval_t *val)case CT_RTX : val->rtx = (xfs_rtblock_t )v;break ; +case CT_RBMBLOCK : + val->rbmblock = (xfs_fileoff_t )v; +break ; +case CT_RBMWORD : + val->rbmword = (unsigned int )v; +break ;case CT_NONE :case NCTS :/* NOTREACHED */
man/man8/xfs_db.8 +10 -0
@@ -1173,6 +1173,16 @@ .SH COMMANDS or .B rtextent (realtime extent) +.HP +.B rbmblock +or +.B rbmb +(realtime bitmap block) +.HP +.B rbmword +or +.B rbmw +(32-bit word within a realtime bitmap block) .PD .RE .IP
PATCH 8/8 xfs_db: convert rtsummary geometry
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Teach the rtconvert command to be able to convert realtime blocks and extents to locations within the rt summary.
--- db/convert.c | 161 ++++++++++++++++++++++++++++++++++++++++++++++++++--- man/man8/xfs_db.8 | 29 ++++++++++ 2 files changed, 182 insertions(+), 8 deletions(-)
db/convert.c +153 -8
@@ -53,6 +53,9 @@ typedef enum {CT_RTX ,/* realtime extent */ CT_RBMBLOCK ,/* block within rt bitmap */ CT_RBMWORD ,/* word within rt bitmap */ +CT_RSUMBLOCK ,/* block within rt summary */ +CT_RSUMLOG ,/* log level for rtsummary computations */ +CT_RSUMINFO ,/* info word within rt summary */ NCTS } ctype_t;
@@ -77,6 +80,7 @@ typedef union {xfs_rtblock_t rtx;xfs_fileoff_t rbmblock;unsigned int rbmword; +xfs_fileoff_t rsumblock; } cval_t;static uint64_t bytevalue (ctype_t ctype,cval_t *val);
@@ -105,6 +109,12 @@ static const char *rtblock_names[] = { "rtblock", "rtb", "rtbno", NULL };static const char *rtx_names[] = {"rtx" ,"rtextent" ,NULL };static const char *rbmblock_names[] = {"rbmblock" ,"rbmb" ,NULL };static const char *rbmword_names[] = {"rbmword" ,"rbmw" ,NULL }; +static const char *rsumblock_names[] = {"rsumblock" ,"rsmb" ,NULL }; +static const char *rsumlog_names[] = {"rsumlog" ,"rsml" ,NULL }; +static const char *rsumword_names[] = {"rsuminfo" ,"rsmi" ,NULL }; + +static int rsuminfo; +static int rsumlog;static const ctydesc_t ctydescs[NCTS ] = { [CT_AGBLOCK ] = {
@@ -181,39 +191,60 @@ static const ctydesc_t ctydescs[NCTS] = {static const ctydesc_t ctydescs_rt[NCTS ] = { [CT_BBOFF ] = { .allowed =M (DADDR ) | -M (RTBLOCK ), +M (RTBLOCK ) | +M (RSUMLOG ), .names = bboff_names, }, [CT_BLKOFF ] = { - .allowed =M (RTBLOCK ), + .allowed =M (RTBLOCK ) | +M (RSUMLOG ), .names = rtblkoff_names, }, [CT_BYTE ] = { - .allowed =0 , + .allowed =M (RSUMLOG ), .names = byte_names, }, [CT_DADDR ] = { - .allowed =M (BBOFF ), + .allowed =M (BBOFF ) | +M (RSUMLOG ), .names = daddr_names, }, [CT_RTBLOCK ] = { .allowed =M (BBOFF ) | -M (BLKOFF ), +M (BLKOFF ) | +M (RSUMLOG ), .names = rtblock_names, }, [CT_RTX ] = { .allowed =M (BBOFF ) | -M (BLKOFF ), +M (BLKOFF ) | +M (RSUMLOG ), .names = rtx_names, }, [CT_RBMBLOCK ] = { - .allowed =M (RBMWORD ), + .allowed =M (RBMWORD ) | +M (RSUMLOG ), .names = rbmblock_names, }, [CT_RBMWORD ] = { - .allowed =M (RBMBLOCK ), + .allowed =M (RBMBLOCK ) | +M (RSUMLOG ), .names = rbmword_names, }, +/* must be specified in order rsumlog -> rsuminfo -> rsumblock */ + [CT_RSUMBLOCK ] = { + .allowed =0 , + .names = rsumblock_names, + }, + [CT_RSUMLOG ] = { + .allowed =M (RSUMINFO ) | +M (RSUMBLOCK ), + .names = rsumlog_names, + }, + [CT_RSUMINFO ] = { + .allowed =M (RSUMBLOCK ), + .names = rsumword_names, + }, }; static const cmdinfo_t convert_cmd =
@@ -224,6 +255,39 @@ static const cmdinfo_t rtconvert_cmd = {"rtconvert" ,NULL , rtconvert_f,3 ,9 ,0 ,"type num [type num]... type" , "convert from one realtime address form to another", NULL }; +static inline uint64_t +rsumblock_to_bytes ( +xfs_fileoff_t rsumblock) +{ +/* +* We compute the rt summary file block with this formula: +* sumoffs = (log2len * sb_rbmblocks) + rbmblock; +* sumblock = sumoffs / blockwsize; +* +* Hence the return value is the inverse of this: +* sumoffs = (rsumblock * blockwsize) + rsuminfo; +* rbmblock = sumoffs % (log2len * sb_rbmblocks); +*/ +xfs_rtsumoff_t sumoff; +xfs_fileoff_t rbmblock; + +if (rsumlog <0 ) { +dbprintf (_ ("need to set rsumlog\n" )); +return 0 ; + } +if (rsuminfo <0 ) { +dbprintf (_ ("need to set rsuminfo\n" )); +return 0 ; + } + + sumoff = rsuminfo + (rsumblock * mp->m_blockwsize); +if (rsumlog) + rbmblock = sumoff % (rsumlog * mp->m_sb.sb_rbmblocks); +else + rbmblock = sumoff; +return rbmblock_to_bytes (rbmblock); +} +static uint64_t bytevalue (ctype_t ctype,cval_t *val) {
@@ -258,6 +322,16 @@ bytevalue(ctype_t ctype, cval_t *val)return rbmblock_to_bytes (val->rbmblock);case CT_RBMWORD :return rbmword_to_bytes (val->rbmword); +case CT_RSUMBLOCK : +return rsumblock_to_bytes (val->rbmblock); +case CT_RSUMLOG : +case CT_RSUMINFO : +/* +* These have to specified before rsumblock, and are stored in +* global variables. Hence they do not adjust the disk address +* value. +*/ +return 0 ;case CT_NONE :case NCTS :break ;
@@ -361,6 +435,9 @@ convert_f(int argc, char **argv)case CT_RTX :case CT_RBMBLOCK :case CT_RBMWORD : +case CT_RSUMBLOCK : +case CT_RSUMLOG : +case CT_RSUMINFO :/* shouldn't get here */ ASSERT (0 );break ;
@@ -373,6 +450,52 @@ convert_f(int argc, char **argv)return 0 ; } +static inline uint64_t +rt_daddr_to_rsumblock ( +struct xfs_mount *mp, +uint64_t input) +{ +xfs_rtblock_t rtbno; +xfs_rtxnum_t rtx; +xfs_fileoff_t rbmblock; +xfs_rtsumoff_t rsumoff; + +if (rsumlog <0 ) { +dbprintf (_ ("need to set rsumlog\n" )); +return 0 ; + } + + rtbno =xfs_daddr_to_rtb (mp, input >>BBSHIFT ); + rtx =xfs_rtb_to_rtx (mp, rtbno); + rbmblock =xfs_rtx_to_rbmblock (mp, rtx); + rsumoff =xfs_rtsumoffs (mp, rsumlog, rbmblock); + +return xfs_rtsumoffs_to_block (mp, rsumoff); +} + +static inline uint64_t +rt_daddr_to_rsuminfo ( +struct xfs_mount *mp, +uint64_t input) +{ +xfs_rtblock_t rtbno; +xfs_rtxnum_t rtx; +xfs_fileoff_t rbmblock; +xfs_rtsumoff_t rsumoff; + +if (rsumlog <0 ) { +dbprintf (_ ("need to set rsumlog\n" )); +return 0 ; + } + + rtbno =xfs_daddr_to_rtb (mp, input >>BBSHIFT ); + rtx =xfs_rtb_to_rtx (mp, rtbno); + rbmblock =xfs_rtx_to_rbmblock (mp, rtx); + rsumoff =xfs_rtsumoffs (mp, rsumlog, rbmblock); + +return xfs_rtsumoffs_to_infoword (mp, rsumoff); +} +static int rtconvert_f (int argc,char **argv) {
@@ -384,6 +507,9 @@ rtconvert_f(int argc, char **argv)uint64_t v;ctype_t wtype; + rsumlog =-1 ; + rsuminfo =-1 ; +/* move past the "rtconvert" command */ argc--; argv++;
@@ -452,6 +578,16 @@ rtconvert_f(int argc, char **argv)xfs_rtb_to_rtx (mp, xfs_daddr_to_rtb(mp, v >> BBSHIFT)));break ; +case CT_RSUMBLOCK : + v =rt_daddr_to_rsumblock (mp, v); +break ; +case CT_RSUMLOG : +dbprintf (_ ("cannot convert to rsumlog\n" )); +return 0 ; +break ; +case CT_RSUMINFO : + v =rt_daddr_to_rsuminfo (mp, v); +break ;case CT_AGBLOCK :case CT_AGINO :case CT_AGNUMBER :
@@ -535,6 +671,15 @@ getvalue(char *s, ctype_t ctype, cval_t *val)case CT_RBMWORD : val->rbmword = (unsigned int )v;break ; +case CT_RSUMBLOCK : + val->rsumblock = (xfs_fileoff_t )v; +break ; +case CT_RSUMLOG : + rsumlog = (unsigned int )v; +break ; +case CT_RSUMINFO : + rsuminfo = (unsigned int )v; +break ;case CT_NONE :case NCTS :/* NOTREACHED */
man/man8/xfs_db.8 +29 -0
@@ -1183,10 +1183,39 @@ .SH COMMANDS or .B rbmw (32-bit word within a realtime bitmap block) +.HP +.B rsumblock +or +.B rsmb +(realtime summary file block) +.HP +.B rsuminfo +or +.B rsmi +(32-bit counter within a realtime summary block) +.HP +.B rsumlog +or +.B rsml +(log2len parameter used for summary file offset computations) .PD .RE .IP Only conversions that "make sense" are allowed. + +Realtime summary file location conversions have the following rules: +Each info word in the rt summary file counts the number of free extents of a +given log2(length) that start in a given rt bitmap block. + +To compute summary file location information for a given rt bitmap block, a +log2(extent length) must be specified as the last type/number pair before the +conversion type, and the type must be +.BR rsumlog . + +To compute the rt bitmap block from summary file location, the type/number pairs +must be specified exactly in the order +.BR rsumlog ", " rsuminfo ", " rsumblock . + .TP .BI "sb [" agno ] Set current address to SB header in allocation group
PATCH 4/7 xfs_metadump: support external devices
In reply to Darrick J. WongHi all,
This series augments the xfs_metadump and xfs_mdrestore utilities to capture the contents of an external log in a metadump, and restore it on the other end. This will enable better debugging analysis of broken filesystems, since it will now be possible to capture external log data. This is a prequisite for the rt groups feature, since we'll also need to capture the rt superblocks written to the rt device.
This also means we can capture the contents of external logs for better analysis by support staff.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=metadump-external-devices-6.12
---
Commits in this patchset:
* xfs_db: allow setting current address to log blocks
--- db/block.c | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++- man/man8/xfs_db.8 | 17 +++++++++ 2 files changed, 119 insertions(+), 1 deletion(-)
PATCH 1/1 xfs_db: allow setting current address to log blocks
From: Darrick J. Wong <[email protected]>
Add commands so that users can target blocks on an external log device.
--- db/block.c | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++- man/man8/xfs_db.8 | 17 +++++++++ 2 files changed, 119 insertions(+), 1 deletion(-)
db/block.c +102 -1
@@ -29,6 +29,8 @@ static int rtblock_f(int argc, char **argv);static void rtblock_help (void );static int rtextent_f (int argc,char **argv);static void rtextent_help (void ); +static int logblock_f (int argc,char **argv); +static void logblock_help (void );static void print_rawdata (void *data,int len); static const cmdinfo_t ablock_cmd =
@@ -49,6 +51,9 @@ static const cmdinfo_t rtblock_cmd =static const cmdinfo_t rtextent_cmd = {"rtextent" ,"rtx" , rtextent_f,0 ,1 ,1 ,N_ ("[rtxno]" ),N_ ("set address to rtextent value" ), rtextent_help }; +static const cmdinfo_t logblock_cmd = + {"logblock" ,"lsb" , logblock_f,0 ,1 ,1 ,N_ ("[logbno]" ), +N_ ("set address to logblock value" ), logblock_help };static void ablock_help (void )
@@ -116,6 +121,7 @@ block_init(void)add_command (&fsblock_cmd);add_command (&rtblock_cmd);add_command (&rtextent_cmd); +add_command (&logblock_cmd); } static void
@@ -132,6 +138,7 @@ daddr_help(void)enum daddr_target {DT_DATA ,DT_RT , +DT_LOG , }; static int
@@ -146,18 +153,27 @@ daddr_f(int c;enum daddr_target tgt =DT_DATA ; -while ((c =getopt (argc, argv,"r" )) !=-1 ) { +while ((c =getopt (argc, argv,"rl" )) !=-1 ) {switch (c) {case 'r' : tgt =DT_RT ; max_daddrs = mp->m_sb.sb_rblocks;break ; +case 'l' : + tgt =DT_LOG ; + max_daddrs = mp->m_sb.sb_logblocks; +break ;default :daddr_help ();return 0 ; } } +if (tgt ==DT_LOG && mp->m_sb.sb_logstart >0 ) { +dbprintf (_ ("filesystem has internal log\n" )); +return 0 ; + } +if (optind == argc) {xfs_daddr_t daddr = iocur_top->off >>BBSHIFT ;
@@ -192,6 +208,9 @@ daddr_f(case DT_RT :set_rt_cur (&typtab[TYP_DATA ], d, bb_count,DB_RING_ADD ,NULL );break ; +case DT_LOG : +set_log_cur (&typtab[TYP_DATA ], d, bb_count,DB_RING_ADD ,NULL ); +break ; }return 0 ; }
@@ -411,6 +430,88 @@ rtextent_f(return 0 ; } +static void +logblock_help (void ) +{ +dbprintf (_ ( +"\n Example:\n" +"\n" +" 'logblock 1023' - sets the file position to the 1023rd log block.\n" +" The external log device or the block offset within the internal log will be\n" +" chosen as appropriate.\n" +)); +} + +static int +logblock_f ( +int argc, +char **argv) +{ +xfs_fsblock_t logblock; +char *p; + +if (argc ==1 ) { +if (mp->m_sb.sb_logstart >0 &&iocur_is_ddev (iocur_top)) { + logblock =XFS_DADDR_TO_FSB (mp, + iocur_top->off >>BBSHIFT ); + +if (logblock < mp->m_sb.sb_logstart || + logblock >= mp->m_sb.sb_logstart + + mp->m_sb.sb_logblocks) { +dbprintf ( +_ ("current address not within internal log\n" )); +return 0 ; + } + +dbprintf (_ ("current logblock is %lld\n" ), + logblock - mp->m_sb.sb_logstart); +return 0 ; + } + +if (mp->m_sb.sb_logstart ==0 && +iocur_is_extlogdev (iocur_top)) { + logblock =XFS_BB_TO_FSB (mp, + iocur_top->off >>BBSHIFT ); + +if (logblock >= mp->m_sb.sb_logblocks) { +dbprintf ( +_ ("current address not within external log\n" )); +return 0 ; + } + +dbprintf (_ ("current logblock is %lld\n" ), logblock); +return 0 ; + } + +dbprintf (_ ("current address does not point to log\n" )); +return 0 ; + } + + logblock =strtoull (argv[1 ], &p,0 ); +if (*p !='\0' ) { +dbprintf (_ ("bad logblock %s\n" ), argv[1 ]); +return 0 ; + } + +if (logblock >= mp->m_sb.sb_logblocks) { +dbprintf (_ ("bad logblock %s\n" ), argv[1 ]); +return 0 ; + } + +ASSERT (typtab[TYP_DATA ].typnm ==TYP_DATA ); + +if (mp->m_sb.sb_logstart) { + logblock += mp->m_sb.sb_logstart; +set_cur (&typtab[TYP_DATA ],XFS_FSB_TO_DADDR (mp, logblock), + blkbb,DB_RING_ADD ,NULL ); + }else { +set_log_cur (&typtab[TYP_DATA ],XFS_FSB_TO_BB (mp, logblock), + blkbb,DB_RING_ADD ,NULL ); + } + +return 0 ; +} + void print_block( const field_t *fields,
man/man8/xfs_db.8 +17 -0
@@ -681,6 +681,9 @@ .SH COMMANDS .B data (uninterpreted). +If an address and the +.B \-l +option are specified, the current address is set to the external log device. If an address and the .B \-r option are specified, the current address is set to the realtime device.
@@ -1009,6 +1012,20 @@ .SH COMMANDS .IR filename , stop logging, or print the current logging status. .TP +.BI "logblock [" logbno ] +Set current address to the log block value given by +.IR logbno . +If no value for +.I logbno +is given the current address is printed, expressed as an fsb. +The type is set to +.B data +(uninterpreted). +If the filesystem has an external log, then the address will be within the log +device. +If the filesystem has an internal log, then the address will be within the +internal log. +.TP .BI "logformat [\-c " cycle "] [\-s " sunit "]" Reformats the log to the specified log cycle and log stripe unit. This has the effect of clearing the log destructively.
PATCH 5/7 mkfs/repair: use new rtbitmap helpers
In reply to Darrick J. WongHi all,
Use the new rtfile helpers to create the rt bitmap and summary files instead of duplicating the logic that the in-kernel growfs already had.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=misc-use-rtbitmap-helpers-6.12
---
Commits in this patchset:
* xfs_repair: checking rt free space metadata must happen during phase 4
* xfs_repair: use xfs_validate_rt_geometry
* mkfs: remove a pointless rtfreesp_init forward declaration
* mkfs: use xfs_rtfile_initialize_blocks
* xfs_repair: use libxfs_rtfile_initialize_blocks
* xfs_repair: stop preallocating blocks in mk_rbmino and mk_rsumino
--- libxfs/libxfs_api_defs.h | 2 mkfs/proto.c | 107 +++-------------- repair/phase4.c | 7 + repair/phase5.c | 6 - repair/phase6.c | 284 +++++++--------------------------------------- repair/sb.c | 40 ------ repair/xfs_repair.c | 3 7 files changed, 73 insertions(+), 376 deletions(-)
PATCH 1/6 xfs_repair: checking rt free space metadata must happen during phase 4
From: Darrick J. Wong <[email protected]>
Back in the really old days, xfs_repair would generate the new free space information for the realtime section during phase 5, and write the contents to the rtbitmap and summary files during phase 6. This was ok because the incore information isn't used until phase 6.
Then I changed the behavior to check the generated information against what was on disk and complain about the discrepancies. Unfortunately, there was a subtle flaw here -- for a non -n run, we'll have regenerated the AG metadata before we actually check the rt free space information. If the AG btree regeneration should clobber one of the old rtbitmap or summary blocks, this will be reported as a corruption even though nothing's wrong.
Move check_rtmetadata to the end of phase 4 so that this doesn't happen.
--- repair/phase4.c | 7 +++++++ repair/phase5.c | 6 ------ repair/xfs_repair.c | 3 --- 3 files changed, 7 insertions(+), 9 deletions(-)
repair/phase4.c +7 -0
@@ -401,4 +401,11 @@ phase4(xfs_mount_t *mp)*/ quotino_check (mp);quota_sb_check (mp); + +/* Check the rt metadata before we rebuild */ +if (mp->m_sb.sb_rblocks) { +do_log ( +_ (" - generate realtime summary info and bitmap...\n" )); +check_rtmetadata (mp); + } }
repair/phase5.c +0 -6
@@ -694,12 +694,6 @@ phase5(xfs_mount_t *mp)free (sb_ifree_ag);free (sb_fdblocks_ag); -if (mp->m_sb.sb_rblocks) { -do_log ( -_ (" - generate realtime summary info and bitmap...\n" )); -check_rtmetadata (mp); - } -do_log (_ (" - reset superblock...\n" ));/*
repair/xfs_repair.c +0 -3
@@ -1318,9 +1318,6 @@ main(int argc, char **argv)if (no_modify) {printf (_ ("No modify flag set, skipping phase 5\n" )); - -if (mp->m_sb.sb_rblocks >0 ) -check_rtmetadata (mp); }else {phase5 (mp); }
PATCH 2/6 xfs_repair: use xfs_validate_rt_geometry
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Use shared libxfs code with the kernel instead of reimplementing it.
--- libxfs/libxfs_api_defs.h | 1 + repair/sb.c | 40 ++-------------------------------------- 2 files changed, 3 insertions(+), 38 deletions(-)
libxfs/libxfs_api_defs.h +1 -0
@@ -309,6 +309,7 @@#define xfs_update_secondary_sbs libxfs_update_secondary_sbs +#define xfs_validate_rt_geometry libxfs_validate_rt_geometry#define xfs_validate_stripe_geometry libxfs_validate_stripe_geometry#define xfs_verify_agbno libxfs_verify_agbno#define xfs_verify_agbext libxfs_verify_agbext
repair/sb.c +2 -38
@@ -447,44 +447,8 @@ verify_sb(char *sb_buf, xfs_sb_t *sb, int is_primary_sb)return (XR_BAD_SECT_SIZE_DATA ); } -/* -* real-time extent size is always set -*/ -if (sb->sb_rextsize * sb->sb_blocksize >XFS_MAX_RTEXTSIZE ) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_rextsize * sb->sb_blocksize <XFS_MIN_RTEXTSIZE ) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_rblocks ==0 ) { -if (sb->sb_rextents !=0 ) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_rbmblocks !=0 ) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_rextslog !=0 ) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_frextents !=0 ) -return (XR_BAD_RT_GEO_DATA ); - }else { -/* -* if we have a real-time partition, sanity-check geometry -*/ -if (sb->sb_rblocks / sb->sb_rextsize != sb->sb_rextents) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_rextents ==0 ) -return XR_BAD_RT_GEO_DATA ; - -if (sb->sb_rextslog !=libxfs_compute_rextslog (sb->sb_rextents)) -return (XR_BAD_RT_GEO_DATA ); - -if (sb->sb_rbmblocks != (xfs_extlen_t )howmany (sb->sb_rextents, -NBBY * sb->sb_blocksize)) -return (XR_BAD_RT_GEO_DATA ); - } +if (!libxfs_validate_rt_geometry (sb)) +return XR_BAD_RT_GEO_DATA ;/* * verify correctness of inode alignment if it's there
PATCH 3/6 mkfs: remove a pointless rtfreesp_init forward declaration
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
--- mkfs/proto.c | 1 - 1 file changed, 1 deletion(-)
mkfs/proto.c +0 -1
@@ -18,7 +18,6 @@ static struct xfs_trans * getres(struct xfs_mount *mp, uint blocks);static void rsvfile (xfs_mount_t *mp,xfs_inode_t *ip,long long len);static char *newregfile (char **pp,int *len);static void rtinit (xfs_mount_t *mp); -static void rtfreesp_init (struct xfs_mount *mp);static long filesize (int fd);static int slashes_are_spaces;
PATCH 4/6 mkfs: use xfs_rtfile_initialize_blocks
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Use the new libxfs helper for initializing the rtbitmap/summary files for rtgroup-enabled file systems. Also skip the zeroing of the blocks for rtgroup file systems as we'll overwrite every block instantly.
--- libxfs/libxfs_api_defs.h | 1 mkfs/proto.c | 106 +++++++--------------------------------------- 2 files changed, 17 insertions(+), 90 deletions(-)
libxfs/libxfs_api_defs.h +1 -0
@@ -263,6 +263,7 @@#define xfs_suminfo_add libxfs_suminfo_add#define xfs_suminfo_get libxfs_suminfo_get#define xfs_rtsummary_wordcount libxfs_rtsummary_wordcount +#define xfs_rtfile_initialize_blocks libxfs_rtfile_initialize_blocks#define xfs_rtfree_extent libxfs_rtfree_extent#define xfs_rtfree_blocks libxfs_rtfree_blocks
mkfs/proto.c +16 -90
124 lines not on this page · raw message
PATCH 5/6 xfs_repair: use libxfs_rtfile_initialize_blocks
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Use libxfs_rtfile_initialize_blocks to write the re-computed rtbitmap and rtsummary contents. This removes duplicate code and prepares for even more sharing once the rtgroup features adds a metadata header to the rtbitmap and rtsummary blocks.
--- repair/phase6.c | 168 ++++++++++--------------------------------------------- 1 file changed, 32 insertions(+), 136 deletions(-)
repair/phase6.c +32 -136
@@ -547,157 +547,60 @@ mk_rbmino(xfs_mount_t *mp)libxfs_irele (ip); } -static int -fill_rbmino (xfs_mount_t *mp) +static void +fill_rbmino ( +struct xfs_mount *mp) { -struct xfs_buf *bp; -xfs_trans_t *tp; -xfs_inode_t *ip; -union xfs_rtword_raw *bmp; -int nmap; -int error; -xfs_fileoff_t bno; -xfs_bmbt_irec_t map; - - bmp = btmcompute; - bno =0 ; +struct xfs_trans *tp; +struct xfs_inode *ip; +int error; error = -libxfs_trans_alloc_rollable (mp,10 , &tp);if (error)res_failed (error); error = -libxfs_iget (mp, tp, mp->m_sb.sb_rbmino,0 , &ip); -if (error) { +libxfs_trans_cancel (tp); +if (error)do_error ( -_ ("couldn't iget realtime bitmap inode -- error - %d\n" ), - error); - } - -while (bno < mp->m_sb.sb_rbmblocks) { -struct xfs_rtalloc_args args = { - .mp = mp, - .tp = tp, - }; -union xfs_rtword_raw *ondisk; - -/* -* fill the file one block at a time -*/ - nmap =1 ; - error = -libxfs_bmapi_write (tp, ip, bno,1 ,0 ,1 , &map, &nmap); -if (error || nmap !=1 ) { -do_error ( -_ ("couldn't map realtime bitmap block %" PRIu64", error = %d\n" ), - bno, error); - } - -ASSERT (map.br_startblock !=HOLESTARTBLOCK ); - - error = -libxfs_trans_read_buf ( - mp, tp, mp->m_dev, -XFS_FSB_TO_DADDR (mp, map.br_startblock), -XFS_FSB_TO_BB (mp,1 ),1 , &bp,NULL ); - -if (error) { -do_warn ( -_ ("can't access block %" PRIu64" (fsbno %" PRIu64") of realtime bitmap inode %" PRIu64"\n" ), - bno, map.br_startblock, mp->m_sb.sb_rbmino); -return (1 ); - } +_ ("couldn't iget realtime bitmap inode, error %d\n" ), error); - args.rbmbp = bp; - ondisk =xfs_rbmblock_wordptr (&args,0 ); -memcpy (ondisk, bmp, mp->m_blockwsize <<XFS_WORDLOG ); - -libxfs_trans_log_buf (tp, bp,0 , mp->m_sb.sb_blocksize -1 ); - - bmp += mp->m_blockwsize; - bno++; - } - -libxfs_trans_ijoin (tp, ip,0 ); - error = -libxfs_trans_commit (tp); + error = -libxfs_rtfile_initialize_blocks (ip,0 , mp->m_sb.sb_rbmblocks, + btmcompute);if (error) -do_error (_ ("%s: commit failed, error %d\n" ), __func__, error); +do_error ( +_ ("couldn't re-initialize realtime bitmap inode, error %d\n" ), error); +libxfs_irele (ip); -return (0 ); } -static int -fill_rsumino (xfs_mount_t *mp) +static void +fill_rsumino ( +struct xfs_mount *mp) { -struct xfs_buf *bp; -xfs_trans_t *tp; -xfs_inode_t *ip; -union xfs_suminfo_raw *smp; -int nmap; -int error; -xfs_fileoff_t bno; -xfs_bmbt_irec_t map; - - smp = sumcompute; - bno =0 ; +struct xfs_trans *tp; +struct xfs_inode *ip; +int error; error = -libxfs_trans_alloc_rollable (mp,10 , &tp);if (error)res_failed (error); error = -libxfs_iget (mp, tp, mp->m_sb.sb_rsumino,0 , &ip); -if (error) { +libxfs_trans_cancel (tp); +if (error)do_error ( -_ ("couldn't iget realtime summary inode -- error - %d\n" ), - error); - } - -while (bno < mp->m_rsumblocks) { -struct xfs_rtalloc_args args = { - .mp = mp, - .tp = tp, - }; -union xfs_suminfo_raw *ondisk; - -/* -* fill the file one block at a time -*/ - nmap =1 ; - error = -libxfs_bmapi_write (tp, ip, bno,1 ,0 ,1 , &map, &nmap); -if (error || nmap !=1 ) { -do_error ( -_ ("couldn't map realtime summary inode block %" PRIu64", error = %d\n" ), - bno, error); - } - -ASSERT (map.br_startblock !=HOLESTARTBLOCK ); - - error = -libxfs_trans_read_buf ( - mp, tp, mp->m_dev, -XFS_FSB_TO_DADDR (mp, map.br_startblock), -XFS_FSB_TO_BB (mp,1 ),1 , &bp,NULL ); - -if (error) { -do_warn ( -_ ("can't access block %" PRIu64" (fsbno %" PRIu64") of realtime summary inode %" PRIu64"\n" ), - bno, map.br_startblock, mp->m_sb.sb_rsumino); -libxfs_irele (ip); -return (1 ); - } +_ ("couldn't iget realtime summary inode, error %d\n" ), error); - args.sumbp = bp; - ondisk =xfs_rsumblock_infoptr (&args,0 ); -memcpy (ondisk, smp, mp->m_blockwsize <<XFS_WORDLOG ); - -libxfs_trans_log_buf (tp, bp,0 , mp->m_sb.sb_blocksize -1 ); - - smp += mp->m_blockwsize; - bno++; - } - -libxfs_trans_ijoin (tp, ip,0 ); - error = -libxfs_trans_commit (tp); + mp->m_rsumip = ip; + error = -libxfs_rtfile_initialize_blocks (ip,0 , mp->m_rsumblocks, + sumcompute); + mp->m_rsumip =NULL ;if (error) -do_error (_ ("%s: commit failed, error %d\n" ), __func__, error); +do_error ( +_ ("couldn't re-initialize realtime summary inode, error %d\n" ), error); +libxfs_irele (ip); -return (0 ); } static void
@@ -3302,15 +3205,8 @@ phase6(xfs_mount_t *mp)if (!no_modify) {do_log (_ (" - resetting contents of realtime bitmap and summary inodes\n" )); -if (fill_rbmino (mp)) { -do_warn ( -_ ("Warning: realtime bitmap may be inconsistent\n" )); - } - -if (fill_rsumino (mp)) { -do_warn ( -_ ("Warning: realtime bitmap may be inconsistent\n" )); - } +fill_rbmino (mp); +fill_rsumino (mp); }mark_standalone_inodes (mp);
PATCH 6/6 xfs_repair: stop preallocating blocks in mk_rbmino and mk_rsumino
In reply to Darrick J. WongFrom: Christoph Hellwig <[email protected]>
Now that repair is using libxfs_rtfile_initialize_blocks to write to the rtbitmap and rtsummary inodes, space allocation is already taken care of that helper and there is no need to preallocate it. Remove the code to do so.
--- repair/phase6.c | 116 +++++++------------------------------------------------ 1 file changed, 14 insertions(+), 102 deletions(-)
repair/phase6.c +14 -102
@@ -475,24 +475,16 @@ reset_sbroot_ino( }static void -mk_rbmino (xfs_mount_t *mp) +mk_rbmino ( +struct xfs_mount *mp) { -xfs_trans_t *tp; -xfs_inode_t *ip; -xfs_bmbt_irec_t *ep; -int i; -int nmap; -int error; -xfs_fileoff_t bno; -xfs_bmbt_irec_t map[XFS_BMAP_MAX_NMAP ]; -uint blocks; +struct xfs_trans *tp; +struct xfs_inode *ip; +int error; -/* -* first set up inode -*/ - i = -libxfs_trans_alloc_rollable (mp,10 , &tp); -if (i) -res_failed (i); + error = -libxfs_trans_alloc_rollable (mp,10 , &tp); +if (error) +res_failed (error); error = -libxfs_iget (mp, tp, mp->m_sb.sb_rbmino,0 , &ip);if (error) {
@@ -508,42 +500,6 @@ mk_rbmino(xfs_mount_t *mp) error = -libxfs_trans_commit (tp);if (error)do_error (_ ("%s: commit failed, error %d\n" ), __func__, error); - -/* -* then allocate blocks for file and fill with zeroes (stolen -* from mkfs) -*/ - blocks = mp->m_sb.sb_rbmblocks + -XFS_BM_MAXLEVELS (mp,XFS_DATA_FORK ) -1 ; - error = -libxfs_trans_alloc_rollable (mp, blocks, &tp); -if (error) -res_failed (error); - -libxfs_trans_ijoin (tp, ip,0 ); - bno =0 ; -while (bno < mp->m_sb.sb_rbmblocks) { - nmap =XFS_BMAP_MAX_NMAP ; - error = -libxfs_bmapi_write (tp, ip, bno, - (xfs_extlen_t)(mp->m_sb.sb_rbmblocks - bno), -0 , mp->m_sb.sb_rbmblocks, map, &nmap); -if (error) { -do_error ( -_ ("couldn't allocate realtime bitmap, error = %d\n" ), - error); - } -for (i =0 , ep = map; i < nmap; i++, ep++) { -libxfs_device_zero (mp->m_ddev_targp, -XFS_FSB_TO_DADDR (mp, ep->br_startblock), -XFS_FSB_TO_BB (mp, ep->br_blockcount)); - bno += ep->br_blockcount; - } - } - error = -libxfs_trans_commit (tp); -if (error) { -do_error ( -_ ("allocation of the realtime bitmap failed, error = %d\n" ), - error); - }libxfs_irele (ip); }
@@ -606,22 +562,13 @@ _("couldn't re-initialize realtime summary inode, error %d\n"), error);static void mk_rsumino (xfs_mount_t *mp) { -xfs_trans_t *tp; -xfs_inode_t *ip; -xfs_bmbt_irec_t *ep; -int i; -int nmap; -int error; -xfs_fileoff_t bno; -xfs_bmbt_irec_t map[XFS_BMAP_MAX_NMAP ]; -uint blocks; +struct xfs_trans *tp; +struct xfs_inode *ip; +int error; -/* -* first set up inode -*/ - i = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_ichange,10 ,0 ,0 , &tp); -if (i) -res_failed (i); + error = -libxfs_trans_alloc (mp, &M_RES (mp)->tr_ichange,10 ,0 ,0 , &tp); +if (error) +res_failed (error); error = -libxfs_iget (mp, tp, mp->m_sb.sb_rsumino,0 , &ip);if (error) {
@@ -637,41 +584,6 @@ mk_rsumino(xfs_mount_t *mp) error = -libxfs_trans_commit (tp);if (error)do_error (_ ("%s: commit failed, error %d\n" ), __func__, error); - -/* -* then allocate blocks for file and fill with zeroes (stolen -* from mkfs) -*/ - blocks = mp->m_rsumblocks +XFS_BM_MAXLEVELS (mp,XFS_DATA_FORK ) -1 ; - error = -libxfs_trans_alloc_rollable (mp, blocks, &tp); -if (error) -res_failed (error); - -libxfs_trans_ijoin (tp, ip,0 ); - bno =0 ; -while (bno < mp->m_rsumblocks) { - nmap =XFS_BMAP_MAX_NMAP ; - error = -libxfs_bmapi_write (tp, ip, bno, - (xfs_extlen_t)(mp->m_rsumblocks - bno), -0 , mp->m_rsumblocks, map, &nmap); -if (error) { -do_error ( -_ ("couldn't allocate realtime summary inode, error = %d\n" ), - error); - } -for (i =0 , ep = map; i < nmap; i++, ep++) { -libxfs_device_zero (mp->m_ddev_targp, -XFS_FSB_TO_DADDR (mp, ep->br_startblock), -XFS_FSB_TO_BB (mp, ep->br_blockcount)); - bno += ep->br_blockcount; - } - } - error = -libxfs_trans_commit (tp); -if (error) { -do_error ( -_ ("allocation of the realtime summary ino failed, error = %d\n" ), - error); - }libxfs_irele (ip); }
PATCH 6/7 xfs_scrub_all: bug fix for 6.12
In reply to Darrick J. WongHi all,
Fix a problem with xfs_scrub_all mistakenly thinking that a service finished before it really did.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-fixes-6.12
---
Commits in this patchset:
* xfs_scrub_all: wait for services to start activating
--- scrub/xfs_scrub_all.in | 52 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
PATCH 1/1 xfs_scrub_all: wait for services to start activating
From: Darrick J. Wong <[email protected]>
It seems that the function call to start a systemd unit completes asynchronously from any change in that unit's active state. On a lightly loaded system, a Start() call followed by an ActiveState() call actually sees the change in state from inactive to activating.
Unfortunately, on a heavily loaded system, the state change may take a few seconds. If this is the case, the wait() call can see that the unit state is "inactive", decide that the service already finished, and exit early, when in reality it hasn't even gotten to 'activating'.
Fix this by adding a second method that watches either for the inactive -> activating state transition or for the last exit from inactivation timestamp to change before waiting for the unit to reach inactive state.
--- scrub/xfs_scrub_all.in | 52 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
scrub/xfs_scrub_all.in +52 -0
@@ -249,6 +249,54 @@ class scrub_service(scrub_control): print(e, file = sys.stderr) return 'failed' + def last_activation(self): + '''Retrieve the last activation time, in microseconds since + boot.''' + global debug + + l = lambda: self.prop.Get('org.freedesktop.systemd1.Unit', + 'InactiveExitTimestampMonotonic') + try: + return self.__dbusrun(l) + except Exception as e: + if debug: + print(e, file = sys.stderr) + return 0 + + def wait_for_startup(self, last_active, wait_for = 30, interval = 0.5): + '''Wait for the service to start up. This is defined as + exiting the inactive state.''' + + for i in range(0, int(wait_for / interval)): + s = self.state() + if debug: + print('waiting for activation %s %s' % (self.unitname, s)) + if s == 'failed': + return 1 + if s != 'inactive': + return 0 + # If the unit is inactive but the last activation time + # doesn't match, then the service ran so quickly that + # it's already gone. + if last_active != self.last_activation(): + return 0 + time.sleep(interval) + + s = self.state() + if debug: + print('waited for startup %s %s' % (self.unitname, s)) + if s == 'failed': + return 1 + if s != 'inactive': + return 0 + + # If the unit is inactive but the last activation time doesn't + # match, then the service ran so quickly that it's already + # gone. + if last_active != self.last_activation(): + return 0 + return 2 + def wait(self, interval = 1): '''Wait until the service finishes.''' global debug
@@ -278,7 +326,11 @@ class scrub_service(scrub_control): print('starting %s' % self.unitname) try: + last_active = self.last_activation() self.__dbusrun(lambda: self.unit.Start('replace')) + ret = self.wait_for_startup(last_active) + if ret > 0: + return ret return self.wait() except dbus.exceptions.DBusException as e: # If the unit was masked, the sysadmin doesn't want us
PATCH 7/7 mkfs: new config file for 6.12 LTS
In reply to Darrick J. WongHi all,
New mkfs config file for 6.12 LTS.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=mkfs-configs-6.12
---
Commits in this patchset:
* mkfs: add a config file for 6.12 LTS kernels
--- mkfs/Makefile | 3 ++- mkfs/lts_6.12.conf | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) create mode 100644 mkfs/lts_6.12.conf
PATCH 1/1 mkfs: add a config file for 6.12 LTS kernels
From: Darrick J. Wong <[email protected]>
We didn't add any new ondisk features in 2023, so the config file is the same.
--- mkfs/Makefile | 3 ++- mkfs/lts_6.12.conf | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) create mode 100644 mkfs/lts_6.12.conf
mkfs/Makefile +2 -1
@@ -16,7 +16,8 @@ CFGFILES = \ lts_5.10.conf \ lts_5.15.conf \ lts_6.1.conf \ - lts_6.6.conf + lts_6.6.conf \ + lts_6.12.conf LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBBLKID) \ $(LIBUUID) $(LIBINIH) $(LIBURCU) $(LIBPTHREAD)
mkfs/lts_6.12.conf new file +19 -0
@@ -0,0 +1,19 @@ +# V5 features that were the mkfs defaults when the upstream Linux 6.12 LTS +# kernel was released at the end of 2024. + +[metadata] +bigtime=1 +crc=1 +finobt=1 +inobtcount=1 +reflink=1 +rmapbt=1 +autofsck=0 + +[inode] +sparse=1 +nrext64=1 +exchange=0 + +[naming] +parent=0