xfs: Extend per-inode extent counters
v5 cover xfs: Extend per-inode extent counters
The commit xfs: fix inode fork extent count overflow (3f8a4f1d876d3e3e49e50b0396eaffcc4ba71b08) mentions that 10 billion data fork extents should be possible to create. However the corresponding on-disk field has a signed 32-bit type. Hence this patchset extends the per-inode data fork extent counter to 64 bits (out of which 48 bits are used to store the extent count).
Also, XFS has an attribute fork extent counter which is 16 bits wide. A workload that,
1. Creates 1 million 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to insert 400,000 new 255-byte sized xattrs causes the xattr extent counter to overflow.
Dave tells me that there are instances where a single file has more than 100 million hardlinks. With parent pointers being stored in xattrs, we will overflow the signed 16-bits wide attribute extent counter when large number of hardlinks are created. Hence this patchset extends the on-disk field to 32-bits.
The following changes are made to accomplish this,
1. A 64-bit inode field is carved out of existing di_pad and di_flushiter fields to hold the 64-bit data fork extent counter.
2. The existing 32-bit inode data fork extent counter will be used to hold the attribute fork extent counter.
3. A new incompat superblock flag to prevent older kernels from mounting the filesystem.
The patchset has been tested by executing xfstests with the following mkfs.xfs options,
1. -m crc=0 -b size=1k
2. -m crc=0 -b size=4k
3. -m crc=0 -b size=512
4. -m rmapbt=1,reflink=1 -b size=1k
5. -m rmapbt=1,reflink=1 -b size=4k
Each of the above test scenarios were executed on the following combinations (For V4 FS test scenario, the last combination was omitted).
|-------------------------------+-----------| | Xfsprogs | Kernel | |-------------------------------+-----------| | Unpatched | Patched | | Patched (disable nrext64) | Unpatched | | Patched (disable nrext64) | Patched | | Patched (enable nrext64) | Patched | |-------------------------------+-----------|
I have also written tests to check if the correct extent counter fields are updated with/without the new incompat flag and to verify upgrading older fs instances to support large extent counters. I have also fixed xfs/270 test to work with the new code base.
These patches can also be obtained from
https://github.com/chandanr/linux.git at branch
xfs-incompat-extend-extcnt-v5.
I will be posting the changes associated with xfsprogs separately.
Changelog:
V4 -> V5:
1. Rebase on xfs-linux/for-next.
2. Use howmany_64() to compute height of maximum bmbt tree.
3. Rename disk and log inode's di_big_dextcnt to di_big_nextents.
4. Rename disk and log inode's di_big_aextcnt to di_big_anextents.
5. Since XFS_IBULK_NREXT64 is not associated with inode walking functionality, define it as the 32nd bit and mask it when passing xfs_ibulk->flags to xfs_iwalk() function.
V3 -> V4:
1. Rebase patchset on xfs-linux/for-next branch.
2. Carve out a 64-bit inode field out of the existing di_pad and di_flushiter fields to hold the 64-bit data fork extent counter.
3. Use the existing 32-bit inode data fork extent counter to hold the attr fork extent counter.
4. Verify the contents of newly introduced inode fields immediately after the inode has been read from the disk.
5. Upgrade inodes to be able to hold large extent counters when reading them from disk.
6. Use XFS_BULK_IREQ_NREXT64 as the flag that userspace can use to indicate that it can read 64-bit data fork extent counter.
7. Bulkstat ioctl returns -EOVERFLOW when userspace is not capable of working with large extent counters and inode's data fork extent count is larger than INT32_MAX.
V2 -> V3:
1. Define maximum extent length as a function of BMBT_BLOCKCOUNT_BITLEN.
2. Introduce xfs_iext_max_nextents() function in the patch series before renaming MAXEXTNUM/MAXAEXTNUM. This is done to reduce proliferation of macros indicating maximum extent count for data and attribute forks.
3. Define xfs_dfork_nextents() as an inline function.
4. Use xfs_rfsblock_t as the data type for variables that hold block count.
5. xfs_dfork_nextents() now returns -EFSCORRUPTED when an invalid fork is passed as an argument.
6. The following changes are done to enable bulkstat ioctl to report 64-bit extent counters,
- Carve out a new 64-bit field xfs_bulkstat->bs_extents64 from
xfs_bulkstat->bs_pad[].
- Carve out a new 64-bit field xfs_bulk_ireq->bulkstat_flags from
xfs_bulk_ireq->reserved[] to hold bulkstat specific operational
flags. Introduce XFS_IBULK_NREXT64 flag to indicate that
userspace has the necessary infrastructure to receive 64-bit
extent counters.
- Define the new flag XFS_BULK_IREQ_BULKSTAT for userspace to
indicate that xfs_bulk_ireq->bulkstat_flags has valid flags set.7. Rename the incompat flag from XFS_SB_FEAT_INCOMPAT_EXTCOUNT_64BIT to XFS_SB_FEAT_INCOMPAT_NREXT64.
8. Add a new helper function xfs_inode_to_disk_iext_counters() to convert from incore inode extent counters to ondisk inode extent counters.
9. Reuse XFS_ERRTAG_REDUCE_MAX_IEXTENTS error tag to skip reporting inodes with more than 10 extents when bulkstat ioctl is invoked by userspace.
10. Introduce the new per-inode XFS_DIFLAG2_NREXT64 flag to indicate that the inode uses 64-bit extent counter. This is used to allow administrators to upgrade existing filesystems.
11. Export presence of XFS_SB_FEAT_INCOMPAT_NREXT64 feature to userspace via XFS_IOC_FSGEOMETRY ioctl.
V1 -> V2:
1. Rebase patches on top of Darrick's btree-dynamic-depth branch.
2. Add new bulkstat ioctl version to support 64-bit data fork extent counter field.
3. Introduce new error tag to verify if the old bulkstat ioctls skip reporting inodes with large data fork extent counters.
Chandan Babu R (16):
xfs: Move extent count limits to xfs_format.h
xfs: Introduce xfs_iext_max_nextents() helper
xfs: Use xfs_extnum_t instead of basic data types
xfs: Introduce xfs_dfork_nextents() helper
xfs: Use basic types to define xfs_log_dinode's di_nextents and
di_anextents
xfs: Promote xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits
respectively
xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs
feature bit
xfs: Introduce XFS_FSOP_GEOM_FLAGS_NREXT64
xfs: Introduce XFS_DIFLAG2_NREXT64 and associated helpers
xfs: Use xfs_rfsblock_t to count maximum blocks that can be used by
BMBT
xfs: Introduce macros to represent new maximum extent counts for
data/attr forks
xfs: Introduce per-inode 64-bit extent counters
xfs: Conditionally upgrade existing inodes to use 64-bit extent
counters
xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters
xfs: Add XFS_SB_FEAT_INCOMPAT_NREXT64 to the list of supported flags
xfs: Define max extent length based on on-disk format definition
fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 82 ++++++++++++++++--------------- fs/xfs/libxfs/xfs_bmap_btree.c | 2 +- fs/xfs/libxfs/xfs_format.h | 60 +++++++++++++++++++---- fs/xfs/libxfs/xfs_fs.h | 13 +++-- fs/xfs/libxfs/xfs_ialloc.c | 2 + fs/xfs/libxfs/xfs_inode_buf.c | 84 +++++++++++++++++++++++++------- fs/xfs/libxfs/xfs_inode_fork.c | 14 +++--- fs/xfs/libxfs/xfs_inode_fork.h | 59 ++++++++++++++++++++++- fs/xfs/libxfs/xfs_log_format.h | 22 +++++++-- fs/xfs/libxfs/xfs_sb.c | 5 ++ fs/xfs/libxfs/xfs_trans_resv.c | 11 +++-- fs/xfs/libxfs/xfs_types.h | 11 +---- fs/xfs/scrub/bmap.c | 2 +- fs/xfs/scrub/inode.c | 20 ++++---- fs/xfs/xfs_bmap_util.c | 14 +++--- fs/xfs/xfs_inode.c | 4 +- fs/xfs/xfs_inode.h | 5 ++ fs/xfs/xfs_inode_item.c | 23 +++++++-- fs/xfs/xfs_inode_item_recover.c | 85 ++++++++++++++++++++++++++++----- fs/xfs/xfs_ioctl.c | 3 ++ fs/xfs/xfs_iomap.c | 28 +++++------ fs/xfs/xfs_itable.c | 27 ++++++++++- fs/xfs/xfs_itable.h | 7 ++- fs/xfs/xfs_iwalk.h | 7 ++- fs/xfs/xfs_mount.h | 2 + fs/xfs/xfs_trace.h | 4 +- 27 files changed, 445 insertions(+), 153 deletions(-)
Signature · 1 line
2.30.2
PATCH v5 1/16 xfs: Move extent count limits to xfs_format.h
Maximum values associated with extent counters i.e. Maximum extent length, Maximum data extents and Maximum xattr extents are dictated by the on-disk format. Hence move these definitions over to xfs_format.h.
--- fs/xfs/libxfs/xfs_format.h | 7 +++++++ fs/xfs/libxfs/xfs_types.h | 7 ------- 2 files changed, 7 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_format.h +7 -0
@@ -869,6 +869,13 @@ enum xfs_dinode_fmt { { XFS_DINODE_FMT_BTREE, "btree" }, \ { XFS_DINODE_FMT_UUID, "uuid" } +/* +* Max values for extlen, extnum, aextnum. +*/ +#define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ +#define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ +#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */ +/* * Inode minimum and maximum sizes. */
fs/xfs/libxfs/xfs_types.h +0 -7
@@ -56,13 +56,6 @@ typedef void * xfs_failaddr_t;#define NULLFSINO ((xfs_ino_t)-1)#define NULLAGINO ((xfs_agino_t)-1) -/* -* Max values for extlen, extnum, aextnum. -*/ -#define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ -#define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ -#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */ -/* * Minimum and maximum blocksize and sectorsize. * The blocksize upper limit is pretty much arbitrary.
Signature · 1 line
2.30.2
PATCH v5 2/16 xfs: Introduce xfs_iext_max_nextents() helper
In reply to Chandan Babu Rxfs_iext_max_nextents() returns the maximum number of extents possible for one of data, cow or attribute fork. This helper will be extended further in a future commit when maximum extent counts associated with data/attribute forks are increased.
--- fs/xfs/libxfs/xfs_bmap.c | 9 ++++----- fs/xfs/libxfs/xfs_inode_buf.c | 8 +++----- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.h | 8 ++++++++ 4 files changed, 16 insertions(+), 11 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +4 -5
@@ -74,13 +74,12 @@ xfs_bmap_compute_maxlevels(* ATTR2 we have to assume the worst case scenario of a minimum size * available. */ -if (whichfork ==XFS_DATA_FORK ) { - maxleafents =MAXEXTNUM ; + maxleafents =xfs_iext_max_nextents (whichfork); +if (whichfork ==XFS_DATA_FORK ) sz =XFS_BMDR_SPACE_CALC (MINDBTPTRS ); - }else { - maxleafents =MAXAEXTNUM ; +else sz =XFS_BMDR_SPACE_CALC (MINABTPTRS ); - } + maxrootrecs =xfs_bmdr_maxrecs (sz,0 ); minleafrecs = mp->m_bmap_dmnr[0 ]; minnoderecs = mp->m_bmap_dmnr[1 ];
fs/xfs/libxfs/xfs_inode_buf.c +3 -5
@@ -337,6 +337,7 @@ xfs_dinode_verify_fork( int whichfork) {uint32_t di_nextents =XFS_DFORK_NEXTENTS (dip, whichfork); +xfs_extnum_t max_extents;switch (XFS_DFORK_FORMAT (dip, whichfork)) {case XFS_DINODE_FMT_LOCAL :
@@ -358,12 +359,9 @@ xfs_dinode_verify_fork(return __this_address;break ;case XFS_DINODE_FMT_BTREE : -if (whichfork ==XFS_ATTR_FORK ) { -if (di_nextents >MAXAEXTNUM ) -return __this_address; - }else if (di_nextents >MAXEXTNUM ) { + max_extents =xfs_iext_max_nextents (whichfork); +if (di_nextents > max_extents)return __this_address; - }break ;default :return __this_address;
fs/xfs/libxfs/xfs_inode_fork.c +1 -1
@@ -744,7 +744,7 @@ xfs_iext_count_may_overflow(if (whichfork ==XFS_COW_FORK )return 0 ; - max_exts = (whichfork ==XFS_ATTR_FORK ) ?MAXAEXTNUM :MAXEXTNUM ; + max_exts =xfs_iext_max_nextents (whichfork);if (XFS_TEST_ERROR (false, ip->i_mount,XFS_ERRTAG_REDUCE_MAX_IEXTENTS )) max_exts =10 ;
fs/xfs/libxfs/xfs_inode_fork.h +8 -0
@@ -133,6 +133,14 @@ static inline int8_t xfs_ifork_format(struct xfs_ifork *ifp)return ifp->if_format; } +static inline xfs_extnum_t xfs_iext_max_nextents (int whichfork) +{ +if (whichfork ==XFS_DATA_FORK || whichfork ==XFS_COW_FORK ) +return MAXEXTNUM ; + +return MAXAEXTNUM ; +} +struct xfs_ifork *xfs_ifork_alloc (enum xfs_dinode_fmt format,xfs_extnum_t nextents);struct xfs_ifork *xfs_iext_state_to_fork (struct xfs_inode *ip,int state);
Signature · 1 line
2.30.2
PATCH v5 3/16 xfs: Use xfs_extnum_t instead of basic data types
In reply to Chandan Babu Rxfs_extnum_t is the type to use to declare variables which have values obtained from xfs_dinode->di_[a]nextents. This commit replaces basic types (e.g. uint32_t) with xfs_extnum_t for such variables.
--- fs/xfs/libxfs/xfs_bmap.c | 2 +- fs/xfs/libxfs/xfs_inode_buf.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/scrub/inode.c | 2 +- fs/xfs/xfs_trace.h | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +1 -1
@@ -54,7 +54,7 @@ xfs_bmap_compute_maxlevels( {int level;/* btree level */ uint maxblocks;/* max blocks at this level */ -uint maxleafents;/* max leaf entries possible */ +xfs_extnum_t maxleafents;/* max leaf entries possible */ int maxrootrecs;/* max records in root block */ int minleafrecs;/* min records in leaf block */ int minnoderecs;/* min records in node block */
fs/xfs/libxfs/xfs_inode_buf.c +1 -1
@@ -336,7 +336,7 @@ xfs_dinode_verify_fork( struct xfs_mount *mp, int whichfork) { - uint32_t di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork); +xfs_extnum_t di_nextents =XFS_DFORK_NEXTENTS (dip, whichfork);xfs_extnum_t max_extents;switch (XFS_DFORK_FORMAT (dip, whichfork)) {
fs/xfs/libxfs/xfs_inode_fork.c +1 -1
@@ -105,7 +105,7 @@ xfs_iformat_extents(struct xfs_mount *mp = ip->i_mount;struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);int state =xfs_bmap_fork_to_state (whichfork); -int nex =XFS_DFORK_NEXTENTS (dip, whichfork); +xfs_extnum_t nex =XFS_DFORK_NEXTENTS (dip, whichfork);int size = nex *sizeof (xfs_bmbt_rec_t);struct xfs_iext_cursor icur;struct xfs_bmbt_rec *dp;
fs/xfs/scrub/inode.c +1 -1
@@ -232,7 +232,7 @@ xchk_dinode(size_t fork_recs;unsigned long long isize;uint64_t flags2; -uint32_t nextents; +xfs_extnum_t nextents;prid_t prid;uint16_t flags;uint16_t mode;
fs/xfs/xfs_trace.h +1 -1
@@ -2169,7 +2169,7 @@ DECLARE_EVENT_CLASS(xfs_swap_extent_class, __field(int, which) __field(xfs_ino_t, ino) __field(int, format) - __field(int, nex) + __field(xfs_extnum_t, nex) __field(int, broot_size)__field (int, fork_off) ),
Signature · 1 line
2.30.2
PATCH v5 4/16 xfs: Introduce xfs_dfork_nextents() helper
In reply to Chandan Babu RThis commit replaces the macro XFS_DFORK_NEXTENTS() with the helper function xfs_dfork_nextents(). As of this commit, xfs_dfork_nextents() returns the same value as XFS_DFORK_NEXTENTS(). A future commit which extends inode's extent counter fields will add more logic to this helper.
This commit also replaces direct accesses to xfs_dinode->di_[a]nextents with calls to xfs_dfork_nextents().
No functional changes have been made.
--- fs/xfs/libxfs/xfs_format.h | 4 ---- fs/xfs/libxfs/xfs_inode_buf.c | 16 +++++++++++----- fs/xfs/libxfs/xfs_inode_fork.c | 10 ++++++---- fs/xfs/libxfs/xfs_inode_fork.h | 32 ++++++++++++++++++++++++++++++++ fs/xfs/scrub/inode.c | 18 ++++++++++-------- 5 files changed, 59 insertions(+), 21 deletions(-)
fs/xfs/libxfs/xfs_format.h +0 -4
@@ -925,10 +925,6 @@ enum xfs_dinode_fmt { ((w) ==XFS_DATA_FORK ? \ (dip)->di_format : \ (dip)->di_aformat) -#define XFS_DFORK_NEXTENTS (dip,w) \ - ((w) == XFS_DATA_FORK ? \ - be32_to_cpu((dip)->di_nextents) : \ - be16_to_cpu((dip)->di_anextents))/* * For block and character special files the 32bit dev_t is stored at the
fs/xfs/libxfs/xfs_inode_buf.c +11 -5
@@ -336,9 +336,11 @@ xfs_dinode_verify_fork( struct xfs_mount *mp, int whichfork) { -xfs_extnum_t di_nextents =XFS_DFORK_NEXTENTS (dip, whichfork); +xfs_extnum_t di_nextents;xfs_extnum_t max_extents; + di_nextents =xfs_dfork_nextents (dip, whichfork); +switch (XFS_DFORK_FORMAT (dip, whichfork)) {case XFS_DINODE_FMT_LOCAL :/*
@@ -405,6 +407,8 @@ xfs_dinode_verify(uint16_t flags;uint64_t flags2;uint64_t di_size; +xfs_extnum_t nextents; +xfs_filblks_t nblocks;if (dip->di_magic !=cpu_to_be16 (XFS_DINODE_MAGIC ))return __this_address;
@@ -435,10 +439,12 @@ xfs_dinode_verify(if ((S_ISLNK (mode) ||S_ISDIR (mode)) && di_size ==0 )return __this_address; + nextents =xfs_dfork_data_extents (dip); + nextents +=xfs_dfork_attr_extents (dip); + nblocks =be64_to_cpu (dip->di_nblocks); +/* Fork checks carried over from xfs_iformat_fork */ -if (mode && -be32_to_cpu (dip->di_nextents) +be16_to_cpu (dip->di_anextents) > -be64_to_cpu (dip->di_nblocks)) +if (mode && nextents > nblocks)return __this_address;if (mode &&XFS_DFORK_BOFF (dip) > mp->m_sb.sb_inodesize)
@@ -495,7 +501,7 @@ xfs_dinode_verify(default :return __this_address; } -if (dip->di_anextents) +if (xfs_dfork_attr_extents (dip))return __this_address; }
fs/xfs/libxfs/xfs_inode_fork.c +6 -4
@@ -105,7 +105,7 @@ xfs_iformat_extents(struct xfs_mount *mp = ip->i_mount;struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);int state =xfs_bmap_fork_to_state (whichfork); -xfs_extnum_t nex =XFS_DFORK_NEXTENTS (dip, whichfork); +xfs_extnum_t nex =xfs_dfork_nextents (dip, whichfork);int size = nex *sizeof (xfs_bmbt_rec_t);struct xfs_iext_cursor icur;struct xfs_bmbt_rec *dp;
@@ -230,7 +230,7 @@ xfs_iformat_data_fork(* depend on it. */ ip->i_df.if_format = dip->di_format; - ip->i_df.if_nextents =be32_to_cpu (dip->di_nextents); + ip->i_df.if_nextents =xfs_dfork_data_extents (dip);switch (inode->i_mode &S_IFMT ) {case S_IFIFO :
@@ -295,14 +295,16 @@ xfs_iformat_attr_fork(struct xfs_inode *ip, struct xfs_dinode *dip) { + xfs_extnum_t naextents;int error =0 ; + naextents =xfs_dfork_attr_extents (dip); +/* * Initialize the extent count early, as the per-format routines may * depend on it. */ - ip->i_afp =xfs_ifork_alloc (dip->di_aformat, -be16_to_cpu (dip->di_anextents)); + ip->i_afp =xfs_ifork_alloc (dip->di_aformat, naextents);switch (ip->i_afp->if_format) {case XFS_DINODE_FMT_LOCAL :
fs/xfs/libxfs/xfs_inode_fork.h +32 -0
@@ -141,6 +141,38 @@ static inline xfs_extnum_t xfs_iext_max_nextents(int whichfork)return MAXAEXTNUM ; } +static inline xfs_extnum_t +xfs_dfork_data_extents ( +struct xfs_dinode *dip) +{ +return be32_to_cpu (dip->di_nextents); +} + +static inline xfs_extnum_t +xfs_dfork_attr_extents ( +struct xfs_dinode *dip) +{ +return be16_to_cpu (dip->di_anextents); +} + +static inline xfs_extnum_t +xfs_dfork_nextents ( +struct xfs_dinode *dip, +int whichfork) +{ +switch (whichfork) { +case XFS_DATA_FORK : +return xfs_dfork_data_extents (dip); +case XFS_ATTR_FORK : +return xfs_dfork_attr_extents (dip); +default : +ASSERT (0 ); +break ; + } + +return 0 ; +} +struct xfs_ifork *xfs_ifork_alloc (enum xfs_dinode_fmt format,xfs_extnum_t nextents);struct xfs_ifork *xfs_iext_state_to_fork (struct xfs_inode *ip,int state);
fs/xfs/scrub/inode.c +10 -8
@@ -233,6 +233,7 @@ xchk_dinode(unsigned long long isize;uint64_t flags2;xfs_extnum_t nextents; +xfs_extnum_t naextents;prid_t prid;uint16_t flags;uint16_t mode;
@@ -391,7 +392,7 @@ xchk_dinode(xchk_inode_extsize (sc, dip, ino, mode, flags);/* di_nextents */ - nextents =be32_to_cpu (dip->di_nextents); + nextents =xfs_dfork_data_extents (dip); fork_recs =XFS_DFORK_DSIZE (dip, mp) /sizeof (struct xfs_bmbt_rec );switch (dip->di_format) {case XFS_DINODE_FMT_EXTENTS :
@@ -408,10 +409,12 @@ xchk_dinode(break ; } + naextents =xfs_dfork_attr_extents (dip); +/* di_forkoff */ if (XFS_DFORK_APTR (dip) >= (char *)dip + mp->m_sb.sb_inodesize)xchk_ino_set_corrupt (sc, ino); -if (dip->di_anextents !=0 && dip->di_forkoff ==0 ) +if (naextents !=0 && dip->di_forkoff ==0 )xchk_ino_set_corrupt (sc, ino);if (dip->di_forkoff ==0 && dip->di_aformat !=XFS_DINODE_FMT_EXTENTS )xchk_ino_set_corrupt (sc, ino);
@@ -423,19 +426,18 @@ xchk_dinode(xchk_ino_set_corrupt (sc, ino);/* di_anextents */ - nextents =be16_to_cpu (dip->di_anextents); fork_recs =XFS_DFORK_ASIZE (dip, mp) /sizeof (struct xfs_bmbt_rec );switch (dip->di_aformat) {case XFS_DINODE_FMT_EXTENTS : -if (nextents > fork_recs) +if (naextents > fork_recs)xchk_ino_set_corrupt (sc, ino);break ;case XFS_DINODE_FMT_BTREE : -if (nextents <= fork_recs) +if (naextents <= fork_recs)xchk_ino_set_corrupt (sc, ino);break ;default : -if (nextents !=0 ) +if (naextents !=0 )xchk_ino_set_corrupt (sc, ino); }
@@ -513,14 +515,14 @@ xchk_inode_xref_bmap( &nextents, &count);if (!xchk_should_check_xref (sc, &error,NULL ))return ; -if (nextents <be32_to_cpu (dip->di_nextents)) +if (nextents <xfs_dfork_data_extents (dip))xchk_ino_xref_set_corrupt (sc, sc->ip->i_ino); error =xfs_bmap_count_blocks (sc->tp, sc->ip,XFS_ATTR_FORK , &nextents, &acount);if (!xchk_should_check_xref (sc, &error,NULL ))return ; -if (nextents !=be16_to_cpu (dip->di_anextents)) +if (nextents !=xfs_dfork_attr_extents (dip))xchk_ino_xref_set_corrupt (sc, sc->ip->i_ino);/* Check nblocks against the inode. */
Signature · 1 line
2.30.2
PATCH v5 5/16 xfs: Use basic types to define xfs_log_dinode's di_nextents and di_anextents
In reply to Chandan Babu RA future commit will increase the width of xfs_extnum_t in order to facilitate larger per-inode extent counters. Hence this patch now uses basic types to define xfs_log_dinode->[di_nextents|dianextents].
--- fs/xfs/libxfs/xfs_log_format.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +2 -2
@@ -396,8 +396,8 @@ struct xfs_log_dinode {xfs_fsize_t di_size;/* number of bytes in file */ xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ -xfs_extnum_t di_nextents;/* number of extents in data fork */ -xfs_aextnum_t di_anextents;/* number of extents in attribute fork*/ +uint32_t di_nextents;/* number of extents in data fork */ +uint16_t di_anextents;/* number of extents in attribute fork*/ uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */
Signature · 1 line
2.30.2
PATCH v5 6/16 xfs: Promote xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits respectively
In reply to Chandan Babu RA future commit will introduce a 64-bit on-disk data extent counter and a 32-bit on-disk attr extent counter. This commit promotes xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits in order to correctly handle in-core versions of these quantities.
--- fs/xfs/libxfs/xfs_bmap.c | 6 +++--- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.h | 2 +- fs/xfs/libxfs/xfs_types.h | 4 ++-- fs/xfs/xfs_inode.c | 4 ++-- fs/xfs/xfs_trace.h | 2 +- 6 files changed, 10 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +3 -3
@@ -52,9 +52,9 @@ xfs_bmap_compute_maxlevels( xfs_mount_t *mp, /* file system mount structure */ int whichfork) /* data or attr fork */ { +xfs_extnum_t maxleafents;/* max leaf entries possible */ int level;/* btree level */ uint maxblocks;/* max blocks at this level */ -xfs_extnum_t maxleafents;/* max leaf entries possible */ int maxrootrecs;/* max records in root block */ int minleafrecs;/* min records in leaf block */ int minnoderecs;/* min records in node block */
@@ -83,7 +83,7 @@ xfs_bmap_compute_maxlevels( maxrootrecs =xfs_bmdr_maxrecs (sz,0 ); minleafrecs = mp->m_bmap_dmnr[0 ]; minnoderecs = mp->m_bmap_dmnr[1 ]; - maxblocks = (maxleafents + minleafrecs -1 ) / minleafrecs; + maxblocks =howmany_64 (maxleafents, minleafrecs);for (level =1 ; maxblocks >1 ; level++) {if (maxblocks <= maxrootrecs) maxblocks =1 ;
@@ -467,7 +467,7 @@ xfs_bmap_check_leaf_extents(if (bp_release)xfs_trans_brelse (NULL , bp); error_norelse: -xfs_warn (mp,"%s: BAD after btree leaves for %d extents" , +xfs_warn (mp,"%s: BAD after btree leaves for %llu extents" , __func__, i);xfs_err (mp,"%s: CORRUPTED BTREE OR SOMETHING" , __func__);xfs_force_shutdown (mp,SHUTDOWN_CORRUPT_INCORE );
fs/xfs/libxfs/xfs_inode_fork.c +1 -1
@@ -117,7 +117,7 @@ xfs_iformat_extents(* we just bail out rather than crash in kmem_alloc() or memcpy() below. */ if (unlikely (size <0 || size >XFS_DFORK_SIZE (dip, mp, whichfork))) { -xfs_warn (ip->i_mount,"corrupt inode %Lu ((a)extents = %d)." , +xfs_warn (ip->i_mount,"corrupt inode %llu ((a)extents = %llu)." , (unsigned long long ) ip->i_ino, nex); xfs_inode_verifier_error(ip, -EFSCORRUPTED, "xfs_iformat_extents(1)", dip, sizeof(*dip),
fs/xfs/libxfs/xfs_inode_fork.h +1 -1
@@ -21,9 +21,9 @@ struct xfs_ifork {void *if_root;/* extent tree root */ char *if_data;/* inline file data */ } if_u1; +xfs_extnum_t if_nextents;/* # of extents in this fork */ short if_broot_bytes;/* bytes allocated for root */ int8_t if_format;/* format of this fork */ -xfs_extnum_t if_nextents;/* # of extents in this fork */ };/*
fs/xfs/libxfs/xfs_types.h +2 -2
@@ -12,8 +12,8 @@ typedef uint32_t xfs_agblock_t; /* blockno in alloc. group */typedef uint32_t xfs_agino_t ;/* inode # within allocation grp */ typedef uint32_t xfs_extlen_t ;/* extent length in blocks */ typedef uint32_t xfs_agnumber_t ;/* allocation group number */ -typedef int32_t xfs_extnum_t ;/* # of extents in a file */ -typedef int16_t xfs_aextnum_t ;/* # extents in an attribute fork */ +typedef uint64_t xfs_extnum_t ;/* # of extents in a file */ +typedef uint32_t xfs_aextnum_t ;/* # extents in an attribute fork */ typedef int64_t xfs_fsize_t ;/* bytes in a file */ typedef uint64_t xfs_ufsize_t ;/* unsigned bytes in a file */
fs/xfs/xfs_inode.c +2 -2
@@ -3495,8 +3495,8 @@ xfs_iflush(if (XFS_TEST_ERROR (ip->i_df.if_nextents +xfs_ifork_nextents (ip->i_afp) > ip->i_nblocks, mp,XFS_ERRTAG_IFLUSH_5 )) {xfs_alert_tag (mp,XFS_PTAG_IFLUSH , -"%s: detected corrupt incore inode %Lu, " -"total extents = %d, nblocks = %Ld, ptr " PTR_FMT , +"%s: detected corrupt incore inode %llu, " +"total extents = %llu nblocks = %lld, ptr " PTR_FMT , __func__, ip->i_ino, ip->i_df.if_nextents +xfs_ifork_nextents (ip->i_afp), ip->i_nblocks, ip);
fs/xfs/xfs_trace.h +1 -1
@@ -2182,7 +2182,7 @@ DECLARE_EVENT_CLASS(xfs_swap_extent_class, __entry->broot_size = ip->i_df.if_broot_bytes; __entry->fork_off =XFS_IFORK_BOFF (ip); ), - TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %d, " + TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %llu, " "broot size %d, forkoff 0x%x", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino,
Signature · 1 line
2.30.2
Re: [PATCH V5 06/16] xfs: Promote xfs_extnum_t and xfs_aextnum_t to 64 and 32-bits respectively
Quoted 7 lines · Chandan Babu R
A future commit will introduce a 64-bit on-disk data extent counter and a 32-bit on-disk attr extent counter. This commit promotes
xfs_extnum_tandxfs_aextnum_tto 64 and 32-bits in order to correctly handle in-core versions of these quantities.Reported-by kernel test robot <[email protected]>Signed-off-by Chandan Babu R <[email protected]>
Looks good now,
--D
Quoted 116 lines
--- fs/xfs/libxfs/xfs_bmap.c | 6 +++--- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.h | 2 +- fs/xfs/libxfs/xfs_types.h | 4 ++-- fs/xfs/xfs_inode.c | 4 ++-- fs/xfs/xfs_trace.h | 2 +- 6 files changed, 10 insertions(+), 10 deletions(-)fs/xfs/libxfs/xfs_bmap.c +3
-3@@ -52,9 +52,9 @@ xfs_bmap_compute_maxlevels( xfs_mount_t *mp, /* file system mount structure */ int whichfork) /* data or attr fork */ { +xfs_extnum_t maxleafents;/* max leaf entries possible */ int level;/* btree level */ uint maxblocks;/* max blocks at this level */ -xfs_extnum_t maxleafents;/* max leaf entries possible */ int maxrootrecs;/* max records in root block */ int minleafrecs;/* min records in leaf block */ int minnoderecs;/* min records in node block */ @@ -83,7 +83,7 @@ xfs_bmap_compute_maxlevels( maxrootrecs =xfs_bmdr_maxrecs (sz,0 ); minleafrecs = mp->m_bmap_dmnr[0 ]; minnoderecs = mp->m_bmap_dmnr[1 ]; - maxblocks = (maxleafents + minleafrecs -1 ) / minleafrecs; + maxblocks =howmany_64 (maxleafents, minleafrecs);for (level =1 ; maxblocks >1 ; level++) {if (maxblocks <= maxrootrecs) maxblocks =1 ;@@ -467,7 +467,7 @@ xfs_bmap_check_leaf_extents(if (bp_release)xfs_trans_brelse (NULL , bp); error_norelse: -xfs_warn (mp,"%s: BAD after btree leaves for %d extents" , +xfs_warn (mp,"%s: BAD after btree leaves for %llu extents" , __func__, i);xfs_err (mp,"%s: CORRUPTED BTREE OR SOMETHING" , __func__);xfs_force_shutdown (mp,SHUTDOWN_CORRUPT_INCORE );fs/xfs/libxfs/xfs_inode_fork.c +1
-1@@ -117,7 +117,7 @@ xfs_iformat_extents(* we just bail out rather than crash in kmem_alloc() or memcpy() below. */ if (unlikely (size <0 || size >XFS_DFORK_SIZE (dip, mp, whichfork))) { -xfs_warn (ip->i_mount,"corrupt inode %Lu ((a)extents = %d)." , +xfs_warn (ip->i_mount,"corrupt inode %llu ((a)extents = %llu)." , (unsigned long long ) ip->i_ino, nex); xfs_inode_verifier_error(ip, -EFSCORRUPTED, "xfs_iformat_extents(1)", dip, sizeof(*dip),fs/xfs/libxfs/xfs_inode_fork.h +1
-1@@ -21,9 +21,9 @@ struct xfs_ifork {void *if_root;/* extent tree root */ char *if_data;/* inline file data */ } if_u1; +xfs_extnum_t if_nextents;/* # of extents in this fork */ short if_broot_bytes;/* bytes allocated for root */ int8_t if_format;/* format of this fork */ -xfs_extnum_t if_nextents;/* # of extents in this fork */ };/* fs/xfs/libxfs/xfs_types.h +2
-2@@ -12,8 +12,8 @@ typedef uint32_t xfs_agblock_t; /* blockno in alloc. group */typedef uint32_t xfs_agino_t ;/* inode # within allocation grp */ typedef uint32_t xfs_extlen_t ;/* extent length in blocks */ typedef uint32_t xfs_agnumber_t ;/* allocation group number */ -typedef int32_t xfs_extnum_t ;/* # of extents in a file */ -typedef int16_t xfs_aextnum_t ;/* # extents in an attribute fork */ +typedef uint64_t xfs_extnum_t ;/* # of extents in a file */ +typedef uint32_t xfs_aextnum_t ;/* # extents in an attribute fork */ typedef int64_t xfs_fsize_t ;/* bytes in a file */ typedef uint64_t xfs_ufsize_t ;/* unsigned bytes in a file */ fs/xfs/xfs_inode.c +2
-2@@ -3495,8 +3495,8 @@ xfs_iflush(if (XFS_TEST_ERROR (ip->i_df.if_nextents +xfs_ifork_nextents (ip->i_afp) > ip->i_nblocks, mp,XFS_ERRTAG_IFLUSH_5 )) {xfs_alert_tag (mp,XFS_PTAG_IFLUSH , -"%s: detected corrupt incore inode %Lu, " -"total extents = %d, nblocks = %Ld, ptr " PTR_FMT , +"%s: detected corrupt incore inode %llu, " +"total extents = %llu nblocks = %lld, ptr " PTR_FMT , __func__, ip->i_ino, ip->i_df.if_nextents +xfs_ifork_nextents (ip->i_afp), ip->i_nblocks, ip);fs/xfs/xfs_trace.h +1
-1@@ -2182,7 +2182,7 @@ DECLARE_EVENT_CLASS(xfs_swap_extent_class, __entry->broot_size = ip->i_df.if_broot_bytes; __entry->fork_off =XFS_IFORK_BOFF (ip); ), - TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %d, " + TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %llu, " "broot size %d, forkoff 0x%x", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino,Signature · 1 line
2.30.2
PATCH v5 7/16 xfs: Introduce XFS_SB_FEAT_INCOMPAT_NREXT64 and associated per-fs feature bit
In reply to Chandan Babu RXFS_SB_FEAT_INCOMPAT_NREXT64 incompat feature bit will be set on filesystems which support large per-inode extent counters. This commit defines the new incompat feature bit and the corresponding per-fs feature bit (along with inline functions to work on it).
--- fs/xfs/libxfs/xfs_format.h | 1 + fs/xfs/libxfs/xfs_sb.c | 3 +++ fs/xfs/xfs_mount.h | 2 ++ 3 files changed, 6 insertions(+)
fs/xfs/libxfs/xfs_format.h +1 -0
@@ -372,6 +372,7 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_META_UUID (1 << 2)/* metadata UUID */ #define XFS_SB_FEAT_INCOMPAT_BIGTIME (1 << 3)/* large timestamps */ #define XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR (1 << 4)/* needs xfs_repair */ +#define XFS_SB_FEAT_INCOMPAT_NREXT64 (1 << 5)/* 64-bit data fork extent counter */ #define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \
fs/xfs/libxfs/xfs_sb.c +3 -0
@@ -124,6 +124,9 @@ xfs_sb_version_to_features( features |=XFS_FEAT_BIGTIME ;if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR ) features |=XFS_FEAT_NEEDSREPAIR ; +if (sbp->sb_features_incompat &XFS_SB_FEAT_INCOMPAT_NREXT64 ) + features |=XFS_FEAT_NREXT64 ; +return features; }
fs/xfs/xfs_mount.h +2 -0
@@ -276,6 +276,7 @@ typedef struct xfs_mount {#define XFS_FEAT_INOBTCNT (1ULL << 23)/* inobt block counts */ #define XFS_FEAT_BIGTIME (1ULL << 24)/* large timestamps */ #define XFS_FEAT_NEEDSREPAIR (1ULL << 25)/* needs xfs_repair */ +#define XFS_FEAT_NREXT64 (1ULL << 26)/* 64-bit inode extent counters */ /* Mount features */ #define XFS_FEAT_NOATTR2 (1ULL << 48)/* disable attr2 creation */
@@ -338,6 +339,7 @@ __XFS_HAS_FEAT(realtime, REALTIME) __XFS_HAS_FEAT(inobtcounts, INOBTCNT) __XFS_HAS_FEAT(bigtime, BIGTIME) __XFS_HAS_FEAT(needsrepair, NEEDSREPAIR) +__XFS_HAS_FEAT (nrext64,NREXT64 )/* * Mount features
Signature · 1 line
2.30.2
PATCH v5 8/16 xfs: Introduce XFS_FSOP_GEOM_FLAGS_NREXT64
In reply to Chandan Babu RXFS_FSOP_GEOM_FLAGS_NREXT64 indicates that the current filesystem instance supports 64-bit per-inode extent counters.
--- fs/xfs/libxfs/xfs_fs.h | 1 + fs/xfs/libxfs/xfs_sb.c | 2 ++ 2 files changed, 3 insertions(+)
fs/xfs/libxfs/xfs_fs.h +1 -0
@@ -251,6 +251,7 @@ typedef struct xfs_fsop_resblks {#define XFS_FSOP_GEOM_FLAGS_REFLINK (1 << 20)/* files can share blocks */ #define XFS_FSOP_GEOM_FLAGS_BIGTIME (1 << 21)/* 64-bit nsec timestamps */ #define XFS_FSOP_GEOM_FLAGS_INOBTCNT (1 << 22)/* inobt btree counter */ +#define XFS_FSOP_GEOM_FLAGS_NREXT64 (1 << 23)/* 64-bit extent counter */ /* * Minimum and maximum sizes need for growth checks.
fs/xfs/libxfs/xfs_sb.c +2 -0
@@ -1138,6 +1138,8 @@ xfs_fs_geometry( }else { geo->logsectsize =BBSIZE ; } +if (xfs_has_nrext64 (mp)) + geo->flags |=XFS_FSOP_GEOM_FLAGS_NREXT64 ; geo->rtsectsize = sbp->sb_blocksize; geo->dirblocksize =xfs_dir2_dirblock_bytes (sbp);
Signature · 1 line
2.30.2
PATCH v5 9/16 xfs: Introduce XFS_DIFLAG2_NREXT64 and associated helpers
In reply to Chandan Babu RThis commit adds the new per-inode flag XFS_DIFLAG2_NREXT64 to indicate that an inode supports 64-bit extent counters. This flag is also enabled by default on newly created inodes when the corresponding filesystem has large extent counter feature bit (i.e. XFS_FEAT_NREXT64) set.
--- fs/xfs/libxfs/xfs_format.h | 10 +++++++++- fs/xfs/libxfs/xfs_ialloc.c | 2 ++ fs/xfs/xfs_inode.h | 5 +++++ fs/xfs/xfs_inode_item_recover.c | 6 ++++++ 4 files changed, 22 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +9 -1
@@ -992,15 +992,17 @@ static inline void xfs_dinode_put_rdev(struct xfs_dinode *dip, xfs_dev_t rdev)#define XFS_DIFLAG2_REFLINK_BIT 1/* file's blocks may be shared */ #define XFS_DIFLAG2_COWEXTSIZE_BIT 2/* copy on write extent size hint */ #define XFS_DIFLAG2_BIGTIME_BIT 3/* big timestamps */ +#define XFS_DIFLAG2_NREXT64_BIT 4/* 64-bit extent counter enabled */ #define XFS_DIFLAG2_DAX (1 << XFS_DIFLAG2_DAX_BIT)#define XFS_DIFLAG2_REFLINK (1 << XFS_DIFLAG2_REFLINK_BIT)#define XFS_DIFLAG2_COWEXTSIZE (1 << XFS_DIFLAG2_COWEXTSIZE_BIT)#define XFS_DIFLAG2_BIGTIME (1 << XFS_DIFLAG2_BIGTIME_BIT) +#define XFS_DIFLAG2_NREXT64 (1 << XFS_DIFLAG2_NREXT64_BIT)#define XFS_DIFLAG2_ANY \ (XFS_DIFLAG2_DAX | XFS_DIFLAG2_REFLINK | XFS_DIFLAG2_COWEXTSIZE | \ - XFS_DIFLAG2_BIGTIME) + XFS_DIFLAG2_BIGTIME | XFS_DIFLAG2_NREXT64)static inline bool xfs_dinode_has_bigtime (const struct xfs_dinode *dip) {
@@ -1008,6 +1010,12 @@ static inline bool xfs_dinode_has_bigtime(const struct xfs_dinode *dip) (dip->di_flags2 &cpu_to_be64 (XFS_DIFLAG2_BIGTIME )); } +static inline bool xfs_dinode_has_nrext64 (const struct xfs_dinode *dip) +{ +return dip->di_version >=3 && + (dip->di_flags2 &cpu_to_be64 (XFS_DIFLAG2_NREXT64 )); +} +/* * Inode number format: * low inopblog bits - offset in block
fs/xfs/libxfs/xfs_ialloc.c +2 -0
@@ -2772,6 +2772,8 @@ xfs_ialloc_setup_geometry( igeo->new_diflags2 =0 ;if (xfs_has_bigtime (mp)) igeo->new_diflags2 |=XFS_DIFLAG2_BIGTIME ; +if (xfs_has_nrext64 (mp)) + igeo->new_diflags2 |=XFS_DIFLAG2_NREXT64 ;/* Compute inode btree geometry. */ igeo->agino_log = sbp->sb_inopblog + sbp->sb_agblklog;
fs/xfs/xfs_inode.h +5 -0
@@ -218,6 +218,11 @@ static inline bool xfs_inode_has_bigtime(struct xfs_inode *ip)return ip->i_diflags2 &XFS_DIFLAG2_BIGTIME ; } +static inline bool xfs_inode_has_nrext64 (struct xfs_inode *ip) +{ +return ip->i_diflags2 &XFS_DIFLAG2_NREXT64 ; +} +/* * Return the buftarg used for data allocations on a given inode. */
fs/xfs/xfs_inode_item_recover.c +6 -0
@@ -142,6 +142,12 @@ xfs_log_dinode_to_disk_ts(return ts; } +static inline bool xfs_log_dinode_has_nrext64 (const struct xfs_log_dinode *ld) +{ +return ld->di_version >=3 && + (ld->di_flags2 &XFS_DIFLAG2_NREXT64 ); +} + STATIC void xfs_log_dinode_to_disk( struct xfs_log_dinode *from,
Signature · 1 line
2.30.2
PATCH v5 10/16 xfs: Use xfs_rfsblock_t to count maximum blocks that can be used by BMBT
In reply to Chandan Babu R--- fs/xfs/libxfs/xfs_bmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +2 -2
@@ -53,8 +53,8 @@ xfs_bmap_compute_maxlevels( int whichfork) /* data or attr fork */ {xfs_extnum_t maxleafents;/* max leaf entries possible */ +xfs_rfsblock_t maxblocks;/* max blocks at this level */ int level;/* btree level */ -uint maxblocks;/* max blocks at this level */ int maxrootrecs;/* max records in root block */ int minleafrecs;/* min records in leaf block */ int minnoderecs;/* min records in node block */
@@ -88,7 +88,7 @@ xfs_bmap_compute_maxlevels(if (maxblocks <= maxrootrecs) maxblocks =1 ;else - maxblocks = (maxblocks + minnoderecs -1 ) / minnoderecs; + maxblocks =howmany_64 (maxblocks, minnoderecs); } mp->m_bm_maxlevels[whichfork] = level;ASSERT (mp->m_bm_maxlevels[whichfork] <=xfs_bmbt_maxlevels_ondisk ());
Signature · 1 line
2.30.2
Re: [PATCH V5 10/16] xfs: Use xfs_rfsblock_t to count maximum blocks that can be used by BMBT
Quoted 2 lines · Chandan Babu R
Reported-by kernel test robot <[email protected]>Signed-off-by Chandan Babu R <[email protected]>
Looks good,
--D
Quoted 30 lines
--- fs/xfs/libxfs/xfs_bmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_bmap.c +2
-2@@ -53,8 +53,8 @@ xfs_bmap_compute_maxlevels( int whichfork) /* data or attr fork */ {xfs_extnum_t maxleafents;/* max leaf entries possible */ +xfs_rfsblock_t maxblocks;/* max blocks at this level */ int level;/* btree level */ -uint maxblocks;/* max blocks at this level */ int maxrootrecs;/* max records in root block */ int minleafrecs;/* min records in leaf block */ int minnoderecs;/* min records in node block */ @@ -88,7 +88,7 @@ xfs_bmap_compute_maxlevels(if (maxblocks <= maxrootrecs) maxblocks =1 ;else - maxblocks = (maxblocks + minnoderecs -1 ) / minnoderecs; + maxblocks =howmany_64 (maxblocks, minnoderecs); } mp->m_bm_maxlevels[whichfork] = level;ASSERT (mp->m_bm_maxlevels[whichfork] <=xfs_bmbt_maxlevels_ondisk ());Signature · 1 line
2.30.2
PATCH v5 11/16 xfs: Introduce macros to represent new maximum extent counts for data/attr forks
In reply to Chandan Babu RThis commit defines new macros to represent maximum extent counts allowed by filesystems which have support for large per-inode extent counters.
--- fs/xfs/libxfs/xfs_bmap.c | 8 +++----- fs/xfs/libxfs/xfs_bmap_btree.c | 2 +- fs/xfs/libxfs/xfs_format.h | 20 ++++++++++++++++---- fs/xfs/libxfs/xfs_inode_buf.c | 3 ++- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.h | 19 +++++++++++++++---- 6 files changed, 38 insertions(+), 16 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +3 -5
@@ -61,10 +61,8 @@ xfs_bmap_compute_maxlevels(int sz;/* root block size */ /* -* The maximum number of extents in a file, hence the maximum number of -* leaf entries, is controlled by the size of the on-disk extent count, -* either a signed 32-bit number for the data fork, or a signed 16-bit -* number for the attr fork. +* The maximum number of extents in a fork, hence the maximum number of +* leaf entries, is controlled by the size of the on-disk extent count. * * Note that we can no longer assume that if we are in ATTR1 that the * fork offset of all the inodes will be
@@ -74,7 +72,7 @@ xfs_bmap_compute_maxlevels(* ATTR2 we have to assume the worst case scenario of a minimum size * available. */ - maxleafents =xfs_iext_max_nextents (whichfork); + maxleafents =xfs_iext_max_nextents (xfs_has_nrext64 (mp), whichfork);if (whichfork ==XFS_DATA_FORK ) sz =XFS_BMDR_SPACE_CALC (MINDBTPTRS );else
fs/xfs/libxfs/xfs_bmap_btree.c +1 -1
@@ -611,7 +611,7 @@ xfs_bmbt_maxlevels_ondisk(void) minrecs[1 ] =xfs_bmbt_block_maxrecs (blocklen, false) /2 ;/* One extra level for the inode root. */ -return xfs_btree_compute_maxlevels (minrecs,MAXEXTNUM ) +1 ; +return xfs_btree_compute_maxlevels (minrecs,XFS_MAX_EXTCNT_DATA_FORK ) +1 ; }/*
fs/xfs/libxfs/xfs_format.h +16 -4
@@ -872,10 +872,22 @@ enum xfs_dinode_fmt {/* * Max values for extlen, extnum, aextnum. -*/ -#define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ -#define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ -#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */ +* +* The newly introduced data fork extent counter is a 64-bit field. However, the +* maximum number of extents in a file is limited to 2^54 extents (assuming one +* blocks per extent) by the 54-bit wide startoff field of an extent record. +* +* A further limitation applies as shown below, +* 2^63 (max file size) / 64k (max block size) = 2^47 +* +* Rounding up 47 to the nearest multiple of bits-per-byte results in 48. Hence +* 2^48 was chosen as the maximum data fork extent count. +*/ +#define MAXEXTLEN ((xfs_extlen_t)((1ULL << 21) - 1))/* 21 bits */ +#define XFS_MAX_EXTCNT_DATA_FORK ((xfs_extnum_t)((1ULL << 48) - 1))/* Unsigned 48-bits */ +#define XFS_MAX_EXTCNT_ATTR_FORK ((xfs_extnum_t)((1ULL << 32) - 1))/* Unsigned 32-bits */ +#define XFS_MAX_EXTCNT_DATA_FORK_OLD ((xfs_extnum_t)((1ULL << 31) - 1))/* Signed 32-bits */ +#define XFS_MAX_EXTCNT_ATTR_FORK_OLD ((xfs_extnum_t)((1ULL << 15) - 1))/* Signed 16-bits */ /* * Inode minimum and maximum sizes.
fs/xfs/libxfs/xfs_inode_buf.c +2 -1
@@ -361,7 +361,8 @@ xfs_dinode_verify_fork(return __this_address;break ;case XFS_DINODE_FMT_BTREE : - max_extents =xfs_iext_max_nextents (whichfork); + max_extents =xfs_iext_max_nextents (xfs_dinode_has_nrext64 (dip), + whichfork);if (di_nextents > max_extents)return __this_address;break ;
fs/xfs/libxfs/xfs_inode_fork.c +1 -1
@@ -746,7 +746,7 @@ xfs_iext_count_may_overflow(if (whichfork ==XFS_COW_FORK )return 0 ; - max_exts =xfs_iext_max_nextents (whichfork); + max_exts =xfs_iext_max_nextents (xfs_inode_has_nrext64 (ip), whichfork);if (XFS_TEST_ERROR (false, ip->i_mount,XFS_ERRTAG_REDUCE_MAX_IEXTENTS )) max_exts =10 ;
fs/xfs/libxfs/xfs_inode_fork.h +15 -4
@@ -133,12 +133,23 @@ static inline int8_t xfs_ifork_format(struct xfs_ifork *ifp)return ifp->if_format; } -static inline xfs_extnum_t xfs_iext_max_nextents (int whichfork) +static inline xfs_extnum_t xfs_iext_max_nextents (bool has_nrext64, +int whichfork) { -if (whichfork ==XFS_DATA_FORK || whichfork ==XFS_COW_FORK ) -return MAXEXTNUM ; +switch (whichfork) { +case XFS_DATA_FORK : +case XFS_COW_FORK : +return has_nrext64 ?XFS_MAX_EXTCNT_DATA_FORK + :XFS_MAX_EXTCNT_DATA_FORK_OLD ; + +case XFS_ATTR_FORK : +return has_nrext64 ?XFS_MAX_EXTCNT_ATTR_FORK + :XFS_MAX_EXTCNT_ATTR_FORK_OLD ; -return MAXAEXTNUM ; +default : +ASSERT (0 ); +return 0 ; + } } static inline xfs_extnum_t
Signature · 1 line
2.30.2
Re: [PATCH V5 11/16] xfs: Introduce macros to represent new maximum extent counts for data/attr forks
Quoted 64 lines · Chandan Babu R
This commit defines new macros to represent maximum extent counts allowed by filesystems which have support for large per-inode extent counters.
Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_bmap.c | 8 +++----- fs/xfs/libxfs/xfs_bmap_btree.c | 2 +- fs/xfs/libxfs/xfs_format.h | 20 ++++++++++++++++---- fs/xfs/libxfs/xfs_inode_buf.c | 3 ++- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.h | 19 +++++++++++++++---- 6 files changed, 38 insertions(+), 16 deletions(-)fs/xfs/libxfs/xfs_bmap.c +3
-5@@ -61,10 +61,8 @@ xfs_bmap_compute_maxlevels(int sz;/* root block size */ /* -* The maximum number of extents in a file, hence the maximum number of -* leaf entries, is controlled by the size of the on-disk extent count, -* either a signed 32-bit number for the data fork, or a signed 16-bit -* number for the attr fork. +* The maximum number of extents in a fork, hence the maximum number of +* leaf entries, is controlled by the size of the on-disk extent count. * * Note that we can no longer assume that if we are in ATTR1 that the * fork offset of all the inodes will be @@ -74,7 +72,7 @@ xfs_bmap_compute_maxlevels(* ATTR2 we have to assume the worst case scenario of a minimum size * available. */ - maxleafents =xfs_iext_max_nextents (whichfork); + maxleafents =xfs_iext_max_nextents (xfs_has_nrext64 (mp), whichfork);if (whichfork ==XFS_DATA_FORK ) sz =XFS_BMDR_SPACE_CALC (MINDBTPTRS );else fs/xfs/libxfs/xfs_bmap_btree.c +1
-1@@ -611,7 +611,7 @@ xfs_bmbt_maxlevels_ondisk(void) minrecs[1 ] =xfs_bmbt_block_maxrecs (blocklen, false) /2 ;/* One extra level for the inode root. */ -return xfs_btree_compute_maxlevels (minrecs,MAXEXTNUM ) +1 ; +return xfs_btree_compute_maxlevels (minrecs,XFS_MAX_EXTCNT_DATA_FORK ) +1 ; }/* fs/xfs/libxfs/xfs_format.h +0
-4@@ -872,10 +872,22 @@ enum xfs_dinode_fmt {/* * Max values for extlen, extnum, aextnum. -*/ -#define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ -#define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ -#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */
fs/xfs/libxfs/xfs_format.h enum xfs_dinode_fmt { · line 884
Quoted 10 lines
fs/xfs/libxfs/xfs_format.h +10
-0+* +* The newly introduced data fork extent counter is a 64-bit field. However, the +* maximum number of extents in a file is limited to 2^54 extents (assuming one +* blocks per extent) by the 54-bit wide startoff field of an extent record. +* +* A further limitation applies as shown below, +* 2^63 (max file size) / 64k (max block size) = 2^47 +* +* Rounding up 47 to the nearest multiple of bits-per-byte results in 48. Hence +* 2^48 was chosen as the maximum data fork extent count.
Ok. I know I've brought up previously the fact that we leave the upper 16 bits of di_big_nextents completely unused, AKA:
It's odd that startoff is a 54-bit field, di_big_nextents is a 64-bit field, but we don't allow more than 2^48 data fork extents even though that means that one cannot populate a file on a 4k-FSB filesystem with one extent record for each file block.
Prior to 5.16, a potential justification was that xfs_btree_cur supported exactly 9 levels and we didn't want to raise that all the way to 12 (or whatever you'd need to support a btree with 2^54 extent records) for *all cursor types* to handle Ultra Extreme Fragmentation.
Now that we have separate cursor caches for all btree types, we could create one bmbt cursor cache for NREXT64 data forks and another for all other cases, which (in my mind anyway) assuages that concern.
The other justification we've covered is that the incore btree for a data fork with 2^48 xfs_bmbt_irec records will consume a bit more than 2^52 bytes of memory, which is (AFAIK) the current x64 memory limit. Assuming that CPU manufacturers keep adding an extra address line bit every other year or so, the extremely wealthy could complain about hitting this limit as early as 2040. That's ~20 or so years out, which is probably enough time either to find a more efficient incore extent map structure due to customer demand or start using the upper 16 bits.
So with those two factors in mind, I /think/ I'm ok with approving this extension to the ondisk format.
IOWs, if anyone has an objection, the time to raise it is NOW.
--D
Quoted 71 lines
fs/xfs/libxfs/xfs_format.h +6
-0+*/ +#define MAXEXTLEN ((xfs_extlen_t)((1ULL << 21) - 1))/* 21 bits */ +#define XFS_MAX_EXTCNT_DATA_FORK ((xfs_extnum_t)((1ULL << 48) - 1))/* Unsigned 48-bits */ +#define XFS_MAX_EXTCNT_ATTR_FORK ((xfs_extnum_t)((1ULL << 32) - 1))/* Unsigned 32-bits */ +#define XFS_MAX_EXTCNT_DATA_FORK_OLD ((xfs_extnum_t)((1ULL << 31) - 1))/* Signed 32-bits */ +#define XFS_MAX_EXTCNT_ATTR_FORK_OLD ((xfs_extnum_t)((1ULL << 15) - 1))/* Signed 16-bits */ /* * Inode minimum and maximum sizes. fs/xfs/libxfs/xfs_inode_buf.c +2
-1@@ -361,7 +361,8 @@ xfs_dinode_verify_fork(return __this_address;break ;case XFS_DINODE_FMT_BTREE : - max_extents =xfs_iext_max_nextents (whichfork); + max_extents =xfs_iext_max_nextents (xfs_dinode_has_nrext64 (dip), + whichfork);if (di_nextents > max_extents)return __this_address;break ;fs/xfs/libxfs/xfs_inode_fork.c +1
-1@@ -746,7 +746,7 @@ xfs_iext_count_may_overflow(if (whichfork ==XFS_COW_FORK )return 0 ; - max_exts =xfs_iext_max_nextents (whichfork); + max_exts =xfs_iext_max_nextents (xfs_inode_has_nrext64 (ip), whichfork);if (XFS_TEST_ERROR (false, ip->i_mount,XFS_ERRTAG_REDUCE_MAX_IEXTENTS )) max_exts =10 ;fs/xfs/libxfs/xfs_inode_fork.h +15
-4@@ -133,12 +133,23 @@ static inline int8_t xfs_ifork_format(struct xfs_ifork *ifp)return ifp->if_format; } -static inline xfs_extnum_t xfs_iext_max_nextents (int whichfork) +static inline xfs_extnum_t xfs_iext_max_nextents (bool has_nrext64, +int whichfork) { -if (whichfork ==XFS_DATA_FORK || whichfork ==XFS_COW_FORK ) -return MAXEXTNUM ; +switch (whichfork) { +case XFS_DATA_FORK : +case XFS_COW_FORK : +return has_nrext64 ?XFS_MAX_EXTCNT_DATA_FORK + :XFS_MAX_EXTCNT_DATA_FORK_OLD ; + +case XFS_ATTR_FORK : +return has_nrext64 ?XFS_MAX_EXTCNT_ATTR_FORK + :XFS_MAX_EXTCNT_ATTR_FORK_OLD ; -return MAXAEXTNUM ; +default : +ASSERT (0 ); +return 0 ; + } } static inline xfs_extnum_tSignature · 1 line
2.30.2
PATCH v5 12/16 xfs: Introduce per-inode 64-bit extent counters
In reply to Chandan Babu RThis commit introduces new fields in the on-disk inode format to support 64-bit data fork extent counters and 32-bit attribute fork extent counters. The new fields will be used only when an inode has XFS_DIFLAG2_NREXT64 flag set. Otherwise we continue to use the regular 32-bit data fork extent counters and 16-bit attribute fork extent counters.
--- fs/xfs/libxfs/xfs_format.h | 22 +++++++-- fs/xfs/libxfs/xfs_inode_buf.c | 49 ++++++++++++++++++-- fs/xfs/libxfs/xfs_inode_fork.h | 6 +++ fs/xfs/libxfs/xfs_log_format.h | 22 +++++++-- fs/xfs/xfs_inode_item.c | 23 ++++++++-- fs/xfs/xfs_inode_item_recover.c | 79 ++++++++++++++++++++++++++++----- 6 files changed, 174 insertions(+), 27 deletions(-)
fs/xfs/libxfs/xfs_format.h +18 -4
@@ -792,16 +792,30 @@ struct xfs_dinode {__be32 di_nlink;/* number of links to file */ __be16 di_projid_lo;/* lower part of owner's project id */ __be16 di_projid_hi;/* higher part owner's project id */ -__u8 di_pad[6 ];/* unused, zeroed space */ -__be16 di_flushiter;/* incremented on flush */ +union { +__be64 di_big_nextents;/* NREXT64 data extents */ +__u8 di_v3_pad[8 ];/* !NREXT64 V3 inode zeroed space */ +struct { +__u8 di_v2_pad[6 ];/* V2 inode zeroed space */ +__be16 di_flushiter;/* V2 inode incremented on flush */ + }; + };xfs_timestamp_t di_atime;/* time last accessed */ xfs_timestamp_t di_mtime;/* time last modified */ xfs_timestamp_t di_ctime;/* time created/inode modified */ __be64 di_size;/* number of bytes in file */ __be64 di_nblocks;/* # of direct & btree blocks used */ __be32 di_extsize;/* basic/minimum extent size for file */ -__be32 di_nextents;/* number of extents in data fork */ -__be16 di_anextents;/* number of extents in attribute fork*/ +union { +struct { +__be32 di_big_anextents;/* NREXT64 attr extents */ +__be16 di_nrext64_pad;/* NREXT64 unused, zero */ + } __packed; +struct { +__be32 di_nextents;/* !NREXT64 data extents */ +__be16 di_anextents;/* !NREXT64 attr extents */ + } __packed; + };__u8 di_forkoff;/* attr fork offs, <<3 for 64b align */ __s8 di_aformat;/* format of attr fork's data */ __be32 di_dmevmask;/* DMIG event mask */
fs/xfs/libxfs/xfs_inode_buf.c +45 -4
89 lines not on this page · raw message
fs/xfs/libxfs/xfs_inode_fork.h +6 -0
@@ -156,6 +156,9 @@ static inline xfs_extnum_txfs_dfork_data_extents ( struct xfs_dinode *dip) { +if (xfs_dinode_has_nrext64 (dip)) +return be64_to_cpu (dip->di_big_nextents); +return be32_to_cpu (dip->di_nextents); }
@@ -163,6 +166,9 @@ static inline xfs_extnum_txfs_dfork_attr_extents ( struct xfs_dinode *dip) { +if (xfs_dinode_has_nrext64 (dip)) +return be32_to_cpu (dip->di_big_anextents); +return be16_to_cpu (dip->di_anextents); }
fs/xfs/libxfs/xfs_log_format.h +18 -4
@@ -388,16 +388,30 @@ struct xfs_log_dinode {uint32_t di_nlink;/* number of links to file */ uint16_t di_projid_lo;/* lower part of owner's project id */ uint16_t di_projid_hi;/* higher part of owner's project id */ -uint8_t di_pad[6 ];/* unused, zeroed space */ -uint16_t di_flushiter;/* incremented on flush */ +union { +uint64_t di_big_nextents;/* NREXT64 data extents */ +uint8_t di_v3_pad[8 ];/* !NREXT64 V3 inode zeroed space */ +struct { +uint8_t di_v2_pad[6 ];/* V2 inode zeroed space */ +uint16_t di_flushiter;/* V2 inode incremented on flush */ + }; + };xfs_log_timestamp_t di_atime;/* time last accessed */ xfs_log_timestamp_t di_mtime;/* time last modified */ xfs_log_timestamp_t di_ctime;/* time created/inode modified */ xfs_fsize_t di_size;/* number of bytes in file */ xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ -uint32_t di_nextents;/* number of extents in data fork */ -uint16_t di_anextents;/* number of extents in attribute fork*/ +union { +struct { +uint32_t di_big_anextents;/* NREXT64 attr extents */ +uint16_t di_nrext64_pad;/* NREXT64 unused, zero */ + } __packed; +struct { +uint32_t di_nextents;/* !NREXT64 data extents */ +uint16_t di_anextents;/* !NREXT64 attr extents */ + } __packed; + };uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */
fs/xfs/xfs_inode_item.c +19 -4
@@ -358,6 +358,21 @@ xfs_copy_dm_fields_to_log_dinode( } } +static inline void +xfs_inode_to_log_dinode_iext_counters ( +struct xfs_inode *ip, +struct xfs_log_dinode *to) +{ +if (xfs_inode_has_nrext64 (ip)) { + to->di_big_nextents =xfs_ifork_nextents (&ip->i_df); + to->di_big_anextents =xfs_ifork_nextents (ip->i_afp); + to->di_nrext64_pad =0 ; + }else { + to->di_nextents =xfs_ifork_nextents (&ip->i_df); + to->di_anextents =xfs_ifork_nextents (ip->i_afp); + } +} + static void xfs_inode_to_log_dinode( struct xfs_inode *ip,
@@ -373,7 +388,6 @@ xfs_inode_to_log_dinode( to->di_projid_lo = ip->i_projid &0xffff ; to->di_projid_hi = ip->i_projid >>16 ; -memset (to->di_pad,0 ,sizeof (to->di_pad));memset (to->di_pad3,0 ,sizeof (to->di_pad3)); to->di_atime =xfs_inode_to_log_dinode_ts (ip, inode->i_atime); to->di_mtime =xfs_inode_to_log_dinode_ts (ip, inode->i_mtime);
@@ -385,8 +399,6 @@ xfs_inode_to_log_dinode( to->di_size = ip->i_disk_size; to->di_nblocks = ip->i_nblocks; to->di_extsize = ip->i_extsize; - to->di_nextents =xfs_ifork_nextents (&ip->i_df); - to->di_anextents =xfs_ifork_nextents (ip->i_afp); to->di_forkoff = ip->i_forkoff; to->di_aformat =xfs_ifork_format (ip->i_afp); to->di_flags = ip->i_diflags;
@@ -406,11 +418,14 @@ xfs_inode_to_log_dinode( to->di_lsn = lsn;memset (to->di_pad2,0 ,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid); - to->di_flushiter =0 ; +memset (to->di_v3_pad,0 ,sizeof (to->di_v3_pad)); }else { to->di_version =2 ; to->di_flushiter = ip->i_flushiter; +memset (to->di_v2_pad,0 ,sizeof (to->di_v2_pad)); } + +xfs_inode_to_log_dinode_iext_counters (ip, to); }/*
fs/xfs/xfs_inode_item_recover.c +68 -11
133 lines not on this page · raw message
Signature · 1 line
2.30.2
kernel test robot
build botRe: [PATCH V5 12/16] xfs: Introduce per-inode 64-bit extent counters
Thank you for the patch! Yet something to improve:
Hi Chandan,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on xfs-linux/for-next]
[also build test ERROR on v5.17-rc1 next-20220125]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Chandan-Babu-R/xfs-Extend-per-inode-extent-counters/20220121-132128 base: https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next config: arm-randconfig-c003-20220122 (https://download.01.org/0day-ci/archive/20220126/202201260622.XxiP4fe5-lkp(a)intel.com/config) compiler: arm-linux-gnueabi-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/f12e8b5064fc3ef50c9d26f15f4a6984db59927c
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Chandan-Babu-R/xfs-Extend-per-inode-extent-counters/20220121-132128 git checkout f12e8b5064fc3ef50c9d26f15f4a6984db59927c
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arm SHELL=/bin/bash fs/xfs/
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
In file included from <command-line>:
In function 'xfs_check_ondisk_structs',
inlined from 'init_xfs_fs' at fs/xfs/xfs_super.c:2223:2:
>> include/linux/compiler_types.h:335:45: error: call to '__compiletime_assert_900' declared with attribute error: XFS: sizeof(struct xfs_dinode) is wrong, expected 176
335 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:316:25: note: in definition of macro '__compiletime_assert'
316 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:335:9: note: in expansion of macro '_compiletime_assert'
335 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
fs/xfs/xfs_ondisk.h:10:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
10 | BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \
| ^~~~~~~~~~~~~~~~
fs/xfs/xfs_ondisk.h:37:9: note: in expansion of macro 'XFS_CHECK_STRUCT_SIZE'
37 | XFS_CHECK_STRUCT_SIZE(struct xfs_dinode, 176);
| ^~~~~~~~~~~~~~~~~~~~~vim +/__compiletime_assert_900 +335 include/linux/compiler_types.h
eb5c2d4b45e3d2 Will Deacon 2020-07-21 321 eb5c2d4b45e3d2 Will Deacon 2020-07-21 322 #define _compiletime_assert(condition, msg, prefix, suffix) \ eb5c2d4b45e3d2 Will Deacon 2020-07-21 323 __compiletime_assert(condition, msg, prefix, suffix) eb5c2d4b45e3d2 Will Deacon 2020-07-21 324 eb5c2d4b45e3d2 Will Deacon 2020-07-21 325 /** eb5c2d4b45e3d2 Will Deacon 2020-07-21 326 * compiletime_assert - break build and emit msg if condition is false eb5c2d4b45e3d2 Will Deacon 2020-07-21 327 * @condition: a compile-time constant condition to check eb5c2d4b45e3d2 Will Deacon 2020-07-21 328 * @msg: a message to emit if condition is false eb5c2d4b45e3d2 Will Deacon 2020-07-21 329 * eb5c2d4b45e3d2 Will Deacon 2020-07-21 330 * In tradition of POSIX assert, this macro will break the build if the eb5c2d4b45e3d2 Will Deacon 2020-07-21 331 * supplied condition is *false*, emitting the supplied error message if the eb5c2d4b45e3d2 Will Deacon 2020-07-21 332 * compiler has support to do so. eb5c2d4b45e3d2 Will Deacon 2020-07-21 333 */ eb5c2d4b45e3d2 Will Deacon 2020-07-21 334 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21 @335 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)eb5c2d4b45e3d2 Will Deacon 2020-07-21 336
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH V5 12/16] xfs: Introduce per-inode 64-bit extent counters
Quoted 37 lines · kernel test robot
Hi Chandan,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on xfs-linux/for-next]
[also build test ERROR on v5.17-rc1 next-20220125]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]url: https://github.com/0day-ci/linux/commits/Chandan-Babu-R/xfs-Extend-per-inode-extent-counters/20220121-132128 base: https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next config: arm-randconfig-c003-20220122 (https://download.01.org/0day-ci/archive/20220126/202201260622.XxiP4fe5-lkp(a)intel.com/config) compiler: arm-linux-gnueabi-gcc (GCC) 11.2.0reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/f12e8b5064fc3ef50c9d26f15f4a6984db59927c
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Chandan-Babu-R/xfs-Extend-per-inode-extent-counters/20220121-132128 git checkoutf12e8b5064fc3ef50c9d26f15f4a6984db59927c
# save the config file to linux build tree
mkdirbuild_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arm SHELL=/bin/bashfs/xfs/If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>All errors (new ones prefixed by >>):
In file included from <command-line>:
In function 'xfs_check_ondisk_structs',inlined from 'init_xfs_fs' at fs/xfs/xfs_super.c:2223:2:Quoted 1 line
Quoted 1 line
include/linux/compiler_types.h:335:45: error: call to '__compiletime_assert_900' declared with attribute error: XFS:sizeof(struct xfs_dinode)is wrong, expected 176335 |
_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:316:25: note: in definition of macro '__compiletime_assert'
Quoted 15 lines
316 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:335:9: note: in expansion of macro '_compiletime_assert'
335 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
fs/xfs/xfs_ondisk.h:10:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
10 | BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \
| ^~~~~~~~~~~~~~~~
fs/xfs/xfs_ondisk.h:37:9: note: in expansion of macro 'XFS_CHECK_STRUCT_SIZE'
37 | XFS_CHECK_STRUCT_SIZE(struct xfs_dinode, 176);
| ^~~~~~~~~~~~~~~~~~~~~The following newly introduced union inside "struct xfs_dinode" and the corresponding one in "struct xfs_log_dinode",
union {
struct {
__be32 di_big_anextents; /* NREXT64 attr extents */
__be16 di_nrext64_pad; /* NREXT64 unused, zero */
} __packed;
struct {
__be32 di_nextents; /* !NREXT64 data extents */
__be16 di_anextents; /* !NREXT64 attr extents */
} __packed;
};needs to be packed as well. I will include this fix in the next version of the patchset.
Quoted 23 lines
vim +/__compiletime_assert_900 +335
include/linux/compiler_types.heb5c2d4b45e3d2 Will Deacon 2020-07-21 321 eb5c2d4b45e3d2 Will Deacon 2020-07-21 322 #define _compiletime_assert(condition, msg, prefix, suffix) \ eb5c2d4b45e3d2 Will Deacon 2020-07-21 323 __compiletime_assert(condition, msg, prefix, suffix) eb5c2d4b45e3d2 Will Deacon 2020-07-21 324 eb5c2d4b45e3d2 Will Deacon 2020-07-21 325 /** eb5c2d4b45e3d2 Will Deacon 2020-07-21 326 * compiletime_assert - break build and emit msg if condition is false eb5c2d4b45e3d2 Will Deacon 2020-07-21 327 * @condition: a compile-time constant condition to check eb5c2d4b45e3d2 Will Deacon 2020-07-21 328 * @msg: a message to emit if condition is false eb5c2d4b45e3d2 Will Deacon 2020-07-21 329 * eb5c2d4b45e3d2 Will Deacon 2020-07-21 330 * In tradition of POSIX assert, this macro will break the build if the eb5c2d4b45e3d2 Will Deacon 2020-07-21 331 * supplied condition is *false*, emitting the supplied error message if the eb5c2d4b45e3d2 Will Deacon 2020-07-21 332 * compiler has support to do so. eb5c2d4b45e3d2 Will Deacon 2020-07-21 333 */ eb5c2d4b45e3d2 Will Deacon 2020-07-21 334 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2Will Deacon 2020-07-21 @335_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)eb5c2d4b45e3d2Will Deacon 2020-07-21 336---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Signature · 1 line
chandan
Re: [PATCH V5 12/16] xfs: Introduce per-inode 64-bit extent counters
Quoted 67 lines · Chandan Babu R
Quoted 37 lines · kernel test robot
Hi Chandan,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on xfs-linux/for-next]
[also build test ERROR on v5.17-rc1 next-20220125]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]url: https://github.com/0day-ci/linux/commits/Chandan-Babu-R/xfs-Extend-per-inode-extent-counters/20220121-132128 base: https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next config: arm-randconfig-c003-20220122 (https://download.01.org/0day-ci/archive/20220126/202201260622.XxiP4fe5-lkp(a)intel.com/config) compiler: arm-linux-gnueabi-gcc (GCC) 11.2.0reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/f12e8b5064fc3ef50c9d26f15f4a6984db59927c
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Chandan-Babu-R/xfs-Extend-per-inode-extent-counters/20220121-132128 git checkoutf12e8b5064fc3ef50c9d26f15f4a6984db59927c
# save the config file to linux build tree
mkdirbuild_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arm SHELL=/bin/bashfs/xfs/If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>All errors (new ones prefixed by >>):
In file included from <command-line>:
In function 'xfs_check_ondisk_structs',inlined from 'init_xfs_fs' at fs/xfs/xfs_super.c:2223:2:Quoted 1 line
Quoted 1 line
include/linux/compiler_types.h:335:45: error: call to '__compiletime_assert_900' declared with attribute error: XFS:sizeof(struct xfs_dinode)is wrong, expected 176335 |
_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:316:25: note: in definition of macro '__compiletime_assert'Quoted 15 lines
316 | prefix ## suffix(); \ | ^~~~~~ include/linux/compiler_types.h:335:9: note: in expansion of macro '_compiletime_assert' 335 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) | ^~~~~~~~~~~~~~~~~~~ include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert' 39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg) | ^~~~~~~~~~~~~~~~~~ fs/xfs/xfs_ondisk.h:10:9: note: in expansion of macro 'BUILD_BUG_ON_MSG' 10 | BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \ | ^~~~~~~~~~~~~~~~ fs/xfs/xfs_ondisk.h:37:9: note: in expansion of macro 'XFS_CHECK_STRUCT_SIZE' 37 | XFS_CHECK_STRUCT_SIZE(struct xfs_dinode, 176); | ^~~~~~~~~~~~~~~~~~~~~The following newly introduced union inside "struct
xfs_dinode" and the corresponding one in "structxfs_log_dinode",union { struct { __be32 di_big_anextents; /* NREXT64 attr extents */ __be16 di_nrext64_pad; /* NREXT64 unused, zero */ } __packed; struct { __be32 di_nextents; /* !NREXT64 data extents */ __be16 di_anextents; /* !NREXT64 attr extents */ } __packed; };
Quoted 2 lines
needs to be packed as well. I will include this fix in the next version of the patchset.
Eughrhrghgg I hate C sometimes.
Thank you for fixing this.
--D
Quoted 27 lines
Quoted 23 lines
vim +/__compiletime_assert_900 +335
include/linux/compiler_types.heb5c2d4b45e3d2 Will Deacon 2020-07-21 321 eb5c2d4b45e3d2 Will Deacon 2020-07-21 322 #define _compiletime_assert(condition, msg, prefix, suffix) \ eb5c2d4b45e3d2 Will Deacon 2020-07-21 323 __compiletime_assert(condition, msg, prefix, suffix) eb5c2d4b45e3d2 Will Deacon 2020-07-21 324 eb5c2d4b45e3d2 Will Deacon 2020-07-21 325 /** eb5c2d4b45e3d2 Will Deacon 2020-07-21 326 * compiletime_assert - break build and emit msg if condition is false eb5c2d4b45e3d2 Will Deacon 2020-07-21 327 * @condition: a compile-time constant condition to check eb5c2d4b45e3d2 Will Deacon 2020-07-21 328 * @msg: a message to emit if condition is false eb5c2d4b45e3d2 Will Deacon 2020-07-21 329 * eb5c2d4b45e3d2 Will Deacon 2020-07-21 330 * In tradition of POSIX assert, this macro will break the build if the eb5c2d4b45e3d2 Will Deacon 2020-07-21 331 * supplied condition is *false*, emitting the supplied error message if the eb5c2d4b45e3d2 Will Deacon 2020-07-21 332 * compiler has support to do so. eb5c2d4b45e3d2 Will Deacon 2020-07-21 333 */ eb5c2d4b45e3d2 Will Deacon 2020-07-21 334 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2Will Deacon 2020-07-21 @335_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)eb5c2d4b45e3d2Will Deacon 2020-07-21 336---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.orgSignature · 1 line
chandan
Re: [PATCH V5 12/16] xfs: Introduce per-inode 64-bit extent counters
In reply to Chandan Babu RQuoted 25 lines · Chandan Babu R
This commit introduces new fields in the on-disk inode format to support 64-bit data fork extent counters and 32-bit attribute fork extent counters. The new fields will be used only when an inode has
XFS_DIFLAG2_NREXT64flag set. Otherwise we continue to use the regular 32-bit data fork extent counters and 16-bit attribute fork extent counters.Signed-off-by Chandan Babu R <[email protected]>Suggested-by Dave Chinner <[email protected]>--- fs/xfs/libxfs/xfs_format.h | 22 +++++++-- fs/xfs/libxfs/xfs_inode_buf.c | 49 ++++++++++++++++++-- fs/xfs/libxfs/xfs_inode_fork.h | 6 +++ fs/xfs/libxfs/xfs_log_format.h | 22 +++++++-- fs/xfs/xfs_inode_item.c | 23 ++++++++-- fs/xfs/xfs_inode_item_recover.c | 79 ++++++++++++++++++++++++++++----- 6 files changed, 174 insertions(+), 27 deletions(-)fs/xfs/libxfs/xfs_format.h +0
-0@@ -792,16 +792,30 @@ struct xfs_dinode {__be32 di_nlink;/* number of links to file */ __be16 di_projid_lo;/* lower part of owner's project id */ __be16 di_projid_hi;/* higher part owner's project id */
fs/xfs/libxfs/xfs_format.h struct xfs_dinode { · line 802
Quoted 10 lines
fs/xfs/libxfs/xfs_format.h +8
-2-__u8 di_pad[6 ];/* unused, zeroed space */ -__be16 di_flushiter;/* incremented on flush */ +union { +__be64 di_big_nextents;/* NREXT64 data extents */ +__u8 di_v3_pad[8 ];/* !NREXT64 V3 inode zeroed space */ +struct { +__u8 di_v2_pad[6 ];/* V2 inode zeroed space */ +__be16 di_flushiter;/* V2 inode incremented on flush */ + }; + };
I think it might be time to reflow part of the comments for these fields away from inline...
union {
/* Number of data fork extents if NREXT64 is set */
__be64 di_big_nextents;
/* Padding for V3 inodes without NREXT64 set. */
__be64 di_v3_pad;
/* Padding and inode flush counter for V2 inodes. */
struct {
__u8 di_v2_pad[6];
__be16 di_flushiter;
};
};Quoted 8 lines
fs/xfs/libxfs/xfs_format.h +0
-2xfs_timestamp_t di_atime;/* time last accessed */ xfs_timestamp_t di_mtime;/* time last modified */ xfs_timestamp_t di_ctime;/* time created/inode modified */ __be64 di_size;/* number of bytes in file */ __be64 di_nblocks;/* # of direct & btree blocks used */ __be32 di_extsize;/* basic/minimum extent size for file */ -__be32 di_nextents;/* number of extents in data fork */ -__be16 di_anextents;/* number of extents in attribute fork*/
fs/xfs/libxfs/xfs_format.h struct xfs_dinode { · line 818
Quoted 10 lines
fs/xfs/libxfs/xfs_format.h +10
-0+union { +struct { +__be32 di_big_anextents;/* NREXT64 attr extents */ +__be16 di_nrext64_pad;/* NREXT64 unused, zero */ + } __packed; +struct { +__be32 di_nextents;/* !NREXT64 data extents */ +__be16 di_anextents;/* !NREXT64 attr extents */ + } __packed; + };
union {
/*
* For V2 inodes and V3 inodes without NREXT64 set, this
* is the number of data and attr fork extents.
*/
struct {
__be32 di_nextents;
__be16 di_anextents;
} __packed;
/* Number of attr fork extents if NREXT64 is set. */
struct {
__be32 di_big_anextents;
__be16 di_nrext64_pad;
} __packed;
} __packed;Quoted 70 lines
fs/xfs/libxfs/xfs_format.h +0
-0__u8 di_forkoff;/* attr fork offs, <<3 for 64b align */ __s8 di_aformat;/* format of attr fork's data */ __be32 di_dmevmask;/* DMIG event mask */ fs/xfs/libxfs/xfs_inode_buf.c +23
-4@@ -279,6 +279,25 @@ xfs_inode_to_disk_ts(return ts; } +static inline void +xfs_inode_to_disk_iext_counters ( +struct xfs_inode *ip, +struct xfs_dinode *to) +{ +if (xfs_inode_has_nrext64 (ip)) { + to->di_big_nextents =cpu_to_be64 (xfs_ifork_nextents (&ip->i_df)); + to->di_big_anextents =cpu_to_be32 (xfs_ifork_nextents (ip->i_afp)); +/* +* We might be upgrading the inode to use larger extent counters +* than was previously used. Hence zero the unused field. +*/ + to->di_nrext64_pad =cpu_to_be16 (0 ); + }else { + to->di_nextents =cpu_to_be32 (xfs_ifork_nextents (&ip->i_df)); + to->di_anextents =cpu_to_be16 (xfs_ifork_nextents (ip->i_afp)); + } +} + void xfs_inode_to_disk( struct xfs_inode *ip,@@ -296,7 +315,6 @@ xfs_inode_to_disk( to->di_projid_lo =cpu_to_be16 (ip->i_projid &0xffff ); to->di_projid_hi =cpu_to_be16 (ip->i_projid >>16 ); -memset (to->di_pad,0 ,sizeof (to->di_pad)); to->di_atime =xfs_inode_to_disk_ts (ip, inode->i_atime); to->di_mtime =xfs_inode_to_disk_ts (ip, inode->i_mtime); to->di_ctime =xfs_inode_to_disk_ts (ip, inode->i_ctime);@@ -307,8 +325,6 @@ xfs_inode_to_disk( to->di_size =cpu_to_be64 (ip->i_disk_size); to->di_nblocks =cpu_to_be64 (ip->i_nblocks); to->di_extsize =cpu_to_be32 (ip->i_extsize); - to->di_nextents =cpu_to_be32 (xfs_ifork_nextents (&ip->i_df)); - to->di_anextents =cpu_to_be16 (xfs_ifork_nextents (ip->i_afp)); to->di_forkoff = ip->i_forkoff; to->di_aformat =xfs_ifork_format (ip->i_afp); to->di_flags =cpu_to_be16 (ip->i_diflags);@@ -323,11 +339,14 @@ xfs_inode_to_disk( to->di_lsn =cpu_to_be64 (lsn);memset (to->di_pad2,0 ,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid); - to->di_flushiter =0 ; +memset (to->di_v3_pad,0 ,sizeof (to->di_v3_pad)); }else { to->di_version =2 ; to->di_flushiter =cpu_to_be16 (ip->i_flushiter); +memset (to->di_v2_pad,0 ,sizeof (to->di_v2_pad)); } + +xfs_inode_to_disk_iext_counters (ip, to); } static xfs_failaddr_t@@ -397,6 +416,24 @@ xfs_dinode_verify_forkoff(return NULL ; }
fs/xfs/libxfs/xfs_inode_buf.c xfs_dinode_verify_forkoff( · line 428
Quoted 10 lines
fs/xfs/libxfs/xfs_inode_buf.c +10
-0+static xfs_failaddr_t +xfs_dinode_verify_nextents ( +struct xfs_mount *mp, +struct xfs_dinode *dip) +{ +if (xfs_dinode_has_nrext64 (dip)) { +if (!xfs_has_nrext64 (mp)) +return __this_address; +if (dip->di_nrext64_pad !=0 ) +return __this_address;
Don't we need to check that:
if (xfs_dfork_data_extents(dip) > XFS_MAX_EXTCNT_DATA_FORK) return __this_address; if (xfs_dfork_attr_extents(dip) > XFS_MAX_EXTCNT_ATTR_FORK) return __this_address;
here?
OH, the actual checking of the extent count fields is in xfs_dinode_verify_fork, isn't it?
I think that means this function exists to check the consistency of the nrext64 inode flag vs. the superblock nrext64 flag and the padding fields, right?
In which case... perhaps this should be xfs_dinode_verify_nrext64() ?
fs/xfs/libxfs/xfs_inode_buf.c xfs_dinode_verify_forkoff( · line 431
Quoted 3 lines
fs/xfs/libxfs/xfs_inode_buf.c +3
-0+ }else { +if (dip->di_version ==3 && dip->di_big_nextents !=0 ) +return __this_address;
We're using tagged unions in xfs_dinode, then "di_big_nextents" is meaningless on an inode that doesn't have NREXT64 set. IOWs,
} else if (dip->di_version >= 3) {
if (dip->di_v3_pad != 0)
return __this_address;
}(Note that I changed the type of di_v3_pad above.)
Quoted 69 lines
fs/xfs/libxfs/xfs_inode_buf.c +9
-0+ } + +return NULL ; +} + xfs_failaddr_t xfs_dinode_verify( struct xfs_mount *mp,@@ -440,6 +477,10 @@ xfs_dinode_verify(if ((S_ISLNK (mode) ||S_ISDIR (mode)) && di_size ==0 )return __this_address; + fa =xfs_dinode_verify_nextents (mp, dip); +if (fa) +return fa; + nextents =xfs_dfork_data_extents (dip); nextents +=xfs_dfork_attr_extents (dip); nblocks =be64_to_cpu (dip->di_nblocks);fs/xfs/libxfs/xfs_inode_fork.h +6
-0@@ -156,6 +156,9 @@ static inline xfs_extnum_txfs_dfork_data_extents ( struct xfs_dinode *dip) { +if (xfs_dinode_has_nrext64 (dip)) +return be64_to_cpu (dip->di_big_nextents); +return be32_to_cpu (dip->di_nextents); }@@ -163,6 +166,9 @@ static inline xfs_extnum_txfs_dfork_attr_extents ( struct xfs_dinode *dip) { +if (xfs_dinode_has_nrext64 (dip)) +return be32_to_cpu (dip->di_big_anextents); +return be16_to_cpu (dip->di_anextents); }fs/xfs/libxfs/xfs_log_format.h +8
-4@@ -388,16 +388,30 @@ struct xfs_log_dinode {uint32_t di_nlink;/* number of links to file */ uint16_t di_projid_lo;/* lower part of owner's project id */ uint16_t di_projid_hi;/* higher part of owner's project id */ -uint8_t di_pad[6 ];/* unused, zeroed space */ -uint16_t di_flushiter;/* incremented on flush */ +union { +uint64_t di_big_nextents;/* NREXT64 data extents */ +uint8_t di_v3_pad[8 ];/* !NREXT64 V3 inode zeroed space */ +struct { +uint8_t di_v2_pad[6 ];/* V2 inode zeroed space */ +uint16_t di_flushiter;/* V2 inode incremented on flush */ + }; + };xfs_log_timestamp_t di_atime;/* time last accessed */ xfs_log_timestamp_t di_mtime;/* time last modified */ xfs_log_timestamp_t di_ctime;/* time created/inode modified */ xfs_fsize_t di_size;/* number of bytes in file */ xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ -uint32_t di_nextents;/* number of extents in data fork */ -uint16_t di_anextents;/* number of extents in attribute fork*/
fs/xfs/libxfs/xfs_log_format.h struct xfs_log_dinode { · line 414
Quoted 10 lines
fs/xfs/libxfs/xfs_log_format.h +10
-0+union { +struct { +uint32_t di_big_anextents;/* NREXT64 attr extents */ +uint16_t di_nrext64_pad;/* NREXT64 unused, zero */ + } __packed; +struct { +uint32_t di_nextents;/* !NREXT64 data extents */ +uint16_t di_anextents;/* !NREXT64 attr extents */ + } __packed; + };
I think you could apply the same transformations as I did to xfs_dinode above.
--D
Quoted 209 lines
fs/xfs/libxfs/xfs_log_format.h +0
-0uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */ fs/xfs/xfs_inode_item.c +19
-4@@ -358,6 +358,21 @@ xfs_copy_dm_fields_to_log_dinode( } } +static inline void +xfs_inode_to_log_dinode_iext_counters ( +struct xfs_inode *ip, +struct xfs_log_dinode *to) +{ +if (xfs_inode_has_nrext64 (ip)) { + to->di_big_nextents =xfs_ifork_nextents (&ip->i_df); + to->di_big_anextents =xfs_ifork_nextents (ip->i_afp); + to->di_nrext64_pad =0 ; + }else { + to->di_nextents =xfs_ifork_nextents (&ip->i_df); + to->di_anextents =xfs_ifork_nextents (ip->i_afp); + } +} + static void xfs_inode_to_log_dinode( struct xfs_inode *ip,@@ -373,7 +388,6 @@ xfs_inode_to_log_dinode( to->di_projid_lo = ip->i_projid &0xffff ; to->di_projid_hi = ip->i_projid >>16 ; -memset (to->di_pad,0 ,sizeof (to->di_pad));memset (to->di_pad3,0 ,sizeof (to->di_pad3)); to->di_atime =xfs_inode_to_log_dinode_ts (ip, inode->i_atime); to->di_mtime =xfs_inode_to_log_dinode_ts (ip, inode->i_mtime);@@ -385,8 +399,6 @@ xfs_inode_to_log_dinode( to->di_size = ip->i_disk_size; to->di_nblocks = ip->i_nblocks; to->di_extsize = ip->i_extsize; - to->di_nextents =xfs_ifork_nextents (&ip->i_df); - to->di_anextents =xfs_ifork_nextents (ip->i_afp); to->di_forkoff = ip->i_forkoff; to->di_aformat =xfs_ifork_format (ip->i_afp); to->di_flags = ip->i_diflags;@@ -406,11 +418,14 @@ xfs_inode_to_log_dinode( to->di_lsn = lsn;memset (to->di_pad2,0 ,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid); - to->di_flushiter =0 ; +memset (to->di_v3_pad,0 ,sizeof (to->di_v3_pad)); }else { to->di_version =2 ; to->di_flushiter = ip->i_flushiter; +memset (to->di_v2_pad,0 ,sizeof (to->di_v2_pad)); } + +xfs_inode_to_log_dinode_iext_counters (ip, to); }/* fs/xfs/xfs_inode_item_recover.c +68
-11@@ -148,6 +148,22 @@ static inline bool xfs_log_dinode_has_nrext64(const struct xfs_log_dinode *ld) (ld->di_flags2 &XFS_DIFLAG2_NREXT64 ); } +static inline void +xfs_log_dinode_to_disk_iext_counters ( +struct xfs_log_dinode *from, +struct xfs_dinode *to) +{ +if (xfs_log_dinode_has_nrext64 (from)) { + to->di_big_nextents =cpu_to_be64 (from->di_big_nextents); + to->di_big_anextents =cpu_to_be32 (from->di_big_anextents); + to->di_nrext64_pad =cpu_to_be16 (from->di_nrext64_pad); + }else { + to->di_nextents =cpu_to_be32 (from->di_nextents); + to->di_anextents =cpu_to_be16 (from->di_anextents); + } + +} + STATIC void xfs_log_dinode_to_disk( struct xfs_log_dinode *from,@@ -164,7 +180,6 @@ xfs_log_dinode_to_disk( to->di_nlink =cpu_to_be32 (from->di_nlink); to->di_projid_lo =cpu_to_be16 (from->di_projid_lo); to->di_projid_hi =cpu_to_be16 (from->di_projid_hi); -memcpy (to->di_pad, from->di_pad,sizeof (to->di_pad)); to->di_atime =xfs_log_dinode_to_disk_ts (from, from->di_atime); to->di_mtime =xfs_log_dinode_to_disk_ts (from, from->di_mtime);@@ -173,8 +188,6 @@ xfs_log_dinode_to_disk( to->di_size =cpu_to_be64 (from->di_size); to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); - to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);@@ -192,10 +205,13 @@ xfs_log_dinode_to_disk( to->di_lsn =cpu_to_be64 (lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &from->di_uuid); - to->di_flushiter =0 ; +memcpy (to->di_v3_pad, from->di_v3_pad,sizeof (to->di_v3_pad)); }else { to->di_flushiter =cpu_to_be16 (from->di_flushiter); +memcpy (to->di_v2_pad, from->di_v2_pad,sizeof (to->di_v2_pad)); } + +xfs_log_dinode_to_disk_iext_counters (from, to); }STATIC int@@ -209,6 +225,8 @@ xlog_recover_inode_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_buf *bp;struct xfs_dinode *dip; +xfs_extnum_t nextents; +xfs_aextnum_t anextents;int len;char *src;char *dest;@@ -348,21 +366,60 @@ xlog_recover_inode_commit_pass2( goto out_release; } } -if (unlikely (ldip->di_nextents + ldip->di_anextents > ldip->di_nblocks)){ -XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(5)" , + +if (xfs_log_dinode_has_nrext64 (ldip)) { +if (!xfs_has_nrext64 (mp) || (ldip->di_nrext64_pad !=0 )) { +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(5)" , +XFS_ERRLEVEL_LOW , mp, ldip, +sizeof (*ldip)); +xfs_alert (mp, +"%s: Bad inode log record, rec ptr " PTR_FMT ", " +"dino ptr " PTR_FMT ", dino bp " PTR_FMT ", " +"ino %Ld, xfs_has_nrext64(mp) = %d, " +"ldip->di_nrext64_pad = %u" , + __func__, item, dip, bp, in_f->ilf_ino, +xfs_has_nrext64 (mp), ldip->di_nrext64_pad); + error = -EFSCORRUPTED ; + goto out_release; + } + }else { +if (ldip->di_version ==3 && ldip->di_big_nextents !=0 ) { +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(6)" , +XFS_ERRLEVEL_LOW , mp, ldip, +sizeof (*ldip)); +xfs_alert (mp, +"%s: Bad inode log record, rec ptr " PTR_FMT ", " +"dino ptr " PTR_FMT ", dino bp " PTR_FMT ", " +"ino %Ld, ldip->di_big_dextcnt = %llu" , + __func__, item, dip, bp, in_f->ilf_ino, + ldip->di_big_nextents); + error = -EFSCORRUPTED ; + goto out_release; + } + } + +if (xfs_log_dinode_has_nrext64 (ldip)) { + nextents = ldip->di_big_nextents; + anextents = ldip->di_big_anextents; + }else { + nextents = ldip->di_nextents; + anextents = ldip->di_anextents; + } + +if (unlikely (nextents + anextents > ldip->di_nblocks)) { +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(7)" ,XFS_ERRLEVEL_LOW , mp, ldip,sizeof (*ldip));xfs_alert (mp,"%s: Bad inode log record, rec ptr " PTR_FMT ", dino ptr " PTR_FMT ", " -"dino bp " PTR_FMT ", ino %Ld, total extents = %d, nblocks = %Ld" , +"dino bp " PTR_FMT ", ino %Ld, total extents = %llu, nblocks = %Ld" , __func__, item, dip, bp, in_f->ilf_ino, - ldip->di_nextents + ldip->di_anextents, - ldip->di_nblocks); + nextents + anextents, ldip->di_nblocks); error = -EFSCORRUPTED ; goto out_release; }if (unlikely (ldip->di_forkoff > mp->m_sb.sb_inodesize)) { -XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(6)" , +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(8)" ,XFS_ERRLEVEL_LOW , mp, ldip,sizeof (*ldip)); xfs_alert(mp,@@ -374,7 +431,7 @@ xlog_recover_inode_commit_pass2( } isize =xfs_log_dinode_size (mp);if (unlikely (item->ri_buf[1 ].i_len > isize)) { -XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(7)" , +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(9)" ,XFS_ERRLEVEL_LOW , mp, ldip,sizeof (*ldip)); xfs_alert(mp,Signature · 1 line
2.30.2
Re: [PATCH V5 12/16] xfs: Introduce per-inode 64-bit extent counters
Quoted 40 lines · Darrick J. Wong
Quoted 25 lines · Chandan Babu R
This commit introduces new fields in the on-disk inode format to support 64-bit data fork extent counters and 32-bit attribute fork extent counters. The new fields will be used only when an inode has
XFS_DIFLAG2_NREXT64flag set. Otherwise we continue to use the regular 32-bit data fork extent counters and 16-bit attribute fork extent counters.Signed-off-by Chandan Babu R <[email protected]>Suggested-by Dave Chinner <[email protected]>--- fs/xfs/libxfs/xfs_format.h | 22 +++++++-- fs/xfs/libxfs/xfs_inode_buf.c | 49 ++++++++++++++++++-- fs/xfs/libxfs/xfs_inode_fork.h | 6 +++ fs/xfs/libxfs/xfs_log_format.h | 22 +++++++-- fs/xfs/xfs_inode_item.c | 23 ++++++++-- fs/xfs/xfs_inode_item_recover.c | 79 ++++++++++++++++++++++++++++----- 6 files changed, 174 insertions(+), 27 deletions(-)fs/xfs/libxfs/xfs_format.h +0
-0@@ -792,16 +792,30 @@ struct xfs_dinode {__be32 di_nlink;/* number of links to file */ __be16 di_projid_lo;/* lower part of owner's project id */ __be16 di_projid_hi;/* higher part owner's project id */ fs/xfs/libxfs/xfs_format.h struct xfs_dinode { · line 802
Quoted 10 lines
fs/xfs/libxfs/xfs_format.h +8
-2-__u8 di_pad[6 ];/* unused, zeroed space */ -__be16 di_flushiter;/* incremented on flush */ +union { +__be64 di_big_nextents;/* NREXT64 data extents */ +__u8 di_v3_pad[8 ];/* !NREXT64 V3 inode zeroed space */ +struct { +__u8 di_v2_pad[6 ];/* V2 inode zeroed space */ +__be16 di_flushiter;/* V2 inode incremented on flush */ + }; + };I think it might be time to reflow part of the comments for these fields away from inline...
Quoted 14 lines
union {
/* Number of data fork extents if NREXT64 is set */
__be64 di_big_nextents;
/* Padding for V3 inodes without NREXT64 set. */
__be64 di_v3_pad;
/* Padding and inode flush counter for V2 inodes. */
struct {
__u8 di_v2_pad[6];
__be16 di_flushiter;
};
};Ok. I will include these changes in the next version.
Quoted 127 lines
Quoted 8 lines
fs/xfs/libxfs/xfs_format.h +0
-2xfs_timestamp_t di_atime;/* time last accessed */ xfs_timestamp_t di_mtime;/* time last modified */ xfs_timestamp_t di_ctime;/* time created/inode modified */ __be64 di_size;/* number of bytes in file */ __be64 di_nblocks;/* # of direct & btree blocks used */ __be32 di_extsize;/* basic/minimum extent size for file */ -__be32 di_nextents;/* number of extents in data fork */ -__be16 di_anextents;/* number of extents in attribute fork*/ fs/xfs/libxfs/xfs_format.h struct xfs_dinode { · line 818
Quoted 10 lines
fs/xfs/libxfs/xfs_format.h +10
-0+union { +struct { +__be32 di_big_anextents;/* NREXT64 attr extents */ +__be16 di_nrext64_pad;/* NREXT64 unused, zero */ + } __packed; +struct { +__be32 di_nextents;/* !NREXT64 data extents */ +__be16 di_anextents;/* !NREXT64 attr extents */ + } __packed; + };union { /* * For V2 inodes and V3 inodes without NREXT64 set, this * is the number of data and attr fork extents. */ struct { __be32 di_nextents; __be16 di_anextents; } __packed; /* Number of attr fork extents if NREXT64 is set. */ struct { __be32 di_big_anextents; __be16 di_nrext64_pad; } __packed; } __packed;Quoted 70 lines
fs/xfs/libxfs/xfs_format.h +0
-0__u8 di_forkoff;/* attr fork offs, <<3 for 64b align */ __s8 di_aformat;/* format of attr fork's data */ __be32 di_dmevmask;/* DMIG event mask */ fs/xfs/libxfs/xfs_inode_buf.c +23
-4@@ -279,6 +279,25 @@ xfs_inode_to_disk_ts(return ts; } +static inline void +xfs_inode_to_disk_iext_counters ( +struct xfs_inode *ip, +struct xfs_dinode *to) +{ +if (xfs_inode_has_nrext64 (ip)) { + to->di_big_nextents =cpu_to_be64 (xfs_ifork_nextents (&ip->i_df)); + to->di_big_anextents =cpu_to_be32 (xfs_ifork_nextents (ip->i_afp)); +/* +* We might be upgrading the inode to use larger extent counters +* than was previously used. Hence zero the unused field. +*/ + to->di_nrext64_pad =cpu_to_be16 (0 ); + }else { + to->di_nextents =cpu_to_be32 (xfs_ifork_nextents (&ip->i_df)); + to->di_anextents =cpu_to_be16 (xfs_ifork_nextents (ip->i_afp)); + } +} + void xfs_inode_to_disk( struct xfs_inode *ip,@@ -296,7 +315,6 @@ xfs_inode_to_disk( to->di_projid_lo =cpu_to_be16 (ip->i_projid &0xffff ); to->di_projid_hi =cpu_to_be16 (ip->i_projid >>16 ); -memset (to->di_pad,0 ,sizeof (to->di_pad)); to->di_atime =xfs_inode_to_disk_ts (ip, inode->i_atime); to->di_mtime =xfs_inode_to_disk_ts (ip, inode->i_mtime); to->di_ctime =xfs_inode_to_disk_ts (ip, inode->i_ctime);@@ -307,8 +325,6 @@ xfs_inode_to_disk( to->di_size =cpu_to_be64 (ip->i_disk_size); to->di_nblocks =cpu_to_be64 (ip->i_nblocks); to->di_extsize =cpu_to_be32 (ip->i_extsize); - to->di_nextents =cpu_to_be32 (xfs_ifork_nextents (&ip->i_df)); - to->di_anextents =cpu_to_be16 (xfs_ifork_nextents (ip->i_afp)); to->di_forkoff = ip->i_forkoff; to->di_aformat =xfs_ifork_format (ip->i_afp); to->di_flags =cpu_to_be16 (ip->i_diflags);@@ -323,11 +339,14 @@ xfs_inode_to_disk( to->di_lsn =cpu_to_be64 (lsn);memset (to->di_pad2,0 ,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid); - to->di_flushiter =0 ; +memset (to->di_v3_pad,0 ,sizeof (to->di_v3_pad)); }else { to->di_version =2 ; to->di_flushiter =cpu_to_be16 (ip->i_flushiter); +memset (to->di_v2_pad,0 ,sizeof (to->di_v2_pad)); } + +xfs_inode_to_disk_iext_counters (ip, to); } static xfs_failaddr_t@@ -397,6 +416,24 @@ xfs_dinode_verify_forkoff(return NULL ; }fs/xfs/libxfs/xfs_inode_buf.c xfs_dinode_verify_forkoff( · line 428
Quoted 10 lines
fs/xfs/libxfs/xfs_inode_buf.c +10
-0+static xfs_failaddr_t +xfs_dinode_verify_nextents ( +struct xfs_mount *mp, +struct xfs_dinode *dip) +{ +if (xfs_dinode_has_nrext64 (dip)) { +if (!xfs_has_nrext64 (mp)) +return __this_address; +if (dip->di_nrext64_pad !=0 ) +return __this_address;Don't we need to check that:
if (xfs_dfork_data_extents(dip) > XFS_MAX_EXTCNT_DATA_FORK) return __this_address; if (xfs_dfork_attr_extents(dip) > XFS_MAX_EXTCNT_ATTR_FORK) return __this_address;here?
Quoted 9 lines
OH, the actual checking of the extent count fields is in
xfs_dinode_verify_fork, isn't it?I think that means this function exists to check the consistency of the nrext64 inode flag vs. the superblock nrext64 flag and the padding fields, right?
In which case... perhaps this should be
xfs_dinode_verify_nrext64()?
You are right. The name xfs_dinode_verify_nextents implies that we are verifying the values of extent counters. I will fix this.
Quoted 7 lines
fs/xfs/libxfs/xfs_inode_buf.c xfs_dinode_verify_forkoff( · line 431
Quoted 3 lines
fs/xfs/libxfs/xfs_inode_buf.c +3
-0+ }else { +if (dip->di_version ==3 && dip->di_big_nextents !=0 ) +return __this_address;We're using tagged unions in
xfs_dinode, then "di_big_nextents" is meaningless on an inode that doesn't have NREXT64 set. IOWs,
Quoted 7 lines
} else if (dip->di_version >= 3) { if (dip->di_v3_pad != 0) return __this_address; }(Note that I changed the type of
di_v3_padabove.)
Thanks for the suggestion. I will fix this as well.
Quoted 74 lines
Quoted 74 lines
fs/xfs/libxfs/xfs_inode_buf.c +9
-0+ } + +return NULL ; +} + xfs_failaddr_t xfs_dinode_verify( struct xfs_mount *mp,@@ -440,6 +477,10 @@ xfs_dinode_verify(if ((S_ISLNK (mode) ||S_ISDIR (mode)) && di_size ==0 )return __this_address; + fa =xfs_dinode_verify_nextents (mp, dip); +if (fa) +return fa; + nextents =xfs_dfork_data_extents (dip); nextents +=xfs_dfork_attr_extents (dip); nblocks =be64_to_cpu (dip->di_nblocks);fs/xfs/libxfs/xfs_inode_fork.h +6
-0@@ -156,6 +156,9 @@ static inline xfs_extnum_txfs_dfork_data_extents ( struct xfs_dinode *dip) { +if (xfs_dinode_has_nrext64 (dip)) +return be64_to_cpu (dip->di_big_nextents); +return be32_to_cpu (dip->di_nextents); }@@ -163,6 +166,9 @@ static inline xfs_extnum_txfs_dfork_attr_extents ( struct xfs_dinode *dip) { +if (xfs_dinode_has_nrext64 (dip)) +return be32_to_cpu (dip->di_big_anextents); +return be16_to_cpu (dip->di_anextents); }fs/xfs/libxfs/xfs_log_format.h +13
-4@@ -388,16 +388,30 @@ struct xfs_log_dinode {uint32_t di_nlink;/* number of links to file */ uint16_t di_projid_lo;/* lower part of owner's project id */ uint16_t di_projid_hi;/* higher part of owner's project id */ -uint8_t di_pad[6 ];/* unused, zeroed space */ -uint16_t di_flushiter;/* incremented on flush */ +union { +uint64_t di_big_nextents;/* NREXT64 data extents */ +uint8_t di_v3_pad[8 ];/* !NREXT64 V3 inode zeroed space */ +struct { +uint8_t di_v2_pad[6 ];/* V2 inode zeroed space */ +uint16_t di_flushiter;/* V2 inode incremented on flush */ + }; + };xfs_log_timestamp_t di_atime;/* time last accessed */ xfs_log_timestamp_t di_mtime;/* time last modified */ xfs_log_timestamp_t di_ctime;/* time created/inode modified */ xfs_fsize_t di_size;/* number of bytes in file */ xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ -uint32_t di_nextents;/* number of extents in data fork */ -uint16_t di_anextents;/* number of extents in attribute fork*/ +union { +struct { +uint32_t di_big_anextents;/* NREXT64 attr extents */ +uint16_t di_nrext64_pad;/* NREXT64 unused, zero */ + } __packed;
Quoted 9 lines
fs/xfs/libxfs/xfs_log_format.h struct xfs_log_dinode { · line 414
Quoted 5 lines
fs/xfs/libxfs/xfs_log_format.h +5
-0+struct { +uint32_t di_nextents;/* !NREXT64 data extents */ +uint16_t di_anextents;/* !NREXT64 attr extents */ + } __packed; + };I think you could apply the same transformations as I did to
xfs_dinodeabove.
Sure. I will make relevant changes to xfs_log_dinode.
Quoted 211 lines
--D
Quoted 209 lines
fs/xfs/libxfs/xfs_log_format.h +0
-0uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */ fs/xfs/xfs_inode_item.c +19
-4@@ -358,6 +358,21 @@ xfs_copy_dm_fields_to_log_dinode( } } +static inline void +xfs_inode_to_log_dinode_iext_counters ( +struct xfs_inode *ip, +struct xfs_log_dinode *to) +{ +if (xfs_inode_has_nrext64 (ip)) { + to->di_big_nextents =xfs_ifork_nextents (&ip->i_df); + to->di_big_anextents =xfs_ifork_nextents (ip->i_afp); + to->di_nrext64_pad =0 ; + }else { + to->di_nextents =xfs_ifork_nextents (&ip->i_df); + to->di_anextents =xfs_ifork_nextents (ip->i_afp); + } +} + static void xfs_inode_to_log_dinode( struct xfs_inode *ip,@@ -373,7 +388,6 @@ xfs_inode_to_log_dinode( to->di_projid_lo = ip->i_projid &0xffff ; to->di_projid_hi = ip->i_projid >>16 ; -memset (to->di_pad,0 ,sizeof (to->di_pad));memset (to->di_pad3,0 ,sizeof (to->di_pad3)); to->di_atime =xfs_inode_to_log_dinode_ts (ip, inode->i_atime); to->di_mtime =xfs_inode_to_log_dinode_ts (ip, inode->i_mtime);@@ -385,8 +399,6 @@ xfs_inode_to_log_dinode( to->di_size = ip->i_disk_size; to->di_nblocks = ip->i_nblocks; to->di_extsize = ip->i_extsize; - to->di_nextents =xfs_ifork_nextents (&ip->i_df); - to->di_anextents =xfs_ifork_nextents (ip->i_afp); to->di_forkoff = ip->i_forkoff; to->di_aformat =xfs_ifork_format (ip->i_afp); to->di_flags = ip->i_diflags;@@ -406,11 +418,14 @@ xfs_inode_to_log_dinode( to->di_lsn = lsn;memset (to->di_pad2,0 ,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &ip->i_mount->m_sb.sb_meta_uuid); - to->di_flushiter =0 ; +memset (to->di_v3_pad,0 ,sizeof (to->di_v3_pad)); }else { to->di_version =2 ; to->di_flushiter = ip->i_flushiter; +memset (to->di_v2_pad,0 ,sizeof (to->di_v2_pad)); } + +xfs_inode_to_log_dinode_iext_counters (ip, to); }/* fs/xfs/xfs_inode_item_recover.c +68
-11@@ -148,6 +148,22 @@ static inline bool xfs_log_dinode_has_nrext64(const struct xfs_log_dinode *ld) (ld->di_flags2 &XFS_DIFLAG2_NREXT64 ); } +static inline void +xfs_log_dinode_to_disk_iext_counters ( +struct xfs_log_dinode *from, +struct xfs_dinode *to) +{ +if (xfs_log_dinode_has_nrext64 (from)) { + to->di_big_nextents =cpu_to_be64 (from->di_big_nextents); + to->di_big_anextents =cpu_to_be32 (from->di_big_anextents); + to->di_nrext64_pad =cpu_to_be16 (from->di_nrext64_pad); + }else { + to->di_nextents =cpu_to_be32 (from->di_nextents); + to->di_anextents =cpu_to_be16 (from->di_anextents); + } + +} + STATIC void xfs_log_dinode_to_disk( struct xfs_log_dinode *from,@@ -164,7 +180,6 @@ xfs_log_dinode_to_disk( to->di_nlink =cpu_to_be32 (from->di_nlink); to->di_projid_lo =cpu_to_be16 (from->di_projid_lo); to->di_projid_hi =cpu_to_be16 (from->di_projid_hi); -memcpy (to->di_pad, from->di_pad,sizeof (to->di_pad)); to->di_atime =xfs_log_dinode_to_disk_ts (from, from->di_atime); to->di_mtime =xfs_log_dinode_to_disk_ts (from, from->di_mtime);@@ -173,8 +188,6 @@ xfs_log_dinode_to_disk( to->di_size =cpu_to_be64 (from->di_size); to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); - to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);@@ -192,10 +205,13 @@ xfs_log_dinode_to_disk( to->di_lsn =cpu_to_be64 (lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2));uuid_copy (&to->di_uuid, &from->di_uuid); - to->di_flushiter =0 ; +memcpy (to->di_v3_pad, from->di_v3_pad,sizeof (to->di_v3_pad)); }else { to->di_flushiter =cpu_to_be16 (from->di_flushiter); +memcpy (to->di_v2_pad, from->di_v2_pad,sizeof (to->di_v2_pad)); } + +xfs_log_dinode_to_disk_iext_counters (from, to); }STATIC int@@ -209,6 +225,8 @@ xlog_recover_inode_commit_pass2(struct xfs_mount *mp = log->l_mp;struct xfs_buf *bp;struct xfs_dinode *dip; +xfs_extnum_t nextents; +xfs_aextnum_t anextents;int len;char *src;char *dest;@@ -348,21 +366,60 @@ xlog_recover_inode_commit_pass2( goto out_release; } } -if (unlikely (ldip->di_nextents + ldip->di_anextents > ldip->di_nblocks)){ -XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(5)" , + +if (xfs_log_dinode_has_nrext64 (ldip)) { +if (!xfs_has_nrext64 (mp) || (ldip->di_nrext64_pad !=0 )) { +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(5)" , +XFS_ERRLEVEL_LOW , mp, ldip, +sizeof (*ldip)); +xfs_alert (mp, +"%s: Bad inode log record, rec ptr " PTR_FMT ", " +"dino ptr " PTR_FMT ", dino bp " PTR_FMT ", " +"ino %Ld, xfs_has_nrext64(mp) = %d, " +"ldip->di_nrext64_pad = %u" , + __func__, item, dip, bp, in_f->ilf_ino, +xfs_has_nrext64 (mp), ldip->di_nrext64_pad); + error = -EFSCORRUPTED ; + goto out_release; + } + }else { +if (ldip->di_version ==3 && ldip->di_big_nextents !=0 ) { +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(6)" , +XFS_ERRLEVEL_LOW , mp, ldip, +sizeof (*ldip)); +xfs_alert (mp, +"%s: Bad inode log record, rec ptr " PTR_FMT ", " +"dino ptr " PTR_FMT ", dino bp " PTR_FMT ", " +"ino %Ld, ldip->di_big_dextcnt = %llu" , + __func__, item, dip, bp, in_f->ilf_ino, + ldip->di_big_nextents); + error = -EFSCORRUPTED ; + goto out_release; + } + } + +if (xfs_log_dinode_has_nrext64 (ldip)) { + nextents = ldip->di_big_nextents; + anextents = ldip->di_big_anextents; + }else { + nextents = ldip->di_nextents; + anextents = ldip->di_anextents; + } + +if (unlikely (nextents + anextents > ldip->di_nblocks)) { +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(7)" ,XFS_ERRLEVEL_LOW , mp, ldip,sizeof (*ldip));xfs_alert (mp,"%s: Bad inode log record, rec ptr " PTR_FMT ", dino ptr " PTR_FMT ", " -"dino bp " PTR_FMT ", ino %Ld, total extents = %d, nblocks = %Ld" , +"dino bp " PTR_FMT ", ino %Ld, total extents = %llu, nblocks = %Ld" , __func__, item, dip, bp, in_f->ilf_ino, - ldip->di_nextents + ldip->di_anextents, - ldip->di_nblocks); + nextents + anextents, ldip->di_nblocks); error = -EFSCORRUPTED ; goto out_release; }if (unlikely (ldip->di_forkoff > mp->m_sb.sb_inodesize)) { -XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(6)" , +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(8)" ,XFS_ERRLEVEL_LOW , mp, ldip,sizeof (*ldip)); xfs_alert(mp,@@ -374,7 +431,7 @@ xlog_recover_inode_commit_pass2( } isize =xfs_log_dinode_size (mp);if (unlikely (item->ri_buf[1 ].i_len > isize)) { -XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(7)" , +XFS_CORRUPTION_ERROR ("xlog_recover_inode_pass2(9)" ,XFS_ERRLEVEL_LOW , mp, ldip,sizeof (*ldip)); xfs_alert(mp,Signature · 1 line
2.30.2
Signature · 1 line
chandan
PATCH v5 13/16 xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
In reply to Chandan Babu RThis commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has XFS_SB_FEAT_INCOMPAT_NREXT64 incompat flag set.
--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/xfs/libxfs/xfs_inode_buf.c +6 -0
@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ; +return 0 ; out_destroy_data_fork:
Signature · 1 line
2.30.2
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 14 lines · Chandan Babu R
This commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has
XFS_SB_FEAT_INCOMPAT_NREXT64incompat flag set.Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_inode_buf.c +0
-0
fs/xfs/libxfs/xfs_inode_buf.c xfs_inode_from_disk( · line 260
Quoted 8 lines
fs/xfs/libxfs/xfs_inode_buf.c +5
-0@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ;
Hmm. Last time around I asked about the oddness of updating the inode feature flags outside of a transaction, and then never responded. :( So to quote you from last time:
Quoted 9 lines
The following is the thought process behind upgrading an inode to
XFS_DIFLAG2_NREXT64when it is read from the disk,1. With support for dynamic upgrade, The extent count limits of an inode needs to be determined by checking flags present within the inode i.e. we need to satisfy self-describing metadata property. This helps tools like
xfs_repairand scrub to verify inode's extent count limits without having to refer to other metadata objects (e.g. superblock feature flags).
I think this makes an even /stronger/ argument for why this update needs to be transactional.
Quoted 5 lines
2. Upgrade when performed inside
xfs_trans_log_inode()may causexfs_iext_count_may_overflow()to return-EFBIGwhen the inode's data/attr extent count is already close to 2^31/2^15 respectively. Hence none of the file operations will be able to add new extents to a file.
Aha, there's the reason why! You're right, xfs_iext_count_may_overflow will abort the operation due to !NREXT64 before we even get a chance to log the inode.
I observe, however, that any time we call that function, we also have a transaction allocated and we hold the ILOCK on the inode being tested. *Most* of those call sites have also joined the inode to the transaction already. I wonder, is that a more appropriate place to be upgrading the inodes? Something like:
/*
* Ensure that the inode has the ability to add the specified number of
* extents. Caller must hold ILOCK_EXCL and have joined the inode to
* the transaction. Upon return, the inode will still be in this state
* upon return and the transaction will be clean.
*/
int
xfs_trans_inode_ensure_nextents(
struct xfs_trans **tpp,
struct xfs_inode *ip,
int whichfork,
int nr_to_add)
{
int error;
error = xfs_iext_count_may_overflow(ip, whichfork, nr_to_add);
if (!error)
return 0;
/*
* Try to upgrade if the extent count fields aren't large
* enough.
*/
if (!xfs_has_nrext64(ip->i_mount) ||
(ip->i_diflags2 & XFS_DIFLAG2_NREXT64))
return error;
ip->i_diflags2 |= XFS_DIFLAG2_NREXT64;
xfs_trans_log_inode(*tpp, ip, XFS_ILOG_CORE);
error = xfs_trans_roll(tpp);
if (error)
return error;
return xfs_iext_count_may_overflow(ip, whichfork, nr_to_add);
}and then the current call sites become:
error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, dblocks, rblocks, false, &tp); if (error) return error; error = xfs_trans_inode_ensure_nextents(&tp, ip, XFS_DATA_FORK, XFS_IEXT_ADD_NOSPLIT_CNT); if (error) goto out_cancel;
What do you think about that?
--D
Quoted 7 lines
fs/xfs/libxfs/xfs_inode_buf.c +1
-0+return 0 ; out_destroy_data_fork:Signature · 1 line
2.30.2
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 106 lines · Darrick J. Wong
Quoted 14 lines · Chandan Babu R
This commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has
XFS_SB_FEAT_INCOMPAT_NREXT64incompat flag set.Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_inode_buf.c +0
-0fs/xfs/libxfs/xfs_inode_buf.c xfs_inode_from_disk( · line 260
Quoted 8 lines
fs/xfs/libxfs/xfs_inode_buf.c +5
-0@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ;Hmm. Last time around I asked about the oddness of updating the inode feature flags outside of a transaction, and then never responded. :( So to quote you from last time:
Quoted 9 lines
The following is the thought process behind upgrading an inode to
XFS_DIFLAG2_NREXT64when it is read from the disk,1. With support for dynamic upgrade, The extent count limits of an inode needs to be determined by checking flags present within the inode i.e. we need to satisfy self-describing metadata property. This helps tools like
xfs_repairand scrub to verify inode's extent count limits without having to refer to other metadata objects (e.g. superblock feature flags).I think this makes an even /stronger/ argument for why this update needs to be transactional.
Quoted 5 lines
2. Upgrade when performed inside
xfs_trans_log_inode()may causexfs_iext_count_may_overflow()to return-EFBIGwhen the inode's data/attr extent count is already close to 2^31/2^15 respectively. Hence none of the file operations will be able to add new extents to a file.Aha, there's the reason why! You're right,
xfs_iext_count_may_overflowwill abort the operation due to !NREXT64 before we even get a chance to log the inode.I observe, however, that any time we call that function, we also have a transaction allocated and we hold the ILOCK on the inode being tested. *Most* of those call sites have also joined the inode to the transaction already. I wonder, is that a more appropriate place to be upgrading the inodes? Something like:
/*
* Ensure that the inode has the ability to add the specified number of
* extents. Caller must holdILOCK_EXCLand have joined the inode to
* the transaction. Upon return, the inode will still be in this state
* upon return and the transaction will be clean.
*/
int
xfs_trans_inode_ensure_nextents(
structxfs_trans**tpp,
structxfs_inode*ip,
int whichfork,
intnr_to_add){ int error; error = xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); if (!error) return 0; /* * Try to upgrade if the extent count fields aren't large * enough. */ if (!xfs_has_nrext64(ip->i_mount) || (ip->i_diflags2 & XFS_DIFLAG2_NREXT64)) return error; ip->i_diflags2 |= XFS_DIFLAG2_NREXT64; xfs_trans_log_inode(*tpp, ip, XFS_ILOG_CORE); error = xfs_trans_roll(tpp); if (error) return error; return xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); }and then the current call sites become:
error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, dblocks, rblocks, false, &tp); if (error) return error; error = xfs_trans_inode_ensure_nextents(&tp, ip, XFS_DATA_FORK, XFS_IEXT_ADD_NOSPLIT_CNT); if (error) goto out_cancel;
Quoted 2 lines
What do you think about that?
I went through all the call sites of xfs_iext_count_may_overflow() and I think that your suggestion can be implemented.
However, wouldn't the current approach suffice in terms of being functionally and logically correct? XFS_DIFLAG2_NREXT64 is set when inode is read from the disk and the first operation to log the changes made to the inode will make sure to include the new value of ip->i_diflags2. Hence we never end up in a situation where a disk inode has more than 2^31 data fork extents without having XFS_DIFLAG2_NREXT64 flag set.
But the approach described above does go against the convention of changing metadata within a transaction. Hence I will try to implement your suggestion and include it in the next version of the patchset.
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 120 lines · Chandan Babu R
Quoted 106 lines · Darrick J. Wong
Quoted 14 lines · Chandan Babu R
This commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has
XFS_SB_FEAT_INCOMPAT_NREXT64incompat flag set.Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_inode_buf.c +0
-0fs/xfs/libxfs/xfs_inode_buf.c xfs_inode_from_disk( · line 260
Quoted 8 lines
fs/xfs/libxfs/xfs_inode_buf.c +5
-0@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ;Hmm. Last time around I asked about the oddness of updating the inode feature flags outside of a transaction, and then never responded. :( So to quote you from last time:
Quoted 9 lines
The following is the thought process behind upgrading an inode to
XFS_DIFLAG2_NREXT64when it is read from the disk,1. With support for dynamic upgrade, The extent count limits of an inode needs to be determined by checking flags present within the inode i.e. we need to satisfy self-describing metadata property. This helps tools like
xfs_repairand scrub to verify inode's extent count limits without having to refer to other metadata objects (e.g. superblock feature flags).I think this makes an even /stronger/ argument for why this update needs to be transactional.
Quoted 5 lines
2. Upgrade when performed inside
xfs_trans_log_inode()may causexfs_iext_count_may_overflow()to return-EFBIGwhen the inode's data/attr extent count is already close to 2^31/2^15 respectively. Hence none of the file operations will be able to add new extents to a file.Aha, there's the reason why! You're right,
xfs_iext_count_may_overflowwill abort the operation due to !NREXT64 before we even get a chance to log the inode.I observe, however, that any time we call that function, we also have a transaction allocated and we hold the ILOCK on the inode being tested. *Most* of those call sites have also joined the inode to the transaction already. I wonder, is that a more appropriate place to be upgrading the inodes? Something like:
/*
* Ensure that the inode has the ability to add the specified number of
* extents. Caller must holdILOCK_EXCLand have joined the inode to
* the transaction. Upon return, the inode will still be in this state
* upon return and the transaction will be clean.
*/
int
xfs_trans_inode_ensure_nextents(
structxfs_trans**tpp,
structxfs_inode*ip,
int whichfork,
intnr_to_add){ int error; error = xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); if (!error) return 0; /* * Try to upgrade if the extent count fields aren't large * enough. */ if (!xfs_has_nrext64(ip->i_mount) || (ip->i_diflags2 & XFS_DIFLAG2_NREXT64)) return error; ip->i_diflags2 |= XFS_DIFLAG2_NREXT64; xfs_trans_log_inode(*tpp, ip, XFS_ILOG_CORE); error = xfs_trans_roll(tpp); if (error) return error; return xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); }and then the current call sites become:
error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, dblocks, rblocks, false, &tp); if (error) return error; error = xfs_trans_inode_ensure_nextents(&tp, ip, XFS_DATA_FORK, XFS_IEXT_ADD_NOSPLIT_CNT); if (error) goto out_cancel;Quoted 2 lines
What do you think about that?
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.However, wouldn't the current approach suffice in terms of being functionally and logically correct?
XFS_DIFLAG2_NREXT64is set when inode is read from the disk and the first operation to log the changes made to the inode will make sure to include the new value ofip->i_diflags2. Hence we never end up in a situation where a disk inode has more than 2^31 data fork extents without havingXFS_DIFLAG2_NREXT64flag set.
Quoted 3 lines
But the approach described above does go against the convention of changing metadata within a transaction. Hence I will try to implement your suggestion and include it in the next version of the patchset.
Ok, that sounds good. :)
--D
Quoted 2 lines
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 108 lines · Darrick J. Wong
Quoted 107 lines · Chandan Babu R
Quoted 106 lines · Darrick J. Wong
Quoted 14 lines · Chandan Babu R
This commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has
XFS_SB_FEAT_INCOMPAT_NREXT64incompat flag set.Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_inode_buf.c +0
-0fs/xfs/libxfs/xfs_inode_buf.c xfs_inode_from_disk( · line 260
Quoted 8 lines
fs/xfs/libxfs/xfs_inode_buf.c +5
-0@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ;Hmm. Last time around I asked about the oddness of updating the inode feature flags outside of a transaction, and then never responded. :( So to quote you from last time:
Quoted 9 lines
The following is the thought process behind upgrading an inode to
XFS_DIFLAG2_NREXT64when it is read from the disk,1. With support for dynamic upgrade, The extent count limits of an inode needs to be determined by checking flags present within the inode i.e. we need to satisfy self-describing metadata property. This helps tools like
xfs_repairand scrub to verify inode's extent count limits without having to refer to other metadata objects (e.g. superblock feature flags).I think this makes an even /stronger/ argument for why this update needs to be transactional.
Quoted 5 lines
2. Upgrade when performed inside
xfs_trans_log_inode()may causexfs_iext_count_may_overflow()to return-EFBIGwhen the inode's data/attr extent count is already close to 2^31/2^15 respectively. Hence none of the file operations will be able to add new extents to a file.Aha, there's the reason why! You're right,
xfs_iext_count_may_overflowwill abort the operation due to !NREXT64 before we even get a chance to log the inode.I observe, however, that any time we call that function, we also have a transaction allocated and we hold the ILOCK on the inode being tested. *Most* of those call sites have also joined the inode to the transaction already. I wonder, is that a more appropriate place to be upgrading the inodes? Something like:
/*
* Ensure that the inode has the ability to add the specified number of
* extents. Caller must holdILOCK_EXCLand have joined the inode to
* the transaction. Upon return, the inode will still be in this state
* upon return and the transaction will be clean.
*/
int
xfs_trans_inode_ensure_nextents(
structxfs_trans**tpp,
structxfs_inode*ip,
int whichfork,
intnr_to_add){ int error; error = xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); if (!error) return 0; /* * Try to upgrade if the extent count fields aren't large * enough. */ if (!xfs_has_nrext64(ip->i_mount) || (ip->i_diflags2 & XFS_DIFLAG2_NREXT64)) return error; ip->i_diflags2 |= XFS_DIFLAG2_NREXT64; xfs_trans_log_inode(*tpp, ip, XFS_ILOG_CORE); error = xfs_trans_roll(tpp); if (error) return error; return xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); }and then the current call sites become:
error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, dblocks, rblocks, false, &tp); if (error) return error; error = xfs_trans_inode_ensure_nextents(&tp, ip, XFS_DATA_FORK, XFS_IEXT_ADD_NOSPLIT_CNT); if (error) goto out_cancel;
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
What do you think about that?
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.
Sorry, I missed/overlooked the usage of xfs_iext_count_may_overflow() in xfs_symlink().
Just after invoking xfs_iext_count_may_overflow(), we execute the following steps,
1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll() invoked in step 4 would mean that we cannot move step 6 to occur before step 1 since xfs_trans_roll would unlock the inode by executing xfs_inode_item_committing().
xfs_create() has a similar flow.
Hence, I think we should retain the current logic of setting XFS_DIFLAG2_NREXT64 just after reading the inode from the disk.
Quoted 14 lines
Quoted 11 lines
However, wouldn't the current approach suffice in terms of being functionally and logically correct?
XFS_DIFLAG2_NREXT64is set when inode is read from the disk and the first operation to log the changes made to the inode will make sure to include the new value ofip->i_diflags2. Hence we never end up in a situation where a disk inode has more than 2^31 data fork extents without havingXFS_DIFLAG2_NREXT64flag set.But the approach described above does go against the convention of changing metadata within a transaction. Hence I will try to implement your suggestion and include it in the next version of the patchset.
Ok, that sounds good. :)
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 134 lines · Chandan Babu R
Quoted 108 lines · Darrick J. Wong
Quoted 107 lines · Chandan Babu R
Quoted 106 lines · Darrick J. Wong
Quoted 14 lines · Chandan Babu R
This commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has
XFS_SB_FEAT_INCOMPAT_NREXT64incompat flag set.Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_inode_buf.c +0
-0fs/xfs/libxfs/xfs_inode_buf.c xfs_inode_from_disk( · line 260
Quoted 8 lines
fs/xfs/libxfs/xfs_inode_buf.c +5
-0@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ;Hmm. Last time around I asked about the oddness of updating the inode feature flags outside of a transaction, and then never responded. :( So to quote you from last time:
Quoted 9 lines
The following is the thought process behind upgrading an inode to
XFS_DIFLAG2_NREXT64when it is read from the disk,1. With support for dynamic upgrade, The extent count limits of an inode needs to be determined by checking flags present within the inode i.e. we need to satisfy self-describing metadata property. This helps tools like
xfs_repairand scrub to verify inode's extent count limits without having to refer to other metadata objects (e.g. superblock feature flags).I think this makes an even /stronger/ argument for why this update needs to be transactional.
Quoted 5 lines
2. Upgrade when performed inside
xfs_trans_log_inode()may causexfs_iext_count_may_overflow()to return-EFBIGwhen the inode's data/attr extent count is already close to 2^31/2^15 respectively. Hence none of the file operations will be able to add new extents to a file.Aha, there's the reason why! You're right,
xfs_iext_count_may_overflowwill abort the operation due to !NREXT64 before we even get a chance to log the inode.I observe, however, that any time we call that function, we also have a transaction allocated and we hold the ILOCK on the inode being tested. *Most* of those call sites have also joined the inode to the transaction already. I wonder, is that a more appropriate place to be upgrading the inodes? Something like:
/*
* Ensure that the inode has the ability to add the specified number of
* extents. Caller must holdILOCK_EXCLand have joined the inode to
* the transaction. Upon return, the inode will still be in this state
* upon return and the transaction will be clean.
*/
int
xfs_trans_inode_ensure_nextents(
structxfs_trans**tpp,
structxfs_inode*ip,
int whichfork,
intnr_to_add){ int error; error = xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); if (!error) return 0; /* * Try to upgrade if the extent count fields aren't large * enough. */ if (!xfs_has_nrext64(ip->i_mount) || (ip->i_diflags2 & XFS_DIFLAG2_NREXT64)) return error; ip->i_diflags2 |= XFS_DIFLAG2_NREXT64; xfs_trans_log_inode(*tpp, ip, XFS_ILOG_CORE); error = xfs_trans_roll(tpp); if (error) return error; return xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); }and then the current call sites become:
error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, dblocks, rblocks, false, &tp); if (error) return error; error = xfs_trans_inode_ensure_nextents(&tp, ip, XFS_DATA_FORK, XFS_IEXT_ADD_NOSPLIT_CNT); if (error) goto out_cancel;Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
What do you think about that?
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
Quoted 8 lines
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.
File creation shouldn't ever run into problems with xfs_iext_count_may_overflow because (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.
--D
Quoted 17 lines
Quoted 14 lines
Quoted 11 lines
However, wouldn't the current approach suffice in terms of being functionally and logically correct?
XFS_DIFLAG2_NREXT64is set when inode is read from the disk and the first operation to log the changes made to the inode will make sure to include the new value ofip->i_diflags2. Hence we never end up in a situation where a disk inode has more than 2^31 data fork extents without havingXFS_DIFLAG2_NREXT64flag set.But the approach described above does go against the convention of changing metadata within a transaction. Hence I will try to implement your suggestion and include it in the next version of the patchset.
Ok, that sounds good. :)
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 141 lines · Darrick J. Wong
Quoted 140 lines · Chandan Babu R
Quoted 108 lines · Darrick J. Wong
Quoted 107 lines · Chandan Babu R
Quoted 106 lines · Darrick J. Wong
Quoted 14 lines · Chandan Babu R
This commit upgrades inodes to use 64-bit extent counters when they are read from disk. Inodes are upgraded only when the filesystem instance has
XFS_SB_FEAT_INCOMPAT_NREXT64incompat flag set.Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_inode_buf.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_inode_buf.c +0
-0fs/xfs/libxfs/xfs_inode_buf.c xfs_inode_from_disk( · line 260
Quoted 8 lines
fs/xfs/libxfs/xfs_inode_buf.c +5
-0@@ -253,6 +253,12 @@ xfs_inode_from_disk( }if (xfs_is_reflink_inode (ip))xfs_ifork_init_cow (ip); + +if ((from->di_version ==3 ) && +xfs_has_nrext64 (ip->i_mount) && + !xfs_dinode_has_nrext64 (from)) + ip->i_diflags2 |=XFS_DIFLAG2_NREXT64 ;Hmm. Last time around I asked about the oddness of updating the inode feature flags outside of a transaction, and then never responded. :( So to quote you from last time:
Quoted 9 lines
The following is the thought process behind upgrading an inode to
XFS_DIFLAG2_NREXT64when it is read from the disk,1. With support for dynamic upgrade, The extent count limits of an inode needs to be determined by checking flags present within the inode i.e. we need to satisfy self-describing metadata property. This helps tools like
xfs_repairand scrub to verify inode's extent count limits without having to refer to other metadata objects (e.g. superblock feature flags).I think this makes an even /stronger/ argument for why this update needs to be transactional.
Quoted 5 lines
2. Upgrade when performed inside
xfs_trans_log_inode()may causexfs_iext_count_may_overflow()to return-EFBIGwhen the inode's data/attr extent count is already close to 2^31/2^15 respectively. Hence none of the file operations will be able to add new extents to a file.Aha, there's the reason why! You're right,
xfs_iext_count_may_overflowwill abort the operation due to !NREXT64 before we even get a chance to log the inode.I observe, however, that any time we call that function, we also have a transaction allocated and we hold the ILOCK on the inode being tested. *Most* of those call sites have also joined the inode to the transaction already. I wonder, is that a more appropriate place to be upgrading the inodes? Something like:
/*
* Ensure that the inode has the ability to add the specified number of
* extents. Caller must holdILOCK_EXCLand have joined the inode to
* the transaction. Upon return, the inode will still be in this state
* upon return and the transaction will be clean.
*/
int
xfs_trans_inode_ensure_nextents(
structxfs_trans**tpp,
structxfs_inode*ip,
int whichfork,
intnr_to_add){ int error; error = xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); if (!error) return 0; /* * Try to upgrade if the extent count fields aren't large * enough. */ if (!xfs_has_nrext64(ip->i_mount) || (ip->i_diflags2 & XFS_DIFLAG2_NREXT64)) return error; ip->i_diflags2 |= XFS_DIFLAG2_NREXT64; xfs_trans_log_inode(*tpp, ip, XFS_ILOG_CORE); error = xfs_trans_roll(tpp); if (error) return error; return xfs_iext_count_may_overflow(ip, whichfork, nr_to_add); }and then the current call sites become:
error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, dblocks, rblocks, false, &tp); if (error) return error; error = xfs_trans_inode_ensure_nextents(&tp, ip, XFS_DATA_FORK, XFS_IEXT_ADD_NOSPLIT_CNT); if (error) goto out_cancel;Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
What do you think about that?
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.
Quoted 9 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field. Similary, xfs_create() can cause data fork extent count field of the parent directory to overflow.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
However, wouldn't the current approach suffice in terms of being functionally and logically correct?
XFS_DIFLAG2_NREXT64is set when inode is read from the disk and the first operation to log the changes made to the inode will make sure to include the new value ofip->i_diflags2. Hence we never end up in a situation where a disk inode has more than 2^31 data fork extents without havingXFS_DIFLAG2_NREXT64flag set.But the approach described above does go against the convention of changing metadata within a transaction. Hence I will try to implement your suggestion and include it in the next version of the patchset.
Ok, that sounds good. :)
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 43 lines · Chandan Babu R
Quoted 33 lines · Darrick J. Wong
Quoted 32 lines · Chandan Babu R
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Chandan Babu R
Quoted 1 line · Darrick J. Wong
On Fri, Jan 21, 2022 at 10:48:54AM +0530, Chandan Babu R wrote:
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Quoted 8 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.
Quoted 4 lines
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field.
I dont' think that can happen. A directory is limited in size to 3 segments of 32GB each. In reality, only the data segment can ever reach 32GB as both the dabtree and free space segments are just compact indexes of the contents of the 32GB data segment.
Hence a directory is never likely to reach more than about 40GB of blocks which is nowhere near large enough to overflowing a 32 bit extent count field.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 49 lines · Dave Chinner
Quoted 43 lines · Chandan Babu R
Quoted 33 lines · Darrick J. Wong
Quoted 32 lines · Chandan Babu R
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Chandan Babu R
Quoted 1 line · Darrick J. Wong
On Fri, Jan 21, 2022 at 10:48:54AM +0530, Chandan Babu R wrote:
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Quoted 8 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.Quoted 4 lines
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field.
Quoted 8 lines
I dont' think that can happen. A directory is limited in size to 3 segments of 32GB each. In reality, only the data segment can ever reach 32GB as both the dabtree and free space segments are just compact indexes of the contents of the 32GB data segment.
Hence a directory is never likely to reach more than about 40GB of blocks which is nowhere near large enough to overflowing a 32 bit extent count field.
I think you are right.
The maximum file size that can be represented by the data fork extent counter in the worst case occurs when all extents are 1 block in size and each block is 1k in size.
With 1k byte sized blocks, a file can reach upto,
1k * (2^31) = 2048 GB
This is much larger than the asymptotic maximum size of a directory i.e. 32GB * 3 = 96GB.
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 65 lines · Chandan Babu R
Quoted 49 lines · Dave Chinner
Quoted 43 lines · Chandan Babu R
Quoted 33 lines · Darrick J. Wong
Quoted 32 lines · Chandan Babu R
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Chandan Babu R
Quoted 1 line · Darrick J. Wong
On Fri, Jan 21, 2022 at 10:48:54AM +0530, Chandan Babu R wrote:
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Quoted 8 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.Quoted 4 lines
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field.
Quoted 8 lines
I dont' think that can happen. A directory is limited in size to 3 segments of 32GB each. In reality, only the data segment can ever reach 32GB as both the dabtree and free space segments are just compact indexes of the contents of the 32GB data segment.
Hence a directory is never likely to reach more than about 40GB of blocks which is nowhere near large enough to overflowing a 32 bit extent count field.
I think you are right.
The maximum file size that can be represented by the data fork extent counter in the worst case occurs when all extents are 1 block in size and each block is 1k in size.
Quoted 5 lines
With 1k byte sized blocks, a file can reach upto,
1k * (2^31) = 2048 GBThis is much larger than the asymptotic maximum size of a directory i.e. 32GB * 3 = 96GB.
Also, I think I should remove extent count overflow checks performed in the following functions,
xfs_create()xfs_rename()xfs_link()xfs_symlink()xfs_bmap_del_extent_real()
... Since they do not accomplish anything.
Please let me know your views on this.
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 66 lines · Chandan Babu R
Quoted 65 lines · Chandan Babu R
Quoted 49 lines · Dave Chinner
Quoted 43 lines · Chandan Babu R
Quoted 33 lines · Darrick J. Wong
Quoted 32 lines · Chandan Babu R
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Chandan Babu R
Quoted 1 line · Darrick J. Wong
On Fri, Jan 21, 2022 at 10:48:54AM +0530, Chandan Babu R wrote:
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Quoted 8 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.Quoted 4 lines
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field.
Quoted 8 lines
I dont' think that can happen. A directory is limited in size to 3 segments of 32GB each. In reality, only the data segment can ever reach 32GB as both the dabtree and free space segments are just compact indexes of the contents of the 32GB data segment.
Hence a directory is never likely to reach more than about 40GB of blocks which is nowhere near large enough to overflowing a 32 bit extent count field.
I think you are right.
The maximum file size that can be represented by the data fork extent counter in the worst case occurs when all extents are 1 block in size and each block is 1k in size.
Quoted 5 lines
Quoted 5 lines
With 1k byte sized blocks, a file can reach upto,
1k * (2^31) = 2048 GBThis is much larger than the asymptotic maximum size of a directory i.e. 32GB * 3 = 96GB.
The downside of getting rid of the checks for directories is that we won't be able to use the error injection knob that limits all forks to 10 extents max to see what happens when that part of directory expansion fails. But if it makes it easier to handle nrext64, then that's probably a good enough reason to forego that.
Quoted 7 lines
Also, I think I should remove extent count overflow checks performed in the following functions,
xfs_create()xfs_rename()xfs_link()xfs_symlink()
Those are probably ok to remove the checks.
Quoted 1 line
xfs_bmap_del_extent_real()
Not sure about this one, since it actually /can/ result in more extents.
--D
Quoted 6 lines
... Since they do not accomplish anything.
Please let me know your views on this.
Signature · 1 line
chandan
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 70 lines · Darrick J. Wong
Quoted 69 lines · Chandan Babu R
Quoted 68 lines · Chandan Babu R
Quoted 49 lines · Dave Chinner
Quoted 43 lines · Chandan Babu R
Quoted 33 lines · Darrick J. Wong
Quoted 32 lines · Chandan Babu R
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Chandan Babu R
Quoted 1 line · Darrick J. Wong
On Fri, Jan 21, 2022 at 10:48:54AM +0530, Chandan Babu R wrote:
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Quoted 8 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.Quoted 4 lines
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field.
Quoted 8 lines
I dont' think that can happen. A directory is limited in size to 3 segments of 32GB each. In reality, only the data segment can ever reach 32GB as both the dabtree and free space segments are just compact indexes of the contents of the 32GB data segment.
Hence a directory is never likely to reach more than about 40GB of blocks which is nowhere near large enough to overflowing a 32 bit extent count field.
I think you are right.
The maximum file size that can be represented by the data fork extent counter in the worst case occurs when all extents are 1 block in size and each block is 1k in size.
With 1k byte sized blocks, a file can reach upto,
1k * (2^31) = 2048 GB
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
This is much larger than the asymptotic maximum size of a directory i.e. 32GB * 3 = 96GB.
The downside of getting rid of the checks for directories is that we won't be able to use the error injection knob that limits all forks to 10 extents max to see what happens when that part of directory expansion fails. But if it makes it easier to handle nrext64, then that's probably a good enough reason to forego that.
If you want error injection to do that, add explicit error injection to the directory code.
Quoted 3 lines
Quoted 1 line
xfs_bmap_del_extent_real()Not sure about this one, since it actually /can/ result in more extents.
Yup, unlikely to ever trigger, but still necessary for correctness.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH V5 13/16] xfs: Conditionally upgrade existing inodes to use 64-bit extent counters
Quoted 74 lines · Dave Chinner
Quoted 73 lines · Darrick J. Wong
Quoted 71 lines · Chandan Babu R
Quoted 70 lines · Chandan Babu R
Quoted 49 lines · Dave Chinner
Quoted 43 lines · Chandan Babu R
Quoted 33 lines · Darrick J. Wong
Quoted 32 lines · Chandan Babu R
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Chandan Babu R
Quoted 1 line · Darrick J. Wong
On Fri, Jan 21, 2022 at 10:48:54AM +0530, Chandan Babu R wrote:
I went through all the call sites of
xfs_iext_count_may_overflow()and I think that your suggestion can be implemented.Sorry, I missed/overlooked the usage of
xfs_iext_count_may_overflow()inxfs_symlink().Just after invoking
xfs_iext_count_may_overflow(), we execute the following steps,1. Allocate inode chunk
2. Initialize inode chunk.
3. Insert record into inobt/finobt.
4. Roll the transaction.
5. Allocate ondisk inode.
6. Add directory inode to transaction.
7. Allocate blocks to store symbolic link path name.
8. Log symlink's inode (data fork contains block mappings).
9. Log data blocks containing symbolic link path name.
10. Add name to directory and log directory's blocks.
11. Log directory inode.
12. Commit transaction.
xfs_trans_roll()invoked in step 4 would mean that we cannot move step 6 to occur before step 1 sincexfs_trans_rollwould unlock the inode by executingxfs_inode_item_committing().
xfs_create()has a similar flow.Quoted 8 lines
Quoted 2 lines
Hence, I think we should retain the current logic of setting
XFS_DIFLAG2_NREXT64just after reading the inode from the disk.File creation shouldn't ever run into problems with
xfs_iext_count_may_overflowbecause (a) only symlinks get created with mapped blocks, and never more than two; and (b) we always set NREXT64 (the inode flag) on new files if NREXT64 (the superblock feature bit) is enabled, so a newly created file will never require upgrading.Quoted 4 lines
The inode representing the symbolic link being created cannot overflow its data fork extent count field. However, the inode representing the directory inside which the symbolic link entry is being created, might overflow its data fork extent count field.
Quoted 8 lines
I dont' think that can happen. A directory is limited in size to 3 segments of 32GB each. In reality, only the data segment can ever reach 32GB as both the dabtree and free space segments are just compact indexes of the contents of the 32GB data segment.
Hence a directory is never likely to reach more than about 40GB of blocks which is nowhere near large enough to overflowing a 32 bit extent count field.
I think you are right.
The maximum file size that can be represented by the data fork extent counter in the worst case occurs when all extents are 1 block in size and each block is 1k in size.
With 1k byte sized blocks, a file can reach upto,
1k * (2^31) = 2048 GBThis is much larger than the asymptotic maximum size of a directory i.e. 32GB * 3 = 96GB.
Quoted 8 lines
Quoted 5 lines
The downside of getting rid of the checks for directories is that we won't be able to use the error injection knob that limits all forks to 10 extents max to see what happens when that part of directory expansion fails. But if it makes it easier to handle nrext64, then that's probably a good enough reason to forego that.
If you want error injection to do that, add explicit error injection to the directory code.
The transaction might already be dirty before entering the directory code (e.g. xfs_dir_createname()). In this case, an error return from xfs_iext_count_may_overflow() will cause the filesystem to be shut down.
On the other hand, removing calls to xfs_iext_count_may_overflow() from the previously listed directory functions would result in the error injection knob to not work for directories. This would require us to delete xfs/533 test.
Leaving the current invocations of xfs_iext_count_may_overflow() in their respective locations would mean that they are essentially no-ops for functions which manipulate directories. However, with functions like xfs_symlink() and xfs_create(), I wouldn't be able to add the inode to the transaction before invoking xfs_iext_count_may_overflow() because this leads to inode being unlocked when rolling the transaction.
Therefore I think we should not change the current code flow w.r.t to functions associated with directory entry manipulation. i.e.
1. Let xfs_iext_count_may_overflow() continue to be no-op w.r.t directory manipulation.
2. Since xfs_iext_count_may_overflow() is a no-op, there is no need to move "add inode to transaction" code to occur before invoking xfs_iext_count_may_overflow().
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
xfs_bmap_del_extent_real()Not sure about this one, since it actually /can/ result in more extents.
Yup, unlikely to ever trigger, but still necessary for correctness.
Signature · 1 line
chandan
PATCH v5 14/16 xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters
In reply to Chandan Babu RThe following changes are made to enable userspace to obtain 64-bit extent counters,
1. Carve out a new 64-bit field xfs_bulkstat->bs_extents64 from xfs_bulkstat->bs_pad[] to hold 64-bit extent counter.
2. Define the new flag XFS_BULK_IREQ_BULKSTAT for userspace to indicate that it is capable of receiving 64-bit extent counters.
--- fs/xfs/libxfs/xfs_fs.h | 12 ++++++++---- fs/xfs/xfs_ioctl.c | 3 +++ fs/xfs/xfs_itable.c | 27 +++++++++++++++++++++++++-- fs/xfs/xfs_itable.h | 7 ++++++- fs/xfs/xfs_iwalk.h | 7 +++++-- 5 files changed, 47 insertions(+), 9 deletions(-)
fs/xfs/libxfs/xfs_fs.h +8 -4
@@ -393,7 +393,7 @@ struct xfs_bulkstat {uint32_t bs_extsize_blks;/* extent size hint, blocks */ uint32_t bs_nlink;/* number of links */ -uint32_t bs_extents;/* number of extents */ +uint32_t bs_extents;/* 32-bit data fork extent counter */ uint32_t bs_aextents;/* attribute number of extents */ uint16_t bs_version;/* structure version */ uint16_t bs_forkoff;/* inode fork offset in bytes */
@@ -402,8 +402,9 @@ struct xfs_bulkstat {uint16_t bs_checked;/* checked inode metadata */ uint16_t bs_mode;/* type and mode */ uint16_t bs_pad2;/* zeroed */ +uint64_t bs_extents64;/* 64-bit data fork extent counter */ -uint64_t bs_pad[7 ];/* zeroed */ +uint64_t bs_pad[6 ];/* zeroed */ };#define XFS_BULKSTAT_VERSION_V1 (1)
@@ -484,8 +485,11 @@ struct xfs_bulk_ireq {*/ #define XFS_BULK_IREQ_SPECIAL (1 << 1) -#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ - XFS_BULK_IREQ_SPECIAL) +#define XFS_BULK_IREQ_NREXT64 (1 << 2) + +#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ + XFS_BULK_IREQ_SPECIAL | \ + XFS_BULK_IREQ_NREXT64)/* Operate on the root directory inode. */ #define XFS_BULK_IREQ_SPECIAL_ROOT (1)
fs/xfs/xfs_ioctl.c +3 -0
@@ -893,6 +893,9 @@ xfs_bulk_ireq_setup(if (XFS_INO_TO_AGNO (mp, breq->startino) >= mp->m_sb.sb_agcount)return -ECANCELED ; +if (hdr->flags &XFS_BULK_IREQ_NREXT64 ) + breq->flags |=XFS_IBULK_NREXT64 ; +return 0 ; }
fs/xfs/xfs_itable.c +25 -2
@@ -20,6 +20,7 @@#include "xfs_icache.h" #include "xfs_health.h" #include "xfs_trans.h" +#include "xfs_errortag.h" /* * Bulk Stat
@@ -64,6 +65,7 @@ xfs_bulkstat_one_int(struct xfs_inode *ip;/* incore inode pointer */ struct inode *inode;struct xfs_bulkstat *buf = bc->buf; +xfs_extnum_t nextents;int error = -EINVAL ;if (xfs_internal_inum (mp, ino))
@@ -102,7 +104,27 @@ xfs_bulkstat_one_int( buf->bs_xflags =xfs_ip2xflags (ip); buf->bs_extsize_blks = ip->i_extsize; - buf->bs_extents =xfs_ifork_nextents (&ip->i_df); + + nextents =xfs_ifork_nextents (&ip->i_df); +if (!(bc->breq->flags &XFS_IBULK_NREXT64 )) { +xfs_extnum_t max_nextents =XFS_MAX_EXTCNT_DATA_FORK_OLD ; + +if (unlikely (XFS_TEST_ERROR (false, mp, +XFS_ERRTAG_REDUCE_MAX_IEXTENTS ))) + max_nextents =10 ; + +if (nextents > max_nextents) { +xfs_iunlock (ip,XFS_ILOCK_SHARED ); +xfs_irele (ip); + error = -EOVERFLOW ; + goto out; + } + + buf->bs_extents = nextents; + }else { + buf->bs_extents64 = nextents; + } +xfs_bulkstat_health (ip, buf); buf->bs_aextents =xfs_ifork_nextents (ip->i_afp); buf->bs_forkoff =XFS_IFORK_BOFF (ip);
@@ -279,7 +301,8 @@ xfs_bulkstat(if (error) goto out; - error =xfs_iwalk (breq->mp, tp, breq->startino, breq->flags, + error =xfs_iwalk (breq->mp, tp, breq->startino, + breq->flags &XFS_IBULK_IWALK_MASK , xfs_bulkstat_iwalk, breq->icount, &bc);xfs_trans_cancel (tp); out:
fs/xfs/xfs_itable.h +6 -1
@@ -13,12 +13,17 @@ struct xfs_ibulk {xfs_ino_t startino;/* start with this inode */ unsigned int icount;/* number of elements in ubuffer */ unsigned int ocount;/* number of records returned */ -unsigned int flags;/* see XFS_IBULK_FLAG_* */ +unsigned long long flags;/* see XFS_IBULK_FLAG_* */ };/* Only iterate within the same AG as startino */ #define XFS_IBULK_SAME_AG (XFS_IWALK_SAME_AG) +#define XFS_IBULK_ONLY_OFFSET 32 +#define XFS_IBULK_IWALK_MASK ((1ULL << XFS_IBULK_ONLY_OFFSET) - 1) + +#define XFS_IBULK_NREXT64 (1ULL << XFS_IBULK_ONLY_OFFSET) +/* * Advance the user buffer pointer by one record of the given size. If the * buffer is now full, return the appropriate error code.
fs/xfs/xfs_iwalk.h +5 -2
@@ -26,9 +26,12 @@ int xfs_iwalk_threaded(struct xfs_mount *mp, xfs_ino_t startino, unsigned int inode_records, bool poll, void *data);/* Only iterate inodes within the same AG as @startino. */ -#define XFS_IWALK_SAME_AG (0x1) +#define XFS_IWALK_SAME_AG (1 << 0) -#define XFS_IWALK_FLAGS_ALL (XFS_IWALK_SAME_AG) +#define XFS_IWALK_NREXT64 (1 << 1) + +#define XFS_IWALK_FLAGS_ALL (XFS_IWALK_SAME_AG | \ + XFS_IWALK_NREXT64)/* Walk all inode btree records in the filesystem starting from @startino. */ typedef int (*xfs_inobt_walk_fn)(struct xfs_mount *mp, struct xfs_trans *tp,
Signature · 1 line
2.30.2
Re: [PATCH V5 14/16] xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters
Quoted 42 lines · Chandan Babu R
The following changes are made to enable userspace to obtain 64-bit extent counters,
1. Carve out a new 64-bit fieldxfs_bulkstat->bs_extents64from xfs_bulkstat->bs_pad[] to hold 64-bit extent counter.
2. Define the new flagXFS_BULK_IREQ_BULKSTATfor userspace to indicate that it is capable of receiving 64-bit extent counters.Suggested-by Darrick J. Wong <[email protected]>Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_fs.h | 12 ++++++++---- fs/xfs/xfs_ioctl.c | 3 +++ fs/xfs/xfs_itable.c | 27 +++++++++++++++++++++++++-- fs/xfs/xfs_itable.h | 7 ++++++- fs/xfs/xfs_iwalk.h | 7 +++++-- 5 files changed, 47 insertions(+), 9 deletions(-)fs/xfs/libxfs/xfs_fs.h +3
-2@@ -393,7 +393,7 @@ struct xfs_bulkstat {uint32_t bs_extsize_blks;/* extent size hint, blocks */ uint32_t bs_nlink;/* number of links */ -uint32_t bs_extents;/* number of extents */ +uint32_t bs_extents;/* 32-bit data fork extent counter */ uint32_t bs_aextents;/* attribute number of extents */ uint16_t bs_version;/* structure version */ uint16_t bs_forkoff;/* inode fork offset in bytes */ @@ -402,8 +402,9 @@ struct xfs_bulkstat {uint16_t bs_checked;/* checked inode metadata */ uint16_t bs_mode;/* type and mode */ uint16_t bs_pad2;/* zeroed */ +uint64_t bs_extents64;/* 64-bit data fork extent counter */ -uint64_t bs_pad[7 ];/* zeroed */ +uint64_t bs_pad[6 ];/* zeroed */ };#define XFS_BULKSTAT_VERSION_V1 (1)
fs/xfs/libxfs/xfs_fs.h struct xfs_bulk_ireq { · line 488
Quoted 6 lines
fs/xfs/libxfs/xfs_fs.h +1
-2@@ -484,8 +485,11 @@ struct xfs_bulk_ireq {*/ #define XFS_BULK_IREQ_SPECIAL (1 << 1) -#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ - XFS_BULK_IREQ_SPECIAL) +#define XFS_BULK_IREQ_NREXT64 (1 << 2)
This needs a comment specifying the behavior of this flag.
If the flag is set and the data fork extent count fits in both fields, will they both be filled out?
If the flag is set but the data fork extent count only fits in bs_extents64, what will be written to bs_extents?
If the flag is not set and the data fork extent count won't fit in bs_extents, do we return an error value? Fill it with garbage?
Quoted 71 lines
fs/xfs/libxfs/xfs_fs.h +4
-0+ +#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ + XFS_BULK_IREQ_SPECIAL | \ + XFS_BULK_IREQ_NREXT64)/* Operate on the root directory inode. */ #define XFS_BULK_IREQ_SPECIAL_ROOT (1)fs/xfs/xfs_ioctl.c +3
-0@@ -893,6 +893,9 @@ xfs_bulk_ireq_setup(if (XFS_INO_TO_AGNO (mp, breq->startino) >= mp->m_sb.sb_agcount)return -ECANCELED ; +if (hdr->flags &XFS_BULK_IREQ_NREXT64 ) + breq->flags |=XFS_IBULK_NREXT64 ; +return 0 ; }fs/xfs/xfs_itable.c +23
-1@@ -20,6 +20,7 @@#include "xfs_icache.h" #include "xfs_health.h" #include "xfs_trans.h" +#include "xfs_errortag.h" /* * Bulk Stat @@ -64,6 +65,7 @@ xfs_bulkstat_one_int(struct xfs_inode *ip;/* incore inode pointer */ struct inode *inode;struct xfs_bulkstat *buf = bc->buf; +xfs_extnum_t nextents;int error = -EINVAL ;if (xfs_internal_inum (mp, ino))@@ -102,7 +104,27 @@ xfs_bulkstat_one_int( buf->bs_xflags =xfs_ip2xflags (ip); buf->bs_extsize_blks = ip->i_extsize; - buf->bs_extents =xfs_ifork_nextents (&ip->i_df); + + nextents =xfs_ifork_nextents (&ip->i_df); +if (!(bc->breq->flags &XFS_IBULK_NREXT64 )) { +xfs_extnum_t max_nextents =XFS_MAX_EXTCNT_DATA_FORK_OLD ; + +if (unlikely (XFS_TEST_ERROR (false, mp, +XFS_ERRTAG_REDUCE_MAX_IEXTENTS ))) + max_nextents =10 ; + +if (nextents > max_nextents) { +xfs_iunlock (ip,XFS_ILOCK_SHARED ); +xfs_irele (ip); + error = -EOVERFLOW ; + goto out; + } + + buf->bs_extents = nextents; + }else { + buf->bs_extents64 = nextents; + } +xfs_bulkstat_health (ip, buf); buf->bs_aextents =xfs_ifork_nextents (ip->i_afp); buf->bs_forkoff =XFS_IFORK_BOFF (ip);
fs/xfs/xfs_itable.c xfs_bulkstat( · line 305
Quoted 6 lines
fs/xfs/xfs_itable.c +2
-1@@ -279,7 +301,8 @@ xfs_bulkstat(if (error) goto out; - error = xfs_iwalk(breq->mp, tp, breq->startino, breq->flags, + error = xfs_iwalk(breq->mp, tp, breq->startino, + breq->flags & XFS_IBULK_IWALK_MASK,
I think it would be cleaner if this function did:
unsigned int iwalk_flags = 0; if (breq->flags & XFS_IBULK_SAME_AG) iwalk_flags |= XFS_IWALK_SAME_AG; ... error = xfs_iwalk(breq->mp, tp, breq->startino, iwalk_flags, xfs_bulkstat_iwalk, breq->icount, &bc);
to make the flags translation explicit. That enables a full cleanup of...
Quoted 12 lines
fs/xfs/xfs_itable.c +0
-0xfs_bulkstat_iwalk, breq->icount, &bc);xfs_trans_cancel (tp); out:fs/xfs/xfs_itable.h +0
-1@@ -13,12 +13,17 @@ struct xfs_ibulk {xfs_ino_t startino;/* start with this inode */ unsigned int icount;/* number of elements in ubuffer */ unsigned int ocount;/* number of records returned */ -unsigned int flags;/* see XFS_IBULK_FLAG_* */
fs/xfs/xfs_itable.h struct xfs_ibulk { · line 25
Quoted 10 lines
fs/xfs/xfs_itable.h +5
-0+unsigned long long flags;/* see XFS_IBULK_FLAG_* */ };/* Only iterate within the same AG as startino */ #define XFS_IBULK_SAME_AG (XFS_IWALK_SAME_AG) +#define XFS_IBULK_ONLY_OFFSET 32 +#define XFS_IBULK_IWALK_MASK ((1ULL << XFS_IBULK_ONLY_OFFSET) - 1) + +#define XFS_IBULK_NREXT64 (1ULL << XFS_IBULK_ONLY_OFFSET)
...the code smells in the XFS_IBULK* flag space:
/* Only iterate within the same AG as startino */
#define XFS_IBULK_SAME_AG (1 << 0)
/* Whatever it is that nrext64 does */
#define XFS_IBULK_NREXT64 (1 << 31)
Quoted 10 lines
fs/xfs/xfs_itable.h +1
-0+/* * Advance the user buffer pointer by one record of the given size. If the * buffer is now full, return the appropriate error code. fs/xfs/xfs_iwalk.h +0
-0@@ -26,9 +26,12 @@ int xfs_iwalk_threaded(struct xfs_mount *mp, xfs_ino_t startino, unsigned int inode_records, bool poll, void *data);
fs/xfs/xfs_iwalk.h int xfs_iwalk_threaded(struct xfs_mount *mp, xfs_ino_t startino, · line 34
Quoted 10 lines
fs/xfs/xfs_iwalk.h +5
-2/* Only iterate inodes within the same AG as @startino. */ -#define XFS_IWALK_SAME_AG (0x1) +#define XFS_IWALK_SAME_AG (1 << 0) -#define XFS_IWALK_FLAGS_ALL (XFS_IWALK_SAME_AG) +#define XFS_IWALK_NREXT64 (1 << 1) + +#define XFS_IWALK_FLAGS_ALL (XFS_IWALK_SAME_AG | \ + XFS_IWALK_NREXT64)
XFS_IWALK_NREXT64 isn't used anywhere.
--D
Quoted 6 lines
fs/xfs/xfs_iwalk.h +0
-0/* Walk all inode btree records in the filesystem starting from @startino. */ typedef int (*xfs_inobt_walk_fn)(struct xfs_mount *mp, struct xfs_trans *tp,Signature · 1 line
2.30.2
Re: [PATCH V5 14/16] xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters
Quoted 46 lines · Darrick J. Wong
Quoted 45 lines · Chandan Babu R
The following changes are made to enable userspace to obtain 64-bit extent counters,
1. Carve out a new 64-bit fieldxfs_bulkstat->bs_extents64from xfs_bulkstat->bs_pad[] to hold 64-bit extent counter.
2. Define the new flagXFS_BULK_IREQ_BULKSTATfor userspace to indicate that it is capable of receiving 64-bit extent counters.Suggested-by Darrick J. Wong <[email protected]>Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_fs.h | 12 ++++++++---- fs/xfs/xfs_ioctl.c | 3 +++ fs/xfs/xfs_itable.c | 27 +++++++++++++++++++++++++-- fs/xfs/xfs_itable.h | 7 ++++++- fs/xfs/xfs_iwalk.h | 7 +++++-- 5 files changed, 47 insertions(+), 9 deletions(-)fs/xfs/libxfs/xfs_fs.h +3
-2@@ -393,7 +393,7 @@ struct xfs_bulkstat {uint32_t bs_extsize_blks;/* extent size hint, blocks */ uint32_t bs_nlink;/* number of links */ -uint32_t bs_extents;/* number of extents */ +uint32_t bs_extents;/* 32-bit data fork extent counter */ uint32_t bs_aextents;/* attribute number of extents */ uint16_t bs_version;/* structure version */ uint16_t bs_forkoff;/* inode fork offset in bytes */ @@ -402,8 +402,9 @@ struct xfs_bulkstat {uint16_t bs_checked;/* checked inode metadata */ uint16_t bs_mode;/* type and mode */ uint16_t bs_pad2;/* zeroed */ +uint64_t bs_extents64;/* 64-bit data fork extent counter */ -uint64_t bs_pad[7 ];/* zeroed */ +uint64_t bs_pad[6 ];/* zeroed */ };#define XFS_BULKSTAT_VERSION_V1 (1)@@ -484,8 +485,11 @@ struct xfs_bulk_ireq {*/ #define XFS_BULK_IREQ_SPECIAL (1 << 1)
Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_bulk_ireq { · line 488
Quoted 3 lines
fs/xfs/libxfs/xfs_fs.h +1
-2-#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ - XFS_BULK_IREQ_SPECIAL) +#define XFS_BULK_IREQ_NREXT64 (1 << 2)This needs a comment specifying the behavior of this flag.
If the flag is set and the data fork extent count fits in both fields, will they both be filled out?
If the flag is set, xfs_bulkstat->bs_extents64 field will be assigned the data fork extent count and xfs_bulkstat->bs_extents will be set to 0 (xfs_bulkstat() allocates xfs_bstat_chunk->buf by invoking kmem_zalloc()).
If the flag is not set, xfs_bulkstat->bs_extents field will be assigned the data fork extent count and xfs_bulkstat->bs_extents64 will be set to 0.
Quoted 3 lines
If the flag is set but the data fork extent count only fits in
bs_extents64, what will be written tobs_extents?
bs_extents will be set to zero.
Quoted 4 lines
If the flag is not set and the data fork extent count won't fit in
bs_extents, do we return an error value? Fill it with garbage?
In this case, we return -EOVERFLOW and the contents of bs_extents will be set to zero. This happens because xfs_bulkstat() will return success even if xfs_iwalk() returned an error provided that we already have collected details about one more inodes. The next call to xfs_ioc_bulkstat() will start from the problematic inode. Here we allocate xfs_bstat_chunk->buf using kmem_zalloc() which zeroes the contents of the allocated memory and returns -EOVERFLOW to userspace.
Quoted 118 lines
Quoted 71 lines
fs/xfs/libxfs/xfs_fs.h +4
-0+ +#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ + XFS_BULK_IREQ_SPECIAL | \ + XFS_BULK_IREQ_NREXT64)/* Operate on the root directory inode. */ #define XFS_BULK_IREQ_SPECIAL_ROOT (1)fs/xfs/xfs_ioctl.c +3
-0@@ -893,6 +893,9 @@ xfs_bulk_ireq_setup(if (XFS_INO_TO_AGNO (mp, breq->startino) >= mp->m_sb.sb_agcount)return -ECANCELED ; +if (hdr->flags &XFS_BULK_IREQ_NREXT64 ) + breq->flags |=XFS_IBULK_NREXT64 ; +return 0 ; }fs/xfs/xfs_itable.c +23
-1@@ -20,6 +20,7 @@#include "xfs_icache.h" #include "xfs_health.h" #include "xfs_trans.h" +#include "xfs_errortag.h" /* * Bulk Stat @@ -64,6 +65,7 @@ xfs_bulkstat_one_int(struct xfs_inode *ip;/* incore inode pointer */ struct inode *inode;struct xfs_bulkstat *buf = bc->buf; +xfs_extnum_t nextents;int error = -EINVAL ;if (xfs_internal_inum (mp, ino))@@ -102,7 +104,27 @@ xfs_bulkstat_one_int( buf->bs_xflags =xfs_ip2xflags (ip); buf->bs_extsize_blks = ip->i_extsize; - buf->bs_extents =xfs_ifork_nextents (&ip->i_df); + + nextents =xfs_ifork_nextents (&ip->i_df); +if (!(bc->breq->flags &XFS_IBULK_NREXT64 )) { +xfs_extnum_t max_nextents =XFS_MAX_EXTCNT_DATA_FORK_OLD ; + +if (unlikely (XFS_TEST_ERROR (false, mp, +XFS_ERRTAG_REDUCE_MAX_IEXTENTS ))) + max_nextents =10 ; + +if (nextents > max_nextents) { +xfs_iunlock (ip,XFS_ILOCK_SHARED ); +xfs_irele (ip); + error = -EOVERFLOW ; + goto out; + } + + buf->bs_extents = nextents; + }else { + buf->bs_extents64 = nextents; + } +xfs_bulkstat_health (ip, buf); buf->bs_aextents =xfs_ifork_nextents (ip->i_afp); buf->bs_forkoff =XFS_IFORK_BOFF (ip);fs/xfs/xfs_itable.c xfs_bulkstat( · line 305
Quoted 6 lines
fs/xfs/xfs_itable.c +2
-1@@ -279,7 +301,8 @@ xfs_bulkstat(if (error) goto out; - error = xfs_iwalk(breq->mp, tp, breq->startino, breq->flags, + error = xfs_iwalk(breq->mp, tp, breq->startino, + breq->flags & XFS_IBULK_IWALK_MASK,I think it would be cleaner if this function did:
unsigned int iwalk_flags = 0; if (breq->flags & XFS_IBULK_SAME_AG) iwalk_flags |= XFS_IWALK_SAME_AG; ... error = xfs_iwalk(breq->mp, tp, breq->startino, iwalk_flags, xfs_bulkstat_iwalk, breq->icount, &bc);to make the flags translation explicit. That enables a full cleanup of...
Quoted 12 lines
fs/xfs/xfs_itable.c +0
-0xfs_bulkstat_iwalk, breq->icount, &bc);xfs_trans_cancel (tp); out:fs/xfs/xfs_itable.h +0
-1@@ -13,12 +13,17 @@ struct xfs_ibulk {xfs_ino_t startino;/* start with this inode */ unsigned int icount;/* number of elements in ubuffer */ unsigned int ocount;/* number of records returned */ -unsigned int flags;/* see XFS_IBULK_FLAG_* */ fs/xfs/xfs_itable.h struct xfs_ibulk { · line 25
Quoted 10 lines
fs/xfs/xfs_itable.h +5
-0+unsigned long long flags;/* see XFS_IBULK_FLAG_* */ };/* Only iterate within the same AG as startino */ #define XFS_IBULK_SAME_AG (XFS_IWALK_SAME_AG) +#define XFS_IBULK_ONLY_OFFSET 32 +#define XFS_IBULK_IWALK_MASK ((1ULL << XFS_IBULK_ONLY_OFFSET) - 1) + +#define XFS_IBULK_NREXT64 (1ULL << XFS_IBULK_ONLY_OFFSET)...the code smells in the XFS_IBULK* flag space:
Quoted 6 lines
/* Only iterate within the same AG as startino */
#defineXFS_IBULK_SAME_AG(1 << 0)/* Whatever it is that nrext64 does */
#defineXFS_IBULK_NREXT64(1 << 31)
Ok. This will indeed make the code more readable. I will include this change when I post the next version of this patch.
Quoted 15 lines
Quoted 15 lines
fs/xfs/xfs_itable.h +1
-0+/* * Advance the user buffer pointer by one record of the given size. If the * buffer is now full, return the appropriate error code. fs/xfs/xfs_iwalk.h +1
-1@@ -26,9 +26,12 @@ int xfs_iwalk_threaded(struct xfs_mount *mp, xfs_ino_t startino, unsigned int inode_records, bool poll, void *data);/* Only iterate inodes within the same AG as @startino. */ -#define XFS_IWALK_SAME_AG (0x1) +#define XFS_IWALK_SAME_AG (1 << 0)
Quoted 8 lines
fs/xfs/xfs_iwalk.h int xfs_iwalk_threaded(struct xfs_mount *mp, xfs_ino_t startino, · line 34
Quoted 5 lines
fs/xfs/xfs_iwalk.h +4
-1-#define XFS_IWALK_FLAGS_ALL (XFS_IWALK_SAME_AG) +#define XFS_IWALK_NREXT64 (1 << 1) + +#define XFS_IWALK_FLAGS_ALL (XFS_IWALK_SAME_AG | \ + XFS_IWALK_NREXT64)
XFS_IWALK_NREXT64isn't used anywhere.
Sorry, I will remove this.
Thanks for all the review comments.
Signature · 1 line
chandan
Re: [PATCH V5 14/16] xfs: Enable bulkstat ioctl to support 64-bit per-inode extent counters
Quoted 74 lines · Chandan Babu R
Quoted 46 lines · Darrick J. Wong
Quoted 45 lines · Chandan Babu R
The following changes are made to enable userspace to obtain 64-bit extent counters,
1. Carve out a new 64-bit fieldxfs_bulkstat->bs_extents64from xfs_bulkstat->bs_pad[] to hold 64-bit extent counter.
2. Define the new flagXFS_BULK_IREQ_BULKSTATfor userspace to indicate that it is capable of receiving 64-bit extent counters.Suggested-by Darrick J. Wong <[email protected]>Signed-off-by Chandan Babu R <[email protected]>--- fs/xfs/libxfs/xfs_fs.h | 12 ++++++++---- fs/xfs/xfs_ioctl.c | 3 +++ fs/xfs/xfs_itable.c | 27 +++++++++++++++++++++++++-- fs/xfs/xfs_itable.h | 7 ++++++- fs/xfs/xfs_iwalk.h | 7 +++++-- 5 files changed, 47 insertions(+), 9 deletions(-)fs/xfs/libxfs/xfs_fs.h +3
-2@@ -393,7 +393,7 @@ struct xfs_bulkstat {uint32_t bs_extsize_blks;/* extent size hint, blocks */ uint32_t bs_nlink;/* number of links */ -uint32_t bs_extents;/* number of extents */ +uint32_t bs_extents;/* 32-bit data fork extent counter */ uint32_t bs_aextents;/* attribute number of extents */ uint16_t bs_version;/* structure version */ uint16_t bs_forkoff;/* inode fork offset in bytes */ @@ -402,8 +402,9 @@ struct xfs_bulkstat {uint16_t bs_checked;/* checked inode metadata */ uint16_t bs_mode;/* type and mode */ uint16_t bs_pad2;/* zeroed */ +uint64_t bs_extents64;/* 64-bit data fork extent counter */ -uint64_t bs_pad[7 ];/* zeroed */ +uint64_t bs_pad[6 ];/* zeroed */ };#define XFS_BULKSTAT_VERSION_V1 (1)@@ -484,8 +485,11 @@ struct xfs_bulk_ireq {*/ #define XFS_BULK_IREQ_SPECIAL (1 << 1)Quoted 8 lines
fs/xfs/libxfs/xfs_fs.h struct xfs_bulk_ireq { · line 488
Quoted 3 lines
fs/xfs/libxfs/xfs_fs.h +1
-2-#define XFS_BULK_IREQ_FLAGS_ALL (XFS_BULK_IREQ_AGNO | \ - XFS_BULK_IREQ_SPECIAL) +#define XFS_BULK_IREQ_NREXT64 (1 << 2)This needs a comment specifying the behavior of this flag.
If the flag is set and the data fork extent count fits in both fields, will they both be filled out?
If the flag is set,
xfs_bulkstat->bs_extents64field will be assigned the data fork extent count andxfs_bulkstat->bs_extentswill be set to 0 (xfs_bulkstat()allocatesxfs_bstat_chunk->bufby invokingkmem_zalloc()).If the flag is not set,
xfs_bulkstat->bs_extentsfield will be assigned the data fork extent count andxfs_bulkstat->bs_extents64will be set to 0.Quoted 3 lines
If the flag is set but the data fork extent count only fits in
bs_extents64, what will be written tobs_extents?
bs_extentswill be set to zero.Quoted 4 lines
If the flag is not set and the data fork extent count won't fit in
bs_extents, do we return an error value? Fill it with garbage?
Quoted 8 lines
In this case, we return
-EOVERFLOWand the contents ofbs_extentswill be set to zero. This happens becausexfs_bulkstat()will return success even ifxfs_iwalk()returned an error provided that we already have collected details about one more inodes. The next call toxfs_ioc_bulkstat()will start from the problematic inode. Here we allocatexfs_bstat_chunk->bufusingkmem_zalloc()which zeroes the contents of the allocated memory and returns-EOVERFLOWto userspace.
Sorry, I forgot to mention that I will add the above data points as a comment before posting the next version of the patchset.
Signature · 1 line
chandan
PATCH v5 15/16 xfs: Add XFS_SB_FEAT_INCOMPAT_NREXT64 to the list of supported flags
In reply to Chandan Babu RThis commit enables XFS module to work with fs instances having 64-bit per-inode extent counters by adding XFS_SB_FEAT_INCOMPAT_NREXT64 flag to the list of supported incompat feature flags.
--- fs/xfs/libxfs/xfs_format.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/xfs/libxfs/xfs_format.h +2 -1
@@ -378,7 +378,8 @@ xfs_sb_has_ro_compat_feature( XFS_SB_FEAT_INCOMPAT_SPINODES| \ XFS_SB_FEAT_INCOMPAT_META_UUID| \ XFS_SB_FEAT_INCOMPAT_BIGTIME| \ - XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR) + XFS_SB_FEAT_INCOMPAT_NEEDSREPAIR| \ + XFS_SB_FEAT_INCOMPAT_NREXT64)#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
Signature · 1 line
2.30.2
PATCH v5 16/16 xfs: Define max extent length based on on-disk format definition
In reply to Chandan Babu RThe maximum extent length depends on maximum block count that can be stored in a BMBT record. Hence this commit defines MAXEXTLEN based on BMBT_BLOCKCOUNT_BITLEN.
While at it, the commit also renames MAXEXTLEN to XFS_MAX_BMBT_EXTLEN.
--- fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 57 +++++++++++++++++----------------- fs/xfs/libxfs/xfs_format.h | 5 +-- fs/xfs/libxfs/xfs_inode_buf.c | 4 +-- fs/xfs/libxfs/xfs_trans_resv.c | 11 ++++--- fs/xfs/scrub/bmap.c | 2 +- fs/xfs/xfs_bmap_util.c | 14 +++++---- fs/xfs/xfs_iomap.c | 28 ++++++++--------- 8 files changed, 64 insertions(+), 59 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +1 -1
@@ -2493,7 +2493,7 @@ __xfs_free_extent_later(ASSERT (bno !=NULLFSBLOCK );ASSERT (len >0 ); -ASSERT (len <=MAXEXTLEN ); +ASSERT (len <=XFS_MAX_BMBT_EXTLEN );ASSERT (!isnullstartblock (bno)); agno =XFS_FSB_TO_AGNO (mp, bno); agbno =XFS_FSB_TO_AGBNO (mp, bno);
fs/xfs/libxfs/xfs_bmap.c +29 -28
168 lines not on this page · raw message
fs/xfs/libxfs/xfs_format.h +3 -2
@@ -886,7 +886,7 @@ enum xfs_dinode_fmt { { XFS_DINODE_FMT_UUID, "uuid" }/* -* Max values for extlen, extnum, aextnum. +* Max values for ondisk inode's extent counters. * * The newly introduced data fork extent counter is a 64-bit field. However, the * maximum number of extents in a file is limited to 2^54 extents (assuming one
@@ -898,7 +898,6 @@ enum xfs_dinode_fmt {* Rounding up 47 to the nearest multiple of bits-per-byte results in 48. Hence * 2^48 was chosen as the maximum data fork extent count. */ -#define MAXEXTLEN ((xfs_extlen_t)((1ULL << 21) - 1))/* 21 bits */ #define XFS_MAX_EXTCNT_DATA_FORK ((xfs_extnum_t)((1ULL << 48) - 1))/* Unsigned 48-bits */ #define XFS_MAX_EXTCNT_ATTR_FORK ((xfs_extnum_t)((1ULL << 32) - 1))/* Unsigned 32-bits */ #define XFS_MAX_EXTCNT_DATA_FORK_OLD ((xfs_extnum_t)((1ULL << 31) - 1))/* Signed 32-bits */
@@ -1635,6 +1634,8 @@ typedef struct xfs_bmdr_block {#define BMBT_STARTOFF_MASK ((1ULL << BMBT_STARTOFF_BITLEN) - 1)#define BMBT_BLOCKCOUNT_MASK ((1ULL << BMBT_BLOCKCOUNT_BITLEN) - 1) +#define XFS_MAX_BMBT_EXTLEN ((xfs_extlen_t)(BMBT_BLOCKCOUNT_MASK)) +/* * bmbt records have a file offset (block) field that is 54 bits wide, so this * is the largest xfs_fileoff_t that we ever expect to see.
fs/xfs/libxfs/xfs_inode_buf.c +2 -2
@@ -691,7 +691,7 @@ xfs_inode_validate_extsize(if (extsize_bytes % blocksize_bytes)return __this_address; -if (extsize >MAXEXTLEN ) +if (extsize >XFS_MAX_BMBT_EXTLEN )return __this_address;if (!rt_flag && extsize > mp->m_sb.sb_agblocks /2 )
@@ -748,7 +748,7 @@ xfs_inode_validate_cowextsize(if (cowextsize_bytes % mp->m_sb.sb_blocksize)return __this_address; -if (cowextsize >MAXEXTLEN ) +if (cowextsize >XFS_MAX_BMBT_EXTLEN )return __this_address;if (cowextsize > mp->m_sb.sb_agblocks /2 )
fs/xfs/libxfs/xfs_trans_resv.c +6 -5
@@ -199,8 +199,8 @@ xfs_calc_inode_chunk_res(/* * Per-extent log reservation for the btree changes involved in freeing or * allocating a realtime extent. We have to be able to log as many rtbitmap -* blocks as needed to mark inuse MAXEXTLEN blocks' worth of realtime extents, -* as well as the realtime summary block. +* blocks as needed to mark inuse XFS_BMBT_MAX_EXTLEN blocks' worth of realtime +* extents, as well as the realtime summary block. */ static unsigned int xfs_rtalloc_log_count(
@@ -210,7 +210,7 @@ xfs_rtalloc_log_count(unsigned int blksz =XFS_FSB_TO_B (mp,1 );unsigned int rtbmp_bytes; - rtbmp_bytes = (MAXEXTLEN / mp->m_sb.sb_rextsize) /NBBY ; + rtbmp_bytes = (XFS_MAX_BMBT_EXTLEN / mp->m_sb.sb_rextsize) /NBBY ;return (howmany (rtbmp_bytes, blksz) +1 ) * num_ops; }
@@ -247,7 +247,7 @@ xfs_rtalloc_log_count(* the inode's bmap btree: max depth * block size * the agfs of the ags from which the extents are allocated: 2 * sector * the superblock free block counter: sector size -* the realtime bitmap: ((MAXEXTLEN / rtextsize) / NBBY) bytes +* the realtime bitmap: ((XFS_BMBT_MAX_EXTLEN / rtextsize) / NBBY) bytes * the realtime summary: 1 block * the allocation btrees: 2 trees * (2 * max depth - 1) * block size * And the bmap_finish transaction can free bmap blocks in a join (t3):
@@ -299,7 +299,8 @@ xfs_calc_write_reservation(* the agf for each of the ags: 2 * sector size * the agfl for each of the ags: 2 * sector size * the super block to reflect the freed blocks: sector size -* the realtime bitmap: 2 exts * ((MAXEXTLEN / rtextsize) / NBBY) bytes +* the realtime bitmap: 2 exts * ((XFS_BMBT_MAX_EXTLEN / rtextsize) / NBBY) +* bytes * the realtime summary: 2 exts * 1 block * worst case split in allocation btrees per extent assuming 2 extents: * 2 exts * 2 trees * (2 * max depth - 1) * block size
fs/xfs/scrub/bmap.c +1 -1
@@ -350,7 +350,7 @@ xchk_bmap_iextent( irec->br_startoff);/* Make sure the extent points to a valid place. */ -if (irec->br_blockcount >MAXEXTLEN ) +if (irec->br_blockcount >XFS_MAX_BMBT_EXTLEN )xchk_fblock_set_corrupt (info->sc, info->whichfork, irec->br_startoff); if (info->is_rt &&
fs/xfs/xfs_bmap_util.c +8 -6
@@ -119,14 +119,14 @@ xfs_bmap_rtalloc(*/ ralen = ap->length / mp->m_sb.sb_rextsize;/* -* If the old value was close enough to MAXEXTLEN that +* If the old value was close enough to XFS_BMBT_MAX_EXTLEN that * we rounded up to it, cut it back so it's valid again. * Note that if it's a really large request (bigger than -* MAXEXTLEN), we don't hear about that number, and can't +* XFS_BMBT_MAX_EXTLEN), we don't hear about that number, and can't * adjust the starting point to match it. */ -if (ralen * mp->m_sb.sb_rextsize >=MAXEXTLEN ) - ralen =MAXEXTLEN / mp->m_sb.sb_rextsize; +if (ralen * mp->m_sb.sb_rextsize >=XFS_MAX_BMBT_EXTLEN ) + ralen =XFS_MAX_BMBT_EXTLEN / mp->m_sb.sb_rextsize;/* * Lock out modifications to both the RT bitmap and summary inodes
@@ -840,9 +840,11 @@ xfs_alloc_file_space(* count, hence we need to limit the number of blocks we are * trying to reserve to avoid an overflow. We can't allocate * more than @nimaps extents, and an extent is limited on disk -* to MAXEXTLEN (21 bits), so use that to enforce the limit. +* to XFS_BMBT_MAX_EXTLEN (21 bits), so use that to enforce the +* limit. */ - resblks =min_t (xfs_fileoff_t, (e - s), (MAXEXTLEN * nimaps)); + resblks =min_t (xfs_fileoff_t, (e - s), + (XFS_MAX_BMBT_EXTLEN * nimaps));if (unlikely (rt)) { dblocks =XFS_DIOSTRAT_SPACE_RES (mp,0 ); rblocks = resblks;
fs/xfs/xfs_iomap.c +14 -14
@@ -395,7 +395,7 @@ xfs_iomap_prealloc_size(*/ plen = prev.br_blockcount;while (xfs_iext_prev_extent (ifp, &ncur, &got)) { -if (plen >MAXEXTLEN /2 || +if (plen >XFS_MAX_BMBT_EXTLEN /2 ||isnullstartblock (got.br_startblock) || got.br_startoff + got.br_blockcount != prev.br_startoff || got.br_startblock + got.br_blockcount != prev.br_startblock)
@@ -407,23 +407,23 @@ xfs_iomap_prealloc_size(/* * If the size of the extents is greater than half the maximum extent * length, then use the current offset as the basis. This ensures that -* for large files the preallocation size always extends to MAXEXTLEN -* rather than falling short due to things like stripe unit/width -* alignment of real extents. +* for large files the preallocation size always extends to +* XFS_BMBT_MAX_EXTLEN rather than falling short due to things like stripe +* unit/width alignment of real extents. */ alloc_blocks = plen *2 ; -if (alloc_blocks >MAXEXTLEN ) +if (alloc_blocks >XFS_MAX_BMBT_EXTLEN ) alloc_blocks =XFS_B_TO_FSB (mp, offset); qblocks = alloc_blocks;/* -* MAXEXTLEN is not a power of two value but we round the prealloc down -* to the nearest power of two value after throttling. To prevent the -* round down from unconditionally reducing the maximum supported -* prealloc size, we round up first, apply appropriate throttling, -* round down and cap the value to MAXEXTLEN. +* XFS_BMBT_MAX_EXTLEN is not a power of two value but we round the prealloc +* down to the nearest power of two value after throttling. To prevent +* the round down from unconditionally reducing the maximum supported +* prealloc size, we round up first, apply appropriate throttling, round +* down and cap the value to XFS_BMBT_MAX_EXTLEN. */ - alloc_blocks =XFS_FILEOFF_MIN (roundup_pow_of_two (MAXEXTLEN ), + alloc_blocks =XFS_FILEOFF_MIN (roundup_pow_of_two (XFS_MAX_BMBT_EXTLEN ), alloc_blocks); freesp =percpu_counter_read_positive (&mp->m_fdblocks);
@@ -471,14 +471,14 @@ xfs_iomap_prealloc_size(*/ if (alloc_blocks) alloc_blocks =rounddown_pow_of_two (alloc_blocks); -if (alloc_blocks >MAXEXTLEN ) - alloc_blocks =MAXEXTLEN ; +if (alloc_blocks >XFS_MAX_BMBT_EXTLEN ) + alloc_blocks =XFS_MAX_BMBT_EXTLEN ;/* * If we are still trying to allocate more space than is * available, squash the prealloc hard. This can happen if we * have a large file on a small filesystem and the above -* lowspace thresholds are smaller than MAXEXTLEN. +* lowspace thresholds are smaller than XFS_BMBT_MAX_EXTLEN. */ while (alloc_blocks && alloc_blocks >= freesp) alloc_blocks >>=4 ;
Signature · 1 line
2.30.2
Re: [PATCH V5 16/16] xfs: Define max extent length based on on-disk format definition
Quoted 8 lines · Chandan Babu R
The maximum extent length depends on maximum block count that can be stored in a BMBT record. Hence this commit defines MAXEXTLEN based on
BMBT_BLOCKCOUNT_BITLEN.While at it, the commit also renames MAXEXTLEN to
XFS_MAX_BMBT_EXTLEN.Suggested-by Darrick J. Wong <[email protected]>Signed-off-by Chandan Babu R <[email protected]>
Looks good now,
--D
Quoted 423 lines
--- fs/xfs/libxfs/xfs_alloc.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 57 +++++++++++++++++----------------- fs/xfs/libxfs/xfs_format.h | 5 +-- fs/xfs/libxfs/xfs_inode_buf.c | 4 +-- fs/xfs/libxfs/xfs_trans_resv.c | 11 ++++--- fs/xfs/scrub/bmap.c | 2 +- fs/xfs/xfs_bmap_util.c | 14 +++++---- fs/xfs/xfs_iomap.c | 28 ++++++++--------- 8 files changed, 64 insertions(+), 59 deletions(-)fs/xfs/libxfs/xfs_alloc.c +1
-1@@ -2493,7 +2493,7 @@ __xfs_free_extent_later(ASSERT (bno !=NULLFSBLOCK );ASSERT (len >0 ); -ASSERT (len <=MAXEXTLEN ); +ASSERT (len <=XFS_MAX_BMBT_EXTLEN );ASSERT (!isnullstartblock (bno)); agno =XFS_FSB_TO_AGNO (mp, bno); agbno =XFS_FSB_TO_AGBNO (mp, bno);fs/xfs/libxfs/xfs_bmap.c +29
-28@@ -1449,7 +1449,7 @@ xfs_bmap_add_extent_delay_real(LEFT .br_startoff +LEFT .br_blockcount == new->br_startoff &&LEFT .br_startblock +LEFT .br_blockcount == new->br_startblock &&LEFT .br_state == new->br_state && - LEFT.br_blockcount + new->br_blockcount <= MAXEXTLEN) + LEFT.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN) state |= BMAP_LEFT_CONTIG;/* @@ -1467,13 +1467,13 @@ xfs_bmap_add_extent_delay_real( new_endoff ==RIGHT .br_startoff && new->br_startblock + new->br_blockcount ==RIGHT .br_startblock && new->br_state ==RIGHT .br_state && - new->br_blockcount +RIGHT .br_blockcount <=MAXEXTLEN && + new->br_blockcount +RIGHT .br_blockcount <=XFS_MAX_BMBT_EXTLEN && ((state & (BMAP_LEFT_CONTIG |BMAP_LEFT_FILLING |BMAP_RIGHT_FILLING )) != (BMAP_LEFT_CONTIG |BMAP_LEFT_FILLING |BMAP_RIGHT_FILLING ) ||LEFT .br_blockcount + new->br_blockcount +RIGHT .br_blockcount - <= MAXEXTLEN)) + <= XFS_MAX_BMBT_EXTLEN)) state |= BMAP_RIGHT_CONTIG; error =0 ;@@ -1997,7 +1997,7 @@ xfs_bmap_add_extent_unwritten_real(LEFT .br_startoff +LEFT .br_blockcount == new->br_startoff &&LEFT .br_startblock +LEFT .br_blockcount == new->br_startblock &&LEFT .br_state == new->br_state && - LEFT.br_blockcount + new->br_blockcount <= MAXEXTLEN) + LEFT.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN) state |= BMAP_LEFT_CONTIG;/* @@ -2015,13 +2015,13 @@ xfs_bmap_add_extent_unwritten_real( new_endoff ==RIGHT .br_startoff && new->br_startblock + new->br_blockcount ==RIGHT .br_startblock && new->br_state ==RIGHT .br_state && - new->br_blockcount +RIGHT .br_blockcount <=MAXEXTLEN && + new->br_blockcount +RIGHT .br_blockcount <=XFS_MAX_BMBT_EXTLEN && ((state & (BMAP_LEFT_CONTIG |BMAP_LEFT_FILLING |BMAP_RIGHT_FILLING )) != (BMAP_LEFT_CONTIG |BMAP_LEFT_FILLING |BMAP_RIGHT_FILLING ) ||LEFT .br_blockcount + new->br_blockcount +RIGHT .br_blockcount - <= MAXEXTLEN)) + <= XFS_MAX_BMBT_EXTLEN)) state |= BMAP_RIGHT_CONTIG;/* @@ -2507,15 +2507,15 @@ xfs_bmap_add_extent_hole_delay(*/ if ((state &BMAP_LEFT_VALID ) && (state &BMAP_LEFT_DELAY ) && left.br_startoff + left.br_blockcount == new->br_startoff && - left.br_blockcount + new->br_blockcount <=MAXEXTLEN ) + left.br_blockcount + new->br_blockcount <=XFS_MAX_BMBT_EXTLEN ) state |=BMAP_LEFT_CONTIG ;if ((state &BMAP_RIGHT_VALID ) && (state &BMAP_RIGHT_DELAY ) && new->br_startoff + new->br_blockcount == right.br_startoff && - new->br_blockcount + right.br_blockcount <=MAXEXTLEN && + new->br_blockcount + right.br_blockcount <=XFS_MAX_BMBT_EXTLEN && (!(state &BMAP_LEFT_CONTIG ) || (left.br_blockcount + new->br_blockcount + - right.br_blockcount <=MAXEXTLEN ))) + right.br_blockcount <=XFS_MAX_BMBT_EXTLEN ))) state |=BMAP_RIGHT_CONTIG ;/* @@ -2658,17 +2658,17 @@ xfs_bmap_add_extent_hole_real( left.br_startoff + left.br_blockcount == new->br_startoff && left.br_startblock + left.br_blockcount == new->br_startblock && left.br_state == new->br_state && - left.br_blockcount + new->br_blockcount <= MAXEXTLEN) + left.br_blockcount + new->br_blockcount <= XFS_MAX_BMBT_EXTLEN) state |= BMAP_LEFT_CONTIG;if ((state &BMAP_RIGHT_VALID ) && !(state &BMAP_RIGHT_DELAY ) && new->br_startoff + new->br_blockcount == right.br_startoff && new->br_startblock + new->br_blockcount == right.br_startblock && new->br_state == right.br_state && - new->br_blockcount + right.br_blockcount <=MAXEXTLEN && + new->br_blockcount + right.br_blockcount <=XFS_MAX_BMBT_EXTLEN && (!(state &BMAP_LEFT_CONTIG ) || left.br_blockcount + new->br_blockcount + - right.br_blockcount <=MAXEXTLEN )) + right.br_blockcount <=XFS_MAX_BMBT_EXTLEN )) state |=BMAP_RIGHT_CONTIG ; error =0 ;@@ -2903,15 +2903,15 @@ xfs_bmap_extsize_align(/* * For large extent hint sizes, the aligned extent might be larger than -* MAXEXTLEN. In that case, reduce the size by an extsz so that it pulls -* the length back under MAXEXTLEN. The outer allocation loops handle -* short allocation just fine, so it is safe to do this. We only want to -* do it when we are forced to, though, because it means more allocation -* operations are required. +* XFS_BMBT_MAX_EXTLEN. In that case, reduce the size by an extsz so +* that it pulls the length back under XFS_BMBT_MAX_EXTLEN. The outer +* allocation loops handle short allocation just fine, so it is safe to +* do this. We only want to do it when we are forced to, though, because +* it means more allocation operations are required. */ -while (align_alen >MAXEXTLEN ) +while (align_alen >XFS_MAX_BMBT_EXTLEN ) align_alen -= extsz; -ASSERT (align_alen <=MAXEXTLEN ); +ASSERT (align_alen <=XFS_MAX_BMBT_EXTLEN );/* * If the previous block overlaps with this proposed allocation @@ -3001,9 +3001,9 @@ xfs_bmap_extsize_align(return -EINVAL ; }else {ASSERT (orig_off >= align_off); -/* see MAXEXTLEN handling above */ +/* see XFS_BMBT_MAX_EXTLEN handling above */ ASSERT (orig_end <= align_off + align_alen || - align_alen + extsz >MAXEXTLEN ); + align_alen + extsz >XFS_MAX_BMBT_EXTLEN ); }#ifdef DEBUG @@ -3968,7 +3968,7 @@ xfs_bmapi_reserve_delalloc(* Cap the alloc length. Keep track of prealloc so we know whether to * tag the inode before we return. */ - alen =XFS_FILBLKS_MIN (len + prealloc,MAXEXTLEN ); + alen =XFS_FILBLKS_MIN (len + prealloc,XFS_MAX_BMBT_EXTLEN );if (!eof) alen =XFS_FILBLKS_MIN (alen, got->br_startoff - aoff);if (prealloc && alen >= len)@@ -4101,7 +4101,7 @@ xfs_bmapi_allocate(if (!xfs_iext_peek_prev_extent (ifp, &bma->icur, &bma->prev)) bma->prev.br_startoff =NULLFILEOFF ; }else { - bma->length =XFS_FILBLKS_MIN (bma->length,MAXEXTLEN ); + bma->length =XFS_FILBLKS_MIN (bma->length,XFS_MAX_BMBT_EXTLEN );if (!bma->eof) bma->length =XFS_FILBLKS_MIN (bma->length, bma->got.br_startoff - bma->offset);@@ -4421,8 +4421,8 @@ xfs_bmapi_write(* xfs_extlen_t and therefore 32 bits. Hence we have to * check for 32-bit overflows and handle them here. */ -if (len > (xfs_filblks_t )MAXEXTLEN ) - bma.length =MAXEXTLEN ; +if (len > (xfs_filblks_t )XFS_MAX_BMBT_EXTLEN ) + bma.length =XFS_MAX_BMBT_EXTLEN ;else bma.length = len;@@ -4557,7 +4557,8 @@ xfs_bmapi_convert_delalloc( bma.ip = ip; bma.wasdel = true; bma.offset = bma.got.br_startoff; - bma.length =max_t (xfs_filblks_t, bma.got.br_blockcount,MAXEXTLEN ); + bma.length =max_t (xfs_filblks_t, bma.got.br_blockcount, +XFS_MAX_BMBT_EXTLEN ); bma.minleft =xfs_bmapi_minleft (tp, ip, whichfork);/* @@ -4638,7 +4639,7 @@ xfs_bmapi_remap( ifp =XFS_IFORK_PTR (ip, whichfork);ASSERT (len >0 ); -ASSERT (len <= (xfs_filblks_t )MAXEXTLEN ); +ASSERT (len <= (xfs_filblks_t )XFS_MAX_BMBT_EXTLEN );ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL ));ASSERT (!(flags & ~(XFS_BMAPI_ATTRFORK |XFS_BMAPI_PREALLOC |XFS_BMAPI_NORMAP )));@@ -5638,7 +5639,7 @@ xfs_bmse_can_merge(if ((left->br_startoff + left->br_blockcount != startoff) || (left->br_startblock + left->br_blockcount != got->br_startblock) || (left->br_state != got->br_state) || - (left->br_blockcount + got->br_blockcount >MAXEXTLEN )) + (left->br_blockcount + got->br_blockcount >XFS_MAX_BMBT_EXTLEN ))return false;return true;fs/xfs/libxfs/xfs_format.h +3
-2@@ -886,7 +886,7 @@ enum xfs_dinode_fmt { { XFS_DINODE_FMT_UUID, "uuid" }/* -* Max values for extlen, extnum, aextnum. +* Max values for ondisk inode's extent counters. * * The newly introduced data fork extent counter is a 64-bit field. However, the * maximum number of extents in a file is limited to 2^54 extents (assuming one @@ -898,7 +898,6 @@ enum xfs_dinode_fmt {* Rounding up 47 to the nearest multiple of bits-per-byte results in 48. Hence * 2^48 was chosen as the maximum data fork extent count. */ -#define MAXEXTLEN ((xfs_extlen_t)((1ULL << 21) - 1))/* 21 bits */ #define XFS_MAX_EXTCNT_DATA_FORK ((xfs_extnum_t)((1ULL << 48) - 1))/* Unsigned 48-bits */ #define XFS_MAX_EXTCNT_ATTR_FORK ((xfs_extnum_t)((1ULL << 32) - 1))/* Unsigned 32-bits */ #define XFS_MAX_EXTCNT_DATA_FORK_OLD ((xfs_extnum_t)((1ULL << 31) - 1))/* Signed 32-bits */ @@ -1635,6 +1634,8 @@ typedef struct xfs_bmdr_block {#define BMBT_STARTOFF_MASK ((1ULL << BMBT_STARTOFF_BITLEN) - 1)#define BMBT_BLOCKCOUNT_MASK ((1ULL << BMBT_BLOCKCOUNT_BITLEN) - 1) +#define XFS_MAX_BMBT_EXTLEN ((xfs_extlen_t)(BMBT_BLOCKCOUNT_MASK)) +/* * bmbt records have a file offset (block) field that is 54 bits wide, so this * is the largest xfs_fileoff_t that we ever expect to see. fs/xfs/libxfs/xfs_inode_buf.c +2
-2@@ -691,7 +691,7 @@ xfs_inode_validate_extsize(if (extsize_bytes % blocksize_bytes)return __this_address; -if (extsize >MAXEXTLEN ) +if (extsize >XFS_MAX_BMBT_EXTLEN )return __this_address;if (!rt_flag && extsize > mp->m_sb.sb_agblocks /2 )@@ -748,7 +748,7 @@ xfs_inode_validate_cowextsize(if (cowextsize_bytes % mp->m_sb.sb_blocksize)return __this_address; -if (cowextsize >MAXEXTLEN ) +if (cowextsize >XFS_MAX_BMBT_EXTLEN )return __this_address;if (cowextsize > mp->m_sb.sb_agblocks /2 )fs/xfs/libxfs/xfs_trans_resv.c +6
-5@@ -199,8 +199,8 @@ xfs_calc_inode_chunk_res(/* * Per-extent log reservation for the btree changes involved in freeing or * allocating a realtime extent. We have to be able to log as many rtbitmap -* blocks as needed to mark inuse MAXEXTLEN blocks' worth of realtime extents, -* as well as the realtime summary block. +* blocks as needed to mark inuse XFS_BMBT_MAX_EXTLEN blocks' worth of realtime +* extents, as well as the realtime summary block. */ static unsigned int xfs_rtalloc_log_count(@@ -210,7 +210,7 @@ xfs_rtalloc_log_count(unsigned int blksz =XFS_FSB_TO_B (mp,1 );unsigned int rtbmp_bytes; - rtbmp_bytes = (MAXEXTLEN / mp->m_sb.sb_rextsize) /NBBY ; + rtbmp_bytes = (XFS_MAX_BMBT_EXTLEN / mp->m_sb.sb_rextsize) /NBBY ;return (howmany (rtbmp_bytes, blksz) +1 ) * num_ops; }@@ -247,7 +247,7 @@ xfs_rtalloc_log_count(* the inode's bmap btree: max depth * block size * the agfs of the ags from which the extents are allocated: 2 * sector * the superblock free block counter: sector size -* the realtime bitmap: ((MAXEXTLEN / rtextsize) / NBBY) bytes +* the realtime bitmap: ((XFS_BMBT_MAX_EXTLEN / rtextsize) / NBBY) bytes * the realtime summary: 1 block * the allocation btrees: 2 trees * (2 * max depth - 1) * block size * And the bmap_finish transaction can free bmap blocks in a join (t3): @@ -299,7 +299,8 @@ xfs_calc_write_reservation(* the agf for each of the ags: 2 * sector size * the agfl for each of the ags: 2 * sector size * the super block to reflect the freed blocks: sector size -* the realtime bitmap: 2 exts * ((MAXEXTLEN / rtextsize) / NBBY) bytes +* the realtime bitmap: 2 exts * ((XFS_BMBT_MAX_EXTLEN / rtextsize) / NBBY) +* bytes * the realtime summary: 2 exts * 1 block * worst case split in allocation btrees per extent assuming 2 extents: * 2 exts * 2 trees * (2 * max depth - 1) * block size fs/xfs/scrub/bmap.c +1
-1@@ -350,7 +350,7 @@ xchk_bmap_iextent( irec->br_startoff);/* Make sure the extent points to a valid place. */ -if (irec->br_blockcount >MAXEXTLEN ) +if (irec->br_blockcount >XFS_MAX_BMBT_EXTLEN )xchk_fblock_set_corrupt (info->sc, info->whichfork, irec->br_startoff); if (info->is_rt &&fs/xfs/xfs_bmap_util.c +8
-6@@ -119,14 +119,14 @@ xfs_bmap_rtalloc(*/ ralen = ap->length / mp->m_sb.sb_rextsize;/* -* If the old value was close enough to MAXEXTLEN that +* If the old value was close enough to XFS_BMBT_MAX_EXTLEN that * we rounded up to it, cut it back so it's valid again. * Note that if it's a really large request (bigger than -* MAXEXTLEN), we don't hear about that number, and can't +* XFS_BMBT_MAX_EXTLEN), we don't hear about that number, and can't * adjust the starting point to match it. */ -if (ralen * mp->m_sb.sb_rextsize >=MAXEXTLEN ) - ralen =MAXEXTLEN / mp->m_sb.sb_rextsize; +if (ralen * mp->m_sb.sb_rextsize >=XFS_MAX_BMBT_EXTLEN ) + ralen =XFS_MAX_BMBT_EXTLEN / mp->m_sb.sb_rextsize;/* * Lock out modifications to both the RT bitmap and summary inodes @@ -840,9 +840,11 @@ xfs_alloc_file_space(* count, hence we need to limit the number of blocks we are * trying to reserve to avoid an overflow. We can't allocate * more than @nimaps extents, and an extent is limited on disk -* to MAXEXTLEN (21 bits), so use that to enforce the limit. +* to XFS_BMBT_MAX_EXTLEN (21 bits), so use that to enforce the +* limit. */ - resblks =min_t (xfs_fileoff_t, (e - s), (MAXEXTLEN * nimaps)); + resblks =min_t (xfs_fileoff_t, (e - s), + (XFS_MAX_BMBT_EXTLEN * nimaps));if (unlikely (rt)) { dblocks =XFS_DIOSTRAT_SPACE_RES (mp,0 ); rblocks = resblks;fs/xfs/xfs_iomap.c +14
-14@@ -395,7 +395,7 @@ xfs_iomap_prealloc_size(*/ plen = prev.br_blockcount;while (xfs_iext_prev_extent (ifp, &ncur, &got)) { -if (plen >MAXEXTLEN /2 || +if (plen >XFS_MAX_BMBT_EXTLEN /2 ||isnullstartblock (got.br_startblock) || got.br_startoff + got.br_blockcount != prev.br_startoff || got.br_startblock + got.br_blockcount != prev.br_startblock)@@ -407,23 +407,23 @@ xfs_iomap_prealloc_size(/* * If the size of the extents is greater than half the maximum extent * length, then use the current offset as the basis. This ensures that -* for large files the preallocation size always extends to MAXEXTLEN -* rather than falling short due to things like stripe unit/width -* alignment of real extents. +* for large files the preallocation size always extends to +* XFS_BMBT_MAX_EXTLEN rather than falling short due to things like stripe +* unit/width alignment of real extents. */ alloc_blocks = plen *2 ; -if (alloc_blocks >MAXEXTLEN ) +if (alloc_blocks >XFS_MAX_BMBT_EXTLEN ) alloc_blocks =XFS_B_TO_FSB (mp, offset); qblocks = alloc_blocks;/* -* MAXEXTLEN is not a power of two value but we round the prealloc down -* to the nearest power of two value after throttling. To prevent the -* round down from unconditionally reducing the maximum supported -* prealloc size, we round up first, apply appropriate throttling, -* round down and cap the value to MAXEXTLEN. +* XFS_BMBT_MAX_EXTLEN is not a power of two value but we round the prealloc +* down to the nearest power of two value after throttling. To prevent +* the round down from unconditionally reducing the maximum supported +* prealloc size, we round up first, apply appropriate throttling, round +* down and cap the value to XFS_BMBT_MAX_EXTLEN. */ - alloc_blocks =XFS_FILEOFF_MIN (roundup_pow_of_two (MAXEXTLEN ), + alloc_blocks =XFS_FILEOFF_MIN (roundup_pow_of_two (XFS_MAX_BMBT_EXTLEN ), alloc_blocks); freesp =percpu_counter_read_positive (&mp->m_fdblocks);@@ -471,14 +471,14 @@ xfs_iomap_prealloc_size(*/ if (alloc_blocks) alloc_blocks =rounddown_pow_of_two (alloc_blocks); -if (alloc_blocks >MAXEXTLEN ) - alloc_blocks =MAXEXTLEN ; +if (alloc_blocks >XFS_MAX_BMBT_EXTLEN ) + alloc_blocks =XFS_MAX_BMBT_EXTLEN ;/* * If we are still trying to allocate more space than is * available, squash the prealloc hard. This can happen if we * have a large file on a small filesystem and the above -* lowspace thresholds are smaller than MAXEXTLEN. +* lowspace thresholds are smaller than XFS_BMBT_MAX_EXTLEN. */ while (alloc_blocks && alloc_blocks >= freesp) alloc_blocks >>=4 ;Signature · 1 line
2.30.2