Extend xattr extent counter to 32-bits
cover Extend xattr extent counter to 32-bits
XFS has a per-inode xattr extent counter which is 16 bits wide. A workload which
1. Creates 1,000,000 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to create 400,000 new 255-byte sized xattrs
causes the following message to be printed on the console,
XFS (loop0): xfs_iflush_int: detected corrupt incore inode 131, total extents = -19916, nblocks = 102937, ptr ffff9ce33b098c00 XFS (loop0): xfs_do_force_shutdown(0x8) called from line 3739 of file fs/xfs/xfs_inode.c. Return address = ffffffffa4a94173
This indicates that we overflowed the 16-bits wide xattr extent counter.
I have been informed that there are instances where a single file has > 100 million hardlinks. With parent pointers being stored in xattr, we will overflow the 16-bits wide xattr extent counter when large number of hardlinks are created.
This patchset also includes the previously posted "Fix log reservation calculation for xattr insert operation" patch as a bug fix. It now replaces the xattr set "mount" and "runtime" reservations with just one static reservation. Hence we don't need the funcationality to calculate maximum sized 'xattr set' reservation separately anymore.
The patches can also be obtained from
https://github.com/chandanr/xfsprogs-dev.git at branch 32bit-anextents-v0.
Chandan Rajendra (2):
xfsprogs: Fix log reservation calculation for xattr insert operation
xfsprogs: Extend attr extent counter to 32-bits
db/bmap.c | 4 +-- db/btdump.c | 2 +- db/check.c | 3 +- db/field.c | 2 -- db/field.h | 1 - db/frag.c | 6 ++-- db/inode.c | 10 ++++--- db/metadump.c | 4 +-- libxfs/xfs_attr.c | 6 +--- libxfs/xfs_format.h | 32 +++++++++++++++------ libxfs/xfs_inode_buf.c | 28 +++++++++++++------ libxfs/xfs_inode_fork.c | 3 +- libxfs/xfs_log_format.h | 5 ++-- libxfs/xfs_log_rlimit.c | 29 ------------------- libxfs/xfs_trans_resv.c | 60 ++++++++++++++++++---------------------- libxfs/xfs_trans_resv.h | 5 +--- libxfs/xfs_trans_space.h | 8 +++++- libxfs/xfs_types.h | 4 +-- logprint/log_misc.c | 9 +++++- logprint/log_print_all.c | 9 +++++- mkfs/xfs_mkfs.c | 1 + repair/attr_repair.c | 2 +- repair/dinode.c | 29 +++++++++++-------- 23 files changed, 140 insertions(+), 122 deletions(-)
Signature · 1 line
2.19.1
PATCH 1/2 xfsprogs: Fix log reservation calculation for xattr insert operation
Log space reservation for xattr insert operation is divided into two parts,
1. Mount time
- Inode
- Superblock for accounting space allocations
- AGF for accounting space used by count, block number, rmap and refcnt btrees.
2. The remaining log space can only be calculated at run time because,
- A local xattr can be large enough to cause a double split of the da btree.
- The value of the xattr can be large enough to be stored in remote blocks. The contents of the remote blocks are not logged.
The log space reservation could be,
- (XFS_DA_NODE_MAXDEPTH + 1) number of blocks. The "+ 1" is required in case xattr is large enough to cause another split of the da btree path.
- BMBT blocks for storing (XFS_DA_NODE_MAXDEPTH + 1) record entries.
- Space for logging blocks of count, block number, rmap and refcnt btrees.
At present, mount time log reservation includes block count required for a single split of the dabtree. The dabtree block count is also taken into account by xfs_attr_calc_size().
Also, AGF log space reservation isn't accounted for.
Due to the reasons mentioned above, log reservation calculation for xattr insert operation gives an incorrect value.
Apart from the above, xfs_log_calc_max_attrsetm_res() passes byte count as an argument to XFS_NEXTENTADD_SPACE_RES() instead of block count.
The above mentioned inconsistencies were discoverd when trying to mount a modified XFS filesystem which uses a 32-bit value as xattr extent counter caused the following warning messages to be printed on the console,
XFS (loop0): Mounting V4 Filesystem
XFS (loop0): Log size 2560 blocks too small, minimum size is 4035 blocks
XFS (loop0): Log size out of supported range.
XFS (loop0): Continuing onwards, but if log hangs are experienced then please report this message in the bug report. XFS (loop0): Ending clean mount
To fix the inconsistencies described above, this commit replaces 'mount' and 'runtime' components with just one static reservation. The new reservation calculates the log space for the worst case possible i.e. it considers,
1. Double split of the da btree.
This happens for large local xattrs.
2. Bmbt blocks required for mapping the contents of a maximum sized (i.e. XATTR_SIZE_MAX bytes in size) remote attribute.
--- libxfs/xfs_attr.c | 6 +--- libxfs/xfs_log_rlimit.c | 29 ------------------- libxfs/xfs_trans_resv.c | 60 ++++++++++++++++++---------------------- libxfs/xfs_trans_resv.h | 5 +--- libxfs/xfs_trans_space.h | 8 +++++- 5 files changed, 36 insertions(+), 72 deletions(-)
libxfs/xfs_attr.c +1 -5
@@ -337,11 +337,7 @@ xfs_attr_set(return error; } - tres.tr_logres =M_RES (mp)->tr_attrsetm.tr_logres + -M_RES (mp)->tr_attrsetrt.tr_logres * - args->total; - tres.tr_logcount =XFS_ATTRSET_LOG_COUNT ; - tres.tr_logflags =XFS_TRANS_PERM_LOG_RES ; + tres =M_RES (mp)->tr_attrset; total = args->total; }else {XFS_STATS_INC (mp, xs_attr_remove);
libxfs/xfs_log_rlimit.c +0 -29
@@ -15,27 +15,6 @@#include "xfs_da_btree.h" #include "xfs_bmap_btree.h" -/* -* Calculate the maximum length in bytes that would be required for a local -* attribute value as large attributes out of line are not logged. -*/ -STATIC int -xfs_log_calc_max_attrsetm_res ( -struct xfs_mount *mp) -{ -int size; -int nblks; - - size =xfs_attr_leaf_entsize_local_max (mp->m_attr_geo->blksize) - -MAXNAMELEN -1 ; - nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); - nblks +=XFS_B_TO_FSB (mp, size); - nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); - -return M_RES (mp)->tr_attrsetm.tr_logres + -M_RES (mp)->tr_attrsetrt.tr_logres * nblks; -} -/* * Iterate over the log space reservation table to figure out and return * the maximum one in terms of the pre-calculated values which were done
@@ -49,9 +28,6 @@ xfs_log_get_max_trans_res(struct xfs_trans_res *resp;struct xfs_trans_res *end_resp;int log_space =0 ; -int attr_space; - - attr_space =xfs_log_calc_max_attrsetm_res (mp); resp = (struct xfs_trans_res *)M_RES (mp); end_resp = (struct xfs_trans_res *)(M_RES (mp) +1 );
@@ -64,11 +40,6 @@ xfs_log_get_max_trans_res( *max_resp = *resp;/* struct copy */ } } - -if (attr_space > log_space) { - *max_resp =M_RES (mp)->tr_attrsetm;/* struct copy */ - max_resp->tr_logres = attr_space; - } }/*
libxfs/xfs_trans_resv.c +27 -33
91 lines not on this page · raw message
libxfs/xfs_trans_resv.h +1 -4
@@ -35,10 +35,7 @@ struct xfs_trans_resv {struct xfs_trans_res tr_writeid;/* write setuid/setgid file */ struct xfs_trans_res tr_attrinval;/* attr fork buffer * invalidation */ -struct xfs_trans_res tr_attrsetm;/* set/create an attribute at -* mount time */ -struct xfs_trans_res tr_attrsetrt;/* set/create an attribute at -* runtime */ +struct xfs_trans_res tr_attrset;/* set/create an attribute */ struct xfs_trans_res tr_attrrm;/* remove an attribute */ struct xfs_trans_res tr_clearagi;/* clear agi unlinked bucket */ struct xfs_trans_res tr_growrtalloc;/* grow realtime allocations */
libxfs/xfs_trans_space.h +7 -1
@@ -38,8 +38,14 @@#define XFS_DAENTER_1B (mp,w) \ ((w) == XFS_DATA_FORK ? (mp)->m_dir_geo->fsbcount : 1) +/* +* xattr set operation can cause the da btree to split twice in the +* worst case. The double split is actually an extra leaf node rather +* than a complete split of blocks in the path from root to a +* leaf. The '1' in the macro below accounts for the extra leaf node. +*/ #define XFS_DAENTER_DBS (mp,w) \ - (XFS_DA_NODE_MAXDEPTH + (((w) == XFS_DATA_FORK) ? 2 : 0)) + (XFS_DA_NODE_MAXDEPTH + (((w) == XFS_DATA_FORK) ? 2 : 1))#define XFS_DAENTER_BLOCKS (mp,w) \ (XFS_DAENTER_1B(mp,w) * XFS_DAENTER_DBS(mp,w))#define XFS_DAENTER_BMAP1B (mp,w) \
Signature · 1 line
2.19.1
PATCH 2/2 xfsprogs: Extend attr extent counter to 32-bits
In reply to Chandan RajendraXFS has a per-inode xattr extent counter which is 16 bits wide. A workload which
1. Creates 1,000,000 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to create 400,000 new 255-byte sized xattrs
causes the following message to be printed on the console,
XFS (loop0): xfs_iflush_int: detected corrupt incore inode 131, total extents = -19916, nblocks = 102937, ptr ffff9ce33b098c00 XFS (loop0): xfs_do_force_shutdown(0x8) called from line 3739 of file fs/xfs/xfs_inode.c. Return address = ffffffffa4a94173
This indicates that we overflowed the 16-bits wide xattr extent counter.
I have been informed that there are instances where a single file has
Quoted 1 line
100 million hardlinks. With parent pointers being stored in xattr,
we will overflow the 16-bits wide xattr extent counter when large number of hardlinks are created.
Hence this commit extends xattr extent counter to 32-bits. It also introduces an incompat flag to prevent older kernels from mounting newer filesystems with 32-bit wide xattr extent counter.
--- db/bmap.c | 4 ++-- db/btdump.c | 2 +- db/check.c | 3 ++- db/field.c | 2 -- db/field.h | 1 - db/frag.c | 6 ++++-- db/inode.c | 10 ++++++---- db/metadump.c | 4 ++-- libxfs/xfs_format.h | 32 ++++++++++++++++++++++++-------- libxfs/xfs_inode_buf.c | 28 ++++++++++++++++++++-------- libxfs/xfs_inode_fork.c | 3 ++- libxfs/xfs_log_format.h | 5 +++-- libxfs/xfs_types.h | 4 ++-- logprint/log_misc.c | 9 ++++++++- logprint/log_print_all.c | 9 ++++++++- mkfs/xfs_mkfs.c | 1 + repair/attr_repair.c | 2 +- repair/dinode.c | 29 ++++++++++++++++++----------- 18 files changed, 104 insertions(+), 50 deletions(-)
db/bmap.c +2 -2
@@ -68,7 +68,7 @@ bmap(ASSERT (fmt ==XFS_DINODE_FMT_LOCAL || fmt ==XFS_DINODE_FMT_EXTENTS || fmt ==XFS_DINODE_FMT_BTREE );if (fmt ==XFS_DINODE_FMT_EXTENTS ) { - nextents =XFS_DFORK_NEXTENTS (dip, whichfork); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); xp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork);for (ep = xp; ep < &xp[nextents] && n < nex; ep++) {if (!bmap_one_extent (ep, &curoffset, eoffset, &n, bep))
@@ -160,7 +160,7 @@ bmap_f( dip = iocur_top->data;if (be32_to_cpu (dip->di_nextents)) dfork =1 ; -if (be16_to_cpu (dip->di_anextents)) +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK )) afork =1 ;pop_cur (); }
db/btdump.c +1 -1
@@ -166,7 +166,7 @@ dump_inode( dip = iocur_top->data;if (attrfork) { -if (!dip->di_anextents || +if (!XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) || dip->di_aformat !=XFS_DINODE_FMT_BTREE ) {dbprintf (_ ("attr fork not in btree format\n" ));return 0 ;
db/check.c +2 -1
@@ -2643,7 +2643,7 @@ process_exinode(xfs_bmbt_rec_t *rp; rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); - *nex =XFS_DFORK_NEXTENTS (dip, whichfork); + *nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);if (*nex <0 || *nex >XFS_DFORK_SIZE (dip, mp, whichfork) /sizeof (xfs_bmbt_rec_t)) {if (!sflag || id->ilist)
@@ -2707,6 +2707,7 @@ process_inode( "dev", "local", "extents", "btree", "uuid" }; + xino.i_mount = mp;libxfs_inode_from_disk (&xino, dip); ino =XFS_AGINO_TO_INO (mp,be32_to_cpu (agf->agf_seqno), agino);
db/field.c +0 -2
@@ -25,8 +25,6 @@#include "symlink.h" const ftattr_t ftattrtab[] = { - {FLDT_AEXTNUM ,"aextnum" , fp_num,"%d" ,SI (bitsz (xfs_aextnum_t)), -FTARG_SIGNED ,NULL ,NULL }, {FLDT_AGBLOCK ,"agblock" , fp_num,"%u" ,SI (bitsz (xfs_agblock_t)),FTARG_DONULL , fa_agblock,NULL }, {FLDT_AGBLOCKNZ ,"agblocknz" , fp_num,"%u" ,SI (bitsz (xfs_agblock_t)),
db/field.h +0 -1
@@ -5,7 +5,6 @@*/ typedef enum fldt { - FLDT_AEXTNUM, FLDT_AGBLOCK, FLDT_AGBLOCKNZ, FLDT_AGF,
db/frag.c +4 -2
@@ -262,9 +262,11 @@ process_exinode( int whichfork) {xfs_bmbt_rec_t *rp; +int nextents; rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); -process_bmbt_reclist (rp,XFS_DFORK_NEXTENTS (dip, whichfork), extmapp); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); +process_bmbt_reclist (rp, nextents, extmapp); } static void
@@ -275,7 +277,7 @@ process_fork(extmap_t *extmap;int nex; - nex =XFS_DFORK_NEXTENTS (dip, whichfork); + nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);if (!nex)return ; extmap =extmap_alloc (nex);
db/inode.c +6 -4
@@ -101,7 +101,7 @@ const field_t inode_core_flds[] = { { "nblocks", FLDT_DRFSBNO, OI(COFF(nblocks)), C1, 0, TYP_NONE }, { "extsize", FLDT_EXTLEN, OI(COFF(extsize)), C1, 0, TYP_NONE }, { "nextents", FLDT_EXTNUM, OI(COFF(nextents)), C1, 0, TYP_NONE }, - { "naextents", FLDT_AEXTNUM, OI(COFF(anextents)), C1, 0, TYP_NONE }, + { "naextents_lo", FLDT_UINT16D, OI(COFF(anextents_lo)), C1, 0, TYP_NONE }, { "forkoff", FLDT_UINT8D, OI(COFF(forkoff)), C1, 0, TYP_NONE }, { "aformat", FLDT_DINODE_FMT, OI(COFF(aformat)), C1, 0, TYP_NONE }, { "dmevmask", FLDT_UINT32X, OI(COFF(dmevmask)), C1, 0, TYP_NONE },
@@ -162,6 +162,7 @@ const field_t inode_v3_flds[] = { { "lsn", FLDT_UINT64X, OI(COFF(lsn)), C1, 0, TYP_NONE }, { "flags2", FLDT_UINT64X, OI(COFF(flags2)), C1, 0, TYP_NONE }, { "cowextsize", FLDT_EXTLEN, OI(COFF(cowextsize)), C1, 0, TYP_NONE }, + { "naextents_hi", FLDT_UINT16D, OI(COFF(anextents_hi)), C1, 0, TYP_NONE }, { "pad2", FLDT_UINT8X, OI(OFF(pad2)), CI(12), FLD_ARRAY|FLD_SKIPALL, TYP_NONE }, { "crtime", FLDT_TIMESTAMP, OI(COFF(crtime)), C1, 0, TYP_NONE }, { "inumber", FLDT_INO, OI(COFF(ino)), C1, 0, TYP_NONE },
@@ -271,7 +272,7 @@ inode_a_bmx_count(return 0 ;ASSERT ((char *)XFS_DFORK_APTR (dip) - (char *)dip ==byteize (startoff));return dip->di_aformat ==XFS_DINODE_FMT_EXTENTS ? -be16_to_cpu (dip->di_anextents) :0 ; +XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) :0 ; } static int
@@ -325,6 +326,7 @@ inode_a_size( {xfs_attr_shortform_t *asf;xfs_dinode_t *dip; +int nextents;ASSERT (startoff ==0 );ASSERT (idx ==0 );
@@ -334,8 +336,8 @@ inode_a_size( asf = (xfs_attr_shortform_t *)XFS_DFORK_APTR (dip);return bitize (be16_to_cpu (asf->hdr.totsize));case XFS_DINODE_FMT_EXTENTS : -return (int )be16_to_cpu (dip->di_anextents) * -bitsz (xfs_bmbt_rec_t); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ); +return nextents *bitsz (xfs_bmbt_rec_t);case XFS_DINODE_FMT_BTREE :return bitize ((int )XFS_DFORK_ASIZE (dip, mp));default :
db/metadump.c +2 -2
@@ -2282,7 +2282,7 @@ process_exinode( whichfork = (itype ==TYP_ATTR ) ?XFS_ATTR_FORK :XFS_DATA_FORK ; - nex =XFS_DFORK_NEXTENTS (dip, whichfork); + nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); used = nex *sizeof (xfs_bmbt_rec_t);if (nex <0 || used >XFS_DFORK_SIZE (dip, mp, whichfork)) {if (show_warnings)
@@ -2335,7 +2335,7 @@ static intprocess_dev_inode ( xfs_dinode_t *dip) { -if (XFS_DFORK_NEXTENTS (dip,XFS_DATA_FORK )) { +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_DATA_FORK )) {if (show_warnings)print_warning ("inode %llu has unexpected extents" , (unsigned long long )cur_ino);
libxfs/xfs_format.h +24 -8
@@ -465,10 +465,13 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_FTYPE (1 << 0)/* filetype in dirent */ #define XFS_SB_FEAT_INCOMPAT_SPINODES (1 << 1)/* sparse inode chunks */ #define XFS_SB_FEAT_INCOMPAT_META_UUID (1 << 2)/* metadata UUID */ -#define XFS_SB_FEAT_INCOMPAT_ALL \ +#define XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR (1 << 3) +#define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ - XFS_SB_FEAT_INCOMPAT_META_UUID) + XFS_SB_FEAT_INCOMPAT_META_UUID | \ + XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR) +#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool
@@ -866,7 +869,7 @@ typedef struct xfs_dinode {__be64 di_nblocks;/* # of direct & btree blocks used */ __be32 di_extsize;/* basic/minimum extent size for file */ __be32 di_nextents;/* number of extents in data fork */ -__be16 di_anextents;/* number of extents in attribute fork*/ +__be16 di_anextents_lo;/* number of extents in attribute fork*/ __u8 di_forkoff;/* attr fork offs, <<3 for 64b align */ __s8 di_aformat;/* format of attr fork's data */ __be32 di_dmevmask;/* DMIG event mask */
@@ -883,7 +886,8 @@ typedef struct xfs_dinode {__be64 di_lsn;/* flush sequence */ __be64 di_flags2;/* more random flags */ __be32 di_cowextsize;/* basic cow extent size for file */ -__u8 di_pad2[12 ];/* more padding for future expansion */ +__be16 di_anextents_hi; +__u8 di_pad2[10 ];/* more padding for future expansion */ /* fields only written to during inode creation */ xfs_timestamp_t di_crtime;/* time created */
@@ -985,10 +989,22 @@ enum xfs_dinode_fmt { ((w) ==XFS_DATA_FORK ? \ (dip)->di_format : \ (dip)->di_aformat) -#define XFS_DFORK_NEXTENTS (dip,w) \ - ((w) == XFS_DATA_FORK ? \ - be32_to_cpu((dip)->di_nextents) : \ - be16_to_cpu((dip)->di_anextents)) + +static inline int32_t XFS_DFORK_NEXTENTS (struct xfs_sb *sbp, +struct xfs_dinode *dip,int whichfork) +{ +int32_t anextents; + +if (whichfork ==XFS_DATA_FORK ) +return be32_to_cpu ((dip)->di_nextents); + + anextents =be16_to_cpu ((dip)->di_anextents_lo); +if (xfs_sb_version_hascrc (sbp)) + anextents |= + ((uint32_t )(be16_to_cpu ((dip)->di_anextents_hi)) <<16 ); + +return anextents; +}/* * For block and character special files the 32bit dev_t is stored at the
libxfs/xfs_inode_buf.c +20 -8
@@ -241,7 +241,8 @@ xfs_inode_from_disk( to->di_nblocks =be64_to_cpu (from->di_nblocks); to->di_extsize =be32_to_cpu (from->di_extsize); to->di_nextents =be32_to_cpu (from->di_nextents); - to->di_anextents =be16_to_cpu (from->di_anextents); + to->di_anextents =XFS_DFORK_NEXTENTS (&ip->i_mount->m_sb, from, +XFS_ATTR_FORK ); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =be32_to_cpu (from->di_dmevmask);
@@ -292,7 +293,8 @@ xfs_inode_to_disk( to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); + to->di_anextents_lo = +cpu_to_be16 ((uint32_t )(from->di_anextents) &0xffff ); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);
@@ -305,6 +307,8 @@ xfs_inode_to_disk( to->di_crtime.t_nsec =cpu_to_be32 (from->di_crtime.tv_nsec); to->di_flags2 =cpu_to_be64 (from->di_flags2); to->di_cowextsize =cpu_to_be32 (from->di_cowextsize); + to->di_anextents_hi = +cpu_to_be16 ((uint32_t )(from->di_anextents) >>16 ); to->di_ino =cpu_to_be64 (ip->i_ino); to->di_lsn =cpu_to_be64 (lsn);memset (to->di_pad2,0 ,sizeof (to->di_pad2));
@@ -343,7 +347,7 @@ xfs_log_dinode_to_disk( to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); + to->di_anextents_lo =cpu_to_be16 (from->di_anextents_lo); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);
@@ -357,6 +361,7 @@ xfs_log_dinode_to_disk( to->di_crtime.t_nsec =cpu_to_be32 (from->di_crtime.t_nsec); to->di_flags2 =cpu_to_be64 (from->di_flags2); to->di_cowextsize =cpu_to_be32 (from->di_cowextsize); + to->di_anextents_hi =cpu_to_be16 (from->di_anextents_hi); to->di_ino =cpu_to_be64 (from->di_ino); to->di_lsn =cpu_to_be64 (from->di_lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2));
@@ -373,7 +378,9 @@ xfs_dinode_verify_fork(struct xfs_mount *mp, int whichfork) { - uint32_t di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork); + uint32_t di_nextents; + + di_nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);switch (XFS_DFORK_FORMAT (dip, whichfork)) {case XFS_DINODE_FMT_LOCAL :
@@ -444,6 +451,9 @@ xfs_dinode_verify(uint16_t flags;uint64_t flags2;uint64_t di_size; +int32_t nextents; +int32_t anextents; +int64_t nblocks;if (dip->di_magic !=cpu_to_be16 (XFS_DINODE_MAGIC ))return __this_address;
@@ -474,10 +484,12 @@ xfs_dinode_verify(if ((S_ISLNK (mode) ||S_ISDIR (mode)) && di_size ==0 )return __this_address; + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_DATA_FORK ); + anextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ); + nblocks =be64_to_cpu (dip->di_nblocks); +/* Fork checks carried over from xfs_iformat_fork */ -if (mode && -be32_to_cpu (dip->di_nextents) +be16_to_cpu (dip->di_anextents) > -be64_to_cpu (dip->di_nblocks)) +if (mode && nextents + anextents > nblocks)return __this_address;if (mode &&XFS_DFORK_BOFF (dip) > mp->m_sb.sb_inodesize)
@@ -534,7 +546,7 @@ xfs_dinode_verify(default :return __this_address; } -if (dip->di_anextents) +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ))return __this_address; }
libxfs/xfs_inode_fork.c +2 -1
@@ -205,9 +205,10 @@ xfs_iformat_extents( int whichfork) {struct xfs_mount *mp = ip->i_mount; +struct xfs_sb *sb = &mp->m_sb;struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);int state =xfs_bmap_fork_to_state (whichfork); -int nex =XFS_DFORK_NEXTENTS (dip, whichfork); +int nex =XFS_DFORK_NEXTENTS (sb, dip, whichfork);int size = nex *sizeof (xfs_bmbt_rec_t);struct xfs_iext_cursor icur;struct xfs_bmbt_rec *dp;
libxfs/xfs_log_format.h +3 -2
@@ -397,7 +397,7 @@ struct xfs_log_dinode {xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ xfs_extnum_t di_nextents;/* number of extents in data fork */ -xfs_aextnum_t di_anextents;/* number of extents in attribute fork*/ +uint16_t di_anextents_lo;/* number of extents in attribute fork*/ uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */
@@ -414,7 +414,8 @@ struct xfs_log_dinode {xfs_lsn_t di_lsn;/* flush sequence */ uint64_t di_flags2;/* more random flags */ uint32_t di_cowextsize;/* basic cow extent size for file */ -uint8_t di_pad2[12 ];/* more padding for future expansion */ +uint16_t di_anextents_hi; +uint8_t di_pad2[10 ];/* more padding for future expansion */ /* fields only written to during inode creation */ xfs_ictimestamp_t di_crtime;/* time created */
libxfs/xfs_types.h +2 -2
@@ -13,7 +13,7 @@ typedef uint32_t xfs_agino_t; /* inode # within allocation grp */typedef uint32_t xfs_extlen_t ;/* extent length in blocks */ typedef uint32_t xfs_agnumber_t ;/* allocation group number */ typedef int32_t xfs_extnum_t ;/* # of extents in a file */ -typedef int16_t xfs_aextnum_t ;/* # extents in an attribute fork */ +typedef int32_t xfs_aextnum_t ;/* # extents in an attribute fork */ typedef int64_t xfs_fsize_t ;/* bytes in a file */ typedef uint64_t xfs_ufsize_t ;/* unsigned bytes in a file */
@@ -60,7 +60,7 @@ typedef void * xfs_failaddr_t;*/ #define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ #define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ -#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */ +#define MAXAEXTNUM ((xfs_aextnum_t)0x7fffffff)/* signed int */ /* * Minimum and maximum blocksize and sectorsize.
logprint/log_misc.c +8 -1
@@ -440,6 +440,8 @@ static voidxlog_print_trans_inode_core ( struct xfs_log_dinode *ip) { +uint32_t naextents; +printf (_ ("INODE CORE\n" )); printf(_("magic 0x%hx mode 0%ho version %d format %d\n"), ip->di_magic, ip->di_mode, (int)ip->di_version,
@@ -451,8 +453,13 @@ xlog_print_trans_inode_core(printf (_ ("size 0x%llx nblocks 0x%llx extsize 0x%x nextents 0x%x\n" ), (unsigned long long )ip->di_size, (unsigned long long )ip->di_nblocks, ip->di_extsize, ip->di_nextents); + + naextents = ip->di_anextents_lo; +if (ip->di_version ==3 ) + naextents |= ((uint32_t )(ip->di_anextents_hi) <<16 ); +printf (_ ("naextents 0x%x forkoff %d dmevmask 0x%x dmstate 0x%hx\n" ), - ip->di_anextents, (int )ip->di_forkoff, ip->di_dmevmask, + naextents, (int )ip->di_forkoff, ip->di_dmevmask, ip->di_dmstate);printf (_ ("flags 0x%x gen 0x%x\n" ), ip->di_flags, ip->di_gen);
logprint/log_print_all.c +8 -1
@@ -240,6 +240,8 @@ STATIC voidxlog_recover_print_inode_core ( struct xfs_log_dinode *di) { +uint32_t anextents; +printf (_ (" CORE inode:\n" ));if (!print_inode)return ;
@@ -252,10 +254,15 @@ xlog_recover_print_inode_core(printf (_ (" atime:%d mtime:%d ctime:%d\n" ), di->di_atime.t_sec, di->di_mtime.t_sec, di->di_ctime.t_sec);printf (_ (" flushiter:%d\n" ), di->di_flushiter); + + anextents = di->di_anextents_lo; +if (di->di_version ==3 ) + anextents |= ((uint32_t )(di->di_anextents_hi) <<16 ); +printf (_ (" size:0x%llx nblks:0x%llx exsize:%d " "nextents:%d anextents:%d\n" ), (unsigned long long ) di->di_size, (unsigned long long )di->di_nblocks, - di->di_extsize, di->di_nextents, (int )di->di_anextents); + di->di_extsize, di->di_nextents, (int )anextents); printf(_(" forkoff:%d dmevmask:0x%x dmstate:%d flags:0x%x " "gen:%u\n"), (int)di->di_forkoff, di->di_dmevmask, (int)di->di_dmstate,
mkfs/xfs_mkfs.c +1 -0
@@ -3018,6 +3018,7 @@ sb_set_features( sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_SPINODES ; } + sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR ; }/*
repair/attr_repair.c +1 -1
@@ -983,7 +983,7 @@ process_longform_attr(if ( bno ==NULLFSBLOCK ) {if (dip->di_aformat ==XFS_DINODE_FMT_EXTENTS && -be16_to_cpu (dip->di_anextents) ==0 ) +XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) ==0 )return (0 );/* the kernel can handle this state */ do_warn( _("block 0 of inode %" PRIu64 " attribute fork is missing\n"),
repair/dinode.c +18 -11
@@ -68,10 +68,12 @@ _("clearing inode %" PRIu64 " attributes\n"), ino_num);fprintf (stderr,_ ("would have cleared inode %" PRIu64" attributes\n" ), ino_num); -if (be16_to_cpu (dino->di_anextents) !=0 ) { +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK ) !=0 ) {if (no_modify)return (1 ); - dino->di_anextents =cpu_to_be16 (0 ); + dino->di_anextents_lo =cpu_to_be16 (0 ); +if (dino->di_version ==3 ) + dino->di_anextents_hi =cpu_to_be16 (0 ); }if (dino->di_aformat !=XFS_DINODE_FMT_EXTENTS ) {
@@ -999,7 +1001,7 @@ process_exinode( lino =XFS_AGINO_TO_INO (mp, agno, ino); rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); *tot =0 ; - numrecs =XFS_DFORK_NEXTENTS (dip, whichfork); + numrecs =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);/* * We've already decided on the maximum number of extents on the inode,
@@ -1836,6 +1838,7 @@ _("bad attr fork offset %d in inode %" PRIu64 ", max=%d\n"),*/ static int process_inode_blocks_and_extents( + xfs_mount_t *mp, xfs_dinode_t *dino, xfs_rfsblock_t nblocks, uint64_t nextents,
@@ -1843,6 +1846,8 @@ process_inode_blocks_and_extents(xfs_ino_t lino, int *dirty) { + int32_t anextents_disk; +if (nblocks !=be64_to_cpu (dino->di_nblocks)) {if (!no_modify) { do_warn(
@@ -1888,19 +1893,21 @@ _("too many attr fork extents (%" PRIu64 ") in inode %" PRIu64 "\n"), anextents, lino);return 1 ; } -if (anextents !=be16_to_cpu (dino->di_anextents)) { + + anextents_disk =XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK ); +if (anextents != anextents_disk) {if (!no_modify) {do_warn (_ ("correcting anextents for inode %" PRIu64", was %d - counted %" PRIu64"\n" ), - lino, -be16_to_cpu (dino->di_anextents), anextents); - dino->di_anextents =cpu_to_be16 (anextents); + lino, anextents_disk, anextents); + dino->di_anextents_lo =cpu_to_be16 (anextents &0xffff ); +if (dino->di_version ==3 ) + dino->di_anextents_hi =cpu_to_be16 (anextents >>16 ); *dirty =1 ; }else {do_warn (_ ("bad anextents %d for inode %" PRIu64", would reset to %" PRIu64"\n" ), -be16_to_cpu (dino->di_anextents), - lino, anextents); + anextents_disk, lino, anextents); } }
@@ -2063,7 +2070,7 @@ process_inode_attr_fork(return 0 ; } - *anextents =be16_to_cpu (dino->di_anextents); + *anextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK );if (*anextents >be64_to_cpu (dino->di_nblocks)) *anextents =1 ;
@@ -2803,7 +2810,7 @@ _("Bad CoW extent size %u on inode %" PRIu64 ", "),/* * correct space counters if required */ -if (process_inode_blocks_and_extents (dino, totblocks + atotblocks, +if (process_inode_blocks_and_extents (mp, dino, totblocks + atotblocks, nextents, anextents, lino, dirty) !=0 ) goto clear_bad_out;
Signature · 1 line
2.19.1
Re: [PATCH 2/2] xfsprogs: Extend attr extent counter to 32-bits
Quoted 22 lines · Chandan Rajendra
XFS has a per-inode xattr extent counter which is 16 bits wide. A workload which
1. Creates 1,000,000 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to create 400,000 new 255-byte sized xattrs
causes the following message to be printed on the console,XFS (loop0):
xfs_iflush_int: detected corrupt incore inode 131, total extents = -19916, nblocks = 102937, ptrffff9ce33b098c00XFS (loop0):xfs_do_force_shutdown(0x8)called from line 3739 of filefs/xfs/xfs_inode.c. Return address =ffffffffa4a94173This indicates that we overflowed the 16-bits wide xattr extent counter.
I have been informed that there are instances where a single file has
Quoted 1 line
100 million hardlinks. With parent pointers being stored in xattr,
we will overflow the 16-bits wide xattr extent counter when large number of hardlinks are created.
Hence this commit extends xattr extent counter to 32-bits. It also introduces an incompat flag to prevent older kernels from mounting newer filesystems with 32-bit wide xattr extent counter.
Signed-off-by Chandan Rajendra <[email protected]>
Quoted 9 lines
--- db/bmap.c | 4 ++-- db/btdump.c | 2 +- db/check.c | 3 ++- db/field.c | 2 -- db/field.h | 1 - db/frag.c | 6 ++++-- db/inode.c | 10 ++++++---- db/metadump.c | 4 ++--
Ooh, xfsprogs patches, thank yoU! :)
Quoted 476 lines
libxfs/xfs_format.h | 32 ++++++++++++++++++++++++-------- libxfs/xfs_inode_buf.c | 28 ++++++++++++++++++++-------- libxfs/xfs_inode_fork.c | 3 ++- libxfs/xfs_log_format.h | 5 +++-- libxfs/xfs_types.h | 4 ++-- logprint/log_misc.c | 9 ++++++++- logprint/log_print_all.c | 9 ++++++++- mkfs/xfs_mkfs.c | 1 + repair/attr_repair.c | 2 +- repair/dinode.c | 29 ++++++++++++++++++----------- 18 files changed, 104 insertions(+), 50 deletions(-)db/bmap.c +2
-2@@ -68,7 +68,7 @@ bmap(ASSERT (fmt ==XFS_DINODE_FMT_LOCAL || fmt ==XFS_DINODE_FMT_EXTENTS || fmt ==XFS_DINODE_FMT_BTREE );if (fmt ==XFS_DINODE_FMT_EXTENTS ) { - nextents =XFS_DFORK_NEXTENTS (dip, whichfork); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); xp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork);for (ep = xp; ep < &xp[nextents] && n < nex; ep++) {if (!bmap_one_extent (ep, &curoffset, eoffset, &n, bep))@@ -160,7 +160,7 @@ bmap_f( dip = iocur_top->data;if (be32_to_cpu (dip->di_nextents)) dfork =1 ; -if (be16_to_cpu (dip->di_anextents)) +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK )) afork =1 ;pop_cur (); }db/btdump.c +1
-1@@ -166,7 +166,7 @@ dump_inode( dip = iocur_top->data;if (attrfork) { -if (!dip->di_anextents || +if (!XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) || dip->di_aformat !=XFS_DINODE_FMT_BTREE ) {dbprintf (_ ("attr fork not in btree format\n" ));return 0 ;db/check.c +2
-1@@ -2643,7 +2643,7 @@ process_exinode(xfs_bmbt_rec_t *rp; rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); - *nex =XFS_DFORK_NEXTENTS (dip, whichfork); + *nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);if (*nex <0 || *nex >XFS_DFORK_SIZE (dip, mp, whichfork) /sizeof (xfs_bmbt_rec_t)) {if (!sflag || id->ilist)@@ -2707,6 +2707,7 @@ process_inode( "dev", "local", "extents", "btree", "uuid" }; + xino.i_mount = mp;libxfs_inode_from_disk (&xino, dip); ino =XFS_AGINO_TO_INO (mp,be32_to_cpu (agf->agf_seqno), agino);db/field.c +0
-2@@ -25,8 +25,6 @@#include "symlink.h" const ftattr_t ftattrtab[] = { - {FLDT_AEXTNUM ,"aextnum" , fp_num,"%d" ,SI (bitsz (xfs_aextnum_t)), -FTARG_SIGNED ,NULL ,NULL }, {FLDT_AGBLOCK ,"agblock" , fp_num,"%u" ,SI (bitsz (xfs_agblock_t)),FTARG_DONULL , fa_agblock,NULL }, {FLDT_AGBLOCKNZ ,"agblocknz" , fp_num,"%u" ,SI (bitsz (xfs_agblock_t)),db/field.h +0
-1@@ -5,7 +5,6 @@*/ typedef enum fldt { - FLDT_AEXTNUM, FLDT_AGBLOCK, FLDT_AGBLOCKNZ, FLDT_AGF,db/frag.c +4
-2@@ -262,9 +262,11 @@ process_exinode( int whichfork) {xfs_bmbt_rec_t *rp; +int nextents; rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); -process_bmbt_reclist (rp,XFS_DFORK_NEXTENTS (dip, whichfork), extmapp); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); +process_bmbt_reclist (rp, nextents, extmapp); } static void@@ -275,7 +277,7 @@ process_fork(extmap_t *extmap;int nex; - nex =XFS_DFORK_NEXTENTS (dip, whichfork); + nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);if (!nex)return ; extmap =extmap_alloc (nex);db/inode.c +6
-4@@ -101,7 +101,7 @@ const field_t inode_core_flds[] = { { "nblocks", FLDT_DRFSBNO, OI(COFF(nblocks)), C1, 0, TYP_NONE }, { "extsize", FLDT_EXTLEN, OI(COFF(extsize)), C1, 0, TYP_NONE }, { "nextents", FLDT_EXTNUM, OI(COFF(nextents)), C1, 0, TYP_NONE }, - { "naextents", FLDT_AEXTNUM, OI(COFF(anextents)), C1, 0, TYP_NONE }, + { "naextents_lo", FLDT_UINT16D, OI(COFF(anextents_lo)), C1, 0, TYP_NONE }, { "forkoff", FLDT_UINT8D, OI(COFF(forkoff)), C1, 0, TYP_NONE }, { "aformat", FLDT_DINODE_FMT, OI(COFF(aformat)), C1, 0, TYP_NONE }, { "dmevmask", FLDT_UINT32X, OI(COFF(dmevmask)), C1, 0, TYP_NONE },@@ -162,6 +162,7 @@ const field_t inode_v3_flds[] = { { "lsn", FLDT_UINT64X, OI(COFF(lsn)), C1, 0, TYP_NONE }, { "flags2", FLDT_UINT64X, OI(COFF(flags2)), C1, 0, TYP_NONE }, { "cowextsize", FLDT_EXTLEN, OI(COFF(cowextsize)), C1, 0, TYP_NONE }, + { "naextents_hi", FLDT_UINT16D, OI(COFF(anextents_hi)), C1, 0, TYP_NONE }, { "pad2", FLDT_UINT8X, OI(OFF(pad2)), CI(12), FLD_ARRAY|FLD_SKIPALL, TYP_NONE }, { "crtime", FLDT_TIMESTAMP, OI(COFF(crtime)), C1, 0, TYP_NONE }, { "inumber", FLDT_INO, OI(COFF(ino)), C1, 0, TYP_NONE },@@ -271,7 +272,7 @@ inode_a_bmx_count(return 0 ;ASSERT ((char *)XFS_DFORK_APTR (dip) - (char *)dip ==byteize (startoff));return dip->di_aformat ==XFS_DINODE_FMT_EXTENTS ? -be16_to_cpu (dip->di_anextents) :0 ; +XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) :0 ; } static int@@ -325,6 +326,7 @@ inode_a_size( {xfs_attr_shortform_t *asf;xfs_dinode_t *dip; +int nextents;ASSERT (startoff ==0 );ASSERT (idx ==0 );@@ -334,8 +336,8 @@ inode_a_size( asf = (xfs_attr_shortform_t *)XFS_DFORK_APTR (dip);return bitize (be16_to_cpu (asf->hdr.totsize));case XFS_DINODE_FMT_EXTENTS : -return (int )be16_to_cpu (dip->di_anextents) * -bitsz (xfs_bmbt_rec_t); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ); +return nextents *bitsz (xfs_bmbt_rec_t);case XFS_DINODE_FMT_BTREE :return bitize ((int )XFS_DFORK_ASIZE (dip, mp));default :db/metadump.c +2
-2@@ -2282,7 +2282,7 @@ process_exinode( whichfork = (itype ==TYP_ATTR ) ?XFS_ATTR_FORK :XFS_DATA_FORK ; - nex =XFS_DFORK_NEXTENTS (dip, whichfork); + nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); used = nex *sizeof (xfs_bmbt_rec_t);if (nex <0 || used >XFS_DFORK_SIZE (dip, mp, whichfork)) {if (show_warnings)@@ -2335,7 +2335,7 @@ static intprocess_dev_inode ( xfs_dinode_t *dip) { -if (XFS_DFORK_NEXTENTS (dip,XFS_DATA_FORK )) { +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_DATA_FORK )) {if (show_warnings)print_warning ("inode %llu has unexpected extents" , (unsigned long long )cur_ino);libxfs/xfs_format.h +24
-8@@ -465,10 +465,13 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_FTYPE (1 << 0)/* filetype in dirent */ #define XFS_SB_FEAT_INCOMPAT_SPINODES (1 << 1)/* sparse inode chunks */ #define XFS_SB_FEAT_INCOMPAT_META_UUID (1 << 2)/* metadata UUID */ -#define XFS_SB_FEAT_INCOMPAT_ALL \ +#define XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR (1 << 3) +#define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ - XFS_SB_FEAT_INCOMPAT_META_UUID) + XFS_SB_FEAT_INCOMPAT_META_UUID | \ + XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR) +#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool@@ -866,7 +869,7 @@ typedef struct xfs_dinode {__be64 di_nblocks;/* # of direct & btree blocks used */ __be32 di_extsize;/* basic/minimum extent size for file */ __be32 di_nextents;/* number of extents in data fork */ -__be16 di_anextents;/* number of extents in attribute fork*/ +__be16 di_anextents_lo;/* number of extents in attribute fork*/ __u8 di_forkoff;/* attr fork offs, <<3 for 64b align */ __s8 di_aformat;/* format of attr fork's data */ __be32 di_dmevmask;/* DMIG event mask */ @@ -883,7 +886,8 @@ typedef struct xfs_dinode {__be64 di_lsn;/* flush sequence */ __be64 di_flags2;/* more random flags */ __be32 di_cowextsize;/* basic cow extent size for file */ -__u8 di_pad2[12 ];/* more padding for future expansion */ +__be16 di_anextents_hi; +__u8 di_pad2[10 ];/* more padding for future expansion */ /* fields only written to during inode creation */ xfs_timestamp_t di_crtime;/* time created */ @@ -985,10 +989,22 @@ enum xfs_dinode_fmt { ((w) ==XFS_DATA_FORK ? \ (dip)->di_format : \ (dip)->di_aformat) -#define XFS_DFORK_NEXTENTS (dip,w) \ - ((w) == XFS_DATA_FORK ? \ - be32_to_cpu((dip)->di_nextents) : \ - be16_to_cpu((dip)->di_anextents)) + +static inline int32_t XFS_DFORK_NEXTENTS (struct xfs_sb *sbp, +struct xfs_dinode *dip,int whichfork) +{ +int32_t anextents; + +if (whichfork ==XFS_DATA_FORK ) +return be32_to_cpu ((dip)->di_nextents); + + anextents =be16_to_cpu ((dip)->di_anextents_lo); +if (xfs_sb_version_hascrc (sbp)) + anextents |= + ((uint32_t )(be16_to_cpu ((dip)->di_anextents_hi)) <<16 ); + +return anextents; +}/* * For block and character special files the 32bit dev_t is stored at the libxfs/xfs_inode_buf.c +20
-8@@ -241,7 +241,8 @@ xfs_inode_from_disk( to->di_nblocks =be64_to_cpu (from->di_nblocks); to->di_extsize =be32_to_cpu (from->di_extsize); to->di_nextents =be32_to_cpu (from->di_nextents); - to->di_anextents =be16_to_cpu (from->di_anextents); + to->di_anextents =XFS_DFORK_NEXTENTS (&ip->i_mount->m_sb, from, +XFS_ATTR_FORK ); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =be32_to_cpu (from->di_dmevmask);@@ -292,7 +293,8 @@ xfs_inode_to_disk( to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); + to->di_anextents_lo = +cpu_to_be16 ((uint32_t )(from->di_anextents) &0xffff ); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);@@ -305,6 +307,8 @@ xfs_inode_to_disk( to->di_crtime.t_nsec =cpu_to_be32 (from->di_crtime.tv_nsec); to->di_flags2 =cpu_to_be64 (from->di_flags2); to->di_cowextsize =cpu_to_be32 (from->di_cowextsize); + to->di_anextents_hi = +cpu_to_be16 ((uint32_t )(from->di_anextents) >>16 ); to->di_ino =cpu_to_be64 (ip->i_ino); to->di_lsn =cpu_to_be64 (lsn);memset (to->di_pad2,0 ,sizeof (to->di_pad2));@@ -343,7 +347,7 @@ xfs_log_dinode_to_disk( to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); + to->di_anextents_lo =cpu_to_be16 (from->di_anextents_lo); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);@@ -357,6 +361,7 @@ xfs_log_dinode_to_disk( to->di_crtime.t_nsec =cpu_to_be32 (from->di_crtime.t_nsec); to->di_flags2 =cpu_to_be64 (from->di_flags2); to->di_cowextsize =cpu_to_be32 (from->di_cowextsize); + to->di_anextents_hi =cpu_to_be16 (from->di_anextents_hi); to->di_ino =cpu_to_be64 (from->di_ino); to->di_lsn =cpu_to_be64 (from->di_lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2));@@ -373,7 +378,9 @@ xfs_dinode_verify_fork(struct xfs_mount *mp, int whichfork) { - uint32_t di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork); + uint32_t di_nextents; + + di_nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);switch (XFS_DFORK_FORMAT (dip, whichfork)) {case XFS_DINODE_FMT_LOCAL :@@ -444,6 +451,9 @@ xfs_dinode_verify(uint16_t flags;uint64_t flags2;uint64_t di_size; +int32_t nextents; +int32_t anextents; +int64_t nblocks;if (dip->di_magic !=cpu_to_be16 (XFS_DINODE_MAGIC ))return __this_address;@@ -474,10 +484,12 @@ xfs_dinode_verify(if ((S_ISLNK (mode) ||S_ISDIR (mode)) && di_size ==0 )return __this_address; + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_DATA_FORK ); + anextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ); + nblocks =be64_to_cpu (dip->di_nblocks); +/* Fork checks carried over from xfs_iformat_fork */ -if (mode && -be32_to_cpu (dip->di_nextents) +be16_to_cpu (dip->di_anextents) > -be64_to_cpu (dip->di_nblocks)) +if (mode && nextents + anextents > nblocks)return __this_address;if (mode &&XFS_DFORK_BOFF (dip) > mp->m_sb.sb_inodesize)@@ -534,7 +546,7 @@ xfs_dinode_verify(default :return __this_address; } -if (dip->di_anextents) +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ))return __this_address; }libxfs/xfs_inode_fork.c +2
-1@@ -205,9 +205,10 @@ xfs_iformat_extents( int whichfork) {struct xfs_mount *mp = ip->i_mount; +struct xfs_sb *sb = &mp->m_sb;struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);int state =xfs_bmap_fork_to_state (whichfork); -int nex =XFS_DFORK_NEXTENTS (dip, whichfork); +int nex =XFS_DFORK_NEXTENTS (sb, dip, whichfork);int size = nex *sizeof (xfs_bmbt_rec_t);struct xfs_iext_cursor icur;struct xfs_bmbt_rec *dp;libxfs/xfs_log_format.h +3
-2@@ -397,7 +397,7 @@ struct xfs_log_dinode {xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ xfs_extnum_t di_nextents;/* number of extents in data fork */ -xfs_aextnum_t di_anextents;/* number of extents in attribute fork*/ +uint16_t di_anextents_lo;/* number of extents in attribute fork*/ uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */ @@ -414,7 +414,8 @@ struct xfs_log_dinode {xfs_lsn_t di_lsn;/* flush sequence */ uint64_t di_flags2;/* more random flags */ uint32_t di_cowextsize;/* basic cow extent size for file */ -uint8_t di_pad2[12 ];/* more padding for future expansion */ +uint16_t di_anextents_hi; +uint8_t di_pad2[10 ];/* more padding for future expansion */ /* fields only written to during inode creation */ xfs_ictimestamp_t di_crtime;/* time created */ libxfs/xfs_types.h +2
-2@@ -13,7 +13,7 @@ typedef uint32_t xfs_agino_t; /* inode # within allocation grp */typedef uint32_t xfs_extlen_t ;/* extent length in blocks */ typedef uint32_t xfs_agnumber_t ;/* allocation group number */ typedef int32_t xfs_extnum_t ;/* # of extents in a file */ -typedef int16_t xfs_aextnum_t ;/* # extents in an attribute fork */ +typedef int32_t xfs_aextnum_t ;/* # extents in an attribute fork */ typedef int64_t xfs_fsize_t ;/* bytes in a file */ typedef uint64_t xfs_ufsize_t ;/* unsigned bytes in a file */ @@ -60,7 +60,7 @@ typedef void * xfs_failaddr_t;*/ #define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ #define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ -#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */ +#define MAXAEXTNUM ((xfs_aextnum_t)0x7fffffff)/* signed int */ /* * Minimum and maximum blocksize and sectorsize. logprint/log_misc.c +8
-1@@ -440,6 +440,8 @@ static voidxlog_print_trans_inode_core ( struct xfs_log_dinode *ip) { +uint32_t naextents; +printf (_ ("INODE CORE\n" )); printf(_("magic 0x%hx mode 0%ho version %d format %d\n"), ip->di_magic, ip->di_mode, (int)ip->di_version,@@ -451,8 +453,13 @@ xlog_print_trans_inode_core(printf (_ ("size 0x%llx nblocks 0x%llx extsize 0x%x nextents 0x%x\n" ), (unsigned long long )ip->di_size, (unsigned long long )ip->di_nblocks, ip->di_extsize, ip->di_nextents); + + naextents = ip->di_anextents_lo; +if (ip->di_version ==3 ) + naextents |= ((uint32_t )(ip->di_anextents_hi) <<16 ); +printf (_ ("naextents 0x%x forkoff %d dmevmask 0x%x dmstate 0x%hx\n" ), - ip->di_anextents, (int )ip->di_forkoff, ip->di_dmevmask, + naextents, (int )ip->di_forkoff, ip->di_dmevmask, ip->di_dmstate);printf (_ ("flags 0x%x gen 0x%x\n" ), ip->di_flags, ip->di_gen);logprint/log_print_all.c +8
-1@@ -240,6 +240,8 @@ STATIC voidxlog_recover_print_inode_core ( struct xfs_log_dinode *di) { +uint32_t anextents; +printf (_ (" CORE inode:\n" ));if (!print_inode)return ;@@ -252,10 +254,15 @@ xlog_recover_print_inode_core(printf (_ (" atime:%d mtime:%d ctime:%d\n" ), di->di_atime.t_sec, di->di_mtime.t_sec, di->di_ctime.t_sec);printf (_ (" flushiter:%d\n" ), di->di_flushiter); + + anextents = di->di_anextents_lo; +if (di->di_version ==3 ) + anextents |= ((uint32_t )(di->di_anextents_hi) <<16 ); +printf (_ (" size:0x%llx nblks:0x%llx exsize:%d " "nextents:%d anextents:%d\n" ), (unsigned long long ) di->di_size, (unsigned long long )di->di_nblocks, - di->di_extsize, di->di_nextents, (int )di->di_anextents); + di->di_extsize, di->di_nextents, (int )anextents); printf(_(" forkoff:%d dmevmask:0x%x dmstate:%d flags:0x%x " "gen:%u\n"), (int)di->di_forkoff, di->di_dmevmask, (int)di->di_dmstate,mkfs/xfs_mkfs.c +0
-0
mkfs/xfs_mkfs.c sb_set_features( · line 3021
Quoted 4 lines
mkfs/xfs_mkfs.c +1
-0@@ -3018,6 +3018,7 @@ sb_set_features( sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_SPINODES ; } + sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR ;
Please don't enable incompat features unconditionally and silently.
--D
Quoted 110 lines
mkfs/xfs_mkfs.c +0
-0}/* repair/attr_repair.c +1
-1@@ -983,7 +983,7 @@ process_longform_attr(if ( bno ==NULLFSBLOCK ) {if (dip->di_aformat ==XFS_DINODE_FMT_EXTENTS && -be16_to_cpu (dip->di_anextents) ==0 ) +XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) ==0 )return (0 );/* the kernel can handle this state */ do_warn( _("block 0 of inode %" PRIu64 " attribute fork is missing\n"),repair/dinode.c +18
-11@@ -68,10 +68,12 @@ _("clearing inode %" PRIu64 " attributes\n"), ino_num);fprintf (stderr,_ ("would have cleared inode %" PRIu64" attributes\n" ), ino_num); -if (be16_to_cpu (dino->di_anextents) !=0 ) { +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK ) !=0 ) {if (no_modify)return (1 ); - dino->di_anextents =cpu_to_be16 (0 ); + dino->di_anextents_lo =cpu_to_be16 (0 ); +if (dino->di_version ==3 ) + dino->di_anextents_hi =cpu_to_be16 (0 ); }if (dino->di_aformat !=XFS_DINODE_FMT_EXTENTS ) {@@ -999,7 +1001,7 @@ process_exinode( lino =XFS_AGINO_TO_INO (mp, agno, ino); rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); *tot =0 ; - numrecs =XFS_DFORK_NEXTENTS (dip, whichfork); + numrecs =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);/* * We've already decided on the maximum number of extents on the inode, @@ -1836,6 +1838,7 @@ _("bad attr fork offset %d in inode %" PRIu64 ", max=%d\n"),*/ static int process_inode_blocks_and_extents( + xfs_mount_t *mp, xfs_dinode_t *dino, xfs_rfsblock_t nblocks, uint64_t nextents,@@ -1843,6 +1846,8 @@ process_inode_blocks_and_extents(xfs_ino_t lino, int *dirty) { + int32_t anextents_disk; +if (nblocks !=be64_to_cpu (dino->di_nblocks)) {if (!no_modify) { do_warn(@@ -1888,19 +1893,21 @@ _("too many attr fork extents (%" PRIu64 ") in inode %" PRIu64 "\n"), anextents, lino);return 1 ; } -if (anextents !=be16_to_cpu (dino->di_anextents)) { + + anextents_disk =XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK ); +if (anextents != anextents_disk) {if (!no_modify) {do_warn (_ ("correcting anextents for inode %" PRIu64", was %d - counted %" PRIu64"\n" ), - lino, -be16_to_cpu (dino->di_anextents), anextents); - dino->di_anextents =cpu_to_be16 (anextents); + lino, anextents_disk, anextents); + dino->di_anextents_lo =cpu_to_be16 (anextents &0xffff ); +if (dino->di_version ==3 ) + dino->di_anextents_hi =cpu_to_be16 (anextents >>16 ); *dirty =1 ; }else {do_warn (_ ("bad anextents %d for inode %" PRIu64", would reset to %" PRIu64"\n" ), -be16_to_cpu (dino->di_anextents), - lino, anextents); + anextents_disk, lino, anextents); } }@@ -2063,7 +2070,7 @@ process_inode_attr_fork(return 0 ; } - *anextents =be16_to_cpu (dino->di_anextents); + *anextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK );if (*anextents >be64_to_cpu (dino->di_nblocks)) *anextents =1 ;@@ -2803,7 +2810,7 @@ _("Bad CoW extent size %u on inode %" PRIu64 ", "),/* * correct space counters if required */ -if (process_inode_blocks_and_extents (dino, totblocks + atotblocks, +if (process_inode_blocks_and_extents (mp, dino, totblocks + atotblocks, nextents, anextents, lino, dirty) !=0 ) goto clear_bad_out;Signature · 1 line
2.19.1
Re: [PATCH 2/2] xfsprogs: Extend attr extent counter to 32-bits
Quoted 515 lines · Darrick J. Wong
Quoted 22 lines · Chandan Rajendra
XFS has a per-inode xattr extent counter which is 16 bits wide. A workload which
1. Creates 1,000,000 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to create 400,000 new 255-byte sized xattrs
causes the following message to be printed on the console,XFS (loop0):
xfs_iflush_int: detected corrupt incore inode 131, total extents = -19916, nblocks = 102937, ptrffff9ce33b098c00XFS (loop0):xfs_do_force_shutdown(0x8)called from line 3739 of filefs/xfs/xfs_inode.c. Return address =ffffffffa4a94173This indicates that we overflowed the 16-bits wide xattr extent counter.
I have been informed that there are instances where a single file has
Quoted 1 line
100 million hardlinks. With parent pointers being stored in xattr,
we will overflow the 16-bits wide xattr extent counter when large number of hardlinks are created.
Hence this commit extends xattr extent counter to 32-bits. It also introduces an incompat flag to prevent older kernels from mounting newer filesystems with 32-bit wide xattr extent counter.
Signed-off-by Chandan Rajendra <[email protected]>Quoted 9 lines
--- db/bmap.c | 4 ++-- db/btdump.c | 2 +- db/check.c | 3 ++- db/field.c | 2 -- db/field.h | 1 - db/frag.c | 6 ++++-- db/inode.c | 10 ++++++---- db/metadump.c | 4 ++--Ooh, xfsprogs patches, thank yoU! :)
Quoted 479 lines
libxfs/xfs_format.h | 32 ++++++++++++++++++++++++-------- libxfs/xfs_inode_buf.c | 28 ++++++++++++++++++++-------- libxfs/xfs_inode_fork.c | 3 ++- libxfs/xfs_log_format.h | 5 +++-- libxfs/xfs_types.h | 4 ++-- logprint/log_misc.c | 9 ++++++++- logprint/log_print_all.c | 9 ++++++++- mkfs/xfs_mkfs.c | 1 + repair/attr_repair.c | 2 +- repair/dinode.c | 29 ++++++++++++++++++----------- 18 files changed, 104 insertions(+), 50 deletions(-)db/bmap.c +2
-2@@ -68,7 +68,7 @@ bmap(ASSERT (fmt ==XFS_DINODE_FMT_LOCAL || fmt ==XFS_DINODE_FMT_EXTENTS || fmt ==XFS_DINODE_FMT_BTREE );if (fmt ==XFS_DINODE_FMT_EXTENTS ) { - nextents =XFS_DFORK_NEXTENTS (dip, whichfork); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); xp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork);for (ep = xp; ep < &xp[nextents] && n < nex; ep++) {if (!bmap_one_extent (ep, &curoffset, eoffset, &n, bep))@@ -160,7 +160,7 @@ bmap_f( dip = iocur_top->data;if (be32_to_cpu (dip->di_nextents)) dfork =1 ; -if (be16_to_cpu (dip->di_anextents)) +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK )) afork =1 ;pop_cur (); }db/btdump.c +1
-1@@ -166,7 +166,7 @@ dump_inode( dip = iocur_top->data;if (attrfork) { -if (!dip->di_anextents || +if (!XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) || dip->di_aformat !=XFS_DINODE_FMT_BTREE ) {dbprintf (_ ("attr fork not in btree format\n" ));return 0 ;db/check.c +2
-1@@ -2643,7 +2643,7 @@ process_exinode(xfs_bmbt_rec_t *rp; rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); - *nex =XFS_DFORK_NEXTENTS (dip, whichfork); + *nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);if (*nex <0 || *nex >XFS_DFORK_SIZE (dip, mp, whichfork) /sizeof (xfs_bmbt_rec_t)) {if (!sflag || id->ilist)@@ -2707,6 +2707,7 @@ process_inode( "dev", "local", "extents", "btree", "uuid" }; + xino.i_mount = mp;libxfs_inode_from_disk (&xino, dip); ino =XFS_AGINO_TO_INO (mp,be32_to_cpu (agf->agf_seqno), agino);db/field.c +0
-2@@ -25,8 +25,6 @@#include "symlink.h" const ftattr_t ftattrtab[] = { - {FLDT_AEXTNUM ,"aextnum" , fp_num,"%d" ,SI (bitsz (xfs_aextnum_t)), -FTARG_SIGNED ,NULL ,NULL }, {FLDT_AGBLOCK ,"agblock" , fp_num,"%u" ,SI (bitsz (xfs_agblock_t)),FTARG_DONULL , fa_agblock,NULL }, {FLDT_AGBLOCKNZ ,"agblocknz" , fp_num,"%u" ,SI (bitsz (xfs_agblock_t)),db/field.h +0
-1@@ -5,7 +5,6 @@*/ typedef enum fldt { - FLDT_AEXTNUM, FLDT_AGBLOCK, FLDT_AGBLOCKNZ, FLDT_AGF,db/frag.c +4
-2@@ -262,9 +262,11 @@ process_exinode( int whichfork) {xfs_bmbt_rec_t *rp; +int nextents; rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); -process_bmbt_reclist (rp,XFS_DFORK_NEXTENTS (dip, whichfork), extmapp); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); +process_bmbt_reclist (rp, nextents, extmapp); } static void@@ -275,7 +277,7 @@ process_fork(extmap_t *extmap;int nex; - nex =XFS_DFORK_NEXTENTS (dip, whichfork); + nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);if (!nex)return ; extmap =extmap_alloc (nex);db/inode.c +6
-4@@ -101,7 +101,7 @@ const field_t inode_core_flds[] = { { "nblocks", FLDT_DRFSBNO, OI(COFF(nblocks)), C1, 0, TYP_NONE }, { "extsize", FLDT_EXTLEN, OI(COFF(extsize)), C1, 0, TYP_NONE }, { "nextents", FLDT_EXTNUM, OI(COFF(nextents)), C1, 0, TYP_NONE }, - { "naextents", FLDT_AEXTNUM, OI(COFF(anextents)), C1, 0, TYP_NONE }, + { "naextents_lo", FLDT_UINT16D, OI(COFF(anextents_lo)), C1, 0, TYP_NONE }, { "forkoff", FLDT_UINT8D, OI(COFF(forkoff)), C1, 0, TYP_NONE }, { "aformat", FLDT_DINODE_FMT, OI(COFF(aformat)), C1, 0, TYP_NONE }, { "dmevmask", FLDT_UINT32X, OI(COFF(dmevmask)), C1, 0, TYP_NONE },@@ -162,6 +162,7 @@ const field_t inode_v3_flds[] = { { "lsn", FLDT_UINT64X, OI(COFF(lsn)), C1, 0, TYP_NONE }, { "flags2", FLDT_UINT64X, OI(COFF(flags2)), C1, 0, TYP_NONE }, { "cowextsize", FLDT_EXTLEN, OI(COFF(cowextsize)), C1, 0, TYP_NONE }, + { "naextents_hi", FLDT_UINT16D, OI(COFF(anextents_hi)), C1, 0, TYP_NONE }, { "pad2", FLDT_UINT8X, OI(OFF(pad2)), CI(12), FLD_ARRAY|FLD_SKIPALL, TYP_NONE }, { "crtime", FLDT_TIMESTAMP, OI(COFF(crtime)), C1, 0, TYP_NONE }, { "inumber", FLDT_INO, OI(COFF(ino)), C1, 0, TYP_NONE },@@ -271,7 +272,7 @@ inode_a_bmx_count(return 0 ;ASSERT ((char *)XFS_DFORK_APTR (dip) - (char *)dip ==byteize (startoff));return dip->di_aformat ==XFS_DINODE_FMT_EXTENTS ? -be16_to_cpu (dip->di_anextents) :0 ; +XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) :0 ; } static int@@ -325,6 +326,7 @@ inode_a_size( {xfs_attr_shortform_t *asf;xfs_dinode_t *dip; +int nextents;ASSERT (startoff ==0 );ASSERT (idx ==0 );@@ -334,8 +336,8 @@ inode_a_size( asf = (xfs_attr_shortform_t *)XFS_DFORK_APTR (dip);return bitize (be16_to_cpu (asf->hdr.totsize));case XFS_DINODE_FMT_EXTENTS : -return (int )be16_to_cpu (dip->di_anextents) * -bitsz (xfs_bmbt_rec_t); + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ); +return nextents *bitsz (xfs_bmbt_rec_t);case XFS_DINODE_FMT_BTREE :return bitize ((int )XFS_DFORK_ASIZE (dip, mp));default :db/metadump.c +2
-2@@ -2282,7 +2282,7 @@ process_exinode( whichfork = (itype ==TYP_ATTR ) ?XFS_ATTR_FORK :XFS_DATA_FORK ; - nex =XFS_DFORK_NEXTENTS (dip, whichfork); + nex =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork); used = nex *sizeof (xfs_bmbt_rec_t);if (nex <0 || used >XFS_DFORK_SIZE (dip, mp, whichfork)) {if (show_warnings)@@ -2335,7 +2335,7 @@ static intprocess_dev_inode ( xfs_dinode_t *dip) { -if (XFS_DFORK_NEXTENTS (dip,XFS_DATA_FORK )) { +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_DATA_FORK )) {if (show_warnings)print_warning ("inode %llu has unexpected extents" , (unsigned long long )cur_ino);libxfs/xfs_format.h +24
-8@@ -465,10 +465,13 @@ xfs_sb_has_ro_compat_feature(#define XFS_SB_FEAT_INCOMPAT_FTYPE (1 << 0)/* filetype in dirent */ #define XFS_SB_FEAT_INCOMPAT_SPINODES (1 << 1)/* sparse inode chunks */ #define XFS_SB_FEAT_INCOMPAT_META_UUID (1 << 2)/* metadata UUID */ -#define XFS_SB_FEAT_INCOMPAT_ALL \ +#define XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR (1 << 3) +#define XFS_SB_FEAT_INCOMPAT_ALL \ (XFS_SB_FEAT_INCOMPAT_FTYPE| \ XFS_SB_FEAT_INCOMPAT_SPINODES| \ - XFS_SB_FEAT_INCOMPAT_META_UUID) + XFS_SB_FEAT_INCOMPAT_META_UUID | \ + XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR) +#define XFS_SB_FEAT_INCOMPAT_UNKNOWN ~XFS_SB_FEAT_INCOMPAT_ALL static inline bool@@ -866,7 +869,7 @@ typedef struct xfs_dinode {__be64 di_nblocks;/* # of direct & btree blocks used */ __be32 di_extsize;/* basic/minimum extent size for file */ __be32 di_nextents;/* number of extents in data fork */ -__be16 di_anextents;/* number of extents in attribute fork*/ +__be16 di_anextents_lo;/* number of extents in attribute fork*/ __u8 di_forkoff;/* attr fork offs, <<3 for 64b align */ __s8 di_aformat;/* format of attr fork's data */ __be32 di_dmevmask;/* DMIG event mask */ @@ -883,7 +886,8 @@ typedef struct xfs_dinode {__be64 di_lsn;/* flush sequence */ __be64 di_flags2;/* more random flags */ __be32 di_cowextsize;/* basic cow extent size for file */ -__u8 di_pad2[12 ];/* more padding for future expansion */ +__be16 di_anextents_hi; +__u8 di_pad2[10 ];/* more padding for future expansion */ /* fields only written to during inode creation */ xfs_timestamp_t di_crtime;/* time created */ @@ -985,10 +989,22 @@ enum xfs_dinode_fmt { ((w) ==XFS_DATA_FORK ? \ (dip)->di_format : \ (dip)->di_aformat) -#define XFS_DFORK_NEXTENTS (dip,w) \ - ((w) == XFS_DATA_FORK ? \ - be32_to_cpu((dip)->di_nextents) : \ - be16_to_cpu((dip)->di_anextents)) + +static inline int32_t XFS_DFORK_NEXTENTS (struct xfs_sb *sbp, +struct xfs_dinode *dip,int whichfork) +{ +int32_t anextents; + +if (whichfork ==XFS_DATA_FORK ) +return be32_to_cpu ((dip)->di_nextents); + + anextents =be16_to_cpu ((dip)->di_anextents_lo); +if (xfs_sb_version_hascrc (sbp)) + anextents |= + ((uint32_t )(be16_to_cpu ((dip)->di_anextents_hi)) <<16 ); + +return anextents; +}/* * For block and character special files the 32bit dev_t is stored at the libxfs/xfs_inode_buf.c +20
-8@@ -241,7 +241,8 @@ xfs_inode_from_disk( to->di_nblocks =be64_to_cpu (from->di_nblocks); to->di_extsize =be32_to_cpu (from->di_extsize); to->di_nextents =be32_to_cpu (from->di_nextents); - to->di_anextents =be16_to_cpu (from->di_anextents); + to->di_anextents =XFS_DFORK_NEXTENTS (&ip->i_mount->m_sb, from, +XFS_ATTR_FORK ); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =be32_to_cpu (from->di_dmevmask);@@ -292,7 +293,8 @@ xfs_inode_to_disk( to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); + to->di_anextents_lo = +cpu_to_be16 ((uint32_t )(from->di_anextents) &0xffff ); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);@@ -305,6 +307,8 @@ xfs_inode_to_disk( to->di_crtime.t_nsec =cpu_to_be32 (from->di_crtime.tv_nsec); to->di_flags2 =cpu_to_be64 (from->di_flags2); to->di_cowextsize =cpu_to_be32 (from->di_cowextsize); + to->di_anextents_hi = +cpu_to_be16 ((uint32_t )(from->di_anextents) >>16 ); to->di_ino =cpu_to_be64 (ip->i_ino); to->di_lsn =cpu_to_be64 (lsn);memset (to->di_pad2,0 ,sizeof (to->di_pad2));@@ -343,7 +347,7 @@ xfs_log_dinode_to_disk( to->di_nblocks =cpu_to_be64 (from->di_nblocks); to->di_extsize =cpu_to_be32 (from->di_extsize); to->di_nextents =cpu_to_be32 (from->di_nextents); - to->di_anextents =cpu_to_be16 (from->di_anextents); + to->di_anextents_lo =cpu_to_be16 (from->di_anextents_lo); to->di_forkoff = from->di_forkoff; to->di_aformat = from->di_aformat; to->di_dmevmask =cpu_to_be32 (from->di_dmevmask);@@ -357,6 +361,7 @@ xfs_log_dinode_to_disk( to->di_crtime.t_nsec =cpu_to_be32 (from->di_crtime.t_nsec); to->di_flags2 =cpu_to_be64 (from->di_flags2); to->di_cowextsize =cpu_to_be32 (from->di_cowextsize); + to->di_anextents_hi =cpu_to_be16 (from->di_anextents_hi); to->di_ino =cpu_to_be64 (from->di_ino); to->di_lsn =cpu_to_be64 (from->di_lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2));@@ -373,7 +378,9 @@ xfs_dinode_verify_fork(struct xfs_mount *mp, int whichfork) { - uint32_t di_nextents = XFS_DFORK_NEXTENTS(dip, whichfork); + uint32_t di_nextents; + + di_nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);switch (XFS_DFORK_FORMAT (dip, whichfork)) {case XFS_DINODE_FMT_LOCAL :@@ -444,6 +451,9 @@ xfs_dinode_verify(uint16_t flags;uint64_t flags2;uint64_t di_size; +int32_t nextents; +int32_t anextents; +int64_t nblocks;if (dip->di_magic !=cpu_to_be16 (XFS_DINODE_MAGIC ))return __this_address;@@ -474,10 +484,12 @@ xfs_dinode_verify(if ((S_ISLNK (mode) ||S_ISDIR (mode)) && di_size ==0 )return __this_address; + nextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_DATA_FORK ); + anextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ); + nblocks =be64_to_cpu (dip->di_nblocks); +/* Fork checks carried over from xfs_iformat_fork */ -if (mode && -be32_to_cpu (dip->di_nextents) +be16_to_cpu (dip->di_anextents) > -be64_to_cpu (dip->di_nblocks)) +if (mode && nextents + anextents > nblocks)return __this_address;if (mode &&XFS_DFORK_BOFF (dip) > mp->m_sb.sb_inodesize)@@ -534,7 +546,7 @@ xfs_dinode_verify(default :return __this_address; } -if (dip->di_anextents) +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ))return __this_address; }libxfs/xfs_inode_fork.c +2
-1@@ -205,9 +205,10 @@ xfs_iformat_extents( int whichfork) {struct xfs_mount *mp = ip->i_mount; +struct xfs_sb *sb = &mp->m_sb;struct xfs_ifork *ifp =XFS_IFORK_PTR (ip, whichfork);int state =xfs_bmap_fork_to_state (whichfork); -int nex =XFS_DFORK_NEXTENTS (dip, whichfork); +int nex =XFS_DFORK_NEXTENTS (sb, dip, whichfork);int size = nex *sizeof (xfs_bmbt_rec_t);struct xfs_iext_cursor icur;struct xfs_bmbt_rec *dp;libxfs/xfs_log_format.h +3
-2@@ -397,7 +397,7 @@ struct xfs_log_dinode {xfs_rfsblock_t di_nblocks;/* # of direct & btree blocks used */ xfs_extlen_t di_extsize;/* basic/minimum extent size for file */ xfs_extnum_t di_nextents;/* number of extents in data fork */ -xfs_aextnum_t di_anextents;/* number of extents in attribute fork*/ +uint16_t di_anextents_lo;/* number of extents in attribute fork*/ uint8_t di_forkoff;/* attr fork offs, <<3 for 64b align */ int8_t di_aformat;/* format of attr fork's data */ uint32_t di_dmevmask;/* DMIG event mask */ @@ -414,7 +414,8 @@ struct xfs_log_dinode {xfs_lsn_t di_lsn;/* flush sequence */ uint64_t di_flags2;/* more random flags */ uint32_t di_cowextsize;/* basic cow extent size for file */ -uint8_t di_pad2[12 ];/* more padding for future expansion */ +uint16_t di_anextents_hi; +uint8_t di_pad2[10 ];/* more padding for future expansion */ /* fields only written to during inode creation */ xfs_ictimestamp_t di_crtime;/* time created */ libxfs/xfs_types.h +2
-2@@ -13,7 +13,7 @@ typedef uint32_t xfs_agino_t; /* inode # within allocation grp */typedef uint32_t xfs_extlen_t ;/* extent length in blocks */ typedef uint32_t xfs_agnumber_t ;/* allocation group number */ typedef int32_t xfs_extnum_t ;/* # of extents in a file */ -typedef int16_t xfs_aextnum_t ;/* # extents in an attribute fork */ +typedef int32_t xfs_aextnum_t ;/* # extents in an attribute fork */ typedef int64_t xfs_fsize_t ;/* bytes in a file */ typedef uint64_t xfs_ufsize_t ;/* unsigned bytes in a file */ @@ -60,7 +60,7 @@ typedef void * xfs_failaddr_t;*/ #define MAXEXTLEN ((xfs_extlen_t)0x001fffff)/* 21 bits */ #define MAXEXTNUM ((xfs_extnum_t)0x7fffffff)/* signed int */ -#define MAXAEXTNUM ((xfs_aextnum_t)0x7fff)/* signed short */ +#define MAXAEXTNUM ((xfs_aextnum_t)0x7fffffff)/* signed int */ /* * Minimum and maximum blocksize and sectorsize. logprint/log_misc.c +8
-1@@ -440,6 +440,8 @@ static voidxlog_print_trans_inode_core ( struct xfs_log_dinode *ip) { +uint32_t naextents; +printf (_ ("INODE CORE\n" )); printf(_("magic 0x%hx mode 0%ho version %d format %d\n"), ip->di_magic, ip->di_mode, (int)ip->di_version,@@ -451,8 +453,13 @@ xlog_print_trans_inode_core(printf (_ ("size 0x%llx nblocks 0x%llx extsize 0x%x nextents 0x%x\n" ), (unsigned long long )ip->di_size, (unsigned long long )ip->di_nblocks, ip->di_extsize, ip->di_nextents); + + naextents = ip->di_anextents_lo; +if (ip->di_version ==3 ) + naextents |= ((uint32_t )(ip->di_anextents_hi) <<16 ); +printf (_ ("naextents 0x%x forkoff %d dmevmask 0x%x dmstate 0x%hx\n" ), - ip->di_anextents, (int )ip->di_forkoff, ip->di_dmevmask, + naextents, (int )ip->di_forkoff, ip->di_dmevmask, ip->di_dmstate);printf (_ ("flags 0x%x gen 0x%x\n" ), ip->di_flags, ip->di_gen);logprint/log_print_all.c +8
-1@@ -240,6 +240,8 @@ STATIC voidxlog_recover_print_inode_core ( struct xfs_log_dinode *di) { +uint32_t anextents; +printf (_ (" CORE inode:\n" ));if (!print_inode)return ;@@ -252,10 +254,15 @@ xlog_recover_print_inode_core(printf (_ (" atime:%d mtime:%d ctime:%d\n" ), di->di_atime.t_sec, di->di_mtime.t_sec, di->di_ctime.t_sec);printf (_ (" flushiter:%d\n" ), di->di_flushiter); + + anextents = di->di_anextents_lo; +if (di->di_version ==3 ) + anextents |= ((uint32_t )(di->di_anextents_hi) <<16 ); +printf (_ (" size:0x%llx nblks:0x%llx exsize:%d " "nextents:%d anextents:%d\n" ), (unsigned long long ) di->di_size, (unsigned long long )di->di_nblocks, - di->di_extsize, di->di_nextents, (int )di->di_anextents); + di->di_extsize, di->di_nextents, (int )anextents); printf(_(" forkoff:%d dmevmask:0x%x dmstate:%d flags:0x%x " "gen:%u\n"), (int)di->di_forkoff, di->di_dmevmask, (int)di->di_dmstate,mkfs/xfs_mkfs.c +0
-0@@ -3018,6 +3018,7 @@ sb_set_features( sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_SPINODES ; }
Quoted 3 lines
mkfs/xfs_mkfs.c sb_set_features( · line 3021
Quoted 1 line
mkfs/xfs_mkfs.c +1
-0+ sbp->sb_features_incompat |=XFS_SB_FEAT_INCOMPAT_32BIT_AEXT_CNTR ;Please don't enable incompat features unconditionally and silently.
Sure. I will implement the new RO-compat method that Dave has suggested.
Quoted 111 lines
--D
Quoted 107 lines
mkfs/xfs_mkfs.c +0
-0}/* repair/attr_repair.c +1
-1@@ -983,7 +983,7 @@ process_longform_attr(if ( bno ==NULLFSBLOCK ) {if (dip->di_aformat ==XFS_DINODE_FMT_EXTENTS && -be16_to_cpu (dip->di_anextents) ==0 ) +XFS_DFORK_NEXTENTS (&mp->m_sb, dip,XFS_ATTR_FORK ) ==0 )return (0 );/* the kernel can handle this state */ do_warn( _("block 0 of inode %" PRIu64 " attribute fork is missing\n"),repair/dinode.c +18
-11@@ -68,10 +68,12 @@ _("clearing inode %" PRIu64 " attributes\n"), ino_num);fprintf (stderr,_ ("would have cleared inode %" PRIu64" attributes\n" ), ino_num); -if (be16_to_cpu (dino->di_anextents) !=0 ) { +if (XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK ) !=0 ) {if (no_modify)return (1 ); - dino->di_anextents =cpu_to_be16 (0 ); + dino->di_anextents_lo =cpu_to_be16 (0 ); +if (dino->di_version ==3 ) + dino->di_anextents_hi =cpu_to_be16 (0 ); }if (dino->di_aformat !=XFS_DINODE_FMT_EXTENTS ) {@@ -999,7 +1001,7 @@ process_exinode( lino =XFS_AGINO_TO_INO (mp, agno, ino); rp = (xfs_bmbt_rec_t *)XFS_DFORK_PTR (dip, whichfork); *tot =0 ; - numrecs =XFS_DFORK_NEXTENTS (dip, whichfork); + numrecs =XFS_DFORK_NEXTENTS (&mp->m_sb, dip, whichfork);/* * We've already decided on the maximum number of extents on the inode, @@ -1836,6 +1838,7 @@ _("bad attr fork offset %d in inode %" PRIu64 ", max=%d\n"),*/ static int process_inode_blocks_and_extents( + xfs_mount_t *mp, xfs_dinode_t *dino, xfs_rfsblock_t nblocks, uint64_t nextents,@@ -1843,6 +1846,8 @@ process_inode_blocks_and_extents(xfs_ino_t lino, int *dirty) { + int32_t anextents_disk; +if (nblocks !=be64_to_cpu (dino->di_nblocks)) {if (!no_modify) { do_warn(@@ -1888,19 +1893,21 @@ _("too many attr fork extents (%" PRIu64 ") in inode %" PRIu64 "\n"), anextents, lino);return 1 ; } -if (anextents !=be16_to_cpu (dino->di_anextents)) { + + anextents_disk =XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK ); +if (anextents != anextents_disk) {if (!no_modify) {do_warn (_ ("correcting anextents for inode %" PRIu64", was %d - counted %" PRIu64"\n" ), - lino, -be16_to_cpu (dino->di_anextents), anextents); - dino->di_anextents =cpu_to_be16 (anextents); + lino, anextents_disk, anextents); + dino->di_anextents_lo =cpu_to_be16 (anextents &0xffff ); +if (dino->di_version ==3 ) + dino->di_anextents_hi =cpu_to_be16 (anextents >>16 ); *dirty =1 ; }else {do_warn (_ ("bad anextents %d for inode %" PRIu64", would reset to %" PRIu64"\n" ), -be16_to_cpu (dino->di_anextents), - lino, anextents); + anextents_disk, lino, anextents); } }@@ -2063,7 +2070,7 @@ process_inode_attr_fork(return 0 ; } - *anextents =be16_to_cpu (dino->di_anextents); + *anextents =XFS_DFORK_NEXTENTS (&mp->m_sb, dino,XFS_ATTR_FORK );if (*anextents >be64_to_cpu (dino->di_nblocks)) *anextents =1 ;@@ -2803,7 +2810,7 @@ _("Bad CoW extent size %u on inode %" PRIu64 ", "),/* * correct space counters if required */ -if (process_inode_blocks_and_extents (dino, totblocks + atotblocks, +if (process_inode_blocks_and_extents (mp, dino, totblocks + atotblocks, nextents, anextents, lino, dirty) !=0 ) goto clear_bad_out;
Signature · 1 line
chandan