xfsprogs: CRC write support for xfs_db
v3 cover xfsprogs: CRC write support for xfs_db
Hi folks,
This is an update version of the patch series posted here previously:
http://oss.sgi.com/archives/xfs/2013-09/msg00805.html
Several of the kernel sync patches have been merged, but there a several more been added (the kernel is a moving target) to bring the code bases back into sync with the current kernel code.
Otherwise the patch set is mostly unchanged as nobody has made any comments on the libxfs/xfs_db code changes in the series yet.
Cheers,
Dave.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 1/30 xfsprogs: fix automatic dependency generation
From: Dave Chinner <[email protected]>
Adding are removing a header file does not result in dependency regeneration like it should. make clean will rebuild the dependencies, but a normal make won't. Fix it.
--- include/buildrules | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
include/buildrules +16 -4
@@ -79,18 +79,30 @@ endif # _BUILDRULES_INCLUDED_ $(_FORCE): # dependency build is automatic, relies on gcc -MM to generate. +# +# This is a bit messy. It regenerates the depenencies on each build so +# that we catch files being added and removed. There are other ways of doing +# this (e.g. per-file dependency files) but that requires more in-depth changes +# to the build system. Compile time is not an issue for us, so the +# rebuild on every make invocation isn't a problem we need to care about. Just +# do it silently so it doesn't make the build unnecessarily noisy. + .PHONY : depend ltdepend install-qa MAKEDEP := $(MAKEDEPEND) $(CFLAGS) -ltdepend: .ltdep +ltdepend: rmltdep .ltdep + +rmltdep: + @rm -f .ltdep .ltdep: $(CFILES) $(HFILES) - @echo " [LTDEP]" $(Q)$(MAKEDEP) $(CFILES) | $(SED) -e 's,^\([^:]*\)\.o,\1.lo,' > .ltdep -depend: .dep +depend: rmdep .dep + +rmdep: + @rm -f .dep .dep: $(CFILES) $(HFILES) - @echo " [DEP]" $(Q)$(MAKEDEP) $(CFILES) > .dep
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 01/30] xfsprogs: fix automatic dependency generation
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Adding are removing a header file does not result in dependency regeneration like it should. make clean will rebuild the dependencies, but a normal make won't. Fix it.
Signed-off-by Dave Chinner <[email protected]>
Looks good. This should also go into the other repositories sharing the build system.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 2/30 xfs: fix some minor sparse warnings
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
A couple of simple locking annotations and 0 vs NULL warnings. Nothing that changes any code behaviour, just removes build noise.
--- libxfs/xfs_bmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/xfs_bmap.c +1 -1
@@ -4415,7 +4415,7 @@ xfs_bmapi_write( {struct xfs_mount *mp = ip->i_mount;struct xfs_ifork *ifp; -struct xfs_bmalloca bma = {0 };/* args for xfs_bmap_alloc */ +struct xfs_bmalloca bma = {NULL };/* args for xfs_bmap_alloc */ xfs_fileoff_t end;/* end of mapped file region */ int eof;/* after the end of extents */ int error;/* error return */
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 02/30] xfs: fix some minor sparse warnings
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 3/30 xfs: create a shared header file for format-related information
In reply to Dave ChinnerAll of the buffer operations structures are needed to be exported for xfs_db, so move them all to a common location rather than spreading them all over the place. They are verifying the on-disk format, so while xfs_format.h might be a good place, it is not part of the on disk format.
Hence we need to create a new header file that we centralise these related definitions. Start by moving the bffer operations structures, and then also move all the other definitions that have crept into xfs_log_format.h and xfs_format.h as there was no other shared header file to put them in.
--- include/Makefile | 1 + include/libxfs.h | 1 + include/xfs_ag.h | 4 - include/xfs_alloc.h | 3 - include/xfs_alloc_btree.h | 2 - include/xfs_attr_leaf.h | 2 - include/xfs_bmap_btree.h | 2 - include/xfs_da_btree.h | 2 - include/xfs_format.h | 10 -- include/xfs_ialloc.h | 2 - include/xfs_ialloc_btree.h | 2 - include/xfs_inode_buf.h | 2 - include/xfs_log_format.h | 177 -------------------------------- include/xfs_sb.h | 3 - include/xfs_shared.h | 244 +++++++++++++++++++++++++++++++++++++++++++++ 15 files changed, 246 insertions(+), 211 deletions(-) create mode 100644 include/xfs_shared.h
include/Makefile +1 -0
@@ -40,6 +40,7 @@ QAHFILES = libxfs.h libxlog.h \ xfs_metadump.h \ xfs_quota_defs.h \ xfs_sb.h \ + xfs_shared.h \ xfs_trace.h \ xfs_trans_resv.h \ xfs_trans_space.h
include/libxfs.h +1 -0
@@ -37,6 +37,7 @@#include <xfs/xfs_fs.h> #include <xfs/xfs_arch.h> +#include <xfs/xfs_shared.h> #include <xfs/xfs_format.h> #include <xfs/xfs_log_format.h> #include <xfs/xfs_quota_defs.h>
include/xfs_ag.h +0 -4
@@ -128,8 +128,6 @@ typedef struct xfs_agf {extern int xfs_read_agf (struct xfs_mount *mp,struct xfs_trans *tp,xfs_agnumber_t agno,int flags,struct xfs_buf **bpp); -extern const struct xfs_buf_ops xfs_agf_buf_ops; -/* * Size of the unlinked inode hash table in the agi. */
@@ -191,8 +189,6 @@ typedef struct xfs_agi {extern int xfs_read_agi (struct xfs_mount *mp,struct xfs_trans *tp,xfs_agnumber_t agno,struct xfs_buf **bpp); -extern const struct xfs_buf_ops xfs_agi_buf_ops; -/* * The third a.g. block contains the a.g. freelist, an array * of block pointers to blocks owned by the allocation btree code.
include/xfs_alloc.h +0 -3
@@ -231,7 +231,4 @@ xfs_alloc_get_rec( xfs_extlen_t *len,/* output: length of extent */ int *stat); /* output: success/failure */ -extern const struct xfs_buf_ops xfs_agf_buf_ops; -extern const struct xfs_buf_ops xfs_agfl_buf_ops; -#endif /* __XFS_ALLOC_H__ */
include/xfs_alloc_btree.h +0 -2
@@ -95,6 +95,4 @@ extern struct xfs_btree_cur *xfs_allocbt_init_cursor(struct xfs_mount *, xfs_agnumber_t, xfs_btnum_t);extern int xfs_allocbt_maxrecs (struct xfs_mount *,int ,int ); -extern const struct xfs_buf_ops xfs_allocbt_buf_ops; -#endif /* __XFS_ALLOC_BTREE_H__ */
include/xfs_attr_leaf.h +0 -2
@@ -106,6 +106,4 @@ void xfs_attr3_leaf_hdr_from_disk(struct xfs_attr3_icleaf_hdr *to,void xfs_attr3_leaf_hdr_to_disk (struct xfs_attr_leafblock *to,struct xfs_attr3_icleaf_hdr *from); -extern const struct xfs_buf_ops xfs_attr3_leaf_buf_ops; -#endif /* __XFS_ATTR_LEAF_H__ */
include/xfs_bmap_btree.h +0 -2
@@ -239,6 +239,4 @@ extern int xfs_bmbt_maxrecs(struct xfs_mount *, int blocklen, int leaf);extern struct xfs_btree_cur *xfs_bmbt_init_cursor (struct xfs_mount *,struct xfs_trans *,struct xfs_inode *,int ); -extern const struct xfs_buf_ops xfs_bmbt_buf_ops; -#endif /* __XFS_BMAP_BTREE_H__ */
include/xfs_da_btree.h +0 -2
@@ -169,8 +169,6 @@ int xfs_da3_node_read(struct xfs_trans *tp, struct xfs_inode *dp, xfs_dablk_t bno, xfs_daddr_t mappedbno, struct xfs_buf **bpp, int which_fork); -extern const struct xfs_buf_ops xfs_da3_node_buf_ops; -/* * Utility routines. */
include/xfs_format.h +0 -10
@@ -156,14 +156,4 @@ struct xfs_dsymlink_hdr { ((bufsize) - (xfs_sb_version_hascrc (&(mp)->m_sb) ? \ sizeof(struct xfs_dsymlink_hdr) : 0)) -int xfs_symlink_blocks (struct xfs_mount *mp,int pathlen); -int xfs_symlink_hdr_set (struct xfs_mount *mp,xfs_ino_t ino,uint32_t offset, -uint32_t size,struct xfs_buf *bp); -bool xfs_symlink_hdr_ok (struct xfs_mount *mp,xfs_ino_t ino,uint32_t offset, -uint32_t size,struct xfs_buf *bp); -void xfs_symlink_local_to_remote (struct xfs_trans *tp,struct xfs_buf *bp, -struct xfs_inode *ip,struct xfs_ifork *ifp); - -extern const struct xfs_buf_ops xfs_symlink_buf_ops; -#endif /* __XFS_FORMAT_H__ */
include/xfs_ialloc.h +0 -2
@@ -158,6 +158,4 @@ int xfs_ialloc_inode_init(struct xfs_mount *mp, struct xfs_trans *tp, xfs_agnumber_t agno, xfs_agblock_t agbno, xfs_agblock_t length, unsigned int gen); -extern const struct xfs_buf_ops xfs_agi_buf_ops; -#endif /* __XFS_IALLOC_H__ */
include/xfs_ialloc_btree.h +0 -2
@@ -110,6 +110,4 @@ extern struct xfs_btree_cur *xfs_inobt_init_cursor(struct xfs_mount *, struct xfs_trans *, struct xfs_buf *, xfs_agnumber_t);extern int xfs_inobt_maxrecs (struct xfs_mount *,int ,int ); -extern const struct xfs_buf_ops xfs_inobt_buf_ops; -#endif /* __XFS_IALLOC_BTREE_H__ */
include/xfs_inode_buf.h +0 -2
@@ -47,6 +47,4 @@ void xfs_inobp_check(struct xfs_mount *, struct xfs_buf *);#define xfs_inobp_check (mp, bp)#endif /* DEBUG */ -extern const struct xfs_buf_ops xfs_inode_buf_ops; -#endif /* __XFS_INODE_BUF_H__ */
include/xfs_log_format.h +0 -177
193 lines not on this page · raw message
include/xfs_sb.h +0 -3
@@ -699,7 +699,4 @@ extern void xfs_sb_from_disk(struct xfs_sb *, struct xfs_dsb *);extern void xfs_sb_to_disk (struct xfs_dsb *,struct xfs_sb *,__int64_t );extern void xfs_sb_quota_from_disk (struct xfs_sb *sbp); -extern const struct xfs_buf_ops xfs_sb_buf_ops; -extern const struct xfs_buf_ops xfs_sb_quiet_buf_ops; -#endif /* __XFS_SB_H__ */
include/xfs_shared.h new file +244 -0
244 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 03/30] xfs: create a shared header file for format-related information
Quoted 6 lines · Dave Chinner
All of the buffer operations structures are needed to be exported for
xfs_db, so move them all to a common location rather than spreading them all over the place. They are verifying the on-disk format, so whilexfs_format.hmight be a good place, it is not part of the on disk format.
Quoted 7 lines
Hence we need to create a new header file that we centralise these related definitions. Start by moving the bffer operations structures, and then also move all the other definitions that have crept into
xfs_log_format.handxfs_format.has there was no other shared header file to put them in.Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 4/30 xfs: split dquot buffer operations out
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Parts of userspace want to be able to read and modify dquot buffers (e.g. xfs_db) so we need to split out the reading and writing of these buffers so it is easy to shared code with libxfs in userspace.
--- include/libxfs.h | 9 ++ libxfs/Makefile | 1 + libxfs/xfs_dquot_buf.c | 273 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 283 insertions(+) create mode 100644 libxfs/xfs_dquot_buf.c
include/libxfs.h +9 -0
@@ -216,6 +216,15 @@ typedef struct xfs_mount {xfs_dablk_t m_dirdatablk;/* blockno of dir data v2 */ xfs_dablk_t m_dirleafblk;/* blockno of dir non-data v2 */ xfs_dablk_t m_dirfreeblk;/* blockno of dirfreeindex v2 */ + +/* +* anonymous struct to allow xfs_dquot_buf.c to compile. +* Pointer is always null in userspace, so code does not use it at all +*/ +struct { +int qi_dqperchunk; + } *m_quotainfo; + } xfs_mount_t;/*
libxfs/Makefile +1 -0
@@ -29,6 +29,7 @@ CFILES = cache.c \ xfs_dir2_leaf.c \ xfs_dir2_node.c \ xfs_dir2_sf.c \ + xfs_dquot_buf.c \ xfs_ialloc.c \ xfs_inode_buf.c \ xfs_inode_fork.c \
libxfs/xfs_dquot_buf.c new file +273 -0
273 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 04/30] xfs: split dquot buffer operations out
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Parts of userspace want to be able to read and modify dquot buffers (e.g.
xfs_db) so we need to split out the reading and writing of these buffers so it is easy to shared code with libxfs in userspace.Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 5/30 xfs: decouple inode and bmap btree header files
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Currently the xfs_inode.h header has a dependency on the definition of the BMAP btree records as the inode fork includes an array of xfs_bmbt_rec_host_t objects in it's definition.
Move all the btree format definitions from xfs_btree.h, xfs_bmap_btree.h, xfs_alloc_btree.h and xfs_ialloc_btree.h to xfs_format.h to continue the process of centralising the on-disk format definitions. With this done, the xfs inode definitions are no longer dependent on btree header files.
The enables a massive culling of unnecessary includes, with close to 200 #include directives removed from the XFS kernel code base.
--- include/xfs_alloc_btree.h | 33 ------ include/xfs_bmap_btree.h | 103 ----------------- include/xfs_btree.h | 80 ------------- include/xfs_format.h | 281 +++++++++++++++++++++++++++++++++++++++++++++ include/xfs_ialloc.h | 3 +- include/xfs_ialloc_btree.h | 49 -------- include/xfs_inode_fork.h | 1 + 7 files changed, 284 insertions(+), 266 deletions(-)
include/xfs_alloc_btree.h +0 -33
@@ -27,39 +27,6 @@ struct xfs_btree_cur;struct xfs_mount ;/* -* There are two on-disk btrees, one sorted by blockno and one sorted -* by blockcount and blockno. All blocks look the same to make the code -* simpler; if we have time later, we'll make the optimizations. -*/ -#define XFS_ABTB_MAGIC 0x41425442/* 'ABTB' for bno tree */ -#define XFS_ABTB_CRC_MAGIC 0x41423342/* 'AB3B' */ -#define XFS_ABTC_MAGIC 0x41425443/* 'ABTC' for cnt tree */ -#define XFS_ABTC_CRC_MAGIC 0x41423343/* 'AB3C' */ - -/* -* Data record/key structure -*/ -typedef struct xfs_alloc_rec { -__be32 ar_startblock;/* starting block number */ -__be32 ar_blockcount;/* count of free blocks */ -}xfs_alloc_rec_t ,xfs_alloc_key_t ; - -typedef struct xfs_alloc_rec_incore { -xfs_agblock_t ar_startblock;/* starting block number */ -xfs_extlen_t ar_blockcount;/* count of free blocks */ -}xfs_alloc_rec_incore_t ; - -/* btree pointer type */ -typedef __be32 xfs_alloc_ptr_t ; - -/* -* Block numbers in the AG: -* SB is sector 0, AGF is sector 1, AGI is sector 2, AGFL is sector 3. -*/ -#define XFS_BNO_BLOCK (mp) ((xfs_agblock_t)(XFS_AGFL_BLOCK(mp) + 1)) -#define XFS_CNT_BLOCK (mp) ((xfs_agblock_t)(XFS_BNO_BLOCK(mp) + 1)) - -/* * Btree block header size depends on a superblock flag. */ #define XFS_ALLOC_BLOCK_LEN (mp) \
include/xfs_bmap_btree.h +0 -103
121 lines not on this page · raw message
include/xfs_btree.h +0 -80
86 lines not on this page · raw message
include/xfs_format.h +281 -0
285 lines not on this page · raw message
include/xfs_ialloc.h +2 -1
@@ -23,6 +23,7 @@ struct xfs_dinode;struct xfs_imap ;struct xfs_mount ;struct xfs_trans ; +struct xfs_btree_cur ;/* * Allocation parameters for inode allocation.
@@ -42,7 +43,7 @@ struct xfs_trans;static inline struct xfs_dinode *xfs_make_iptr (struct xfs_mount *mp,struct xfs_buf *b,int o) { -return (xfs_dinode_t *) +return (struct xfs_dinode *) (xfs_buf_offset (b, o << (mp)->m_sb.sb_inodelog)); }
include/xfs_ialloc_btree.h +0 -49
55 lines not on this page · raw message
include/xfs_inode_fork.h +1 -0
@@ -19,6 +19,7 @@#define __XFS_INODE_FORK_H__struct xfs_inode_log_item ; +struct xfs_dinode ;/* * The following xfs_ext_irec_t struct introduces a second (top) level
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 05/30] xfs: decouple inode and bmap btree header files
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 6/30 libxfs: unify xfs_btree.c with kernel code
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The libxfs/xfs_btree.c code does not contain a small amount of code for btree block readahead that the kernel code does. Instead, it short circuits it at a higher layer and doesn't include the lower layer functions. There is no harm in calling the lower lay functions and have them do nothing, and doing so unifies the kernel and userspace code.
--- libxfs/xfs.h | 8 +++++--- libxfs/xfs_btree.c | 48 +++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 48 insertions(+), 8 deletions(-)
libxfs/xfs.h +5 -3
@@ -319,10 +319,12 @@ roundup_64(__uint64_t x, __uint32_t y)#define xfs_trans_buf_copy_type (dbp, sbp) -#define xfs_buf_readahead (a,b,c,ops) ((void) 0)/* no readahead */ +/* no readahead, need to avoid set-but-unused var warnings. */ +#define xfs_buf_readahead(a,d,c,ops) ({ \ + xfs_daddr_t __d = d; \ + __d = __d; /* no set-but-unused warning */ \ +})#define xfs_buf_readahead_map (a,b,c,ops) ((void) 0)/* no readahead */ -#define xfs_btree_reada_bufl (m,fsb,c,ops) ((void) 0) -#define xfs_btree_reada_bufs (m,fsb,c,x,ops) ((void) 0)#define xfs_buftrace (x,y) ((void) 0)/* debug only */ #define xfs_cmn_err(tag,level,mp,fmt,args...) cmn_err(level,fmt, ## args)
libxfs/xfs_btree.c +43 -5
84 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 06/30] libxfs: unify xfs_btree.c with kernel code
Quoted 10 lines · Dave Chinner
From: Dave Chinner <[email protected]>
The
libxfs/xfs_btree.ccode does not contain a small amount of code for btree block readahead that the kernel code does. Instead, it short circuits it at a higher layer and doesn't include the lower layer functions. There is no harm in calling the lower lay functions and have them do nothing, and doing so unifies the kernel and userspace code.Signed-off-by Dave Chinner <[email protected]>
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 7/30 libxfs: bmap btree owner swap support
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
For CRC enabled filesystems, we can't just swap inode forks from one inode to another when defragmenting a file - the blocks in the inode fork bmap btree contain pointers back to the owner inode. Hence if we are to swap the inode forks we have to atomically modify every block in the btree during the transaction.
This patch brings across the kernel code for doing the owner swap of an entire fork - something that we are likely to end up needing in xfs_repair when reparenting stray inodes to lost+found - without all the associated swap extents transaction and recovery cruft as those parts are not needed in userspace.
--- include/xfs_bmap_btree.h | 4 ++ include/xfs_btree.h | 19 ++++-- include/xfs_inode_buf.h | 18 ++--- include/xfs_log_format.h | 8 ++- libxfs/xfs_bmap_btree.c | 44 ++++++++++++ libxfs/xfs_btree.c | 170 ++++++++++++++++++++++++++++++++++++++++++----- 6 files changed, 227 insertions(+), 36 deletions(-)
include/xfs_bmap_btree.h +4 -0
@@ -133,6 +133,10 @@ extern int xfs_bmbt_get_maxrecs(struct xfs_btree_cur *, int level);extern int xfs_bmdr_maxrecs (struct xfs_mount *,int blocklen,int leaf);extern int xfs_bmbt_maxrecs (struct xfs_mount *,int blocklen,int leaf); +extern int xfs_bmbt_change_owner (struct xfs_trans *tp,struct xfs_inode *ip, +int whichfork,xfs_ino_t new_owner, +struct list_head *buffer_list); +extern struct xfs_btree_cur *xfs_bmbt_init_cursor (struct xfs_mount *,struct xfs_trans *,struct xfs_inode *,int );
include/xfs_btree.h +12 -7
@@ -41,15 +41,18 @@ extern kmem_zone_t *xfs_btree_cur_zone;/* * For logging record fields. */ -#define XFS_BB_MAGIC 0x01 -#define XFS_BB_LEVEL 0x02 -#define XFS_BB_NUMRECS 0x04 -#define XFS_BB_LEFTSIB 0x08 -#define XFS_BB_RIGHTSIB 0x10 -#define XFS_BB_BLKNO 0x20 +#define XFS_BB_MAGIC (1 << 0) +#define XFS_BB_LEVEL (1 << 1) +#define XFS_BB_NUMRECS (1 << 2) +#define XFS_BB_LEFTSIB (1 << 3) +#define XFS_BB_RIGHTSIB (1 << 4) +#define XFS_BB_BLKNO (1 << 5) +#define XFS_BB_LSN (1 << 6) +#define XFS_BB_UUID (1 << 7) +#define XFS_BB_OWNER (1 << 8)#define XFS_BB_NUM_BITS 5#define XFS_BB_ALL_BITS ((1 << XFS_BB_NUM_BITS) - 1) -#define XFS_BB_NUM_BITS_CRC 8 +#define XFS_BB_NUM_BITS_CRC 9#define XFS_BB_ALL_BITS_CRC ((1 << XFS_BB_NUM_BITS_CRC) - 1)/*
@@ -381,6 +384,8 @@ int xfs_btree_new_iroot(struct xfs_btree_cur *, int *, int *);int xfs_btree_insert (struct xfs_btree_cur *,int *);int xfs_btree_delete (struct xfs_btree_cur *,int *);int xfs_btree_get_rec (struct xfs_btree_cur *,union xfs_btree_rec **,int *); +int xfs_btree_change_owner (struct xfs_btree_cur *cur,__uint64_t new_owner, +struct list_head *buffer_list);/* * btree block CRC helpers
include/xfs_inode_buf.h +9 -9
@@ -32,17 +32,17 @@ struct xfs_imap {ushort im_boffset;/* inode offset in block in bytes */ }; -int xfs_imap_to_bp (struct xfs_mount *,struct xfs_trans *, -struct xfs_imap *,struct xfs_dinode **, -struct xfs_buf **,uint ,uint ); -int xfs_iread (struct xfs_mount *,struct xfs_trans *, -struct xfs_inode *,uint ); -void xfs_dinode_calc_crc (struct xfs_mount *,struct xfs_dinode *); -void xfs_dinode_to_disk (struct xfs_dinode *, -struct xfs_icdinode *); +int xfs_imap_to_bp (struct xfs_mount *,struct xfs_trans *, +struct xfs_imap *,struct xfs_dinode **, +struct xfs_buf **,uint ,uint ); +int xfs_iread (struct xfs_mount *,struct xfs_trans *, +struct xfs_inode *,uint ); +void xfs_dinode_calc_crc (struct xfs_mount *,struct xfs_dinode *); +void xfs_dinode_to_disk (struct xfs_dinode *to,struct xfs_icdinode *from); +void xfs_dinode_from_disk (struct xfs_icdinode *to,struct xfs_dinode *from);#if defined(DEBUG ) -void xfs_inobp_check (struct xfs_mount *,struct xfs_buf *); +void xfs_inobp_check (struct xfs_mount *,struct xfs_buf *);#else #define xfs_inobp_check (mp, bp)#endif /* DEBUG */
include/xfs_log_format.h +6 -2
@@ -302,6 +302,8 @@ typedef struct xfs_inode_log_format_64 {#define XFS_ILOG_ADATA 0x040/* log i_af.if_data */ #define XFS_ILOG_AEXT 0x080/* log i_af.if_extents */ #define XFS_ILOG_ABROOT 0x100/* log i_af.i_broot */ +#define XFS_ILOG_DOWNER 0x200/* change the data fork owner on replay */ +#define XFS_ILOG_AOWNER 0x400/* change the attr fork owner on replay */ /*
@@ -315,7 +317,8 @@ typedef struct xfs_inode_log_format_64 {#define XFS_ILOG_NONCORE (XFS_ILOG_DDATA | XFS_ILOG_DEXT | \ XFS_ILOG_DBROOT | XFS_ILOG_DEV | \ XFS_ILOG_UUID | XFS_ILOG_ADATA | \ - XFS_ILOG_AEXT | XFS_ILOG_ABROOT) + XFS_ILOG_AEXT | XFS_ILOG_ABROOT | \ + XFS_ILOG_DOWNER | XFS_ILOG_AOWNER)#define XFS_ILOG_DFORK (XFS_ILOG_DDATA | XFS_ILOG_DEXT | \ XFS_ILOG_DBROOT)
@@ -327,7 +330,8 @@ typedef struct xfs_inode_log_format_64 { XFS_ILOG_DEXT | XFS_ILOG_DBROOT | \ XFS_ILOG_DEV | XFS_ILOG_UUID | \ XFS_ILOG_ADATA | XFS_ILOG_AEXT | \ - XFS_ILOG_ABROOT | XFS_ILOG_TIMESTAMP) + XFS_ILOG_ABROOT | XFS_ILOG_TIMESTAMP | \ + XFS_ILOG_DOWNER | XFS_ILOG_AOWNER)static inline int xfs_ilog_fbroot (int w) {
libxfs/xfs_bmap_btree.c +44 -0
47 lines not on this page · raw message
libxfs/xfs_btree.c +152 -18
185 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 07/30] libxfs: bmap btree owner swap support
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
For CRC enabled filesystems, we can't just swap inode forks from one inode to another when defragmenting a file - the blocks in the inode fork bmap btree contain pointers back to the owner inode. Hence if we are to swap the inode forks we have to atomically modify every block in the btree during the transaction.
Quoted 7 lines
This patch brings across the kernel code for doing the owner swap of an entire fork - something that we are likely to end up needing in
xfs_repairwhen reparenting stray inodes to lost+found - without all the associated swap extents transaction and recovery cruft as those parts are not needed in userspace.Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 8/30 libxfs: xfs_rtalloc.c becomes xfs_rtbitmap.c
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
To match the split-up of the kernel xfs_rtalloc.c file, convert the libxfs version of xfs_rtalloc.c to match the newly shared kernel source file with all the realtime bitmap functions in it, xfs_rtbitmap.c.
--- libxfs/Makefile | 2 +- libxfs/xfs_rtalloc.c | 776 ---------------------------------------- libxfs/xfs_rtbitmap.c | 951 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 952 insertions(+), 777 deletions(-) delete mode 100644 libxfs/xfs_rtalloc.c create mode 100644 libxfs/xfs_rtbitmap.c
libxfs/Makefile +1 -1
@@ -35,7 +35,7 @@ CFILES = cache.c \ xfs_inode_fork.c \ xfs_ialloc_btree.c \ xfs_log_rlimit.c \ - xfs_rtalloc.c \ + xfs_rtbitmap.c \ xfs_sb.c \ xfs_symlink_remote.c \ xfs_trans_resv.c
libxfs/xfs_rtalloc.c deleted +0 -776
776 lines not on this page · raw message
libxfs/xfs_rtbitmap.c new file +951 -0
951 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 08/30] libxfs: xfs_rtalloc.c becomes xfs_rtbitmap.c
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
To match the split-up of the kernel
xfs_rtalloc.cfile, convert the libxfs version ofxfs_rtalloc.cto match the newly shared kernel source file with all the realtime bitmap functions in it,xfs_rtbitmap.c.Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 9/30 libxfs: bring across inode buffer readahead verifier changes
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
These were made for log recovery readahead in the kernel, so are not directly used in userspace. Hence bringing the change across is simply to keep files in sync.
--- libxfs/xfs_inode_buf.c | 41 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 5 deletions(-)
libxfs/xfs_inode_buf.c +36 -5
@@ -46,9 +46,22 @@ xfs_inobp_check( }#endif +/* +* If we are doing readahead on an inode buffer, we might be in log recovery +* reading an inode allocation buffer that hasn't yet been replayed, and hence +* has not had the inode cores stamped into it. Hence for readahead, the buffer +* may be potentially invalid. +* +* If the readahead buffer is invalid, we don't want to mark it with an error, +* but we do want to clear the DONE status of the buffer so that a followup read +* will re-read it from disk. This will ensure that we don't get an unnecessary +* warnings during log recovery and we don't get unnecssary panics on debug +* kernels. +*/ static void xfs_inode_buf_verify ( -struct xfs_buf *bp) +struct xfs_buf *bp, +bool readahead) {struct xfs_mount *mp = bp->b_target->bt_mount;int i;
@@ -69,6 +82,11 @@ xfs_inode_buf_verify(if (unlikely (XFS_TEST_ERROR (!di_ok, mp,XFS_ERRTAG_ITOBP_INOTOBP ,XFS_RANDOM_ITOBP_INOTOBP ))) { +if (readahead) { + bp->b_flags &= ~XBF_DONE ; +return ; + } +xfs_buf_ioerror (bp,EFSCORRUPTED );XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_HIGH , mp, dip);
@@ -87,14 +105,21 @@ static voidxfs_inode_buf_read_verify ( struct xfs_buf *bp) { -xfs_inode_buf_verify (bp); +xfs_inode_buf_verify (bp, false); +} + +static void +xfs_inode_buf_readahead_verify ( +struct xfs_buf *bp) +{ +xfs_inode_buf_verify (bp, true); }static void xfs_inode_buf_write_verify (struct xfs_buf *bp) { -xfs_inode_buf_verify (bp); +xfs_inode_buf_verify (bp, false); }const struct xfs_buf_ops xfs_inode_buf_ops = {
@@ -102,6 +127,12 @@ const struct xfs_buf_ops xfs_inode_buf_ops = { .verify_write = xfs_inode_buf_write_verify, }; +const struct xfs_buf_ops xfs_inode_buf_ra_ops = { + .verify_read = xfs_inode_buf_readahead_verify, + .verify_write = xfs_inode_buf_write_verify, +}; + +/* * This routine is called to map an inode to the buffer containing the on-disk * version of the inode. It returns a pointer to the buffer containing the
@@ -191,7 +222,7 @@ xfs_dinode_from_disk( to->di_ino =be64_to_cpu (from->di_ino); to->di_lsn =be64_to_cpu (from->di_lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2)); -platform_uuid_copy (&to->di_uuid, &from->di_uuid); +uuid_copy (&to->di_uuid, &from->di_uuid); } }
@@ -237,7 +268,7 @@ xfs_dinode_to_disk( to->di_ino =cpu_to_be64 (from->di_ino); to->di_lsn =cpu_to_be64 (from->di_lsn);memcpy (to->di_pad2, from->di_pad2,sizeof (to->di_pad2)); -platform_uuid_copy (&to->di_uuid, &from->di_uuid); +uuid_copy (&to->di_uuid, &from->di_uuid); to->di_flushiter =0 ; }else { to->di_flushiter =cpu_to_be16 (from->di_flushiter);
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 09/30] libxfs: bring across inode buffer readahead verifier changes
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
These were made for log recovery readahead in the kernel, so are not directly used in userspace. Hence bringing the change across is simply to keep files in sync.
Signed-off-by Dave Chinner <[email protected]>
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 10/30 libxfs: Minor cleanup and bug fix sync
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
These bring all the small single line comment, whitespace and minor code differences into sync with the kernel code. Anything left at this point is an intentional difference.
--- include/xfs_fs.h | 4 ++-- include/xfs_quota_defs.h | 4 ++++ libxfs/xfs_attr_leaf.c | 4 +--- libxfs/xfs_bmap.c | 6 +++--- libxfs/xfs_bmap_btree.c | 2 +- libxfs/xfs_dir2_leaf.c | 1 - libxfs/xfs_dir2_node.c | 2 +- libxfs/xfs_ialloc.c | 4 ++-- libxfs/xfs_inode_buf.c | 2 ++ libxfs/xfs_inode_fork.c | 5 +++-- libxfs/xfs_trans_resv.c | 2 +- 11 files changed, 20 insertions(+), 16 deletions(-)
include/xfs_fs.h +2 -2
@@ -358,7 +358,7 @@ typedef struct xfs_error_injection {* Speculative preallocation trimming. */ #define XFS_EOFBLOCKS_VERSION 1 -struct xfs_eofblocks { +struct xfs_fs_eofblocks {__u32 eof_version;__u32 eof_flags;uid_t eof_uid;
@@ -516,7 +516,7 @@ typedef struct xfs_swapext/* XFS_IOC_GETBIOSIZE ---- deprecated 47 */ #define XFS_IOC_GETBMAPX _IOWR('X', 56, struct getbmap)#define XFS_IOC_ZERO_RANGE _IOW ('X', 57, struct xfs_flock64) -#define XFS_IOC_FREE_EOFBLOCKS _IOR ('X', 58, struct xfs_eofblocks) +#define XFS_IOC_FREE_EOFBLOCKS _IOR ('X', 58, struct xfs_fs_eofblocks)/* * ioctl commands that replace IRIX syssgi()'s
include/xfs_quota_defs.h +4 -0
@@ -154,4 +154,8 @@ typedef __uint16_t xfs_qwarncnt_t; (XFS_QMOPT_UQUOTA | XFS_QMOPT_PQUOTA | XFS_QMOPT_GQUOTA)#define XFS_QMOPT_RESBLK_MASK (XFS_QMOPT_RES_REGBLKS | XFS_QMOPT_RES_RTBLKS) +extern int xfs_dqcheck (struct xfs_mount *mp,xfs_disk_dquot_t *ddq, +xfs_dqid_t id,uint type,uint flags,char *str); +extern int xfs_calc_dquots_per_chunk (struct xfs_mount *mp,unsigned int nbblks); +#endif /* __XFS_QUOTA_H__ */
libxfs/xfs_attr_leaf.c +1 -3
@@ -599,7 +599,7 @@ xfs_attr_shortform_getvalue(xfs_da_args_t *args)xfs_attr_sf_entry_t *sfe;int i; -ASSERT (args->dp->i_d.di_aformat ==XFS_IFINLINE ); +ASSERT (args->dp->i_afp->if_flags ==XFS_IFINLINE ); sf = (xfs_attr_shortform_t *)args->dp->i_afp->if_u1.if_data; sfe = &sf->list[0 ]; for (i = 0; i < sf->hdr.count;
@@ -909,7 +909,6 @@ out:return error; } -/*======================================================================== * Routines used for growing the Btree. *========================================================================*/
@@ -1270,7 +1269,6 @@ xfs_attr3_leaf_compact( ichdr_dst->freemap[0 ].size = ichdr_dst->firstused - ichdr_dst->freemap[0 ].base; -/* write the header back to initialise the underlying buffer */ xfs_attr3_leaf_hdr_to_disk (leaf_dst, ichdr_dst);
libxfs/xfs_bmap.c +3 -3
@@ -510,8 +510,8 @@ xfs_bmap_trace_exlist(/* * Validate that the bmbt_irecs being returned from bmapi are valid -* given the callers original parameters. Specifically check the -* ranges of the returned irecs to ensure that they only extent beyond +* given the caller's original parameters. Specifically check the +* ranges of the returned irecs to ensure that they only extend beyond * the given parameters if the XFS_BMAPI_ENTIRE flag was set. */ STATIC void
@@ -1515,7 +1515,7 @@ xfs_bmap_first_unused( }/* -* Returns the file-relative block number of the last block + 1 before +* Returns the file-relative block number of the last block - 1 before * last_block (input value) in the file. * This is not based on i_size, it is based on the extent records. * Returns 0 for local files, as they do not have extent records.
libxfs/xfs_bmap_btree.c +1 -1
@@ -737,7 +737,7 @@ xfs_bmbt_verify(* precise. */ level =be16_to_cpu (block->bb_level); -if (level >MAX (mp->m_bm_maxlevels[0 ], mp->m_bm_maxlevels[1 ])) +if (level >max (mp->m_bm_maxlevels[0 ], mp->m_bm_maxlevels[1 ]))return false;if (be16_to_cpu (block->bb_numrecs) > mp->m_bmap_dmxr[level !=0 ])return false;
libxfs/xfs_dir2_leaf.c +0 -1
@@ -1072,7 +1072,6 @@ xfs_dir3_leaf_compact_x1( *highstalep = highstale; } -/* * Log the bests entries indicated from a leaf1 block. */
libxfs/xfs_dir2_node.c +1 -1
@@ -1796,7 +1796,7 @@ xfs_dir2_node_addname_int(/* * Look at the current free entry. Is it good enough? * -* The bests initialisation should be where the buffer is read in +* The bests initialisation should be where the bufer is read in * the above branch. But gcc is too stupid to realise that bests * and the freehdr are actually initialised if they are placed * there, so we have to do it here to avoid warnings. Blech.
libxfs/xfs_ialloc.c +2 -2
@@ -484,7 +484,7 @@ xfs_ialloc_next_ag(/* * Select an allocation group to look for a free inode in, based on the parent -* inode and then mode. Return the allocation group buffer. +* inode and the mode. Return the allocation group buffer. */ STATIC xfs_agnumber_t xfs_ialloc_ag_select(
@@ -706,7 +706,7 @@ xfs_dialloc_ag( error =xfs_inobt_get_rec (cur, &rec, &j);if (error) goto error0; -XFS_WANT_CORRUPTED_GOTO (i ==1 , error0); +XFS_WANT_CORRUPTED_GOTO (j ==1 , error0);if (rec.ir_freecount >0 ) {/*
libxfs/xfs_inode_buf.c +2 -0
@@ -101,6 +101,7 @@ xfs_inode_buf_verify(xfs_inobp_check (mp, bp); } +static void xfs_inode_buf_read_verify (struct xfs_buf *bp)
@@ -299,6 +300,7 @@ xfs_dinode_verify(return false;return true; } + void xfs_dinode_calc_crc( struct xfs_mount *mp,
libxfs/xfs_inode_fork.c +3 -2
@@ -138,7 +138,8 @@ xfs_iformat_fork( } di_size =be64_to_cpu (dip->di_size); -if (unlikely (di_size >XFS_DFORK_DSIZE (dip, ip->i_mount))) { +if (unlikely (di_size <0 || + di_size >XFS_DFORK_DSIZE (dip, ip->i_mount))) { xfs_warn(ip->i_mount, "corrupt inode %Lu (bad size %Ld for local inode).", (unsigned long long) ip->i_ino,
@@ -444,7 +445,7 @@ xfs_iread_extents(* * The caller must not request to add more records than would fit in * the on-disk inode root. If the if_broot is currently NULL, then -* if we adding records one will be allocated. The caller must also +* if we are adding records, one will be allocated. The caller must also * not request that the number of records go below zero, although * it can go to zero. *
libxfs/xfs_trans_resv.c +1 -1
@@ -49,7 +49,7 @@ xfs_calc_buf_res(/* * Logging inodes is really tricksy. They are logged in memory format, -* which means that what we write into the log doesn't directory translate into +* which means that what we write into the log doesn't directly translate into * the amount of space they use on disk. * * Case in point - btree format forks in memory format use more space than the
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 10/30] libxfs: Minor cleanup and bug fix sync
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
These bring all the small single line comment, whitespace and minor code differences into sync with the kernel code. Anything left at this point is an intentional difference.
Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 11/30 xfs: remove newlines from strings passed to __xfs_printk
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
__xfs_printk adds its own "\n". Having it in the original string leads to unintentional blank lines from these messages.
Ported from kernel commit 08e96e1a.
--- libxfs/xfs_bmap.c | 2 +- libxfs/xfs_dir2_node.c | 2 +- libxfs/xfs_sb.c | 4 ++-- libxlog/xfs_log_recover.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)
libxfs/xfs_bmap.c +1 -1
@@ -1447,7 +1447,7 @@ xfs_bmap_search_extents( xfs_alert_tag(ip->i_mount, XFS_PTAG_FSBLOCK_ZERO, "Access to block zero in inode %llu " "start_block: %llx start_off: %llx " - "blkcnt: %llx extent-state: %x lastx: %x\n", + "blkcnt: %llx extent-state: %x lastx: %x", (unsigned long long)ip->i_ino, (unsigned long long)gotp->br_startblock, (unsigned long long)gotp->br_startoff,
libxfs/xfs_dir2_node.c +1 -1
@@ -1083,7 +1083,7 @@ xfs_dir2_leafn_rebalance( state->inleaf =1 ; blk2->index =0 ;xfs_alert (args->dp->i_mount, -"%s: picked the wrong leaf? reverting original leaf: blk1->index %d\n" , +"%s: picked the wrong leaf? reverting original leaf: blk1->index %d" , __func__, blk1->index); } }
libxfs/xfs_sb.c +2 -2
@@ -215,13 +215,13 @@ xfs_mount_validate_sb(if (xfs_sb_version_has_pquotino (sbp)) {if (sbp->sb_qflags & (XFS_OQUOTA_ENFD |XFS_OQUOTA_CHKD )) {xfs_notice (mp, -"Version 5 of Super block has XFS_OQUOTA bits.\n" ); +"Version 5 of Super block has XFS_OQUOTA bits." );return XFS_ERROR (EFSCORRUPTED ); } }else if (sbp->sb_qflags & (XFS_PQUOTA_ENFD |XFS_GQUOTA_ENFD |XFS_PQUOTA_CHKD |XFS_GQUOTA_CHKD )) {xfs_notice (mp, -"Superblock earlier than Version 5 has XFS_[PQ]UOTA_{ENFD|CHKD} bits.\n" ); +"Superblock earlier than Version 5 has XFS_[PQ]UOTA_{ENFD|CHKD} bits." );return XFS_ERROR (EFSCORRUPTED ); }
libxlog/xfs_log_recover.c +1 -1
@@ -1330,7 +1330,7 @@ xlog_unpack_data_crc(if (crc != rhead->h_crc) {if (rhead->h_crc ||xfs_sb_version_hascrc (&log->l_mp->m_sb)) {xfs_alert (log->l_mp, -"log record CRC mismatch: found 0x%x, expected 0x%x.\n" , +"log record CRC mismatch: found 0x%x, expected 0x%x." ,le32_to_cpu (rhead->h_crc),le32_to_cpu (crc));xfs_hex_dump (dp,32 );
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 11/30] xfs: remove newlines from strings passed to __xfs_printk
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
__xfs_printkadds its own "\n". Having it in the original string leads to unintentional blank lines from these messages.Ported from kernel commit
08e96e1a.Signed-off-by Dave Chinner <[email protected]>
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 12/30 xfs: fix the wrong new_size/rnew_size at xfs_iext_realloc_direct()
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
At xfs_iext_realloc_direct(), the new_size is changed by adding if_bytes if originally the extent records are stored at the inline extent buffer, and we have to switch from it to a direct extent list for those new allocated extents, this is wrong.
This patch fix above problem and revise the new_size comments at xfs_iext_realloc_direct() to make it more readable. Also, fix the comments while switching from the inline extent buffer to a direct extent list to reflect this change.
Ported from kernel commit 17ec81c1.
--- libxfs/xfs_inode_fork.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-)
libxfs/xfs_inode_fork.c +2 -7
@@ -1330,7 +1330,7 @@ xfs_iext_remove_indirect(void xfs_iext_realloc_direct (xfs_ifork_t *ifp,/* inode fork pointer */ -int new_size)/* new size of extents */ +int new_size)/* new size of extents after adding */ {int rnew_size;/* real new size of extents */
@@ -1368,13 +1368,8 @@ xfs_iext_realloc_direct( rnew_size - ifp->if_real_bytes); } } -/* -* Switch from the inline extent buffer to a direct -* extent list. Be sure to include the inline extent -* bytes in new_size. -*/ +/* Switch from the inline extent buffer to a direct extent list */ else { - new_size += ifp->if_bytes;if (!is_power_of_2 (new_size)) { rnew_size =roundup_pow_of_two (new_size); }
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 12/30] xfs: fix the wrong new_size/rnew_size at xfs_iext_realloc_direct()
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 13/30 xfs: fix node forward in xfs_node_toosmall
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When a node is considered for a merge with a sibling, it overwrites the sibling pointers of the original incore nodehdr with the sibling's pointers. This leads to loop considering the original node as a merge candidate with itself in the second pass, and so it incorrectly determines a merge should occur.)
Ported from equivalent kernel commit 997def25.
--- libxfs/xfs_da_btree.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
libxfs/xfs_da_btree.c +3 -2
@@ -1201,6 +1201,7 @@ xfs_da3_node_toosmall(/* start with smaller blk num */ forward = nodehdr.forw < nodehdr.back;for (i =0 ; i <2 ; forward = !forward, i++) { +struct xfs_da3_icnode_hdr thdr;if (forward) blkno = nodehdr.forw;else
@@ -1213,10 +1214,10 @@ xfs_da3_node_toosmall(return (error); node = bp->b_addr; -xfs_da3_node_hdr_from_disk (&nodehdr, node); +xfs_da3_node_hdr_from_disk (&thdr, node);xfs_trans_brelse (state->args->trans, bp); -if (count - nodehdr.count >=0 ) +if (count - thdr.count >=0 )break ;/* fits with at least 25% to spare */ }if (i >=2 ) {
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 13/30] xfs: fix node forward in xfs_node_toosmall
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 14/30 xfs: don't emit corruption noise on fs probes
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
If we get EWRONGFS due to probing of non-xfs filesystems, there's no need to issue the scary corruption error and backtrace.
Ported from kernel commit 31625f28.
--- libxfs/xfs_sb.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
libxfs/xfs_sb.c +3 -2
@@ -574,8 +574,9 @@ xfs_sb_read_verify( out_error:if (error) { -XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , - mp, bp->b_addr); +if (error !=EWRONGFS ) +XFS_CORRUPTION_ERROR (__func__,XFS_ERRLEVEL_LOW , + mp, bp->b_addr);xfs_buf_ioerror (bp, error); } }
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 14/30] xfs: don't emit corruption noise on fs probes
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 15/30 db: separate out straight buffer IO from map based IO.
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
First step in converting to libxfs based IO.
--- db/init.c | 7 ++-- db/io.c | 125 +++++++++++++++++++++++++++++++++++++++++++++++++------------- db/io.h | 5 +-- 3 files changed, 104 insertions(+), 33 deletions(-)
db/init.c +3 -4
@@ -55,7 +55,7 @@ init( char **argv) {xfs_sb_t *sbp; -void *bufp =NULL ; +char bufp[BBSIZE ];int c;setlocale (LC_ALL ,"" );
@@ -115,15 +115,14 @@ init(exit (1 ); } -if (read_bbs (XFS_SB_DADDR ,1 , &bufp,NULL )) { +if (read_buf (XFS_SB_DADDR ,1 , bufp)) {fprintf (stderr,_ ("%s: %s is invalid (cannot read first 512 " "bytes)\n" ), progname, fsdevice);exit (1 ); }/* copy SB from buffer to in-core, converting architecture as we go */ -libxfs_sb_from_disk (&xmount.m_sb, bufp); -xfree (bufp); +libxfs_sb_from_disk (&xmount.m_sb, (struct xfs_dsb *)bufp); sbp = &xmount.m_sb;if (sbp->sb_magicnum !=XFS_SB_MAGIC ) {
db/io.c +100 -25
199 lines not on this page · raw message
db/io.h +1 -4
@@ -52,10 +52,7 @@ extern void off_cur(int off, int len);extern void pop_cur (void );extern void print_iocur (char *tag,iocur_t *ioc);extern void push_cur (void ); -extern int read_bbs (__int64_t daddr,int count,void **bufp, -bbmap_t *bbmap); -extern int write_bbs (__int64_t daddr,int count,void *bufp, -bbmap_t *bbmap); +extern int read_buf (__int64_t daddr,int count,void *bufp);extern void write_cur (void );extern void set_cur (const struct typ *t,__int64_t d,int c,int ring_add,bbmap_t *bbmap);
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 15/30] db: separate out straight buffer IO from map based IO.
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
First step in converting to libxfs based IO.
Signed-off-by Dave Chinner <[email protected]>
The patch description is a little too short, there's not real explanation of what it actually does.
db/init.c init( · line 118
Quoted 2 lines
db/init.c +1
-1-if (read_bbs (XFS_SB_DADDR ,1 , &bufp,NULL )) { +if (read_buf (XFS_SB_DADDR ,1 , bufp)) {
E.g. why isn't this already using the normal libxfs routines?
(there probably is an explanation but I don't quite see it yet..)
db/io.c ring_add(void) · line 425
Quoted 6 lines
int
db/io.c +5
-0+read_buf ( + xfs_daddr_t bbno, + int count, + void *bufp) +{
Is read_buf really a good name for something that is a trivial pread wrapper and doesn't deal with buffers? Should the function have some comments explaining when to use it? The same also applies to the write side.
Quoted 21 lines
db/io.c +20
-0+int err; + + err =pwrite64 (x.dfd, bufp,BBTOB (count),BBTOB (bbno)); +if (err <0 ) + err = errno; +else if (err < count) + err =-1 ; +return err; +} +static void +write_cur_buf (void ) +{ +int ret; + + ret =write_buf (iocur_top->bb, iocur_top->blen, iocur_top->buf); + +if (ret ==-1 ) +dbprintf (_ ("incomplete write, block: %lld\n" ), + (iocur_base + iocur_sp)->bb); +else if (ret !=0 )
db/io.c ring_add(void) · line 472
Quoted 10 lines
db/io.c +10
-0+dbprintf (_ ("write error: %s\n" ),strerror (ret)); + +/* re-read buffer from disk */ + ret =read_buf (iocur_top->bb, iocur_top->blen, iocur_top->buf); +if (ret ==-1 ) +dbprintf (_ ("incomplete read, block: %lld\n" ), + (iocur_base + iocur_sp)->bb); +else if (ret !=0 ) +dbprintf (_ ("read error: %s\n" ),strerror (ret)); +}
What is the point of the write and re-read cycle?
db/io.c write_bbs( · line 494
Quoted 10 lines
db/io.c +3
-1+for (j =0 ; j < count; j++) { + bbno = bbmap->b[j];if (lseek64 (x.dfd, bbno <<BBSHIFT ,SEEK_SET ) <0 ) { rval = errno;dbprintf (_ ("can't seek in filesystem at bb %lld\n" ), bbno);return rval; } - c =BBTOB (bbmap ?1 : count); + c =BBTOB (1 ); i = (int )write (x.dfd, (char *)bufp +BBTOB (j), c);
Shoiuldn't this use the write_buf helper above?
db/io.c read_bbs( · line 538
Quoted 12 lines
db/io.c +3
-1+for (j =0 ; j < count; j++) { + bbno = bbmap->b[j];if (lseek64 (x.dfd, bbno <<BBSHIFT ,SEEK_SET ) <0 ) { rval = errno;dbprintf (_ ("can't seek in filesystem at bb %lld\n" ), bbno);@@ -483,7 +534,7 @@ read_bbs(xfree (buf); buf =NULL ; }else { - c =BBTOB (bbmap ?1 : count); + c =BBTOB (1 ); i = (int )read (x.dfd, (char *)buf +BBTOB (j), c);
And read_buf here?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 15/30] db: separate out straight buffer IO from map based IO.
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
First step in converting to libxfs based IO.
Signed-off-by Dave Chinner <[email protected]>The patch description is a little too short, there's not real explanation of what it actually does.
I can fix that.
Quoted 7 lines
db/init.c init( · line 118
Quoted 3 lines
db/init.c +1
-1-if (read_bbs (XFS_SB_DADDR ,1 , &bufp,NULL )) { +if (read_buf (XFS_SB_DADDR ,1 , bufp)) {E.g. why isn't this already using the normal libxfs routines?
(there probably is an explanation but I don't quite see it yet..)
Because the first step is to get rid of the dependency on a basic block map array interface for all callers. i.e. this patch separates out the read/write functions into contiguous buffer IO and non-contiguous buffer IO to match the two libxfs_buf IO interfaces.
Quoted 11 lines
db/io.c ring_add(void) · line 425
Quoted 6 lines
int
db/io.c +5
-0+read_buf ( + xfs_daddr_t bbno, + int count, + void *bufp) +{Is
read_bufreally a good name for something that is a trivial pread wrapper and doesn't deal with buffers? Should the function have some comments explaining when to use it? The same also applies to the write side.
It ends up going away - it's just a temporary step in switching over to the libxfs code. i.e. this changes the API without changing the implementation.
Quoted 15 lines
Quoted 15 lines
db/io.c +15
-0+static void +write_cur_buf (void ) +{ +int ret; + + ret =write_buf (iocur_top->bb, iocur_top->blen, iocur_top->buf); + +if (ret ==-1 ) +dbprintf (_ ("incomplete write, block: %lld\n" ), + (iocur_base + iocur_sp)->bb); +else if (ret !=0 ) +dbprintf (_ ("write error: %s\n" ),strerror (ret)); + +/* re-read buffer from disk */ + ret =read_buf (iocur_top->bb, iocur_top->blen, iocur_top->buf);
Quoted 8 lines
db/io.c ring_add(void) · line 472
Quoted 6 lines
db/io.c +6
-0+if (ret ==-1 ) +dbprintf (_ ("incomplete read, block: %lld\n" ), + (iocur_base + iocur_sp)->bb); +else if (ret !=0 ) +dbprintf (_ ("read error: %s\n" ),strerror (ret)); +}What is the point of the write and re-read cycle?
That's just what the current code does - I'm not changing the logic of operation here. I can add patches at the end of the series to do this - I assume it was intended to catch bad writes (e.g. media errors) back in the days of Irix...
Quoted 4 lines
Quoted 4 lines
db/io.c +2
-0+for (j =0 ; j < count; j++) { + bbno = bbmap->b[j];if (lseek64 (x.dfd, bbno <<BBSHIFT ,SEEK_SET ) <0 ) { rval = errno;
Quoted 8 lines
db/io.c write_bbs( · line 494
Quoted 6 lines
db/io.c +1
-1dbprintf (_ ("can't seek in filesystem at bb %lld\n" ), bbno);return rval; } - c =BBTOB (bbmap ?1 : count); + c =BBTOB (1 ); i = (int )write (x.dfd, (char *)bufp +BBTOB (j), c);Shoiuldn't this use the
write_bufhelper above?
Possibly, but this is just removing the single contiguous buffer case from the implementation now that it is never called in that way. The entire implemenation is converted to libxfs_buf*map IO calls later on, so there's not much point modifying it further at this point.
Quoted 1 line
And
read_bufhere?
Same again.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 15/30] db: separate out straight buffer IO from map based IO.
Quoted 4 lines · Dave Chinner
Quoted 2 lines
The patch description is a little too short, there's not real explanation of what it actually does.
I can fix that.
Thanks. I think all the answers you gave to my comments make a good enough explanation for this patch.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 16/30 db: rewrite bbmap to use xfs_buf_map
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Use the libxfs struct xfs_buf_map for recording the extent layout of discontiguous buffers and convert the read/write to decode them directory and use read_buf/write_buf to do the extent IO. This brings the physical xfs_db IO code to be very close to the model that libxfs uses.
--- db/bmap.c | 15 ++++----------- db/io.c | 58 ++++++++++++---------------------------------------------- db/io.h | 3 ++- 3 files changed, 18 insertions(+), 58 deletions(-)
db/bmap.c +4 -11
@@ -293,20 +293,13 @@ make_bbmap(int nex, bmap_ext_t *bmp) { - int d; -xfs_dfsbno_t dfsbno; int i; -int j; -int k; -for (i =0 , d =0 ; i < nex; i++) { - dfsbno = bmp[i].startblock; -for (j =0 ; j < bmp[i].blockcount; j++, dfsbno++) { -for (k =0 ; k < blkbb; k++) - bbmap->b[d++] = -XFS_FSB_TO_DADDR (mp, dfsbno) + k; - } +for (i =0 ; i < nex; i++) { + bbmap->b[i].bm_bn =XFS_FSB_TO_DADDR (mp, bmp[i].startblock); + bbmap->b[i].bm_len =XFS_FSB_TO_BB (mp, bmp[i].blockcount); } + bbmap->nmaps = nex; } static xfs_fsblock_t
db/io.c +12 -46
85 lines not on this page · raw message
db/io.h +2 -1
@@ -20,7 +20,8 @@ struct typ;#define BBMAP_SIZE (XFS_MAX_BLOCKSIZE / BBSIZE)typedef struct bbmap { -__int64_t b[BBMAP_SIZE ]; +int nmaps; +struct xfs_buf_map b[BBMAP_SIZE ]; }bbmap_t ; typedef struct iocur {
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 16/30] db: rewrite bbmap to use xfs_buf_map
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Use the libxfs struct
xfs_buf_mapfor recording the extent layout of discontiguous buffers and convert the read/write to decode them directory and use read_buf/write_buf to do the extent IO. This brings the physicalxfs_dbIO code to be very close to the model that libxfs uses.
Looks reasonable.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 17/30 db: rewrite IO engine to use libxfs
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Now that we have buffers and xfs_buf_maps, it is relatively easy to convert the IO engine to use libxfs routines. This gets rid of the most of the differences between mapped and straight buffer reads, and tracks xfs_bufs directly in the IO context that is being used.
This is not yet a perfect solution, as xfs_db does different sized IOs for the same block range which will throw warnings like:
xfs_db> inode 647ffff7fde740: Badness in key lookup (length) bp=(bno 0x40, len 8192 bytes) key=(bno 0x40, len 4096 bytes) xfs_db>
This is when first displaying an inode in the root inode chunk. These will need to be dealt with on a case by case basis.
--- db/init.c | 25 ++++++-- db/io.c | 176 ++++++++++++++----------------------------------------- db/io.h | 4 +- include/libxfs.h | 3 + libxfs/init.c | 5 ++ libxfs/rdwr.c | 61 ++++++++++++------- 6 files changed, 116 insertions(+), 158 deletions(-)
db/init.c +21 -4
@@ -54,8 +54,8 @@ init(int argc, char **argv) { - xfs_sb_t *sbp; -char bufp[BBSIZE ]; + struct xfs_sb *sbp; +struct xfs_buf *bp;int c;setlocale (LC_ALL ,"" );
@@ -115,14 +115,25 @@ init(exit (1 ); } -if (read_buf (XFS_SB_DADDR ,1 , bufp)) { +/* +* Read the superblock, but don't validate it - we are a diagnostic +* tool and so need to be able to mount busted filesystems. +*/ +memset (&xmount,0 ,sizeof (struct xfs_mount )); +libxfs_buftarg_init (&xmount, x.ddev, x.logdev, x.rtdev); + bp =libxfs_readbuf (xmount.m_ddev_targp,XFS_SB_DADDR , +1 << (XFS_MAX_SECTORSIZE_LOG -BBSHIFT ),0 ,NULL ); + +if (!bp || bp->b_error) {fprintf (stderr,_ ("%s: %s is invalid (cannot read first 512 " "bytes)\n" ), progname, fsdevice);exit (1 ); }/* copy SB from buffer to in-core, converting architecture as we go */ -libxfs_sb_from_disk (&xmount.m_sb, (struct xfs_dsb *)bufp); +libxfs_sb_from_disk (&xmount.m_sb,XFS_BUF_TO_SBP (bp)); +libxfs_putbuf (bp); +libxfs_purgebuf (bp); sbp = &xmount.m_sb;if (sbp->sb_magicnum !=XFS_SB_MAGIC ) {
@@ -201,6 +212,12 @@ main( } close_devices: +/* +* make sure that we pop the last buffer context we held so that the +* buffer is released before purge the caches during unmount. +*/ +pop_cur (); +libxfs_umount (mp);if (x.ddev)libxfs_device_close (x.ddev);if (x.logdev && x.logdev != x.ddev)
db/io.c +45 -131
261 lines not on this page · raw message
db/io.h +2 -2
@@ -36,8 +36,8 @@ typedef struct iocur {__uint16_t mode;/* current inode's mode */ xfs_off_t off;/* fs offset of "data" in bytes */ const struct typ *typ;/* type of "data" */ -int use_bbmap;/* set if bbmap is valid */ -bbmap_t bbmap;/* map daddr if fragmented */ +bbmap_t *bbmap;/* map daddr if fragmented */ +struct xfs_buf *bp;/* underlying buffer */ } iocur_t;#define DB_RING_ADD 1/* add to ring on set_cur */
include/libxfs.h +3 -0
@@ -447,7 +447,10 @@ extern xfs_buf_t *libxfs_getbufr(struct xfs_buftarg *, xfs_daddr_t, int);extern void libxfs_putbufr (xfs_buf_t *);extern int libxfs_writebuf_int (xfs_buf_t *,int ); +extern int libxfs_writebufr (struct xfs_buf *);extern int libxfs_readbufr (struct xfs_buftarg *,xfs_daddr_t ,xfs_buf_t *,int ,int ); +extern int libxfs_readbufr_map (struct xfs_buftarg *,struct xfs_buf *, +struct xfs_buf_map *,int ,int );extern int libxfs_bhash_size;extern int libxfs_ihash_size;
libxfs/init.c +5 -0
@@ -688,6 +688,9 @@ libxfs_mount(libxfs_buftarg_init (mp, dev, logdev, rtdev); mp->m_flags = (LIBXFS_MOUNT_32BITINODES |LIBXFS_MOUNT_32BITINOOPT ); +if (flags &LIBXFS_MOUNT_ROOTINOS ) + mp->m_flags |=LIBXFS_MOUNT_ROOTINOS ; + mp->m_sb = *sb;INIT_RADIX_TREE (&mp->m_perag_tree,GFP_KERNEL ); sbp = &(mp->m_sb);
@@ -866,6 +869,8 @@ libxfs_umount(xfs_mount_t *mp)int agno;libxfs_rtmount_destroy (mp); +if ((mp->m_flags &LIBXFS_MOUNT_ROOTINOS ) && mp->m_rootip) +libxfs_iput (mp->m_rootip,0 );libxfs_icache_purge ();libxfs_bcache_purge ();
libxfs/rdwr.c +40 -21
76 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 17/30] db: rewrite IO engine to use libxfs
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Now that we have buffers and
xfs_buf_maps, it is relatively easy to convert the IO engine to use libxfs routines. This gets rid of the most of the differences between mapped and straight buffer reads, and tracksxfs_bufsdirectly in the IO context that is being used.
Quoted 7 lines
This is not yet a perfect solution, as
xfs_dbdoes different sized IOs for the same block range which will throw warnings like:
xfs_db> inode 647ffff7fde740: Badness in key lookup (length) bp=(bno0x40, len 8192 bytes) key=(bno0x40, len 4096 bytes)xfs_db>
Maybe we should use uncached buffer I/O in xfs_db, similar to what we do in the mount path in the kernel?
libxfs/init.c libxfs_umount(xfs_mount_t *mp) · line 873
Quoted 13 lines
libxfs/init.c +5
-0mp->m_flags = (LIBXFS_MOUNT_32BITINODES |LIBXFS_MOUNT_32BITINOOPT ); +if (flags &LIBXFS_MOUNT_ROOTINOS ) + mp->m_flags |=LIBXFS_MOUNT_ROOTINOS ; + mp->m_sb = *sb;INIT_RADIX_TREE (&mp->m_perag_tree,GFP_KERNEL ); sbp = &(mp->m_sb);@@ -866,6 +869,8 @@ libxfs_umount(xfs_mount_t *mp)int agno;libxfs_rtmount_destroy (mp); +if ((mp->m_flags &LIBXFS_MOUNT_ROOTINOS ) && mp->m_rootip) +libxfs_iput (mp->m_rootip,0 );
Seems like the patch to remove m_rootip should go before this one?
libxfs/rdwr.c libxfs_readbuf(struct xfs_buftarg *btp, xfs_daddr_t blkno, int len, int flags, · line 724
Quoted 6 lines
libxfs/rdwr.c +3
-3-struct xfs_buf * -libxfs_readbuf_map (struct xfs_buftarg *btp,struct xfs_buf_map *map,int nmaps, -int flags,const struct xfs_buf_ops *ops) +int +libxfs_readbufr_map (struct xfs_buftarg *btp,struct xfs_buf *bp, +struct xfs_buf_map *map,int nmaps,int flags)
Shouldn't these sort of changes go into a separate patch?
Otherwise looks good to me.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 17/30] db: rewrite IO engine to use libxfs
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Now that we have buffers and
xfs_buf_maps, it is relatively easy to convert the IO engine to use libxfs routines. This gets rid of the most of the differences between mapped and straight buffer reads, and tracksxfs_bufsdirectly in the IO context that is being used.This is not yet a perfect solution, as
xfs_dbdoes different sized IOs for the same block range which will throw warnings like:
Quoted 7 lines
Quoted 4 lines
xfs_db> inode 647ffff7fde740: Badness in key lookup (length) bp=(bno0x40, len 8192 bytes) key=(bno0x40, len 4096 bytes)xfs_db>Maybe we should use uncached buffer I/O in
xfs_db, similar to what we do in the mount path in the kernel?
Possibly. I have thought about it, but we'd need a new interface and a rework of the libxfs read/write code to do that, and that has wider impact than just xfs_db. I also found lots of interesting issues as a result of the re-read behaviour which I wouldn't have easily found otherwise. i.e. the output is an early warning that something isn't quite right in xfs_db, so it does have some use right now.
Quoted 10 lines
Quoted 10 lines
libxfs/init.c +3
-0mp->m_flags = (LIBXFS_MOUNT_32BITINODES |LIBXFS_MOUNT_32BITINOOPT ); +if (flags &LIBXFS_MOUNT_ROOTINOS ) + mp->m_flags |=LIBXFS_MOUNT_ROOTINOS ; + mp->m_sb = *sb;INIT_RADIX_TREE (&mp->m_perag_tree,GFP_KERNEL ); sbp = &(mp->m_sb);@@ -866,6 +869,8 @@ libxfs_umount(xfs_mount_t *mp)int agno;
Quoted 5 lines
libxfs/init.c libxfs_umount(xfs_mount_t *mp) · line 873
Quoted 3 lines
libxfs/init.c +2
-0libxfs_rtmount_destroy (mp); +if ((mp->m_flags &LIBXFS_MOUNT_ROOTINOS ) && mp->m_rootip) +libxfs_iput (mp->m_rootip,0 );Seems like the patch to remove
m_rootipshould go before this one?
Yeah, good idea.
Quoted 8 lines
libxfs/rdwr.c libxfs_readbuf(struct xfs_buftarg *btp, xfs_daddr_t blkno, int len, int flags, · line 724
Quoted 6 lines
libxfs/rdwr.c +3
-3-struct xfs_buf * -libxfs_readbuf_map (struct xfs_buftarg *btp,struct xfs_buf_map *map,int nmaps, -int flags,const struct xfs_buf_ops *ops) +int +libxfs_readbufr_map (struct xfs_buftarg *btp,struct xfs_buf *bp, +struct xfs_buf_map *map,int nmaps,int flags)Shouldn't these sort of changes go into a separate patch?
I can do that.
Thanks for looking at these patches, Christoph.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 18/30 db: introduce verifier support into set_cur
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
To be able to use read and write verifiers, we need to pass the verifier to the IO routines. We do this via the set_cur() function used to trigger reading the buffer.
For most metadata types, there is only one type of verifier needed. For these, we can simply add the verifier to the type table entry for the given type and use that directly. This type entry is already carried around by the IO context, so if we ever need to get it again we have direct access to it in the context we'll be doing IO.
Only attach the verifiers to the v5 filesystem type table; there is not need for them on v4 filesystems as we don't have to verify or calculate CRCs for them.
There are some metadata types that have more than one buffer format, or aren't based in directly in buffers. For these, leave the type table verifier NULL for now - these will need to be addressed individually.
--- db/io.c | 13 +++++++--- db/type.c | 88 ++++++++++++++++++++++++++++++++++----------------------------- db/type.h | 1 + 3 files changed, 59 insertions(+), 43 deletions(-)
db/io.c +10 -3
@@ -480,12 +480,14 @@ set_cur(xfs_ino_t dirino;xfs_ino_t ino;__uint16_t mode; +const struct xfs_buf_ops *ops = t ? t->bops :NULL ;if (iocur_sp <0 ) {dbprintf (_ ("set_cur no stack element to set\n" ));return ; } + ino = iocur_top->ino; dirino = iocur_top->dirino; mode = iocur_top->mode;
@@ -507,12 +509,17 @@ set_cur(return ;memcpy (iocur_top->bbmap, bbmap,sizeof (struct bbmap )); bp =libxfs_readbuf_map (mp->m_ddev_targp, bbmap->b, - bbmap->nmaps,0 ,NULL ); + bbmap->nmaps,0 , ops); }else { - bp =libxfs_readbuf (mp->m_ddev_targp, d, c,0 ,NULL ); + bp =libxfs_readbuf (mp->m_ddev_targp, d, c,0 , ops); iocur_top->bbmap =NULL ; } -if (!bp || bp->b_error) + +/* +* keep the buffer even if the verifier says it is corrupted. +* We're a diagnostic tool, after all. +*/ +if (!bp || (bp->b_error && bp->b_error !=EFSCORRUPTED ))return ; iocur_top->buf = bp->b_addr; iocur_top->bp = bp;
db/type.c +48 -40
98 lines not on this page · raw message
db/type.h +1 -0
@@ -42,6 +42,7 @@ typedef struct typchar *name;pfunc_t pfunc;const struct field *fields; +const struct xfs_buf_ops *bops; } typ_t;extern const typ_t *typtab, *cur_typ;
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 18/30] db: introduce verifier support into set_cur
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 19/30 db: indicate if the CRC on a buffer is correct or not
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When dumping metadata that has a CRC in it, output not only the CRC but text to tell us whether the value is correct or not. Hence we can see at a glance if there's something wrong or not.
Do this by peeking at the buffer attached to the current IO contexted. If there was a CRC error, then it will be marked with a EFSCORRUPTED error. Use this to determine what to output.
--- db/agf.c | 2 +- db/agfl.c | 2 +- db/agi.c | 2 +- db/btblock.c | 10 +++++----- db/dir2.c | 4 ++-- db/dquot.c | 2 +- db/field.c | 5 +++++ db/field.h | 4 ++++ db/fprint.c | 39 +++++++++++++++++++++++++++++++++++++++ db/fprint.h | 2 ++ db/inode.c | 2 +- db/io.h | 6 ++++++ db/sb.c | 2 +- db/symlink.c | 2 +- 14 files changed, 70 insertions(+), 14 deletions(-)
db/agf.c +1 -1
@@ -71,7 +71,7 @@ const field_t agf_flds[] = { { "btreeblks", FLDT_UINT32D, OI(OFF(btreeblks)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE }, { "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE }, { NULL } };
db/agfl.c +1 -1
@@ -58,7 +58,7 @@ const field_t agfl_crc_flds[] = { { "seqno", FLDT_AGNUMBER, OI(OFF(seqno)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE }, { "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE }, { "bno", FLDT_AGBLOCKNZ, OI(OFF(bno)), agfl_bno_size, FLD_ARRAY|FLD_COUNT, TYP_DATA }, { NULL }
db/agi.c +1 -1
@@ -56,7 +56,7 @@ const field_t agi_flds[] = { CI(XFS_AGI_UNLINKED_BUCKETS), FLD_ARRAY, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE }, { "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE }, { NULL } };
db/btblock.c +5 -5
@@ -295,7 +295,7 @@ const field_t bmapbta_crc_flds[] = { { "lsn", FLDT_UINT64X, OI(OFF(u.l.bb_lsn)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(u.l.bb_uuid)), C1, 0, TYP_NONE }, { "owner", FLDT_INO, OI(OFF(u.l.bb_owner)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE }, { "recs", FLDT_BMAPBTAREC, btblock_rec_offset, btblock_rec_count, FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "keys", FLDT_BMAPBTAKEY, btblock_key_offset, btblock_key_count,
@@ -314,7 +314,7 @@ const field_t bmapbtd_crc_flds[] = { { "lsn", FLDT_UINT64X, OI(OFF(u.l.bb_lsn)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(u.l.bb_uuid)), C1, 0, TYP_NONE }, { "owner", FLDT_INO, OI(OFF(u.l.bb_owner)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE }, { "recs", FLDT_BMAPBTDREC, btblock_rec_offset, btblock_rec_count, FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "keys", FLDT_BMAPBTDKEY, btblock_key_offset, btblock_key_count,
@@ -405,7 +405,7 @@ const field_t inobt_crc_flds[] = { { "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE }, { "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE }, { "recs", FLDT_INOBTREC, btblock_rec_offset, btblock_rec_count, FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "keys", FLDT_INOBTKEY, btblock_key_offset, btblock_key_count,
@@ -471,7 +471,7 @@ const field_t bnobt_crc_flds[] = { { "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE }, { "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE }, { "recs", FLDT_BNOBTREC, btblock_rec_offset, btblock_rec_count, FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "keys", FLDT_BNOBTKEY, btblock_key_offset, btblock_key_count,
@@ -533,7 +533,7 @@ const field_t cntbt_crc_flds[] = { { "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE }, { "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE }, { "recs", FLDT_CNTBTREC, btblock_rec_offset, btblock_rec_count, FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE }, { "keys", FLDT_CNTBTKEY, btblock_key_offset, btblock_key_count,
db/dir2.c +2 -2
@@ -922,7 +922,7 @@ const field_t dir3_data_union_flds[] = {#define DBH3OFF (f) bitize(offsetof(struct xfs_dir3_blk_hdr, f))const field_t dir3_blkhdr_flds[] = { {"magic" ,FLDT_UINT32X ,OI (DBH3OFF (magic)),C1 ,0 ,TYP_NONE }, - {"crc" ,FLDT_UINT32X ,OI (DBH3OFF (crc)),C1 ,0 ,TYP_NONE }, + {"crc" ,FLDT_CRC ,OI (DBH3OFF (crc)),C1 ,0 ,TYP_NONE }, {"bno" ,FLDT_DFSBNO ,OI (DBH3OFF (blkno)),C1 ,0 ,TYP_BMAPBTD }, {"lsn" ,FLDT_UINT64X ,OI (DBH3OFF (lsn)),C1 ,0 ,TYP_NONE }, {"uuid" ,FLDT_UUID ,OI (DBH3OFF (uuid)),C1 ,0 ,TYP_NONE },
@@ -959,7 +959,7 @@ const field_t dir3_free_hdr_flds[] = {#define DB3OFF (f) bitize(offsetof(struct xfs_da3_blkinfo, f))const field_t da3_blkinfo_flds[] = { {"hdr" ,FLDT_DA_BLKINFO ,OI (DB3OFF (hdr)),C1 ,0 ,TYP_NONE }, - {"crc" ,FLDT_UINT32X ,OI (DB3OFF (crc)),C1 ,0 ,TYP_NONE }, + {"crc" ,FLDT_CRC ,OI (DB3OFF (crc)),C1 ,0 ,TYP_NONE }, {"bno" ,FLDT_DFSBNO ,OI (DB3OFF (blkno)),C1 ,0 ,TYP_BMAPBTD }, {"lsn" ,FLDT_UINT64X ,OI (DB3OFF (lsn)),C1 ,0 ,TYP_NONE }, {"uuid" ,FLDT_UUID ,OI (DB3OFF (uuid)),C1 ,0 ,TYP_NONE },
db/dquot.c +1 -1
@@ -48,7 +48,7 @@ const field_t dqblk_flds[] = { { "diskdq", FLDT_DISK_DQUOT, OI(DDOFF(diskdq)), C1, 0, TYP_NONE }, { "fill", FLDT_CHARS, OI(DDOFF(fill)), CI(DDSZC(fill)), FLD_SKIPALL, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(DDOFF(crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(DDOFF(crc)), C1, 0, TYP_NONE }, { "lsn", FLDT_UINT64X, OI(DDOFF(lsn)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(DDOFF(uuid)), C1, 0, TYP_NONE }, { NULL }
db/field.c +5 -0
@@ -163,6 +163,11 @@ const ftattr_t ftattrtab[] = { 0, fa_agblock, NULL }, {FLDT_CNTBTREC ,"cntbtrec" , fp_sarray, (char *)cntbt_rec_flds, SI(bitsz(xfs_alloc_rec_t)), 0, NULL, cntbt_rec_flds }, + +/* CRC field */ + { FLDT_CRC, "crc", fp_crc, "%#x (%s)", SI(bitsz(__uint32_t)), + 0, NULL, NULL }, + { FLDT_DEV, "dev", fp_num, "%#x", SI(bitsz(xfs_dev_t)), 0, NULL, NULL }, { FLDT_DFILOFFA, "dfiloffa", fp_num, "%llu", SI(bitsz(xfs_dfiloff_t)), 0, fa_dfiloffa, NULL },
db/field.h +4 -0
@@ -80,6 +80,10 @@ typedef enum fldt {FLDT_CNTBTKEY ,FLDT_CNTBTPTR ,FLDT_CNTBTREC , + +/* CRC field type */ +FLDT_CRC , +FLDT_DEV ,FLDT_DFILOFFA , FLDT_DFILOFFD,
db/fprint.c +39 -0
@@ -30,6 +30,7 @@#include "output.h" #include "sig.h" #include "malloc.h" +#include "io.h" int fp_charns(
@@ -184,3 +185,41 @@ fp_uuid( }return 1 ; } + +/* +* CRC is correct is the current buffer it is being pulled out +* of is not marked with a EFSCORRUPTED error. +*/ +int +fp_crc ( +void *obj, +int bit, +int count, +char *fmtstr, +int size, +int arg, +int base, +int array) +{ +int bitpos; +int i; +__int64_t val; +char *ok; + + ok =iocur_crc_valid () ?"correct" :"bad" ; + +for (i =0 , bitpos = bit; + i < count && !seenint (); + i++, bitpos += size) { +if (array) +dbprintf ("%d:" , i + base); + val =getbitval (obj, bitpos, size,BVUNSIGNED ); +if (size >32 ) +dbprintf (fmtstr, val, ok); +else +dbprintf (fmtstr, (__int32_t )val, ok); +if (i < count -1 ) +dbprintf (" " ); + } +return 1 ; +}
db/fprint.h +2 -0
@@ -29,3 +29,5 @@ extern int fp_time(void *obj, int bit, int count, char *fmtstr, int size, int arg, int base, int array);extern int fp_uuid (void *obj,int bit,int count,char *fmtstr,int size,int arg,int base,int array); +extern int fp_crc (void *obj,int bit,int count,char *fmtstr,int size, +int arg,int base,int array);
db/inode.c +1 -1
@@ -168,7 +168,7 @@ const field_t inode_core_flds[] = { };const field_t inode_v3_flds[] = { - {"crc" ,FLDT_UINT32X ,OI (COFF (crc)),C1 ,0 ,TYP_NONE }, + {"crc" ,FLDT_CRC ,OI (COFF (crc)),C1 ,0 ,TYP_NONE }, {"change_count" ,FLDT_UINT64D ,OI (COFF (changecount)),C1 ,0 ,TYP_NONE }, {"lsn" ,FLDT_UINT64X ,OI (COFF (lsn)),C1 ,0 ,TYP_NONE }, {"flags2" ,FLDT_UINT64X ,OI (COFF (flags2)),C1 ,0 ,TYP_NONE },
db/io.h +6 -0
@@ -58,3 +58,9 @@ extern void write_cur(void);extern void set_cur (const struct typ *t,__int64_t d,int c,int ring_add,bbmap_t *bbmap);extern void ring_add (void ); + +static inline bool +iocur_crc_valid () +{ +return (iocur_top->bp && iocur_top->bp->b_error !=EFSCORRUPTED ); +}
db/sb.c +1 -1
@@ -118,7 +118,7 @@ const field_t sb_flds[] = { C1, 0, TYP_NONE }, {"features_log_incompat" ,FLDT_UINT32X ,OI (OFF (features_log_incompat)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE }, { "pquotino", FLDT_INO, OI(OFF(pquotino)), C1, 0, TYP_INODE }, { "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE }, { NULL }
db/symlink.c +1 -1
@@ -69,7 +69,7 @@ const struct field symlink_crc_flds[] = { { "magic", FLDT_UINT32X, OI(OFF(magic)), C1, 0, TYP_NONE }, { "offset", FLDT_UINT32D, OI(OFF(offset)), C1, 0, TYP_NONE }, { "bytes", FLDT_UINT32D, OI(OFF(bytes)), C1, 0, TYP_NONE }, - { "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE }, + { "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE }, { "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE }, { "owner", FLDT_INO, OI(OFF(owner)), C1, 0, TYP_NONE }, { "bno", FLDT_DFSBNO, OI(OFF(blkno)), C1, 0, TYP_BMAPBTD },
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 19/30] db: indicate if the CRC on a buffer is correct or not
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 20/30 db: verify and calculate inode CRCs
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When we set the current Io cursor to point at an inode, verify that the inode CRC is intact. And prior to writing such an IO cursor, calculate the inode CRC.
--- db/inode.c | 2 ++ db/io.c | 4 ++++ db/io.h | 6 +++++- include/libxfs.h | 4 ++++ libxfs/xfs_inode_buf.c | 8 ++++---- 5 files changed, 19 insertions(+), 5 deletions(-)
db/inode.c +2 -0
@@ -655,6 +655,8 @@ set_cur_inode( blkbb, DB_RING_IGN, NULL);off_cur (offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize); dip = iocur_top->data; + iocur_top->ino_crc_ok =libxfs_dinode_verify (mp, ino, dip); + iocur_top->ino_buf =1 ; iocur_top->ino = ino; iocur_top->mode =be16_to_cpu (dip->di_mode);if ((iocur_top->mode &S_IFMT ) ==S_IFDIR )
db/io.c +4 -0
@@ -462,6 +462,9 @@ write_cur(void)return ; } +if (iocur_top->ino_buf) +libxfs_dinode_calc_crc (mp, iocur_top->data); +if (iocur_top->bbmap)write_cur_bbs ();else
@@ -534,6 +537,7 @@ set_cur( iocur_top->ino = ino; iocur_top->dirino = dirino; iocur_top->mode = mode; + iocur_top->ino_buf =0 ;/* store location in ring */ if (ring_flag)
db/io.h +5 -1
@@ -38,6 +38,8 @@ typedef struct iocur {const struct typ *typ;/* type of "data" */ bbmap_t *bbmap;/* map daddr if fragmented */ struct xfs_buf *bp;/* underlying buffer */ + int ino_crc_ok:1; + int ino_buf:1; } iocur_t;#define DB_RING_ADD 1/* add to ring on set_cur */
@@ -62,5 +64,7 @@ extern void ring_add(void);static inline bool iocur_crc_valid () { -return (iocur_top->bp && iocur_top->bp->b_error !=EFSCORRUPTED ); +return (iocur_top->bp && + iocur_top->bp->b_error !=EFSCORRUPTED && + (!iocur_top->ino_buf || iocur_top->ino_crc_ok)); }
include/libxfs.h +4 -0
@@ -755,6 +755,10 @@ void xfs_dinode_from_disk(struct xfs_icdinode *,#define libxfs_idata_realloc xfs_idata_realloc#define libxfs_idestroy_fork xfs_idestroy_fork +#define libxfs_dinode_verify xfs_dinode_verify +bool xfs_dinode_verify (struct xfs_mount *mp,xfs_ino_t ino, +struct xfs_dinode *dip); +/* xfs_sb.h */ #define libxfs_mod_sb xfs_mod_sb#define libxfs_sb_from_disk xfs_sb_from_disk
libxfs/xfs_inode_buf.c +4 -4
@@ -276,10 +276,10 @@ xfs_dinode_to_disk( } } -static bool +bool xfs_dinode_verify (struct xfs_mount *mp, -struct xfs_inode *ip, +xfs_ino_t ino,struct xfs_dinode *dip) {if (dip->di_magic !=cpu_to_be16 (XFS_DINODE_MAGIC ))
@@ -294,7 +294,7 @@ xfs_dinode_verify(if (!xfs_verify_cksum ((char *)dip, mp->m_sb.sb_inodesize, offsetof(struct xfs_dinode , di_crc)))return false; -if (be64_to_cpu (dip->di_ino) != ip->i_ino) +if (be64_to_cpu (dip->di_ino) != ino)return false;if (!uuid_equal (&dip->di_uuid, &mp->m_sb.sb_uuid))return false;
@@ -346,7 +346,7 @@ xfs_iread(return error;/* even unallocated inodes are verified */ -if (!xfs_dinode_verify (mp, ip, dip)) { +if (!xfs_dinode_verify (mp, ip->i_ino, dip)) {xfs_alert (mp,"%s: validation failed for inode %lld failed" , __func__, ip->i_ino);
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 20/30] db: verify and calculate inode CRCs
Quoted 1 line
+#define
libxfs_dinode_verifyxfs_dinode_verify
Is there really any point in adding this libxfs renaming in new code?
Otherwise looks fine,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 20/30] db: verify and calculate inode CRCs
Quoted 3 lines · Christoph Hellwig
Quoted 1 line
+#define
libxfs_dinode_verifyxfs_dinode_verifyIs there really any point in adding this libxfs renaming in new code?
Just following the convention used by all the other code...
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 21/30 db: verify and calculate dquot CRCs
In reply to Dave ChinnerWhen we set the current Io cursor to point at a dquot block, verify that the dquot CRC is intact. And prior to writing such an IO cursor, calculate the dquot CRC.
--- db/io.c | 5 ++++- db/io.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-)
db/io.c +4 -1
@@ -464,7 +464,9 @@ write_cur(void)if (iocur_top->ino_buf)libxfs_dinode_calc_crc (mp, iocur_top->data); - +if (iocur_top->dquot_buf) +xfs_update_cksum (iocur_top->data,sizeof (struct xfs_dqblk ), +XFS_DQUOT_CRC_OFF );if (iocur_top->bbmap)write_cur_bbs ();else
@@ -538,6 +540,7 @@ set_cur( iocur_top->dirino = dirino; iocur_top->mode = mode; iocur_top->ino_buf =0 ; + iocur_top->dquot_buf =0 ;/* store location in ring */ if (ring_flag)
db/io.h +1 -0
@@ -40,6 +40,7 @@ typedef struct iocur {struct xfs_buf *bp;/* underlying buffer */ int ino_crc_ok:1; int ino_buf:1; + int dquot_buf:1; } iocur_t;#define DB_RING_ADD 1/* add to ring on set_cur */
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 21/30] db: verify and calculate dquot CRCs
Quoted 15 lines · Dave Chinner
When we set the current Io cursor to point at a dquot block, verify that the dquot CRC is intact. And prior to writing such an IO cursor, calculate the dquot CRC.
Signed-off-by Dave Chinner <[email protected]>--- db/io.c | 5 ++++- db/io.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-)db/io.c +0
-0
db/io.c write_cur(void) · line 469
Quoted 7 lines
db/io.c +3
-1@@ -464,7 +464,9 @@ write_cur(void)if (iocur_top->ino_buf)libxfs_dinode_calc_crc (mp, iocur_top->data); - +if (iocur_top->dquot_buf) +xfs_update_cksum (iocur_top->data,sizeof (struct xfs_dqblk ), +XFS_DQUOT_CRC_OFF );
The patch looks good to get going, but it seems we'd be better off going through an ops vector here in some form..
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 21/30] db: verify and calculate dquot CRCs
Quoted 17 lines · Christoph Hellwig
Quoted 16 lines · Dave Chinner
When we set the current Io cursor to point at a dquot block, verify that the dquot CRC is intact. And prior to writing such an IO cursor, calculate the dquot CRC.
Signed-off-by Dave Chinner <[email protected]>--- db/io.c | 5 ++++- db/io.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-)db/io.c +0
-0@@ -464,7 +464,9 @@ write_cur(void)
Quoted 9 lines
db/io.c write_cur(void) · line 469
Quoted 6 lines
db/io.c +3
-1if (iocur_top->ino_buf)libxfs_dinode_calc_crc (mp, iocur_top->data); - +if (iocur_top->dquot_buf) +xfs_update_cksum (iocur_top->data,sizeof (struct xfs_dqblk ), +XFS_DQUOT_CRC_OFF );The patch looks good to get going, but it seems we'd be better off going through an ops vector here in some form..
Potentially - if we need a third object-within-a-buffer verification then it will be worthwhile to do this. However, for the moment we only have to support inodes and dquots and so for the moment I don't see that further abstraction is necessary.
Hence right now I'd prefer to make it explicit that we are doing these CRC calculations here and just keep in mind that if we are going to add more we need to abstract it into an ops vector at that time...
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 22/30 db: add a special directory buffer verifier
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Because we only have a single directory type that is used for all the different buffer types, we need to provide a special verifier for the read code. That verifier needs to know all the directory types and when it find one it knows about, switch to the correct verifier and call it.
We already do this for certain readahead cases in the directory code, so there is precedence for this. If we don't find a magic number we recognise, the verifier fails...
--- db/dir2.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ db/dir2.h | 2 ++ db/type.c | 3 ++- 3 files changed, 65 insertions(+), 1 deletion(-)
db/dir2.c +61 -0
@@ -24,6 +24,7 @@#include "field.h" #include "dir2.h" #include "init.h" +#include "output.h" static int dir2_block_hdr_count (void *obj,int startoff);static int dir2_block_leaf_count (void *obj,int startoff);
@@ -975,3 +976,63 @@ const field_t da3_node_hdr_flds[] = { { "pad", FLDT_UINT32D, OI(H3OFF(__pad32)), C1, 0, TYP_NONE }, { NULL } }; + +/* +* Special read verifier for directory buffers. detect the magic number +* appropriately and set the correct verifier and call it. +*/ +static void +xfs_dir3_db_read_verify ( +struct xfs_buf *bp) +{ +__be32 magic32; +__be16 magic16; + + magic32 = *(__be32 *)bp->b_addr; + magic16 = ((struct xfs_da_blkinfo *)bp->b_addr)->magic; + +switch (magic32) { +case cpu_to_be32 (XFS_DIR3_BLOCK_MAGIC ): + bp->b_ops = &xfs_dir3_block_buf_ops; + goto verify; +case cpu_to_be32 (XFS_DIR3_DATA_MAGIC ): + bp->b_ops = &xfs_dir3_data_buf_ops; + goto verify; +case cpu_to_be32 (XFS_DIR3_FREE_MAGIC ): + bp->b_ops = &xfs_dir3_free_buf_ops; + goto verify; +default : +break ; + } + +switch (magic16) { +case cpu_to_be16 (XFS_DIR3_LEAF1_MAGIC ): + bp->b_ops = &xfs_dir3_leaf1_buf_ops; +break ; +case cpu_to_be16 (XFS_DIR3_LEAFN_MAGIC ): + bp->b_ops = &xfs_dir3_leafn_buf_ops; +break ; +case cpu_to_be16 (XFS_DA3_NODE_MAGIC ): + bp->b_ops = &xfs_da3_node_buf_ops; +break ; +default : +dbprintf (_ ("Unknown directory buffer type!\n" )); +xfs_buf_ioerror (bp,EFSCORRUPTED ); +return ; + } +verify: + bp->b_ops->verify_read (bp); +} + +static void +xfs_dir3_db_write_verify ( +struct xfs_buf *bp) +{ +dbprintf (_ ("Writing unknown directory buffer type!\n" )); +xfs_buf_ioerror (bp,EFSCORRUPTED ); +} + +const struct xfs_buf_ops xfs_dir3_db_buf_ops = { + .verify_read = xfs_dir3_db_read_verify, + .verify_write = xfs_dir3_db_write_verify, +};
db/dir2.h +2 -0
@@ -60,3 +60,5 @@ static inline xfs_dir2_inou_t *xfs_dir2_sf_inumberp(xfs_dir2_sf_entry_t *sfep)extern int dir2_data_union_size (void *obj,int startoff,int idx);extern int dir2_size (void *obj,int startoff,int idx); + +extern const struct xfs_buf_ops xfs_dir3_db_buf_ops;
db/type.c +2 -1
@@ -87,7 +87,8 @@ static const typ_t __typtab_crc[] = { {TYP_CNTBT ,"cntbt" , handle_struct, cntbt_crc_hfld, &xfs_allocbt_buf_ops }, { TYP_DATA, "data", handle_block, NULL, NULL }, - { TYP_DIR2, "dir3", handle_struct, dir3_hfld, NULL }, + { TYP_DIR2, "dir3", handle_struct, dir3_hfld, + &xfs_dir3_db_buf_ops }, { TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld, &xfs_dquot_buf_ops }, { TYP_INOBT, "inobt", handle_struct, inobt_crc_hfld,
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 22/30] db: add a special directory buffer verifier
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Because we only have a single directory type that is used for all the different buffer types, we need to provide a special verifier for the read code. That verifier needs to know all the directory types and when it find one it knows about, switch to the correct verifier and call it.
Quoted 5 lines
We already do this for certain readahead cases in the directory code, so there is precedence for this. If we don't find a magic number we recognise, the verifier fails...
Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 23/30 db: add a special attribute buffer verifier
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Because we only have a single attribute type that is used for all the attribute buffer types, we need to provide a special verifier for the read code. That verifier needs to know all the attribute types and when it find one it knows about, switch to the correct verifier and call it.
--- db/attr.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ db/attr.h | 2 ++ db/type.c | 3 ++- 3 files changed, 55 insertions(+), 1 deletion(-)
db/attr.c +51 -0
@@ -25,6 +25,7 @@#include "attr.h" #include "io.h" #include "init.h" +#include "output.h" static int attr_leaf_entries_count (void *obj,int startoff);static int attr_leaf_hdr_count (void *obj,int startoff);
@@ -522,3 +523,53 @@ const field_t attr3_leaf_hdr_flds[] = { { NULL } }; +/* +* Special read verifier for attribute buffers. detect the magic number +* appropriately and set the correct verifier and call it. +*/ +static void +xfs_attr3_db_read_verify ( +struct xfs_buf *bp) +{ +__be32 magic32; +__be16 magic16; + + magic32 = *(__be32 *)bp->b_addr; + magic16 = ((struct xfs_da_blkinfo *)bp->b_addr)->magic; + +switch (magic16) { +case cpu_to_be16 (XFS_ATTR3_LEAF_MAGIC ): + bp->b_ops = &xfs_attr3_leaf_buf_ops; + goto verify; +case cpu_to_be16 (XFS_DA3_NODE_MAGIC ): + bp->b_ops = &xfs_da3_node_buf_ops; + goto verify; +default : +break ; + } + +switch (magic32) { +case cpu_to_be32 (XFS_ATTR3_RMT_MAGIC ): + bp->b_ops = &xfs_attr3_rmt_buf_ops; +break ; +default : +dbprintf (_ ("Unknown attribute buffer type!\n" )); +xfs_buf_ioerror (bp,EFSCORRUPTED ); +return ; + } +verify: + bp->b_ops->verify_read (bp); +} + +static void +xfs_attr3_db_write_verify ( +struct xfs_buf *bp) +{ +dbprintf (_ ("Writing unknown attribute buffer type!\n" )); +xfs_buf_ioerror (bp,EFSCORRUPTED ); +} + +const struct xfs_buf_ops xfs_attr3_db_buf_ops = { + .verify_read = xfs_attr3_db_read_verify, + .verify_write = xfs_attr3_db_write_verify, +};
db/attr.h +2 -0
@@ -33,3 +33,5 @@ extern const field_t attr3_node_hdr_flds[];extern int attr_leaf_name_size (void *obj,int startoff,int idx);extern int attr_size (void *obj,int startoff,int idx); + +extern const struct xfs_buf_ops xfs_attr3_db_buf_ops;
db/type.c +2 -1
@@ -77,7 +77,8 @@ static const typ_t __typtab_crc[] = { { TYP_AGF, "agf", handle_struct, agf_hfld, &xfs_agf_buf_ops }, { TYP_AGFL, "agfl", handle_struct, agfl_crc_hfld, &xfs_agfl_buf_ops }, { TYP_AGI, "agi", handle_struct, agi_hfld, &xfs_agfl_buf_ops }, - { TYP_ATTR, "attr3", handle_struct, attr3_hfld, NULL }, + { TYP_ATTR, "attr3", handle_struct, attr3_hfld, + &xfs_attr3_db_buf_ops }, { TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_crc_hfld, &xfs_bmbt_buf_ops }, { TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_crc_hfld,
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 23/30] db: add a special attribute buffer verifier
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 24/30 db: re-enable write support for v5 filesystems.
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
As we can now verify and recalculate CRCs on IO, we can modify the on-disk structures without corrupting the filesyste, This makes it safe to turn write support on for v5 filesystems for the first time.
--- db/init.c | 15 --------------- 1 file changed, 15 deletions(-)
db/init.c +0 -15
@@ -143,21 +143,6 @@ init(exit (EXIT_FAILURE ); } -/* -* Don't allow modifications to CRC enabled filesystems until we support -* CRC recalculation in the IO path. Unless, of course, the user is in -* the process of hitting us with a big hammer. -*/ -if (XFS_SB_VERSION_NUM (sbp) >=XFS_SB_VERSION_5 && - !(x.isreadonly &LIBXFS_ISREADONLY )) { -fprintf (stderr, -_ ("%s: modifications to %s are not supported in thi version.\n" -"Use \"-r\" to run %s in read-only mode on this filesystem .\n" ), - progname, fsdevice, progname); -if (!force) -exit (EXIT_FAILURE ); - } - mp =libxfs_mount (&xmount, sbp, x.ddev, x.logdev, x.rtdev,LIBXFS_MOUNT_ROOTINOS |LIBXFS_MOUNT_DEBUGGER );if (!mp) {
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 24/30] db: re-enable write support for v5 filesystems.
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 25/30 libxfs: fix root inode handling inconsistencies
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When "mounting" a filesystem via libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem, libxfs_unmount() used to unconditionally free root inodes if they were present.
This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell libxfs_mount() to read the root inode, and so when unmounting throws an error like:
cache_node_put: node put on refcount 0 (node=0x684610)
When a second libxfs_iput() call is made on the root inode.
Clean this up and fix up all the callers that use magic numbers rather than symbolic values to tell libxfs tomount the root inodes.
--- copy/xfs_copy.c | 3 ++- mkfs/xfs_mkfs.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)
copy/xfs_copy.c +2 -1
@@ -684,7 +684,8 @@ main(int argc, char **argv) sb = &mbuf.m_sb;libxfs_sb_from_disk (sb,XFS_BUF_TO_SBP (sbp)); - mp =libxfs_mount (&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev,1 ); + mp =libxfs_mount (&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, +LIBXFS_MOUNT_ROOTINOS );if (mp ==NULL ) {do_log (_ ("%s: %s filesystem failed to initialize\n" "%s: Aborting.\n" ), progname, source_name, progname);
mkfs/xfs_mkfs.c +2 -1
@@ -2582,6 +2582,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),memset (XFS_BUF_PTR (buf),0 , sectorsize);libxfs_sb_to_disk ((void *)XFS_BUF_PTR (buf), sbp,XFS_SB_ALL_BITS );libxfs_writebuf (buf,LIBXFS_EXIT_ON_FAILURE ); +libxfs_purgebuf (buf);/* * If the data area is a file, then grow it out to its final size
@@ -2616,7 +2617,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"), (xfs_extlen_t )XFS_FSB_TO_BB (mp, logblocks), &sbp->sb_uuid, logversion, lsunit, XLOG_FMT); - mp =libxfs_mount (mp, sbp, xi.ddev, xi.logdev, xi.rtdev,1 ); + mp =libxfs_mount (mp, sbp, xi.ddev, xi.logdev, xi.rtdev,0 );if (mp ==NULL ) {fprintf (stderr,_ ("%s: filesystem failed to initialize\n" ), progname);
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30] libxfs: fix root inode handling inconsistencies
Quoted 12 lines · Dave Chinner
From: Dave Chinner <[email protected]>
When "mounting" a filesystem via
libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem,libxfs_unmount()used to unconditionally free root inodes if they were present.This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell
libxfs_mount()to read the root inode, and so when unmounting throws an error like:
Quoted 6 lines
cache_node_put: node put on refcount 0 (node=0x684610)When a second
libxfs_iput()call is made on the root inode.Clean this up and fix up all the callers that use magic numbers rather than symbolic values to tell libxfs tomount the root inodes.
I think the right fix is to kill LIBXFS_MOUNT_ROOTINOS as nothing ever uses mp->m_rootip.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30] libxfs: fix root inode handling inconsistencies
Quoted 15 lines · Christoph Hellwig
Quoted 14 lines · Dave Chinner
From: Dave Chinner <[email protected]>
When "mounting" a filesystem via
libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem,libxfs_unmount()used to unconditionally free root inodes if they were present.This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell
libxfs_mount()to read the root inode, and so when unmounting throws an error like:
cache_node_put: node put on refcount 0 (node=0x684610)
Quoted 7 lines
Quoted 4 lines
When a second
libxfs_iput()call is made on the root inode.Clean this up and fix up all the callers that use magic numbers rather than symbolic values to tell libxfs tomount the root inodes.
I think the right fix is to kill
LIBXFS_MOUNT_ROOTINOSas nothing ever usesmp->m_rootip.
Good point - I never looked at the users of mp->m_rootip, so didn't notice this. I'll redo this patch to kill LIBXFS_MOUNT_ROOTINOS.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH v2 25/30 libxfs: fix root inode handling inconsistencies
From: Dave Chinner <[email protected]>
When "mounting" a filesystem via libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem, libxfs_unmount() used to unconditionally free root inodes if they were present.
This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell libxfs_mount() to read the root inode, and so when unmounting throws an error like:
cache_node_put: node put on refcount 0 (node=0x684610)
when a second libxfs_iput() call is made on the root inode.
As it turns out, nothing ever uses mp->m_rootip and so we don't need to read it in or free it, or even have a pointer to it in the struct xfs_mount, so kill the m_rootip and LIBXFS_MOUNT_ROOTINOS parameter completely.
---
V2: just kill m_rootino and LIBXFS_MOUNT_ROOTINOS altogether.
copy/xfs_copy.c | 2 +- db/init.c | 13 +++++-------- include/libxfs.h | 12 +++++------- libxfs/init.c | 27 ++++----------------------- mkfs/proto.c | 1 - mkfs/xfs_mkfs.c | 4 ++-- repair/phase6.c | 2 -- 7 files changed, 17 insertions(+), 44 deletions(-)
copy/xfs_copy.c +1 -1
@@ -684,7 +684,7 @@ main(int argc, char **argv) sb = &mbuf.m_sb;libxfs_sb_from_disk (sb,XFS_BUF_TO_SBP (sbp)); - mp =libxfs_mount (&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev,1 ); + mp =libxfs_mount (&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev,0 );if (mp ==NULL ) {do_log (_ ("%s: %s filesystem failed to initialize\n" "%s: Aborting.\n" ), progname, source_name, progname);
db/init.c +5 -8
@@ -144,15 +144,12 @@ init( } mp =libxfs_mount (&xmount, sbp, x.ddev, x.logdev, x.rtdev, -LIBXFS_MOUNT_ROOTINOS |LIBXFS_MOUNT_DEBUGGER ); +LIBXFS_MOUNT_DEBUGGER );if (!mp) { - mp =libxfs_mount (&xmount, sbp, x.ddev, x.logdev, x.rtdev, -LIBXFS_MOUNT_DEBUGGER ); -if (!mp) { -fprintf (stderr,_ ("%s: device %s unusable (not an XFS " -"filesystem?)\n" ), progname, fsdevice); -exit (1 ); - } +fprintf (stderr, +_ ("%s: device %s unusable (not an XFS filesystem?)\n" ), + progname, fsdevice); +exit (1 ); } blkbb =1 << mp->m_blkbb_log;
include/libxfs.h +5 -7
@@ -167,7 +167,6 @@ typedef struct xfs_mount {uint m_rsumsize;/* size of rt summary, bytes */ struct xfs_inode *m_rbmip;/* pointer to bitmap inode */ struct xfs_inode *m_rsumip;/* pointer to summary inode */ -struct xfs_inode *m_rootip;/* pointer to root directory */ struct xfs_buftarg *m_ddev_targp;struct xfs_buftarg *m_logdev_targp;struct xfs_buftarg *m_rtdev_targp;
@@ -259,12 +258,11 @@ typedef struct xfs_perag {int pagb_count;/* pagb slots in use */ } xfs_perag_t; -#define LIBXFS_MOUNT_ROOTINOS 0x0001 -#define LIBXFS_MOUNT_DEBUGGER 0x0002 -#define LIBXFS_MOUNT_32BITINODES 0x0004 -#define LIBXFS_MOUNT_32BITINOOPT 0x0008 -#define LIBXFS_MOUNT_COMPAT_ATTR 0x0010 -#define LIBXFS_MOUNT_ATTR2 0x0020 +#define LIBXFS_MOUNT_DEBUGGER 0x0001 +#define LIBXFS_MOUNT_32BITINODES 0x0002 +#define LIBXFS_MOUNT_32BITINOOPT 0x0004 +#define LIBXFS_MOUNT_COMPAT_ATTR 0x0008 +#define LIBXFS_MOUNT_ATTR2 0x0010#define LIBXFS_IHASHSIZE (sbp) (1<<10)#define LIBXFS_BHASHSIZE (sbp) (1<<10)
libxfs/init.c +4 -23
@@ -688,8 +688,6 @@ libxfs_mount(libxfs_buftarg_init (mp, dev, logdev, rtdev); mp->m_flags = (LIBXFS_MOUNT_32BITINODES |LIBXFS_MOUNT_32BITINOOPT ); -if (flags &LIBXFS_MOUNT_ROOTINOS ) - mp->m_flags |=LIBXFS_MOUNT_ROOTINOS ; mp->m_sb = *sb;INIT_RADIX_TREE (&mp->m_perag_tree,GFP_KERNEL );
@@ -813,30 +811,15 @@ libxfs_mount(exit (1 ); } -/* -* mkfs calls mount before the root inode is allocated. -*/ -if ((flags &LIBXFS_MOUNT_ROOTINOS ) && sbp->sb_rootino !=NULLFSINO ) { - error =libxfs_iget (mp,NULL , sbp->sb_rootino,0 , - &mp->m_rootip,0 ); -if (error) { -fprintf (stderr,_ ("%s: cannot read root inode (%d)\n" ), - progname, error); -if (!(flags &LIBXFS_MOUNT_DEBUGGER )) -return NULL ; - } -ASSERT (mp->m_rootip !=NULL ); - } -if ((flags &LIBXFS_MOUNT_ROOTINOS ) &&rtmount_inodes (mp)) { -if (mp->m_rootip) -libxfs_iput (mp->m_rootip,0 ); +/* set up the realtime inodes if they exist */ + error =rtmount_inodes (mp); +if (error)return NULL ; - }/* * mkfs calls mount before the AGF/AGI structures are written. */ -if ((flags &LIBXFS_MOUNT_ROOTINOS ) && sbp->sb_rootino !=NULLFSINO && +if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) { error =xfs_initialize_perag_data (mp, sbp->sb_agcount);if (error) {
@@ -869,8 +852,6 @@ libxfs_umount(xfs_mount_t *mp)int agno;libxfs_rtmount_destroy (mp); -if ((mp->m_flags &LIBXFS_MOUNT_ROOTINOS ) && mp->m_rootip) -libxfs_iput (mp->m_rootip,0 );libxfs_icache_purge ();libxfs_bcache_purge ();
mkfs/proto.c +0 -1
@@ -543,7 +543,6 @@ parseproto( pip = ip; mp->m_sb.sb_rootino = ip->i_ino;libxfs_mod_sb (tp,XFS_SB_ROOTINO ); - mp->m_rootip = ip; isroot =1 ; }else {libxfs_trans_ijoin (tp, pip,0 );
mkfs/xfs_mkfs.c +2 -2
@@ -2582,6 +2582,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),memset (XFS_BUF_PTR (buf),0 , sectorsize);libxfs_sb_to_disk ((void *)XFS_BUF_PTR (buf), sbp,XFS_SB_ALL_BITS );libxfs_writebuf (buf,LIBXFS_EXIT_ON_FAILURE ); +libxfs_purgebuf (buf);/* * If the data area is a file, then grow it out to its final size
@@ -2616,7 +2617,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"), (xfs_extlen_t )XFS_FSB_TO_BB (mp, logblocks), &sbp->sb_uuid, logversion, lsunit, XLOG_FMT); - mp =libxfs_mount (mp, sbp, xi.ddev, xi.logdev, xi.rtdev,1 ); + mp =libxfs_mount (mp, sbp, xi.ddev, xi.logdev, xi.rtdev,0 );if (mp ==NULL ) {fprintf (stderr,_ ("%s: filesystem failed to initialize\n" ), progname);
@@ -2887,7 +2888,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),/* * Allocate the root inode and anything else in the proto file. */ - mp->m_rootip =NULL ;parse_proto (mp, &fsx, &protostring);/*
repair/phase6.c +0 -2
@@ -856,8 +856,6 @@ mk_root_dir(xfs_mount_t *mp) ip->i_df.if_bytes = ip->i_df.if_real_bytes =0 ; ip->i_df.if_u1.if_extents =NULL ; - mp->m_rootip = ip; -/* * initialize the directory */
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30 V2] libxfs: fix root inode handling inconsistencies
Quoted 5 lines · Dave Chinner
libxfs/init.c +2
-3-if ((flags &LIBXFS_MOUNT_ROOTINOS ) &&rtmount_inodes (mp)) { -if (mp->m_rootip) -libxfs_iput (mp->m_rootip,0 ); +/* set up the realtime inodes if they exist */ + error =rtmount_inodes (mp);
libxfs/init.c libxfs_mount( · line 823
Quoted 10 lines
libxfs/init.c +2
-2+if (error)return NULL ; - }/* * mkfs calls mount before the AGF/AGI structures are written. */ -if ((flags &LIBXFS_MOUNT_ROOTINOS ) && sbp->sb_rootino !=NULLFSINO && +if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) {
Oh, I hadn't noticed that LIBXFS_MOUNT_ROOTINOS does more than reading the root inode. Seems like mkfs might need the rt inodes if a file with the rt flag is specified in the proto file.
I can't see how xfs_copy could need either the rt inodes nor the perag data.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30 V2] libxfs: fix root inode handling inconsistencies
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Dave Chinner
libxfs/init.c +3
-4-if ((flags &LIBXFS_MOUNT_ROOTINOS ) &&rtmount_inodes (mp)) { -if (mp->m_rootip) -libxfs_iput (mp->m_rootip,0 ); +/* set up the realtime inodes if they exist */ + error =rtmount_inodes (mp); +if (error)return NULL ; - }
Quoted 10 lines
libxfs/init.c libxfs_mount( · line 823
Quoted 6 lines
libxfs/init.c +1
-1/* * mkfs calls mount before the AGF/AGI structures are written. */ -if ((flags &LIBXFS_MOUNT_ROOTINOS ) && sbp->sb_rootino !=NULLFSINO && +if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) {Oh, I hadn't noticed that
LIBXFS_MOUNT_ROOTINOSdoes more than reading the root inode. Seems like mkfs might need the rt inodes if a file with the rt flag is specified in the proto file.
mkfs always allocates the rt inodes directly after the root directory is created.
Quoted 2 lines
I can't see how
xfs_copycould need either the rt inodes nor the perag data.
Right, it doesn't need them, but it doesn't hurt at all to initialise them because all the ag headers are about to be read to find all the used space, anyway.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30 V2] libxfs: fix root inode handling inconsistencies
Quoted 10 lines · Dave Chinner
Quoted 7 lines
libxfs/init.c libxfs_mount( · line 823
Quoted 3 lines
libxfs/init.c +1
-1-if ((flags &LIBXFS_MOUNT_ROOTINOS ) && sbp->sb_rootino !=NULLFSINO && +if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) {Oh, I hadn't noticed that
LIBXFS_MOUNT_ROOTINOSdoes more than reading the root inode. Seems like mkfs might need the rt inodes if a file with the rt flag is specified in the proto file.mkfs always allocates the rt inodes directly after the root directory is created.
True, we obviosuly create them in mkfs. In fact looking over the code the only call into the rtalloc code in userspace ever is mkfs initializing those using libxfs_rtfree_extent.
So removing the call to rtmount_inodes is fine, and we should remove the declaration of it as well as it's now unused. In the future I suspect we should simplify the mkfs code to not even require xfs_rtalloc.c for just marking the inodes as entirely free, but that's not something for this series.
Quoted 6 lines
Quoted 2 lines
I can't see how
xfs_copycould need either the rt inodes nor the perag data.Right, it doesn't need them, but it doesn't hurt at all to initialise them because all the ag headers are about to be read to find all the used space, anyway.
xfs_initialize_perag_data does the following:
- read in AGI/AGF. This is something we lazily do whenever we need it anyway, so no one should rely on it.
- update the in-core superblock global counters. Seems like the old xfs_check relies on this and still needs an equivalent if we care enough. No one else seems to care.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30 V2] libxfs: fix root inode handling inconsistencies
Quoted 12 lines · Christoph Hellwig
Quoted 10 lines · Dave Chinner
Quoted 7 lines
libxfs/init.c libxfs_mount( · line 823
Quoted 3 lines
libxfs/init.c +1
-1-if ((flags &LIBXFS_MOUNT_ROOTINOS ) && sbp->sb_rootino !=NULLFSINO && +if (sbp->sb_rootino !=NULLFSINO &&xfs_sb_version_haslazysbcount (&mp->m_sb)) {Oh, I hadn't noticed that
LIBXFS_MOUNT_ROOTINOSdoes more than reading the root inode. Seems like mkfs might need the rt inodes if a file with the rt flag is specified in the proto file.mkfs always allocates the rt inodes directly after the root directory is created.
Quoted 6 lines
True, we obviosuly create them in mkfs. In fact looking over the code the only call into the rtalloc code in userspace ever is mkfs initializing those using
libxfs_rtfree_extent.So removing the call to
rtmount_inodesis fine, and we should remove the declaration of it as well as it's now unused.
You are right, they are otherwise unused. repair and db just look at the raw inode numbers, and do direct libxfs_iget calls to get an xfs_inode and hence don't use the cached values at all. I'll kill it completely, then.
Quoted 4 lines
In the future I
suspect we should simplify the mkfs code to not even requirexfs_rtalloc.cfor just marking the inodes as entirely free, but that's not something for this series.
Yeah, it seems that way - xfs_repair does all the rebuild itself without needing the libxfs code, so that seems quite doable.
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
I can't see how
xfs_copycould need either the rt inodes nor the perag data.Right, it doesn't need them, but it doesn't hurt at all to initialise them because all the ag headers are about to be read to find all the used space, anyway.
Quoted 7 lines
xfs_initialize_perag_datadoes the following:- read in AGI/AGF. This is something we lazily do whenever we need it anyway, so no one should rely on it.
- update the in-core superblock global counters. Seems like the oldxfs_checkrelies on this and still needs an equivalent if we care enough. No one else seems to care.
So you are suggesting that I move that initialisation to the xfs_check code rather than just doing it in the mount code? Or something else?
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 25/30 V2] libxfs: fix root inode handling inconsistencies
Quoted 11 lines · Dave Chinner
Quoted 7 lines
xfs_initialize_perag_datadoes the following:- read in AGI/AGF. This is something we lazily do whenever we need it anyway, so no one should rely on it.
- update the in-core superblock global counters. Seems like the oldxfs_checkrelies on this and still needs an equivalent if we care enough. No one else seems to care.So you are suggesting that I move that initialisation to the
xfs_checkcode rather than just doing it in the mount code? Or something else?
That seems to be the sanest option if it works out. Otherwise we'd need to keep a flag to let libxfs do it.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 26/30 xfs_db: avoid libxfs buffer lookup warnings
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When we mount the filesystem inside xfs_db, libxfs is tasked with reading some information from disk, such as root inodes. Because libxfs does this inode reading, it uses inode cluster buffers to read the inodes. xfs_db, OTOH, just uses FSB sized buffers to read inodes, and hence xfs_db throws a warning when reading the root inode block like so:
$ sudo xfs_db -c "sb 0" -c "p rootino" -c "inode 32" /dev/vda Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled! Use of these features is at your own risk! rootino = 32 7f59f20e6740: Badness in key lookup (length) bp=(bno 0x20, len 8192 bytes) key=(bno 0x20, len 1024 bytes) $
There is another way this can happen, and that is dumping raw data from disk using either the "fsb NNN" or "daddr MMM" commands to dump untyped information. This is always read in sector or filesystem block units, and so will cause similar badness warnings.
xfs_db is unique in the way it can read the same blocks with different lengths, so we really need a way to avoid having duplicate buffers in the cache. To handle this in a generic way, introduce a "purge on compare failure" feature to libxfs.
What this feature does is instead of throwing a warning when a buffer miscompare occurs (e.g. due to a length mismatch), it purges the buffer that is in cache from the cache. We can do this safely in the context of xfs_db because it always writes back changes made to buffers before it releases the reference to the buffer. Hence we can purge buffers directly from the lookup code without having to worry about whether they are dirty or not.
Doing this purge on miscompare operation avoids the problem that libxfs is currently warning about, and hence if the feature flag is set then we don't need to warn about miscompares any more. Hence the whole problem goes away entirely for xfs_db, without affecting any of the other users of libxfs based IO.
--- db/init.c | 1 + db/inode.c | 33 +++++++++++++++++++-- db/io.c | 4 ++- include/cache.h | 22 +++++++++++++- include/libxfs.h | 2 ++ libxfs/cache.c | 83 +++++++++++++++++++++++++++++++++++++++-------------- libxfs/init.c | 6 ++-- libxfs/rdwr.c | 30 ++++++++++--------- repair/xfs_repair.c | 4 +-- 9 files changed, 143 insertions(+), 42 deletions(-)
db/init.c +1 -0
@@ -109,6 +109,7 @@ init(else x.dname = fsdevice; + x.bcache_flags =CACHE_MISCOMPARE_PURGE ;if (!libxfs_init (&x)) {fputs (_ ("\nfatal error -- couldn't initialize XFS library\n" ), stderr);
db/inode.c +31 -2
@@ -623,6 +623,14 @@ inode_u_symlink_count( (int)be64_to_cpu(dip->di_size) : 0; } +/* +* We are now using libxfs for our IO backend, so we should always try to use +* inode cluster buffers rather than filesystem block sized buffers for reading +* inodes. This means that we always use the same buffers as libxfs operations +* does, and that avoids buffer cache issues caused by overlapping buffers. This +* can be seen clearly when trying to read the root inode. Much of this logic is +* similar to libxfs_imap(). +*/ void set_cur_inode (xfs_ino_t ino)
@@ -632,6 +640,9 @@ set_cur_inode(xfs_agnumber_t agno;xfs_dinode_t *dip;int offset; +int numblks = blkbb; +xfs_agblock_t cluster_agbno; + agno =XFS_INO_TO_AGNO (mp, ino); agino =XFS_INO_TO_AGINO (mp, ino);
@@ -644,6 +655,24 @@ set_cur_inode(return ; } cur_agno = agno; + +if (mp->m_inode_cluster_size > mp->m_sb.sb_blocksize && + mp->m_inoalign_mask) { +xfs_agblock_t chunk_agbno; +xfs_agblock_t offset_agbno; +int blks_per_cluster; + + blks_per_cluster = mp->m_inode_cluster_size >> + mp->m_sb.sb_blocklog; + offset_agbno = agbno & mp->m_inoalign_mask; + chunk_agbno = agbno - offset_agbno; + cluster_agbno = chunk_agbno + + ((offset_agbno / blks_per_cluster) * blks_per_cluster); + offset += ((agbno - cluster_agbno) * mp->m_sb.sb_inopblock); + numblks =XFS_FSB_TO_BB (mp, blks_per_cluster); + }else + cluster_agbno = agbno; +/* * First set_cur to the block with the inode * then use off_cur to get the right part of the buffer.
@@ -651,8 +680,8 @@ set_cur_inode(ASSERT (typtab[TYP_INODE ].typnm ==TYP_INODE );/* ingore ring update here, do it explicitly below */ -set_cur (&typtab[TYP_INODE ],XFS_AGB_TO_DADDR (mp, agno, agbno), - blkbb,DB_RING_IGN ,NULL ); +set_cur (&typtab[TYP_INODE ],XFS_AGB_TO_DADDR (mp, agno, cluster_agbno), + numblks,DB_RING_IGN ,NULL );off_cur (offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize); dip = iocur_top->data; iocur_top->ino_crc_ok =libxfs_dinode_verify (mp, ino, dip);
db/io.c +3 -1
@@ -104,8 +104,10 @@ pop_cur(void)dbprintf (_ ("can't pop anything from I/O stack\n" ));return ; } -if (iocur_top->bp) +if (iocur_top->bp) {libxfs_putbuf (iocur_top->bp); + iocur_top->bp =NULL ; + }if (iocur_top->bbmap) {free (iocur_top->bbmap); iocur_top->bbmap =NULL ;
include/cache.h +21 -1
@@ -18,6 +18,25 @@#ifndef __CACHE_H__#define __CACHE_H__ +/* +* initialisation flags +*/ +/* +* xfs_db always writes changes immediately, and so we need to purge buffers +* when we get a buffer lookup mismatch due to readin the same block with a +* different buffer configuration. +*/ +#define CACHE_MISCOMPARE_PURGE (1 << 0) + +/* +* cache object campare return values +*/ +enum { +CACHE_HIT , +CACHE_MISS , +CACHE_PURGE , +}; +#define HASH_CACHE_RATIO 8/*
@@ -82,6 +101,7 @@ struct cache_node { };struct cache { +int c_flags;/* behavioural flags */ unsigned int c_maxcount;/* max cache nodes */ unsigned int c_count;/* count of nodes */ pthread_mutex_t c_mutex;/* node count mutex */
@@ -99,7 +119,7 @@ struct cache {unsigned int c_max;/* max nodes ever used */ }; -struct cache *cache_init (unsigned int ,struct cache_operations *); +struct cache *cache_init (int ,unsigned int ,struct cache_operations *);void cache_destroy (struct cache *);void cache_walk (struct cache *,cache_walk_t );void cache_purge (struct cache *);
include/libxfs.h +2 -0
@@ -110,6 +110,8 @@ typedef struct {int dfd;/* data subvolume file descriptor */ int logfd;/* log subvolume file descriptor */ int rtfd;/* realtime subvolume file descriptor */ +int icache_flags;/* cache init flags */ +int bcache_flags;/* cache init flags */ } libxfs_init_t;#define LIBXFS_EXIT_ON_FAILURE 0x0001/* exit the program if a call fails */
libxfs/cache.c +62 -21
140 lines not on this page · raw message
libxfs/init.c +4 -2
@@ -337,10 +337,12 @@ libxfs_init(libxfs_init_t *a)chdir (curdir);if (!libxfs_ihash_size) libxfs_ihash_size =LIBXFS_IHASHSIZE (sbp); - libxfs_icache =cache_init (libxfs_ihash_size, &libxfs_icache_operations); + libxfs_icache =cache_init (a->icache_flags, libxfs_ihash_size, + &libxfs_icache_operations);if (!libxfs_bhash_size) libxfs_bhash_size =LIBXFS_BHASHSIZE (sbp); - libxfs_bcache =cache_init (libxfs_bhash_size, &libxfs_bcache_operations); + libxfs_bcache =cache_init (a->bcache_flags, libxfs_bhash_size, + &libxfs_bcache_operations); use_xfs_buf_lock = a->usebuflock;manage_zones (0 ); rval =1 ;
libxfs/rdwr.c +17 -13
@@ -323,20 +323,24 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key)struct xfs_buf *bp = (struct xfs_buf *)node;struct xfs_bufkey *bkey = (struct xfs_bufkey *)key; -#ifdef IO_BCOMPARE_CHECK if (bp->b_target->dev == bkey->buftarg->dev && - bp->b_bn == bkey->blkno && - bp->b_bcount !=BBTOB (bkey->bblen)) -fprintf (stderr,"%lx: Badness in key lookup (length)\n" -"bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n" , -pthread_self (), - (unsigned long long )bp->b_bn, (int )bp->b_bcount, - (unsigned long long )bkey->blkno,BBTOB (bkey->bblen)); + bp->b_bn == bkey->blkno) { +if (bp->b_bcount ==BBTOB (bkey->bblen)) +return CACHE_HIT ; +#ifdef IO_BCOMPARE_CHECK +if (!(libxfs_bcache->c_flags &CACHE_MISCOMPARE_PURGE )) { +fprintf (stderr, +"%lx: Badness in key lookup (length)\n" +"bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n" , +pthread_self (), + (unsigned long long )bp->b_bn, (int )bp->b_bcount, + (unsigned long long )bkey->blkno, +BBTOB (bkey->bblen)); + }#endif - -return (bp->b_target->dev == bkey->buftarg->dev && - bp->b_bn == bkey->blkno && - bp->b_bcount ==BBTOB (bkey->bblen)); +return CACHE_PURGE ; + } +return CACHE_MISS ; }void
@@ -1029,7 +1033,7 @@ libxfs_icompare(struct cache_node *node, cache_key_t key) {xfs_inode_t *ip = (xfs_inode_t *)node; -return (ip->i_ino == *(xfs_ino_t *)key); +return (ip->i_ino == *(xfs_ino_t *)key) ?CACHE_HIT :CACHE_MISS ; }int
repair/xfs_repair.c +2 -2
@@ -712,9 +712,9 @@ main(int argc, char **argv)if (!ihash_option_used) libxfs_ihash_size = libxfs_bhash_size; - libxfs_icache =cache_init (libxfs_ihash_size, + libxfs_icache =cache_init (0 , libxfs_ihash_size, &libxfs_icache_operations); - libxfs_bcache =cache_init (libxfs_bhash_size, + libxfs_bcache =cache_init (0 , libxfs_bhash_size, &libxfs_bcache_operations); }
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 26/30] xfs_db: avoid libxfs buffer lookup warnings
Quoted 34 lines
db/inode.c +19
-0+/* +* We are now using libxfs for our IO backend, so we should always try to use +* inode cluster buffers rather than filesystem block sized buffers for reading +* inodes. This means that we always use the same buffers as libxfs operations +* does, and that avoids buffer cache issues caused by overlapping buffers. This +* can be seen clearly when trying to read the root inode. Much of this logic is +* similar to libxfs_imap(). +*/ void set_cur_inode (xfs_ino_t ino)@@ -632,6 +640,9 @@ set_cur_inode(xfs_agnumber_t agno;xfs_dinode_t *dip;int offset; +int numblks = blkbb; +xfs_agblock_t cluster_agbno; + agno =XFS_INO_TO_AGNO (mp, ino); agino =XFS_INO_TO_AGINO (mp, ino);@@ -644,6 +655,24 @@ set_cur_inode(return ; } cur_agno = agno; + +if (mp->m_inode_cluster_size > mp->m_sb.sb_blocksize && + mp->m_inoalign_mask) { +xfs_agblock_t chunk_agbno; +xfs_agblock_t offset_agbno; +int blks_per_cluster; + + blks_per_cluster = mp->m_inode_cluster_size >>
db/inode.c set_cur_inode( · line 675
Quoted 10 lines
db/inode.c +10
-0+ mp->m_sb.sb_blocklog; + offset_agbno = agbno & mp->m_inoalign_mask; + chunk_agbno = agbno - offset_agbno; + cluster_agbno = chunk_agbno + + ((offset_agbno / blks_per_cluster) * blks_per_cluster); + offset += ((agbno - cluster_agbno) * mp->m_sb.sb_inopblock); + numblks =XFS_FSB_TO_BB (mp, blks_per_cluster); + }else + cluster_agbno = agbno; +
Seems like this should be a separate patch?
db/io.c pop_cur(void) · line 110
Quoted 5 lines
db/io.c +3
-1-if (iocur_top->bp) +if (iocur_top->bp) {libxfs_putbuf (iocur_top->bp); + iocur_top->bp =NULL ; + }
This should probably be folded into the patch that started using buffers in xfs_db.
include/libxfs.h typedef struct { · line 114
Quoted 2 lines
include/libxfs.h +2
-0+int icache_flags;/* cache init flags */ +int bcache_flags;/* cache init flags */
The icache_flags aren't used anywhere. Also I'd really prefer to have my patch to kill the inode cache istead of adding more changes to it.
libxfs/rdwr.c libxfs_bcompare(struct cache_node *node, cache_key_t key) · line 340
Quoted 11 lines
libxfs/rdwr.c +10
-0+#ifdef IO_BCOMPARE_CHECK +if (!(libxfs_bcache->c_flags &CACHE_MISCOMPARE_PURGE )) { +fprintf (stderr, +"%lx: Badness in key lookup (length)\n" +"bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n" , +pthread_self (), + (unsigned long long )bp->b_bn, (int )bp->b_bcount, + (unsigned long long )bkey->blkno, +BBTOB (bkey->bblen)); + }#endif
What is the point of the IO_BCOMPARE_CHECK ifdef if we unconditionally define it?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 26/30] xfs_db: avoid libxfs buffer lookup warnings
Quoted 38 lines · Christoph Hellwig
Quoted 38 lines
db/inode.c +23
-0+/* +* We are now using libxfs for our IO backend, so we should always try to use +* inode cluster buffers rather than filesystem block sized buffers for reading +* inodes. This means that we always use the same buffers as libxfs operations +* does, and that avoids buffer cache issues caused by overlapping buffers. This +* can be seen clearly when trying to read the root inode. Much of this logic is +* similar to libxfs_imap(). +*/ void set_cur_inode (xfs_ino_t ino)@@ -632,6 +640,9 @@ set_cur_inode(xfs_agnumber_t agno;xfs_dinode_t *dip;int offset; +int numblks = blkbb; +xfs_agblock_t cluster_agbno; + agno =XFS_INO_TO_AGNO (mp, ino); agino =XFS_INO_TO_AGINO (mp, ino);@@ -644,6 +655,24 @@ set_cur_inode(return ; } cur_agno = agno; + +if (mp->m_inode_cluster_size > mp->m_sb.sb_blocksize && + mp->m_inoalign_mask) { +xfs_agblock_t chunk_agbno; +xfs_agblock_t offset_agbno; +int blks_per_cluster; + + blks_per_cluster = mp->m_inode_cluster_size >> + mp->m_sb.sb_blocklog; + offset_agbno = agbno & mp->m_inoalign_mask; + chunk_agbno = agbno - offset_agbno; + cluster_agbno = chunk_agbno +
Quoted 8 lines
db/inode.c set_cur_inode( · line 675
Quoted 6 lines
db/inode.c +6
-0+ ((offset_agbno / blks_per_cluster) * blks_per_cluster); + offset += ((agbno - cluster_agbno) * mp->m_sb.sb_inopblock); + numblks =XFS_FSB_TO_BB (mp, blks_per_cluster); + }else + cluster_agbno = agbno; +Seems like this should be a separate patch?
Yes, probably should be.
Quoted 8 lines
db/io.c pop_cur(void) · line 110
Quoted 5 lines
db/io.c +3
-1-if (iocur_top->bp) +if (iocur_top->bp) {libxfs_putbuf (iocur_top->bp); + iocur_top->bp =NULL ; + }This should probably be folded into the patch that started using buffers in
xfs_db.
Will do.
Quoted 6 lines
include/libxfs.h typedef struct { · line 114
Quoted 3 lines
include/libxfs.h +2
-0+int icache_flags;/* cache init flags */ +int bcache_flags;/* cache init flags */ The
icache_flagsaren't used anywhere. Also I'd really prefer to have my patch to kill the inode cache istead of adding more changes to it.
Sure - should I prepend your series to kill the icache to this one?
Quoted 6 lines
Quoted 6 lines
libxfs/rdwr.c +6
-0+#ifdef IO_BCOMPARE_CHECK +if (!(libxfs_bcache->c_flags &CACHE_MISCOMPARE_PURGE )) { + fprintf(stderr, + "%lx: Badness in key lookup (length)\n" + "bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n", + pthread_self(),
Quoted 8 lines
libxfs/rdwr.c libxfs_bcompare(struct cache_node *node, cache_key_t key) · line 340
Quoted 5 lines
libxfs/rdwr.c +4
-0+ (unsigned long long )bp->b_bn, (int )bp->b_bcount, + (unsigned long long )bkey->blkno, + BBTOB(bkey->bblen)); + }#endif What is the point of the
IO_BCOMPARE_CHECKifdef if we unconditionally define it?
I'd like to - eventually - turn it off. In fact, I'd like to end up with a real debug xfsprogs build like we do for the kernel code so that the version that distros ship don't have this noise enabled but developers can easily enable all the different debug/tracing stuff to be able to find problems in the code....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 26/30] xfs_db: avoid libxfs buffer lookup warnings
Quoted 4 lines · Dave Chinner
Quoted 2 lines
The
icache_flagsaren't used anywhere. Also I'd really prefer to have my patch to kill the inode cache istead of adding more changes to it.Sure - should I prepend your series to kill the icache to this one?
It's just a single patch. Just include it in your series.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 27/30 libxfs: work around do_div() not handling 32 bit numerators
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The libxfs dquot buffer code uses do_div() with a 32 bit numerator. This gives incorrect results as do_div() passes the numerator by reference as a pointer to a 64 bit value. Hence it does the division using 32 bits of garbage gives the wrong result. The kernel code handles 32 bit numerators just fine, so this patch is a temporary workaround in the dquot buffer code until we fix do_div() to handle 32 bit numerators correctly.
--- libxfs/xfs_dquot_buf.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
libxfs/xfs_dquot_buf.c +12 -2
@@ -18,18 +18,28 @@*/ #include "xfs.h" +/* +* XXX: the userspace implementation of the do_div() macro does not handle 32 +* bit numerators properly as it passes it by reference as a pointer to a 64 bit +* variable and dereferences it as such. Hence the result is way, way off +* because it uses 32 bits of garbage for the upper 32 bits of the numerator. +* +* This is being left here as a reminder that we need to fix do_div() in +* userspace as every time we do a libxfs kernel/userspace diff we'll see this +* comment. +*/ int xfs_calc_dquots_per_chunk (struct xfs_mount *mp,unsigned int nbblks)/* basic block units */ { -unsigned int ndquots; +uint64_t ndquots;ASSERT (nbblks >0 ); ndquots =BBTOB (nbblks);do_div (ndquots,sizeof (xfs_dqblk_t)); -return ndquots; +return (int )ndquots; }/*
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 27/30] libxfs: work around do_div() not handling 32 bit numerators
Quoted 9 lines · Dave Chinner
From: Dave Chinner <[email protected]>
The libxfs dquot buffer code uses
do_div()with a 32 bit numerator. This gives incorrect results asdo_div()passes the numerator by reference as a pointer to a 64 bit value. Hence it does the division using 32 bits of garbage gives the wrong result. The kernel code handles 32 bit numerators just fine, so this patch is a temporary workaround in the dquot buffer code until we fixdo_div()to handle 32 bit numerators correctly.
The right fix is to simply stop using do_div here, both in kernel and userspace.
Quoted 4 lines
xfs_calc_dquots_per_chunk(
libxfs/xfs_dquot_buf.c +0
-0struct xfs_mount *mp, unsigned int nbblks) /* basic block units */
libxfs/xfs_dquot_buf.c line 42
Quoted 10 lines
libxfs/xfs_dquot_buf.c +2
-2{ -unsigned int ndquots; +uint64_t ndquots;ASSERT (nbblks >0 ); ndquots =BBTOB (nbblks);do_div (ndquots,sizeof (xfs_dqblk_t)); -return ndquots; +return (int )ndquots;
ndquots = BBTOB(nbblks) / sizeof(xfs_dqblk_t);
Also the current xfs_calc_dquots_per_chunk doesn't use the mp argument anyway, but all callers do. I think in the end we'd want to take that in from the calers end end up with a version that includes the check from the quotainfo. But for getting userspace up and running my above minimal version should do it.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 27/30] libxfs: work around do_div() not handling 32 bit numerators
Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Dave Chinner
From: Dave Chinner <[email protected]>
The libxfs dquot buffer code uses
do_div()with a 32 bit numerator. This gives incorrect results asdo_div()passes the numerator by reference as a pointer to a 64 bit value. Hence it does the division using 32 bits of garbage gives the wrong result. The kernel code handles 32 bit numerators just fine, so this patch is a temporary workaround in the dquot buffer code until we fixdo_div()to handle 32 bit numerators correctly.
Quoted 2 lines
The right fix is to simply stop using
do_divhere, both in kernel and userspace.
Oh, right, I was thinking that BBTOB() resulted in a u64 value like BTOBB() and BTOBBT() do. It doesn't have a built in cast, so takes whatever type the source variable has.
Quoted 7 lines
Quoted 7 lines
xfs_calc_dquots_per_chunk(
libxfs/xfs_dquot_buf.c +1
-1struct xfs_mount *mp, unsigned int nbblks) /* basic block units */ { -unsigned int ndquots; +uint64_t ndquots;
Quoted 8 lines
libxfs/xfs_dquot_buf.c line 42
Quoted 6 lines
libxfs/xfs_dquot_buf.c +1
-1ASSERT (nbblks >0 ); ndquots =BBTOB (nbblks);do_div (ndquots,sizeof (xfs_dqblk_t)); -return ndquots; +return (int )ndquots;ndquots = BBTOB(nbblks) / sizeof(xfs_dqblk_t);
Yup, that should work.
Quoted 5 lines
Also the current
xfs_calc_dquots_per_chunkdoesn't use the mp argument anyway, but all callers do. I think in the end we'd want to take that in from the calers end end up with a version that includes the check from the quotainfo. But for getting userspace up and running my above minimal version should do it.
Yes, changed to fix. I'll drive further cleanups from the kernel side here...
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 28/30 db: enable metadump on CRC filesystems
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Now that we can calculate CRCs through xfs_db, we can add support for recalculating CRCs on obfuscated metadump images. This simply requires us to call the write verifier manually before writing the buffer to the metadump image.
We don't need to do anything special to mdrestore, as the metadata blocks it reads from the image file should already have all the correct CRCs in them. hence it can be mostly oblivious to the fact that the filesystem it is resotring contains CRCs.
While there, add IO stack unwinding to xfs_db to prevent metadump and other db programs from exiting with unreleased buffers in the IO stack and hence emitting warnings like:
cache_purge: shake on cache 0x69e4f0 left 7 nodes!?
--- db/init.c | 5 ++++- db/metadump.c | 24 +++++++++++++++++++----- mdrestore/xfs_mdrestore.c | 3 --- 3 files changed, 23 insertions(+), 9 deletions(-)
db/init.c +4 -1
@@ -173,9 +173,11 @@ main(int c, i, done =0 ;char *input;char **v; +int start_iocur_sp;pushfile (stdin);init (argc, argv); + start_iocur_sp = iocur_sp;for (i =0 ; !done && i < ncmdline; i++) { v =breakline (cmdline[i], &c);
@@ -202,7 +204,8 @@ close_devices:* make sure that we pop the last buffer context we held so that the * buffer is released before purge the caches during unmount. */ -pop_cur (); +while (iocur_sp > start_iocur_sp) +pop_cur ();libxfs_umount (mp);if (x.ddev)libxfs_device_close (x.ddev);
db/metadump.c +19 -5
@@ -172,6 +172,22 @@ write_buf(__int64_t off;int i; +/* +* Run the write verifier to recalculate the buffer CRCs and check +* we are writing something valid to disk +*/ +if (buf->bp && buf->bp->b_ops) { + buf->bp->b_error =0 ; + buf->bp->b_ops->verify_write (buf->bp); +if (buf->bp->b_error) { +fprintf (stderr, +_ ("%s: write verifer failed on bno 0x%llx/0x%x\n" ), + __func__, (long long )buf->bp->b_bn, + buf->bp->b_bcount); +return buf->bp->b_error; + } + } +for (i =0 , off = buf->bb, data = buf->data; i < buf->blen; i++, off++, data +=BBSIZE ) {
@@ -1727,6 +1743,9 @@ copy_inode_chunk(if (!process_inode (agno, agino + i, dip)) goto pop_out; + +/* calculate the new CRC for the inode */ +xfs_dinode_calc_crc (mp, dip); } skip_processing:if (!write_buf (iocur_top))
@@ -2053,11 +2072,6 @@ metadump_f(return 0 ; } -if (xfs_sb_version_hascrc (&mp->m_sb) && dont_obfuscate ==0 ) { -print_warning ("Can't obfuscate CRC enabled filesystems yet." ); -return 0 ; - } - metablock = (xfs_metablock_t *)calloc (BBSIZE +1 ,BBSIZE );if (metablock ==NULL ) {print_warning ("memory allocation failure" );
mdrestore/xfs_mdrestore.c +0 -3
@@ -109,9 +109,6 @@ perform_restore(if (sb.sb_magicnum !=XFS_SB_MAGIC )fatal ("bad magic number for primary superblock\n" ); -if (xfs_sb_version_hascrc (&sb)) -fatal ("Can't restore CRC enabled filesystems yet.\n" ); - ((xfs_dsb_t *)block_buffer)->sb_inprogress =1 ;if (is_target_file) {
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 28/30] db: enable metadump on CRC filesystems
Quoted 5 lines · Dave Chinner
While there, add IO stack unwinding to
xfs_dbto prevent metadump and other db programs from exiting with unreleased buffers in the IO stack and hence emitting warnings like:
cache_purge: shake on cache0x69e4f0left 7 nodes!?
Seems like that should be a separate patch.
db/init.c close_devices: · line 208
Quoted 7 lines
db/init.c +2
-1@@ -202,7 +204,8 @@ close_devices:* make sure that we pop the last buffer context we held so that the * buffer is released before purge the caches during unmount. */ -pop_cur (); +while (iocur_sp > start_iocur_sp) +pop_cur ();
The comment above this needs an update.
Otherwise looks fine.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 29/30 xfs: support larger inode clusters on v5 filesystems
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
To allow the kernel to use larger inode clusters than the standard 8192 bytes, we need to set the inode alignment fields appropriately so that the kernel is consistent in it's inode to buffer mappings. We set the alignment to allow a constant 32 inodes per cluster, instead of a fixed 8k cluster size.
--- include/libxfs.h | 2 +- mkfs/xfs_mkfs.c | 5 ++++- repair/sb.c | 41 ++++++++++++++++++++++++++++------------- 3 files changed, 33 insertions(+), 15 deletions(-)
include/libxfs.h +1 -1
@@ -182,7 +182,7 @@ typedef struct xfs_mount {__uint8_t m_sectbb_log;/* sectorlog - BBSHIFT */ __uint8_t m_agno_log;/* log #ag's */ __uint8_t m_agino_log;/* #bits for agino in inum */ -__uint16_t m_inode_cluster_size;/* min inode buf size */ +uint m_inode_cluster_size;/* min inode buf size */ uint m_blockmask;/* sb_blocksize-1 */ uint m_blockwsize;/* sb_blocksize in words */ uint m_blockwmask;/* blockwsize-1 */
mkfs/xfs_mkfs.c +4 -1
@@ -2532,7 +2532,10 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"), }else sbp->sb_logsunit =0 ;if (iaflag) { - sbp->sb_inoalignmt =XFS_INODE_BIG_CLUSTER_SIZE >> blocklog; +int cluster_size =XFS_INODE_BIG_CLUSTER_SIZE ; +if (crcs_enabled) + cluster_size *= isize /XFS_DINODE_MIN_SIZE ; + sbp->sb_inoalignmt = cluster_size >> blocklog; iaflag = sbp->sb_inoalignmt !=0 ; }else sbp->sb_inoalignmt =0 ;
repair/sb.c +28 -13
64 lines not on this page · raw message
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 29/30] xfs: support larger inode clusters on v5 filesystems
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v3 30/30 xfsprogs: kill experimental warnings for v5 filesystems
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
With xfsprogs now being close to feature complete on v5 filesystems, remove the experimental warnings from the superblock verifier. This means that we don't need to filter such warnings from the output in xfstests and so we can see exactly what tests are failing due to code deficiencies rather than from detecting warning noise.
--- libxfs/xfs_sb.c | 4 ---- 1 file changed, 4 deletions(-)
libxfs/xfs_sb.c +0 -4
@@ -180,10 +180,6 @@ xfs_mount_validate_sb(* write validation, we don't need to check feature masks. */ if (check_version &&XFS_SB_VERSION_NUM (sbp) ==XFS_SB_VERSION_5 ) { -xfs_alert (mp, -"Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled!\n" -"Use of these features is at your own risk!" ); -if (xfs_sb_has_compat_feature (sbp,XFS_SB_FEAT_COMPAT_UNKNOWN )) { xfs_warn(mp,
Signature · 6 lines
1.8.4.rc3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 30/30] xfsprogs: kill experimental warnings for v5 filesystems
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
With xfsprogs now being close to feature complete on v5 filesystems, remove the experimental warnings from the superblock verifier. This means that we don't need to filter such warnings from the output in xfstests and so we can see exactly what tests are failing due to code deficiencies rather than from detecting warning noise.
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs