Skip to contentSkip to conversation

xfsprogs: CRC write support for xfs_db + more

PATCH Superseded Dave Chinner 3 participants 58 messages Nov 6, 2013 – Nov 13, 2013

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover xfsprogs: CRC write support for xfs_db + more

Hi folks,

Many thanks to Christoph for reviewing V3 of this patchset, which was posted here:

V3: http://oss.sgi.com/archives/xfs/2013-10/msg00914.html

The series was described in the V2 posting here:

V2: http://oss.sgi.com/archives/xfs/2013-09/msg00805.html

Changes made since V3:
- moved LIBXFS_MOUNT_ROOTINOS removal patch to start of xfs_db series
- performance tested and reviewed Christoph's icache removal patch and added it before start of xfs_db series
- split several of the patches up into independent changes
- updated the commit message for the first step of the xfs_db libxfs conversion
- updated comments as requested by Christoph

Unrelated new patches at end of series:
- xfs_repair speedups that I wrote to speed up large filesystem testing. These reduce the default configuration repair time for a 100TB filesystem with 50M inodes in it on my test rig by 70%.
- xfs_repair bug fix found while checking large directories.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw

PATCH v4 1/37 xfsprogs: fix automatic dependency generation

From: Dave Chinner <[email protected]>

Adding are removing a header file does not result in dependency regeneration like it should. make clean will rebuild the dependencies, but a normal make won't. Fix it.

---
 include/buildrules | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)
include/buildrules +16 -4
index 49cb2a4..edb1beb 100644
@@ -79,18 +79,30 @@ endif # _BUILDRULES_INCLUDED_
 $(_FORCE):
 
 # dependency build is automatic, relies on gcc -MM to generate.
+#
+# This is a bit messy. It regenerates the depenencies on each build so
+# that we catch files being added and removed. There are other ways of doing
+# this (e.g. per-file dependency files) but that requires more in-depth changes
+# to the build system. Compile time is not an issue for us, so the
+# rebuild on every make invocation isn't a problem we need to care about. Just
+# do it silently so it doesn't make the build unnecessarily noisy.
+
 .PHONY : depend ltdepend install-qa
 
 MAKEDEP := $(MAKEDEPEND) $(CFLAGS)
 
-ltdepend: .ltdep
+ltdepend: rmltdep .ltdep
+
+rmltdep:
+	@rm -f .ltdep
 
 .ltdep: $(CFILES) $(HFILES)
-	@echo "    [LTDEP]"
 	$(Q)$(MAKEDEP) $(CFILES) | $(SED) -e 's,^\([^:]*\)\.o,\1.lo,' > .ltdep
 
-depend: .dep
+depend: rmdep .dep
+
+rmdep:
+	@rm -f .dep
 
 .dep: $(CFILES) $(HFILES)
-	@echo "    [DEP]"
 	$(Q)$(MAKEDEP) $(CFILES) > .dep
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#2 lore raw

PATCH v4 2/37 xfs: fix some minor sparse warnings

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

A couple of simple locking annotations and 0 vs NULL warnings. Nothing that changes any code behaviour, just removes build noise.

---
 libxfs/xfs_bmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libxfs/xfs_bmap.c +1 -1
index 2d480cc..7336abf 100644
@@ -4415,7 +4415,7 @@ xfs_bmapi_write(
 {
 	struct xfs_mount	*mp = ip->i_mount;
 	struct xfs_ifork	*ifp;
-	struct xfs_bmalloca	bma = { 0 };	/* args for xfs_bmap_alloc */
+	struct xfs_bmalloca	bma = { NULL };	/* args for xfs_bmap_alloc */
 	xfs_fileoff_t		end;		/* end of mapped file region */
 	int			eof;		/* after the end of extents */
 	int			error;		/* error return */
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#3 lore raw

PATCH v4 3/37 xfs: create a shared header file for format-related information

In reply to Dave Chinner

All of the buffer operations structures are needed to be exported for xfs_db, so move them all to a common location rather than spreading them all over the place. They are verifying the on-disk format, so while xfs_format.h might be a good place, it is not part of the on disk format.

Hence we need to create a new header file that we centralise these related definitions. Start by moving the bffer operations structures, and then also move all the other definitions that have crept into xfs_log_format.h and xfs_format.h as there was no other shared header file to put them in.

---
 include/Makefile           |   1 +
 include/libxfs.h           |   1 +
 include/xfs_ag.h           |   4 -
 include/xfs_alloc.h        |   3 -
 include/xfs_alloc_btree.h  |   2 -
 include/xfs_attr_leaf.h    |   2 -
 include/xfs_bmap_btree.h   |   2 -
 include/xfs_da_btree.h     |   2 -
 include/xfs_format.h       |  10 --
 include/xfs_ialloc.h       |   2 -
 include/xfs_ialloc_btree.h |   2 -
 include/xfs_inode_buf.h    |   2 -
 include/xfs_log_format.h   | 177 --------------------------------
 include/xfs_sb.h           |   3 -
 include/xfs_shared.h       | 244 +++++++++++++++++++++++++++++++++++++++++++++
 15 files changed, 246 insertions(+), 211 deletions(-)
 create mode 100644 include/xfs_shared.h
include/Makefile +1 -0
index dc6a8bb..6682b9d 100644
@@ -40,6 +40,7 @@ QAHFILES = libxfs.h libxlog.h \
 	xfs_metadump.h \
 	xfs_quota_defs.h \
 	xfs_sb.h \
+	xfs_shared.h \
 	xfs_trace.h \
 	xfs_trans_resv.h \
 	xfs_trans_space.h
include/libxfs.h +1 -0
index b837072..835ba37 100644
@@ -37,6 +37,7 @@
 #include <xfs/xfs_fs.h>
 #include <xfs/xfs_arch.h>
 
+#include <xfs/xfs_shared.h>
 #include <xfs/xfs_format.h>
 #include <xfs/xfs_log_format.h>
 #include <xfs/xfs_quota_defs.h>
include/xfs_ag.h +0 -4
index 1cb740a..3fc1098 100644
@@ -128,8 +128,6 @@ typedef struct xfs_agf {
 extern int xfs_read_agf(struct xfs_mount *mp, struct xfs_trans *tp,
 			xfs_agnumber_t agno, int flags, struct xfs_buf **bpp);
 
-extern const struct xfs_buf_ops xfs_agf_buf_ops;
-
 /*
  * Size of the unlinked inode hash table in the agi.
  */
@@ -191,8 +189,6 @@ typedef struct xfs_agi {
 extern int xfs_read_agi(struct xfs_mount *mp, struct xfs_trans *tp,
 				xfs_agnumber_t agno, struct xfs_buf **bpp);
 
-extern const struct xfs_buf_ops xfs_agi_buf_ops;
-
 /*
  * The third a.g. block contains the a.g. freelist, an array
  * of block pointers to blocks owned by the allocation btree code.
include/xfs_alloc.h +0 -3
index 99d0a61..feacb06 100644
@@ -231,7 +231,4 @@ xfs_alloc_get_rec(
 	xfs_extlen_t		*len,	/* output: length of extent */
 	int			*stat);	/* output: success/failure */
 
-extern const struct xfs_buf_ops xfs_agf_buf_ops;
-extern const struct xfs_buf_ops xfs_agfl_buf_ops;
-
 #endif	/* __XFS_ALLOC_H__ */
include/xfs_alloc_btree.h +0 -2
index e3a3f74..72676c3 100644
@@ -95,6 +95,4 @@ extern struct xfs_btree_cur *xfs_allocbt_init_cursor(struct xfs_mount *,
 		xfs_agnumber_t, xfs_btnum_t);
 extern int xfs_allocbt_maxrecs(struct xfs_mount *, int, int);
 
-extern const struct xfs_buf_ops xfs_allocbt_buf_ops;
-
 #endif	/* __XFS_ALLOC_BTREE_H__ */
include/xfs_attr_leaf.h +0 -2
index d9b148f..3ec5ec0 100644
@@ -106,6 +106,4 @@ void	xfs_attr3_leaf_hdr_from_disk(struct xfs_attr3_icleaf_hdr *to,
 void	xfs_attr3_leaf_hdr_to_disk(struct xfs_attr_leafblock *to,
 				   struct xfs_attr3_icleaf_hdr *from);
 
-extern const struct xfs_buf_ops xfs_attr3_leaf_buf_ops;
-
 #endif	/* __XFS_ATTR_LEAF_H__ */
include/xfs_bmap_btree.h +0 -2
index 1b726d6..e307978 100644
@@ -239,6 +239,4 @@ extern int xfs_bmbt_maxrecs(struct xfs_mount *, int blocklen, int leaf);
 extern struct xfs_btree_cur *xfs_bmbt_init_cursor(struct xfs_mount *,
 		struct xfs_trans *, struct xfs_inode *, int);
 
-extern const struct xfs_buf_ops xfs_bmbt_buf_ops;
-
 #endif	/* __XFS_BMAP_BTREE_H__ */
include/xfs_da_btree.h +0 -2
index 9323b0e..e492dca 100644
@@ -169,8 +169,6 @@ int	xfs_da3_node_read(struct xfs_trans *tp, struct xfs_inode *dp,
 			 xfs_dablk_t bno, xfs_daddr_t mappedbno,
 			 struct xfs_buf **bpp, int which_fork);
 
-extern const struct xfs_buf_ops xfs_da3_node_buf_ops;
-
 /*
  * Utility routines.
  */
include/xfs_format.h +0 -10
index 35c08ff..a790428 100644
@@ -156,14 +156,4 @@ struct xfs_dsymlink_hdr {
 	((bufsize) - (xfs_sb_version_hascrc(&(mp)->m_sb) ? \
 			sizeof(struct xfs_dsymlink_hdr) : 0))
 
-int xfs_symlink_blocks(struct xfs_mount *mp, int pathlen);
-int xfs_symlink_hdr_set(struct xfs_mount *mp, xfs_ino_t ino, uint32_t offset,
-			uint32_t size, struct xfs_buf *bp);
-bool xfs_symlink_hdr_ok(struct xfs_mount *mp, xfs_ino_t ino, uint32_t offset,
-			uint32_t size, struct xfs_buf *bp);
-void xfs_symlink_local_to_remote(struct xfs_trans *tp, struct xfs_buf *bp,
-				 struct xfs_inode *ip, struct xfs_ifork *ifp);
-
-extern const struct xfs_buf_ops xfs_symlink_buf_ops;
-
 #endif /* __XFS_FORMAT_H__ */
include/xfs_ialloc.h +0 -2
index 68c0732..1557798 100644
@@ -158,6 +158,4 @@ int xfs_ialloc_inode_init(struct xfs_mount *mp, struct xfs_trans *tp,
 			  xfs_agnumber_t agno, xfs_agblock_t agbno,
 			  xfs_agblock_t length, unsigned int gen);
 
-extern const struct xfs_buf_ops xfs_agi_buf_ops;
-
 #endif	/* __XFS_IALLOC_H__ */
include/xfs_ialloc_btree.h +0 -2
index 3ac36b7..cfbfe46 100644
@@ -110,6 +110,4 @@ extern struct xfs_btree_cur *xfs_inobt_init_cursor(struct xfs_mount *,
 		struct xfs_trans *, struct xfs_buf *, xfs_agnumber_t);
 extern int xfs_inobt_maxrecs(struct xfs_mount *, int, int);
 
-extern const struct xfs_buf_ops xfs_inobt_buf_ops;
-
 #endif	/* __XFS_IALLOC_BTREE_H__ */
include/xfs_inode_buf.h +0 -2
index aae9fc4..e8fd3bd 100644
@@ -47,6 +47,4 @@ void		xfs_inobp_check(struct xfs_mount *, struct xfs_buf *);
 #define	xfs_inobp_check(mp, bp)
 #endif /* DEBUG */
 
-extern const struct xfs_buf_ops xfs_inode_buf_ops;
-
 #endif	/* __XFS_INODE_BUF_H__ */
include/xfs_log_format.h +0 -177

193 lines not on this page · raw message

include/xfs_sb.h +0 -3
index 6835b44..35061d4 100644
@@ -699,7 +699,4 @@ extern void	xfs_sb_from_disk(struct xfs_sb *, struct xfs_dsb *);
 extern void	xfs_sb_to_disk(struct xfs_dsb *, struct xfs_sb *, __int64_t);
 extern void	xfs_sb_quota_from_disk(struct xfs_sb *sbp);
 
-extern const struct xfs_buf_ops xfs_sb_buf_ops;
-extern const struct xfs_buf_ops xfs_sb_quiet_buf_ops;
-
 #endif	/* __XFS_SB_H__ */
include/xfs_shared.h new file +244 -0

244 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#4 lore raw

PATCH v4 4/37 xfs: split dquot buffer operations out

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Parts of userspace want to be able to read and modify dquot buffers (e.g. xfs_db) so we need to split out the reading and writing of these buffers so it is easy to shared code with libxfs in userspace.

---
 include/libxfs.h       |   9 ++
 libxfs/Makefile        |   1 +
 libxfs/xfs_dquot_buf.c | 273 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 283 insertions(+)
 create mode 100644 libxfs/xfs_dquot_buf.c
include/libxfs.h +9 -0
index 835ba37..f10ab59 100644
@@ -216,6 +216,15 @@ typedef struct xfs_mount {
 	xfs_dablk_t		m_dirdatablk;	/* blockno of dir data v2 */
 	xfs_dablk_t		m_dirleafblk;	/* blockno of dir non-data v2 */
 	xfs_dablk_t		m_dirfreeblk;	/* blockno of dirfreeindex v2 */
+
+	/*
+	 * anonymous struct to allow xfs_dquot_buf.c to compile.
+	 * Pointer is always null in userspace, so code does not use it at all
+	 */
+	struct {
+		int	qi_dqperchunk;
+	}			*m_quotainfo;
+
 } xfs_mount_t;
 
 /*
libxfs/Makefile +1 -0
index f0cbae3..4522218 100644
@@ -29,6 +29,7 @@ CFILES = cache.c \
 	xfs_dir2_leaf.c \
 	xfs_dir2_node.c \
 	xfs_dir2_sf.c \
+	xfs_dquot_buf.c \
 	xfs_ialloc.c \
 	xfs_inode_buf.c \
 	xfs_inode_fork.c \
libxfs/xfs_dquot_buf.c new file +273 -0

273 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#5 lore raw

PATCH v4 5/37 xfs: decouple inode and bmap btree header files

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Currently the xfs_inode.h header has a dependency on the definition of the BMAP btree records as the inode fork includes an array of xfs_bmbt_rec_host_t objects in it's definition.

Move all the btree format definitions from xfs_btree.h, xfs_bmap_btree.h, xfs_alloc_btree.h and xfs_ialloc_btree.h to xfs_format.h to continue the process of centralising the on-disk format definitions. With this done, the xfs inode definitions are no longer dependent on btree header files.

The enables a massive culling of unnecessary includes, with close to 200 #include directives removed from the XFS kernel code base.

---
 include/xfs_alloc_btree.h  |  33 ------
 include/xfs_bmap_btree.h   | 103 -----------------
 include/xfs_btree.h        |  80 -------------
 include/xfs_format.h       | 281 +++++++++++++++++++++++++++++++++++++++++++++
 include/xfs_ialloc.h       |   3 +-
 include/xfs_ialloc_btree.h |  49 --------
 include/xfs_inode_fork.h   |   1 +
 7 files changed, 284 insertions(+), 266 deletions(-)
include/xfs_alloc_btree.h +0 -33
index 72676c3..45e189e 100644
@@ -27,39 +27,6 @@ struct xfs_btree_cur;
 struct xfs_mount;
 
 /*
- * There are two on-disk btrees, one sorted by blockno and one sorted
- * by blockcount and blockno.  All blocks look the same to make the code
- * simpler; if we have time later, we'll make the optimizations.
- */
-#define	XFS_ABTB_MAGIC		0x41425442	/* 'ABTB' for bno tree */
-#define	XFS_ABTB_CRC_MAGIC	0x41423342	/* 'AB3B' */
-#define	XFS_ABTC_MAGIC		0x41425443	/* 'ABTC' for cnt tree */
-#define	XFS_ABTC_CRC_MAGIC	0x41423343	/* 'AB3C' */
-
-/*
- * Data record/key structure
- */
-typedef struct xfs_alloc_rec {
-	__be32		ar_startblock;	/* starting block number */
-	__be32		ar_blockcount;	/* count of free blocks */
-} xfs_alloc_rec_t, xfs_alloc_key_t;
-
-typedef struct xfs_alloc_rec_incore {
-	xfs_agblock_t	ar_startblock;	/* starting block number */
-	xfs_extlen_t	ar_blockcount;	/* count of free blocks */
-} xfs_alloc_rec_incore_t;
-
-/* btree pointer type */
-typedef __be32 xfs_alloc_ptr_t;
-
-/*
- * Block numbers in the AG:
- * SB is sector 0, AGF is sector 1, AGI is sector 2, AGFL is sector 3.
- */
-#define	XFS_BNO_BLOCK(mp)	((xfs_agblock_t)(XFS_AGFL_BLOCK(mp) + 1))
-#define	XFS_CNT_BLOCK(mp)	((xfs_agblock_t)(XFS_BNO_BLOCK(mp) + 1))
-
-/*
  * Btree block header size depends on a superblock flag.
  */
 #define XFS_ALLOC_BLOCK_LEN(mp) \
include/xfs_bmap_btree.h +0 -103

121 lines not on this page · raw message

include/xfs_btree.h +0 -80

86 lines not on this page · raw message

include/xfs_format.h +281 -0

285 lines not on this page · raw message

include/xfs_ialloc.h +2 -1
index 1557798..a8f76a5 100644
@@ -23,6 +23,7 @@ struct xfs_dinode;
 struct xfs_imap;
 struct xfs_mount;
 struct xfs_trans;
+struct xfs_btree_cur;
 
 /*
  * Allocation parameters for inode allocation.
@@ -42,7 +43,7 @@ struct xfs_trans;
 static inline struct xfs_dinode *
 xfs_make_iptr(struct xfs_mount *mp, struct xfs_buf *b, int o)
 {
-	return (xfs_dinode_t *)
+	return (struct xfs_dinode *)
 		(xfs_buf_offset(b, o << (mp)->m_sb.sb_inodelog));
 }
 
include/xfs_ialloc_btree.h +0 -49

55 lines not on this page · raw message

include/xfs_inode_fork.h +1 -0
index 28661a0..eb329a1 100644
@@ -19,6 +19,7 @@
 #define	__XFS_INODE_FORK_H__
 
 struct xfs_inode_log_item;
+struct xfs_dinode;
 
 /*
  * The following xfs_ext_irec_t struct introduces a second (top) level
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#6 lore raw

PATCH v4 6/37 libxfs: unify xfs_btree.c with kernel code

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

The libxfs/xfs_btree.c code does not contain a small amount of code for btree block readahead that the kernel code does. Instead, it short circuits it at a higher layer and doesn't include the lower layer functions. There is no harm in calling the lower lay functions and have them do nothing, and doing so unifies the kernel and userspace code.

---
 libxfs/xfs.h       |  8 +++++---
 libxfs/xfs_btree.c | 48 +++++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 48 insertions(+), 8 deletions(-)
libxfs/xfs.h +5 -3
index 31acf1b..364fd83 100644
@@ -319,10 +319,12 @@ roundup_64(__uint64_t x, __uint32_t y)
 
 #define xfs_trans_buf_copy_type(dbp, sbp)
 
-#define xfs_buf_readahead(a,b,c,ops)		((void) 0)	/* no readahead */
+/* no readahead, need to avoid set-but-unused var warnings. */
+#define xfs_buf_readahead(a,d,c,ops)		({	\
+	xfs_daddr_t __d = d;				\
+	__d = __d; /* no set-but-unused warning */	\
+})
 #define xfs_buf_readahead_map(a,b,c,ops)	((void) 0)	/* no readahead */
-#define xfs_btree_reada_bufl(m,fsb,c,ops)	((void) 0)
-#define xfs_btree_reada_bufs(m,fsb,c,x,ops)	((void) 0)
 #define xfs_buftrace(x,y)			((void) 0)	/* debug only */
 
 #define xfs_cmn_err(tag,level,mp,fmt,args...)	cmn_err(level,fmt, ## args)
libxfs/xfs_btree.c +43 -5

84 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#7 lore raw

PATCH v4 7/37 libxfs: bmap btree owner swap support

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

For CRC enabled filesystems, we can't just swap inode forks from one inode to another when defragmenting a file - the blocks in the inode fork bmap btree contain pointers back to the owner inode. Hence if we are to swap the inode forks we have to atomically modify every block in the btree during the transaction.

This patch brings across the kernel code for doing the owner swap of an entire fork - something that we are likely to end up needing in xfs_repair when reparenting stray inodes to lost+found - without all the associated swap extents transaction and recovery cruft as those parts are not needed in userspace.

---
 include/xfs_bmap_btree.h |   4 ++
 include/xfs_btree.h      |  19 ++++--
 include/xfs_inode_buf.h  |  18 ++---
 include/xfs_log_format.h |   8 ++-
 libxfs/xfs_bmap_btree.c  |  44 ++++++++++++
 libxfs/xfs_btree.c       | 170 ++++++++++++++++++++++++++++++++++++++++++-----
 6 files changed, 227 insertions(+), 36 deletions(-)
include/xfs_bmap_btree.h +4 -0
index 2379d33..6e42e1e 100644
@@ -133,6 +133,10 @@ extern int xfs_bmbt_get_maxrecs(struct xfs_btree_cur *, int level);
 extern int xfs_bmdr_maxrecs(struct xfs_mount *, int blocklen, int leaf);
 extern int xfs_bmbt_maxrecs(struct xfs_mount *, int blocklen, int leaf);
 
+extern int xfs_bmbt_change_owner(struct xfs_trans *tp, struct xfs_inode *ip,
+				 int whichfork, xfs_ino_t new_owner,
+				 struct list_head *buffer_list);
+
 extern struct xfs_btree_cur *xfs_bmbt_init_cursor(struct xfs_mount *,
 		struct xfs_trans *, struct xfs_inode *, int);
 
include/xfs_btree.h +12 -7
index 227bfa5..6afe0b2 100644
@@ -41,15 +41,18 @@ extern kmem_zone_t	*xfs_btree_cur_zone;
 /*
  * For logging record fields.
  */
-#define	XFS_BB_MAGIC		0x01
-#define	XFS_BB_LEVEL		0x02
-#define	XFS_BB_NUMRECS		0x04
-#define	XFS_BB_LEFTSIB		0x08
-#define	XFS_BB_RIGHTSIB		0x10
-#define	XFS_BB_BLKNO		0x20
+#define	XFS_BB_MAGIC		(1 << 0)
+#define	XFS_BB_LEVEL		(1 << 1)
+#define	XFS_BB_NUMRECS		(1 << 2)
+#define	XFS_BB_LEFTSIB		(1 << 3)
+#define	XFS_BB_RIGHTSIB		(1 << 4)
+#define	XFS_BB_BLKNO		(1 << 5)
+#define	XFS_BB_LSN		(1 << 6)
+#define	XFS_BB_UUID		(1 << 7)
+#define	XFS_BB_OWNER		(1 << 8)
 #define	XFS_BB_NUM_BITS		5
 #define	XFS_BB_ALL_BITS		((1 << XFS_BB_NUM_BITS) - 1)
-#define	XFS_BB_NUM_BITS_CRC	8
+#define	XFS_BB_NUM_BITS_CRC	9
 #define	XFS_BB_ALL_BITS_CRC	((1 << XFS_BB_NUM_BITS_CRC) - 1)
 
 /*
@@ -381,6 +384,8 @@ int xfs_btree_new_iroot(struct xfs_btree_cur *, int *, int *);
 int xfs_btree_insert(struct xfs_btree_cur *, int *);
 int xfs_btree_delete(struct xfs_btree_cur *, int *);
 int xfs_btree_get_rec(struct xfs_btree_cur *, union xfs_btree_rec **, int *);
+int xfs_btree_change_owner(struct xfs_btree_cur *cur, __uint64_t new_owner,
+			   struct list_head *buffer_list);
 
 /*
  * btree block CRC helpers
include/xfs_inode_buf.h +9 -9
index e8fd3bd..9308c47 100644
@@ -32,17 +32,17 @@ struct xfs_imap {
 	ushort		im_boffset;	/* inode offset in block in bytes */
 };
 
-int		xfs_imap_to_bp(struct xfs_mount *, struct xfs_trans *,
-			       struct xfs_imap *, struct xfs_dinode **,
-			       struct xfs_buf **, uint, uint);
-int		xfs_iread(struct xfs_mount *, struct xfs_trans *,
-			  struct xfs_inode *, uint);
-void		xfs_dinode_calc_crc(struct xfs_mount *, struct xfs_dinode *);
-void		xfs_dinode_to_disk(struct xfs_dinode *,
-				   struct xfs_icdinode *);
+int	xfs_imap_to_bp(struct xfs_mount *, struct xfs_trans *,
+		       struct xfs_imap *, struct xfs_dinode **,
+		       struct xfs_buf **, uint, uint);
+int	xfs_iread(struct xfs_mount *, struct xfs_trans *,
+		  struct xfs_inode *, uint);
+void	xfs_dinode_calc_crc(struct xfs_mount *, struct xfs_dinode *);
+void	xfs_dinode_to_disk(struct xfs_dinode *to, struct xfs_icdinode *from);
+void	xfs_dinode_from_disk(struct xfs_icdinode *to, struct xfs_dinode *from);
 
 #if defined(DEBUG)
-void		xfs_inobp_check(struct xfs_mount *, struct xfs_buf *);
+void	xfs_inobp_check(struct xfs_mount *, struct xfs_buf *);
 #else
 #define	xfs_inobp_check(mp, bp)
 #endif /* DEBUG */
include/xfs_log_format.h +6 -2
index aeaa715..f0969c7 100644
@@ -302,6 +302,8 @@ typedef struct xfs_inode_log_format_64 {
 #define	XFS_ILOG_ADATA	0x040	/* log i_af.if_data */
 #define	XFS_ILOG_AEXT	0x080	/* log i_af.if_extents */
 #define	XFS_ILOG_ABROOT	0x100	/* log i_af.i_broot */
+#define XFS_ILOG_DOWNER	0x200	/* change the data fork owner on replay */
+#define XFS_ILOG_AOWNER	0x400	/* change the attr fork owner on replay */
 
 
 /*
@@ -315,7 +317,8 @@ typedef struct xfs_inode_log_format_64 {
 #define	XFS_ILOG_NONCORE	(XFS_ILOG_DDATA | XFS_ILOG_DEXT | \
 				 XFS_ILOG_DBROOT | XFS_ILOG_DEV | \
 				 XFS_ILOG_UUID | XFS_ILOG_ADATA | \
-				 XFS_ILOG_AEXT | XFS_ILOG_ABROOT)
+				 XFS_ILOG_AEXT | XFS_ILOG_ABROOT | \
+				 XFS_ILOG_DOWNER | XFS_ILOG_AOWNER)
 
 #define	XFS_ILOG_DFORK		(XFS_ILOG_DDATA | XFS_ILOG_DEXT | \
 				 XFS_ILOG_DBROOT)
@@ -327,7 +330,8 @@ typedef struct xfs_inode_log_format_64 {
 				 XFS_ILOG_DEXT | XFS_ILOG_DBROOT | \
 				 XFS_ILOG_DEV | XFS_ILOG_UUID | \
 				 XFS_ILOG_ADATA | XFS_ILOG_AEXT | \
-				 XFS_ILOG_ABROOT | XFS_ILOG_TIMESTAMP)
+				 XFS_ILOG_ABROOT | XFS_ILOG_TIMESTAMP | \
+				 XFS_ILOG_DOWNER | XFS_ILOG_AOWNER)
 
 static inline int xfs_ilog_fbroot(int w)
 {
libxfs/xfs_bmap_btree.c +44 -0

47 lines not on this page · raw message

libxfs/xfs_btree.c +152 -18

185 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#8 lore raw

PATCH v4 8/37 libxfs: xfs_rtalloc.c becomes xfs_rtbitmap.c

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

To match the split-up of the kernel xfs_rtalloc.c file, convert the libxfs version of xfs_rtalloc.c to match the newly shared kernel source file with all the realtime bitmap functions in it, xfs_rtbitmap.c.

---
 libxfs/Makefile       |   2 +-
 libxfs/xfs_rtalloc.c  | 776 ----------------------------------------
 libxfs/xfs_rtbitmap.c | 951 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 952 insertions(+), 777 deletions(-)
 delete mode 100644 libxfs/xfs_rtalloc.c
 create mode 100644 libxfs/xfs_rtbitmap.c
libxfs/Makefile +1 -1
index 4522218..ae15a5d 100644
@@ -35,7 +35,7 @@ CFILES = cache.c \
 	xfs_inode_fork.c \
 	xfs_ialloc_btree.c \
 	xfs_log_rlimit.c \
-	xfs_rtalloc.c \
+	xfs_rtbitmap.c \
 	xfs_sb.c \
 	xfs_symlink_remote.c \
 	xfs_trans_resv.c
libxfs/xfs_rtalloc.c deleted +0 -776

776 lines not on this page · raw message

libxfs/xfs_rtbitmap.c new file +951 -0

951 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#9 lore raw

PATCH v4 9/37 libxfs: bring across inode buffer readahead verifier changes

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

These were made for log recovery readahead in the kernel, so are not directly used in userspace. Hence bringing the change across is simply to keep files in sync.

---
 libxfs/xfs_inode_buf.c | 41 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 36 insertions(+), 5 deletions(-)
libxfs/xfs_inode_buf.c +36 -5
index b096f77..67d5eb4 100644
@@ -46,9 +46,22 @@ xfs_inobp_check(
 }
 #endif
 
+/*
+ * If we are doing readahead on an inode buffer, we might be in log recovery
+ * reading an inode allocation buffer that hasn't yet been replayed, and hence
+ * has not had the inode cores stamped into it. Hence for readahead, the buffer
+ * may be potentially invalid.
+ *
+ * If the readahead buffer is invalid, we don't want to mark it with an error,
+ * but we do want to clear the DONE status of the buffer so that a followup read
+ * will re-read it from disk. This will ensure that we don't get an unnecessary
+ * warnings during log recovery and we don't get unnecssary panics on debug
+ * kernels.
+ */
 static void
 xfs_inode_buf_verify(
-	struct xfs_buf	*bp)
+	struct xfs_buf	*bp,
+	bool		readahead)
 {
 	struct xfs_mount *mp = bp->b_target->bt_mount;
 	int		i;
@@ -69,6 +82,11 @@ xfs_inode_buf_verify(
 		if (unlikely(XFS_TEST_ERROR(!di_ok, mp,
 						XFS_ERRTAG_ITOBP_INOTOBP,
 						XFS_RANDOM_ITOBP_INOTOBP))) {
+			if (readahead) {
+				bp->b_flags &= ~XBF_DONE;
+				return;
+			}
+
 			xfs_buf_ioerror(bp, EFSCORRUPTED);
 			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_HIGH,
 					     mp, dip);
@@ -87,14 +105,21 @@ static void
 xfs_inode_buf_read_verify(
 	struct xfs_buf	*bp)
 {
-	xfs_inode_buf_verify(bp);
+	xfs_inode_buf_verify(bp, false);
+}
+
+static void
+xfs_inode_buf_readahead_verify(
+	struct xfs_buf	*bp)
+{
+	xfs_inode_buf_verify(bp, true);
 }
 
 static void
 xfs_inode_buf_write_verify(
 	struct xfs_buf	*bp)
 {
-	xfs_inode_buf_verify(bp);
+	xfs_inode_buf_verify(bp, false);
 }
 
 const struct xfs_buf_ops xfs_inode_buf_ops = {
@@ -102,6 +127,12 @@ const struct xfs_buf_ops xfs_inode_buf_ops = {
 	.verify_write = xfs_inode_buf_write_verify,
 };
 
+const struct xfs_buf_ops xfs_inode_buf_ra_ops = {
+	.verify_read = xfs_inode_buf_readahead_verify,
+	.verify_write = xfs_inode_buf_write_verify,
+};
+
+
 /*
  * This routine is called to map an inode to the buffer containing the on-disk
  * version of the inode.  It returns a pointer to the buffer containing the
@@ -191,7 +222,7 @@ xfs_dinode_from_disk(
 		to->di_ino = be64_to_cpu(from->di_ino);
 		to->di_lsn = be64_to_cpu(from->di_lsn);
 		memcpy(to->di_pad2, from->di_pad2, sizeof(to->di_pad2));
-		platform_uuid_copy(&to->di_uuid, &from->di_uuid);
+		uuid_copy(&to->di_uuid, &from->di_uuid);
 	}
 }
 
@@ -237,7 +268,7 @@ xfs_dinode_to_disk(
 		to->di_ino = cpu_to_be64(from->di_ino);
 		to->di_lsn = cpu_to_be64(from->di_lsn);
 		memcpy(to->di_pad2, from->di_pad2, sizeof(to->di_pad2));
-		platform_uuid_copy(&to->di_uuid, &from->di_uuid);
+		uuid_copy(&to->di_uuid, &from->di_uuid);
 		to->di_flushiter = 0;
 	} else {
 		to->di_flushiter = cpu_to_be16(from->di_flushiter);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#10 lore raw

PATCH v4 10/37 libxfs: Minor cleanup and bug fix sync

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

These bring all the small single line comment, whitespace and minor code differences into sync with the kernel code. Anything left at this point is an intentional difference.

---
 include/xfs_fs.h         | 4 ++--
 include/xfs_quota_defs.h | 4 ++++
 libxfs/xfs_attr_leaf.c   | 4 +---
 libxfs/xfs_bmap.c        | 6 +++---
 libxfs/xfs_bmap_btree.c  | 2 +-
 libxfs/xfs_dir2_leaf.c   | 1 -
 libxfs/xfs_dir2_node.c   | 2 +-
 libxfs/xfs_ialloc.c      | 4 ++--
 libxfs/xfs_inode_buf.c   | 2 ++
 libxfs/xfs_inode_fork.c  | 5 +++--
 libxfs/xfs_trans_resv.c  | 2 +-
 11 files changed, 20 insertions(+), 16 deletions(-)
include/xfs_fs.h +2 -2
index c43ba98..554fd66 100644
@@ -358,7 +358,7 @@ typedef struct xfs_error_injection {
  * Speculative preallocation trimming.
  */
 #define XFS_EOFBLOCKS_VERSION		1
-struct xfs_eofblocks {
+struct xfs_fs_eofblocks {
 	__u32		eof_version;
 	__u32		eof_flags;
 	uid_t		eof_uid;
@@ -516,7 +516,7 @@ typedef struct xfs_swapext
 /*	XFS_IOC_GETBIOSIZE ---- deprecated 47	   */
 #define XFS_IOC_GETBMAPX	_IOWR('X', 56, struct getbmap)
 #define XFS_IOC_ZERO_RANGE	_IOW ('X', 57, struct xfs_flock64)
-#define XFS_IOC_FREE_EOFBLOCKS	_IOR ('X', 58, struct xfs_eofblocks)
+#define XFS_IOC_FREE_EOFBLOCKS	_IOR ('X', 58, struct xfs_fs_eofblocks)
 
 /*
  * ioctl commands that replace IRIX syssgi()'s
include/xfs_quota_defs.h +4 -0
index e6b0d6e..b3b2b10 100644
@@ -154,4 +154,8 @@ typedef __uint16_t	xfs_qwarncnt_t;
 		(XFS_QMOPT_UQUOTA | XFS_QMOPT_PQUOTA | XFS_QMOPT_GQUOTA)
 #define XFS_QMOPT_RESBLK_MASK	(XFS_QMOPT_RES_REGBLKS | XFS_QMOPT_RES_RTBLKS)
 
+extern int xfs_dqcheck(struct xfs_mount *mp, xfs_disk_dquot_t *ddq,
+		       xfs_dqid_t id, uint type, uint flags, char *str);
+extern int xfs_calc_dquots_per_chunk(struct xfs_mount *mp, unsigned int nbblks);
+
 #endif	/* __XFS_QUOTA_H__ */
libxfs/xfs_attr_leaf.c +1 -3
index c09b0f3..fd52397 100644
@@ -599,7 +599,7 @@ xfs_attr_shortform_getvalue(xfs_da_args_t *args)
 	xfs_attr_sf_entry_t *sfe;
 	int i;
 
-	ASSERT(args->dp->i_d.di_aformat == XFS_IFINLINE);
+	ASSERT(args->dp->i_afp->if_flags == XFS_IFINLINE);
 	sf = (xfs_attr_shortform_t *)args->dp->i_afp->if_u1.if_data;
 	sfe = &sf->list[0];
 	for (i = 0; i < sf->hdr.count;
@@ -909,7 +909,6 @@ out:
 	return error;
 }
 
-
 /*========================================================================
  * Routines used for growing the Btree.
  *========================================================================*/
@@ -1270,7 +1269,6 @@ xfs_attr3_leaf_compact(
 	ichdr_dst->freemap[0].size = ichdr_dst->firstused -
 						ichdr_dst->freemap[0].base;
 
-
 	/* write the header back to initialise the underlying buffer */
 	xfs_attr3_leaf_hdr_to_disk(leaf_dst, ichdr_dst);
 
libxfs/xfs_bmap.c +3 -3
index 7336abf..3e80c64 100644
@@ -510,8 +510,8 @@ xfs_bmap_trace_exlist(
 
 /*
  * Validate that the bmbt_irecs being returned from bmapi are valid
- * given the callers original parameters.  Specifically check the
- * ranges of the returned irecs to ensure that they only extent beyond
+ * given the caller's original parameters.  Specifically check the
+ * ranges of the returned irecs to ensure that they only extend beyond
  * the given parameters if the XFS_BMAPI_ENTIRE flag was set.
  */
 STATIC void
@@ -1515,7 +1515,7 @@ xfs_bmap_first_unused(
 }
 
 /*
- * Returns the file-relative block number of the last block + 1 before
+ * Returns the file-relative block number of the last block - 1 before
  * last_block (input value) in the file.
  * This is not based on i_size, it is based on the extent records.
  * Returns 0 for local files, as they do not have extent records.
libxfs/xfs_bmap_btree.c +1 -1
index 2f6b48a..6211dc2 100644
@@ -737,7 +737,7 @@ xfs_bmbt_verify(
 	 * precise.
 	 */
 	level = be16_to_cpu(block->bb_level);
-	if (level > MAX(mp->m_bm_maxlevels[0], mp->m_bm_maxlevels[1]))
+	if (level > max(mp->m_bm_maxlevels[0], mp->m_bm_maxlevels[1]))
 		return false;
 	if (be16_to_cpu(block->bb_numrecs) > mp->m_bmap_dmxr[level != 0])
 		return false;
libxfs/xfs_dir2_leaf.c +0 -1
index c035c4d..683536e 100644
@@ -1072,7 +1072,6 @@ xfs_dir3_leaf_compact_x1(
 	*highstalep = highstale;
 }
 
-
 /*
  * Log the bests entries indicated from a leaf1 block.
  */
libxfs/xfs_dir2_node.c +1 -1
index 6a245e5..10d1d81 100644
@@ -1796,7 +1796,7 @@ xfs_dir2_node_addname_int(
 		/*
 		 * Look at the current free entry.  Is it good enough?
 		 *
-		 * The bests initialisation should be where the buffer is read in
+		 * The bests initialisation should be where the bufer is read in
 		 * the above branch. But gcc is too stupid to realise that bests
 		 * and the freehdr are actually initialised if they are placed
 		 * there, so we have to do it here to avoid warnings. Blech.
libxfs/xfs_ialloc.c +2 -2
index 4683287..afe1a82 100644
@@ -484,7 +484,7 @@ xfs_ialloc_next_ag(
 
 /*
  * Select an allocation group to look for a free inode in, based on the parent
- * inode and then mode.  Return the allocation group buffer.
+ * inode and the mode.  Return the allocation group buffer.
  */
 STATIC xfs_agnumber_t
 xfs_ialloc_ag_select(
@@ -706,7 +706,7 @@ xfs_dialloc_ag(
 		error = xfs_inobt_get_rec(cur, &rec, &j);
 		if (error)
 			goto error0;
-		XFS_WANT_CORRUPTED_GOTO(i == 1, error0);
+		XFS_WANT_CORRUPTED_GOTO(j == 1, error0);
 
 		if (rec.ir_freecount > 0) {
 			/*
libxfs/xfs_inode_buf.c +2 -0
index 67d5eb4..b796556 100644
@@ -101,6 +101,7 @@ xfs_inode_buf_verify(
 	xfs_inobp_check(mp, bp);
 }
 
+
 static void
 xfs_inode_buf_read_verify(
 	struct xfs_buf	*bp)
@@ -299,6 +300,7 @@ xfs_dinode_verify(
 		return false;
 	return true;
 }
+
 void
 xfs_dinode_calc_crc(
 	struct xfs_mount	*mp,
libxfs/xfs_inode_fork.c +3 -2
index 1c006f9..190690c 100644
@@ -138,7 +138,8 @@ xfs_iformat_fork(
 			}
 
 			di_size = be64_to_cpu(dip->di_size);
-			if (unlikely(di_size > XFS_DFORK_DSIZE(dip, ip->i_mount))) {
+			if (unlikely(di_size < 0 ||
+				     di_size > XFS_DFORK_DSIZE(dip, ip->i_mount))) {
 				xfs_warn(ip->i_mount,
 			"corrupt inode %Lu (bad size %Ld for local inode).",
 					(unsigned long long) ip->i_ino,
@@ -444,7 +445,7 @@ xfs_iread_extents(
  *
  * The caller must not request to add more records than would fit in
  * the on-disk inode root.  If the if_broot is currently NULL, then
- * if we adding records one will be allocated.  The caller must also
+ * if we are adding records, one will be allocated.  The caller must also
  * not request that the number of records go below zero, although
  * it can go to zero.
  *
libxfs/xfs_trans_resv.c +1 -1
index 3e14b1c..1e59fad 100644
@@ -49,7 +49,7 @@ xfs_calc_buf_res(
 
 /*
  * Logging inodes is really tricksy. They are logged in memory format,
- * which means that what we write into the log doesn't directory translate into
+ * which means that what we write into the log doesn't directly translate into
  * the amount of space they use on disk.
  *
  * Case in point - btree format forks in memory format use more space than the
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#11 lore raw

PATCH v4 11/37 xfs: remove newlines from strings passed to __xfs_printk

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

__xfs_printk adds its own "\n". Having it in the original string leads to unintentional blank lines from these messages.

Ported from kernel commit 08e96e1a.

---
 libxfs/xfs_bmap.c         | 2 +-
 libxfs/xfs_dir2_node.c    | 2 +-
 libxfs/xfs_sb.c           | 4 ++--
 libxlog/xfs_log_recover.c | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)
libxfs/xfs_bmap.c +1 -1
index 3e80c64..c45b91a 100644
@@ -1447,7 +1447,7 @@ xfs_bmap_search_extents(
 		xfs_alert_tag(ip->i_mount, XFS_PTAG_FSBLOCK_ZERO,
 				"Access to block zero in inode %llu "
 				"start_block: %llx start_off: %llx "
-				"blkcnt: %llx extent-state: %x lastx: %x\n",
+				"blkcnt: %llx extent-state: %x lastx: %x",
 			(unsigned long long)ip->i_ino,
 			(unsigned long long)gotp->br_startblock,
 			(unsigned long long)gotp->br_startoff,
libxfs/xfs_dir2_node.c +1 -1
index 10d1d81..ced8c58 100644
@@ -1083,7 +1083,7 @@ xfs_dir2_leafn_rebalance(
 		state->inleaf = 1;
 		blk2->index = 0;
 		xfs_alert(args->dp->i_mount,
-	"%s: picked the wrong leaf? reverting original leaf: blk1->index %d\n",
+	"%s: picked the wrong leaf? reverting original leaf: blk1->index %d",
 			__func__, blk1->index);
 	}
 }
libxfs/xfs_sb.c +2 -2
index 8b90b88..11353bb 100644
@@ -215,13 +215,13 @@ xfs_mount_validate_sb(
 	if (xfs_sb_version_has_pquotino(sbp)) {
 		if (sbp->sb_qflags & (XFS_OQUOTA_ENFD | XFS_OQUOTA_CHKD)) {
 			xfs_notice(mp,
-			   "Version 5 of Super block has XFS_OQUOTA bits.\n");
+			   "Version 5 of Super block has XFS_OQUOTA bits.");
 			return XFS_ERROR(EFSCORRUPTED);
 		}
 	} else if (sbp->sb_qflags & (XFS_PQUOTA_ENFD | XFS_GQUOTA_ENFD |
 				XFS_PQUOTA_CHKD | XFS_GQUOTA_CHKD)) {
 			xfs_notice(mp,
-"Superblock earlier than Version 5 has XFS_[PQ]UOTA_{ENFD|CHKD} bits.\n");
+"Superblock earlier than Version 5 has XFS_[PQ]UOTA_{ENFD|CHKD} bits.");
 			return XFS_ERROR(EFSCORRUPTED);
 	}
 
libxlog/xfs_log_recover.c +1 -1
index f3cda77..3f22921 100644
@@ -1330,7 +1330,7 @@ xlog_unpack_data_crc(
 	if (crc != rhead->h_crc) {
 		if (rhead->h_crc || xfs_sb_version_hascrc(&log->l_mp->m_sb)) {
 			xfs_alert(log->l_mp,
-		"log record CRC mismatch: found 0x%x, expected 0x%x.\n",
+		"log record CRC mismatch: found 0x%x, expected 0x%x.",
 					le32_to_cpu(rhead->h_crc),
 					le32_to_cpu(crc));
 			xfs_hex_dump(dp, 32);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#12 lore raw

PATCH v4 12/37 xfs: fix the wrong new_size/rnew_size at xfs_iext_realloc_direct()

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

At xfs_iext_realloc_direct(), the new_size is changed by adding if_bytes if originally the extent records are stored at the inline extent buffer, and we have to switch from it to a direct extent list for those new allocated extents, this is wrong.

This patch fix above problem and revise the new_size comments at xfs_iext_realloc_direct() to make it more readable. Also, fix the comments while switching from the inline extent buffer to a direct extent list to reflect this change.

Ported from kernel commit 17ec81c1.

---
 libxfs/xfs_inode_fork.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
libxfs/xfs_inode_fork.c +2 -7
index 190690c..dfa86ae 100644
@@ -1330,7 +1330,7 @@ xfs_iext_remove_indirect(
 void
 xfs_iext_realloc_direct(
 	xfs_ifork_t	*ifp,		/* inode fork pointer */
-	int		new_size)	/* new size of extents */
+	int		new_size)	/* new size of extents after adding */
 {
 	int		rnew_size;	/* real new size of extents */
 
@@ -1368,13 +1368,8 @@ xfs_iext_realloc_direct(
 				rnew_size - ifp->if_real_bytes);
 		}
 	}
-	/*
-	 * Switch from the inline extent buffer to a direct
-	 * extent list. Be sure to include the inline extent
-	 * bytes in new_size.
-	 */
+	/* Switch from the inline extent buffer to a direct extent list */
 	else {
-		new_size += ifp->if_bytes;
 		if (!is_power_of_2(new_size)) {
 			rnew_size = roundup_pow_of_two(new_size);
 		}
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#13 lore raw

PATCH v4 13/37 xfs: fix node forward in xfs_node_toosmall

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When a node is considered for a merge with a sibling, it overwrites the sibling pointers of the original incore nodehdr with the sibling's pointers. This leads to loop considering the original node as a merge candidate with itself in the second pass, and so it incorrectly determines a merge should occur.)

Ported from equivalent kernel commit 997def25.

---
 libxfs/xfs_da_btree.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libxfs/xfs_da_btree.c +3 -2
index f106e06..53414f5 100644
@@ -1201,6 +1201,7 @@ xfs_da3_node_toosmall(
 	/* start with smaller blk num */
 	forward = nodehdr.forw < nodehdr.back;
 	for (i = 0; i < 2; forward = !forward, i++) {
+		struct xfs_da3_icnode_hdr thdr;
 		if (forward)
 			blkno = nodehdr.forw;
 		else
@@ -1213,10 +1214,10 @@ xfs_da3_node_toosmall(
 			return(error);
 
 		node = bp->b_addr;
-		xfs_da3_node_hdr_from_disk(&nodehdr, node);
+		xfs_da3_node_hdr_from_disk(&thdr, node);
 		xfs_trans_brelse(state->args->trans, bp);
 
-		if (count - nodehdr.count >= 0)
+		if (count - thdr.count >= 0)
 			break;	/* fits with at least 25% to spare */
 	}
 	if (i >= 2) {
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#14 lore raw

PATCH v4 14/37 xfs: don't emit corruption noise on fs probes

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

If we get EWRONGFS due to probing of non-xfs filesystems, there's no need to issue the scary corruption error and backtrace.

Ported from kernel commit 31625f28.

---
 libxfs/xfs_sb.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
libxfs/xfs_sb.c +3 -2
index 11353bb..65ddc2f 100644
@@ -574,8 +574,9 @@ xfs_sb_read_verify(
 
 out_error:
 	if (error) {
-		XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW,
-				     mp, bp->b_addr);
+		if (error != EWRONGFS)
+			XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW,
+					     mp, bp->b_addr);
 		xfs_buf_ioerror(bp, error);
 	}
 }
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#15 lore raw

PATCH v4 15/37 libxfs: fix root inode handling inconsistencies

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When "mounting" a filesystem via libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem, libxfs_unmount() used to unconditionally free root inodes if they were present.

This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell libxfs_mount() to read the root inodes and so can result in unbalanced freeing of inodes when cleaning up during the unmount proceedure.

As it turns out, nothing ever uses mp->m_rootip and so we don't need to read it in or free it, or even have a pointer to it in the struct xfs_mount. Similarly, the only user of the realtime inodes is mkfs, and it initialises them itself. Hence we can kill the m_rootip and the realtime inode mounting code.

This leaves one user of LIBXFS_MOUNT_ROOTINOS - xfs_db - and that is only used to initialise the in-core superblock counter values from the ag header for xfs_check. Move this code to the xfs_db init functions so we can get rid of the mount parameter previously used to trigger all these behavours (LIBXFS_MOUNT_ROOTINOS) completely.

---
 copy/xfs_copy.c  |   2 +-
 db/init.c        |  26 +++++++----
 include/libxfs.h |  12 +++---
 libxfs/init.c    | 128 -------------------------------------------------------
 mkfs/proto.c     |   1 -
 mkfs/xfs_mkfs.c  |   4 +-
 repair/phase6.c  |   2 -
 7 files changed, 26 insertions(+), 149 deletions(-)
copy/xfs_copy.c +1 -1
index bb37279..9986fbf 100644
@@ -684,7 +684,7 @@ main(int argc, char **argv)
 	sb = &mbuf.m_sb;
 	libxfs_sb_from_disk(sb, XFS_BUF_TO_SBP(sbp));
 
-	mp = libxfs_mount(&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, 1);
+	mp = libxfs_mount(&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, 0);
 	if (mp == NULL) {
 		do_log(_("%s: %s filesystem failed to initialize\n"
 			"%s: Aborting.\n"), progname, source_name, progname);
db/init.c +18 -8
index 2932e51..54f3f9e 100644
@@ -149,18 +149,28 @@ init(
 	}
 
 	mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_ROOTINOS | LIBXFS_MOUNT_DEBUGGER);
+			  LIBXFS_MOUNT_DEBUGGER);
 	if (!mp) {
-		mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_DEBUGGER);
-		if (!mp) {
-			fprintf(stderr, _("%s: device %s unusable (not an XFS "
-				"filesystem?)\n"), progname, fsdevice);
-			exit(1);
-		}
+		fprintf(stderr,
+			_("%s: device %s unusable (not an XFS filesystem?)\n"),
+			progname, fsdevice);
+		exit(1);
 	}
 	blkbb = 1 << mp->m_blkbb_log;
 
+	/*
+	 * xfs_check needs corrected incore superblock values
+	 */
+	if (sbp->sb_rootino != NULLFSINO &&
+	    xfs_sb_version_haslazysbcount(&mp->m_sb)) {
+		int error = xfs_initialize_perag_data(mp, sbp->sb_agcount);
+		if (error) {
+			fprintf(stderr, _("%s: cannot init perag data (%d)\n"),
+				progname, error);
+			exit(EXIT_FAILURE);
+		}
+	}
+
 	if (xfs_sb_version_hascrc(&mp->m_sb))
 		type_set_tab_crc();
 
include/libxfs.h +5 -7
index f10ab59..3df8c07 100644
@@ -167,7 +167,6 @@ typedef struct xfs_mount {
 	uint			m_rsumsize;	/* size of rt summary, bytes */
 	struct xfs_inode	*m_rbmip;	/* pointer to bitmap inode */
 	struct xfs_inode	*m_rsumip;	/* pointer to summary inode */
-	struct xfs_inode	*m_rootip;	/* pointer to root directory */
 	struct xfs_buftarg	*m_ddev_targp;
 	struct xfs_buftarg	*m_logdev_targp;
 	struct xfs_buftarg	*m_rtdev_targp;
@@ -259,12 +258,11 @@ typedef struct xfs_perag {
 	int		pagb_count;	/* pagb slots in use */
 } xfs_perag_t;
 
-#define LIBXFS_MOUNT_ROOTINOS		0x0001
-#define LIBXFS_MOUNT_DEBUGGER		0x0002
-#define LIBXFS_MOUNT_32BITINODES	0x0004
-#define LIBXFS_MOUNT_32BITINOOPT	0x0008
-#define LIBXFS_MOUNT_COMPAT_ATTR	0x0010
-#define LIBXFS_MOUNT_ATTR2		0x0020
+#define LIBXFS_MOUNT_DEBUGGER		0x0001
+#define LIBXFS_MOUNT_32BITINODES	0x0002
+#define LIBXFS_MOUNT_32BITINOOPT	0x0004
+#define LIBXFS_MOUNT_COMPAT_ATTR	0x0008
+#define LIBXFS_MOUNT_ATTR2		0x0010
 
 #define LIBXFS_IHASHSIZE(sbp)		(1<<10)
 #define LIBXFS_BHASHSIZE(sbp) 		(1<<10)
libxfs/init.c +0 -128

146 lines not on this page · raw message

mkfs/proto.c +0 -1
index 0cdef41..4cc0df6 100644
@@ -543,7 +543,6 @@ parseproto(
 			pip = ip;
 			mp->m_sb.sb_rootino = ip->i_ino;
 			libxfs_mod_sb(tp, XFS_SB_ROOTINO);
-			mp->m_rootip = ip;
 			isroot = 1;
 		} else {
 			libxfs_trans_ijoin(tp, pip, 0);
mkfs/xfs_mkfs.c +2 -2
index 355708c..d37e948 100644
@@ -2582,6 +2582,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	memset(XFS_BUF_PTR(buf), 0, sectorsize);
 	libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp, XFS_SB_ALL_BITS);
 	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	libxfs_purgebuf(buf);
 
 	/*
 	 * If the data area is a file, then grow it out to its final size
@@ -2616,7 +2617,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 		(xfs_extlen_t)XFS_FSB_TO_BB(mp, logblocks),
 		&sbp->sb_uuid, logversion, lsunit, XLOG_FMT);
 
-	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 1);
+	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 0);
 	if (mp == NULL) {
 		fprintf(stderr, _("%s: filesystem failed to initialize\n"),
 			progname);
@@ -2887,7 +2888,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	/*
 	 * Allocate the root inode and anything else in the proto file.
 	 */
-	mp->m_rootip = NULL;
 	parse_proto(mp, &fsx, &protostring);
 
 	/*
repair/phase6.c +0 -2
index 2a37438..5307acf 100644
@@ -856,8 +856,6 @@ mk_root_dir(xfs_mount_t *mp)
 	ip->i_df.if_bytes = ip->i_df.if_real_bytes = 0;
 	ip->i_df.if_u1.if_extents = NULL;
 
-	mp->m_rootip = ip;
-
 	/*
 	 * initialize the directory
 	 */
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v4 15/37
#16 lore raw

Re: [PATCH 15/37] libxfs: fix root inode handling inconsistencies

Quoted 18 lines · Dave Chinner
db/init.c +5 -8
 	mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_ROOTINOS | LIBXFS_MOUNT_DEBUGGER);
+			  LIBXFS_MOUNT_DEBUGGER);
 	if (!mp) {
-		mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_DEBUGGER);
-		if (!mp) {
-			fprintf(stderr, _("%s: device %s unusable (not an XFS "
-				"filesystem?)\n"), progname, fsdevice);
-			exit(1);
-		}
+		fprintf(stderr,
+			_("%s: device %s unusable (not an XFS filesystem?)\n"),
+			progname, fsdevice);
+		exit(1);
 	}
 	blkbb = 1 << mp->m_blkbb_log;
 

db/init.c init( · line 170

Quoted 10 lines
db/init.c +10 -0
+	/*
+	 * xfs_check needs corrected incore superblock values
+	 */
+	if (sbp->sb_rootino != NULLFSINO &&
+	    xfs_sb_version_haslazysbcount(&mp->m_sb)) {
+		int error = xfs_initialize_perag_data(mp, sbp->sb_agcount);
+		if (error) {
+			fprintf(stderr, _("%s: cannot init perag data (%d)\n"),
+				progname, error);
+			exit(EXIT_FAILURE);

The old code above just goes ahead without the perage data due to the retry without LIBXFS_MOUNT_ROOTINOS. I think we should keep that behaviour as we want to be able to run xfs_db under as many circumstances as possible.

Otherwise looks good,

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#17 lore raw

PATCH v2 15/37 libxfs: fix root inode handling inconsistencies

libxfs: fix root inode handling inconsistencies

From: Dave Chinner <[email protected]>

When "mounting" a filesystem via libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem, libxfs_unmount() used to unconditionally free root inodes if they were present.

This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell libxfs_mount() to read the root inodes and so can result in unbalanced freeing of inodes when cleaning up during the unmount proceedure.

As it turns out, nothing ever uses mp->m_rootip and so we don't need to read it in or free it, or even have a pointer to it in the struct xfs_mount. Similarly, the only user of the realtime inodes is mkfs, and it initialises them itself. Hence we can kill the m_rootip and the realtime inode mounting code.

This leaves one user of LIBXFS_MOUNT_ROOTINOS - xfs_db - and that is only used to initialise the in-core superblock counter values from the ag header for xfs_check. Move this code to the xfs_db init functions so we can get rid of the mount parameter previously used to trigger all these behavours (LIBXFS_MOUNT_ROOTINOS) completely.

---
V2: removed the exit() from the case where the perag data could not be read so that we maintain the behaviour of xfs_db being able to mount corrupted filesystems.

 copy/xfs_copy.c  |   2 +-
 db/init.c        |  26 +++++++----
 include/libxfs.h |  12 +++---
 libxfs/init.c    | 128 -------------------------------------------------------
 mkfs/proto.c     |   1 -
 mkfs/xfs_mkfs.c  |   4 +-
 repair/phase6.c  |   2 -
 7 files changed, 26 insertions(+), 149 deletions(-)
copy/xfs_copy.c +1 -1
index bb37279..9986fbf 100644
@@ -684,7 +684,7 @@ main(int argc, char **argv)
 	sb = &mbuf.m_sb;
 	libxfs_sb_from_disk(sb, XFS_BUF_TO_SBP(sbp));
 
-	mp = libxfs_mount(&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, 1);
+	mp = libxfs_mount(&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, 0);
 	if (mp == NULL) {
 		do_log(_("%s: %s filesystem failed to initialize\n"
 			"%s: Aborting.\n"), progname, source_name, progname);
db/init.c +18 -8
index 2932e51..d73d549 100644
@@ -149,18 +149,28 @@ init(
 	}
 
 	mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_ROOTINOS | LIBXFS_MOUNT_DEBUGGER);
+			  LIBXFS_MOUNT_DEBUGGER);
 	if (!mp) {
-		mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_DEBUGGER);
-		if (!mp) {
-			fprintf(stderr, _("%s: device %s unusable (not an XFS "
-				"filesystem?)\n"), progname, fsdevice);
-			exit(1);
-		}
+		fprintf(stderr,
+			_("%s: device %s unusable (not an XFS filesystem?)\n"),
+			progname, fsdevice);
+		exit(1);
 	}
 	blkbb = 1 << mp->m_blkbb_log;
 
+	/*
+	 * xfs_check needs corrected incore superblock values
+	 */
+	if (sbp->sb_rootino != NULLFSINO &&
+	    xfs_sb_version_haslazysbcount(&mp->m_sb)) {
+		int error = xfs_initialize_perag_data(mp, sbp->sb_agcount);
+		if (error) {
+			fprintf(stderr,
+	_("%s: cannot init perag data (%d). Continuing anyway.\n"),
+				progname, error);
+		}
+	}
+
 	if (xfs_sb_version_hascrc(&mp->m_sb))
 		type_set_tab_crc();
 
include/libxfs.h +5 -7
index f10ab59..3df8c07 100644
@@ -167,7 +167,6 @@ typedef struct xfs_mount {
 	uint			m_rsumsize;	/* size of rt summary, bytes */
 	struct xfs_inode	*m_rbmip;	/* pointer to bitmap inode */
 	struct xfs_inode	*m_rsumip;	/* pointer to summary inode */
-	struct xfs_inode	*m_rootip;	/* pointer to root directory */
 	struct xfs_buftarg	*m_ddev_targp;
 	struct xfs_buftarg	*m_logdev_targp;
 	struct xfs_buftarg	*m_rtdev_targp;
@@ -259,12 +258,11 @@ typedef struct xfs_perag {
 	int		pagb_count;	/* pagb slots in use */
 } xfs_perag_t;
 
-#define LIBXFS_MOUNT_ROOTINOS		0x0001
-#define LIBXFS_MOUNT_DEBUGGER		0x0002
-#define LIBXFS_MOUNT_32BITINODES	0x0004
-#define LIBXFS_MOUNT_32BITINOOPT	0x0008
-#define LIBXFS_MOUNT_COMPAT_ATTR	0x0010
-#define LIBXFS_MOUNT_ATTR2		0x0020
+#define LIBXFS_MOUNT_DEBUGGER		0x0001
+#define LIBXFS_MOUNT_32BITINODES	0x0002
+#define LIBXFS_MOUNT_32BITINOOPT	0x0004
+#define LIBXFS_MOUNT_COMPAT_ATTR	0x0008
+#define LIBXFS_MOUNT_ATTR2		0x0010
 
 #define LIBXFS_IHASHSIZE(sbp)		(1<<10)
 #define LIBXFS_BHASHSIZE(sbp) 		(1<<10)
libxfs/init.c +0 -128

146 lines not on this page · raw message

mkfs/proto.c +0 -1
index 0cdef41..4cc0df6 100644
@@ -543,7 +543,6 @@ parseproto(
 			pip = ip;
 			mp->m_sb.sb_rootino = ip->i_ino;
 			libxfs_mod_sb(tp, XFS_SB_ROOTINO);
-			mp->m_rootip = ip;
 			isroot = 1;
 		} else {
 			libxfs_trans_ijoin(tp, pip, 0);
mkfs/xfs_mkfs.c +2 -2
index 355708c..d37e948 100644
@@ -2582,6 +2582,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	memset(XFS_BUF_PTR(buf), 0, sectorsize);
 	libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp, XFS_SB_ALL_BITS);
 	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	libxfs_purgebuf(buf);
 
 	/*
 	 * If the data area is a file, then grow it out to its final size
@@ -2616,7 +2617,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 		(xfs_extlen_t)XFS_FSB_TO_BB(mp, logblocks),
 		&sbp->sb_uuid, logversion, lsunit, XLOG_FMT);
 
-	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 1);
+	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 0);
 	if (mp == NULL) {
 		fprintf(stderr, _("%s: filesystem failed to initialize\n"),
 			progname);
@@ -2887,7 +2888,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	/*
 	 * Allocate the root inode and anything else in the proto file.
 	 */
-	mp->m_rootip = NULL;
 	parse_proto(mp, &fsx, &protostring);
 
 	/*
repair/phase6.c +0 -2
index 2a37438..5307acf 100644
@@ -856,8 +856,6 @@ mk_root_dir(xfs_mount_t *mp)
 	ip->i_df.if_bytes = ip->i_df.if_real_bytes = 0;
 	ip->i_df.if_u1.if_extents = NULL;
 
-	mp->m_rootip = ip;
-
 	/*
 	 * initialize the directory
 	 */

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#18 lore raw

PATCH v3 15/37 libxfs: fix root inode handling inconsistencies

From: Dave Chinner <[email protected]>

When "mounting" a filesystem via libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem, libxfs_unmount() used to unconditionally free root inodes if they were present.

This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell libxfs_mount() to read the root inodes and so can result in unbalanced freeing of inodes when cleaning up during the unmount proceedure.

As it turns out, nothing ever uses mp->m_rootip and so we don't need to read it in or free it, or even have a pointer to it in the struct xfs_mount. Similarly, the only user of the realtime inodes is mkfs, and it initialises them itself. Hence we can kill the m_rootip and the realtime inode mounting code.

This leaves one user of LIBXFS_MOUNT_ROOTINOS - xfs_db - and that is only used to initialise the in-core superblock counter values from the ag header for xfs_check. Move this code to the xfs_db init functions so we can get rid of the mount parameter previously used to trigger all these behavours (LIBXFS_MOUNT_ROOTINOS) completely.

---

V3: rtmount_init() is still needed to initialise the realtime device variables so that the code in libxfs/xfs_rtbitmap.c functions correctly. It was a bit hasty to remove it, and breaks mkfs. This code can be removed if we rework xfsprogs not to need the rtbitmap functions....

V2: removed the exit() from the case where the perag data could not be read so that we maintain the behaviour of xfs_db being able to mount corrupted filesystems.

 copy/xfs_copy.c  |  2 +-
 db/init.c        | 26 +++++++++++++++-------
 include/libxfs.h | 12 +++++-----
 libxfs/init.c    | 67 --------------------------------------------------------
 mkfs/proto.c     |  1 -
 mkfs/xfs_mkfs.c  |  4 ++--
 repair/phase6.c  |  2 --
 7 files changed, 26 insertions(+), 88 deletions(-)
copy/xfs_copy.c +1 -1
index bb37279..9986fbf 100644
@@ -684,7 +684,7 @@ main(int argc, char **argv)
 	sb = &mbuf.m_sb;
 	libxfs_sb_from_disk(sb, XFS_BUF_TO_SBP(sbp));
 
-	mp = libxfs_mount(&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, 1);
+	mp = libxfs_mount(&mbuf, sb, xargs.ddev, xargs.logdev, xargs.rtdev, 0);
 	if (mp == NULL) {
 		do_log(_("%s: %s filesystem failed to initialize\n"
 			"%s: Aborting.\n"), progname, source_name, progname);
db/init.c +18 -8
index 2932e51..d73d549 100644
@@ -149,18 +149,28 @@ init(
 	}
 
 	mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_ROOTINOS | LIBXFS_MOUNT_DEBUGGER);
+			  LIBXFS_MOUNT_DEBUGGER);
 	if (!mp) {
-		mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
-				LIBXFS_MOUNT_DEBUGGER);
-		if (!mp) {
-			fprintf(stderr, _("%s: device %s unusable (not an XFS "
-				"filesystem?)\n"), progname, fsdevice);
-			exit(1);
-		}
+		fprintf(stderr,
+			_("%s: device %s unusable (not an XFS filesystem?)\n"),
+			progname, fsdevice);
+		exit(1);
 	}
 	blkbb = 1 << mp->m_blkbb_log;
 
+	/*
+	 * xfs_check needs corrected incore superblock values
+	 */
+	if (sbp->sb_rootino != NULLFSINO &&
+	    xfs_sb_version_haslazysbcount(&mp->m_sb)) {
+		int error = xfs_initialize_perag_data(mp, sbp->sb_agcount);
+		if (error) {
+			fprintf(stderr,
+	_("%s: cannot init perag data (%d). Continuing anyway.\n"),
+				progname, error);
+		}
+	}
+
 	if (xfs_sb_version_hascrc(&mp->m_sb))
 		type_set_tab_crc();
 
include/libxfs.h +5 -7
index f10ab59..3df8c07 100644
@@ -167,7 +167,6 @@ typedef struct xfs_mount {
 	uint			m_rsumsize;	/* size of rt summary, bytes */
 	struct xfs_inode	*m_rbmip;	/* pointer to bitmap inode */
 	struct xfs_inode	*m_rsumip;	/* pointer to summary inode */
-	struct xfs_inode	*m_rootip;	/* pointer to root directory */
 	struct xfs_buftarg	*m_ddev_targp;
 	struct xfs_buftarg	*m_logdev_targp;
 	struct xfs_buftarg	*m_rtdev_targp;
@@ -259,12 +258,11 @@ typedef struct xfs_perag {
 	int		pagb_count;	/* pagb slots in use */
 } xfs_perag_t;
 
-#define LIBXFS_MOUNT_ROOTINOS		0x0001
-#define LIBXFS_MOUNT_DEBUGGER		0x0002
-#define LIBXFS_MOUNT_32BITINODES	0x0004
-#define LIBXFS_MOUNT_32BITINOOPT	0x0008
-#define LIBXFS_MOUNT_COMPAT_ATTR	0x0010
-#define LIBXFS_MOUNT_ATTR2		0x0020
+#define LIBXFS_MOUNT_DEBUGGER		0x0001
+#define LIBXFS_MOUNT_32BITINODES	0x0002
+#define LIBXFS_MOUNT_32BITINOOPT	0x0004
+#define LIBXFS_MOUNT_COMPAT_ATTR	0x0008
+#define LIBXFS_MOUNT_ATTR2		0x0010
 
 #define LIBXFS_IHASHSIZE(sbp)		(1<<10)
 #define LIBXFS_BHASHSIZE(sbp) 		(1<<10)
libxfs/init.c +0 -67

79 lines not on this page · raw message

mkfs/proto.c +0 -1
index 0cdef41..4cc0df6 100644
@@ -543,7 +543,6 @@ parseproto(
 			pip = ip;
 			mp->m_sb.sb_rootino = ip->i_ino;
 			libxfs_mod_sb(tp, XFS_SB_ROOTINO);
-			mp->m_rootip = ip;
 			isroot = 1;
 		} else {
 			libxfs_trans_ijoin(tp, pip, 0);
mkfs/xfs_mkfs.c +2 -2
index 355708c..d37e948 100644
@@ -2582,6 +2582,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	memset(XFS_BUF_PTR(buf), 0, sectorsize);
 	libxfs_sb_to_disk((void *)XFS_BUF_PTR(buf), sbp, XFS_SB_ALL_BITS);
 	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+	libxfs_purgebuf(buf);
 
 	/*
 	 * If the data area is a file, then grow it out to its final size
@@ -2616,7 +2617,7 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 		(xfs_extlen_t)XFS_FSB_TO_BB(mp, logblocks),
 		&sbp->sb_uuid, logversion, lsunit, XLOG_FMT);
 
-	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 1);
+	mp = libxfs_mount(mp, sbp, xi.ddev, xi.logdev, xi.rtdev, 0);
 	if (mp == NULL) {
 		fprintf(stderr, _("%s: filesystem failed to initialize\n"),
 			progname);
@@ -2887,7 +2888,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	/*
 	 * Allocate the root inode and anything else in the proto file.
 	 */
-	mp->m_rootip = NULL;
 	parse_proto(mp, &fsx, &protostring);
 
 	/*
repair/phase6.c +0 -2
index 2a37438..5307acf 100644
@@ -856,8 +856,6 @@ mk_root_dir(xfs_mount_t *mp)
 	ip->i_df.if_bytes = ip->i_df.if_real_bytes = 0;
 	ip->i_df.if_u1.if_extents = NULL;
 
-	mp->m_rootip = ip;
-
 	/*
 	 * initialize the directory
 	 */

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by on v3 15/37
#19 lore raw

Re: [PATCH 15/37 V3] libxfs: fix root inode handling inconsistencies

Quoted 19 lines · Dave Chinner

From: Dave Chinner <[email protected]>

When "mounting" a filesystem via libxfs_mount(), callers can tell libxfs to read the root and realtime inodes into cache. However, when unmounting the filesystem, libxfs_unmount() used to unconditionally free root inodes if they were present.

This leads to interesting issues like in mkfs, when it handles creation, reading and freeing of the root and rt inodes itself. It, however, passes in the flag to tell libxfs_mount() to read the root inodes and so can result in unbalanced freeing of inodes when cleaning up during the unmount proceedure.

As it turns out, nothing ever uses mp->m_rootip and so we don't need to read it in or free it, or even have a pointer to it in the struct xfs_mount. Similarly, the only user of the realtime inodes is mkfs, and it initialises them itself. Hence we can kill the m_rootip and the realtime inode mounting code.

Quoted 7 lines

This leaves one user of LIBXFS_MOUNT_ROOTINOS - xfs_db - and that is only used to initialise the in-core superblock counter values from the ag header for xfs_check. Move this code to the xfs_db init functions so we can get rid of the mount parameter previously used to trigger all these behavours (LIBXFS_MOUNT_ROOTINOS) completely.

Looks good,

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#20 lore raw

PATCH v4 16/37 libxfs: stop caching inode structures

In reply to Dave Chinner

From: Christoph Hellwig <[email protected]>

Currently libxfs has a cache for xfs_inode structures. Unlike in kernelspace where the inode cache, and the associated page cache for file data is used for all filesystem operations the libxfs inode cache is only used in few places:

- the libxfs init code reads the root and realtime inodes when called from xfs_db using a special flag, but these inode structure are never referenced again
- mkfs uses namespace and bmap routines that take the xfs_inode structure to create the root and realtime inodes, as well as any additional files specified in the proto file
- the xfs_db attr code uses xfs_inode-based attr routines in the attrset and attrget commands
- phase6 of xfs_repair uses xfs_inode-based routines for rebuilding directories and moving files to the lost+found directory.
- phase7 of xfs_repair uses struct xfs_inode to modify the nlink count of inodes.

So except in repair we never ever reuse a cached inode, and even in repair the logical inode caching doesn't help:

- in phase 6a we iterate over each inode in the incore inode tree, and if it's a directory check/rebuild it
- phase6b then updates the "." and ".." entries for directories that need, which means we require the backing buffers.
- phase6c moves disconnected inodes to lost_found, which again needs the backing buffer to actually do anything.
- phase7 then only touches inodes for which we need to reset i_nlink, which always involves reading, modifying and writing the physical inode.
which always involves modifying the . and .. entries.

Given these facts stop caching the inodes to reduce memory usage especially in xfs_repair, where this makes a different for large inode count inodes. On the upper end this allows repair to complete for filesystem / amount of memory combinations that previously wouldn't.

With this we probably could increase the memory available to the buffer cache in xfs_repair, but trying to do so I got a bit lost - the current formula seems to magic to me to make any sense, and simply doubling the buffer cache size causes us to run out of memory given that the data cached in the buffer cache (typically lots of 8k inode buffers and few 4k other metadata buffers) are much bigger than the inodes cached in the inode cache. We probably need a sizing scheme that takes the actual amount of memory allocated to the buffer cache into account to solve this better.

---
 include/libxfs.h      |  5 ---
 libxfs/init.c         |  9 ------
 libxfs/rdwr.c         | 87 ++++++++++++---------------------------------------
 man/man8/xfs_repair.8 |  6 ----
 mkfs/xfs_mkfs.c       |  1 -
 repair/xfs_repair.c   | 14 ++-------
 6 files changed, 23 insertions(+), 99 deletions(-)
include/libxfs.h +0 -5
index 3df8c07..e017b32 100644
@@ -264,7 +264,6 @@ typedef struct xfs_perag {
 #define LIBXFS_MOUNT_COMPAT_ATTR	0x0008
 #define LIBXFS_MOUNT_ATTR2		0x0010
 
-#define LIBXFS_IHASHSIZE(sbp)		(1<<10)
 #define LIBXFS_BHASHSIZE(sbp) 		(1<<10)
 
 extern xfs_mount_t	*libxfs_mount (xfs_mount_t *, xfs_sb_t *,
@@ -448,7 +447,6 @@ extern int	libxfs_writebuf_int(xfs_buf_t *, int);
 extern int	libxfs_readbufr(struct xfs_buftarg *, xfs_daddr_t, xfs_buf_t *, int, int);
 
 extern int libxfs_bhash_size;
-extern int libxfs_ihash_size;
 
 #define LIBXFS_BREAD	0x1
 #define LIBXFS_BWRITE	0x2
@@ -648,9 +646,6 @@ extern void	libxfs_trans_ichgtime(struct xfs_trans *,
 extern int	libxfs_iflush_int (xfs_inode_t *, xfs_buf_t *);
 
 /* Inode Cache Interfaces */
-extern struct cache	*libxfs_icache;
-extern struct cache_operations	libxfs_icache_operations;
-extern void	libxfs_icache_purge (void);
 extern int	libxfs_iget (xfs_mount_t *, xfs_trans_t *, xfs_ino_t,
 				uint, xfs_inode_t **, xfs_daddr_t);
 extern void	libxfs_iput (xfs_inode_t *, uint);
libxfs/init.c +0 -9
index ee6165e..abe3316 100644
@@ -22,9 +22,6 @@
 
 char *progname = "libxfs";	/* default, changed by each tool */
 
-struct cache *libxfs_icache;	/* global inode cache */
-int libxfs_ihash_size;		/* #buckets in icache */
-
 struct cache *libxfs_bcache;	/* global buffer cache */
 int libxfs_bhash_size;		/* #buckets in bcache */
 
@@ -335,9 +332,6 @@ libxfs_init(libxfs_init_t *a)
 	}
 	if (needcd)
 		chdir(curdir);
-	if (!libxfs_ihash_size)
-		libxfs_ihash_size = LIBXFS_IHASHSIZE(sbp);
-	libxfs_icache = cache_init(libxfs_ihash_size, &libxfs_icache_operations);
 	if (!libxfs_bhash_size)
 		libxfs_bhash_size = LIBXFS_BHASHSIZE(sbp);
 	libxfs_bcache = cache_init(libxfs_bhash_size, &libxfs_bcache_operations);
@@ -738,7 +732,6 @@ libxfs_umount(xfs_mount_t *mp)
 	int			agno;
 
 	libxfs_rtmount_destroy(mp);
-	libxfs_icache_purge();
 	libxfs_bcache_purge();
 
 	for (agno = 0; agno < mp->m_maxagi; agno++) {
@@ -754,7 +747,6 @@ void
 libxfs_destroy(void)
 {
 	manage_zones(1);
-	cache_destroy(libxfs_icache);
 	cache_destroy(libxfs_bcache);
 }
 
@@ -770,7 +762,6 @@ libxfs_report(FILE *fp)
 	time_t t;
 	char *c;
 
-	cache_report(fp, "libxfs_icache", libxfs_icache);
 	cache_report(fp, "libxfs_bcache", libxfs_bcache);
 
 	t = time(NULL);
libxfs/rdwr.c +20 -67

112 lines not on this page · raw message

man/man8/xfs_repair.8 +0 -6
index 96adb29..47436ec 100644
@@ -130,12 +130,6 @@ The
 supported are:
 .RS 1.0i
 .TP
-.BI ihash= ihashsize
-overrides the default inode cache hash size. The total number of
-inode cache entries are limited to 8 times this amount. The default
-.I ihashsize
-is 1024 (for a total of 8192 entries).
-.TP
 .BI bhash= bhashsize
 overrides the default buffer cache hash size. The total number of
 buffer cache entries are limited to 8 times this amount. The default
mkfs/xfs_mkfs.c +0 -1
index d37e948..3a032c0 100644
@@ -2935,7 +2935,6 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	 * Need to drop references to inodes we still hold, first.
 	 */
 	libxfs_rtmount_destroy(mp);
-	libxfs_icache_purge();
 	libxfs_bcache_purge();
 
 	/*
repair/xfs_repair.c +3 -11
index 820e7a2..214b7fa 100644
@@ -69,7 +69,6 @@ static char *c_opts[] = {
 };
 
 
-static int	ihash_option_used;
 static int	bhash_option_used;
 static long	max_mem_specified;	/* in megabytes */
 static int	phase2_threads = 32;
@@ -239,13 +238,13 @@ process_args(int argc, char **argv)
 					pre_65_beta = 1;
 					break;
 				case IHASH_SIZE:
-					libxfs_ihash_size = (int)strtol(val, NULL, 0);
-					ihash_option_used = 1;
+					do_warn(
+		_("-o ihash option has been removed and will be ignored\n"));
 					break;
 				case BHASH_SIZE:
 					if (max_mem_specified)
 						do_abort(
-			_("-o bhash option cannot be used with -m option\n"));
+		_("-o bhash option cannot be used with -m option\n"));
 					libxfs_bhash_size = (int)strtol(val, NULL, 0);
 					bhash_option_used = 1;
 					break;
@@ -648,9 +647,7 @@ main(int argc, char **argv)
 		unsigned long	max_mem;
 		struct rlimit	rlim;
 
-		libxfs_icache_purge();
 		libxfs_bcache_purge();
-		cache_destroy(libxfs_icache);
 		cache_destroy(libxfs_bcache);
 
 		mem_used = (mp->m_sb.sb_icount >> (10 - 2)) +
@@ -709,11 +706,6 @@ main(int argc, char **argv)
 			do_log(_("        - block cache size set to %d entries\n"),
 				libxfs_bhash_size * HASH_CACHE_RATIO);
 
-		if (!ihash_option_used)
-			libxfs_ihash_size = libxfs_bhash_size;
-
-		libxfs_icache = cache_init(libxfs_ihash_size,
-						&libxfs_icache_operations);
 		libxfs_bcache = cache_init(libxfs_bhash_size,
 						&libxfs_bcache_operations);
 	}
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#21 lore raw

PATCH v4 17/37 db: separate out straight buffer IO from map based IO.

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Libxfs has two different interfaces for getting and reading buffers. The first is a block/length interface for reading contiguous regions, and the second is based on extent based xfs_buf_map arrays for discontiguous regions. The xfs-db code is solely based on a basic block array interface regardless of the type of region being read, and so doesn't match to either libxfs interface.

As a first step to converting xfs_db to the libxfs interfaces, add a simple block/length buffer API and implement it using pread/pwrite. Then remove the single region conditionals from the basic block array based interfaces, and convert all the contiguous block read cases to use the new API.

This new API is temporary - it will be replaced by the equivalent libxfs interface calls once all the infrastructure preparation for the changeover has been completed.

---
 db/init.c |   7 ++--
 db/io.c   | 125 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
 db/io.h   |   5 +--
 3 files changed, 104 insertions(+), 33 deletions(-)
db/init.c +3 -4
index 54f3f9e..b93a194 100644
@@ -55,7 +55,7 @@ init(
 	char		**argv)
 {
 	xfs_sb_t	*sbp;
-	void		*bufp = NULL;
+	char		bufp[BBSIZE];
 	int		c;
 
 	setlocale(LC_ALL, "");
@@ -115,15 +115,14 @@ init(
 		exit(1);
 	}
 
-	if (read_bbs(XFS_SB_DADDR, 1, &bufp, NULL)) {
+	if (read_buf(XFS_SB_DADDR, 1, bufp)) {
 		fprintf(stderr, _("%s: %s is invalid (cannot read first 512 "
 			"bytes)\n"), progname, fsdevice);
 		exit(1);
 	}
 
 	/* copy SB from buffer to in-core, converting architecture as we go */
-	libxfs_sb_from_disk(&xmount.m_sb, bufp);
-	xfree(bufp);
+	libxfs_sb_from_disk(&xmount.m_sb, (struct xfs_dsb *)bufp);
 
 	sbp = &xmount.m_sb;
 	if (sbp->sb_magicnum != XFS_SB_MAGIC) {
db/io.c +100 -25

199 lines not on this page · raw message

db/io.h +1 -4
index 549aad9..9ea6223 100644
@@ -52,10 +52,7 @@ extern void	off_cur(int off, int len);
 extern void	pop_cur(void);
 extern void	print_iocur(char *tag, iocur_t *ioc);
 extern void	push_cur(void);
-extern int	read_bbs(__int64_t daddr, int count, void **bufp,
-			 bbmap_t *bbmap);
-extern int	write_bbs(__int64_t daddr, int count, void *bufp,
-			  bbmap_t *bbmap);
+extern int	read_buf(__int64_t daddr, int count, void *bufp);
 extern void     write_cur(void);
 extern void	set_cur(const struct typ *t, __int64_t d, int c, int ring_add,
 			bbmap_t *bbmap);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#23 lore raw

PATCH v4 18/37 db: rewrite bbmap to use xfs_buf_map

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Use the libxfs struct xfs_buf_map for recording the extent layout of discontiguous buffers and convert the read/write to decode them directory and use read_buf/write_buf to do the extent IO. This brings the physical xfs_db IO code to be very close to the model that libxfs uses.

---
 db/bmap.c | 15 ++++-----------
 db/io.c   | 58 ++++++++++++----------------------------------------------
 db/io.h   |  3 ++-
 3 files changed, 18 insertions(+), 58 deletions(-)
db/bmap.c +4 -11
index 0ef7a62..3951b9f 100644
@@ -293,20 +293,13 @@ make_bbmap(
 	int		nex,
 	bmap_ext_t	*bmp)
 {
-	int		d;
-	xfs_dfsbno_t	dfsbno;
 	int		i;
-	int		j;
-	int		k;
 
-	for (i = 0, d = 0; i < nex; i++) {
-		dfsbno = bmp[i].startblock;
-		for (j = 0; j < bmp[i].blockcount; j++, dfsbno++) {
-			for (k = 0; k < blkbb; k++)
-				bbmap->b[d++] =
-					XFS_FSB_TO_DADDR(mp, dfsbno) + k;
-		}
+	for (i = 0; i < nex; i++) {
+		bbmap->b[i].bm_bn = XFS_FSB_TO_DADDR(mp, bmp[i].startblock);
+		bbmap->b[i].bm_len = XFS_FSB_TO_BB(mp, bmp[i].blockcount);
 	}
+	bbmap->nmaps = nex;
 }
 
 static xfs_fsblock_t
db/io.c +12 -46

85 lines not on this page · raw message

db/io.h +2 -1
index 9ea6223..c7641d5 100644
@@ -20,7 +20,8 @@ struct typ;
 
 #define	BBMAP_SIZE		(XFS_MAX_BLOCKSIZE / BBSIZE)
 typedef struct bbmap {
-	__int64_t		b[BBMAP_SIZE];
+	int			nmaps;
+	struct xfs_buf_map	b[BBMAP_SIZE];
 } bbmap_t;
 
 typedef struct iocur {
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#24 lore raw

PATCH v4 19/37 libxfs: refactor libxfs_buf_read_map for xfs_db

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

xfs_db requires low level read/write buffer primitives that are the equivalent of libxfs_readbufr/writebufr. The implementation of libxfs_writebufr already handles discontiguous buffers, but there is no equivalent libxfs_readbufr_map support in the code.

Refactor libxfs_readbuf_map into two parts - one that does the buffer cache lookup, and the other that does the read IO. This provides the implementation of libxfs_readbufr_map that is required for xfs_db.

---
 include/libxfs.h |  3 +++
 libxfs/rdwr.c    | 61 +++++++++++++++++++++++++++++++++++++-------------------
 2 files changed, 43 insertions(+), 21 deletions(-)
include/libxfs.h +3 -0
index e017b32..b097bd2 100644
@@ -444,7 +444,10 @@ extern xfs_buf_t *libxfs_getbufr(struct xfs_buftarg *, xfs_daddr_t, int);
 extern void	libxfs_putbufr(xfs_buf_t *);
 
 extern int	libxfs_writebuf_int(xfs_buf_t *, int);
+extern int	libxfs_writebufr(struct xfs_buf *);
 extern int	libxfs_readbufr(struct xfs_buftarg *, xfs_daddr_t, xfs_buf_t *, int, int);
+extern int	libxfs_readbufr_map(struct xfs_buftarg *, struct xfs_buf *,
+				    struct xfs_buf_map *, int, int);
 
 extern int libxfs_bhash_size;
 
libxfs/rdwr.c +40 -21

76 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v4 19/37
#25 lore raw

Re: [PATCH 19/37] libxfs: refactor libxfs_buf_read_map for xfs_db

Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs_db requires low level read/write buffer primitives that are the equivalent of libxfs_readbufr/writebufr. The implementation of libxfs_writebufr already handles discontiguous buffers, but there is no equivalent libxfs_readbufr_map support in the code.

Quoted 6 lines

Refactor libxfs_readbuf_map into two parts - one that does the buffer cache lookup, and the other that does the read IO. This provides the implementation of libxfs_readbufr_map that is required for xfs_db.

Looks good,

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#26 lore raw

PATCH v4 20/37 db: rewrite IO engine to use libxfs

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Now that we have buffers and xfs_buf_maps, it is relatively easy to convert the IO engine to use libxfs routines. This gets rid of the most of the differences between mapped and straight buffer reads, and tracks xfs_bufs directly in the IO context that is being used.

This is not yet a perfect solution, as xfs_db does different sized IOs for the same block range which will throw warnings like:

xfs_db> inode 64
7ffff7fde740: Badness in key lookup (length) bp=(bno 0x40, len 8192 bytes) key=(bno 0x40, len 4096 bytes) xfs_db>

This is when first displaying an inode in the root inode chunk. These will need to be dealt with on a case by case basis.

---
 db/init.c |  25 +++++++--
 db/io.c   | 178 +++++++++++++++++---------------------------------------------
 db/io.h   |   4 +-
 3 files changed, 70 insertions(+), 137 deletions(-)
db/init.c +21 -4
index b93a194..48b8dd1 100644
@@ -54,8 +54,8 @@ init(
 	int		argc,
 	char		**argv)
 {
-	xfs_sb_t	*sbp;
-	char		bufp[BBSIZE];
+	struct xfs_sb	*sbp;
+	struct xfs_buf	*bp;
 	int		c;
 
 	setlocale(LC_ALL, "");
@@ -115,14 +115,25 @@ init(
 		exit(1);
 	}
 
-	if (read_buf(XFS_SB_DADDR, 1, bufp)) {
+	/*
+	 * Read the superblock, but don't validate it - we are a diagnostic
+	 * tool and so need to be able to mount busted filesystems.
+	 */
+	memset(&xmount, 0, sizeof(struct xfs_mount));
+	libxfs_buftarg_init(&xmount, x.ddev, x.logdev, x.rtdev);
+	bp = libxfs_readbuf(xmount.m_ddev_targp, XFS_SB_DADDR,
+			    1 << (XFS_MAX_SECTORSIZE_LOG - BBSHIFT), 0, NULL);
+
+	if (!bp || bp->b_error) {
 		fprintf(stderr, _("%s: %s is invalid (cannot read first 512 "
 			"bytes)\n"), progname, fsdevice);
 		exit(1);
 	}
 
 	/* copy SB from buffer to in-core, converting architecture as we go */
-	libxfs_sb_from_disk(&xmount.m_sb, (struct xfs_dsb *)bufp);
+	libxfs_sb_from_disk(&xmount.m_sb, XFS_BUF_TO_SBP(bp));
+	libxfs_putbuf(bp);
+	libxfs_purgebuf(bp);
 
 	sbp = &xmount.m_sb;
 	if (sbp->sb_magicnum != XFS_SB_MAGIC) {
@@ -211,6 +222,12 @@ main(
 	}
 
 close_devices:
+	/*
+	 * make sure that we pop the last buffer context we held so that the
+	 * buffer is released before purge the caches during unmount.
+	 */
+	pop_cur();
+	libxfs_umount(mp);
 	if (x.ddev)
 		libxfs_device_close(x.ddev);
 	if (x.logdev && x.logdev != x.ddev)
db/io.c +47 -131

263 lines not on this page · raw message

db/io.h +2 -2
index c7641d5..2c47ccc 100644
@@ -36,8 +36,8 @@ typedef struct iocur {
 	__uint16_t		mode;	/* current inode's mode */
 	xfs_off_t		off;	/* fs offset of "data" in bytes */
 	const struct typ	*typ;	/* type of "data" */
-	int			use_bbmap; /* set if bbmap is valid */
-	bbmap_t			bbmap;	/* map daddr if fragmented */
+	bbmap_t			*bbmap;	/* map daddr if fragmented */
+	struct xfs_buf		*bp;	/* underlying buffer */
 } iocur_t;
 
 #define DB_RING_ADD 1                   /* add to ring on set_cur */
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#27 lore raw

PATCH v4 21/37 db: introduce verifier support into set_cur

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

To be able to use read and write verifiers, we need to pass the verifier to the IO routines. We do this via the set_cur() function used to trigger reading the buffer.

For most metadata types, there is only one type of verifier needed. For these, we can simply add the verifier to the type table entry for the given type and use that directly. This type entry is already carried around by the IO context, so if we ever need to get it again we have direct access to it in the context we'll be doing IO.

Only attach the verifiers to the v5 filesystem type table; there is not need for them on v4 filesystems as we don't have to verify or calculate CRCs for them.

There are some metadata types that have more than one buffer format, or aren't based in directly in buffers. For these, leave the type table verifier NULL for now - these will need to be addressed individually.

---
 db/io.c   | 13 +++++++---
 db/type.c | 88 ++++++++++++++++++++++++++++++++++-----------------------------
 db/type.h |  1 +
 3 files changed, 59 insertions(+), 43 deletions(-)
db/io.c +10 -3
index ca89354..2d1cc56 100644
@@ -482,12 +482,14 @@ set_cur(
 	xfs_ino_t	dirino;
 	xfs_ino_t	ino;
 	__uint16_t	mode;
+	const struct xfs_buf_ops *ops = t ? t->bops : NULL;
 
 	if (iocur_sp < 0) {
 		dbprintf(_("set_cur no stack element to set\n"));
 		return;
 	}
 
+
 	ino = iocur_top->ino;
 	dirino = iocur_top->dirino;
 	mode = iocur_top->mode;
@@ -509,12 +511,17 @@ set_cur(
 			return;
 		memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
 		bp = libxfs_readbuf_map(mp->m_ddev_targp, bbmap->b,
-					bbmap->nmaps, 0, NULL);
+					bbmap->nmaps, 0, ops);
 	} else {
-		bp = libxfs_readbuf(mp->m_ddev_targp, d, c, 0, NULL);
+		bp = libxfs_readbuf(mp->m_ddev_targp, d, c, 0, ops);
 		iocur_top->bbmap = NULL;
 	}
-	if (!bp || bp->b_error)
+
+	/*
+	 * keep the buffer even if the verifier says it is corrupted.
+	 * We're a diagnostic tool, after all.
+	 */
+	if (!bp || (bp->b_error && bp->b_error != EFSCORRUPTED))
 		return;
 	iocur_top->buf = bp->b_addr;
 	iocur_top->bp = bp;
db/type.c +48 -40

98 lines not on this page · raw message

db/type.h +1 -0
index c41aca4d..3bb26f1 100644
@@ -42,6 +42,7 @@ typedef struct typ
 	char			*name;
 	pfunc_t			pfunc;
 	const struct field	*fields;
+	const struct xfs_buf_ops *bops;
 } typ_t;
 extern const typ_t	*typtab, *cur_typ;
 
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#28 lore raw

PATCH v4 22/37 db: indicate if the CRC on a buffer is correct or not

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When dumping metadata that has a CRC in it, output not only the CRC but text to tell us whether the value is correct or not. Hence we can see at a glance if there's something wrong or not.

Do this by peeking at the buffer attached to the current IO contexted. If there was a CRC error, then it will be marked with a EFSCORRUPTED error. Use this to determine what to output.

---
 db/agf.c     |  2 +-
 db/agfl.c    |  2 +-
 db/agi.c     |  2 +-
 db/btblock.c | 10 +++++-----
 db/dir2.c    |  4 ++--
 db/dquot.c   |  2 +-
 db/field.c   |  5 +++++
 db/field.h   |  4 ++++
 db/fprint.c  | 39 +++++++++++++++++++++++++++++++++++++++
 db/fprint.h  |  2 ++
 db/inode.c   |  2 +-
 db/io.h      |  6 ++++++
 db/sb.c      |  2 +-
 db/symlink.c |  2 +-
 14 files changed, 70 insertions(+), 14 deletions(-)
db/agf.c +1 -1
index 389cb43..d9a07ca 100644
@@ -71,7 +71,7 @@ const field_t	agf_flds[] = {
 	{ "btreeblks", FLDT_UINT32D, OI(OFF(btreeblks)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ NULL }
 };
 
db/agfl.c +1 -1
index e2340e6..b29538f 100644
@@ -58,7 +58,7 @@ const field_t	agfl_crc_flds[] = {
 	{ "seqno", FLDT_AGNUMBER, OI(OFF(seqno)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_AGBLOCKNZ, OI(OFF(bno)), agfl_bno_size,
 	  FLD_ARRAY|FLD_COUNT, TYP_DATA },
 	{ NULL }
db/agi.c +1 -1
index 6b2e889..398bdbb 100644
@@ -56,7 +56,7 @@ const field_t	agi_flds[] = {
 	  CI(XFS_AGI_UNLINKED_BUCKETS), FLD_ARRAY, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ NULL }
 };
 
db/btblock.c +5 -5
index 34188db..1ea0cff 100644
@@ -295,7 +295,7 @@ const field_t	bmapbta_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.l.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.l.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_INO, OI(OFF(u.l.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_BMAPBTAREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_BMAPBTAKEY, btblock_key_offset, btblock_key_count,
@@ -314,7 +314,7 @@ const field_t	bmapbtd_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.l.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.l.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_INO, OI(OFF(u.l.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_BMAPBTDREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_BMAPBTDKEY, btblock_key_offset, btblock_key_count,
@@ -405,7 +405,7 @@ const field_t	inobt_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_INOBTREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_INOBTKEY, btblock_key_offset, btblock_key_count,
@@ -471,7 +471,7 @@ const field_t	bnobt_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_BNOBTREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_BNOBTKEY, btblock_key_offset, btblock_key_count,
@@ -533,7 +533,7 @@ const field_t	cntbt_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_CNTBTREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_CNTBTKEY, btblock_key_offset, btblock_key_count,
db/dir2.c +2 -2
index 8b08d48..2ec64e0 100644
@@ -922,7 +922,7 @@ const field_t	dir3_data_union_flds[] = {
 #define	DBH3OFF(f)	bitize(offsetof(struct xfs_dir3_blk_hdr, f))
 const field_t	dir3_blkhdr_flds[] = {
 	{ "magic", FLDT_UINT32X, OI(DBH3OFF(magic)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(DBH3OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(DBH3OFF(crc)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_DFSBNO, OI(DBH3OFF(blkno)), C1, 0, TYP_BMAPBTD },
 	{ "lsn", FLDT_UINT64X, OI(DBH3OFF(lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(DBH3OFF(uuid)), C1, 0, TYP_NONE },
@@ -959,7 +959,7 @@ const field_t	dir3_free_hdr_flds[] = {
 #define	DB3OFF(f)	bitize(offsetof(struct xfs_da3_blkinfo, f))
 const field_t	da3_blkinfo_flds[] = {
 	{ "hdr", FLDT_DA_BLKINFO, OI(DB3OFF(hdr)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(DB3OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(DB3OFF(crc)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_DFSBNO, OI(DB3OFF(blkno)), C1, 0, TYP_BMAPBTD },
 	{ "lsn", FLDT_UINT64X, OI(DB3OFF(lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(DB3OFF(uuid)), C1, 0, TYP_NONE },
db/dquot.c +1 -1
index 6927956..2f7d463 100644
@@ -48,7 +48,7 @@ const field_t	dqblk_flds[] = {
 	{ "diskdq", FLDT_DISK_DQUOT, OI(DDOFF(diskdq)), C1, 0, TYP_NONE },
 	{ "fill", FLDT_CHARS, OI(DDOFF(fill)), CI(DDSZC(fill)), FLD_SKIPALL,
 	  TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(DDOFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(DDOFF(crc)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(DDOFF(lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(DDOFF(uuid)), C1, 0, TYP_NONE },
 	{ NULL }
db/field.c +5 -0
index c6d7404..4582097 100644
@@ -163,6 +163,11 @@ const ftattr_t	ftattrtab[] = {
 	  0, fa_agblock, NULL },
 	{ FLDT_CNTBTREC, "cntbtrec", fp_sarray, (char *)cntbt_rec_flds,
 	  SI(bitsz(xfs_alloc_rec_t)), 0, NULL, cntbt_rec_flds },
+
+/* CRC field */
+	{ FLDT_CRC, "crc", fp_crc, "%#x (%s)", SI(bitsz(__uint32_t)),
+	  0, NULL, NULL },
+
 	{ FLDT_DEV, "dev", fp_num, "%#x", SI(bitsz(xfs_dev_t)), 0, NULL, NULL },
 	{ FLDT_DFILOFFA, "dfiloffa", fp_num, "%llu", SI(bitsz(xfs_dfiloff_t)),
 	  0, fa_dfiloffa, NULL },
db/field.h +4 -0
index aecdf9f..6343c9a 100644
@@ -80,6 +80,10 @@ typedef enum fldt	{
 	FLDT_CNTBTKEY,
 	FLDT_CNTBTPTR,
 	FLDT_CNTBTREC,
+
+	/* CRC field type */
+	FLDT_CRC,
+
 	FLDT_DEV,
 	FLDT_DFILOFFA,
 	FLDT_DFILOFFD,
db/fprint.c +39 -0
index 1d2f29c..435d984 100644
@@ -30,6 +30,7 @@
 #include "output.h"
 #include "sig.h"
 #include "malloc.h"
+#include "io.h"
 
 int
 fp_charns(
@@ -184,3 +185,41 @@ fp_uuid(
 	}
 	return 1;
 }
+
+/*
+ * CRC is correct is the current buffer it is being pulled out
+ * of is not marked with a EFSCORRUPTED error.
+ */
+int
+fp_crc(
+	void	*obj,
+	int	bit,
+	int	count,
+	char	*fmtstr,
+	int	size,
+	int	arg,
+	int	base,
+	int	array)
+{
+	int		bitpos;
+	int		i;
+	__int64_t	val;
+	char		*ok;
+
+	ok = iocur_crc_valid() ? "correct" : "bad";
+
+	for (i = 0, bitpos = bit;
+	     i < count && !seenint();
+	     i++, bitpos += size) {
+		if (array)
+			dbprintf("%d:", i + base);
+		val = getbitval(obj, bitpos, size, BVUNSIGNED);
+		if (size > 32)
+			dbprintf(fmtstr, val, ok);
+		else
+			dbprintf(fmtstr, (__int32_t)val, ok);
+		if (i < count - 1)
+			dbprintf(" ");
+	}
+	return 1;
+}
db/fprint.h +2 -0
index b032dbd..6a6d77e 100644
@@ -29,3 +29,5 @@ extern int	fp_time(void *obj, int bit, int count, char *fmtstr, int size,
 			int arg, int base, int array);
 extern int	fp_uuid(void *obj, int bit, int count, char *fmtstr, int size,
 			int arg, int base, int array);
+extern int	fp_crc(void *obj, int bit, int count, char *fmtstr, int size,
+		       int arg, int base, int array);
db/inode.c +1 -1
index 634dc30..ec533ee 100644
@@ -168,7 +168,7 @@ const field_t	inode_core_flds[] = {
 };
 
 const field_t	inode_v3_flds[] = {
-	{ "crc", FLDT_UINT32X, OI(COFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(COFF(crc)), C1, 0, TYP_NONE },
 	{ "change_count", FLDT_UINT64D, OI(COFF(changecount)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(COFF(lsn)), C1, 0, TYP_NONE },
 	{ "flags2", FLDT_UINT64X, OI(COFF(flags2)), C1, 0, TYP_NONE },
db/io.h +6 -0
index 2c47ccc..d647284 100644
@@ -58,3 +58,9 @@ extern void     write_cur(void);
 extern void	set_cur(const struct typ *t, __int64_t d, int c, int ring_add,
 			bbmap_t *bbmap);
 extern void     ring_add(void);
+
+static inline bool
+iocur_crc_valid()
+{
+	return (iocur_top->bp && iocur_top->bp->b_error != EFSCORRUPTED);
+}
db/sb.c +1 -1
index 4929152..6cb665d 100644
@@ -118,7 +118,7 @@ const field_t	sb_flds[] = {
 		C1, 0, TYP_NONE },
 	{ "features_log_incompat", FLDT_UINT32X, OI(OFF(features_log_incompat)),
 		C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ "pquotino", FLDT_INO, OI(OFF(pquotino)), C1, 0, TYP_INODE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
 	{ NULL }
db/symlink.c +1 -1
index 9f3d0b9..a4f420f 100644
@@ -69,7 +69,7 @@ const struct field	symlink_crc_flds[] = {
 	{ "magic", FLDT_UINT32X, OI(OFF(magic)), C1, 0, TYP_NONE },
 	{ "offset", FLDT_UINT32D, OI(OFF(offset)), C1, 0, TYP_NONE },
 	{ "bytes", FLDT_UINT32D, OI(OFF(bytes)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_INO, OI(OFF(owner)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_DFSBNO, OI(OFF(blkno)), C1, 0, TYP_BMAPBTD },
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#29 lore raw

PATCH v4 23/37 db: verify and calculate inode CRCs

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When we set the current IO cursor to point at an inode, verify that the inode CRC is intact. And prior to writing such an IO cursor, calculate the inode CRC.

---
 db/inode.c             | 2 ++
 db/io.c                | 4 ++++
 db/io.h                | 6 +++++-
 include/libxfs.h       | 4 ++++
 libxfs/xfs_inode_buf.c | 8 ++++----
 5 files changed, 19 insertions(+), 5 deletions(-)
db/inode.c +2 -0
index ec533ee..4090855 100644
@@ -655,6 +655,8 @@ set_cur_inode(
 		blkbb, DB_RING_IGN, NULL);
 	off_cur(offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize);
 	dip = iocur_top->data;
+	iocur_top->ino_crc_ok = libxfs_dinode_verify(mp, ino, dip);
+	iocur_top->ino_buf = 1;
 	iocur_top->ino = ino;
 	iocur_top->mode = be16_to_cpu(dip->di_mode);
 	if ((iocur_top->mode & S_IFMT) == S_IFDIR)
db/io.c +4 -0
index 2d1cc56..6e3282e 100644
@@ -464,6 +464,9 @@ write_cur(void)
 		return;
 	}
 
+	if (iocur_top->ino_buf)
+		libxfs_dinode_calc_crc(mp, iocur_top->data);
+
 	if (iocur_top->bbmap)
 		write_cur_bbs();
 	else
@@ -536,6 +539,7 @@ set_cur(
 	iocur_top->ino = ino;
 	iocur_top->dirino = dirino;
 	iocur_top->mode = mode;
+	iocur_top->ino_buf = 0;
 
 	/* store location in ring */
 	if (ring_flag)
db/io.h +5 -1
index d647284..1f8270d 100644
@@ -38,6 +38,8 @@ typedef struct iocur {
 	const struct typ	*typ;	/* type of "data" */
 	bbmap_t			*bbmap;	/* map daddr if fragmented */
 	struct xfs_buf		*bp;	/* underlying buffer */
+	int			ino_crc_ok:1;
+	int			ino_buf:1;
 } iocur_t;
 
 #define DB_RING_ADD 1                   /* add to ring on set_cur */
@@ -62,5 +64,7 @@ extern void     ring_add(void);
 static inline bool
 iocur_crc_valid()
 {
-	return (iocur_top->bp && iocur_top->bp->b_error != EFSCORRUPTED);
+	return (iocur_top->bp &&
+		iocur_top->bp->b_error != EFSCORRUPTED &&
+		(!iocur_top->ino_buf || iocur_top->ino_crc_ok));
 }
include/libxfs.h +4 -0
index b097bd2..cbb5757 100644
@@ -748,6 +748,10 @@ void	xfs_dinode_from_disk(struct xfs_icdinode *,
 #define libxfs_idata_realloc		xfs_idata_realloc
 #define libxfs_idestroy_fork		xfs_idestroy_fork
 
+#define libxfs_dinode_verify		xfs_dinode_verify
+bool xfs_dinode_verify(struct xfs_mount *mp, xfs_ino_t ino,
+		       struct xfs_dinode *dip);
+
 /* xfs_sb.h */
 #define libxfs_mod_sb			xfs_mod_sb
 #define libxfs_sb_from_disk		xfs_sb_from_disk
libxfs/xfs_inode_buf.c +4 -4
index b796556..728ef71 100644
@@ -276,10 +276,10 @@ xfs_dinode_to_disk(
 	}
 }
 
-static bool
+bool
 xfs_dinode_verify(
 	struct xfs_mount	*mp,
-	struct xfs_inode	*ip,
+	xfs_ino_t		ino,
 	struct xfs_dinode	*dip)
 {
 	if (dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC))
@@ -294,7 +294,7 @@ xfs_dinode_verify(
 	if (!xfs_verify_cksum((char *)dip, mp->m_sb.sb_inodesize,
 			      offsetof(struct xfs_dinode, di_crc)))
 		return false;
-	if (be64_to_cpu(dip->di_ino) != ip->i_ino)
+	if (be64_to_cpu(dip->di_ino) != ino)
 		return false;
 	if (!uuid_equal(&dip->di_uuid, &mp->m_sb.sb_uuid))
 		return false;
@@ -346,7 +346,7 @@ xfs_iread(
 		return error;
 
 	/* even unallocated inodes are verified */
-	if (!xfs_dinode_verify(mp, ip, dip)) {
+	if (!xfs_dinode_verify(mp, ip->i_ino, dip)) {
 		xfs_alert(mp, "%s: validation failed for inode %lld failed",
 				__func__, ip->i_ino);
 
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#30 lore raw

PATCH v4 24/37 db: verify and calculate dquot CRCs

In reply to Dave Chinner

When we set the current Io cursor to point at a dquot block, verify that the dquot CRC is intact. And prior to writing such an IO cursor, calculate the dquot CRC.

---
 db/io.c | 5 ++++-
 db/io.h | 1 +
 2 files changed, 5 insertions(+), 1 deletion(-)
db/io.c +4 -1
index 6e3282e..123214d 100644
@@ -466,7 +466,9 @@ write_cur(void)
 
 	if (iocur_top->ino_buf)
 		libxfs_dinode_calc_crc(mp, iocur_top->data);
-
+	if (iocur_top->dquot_buf)
+		xfs_update_cksum(iocur_top->data, sizeof(struct xfs_dqblk),
+				 XFS_DQUOT_CRC_OFF);
 	if (iocur_top->bbmap)
 		write_cur_bbs();
 	else
@@ -540,6 +542,7 @@ set_cur(
 	iocur_top->dirino = dirino;
 	iocur_top->mode = mode;
 	iocur_top->ino_buf = 0;
+	iocur_top->dquot_buf = 0;
 
 	/* store location in ring */
 	if (ring_flag)
db/io.h +1 -0
index 1f8270d..4f24c83 100644
@@ -40,6 +40,7 @@ typedef struct iocur {
 	struct xfs_buf		*bp;	/* underlying buffer */
 	int			ino_crc_ok:1;
 	int			ino_buf:1;
+	int			dquot_buf:1;
 } iocur_t;
 
 #define DB_RING_ADD 1                   /* add to ring on set_cur */
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#31 lore raw

PATCH v4 25/37 db: add a special directory buffer verifier

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Because we only have a single directory type that is used for all the different buffer types, we need to provide a special verifier for the read code. That verifier needs to know all the directory types and when it find one it knows about, switch to the correct verifier and call it.

We already do this for certain readahead cases in the directory code, so there is precedence for this. If we don't find a magic number we recognise, the verifier fails...

---
 db/dir2.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 db/dir2.h |  2 ++
 db/type.c |  3 ++-
 3 files changed, 65 insertions(+), 1 deletion(-)
db/dir2.c +61 -0
index 2ec64e0..5a10955 100644
@@ -24,6 +24,7 @@
 #include "field.h"
 #include "dir2.h"
 #include "init.h"
+#include "output.h"
 
 static int	dir2_block_hdr_count(void *obj, int startoff);
 static int	dir2_block_leaf_count(void *obj, int startoff);
@@ -975,3 +976,63 @@ const field_t	da3_node_hdr_flds[] = {
 	{ "pad", FLDT_UINT32D, OI(H3OFF(__pad32)), C1, 0, TYP_NONE },
 	{ NULL }
 };
+
+/*
+ * Special read verifier for directory buffers. detect the magic number
+ * appropriately and set the correct verifier and call it.
+ */
+static void
+xfs_dir3_db_read_verify(
+	struct xfs_buf		*bp)
+{
+	__be32			magic32;
+	__be16			magic16;
+
+	magic32 = *(__be32 *)bp->b_addr;
+	magic16 = ((struct xfs_da_blkinfo *)bp->b_addr)->magic;
+
+	switch (magic32) {
+	case cpu_to_be32(XFS_DIR3_BLOCK_MAGIC):
+		bp->b_ops = &xfs_dir3_block_buf_ops;
+		goto verify;
+	case cpu_to_be32(XFS_DIR3_DATA_MAGIC):
+		bp->b_ops = &xfs_dir3_data_buf_ops;
+		goto verify;
+	case cpu_to_be32(XFS_DIR3_FREE_MAGIC):
+		bp->b_ops = &xfs_dir3_free_buf_ops;
+		goto verify;
+	default:
+		break;
+	}
+
+	switch (magic16) {
+	case cpu_to_be16(XFS_DIR3_LEAF1_MAGIC):
+		bp->b_ops = &xfs_dir3_leaf1_buf_ops;
+		break;
+	case cpu_to_be16(XFS_DIR3_LEAFN_MAGIC):
+		bp->b_ops = &xfs_dir3_leafn_buf_ops;
+		break;
+	case cpu_to_be16(XFS_DA3_NODE_MAGIC):
+		bp->b_ops = &xfs_da3_node_buf_ops;
+		break;
+	default:
+		dbprintf(_("Unknown directory buffer type!\n"));
+		xfs_buf_ioerror(bp, EFSCORRUPTED);
+		return;
+	}
+verify:
+	bp->b_ops->verify_read(bp);
+}
+
+static void
+xfs_dir3_db_write_verify(
+	struct xfs_buf		*bp)
+{
+	dbprintf(_("Writing unknown directory buffer type!\n"));
+	xfs_buf_ioerror(bp, EFSCORRUPTED);
+}
+
+const struct xfs_buf_ops xfs_dir3_db_buf_ops = {
+	.verify_read = xfs_dir3_db_read_verify,
+	.verify_write = xfs_dir3_db_write_verify,
+};
db/dir2.h +2 -0
index b3651d5..5054493 100644
@@ -60,3 +60,5 @@ static inline xfs_dir2_inou_t *xfs_dir2_sf_inumberp(xfs_dir2_sf_entry_t *sfep)
 
 extern int	dir2_data_union_size(void *obj, int startoff, int idx);
 extern int	dir2_size(void *obj, int startoff, int idx);
+
+extern const struct xfs_buf_ops xfs_dir3_db_buf_ops;
db/type.c +2 -1
index b3f3d87..2c3431e 100644
@@ -87,7 +87,8 @@ static const typ_t	__typtab_crc[] = {
 	{ TYP_CNTBT, "cntbt", handle_struct, cntbt_crc_hfld,
 		&xfs_allocbt_buf_ops },
 	{ TYP_DATA, "data", handle_block, NULL, NULL },
-	{ TYP_DIR2, "dir3", handle_struct, dir3_hfld, NULL },
+	{ TYP_DIR2, "dir3", handle_struct, dir3_hfld,
+		&xfs_dir3_db_buf_ops },
 	{ TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld,
 		&xfs_dquot_buf_ops },
 	{ TYP_INOBT, "inobt", handle_struct, inobt_crc_hfld,
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#32 lore raw

PATCH v4 26/37 db: add a special attribute buffer verifier

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Because we only have a single attribute type that is used for all the attribute buffer types, we need to provide a special verifier for the read code. That verifier needs to know all the attribute types and when it find one it knows about, switch to the correct verifier and call it.

---
 db/attr.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 db/attr.h |  2 ++
 db/type.c |  3 ++-
 3 files changed, 55 insertions(+), 1 deletion(-)
db/attr.c +51 -0
index cd95a0a..359af7b 100644
@@ -25,6 +25,7 @@
 #include "attr.h"
 #include "io.h"
 #include "init.h"
+#include "output.h"
 
 static int	attr_leaf_entries_count(void *obj, int startoff);
 static int	attr_leaf_hdr_count(void *obj, int startoff);
@@ -522,3 +523,53 @@ const field_t	attr3_leaf_hdr_flds[] = {
 	{ NULL }
 };
 
+/*
+ * Special read verifier for attribute buffers. detect the magic number
+ * appropriately and set the correct verifier and call it.
+ */
+static void
+xfs_attr3_db_read_verify(
+	struct xfs_buf		*bp)
+{
+	__be32			magic32;
+	__be16			magic16;
+
+	magic32 = *(__be32 *)bp->b_addr;
+	magic16 = ((struct xfs_da_blkinfo *)bp->b_addr)->magic;
+
+	switch (magic16) {
+	case cpu_to_be16(XFS_ATTR3_LEAF_MAGIC):
+		bp->b_ops = &xfs_attr3_leaf_buf_ops;
+		goto verify;
+	case cpu_to_be16(XFS_DA3_NODE_MAGIC):
+		bp->b_ops = &xfs_da3_node_buf_ops;
+		goto verify;
+	default:
+		break;
+	}
+
+	switch (magic32) {
+	case cpu_to_be32(XFS_ATTR3_RMT_MAGIC):
+		bp->b_ops = &xfs_attr3_rmt_buf_ops;
+		break;
+	default:
+		dbprintf(_("Unknown attribute buffer type!\n"));
+		xfs_buf_ioerror(bp, EFSCORRUPTED);
+		return;
+	}
+verify:
+	bp->b_ops->verify_read(bp);
+}
+
+static void
+xfs_attr3_db_write_verify(
+	struct xfs_buf		*bp)
+{
+	dbprintf(_("Writing unknown attribute buffer type!\n"));
+	xfs_buf_ioerror(bp, EFSCORRUPTED);
+}
+
+const struct xfs_buf_ops xfs_attr3_db_buf_ops = {
+	.verify_read = xfs_attr3_db_read_verify,
+	.verify_write = xfs_attr3_db_write_verify,
+};
db/attr.h +2 -0
index 3065372..bc3431f 100644
@@ -33,3 +33,5 @@ extern const field_t	attr3_node_hdr_flds[];
 
 extern int	attr_leaf_name_size(void *obj, int startoff, int idx);
 extern int	attr_size(void *obj, int startoff, int idx);
+
+extern const struct xfs_buf_ops xfs_attr3_db_buf_ops;
db/type.c +2 -1
index 2c3431e..04d0d56 100644
@@ -77,7 +77,8 @@ static const typ_t	__typtab_crc[] = {
 	{ TYP_AGF, "agf", handle_struct, agf_hfld, &xfs_agf_buf_ops },
 	{ TYP_AGFL, "agfl", handle_struct, agfl_crc_hfld, &xfs_agfl_buf_ops },
 	{ TYP_AGI, "agi", handle_struct, agi_hfld, &xfs_agfl_buf_ops },
-	{ TYP_ATTR, "attr3", handle_struct, attr3_hfld, NULL },
+	{ TYP_ATTR, "attr3", handle_struct, attr3_hfld,
+		&xfs_attr3_db_buf_ops },
 	{ TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_crc_hfld,
 		&xfs_bmbt_buf_ops },
 	{ TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_crc_hfld,
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#33 lore raw

PATCH v4 27/37 db: re-enable write support for v5 filesystems.

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

As we can now verify and recalculate CRCs on IO, we can modify the on-disk structures without corrupting the filesyste, This makes it safe to turn write support on for v5 filesystems for the first time.

---
 db/init.c | 15 ---------------
 1 file changed, 15 deletions(-)
db/init.c +0 -15
index 48b8dd1..838ea3c 100644
@@ -143,21 +143,6 @@ init(
 			exit(EXIT_FAILURE);
 	}
 
-	/*
-	 * Don't allow modifications to CRC enabled filesystems until we support
-	 * CRC recalculation in the IO path. Unless, of course, the user is in
-	 * the process of hitting us with a big hammer.
-	 */
-	if (XFS_SB_VERSION_NUM(sbp) >= XFS_SB_VERSION_5 &&
-	    !(x.isreadonly & LIBXFS_ISREADONLY)) {
-		fprintf(stderr, 
-	_("%s: modifications to %s are not supported in thi version.\n"
-	"Use \"-r\" to run %s in read-only mode on this filesystem .\n"),
-			progname, fsdevice, progname);
-		if (!force)
-			exit(EXIT_FAILURE);
-	}
-
 	mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
 			  LIBXFS_MOUNT_DEBUGGER);
 	if (!mp) {
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#34 lore raw

PATCH v4 28/37 xfs_db: avoid libxfs buffer lookup warnings

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When we mount the filesystem inside xfs_db, libxfs is tasked with reading some information from disk, such as root inodes. Because libxfs does this inode reading, it uses inode cluster buffers to read the inodes. xfs_db, OTOH, just uses FSB sized buffers to read inodes, and hence xfs_db throws a warning when reading the root inode block like so:

$ sudo xfs_db -c "sb 0" -c "p rootino" -c "inode 32" /dev/vda
Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled!
Use of these features is at your own risk!
rootino = 32
7f59f20e6740: Badness in key lookup (length)
bp=(bno 0x20, len 8192 bytes) key=(bno 0x20, len 1024 bytes)
$

There is another way this can happen, and that is dumping raw data from disk using either the "fsb NNN" or "daddr MMM" commands to dump untyped information. This is always read in sector or filesystem block units, and so will cause similar badness warnings.

To avoid this problem when reading inodes, teach xfs_db to read inode clusters rather individual filesystem blocks when asked to read an inode.

---
 db/inode.c | 33 +++++++++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
db/inode.c +31 -2
index 4090855..24170ba 100644
@@ -623,6 +623,14 @@ inode_u_symlink_count(
 		(int)be64_to_cpu(dip->di_size) : 0;
 }
 
+/*
+ * We are now using libxfs for our IO backend, so we should always try to use
+ * inode cluster buffers rather than filesystem block sized buffers for reading
+ * inodes. This means that we always use the same buffers as libxfs operations
+ * does, and that avoids buffer cache issues caused by overlapping buffers. This
+ * can be seen clearly when trying to read the root inode. Much of this logic is
+ * similar to libxfs_imap().
+ */
 void
 set_cur_inode(
 	xfs_ino_t	ino)
@@ -632,6 +640,9 @@ set_cur_inode(
 	xfs_agnumber_t	agno;
 	xfs_dinode_t	*dip;
 	int		offset;
+	int		numblks = blkbb;
+	xfs_agblock_t	cluster_agbno;
+
 
 	agno = XFS_INO_TO_AGNO(mp, ino);
 	agino = XFS_INO_TO_AGINO(mp, ino);
@@ -644,6 +655,24 @@ set_cur_inode(
 		return;
 	}
 	cur_agno = agno;
+
+	if (mp->m_inode_cluster_size > mp->m_sb.sb_blocksize &&
+	    mp->m_inoalign_mask) {
+		xfs_agblock_t	chunk_agbno;
+		xfs_agblock_t	offset_agbno;
+		int		blks_per_cluster;
+
+		blks_per_cluster = mp->m_inode_cluster_size >>
+							mp->m_sb.sb_blocklog;
+		offset_agbno = agbno & mp->m_inoalign_mask;
+		chunk_agbno = agbno - offset_agbno;
+		cluster_agbno = chunk_agbno +
+			((offset_agbno / blks_per_cluster) * blks_per_cluster);
+		offset += ((agbno - cluster_agbno) * mp->m_sb.sb_inopblock);
+		numblks = XFS_FSB_TO_BB(mp, blks_per_cluster);
+	} else
+		cluster_agbno = agbno;
+
 	/*
 	 * First set_cur to the block with the inode
 	 * then use off_cur to get the right part of the buffer.
@@ -651,8 +680,8 @@ set_cur_inode(
 	ASSERT(typtab[TYP_INODE].typnm == TYP_INODE);
 
 	/* ingore ring update here, do it explicitly below */
-	set_cur(&typtab[TYP_INODE], XFS_AGB_TO_DADDR(mp, agno, agbno),
-		blkbb, DB_RING_IGN, NULL);
+	set_cur(&typtab[TYP_INODE], XFS_AGB_TO_DADDR(mp, agno, cluster_agbno),
+		numblks, DB_RING_IGN, NULL);
 	off_cur(offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize);
 	dip = iocur_top->data;
 	iocur_top->ino_crc_ok = libxfs_dinode_verify(mp, ino, dip);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#35 lore raw

Re: [PATCH 28/37] xfs_db: avoid libxfs buffer lookup warnings

Looks like I messed up the patch subject. It should be:

"xfs_db: use inode cluster buffers for inode IO"

Quoted 104 lines · Dave Chinner

From: Dave Chinner <[email protected]>

When we mount the filesystem inside xfs_db, libxfs is tasked with reading some information from disk, such as root inodes. Because libxfs does this inode reading, it uses inode cluster buffers to read the inodes. xfs_db, OTOH, just uses FSB sized buffers to read inodes, and hence xfs_db throws a warning when reading the root inode block like so:

$ sudo xfs_db -c "sb 0" -c "p rootino" -c "inode 32" /dev/vda
Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled!
Use of these features is at your own risk!
rootino = 32
7f59f20e6740: Badness in key lookup (length)
bp=(bno 0x20, len 8192 bytes) key=(bno 0x20, len 1024 bytes)
$

There is another way this can happen, and that is dumping raw data from disk using either the "fsb NNN" or "daddr MMM" commands to dump untyped information. This is always read in sector or filesystem block units, and so will cause similar badness warnings.

To avoid this problem when reading inodes, teach xfs_db to read inode clusters rather individual filesystem blocks when asked to read an inode.

---
 db/inode.c | 33 +++++++++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)
db/inode.c +31 -2
index 4090855..24170ba 100644
@@ -623,6 +623,14 @@ inode_u_symlink_count(
 		(int)be64_to_cpu(dip->di_size) : 0;
 }
 
+/*
+ * We are now using libxfs for our IO backend, so we should always try to use
+ * inode cluster buffers rather than filesystem block sized buffers for reading
+ * inodes. This means that we always use the same buffers as libxfs operations
+ * does, and that avoids buffer cache issues caused by overlapping buffers. This
+ * can be seen clearly when trying to read the root inode. Much of this logic is
+ * similar to libxfs_imap().
+ */
 void
 set_cur_inode(
 	xfs_ino_t	ino)
@@ -632,6 +640,9 @@ set_cur_inode(
 	xfs_agnumber_t	agno;
 	xfs_dinode_t	*dip;
 	int		offset;
+	int		numblks = blkbb;
+	xfs_agblock_t	cluster_agbno;
+
 
 	agno = XFS_INO_TO_AGNO(mp, ino);
 	agino = XFS_INO_TO_AGINO(mp, ino);
@@ -644,6 +655,24 @@ set_cur_inode(
 		return;
 	}
 	cur_agno = agno;
+
+	if (mp->m_inode_cluster_size > mp->m_sb.sb_blocksize &&
+	    mp->m_inoalign_mask) {
+		xfs_agblock_t	chunk_agbno;
+		xfs_agblock_t	offset_agbno;
+		int		blks_per_cluster;
+
+		blks_per_cluster = mp->m_inode_cluster_size >>
+							mp->m_sb.sb_blocklog;
+		offset_agbno = agbno & mp->m_inoalign_mask;
+		chunk_agbno = agbno - offset_agbno;
+		cluster_agbno = chunk_agbno +
+			((offset_agbno / blks_per_cluster) * blks_per_cluster);
+		offset += ((agbno - cluster_agbno) * mp->m_sb.sb_inopblock);
+		numblks = XFS_FSB_TO_BB(mp, blks_per_cluster);
+	} else
+		cluster_agbno = agbno;
+
 	/*
 	 * First set_cur to the block with the inode
 	 * then use off_cur to get the right part of the buffer.
@@ -651,8 +680,8 @@ set_cur_inode(
 	ASSERT(typtab[TYP_INODE].typnm == TYP_INODE);
 
 	/* ingore ring update here, do it explicitly below */
-	set_cur(&typtab[TYP_INODE], XFS_AGB_TO_DADDR(mp, agno, agbno),
-		blkbb, DB_RING_IGN, NULL);
+	set_cur(&typtab[TYP_INODE], XFS_AGB_TO_DADDR(mp, agno, cluster_agbno),
+		numblks, DB_RING_IGN, NULL);
 	off_cur(offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize);
 	dip = iocur_top->data;
 	iocur_top->ino_crc_ok = libxfs_dinode_verify(mp, ino, dip);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#37 lore raw

PATCH v4 29/37 xfs_db: avoid libxfs buffer lookup warnings

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

xfs_db is unique in the way it can read the same blocks with different lengths from disk, so we really need a way to avoid having duplicate buffers in the cache. To handle this in a generic way, introduce a "purge on compare failure" feature to libxfs.

What this feature does is instead of throwing a warning when a buffer miscompare occurs (e.g. due to a length mismatch), it purges the buffer that is in cache from the cache. We can do this safely in the context of xfs_db because it always writes back changes made to buffers before it releases the reference to the buffer. Hence we can purge buffers directly from the lookup code without having to worry about whether they are dirty or not.

Doing this purge on miscompare operation avoids the problem that libxfs is currently warning about, and hence if the feature flag is set then we don't need to warn about miscompares any more. Hence the whole problem goes away entirely for xfs_db, without affecting any of the other users of libxfs based IO.

---
 db/init.c           |  1 +
 include/cache.h     | 22 +++++++++++++-
 include/libxfs.h    |  2 ++
 libxfs/cache.c      | 83 +++++++++++++++++++++++++++++++++++++++--------------
 libxfs/init.c       |  3 +-
 libxfs/rdwr.c       | 28 ++++++++++--------
 repair/xfs_repair.c |  2 +-
 7 files changed, 105 insertions(+), 36 deletions(-)
db/init.c +1 -0
index 838ea3c..4bc048a 100644
@@ -109,6 +109,7 @@ init(
 	else
 		x.dname = fsdevice;
 
+	x.bcache_flags = CACHE_MISCOMPARE_PURGE;
 	if (!libxfs_init(&x)) {
 		fputs(_("\nfatal error -- couldn't initialize XFS library\n"),
 			stderr);
include/cache.h +21 -1
index 0c0a1c5..c5757d0 100644
@@ -18,6 +18,25 @@
 #ifndef __CACHE_H__
 #define __CACHE_H__
 
+/*
+ * initialisation flags
+ */
+/*
+ * xfs_db always writes changes immediately, and so we need to purge buffers
+ * when we get a buffer lookup mismatch due to readin the same block with a
+ * different buffer configuration.
+ */
+#define CACHE_MISCOMPARE_PURGE	(1 << 0)
+
+/*
+ * cache object campare return values
+ */
+enum {
+	CACHE_HIT,
+	CACHE_MISS,
+	CACHE_PURGE,
+};
+
 #define	HASH_CACHE_RATIO	8
 
 /*
@@ -82,6 +101,7 @@ struct cache_node {
 };
 
 struct cache {
+	int			c_flags;	/* behavioural flags */
 	unsigned int		c_maxcount;	/* max cache nodes */
 	unsigned int		c_count;	/* count of nodes */
 	pthread_mutex_t		c_mutex;	/* node count mutex */
@@ -99,7 +119,7 @@ struct cache {
 	unsigned int 		c_max;		/* max nodes ever used */
 };
 
-struct cache *cache_init(unsigned int, struct cache_operations *);
+struct cache *cache_init(int, unsigned int, struct cache_operations *);
 void cache_destroy(struct cache *);
 void cache_walk(struct cache *, cache_walk_t);
 void cache_purge(struct cache *);
include/libxfs.h +2 -0
index cbb5757..40a950e 100644
@@ -110,6 +110,8 @@ typedef struct {
 	int             dfd;            /* data subvolume file descriptor */
 	int             logfd;          /* log subvolume file descriptor */
 	int             rtfd;           /* realtime subvolume file descriptor */
+	int		icache_flags;	/* cache init flags */
+	int		bcache_flags;	/* cache init flags */
 } libxfs_init_t;
 
 #define LIBXFS_EXIT_ON_FAILURE	0x0001	/* exit the program if a call fails */
libxfs/cache.c +62 -21

140 lines not on this page · raw message

libxfs/init.c +2 -1
index abe3316..5ba4fe6 100644
@@ -334,7 +334,8 @@ libxfs_init(libxfs_init_t *a)
 		chdir(curdir);
 	if (!libxfs_bhash_size)
 		libxfs_bhash_size = LIBXFS_BHASHSIZE(sbp);
-	libxfs_bcache = cache_init(libxfs_bhash_size, &libxfs_bcache_operations);
+	libxfs_bcache = cache_init(a->bcache_flags, libxfs_bhash_size,
+				   &libxfs_bcache_operations);
 	use_xfs_buf_lock = a->usebuflock;
 	manage_zones(0);
 	rval = 1;
libxfs/rdwr.c +16 -12
index 7eaea0a..0aa2eba 100644
@@ -323,20 +323,24 @@ libxfs_bcompare(struct cache_node *node, cache_key_t key)
 	struct xfs_buf	*bp = (struct xfs_buf *)node;
 	struct xfs_bufkey *bkey = (struct xfs_bufkey *)key;
 
-#ifdef IO_BCOMPARE_CHECK
 	if (bp->b_target->dev == bkey->buftarg->dev &&
-	    bp->b_bn == bkey->blkno &&
-	    bp->b_bcount != BBTOB(bkey->bblen))
-		fprintf(stderr, "%lx: Badness in key lookup (length)\n"
-			"bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n",
-			pthread_self(),
-			(unsigned long long)bp->b_bn, (int)bp->b_bcount,
-			(unsigned long long)bkey->blkno, BBTOB(bkey->bblen));
+	    bp->b_bn == bkey->blkno) {
+		if (bp->b_bcount == BBTOB(bkey->bblen))
+			return CACHE_HIT;
+#ifdef IO_BCOMPARE_CHECK
+		if (!(libxfs_bcache->c_flags & CACHE_MISCOMPARE_PURGE)) {
+			fprintf(stderr,
+	"%lx: Badness in key lookup (length)\n"
+	"bp=(bno 0x%llx, len %u bytes) key=(bno 0x%llx, len %u bytes)\n",
+				pthread_self(),
+				(unsigned long long)bp->b_bn, (int)bp->b_bcount,
+				(unsigned long long)bkey->blkno,
+				BBTOB(bkey->bblen));
+		}
 #endif
-
-	return (bp->b_target->dev == bkey->buftarg->dev &&
-		bp->b_bn == bkey->blkno &&
-		bp->b_bcount == BBTOB(bkey->bblen));
+		return CACHE_PURGE;
+	}
+	return CACHE_MISS;
 }
 
 void
repair/xfs_repair.c +1 -1
index 214b7fa..77a040e 100644
@@ -706,7 +706,7 @@ main(int argc, char **argv)
 			do_log(_("        - block cache size set to %d entries\n"),
 				libxfs_bhash_size * HASH_CACHE_RATIO);
 
-		libxfs_bcache = cache_init(libxfs_bhash_size,
+		libxfs_bcache = cache_init(0, libxfs_bhash_size,
 						&libxfs_bcache_operations);
 	}
 
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#39 lore raw

PATCH v4 30/37 libxfs: work around do_div() not handling 32 bit numerators

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

The libxfs dquot buffer code uses do_div() with a 32 bit numerator. This gives incorrect results as do_div() passes the numerator by reference as a pointer to a 64 bit value. Hence it does the division using 32 bits of garbage gives the wrong result.

As per Christoph's suggestion, we can kill the usage of do_div() here completely and just do the division directly, both in userspace and kernel space.

---
 libxfs/xfs_dquot_buf.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
libxfs/xfs_dquot_buf.c +1 -6
index 620d9d3..6bbb0ff 100644
@@ -23,13 +23,8 @@ xfs_calc_dquots_per_chunk(
 	struct xfs_mount	*mp,
 	unsigned int		nbblks)	/* basic block units */
 {
-	unsigned int	ndquots;
-
 	ASSERT(nbblks > 0);
-	ndquots = BBTOB(nbblks);
-	do_div(ndquots, sizeof(xfs_dqblk_t));
-
-	return ndquots;
+	return BBTOB(nbblks) / sizeof(xfs_dqblk_t);
 }
 
 /*
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v4 30/37
#40 lore raw

Re: [PATCH 30/37] libxfs: work around do_div() not handling 32 bit numerators

Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The libxfs dquot buffer code uses do_div() with a 32 bit numerator. This gives incorrect results as do_div() passes the numerator by reference as a pointer to a 64 bit value. Hence it does the division using 32 bits of garbage gives the wrong result.

Quoted 5 lines

As per Christoph's suggestion, we can kill the usage of do_div() here completely and just do the division directly, both in userspace and kernel space.

Looks good,

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#41 lore raw

PATCH v4 31/37 db: Unwind the IO stack on exit

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

xfs_db can build up a large IO stack by the time it has run to completion. If we don't unwind this IO stack before we shut down the libxfs caches, metadump and other db programs will exit with unreleased buffers and emit warnings like:

cache_purge: shake on cache 0x69e4f0 left 7 nodes!?

Hence we need to unwind the iostack as we shut down.

---
 db/init.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
db/init.c +6 -3
index 4bc048a..8ad21da 100644
@@ -183,9 +183,11 @@ main(
 	int	c, i, done = 0;
 	char	*input;
 	char	**v;
+	int	start_iocur_sp;
 
 	pushfile(stdin);
 	init(argc, argv);
+	start_iocur_sp = iocur_sp;
 
 	for (i = 0; !done && i < ncmdline; i++) {
 		v = breakline(cmdline[i], &c);
@@ -209,10 +211,11 @@ main(
 
 close_devices:
 	/*
-	 * make sure that we pop the last buffer context we held so that the
-	 * buffer is released before purge the caches during unmount.
+	 * Make sure that we pop the all the buffer contexts we hold so that
+	 * they are released before we purge the caches during unmount.
 	 */
-	pop_cur();
+	while (iocur_sp > start_iocur_sp)
+		pop_cur();
 	libxfs_umount(mp);
 	if (x.ddev)
 		libxfs_device_close(x.ddev);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v4 31/37
#42 lore raw

Re: [PATCH 31/37] db: Unwind the IO stack on exit

Quoted 10 lines · Dave Chinner

From: Dave Chinner <[email protected]>

xfs_db can build up a large IO stack by the time it has run to completion. If we don't unwind this IO stack before we shut down the libxfs caches, metadump and other db programs will exit with unreleased buffers and emit warnings like:

cache_purge: shake on cache 0x69e4f0 left 7 nodes!?

Hence we need to unwind the iostack as we shut down.

Seems like this should be folded into patch 20, as that adds the first pop, but it seems to fail goind all the way.

Otherwise looks good,

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#43 lore raw

PATCH v4 32/37 db: enable metadump on CRC filesystems

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Now that we can calculate CRCs through xfs_db, we can add support for recalculating CRCs on obfuscated metadump images. This simply requires us to call the write verifier manually before writing the buffer to the metadump image.

We don't need to do anything special to mdrestore, as the metadata blocks it reads from the image file will already have all the correct CRCs in them. Hence it can be mostly oblivious to the fact that the filesystem it is restoring contains CRCs.

---
 db/metadump.c             | 24 +++++++++++++++++++-----
 mdrestore/xfs_mdrestore.c |  3 ---
 2 files changed, 19 insertions(+), 8 deletions(-)
db/metadump.c +19 -5
index ac6a4d6..117dc42 100644
@@ -172,6 +172,22 @@ write_buf(
 	__int64_t	off;
 	int		i;
 
+	/*
+	 * Run the write verifier to recalculate the buffer CRCs and check
+	 * we are writing something valid to disk
+	 */
+	if (buf->bp && buf->bp->b_ops) {
+		buf->bp->b_error = 0;
+		buf->bp->b_ops->verify_write(buf->bp);
+		if (buf->bp->b_error) {
+			fprintf(stderr,
+	_("%s: write verifer failed on bno 0x%llx/0x%x\n"),
+				__func__, (long long)buf->bp->b_bn,
+				buf->bp->b_bcount);
+			return buf->bp->b_error;
+		}
+	}
+
 	for (i = 0, off = buf->bb, data = buf->data;
 			i < buf->blen;
 			i++, off++, data += BBSIZE) {
@@ -1727,6 +1743,9 @@ copy_inode_chunk(
 
 		if (!process_inode(agno, agino + i, dip))
 			goto pop_out;
+
+		/* calculate the new CRC for the inode */
+		xfs_dinode_calc_crc(mp, dip);
 	}
 skip_processing:
 	if (!write_buf(iocur_top))
@@ -2053,11 +2072,6 @@ metadump_f(
 		return 0;
 	}
 
-	if (xfs_sb_version_hascrc(&mp->m_sb) && dont_obfuscate == 0) {
-		print_warning("Can't obfuscate CRC enabled filesystems yet.");
-		return 0;
-	}
-
 	metablock = (xfs_metablock_t *)calloc(BBSIZE + 1, BBSIZE);
 	if (metablock == NULL) {
 		print_warning("memory allocation failure");
mdrestore/xfs_mdrestore.c +0 -3
index fe61766..e57bdb2 100644
@@ -109,9 +109,6 @@ perform_restore(
 	if (sb.sb_magicnum != XFS_SB_MAGIC)
 		fatal("bad magic number for primary superblock\n");
 
-	if (xfs_sb_version_hascrc(&sb))
-		fatal("Can't restore CRC enabled filesystems yet.\n");
-
 	((xfs_dsb_t*)block_buffer)->sb_inprogress = 1;
 
 	if (is_target_file)  {
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#44 lore raw

PATCH v4 33/37 xfs: support larger inode clusters on v5 filesystems

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

To allow the kernel to use larger inode clusters than the standard 8192 bytes, we need to set the inode alignment fields appropriately so that the kernel is consistent in it's inode to buffer mappings. We set the alignment to allow a constant 32 inodes per cluster, instead of a fixed 8k cluster size.

---
 include/libxfs.h |  2 +-
 mkfs/xfs_mkfs.c  |  5 ++++-
 repair/sb.c      | 41 ++++++++++++++++++++++++++++-------------
 3 files changed, 33 insertions(+), 15 deletions(-)
include/libxfs.h +1 -1
index 40a950e..4bf331c 100644
@@ -181,7 +181,7 @@ typedef struct xfs_mount {
 	__uint8_t		m_sectbb_log;	/* sectorlog - BBSHIFT */
 	__uint8_t		m_agno_log;	/* log #ag's */
 	__uint8_t		m_agino_log;	/* #bits for agino in inum */
-	__uint16_t		m_inode_cluster_size;/* min inode buf size */
+	uint			m_inode_cluster_size;/* min inode buf size */
 	uint			m_blockmask;	/* sb_blocksize-1 */
 	uint			m_blockwsize;	/* sb_blocksize in words */
 	uint			m_blockwmask;	/* blockwsize-1 */
mkfs/xfs_mkfs.c +4 -1
index 3a032c0..d82128c 100644
@@ -2532,7 +2532,10 @@ _("size %s specified for log subvolume is too large, maximum is %lld blocks\n"),
 	} else
 		sbp->sb_logsunit = 0;
 	if (iaflag) {
-		sbp->sb_inoalignmt = XFS_INODE_BIG_CLUSTER_SIZE >> blocklog;
+		int	cluster_size = XFS_INODE_BIG_CLUSTER_SIZE;
+		if (crcs_enabled)
+			cluster_size *= isize / XFS_DINODE_MIN_SIZE;
+		sbp->sb_inoalignmt = cluster_size >> blocklog;
 		iaflag = sbp->sb_inoalignmt != 0;
 	} else
 		sbp->sb_inoalignmt = 0;
repair/sb.c +28 -13

64 lines not on this page · raw message

Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#45 lore raw

PATCH v4 34/37 xfsprogs: kill experimental warnings for v5 filesystems

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

With xfsprogs now being close to feature complete on v5 filesystems, remove the experimental warnings from the superblock verifier. This means that we don't need to filter such warnings from the output in xfstests and so we can see exactly what tests are failing due to code deficiencies rather than from detecting warning noise.

---
 libxfs/xfs_sb.c | 4 ----
 1 file changed, 4 deletions(-)
libxfs/xfs_sb.c +0 -4
index 65ddc2f..48b1a97 100644
@@ -180,10 +180,6 @@ xfs_mount_validate_sb(
 	 * write validation, we don't need to check feature masks.
 	 */
 	if (check_version && XFS_SB_VERSION_NUM(sbp) == XFS_SB_VERSION_5) {
-		xfs_alert(mp,
-"Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled!\n"
-"Use of these features is at your own risk!");
-
 		if (xfs_sb_has_compat_feature(sbp,
 					XFS_SB_FEAT_COMPAT_UNKNOWN)) {
 			xfs_warn(mp,
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#46 lore raw

PATCH v4 35/37 repair: prefetching is turned off unnecessarily

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When we have a large filesystem, prefetching is only enabled when there is a significant amount of RAM available - roughly 16GB RAM for every 100TB of disk space. For large filesystems, this memory usage calculation is mostly derived from the memory needed to track used space rather than inodes. That is, for a 100TB filesystem with 50 million inodes, only 50M * 4 bytes or 200MB of the the required 16GB of RAM is used for tracking inodes. Hence with prefetching turned off, such a filesystem only uses 230MB of memory to run repair to completion.

With prefetching turned on, this increases to about 900MB of RAM, but it is still far, far less than the predicted 16GB of RAM needed to enable prefetching. Hence we are turning off prefetching when we really don't need to and hence large filesystems are being checked slower than they could be.

This patch makes prefetching always be enabled, but adds warnings in the case that we might not have enough memory to complete successfully and if it fails to run again with prefetching disabled:

Memory available for repair (12031MB) may not be sufficient. At least 13044MB is needed to repair this filesystem efficiently If repair fails due to lack of memory, please
turn prefetching off (-P) to reduce the memory footprint.

A similar warning is also added when prefetching is disabled and xfs_repair exhausts memory then more RAM/swap should be added to the system.

---
 repair/xfs_repair.c | 42 ++++++++++++++++++++++--------------------
 1 file changed, 22 insertions(+), 20 deletions(-)
repair/xfs_repair.c +22 -20
index 77a040e..78f8363 100644
@@ -674,34 +674,36 @@ main(int argc, char **argv)
 				mp->m_sb.sb_dblocks >> (10 + 1));
 
 		if (max_mem <= mem_used) {
-			/*
-			 * Turn off prefetch and minimise libxfs cache if
-			 * physical memory is deemed insufficient
-			 */
 			if (max_mem_specified) {
 				do_abort(
 	_("Required memory for repair is greater that the maximum specified\n"
 	  "with the -m option. Please increase it to at least %lu.\n"),
 					mem_used / 1024);
-			} else {
-				do_warn(
-	_("Not enough RAM available for repair to enable prefetching.\n"
-	  "This will be _slow_.\n"
-	  "You need at least %luMB RAM to run with prefetching enabled.\n"),
-					mem_used * 1280 / (1024 * 1024));
 			}
-			do_prefetch = 0;
-			libxfs_bhash_size = 64;
-		} else {
-			max_mem -= mem_used;
-			if (max_mem >= (1 << 30))
-				max_mem = 1 << 30;
-			libxfs_bhash_size = max_mem / (HASH_CACHE_RATIO *
-					(mp->m_inode_cluster_size >> 10));
-			if (libxfs_bhash_size < 512)
-				libxfs_bhash_size = 512;
+			do_warn(
+	_("Memory available for repair (%luMB) may not be sufficient.\n"
+	  "At least %luMB is needed to repair this filesystem efficiently\n"
+	  "If repair fails due to lack of memory, please\n"),
+				max_mem / 1024, mem_used / 1024);
+			if (do_prefetch)
+				do_warn(
+	_("turn prefetching off (-P) to reduce the memory footprint.\n"));
+			else
+				do_warn(
+	_("increase system RAM and/or swap space to at least %luMB.\n"),
+			mem_used * 2 / 1024);
+
+			max_mem = mem_used;
 		}
 
+		max_mem -= mem_used;
+		if (max_mem >= (1 << 30))
+			max_mem = 1 << 30;
+		libxfs_bhash_size = max_mem / (HASH_CACHE_RATIO *
+				(mp->m_inode_cluster_size >> 10));
+		if (libxfs_bhash_size < 512)
+			libxfs_bhash_size = 512;
+
 		if (verbose)
 			do_log(_("        - block cache size set to %d entries\n"),
 				libxfs_bhash_size * HASH_CACHE_RATIO);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#48 lore raw

PATCH v4 36/37 repair: Increase default repair parallelism on large filesystems

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Large filesystems or high AG count filesystems generally have more inherent parallelism in the backing storage. We shoul dmake use of this by default to speed up repair times. Make xfs_repair use an "auto-stride" configuration on filesystems with enough AGs to be considered "multidisk" configurations.

This difference in elaspsed time to repair a 100TB filesystem with 50 million inodes in it with all metadata in flash is:

Time IOPS BW CPU RAM
vanilla: 2719s 2900 55MB/s 25% 0.95GB
patched: 908s varied varied varied 2.33GB

With the patched kernel, there were IO peaks of over 1.3GB/s during AG scanning. Some phases now run at noticably different speeds
- phase 3 ran at ~180% CPU, 18,000 IOPS and 130MB/s,
- phase 4 ran at ~280% CPU, 12,000 IOPS and 100MB/s
- the other phases were similar to the vanilla repair.

Memory usage is increased because of the increased buffer cache size as a result of concurrent AG scanning using it.

---
 repair/xfs_repair.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
repair/xfs_repair.c +17 -0
index 78f8363..a863337 100644
@@ -614,6 +614,23 @@ main(int argc, char **argv)
 	inodes_per_cluster = MAX(mp->m_sb.sb_inopblock,
 			XFS_INODE_CLUSTER_SIZE(mp) >> mp->m_sb.sb_inodelog);
 
+	/*
+	 * Automatic striding for high agcount filesystems.
+	 *
+	 * More AGs indicates that the filesystem is either large or can handle
+	 * more IO parallelism. Either way, we should try to process multiple
+	 * AGs at a time in such a configuration to try to saturate the
+	 * underlying storage and speed the repair process. Only do this if
+	 * prefetching is enabled.
+	 *
+	 * Given mkfs defaults for 16AGs for "multidisk" configurations, we want
+	 * to target these for an increase in thread count. Hence a stride value
+	 * of 15 is chosen to ensure we get at least 2 AGs being scanned at once
+	 * on such filesystems.
+	 */
+	if (!ag_stride && glob_agcount >= 16 && do_prefetch)
+		ag_stride = 15;
+
 	if (ag_stride) {
 		thread_count = (glob_agcount + ag_stride - 1) / ag_stride;
 		thread_init();
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#49 lore raw

Re: [PATCH 36/37] repair: Increase default repair parallelism on large filesystems

repair/xfs_repair.c main(int argc, char **argv) · line 632

Quoted 2 lines · Dave Chinner
repair/xfs_repair.c +2 -0
+	if (!ag_stride && glob_agcount >= 16 && do_prefetch)
+		ag_stride = 15;

Shouldn't we have a define for the agcount shared by mkfs and repair, and derive the stride from it?

Otherwise looks fine.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#50 lore raw

Re: [PATCH 36/37] repair: Increase default repair parallelism on large filesystems

Quoted 8 lines · Christoph Hellwig

repair/xfs_repair.c main(int argc, char **argv) · line 632

Quoted 2 lines · Dave Chinner
repair/xfs_repair.c +2 -0
+	if (!ag_stride && glob_agcount >= 16 && do_prefetch)
+		ag_stride = 15;

Shouldn't we have a define for the agcount shared by mkfs and repair, and derive the stride from it?

Otherwise looks fine.

Yes, we could.

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#51 lore raw

Re: [PATCH 36/37] repair: Increase default repair parallelism on large filesystems

Quoted 11 lines · Dave Chinner
Quoted 8 lines · Christoph Hellwig

repair/xfs_repair.c main(int argc, char **argv) · line 632

Quoted 2 lines · Dave Chinner
repair/xfs_repair.c +2 -0
+	if (!ag_stride && glob_agcount >= 16 && do_prefetch)
+		ag_stride = 15;

Shouldn't we have a define for the agcount shared by mkfs and repair, and derive the stride from it?

Otherwise looks fine.

Yes, we could.

More complex than it looks - the mkfs calculation in calc_default_ag_geometry() is based on byte counts and a sliding scale of shifts and hence the resultant multidisk AG counts are implicit side effects of the those AG size calculations. Hence to make use of a common #define for this sort of striding, we'd need to rewrite calc_default_ag_geometry().

I'll do that in a separate patch series, as it's not directly related to this initial introduction of the feature to repair.

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#52 lore raw

Re: [PATCH 36/37] repair: Increase default repair parallelism on large filesystems

Quoted 2 lines · Dave Chinner

I'll do that in a separate patch series, as it's not directly related to this initial introduction of the feature to repair.

Ok, fine with me.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#53 lore raw

PATCH v4 37/37 repair: fix leaf node directory data check

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When walking the leaf node format blocks (LEAFN) in the hash index of a large directory, we could trip over btree node blocks (DA_NODE) in the address space if there are enough entries in the directory. These cause a verifier failure, and hence the directory is considered corrupt and is trashed and rebuilt unnecesarily. Fix this by using the correct verifier that can handle both types of blocks without triggering failures.

---
 repair/phase6.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
repair/phase6.c +9 -1
index 5307acf..d2d4a44 100644
@@ -1937,8 +1937,16 @@ longform_dir2_check_node(
 		next_da_bno = da_bno + mp->m_dirblkfsbs - 1;
 		if (bmap_next_offset(NULL, ip, &next_da_bno, XFS_DATA_FORK))
 			break;
+
+		/*
+		 * we need to use the da3 node verifier here as it handles the
+		 * fact that reading the leaf hash tree blocks can return either
+		 * leaf or node blocks and calls the correct verifier. If we get
+		 * a node block, then we'll skip it below based on a magic
+		 * number check.
+		 */
 		if (libxfs_da_read_buf(NULL, ip, da_bno, -1, &bp,
-				XFS_DATA_FORK, &xfs_dir3_leafn_buf_ops)) {
+				XFS_DATA_FORK, &xfs_da3_node_buf_ops)) {
 			do_warn(
 	_("can't read leaf block %u for directory inode %" PRIu64 "\n"),
 				da_bno, ip->i_ino);
Signature · 6 lines
1.8.4.rc3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v4 37/37
#54 lore raw

Re: [PATCH 37/37] repair: fix leaf node directory data check

Quoted 11 lines · Dave Chinner

From: Dave Chinner <[email protected]>

When walking the leaf node format blocks (LEAFN) in the hash index of a large directory, we could trip over btree node blocks (DA_NODE) in the address space if there are enough entries in the directory. These cause a verifier failure, and hence the directory is considered corrupt and is trashed and rebuilt unnecesarily. Fix this by using the correct verifier that can handle both types of blocks without triggering failures.

Looks good,

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#56 lore raw
Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Dave Chinner

Hi folks,

Many thanks to Christoph for reviewing V3 of this patchset, which was posted here:

Seems like you mostly lost my review tags so I'll have to wade through the whole series again..

Oh, sorry, I forgot to add them when I rebased it. My fault - I was more concerned with making sure I got al the changes... I'll add them if I need to post it again...

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#57 lore raw
In reply to Dave Chinner

Hey Dave,

I am ready to pull in patches 1-16 of this series. Is that OK, or would you prefer to wait for all the patches?

The outstanding work I see is:
[20/37] db: rewrite IO engine to use libxfs [24/37] db: verify and calculate dquot CRCs [32/37] db: enable metadump on CRC filesystems Needs a review

[31/37] db: Unwind the IO stack on exit
Reviewed, but needs a response to Christoph's question "Should be folded into patch 20?" (listed as unreviewed above).

[36/37] repair: Increase default repair parallelism ... Waiting for another version from you.

Please let me know if I missed something.

Thanks
--Rich

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs