xfsprogs: metadata CRC support V2
PATCH 1/1 xfsprogs: metadata CRC support V2
Hi folks,
I've attached a tarball containing the patch series that adds metadata CRC support to xfsprogs. It is still very much a work in progress, but is sufficient to start running xfstests on CRC enable filesystems.
THere is much missing functionality, especially with respect to directly/attribute modifications. Only mkfs and xfs_repair have been updated to explicitly support CRC on directory/sttribute blocks, though all the other tools will stil run the result might be a little unpredictable.
The patch series is as follows:
Dave Chinner (26):
xfsprogs: sync code to current kernel code
xfsprogs: update libxfs to 3.9-rc1 + xfsdev
xfsprogs: add CRC32c infrastructure
xfsprogs: updata libxlog to current kernel code
libxfs: add crc format changes to generic btrees
xfsprogs: add crc format chagnes to ag headers
libxfs: change quota buffer formats
libxfs: add version 3 inode support
libxfs: add support for crc headers on remote symlinks
xfs: add CRC checks to block format directory blocks
xfs: add CRC checking to dir2 free blocks
xfs: add CRC checking to dir2 data blocks
xfs: add CRC checking to dir2 leaf blocks
xfs: shortform directory offsets change for dir3 format
xfs: add CRCs to dir2/da node blocks
xfs: add CRCs to attr leaf blocks
xfs: split remote attribute code out
xfs: add CRC protection to remote attributes
xfs: add buffer types to directory and attribute buffers
xfs: buffer type overruns blf_flags field
xfs: add CRC checks to the superblock
xfs: implement extended feature masks
xfsprogs: introduce CRC support into mkfs.xfs
xfsprogs: add crc format support to repair
xfsprogs: add crc format support to db
xfs_repair: update for dir/attr crc format changes.
Diffstat is below [1].
The changes are roughly:
- sync to current kernel code
- apply userspace converted version of kernel patches
- update xfsprogs utilities to be aware of CRCs
Much of the last 4 patches needs to be split up and redirected back to the patches that introduce the bugs that are fixed or conversions that are missed.
Repair needs support for remote attribute and symlink block CRC support, as well as verifying the new feature masks. mkfs needs to handle thenew feature masks correctly, too. xfs_db needs to be able to display all crc enabled formats, and needs to ensure all read-only functions work properly on CRC enabled filesystems. I think that metadump will also need some attention. So there's plenty of work yet to do, though it is already usable.
For non-CRC filesystems, the utilities appear to work without regressions in xfstests. Most of the testing I have done is on non-CRC filesytsems.
For CRC enabled filesystems, xfstests will run for a short while before the kernel bugs out with an attribute corruption. Over the next week I hope to get the kernel code to the point where it will run most of xfstests with CRCs enabled:
$ sudo MKFS_OPTIONS="-m crc=1" ./check -g auto FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 test-4 3.9.0-rc4-dgc+ MKFS_OPTIONS -- -f -m crc=1 /dev/vdb MOUNT_OPTIONS -- /dev/vdb /mnt/scr
generic/001 4s generic/002 0s generic/005 0s generic/006 1s generic/007 1s generic/010 [not run] dbtest was not built for this platform generic/011 1s generic/013 12s generic/014 3s generic/015 0s
generic/020
Message from syslogd@test-4 at Apr 13 00:35:45 ...
kernel:[ 451.268820] XFS: Assertion failed: ichdr->firstused >= ichdr->count * sizeof(xfs_attr_leaf_entry_t) + xfs_attr3_leaf_hdr_size(leaf), file: fs/xfs/xfs_attr_leaf.c, line: 1427
$
Finally, I'm sending this update out as a quilt series style tarball rather than email patches as some of the patches are too large for the list and will get blocked. Hence you get patches and a series file that I've just exported from my current guilt tree.
Obligatory warning: If you enable CRCs, the code will eat your data. Use this only only for testing with data you don't care about.
Comments, flames and testing results all welcome.
Cheers,
Dave.
[1] diffstat:
copy/xfs_copy.c | 5 +- db/agf.c | 3 + db/agfl.c | 16 + db/agfl.h | 2 + db/agi.c | 3 + db/attr.c | 20 +- db/btblock.c | 145 + db/btblock.h | 10 + db/check.c | 55 +- db/dir.c | 8 +- db/dir.h | 6 + db/dir2.c | 16 +- db/dir2.h | 32 + db/dir2sf.c | 24 +- db/dirshort.c | 1 + db/field.c | 16 + db/field.h | 8 + db/freesp.c | 11 +- db/init.c | 4 + db/inode.c | 27 + db/inode.h | 3 + db/metadump.c | 27 +- db/sb.c | 40 +- db/type.c | 35 +- db/type.h | 3 +- estimate/xfs_estimate.c | 3 + fsr/xfs_fsr.c | 2 +- include/Makefile | 13 +- include/libxfs.h | 222 +- include/libxlog.h | 34 +- include/linux.h | 1 + include/platform_defs.h.in | 5 + include/swab.h | 38 + include/xfs.h | 1 + include/xfs_ag.h | 81 +- include/xfs_alloc.h | 57 +- include/xfs_alloc_btree.h | 29 +- include/xfs_arch.h | 16 + include/xfs_attr_leaf.h | 135 +- include/xfs_attr_remote.h | 52 + include/xfs_bmap.h | 325 +- include/xfs_bmap_btree.h | 21 +- include/xfs_btree.h | 87 +- include/xfs_buf_item.h | 117 +- include/xfs_cksum.h | 63 + include/xfs_da_btree.h | 185 +- include/xfs_dinode.h | 47 +- include/xfs_dir1.h | 13 + include/xfs_dir2.h | 95 +- include/xfs_dir2_block.h | 92 - include/xfs_dir2_data.h | 184 - include/xfs_dir2_format.h | 836 +++++ include/xfs_dir2_leaf.h | 253 -- include/xfs_dir2_node.h | 100 - include/xfs_dir2_sf.h | 171 - include/xfs_fs.h | 38 +- include/xfs_ialloc.h | 17 +- include/xfs_ialloc_btree.h | 12 +- include/xfs_inode.h | 263 +- include/xfs_inode_item.h | 22 +- include/xfs_inum.h | 16 - include/xfs_log.h | 31 +- include/xfs_log_priv.h | 101 +- include/xfs_mount.h | 48 +- include/xfs_quota.h | 58 +- include/xfs_rtalloc.h | 4 +- include/xfs_sb.h | 172 +- include/xfs_symlink.h | 45 + include/xfs_trace.h | 69 +- include/xfs_trans.h | 88 +- include/xfs_types.h | 25 +- libxfs/Makefile | 26 +- libxfs/crc32.c | 1036 ++++++ libxfs/crc32defs.h | 72 + libxfs/gen_crc32table.c | 144 + libxfs/init.c | 26 +- libxfs/logitem.c | 20 +- libxfs/rdwr.c | 522 ++- libxfs/trans.c | 55 +- libxfs/util.c | 230 +- libxfs/xfs.h | 147 +- libxfs/xfs_alloc.c | 694 ++-- libxfs/xfs_alloc_btree.c | 140 +- libxfs/xfs_attr.c | 546 +-- libxfs/xfs_attr_leaf.c | 1858 ++++++---- libxfs/xfs_attr_remote.c | 527 +++ libxfs/xfs_bmap.c | 8950 ++++++++++++++++++++++++---------------------- libxfs/xfs_bmap_btree.c | 124 +- libxfs/xfs_btree.c | 415 ++- libxfs/xfs_da_btree.c | 2598 ++++++++------ libxfs/xfs_dir2.c | 149 +- libxfs/xfs_dir2_block.c | 806 +++-- libxfs/xfs_dir2_data.c | 604 ++-- libxfs/xfs_dir2_leaf.c | 1342 ++++--- libxfs/xfs_dir2_node.c | 1455 ++++---- libxfs/xfs_dir2_priv.h | 160 + libxfs/xfs_dir2_sf.c | 326 +- libxfs/xfs_ialloc.c | 632 ++-- libxfs/xfs_ialloc_btree.c | 87 +- libxfs/xfs_inode.c | 505 ++- libxfs/xfs_mount.c | 259 +- libxfs/xfs_rtalloc.c | 77 +- libxfs/xfs_symlink.c | 154 + libxfs/xfs_trans.c | 9 +- libxlog/xfs_log_recover.c | 299 +- logprint/log_copy.c | 2 +- logprint/log_dump.c | 2 +- logprint/log_misc.c | 9 +- logprint/log_print_all.c | 7 +- logprint/log_print_trans.c | 22 +- logprint/logprint.c | 8 +- logprint/logprint.h | 10 +- mdrestore/Makefile | 2 +- mkfs/maxtrres.c | 4 +- mkfs/proto.c | 14 +- mkfs/xfs_mkfs.c | 162 +- mkfs/xfs_mkfs.h | 12 +- repair/agheader.c | 36 +- repair/attr_repair.c | 63 +- repair/dino_chunks.c | 13 +- repair/dinode.c | 208 +- repair/dir.c | 96 +- repair/dir2.c | 497 +-- repair/dir2.h | 28 +- repair/phase2.c | 13 +- repair/phase3.c | 2 +- repair/phase5.c | 157 +- repair/phase6.c | 327 +- repair/prefetch.c | 11 +- repair/rt.c | 4 +- repair/scan.c | 166 +- repair/scan.h | 12 +- repair/versions.c | 2 +- repair/xfs_repair.c | 9 +- 134 files changed, 19036 insertions(+), 12663 deletions(-) create mode 100644 include/xfs_attr_remote.h create mode 100644 include/xfs_cksum.h create mode 100644 include/xfs_dir1.h delete mode 100644 include/xfs_dir2_block.h delete mode 100644 include/xfs_dir2_data.h create mode 100644 include/xfs_dir2_format.h delete mode 100644 include/xfs_dir2_leaf.h delete mode 100644 include/xfs_dir2_node.h delete mode 100644 include/xfs_dir2_sf.h create mode 100644 include/xfs_symlink.h create mode 100644 libxfs/crc32.c create mode 100644 libxfs/crc32defs.h create mode 100644 libxfs/gen_crc32table.c create mode 100644 libxfs/xfs_attr_remote.c create mode 100644 libxfs/xfs_dir2_priv.h create mode 100644 libxfs/xfs_symlink.c
Signature · 2 lines
Dave Chinner [email protected]
xfsprogs-crc-patchset-v2/ 0000755 0001750 0001750 00000000000 12132011610 014237 5 ustar dave dave xfsprogs-crc-patchset-v2/series 0000644 0001750 0001750 00000001436 12132011546 015470 0 ustar dave dave repair-fix-extent-count logprint-agi-dump
xfsprogs-kernel-3.8-update
xfsprogs-kernel-3.9-rc1-dev-update
xfsprogs-crc-infrastructure
xfsprogs-libxlog-kernel-sync
btree-format-changes
ag-hdr-format-changes
quota-format-changes
inode-v3
remote-symlink-crc
xfs-add-crc-to-dir2-block-2
xfs-add-crc-to-dir2-free-2
xfs-add-crc-to-dir2-data-2
xfs-add-crc-to-dir2-leaf-3
dir2-sf
xfs-add-crc-to-da-node
xfs-add-crc-to-attr-leaf
xfs-attr-rmt-move
xfs-add-crc-to-remote-attr
xfs-da-blf-types
xfs-trans-type-blf-change
xfs__add_crc_checks_to_the_superblock-2
xfs-sb-v5-feature-masks
xfsprogs-mkfs-crc-support
xfsprogs-repair-crc-support
xfsprogs-db-crc-support
xfsprogs-repair-dir-crc-support
debug
bpm-progs-deb
xfsprogs-freespace
spaceman-trim
spaceman-prealloc-flush
spaceman-ag-control
xfsprogs-freespace-2