btrfs: add test for BTRFS_IOC_GET_CSUMS ioctl
PATCH 1/1 btrfs: add test for BTRFS_IOC_GET_CSUMS ioctl
Test the new BTRFS_IOC_GET_CSUMS ioctl which retrieves file checksums from the kernel's csum tree. The test covers:
1. Regular data extents returning HAS_CSUMS with checksum data
2. Holes returning SPARSE entries
3. Preallocated extents returning SPARSE entries
4. Buffer continuation (small buffer forcing multiple calls)
5. Checksum consistency (identical data produces identical csums)
6. Querying past file data returning SPARSE
7. Precomputed values
--- src/Makefile | 2 +- src/btrfs_get_csums.c | 198 ++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/343 | 196 +++++++++++++++++++++++++++++++++++++++++ tests/btrfs/343.out | 33 +++++++ 4 files changed, 428 insertions(+), 1 deletion(-) create mode 100644 src/btrfs_get_csums.c create mode 100755 tests/btrfs/343 create mode 100644 tests/btrfs/343.out
src/Makefile +1 -1
@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint + rw_hint btrfs_get_csums EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \
src/btrfs_get_csums.c new file +198 -0
198 lines not on this page · raw message
tests/btrfs/343 new file +196 -0
196 lines not on this page · raw message
tests/btrfs/343.out new file +33 -0
@@ -0,0 +1,33 @@ +QA output created by 343 +=== Test 1: regular data === +HAS_CSUMS +has csums +complete +=== Test 2: hole === +HAS_CSUMS +SPARSE +HAS_CSUMS +complete +=== Test 3: prealloc === +SPARSE +complete +=== Test 4: continuation === +SPARSE +incomplete +HAS_CSUMS +complete +=== Test 5: csum consistency === +csums match +=== Test 6: past data === +SPARSE +complete +=== Test 7: precomputed csums === +--- crc32c --- +csums verified +--- xxhash --- +csums verified +--- sha256 --- +csums verified +--- blake2 --- +csums verified +=== done ===
Signature · 1 line
2.52.0
Quoted 2 lines · Mark Harmstone
Test the new
BTRFS_IOC_GET_CSUMSioctl which retrieves file checksums from the kernel's csum tree. The test covers:
I think it's a great idea to have a dedicated test case for the new ioctl.
Although that ioctl has some changes and a refresh will be appriciated.
Quoted 2 lines
1. Regular data extents returning
HAS_CSUMSwith checksum data
Maybe also test an inlined extent?
Quoted 5 lines
2. Holes returning SPARSE entries
3. Preallocated extents returning SPARSE entries
4. Buffer continuation (small buffer forcing multiple calls)
5. Checksum consistency (identical data produces identical csums)
6. Querying past file data returning SPARSE
I guess we may also want some corner cases related to the last block:
- The file size is not aligned
* Test get csum of the last block
Which should return the csum of the last full block.
* Test get csum beyond the last block
Which should return ZEROED.
- The file size is aligned, but the last extent is truncated E.g. the file size is 4K, but the data extent is 16K (e.g. reflinked)
* Test get csum of the [4K, 8K)
Which should return ZEROED
Quoted 3 lines
7. Precomputed values
Signed-off-by Mark Harmstone <[email protected]>
[...]
tests/btrfs/343 line 129
Quoted 5 lines
tests/btrfs/343 new file +5
-0+ +# Test 7: Verify checksum values against precomputed values for all algorithms. +# Uses explicit -s 4096 as the expected values are precomputed for 4096-byte +# sectors. Write three sectors: all 0x00, all 0xff, all 0xaa. +echo "=== Test 7: precomputed csums ==="
One idea to get rid of the pre-calculated csum is, to read out the data block by block from the file (as long as the file size is aligned to block size), then calculate the csum.
For xxhash/sha256/blake2, the checksum can be calculated in user space and should match the one in csum tree.
It's only crc32c that we do not have a handy tool in user-space.
Thanks,
Qu
Quoted 106 lines
tests/btrfs/343 new file +67
-0+_scratch_unmount +for csum_algo in crc32c xxhash sha256 blake2; do + echo "--- $csum_algo ---" + + _scratch_mkfs -s 4096 --csum $csum_algo >> $seqres.full 2>&1 + _scratch_mount + + case "$csum_algo" in + crc32c) + expect_00="98f94189" + expect_ff="25c1fe13" + expect_aa="4ed66b65" + ;; + xxhash) + expect_00="ac869b6f32d8bbdb" + expect_ff="10af2cb94282321f" + expect_aa="e35ac2d66625ceaa" + ;; + sha256) + expect_00="ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7" + expect_ff="f47a8ec3e9aff2318d896942282ad4fe37d6391c82914f54a5da8a37de1300c6" + expect_aa="c622005493c4cb75f3e08eda4cc0bfe172e2c5eeca661ec4908c5490fc3d6994" + ;; + blake2) + expect_00="686ede9288c391e7e05026e56f2f91bfd879987a040ea98445dabc76f55b8e5f" + expect_ff="1e23d2944a4523734ef9c8b536eed668fa8a99b272bfc10988864e1ef135197d" + expect_aa="8337334ce71745681a24e9aa409fdf85fe4081242c39815ccf266df6d33355a2" + ;; + esac + + $XFS_IO_PROG -f \ + -c "pwrite -S 0x00 0 4096" \ + -c "pwrite -S 0xff 4096 4096" \ + -c "pwrite -S 0xaa 8192 4096" \ + $SCRATCH_MNT/file7 >> $seqres.full + sync + + output=$($GET_CSUMS $SCRATCH_MNT/file7 0 12288) + echo "$output" >> $seqres.full + + got_csums=$(echo "$output" | grep "^HAS_CSUMS" | \ + awk '{for(i=4;i<=NF;i++) printf $i " "; print ""}') + got_00=$(echo $got_csums | awk '{print $1}') + got_ff=$(echo $got_csums | awk '{print $2}') + got_aa=$(echo $got_csums | awk '{print $3}') + + pass=true + if [ "$got_00" != "$expect_00" ]; then + echo "FAIL: 0x00 sector csum $got_00 != $expect_00" + pass=false + fi + if [ "$got_ff" != "$expect_ff" ]; then + echo "FAIL: 0xff sector csum $got_ff != $expect_ff" + pass=false + fi + if [ "$got_aa" != "$expect_aa" ]; then + echo "FAIL: 0xaa sector csum $got_aa != $expect_aa" + pass=false + fi + $pass && echo "csums verified" + + _scratch_unmount +done + +echo "=== done ===" +status=0 +exittests/btrfs/343.out new file +33
-0@@ -0,0 +1,33 @@ +QA output created by 343 +=== Test 1: regular data === +HAS_CSUMS +has csums +complete +=== Test 2: hole === +HAS_CSUMS +SPARSE +HAS_CSUMS +complete +=== Test 3: prealloc === +SPARSE +complete +=== Test 4: continuation === +SPARSE +incomplete +HAS_CSUMS +complete +=== Test 5: csum consistency === +csums match +=== Test 6: past data === +SPARSE +complete +=== Test 7: precomputed csums === +--- crc32c --- +csums verified +--- xxhash --- +csums verified +--- sha256 --- +csums verified +--- blake2 --- +csums verified +=== done ===