xfstests: update verity tests for non-4K block and page size
Eric Biggers
maintainerv3 cover xfstests: update verity tests for non-4K block and page size
This series updates the verity xfstests to eliminate implicit assumptions that 'merkle_tree_block_size == fs_block_size == page_size == 4096', and to provide some test coverage for cases where merkle_tree_block_size differs from fs_block_size and/or page_size. It doesn't add any new test scripts, but it does update some of the existing test scripts to test multiple block sizes.
This goes along with my kernel patch series
"fsverity: support for non-4K pages"
(https://lore.kernel.org/linux-fsdevel/[email protected]/T/#u).
However, it's not necessary to wait for that kernel patch series to be applied before applying this xfstests patch series.
Changed in v3:
- Fixed generic/574 failure with some bash versions.
Changed in v2:
- Adjusted the output of generic/574, generic/575, and generic/624 slightly to avoid confusion.
Eric Biggers (10):
common/verity: add and use _fsv_can_enable() common/verity: set FSV_BLOCK_SIZE to an appropriate value common/verity: use FSV_BLOCK_SIZE by default common/verity: add _filter_fsverity_digest() generic/572: support non-4K Merkle tree block size generic/573: support non-4K Merkle tree block size generic/577: support non-4K Merkle tree block size generic/574: test multiple Merkle tree block sizes generic/624: test multiple Merkle tree block sizes generic/575: test 1K Merkle tree block size
common/verity | 84 ++++++++++++---- tests/generic/572 | 21 ++-- tests/generic/572.out | 10 +- tests/generic/573 | 8 +- tests/generic/574 | 219 +++++++++++++++++++++++++----------------- tests/generic/574.out | 83 +--------------- tests/generic/575 | 57 ++++++++--- tests/generic/575.out | 8 +- tests/generic/577 | 24 ++--- tests/generic/577.out | 10 +- tests/generic/624 | 119 ++++++++++++++++------- tests/generic/624.out | 15 +-- 12 files changed, 370 insertions(+), 288 deletions(-)
base-commit: 3dc46f477b39d732e1841e6f5a180759cee3e8ce
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 5/10 generic/572: support non-4K Merkle tree block size
From: Eric Biggers <[email protected]>
Update generic/572 to not implicitly assume that the Merkle tree block size being used is 4096 bytes. Also remove an unused function.
--- tests/generic/572 | 21 ++++++--------------- tests/generic/572.out | 10 +++------- 2 files changed, 9 insertions(+), 22 deletions(-)
tests/generic/572 +6 -15
@@ -9,7 +9,7 @@ # - conditions for enabling verity # - verity files have correct contents and size # - can't change contents of verity files, but can change metadata -# - can retrieve a verity file's measurement via FS_IOC_MEASURE_VERITY +# - can retrieve a verity file's digest via FS_IOC_MEASURE_VERITY # . ./common/preamble _begin_fstest auto quick verity
@@ -48,15 +48,6 @@ verify_data_readable() md5sum $file > /dev/null } -verify_data_unreadable() -{ - local file=$1 - - # try both reading just the first data block, and reading until EOF - head -c $FSV_BLOCK_SIZE $file 2>&1 >/dev/null | filter_output - md5sum $file |& filter_output -} - _fsv_scratch_begin_subtest "Enabling verity on file with verity already enabled fails with EEXIST" _fsv_create_enable_file $fsv_file echo "(trying again)"
@@ -94,7 +85,7 @@ verify_data_readable $fsv_file _fsv_scratch_begin_subtest "Enabling verity can be interrupted" dd if=/dev/zero of=$fsv_file bs=1 count=0 seek=$((1 << 34)) status=none start_time=$(date +%s) -$FSVERITY_PROG enable $fsv_file & +$FSVERITY_PROG enable --block-size=$FSV_BLOCK_SIZE $fsv_file & sleep 0.5 kill %1 wait
@@ -106,7 +97,7 @@ fi _fsv_scratch_begin_subtest "Enabling verity on file with verity already being enabled fails with EBUSY" dd if=/dev/zero of=$fsv_file bs=1 count=0 seek=$((1 << 34)) status=none start_time=$(date +%s) -$FSVERITY_PROG enable $fsv_file & +$FSVERITY_PROG enable --block-size=$FSV_BLOCK_SIZE $fsv_file & sleep 0.5 _fsv_enable $fsv_file |& filter_output kill %1
@@ -129,7 +120,7 @@ verify_data_readable $fsv_file _fsv_scratch_begin_subtest "verity file can be measured" _fsv_create_enable_file $fsv_file >> $seqres.full -_fsv_measure $fsv_file +_fsv_measure $fsv_file | _filter_fsverity_digest _fsv_scratch_begin_subtest "verity file can be renamed" _fsv_create_enable_file $fsv_file
@@ -170,8 +161,8 @@ verify_data_readable $fsv_file # Test files <= 1 block in size. These are a bit of a special case since there # are no hash blocks; the root hash is calculated directly over the data block. +_fsv_scratch_begin_subtest "verity on small files" for size in 1 $((FSV_BLOCK_SIZE - 1)) $FSV_BLOCK_SIZE; do - _fsv_scratch_begin_subtest "verity on $size-byte file" head -c $size /dev/urandom > $fsv_orig_file cp $fsv_orig_file $fsv_file _fsv_enable $fsv_file
@@ -179,7 +170,7 @@ for size in 1 $((FSV_BLOCK_SIZE - 1)) $FSV_BLOCK_SIZE; do rm -f $fsv_file done -_fsv_scratch_begin_subtest "verity on 100M file (multiple levels in hash tree)" +_fsv_scratch_begin_subtest "verity on 100MB file (multiple levels in hash tree)" head -c 100000000 /dev/urandom > $fsv_orig_file cp $fsv_orig_file $fsv_file _fsv_enable $fsv_file
tests/generic/572.out +3 -7
@@ -39,7 +39,7 @@ bash: SCRATCH_MNT/file.fsv: Operation not permitted # verity file can be read # verity file can be measured -sha256:be54121da3877f8852c65136d731784f134c4dd9d95071502e80d7be9f99b263 +sha256:<digest> # verity file can be renamed
@@ -58,16 +58,12 @@ sha256:be54121da3877f8852c65136d731784f134c4dd9d95071502e80d7be9f99b263 # Trying to measure non-verity file fails with ENODATA ERROR: FS_IOC_MEASURE_VERITY failed on 'SCRATCH_MNT/file.fsv': No data available -# verity on 1-byte file +# verity on small files Files matched - -# verity on 4095-byte file Files matched - -# verity on 4096-byte file Files matched -# verity on 100M file (multiple levels in hash tree) +# verity on 100MB file (multiple levels in hash tree) Files matched # verity on sparse file
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 6/10 generic/573: support non-4K Merkle tree block size
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Update generic/573 to not implicitly assume that the Merkle tree block size being used is 4096 bytes.
--- tests/generic/573 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tests/generic/573 +4 -4
@@ -36,23 +36,23 @@ fsv_file=$SCRATCH_MNT/file.fsv _fsv_scratch_begin_subtest "FS_IOC_ENABLE_VERITY doesn't require root" echo foo > $fsv_file chmod 666 $fsv_file -_user_do "$FSVERITY_PROG enable $fsv_file" +_user_do "$FSVERITY_PROG enable --block-size=$FSV_BLOCK_SIZE $fsv_file" _fsv_scratch_begin_subtest "FS_IOC_ENABLE_VERITY requires write access" echo foo > $fsv_file >> $seqres.full chmod 444 $fsv_file -_user_do "$FSVERITY_PROG enable $fsv_file" |& _filter_scratch +_user_do "$FSVERITY_PROG enable --block-size=$FSV_BLOCK_SIZE $fsv_file" |& _filter_scratch _fsv_scratch_begin_subtest "FS_IOC_ENABLE_VERITY requires !append-only" echo foo > $fsv_file >> $seqres.full $CHATTR_PROG +a $fsv_file -$FSVERITY_PROG enable $fsv_file |& _filter_scratch +_fsv_enable $fsv_file |& _filter_scratch $CHATTR_PROG -a $fsv_file _fsv_scratch_begin_subtest "FS_IOC_ENABLE_VERITY requires !immutable" echo foo > $fsv_file >> $seqres.full $CHATTR_PROG +i $fsv_file -$FSVERITY_PROG enable $fsv_file |& _filter_scratch +_fsv_enable $fsv_file |& _filter_scratch $CHATTR_PROG -i $fsv_file _fsv_scratch_begin_subtest "FS_IOC_MEASURE_VERITY doesn't require root"
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 7/10 generic/577: support non-4K Merkle tree block size
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Update generic/577 to not implicitly assume that the Merkle tree block size being used is 4096 bytes.
--- tests/generic/577 | 22 ++++++++++++---------- tests/generic/577.out | 10 +++++----- 2 files changed, 17 insertions(+), 15 deletions(-)
tests/generic/577 +12 -10
@@ -38,6 +38,11 @@ sigfile=$tmp.sig otherfile=$SCRATCH_MNT/otherfile othersigfile=$tmp.othersig +sign() +{ + _fsv_sign "$@" | _filter_scratch | _filter_fsverity_digest +} + # Setup echo -e "\n# Generating certificates and private keys"
@@ -57,14 +62,13 @@ _enable_fsverity_signatures echo -e "\n# Generating file and signing it for fs-verity" head -c 100000 /dev/zero > $fsv_orig_file for suffix in '' '.2'; do - _fsv_sign $fsv_orig_file $sigfile$suffix --key=$keyfile$suffix \ - --cert=$certfile$suffix | _filter_scratch + sign $fsv_orig_file $sigfile$suffix --key=$keyfile$suffix \ + --cert=$certfile$suffix done echo -e "\n# Signing a different file for fs-verity" head -c 100000 /dev/zero | tr '\0' 'X' > $otherfile -_fsv_sign $otherfile $othersigfile --key=$keyfile --cert=$certfile \ - | _filter_scratch +sign $otherfile $othersigfile --key=$keyfile --cert=$certfile # Actual tests
@@ -106,16 +110,15 @@ _fsv_enable $fsv_file --signature=$tmp.malformed_sig |& _filter_scratch echo -e "\n# Testing salt" reset_fsv_file -_fsv_sign $fsv_orig_file $sigfile.salted --key=$keyfile --cert=$certfile \ - --salt=abcd | _filter_scratch +sign $fsv_orig_file $sigfile.salted --key=$keyfile --cert=$certfile --salt=abcd _fsv_enable $fsv_file --signature=$sigfile.salted --salt=abcd cmp $fsv_file $fsv_orig_file echo -e "\n# Testing non-default hash algorithm" if _fsv_can_enable $fsv_file --hash-alg=sha512; then reset_fsv_file - _fsv_sign $fsv_orig_file $sigfile.sha512 --key=$keyfile \ - --cert=$certfile --hash-alg=sha512 > /dev/null + sign $fsv_orig_file $sigfile.sha512 --key=$keyfile --cert=$certfile \ + --hash-alg=sha512 > /dev/null _fsv_enable $fsv_file --signature=$sigfile.sha512 --hash-alg=sha512 cmp $fsv_file $fsv_orig_file fi
@@ -123,8 +126,7 @@ fi echo -e "\n# Testing empty file" rm -f $fsv_file echo -n > $fsv_file -_fsv_sign $fsv_file $sigfile.emptyfile --key=$keyfile --cert=$certfile | \ - _filter_scratch +sign $fsv_file $sigfile.emptyfile --key=$keyfile --cert=$certfile _fsv_enable $fsv_file --signature=$sigfile.emptyfile # success, all done
tests/generic/577.out +5 -5
@@ -9,11 +9,11 @@ QA output created by 577 # Enabling fs.verity.require_signatures # Generating file and signing it for fs-verity -Signed file 'SCRATCH_MNT/file' (sha256:ecabbfca4efd69a721be824965da10d27900b109549f96687b35a4d91d810dac) -Signed file 'SCRATCH_MNT/file' (sha256:ecabbfca4efd69a721be824965da10d27900b109549f96687b35a4d91d810dac) +Signed file 'SCRATCH_MNT/file' (sha256:<digest>) +Signed file 'SCRATCH_MNT/file' (sha256:<digest>) # Signing a different file for fs-verity -Signed file 'SCRATCH_MNT/otherfile' (sha256:b2a419c5a8c767a78c6275d6729794bf51e52ddf8713e31d12a93d61d961f49f) +Signed file 'SCRATCH_MNT/otherfile' (sha256:<digest>) # Enabling verity with valid signature (should succeed)
@@ -33,9 +33,9 @@ ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Key was rejected b ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Bad message # Testing salt -Signed file 'SCRATCH_MNT/file' (sha256:1cb173bcd199133eb80e9ea4f0f741001b9e73227aa8812685156f2bc8ff45f5) +Signed file 'SCRATCH_MNT/file' (sha256:<digest>) # Testing non-default hash algorithm # Testing empty file -Signed file 'SCRATCH_MNT/file.fsv' (sha256:3d248ca542a24fc62d1c43b916eae5016878e2533c88238480b26128a1f1af95) +Signed file 'SCRATCH_MNT/file.fsv' (sha256:<digest>)
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 9/10 generic/624: test multiple Merkle tree block sizes
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Instead of only testing 4K Merkle tree blocks, test FSV_BLOCK_SIZE, and also other sizes if they happen to be supported. This allows this test to run in cases where it couldn't before and improves test coverage in cases where it did run before.
This required reworking the test to compute the expected digests using the 'fsverity digest' command, instead of relying on .out file comparisons. (There isn't much downside to this, since comparison to known-good file digests already happens in generic/575. So if both the kernel and fsverity-utils were to be broken in the same way, generic/575 would detect it. generic/624 serves a different purpose.)
--- common/verity | 11 ++++ tests/generic/624 | 119 ++++++++++++++++++++++++++++++------------ tests/generic/624.out | 15 ++---- 3 files changed, 101 insertions(+), 44 deletions(-)
common/verity +11 -0
@@ -263,6 +263,17 @@ _fsv_measure() $FSVERITY_PROG measure "$@" | awk '{print $1}' } +_fsv_digest() +{ + local args=("$@") + # If the caller didn't explicitly specify a Merkle tree block size, then + # use FSV_BLOCK_SIZE. + if ! [[ " $*" =~ " --block-size" ]]; then + args+=("--block-size=$FSV_BLOCK_SIZE") + fi + $FSVERITY_PROG digest "${args[@]}" | awk '{print $1}' +} + _fsv_sign() { local args=("$@")
tests/generic/624 +85 -34
133 lines not on this page · raw message
tests/generic/624.out +5 -10
@@ -1,16 +1,11 @@ QA output created by 624 -# Creating a verity file -sha256:11e4f886bf2d70a6ef3a8b6ce8e8c62c9e5d3263208b9f120ae46791f124be73 +# Testing block_size=FSV_BLOCK_SIZE -# Dumping Merkle tree -db88cdad554734cd648a1bfbb5be7f86646c54397847aab0b3f42a28829fed17 - +# Testing block_size=1024 if supported -# Dumping Merkle tree (in chunks) -db88cdad554734cd648a1bfbb5be7f86646c54397847aab0b3f42a28829fed17 - +# Testing block_size=4096 if supported -# Dumping descriptor -11e4f886bf2d70a6ef3a8b6ce8e8c62c9e5d3263208b9f120ae46791f124be73 - +# Testing block_size=16384 if supported -# Dumping descriptor (in chunks) -11e4f886bf2d70a6ef3a8b6ce8e8c62c9e5d3263208b9f120ae46791f124be73 - +# Testing block_size=65536 if supported
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 4/10 common/verity: add _filter_fsverity_digest()
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Add a filter that replaces fs-verity digests with a fixed string. This is needed because the fs-verity digests that some tests print are going to start depending on the default Merkle tree block size.
--- common/verity | 8 ++++++++ 1 file changed, 8 insertions(+)
common/verity +8 -0
@@ -406,3 +406,11 @@ _require_fsverity_max_file_size_limit() ;; esac } + +# Replace fs-verity digests, as formatted by the 'fsverity' tool, with <digest>. +# This function can be used by tests where fs-verity digests depend on the +# default Merkle tree block size (FSV_BLOCK_SIZE). +_filter_fsverity_digest() +{ + sed -E 's/\b(sha(256|512)):[a-f0-9]{64,}\b/\1:<digest>/' +}
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 2/10 common/verity: set FSV_BLOCK_SIZE to an appropriate value
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
In order to maximize the chance that the verity tests can actually be run, FSV_BLOCK_SIZE (the default Merkle tree size for the verity tests) needs to be min(fs_block_size, page_size), not simply page_size. The only reason that page_size was okay before was because the kernel only supported merkle_tree_block_size == fs_block_size == page_size anyway. But that is changing.
--- common/verity | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-)
common/verity +27 -5
@@ -13,6 +13,11 @@ if [ "$FSTYP" == "btrfs" ]; then fi fi +# Require fs-verity support on the scratch filesystem. +# +# FSV_BLOCK_SIZE will be set to a Merkle tree block size that is supported by +# the filesystem. Other sizes may be supported too, but FSV_BLOCK_SIZE is the +# only size that is guaranteed to work without any additional checks. _require_scratch_verity() { _require_scratch
@@ -27,7 +32,7 @@ _require_scratch_verity() # Try to mount the filesystem. If this fails then either the kernel # isn't aware of fs-verity, or the mkfs options were not compatible with - # verity (e.g. ext4 with block size != PAGE_SIZE). + # verity (e.g. ext4 with block size != PAGE_SIZE on old kernels). if ! _try_scratch_mount &>>$seqres.full; then _notrun "kernel is unaware of $FSTYP verity feature," \ "or mkfs options are not compatible with verity"
@@ -39,6 +44,27 @@ _require_scratch_verity() _notrun "kernel $FSTYP isn't configured with verity support" fi + # Select a default Merkle tree block size for when tests don't + # explicitly specify one. + # + # For consistency reasons, all 'fsverity' subcommands, including + # 'fsverity enable', default to 4K Merkle tree blocks. That's generally + # not ideal for tests, since it's possible that the filesystem doesn't + # support 4K blocks but does support another size. Specifically, the + # kernel originally supported only merkle_tree_block_size == + # fs_block_size == page_size, and later it was updated to support + # merkle_tree_block_size <= min(fs_block_size, page_size). + # + # Therefore, we default to merkle_tree_block_size == min(fs_block_size, + # page_size). That maximizes the chance of verity actually working. + local fs_block_size=$(_get_block_size $SCRATCH_MNT) + local page_size=$(get_page_size) + if (( fs_block_size <= page_size )); then + FSV_BLOCK_SIZE=$fs_block_size + else + FSV_BLOCK_SIZE=$page_size + fi + # The filesystem may have fs-verity enabled but not actually usable by # default. E.g., ext4 only supports verity on extent-based files, so it # doesn't work on ext3-style filesystems. So, try actually using it.
@@ -47,10 +73,6 @@ _require_scratch_verity() fi _scratch_unmount - - # Merkle tree block size. Currently all filesystems only support - # PAGE_SIZE for this. This is also the default for 'fsverity enable'. - FSV_BLOCK_SIZE=$(get_page_size) } # Check for CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y, as well as the userspace
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 10/10 generic/575: test 1K Merkle tree block size
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
In addition to 4K, test 1K Merkle tree blocks. Also always run this test, regardless of FSV_BLOCK_SIZE, but allow skipping tests of parameters that are unsupported, unless they are the default.
--- tests/generic/575 | 57 +++++++++++++++++++++++++++++++------------ tests/generic/575.out | 8 ++++-- 2 files changed, 47 insertions(+), 18 deletions(-)
tests/generic/575 +41 -16
@@ -4,7 +4,7 @@ # # FS QA Test generic/575 # -# Test that fs-verity is using the correct measurement values. This test +# Test that fs-verity is using the correct file digest values. This test # verifies that fs-verity is doing its Merkle tree-based hashing correctly, # i.e. that it hasn't been broken by a change. #
@@ -26,9 +26,6 @@ _cleanup() # real QA test starts here _supported_fs generic _require_scratch_verity -if [ $FSV_BLOCK_SIZE != 4096 ]; then - _notrun "4096-byte verity block size not supported on this platform" -fi _disable_fsverity_signatures _scratch_mkfs_verity &>> $seqres.full
@@ -36,24 +33,42 @@ _scratch_mount fsv_orig_file=$SCRATCH_MNT/file fsv_file=$SCRATCH_MNT/file.fsv +# Try multiple hash algorithms. algs=(sha256 sha512) +# Try multiple Merkle tree block sizes. +block_sizes=(1024 4096) + # Try files with 0, 1, and multiple Merkle tree levels. file_sizes=(0 4096 65536 65536 100000000) # Try both unsalted and salted, and check that empty salt is the same as no salt salts=('' '' '' '--salt=' '--salt=f3c93fa6fb828c0e1587e5714ecf6f56') -# The expected file measurements are here rather than in the expected output -# file because not all hash algorithms may be available. -sha256_vals=( +# The expected file digests are here rather than in the expected output file +# because the kernel might not support all hash algorithms and block sizes. +sha256_vals_bsize1024=( +sha256:f2cca36b9b1b7f07814e4284b10121809133e7cb9c4528c8f6846e85fc624ffa +sha256:ea08590a4fe9c3d6c9dafe0eedacd9dffff8f24e24f1865ee3af132a495ab087 +sha256:527496288d703686e31092f5cca7e1306b2467f00b247ad01056ee5ec35a4bb9 +sha256:527496288d703686e31092f5cca7e1306b2467f00b247ad01056ee5ec35a4bb9 +sha256:087818b23312acb15dff9ff6e2b4f601406d08bb36013542444cc15248f47016 +) +sha256_vals_bsize4096=( sha256:3d248ca542a24fc62d1c43b916eae5016878e2533c88238480b26128a1f1af95 sha256:babc284ee4ffe7f449377fbf6692715b43aec7bc39c094a95878904d34bac97e sha256:011e3f2b1dc89b75d78cddcc2a1b85cd8a64b2883e5f20f277ae4c0617e0404f sha256:011e3f2b1dc89b75d78cddcc2a1b85cd8a64b2883e5f20f277ae4c0617e0404f sha256:9d33cab743468fcbe4edab91a275b30dd543c12dd5e6ce6f2f737f66a1558f06 ) -sha512_vals=( +sha512_vals_bsize1024=( +sha512:8451664f25b2ad3f24391280e0c5681cb843389c180baa719f8fdfb063f5ddfa2d1c4433e55e2b6fbb3ba6aa2df8a4f41bf56cb7e0a3b617b6919a42c80f034c +sha512:ab3c6444ab377bbe54c604c26cad241b146d85dc29727703a0d8134f70a8172fb3fa67d171355106b69cc0a9e7e9debb335f9461b3aba44093914867f7c73233 +sha512:e6a11353c24dd7b4603cb8ffa50a7041dbea7382d4698474ccbc2d8b34f3a83d8bf16df25c64ed31ee27213a8a3cbd001fb1ccde46384c23b81305c2393c1046 +sha512:e6a11353c24dd7b4603cb8ffa50a7041dbea7382d4698474ccbc2d8b34f3a83d8bf16df25c64ed31ee27213a8a3cbd001fb1ccde46384c23b81305c2393c1046 +sha512:517d573bd50d5f3787f5766c2ac60c7af854b0901b69757b4ef8dd70aa6b30fcc10d81629ce923bdd062a01c20fad0f063a081a2f3b0814ac06547b26bedc0d9 +) +sha512_vals_bsize4096=( sha512:ccf9e5aea1c2a64efa2f2354a6024b90dffde6bbc017825045dce374474e13d10adb9dadcc6ca8e17a3c075fbd31336e8f266ae6fa93a6c3bed66f9e784e5abf sha512:928922686c4caf32175f5236a7f964e9925d10a74dc6d8344a8bd08b23c228ff5792573987d7895f628f39c4f4ebe39a7367d7aeb16aaa0cd324ac1d53664e61 sha512:eab7224ce374a0a4babcb2db25e24836247f38b87806ad9be9e5ba4daac2f5b814fc0cbdfd9f1f8499b3c9a6c1b38fe08974cce49883ab4ccd04462fd2f9507f
@@ -61,20 +76,27 @@ sha512:eab7224ce374a0a4babcb2db25e24836247f38b87806ad9be9e5ba4daac2f5b814fc0cbdf sha512:f7083a38644880d25539488313e9e5b41a4d431a0e383945129ad2c36e3c1d0f28928a424641bb1363c12b6e770578102566acea73baf1ce8ee15336f5ba2446 ) -test_alg() +test_alg_with_block_size() { local alg=$1 - local -n vals=${alg}_vals + local block_size=$2 + local -n vals=${alg}_vals_bsize${block_size} local i local file_size local expected actual salt_arg - _fsv_scratch_begin_subtest "Check for expected measurement values ($alg)" + _fsv_scratch_begin_subtest "Testing alg=$alg, block_size=$block_size if supported" - if ! _fsv_can_enable $fsv_file --hash-alg=$alg; then - if [ "$alg" = sha256 ]; then - _fail "Something is wrong - sha256 hash should always be available" + if ! _fsv_can_enable $fsv_file --hash-alg=$alg --block-size=$block_size + then + # Since this is after _require_scratch_verity, sha256 with + # FSV_BLOCK_SIZE must be supported. + if [ "$alg" = "sha256" -a "$block_size" = "$FSV_BLOCK_SIZE" ] + then + _fail "Failed to enable verity with default params" fi + # This combination of parameters is unsupported; skip it. + echo "alg=$alg, block_size=$block_size is unsupported" >> $seqres.full return 0 fi
@@ -85,7 +107,8 @@ test_alg() head -c $file_size /dev/zero > $fsv_orig_file cp $fsv_orig_file $fsv_file - _fsv_enable --hash-alg=$alg $salt_arg $fsv_file + _fsv_enable $fsv_file --hash-alg=$alg --block-size=$block_size \ + $salt_arg actual=$(_fsv_measure $fsv_file) if [ "$actual" != "$expected" ]; then echo "Mismatch: expected $expected, kernel calculated $actual (file_size=$file_size)"
@@ -96,7 +119,9 @@ test_alg() } for alg in ${algs[@]}; do - test_alg $alg + for block_size in ${block_sizes[@]}; do + test_alg_with_block_size $alg $block_size + done done # success, all done
tests/generic/575.out +6 -2
@@ -1,5 +1,9 @@ QA output created by 575 -# Check for expected measurement values (sha256) +# Testing alg=sha256, block_size=1024 if supported -# Check for expected measurement values (sha512) +# Testing alg=sha256, block_size=4096 if supported + +# Testing alg=sha512, block_size=1024 if supported + +# Testing alg=sha512, block_size=4096 if supported
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 8/10 generic/574: test multiple Merkle tree block sizes
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Instead of only testing 4K Merkle tree blocks, test FSV_BLOCK_SIZE, and also other sizes if they happen to be supported. This allows this test to run in cases where it couldn't before and improves test coverage in cases where it did run before.
Given that the list of Merkle tree block sizes that will actually work is not fixed, this required reworking the test to not rely on the .out file so heavily.
--- tests/generic/574 | 219 +++++++++++++++++++++++++----------------- tests/generic/574.out | 83 +--------------- 2 files changed, 136 insertions(+), 166 deletions(-)
tests/generic/574 +131 -88
@@ -37,23 +37,19 @@ fsv_file=$SCRATCH_MNT/file.fsv setup_zeroed_file() { - local len=$1 - local sparse=$2 + local block_size=$1 + local file_len=$2 + local sparse=$3 if $sparse; then - dd if=/dev/zero of=$fsv_orig_file bs=1 count=0 seek=$len \ + dd if=/dev/zero of=$fsv_orig_file bs=1 count=0 seek=$file_len \ status=none else - head -c $len /dev/zero > $fsv_orig_file + head -c $file_len /dev/zero > $fsv_orig_file fi cp $fsv_orig_file $fsv_file - _fsv_enable $fsv_file - md5sum $fsv_file |& _filter_scratch -} - -filter_sigbus() -{ - sed -e 's/.*Bus error.*/Bus error/' + _fsv_enable $fsv_file --block-size=$block_size + cmp $fsv_orig_file $fsv_file } round_up_to_page_boundary()
@@ -64,65 +60,100 @@ round_up_to_page_boundary() echo $(( (n + page_size - 1) & ~(page_size - 1) )) } +mread() +{ + local file=$1 + local offset=$2 + local length=$3 + local map_len=$(round_up_to_page_boundary $(_get_filesize $file)) + + # Some callers expect xfs_io to crash with SIGBUS due to the mread, + # causing the shell to print "Bus error" to stderr. To allow this + # message to be redirected, execute xfs_io in a new shell instance. + # However, for this to work reliably, we also need to prevent the new + # shell instance from optimizing out the fork and directly exec'ing + # xfs_io. The easiest way to do that is to append 'true' to the + # commands, so that xfs_io is no longer the last command the shell sees. + bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $file \ + -c 'mmap -r 0 $map_len' \ + -c 'mread -v $offset $length'; true" +} + corruption_test() { - local file_len=$1 - local zap_offset=$2 - local zap_len=$3 - local is_merkle_tree=${4:-false} # if true, zap tree instead of data - local use_sparse_file=${5:-false} - local page_aligned_eof=$(round_up_to_page_boundary $file_len) - local measurement + local block_size=$1 + local file_len=$2 + local zap_offset=$3 + local zap_len=$4 + local is_merkle_tree=${5:-false} # if true, zap tree instead of data + local use_sparse_file=${6:-false} + + local paramstr="block_size=$block_size" + paramstr+=", file_len=$file_len" + paramstr+=", zap_offset=$zap_offset" + paramstr+=", zap_len=$zap_len" + paramstr+=", is_merkle_tree=$is_merkle_tree" + paramstr+=", use_sparse_file=$use_sparse_file" if $is_merkle_tree; then local corrupt_func=_fsv_scratch_corrupt_merkle_tree else local corrupt_func=_fsv_scratch_corrupt_bytes fi + local fs_block_size=$(_get_block_size $SCRATCH_MNT) - local msg="Corruption test:" - msg+=" file_len=$file_len" - if $use_sparse_file; then - msg+=" (sparse)" - fi - msg+=" zap_offset=$zap_offset" - if $is_merkle_tree; then - msg+=" (in Merkle tree)" - fi - msg+=" zap_len=$zap_len" + rm -rf "${SCRATCH_MNT:?}"/* + setup_zeroed_file $block_size $file_len $use_sparse_file - _fsv_scratch_begin_subtest "$msg" - setup_zeroed_file $file_len $use_sparse_file - cmp $fsv_file $fsv_orig_file - echo "Corrupting bytes..." + # Corrupt part of the file (data or Merkle tree). head -c $zap_len /dev/zero | tr '\0' X \ | $corrupt_func $fsv_file $zap_offset - echo "Validating corruption (reading full file)..." + # Reading the full file with buffered I/O should fail. _scratch_cycle_mount - md5sum $fsv_file |& _filter_scratch + if cat $fsv_file >/dev/null 2>$tmp.err; then + echo "Unexpectedly was able to read full file ($paramstr)" + elif ! grep -q 'Input/output error' $tmp.err; then + echo "Wrong error reading full file ($paramstr):" + cat $tmp.err + fi - echo "Validating corruption (direct I/O)..." + # Reading the full file with direct I/O should fail. _scratch_cycle_mount - dd if=$fsv_file bs=$FSV_BLOCK_SIZE iflag=direct status=none \ - of=/dev/null |& _filter_scratch + if dd if=$fsv_file bs=$fs_block_size iflag=direct status=none \ + of=/dev/null 2>$tmp.err + then + echo "Unexpectedly was able to read full file with DIO ($paramstr)" + elif ! grep -q 'Input/output error' $tmp.err; then + echo "Wrong error reading full file with DIO ($paramstr):" + cat $tmp.err + fi - if (( zap_offset < file_len )) && ! $is_merkle_tree; then - echo "Validating corruption (reading just corrupted part)..." - dd if=$fsv_file bs=1 skip=$zap_offset count=$zap_len \ - of=/dev/null status=none |& _filter_scratch + # Reading just the corrupted part of the file should fail. + if ! $is_merkle_tree; then + if dd if=$fsv_file bs=1 skip=$zap_offset count=$zap_len \ + of=/dev/null status=none 2>$tmp.err; then + echo "Unexpectedly was able to read corrupted part ($paramstr)" + elif ! grep -q 'Input/output error' $tmp.err; then + echo "Wrong error reading corrupted part ($paramstr):" + cat $tmp.err + fi fi - echo "Validating corruption (reading full file via mmap)..." - bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ - -c 'mmap -r 0 $page_aligned_eof' \ - -c 'mread 0 $file_len'" |& filter_sigbus + # Reading the full file via mmap should fail. + mread $fsv_file 0 $file_len >/dev/null 2>$tmp.err + if ! grep -q 'Bus error' $tmp.err; then + echo "Didn't see SIGBUS when reading file via mmap" + cat $tmp.err + fi + # Reading just the corrupted part via mmap should fail. if ! $is_merkle_tree; then - echo "Validating corruption (reading just corrupted part via mmap)..." - bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ - -c 'mmap -r 0 $page_aligned_eof' \ - -c 'mread $zap_offset $zap_len'" |& filter_sigbus + mread $fsv_file $zap_offset $zap_len >/dev/null 2>$tmp.err + if ! grep -q 'Bus error' $tmp.err; then + echo "Didn't see SIGBUS when reading corrupted part via mmap" + cat $tmp.err + fi fi }
@@ -131,52 +162,64 @@ corruption_test() # return zeros in the last block past EOF, regardless of the contents on # disk. In the former, corruption should be detected and result in SIGBUS, # while in the latter we would expect zeros past EOF, but no error. -corrupt_eof_block_test() { - local file_len=$1 - local zap_len=$2 - local page_aligned_eof=$(round_up_to_page_boundary $file_len) - _fsv_scratch_begin_subtest "Corruption test: EOF block" - setup_zeroed_file $file_len false - cmp $fsv_file $fsv_orig_file - echo "Corrupting bytes..." +corrupt_eof_block_test() +{ + local block_size=$1 + local file_len=$2 + local zap_len=$3 + + rm -rf "${SCRATCH_MNT:?}"/* + setup_zeroed_file $block_size $file_len false head -c $zap_len /dev/zero | tr '\0' X \ | _fsv_scratch_corrupt_bytes $fsv_file $file_len - echo "Reading eof block via mmap into a temporary file..." - bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ - -c 'mmap -r 0 $page_aligned_eof' \ - -c 'mread -v $file_len $zap_len'" \ - |& filter_sigbus >$tmp.eof_block_read 2>&1 - - head -c $file_len /dev/zero > $tmp.zero_cmp_file - $XFS_IO_PROG -r $tmp.zero_cmp_file \ - -c "mmap -r 0 $page_aligned_eof" \ - -c "mread -v $file_len $zap_len" >$tmp.eof_zero_read - - echo "Checking for SIGBUS or zeros..." - grep -q -e '^Bus error$' $tmp.eof_block_read \ - || diff $tmp.eof_block_read $tmp.eof_zero_read \ - && echo "OK" -} - -# Note: these tests just overwrite some bytes without checking their original -# values. Therefore, make sure to overwrite at least 5 or so bytes, to make it -# nearly guaranteed that there will be a change -- even when the test file is -# encrypted due to the test_dummy_encryption mount option being specified. + mread $fsv_file $file_len $zap_len >$tmp.out 2>$tmp.err -corruption_test 131072 0 5 -corruption_test 131072 4091 5 -corruption_test 131072 65536 65536 -corruption_test 131072 131067 5 + head -c $file_len /dev/zero >$tmp.zeroes + mread $tmp.zeroes $file_len $zap_len >$tmp.zeroes_out -corrupt_eof_block_test 130999 72 + grep -q 'Bus error' $tmp.err || diff $tmp.out $tmp.zeroes_out +} -# Merkle tree corruption. -corruption_test 200000 100 10 true +test_block_size() +{ + local block_size=$1 + + # Note: these tests just overwrite some bytes without checking their + # original values. Therefore, make sure to overwrite at least 5 or so + # bytes, to make it nearly guaranteed that there will be a change -- + # even when the test file is encrypted due to the test_dummy_encryption + # mount option being specified. + corruption_test $block_size 131072 0 5 + corruption_test $block_size 131072 4091 5 + corruption_test $block_size 131072 65536 65536 + corruption_test $block_size 131072 131067 5 + + corrupt_eof_block_test $block_size 130999 72 + + # Merkle tree corruption. + corruption_test $block_size 200000 100 10 true + + # Sparse file. Corrupting the Merkle tree should still cause reads to + # fail, i.e. the filesystem must verify holes. + corruption_test $block_size 200000 100 10 true true +} -# Sparse file. Corrupting the Merkle tree should still cause reads to fail, -# i.e. the filesystem must verify holes. -corruption_test 200000 100 10 true true +# Always test FSV_BLOCK_SIZE. Also test some other block sizes if they happen +# to be supported. +_fsv_scratch_begin_subtest "Testing block_size=FSV_BLOCK_SIZE" +test_block_size $FSV_BLOCK_SIZE +for block_size in 1024 4096 16384 65536; do + _fsv_scratch_begin_subtest "Testing block_size=$block_size if supported" + if (( block_size == FSV_BLOCK_SIZE )); then + continue # Skip redundant test case. + fi + if ! _fsv_can_enable $fsv_file --block-size=$block_size; then + echo "block_size=$block_size is unsupported" >> $seqres.full + continue + fi + test_block_size $block_size +done # success, all done status=0
tests/generic/574.out +5 -78
89 lines not on this page · raw message
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 1/10 common/verity: add and use _fsv_can_enable()
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Replace _fsv_have_hash_algorithm() with a more general function _fsv_can_enable() which checks whether 'fsverity enable' with the given parameters works. For now it is just used with --hash-alg or with no parameters, but soon it will be used with --block-size too.
--- common/verity | 17 ++++++----------- tests/generic/575 | 2 +- tests/generic/577 | 2 +- 3 files changed, 8 insertions(+), 13 deletions(-)
common/verity +6 -11
@@ -42,13 +42,7 @@ _require_scratch_verity() # The filesystem may have fs-verity enabled but not actually usable by # default. E.g., ext4 only supports verity on extent-based files, so it # doesn't work on ext3-style filesystems. So, try actually using it. - echo foo > $SCRATCH_MNT/tmpfile - _disable_fsverity_signatures - _fsv_enable $SCRATCH_MNT/tmpfile - local status=$? - _restore_prev_fsverity_signatures - rm -f $SCRATCH_MNT/tmpfile - if (( $status != 0 )); then + if ! _fsv_can_enable $SCRATCH_MNT/tmpfile; then _notrun "$FSTYP verity isn't usable by default with these mkfs options" fi
@@ -256,15 +250,16 @@ _fsv_create_enable_file() _fsv_enable "$file" "$@" } -_fsv_have_hash_algorithm() +_fsv_can_enable() { - local hash_alg=$1 - local test_file=$2 + local test_file=$1 + shift + local params=("$@") _disable_fsverity_signatures rm -f $test_file head -c 4096 /dev/zero > $test_file - _fsv_enable --hash-alg=$hash_alg $test_file &>> $seqres.full + _fsv_enable $test_file "${params[@]}" &>> $seqres.full local status=$? _restore_prev_fsverity_signatures rm -f $test_file
tests/generic/575 +1 -1
@@ -71,7 +71,7 @@ test_alg() _fsv_scratch_begin_subtest "Check for expected measurement values ($alg)" - if ! _fsv_have_hash_algorithm $alg $fsv_file; then + if ! _fsv_can_enable $fsv_file --hash-alg=$alg; then if [ "$alg" = sha256 ]; then _fail "Something is wrong - sha256 hash should always be available" fi
tests/generic/577 +1 -1
@@ -112,7 +112,7 @@ _fsv_enable $fsv_file --signature=$sigfile.salted --salt=abcd cmp $fsv_file $fsv_orig_file echo -e "\n# Testing non-default hash algorithm" -if _fsv_have_hash_algorithm sha512 $fsv_file; then +if _fsv_can_enable $fsv_file --hash-alg=sha512; then reset_fsv_file _fsv_sign $fsv_orig_file $sigfile.sha512 --key=$keyfile \ --cert=$certfile --hash-alg=sha512 > /dev/null
Signature · 1 line
2.39.0
Eric Biggers
maintainerPATCH v3 3/10 common/verity: use FSV_BLOCK_SIZE by default
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Make _fsv_enable() and _fsv_sign() default to FSV_BLOCK_SIZE if no block size is explicitly specified, so that the individual tests don't have to do this themselves. This overrides the fsverity-utils default of 4096 bytes, or the page size in older versions of fsverity-utils, both of which may differ from FSV_BLOCK_SIZE.
--- common/verity | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
common/verity +14 -2
@@ -249,7 +249,13 @@ _fsv_dump_signature() _fsv_enable() { - $FSVERITY_PROG enable "$@" + local args=("$@") + # If the caller didn't explicitly specify a Merkle tree block size, then + # use FSV_BLOCK_SIZE. + if ! [[ " $*" =~ " --block-size" ]]; then + args+=("--block-size=$FSV_BLOCK_SIZE") + fi + $FSVERITY_PROG enable "${args[@]}" } _fsv_measure()
@@ -259,7 +265,13 @@ _fsv_measure() _fsv_sign() { - $FSVERITY_PROG sign "$@" + local args=("$@") + # If the caller didn't explicitly specify a Merkle tree block size, then + # use FSV_BLOCK_SIZE. + if ! [[ " $*" =~ " --block-size" ]]; then + args+=("--block-size=$FSV_BLOCK_SIZE") + fi + $FSVERITY_PROG sign "${args[@]}" } # Generate a file, then enable verity on it.
Signature · 1 line
2.39.0
Quoted 7 lines · Eric Biggers
This series updates the verity xfstests to eliminate implicit assumptions that '
merkle_tree_block_size==fs_block_size==page_size== 4096', and to provide some test coverage for cases wheremerkle_tree_block_sizediffers fromfs_block_sizeand/orpage_size. It doesn't add any new test scripts, but it does update some of the existing test scripts to test multiple block sizes.
Quoted 8 lines
This goes along with my kernel patch series
"fsverity: support for non-4K pages"
(https://lore.kernel.org/linux-fsdevel/[email protected]/T/#u).
However, it's not necessary to wait for that kernel patch series to be applied before applying this xfstests patch series.Changed in v3:
- Fixed generic/574 failure with some bash versions.
Thanks for fixing this failure, this version looks good to me. I'd like to merge this patchset with my RVB.
Quoted 36 lines
Changed in v2:
- Adjusted the output of generic/574, generic/575, and generic/624 slightly to avoid confusion.Eric Biggers (10):
common/verity: add and use_fsv_can_enable()common/verity: setFSV_BLOCK_SIZEto an appropriate value common/verity: useFSV_BLOCK_SIZEby default common/verity: add_filter_fsverity_digest()generic/572: support non-4K Merkle tree block size generic/573: support non-4K Merkle tree block size generic/577: support non-4K Merkle tree block size generic/574: test multiple Merkle tree block sizes generic/624: test multiple Merkle tree block sizes generic/575: test 1K Merkle tree block sizecommon/verity | 84 ++++++++++++---- tests/generic/572 | 21 ++-- tests/generic/572.out | 10 +- tests/generic/573 | 8 +- tests/generic/574 | 219 +++++++++++++++++++++++++----------------- tests/generic/574.out | 83 +--------------- tests/generic/575 | 57 ++++++++--- tests/generic/575.out | 8 +- tests/generic/577 | 24 ++--- tests/generic/577.out | 10 +- tests/generic/624 | 119 ++++++++++++++++------- tests/generic/624.out | 15 +-- 12 files changed, 370 insertions(+), 288 deletions(-)base-commit:
3dc46f477b39d732e1841e6f5a180759cee3e8ceSignature · 1 line
2.39.0