xfstests: add fs-verity tests
Eric Biggers
maintainercover xfstests: add fs-verity tests
Add tests for fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running these tests requires:
- A kernel with the fs-verity patches from
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/log/
(should be merged in 4.21) and configured with CONFIG_FS_VERITY.
- The fsverity utility program, which can be installed from
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git/
- e2fsprogs v1.44.4-2 or later for ext4 tests, or f2fs-tools v1.11.0 or later for f2fs tests.
Example with kvm-xfstests:
$ kvm-xfstests -c ext4,f2fs -g verity
See the file Documentation/filesystem/fsverity.rst in the kernel tree for more information about fs-verity.
Eric Biggers (7):
common/verity: add common functions for testing fs-verity
generic: test general behavior of verity files
generic: test access controls on the fs-verity ioctls
generic: test fs-verity descriptor validation
generic: test corrupting verity files
generic: test that fs-verity is using the correct measurement values
generic: test using fs-verity and fscrypt simultaneously
common/config | 1 + common/verity | 198 ++++++++++++++++++++++++++ tests/generic/900 | 165 +++++++++++++++++++++ tests/generic/900.out | 54 +++++++ tests/generic/901 | 71 ++++++++++ tests/generic/901.out | 14 ++ tests/generic/902 | 323 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/902.out | 125 ++++++++++++++++ tests/generic/903 | 126 ++++++++++++++++ tests/generic/903.out | 91 ++++++++++++ tests/generic/904 | 124 ++++++++++++++++ tests/generic/904.out | 7 + tests/generic/905 | 81 +++++++++++ tests/generic/905.out | 12 ++ tests/generic/group | 6 + 15 files changed, 1398 insertions(+) create mode 100644 common/verity create mode 100755 tests/generic/900 create mode 100644 tests/generic/900.out create mode 100755 tests/generic/901 create mode 100644 tests/generic/901.out create mode 100755 tests/generic/902 create mode 100644 tests/generic/902.out create mode 100755 tests/generic/903 create mode 100644 tests/generic/903.out create mode 100755 tests/generic/904 create mode 100644 tests/generic/904.out create mode 100755 tests/generic/905 create mode 100644 tests/generic/905.out
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 1/7 common/verity: add common functions for testing fs-verity
From: Eric Biggers <[email protected]>
Add common functions for setting up and testing fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running the fs-verity tests requires:
- A kernel with the fs-verity patches from
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/log/
(should be merged in 4.21) and configured with CONFIG_FS_VERITY.
- The fsverity utility program, which can be installed from
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git/
- e2fsprogs v1.44.4-2 or later for ext4 tests, or f2fs-tools v1.11.0 or later for f2fs tests.
See the file Documentation/filesystem/fsverity.rst in the kernel tree for more information about fs-verity.
--- common/config | 1 + common/verity | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 199 insertions(+) create mode 100644 common/verity
common/config +1 -0
@@ -194,6 +194,7 @@ export GETCAP_PROG="$(type -P getcap)" export CHECKBASHISMS_PROG="$(type -P checkbashisms)" export XFS_INFO_PROG="$(type -P xfs_info)" export DUPEREMOVE_PROG="$(type -P duperemove)" +export FSVERITY_PROG="$(type -P fsverity)" # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled. # newer systems have udevadm command but older systems like RHEL5 don't.
common/verity new file +198 -0
198 lines not on this page · raw message
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 2/7 generic: test general behavior of verity files
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
This is a basic fs-verity test which verifies:
- conditions for enabling verity
- verity files have expected contents and size
- can't write to verity files
- can retrieve a verity file's measurement via FS_IOC_MEASURE_VERITY
--- tests/generic/900 | 165 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/900.out | 54 ++++++++++++++ tests/generic/group | 1 + 3 files changed, 220 insertions(+) create mode 100755 tests/generic/900 create mode 100644 tests/generic/900.out
tests/generic/900 new file +165 -0
@@ -0,0 +1,165 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/900 +# +# This is a basic fs-verity test which verifies: +# +# - conditions for enabling verity +# - verity files have correct contents and size +# - can't write to verity files +# - can retrieve a verity file's measurement via FS_IOC_MEASURE_VERITY +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_orig_file=$SCRATCH_MNT/file +fsv_file=$SCRATCH_MNT/file.fsv + +verify_data_readable() +{ + local file=$1 + + md5sum $file > /dev/null +} + +verify_data_unreadable() +{ + local file=$1 + + # try both reading just the first data block, and reading until EOF + head -c $FSV_BLOCK_SIZE $file 2>&1 >/dev/null | _filter_scratch + md5sum $file |& _filter_scratch +} + +_fsv_begin_subtest "Enabling fs-verity on directory fails with EISDIR" +mkdir $SCRATCH_MNT/dir +_fsv_enable $SCRATCH_MNT/dir |& _filter_scratch + +_fsv_begin_subtest "Enabling fs-verity on file open for writing fails with ETXTBSY" +_fsv_create_setup_file $fsv_file >> $seqres.full +exec 3<> "$fsv_file" +_fsv_enable $fsv_file |& _filter_scratch +exec 3<&- +verify_data_readable $fsv_file + +_fsv_begin_subtest "Enabling fs-verity on file on read-only filesystem fails with EROFS" +_fsv_create_setup_file $fsv_file >> $seqres.full +_scratch_remount ro +_fsv_enable $fsv_file |& _filter_scratch +_scratch_remount rw + +_fsv_begin_subtest "Enabling fs-verity on a file twice fails with EEXIST" +_fsv_create_setup_file $fsv_file >> $seqres.full +_fsv_enable $fsv_file +echo "(trying again)" +_fsv_enable $fsv_file |& _filter_scratch + +_fsv_begin_subtest "fs-verity file can't be opened for writing" +_fsv_create_enable_file $fsv_file >> $seqres.full +echo "* reading" +$XFS_IO_PROG -r $fsv_file -c '' +echo "* xfs_io writing, should be O_RDWR" +$XFS_IO_PROG $fsv_file -c '' |& _filter_scratch +echo "* bash >>, should be O_APPEND" +bash -c "echo >> $fsv_file" |& _filter_scratch +echo "* bash >, should be O_WRONLY|O_CREAT|O_TRUNC" +bash -c "echo > $fsv_file" |& _filter_scratch + +_fsv_begin_subtest "fs-verity file can be read" +_fsv_create_enable_file $fsv_file >> $seqres.full +verify_data_readable $fsv_file + +_fsv_begin_subtest "fs-verity file reports measurement calculated earlier" +head -c 100000 /dev/urandom > $fsv_orig_file +expected_hash=$(_fsv_setup $fsv_orig_file $fsv_file) +_fsv_enable $fsv_file +actual_hash=$(_fsv_measure $fsv_file) +if [ "$expected_hash" != "$actual_hash" ]; then + echo "Measurement changed: $expected_hash => $actual_hash" +fi + +_fsv_begin_subtest "fs-verity file has correct contents" +head -c 100000 /dev/urandom > $fsv_orig_file +_fsv_setup $fsv_orig_file $fsv_file >> $seqres.full +_fsv_enable $fsv_file +cmp $fsv_file $fsv_orig_file + +_fsv_begin_subtest "Trying to measure non-verity file fails with ENODATA" +_fsv_create_setup_file $fsv_file >> $seqres.full +_fsv_measure $fsv_file |& _filter_scratch +verify_data_readable $fsv_file + +_fsv_begin_subtest "fs-verity file has adjusted i_size" +head -c 100000 /dev/zero > $fsv_orig_file +measurement=$(_fsv_setup $fsv_orig_file $fsv_file) +stat -c %s $fsv_file +_fsv_enable $fsv_file +stat -c %s $fsv_file +_scratch_cycle_mount +stat -c %s $fsv_file + +# This verifies that the adjusted i_size is not written to the real on-disk +# i_size, preventing the fs-verity header from being found again. +_fsv_begin_subtest "fs-verity file can still be read after mount cycle" +head -c 100000 /dev/urandom > $fsv_orig_file +measurement=$(_fsv_setup $fsv_orig_file $fsv_file) +_fsv_enable $fsv_file +cmp $fsv_file $fsv_orig_file +echo "(chmod file)" +# make the inode dirty, so it gets written +chmod 444 $fsv_file +chmod 400 $fsv_file +echo "(cycle mount)" +_scratch_cycle_mount +echo "(read file)" +cmp $fsv_file $fsv_orig_file + +# Test files <= 1 block in size. These are a bit of a special case since there +# are no hash blocks, so the root hash has to be calculated over the data block. +for size in 1 4095 4096; do + _fsv_begin_subtest "fs-verity on $size-byte file" + head -c $size /dev/urandom > $fsv_orig_file + measurement=$(_fsv_setup $fsv_orig_file $fsv_file) + _fsv_enable $fsv_file + cmp $fsv_orig_file $fsv_file && echo "Files matched" + rm -f $fsv_file +done + +_fsv_begin_subtest "fs-verity on 100M file (multiple levels in hash tree)" +head -c 100000000 /dev/urandom > $fsv_orig_file +_fsv_setup $fsv_orig_file $fsv_file >> $seqres.full +_fsv_enable $fsv_file +cmp $fsv_orig_file $fsv_file && echo "Files matched" + +# success, all done +status=0 +exit
tests/generic/900.out new file +54 -0
54 lines not on this page · raw message
tests/generic/group +1 -0
@@ -525,3 +525,4 @@ 520 auto quick log 521 soak long_rw 522 soak long_rw +900 auto quick verity
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 3/7 generic: test access controls on the fs-verity ioctls
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Test access controls on the fs-verity ioctls. FS_IOC_MEASURE_VERITY is allowed on any file, whereas FS_IOC_ENABLE_VERITY requires write access.
--- tests/generic/901 | 71 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/901.out | 14 +++++++++ tests/generic/group | 1 + 3 files changed, 86 insertions(+) create mode 100755 tests/generic/901 create mode 100644 tests/generic/901.out
tests/generic/901 new file +71 -0
@@ -0,0 +1,71 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/901 +# +# Test access controls on the fs-verity ioctls. FS_IOC_MEASURE_VERITY is +# allowed on any file, whereas FS_IOC_ENABLE_VERITY requires write access. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity +_require_user + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_file=$SCRATCH_MNT/file.fsv + +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY doesn't require root" +_fsv_create_setup_file $fsv_file >> $seqres.full +chmod 666 $fsv_file +su $qa_user -c "$FSVERITY_PROG enable $fsv_file" + +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY requires write access" +_fsv_create_setup_file $fsv_file >> $seqres.full +chmod 444 $fsv_file +su $qa_user -c "$FSVERITY_PROG enable $fsv_file" |& _filter_scratch + +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY requires !append-only" +_fsv_create_setup_file $fsv_file >> $seqres.full +chattr +a $fsv_file +$FSVERITY_PROG enable $fsv_file |& _filter_scratch +chattr -a $fsv_file + +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY requires !immutable" +_fsv_create_setup_file $fsv_file >> $seqres.full +chattr +i $fsv_file +$FSVERITY_PROG enable $fsv_file |& _filter_scratch +chattr -i $fsv_file + +_fsv_begin_subtest "FS_IOC_MEASURE_VERITY doesn't require root" +_fsv_create_enable_file $fsv_file >> $seqres.full +su $qa_user -c "$FSVERITY_PROG measure $fsv_file" >> $seqres.full + +# success, all done +status=0 +exit
tests/generic/901.out new file +14 -0
@@ -0,0 +1,14 @@ +QA output created by 901 + +# FS_IOC_ENABLE_VERITY doesn't require root + +# FS_IOC_ENABLE_VERITY requires write access +ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Permission denied + +# FS_IOC_ENABLE_VERITY requires !append-only +ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Operation not permitted + +# FS_IOC_ENABLE_VERITY requires !immutable +ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Operation not permitted + +# FS_IOC_MEASURE_VERITY doesn't require root
tests/generic/group +1 -0
@@ -526,3 +526,4 @@ 521 soak long_rw 522 soak long_rw 900 auto quick verity +901 auto quick verity
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 4/7 generic: test fs-verity descriptor validation
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
This test tries corrupting various fields in the fsverity_descriptor and verifies that this causes FS_IOC_ENABLE_VERITY to fail.
--- tests/generic/902 | 323 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/902.out | 125 ++++++++++++++++ tests/generic/group | 1 + 3 files changed, 449 insertions(+) create mode 100755 tests/generic/902 create mode 100644 tests/generic/902.out
tests/generic/902 new file +323 -0
@@ -0,0 +1,323 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/902 +# +# Test fs-verity descriptor validation. This test tries corrupting various +# fields in the fsverity_descriptor and verifies that this causes +# FS_IOC_ENABLE_VERITY to fail. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_orig_file=$SCRATCH_MNT/file +fsv_file=$SCRATCH_MNT/file.fsv + +# Serialize an integer into a little endian hex bytestring of the given length, +# e.g. `num_to_hex 1000 4` == "\xe8\x03\x00\x00" +num_to_hex() +{ + local value=$1 + local nbytes=$2 + local i + + for (( i = 0; i < nbytes; i++, value >>= 8 )); do + printf '\\x%02x' $((value & 0xff)) + done +} + +# Number of bytes in a hex bytestring, e.g. `hexstr_len "\xe8\x03"` == 2 +hexstr_len() +{ + echo -n -e "$1" | wc -c +} + +# Get a file's SHA-256 digest as a hex bytestring for "echo -e" +do_sha256sum() +{ + sha256sum "$@" | awk '{print $1}' | sed 's/../\\x\0/g' +} + +# Append a field to 'str', allowing override commands. See make_test_file(). +do_add_field() +{ + local fixed_length=$1 + local -n str=$2 + local fieldname=$3 + local default_val=$4 + shift 4 + local cmds=("$@") + local cmd + local val="$default_val" + + for cmd in "${cmds[@]}"; do + if [ $(echo "$cmd" | cut -d' ' -f1) = $fieldname ]; then + val=$(echo "$cmd" | sed "s/^$fieldname *//") + if $fixed_length && \ + [ $(hexstr_len "$val") != $(hexstr_len "$default_val") ] + then + _fail "wrong value length in '$cmd'" + fi + break + fi + done + str+="$val" +} + +add_field() +{ + do_add_field true "$@" +} + +add_varfield() +{ + do_add_field false "$@" +} + +FS_VERITY_EXT_ROOT_HASH=1 +FS_VERITY_EXT_SALT=2 +FS_VERITY_EXT_PKCS7_SIGNATURE=3 +FS_VERITY_EXT_ELIDE=4 +FS_VERITY_EXT_PATCH=5 + +EXTHDR_SIZE=8 + +# Create an extension header (struct fsverity_extension) +create_exthdr() +{ + local length=$1 + local type=$2 + if [ $# -ge 3 ]; then + local reserved=$3 + else + local reserved=0 + fi + + num_to_hex $length 4 + num_to_hex $type 2 + num_to_hex $reserved 2 +} + +# Create an extension item, given the type and payload +create_ext() +{ + local type=$1 + local payload=$2 + local payload_size=$(hexstr_len "$payload") + + create_exthdr $(( EXTHDR_SIZE + payload_size )) $type + echo -n "$payload" + num_to_hex 0 $(( -payload_size & 7 )) +} + +# Create a ROOT_HASH extension item +create_root_hash_ext() +{ + local root_hash=$1 + + create_ext $FS_VERITY_EXT_ROOT_HASH "$root_hash" +} + +DEFAULT_AUTH_EXT_COUNT=1 # root hash +DEFAULT_UNAUTH_EXT_COUNT=0 # none + +# +# Generate a file and append fs-verity metadata to it, allowing metadata fields +# to be overridden. The overrides are given as command strings in the format +# "$field $value". E.g., "major_version \x01" sets the major_version field to +# the byte \x01 (binary 1, not ASCII 1). For fixed-length fields (add_field()) +# the override must be the same length as the default value; for variable-length +# fields (add_varfield()) the override can be any length. +# +make_test_file() +{ + local cmds=("$@") + local out="" + + # 8 KiB file + head -c 8192 /dev/urandom > $fsv_orig_file + cp $fsv_orig_file $fsv_file + + # Generate the Merkle tree.. there are just 2 data blocks, so it's easy. + local hash1=$(head -c 4096 $fsv_file | do_sha256sum) + local hash2=$(tail -c 4096 $fsv_file | do_sha256sum) + echo -n -e "$hash1" >> $fsv_file + echo -n -e "$hash2" >> $fsv_file + head -c $((4096 - (32*2))) /dev/zero >> $fsv_file + local root_hash=$(tail -c 4096 $fsv_file | do_sha256sum) + + # Append the 'struct fsverity_descriptor' + add_field out magic "FSVerity" "${cmds[@]}" + add_field out major_version "\x01" "${cmds[@]}" + add_field out minor_version "\x00" "${cmds[@]}" + add_field out log_data_blocksize "$(num_to_hex 12 1)" "${cmds[@]}" # 4K block size + add_field out log_tree_blocksize "$(num_to_hex 12 1)" "${cmds[@]}" + add_field out data_algorithm "$(num_to_hex 1 2)" "${cmds[@]}" # SHA-256 + add_field out tree_algorithm "$(num_to_hex 1 2)" "${cmds[@]}" + add_field out flags "$(num_to_hex 0 4)" "${cmds[@]}" + add_field out reserved1 "$(num_to_hex 0 4)" "${cmds[@]}" + add_field out orig_file_size "$(num_to_hex 8192 8)" "${cmds[@]}" + add_field out auth_ext_count "$(num_to_hex $DEFAULT_AUTH_EXT_COUNT 2)" "${cmds[@]}" + add_field out reserved2 "$(num_to_hex 0 30)" "${cmds[@]}" + + # Append the authenticated extensions (default: just the root hash) + add_varfield out root_hash "$(create_root_hash_ext "$root_hash")" "${cmds[@]}" + add_varfield out auth_extensions "" "${cmds[@]}" + + # Append the unauthenticated extensions (default: none) + add_field out unauth_ext_count "$(num_to_hex $DEFAULT_UNAUTH_EXT_COUNT 2)" "${cmds[@]}" + add_field out unauth_ext_count_padding "$(num_to_hex 0 6)" "${cmds[@]}" + add_varfield out unauth_extensions "" "${cmds[@]}" + + # No gap before the footer by default + add_varfield out gap_before_footer "" "${cmds[@]}" + + # Append the footer + local desc_reverse_offset=$((12 + $(hexstr_len "$out") )) + add_field out desc_reverse_offset "$(num_to_hex $desc_reverse_offset 4)" "${cmds[@]}" + add_field out ftr_magic "FSVerity" "${cmds[@]}" + + echo -n -e "$out" >> $fsv_file +} + +desc_test() +{ + local description=$1 + shift + local cmds=("$@") + + _fsv_begin_subtest "$description" + make_test_file "${cmds[@]}" + { + if _fsv_enable $fsv_file; then + cmp $fsv_file $fsv_orig_file + fi + } |& _filter_scratch +} + +ext_count() +{ + local type=$1 + local count=$2 + local default_count=$3 + local sign=${count:0:1} + if [ $sign = '+' ] || [ $sign = '-' ]; then + count=$(( default_count + $count )) + fi + echo "$type $(num_to_hex $count 2)" +} + +auth_ext_count() +{ + ext_count "auth_ext_count" "$1" $DEFAULT_AUTH_EXT_COUNT +} + +unauth_ext_count() +{ + ext_count "unauth_ext_count" "$1" $DEFAULT_UNAUTH_EXT_COUNT +} + +desc_test "control case, valid file" +desc_test "multiple pages, valid file" "gap_before_footer $(num_to_hex 0 10000)" + +desc_test "bad magic: XXXXXXXX" "magic XXXXXXXX" +desc_test "bad magic: FSVeritY" "magic FSVeritY" +desc_test "bad major_version" "major_version \xff" +desc_test "bad minor_version" "minor_version \xff" +desc_test "bad log_data_blocksize: 0x00" "log_data_blocksize \x00" +desc_test "bad log_data_blocksize: 0xff" "log_data_blocksize \xff" +desc_test "bad log_tree_blocksize: 0x00" "log_tree_blocksize \x00" +desc_test "bad log_tree_blocksize: 0xff" "log_tree_blocksize \xff" +desc_test "bad data_algorithm: 0x0000" "data_algorithm \x00\x00" +desc_test "bad data_algorithm: 0xffff" "data_algorithm \xff\xff" +desc_test "bad tree_algorithm: 0x0000" "tree_algorithm \x00\x00" +desc_test "bad tree_algorithm: 0xffff" "tree_algorithm \xff\xff" +desc_test "mismatched block sizes" "log_data_blocksize \x10" "log_tree_blocksize \x0C" +desc_test "mismatched algorithms" "data_algorithm \x01\x00" "tree_algorithm \x02\x00" +desc_test "bad flags" "flags \xff\xff\xff\xff" +desc_test "bad reserved1" "reserved1 \xff\xff\xff\xff" +desc_test "bad orig_file_size: 0" "orig_file_size $(num_to_hex 0 8)" +desc_test "bad orig_file_size: > full_isize" "orig_file_size $(num_to_hex 100000 8)" +desc_test "bad orig_file_size: UINT64_MAX" "orig_file_size \xff\xff\xff\xff\xff\xff\xff\xff" +desc_test "bad auth_ext_count" "$(auth_ext_count 65535)" +desc_test "bad reserved2" "reserved2 $(perl -e 'print "\\xff" x 30')" + +desc_test "bad desc_reverse_offset: 0" "desc_reverse_offset $(num_to_hex 0 4)" +desc_test "bad desc_reverse_offset: 64" "desc_reverse_offset $(num_to_hex 64 4)" +desc_test "bad desc_reverse_offset: 69" "desc_reverse_offset $(num_to_hex 69 4)" +desc_test "bad desc_reverse_offset: > full_isize" "desc_reverse_offset $(num_to_hex 100000 4)" +desc_test "bad desc_reverse_offset: UINT32_MAX" "desc_reverse_offset \xff\xff\xff\xff" +desc_test "bad ftr_magic: XXXXXXXX" "ftr_magic XXXXXXXX" +desc_test "bad ftr_magic: FSVeritY" "ftr_magic FSVeritY" + +desc_test "root hash length wrong: 0" \ + "root_hash $(create_root_hash_ext "$(num_to_hex 0 0)")" +desc_test "root hash length wrong: too short" \ + "root_hash $(create_root_hash_ext "$(num_to_hex 0 16)")" +desc_test "root hash length wrong: too long" \ + "root_hash $(create_root_hash_ext "$(num_to_hex 0 100)")" +desc_test "multiple root hashes" "$(auth_ext_count +1)" \ + "auth_extensions $(create_root_hash_ext "$(num_to_hex 0 32)")" +desc_test "no root hash" "$(auth_ext_count -1)" "root_hash" + +desc_test "root hash is unauthenticated" \ + "$(auth_ext_count -1)" \ + "$(unauth_ext_count +1)" \ + "root_hash" \ + "unauth_extensions $(create_root_hash_ext $(num_to_hex 0 32))" + +desc_test "salt is unauthenticated" \ + "$(unauth_ext_count +1)" \ + "unauth_extensions $(create_ext $FS_VERITY_EXT_SALT foo)" + +desc_test "unknown extension type" \ + "$(auth_ext_count +1)" \ + "auth_extensions $(create_ext 255 "")" + +desc_test "length in extension header smaller than header" \ + "$(auth_ext_count +1)" \ + "auth_extensions $(create_exthdr 0 $FS_VERITY_EXT_SALT)" + +desc_test "extension length overflows buffer" \ + "$(auth_ext_count +1)" \ + "auth_extensions $(create_exthdr 50000 $FS_VERITY_EXT_SALT)" + +desc_test "extension length wraps to 0 after rounding" \ + "$(auth_ext_count +1)" \ + "auth_extensions $(create_exthdr 0xffffffff $FS_VERITY_EXT_SALT)" + +desc_test "reserved bits set in extension header" \ + "$(auth_ext_count +1)" \ + "auth_extensions $(create_exthdr $EXTHDR_SIZE $FS_VERITY_EXT_SALT 1000)" + +# success, all done +status=0 +exit
tests/generic/902.out new file +125 -0
125 lines not on this page · raw message
tests/generic/group +1 -0
@@ -527,3 +527,4 @@ 522 soak long_rw 900 auto quick verity 901 auto quick verity +902 auto quick verity
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 5/7 generic: test corrupting verity files
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
This test zaps various parts of the contents of a verity file, or parts of its Merkle tree, by writing directly to the block device. It verifies that this causes I/O errors when the relevant part of the contents is later read by any means.
--- tests/generic/903 | 126 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/903.out | 91 ++++++++++++++++++++++++++++++ tests/generic/group | 1 + 3 files changed, 218 insertions(+) create mode 100755 tests/generic/903 create mode 100644 tests/generic/903.out
tests/generic/903 new file +126 -0
@@ -0,0 +1,126 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/903 +# +# Test corrupting verity files. This test zaps various parts of the contents of +# a verity file, or parts of its Merkle tree, by writing directly to the block +# device. It verifies that this causes I/O errors when the relevant part of the +# contents is later read by any means. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_orig_file=$SCRATCH_MNT/file +fsv_file=$SCRATCH_MNT/file.fsv + +setup_zeroed_file() +{ + local len=$1 + + head -c $len /dev/zero > $fsv_orig_file + _fsv_setup $fsv_orig_file $fsv_file >> $seqres.full + _fsv_enable $fsv_file + md5sum $fsv_file |& _filter_scratch 1>&2 +} + +filter_sigbus() +{ + sed -e 's/.*Bus error.*/Bus error/' +} + +page_boundary() +{ + local n=$1 + local page_size=$(getconf PAGE_SIZE) + + echo $(( (n + page_size - 1) & ~(page_size - 1) )) +} + +corruption_test() +{ + local file_len=$1 + local zap_offset=$2 + local zap_len=$3 + local metadata_offset=$(page_boundary $file_len) + local measurement + + _fsv_begin_subtest "Corruption test: file_len=$file_len zap_offset=$zap_offset zap_len=$zap_len" + setup_zeroed_file $file_len + cmp $fsv_file $fsv_orig_file + echo "Corrupting bytes..." + head -c $zap_len /dev/zero | tr '\0' X \ + | _fsv_corrupt_bytes $fsv_file $zap_offset + + echo "Validating corruption (reading full file)..." + _scratch_cycle_mount + md5sum $fsv_file |& _filter_scratch + + echo "Validating corruption (direct I/O)..." + _scratch_cycle_mount + dd if=$fsv_file bs=$FSV_BLOCK_SIZE iflag=direct status=none \ + of=/dev/null |& _filter_scratch + + if (( zap_offset < metadata_offset )); then + echo "Validating corruption (reading just corrupted part)..." + dd if=$fsv_file bs=1 skip=$zap_offset count=$zap_len \ + of=/dev/null status=none |& _filter_scratch + fi + + echo "Validating corruption (reading full file via mmap)..." + bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ + -c 'mmap -r 0 $metadata_offset' \ + -c 'mread 0 $file_len'" |& filter_sigbus + + if (( zap_offset < metadata_offset )); then + echo "Validating corruption (reading just corrupted part via mmap)..." + bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ + -c 'mmap -r 0 $metadata_offset' \ + -c 'mread $zap_offset $zap_len'" |& filter_sigbus + fi +} + +corruption_test 131072 0 1 +corruption_test 131072 4095 1 +corruption_test 131072 65536 65536 +corruption_test 131072 131071 1 + +# Non-zeroed bytes in the final partial block beyond EOF should cause reads to +# fail too. Such bytes would be visible via mmap(). +corruption_test 129999 131000 72 + +# Hash tree corruption +corruption_test 1048576 1052672 4096 +corruption_test 1048576 1056767 1 + +# success, all done +status=0 +exit
tests/generic/903.out new file +91 -0
91 lines not on this page · raw message
tests/generic/group +1 -0
@@ -528,3 +528,4 @@ 900 auto quick verity 901 auto quick verity 902 auto quick verity +903 auto quick verity
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 6/7 generic: test that fs-verity is using the correct measurement values
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
This test verifies that fs-verity is doing its Merkle tree-based hashing correctly, i.e. that it hasn't been broken by a change.
--- tests/generic/904 | 124 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/904.out | 7 +++ tests/generic/group | 1 + 3 files changed, 132 insertions(+) create mode 100755 tests/generic/904 create mode 100644 tests/generic/904.out
tests/generic/904 new file +124 -0
@@ -0,0 +1,124 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/904 +# +# Test that fs-verity is using the correct measurement values. This test +# verifies that fs-verity is doing its Merkle tree-based hashing correctly, +# i.e. that it hasn't been broken by a change. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_orig_file=$SCRATCH_MNT/file +fsv_file=$SCRATCH_MNT/file.fsv + +have_hash_algorithm() +{ + local alg=$1 + + head -c 4096 /dev/zero > $fsv_file + if ! _fsv_setup --hash=$alg $fsv_file &>> $seqres.full; then + # no userspace tool support + return 1 + fi + if ! _fsv_enable $fsv_file &>> $seqres.full; then + # no kernel support + return 1 + fi + rm -f $fsv_file + return 0 +} + +algs=(sha256 sha512 crc32c) +file_sizes=(4096 65536 100000000) + +# The expected values are here rather than in the expected output file because +# not all hash algorithms may be available. +sha256_vals=( +sha256:47b92f80eedc47a224cc4f922de978c8933bc205844a7fd06da848d2c37471ab +sha256:8f6f7df7c0babebacc1b4a158ba9d40ecfe3ad5e66647d1aaf715244ae882a80 +sha256:66fcfdf105061a621b13da383ff57d0ac63c61311215b46c1f4fc4959db69ef6 +) +sha512_vals=( +sha512:2709f8fc180abb2444b6a57143442737ab3ece8d6e463e1bf328a78e5b0a902f0031cc7ac58797e40c6cfbb29e3005c4730800932308bc549df5375fb1859d37 +sha512:bfdcd4c1a9493c830e2e175da7fda02e60e2deeb21f2787cea3a70e545fd34b9d0266738cbe2435f81bc5a44f58d8ae404f9e5f835c8b989a7a0b234e4cadffa +sha512:f08e75b685b65f43d9ffa71b85e400cbdc67215cfa755c126237d8dc96ba9570225387586ee47f97d552aeba9e73a318fcb65c6b27ffc58106d803f0acea29e3 +) +crc32c_vals=( +crc32c:91806377 +crc32c:cf55a43e +crc32c:d672241e +) + +test_alg() +{ + local alg=$1 + local -n vals=${alg}_vals + local i + local file_size + local expected actual + + _fsv_begin_subtest "Check for expected measurement values ($alg)" + + if ! have_hash_algorithm $alg; then + if [ "$alg" = sha256 ]; then + _fail "Something is wrong - sha256 hash should always be available" + fi + return 0 + fi + + for i in ${!file_sizes[@]}; do + file_size=${file_sizes[$i]} + expected=${vals[$i]} + + head -c $file_size /dev/zero > $fsv_orig_file + actual=$(_fsv_setup --hash=$alg $fsv_orig_file $fsv_file) + if [ "$actual" != "$expected" ]; then + echo "Mismatch: expected $expected, 'fsverity setup' calculated $actual (file_size=$file_size)" + fi + _fsv_enable $fsv_file + actual=$(_fsv_measure $fsv_file) + if [ "$actual" != "$expected" ]; then + echo "Mismatch: expected $expected, kernel calculated $actual (file_size=$file_size)" + fi + cmp $fsv_orig_file $fsv_file + rm -f $fsv_file + done +} + +for alg in ${algs[@]}; do + test_alg $alg +done + +# success, all done +status=0 +exit
tests/generic/904.out new file +7 -0
@@ -0,0 +1,7 @@ +QA output created by 904 + +# Check for expected measurement values (sha256) + +# Check for expected measurement values (sha512) + +# Check for expected measurement values (crc32c)
tests/generic/group +1 -0
@@ -529,3 +529,4 @@ 901 auto quick verity 902 auto quick verity 903 auto quick verity +904 auto quick verity
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Eric Biggers
maintainerPATCH 7/7 generic: test using fs-verity and fscrypt simultaneously
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
This primarily verifies correct ordering of the hooks for each feature:
fscrypt needs to be first.
--- tests/generic/905 | 81 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/905.out | 12 +++++++ tests/generic/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/generic/905 create mode 100644 tests/generic/905.out
tests/generic/905 new file +81 -0
@@ -0,0 +1,81 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/905 +# +# Test using fs-verity and fscrypt simultaneously. This primarily verifies +# correct ordering of the hooks for each feature: fscrypt needs to be first. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity +. ./common/encrypt + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity +_require_scratch_encryption +_require_xfs_io_command "set_encpolicy" +_require_command "$KEYCTL_PROG" keyctl + +_scratch_mkfs_encrypted_verity &>> $seqres.full +_scratch_mount + +fsv_orig_file=$tmp.file +edir=$SCRATCH_MNT/edir +fsv_file=$edir/file.fsv + +# Set up an encrypted directory. +_new_session_keyring +keydesc=$(_generate_encryption_key) +mkdir $edir +$XFS_IO_PROG -c "set_encpolicy $keydesc" $edir + +# Set up an fs-verity file within the encrypted directory. +# Verity that it actually has an encryption policy. +head -c 100000 /dev/zero > $fsv_orig_file +_fsv_setup $fsv_orig_file $fsv_file >> $seqres.full +_fsv_enable $fsv_file +echo +$XFS_IO_PROG -r -c "get_encpolicy" $fsv_file |& _filter_scratch \ + | sed 's/Master key descriptor:.*/Master key descriptor: 0000000000000000/' +echo + +# Verify that the file contents are as expected. This should be going through +# both the decryption and verity I/O paths. +cmp $fsv_orig_file $fsv_file && echo "Files matched" + +# Just in case, try again after a mount cycle to empty the page cache. +_scratch_cycle_mount +cmp $fsv_orig_file $fsv_file && echo "Files matched" + +# Corrupt some bytes as a sanity check that fs-verity is really working. +# This also verifies that the data on-disk is really encrypted, since otherwise +# the data being written here would be identical to the old data. +head -c 1000 /dev/zero | _fsv_corrupt_bytes $fsv_file 50000 +md5sum $fsv_file |& _filter_scratch + +# success, all done +status=0 +exit
tests/generic/905.out new file +12 -0
@@ -0,0 +1,12 @@ +QA output created by 905 + +Encryption policy for SCRATCH_MNT/edir/file.fsv: + Policy version: 0 + Master key descriptor: 0000000000000000 + Contents encryption mode: 1 (AES-256-XTS) + Filenames encryption mode: 4 (AES-256-CTS) + Flags: 0x02 + +Files matched +Files matched +md5sum: SCRATCH_MNT/edir/file.fsv: Input/output error
tests/generic/group +1 -0
@@ -530,3 +530,4 @@ 902 auto quick verity 903 auto quick verity 904 auto quick verity +905 auto quick verity encrypt
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Quoted 3 lines · Eric Biggers
Add tests for fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running these tests requires:
Seriously, given how broken the current fs-verify support is we should neither merge it into the kernel nor fstests.
Nothing against the highlevel feature per-se, but the implementation posted to linux-fsdevel a while ago is simply too broken.
Eric Biggers
maintainerHi Christoph,
Quoted 10 lines · Christoph Hellwig
Quoted 3 lines · Eric Biggers
Add tests for fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running these tests requires:
Seriously, given how broken the current fs-verify support is we should neither merge it into the kernel nor fstests.
Nothing against the highlevel feature per-se, but the implementation posted to linux-fsdevel a while ago is simply too broken.
Please explain. Broken, how so? What changes do you suggest, specifically?
Also, if you have feedback it would be helpful if you posted it in response to the actual kernel patchset, not just the tests. AFAICS, this is your first time responding to any fs-verity related thread, despite it being out for review for months; is that correct or did I miss something you previously posted?
- Eric
Theodore Y. Ts'o
maintainerQuoted 10 lines · Christoph Hellwig
Quoted 3 lines · Eric Biggers
Add tests for fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running these tests requires:
Seriously, given how broken the current fs-verify support is we should neither merge it into the kernel nor fstests.
Nothing against the highlevel feature per-se, but the implementation posted to linux-fsdevel a while ago is simply too broken.
What do you think is broken?
- Ted
Quoted 8 lines · Eric Biggers
Hi Christoph,
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Eric Biggers
Add tests for fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running these tests requires:
Quoted 7 lines
Quoted 5 lines
Seriously, given how broken the current fs-verify support is we should neither merge it into the kernel nor fstests.
Nothing against the highlevel feature per-se, but the implementation posted to linux-fsdevel a while ago is simply too broken.
Please explain. Broken, how so? What changes do you suggest, specifically?
Don't magically pass the hash in the file data, that is just broken beyond repairable.
Quoted 4 lines
Also, if you have feedback it would be helpful if you posted it in response to the actual kernel patchset, not just the tests. AFAICS, this is your first time responding to any fs-verity related thread, despite it being out for review for months; is that correct or did I miss something you previously posted?
No. But given how broken it I'd rather ignore it rather than wasting my time. It isn't like it did get any other serious review either.
Quoted 19 lines · Eric Biggers
Add tests for fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running these tests requires:
- A kernel with the fs-verity patches from
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/log/
(should be merged in 4.21) and configured withCONFIG_FS_VERITY.
- The fsverity utility program, which can be installed from
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git/
- e2fsprogs v1.44.4-2 or later for ext4 tests, or f2fs-tools v1.11.0 or later for f2fs tests.Example with kvm-xfstests:
$ kvm-xfstests -c ext4,f2fs -g veritySee the file
Documentation/filesystem/fsverity.rstin the kernel tree for more information about fs-verity.
Quoted 8 lines
Eric Biggers (7):
common/verity: add common functions for testing fs-verity
generic: test general behavior of verity files
generic: test access controls on the fs-verity ioctls
generic: test fs-verity descriptor validation
generic: test corrupting verity files
generic: test that fs-verity is using the correct measurement values
generic: test using fs-verity and fscrypt simultaneously
The tests look good to me overall, and tests ran fine on my test vm (I only tested with ext4 though), thanks a lot! There're just a few minor issues, the comments go to individual patch.
But it seems that there're still disagreements on the implemention of fsverity, I'd like to wait and see what's the final decision before taking the tests.
Thanks,
Eryu
Re: [PATCH 1/7] common/verity: add common functions for testing fs-verity
In reply to Eric BiggersQuoted 68 lines · Eric Biggers
From: Eric Biggers <[email protected]>
Add common functions for setting up and testing fs-verity, a new feature for read-only file-based authenticity protection. fs-verity will be supported by ext4 and f2fs, and perhaps other filesystems later. Running the fs-verity tests requires:
- A kernel with the fs-verity patches from
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/log/
(should be merged in 4.21) and configured withCONFIG_FS_VERITY.
- The fsverity utility program, which can be installed from
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git/
- e2fsprogs v1.44.4-2 or later for ext4 tests, or f2fs-tools v1.11.0 or later for f2fs tests.See the file
Documentation/filesystem/fsverity.rstin the kernel tree for more information about fs-verity.Signed-off-by Eric Biggers <[email protected]>--- common/config | 1 + common/verity | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 199 insertions(+) create mode 100644 common/veritycommon/config +1
-0@@ -194,6 +194,7 @@ export GETCAP_PROG="$(type -P getcap)" export CHECKBASHISMS_PROG="$(type -P checkbashisms)" export XFS_INFO_PROG="$(type -P xfs_info)" export DUPEREMOVE_PROG="$(type -P duperemove)" +export FSVERITY_PROG="$(type -P fsverity)" # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled. # newer systems have udevadm command but older systems like RHEL5 don't.common/verity new file +25
-0@@ -0,0 +1,198 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# Functions for setting up and testing fs-verity + +FSV_BLOCK_SIZE=4096 + +_require_scratch_verity() +{ + _require_scratch + _require_command "$FSVERITY_PROG" fsverity + + if ! _scratch_mkfs_verity &>>$seqres.full; then + # ext4: need e2fsprogs v1.44.4-2 or later + # f2fs: need f2fs-tools v1.11.0 or later + _notrun "$FSTYP userspace tools don't support fs-verity" + fi + + # Try to mount the filesystem. If this fails, then the filesystem is + # unaware of the fs-verity feature. + if ! _try_scratch_mount &>>$seqres.full; then + _notrun "kernel doesn't know about $FSTYP verity feature" + fi + _scratch_unmount +
common/verity line 35
Quoted 10 lines
common/verity new file +10
-0+ # The filesystem may be aware of fs-verity but have it disabled by + # CONFIG_FS_VERITY=n. Detect support via sysfs. + if [ ! -e /sys/fs/$FSTYP/features/verity ]; then + _notrun "kernel $FSTYP isn't configured with verity support" + fi + + # fs-verity with block_size != PAGE_SIZE isn't implemented yet. + # ("block_size" here refers to the fs-verity block size, not to the + # filesystem's block size.) + if [ "$(getconf PAGE_SIZE)" != $FSV_BLOCK_SIZE ]; then
We could use helper "get_page_size" here.
Quoted 29 lines
common/verity new file +29
-0+ _notrun "verity not yet supported for PAGE_SIZE != $FSV_BLOCK_SIZE" + fi +} + +_scratch_mkfs_verity() +{ + case $FSTYP in + ext4|f2fs) + _scratch_mkfs -O verity + ;; + *) + _notrun "No verity support for $FSTYP" + ;; + esac +} + +_scratch_mkfs_encrypted_verity() +{ + case $FSTYP in + ext4) + _scratch_mkfs -O encrypt,verity + ;; + f2fs) + # f2fs-tools as of v1.11.0 doesn't allow comma-separated + # features with -O. Instead -O must be supplied multiple times. + _scratch_mkfs -O encrypt -O verity + ;; + *) + _notrun "$FSTYP not supported in _scratch_mkfs_encrypted_verity"
common/verity line 74
Quoted 10 lines
common/verity new file +10
-0+ ;; + esac +} + +_fsv_randstring() +{ + local nchars=$1 + + tr -d -C 0-9a-f < /dev/urandom | head -c "$nchars" +}
This function has no caller? And if it has a caller, I think it's generic enough to move it to common/rc and rename it to a more generic name. But why limits the string to 0-9a-f, if it's expected to generate random string?
common/verity line 80
Quoted 6 lines
common/verity new file +6
-0+ +_fsv_begin_subtest() +{ + local msg=$1 + + rm -rf "${SCRATCH_MNT:?}"/*
It assumes the test is run against $SCRATCH_DEV/$SCRATCH_MNT, it's better to either rename the function to _fsv_scratch_begin_subtest to indicate it takes use of $SCRATCH_DEV/MNT or just pass the working dir as a argument.
Quoted 38 lines
common/verity new file +38
-0+ echo -e "\n# $msg" +} + +_fsv_setup() +{ + $FSVERITY_PROG setup "$@" | awk '/^File measurement: /{print $3}' +} + +_fsv_enable() +{ + $FSVERITY_PROG enable "$@" +} + +_fsv_measure() +{ + $FSVERITY_PROG measure "$@" | awk '{print $1}' +} + +# Generate a file with verity metadata, but don't actually enable verity yet +_fsv_create_setup_file() +{ + local file=$1 + + head -c $((FSV_BLOCK_SIZE * 2)) /dev/zero > "$file" + _fsv_setup "$file" +} + +# Generate a file with verity metadata, then enable verity +_fsv_create_enable_file() +{ + local file=$1 + + _fsv_create_setup_file "$file" + _fsv_enable "$file" +} + +# +# _fsv_corrupt_bytes - Write some bytes to a file, bypassing the filesystem
common/verity line 128
Quoted 10 lines
common/verity new file +10
-0+# +# Write the bytes sent on stdin to the given offset in the given file, but do so +# by writing directly to the extents on the block device, with the filesystem +# unmounted. This can be used to corrupt a verity file for testing purposes, +# bypassing the restrictions imposed by the filesystem. On ext4 and f2fs this +# can also write into the metadata region of a verity file. +# +# The file is assumed to be located on $SCRATCH_DEV. +# +_fsv_corrupt_bytes()
Same here. Either use rename it to _fsv_scratch_corrupt_bytes or pass the block device to it.
Quoted 26 lines
common/verity new file +26
-0+{ + local file=$1 + local offset=$2 + local lstarts=() # extent logical starts, in bytes + local pstarts=() # extent physical starts, in bytes + local lens=() # extent lengths, in bytes + local line + local cmd + local dd_cmds=() + local eidx=0 + + sync # Sync to avoid unwritten extents + + cat > $tmp.bytes + local end=$(( offset + $(stat -c %s $tmp.bytes ) )) + + # Get the list of extents that intersect the requested range + while read -r line; do \ + local fields=($line) + local lstart=${fields[0]} + local lend=${fields[1]} + local pstart=${fields[2]} + local pend=${fields[3]} + local llen=$((lend + 1 - lstart)) + local plen=$((pend + 1 - pstart)) + if (( llen != plen )); then
common/verity line 164
Quoted 10 lines
common/verity new file +10
-0+ _fail "Logical and physical extent lengths differ! $line" + fi + lstarts+=( $((lstart * 512)) ) + pstarts+=( $((pstart * 512)) ) + lens+=( $((llen * 512)) ) + done < <($XFS_IO_PROG -r -c "fiemap $offset $((end - offset))" "$file" \ + | grep -E '^[[:space:]]+[0-9]+:' \ + | grep -v '\<hole\>' \ + | sed -E 's/^[[:space:]]+[0-9]+://' \ + | tr '][.:' ' ')
Introduce a new _filter_xfs_io_fiemap helper? We already have _filter_filefrag which does similar jobs.
Thanks,
Eryu
Quoted 37 lines
common/verity new file +34
-0+ + while (( offset < end )); do + # Find the next extent to write to + while true; do + if (( eidx >= ${#lstarts[@]} )); then + _fail "Extents ended before byte $offset" + fi + if (( offset < ${lstarts[$eidx]} )); then + _fail "Hole in file at byte $offset" + fi + local lend=$(( ${lstarts[$eidx]} + ${lens[$eidx]} )) + if (( offset < lend )); then + break + fi + (( eidx += 1 )) + done + # Add a command that writes to the next extent + local len=$((lend - offset)) + local seek=$(( offset + ${pstarts[$eidx]} - ${lstarts[$eidx]} )) + if (( len > end - offset )); then + len=$((end - offset)) + fi + dd_cmds+=("head -c $len | dd of=$SCRATCH_DEV oflag=seek_bytes seek=$seek status=none") + (( offset += len )) + done + + # Execute the commands to write the data + _scratch_unmount + for cmd in "${dd_cmds[@]}"; do + eval "$cmd" + done < $tmp.bytes + sync # Sync to flush the block device's pagecache + _scratch_mount +}Signature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Re: [PATCH 3/7] generic: test access controls on the fs-verity ioctls
In reply to Eric BiggersQuoted 56 lines · Eric Biggers
From: Eric Biggers <[email protected]>
Test access controls on the fs-verity ioctls.
FS_IOC_MEASURE_VERITYis allowed on any file, whereasFS_IOC_ENABLE_VERITYrequires write access.Signed-off-by Eric Biggers <[email protected]>--- tests/generic/901 | 71 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/901.out | 14 +++++++++ tests/generic/group | 1 + 3 files changed, 86 insertions(+) create mode 100755 tests/generic/901 create mode 100644 tests/generic/901.outtests/generic/901 new file +36
-0@@ -0,0 +1,71 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/901 +# +# Test access controls on the fs-verity ioctls. FS_IOC_MEASURE_VERITY is +# allowed on any file, whereas FS_IOC_ENABLE_VERITY requires write access. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity
tests/generic/901 line 46
Quoted 10 lines
tests/generic/901 new file +10
-0+_require_user + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_file=$SCRATCH_MNT/file.fsv + +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY doesn't require root" +_fsv_create_setup_file $fsv_file >> $seqres.full +chmod 666 $fsv_file +su $qa_user -c "$FSVERITY_PROG enable $fsv_file"
Use _user_do instead of a bare su?
tests/generic/901 line 57
Quoted 11 lines
tests/generic/901 new file +11
-0+ +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY requires write access" +_fsv_create_setup_file $fsv_file >> $seqres.full +chmod 444 $fsv_file +su $qa_user -c "$FSVERITY_PROG enable $fsv_file" |& _filter_scratch + +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY requires !append-only" +_fsv_create_setup_file $fsv_file >> $seqres.full +chattr +a $fsv_file +$FSVERITY_PROG enable $fsv_file |& _filter_scratch +chattr -a $fsv_file
Need "_require_chattr ia" then, and use $CHATTR_PROG.
Thanks,
Eryu
Quoted 46 lines
tests/generic/901 new file +14
-0+ +_fsv_begin_subtest "FS_IOC_ENABLE_VERITY requires !immutable" +_fsv_create_setup_file $fsv_file >> $seqres.full +chattr +i $fsv_file +$FSVERITY_PROG enable $fsv_file |& _filter_scratch +chattr -i $fsv_file + +_fsv_begin_subtest "FS_IOC_MEASURE_VERITY doesn't require root" +_fsv_create_enable_file $fsv_file >> $seqres.full +su $qa_user -c "$FSVERITY_PROG measure $fsv_file" >> $seqres.full + +# success, all done +status=0 +exittests/generic/901.out new file +14
-0@@ -0,0 +1,14 @@ +QA output created by 901 + +# FS_IOC_ENABLE_VERITY doesn't require root + +# FS_IOC_ENABLE_VERITY requires write access +ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Permission denied + +# FS_IOC_ENABLE_VERITY requires !append-only +ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Operation not permitted + +# FS_IOC_ENABLE_VERITY requires !immutable +ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': Operation not permitted + +# FS_IOC_MEASURE_VERITY doesn't require roottests/generic/group +1
-0@@ -526,3 +526,4 @@ 521 soak long_rw 522 soak long_rw 900 auto quick verity +901 auto quick veritySignature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog
Re: [PATCH 5/7] generic: test corrupting verity files
In reply to Eric BiggersQuoted 75 lines · Eric Biggers
From: Eric Biggers <[email protected]>
This test zaps various parts of the contents of a verity file, or parts of its Merkle tree, by writing directly to the block device. It verifies that this causes I/O errors when the relevant part of the contents is later read by any means.
Signed-off-by Eric Biggers <[email protected]>--- tests/generic/903 | 126 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/903.out | 91 ++++++++++++++++++++++++++++++ tests/generic/group | 1 + 3 files changed, 218 insertions(+) create mode 100755 tests/generic/903 create mode 100644 tests/generic/903.outtests/generic/903 new file +53
-0@@ -0,0 +1,126 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2018 Google LLC +# +# FS QA Test generic/903 +# +# Test corrupting verity files. This test zaps various parts of the contents of +# a verity file, or parts of its Merkle tree, by writing directly to the block +# device. It verifies that this causes I/O errors when the relevant part of the +# contents is later read by any means. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/verity + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch_verity + +_scratch_mkfs_verity &>> $seqres.full +_scratch_mount +fsv_orig_file=$SCRATCH_MNT/file +fsv_file=$SCRATCH_MNT/file.fsv + +setup_zeroed_file() +{ + local len=$1 + + head -c $len /dev/zero > $fsv_orig_file + _fsv_setup $fsv_orig_file $fsv_file >> $seqres.full + _fsv_enable $fsv_file + md5sum $fsv_file |& _filter_scratch 1>&2 +}
tests/generic/903 line 63
Quoted 10 lines
tests/generic/903 new file +10
-0+ +filter_sigbus() +{ + sed -e 's/.*Bus error.*/Bus error/' +} + +page_boundary() +{ + local n=$1 + local page_size=$(getconf PAGE_SIZE)
get_page_size. (I think we really should rename it to _get_page_size and fix all the callers..)
Thanks,
Eryu
Quoted 172 lines
tests/generic/903 new file +63
-0+ + echo $(( (n + page_size - 1) & ~(page_size - 1) )) +} + +corruption_test() +{ + local file_len=$1 + local zap_offset=$2 + local zap_len=$3 + local metadata_offset=$(page_boundary $file_len) + local measurement + + _fsv_begin_subtest "Corruption test: file_len=$file_len zap_offset=$zap_offset zap_len=$zap_len" + setup_zeroed_file $file_len + cmp $fsv_file $fsv_orig_file + echo "Corrupting bytes..." + head -c $zap_len /dev/zero | tr '\0' X \ + | _fsv_corrupt_bytes $fsv_file $zap_offset + + echo "Validating corruption (reading full file)..." + _scratch_cycle_mount + md5sum $fsv_file |& _filter_scratch + + echo "Validating corruption (direct I/O)..." + _scratch_cycle_mount + dd if=$fsv_file bs=$FSV_BLOCK_SIZE iflag=direct status=none \ + of=/dev/null |& _filter_scratch + + if (( zap_offset < metadata_offset )); then + echo "Validating corruption (reading just corrupted part)..." + dd if=$fsv_file bs=1 skip=$zap_offset count=$zap_len \ + of=/dev/null status=none |& _filter_scratch + fi + + echo "Validating corruption (reading full file via mmap)..." + bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ + -c 'mmap -r 0 $metadata_offset' \ + -c 'mread 0 $file_len'" |& filter_sigbus + + if (( zap_offset < metadata_offset )); then + echo "Validating corruption (reading just corrupted part via mmap)..." + bash -c "trap '' SIGBUS; $XFS_IO_PROG -r $fsv_file \ + -c 'mmap -r 0 $metadata_offset' \ + -c 'mread $zap_offset $zap_len'" |& filter_sigbus + fi +} + +corruption_test 131072 0 1 +corruption_test 131072 4095 1 +corruption_test 131072 65536 65536 +corruption_test 131072 131071 1 + +# Non-zeroed bytes in the final partial block beyond EOF should cause reads to +# fail too. Such bytes would be visible via mmap(). +corruption_test 129999 131000 72 + +# Hash tree corruption +corruption_test 1048576 1052672 4096 +corruption_test 1048576 1056767 1 + +# success, all done +status=0 +exittests/generic/903.out new file +91
-0@@ -0,0 +1,91 @@ +QA output created by 903 + +# Corruption test: file_len=131072 zap_offset=0 zap_len=1 +0dfbe8aa4c20b52e1b8bf3cb6cbdf193 SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading just corrupted part)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus error +Validating corruption (reading just corrupted part via mmap)... +Bus error + +# Corruption test: file_len=131072 zap_offset=4095 zap_len=1 +0dfbe8aa4c20b52e1b8bf3cb6cbdf193 SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading just corrupted part)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus error +Validating corruption (reading just corrupted part via mmap)... +Bus error + +# Corruption test: file_len=131072 zap_offset=65536 zap_len=65536 +0dfbe8aa4c20b52e1b8bf3cb6cbdf193 SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading just corrupted part)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus error +Validating corruption (reading just corrupted part via mmap)... +Bus error + +# Corruption test: file_len=131072 zap_offset=131071 zap_len=1 +0dfbe8aa4c20b52e1b8bf3cb6cbdf193 SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading just corrupted part)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus error +Validating corruption (reading just corrupted part via mmap)... +Bus error + +# Corruption test: file_len=129999 zap_offset=131000 zap_len=72 +0ed66e88b29ce0c585cedf35ee127213 SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading just corrupted part)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus error +Validating corruption (reading just corrupted part via mmap)... +Bus error + +# Corruption test: file_len=1048576 zap_offset=1052672 zap_len=4096 +b6d81b360a5672d80c27430f39153e2c SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus error + +# Corruption test: file_len=1048576 zap_offset=1056767 zap_len=1 +b6d81b360a5672d80c27430f39153e2c SCRATCH_MNT/file.fsv +Corrupting bytes... +Validating corruption (reading full file)... +md5sum: SCRATCH_MNT/file.fsv: Input/output error +Validating corruption (direct I/O)... +dd: error reading 'SCRATCH_MNT/file.fsv': Input/output error +Validating corruption (reading full file via mmap)... +Bus errortests/generic/group +1
-0@@ -528,3 +528,4 @@ 900 auto quick verity 901 auto quick verity 902 auto quick verity +903 auto quick veritySignature · 1 line
2.20.0.rc2.403.gdbc3b29805-goog