fstests: add btrfs encryption testing
Subsystems (3)
cover fstests: add btrfs encryption testing
This is a preliminary fstests side of the btrfs encryption feature; more tests are needed, but this gets the existing encryption tests working with btrfs. This requires the progs and the kernel changes: [1] [2].
Marked as RFC because they're not ready to merge until all its dependencies finish landing.
[1] https://lore.kernel.org/linux-btrfs/[email protected]/T/#t
[2] https://lore.kernel.org/linux-btrfs/[email protected]/T/#t
Sweet Tea Dorminy (8):
common/encrypt: separate data and inode nonces common/encrypt: add btrfs to get_encryption_*nonce common/encrypt: add btrfs to get_ciphertext_filename common/encrypt: enable making a encrypted btrfs filesystem generic/613: write some actual data for btrfs
tests: adjust encryption tests for extent encryption common/verity: explicitly don't allow btrfs encryption
btrfs: add simple test of reflink of encrypted data
common/encrypt | 86 +++++++++++++++++++++++++++++++++++++++++---- common/verity | 4 +++ tests/btrfs/613 | 62 ++++++++++++++++++++++++++++++++ tests/btrfs/613.out | 13 +++++++ tests/generic/429 | 6 ++++ tests/generic/580 | 4 +++ tests/generic/595 | 4 +++ tests/generic/613 | 12 ++++--- 8 files changed, 180 insertions(+), 11 deletions(-) create mode 100755 tests/btrfs/613 create mode 100644 tests/btrfs/613.out
base-commit: 87f90a2dae7a4adb7a0a314e27abae9aa1de78fb
Signature · 1 line
2.40.1
RFC PATCH 4/8 common/encrypt: enable making a encrypted btrfs filesystem
--- common/encrypt | 6 ++++++ 1 file changed, 6 insertions(+)
common/encrypt +6 -0
@@ -153,6 +153,9 @@ _scratch_mkfs_encrypted() # erase the UBI volume; reformated automatically on next mount $UBIUPDATEVOL_PROG ${SCRATCH_DEV} -t ;; + btrfs) + _scratch_mkfs + ;; ceph) _scratch_cleanup_files ;;
@@ -168,6 +171,9 @@ _scratch_mkfs_sized_encrypted() ext4|f2fs) MKFS_OPTIONS="$MKFS_OPTIONS -O encrypt" _scratch_mkfs_sized $* ;; + btrfs) + _scratch_mkfs_sized $* + ;; *) _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized_encrypted" ;;
Signature · 1 line
2.40.1
RFC PATCH 5/8 generic/613: write some actual data for btrfs
In reply to Sweet Tea DorminyCurrently, the test touches a file and assumes that that is sufficient to generate a new nonce to test for that file. However, btrfs doesn't store an encryption context for a leaf inode, and doesn't store an encryption context for data within a leaf inode until data is actually written. Thus, merely touching the file on btrfs doesn't actually generate a testable nonce.
Instead, write a trivial bit of data to each file, which provokes btrfs to generate a encryption context for the data and thus a testable nonce.
--- tests/generic/613 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/generic/613 +2 -2
@@ -53,11 +53,11 @@ for i in {1..50}; do done for i in {1..50}; do file=$SCRATCH_MNT/v1_policy_dir_1/$i - touch $file + echo "0" > $file inodes+=("$(stat -c %i $file)") file=$SCRATCH_MNT/v2_policy_dir_1/$i - touch $file + echo "0" > $file inodes+=("$(stat -c %i $file)") done _scratch_unmount
Signature · 1 line
2.40.1
RFC PATCH 1/8 common/encrypt: separate data and inode nonces
In reply to Sweet Tea Dorminybtrfs will have different inode and data nonces, so we need to be specific about which nonce each use needs. For now, there is no difference in the two functions.
--- common/encrypt | 33 ++++++++++++++++++++++++++------- tests/generic/613 | 4 ++-- 2 files changed, 28 insertions(+), 9 deletions(-)
common/encrypt +26 -7
@@ -488,7 +488,7 @@ _add_fscrypt_provisioning_key() # Retrieve the encryption nonce of the given inode as a hex string. The nonce # was randomly generated by the filesystem and isn't exposed directly to # userspace. But it can be read using the filesystem's debugging tools. -_get_encryption_nonce() +_get_encryption_file_nonce() { local device=$1 local inode=$2
@@ -532,15 +532,34 @@ _get_encryption_nonce() }' ;; *) - _fail "_get_encryption_nonce() isn't implemented on $FSTYP" + _fail "_get_encryption_file_nonce() isn't implemented on $FSTYP" ;; esac } -# Require support for _get_encryption_nonce() +# Retrieve the encryption nonce used to encrypt the data of the given inode as +# a hex string. The nonce was randomly generated by the filesystem and isn't +# exposed directly to userspace. But it can be read using the filesystem's +# debugging tools. +_get_encryption_data_nonce() +{ + local device=$1 + local inode=$2 + + case $FSTYP in + ext4|f2fs) + _get_encryption_file_nonce $device $inode + ;; + *) + _fail "_get_encryption_data_nonce() isn't implemented on $FSTYP" + ;; + esac +} + +# Require support for _get_encryption_*nonce() _require_get_encryption_nonce_support() { - echo "Checking for _get_encryption_nonce() support for $FSTYP" >> $seqres.full + echo "Checking for _get_encryption_*nonce() support for $FSTYP" >> $seqres.full case $FSTYP in ext4) _require_command "$DEBUGFS_PROG" debugfs
@@ -554,7 +573,7 @@ _require_get_encryption_nonce_support() # the test fail in that case, as it was an f2fs-tools bug... ;; *) - _notrun "_get_encryption_nonce() isn't implemented on $FSTYP" + _notrun "_get_encryption_*nonce() isn't implemented on $FSTYP" ;; esac }
@@ -760,7 +779,7 @@ _do_verify_ciphertext_for_encryption_policy() echo "Verifying encrypted file contents" >> $seqres.full for f in "${test_contents_files[@]}"; do read -r src inode blocklist <<< "$f" - nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) + nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) _dump_ciphertext_blocks $SCRATCH_DEV $blocklist > $tmp.actual_contents $crypt_contents_cmd $contents_encryption_mode $raw_key_hex \ --file-nonce=$nonce --block-size=$blocksize \
@@ -780,7 +799,7 @@ _do_verify_ciphertext_for_encryption_policy() echo "Verifying encrypted file names" >> $seqres.full for f in "${test_filenames_files[@]}"; do read -r name inode dir_inode padding <<< "$f" - nonce=$(_get_encryption_nonce $SCRATCH_DEV $dir_inode) + nonce=$(_get_encryption_file_nonce $SCRATCH_DEV $dir_inode) _get_ciphertext_filename $SCRATCH_DEV $inode $dir_inode \ > $tmp.actual_name echo -n "$name" | \
tests/generic/613 +2 -2
@@ -68,10 +68,10 @@ echo -e "\n# Getting encryption nonces from inodes" echo -n > $tmp.nonces_hex echo -n > $tmp.nonces_bin for inode in "${inodes[@]}"; do - nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) + nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) if (( ${#nonce} != 32 )) || [ -n "$(echo "$nonce" | tr -d 0-9a-fA-F)" ] then - _fail "Expected nonce to be 16 bytes (32 hex characters), but got \"$nonce\"" + _fail "Expected nonce for inode $inode to be 16 bytes (32 hex characters), but got \"$nonce\"" fi echo $nonce >> $tmp.nonces_hex echo -ne "$(echo $nonce | sed 's/[0-9a-fA-F]\{2\}/\\x\0/g')" \
Signature · 1 line
2.40.1
RFC PATCH 2/8 common/encrypt: add btrfs to get_encryption_*nonce
In reply to Sweet Tea DorminyAdd the modes of getting the encryption nonces, either inode or extent, to the various get_encryption_nonce functions. For now, no encrypt test makes a file with more than one extent, so we can just grab the first extent's nonce for the data nonce; when we write a bigger file test, we'll need to change that.
--- common/encrypt | 31 +++++++++++++++++++++++++++++++ tests/generic/613 | 4 ++++ 2 files changed, 35 insertions(+)
common/encrypt +31 -0
@@ -531,6 +531,17 @@ _get_encryption_file_nonce() found = 0; }' ;; + btrfs) + # Retrieve the fscrypt context for an inode as a hex string. + # btrfs prints these like: + # item 14 key ($inode FSCRYPT_CTXT_ITEM 0) itemoff 15491 itemsize 40 + # value: 02010400000000008fabf3dd745d41856e812458cd765bf0140f41d62853f4c0351837daff4dcc8f + + $BTRFS_UTIL_PROG inspect-internal dump-tree $device | \ + grep -A 1 "key ($inode FSCRYPT_CTXT_ITEM 0)" | \ + grep --only-matching 'value: [[:xdigit:]]\+' | \ + tr -d ' \n' | tail -c 32 + ;; *) _fail "_get_encryption_file_nonce() isn't implemented on $FSTYP" ;;
@@ -550,6 +561,23 @@ _get_encryption_data_nonce() ext4|f2fs) _get_encryption_file_nonce $device $inode ;; + btrfs) + # Retrieve the encryption IV of the first file extent in an inode as a hex + # string. btrfs prints the file extents (for simple unshared + # inodes) like: + # item 21 key ($inode EXTENT_DATA 0) itemoff 2534 itemsize 69 + # generation 7 type 1 (regular) + # extent data disk byte 5304320 nr 1048576 + # extent data offset 0 nr 1048576 ram 1048576 + # extent compression 0 (none) + # extent encryption 161 ((1, 40: context 0201040200000000116a77667261d7422a4b1ed8c427e685edb7a0d370d0c9d40030333033333330)) + + + $BTRFS_UTIL_PROG inspect-internal dump-tree $device | \ + grep -A 5 "key ($inode EXTENT_DATA 0)" | \ + grep --only-matching 'context [[:xdigit:]]\+' | \ + tr -d ' \n' | tail -c 32 + ;; *) _fail "_get_encryption_data_nonce() isn't implemented on $FSTYP" ;;
@@ -572,6 +600,9 @@ _require_get_encryption_nonce_support() # Otherwise the xattr is incorrectly parsed as v1. But just let # the test fail in that case, as it was an f2fs-tools bug... ;; + btrfs) + _require_command "$BTRFS_UTIL_PROG" btrfs + ;; *) _notrun "_get_encryption_*nonce() isn't implemented on $FSTYP" ;;
tests/generic/613 +4 -0
@@ -69,6 +69,10 @@ echo -n > $tmp.nonces_hex echo -n > $tmp.nonces_bin for inode in "${inodes[@]}"; do nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) + if [ "$FSTYP" == "btrfs" ] && [ "$nonce" == "" ] + then + nonce=$(_get_encryption_file_nonce $SCRATCH_DEV $inode) + fi if (( ${#nonce} != 32 )) || [ -n "$(echo "$nonce" | tr -d 0-9a-fA-F)" ] then _fail "Expected nonce for inode $inode to be 16 bytes (32 hex characters), but got \"$nonce\""
Signature · 1 line
2.40.1
RFC PATCH 6/8 tests: adjust encryption tests for extent encryption
In reply to Sweet Tea DorminyExtent encryption is different from the existing inode-based encryption insofar as it only generates encryption keys for data encryption at the moment at which the data is written. This means that when a key is removed, even if there's an open file using it, that file immediately becomes unreadable and unwritable.
This contradicts the assumptions in three tests. In generic/429, we can issue a sync to push the dirty data to the filesystem before dropping the key. However, generic/580 explicitly wants to write data after dropping the key, and generic/595 wants to intermingle key removals and data writes, which is unpredictable in effect. So just disable those two.
--- tests/generic/429 | 6 ++++++ tests/generic/580 | 4 ++++ tests/generic/595 | 4 ++++ 3 files changed, 14 insertions(+)
tests/generic/429 +6 -0
@@ -68,6 +68,12 @@ show_directory_with_key() show_file_contents } +# btrfs needs to have dirty data pushed into it before session keyring +# is unlinked, as it doesn't set up the data encryption key until then. +if [ "$FSTYP" = "btrfs" ]; then + sync +fi + # View the directory without the encryption key. The plaintext names shouldn't # exist, but 'cat' each to verify this, which also should create negative # dentries. The no-key names are unpredictable by design, but verify that the
tests/generic/580 +4 -0
@@ -23,6 +23,10 @@ _require_scratch_encryption -v 2 _scratch_mkfs_encrypted &>> $seqres.full _scratch_mount +if [ $FSTYP = "btrfs" ]; then + _notrun "extent encryption locks open files immediately on key removal" +fi + test_with_policy_version() { local vers=$1
tests/generic/595 +4 -0
@@ -35,6 +35,10 @@ _require_command "$KEYCTL_PROG" keyctl _scratch_mkfs_encrypted &>> $seqres.full _scratch_mount +if [ $FSTYP = "btrfs" ]; then + _notrun "extent encryption locks open files immediately on key removal" +fi + dir=$SCRATCH_MNT/dir runtime=$((4 * TIME_FACTOR))
Signature · 1 line
2.40.1
RFC PATCH 7/8 common/verity: explicitly don't allow btrfs encryption
In reply to Sweet Tea DorminyCurrently btrfs encryption doesn't support verity, but it is planned to one day. To be explicit about the lack of support, add a custom error message to the combination.
--- common/verity | 4 ++++ 1 file changed, 4 insertions(+)
common/verity +4 -0
@@ -218,6 +218,10 @@ _scratch_mkfs_encrypted_verity() # features with -O. Instead -O must be supplied multiple times. _scratch_mkfs -O encrypt -O verity ;; + btrfs) + # currently verity + encryption is not supported + _notrun "btrfs doesn't currently support verity + encryption" + ;; *) _notrun "$FSTYP not supported in _scratch_mkfs_encrypted_verity" ;;
Signature · 1 line
2.40.1
RFC PATCH 3/8 common/encrypt: add btrfs to get_ciphertext_filename
In reply to Sweet Tea DorminyAdd the relevant call to get an encrypted filename from btrfs.
--- common/encrypt | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
common/encrypt +16 -0
@@ -618,6 +618,19 @@ _get_ciphertext_filename() local dir_inode=$3 case $FSTYP in + btrfs) + # Extract the filename from the inode_ref object, similar to: + # item 24 key (259 INODE_REF 257) itemoff 14826 itemsize 26 + # index 3 namelen 16 name: J\xf7\x15tD\x8eL\xae/\x98\x9f\x09\xc1\xb6\x09> + # + $BTRFS_UTIL_PROG inspect-internal dump-tree $device | \ + grep -A 1 "key ($inode INODE_REF " | tail -n 1 | \ + perl -ne ' + s/.*?name: //; + chomp; + s/\\x([[:xdigit:]]{2})/chr hex $1/eg; + print;' + ;; ext4) # Extract the filename from the debugfs output line like: #
@@ -715,6 +728,9 @@ _require_get_ciphertext_filename_support() _notrun "dump.f2fs (f2fs-tools) is too old; doesn't support showing unambiguous on-disk filenames" fi ;; + btrfs) + _require_command "$BTRFS_UTIL_PROG" btrfs + ;; *) _notrun "_get_ciphertext_filename() isn't implemented on $FSTYP" ;;
Signature · 1 line
2.40.1
RFC PATCH 8/8 btrfs: add simple test of reflink of encrypted data
In reply to Sweet Tea DorminyMake sure that we succeed at reflinking encrypted data.
Test deliberately numbered with a high number so it won't conflict with tests between now and merge.
--- tests/btrfs/613 | 62 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/613.out | 13 ++++++++++ 2 files changed, 75 insertions(+) create mode 100755 tests/btrfs/613 create mode 100644 tests/btrfs/613.out
tests/btrfs/613 new file +62 -0
@@ -0,0 +1,62 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 YOUR NAME HERE. All Rights Reserved. +# +# FS QA Test 613 +# +# Check if reflinking one encrypted file on btrfs succeeds. +# +. ./common/preamble +_begin_fstest auto encrypt + +# Import common functions. +. ./common/encrypt +. ./common/filter +. ./common/reflink + +# real QA test starts here + +# Modify as appropriate. +_supported_fs btrfs + +_require_test +_require_scratch +_require_cp_reflink +_require_scratch_encryption -v 2 +_require_command "$KEYCTL_PROG" keyctl + +_scratch_mkfs_encrypted &>> $seqres.full +_scratch_mount + +dir=$SCRATCH_MNT/dir +mkdir $dir +_set_encpolicy $dir $TEST_KEY_IDENTIFIER +_add_enckey $SCRATCH_MNT "$TEST_RAW_KEY" +echo "Creating and reflinking a file" +$XFS_IO_PROG -t -f -c "pwrite 0 33k" $dir/test > /dev/null +sync +cp --reflink=always $dir/test $dir/test2 +sync + +echo "Can't reflink encrypted and unencrypted" +cp --reflink=always $dir/test $SCRATCH_MNT/fail |& _filter_scratch + +echo "Diffing the file and its copy" +diff $dir/test $dir/test2 + +echo "Verifying the files are reflinked" +_verify_reflink $dir/test $dir/test2 + +echo "Diffing the files after remount" +_scratch_cycle_mount +_add_enckey $SCRATCH_MNT "$TEST_RAW_KEY" +diff $dir/test $dir/test2 + +echo "Diffing the files after key remove" +echo 2 > /proc/sys/vm/drop_caches +_rm_enckey $SCRATCH_MNT $TEST_KEY_IDENTIFIER +diff $dir/test $dir/test2 |& _filter_scratch + +# success, all done +status=0 +exit
tests/btrfs/613.out new file +13 -0
@@ -0,0 +1,13 @@ +QA output created by 613 +Added encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +Creating and reflinking a file +Can't reflink encrypted and unencrypted +cp: failed to clone 'SCRATCH_MNT/fail' from 'SCRATCH_MNT/dir/test': Invalid argument +Diffing the file and its copy +Verifying the files are reflinked +Diffing the files after remount +Added encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +Diffing the files after key remove +Removed encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +diff: SCRATCH_MNT/dir/test: No such file or directory +diff: SCRATCH_MNT/dir/test2: No such file or directory
Signature · 1 line
2.40.1
Re: [RFC PATCH 8/8] btrfs: add simple test of reflink of encrypted data
Quoted 18 lines · Sweet Tea Dorminy
Make sure that we succeed at reflinking encrypted data.
Test deliberately numbered with a high number so it won't conflict with tests between now and merge.
--- tests/btrfs/613 | 62 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/613.out | 13 ++++++++++ 2 files changed, 75 insertions(+) create mode 100755 tests/btrfs/613 create mode 100644 tests/btrfs/613.outtests/btrfs/613 new file +0
-0
tests/btrfs/613 line 3
Quoted 3 lines
tests/btrfs/613 new file +3
-0@@ -0,0 +1,62 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 YOUR NAME HERE. All Rights Reserved.
Don't forget to fill this...
Quoted 23 lines
tests/btrfs/613 new file +23
-0+# +# FS QA Test 613 +# +# Check if reflinking one encrypted file on btrfs succeeds. +# +. ./common/preamble +_begin_fstest auto encrypt + +# Import common functions. +. ./common/encrypt +. ./common/filter +. ./common/reflink + +# real QA test starts here + +# Modify as appropriate. +_supported_fs btrfs + +_require_test +_require_scratch +_require_cp_reflink +_require_scratch_encryption -v 2 +_require_command "$KEYCTL_PROG" keyctl
tests/btrfs/613 line 36
Quoted 10 lines
tests/btrfs/613 new file +10
-0+ +_scratch_mkfs_encrypted &>> $seqres.full +_scratch_mount + +dir=$SCRATCH_MNT/dir +mkdir $dir +_set_encpolicy $dir $TEST_KEY_IDENTIFIER +_add_enckey $SCRATCH_MNT "$TEST_RAW_KEY" +echo "Creating and reflinking a file" +$XFS_IO_PROG -t -f -c "pwrite 0 33k" $dir/test > /dev/null
Why the -t (truncate)? We are creating the file on a brand new fs.
tests/btrfs/613 line 37
Quoted 1 line
tests/btrfs/613 new file +1
-0+sync
What's this sync for? The reflink code flushes delalloc (for both source and destination inodes, always).
Is this really necessary? If so please add a comment explaining why it's needed, otherwise remove it.
tests/btrfs/613 line 39
Quoted 2 lines
tests/btrfs/613 new file +2
-0+cp --reflink=always $dir/test $dir/test2 +sync
Same here. What is this sync for?
Please add a comment explaining why it's needed, otherwise remove it.
Thanks.
Quoted 45 lines
tests/btrfs/613 new file +23
-0+ +echo "Can't reflink encrypted and unencrypted" +cp --reflink=always $dir/test $SCRATCH_MNT/fail |& _filter_scratch + +echo "Diffing the file and its copy" +diff $dir/test $dir/test2 + +echo "Verifying the files are reflinked" +_verify_reflink $dir/test $dir/test2 + +echo "Diffing the files after remount" +_scratch_cycle_mount +_add_enckey $SCRATCH_MNT "$TEST_RAW_KEY" +diff $dir/test $dir/test2 + +echo "Diffing the files after key remove" +echo 2 > /proc/sys/vm/drop_caches +_rm_enckey $SCRATCH_MNT $TEST_KEY_IDENTIFIER +diff $dir/test $dir/test2 |& _filter_scratch + +# success, all done +status=0 +exittests/btrfs/613.out new file +13
-1@@ -0,0 +1,13 @@ +QA output created by 613 +Added encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +Creating and reflinking a file +Can't reflink encrypted and unencrypted +cp: failed to clone 'SCRATCH_MNT/fail' from 'SCRATCH_MNT/dir/test': Invalid argument +Diffing the file and its copy +Verifying the files are reflinked +Diffing the files after remount +Added encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +Diffing the files after key remove +Removed encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +diff: SCRATCH_MNT/dir/test: No such file or directory +diff: SCRATCH_MNT/dir/test2: No such file or directory --2.40.1