fstests: add btrfs encryption testing
Subsystems (3)
v2 cover fstests: add btrfs encryption testing
This is a preliminary fstests side of the btrfs encryption feature; more tests are needed, but this gets the existing encryption tests working with btrfs. This requires the latest related progs and kernel changesets.
Marked as RFC because they're not ready to merge until all its dependencies finish landing; this is primarily to demonstrate that extent encryption, between fscrypt and btrfs, does not significantly change user-visible behavior.
Changelog:
RFC v2:
- Reverted changes to generic/580 and generic/595 to match the new 'soft-delete' behavior introduced in v2 of kernel patchset "fscrypt: add extent encryption". (change 6)
- Removed extraneous syncs/drop_caches and added copyright to new test (change 8), as per Filipe's comments.
RFC v1:
- https://lore.kernel.org/linux-btrfs/[email protected]/T/#t
Sweet Tea Dorminy (8):
common/encrypt: separate data and inode nonces common/encrypt: add btrfs to get_encryption_*nonce common/encrypt: add btrfs to get_ciphertext_filename common/encrypt: enable making a encrypted btrfs filesystem generic/613: write some actual data for btrfs
tests: adjust generic/429 for extent encryption common/verity: explicitly don't allow btrfs encryption
btrfs: add simple test of reflink of encrypted data
common/encrypt | 86 +++++++++++++++++++++++++++++++++++++++++---- common/verity | 4 +++ tests/btrfs/613 | 59 +++++++++++++++++++++++++++++++ tests/btrfs/613.out | 13 +++++++ tests/generic/429 | 6 ++++ tests/generic/613 | 12 ++++--- 6 files changed, 169 insertions(+), 11 deletions(-) create mode 100755 tests/btrfs/613 create mode 100644 tests/btrfs/613.out
base-commit: 87f90a2dae7a4adb7a0a314e27abae9aa1de78fb
Signature · 1 line
2.40.1
RFC PATCH v2 1/8 common/encrypt: separate data and inode nonces
btrfs will have different inode and data nonces, so we need to be specific about which nonce each use needs. For now, there is no difference in the two functions.
--- common/encrypt | 33 ++++++++++++++++++++++++++------- tests/generic/613 | 4 ++-- 2 files changed, 28 insertions(+), 9 deletions(-)
common/encrypt +26 -7
@@ -488,7 +488,7 @@ _add_fscrypt_provisioning_key() # Retrieve the encryption nonce of the given inode as a hex string. The nonce # was randomly generated by the filesystem and isn't exposed directly to # userspace. But it can be read using the filesystem's debugging tools. -_get_encryption_nonce() +_get_encryption_file_nonce() { local device=$1 local inode=$2
@@ -532,15 +532,34 @@ _get_encryption_nonce() }' ;; *) - _fail "_get_encryption_nonce() isn't implemented on $FSTYP" + _fail "_get_encryption_file_nonce() isn't implemented on $FSTYP" ;; esac } -# Require support for _get_encryption_nonce() +# Retrieve the encryption nonce used to encrypt the data of the given inode as +# a hex string. The nonce was randomly generated by the filesystem and isn't +# exposed directly to userspace. But it can be read using the filesystem's +# debugging tools. +_get_encryption_data_nonce() +{ + local device=$1 + local inode=$2 + + case $FSTYP in + ext4|f2fs) + _get_encryption_file_nonce $device $inode + ;; + *) + _fail "_get_encryption_data_nonce() isn't implemented on $FSTYP" + ;; + esac +} + +# Require support for _get_encryption_*nonce() _require_get_encryption_nonce_support() { - echo "Checking for _get_encryption_nonce() support for $FSTYP" >> $seqres.full + echo "Checking for _get_encryption_*nonce() support for $FSTYP" >> $seqres.full case $FSTYP in ext4) _require_command "$DEBUGFS_PROG" debugfs
@@ -554,7 +573,7 @@ _require_get_encryption_nonce_support() # the test fail in that case, as it was an f2fs-tools bug... ;; *) - _notrun "_get_encryption_nonce() isn't implemented on $FSTYP" + _notrun "_get_encryption_*nonce() isn't implemented on $FSTYP" ;; esac }
@@ -760,7 +779,7 @@ _do_verify_ciphertext_for_encryption_policy() echo "Verifying encrypted file contents" >> $seqres.full for f in "${test_contents_files[@]}"; do read -r src inode blocklist <<< "$f" - nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) + nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) _dump_ciphertext_blocks $SCRATCH_DEV $blocklist > $tmp.actual_contents $crypt_contents_cmd $contents_encryption_mode $raw_key_hex \ --file-nonce=$nonce --block-size=$blocksize \
@@ -780,7 +799,7 @@ _do_verify_ciphertext_for_encryption_policy() echo "Verifying encrypted file names" >> $seqres.full for f in "${test_filenames_files[@]}"; do read -r name inode dir_inode padding <<< "$f" - nonce=$(_get_encryption_nonce $SCRATCH_DEV $dir_inode) + nonce=$(_get_encryption_file_nonce $SCRATCH_DEV $dir_inode) _get_ciphertext_filename $SCRATCH_DEV $inode $dir_inode \ > $tmp.actual_name echo -n "$name" | \
tests/generic/613 +2 -2
@@ -68,10 +68,10 @@ echo -e "\n# Getting encryption nonces from inodes" echo -n > $tmp.nonces_hex echo -n > $tmp.nonces_bin for inode in "${inodes[@]}"; do - nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) + nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) if (( ${#nonce} != 32 )) || [ -n "$(echo "$nonce" | tr -d 0-9a-fA-F)" ] then - _fail "Expected nonce to be 16 bytes (32 hex characters), but got \"$nonce\"" + _fail "Expected nonce for inode $inode to be 16 bytes (32 hex characters), but got \"$nonce\"" fi echo $nonce >> $tmp.nonces_hex echo -ne "$(echo $nonce | sed 's/[0-9a-fA-F]\{2\}/\\x\0/g')" \
Signature · 1 line
2.40.1
RFC PATCH v2 2/8 common/encrypt: add btrfs to get_encryption_*nonce
In reply to Sweet Tea DorminyAdd the modes of getting the encryption nonces, either inode or extent, to the various get_encryption_nonce functions. For now, no encrypt test makes a file with more than one extent, so we can just grab the first extent's nonce for the data nonce; when we write a bigger file test, we'll need to change that.
--- common/encrypt | 31 +++++++++++++++++++++++++++++++ tests/generic/613 | 4 ++++ 2 files changed, 35 insertions(+)
common/encrypt +31 -0
@@ -531,6 +531,17 @@ _get_encryption_file_nonce() found = 0; }' ;; + btrfs) + # Retrieve the fscrypt context for an inode as a hex string. + # btrfs prints these like: + # item 14 key ($inode FSCRYPT_CTXT_ITEM 0) itemoff 15491 itemsize 40 + # value: 02010400000000008fabf3dd745d41856e812458cd765bf0140f41d62853f4c0351837daff4dcc8f + + $BTRFS_UTIL_PROG inspect-internal dump-tree $device | \ + grep -A 1 "key ($inode FSCRYPT_CTXT_ITEM 0)" | \ + grep --only-matching 'value: [[:xdigit:]]\+' | \ + tr -d ' \n' | tail -c 32 + ;; *) _fail "_get_encryption_file_nonce() isn't implemented on $FSTYP" ;;
@@ -550,6 +561,23 @@ _get_encryption_data_nonce() ext4|f2fs) _get_encryption_file_nonce $device $inode ;; + btrfs) + # Retrieve the encryption IV of the first file extent in an inode as a hex + # string. btrfs prints the file extents (for simple unshared + # inodes) like: + # item 21 key ($inode EXTENT_DATA 0) itemoff 2534 itemsize 69 + # generation 7 type 1 (regular) + # extent data disk byte 5304320 nr 1048576 + # extent data offset 0 nr 1048576 ram 1048576 + # extent compression 0 (none) + # extent encryption 161 ((1, 40: context 0201040200000000116a77667261d7422a4b1ed8c427e685edb7a0d370d0c9d40030333033333330)) + + + $BTRFS_UTIL_PROG inspect-internal dump-tree $device | \ + grep -A 5 "key ($inode EXTENT_DATA 0)" | \ + grep --only-matching 'context [[:xdigit:]]\+' | \ + tr -d ' \n' | tail -c 32 + ;; *) _fail "_get_encryption_data_nonce() isn't implemented on $FSTYP" ;;
@@ -572,6 +600,9 @@ _require_get_encryption_nonce_support() # Otherwise the xattr is incorrectly parsed as v1. But just let # the test fail in that case, as it was an f2fs-tools bug... ;; + btrfs) + _require_command "$BTRFS_UTIL_PROG" btrfs + ;; *) _notrun "_get_encryption_*nonce() isn't implemented on $FSTYP" ;;
tests/generic/613 +4 -0
@@ -69,6 +69,10 @@ echo -n > $tmp.nonces_hex echo -n > $tmp.nonces_bin for inode in "${inodes[@]}"; do nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) + if [ "$FSTYP" == "btrfs" ] && [ "$nonce" == "" ] + then + nonce=$(_get_encryption_file_nonce $SCRATCH_DEV $inode) + fi if (( ${#nonce} != 32 )) || [ -n "$(echo "$nonce" | tr -d 0-9a-fA-F)" ] then _fail "Expected nonce for inode $inode to be 16 bytes (32 hex characters), but got \"$nonce\""
Signature · 1 line
2.40.1
RFC PATCH v2 4/8 common/encrypt: enable making a encrypted btrfs filesystem
In reply to Sweet Tea Dorminy--- common/encrypt | 6 ++++++ 1 file changed, 6 insertions(+)
common/encrypt +6 -0
@@ -153,6 +153,9 @@ _scratch_mkfs_encrypted() # erase the UBI volume; reformated automatically on next mount $UBIUPDATEVOL_PROG ${SCRATCH_DEV} -t ;; + btrfs) + _scratch_mkfs + ;; ceph) _scratch_cleanup_files ;;
@@ -168,6 +171,9 @@ _scratch_mkfs_sized_encrypted() ext4|f2fs) MKFS_OPTIONS="$MKFS_OPTIONS -O encrypt" _scratch_mkfs_sized $* ;; + btrfs) + _scratch_mkfs_sized $* + ;; *) _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized_encrypted" ;;
Signature · 1 line
2.40.1
RFC PATCH v2 3/8 common/encrypt: add btrfs to get_ciphertext_filename
In reply to Sweet Tea DorminyAdd the relevant call to get an encrypted filename from btrfs.
--- common/encrypt | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
common/encrypt +16 -0
@@ -618,6 +618,19 @@ _get_ciphertext_filename() local dir_inode=$3 case $FSTYP in + btrfs) + # Extract the filename from the inode_ref object, similar to: + # item 24 key (259 INODE_REF 257) itemoff 14826 itemsize 26 + # index 3 namelen 16 name: J\xf7\x15tD\x8eL\xae/\x98\x9f\x09\xc1\xb6\x09> + # + $BTRFS_UTIL_PROG inspect-internal dump-tree $device | \ + grep -A 1 "key ($inode INODE_REF " | tail -n 1 | \ + perl -ne ' + s/.*?name: //; + chomp; + s/\\x([[:xdigit:]]{2})/chr hex $1/eg; + print;' + ;; ext4) # Extract the filename from the debugfs output line like: #
@@ -715,6 +728,9 @@ _require_get_ciphertext_filename_support() _notrun "dump.f2fs (f2fs-tools) is too old; doesn't support showing unambiguous on-disk filenames" fi ;; + btrfs) + _require_command "$BTRFS_UTIL_PROG" btrfs + ;; *) _notrun "_get_ciphertext_filename() isn't implemented on $FSTYP" ;;
Signature · 1 line
2.40.1
RFC PATCH v2 5/8 generic/613: write some actual data for btrfs
In reply to Sweet Tea DorminyCurrently, the test touches a file and assumes that that is sufficient to generate a new nonce to test for that file. However, btrfs doesn't store an encryption context for a leaf inode, and doesn't store an encryption context for data within a leaf inode until data is actually written. Thus, merely touching the file on btrfs doesn't actually generate a testable nonce.
Instead, write a trivial bit of data to each file, which provokes btrfs to generate a encryption context for the data and thus a testable nonce.
--- tests/generic/613 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/generic/613 +2 -2
@@ -53,11 +53,11 @@ for i in {1..50}; do done for i in {1..50}; do file=$SCRATCH_MNT/v1_policy_dir_1/$i - touch $file + echo "0" > $file inodes+=("$(stat -c %i $file)") file=$SCRATCH_MNT/v2_policy_dir_1/$i - touch $file + echo "0" > $file inodes+=("$(stat -c %i $file)") done _scratch_unmount
Signature · 1 line
2.40.1
RFC PATCH v2 6/8 tests: adjust generic/429 for extent encryption
In reply to Sweet Tea DorminyExtent encryption is different from the existing inode-based encryption insofar as it only generates encryption keys for data encryption at the moment at which the data is written. This means that when a session key is removed, even if there's an open file using it, that file immediately becomes unreadable and unwritable.
This isn't an issue for non-session keys, which are soft deleted by fscrypt and stick around until there are no more open files with extent encryption using them. But for session keys, which are managed by the kernel keyring directly instead of through fscrypt, when they're removed they're removed.
generic/429 uses session keys and expects to use the written data after key removal; while it's not quite what the test means for other filesystems, most of the test is still meaningful if we push the dirty data into the filesystem with a sync before dropping the key.
--- tests/generic/429 | 6 ++++++ 1 file changed, 6 insertions(+)
tests/generic/429 +6 -0
@@ -68,6 +68,12 @@ show_directory_with_key() show_file_contents } +# btrfs needs to have dirty data pushed into it before session keyring +# is unlinked, as it doesn't set up the data encryption key until then. +if [ "$FSTYP" = "btrfs" ]; then + sync +fi + # View the directory without the encryption key. The plaintext names shouldn't # exist, but 'cat' each to verify this, which also should create negative # dentries. The no-key names are unpredictable by design, but verify that the
Signature · 1 line
2.40.1
RFC PATCH v2 7/8 common/verity: explicitly don't allow btrfs encryption
In reply to Sweet Tea DorminyCurrently btrfs encryption doesn't support verity, but it is planned to one day. To be explicit about the lack of support, add a custom error message to the combination.
--- common/verity | 4 ++++ 1 file changed, 4 insertions(+)
common/verity +4 -0
@@ -218,6 +218,10 @@ _scratch_mkfs_encrypted_verity() # features with -O. Instead -O must be supplied multiple times. _scratch_mkfs -O encrypt -O verity ;; + btrfs) + # currently verity + encryption is not supported + _notrun "btrfs doesn't currently support verity + encryption" + ;; *) _notrun "$FSTYP not supported in _scratch_mkfs_encrypted_verity" ;;
Signature · 1 line
2.40.1
RFC PATCH v2 8/8 btrfs: add simple test of reflink of encrypted data
In reply to Sweet Tea DorminyMake sure that we succeed at reflinking encrypted data.
Test deliberately numbered with a high number so it won't conflict with tests between now and merge.
--- tests/btrfs/613 | 59 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/613.out | 13 ++++++++++ 2 files changed, 72 insertions(+) create mode 100755 tests/btrfs/613 create mode 100644 tests/btrfs/613.out
tests/btrfs/613 new file +59 -0
@@ -0,0 +1,59 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Meta Platforms, Inc. All Rights Reserved. +# +# FS QA Test 613 +# +# Check if reflinking one encrypted file on btrfs succeeds. +# +. ./common/preamble +_begin_fstest auto encrypt + +# Import common functions. +. ./common/encrypt +. ./common/filter +. ./common/reflink + +# real QA test starts here + +# Modify as appropriate. +_supported_fs btrfs + +_require_test +_require_scratch +_require_cp_reflink +_require_scratch_encryption -v 2 +_require_command "$KEYCTL_PROG" keyctl + +_scratch_mkfs_encrypted &>> $seqres.full +_scratch_mount + +dir=$SCRATCH_MNT/dir +mkdir $dir +_set_encpolicy $dir $TEST_KEY_IDENTIFIER +_add_enckey $SCRATCH_MNT "$TEST_RAW_KEY" +echo "Creating and reflinking a file" +$XFS_IO_PROG -t -f -c "pwrite 0 33k" $dir/test > /dev/null +cp --reflink=always $dir/test $dir/test2 + +echo "Can't reflink encrypted and unencrypted" +cp --reflink=always $dir/test $SCRATCH_MNT/fail |& _filter_scratch + +echo "Diffing the file and its copy" +diff $dir/test $dir/test2 + +echo "Verifying the files are reflinked" +_verify_reflink $dir/test $dir/test2 + +echo "Diffing the files after remount" +_scratch_cycle_mount +_add_enckey $SCRATCH_MNT "$TEST_RAW_KEY" +diff $dir/test $dir/test2 + +echo "Diffing the files after key remove" +_rm_enckey $SCRATCH_MNT $TEST_KEY_IDENTIFIER +diff $dir/test $dir/test2 |& _filter_scratch + +# success, all done +status=0 +exit
tests/btrfs/613.out new file +13 -0
@@ -0,0 +1,13 @@ +QA output created by 613 +Added encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +Creating and reflinking a file +Can't reflink encrypted and unencrypted +cp: failed to clone 'SCRATCH_MNT/fail' from 'SCRATCH_MNT/dir/test': Invalid argument +Diffing the file and its copy +Verifying the files are reflinked +Diffing the files after remount +Added encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +Diffing the files after key remove +Removed encryption key with identifier 69b2f6edeee720cce0577937eb8a6751 +diff: SCRATCH_MNT/dir/test: No such file or directory +diff: SCRATCH_MNT/dir/test2: No such file or directory
Signature · 1 line
2.40.1
Re: [RFC PATCH v2 1/8] common/encrypt: separate data and inode nonces
In reply to Sweet Tea DorminyQuoted 18 lines
tests/generic/613 +2
-2@@ -68,10 +68,10 @@ echo -e "\n# Getting encryption nonces from inodes" echo -n > $tmp.nonces_hex echo -n > $tmp.nonces_bin for inode in "${inodes[@]}"; do - nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) + nonce=$(_get_encryption_data_nonce $SCRATCH_DEV $inode) if (( ${#nonce} != 32 )) || [ -n "$(echo "$nonce" | tr -d 0-9a-fA-F)" ] then - _fail "Expected nonce to be 16 bytes (32 hex characters), but got \"$nonce\"" + _fail "Expected nonce for inode $inode to be 16 bytes (32 hex characters), but got \"$nonce\"" fi
Quoted 2 lines
echo $nonce >> $tmp.nonces_hex
echo -ne "$(echo $nonce | sed 's/[0-9a-fA-F]\{2\}/\\x\0/g')" \Also, the test case f2fs/002 refers to _get_encryption_nonce().
nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode)
Thanks, Anand