Inline Encryption Support
Subsystems (8)
v6 cover Inline Encryption Support
This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.
Note that the patches in this series for the block layer (i.e. patches 1, 2 and 3) can be applied independently of the subsequent patches in this series.
Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device. Inline Encryption hardware has become increasingly common, and we want to support it in the kernel.
Inline Encryption hardware implementations often function around the concept of a limited number of "keyslots", which can hold an encryption context each. The storage device can be directed to en/decrypt any particular request with the encryption context stored in any particular keyslot.
Patch 1 introduces a Keyslot Manager to efficiently manage keyslots. The keyslot manager also functions as the interface that blk-crypto (introduced in Patch 3), will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.
Patch 2 introduces struct bio_crypt_ctx, and a ptr to one in struct bio, which allows struct bio to represent an encryption context that can be passed down the storage stack from the filesystem layer to the storage driver.
Patch 3 introduces blk-crypto. Blk-crypto delegates crypto operations to inline encryption hardware when available, and also contains a software fallback to the kernel crypto API. Blk-crypto also makes it possible for layered devices like device mapper to make use of inline encryption hardware. Given that blk-crypto works as a software fallback, it may be possible to remove file content en/decryption from fscrypt and simply use blk-crypto in a future patch. For more details on blk-crypto, refer to Documentation/block/inline-encryption.rst.
Patches 4-6 add support for inline encryption into the UFS driver according to the JEDEC UFS HCI v2.1 specification. Inline encryption support for other drivers (like eMMC) may be added in the same way - the device driver should set up a Keyslot Manager in the device's request_queue (refer to the UFS crypto additions in ufshcd-crypto.c and ufshcd.c for an example).
Patch 7 adds support to fscrypt - to use inline encryption with fscrypt, the filesystem must be mounted with '-o inlinecrypt' - when this option is specified, the contents of any AES-256-XTS encrypted file will be encrypted using blk-crypto.
Patches 8 and 9 add support to f2fs and ext4 respectively, so that we have a complete stack that can make use of inline encryption.
The patches were tested running kvm-xfstests, by specifying the introduced "inlinecrypt" mount option, so that en/decryption happens with the blk-crypto fallback. The patches were also tested on a Pixel 4 with UFS hardware that has support for inline encryption.
There have been a few patch sets addressing Inline Encryption Support in the past. Briefly, this patch set differs from those as follows:
1) "crypto: qce: ice: Add support for Inline Crypto Engine" is specific to certain hardware, while our patch set's Inline Encryption support for UFS is implemented according to the JEDEC UFS specification.
2) "scsi: ufs: UFS Host Controller crypto changes" registers inline encryption support as a kernel crypto algorithm. Our patch views inline encryption as being fundamentally different from a generic crypto provider (in that inline encryption is tied to a device), and so does not use the kernel crypto API to represent inline encryption hardware.
3) "scsi: ufs: add real time/inline crypto support to UFS HCD" requires the device mapper to work - our patch does not.
Changes v5 => v6:
- Blk-crypto's kernel crypto API fallback is no longer restricted to 8-byte DUNs. It's also now separately configurable from blk-crypto, and can be disabled entirely, while still allowing the kernel to use inline encryption hardware. Further, struct bio_crypt_ctx takes up less space, and no longer contains the information needed by the crypto API fallback - the fallback allocates the required memory when necessary.
- Blk-crypto now supports all file content encryption modes supported by fscrypt.
- Fixed bio merging logic in blk-merge.c
- Fscrypt now supports inline encryption with the direct key policy, since blk-crypto now has support for larger DUNs.
- Keyslot manager now uses a hashtable to lookup which keyslot contains any particular key (thanks Eric!)
- Fscrypt support for inline encryption now handles filesystems with multiple underlying block devices (thanks Eric!)
- Numerous cleanups
Changes v4 => v5:
- The fscrypt patch has been separated into 2. The first adds support for the IV_INO_LBLK_64 policy (which was called INLINE_CRYPT_OPTIMIZED in past versions of this series). This policy is now purely an on disk format, and doesn't dictate whether blk-crypto is used for file content encryption or not. Instead, this is now decided based on the "inlinecrypt" mount option.
- Inline crypto key eviction is now handled by blk-crypto instead of fscrypt.
- More refactoring.
Changes v3 => v4:
- Fixed the issue with allocating crypto_skcipher in blk_crypto_keyslot_program.
- bio_crypto_alloc_ctx is now mempool backed.
- In f2fs, a bio's bi_crypt_context is now set up when the bio is allocated, rather than just before the bio is submitted - this fixes bugs in certain cases, like when an encrypted block is being moved without decryption.
- Lots of refactoring and cleanup of blk-crypto - thanks Eric!
Changes v2 => v3:
- Overhauled keyslot manager's get keyslot logic and optimized LRU.
- Block crypto en/decryption fallback now supports data unit sizes that divide the bvec length (instead of requiring each bvec's length to be the same as the data unit size).
- fscrypt master key is now keyed additionally by super_block and ci_ctfm != NULL.
- all references of "hw encryption" are replaced by inline encryption.
- address various other review comments from Eric.
Changes v1 => v2:
- Block layer and UFS changes are split into 3 patches each.
- We now only have a ptr to a struct bio_crypt_ctx in struct bio, instead of the struct itself.
- struct bio_crypt_ctx no longer has flags.
- blk-crypto now correctly handles the case when it fails to init (because of insufficient memory), but kernel continues to boot.
- ufshcd-crypto now works on big endian cpus.
- Many cleanups.
Satya Tangirala (9):
block: Keyslot Manager for Inline Encryption
block: Add encryption context to struct bio
block: blk-crypto for Inline Encryption
scsi: ufs: UFS driver v2.1 spec crypto additions
scsi: ufs: UFS crypto API
scsi: ufs: Add inline encryption support to UFS
fscrypt: add inline encryption support
f2fs: add inline encryption support
ext4: add inline encryption support
Documentation/block/index.rst | 1 + Documentation/block/inline-encryption.rst | 183 ++++++ block/Kconfig | 17 + block/Makefile | 3 + block/bio-crypt-ctx.c | 140 +++++ block/bio.c | 21 +- block/blk-core.c | 16 +- block/blk-crypto-fallback.c | 648 ++++++++++++++++++++++ block/blk-crypto-internal.h | 58 ++ block/blk-crypto.c | 242 ++++++++ block/blk-merge.c | 11 + block/bounce.c | 12 +- block/keyslot-manager.c | 426 ++++++++++++++ drivers/md/dm.c | 3 +- drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd-crypto.c | 391 +++++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 107 ++++ drivers/scsi/ufs/ufshcd.c | 56 +- drivers/scsi/ufs/ufshcd.h | 25 + drivers/scsi/ufs/ufshci.h | 67 ++- fs/buffer.c | 2 + fs/crypto/Kconfig | 6 + fs/crypto/Makefile | 1 + fs/crypto/bio.c | 28 +- fs/crypto/crypto.c | 2 +- fs/crypto/fname.c | 4 +- fs/crypto/fscrypt_private.h | 122 +++- fs/crypto/inline_crypt.c | 319 +++++++++++ fs/crypto/keyring.c | 4 +- fs/crypto/keysetup.c | 102 ++-- fs/crypto/keysetup_v1.c | 16 +- fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 4 +- fs/ext4/page-io.c | 6 +- fs/ext4/readpage.c | 11 +- fs/ext4/super.c | 13 + fs/f2fs/data.c | 65 ++- fs/f2fs/f2fs.h | 3 + fs/f2fs/super.c | 41 ++ include/linux/bio-crypt-ctx.h | 193 +++++++ include/linux/bio.h | 1 + include/linux/blk-crypto.h | 63 +++ include/linux/blk_types.h | 6 + include/linux/blkdev.h | 6 + include/linux/fscrypt.h | 58 ++ include/linux/keyslot-manager.h | 60 ++ 47 files changed, 3462 insertions(+), 112 deletions(-) create mode 100644 Documentation/block/inline-encryption.rst create mode 100644 block/bio-crypt-ctx.c create mode 100644 block/blk-crypto-fallback.c create mode 100644 block/blk-crypto-internal.h create mode 100644 block/blk-crypto.c create mode 100644 block/keyslot-manager.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h create mode 100644 fs/crypto/inline_crypt.c create mode 100644 include/linux/bio-crypt-ctx.h create mode 100644 include/linux/blk-crypto.h create mode 100644 include/linux/keyslot-manager.h
Signature · 1 line
2.24.1.735.g03f4e72817-goog
PATCH v6 1/9 block: Keyslot Manager for Inline Encryption
Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.
Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold an encryption context (we say that an encryption context can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the encryption context programmed into that associated keyslot. As keyslots are limited, and programming keys may be expensive in many implementations, and multiple requests may use exactly the same encryption contexts, we introduce a Keyslot Manager to efficiently manage keyslots. We also introduce a blk_crypto_key, which will represent the key that's programmed into keyslots managed by keyslot managers. The keyslot manager also functions as the interface that upper layers will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.
--- block/Kconfig | 7 + block/Makefile | 1 + block/keyslot-manager.c | 426 ++++++++++++++++++++++++++++++++ include/linux/bio-crypt-ctx.h | 49 ++++ include/linux/bio.h | 1 + include/linux/blkdev.h | 6 + include/linux/keyslot-manager.h | 60 +++++ 7 files changed, 550 insertions(+) create mode 100644 block/keyslot-manager.c create mode 100644 include/linux/bio-crypt-ctx.h create mode 100644 include/linux/keyslot-manager.h
block/Kconfig +7 -0
@@ -181,6 +181,13 @@ config BLK_SED_OPAL Enabling this option enables users to setup/unlock/lock Locking ranges for SED devices using the Opal protocol. +config BLK_INLINE_ENCRYPTION + bool "Enable inline encryption support in block layer" + help + Build the blk-crypto subsystem. Enabling this lets the + block layer handle encryption, so users can take + advantage of inline encryption hardware if present. + menu "Partition Types" source "block/partitions/Kconfig"
block/Makefile +1 -0
@@ -37,3 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o
block/keyslot-manager.c new file +426 -0
426 lines not on this page · raw message
include/linux/bio-crypt-ctx.h new file +49 -0
49 lines not on this page · raw message
include/linux/bio.h +1 -0
@@ -8,6 +8,7 @@#include <linux/highmem.h> #include <linux/mempool.h> #include <linux/ioprio.h> +#include <linux/bio-crypt-ctx.h> #ifdef CONFIG_BLOCK /* struct bio, bio_vec and BIO_* flags are defined in blk_types.h */
include/linux/blkdev.h +6 -0
@@ -43,6 +43,7 @@ struct pr_ops;struct rq_qos ;struct blk_queue_stats ;struct blk_stat_callback ; +struct keyslot_manager ;#define BLKDEV_MIN_RQ 4#define BLKDEV_MAX_RQ 128/* Default maximum */
@@ -474,6 +475,11 @@ struct request_queue {unsigned int dma_pad_mask;unsigned int dma_alignment; +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +/* Inline crypto capabilities */ +struct keyslot_manager *ksm; +#endif +unsigned int rq_timeout;int poll_nsec;
include/linux/keyslot-manager.h new file +60 -0
60 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 1/9] block: Keyslot Manager for Inline Encryption
Quoted 2 lines · Satya Tangirala
Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.)
These four things (key, algorithm, DUN, and data unit size) fully specify the crypto that is done. So the use of "etc." is a bit misleading.
Quoted 6 lines
along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.
Quoted 16 lines
Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold an encryption context (we say that an encryption context can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the encryption context programmed into that associated keyslot. As keyslots are limited, and programming keys may be expensive in many implementations, and multiple requests may use exactly the same encryption contexts, we introduce a Keyslot Manager to efficiently manage keyslots. We also introduce a
blk_crypto_key, which will represent the key that's programmed into keyslots managed by keyslot managers. The keyslot manager also functions as the interface that upper layers will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found inblock/keyslot-manager.candlinux/keyslot-manager.h.
Long paragraphs are hard to read. Maybe split this into multiple paragraphs.
Quoted 17 lines
block/keyslot-manager.c new file +17
-0+/** +* keyslot_manager_crypto_mode_supported() - Find out if a crypto_mode/data +* unit size combination is supported +* by a ksm. +* @ksm: The keyslot manager to check +* @crypto_mode: The crypto mode to check for. +* @data_unit_size: The data_unit_size for the mode. +* +* Calls and returns the result of the crypto_mode_supported function specified +* by the ksm. +* +* Context: Process context. +* Return: Whether or not this ksm supports the specified crypto_mode/ +* data_unit_size combo. +*/ +bool keyslot_manager_crypto_mode_supported(struct keyslot_manager *ksm, + enum blk_crypto_mode_num crypto_mode,
block/keyslot-manager.c line 333
Quoted 10 lines
block/keyslot-manager.c new file +10
-0+ unsigned int data_unit_size) +{ +if (!ksm) +return false; +if (WARN_ON (crypto_mode >=BLK_ENCRYPTION_MODE_MAX )) +return false; +if (WARN_ON (!is_power_of_2 (data_unit_size))) +return false; +return ksm->crypto_mode_supported[crypto_mode] & data_unit_size; +}
There's no crypto_mode_supported() function anymore, so the comment above this function is outdated, including both the part that mentions crypto_mode_supported() and the part that says "Process context".
Also, since C enums are signed, and there's already a check for invalid crypto_mode, it might be a good idea to catch crypto_mode < 0 too:
if (WARN_ON((unsigned int)crypto_mode >= BLK_ENCRYPTION_MODE_MAX))
Quoted 15 lines
block/keyslot-manager.c new file +15
-0+/** +* keyslot_manager_evict_key() - Evict a key from the lower layer device. +* @ksm: The keyslot manager to evict from +* @key: The key to evict +* +* Find the keyslot that the specified key was programmed into, and evict that +* slot from the lower layer device if that slot is not currently in use. +* +* Context: Process context. Takes and releases ksm->lock. +* Return: 0 on success, -EBUSY if the key is still in use, or another +* -errno value on other error. +*/ +int keyslot_manager_evict_key (struct keyslot_manager *ksm, +const struct blk_crypto_key *key) +{
block/keyslot-manager.c line 359
Quoted 10 lines
block/keyslot-manager.c new file +10
-0+int slot; +int err; +struct keyslot *slotp; + +down_write (&ksm->lock); + slot =find_keyslot (ksm, key); +if (slot <0 ) { + err = slot; + goto out_unlock; + }
I think this function should return 0 (rather than fail with -ENOKEY) if the key is not currently programmed into a keyslot. Otherwise anyone who wants to print a warning if key eviction failed will have to ignore the -ENOKEY error.
(Note that this change would require an small update to the function comment.)
- Eric
Re: [PATCH v6 1/9] block: Keyslot Manager for Inline Encryption
In reply to Satya TangiralaQuoted 16 lines
block/keyslot-manager.c new file +16
-0+struct keyslot_manager { +unsigned int num_slots; +struct keyslot_mgmt_ll_ops ksm_ll_ops; +unsigned int crypto_mode_supported[BLK_ENCRYPTION_MODE_MAX ]; +void *ll_priv_data; + +/* Protects programming and evicting keys from the device */ +struct rw_semaphore lock; + +/* List of idle slots, with least recently used slot at front */ +wait_queue_head_t idle_slots_wait_queue; +struct list_head idle_slots; +spinlock_t idle_slots_lock; + +/* +* Hash table which maps key hashes to keyslots, so that we can find a
block/keyslot-manager.c line 67
Quoted 10 lines
block/keyslot-manager.c new file +10
-0+* key's keyslot in O(1) time rather than O(num_slots). Protected by +* 'lock'. A cryptographic hash function is used so that timing attacks +* can't leak information about the raw keys. +*/ +struct hlist_head *slot_hashtable; +unsigned int slot_hashtable_size; + +/* Per-keyslot data */ +struct keyslot slots[]; +};
Is there a rationale for making this structure private? If it was exposed we could embedd it into the containing structure (although the slots would need a dynamic allocation), and instead of the keyslot_manager_private helper, the caller could simply use container_of.
block/keyslot-manager.c line 93
Quoted 4 lines
block/keyslot-manager.c new file +4
-0+struct keyslot_manager *keyslot_manager_create (unsigned int num_slots, +const struct keyslot_mgmt_ll_ops *ksm_ll_ops, +const unsigned int crypto_mode_supported[BLK_ENCRYPTION_MODE_MAX ], +void *ll_priv_data)
.. and then the caller could simply set the ops and the supported modes array directly in the structure, simplifying the interface even further.
Quoted 5 lines
block/keyslot-manager.c new file +5
-0+static int find_keyslot (struct keyslot_manager *ksm, +const struct blk_crypto_key *key) +{ +const struct hlist_head *head =hash_bucket_for_key (ksm, key); +const struct keyslot *slotp;
block/keyslot-manager.c line 176
Quoted 10 lines
block/keyslot-manager.c new file +10
-0+ +hlist_for_each_entry (slotp, head, hash_node) { +if (slotp->key.hash == key->hash && + slotp->key.crypto_mode == key->crypto_mode && + slotp->key.data_unit_size == key->data_unit_size && + !crypto_memneq (slotp->key.raw, key->raw, key->size)) +return slotp - ksm->slots; + } +return -ENOKEY ; +}
I'd return the actual slot pointer here, as that seems the more natural fit. Then factor the pointer arithmetics into a little helper to make it obvious for those few places that need the actual slot number.
Also can you add proper subsystem prefix to the various symbol names?
Quoted 20 lines
block/keyslot-manager.c new file +20
-0+void keyslot_manager_get_slot (struct keyslot_manager *ksm,unsigned int slot) +{ +if (WARN_ON (slot >= ksm->num_slots)) +return ; + +WARN_ON (atomic_inc_return (&ksm->slots[slot].slot_refs) <2 ); +} + +/** +* keyslot_manager_put_slot() - Release a reference to a slot +* @ksm: The keyslot manager to release the reference from. +* @slot: The slot to release the reference from. +* +* Context: Any context. +*/ +void keyslot_manager_put_slot (struct keyslot_manager *ksm,unsigned int slot) +{ +unsigned long flags; + +if (WARN_ON (slot >= ksm->num_slots))
block/keyslot-manager.c line 305
Quoted 10 lines
block/keyslot-manager.c new file +10
-0+return ; + +if (atomic_dec_and_lock_irqsave (&ksm->slots[slot].slot_refs, + &ksm->idle_slots_lock, flags)) { +list_add_tail (&ksm->slots[slot].idle_slot_node, + &ksm->idle_slots); +spin_unlock_irqrestore (&ksm->idle_slots_lock, flags); +wake_up (&ksm->idle_slots_wait_queue); + } +}
How about passing the bio_crypt_ctx structure instead of the not very nicely typed slot index? Also if we merge the files both these helpers should probably just go away and me merged into the 1 or 2 callers that exist.
Quoted 20 lines
include/linux/bio-crypt-ctx.h new file +20
-0+#ifdef CONFIG_BLK_INLINE_ENCRYPTION + +#define BLK_CRYPTO_MAX_KEY_SIZE 64 + +/** +* struct blk_crypto_key - an inline encryption key +* @crypto_mode: encryption algorithm this key is for +* @data_unit_size: the data unit size for all encryption/decryptions with this +* key. This is the size in bytes of each individual plaintext and +* ciphertext. This is always a power of 2. It might be e.g. the +* filesystem block size or the disk sector size. +* @data_unit_size_bits: log2 of data_unit_size +* @size: size of this key in bytes (determined by @crypto_mode) +* @hash: hash of this key, for keyslot manager use only +* @raw: the raw bytes of this key. Only the first @size bytes are used. +* +* A blk_crypto_key is immutable once created, and many bios can reference it at +* the same time. It must not be freed until all bios using it have completed. +*/ +struct blk_crypto_key {
include/linux/bio-crypt-ctx.h line 48
Quoted 10 lines
include/linux/bio-crypt-ctx.h new file +10
-0+enum blk_crypto_mode_num crypto_mode; +unsigned int data_unit_size; +unsigned int data_unit_size_bits; +unsigned int size; +unsigned int hash; +u8 raw[BLK_CRYPTO_MAX_KEY_SIZE ]; +}; + +#endif /* CONFIG_BLK_INLINE_ENCRYPTION */ +#endif /* CONFIG_BLOCK */
I don't think we need any ifdefs around these declarations.
PATCH v6 2/9 block: Add encryption context to struct bio
In reply to Satya TangiralaWe must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the filesystem/fscrypt that knows about and manages encryption contexts. As such, when the filesystem layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.
We want to communicate the encryption context from the filesystem layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct bio_crypt_ctx to struct bio, which can represent an encryption context (note that we can't use the bi_private field in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate the bio_crypt_ctx and make the bio/request merging logic aware of the bio_crypt_ctx.
--- block/Makefile | 2 +- block/bio-crypt-ctx.c | 131 ++++++++++++++++++++++++++++++ block/bio.c | 16 ++-- block/blk-core.c | 3 + block/blk-merge.c | 11 +++ block/bounce.c | 12 ++- drivers/md/dm.c | 3 +- include/linux/bio-crypt-ctx.h | 146 +++++++++++++++++++++++++++++++++- include/linux/blk_types.h | 6 ++ 9 files changed, 312 insertions(+), 18 deletions(-) create mode 100644 block/bio-crypt-ctx.c
block/Makefile +1 -1
@@ -37,4 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.o
block/bio-crypt-ctx.c new file +131 -0
131 lines not on this page · raw message
block/bio.c +8 -8
@@ -236,6 +236,8 @@ void bio_uninit(struct bio *bio)if (bio_integrity (bio))bio_integrity_free (bio); + +bio_crypt_free_ctx (bio); }EXPORT_SYMBOL (bio_uninit);
@@ -615,15 +617,12 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)__bio_clone_fast (b, bio); -if (bio_integrity (bio)) { -int ret; - - ret =bio_integrity_clone (b, bio, gfp_mask); +bio_crypt_clone (b, bio, gfp_mask); -if (ret <0 ) { -bio_put (b); -return NULL ; - } +if (bio_integrity (bio) && +bio_integrity_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ; }return b;
@@ -997,6 +996,7 @@ void bio_advance(struct bio *bio, unsigned bytes)if (bio_integrity (bio))bio_integrity_advance (bio, bytes); +bio_crypt_advance (bio, bytes);bio_advance_iter (bio, &bio->bi_iter, bytes); }EXPORT_SYMBOL (bio_advance);
block/blk-core.c +3 -0
@@ -1810,5 +1810,8 @@ int __init blk_dev_init(void) blk_debugfs_root =debugfs_create_dir ("block" ,NULL );#endif +if (bio_crypt_ctx_init () <0 ) +panic ("Failed to allocate mem for bio crypt ctxs\n" ); +return 0 ; }
block/blk-merge.c +11 -0
@@ -596,6 +596,8 @@ int ll_back_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs)req_set_nomerge (req->q, req);return 0 ; } +if (!bio_crypt_ctx_mergeable (req->bio,blk_rq_bytes (req), bio)) +return 0 ;return ll_new_hw_segment (req, bio, nr_segs); }
@@ -612,6 +614,8 @@ int ll_front_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segsreq_set_nomerge (req->q, req);return 0 ; } +if (!bio_crypt_ctx_mergeable (bio, bio->bi_iter.bi_size, req->bio)) +return 0 ;return ll_new_hw_segment (req, bio, nr_segs); }
@@ -656,6 +660,9 @@ static int ll_merge_requests_fn(struct request_queue *q, struct request *req,if (blk_integrity_merge_rq (q, req, next) == false)return 0 ; +if (!bio_crypt_ctx_mergeable (req->bio,blk_rq_bytes (req), next->bio)) +return 0 ; +/* Merge is OK... */ req->nr_phys_segments = total_phys_segments;return 1 ;
@@ -895,6 +902,10 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)if (rq->ioprio !=bio_prio (bio))return false; +/* Only merge if the crypt contexts are compatible */ +if (!bio_crypt_ctx_compatible (bio, rq->bio)) +return false; +return true; }
block/bounce.c +5 -7
@@ -267,14 +267,12 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,break ; } -if (bio_integrity (bio_src)) { -int ret; +bio_crypt_clone (bio, bio_src, gfp_mask); - ret =bio_integrity_clone (bio, bio_src, gfp_mask); -if (ret <0 ) { -bio_put (bio); -return NULL ; - } +if (bio_integrity (bio_src) && +bio_integrity_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ; }bio_clone_blkg_association (bio, bio_src);
drivers/md/dm.c +2 -1
@@ -1304,9 +1304,10 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,__bio_clone_fast (clone, bio); +bio_crypt_clone (clone, bio,GFP_NOIO ); +if (bio_integrity (bio)) {int r; -if (unlikely (!dm_target_has_integrity (tio->ti->type) && !dm_target_passes_integrity (tio->ti->type))) { DMWARN("%s: the target %s doesn't support integrity data.",
include/linux/bio-crypt-ctx.h +145 -1
158 lines not on this page · raw message
include/linux/blk_types.h +6 -0
@@ -18,6 +18,7 @@ struct block_device;struct io_context ;struct cgroup_subsys_state ;typedef void (bio_end_io_t ) (struct bio *); +struct bio_crypt_ctx ;/* * Block error status values. See block/blk-core:blk_errors for the details.
@@ -173,6 +174,11 @@ struct bio {u64 bi_iocost_cost;#endif #endif + +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *bi_crypt_context; +#endif +union {#if defined(CONFIG_BLK_DEV_INTEGRITY )struct bio_integrity_payload *bi_integrity;/* data integrity */
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 2/9] block: Add encryption context to struct bio
Quoted 8 lines · Satya Tangirala
We must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the filesystem/fscrypt that knows about and manages encryption contexts. As such, when the filesystem layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.
Quoted 8 lines
We want to communicate the encryption context from the filesystem layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct
bio_crypt_ctxto struct bio, which can represent an encryption context (note that we can't use thebi_privatefield in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate thebio_crypt_ctxand make the bio/request merging logic aware of thebio_crypt_ctx.
You might want to reword this to clarify that this could potentially be used by any user of the block layer (e.g., device-mapper targets), not just filesystems and not just fscrypt. fscrypt is just the initial use case.
Quoted 54 lines
block/bio-crypt-ctx.c new file +48
-0@@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright 2019 Google LLC +*/ + +#include <linux/bio.h> +#include <linux/blkdev.h> +#include <linux/keyslot-manager.h> +#include <linux/module.h> +#include <linux/slab.h> + + +static int num_prealloc_crypt_ctxs =128 ; + +module_param (num_prealloc_crypt_ctxs, int,0444 ); +MODULE_PARM_DESC (num_prealloc_crypt_ctxs, +"Number of bio crypto contexts to preallocate" ); + +static struct kmem_cache *bio_crypt_ctx_cache; +static mempool_t *bio_crypt_ctx_pool; + +int __init bio_crypt_ctx_init (void ) +{ + bio_crypt_ctx_cache =KMEM_CACHE (bio_crypt_ctx,0 ); +if (!bio_crypt_ctx_cache) +return -ENOMEM ; + + bio_crypt_ctx_pool =mempool_create_slab_pool (num_prealloc_crypt_ctxs, + bio_crypt_ctx_cache); +if (!bio_crypt_ctx_pool) +return -ENOMEM ; + +/* This is assumed in various places. */ +BUILD_BUG_ON (BLK_ENCRYPTION_MODE_INVALID !=0 ); + +return 0 ; +} + +struct bio_crypt_ctx *bio_crypt_alloc_ctx (gfp_t gfp_mask) +{ +return mempool_alloc (bio_crypt_ctx_pool, gfp_mask); +} + +void bio_crypt_free_ctx (struct bio *bio) +{ +mempool_free (bio->bi_crypt_context, bio_crypt_ctx_pool); + bio->bi_crypt_context =NULL ; +}
block/bio-crypt-ctx.c line 58
Quoted 10 lines
block/bio-crypt-ctx.c new file +10
-0+ +void bio_crypt_clone (struct bio *dst,struct bio *src,gfp_t gfp_mask) +{ +const struct bio_crypt_ctx *src_bc = src->bi_crypt_context; + +/* +* If a bio is swhandled, then it will be decrypted when bio_endio +* is called. As we only want the data to be decrypted once, copies +* of the bio must not have have a crypt context. +*/
This comment belongs in the patch that adds the crypto API fallback, not in this patch.
Quoted 4 lines
block/bio-crypt-ctx.c new file +4
-0+/* +* Checks that two bio crypt contexts are compatible - i.e. that +* they are mergeable except for data_unit_num continuity. +*/
block/bio-crypt-ctx.c line 94
Quoted 10 lines
block/bio-crypt-ctx.c new file +10
-0+bool bio_crypt_ctx_compatible (struct bio *b_1,struct bio *b_2) +{ +struct bio_crypt_ctx *bc1 = b_1->bi_crypt_context; +struct bio_crypt_ctx *bc2 = b_2->bi_crypt_context; + +if (bc1 != bc2) +return false; + +return !bc1 || bc1->bc_key == bc2->bc_key; +}
As we've been discussing, this got broken because it now always returns false when the bio_crypt_ctx's are different. My suggestion is:
if (!bc1) return !bc2; return bc2 && bc1->bc_key == bc2->bc_key;
Quoted 9 lines
block/bio-crypt-ctx.c new file +1
-4@@ -615,15 +617,12 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)__bio_clone_fast (b, bio); -if (bio_integrity (bio)) { -int ret; - - ret =bio_integrity_clone (b, bio, gfp_mask); +bio_crypt_clone (b, bio, gfp_mask);
block/bio-crypt-ctx.c struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs) · line 626
Quoted 10 lines
block/bio-crypt-ctx.c new file +4
-4-if (ret <0 ) { -bio_put (b); -return NULL ; - } +if (bio_integrity (bio) && +bio_integrity_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ; }
Now that bio_crypt_clone() can't fail, this patch probably shouldn't bother to clean up the blk-integrity error handling.
Quoted 33 lines
block/blk-merge.c +7
-0@@ -596,6 +596,8 @@ int ll_back_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs)req_set_nomerge (req->q, req);return 0 ; } +if (!bio_crypt_ctx_mergeable (req->bio,blk_rq_bytes (req), bio)) +return 0 ;return ll_new_hw_segment (req, bio, nr_segs); }@@ -612,6 +614,8 @@ int ll_front_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segsreq_set_nomerge (req->q, req);return 0 ; } +if (!bio_crypt_ctx_mergeable (bio, bio->bi_iter.bi_size, req->bio)) +return 0 ;return ll_new_hw_segment (req, bio, nr_segs); }@@ -656,6 +660,9 @@ static int ll_merge_requests_fn(struct request_queue *q, struct request *req,if (blk_integrity_merge_rq (q, req, next) == false)return 0 ; +if (!bio_crypt_ctx_mergeable (req->bio,blk_rq_bytes (req), next->bio)) +return 0 ; +/* Merge is OK... */ req->nr_phys_segments = total_phys_segments;return 1 ;
block/blk-merge.c bool blk_rq_merge_ok(struct request *rq, struct bio *bio) · line 910
Quoted 9 lines
block/blk-merge.c +4
-0@@ -895,6 +902,10 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)if (rq->ioprio !=bio_prio (bio))return false; +/* Only merge if the crypt contexts are compatible */ +if (!bio_crypt_ctx_compatible (bio, rq->bio)) +return false; +return true; }
Thanks, this looks much better now. This fixes the bug in v5 where the DUN continuity check in blk_try_merge() was insufficient, as blk_try_merge() isn't always called, but ll_front_merge_fn() and ll_back_merge_fn() are.
As an optional cleanup, could you also move the blk-crypto checks in ll_back_merge_fn(), ll_front_merge_fn(), and blk_rq_merge_ok() to be immediately below the corresponding blk-integrity checks? blk-integrity and blk-crypto are both optional block layer features which have constraints for front and back merges, so they require merge checks in all the same places (except in one case where blk-integrity requires an extra check). That's partly how I found the bug (which for the record, Satya found independently too) -- I was comparing the blk-crypto checks to blk-integrity.
So having the blk-crypto and blk-integrity checks be immediately next to each other would be helpful from a readability standpoint.
Could you also flip the argument order in the call to bio_crypt_ctx_compatible() so that it matches the order for the other checks in blk_rq_merge_ok()?
None of this "actually matters", but IMO it's really important that we make these merge checks as straightforward and auditable as possible, as bugs in them cause data corruption, as we've seen.
Quoted 15 lines
block/bounce.c +1
-4@@ -267,14 +267,12 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,break ; } -if (bio_integrity (bio_src)) { -int ret; +bio_crypt_clone (bio, bio_src, gfp_mask); - ret =bio_integrity_clone (bio, bio_src, gfp_mask); -if (ret <0 ) {
block/bounce.c static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask, · line 278
Quoted 10 lines
block/bounce.c +4
-3-bio_put (bio); -return NULL ; - } +if (bio_integrity (bio_src) && +bio_integrity_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ; }bio_clone_blkg_association (bio, bio_src);
Like in bio_clone_fast(): now that bio_crypt_clone() can't fail, this patch probably shouldn't bother cleaning up the blk-integrity error handling.
drivers/md/dm.c static int clone_bio(struct dm_target_io *tio, struct bio *bio, · line 1309
Quoted 13 lines
drivers/md/dm.c +2
-1@@ -1304,9 +1304,10 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,__bio_clone_fast (clone, bio); +bio_crypt_clone (clone, bio,GFP_NOIO ); +if (bio_integrity (bio)) {int r; -
Unnecessary deleted line.
include/linux/bio-crypt-ctx.h line 11
Quoted 10 lines
include/linux/bio-crypt-ctx.h +1
-1@@ -8,7 +8,7 @@enum blk_crypto_mode_num {BLK_ENCRYPTION_MODE_INVALID ,BLK_ENCRYPTION_MODE_AES_256_XTS , -BLK_ENCRYPTION_MODE_AES_128_CBC , +BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV ,
This should be folded into the patch which introduced blk_crypto_mode_num.
Quoted 16 lines
include/linux/bio-crypt-ctx.h +16
-0+/** +* struct bio_crypt_ctx - an inline encryption context +* @bc_key: the key, algorithm, and data unit size to use +* @bc_keyslot: the keyslot that has been assigned for this key in @bc_ksm, +* or -1 if no keyslot has been assigned yet. +* @bc_dun: the data unit number (starting IV) to use +* @bc_ksm: the keyslot manager into which the key has been programmed with +* @bc_keyslot, or NULL if this key hasn't yet been programmed. +* +* A bio_crypt_ctx specifies that the contents of the bio will be encrypted (for +* write requests) or decrypted (for read requests) inline by the storage device +* or controller, or by the crypto API fallback. +*/ +struct bio_crypt_ctx { +const struct blk_crypto_key *bc_key; +int bc_keyslot;
include/linux/bio-crypt-ctx.h struct blk_crypto_key { · line 75
Quoted 10 lines
include/linux/bio-crypt-ctx.h +10
-0+ +/* Data unit number */ +u64 bc_dun[BLK_CRYPTO_DUN_ARRAY_SIZE ]; + +/* +* The keyslot manager where the key has been programmed +* with keyslot. +*/ +struct keyslot_manager *bc_ksm; +};
The two comments inside the struct definition are not needed now that there is a kerneldoc comment above the struct which documents all the fields.
FWIW, I also think it would be slightly more logical to order the fields like:
struct bio_crypt_ctx {
const struct blk_crypto_key *bc_key;
u64 bc_dun[BLK_CRYPTO_DUN_ARRAY_SIZE];
struct keyslot_manager *bc_ksm;
int bc_keyslot;
};... because (key, dun) go together as they're what the user provides, while (ksm, keyslot) go together as they're managed by the block layer.
Quoted 6 lines
include/linux/bio-crypt-ctx.h +6
-0+static inline bool bio_crypt_dun_is_contiguous (const struct bio_crypt_ctx *bc, +unsigned int bytes, +u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE ]) +{ +int i =0 ; +unsigned int inc = bytes >> bc->bc_key->data_unit_size_bits;
include/linux/bio-crypt-ctx.h struct blk_crypto_key { · line 128
Quoted 10 lines
include/linux/bio-crypt-ctx.h +10
-0+ +while (inc && i <BLK_CRYPTO_DUN_ARRAY_SIZE ) { +if (bc->bc_dun[i] + inc != next_dun[i]) +return false; + inc = ((bc->bc_dun[i] + inc) < inc); + i++; + } + +return true; +}
This incorrectly returns true in some cases. It needs to compare the whole DUN, not stop as soon as 'inc' becomes 0.
I'm also always a bit nervious of code that checks for integer wraparound without casting or assigning the result, due to the presence of integer promotion in C... That's partly why I had the 'u64 sum' variable in the version I suggested. But this specific case is fine because the type is u64.
- Eric
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
In reply to Satya TangiralaQuoted 495 lines · Satya Tangirala
We must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the filesystem/fscrypt that knows about and manages encryption contexts. As such, when the filesystem layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.
We want to communicate the encryption context from the filesystem layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct
bio_crypt_ctxto struct bio, which can represent an encryption context (note that we can't use thebi_privatefield in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate thebio_crypt_ctxand make the bio/request merging logic aware of thebio_crypt_ctx.Signed-off-by Satya Tangirala <[email protected]>--- block/Makefile | 2 +- block/bio-crypt-ctx.c | 131 ++++++++++++++++++++++++++++++ block/bio.c | 16 ++-- block/blk-core.c | 3 + block/blk-merge.c | 11 +++ block/bounce.c | 12 ++- drivers/md/dm.c | 3 +- include/linux/bio-crypt-ctx.h | 146 +++++++++++++++++++++++++++++++++- include/linux/blk_types.h | 6 ++ 9 files changed, 312 insertions(+), 18 deletions(-) create mode 100644 block/bio-crypt-ctx.cblock/Makefile +1
-1@@ -37,4 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.oblock/bio-crypt-ctx.c new file +131
-0@@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright 2019 Google LLC +*/ + +#include <linux/bio.h> +#include <linux/blkdev.h> +#include <linux/keyslot-manager.h> +#include <linux/module.h> +#include <linux/slab.h> + + +static int num_prealloc_crypt_ctxs =128 ; + +module_param (num_prealloc_crypt_ctxs, int,0444 ); +MODULE_PARM_DESC (num_prealloc_crypt_ctxs, +"Number of bio crypto contexts to preallocate" ); + +static struct kmem_cache *bio_crypt_ctx_cache; +static mempool_t *bio_crypt_ctx_pool; + +int __init bio_crypt_ctx_init (void ) +{ + bio_crypt_ctx_cache =KMEM_CACHE (bio_crypt_ctx,0 ); +if (!bio_crypt_ctx_cache) +return -ENOMEM ; + + bio_crypt_ctx_pool =mempool_create_slab_pool (num_prealloc_crypt_ctxs, + bio_crypt_ctx_cache); +if (!bio_crypt_ctx_pool) +return -ENOMEM ; + +/* This is assumed in various places. */ +BUILD_BUG_ON (BLK_ENCRYPTION_MODE_INVALID !=0 ); + +return 0 ; +} + +struct bio_crypt_ctx *bio_crypt_alloc_ctx (gfp_t gfp_mask) +{ +return mempool_alloc (bio_crypt_ctx_pool, gfp_mask); +} + +void bio_crypt_free_ctx (struct bio *bio) +{ +mempool_free (bio->bi_crypt_context, bio_crypt_ctx_pool); + bio->bi_crypt_context =NULL ; +} + +void bio_crypt_clone (struct bio *dst,struct bio *src,gfp_t gfp_mask) +{ +const struct bio_crypt_ctx *src_bc = src->bi_crypt_context; + +/* +* If a bio is swhandled, then it will be decrypted when bio_endio +* is called. As we only want the data to be decrypted once, copies +* of the bio must not have have a crypt context. +*/ +if (!src_bc) +return ; + + dst->bi_crypt_context =bio_crypt_alloc_ctx (gfp_mask); + *dst->bi_crypt_context = *src_bc; + +if (src_bc->bc_keyslot >=0 ) +keyslot_manager_get_slot (src_bc->bc_ksm, src_bc->bc_keyslot); +} +EXPORT_SYMBOL_GPL (bio_crypt_clone); + +bool bio_crypt_should_process (struct request *rq) +{ +struct bio *bio = rq->bio; + +if (!bio || !bio->bi_crypt_context) +return false; + +return rq->q->ksm == bio->bi_crypt_context->bc_ksm; +} +EXPORT_SYMBOL_GPL (bio_crypt_should_process); + +/* +* Checks that two bio crypt contexts are compatible - i.e. that +* they are mergeable except for data_unit_num continuity. +*/ +bool bio_crypt_ctx_compatible (struct bio *b_1,struct bio *b_2) +{ +struct bio_crypt_ctx *bc1 = b_1->bi_crypt_context; +struct bio_crypt_ctx *bc2 = b_2->bi_crypt_context; + +if (bc1 != bc2) +return false; + +return !bc1 || bc1->bc_key == bc2->bc_key; +} + +/* +* Checks that two bio crypt contexts are compatible, and also +* that their data_unit_nums are continuous (and can hence be merged) +* in the order b_1 followed by b_2. +*/ +bool bio_crypt_ctx_mergeable (struct bio *b_1,unsigned int b1_bytes, +struct bio *b_2) +{ +struct bio_crypt_ctx *bc1 = b_1->bi_crypt_context; +struct bio_crypt_ctx *bc2 = b_2->bi_crypt_context; + +if (!bio_crypt_ctx_compatible (b_1, b_2)) +return false; + +return !bc1 ||bio_crypt_dun_is_contiguous (bc1, b1_bytes, bc2->bc_dun); +} + +void bio_crypt_ctx_release_keyslot (struct bio_crypt_ctx *bc) +{ +keyslot_manager_put_slot (bc->bc_ksm, bc->bc_keyslot); + bc->bc_ksm =NULL ; + bc->bc_keyslot =-1 ; +} + +int bio_crypt_ctx_acquire_keyslot (struct bio_crypt_ctx *bc, +struct keyslot_manager *ksm) +{ +int slot =keyslot_manager_get_slot_for_key (ksm, bc->bc_key); + +if (slot <0 ) +return slot; + + bc->bc_keyslot = slot; + bc->bc_ksm = ksm; +return 0 ; +}block/bio.c +8
-8@@ -236,6 +236,8 @@ void bio_uninit(struct bio *bio)if (bio_integrity (bio))bio_integrity_free (bio); + +bio_crypt_free_ctx (bio); }EXPORT_SYMBOL (bio_uninit);@@ -615,15 +617,12 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)__bio_clone_fast (b, bio); -if (bio_integrity (bio)) { -int ret; - - ret =bio_integrity_clone (b, bio, gfp_mask); +bio_crypt_clone (b, bio, gfp_mask); -if (ret <0 ) { -bio_put (b); -return NULL ; - } +if (bio_integrity (bio) && +bio_integrity_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ; }return b;@@ -997,6 +996,7 @@ void bio_advance(struct bio *bio, unsigned bytes)if (bio_integrity (bio))bio_integrity_advance (bio, bytes); +bio_crypt_advance (bio, bytes);bio_advance_iter (bio, &bio->bi_iter, bytes); }EXPORT_SYMBOL (bio_advance);block/blk-core.c +3
-0@@ -1810,5 +1810,8 @@ int __init blk_dev_init(void) blk_debugfs_root =debugfs_create_dir ("block" ,NULL );#endif +if (bio_crypt_ctx_init () <0 ) +panic ("Failed to allocate mem for bio crypt ctxs\n" ); +return 0 ; }block/blk-merge.c +11
-0@@ -596,6 +596,8 @@ int ll_back_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs)req_set_nomerge (req->q, req);return 0 ; } +if (!bio_crypt_ctx_mergeable (req->bio,blk_rq_bytes (req), bio)) +return 0 ;return ll_new_hw_segment (req, bio, nr_segs); }@@ -612,6 +614,8 @@ int ll_front_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segsreq_set_nomerge (req->q, req);return 0 ; } +if (!bio_crypt_ctx_mergeable (bio, bio->bi_iter.bi_size, req->bio)) +return 0 ;return ll_new_hw_segment (req, bio, nr_segs); }@@ -656,6 +660,9 @@ static int ll_merge_requests_fn(struct request_queue *q, struct request *req,if (blk_integrity_merge_rq (q, req, next) == false)return 0 ; +if (!bio_crypt_ctx_mergeable (req->bio,blk_rq_bytes (req), next->bio)) +return 0 ; +/* Merge is OK... */ req->nr_phys_segments = total_phys_segments;return 1 ;@@ -895,6 +902,10 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)if (rq->ioprio !=bio_prio (bio))return false; +/* Only merge if the crypt contexts are compatible */ +if (!bio_crypt_ctx_compatible (bio, rq->bio)) +return false; +return true; }block/bounce.c +5
-7@@ -267,14 +267,12 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,break ; } -if (bio_integrity (bio_src)) { -int ret; +bio_crypt_clone (bio, bio_src, gfp_mask); - ret =bio_integrity_clone (bio, bio_src, gfp_mask); -if (ret <0 ) { -bio_put (bio); -return NULL ; - } +if (bio_integrity (bio_src) && +bio_integrity_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ; }bio_clone_blkg_association (bio, bio_src);drivers/md/dm.c +2
-1@@ -1304,9 +1304,10 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,__bio_clone_fast (clone, bio); +bio_crypt_clone (clone, bio,GFP_NOIO ); +if (bio_integrity (bio)) {int r; -if (unlikely (!dm_target_has_integrity (tio->ti->type) && !dm_target_passes_integrity (tio->ti->type))) { DMWARN("%s: the target %s doesn't support integrity data.",include/linux/bio-crypt-ctx.h +145
-1@@ -8,7 +8,7 @@enum blk_crypto_mode_num {BLK_ENCRYPTION_MODE_INVALID ,BLK_ENCRYPTION_MODE_AES_256_XTS , -BLK_ENCRYPTION_MODE_AES_128_CBC , +BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV ,BLK_ENCRYPTION_MODE_ADIANTUM ,BLK_ENCRYPTION_MODE_MAX , };@@ -44,6 +44,150 @@ struct blk_crypto_key {u8 raw[BLK_CRYPTO_MAX_KEY_SIZE ]; }; +#define BLK_CRYPTO_MAX_IV_SIZE 32 +#define BLK_CRYPTO_DUN_ARRAY_SIZE (BLK_CRYPTO_MAX_IV_SIZE/sizeof(u64)) + +/** +* struct bio_crypt_ctx - an inline encryption context +* @bc_key: the key, algorithm, and data unit size to use +* @bc_keyslot: the keyslot that has been assigned for this key in @bc_ksm, +* or -1 if no keyslot has been assigned yet. +* @bc_dun: the data unit number (starting IV) to use +* @bc_ksm: the keyslot manager into which the key has been programmed with +* @bc_keyslot, or NULL if this key hasn't yet been programmed. +* +* A bio_crypt_ctx specifies that the contents of the bio will be encrypted (for +* write requests) or decrypted (for read requests) inline by the storage device +* or controller, or by the crypto API fallback. +*/ +struct bio_crypt_ctx { +const struct blk_crypto_key *bc_key; +int bc_keyslot; + +/* Data unit number */ +u64 bc_dun[BLK_CRYPTO_DUN_ARRAY_SIZE ]; + +/* +* The keyslot manager where the key has been programmed +* with keyslot. +*/ +struct keyslot_manager *bc_ksm; +}; + +int bio_crypt_ctx_init (void ); + +struct bio_crypt_ctx *bio_crypt_alloc_ctx (gfp_t gfp_mask); + +void bio_crypt_free_ctx (struct bio *bio); + +static inline bool bio_has_crypt_ctx (struct bio *bio) +{ +return bio->bi_crypt_context; +} + +void bio_crypt_clone (struct bio *dst,struct bio *src,gfp_t gfp_mask); + +static inline void bio_crypt_set_ctx (struct bio *bio, +const struct blk_crypto_key *key, +u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE ], +gfp_t gfp_mask) +{ +struct bio_crypt_ctx *bc =bio_crypt_alloc_ctx (gfp_mask); + + bc->bc_key = key; +memcpy (bc->bc_dun, dun,sizeof (bc->bc_dun)); + bc->bc_ksm =NULL ; + bc->bc_keyslot =-1 ; + + bio->bi_crypt_context = bc; +} + +void bio_crypt_ctx_release_keyslot (struct bio_crypt_ctx *bc); + +int bio_crypt_ctx_acquire_keyslot (struct bio_crypt_ctx *bc, +struct keyslot_manager *ksm); + +struct request ; +bool bio_crypt_should_process (struct request *rq); + +static inline bool bio_crypt_dun_is_contiguous (const struct bio_crypt_ctx *bc, +unsigned int bytes, +u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE ]) +{ +int i =0 ; +unsigned int inc = bytes >> bc->bc_key->data_unit_size_bits; + +while (inc && i <BLK_CRYPTO_DUN_ARRAY_SIZE ) { +if (bc->bc_dun[i] + inc != next_dun[i]) +return false; + inc = ((bc->bc_dun[i] + inc) < inc); + i++; + } + +return true; +} + + +static inline void bio_crypt_dun_increment (u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE ], +unsigned int inc) +{ +int i =0 ; + +while (inc && i <BLK_CRYPTO_DUN_ARRAY_SIZE ) { + dun[i] += inc; + inc = (dun[i] < inc); + i++; + } +} + +static inline void bio_crypt_advance (struct bio *bio,unsigned int bytes) +{ +struct bio_crypt_ctx *bc = bio->bi_crypt_context; + +if (!bc) +return ; + +bio_crypt_dun_increment (bc->bc_dun, + bytes >> bc->bc_key->data_unit_size_bits); +} + +bool bio_crypt_ctx_compatible (struct bio *b_1,struct bio *b_2); + +bool bio_crypt_ctx_mergeable (struct bio *b_1,unsigned int b1_bytes, +struct bio *b_2); + +#else /* CONFIG_BLK_INLINE_ENCRYPTION */ +static inline int bio_crypt_ctx_init (void ) +{ +return 0 ; +} + +static inline bool bio_has_crypt_ctx (struct bio *bio) +{ +return false; +} + +static inline void bio_crypt_clone (struct bio *dst,struct bio *src, +gfp_t gfp_mask) { } + +static inline void bio_crypt_free_ctx (struct bio *bio) { } + +static inline void bio_crypt_advance (struct bio *bio,unsigned int bytes) { } + +static inline bool bio_crypt_ctx_compatible (struct bio *b_1,struct bio *b_2) +{ +return true; +} + +static inline bool bio_crypt_ctx_mergeable (struct bio *b_1, +unsigned int b1_bytes, +struct bio *b_2) +{ +return true; +} +#endif /* CONFIG_BLK_INLINE_ENCRYPTION */ +#endif /* CONFIG_BLOCK */ +#endif /* __LINUX_BIO_CRYPT_CTX_H */ include/linux/blk_types.h +1
-0@@ -18,6 +18,7 @@ struct block_device;struct io_context ;struct cgroup_subsys_state ;typedef void (bio_end_io_t ) (struct bio *); +struct bio_crypt_ctx ;/* * Block error status values. See block/blk-core:blk_errors for the details.
include/linux/blk_types.h struct bio { · line 180
Quoted 7 lines
include/linux/blk_types.h +4
-0@@ -173,6 +174,11 @@ struct bio {u64 bi_iocost_cost;#endif #endif + +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *bi_crypt_context; +#endif
This grows struct bio even if we aren't actively using bi_crypt_context, and I thought Jens told us to stop making it bigger. :)
--D
Quoted 7 lines
include/linux/blk_types.h +1
-0+union {#if defined(CONFIG_BLK_DEV_INTEGRITY )struct bio_integrity_payload *bi_integrity;/* data integrity */ Signature · 1 line
2.24.1.735.g03f4e72817-goog
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
Darrick,
Quoted 6 lines
include/linux/blk_types.h struct bio { · line 180
Quoted 3 lines
include/linux/blk_types.h +3
-0+#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *bi_crypt_context; +#endif This grows struct bio even if we aren't actively using
bi_crypt_context, and I thought Jens told us to stop making it bigger. :)
Yeah. Why not use the bio integrity plumbing? It was explicitly designed to attach things to a bio and have them consumed by the device driver.
Signature · 7 lines
Martin K. Petersen Oracle Linux Engineering _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Eric Biggers
maintainerRe: [PATCH v6 2/9] block: Add encryption context to struct bio
Quoted 10 lines · Martin K. Petersen
Darrick,
Quoted 6 lines
include/linux/blk_types.h struct bio { · line 180
Quoted 3 lines
include/linux/blk_types.h +3
-0+#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *bi_crypt_context; +#endif This grows struct bio even if we aren't actively using
bi_crypt_context, and I thought Jens told us to stop making it bigger. :)
Quoted 3 lines
Yeah. Why not use the bio integrity plumbing? It was explicitly designed to attach things to a bio and have them consumed by the device driver.
There's not really any such thing as "use the bio integrity plumbing". blk-integrity just does blk-integrity; it's not a plumbing layer that allows other features to be supported. Well, in theory we could refactor and rename all the hooks to "blk-extra" and make them delegate to either blk-integrity or blk-crypto, but I think that would be overkill.
What we could do, though, is say that at most one of blk-crypto and blk-integrity can be used at once on a given bio, and put the bi_integrity and bi_crypt_context pointers in union. (That would require allocating a REQ_INLINECRYPT bit so that we can tell what the pointer points to.)
- Eric
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
Eric,
Quoted 6 lines
There's not really any such thing as "use the bio integrity plumbing". blk-integrity just does blk-integrity; it's not a plumbing layer that allows other features to be supported. Well, in theory we could refactor and rename all the hooks to "blk-extra" and make them delegate to either blk-integrity or blk-crypto, but I think that would be overkill.
I certainly don't expect your crypto stuff to plug in without any modification to what we currently have. I'm just observing that the existing plumbing is designed to have pluggable functions that let filesystems attach additional information to bios on writes and process additional attached information on reads. And the block layer already handles slicing and dicing these attachments as the I/O traverses the stack.
There's also other stuff that probably won't be directly applicable or interesting for your use case. It just seems like identifying actual commonalities and differences would be worthwhile.
Note that substantial changes to the integrity code would inevitably lead to a lot of pain and suffering for me. So from that perspective I am very happy if you leave it alone. From an architectural viewpoint, however, it seems that there are more similarities than differences between crypto and integrity. And we should avoid duplication where possible. That's all.
Quoted 5 lines
What we could do, though, is say that at most one of blk-crypto and blk-integrity can be used at once on a given bio, and put the
bi_integrityandbi_crypt_contextpointers in union. (That would require allocating aREQ_INLINECRYPTbit so that we can tell what the pointer points to.)
Absolutely. That's why it's a union. Putting your stuff there is a prerequisite as far as I'm concerned. No need to grow the bio when the two features are unlikely to coexist. We can revisit that later should the need arise.
Signature · 7 lines
Martin K. Petersen Oracle Linux Engineering _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Eric Biggers
maintainerRe: [PATCH v6 2/9] block: Add encryption context to struct bio
Quoted 22 lines · Martin K. Petersen
Eric,
Quoted 6 lines
There's not really any such thing as "use the bio integrity plumbing". blk-integrity just does blk-integrity; it's not a plumbing layer that allows other features to be supported. Well, in theory we could refactor and rename all the hooks to "blk-extra" and make them delegate to either blk-integrity or blk-crypto, but I think that would be overkill.
I certainly don't expect your crypto stuff to plug in without any modification to what we currently have. I'm just observing that the existing plumbing is designed to have pluggable functions that let filesystems attach additional information to bios on writes and process additional attached information on reads. And the block layer already handles slicing and dicing these attachments as the I/O traverses the stack.
There's also other stuff that probably won't be directly applicable or interesting for your use case. It just seems like identifying actual commonalities and differences would be worthwhile.
Quoted 6 lines
Note that substantial changes to the integrity code would inevitably lead to a lot of pain and suffering for me. So from that perspective I am very happy if you leave it alone. From an architectural viewpoint, however, it seems that there are more similarities than differences between crypto and integrity. And we should avoid duplication where possible. That's all.
There are some similarities, like both being optional features that need extra per-bio information and hooks for bio merging, freeing, cloning, and advancing.
However, the nature of the per-bio information is very different. Most of the complexity in blk-integrity is around managing of a separate integrity scatterlist for each bio, alongside the regular data scatterlist.
That's not something we need or want for inline encryption. For each bio we just need a key, algorithm, data unit number, and data unit size. Since the data unit number (IV) is automatically incremented for each sector and the encryption is length-preserving, there's no per-sector data.
(Granted, from a crypto perspective ideally one would use authenticated encryption, which does require per-sector data. However, no one seems interested in building hardware that supports it. So for the forseeable future, only length-preserving encryption is in scope for this.)
Also, blk-crypto actually transforms the data whereas blk-integrity does not.
Quoted 10 lines
Quoted 5 lines
What we could do, though, is say that at most one of blk-crypto and blk-integrity can be used at once on a given bio, and put the
bi_integrityandbi_crypt_contextpointers in union. (That would require allocating aREQ_INLINECRYPTbit so that we can tell what the pointer points to.)Absolutely. That's why it's a union. Putting your stuff there is a prerequisite as far as I'm concerned. No need to grow the bio when the two features are unlikely to coexist. We can revisit that later should the need arise.
There are some ways the two features could be supported simultaneously without using more space, like making the pointer point to a linked list of tagged structs, or making the struct contain both a bio_crypt_ctx and bio_integrity_payload (or whichever combination is enabled in kconfig).
But it would be painful and I don't think people need this for now. So if people really aren't willing to accept the extra 8 bytes per bio even behind a kconfig option, my vote is we that we put bi_crypt_context in the union with bi_integrity, and add a flag REQ_INLINECRYPT (like REQ_INTEGRITY) that indicates that the bi_crypt_context member of the union is valid.
We'd also need some error-handling to prevent the two features from actually being used together. It looks like there are several cases to consider. One of them is what happens if bio_crypt_set_ctx() is called when blk-integrity verification or generation is enabled for the disk. I suppose it could either return an error, or we could make blk-crypto use the crypto API fallback provided that it was modified to make the decryption stop relying on ->bi_crypt_context, which could be done by cloning the bio and using ->bi_private instead.
- Eric
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
Eric,
Quoted 10 lines
However, the nature of the per-bio information is very different. Most of the complexity in blk-integrity is around managing of a separate integrity scatterlist for each bio, alongside the regular data scatterlist.
That's not something we need or want for inline encryption. For each bio we just need a key, algorithm, data unit number, and data unit size. Since the data unit number (IV) is automatically incremented for each sector and the encryption is length-preserving, there's no per-sector data.
Fair enough. I just wanted to make sure that you guys had actually looked at the integrity stuff and determined it wasn't a good fit.
Quoted 5 lines
There are some ways the two features could be supported simultaneously without using more space, like making the pointer point to a linked list of tagged structs, or making the struct contain both a
bio_crypt_ctxandbio_integrity_payload(or whichever combination is enabled in kconfig).
We have previously discussed having a facility in which you could chain several different things (with different prep/endio functions) off a bio. Similar to how we allow arbitrary stacking of block_devices. That was actually my main interest in terms of opening the integrity can of worms in this thread. Trying to find out which pieces of the plumbing, if any, could potentially be made generic and feature-independent.
The copy offload efforts, which are now again picking up momentum, also need to hang things off of the bio. That's the original reason the integrity field became a union, fwiw.
Quoted 5 lines
So if people really aren't willing to accept the extra 8 bytes per bio even behind a kconfig option, my vote is we that we put
bi_crypt_contextin the union withbi_integrity, and add a flagREQ_INLINECRYPT(likeREQ_INTEGRITY) that indicates that thebi_crypt_contextmember of the union is valid.
Agreed.
Quoted 5 lines
We'd also need some error-handling to prevent the two features from actually being used together. It looks like there are several cases to consider. One of them is what happens if
bio_crypt_set_ctx()is called when blk-integrity verification or generation is enabled for the disk.
The integrity profile is only attached if the device driver identifies a discovered device as capable. At least in the short term no device should indicate simultaneous support for DIX and your crypto interface.
Not saying that sanity checks shouldn't exist. But I think both of these features fall into things that are registered at device discovery time so we shouldn't need to clutter the I/O hot path with mutual exclusivity checks.
Signature · 7 lines
Martin K. Petersen Oracle Linux Engineering _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
In reply to Martin K. PetersenQuoted 4 lines · Martin K. Petersen
Absolutely. That's why it's a union. Putting your stuff there is a prerequisite as far as I'm concerned. No need to grow the bio when the two features are unlikely to coexist. We can revisit that later should the need arise.
With NVMe key per I/O support some form of inline encryption and PI are very likely to be used together in the not too far future.
Eric Biggers
maintainerRe: [PATCH v6 2/9] block: Add encryption context to struct bio
Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Martin K. Petersen
Absolutely. That's why it's a union. Putting your stuff there is a prerequisite as far as I'm concerned. No need to grow the bio when the two features are unlikely to coexist. We can revisit that later should the need arise.
With NVMe key per I/O support some form of inline encryption and PI are very likely to be used together in the not too far future.
The NVMe "key per I/O" draft is heavily flawed, and I don't think it will be useful at all in the Linux kernel context. The problem is that, as far as I can tell, it doesn't allow the encryption algorithm and IVs to be selected, or even standardized or made discoverable in any way. It does say that AES-256 must be supported, but it doesn't say which mode of operation (i.e. it could be something inappropriate for disk encryption, like ECB), nor does it say whether AES-256 has to be the default or not, and if it's not the default how to discover that and select AES-256. IV generation is also unspecified, so it could be something insecure like always using the same IV.
So effectively the NVMe encryption will be unspecified, untestable, and unverifiable. That means that vendors are likely to implement it insecurely, similar to how they're implementing self-encrypting drives insecurely [1]. (Granted, there are some reasons to think that vendors are less likely to screw up key per I/O. But inevitably some will still get it wrong.)
[1] https://www.ieee-security.org/TC/SP2019/papers/310.pdf
Also, since "key per I/O" won't allow selecting IVs, all the encrypted data will be tied to its physical location on-disk. That will make "key per I/O" unusable in any case where encrypted blocks are moved without the key, e.g. filesystem-level encryption on many filesystems.
And since the way that dm-crypt and fscrypt work is that you select which algorithm and IV generator you want to use, to even use NVMe "key per I/O" with them we'd have to add magic settings that say to use some unspecified hardware-specific encryption format, which could be completely insecure. As one of the fscrypt maintainers I'd be really hesistant to accept any such patch, and I think the dm-crypt people would feel the same way.
I've already raised these concerns in the NVMe and TCG Storage working groups, and the people working on it refused to make any changes, as they consider "key per I/O" to be more akin to the TCG Opal self-encrypting drive specification, and not actually intended to be "inline encryption".
So let's not over-engineer this kernel patchset to support some broken vaporware, please.
- Eric
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
Hi Eric,
Quoted 8 lines · Eric Biggers
The NVMe "key per I/O" draft is heavily flawed, and I don't think it will be useful at all in the Linux kernel context. The problem is that, as far as I can tell, it doesn't allow the encryption algorithm and IVs to be selected, or even standardized or made discoverable in any way. It does say that AES-256 must be supported, but it doesn't say which mode of operation (i.e. it could be something inappropriate for disk encryption, like ECB), nor does it say whether AES-256 has to be the default or not, and if it's not the default how to discover that and select AES-256.
I've talked to people involved with the TCG side of this spec, where all the interesting crypto happens. Currently the plan is to support KMIP wrapper keys, which specify the exact algorithm and operation mode, and algorithms and modes for the initial version are planned to be AES 256/512 XTS. I also had a chat with an involved person and they understand the principle that for the inline crypto to be trusted it needs to be interoperable with (trusted) software algorithms. So I don't think it is all doom.
Quoted 2 lines
IV generation is also unspecified, so it
could be something insecure like always using the same IV.
From talking to one of the initiators of the spec, no it is not intended to be unspecified, but indeed tied to the LBA (see below).
Quoted 4 lines
Also, since "key per I/O" won't allow selecting IVs, all the encrypted data will be tied to its physical location on-disk. That will make "key per I/O" unusable in any case where encrypted blocks are moved without the key, e.g. filesystem-level encryption on many filesystems.
File systems don't move data around all that often (saying that with my fs developer hat on). In traditional file systems only defragmentation will move data around, with extent refcounting it can also happen for dedup, and for file systems that write out of place data gets moved when parts of a block are rewritten, but in that case a read modify write cycle is perfomed in the Linux code anyway, so it will go through the inline encryption engined on the way and the way out.
So in other words - specifying an IV would be useful for some use cases, but I don't think it is a deal blocker. Even without that is is useful for block device level encryption, and could have some usefulness for file system encryption usage.
I think that adding an IV would eventually be useful, but fitting that into NVMe won't be easy, as you'd need to find a way to specify the IV for each submission queue entry, which requires growing it, or finding some way to extend it out of band.
Quoted 4 lines
I've already raised these concerns in the NVMe and TCG Storage working groups, and the people working on it refused to make any changes, as they consider "key per I/O" to be more akin to the TCG Opal self-encrypting drive specification, and not actually intended to be "inline encryption".
While I have my fair share of issues how the spec is developed that isn't my impression, and at least for the verifyable part I heard contrary statements. Feel free to contact me offline to make sure we can move this into the right direction.
Quoted 2 lines
So let's not over-engineer this kernel patchset to support some broken vaporware, please.
Not sharing bio fields for integrity and encryption actually keeps the patchset simpler (although uses more memory if both options are enabled). So my main point here is to not over engineer it for broken premise that won't be true soon.
Eric Biggers
maintainerRe: [PATCH v6 2/9] block: Add encryption context to struct bio
Quoted 14 lines · Christoph Hellwig
File systems don't move data around all that often (saying that with my fs developer hat on). In traditional file systems only defragmentation will move data around, with extent refcounting it can also happen for dedup, and for file systems that write out of place data gets moved when parts of a block are rewritten, but in that case a read modify write cycle is perfomed in the Linux code anyway, so it will go through the inline encryption engined on the way and the way out.
So in other words - specifying an IV would be useful for some use cases, but I don't think it is a deal blocker. Even without that is is useful for block device level encryption, and could have some usefulness for file system encryption usage.
Quoted 4 lines
I think that adding an IV would eventually be useful, but fitting that into NVMe won't be easy, as you'd need to find a way to specify the IV for each submission queue entry, which requires growing it, or finding some way to extend it out of band.
Sure, people have even done inline crypto on ext4 before (downstream), using the LBA for the IV. But log-structured filesystems like f2fs move data blocks around *without the encryption key*; and at least for fscrypt, f2fs support is essential. In any case it's also awkward having the physical on-disk location determine the ciphertext on-disk, as then the result isn't fully controlled by the encryption settings you set, but also based on where your filesystem is located on-disk (with extra fun occurring if there's any sort of remapping layer in between). But sure, it's not *useless* to not be able to specify the IV, it's just annoying and less useful.
[I was also a bit surprised to see that NVMe won't actually allow specify the IV, as I thought you had objected to the naming of the INLINE_CRYPT_OPTIMIZED fscrypt policy flag partly on the grounds that NVMe would support IVs longer than the 64 bits that UFS is limited to. Perhaps I misunderstood though.]
Quoted 7 lines
Quoted 2 lines
So let's not over-engineer this kernel patchset to support some broken vaporware, please.
Not sharing bio fields for integrity and encryption actually keeps the patchset simpler (although uses more memory if both options are enabled). So my main point here is to not over engineer it for broken premise that won't be true soon.
Well there are 3 options:
(a) Separate fields for bi_crypt_context and bi_integrity
(b) bi_crypt_context and bi_integrity in union
(c) One pointer that can support both features, e.g. linked list of tagged structs.
It sounds like you're advocating for (a), but I had misunderstood and thought you're advocating for (c). We'd of course be fine with (a) as it's the simplest, but other people are saying they prefer (b).
Satya, to resolve this I think you should check how hard (b) is to implement -- i.e. is it easy, or is it really tricky to ensure the features are never used together? (Considering that it's probably not just a matter of whether any hardware supports both features, as dm-integrity supports blk-integrity in software and blk-crypto-fallback supports blk-crypto in software.)
- Eric
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
Quoted 54 lines · Eric Biggers
Quoted 14 lines · Christoph Hellwig
File systems don't move data around all that often (saying that with my fs developer hat on). In traditional file systems only defragmentation will move data around, with extent refcounting it can also happen for dedup, and for file systems that write out of place data gets moved when parts of a block are rewritten, but in that case a read modify write cycle is perfomed in the Linux code anyway, so it will go through the inline encryption engined on the way and the way out.
So in other words - specifying an IV would be useful for some use cases, but I don't think it is a deal blocker. Even without that is is useful for block device level encryption, and could have some usefulness for file system encryption usage.
Quoted 4 lines
I think that adding an IV would eventually be useful, but fitting that into NVMe won't be easy, as you'd need to find a way to specify the IV for each submission queue entry, which requires growing it, or finding some way to extend it out of band.
Sure, people have even done inline crypto on ext4 before (downstream), using the LBA for the IV. But log-structured filesystems like f2fs move data blocks around *without the encryption key*; and at least for fscrypt, f2fs support is essential. In any case it's also awkward having the physical on-disk location determine the ciphertext on-disk, as then the result isn't fully controlled by the encryption settings you set, but also based on where your filesystem is located on-disk (with extra fun occurring if there's any sort of remapping layer in between). But sure, it's not *useless* to not be able to specify the IV, it's just annoying and less useful.
[I was also a bit surprised to see that NVMe won't actually allow specify the IV, as I thought you had objected to the naming of the
INLINE_CRYPT_OPTIMIZEDfscrypt policy flag partly on the grounds that NVMe would support IVs longer than the 64 bits that UFS is limited to. Perhaps I misunderstood though.]Quoted 7 lines
Quoted 2 lines
So let's not over-engineer this kernel patchset to support some broken vaporware, please.
Not sharing bio fields for integrity and encryption actually keeps the patchset simpler (although uses more memory if both options are enabled). So my main point here is to not over engineer it for broken premise that won't be true soon.
Well there are 3 options:
(a) Separate fields for
bi_crypt_contextandbi_integrity
(b)bi_crypt_contextandbi_integrityin union
(c) One pointer that can support both features, e.g. linked list of tagged structs.It sounds like you're advocating for (a), but I had misunderstood and thought you're advocating for (c). We'd of course be fine with (a) as it's the simplest, but other people are saying they prefer (b).
Quoted 7 lines
Satya, to resolve this I think you should check how hard (b) is to implement -- i.e. is it easy, or is it really tricky to ensure the features are never used together? (Considering that it's probably not just a matter of whether any hardware supports both features, as dm-integrity supports blk-integrity in software and blk-crypto-fallback supports blk-crypto in software.)
- Eric
What I have right now for v7 of the patch series was my attempt at (b) (since some were saying they prefer it) - it may still be incomplete though because I missed something, but here's what I think it involved:
1) bi_crypt_context is never set after bi_integrity, so I don't check if bi_integrity is set or not when setting bi_crypt_context - this keeps the code cleaner when setting the bi_crypt_context.
2) I made it error whenever we try to set bi_integrity on a bio with REQ_BLK_CRYPTO.
3) There is an issue with the way the blk-crypto-fallback interacts with bio integrity. One of the goals of the fallback is to make it inline encryption transparent to the upper layers (i.e. as far as the upper layers are concerned there should be no difference whether there is actual hardware inline encryption support or whether the fallback is used instead).
The issue is, when the fallback encrypts a bio, it clones the bio and encrypts the pages in the clone, but the clone won't have a bi_crypt_context (since otherwise, when the clone is resubmitted, blk-crypto would incorrectly assume that the clone needs to be encrypted once again). When bi_integrity is set on the clone, there won't be any error detected since REQ_BLK_CRYPTO won't be set on the clone. However, if hardware inline encryption support had been present, and the blk-crypto had not used the fallback, the same bio would have bi_crypt_context set when we try to set bi_integrity, which would then fail.
To fix this issue, I introduced another flag REQ_NO_SPECIAL (subject to being renamed when I think of/someone suggests a better name), which when set on the bio, indicates that bi_integrity (and in future whatever other fields share the same union, shouldn't be set on that bio. I can probably do away with the new flag and just set REQ_BLK_CRYPTO and also set bi_crypt_context = NULL to signify the same thing, but either way, it doesn't seem like a very clean solution to me.
4) I don't think there's actually an issue with dm-integrity as long as when we add support for inline encryption to dm, we ensure that if any of the child targets is dm-integrity, then the dm device says it doesn't support any crypto mode. This way, encryption is always consistently done before integrity calculation, which is always consistently done before decryption (and when dm-integrity is not using the internal hash, then the bio will fail because of (2), but this is still consistent whether or not hardware inline encryption support is present.
However, even if we decide to do (a), I'll still need to ensure that when bio integrity and blk-crypto are used together, the results are consistent regardless of whether hardware inline crypto support is present. So I either disallow bio integrity to be used with blk-crypto, in which case I'll need to do something similar to what I had to do in (3) like introduce REQ_NO_SPECIAL and we also lose the advantage of not having the two fields shared in a union, or I allow them to be used together while ensuring consistent results by maybe doing something like forcing blk-crypto to fallback to the crypto API whenever the target device for a bio has a bio integrity profile (and whatever checks are done in bio_integrity_prep).
So I'm not sure if (a) is a lot easier or cleaner. But unlike (b), (a) does give us the option of using the two features together....if it seems likely that we'll want to use both these features together at some point, then maybe I should switch back to (a), and for now just disallow the two features from being used together since that's still easier than trying to get them both to work together, and once we know for sure that we want to use both together, we can make the necessary additions then?
Re: [PATCH v6 2/9] block: Add encryption context to struct bio
In reply to Christoph HellwigChristoph,
Quoted 7 lines
Quoted 4 lines
Absolutely. That's why it's a union. Putting your stuff there is a prerequisite as far as I'm concerned. No need to grow the bio when the two features are unlikely to coexist. We can revisit that later should the need arise.
With NVMe key per I/O support some form of inline encryption and PI are very likely to be used together in the not too far future.
I don't disagree that we'll have to manage coexistence eventually. Hence my comments about being able to chain multiple things to a bio.
In the immediate term, though, I think it makes sense to leverage the integrity pointer to avoid growing struct bio.
Signature · 7 lines
Martin K. Petersen Oracle Linux Engineering _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Eric Biggers
maintainerRe: [PATCH v6 2/9] block: Add encryption context to struct bio
In reply to Satya TangiralaQuoted 4 lines · Satya Tangirala
include/linux/bio-crypt-ctx.h +4
-0+static inline void bio_crypt_set_ctx (struct bio *bio, +const struct blk_crypto_key *key, +u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE ], +gfp_t gfp_mask)
include/linux/bio-crypt-ctx.h struct blk_crypto_key { · line 103
Quoted 10 lines
include/linux/bio-crypt-ctx.h +10
-0+{ +struct bio_crypt_ctx *bc =bio_crypt_alloc_ctx (gfp_mask); + + bc->bc_key = key; +memcpy (bc->bc_dun, dun,sizeof (bc->bc_dun)); + bc->bc_ksm =NULL ; + bc->bc_keyslot =-1 ; + + bio->bi_crypt_context = bc; +}
The 'dun' argument should be const.
- Eric
PATCH v6 3/9 block: blk-crypto for Inline Encryption
In reply to Satya TangiralaWe introduce blk-crypto, which manages programming keyslots for struct bios. With blk-crypto, filesystems only need to call bio_crypt_set_ctx with the encryption key, algorithm and data_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-crypto handles that. Blk-crypto also makes it possible for layered devices like device mapper to make use of inline encryption hardware.
Blk-crypto delegates crypto operations to inline encryption hardware when available. The separately configurable blk-crypto-fallback also contains a software fallback to the kernel crypto API - when enabled, blk-crypto will use this fallback for en/decryption when inline encryption hardware is not available. For more details, refer to Documentation/block/inline-encryption.rst.
--- Documentation/block/index.rst | 1 + Documentation/block/inline-encryption.rst | 183 ++++++ block/Kconfig | 10 + block/Makefile | 4 +- block/bio-crypt-ctx.c | 17 +- block/bio.c | 5 + block/blk-core.c | 13 +- block/blk-crypto-fallback.c | 648 ++++++++++++++++++++++ block/blk-crypto-internal.h | 58 ++ block/blk-crypto.c | 242 ++++++++ include/linux/blk-crypto.h | 63 +++ 11 files changed, 1236 insertions(+), 8 deletions(-) create mode 100644 Documentation/block/inline-encryption.rst create mode 100644 block/blk-crypto-fallback.c create mode 100644 block/blk-crypto-internal.h create mode 100644 block/blk-crypto.c create mode 100644 include/linux/blk-crypto.h
Documentation/block/index.rst +1 -0
@@ -14,6 +14,7 @@ Block cmdline-partition data-integrity deadline-iosched + inline-encryption ioprio kyber-iosched null_blk
Documentation/block/inline-encryption.rst new file +183 -0
183 lines not on this page · raw message
block/Kconfig +10 -0
@@ -188,6 +188,16 @@ config BLK_INLINE_ENCRYPTION block layer handle encryption, so users can take advantage of inline encryption hardware if present. +config BLK_INLINE_ENCRYPTION_FALLBACK + bool "Enable crypto API fallback for blk-crypto" + depends on BLK_INLINE_ENCRYPTION + select CRYPTO + select CRYPTO_SKCIPHER + help + Enabling this lets the block layer handle inline encryption + by falling back to the kernel crypto API when inline + encryption hardware is not present. + menu "Partition Types" source "block/partitions/Kconfig"
block/Makefile +3 -1
@@ -37,4 +37,6 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.o \ + blk-crypto.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) += blk-crypto-fallback.o
block/bio-crypt-ctx.c +13 -4
@@ -9,6 +9,7 @@#include <linux/module.h> #include <linux/slab.h> +#include "blk-crypto-internal.h" static int num_prealloc_crypt_ctxs =128 ;
@@ -21,6 +22,8 @@ static mempool_t *bio_crypt_ctx_pool;int __init bio_crypt_ctx_init (void ) { +size_t i; + bio_crypt_ctx_cache =KMEM_CACHE (bio_crypt_ctx,0 );if (!bio_crypt_ctx_cache)return -ENOMEM ;
@@ -33,6 +36,12 @@ int __init bio_crypt_ctx_init(void)/* This is assumed in various places. */ BUILD_BUG_ON (BLK_ENCRYPTION_MODE_INVALID !=0 ); +/* Sanity check that no algorithm exceeds the defined limits. */ +for (i =0 ; i <BLK_ENCRYPTION_MODE_MAX ; i++) { +BUG_ON (blk_crypto_modes[i].keysize >BLK_CRYPTO_MAX_KEY_SIZE ); +BUG_ON (blk_crypto_modes[i].ivsize >BLK_CRYPTO_MAX_IV_SIZE ); + } +return 0 ; }
@@ -52,11 +61,11 @@ void bio_crypt_clone(struct bio *dst, struct bio *src, gfp_t gfp_mask)const struct bio_crypt_ctx *src_bc = src->bi_crypt_context;/* -* If a bio is swhandled, then it will be decrypted when bio_endio -* is called. As we only want the data to be decrypted once, copies -* of the bio must not have have a crypt context. +* If a bio is fallback_crypted, then it will be decrypted when +* bio_endio is called. As we only want the data to be decrypted once, +* copies of the bio must not have have a crypt context. */ -if (!src_bc) +if (!src_bc ||bio_crypt_fallback_crypted (src_bc))return ; dst->bi_crypt_context =bio_crypt_alloc_ctx (gfp_mask);
block/bio.c +5 -0
@@ -17,6 +17,7 @@#include <linux/cgroup.h> #include <linux/blk-cgroup.h> #include <linux/highmem.h> +#include <linux/blk-crypto.h> #include <trace/events/block.h> #include "blk.h"
@@ -1791,6 +1792,10 @@ void bio_endio(struct bio *bio) again:if (!bio_remaining_done (bio))return ; + +if (!blk_crypto_endio (bio)) +return ; +if (!bio_integrity_endio (bio))return ;
block/blk-core.c +10 -3
@@ -38,6 +38,7 @@#include <linux/debugfs.h> #include <linux/bpf.h> #include <linux/psi.h> +#include <linux/blk-crypto.h> #define CREATE_TRACE_POINTS #include <trace/events/block.h>
@@ -1063,7 +1064,9 @@ blk_qc_t generic_make_request(struct bio *bio)/* Create a fresh bio_list for all subordinate requests */ bio_list_on_stack[1 ] = bio_list_on_stack[0 ];bio_list_init (&bio_list_on_stack[0 ]); - ret = q->make_request_fn (q, bio); + +if (!blk_crypto_submit_bio (&bio)) + ret = q->make_request_fn (q, bio);blk_queue_exit (q);
@@ -1111,7 +1114,7 @@ blk_qc_t direct_make_request(struct bio *bio) {struct request_queue *q = bio->bi_disk->queue;bool nowait = bio->bi_opf &REQ_NOWAIT ; -blk_qc_t ret; +blk_qc_t ret =BLK_QC_T_NONE ;if (!generic_make_request_checks (bio))return BLK_QC_T_NONE ;
@@ -1125,7 +1128,8 @@ blk_qc_t direct_make_request(struct bio *bio)return BLK_QC_T_NONE ; } - ret = q->make_request_fn (q, bio); +if (!blk_crypto_submit_bio (&bio)) + ret = q->make_request_fn (q, bio);blk_queue_exit (q);return ret; }
@@ -1813,5 +1817,8 @@ int __init blk_dev_init(void)if (bio_crypt_ctx_init () <0 )panic ("Failed to allocate mem for bio crypt ctxs\n" ); +if (blk_crypto_fallback_init () <0 ) +panic ("Failed to init blk-crypto-fallback\n" ); +return 0 ; }
block/blk-crypto-fallback.c new file +648 -0
648 lines not on this page · raw message
block/blk-crypto-internal.h new file +58 -0
58 lines not on this page · raw message
block/blk-crypto.c new file +242 -0
242 lines not on this page · raw message
include/linux/blk-crypto.h new file +63 -0
63 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 3/9] block: blk-crypto for Inline Encryption
Quoted 1 line · Satya Tangirala
block/blk-crypto-fallback.c +0
-0
[...]
block/blk-crypto-fallback.c line 52
Quoted 4 lines
block/blk-crypto-fallback.c +4
-0+ +/* The following few vars are only used during the crypto API fallback */ +static struct kmem_cache *bio_fallback_crypt_ctx_cache; +static mempool_t *bio_fallback_crypt_ctx_pool;
The above comment is redundant now that this file only contains the crypto API fallback.
block/blk-crypto-fallback.c line 120
Quoted 12 lines
block/blk-crypto-fallback.c +12
-0+static int blk_crypto_keyslot_program (struct keyslot_manager *ksm, +const struct blk_crypto_key *key, +unsigned int slot) +{ +struct blk_crypto_keyslot *slotp = &blk_crypto_keyslots[slot]; +const enum blk_crypto_mode_num crypto_mode = key->crypto_mode; +int err; + +if (crypto_mode != slotp->crypto_mode && + slotp->crypto_mode !=BLK_ENCRYPTION_MODE_INVALID ) { +blk_crypto_evict_keyslot (slot); + }
Unnecessary braces.
block/blk-crypto-fallback.c line 123
Quoted 3 lines
block/blk-crypto-fallback.c +3
-0+ +if (!slotp->tfms[crypto_mode]) +return -ENOMEM ;
This check seems pointless since blk_crypto_fallback_submit_bio() already checked whether the tfms have been initialized for the crypto_mode or not.
Quoted 9 lines
block/blk-crypto-fallback.c +9
-0+/* +* The crypto API fallback's encryption routine. +* Allocate a bounce bio for encryption, encrypt the input bio using crypto API, +* and replace *bio_ptr with the bounce bio. May split input bio if it's too +* large. +*/ +static int blk_crypto_encrypt_bio (struct bio **bio_ptr) +{ +struct bio *src_bio;
block/blk-crypto-fallback.c line 282
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+struct skcipher_request *ciph_req =NULL ; +DECLARE_CRYPTO_WAIT (wait); +u64 curr_dun[BLK_CRYPTO_DUN_ARRAY_SIZE ]; +union blk_crypto_iv iv; +struct scatterlist src, dst; +struct bio *enc_bio; +unsigned int i, j; +int data_unit_size; +struct bio_crypt_ctx *bc; +int err =0 ;
When there are a lot of variables I feel it helps readability to declare them in the order in which they are used.
Quoted 14 lines
block/blk-crypto-fallback.c +14
-0+/* Encrypt each data unit in this page */ +for (j =0 ; j < enc_bvec->bv_len; j += data_unit_size) { +blk_crypto_dun_to_iv (curr_dun, &iv); + err =crypto_wait_req (crypto_skcipher_encrypt (ciph_req), + &wait); +if (err) { + i++; + src_bio->bi_status =BLK_STS_RESOURCE ; + goto out_free_bounce_pages; + } +bio_crypt_dun_increment (curr_dun,1 ); + src.offset += data_unit_size; + dst.offset += data_unit_size; + }
[...]
block/blk-crypto-fallback.c line 449
Quoted 12 lines
block/blk-crypto-fallback.c +12
-0+/* Decrypt each data unit in the segment */ +for (i =0 ; i < bv.bv_len; i += data_unit_size) { +blk_crypto_dun_to_iv (curr_dun, &iv); +if (crypto_wait_req (crypto_skcipher_decrypt (ciph_req), + &wait)) { + bio->bi_status =BLK_STS_IOERR ; + goto out; + } +bio_crypt_dun_increment (curr_dun,1 ); + sg.offset += data_unit_size; + } + }
Encryption failure is using BLK_STS_RESOURCE whereas decryption failure is using BLK_STS_IOERR. They should use the same error code.
Quoted 18 lines
block/blk-crypto-fallback.c +18
-0+/* +* Queue bio for decryption. +* Returns true iff bio was queued for decryption. +*/ +bool blk_crypto_queue_decrypt_bio (struct bio *bio) +{ +struct blk_crypto_decrypt_work *decrypt_work; + +/* If there was an IO error, don't queue for decrypt. */ +if (bio->bi_status) + goto out; + + decrypt_work =kmem_cache_zalloc (blk_crypto_decrypt_work_cache, +GFP_ATOMIC ); +if (!decrypt_work) { + bio->bi_status =BLK_STS_RESOURCE ; + goto out; + }
block/blk-crypto-fallback.c line 487
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+ +INIT_WORK (&decrypt_work->work, blk_crypto_decrypt_bio); + decrypt_work->bio = bio; +queue_work (blk_crypto_wq, &decrypt_work->work); + +return true; +out: +blk_crypto_free_fallback_crypt_ctx (bio); +return false; +}
Needing to allocate a struct blk_crypto_decrypt_work for every bio adds some complexity and introduces another point of failure. How about embedding the work_struct and back-pointer to the bio in the struct bio_fallback_crypt_ctx instead? Since bio_fallback_crypt_ctx and bio_crypt_ctx are now separate, this approach would no longer add any overhead to hardware inline encryption.
Quoted 6 lines
block/blk-crypto-fallback.c +6
-0+ +/** +* blk_crypto_start_using_mode() - Start using a crypto algorithm on a device +* @mode_num: the blk_crypto_mode we want to allocate ciphers for. +* @data_unit_size: the data unit size that will be used +* @q: the request queue for the device
block/blk-crypto-fallback.c line 503
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+* +* Upper layers must call this function to ensure that a the crypto API fallback +* has transforms for this algorithm, if they become necessary. +* +* Return: 0 on success and -err on error. +*/ +int blk_crypto_start_using_mode (enum blk_crypto_mode_num mode_num, +unsigned int data_unit_size, +struct request_queue *q) +{
I think it would make more sense to put the request_queue parameter first.
Quoted 13 lines
block/blk-crypto-fallback.c +13
-0+struct blk_crypto_keyslot *slotp; +unsigned int i; +int err =0 ; + +/* +* Fast path +* Ensure that updates to blk_crypto_keyslots[i].tfms[mode_num] +* for each i are visible before we try to access them. +*/ +if (likely (smp_load_acquire (&tfms_inited[mode_num]))) +return 0 ; + +/*
block/blk-crypto-fallback.c line 526
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+* If the keyslot manager of the request queue supports this +* crypto mode, then we don't need to allocate this mode. +*/ +if (keyslot_manager_crypto_mode_supported (q->ksm, mode_num, + data_unit_size)) +return 0 ; + +mutex_lock (&tfms_init_lock); +if (likely (tfms_inited[mode_num])) + goto out;
Drop the second likely() since that case isn't really likely.
block/blk-crypto-fallback.c line 575
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+int blk_crypto_fallback_submit_bio (struct bio **bio_ptr) +{ +struct bio *bio = *bio_ptr; +struct bio_crypt_ctx *bc = bio->bi_crypt_context; +struct bio_fallback_crypt_ctx *f_ctx; + +if (WARN_ON_ONCE (!tfms_inited[bc->bc_key->crypto_mode])) { + bio->bi_status =BLK_STS_IOERR ; +return -EIO ; + }
The reason I had suggested the WARN here is because it went with a change to make blk-crypto no longer try to fall back to the crypto API on keyslot programming error. But that change wasn't included in v6. So this WARN can currently be hit, so it's not appropriate for it to be a WARN.
Quoted 42 lines
block/blk-crypto-fallback.c +42
-0+int __init blk_crypto_fallback_init (void ) +{ +int i; +unsigned int crypto_mode_supported[BLK_ENCRYPTION_MODE_MAX ]; + +prandom_bytes (blank_key,BLK_CRYPTO_MAX_KEY_SIZE ); + +/* All blk-crypto modes have a crypto API fallback. */ +for (i =0 ; i <BLK_ENCRYPTION_MODE_MAX ; i++) + crypto_mode_supported[i] =0xFFFFFFFF ; + crypto_mode_supported[BLK_ENCRYPTION_MODE_INVALID ] =0 ; + + blk_crypto_ksm =keyslot_manager_create (blk_crypto_num_keyslots, + &blk_crypto_ksm_ll_ops, + crypto_mode_supported,NULL ); +if (!blk_crypto_ksm) +return -ENOMEM ; + + blk_crypto_wq =alloc_workqueue ("blk_crypto_wq" , +WQ_UNBOUND |WQ_HIGHPRI | +WQ_MEM_RECLAIM ,num_online_cpus ()); +if (!blk_crypto_wq) +return -ENOMEM ; + + blk_crypto_keyslots =kcalloc (blk_crypto_num_keyslots, +sizeof (blk_crypto_keyslots[0 ]), +GFP_KERNEL ); +if (!blk_crypto_keyslots) +return -ENOMEM ; + + blk_crypto_bounce_page_pool = +mempool_create_page_pool (num_prealloc_bounce_pg,0 ); +if (!blk_crypto_bounce_page_pool) +return -ENOMEM ; + + blk_crypto_decrypt_work_cache =KMEM_CACHE (blk_crypto_decrypt_work, +SLAB_RECLAIM_ACCOUNT ); +if (!blk_crypto_decrypt_work_cache) +return -ENOMEM ; + + bio_fallback_crypt_ctx_cache =KMEM_CACHE (bio_fallback_crypt_ctx,0 ); +if (!bio_fallback_crypt_ctx_cache)
block/blk-crypto-fallback.c line 648
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+return -ENOMEM ; + + bio_fallback_crypt_ctx_pool = +mempool_create_slab_pool (num_prealloc_fallback_crypt_ctxs, + bio_fallback_crypt_ctx_cache); +if (!bio_fallback_crypt_ctx_pool) +return -ENOMEM ; + +return 0 ; +}
All this memory gets pre-allocated at boot time when CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK=y even if nothing uses it.
How about delaying all this initialization to blk_crypto_start_using_mode(), similar to the crypto tfm allocations, so that people who don't need the fallback don't pay the price of these memory allocations?
Quoted 57 lines
block/blk-crypto-fallback.c +57
-0+/** +* blk_crypto_submit_bio - handle submitting bio for inline encryption +* +* @bio_ptr: pointer to original bio pointer +* +* If the bio doesn't have inline encryption enabled or the submitter already +* specified a keyslot for the target device, do nothing. Else, a raw key must +* have been provided, so acquire a device keyslot for it if supported. Else, +* use the crypto API fallback. +* +* When the crypto API fallback is used for encryption, blk-crypto may choose to +* split the bio into 2 - the first one that will continue to be processed and +* the second one that will be resubmitted via generic_make_request. +* A bounce bio will be allocated to encrypt the contents of the aforementioned +* "first one", and *bio_ptr will be updated to this bounce bio. +* +* Return: 0 if bio submission should continue; nonzero if bio_endio() was +* already called so bio submission should abort. +*/ +int blk_crypto_submit_bio (struct bio **bio_ptr) +{ +struct bio *bio = *bio_ptr; +struct request_queue *q; +struct bio_crypt_ctx *bc = bio->bi_crypt_context; +int err; + +if (!bc || !bio_has_data (bio)) +return 0 ; + +/* +* When a read bio is marked for fallback decryption, its bi_iter is +* saved so that when we decrypt the bio later, we know what part of it +* was marked for fallback decryption (when the bio is passed down after +* blk_crypto_submit bio, it may be split or advanced so we cannot rely +* on the bi_iter while decrypting in blk_crypto_endio) +*/ +if (bio_crypt_fallback_crypted (bc)) +return 0 ; + + err =bio_crypt_check_alignment (bio); +if (err) { + bio->bi_status =BLK_STS_IOERR ; + goto out; + } + + q = bio->bi_disk->queue; + +if (bc->bc_ksm) { +/* Key already programmed into device? */ +if (q->ksm == bc->bc_ksm) +return 0 ; + +/* Nope, release the existing keyslot. */ +bio_crypt_ctx_release_keyslot (bc); + } + +/* Get device keyslot if supported */
block/blk-crypto-fallback.c
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+if (keyslot_manager_crypto_mode_supported (q->ksm, + bc->bc_key->crypto_mode, + bc->bc_key->data_unit_size)) { + err =bio_crypt_ctx_acquire_keyslot (bc, q->ksm); +if (!err) +return 0 ; + +pr_warn_once ("Failed to acquire keyslot for %s (err=%d). Falling back to crypto API.\n" , + bio->bi_disk->disk_name, err); + }
I'm still not sure we should bother trying to fall back to the crypto API when hardware keyslot programming fails, given that the crypto tfm's are unlikely to have been allocated anyway.
Quoted 4 lines
block/blk-crypto-fallback.c +4
-0+/** +* blk_crypto_init_key() - Prepare a key for use with blk-crypto +* @blk_key: Pointer to the blk_crypto_key to initialize. +* @raw_key: Pointer to the raw key. Must be the correct length for the chosen
block/blk-crypto-fallback.c
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+* @crypto_mode; see blk_crypto_modes[]. +* @crypto_mode: identifier for the encryption algorithm to use +* @data_unit_size: the data unit size to use for en/decryption +* +* Return: The blk_crypto_key that was prepared, or an ERR_PTR() on error. When +* done using the key, it must be freed with blk_crypto_free_key(). +*/ +int blk_crypto_init_key (struct blk_crypto_key *blk_key,const u8 *raw_key, +enum blk_crypto_mode_num crypto_mode, +unsigned int data_unit_size)
The "Return:" comment is outdated as this function got changed from allocation semantics to init semantics, so that struct blk_crypto_key could be embedded in another struct. It should be changed to something that emphasizes that the caller is responsible for zeroizing the keys, e.g.:
Return: 0 on success, -errno on failure. The caller is responsible for
zeroizing both blk_key and raw_key when done with them.
Quoted 15 lines
block/blk-crypto-fallback.c +15
-0+/** +* blk_crypto_evict_key() - Evict a key from any inline encryption hardware +* it may have been programmed into +* @q: The request queue who's keyslot manager this key might have been +* programmed into +* @key: The key to evict +* +* Upper layers (filesystems) should call this function to ensure that a key +* is evicted from hardware that it might have been programmed into. This +* will call keyslot_manager_evict_key on the queue's keyslot manager, if one +* exists, and supports the crypto algorithm with the specified data unit size. +* Otherwise, it will evict the key from the blk-crypto-fallback's ksm. +* +* Return: 0 on success, -err on error. +*/
block/blk-crypto-fallback.c
Quoted 10 lines
block/blk-crypto-fallback.c +10
-0+int blk_crypto_evict_key (struct request_queue *q, +const struct blk_crypto_key *key) +{ +if (q->ksm && +keyslot_manager_crypto_mode_supported (q->ksm, key->crypto_mode, + key->data_unit_size)) +return keyslot_manager_evict_key (q->ksm, key); + +return blk_crypto_fallback_evict_key (key); +}
As I mentioned on the keyslot manager patch, this should return 0 if the key does not currently have a keyslot, as callers will consider that to be success.
- Eric
Eric Biggers
maintainerRe: [PATCH v6 3/9] block: blk-crypto for Inline Encryption
In reply to Satya TangiralaQuoted 30 lines · Satya Tangirala
Documentation/block/inline-encryption.rst new file +24
-0@@ -0,0 +1,183 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Inline Encryption +================= + +Objective +========= + +We want to support inline encryption (IE) in the kernel. +To allow for testing, we also want a crypto API fallback when actual +IE hardware is absent. We also want IE to work with layered devices +like dm and loopback (i.e. we want to be able to use the IE hardware +of the underlying devices if present, or else fall back to crypto API +en/decryption). + + +Constraints and notes +===================== + +- IE hardware have a limited number of "keyslots" that can be programmed + with an encryption context (key, algorithm, data unit size, etc.) at any time. + One can specify a keyslot in a data request made to the device, and the + device will en/decrypt the data using the encryption context programmed into
Documentation/block/inline-encryption.rst line 34
Quoted 10 lines
Documentation/block/inline-encryption.rst new file +10
-0+ that specified keyslot. When possible, we want to make multiple requests with + the same encryption context share the same keyslot. + +- We need a way for filesystems to specify an encryption context to use for + en/decrypting a struct bio, and a device driver (like UFS) needs to be able + to use that encryption context when it processes the bio. + +- We need a way for device drivers to expose their capabilities in a unified + way to the upper layers. +
Can you add an explicit explanation about how inline encryption is different from "self-encrypting drives", like ones based on the TCG Opal or ATA Security standards? That seems to be a common point of confusion.
Also, this documentation file really ought to start by briefly explaining what inline encryption is and why it's important. Then only after that get into the implementation. Don't assume the reader has already read a bunch of commit messages, cover letters, articles, code, etc.
- Eric
Eric Biggers
maintainerRe: [PATCH v6 3/9] block: blk-crypto for Inline Encryption
In reply to Satya Tangiralablock/blk-crypto-internal.h line 42
Quoted 6 lines · Satya Tangirala
block/blk-crypto-internal.h new file +6
-0+static inline int blk_crypto_fallback_submit_bio (struct bio **bio_ptr) +{ +pr_warn_once ("blk-crypto crypto API fallback disabled; failing request" ); + (*bio_ptr)->bi_status =BLK_STS_NOTSUPP ; +return -EIO ; +}
There needs to be a "\n" at the end of this log message. Also, due to the pr_fmt() in blk-crypto.c, this prints as:
blk-crypto: blk-crypto crypto API fallback disabled; failing request
... so the second "blk-crypto" is redundant.
- Eric
PATCH v6 4/9 scsi: ufs: UFS driver v2.1 spec crypto additions
In reply to Satya TangiralaAdd the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI specification in preparation to add support for inline encryption to UFS.
--- drivers/scsi/ufs/ufshcd.c | 2 ++ drivers/scsi/ufs/ufshcd.h | 5 +++ drivers/scsi/ufs/ufshci.h | 67 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 2 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
@@ -4789,6 +4789,8 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp)case OCS_MISMATCH_RESP_UPIU_SIZE :case OCS_PEER_COMM_FAILURE :case OCS_FATAL_ERROR : +case OCS_INVALID_CRYPTO_CONFIG : +case OCS_GENERAL_CRYPTO_ERROR :default : result |=DID_ERROR <<16 ; dev_err(hba->dev,
drivers/scsi/ufs/ufshcd.h +5 -0
@@ -722,6 +722,11 @@ struct ufs_hba {* for userspace to control the power management. */ #define UFSHCD_CAP_RPM_AUTOSUSPEND (1 << 6) +/* +* This capability allows the host controller driver to use the +* inline crypto engine, if it is present +*/ +#define UFSHCD_CAP_CRYPTO (1 << 7)struct devfreq *devfreq;struct ufs_clk_scaling clk_scaling;
drivers/scsi/ufs/ufshci.h +65 -2
106 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v6 4/9] scsi: ufs: UFS driver v2.1 spec crypto additions
Quoted 10 lines · Satya Tangirala
Add the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI specification in preparation to add support for inline encryption to UFS.
Signed-off-by Satya Tangirala <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 2 ++ drivers/scsi/ufs/ufshcd.h | 5 +++ drivers/scsi/ufs/ufshci.h | 67 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 2 deletions(-)
I'd merge this into the next patch.
PATCH v6 5/9 scsi: ufs: UFS crypto API
In reply to Satya TangiralaIntroduce functions to manipulate UFS inline encryption hardware in line with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot manager.
--- drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd-crypto.c | 361 +++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 86 ++++++++ drivers/scsi/ufs/ufshcd.h | 12 + 5 files changed, 469 insertions(+) create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
drivers/scsi/ufs/Kconfig +9 -0
@@ -160,3 +160,12 @@ config SCSI_UFS_BSG Select this if you need a bsg device node for your UFS controller. If unsure, say N. + +config SCSI_UFS_CRYPTO + bool "UFS Crypto Engine Support" + depends on SCSI_UFSHCD && BLK_INLINE_ENCRYPTION + help + Enable Crypto Engine Support in UFS. + Enabling this makes it possible for the kernel to use the crypto + capabilities of the UFS device (if present) to perform crypto + operations on data being transferred to/from the device.
drivers/scsi/ufs/Makefile +1 -0
@@ -12,3 +12,4 @@ obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o obj-$(CONFIG_SCSI_UFS_MEDIATEK) += ufs-mediatek.o obj-$(CONFIG_SCSI_UFS_TI_J721E) += ti-j721e-ufs.o +ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o
drivers/scsi/ufs/ufshcd-crypto.c new file +361 -0
361 lines not on this page · raw message
drivers/scsi/ufs/ufshcd-crypto.h new file +86 -0
86 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +12 -0
@@ -525,6 +525,10 @@ struct ufs_stats {* @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for * device is known or not. * @scsi_block_reqs_cnt: reference counting for scsi block requests +* @crypto_capabilities: Content of crypto capabilities register (0x100) +* @crypto_cap_array: Array of crypto capabilities +* @crypto_cfg_register: Start of the crypto cfg array +* @ksm: the keyslot manager tied to this hba */ struct ufs_hba {void __iomem *mmio_base;
@@ -741,6 +745,14 @@ struct ufs_hba {struct device bsg_dev;struct request_queue *bsg_queue; + +#ifdef CONFIG_SCSI_UFS_CRYPTO +/* crypto */ +union ufs_crypto_capabilities crypto_capabilities; +union ufs_crypto_cap_entry *crypto_cap_array; +u32 crypto_cfg_register; +struct keyslot_manager *ksm; +#endif /* CONFIG_SCSI_UFS_CRYPTO */ };/* Returns true if clocks can be gated. Otherwise false */
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 5/9] scsi: ufs: UFS crypto API
Quoted 10 lines · Satya Tangirala
drivers/scsi/ufs/ufshcd-crypto.c new file +10
-0+/** +* ufshcd_hba_init_crypto - Read crypto capabilities, init crypto fields in hba +* @hba: Per adapter instance +* +* Return: 0 if crypto was initialized or is not supported, else a -errno value. +*/ +int ufshcd_hba_init_crypto (struct ufs_hba *hba) +{ +int cap_idx =0 ; +int err =0 ;
drivers/scsi/ufs/ufshcd-crypto.c line 283
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10
-0+unsigned int crypto_modes_supported[BLK_ENCRYPTION_MODE_MAX ]; +enum blk_crypto_mode_num blk_mode_num; + +/* Default to disabling crypto */ + hba->caps &= ~UFSHCD_CAP_CRYPTO ; + +/* Return 0 if crypto support isn't present */ +if (!(hba->capabilities &MASK_CRYPTO_SUPPORT )) + goto out; +
Note that unfortunately, this patch doesn't work yet on some UFS host controllers that claim to support the UFS standard crypto, due to issues like deviations from the UFS standard and missing device tree changes -- and this can even cause boot-time crashes.
So if we can't fix everything right away (which can be really hard without help from the relevant vendor) I think we have to define a bit UFSHCD_QUIRK_BROKEN_CRYPTO in ufs_hba::quirks which can be set on host controllers where the proper tricks to get the crypto working correctly haven't been figured out yet. The crypto support would be ignored if that bit is set.
- Eric
Eric Biggers
maintainerRe: [PATCH v6 5/9] scsi: ufs: UFS crypto API
Quoted 27 lines · Eric Biggers
Quoted 10 lines · Satya Tangirala
drivers/scsi/ufs/ufshcd-crypto.c new file +10
-0+/** +* ufshcd_hba_init_crypto - Read crypto capabilities, init crypto fields in hba +* @hba: Per adapter instance +* +* Return: 0 if crypto was initialized or is not supported, else a -errno value. +*/ +int ufshcd_hba_init_crypto (struct ufs_hba *hba) +{ +int cap_idx =0 ; +int err =0 ;drivers/scsi/ufs/ufshcd-crypto.c line 283
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10
-0+unsigned int crypto_modes_supported[BLK_ENCRYPTION_MODE_MAX ]; +enum blk_crypto_mode_num blk_mode_num; + +/* Default to disabling crypto */ + hba->caps &= ~UFSHCD_CAP_CRYPTO ; + +/* Return 0 if crypto support isn't present */ +if (!(hba->capabilities &MASK_CRYPTO_SUPPORT )) + goto out; +Note that unfortunately, this patch doesn't work yet on some UFS host controllers that claim to support the UFS standard crypto, due to issues like deviations from the UFS standard and missing device tree changes -- and this can even cause boot-time crashes.
Quoted 5 lines
So if we can't fix everything right away (which can be really hard without help from the relevant vendor) I think we have to define a bit
UFSHCD_QUIRK_BROKEN_CRYPTOinufs_hba::quirkswhich can be set on host controllers where the proper tricks to get the crypto working correctly haven't been figured out yet. The crypto support would be ignored if that bit is set.
I included a patch defining UFSHCD_QUIRK_BROKEN_CRYPTO in my RFC patchset that wires up inline crypto support on Dragonboard 845c:
https://lkml.kernel.org/linux-scsi/[email protected]/
Satya, feel free to include that patch in v7 of your patchset.
- Eric
Re: [PATCH v6 5/9] scsi: ufs: UFS crypto API
In reply to Satya Tangiraladrivers/scsi/ufs/Makefile obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o · line 15
Quoted 8 lines
index
94c6c5d7334b..e88cdcde83fd100644drivers/scsi/ufs/Makefile +1
-0@@ -12,3 +12,4 @@ obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o obj-$(CONFIG_SCSI_UFS_MEDIATEK) += ufs-mediatek.o obj-$(CONFIG_SCSI_UFS_TI_J721E) += ti-j721e-ufs.o +ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o
This line should be moved up to just below the previous statement adding and object to fshcd-core-.
Quoted 5 lines
drivers/scsi/ufs/Makefile +5
-0+static bool ufshcd_cap_idx_valid(struct ufs_hba *hba, unsigned int cap_idx) +{ + return cap_idx < hba->crypto_capabilities.num_crypto_cap; +} +
drivers/scsi/ufs/Makefile
Quoted 10 lines
drivers/scsi/ufs/Makefile +10
-0+static u8 get_data_unit_size_mask(unsigned int data_unit_size) +{ + if (data_unit_size < 512 || data_unit_size > 65536 || + !is_power_of_2(data_unit_size)) + return 0; + + return data_unit_size / 512; +} + +static size_t get_keysize_bytes(enum ufs_crypto_key_size size)
Please add ufshcd_ prefixes to all the helpers.
Quoted 11 lines
drivers/scsi/ufs/Makefile +11
-0+ pm_runtime_get_sync(hba->dev); + ufshcd_hold(hba, false); + /* Clear the dword 16 */ + ufshcd_writel(hba, 0, slot_offset + 16 * sizeof(cfg->reg_val[0])); + /* Ensure that CFGE is cleared before programming the key */ + wmb(); + for (i = 0; i < 16; i++) { + ufshcd_writel(hba, le32_to_cpu(cfg->reg_val[i]), + slot_offset + i * sizeof(cfg->reg_val[0])); + /* Spec says each dword in key must be written sequentially */ + wmb();
drivers/scsi/ufs/Makefile
Quoted 10 lines
drivers/scsi/ufs/Makefile +10
-0+ } + /* Write dword 17 */ + ufshcd_writel(hba, le32_to_cpu(cfg->reg_val[17]), + slot_offset + 17 * sizeof(cfg->reg_val[0])); + /* Dword 16 must be written last */ + wmb(); + /* Write dword 16 */ + ufshcd_writel(hba, le32_to_cpu(cfg->reg_val[16]), + slot_offset + 16 * sizeof(cfg->reg_val[0])); + wmb();
All these wmb calls look bogus as writel itself orders mmio writes, while wmb is not guaranteed to have any effect on mmio space.
drivers/scsi/ufs/Makefile
Quoted 4 lines
drivers/scsi/ufs/Makefile +3
-0+EXPORT_SYMBOL_GPL(ufshcd_crypto_enable); +} +EXPORT_SYMBOL_GPL(ufshcd_crypto_disable);
None of the exported symbols in this file is used outside ufshcd-core.ko, so all the exports caan be dropped.
drivers/scsi/ufs/Makefile
Quoted 3 lines
drivers/scsi/ufs/Makefile +3
-0+ if (ufs_crypto_alg == UFS_CRYPTO_ALG_AES_XTS && + key_size == UFS_CRYPTO_KEY_SIZE_256) + return BLK_ENCRYPTION_MODE_AES_256_XTS;
Please don't indent continuation lines of conditional with a single tab.
drivers/scsi/ufs/Makefile
Quoted 3 lines
drivers/scsi/ufs/Makefile +2
-0+static inline int ufshcd_num_keyslots(struct ufs_hba *hba) +static inline bool ufshcd_keyslot_valid(struct ufs_hba *hba, unsigned int slot)
The two functions are only used in ufshcd-crypto.c and can be moved there.
drivers/scsi/ufs/Makefile
Quoted 4 lines
drivers/scsi/ufs/Makefile +4
-0+static inline bool ufshcd_is_crypto_enabled(struct ufs_hba *hba) +{ + return hba->caps & UFSHCD_CAP_CRYPTO; +}
I think this one would be clearer to just open code in the three callers.
PATCH v6 6/9 scsi: ufs: Add inline encryption support to UFS
In reply to Satya TangiralaWire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.
--- drivers/scsi/ufs/ufshcd-crypto.c | 30 ++++++++++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 21 +++++++++++++ drivers/scsi/ufs/ufshcd.c | 54 +++++++++++++++++++++++++++++--- drivers/scsi/ufs/ufshcd.h | 8 +++++ 4 files changed, 108 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd-crypto.c +30 -0
@@ -352,6 +352,36 @@ void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba, }EXPORT_SYMBOL_GPL (ufshcd_crypto_setup_rq_keyslot_manager); +int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +struct bio_crypt_ctx *bc; + +if (!bio_crypt_should_process (cmd->request)) { + lrbp->crypto_enable = false; +return 0 ; + } + bc = cmd->request->bio->bi_crypt_context; + +if (WARN_ON (!ufshcd_is_crypto_enabled (hba))) { +/* +* Upper layer asked us to do inline encryption +* but that isn't enabled, so we fail this request. +*/ +return -EINVAL ; + } +if (!ufshcd_keyslot_valid (hba, bc->bc_keyslot)) +return -EINVAL ; + + lrbp->crypto_enable = true; + lrbp->crypto_key_slot = bc->bc_keyslot; + lrbp->data_unit_num = bc->bc_dun[0 ]; + +return 0 ; +} +EXPORT_SYMBOL_GPL (ufshcd_prepare_lrbp_crypto); +void ufshcd_crypto_destroy_rq_keyslot_manager (struct ufs_hba *hba,struct request_queue *q) {
drivers/scsi/ufs/ufshcd-crypto.h +21 -0
@@ -36,6 +36,15 @@ static inline bool ufshcd_is_crypto_enabled(struct ufs_hba *hba) }/* Functions implementing UFSHCI v2.1 specification behaviour */ +int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp); + +static inline bool ufshcd_lrbp_crypto_enabled (struct ufshcd_lrb *lrbp) +{ +return lrbp->crypto_enable; +} +void ufshcd_crypto_enable (struct ufs_hba *hba);void ufshcd_crypto_disable (struct ufs_hba *hba);
@@ -81,6 +90,18 @@ static inline void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba,static inline void ufshcd_crypto_destroy_rq_keyslot_manager (struct ufs_hba *hba,struct request_queue *q) { } +static inline int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +return 0 ; +} + +static inline bool ufshcd_lrbp_crypto_enabled (struct ufshcd_lrb *lrbp) +{ +return false; +} +#endif /* CONFIG_SCSI_UFS_CRYPTO */ #endif /* _UFSHCD_CRYPTO_H */
drivers/scsi/ufs/ufshcd.c +49 -5
121 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +8 -0
@@ -167,6 +167,9 @@ struct ufs_pm_lvl_states {* @intr_cmd: Interrupt command (doesn't participate in interrupt aggregation) * @issue_time_stamp: time stamp for debug purposes * @compl_time_stamp: time stamp for statistics +* @crypto_enable: whether or not the request needs inline crypto operations +* @crypto_key_slot: the key slot to use for inline crypto +* @data_unit_num: the data unit number for the first block for inline crypto * @req_abort_skip: skip request abort task flag */ struct ufshcd_lrb {
@@ -191,6 +194,11 @@ struct ufshcd_lrb {bool intr_cmd;ktime_t issue_time_stamp;ktime_t compl_time_stamp; +#if IS_ENABLED (CONFIG_SCSI_UFS_CRYPTO ) +bool crypto_enable; +u8 crypto_key_slot; +u64 data_unit_num; +#endif /* CONFIG_SCSI_UFS_CRYPTO */ bool req_abort_skip; };
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 6/9] scsi: ufs: Add inline encryption support to UFS
Quoted 38 lines · Satya Tangirala
Wire up
ufshcd.cwith the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.Signed-off-by Satya Tangirala <[email protected]>--- drivers/scsi/ufs/ufshcd-crypto.c | 30 ++++++++++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 21 +++++++++++++ drivers/scsi/ufs/ufshcd.c | 54 +++++++++++++++++++++++++++++--- drivers/scsi/ufs/ufshcd.h | 8 +++++ 4 files changed, 108 insertions(+), 5 deletions(-)drivers/scsi/ufs/ufshcd-crypto.c +19
-0@@ -352,6 +352,36 @@ void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba, }EXPORT_SYMBOL_GPL (ufshcd_crypto_setup_rq_keyslot_manager); +int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +struct bio_crypt_ctx *bc; + +if (!bio_crypt_should_process (cmd->request)) { + lrbp->crypto_enable = false; +return 0 ; + } + bc = cmd->request->bio->bi_crypt_context; + +if (WARN_ON (!ufshcd_is_crypto_enabled (hba))) { +/* +* Upper layer asked us to do inline encryption +* but that isn't enabled, so we fail this request. +*/ +return -EINVAL ; + }
drivers/scsi/ufs/ufshcd-crypto.c void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba, · line 383
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c +10
-0+if (!ufshcd_keyslot_valid (hba, bc->bc_keyslot)) +return -EINVAL ; + + lrbp->crypto_enable = true; + lrbp->crypto_key_slot = bc->bc_keyslot; + lrbp->data_unit_num = bc->bc_dun[0 ]; + +return 0 ; +} +EXPORT_SYMBOL_GPL (ufshcd_prepare_lrbp_crypto);
The UFS driver only uses the first 64 bits of the DUN, but in this version of the patchset the DUN in the bio_crypt_ctx can be up to the real length of the algorithm's IV -- which for AES-256-XTS is 128 bits. So if the user were to specify anything nonzero in bits 64-127, the crypto would be done incorrectly.
(This case isn't encountered with fscrypt. But it's still an issue with the overall approach.)
So there needs to be a way for drivers to declare the max_dun_size they support, and prevent them from being used with longer DUNs.
- Eric
Re: [PATCH v6 6/9] scsi: ufs: Add inline encryption support to UFS
In reply to Satya TangiralaQuoted 2 lines · Satya Tangirala
Wire up
ufshcd.cwith the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.
I think this patch should be merged into the previous patch, as the previous one isn't useful without wiring it up.
drivers/scsi/ufs/ufshcd-crypto.c void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba, · line 364
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c +10
-0+int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +struct bio_crypt_ctx *bc; + +if (!bio_crypt_should_process (cmd->request)) { + lrbp->crypto_enable = false; +return 0 ; + }
I think this check belongs into the caller so that there is no function call overhead for the !inline crypto case. If you extend the conditional
to
if (!IS_ENABLED(CONFIG_SCSI_UFS_CRYPTO) || !bio_crypt_should_process(cmd->request))
you also don't need the stub for ufshcd_prepare_lrbp_crypto.
Quoted 1 line
+EXPORT_SYMBOL_GPL(ufshcd_prepare_lrbp_crypto);
No need to export this function.
drivers/scsi/ufs/ufshcd.c static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp, · line 2233
Quoted 9 lines
drivers/scsi/ufs/ufshcd.c +9
-0+if (ufshcd_lrbp_crypto_enabled (lrbp)) { +#if IS_ENABLED (CONFIG_SCSI_UFS_CRYPTO ) + dword_0 |=UTP_REQ_DESC_CRYPTO_ENABLE_CMD ; + dword_0 |= lrbp->crypto_key_slot; + req_desc->header.dword_1 = +cpu_to_le32 (lower_32_bits (lrbp->data_unit_num)); + req_desc->header.dword_3 = +cpu_to_le32 (upper_32_bits (lrbp->data_unit_num)); +#endif /* CONFIG_SCSI_UFS_CRYPTO */
This can be a plain old ifdef without the IS_ENABLED obsfucation.
drivers/scsi/ufs/ufshcd.c static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, · line 2537
Quoted 3 lines
drivers/scsi/ufs/ufshcd.c +3
-0+#if IS_ENABLED (CONFIG_SCSI_UFS_CRYPTO ) + lrbp->crypto_enable = false;/* No crypto operations */ +#endif
Same here.
drivers/scsi/ufs/ufshcd.h struct ufshcd_lrb { · line 201
Quoted 5 lines
drivers/scsi/ufs/ufshcd.h +5
-0+#if IS_ENABLED (CONFIG_SCSI_UFS_CRYPTO ) +bool crypto_enable; +u8 crypto_key_slot; +u64 data_unit_num; +#endif /* CONFIG_SCSI_UFS_CRYPTO */
.. and here.
Eric Biggers
maintainerRe: [PATCH v6 6/9] scsi: ufs: Add inline encryption support to UFS
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Satya Tangirala
Wire up
ufshcd.cwith the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.I think this patch should be merged into the previous patch, as the previous one isn't useful without wiring it up.
Satya actually did this originally but then one of the UFS maintainers requested the separate patches for (1) new registers, (2) ufshcd-crypto, and (3) ufshcd.c:
https://lore.kernel.org/linux-block/SN6PR04MB49259F70346E2055C9E0F401FC310@SN6PR04MB4925.namprd04.prod.outlook.com/
So, he's not going to be able to make everyone happy :-)
I personally would be fine with either way.
- Eric
Re: [PATCH v6 6/9] scsi: ufs: Add inline encryption support to UFS
Quoted 9 lines · Eric Biggers
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · Satya Tangirala
Wire up
ufshcd.cwith the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.I think this patch should be merged into the previous patch, as the previous one isn't useful without wiring it up.
Quoted 7 lines
Satya actually did this originally but then one of the UFS maintainers requested the separate patches for (1) new registers, (2) ufshcd-crypto, and (3)
ufshcd.c:
https://lore.kernel.org/linux-block/SN6PR04MB49259F70346E2055C9E0F401FC310@SN6PR04MB4925.namprd04.prod.outlook.com/So, he's not going to be able to make everyone happy :-)
I personally would be fine with either way.
Oh well, the split between adding functions and callers is highly unusual for Linux development. Adding the defines I can see, especially if they are large (which these aren't). But you'll need to get this accepted by the UFS folks, so I'll shut up now.
Eric Biggers
maintainerRe: [PATCH v6 6/9] scsi: ufs: Add inline encryption support to UFS
In reply to Satya TangiralaQuoted 18 lines · Satya Tangirala
drivers/scsi/ufs/ufshcd.c +3
-0@@ -4654,6 +4686,8 @@ static int ufshcd_slave_configure(struct scsi_device *sdev)if (ufshcd_is_rpm_autosuspend_allowed (hba)) sdev->rpm_autosuspend =1 ; +ufshcd_crypto_setup_rq_keyslot_manager (hba, q); +return 0 ; }@@ -4664,6 +4698,7 @@ static int ufshcd_slave_configure(struct scsi_device *sdev)static void ufshcd_slave_destroy (struct scsi_device *sdev) {struct ufs_hba *hba; +struct request_queue *q = sdev->request_queue; hba =shost_priv (sdev->host);/* Drop the reference as it won't be needed anymore */
drivers/scsi/ufs/ufshcd.c static void ufshcd_slave_destroy(struct scsi_device *sdev) · line 4714
Quoted 6 lines
drivers/scsi/ufs/ufshcd.c +2
-0@@ -4674,6 +4709,8 @@ static void ufshcd_slave_destroy(struct scsi_device *sdev) hba->sdev_ufs_device =NULL ;spin_unlock_irqrestore (hba->host->host_lock, flags); } + +ufshcd_crypto_destroy_rq_keyslot_manager (hba, q); }
Just noticed this --- this is still destroying the keyslot manager when a SCSI device is destroyed. The keyslot manager is associated with the host controller (which might control multiple devices), so it must not be destroyed until the ufs_hba is destroyed, i.e. in ufshcd_dealloc_host().
(I was also thinking about whether we could use devm so that the keyslot_manager doesn't need to be explicitly freed. But that wouldn't actually help because we still need to ensure that all the crypto keys get zeroed.)
- Eric
Eric Biggers
maintainerRe: [PATCH v6 6/9] scsi: ufs: Add inline encryption support to UFS
In reply to Satya Tangiraladrivers/scsi/ufs/ufshcd.c static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) · line 2501
Quoted 11 lines · Satya Tangirala
drivers/scsi/ufs/ufshcd.c +7
-0@@ -2472,6 +2492,13 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) lrbp->task_tag = tag; lrbp->lun =ufshcd_scsi_to_upiu_lun (cmd->device->lun); lrbp->intr_cmd = !ufshcd_is_intr_aggr_allowed (hba) ? true : false; + + err =ufshcd_prepare_lrbp_crypto (hba, cmd, lrbp); +if (err) { + lrbp->cmd =NULL ; +clear_bit_unlock (tag, &hba->lrb_in_use); + goto out; + }
The error path here is missing a call to ufshcd_release().
- Eric
PATCH v6 7/9 fscrypt: add inline encryption support
In reply to Satya TangiralaAdd support for inline encryption to fs/crypto/. With "inline encryption", the block layer handles the decryption/encryption as part of the bio, instead of the filesystem doing the crypto itself via Linux's crypto API. This model is needed in order to take advantage of the inline encryption hardware present on most modern mobile SoCs.
To use inline encryption, the filesystem needs to be mounted with '-o inlinecrypt'. The contents of any encrypted files will then be encrypted using blk-crypto, instead of using the traditional filesystem-layer crypto. Fscrypt still provides the key and IV to use, and the actual ciphertext on-disk is still the same; therefore it's testable using the existing fscrypt ciphertext verification tests.
Note that since blk-crypto has a fallack to Linux's crypto API, and also supports all the encryption modes currently supported by fscrypt, this feature is usable and testable even without actual inline encryption hardware.
Per-filesystem changes will be needed to set encryption contexts when submitting bios and to implement the 'inlinecrypt' mount option. This patch just adds the common code.
--- fs/crypto/Kconfig | 6 + fs/crypto/Makefile | 1 + fs/crypto/bio.c | 28 +++- fs/crypto/crypto.c | 2 +- fs/crypto/fname.c | 4 +- fs/crypto/fscrypt_private.h | 122 ++++++++++++-- fs/crypto/inline_crypt.c | 319 ++++++++++++++++++++++++++++++++++++ fs/crypto/keyring.c | 4 +- fs/crypto/keysetup.c | 102 ++++++++---- fs/crypto/keysetup_v1.c | 16 +- include/linux/fscrypt.h | 58 +++++++ 11 files changed, 591 insertions(+), 71 deletions(-) create mode 100644 fs/crypto/inline_crypt.c
fs/crypto/Kconfig +6 -0
@@ -16,3 +16,9 @@ config FS_ENCRYPTION efficient since it avoids caching the encrypted and decrypted pages in the page cache. Currently Ext4, F2FS and UBIFS make use of this feature. + +config FS_ENCRYPTION_INLINE_CRYPT + bool "Enable fscrypt to use inline crypto" + depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION + help + Enable fscrypt to use inline encryption hardware if available.
fs/crypto/Makefile +1 -0
@@ -11,3 +11,4 @@ fscrypto-y := crypto.o \ policy.o fscrypto-$(CONFIG_BLOCK) += bio.o +fscrypto-$(CONFIG_FS_ENCRYPTION_INLINE_CRYPT) += inline_crypt.o
fs/crypto/bio.c +19 -9
@@ -46,26 +46,35 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, {const unsigned int blockbits = inode->i_blkbits;const unsigned int blocksize =1 << blockbits; +const bool inlinecrypt =fscrypt_inode_uses_inline_crypto (inode);struct page *ciphertext_page;struct bio *bio;int ret, err =0 ; - ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); -if (!ciphertext_page) -return -ENOMEM ; +if (inlinecrypt) { + ciphertext_page =ZERO_PAGE (0 ); + }else { + ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); +if (!ciphertext_page) +return -ENOMEM ; + }while (len--) { - err =fscrypt_crypt_block (inode,FS_ENCRYPT , lblk, -ZERO_PAGE (0 ), ciphertext_page, - blocksize,0 ,GFP_NOFS ); -if (err) - goto errout; +if (!inlinecrypt) { + err =fscrypt_crypt_block (inode,FS_ENCRYPT , lblk, +ZERO_PAGE (0 ), ciphertext_page, + blocksize,0 ,GFP_NOFS ); +if (err) + goto errout; + } bio =bio_alloc (GFP_NOWAIT ,1 );if (!bio) { err = -ENOMEM ; goto errout; } +fscrypt_set_bio_crypt_ctx (bio, inode, lblk,GFP_NOIO ); +bio_set_dev (bio, inode->i_sb->s_bdev); bio->bi_iter.bi_sector = pblk << (blockbits -9 );bio_set_op_attrs (bio,REQ_OP_WRITE ,0 );
@@ -87,7 +96,8 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, } err =0 ; errout: -fscrypt_free_bounce_page (ciphertext_page); +if (!inlinecrypt) +fscrypt_free_bounce_page (ciphertext_page);return err; }EXPORT_SYMBOL (fscrypt_zeroout_range);
fs/crypto/crypto.c +1 -1
@@ -97,7 +97,7 @@ int fscrypt_crypt_block(const struct inode *inode, fscrypt_direction_t rw,DECLARE_CRYPTO_WAIT (wait);struct scatterlist dst, src;struct fscrypt_info *ci = inode->i_crypt_info; -struct crypto_skcipher *tfm = ci->ci_ctfm; +struct crypto_skcipher *tfm = ci->ci_key.tfm;int res =0 ;if (WARN_ON_ONCE (len <=0 ))
fs/crypto/fname.c +2 -2
@@ -40,7 +40,7 @@ int fname_encrypt(struct inode *inode, const struct qstr *iname,struct skcipher_request *req =NULL ;DECLARE_CRYPTO_WAIT (wait);struct fscrypt_info *ci = inode->i_crypt_info; -struct crypto_skcipher *tfm = ci->ci_ctfm; +struct crypto_skcipher *tfm = ci->ci_key.tfm;union fscrypt_iv iv;struct scatterlist sg;int res;
@@ -93,7 +93,7 @@ static int fname_decrypt(struct inode *inode,DECLARE_CRYPTO_WAIT (wait);struct scatterlist src_sg, dst_sg;struct fscrypt_info *ci = inode->i_crypt_info; -struct crypto_skcipher *tfm = ci->ci_ctfm; +struct crypto_skcipher *tfm = ci->ci_key.tfm;union fscrypt_iv iv;int res;
fs/crypto/fscrypt_private.h +109 -13
183 lines not on this page · raw message
fs/crypto/inline_crypt.c new file +319 -0
319 lines not on this page · raw message
fs/crypto/keyring.c +2 -2
@@ -44,8 +44,8 @@ static void free_master_key(struct fscrypt_master_key *mk)wipe_master_key_secret (&mk->mk_secret);for (i =0 ; i <= __FSCRYPT_MODE_MAX; i++) { -crypto_free_skcipher (mk->mk_direct_tfms[i]); -crypto_free_skcipher (mk->mk_iv_ino_lblk_64_tfms[i]); +fscrypt_destroy_prepared_key (&mk->mk_direct_keys[i]); +fscrypt_destroy_prepared_key (&mk->mk_iv_ino_lblk_64_keys[i]); }key_put (mk->mk_users);
fs/crypto/keysetup.c +67 -35
198 lines not on this page · raw message
fs/crypto/keysetup_v1.c +7 -9
@@ -146,7 +146,7 @@ struct fscrypt_direct_key {struct hlist_node dk_node;refcount_t dk_refcount;const struct fscrypt_mode *dk_mode; -struct crypto_skcipher *dk_ctfm; +struct fscrypt_prepared_key dk_key;u8 dk_descriptor[FSCRYPT_KEY_DESCRIPTOR_SIZE ];u8 dk_raw[FSCRYPT_MAX_KEY_SIZE ]; };
@@ -154,7 +154,7 @@ struct fscrypt_direct_key {static void free_direct_key (struct fscrypt_direct_key *dk) {if (dk) { -crypto_free_skcipher (dk->dk_ctfm); +fscrypt_destroy_prepared_key (&dk->dk_key);kzfree (dk); } }
@@ -199,6 +199,8 @@ find_or_insert_direct_key(struct fscrypt_direct_key *to_insert,continue ;if (ci->ci_mode != dk->dk_mode)continue ; +if (!fscrypt_is_key_prepared (&dk->dk_key, ci)) +continue ;if (crypto_memneq (raw_key, dk->dk_raw, ci->ci_mode->keysize))continue ;/* using existing tfm with same (descriptor, mode, raw_key) */
@@ -231,13 +233,9 @@ fscrypt_get_direct_key(const struct fscrypt_info *ci, const u8 *raw_key)return ERR_PTR (-ENOMEM );refcount_set (&dk->dk_refcount,1 ); dk->dk_mode = ci->ci_mode; - dk->dk_ctfm =fscrypt_allocate_skcipher (ci->ci_mode, raw_key, - ci->ci_inode); -if (IS_ERR (dk->dk_ctfm)) { - err =PTR_ERR (dk->dk_ctfm); - dk->dk_ctfm =NULL ; + err =fscrypt_prepare_key (&dk->dk_key, raw_key, ci); +if (err) goto err_free_dk; - }memcpy (dk->dk_descriptor, ci->ci_policy.v1.master_key_descriptor,FSCRYPT_KEY_DESCRIPTOR_SIZE );memcpy (dk->dk_raw, raw_key, ci->ci_mode->keysize);
@@ -274,7 +272,7 @@ static int setup_v1_file_key_direct(struct fscrypt_info *ci,if (IS_ERR (dk))return PTR_ERR (dk); ci->ci_direct_key = dk; - ci->ci_ctfm = dk->dk_ctfm; + ci->ci_key = dk->dk_key;return 0 ; }
include/linux/fscrypt.h +58 -0
70 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 7/9] fscrypt: add inline encryption support
Quoted 49 lines · Satya Tangirala
fs/crypto/bio.c +17
-8@@ -46,26 +46,35 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, {const unsigned int blockbits = inode->i_blkbits;const unsigned int blocksize =1 << blockbits; +const bool inlinecrypt =fscrypt_inode_uses_inline_crypto (inode);struct page *ciphertext_page;struct bio *bio;int ret, err =0 ; - ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); -if (!ciphertext_page) -return -ENOMEM ; +if (inlinecrypt) { + ciphertext_page =ZERO_PAGE (0 ); + }else { + ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); +if (!ciphertext_page) +return -ENOMEM ; + }while (len--) { - err =fscrypt_crypt_block (inode,FS_ENCRYPT , lblk, -ZERO_PAGE (0 ), ciphertext_page, - blocksize,0 ,GFP_NOFS ); -if (err) - goto errout; +if (!inlinecrypt) { + err =fscrypt_crypt_block (inode,FS_ENCRYPT , lblk, +ZERO_PAGE (0 ), ciphertext_page, + blocksize,0 ,GFP_NOFS ); +if (err) + goto errout; + } bio =bio_alloc (GFP_NOWAIT ,1 );if (!bio) { err = -ENOMEM ; goto errout; } +fscrypt_set_bio_crypt_ctx (bio, inode, lblk,GFP_NOIO ); +bio_set_dev (bio, inode->i_sb->s_bdev); bio->bi_iter.bi_sector = pblk << (blockbits -9 );bio_set_op_attrs (bio,REQ_OP_WRITE ,0 );
fs/crypto/bio.c int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, · line 103
Quoted 9 lines
fs/crypto/bio.c +2
-1@@ -87,7 +96,8 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, } err =0 ; errout: -fscrypt_free_bounce_page (ciphertext_page); +if (!inlinecrypt) +fscrypt_free_bounce_page (ciphertext_page);return err; }EXPORT_SYMBOL (fscrypt_zeroout_range);
FYI, I've just applied a patch
(https://lore.kernel.org/r/[email protected]/)
to fscrypt.git#master that optimizes this function to write multiple pages at a time. So this part of this patch will need to be reworked. I suggest just handling the inline and fs-layer encryption cases separately.
I maintain a testing branch that has all the pending patches I'm interested in applied, so I actually already hacked together the following to resolve the conflict. Please double check it carefully before using it in v7 though:
static int fscrypt_zeroout_range_inlinecrypt(const struct inode *inode,
pgoff_t lblk, sector_t pblk, unsigned int len)
{
const unsigned int blockbits = inode->i_blkbits;
const unsigned int blocks_per_page_bits = PAGE_SHIFT - blockbits;
const unsigned int blocks_per_page = 1 << blocks_per_page_bits;
unsigned int i;
struct bio *bio;
int ret, err;
/* This always succeeds since __GFP_DIRECT_RECLAIM is set. */
bio = bio_alloc(GFP_NOFS, BIO_MAX_PAGES);
do {
bio_set_dev(bio, inode->i_sb->s_bdev);
bio->bi_iter.bi_sector = pblk << (blockbits - 9);
bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
fscrypt_set_bio_crypt_ctx(bio, inode, lblk, GFP_NOFS);
i = 0;
do {
unsigned int blocks_this_page =
min(len, blocks_per_page);
unsigned int bytes_this_page =
blocks_this_page << blockbits;
ret = bio_add_page(bio, ZERO_PAGE(0),
bytes_this_page, 0);
if (WARN_ON(ret != bytes_this_page)) {
err = -EIO;
goto out;
}
lblk += blocks_this_page;
pblk += blocks_this_page;
len -= blocks_this_page;
} while (++i != BIO_MAX_PAGES && len != 0);
err = submit_bio_wait(bio);
if (err)
goto out;
bio_reset(bio);
} while (len != 0);
err = 0;
out:
bio_put(bio);
return err;
}PATCH v6 8/9 f2fs: add inline encryption support
In reply to Satya TangiralaWire up f2fs to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:
- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
--- fs/f2fs/data.c | 65 +++++++++++++++++++++++++++++++++++++++++++------ fs/f2fs/f2fs.h | 3 +++ fs/f2fs/super.c | 41 +++++++++++++++++++++++++++++++ 3 files changed, 101 insertions(+), 8 deletions(-)
fs/f2fs/data.c +57 -8
@@ -308,6 +308,33 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)return bio; } +static void f2fs_set_bio_crypt_ctx (struct bio *bio,const struct inode *inode, +pgoff_t first_idx, +const struct f2fs_io_info *fio, +gfp_t gfp_mask) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (!fio || !fio->encrypted_page) +fscrypt_set_bio_crypt_ctx (bio, inode, first_idx, gfp_mask); +} + +static bool f2fs_crypt_mergeable_bio (struct bio *bio,const struct inode *inode, +pgoff_t next_idx, +const struct f2fs_io_info *fio) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (fio && fio->encrypted_page) +return !bio_has_crypt_ctx (bio); + +return fscrypt_mergeable_bio (bio, inode, next_idx); +} +static inline void __submit_bio (struct f2fs_sb_info *sbi,struct bio *bio,enum page_type type) {
@@ -491,6 +518,9 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)/* Allocate a new bio */ bio =__bio_alloc (fio,1 ); +f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio,GFP_NOIO ); +if (bio_add_page (bio, page,PAGE_SIZE ,0 ) <PAGE_SIZE ) {bio_put (bio);return -EFAULT ;
@@ -678,12 +708,18 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)trace_f2fs_submit_page_bio (page, fio);f2fs_trace_ios (fio,0 ); -if (bio && !page_is_mergeable (fio->sbi, bio, *fio->last_block, - fio->new_blkaddr)) +if (bio && (!page_is_mergeable (fio->sbi, bio, *fio->last_block, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (bio, fio->page->mapping->host, + fio->page->index, fio))) {f2fs_submit_merged_ipu_write (fio->sbi, &bio,NULL ); +} alloc_new:if (!bio) { bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO );bio_set_op_attrs (bio, fio->op, fio->op_flags);add_bio_entry (fio->sbi, bio, page, fio->temp);
@@ -735,8 +771,11 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)inc_page_count (sbi,WB_DATA_TYPE (bio_page)); -if (io->bio && !io_is_mergeable (sbi, io->bio, io, fio, - io->last_block_in_bio, fio->new_blkaddr)) +if (io->bio && + (!io_is_mergeable (sbi, io->bio, io, fio, io->last_block_in_bio, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (io->bio, fio->page->mapping->host, + fio->page->index, fio)))__submit_merged_bio (io); alloc_new:if (io->bio ==NULL ) {
@@ -748,6 +787,9 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio) goto skip; } io->bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (io->bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO ); io->fio = *fio; }
@@ -791,11 +833,14 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode, block_t blkaddr, bio =f2fs_bio_alloc (sbi,min_t (int, nr_pages,BIO_MAX_PAGES ), false);if (!bio)return ERR_PTR (-ENOMEM ); + +f2fs_set_bio_crypt_ctx (bio, inode, first_idx,NULL ,GFP_NOFS ); +f2fs_target_device (sbi, blkaddr, bio); bio->bi_end_io = f2fs_read_end_io;bio_set_op_attrs (bio,REQ_OP_READ , op_flag); -if (f2fs_encrypted_file (inode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (f2fs_need_verity (inode, first_idx))
@@ -1832,8 +1877,9 @@ static int f2fs_read_single_page(struct inode *inode, struct page *page,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && !page_is_mergeable (F2FS_I_SB (inode), bio, - *last_block_in_bio, block_nr)) { +if (bio && (!page_is_mergeable (F2FS_I_SB (inode), bio, + *last_block_in_bio, block_nr) || + !f2fs_crypt_mergeable_bio (bio, inode, page->index,NULL ))) { submit_and_realloc:__submit_bio (F2FS_I_SB (inode), bio,DATA ); bio =NULL ;
@@ -1973,6 +2019,9 @@ static int encrypt_one_page(struct f2fs_io_info *fio)/* wait for GCed page writeback via META_MAPPING */ f2fs_wait_on_block_writeback (inode, fio->old_blkaddr); +if (fscrypt_inode_uses_inline_crypto (inode)) +return 0 ; + retry_encrypt: fio->encrypted_page = fscrypt_encrypt_pagecache_blocks(fio->page, PAGE_SIZE, 0,
@@ -2147,7 +2196,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)f2fs_unlock_op (fio->sbi); err =f2fs_inplace_write_data (fio);if (err) { -if (f2fs_encrypted_file (inode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode))fscrypt_finalize_bounce_page (&fio->encrypted_page);if (PageWriteback (page))end_page_writeback (page);
fs/f2fs/f2fs.h +3 -0
@@ -137,6 +137,9 @@ struct f2fs_mount_info {int alloc_mode;/* segment allocation policy */ int fsync_mode;/* fsync policy */ bool test_dummy_encryption;/* test dummy encryption */ +#ifdef CONFIG_FS_ENCRYPTION +bool inlinecrypt;/* inline encryption enabled */ +#endif block_t unusable_cap;/* Amount of space allowed to be * unusable when disabling checkpoint */
fs/f2fs/super.c +41 -0
83 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 8/9] f2fs: add inline encryption support
Quoted 73 lines · Satya Tangirala
Wire up f2fs to support inline encryption via the helper functions which
fs/crypto/now provides. This includes:- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the
bio_crypt_ctxon bios that will be submitted to an inline-encrypted file.- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
Co-developed-by Eric Biggers <[email protected]>Signed-off-by Eric Biggers <[email protected]>Signed-off-by Satya Tangirala <[email protected]>--- fs/f2fs/data.c | 65 +++++++++++++++++++++++++++++++++++++++++++------ fs/f2fs/f2fs.h | 3 +++ fs/f2fs/super.c | 41 +++++++++++++++++++++++++++++++ 3 files changed, 101 insertions(+), 8 deletions(-)fs/f2fs/data.c +30
-0@@ -308,6 +308,33 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)return bio; } +static void f2fs_set_bio_crypt_ctx (struct bio *bio,const struct inode *inode, +pgoff_t first_idx, +const struct f2fs_io_info *fio, +gfp_t gfp_mask) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (!fio || !fio->encrypted_page) +fscrypt_set_bio_crypt_ctx (bio, inode, first_idx, gfp_mask); +} + +static bool f2fs_crypt_mergeable_bio (struct bio *bio,const struct inode *inode, +pgoff_t next_idx, +const struct f2fs_io_info *fio) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (fio && fio->encrypted_page) +return !bio_has_crypt_ctx (bio); + +return fscrypt_mergeable_bio (bio, inode, next_idx); +} +static inline void __submit_bio (struct f2fs_sb_info *sbi,struct bio *bio,enum page_type type) {@@ -491,6 +518,9 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)/* Allocate a new bio */ bio =__bio_alloc (fio,1 ); +f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio,GFP_NOIO ); +if (bio_add_page (bio, page,PAGE_SIZE ,0 ) <PAGE_SIZE ) {bio_put (bio);return -EFAULT ;@@ -678,12 +708,18 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)trace_f2fs_submit_page_bio (page, fio);
fs/f2fs/data.c int f2fs_merge_page_bio(struct f2fs_io_info *fio) · line 716
Quoted 10 lines
fs/f2fs/data.c +5
-2f2fs_trace_ios (fio,0 ); -if (bio && !page_is_mergeable (fio->sbi, bio, *fio->last_block, - fio->new_blkaddr)) +if (bio && (!page_is_mergeable (fio->sbi, bio, *fio->last_block, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (bio, fio->page->mapping->host, + fio->page->index, fio))) {f2fs_submit_merged_ipu_write (fio->sbi, &bio,NULL ); +}
Unnecessary (and wrongly formatted) braces here.
fs/f2fs/data.c int f2fs_merge_page_bio(struct f2fs_io_info *fio) · line 723
Quoted 7 lines
fs/f2fs/data.c +3
-0alloc_new:if (!bio) { bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO );bio_set_op_attrs (bio, fio->op, fio->op_flags);
Nit: GFP_NOIO can be joined with the previous line.
Quoted 17 lines
fs/f2fs/data.c +5
-2add_bio_entry (fio->sbi, bio, page, fio->temp);@@ -735,8 +771,11 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)inc_page_count (sbi,WB_DATA_TYPE (bio_page)); -if (io->bio && !io_is_mergeable (sbi, io->bio, io, fio, - io->last_block_in_bio, fio->new_blkaddr)) +if (io->bio && + (!io_is_mergeable (sbi, io->bio, io, fio, io->last_block_in_bio, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (io->bio, fio->page->mapping->host, + fio->page->index, fio)))__submit_merged_bio (io); alloc_new:if (io->bio ==NULL ) {
fs/f2fs/data.c void f2fs_submit_page_write(struct f2fs_io_info *fio) · line 793
Quoted 7 lines
fs/f2fs/data.c +3
-0@@ -748,6 +787,9 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio) goto skip; } io->bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (io->bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO ); io->fio = *fio;
Likewise.
- Eric
PATCH v6 9/9 ext4: add inline encryption support
In reply to Satya TangiralaWire up ext4 to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:
- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.
Note: submit_bh_wbc() in fs/buffer.c also needed to be patched for this part, since ext4 sometimes uses ll_rw_block() on file data.
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
--- fs/buffer.c | 2 ++ fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 4 ++-- fs/ext4/page-io.c | 6 ++++-- fs/ext4/readpage.c | 11 ++++++++--- fs/ext4/super.c | 13 +++++++++++++ 6 files changed, 30 insertions(+), 7 deletions(-)
fs/buffer.c +2 -0
@@ -3108,6 +3108,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,*/ bio =bio_alloc (GFP_NOIO ,1 ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); + bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_write_hint = write_hint;
fs/ext4/ext4.h +1 -0
@@ -1153,6 +1153,7 @@ struct ext4_inode_info {#define EXT4_MOUNT_JOURNAL_CHECKSUM 0x800000/* Journal checksums */ #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT 0x1000000/* Journal Async Commit */ #define EXT4_MOUNT_WARN_ON_ERROR 0x2000000/* Trigger WARN_ON on error */ +#define EXT4_MOUNT_INLINECRYPT 0x4000000/* Inline encryption support */ #define EXT4_MOUNT_DELALLOC 0x8000000/* Delalloc support */ #define EXT4_MOUNT_DATA_ERR_ABORT 0x10000000/* Abort on file data write */ #define EXT4_MOUNT_BLOCK_VALIDITY 0x20000000/* Block validity checking */
fs/ext4/inode.c +2 -2
@@ -1090,7 +1090,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len, }if (unlikely (err)) {page_zero_new_buffers (page, from, to); - }else if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) { + }else if (fscrypt_inode_uses_fs_layer_crypto (inode)) {for (i =0 ; i < nr_wait; i++) {int err2;
@@ -3698,7 +3698,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,/* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate (bh)) goto unlock; -if (S_ISREG (inode->i_mode) &&IS_ENCRYPTED (inode)) { +if (fscrypt_inode_uses_fs_layer_crypto (inode)) {/* We expect the key to be set. */ BUG_ON (!fscrypt_has_encryption_key (inode)); WARN_ON_ONCE(fscrypt_decrypt_pagecache_blocks(
fs/ext4/page-io.c +4 -2
@@ -404,6 +404,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,* __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset(). */ bio =bio_alloc (GFP_NOIO ,BIO_MAX_PAGES ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_end_io = ext4_end_bio;
@@ -420,7 +421,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io, {int ret; -if (io->io_bio && bh->b_blocknr != io->io_next_block) { +if (io->io_bio && (bh->b_blocknr != io->io_next_block || + !fscrypt_mergeable_bio_bh (io->io_bio, bh))) { submit_and_retry:ext4_io_submit (io); }
@@ -508,7 +510,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,* (e.g. holes) to be unnecessarily encrypted, but this is rare and * can't happen in the common case of blocksize == PAGE_SIZE. */ -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode) && nr_to_submit) { +if (fscrypt_inode_uses_fs_layer_crypto (inode) && nr_to_submit) {gfp_t gfp_flags =GFP_NOFS ;unsigned int enc_bytes =round_up (len,i_blocksize (inode));
fs/ext4/readpage.c +8 -3
@@ -183,7 +183,7 @@ static struct bio_post_read_ctx *get_bio_post_read_ctx(struct inode *inode,unsigned int post_read_steps =0 ;struct bio_post_read_ctx *ctx =NULL ; -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (ext4_need_verity (inode, first_idx))
@@ -220,6 +220,7 @@ int ext4_mpage_readpages(struct address_space *mapping,const unsigned blkbits = inode->i_blkbits;const unsigned blocks_per_page =PAGE_SIZE >> blkbits;const unsigned blocksize =1 << blkbits; +sector_t next_block;sector_t block_in_file;sector_t last_block;sector_t last_block_in_file;
@@ -252,7 +253,8 @@ int ext4_mpage_readpages(struct address_space *mapping,if (page_has_buffers (page)) goto confused; - block_in_file = (sector_t )page->index << (PAGE_SHIFT - blkbits); + block_in_file = next_block = + (sector_t )page->index << (PAGE_SHIFT - blkbits); last_block = block_in_file + nr_pages * blocks_per_page; last_block_in_file = (ext4_readpage_limit (inode) + blocksize -1 ) >> blkbits;
@@ -352,7 +354,8 @@ int ext4_mpage_readpages(struct address_space *mapping,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && (last_block_in_bio != blocks[0 ] -1 )) { +if (bio && (last_block_in_bio != blocks[0 ] -1 || + !fscrypt_mergeable_bio (bio, inode, next_block))) { submit_and_realloc:submit_bio (bio); bio =NULL ;
@@ -366,6 +369,8 @@ int ext4_mpage_readpages(struct address_space *mapping,*/ bio =bio_alloc (GFP_KERNEL ,min_t (int, nr_pages,BIO_MAX_PAGES )); +fscrypt_set_bio_crypt_ctx (bio, inode, next_block, +GFP_KERNEL ); ctx =get_bio_post_read_ctx (inode, bio, page->index);if (IS_ERR (ctx)) {bio_put (bio);
fs/ext4/super.c +13 -0
@@ -1357,6 +1357,11 @@ static void ext4_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (ext4_lblk_t); } +static bool ext4_inline_crypt_enabled (struct super_block *sb) +{ +return test_opt (sb,INLINECRYPT ); +} +static const struct fscrypt_operations ext4_cryptops = { .key_prefix ="ext4:" , .get_context = ext4_get_context,
@@ -1366,6 +1371,7 @@ static const struct fscrypt_operations ext4_cryptops = { .max_namelen = EXT4_NAME_LEN, .has_stable_inodes = ext4_has_stable_inodes, .get_ino_and_lblk_bits = ext4_get_ino_and_lblk_bits, + .inline_crypt_enabled = ext4_inline_crypt_enabled, };#endif
@@ -1460,6 +1466,7 @@ enum { Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit, Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,
@@ -1556,6 +1563,7 @@ static const match_table_t tokens = { {Opt_noinit_itable, "noinit_itable"}, {Opt_max_dir_size_kb, "max_dir_size_kb=%u"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_nombcache, "nombcache"}, {Opt_nombcache, "no_mbcache"}, /* for backward compatibility */ {Opt_removed, "check=none"}, /* mount option from ext2/3 */
@@ -1767,6 +1775,11 @@ static const struct mount_opts { {Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT}, {Opt_max_dir_size_kb, 0, MOPT_GTE0}, {Opt_test_dummy_encryption, 0, MOPT_GTE0}, +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET}, +#else + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT}, +#endif {Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET}, {Opt_err, 0, 0} };
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Eric Biggers
maintainerRe: [PATCH v6 9/9] ext4: add inline encryption support
[+Cc linux-ext4]
Quoted 12 lines · Satya Tangirala
Wire up ext4 to support inline encryption via the helper functions which
fs/crypto/now provides. This includes:- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the
bio_crypt_ctxon bios that will be submitted to an inline-encrypted file.Note:
submit_bh_wbc()infs/buffer.calso needed to be patched for this part, since ext4 sometimes usesll_rw_block()on file data.
Quoted 7 lines
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
Signed-off-by Eric Biggers <[email protected]>Signed-off-by Satya Tangirala <[email protected]>
In the next version can you please add linux-ext4 to Cc for the series?
Quoted 14 lines
--- fs/buffer.c | 2 ++ fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 4 ++-- fs/ext4/page-io.c | 6 ++++-- fs/ext4/readpage.c | 11 ++++++++--- fs/ext4/super.c | 13 +++++++++++++ 6 files changed, 30 insertions(+), 7 deletions(-)fs/buffer.c +0
-0
fs/buffer.c static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh, · line 3115
Quoted 8 lines
fs/buffer.c +2
-0@@ -3108,6 +3108,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,*/ bio =bio_alloc (GFP_NOIO ,1 ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); + bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_write_hint = write_hint;
In 5.5, there was some decryption code added to fs/buffer.c in order to support ext4 encryption with blocksize < PAGE_SIZE:
static void end_buffer_async_read_io(struct buffer_head *bh, int uptodate)
{
/* Decrypt if needed */
if (uptodate && IS_ENABLED(CONFIG_FS_ENCRYPTION) &&
IS_ENCRYPTED(bh->b_page->mapping->host) &&
S_ISREG(bh->b_page->mapping->host->i_mode)) {
...This needs to be updated to use fscrypt_inode_uses_fs_layer_crypto() instead, so that the filesystem-layer decryption is not done when the file is already using inline decryption.
Quoted 157 lines
fs/ext4/ext4.h +1
-0@@ -1153,6 +1153,7 @@ struct ext4_inode_info {#define EXT4_MOUNT_JOURNAL_CHECKSUM 0x800000/* Journal checksums */ #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT 0x1000000/* Journal Async Commit */ #define EXT4_MOUNT_WARN_ON_ERROR 0x2000000/* Trigger WARN_ON on error */ +#define EXT4_MOUNT_INLINECRYPT 0x4000000/* Inline encryption support */ #define EXT4_MOUNT_DELALLOC 0x8000000/* Delalloc support */ #define EXT4_MOUNT_DATA_ERR_ABORT 0x10000000/* Abort on file data write */ #define EXT4_MOUNT_BLOCK_VALIDITY 0x20000000/* Block validity checking */ fs/ext4/inode.c +2
-2@@ -1090,7 +1090,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len, }if (unlikely (err)) {page_zero_new_buffers (page, from, to); - }else if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) { + }else if (fscrypt_inode_uses_fs_layer_crypto (inode)) {for (i =0 ; i < nr_wait; i++) {int err2;@@ -3698,7 +3698,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,/* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate (bh)) goto unlock; -if (S_ISREG (inode->i_mode) &&IS_ENCRYPTED (inode)) { +if (fscrypt_inode_uses_fs_layer_crypto (inode)) {/* We expect the key to be set. */ BUG_ON (!fscrypt_has_encryption_key (inode)); WARN_ON_ONCE(fscrypt_decrypt_pagecache_blocks(fs/ext4/page-io.c +4
-2@@ -404,6 +404,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,* __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset(). */ bio =bio_alloc (GFP_NOIO ,BIO_MAX_PAGES ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_end_io = ext4_end_bio;@@ -420,7 +421,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io, {int ret; -if (io->io_bio && bh->b_blocknr != io->io_next_block) { +if (io->io_bio && (bh->b_blocknr != io->io_next_block || + !fscrypt_mergeable_bio_bh (io->io_bio, bh))) { submit_and_retry:ext4_io_submit (io); }@@ -508,7 +510,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,* (e.g. holes) to be unnecessarily encrypted, but this is rare and * can't happen in the common case of blocksize == PAGE_SIZE. */ -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode) && nr_to_submit) { +if (fscrypt_inode_uses_fs_layer_crypto (inode) && nr_to_submit) {gfp_t gfp_flags =GFP_NOFS ;unsigned int enc_bytes =round_up (len,i_blocksize (inode));fs/ext4/readpage.c +8
-3@@ -183,7 +183,7 @@ static struct bio_post_read_ctx *get_bio_post_read_ctx(struct inode *inode,unsigned int post_read_steps =0 ;struct bio_post_read_ctx *ctx =NULL ; -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (ext4_need_verity (inode, first_idx))@@ -220,6 +220,7 @@ int ext4_mpage_readpages(struct address_space *mapping,const unsigned blkbits = inode->i_blkbits;const unsigned blocks_per_page =PAGE_SIZE >> blkbits;const unsigned blocksize =1 << blkbits; +sector_t next_block;sector_t block_in_file;sector_t last_block;sector_t last_block_in_file;@@ -252,7 +253,8 @@ int ext4_mpage_readpages(struct address_space *mapping,if (page_has_buffers (page)) goto confused; - block_in_file = (sector_t )page->index << (PAGE_SHIFT - blkbits); + block_in_file = next_block = + (sector_t )page->index << (PAGE_SHIFT - blkbits); last_block = block_in_file + nr_pages * blocks_per_page; last_block_in_file = (ext4_readpage_limit (inode) + blocksize -1 ) >> blkbits;@@ -352,7 +354,8 @@ int ext4_mpage_readpages(struct address_space *mapping,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && (last_block_in_bio != blocks[0 ] -1 )) { +if (bio && (last_block_in_bio != blocks[0 ] -1 || + !fscrypt_mergeable_bio (bio, inode, next_block))) { submit_and_realloc:submit_bio (bio); bio =NULL ;@@ -366,6 +369,8 @@ int ext4_mpage_readpages(struct address_space *mapping,*/ bio =bio_alloc (GFP_KERNEL ,min_t (int, nr_pages,BIO_MAX_PAGES )); +fscrypt_set_bio_crypt_ctx (bio, inode, next_block, +GFP_KERNEL ); ctx =get_bio_post_read_ctx (inode, bio, page->index);if (IS_ERR (ctx)) {bio_put (bio);fs/ext4/super.c +8
-0@@ -1357,6 +1357,11 @@ static void ext4_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (ext4_lblk_t); } +static bool ext4_inline_crypt_enabled (struct super_block *sb) +{ +return test_opt (sb,INLINECRYPT ); +} +static const struct fscrypt_operations ext4_cryptops = { .key_prefix ="ext4:" , .get_context = ext4_get_context,@@ -1366,6 +1371,7 @@ static const struct fscrypt_operations ext4_cryptops = { .max_namelen = EXT4_NAME_LEN, .has_stable_inodes = ext4_has_stable_inodes, .get_ino_and_lblk_bits = ext4_get_ino_and_lblk_bits, + .inline_crypt_enabled = ext4_inline_crypt_enabled, };#endif @@ -1460,6 +1466,7 @@ enum { Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit, Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,@@ -1556,6 +1563,7 @@ static const match_table_t tokens = { {Opt_noinit_itable, "noinit_itable"}, {Opt_max_dir_size_kb, "max_dir_size_kb=%u"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_nombcache, "nombcache"}, {Opt_nombcache, "no_mbcache"}, /* for backward compatibility */ {Opt_removed, "check=none"}, /* mount option from ext2/3 */@@ -1767,6 +1775,11 @@ static const struct mount_opts { {Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT},
fs/ext4/super.c static const struct mount_opts { · line 1785
Quoted 10 lines
fs/ext4/super.c +5
-0{Opt_max_dir_size_kb, 0, MOPT_GTE0}, {Opt_test_dummy_encryption, 0, MOPT_GTE0}, +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET}, +#else + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT}, +#endif {Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET}, {Opt_err, 0, 0} };
On v5 there was a discussion about using a flag like SB_INLINECRYPT to replace the ->inline_crypt_enabled() function. Was this considered?
- Eric
Re: [PATCH v6 9/9] ext4: add inline encryption support
I messed up while using git and managed to overwrite the author field in the patch - it should read "Eric Biggers <[email protected]>" instead. I'll fix that in the next patch version, sorry.
Quoted 18 lines · Eric Biggers
[+Cc linux-ext4]
Quoted 15 lines · Satya Tangirala
Wire up ext4 to support inline encryption via the helper functions which
fs/crypto/now provides. This includes:- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the
bio_crypt_ctxon bios that will be submitted to an inline-encrypted file.Note:
submit_bh_wbc()infs/buffer.calso needed to be patched for this part, since ext4 sometimes usesll_rw_block()on file data.- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
Quoted 7 lines
Quoted 4 lines
- Not doing filesystem-layer crypto on inline-encrypted files.
Signed-off-by Eric Biggers <[email protected]>Signed-off-by Satya Tangirala <[email protected]>In the next version can you please add linux-ext4 to Cc for the series?
Ok, I will.
Quoted 35 lines
Quoted 14 lines
--- fs/buffer.c | 2 ++ fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 4 ++-- fs/ext4/page-io.c | 6 ++++-- fs/ext4/readpage.c | 11 ++++++++--- fs/ext4/super.c | 13 +++++++++++++ 6 files changed, 30 insertions(+), 7 deletions(-)fs/buffer.c +0
-0fs/buffer.c static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh, · line 3115
Quoted 8 lines
fs/buffer.c +2
-0@@ -3108,6 +3108,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,*/ bio =bio_alloc (GFP_NOIO ,1 ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); + bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_write_hint = write_hint;In 5.5, there was some decryption code added to
fs/buffer.cin order to support ext4 encryption with blocksize <PAGE_SIZE:static void
end_buffer_async_read_io(struct buffer_head *bh, int uptodate){ /* Decrypt if needed */ if (uptodate && IS_ENABLED(CONFIG_FS_ENCRYPTION) && IS_ENCRYPTED(bh->b_page->mapping->host) && S_ISREG(bh->b_page->mapping->host->i_mode)) { ...
Quoted 4 lines
This needs to be updated to use
fscrypt_inode_uses_fs_layer_crypto()instead, so that the filesystem-layer decryption is not done when the file is already using inline decryption.
Will fix in the next version.
Quoted 162 lines
Quoted 162 lines
fs/ext4/ext4.h +1
-0@@ -1153,6 +1153,7 @@ struct ext4_inode_info {#define EXT4_MOUNT_JOURNAL_CHECKSUM 0x800000/* Journal checksums */ #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT 0x1000000/* Journal Async Commit */ #define EXT4_MOUNT_WARN_ON_ERROR 0x2000000/* Trigger WARN_ON on error */ +#define EXT4_MOUNT_INLINECRYPT 0x4000000/* Inline encryption support */ #define EXT4_MOUNT_DELALLOC 0x8000000/* Delalloc support */ #define EXT4_MOUNT_DATA_ERR_ABORT 0x10000000/* Abort on file data write */ #define EXT4_MOUNT_BLOCK_VALIDITY 0x20000000/* Block validity checking */ fs/ext4/inode.c +2
-2@@ -1090,7 +1090,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len, }if (unlikely (err)) {page_zero_new_buffers (page, from, to); - }else if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) { + }else if (fscrypt_inode_uses_fs_layer_crypto (inode)) {for (i =0 ; i < nr_wait; i++) {int err2;@@ -3698,7 +3698,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,/* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate (bh)) goto unlock; -if (S_ISREG (inode->i_mode) &&IS_ENCRYPTED (inode)) { +if (fscrypt_inode_uses_fs_layer_crypto (inode)) {/* We expect the key to be set. */ BUG_ON (!fscrypt_has_encryption_key (inode)); WARN_ON_ONCE(fscrypt_decrypt_pagecache_blocks(fs/ext4/page-io.c +4
-2@@ -404,6 +404,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,* __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset(). */ bio =bio_alloc (GFP_NOIO ,BIO_MAX_PAGES ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_end_io = ext4_end_bio;@@ -420,7 +421,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io, {int ret; -if (io->io_bio && bh->b_blocknr != io->io_next_block) { +if (io->io_bio && (bh->b_blocknr != io->io_next_block || + !fscrypt_mergeable_bio_bh (io->io_bio, bh))) { submit_and_retry:ext4_io_submit (io); }@@ -508,7 +510,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,* (e.g. holes) to be unnecessarily encrypted, but this is rare and * can't happen in the common case of blocksize == PAGE_SIZE. */ -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode) && nr_to_submit) { +if (fscrypt_inode_uses_fs_layer_crypto (inode) && nr_to_submit) {gfp_t gfp_flags =GFP_NOFS ;unsigned int enc_bytes =round_up (len,i_blocksize (inode));fs/ext4/readpage.c +8
-3@@ -183,7 +183,7 @@ static struct bio_post_read_ctx *get_bio_post_read_ctx(struct inode *inode,unsigned int post_read_steps =0 ;struct bio_post_read_ctx *ctx =NULL ; -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (ext4_need_verity (inode, first_idx))@@ -220,6 +220,7 @@ int ext4_mpage_readpages(struct address_space *mapping,const unsigned blkbits = inode->i_blkbits;const unsigned blocks_per_page =PAGE_SIZE >> blkbits;const unsigned blocksize =1 << blkbits; +sector_t next_block;sector_t block_in_file;sector_t last_block;sector_t last_block_in_file;@@ -252,7 +253,8 @@ int ext4_mpage_readpages(struct address_space *mapping,if (page_has_buffers (page)) goto confused; - block_in_file = (sector_t )page->index << (PAGE_SHIFT - blkbits); + block_in_file = next_block = + (sector_t )page->index << (PAGE_SHIFT - blkbits); last_block = block_in_file + nr_pages * blocks_per_page; last_block_in_file = (ext4_readpage_limit (inode) + blocksize -1 ) >> blkbits;@@ -352,7 +354,8 @@ int ext4_mpage_readpages(struct address_space *mapping,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && (last_block_in_bio != blocks[0 ] -1 )) { +if (bio && (last_block_in_bio != blocks[0 ] -1 || + !fscrypt_mergeable_bio (bio, inode, next_block))) { submit_and_realloc:submit_bio (bio); bio =NULL ;@@ -366,6 +369,8 @@ int ext4_mpage_readpages(struct address_space *mapping,*/ bio =bio_alloc (GFP_KERNEL ,min_t (int, nr_pages,BIO_MAX_PAGES )); +fscrypt_set_bio_crypt_ctx (bio, inode, next_block, +GFP_KERNEL ); ctx =get_bio_post_read_ctx (inode, bio, page->index);if (IS_ERR (ctx)) {bio_put (bio);fs/ext4/super.c +11
-0@@ -1357,6 +1357,11 @@ static void ext4_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (ext4_lblk_t); } +static bool ext4_inline_crypt_enabled (struct super_block *sb) +{ +return test_opt (sb,INLINECRYPT ); +} +static const struct fscrypt_operations ext4_cryptops = { .key_prefix ="ext4:" , .get_context = ext4_get_context,@@ -1366,6 +1371,7 @@ static const struct fscrypt_operations ext4_cryptops = { .max_namelen = EXT4_NAME_LEN, .has_stable_inodes = ext4_has_stable_inodes, .get_ino_and_lblk_bits = ext4_get_ino_and_lblk_bits, + .inline_crypt_enabled = ext4_inline_crypt_enabled, };#endif @@ -1460,6 +1466,7 @@ enum { Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit, Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,@@ -1556,6 +1563,7 @@ static const match_table_t tokens = { {Opt_noinit_itable, "noinit_itable"}, {Opt_max_dir_size_kb, "max_dir_size_kb=%u"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_nombcache, "nombcache"}, {Opt_nombcache, "no_mbcache"}, /* for backward compatibility */ {Opt_removed, "check=none"}, /* mount option from ext2/3 */@@ -1767,6 +1775,11 @@ static const struct mount_opts { {Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT}, {Opt_max_dir_size_kb, 0, MOPT_GTE0}, {Opt_test_dummy_encryption, 0, MOPT_GTE0}, +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET}, +#else
Quoted 9 lines
fs/ext4/super.c static const struct mount_opts { · line 1785
Quoted 5 lines
fs/ext4/super.c +2
-0+ {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT}, +#endif {Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET}, {Opt_err, 0, 0} };On v5 there was a discussion about using a flag like
SB_INLINECRYPTto replace the ->inline_crypt_enabled() function. Was this considered?
Right, I'll add this change to the next version too.
Quoted 1 line
- Eric
Eric Biggers
maintainerRe: [PATCH v6 9/9] ext4: add inline encryption support
In reply to Satya TangiralaQuoted 18 lines · Satya Tangirala
fs/ext4/super.c +2
-0@@ -1460,6 +1466,7 @@ enum { Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit, Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,@@ -1556,6 +1563,7 @@ static const match_table_t tokens = { {Opt_noinit_itable, "noinit_itable"}, {Opt_max_dir_size_kb, "max_dir_size_kb=%u"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_nombcache, "nombcache"}, {Opt_nombcache, "no_mbcache"}, /* for backward compatibility */ {Opt_removed, "check=none"}, /* mount option from ext2/3 */@@ -1767,6 +1775,11 @@ static const struct mount_opts { {Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT},
fs/ext4/super.c static const struct mount_opts { · line 1785
Quoted 10 lines
fs/ext4/super.c +5
-0{Opt_max_dir_size_kb, 0, MOPT_GTE0}, {Opt_test_dummy_encryption, 0, MOPT_GTE0}, +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET}, +#else + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT}, +#endif {Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET}, {Opt_err, 0, 0} };
This mount option will need to be documented inDocumentation/admin-guide/ext4.rst for ext4 andDocumentation/filesystems/f2fs.txt for f2fs.
- Eric
I haven't been able to deep dive into the details, but the structure of this still makes me very unhappy.
Most of it is related to the software fallback again. Please split the fallback into a separate file, and also into a separate data structure. There is abslutely no need to have the overhead of the software only fields for the hardware case.
On the counter side I think all the core block layer code added should go into a single file instead of split into three with some odd layering.
Also what I don't understand is why this managed key-slots on a per-bio basis. Wou;dn't it make a whole lot more sense to manage them on a struct request basis once most of the merging has been performed?
Quoted 8 lines · Christoph Hellwig
I haven't been able to deep dive into the details, but the structure of this still makes me very unhappy.
Most of it is related to the software fallback again. Please split the fallback into a separate file, and also into a separate data structure. There is abslutely no need to have the overhead of the software only fields for the hardware case.
The fallback actually is in a separate file, and the software only fields are not allocated in the hardware case anymore, either - I should have made that clear(er) in the coverletter.
Quoted 4 lines
On the counter side I think all the core block layer code added should go into a single file instead of split into three with some odd layering.
Alright, I'll look into this. I still think that the keyslot manager should maybe go in a separate file because it does a specific, fairly self contained task and isn't just block layer code - it's the interface between the device drivers and any upper layer.
Quoted 3 lines
Also what I don't understand is why this managed key-slots on a per-bio basis. Wou;dn't it make a whole lot more sense to manage them on a struct request basis once most of the merging has been performed?
I don't immediately see an issue with making it work on a struct request basis. I'll look into this more carefully.
Thanks!
Satya
Hi Satya,
Quoted 3 lines · Satya Tangirala
The fallback actually is in a separate file, and the software only fields are not allocated in the hardware case anymore, either - I should have made that clear(er) in the coverletter.
I see this now, thanks. Either the changes weren't pushed to the fscrypt report by the time I saw you mail, or I managed to look at a stale local copy.
Quoted 4 lines
Alright, I'll look into this. I still think that the keyslot manager should maybe go in a separate file because it does a specific, fairly self contained task and isn't just block layer code - it's the interface between the device drivers and any upper layer.
So are various other functions in the code like bio_crypt_clone or bio_crypt_should_process. Also the keyslot_* naming is way to generic, it really needs a blk_ or blk_crypto_ prefix.
Quoted 5 lines
Quoted 3 lines
Also what I don't understand is why this managed key-slots on a per-bio basis. Wou;dn't it make a whole lot more sense to manage them on a struct request basis once most of the merging has been performed?
I don't immediately see an issue with making it work on a struct request basis. I'll look into this more carefully.
I think that should end up being simpler and more efficient.
Quoted 20 lines · Christoph Hellwig
Hi Satya,
Quoted 3 lines · Satya Tangirala
The fallback actually is in a separate file, and the software only fields are not allocated in the hardware case anymore, either - I should have made that clear(er) in the coverletter.
I see this now, thanks. Either the changes weren't pushed to the fscrypt report by the time I saw you mail, or I managed to look at a stale local copy.
Quoted 4 lines
Alright, I'll look into this. I still think that the keyslot manager should maybe go in a separate file because it does a specific, fairly self contained task and isn't just block layer code - it's the interface between the device drivers and any upper layer.
So are various other functions in the code like
bio_crypt_cloneorbio_crypt_should_process. Also the keyslot_* naming is way to generic, it really needs ablk_orblk_crypto_prefix.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Also what I don't understand is why this managed key-slots on a per-bio basis. Wou;dn't it make a whole lot more sense to manage them on a struct request basis once most of the merging has been performed?
I don't immediately see an issue with making it work on a struct request basis. I'll look into this more carefully.
I think that should end up being simpler and more efficient.
So I tried reading through more of blk-mq and the IO schedulers to figure out how to do this. As far as I can tell, requests may be merged with each other until they're taken off the scheduler. So ideally, we'd program a keyslot for a request when it's taken off the scheduler, but this happens from within an atomic context. Otoh, programming a keyslot might cause the thread to sleep (in the event that all keyslots are in use by other in-flight requests). Unless I'm missing something, or you had some other different idea in mind, I think it's a lot easier to stick to letting blk-crypto program keyslots and manage them per-bio...
Quoted 9 lines · Satya Tangirala
So I tried reading through more of blk-mq and the IO schedulers to figure out how to do this. As far as I can tell, requests may be merged with each other until they're taken off the scheduler. So ideally, we'd program a keyslot for a request when it's taken off the scheduler, but this happens from within an atomic context. Otoh, programming a keyslot might cause the thread to sleep (in the event that all keyslots are in use by other in-flight requests). Unless I'm missing something, or you had some other different idea in mind, I think it's a lot easier to stick to letting blk-crypto program keyslots and manage them per-bio...
But as far as I understand from reading the code it only sleeps because it waits for another key slot to be released. Which is exactly like any other resource constraint in the storage device. In that case ->queue_rq returns BLK_STS_RESOURCE (or you do the equivalent in the blk-mq code) and the queue gets woken again once the resource is available.
Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Satya Tangirala
So I tried reading through more of blk-mq and the IO schedulers to figure out how to do this. As far as I can tell, requests may be merged with each other until they're taken off the scheduler. So ideally, we'd program a keyslot for a request when it's taken off the scheduler, but this happens from within an atomic context. Otoh, programming a keyslot might cause the thread to sleep (in the event that all keyslots are in use by other in-flight requests). Unless I'm missing something, or you had some other different idea in mind, I think it's a lot easier to stick to letting blk-crypto program keyslots and manage them per-bio...
Quoted 6 lines
But as far as I understand from reading the code it only sleeps because it waits for another key slot to be released. Which is exactly like any other resource constraint in the storage device. In that case ->queue_rq returns
BLK_STS_RESOURCE(or you do the equivalent in the blk-mq code) and the queue gets woken again once the resource is available.
Wouldn't that mean that all the other requests in the queue, even ones that don't even need any inline encryption, also don't get processed until the queue is woken up again? And if so, are we really ok with that?
As you said, we'd need the queue to wake up once a keyslot is available. It's possible that only some hardware queues and not others get blocked because of keyslot programming, so ideally, we could somehow make the correct hardware queue(s) wake up once a keyslot is freed. But the keyslot manager can't assume that it's actually blk-mq that's being used underneath, so if we want to get the keyslot manager to do something once a keyslot was freed, it would need some generic way to signal that to blk-mq. We can also just wait around for the queue to restart by itself after some time delay and try to program the keyslot again at that point, although I wouldn't want to do that because in the current design we know exactly when a keyslot is freed, and we don't need to rely on potentially inefficient guesswork about when we can successfully program a keyslot. Maybe we're alright with waking up all the queues rather than only the ones that really need it? But in any case, I don't know yet what the best way to solve this problem is.
We would also need to make changes to handle programming keyslots in some of the other make_request_fns besides blk_mq_make_request too (wherever relevant, at least) which adds more complexity. Overall, it seems to me like trying to manage programming of keyslots on a per-request basis is maybe more code than what we have now, and I'm not sure what we're really buying by doing it (other than perhaps the performance benefit of having to get fewer refcounts on a variable and fewer comparisions of cryptographic keys).
Also I forgot to mention this in my previous mail, but there may be some drivers/devices whose keyslots cannot be programmed from an atomic context, so this approach which might make things difficult in those situations (the UFS v2.1 spec, which I followed while implementing support for inline crypto for UFS, does not care whether we're in an atomic context or not, but there might be specifications for other drivers, or even some particular UFS inline encryption hardware that do).
So unless you have strong objections, I'd want to continue programming keyslots per-bio for the above reasons.
Quoted 3 lines · Satya Tangirala
Wouldn't that mean that all the other requests in the queue, even ones that don't even need any inline encryption, also don't get processed until the queue is woken up again?
For the basic implementation yes.
Quoted 1 line
And if so, are we really ok with that?
That depends on the use cases. With the fscrypt setup are we still going to see unencrypted I/O to the device as well? If so we'll need to refine the setup and only queue up unencrypted requests. But I'd still try to dumb version first and then refine it.
Quoted 6 lines
As you said, we'd need the queue to wake up once a keyslot is available. It's possible that only some hardware queues and not others get blocked because of keyslot programming, so ideally, we could somehow make the correct hardware queue(s) wake up once a keyslot is freed. But the keyslot manager can't assume that it's actually blk-mq that's being used underneath,
Why? The legacy requet code is long gone.
Quoted 7 lines
Also I forgot to mention this in my previous mail, but there may be some
drivers/deviceswhose keyslots cannot be programmed from an atomic context, so this approach which might make things difficult in those situations (the UFS v2.1 spec, which I followed while implementing support for inline crypto for UFS, does not care whether we're in an atomic context or not, but there might be specifications for other drivers, or even some particular UFS inline encryption hardware that do).
We have an option to never call ->queue_rq from atomic context (BLK_MQ_F_BLOCKING). But do you know of existing hardware that behaves like this or is it just hypothetical?
Quoted 2 lines
So unless you have strong objections, I'd want to continue programming keyslots per-bio for the above reasons.
I'm pretty sure from looking at the code that doing inline encryption at the bio level is the wrong approach. That isn't supposed to end the discussion, but especially things like waking up after a keyslot becomes available fits much better into the request layer resource model that is built around queuing limitations, and not the make_request model that assumes the driver can always queue.
Eric Biggers
maintainerQuoted 5 lines · Christoph Hellwig
Quoted 3 lines · Satya Tangirala
Wouldn't that mean that all the other requests in the queue, even ones that don't even need any inline encryption, also don't get processed until the queue is woken up again?
Quoted 8 lines
For the basic implementation yes.
Quoted 1 line
And if so, are we really ok with that?
That depends on the use cases. With the fscrypt setup are we still going to see unencrypted I/O to the device as well? If so we'll need to refine the setup and only queue up unencrypted requests. But I'd still try to dumb version first and then refine it.
Definitely, for several reasons:
- Not all files on the filesystem are necessarily encrypted.
- Filesystem metadata is not encrypted (except for filenames, but those don't use inline encryption).
- Encryption isn't necessarily being used on all partitions on the disk.
It's also not just about unencrypted vs. encrypted, since just because someone is waiting for one keyslot doesn't mean we should pause all encrypted I/O to the device for all keyslots.
Quoted 18 lines
Quoted 7 lines
As you said, we'd need the queue to wake up once a keyslot is available. It's possible that only some hardware queues and not others get blocked because of keyslot programming, so ideally, we could somehow make the correct hardware queue(s) wake up once a keyslot is freed. But the keyslot manager can't assume that it's actually blk-mq that's being used underneath,
Why? The legacy requet code is long gone.
Quoted 7 lines
Also I forgot to mention this in my previous mail, but there may be some
drivers/deviceswhose keyslots cannot be programmed from an atomic context, so this approach which might make things difficult in those situations (the UFS v2.1 spec, which I followed while implementing support for inline crypto for UFS, does not care whether we're in an atomic context or not, but there might be specifications for other drivers, or even some particular UFS inline encryption hardware that do).
Quoted 3 lines
We have an option to never call ->queue_rq from atomic context (
BLK_MQ_F_BLOCKING). But do you know of existing hardware that behaves like this or is it just hypothetical?
Maybe -- check the Qualcomm ICE (Inline Crypto Engine) driver I posted at
https://lkml.kernel.org/linux-block/[email protected]/.
The hardware requires vendor-specific SMC calls to program keys, rather than the UFS standard way. It's currently blocking, since the code to make the SMC calls in drivers/firmware/qcom_scm*.c uses GFP_KERNEL and mutex_lock().
I'll test whether it can work in atomic context by using GFP_ATOMIC and qcom_scm_call_atomic() instead. (Adding a spinlock might be needed too.)
- Eric
Eric Biggers
maintainerQuoted 48 lines · Eric Biggers
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Satya Tangirala
Wouldn't that mean that all the other requests in the queue, even ones that don't even need any inline encryption, also don't get processed until the queue is woken up again?
Quoted 8 lines
For the basic implementation yes.
Quoted 1 line
And if so, are we really ok with that?
That depends on the use cases. With the fscrypt setup are we still going to see unencrypted I/O to the device as well? If so we'll need to refine the setup and only queue up unencrypted requests. But I'd still try to dumb version first and then refine it.
Definitely, for several reasons:
- Not all files on the filesystem are necessarily encrypted.
- Filesystem metadata is not encrypted (except for filenames, but those don't use inline encryption).
- Encryption isn't necessarily being used on all partitions on the disk.It's also not just about unencrypted vs. encrypted, since just because someone is waiting for one keyslot doesn't mean we should pause all encrypted I/O to the device for all keyslots.
Quoted 18 lines
Quoted 7 lines
As you said, we'd need the queue to wake up once a keyslot is available. It's possible that only some hardware queues and not others get blocked because of keyslot programming, so ideally, we could somehow make the correct hardware queue(s) wake up once a keyslot is freed. But the keyslot manager can't assume that it's actually blk-mq that's being used underneath,
Why? The legacy requet code is long gone.
Quoted 7 lines
Also I forgot to mention this in my previous mail, but there may be some
drivers/deviceswhose keyslots cannot be programmed from an atomic context, so this approach which might make things difficult in those situations (the UFS v2.1 spec, which I followed while implementing support for inline crypto for UFS, does not care whether we're in an atomic context or not, but there might be specifications for other drivers, or even some particular UFS inline encryption hardware that do).Quoted 3 lines
We have an option to never call ->queue_rq from atomic context (
BLK_MQ_F_BLOCKING). But do you know of existing hardware that behaves like this or is it just hypothetical?
Quoted 9 lines
Maybe -- check the Qualcomm ICE (Inline Crypto Engine) driver I posted at
https://lkml.kernel.org/linux-block/[email protected]/.
The hardware requires vendor-specific SMC calls to program keys, rather than the UFS standard way. It's currently blocking, since the code to make the SMC calls indrivers/firmware/qcom_scm*.cusesGFP_KERNELandmutex_lock().I'll test whether it can work in atomic context by using
GFP_ATOMICandqcom_scm_call_atomic()instead. (Adding a spinlock might be needed too.)
The vendor-specific SMC calls do seem to work in atomic context, at least on SDA845. However, in ufshcd_program_key(), the calls to pm_runtime_get_sync() and ufshcd_hold() can also sleep.
I think we can move the pm_runtime_get_sync() to ufshcd_crypto_keyslot_evict(), since the block layer already ensures the device is not runtime-suspended while requests are being processed (see blk_queue_enter()). I.e., keyslots can be evicted independently of any bio, but that's not the case for programming them.
That still leaves ufshcd_hold(), which is still needed to ungate the UFS clocks. It does accept an 'async' argument, which is used by ufshcd_queuecommand() to schedule work to ungate the clocks and return SCSI_MLQUEUE_HOST_BUSY.
So in blk_mq_dispatch_rq_list(), we could potentially try to acquire the keyslot, and if it can't be done because either none are available or because something else needs to be waited for, we can put the request back on the dispatch list -- similar to how failure to get a driver tag is handled.
However, if I understand correctly, that would mean that all requests to the same hardware queue would be blocked whenever someone is waiting for a keyslot -- even unencrypted requests and requests for unrelated keyslots.
It's possible that would still be fine for the Android use case, as vendors tend to add enough keyslots to work with Android, provided that they choose the fscrypt format that uses one key per encryption policy rather than one key per file. I.e., it might be the case that no one waits for keyslots in practice anyway. But, it seems it would be undesirable for a general Linux kernel framework, which could potentially be used with per-file keys or with hardware that only has a *very* small number of keyslots.
Another option would be to allocate the keyslot in blk_mq_get_request(), where sleeping is still allowed, but some merging was already done.
- Eric
Quoted 8 lines · Eric Biggers
The vendor-specific SMC calls do seem to work in atomic context, at least on SDA845. However, in
ufshcd_program_key(), the calls topm_runtime_get_sync()andufshcd_hold()can also sleep.I think we can move the
pm_runtime_get_sync()toufshcd_crypto_keyslot_evict(), since the block layer already ensures the device is not runtime-suspended while requests are being processed (seeblk_queue_enter()). I.e., keyslots can be evicted independently of any bio, but that's not the case for programming them.
Yes.
Quoted 8 lines
That still leaves
ufshcd_hold(), which is still needed to ungate the UFS clocks. It does accept an 'async' argument, which is used byufshcd_queuecommand()to schedule work to ungate the clocks and returnSCSI_MLQUEUE_HOST_BUSY.So in
blk_mq_dispatch_rq_list(), we could potentially try to acquire the keyslot, and if it can't be done because either none are available or because something else needs to be waited for, we can put the request back on the dispatch list -- similar to how failure to get a driver tag is handled.
Yes, that is what I had in mind.
Quoted 3 lines
However, if I understand correctly, that would mean that all requests to the same hardware queue would be blocked whenever someone is waiting for a keyslot -- even unencrypted requests and requests for unrelated keyslots.
At least for an initial dumb implementation, yes. But if we care enough we can improve the code to check for the encrypted flag and only put back encrypted flags in that case.
Quoted 10 lines
It's possible that would still be fine for the Android use case, as vendors tend to add enough keyslots to work with Android, provided that they choose the fscrypt format that uses one key per encryption policy rather than one key per file. I.e., it might be the case that no one waits for keyslots in practice anyway. But, it seems it would be undesirable for a general Linux kernel framework, which could potentially be used with per-file keys or with hardware that only has a *very* small number of keyslots.
Another option would be to allocate the keyslot in
blk_mq_get_request(), where sleeping is still allowed, but some merging was already done.
That is another good idea. In blk_mq_get_request we acquire other resources like the tag, so this would be a very logical places to acquire the key slots. We can should also be able to still merge into the request while it is waiting.
Hi Christoph,
I sent out v7 of the patch series. It's at
https://lore.kernel.org/linux-fscrypt/[email protected]/T/#t
It manages keyslots on a per-request basis now - I made it get keyslots in blk_mq_get_request, because that way I wouldn't have to worry about programming keys in an atomic context. What do you think of the new approach?
Quoted 46 lines · Christoph Hellwig
Quoted 8 lines · Eric Biggers
The vendor-specific SMC calls do seem to work in atomic context, at least on SDA845. However, in
ufshcd_program_key(), the calls topm_runtime_get_sync()andufshcd_hold()can also sleep.I think we can move the
pm_runtime_get_sync()toufshcd_crypto_keyslot_evict(), since the block layer already ensures the device is not runtime-suspended while requests are being processed (seeblk_queue_enter()). I.e., keyslots can be evicted independently of any bio, but that's not the case for programming them.Yes.
Quoted 8 lines
That still leaves
ufshcd_hold(), which is still needed to ungate the UFS clocks. It does accept an 'async' argument, which is used byufshcd_queuecommand()to schedule work to ungate the clocks and returnSCSI_MLQUEUE_HOST_BUSY.So in
blk_mq_dispatch_rq_list(), we could potentially try to acquire the keyslot, and if it can't be done because either none are available or because something else needs to be waited for, we can put the request back on the dispatch list -- similar to how failure to get a driver tag is handled.Yes, that is what I had in mind.
Quoted 3 lines
However, if I understand correctly, that would mean that all requests to the same hardware queue would be blocked whenever someone is waiting for a keyslot -- even unencrypted requests and requests for unrelated keyslots.
At least for an initial dumb implementation, yes. But if we care enough we can improve the code to check for the encrypted flag and only put back encrypted flags in that case.
Quoted 10 lines
It's possible that would still be fine for the Android use case, as vendors tend to add enough keyslots to work with Android, provided that they choose the fscrypt format that uses one key per encryption policy rather than one key per file. I.e., it might be the case that no one waits for keyslots in practice anyway. But, it seems it would be undesirable for a general Linux kernel framework, which could potentially be used with per-file keys or with hardware that only has a *very* small number of keyslots.
Another option would be to allocate the keyslot in
blk_mq_get_request(), where sleeping is still allowed, but some merging was already done.That is another good idea. In
blk_mq_get_requestwe acquire other resources like the tag, so this would be a very logical places to acquire the key slots. We can should also be able to still merge into the request while it is waiting.
Quoted 9 lines · Satya Tangirala
Hi Christoph,
I sent out v7 of the patch series. It's at
https://lore.kernel.org/linux-fscrypt/[email protected]/T/#tIt manages keyslots on a per-request basis now - I made it get keyslots in
blk_mq_get_request, because that way I wouldn't have to worry about programming keys in an atomic context. What do you think of the new approach?
I'll try to review it soon, thanks!