Inline Encryption Support
Subsystems (8)
v5 cover Inline Encryption Support
This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.
This patchset applies to fscrypt.git#master with the additional patchset "[PATCH 0/3] fscrypt: support for inline-encryption-optimized policies" applied. It can be retrieved from git at
https://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git/log/?h=inline-encryption-v5
Note however, that the patches for the block layer (i.e. patches 1, 2 and 3) can be applied independently.
Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device. Inline Encryption hardware has become increasingly common, and we want to support it in the kernel.
Inline Encryption hardware implementations often function around the concept of a limited number of "keyslots", which can hold an encryption context each. The storage device can be directed to en/decrypt any particular request with the encryption context stored in any particular keyslot.
Patch 1 introduces a Keyslot Manager to efficiently manage keyslots. The keyslot manager also functions as the interface that blk-crypto (introduced in Patch 3), will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.
Patch 2 introduces struct bio_crypt_ctx, and a ptr to one in struct bio, which allows struct bio to represent an encryption context that can be passed down the storage stack from the filesystem layer to the storage driver.
Patch 3 introduces blk-crypto. Blk-crypto delegates crypto operations to inline encryption hardware when available, and also contains a software fallback to the kernel crypto API. Blk-crypto also makes it possible for layered devices like device mapper to make use of inline encryption hardware. Given that blk-crypto works as a software fallback, we are considering removing file content en/decryption from fscrypt and simply using blk-crypto in a future patch. For more details on blk-crypto, refer to Documentation/block/inline-encryption.rst.
Patches 4-6 add support for inline encryption into the UFS driver according to the JEDEC UFS HCI v2.1 specification. Inline encryption support for other drivers (like eMMC) may be added in the same way - the device driver should set up a Keyslot Manager in the device's request_queue (refer to the UFS crypto additions in ufshcd-crypto.c and ufshcd.c for an example).
Patch 7 adds support to fscrypt - to use inline encryption with fscrypt, the filesystem must be mounted with '-o inlinecrypt' - when this option is specified, the contents of any AES-256-XTS encrypted file will be encrypted using blk-crypto.
Patches 8 and 9 add support to f2fs and ext4 respectively, so that we have a complete stack that can make use of inline encryption.
The patches were tested running kvm-xfstests, by specifying the introduced "inlinecrypt" mount option, so that encryption happens in blk-crypto.
There have been a few patch sets addressing Inline Encryption Support in the past. Briefly, this patch set differs from those as follows:
1) "crypto: qce: ice: Add support for Inline Crypto Engine" is specific to certain hardware, while our patch set's Inline Encryption support for UFS is implemented according to the JEDEC UFS specification.
2) "scsi: ufs: UFS Host Controller crypto changes" registers inline encryption support as a kernel crypto algorithm. Our patch views inline encryption as being fundamentally different from a generic crypto provider (in that inline encryption is tied to a device), and so does not use the kernel crypto API to represent inline encryption hardware.
3) "scsi: ufs: add real time/inline crypto support to UFS HCD" requires the device mapper to work - our patch does not.
Changes v4 => v5:
- The fscrypt patch has been separated into 2. The first adds support for the IV_INO_LBLK_64 policy (which was called INLINE_CRYPT_OPTIMIZED in past versions of this series). This policy is now purely an on disk format, and doesn't dictate whether blk-crypto is used for file content encryption or not. Instead, this is now decided based on the "inlinecrypt" mount option.
- Inline crypto key eviction is now handled by blk-crypto instead of fscrypt.
- More refactoring.
Changes v3 => v4:
- Fixed the issue with allocating crypto_skcipher in blk_crypto_keyslot_program.
- bio_crypto_alloc_ctx is now mempool backed.
- In f2fs, a bio's bi_crypt_context is now set up when the bio is allocated, rather than just before the bio is submitted - this fixes bugs in certain cases, like when an encrypted block is being moved without decryption.
- Lots of refactoring and cleanup of blk-crypto - thanks Eric!
Changes v2 => v3:
- Overhauled keyslot manager's get keyslot logic and optimized LRU.
- Block crypto en/decryption fallback now supports data unit sizes that divide the bvec length (instead of requiring each bvec's length to be the same as the data unit size).
- fscrypt master key is now keyed additionally by super_block and ci_ctfm != NULL.
- all references of "hw encryption" are replaced by inline encryption.
- address various other review comments from Eric.
Changes v1 => v2:
- Block layer and UFS changes are split into 3 patches each.
- We now only have a ptr to a struct bio_crypt_ctx in struct bio, instead of the struct itself.
- struct bio_crypt_ctx no longer has flags.
- blk-crypto now correctly handles the case when it fails to init (because of insufficient memory), but kernel continues to boot.
- ufshcd-crypto now works on big endian cpus.
- Many cleanups.
Eric Biggers (1):
ext4: add inline encryption support
Satya Tangirala (8):
block: Keyslot Manager for Inline Encryption
block: Add encryption context to struct bio
block: blk-crypto for Inline Encryption
scsi: ufs: UFS driver v2.1 spec crypto additions
scsi: ufs: UFS crypto API
scsi: ufs: Add inline encryption support to UFS
fscrypt: add inline encryption support
f2fs: add inline encryption support
Documentation/block/index.rst | 1 + Documentation/block/inline-encryption.rst | 183 +++++ block/Kconfig | 10 + block/Makefile | 2 + block/bio-crypt-ctx.c | 142 ++++ block/bio.c | 23 +- block/blk-core.c | 14 +- block/blk-crypto.c | 798 ++++++++++++++++++++++ block/blk-merge.c | 35 +- block/bounce.c | 15 +- block/keyslot-manager.c | 352 ++++++++++ drivers/md/dm.c | 15 +- drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd-crypto.c | 391 +++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 86 +++ drivers/scsi/ufs/ufshcd.c | 85 ++- drivers/scsi/ufs/ufshcd.h | 25 + drivers/scsi/ufs/ufshci.h | 67 +- fs/buffer.c | 3 + fs/crypto/Kconfig | 6 + fs/crypto/Makefile | 1 + fs/crypto/bio.c | 31 +- fs/crypto/fscrypt_private.h | 72 ++ fs/crypto/inline_crypt.c | 390 +++++++++++ fs/crypto/keyring.c | 2 + fs/crypto/keysetup.c | 18 +- fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 4 +- fs/ext4/page-io.c | 11 +- fs/ext4/readpage.c | 15 +- fs/ext4/super.c | 13 + fs/f2fs/data.c | 76 ++- fs/f2fs/f2fs.h | 3 + fs/f2fs/super.c | 20 + include/linux/bio-crypt-ctx.h | 226 ++++++ include/linux/bio.h | 1 + include/linux/blk-crypto.h | 62 ++ include/linux/blk_types.h | 6 + include/linux/blkdev.h | 6 + include/linux/fscrypt.h | 60 ++ include/linux/keyslot-manager.h | 98 +++ 42 files changed, 3318 insertions(+), 61 deletions(-) create mode 100644 Documentation/block/inline-encryption.rst create mode 100644 block/bio-crypt-ctx.c create mode 100644 block/blk-crypto.c create mode 100644 block/keyslot-manager.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h create mode 100644 fs/crypto/inline_crypt.c create mode 100644 include/linux/bio-crypt-ctx.h create mode 100644 include/linux/blk-crypto.h create mode 100644 include/linux/keyslot-manager.h
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
PATCH v5 1/9 block: Keyslot Manager for Inline Encryption
Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.
Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold an encryption context (we say that an encryption context can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the encryption context programmed into that associated keyslot. As keyslots are limited, and programming keys may be expensive in many implementations, and multiple requests may use exactly the same encryption contexts, we introduce a Keyslot Manager to efficiently manage keyslots. The keyslot manager also functions as the interface that upper layers will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.
--- block/Kconfig | 8 + block/Makefile | 1 + block/keyslot-manager.c | 352 ++++++++++++++++++++++++++++++++ include/linux/bio.h | 5 + include/linux/blkdev.h | 6 + include/linux/keyslot-manager.h | 98 +++++++++ 6 files changed, 470 insertions(+) create mode 100644 block/keyslot-manager.c create mode 100644 include/linux/keyslot-manager.h
block/Kconfig +8 -0
@@ -177,6 +177,14 @@ config BLK_SED_OPAL Enabling this option enables users to setup/unlock/lock Locking ranges for SED devices using the Opal protocol. +config BLK_INLINE_ENCRYPTION + bool "Enable inline encryption support in block layer" + help + Build the blk-crypto subsystem. + Enabling this lets the block layer handle encryption, + so users can take advantage of inline encryption + hardware if present. + menu "Partition Types" source "block/partitions/Kconfig"
block/Makefile +1 -0
@@ -36,3 +36,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o
block/keyslot-manager.c new file +352 -0
352 lines not on this page · raw message
include/linux/bio.h +5 -0
@@ -564,6 +564,11 @@ static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags) }#endif +enum blk_crypto_mode_num { +BLK_ENCRYPTION_MODE_INVALID =0 , +BLK_ENCRYPTION_MODE_AES_256_XTS =1 , +}; +/* * BIO list management for use by remapping drivers (e.g. DM or MD) and loop. *
include/linux/blkdev.h +6 -0
@@ -43,6 +43,7 @@ struct pr_ops;struct rq_qos ;struct blk_queue_stats ;struct blk_stat_callback ; +struct keyslot_manager ;#define BLKDEV_MIN_RQ 4#define BLKDEV_MAX_RQ 128/* Default maximum */
@@ -481,6 +482,11 @@ struct request_queue {unsigned int dma_pad_mask;unsigned int dma_alignment; +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +/* Inline crypto capabilities */ +struct keyslot_manager *ksm; +#endif +unsigned int rq_timeout;int poll_nsec;
include/linux/keyslot-manager.h new file +98 -0
98 lines not on this page · raw message
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 1/9] block: Keyslot Manager for Inline Encryption
Quoted 2 lines · Satya Tangirala
+/*
+ *keyslot-manager.c
Never mention the file name in top of the file comments, it is going to be out of data sooner than you'll get the series merged..
Quoted 1 line
+EXPORT_SYMBOL(keyslot_manager_create);
please use EXPORT_SYMBOL_GPL like all new low-level block layer exports.
Quoted 1 line
+EXPORT_SYMBOL(keyslot_manager_get_slot_for_key);
This is only used in block/bio-crypt-ctx.c, no need for an export.
block/keyslot-manager.c line 239
Quoted 8 lines
block/keyslot-manager.c new file +8
-0+void keyslot_manager_get_slot (struct keyslot_manager *ksm,unsigned int slot) +{ +if (WARN_ON (slot >= ksm->num_slots)) +return ; + +WARN_ON (atomic_inc_return (&ksm->slots[slot].slot_refs) <2 ); +} +EXPORT_SYMBOL (keyslot_manager_get_slot);
Same here.
Quoted 1 line
+EXPORT_SYMBOL(keyslot_manager_put_slot);
And here.
block/keyslot-manager.c line 291
Quoted 11 lines
block/keyslot-manager.c new file +11
-0+bool keyslot_manager_crypto_mode_supported (struct keyslot_manager *ksm, +enum blk_crypto_mode_num crypto_mode, +unsigned int data_unit_size) +{ +if (!ksm) +return false; +return ksm->ksm_ll_ops.crypto_mode_supported (ksm->ll_priv_data, + crypto_mode, + data_unit_size); +} +EXPORT_SYMBOL (keyslot_manager_crypto_mode_supported);
And here as well. In fact this one is so trivial that it is better open coded into the two callers.
block/keyslot-manager.c line 300
Quoted 8 lines
block/keyslot-manager.c new file +8
-0+bool keyslot_manager_rq_crypto_mode_supported (struct request_queue *q, +enum blk_crypto_mode_num crypto_mode, +unsigned int data_unit_size) +{ +return keyslot_manager_crypto_mode_supported (q->ksm, crypto_mode, + data_unit_size); +} +EXPORT_SYMBOL (keyslot_manager_rq_crypto_mode_supported);
And this one is entirely unused.
Quoted 1 line
+EXPORT_SYMBOL(keyslot_manager_evict_key);
No used outside blk-crypto.c either.
In fact given how small block/blk-crypto.c and block/keyslot-manager.c are, and given that all but two functions in the latter are only called from the former you should seriously consider merging the two files.
include/linux/bio.h static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags) · line 570
Quoted 12 lines
include/linux/bio.h +4
-0@@ -564,6 +564,11 @@ static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags) }#endif +enum blk_crypto_mode_num { +BLK_ENCRYPTION_MODE_INVALID =0 , +BLK_ENCRYPTION_MODE_AES_256_XTS =1 , +};
This one moves to include/linux/bio-crypt-ctx.h later in the series, please introduce it in the right place from the start. Also is there a need to explicitly assign code points here?
include/linux/bio.h static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags) · line 573
Quoted 3 lines
include/linux/bio.h +3
-0+extern struct keyslot_manager *keyslot_manager_create (unsigned int num_slots, +const struct keyslot_mgmt_ll_ops *ksm_ops, +void *ll_priv_data);
There is no nee for externs on function declarations in headers.
PATCH v5 2/9 block: Add encryption context to struct bio
In reply to Satya TangiralaWe must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the filesystem/fscrypt that knows about and manages encryption contexts. As such, when the filesystem layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.
We want to communicate the encryption context from the filesystem layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct bio_crypt_ctx to struct bio, which can represent an encryption context (note that we can't use the bi_private field in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate the bio_crypt_ctx and make the bio/request merging logic aware of the bio_crypt_ctx.
--- block/Makefile | 2 +- block/bio-crypt-ctx.c | 137 +++++++++++++++++++++ block/bio.c | 18 +-- block/blk-core.c | 3 + block/blk-merge.c | 35 +++++- block/bounce.c | 15 +-- drivers/md/dm.c | 15 ++- include/linux/bio-crypt-ctx.h | 219 ++++++++++++++++++++++++++++++++++ include/linux/bio.h | 6 +- include/linux/blk_types.h | 6 + 10 files changed, 426 insertions(+), 30 deletions(-) create mode 100644 block/bio-crypt-ctx.c create mode 100644 include/linux/bio-crypt-ctx.h
block/Makefile +1 -1
@@ -36,4 +36,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.o
block/bio-crypt-ctx.c new file +137 -0
137 lines not on this page · raw message
block/bio.c +10 -8
@@ -241,6 +241,7 @@ static void bio_free(struct bio *bio)struct bio_set *bs = bio->bi_pool;void *p; +bio_crypt_free_ctx (bio);bio_uninit (bio);if (bs) {
@@ -612,15 +613,15 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)__bio_clone_fast (b, bio); -if (bio_integrity (bio)) { -int ret; - - ret =bio_integrity_clone (b, bio, gfp_mask); +if (bio_crypt_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ; + } -if (ret <0 ) { -bio_put (b); -return NULL ; - } +if (bio_integrity (bio) && +bio_integrity_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ; }return b;
@@ -992,6 +993,7 @@ void bio_advance(struct bio *bio, unsigned bytes)if (bio_integrity (bio))bio_integrity_advance (bio, bytes); +bio_crypt_advance (bio, bytes);bio_advance_iter (bio, &bio->bi_iter, bytes); }EXPORT_SYMBOL (bio_advance);
block/blk-core.c +3 -0
@@ -1807,5 +1807,8 @@ int __init blk_dev_init(void) blk_debugfs_root =debugfs_create_dir ("block" ,NULL );#endif +if (bio_crypt_ctx_init () <0 ) +panic ("Failed to allocate mem for bio crypt ctxs\n" ); +return 0 ; }
block/blk-merge.c +31 -4
@@ -557,6 +557,9 @@ static inline int ll_new_hw_segment(struct request *req, struct bio *bio,if (blk_integrity_merge_bio (req->q, req, bio) == false) goto no_merge; +if (WARN_ON_ONCE (!bio_crypt_ctx_compatible (bio, req->bio))) + goto no_merge; +/* * This will form the start of a new hw segment. Bump both * counters.
@@ -711,8 +714,14 @@ static enum elv_merge blk_try_req_merge(struct request *req, {if (blk_discard_mergable (req))return ELEVATOR_DISCARD_MERGE ; -else if (blk_rq_pos (req) +blk_rq_sectors (req) ==blk_rq_pos (next)) +else if (blk_rq_pos (req) +blk_rq_sectors (req) ==blk_rq_pos (next)) { +if (!bio_crypt_ctx_back_mergeable (req->bio, +blk_rq_sectors (req), + next->bio)) { +return ELEVATOR_NO_MERGE ; + }return ELEVATOR_BACK_MERGE ; + }return ELEVATOR_NO_MERGE ; }
@@ -748,6 +757,9 @@ static struct request *attempt_merge(struct request_queue *q,if (req->ioprio != next->ioprio)return NULL ; +if (!bio_crypt_ctx_compatible (req->bio, next->bio)) +return NULL ; +/* * If we are allowed to merge, then append bio list * from next to rq and release next. merge_requests_fn
@@ -880,16 +892,31 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)if (rq->ioprio !=bio_prio (bio))return false; +/* Only merge if the crypt contexts are compatible */ +if (!bio_crypt_ctx_compatible (bio, rq->bio)) +return false; +return true; }enum elv_merge blk_try_merge (struct request *rq,struct bio *bio) { -if (blk_discard_mergable (rq)) +if (blk_discard_mergable (rq)) {return ELEVATOR_DISCARD_MERGE ; -else if (blk_rq_pos (rq) +blk_rq_sectors (rq) == bio->bi_iter.bi_sector) + }else if (blk_rq_pos (rq) +blk_rq_sectors (rq) == + bio->bi_iter.bi_sector) { +if (!bio_crypt_ctx_back_mergeable (rq->bio, +blk_rq_sectors (rq), bio)) { +return ELEVATOR_NO_MERGE ; + }return ELEVATOR_BACK_MERGE ; -else if (blk_rq_pos (rq) -bio_sectors (bio) == bio->bi_iter.bi_sector) + }else if (blk_rq_pos (rq) -bio_sectors (bio) == + bio->bi_iter.bi_sector) { +if (!bio_crypt_ctx_back_mergeable (bio, +bio_sectors (bio), rq->bio)) { +return ELEVATOR_NO_MERGE ; + }return ELEVATOR_FRONT_MERGE ; + }return ELEVATOR_NO_MERGE ; }
block/bounce.c +8 -7
@@ -267,14 +267,15 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,break ; } -if (bio_integrity (bio_src)) { -int ret; +if (bio_crypt_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ; + } - ret =bio_integrity_clone (bio, bio_src, gfp_mask); -if (ret <0 ) { -bio_put (bio); -return NULL ; - } +if (bio_integrity (bio_src) && +bio_integrity_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ; }bio_clone_blkg_association (bio, bio_src);
drivers/md/dm.c +10 -5
@@ -1322,12 +1322,15 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio, sector_t sector, unsigned len) {struct bio *clone = &tio->clone; +int ret;__bio_clone_fast (clone, bio); -if (bio_integrity (bio)) { -int r; + ret =bio_crypt_clone (clone, bio,GFP_NOIO ); +if (ret <0 ) +return ret; +if (bio_integrity (bio)) {if (unlikely (!dm_target_has_integrity (tio->ti->type) && !dm_target_passes_integrity (tio->ti->type))) { DMWARN("%s: the target %s doesn't support integrity data.",
@@ -1336,9 +1339,11 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,return -EIO ; } - r =bio_integrity_clone (clone, bio,GFP_NOIO ); -if (r <0 ) -return r; + ret =bio_integrity_clone (clone, bio,GFP_NOIO ); +if (ret <0 ) { +bio_crypt_free_ctx (clone); +return ret; + } }bio_advance (clone,to_bytes (sector - clone->bi_iter.bi_sector));
include/linux/bio-crypt-ctx.h new file +219 -0
219 lines not on this page · raw message
include/linux/bio.h +1 -5
@@ -8,6 +8,7 @@#include <linux/highmem.h> #include <linux/mempool.h> #include <linux/ioprio.h> +#include <linux/bio-crypt-ctx.h> #ifdef CONFIG_BLOCK /* struct bio, bio_vec and BIO_* flags are defined in blk_types.h */
@@ -564,11 +565,6 @@ static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags) }#endif -enum blk_crypto_mode_num { -BLK_ENCRYPTION_MODE_INVALID =0 , -BLK_ENCRYPTION_MODE_AES_256_XTS =1 , -}; -/* * BIO list management for use by remapping drivers (e.g. DM or MD) and loop. *
include/linux/blk_types.h +6 -0
@@ -18,6 +18,7 @@ struct block_device;struct io_context ;struct cgroup_subsys_state ;typedef void (bio_end_io_t ) (struct bio *); +struct bio_crypt_ctx ;/* * Block error status values. See block/blk-core:blk_errors for the details.
@@ -173,6 +174,11 @@ struct bio {u64 bi_iocost_cost;#endif #endif + +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *bi_crypt_context; +#endif +union {#if defined(CONFIG_BLK_DEV_INTEGRITY )struct bio_integrity_payload *bi_integrity;/* data integrity */
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 2/9] block: Add encryption context to struct bio
Quoted 1 line
+static int
num_prealloc_crypt_ctxs= 128;
Where does that magic number come from?
block/bio-crypt-ctx.c line 35
Quoted 5 lines
block/bio-crypt-ctx.c new file +5
-0+struct bio_crypt_ctx *bio_crypt_alloc_ctx (gfp_t gfp_mask) +{ +return mempool_alloc (bio_crypt_ctx_pool, gfp_mask); +} +EXPORT_SYMBOL (bio_crypt_alloc_ctx);
This isn't used by an modular code.
block/bio-crypt-ctx.c line 42
Quoted 6 lines
block/bio-crypt-ctx.c new file +6
-0+void bio_crypt_free_ctx (struct bio *bio) +{ +mempool_free (bio->bi_crypt_context, bio_crypt_ctx_pool); + bio->bi_crypt_context =NULL ; +} +EXPORT_SYMBOL (bio_crypt_free_ctx);
This one is called from modular code, but I think the usage in DM is bogus, as the caller of the function eventually does a bio_put, which ends up in bio_free and takes care of the freeing as well.
block/bio-crypt-ctx.c line 70
Quoted 8 lines
block/bio-crypt-ctx.c new file +8
-0+bool bio_crypt_should_process (struct bio *bio,struct request_queue *q) +{ +if (!bio_has_crypt_ctx (bio)) +return false; + +WARN_ON (!bio_crypt_has_keyslot (bio)); +return q->ksm == bio->bi_crypt_context->processing_ksm; +}
Passing a struct request here and also adding the ->bio != NULL check here would simplify the only caller in ufs a bit.
Quoted 8 lines
block/bio-crypt-ctx.c new file +8
-0+/* +* Checks that two bio crypt contexts are compatible - i.e. that +* they are mergeable except for data_unit_num continuity. +*/ +bool bio_crypt_ctx_compatible (struct bio *b_1,struct bio *b_2) +{ +struct bio_crypt_ctx *bc1 = b_1->bi_crypt_context; +struct bio_crypt_ctx *bc2 = b_2->bi_crypt_context;
block/bio-crypt-ctx.c line 90
Quoted 10 lines
block/bio-crypt-ctx.c new file +10
-0+ +if (bio_has_crypt_ctx (b_1) !=bio_has_crypt_ctx (b_2)) +return false; + +if (!bio_has_crypt_ctx (b_1)) +return true; + +return bc1->keyslot == bc2->keyslot && + bc1->data_unit_size_bits == bc2->data_unit_size_bits; +}
I think we'd normally call this bio_crypt_ctx_mergeable.
block/bio.c struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs) · line 624
Quoted 13 lines
block/bio.c +8
-4+if (bio_crypt_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ; + } -if (ret <0 ) { -bio_put (b); -return NULL ; - } +if (bio_integrity (bio) && +bio_integrity_clone (b, bio, gfp_mask) <0 ) { +bio_put (b); +return NULL ;
Pleae use a goto to merge the common error handling path
block/blk-merge.c static enum elv_merge blk_try_req_merge(struct request *req, · line 722
Quoted 5 lines
block/blk-merge.c +5
-0+if (!bio_crypt_ctx_back_mergeable (req->bio, +blk_rq_sectors (req), + next->bio)) { +return ELEVATOR_NO_MERGE ; + }
No neef for the braces. And pretty weird alignment, normal Linux style would be:
if (!bio_crypt_ctx_back_mergeable(req->bio, blk_rq_sectors(req), next->bio)) return ELEVATOR_NO_MERGE;
block/blk-merge.c bool blk_rq_merge_ok(struct request *rq, struct bio *bio) · line 918
Quoted 12 lines
block/blk-merge.c +10
-1+if (!bio_crypt_ctx_back_mergeable (rq->bio, +blk_rq_sectors (rq), bio)) { +return ELEVATOR_NO_MERGE ; + }return ELEVATOR_BACK_MERGE ; - else if (blk_rq_pos(rq) - bio_sectors(bio) == bio->bi_iter.bi_sector) + }else if (blk_rq_pos (rq) -bio_sectors (bio) == + bio->bi_iter.bi_sector) { +if (!bio_crypt_ctx_back_mergeable (bio, +bio_sectors (bio), rq->bio)) { +return ELEVATOR_NO_MERGE ; + }
Same for these two.
Quoted 11 lines
+++
b/block/bounce.cblock/bounce.c +4
-2@@ -267,14 +267,15 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,break ; } -if (bio_integrity (bio_src)) { -int ret; +if (bio_crypt_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ; + }
block/bounce.c static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask, · line 278
Quoted 10 lines
block/bounce.c +4
-5- ret =bio_integrity_clone (bio, bio_src, gfp_mask); -if (ret <0 ) { -bio_put (bio); -return NULL ; - } +if (bio_integrity (bio_src) && +bio_integrity_clone (bio, bio_src, gfp_mask) <0 ) { +bio_put (bio); +return NULL ;
Use a common error path with a goto, please.
include/linux/bio-crypt-ctx.h line 72
Quoted 6 lines
include/linux/bio-crypt-ctx.h new file +6
-0+static inline int bio_crypt_set_ctx (struct bio *bio, +const u8 *raw_key, +enum blk_crypto_mode_num crypto_mode, +u64 dun, +unsigned int dun_bits, +gfp_t gfp_mask)
Pleae just open code this in the only caller.
include/linux/bio-crypt-ctx.h line 78
Quoted 6 lines
include/linux/bio-crypt-ctx.h new file +6
-0+{ +struct bio_crypt_ctx *crypt_ctx; + + crypt_ctx =bio_crypt_alloc_ctx (gfp_mask); +if (!crypt_ctx) +return -ENOMEM ;
Also bio_crypt_alloc_ctx with a waiting mask will never return an error. Changing this function and its call chain to void returns will clean up a lot of code in this series.
include/linux/bio-crypt-ctx.h line 94
Quoted 4 lines
include/linux/bio-crypt-ctx.h new file +4
-0+static inline void bio_set_data_unit_num (struct bio *bio,u64 dun) +{ + bio->bi_crypt_context->data_unit_num = dun; +}
This function is never used and can be removed.
include/linux/bio-crypt-ctx.h line 107
Quoted 7 lines
include/linux/bio-crypt-ctx.h new file +7
-0+static inline void bio_crypt_set_keyslot (struct bio *bio, +unsigned int keyslot, +struct keyslot_manager *ksm) +{ + bio->bi_crypt_context->keyslot = keyslot; + bio->bi_crypt_context->processing_ksm = ksm; +}
Just adding these two lines to the only caller will be a lot cleaner.
include/linux/bio-crypt-ctx.h line 117
Quoted 4 lines
include/linux/bio-crypt-ctx.h new file +4
-0+static inline const u8 *bio_crypt_raw_key (struct bio *bio) +{ +return bio->bi_crypt_context->raw_key; +}
Can be inlined into the only caller.
include/linux/bio-crypt-ctx.h line 122
Quoted 5 lines
include/linux/bio-crypt-ctx.h new file +5
-0+ +static inline enum blk_crypto_mode_num bio_crypto_mode (struct bio *bio) +{ +return bio->bi_crypt_context->crypto_mode; +}
Same here.
include/linux/bio-crypt-ctx.h line 132
Quoted 4 lines
include/linux/bio-crypt-ctx.h new file +4
-0+static inline u64 bio_crypt_sw_data_unit_num (struct bio *bio) +{ +return bio->bi_crypt_context->sw_data_unit_num; +}
Same here.
PATCH v5 3/9 block: blk-crypto for Inline Encryption
In reply to Satya TangiralaWe introduce blk-crypto, which manages programming keyslots for struct bios. With blk-crypto, filesystems only need to call bio_crypt_set_ctx with the encryption key, algorithm and data_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-crypto handles that. Blk-crypto also makes it possible for layered devices like device mapper to make use of inline encryption hardware.
Blk-crypto delegates crypto operations to inline encryption hardware when available, and also contains a software fallback to the kernel crypto API. For more details, refer to Documentation/block/inline-encryption.rst.
--- Documentation/block/index.rst | 1 + Documentation/block/inline-encryption.rst | 183 +++++ block/Kconfig | 2 + block/Makefile | 3 +- block/bio-crypt-ctx.c | 7 +- block/bio.c | 5 + block/blk-core.c | 11 +- block/blk-crypto.c | 798 ++++++++++++++++++++++ include/linux/bio-crypt-ctx.h | 7 + include/linux/blk-crypto.h | 62 ++ 10 files changed, 1076 insertions(+), 3 deletions(-) create mode 100644 Documentation/block/inline-encryption.rst create mode 100644 block/blk-crypto.c create mode 100644 include/linux/blk-crypto.h
Documentation/block/index.rst +1 -0
@@ -14,6 +14,7 @@ Block cmdline-partition data-integrity deadline-iosched + inline-encryption ioprio kyber-iosched null_blk
Documentation/block/inline-encryption.rst new file +183 -0
183 lines not on this page · raw message
block/Kconfig +2 -0
@@ -179,6 +179,8 @@ config BLK_SED_OPAL config BLK_INLINE_ENCRYPTION bool "Enable inline encryption support in block layer" + select CRYPTO + select CRYPTO_BLKCIPHER help Build the blk-crypto subsystem. Enabling this lets the block layer handle encryption,
block/Makefile +2 -1
@@ -36,4 +36,5 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o bio-crypt-ctx.o \ + blk-crypto.o
block/bio-crypt-ctx.c +6 -1
@@ -43,7 +43,12 @@ EXPORT_SYMBOL(bio_crypt_free_ctx);int bio_crypt_clone (struct bio *dst,struct bio *src,gfp_t gfp_mask) { -if (!bio_has_crypt_ctx (src)) +/* +* If a bio is swhandled, then it will be decrypted when bio_endio +* is called. As we only want the data to be decrypted once, copies +* of the bio must not have have a crypt context. +*/ +if (!bio_has_crypt_ctx (src) ||bio_crypt_swhandled (src))return 0 ; dst->bi_crypt_context =bio_crypt_alloc_ctx (gfp_mask);
block/bio.c +5 -0
@@ -17,6 +17,7 @@#include <linux/cgroup.h> #include <linux/blk-cgroup.h> #include <linux/highmem.h> +#include <linux/blk-crypto.h> #include <trace/events/block.h> #include "blk.h"
@@ -1788,6 +1789,10 @@ void bio_endio(struct bio *bio) again:if (!bio_remaining_done (bio))return ; + +if (!blk_crypto_endio (bio)) +return ; +if (!bio_integrity_endio (bio))return ;
block/blk-core.c +10 -1
@@ -38,6 +38,7 @@#include <linux/debugfs.h> #include <linux/bpf.h> #include <linux/psi.h> +#include <linux/blk-crypto.h> #define CREATE_TRACE_POINTS #include <trace/events/block.h>
@@ -1061,7 +1062,9 @@ blk_qc_t generic_make_request(struct bio *bio)/* Create a fresh bio_list for all subordinate requests */ bio_list_on_stack[1 ] = bio_list_on_stack[0 ];bio_list_init (&bio_list_on_stack[0 ]); - ret = q->make_request_fn (q, bio); + +if (!blk_crypto_submit_bio (&bio)) + ret = q->make_request_fn (q, bio);blk_queue_exit (q);
@@ -1114,6 +1117,9 @@ blk_qc_t direct_make_request(struct bio *bio)if (!generic_make_request_checks (bio))return BLK_QC_T_NONE ; +if (blk_crypto_submit_bio (&bio)) +return BLK_QC_T_NONE ; +if (unlikely (blk_queue_enter (q, nowait ?BLK_MQ_REQ_NOWAIT :0 ))) {if (nowait && !blk_queue_dying (q)) bio->bi_status =BLK_STS_AGAIN ;
@@ -1810,5 +1816,8 @@ int __init blk_dev_init(void)if (bio_crypt_ctx_init () <0 )panic ("Failed to allocate mem for bio crypt ctxs\n" ); +if (blk_crypto_init () <0 ) +panic ("Failed to init blk-crypto\n" ); +return 0 ; }
block/blk-crypto.c new file +798 -0
798 lines not on this page · raw message
include/linux/bio-crypt-ctx.h +7 -0
@@ -53,6 +53,8 @@ static inline void bio_crypt_advance(struct bio *bio, unsigned int bytes) } } +extern bool bio_crypt_swhandled (struct bio *bio); +static inline bool bio_crypt_has_keyslot (struct bio *bio) {return bio->bi_crypt_context->keyslot >=0 ;
@@ -170,6 +172,11 @@ static inline void bio_crypt_set_ctx(struct bio *bio, unsigned int dun_bits, gfp_t gfp_mask) { } +static inline bool bio_crypt_swhandled (struct bio *bio) +{ +return false; +} +static inline void bio_set_data_unit_num (struct bio *bio,u64 dun) { }static inline bool bio_crypt_has_keyslot (struct bio *bio)
include/linux/blk-crypto.h new file +62 -0
62 lines not on this page · raw message
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 3/9] block: blk-crypto for Inline Encryption
Quoted 10 lines · Satya Tangirala
We introduce blk-crypto, which manages programming keyslots for struct bios. With blk-crypto, filesystems only need to call
bio_crypt_set_ctxwith the encryption key, algorithm anddata_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-crypto handles that. Blk-crypto also makes it possible for layered devices like device mapper to make use of inline encryption hardware.Blk-crypto delegates crypto operations to inline encryption hardware when available, and also contains a software fallback to the kernel crypto API. For more details, refer to
Documentation/block/inline-encryption.rst.
Can you explain why we need this software fallback that basically just duplicates logic already in fscrypt? As far as I can tell this fallback logic actually is more code than the actual inline encryption, and nasty code at that, e.g. the whole crypt_iter thing.
Theodore Y. Ts'o
maintainerRe: [PATCH v5 3/9] block: blk-crypto for Inline Encryption
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Satya Tangirala
We introduce blk-crypto, which manages programming keyslots for struct bios. With blk-crypto, filesystems only need to call
bio_crypt_set_ctxwith the encryption key, algorithm anddata_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-crypto handles that. Blk-crypto also makes it possible for layered devices like device mapper to make use of inline encryption hardware.
Quoted 8 lines
Quoted 3 lines
Blk-crypto delegates crypto operations to inline encryption hardware when available, and also contains a software fallback to the kernel crypto API. For more details, refer to
Documentation/block/inline-encryption.rst.Can you explain why we need this software fallback that basically just duplicates logic already in fscrypt? As far as I can tell this fallback logic actually is more code than the actual inline encryption, and nasty code at that, e.g. the whole
crypt_iterthing.
One of the reasons I really want this is so I (as an upstream maintainer of ext4 and fscrypt) can test the new code paths using xfstests on GCE, without needing special pre-release hardware that has the ICE support.
Yeah, I could probably get one of those dev boards internally at Google, but they're a pain in the tuckus to use, and I'd much rather be able to have my normal test infrastructure using gce-xfstests and kvm-xfstests be able to test inline-crypto. So in terms of CI testing, having the blk-crypto is really going to be helpful.
- Ted
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [PATCH v5 3/9] block: blk-crypto for Inline Encryption
Quoted 10 lines · Theodore Y. Ts'o
One of the reasons I really want this is so I (as an upstream maintainer of ext4 and fscrypt) can test the new code paths using xfstests on GCE, without needing special pre-release hardware that has the ICE support.
Yeah, I could probably get one of those dev boards internally at Google, but they're a pain in the tuckus to use, and I'd much rather be able to have my normal test infrastructure using gce-xfstests and kvm-xfstests be able to test inline-crypto. So in terms of CI testing, having the blk-crypto is really going to be helpful.
Implementing the support in qemu or a special device mapper mode seems like a much better idea for that use case over carrying the code in the block layer and severely bloating the per-I/O data structure.
Eric Biggers
maintainerRe: [PATCH v5 3/9] block: blk-crypto for Inline Encryption
Quoted 12 lines · Christoph Hellwig
Quoted 10 lines · Theodore Y. Ts'o
One of the reasons I really want this is so I (as an upstream maintainer of ext4 and fscrypt) can test the new code paths using xfstests on GCE, without needing special pre-release hardware that has the ICE support.
Yeah, I could probably get one of those dev boards internally at Google, but they're a pain in the tuckus to use, and I'd much rather be able to have my normal test infrastructure using gce-xfstests and kvm-xfstests be able to test inline-crypto. So in terms of CI testing, having the blk-crypto is really going to be helpful.
Quoted 5 lines
Implementing the support in qemu or a special device mapper mode seems like a much better idea for that use case over carrying the code in the block layer and severely bloating the per-I/O data structure.
QEMU doesn't support UFS, but even if it did and we added the UFS v2.1 crypto support, it would preclude testing with anything other than a custom QEMU VM or a system with real inline encryption hardware. gce-xfstests wouldn't work. So it would be much harder to test inline encrypted I/O, so e.g. in practice it wouldn't be tested as part of the regular ext4 regression testing.
The advantages of blk-crypto over a device mapper target like "dm-inlinecrypt" are (a) blk-crypto is much easier for userspace to use, and (b) blk-crypto allows upper layers to simply use inline encryption rather than have to implement encryption twice, once manually and once with inline encryption.
It's true that as of this patchset, the only user of this stuff (fscrypt) still implements both I/O paths anyway. But that's something that could change later once blk-crypto is ready for it, with longer IV support, O(1) keyslot lookups, and a way to configure whether hardware is used or not. Satya is already looking into longer IV support, and I have a proposal for making the keyslot lookups O(1) using a hash table.
I think that "Severely bloating the per-I/O data structure" is an exaggeration, since that it's only 32 bytes, and it isn't in struct bio directly but rather in struct bio_crypt_ctx...
In any case, Satya, it might be a good idea to reorganize this patchset so that it first adds all logic that's needed for "real" inline encryption support (including the needed parts of blk-crypto.c), then adds the crypto API fallback as a separate patch. That would separate the concerns more cleanly and make the patchset easier to review, and make it easier to make the fallback de-configurable or even remove it entirely if that turns out to be needed.
- Eric
Re: [PATCH v5 3/9] block: blk-crypto for Inline Encryption
Quoted 3 lines · Eric Biggers
I think that "Severely bloating the per-I/O data structure" is an exaggeration, since that it's only 32 bytes, and it isn't in struct bio directly but rather in struct
bio_crypt_ctx...
Yes, and none of that is needed for the real inline crypto. And I think we can further reduce the overhead of bio_crypt_ctx once we have the basiscs sorted out. If we want to gain more traction we need to reduce the I/O to a minimum.
Quoted 6 lines
In any case, Satya, it might be a good idea to reorganize this patchset so that it first adds all logic that's needed for "real" inline encryption support (including the needed parts of
blk-crypto.c), then adds the crypto API fallback as a separate patch. That would separate the concerns more cleanly and make the patchset easier to review, and make it easier to make the fallback de-configurable or even remove it entirely if that turns out to be needed.
Yes, that is a good idea. Not just in terms of patch, but also in terms of code organization. The current structure is pretty weird with 3 files that are mostly tighly integrated, except that one also has the software implementations. So what I think we need at a minimum is:
- reoranizize that we have say block/blk-crypt.c for all the inline crypto infrastructure, and block/blk-crypy-sw.c for the actual software crypto implementation.
- remove all the fields only needed for software crypto from bio_crypt_ctx, and instead clone the bio into a bioset with the additional fields only when we use the software implementation, so that there is no overhead for the hardware path.
PATCH v5 4/9 scsi: ufs: UFS driver v2.1 spec crypto additions
In reply to Satya TangiralaAdd the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI specification in preparation to add support for inline encryption to UFS.
--- drivers/scsi/ufs/ufshcd.c | 2 ++ drivers/scsi/ufs/ufshcd.h | 5 +++ drivers/scsi/ufs/ufshci.h | 67 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 2 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
@@ -4769,6 +4769,8 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp)case OCS_MISMATCH_RESP_UPIU_SIZE :case OCS_PEER_COMM_FAILURE :case OCS_FATAL_ERROR : +case OCS_INVALID_CRYPTO_CONFIG : +case OCS_GENERAL_CRYPTO_ERROR :default : result |=DID_ERROR <<16 ; dev_err(hba->dev,
drivers/scsi/ufs/ufshcd.h +5 -0
@@ -716,6 +716,11 @@ struct ufs_hba {* the performance of ongoing read/write operations. */ #define UFSHCD_CAP_KEEP_AUTO_BKOPS_ENABLED_EXCEPT_SUSPEND (1 << 5) +/* +* This capability allows the host controller driver to use the +* inline crypto engine, if it is present +*/ +#define UFSHCD_CAP_CRYPTO (1 << 6)struct devfreq *devfreq;struct ufs_clk_scaling clk_scaling;
drivers/scsi/ufs/ufshci.h +65 -2
106 lines not on this page · raw message
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
PATCH v5 5/9 scsi: ufs: UFS crypto API
In reply to Satya TangiralaIntroduce functions to manipulate UFS inline encryption hardware in line with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot manager.
--- drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd-crypto.c | 391 +++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 86 +++++++ drivers/scsi/ufs/ufshcd.h | 14 ++ 5 files changed, 501 insertions(+) create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
drivers/scsi/ufs/Kconfig +9 -0
@@ -150,3 +150,12 @@ config SCSI_UFS_BSG Select this if you need a bsg device node for your UFS controller. If unsure, say N. + +config SCSI_UFS_CRYPTO + bool "UFS Crypto Engine Support" + depends on SCSI_UFSHCD && BLK_INLINE_ENCRYPTION + help + Enable Crypto Engine Support in UFS. + Enabling this makes it possible for the kernel to use the crypto + capabilities of the UFS device (if present) to perform crypto + operations on data being transferred to/from the device.
drivers/scsi/ufs/Makefile +1 -0
@@ -11,3 +11,4 @@ obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o obj-$(CONFIG_SCSI_UFS_MEDIATEK) += ufs-mediatek.o +ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o
drivers/scsi/ufs/ufshcd-crypto.c new file +391 -0
391 lines not on this page · raw message
drivers/scsi/ufs/ufshcd-crypto.h new file +86 -0
86 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +14 -0
@@ -525,6 +525,11 @@ struct ufs_stats {* @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for * device is known or not. * @scsi_block_reqs_cnt: reference counting for scsi block requests +* @crypto_capabilities: Content of crypto capabilities register (0x100) +* @crypto_cap_array: Array of crypto capabilities +* @crypto_cfg_register: Start of the crypto cfg array +* @crypto_cfgs: Array of crypto configurations (i.e. config for each slot) +* @ksm: the keyslot manager tied to this hba */ struct ufs_hba {void __iomem *mmio_base;
@@ -735,6 +740,15 @@ struct ufs_hba {struct device bsg_dev;struct request_queue *bsg_queue; + +#ifdef CONFIG_SCSI_UFS_CRYPTO +/* crypto */ +union ufs_crypto_capabilities crypto_capabilities; +union ufs_crypto_cap_entry *crypto_cap_array; +u32 crypto_cfg_register; +union ufs_crypto_cfg_entry *crypto_cfgs; +struct keyslot_manager *ksm; +#endif /* CONFIG_SCSI_UFS_CRYPTO */ };/* Returns true if clocks can be gated. Otherwise false */
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 5/9] scsi: ufs: UFS crypto API
drivers/scsi/ufs/ufshcd-crypto.c line 34
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10
-0+static size_t get_keysize_bytes (enum ufs_crypto_key_size size) +{ +switch (size) { +case UFS_CRYPTO_KEY_SIZE_128 :return 16 ; +case UFS_CRYPTO_KEY_SIZE_192 :return 24 ; +case UFS_CRYPTO_KEY_SIZE_256 :return 32 ; +case UFS_CRYPTO_KEY_SIZE_512 :return 64 ; +default :return 0 ; + } +}
Please fix the indentation and move all the returns to their own lines. There are various more spots that will need to be fixed like this as well later in the patch.
drivers/scsi/ufs/ufshcd-crypto.c line 40
Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6
-0+ +static int ufshcd_crypto_cap_find (void *hba_p, +enum blk_crypto_mode_num crypto_mode, +unsigned int data_unit_size) +{ +struct ufs_hba *hba = hba_p;
Please properly type the first argument.
drivers/scsi/ufs/ufshcd-crypto.c line 104
Quoted 1 line
drivers/scsi/ufs/ufshcd-crypto.c new file +1
-0+case UFS_CRYPTO_ALG_BITLOCKER_AES_CBC :// fallthrough
Please don't use // comments.
drivers/scsi/ufs/ufshcd-crypto.c line 116
Quoted 3 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +3
-0+static void program_key (struct ufs_hba *hba, +const union ufs_crypto_cfg_entry *cfg, +int slot)
The function name needs a ufshcd prefix.
Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6
-0+wmb (); +for (i =0 ; i <16 ; i++) { +ufshcd_writel (hba,le32_to_cpu (cfg->reg_val[i]), + slot_offset + i *sizeof (cfg->reg_val[0 ])); +/* Spec says each dword in key must be written sequentially */ +wmb ();
drivers/scsi/ufs/ufshcd-crypto.c line 139
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10
-0+ } +/* Write dword 17 */ +ufshcd_writel (hba,le32_to_cpu (cfg->reg_val[17 ]), + slot_offset +17 *sizeof (cfg->reg_val[0 ])); +/* Dword 16 must be written last */ +wmb (); +/* Write dword 16 */ +ufshcd_writel (hba,le32_to_cpu (cfg->reg_val[16 ]), + slot_offset +16 *sizeof (cfg->reg_val[0 ])); +wmb ();
wmb() has no meaning for MMIO operations, something looks very fishy here.
drivers/scsi/ufs/ufshcd-crypto.c line 147
Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6
-0+static int ufshcd_crypto_keyslot_program (void *hba_p,const u8 *key, +enum blk_crypto_mode_num crypto_mode, +unsigned int data_unit_size, +unsigned int slot) +{ +struct ufs_hba *hba = hba_p;
This is not a very type safe API. I think the proper thing to do would be to allocte the struct keyslot_manager in the driver (ufshcd) as part of the containing structure (ufs_hba) and then just have a keyslot_manager_init that initializes the field. Then pass the struct keyslot_manager to the methods, which can use container_of to get the containing structure.
Quoted 1 line
+#define
NUM_KEYSLOTS(hba)(hba->crypto_capabilities.config_count+ 1)
Please make this an inline function.
PATCH v5 6/9 scsi: ufs: Add inline encryption support to UFS
In reply to Satya TangiralaWire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.
--- drivers/scsi/ufs/ufshcd.c | 83 ++++++++++++++++++++++++++++++++++++--- drivers/scsi/ufs/ufshcd.h | 6 +++ 2 files changed, 84 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd.c +78 -5
@@ -47,6 +47,7 @@#include "unipro.h" #include "ufs-sysfs.h" #include "ufs_bsg.h" +#include "ufshcd-crypto.h" #define CREATE_TRACE_POINTS #include <trace/events/ufs.h>
@@ -857,7 +858,14 @@ static void ufshcd_enable_run_stop_reg(struct ufs_hba *hba)*/ static inline void ufshcd_hba_start (struct ufs_hba *hba) { -ufshcd_writel (hba,CONTROLLER_ENABLE ,REG_CONTROLLER_ENABLE ); +u32 val =CONTROLLER_ENABLE ; + +if (ufshcd_hba_is_crypto_supported (hba)) { +ufshcd_crypto_enable (hba); + val |=CRYPTO_GENERAL_ENABLE ; + } + +ufshcd_writel (hba, val,REG_CONTROLLER_ENABLE ); }/**
@@ -2211,9 +2219,21 @@ static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp, dword_0 |=UTP_REQ_DESC_INT_CMD ;/* Transfer request descriptor header fields */ +if (lrbp->crypto_enable) { + dword_0 |=UTP_REQ_DESC_CRYPTO_ENABLE_CMD ; + dword_0 |= lrbp->crypto_key_slot; + req_desc->header.dword_1 = +cpu_to_le32 ((u32 )lrbp->data_unit_num); + req_desc->header.dword_3 = +cpu_to_le32 ((u32)(lrbp->data_unit_num >>32 )); + }else { +/* dword_1 and dword_3 are reserved, hence they are set to 0 */ + req_desc->header.dword_1 =0 ; + req_desc->header.dword_3 =0 ; + } + req_desc->header.dword_0 =cpu_to_le32 (dword_0); -/* dword_1 is reserved, hence it is set to 0 */ - req_desc->header.dword_1 =0 ; +/* * assigning invalid value for command status. Controller * updates OCS on command completion, with the command
@@ -2221,8 +2241,6 @@ static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp,*/ req_desc->header.dword_2 =cpu_to_le32 (OCS_INVALID_COMMAND_STATUS ); -/* dword_3 is reserved, hence it is set to 0 */ - req_desc->header.dword_3 =0 ; req_desc->prd_table_length =0 ; }
@@ -2382,6 +2400,37 @@ static inline u16 ufshcd_upiu_wlun_to_scsi_wlun(u8 upiu_wlun_id)return (upiu_wlun_id & ~UFS_UPIU_WLUN_ID ) |SCSI_W_LUN_BASE ; } +static inline int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +int key_slot; + +if (!cmd->request->bio || + !bio_crypt_should_process (cmd->request->bio, cmd->request->q)) { + lrbp->crypto_enable = false; +return 0 ; + } + +if (WARN_ON (!ufshcd_is_crypto_enabled (hba))) { +/* +* Upper layer asked us to do inline encryption +* but that isn't enabled, so we fail this request. +*/ +return -EINVAL ; + } + key_slot =bio_crypt_get_keyslot (cmd->request->bio); +if (!ufshcd_keyslot_valid (hba, key_slot)) +return -EINVAL ; + + lrbp->crypto_enable = true; + lrbp->crypto_key_slot = key_slot; + lrbp->data_unit_num =bio_crypt_data_unit_num (cmd->request->bio); + +return 0 ; +} + +/** * ufshcd_queuecommand - main entry point for SCSI requests * @host: SCSI host pointer
@@ -2469,6 +2518,13 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) lrbp->task_tag = tag; lrbp->lun =ufshcd_scsi_to_upiu_lun (cmd->device->lun); lrbp->intr_cmd = !ufshcd_is_intr_aggr_allowed (hba) ? true : false; + + err =ufshcd_prepare_lrbp_crypto (hba, cmd, lrbp); +if (err) { + lrbp->cmd =NULL ; +clear_bit_unlock (tag, &hba->lrb_in_use); + goto out; + } lrbp->req_abort_skip = false;ufshcd_comp_scsi_upiu (hba, lrbp);
@@ -2502,6 +2558,7 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, lrbp->task_tag = tag; lrbp->lun =0 ;/* device management cmd is not specific to any LUN */ lrbp->intr_cmd = true;/* No interrupt aggregation */ + lrbp->crypto_enable = false;/* No crypto operations */ hba->dev_cmd.type = cmd_type;return ufshcd_comp_devman_upiu (hba, lrbp);
@@ -4229,6 +4286,8 @@ static inline void ufshcd_hba_stop(struct ufs_hba *hba, bool can_sleep) {int err; +ufshcd_crypto_disable (hba); +ufshcd_writel (hba,CONTROLLER_DISABLE ,REG_CONTROLLER_ENABLE ); err = ufshcd_wait_for_register(hba, REG_CONTROLLER_ENABLE, CONTROLLER_ENABLE, CONTROLLER_DISABLE,
@@ -4632,8 +4691,12 @@ static int ufshcd_change_queue_depth(struct scsi_device *sdev, int depth)static int ufshcd_slave_configure (struct scsi_device *sdev) {struct request_queue *q = sdev->request_queue; +struct ufs_hba *hba =shost_priv (sdev->host);blk_queue_update_dma_pad (q,PRDT_DATA_BYTE_COUNT_PAD -1 ); + +ufshcd_crypto_setup_rq_keyslot_manager (hba, q); +return 0 ; }
@@ -4644,6 +4707,7 @@ static int ufshcd_slave_configure(struct scsi_device *sdev)static void ufshcd_slave_destroy (struct scsi_device *sdev) {struct ufs_hba *hba; +struct request_queue *q = sdev->request_queue; hba =shost_priv (sdev->host);/* Drop the reference as it won't be needed anymore */
@@ -4654,6 +4718,8 @@ static void ufshcd_slave_destroy(struct scsi_device *sdev) hba->sdev_ufs_device =NULL ;spin_unlock_irqrestore (hba->host->host_lock, flags); } + +ufshcd_crypto_destroy_rq_keyslot_manager (hba, q); }/**
@@ -8380,6 +8446,13 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)/* Reset the attached device */ ufshcd_vops_device_reset (hba); +/* Init crypto */ + err =ufshcd_hba_init_crypto (hba); +if (err) { +dev_err (hba->dev,"crypto setup failed\n" ); + goto out_remove_scsi_host; + } +/* Host controller enable */ err =ufshcd_hba_enable (hba);if (err) {
drivers/scsi/ufs/ufshcd.h +6 -0
@@ -167,6 +167,9 @@ struct ufs_pm_lvl_states {* @intr_cmd: Interrupt command (doesn't participate in interrupt aggregation) * @issue_time_stamp: time stamp for debug purposes * @compl_time_stamp: time stamp for statistics +* @crypto_enable: whether or not the request needs inline crypto operations +* @crypto_key_slot: the key slot to use for inline crypto +* @data_unit_num: the data unit number for the first block for inline crypto * @req_abort_skip: skip request abort task flag */ struct ufshcd_lrb {
@@ -191,6 +194,9 @@ struct ufshcd_lrb {bool intr_cmd;ktime_t issue_time_stamp;ktime_t compl_time_stamp; +bool crypto_enable; +u8 crypto_key_slot; +u64 data_unit_num;bool req_abort_skip; };
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 6/9] scsi: ufs: Add inline encryption support to UFS
drivers/scsi/ufs/ufshcd.c static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp, · line 2222
Quoted 2 lines
drivers/scsi/ufs/ufshcd.c +1
-0/* Transfer request descriptor header fields */ +if (lrbp->crypto_enable) {
Maybe we want a little inline function so that we can use IS_ENABLED to make sure the compiler eliminates the dead code if crypt config option is not set.
a) don't have to define the crypto_enable if the config options are not set
drivers/scsi/ufs/ufshcd.c static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp, · line 2228
Quoted 6 lines
drivers/scsi/ufs/ufshcd.c +6
-0+ dword_0 |=UTP_REQ_DESC_CRYPTO_ENABLE_CMD ; + dword_0 |= lrbp->crypto_key_slot; + req_desc->header.dword_1 = +cpu_to_le32 ((u32 )lrbp->data_unit_num); + req_desc->header.dword_3 = +cpu_to_le32 ((u32)(lrbp->data_unit_num >>32 ));
This should use ther upper_32_bits / lower_32_bits helpers.
Quoted 18 lines
drivers/scsi/ufs/ufshcd.c +18
-0+static inline int ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +int key_slot; + +if (!cmd->request->bio || + !bio_crypt_should_process (cmd->request->bio, cmd->request->q)) { + lrbp->crypto_enable = false; +return 0 ; + } + +if (WARN_ON (!ufshcd_is_crypto_enabled (hba))) { +/* +* Upper layer asked us to do inline encryption +* but that isn't enabled, so we fail this request. +*/ +return -EINVAL ;
drivers/scsi/ufs/ufshcd.c static inline u16 ufshcd_upiu_wlun_to_scsi_wlun(u8 upiu_wlun_id) · line 2430
Quoted 10 lines
drivers/scsi/ufs/ufshcd.c +10
-0+ } + key_slot =bio_crypt_get_keyslot (cmd->request->bio); +if (!ufshcd_keyslot_valid (hba, key_slot)) +return -EINVAL ; + + lrbp->crypto_enable = true; + lrbp->crypto_key_slot = key_slot; + lrbp->data_unit_num =bio_crypt_data_unit_num (cmd->request->bio); + +return 0 ;
I think this should go into ufshcd-crypto.c so that it can be stubbed out for non-crypto builds. That also means we can remove various stubs for the block layer helpers and just dereference the fields directly, helping with code readability.
PATCH v5 7/9 fscrypt: add inline encryption support
In reply to Satya TangiralaAdd support for inline encryption to fs/crypto/. With "inline encryption", the block layer handles the decryption/encryption as part of the bio, instead of the filesystem doing the crypto itself via Linux's crypto API. This model is needed in order to take advantage of the inline encryption hardware present on most modern mobile SoCs.
To use inline encryption, the filesystem needs to be mounted with '-o inlinecrypt'. The contents of any AES-256-XTS encrypted files will then be encrypted using blk-crypto, instead of using the traditional filesystem-layer crypto. fscrypt still provides the key and IV to use, and the actual ciphertext on-disk is still the same; therefore it's testable using the existing fscrypt ciphertext verification tests.
Note that since blk-crypto has a fallack to Linux's crypto API, this feature is usable and testable even without actual inline encryption hardware.
Per-filesystem changes will be needed to set encryption contexts when submitting bios and to implement the 'inlinecrypt' mount option. This patch just adds the common code.
--- fs/crypto/Kconfig | 6 + fs/crypto/Makefile | 1 + fs/crypto/bio.c | 31 ++- fs/crypto/fscrypt_private.h | 72 +++++++ fs/crypto/inline_crypt.c | 390 ++++++++++++++++++++++++++++++++++++ fs/crypto/keyring.c | 2 + fs/crypto/keysetup.c | 18 +- include/linux/fscrypt.h | 60 ++++++ 8 files changed, 566 insertions(+), 14 deletions(-) create mode 100644 fs/crypto/inline_crypt.c
fs/crypto/Kconfig +6 -0
@@ -16,3 +16,9 @@ config FS_ENCRYPTION efficient since it avoids caching the encrypted and decrypted pages in the page cache. Currently Ext4, F2FS and UBIFS make use of this feature. + +config FS_ENCRYPTION_INLINE_CRYPT + bool "Enable fscrypt to use inline crypto" + depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION + help + Enable fscrypt to use inline encryption hardware if available.
fs/crypto/Makefile +1 -0
@@ -11,3 +11,4 @@ fscrypto-y := crypto.o \ policy.o fscrypto-$(CONFIG_BLOCK) += bio.o +fscrypto-$(CONFIG_FS_ENCRYPTION_INLINE_CRYPT) += inline_crypt.o
fs/crypto/bio.c +22 -9
@@ -46,26 +46,38 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, {const unsigned int blockbits = inode->i_blkbits;const unsigned int blocksize =1 << blockbits; +const bool inlinecrypt =fscrypt_inode_uses_inline_crypto (inode);struct page *ciphertext_page;struct bio *bio;int ret, err =0 ; - ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); -if (!ciphertext_page) -return -ENOMEM ; +if (inlinecrypt) { + ciphertext_page =ZERO_PAGE (0 ); + }else { + ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); +if (!ciphertext_page) +return -ENOMEM ; + }while (len--) { - err =fscrypt_crypt_block (inode,FS_ENCRYPT , lblk, -ZERO_PAGE (0 ), ciphertext_page, - blocksize,0 ,GFP_NOFS ); -if (err) - goto errout; +if (!inlinecrypt) { + err =fscrypt_crypt_block (inode,FS_ENCRYPT , lblk, +ZERO_PAGE (0 ), ciphertext_page, + blocksize,0 ,GFP_NOFS ); +if (err) + goto errout; + } bio =bio_alloc (GFP_NOWAIT ,1 );if (!bio) { err = -ENOMEM ; goto errout; } + err =fscrypt_set_bio_crypt_ctx (bio, inode, lblk,GFP_NOIO ); +if (err) { +bio_put (bio); + goto errout; + }bio_set_dev (bio, inode->i_sb->s_bdev); bio->bi_iter.bi_sector = pblk << (blockbits -9 );bio_set_op_attrs (bio,REQ_OP_WRITE ,0 );
@@ -87,7 +99,8 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, } err =0 ; errout: -fscrypt_free_bounce_page (ciphertext_page); +if (!inlinecrypt) +fscrypt_free_bounce_page (ciphertext_page);return err; }EXPORT_SYMBOL (fscrypt_zeroout_range);
fs/crypto/fscrypt_private.h +72 -0
105 lines not on this page · raw message
fs/crypto/inline_crypt.c new file +390 -0
390 lines not on this page · raw message
fs/crypto/keyring.c +2 -0
@@ -48,6 +48,8 @@ static void free_master_key(struct fscrypt_master_key *mk)crypto_free_skcipher (mk->mk_iv_ino_lblk_64_tfms[i]); } +fscrypt_evict_inline_crypt_keys (mk); +key_put (mk->mk_users);kzfree (mk); }
fs/crypto/keysetup.c +13 -5
@@ -13,12 +13,13 @@#include "fscrypt_private.h" -static struct fscrypt_mode available_modes[] = { +struct fscrypt_mode fscrypt_modes[] = { [FSCRYPT_MODE_AES_256_XTS ] = { .friendly_name ="AES-256-XTS" , .cipher_str ="xts(aes)" , .keysize =64 , .ivsize =16 , + .blk_crypto_mode =BLK_ENCRYPTION_MODE_AES_256_XTS , }, [FSCRYPT_MODE_AES_256_CTS ] = { .friendly_name ="AES-256-CTS-CBC" ,
@@ -51,10 +52,10 @@ select_encryption_mode(const union fscrypt_policy *policy, const struct inode *inode) {if (S_ISREG (inode->i_mode)) -return &available_modes[fscrypt_policy_contents_mode (policy)]; +return &fscrypt_modes[fscrypt_policy_contents_mode (policy)];if (S_ISDIR (inode->i_mode) ||S_ISLNK (inode->i_mode)) -return &available_modes[fscrypt_policy_fnames_mode (policy)]; +return &fscrypt_modes[fscrypt_policy_fnames_mode (policy)];WARN_ONCE (1 ,"fscrypt: filesystem tried to load encryption info for inode %lu, which is not encryptable (file type %d)\n" , inode->i_ino, (inode->i_mode &S_IFMT ));
@@ -111,6 +112,9 @@ int fscrypt_set_derived_key(struct fscrypt_info *ci, const u8 *derived_key) {struct crypto_skcipher *tfm; +if (fscrypt_should_use_inline_encryption (ci)) +return fscrypt_set_inline_crypt_key (ci, derived_key); + tfm =fscrypt_allocate_skcipher (ci->ci_mode, derived_key, ci->ci_inode);if (IS_ERR (tfm))return PTR_ERR (tfm);
@@ -128,7 +132,7 @@ static int setup_per_mode_key(struct fscrypt_info *ci,const struct inode *inode = ci->ci_inode;const struct super_block *sb = inode->i_sb;struct fscrypt_mode *mode = ci->ci_mode; -u8 mode_num = mode - available_modes; +const u8 mode_num = mode - fscrypt_modes;struct crypto_skcipher *tfm, *prev_tfm;u8 mode_key[FSCRYPT_MAX_KEY_SIZE ];u8 hkdf_info[sizeof (mode_num) +sizeof (sb->s_uuid)];
@@ -204,6 +208,8 @@ static int fscrypt_setup_v2_file_key(struct fscrypt_info *ci,* the IVs. This format is optimized for use with inline * encryption hardware compliant with the UFS or eMMC standards. */ +if (fscrypt_should_use_inline_encryption (ci)) +return fscrypt_setup_per_mode_inline_crypt_key (ci, mk);return setup_per_mode_key (ci, mk, mk->mk_iv_ino_lblk_64_tfms,HKDF_CONTEXT_IV_INO_LBLK_64_KEY , true);
@@ -330,8 +336,10 @@ static void put_crypt_info(struct fscrypt_info *ci)if (ci->ci_direct_key)fscrypt_put_direct_key (ci->ci_direct_key); -else if (ci->ci_owns_key) +else if (ci->ci_owns_key) {crypto_free_skcipher (ci->ci_ctfm); +fscrypt_free_inline_crypt_key (ci); + } key = ci->ci_master_key;if (key) {
include/linux/fscrypt.h +60 -0
72 lines not on this page · raw message
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 7/9] fscrypt: add inline encryption support
Quoted 12 lines
fs/crypto/bio.c +1
-0@@ -46,26 +46,38 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, {const unsigned int blockbits = inode->i_blkbits;const unsigned int blocksize =1 << blockbits; +const bool inlinecrypt =fscrypt_inode_uses_inline_crypto (inode);struct page *ciphertext_page;struct bio *bio;int ret, err =0 ;
fs/crypto/bio.c int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, · line 59
Quoted 10 lines
fs/crypto/bio.c +6
-3- ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); -if (!ciphertext_page) -return -ENOMEM ; +if (inlinecrypt) { + ciphertext_page =ZERO_PAGE (0 ); + }else { + ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); +if (!ciphertext_page) +return -ENOMEM ;
I think you just want to split this into two functions for the inline crypto vs not cases.
fs/crypto/bio.c struct fscrypt_master_key { · line 461
Quoted 13 lines
fs/crypto/bio.c +9
-0@@ -391,6 +450,16 @@ struct fscrypt_master_key {*/ struct crypto_skcipher *mk_iv_ino_lblk_64_tfms[__FSCRYPT_MODE_MAX +1 ]; +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT +/* Raw keys for IV_INO_LBLK_64 policies, allocated on-demand */ +u8 *mk_iv_ino_lblk_64_raw_keys[__FSCRYPT_MODE_MAX +1 ]; + +/* The data unit size being used for inline encryption */ +unsigned int mk_data_unit_size; + +/* The filesystem's block device */ +struct block_device *mk_bdev;
File systems (including f2fs) can have multiple underlying block devices.
fs/crypto/bio.c struct fscrypt_master_key {
Quoted 7 lines
fs/crypto/bio.c +7
-0+{ +const struct inode *inode = ci->ci_inode; +struct super_block *sb = inode->i_sb; + +/* The file must need contents encryption, not filenames encryption */ +if (!S_ISREG (inode->i_mode)) +return false;
But that isn't really what the check checks for..
fs/crypto/bio.c
Quoted 4 lines
fs/crypto/bio.c +4
-0+/* The filesystem must be mounted with -o inlinecrypt */ +if (!sb->s_cop->inline_crypt_enabled || + !sb->s_cop->inline_crypt_enabled (sb)) +return false;
So please add a SB_* flag for that option instead of the weird indirection.
Quoted 2 lines
+/**
+ *fscrypt_inode_uses_inline_crypto- test whether an inode uses inline encryption
This adds an overly long line. This happens many more times in the patch.
Btw, I'm not happy about the 8-byte IV assumptions everywhere here. That really should be a parameter, not hardcoded.
Eric Biggers
maintainerRe: [PATCH v5 7/9] fscrypt: add inline encryption support
Hi Christoph, thanks for reviewing this.
Quoted 30 lines · Christoph Hellwig
Quoted 12 lines
fs/crypto/bio.c +1
-0@@ -46,26 +46,38 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, {const unsigned int blockbits = inode->i_blkbits;const unsigned int blocksize =1 << blockbits; +const bool inlinecrypt =fscrypt_inode_uses_inline_crypto (inode);struct page *ciphertext_page;struct bio *bio;int ret, err =0 ;fs/crypto/bio.c int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, · line 59
Quoted 10 lines
fs/crypto/bio.c +6
-3- ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); -if (!ciphertext_page) -return -ENOMEM ; +if (inlinecrypt) { + ciphertext_page =ZERO_PAGE (0 ); + }else { + ciphertext_page =fscrypt_alloc_bounce_page (GFP_NOWAIT ); +if (!ciphertext_page) +return -ENOMEM ;I think you just want to split this into two functions for the inline crypto vs not cases.
Quoted 4 lines
fs/crypto/bio.c +0
-0@@ -391,6 +450,16 @@ struct fscrypt_master_key {*/ struct crypto_skcipher *mk_iv_ino_lblk_64_tfms[__FSCRYPT_MODE_MAX +1 ];
Quoted 12 lines
fs/crypto/bio.c struct fscrypt_master_key { · line 461
Quoted 9 lines
fs/crypto/bio.c +9
-0+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT +/* Raw keys for IV_INO_LBLK_64 policies, allocated on-demand */ +u8 *mk_iv_ino_lblk_64_raw_keys[__FSCRYPT_MODE_MAX +1 ]; + +/* The data unit size being used for inline encryption */ +unsigned int mk_data_unit_size; + +/* The filesystem's block device */ +struct block_device *mk_bdev;File systems (including f2fs) can have multiple underlying block devices.
Yes, this is a bug: it causes the key to be evicted from only one device.
I'm thinking this might point to deeper problems with the inline encryption API exposed to filesystems being too difficult to use correctly. We might want to move to something like:
struct blk_crypto_key_handle * blk_crypto_prepare_key(struct request_queue *q, const u8 *raw_key,
enum blk_crypto_mode_num crypto_mode, unsigned int data_unit_size);
int bio_crypt_set_ctx(struct bio *bio, struct blk_crypto_key_handle *h,
u64 dun, gfp_t gfp_mask);
void blk_crypto_destroy_key(struct blk_crypto_key_handle *h);
Each handle would associate a raw_key, mode, and data_unit_size with a specific keyslot_manager, which would allocate/evict a keyslot as needed for I/O. blk_crypto_destroy_key() would both evict the keyslot and free the key.
For each key, multi-device filesystems would then need to allocate a handle for each device. This would force them to handle key eviction correctly.
Satya, any thoughts on this?
Quoted 10 lines
fs/crypto/bio.c struct fscrypt_master_key {
Quoted 8 lines
fs/crypto/bio.c +7
-0+{ +const struct inode *inode = ci->ci_inode; +struct super_block *sb = inode->i_sb; + +/* The file must need contents encryption, not filenames encryption */ +if (!S_ISREG (inode->i_mode)) +return false;But that isn't really what the check checks for..
This is how fscrypt has always worked. The type of encryption to do is determined as follows:
S_ISREG() => contents encryptionS_ISDIR() || S_ISLNK() => filenames encryption
See e.g. select_encryption_mode(), and similar checks elsewhere in fs/{crypto,ext4,f2fs}/.
Do you have a suggestion to make it clearer?
Quoted 8 lines
fs/crypto/bio.c
Quoted 5 lines
fs/crypto/bio.c +4
-0+/* The filesystem must be mounted with -o inlinecrypt */ +if (!sb->s_cop->inline_crypt_enabled || + !sb->s_cop->inline_crypt_enabled (sb)) +return false;So please add a SB_* flag for that option instead of the weird indirection.
IMO it's not really "weird" given that the point of the fscrypt_operations is to expose filesystem-specific stuff to fs/crypto/. But yes, using one of the SB_* bits would make it simpler, so if people are fine with that we'll do that.
Quoted 6 lines
Quoted 3 lines
+/**
+ *fscrypt_inode_uses_inline_crypto- test whether an inode uses inline encryptionThis adds an overly long line. This happens many more times in the patch.
checkpatch reports 4 overly long lines in the patch, 3 of which are the first line of kerneldoc comments. For some reason I thought those are recommended to be one line even if it exceeds 80 characters, but maybe I'm mistaken.
Quoted 3 lines
Btw, I'm not happy about the 8-byte IV assumptions everywhere here. That really should be a parameter, not hardcoded.
To be clear, the 8-byte IV assumption doesn't really come from fs/crypto/, but rather in what the blk-crypto API provides. If blk-crypto were to provide longer IV support, fs/crypto/ would pretty easily be able to make use of it.
(And if IVs >= 24 bytes were supported and we added AES-128-CBC-ESSIV and Adiantum support to blk-crypto.c, then inline encryption would be able to do everything that the existing filesystem-layer contents encryption can do.)
Do you have anything in mind for how to make the API support longer IVs in a clean way? Are you thinking of something like:
#define BLK_CRYPTO_MAX_DUN_SIZE 24 u8 dun[BLK_CRYPTO_MAX_DUN_SIZE]; int dun_size;
We do have to perform arithmetic operations on it, so a byte array would make it ugly and slower, but it should be possible...
- Eric
Re: [PATCH v5 7/9] fscrypt: add inline encryption support
Quoted 9 lines · Eric Biggers
Quoted 5 lines
fs/crypto/inline_crypt.c line 30
Quoted 3 lines
fs/crypto/inline_crypt.c new file +3
-0+/* The file must need contents encryption, not filenames encryption */ +if (!S_ISREG (inode->i_mode)) +return false;But that isn't really what the check checks for..
This is how fscrypt has always worked. The type of encryption to do is determined as follows:
Quoted 7 lines
S_ISREG()=> contents encryptionS_ISDIR()||S_ISLNK()=> filenames encryptionSee e.g.
select_encryption_mode(), and similar checks elsewhere infs/{crypto,ext4,f2fs}/.Do you have a suggestion to make it clearer?
Maybe have a fscrypt_content_encryption helper that currently evaluates to S_ISREG(inode->i_mode) as that documents the intent?
Quoted 11 lines
Quoted 7 lines
fs/crypto/inline_crypt.c line 43
Quoted 4 lines
fs/crypto/inline_crypt.c new file +4
-0+/* The filesystem must be mounted with -o inlinecrypt */ +if (!sb->s_cop->inline_crypt_enabled || + !sb->s_cop->inline_crypt_enabled (sb)) +return false;So please add a SB_* flag for that option instead of the weird indirection.
IMO it's not really "weird" given that the point of the
fscrypt_operationsis to expose filesystem-specific stuff tofs/crypto/. But yes, using one of the SB_* bits would make it simpler, so if people are fine with that we'll do that.
I think that is much simpler. Additionally it could also replace the need for the has_stable_inodes and get_ino_and_lblk_bits methods, which are pretty weird. Instead just document the requirements for the SB_INLINE_CRYPT flag and handle the rest in the file system. That is for f2f always set it, and for ext4 set it based on s_inodes_count. Which brings me to:
Quoted 7 lines
Quoted 3 lines
Btw, I'm not happy about the 8-byte IV assumptions everywhere here. That really should be a parameter, not hardcoded.
To be clear, the 8-byte IV assumption doesn't really come from
fs/crypto/, but rather in what the blk-crypto API provides. If blk-crypto were to provide longer IV support,fs/crypto/would pretty easily be able to make use of it.
That's what I meant - we hardcode the value in fscrypt. Instead we need to expose the size from blk-crypt and check for it.
Quoted 8 lines
(And if IVs >= 24 bytes were supported and we added AES-128-CBC-ESSIV and Adiantum support to
blk-crypto.c, then inline encryption would be able to do everything that the existing filesystem-layer contents encryption can do.)Do you have anything in mind for how to make the API support longer IVs in a clean way? Are you thinking of something like:
Quoted 7 lines
#define BLK_CRYPTO_MAX_DUN_SIZE 24 u8 dun[BLK_CRYPTO_MAX_DUN_SIZE]; int dun_size;We do have to perform arithmetic operations on it, so a byte array would make it ugly and slower, but it should be possible...
Well, we could make it an array of u64s, which means we can do all the useful arithmetics on components on one of them. But I see the point, this adds significant complexity for no real short term gain, and we should probably postponed it until needed. Maybe just document the assumptions a little better.
Eric Biggers
maintainerRe: [PATCH v5 7/9] fscrypt: add inline encryption support
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Eric Biggers
Quoted 5 lines
fs/crypto/inline_crypt.c line 30
Quoted 3 lines
fs/crypto/inline_crypt.c new file +3
-0+/* The file must need contents encryption, not filenames encryption */ +if (!S_ISREG (inode->i_mode)) +return false;But that isn't really what the check checks for..
This is how fscrypt has always worked. The type of encryption to do is determined as follows:
S_ISREG()=> contents encryptionS_ISDIR()||S_ISLNK()=> filenames encryption
Quoted 7 lines
Quoted 4 lines
See e.g.
select_encryption_mode(), and similar checks elsewhere infs/{crypto,ext4,f2fs}/.Do you have a suggestion to make it clearer?
Maybe have a
fscrypt_content_encryptionhelper that currently evaluates toS_ISREG(inode->i_mode)as that documents the intent?
It's more important to clean up the IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) checks that are duplicated in fs/{ext4,f2fs}/, so I've been thinking of adding a helper:
static inline bool fscrypt_needs_contents_encryption(const struct inode *inode)
{
return IS_ENABLED(CONFIG_FS_ENCRYPTION) && IS_ENCRYPTED(inode) &&
S_ISREG(inode->i_mode);
}It could be used here too, though only S_ISREG() is actually needed here, so it wouldn't be as good of a fit.
Anyway, this will need to be a separate cleanup.
Quoted 13 lines
Quoted 9 lines
Quoted 8 lines
fs/crypto/inline_crypt.c line 43
Quoted 5 lines
fs/crypto/inline_crypt.c new file +4
-0+/* The filesystem must be mounted with -o inlinecrypt */ +if (!sb->s_cop->inline_crypt_enabled || + !sb->s_cop->inline_crypt_enabled (sb)) +return false;So please add a SB_* flag for that option instead of the weird indirection.
Quoted 3 lines
IMO it's not really "weird" given that the point of the
fscrypt_operationsis to expose filesystem-specific stuff tofs/crypto/. But yes, using one of the SB_* bits would make it simpler, so if people are fine with that we'll do that.
Quoted 7 lines
I think that is much simpler. Additionally it could also replace the need for the
has_stable_inodesandget_ino_and_lblk_bitsmethods, which are pretty weird. Instead just document the requirements for theSB_INLINE_CRYPTflag and handle the rest in the file system. That is for f2f always set it, and for ext4 set it based ons_inodes_count. Which brings me to:
I don't think combining these things is a good idea because it would restrict the use of inline encryption to filesystems that allow IV_INO_LBLK_64 encryption policies, i.e. filesystems that have stable inode numbers, 32-bit inodes, and 32-bit file logical block numbers.
The on-disk format (i.e. the type of encryption policy chosen) and the implementation (inline or filesystem-layer crypto) are really two separate things. This was one of the changes in v4 => v5 of this patchset; these two things used to be conflated but now they are separate. Now you can use inline encryption with the existing fscrypt policies too.
We could use two separate SB_* flags, like SB_INLINE_CRYPT and SB_IV_INO_LBLK_64_SUPPORT. However, the ->has_stable_inodes() and ->get_ino_and_lblk_bits() methods are nice because they separate the filesystem properties from the question of "is this encryption policy supported". Declaring the filesystem properties is easier to do because it doesn't require any fscrypt-specific knowledge. Also, fs/crypto/ could use these properties in different ways in the future, e.g. if another IV generation scheme is added.
Quoted 11 lines
Quoted 7 lines
Quoted 3 lines
Btw, I'm not happy about the 8-byte IV assumptions everywhere here. That really should be a parameter, not hardcoded.
To be clear, the 8-byte IV assumption doesn't really come from
fs/crypto/, but rather in what the blk-crypto API provides. If blk-crypto were to provide longer IV support,fs/crypto/would pretty easily be able to make use of it.That's what I meant - we hardcode the value in fscrypt. Instead we need to expose the size from blk-crypt and check for it.
If we add variable-length IV support, I think it should be handled in the same way as crypto algorithms. The current model is that the bio submitter doesn't need to check whether the crypto algorithm is supported by the device, because blk-crypto will fall back to the crypto API if needed. Unsupported IV lengths could be handled in the same way.
- Eric
Re: [PATCH v5 7/9] fscrypt: add inline encryption support
Quoted 9 lines · Eric Biggers
It's more important to clean up the
IS_ENCRYPTED(inode)&&S_ISREG(inode->i_mode)checks that are duplicated infs/{ext4,f2fs}/, so I've been thinking of adding a helper:static inline bool
fscrypt_needs_contents_encryption(const struct inode *inode){ return IS_ENABLED(CONFIG_FS_ENCRYPTION) && IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode); }
Sounds fine.
Quoted 5 lines
I don't think combining these things is a good idea because it would restrict the use of inline encryption to filesystems that allow
IV_INO_LBLK_64encryption policies, i.e. filesystems that have stable inode numbers, 32-bit inodes, and 32-bit file logical block numbers.
Quoted 8 lines
The on-disk format (i.e. the type of encryption policy chosen) and the implementation (inline or filesystem-layer crypto) are really two separate things. This was one of the changes in v4 => v5 of this patchset; these two things used to be conflated but now they are separate. Now you can use inline encryption with the existing fscrypt policies too.
We could use two separate SB_* flags, like
SB_INLINE_CRYPTandSB_IV_INO_LBLK_64_SUPPORT.
Yes, I think that is a good idea.
Quoted 6 lines
However, the ->has_stable_inodes() and
->get_ino_and_lblk_bits() methods are nice because they separate the filesystem properties from the question of "is this encryption policy supported". Declaring the filesystem properties is easier to do because it doesn't require any fscrypt-specific knowledge. Also,fs/crypto/could use these properties in different ways in the future, e.g. if another IV generation scheme is added.
I don't really like writing up method boilerplates for something that is a simple boolean flag.
Eric Biggers
maintainerRe: [PATCH v5 7/9] fscrypt: add inline encryption support
Quoted 23 lines · Christoph Hellwig
Quoted 5 lines
I don't think combining these things is a good idea because it would restrict the use of inline encryption to filesystems that allow
IV_INO_LBLK_64encryption policies, i.e. filesystems that have stable inode numbers, 32-bit inodes, and 32-bit file logical block numbers.Quoted 8 lines
The on-disk format (i.e. the type of encryption policy chosen) and the implementation (inline or filesystem-layer crypto) are really two separate things. This was one of the changes in v4 => v5 of this patchset; these two things used to be conflated but now they are separate. Now you can use inline encryption with the existing fscrypt policies too.
We could use two separate SB_* flags, like
SB_INLINE_CRYPTandSB_IV_INO_LBLK_64_SUPPORT.Yes, I think that is a good idea.
Quoted 6 lines
However, the ->has_stable_inodes() and
->get_ino_and_lblk_bits() methods are nice because they separate the filesystem properties from the question of "is this encryption policy supported". Declaring the filesystem properties is easier to do because it doesn't require any fscrypt-specific knowledge. Also,fs/crypto/could use these properties in different ways in the future, e.g. if another IV generation scheme is added.
Quoted 2 lines
I don't really like writing up method boilerplates for something that is a simple boolean flag.
fs/crypto/ uses ->has_stable_inodes() and ->get_ino_and_lblk_bits() to print an appropriate error message. If we changed it to a simple flag we'd have to print a less useful error message. Also, people are basically guaranteed to not understand what "SB_IV_INO_LBLK_64_SUPPORT" means exactly, and are likely to copy-and-paste it incorrectly when adding fscrypt support to a new filesystem. Also it would make it more difficult to add other fscrypt IV generation schemes in the future as we'd then need to add another sb flag (e.g. SB_IV_INO_LBLK_128) and make filesystem-specific changes, rather than change fs/crypto/ only.
So personally I'd prefer to keep ->has_stable_inodes() and ->get_ino_and_lblk_bits() for now.
Replacing ->inline_crypt_enabled() with SB_INLINE_CRYPT makes much more sense though.
- Eric
Eric Biggers
maintainerRe: [PATCH v5 7/9] fscrypt: add inline encryption support
In reply to Christoph HellwigQuoted 24 lines · Christoph Hellwig
Quoted 7 lines
Quoted 3 lines
Btw, I'm not happy about the 8-byte IV assumptions everywhere here. That really should be a parameter, not hardcoded.
To be clear, the 8-byte IV assumption doesn't really come from
fs/crypto/, but rather in what the blk-crypto API provides. If blk-crypto were to provide longer IV support,fs/crypto/would pretty easily be able to make use of it.That's what I meant - we hardcode the value in fscrypt. Instead we need to expose the size from blk-crypt and check for it.
Quoted 13 lines
(And if IVs >= 24 bytes were supported and we added AES-128-CBC-ESSIV and Adiantum support to
blk-crypto.c, then inline encryption would be able to do everything that the existing filesystem-layer contents encryption can do.)Do you have anything in mind for how to make the API support longer IVs in a clean way? Are you thinking of something like:
#define BLK_CRYPTO_MAX_DUN_SIZE 24 u8 dun[BLK_CRYPTO_MAX_DUN_SIZE]; int dun_size;
Quoted 8 lines
Quoted 2 lines
We do have to perform arithmetic operations on it, so a byte array would make it ugly and slower, but it should be possible...
Well, we could make it an array of u64s, which means we can do all the useful arithmetics on components on one of them. But I see the point, this adds significant complexity for no real short term gain, and we should probably postponed it until needed. Maybe just document the assumptions a little better.
Just in case it's not obvious to anyone, I should also mention that being limited to specifying a 64-bit DUN doesn't prevent hardware that accepts a longer IV (e.g. 128 bits) from being used. It would just be a matter of zero-padding the IV in the driver rather than in hardware.
The actual limitation we're talking about here is in the range of IVs that can be specified. A 64-bit DUN only allows the first 64 bits of the IV to be nonzero. That works for fscrypt in all cases except DIRECT_KEY policies, and it would work for dm-crypt using the usual dm-crypt IV generator (plain64).
But for inline encryption to be compatible with DIRECT_KEY fscrypt policies or with certain other dm-crypt IV generators, we'd need the ability to specify more IV bits.
- Eric
PATCH v5 8/9 f2fs: add inline encryption support
In reply to Satya TangiralaWire up f2fs to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:
- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
--- fs/f2fs/data.c | 76 +++++++++++++++++++++++++++++++++++++++++++------ fs/f2fs/f2fs.h | 3 ++ fs/f2fs/super.c | 20 +++++++++++++ 3 files changed, 91 insertions(+), 8 deletions(-)
fs/f2fs/data.c +68 -8
@@ -306,6 +306,35 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)return bio; } +static int f2fs_set_bio_crypt_ctx (struct bio *bio,const struct inode *inode, +pgoff_t first_idx, +const struct f2fs_io_info *fio, +gfp_t gfp_mask) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (fio && fio->encrypted_page) +return 0 ; + +return fscrypt_set_bio_crypt_ctx (bio, inode, first_idx, gfp_mask); +} + +static bool f2fs_crypt_mergeable_bio (struct bio *bio,const struct inode *inode, +pgoff_t next_idx, +const struct f2fs_io_info *fio) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (fio && fio->encrypted_page) +return true; + +return fscrypt_mergeable_bio (bio, inode, next_idx); +} +static inline void __submit_bio (struct f2fs_sb_info *sbi,struct bio *bio,enum page_type type) {
@@ -477,6 +506,7 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)struct bio *bio;struct page *page = fio->encrypted_page ? fio->encrypted_page : fio->page; +int err; if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr, fio->is_por ? META_POR : (__is_meta_io(fio) ?
@@ -489,6 +519,13 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)/* Allocate a new bio */ bio =__bio_alloc (fio,1 ); + err =f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio,GFP_NOIO ); +if (err) { +bio_put (bio); +return err; + } +if (bio_add_page (bio, page,PAGE_SIZE ,0 ) <PAGE_SIZE ) {bio_put (bio);return -EFAULT ;
@@ -556,14 +593,19 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)trace_f2fs_submit_page_bio (page, fio);f2fs_trace_ios (fio,0 ); -if (bio && !page_is_mergeable (fio->sbi, bio, *fio->last_block, - fio->new_blkaddr)) { +if (bio && (!page_is_mergeable (fio->sbi, bio, *fio->last_block, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (bio, fio->page->mapping->host, + fio->page->index, fio))) {__submit_bio (fio->sbi, bio, fio->type); bio =NULL ; } alloc_new:if (!bio) { bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO | __GFP_NOFAIL);bio_set_op_attrs (bio, fio->op, fio->op_flags); }
@@ -629,8 +671,11 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)inc_page_count (sbi,WB_DATA_TYPE (bio_page)); -if (io->bio && !io_is_mergeable (sbi, io->bio, io, fio, - io->last_block_in_bio, fio->new_blkaddr)) +if (io->bio && + (!io_is_mergeable (sbi, io->bio, io, fio, io->last_block_in_bio, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (io->bio, fio->page->mapping->host, + fio->page->index, fio)))__submit_merged_bio (io); alloc_new:if (io->bio ==NULL ) {
@@ -642,6 +687,9 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio) goto skip; } io->bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (io->bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO | __GFP_NOFAIL); io->fio = *fio; }
@@ -681,15 +729,23 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode, block_t blkaddr,struct bio *bio;struct bio_post_read_ctx *ctx;unsigned int post_read_steps =0 ; +int err; bio =f2fs_bio_alloc (sbi,min_t (int, nr_pages,BIO_MAX_PAGES ), false);if (!bio)return ERR_PTR (-ENOMEM ); + + err =f2fs_set_bio_crypt_ctx (bio, inode, first_idx,NULL ,GFP_NOFS ); +if (err) { +bio_put (bio); +return ERR_PTR (err); + } +f2fs_target_device (sbi, blkaddr, bio); bio->bi_end_io = f2fs_read_end_io;bio_set_op_attrs (bio,REQ_OP_READ , op_flag); -if (f2fs_encrypted_file (inode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (f2fs_need_verity (inode, first_idx))
@@ -1726,8 +1782,9 @@ static int f2fs_read_single_page(struct inode *inode, struct page *page,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && !page_is_mergeable (F2FS_I_SB (inode), bio, - *last_block_in_bio, block_nr)) { +if (bio && (!page_is_mergeable (F2FS_I_SB (inode), bio, + *last_block_in_bio, block_nr) || + !f2fs_crypt_mergeable_bio (bio, inode, page->index,NULL ))) { submit_and_realloc:__submit_bio (F2FS_I_SB (inode), bio,DATA ); bio =NULL ;
@@ -1867,6 +1924,9 @@ static int encrypt_one_page(struct f2fs_io_info *fio)/* wait for GCed page writeback via META_MAPPING */ f2fs_wait_on_block_writeback (inode, fio->old_blkaddr); +if (fscrypt_inode_uses_inline_crypto (inode)) +return 0 ; + retry_encrypt: fio->encrypted_page = fscrypt_encrypt_pagecache_blocks(fio->page, PAGE_SIZE, 0,
@@ -2041,7 +2101,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)f2fs_unlock_op (fio->sbi); err =f2fs_inplace_write_data (fio);if (err) { -if (f2fs_encrypted_file (inode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode))fscrypt_finalize_bounce_page (&fio->encrypted_page);if (PageWriteback (page))end_page_writeback (page);
fs/f2fs/f2fs.h +3 -0
@@ -137,6 +137,9 @@ struct f2fs_mount_info {int alloc_mode;/* segment allocation policy */ int fsync_mode;/* fsync policy */ bool test_dummy_encryption;/* test dummy encryption */ +#ifdef CONFIG_FS_ENCRYPTION +bool inlinecrypt;/* inline encryption enabled */ +#endif block_t unusable_cap;/* Amount of space allowed to be * unusable when disabling checkpoint */
fs/f2fs/super.c +20 -0
@@ -137,6 +137,7 @@ enum { Opt_alloc, Opt_fsync, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_checkpoint_disable, Opt_checkpoint_disable_cap, Opt_checkpoint_disable_cap_perc,
@@ -199,6 +200,7 @@ static match_table_t f2fs_tokens = { {Opt_alloc, "alloc_mode=%s"}, {Opt_fsync, "fsync_mode=%s"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_checkpoint_disable, "checkpoint=disable"}, {Opt_checkpoint_disable_cap, "checkpoint=disable:%u"}, {Opt_checkpoint_disable_cap_perc, "checkpoint=disable:%u%%"},
@@ -783,6 +785,13 @@ static int parse_options(struct super_block *sb, char *options)f2fs_info (sbi,"Test dummy encryption mode enabled" );#else f2fs_info (sbi,"Test dummy encryption mount option ignored" ); +#endif +break ; +case Opt_inlinecrypt: +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT +F2FS_OPTION (sbi).inlinecrypt = true; +#else +f2fs_info (sbi,"inline encryption not supported" );#endif break ;case Opt_checkpoint_disable_cap_perc:
@@ -1438,6 +1447,8 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root)#ifdef CONFIG_FS_ENCRYPTION if (F2FS_OPTION (sbi).test_dummy_encryption)seq_puts (seq,",test_dummy_encryption" ); +if (F2FS_OPTION (sbi).inlinecrypt) +seq_puts (seq,",inlinecrypt" );#endif if (F2FS_OPTION (sbi).alloc_mode ==ALLOC_MODE_DEFAULT )
@@ -1466,6 +1477,9 @@ static void default_options(struct f2fs_sb_info *sbi)F2FS_OPTION (sbi).alloc_mode =ALLOC_MODE_DEFAULT ;F2FS_OPTION (sbi).fsync_mode =FSYNC_MODE_POSIX ;F2FS_OPTION (sbi).test_dummy_encryption = false; +#ifdef CONFIG_FS_ENCRYPTION +F2FS_OPTION (sbi).inlinecrypt = false; +#endif F2FS_OPTION (sbi).s_resuid =make_kuid (&init_user_ns,F2FS_DEF_RESUID );F2FS_OPTION (sbi).s_resgid =make_kgid (&init_user_ns,F2FS_DEF_RESGID );
@@ -2320,6 +2334,11 @@ static void f2fs_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (block_t); } +static bool f2fs_inline_crypt_enabled (struct super_block *sb) +{ +return F2FS_OPTION (F2FS_SB (sb)).inlinecrypt; +} +static const struct fscrypt_operations f2fs_cryptops = { .key_prefix ="f2fs:" , .get_context = f2fs_get_context,
@@ -2329,6 +2348,7 @@ static const struct fscrypt_operations f2fs_cryptops = { .max_namelen = F2FS_NAME_LEN, .has_stable_inodes = f2fs_has_stable_inodes, .get_ino_and_lblk_bits = f2fs_get_ino_and_lblk_bits, + .inline_crypt_enabled = f2fs_inline_crypt_enabled, };#endif
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v5 8/9] f2fs: add inline encryption support
Quoted 9 lines · Satya Tangirala
Wire up f2fs to support inline encryption via the helper functions which
fs/crypto/now provides. This includes:- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the
bio_crypt_ctxon bios that will be submitted to an inline-encrypted file.
Quoted 8 lines
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
Co-developed-by Eric Biggers <[email protected]>Signed-off-by Eric Biggers <[email protected]>Signed-off-by Satya Tangirala <[email protected]>
Quoted 259 lines
--- fs/f2fs/data.c | 76 +++++++++++++++++++++++++++++++++++++++++++------ fs/f2fs/f2fs.h | 3 ++ fs/f2fs/super.c | 20 +++++++++++++ 3 files changed, 91 insertions(+), 8 deletions(-)fs/f2fs/data.c +68
-8@@ -306,6 +306,35 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)return bio; } +static int f2fs_set_bio_crypt_ctx (struct bio *bio,const struct inode *inode, +pgoff_t first_idx, +const struct f2fs_io_info *fio, +gfp_t gfp_mask) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (fio && fio->encrypted_page) +return 0 ; + +return fscrypt_set_bio_crypt_ctx (bio, inode, first_idx, gfp_mask); +} + +static bool f2fs_crypt_mergeable_bio (struct bio *bio,const struct inode *inode, +pgoff_t next_idx, +const struct f2fs_io_info *fio) +{ +/* +* The f2fs garbage collector sets ->encrypted_page when it wants to +* read/write raw data without encryption. +*/ +if (fio && fio->encrypted_page) +return true; + +return fscrypt_mergeable_bio (bio, inode, next_idx); +} +static inline void __submit_bio (struct f2fs_sb_info *sbi,struct bio *bio,enum page_type type) {@@ -477,6 +506,7 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)struct bio *bio;struct page *page = fio->encrypted_page ? fio->encrypted_page : fio->page; +int err; if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr, fio->is_por ? META_POR : (__is_meta_io(fio) ?@@ -489,6 +519,13 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)/* Allocate a new bio */ bio =__bio_alloc (fio,1 ); + err =f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio,GFP_NOIO ); +if (err) { +bio_put (bio); +return err; + } +if (bio_add_page (bio, page,PAGE_SIZE ,0 ) <PAGE_SIZE ) {bio_put (bio);return -EFAULT ;@@ -556,14 +593,19 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)trace_f2fs_submit_page_bio (page, fio);f2fs_trace_ios (fio,0 ); -if (bio && !page_is_mergeable (fio->sbi, bio, *fio->last_block, - fio->new_blkaddr)) { +if (bio && (!page_is_mergeable (fio->sbi, bio, *fio->last_block, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (bio, fio->page->mapping->host, + fio->page->index, fio))) {__submit_bio (fio->sbi, bio, fio->type); bio =NULL ; } alloc_new:if (!bio) { bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO | __GFP_NOFAIL);bio_set_op_attrs (bio, fio->op, fio->op_flags); }@@ -629,8 +671,11 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)inc_page_count (sbi,WB_DATA_TYPE (bio_page)); -if (io->bio && !io_is_mergeable (sbi, io->bio, io, fio, - io->last_block_in_bio, fio->new_blkaddr)) +if (io->bio && + (!io_is_mergeable (sbi, io->bio, io, fio, io->last_block_in_bio, + fio->new_blkaddr) || + !f2fs_crypt_mergeable_bio (io->bio, fio->page->mapping->host, + fio->page->index, fio)))__submit_merged_bio (io); alloc_new:if (io->bio ==NULL ) {@@ -642,6 +687,9 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio) goto skip; } io->bio =__bio_alloc (fio,BIO_MAX_PAGES ); +f2fs_set_bio_crypt_ctx (io->bio, fio->page->mapping->host, + fio->page->index, fio, +GFP_NOIO | __GFP_NOFAIL); io->fio = *fio; }@@ -681,15 +729,23 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode, block_t blkaddr,struct bio *bio;struct bio_post_read_ctx *ctx;unsigned int post_read_steps =0 ; +int err; bio =f2fs_bio_alloc (sbi,min_t (int, nr_pages,BIO_MAX_PAGES ), false);if (!bio)return ERR_PTR (-ENOMEM ); + + err =f2fs_set_bio_crypt_ctx (bio, inode, first_idx,NULL ,GFP_NOFS ); +if (err) { +bio_put (bio); +return ERR_PTR (err); + } +f2fs_target_device (sbi, blkaddr, bio); bio->bi_end_io = f2fs_read_end_io;bio_set_op_attrs (bio,REQ_OP_READ , op_flag); -if (f2fs_encrypted_file (inode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (f2fs_need_verity (inode, first_idx))@@ -1726,8 +1782,9 @@ static int f2fs_read_single_page(struct inode *inode, struct page *page,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && !page_is_mergeable (F2FS_I_SB (inode), bio, - *last_block_in_bio, block_nr)) { +if (bio && (!page_is_mergeable (F2FS_I_SB (inode), bio, + *last_block_in_bio, block_nr) || + !f2fs_crypt_mergeable_bio (bio, inode, page->index,NULL ))) { submit_and_realloc:__submit_bio (F2FS_I_SB (inode), bio,DATA ); bio =NULL ;@@ -1867,6 +1924,9 @@ static int encrypt_one_page(struct f2fs_io_info *fio)/* wait for GCed page writeback via META_MAPPING */ f2fs_wait_on_block_writeback (inode, fio->old_blkaddr); +if (fscrypt_inode_uses_inline_crypto (inode)) +return 0 ; + retry_encrypt: fio->encrypted_page = fscrypt_encrypt_pagecache_blocks(fio->page, PAGE_SIZE, 0,@@ -2041,7 +2101,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)f2fs_unlock_op (fio->sbi); err =f2fs_inplace_write_data (fio);if (err) { -if (f2fs_encrypted_file (inode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode))fscrypt_finalize_bounce_page (&fio->encrypted_page);if (PageWriteback (page))end_page_writeback (page);fs/f2fs/f2fs.h +3
-0@@ -137,6 +137,9 @@ struct f2fs_mount_info {int alloc_mode;/* segment allocation policy */ int fsync_mode;/* fsync policy */ bool test_dummy_encryption;/* test dummy encryption */ +#ifdef CONFIG_FS_ENCRYPTION +bool inlinecrypt;/* inline encryption enabled */ +#endif block_t unusable_cap;/* Amount of space allowed to be * unusable when disabling checkpoint */ fs/f2fs/super.c +20
-0@@ -137,6 +137,7 @@ enum { Opt_alloc, Opt_fsync, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_checkpoint_disable, Opt_checkpoint_disable_cap, Opt_checkpoint_disable_cap_perc,@@ -199,6 +200,7 @@ static match_table_t f2fs_tokens = { {Opt_alloc, "alloc_mode=%s"}, {Opt_fsync, "fsync_mode=%s"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_checkpoint_disable, "checkpoint=disable"}, {Opt_checkpoint_disable_cap, "checkpoint=disable:%u"}, {Opt_checkpoint_disable_cap_perc, "checkpoint=disable:%u%%"},@@ -783,6 +785,13 @@ static int parse_options(struct super_block *sb, char *options)f2fs_info (sbi,"Test dummy encryption mode enabled" );#else f2fs_info (sbi,"Test dummy encryption mount option ignored" ); +#endif +break ; +case Opt_inlinecrypt: +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT +F2FS_OPTION (sbi).inlinecrypt = true; +#else +f2fs_info (sbi,"inline encryption not supported" );#endif break ;case Opt_checkpoint_disable_cap_perc:@@ -1438,6 +1447,8 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root)#ifdef CONFIG_FS_ENCRYPTION if (F2FS_OPTION (sbi).test_dummy_encryption)seq_puts (seq,",test_dummy_encryption" ); +if (F2FS_OPTION (sbi).inlinecrypt) +seq_puts (seq,",inlinecrypt" );#endif if (F2FS_OPTION (sbi).alloc_mode ==ALLOC_MODE_DEFAULT )@@ -1466,6 +1477,9 @@ static void default_options(struct f2fs_sb_info *sbi)F2FS_OPTION (sbi).alloc_mode =ALLOC_MODE_DEFAULT ;F2FS_OPTION (sbi).fsync_mode =FSYNC_MODE_POSIX ;F2FS_OPTION (sbi).test_dummy_encryption = false; +#ifdef CONFIG_FS_ENCRYPTION +F2FS_OPTION (sbi).inlinecrypt = false; +#endif F2FS_OPTION (sbi).s_resuid =make_kuid (&init_user_ns,F2FS_DEF_RESUID );F2FS_OPTION (sbi).s_resgid =make_kgid (&init_user_ns,F2FS_DEF_RESGID );@@ -2320,6 +2334,11 @@ static void f2fs_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (block_t); } +static bool f2fs_inline_crypt_enabled (struct super_block *sb) +{ +return F2FS_OPTION (F2FS_SB (sb)).inlinecrypt; +} +static const struct fscrypt_operations f2fs_cryptops = { .key_prefix ="f2fs:" , .get_context = f2fs_get_context,@@ -2329,6 +2348,7 @@ static const struct fscrypt_operations f2fs_cryptops = { .max_namelen = F2FS_NAME_LEN, .has_stable_inodes = f2fs_has_stable_inodes, .get_ino_and_lblk_bits = f2fs_get_ino_and_lblk_bits, + .inline_crypt_enabled = f2fs_inline_crypt_enabled, };#endif Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
PATCH v5 9/9 ext4: add inline encryption support
In reply to Satya TangiralaFrom: Eric Biggers <[email protected]>
Wire up ext4 to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:
- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.
Note: submit_bh_wbc() in fs/buffer.c also needed to be patched for this part, since ext4 sometimes uses ll_rw_block() on file data.
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
--- fs/buffer.c | 3 +++ fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 4 ++-- fs/ext4/page-io.c | 11 +++++++++-- fs/ext4/readpage.c | 15 ++++++++++++--- fs/ext4/super.c | 13 +++++++++++++ 6 files changed, 40 insertions(+), 7 deletions(-)
fs/buffer.c +3 -0
@@ -47,6 +47,7 @@#include <linux/pagevec.h> #include <linux/sched/mm.h> #include <trace/events/block.h> +#include <linux/fscrypt.h> static int fsync_buffers_list (spinlock_t *lock,struct list_head *list); static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
@@ -3068,6 +3069,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,*/ bio =bio_alloc (GFP_NOIO ,1 ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO | __GFP_NOFAIL); + bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_write_hint = write_hint;
fs/ext4/ext4.h +1 -0
@@ -1148,6 +1148,7 @@ struct ext4_inode_info {#define EXT4_MOUNT_JOURNAL_CHECKSUM 0x800000/* Journal checksums */ #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT 0x1000000/* Journal Async Commit */ #define EXT4_MOUNT_WARN_ON_ERROR 0x2000000/* Trigger WARN_ON on error */ +#define EXT4_MOUNT_INLINECRYPT 0x4000000/* Inline encryption support */ #define EXT4_MOUNT_DELALLOC 0x8000000/* Delalloc support */ #define EXT4_MOUNT_DATA_ERR_ABORT 0x10000000/* Abort on file data write */ #define EXT4_MOUNT_BLOCK_VALIDITY 0x20000000/* Block validity checking */
fs/ext4/inode.c +2 -2
@@ -1237,7 +1237,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len, }if (unlikely (err)) {page_zero_new_buffers (page, from, to); - }else if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) { + }else if (fscrypt_inode_uses_fs_layer_crypto (inode)) {for (i =0 ; i < nr_wait; i++) {int err2;
@@ -4034,7 +4034,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,/* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate (bh)) goto unlock; -if (S_ISREG (inode->i_mode) &&IS_ENCRYPTED (inode)) { +if (fscrypt_inode_uses_fs_layer_crypto (inode)) {/* We expect the key to be set. */ BUG_ON (!fscrypt_has_encryption_key (inode)); WARN_ON_ONCE(fscrypt_decrypt_pagecache_blocks(
fs/ext4/page-io.c +9 -2
@@ -362,10 +362,16 @@ static int io_submit_init_bio(struct ext4_io_submit *io, struct buffer_head *bh) {struct bio *bio; +int err; bio =bio_alloc (GFP_NOIO ,BIO_MAX_PAGES );if (!bio)return -ENOMEM ; + err =fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); +if (err) { +bio_put (bio); +return err; + } bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_end_io = ext4_end_bio;
@@ -383,7 +389,8 @@ static int io_submit_add_bh(struct ext4_io_submit *io, {int ret; -if (io->io_bio && bh->b_blocknr != io->io_next_block) { +if (io->io_bio && (bh->b_blocknr != io->io_next_block || + !fscrypt_mergeable_bio_bh (io->io_bio, bh))) { submit_and_retry:ext4_io_submit (io); }
@@ -474,7 +481,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,* (e.g. holes) to be unnecessarily encrypted, but this is rare and * can't happen in the common case of blocksize == PAGE_SIZE. */ -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode) && nr_to_submit) { +if (fscrypt_inode_uses_fs_layer_crypto (inode) && nr_to_submit) {gfp_t gfp_flags =GFP_NOFS ;unsigned int enc_bytes =round_up (len,i_blocksize (inode));
fs/ext4/readpage.c +12 -3
@@ -183,7 +183,7 @@ static struct bio_post_read_ctx *get_bio_post_read_ctx(struct inode *inode,unsigned int post_read_steps =0 ;struct bio_post_read_ctx *ctx =NULL ; -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (ext4_need_verity (inode, first_idx))
@@ -220,6 +220,7 @@ int ext4_mpage_readpages(struct address_space *mapping,const unsigned blkbits = inode->i_blkbits;const unsigned blocks_per_page =PAGE_SIZE >> blkbits;const unsigned blocksize =1 << blkbits; +sector_t next_block;sector_t block_in_file;sector_t last_block;sector_t last_block_in_file;
@@ -252,7 +253,8 @@ int ext4_mpage_readpages(struct address_space *mapping,if (page_has_buffers (page)) goto confused; - block_in_file = (sector_t )page->index << (PAGE_SHIFT - blkbits); + block_in_file = next_block = + (sector_t )page->index << (PAGE_SHIFT - blkbits); last_block = block_in_file + nr_pages * blocks_per_page; last_block_in_file = (ext4_readpage_limit (inode) + blocksize -1 ) >> blkbits;
@@ -352,7 +354,8 @@ int ext4_mpage_readpages(struct address_space *mapping,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && (last_block_in_bio != blocks[0 ] -1 )) { +if (bio && (last_block_in_bio != blocks[0 ] -1 || + !fscrypt_mergeable_bio (bio, inode, next_block))) { submit_and_realloc:submit_bio (bio); bio =NULL ;
@@ -364,6 +367,12 @@ int ext4_mpage_readpages(struct address_space *mapping, min_t(int, nr_pages, BIO_MAX_PAGES));if (!bio) goto set_error_page; +if (fscrypt_set_bio_crypt_ctx (bio, inode, next_block, +GFP_KERNEL ) !=0 ) { +bio_put (bio); + bio =NULL ; + goto set_error_page; + } ctx =get_bio_post_read_ctx (inode, bio, page->index);if (IS_ERR (ctx)) {bio_put (bio);
fs/ext4/super.c +13 -0
@@ -1357,6 +1357,11 @@ static void ext4_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (ext4_lblk_t); } +static bool ext4_inline_crypt_enabled (struct super_block *sb) +{ +return test_opt (sb,INLINECRYPT ); +} +static const struct fscrypt_operations ext4_cryptops = { .key_prefix ="ext4:" , .get_context = ext4_get_context,
@@ -1366,6 +1371,7 @@ static const struct fscrypt_operations ext4_cryptops = { .max_namelen = EXT4_NAME_LEN, .has_stable_inodes = ext4_has_stable_inodes, .get_ino_and_lblk_bits = ext4_get_ino_and_lblk_bits, + .inline_crypt_enabled = ext4_inline_crypt_enabled, };#endif
@@ -1461,6 +1467,7 @@ enum { Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit, Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,
@@ -1557,6 +1564,7 @@ static const match_table_t tokens = { {Opt_noinit_itable, "noinit_itable"}, {Opt_max_dir_size_kb, "max_dir_size_kb=%u"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_nombcache, "nombcache"}, {Opt_nombcache, "no_mbcache"}, /* for backward compatibility */ {Opt_removed, "check=none"}, /* mount option from ext2/3 */
@@ -1768,6 +1776,11 @@ static const struct mount_opts { {Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT}, {Opt_max_dir_size_kb, 0, MOPT_GTE0}, {Opt_test_dummy_encryption, 0, MOPT_GTE0}, +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET}, +#else + {Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT}, +#endif {Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET}, {Opt_err, 0, 0} };
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog