blk-crypto cleanups
Subsystems (6)
v5 cover blk-crypto cleanups
This series renames struct blk_keyslot_manager to struct blk_crypto_profile, as it is misnamed; it doesn't always manage keyslots. It's much more logical to think of it as the "blk-crypto profile" of a device, similar to blk_integrity_profile.
This series also improves the inline-encryption.rst documentation file, and cleans up blk-crypto-fallback a bit.
This series applies to block/for-next.
Changed v4 => v5:
- Rebased onto block/for-next again
- Added Reviewed-by tags
Changed v3 => v4:
- Rebased onto block/for-next to resolve a conflict due to 'struct request' being moved.
Changed v2 => v3:
- Made some minor tweaks to patches 3 and 4, mostly comments and documentation.
- Clarified a commit message to mention no change in behavior.
- Added a Reviewed-by tag.
Changed v1 => v2:
- Fixed a build error in blk-integrity.c.
- Removed a mention of "ksm" from a comment.
- Dropped the patch "blk-crypto-fallback: consolidate static variables".
- Added Acked-by and Reviewed-by tags.
Eric Biggers (4):
blk-crypto-fallback: properly prefix function and struct names
blk-crypto: rename keyslot-manager files to blk-crypto-profile
blk-crypto: rename blk_keyslot_manager to blk_crypto_profile
blk-crypto: update inline encryption documentation
Documentation/block/inline-encryption.rst | 451 +++++++++-------- block/Makefile | 2 +- block/blk-crypto-fallback.c | 118 ++--- block/blk-crypto-profile.c | 565 +++++++++++++++++++++ block/blk-crypto.c | 29 +- block/blk-integrity.c | 4 +- block/keyslot-manager.c | 579 ---------------------- drivers/md/dm-core.h | 4 +- drivers/md/dm-table.c | 168 +++---- drivers/md/dm.c | 10 +- drivers/mmc/core/crypto.c | 11 +- drivers/mmc/host/cqhci-crypto.c | 33 +- drivers/scsi/ufs/ufshcd-crypto.c | 32 +- drivers/scsi/ufs/ufshcd-crypto.h | 9 +- drivers/scsi/ufs/ufshcd.c | 2 +- drivers/scsi/ufs/ufshcd.h | 6 +- include/linux/blk-crypto-profile.h | 166 +++++++ include/linux/blk-mq.h | 2 +- include/linux/blkdev.h | 16 +- include/linux/device-mapper.h | 4 +- include/linux/keyslot-manager.h | 120 ----- include/linux/mmc/host.h | 4 +- 22 files changed, 1204 insertions(+), 1131 deletions(-) create mode 100644 block/blk-crypto-profile.c delete mode 100644 block/keyslot-manager.c create mode 100644 include/linux/blk-crypto-profile.h delete mode 100644 include/linux/keyslot-manager.h
base-commit: 960d083b6eca8f150fcd6dc7cf56b0005635b649
Signature · 1 line
2.33.0
PATCH v5 1/4 blk-crypto-fallback: properly prefix function and struct names
From: Eric Biggers <[email protected]>
For clarity, avoid using just the "blk_crypto_" prefix for functions and structs that are specific to blk-crypto-fallback. Instead, use "blk_crypto_fallback_". Some places already did this, but others didn't.
This is also a prerequisite for using "struct blk_crypto_keyslot" to mean a generic blk-crypto keyslot (which is what it sounds like). Rename the fallback one to "struct blk_crypto_fallback_keyslot".
No change in behavior.
Reviewed-by Christoph Hellwig <[email protected]>
Reviewed-by Chaitanya Kulkarni <[email protected]>
Reviewed-by Mike Snitzer <[email protected]>
Signed-off-by Eric Biggers <[email protected]>
--- block/blk-crypto-fallback.c | 59 +++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 29 deletions(-)
block/blk-crypto-fallback.c +30 -29
@@ -73,7 +73,7 @@ static mempool_t *bio_fallback_crypt_ctx_pool;static DEFINE_MUTEX (tfms_init_lock);static bool tfms_inited[BLK_ENCRYPTION_MODE_MAX ]; -static struct blk_crypto_keyslot { +static struct blk_crypto_fallback_keyslot {enum blk_crypto_mode_num crypto_mode;struct crypto_skcipher *tfms[BLK_ENCRYPTION_MODE_MAX ]; } *blk_crypto_keyslots;
@@ -89,9 +89,9 @@ static struct bio_set crypto_bio_split;*/ static u8 blank_key[BLK_CRYPTO_MAX_KEY_SIZE ]; -static void blk_crypto_evict_keyslot (unsigned int slot) +static void blk_crypto_fallback_evict_keyslot (unsigned int slot) { -struct blk_crypto_keyslot *slotp = &blk_crypto_keyslots[slot]; +struct blk_crypto_fallback_keyslot *slotp = &blk_crypto_keyslots[slot];enum blk_crypto_mode_num crypto_mode = slotp->crypto_mode;int err;
@@ -104,34 +104,34 @@ static void blk_crypto_evict_keyslot(unsigned int slot) slotp->crypto_mode =BLK_ENCRYPTION_MODE_INVALID ; } -static int blk_crypto_keyslot_program (struct blk_keyslot_manager *ksm, -const struct blk_crypto_key *key, -unsigned int slot) +static int blk_crypto_fallback_keyslot_program (struct blk_keyslot_manager *ksm, +const struct blk_crypto_key *key, +unsigned int slot) { -struct blk_crypto_keyslot *slotp = &blk_crypto_keyslots[slot]; +struct blk_crypto_fallback_keyslot *slotp = &blk_crypto_keyslots[slot];const enum blk_crypto_mode_num crypto_mode = key->crypto_cfg.crypto_mode;int err;if (crypto_mode != slotp->crypto_mode && slotp->crypto_mode !=BLK_ENCRYPTION_MODE_INVALID ) -blk_crypto_evict_keyslot (slot); +blk_crypto_fallback_evict_keyslot (slot); slotp->crypto_mode = crypto_mode; err =crypto_skcipher_setkey (slotp->tfms[crypto_mode], key->raw, key->size);if (err) { -blk_crypto_evict_keyslot (slot); +blk_crypto_fallback_evict_keyslot (slot);return err; }return 0 ; } -static int blk_crypto_keyslot_evict (struct blk_keyslot_manager *ksm, -const struct blk_crypto_key *key, -unsigned int slot) +static int blk_crypto_fallback_keyslot_evict (struct blk_keyslot_manager *ksm, +const struct blk_crypto_key *key, +unsigned int slot) { -blk_crypto_evict_keyslot (slot); +blk_crypto_fallback_evict_keyslot (slot);return 0 ; }
@@ -141,8 +141,8 @@ static int blk_crypto_keyslot_evict(struct blk_keyslot_manager *ksm,* hardware. */ static const struct blk_ksm_ll_ops blk_crypto_ksm_ll_ops = { - .keyslot_program = blk_crypto_keyslot_program, - .keyslot_evict = blk_crypto_keyslot_evict, + .keyslot_program = blk_crypto_fallback_keyslot_program, + .keyslot_evict = blk_crypto_fallback_keyslot_evict, };static void blk_crypto_fallback_encrypt_endio (struct bio *enc_bio)
@@ -160,7 +160,7 @@ static void blk_crypto_fallback_encrypt_endio(struct bio *enc_bio)bio_endio (src_bio); } -static struct bio *blk_crypto_clone_bio (struct bio *bio_src) +static struct bio *blk_crypto_fallback_clone_bio (struct bio *bio_src) {struct bvec_iter iter;struct bio_vec bv;
@@ -187,12 +187,13 @@ static struct bio *blk_crypto_clone_bio(struct bio *bio_src)return bio; } -static bool blk_crypto_alloc_cipher_req (struct blk_ksm_keyslot *slot, -struct skcipher_request **ciph_req_ret, -struct crypto_wait *wait) +static bool +blk_crypto_fallback_alloc_cipher_req (struct blk_ksm_keyslot *slot, +struct skcipher_request **ciph_req_ret, +struct crypto_wait *wait) {struct skcipher_request *ciph_req; -const struct blk_crypto_keyslot *slotp; +const struct blk_crypto_fallback_keyslot *slotp;int keyslot_idx =blk_ksm_get_slot_idx (slot); slotp = &blk_crypto_keyslots[keyslot_idx];
@@ -210,7 +211,7 @@ static bool blk_crypto_alloc_cipher_req(struct blk_ksm_keyslot *slot,return true; } -static bool blk_crypto_split_bio_if_needed (struct bio **bio_ptr) +static bool blk_crypto_fallback_split_bio_if_needed (struct bio **bio_ptr) {struct bio *bio = *bio_ptr;unsigned int i =0 ;
@@ -277,7 +278,7 @@ static bool blk_crypto_fallback_encrypt_bio(struct bio **bio_ptr)blk_status_t blk_st;/* Split the bio if it's too big for single page bvec */ -if (!blk_crypto_split_bio_if_needed (bio_ptr)) +if (!blk_crypto_fallback_split_bio_if_needed (bio_ptr))return false; src_bio = *bio_ptr;
@@ -285,7 +286,7 @@ static bool blk_crypto_fallback_encrypt_bio(struct bio **bio_ptr) data_unit_size = bc->bc_key->crypto_cfg.data_unit_size;/* Allocate bounce bio for encryption */ - enc_bio =blk_crypto_clone_bio (src_bio); + enc_bio =blk_crypto_fallback_clone_bio (src_bio);if (!enc_bio) { src_bio->bi_status =BLK_STS_RESOURCE ;return false;
@@ -302,7 +303,7 @@ static bool blk_crypto_fallback_encrypt_bio(struct bio **bio_ptr) }/* and then allocate an skcipher_request for it */ -if (!blk_crypto_alloc_cipher_req (slot, &ciph_req, &wait)) { +if (!blk_crypto_fallback_alloc_cipher_req (slot, &ciph_req, &wait)) { src_bio->bi_status =BLK_STS_RESOURCE ; goto out_release_keyslot; }
@@ -404,7 +405,7 @@ static void blk_crypto_fallback_decrypt_bio(struct work_struct *work) }/* and then allocate an skcipher_request for it */ -if (!blk_crypto_alloc_cipher_req (slot, &ciph_req, &wait)) { +if (!blk_crypto_fallback_alloc_cipher_req (slot, &ciph_req, &wait)) { bio->bi_status =BLK_STS_RESOURCE ; goto out; }
@@ -474,9 +475,9 @@ static void blk_crypto_fallback_decrypt_endio(struct bio *bio)* @bio_ptr: pointer to the bio to prepare * * If bio is doing a WRITE operation, this splits the bio into two parts if it's -* too big (see blk_crypto_split_bio_if_needed). It then allocates a bounce bio -* for the first part, encrypts it, and update bio_ptr to point to the bounce -* bio. +* too big (see blk_crypto_fallback_split_bio_if_needed()). It then allocates a +* bounce bio for the first part, encrypts it, and updates bio_ptr to point to +* the bounce bio. * * For a READ operation, we mark the bio for decryption by using bi_private and * bi_end_io.
@@ -611,7 +612,7 @@ static int blk_crypto_fallback_init(void)int blk_crypto_fallback_start_using_mode (enum blk_crypto_mode_num mode_num) {const char *cipher_str = blk_crypto_modes[mode_num].cipher_str; -struct blk_crypto_keyslot *slotp; +struct blk_crypto_fallback_keyslot *slotp;unsigned int i;int err =0 ;
Signature · 1 line
2.33.0
PATCH v5 2/4 blk-crypto: rename keyslot-manager files to blk-crypto-profile
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
In preparation for renaming struct blk_keyslot_manager to struct blk_crypto_profile, rename the keyslot-manager.h and keyslot-manager.c source files. Renaming these files separately before making a lot of changes to their contents makes it easier for git to understand that they were renamed.
Reviewed-by Christoph Hellwig <[email protected]>
Reviewed-by Mike Snitzer <[email protected]>
Signed-off-by Eric Biggers <[email protected]>
---
block/Makefile | 2 +-
block/blk-crypto-fallback.c | 2 +-
block/{keyslot-manager.c => blk-crypto-profile.c} | 2 +-
block/blk-crypto.c | 2 +-
drivers/md/dm-core.h | 2 +-
drivers/md/dm.c | 2 +-
drivers/mmc/host/cqhci-crypto.c | 2 +-
drivers/scsi/ufs/ufshcd.h | 2 +-
include/linux/{keyslot-manager.h => blk-crypto-profile.h} | 0
include/linux/mmc/host.h | 2 +-
10 files changed, 9 insertions(+), 9 deletions(-)
rename block/{keyslot-manager.c => blk-crypto-profile.c} (99%)
rename include/linux/{keyslot-manager.h => blk-crypto-profile.h} (100%)block/Makefile +1 -1
@@ -36,6 +36,6 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o blk-crypto.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += blk-crypto.o blk-crypto-profile.o obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) += blk-crypto-fallback.o obj-$(CONFIG_BLOCK_HOLDER_DEPRECATED) += holder.o
block/blk-crypto-fallback.c +1 -1
@@ -12,9 +12,9 @@#include <crypto/skcipher.h> #include <linux/blk-cgroup.h> #include <linux/blk-crypto.h> +#include <linux/blk-crypto-profile.h> #include <linux/blkdev.h> #include <linux/crypto.h> -#include <linux/keyslot-manager.h> #include <linux/mempool.h> #include <linux/module.h> #include <linux/random.h>
block/blk-crypto-profile.c renamed from block/keyslot-manager.c +1 -1
@@ -28,7 +28,7 @@#define pr_fmt (fmt) "blk-crypto: " fmt -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> #include <linux/device.h> #include <linux/atomic.h> #include <linux/mutex.h>
block/blk-crypto.c +1 -1
@@ -11,7 +11,7 @@#include <linux/bio.h> #include <linux/blkdev.h> -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> #include <linux/module.h> #include <linux/slab.h>
drivers/md/dm-core.h +1 -1
@@ -13,7 +13,7 @@#include <linux/ktime.h> #include <linux/genhd.h> #include <linux/blk-mq.h> -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> #include <trace/events/block.h>
drivers/md/dm.c +1 -1
@@ -29,7 +29,7 @@#include <linux/refcount.h> #include <linux/part_stat.h> #include <linux/blk-crypto.h> -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> #define DM_MSG_PREFIX "core"
drivers/mmc/host/cqhci-crypto.c +1 -1
@@ -6,7 +6,7 @@*/ #include <linux/blk-crypto.h> -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> #include <linux/mmc/host.h> #include "cqhci-crypto.h"
drivers/scsi/ufs/ufshcd.h +1 -1
@@ -32,7 +32,7 @@#include <linux/regulator/consumer.h> #include <linux/bitfield.h> #include <linux/devfreq.h> -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> #include "unipro.h" #include <asm/irq.h>
include/linux/blk-crypto-profile.h renamed from include/linux/keyslot-manager.h +0 -0
include/linux/mmc/host.h +1 -1
@@ -15,7 +15,7 @@#include <linux/mmc/card.h> #include <linux/mmc/pm.h> #include <linux/dma-direction.h> -#include <linux/keyslot-manager.h> +#include <linux/blk-crypto-profile.h> struct mmc_ios {unsigned int clock;/* clock rate */
Signature · 1 line
2.33.0
PATCH v5 3/4 blk-crypto: rename blk_keyslot_manager to blk_crypto_profile
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
blk_keyslot_manager is misnamed because it doesn't necessarily manage keyslots. It actually does several different things:
- Contains the crypto capabilities of the device.
- Provides functions to control the inline encryption hardware. Originally these were just for programming/evicting keyslots; however, new functionality (hardware-wrapped keys) will require new functions here which are unrelated to keyslots. Moreover, device-mapper devices already (ab)use "keyslot_evict" to pass key eviction requests to their underlying devices even though device-mapper devices don't have any keyslots themselves (so it really should be "evict_key", not "keyslot_evict").
- Sometimes (but not always!) it manages keyslots. Originally it always did, but device-mapper devices don't have keyslots themselves, so they use a "passthrough keyslot manager" which doesn't actually manage keyslots. This hack works, but the terminology is unnatural. Also, some hardware doesn't have keyslots and thus also uses a "passthrough keyslot manager" (support for such hardware is yet to be upstreamed, but it will happen eventually).
Let's stop having keyslot managers which don't actually manage keyslots. Instead, rename blk_keyslot_manager to blk_crypto_profile.
This is a fairly big change, since for consistency it also has to update keyslot manager-related function names, variable names, and comments -- not just the actual struct name. However it's still a fairly straightforward change, as it doesn't change any actual functionality.
Reviewed-by Mike Snitzer <[email protected]>
Signed-off-by Eric Biggers <[email protected]>
--- block/blk-crypto-fallback.c | 71 ++-- block/blk-crypto-profile.c | 520 ++++++++++++++--------------- block/blk-crypto.c | 27 +- block/blk-integrity.c | 4 +- drivers/md/dm-core.h | 2 +- drivers/md/dm-table.c | 168 +++++----- drivers/md/dm.c | 8 +- drivers/mmc/core/crypto.c | 11 +- drivers/mmc/host/cqhci-crypto.c | 31 +- drivers/scsi/ufs/ufshcd-crypto.c | 32 +- drivers/scsi/ufs/ufshcd-crypto.h | 9 +- drivers/scsi/ufs/ufshcd.c | 2 +- drivers/scsi/ufs/ufshcd.h | 4 +- include/linux/blk-crypto-profile.h | 164 +++++---- include/linux/blk-mq.h | 2 +- include/linux/blkdev.h | 16 +- include/linux/device-mapper.h | 4 +- include/linux/mmc/host.h | 2 +- 18 files changed, 555 insertions(+), 522 deletions(-)
block/blk-crypto-fallback.c +35 -36
@@ -78,7 +78,7 @@ static struct blk_crypto_fallback_keyslot {struct crypto_skcipher *tfms[BLK_ENCRYPTION_MODE_MAX ]; } *blk_crypto_keyslots; -static struct blk_keyslot_manager blk_crypto_ksm; +static struct blk_crypto_profile blk_crypto_fallback_profile;static struct workqueue_struct *blk_crypto_wq;static mempool_t *blk_crypto_bounce_page_pool;static struct bio_set crypto_bio_split;
@@ -104,9 +104,10 @@ static void blk_crypto_fallback_evict_keyslot(unsigned int slot) slotp->crypto_mode =BLK_ENCRYPTION_MODE_INVALID ; } -static int blk_crypto_fallback_keyslot_program (struct blk_keyslot_manager *ksm, -const struct blk_crypto_key *key, -unsigned int slot) +static int +blk_crypto_fallback_keyslot_program (struct blk_crypto_profile *profile, +const struct blk_crypto_key *key, +unsigned int slot) {struct blk_crypto_fallback_keyslot *slotp = &blk_crypto_keyslots[slot]; const enum blk_crypto_mode_num crypto_mode =
@@ -127,7 +128,7 @@ static int blk_crypto_fallback_keyslot_program(struct blk_keyslot_manager *ksm,return 0 ; } -static int blk_crypto_fallback_keyslot_evict (struct blk_keyslot_manager *ksm, +static int blk_crypto_fallback_keyslot_evict (struct blk_crypto_profile *profile,const struct blk_crypto_key *key,unsigned int slot) {
@@ -135,14 +136,9 @@ static int blk_crypto_fallback_keyslot_evict(struct blk_keyslot_manager *ksm,return 0 ; } -/* -* The crypto API fallback KSM ops - only used for a bio when it specifies a -* blk_crypto_key that was not supported by the device's inline encryption -* hardware. -*/ -static const struct blk_ksm_ll_ops blk_crypto_ksm_ll_ops = { - .keyslot_program = blk_crypto_fallback_keyslot_program, - .keyslot_evict = blk_crypto_fallback_keyslot_evict, +static const struct blk_crypto_ll_ops blk_crypto_fallback_ll_ops = { + .keyslot_program = blk_crypto_fallback_keyslot_program, + .keyslot_evict = blk_crypto_fallback_keyslot_evict, };static void blk_crypto_fallback_encrypt_endio (struct bio *enc_bio)
@@ -188,13 +184,13 @@ static struct bio *blk_crypto_fallback_clone_bio(struct bio *bio_src) }static bool -blk_crypto_fallback_alloc_cipher_req (struct blk_ksm_keyslot *slot, +blk_crypto_fallback_alloc_cipher_req (struct blk_crypto_keyslot *slot,struct skcipher_request **ciph_req_ret,struct crypto_wait *wait) {struct skcipher_request *ciph_req;const struct blk_crypto_fallback_keyslot *slotp; -int keyslot_idx =blk_ksm_get_slot_idx (slot); +int keyslot_idx =blk_crypto_keyslot_index (slot); slotp = &blk_crypto_keyslots[keyslot_idx]; ciph_req = skcipher_request_alloc(slotp->tfms[slotp->crypto_mode],
@@ -266,7 +262,7 @@ static bool blk_crypto_fallback_encrypt_bio(struct bio **bio_ptr) {struct bio *src_bio, *enc_bio;struct bio_crypt_ctx *bc; -struct blk_ksm_keyslot *slot; +struct blk_crypto_keyslot *slot;int data_unit_size;struct skcipher_request *ciph_req =NULL ;DECLARE_CRYPTO_WAIT (wait);
@@ -293,10 +289,11 @@ static bool blk_crypto_fallback_encrypt_bio(struct bio **bio_ptr) }/* -* Use the crypto API fallback keyslot manager to get a crypto_skcipher -* for the algorithm and key specified for this bio. +* Get a blk-crypto-fallback keyslot that contains a crypto_skcipher for +* this bio's algorithm and key. */ - blk_st =blk_ksm_get_slot_for_key (&blk_crypto_ksm, bc->bc_key, &slot); + blk_st =blk_crypto_get_keyslot (&blk_crypto_fallback_profile, + bc->bc_key, &slot);if (blk_st !=BLK_STS_OK ) { src_bio->bi_status = blk_st; goto out_put_enc_bio;
@@ -364,7 +361,7 @@ static bool blk_crypto_fallback_encrypt_bio(struct bio **bio_ptr) out_free_ciph_req:skcipher_request_free (ciph_req); out_release_keyslot: -blk_ksm_put_slot (slot); +blk_crypto_put_keyslot (slot); out_put_enc_bio:if (enc_bio)bio_put (enc_bio);
@@ -382,7 +379,7 @@ static void blk_crypto_fallback_decrypt_bio(struct work_struct *work) container_of(work, struct bio_fallback_crypt_ctx, work);struct bio *bio = f_ctx->bio;struct bio_crypt_ctx *bc = &f_ctx->crypt_ctx; -struct blk_ksm_keyslot *slot; +struct blk_crypto_keyslot *slot;struct skcipher_request *ciph_req =NULL ;DECLARE_CRYPTO_WAIT (wait);u64 curr_dun[BLK_CRYPTO_DUN_ARRAY_SIZE ];
@@ -395,10 +392,11 @@ static void blk_crypto_fallback_decrypt_bio(struct work_struct *work)blk_status_t blk_st;/* -* Use the crypto API fallback keyslot manager to get a crypto_skcipher -* for the algorithm and key specified for this bio. +* Get a blk-crypto-fallback keyslot that contains a crypto_skcipher for +* this bio's algorithm and key. */ - blk_st =blk_ksm_get_slot_for_key (&blk_crypto_ksm, bc->bc_key, &slot); + blk_st =blk_crypto_get_keyslot (&blk_crypto_fallback_profile, + bc->bc_key, &slot);if (blk_st !=BLK_STS_OK ) { bio->bi_status = blk_st; goto out_no_keyslot;
@@ -436,7 +434,7 @@ static void blk_crypto_fallback_decrypt_bio(struct work_struct *work) out:skcipher_request_free (ciph_req); -blk_ksm_put_slot (slot); +blk_crypto_put_keyslot (slot); out_no_keyslot:mempool_free (f_ctx, bio_fallback_crypt_ctx_pool);bio_endio (bio);
@@ -501,8 +499,8 @@ bool blk_crypto_fallback_bio_prep(struct bio **bio_ptr)return false; } -if (!blk_ksm_crypto_cfg_supported (&blk_crypto_ksm, - &bc->bc_key->crypto_cfg)) { +if (!__blk_crypto_cfg_supported (&blk_crypto_fallback_profile, + &bc->bc_key->crypto_cfg)) { bio->bi_status =BLK_STS_NOTSUPP ;return false; }
@@ -528,7 +526,7 @@ bool blk_crypto_fallback_bio_prep(struct bio **bio_ptr)int blk_crypto_fallback_evict_key (const struct blk_crypto_key *key) { -return blk_ksm_evict_key (&blk_crypto_ksm, key); +return __blk_crypto_evict_key (&blk_crypto_fallback_profile, key); }static bool blk_crypto_fallback_inited;
@@ -536,6 +534,7 @@ static int blk_crypto_fallback_init(void) {int i;int err; +struct blk_crypto_profile *profile = &blk_crypto_fallback_profile;if (blk_crypto_fallback_inited)return 0 ;
@@ -546,24 +545,24 @@ static int blk_crypto_fallback_init(void)if (err) goto out; - err =blk_ksm_init (&blk_crypto_ksm, blk_crypto_num_keyslots); + err =blk_crypto_profile_init (profile, blk_crypto_num_keyslots);if (err) goto fail_free_bioset; err = -ENOMEM ; - blk_crypto_ksm.ksm_ll_ops = blk_crypto_ksm_ll_ops; - blk_crypto_ksm.max_dun_bytes_supported =BLK_CRYPTO_MAX_IV_SIZE ; + profile->ll_ops = blk_crypto_fallback_ll_ops; + profile->max_dun_bytes_supported =BLK_CRYPTO_MAX_IV_SIZE ;/* All blk-crypto modes have a crypto API fallback. */ for (i =0 ; i <BLK_ENCRYPTION_MODE_MAX ; i++) - blk_crypto_ksm.crypto_modes_supported[i] =0xFFFFFFFF ; - blk_crypto_ksm.crypto_modes_supported[BLK_ENCRYPTION_MODE_INVALID ] =0 ; + profile->modes_supported[i] =0xFFFFFFFF ; + profile->modes_supported[BLK_ENCRYPTION_MODE_INVALID ] =0 ; blk_crypto_wq =alloc_workqueue ("blk_crypto_wq" ,WQ_UNBOUND |WQ_HIGHPRI |WQ_MEM_RECLAIM ,num_online_cpus ());if (!blk_crypto_wq) - goto fail_free_ksm; + goto fail_destroy_profile; blk_crypto_keyslots = kcalloc(blk_crypto_num_keyslots, sizeof(blk_crypto_keyslots[0]),
@@ -597,8 +596,8 @@ static int blk_crypto_fallback_init(void)kfree (blk_crypto_keyslots); fail_free_wq:destroy_workqueue (blk_crypto_wq); -fail_free_ksm: -blk_ksm_destroy (&blk_crypto_ksm); +fail_destroy_profile: +blk_crypto_profile_destroy (profile); fail_free_bioset:bioset_exit (&crypto_bio_split); out:
block/blk-crypto-profile.c +253 -267
810 lines not on this page · raw message
block/blk-crypto.c +14 -13
@@ -218,8 +218,9 @@ static bool bio_crypt_check_alignment(struct bio *bio)blk_status_t __blk_crypto_init_request (struct request *rq) { -return blk_ksm_get_slot_for_key (rq->q->ksm, rq->crypt_ctx->bc_key, - &rq->crypt_keyslot); +return blk_crypto_get_keyslot (rq->q->crypto_profile, + rq->crypt_ctx->bc_key, + &rq->crypt_keyslot); }/**
@@ -233,7 +234,7 @@ blk_status_t __blk_crypto_init_request(struct request *rq)*/ void __blk_crypto_free_request (struct request *rq) { -blk_ksm_put_slot (rq->crypt_keyslot); +blk_crypto_put_keyslot (rq->crypt_keyslot);mempool_free (rq->crypt_ctx, bio_crypt_ctx_pool);blk_crypto_rq_set_defaults (rq); }
@@ -264,6 +265,7 @@ bool __blk_crypto_bio_prep(struct bio **bio_ptr) {struct bio *bio = *bio_ptr;const struct blk_crypto_key *bc_key = bio->bi_crypt_context->bc_key; +struct blk_crypto_profile *profile;/* Error if bio has no data. */ if (WARN_ON_ONCE (!bio_has_data (bio))) {
@@ -280,8 +282,8 @@ bool __blk_crypto_bio_prep(struct bio **bio_ptr)* Success if device supports the encryption context, or if we succeeded * in falling back to the crypto API. */ -if (blk_ksm_crypto_cfg_supported (bio->bi_bdev->bd_disk->queue->ksm, - &bc_key->crypto_cfg)) + profile = bio->bi_bdev->bd_disk->queue->crypto_profile; +if (__blk_crypto_cfg_supported (profile, &bc_key->crypto_cfg))return true;if (blk_crypto_fallback_bio_prep (bio_ptr))
@@ -357,7 +359,7 @@ bool blk_crypto_config_supported(struct request_queue *q, const struct blk_crypto_config *cfg) {return IS_ENABLED (CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK ) || -blk_ksm_crypto_cfg_supported (q->ksm, cfg); +__blk_crypto_cfg_supported (q->crypto_profile, cfg); }/**
@@ -378,7 +380,7 @@ bool blk_crypto_config_supported(struct request_queue *q,int blk_crypto_start_using_key (const struct blk_crypto_key *key,struct request_queue *q) { -if (blk_ksm_crypto_cfg_supported (q->ksm, &key->crypto_cfg)) +if (__blk_crypto_cfg_supported (q->crypto_profile, &key->crypto_cfg))return 0 ;return blk_crypto_fallback_start_using_mode (key->crypto_cfg.crypto_mode); }
@@ -394,18 +396,17 @@ int blk_crypto_start_using_key(const struct blk_crypto_key *key,* evicted from any hardware that it might have been programmed into. The key * must not be in use by any in-flight IO when this function is called. * -* Return: 0 on success or if key is not present in the q's ksm, -err on error. +* Return: 0 on success or if the key wasn't in any keyslot; -errno on error. */ int blk_crypto_evict_key (struct request_queue *q,const struct blk_crypto_key *key) { -if (blk_ksm_crypto_cfg_supported (q->ksm, &key->crypto_cfg)) -return blk_ksm_evict_key (q->ksm, key); +if (__blk_crypto_cfg_supported (q->crypto_profile, &key->crypto_cfg)) +return __blk_crypto_evict_key (q->crypto_profile, key);/* -* If the request queue's associated inline encryption hardware didn't -* have support for the key, then the key might have been programmed -* into the fallback keyslot manager, so try to evict from there. +* If the request_queue didn't support the key, then blk-crypto-fallback +* may have been used, so try to evict the key from blk-crypto-fallback. */ return blk_crypto_fallback_evict_key (key); }
block/blk-integrity.c +2 -2
@@ -409,9 +409,9 @@ void blk_integrity_register(struct gendisk *disk, struct blk_integrity *templateblk_queue_flag_set (QUEUE_FLAG_STABLE_WRITES , disk->queue);#ifdef CONFIG_BLK_INLINE_ENCRYPTION -if (disk->queue->ksm) { +if (disk->queue->crypto_profile) {pr_warn ("blk-integrity: Integrity and hardware inline encryption are not supported together. Disabling hardware inline encryption.\n" ); -blk_ksm_unregister (disk->queue); +blk_crypto_unregister (disk->queue); }#endif }
drivers/md/dm-core.h +1 -1
@@ -200,7 +200,7 @@ struct dm_table {struct dm_md_mempools *mempools;#ifdef CONFIG_BLK_INLINE_ENCRYPTION -struct blk_keyslot_manager *ksm; +struct blk_crypto_profile *crypto_profile;#endif };
drivers/md/dm-table.c +82 -86
283 lines not on this page · raw message
drivers/md/dm.c +4 -4
@@ -1660,14 +1660,14 @@ static const struct dax_operations dm_dax_ops;static void dm_wq_work (struct work_struct *work);#ifdef CONFIG_BLK_INLINE_ENCRYPTION -static void dm_queue_destroy_keyslot_manager (struct request_queue *q) +static void dm_queue_destroy_crypto_profile (struct request_queue *q) { -dm_destroy_keyslot_manager (q->ksm); +dm_destroy_crypto_profile (q->crypto_profile); }#else /* CONFIG_BLK_INLINE_ENCRYPTION */ -static inline void dm_queue_destroy_keyslot_manager (struct request_queue *q) +static inline void dm_queue_destroy_crypto_profile (struct request_queue *q) { }#endif /* !CONFIG_BLK_INLINE_ENCRYPTION */
@@ -1693,7 +1693,7 @@ static void cleanup_mapped_device(struct mapped_device *md)dm_sysfs_exit (md);del_gendisk (md->disk); } -dm_queue_destroy_keyslot_manager (md->queue); +dm_queue_destroy_crypto_profile (md->queue);blk_cleanup_disk (md->disk); }
drivers/mmc/core/crypto.c +7 -4
@@ -16,13 +16,13 @@ void mmc_crypto_set_initial_state(struct mmc_host *host) {/* Reset might clear all keys, so reprogram all the keys. */ if (host->caps2 &MMC_CAP2_CRYPTO ) -blk_ksm_reprogram_all_keys (&host->ksm); +blk_crypto_reprogram_all_keys (&host->crypto_profile); }void mmc_crypto_setup_queue (struct request_queue *q,struct mmc_host *host) {if (host->caps2 &MMC_CAP2_CRYPTO ) -blk_ksm_register (&host->ksm, q); +blk_crypto_register (&host->crypto_profile, q); }EXPORT_SYMBOL_GPL (mmc_crypto_setup_queue);
@@ -30,12 +30,15 @@ void mmc_crypto_prepare_req(struct mmc_queue_req *mqrq) {struct request *req =mmc_queue_req_to_req (mqrq);struct mmc_request *mrq = &mqrq->brq.mrq; +struct blk_crypto_keyslot *keyslot;if (!req->crypt_ctx)return ; mrq->crypto_ctx = req->crypt_ctx; -if (req->crypt_keyslot) - mrq->crypto_key_slot =blk_ksm_get_slot_idx (req->crypt_keyslot); + + keyslot = req->crypt_keyslot; +if (keyslot) + mrq->crypto_key_slot =blk_crypto_keyslot_index (keyslot); }EXPORT_SYMBOL_GPL (mmc_crypto_prepare_req);
drivers/mmc/host/cqhci-crypto.c +16 -15
@@ -23,9 +23,10 @@ static const struct cqhci_crypto_alg_entry { };static inline struct cqhci_host * -cqhci_host_from_ksm (struct blk_keyslot_manager *ksm) +cqhci_host_from_crypto_profile (struct blk_crypto_profile *profile) { - struct mmc_host *mmc = container_of(ksm, struct mmc_host, ksm); +struct mmc_host *mmc = + container_of(profile, struct mmc_host, crypto_profile);return mmc->cqe_private; }
@@ -57,12 +58,12 @@ static int cqhci_crypto_program_key(struct cqhci_host *cq_host,return 0 ; } -static int cqhci_crypto_keyslot_program (struct blk_keyslot_manager *ksm, +static int cqhci_crypto_keyslot_program (struct blk_crypto_profile *profile,const struct blk_crypto_key *key,unsigned int slot) { -struct cqhci_host *cq_host =cqhci_host_from_ksm (ksm); +struct cqhci_host *cq_host =cqhci_host_from_crypto_profile (profile);const union cqhci_crypto_cap_entry *ccap_array = cq_host->crypto_cap_array; const struct cqhci_crypto_alg_entry *alg =
@@ -115,11 +116,11 @@ static int cqhci_crypto_clear_keyslot(struct cqhci_host *cq_host, int slot)return cqhci_crypto_program_key (cq_host, &cfg, slot); } -static int cqhci_crypto_keyslot_evict (struct blk_keyslot_manager *ksm, +static int cqhci_crypto_keyslot_evict (struct blk_crypto_profile *profile,const struct blk_crypto_key *key,unsigned int slot) { -struct cqhci_host *cq_host =cqhci_host_from_ksm (ksm); +struct cqhci_host *cq_host =cqhci_host_from_crypto_profile (profile);return cqhci_crypto_clear_keyslot (cq_host, slot); }
@@ -132,7 +133,7 @@ static int cqhci_crypto_keyslot_evict(struct blk_keyslot_manager *ksm,* "enabled" when these are called, i.e. CQHCI_ENABLE might not be set in the * CQHCI_CFG register. But the hardware allows that. */ -static const struct blk_ksm_ll_ops cqhci_ksm_ops = { +static const struct blk_crypto_ll_ops cqhci_crypto_ops = { .keyslot_program = cqhci_crypto_keyslot_program, .keyslot_evict = cqhci_crypto_keyslot_evict, };
@@ -157,8 +158,8 @@ cqhci_find_blk_crypto_mode(union cqhci_crypto_cap_entry cap)* * If the driver previously set MMC_CAP2_CRYPTO and the CQE declares * CQHCI_CAP_CS, initialize the crypto support. This involves reading the -* crypto capability registers, initializing the keyslot manager, clearing all -* keyslots, and enabling 128-bit task descriptors. +* crypto capability registers, initializing the blk_crypto_profile, clearing +* all keyslots, and enabling 128-bit task descriptors. * * Return: 0 if crypto was initialized or isn't supported; whether * MMC_CAP2_CRYPTO remains set indicates which one of those cases it is.
@@ -168,7 +169,7 @@ int cqhci_crypto_init(struct cqhci_host *cq_host) {struct mmc_host *mmc = cq_host->mmc;struct device *dev =mmc_dev (mmc); -struct blk_keyslot_manager *ksm = &mmc->ksm; +struct blk_crypto_profile *profile = &mmc->crypto_profile;unsigned int num_keyslots;unsigned int cap_idx;enum blk_crypto_mode_num blk_mode_num;
@@ -199,15 +200,15 @@ int cqhci_crypto_init(struct cqhci_host *cq_host)*/ num_keyslots = cq_host->crypto_capabilities.config_count +1 ; - err =devm_blk_ksm_init (dev, ksm, num_keyslots); + err =devm_blk_crypto_profile_init (dev, profile, num_keyslots);if (err) goto out; - ksm->ksm_ll_ops = cqhci_ksm_ops; - ksm->dev = dev; + profile->ll_ops = cqhci_crypto_ops; + profile->dev = dev;/* Unfortunately, CQHCI crypto only supports 32 DUN bits. */ - ksm->max_dun_bytes_supported =4 ; + profile->max_dun_bytes_supported =4 ;/* * Cache all the crypto capabilities and advertise the supported crypto
@@ -223,7 +224,7 @@ int cqhci_crypto_init(struct cqhci_host *cq_host) cq_host->crypto_cap_array[cap_idx]);if (blk_mode_num ==BLK_ENCRYPTION_MODE_INVALID )continue ; - ksm->crypto_modes_supported[blk_mode_num] |= + profile->modes_supported[blk_mode_num] |= cq_host->crypto_cap_array[cap_idx].sdus_mask *512 ; }
drivers/scsi/ufs/ufshcd-crypto.c +17 -15
@@ -48,11 +48,12 @@ static int ufshcd_program_key(struct ufs_hba *hba,return err; } -static int ufshcd_crypto_keyslot_program (struct blk_keyslot_manager *ksm, +static int ufshcd_crypto_keyslot_program (struct blk_crypto_profile *profile,const struct blk_crypto_key *key,unsigned int slot) { - struct ufs_hba *hba = container_of(ksm, struct ufs_hba, ksm); +struct ufs_hba *hba = + container_of(profile, struct ufs_hba, crypto_profile);const union ufs_crypto_cap_entry *ccap_array = hba->crypto_cap_array;const struct ufs_crypto_alg_entry *alg = &ufs_crypto_algs[key->crypto_cfg.crypto_mode];
@@ -105,11 +106,12 @@ static int ufshcd_clear_keyslot(struct ufs_hba *hba, int slot)return ufshcd_program_key (hba, &cfg, slot); } -static int ufshcd_crypto_keyslot_evict (struct blk_keyslot_manager *ksm, +static int ufshcd_crypto_keyslot_evict (struct blk_crypto_profile *profile,const struct blk_crypto_key *key,unsigned int slot) { - struct ufs_hba *hba = container_of(ksm, struct ufs_hba, ksm); +struct ufs_hba *hba = + container_of(profile, struct ufs_hba, crypto_profile);return ufshcd_clear_keyslot (hba, slot); }
@@ -120,11 +122,11 @@ bool ufshcd_crypto_enable(struct ufs_hba *hba)return false;/* Reset might clear all keys, so reprogram all the keys. */ -blk_ksm_reprogram_all_keys (&hba->ksm); +blk_crypto_reprogram_all_keys (&hba->crypto_profile);return true; } -static const struct blk_ksm_ll_ops ufshcd_ksm_ops = { +static const struct blk_crypto_ll_ops ufshcd_crypto_ops = { .keyslot_program = ufshcd_crypto_keyslot_program, .keyslot_evict = ufshcd_crypto_keyslot_evict, };
@@ -179,15 +181,16 @@ int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba) }/* The actual number of configurations supported is (CFGC+1) */ - err =devm_blk_ksm_init (hba->dev, &hba->ksm, - hba->crypto_capabilities.config_count +1 ); + err =devm_blk_crypto_profile_init ( + hba->dev, &hba->crypto_profile, + hba->crypto_capabilities.config_count +1 );if (err) goto out; - hba->ksm.ksm_ll_ops = ufshcd_ksm_ops; + hba->crypto_profile.ll_ops = ufshcd_crypto_ops;/* UFS only supports 8 bytes for any DUN */ - hba->ksm.max_dun_bytes_supported =8 ; - hba->ksm.dev = hba->dev; + hba->crypto_profile.max_dun_bytes_supported =8 ; + hba->crypto_profile.dev = hba->dev;/* * Cache all the UFS crypto capabilities and advertise the supported
@@ -202,7 +205,7 @@ int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba) blk_mode_num =ufshcd_find_blk_crypto_mode ( hba->crypto_cap_array[cap_idx]);if (blk_mode_num !=BLK_ENCRYPTION_MODE_INVALID ) - hba->ksm.crypto_modes_supported[blk_mode_num] |= + hba->crypto_profile.modes_supported[blk_mode_num] |= hba->crypto_cap_array[cap_idx].sdus_mask *512 ; }
@@ -230,9 +233,8 @@ void ufshcd_init_crypto(struct ufs_hba *hba)ufshcd_clear_keyslot (hba, slot); } -void ufshcd_crypto_setup_rq_keyslot_manager (struct ufs_hba *hba, -struct request_queue *q) +void ufshcd_crypto_register (struct ufs_hba *hba,struct request_queue *q) {if (hba->caps &UFSHCD_CAP_CRYPTO ) -blk_ksm_register (&hba->ksm, q); +blk_crypto_register (&hba->crypto_profile, q); }
drivers/scsi/ufs/ufshcd-crypto.h +4 -5
@@ -18,7 +18,7 @@ static inline void ufshcd_prepare_lrbp_crypto(struct request *rq,return ; } - lrbp->crypto_key_slot =blk_ksm_get_slot_idx (rq->crypt_keyslot); + lrbp->crypto_key_slot =blk_crypto_keyslot_index (rq->crypt_keyslot); lrbp->data_unit_num = rq->crypt_ctx->bc_dun[0 ]; }
@@ -40,8 +40,7 @@ int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba);void ufshcd_init_crypto (struct ufs_hba *hba); -void ufshcd_crypto_setup_rq_keyslot_manager (struct ufs_hba *hba, -struct request_queue *q); +void ufshcd_crypto_register (struct ufs_hba *hba,struct request_queue *q);#else /* CONFIG_SCSI_UFS_CRYPTO */
@@ -64,8 +63,8 @@ static inline int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba)static inline void ufshcd_init_crypto (struct ufs_hba *hba) { } -static inline void ufshcd_crypto_setup_rq_keyslot_manager (struct ufs_hba *hba, -struct request_queue *q) { } +static inline void ufshcd_crypto_register (struct ufs_hba *hba, +struct request_queue *q) { }#endif /* CONFIG_SCSI_UFS_CRYPTO */
drivers/scsi/ufs/ufshcd.c +1 -1
@@ -4986,7 +4986,7 @@ static int ufshcd_slave_configure(struct scsi_device *sdev)else if (ufshcd_is_rpm_autosuspend_allowed (hba)) sdev->rpm_autosuspend =1 ; -ufshcd_crypto_setup_rq_keyslot_manager (hba, q); +ufshcd_crypto_register (hba, q);return 0 ; }
drivers/scsi/ufs/ufshcd.h +2 -2
@@ -766,7 +766,7 @@ struct ufs_hba_monitor {* @crypto_capabilities: Content of crypto capabilities register (0x100) * @crypto_cap_array: Array of crypto capabilities * @crypto_cfg_register: Start of the crypto cfg array -* @ksm: the keyslot manager tied to this hba +* @crypto_profile: the crypto profile of this hba (if applicable) */ struct ufs_hba {void __iomem *mmio_base;
@@ -911,7 +911,7 @@ struct ufs_hba {union ufs_crypto_capabilities crypto_capabilities;union ufs_crypto_cap_entry *crypto_cap_array;u32 crypto_cfg_register; -struct blk_keyslot_manager ksm; +struct blk_crypto_profile crypto_profile;#endif #ifdef CONFIG_DEBUG_FS struct dentry *debugfs_root;
include/linux/blk-crypto-profile.h +105 -59
213 lines not on this page · raw message
include/linux/blk-mq.h +1 -1
@@ -171,7 +171,7 @@ struct request {#ifdef CONFIG_BLK_INLINE_ENCRYPTION struct bio_crypt_ctx *crypt_ctx; -struct blk_ksm_keyslot *crypt_keyslot; +struct blk_crypto_keyslot *crypt_keyslot;#endif unsigned short write_hint;
include/linux/blkdev.h +8 -8
@@ -30,7 +30,7 @@ struct pr_ops;struct rq_qos ;struct blk_queue_stats ;struct blk_stat_callback ; -struct blk_keyslot_manager ; +struct blk_crypto_profile ;/* Must be consistent with blk_mq_poll_stats_bkt() */ #define BLK_MQ_POLL_STATS_BKTS 16
@@ -224,8 +224,7 @@ struct request_queue {unsigned int dma_alignment;#ifdef CONFIG_BLK_INLINE_ENCRYPTION -/* Inline crypto capabilities */ -struct blk_keyslot_manager *ksm; +struct blk_crypto_profile *crypto_profile;#endif unsigned int rq_timeout;
@@ -1160,19 +1159,20 @@ int kblockd_mod_delayed_work_on(int cpu, struct delayed_work *dwork, unsigned lo#ifdef CONFIG_BLK_INLINE_ENCRYPTION -bool blk_ksm_register (struct blk_keyslot_manager *ksm,struct request_queue *q); +bool blk_crypto_register (struct blk_crypto_profile *profile, +struct request_queue *q); -void blk_ksm_unregister (struct request_queue *q); +void blk_crypto_unregister (struct request_queue *q);#else /* CONFIG_BLK_INLINE_ENCRYPTION */ -static inline bool blk_ksm_register (struct blk_keyslot_manager *ksm, -struct request_queue *q) +static inline bool blk_crypto_register (struct blk_crypto_profile *profile, +struct request_queue *q) {return true; } -static inline void blk_ksm_unregister (struct request_queue *q) { } +static inline void blk_crypto_unregister (struct request_queue *q) { }#endif /* CONFIG_BLK_INLINE_ENCRYPTION */
include/linux/device-mapper.h +2 -2
@@ -576,9 +576,9 @@ struct dm_table *dm_swap_table(struct mapped_device *md, struct dm_table *t);/* -* Table keyslot manager functions +* Table blk_crypto_profile functions */ -void dm_destroy_keyslot_manager (struct blk_keyslot_manager *ksm); +void dm_destroy_crypto_profile (struct blk_crypto_profile *profile);/*----------------------------------------------------------------- * Macros.
include/linux/mmc/host.h +1 -1
@@ -492,7 +492,7 @@ struct mmc_host {/* Inline encryption support */ #ifdef CONFIG_MMC_CRYPTO -struct blk_keyslot_manager ksm; +struct blk_crypto_profile crypto_profile;#endif /* Host Software Queue support */
Signature · 1 line
2.33.0
PATCH v5 4/4 blk-crypto: update inline encryption documentation
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Rework most of inline-encryption.rst to be easier to follow, to correct some information, to add some important details and remove some unimportant details, and to take into account the renaming from blk_keyslot_manager to blk_crypto_profile.
Reviewed-by Mike Snitzer <[email protected]>
Signed-off-by Eric Biggers <[email protected]>
--- Documentation/block/inline-encryption.rst | 451 ++++++++++++---------- 1 file changed, 245 insertions(+), 206 deletions(-)
Documentation/block/inline-encryption.rst +245 -206
@@ -7,230 +7,269 @@ Inline Encryption Background ========== -Inline encryption hardware sits logically between memory and the disk, and can -en/decrypt data as it goes in/out of the disk. Inline encryption hardware has a -fixed number of "keyslots" - slots into which encryption contexts (i.e. the -encryption key, encryption algorithm, data unit size) can be programmed by the -kernel at any time. Each request sent to the disk can be tagged with the index -of a keyslot (and also a data unit number to act as an encryption tweak), and -the inline encryption hardware will en/decrypt the data in the request with the -encryption context programmed into that keyslot. This is very different from -full disk encryption solutions like self encrypting drives/TCG OPAL/ATA -Security standards, since with inline encryption, any block on disk could be -encrypted with any encryption context the kernel chooses. - +Inline encryption hardware sits logically between memory and disk, and can +en/decrypt data as it goes in/out of the disk. For each I/O request, software +can control exactly how the inline encryption hardware will en/decrypt the data +in terms of key, algorithm, data unit size (the granularity of en/decryption), +and data unit number (a value that determines the initialization vector(s)). + +Some inline encryption hardware accepts all encryption parameters including raw +keys directly in low-level I/O requests. However, most inline encryption +hardware instead has a fixed number of "keyslots" and requires that the key, +algorithm, and data unit size first be programmed into a keyslot. Each +low-level I/O request then just contains a keyslot index and data unit number. + +Note that inline encryption hardware is very different from traditional crypto +accelerators, which are supported through the kernel crypto API. Traditional +crypto accelerators operate on memory regions, whereas inline encryption +hardware operates on I/O requests. Thus, inline encryption hardware needs to be +managed by the block layer, not the kernel crypto API. + +Inline encryption hardware is also very different from "self-encrypting drives", +such as those based on the TCG Opal or ATA Security standards. Self-encrypting +drives don't provide fine-grained control of encryption and provide no way to +verify the correctness of the resulting ciphertext. Inline encryption hardware +provides fine-grained control of encryption, including the choice of key and +initialization vector for each sector, and can be tested for correctness. Objective ========= -We want to support inline encryption (IE) in the kernel. -To allow for testing, we also want a crypto API fallback when actual -IE hardware is absent. We also want IE to work with layered devices -like dm and loopback (i.e. we want to be able to use the IE hardware -of the underlying devices if present, or else fall back to crypto API -en/decryption). - +We want to support inline encryption in the kernel. To make testing easier, we +also want support for falling back to the kernel crypto API when actual inline +encryption hardware is absent. We also want inline encryption to work with +layered devices like device-mapper and loopback (i.e. we want to be able to use +the inline encryption hardware of the underlying devices if present, or else +fall back to crypto API en/decryption). Constraints and notes ===================== -- IE hardware has a limited number of "keyslots" that can be programmed - with an encryption context (key, algorithm, data unit size, etc.) at any time. - One can specify a keyslot in a data request made to the device, and the - device will en/decrypt the data using the encryption context programmed into - that specified keyslot. When possible, we want to make multiple requests with - the same encryption context share the same keyslot. - -- We need a way for upper layers like filesystems to specify an encryption - context to use for en/decrypting a struct bio, and a device driver (like UFS) - needs to be able to use that encryption context when it processes the bio. - -- We need a way for device drivers to expose their inline encryption - capabilities in a unified way to the upper layers. - - -Design -====== - -We add a struct bio_crypt_ctx to struct bio that can -represent an encryption context, because we need to be able to pass this -encryption context from the upper layers (like the fs layer) to the -device driver to act upon. - -While IE hardware works on the notion of keyslots, the FS layer has no -knowledge of keyslots - it simply wants to specify an encryption context to -use while en/decrypting a bio. - -We introduce a keyslot manager (KSM) that handles the translation from -encryption contexts specified by the FS to keyslots on the IE hardware. -This KSM also serves as the way IE hardware can expose its capabilities to -upper layers. The generic mode of operation is: each device driver that wants -to support IE will construct a KSM and set it up in its struct request_queue. -Upper layers that want to use IE on this device can then use this KSM in -the device's struct request_queue to translate an encryption context into -a keyslot. The presence of the KSM in the request queue shall be used to mean -that the device supports IE. - -The KSM uses refcounts to track which keyslots are idle (either they have no -encryption context programmed, or there are no in-flight struct bios -referencing that keyslot). When a new encryption context needs a keyslot, it -tries to find a keyslot that has already been programmed with the same -encryption context, and if there is no such keyslot, it evicts the least -recently used idle keyslot and programs the new encryption context into that -one. If no idle keyslots are available, then the caller will sleep until there -is at least one. - - -blk-mq changes, other block layer changes and blk-crypto-fallback -================================================================= - -We add a pointer to a ``bi_crypt_context`` and ``keyslot`` to -struct request. These will be referred to as the ``crypto fields`` -for the request. This ``keyslot`` is the keyslot into which the -``bi_crypt_context`` has been programmed in the KSM of the ``request_queue`` -that this request is being sent to. - -We introduce ``block/blk-crypto-fallback.c``, which allows upper layers to remain -blissfully unaware of whether or not real inline encryption hardware is present -underneath. When a bio is submitted with a target ``request_queue`` that doesn't -support the encryption context specified with the bio, the block layer will -en/decrypt the bio with the blk-crypto-fallback. - -If the bio is a ``WRITE`` bio, a bounce bio is allocated, and the data in the bio -is encrypted stored in the bounce bio - blk-mq will then proceed to process the -bounce bio as if it were not encrypted at all (except when blk-integrity is -concerned). ``blk-crypto-fallback`` sets the bounce bio's ``bi_end_io`` to an -internal function that cleans up the bounce bio and ends the original bio. - -If the bio is a ``READ`` bio, the bio's ``bi_end_io`` (and also ``bi_private``) -is saved and overwritten by ``blk-crypto-fallback`` to -``bio_crypto_fallback_decrypt_bio``. The bio's ``bi_crypt_context`` is also -overwritten with ``NULL``, so that to the rest of the stack, the bio looks -as if it was a regular bio that never had an encryption context specified. -``bio_crypto_fallback_decrypt_bio`` will decrypt the bio, restore the original -``bi_end_io`` (and also ``bi_private``) and end the bio again. - -Regardless of whether real inline encryption hardware is used or the +- We need a way for upper layers (e.g. filesystems) to specify an encryption + context to use for en/decrypting a bio, and device drivers (e.g. UFSHCD) need + to be able to use that encryption context when they process the request. + Encryption contexts also introduce constraints on bio merging; the block layer + needs to be aware of these constraints. + +- Different inline encryption hardware has different supported algorithms, + supported data unit sizes, maximum data unit numbers, etc. We call these + properties the "crypto capabilities". We need a way for device drivers to + advertise crypto capabilities to upper layers in a generic way. + +- Inline encryption hardware usually (but not always) requires that keys be + programmed into keyslots before being used. Since programming keyslots may be + slow and there may not be very many keyslots, we shouldn't just program the + key for every I/O request, but rather keep track of which keys are in the + keyslots and reuse an already-programmed keyslot when possible. + +- Upper layers typically define a specific end-of-life for crypto keys, e.g. + when an encrypted directory is locked or when a crypto mapping is torn down. + At these times, keys are wiped from memory. We must provide a way for upper + layers to also evict keys from any keyslots they are present in. + +- When possible, device-mapper devices must be able to pass through the inline + encryption support of their underlying devices. However, it doesn't make + sense for device-mapper devices to have keyslots themselves. + +Basic design +============ + +We introduce ``struct blk_crypto_key`` to represent an inline encryption key and +how it will be used. This includes the actual bytes of the key; the size of the +key; the algorithm and data unit size the key will be used with; and the number +of bytes needed to represent the maximum data unit number the key will be used +with. + +We introduce ``struct bio_crypt_ctx`` to represent an encryption context. It +contains a data unit number and a pointer to a blk_crypto_key. We add pointers +to a bio_crypt_ctx to ``struct bio`` and ``struct request``; this allows users +of the block layer (e.g. filesystems) to provide an encryption context when +creating a bio and have it be passed down the stack for processing by the block +layer and device drivers. Note that the encryption context doesn't explicitly +say whether to encrypt or decrypt, as that is implicit from the direction of the +bio; WRITE means encrypt, and READ means decrypt. + +We also introduce ``struct blk_crypto_profile`` to contain all generic inline +encryption-related state for a particular inline encryption device. The +blk_crypto_profile serves as the way that drivers for inline encryption hardware +advertise their crypto capabilities and provide certain functions (e.g., +functions to program and evict keys) to upper layers. Each device driver that +wants to support inline encryption will construct a blk_crypto_profile, then +associate it with the disk's request_queue. + +The blk_crypto_profile also manages the hardware's keyslots, when applicable. +This happens in the block layer, so that users of the block layer can just +specify encryption contexts and don't need to know about keyslots at all, nor do +device drivers need to care about most details of keyslot management. + +Specifically, for each keyslot, the block layer (via the blk_crypto_profile) +keeps track of which blk_crypto_key that keyslot contains (if any), and how many +in-flight I/O requests are using it. When the block layer creates a +``struct request`` for a bio that has an encryption context, it grabs a keyslot +that already contains the key if possible. Otherwise it waits for an idle +keyslot (a keyslot that isn't in-use by any I/O), then programs the key into the +least-recently-used idle keyslot using the function the device driver provided. +In both cases, the resulting keyslot is stored in the ``crypt_keyslot`` field of +the request, where it is then accessible to device drivers and is released after +the request completes. + +``struct request`` also contains a pointer to the original bio_crypt_ctx. +Requests can be built from multiple bios, and the block layer must take the +encryption context into account when trying to merge bios and requests. For two +bios/requests to be merged, they must have compatible encryption contexts: both +unencrypted, or both encrypted with the same key and contiguous data unit +numbers. Only the encryption context for the first bio in a request is +retained, since the remaining bios have been verified to be merge-compatible +with the first bio. + +To make it possible for inline encryption to work with request_queue based +layered devices, when a request is cloned, its encryption context is cloned as +well. When the cloned request is submitted, it is then processed as usual; this +includes getting a keyslot from the clone's target device if needed. + +blk-crypto-fallback +=================== + +It is desirable for the inline encryption support of upper layers (e.g. +filesystems) to be testable without real inline encryption hardware, and +likewise for the block layer's keyslot management logic. It is also desirable +to allow upper layers to just always use inline encryption rather than have to +implement encryption in multiple ways. + +Therefore, we also introduce *blk-crypto-fallback*, which is an implementation +of inline encryption using the kernel crypto API. blk-crypto-fallback is built +into the block layer, so it works on any block device without any special setup. +Essentially, when a bio with an encryption context is submitted to a +request_queue that doesn't support that encryption context, the block layer will +handle en/decryption of the bio using blk-crypto-fallback. + +For encryption, the data cannot be encrypted in-place, as callers usually rely +on it being unmodified. Instead, blk-crypto-fallback allocates bounce pages, +fills a new bio with those bounce pages, encrypts the data into those bounce +pages, and submits that "bounce" bio. When the bounce bio completes, +blk-crypto-fallback completes the original bio. If the original bio is too +large, multiple bounce bios may be required; see the code for details. + +For decryption, blk-crypto-fallback "wraps" the bio's completion callback +(``bi_complete``) and private data (``bi_private``) with its own, unsets the +bio's encryption context, then submits the bio. If the read completes +successfully, blk-crypto-fallback restores the bio's original completion +callback and private data, then decrypts the bio's data in-place using the +kernel crypto API. Decryption happens from a workqueue, as it may sleep. +Afterwards, blk-crypto-fallback completes the bio. + +In both cases, the bios that blk-crypto-fallback submits no longer have an +encryption context. Therefore, lower layers only see standard unencrypted I/O. + +blk-crypto-fallback also defines its own blk_crypto_profile and has its own +"keyslots"; its keyslots contain ``struct crypto_skcipher`` objects. The reason +for this is twofold. First, it allows the keyslot management logic to be tested +without actual inline encryption hardware. Second, similar to actual inline +encryption hardware, the crypto API doesn't accept keys directly in requests but +rather requires that keys be set ahead of time, and setting keys can be +expensive; moreover, allocating a crypto_skcipher can't happen on the I/O path +at all due to the locks it takes. Therefore, the concept of keyslots still +makes sense for blk-crypto-fallback. + +Note that regardless of whether real inline encryption hardware or blk-crypto-fallback is used, the ciphertext written to disk (and hence the -on-disk format of data) will be the same (assuming the hardware's implementation -of the algorithm being used adheres to spec and functions correctly). - -If a ``request queue``'s inline encryption hardware claimed to support the -encryption context specified with a bio, then it will not be handled by the -``blk-crypto-fallback``. We will eventually reach a point in blk-mq when a -struct request needs to be allocated for that bio. At that point, -blk-mq tries to program the encryption context into the ``request_queue``'s -keyslot_manager, and obtain a keyslot, which it stores in its newly added -``keyslot`` field. This keyslot is released when the request is completed. - -When the first bio is added to a request, ``blk_crypto_rq_bio_prep`` is called, -which sets the request's ``crypt_ctx`` to a copy of the bio's -``bi_crypt_context``. bio_crypt_do_front_merge is called whenever a subsequent -bio is merged to the front of the request, which updates the ``crypt_ctx`` of -the request so that it matches the newly merged bio's ``bi_crypt_context``. In particular, the request keeps a copy of the ``bi_crypt_context`` of the first -bio in its bio-list (blk-mq needs to be careful to maintain this invariant -during bio and request merges). - -To make it possible for inline encryption to work with request queue based -layered devices, when a request is cloned, its ``crypto fields`` are cloned as -well. When the cloned request is submitted, blk-mq programs the -``bi_crypt_context`` of the request into the clone's request_queue's keyslot -manager, and stores the returned keyslot in the clone's ``keyslot``. +on-disk format of data) will be the same (assuming that both the inline +encryption hardware's implementation and the kernel crypto API's implementation +of the algorithm being used adhere to spec and function correctly). +blk-crypto-fallback is optional and is controlled by the +``CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK`` kernel configuration option. API presented to users of the block layer ========================================= -``struct blk_crypto_key`` represents a crypto key (the raw key, size of the -key, the crypto algorithm to use, the data unit size to use, and the number of -bytes required to represent data unit numbers that will be specified with the -``bi_crypt_context``). - -``blk_crypto_init_key`` allows upper layers to initialize such a -``blk_crypto_key``. - -``bio_crypt_set_ctx`` should be called on any bio that a user of -the block layer wants en/decrypted via inline encryption (or the -blk-crypto-fallback, if hardware support isn't available for the desired -crypto configuration). This function takes the ``blk_crypto_key`` and the -data unit number (DUN) to use when en/decrypting the bio. - -``blk_crypto_config_supported`` allows upper layers to query whether or not the -an encryption context passed to request queue can be handled by blk-crypto -(either by real inline encryption hardware, or by the blk-crypto-fallback). -This is useful e.g. when blk-crypto-fallback is disabled, and the upper layer -wants to use an algorithm that may not supported by hardware - this function -lets the upper layer know ahead of time that the algorithm isn't supported, -and the upper layer can fallback to something else if appropriate. - -``blk_crypto_start_using_key`` - Upper layers must call this function on -``blk_crypto_key`` and a ``request_queue`` before using the key with any bio -headed for that ``request_queue``. This function ensures that either the -hardware supports the key's crypto settings, or the crypto API fallback has -transforms for the needed mode allocated and ready to go. Note that this -function may allocate an ``skcipher``, and must not be called from the data -path, since allocating ``skciphers`` from the data path can deadlock. - -``blk_crypto_evict_key`` *must* be called by upper layers before a -``blk_crypto_key`` is freed. Further, it *must* only be called only once -there are no more in-flight requests that use that ``blk_crypto_key``. -``blk_crypto_evict_key`` will ensure that a key is removed from any keyslots in -inline encryption hardware that the key might have been programmed into (or the blk-crypto-fallback). +``blk_crypto_config_supported()`` allows users to check ahead of time whether +inline encryption with particular crypto settings will work on a particular +request_queue -- either via hardware or via blk-crypto-fallback. This function +takes in a ``struct blk_crypto_config`` which is like blk_crypto_key, but omits +the actual bytes of the key and instead just contains the algorithm, data unit +size, etc. This function can be useful if blk-crypto-fallback is disabled. + +``blk_crypto_init_key()`` allows users to initialize a blk_crypto_key. + +Users must call ``blk_crypto_start_using_key()`` before actually starting to use +a blk_crypto_key on a request_queue (even if ``blk_crypto_config_supported()`` +was called earlier). This is needed to initialize blk-crypto-fallback if it +will be needed. This must not be called from the data path, as this may have to +allocate resources, which may deadlock in that case. + +Next, to attach an encryption context to a bio, users should call +``bio_crypt_set_ctx()``. This function allocates a bio_crypt_ctx and attaches +it to a bio, given the blk_crypto_key and the data unit number that will be used +for en/decryption. Users don't need to worry about freeing the bio_crypt_ctx +later, as that happens automatically when the bio is freed or reset. + +Finally, when done using inline encryption with a blk_crypto_key on a +request_queue, users must call ``blk_crypto_evict_key()``. This ensures that +the key is evicted from all keyslots it may be programmed into and unlinked from +any kernel data structures it may be linked into. + +In summary, for users of the block layer, the lifecycle of a blk_crypto_key is +as follows: + +1. ``blk_crypto_config_supported()`` (optional) +2. ``blk_crypto_init_key()`` +3. ``blk_crypto_start_using_key()`` +4. ``bio_crypt_set_ctx()`` (potentially many times) +5. ``blk_crypto_evict_key()`` (after all I/O has completed) +6. Zeroize the blk_crypto_key (this has no dedicated function) + +If a blk_crypto_key is being used on multiple request_queues, then +``blk_crypto_config_supported()`` (if used), ``blk_crypto_start_using_key()``, +and ``blk_crypto_evict_key()`` must be called on each request_queue. API presented to device drivers =============================== -A :c:type:``struct blk_keyslot_manager`` should be set up by device drivers in -the ``request_queue`` of the device. The device driver needs to call -``blk_ksm_init`` (or its resource-managed variant ``devm_blk_ksm_init``) on the -``blk_keyslot_manager``, while specifying the number of keyslots supported by -the hardware. - -The device driver also needs to tell the KSM how to actually manipulate the -IE hardware in the device to do things like programming the crypto key into -the IE hardware into a particular keyslot. All this is achieved through the -struct blk_ksm_ll_ops field in the KSM that the device driver -must fill up after initing the ``blk_keyslot_manager``. - -The KSM also handles runtime power management for the device when applicable -(e.g. when it wants to program a crypto key into the IE hardware, the device -must be runtime powered on) - so the device driver must also set the ``dev`` -field in the ksm to point to the `struct device` for the KSM to use for runtime -power management. - -``blk_ksm_reprogram_all_keys`` can be called by device drivers if the device -needs each and every of its keyslots to be reprogrammed with the key it -"should have" at the point in time when the function is called. This is useful -e.g. if a device loses all its keys on runtime power down/up. - -If the driver used ``blk_ksm_init`` instead of ``devm_blk_ksm_init``, then -``blk_ksm_destroy`` should be called to free up all resources used by a -``blk_keyslot_manager`` once it is no longer needed. +A device driver that wants to support inline encryption must set up a +blk_crypto_profile in the request_queue of its device. To do this, it first +must call ``blk_crypto_profile_init()`` (or its resource-managed variant +``devm_blk_crypto_profile_init()``), providing the number of keyslots. + +Next, it must advertise its crypto capabilities by setting fields in the +blk_crypto_profile, e.g. ``modes_supported`` and ``max_dun_bytes_supported``. + +It then must set function pointers in the ``ll_ops`` field of the +blk_crypto_profile to tell upper layers how to control the inline encryption +hardware, e.g. how to program and evict keyslots. Most drivers will need to +implement ``keyslot_program`` and ``keyslot_evict``. For details, see the +comments for ``struct blk_crypto_ll_ops``. + +Once the driver registers a blk_crypto_profile with a request_queue, I/O +requests the driver receives via that queue may have an encryption context. All +encryption contexts will be compatible with the crypto capabilities declared in +the blk_crypto_profile, so drivers don't need to worry about handling +unsupported requests. Also, if a nonzero number of keyslots was declared in the +blk_crypto_profile, then all I/O requests that have an encryption context will +also have a keyslot which was already programmed with the appropriate key. + +If the driver implements runtime suspend and its blk_crypto_ll_ops don't work +while the device is runtime-suspended, then the driver must also set the ``dev`` +field of the blk_crypto_profile to point to the ``struct device`` that will be +resumed before any of the low-level operations are called. + +If there are situations where the inline encryption hardware loses the contents +of its keyslots, e.g. device resets, the driver must handle reprogramming the +keyslots. To do this, the driver may call ``blk_crypto_reprogram_all_keys()``. + +Finally, if the driver used ``blk_crypto_profile_init()`` instead of +``devm_blk_crypto_profile_init()``, then it is responsible for calling +``blk_crypto_profile_destroy()`` when the crypto profile is no longer needed. Layered Devices =============== -Request queue based layered devices like dm-rq that wish to support IE need to -create their own keyslot manager for their request queue, and expose whatever -functionality they choose. When a layered device wants to pass a clone of that -request to another ``request_queue``, blk-crypto will initialize and prepare the -clone as necessary - see ``blk_crypto_insert_cloned_request`` in -``blk-crypto.c``. - - -Future Optimizations for layered devices -======================================== - -Creating a keyslot manager for a layered device uses up memory for each -keyslot, and in general, a layered device merely passes the request on to a -"child" device, so the keyslots in the layered device itself are completely -unused, and don't need any refcounting or keyslot programming. We can instead -define a new type of KSM; the "passthrough KSM", that layered devices can use -to advertise an unlimited number of keyslots, and support for any encryption -algorithms they choose, while not actually using any memory for each keyslot. -Another use case for the "passthrough KSM" is for IE devices that do not have a -limited number of keyslots. - +Request queue based layered devices like dm-rq that wish to support inline +encryption need to create their own blk_crypto_profile for their request_queue, +and expose whatever functionality they choose. When a layered device wants to +pass a clone of that request to another request_queue, blk-crypto will +initialize and prepare the clone as necessary; see +``blk_crypto_insert_cloned_request()``. Interaction between inline encryption and blk integrity =======================================================
@@ -257,7 +296,7 @@ Because there isn't any real hardware yet, it seems prudent to assume that hardware implementations might not implement both features together correctly, and disallow the combination for now. Whenever a device supports integrity, the kernel will pretend that the device does not support hardware inline encryption -(by essentially setting the keyslot manager in the request_queue of the device -to NULL). When the crypto API fallback is enabled, this means that all bios with -and encryption context will use the fallback, and IO will complete as usual. -When the fallback is disabled, a bio with an encryption context will be failed. +(by setting the blk_crypto_profile in the request_queue of the device to NULL). +When the crypto API fallback is enabled, this means that all bios with and +encryption context will use the fallback, and IO will complete as usual. When +the fallback is disabled, a bio with an encryption context will be failed.
Signature · 1 line
2.33.0
Eric,
Quoted 4 lines
This series renames struct
blk_keyslot_managerto structblk_crypto_profile, as it is misnamed; it doesn't always manage keyslots. It's much more logical to think of it as the "blk-crypto profile" of a device, similar toblk_integrity_profile.
Looks good to me, easier to follow.
Reviewed-by Martin K. Petersen <[email protected]>
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering