Skip to contentSkip to conversation

Inline Encryption Support

PATCH Superseded Satya Tangirala 3 participants 23 messages Mar 12, 2020 – Mar 19, 2020

Subsystems (8)
Documentation VFS SCSI Block layer +4 more subsystems

v8 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#0 lore raw

v8 cover Inline Encryption Support

This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.

Note that the patches in this series for the block layer (i.e. patches 1, 2, 3 and 4) can be applied independently of the subsequent patches in this series.

Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device. Inline Encryption hardware has become increasingly common, and we want to support it in the kernel.

Inline Encryption hardware implementations often function around the concept of a limited number of "keyslots", which can hold an encryption context each. The storage device can be directed to en/decrypt any particular request with the encryption context stored in any particular keyslot.

Patch 1 introduces a Keyslot Manager to efficiently manage keyslots. The keyslot manager also functions as the interface that blk-crypto (introduced in Patch 2), will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.

Patch 2 adds the block layer changes for inline encryption support. It introduces struct bio_crypt_ctx, and a ptr to one in struct bio, which allows struct bio to represent an encryption context that can be passed down the storage stack from the filesystem layer to the storage driver.

Patch 3 precludes inline encryption support in a device whenever it supports blk-integrity, because there is currently no known hardware that supports both features, and it is not completely straightfoward to support both of them properly, and doing it improperly might result in leaks of information about the plaintext.

Patch 4 introduces blk-crypto-fallback - a kernel crypto API fallback for blk-crypto to use when inline encryption hardware isn't present. This allows filesystems to specify encryption contexts for bios without having to worry about whether the underlying hardware has inline encryption support, and allows for testing without real hardware inline encryption support. This fallback is separately configurable from blk-crypto, and can be disabled if desired while keeping inline encryption support. It may also be possible to remove file content en/decryption from fscrypt and simply use blk-crypto-fallback in a future patch. For more details on blk-crypto and the fallback, refer to Documentation/block/inline-encryption.rst.

Patches 5-7 add support for inline encryption into the UFS driver according to the JEDEC UFS HCI v2.1 specification. Inline encryption support for other drivers (like eMMC) may be added in the same way - the device driver should set up a Keyslot Manager in the device's request_queue (refer to the UFS crypto additions in ufshcd-crypto.c and ufshcd.c for an example).

Patch 8 adds the SB_INLINECRYPT mount flag to the fs layer, which filesystems must set to indicate that they want to use blk-crypto for en/decryption of file contents.

Patch 9 adds support to fscrypt - to use inline encryption with fscrypt, the filesystem must be mounted with '-o inlinecrypt' - when this option is specified, the contents of any AES-256-XTS encrypted file will be encrypted using blk-crypto.

Patches 10 and 11 add support to f2fs and ext4 respectively, so that we have a complete stack that can make use of inline encryption.

The patches were tested running kvm-xfstests, by specifying the introduced "inlinecrypt" mount option, so that en/decryption happens with the blk-crypto fallback. The patches were also tested on a Pixel 4 with UFS hardware that has support for inline encryption.

There have been a few patch sets addressing Inline Encryption Support in the past. Briefly, this patch set differs from those as follows:

1) "crypto: qce: ice: Add support for Inline Crypto Engine" is specific to certain hardware, while our patch set's Inline Encryption support for UFS is implemented according to the JEDEC UFS specification.

2) "scsi: ufs: UFS Host Controller crypto changes" registers inline encryption support as a kernel crypto algorithm. Our patch views inline encryption as being fundamentally different from a generic crypto provider (in that inline encryption is tied to a device), and so does not use the kernel crypto API to represent inline encryption hardware.

3) "scsi: ufs: add real time/inline crypto support to UFS HCD" requires the device mapper to work - our patch does not.

Changes v7 => v8:
- Pass a struct blk_ksm_keyslot * around instead of slot numbers which simplifies some functions and passes around arguments with better types
- Make bios with no encryption context avoid making calls into blk-crypto by checking for the presence of bi_crypt_context before making the call
- Make blk-integrity preclude inline encryption support at probe time
- Many many cleanups

Changes v6 => v7:
- Keyslot management is now done on a per-request basis rather than a per-bio basis.
- Storage drivers can now specify the maximum number of bytes they can accept for the data unit number (DUN) for each crypto algorithm, and upper layers can specify the minimum number of bytes of DUN they want with the blk_crypto_key they send with the bio - a driver is only considered to support a blk_crypto_key if the driver supports at least as many DUN bytes as the upper layer wants. This is necessary because storage drivers may not support as many bytes as the algorithm specification dictates (for e.g. UFS only supports 8 byte DUNs for AES-256-XTS, even though the algorithm specification says DUNs are 16 bytes long).
- Introduce SB_INLINECRYPT to keep track of whether inline encryption is enabled for a filesystem (instead of using an fscrypt_operation).
- Expose keyslot manager declaration and embed it within ufs_hba to clean up code.
- Make blk-crypto preclude blk-integrity.
- Some bug fixes
- Introduce UFSHCD_QUIRK_BROKEN_CRYPTO for UFS drivers that don't support inline encryption (yet)

Changes v5 => v6:
- Blk-crypto's kernel crypto API fallback is no longer restricted to 8-byte DUNs. It's also now separately configurable from blk-crypto, and can be disabled entirely, while still allowing the kernel to use inline encryption hardware. Further, struct bio_crypt_ctx takes up less space, and no longer contains the information needed by the crypto API fallback - the fallback allocates the required memory when necessary.
- Blk-crypto now supports all file content encryption modes supported by fscrypt.
- Fixed bio merging logic in blk-merge.c
- Fscrypt now supports inline encryption with the direct key policy, since blk-crypto now has support for larger DUNs.
- Keyslot manager now uses a hashtable to lookup which keyslot contains any particular key (thanks Eric!)
- Fscrypt support for inline encryption now handles filesystems with multiple underlying block devices (thanks Eric!)
- Numerous cleanups

Changes v4 => v5:
- The fscrypt patch has been separated into 2. The first adds support for the IV_INO_LBLK_64 policy (which was called INLINE_CRYPT_OPTIMIZED in past versions of this series). This policy is now purely an on disk format, and doesn't dictate whether blk-crypto is used for file content encryption or not. Instead, this is now decided based on the "inlinecrypt" mount option.
- Inline crypto key eviction is now handled by blk-crypto instead of fscrypt.
- More refactoring.

Changes v3 => v4:
- Fixed the issue with allocating crypto_skcipher in blk_crypto_keyslot_program.
- bio_crypto_alloc_ctx is now mempool backed.
- In f2fs, a bio's bi_crypt_context is now set up when the bio is allocated, rather than just before the bio is submitted - this fixes bugs in certain cases, like when an encrypted block is being moved without decryption.
- Lots of refactoring and cleanup of blk-crypto - thanks Eric!

Changes v2 => v3:
- Overhauled keyslot manager's get keyslot logic and optimized LRU.
- Block crypto en/decryption fallback now supports data unit sizes that divide the bvec length (instead of requiring each bvec's length to be the same as the data unit size).
- fscrypt master key is now keyed additionally by super_block and ci_ctfm != NULL.
- all references of "hw encryption" are replaced by inline encryption.
- address various other review comments from Eric.

Changes v1 => v2:
- Block layer and UFS changes are split into 3 patches each.
- We now only have a ptr to a struct bio_crypt_ctx in struct bio, instead of the struct itself.
- struct bio_crypt_ctx no longer has flags.
- blk-crypto now correctly handles the case when it fails to init (because of insufficient memory), but kernel continues to boot.
- ufshcd-crypto now works on big endian cpus.
- Many cleanups.

Eric Biggers (1):
ext4: add inline encryption support

Satya Tangirala (10):
block: Keyslot Manager for Inline Encryption
block: Inline encryption support for blk-mq
block: Make blk-integrity preclude hardware inline encryption
block: blk-crypto-fallback for Inline Encryption
scsi: ufs: UFS driver v2.1 spec crypto additions
scsi: ufs: UFS crypto API
scsi: ufs: Add inline encryption support to UFS
fs: introduce SB_INLINECRYPT
fscrypt: add inline encryption support
f2fs: add inline encryption support

 Documentation/admin-guide/ext4.rst        |   6 +
 Documentation/block/index.rst             |   1 +
 Documentation/block/inline-encryption.rst | 195 +++++++
 Documentation/filesystems/f2fs.txt        |   6 +
 block/Kconfig                             |  17 +
 block/Makefile                            |   2 +
 block/bio-integrity.c                     |   5 +
 block/bio.c                               |   9 +-
 block/blk-core.c                          |  27 +-
 block/blk-crypto-fallback.c               | 662 ++++++++++++++++++++++
 block/blk-crypto-internal.h               | 168 ++++++
 block/blk-crypto.c                        | 429 ++++++++++++++
 block/blk-integrity.c                     |   7 +
 block/blk-map.c                           |   2 +
 block/blk-merge.c                         |  11 +
 block/blk-mq.c                            |  15 +
 block/blk.h                               |   5 +
 block/bounce.c                            |   3 +
 block/keyslot-manager.c                   | 397 +++++++++++++
 drivers/md/dm.c                           |   4 +
 drivers/scsi/ufs/Kconfig                  |   9 +
 drivers/scsi/ufs/Makefile                 |   1 +
 drivers/scsi/ufs/ufshcd-crypto.c          | 397 +++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h          |  68 +++
 drivers/scsi/ufs/ufshcd.c                 |  61 +-
 drivers/scsi/ufs/ufshcd.h                 |  27 +
 drivers/scsi/ufs/ufshci.h                 |  67 ++-
 fs/buffer.c                               |   7 +-
 fs/crypto/Kconfig                         |   6 +
 fs/crypto/Makefile                        |   1 +
 fs/crypto/bio.c                           |  51 ++
 fs/crypto/crypto.c                        |   2 +-
 fs/crypto/fname.c                         |   4 +-
 fs/crypto/fscrypt_private.h               | 120 +++-
 fs/crypto/inline_crypt.c                  | 328 +++++++++++
 fs/crypto/keyring.c                       |   4 +-
 fs/crypto/keysetup.c                      |  92 ++-
 fs/crypto/keysetup_v1.c                   |  16 +-
 fs/ext4/inode.c                           |   4 +-
 fs/ext4/page-io.c                         |   6 +-
 fs/ext4/readpage.c                        |  11 +-
 fs/ext4/super.c                           |   9 +
 fs/f2fs/compress.c                        |   2 +-
 fs/f2fs/data.c                            |  68 ++-
 fs/f2fs/f2fs.h                            |   3 +
 fs/f2fs/super.c                           |  33 ++
 fs/proc_namespace.c                       |   1 +
 include/linux/blk-crypto.h                | 130 +++++
 include/linux/blk_types.h                 |   6 +
 include/linux/blkdev.h                    |  41 ++
 include/linux/fs.h                        |   1 +
 include/linux/fscrypt.h                   |  57 ++
 include/linux/keyslot-manager.h           | 108 ++++
 53 files changed, 3622 insertions(+), 90 deletions(-)
 create mode 100644 Documentation/block/inline-encryption.rst
 create mode 100644 block/blk-crypto-fallback.c
 create mode 100644 block/blk-crypto-internal.h
 create mode 100644 block/blk-crypto.c
 create mode 100644 block/keyslot-manager.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
 create mode 100644 fs/crypto/inline_crypt.c
 create mode 100644 include/linux/blk-crypto.h
 create mode 100644 include/linux/keyslot-manager.h
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
#1 lore raw

PATCH v8 1/11 block: Keyslot Manager for Inline Encryption

Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.

Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold a key (we say that a key can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot and a data unit number associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the key programmed into that associated keyslot and the data unit number specified with the request.

As keyslots are limited, and programming keys may be expensive in many implementations, and multiple requests may use exactly the same encryption contexts, we introduce a Keyslot Manager to efficiently manage keyslots.

We also introduce a blk_crypto_key, which will represent the key that's programmed into keyslots managed by keyslot managers. The keyslot manager also functions as the interface that upper layers will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.

---
 block/Kconfig                   |   7 +
 block/Makefile                  |   1 +
 block/keyslot-manager.c         | 377 ++++++++++++++++++++++++++++++++
 include/linux/blk-crypto.h      |  45 ++++
 include/linux/blkdev.h          |   6 +
 include/linux/keyslot-manager.h | 108 +++++++++
 6 files changed, 544 insertions(+)
 create mode 100644 block/keyslot-manager.c
 create mode 100644 include/linux/blk-crypto.h
 create mode 100644 include/linux/keyslot-manager.h
block/Kconfig +7 -0
index 3bc76bb113a0..c04a1d500842 100644
@@ -185,6 +185,13 @@ config BLK_SED_OPAL
 	Enabling this option enables users to setup/unlock/lock
 	Locking ranges for SED devices using the Opal protocol.
 
+config BLK_INLINE_ENCRYPTION
+	bool "Enable inline encryption support in block layer"
+	help
+	  Build the blk-crypto subsystem. Enabling this lets the
+	  block layer handle encryption, so users can take
+	  advantage of inline encryption hardware if present.
+
 menu "Partition Types"
 
 source "block/partitions/Kconfig"
block/Makefile +1 -0
index 1a43750f4b01..ef3a05dcf1f2 100644
@@ -37,3 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS)	+= blk-mq-debugfs.o
 obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o
block/keyslot-manager.c new file +377 -0

377 lines not on this page · raw message

include/linux/blk-crypto.h new file +45 -0

45 lines not on this page · raw message

include/linux/blkdev.h +6 -0
index 10455b2bbbb4..c447361952d1 100644
@@ -43,6 +43,7 @@ struct pr_ops;
 struct rq_qos;
 struct blk_queue_stats;
 struct blk_stat_callback;
+struct keyslot_manager;
 
 #define BLKDEV_MIN_RQ	4
 #define BLKDEV_MAX_RQ	128	/* Default maximum */
@@ -474,6 +475,11 @@ struct request_queue {
 	unsigned int		dma_pad_mask;
 	unsigned int		dma_alignment;
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	/* Inline crypto capabilities */
+	struct keyslot_manager *ksm;
+#endif
+
 	unsigned int		rq_timeout;
 	int			poll_nsec;
 
include/linux/keyslot-manager.h new file +108 -0

108 lines not on this page · raw message

Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#2 lore raw

Re: [PATCH v8 01/11] block: Keyslot Manager for Inline Encryption

Quoted 18 lines · Satya Tangirala
block/keyslot-manager.c new file +18 -0
+/**
+ * blk_ksm_init() - Initialize a keyslot manager
+ * @ksm: The keyslot_manager to initialize.
+ * @dev: Device for runtime power management (NULL if none)
+ * @num_slots: The number of key slots to manage.
+ *
+ * Allocate memory for keyslots and initialize a keyslot manager. Called by
+ * e.g. storage drivers to set up a keyslot manager in their request_queue.
+ *
+ * Return: 0 on success, or else a negative error code.
+ */
+int blk_ksm_init(struct keyslot_manager *ksm, struct device *dev,
+		 unsigned int num_slots)
+{
+	unsigned int slot;
+	unsigned int i;
+
+	memset(ksm, 0, sizeof(*ksm));

block/keyslot-manager.c line 90

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+
+	if (num_slots == 0)
+		return -EINVAL;
+
+	ksm->slots = kvcalloc(num_slots, sizeof(ksm->slots[0]), GFP_KERNEL);
+	if (!ksm->slots)
+		return -ENOMEM;
+
+	ksm->num_slots = num_slots;
+	ksm->dev = dev;

Seems that now that keyslot_manager::dev is an unconditional field and callers initialize other fields of the keyslot_manager, the 'dev' parameter should be removed from blk_ksm_init() and the caller should be responsible for setting 'dev' if they need runtime power management.

Quoted 8 lines

+/**
+ * blk_ksm_get_slot_for_key() - Program a key into a keyslot.
+ * @ksm: The keyslot manager to program the key into.
+ * @key: Pointer to the key object to program, including the raw key, crypto
+ * mode, and data unit size.
+ * @keyslot: A pointer to return the pointer of the allocated keyslot.
+ *
+ * Get a keyslot that's been programmed with the specified key. If one already

Quoted 8 lines

+ * exists, return it with incremented refcount. Otherwise, wait for a keyslot
+ * to become idle and program it.
+ *
+ * Context: Process context. Takes and releases ksm->lock.
+ * Return: BLK_STATUS_OK on success (and keyslot is set to the pointer of the
+ * allocated keyslot), and BLK_STATUS_IOERR otherwise (and keyslot is
+ * set to NULL).
+ */

There can be other errors besides BLK_STATUS_IOERR returned.

Quoted 8 lines
block/keyslot-manager.c new file +8 -0
+/**
+ * blk_ksm_put_slot() - Release a reference to a slot
+ * @slot: The keyslot to release the reference of.
+ *
+ * Context: Any context.
+ */
+void blk_ksm_put_slot(struct blk_ksm_keyslot *slot)
+{

block/keyslot-manager.c line 272

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+	struct keyslot_manager *ksm = slot->ksm;
+	unsigned long flags;
+
+	if (!slot)
+		return;
+
+	if (atomic_dec_and_lock_irqsave(&slot->slot_refs,
+					&ksm->idle_slots_lock, flags)) {
+		list_add_tail(&slot->idle_slot_node,
+			      &ksm->idle_slots);

Nit: the arguments to list_add_tail() fit on one line

Quoted 30 lines
include/linux/keyslot-manager.h new file +24 -0
new file mode 100644
index 000000000000..7f88ed02faee
@@ -0,0 +1,108 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright 2019 Google LLC
+ */
+
+#ifndef __LINUX_KEYSLOT_MANAGER_H
+#define __LINUX_KEYSLOT_MANAGER_H
+
+#include <linux/bio.h>
+#include <linux/blk-crypto.h>
+
+struct keyslot_manager;
+
+/**
+ * struct keyslot_mgmt_ll_ops - functions to manage keyslots in hardware
+ * @keyslot_program:	Program the specified key into the specified slot in the
+ *			inline encryption hardware.
+ * @keyslot_evict:	Evict key from the specified keyslot in the hardware.
+ *			The key is provided so that e.g. dm layers can evict
+ *			keys from the devices that they map over.
+ *			Returns 0 on success, -errno otherwise.
+ *
+ * This structure should be provided by storage device drivers when they set up
+ * a keyslot manager - this structure holds the function ptrs that the keyslot

include/linux/keyslot-manager.h line 34

Quoted 10 lines
include/linux/keyslot-manager.h new file +10 -0
+ * manager will use to manipulate keyslots in the hardware.
+ */
+struct keyslot_mgmt_ll_ops {
+	blk_status_t (*keyslot_program)(struct keyslot_manager *ksm,
+					const struct blk_crypto_key *key,
+					unsigned int slot);
+	int (*keyslot_evict)(struct keyslot_manager *ksm,
+			     const struct blk_crypto_key *key,
+			     unsigned int slot);
+};

As I mentioned on one of the UFS patches, I'm not sure it's better to make keyslot_program return blk_status_t rather than errno as it did before. blk_ksm_get_slot_for_key() could just do the translation to blk_status_t.

include/linux/keyslot-manager.h line 36

Quoted 1 line
include/linux/keyslot-manager.h new file +1 -0
+struct keyslot_manager {

Now that the "blk_" prefix has been added to all the functions, perhaps the actual struct should be renamed to "blk_keyslot_manager" too? And likewise "blk_keyslot_mgmt_ll_ops".

#3 lore raw

PATCH v8 2/11 block: Inline encryption support for blk-mq

In reply to Satya Tangirala

We must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the upper layers (like the filesystem/fscrypt) that know about and manages encryption contexts. As such, when the upper layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.

We want to communicate the encryption context from the upper layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct bio_crypt_ctx to struct bio, which can represent an encryption context (note that we can't use the bi_private field in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate the bio_crypt_ctx and make the bio/request merging logic aware of the bio_crypt_ctx.

We also make changes to blk-mq to make it handle bios with encryption contexts. blk-mq can merge many bios into the same request. These bios need to have contiguous data unit numbers (the necessary changes to blk-merge are also made to ensure this) - as such, it suffices to keep the data unit number of just the first bio, since that's all a storage driver needs to infer the data unit number to use for each data block in each bio in a request. blk-mq keeps track of the encryption context to be used for all the bios in a request with the request's rq_crypt_ctx. When the first bio is added to an empty request, blk-mq will program the encryption context of that bio into the request_queue's keyslot manager, and store the returned keyslot in the request's rq_crypt_ctx. All the functions to operate on encryption contexts are in blk-crypto.c.

Upper layers only need to call bio_crypt_set_ctx with the encryption key, algorithm and data_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-mq/blk-crypto handles that. Blk-crypto also makes it possible for request-based layered devices like dm-rq to make use of inline encryption hardware by cloning the rq_crypt_ctx and programming a keyslot in the new request_queue when necessary.

Note that any user of the block layer can submit bios with an encryption context, such as filesystems, device-mapper targets, etc.

---
 block/Makefile              |   2 +-
 block/bio.c                 |   9 +-
 block/blk-core.c            |  27 ++-
 block/blk-crypto-internal.h | 144 +++++++++++++
 block/blk-crypto.c          | 408 ++++++++++++++++++++++++++++++++++++
 block/blk-map.c             |   2 +
 block/blk-merge.c           |  11 +
 block/blk-mq.c              |  15 ++
 block/blk.h                 |   5 +
 block/bounce.c              |   3 +
 drivers/md/dm.c             |   4 +
 include/linux/blk-crypto.h  |  72 ++++++-
 include/linux/blk_types.h   |   6 +
 include/linux/blkdev.h      |   5 +
 14 files changed, 706 insertions(+), 7 deletions(-)
 create mode 100644 block/blk-crypto-internal.h
 create mode 100644 block/blk-crypto.c
block/Makefile +1 -1
index ef3a05dcf1f2..82f42ca3f769 100644
@@ -37,4 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS)	+= blk-mq-debugfs.o
 obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
-obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o blk-crypto.o
block/bio.c +8 -1
index 94d697217887..9e2542809279 100644
@@ -17,6 +17,7 @@
 #include <linux/cgroup.h>
 #include <linux/blk-cgroup.h>
 #include <linux/highmem.h>
+#include <linux/blk-crypto.h>
 
 #include <trace/events/block.h>
 #include "blk.h"
@@ -236,6 +237,9 @@ void bio_uninit(struct bio *bio)
 
 	if (bio_integrity(bio))
 		bio_integrity_free(bio);
+
+	if (bio_has_crypt_ctx(bio))
+		bio_crypt_free_ctx(bio);
 }
 EXPORT_SYMBOL(bio_uninit);
 
@@ -664,11 +668,13 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)
 
 	__bio_clone_fast(b, bio);
 
+	if (bio_has_crypt_ctx(bio))
+		bio_crypt_clone(b, bio, gfp_mask);
+
 	if (bio_integrity(bio)) {
 		int ret;
 
 		ret = bio_integrity_clone(b, bio, gfp_mask);
-
 		if (ret < 0) {
 			bio_put(b);
 			return NULL;
@@ -1046,6 +1052,7 @@ void bio_advance(struct bio *bio, unsigned bytes)
 	if (bio_integrity(bio))
 		bio_integrity_advance(bio, bytes);
 
+	bio_crypt_advance(bio, bytes);
 	bio_advance_iter(bio, &bio->bi_iter, bytes);
 }
 EXPORT_SYMBOL(bio_advance);
block/blk-core.c +23 -4
index 089e890ab208..199837d1c15d 100644
@@ -38,6 +38,7 @@
 #include <linux/debugfs.h>
 #include <linux/bpf.h>
 #include <linux/psi.h>
+#include <linux/blk-crypto.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/block.h>
@@ -120,6 +121,7 @@ void blk_rq_init(struct request_queue *q, struct request *rq)
 	rq->start_time_ns = ktime_get_ns();
 	rq->part = NULL;
 	refcount_set(&rq->ref, 1);
+	blk_crypto_rq_set_defaults(rq);
 }
 EXPORT_SYMBOL(blk_rq_init);
 
@@ -617,6 +619,9 @@ bool bio_attempt_back_merge(struct request *req, struct bio *bio,
 	req->biotail = bio;
 	req->__data_len += bio->bi_iter.bi_size;
 
+	if (bio_has_crypt_ctx(bio))
+		bio_crypt_free_ctx(bio);
+
 	blk_account_io_start(req, false);
 	return true;
 }
@@ -641,6 +646,9 @@ bool bio_attempt_front_merge(struct request *req, struct bio *bio,
 	req->__sector = bio->bi_iter.bi_sector;
 	req->__data_len += bio->bi_iter.bi_size;
 
+	if (bio_has_crypt_ctx(bio))
+		blk_crypto_rq_bio_prep(req, bio);
+
 	blk_account_io_start(req, false);
 	return true;
 }
@@ -1066,7 +1074,9 @@ blk_qc_t generic_make_request(struct bio *bio)
 			/* Create a fresh bio_list for all subordinate requests */
 			bio_list_on_stack[1] = bio_list_on_stack[0];
 			bio_list_init(&bio_list_on_stack[0]);
-			ret = q->make_request_fn(q, bio);
+			if (!bio_has_crypt_ctx(bio) ||
+			    !blk_crypto_bio_prep(&bio))
+				ret = q->make_request_fn(q, bio);
 
 			blk_queue_exit(q);
 
@@ -1114,7 +1124,7 @@ blk_qc_t direct_make_request(struct bio *bio)
 {
 	struct request_queue *q = bio->bi_disk->queue;
 	bool nowait = bio->bi_opf & REQ_NOWAIT;
-	blk_qc_t ret;
+	blk_qc_t ret = BLK_QC_T_NONE;
 
 	if (!generic_make_request_checks(bio))
 		return BLK_QC_T_NONE;
@@ -1127,8 +1137,8 @@ blk_qc_t direct_make_request(struct bio *bio)
 		bio_endio(bio);
 		return BLK_QC_T_NONE;
 	}
-
-	ret = q->make_request_fn(q, bio);
+	if (!bio_has_crypt_ctx(bio) || !blk_crypto_bio_prep(&bio))
+		ret = q->make_request_fn(q, bio);
 	blk_queue_exit(q);
 	return ret;
 }
@@ -1258,6 +1268,10 @@ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request *
 	    should_fail_request(&rq->rq_disk->part0, blk_rq_bytes(rq)))
 		return BLK_STS_IOERR;
 
+	if (blk_crypto_rq_is_encrypted(rq) &&
+	    blk_crypto_insert_cloned_request(rq))
+		return BLK_STS_IOERR;
+
 	if (blk_queue_io_stat(q))
 		blk_account_io_start(rq, true);
 
@@ -1646,6 +1660,9 @@ int blk_rq_prep_clone(struct request *rq, struct request *rq_src,
 
 	__blk_rq_prep_clone(rq, rq_src);
 
+	if (blk_crypto_rq_is_encrypted(rq_src))
+		blk_crypto_rq_prep_clone(rq, rq_src);
+
 	return 0;
 
 free_and_out:
@@ -1813,5 +1830,7 @@ int __init blk_dev_init(void)
 	blk_debugfs_root = debugfs_create_dir("block", NULL);
 #endif
 
+	bio_crypt_ctx_init();
+
 	return 0;
 }
block/blk-crypto-internal.h new file +144 -0

144 lines not on this page · raw message

block/blk-crypto.c new file +408 -0

408 lines not on this page · raw message

block/blk-map.c +2 -0
index b0790268ed9d..1c8acc39584d 100644
@@ -41,6 +41,8 @@ int blk_rq_append_bio(struct request *rq, struct bio **bio)
 		rq->biotail->bi_next = *bio;
 		rq->biotail = *bio;
 		rq->__data_len += (*bio)->bi_iter.bi_size;
+		if (bio_has_crypt_ctx(*bio))
+			bio_crypt_free_ctx(*bio);
 	}
 
 	return 0;
block/blk-merge.c +11 -0
index 1534ed736363..a0c24b6e0eb3 100644
@@ -596,6 +596,8 @@ int ll_back_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs)
 	if (blk_integrity_rq(req) &&
 	    integrity_req_gap_back_merge(req, bio))
 		return 0;
+	if (!bio_crypt_ctx_back_mergeable(req, bio))
+		return 0;
 	if (blk_rq_sectors(req) + bio_sectors(bio) >
 	    blk_rq_get_max_sectors(req, blk_rq_pos(req))) {
 		req_set_nomerge(req->q, req);
@@ -612,6 +614,8 @@ int ll_front_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs
 	if (blk_integrity_rq(req) &&
 	    integrity_req_gap_front_merge(req, bio))
 		return 0;
+	if (!bio_crypt_ctx_front_mergeable(req, bio))
+		return 0;
 	if (blk_rq_sectors(req) + bio_sectors(bio) >
 	    blk_rq_get_max_sectors(req, bio->bi_iter.bi_sector)) {
 		req_set_nomerge(req->q, req);
@@ -661,6 +665,9 @@ static int ll_merge_requests_fn(struct request_queue *q, struct request *req,
 	if (blk_integrity_merge_rq(q, req, next) == false)
 		return 0;
 
+	if (!bio_crypt_ctx_merge_rq(req, next))
+		return 0;
+
 	/* Merge is OK... */
 	req->nr_phys_segments = total_phys_segments;
 	return 1;
@@ -885,6 +892,10 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)
 	if (blk_integrity_merge_bio(rq->q, rq, bio) == false)
 		return false;
 
+	/* Only merge if the crypt contexts are compatible */
+	if (!bio_crypt_rq_ctx_compatible(rq, bio))
+		return false;
+
 	/* must be using the same buffer */
 	if (req_op(rq) == REQ_OP_WRITE_SAME &&
 	    !blk_write_same_mergeable(rq->bio, bio))
block/blk-mq.c +15 -0
index d92088dec6c3..b98dd06daf8f 100644
@@ -26,6 +26,7 @@
 #include <linux/delay.h>
 #include <linux/crash_dump.h>
 #include <linux/prefetch.h>
+#include <linux/blk-crypto.h>
 
 #include <trace/events/block.h>
 
@@ -317,6 +318,7 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,
 #if defined(CONFIG_BLK_DEV_INTEGRITY)
 	rq->nr_integrity_segments = 0;
 #endif
+	blk_crypto_rq_set_defaults(rq);
 	/* tag was already set */
 	rq->extra_len = 0;
 	WRITE_ONCE(rq->deadline, 0);
@@ -474,6 +476,8 @@ static void __blk_mq_free_request(struct request *rq)
 	struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
 	const int sched_tag = rq->internal_tag;
 
+	if (blk_crypto_rq_is_encrypted(rq))
+		blk_crypto_free_request(rq);
 	blk_pm_mark_last_busy(rq);
 	rq->mq_hctx = NULL;
 	if (rq->tag != -1)
@@ -1971,6 +1975,7 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 	struct request *same_queue_rq = NULL;
 	unsigned int nr_segs;
 	blk_qc_t cookie;
+	blk_status_t err;
 
 	blk_queue_bounce(q, &bio);
 	__blk_queue_split(q, &bio, &nr_segs);
@@ -2002,6 +2007,16 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 
 	cookie = request_to_qc_t(data.hctx, rq);
 
+	if (bio_has_crypt_ctx(bio)) {
+		err = blk_crypto_init_request(rq, bio);
+		if (err != BLK_STS_OK) {
+			bio->bi_status = err;
+			bio_endio(bio);
+			blk_mq_end_request(rq, err);
+			return BLK_QC_T_NONE;
+		}
+	}
+
 	blk_mq_bio_to_request(rq, bio, nr_segs);
 
 	plug = blk_mq_plug(q, bio);
block/blk.h +5 -0
index 0b8884353f6b..35e675ac845d 100644
@@ -4,7 +4,9 @@
 
 #include <linux/idr.h>
 #include <linux/blk-mq.h>
+#include <linux/blk-crypto.h>
 #include <xen/xen.h>
+#include "blk-crypto-internal.h"
 #include "blk-mq.h"
 #include "blk-mq-sched.h"
 
@@ -117,6 +119,9 @@ static inline void blk_rq_bio_prep(struct request *rq, struct bio *bio,
 
 	if (bio->bi_disk)
 		rq->rq_disk = bio->bi_disk;
+
+	if (bio_has_crypt_ctx(bio))
+		blk_crypto_rq_bio_prep(rq, bio);
 }
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
block/bounce.c +3 -0
index f8ed677a1bf7..c345132e91dd 100644
@@ -267,6 +267,9 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,
 		break;
 	}
 
+	if (bio_has_crypt_ctx(bio_src))
+		bio_crypt_clone(bio, bio_src, gfp_mask);
+
 	if (bio_integrity(bio_src)) {
 		int ret;
 
drivers/md/dm.c +4 -0
index b89f07ee2eff..d3f8fab33ee2 100644
@@ -25,6 +25,7 @@
 #include <linux/wait.h>
 #include <linux/pr.h>
 #include <linux/refcount.h>
+#include <linux/blk-crypto.h>
 
 #define DM_MSG_PREFIX "core"
 
@@ -1304,6 +1305,9 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,
 
 	__bio_clone_fast(clone, bio);
 
+	if (bio_has_crypt_ctx(bio))
+		bio_crypt_clone(clone, bio, GFP_NOIO);
+
 	if (bio_integrity(bio)) {
 		int r;
 
include/linux/blk-crypto.h +71 -1

88 lines not on this page · raw message

include/linux/blk_types.h +6 -0
index 70254ae11769..1996689c51d3 100644
@@ -18,6 +18,7 @@ struct block_device;
 struct io_context;
 struct cgroup_subsys_state;
 typedef void (bio_end_io_t) (struct bio *);
+struct bio_crypt_ctx;
 
 /*
  * Block error status values.  See block/blk-core:blk_errors for the details.
@@ -173,6 +174,11 @@ struct bio {
 	u64			bi_iocost_cost;
 #endif
 #endif
+
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	struct bio_crypt_ctx	*bi_crypt_context;
+#endif
+
 	union {
 #if defined(CONFIG_BLK_DEV_INTEGRITY)
 		struct bio_integrity_payload *bi_integrity; /* data integrity */
include/linux/blkdev.h +5 -0
index c447361952d1..c6ea578c1f79 100644
@@ -224,6 +224,11 @@ struct request {
 	unsigned short nr_integrity_segments;
 #endif
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	struct bio_crypt_ctx *crypt_ctx;
+	struct blk_ksm_keyslot *crypt_keyslot;
+#endif
+
 	unsigned short write_hint;
 	unsigned short ioprio;
 
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
#4 lore raw

Re: [PATCH v8 02/11] block: Inline encryption support for blk-mq

block/bio.c struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs) · line 672

Quoted 2 lines · Satya Tangirala
block/bio.c +2 -0
+	if (bio_has_crypt_ctx(bio))
+		bio_crypt_clone(b, bio, gfp_mask);

FYI, what I had tried to suggest when moving the the bio_has_crypt_ctx checks out was not to open code them, but to have inline functions.

E.g. your current bio_crypt_clone becomes __bio_crypt_clone,

and then a wrapper is added ala:

static inline void bio_crypt_clone(struct bio *dst, struct bio *src,
gfp_t gfp_mask)

{
	if (bio_has_crypt_ctx(bio))
		__bio_crypt_clone(dst, src, gfp_mask);
}

Which also means in all the headers you can now declare everything unconditional as long as bio_has_crypt_ctx is stubbed out for the case where blk crypto is disabled.

block/bio.c struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs) · line 680

Quoted 8 lines
block/bio.c +0 -1
 	if (bio_integrity(bio)) {
 		int ret;
 
 		ret = bio_integrity_clone(b, bio, gfp_mask);
-
 		if (ret < 0) {
 			bio_put(b);
 			return NULL;

Spurious whitespace change.

block/blk-core.c int __init blk_dev_init(void) · line 1834

Quoted 7 lines

free_and_out:

block/blk-core.c +2 -0
@@ -1813,5 +1830,7 @@ int __init blk_dev_init(void)
 	blk_debugfs_root = debugfs_create_dir("block", NULL);
 #endif
 
+	bio_crypt_ctx_init();
+

Is there any good reason to explicitly call bio_crypt_ctx_init vs just making it a local subsys_initcall?

block/blk-crypto.c line 135

Quoted 11 lines
block/blk-crypto.c new file +11 -0
+bool bio_crypt_dun_is_contiguous(const struct bio_crypt_ctx *bc,
+				 unsigned int bytes,
+				 u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE])
+{
+	int i = 0;
+	unsigned int inc = bytes >> bc->bc_key->data_unit_size_bits;
+
+	while (i < BLK_CRYPTO_DUN_ARRAY_SIZE) {
+		if (bc->bc_dun[i] + inc != next_dun[i])
+			return false;
+		inc = ((bc->bc_dun[i] + inc)  < inc);

Besides the bracing and double whitespace issue this code looks weird to me.

So inc starts out as the number of bytes shifted to the dun size.

We then check if it matches the next dun for every entry in the array.

But then inc is turned into a bollean for the next iteration. At that point I'm a little lost, can you add comments or make the code more explicit?

block/blk-crypto.c line 219

Quoted 9 lines
block/blk-crypto.c new file +9 -0
+	blk_crypto_rq_set_defaults(rq);
+
+	err = blk_ksm_get_slot_for_key(rq->q->ksm,
+				       bio->bi_crypt_context->bc_key,
+				       &rq->crypt_keyslot);
+	if (err != BLK_STS_OK)
+		pr_warn_once("Failed to acquire keyslot for %s (err=%d).\n",
+			     bio->bi_disk->disk_name, err);
+	return err;

Is this error really that important? If someone prints an error here I'd expect the low-level driver to do that, as that is the only place knowing what kind of error we could have here.

block/blk-crypto.c line 264

Quoted 12 lines
block/blk-crypto.c new file +12 -0
+int blk_crypto_bio_prep(struct bio **bio_ptr)
+{
+	struct bio *bio = *bio_ptr;
+
+	/*
+	 * If bio has no data, just pretend it didn't have an encryption
+	 * context.
+	 */
+	if (!bio_has_data(bio)) {
+		bio_crypt_free_ctx(bio);
+		return 0;
+	}

Shouldn't a submitted bio without data but with a crypt context be a hard error?

block/blk-crypto.c line 268

Quoted 3 lines
block/blk-crypto.c new file +3 -0
+	bio_crypt_check_alignment(bio);
+	if (bio->bi_status != BLK_STS_OK)
+		goto fail;

Weird calling convention. Why doesn't bio_crypt_check_alignment return a bool, and then this becomes the much more obvious:

	if (!bio_crypt_check_alignment(bio)) {
		bio->bi_status = BLK_STS_IOERR;
		goto fail;
	}
Quoted 4 lines
block/blk-crypto.c new file +4 -0
+	/*
+	 * Success if device supports the encryption context, and blk-integrity
+	 * isn't supported by device/is turned off.
+	 */

block/blk-crypto.c line 283

Quoted 10 lines
block/blk-crypto.c new file +10 -0
+	if (!blk_ksm_crypto_key_supported(bio->bi_disk->queue->ksm,
+					  bio->bi_crypt_context->bc_key)) {
+		bio->bi_status = BLK_STS_NOTSUPP;
+		goto fail;
+	}
+
+	return 0;
+fail:
+	bio_endio(*bio_ptr);
+	return -EIO;

Weird calling convention again. If the actual error is in the bio, this should just be a bool.

block/blk-crypto.c line 305

Quoted 4 lines
block/blk-crypto.c new file +4 -0
+void blk_crypto_rq_prep_clone(struct request *dst, struct request *src)
+{
+	dst->crypt_ctx = src->crypt_ctx;
+}

This seems reasonable to inline in the header..

block/blk-mq.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2012

Quoted 11 lines
block/blk-mq.c +4 -0
+	blk_status_t err;
 
 	blk_queue_bounce(q, &bio);
 	__blk_queue_split(q, &bio, &nr_segs);
@@ -2002,6 +2007,16 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 
 	cookie = request_to_qc_t(data.hctx, rq);
 
+	if (bio_has_crypt_ctx(bio)) {
+		err = blk_crypto_init_request(rq, bio);
+		if (err != BLK_STS_OK) {

The err declaration can go into this scope. I'd also rather call it ret as err is usually used for errno codes.

Also blk_crypto_init_request doesn't really need the bio, but just the key. So I'd rather pass the key to avoid confusion what this function might do with the bio.

block/blk-mq.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2016

Quoted 4 lines
block/blk-mq.c +4 -0
+			bio->bi_status = err;
+			bio_endio(bio);
+			blk_mq_end_request(rq, err);
+			return BLK_QC_T_NONE;

Shoudn't the blk_mq_end_request just be a blk_mq_free_request here?

include/linux/blk-crypto.h struct blk_crypto_key { · line 76

Quoted 9 lines
include/linux/blk-crypto.h +9 -0
+#ifdef CONFIG_BLOCK
+
+#include <linux/blk_types.h>
+#include <linux/blkdev.h>
+
+struct request;
+struct request_queue;
+
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION

Does the #ifdef CONFIG_BLOCK buy us anything?

#5 lore raw

PATCH v8 3/11 block: Make blk-integrity preclude hardware inline encryption

In reply to Satya Tangirala

Whenever a device supports blk-integrity, the kernel will now always pretend that the device doesn't support inline encryption (essentially by setting the keyslot manager in the request queue to NULL).

There's no hardware currently that supports both integrity and inline encryption. However, it seems possible that there will be in the near future, based on discussion at
https://lore.kernel.org/r/[email protected]/
But properly integrating both features is not trivial, and without real hardware that implements both, it is difficult to tell if it will be done correctly by the majority of hardware that support both, and through discussions at
https://lore.kernel.org/r/[email protected]/
it seems best not to support both features together right now, and to decide what to do at probe time.

---
 block/bio-integrity.c   |  5 +++++
 block/blk-integrity.c   |  7 +++++++
 block/keyslot-manager.c | 20 ++++++++++++++++++++
 include/linux/blkdev.h  | 30 ++++++++++++++++++++++++++++++
 4 files changed, 62 insertions(+)
block/bio-integrity.c +5 -0
index bf62c25cde8f..a5c57991c6fa 100644
@@ -42,6 +42,11 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,
 	struct bio_set *bs = bio->bi_pool;
 	unsigned inline_vecs;
 
+	if (bio_has_crypt_ctx(bio)) {
+		pr_warn("blk-integrity can't be used together with inline en/decryption.");
+		return ERR_PTR(-EOPNOTSUPP);
+	}
+
 	if (!bs || !mempool_initialized(&bs->bio_integrity_pool)) {
 		bip = kmalloc(struct_size(bip, bip_inline_vecs, nr_vecs), gfp_mask);
 		inline_vecs = nr_vecs;
block/blk-integrity.c +7 -0
index ff1070edbb40..793ba23e8688 100644
@@ -409,6 +409,13 @@ void blk_integrity_register(struct gendisk *disk, struct blk_integrity *template
 	bi->tag_size = template->tag_size;
 
 	disk->queue->backing_dev_info->capabilities |= BDI_CAP_STABLE_WRITES;
+
+#ifdef BLK_INLINE_ENCRYPTION
+	if (disk->queue->ksm) {
+		pr_warn("blk-integrity: Integrity and hardware inline encryption are not supported together. Unregistering keyslot manager from request queue, to disable hardware inline encryption.");
+		blk_ksm_unregister(disk->queue);
+	}
+#endif
 }
 EXPORT_SYMBOL(blk_integrity_register);
 
block/keyslot-manager.c +20 -0
index 38df0652df80..a7970e18a122 100644
@@ -25,6 +25,9 @@
  * Upper layers will call blk_ksm_get_slot_for_key() to program a
  * key into some slot in the inline encryption hardware.
  */
+
+#define pr_fmt(fmt) "blk_ksm: " fmt
+
 #include <crypto/algapi.h>
 #include <linux/keyslot-manager.h>
 #include <linux/atomic.h>
@@ -375,3 +378,20 @@ void blk_ksm_destroy(struct keyslot_manager *ksm)
 	memzero_explicit(ksm, sizeof(*ksm));
 }
 EXPORT_SYMBOL_GPL(blk_ksm_destroy);
+
+bool blk_ksm_register(struct keyslot_manager *ksm, struct request_queue *q)
+{
+	if (blk_integrity_queue_supports_integrity(q)) {
+		pr_warn("Integrity and hardware inline encryption are not supported together. Won't register keyslot manager with request queue.");
+		return false;
+	}
+	q->ksm = ksm;
+	return true;
+}
+EXPORT_SYMBOL_GPL(blk_ksm_register);
+
+void blk_ksm_unregister(struct request_queue *q)
+{
+	q->ksm = NULL;
+}
+EXPORT_SYMBOL_GPL(blk_ksm_unregister);
include/linux/blkdev.h +30 -0
index c6ea578c1f79..abe886d48cc4 100644
@@ -1570,6 +1570,12 @@ struct blk_integrity *bdev_get_integrity(struct block_device *bdev)
 	return blk_get_integrity(bdev->bd_disk);
 }
 
+static inline bool
+blk_integrity_queue_supports_integrity(struct request_queue *q)
+{
+	return q->integrity.profile;
+}
+
 static inline bool blk_integrity_rq(struct request *rq)
 {
 	return rq->cmd_flags & REQ_INTEGRITY;
@@ -1650,6 +1656,11 @@ static inline struct blk_integrity *blk_get_integrity(struct gendisk *disk)
 {
 	return NULL;
 }
+static inline bool
+blk_integrity_queue_supports_integrity(struct request_queue *q)
+{
+	return false;
+}
 static inline int blk_integrity_compare(struct gendisk *a, struct gendisk *b)
 {
 	return 0;
@@ -1701,6 +1712,25 @@ static inline struct bio_vec *rq_integrity_vec(struct request *rq)
 
 #endif /* CONFIG_BLK_DEV_INTEGRITY */
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+
+bool blk_ksm_register(struct keyslot_manager *ksm, struct request_queue *q);
+
+void blk_ksm_unregister(struct request_queue *q);
+
+#else /* CONFIG_BLK_INLINE_ENCRYPTION */
+
+static inline bool blk_ksm_register(struct keyslot_manager *ksm,
+				    struct request_queue *q)
+{
+	return true;
+}
+
+static inline void blk_ksm_unregister(struct request_queue *q) { }
+
+#endif /* CONFIG_BLK_INLINE_ENCRYPTION */
+
+
 struct block_device_operations {
 	int (*open) (struct block_device *, fmode_t);
 	void (*release) (struct gendisk *, fmode_t);
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#6 lore raw

Re: [PATCH v8 03/11] block: Make blk-integrity preclude hardware inline encryption

Quoted 7 lines · Satya Tangirala
block/blk-integrity.c +0 -0
index ff1070edbb40..793ba23e8688 100644
@@ -409,6 +409,13 @@ void blk_integrity_register(struct gendisk *disk, struct blk_integrity *template
 	bi->tag_size = template->tag_size;
 

block/blk-integrity.c void blk_integrity_register(struct gendisk *disk, struct blk_integrity *template · line 420

Quoted 10 lines
block/blk-integrity.c +7 -0
 	disk->queue->backing_dev_info->capabilities |= BDI_CAP_STABLE_WRITES;
+
+#ifdef BLK_INLINE_ENCRYPTION
+	if (disk->queue->ksm) {
+		pr_warn("blk-integrity: Integrity and hardware inline encryption are not supported together. Unregistering keyslot manager from request queue, to disable hardware inline encryption.");
+		blk_ksm_unregister(disk->queue);
+	}
+#endif
 }
 EXPORT_SYMBOL(blk_integrity_register);

This ifdef is wrong, it should be CONFIG_BLK_INLINE_ENCRYPTION.

Also the log message is missing a trailing newline.

block/keyslot-manager.c line 29

Quoted 11 lines
block/keyslot-manager.c +2 -0
index 38df0652df80..a7970e18a122 100644
@@ -25,6 +25,9 @@
  * Upper layers will call blk_ksm_get_slot_for_key() to program a
  * key into some slot in the inline encryption hardware.
  */
+
+#define pr_fmt(fmt) "blk_ksm: " fmt

People aren't going to know what "blk_ksm" means in the logs. I think just use "blk-crypto" instead.

Quoted 9 lines
block/keyslot-manager.c +2 -0
+
 #include <crypto/algapi.h>
 #include <linux/keyslot-manager.h>
 #include <linux/atomic.h>
@@ -375,3 +378,20 @@ void blk_ksm_destroy(struct keyslot_manager *ksm)
 	memzero_explicit(ksm, sizeof(*ksm));
 }
 EXPORT_SYMBOL_GPL(blk_ksm_destroy);
+

block/keyslot-manager.c void blk_ksm_destroy(struct keyslot_manager *ksm) · line 391

Quoted 10 lines
block/keyslot-manager.c +10 -0
+bool blk_ksm_register(struct keyslot_manager *ksm, struct request_queue *q)
+{
+	if (blk_integrity_queue_supports_integrity(q)) {
+		pr_warn("Integrity and hardware inline encryption are not supported together. Won't register keyslot manager with request queue.");
+		return false;
+	}
+	q->ksm = ksm;
+	return true;
+}
+EXPORT_SYMBOL_GPL(blk_ksm_register);

People reading the logs won't know what a keyslot manager is and why they should care that one wasn't registered. It would be better to say that hardware inline encryption is being disabled.

Ideally the device name would be included in the message too.

block/keyslot-manager.c void blk_ksm_destroy(struct keyslot_manager *ksm) · line 397

Quoted 6 lines
block/keyslot-manager.c +6 -0
+
+void blk_ksm_unregister(struct request_queue *q)
+{
+	q->ksm = NULL;
+}
+EXPORT_SYMBOL_GPL(blk_ksm_unregister);

blk_ksm_unregister() doesn't need to be exported.

#7 lore raw

Re: [PATCH v8 03/11] block: Make blk-integrity preclude hardware inline encryption

In reply to Satya Tangirala
Quoted 11 lines · Satya Tangirala

There's no hardware currently that supports both integrity and inline encryption. However, it seems possible that there will be in the near future, based on discussion at
https://lore.kernel.org/r/[email protected]/
But properly integrating both features is not trivial, and without real hardware that implements both, it is difficult to tell if it will be done correctly by the majority of hardware that support both, and through discussions at
https://lore.kernel.org/r/[email protected]/
it seems best not to support both features together right now, and to decide what to do at probe time.

Please don't reference web links, just inline the important information.

block/bio-integrity.c struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, · line 48

Quoted 12 lines
block/bio-integrity.c +4 -0
index bf62c25cde8f..a5c57991c6fa 100644
@@ -42,6 +42,11 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,
 	struct bio_set *bs = bio->bi_pool;
 	unsigned inline_vecs;
 
+	if (bio_has_crypt_ctx(bio)) {
+		pr_warn("blk-integrity can't be used together with inline en/decryption.");
+		return ERR_PTR(-EOPNOTSUPP);
+	}

This is a hard error and should just be a WARN_ON_ONCE.

I'm also not sure we need the register time warnings at all.

#8 lore raw

PATCH v8 4/11 block: blk-crypto-fallback for Inline Encryption

In reply to Satya Tangirala

Blk-crypto delegates crypto operations to inline encryption hardware when available. The separately configurable blk-crypto-fallback contains a software fallback to the kernel crypto API - when enabled, blk-crypto will use this fallback for en/decryption when inline encryption hardware is not available. This lets upper layers not have to worry about whether or not the underlying device has support for inline encryption before deciding to specify an encryption context for a bio, and also allows for testing without actual inline encryption hardware. For more details, refer to Documentation/block/inline-encryption.rst.

---
 Documentation/block/index.rst             |   1 +
 Documentation/block/inline-encryption.rst | 195 +++++++
 block/Kconfig                             |  10 +
 block/Makefile                            |   1 +
 block/blk-crypto-fallback.c               | 662 ++++++++++++++++++++++
 block/blk-crypto-internal.h               |  24 +
 block/blk-crypto.c                        |  47 +-
 include/linux/blk-crypto.h                |  17 +-
 8 files changed, 943 insertions(+), 14 deletions(-)
 create mode 100644 Documentation/block/inline-encryption.rst
 create mode 100644 block/blk-crypto-fallback.c
Documentation/block/index.rst +1 -0
index 3fa7a52fafa4..026addfc69bc 100644
@@ -14,6 +14,7 @@ Block
    cmdline-partition
    data-integrity
    deadline-iosched
+   inline-encryption
    ioprio
    kyber-iosched
    null_blk
Documentation/block/inline-encryption.rst new file +195 -0

195 lines not on this page · raw message

block/Kconfig +10 -0
index c04a1d500842..0af387623774 100644
@@ -192,6 +192,16 @@ config BLK_INLINE_ENCRYPTION
 	  block layer handle encryption, so users can take
 	  advantage of inline encryption hardware if present.
 
+config BLK_INLINE_ENCRYPTION_FALLBACK
+	bool "Enable crypto API fallback for blk-crypto"
+	depends on BLK_INLINE_ENCRYPTION
+	select CRYPTO
+	select CRYPTO_SKCIPHER
+	help
+	  Enabling this lets the block layer handle inline encryption
+	  by falling back to the kernel crypto API when inline
+	  encryption hardware is not present.
+
 menu "Partition Types"
 
 source "block/partitions/Kconfig"
block/Makefile +1 -0
index 82f42ca3f769..9464fb6ae423 100644
@@ -38,3 +38,4 @@ obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
 obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o blk-crypto.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK)	+= blk-crypto-fallback.o
block/blk-crypto-fallback.c new file +662 -0

662 lines not on this page · raw message

block/blk-crypto-internal.h +24 -0
index 5cdf45167117..a8b7c9c4b8da 100644
@@ -16,6 +16,8 @@ struct blk_crypto_mode {
 	unsigned int ivsize; /* iv size in bytes */
 };
 
+extern const struct blk_crypto_mode blk_crypto_modes[];
+
 #ifdef CONFIG_BLK_INLINE_ENCRYPTION
 
 void bio_crypt_ctx_init(void);
@@ -141,4 +143,26 @@ static inline blk_status_t blk_crypto_insert_cloned_request(struct request *rq)
 
 #endif /* CONFIG_BLK_INLINE_ENCRYPTION */
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK
+
+void blk_crypto_fallback_bio_prep(struct bio **bio_ptr);
+
+int blk_crypto_fallback_evict_key(const struct blk_crypto_key *key);
+
+#else /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
+static inline void blk_crypto_fallback_bio_prep(struct bio **bio_ptr)
+{
+	pr_warn_once("crypto API fallback disabled; failing request.\n");
+	(*bio_ptr)->bi_status = BLK_STS_NOTSUPP;
+}
+
+static inline int
+blk_crypto_fallback_evict_key(const struct blk_crypto_key *key)
+{
+	return 0;
+}
+
+#endif /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
 #endif /* __LINUX_BLK_CRYPTO_INTERNAL_H */
block/blk-crypto.c +34 -13

95 lines not on this page · raw message

include/linux/blk-crypto.h +16 -1
index 90a28df26106..9c54bc34044e 100644
@@ -58,7 +58,7 @@ struct blk_crypto_key {
  *
  * A bio_crypt_ctx specifies that the contents of the bio will be encrypted (for
  * write requests) or decrypted (for read requests) inline by the storage device
- * or controller.
+ * or controller, or by the crypto API fallback.
  */
 struct bio_crypt_ctx {
 	const struct blk_crypto_key	*bc_key;
@@ -110,6 +110,21 @@ static inline void bio_crypt_clone(struct bio *dst, struct bio *src,
 
 #endif /* CONFIG_BLK_INLINE_ENCRYPTION */
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK
+
+int blk_crypto_start_using_key(struct blk_crypto_key *key,
+			       struct request_queue *q);
+
+#else /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
+static inline int blk_crypto_start_using_key(struct blk_crypto_key *key,
+					     struct request_queue *q)
+{
+	return 0;
+}
+
+#endif /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
 #endif /* CONFIG_BLOCK */
 
 #endif /* __LINUX_BLK_CRYPTO_H */
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
#9 lore raw

PATCH v8 5/11 scsi: ufs: UFS driver v2.1 spec crypto additions

In reply to Satya Tangirala

Add the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI specification in preparation to add support for inline encryption to UFS.

---
 drivers/scsi/ufs/ufshcd.c |  2 ++
 drivers/scsi/ufs/ufshcd.h |  5 +++
 drivers/scsi/ufs/ufshci.h | 67 +++++++++++++++++++++++++++++++++++++--
 3 files changed, 72 insertions(+), 2 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
index abd0e6b05f79..825d9eb34f10 100644
@@ -4759,6 +4759,8 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp)
 	case OCS_MISMATCH_RESP_UPIU_SIZE:
 	case OCS_PEER_COMM_FAILURE:
 	case OCS_FATAL_ERROR:
+	case OCS_INVALID_CRYPTO_CONFIG:
+	case OCS_GENERAL_CRYPTO_ERROR:
 	default:
 		result |= DID_ERROR << 16;
 		dev_err(hba->dev,
drivers/scsi/ufs/ufshcd.h +5 -0
index 2ae6c7c8528c..978781c538c4 100644
@@ -716,6 +716,11 @@ struct ufs_hba {
 	 * for userspace to control the power management.
 	 */
 #define UFSHCD_CAP_RPM_AUTOSUSPEND (1 << 6)
+	/*
+	 * This capability allows the host controller driver to use the
+	 * inline crypto engine, if it is present
+	 */
+#define UFSHCD_CAP_CRYPTO (1 << 7)
 
 	struct devfreq *devfreq;
 	struct ufs_clk_scaling clk_scaling;
drivers/scsi/ufs/ufshci.h +65 -2

106 lines not on this page · raw message

Signature · 1 line
2.25.1.481.gfbce0eb801-goog
#10 lore raw

PATCH v8 6/11 scsi: ufs: UFS crypto API

In reply to Satya Tangirala

Introduce functions to manipulate UFS inline encryption hardware in line with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot manager.

The UFS crypto API will assume by default that a vendor driver doesn't support UFS crypto, even if the hardware advertises the capability, because a lot of hardware requires some special handling that's not specified in the aforementioned JEDEC spec. Each vendor driver must explicity set hba->caps |= UFSHCD_CAP_CRYPTO before ufshcd_hba_init_crypto is called to opt-in to UFS crypto support.

---
 drivers/scsi/ufs/Kconfig         |   9 +
 drivers/scsi/ufs/Makefile        |   1 +
 drivers/scsi/ufs/ufshcd-crypto.c | 370 +++++++++++++++++++++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h |  54 +++++
 drivers/scsi/ufs/ufshcd.h        |  14 ++
 5 files changed, 448 insertions(+)
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
drivers/scsi/ufs/Kconfig +9 -0
index d14c2243e02a..c69f1b49167b 100644
@@ -160,3 +160,12 @@ config SCSI_UFS_BSG
 
 	  Select this if you need a bsg device node for your UFS controller.
 	  If unsure, say N.
+
+config SCSI_UFS_CRYPTO
+	bool "UFS Crypto Engine Support"
+	depends on SCSI_UFSHCD && BLK_INLINE_ENCRYPTION
+	help
+	  Enable Crypto Engine Support in UFS.
+	  Enabling this makes it possible for the kernel to use the crypto
+	  capabilities of the UFS device (if present) to perform crypto
+	  operations on data being transferred to/from the device.
drivers/scsi/ufs/Makefile +1 -0
index 94c6c5d7334b..197e178f44bc 100644
@@ -7,6 +7,7 @@ obj-$(CONFIG_SCSI_UFS_QCOM) += ufs-qcom.o
 obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o
 ufshcd-core-y				+= ufshcd.o ufs-sysfs.o
 ufshcd-core-$(CONFIG_SCSI_UFS_BSG)	+= ufs_bsg.o
+ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o
 obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o
 obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o
 obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd-crypto.c new file +370 -0

370 lines not on this page · raw message

drivers/scsi/ufs/ufshcd-crypto.h new file +54 -0

54 lines not on this page · raw message

drivers/scsi/ufs/ufshcd.h +14 -0
index 978781c538c4..ee37dd44d5e6 100644
@@ -55,6 +55,7 @@
 #include <linux/clk.h>
 #include <linux/completion.h>
 #include <linux/regulator/consumer.h>
+#include <linux/keyslot-manager.h>
 #include "unipro.h"
 
 #include <asm/irq.h>
@@ -521,6 +522,10 @@ struct ufs_stats {
  * @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for
  *  device is known or not.
  * @scsi_block_reqs_cnt: reference counting for scsi block requests
+ * @crypto_capabilities: Content of crypto capabilities register (0x100)
+ * @crypto_cap_array: Array of crypto capabilities
+ * @crypto_cfg_register: Start of the crypto cfg array
+ * @ksm: the keyslot manager tied to this hba
  */
 struct ufs_hba {
 	void __iomem *mmio_base;
@@ -634,6 +639,7 @@ struct ufs_hba {
 	 * enabled via HCE register.
 	 */
 	#define UFSHCI_QUIRK_BROKEN_HCE				0x400
+
 	unsigned int quirks;	/* Deviations from standard UFSHCI spec. */
 
 	/* Device deviations from standard UFS device spec. */
@@ -735,6 +741,14 @@ struct ufs_hba {
 
 	struct device		bsg_dev;
 	struct request_queue	*bsg_queue;
+
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+	/* crypto */
+	union ufs_crypto_capabilities crypto_capabilities;
+	union ufs_crypto_cap_entry *crypto_cap_array;
+	u32 crypto_cfg_register;
+	struct keyslot_manager ksm;
+#endif /* CONFIG_SCSI_UFS_CRYPTO */
 };
 
 /* Returns true if clocks can be gated. Otherwise false */
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#11 lore raw

Re: [PATCH v8 06/11] scsi: ufs: UFS crypto API

Quoted 57 lines · Satya Tangirala

Introduce functions to manipulate UFS inline encryption hardware in line with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot manager.

The UFS crypto API will assume by default that a vendor driver doesn't support UFS crypto, even if the hardware advertises the capability, because a lot of hardware requires some special handling that's not specified in the aforementioned JEDEC spec. Each vendor driver must explicity set hba->caps |= UFSHCD_CAP_CRYPTO before ufshcd_hba_init_crypto is called to opt-in to UFS crypto support.

---
 drivers/scsi/ufs/Kconfig         |   9 +
 drivers/scsi/ufs/Makefile        |   1 +
 drivers/scsi/ufs/ufshcd-crypto.c | 370 +++++++++++++++++++++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h |  54 +++++
 drivers/scsi/ufs/ufshcd.h        |  14 ++
 5 files changed, 448 insertions(+)
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
drivers/scsi/ufs/Kconfig +9 -0
index d14c2243e02a..c69f1b49167b 100644
@@ -160,3 +160,12 @@ config SCSI_UFS_BSG
 
 	  Select this if you need a bsg device node for your UFS controller.
 	  If unsure, say N.
+
+config SCSI_UFS_CRYPTO
+	bool "UFS Crypto Engine Support"
+	depends on SCSI_UFSHCD && BLK_INLINE_ENCRYPTION
+	help
+	  Enable Crypto Engine Support in UFS.
+	  Enabling this makes it possible for the kernel to use the crypto
+	  capabilities of the UFS device (if present) to perform crypto
+	  operations on data being transferred to/from the device.
drivers/scsi/ufs/Makefile +1 -0
index 94c6c5d7334b..197e178f44bc 100644
@@ -7,6 +7,7 @@ obj-$(CONFIG_SCSI_UFS_QCOM) += ufs-qcom.o
 obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o
 ufshcd-core-y				+= ufshcd.o ufs-sysfs.o
 ufshcd-core-$(CONFIG_SCSI_UFS_BSG)	+= ufs_bsg.o
+ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o
 obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o
 obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o
 obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd-crypto.c new file +0 -0
new file mode 100644
index 000000000000..8b6f7c83f77f

drivers/scsi/ufs/ufshcd-crypto.c line 8

Quoted 8 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +8 -0
@@ -0,0 +1,370 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2019 Google LLC
+ */
+
+#include <linux/keyslot-manager.h>
+#include "ufshcd.h"
+#include "ufshcd-crypto.h"

Since keyslot-manager.h now has to be included in ufshcd.h anyway since now there's a struct keyslot_manager embedded in struct ufs_hba, there's no need to include keyslot-manager.h here and in ufshcd-crypto.h too.

Quoted 9 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +9 -0
+
+static inline int ufshcd_num_keyslots(struct ufs_hba *hba)
+{
+	return hba->crypto_capabilities.config_count + 1;
+}
+
+static inline bool ufshcd_keyslot_valid(struct ufs_hba *hba, unsigned int slot)
+{
+	/*

drivers/scsi/ufs/ufshcd-crypto.c line 27

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	 * The actual number of configurations supported is (CFGC+1), so slot
+	 * numbers range from 0 to config_count inclusive.
+	 */
+	return slot < ufshcd_num_keyslots(hba);
+}
+
+static bool ufshcd_cap_idx_valid(struct ufs_hba *hba, unsigned int cap_idx)
+{
+	return cap_idx < hba->crypto_capabilities.num_crypto_cap;
+}

ufshcd_cap_idx_valid() isn't needed since it's only used to check the return value of ufshcd_crypto_cap_find().

drivers/scsi/ufs/ufshcd-crypto.c line 36

Quoted 9 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +9 -0
+
+static u8 ufshcd_get_data_unit_size_mask(unsigned int data_unit_size)
+{
+	if (data_unit_size < 512 || data_unit_size > 65536 ||
+	    !is_power_of_2(data_unit_size))
+		return 0;
+
+	return data_unit_size / 512;
+}

ufshcd_get_data_unit_size_mask() isn't needed since it's only used via ufshcd_crypto_keyslot_program() which already knows it has a data_unit_size the driver supports. So just do 'ufs_data_unit_size = key->data_unit_size / 512'

Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6 -0
+
+static size_t ufshcd_get_keysize_bytes(enum ufs_crypto_key_size size)
+{
+	switch (size) {
+	case UFS_CRYPTO_KEY_SIZE_128:
+		return 16;

drivers/scsi/ufs/ufshcd-crypto.c line 52

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	case UFS_CRYPTO_KEY_SIZE_192:
+		return 24;
+	case UFS_CRYPTO_KEY_SIZE_256:
+		return 32;
+	case UFS_CRYPTO_KEY_SIZE_512:
+		return 64;
+	default:
+		return 0;
+	}
+}

ufshcd_get_keysize_bytes() isn't needed since it's only used via ufshcd_crypto_keyslot_program() which already has access to key->size.

Quoted 5 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +5 -0
+
+/* Blk-crypto modes supported by UFS crypto */
+static const struct {
+	enum ufs_crypto_alg ufs_alg;
+	enum ufs_crypto_key_size ufs_key_size;

drivers/scsi/ufs/ufshcd-crypto.c line 67

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	enum blk_crypto_mode_num blk_mode;
+	bool supported;
+} ufs_crypto_algs[BLK_ENCRYPTION_MODE_MAX] = {
+	[BLK_ENCRYPTION_MODE_AES_256_XTS] = {
+		.ufs_alg = UFS_CRYPTO_ALG_AES_XTS,
+		.ufs_key_size = UFS_CRYPTO_KEY_SIZE_256,
+		.blk_mode = BLK_ENCRYPTION_MODE_AES_256_XTS,
+		.supported = true,
+	},
+};

The '.blk_mode' field is unnecessary since it's the same as the array index.

I was also confused by the '.supported' field and under what circumstances it would be set to false. It's actually just used to indicate that the array entry is used. That's not necessary since UFS_CRYPTO_KEY_SIZE_INVALID == 0. (To make the assumption clear, we can add BUILD_BUG_ON(UFS_CRYPTO_KEY_SIZE_INVALID != 0))

Quoted 21 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +21 -0
+static int ufshcd_crypto_cap_find(struct ufs_hba *hba,
+				  enum blk_crypto_mode_num crypto_mode,
+				  unsigned int data_unit_size)
+{
+	enum ufs_crypto_alg ufs_alg;
+	u8 data_unit_mask;
+	int cap_idx;
+	enum ufs_crypto_key_size ufs_key_size;
+	union ufs_crypto_cap_entry *ccap_array = hba->crypto_cap_array;
+
+	if (!ufshcd_hba_is_crypto_supported(hba))
+		return -EINVAL;
+
+	if (!ufs_crypto_algs[crypto_mode].supported)
+		return -EINVAL;
+
+	ufs_alg = ufs_crypto_algs[crypto_mode].ufs_alg;
+	ufs_key_size = ufs_crypto_algs[crypto_mode].ufs_key_size;
+
+	data_unit_mask = ufshcd_get_data_unit_size_mask(data_unit_size);
+

drivers/scsi/ufs/ufshcd-crypto.c line 99

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	for (cap_idx = 0; cap_idx < hba->crypto_capabilities.num_crypto_cap;
+	     cap_idx++) {
+		if (ccap_array[cap_idx].algorithm_id == ufs_alg &&
+		    (ccap_array[cap_idx].sdus_mask & data_unit_mask) &&
+		    ccap_array[cap_idx].key_size == ufs_key_size)
+			return cap_idx;
+	}
+
+	return -EINVAL;
+}

This is only called once. It would be simpler to just inline it into the caller. The ufshcd_hba_is_crypto_supported() and ufs_crypto_algs[crypto_mode].supported checks are also unnecessary.

Quoted 36 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +36 -0
+
+/**
+ * ufshcd_crypto_cfg_entry_write_key - Write a key into a crypto_cfg_entry
+ *
+ *	Writes the key with the appropriate format - for AES_XTS,
+ *	the first half of the key is copied as is, the second half is
+ *	copied with an offset halfway into the cfg->crypto_key array.
+ *	For the other supported crypto algs, the key is just copied.
+ *
+ * @cfg: The crypto config to write to
+ * @key: The key to write
+ * @cap: The crypto capability (which specifies the crypto alg and key size)
+ *
+ * Returns BLK_STS_OK on success, or BLK_STS_IOERR on error
+ */
+static blk_status_t ufshcd_crypto_cfg_entry_write_key(
+						union ufs_crypto_cfg_entry *cfg,
+						const u8 *key,
+						union ufs_crypto_cap_entry cap)
+{
+	size_t key_size_bytes = ufshcd_get_keysize_bytes(cap.key_size);
+
+	if (key_size_bytes == 0)
+		return BLK_STS_IOERR;
+
+	switch (cap.algorithm_id) {
+	case UFS_CRYPTO_ALG_AES_XTS:
+		key_size_bytes *= 2;
+		if (key_size_bytes > UFS_CRYPTO_KEY_MAX_SIZE)
+			return BLK_STS_IOERR;
+
+		memcpy(cfg->crypto_key, key, key_size_bytes/2);
+		memcpy(cfg->crypto_key + UFS_CRYPTO_KEY_MAX_SIZE/2,
+		       key + key_size_bytes/2, key_size_bytes/2);
+		return BLK_STS_OK;
+	case UFS_CRYPTO_ALG_BITLOCKER_AES_CBC:

drivers/scsi/ufs/ufshcd-crypto.c line 145

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+		/* fall through */
+	case UFS_CRYPTO_ALG_AES_ECB:
+		/* fall through */
+	case UFS_CRYPTO_ALG_ESSIV_AES_CBC:
+		memcpy(cfg->crypto_key, key, key_size_bytes);
+		return BLK_STS_OK;
+	}
+
+	return BLK_STS_IOERR;
+}

Likewise, this is only called once, and the error checks are unnecessary as this is only called from ufshcd_crypto_keyslot_program() where we know the algorithm is supported by the UFS driver. It would be much simpler to just inline this into the caller.

The switch statement can also be simplified to an 'if', since XTS is the only algorithm that requires special handling and it's likely to remain that way. The "two keys" thing is pretty unique to XTS.

Quoted 20 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +20 -0
+
+static void ufshcd_program_key(struct ufs_hba *hba,
+			       const union ufs_crypto_cfg_entry *cfg,
+			       int slot)
+{
+	int i;
+	u32 slot_offset = hba->crypto_cfg_register + slot * sizeof(*cfg);
+
+	ufshcd_hold(hba, false);
+	/* Ensure that CFGE is cleared before programming the key */
+	ufshcd_writel(hba, 0, slot_offset + 16 * sizeof(cfg->reg_val[0]));
+	for (i = 0; i < 16; i++) {
+		ufshcd_writel(hba, le32_to_cpu(cfg->reg_val[i]),
+			      slot_offset + i * sizeof(cfg->reg_val[0]));
+	}
+	/* Write dword 17 */
+	ufshcd_writel(hba, le32_to_cpu(cfg->reg_val[17]),
+		      slot_offset + 17 * sizeof(cfg->reg_val[0]));
+	/* Dword 16 must be written last */
+	ufshcd_writel(hba, le32_to_cpu(cfg->reg_val[16]),

drivers/scsi/ufs/ufshcd-crypto.c line 175

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+		      slot_offset + 16 * sizeof(cfg->reg_val[0]));
+	ufshcd_release(hba);
+}
+
+static void ufshcd_clear_keyslot(struct ufs_hba *hba, int slot)
+{
+	union ufs_crypto_cfg_entry cfg = { 0 };
+
+	ufshcd_program_key(hba, &cfg, slot);
+}

It may make sense to move ufshcd_clear_keyslot() to just above ufshcd_crypto_keyslot_evict(), so that the key eviction code is grouped together.

drivers/scsi/ufs/ufshcd-crypto.c line 184

Quoted 8 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +8 -0
+/* Clear all keyslots at driver init time */
+static void ufshcd_clear_all_keyslots(struct ufs_hba *hba)
+{
+	int slot;
+
+	for (slot = 0; slot < ufshcd_num_keyslots(hba); slot++)
+		ufshcd_clear_keyslot(hba, slot);
+}

This is only called once. It would be simpler to just inline it into ufshcd_hba_init_crypto().

Quoted 28 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +28 -0
+static blk_status_t ufshcd_crypto_keyslot_program(struct keyslot_manager *ksm,
+					const struct blk_crypto_key *key,
+					unsigned int slot)
+{
+	struct ufs_hba *hba = container_of(ksm, struct ufs_hba, ksm);
+	blk_status_t err = BLK_STS_OK;
+	u8 data_unit_mask;
+	union ufs_crypto_cfg_entry cfg;
+	int cap_idx;
+
+	cap_idx = ufshcd_crypto_cap_find(hba, key->crypto_mode,
+					 key->data_unit_size);
+
+	if (!(hba->caps & UFSHCD_CAP_CRYPTO) ||
+	    !ufshcd_keyslot_valid(hba, slot) ||
+	    !ufshcd_cap_idx_valid(hba, cap_idx))
+		return BLK_STS_IOERR;
+
+	data_unit_mask = ufshcd_get_data_unit_size_mask(key->data_unit_size);
+
+	if (!(data_unit_mask & hba->crypto_cap_array[cap_idx].sdus_mask))
+		return BLK_STS_IOERR;
+
+	memset(&cfg, 0, sizeof(cfg));
+	cfg.data_unit_size = data_unit_mask;
+	cfg.crypto_cap_idx = cap_idx;
+	cfg.config_enable |= UFS_CRYPTO_CONFIGURATION_ENABLE;
+

drivers/scsi/ufs/ufshcd-crypto.c line 223

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	err = ufshcd_crypto_cfg_entry_write_key(&cfg, key->raw,
+						hba->crypto_cap_array[cap_idx]);
+	if (err)
+		return err;
+
+	ufshcd_program_key(hba, &cfg, slot);
+
+	memzero_explicit(&cfg, sizeof(cfg));
+	return BLK_STS_OK;
+}

ufshcd_crypto_keyslot_program() is much more complicated than necessary. It calls several functions that are only called once, and could be simplified if inlined. It also does lots of checks that are unnecessary because the block layer already guarantees that it's called with a valid keyslot and a (crypto_mode, data unit_size) pair the driver supports.

I also don't see any benefit to the change from errno to blk_status_t here. It's inconsistent with keyslot_evict(), which is going to cause confusion. It also doesn't match what blk-crypto-fallback wants. It seems it would be better to just have blk_ksm_get_slot_for_key() use errno_to_blk_status() instead.

So how about something like the following:

static int ufshcd_crypto_keyslot_program(struct keyslot_manager *ksm,
const struct blk_crypto_key *key, unsigned int slot)

{
	struct ufs_hba *hba = container_of(ksm, struct ufs_hba, ksm);
	const union ufs_crypto_cap_entry *ccaps = hba->crypto_cap_array;
	const struct ufs_crypto_alg_entry *alg =
		&ufs_crypto_algs[key->crypto_mode];
	unsigned int ufs_data_unit_size = key->data_unit_size / 512;
	int cap_idx;
	union ufs_crypto_cfg_entry cfg = { 0 };

	/* Find the corresponding crypto capability */
	for (cap_idx = 0; cap_idx < hba->crypto_capabilities.num_crypto_cap;
	     cap_idx++) {
		if (ccaps[cap_idx].algorithm_id == alg->ufs_alg &&
		    ccaps[cap_idx].key_size == alg->ufs_key_size &&
		    (ccaps[cap_idx].sdus_mask & ufs_data_unit_size))
			break;
	}
	if (WARN_ON(cap_idx == hba->crypto_capabilities.num_crypto_cap))
		return -EOPNOTSUPP;

	if (alg->ufs_alg == UFS_CRYPTO_ALG_AES_XTS) {
		memcpy(cfg.crypto_key, key->raw, key->size / 2);
		memcpy(&cfg.crypto_key[UFS_CRYPTO_KEY_MAX_SIZE / 2],
		       &key->raw[key->size / 2], key->size / 2);
	} else {
		memcpy(cfg.crypto_key, key->raw, key->size);
	}
	cfg.data_unit_size = ufs_data_unit_size;
	cfg.crypto_cap_idx = cap_idx;
	cfg.config_enable = UFS_CRYPTO_CONFIGURATION_ENABLE;

	ufshcd_program_key(hba, &cfg, slot);

	memzero_explicit(&cfg, sizeof(cfg));
	return 0;
}

That does make ufshcd_crypto_keyslot_program() slightly longer, but it eliminates lots of other functions. So overall it makes the code much shorter and easier to understand.

drivers/scsi/ufs/ufshcd-crypto.c line 233

Quoted 9 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +9 -0
+static int ufshcd_crypto_keyslot_evict(struct keyslot_manager *ksm,
+				       const struct blk_crypto_key *key,
+				       unsigned int slot)
+{
+	struct ufs_hba *hba = container_of(ksm, struct ufs_hba, ksm);
+
+	if (!(hba->caps & UFSHCD_CAP_CRYPTO) ||
+	    !ufshcd_keyslot_valid(hba, slot))
+		return -EINVAL;

Similarly, there's need for these checks, since the block layer ensures this is only called with a valid keyslot.

drivers/scsi/ufs/ufshcd-crypto.c line 239

Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6 -0
+
+	/*
+	 * Clear the crypto cfg on the device. Clearing CFGE
+	 * might not be sufficient, so just clear the entire cfg.
+	 */
+	ufshcd_clear_keyslot(hba, slot);

This comment seems to be in the wrong place. It should be in ufshcd_clear_keyslot(), where the cfg is actually initialized to 0.

Quoted 5 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +5 -0
+void ufshcd_crypto_enable(struct ufs_hba *hba)
+{
+	if (!ufshcd_hba_is_crypto_supported(hba))
+		return;
+

drivers/scsi/ufs/ufshcd-crypto.c line 258

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	hba->caps |= UFSHCD_CAP_CRYPTO;
+
+	/* Reset might clear all keys, so reprogram all the keys. */
+	blk_ksm_reprogram_all_keys(&hba->ksm);
+}
+
+void ufshcd_crypto_disable(struct ufs_hba *hba)
+{
+	hba->caps &= ~UFSHCD_CAP_CRYPTO;
+}

What's the point of setting and clearing UFSHCD_CAP_CRYPTO every time the host controller is stopped / started? The block layer already ensures that the host controller is runtime-resumed before calling any of the keyslot manager functions. So the crypto support isn't going to transiently go away.

Also, UFSHCD_CAP_CRYPTO is now the means by which the vendor driver indicates that crypto is supported. So what's the point of both ufshcd_hba_is_crypto_supported() and UFSHCD_CAP_CRYPTO? Why not just consistently use UFSHCD_CAP_CRYPTO?

I.e., remove ufshcd_crypto_disable(), and ufshcd_crypto_enable() would be:

void ufshcd_crypto_enable(struct ufs_hba *hba)

{
        /* Reset might clear all keys, so reprogram all the keys. */
        if (hba->caps & UFSHCD_CAP_CRYPTO)
                blk_ksm_reprogram_all_keys(&hba->ksm);
}

ufshcd_hba_init_crypto() would then clear UFSHCD_CAP_CRYPTO if crypto initialization failed / isn't supported. ufshcd_hba_init_crypto() would be the only place that checks the crypto capabilities register.

That all would be much simpler. Any reason not to do that?

Quoted 5 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +5 -0
+static enum blk_crypto_mode_num ufshcd_blk_crypto_mode_num_for_alg_keysize(
+					enum ufs_crypto_alg ufs_crypto_alg,
+					enum ufs_crypto_key_size key_size)
+{
+	int i;

drivers/scsi/ufs/ufshcd-crypto.c line 279

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+
+	for (i = 0; i < ARRAY_SIZE(ufs_crypto_algs); i++) {
+		if (ufs_crypto_algs[i].supported &&
+		    ufs_crypto_algs[i].ufs_alg == ufs_crypto_alg &&
+		    ufs_crypto_algs[i].ufs_key_size == key_size) {
+			return ufs_crypto_algs[i].blk_mode;
+		}
+	}
+	return BLK_ENCRYPTION_MODE_INVALID;
+}

This function name is far too long. Also, the 'supported' and 'blk_mode' fields are unnecessary as I mentioned above. Also, it would be simpler to pass in 'union ufs_crypto_cap_entry'. So how about:

static enum blk_crypto_mode_num ufshcd_find_blk_crypto_mode(union ufs_crypto_cap_entry cap)

{
        int i;

        for (i = 0; i < ARRAY_SIZE(ufs_crypto_algs); i++) {
                BUILD_BUG_ON(UFS_CRYPTO_KEY_SIZE_INVALID != 0);
                if (ufs_crypto_algs[i].ufs_alg == cap.algorithm_id &&
                    ufs_crypto_algs[i].ufs_key_size == cap.key_size)
                        return i;
        }
        return BLK_ENCRYPTION_MODE_INVALID;
}
Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+
+/**
+ * ufshcd_hba_init_crypto - Read crypto capabilities, init crypto fields in hba
+ * @hba: Per adapter instance
+ *
+ * Return: 0 if crypto was initialized or is not supported, else a -errno value.
+ */
+int ufshcd_hba_init_crypto(struct ufs_hba *hba)
+{
+	int cap_idx = 0;

drivers/scsi/ufs/ufshcd-crypto.c line 299

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	int err = 0;
+	enum blk_crypto_mode_num blk_mode_num;
+
+	/*
+	 * Return 0 if crypto support isn't present/not advertised by vendor
+	 * specific driver.
+	 */
+	if (!(hba->capabilities & MASK_CRYPTO_SUPPORT) ||
+	    !(hba->caps & UFSHCD_CAP_CRYPTO))
+		goto out;

People will be confused by the difference between capabilities and caps. This comment could use some elaboration, like:

/*
* Don't use crypto if either the hardware doesn't advertise the
* standard crypto capability bit *or* if the vendor specific driver
* hasn't advertised that crypto is supported.
*/

drivers/scsi/ufs/ufshcd-crypto.c line 305

Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6 -0
+
+	/*
+	 * Crypto Capabilities should never be 0, because the
+	 * config_array_ptr > 04h. So we use a 0 value to indicate that
+	 * crypto init failed, and can't be enabled.
+	 */

This comment would become unnecessary if we were to stop using the cached hardware crypto capabilities register to indicate whether crypto is supported, but rather just UFSHCD_CAP_CRYPTO -- as I suggested above.

drivers/scsi/ufs/ufshcd-crypto.c line 314

Quoted 9 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +9 -0
+	hba->crypto_capabilities.reg_val =
+			cpu_to_le32(ufshcd_readl(hba, REG_UFS_CCAP));
+	hba->crypto_cfg_register =
+		(u32)hba->crypto_capabilities.config_array_ptr * 0x100;
+	hba->crypto_cap_array =
+		devm_kcalloc(hba->dev,
+			     hba->crypto_capabilities.num_crypto_cap,
+			     sizeof(hba->crypto_cap_array[0]),
+			     GFP_KERNEL);

The arguments to devm_kcalloc() can fit on two lines.

drivers/scsi/ufs/ufshcd-crypto.c line 322

Quoted 8 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +8 -0
+	if (!hba->crypto_cap_array) {
+		err = -ENOMEM;
+		goto out;
+	}
+
+	err = blk_ksm_init(&hba->ksm, hba->dev, ufshcd_num_keyslots(hba));
+	if (err)
+		goto out_free_caps;

If the unnecessary checks are removed from other places in this file, this would be the only remaining caller of ufshcd_num_keyslots(), which could then be removed and the comment about the off-by-one quirk could be moved to here:

        /* The actual number of configurations supported is (CFGC+1) */
        num_keyslots = hba->crypto_capabilities.config_count + 1;

	err = blk_ksm_init(&hba->ksm, hba->dev, num_keyslots);

	...


        /* Clear all keyslots at driver init time */
        for (slot = 0; slot < num_keyslots; slot++)
                ufshcd_clear_keyslot(hba, slot);

drivers/scsi/ufs/ufshcd-crypto.c line 331

Quoted 9 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +9 -0
+
+	hba->ksm.ksm_ll_ops = ufshcd_ksm_ops;
+	/* UFS only supports 8 bytes for any DUN */
+	hba->ksm.max_dun_bytes_supported = 8;
+
+	/*
+	 * Store all the capabilities and fill up the keyslot manager's
+	 * supported crypto modes.
+	 */

"fill up the keyslot manager's supported crypto modes" is what the code does, but in comments it's better to describe the goal at a higher level, like

/*
* Cache all the UFS crypto capabilities and advertise the supported
* crypto modes and data unit sizes to the block layer. */

drivers/scsi/ufs/ufshcd-crypto.c line 340

Quoted 9 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +9 -0
+	for (cap_idx = 0; cap_idx < hba->crypto_capabilities.num_crypto_cap;
+	     cap_idx++) {
+		hba->crypto_cap_array[cap_idx].reg_val =
+			cpu_to_le32(ufshcd_readl(hba,
+						 REG_UFS_CRYPTOCAP +
+						 cap_idx * sizeof(__le32)));
+		blk_mode_num = ufshcd_blk_crypto_mode_num_for_alg_keysize(
+				hba->crypto_cap_array[cap_idx].algorithm_id,
+				hba->crypto_cap_array[cap_idx].key_size);

As I mentioned above, this could simply pass the capability directly.

Quoted 6 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +6 -0
+		if (blk_mode_num != BLK_ENCRYPTION_MODE_INVALID)
+			hba->ksm.crypto_modes_supported[blk_mode_num] |=
+				hba->crypto_cap_array[cap_idx].sdus_mask * 512;
+	}
+
+	ufshcd_clear_all_keyslots(hba);

drivers/scsi/ufs/ufshcd-crypto.c line 356

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+
+	return 0;
+
+out_free_caps:
+	devm_kfree(hba->dev, hba->crypto_cap_array);
+out:
+	/* Indicate that init failed by setting crypto_capabilities to 0 */
+	hba->crypto_capabilities.reg_val = 0;
+	return err;
+}

As I mentioned above, why not clear UFSHCD_CAP_CRYPTO here instead of crypto_capabilities, and use UFSHCD_CAP_CRYPTO consistently everywhere?

Quoted 4 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +4 -0
+
+void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba,
+					    struct request_queue *q)
+{

drivers/scsi/ufs/ufshcd-crypto.c line 370

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	if (!ufshcd_hba_is_crypto_supported(hba) || !q)
+		return;
+
+	blk_ksm_register(&hba->ksm, q);
+}
+
+void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
+{
+	blk_ksm_destroy(&hba->ksm);
+}

q can't be NULL here.

Quoted 24 lines
drivers/scsi/ufs/ufshcd.h +5 -0
index 978781c538c4..ee37dd44d5e6 100644
@@ -55,6 +55,7 @@
 #include <linux/clk.h>
 #include <linux/completion.h>
 #include <linux/regulator/consumer.h>
+#include <linux/keyslot-manager.h>
 #include "unipro.h"
 
 #include <asm/irq.h>
@@ -521,6 +522,10 @@ struct ufs_stats {
  * @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for
  *  device is known or not.
  * @scsi_block_reqs_cnt: reference counting for scsi block requests
+ * @crypto_capabilities: Content of crypto capabilities register (0x100)
+ * @crypto_cap_array: Array of crypto capabilities
+ * @crypto_cfg_register: Start of the crypto cfg array
+ * @ksm: the keyslot manager tied to this hba
  */
 struct ufs_hba {
 	void __iomem *mmio_base;

drivers/scsi/ufs/ufshcd.h struct ufs_hba { · line 643

Quoted 5 lines
drivers/scsi/ufs/ufshcd.h +1 -0
@@ -634,6 +639,7 @@ struct ufs_hba {
 	 * enabled via HCE register.
 	 */
 	#define UFSHCI_QUIRK_BROKEN_HCE				0x400
+
 	unsigned int quirks;	/* Deviations from standard UFSHCI spec. */

Unnecessary whitespace change.

Quoted 4 lines
drivers/scsi/ufs/ufshcd.h +0 -0
 	/* Device deviations from standard UFS device spec. */
@@ -735,6 +741,14 @@ struct ufs_hba {
 

drivers/scsi/ufs/ufshcd.h struct ufs_hba { · line 751

Quoted 10 lines
drivers/scsi/ufs/ufshcd.h +8 -0
 	struct device		bsg_dev;
 	struct request_queue	*bsg_queue;
+
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+	/* crypto */
+	union ufs_crypto_capabilities crypto_capabilities;
+	union ufs_crypto_cap_entry *crypto_cap_array;
+	u32 crypto_cfg_register;
+	struct keyslot_manager ksm;
+#endif /* CONFIG_SCSI_UFS_CRYPTO */

These two comments don't really add anything. It's already obviously "crypto" fields based on the #ifdef and variable names. And a 4 line #ifdef isn't long enough for a comment on the #endif to be helpful.

- Eric

#12 lore raw

PATCH v8 7/11 scsi: ufs: Add inline encryption support to UFS

In reply to Satya Tangirala

Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.

---
 drivers/scsi/ufs/ufshcd-crypto.c | 27 +++++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h | 14 ++++++++
 drivers/scsi/ufs/ufshcd.c        | 59 +++++++++++++++++++++++++++++---
 drivers/scsi/ufs/ufshcd.h        |  8 +++++
 4 files changed, 103 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd-crypto.c +27 -0
index 8b6f7c83f77f..37254472326a 100644
@@ -368,3 +368,30 @@ void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
 {
 	blk_ksm_destroy(&hba->ksm);
 }
+
+int ufshcd_prepare_lrbp_crypto(struct ufs_hba *hba,
+			       struct scsi_cmnd *cmd,
+			       struct ufshcd_lrb *lrbp)
+{
+	struct request *rq = cmd->request;
+	struct bio_crypt_ctx *bc = rq->crypt_ctx;
+	unsigned int slot_idx = blk_ksm_get_slot_idx(rq->crypt_keyslot);
+
+	lrbp->crypto_enable = false;
+
+	if (WARN_ON(!(hba->caps & UFSHCD_CAP_CRYPTO))) {
+		/*
+		 * Upper layer asked us to do inline encryption
+		 * but that isn't enabled, so we fail this request.
+		 */
+		return -EINVAL;
+	}
+	if (!ufshcd_keyslot_valid(hba, slot_idx))
+		return -EINVAL;
+
+	lrbp->crypto_enable = true;
+	lrbp->crypto_key_slot = slot_idx;
+	lrbp->data_unit_num = bc->bc_dun[0];
+
+	return 0;
+}
drivers/scsi/ufs/ufshcd-crypto.h +14 -0
index 8270c0c5081a..c76f93ede51c 100644
@@ -16,6 +16,15 @@ static inline bool ufshcd_hba_is_crypto_supported(struct ufs_hba *hba)
 	return hba->crypto_capabilities.reg_val != 0;
 }
 
+int ufshcd_prepare_lrbp_crypto(struct ufs_hba *hba,
+			       struct scsi_cmnd *cmd,
+			       struct ufshcd_lrb *lrbp);
+
+static inline bool ufshcd_lrbp_crypto_enabled(struct ufshcd_lrb *lrbp)
+{
+	return lrbp->crypto_enable;
+}
+
 void ufshcd_crypto_enable(struct ufs_hba *hba);
 
 void ufshcd_crypto_disable(struct ufs_hba *hba);
@@ -49,6 +58,11 @@ static inline void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba,
 static inline void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
 { }
 
+static inline bool ufshcd_lrbp_crypto_enabled(struct ufshcd_lrb *lrbp)
+{
+	return false;
+}
+
 #endif /* CONFIG_SCSI_UFS_CRYPTO */
 
 #endif /* _UFSHCD_CRYPTO_H */
drivers/scsi/ufs/ufshcd.c +54 -5

120 lines not on this page · raw message

drivers/scsi/ufs/ufshcd.h +8 -0
index ee37dd44d5e6..78397864f911 100644
@@ -168,6 +168,9 @@ struct ufs_pm_lvl_states {
  * @intr_cmd: Interrupt command (doesn't participate in interrupt aggregation)
  * @issue_time_stamp: time stamp for debug purposes
  * @compl_time_stamp: time stamp for statistics
+ * @crypto_enable: whether or not the request needs inline crypto operations
+ * @crypto_key_slot: the key slot to use for inline crypto
+ * @data_unit_num: the data unit number for the first block for inline crypto
  * @req_abort_skip: skip request abort task flag
  */
 struct ufshcd_lrb {
@@ -192,6 +195,11 @@ struct ufshcd_lrb {
 	bool intr_cmd;
 	ktime_t issue_time_stamp;
 	ktime_t compl_time_stamp;
+#if IS_ENABLED(CONFIG_SCSI_UFS_CRYPTO)
+	bool crypto_enable;
+	u8 crypto_key_slot;
+	u64 data_unit_num;
+#endif /* CONFIG_SCSI_UFS_CRYPTO */
 
 	bool req_abort_skip;
 };
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#13 lore raw

Re: [PATCH v8 07/11] scsi: ufs: Add inline encryption support to UFS

Quoted 163 lines · Satya Tangirala

Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.

---
 drivers/scsi/ufs/ufshcd-crypto.c | 27 +++++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h | 14 ++++++++
 drivers/scsi/ufs/ufshcd.c        | 59 +++++++++++++++++++++++++++++---
 drivers/scsi/ufs/ufshcd.h        |  8 +++++
 4 files changed, 103 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd-crypto.c +27 -0
index 8b6f7c83f77f..37254472326a 100644
@@ -368,3 +368,30 @@ void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
 {
 	blk_ksm_destroy(&hba->ksm);
 }
+
+int ufshcd_prepare_lrbp_crypto(struct ufs_hba *hba,
+			       struct scsi_cmnd *cmd,
+			       struct ufshcd_lrb *lrbp)
+{
+	struct request *rq = cmd->request;
+	struct bio_crypt_ctx *bc = rq->crypt_ctx;
+	unsigned int slot_idx = blk_ksm_get_slot_idx(rq->crypt_keyslot);
+
+	lrbp->crypto_enable = false;
+
+	if (WARN_ON(!(hba->caps & UFSHCD_CAP_CRYPTO))) {
+		/*
+		 * Upper layer asked us to do inline encryption
+		 * but that isn't enabled, so we fail this request.
+		 */
+		return -EINVAL;
+	}
+	if (!ufshcd_keyslot_valid(hba, slot_idx))
+		return -EINVAL;
+
+	lrbp->crypto_enable = true;
+	lrbp->crypto_key_slot = slot_idx;
+	lrbp->data_unit_num = bc->bc_dun[0];
+
+	return 0;
+}
drivers/scsi/ufs/ufshcd-crypto.h +14 -0
index 8270c0c5081a..c76f93ede51c 100644
@@ -16,6 +16,15 @@ static inline bool ufshcd_hba_is_crypto_supported(struct ufs_hba *hba)
 	return hba->crypto_capabilities.reg_val != 0;
 }
 
+int ufshcd_prepare_lrbp_crypto(struct ufs_hba *hba,
+			       struct scsi_cmnd *cmd,
+			       struct ufshcd_lrb *lrbp);
+
+static inline bool ufshcd_lrbp_crypto_enabled(struct ufshcd_lrb *lrbp)
+{
+	return lrbp->crypto_enable;
+}
+
 void ufshcd_crypto_enable(struct ufs_hba *hba);
 
 void ufshcd_crypto_disable(struct ufs_hba *hba);
@@ -49,6 +58,11 @@ static inline void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba,
 static inline void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
 { }
 
+static inline bool ufshcd_lrbp_crypto_enabled(struct ufshcd_lrb *lrbp)
+{
+	return false;
+}
+
 #endif /* CONFIG_SCSI_UFS_CRYPTO */
 
 #endif /* _UFSHCD_CRYPTO_H */
drivers/scsi/ufs/ufshcd.c +39 -5
index 825d9eb34f10..3a19966dbee9 100644
@@ -47,6 +47,7 @@
 #include "unipro.h"
 #include "ufs-sysfs.h"
 #include "ufs_bsg.h"
+#include "ufshcd-crypto.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/ufs.h>
@@ -816,7 +817,14 @@ static void ufshcd_enable_run_stop_reg(struct ufs_hba *hba)
  */
 static inline void ufshcd_hba_start(struct ufs_hba *hba)
 {
-	ufshcd_writel(hba, CONTROLLER_ENABLE, REG_CONTROLLER_ENABLE);
+	u32 val = CONTROLLER_ENABLE;
+
+	if (ufshcd_hba_is_crypto_supported(hba)) {
+		ufshcd_crypto_enable(hba);
+		val |= CRYPTO_GENERAL_ENABLE;
+	}
+
+	ufshcd_writel(hba, val, REG_CONTROLLER_ENABLE);
 }
 
 /**
@@ -2192,9 +2200,23 @@ static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp,
 		dword_0 |= UTP_REQ_DESC_INT_CMD;
 
 	/* Transfer request descriptor header fields */
+	if (ufshcd_lrbp_crypto_enabled(lrbp)) {
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+		dword_0 |= UTP_REQ_DESC_CRYPTO_ENABLE_CMD;
+		dword_0 |= lrbp->crypto_key_slot;
+		req_desc->header.dword_1 =
+			cpu_to_le32(lower_32_bits(lrbp->data_unit_num));
+		req_desc->header.dword_3 =
+			cpu_to_le32(upper_32_bits(lrbp->data_unit_num));
+#endif /* CONFIG_SCSI_UFS_CRYPTO */
+	} else {
+		/* dword_1 and dword_3 are reserved, hence they are set to 0 */
+		req_desc->header.dword_1 = 0;
+		req_desc->header.dword_3 = 0;
+	}
+
 	req_desc->header.dword_0 = cpu_to_le32(dword_0);
-	/* dword_1 is reserved, hence it is set to 0 */
-	req_desc->header.dword_1 = 0;
+
 	/*
 	 * assigning invalid value for command status. Controller
 	 * updates OCS on command completion, with the command
@@ -2202,8 +2224,6 @@ static void ufshcd_prepare_req_desc_hdr(struct ufshcd_lrb *lrbp,
 	 */
 	req_desc->header.dword_2 =
 		cpu_to_le32(OCS_INVALID_COMMAND_STATUS);
-	/* dword_3 is reserved, hence it is set to 0 */
-	req_desc->header.dword_3 = 0;
 
 	req_desc->prd_table_length = 0;
 }
@@ -2437,6 +2457,20 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd)
 	lrbp->task_tag = tag;
 	lrbp->lun = ufshcd_scsi_to_upiu_lun(cmd->device->lun);
 	lrbp->intr_cmd = !ufshcd_is_intr_aggr_allowed(hba) ? true : false;
+
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+	if (cmd->request->crypt_keyslot) {
+		err = ufshcd_prepare_lrbp_crypto(hba, cmd, lrbp);
+		if (err) {
+			lrbp->cmd = NULL;
+			ufshcd_release(hba);
+			goto out;
+		}
+	} else {
+		lrbp->crypto_enable = false;
+	}
+#endif
+
 	lrbp->req_abort_skip = false;
 
 	ufshcd_comp_scsi_upiu(hba, lrbp);

drivers/scsi/ufs/ufshcd.c static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, · line 2512

Quoted 9 lines
drivers/scsi/ufs/ufshcd.c +3 -0
@@ -2470,6 +2504,9 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba,
 	lrbp->task_tag = tag;
 	lrbp->lun = 0; /* device management cmd is not specific to any LUN */
 	lrbp->intr_cmd = true; /* No interrupt aggregation */
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+	lrbp->crypto_enable = false; /* No crypto operations */
+#endif
 	hba->dev_cmd.type = cmd_type;
 
 	return ufshcd_comp_devman_upiu(hba, lrbp);

This is all much more complicated than needed.

- We don't need the 'crypto_enable' field. Just make 'crypto_key_slot' an int and use 'crypto_key_slot = -1' instead.

- ufshcd_lrbp_crypto_enabled() isn't needed since it's only used in one place which uses an ifdef anyway. It can simply check crypto_key_slot >= 0.

- The checks in ufshcd_prepare_lrbp_crypto() are unneeded, as the block layer already ensures that a valid keyslot is provided.

So for the per-command code, all we actually need is:

#ifdef CONFIG_SCSI_UFS_CRYPTO
static inline void ufshcd_prepare_lrbp_crypto(struct scsi_cmnd *cmd,
struct ufshcd_lrb *lrbp)

{       
        struct request *rq = cmd->request; 
        
        if (rq->crypt_keyslot) {
                lrbp->crypto_key_slot = blk_ksm_get_slot_idx(rq->crypt_keyslot);
                lrbp->data_unit_num = rq->crypt_ctx->bc_dun[0];
        } else {
                lrbp->crypto_key_slot = -1;
        }
}                              

...
#else
static inline void ufshcd_prepare_lrbp_crypto(struct scsi_cmnd *cmd,
struct ufshcd_lrb *lrbp)
{
}
...
#endif

Then in ufshcd_queuecommand():

	...

        lrbp->intr_cmd = !ufshcd_is_intr_aggr_allowed(hba) ? true : false;
        lrbp->req_abort_skip = false;

        ufshcd_prepare_lrbp_crypto(cmd, lrbp);

        ufshcd_comp_scsi_upiu(hba, lrbp);

	...

Then in ufshcd_prepare_req_desc_hdr():

        u32 dword_1 = 0;
        u32 dword_3 = 0;

	...
#ifdef CONFIG_SCSI_UFS_CRYPTO
        if (lrbp->crypto_key_slot >= 0) {
                dword_0 |= UTP_REQ_DESC_CRYPTO_ENABLE_CMD;
                dword_0 |= lrbp->crypto_key_slot;
                dword_1 = lower_32_bits(lrbp->data_unit_num);
                dword_3 = upper_32_bits(lrbp->data_unit_num);
        }

#endif

	req_desc->header.dword_0 = cpu_to_le32(dword_0);
	req_desc->header.dword_1 = cpu_to_le32(dword_1);
	...
	req_desc->header.dword_2 =
		cpu_to_le32(OCS_INVALID_COMMAND_STATUS);
	req_desc->header.dword_3 = cpu_to_le32(dword_3);
maintainer
#14 lore raw

Re: [PATCH v8 07/11] scsi: ufs: Add inline encryption support to UFS

In reply to Satya Tangirala

drivers/scsi/ufs/ufshcd.c static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, · line 2510

Quoted 8 lines · Satya Tangirala
drivers/scsi/ufs/ufshcd.c +3 -0
@@ -2470,6 +2504,9 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba,
 	lrbp->task_tag = tag;
 	lrbp->lun = 0; /* device management cmd is not specific to any LUN */
 	lrbp->intr_cmd = true; /* No interrupt aggregation */
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+	lrbp->crypto_enable = false; /* No crypto operations */
+#endif
 	hba->dev_cmd.type = cmd_type;

Doesn't this need to be initialized in ufshcd_issue_devman_upiu_cmd() too?

#15 lore raw

PATCH v8 8/11 fs: introduce SB_INLINECRYPT

In reply to Satya Tangirala

Introduce SB_INLINECRYPT, which is set by filesystems that wish to use blk-crypto for file content en/decryption.

---
 fs/proc_namespace.c | 1 +
 include/linux/fs.h  | 1 +
 2 files changed, 2 insertions(+)
fs/proc_namespace.c +1 -0
index 273ee82d8aa9..8bf195d3bda6 100644
@@ -49,6 +49,7 @@ static int show_sb_opts(struct seq_file *m, struct super_block *sb)
 		{ SB_DIRSYNC, ",dirsync" },
 		{ SB_MANDLOCK, ",mand" },
 		{ SB_LAZYTIME, ",lazytime" },
+		{ SB_INLINECRYPT, ",inlinecrypt" },
 		{ 0, NULL }
 	};
 	const struct proc_fs_info *fs_infop;
include/linux/fs.h +1 -0
index 3cd4fe6b845e..08a0395674dd 100644
@@ -1370,6 +1370,7 @@ extern int send_sigurg(struct fown_struct *fown);
 #define SB_NODIRATIME	2048	/* Do not update directory access times */
 #define SB_SILENT	32768
 #define SB_POSIXACL	(1<<16)	/* VFS does not apply the umask */
+#define SB_INLINECRYPT	(1<<17)	/* inodes in SB use blk-crypto */
 #define SB_KERNMOUNT	(1<<22) /* this is a kern_mount call */
 #define SB_I_VERSION	(1<<23) /* Update inode I_version field */
 #define SB_LAZYTIME	(1<<25) /* Update the on-disk [acm]times lazily */
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#16 lore raw

Re: [PATCH v8 08/11] fs: introduce SB_INLINECRYPT

Quoted 26 lines · Satya Tangirala

Introduce SB_INLINECRYPT, which is set by filesystems that wish to use blk-crypto for file content en/decryption.

---
 fs/proc_namespace.c | 1 +
 include/linux/fs.h  | 1 +
 2 files changed, 2 insertions(+)
fs/proc_namespace.c +1 -0
index 273ee82d8aa9..8bf195d3bda6 100644
@@ -49,6 +49,7 @@ static int show_sb_opts(struct seq_file *m, struct super_block *sb)
 		{ SB_DIRSYNC, ",dirsync" },
 		{ SB_MANDLOCK, ",mand" },
 		{ SB_LAZYTIME, ",lazytime" },
+		{ SB_INLINECRYPT, ",inlinecrypt" },
 		{ 0, NULL }
 	};
 	const struct proc_fs_info *fs_infop;
include/linux/fs.h +0 -0
index 3cd4fe6b845e..08a0395674dd 100644

include/linux/fs.h extern int send_sigurg(struct fown_struct *fown); · line 1373

Quoted 4 lines
include/linux/fs.h +1 -0
@@ -1370,6 +1370,7 @@ extern int send_sigurg(struct fown_struct *fown);
 #define SB_NODIRATIME	2048	/* Do not update directory access times */
 #define SB_SILENT	32768
 #define SB_POSIXACL	(1<<16)	/* VFS does not apply the umask */
+#define SB_INLINECRYPT	(1<<17)	/* inodes in SB use blk-crypto */

"inodes use blk-crypto" isn't very clear. It could be misunderstand as meaning something like "does the filesystem contain any encrypted files". I think the following would be a bit clearer:

	/* Use blk-crypto for encrypted files */

(And these flags are obviously per-sb, so there's no need to write "in SB".)

#17 lore raw

PATCH v8 9/11 fscrypt: add inline encryption support

In reply to Satya Tangirala

Add support for inline encryption to fs/crypto/. With "inline encryption", the block layer handles the decryption/encryption as part of the bio, instead of the filesystem doing the crypto itself via Linux's crypto API. This model is needed in order to take advantage of the inline encryption hardware present on most modern mobile SoCs.

To use inline encryption, the filesystem needs to be mounted with '-o inlinecrypt'. The contents of any encrypted files will then be encrypted using blk-crypto, instead of using the traditional filesystem-layer crypto. Fscrypt still provides the key and IV to use, and the actual ciphertext on-disk is still the same; therefore it's testable using the existing fscrypt ciphertext verification tests.

Note that since blk-crypto has a fallack to Linux's crypto API, and also supports all the encryption modes currently supported by fscrypt, this feature is usable and testable even without actual inline encryption hardware.

Per-filesystem changes will be needed to set encryption contexts when submitting bios and to implement the 'inlinecrypt' mount option. This patch just adds the common code.

---
 fs/crypto/Kconfig           |   6 +
 fs/crypto/Makefile          |   1 +
 fs/crypto/bio.c             |  51 ++++++
 fs/crypto/crypto.c          |   2 +-
 fs/crypto/fname.c           |   4 +-
 fs/crypto/fscrypt_private.h | 120 +++++++++++--
 fs/crypto/inline_crypt.c    | 328 ++++++++++++++++++++++++++++++++++++
 fs/crypto/keyring.c         |   4 +-
 fs/crypto/keysetup.c        |  92 ++++++----
 fs/crypto/keysetup_v1.c     |  16 +-
 include/linux/fscrypt.h     |  57 +++++++
 11 files changed, 624 insertions(+), 57 deletions(-)
 create mode 100644 fs/crypto/inline_crypt.c
fs/crypto/Kconfig +6 -0
index 8046d7c7a3e9..f1f11a6228eb 100644
@@ -24,3 +24,9 @@ config FS_ENCRYPTION_ALGS
 	select CRYPTO_SHA256
 	select CRYPTO_SHA512
 	select CRYPTO_XTS
+
+config FS_ENCRYPTION_INLINE_CRYPT
+	bool "Enable fscrypt to use inline crypto"
+	depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION
+	help
+	  Enable fscrypt to use inline encryption hardware if available.
fs/crypto/Makefile +1 -0
index 232e2bb5a337..652c7180ec6d 100644
@@ -11,3 +11,4 @@ fscrypto-y := crypto.o \
 	      policy.o
 
 fscrypto-$(CONFIG_BLOCK) += bio.o
+fscrypto-$(CONFIG_FS_ENCRYPTION_INLINE_CRYPT) += inline_crypt.o
fs/crypto/bio.c +51 -0

69 lines not on this page · raw message

fs/crypto/crypto.c +1 -1
index 1ecaac7ee3cb..263bc676c73d 100644
@@ -95,7 +95,7 @@ int fscrypt_crypt_block(const struct inode *inode, fscrypt_direction_t rw,
 	DECLARE_CRYPTO_WAIT(wait);
 	struct scatterlist dst, src;
 	struct fscrypt_info *ci = inode->i_crypt_info;
-	struct crypto_skcipher *tfm = ci->ci_ctfm;
+	struct crypto_skcipher *tfm = ci->ci_key.tfm;
 	int res = 0;
 
 	if (WARN_ON_ONCE(len <= 0))
fs/crypto/fname.c +2 -2
index 4c212442a8f7..0fca2d7a5645 100644
@@ -117,7 +117,7 @@ int fscrypt_fname_encrypt(const struct inode *inode, const struct qstr *iname,
 	struct skcipher_request *req = NULL;
 	DECLARE_CRYPTO_WAIT(wait);
 	const struct fscrypt_info *ci = inode->i_crypt_info;
-	struct crypto_skcipher *tfm = ci->ci_ctfm;
+	struct crypto_skcipher *tfm = ci->ci_key.tfm;
 	union fscrypt_iv iv;
 	struct scatterlist sg;
 	int res;
@@ -170,7 +170,7 @@ static int fname_decrypt(const struct inode *inode,
 	DECLARE_CRYPTO_WAIT(wait);
 	struct scatterlist src_sg, dst_sg;
 	const struct fscrypt_info *ci = inode->i_crypt_info;
-	struct crypto_skcipher *tfm = ci->ci_ctfm;
+	struct crypto_skcipher *tfm = ci->ci_key.tfm;
 	union fscrypt_iv iv;
 	int res;
 
fs/crypto/fscrypt_private.h +107 -13

177 lines not on this page · raw message

fs/crypto/inline_crypt.c new file +328 -0

328 lines not on this page · raw message

fs/crypto/keyring.c +2 -2
index ab41b25d4fa1..d8ab33f631ba 100644
@@ -44,8 +44,8 @@ static void free_master_key(struct fscrypt_master_key *mk)
 	wipe_master_key_secret(&mk->mk_secret);
 
 	for (i = 0; i <= __FSCRYPT_MODE_MAX; i++) {
-		crypto_free_skcipher(mk->mk_direct_tfms[i]);
-		crypto_free_skcipher(mk->mk_iv_ino_lblk_64_tfms[i]);
+		fscrypt_destroy_prepared_key(&mk->mk_direct_keys[i]);
+		fscrypt_destroy_prepared_key(&mk->mk_iv_ino_lblk_64_keys[i]);
 	}
 
 	key_put(mk->mk_users);
fs/crypto/keysetup.c +62 -30

177 lines not on this page · raw message

fs/crypto/keysetup_v1.c +7 -9
index 801b48c0cd7f..59c520b200cb 100644
@@ -146,7 +146,7 @@ struct fscrypt_direct_key {
 	struct hlist_node		dk_node;
 	refcount_t			dk_refcount;
 	const struct fscrypt_mode	*dk_mode;
-	struct crypto_skcipher		*dk_ctfm;
+	struct fscrypt_prepared_key	dk_key;
 	u8				dk_descriptor[FSCRYPT_KEY_DESCRIPTOR_SIZE];
 	u8				dk_raw[FSCRYPT_MAX_KEY_SIZE];
 };
@@ -154,7 +154,7 @@ struct fscrypt_direct_key {
 static void free_direct_key(struct fscrypt_direct_key *dk)
 {
 	if (dk) {
-		crypto_free_skcipher(dk->dk_ctfm);
+		fscrypt_destroy_prepared_key(&dk->dk_key);
 		kzfree(dk);
 	}
 }
@@ -199,6 +199,8 @@ find_or_insert_direct_key(struct fscrypt_direct_key *to_insert,
 			continue;
 		if (ci->ci_mode != dk->dk_mode)
 			continue;
+		if (!fscrypt_is_key_prepared(&dk->dk_key, ci))
+			continue;
 		if (crypto_memneq(raw_key, dk->dk_raw, ci->ci_mode->keysize))
 			continue;
 		/* using existing tfm with same (descriptor, mode, raw_key) */
@@ -231,13 +233,9 @@ fscrypt_get_direct_key(const struct fscrypt_info *ci, const u8 *raw_key)
 		return ERR_PTR(-ENOMEM);
 	refcount_set(&dk->dk_refcount, 1);
 	dk->dk_mode = ci->ci_mode;
-	dk->dk_ctfm = fscrypt_allocate_skcipher(ci->ci_mode, raw_key,
-						ci->ci_inode);
-	if (IS_ERR(dk->dk_ctfm)) {
-		err = PTR_ERR(dk->dk_ctfm);
-		dk->dk_ctfm = NULL;
+	err = fscrypt_prepare_key(&dk->dk_key, raw_key, ci);
+	if (err)
 		goto err_free_dk;
-	}
 	memcpy(dk->dk_descriptor, ci->ci_policy.v1.master_key_descriptor,
 	       FSCRYPT_KEY_DESCRIPTOR_SIZE);
 	memcpy(dk->dk_raw, raw_key, ci->ci_mode->keysize);
@@ -259,7 +257,7 @@ static int setup_v1_file_key_direct(struct fscrypt_info *ci,
 	if (IS_ERR(dk))
 		return PTR_ERR(dk);
 	ci->ci_direct_key = dk;
-	ci->ci_ctfm = dk->dk_ctfm;
+	ci->ci_key = dk->dk_key;
 	return 0;
 }
 
include/linux/fscrypt.h +57 -0

69 lines not on this page · raw message

Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#18 lore raw

Re: [PATCH v8 09/11] fscrypt: add inline encryption support

fs/crypto/inline_crypt.c line 56

Quoted 8 lines · Satya Tangirala
fs/crypto/inline_crypt.c new file +8 -0
+int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
+				     const u8 *raw_key,
+				     const struct fscrypt_info *ci)
+{
+	const struct inode *inode = ci->ci_inode;
+	struct super_block *sb = inode->i_sb;
+	enum blk_crypto_mode_num crypto_mode = ci->ci_mode->blk_crypto_mode;
+	unsigned int blk_crypto_dun_bytes;

'blk_crypto_dun_bytes' is overly verbose. 'dun_bytes' would still be just as clear.

This comment also applies to the parameter to blk_crypto_init_key().

Quoted 11 lines
fs/crypto/inline_crypt.c new file +11 -0
+/**
+ * fscrypt_set_bio_crypt_ctx_bh - prepare a file contents bio for inline
+ *				  encryption
+ * @bio: a bio which will eventually be submitted to the file
+ * @first_bh: the first buffer_head for which I/O will be submitted
+ * @gfp_mask: memory allocation flags
+ *
+ * Same as fscrypt_set_bio_crypt_ctx(), except this takes a buffer_head instead
+ * of an inode and block number directly.
+ */
+void fscrypt_set_bio_crypt_ctx_bh(struct bio *bio,

fs/crypto/inline_crypt.c line 264

Quoted 10 lines
fs/crypto/inline_crypt.c new file +10 -0
+				 const struct buffer_head *first_bh,
+				 gfp_t gfp_mask)
+{
+	const struct inode *inode;
+	u64 first_lblk;
+
+	if (bh_get_inode_and_lblk_num(first_bh, &inode, &first_lblk))
+		fscrypt_set_bio_crypt_ctx(bio, inode, first_lblk, gfp_mask);
+}
+EXPORT_SYMBOL_GPL(fscrypt_set_bio_crypt_ctx_bh);

Nit: the continuation lines for the function arguments aren't aligned.

- Eric

#19 lore raw

PATCH v8 10/11 f2fs: add inline encryption support

In reply to Satya Tangirala

Wire up f2fs to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:

- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.

- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.

- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.

- Not doing filesystem-layer crypto on inline-encrypted files.

---
 Documentation/filesystems/f2fs.txt |  6 +++
 fs/f2fs/compress.c                 |  2 +-
 fs/f2fs/data.c                     | 68 +++++++++++++++++++++++++-----
 fs/f2fs/f2fs.h                     |  3 ++
 fs/f2fs/super.c                    | 33 +++++++++++++++
 5 files changed, 101 insertions(+), 11 deletions(-)
Documentation/filesystems/f2fs.txt +6 -0
index 4eb3e2ddd00e..300d5843cbe4 100644
@@ -246,6 +246,12 @@ compress_extension=%s  Support adding specified extension, so that f2fs can enab
                        on compression extension list and enable compression on
                        these file by default rather than to enable it via ioctl.
                        For other files, we can still enable compression via ioctl.
+inlinecrypt
+                       Encrypt/decrypt the contents of encrypted files using the
+                       blk-crypto framework rather than filesystem-layer encryption.
+                       This allows the use of inline encryption hardware. The on-disk
+                       format is unaffected. For more details, see
+                       Documentation/block/inline-encryption.rst.
 
 ================================================================================
 DEBUGFS ENTRIES
fs/f2fs/compress.c +1 -1
index d8a64be90a50..6ae313ea5432 100644
@@ -775,7 +775,7 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
 		.need_lock = LOCK_RETRY,
 		.io_type = io_type,
 		.io_wbc = wbc,
-		.encrypted = f2fs_encrypted_file(cc->inode),
+		.encrypted = fscrypt_inode_uses_fs_layer_crypto(cc->inode),
 	};
 	struct dnode_of_data dn;
 	struct node_info ni;
fs/f2fs/data.c +58 -10

142 lines not on this page · raw message

fs/f2fs/f2fs.h +3 -0
index 5355be6b6755..75817f0dc6f8 100644
@@ -139,6 +139,9 @@ struct f2fs_mount_info {
 	int alloc_mode;			/* segment allocation policy */
 	int fsync_mode;			/* fsync policy */
 	bool test_dummy_encryption;	/* test dummy encryption */
+#ifdef CONFIG_FS_ENCRYPTION
+	bool inlinecrypt;		/* inline encryption enabled */
+#endif
 	block_t unusable_cap;		/* Amount of space allowed to be
 					 * unusable when disabling checkpoint
 					 */
fs/f2fs/super.c +33 -0
index 65a7a432dfee..aedbf4cae083 100644
@@ -137,6 +137,7 @@ enum {
 	Opt_alloc,
 	Opt_fsync,
 	Opt_test_dummy_encryption,
+	Opt_inlinecrypt,
 	Opt_checkpoint_disable,
 	Opt_checkpoint_disable_cap,
 	Opt_checkpoint_disable_cap_perc,
@@ -202,6 +203,7 @@ static match_table_t f2fs_tokens = {
 	{Opt_alloc, "alloc_mode=%s"},
 	{Opt_fsync, "fsync_mode=%s"},
 	{Opt_test_dummy_encryption, "test_dummy_encryption"},
+	{Opt_inlinecrypt, "inlinecrypt"},
 	{Opt_checkpoint_disable, "checkpoint=disable"},
 	{Opt_checkpoint_disable_cap, "checkpoint=disable:%u"},
 	{Opt_checkpoint_disable_cap_perc, "checkpoint=disable:%u%%"},
@@ -790,6 +792,13 @@ static int parse_options(struct super_block *sb, char *options)
 			f2fs_info(sbi, "Test dummy encryption mode enabled");
 #else
 			f2fs_info(sbi, "Test dummy encryption mount option ignored");
+#endif
+			break;
+		case Opt_inlinecrypt:
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+			sb->s_flags |= SB_INLINECRYPT;
+#else
+			f2fs_info(sbi, "inline encryption not supported");
 #endif
 			break;
 		case Opt_checkpoint_disable_cap_perc:
@@ -1568,6 +1577,9 @@ static void default_options(struct f2fs_sb_info *sbi)
 	F2FS_OPTION(sbi).alloc_mode = ALLOC_MODE_DEFAULT;
 	F2FS_OPTION(sbi).fsync_mode = FSYNC_MODE_POSIX;
 	F2FS_OPTION(sbi).test_dummy_encryption = false;
+#ifdef CONFIG_FS_ENCRYPTION
+	sbi->sb->s_flags &= ~SB_INLINECRYPT;
+#endif
 	F2FS_OPTION(sbi).s_resuid = make_kuid(&init_user_ns, F2FS_DEF_RESUID);
 	F2FS_OPTION(sbi).s_resgid = make_kgid(&init_user_ns, F2FS_DEF_RESGID);
 	F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZO;
@@ -2415,6 +2427,25 @@ static void f2fs_get_ino_and_lblk_bits(struct super_block *sb,
 	*lblk_bits_ret = 8 * sizeof(block_t);
 }
 
+static int f2fs_get_num_devices(struct super_block *sb)
+{
+	struct f2fs_sb_info *sbi = F2FS_SB(sb);
+
+	if (f2fs_is_multi_device(sbi))
+		return sbi->s_ndevs;
+	return 1;
+}
+
+static void f2fs_get_devices(struct super_block *sb,
+			     struct request_queue **devs)
+{
+	struct f2fs_sb_info *sbi = F2FS_SB(sb);
+	int i;
+
+	for (i = 0; i < sbi->s_ndevs; i++)
+		devs[i] = bdev_get_queue(FDEV(i).bdev);
+}
+
 static const struct fscrypt_operations f2fs_cryptops = {
 	.key_prefix		= "f2fs:",
 	.get_context		= f2fs_get_context,
@@ -2424,6 +2455,8 @@ static const struct fscrypt_operations f2fs_cryptops = {
 	.max_namelen		= F2FS_NAME_LEN,
 	.has_stable_inodes	= f2fs_has_stable_inodes,
 	.get_ino_and_lblk_bits	= f2fs_get_ino_and_lblk_bits,
+	.get_num_devices	= f2fs_get_num_devices,
+	.get_devices		= f2fs_get_devices,
 };
 #endif
 
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#20 lore raw

Re: [PATCH v8 10/11] f2fs: add inline encryption support

Quoted 5 lines · Satya Tangirala
fs/f2fs/f2fs.h +0 -0
index 5355be6b6755..75817f0dc6f8 100644

fs/f2fs/f2fs.h struct f2fs_mount_info { · line 147

Quoted 9 lines
fs/f2fs/f2fs.h +3 -0
@@ -139,6 +139,9 @@ struct f2fs_mount_info {
 	int alloc_mode;			/* segment allocation policy */
 	int fsync_mode;			/* fsync policy */
 	bool test_dummy_encryption;	/* test dummy encryption */
+#ifdef CONFIG_FS_ENCRYPTION
+	bool inlinecrypt;		/* inline encryption enabled */
+#endif
 	block_t unusable_cap;		/* Amount of space allowed to be
 					 * unusable when disabling checkpoint
 					 */

This bool is unused now.

fs/f2fs/f2fs.h static void default_options(struct f2fs_sb_info *sbi) · line 1582

Quoted 7 lines
fs/f2fs/f2fs.h +3 -0
@@ -1568,6 +1577,9 @@ static void default_options(struct f2fs_sb_info *sbi)
 	F2FS_OPTION(sbi).alloc_mode = ALLOC_MODE_DEFAULT;
 	F2FS_OPTION(sbi).fsync_mode = FSYNC_MODE_POSIX;
 	F2FS_OPTION(sbi).test_dummy_encryption = false;
+#ifdef CONFIG_FS_ENCRYPTION
+	sbi->sb->s_flags &= ~SB_INLINECRYPT;
+#endif

This really should be CONFIG_FS_ENCRYPTION_INLINE_CRYPT, but actually there's no need for the #ifdef at all. Just clear the flag unconditionally.

- Eric

#21 lore raw

PATCH v8 11/11 ext4: add inline encryption support

In reply to Satya Tangirala

From: Eric Biggers <[email protected]>

Wire up ext4 to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:

- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.

- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.

Note: submit_bh_wbc() in fs/buffer.c also needed to be patched for this part, since ext4 sometimes uses ll_rw_block() on file data.

- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.

- Not doing filesystem-layer crypto on inline-encrypted files.

---
 Documentation/admin-guide/ext4.rst |  6 ++++++
 fs/buffer.c                        |  7 ++++---
 fs/ext4/inode.c                    |  4 ++--
 fs/ext4/page-io.c                  |  6 ++++--
 fs/ext4/readpage.c                 | 11 ++++++++---
 fs/ext4/super.c                    |  9 +++++++++
 6 files changed, 33 insertions(+), 10 deletions(-)
Documentation/admin-guide/ext4.rst +6 -0
index 9443fcef1876..ed997e376678 100644
@@ -395,6 +395,12 @@ When mounting an ext4 filesystem, the following option are accepted:
         Documentation/filesystems/dax.txt.  Note that this option is
         incompatible with data=journal.
 
+  inlinecrypt
+        Encrypt/decrypt the contents of encrypted files using the blk-crypto
+        framework rather than filesystem-layer encryption. This allows the use
+        of inline encryption hardware. The on-disk format is unaffected. For
+        more details, see Documentation/block/inline-encryption.rst.
+
 Data Mode
 =========
 There are 3 different data modes:
fs/buffer.c +4 -3
index b8d28370cfd7..226f1784eda7 100644
@@ -331,9 +331,8 @@ static void decrypt_bh(struct work_struct *work)
 static void end_buffer_async_read_io(struct buffer_head *bh, int uptodate)
 {
 	/* Decrypt if needed */
-	if (uptodate && IS_ENABLED(CONFIG_FS_ENCRYPTION) &&
-	    IS_ENCRYPTED(bh->b_page->mapping->host) &&
-	    S_ISREG(bh->b_page->mapping->host->i_mode)) {
+	if (uptodate &&
+	    fscrypt_inode_uses_fs_layer_crypto(bh->b_page->mapping->host)) {
 		struct decrypt_bh_ctx *ctx = kmalloc(sizeof(*ctx), GFP_ATOMIC);
 
 		if (ctx) {
@@ -3085,6 +3084,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
 	 */
 	bio = bio_alloc(GFP_NOIO, 1);
 
+	fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
+
 	bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
 	bio_set_dev(bio, bh->b_bdev);
 	bio->bi_write_hint = write_hint;
fs/ext4/inode.c +2 -2
index fa0ff78dc033..5714ff6395a1 100644
@@ -1089,7 +1089,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len,
 	}
 	if (unlikely(err)) {
 		page_zero_new_buffers(page, from, to);
-	} else if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode)) {
+	} else if (fscrypt_inode_uses_fs_layer_crypto(inode)) {
 		for (i = 0; i < nr_wait; i++) {
 			int err2;
 
@@ -3720,7 +3720,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,
 		/* Uhhuh. Read error. Complain and punt. */
 		if (!buffer_uptodate(bh))
 			goto unlock;
-		if (S_ISREG(inode->i_mode) && IS_ENCRYPTED(inode)) {
+		if (fscrypt_inode_uses_fs_layer_crypto(inode)) {
 			/* We expect the key to be set. */
 			BUG_ON(!fscrypt_has_encryption_key(inode));
 			err = fscrypt_decrypt_pagecache_blocks(page, blocksize,
fs/ext4/page-io.c +4 -2
index 68b39e75446a..0757145a31b2 100644
@@ -404,6 +404,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,
 	 * __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset().
 	 */
 	bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES);
+	fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
 	bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
 	bio_set_dev(bio, bh->b_bdev);
 	bio->bi_end_io = ext4_end_bio;
@@ -420,7 +421,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io,
 {
 	int ret;
 
-	if (io->io_bio && bh->b_blocknr != io->io_next_block) {
+	if (io->io_bio && (bh->b_blocknr != io->io_next_block ||
+			   !fscrypt_mergeable_bio_bh(io->io_bio, bh))) {
 submit_and_retry:
 		ext4_io_submit(io);
 	}
@@ -508,7 +510,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,
 	 * (e.g. holes) to be unnecessarily encrypted, but this is rare and
 	 * can't happen in the common case of blocksize == PAGE_SIZE.
 	 */
-	if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) && nr_to_submit) {
+	if (fscrypt_inode_uses_fs_layer_crypto(inode) && nr_to_submit) {
 		gfp_t gfp_flags = GFP_NOFS;
 		unsigned int enc_bytes = round_up(len, i_blocksize(inode));
 
fs/ext4/readpage.c +8 -3
index c1769afbf799..68eac0aeffad 100644
@@ -195,7 +195,7 @@ static void ext4_set_bio_post_read_ctx(struct bio *bio,
 {
 	unsigned int post_read_steps = 0;
 
-	if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode))
+	if (fscrypt_inode_uses_fs_layer_crypto(inode))
 		post_read_steps |= 1 << STEP_DECRYPT;
 
 	if (ext4_need_verity(inode, first_idx))
@@ -232,6 +232,7 @@ int ext4_mpage_readpages(struct address_space *mapping,
 	const unsigned blkbits = inode->i_blkbits;
 	const unsigned blocks_per_page = PAGE_SIZE >> blkbits;
 	const unsigned blocksize = 1 << blkbits;
+	sector_t next_block;
 	sector_t block_in_file;
 	sector_t last_block;
 	sector_t last_block_in_file;
@@ -264,7 +265,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 		if (page_has_buffers(page))
 			goto confused;
 
-		block_in_file = (sector_t)page->index << (PAGE_SHIFT - blkbits);
+		block_in_file = next_block =
+			(sector_t)page->index << (PAGE_SHIFT - blkbits);
 		last_block = block_in_file + nr_pages * blocks_per_page;
 		last_block_in_file = (ext4_readpage_limit(inode) +
 				      blocksize - 1) >> blkbits;
@@ -364,7 +366,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 		 * This page will go to BIO.  Do we need to send this
 		 * BIO off first?
 		 */
-		if (bio && (last_block_in_bio != blocks[0] - 1)) {
+		if (bio && (last_block_in_bio != blocks[0] - 1 ||
+			    !fscrypt_mergeable_bio(bio, inode, next_block))) {
 		submit_and_realloc:
 			submit_bio(bio);
 			bio = NULL;
@@ -376,6 +379,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 			 */
 			bio = bio_alloc(GFP_KERNEL,
 				min_t(int, nr_pages, BIO_MAX_PAGES));
+			fscrypt_set_bio_crypt_ctx(bio, inode, next_block,
+						  GFP_KERNEL);
 			ext4_set_bio_post_read_ctx(bio, inode, page->index);
 			bio_set_dev(bio, bdev);
 			bio->bi_iter.bi_sector = blocks[0] << (blkbits - 9);
fs/ext4/super.c +9 -0
index 0c7c4adb664e..2054da2ddde4 100644
@@ -1514,6 +1514,7 @@ enum {
 	Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit,
 	Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
 	Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption,
+	Opt_inlinecrypt,
 	Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
 	Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota,
 	Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,
@@ -1611,6 +1612,7 @@ static const match_table_t tokens = {
 	{Opt_noinit_itable, "noinit_itable"},
 	{Opt_max_dir_size_kb, "max_dir_size_kb=%u"},
 	{Opt_test_dummy_encryption, "test_dummy_encryption"},
+	{Opt_inlinecrypt, "inlinecrypt"},
 	{Opt_nombcache, "nombcache"},
 	{Opt_nombcache, "no_mbcache"},	/* for backward compatibility */
 	{Opt_removed, "check=none"},	/* mount option from ext2/3 */
@@ -1898,6 +1900,13 @@ static int handle_mount_opt(struct super_block *sb, char *opt, int token,
 	case Opt_nolazytime:
 		sb->s_flags &= ~SB_LAZYTIME;
 		return 1;
+	case Opt_inlinecrypt:
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+		sb->s_flags |= SB_INLINECRYPT;
+#else
+		ext4_msg(sb, KERN_ERR, "inline encryption not supported");
+#endif
+		return 1;
 	}
 
 	for (m = ext4_mount_opts; m->token != Opt_err; m++)
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
maintainer
#22 lore raw
In reply to Satya Tangirala
Quoted 3 lines · Satya Tangirala

This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.

This patch series can also be retrieved from

Repo: https://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git
Tag: inline-encryption-v8

I based it on v5.6-rc2 (same base commit as v7) since it still applies cleanly to it. You can see what changed by:

git diff inline-encryption-v7..inline-encryption-v8

- Eric