Skip to contentSkip to conversation

Inline Encryption Support

PATCH Superseded Satya Tangirala 5 participants 40 messages Feb 21, 2020 – Mar 5, 2020

Subsystems (8)
Documentation VFS SCSI Block layer +4 more subsystems

v7 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#0 lore raw

v7 cover Inline Encryption Support

This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.

Note that the patches in this series for the block layer (i.e. patches 1, 2 and 3) can be applied independently of the subsequent patches in this series.

Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device. Inline Encryption hardware has become increasingly common, and we want to support it in the kernel.

Inline Encryption hardware implementations often function around the concept of a limited number of "keyslots", which can hold an encryption context each. The storage device can be directed to en/decrypt any particular request with the encryption context stored in any particular keyslot.

Patch 1 introduces a Keyslot Manager to efficiently manage keyslots. The keyslot manager also functions as the interface that blk-crypto (introduced in Patch 2), will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.

Patch 2 adds the block layer changes for inline encryption support. It introduces struct bio_crypt_ctx, and a ptr to one in struct bio, which allows struct bio to represent an encryption context that can be passed down the storage stack from the filesystem layer to the storage driver.

Patch 3 introduces blk-crypto-fallback - a kernel crypto API fallback for blk-crypto to use when inline encryption hardware isn't present. This allows filesystems to specify encryption contexts for bios without having to worry about whether the underlying hardware has inline encryption support, and allows for testing without real hardware inline encryption support. This fallback is separately configurable from blk-crypto, and can be disabled if desired while keeping inline encryption support. It may also be possible to remove file content en/decryption from fscrypt and simply use blk-crypto-fallback in a future patch. For more details on blk-crypto and the fallback, refer to Documentation/block/inline-encryption.rst.

Patches 4-6 add support for inline encryption into the UFS driver according to the JEDEC UFS HCI v2.1 specification. Inline encryption support for other drivers (like eMMC) may be added in the same way - the device driver should set up a Keyslot Manager in the device's request_queue (refer to the UFS crypto additions in ufshcd-crypto.c and ufshcd.c for an example).

Patch 7 adds support to fscrypt - to use inline encryption with fscrypt, the filesystem must be mounted with '-o inlinecrypt' - when this option is specified, the contents of any AES-256-XTS encrypted file will be encrypted using blk-crypto.

Patches 8 and 9 add support to f2fs and ext4 respectively, so that we have a complete stack that can make use of inline encryption.

The patches were tested running kvm-xfstests, by specifying the introduced "inlinecrypt" mount option, so that en/decryption happens with the blk-crypto fallback. The patches were also tested on a Pixel 4 with UFS hardware that has support for inline encryption.

There have been a few patch sets addressing Inline Encryption Support in the past. Briefly, this patch set differs from those as follows:

1) "crypto: qce: ice: Add support for Inline Crypto Engine" is specific to certain hardware, while our patch set's Inline Encryption support for UFS is implemented according to the JEDEC UFS specification.

2) "scsi: ufs: UFS Host Controller crypto changes" registers inline encryption support as a kernel crypto algorithm. Our patch views inline encryption as being fundamentally different from a generic crypto provider (in that inline encryption is tied to a device), and so does not use the kernel crypto API to represent inline encryption hardware.

3) "scsi: ufs: add real time/inline crypto support to UFS HCD" requires the device mapper to work - our patch does not.

Changes v6 => v7:
- Keyslot management is now done on a per-request basis rather than a per-bio basis.
- Storage drivers can now specify the maximum number of bytes they can accept for the data unit number (DUN) for each crypto algorithm, and upper layers can specify the minimum number of bytes of DUN they want with the blk_crypto_key they send with the bio - a driver is only considered to support a blk_crypto_key if the driver supports at least as many DUN bytes as the upper layer wants. This is necessary because storage drivers may not support as many bytes as the algorithm specification dictates (for e.g. UFS only supports 8 byte DUNs for AES-256-XTS, even though the algorithm specification says DUNs are 16 bytes long).
- Introduce SB_INLINECRYPT to keep track of whether inline encryption is enabled for a filesystem (instead of using an fscrypt_operation).
- Expose keyslot manager declaration and embed it within ufs_hba to clean up code.
- Make blk-crypto preclude blk-integrity.
- Some bug fixes
- Introduce UFSHCD_QUIRK_BROKEN_CRYPTO for UFS drivers that don't support inline encryption (yet)

Changes v5 => v6:
- Blk-crypto's kernel crypto API fallback is no longer restricted to 8-byte DUNs. It's also now separately configurable from blk-crypto, and can be disabled entirely, while still allowing the kernel to use inline encryption hardware. Further, struct bio_crypt_ctx takes up less space, and no longer contains the information needed by the crypto API fallback - the fallback allocates the required memory when necessary.
- Blk-crypto now supports all file content encryption modes supported by fscrypt.
- Fixed bio merging logic in blk-merge.c
- Fscrypt now supports inline encryption with the direct key policy, since blk-crypto now has support for larger DUNs.
- Keyslot manager now uses a hashtable to lookup which keyslot contains any particular key (thanks Eric!)
- Fscrypt support for inline encryption now handles filesystems with multiple underlying block devices (thanks Eric!)
- Numerous cleanups

Changes v4 => v5:
- The fscrypt patch has been separated into 2. The first adds support for the IV_INO_LBLK_64 policy (which was called INLINE_CRYPT_OPTIMIZED in past versions of this series). This policy is now purely an on disk format, and doesn't dictate whether blk-crypto is used for file content encryption or not. Instead, this is now decided based on the "inlinecrypt" mount option.
- Inline crypto key eviction is now handled by blk-crypto instead of fscrypt.
- More refactoring.

Changes v3 => v4:
- Fixed the issue with allocating crypto_skcipher in blk_crypto_keyslot_program.
- bio_crypto_alloc_ctx is now mempool backed.
- In f2fs, a bio's bi_crypt_context is now set up when the bio is allocated, rather than just before the bio is submitted - this fixes bugs in certain cases, like when an encrypted block is being moved without decryption.
- Lots of refactoring and cleanup of blk-crypto - thanks Eric!

Changes v2 => v3:
- Overhauled keyslot manager's get keyslot logic and optimized LRU.
- Block crypto en/decryption fallback now supports data unit sizes that divide the bvec length (instead of requiring each bvec's length to be the same as the data unit size).
- fscrypt master key is now keyed additionally by super_block and ci_ctfm != NULL.
- all references of "hw encryption" are replaced by inline encryption.
- address various other review comments from Eric.

Changes v1 => v2:
- Block layer and UFS changes are split into 3 patches each.
- We now only have a ptr to a struct bio_crypt_ctx in struct bio, instead of the struct itself.
- struct bio_crypt_ctx no longer has flags.
- blk-crypto now correctly handles the case when it fails to init (because of insufficient memory), but kernel continues to boot.
- ufshcd-crypto now works on big endian cpus.
- Many cleanups.

Eric Biggers (1):
ext4: add inline encryption support

Satya Tangirala (8):
block: Keyslot Manager for Inline Encryption
block: Inline encryption support for blk-mq
block: blk-crypto-fallback for Inline Encryption
scsi: ufs: UFS driver v2.1 spec crypto additions
scsi: ufs: UFS crypto API
scsi: ufs: Add inline encryption support to UFS
fscrypt: add inline encryption support
f2fs: add inline encryption support

 Documentation/admin-guide/ext4.rst        |   4 +
 Documentation/block/index.rst             |   1 +
 Documentation/block/inline-encryption.rst | 162 ++++++
 Documentation/filesystems/f2fs.txt        |   4 +
 block/Kconfig                             |  17 +
 block/Makefile                            |   2 +
 block/bio-integrity.c                     |   5 +
 block/bio.c                               |   7 +-
 block/blk-core.c                          |  16 +
 block/blk-crypto-fallback.c               | 673 ++++++++++++++++++++++
 block/blk-crypto-internal.h               |  51 ++
 block/blk-crypto.c                        | 441 ++++++++++++++
 block/blk-map.c                           |   1 +
 block/blk-merge.c                         |  11 +
 block/blk-mq.c                            |  16 +
 block/blk.h                               |   3 +
 block/bounce.c                            |   2 +
 block/keyslot-manager.c                   | 426 ++++++++++++++
 drivers/md/dm.c                           |   2 +
 drivers/scsi/ufs/Kconfig                  |   9 +
 drivers/scsi/ufs/Makefile                 |   1 +
 drivers/scsi/ufs/ufs-hisi.c               |   8 +
 drivers/scsi/ufs/ufs-qcom.c               |   7 +
 drivers/scsi/ufs/ufshcd-crypto.c          | 393 +++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h          |  68 +++
 drivers/scsi/ufs/ufshcd.c                 |  61 +-
 drivers/scsi/ufs/ufshcd.h                 |  33 ++
 drivers/scsi/ufs/ufshci.h                 |  67 ++-
 fs/buffer.c                               |   7 +-
 fs/crypto/Kconfig                         |   6 +
 fs/crypto/Makefile                        |   1 +
 fs/crypto/bio.c                           |  55 ++
 fs/crypto/crypto.c                        |   2 +-
 fs/crypto/fname.c                         |   4 +-
 fs/crypto/fscrypt_private.h               | 121 +++-
 fs/crypto/inline_crypt.c                  | 324 +++++++++++
 fs/crypto/keyring.c                       |   4 +-
 fs/crypto/keysetup.c                      |  95 ++-
 fs/crypto/keysetup_v1.c                   |  16 +-
 fs/ext4/ext4.h                            |   1 +
 fs/ext4/inode.c                           |   4 +-
 fs/ext4/page-io.c                         |   6 +-
 fs/ext4/readpage.c                        |  11 +-
 fs/ext4/super.c                           |  19 +
 fs/f2fs/compress.c                        |   2 +-
 fs/f2fs/data.c                            |  67 ++-
 fs/f2fs/f2fs.h                            |   3 +
 fs/f2fs/super.c                           |  35 ++
 include/linux/bio.h                       |   1 +
 include/linux/blk-crypto.h                | 271 +++++++++
 include/linux/blk_types.h                 |  12 +
 include/linux/blkdev.h                    |  16 +
 include/linux/fs.h                        |   1 +
 include/linux/fscrypt.h                   |  57 ++
 include/linux/keyslot-manager.h           | 108 ++++
 55 files changed, 3654 insertions(+), 86 deletions(-)
 create mode 100644 Documentation/block/inline-encryption.rst
 create mode 100644 block/blk-crypto-fallback.c
 create mode 100644 block/blk-crypto-internal.h
 create mode 100644 block/blk-crypto.c
 create mode 100644 block/keyslot-manager.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
 create mode 100644 fs/crypto/inline_crypt.c
 create mode 100644 include/linux/blk-crypto.h
 create mode 100644 include/linux/keyslot-manager.h
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#1 lore raw

PATCH v7 1/9 block: Keyslot Manager for Inline Encryption

Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.

Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold an encryption context (we say that an encryption context can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the encryption context programmed into that associated keyslot.

As keyslots are limited, and programming keys may be expensive in many implementations, and multiple requests may use exactly the same encryption contexts, we introduce a Keyslot Manager to efficiently manage keyslots.

We also introduce a blk_crypto_key, which will represent the key that's programmed into keyslots managed by keyslot managers. The keyslot manager also functions as the interface that upper layers will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.

---
 block/Kconfig                   |   7 +
 block/Makefile                  |   1 +
 block/keyslot-manager.c         | 426 ++++++++++++++++++++++++++++++++
 include/linux/bio.h             |   1 +
 include/linux/blk-crypto.h      |  45 ++++
 include/linux/blkdev.h          |   6 +
 include/linux/keyslot-manager.h | 108 ++++++++
 7 files changed, 594 insertions(+)
 create mode 100644 block/keyslot-manager.c
 create mode 100644 include/linux/blk-crypto.h
 create mode 100644 include/linux/keyslot-manager.h
block/Kconfig +7 -0
index 3bc76bb113a0..c04a1d500842 100644
@@ -185,6 +185,13 @@ config BLK_SED_OPAL
 	Enabling this option enables users to setup/unlock/lock
 	Locking ranges for SED devices using the Opal protocol.
 
+config BLK_INLINE_ENCRYPTION
+	bool "Enable inline encryption support in block layer"
+	help
+	  Build the blk-crypto subsystem. Enabling this lets the
+	  block layer handle encryption, so users can take
+	  advantage of inline encryption hardware if present.
+
 menu "Partition Types"
 
 source "block/partitions/Kconfig"
block/Makefile +1 -0
index 1a43750f4b01..ef3a05dcf1f2 100644
@@ -37,3 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS)	+= blk-mq-debugfs.o
 obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o
block/keyslot-manager.c new file +426 -0

426 lines not on this page · raw message

include/linux/bio.h +1 -0
index 853d92ceee64..b2103e207ed5 100644
@@ -8,6 +8,7 @@
 #include <linux/highmem.h>
 #include <linux/mempool.h>
 #include <linux/ioprio.h>
+#include <linux/blk-crypto.h>
 
 #ifdef CONFIG_BLOCK
 /* struct bio, bio_vec and BIO_* flags are defined in blk_types.h */
include/linux/blk-crypto.h new file +45 -0

45 lines not on this page · raw message

include/linux/blkdev.h +6 -0
index 053ea4b51988..8881b25ef58b 100644
@@ -43,6 +43,7 @@ struct pr_ops;
 struct rq_qos;
 struct blk_queue_stats;
 struct blk_stat_callback;
+struct keyslot_manager;
 
 #define BLKDEV_MIN_RQ	4
 #define BLKDEV_MAX_RQ	128	/* Default maximum */
@@ -474,6 +475,11 @@ struct request_queue {
 	unsigned int		dma_pad_mask;
 	unsigned int		dma_alignment;
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	/* Inline crypto capabilities */
+	struct keyslot_manager *ksm;
+#endif
+
 	unsigned int		rq_timeout;
 	int			poll_nsec;
 
include/linux/keyslot-manager.h new file +108 -0

108 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#2 lore raw

Re: [PATCH v7 1/9] block: Keyslot Manager for Inline Encryption

Quoted 23 lines
block/keyslot-manager.c new file +23 -0
+#ifdef CONFIG_PM
+static inline void blk_ksm_set_dev(struct keyslot_manager *ksm,
+				   struct device *dev)
+{
+	ksm->dev = dev;
+}
+
+/* If there's an underlying device and it's suspended, resume it. */
+static inline void blk_ksm_pm_get(struct keyslot_manager *ksm)
+{
+	if (ksm->dev)
+		pm_runtime_get_sync(ksm->dev);
+}
+
+static inline void blk_ksm_pm_put(struct keyslot_manager *ksm)
+{
+	if (ksm->dev)
+		pm_runtime_put_sync(ksm->dev);
+}
+#else /* CONFIG_PM */
+static inline void blk_ksm_set_dev(struct keyslot_manager *ksm,
+				   struct device *dev)
+{

block/keyslot-manager.c line 75

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+}
+
+static inline void blk_ksm_pm_get(struct keyslot_manager *ksm)
+{
+}
+
+static inline void blk_ksm_pm_put(struct keyslot_manager *ksm)
+{
+}
+#endif /* !CONFIG_PM */

I think no one is hurt by an unused dev field for the non-pm case. I'd suggest to define the field unconditionally, and replace all the above with direct calls below.

block/keyslot-manager.c line 290

Quoted 13 lines
block/keyslot-manager.c new file +13 -0
+/**
+ * blk_ksm_get_slot() - Increment the refcount on the specified slot.
+ * @ksm: The keyslot manager that we want to modify.
+ * @slot: The slot to increment the refcount of.
+ *
+ * This function assumes that there is already an active reference to that slot
+ * and simply increments the refcount. This is useful when cloning a bio that
+ * already has a reference to a keyslot, and we want the cloned bio to also have
+ * its own reference.
+ *
+ * Context: Any context.
+ */
+void blk_ksm_get_slot(struct keyslot_manager *ksm, unsigned int slot)

This function doesn't appear to be used at all in the whole series.

Quoted 11 lines
block/keyslot-manager.c new file +11 -0
+/**
+ * blk_ksm_put_slot() - Release a reference to a slot
+ * @ksm: The keyslot manager to release the reference from.
+ * @slot: The slot to release the reference from.
+ *
+ * Context: Any context.
+ */
+void blk_ksm_put_slot(struct keyslot_manager *ksm, unsigned int slot)
+{
+	unsigned long flags;
+

block/keyslot-manager.c line 318

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+	if (WARN_ON(slot >= ksm->num_slots))
+		return;
+
+	if (atomic_dec_and_lock_irqsave(&ksm->slots[slot].slot_refs,
+					&ksm->idle_slots_lock, flags)) {
+		list_add_tail(&ksm->slots[slot].idle_slot_node,
+			      &ksm->idle_slots);
+		spin_unlock_irqrestore(&ksm->idle_slots_lock, flags);
+		wake_up(&ksm->idle_slots_wait_queue);
+	}

Given that blk_ksm_get_slot_for_key returns a signed keyslot that can return errors, and the only callers stores it in a signed variable I think this function should take a signed slot as well, and the check for a non-negative slot should be moved here from the only caller.

include/linux/bio.h line 14

Quoted 12 lines
include/linux/bio.h +1 -0
index 853d92ceee64..b2103e207ed5 100644
@@ -8,6 +8,7 @@
 #include <linux/highmem.h>
 #include <linux/mempool.h>
 #include <linux/ioprio.h>
+#include <linux/blk-crypto.h>
 
 #ifdef CONFIG_BLOCK
 /* struct bio, bio_vec and BIO_* flags are defined in blk_types.h */

This doesn't belong here, but into the patch that actually requires crypto definitions in bio.h.

#3 lore raw

Re: [PATCH v7 1/9] block: Keyslot Manager for Inline Encryption

Quoted 4 lines · Christoph Hellwig

Given that blk_ksm_get_slot_for_key returns a signed keyslot that can return errors, and the only callers stores it in a signed variable I think this function should take a signed slot as well, and the check for a non-negative slot should be moved here from the only caller.

Actually looking over the code again I think it might be better to return only the error code (and that might actually be a blk_status_t), and then use an argument to return a pointer to the actual struct keyslot. That gives us much easier to understand code and better type safety.

maintainer
#4 lore raw

Re: [PATCH v7 1/9] block: Keyslot Manager for Inline Encryption

Quoted 11 lines · Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

Given that blk_ksm_get_slot_for_key returns a signed keyslot that can return errors, and the only callers stores it in a signed variable I think this function should take a signed slot as well, and the check for a non-negative slot should be moved here from the only caller.

Actually looking over the code again I think it might be better to return only the error code (and that might actually be a blk_status_t), and then use an argument to return a pointer to the actual struct keyslot. That gives us much easier to understand code and better type safety.

That doesn't make sense because the caller only cares about the keyslot number, not the 'struct keyslot'. The 'struct keyslot' is internal to keyslot-manager.c, as it only contains keyslot management information.

Your earlier suggestion of making blk_ksm_put_slot() be a no-op on a negative keyslot number sounds fine though.

- Eric

#5 lore raw

Re: [PATCH v7 1/9] block: Keyslot Manager for Inline Encryption

Quoted 13 lines · Eric Biggers
Quoted 11 lines · Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

Given that blk_ksm_get_slot_for_key returns a signed keyslot that can return errors, and the only callers stores it in a signed variable I think this function should take a signed slot as well, and the check for a non-negative slot should be moved here from the only caller.

Actually looking over the code again I think it might be better to return only the error code (and that might actually be a blk_status_t), and then use an argument to return a pointer to the actual struct keyslot. That gives us much easier to understand code and better type safety.

Quoted 4 lines

That doesn't make sense because the caller only cares about the keyslot number, not the 'struct keyslot'. The 'struct keyslot' is internal to keyslot-manager.c, as it only contains keyslot management information.

I think it does make some sense at least to make the keyslot type opaque to most of the system other than the driver itself (the driver will now have to call a function like blk_ksm_slot_idx_for_keyslot to actually get a keyslot number at the end of the day). Also this way, the keyslot manager can verify that the keyslot passed to blk_ksm_put_slot is actually part of that keyslot manager (and that somebody isn't releasing a slot number that was actually acquired from a different keyslot manager). I don't think it's much benefit or loss either way, but I already switched to passing pointers to struct keyslot around instead of ints, so I'll keep it that way unless you strongly feel that using ints in this case is better than struct keyslot *.

Quoted 4 lines

Your earlier suggestion of making blk_ksm_put_slot() be a no-op on a negative keyslot number sounds fine though.

- Eric

#6 lore raw

Re: [PATCH v7 1/9] block: Keyslot Manager for Inline Encryption

Quoted 11 lines · Satya Tangirala

I think it does make some sense at least to make the keyslot type opaque to most of the system other than the driver itself (the driver will now have to call a function like blk_ksm_slot_idx_for_keyslot to actually get a keyslot number at the end of the day). Also this way, the keyslot manager can verify that the keyslot passed to blk_ksm_put_slot is actually part of that keyslot manager (and that somebody isn't releasing a slot number that was actually acquired from a different keyslot manager). I don't think it's much benefit or loss either way, but I already switched to passing pointers to struct keyslot around instead of ints, so I'll keep it that way unless you strongly feel that using ints in this case is better than struct keyslot *.

Exactly. This provides a little type safety.

maintainer
#7 lore raw

Re: [PATCH v7 1/9] block: Keyslot Manager for Inline Encryption

In reply to Satya Tangirala
Quoted 7 lines · Satya Tangirala

Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.

Quoted 7 lines

Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold an encryption context (we say that an encryption context can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the encryption context programmed into that associated keyslot.

The second paragraph uses the phrase "encryption context" differently from the first. In the first it's (key, alg, dun, dusize) while in the second it's (key, alg, dusize). Maybe simply say "key" in the second? "Key" is already used as shorthand in lots of places without specifying that it really means also the (alg, dusize), like in "keyslot" and "keyslot manager".

This inconsistency shows up in some other places in the patchset too.

Quoted 5 lines
block/keyslot-manager.c new file +0 -0
new file mode 100644
index 000000000000..15dfc0c12c7f

[...]

Quoted 15 lines
block/keyslot-manager.c new file +15 -0
+/**
+ * blk_ksm_init() - Initialize a keyslot manager
+ * @ksm: The keyslot_manager to initialize.
+ * @dev: Device for runtime power management (NULL if none)
+ * @num_slots: The number of key slots to manage.
+ *
+ * Allocate memory for keyslots and initialize a keyslot manager. Called by
+ * e.g. storage drivers to set up a keyslot manager in their request_queue.
+ *
+ * Return: 0 on success, or else a negative error code.
+ */
+int blk_ksm_init(struct keyslot_manager *ksm, struct device *dev,
+		 unsigned int num_slots)
+{
+	unsigned int slot;

block/keyslot-manager.c line 118

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+	unsigned int i;
+
+	memset(ksm, 0, sizeof(*ksm));
+
+	if (num_slots == 0)
+		return -EINVAL;
+
+	ksm->slots = kvzalloc(sizeof(ksm->slots[0]) * num_slots, GFP_KERNEL);
+	if (!ksm->slots)
+		return -ENOMEM;

This should use kvcalloc() so that an integer overflow check is included.

Quoted 22 lines
block/keyslot-manager.c new file +22 -0
+
+	ksm->num_slots = num_slots;
+	blk_ksm_set_dev(ksm, dev);
+
+	init_rwsem(&ksm->lock);
+
+	init_waitqueue_head(&ksm->idle_slots_wait_queue);
+	INIT_LIST_HEAD(&ksm->idle_slots);
+
+	for (slot = 0; slot < num_slots; slot++) {
+		list_add_tail(&ksm->slots[slot].idle_slot_node,
+			      &ksm->idle_slots);
+	}
+
+	spin_lock_init(&ksm->idle_slots_lock);
+
+	ksm->slot_hashtable_size = roundup_pow_of_two(num_slots);
+	ksm->slot_hashtable = kvmalloc_array(ksm->slot_hashtable_size,
+					     sizeof(ksm->slot_hashtable[0]),
+					     GFP_KERNEL);
+	if (!ksm->slot_hashtable)
+		goto err_free_ksm;

block/keyslot-manager.c line 150

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+	for (i = 0; i < ksm->slot_hashtable_size; i++)
+		INIT_HLIST_HEAD(&ksm->slot_hashtable[i]);
+
+	return 0;
+
+err_free_ksm:
+	blk_ksm_destroy(ksm);
+	return -ENOMEM;
+}
+EXPORT_SYMBOL_GPL(blk_ksm_init);

The label should be called 'err_destroy_ksm' now that it's not actually freeing the memory of the 'struct keyslot_manager' itself.

Quoted 17 lines
block/keyslot-manager.c new file +17 -0
+/**
+ * blk_ksm_crypto_mode_supported() - Find out if a crypto_mode/data unit size
+ *				     combination is supported by a ksm.
+ * @ksm: The keyslot manager to check
+ * @crypto_mode: The crypto mode to check for.
+ * @blk_crypto_dun_bytes: The minimum number of bytes needed for specifying DUNs
+ * @data_unit_size: The data_unit_size for the mode.
+ *
+ * Checks for crypto_mode/data unit size support.
+ *
+ * Return: Whether or not this ksm supports the specified crypto_mode/
+ *	   data_unit_size combo.
+ */
+bool blk_ksm_crypto_mode_supported(struct keyslot_manager *ksm,
+				   const struct blk_crypto_key *key)
+{
+	if (!ksm)

block/keyslot-manager.c line 347

Quoted 10 lines
block/keyslot-manager.c new file +10 -0
+		return false;
+	if (WARN_ON((unsigned int)key->crypto_mode >= BLK_ENCRYPTION_MODE_MAX))
+		return false;
+	if (WARN_ON(!is_power_of_2(key->data_unit_size)))
+		return false;
+	return (ksm->crypto_modes_supported[key->crypto_mode] &
+		key->data_unit_size) &&
+	       (ksm->max_dun_bytes_supported[key->crypto_mode] >=
+		key->dun_bytes);
+}

The kerneldoc for this function is outdated now that it was changed to take in a 'struct blk_crypto_key'.

Also, since the crypto_mode and data_unit_size fields of 'struct blk_crypto_key' are already validated by blk_crypto_init_key(), they don't really need to validated again here anymore.

block/keyslot-manager.c line 426

Quoted 9 lines
block/keyslot-manager.c new file +9 -0
+void blk_ksm_destroy(struct keyslot_manager *ksm)
+{
+	if (!ksm)
+		return;
+	kvfree(ksm->slot_hashtable);
+	kvfree(ksm->slots);
+	memzero_explicit(ksm, sizeof(*ksm));
+}
+EXPORT_SYMBOL_GPL(blk_ksm_destroy);

In v6 => v7, zeroing the keys was lost. We need:

	memzero_explicit(ksm->slots, sizeof(ksm->slots[0]) * ksm->num_slots);
Quoted 23 lines
include/linux/blk-crypto.h new file +17 -0
new file mode 100644
index 000000000000..b8d54eca1c0d
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright 2019 Google LLC
+ */
+
+#ifndef __LINUX_BLK_CRYPTO_H
+#define __LINUX_BLK_CRYPTO_H
+
+enum blk_crypto_mode_num {
+	BLK_ENCRYPTION_MODE_INVALID,
+	BLK_ENCRYPTION_MODE_AES_256_XTS,
+	BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV,
+	BLK_ENCRYPTION_MODE_ADIANTUM,
+	BLK_ENCRYPTION_MODE_MAX,
+};
+
+#define BLK_CRYPTO_MAX_KEY_SIZE		64

include/linux/blk-crypto.h line 27

Quoted 10 lines
include/linux/blk-crypto.h new file +10 -0
+
+/**
+ * struct blk_crypto_key - an inline encryption key
+ * @crypto_mode: encryption algorithm this key is for
+ * @data_unit_size: the data unit size for all encryption/decryptions with this
+ *	key.  This is the size in bytes of each individual plaintext and
+ *	ciphertext.  This is always a power of 2.  It might be e.g. the
+ *	filesystem block size or the disk sector size.
+ * @data_unit_size_bits: log2 of data_unit_size
+ * @dun_bytes: the number of bytes of DUN used when using this key

Shouldn't @dun_bytes be:

the maximum number of bytes of DUN needed when using this key

Quoted 42 lines
include/linux/blk-crypto.h new file +42 -0
+struct keyslot_manager {
+	unsigned int num_slots;
+
+	/*
+	 * The struct keyslot_mgmt_ll_ops that this keyslot manager will use
+	 * to perform operations like programming and evicting keys on the
+	 * device
+	 */
+	struct keyslot_mgmt_ll_ops ksm_ll_ops;
+
+	/*
+	 * Array of size BLK_ENCRYPTION_MODE_MAX of bitmasks that represents
+	 * whether a crypto mode and data unit size are supported. The i'th
+	 * bit of crypto_mode_supported[crypto_mode] is set iff a data unit
+	 * size of (1 << i) is supported. We only support data unit sizes
+	 * that are powers of 2.
+	 */
+	unsigned int crypto_modes_supported[BLK_ENCRYPTION_MODE_MAX];
+	/*
+	 * Array of size BLK_ENCRYPTION_MODE_MAX. The i'th entry specifies the
+	 * maximum number of dun bytes supported by the i'th crypto mode.
+	 */
+	unsigned int max_dun_bytes_supported[BLK_ENCRYPTION_MODE_MAX];
+
+	/* Private data unused by keyslot_manager. */
+	void *ll_priv_data;
+
+#ifdef CONFIG_PM
+	/* Device for runtime power management (NULL if none) */
+	struct device *dev;
+#endif
+
+	/* Protects programming and evicting keys from the device */
+	struct rw_semaphore lock;
+
+	/* List of idle slots, with least recently used slot at front */
+	wait_queue_head_t idle_slots_wait_queue;
+	struct list_head idle_slots;
+	spinlock_t idle_slots_lock;
+
+	/*
+	 * Hash table which maps key hashes to keyslots, so that we can find a

include/linux/blk-crypto.h

Quoted 10 lines
include/linux/blk-crypto.h new file +10 -0
+	 * key's keyslot in O(1) time rather than O(num_slots).  Protected by
+	 * 'lock'.  A cryptographic hash function is used so that timing attacks
+	 * can't leak information about the raw keys.
+	 */
+	struct hlist_head *slot_hashtable;
+	unsigned int slot_hashtable_size;
+
+	/* Per-keyslot data */
+	struct keyslot *slots;
+};

Now that struct keyslot_manager is exposed to everyone, it would be helpful if the comments and field order made the fields clearly divided into two parts: one part that's public, and one part that's for keyslot manager use only.

- Eric

#8 lore raw

PATCH v7 2/9 block: Inline encryption support for blk-mq

In reply to Satya Tangirala

We must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the upper layers (like the filesystem/fscrypt) that knows about and manages encryption contexts. As such, when the upper layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.

We want to communicate the encryption context from the upper layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct bio_crypt_ctx to struct bio, which can represent an encryption context (note that we can't use the bi_private field in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate the bio_crypt_ctx and make the bio/request merging logic aware of the bio_crypt_ctx.

We also make changes to blk-mq to make it handle bios with encryption contexts. blk-mq can merge many bios into the same request. These bios need to have contiguous data unit numbers (the necessary changes to blk-merge are also made to ensure this) - as such, it suffices to keep the data unit number of just the first bio, since that's all a storage driver needs to infer the data unit number to use for each data block in each bio in a request. blk-mq keeps track of the encryption context to be used for all the bios in a request with the request's rq_crypt_ctx. When the first bio is added to an empty request, blk-mq will program the encryption context of that bio into the request_queue's keyslot manager, and store the returned keyslot in the request's rq_crypt_ctx. All the functions to operate on encryption contexts are in blk-crypto.c.

For now, blk-crypto and blk-integrity can't be used with each other (and any attempts to do both on a bio will cause the bio to fail).

Upper layers only need to call bio_crypt_set_ctx with the encryption key, algorithm and data_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-mq/blk-crypto handles that. Blk-crypto also makes it possible for request-based layered devices like dm-rq to make use of inline encryption hardware by cloning the rq_crypt_ctx and programming a keyslot in the new request_queue when necessary.

Note that any user of the block layer can submit bios with an encryption context, such as filesystems, device-mapper targets, etc.

---
 block/Makefile              |   2 +-
 block/bio-integrity.c       |   5 +
 block/bio.c                 |   7 +-
 block/blk-core.c            |  16 ++
 block/blk-crypto-internal.h |  19 ++
 block/blk-crypto.c          | 412 ++++++++++++++++++++++++++++++++++++
 block/blk-map.c             |   1 +
 block/blk-merge.c           |  11 +
 block/blk-mq.c              |  16 ++
 block/blk.h                 |   3 +
 block/bounce.c              |   2 +
 drivers/md/dm.c             |   2 +
 include/linux/blk-crypto.h  | 211 ++++++++++++++++++
 include/linux/blk_types.h   |   6 +
 include/linux/blkdev.h      |  10 +
 15 files changed, 721 insertions(+), 2 deletions(-)
 create mode 100644 block/blk-crypto-internal.h
 create mode 100644 block/blk-crypto.c
block/Makefile +1 -1
index ef3a05dcf1f2..82f42ca3f769 100644
@@ -37,4 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS)	+= blk-mq-debugfs.o
 obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
-obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o blk-crypto.o
block/bio-integrity.c +5 -0
index bf62c25cde8f..bce563031e7c 100644
@@ -42,6 +42,11 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,
 	struct bio_set *bs = bio->bi_pool;
 	unsigned inline_vecs;
 
+	if (bio_has_crypt_ctx(bio)) {
+		pr_warn("blk-integrity can't be used together with blk-crypto en/decryption.");
+		return ERR_PTR(-EOPNOTSUPP);
+	}
+
 	if (!bs || !mempool_initialized(&bs->bio_integrity_pool)) {
 		bip = kmalloc(struct_size(bip, bip_inline_vecs, nr_vecs), gfp_mask);
 		inline_vecs = nr_vecs;
block/bio.c +6 -1
index 94d697217887..115fd5960508 100644
@@ -17,6 +17,7 @@
 #include <linux/cgroup.h>
 #include <linux/blk-cgroup.h>
 #include <linux/highmem.h>
+#include <linux/blk-crypto.h>
 
 #include <trace/events/block.h>
 #include "blk.h"
@@ -236,6 +237,8 @@ void bio_uninit(struct bio *bio)
 
 	if (bio_integrity(bio))
 		bio_integrity_free(bio);
+
+	bio_crypt_free_ctx(bio);
 }
 EXPORT_SYMBOL(bio_uninit);
 
@@ -664,11 +667,12 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)
 
 	__bio_clone_fast(b, bio);
 
+	bio_crypt_clone(b, bio, gfp_mask);
+
 	if (bio_integrity(bio)) {
 		int ret;
 
 		ret = bio_integrity_clone(b, bio, gfp_mask);
-
 		if (ret < 0) {
 			bio_put(b);
 			return NULL;
@@ -1046,6 +1050,7 @@ void bio_advance(struct bio *bio, unsigned bytes)
 	if (bio_integrity(bio))
 		bio_integrity_advance(bio, bytes);
 
+	bio_crypt_advance(bio, bytes);
 	bio_advance_iter(bio, &bio->bi_iter, bytes);
 }
 EXPORT_SYMBOL(bio_advance);
block/blk-core.c +16 -0
index 089e890ab208..1d8f3fa5cb6c 100644
@@ -38,6 +38,7 @@
 #include <linux/debugfs.h>
 #include <linux/bpf.h>
 #include <linux/psi.h>
+#include <linux/blk-crypto.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/block.h>
@@ -120,6 +121,9 @@ void blk_rq_init(struct request_queue *q, struct request *rq)
 	rq->start_time_ns = ktime_get_ns();
 	rq->part = NULL;
 	refcount_set(&rq->ref, 1);
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	rq->rq_crypt_ctx.keyslot = -EINVAL;
+#endif
 }
 EXPORT_SYMBOL(blk_rq_init);
 
@@ -617,6 +621,8 @@ bool bio_attempt_back_merge(struct request *req, struct bio *bio,
 	req->biotail = bio;
 	req->__data_len += bio->bi_iter.bi_size;
 
+	bio_crypt_free_ctx(bio);
+
 	blk_account_io_start(req, false);
 	return true;
 }
@@ -641,6 +647,8 @@ bool bio_attempt_front_merge(struct request *req, struct bio *bio,
 	req->__sector = bio->bi_iter.bi_sector;
 	req->__data_len += bio->bi_iter.bi_size;
 
+	blk_crypto_rq_bio_prep(req, bio);
+
 	blk_account_io_start(req, false);
 	return true;
 }
@@ -1258,6 +1266,9 @@ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request *
 	    should_fail_request(&rq->rq_disk->part0, blk_rq_bytes(rq)))
 		return BLK_STS_IOERR;
 
+	if (blk_crypto_insert_cloned_request(q, rq))
+		return BLK_STS_IOERR;
+
 	if (blk_queue_io_stat(q))
 		blk_account_io_start(rq, true);
 
@@ -1646,6 +1657,8 @@ int blk_rq_prep_clone(struct request *rq, struct request *rq_src,
 
 	__blk_rq_prep_clone(rq, rq_src);
 
+	blk_crypto_rq_prep_clone(rq, rq_src);
+
 	return 0;
 
 free_and_out:
@@ -1813,5 +1826,8 @@ int __init blk_dev_init(void)
 	blk_debugfs_root = debugfs_create_dir("block", NULL);
 #endif
 
+	if (bio_crypt_ctx_init() < 0)
+		panic("Failed to allocate mem for bio crypt ctxs\n");
+
 	return 0;
 }
block/blk-crypto-internal.h new file +19 -0
new file mode 100644
index 000000000000..2452b5dee140
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright 2019 Google LLC
+ */
+
+#ifndef __LINUX_BLK_CRYPTO_INTERNAL_H
+#define __LINUX_BLK_CRYPTO_INTERNAL_H
+
+#include <linux/bio.h>
+#include <linux/blkdev.h>
+
+/* Represents a crypto mode supported by blk-crypto  */
+struct blk_crypto_mode {
+	const char *cipher_str; /* crypto API name (for fallback case) */
+	unsigned int keysize; /* key size in bytes */
+	unsigned int ivsize; /* iv size in bytes */
+};
+
+#endif /* __LINUX_BLK_CRYPTO_INTERNAL_H */
block/blk-crypto.c new file +412 -0

412 lines not on this page · raw message

block/blk-map.c +1 -0
index b0790268ed9d..4484e37d316e 100644
@@ -41,6 +41,7 @@ int blk_rq_append_bio(struct request *rq, struct bio **bio)
 		rq->biotail->bi_next = *bio;
 		rq->biotail = *bio;
 		rq->__data_len += (*bio)->bi_iter.bi_size;
+		bio_crypt_free_ctx(*bio);
 	}
 
 	return 0;
block/blk-merge.c +11 -0
index 1534ed736363..a0c24b6e0eb3 100644
@@ -596,6 +596,8 @@ int ll_back_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs)
 	if (blk_integrity_rq(req) &&
 	    integrity_req_gap_back_merge(req, bio))
 		return 0;
+	if (!bio_crypt_ctx_back_mergeable(req, bio))
+		return 0;
 	if (blk_rq_sectors(req) + bio_sectors(bio) >
 	    blk_rq_get_max_sectors(req, blk_rq_pos(req))) {
 		req_set_nomerge(req->q, req);
@@ -612,6 +614,8 @@ int ll_front_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs
 	if (blk_integrity_rq(req) &&
 	    integrity_req_gap_front_merge(req, bio))
 		return 0;
+	if (!bio_crypt_ctx_front_mergeable(req, bio))
+		return 0;
 	if (blk_rq_sectors(req) + bio_sectors(bio) >
 	    blk_rq_get_max_sectors(req, bio->bi_iter.bi_sector)) {
 		req_set_nomerge(req->q, req);
@@ -661,6 +665,9 @@ static int ll_merge_requests_fn(struct request_queue *q, struct request *req,
 	if (blk_integrity_merge_rq(q, req, next) == false)
 		return 0;
 
+	if (!bio_crypt_ctx_merge_rq(req, next))
+		return 0;
+
 	/* Merge is OK... */
 	req->nr_phys_segments = total_phys_segments;
 	return 1;
@@ -885,6 +892,10 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)
 	if (blk_integrity_merge_bio(rq->q, rq, bio) == false)
 		return false;
 
+	/* Only merge if the crypt contexts are compatible */
+	if (!bio_crypt_rq_ctx_compatible(rq, bio))
+		return false;
+
 	/* must be using the same buffer */
 	if (req_op(rq) == REQ_OP_WRITE_SAME &&
 	    !blk_write_same_mergeable(rq->bio, bio))
block/blk-mq.c +16 -0
index a12b1763508d..6b437637d029 100644
@@ -26,6 +26,7 @@
 #include <linux/delay.h>
 #include <linux/crash_dump.h>
 #include <linux/prefetch.h>
+#include <linux/blk-crypto.h>
 
 #include <trace/events/block.h>
 
@@ -316,6 +317,10 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,
 	rq->nr_phys_segments = 0;
 #if defined(CONFIG_BLK_DEV_INTEGRITY)
 	rq->nr_integrity_segments = 0;
+#endif
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	rq->rq_crypt_ctx.bc = NULL;
+	rq->rq_crypt_ctx.keyslot = -EINVAL;
 #endif
 	/* tag was already set */
 	rq->extra_len = 0;
@@ -474,6 +479,7 @@ static void __blk_mq_free_request(struct request *rq)
 	struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
 	const int sched_tag = rq->internal_tag;
 
+	blk_crypto_free_request(rq);
 	blk_pm_mark_last_busy(rq);
 	rq->mq_hctx = NULL;
 	if (rq->tag != -1)
@@ -1968,6 +1974,9 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 	unsigned int nr_segs;
 	blk_qc_t cookie;
 
+	if (blk_crypto_bio_prep(&bio))
+		return BLK_QC_T_NONE;
+
 	blk_queue_bounce(q, &bio);
 	__blk_queue_split(q, &bio, &nr_segs);
 
@@ -1998,6 +2007,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 
 	cookie = request_to_qc_t(data.hctx, rq);
 
+	if (blk_crypto_init_request(rq, q, bio)) {
+		bio->bi_status = BLK_STS_RESOURCE;
+		bio_endio(bio);
+		blk_mq_end_request(rq, BLK_STS_RESOURCE);
+		return BLK_QC_T_NONE;
+	}
+
 	blk_mq_bio_to_request(rq, bio, nr_segs);
 
 	plug = blk_mq_plug(q, bio);
block/blk.h +3 -0
index 0b8884353f6b..3929900fca02 100644
@@ -4,6 +4,7 @@
 
 #include <linux/idr.h>
 #include <linux/blk-mq.h>
+#include <linux/blk-crypto.h>
 #include <xen/xen.h>
 #include "blk-mq.h"
 #include "blk-mq-sched.h"
@@ -117,6 +118,8 @@ static inline void blk_rq_bio_prep(struct request *rq, struct bio *bio,
 
 	if (bio->bi_disk)
 		rq->rq_disk = bio->bi_disk;
+
+	blk_crypto_rq_bio_prep(rq, bio);
 }
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
block/bounce.c +2 -0
index f8ed677a1bf7..c3aaed070124 100644
@@ -267,6 +267,8 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,
 		break;
 	}
 
+	bio_crypt_clone(bio, bio_src, gfp_mask);
+
 	if (bio_integrity(bio_src)) {
 		int ret;
 
drivers/md/dm.c +2 -0
index b89f07ee2eff..e5b5ffa73619 100644
@@ -1304,6 +1304,8 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,
 
 	__bio_clone_fast(clone, bio);
 
+	bio_crypt_clone(clone, bio, GFP_NOIO);
+
 	if (bio_integrity(bio)) {
 		int r;
 
include/linux/blk-crypto.h +211 -0

221 lines not on this page · raw message

include/linux/blk_types.h +6 -0
index 70254ae11769..1996689c51d3 100644
@@ -18,6 +18,7 @@ struct block_device;
 struct io_context;
 struct cgroup_subsys_state;
 typedef void (bio_end_io_t) (struct bio *);
+struct bio_crypt_ctx;
 
 /*
  * Block error status values.  See block/blk-core:blk_errors for the details.
@@ -173,6 +174,11 @@ struct bio {
 	u64			bi_iocost_cost;
 #endif
 #endif
+
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	struct bio_crypt_ctx	*bi_crypt_context;
+#endif
+
 	union {
 #if defined(CONFIG_BLK_DEV_INTEGRITY)
 		struct bio_integrity_payload *bi_integrity; /* data integrity */
include/linux/blkdev.h +10 -0
index 8881b25ef58b..a90332299f29 100644
@@ -27,6 +27,7 @@
 #include <linux/percpu-refcount.h>
 #include <linux/scatterlist.h>
 #include <linux/blkzoned.h>
+#include <linux/blk-crypto.h>
 
 struct module;
 struct scsi_ioctl_command;
@@ -124,6 +125,11 @@ enum mq_rq_state {
 	MQ_RQ_COMPLETE		= 2,
 };
 
+struct rq_crypt_ctx {
+	struct bio_crypt_ctx *bc;
+	int keyslot;
+};
+
 /*
  * Try to put the fields that are referenced together in the same cacheline.
  *
@@ -224,6 +230,10 @@ struct request {
 	unsigned short nr_integrity_segments;
 #endif
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	struct rq_crypt_ctx rq_crypt_ctx;
+#endif
+
 	unsigned short write_hint;
 	unsigned short ioprio;
 
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#9 lore raw

Re: [PATCH v7 2/9] block: Inline encryption support for blk-mq

block/bio-integrity.c struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, · line 48

Quoted 11 lines

index bf62c25cde8f..bce563031e7c 100644

block/bio-integrity.c +4 -0
@@ -42,6 +42,11 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,
 	struct bio_set *bs = bio->bi_pool;
 	unsigned inline_vecs;
 
+	if (bio_has_crypt_ctx(bio)) {
+		pr_warn("blk-integrity can't be used together with blk-crypto en/decryption.");
+		return ERR_PTR(-EOPNOTSUPP);
+	}

What is the rationale for this limitation? Restricting unrelated features from being used together is a pretty bad design pattern and should be avoided where possible. If it can't it needs to be documented very clearly.

block/bio-integrity.c struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, · line 51

Quoted 3 lines
block/bio-integrity.c +3 -0
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	rq->rq_crypt_ctx.keyslot = -EINVAL;
+#endif

All the other core block calls to the crypto code are in helpers that are stubbed out. It might make sense to follow that style here.

block/bio-integrity.c int __init blk_dev_init(void) · line 1830

Quoted 8 lines
block/bio-integrity.c +2 -0
 free_and_out:
@@ -1813,5 +1826,8 @@ int __init blk_dev_init(void)
 	blk_debugfs_root = debugfs_create_dir("block", NULL);
 #endif
 
+	if (bio_crypt_ctx_init() < 0)
+		panic("Failed to allocate mem for bio crypt ctxs\n");

Maybe move that panic into bio_crypt_ctx_init itself?

block/bio-integrity.c int __init blk_dev_init(void)

Quoted 5 lines
block/bio-integrity.c +5 -0
+static int num_prealloc_crypt_ctxs = 128;
+
+module_param(num_prealloc_crypt_ctxs, int, 0444);
+MODULE_PARM_DESC(num_prealloc_crypt_ctxs,
+		"Number of bio crypto contexts to preallocate");

Please write a comment why this is a tunable, how the default is choosen and why someone might want to chane it.

block/bio-integrity.c

Quoted 4 lines
block/bio-integrity.c +4 -0
+struct bio_crypt_ctx *bio_crypt_alloc_ctx(gfp_t gfp_mask)
+{
+	return mempool_alloc(bio_crypt_ctx_pool, gfp_mask);
+}

I'd rather move bio_crypt_set_ctx out of line, at which point we don't really need this helper.

Quoted 6 lines
block/bio-integrity.c +6 -0
+/* Return: 0 on success, negative on error */
+int rq_crypt_ctx_acquire_keyslot(struct bio_crypt_ctx *bc,
+				  struct keyslot_manager *ksm,
+				  struct rq_crypt_ctx *rc)
+{
+	rc->keyslot = blk_ksm_get_slot_for_key(ksm, bc->bc_key);

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+	return rc->keyslot >= 0 ? 0 : rc->keyslot;
+}
+
+void rq_crypt_ctx_release_keyslot(struct keyslot_manager *ksm,
+				  struct rq_crypt_ctx *rc)
+{
+	if (rc->keyslot >= 0)
+		blk_ksm_put_slot(ksm, rc->keyslot);
+	rc->keyslot = -EINVAL;
+}

Is there really much of a need for these helpers? I think the callers would generally be simpler without them. Especially the fallback code can avoid having to declare rq_crypt_ctx variables on stack without the helpers.

block/bio-integrity.c

Quoted 2 lines
block/bio-integrity.c +2 -0
+int blk_crypto_init_request(struct request *rq, struct request_queue *q,
+			    struct bio *bio)

We can always derive the request_queue from rq->q, so there is no need to pass it explicitly (even if a lot of legacy block code does, but it is slowly getting cleaned up).

Quoted 4 lines
block/bio-integrity.c +4 -0
+{
+	struct rq_crypt_ctx *rc = &rq->rq_crypt_ctx;
+	struct bio_crypt_ctx *bc;
+	int err;

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+
+	rc->bc = NULL;
+	rc->keyslot = -EINVAL;
+
+	if (!bio)
+		return 0;
+
+	bc = bio->bi_crypt_context;
+	if (!bc)
+		return 0;

Shouldn't the checks if the bio actually requires crypto handling be done by the caller based on a new handler ala:

static inline bool bio_is_encrypted(struct bio *bio)

{
	return bio && bio->bi_crypt_context;
}

and maybe some inline helpers to reduce the clutter?

That way a kernel with blk crypto support, but using non-crypto I/O saves all the function calls to blk-crypto.

block/bio-integrity.c

Quoted 3 lines
block/bio-integrity.c +3 -0
+	err = bio_crypt_check_alignment(bio);
+	if (err)
+		goto fail;

This seems pretty late to check the alignment, it would be more useful in bio_add_page. Then again Jens didn't like alignment checks in the block layer at all even for the normal non-crypto alignment, so I don't see why we'd have them here but not for the general case (I'd actually like to ee them for the general case, btw).

block/bio-integrity.c

Quoted 8 lines
block/bio-integrity.c +8 -0
+int blk_crypto_evict_key(struct request_queue *q,
+			 const struct blk_crypto_key *key)
+{
+	if (q->ksm && blk_ksm_crypto_mode_supported(q->ksm, key))
+		return blk_ksm_evict_key(q->ksm, key);
+
+	return 0;
+}

Is there any point in this wrapper that just has a single caller? Als why doesn't blk_ksm_evict_key have the blk_ksm_crypto_mode_supported sanity check itself?

block/bio-integrity.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2015

Quoted 10 lines
block/bio-integrity.c +6 -0
@@ -1998,6 +2007,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 
 	cookie = request_to_qc_t(data.hctx, rq);
 
+	if (blk_crypto_init_request(rq, q, bio)) {
+		bio->bi_status = BLK_STS_RESOURCE;
+		bio_endio(bio);
+		blk_mq_end_request(rq, BLK_STS_RESOURCE);
+		return BLK_QC_T_NONE;
+	}

This looks fundamentally wrong given that layers above blk-mq can't handle BLK_STS_RESOURCE. It will just show up as an error in the calller insteaf of being requeued.

That being said I think the only error return from blk_crypto_init_request is and actual hardware error. So failing this might be ok, but it should be BLK_STS_IOERR, or even better an error directly propagated from the driver.

block/bio-integrity.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)

Quoted 5 lines
block/bio-integrity.c +5 -0
+int bio_crypt_ctx_init(void);
+
+struct bio_crypt_ctx *bio_crypt_alloc_ctx(gfp_t gfp_mask);
+
+void bio_crypt_free_ctx(struct bio *bio);

These can go into block layer internal headers.

Quoted 18 lines
block/bio-integrity.c +18 -0
+static inline bool bio_crypt_dun_is_contiguous(const struct bio_crypt_ctx *bc,
+					       unsigned int bytes,
+					u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE])
+{
+	int i = 0;
+	unsigned int inc = bytes >> bc->bc_key->data_unit_size_bits;
+
+	while (i < BLK_CRYPTO_DUN_ARRAY_SIZE) {
+		if (bc->bc_dun[i] + inc != next_dun[i])
+			return false;
+		inc = ((bc->bc_dun[i] + inc)  < inc);
+		i++;
+	}
+
+	return true;
+}
+
+static inline void bio_crypt_dun_increment(u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE],

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+					   unsigned int inc)
+{
+	int i = 0;
+
+	while (inc && i < BLK_CRYPTO_DUN_ARRAY_SIZE) {
+		dun[i] += inc;
+		inc = (dun[i] < inc);
+		i++;
+	}
+}

Should these really be inline?

Quoted 23 lines
block/bio-integrity.c +23 -0
+bool bio_crypt_rq_ctx_compatible(struct request *rq, struct bio *bio);
+
+bool bio_crypt_ctx_front_mergeable(struct request *req, struct bio *bio);
+
+bool bio_crypt_ctx_back_mergeable(struct request *req, struct bio *bio);
+
+bool bio_crypt_ctx_merge_rq(struct request *req, struct request *next);
+
+void blk_crypto_bio_back_merge(struct request *req, struct bio *bio);
+
+void blk_crypto_bio_front_merge(struct request *req, struct bio *bio);
+
+void blk_crypto_free_request(struct request *rq);
+
+int blk_crypto_init_request(struct request *rq, struct request_queue *q,
+			    struct bio *bio);
+
+int blk_crypto_bio_prep(struct bio **bio_ptr);
+
+void blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio);
+
+void blk_crypto_rq_prep_clone(struct request *dst, struct request *src);
+

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+int blk_crypto_insert_cloned_request(struct request_queue *q,
+				     struct request *rq);
+
+int blk_crypto_init_key(struct blk_crypto_key *blk_key, const u8 *raw_key,
+			enum blk_crypto_mode_num crypto_mode,
+			unsigned int blk_crypto_dun_bytes,
+			unsigned int data_unit_size);
+
+int blk_crypto_evict_key(struct request_queue *q,
+			 const struct blk_crypto_key *key);

Most of this should be block layer private.

Quoted 9 lines
block/bio-integrity.c +5 -0
+struct rq_crypt_ctx {
+	struct bio_crypt_ctx *bc;
+	int keyslot;
+};
+
 /*
  * Try to put the fields that are referenced together in the same cacheline.
  *

block/bio-integrity.c struct request { · line 235

Quoted 6 lines
block/bio-integrity.c +3 -0
@@ -224,6 +230,10 @@ struct request {
 	unsigned short nr_integrity_segments;
 #endif
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	struct rq_crypt_ctx rq_crypt_ctx;
+#endif

I'd be tempted to just add

	struct bio_crypt_ctx *crypt_ctx;
	int crypt_keyslot;

directly to struct request.

#10 lore raw

Re: [PATCH v7 2/9] block: Inline encryption support for blk-mq

Quoted 7 lines · Christoph Hellwig
Quoted 7 lines

index bf62c25cde8f..bce563031e7c 100644

block/bio-integrity.c +0 -0
@@ -42,6 +42,11 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,
 	struct bio_set *bs = bio->bi_pool;
 	unsigned inline_vecs;
 
Quoted 10 lines

block/bio-integrity.c struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, · line 48

Quoted 4 lines
block/bio-integrity.c +4 -0
+	if (bio_has_crypt_ctx(bio)) {
+		pr_warn("blk-integrity can't be used together with blk-crypto en/decryption.");
+		return ERR_PTR(-EOPNOTSUPP);
+	}

What is the rationale for this limitation? Restricting unrelated features from being used together is a pretty bad design pattern and should be avoided where possible. If it can't it needs to be documented very clearly.

My understanding of blk-integrity is that for writes, blk-integrity generates some integrity info for a bio and sends it along with the bio, and the device on the other end verifies that the data it received to write matches up with the integrity info provided with the bio, and saves the integrity info along with the data. As for reads, the device sends the data along with the saved integrity info and blk-integrity verifies that the data received matches up with the integrity info.

So for encryption and integrity to work together, in systems without hardware inline encryption support, encryption of data in a bio must happen in the kernel before bio_integrity_prep, since we shouldn't modify data in the bio after integrity calculation, and similarly, decryption of data in the bio must happen in the kernel after integrity verification. The integrity info saved on disk is the integrity info of the ciphertext.

But in systems with hardware inline encryption support, during write, the device will receive integrity info of the plaintext data, and during read, it will send integrity info of the plaintext data to the kernel. I'm worried that the device may not do anything special and simply save the integrity info of the plaintext on disk which will cause the on disk format to no longer be the same whether or not hardware inline encryption support is present, which will cause issues like not being able to ever switch between using the "inlinecrypt" mount option (and in particular, that means people with existing encrypted filesystems on disks with integrity can't just turn on the "inlinecrypt" mount option).

As far as I can tell, I think all the issues go away if the hardware explicitly computes integrity info for the ciphertext and stores that on disk instead of what the kernel passes it, and on the read path, it decrypts the data on disk and generates integrity info for the plain text, and passes it to the kernel. Eric also points out that a device that just stores integrity info of the plaintext on disk would be broken since the integrity info would leak information about the plaintext, and that the correct thing for the device to do is store integrity info computed using only the ciphertext.

So if we're alright with assuming for now that hardware vendors will at least plan to do the "right" thing, I'll just remove the exclusivity check in bio-integrity and also the REQ_NO_SPECIAL code :) (and figure out how to handle dm-integrity) because the rest of the code will otherwise work as intended w.r.t encryption with integrity. As for handling cases like devices that actually don't do the "right" thing, or devices that support both features but just not at the same time, I think I'll leave that to when we actually have hardware that has both these features.

Quoted 232 lines

block/bio-integrity.c struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, · line 51

Quoted 3 lines
block/bio-integrity.c +3 -0
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	rq->rq_crypt_ctx.keyslot = -EINVAL;
+#endif

All the other core block calls to the crypto code are in helpers that are stubbed out. It might make sense to follow that style here.

block/bio-integrity.c int __init blk_dev_init(void) · line 1830

Quoted 8 lines
block/bio-integrity.c +2 -0
 free_and_out:
@@ -1813,5 +1826,8 @@ int __init blk_dev_init(void)
 	blk_debugfs_root = debugfs_create_dir("block", NULL);
 #endif
 
+	if (bio_crypt_ctx_init() < 0)
+		panic("Failed to allocate mem for bio crypt ctxs\n");

Maybe move that panic into bio_crypt_ctx_init itself?

block/bio-integrity.c int __init blk_dev_init(void)

Quoted 5 lines
block/bio-integrity.c +5 -0
+static int num_prealloc_crypt_ctxs = 128;
+
+module_param(num_prealloc_crypt_ctxs, int, 0444);
+MODULE_PARM_DESC(num_prealloc_crypt_ctxs,
+		"Number of bio crypto contexts to preallocate");

Please write a comment why this is a tunable, how the default is choosen and why someone might want to chane it.

block/bio-integrity.c

Quoted 4 lines
block/bio-integrity.c +4 -0
+struct bio_crypt_ctx *bio_crypt_alloc_ctx(gfp_t gfp_mask)
+{
+	return mempool_alloc(bio_crypt_ctx_pool, gfp_mask);
+}

I'd rather move bio_crypt_set_ctx out of line, at which point we don't really need this helper.

Quoted 6 lines
block/bio-integrity.c +6 -0
+/* Return: 0 on success, negative on error */
+int rq_crypt_ctx_acquire_keyslot(struct bio_crypt_ctx *bc,
+				  struct keyslot_manager *ksm,
+				  struct rq_crypt_ctx *rc)
+{
+	rc->keyslot = blk_ksm_get_slot_for_key(ksm, bc->bc_key);

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+	return rc->keyslot >= 0 ? 0 : rc->keyslot;
+}
+
+void rq_crypt_ctx_release_keyslot(struct keyslot_manager *ksm,
+				  struct rq_crypt_ctx *rc)
+{
+	if (rc->keyslot >= 0)
+		blk_ksm_put_slot(ksm, rc->keyslot);
+	rc->keyslot = -EINVAL;
+}

Is there really much of a need for these helpers? I think the callers would generally be simpler without them. Especially the fallback code can avoid having to declare rq_crypt_ctx variables on stack without the helpers.

block/bio-integrity.c

Quoted 2 lines
block/bio-integrity.c +2 -0
+int blk_crypto_init_request(struct request *rq, struct request_queue *q,
+			    struct bio *bio)

We can always derive the request_queue from rq->q, so there is no need to pass it explicitly (even if a lot of legacy block code does, but it is slowly getting cleaned up).

Quoted 4 lines
block/bio-integrity.c +4 -0
+{
+	struct rq_crypt_ctx *rc = &rq->rq_crypt_ctx;
+	struct bio_crypt_ctx *bc;
+	int err;

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+
+	rc->bc = NULL;
+	rc->keyslot = -EINVAL;
+
+	if (!bio)
+		return 0;
+
+	bc = bio->bi_crypt_context;
+	if (!bc)
+		return 0;

Shouldn't the checks if the bio actually requires crypto handling be done by the caller based on a new handler ala:

static inline bool bio_is_encrypted(struct bio *bio)

{
	return bio && bio->bi_crypt_context;
}

and maybe some inline helpers to reduce the clutter?

That way a kernel with blk crypto support, but using non-crypto I/O saves all the function calls to blk-crypto.

block/bio-integrity.c

Quoted 3 lines
block/bio-integrity.c +3 -0
+	err = bio_crypt_check_alignment(bio);
+	if (err)
+		goto fail;

This seems pretty late to check the alignment, it would be more useful in bio_add_page. Then again Jens didn't like alignment checks in the block layer at all even for the normal non-crypto alignment, so I don't see why we'd have them here but not for the general case (I'd actually like to ee them for the general case, btw).

block/bio-integrity.c

Quoted 8 lines
block/bio-integrity.c +8 -0
+int blk_crypto_evict_key(struct request_queue *q,
+			 const struct blk_crypto_key *key)
+{
+	if (q->ksm && blk_ksm_crypto_mode_supported(q->ksm, key))
+		return blk_ksm_evict_key(q->ksm, key);
+
+	return 0;
+}

Is there any point in this wrapper that just has a single caller? Als why doesn't blk_ksm_evict_key have the blk_ksm_crypto_mode_supported sanity check itself?

block/bio-integrity.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2015

Quoted 10 lines
block/bio-integrity.c +6 -0
@@ -1998,6 +2007,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
 
 	cookie = request_to_qc_t(data.hctx, rq);
 
+	if (blk_crypto_init_request(rq, q, bio)) {
+		bio->bi_status = BLK_STS_RESOURCE;
+		bio_endio(bio);
+		blk_mq_end_request(rq, BLK_STS_RESOURCE);
+		return BLK_QC_T_NONE;
+	}

This looks fundamentally wrong given that layers above blk-mq can't handle BLK_STS_RESOURCE. It will just show up as an error in the calller insteaf of being requeued.

That being said I think the only error return from blk_crypto_init_request is and actual hardware error. So failing this might be ok, but it should be BLK_STS_IOERR, or even better an error directly propagated from the driver.

block/bio-integrity.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)

Quoted 5 lines
block/bio-integrity.c +5 -0
+int bio_crypt_ctx_init(void);
+
+struct bio_crypt_ctx *bio_crypt_alloc_ctx(gfp_t gfp_mask);
+
+void bio_crypt_free_ctx(struct bio *bio);

These can go into block layer internal headers.

Quoted 18 lines
block/bio-integrity.c +18 -0
+static inline bool bio_crypt_dun_is_contiguous(const struct bio_crypt_ctx *bc,
+					       unsigned int bytes,
+					u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE])
+{
+	int i = 0;
+	unsigned int inc = bytes >> bc->bc_key->data_unit_size_bits;
+
+	while (i < BLK_CRYPTO_DUN_ARRAY_SIZE) {
+		if (bc->bc_dun[i] + inc != next_dun[i])
+			return false;
+		inc = ((bc->bc_dun[i] + inc)  < inc);
+		i++;
+	}
+
+	return true;
+}
+
+static inline void bio_crypt_dun_increment(u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE],

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+					   unsigned int inc)
+{
+	int i = 0;
+
+	while (inc && i < BLK_CRYPTO_DUN_ARRAY_SIZE) {
+		dun[i] += inc;
+		inc = (dun[i] < inc);
+		i++;
+	}
+}

Should these really be inline?

Quoted 23 lines
block/bio-integrity.c +23 -0
+bool bio_crypt_rq_ctx_compatible(struct request *rq, struct bio *bio);
+
+bool bio_crypt_ctx_front_mergeable(struct request *req, struct bio *bio);
+
+bool bio_crypt_ctx_back_mergeable(struct request *req, struct bio *bio);
+
+bool bio_crypt_ctx_merge_rq(struct request *req, struct request *next);
+
+void blk_crypto_bio_back_merge(struct request *req, struct bio *bio);
+
+void blk_crypto_bio_front_merge(struct request *req, struct bio *bio);
+
+void blk_crypto_free_request(struct request *rq);
+
+int blk_crypto_init_request(struct request *rq, struct request_queue *q,
+			    struct bio *bio);
+
+int blk_crypto_bio_prep(struct bio **bio_ptr);
+
+void blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio);
+
+void blk_crypto_rq_prep_clone(struct request *dst, struct request *src);
+

block/bio-integrity.c

Quoted 10 lines
block/bio-integrity.c +10 -0
+int blk_crypto_insert_cloned_request(struct request_queue *q,
+				     struct request *rq);
+
+int blk_crypto_init_key(struct blk_crypto_key *blk_key, const u8 *raw_key,
+			enum blk_crypto_mode_num crypto_mode,
+			unsigned int blk_crypto_dun_bytes,
+			unsigned int data_unit_size);
+
+int blk_crypto_evict_key(struct request_queue *q,
+			 const struct blk_crypto_key *key);

Most of this should be block layer private.

Quoted 9 lines
block/bio-integrity.c +5 -0
+struct rq_crypt_ctx {
+	struct bio_crypt_ctx *bc;
+	int keyslot;
+};
+
 /*
  * Try to put the fields that are referenced together in the same cacheline.
  *

block/bio-integrity.c struct request { · line 235

Quoted 6 lines
block/bio-integrity.c +3 -0
@@ -224,6 +230,10 @@ struct request {
 	unsigned short nr_integrity_segments;
 #endif
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION
+	struct rq_crypt_ctx rq_crypt_ctx;
+#endif

I'd be tempted to just add

	struct bio_crypt_ctx *crypt_ctx;
	int crypt_keyslot;

directly to struct request.

#11 lore raw

Re: [PATCH v7 2/9] block: Inline encryption support for blk-mq

Quoted 5 lines · Satya Tangirala
Quoted 5 lines

What is the rationale for this limitation? Restricting unrelated features from being used together is a pretty bad design pattern and should be avoided where possible. If it can't it needs to be documented very clearly.

Quoted 7 lines

My understanding of blk-integrity is that for writes, blk-integrity generates some integrity info for a bio and sends it along with the bio, and the device on the other end verifies that the data it received to write matches up with the integrity info provided with the bio, and saves the integrity info along with the data. As for reads, the device sends the data along with the saved integrity info and blk-integrity verifies that the data received matches up with the integrity info.

Yes, a device supporting inline encryption and integrity will have to update the guard tag to match the encrypted data as well. That alone is a good enough reason to reject the combination for now until it is fully supported. It needs to be properly document, and I think we should also do it at probe time if possible, not when submitting I/O.

maintainer
#12 lore raw

Re: [PATCH v7 2/9] block: Inline encryption support for blk-mq

In reply to Christoph Hellwig
Quoted 4 lines · Christoph Hellwig
Quoted 4 lines
block/blk-crypto.c new file +4 -0
+int blk_crypto_evict_key(struct request_queue *q,
+			 const struct blk_crypto_key *key)
+{
+	if (q->ksm && blk_ksm_crypto_mode_supported(q->ksm, key))
Quoted 8 lines

block/blk-crypto.c line 412

Quoted 4 lines
block/blk-crypto.c new file +4 -0
+		return blk_ksm_evict_key(q->ksm, key);
+
+	return 0;
+}

Is there any point in this wrapper that just has a single caller? Als why doesn't blk_ksm_evict_key have the blk_ksm_crypto_mode_supported sanity check itself?

Later in the series it's changed to:

int blk_crypto_evict_key(struct request_queue *q,
const struct blk_crypto_key *key)

{
        if (q->ksm && blk_ksm_crypto_mode_supported(q->ksm, key))
                return blk_ksm_evict_key(q->ksm, key);

        return blk_crypto_fallback_evict_key(key);
}

I.e. if the encryption mode is using hardware, then the key needs to be evicted from q->ksm. Otherwise the key needs to be evicted from the fallback.

Also keep in mind that our goal is to define a clean API for any user of the block layer to use encryption, not just fs/crypto/. That API includes:

	blk_crypto_init_key()
	blk_crypto_start_using_key()
	bio_crypt_set_ctx()
	blk_crypto_evict_key()

If anyone else decides to use inline encryption (e.g., if inline encryption support were added to dm-crypt or another device-mapper target), they'll use these same functions.

So IMO it's important to define a clean API that won't need to be refactored as soon as anyone else starts using it, and not e.g. micro-optimize for code length based on there currently being only one user.

- Eric

#13 lore raw

PATCH v7 3/9 block: blk-crypto-fallback for Inline Encryption

In reply to Satya Tangirala

Blk-crypto delegates crypto operations to inline encryption hardware when available. The separately configurable blk-crypto-fallback contains a software fallback to the kernel crypto API - when enabled, blk-crypto will use this fallback for en/decryption when inline encryption hardware is not available. This lets upper layers not have to worry about whether or not the underlying device has support for inline encryption before deciding to specify an encryption context for a bio, and also allows for testing without actual inline encryption hardware. For more details, refer to Documentation/block/inline-encryption.rst.

---
 Documentation/block/index.rst             |   1 +
 Documentation/block/inline-encryption.rst | 162 ++++++
 block/Kconfig                             |  10 +
 block/Makefile                            |   1 +
 block/bio-integrity.c                     |   2 +-
 block/blk-crypto-fallback.c               | 673 ++++++++++++++++++++++
 block/blk-crypto-internal.h               |  32 +
 block/blk-crypto.c                        |  43 +-
 include/linux/blk-crypto.h                |  17 +-
 include/linux/blk_types.h                 |   6 +
 10 files changed, 938 insertions(+), 9 deletions(-)
 create mode 100644 Documentation/block/inline-encryption.rst
 create mode 100644 block/blk-crypto-fallback.c
Documentation/block/index.rst +1 -0
index 3fa7a52fafa4..026addfc69bc 100644
@@ -14,6 +14,7 @@ Block
    cmdline-partition
    data-integrity
    deadline-iosched
+   inline-encryption
    ioprio
    kyber-iosched
    null_blk
Documentation/block/inline-encryption.rst new file +162 -0

162 lines not on this page · raw message

block/Kconfig +10 -0
index c04a1d500842..0af387623774 100644
@@ -192,6 +192,16 @@ config BLK_INLINE_ENCRYPTION
 	  block layer handle encryption, so users can take
 	  advantage of inline encryption hardware if present.
 
+config BLK_INLINE_ENCRYPTION_FALLBACK
+	bool "Enable crypto API fallback for blk-crypto"
+	depends on BLK_INLINE_ENCRYPTION
+	select CRYPTO
+	select CRYPTO_SKCIPHER
+	help
+	  Enabling this lets the block layer handle inline encryption
+	  by falling back to the kernel crypto API when inline
+	  encryption hardware is not present.
+
 menu "Partition Types"
 
 source "block/partitions/Kconfig"
block/Makefile +1 -0
index 82f42ca3f769..9464fb6ae423 100644
@@ -38,3 +38,4 @@ obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
 obj-$(CONFIG_BLK_SED_OPAL)	+= sed-opal.o
 obj-$(CONFIG_BLK_PM)		+= blk-pm.o
 obj-$(CONFIG_BLK_INLINE_ENCRYPTION)	+= keyslot-manager.o blk-crypto.o
+obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK)	+= blk-crypto-fallback.o
block/bio-integrity.c +1 -1
index bce563031e7c..6f6da02b10ef 100644
@@ -42,7 +42,7 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,
 	struct bio_set *bs = bio->bi_pool;
 	unsigned inline_vecs;
 
-	if (bio_has_crypt_ctx(bio)) {
+	if (bio_has_crypt_ctx(bio) || (bio->bi_opf & REQ_NO_SPECIAL)) {
 		pr_warn("blk-integrity can't be used together with blk-crypto en/decryption.");
 		return ERR_PTR(-EOPNOTSUPP);
 	}
block/blk-crypto-fallback.c new file +673 -0

673 lines not on this page · raw message

block/blk-crypto-internal.h +32 -0
index 2452b5dee140..4cccaa9b81ef 100644
@@ -16,4 +16,36 @@ struct blk_crypto_mode {
 	unsigned int ivsize; /* iv size in bytes */
 };
 
+extern const struct blk_crypto_mode blk_crypto_modes[];
+
+void rq_crypt_ctx_release_keyslot(struct keyslot_manager *ksm,
+				  struct rq_crypt_ctx *rc);
+
+int rq_crypt_ctx_acquire_keyslot(struct bio_crypt_ctx *bc,
+				 struct keyslot_manager *ksm,
+				 struct rq_crypt_ctx *rc);
+
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK
+
+int blk_crypto_fallback_bio_prep(struct bio **bio_ptr);
+
+int blk_crypto_fallback_evict_key(const struct blk_crypto_key *key);
+
+#else /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
+static inline int blk_crypto_fallback_bio_prep(struct bio **bio_ptr)
+{
+	pr_warn_once("crypto API fallback disabled; failing request.\n");
+	(*bio_ptr)->bi_status = BLK_STS_NOTSUPP;
+	return false;
+}
+
+static inline int
+blk_crypto_fallback_evict_key(const struct blk_crypto_key *key)
+{
+	return 0;
+}
+
+#endif /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
 #endif /* __LINUX_BLK_CRYPTO_INTERNAL_H */
block/blk-crypto.c +36 -7

92 lines not on this page · raw message

include/linux/blk-crypto.h +16 -1
index f855895770f6..bde2e08879b6 100644
@@ -58,7 +58,7 @@ struct blk_crypto_key {
  *
  * A bio_crypt_ctx specifies that the contents of the bio will be encrypted (for
  * write requests) or decrypted (for read requests) inline by the storage device
- * or controller.
+ * or controller, or by the crypto API fallback.
  */
 struct bio_crypt_ctx {
 	const struct blk_crypto_key	*bc_key;
@@ -251,6 +251,21 @@ static inline int blk_crypto_insert_cloned_request(struct request_queue *q,
 
 #endif /* CONFIG_BLK_INLINE_ENCRYPTION */
 
+#ifdef CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK
+
+int blk_crypto_start_using_key(struct blk_crypto_key *key,
+			       struct request_queue *q);
+
+#else /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
+static inline int blk_crypto_start_using_key(struct blk_crypto_key *key,
+					     struct request_queue *q)
+{
+	return 0;
+}
+
+#endif /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */
+
 #endif /* CONFIG_BLOCK */
 
 #endif /* __LINUX_BLK_CRYPTO_H */
include/linux/blk_types.h +6 -0
index 1996689c51d3..520254f92a81 100644
@@ -324,6 +324,11 @@ enum req_flag_bits {
 	__REQ_NOMERGE,		/* don't touch this for merging */
 	__REQ_IDLE,		/* anticipate more IO after this one */
 	__REQ_INTEGRITY,	/* I/O includes block integrity payload */
+	/*
+	 * I/O cannot include any special tags like block integrity or
+	 * bi_crypt_context
+	 */
+	__REQ_NO_SPECIAL,
 	__REQ_FUA,		/* forced unit access */
 	__REQ_PREFLUSH,		/* request for cache flush */
 	__REQ_RAHEAD,		/* read ahead, can fail anytime */
@@ -359,6 +364,7 @@ enum req_flag_bits {
 #define REQ_NOMERGE		(1ULL << __REQ_NOMERGE)
 #define REQ_IDLE		(1ULL << __REQ_IDLE)
 #define REQ_INTEGRITY		(1ULL << __REQ_INTEGRITY)
+#define REQ_NO_SPECIAL		(1ULL << __REQ_NO_SPECIAL)
 #define REQ_FUA			(1ULL << __REQ_FUA)
 #define REQ_PREFLUSH		(1ULL << __REQ_PREFLUSH)
 #define REQ_RAHEAD		(1ULL << __REQ_RAHEAD)
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
reviewer
#14 lore raw

Re: [PATCH v7 3/9] block: blk-crypto-fallback for Inline Encryption

Quoted 33 lines · Satya Tangirala

Blk-crypto delegates crypto operations to inline encryption hardware when available. The separately configurable blk-crypto-fallback contains a software fallback to the kernel crypto API - when enabled, blk-crypto will use this fallback for en/decryption when inline encryption hardware is not available. This lets upper layers not have to worry about whether or not the underlying device has support for inline encryption before deciding to specify an encryption context for a bio, and also allows for testing without actual inline encryption hardware. For more details, refer to Documentation/block/inline-encryption.rst.

---
 Documentation/block/index.rst             |   1 +
 Documentation/block/inline-encryption.rst | 162 ++++++
 block/Kconfig                             |  10 +
 block/Makefile                            |   1 +
 block/bio-integrity.c                     |   2 +-
 block/blk-crypto-fallback.c               | 673 ++++++++++++++++++++++
 block/blk-crypto-internal.h               |  32 +
 block/blk-crypto.c                        |  43 +-
 include/linux/blk-crypto.h                |  17 +-
 include/linux/blk_types.h                 |   6 +
 10 files changed, 938 insertions(+), 9 deletions(-)
 create mode 100644 Documentation/block/inline-encryption.rst
 create mode 100644 block/blk-crypto-fallback.c
Documentation/block/inline-encryption.rst new file +0 -0
new file mode 100644
index 000000000000..02abea993975

Documentation/block/inline-encryption.rst line 10

Quoted 10 lines
Documentation/block/inline-encryption.rst new file +10 -0
@@ -0,0 +1,162 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Inline Encryption
+=================
+
+Background
+==========
+
+Inline encryption hardware sit logically between memory and the disk, and can

sits

Documentation/block/inline-encryption.rst line 11

Quoted 1 line
Documentation/block/inline-encryption.rst new file +1 -0
+en/decrypt data as it goes in/out of the disk. Inline encryption hardware have a

has

Quoted 14 lines
Documentation/block/inline-encryption.rst new file +14 -0
+fixed number of "keyslots" - slots into which encryption contexts (i.e. the
+encryption key, encryption algorithm, data unit size) can be programmed by the
+kernel at any time. Each request sent to the disk can be tagged with the index
+of a keyslot (and also a data unit number to act as an encryption tweak), and
+the inline encryption hardware will en/decrypt the data in the request with the
+encryption context programmed into that keyslot. This is very different from
+full disk encryption solutions like self encrypting drives/TCG OPAL/ATA
+Security standards, since with inline encryption, any block on disk could be
+encrypted with any encryption context the kernel chooses.
+
+
+Objective
+=========
+

...

Documentation/block/inline-encryption.rst line 37

Quoted 6 lines
Documentation/block/inline-encryption.rst new file +6 -0
+
+
+Constraints and notes
+=====================
+
+- IE hardware have a limited number of "keyslots" that can be programmed

has

Quoted 6 lines
Documentation/block/inline-encryption.rst new file +6 -0
+  with an encryption context (key, algorithm, data unit size, etc.) at any time.
+  One can specify a keyslot in a data request made to the device, and the
+  device will en/decrypt the data using the encryption context programmed into
+  that specified keyslot. When possible, we want to make multiple requests with
+  the same encryption context share the same keyslot.
+

...

Quoted 6 lines
Documentation/block/inline-encryption.rst new file +6 -0
+
+
+Design
+======
+
+We add a :c:type:`struct bio_crypt_ctx` to :c:type:`struct bio` that can

Documentation/block/inline-encryption.rst line 65

Quoted 10 lines
Documentation/block/inline-encryption.rst new file +10 -0
+represent an encryption context, because we need to be able to pass this
+encryption context from the FS layer to the device driver to act upon.
+
+While IE hardware works on the notion of keyslots, the FS layer has no
+knowledge of keyslots - it simply wants to specify an encryption context to
+use while en/decrypting a bio.
+
+We introduce a keyslot manager (KSM) that handles the translation from
+encryption contexts specified by the FS to keyslots on the IE hardware.
+This KSM also serves as the way IE hardware can expose their capabilities to

its

Documentation/block/inline-encryption.rst line 72

Quoted 7 lines
Documentation/block/inline-encryption.rst new file +7 -0
+upper layers. The generic mode of operation is: each device driver that wants
+to support IE will construct a KSM and set it up in its struct request_queue.
+Upper layers that want to use IE on this device can then use this KSM in
+the device's struct request_queue to translate an encryption context into
+a keyslot. The presence of the KSM in the request queue shall be used to mean
+that the device supports IE.
+

...

Hi Satya,
ISTM that we disagree on whether "hardware" is singular or plural. ;)

My interface search foo (FWIW) says that "hardware" has no plural version. Anyway, my best evidence is in your intro/commit message, where it says:
"when inline encryption hardware is not available",
so it must be singular. :)

cheers.

Signature · 1 line
~Randy
#15 lore raw

Re: [PATCH v7 3/9] block: blk-crypto-fallback for Inline Encryption

In reply to Satya Tangirala

That REQ_NO_SPECIAL think really needs a separate patch and be explained. In fact I hope we can get away without it as it is horrible, so maybe the explanation will lead to a better outcome.

#16 lore raw

Re: [PATCH v7 3/9] block: blk-crypto-fallback for Inline Encryption

In reply to Satya Tangirala

High-level question: Does the whole keyslot manager concept even make sense for the fallback? With the work-queue we have item that exectutes at a time per cpu. So just allocatea per-cpu crypto_skcipher for each encryption mode and there should never be a slot limitation. Or do I miss something?

maintainer
#17 lore raw

Re: [PATCH v7 3/9] block: blk-crypto-fallback for Inline Encryption

Quoted 5 lines · Christoph Hellwig

High-level question: Does the whole keyslot manager concept even make sense for the fallback? With the work-queue we have item that exectutes at a time per cpu. So just allocatea per-cpu crypto_skcipher for each encryption mode and there should never be a slot limitation. Or do I miss something?

It does make sense because if blk-crypto-fallback didn't use a keyslot manager, it would have to call crypto_skcipher_setkey() on the I/O path for every bio to ensure that the CPU's crypto_skcipher has the correct key. That's undesirable, because setting a new key can be expensive with some encryption algorithms, and also it can require a memory allocation which can fail. For example, with the Adiantum algorithm, setting a key requires encrypting ~1100 bytes of data in order to generate subkeys. It's better to set a key once and use it many times.

Making blk-crypto-fallback use the keyslot manager also allows the keyslot manager to be tested by routine filesystem regression testing, e.g. 'gce-xfstests -c ext4/encrypt -g auto -m inlinecrypt'.

- Eric

#18 lore raw

Re: [PATCH v7 3/9] block: blk-crypto-fallback for Inline Encryption

Quoted 7 lines · Eric Biggers
Quoted 5 lines · Christoph Hellwig

High-level question: Does the whole keyslot manager concept even make sense for the fallback? With the work-queue we have item that exectutes at a time per cpu. So just allocatea per-cpu crypto_skcipher for each encryption mode and there should never be a slot limitation. Or do I miss something?

Quoted 7 lines

It does make sense because if blk-crypto-fallback didn't use a keyslot manager, it would have to call crypto_skcipher_setkey() on the I/O path for every bio to ensure that the CPU's crypto_skcipher has the correct key. That's undesirable, because setting a new key can be expensive with some encryption algorithms, and also it can require a memory allocation which can fail. For example, with the Adiantum algorithm, setting a key requires encrypting ~1100 bytes of data in order to generate subkeys. It's better to set a key once and use it many times.

I didn't think of such expensive operations when setting the key. Note that you would not have to do it on every I/O, as chances are high you'll get I/O from the same submitter and thus the same key, and we can optimize for that case pretty easily.

But if you think the keyslot manager is better I accept that, this was just a throught when looking over the code.

maintainer
#19 lore raw

Re: [PATCH v7 3/9] block: blk-crypto-fallback for Inline Encryption

In reply to Satya Tangirala
Quoted 11 lines · Satya Tangirala

Blk-crypto delegates crypto operations to inline encryption hardware when available. The separately configurable blk-crypto-fallback contains a software fallback to the kernel crypto API - when enabled, blk-crypto will use this fallback for en/decryption when inline encryption hardware is not available. This lets upper layers not have to worry about whether or not the underlying device has support for inline encryption before deciding to specify an encryption context for a bio, and also allows for testing without actual inline encryption hardware. For more details, refer to Documentation/block/inline-encryption.rst.

In v7, only blk_mq_make_request() actually calls blk_crypto_bio_prep(). That will make the crypto contexts be silently ignored (no fallback) if q->make_request_fn != blk_mq_make_request.

In recent kernels that *hopefully* won't matter in practice since almost everyone is using blk_mq_make_request. But it still seems like a poor design. First, it's super important that if someone requests encryption, then they either get it or get an error; it should *never* be silently ignored. Second, part of the goal of blk-crypto-fallback is that it should always work, so that in principle users don't have to implement the encryption twice, once via blk-crypto and once via fs or dm-layer crypto.

So is there any reason not to keep the blk_crypto_bio_prep() call in generic_make_request()?

I think performance can't be much of a complaint, since if almost everyone is using blk_mq_make_request() then they are making the function call anyway...

- Eric

#20 lore raw

PATCH v7 4/9 scsi: ufs: UFS driver v2.1 spec crypto additions

In reply to Satya Tangirala

Add the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI specification in preparation to add support for inline encryption to UFS.

---
 drivers/scsi/ufs/ufshcd.c |  2 ++
 drivers/scsi/ufs/ufshcd.h |  5 +++
 drivers/scsi/ufs/ufshci.h | 67 +++++++++++++++++++++++++++++++++++++--
 3 files changed, 72 insertions(+), 2 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
index abd0e6b05f79..825d9eb34f10 100644
@@ -4759,6 +4759,8 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp)
 	case OCS_MISMATCH_RESP_UPIU_SIZE:
 	case OCS_PEER_COMM_FAILURE:
 	case OCS_FATAL_ERROR:
+	case OCS_INVALID_CRYPTO_CONFIG:
+	case OCS_GENERAL_CRYPTO_ERROR:
 	default:
 		result |= DID_ERROR << 16;
 		dev_err(hba->dev,
drivers/scsi/ufs/ufshcd.h +5 -0
index 2ae6c7c8528c..978781c538c4 100644
@@ -716,6 +716,11 @@ struct ufs_hba {
 	 * for userspace to control the power management.
 	 */
 #define UFSHCD_CAP_RPM_AUTOSUSPEND (1 << 6)
+	/*
+	 * This capability allows the host controller driver to use the
+	 * inline crypto engine, if it is present
+	 */
+#define UFSHCD_CAP_CRYPTO (1 << 7)
 
 	struct devfreq *devfreq;
 	struct ufs_clk_scaling clk_scaling;
drivers/scsi/ufs/ufshci.h +65 -2

106 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#21 lore raw

PATCH v7 5/9 scsi: ufs: UFS crypto API

In reply to Satya Tangirala

Introduce functions to manipulate UFS inline encryption hardware in line with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot manager.

---
 drivers/scsi/ufs/Kconfig         |   9 +
 drivers/scsi/ufs/Makefile        |   1 +
 drivers/scsi/ufs/ufshcd-crypto.c | 367 +++++++++++++++++++++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h |  54 +++++
 drivers/scsi/ufs/ufshcd.h        |  20 ++
 5 files changed, 451 insertions(+)
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c
 create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
drivers/scsi/ufs/Kconfig +9 -0
index d14c2243e02a..c69f1b49167b 100644
@@ -160,3 +160,12 @@ config SCSI_UFS_BSG
 
 	  Select this if you need a bsg device node for your UFS controller.
 	  If unsure, say N.
+
+config SCSI_UFS_CRYPTO
+	bool "UFS Crypto Engine Support"
+	depends on SCSI_UFSHCD && BLK_INLINE_ENCRYPTION
+	help
+	  Enable Crypto Engine Support in UFS.
+	  Enabling this makes it possible for the kernel to use the crypto
+	  capabilities of the UFS device (if present) to perform crypto
+	  operations on data being transferred to/from the device.
drivers/scsi/ufs/Makefile +1 -0
index 94c6c5d7334b..197e178f44bc 100644
@@ -7,6 +7,7 @@ obj-$(CONFIG_SCSI_UFS_QCOM) += ufs-qcom.o
 obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o
 ufshcd-core-y				+= ufshcd.o ufs-sysfs.o
 ufshcd-core-$(CONFIG_SCSI_UFS_BSG)	+= ufs_bsg.o
+ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o
 obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o
 obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o
 obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd-crypto.c new file +367 -0

367 lines not on this page · raw message

drivers/scsi/ufs/ufshcd-crypto.h new file +54 -0

54 lines not on this page · raw message

drivers/scsi/ufs/ufshcd.h +20 -0
index 978781c538c4..7b8a87418f0c 100644
@@ -55,6 +55,7 @@
 #include <linux/clk.h>
 #include <linux/completion.h>
 #include <linux/regulator/consumer.h>
+#include <linux/keyslot-manager.h>
 #include "unipro.h"
 
 #include <asm/irq.h>
@@ -521,6 +522,10 @@ struct ufs_stats {
  * @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for
  *  device is known or not.
  * @scsi_block_reqs_cnt: reference counting for scsi block requests
+ * @crypto_capabilities: Content of crypto capabilities register (0x100)
+ * @crypto_cap_array: Array of crypto capabilities
+ * @crypto_cfg_register: Start of the crypto cfg array
+ * @ksm: the keyslot manager tied to this hba
  */
 struct ufs_hba {
 	void __iomem *mmio_base;
@@ -634,6 +639,13 @@ struct ufs_hba {
 	 * enabled via HCE register.
 	 */
 	#define UFSHCI_QUIRK_BROKEN_HCE				0x400
+
+	/*
+	 * This quirk needs to be enabled if the host controller advertises
+	 * inline encryption support but it doesn't work correctly.
+	 */
+	#define UFSHCD_QUIRK_BROKEN_CRYPTO			0x800
+
 	unsigned int quirks;	/* Deviations from standard UFSHCI spec. */
 
 	/* Device deviations from standard UFS device spec. */
@@ -735,6 +747,14 @@ struct ufs_hba {
 
 	struct device		bsg_dev;
 	struct request_queue	*bsg_queue;
+
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+	/* crypto */
+	union ufs_crypto_capabilities crypto_capabilities;
+	union ufs_crypto_cap_entry *crypto_cap_array;
+	u32 crypto_cfg_register;
+	struct keyslot_manager ksm;
+#endif /* CONFIG_SCSI_UFS_CRYPTO */
 };
 
 /* Returns true if clocks can be gated. Otherwise false */
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
maintainer
#22 lore raw

Re: [PATCH v7 5/9] scsi: ufs: UFS crypto API

Quoted 21 lines · Satya Tangirala
drivers/scsi/ufs/ufshcd-crypto.c new file +21 -0
+static int ufshcd_crypto_cap_find(struct ufs_hba *hba,
+				  enum blk_crypto_mode_num crypto_mode,
+				  unsigned int data_unit_size)
+{
+	enum ufs_crypto_alg ufs_alg;
+	u8 data_unit_mask;
+	int cap_idx;
+	enum ufs_crypto_key_size ufs_key_size;
+	union ufs_crypto_cap_entry *ccap_array = hba->crypto_cap_array;
+
+	if (!ufshcd_hba_is_crypto_supported(hba))
+		return -EINVAL;
+
+	switch (crypto_mode) {
+	case BLK_ENCRYPTION_MODE_AES_256_XTS:
+		ufs_alg = UFS_CRYPTO_ALG_AES_XTS;
+		ufs_key_size = UFS_CRYPTO_KEY_SIZE_256;
+		break;
+	default:
+		return -EINVAL;
+	}

[...]

Quoted 8 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +8 -0
+bool ufshcd_blk_crypto_mode_num_for_alg_dusize(
+					enum ufs_crypto_alg ufs_crypto_alg,
+					enum ufs_crypto_key_size key_size,
+					enum blk_crypto_mode_num *blk_mode_num,
+					unsigned int *max_dun_bytes_supported)
+{
+	/*
+	 * This is currently the only mode that UFS and blk-crypto both support.

drivers/scsi/ufs/ufshcd-crypto.c line 269

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+	 */
+	if (ufs_crypto_alg == UFS_CRYPTO_ALG_AES_XTS &&
+	    key_size == UFS_CRYPTO_KEY_SIZE_256) {
+		*blk_mode_num = BLK_ENCRYPTION_MODE_AES_256_XTS;
+		*max_dun_bytes_supported = 8;
+		return true;
+	}
+
+	return false;
+}

In UFS, max_dun_bytes_supported is always 8 because it's a property of how the DUN is conveyed in the UFS standard, not specific to the crypto algorithm. So, ufshcd_hba_init_crypto() should just set to 8, and there's no need for this code that pretends like it could be a per-algorithm thing.

Also, perhaps ufshcd_crypto_cap_find() and this would be better served by a table that maps between the different conventions for representing the algorithms? For now it would just have one entry:

	static const struct {
		enum ufs_crypto_alg ufs_alg;
		enum ufs_crypto_key_size ufs_key_size;
		enum blk_crypto_mode_num blk_mode;
	} ufs_crypto_algs[] = {
		{
			.ufs_alg = UFS_CRYPTO_ALG_AES_XTS,
			.ufs_key_size = UFS_CRYPTO_KEY_SIZE_256, 
			.blk_mode = BLK_ENCRYPTION_MODE_AES_256_XTS,
		},
	};      

But then it would be super easy to add another entry later.

I think the only reason not to do that is if we didn't expect any more algorithms to be added later. But in that case it would be simpler to remove ufshcd_blk_crypto_mode_num_for_alg_dusize() and just hard-code AES-256-XTS, and likewise make ufshcd_crypto_cap_find() use 'if' instead of 'switch'.

(Note that ufshcd_blk_crypto_mode_num_for_alg_dusize() is also misnamed, as it doesn't have anything to do with the data unit size. And it should be 'static'.)

Quoted 37 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +37 -0
+
+/**
+ * ufshcd_hba_init_crypto - Read crypto capabilities, init crypto fields in hba
+ * @hba: Per adapter instance
+ *
+ * Return: 0 if crypto was initialized or is not supported, else a -errno value.
+ */
+int ufshcd_hba_init_crypto(struct ufs_hba *hba)
+{
+	int cap_idx = 0;
+	int err = 0;
+	enum blk_crypto_mode_num blk_mode_num;
+	unsigned int max_dun_bytes;
+
+	/* Default to disabling crypto */
+	hba->caps &= ~UFSHCD_CAP_CRYPTO;
+
+	/* Return 0 if crypto support isn't present */
+	if (!(hba->capabilities & MASK_CRYPTO_SUPPORT) ||
+	    (hba->quirks & UFSHCD_QUIRK_BROKEN_CRYPTO))
+		goto out;
+
+	/*
+	 * Crypto Capabilities should never be 0, because the
+	 * config_array_ptr > 04h. So we use a 0 value to indicate that
+	 * crypto init failed, and can't be enabled.
+	 */
+	hba->crypto_capabilities.reg_val =
+			cpu_to_le32(ufshcd_readl(hba, REG_UFS_CCAP));
+	hba->crypto_cfg_register =
+		(u32)hba->crypto_capabilities.config_array_ptr * 0x100;
+	hba->crypto_cap_array =
+		devm_kcalloc(hba->dev,
+			     hba->crypto_capabilities.num_crypto_cap,
+			     sizeof(hba->crypto_cap_array[0]),
+			     GFP_KERNEL);
+	if (!hba->crypto_cap_array) {

drivers/scsi/ufs/ufshcd-crypto.c line 316

Quoted 10 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +10 -0
+		err = -ENOMEM;
+		goto out;
+	}
+
+	err = blk_ksm_init(&hba->ksm, hba->dev, ufshcd_num_keyslots(hba));
+	if (err)
+		goto out_free_caps;
+
+	hba->ksm.ksm_ll_ops = ufshcd_ksm_ops;
+	hba->ksm.ll_priv_data = hba;

ll_priv_data isn't used anymore, so it should be removed.

drivers/scsi/ufs/ufshcd-crypto.c line 321

Quoted 5 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +5 -0
+
+	memset(hba->ksm.crypto_modes_supported, 0,
+	       sizeof(hba->ksm.crypto_modes_supported));
+	memset(hba->ksm.max_dun_bytes_supported, 0,
+	       sizeof(hba->ksm.max_dun_bytes_supported));

No need to zero these arrays here, since it's already done by blk_ksm_init().

drivers/scsi/ufs/ufshcd-crypto.c line 325

Quoted 4 lines
drivers/scsi/ufs/ufshcd-crypto.c new file +4 -0
+	/*
+	 * Store all the capabilities now so that we don't need to repeatedly
+	 * access the device each time we want to know its capabilities
+	 */

This comment is a bit misleading now, since now this loop also initializes the crypto_modes_supported array, which is *required* and not just a performance optimization.

- Eric

#23 lore raw

PATCH v7 6/9 scsi: ufs: Add inline encryption support to UFS

In reply to Satya Tangirala

Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.

Also, introduce UFSHCD_QUIRK_BROKEN_CRYPTO that certain UFS drivers that don't yet support inline encryption need to use - taken from patches by John Stultz <[email protected]>
(https://android-review.googlesource.com/c/kernel/common/+/1162224/5)
(https://android-review.googlesource.com/c/kernel/common/+/1162225/5)
(https://android-review.googlesource.com/c/kernel/common/+/1164506/1)

---
 drivers/scsi/ufs/ufs-hisi.c      |  8 +++++
 drivers/scsi/ufs/ufs-qcom.c      |  7 ++++
 drivers/scsi/ufs/ufshcd-crypto.c | 26 ++++++++++++++
 drivers/scsi/ufs/ufshcd-crypto.h | 14 ++++++++
 drivers/scsi/ufs/ufshcd.c        | 59 +++++++++++++++++++++++++++++---
 drivers/scsi/ufs/ufshcd.h        |  8 +++++
 6 files changed, 117 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufs-hisi.c +8 -0
index 5d6487350a6c..fa9d4d1c43c9 100644
@@ -475,6 +475,14 @@ static int ufs_hisi_init_common(struct ufs_hba *hba)
 	if (!host)
 		return -ENOMEM;
 
+	/*
+	 * Inline crypto is currently broken with ufs-hisi because the keyslots
+	 * overlap with the vendor-specific SYS CTRL registers -- and even if
+	 * software uses only non-overlapping keyslots, the kernel crashes when
+	 * programming a key or a UFS error occurs on the first encrypted I/O.
+	 */
+	hba->quirks |= UFSHCD_QUIRK_BROKEN_CRYPTO;
+
 	host->hba = hba;
 	ufshcd_set_variant(hba, host);
 
drivers/scsi/ufs/ufs-qcom.c +7 -0
index c69c29a1ceb9..4b2ec3745a16 100644
@@ -1002,6 +1002,13 @@ static void ufs_qcom_advertise_quirks(struct ufs_hba *hba)
 				| UFSHCD_QUIRK_DME_PEER_ACCESS_AUTO_MODE
 				| UFSHCD_QUIRK_BROKEN_PA_RXHSUNTERMCAP);
 	}
+
+	/*
+	 * Inline crypto is currently broken with ufs-qcom at least because the
+	 * device tree doesn't include the crypto registers.  There are likely
+	 * to be other issues that will need to be addressed too.
+	 */
+	hba->quirks |= UFSHCD_QUIRK_BROKEN_CRYPTO;
 }
 
 static void ufs_qcom_set_caps(struct ufs_hba *hba)
drivers/scsi/ufs/ufshcd-crypto.c +26 -0
index 1b8e14d30c04..cd7ca50a1dd9 100644
@@ -365,3 +365,29 @@ void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
 {
 	blk_ksm_destroy(&hba->ksm);
 }
+
+int ufshcd_prepare_lrbp_crypto(struct ufs_hba *hba,
+			       struct scsi_cmnd *cmd,
+			       struct ufshcd_lrb *lrbp)
+{
+	struct rq_crypt_ctx *rc = &cmd->request->rq_crypt_ctx;
+	struct bio_crypt_ctx *bc = rc->bc;
+
+	lrbp->crypto_enable = false;
+
+	if (WARN_ON(!(hba->caps & UFSHCD_CAP_CRYPTO))) {
+		/*
+		 * Upper layer asked us to do inline encryption
+		 * but that isn't enabled, so we fail this request.
+		 */
+		return -EINVAL;
+	}
+	if (!ufshcd_keyslot_valid(hba, rc->keyslot))
+		return -EINVAL;
+
+	lrbp->crypto_enable = true;
+	lrbp->crypto_key_slot = rc->keyslot;
+	lrbp->data_unit_num = bc->bc_dun[0];
+
+	return 0;
+}
drivers/scsi/ufs/ufshcd-crypto.h +14 -0
index 8270c0c5081a..c76f93ede51c 100644
@@ -16,6 +16,15 @@ static inline bool ufshcd_hba_is_crypto_supported(struct ufs_hba *hba)
 	return hba->crypto_capabilities.reg_val != 0;
 }
 
+int ufshcd_prepare_lrbp_crypto(struct ufs_hba *hba,
+			       struct scsi_cmnd *cmd,
+			       struct ufshcd_lrb *lrbp);
+
+static inline bool ufshcd_lrbp_crypto_enabled(struct ufshcd_lrb *lrbp)
+{
+	return lrbp->crypto_enable;
+}
+
 void ufshcd_crypto_enable(struct ufs_hba *hba);
 
 void ufshcd_crypto_disable(struct ufs_hba *hba);
@@ -49,6 +58,11 @@ static inline void ufshcd_crypto_setup_rq_keyslot_manager(struct ufs_hba *hba,
 static inline void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba)
 { }
 
+static inline bool ufshcd_lrbp_crypto_enabled(struct ufshcd_lrb *lrbp)
+{
+	return false;
+}
+
 #endif /* CONFIG_SCSI_UFS_CRYPTO */
 
 #endif /* _UFSHCD_CRYPTO_H */
drivers/scsi/ufs/ufshcd.c +54 -5

120 lines not on this page · raw message

drivers/scsi/ufs/ufshcd.h +8 -0
index 7b8a87418f0c..c8f948aa5e3d 100644
@@ -168,6 +168,9 @@ struct ufs_pm_lvl_states {
  * @intr_cmd: Interrupt command (doesn't participate in interrupt aggregation)
  * @issue_time_stamp: time stamp for debug purposes
  * @compl_time_stamp: time stamp for statistics
+ * @crypto_enable: whether or not the request needs inline crypto operations
+ * @crypto_key_slot: the key slot to use for inline crypto
+ * @data_unit_num: the data unit number for the first block for inline crypto
  * @req_abort_skip: skip request abort task flag
  */
 struct ufshcd_lrb {
@@ -192,6 +195,11 @@ struct ufshcd_lrb {
 	bool intr_cmd;
 	ktime_t issue_time_stamp;
 	ktime_t compl_time_stamp;
+#if IS_ENABLED(CONFIG_SCSI_UFS_CRYPTO)
+	bool crypto_enable;
+	u8 crypto_key_slot;
+	u64 data_unit_num;
+#endif /* CONFIG_SCSI_UFS_CRYPTO */
 
 	bool req_abort_skip;
 };
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#24 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Quoted 9 lines · Satya Tangirala

Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.

Also, introduce UFSHCD_QUIRK_BROKEN_CRYPTO that certain UFS drivers that don't yet support inline encryption need to use - taken from patches by John Stultz <[email protected]>
(https://android-review.googlesource.com/c/kernel/common/+/1162224/5)
(https://android-review.googlesource.com/c/kernel/common/+/1162225/5)
(https://android-review.googlesource.com/c/kernel/common/+/1164506/1)

Between all these quirks, with what upstream SOC does this feature actually work?

maintainer
#25 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Satya Tangirala

Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.

Also, introduce UFSHCD_QUIRK_BROKEN_CRYPTO that certain UFS drivers that don't yet support inline encryption need to use - taken from patches by John Stultz <[email protected]>
(https://android-review.googlesource.com/c/kernel/common/+/1162224/5)
(https://android-review.googlesource.com/c/kernel/common/+/1162225/5)
(https://android-review.googlesource.com/c/kernel/common/+/1164506/1)

Quoted 2 lines

Between all these quirks, with what upstream SOC does this feature actually work?

It will work on DragonBoard 845c, i.e. Qualcomm's Snapdragon 845 SoC, if we apply my patchset
https://lkml.kernel.org/linux-block/[email protected]/.
It's currently based on Satya's v6 patchset, but I'll be rebasing it onto v7 and resending. It uses all the UFS standard crypto code that Satya is adding except for ufshcd_program_key(), which has to be replaced with a vendor-specific operation. It does also add vendor-specific code to ufs-qcom to initialize the crypto hardware, but that's in addition to the standard code, not replacing it.

DragonBoard 845c is a commercially available development board that boots the mainline kernel (modulo two arm-smmu IOMMU patches that Linaro is working on), so I think it counts as an "upstream SoC".

That's all that we currently have the hardware to verify ourselves, though Mediatek says that Satya's patches are working on their hardware too. And the UFS controller on Mediatek SoCs is supported by the upstream kernel via ufs-mediatek. But I don't know whether it just works exactly as-is or whether they needed to patch ufs-mediatek too. Stanley or Kuohong, can you confirm?

We're also hoping that the patches are usable with the UFS controllers from Cadence Design Systems and Synopsys, which have upstream kernel support in drivers/scsi/ufs/cdns-pltfrm.c and drivers/scsi/ufs/ufshcd-dwc.c. But we don't currently have a way to verify this. But in 2018, both companies had tried to get the UFS v2.1 standard crypto support upstream, so presumably they must have implemented it in their hardware. +Cc the people who were working on that.

- Eric

#26 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Hi,

Quoted 28 lines · Eric Biggers
Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Satya Tangirala

Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.

Also, introduce UFSHCD_QUIRK_BROKEN_CRYPTO that certain UFS drivers that don't yet support inline encryption need to use - taken from patches by John Stultz <[email protected]>
(https://android-review.googlesource.com/c/kernel/common/+/1162224/5)
(https://android-review.googlesource.com/c/kernel/common/+/1162225/5)
(https://android-review.googlesource.com/c/kernel/common/+/1164506/1)

Quoted 2 lines

Between all these quirks, with what upstream SOC does this feature actually work?

It will work on DragonBoard 845c, i.e. Qualcomm's Snapdragon 845 SoC, if we apply my patchset
https://lkml.kernel.org/linux-block/[email protected]/.
It's currently based on Satya's v6 patchset, but I'll be rebasing it onto v7 and resending. It uses all the UFS standard crypto code that Satya is adding except for ufshcd_program_key(), which has to be replaced with a vendor-specific operation. It does also add vendor-specific code to ufs-qcom to initialize the crypto hardware, but that's in addition to the standard code, not replacing it.

DragonBoard 845c is a commercially available development board that boots the mainline kernel (modulo two arm-smmu IOMMU patches that Linaro is working on), so I think it counts as an "upstream SoC".

Quoted 5 lines

That's all that we currently have the hardware to verify ourselves, though Mediatek says that Satya's patches are working on their hardware too. And the UFS controller on Mediatek SoCs is supported by the upstream kernel via ufs-mediatek. But I don't know whether it just works exactly as-is or whether they needed to patch ufs-mediatek too. Stanley or Kuohong, can you confirm?

Yes, MediaTek is keeping work closely with inline encryption patch sets. Currently the v6 version can work well (without UFSHCD_QUIRK_BROKEN_CRYPTO quirk) at least in our MT6779 SoC platform which basic SoC support and some other peripheral drivers are under upstreaming as below link,

https://patchwork.kernel.org/project/linux-mediatek/list/?state=%
2A&q=6779&series=&submitter=&delegate=&archive=both

The integration with inline encryption patch set needs to patch ufs-mediatek and patches are ready in downstream. We plan to upstream them soon after inline encryption patch sets get merged.

Quoted 9 lines

We're also hoping that the patches are usable with the UFS controllers from Cadence Design Systems and Synopsys, which have upstream kernel support in drivers/scsi/ufs/cdns-pltfrm.c and drivers/scsi/ufs/ufshcd-dwc.c. But we don't currently have a way to verify this. But in 2018, both companies had tried to get the UFS v2.1 standard crypto support upstream, so presumably they must have implemented it in their hardware. +Cc the people who were working on that.

- Eric

Thanks,
Stanley Chu

#27 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Quoted 6 lines · Stanley Chu

Yes, MediaTek is keeping work closely with inline encryption patch sets. Currently the v6 version can work well (without UFSHCD_QUIRK_BROKEN_CRYPTO quirk) at least in our MT6779 SoC platform which basic SoC support and some other peripheral drivers are under upstreaming as below link,

Quoted 6 lines

https://patchwork.kernel.org/project/linux-mediatek/list/?state=%
2A&q=6779&series=&submitter=&delegate=&archive=both

The integration with inline encryption patch set needs to patch ufs-mediatek and patches are ready in downstream. We plan to upstream them soon after inline encryption patch sets get merged.

What amount of support do you need in ufs-mediatek? It seems like pretty much every ufs low-level driver needs some kind of specific support now, right? I wonder if we should instead opt into the support instead of all the quirking here.

#28 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Hi Christoph,

Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Stanley Chu

Yes, MediaTek is keeping work closely with inline encryption patch sets. Currently the v6 version can work well (without UFSHCD_QUIRK_BROKEN_CRYPTO quirk) at least in our MT6779 SoC platform which basic SoC support and some other peripheral drivers are under upstreaming as below link,

https://patchwork.kernel.org/project/linux-mediatek/list/?state=%
2A&q=6779&series=&submitter=&delegate=&archive=both

Quoted 8 lines
Quoted 3 lines

The integration with inline encryption patch set needs to patch ufs-mediatek and patches are ready in downstream. We plan to upstream them soon after inline encryption patch sets get merged.

What amount of support do you need in ufs-mediatek? It seems like pretty much every ufs low-level driver needs some kind of specific support now, right? I wonder if we should instead opt into the support instead of all the quirking here.

The patch in ufs-mediatek is aimed to issue vendor-specific SMC calls for host initialization and configuration. This is because MediaTek UFS host has some "secure-protected" registers/features which need to be accessed/switched in secure world.

Such protection is not mentioned by UFSHCI specifications thus inline encryption patch set assumes that every registers in UFSHCI can be accessed normally in kernel. This makes sense and surely the patchset can work fine in any "standard" UFS host. However if host has special design then it can work normally only if some vendor-specific treatment is applied.

I think one of the reason to apply UFSHCD_QUIRK_BROKEN_CRYPTO quirk in ufs-qcom host is similar to above case.

Thanks,
Stanley Chu

maintainer
#29 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Quoted 34 lines · Stanley Chu

Hi Christoph,

Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Stanley Chu

Yes, MediaTek is keeping work closely with inline encryption patch sets. Currently the v6 version can work well (without UFSHCD_QUIRK_BROKEN_CRYPTO quirk) at least in our MT6779 SoC platform which basic SoC support and some other peripheral drivers are under upstreaming as below link,

https://patchwork.kernel.org/project/linux-mediatek/list/?state=%
2A&q=6779&series=&submitter=&delegate=&archive=both

Quoted 8 lines
Quoted 3 lines

The integration with inline encryption patch set needs to patch ufs-mediatek and patches are ready in downstream. We plan to upstream them soon after inline encryption patch sets get merged.

What amount of support do you need in ufs-mediatek? It seems like pretty much every ufs low-level driver needs some kind of specific support now, right? I wonder if we should instead opt into the support instead of all the quirking here.

The patch in ufs-mediatek is aimed to issue vendor-specific SMC calls for host initialization and configuration. This is because MediaTek UFS host has some "secure-protected" registers/features which need to be accessed/switched in secure world.

Such protection is not mentioned by UFSHCI specifications thus inline encryption patch set assumes that every registers in UFSHCI can be accessed normally in kernel. This makes sense and surely the patchset can work fine in any "standard" UFS host. However if host has special design then it can work normally only if some vendor-specific treatment is applied.

Quoted 2 lines

I think one of the reason to apply UFSHCD_QUIRK_BROKEN_CRYPTO quirk in ufs-qcom host is similar to above case.

So, I had originally assumed that most kernel developers would prefer to make the UFS crypto support opt-out rather than opt-in, since that matches the normal Linux way of doing things. I.e. normally the kernel's default assumption is that devices implement the relevant standard, and only when a device is known to deviate from the standard does the driver apply quirks.

But indeed, as we've investigated more vendors' UFS hardware, it seems that everyone has some quirk that needs to be handled in the platform driver:

- ufs-qcom (tested on DragonBoard 845c with upstream kernel) needs vendor-specific crypto initialization logic and SMC calls to set keys

- ufs-mediatek needs the quirks that Stanley mentioned above

- ufs-hisi (tested on Hikey960 with upstream kernel) needs to write a vendor-specific register to use high keyslots, but even then I still couldn't get the crypto support working correctly.

I'm not sure about the UFS controllers from Synopsys, Cadence, or Samsung, all of which apparently have implemented some form of the crypto support too. But I wouldn't get my hopes up that everyone followed the UFS standard precisely.

So if there are no objections, IMO we should make the crypto support opt-in.

That makes it even more important to upstream the crypto support for specific hardware like ufs-qcom and ufs-mediatek, since otherwise the ufshcd-crypto code would be unusable even theoretically. I'm volunteering to handle ufs-qcom with
https://lkml.kernel.org/linux-block/[email protected]/.
Stanley, could you send out ufs-mediatek support as an RFC so people can see better what it involves?

- Eric

#30 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Hi Eric,

Quoted 60 lines · Eric Biggers
Quoted 34 lines · Stanley Chu

Hi Christoph,

Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Stanley Chu

Yes, MediaTek is keeping work closely with inline encryption patch sets. Currently the v6 version can work well (without UFSHCD_QUIRK_BROKEN_CRYPTO quirk) at least in our MT6779 SoC platform which basic SoC support and some other peripheral drivers are under upstreaming as below link,

https://patchwork.kernel.org/project/linux-mediatek/list/?state=%
2A&q=6779&series=&submitter=&delegate=&archive=both

Quoted 8 lines
Quoted 3 lines

The integration with inline encryption patch set needs to patch ufs-mediatek and patches are ready in downstream. We plan to upstream them soon after inline encryption patch sets get merged.

What amount of support do you need in ufs-mediatek? It seems like pretty much every ufs low-level driver needs some kind of specific support now, right? I wonder if we should instead opt into the support instead of all the quirking here.

The patch in ufs-mediatek is aimed to issue vendor-specific SMC calls for host initialization and configuration. This is because MediaTek UFS host has some "secure-protected" registers/features which need to be accessed/switched in secure world.

Such protection is not mentioned by UFSHCI specifications thus inline encryption patch set assumes that every registers in UFSHCI can be accessed normally in kernel. This makes sense and surely the patchset can work fine in any "standard" UFS host. However if host has special design then it can work normally only if some vendor-specific treatment is applied.

Quoted 2 lines

I think one of the reason to apply UFSHCD_QUIRK_BROKEN_CRYPTO quirk in ufs-qcom host is similar to above case.

So, I had originally assumed that most kernel developers would prefer to make the UFS crypto support opt-out rather than opt-in, since that matches the normal Linux way of doing things. I.e. normally the kernel's default assumption is that devices implement the relevant standard, and only when a device is known to deviate from the standard does the driver apply quirks.

But indeed, as we've investigated more vendors' UFS hardware, it seems that everyone has some quirk that needs to be handled in the platform driver:

- ufs-qcom (tested on DragonBoard 845c with upstream kernel) needs vendor-specific crypto initialization logic and SMC calls to set keys

- ufs-mediatek needs the quirks that Stanley mentioned above

- ufs-hisi (tested on Hikey960 with upstream kernel) needs to write a vendor-specific register to use high keyslots, but even then I still couldn't get the crypto support working correctly.

I'm not sure about the UFS controllers from Synopsys, Cadence, or Samsung, all of which apparently have implemented some form of the crypto support too. But I wouldn't get my hopes up that everyone followed the UFS standard precisely.

Quoted 8 lines

So if there are no objections, IMO we should make the crypto support opt-in.

That makes it even more important to upstream the crypto support for specific hardware like ufs-qcom and ufs-mediatek, since otherwise the ufshcd-crypto code would be unusable even theoretically. I'm volunteering to handle ufs-qcom with
https://lkml.kernel.org/linux-block/[email protected]/.
Stanley, could you send out ufs-mediatek support as an RFC so people can see better what it involves?

Sure, I will send out our RFC patches. Please allow me some time for submission.

Thanks,
Stanley Chu

Quoted 1 line

- Eric

#31 lore raw

Re: [PATCH v7 6/9] scsi: ufs: Add inline encryption support to UFS

Hi Eric and all,

Quoted 18 lines · Stanley Chu

Hi Eric,

Quoted 5 lines · Eric Biggers

I'm not sure about the UFS controllers from Synopsys, Cadence, or Samsung, all of which apparently have implemented some form of the crypto support too. But I wouldn't get my hopes up that everyone followed the UFS standard precisely.

Quoted 8 lines

So if there are no objections, IMO we should make the crypto support opt-in.

That makes it even more important to upstream the crypto support for specific hardware like ufs-qcom and ufs-mediatek, since otherwise the ufshcd-crypto code would be unusable even theoretically. I'm volunteering to handle ufs-qcom with
https://lkml.kernel.org/linux-block/[email protected]/.
Stanley, could you send out ufs-mediatek support as an RFC so people can see better what it involves?

Quoted 2 lines

Sure, I will send out our RFC patches. Please allow me some time for submission.

The ufs-mediatek RFC patch is uploaded as
https://patchwork.kernel.org/patch/11415051/

This patch is rebased to the latest wip-inline-encryption branch in Eric Biggers's git:
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/

Thanks,
Stanley Chu

#32 lore raw

PATCH v7 7/9 fscrypt: add inline encryption support

In reply to Satya Tangirala

Add support for inline encryption to fs/crypto/. With "inline encryption", the block layer handles the decryption/encryption as part of the bio, instead of the filesystem doing the crypto itself via Linux's crypto API. This model is needed in order to take advantage of the inline encryption hardware present on most modern mobile SoCs.

To use inline encryption, the filesystem needs to be mounted with '-o inlinecrypt'. The contents of any encrypted files will then be encrypted using blk-crypto, instead of using the traditional filesystem-layer crypto. Fscrypt still provides the key and IV to use, and the actual ciphertext on-disk is still the same; therefore it's testable using the existing fscrypt ciphertext verification tests.

Note that since blk-crypto has a fallack to Linux's crypto API, and also supports all the encryption modes currently supported by fscrypt, this feature is usable and testable even without actual inline encryption hardware.

Per-filesystem changes will be needed to set encryption contexts when submitting bios and to implement the 'inlinecrypt' mount option. This patch just adds the common code.

---
 fs/crypto/Kconfig           |   6 +
 fs/crypto/Makefile          |   1 +
 fs/crypto/bio.c             |  55 ++++++
 fs/crypto/crypto.c          |   2 +-
 fs/crypto/fname.c           |   4 +-
 fs/crypto/fscrypt_private.h | 121 ++++++++++++--
 fs/crypto/inline_crypt.c    | 324 ++++++++++++++++++++++++++++++++++++
 fs/crypto/keyring.c         |   4 +-
 fs/crypto/keysetup.c        |  95 +++++++----
 fs/crypto/keysetup_v1.c     |  16 +-
 include/linux/fs.h          |   1 +
 include/linux/fscrypt.h     |  57 +++++++
 12 files changed, 629 insertions(+), 57 deletions(-)
 create mode 100644 fs/crypto/inline_crypt.c
fs/crypto/Kconfig +6 -0
index 8046d7c7a3e9..f1f11a6228eb 100644
@@ -24,3 +24,9 @@ config FS_ENCRYPTION_ALGS
 	select CRYPTO_SHA256
 	select CRYPTO_SHA512
 	select CRYPTO_XTS
+
+config FS_ENCRYPTION_INLINE_CRYPT
+	bool "Enable fscrypt to use inline crypto"
+	depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION
+	help
+	  Enable fscrypt to use inline encryption hardware if available.
fs/crypto/Makefile +1 -0
index 232e2bb5a337..652c7180ec6d 100644
@@ -11,3 +11,4 @@ fscrypto-y := crypto.o \
 	      policy.o
 
 fscrypto-$(CONFIG_BLOCK) += bio.o
+fscrypto-$(CONFIG_FS_ENCRYPTION_INLINE_CRYPT) += inline_crypt.o
fs/crypto/bio.c +55 -0

79 lines not on this page · raw message

fs/crypto/crypto.c +1 -1
index 1ecaac7ee3cb..263bc676c73d 100644
@@ -95,7 +95,7 @@ int fscrypt_crypt_block(const struct inode *inode, fscrypt_direction_t rw,
 	DECLARE_CRYPTO_WAIT(wait);
 	struct scatterlist dst, src;
 	struct fscrypt_info *ci = inode->i_crypt_info;
-	struct crypto_skcipher *tfm = ci->ci_ctfm;
+	struct crypto_skcipher *tfm = ci->ci_key.tfm;
 	int res = 0;
 
 	if (WARN_ON_ONCE(len <= 0))
fs/crypto/fname.c +2 -2
index 4c212442a8f7..0fca2d7a5645 100644
@@ -117,7 +117,7 @@ int fscrypt_fname_encrypt(const struct inode *inode, const struct qstr *iname,
 	struct skcipher_request *req = NULL;
 	DECLARE_CRYPTO_WAIT(wait);
 	const struct fscrypt_info *ci = inode->i_crypt_info;
-	struct crypto_skcipher *tfm = ci->ci_ctfm;
+	struct crypto_skcipher *tfm = ci->ci_key.tfm;
 	union fscrypt_iv iv;
 	struct scatterlist sg;
 	int res;
@@ -170,7 +170,7 @@ static int fname_decrypt(const struct inode *inode,
 	DECLARE_CRYPTO_WAIT(wait);
 	struct scatterlist src_sg, dst_sg;
 	const struct fscrypt_info *ci = inode->i_crypt_info;
-	struct crypto_skcipher *tfm = ci->ci_ctfm;
+	struct crypto_skcipher *tfm = ci->ci_key.tfm;
 	union fscrypt_iv iv;
 	int res;
 
fs/crypto/fscrypt_private.h +108 -13

178 lines not on this page · raw message

fs/crypto/inline_crypt.c new file +324 -0

324 lines not on this page · raw message

fs/crypto/keyring.c +2 -2
index ab41b25d4fa1..d8ab33f631ba 100644
@@ -44,8 +44,8 @@ static void free_master_key(struct fscrypt_master_key *mk)
 	wipe_master_key_secret(&mk->mk_secret);
 
 	for (i = 0; i <= __FSCRYPT_MODE_MAX; i++) {
-		crypto_free_skcipher(mk->mk_direct_tfms[i]);
-		crypto_free_skcipher(mk->mk_iv_ino_lblk_64_tfms[i]);
+		fscrypt_destroy_prepared_key(&mk->mk_direct_keys[i]);
+		fscrypt_destroy_prepared_key(&mk->mk_iv_ino_lblk_64_keys[i]);
 	}
 
 	key_put(mk->mk_users);
fs/crypto/keysetup.c +65 -30

180 lines not on this page · raw message

fs/crypto/keysetup_v1.c +7 -9
index 801b48c0cd7f..59c520b200cb 100644
@@ -146,7 +146,7 @@ struct fscrypt_direct_key {
 	struct hlist_node		dk_node;
 	refcount_t			dk_refcount;
 	const struct fscrypt_mode	*dk_mode;
-	struct crypto_skcipher		*dk_ctfm;
+	struct fscrypt_prepared_key	dk_key;
 	u8				dk_descriptor[FSCRYPT_KEY_DESCRIPTOR_SIZE];
 	u8				dk_raw[FSCRYPT_MAX_KEY_SIZE];
 };
@@ -154,7 +154,7 @@ struct fscrypt_direct_key {
 static void free_direct_key(struct fscrypt_direct_key *dk)
 {
 	if (dk) {
-		crypto_free_skcipher(dk->dk_ctfm);
+		fscrypt_destroy_prepared_key(&dk->dk_key);
 		kzfree(dk);
 	}
 }
@@ -199,6 +199,8 @@ find_or_insert_direct_key(struct fscrypt_direct_key *to_insert,
 			continue;
 		if (ci->ci_mode != dk->dk_mode)
 			continue;
+		if (!fscrypt_is_key_prepared(&dk->dk_key, ci))
+			continue;
 		if (crypto_memneq(raw_key, dk->dk_raw, ci->ci_mode->keysize))
 			continue;
 		/* using existing tfm with same (descriptor, mode, raw_key) */
@@ -231,13 +233,9 @@ fscrypt_get_direct_key(const struct fscrypt_info *ci, const u8 *raw_key)
 		return ERR_PTR(-ENOMEM);
 	refcount_set(&dk->dk_refcount, 1);
 	dk->dk_mode = ci->ci_mode;
-	dk->dk_ctfm = fscrypt_allocate_skcipher(ci->ci_mode, raw_key,
-						ci->ci_inode);
-	if (IS_ERR(dk->dk_ctfm)) {
-		err = PTR_ERR(dk->dk_ctfm);
-		dk->dk_ctfm = NULL;
+	err = fscrypt_prepare_key(&dk->dk_key, raw_key, ci);
+	if (err)
 		goto err_free_dk;
-	}
 	memcpy(dk->dk_descriptor, ci->ci_policy.v1.master_key_descriptor,
 	       FSCRYPT_KEY_DESCRIPTOR_SIZE);
 	memcpy(dk->dk_raw, raw_key, ci->ci_mode->keysize);
@@ -259,7 +257,7 @@ static int setup_v1_file_key_direct(struct fscrypt_info *ci,
 	if (IS_ERR(dk))
 		return PTR_ERR(dk);
 	ci->ci_direct_key = dk;
-	ci->ci_ctfm = dk->dk_ctfm;
+	ci->ci_key = dk->dk_key;
 	return 0;
 }
 
include/linux/fs.h +1 -0
index 3cd4fe6b845e..2331ff0464b2 100644
@@ -1370,6 +1370,7 @@ extern int send_sigurg(struct fown_struct *fown);
 #define SB_NODIRATIME	2048	/* Do not update directory access times */
 #define SB_SILENT	32768
 #define SB_POSIXACL	(1<<16)	/* VFS does not apply the umask */
+#define SB_INLINE_CRYPT	(1<<17)	/* inodes in SB use blk-crypto */
 #define SB_KERNMOUNT	(1<<22) /* this is a kern_mount call */
 #define SB_I_VERSION	(1<<23) /* Update inode I_version field */
 #define SB_LAZYTIME	(1<<25) /* Update the on-disk [acm]times lazily */
include/linux/fscrypt.h +57 -0

69 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
maintainer
#33 lore raw

Re: [PATCH v7 7/9] fscrypt: add inline encryption support

fs/crypto/bio.c line 27

Quoted 9 lines · Satya Tangirala
fs/crypto/bio.c +1 -0
index 4fa18fff9c4e..82d06cf4b94a 100644
@@ -24,6 +24,8 @@
 #include <linux/module.h>
 #include <linux/bio.h>
 #include <linux/namei.h>
+#include <linux/fscrypt.h>

No need to include <linux/fscrypt.h> explicitly here, since everything in fs/crypto/ already gets it via "fscrypt_private.h".

Quoted 9 lines
fs/crypto/bio.c +9 -0
+static int fscrypt_zeroout_range_inline_crypt(const struct inode *inode,
+					      pgoff_t lblk, sector_t pblk,
+					      unsigned int len)
+{
+	const unsigned int blockbits = inode->i_blkbits;
+	const unsigned int blocks_per_page = 1 << (PAGE_SHIFT - blockbits);
+	struct bio *bio;
+	int ret, err = 0;
+	int num_pages = 0;

fs/crypto/bio.c void fscrypt_decrypt_bio(struct bio *bio) · line 64

Quoted 10 lines
fs/crypto/bio.c +10 -0
+
+	/* This always succeeds since __GFP_DIRECT_RECLAIM is set. */
+	bio = bio_alloc(GFP_NOFS, BIO_MAX_PAGES);
+
+	while (len) {
+		unsigned int blocks_this_page = min(len, blocks_per_page);
+		unsigned int bytes_this_page = blocks_this_page << blockbits;
+
+		if (num_pages == 0) {
+			fscrypt_set_bio_crypt_ctx(bio, inode, lblk, GFP_NOIO);

This should use GFP_NOFS rather than the stricter GFP_NOIO.

Quoted 8 lines
fs/crypto/bio.c +8 -0
+			bio_set_dev(bio, inode->i_sb->s_bdev);
+			bio->bi_iter.bi_sector =
+					pblk << (blockbits - SECTOR_SHIFT);
+			bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
+		}
+		ret = bio_add_page(bio, ZERO_PAGE(0), bytes_this_page, 0);
+		if (WARN_ON(ret != bytes_this_page)) {
+			err = -EIO;

fs/crypto/bio.c void fscrypt_decrypt_bio(struct bio *bio) · line 82

Quoted 10 lines
fs/crypto/bio.c +10 -0
+			goto out;
+		}
+		num_pages++;
+		len -= blocks_this_page;
+		lblk += blocks_this_page;
+		pblk += blocks_this_page;
+		if (num_pages == BIO_MAX_PAGES || !len) {
+			err = submit_bio_wait(bio);
+			if (!err && bio->bi_status)
+				err = -EIO;

submit_bio_wait() already checks bi_status and reflects it in the returned error, so checking it again here is redundant.

Quoted 5 lines
fs/crypto/bio.c +1 -0
@@ -69,12 +119,17 @@ int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk,
 	unsigned int nr_pages;
 	unsigned int i;
 	unsigned int offset;
+	const bool inlinecrypt = fscrypt_inode_uses_inline_crypto(inode);

fs/crypto/bio.c int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk, · line 132

Quoted 10 lines
fs/crypto/bio.c +4 -0
 	struct bio *bio;
 	int ret, err;
 
 	if (len == 0)
 		return 0;
 
+	if (inlinecrypt)
+		return fscrypt_zeroout_range_inline_crypt(inode, lblk, pblk,
+							  len);
+

No need for the 'inlinecrypt' bool variable. Just do:

if (fscrypt_inode_uses_inline_crypto(inode))

FYI, I had suggested a merge resolution to use here which didn't have the above problems. Looks like you missed it?
https://lkml.kernel.org/linux-block/[email protected]/

- Eric

maintainer
#34 lore raw

Re: [PATCH v7 7/9] fscrypt: add inline encryption support

In reply to Satya Tangirala
Quoted 23 lines · Satya Tangirala
fs/crypto/keysetup.c +4 -0
index 65cb09fa6ead..7c157130c16a 100644
@@ -19,6 +19,8 @@ struct fscrypt_mode fscrypt_modes[] = {
 		.cipher_str = "xts(aes)",
 		.keysize = 64,
 		.ivsize = 16,
+		.blk_crypto_mode = BLK_ENCRYPTION_MODE_AES_256_XTS,
+		.blk_crypto_dun_bytes_required = 8,
 	},
 	[FSCRYPT_MODE_AES_256_CTS] = {
 		.friendly_name = "AES-256-CTS-CBC",
@@ -31,6 +33,8 @@ struct fscrypt_mode fscrypt_modes[] = {
 		.cipher_str = "essiv(cbc(aes),sha256)",
 		.keysize = 16,
 		.ivsize = 16,
+		.blk_crypto_mode = BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV,
+		.blk_crypto_dun_bytes_required = 8,
 	},
 	[FSCRYPT_MODE_AES_128_CTS] = {
 		.friendly_name = "AES-128-CTS-CBC",

fs/crypto/keysetup.c struct fscrypt_mode fscrypt_modes[] = { · line 53

Quoted 7 lines
fs/crypto/keysetup.c +2 -0
@@ -43,6 +47,8 @@ struct fscrypt_mode fscrypt_modes[] = {
 		.cipher_str = "adiantum(xchacha12,aes)",
 		.keysize = 32,
 		.ivsize = 32,
+		.blk_crypto_mode = BLK_ENCRYPTION_MODE_ADIANTUM,
+		.blk_crypto_dun_bytes_required = 24,
 	},
 };

The DUN bytes required is actually determined by the IV generation method too. Currently fscrypt has the following combinations:

	AES-256-XTS: 8 bytes
	AES-128-CBC-ESSIV: 8 bytes
	Adiantum without DIRECT_KEY: 8 bytes
	Adiantum with DIRECT_KEY: 24 bytes

I.e., DIRECT_KEY is only allowed with Adiantum, but not required for it.

So it's technically incorrect to always pass dun_bytes_required=24 for Adiantum.

And it's conceivable that in the future we could add an fscrypt setting that uses AES-256-XTS with 16 IV bytes. Such a setting wouldn't be usable with UFS inline encryption, yet the existing AES-256-XTS settings still would.

So, how about instead of putting .blk_crypto_dun_bytes_required in the crypto_mode table, using logic like:

	dun_bytes_required = 8;
	if (flags & FSCRYPT_POLICY_FLAG_DIRECT_KEY)
		dun_bytes_required += 16;

include/linux/fs.h extern int send_sigurg(struct fown_struct *fown); · line 1376

Quoted 12 lines
include/linux/fs.h +1 -0
index 3cd4fe6b845e..2331ff0464b2 100644
@@ -1370,6 +1370,7 @@ extern int send_sigurg(struct fown_struct *fown);
 #define SB_NODIRATIME	2048	/* Do not update directory access times */
 #define SB_SILENT	32768
 #define SB_POSIXACL	(1<<16)	/* VFS does not apply the umask */
+#define SB_INLINE_CRYPT	(1<<17)	/* inodes in SB use blk-crypto */
 #define SB_KERNMOUNT	(1<<22) /* this is a kern_mount call */
 #define SB_I_VERSION	(1<<23) /* Update inode I_version field */
 #define SB_LAZYTIME	(1<<25) /* Update the on-disk [acm]times lazily */

This flag probably should be called "SB_INLINECRYPT" to match the mount option, which is "inlinecrypt" not "inline_crypt".

Also, the addition of this flag, along with the update to show_sb_opts() in fs/proc_namespace.c which I think is needed, maybe should go in a separate patch whose subject is prefixed with "fs: " to make it clearer to reviewers that this part is a VFS-level change.

- Eric

maintainer
#35 lore raw

Re: [PATCH v7 7/9] fscrypt: add inline encryption support

In reply to Satya Tangirala
Quoted 21 lines · Satya Tangirala
(quoted diff) +21 -0
+/**
+ * fscrypt_inode_uses_inline_crypto - test whether an inode uses inline
+ *				      encryption
+ * @inode: an inode
+ *
+ * Return: true if the inode requires file contents encryption and if the
+ *	   encryption should be done in the block layer via blk-crypto rather
+ *	   than in the filesystem layer.
+ */
+bool fscrypt_inode_uses_inline_crypto(const struct inode *inode)
+{
+	return IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) &&
+		inode->i_crypt_info->ci_inlinecrypt;
+}
+EXPORT_SYMBOL_GPL(fscrypt_inode_uses_inline_crypto);
+
+/**
+ * fscrypt_inode_uses_fs_layer_crypto - test whether an inode uses fs-layer
+ *					encryption
+ * @inode: an inode
+ *

Review

Quoted 10 lines
(quoted diff) +10 -0
+ * Return: true if the inode requires file contents encryption and if the
+ *	   encryption should be done in the filesystem layer rather than in the
+ *	   block layer via blk-crypto.
+ */
+bool fscrypt_inode_uses_fs_layer_crypto(const struct inode *inode)
+{
+	return IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) &&
+		!inode->i_crypt_info->ci_inlinecrypt;
+}
+EXPORT_SYMBOL_GPL(fscrypt_inode_uses_fs_layer_crypto);

We should use the fscrypt_needs_contents_encryption() helper function which I added in v5.6. I.e.:

fs/crypto/inline_crypt.c +3 -3
index 72692366795aa9..36510802a3665a 100644
@@ -32,7 +32,7 @@ void fscrypt_select_encryption_impl(struct fscrypt_info *ci)
 	struct super_block *sb = inode->i_sb;
 
 	/* The file must need contents encryption, not filenames encryption */
-	if (!S_ISREG(inode->i_mode))
+	if (!fscrypt_needs_contents_encryption(inode))
 		return;
 
 	/* blk-crypto must implement the needed encryption algorithm */
@@ -148,7 +148,7 @@ void fscrypt_destroy_inline_crypt_key(struct fscrypt_prepared_key *prep_key)
  */
 bool fscrypt_inode_uses_inline_crypto(const struct inode *inode)
 {
-	return IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) &&
+	return fscrypt_needs_contents_encryption(inode) &&
 		inode->i_crypt_info->ci_inlinecrypt;
 }
 EXPORT_SYMBOL_GPL(fscrypt_inode_uses_inline_crypto);
@@ -164,7 +164,7 @@ EXPORT_SYMBOL_GPL(fscrypt_inode_uses_inline_crypto);
  */
 bool fscrypt_inode_uses_fs_layer_crypto(const struct inode *inode)
 {
-	return IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) &&
+	return fscrypt_needs_contents_encryption(inode) &&
 		!inode->i_crypt_info->ci_inlinecrypt;
 }
 EXPORT_SYMBOL_GPL(fscrypt_inode_uses_fs_layer_crypto);
include/linux/fscrypt.h +1 -1
index 2a84131ab270fd..1d9810eb88b113 100644
@@ -528,7 +528,7 @@ static inline bool fscrypt_inode_uses_inline_crypto(const struct inode *inode)
 
 static inline bool fscrypt_inode_uses_fs_layer_crypto(const struct inode *inode)
 {
-	return IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode);
+	return fscrypt_needs_contents_encryption(inode);
 }
 
 static inline void fscrypt_set_bio_crypt_ctx(struct bio *bio,
#36 lore raw

PATCH v7 8/9 f2fs: add inline encryption support

In reply to Satya Tangirala

Wire up f2fs to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:

- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.

- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.

- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.

- Not doing filesystem-layer crypto on inline-encrypted files.

---
 Documentation/filesystems/f2fs.txt |  4 ++
 fs/f2fs/compress.c                 |  2 +-
 fs/f2fs/data.c                     | 67 +++++++++++++++++++++++++-----
 fs/f2fs/f2fs.h                     |  3 ++
 fs/f2fs/super.c                    | 35 ++++++++++++++++
 5 files changed, 100 insertions(+), 11 deletions(-)
Documentation/filesystems/f2fs.txt +4 -0
index 4eb3e2ddd00e..e8d6a9240f55 100644
@@ -246,6 +246,10 @@ compress_extension=%s  Support adding specified extension, so that f2fs can enab
                        on compression extension list and enable compression on
                        these file by default rather than to enable it via ioctl.
                        For other files, we can still enable compression via ioctl.
+inlinecrypt
+                       Use blk-crypto, rather than fscrypt, for file content
+                       en/decryption. See
+                       Documentation/block/inline-encryption.rst
 
 ================================================================================
 DEBUGFS ENTRIES
fs/f2fs/compress.c +1 -1
index d8a64be90a50..6ae313ea5432 100644
@@ -775,7 +775,7 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
 		.need_lock = LOCK_RETRY,
 		.io_type = io_type,
 		.io_wbc = wbc,
-		.encrypted = f2fs_encrypted_file(cc->inode),
+		.encrypted = fscrypt_inode_uses_fs_layer_crypto(cc->inode),
 	};
 	struct dnode_of_data dn;
 	struct node_info ni;
fs/f2fs/data.c +57 -10

135 lines not on this page · raw message

fs/f2fs/f2fs.h +3 -0
index 5355be6b6755..75817f0dc6f8 100644
@@ -139,6 +139,9 @@ struct f2fs_mount_info {
 	int alloc_mode;			/* segment allocation policy */
 	int fsync_mode;			/* fsync policy */
 	bool test_dummy_encryption;	/* test dummy encryption */
+#ifdef CONFIG_FS_ENCRYPTION
+	bool inlinecrypt;		/* inline encryption enabled */
+#endif
 	block_t unusable_cap;		/* Amount of space allowed to be
 					 * unusable when disabling checkpoint
 					 */
fs/f2fs/super.c +35 -0
index 65a7a432dfee..fb94ea555c30 100644
@@ -137,6 +137,7 @@ enum {
 	Opt_alloc,
 	Opt_fsync,
 	Opt_test_dummy_encryption,
+	Opt_inlinecrypt,
 	Opt_checkpoint_disable,
 	Opt_checkpoint_disable_cap,
 	Opt_checkpoint_disable_cap_perc,
@@ -202,6 +203,7 @@ static match_table_t f2fs_tokens = {
 	{Opt_alloc, "alloc_mode=%s"},
 	{Opt_fsync, "fsync_mode=%s"},
 	{Opt_test_dummy_encryption, "test_dummy_encryption"},
+	{Opt_inlinecrypt, "inlinecrypt"},
 	{Opt_checkpoint_disable, "checkpoint=disable"},
 	{Opt_checkpoint_disable_cap, "checkpoint=disable:%u"},
 	{Opt_checkpoint_disable_cap_perc, "checkpoint=disable:%u%%"},
@@ -790,6 +792,13 @@ static int parse_options(struct super_block *sb, char *options)
 			f2fs_info(sbi, "Test dummy encryption mode enabled");
 #else
 			f2fs_info(sbi, "Test dummy encryption mount option ignored");
+#endif
+			break;
+		case Opt_inlinecrypt:
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+			sb->s_flags |= SB_INLINE_CRYPT;
+#else
+			f2fs_info(sbi, "inline encryption not supported");
 #endif
 			break;
 		case Opt_checkpoint_disable_cap_perc:
@@ -1538,6 +1547,8 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root)
 #ifdef CONFIG_FS_ENCRYPTION
 	if (F2FS_OPTION(sbi).test_dummy_encryption)
 		seq_puts(seq, ",test_dummy_encryption");
+	if (sbi->sb->s_flags & SB_INLINE_CRYPT)
+		seq_puts(seq, ",inlinecrypt");
 #endif
 
 	if (F2FS_OPTION(sbi).alloc_mode == ALLOC_MODE_DEFAULT)
@@ -1568,6 +1579,9 @@ static void default_options(struct f2fs_sb_info *sbi)
 	F2FS_OPTION(sbi).alloc_mode = ALLOC_MODE_DEFAULT;
 	F2FS_OPTION(sbi).fsync_mode = FSYNC_MODE_POSIX;
 	F2FS_OPTION(sbi).test_dummy_encryption = false;
+#ifdef CONFIG_FS_ENCRYPTION
+	sbi->sb->s_flags &= ~SB_INLINE_CRYPT;
+#endif
 	F2FS_OPTION(sbi).s_resuid = make_kuid(&init_user_ns, F2FS_DEF_RESUID);
 	F2FS_OPTION(sbi).s_resgid = make_kgid(&init_user_ns, F2FS_DEF_RESGID);
 	F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZO;
@@ -2415,6 +2429,25 @@ static void f2fs_get_ino_and_lblk_bits(struct super_block *sb,
 	*lblk_bits_ret = 8 * sizeof(block_t);
 }
 
+static int f2fs_get_num_devices(struct super_block *sb)
+{
+	struct f2fs_sb_info *sbi = F2FS_SB(sb);
+
+	if (f2fs_is_multi_device(sbi))
+		return sbi->s_ndevs;
+	return 1;
+}
+
+static void f2fs_get_devices(struct super_block *sb,
+			     struct request_queue **devs)
+{
+	struct f2fs_sb_info *sbi = F2FS_SB(sb);
+	int i;
+
+	for (i = 0; i < sbi->s_ndevs; i++)
+		devs[i] = bdev_get_queue(FDEV(i).bdev);
+}
+
 static const struct fscrypt_operations f2fs_cryptops = {
 	.key_prefix		= "f2fs:",
 	.get_context		= f2fs_get_context,
@@ -2424,6 +2457,8 @@ static const struct fscrypt_operations f2fs_cryptops = {
 	.max_namelen		= F2FS_NAME_LEN,
 	.has_stable_inodes	= f2fs_has_stable_inodes,
 	.get_ino_and_lblk_bits	= f2fs_get_ino_and_lblk_bits,
+	.get_num_devices	= f2fs_get_num_devices,
+	.get_devices		= f2fs_get_devices,
 };
 #endif
 
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#37 lore raw

PATCH v7 9/9 ext4: add inline encryption support

In reply to Satya Tangirala

From: Eric Biggers <[email protected]>

Wire up ext4 to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:

- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.

- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.

Note: submit_bh_wbc() in fs/buffer.c also needed to be patched for this part, since ext4 sometimes uses ll_rw_block() on file data.

- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.

- Not doing filesystem-layer crypto on inline-encrypted files.

---
 Documentation/admin-guide/ext4.rst |  4 ++++
 fs/buffer.c                        |  7 ++++---
 fs/ext4/ext4.h                     |  1 +
 fs/ext4/inode.c                    |  4 ++--
 fs/ext4/page-io.c                  |  6 ++++--
 fs/ext4/readpage.c                 | 11 ++++++++---
 fs/ext4/super.c                    | 19 +++++++++++++++++++
 7 files changed, 42 insertions(+), 10 deletions(-)
Documentation/admin-guide/ext4.rst +4 -0
index 9443fcef1876..b7cd3bdf827e 100644
@@ -395,6 +395,10 @@ When mounting an ext4 filesystem, the following option are accepted:
         Documentation/filesystems/dax.txt.  Note that this option is
         incompatible with data=journal.
 
+  inlinecrypt
+        Use blk-crypto, rather than fscrypt, for file content en/decryption.
+        See Documentation/block/inline-encryption.rst.
+
 Data Mode
 =========
 There are 3 different data modes:
fs/buffer.c +4 -3
index b8d28370cfd7..226f1784eda7 100644
@@ -331,9 +331,8 @@ static void decrypt_bh(struct work_struct *work)
 static void end_buffer_async_read_io(struct buffer_head *bh, int uptodate)
 {
 	/* Decrypt if needed */
-	if (uptodate && IS_ENABLED(CONFIG_FS_ENCRYPTION) &&
-	    IS_ENCRYPTED(bh->b_page->mapping->host) &&
-	    S_ISREG(bh->b_page->mapping->host->i_mode)) {
+	if (uptodate &&
+	    fscrypt_inode_uses_fs_layer_crypto(bh->b_page->mapping->host)) {
 		struct decrypt_bh_ctx *ctx = kmalloc(sizeof(*ctx), GFP_ATOMIC);
 
 		if (ctx) {
@@ -3085,6 +3084,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
 	 */
 	bio = bio_alloc(GFP_NOIO, 1);
 
+	fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
+
 	bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
 	bio_set_dev(bio, bh->b_bdev);
 	bio->bi_write_hint = write_hint;
fs/ext4/ext4.h +1 -0
index 4441331d06cc..3917f4b3ec30 100644
@@ -1151,6 +1151,7 @@ struct ext4_inode_info {
 #define EXT4_MOUNT_JOURNAL_CHECKSUM	0x800000 /* Journal checksums */
 #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT	0x1000000 /* Journal Async Commit */
 #define EXT4_MOUNT_WARN_ON_ERROR	0x2000000 /* Trigger WARN_ON on error */
+#define EXT4_MOUNT_INLINECRYPT		0x4000000 /* Inline encryption support */
 #define EXT4_MOUNT_DELALLOC		0x8000000 /* Delalloc support */
 #define EXT4_MOUNT_DATA_ERR_ABORT	0x10000000 /* Abort on file data write */
 #define EXT4_MOUNT_BLOCK_VALIDITY	0x20000000 /* Block validity checking */
fs/ext4/inode.c +2 -2
index e60aca791d3f..9ac1dd131586 100644
@@ -1089,7 +1089,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len,
 	}
 	if (unlikely(err)) {
 		page_zero_new_buffers(page, from, to);
-	} else if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode)) {
+	} else if (fscrypt_inode_uses_fs_layer_crypto(inode)) {
 		for (i = 0; i < nr_wait; i++) {
 			int err2;
 
@@ -3720,7 +3720,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,
 		/* Uhhuh. Read error. Complain and punt. */
 		if (!buffer_uptodate(bh))
 			goto unlock;
-		if (S_ISREG(inode->i_mode) && IS_ENCRYPTED(inode)) {
+		if (fscrypt_inode_uses_fs_layer_crypto(inode)) {
 			/* We expect the key to be set. */
 			BUG_ON(!fscrypt_has_encryption_key(inode));
 			err = fscrypt_decrypt_pagecache_blocks(page, blocksize,
fs/ext4/page-io.c +4 -2
index 68b39e75446a..0757145a31b2 100644
@@ -404,6 +404,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,
 	 * __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset().
 	 */
 	bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES);
+	fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
 	bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
 	bio_set_dev(bio, bh->b_bdev);
 	bio->bi_end_io = ext4_end_bio;
@@ -420,7 +421,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io,
 {
 	int ret;
 
-	if (io->io_bio && bh->b_blocknr != io->io_next_block) {
+	if (io->io_bio && (bh->b_blocknr != io->io_next_block ||
+			   !fscrypt_mergeable_bio_bh(io->io_bio, bh))) {
 submit_and_retry:
 		ext4_io_submit(io);
 	}
@@ -508,7 +510,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,
 	 * (e.g. holes) to be unnecessarily encrypted, but this is rare and
 	 * can't happen in the common case of blocksize == PAGE_SIZE.
 	 */
-	if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) && nr_to_submit) {
+	if (fscrypt_inode_uses_fs_layer_crypto(inode) && nr_to_submit) {
 		gfp_t gfp_flags = GFP_NOFS;
 		unsigned int enc_bytes = round_up(len, i_blocksize(inode));
 
fs/ext4/readpage.c +8 -3
index c1769afbf799..68eac0aeffad 100644
@@ -195,7 +195,7 @@ static void ext4_set_bio_post_read_ctx(struct bio *bio,
 {
 	unsigned int post_read_steps = 0;
 
-	if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode))
+	if (fscrypt_inode_uses_fs_layer_crypto(inode))
 		post_read_steps |= 1 << STEP_DECRYPT;
 
 	if (ext4_need_verity(inode, first_idx))
@@ -232,6 +232,7 @@ int ext4_mpage_readpages(struct address_space *mapping,
 	const unsigned blkbits = inode->i_blkbits;
 	const unsigned blocks_per_page = PAGE_SIZE >> blkbits;
 	const unsigned blocksize = 1 << blkbits;
+	sector_t next_block;
 	sector_t block_in_file;
 	sector_t last_block;
 	sector_t last_block_in_file;
@@ -264,7 +265,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 		if (page_has_buffers(page))
 			goto confused;
 
-		block_in_file = (sector_t)page->index << (PAGE_SHIFT - blkbits);
+		block_in_file = next_block =
+			(sector_t)page->index << (PAGE_SHIFT - blkbits);
 		last_block = block_in_file + nr_pages * blocks_per_page;
 		last_block_in_file = (ext4_readpage_limit(inode) +
 				      blocksize - 1) >> blkbits;
@@ -364,7 +366,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 		 * This page will go to BIO.  Do we need to send this
 		 * BIO off first?
 		 */
-		if (bio && (last_block_in_bio != blocks[0] - 1)) {
+		if (bio && (last_block_in_bio != blocks[0] - 1 ||
+			    !fscrypt_mergeable_bio(bio, inode, next_block))) {
 		submit_and_realloc:
 			submit_bio(bio);
 			bio = NULL;
@@ -376,6 +379,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 			 */
 			bio = bio_alloc(GFP_KERNEL,
 				min_t(int, nr_pages, BIO_MAX_PAGES));
+			fscrypt_set_bio_crypt_ctx(bio, inode, next_block,
+						  GFP_KERNEL);
 			ext4_set_bio_post_read_ctx(bio, inode, page->index);
 			bio_set_dev(bio, bdev);
 			bio->bi_iter.bi_sector = blocks[0] << (blkbits - 9);
fs/ext4/super.c +19 -0
index f464dff09774..490e748b048c 100644
@@ -1504,6 +1504,7 @@ enum {
 	Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit,
 	Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
 	Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption,
+	Opt_inlinecrypt,
 	Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
 	Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota,
 	Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,
@@ -1601,6 +1602,7 @@ static const match_table_t tokens = {
 	{Opt_noinit_itable, "noinit_itable"},
 	{Opt_max_dir_size_kb, "max_dir_size_kb=%u"},
 	{Opt_test_dummy_encryption, "test_dummy_encryption"},
+	{Opt_inlinecrypt, "inlinecrypt"},
 	{Opt_nombcache, "nombcache"},
 	{Opt_nombcache, "no_mbcache"},	/* for backward compatibility */
 	{Opt_removed, "check=none"},	/* mount option from ext2/3 */
@@ -1812,6 +1814,11 @@ static const struct mount_opts {
 	{Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT},
 	{Opt_max_dir_size_kb, 0, MOPT_GTE0},
 	{Opt_test_dummy_encryption, 0, MOPT_GTE0},
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+	{Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET},
+#else
+	{Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT},
+#endif
 	{Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET},
 	{Opt_err, 0, 0}
 };
@@ -1888,6 +1895,13 @@ static int handle_mount_opt(struct super_block *sb, char *opt, int token,
 	case Opt_nolazytime:
 		sb->s_flags &= ~SB_LAZYTIME;
 		return 1;
+	case Opt_inlinecrypt:
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+		sb->s_flags |= SB_INLINE_CRYPT;
+#else
+		ext4_msg(sb, KERN_ERR, "inline encryption not supported");
+#endif
+		return 1;
 	}
 
 	for (m = ext4_mount_opts; m->token != Opt_err; m++)
@@ -2300,6 +2314,8 @@ static int _ext4_show_options(struct seq_file *seq, struct super_block *sb,
 		SEQ_OPTS_PUTS("data_err=abort");
 	if (DUMMY_ENCRYPTION_ENABLED(sbi))
 		SEQ_OPTS_PUTS("test_dummy_encryption");
+	if (sb->s_flags & SB_INLINE_CRYPT)
+		SEQ_OPTS_PUTS("inlinecrypt");
 
 	ext4_show_quota_options(seq, sb);
 	return 0;
@@ -3808,6 +3824,9 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)
 	 */
 	sbi->s_li_wait_mult = EXT4_DEF_LI_WAIT_MULT;
 
+	/* disable blk-crypto file content encryption by default */
+	sb->s_flags &= ~SB_INLINE_CRYPT;
+
 	blocksize = BLOCK_SIZE << le32_to_cpu(es->s_log_block_size);
 	if (blocksize < EXT4_MIN_BLOCK_SIZE ||
 	    blocksize > EXT4_MAX_BLOCK_SIZE) {
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
maintainer
#38 lore raw

Re: [PATCH v7 9/9] ext4: add inline encryption support

Quoted 14 lines · Satya Tangirala

From: Eric Biggers <[email protected]>

Wire up ext4 to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:

- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.

- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.

Note: submit_bh_wbc() in fs/buffer.c also needed to be patched for this part, since ext4 sometimes uses ll_rw_block() on file data.

Quoted 7 lines

- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.

- Not doing filesystem-layer crypto on inline-encrypted files.

You've changed enough in this patch that you should probably should add your Co-developed-by, or at least summarize your changes in between the Signed-off-by (e.g. "[satyat: use SB_INLINECRYPT and update docs]")

Quoted 15 lines
---
 Documentation/admin-guide/ext4.rst |  4 ++++
 fs/buffer.c                        |  7 ++++---
 fs/ext4/ext4.h                     |  1 +
 fs/ext4/inode.c                    |  4 ++--
 fs/ext4/page-io.c                  |  6 ++++--
 fs/ext4/readpage.c                 | 11 ++++++++---
 fs/ext4/super.c                    | 19 +++++++++++++++++++
 7 files changed, 42 insertions(+), 10 deletions(-)
Documentation/admin-guide/ext4.rst +0 -0
index 9443fcef1876..b7cd3bdf827e 100644

Documentation/admin-guide/ext4.rst When mounting an ext4 filesystem, the following option are accepted: · line 400

Quoted 6 lines
Documentation/admin-guide/ext4.rst +3 -0
@@ -395,6 +395,10 @@ When mounting an ext4 filesystem, the following option are accepted:
         Documentation/filesystems/dax.txt.  Note that this option is
         incompatible with data=journal.
 
+  inlinecrypt
+        Use blk-crypto, rather than fscrypt, for file content en/decryption.
+        See Documentation/block/inline-encryption.rst.

"fscrypt" refers to the ext4 encryption feature itself. I.e., when inline encryption is enabled, fscrypt is still being used. Instead you probably want to write something like:

Encrypt/decrypt the contents of encrypted files using the blk-crypto framework rather than filesystem-layer encryption. This allows the use of inline encryption hardware. The on-disk format is unaffected. For more details, see Documentation/block/inline-encryption.rst.

fs/ext4/ext4.h struct ext4_inode_info { · line 1157

Quoted 12 lines
fs/ext4/ext4.h +1 -0
index 4441331d06cc..3917f4b3ec30 100644
@@ -1151,6 +1151,7 @@ struct ext4_inode_info {
 #define EXT4_MOUNT_JOURNAL_CHECKSUM	0x800000 /* Journal checksums */
 #define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT	0x1000000 /* Journal Async Commit */
 #define EXT4_MOUNT_WARN_ON_ERROR	0x2000000 /* Trigger WARN_ON on error */
+#define EXT4_MOUNT_INLINECRYPT		0x4000000 /* Inline encryption support */
 #define EXT4_MOUNT_DELALLOC		0x8000000 /* Delalloc support */
 #define EXT4_MOUNT_DATA_ERR_ABORT	0x10000000 /* Abort on file data write */
 #define EXT4_MOUNT_BLOCK_VALIDITY	0x20000000 /* Block validity checking */

Isn't this ext4-specific flag unused now that you switched to SB_INLINECRYPT instead?

Quoted 150 lines
fs/ext4/inode.c +2 -2
index e60aca791d3f..9ac1dd131586 100644
@@ -1089,7 +1089,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len,
 	}
 	if (unlikely(err)) {
 		page_zero_new_buffers(page, from, to);
-	} else if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode)) {
+	} else if (fscrypt_inode_uses_fs_layer_crypto(inode)) {
 		for (i = 0; i < nr_wait; i++) {
 			int err2;
 
@@ -3720,7 +3720,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,
 		/* Uhhuh. Read error. Complain and punt. */
 		if (!buffer_uptodate(bh))
 			goto unlock;
-		if (S_ISREG(inode->i_mode) && IS_ENCRYPTED(inode)) {
+		if (fscrypt_inode_uses_fs_layer_crypto(inode)) {
 			/* We expect the key to be set. */
 			BUG_ON(!fscrypt_has_encryption_key(inode));
 			err = fscrypt_decrypt_pagecache_blocks(page, blocksize,
fs/ext4/page-io.c +4 -2
index 68b39e75446a..0757145a31b2 100644
@@ -404,6 +404,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,
 	 * __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset().
 	 */
 	bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES);
+	fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
 	bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
 	bio_set_dev(bio, bh->b_bdev);
 	bio->bi_end_io = ext4_end_bio;
@@ -420,7 +421,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io,
 {
 	int ret;
 
-	if (io->io_bio && bh->b_blocknr != io->io_next_block) {
+	if (io->io_bio && (bh->b_blocknr != io->io_next_block ||
+			   !fscrypt_mergeable_bio_bh(io->io_bio, bh))) {
 submit_and_retry:
 		ext4_io_submit(io);
 	}
@@ -508,7 +510,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,
 	 * (e.g. holes) to be unnecessarily encrypted, but this is rare and
 	 * can't happen in the common case of blocksize == PAGE_SIZE.
 	 */
-	if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode) && nr_to_submit) {
+	if (fscrypt_inode_uses_fs_layer_crypto(inode) && nr_to_submit) {
 		gfp_t gfp_flags = GFP_NOFS;
 		unsigned int enc_bytes = round_up(len, i_blocksize(inode));
 
fs/ext4/readpage.c +8 -3
index c1769afbf799..68eac0aeffad 100644
@@ -195,7 +195,7 @@ static void ext4_set_bio_post_read_ctx(struct bio *bio,
 {
 	unsigned int post_read_steps = 0;
 
-	if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode))
+	if (fscrypt_inode_uses_fs_layer_crypto(inode))
 		post_read_steps |= 1 << STEP_DECRYPT;
 
 	if (ext4_need_verity(inode, first_idx))
@@ -232,6 +232,7 @@ int ext4_mpage_readpages(struct address_space *mapping,
 	const unsigned blkbits = inode->i_blkbits;
 	const unsigned blocks_per_page = PAGE_SIZE >> blkbits;
 	const unsigned blocksize = 1 << blkbits;
+	sector_t next_block;
 	sector_t block_in_file;
 	sector_t last_block;
 	sector_t last_block_in_file;
@@ -264,7 +265,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 		if (page_has_buffers(page))
 			goto confused;
 
-		block_in_file = (sector_t)page->index << (PAGE_SHIFT - blkbits);
+		block_in_file = next_block =
+			(sector_t)page->index << (PAGE_SHIFT - blkbits);
 		last_block = block_in_file + nr_pages * blocks_per_page;
 		last_block_in_file = (ext4_readpage_limit(inode) +
 				      blocksize - 1) >> blkbits;
@@ -364,7 +366,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 		 * This page will go to BIO.  Do we need to send this
 		 * BIO off first?
 		 */
-		if (bio && (last_block_in_bio != blocks[0] - 1)) {
+		if (bio && (last_block_in_bio != blocks[0] - 1 ||
+			    !fscrypt_mergeable_bio(bio, inode, next_block))) {
 		submit_and_realloc:
 			submit_bio(bio);
 			bio = NULL;
@@ -376,6 +379,8 @@ int ext4_mpage_readpages(struct address_space *mapping,
 			 */
 			bio = bio_alloc(GFP_KERNEL,
 				min_t(int, nr_pages, BIO_MAX_PAGES));
+			fscrypt_set_bio_crypt_ctx(bio, inode, next_block,
+						  GFP_KERNEL);
 			ext4_set_bio_post_read_ctx(bio, inode, page->index);
 			bio_set_dev(bio, bdev);
 			bio->bi_iter.bi_sector = blocks[0] << (blkbits - 9);
fs/ext4/super.c +14 -0
index f464dff09774..490e748b048c 100644
@@ -1504,6 +1504,7 @@ enum {
 	Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit,
 	Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback,
 	Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption,
+	Opt_inlinecrypt,
 	Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota,
 	Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota,
 	Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,
@@ -1601,6 +1602,7 @@ static const match_table_t tokens = {
 	{Opt_noinit_itable, "noinit_itable"},
 	{Opt_max_dir_size_kb, "max_dir_size_kb=%u"},
 	{Opt_test_dummy_encryption, "test_dummy_encryption"},
+	{Opt_inlinecrypt, "inlinecrypt"},
 	{Opt_nombcache, "nombcache"},
 	{Opt_nombcache, "no_mbcache"},	/* for backward compatibility */
 	{Opt_removed, "check=none"},	/* mount option from ext2/3 */
@@ -1812,6 +1814,11 @@ static const struct mount_opts {
 	{Opt_jqfmt_vfsv1, QFMT_VFS_V1, MOPT_QFMT},
 	{Opt_max_dir_size_kb, 0, MOPT_GTE0},
 	{Opt_test_dummy_encryption, 0, MOPT_GTE0},
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+	{Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_SET},
+#else
+	{Opt_inlinecrypt, EXT4_MOUNT_INLINECRYPT, MOPT_NOSUPPORT},
+#endif
 	{Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET},
 	{Opt_err, 0, 0}
 };
@@ -1888,6 +1895,13 @@ static int handle_mount_opt(struct super_block *sb, char *opt, int token,
 	case Opt_nolazytime:
 		sb->s_flags &= ~SB_LAZYTIME;
 		return 1;
+	case Opt_inlinecrypt:
+#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
+		sb->s_flags |= SB_INLINE_CRYPT;
+#else
+		ext4_msg(sb, KERN_ERR, "inline encryption not supported");
+#endif
+		return 1;
 	}
 
 	for (m = ext4_mount_opts; m->token != Opt_err; m++)

fs/ext4/super.c static int _ext4_show_options(struct seq_file *seq, struct super_block *sb, · line 2318

Quoted 5 lines
fs/ext4/super.c +2 -0
@@ -2300,6 +2314,8 @@ static int _ext4_show_options(struct seq_file *seq, struct super_block *sb,
 		SEQ_OPTS_PUTS("data_err=abort");
 	if (DUMMY_ENCRYPTION_ENABLED(sbi))
 		SEQ_OPTS_PUTS("test_dummy_encryption");
+	if (sb->s_flags & SB_INLINE_CRYPT)
+		SEQ_OPTS_PUTS("inlinecrypt");

If we're going with the VFS-level mount option flag, it looks like the code to show it should go in show_sb_opts() in fs/proc_namespace.c instead of here.

fs/ext4/super.c static int ext4_fill_super(struct super_block *sb, void *data, int silent) · line 3829

Quoted 7 lines
fs/ext4/super.c +3 -0
@@ -3808,6 +3824,9 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)
 	 */
 	sbi->s_li_wait_mult = EXT4_DEF_LI_WAIT_MULT;
 
+	/* disable blk-crypto file content encryption by default */
+	sb->s_flags &= ~SB_INLINE_CRYPT;
+

Don't the flags default to 0? If they do, this part is unnecessary.

- Eric

maintainer
#39 lore raw
In reply to Satya Tangirala
Quoted 2 lines · Satya Tangirala

This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.

[...]

Quoted 13 lines

Changes v6 => v7:
- Keyslot management is now done on a per-request basis rather than a per-bio basis.
- Storage drivers can now specify the maximum number of bytes they can accept for the data unit number (DUN) for each crypto algorithm, and upper layers can specify the minimum number of bytes of DUN they want with the blk_crypto_key they send with the bio - a driver is only considered to support a blk_crypto_key if the driver supports at least as many DUN bytes as the upper layer wants. This is necessary because storage drivers may not support as many bytes as the algorithm specification dictates (for e.g. UFS only supports 8 byte DUNs for AES-256-XTS, even though the algorithm specification says DUNs are 16 bytes long).

Quoted 8 lines

- Introduce SB_INLINECRYPT to keep track of whether inline encryption is enabled for a filesystem (instead of using an fscrypt_operation).
- Expose keyslot manager declaration and embed it within ufs_hba to clean up code.
- Make blk-crypto preclude blk-integrity.
- Some bug fixes
- Introduce UFSHCD_QUIRK_BROKEN_CRYPTO for UFS drivers that don't support inline encryption (yet)

This patchset can also be retrieved from

Repo: https://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git
Tag: inline-encryption-v7

For review purposes I also created a tag inline-encryption-v6-rebased-onto-v7-base which is the v6 patchset rebased onto the same base commit (v5.6-rc2). So it's possible to see what changed by

git diff inline-encryption-v6-rebased-onto-v7-base..inline-encryption-v7

(Although, I had to resolve conflicts in fs/crypto/ to do the rebase, so it's not *exactly* v6.)

- Eric