Inline Encryption Support
Subsystems (8)
v10 cover Inline Encryption Support
This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.
Note that the patches in this series for the block layer (i.e. patches 1, 2, 3, 4 and 5) can be applied independently of the subsequent patches in this series.
Inline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size, etc.) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device. Inline Encryption hardware has become increasingly common, and we want to support it in the kernel.
Inline Encryption hardware implementations often function around the concept of a limited number of "keyslots", which can hold an encryption context each. The storage device can be directed to en/decrypt any particular request with the encryption context stored in any particular keyslot.
Patch 1 documents the whole series.
Patch 2 introduces a Keyslot Manager to efficiently manage keyslots. The keyslot manager also functions as the interface that blk-crypto (introduced in Patch 3), will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.
Patch 3 adds the block layer changes for inline encryption support. It introduces struct bio_crypt_ctx, and a ptr to one in struct bio, which allows struct bio to represent an encryption context that can be passed down the storage stack from the filesystem layer to the storage driver.
Patch 4 precludes inline encryption support in a device whenever it supports blk-integrity, because there is currently no known hardware that supports both features, and it is not completely straightfoward to support both of them properly, and doing it improperly might result in leaks of information about the plaintext.
Patch 5 introduces blk-crypto-fallback - a kernel crypto API fallback for blk-crypto to use when inline encryption hardware isn't present. This allows filesystems to specify encryption contexts for bios without having to worry about whether the underlying hardware has inline encryption support, and allows for testing without real hardware inline encryption support. This fallback is separately configurable from blk-crypto, and can be disabled if desired while keeping inline encryption support. It may also be possible to remove file content en/decryption from fscrypt and simply use blk-crypto-fallback in a future patch. For more details on blk-crypto and the fallback, refer to Documentation/block/inline-encryption.rst.
Patches 6-8 add support for inline encryption into the UFS driver according to the JEDEC UFS HCI v2.1 specification. Inline encryption support for other drivers (like eMMC) may be added in the same way - the device driver should set up a Keyslot Manager in the device's request_queue (refer to the UFS crypto additions in ufshcd-crypto.c and ufshcd.c for an example).
Patch 9 adds the SB_INLINECRYPT mount flag to the fs layer, which filesystems must set to indicate that they want to use blk-crypto for en/decryption of file contents.
Patch 10 adds support to fscrypt - to use inline encryption with fscrypt, the filesystem must be mounted with '-o inlinecrypt' - when this option is specified, the contents of any AES-256-XTS encrypted file will be encrypted using blk-crypto.
Patches 11 and 12 add support to f2fs and ext4 respectively, so that we have a complete stack that can make use of inline encryption.
The patches were tested running kvm-xfstests, by specifying the introduced "inlinecrypt" mount option, so that en/decryption happens with the blk-crypto fallback. The patches were also tested on a Pixel 4 with UFS hardware that has support for inline encryption.
There have been a few patch sets addressing Inline Encryption Support in the past. Briefly, this patch set differs from those as follows:
1) "crypto: qce: ice: Add support for Inline Crypto Engine" is specific to certain hardware, while our patch set's Inline Encryption support for UFS is implemented according to the JEDEC UFS specification.
2) "scsi: ufs: UFS Host Controller crypto changes" registers inline encryption support as a kernel crypto algorithm. Our patch views inline encryption as being fundamentally different from a generic crypto provider (in that inline encryption is tied to a device), and so does not use the kernel crypto API to represent inline encryption hardware.
3) "scsi: ufs: add real time/inline crypto support to UFS HCD" requires the device mapper to work - our patch does not.
Changes v9 => v10:
- Incorporate Eric's fix for allowing en/decryption to happen as usual via fscrypt in the case that hardware doesn't support the desired crypto configuration, but blk-crypto-fallback is disabled. (Introduce struct blk_crypto_config and blk_crypto_config_supported for fscrypt to call, to check that either blk-crypto-fallback is enabled or the device supports the crypto configuration).
- Update docs
- Lots of cleanups
Changes v8 => v9:
- Don't open code bio_has_crypt_ctx into callers of blk-crypto functions.
- Lots of cleanups
Changes v7 => v8:
- Pass a struct blk_ksm_keyslot * around instead of slot numbers which simplifies some functions and passes around arguments with better types
- Make bios with no encryption context avoid making calls into blk-crypto by checking for the presence of bi_crypt_context before making the call
- Make blk-integrity preclude inline encryption support at probe time
- Many many cleanups
Changes v6 => v7:
- Keyslot management is now done on a per-request basis rather than a per-bio basis.
- Storage drivers can now specify the maximum number of bytes they can accept for the data unit number (DUN) for each crypto algorithm, and upper layers can specify the minimum number of bytes of DUN they want with the blk_crypto_key they send with the bio - a driver is only considered to support a blk_crypto_key if the driver supports at least as many DUN bytes as the upper layer wants. This is necessary because storage drivers may not support as many bytes as the algorithm specification dictates (for e.g. UFS only supports 8 byte DUNs for AES-256-XTS, even though the algorithm specification says DUNs are 16 bytes long).
- Introduce SB_INLINECRYPT to keep track of whether inline encryption is enabled for a filesystem (instead of using an fscrypt_operation).
- Expose keyslot manager declaration and embed it within ufs_hba to clean up code.
- Make blk-crypto preclude blk-integrity.
- Some bug fixes
- Introduce UFSHCD_QUIRK_BROKEN_CRYPTO for UFS drivers that don't support inline encryption (yet)
Changes v5 => v6:
- Blk-crypto's kernel crypto API fallback is no longer restricted to 8-byte DUNs. It's also now separately configurable from blk-crypto, and can be disabled entirely, while still allowing the kernel to use inline encryption hardware. Further, struct bio_crypt_ctx takes up less space, and no longer contains the information needed by the crypto API fallback - the fallback allocates the required memory when necessary.
- Blk-crypto now supports all file content encryption modes supported by fscrypt.
- Fixed bio merging logic in blk-merge.c
- Fscrypt now supports inline encryption with the direct key policy, since blk-crypto now has support for larger DUNs.
- Keyslot manager now uses a hashtable to lookup which keyslot contains any particular key (thanks Eric!)
- Fscrypt support for inline encryption now handles filesystems with multiple underlying block devices (thanks Eric!)
- Numerous cleanups
Changes v4 => v5:
- The fscrypt patch has been separated into 2. The first adds support for the IV_INO_LBLK_64 policy (which was called INLINE_CRYPT_OPTIMIZED in past versions of this series). This policy is now purely an on disk format, and doesn't dictate whether blk-crypto is used for file content encryption or not. Instead, this is now decided based on the "inlinecrypt" mount option.
- Inline crypto key eviction is now handled by blk-crypto instead of fscrypt.
- More refactoring.
Changes v3 => v4:
- Fixed the issue with allocating crypto_skcipher in blk_crypto_keyslot_program.
- bio_crypto_alloc_ctx is now mempool backed.
- In f2fs, a bio's bi_crypt_context is now set up when the bio is allocated, rather than just before the bio is submitted - this fixes bugs in certain cases, like when an encrypted block is being moved without decryption.
- Lots of refactoring and cleanup of blk-crypto - thanks Eric!
Changes v2 => v3:
- Overhauled keyslot manager's get keyslot logic and optimized LRU.
- Block crypto en/decryption fallback now supports data unit sizes that divide the bvec length (instead of requiring each bvec's length to be the same as the data unit size).
- fscrypt master key is now keyed additionally by super_block and ci_ctfm != NULL.
- all references of "hw encryption" are replaced by inline encryption.
- address various other review comments from Eric.
Changes v1 => v2:
- Block layer and UFS changes are split into 3 patches each.
- We now only have a ptr to a struct bio_crypt_ctx in struct bio, instead of the struct itself.
- struct bio_crypt_ctx no longer has flags.
- blk-crypto now correctly handles the case when it fails to init (because of insufficient memory), but kernel continues to boot.
- ufshcd-crypto now works on big endian cpus.
- Many cleanups.
Eric Biggers (1):
ext4: add inline encryption support
Satya Tangirala (11):
Documentation: Document the blk-crypto framework
block: Keyslot Manager for Inline Encryption
block: Inline encryption support for blk-mq
block: Make blk-integrity preclude hardware inline encryption
block: blk-crypto-fallback for Inline Encryption
scsi: ufs: UFS driver v2.1 spec crypto additions
scsi: ufs: UFS crypto API
scsi: ufs: Add inline encryption support to UFS
fs: introduce SB_INLINECRYPT
fscrypt: add inline encryption support
f2fs: add inline encryption support
Documentation/admin-guide/ext4.rst | 6 + Documentation/block/index.rst | 1 + Documentation/block/inline-encryption.rst | 260 +++++++++ Documentation/filesystems/f2fs.rst | 7 +- block/Kconfig | 17 + block/Makefile | 2 + block/bio-integrity.c | 3 + block/bio.c | 6 + block/blk-core.c | 20 +- block/blk-crypto-fallback.c | 655 ++++++++++++++++++++++ block/blk-crypto-internal.h | 192 +++++++ block/blk-crypto.c | 423 ++++++++++++++ block/blk-integrity.c | 7 + block/blk-map.c | 1 + block/blk-merge.c | 11 + block/blk-mq.c | 13 + block/blk.h | 4 + block/bounce.c | 2 + block/keyslot-manager.c | 397 +++++++++++++ drivers/md/dm.c | 3 + drivers/scsi/ufs/Kconfig | 9 + drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd-crypto.c | 226 ++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 60 ++ drivers/scsi/ufs/ufshcd.c | 46 +- drivers/scsi/ufs/ufshcd.h | 24 + drivers/scsi/ufs/ufshci.h | 67 ++- fs/buffer.c | 7 +- fs/crypto/Kconfig | 6 + fs/crypto/Makefile | 1 + fs/crypto/bio.c | 50 ++ fs/crypto/crypto.c | 2 +- fs/crypto/fname.c | 4 +- fs/crypto/fscrypt_private.h | 120 +++- fs/crypto/inline_crypt.c | 365 ++++++++++++ fs/crypto/keyring.c | 4 +- fs/crypto/keysetup.c | 92 ++- fs/crypto/keysetup_v1.c | 16 +- fs/ext4/inode.c | 4 +- fs/ext4/page-io.c | 6 +- fs/ext4/readpage.c | 11 +- fs/ext4/super.c | 9 + fs/f2fs/compress.c | 2 +- fs/f2fs/data.c | 68 ++- fs/f2fs/super.c | 32 ++ fs/proc_namespace.c | 1 + include/linux/blk-crypto.h | 124 ++++ include/linux/blk_types.h | 6 + include/linux/blkdev.h | 41 ++ include/linux/fs.h | 1 + include/linux/fscrypt.h | 57 ++ include/linux/keyslot-manager.h | 107 ++++ 52 files changed, 3509 insertions(+), 90 deletions(-) create mode 100644 Documentation/block/inline-encryption.rst create mode 100644 block/blk-crypto-fallback.c create mode 100644 block/blk-crypto-internal.h create mode 100644 block/blk-crypto.c create mode 100644 block/keyslot-manager.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h create mode 100644 fs/crypto/inline_crypt.c create mode 100644 include/linux/blk-crypto.h create mode 100644 include/linux/keyslot-manager.h
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 1/12 Documentation: Document the blk-crypto framework
The blk-crypto framework adds support for inline encryption. There are numerous changes throughout the storage stack. This patch documents the main design choices in the block layer, the API presented to users of the block layer (like fscrypt or layered devices) and the API presented to drivers for adding support for inline encryption.
--- Documentation/block/inline-encryption.rst | 260 ++++++++++++++++++++++ 1 file changed, 260 insertions(+) create mode 100644 Documentation/block/inline-encryption.rst
Documentation/block/inline-encryption.rst new file +260 -0
@@ -0,0 +1,260 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Inline Encryption +================= + +Background +========== + +Inline encryption hardware sits logically between memory and the disk, and can +en/decrypt data as it goes in/out of the disk. Inline encryption hardware has a +fixed number of "keyslots" - slots into which encryption contexts (i.e. the +encryption key, encryption algorithm, data unit size) can be programmed by the +kernel at any time. Each request sent to the disk can be tagged with the index +of a keyslot (and also a data unit number to act as an encryption tweak), and +the inline encryption hardware will en/decrypt the data in the request with the +encryption context programmed into that keyslot. This is very different from +full disk encryption solutions like self encrypting drives/TCG OPAL/ATA +Security standards, since with inline encryption, any block on disk could be +encrypted with any encryption context the kernel chooses. + + +Objective +========= + +We want to support inline encryption (IE) in the kernel. +To allow for testing, we also want a crypto API fallback when actual +IE hardware is absent. We also want IE to work with layered devices +like dm and loopback (i.e. we want to be able to use the IE hardware +of the underlying devices if present, or else fall back to crypto API +en/decryption). + + +Constraints and notes +===================== + +- IE hardware has a limited number of "keyslots" that can be programmed + with an encryption context (key, algorithm, data unit size, etc.) at any time. + One can specify a keyslot in a data request made to the device, and the + device will en/decrypt the data using the encryption context programmed into + that specified keyslot. When possible, we want to make multiple requests with + the same encryption context share the same keyslot. + +- We need a way for upper layers like filesystems to specify an encryption + context to use for en/decrypting a struct bio, and a device driver (like UFS) + needs to be able to use that encryption context when it processes the bio. + +- We need a way for device drivers to expose their inline encryption + capabilities in a unified way to the upper layers. + + +Design +====== + +We add a :c:type:`struct bio_crypt_ctx` to :c:type:`struct bio` that can +represent an encryption context, because we need to be able to pass this +encryption context from the upper layers (like the fs layer) to the +device driver to act upon. + +While IE hardware works on the notion of keyslots, the FS layer has no +knowledge of keyslots - it simply wants to specify an encryption context to +use while en/decrypting a bio. + +We introduce a keyslot manager (KSM) that handles the translation from +encryption contexts specified by the FS to keyslots on the IE hardware. +This KSM also serves as the way IE hardware can expose its capabilities to +upper layers. The generic mode of operation is: each device driver that wants +to support IE will construct a KSM and set it up in its struct request_queue. +Upper layers that want to use IE on this device can then use this KSM in +the device's struct request_queue to translate an encryption context into +a keyslot. The presence of the KSM in the request queue shall be used to mean +that the device supports IE. + +The KSM uses refcounts to track which keyslots are idle (either they have no +encryption context programmed, or there are no in-flight struct bios +referencing that keyslot). When a new encryption context needs a keyslot, it +tries to find a keyslot that has already been programmed with the same +encryption context, and if there is no such keyslot, it evicts the least +recently used idle keyslot and programs the new encryption context into that +one. If no idle keyslots are available, then the caller will sleep until there +is at least one. + + +blk-mq changes, other block layer changes and blk-crypto-fallback +================================================================= + +We add a pointer to a ``bi_crypt_context`` and ``keyslot`` to +:c:type:`struct request`. These will be referred to as the ``crypto fields`` +for the request. This ``keyslot`` is the keyslot into which the +``bi_crypt_context`` has been programmed in the KSM of the ``request_queue`` +that this request is being sent to. + +We introduce ``block/blk-crypto-fallback.c``, which allows upper layers to remain +blissfully unaware of whether or not real inline encryption hardware is present +underneath. When a bio is submitted with a target ``request_queue`` that doesn't +support the encryption context specified with the bio, the block layer will +en/decrypt the bio with the blk-crypto-fallback. + +If the bio is a ``WRITE`` bio, a bounce bio is allocated, and the data in the bio +is encrypted stored in the bounce bio - blk-mq will then proceed to process the +bounce bio as if it were not encrypted at all (except when blk-integrity is +concerned). ``blk-crypto-fallback`` sets the bounce bio's ``bi_end_io`` to an +internal function that cleans up the bounce bio and ends the original bio. + +If the bio is a ``READ`` bio, the bio's ``bi_end_io`` (and also ``bi_private``) +is saved and overwritten by ``blk-crypto-fallback`` to +``bio_crypto_fallback_decrypt_bio``. The bio's ``bi_crypt_context`` is also +overwritten with ``NULL``, so that to the rest of the stack, the bio looks +as if it was a regular bio that never had an encryption context specified. +``bio_crypto_fallback_decrypt_bio`` will decrypt the bio, restore the original +``bi_end_io`` (and also ``bi_private``) and end the bio again. + +Regardless of whether real inline encryption hardware is used or the +blk-crypto-fallback is used, the ciphertext written to disk (and hence the +on-disk format of data) will be the same (assuming the hardware's implementation +of the algorithm being used adheres to spec and functions correctly). + +If a ``request queue``'s inline encryption hardware claimed to support the +encryption context specified with a bio, then it will not be handled by the +``blk-crypto-fallback``. We will eventually reach a point in blk-mq when a +:c:type:`struct request` needs to be allocated for that bio. At that point, +blk-mq tries to program the encryption context into the ``request_queue``'s +keyslot_manager, and obtain a keyslot, which it stores in its newly added +``keyslot`` field. This keyslot is released when the request is completed. + +When a bio is added to a request, the request takes over ownership of the +``bi_crypt_context`` of the bio - in particular, the request keeps the +``bi_crypt_context`` of the first bio in its bio-list, and frees the rest +(blk-mq needs to be careful to maintain this invariant during bio and request +merges). + +To make it possible for inline encryption to work with request queue based +layered devices, when a request is cloned, its ``crypto fields`` are cloned as +well. When the cloned request is submitted, blk-mq programs the +``bi_crypt_context`` of the request into the clone's request_queue's keyslot +manager, and stores the returned keyslot in the clone's ``keyslot``. + + +API presented to users of the block layer +========================================= + +``struct blk_crypto_key`` represents a crypto key (the raw key, size of the +key, the crypto algorithm to use, the data unit size to use, and the number of +bytes required to represent data unit numbers that will be specified with the +``bi_crypt_context``). + +``blk_crypto_init_key`` allows upper layers to initialize such a +``blk_crypto_key``. + +``bio_crypt_set_ctx`` should be called on any bio that a user of +the block layer wants en/decrypted via inline encryption (or the +blk-crypto-fallback, if hardware support isn't available for the desired +crypto configuration). This function takes the ``blk_crypto_key`` and the +data unit number (DUN) to use when en/decrypting the bio. + +``blk_crypto_config_supported`` allows upper layers to query whether or not the +an encryption context passed to request queue can be handled by blk-crypto +(either by real inline encryption hardware, or by the blk-crypto-fallback). +This is useful e.g. when blk-crypto-fallback is disabled, and the upper layer +wants to use an algorithm that may not supported by hardware - this function +lets the upper layer know ahead of time that the algorithm isn't supported, +and the upper layer can fallback to something else if appropriate. + +``blk_crypto_start_using_key`` - Upper layers must call this function on +``blk_crypto_key`` and a ``request_queue`` before using the key with any bio +headed for that ``request_queue``. This function ensures that either the +hardware supports the key's crypto settings, or the crypto API fallback has +transforms for the needed mode allocated and ready to go. Note that this +function may allocate an ``skcipher``, and must not be called from the data +path, since allocating ``skciphers`` from the data path can deadlock. + +``blk_crypto_evict_key`` should be called by upper layers when they want +to ensure that a key is removed from memory and from any keyslots in inline +encryption hardware that the key might have been programmed into (or the +blk-crypto-fallback). + +API presented to device drivers +=============================== + +A :c:type:``struct keyslot_manager`` should be set up by device drivers in the +``request_queue`` of the device. The device driver needs to call +``blk_ksm_init`` on the ``keyslot_manager``, which specfying the number of +keyslots supported by the hardware. + +The device driver also needs to tell the KSM how to actually manipulate the +IE hardware in the device to do things like programming the crypto key into +the IE hardware into a particular keyslot. All this is achieved through the +:c:type:`struct keyslot_mgmt_ll_ops` field in the KSM that the device driver +must fill up after initing the ``keyslot_manager``. + +The KSM also handles runtime power management for the device when applicable +(e.g. when it wants to program a crypto key into the IE hardware, the device +must be runtime powered on) - so the device driver must also set the ``dev`` +field in the ksm to point to the `struct device` for the KSM to use for runtime +power management. + +``blk_ksm_reprogram_all_keys`` can be called by device drivers if the device +needs each and every of its keyslots to be reprogrammed with the key it +"should have" at the point in time when the function is called. This is useful +e.g. if a device loses all its keys on runtime power down/up. + +``blk_ksm_destroy`` should be called to free up all resources used by a keyslot +manager upon ``blk_ksm_init``, once the ``keyslot_manager`` is no longer +needed. + + +Layered Devices +=============== + +Request queue based layered devices like dm-rq that wish to support IE need to +create their own keyslot manager for their request queue, and expose whatever +functionality they choose. When a layered device wants to pass a clone of that +request to another ``request_queue``, blk-crypto will initialize and prepare the +clone as necessary - see ``blk_crypto_rq_prep_clone`` and +``blk_crypto_insert_cloned_request`` in ``blk-crypto.c``. + + +Future Optimizations for layered devices +======================================== + +Creating a keyslot manager for a layered device uses up memory for each +keyslot, and in general, a layered device merely passes the request on to a +"child" device, so the keyslots in the layered device itself are completely +unused, and don't need any refcounting or keyslot programming. We can instead +define a new type of KSM; the "passthrough KSM", that layered devices can use +to advertise an unlimited number of keyslots, and support for any encryption +algorithms they choose, while not actually using any memory for each keyslot. +Another use case for the "passthrough KSM" is for IE devices that do not have a +limited number of keyslots. + + +Interaction between inline encryption and blk integrity +======================================================= + +At the time of this patch, there is no real hardware that supports both these +features. However, these features do interact with each other, and it's not +completely trivial to make them both work together properly. In particular, +when a WRITE bio wants to use inline encryption on a device that supports both +features, the bio will have an encryption context specified, after which +its integrity information is calculated (using the plaintext data, since +the encryption will happen while data is being written), and the data and +integrity info is sent to the device. Obviously, the integrity info must be +verified before the data is encrypted. After the data is encrypted, the device +must not store the integrity info that it received with the plaintext data +since that might reveal information about the plaintext data. As such, it must +re-generate the integrity info from the ciphertext data and store that on disk +instead. Another issue with storing the integrity info of the plaintext data is +that it changes the on disk format depending on whether hardware inline +encryption support is present or the kernel crypto API fallback is used (since +if the fallback is used, the device will receive the integrity info of the +ciphertext, not that of the plaintext). + +Because there isn't any real hardware yet, it seems prudent to assume that +hardware implementations might not implement both features together correctly, +and disallow the combination for now. Whenever a device supports integrity, the +kernel will pretend that the device does not support hardware inline encryption +(by essentially setting the keyslot manager in the request_queue of the device +to NULL). When the crypto API fallback is enabled, this means that all bios with +and encryption context will use the fallback, and IO will complete as usual. +When the fallback is disabled, a bio with an encryption context will be failed.
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 2/12 block: Keyslot Manager for Inline Encryption
In reply to Satya TangiralaInline Encryption hardware allows software to specify an encryption context (an encryption key, crypto algorithm, data unit num, data unit size) along with a data transfer request to a storage device, and the inline encryption hardware will use that context to en/decrypt the data. The inline encryption hardware is part of the storage device, and it conceptually sits on the data path between system memory and the storage device.
Inline Encryption hardware implementations often function around the concept of "keyslots". These implementations often have a limited number of "keyslots", each of which can hold a key (we say that a key can be "programmed" into a keyslot). Requests made to the storage device may have a keyslot and a data unit number associated with them, and the inline encryption hardware will en/decrypt the data in the requests using the key programmed into that associated keyslot and the data unit number specified with the request.
As keyslots are limited, and programming keys may be expensive in many implementations, and multiple requests may use exactly the same encryption contexts, we introduce a Keyslot Manager to efficiently manage keyslots.
We also introduce a blk_crypto_key, which will represent the key that's programmed into keyslots managed by keyslot managers. The keyslot manager also functions as the interface that upper layers will use to program keys into inline encryption hardware. For more information on the Keyslot Manager, refer to documentation found in block/keyslot-manager.c and linux/keyslot-manager.h.
--- block/Kconfig | 7 + block/Makefile | 1 + block/keyslot-manager.c | 378 ++++++++++++++++++++++++++++++++ include/linux/blk-crypto.h | 53 +++++ include/linux/blkdev.h | 6 + include/linux/keyslot-manager.h | 107 +++++++++ 6 files changed, 552 insertions(+) create mode 100644 block/keyslot-manager.c create mode 100644 include/linux/blk-crypto.h create mode 100644 include/linux/keyslot-manager.h
block/Kconfig +7 -0
@@ -185,6 +185,13 @@ config BLK_SED_OPAL Enabling this option enables users to setup/unlock/lock Locking ranges for SED devices using the Opal protocol. +config BLK_INLINE_ENCRYPTION + bool "Enable inline encryption support in block layer" + help + Build the blk-crypto subsystem. Enabling this lets the + block layer handle encryption, so users can take + advantage of inline encryption hardware if present. + menu "Partition Types" source "block/partitions/Kconfig"
block/Makefile +1 -0
@@ -36,3 +36,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o
block/keyslot-manager.c new file +378 -0
378 lines not on this page · raw message
include/linux/blk-crypto.h new file +53 -0
53 lines not on this page · raw message
include/linux/blkdev.h +6 -0
@@ -43,6 +43,7 @@ struct pr_ops;struct rq_qos ;struct blk_queue_stats ;struct blk_stat_callback ; +struct blk_keyslot_manager ;#define BLKDEV_MIN_RQ 4#define BLKDEV_MAX_RQ 128/* Default maximum */
@@ -474,6 +475,11 @@ struct request_queue {unsigned int dma_pad_mask;unsigned int dma_alignment; +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +/* Inline crypto capabilities */ +struct blk_keyslot_manager *ksm; +#endif +unsigned int rq_timeout;int poll_nsec;
include/linux/keyslot-manager.h new file +107 -0
107 lines not on this page · raw message
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Re: [PATCH v10 02/12] block: Keyslot Manager for Inline Encryption
block/keyslot-manager.c line 297
Quoted 8 lines
block/keyslot-manager.c new file +8
-0+bool blk_ksm_crypto_cfg_supported (struct blk_keyslot_manager *ksm, +const struct blk_crypto_config *cfg) +{ +if (!ksm) +return false; +return (ksm->crypto_modes_supported[cfg->crypto_mode] & + cfg->data_unit_size) && + (ksm->max_dun_bytes_supported >= cfg->dun_bytes);
Nit: why not expand this a bit to be more readable:
if (!(ksm->crypto_modes_supported[cfg->crypto_mode] & cfg->data_unit_size)) return false; if (ksm->max_dun_bytes_supported < cfg->dun_bytes) return false; return true;
Quoted 5 lines
block/keyslot-manager.c new file +5
-0+int blk_ksm_evict_key (struct blk_keyslot_manager *ksm, +const struct blk_crypto_key *key) +{ +struct blk_ksm_keyslot *slot; +int err =0 ;
block/keyslot-manager.c line 326
Quoted 10 lines
block/keyslot-manager.c new file +10
-0+ +blk_ksm_hw_enter (ksm); + slot =blk_ksm_find_keyslot (ksm, key); +if (!slot) + goto out_unlock; + +if (atomic_read (&slot->slot_refs) !=0 ) { + err = -EBUSY ; + goto out_unlock; + }
This check looks racy.
Re: [PATCH v10 02/12] block: Keyslot Manager for Inline Encryption
Quoted 27 lines · Christoph Hellwig
block/keyslot-manager.c line 297
Quoted 8 lines
block/keyslot-manager.c new file +8
-0+bool blk_ksm_crypto_cfg_supported (struct blk_keyslot_manager *ksm, +const struct blk_crypto_config *cfg) +{ +if (!ksm) +return false; +return (ksm->crypto_modes_supported[cfg->crypto_mode] & + cfg->data_unit_size) && + (ksm->max_dun_bytes_supported >= cfg->dun_bytes);Nit: why not expand this a bit to be more readable:
if (!(ksm->crypto_modes_supported[cfg->crypto_mode] & cfg->data_unit_size)) return false; if (ksm->max_dun_bytes_supported < cfg->dun_bytes) return false; return true;Quoted 9 lines
block/keyslot-manager.c new file +9
-0+int blk_ksm_evict_key (struct blk_keyslot_manager *ksm, +const struct blk_crypto_key *key) +{ +struct blk_ksm_keyslot *slot; +int err =0 ; + +blk_ksm_hw_enter (ksm); + slot =blk_ksm_find_keyslot (ksm, key); +if (!slot)
Quoted 8 lines
block/keyslot-manager.c line 326
Quoted 6 lines
block/keyslot-manager.c new file +6
-0+ goto out_unlock; + +if (atomic_read (&slot->slot_refs) !=0 ) { + err = -EBUSY ; + goto out_unlock; + }This check looks racy.
Yes, this could in theory race with blk_ksm_put_slot (e.g. if it's called while there's still IO in flight/IO that just finished) - But it's currently only called by fscrypt when a key is being destroyed, which only happens after all the inodes using that key are evicted, and no data is in flight, so when this function is called, slot->slot_refs will be 0. In particular, this function should only be called when the key isn't being used for IO anymore. I'll add a WARN_ON_ONCE and also make the assumption clearer. We could also instead make this wait for the slot_refs to become 0 and then evict the key instead of just returning -EBUSY as it does now, but I'm not sure if it's really what we want to do/worth doing right now...
Eric Biggers
maintainerRe: [PATCH v10 02/12] block: Keyslot Manager for Inline Encryption
Quoted 17 lines · Satya Tangirala
Quoted 9 lines
Quoted 9 lines
block/keyslot-manager.c new file +9
-0+int blk_ksm_evict_key (struct blk_keyslot_manager *ksm, +const struct blk_crypto_key *key) +{ +struct blk_ksm_keyslot *slot; +int err =0 ; + +blk_ksm_hw_enter (ksm); + slot =blk_ksm_find_keyslot (ksm, key); +if (!slot)Quoted 8 lines
block/keyslot-manager.c line 326
Quoted 6 lines
block/keyslot-manager.c new file +6
-0+ goto out_unlock; + +if (atomic_read (&slot->slot_refs) !=0 ) { + err = -EBUSY ; + goto out_unlock; + }This check looks racy.
Quoted 11 lines
Yes, this could in theory race with
blk_ksm_put_slot(e.g. if it's called while there's still IO in flight/IO that just finished) - But it's currently only called by fscrypt when a key is being destroyed, which only happens after all the inodes using that key are evicted, and no data is in flight, so when this function is called,slot->slot_refswill be 0. In particular, this function should only be called when the key isn't being used for IO anymore. I'll add aWARN_ON_ONCEand also make the assumption clearer. We could also instead make this wait for theslot_refsto become 0 and then evict the key instead of just returning-EBUSYas it does now, but I'm not sure if it's really what we want to do/worth doing right now...
Note that we're holding down_write(&ksm->lock) here, which synchronizes with someone getting the keyslot (in particular, incrementing its refcount from 0) because that uses down_read(&ksm->lock).
So I don't think there's a race. The behavior is just that if someone tries to evict a key that's still in-use, then we'll correctly fail to evict the key.
"Evicting a key that's still in-use" isn't supposed to happen, so printing a warning is a good idea. But I think it needs to be pr_warn_once(), not WARN_ON_ONCE(), because WARN_ON_ONCE() is for kernel bugs only, not userspace bugs. It's theoretically possible for userspace to cause the same key to be used multiple times on the same disk but via different blk_crypto_key's. The keyslot manager will put these in the same keyslot, but there will be a separate eviction attempt for each blk_crypto_key.
For example, with fscrypt with -o inlinecrypt and blk-crypto-fallback, userspace could create an encrypted file using FSCRYPT_MODE_ADIANTUM and flags == 0, then get its encryption nonce and derive the file's encryption key. Then in another directory, they could set FSCRYPT_MODE_ADIANTUM and flags == FSCRYPT_POLICY_FLAG_DIRECT_KEY, and use the other file's encryption key as the *master* key.
That would be totally insane for userspace to do. But it's possible, so we can't use WARN_ON_ONCE().
- Eric
Eric Biggers
maintainerRe: [PATCH v10 02/12] block: Keyslot Manager for Inline Encryption
Quoted 52 lines · Eric Biggers
Quoted 17 lines · Satya Tangirala
Quoted 9 lines
Quoted 9 lines
block/keyslot-manager.c new file +9
-0+int blk_ksm_evict_key (struct blk_keyslot_manager *ksm, +const struct blk_crypto_key *key) +{ +struct blk_ksm_keyslot *slot; +int err =0 ; + +blk_ksm_hw_enter (ksm); + slot =blk_ksm_find_keyslot (ksm, key); +if (!slot)Quoted 8 lines
block/keyslot-manager.c line 326
Quoted 6 lines
block/keyslot-manager.c new file +6
-0+ goto out_unlock; + +if (atomic_read (&slot->slot_refs) !=0 ) { + err = -EBUSY ; + goto out_unlock; + }This check looks racy.
Quoted 11 lines
Yes, this could in theory race with
blk_ksm_put_slot(e.g. if it's called while there's still IO in flight/IO that just finished) - But it's currently only called by fscrypt when a key is being destroyed, which only happens after all the inodes using that key are evicted, and no data is in flight, so when this function is called,slot->slot_refswill be 0. In particular, this function should only be called when the key isn't being used for IO anymore. I'll add aWARN_ON_ONCEand also make the assumption clearer. We could also instead make this wait for theslot_refsto become 0 and then evict the key instead of just returning-EBUSYas it does now, but I'm not sure if it's really what we want to do/worth doing right now...Note that we're holding
down_write(&ksm->lock)here, which synchronizes with someone getting the keyslot (in particular, incrementing its refcount from 0) because that usesdown_read(&ksm->lock).So I don't think there's a race. The behavior is just that if someone tries to evict a key that's still in-use, then we'll correctly fail to evict the key.
"Evicting a key that's still in-use" isn't supposed to happen, so printing a warning is a good idea. But I think it needs to be
pr_warn_once(), notWARN_ON_ONCE(), becauseWARN_ON_ONCE()is for kernel bugs only, not userspace bugs. It's theoretically possible for userspace to cause the same key to be used multiple times on the same disk but via different blk_crypto_key's. The keyslot manager will put these in the same keyslot, but there will be a separate eviction attempt for eachblk_crypto_key.For example, with fscrypt with -o inlinecrypt and blk-crypto-fallback, userspace could create an encrypted file using
FSCRYPT_MODE_ADIANTUMand flags == 0, then get its encryption nonce and derive the file's encryption key. Then in another directory, they could setFSCRYPT_MODE_ADIANTUMand flags ==FSCRYPT_POLICY_FLAG_DIRECT_KEY, and use the other file's encryption key as the *master* key.
Quoted 3 lines
That would be totally insane for userspace to do. But it's possible, so we can't use
WARN_ON_ONCE().
Or maybe 'struct blk_ksm_keyslot' should contain a pointer to the 'struct blk_crypto_key' rather than a copy of it? If we did that, then:
- Each duplicate blk_crypto_key would use its own keyslot and not interfere with any others.
- blk_crypto_evict_key() would be *required* to be called.
- It would be a kernel bug if blk_crypto_evict_key() were called with any pending I/O, so WARN_ON_ONCE() would be the right thing to do.
- The hash function used to find a key's keyslot would be hash_ptr(blk_crypto_key, ksm->log_slot_hashtable_size) instead of SipHash(key=perboot_key, data=raw_key).
I might be forgetting something; was there a reason we didn't do that? It wouldn't be as robust against users forgetting to call blk_crypto_evict_key(), but that would be a bug anyway.
- Eric
Re: [PATCH v10 02/12] block: Keyslot Manager for Inline Encryption
Quoted 11 lines · Eric Biggers
Or maybe 'struct
blk_ksm_keyslot' should contain a pointer to the 'structblk_crypto_key' rather than a copy of it? If we did that, then:- Each duplicate
blk_crypto_keywould use its own keyslot and not interfere with any others.-
blk_crypto_evict_key()would be *required* to be called.- It would be a kernel bug if
blk_crypto_evict_key()were called with any pending I/O, soWARN_ON_ONCE()would be the right thing to do.
Quoted 7 lines
- The hash function used to find a key's keyslot would be
hash_ptr(blk_crypto_key, ksm->log_slot_hashtable_size)instead ofSipHash(key=perboot_key, data=raw_key).I might be forgetting something; was there a reason we didn't do that? It wouldn't be as robust against users forgetting to call
blk_crypto_evict_key(), but that would be a bug anyway.
The above sounds pretty sensible to me (but I'm everything but an expert in the area).
PATCH v10 3/12 block: Inline encryption support for blk-mq
In reply to Satya TangiralaWe must have some way of letting a storage device driver know what encryption context it should use for en/decrypting a request. However, it's the upper layers (like the filesystem/fscrypt) that know about and manages encryption contexts. As such, when the upper layer submits a bio to the block layer, and this bio eventually reaches a device driver with support for inline encryption, the device driver will need to have been told the encryption context for that bio.
We want to communicate the encryption context from the upper layer to the storage device along with the bio, when the bio is submitted to the block layer. To do this, we add a struct bio_crypt_ctx to struct bio, which can represent an encryption context (note that we can't use the bi_private field in struct bio to do this because that field does not function to pass information across layers in the storage stack). We also introduce various functions to manipulate the bio_crypt_ctx and make the bio/request merging logic aware of the bio_crypt_ctx.
We also make changes to blk-mq to make it handle bios with encryption contexts. blk-mq can merge many bios into the same request. These bios need to have contiguous data unit numbers (the necessary changes to blk-merge are also made to ensure this) - as such, it suffices to keep the data unit number of just the first bio, since that's all a storage driver needs to infer the data unit number to use for each data block in each bio in a request. blk-mq keeps track of the encryption context to be used for all the bios in a request with the request's rq_crypt_ctx. When the first bio is added to an empty request, blk-mq will program the encryption context of that bio into the request_queue's keyslot manager, and store the returned keyslot in the request's rq_crypt_ctx. All the functions to operate on encryption contexts are in blk-crypto.c.
Upper layers only need to call bio_crypt_set_ctx with the encryption key, algorithm and data_unit_num; they don't have to worry about getting a keyslot for each encryption context, as blk-mq/blk-crypto handles that. Blk-crypto also makes it possible for request-based layered devices like dm-rq to make use of inline encryption hardware by cloning the rq_crypt_ctx and programming a keyslot in the new request_queue when necessary.
Note that any user of the block layer can submit bios with an encryption context, such as filesystems, device-mapper targets, etc.
--- block/Makefile | 2 +- block/bio.c | 6 + block/blk-core.c | 20 +- block/blk-crypto-internal.h | 157 ++++++++++++++ block/blk-crypto.c | 396 ++++++++++++++++++++++++++++++++++++ block/blk-map.c | 1 + block/blk-merge.c | 11 + block/blk-mq.c | 13 ++ block/blk.h | 4 + block/bounce.c | 2 + drivers/md/dm.c | 3 + include/linux/blk-crypto.h | 71 +++++++ include/linux/blk_types.h | 6 + include/linux/blkdev.h | 5 + 14 files changed, 692 insertions(+), 5 deletions(-) create mode 100644 block/blk-crypto-internal.h create mode 100644 block/blk-crypto.c
block/Makefile +1 -1
@@ -36,4 +36,4 @@ obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o -obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o blk-crypto.o
block/bio.c +6 -0
@@ -18,6 +18,7 @@#include <linux/blk-cgroup.h> #include <linux/highmem.h> #include <linux/sched/sysctl.h> +#include <linux/blk-crypto.h> #include <trace/events/block.h> #include "blk.h"
@@ -237,6 +238,8 @@ void bio_uninit(struct bio *bio)if (bio_integrity (bio))bio_integrity_free (bio); + +bio_crypt_free_ctx (bio); }EXPORT_SYMBOL (bio_uninit);
@@ -708,6 +711,8 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)__bio_clone_fast (b, bio); +bio_crypt_clone (b, bio, gfp_mask); +if (bio_integrity (bio)) {int ret;
@@ -1105,6 +1110,7 @@ void bio_advance(struct bio *bio, unsigned bytes)if (bio_integrity (bio))bio_integrity_advance (bio, bytes); +bio_crypt_advance (bio, bytes);bio_advance_iter (bio, &bio->bi_iter, bytes); }EXPORT_SYMBOL (bio_advance);
block/blk-core.c +16 -4
@@ -38,6 +38,7 @@#include <linux/debugfs.h> #include <linux/bpf.h> #include <linux/psi.h> +#include <linux/blk-crypto.h> #define CREATE_TRACE_POINTS #include <trace/events/block.h>
@@ -120,6 +121,7 @@ void blk_rq_init(struct request_queue *q, struct request *rq) rq->start_time_ns =ktime_get_ns (); rq->part =NULL ;refcount_set (&rq->ref,1 ); +blk_crypto_rq_set_defaults (rq); }EXPORT_SYMBOL (blk_rq_init);
@@ -623,6 +625,8 @@ bool bio_attempt_back_merge(struct request *req, struct bio *bio, req->biotail = bio; req->__data_len += bio->bi_iter.bi_size; +bio_crypt_free_ctx (bio); +blk_account_io_start (req, false);return true; }
@@ -647,6 +651,8 @@ bool bio_attempt_front_merge(struct request *req, struct bio *bio, req->__sector = bio->bi_iter.bi_sector; req->__data_len += bio->bi_iter.bi_size; +blk_crypto_rq_bio_prep (req, bio); +blk_account_io_start (req, false);return true; }
@@ -1072,7 +1078,8 @@ blk_qc_t generic_make_request(struct bio *bio)/* Create a fresh bio_list for all subordinate requests */ bio_list_on_stack[1 ] = bio_list_on_stack[0 ];bio_list_init (&bio_list_on_stack[0 ]); - ret = q->make_request_fn (q, bio); +if (blk_crypto_bio_prep (&bio)) + ret = q->make_request_fn (q, bio);blk_queue_exit (q);
@@ -1120,7 +1127,7 @@ blk_qc_t direct_make_request(struct bio *bio) {struct request_queue *q = bio->bi_disk->queue;bool nowait = bio->bi_opf &REQ_NOWAIT ; -blk_qc_t ret; +blk_qc_t ret =BLK_QC_T_NONE ;if (!generic_make_request_checks (bio))return BLK_QC_T_NONE ;
@@ -1132,8 +1139,8 @@ blk_qc_t direct_make_request(struct bio *bio)bio_io_error (bio);return BLK_QC_T_NONE ; } - - ret = q->make_request_fn (q, bio); +if (blk_crypto_bio_prep (&bio)) + ret = q->make_request_fn (q, bio);blk_queue_exit (q);return ret; }
@@ -1263,6 +1270,9 @@ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request * should_fail_request(&rq->rq_disk->part0, blk_rq_bytes(rq)))return BLK_STS_IOERR ; +if (blk_crypto_insert_cloned_request (rq)) +return BLK_STS_IOERR ; +if (blk_queue_io_stat (q))blk_account_io_start (rq, true);
@@ -1640,6 +1650,8 @@ int blk_rq_prep_clone(struct request *rq, struct request *rq_src, rq->ioprio = rq_src->ioprio; rq->extra_len = rq_src->extra_len; +blk_crypto_rq_prep_clone (rq, rq_src); +return 0 ; free_and_out:
block/blk-crypto-internal.h new file +157 -0
157 lines not on this page · raw message
block/blk-crypto.c new file +396 -0
396 lines not on this page · raw message
block/blk-map.c +1 -0
@@ -549,6 +549,7 @@ int blk_rq_append_bio(struct request *rq, struct bio **bio) rq->biotail->bi_next = *bio; rq->biotail = *bio; rq->__data_len += (*bio)->bi_iter.bi_size; +bio_crypt_free_ctx (*bio); }return 0 ;
block/blk-merge.c +11 -0
@@ -596,6 +596,8 @@ int ll_back_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segs)if (blk_integrity_rq (req) &&integrity_req_gap_back_merge (req, bio))return 0 ; +if (!bio_crypt_ctx_back_mergeable (req, bio)) +return 0 ;if (blk_rq_sectors (req) +bio_sectors (bio) >blk_rq_get_max_sectors (req,blk_rq_pos (req))) {req_set_nomerge (req->q, req);
@@ -612,6 +614,8 @@ int ll_front_merge_fn(struct request *req, struct bio *bio, unsigned int nr_segsif (blk_integrity_rq (req) &&integrity_req_gap_front_merge (req, bio))return 0 ; +if (!bio_crypt_ctx_front_mergeable (req, bio)) +return 0 ;if (blk_rq_sectors (req) +bio_sectors (bio) >blk_rq_get_max_sectors (req, bio->bi_iter.bi_sector)) {req_set_nomerge (req->q, req);
@@ -661,6 +665,9 @@ static int ll_merge_requests_fn(struct request_queue *q, struct request *req,if (blk_integrity_merge_rq (q, req, next) == false)return 0 ; +if (!bio_crypt_ctx_merge_rq (req, next)) +return 0 ; +/* Merge is OK... */ req->nr_phys_segments = total_phys_segments;return 1 ;
@@ -885,6 +892,10 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)if (blk_integrity_merge_bio (rq->q, rq, bio) == false)return false; +/* Only merge if the crypt contexts are compatible */ +if (!bio_crypt_rq_ctx_compatible (rq, bio)) +return false; +/* must be using the same buffer */ if (req_op (rq) ==REQ_OP_WRITE_SAME && !blk_write_same_mergeable (rq->bio, bio))
block/blk-mq.c +13 -0
@@ -26,6 +26,7 @@#include <linux/delay.h> #include <linux/crash_dump.h> #include <linux/prefetch.h> +#include <linux/blk-crypto.h> #include <trace/events/block.h>
@@ -317,6 +318,7 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,#if defined(CONFIG_BLK_DEV_INTEGRITY ) rq->nr_integrity_segments =0 ;#endif +blk_crypto_rq_set_defaults (rq);/* tag was already set */ rq->extra_len =0 ;WRITE_ONCE (rq->deadline,0 );
@@ -474,6 +476,7 @@ static void __blk_mq_free_request(struct request *rq)struct blk_mq_hw_ctx *hctx = rq->mq_hctx;const int sched_tag = rq->internal_tag; +blk_crypto_free_request (rq);blk_pm_mark_last_busy (rq); rq->mq_hctx =NULL ;if (rq->tag !=-1 )
@@ -1978,6 +1981,7 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)struct request *same_queue_rq =NULL ;unsigned int nr_segs;blk_qc_t cookie; +blk_status_t ret;blk_queue_bounce (q, &bio);__blk_queue_split (q, &bio, &nr_segs);
@@ -2011,6 +2015,15 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)blk_mq_bio_to_request (rq, bio, nr_segs); + ret =blk_crypto_init_request (rq); +if (ret !=BLK_STS_OK ) { + bio->bi_status = ret; +bio_endio (bio); +blk_mq_free_request (rq); +return BLK_QC_T_NONE ; + } + + plug =blk_mq_plug (q, bio);if (unlikely (is_flush_fua)) {/* Bypass scheduler for flush requests */
block/blk.h +4 -0
@@ -5,7 +5,9 @@#include <linux/idr.h> #include <linux/blk-mq.h> #include <linux/part_stat.h> +#include <linux/blk-crypto.h> #include <xen/xen.h> +#include "blk-crypto-internal.h" #include "blk-mq.h" #include "blk-mq-sched.h"
@@ -118,6 +120,8 @@ static inline void blk_rq_bio_prep(struct request *rq, struct bio *bio,if (bio->bi_disk) rq->rq_disk = bio->bi_disk; + +blk_crypto_rq_bio_prep (rq, bio); }#ifdef CONFIG_BLK_DEV_INTEGRITY
block/bounce.c +2 -0
@@ -267,6 +267,8 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,break ; } +bio_crypt_clone (bio, bio_src, gfp_mask); +if (bio_integrity (bio_src)) {int ret;
drivers/md/dm.c +3 -0
@@ -26,6 +26,7 @@#include <linux/pr.h> #include <linux/refcount.h> #include <linux/part_stat.h> +#include <linux/blk-crypto.h> #define DM_MSG_PREFIX "core"
@@ -1305,6 +1306,8 @@ static int clone_bio(struct dm_target_io *tio, struct bio *bio,__bio_clone_fast (clone, bio); +bio_crypt_clone (clone, bio,GFP_NOIO ); +if (bio_integrity (bio)) {int r;
include/linux/blk-crypto.h +71 -0
81 lines not on this page · raw message
include/linux/blk_types.h +6 -0
@@ -18,6 +18,7 @@ struct block_device;struct io_context ;struct cgroup_subsys_state ;typedef void (bio_end_io_t ) (struct bio *); +struct bio_crypt_ctx ;/* * Block error status values. See block/blk-core:blk_errors for the details.
@@ -173,6 +174,11 @@ struct bio {u64 bi_iocost_cost;#endif #endif + +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *bi_crypt_context; +#endif +union {#if defined(CONFIG_BLK_DEV_INTEGRITY )struct bio_integrity_payload *bi_integrity;/* data integrity */
include/linux/blkdev.h +5 -0
@@ -224,6 +224,11 @@ struct request {unsigned short nr_integrity_segments;#endif +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +struct bio_crypt_ctx *crypt_ctx; +struct blk_ksm_keyslot *crypt_keyslot; +#endif +unsigned short write_hint;unsigned short ioprio;
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Re: [PATCH v10 03/12] block: Inline encryption support for blk-mq
Quoted 8 lines
block/blk-crypto.c new file +8
-0+/** +* __blk_crypto_bio_prep - Prepare bio for inline encryption +* +* @bio_ptr: pointer to original bio pointer +* +* Succeeds if the bio doesn't have inline encryption enabled or if the bio +* crypt context provided for the bio is supported by the underlying device's +* inline encryption hardware. Ends the bio with error otherwise.
block/blk-crypto.c line 241
Quoted 10 lines
block/blk-crypto.c new file +10
-0+* +* Caller must ensure bio has bio_crypt_ctx. +* +* Return: true on success; false on error (and bio->bi_status will be set +* appropriately, and bio_endio() will have been called so bio +* submission should abort). +*/ +bool __blk_crypto_bio_prep (struct bio **bio_ptr) +{ +struct bio *bio = *bio_ptr;
Why is the bio passed by references? As far as I can see it is never changed.
Quoted 4 lines
block/blk-crypto.c new file +4
-0+ +/* Error if bio has no data. */ +if (WARN_ON_ONCE (!bio_has_data (bio))) + goto fail;
block/blk-crypto.c line 255
Quoted 10 lines
block/blk-crypto.c new file +10
-0+ +if (!bio_crypt_check_alignment (bio)) + goto fail; + +/* +* Success if device supports the encryption context, and blk-integrity +* isn't supported by device/is turned off. +*/ +if (!blk_ksm_crypto_cfg_supported (bio->bi_disk->queue->ksm, + &bio->bi_crypt_context->bc_key->crypto_cfg)) {
The indentation here looks odd.
block/blk-crypto.c line 262
Quoted 7 lines
block/blk-crypto.c new file +7
-0+ bio->bi_status =BLK_STS_NOTSUPP ; + goto fail; + } + +return true; +fail: +bio_endio (*bio_ptr);
This seems to fail to set a status for the bio_has_data case, and setting the status for bio_crypt_check_alignment should be moved to here, where we also end the IO.
Quoted 8 lines
+ *
__blk_crypto_rq_bio_prep- Prepare a request when its first bio is inserted
+ *
+ * @rq: The request to prepare
+ * @bio: The first bio being inserted into the request
+ *
+ * Frees the bio crypt context in the request's oldrq->crypt_ctx, if any, and
+ * moves the bio crypt context of the bio into the request'srq->crypt_ctx.
+ */
block/blk-crypto.c line 280
Quoted 6 lines
block/blk-crypto.c new file +6
-0+void __blk_crypto_rq_bio_prep (struct request *rq,struct bio *bio) +{ +mempool_free (rq->crypt_ctx, bio_crypt_ctx_pool); + rq->crypt_ctx = bio->bi_crypt_context; + bio->bi_crypt_context =NULL ; +}
This seems to be called in case of the both the initial request creation and merging a bio into the request (although only the front merge and not the back merge, which seems weird).
But even then the behvior seems odd as bio->bi_crypt_context becomes NULL as soon as the bio is attached to a request, which seems like a somewhat dangerous condition. Maybe it works, but is it really worth saving a little memory? Why not just propagate the context of the first bio to the request, and free them when the bio is completed?
Why do we always take the context from the bio instead of keeping the one in the request? After all we merge the bio into the request.
block/blk-crypto.c line 284
Quoted 3 lines
block/blk-crypto.c new file +3
-0+void __blk_crypto_rq_prep_clone (struct request *dst,struct request *src) +{ + dst->crypt_ctx = src->crypt_ctx;
Probably worth just open coding in the only caller..
block/blk-crypto.c line 296
Quoted 11 lines
block/blk-crypto.c new file +11
-0+ +/** +* __blk_crypto_insert_cloned_request - Prepare a cloned request to be inserted +* into a request queue. +* @rq: the request being queued +* +* Return: BLK_STS_OK on success, nonzero on error. +*/ +blk_status_t __blk_crypto_insert_cloned_request (struct request *rq) +{ +return blk_crypto_init_request (rq);
Same.
block/blk-mq.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2024
Quoted 12 lines
block/blk-mq.c +7
-0__blk_queue_split (q, &bio, &nr_segs);@@ -2011,6 +2015,15 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)blk_mq_bio_to_request (rq, bio, nr_segs); + ret =blk_crypto_init_request (rq); +if (ret !=BLK_STS_OK ) { + bio->bi_status = ret; +bio_endio (bio); +blk_mq_free_request (rq); +return BLK_QC_T_NONE ; + }
Didn't Eric have a comment last round that we shoul dtry this before attaching the bio to simplify error handling?
include/linux/blk-crypto.h struct blk_crypto_key { · line 56
Quoted 1 line
include/linux/blk-crypto.h +1
-0+#define BLK_CRYPTO_DUN_ARRAY_SIZE (BLK_CRYPTO_MAX_IV_SIZE/sizeof(u64))
Please use whitespace before and after operators.
Re: [PATCH v10 03/12] block: Inline encryption support for blk-mq
Quoted 13 lines · Christoph Hellwig
Quoted 13 lines
+/**
+ *__blk_crypto_bio_prep- Prepare bio for inline encryption
+ *
+ * @bio_ptr: pointer to original bio pointer
+ *
+ * Succeeds if the bio doesn't have inline encryption enabled or if the bio
+ * crypt context provided for the bio is supported by the underlying device's
+ * inline encryption hardware. Ends the bio with error otherwise.
+ *
+ * Caller must ensure bio hasbio_crypt_ctx.
+ *
+ * Return: true on success; false on error (andbio->bi_statuswill be set
+ * appropriately, andbio_endio()will have been called so bio
Quoted 9 lines
block/blk-crypto.c line 241
Quoted 5 lines
+ * submission should abort).
+ */block/blk-crypto.c new file +3
-0+bool __blk_crypto_bio_prep (struct bio **bio_ptr) +{ +struct bio *bio = *bio_ptr;Why is the bio passed by references? As far as I can see it is never changed.
It's modified by additions in the next patch in the series and I thought I should introduce the function with the final type from the get go - is that alright?
Quoted 82 lines
Quoted 4 lines
block/blk-crypto.c new file +4
-0+ +/* Error if bio has no data. */ +if (WARN_ON_ONCE (!bio_has_data (bio))) + goto fail;block/blk-crypto.c line 255
Quoted 10 lines
block/blk-crypto.c new file +10
-0+ +if (!bio_crypt_check_alignment (bio)) + goto fail; + +/* +* Success if device supports the encryption context, and blk-integrity +* isn't supported by device/is turned off. +*/ +if (!blk_ksm_crypto_cfg_supported (bio->bi_disk->queue->ksm, + &bio->bi_crypt_context->bc_key->crypto_cfg)) {The indentation here looks odd.
block/blk-crypto.c line 262
Quoted 7 lines
block/blk-crypto.c new file +7
-0+ bio->bi_status =BLK_STS_NOTSUPP ; + goto fail; + } + +return true; +fail: +bio_endio (*bio_ptr);This seems to fail to set a status for the
bio_has_datacase, and setting the status forbio_crypt_check_alignmentshould be moved to here, where we also end the IO.Quoted 8 lines
+ *
__blk_crypto_rq_bio_prep- Prepare a request when its first bio is inserted
+ *
+ * @rq: The request to prepare
+ * @bio: The first bio being inserted into the request
+ *
+ * Frees the bio crypt context in the request's oldrq->crypt_ctx, if any, and
+ * moves the bio crypt context of the bio into the request'srq->crypt_ctx.
+ */block/blk-crypto.c line 280
Quoted 6 lines
block/blk-crypto.c new file +6
-0+void __blk_crypto_rq_bio_prep (struct request *rq,struct bio *bio) +{ +mempool_free (rq->crypt_ctx, bio_crypt_ctx_pool); + rq->crypt_ctx = bio->bi_crypt_context; + bio->bi_crypt_context =NULL ; +}This seems to be called in case of the both the initial request creation and merging a bio into the request (although only the front merge and not the back merge, which seems weird).
But even then the behvior seems odd as
bio->bi_crypt_contextbecomes NULL as soon as the bio is attached to a request, which seems like a somewhat dangerous condition. Maybe it works, but is it really worth saving a little memory? Why not just propagate the context of the first bio to the request, and free them when the bio is completed?Why do we always take the context from the bio instead of keeping the one in the request? After all we merge the bio into the request.
block/blk-crypto.c line 284
Quoted 3 lines
block/blk-crypto.c new file +3
-0+void __blk_crypto_rq_prep_clone (struct request *dst,struct request *src) +{ + dst->crypt_ctx = src->crypt_ctx;Probably worth just open coding in the only caller..
block/blk-crypto.c line 296
Quoted 11 lines
block/blk-crypto.c new file +11
-0+ +/** +* __blk_crypto_insert_cloned_request - Prepare a cloned request to be inserted +* into a request queue. +* @rq: the request being queued +* +* Return: BLK_STS_OK on success, nonzero on error. +*/ +blk_status_t __blk_crypto_insert_cloned_request (struct request *rq) +{ +return blk_crypto_init_request (rq);Same.
Quoted 5 lines
block/blk-mq.c +0
-0__blk_queue_split (q, &bio, &nr_segs);@@ -2011,6 +2015,15 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)blk_mq_bio_to_request (rq, bio, nr_segs);
Quoted 11 lines
block/blk-mq.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2024
Quoted 7 lines
block/blk-mq.c +7
-0+ ret =blk_crypto_init_request (rq); +if (ret !=BLK_STS_OK ) { + bio->bi_status = ret; +bio_endio (bio); +blk_mq_free_request (rq); +return BLK_QC_T_NONE ; + }Didn't Eric have a comment last round that we shoul dtry this before attaching the bio to simplify error handling?
In the previous round, I believe Eric commented that I should call blk_crypto_init_request after bio_to_request so that we can do away with some of the arguments to blk_crypto_init_request and also a boilerplate function used only while calling blk_crypto_init_request. I realize you wrote "And we can fail just the request on an error, so yes this doesn't seem too bad." in response to this particular comment of Eric's, and it seems I might not actually have understood what that meant - did you have something in mind different from what I'm doing here?
Quoted 3 lines
include/linux/blk-crypto.h struct blk_crypto_key { · line 56
Quoted 1 line
include/linux/blk-crypto.h +1
-0+#define BLK_CRYPTO_DUN_ARRAY_SIZE (BLK_CRYPTO_MAX_IV_SIZE/sizeof(u64))Please use whitespace before and after operators.
Re: [PATCH v10 03/12] block: Inline encryption support for blk-mq
Quoted 3 lines · Satya Tangirala
It's modified by additions in the next patch in the series and I thought I should introduce the function with the final type from the get go - is that alright?
It is probably ok, let me review the next patch in more detail.
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
block/blk-mq.c +0
-0__blk_queue_split (q, &bio, &nr_segs);@@ -2011,6 +2015,15 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)blk_mq_bio_to_request (rq, bio, nr_segs);Quoted 11 lines
block/blk-mq.c static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) · line 2024
Quoted 7 lines
block/blk-mq.c +7
-0+ ret =blk_crypto_init_request (rq); +if (ret !=BLK_STS_OK ) { + bio->bi_status = ret; +bio_endio (bio); +blk_mq_free_request (rq); +return BLK_QC_T_NONE ; + }Didn't Eric have a comment last round that we shoul dtry this before attaching the bio to simplify error handling?
Quoted 9 lines
In the previous round, I believe Eric commented that I should call
blk_crypto_init_requestafterbio_to_requestso that we can do away with some of the arguments toblk_crypto_init_requestand also a boilerplate function used only while callingblk_crypto_init_request. I realize you wrote "And we can fail just the request on an error, so yes this doesn't seem too bad." in response to this particular comment of Eric's, and it seems I might not actually have understood what that meant - did you have something in mind different from what I'm doing here?
No, this looks ok, sorry for the noise.
PATCH v10 4/12 block: Make blk-integrity preclude hardware inline encryption
In reply to Satya TangiralaWhenever a device supports blk-integrity, the kernel will now always pretend that the device doesn't support inline encryption (essentially by setting the keyslot manager in the request queue to NULL).
There's no hardware currently that supports both integrity and inline encryption. However, it seems possible that there will be such hardware in the near future (like the NVMe key per I/O support that might support both inline encryption and PI).
But properly integrating both features is not trivial, and without real hardware that implements both, it is difficult to tell if it will be done correctly by the majority of hardware that support both. So it seems best not to support both features together right now, and to decide what to do at probe time.
--- block/bio-integrity.c | 3 +++ block/blk-integrity.c | 7 +++++++ block/keyslot-manager.c | 19 +++++++++++++++++++ include/linux/blkdev.h | 30 ++++++++++++++++++++++++++++++ 4 files changed, 59 insertions(+)
block/bio-integrity.c +3 -0
@@ -42,6 +42,9 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio,struct bio_set *bs = bio->bi_pool;unsigned inline_vecs; +if (WARN_ON_ONCE (bio_has_crypt_ctx (bio))) +return ERR_PTR (-EOPNOTSUPP ); +if (!bs || !mempool_initialized (&bs->bio_integrity_pool)) { bip =kmalloc (struct_size (bip, bip_inline_vecs, nr_vecs), gfp_mask); inline_vecs = nr_vecs;
block/blk-integrity.c +7 -0
@@ -409,6 +409,13 @@ void blk_integrity_register(struct gendisk *disk, struct blk_integrity *template bi->tag_size = template->tag_size; disk->queue->backing_dev_info->capabilities |=BDI_CAP_STABLE_WRITES ; + +#ifdef CONFIG_BLK_INLINE_ENCRYPTION +if (disk->queue->ksm) { +pr_warn ("blk-integrity: Integrity and hardware inline encryption are not supported together. Unregistering keyslot manager from request queue, to disable hardware inline encryption.\n" ); +blk_ksm_unregister (disk->queue); + } +#endif }EXPORT_SYMBOL (blk_integrity_register);
block/keyslot-manager.c +19 -0
@@ -25,6 +25,9 @@* Upper layers will call blk_ksm_get_slot_for_key() to program a * key into some slot in the inline encryption hardware. */ + +#define pr_fmt (fmt) "blk_crypto: " fmt +#include <crypto/algapi.h> #include <linux/keyslot-manager.h> #include <linux/atomic.h>
@@ -376,3 +379,19 @@ void blk_ksm_destroy(struct blk_keyslot_manager *ksm)memzero_explicit (ksm,sizeof (*ksm)); }EXPORT_SYMBOL_GPL (blk_ksm_destroy); + +bool blk_ksm_register (struct blk_keyslot_manager *ksm,struct request_queue *q) +{ +if (blk_integrity_queue_supports_integrity (q)) { +pr_warn ("Integrity and hardware inline encryption are not supported together. Hardware inline encryption is being disabled.\n" ); +return false; + } + q->ksm = ksm; +return true; +} +EXPORT_SYMBOL_GPL (blk_ksm_register); + +void blk_ksm_unregister (struct request_queue *q) +{ + q->ksm =NULL ; +}
include/linux/blkdev.h +30 -0
@@ -1562,6 +1562,12 @@ struct blk_integrity *bdev_get_integrity(struct block_device *bdev)return blk_get_integrity (bdev->bd_disk); } +static inline bool +blk_integrity_queue_supports_integrity (struct request_queue *q) +{ +return q->integrity.profile; +} +static inline bool blk_integrity_rq (struct request *rq) {return rq->cmd_flags &REQ_INTEGRITY ;
@@ -1642,6 +1648,11 @@ static inline struct blk_integrity *blk_get_integrity(struct gendisk *disk) {return NULL ; } +static inline bool +blk_integrity_queue_supports_integrity (struct request_queue *q) +{ +return false; +}static inline int blk_integrity_compare (struct gendisk *a,struct gendisk *b) {return 0 ;
@@ -1693,6 +1704,25 @@ static inline struct bio_vec *rq_integrity_vec(struct request *rq)#endif /* CONFIG_BLK_DEV_INTEGRITY */ +#ifdef CONFIG_BLK_INLINE_ENCRYPTION + +bool blk_ksm_register (struct blk_keyslot_manager *ksm,struct request_queue *q); + +void blk_ksm_unregister (struct request_queue *q); + +#else /* CONFIG_BLK_INLINE_ENCRYPTION */ + +static inline bool blk_ksm_register (struct blk_keyslot_manager *ksm, +struct request_queue *q) +{ +return true; +} + +static inline void blk_ksm_unregister (struct request_queue *q) { } + +#endif /* CONFIG_BLK_INLINE_ENCRYPTION */ + +struct block_device_operations {int (*open) (struct block_device *,fmode_t );void (*release) (struct gendisk *,fmode_t );
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 5/12 block: blk-crypto-fallback for Inline Encryption
In reply to Satya TangiralaBlk-crypto delegates crypto operations to inline encryption hardware when available. The separately configurable blk-crypto-fallback contains a software fallback to the kernel crypto API - when enabled, blk-crypto will use this fallback for en/decryption when inline encryption hardware is not available. This lets upper layers not have to worry about whether or not the underlying device has support for inline encryption before deciding to specify an encryption context for a bio. It also allows for testing without actual inline encryption hardware - in particular, it makes it possible to test the inline encryption code in ext4 and f2fs simply by running xfstests with the inlinecrypt mount option, which in turn allows for things like the regular upstream regression testing of ext4 to cover the inline encryption code paths. For more details, refer to Documentation/block/inline-encryption.rst.
--- Documentation/block/index.rst | 1 + block/Kconfig | 10 + block/Makefile | 1 + block/blk-crypto-fallback.c | 655 ++++++++++++++++++++++++++++++++++ block/blk-crypto-internal.h | 35 ++ block/blk-crypto.c | 67 ++-- include/linux/blk-crypto.h | 2 +- 7 files changed, 750 insertions(+), 21 deletions(-) create mode 100644 block/blk-crypto-fallback.c
Documentation/block/index.rst +1 -0
@@ -14,6 +14,7 @@ Block cmdline-partition data-integrity deadline-iosched + inline-encryption ioprio kyber-iosched null_blk
block/Kconfig +10 -0
@@ -192,6 +192,16 @@ config BLK_INLINE_ENCRYPTION block layer handle encryption, so users can take advantage of inline encryption hardware if present. +config BLK_INLINE_ENCRYPTION_FALLBACK + bool "Enable crypto API fallback for blk-crypto" + depends on BLK_INLINE_ENCRYPTION + select CRYPTO + select CRYPTO_SKCIPHER + help + Enabling this lets the block layer handle inline encryption + by falling back to the kernel crypto API when inline + encryption hardware is not present. + menu "Partition Types" source "block/partitions/Kconfig"
block/Makefile +1 -0
@@ -37,3 +37,4 @@ obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o obj-$(CONFIG_BLK_PM) += blk-pm.o obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o blk-crypto.o +obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) += blk-crypto-fallback.o
block/blk-crypto-fallback.c new file +655 -0
655 lines not on this page · raw message
block/blk-crypto-internal.h +35 -0
@@ -11,10 +11,13 @@/* Represents a crypto mode supported by blk-crypto */ struct blk_crypto_mode { +const char *cipher_str;/* crypto API name (for fallback case) */ unsigned int keysize;/* key size in bytes */ unsigned int ivsize;/* iv size in bytes */ }; +extern const struct blk_crypto_mode blk_crypto_modes[]; +#ifdef CONFIG_BLK_INLINE_ENCRYPTION void bio_crypt_dun_increment(u64 dun[BLK_CRYPTO_DUN_ARRAY_SIZE],
@@ -154,4 +157,36 @@ static inline blk_status_t blk_crypto_insert_cloned_request(struct request *rq) return BLK_STS_OK; } +#ifdef CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK + +int blk_crypto_fallback_start_using_mode (enum blk_crypto_mode_num mode_num); + +bool blk_crypto_fallback_bio_prep (struct bio **bio_ptr); + +int blk_crypto_fallback_evict_key (const struct blk_crypto_key *key); + +#else /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */ + +static inline int +blk_crypto_fallback_start_using_mode (enum blk_crypto_mode_num mode_num) +{ +pr_warn_once ("crypto API fallback is disabled\n" ); +return -ENOPKG ; +} + +static inline bool blk_crypto_fallback_bio_prep (struct bio **bio_ptr) +{ +pr_warn_once ("crypto API fallback disabled; failing request.\n" ); + (*bio_ptr)->bi_status =BLK_STS_NOTSUPP ; +return false; +} + +static inline int +blk_crypto_fallback_evict_key (const struct blk_crypto_key *key) +{ +return 0 ; +} + +#endif /* CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK */ +#endif /* __LINUX_BLK_CRYPTO_INTERNAL_H */
block/blk-crypto.c +47 -20
133 lines not on this page · raw message
include/linux/blk-crypto.h +1 -1
@@ -62,7 +62,7 @@ struct blk_crypto_key {* * A bio_crypt_ctx specifies that the contents of the bio will be encrypted (for * write requests) or decrypted (for read requests) inline by the storage device -* or controller. +* or controller, or by the crypto API fallback. */ struct bio_crypt_ctx {const struct blk_crypto_key *bc_key;
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Re: [PATCH v10 05/12] block: blk-crypto-fallback for Inline Encryption
Quoted 18 lines
Documentation/block/index.rst +1
-0@@ -14,6 +14,7 @@ Block cmdline-partition data-integrity deadline-iosched + inline-encryption ioprio kyber-iosched null_blkblock/blk-crypto.c +0
-0
block/blk-crypto.c line 9
Quoted 7 lines
block/blk-crypto.c +4
-0@@ -3,6 +3,10 @@* Copyright 2019 Google LLC */ +/* +* Refer to Documentation/block/inline-encryption.rst for detailed explanation. +*/ +
These hunks should be in other patches.
PATCH v10 6/12 scsi: ufs: UFS driver v2.1 spec crypto additions
In reply to Satya TangiralaAdd the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI specification in preparation to add support for inline encryption to UFS.
--- drivers/scsi/ufs/ufshcd.c | 2 ++ drivers/scsi/ufs/ufshcd.h | 6 ++++ drivers/scsi/ufs/ufshci.h | 67 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 73 insertions(+), 2 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
@@ -4759,6 +4759,8 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp)case OCS_MISMATCH_RESP_UPIU_SIZE :case OCS_PEER_COMM_FAILURE :case OCS_FATAL_ERROR : +case OCS_INVALID_CRYPTO_CONFIG : +case OCS_GENERAL_CRYPTO_ERROR :default : result |=DID_ERROR <<16 ; dev_err(hba->dev,
drivers/scsi/ufs/ufshcd.h +6 -0
@@ -542,6 +542,12 @@ enum ufshcd_caps {* for userspace to control the power management. */ UFSHCD_CAP_RPM_AUTOSUSPEND =1 <<6 , + +/* +* This capability allows the host controller driver to use the +* inline crypto engine, if it is present +*/ + UFSHCD_CAP_CRYPTO = (1 << 7), };/**
drivers/scsi/ufs/ufshci.h +65 -2
106 lines not on this page · raw message
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 7/12 scsi: ufs: UFS crypto API
In reply to Satya TangiralaIntroduce functions to manipulate UFS inline encryption hardware in line with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot manager.
The UFS crypto API will assume by default that a vendor driver doesn't support UFS crypto, even if the hardware advertises the capability, because a lot of hardware requires some special handling that's not specified in the aforementioned JEDEC spec. Each vendor driver must explicity set hba->caps |= UFSHCD_CAP_CRYPTO before ufshcd_hba_init_crypto is called to opt-in to UFS crypto support.
--- drivers/scsi/ufs/Kconfig | 9 ++ drivers/scsi/ufs/Makefile | 1 + drivers/scsi/ufs/ufshcd-crypto.c | 226 +++++++++++++++++++++++++++++++ drivers/scsi/ufs/ufshcd-crypto.h | 42 ++++++ drivers/scsi/ufs/ufshcd.h | 12 ++ 5 files changed, 290 insertions(+) create mode 100644 drivers/scsi/ufs/ufshcd-crypto.c create mode 100644 drivers/scsi/ufs/ufshcd-crypto.h
drivers/scsi/ufs/Kconfig +9 -0
@@ -160,3 +160,12 @@ config SCSI_UFS_BSG Select this if you need a bsg device node for your UFS controller. If unsure, say N. + +config SCSI_UFS_CRYPTO + bool "UFS Crypto Engine Support" + depends on SCSI_UFSHCD && BLK_INLINE_ENCRYPTION + help + Enable Crypto Engine Support in UFS. + Enabling this makes it possible for the kernel to use the crypto + capabilities of the UFS device (if present) to perform crypto + operations on data being transferred to/from the device.
drivers/scsi/ufs/Makefile +1 -0
@@ -7,6 +7,7 @@ obj-$(CONFIG_SCSI_UFS_QCOM) += ufs-qcom.o obj-$(CONFIG_SCSI_UFSHCD) += ufshcd-core.o ufshcd-core-y += ufshcd.o ufs-sysfs.o ufshcd-core-$(CONFIG_SCSI_UFS_BSG) += ufs_bsg.o +ufshcd-core-$(CONFIG_SCSI_UFS_CRYPTO) += ufshcd-crypto.o obj-$(CONFIG_SCSI_UFSHCD_PCI) += ufshcd-pci.o obj-$(CONFIG_SCSI_UFSHCD_PLATFORM) += ufshcd-pltfrm.o obj-$(CONFIG_SCSI_UFS_HISI) += ufs-hisi.o
drivers/scsi/ufs/ufshcd-crypto.c new file +226 -0
226 lines not on this page · raw message
drivers/scsi/ufs/ufshcd-crypto.h new file +42 -0
42 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +12 -0
@@ -57,6 +57,7 @@#include <linux/regulator/consumer.h> #include <linux/bitfield.h> #include <linux/devfreq.h> +#include <linux/keyslot-manager.h> #include "unipro.h" #include <asm/irq.h>
@@ -601,6 +602,10 @@ enum ufshcd_caps {* @is_urgent_bkops_lvl_checked: keeps track if the urgent bkops level for * device is known or not. * @scsi_block_reqs_cnt: reference counting for scsi block requests +* @crypto_capabilities: Content of crypto capabilities register (0x100) +* @crypto_cap_array: Array of crypto capabilities +* @crypto_cfg_register: Start of the crypto cfg array +* @ksm: the keyslot manager tied to this hba */ struct ufs_hba {void __iomem *mmio_base;
@@ -720,6 +725,13 @@ struct ufs_hba {struct device bsg_dev;struct request_queue *bsg_queue; + +#ifdef CONFIG_SCSI_UFS_CRYPTO +union ufs_crypto_capabilities crypto_capabilities; +union ufs_crypto_cap_entry *crypto_cap_array; +u32 crypto_cfg_register; +struct blk_keyslot_manager ksm; +#endif };/* Returns true if clocks can be gated. Otherwise false */
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 8/12 scsi: ufs: Add inline encryption support to UFS
In reply to Satya TangiralaWire up ufshcd.c with the UFS Crypto API, the block layer inline encryption additions and the keyslot manager.
--- drivers/scsi/ufs/ufshcd-crypto.h | 18 +++++++++++++ drivers/scsi/ufs/ufshcd.c | 44 ++++++++++++++++++++++++++++---- drivers/scsi/ufs/ufshcd.h | 6 +++++ 3 files changed, 63 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd-crypto.h +18 -0
@@ -10,6 +10,20 @@#include "ufshcd.h" #include "ufshci.h" +static inline void ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) +{ +struct request *rq = cmd->request; + +if (rq->crypt_keyslot) { + lrbp->crypto_key_slot =blk_ksm_get_slot_idx (rq->crypt_keyslot); + lrbp->data_unit_num = rq->crypt_ctx->bc_dun[0 ]; + }else { + lrbp->crypto_key_slot =-1 ; + } +} +bool ufshcd_crypto_enable (struct ufs_hba *hba);int ufshcd_hba_init_crypto (struct ufs_hba *hba);
@@ -21,6 +35,10 @@ void ufshcd_crypto_destroy_keyslot_manager(struct ufs_hba *hba);#else /* CONFIG_SCSI_UFS_CRYPTO */ +static inline void ufshcd_prepare_lrbp_crypto (struct ufs_hba *hba, +struct scsi_cmnd *cmd, +struct ufshcd_lrb *lrbp) { } +static inline bool ufshcd_crypto_enable (struct ufs_hba *hba) {return false;
drivers/scsi/ufs/ufshcd.c +39 -5
111 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +6 -0
@@ -170,6 +170,8 @@ struct ufs_pm_lvl_states {* @intr_cmd: Interrupt command (doesn't participate in interrupt aggregation) * @issue_time_stamp: time stamp for debug purposes * @compl_time_stamp: time stamp for statistics +* @crypto_key_slot: the key slot to use for inline crypto (-1 if none) +* @data_unit_num: the data unit number for the first block for inline crypto * @req_abort_skip: skip request abort task flag */ struct ufshcd_lrb {
@@ -194,6 +196,10 @@ struct ufshcd_lrb {bool intr_cmd;ktime_t issue_time_stamp;ktime_t compl_time_stamp; +#ifdef CONFIG_SCSI_UFS_CRYPTO +int crypto_key_slot; +u64 data_unit_num; +#endif bool req_abort_skip; };
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 9/12 fs: introduce SB_INLINECRYPT
In reply to Satya TangiralaIntroduce SB_INLINECRYPT, which is set by filesystems that wish to use blk-crypto for file content en/decryption. This flag maps to the '-o inlinecrypt' mount option which multiple filesystems will implement, and code in fs/crypto/ needs to be able to check for this mount option in a filesystem-independent way.
--- fs/proc_namespace.c | 1 + include/linux/fs.h | 1 + 2 files changed, 2 insertions(+)
fs/proc_namespace.c +1 -0
@@ -49,6 +49,7 @@ static int show_sb_opts(struct seq_file *m, struct super_block *sb) { SB_DIRSYNC, ",dirsync" }, { SB_MANDLOCK, ",mand" }, { SB_LAZYTIME, ",lazytime" }, + { SB_INLINECRYPT, ",inlinecrypt" }, { 0, NULL } };const struct proc_fs_info *fs_infop;
include/linux/fs.h +1 -0
@@ -1376,6 +1376,7 @@ extern int send_sigurg(struct fown_struct *fown);#define SB_NODIRATIME 2048/* Do not update directory access times */ #define SB_SILENT 32768#define SB_POSIXACL (1<<16)/* VFS does not apply the umask */ +#define SB_INLINECRYPT (1<<17)/* Use blk-crypto for encrypted files */ #define SB_KERNMOUNT (1<<22)/* this is a kern_mount call */ #define SB_I_VERSION (1<<23)/* Update inode I_version field */ #define SB_LAZYTIME (1<<25)/* Update the on-disk [acm]times lazily */
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 10/12 fscrypt: add inline encryption support
In reply to Satya TangiralaAdd support for inline encryption to fs/crypto/. With "inline encryption", the block layer handles the decryption/encryption as part of the bio, instead of the filesystem doing the crypto itself via Linux's crypto API. This model is needed in order to take advantage of the inline encryption hardware present on most modern mobile SoCs.
To use inline encryption, the filesystem needs to be mounted with '-o inlinecrypt'. The contents of any encrypted files will then be encrypted using blk-crypto, instead of using the traditional filesystem-layer crypto. Fscrypt still provides the key and IV to use, and the actual ciphertext on-disk is still the same; therefore it's testable using the existing fscrypt ciphertext verification tests.
Note that since blk-crypto has a fallack to Linux's crypto API, and also supports all the encryption modes currently supported by fscrypt, this feature is usable and testable even without actual inline encryption hardware.
Per-filesystem changes will be needed to set encryption contexts when submitting bios and to implement the 'inlinecrypt' mount option. This patch just adds the common code.
--- fs/crypto/Kconfig | 6 + fs/crypto/Makefile | 1 + fs/crypto/bio.c | 50 +++++ fs/crypto/crypto.c | 2 +- fs/crypto/fname.c | 4 +- fs/crypto/fscrypt_private.h | 120 ++++++++++-- fs/crypto/inline_crypt.c | 365 ++++++++++++++++++++++++++++++++++++ fs/crypto/keyring.c | 4 +- fs/crypto/keysetup.c | 92 ++++++--- fs/crypto/keysetup_v1.c | 16 +- include/linux/fscrypt.h | 57 ++++++ 11 files changed, 660 insertions(+), 57 deletions(-) create mode 100644 fs/crypto/inline_crypt.c
fs/crypto/Kconfig +6 -0
@@ -24,3 +24,9 @@ config FS_ENCRYPTION_ALGS select CRYPTO_SHA256 select CRYPTO_SHA512 select CRYPTO_XTS + +config FS_ENCRYPTION_INLINE_CRYPT + bool "Enable fscrypt to use inline crypto" + depends on FS_ENCRYPTION && BLK_INLINE_ENCRYPTION + help + Enable fscrypt to use inline encryption hardware if available.
fs/crypto/Makefile +1 -0
@@ -11,3 +11,4 @@ fscrypto-y := crypto.o \ policy.o fscrypto-$(CONFIG_BLOCK) += bio.o +fscrypto-$(CONFIG_FS_ENCRYPTION_INLINE_CRYPT) += inline_crypt.o
fs/crypto/bio.c +50 -0
62 lines not on this page · raw message
fs/crypto/crypto.c +1 -1
@@ -95,7 +95,7 @@ int fscrypt_crypt_block(const struct inode *inode, fscrypt_direction_t rw,DECLARE_CRYPTO_WAIT (wait);struct scatterlist dst, src;struct fscrypt_info *ci = inode->i_crypt_info; -struct crypto_skcipher *tfm = ci->ci_ctfm; +struct crypto_skcipher *tfm = ci->ci_key.tfm;int res =0 ;if (WARN_ON_ONCE (len <=0 ))
fs/crypto/fname.c +2 -2
@@ -117,7 +117,7 @@ int fscrypt_fname_encrypt(const struct inode *inode, const struct qstr *iname,struct skcipher_request *req =NULL ;DECLARE_CRYPTO_WAIT (wait);const struct fscrypt_info *ci = inode->i_crypt_info; -struct crypto_skcipher *tfm = ci->ci_ctfm; +struct crypto_skcipher *tfm = ci->ci_key.tfm;union fscrypt_iv iv;struct scatterlist sg;int res;
@@ -170,7 +170,7 @@ static int fname_decrypt(const struct inode *inode,DECLARE_CRYPTO_WAIT (wait);struct scatterlist src_sg, dst_sg;const struct fscrypt_info *ci = inode->i_crypt_info; -struct crypto_skcipher *tfm = ci->ci_ctfm; +struct crypto_skcipher *tfm = ci->ci_key.tfm;union fscrypt_iv iv;int res;
fs/crypto/fscrypt_private.h +107 -13
177 lines not on this page · raw message
fs/crypto/inline_crypt.c new file +365 -0
365 lines not on this page · raw message
fs/crypto/keyring.c +2 -2
@@ -44,8 +44,8 @@ static void free_master_key(struct fscrypt_master_key *mk)wipe_master_key_secret (&mk->mk_secret);for (i =0 ; i <= __FSCRYPT_MODE_MAX; i++) { -crypto_free_skcipher (mk->mk_direct_tfms[i]); -crypto_free_skcipher (mk->mk_iv_ino_lblk_64_tfms[i]); +fscrypt_destroy_prepared_key (&mk->mk_direct_keys[i]); +fscrypt_destroy_prepared_key (&mk->mk_iv_ino_lblk_64_keys[i]); }key_put (mk->mk_users);
fs/crypto/keysetup.c +62 -30
177 lines not on this page · raw message
fs/crypto/keysetup_v1.c +7 -9
@@ -146,7 +146,7 @@ struct fscrypt_direct_key {struct hlist_node dk_node;refcount_t dk_refcount;const struct fscrypt_mode *dk_mode; -struct crypto_skcipher *dk_ctfm; +struct fscrypt_prepared_key dk_key;u8 dk_descriptor[FSCRYPT_KEY_DESCRIPTOR_SIZE ];u8 dk_raw[FSCRYPT_MAX_KEY_SIZE ]; };
@@ -154,7 +154,7 @@ struct fscrypt_direct_key {static void free_direct_key (struct fscrypt_direct_key *dk) {if (dk) { -crypto_free_skcipher (dk->dk_ctfm); +fscrypt_destroy_prepared_key (&dk->dk_key);kzfree (dk); } }
@@ -199,6 +199,8 @@ find_or_insert_direct_key(struct fscrypt_direct_key *to_insert,continue ;if (ci->ci_mode != dk->dk_mode)continue ; +if (!fscrypt_is_key_prepared (&dk->dk_key, ci)) +continue ;if (crypto_memneq (raw_key, dk->dk_raw, ci->ci_mode->keysize))continue ;/* using existing tfm with same (descriptor, mode, raw_key) */
@@ -231,13 +233,9 @@ fscrypt_get_direct_key(const struct fscrypt_info *ci, const u8 *raw_key)return ERR_PTR (-ENOMEM );refcount_set (&dk->dk_refcount,1 ); dk->dk_mode = ci->ci_mode; - dk->dk_ctfm =fscrypt_allocate_skcipher (ci->ci_mode, raw_key, - ci->ci_inode); -if (IS_ERR (dk->dk_ctfm)) { - err =PTR_ERR (dk->dk_ctfm); - dk->dk_ctfm =NULL ; + err =fscrypt_prepare_key (&dk->dk_key, raw_key, ci); +if (err) goto err_free_dk; - }memcpy (dk->dk_descriptor, ci->ci_policy.v1.master_key_descriptor,FSCRYPT_KEY_DESCRIPTOR_SIZE );memcpy (dk->dk_raw, raw_key, ci->ci_mode->keysize);
@@ -259,7 +257,7 @@ static int setup_v1_file_key_direct(struct fscrypt_info *ci,if (IS_ERR (dk))return PTR_ERR (dk); ci->ci_direct_key = dk; - ci->ci_ctfm = dk->dk_ctfm; + ci->ci_key = dk->dk_key;return 0 ; }
include/linux/fscrypt.h +57 -0
69 lines not on this page · raw message
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 11/12 f2fs: add inline encryption support
In reply to Satya TangiralaWire up f2fs to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:
- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
--- Documentation/filesystems/f2fs.rst | 7 ++- fs/f2fs/compress.c | 2 +- fs/f2fs/data.c | 68 +++++++++++++++++++++++++----- fs/f2fs/super.c | 32 ++++++++++++++ 4 files changed, 97 insertions(+), 12 deletions(-)
Documentation/filesystems/f2fs.rst +6 -1
@@ -254,7 +254,12 @@ compress_extension=%s Support adding specified extension, so that f2fs can enab on compression extension list and enable compression on these file by default rather than to enable it via ioctl. For other files, we can still enable compression via ioctl. -====================== ============================================================ +inlinecrypt + Encrypt/decrypt the contents of encrypted files using the + blk-crypto framework rather than filesystem-layer encryption. + This allows the use of inline encryption hardware. The on-disk + format is unaffected. For more details, see + Documentation/block/inline-encryption.rst. Debugfs Entries ===============
fs/f2fs/compress.c +1 -1
@@ -975,7 +975,7 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc, .submitted = false, .io_type = io_type, .io_wbc = wbc, - .encrypted = f2fs_encrypted_file(cc->inode), + .encrypted = fscrypt_inode_uses_fs_layer_crypto(cc->inode), };struct dnode_of_data dn;struct node_info ni;
fs/f2fs/data.c +58 -10
142 lines not on this page · raw message
fs/f2fs/super.c +32 -0
@@ -138,6 +138,7 @@ enum { Opt_alloc, Opt_fsync, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_checkpoint_disable, Opt_checkpoint_disable_cap, Opt_checkpoint_disable_cap_perc,
@@ -203,6 +204,7 @@ static match_table_t f2fs_tokens = { {Opt_alloc, "alloc_mode=%s"}, {Opt_fsync, "fsync_mode=%s"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_checkpoint_disable, "checkpoint=disable"}, {Opt_checkpoint_disable_cap, "checkpoint=disable:%u"}, {Opt_checkpoint_disable_cap_perc, "checkpoint=disable:%u%%"},
@@ -788,6 +790,13 @@ static int parse_options(struct super_block *sb, char *options)f2fs_info (sbi,"Test dummy encryption mode enabled" );#else f2fs_info (sbi,"Test dummy encryption mount option ignored" ); +#endif +break ; +case Opt_inlinecrypt: +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + sb->s_flags |=SB_INLINECRYPT ; +#else +f2fs_info (sbi,"inline encryption not supported" );#endif break ;case Opt_checkpoint_disable_cap_perc:
@@ -1583,6 +1592,8 @@ static void default_options(struct f2fs_sb_info *sbi)F2FS_OPTION (sbi).compress_ext_cnt =0 ;F2FS_OPTION (sbi).bggc_mode =BGGC_MODE_ON ; + sbi->sb->s_flags &= ~SB_INLINECRYPT ; +set_opt (sbi,INLINE_XATTR );set_opt (sbi,INLINE_DATA );set_opt (sbi,INLINE_DENTRY );
@@ -2427,6 +2438,25 @@ static void f2fs_get_ino_and_lblk_bits(struct super_block *sb, *lblk_bits_ret =8 *sizeof (block_t); } +static int f2fs_get_num_devices (struct super_block *sb) +{ +struct f2fs_sb_info *sbi =F2FS_SB (sb); + +if (f2fs_is_multi_device (sbi)) +return sbi->s_ndevs; +return 1 ; +} + +static void f2fs_get_devices (struct super_block *sb, +struct request_queue **devs) +{ +struct f2fs_sb_info *sbi =F2FS_SB (sb); +int i; + +for (i =0 ; i < sbi->s_ndevs; i++) + devs[i] =bdev_get_queue (FDEV (i).bdev); +} +static const struct fscrypt_operations f2fs_cryptops = { .key_prefix ="f2fs:" , .get_context = f2fs_get_context,
@@ -2436,6 +2466,8 @@ static const struct fscrypt_operations f2fs_cryptops = { .max_namelen = F2FS_NAME_LEN, .has_stable_inodes = f2fs_has_stable_inodes, .get_ino_and_lblk_bits = f2fs_get_ino_and_lblk_bits, + .get_num_devices = f2fs_get_num_devices, + .get_devices = f2fs_get_devices, };#endif
Signature · 1 line
2.26.0.110.g2183baf09c-goog
PATCH v10 12/12 ext4: add inline encryption support
In reply to Satya TangiralaFrom: Eric Biggers <[email protected]>
Wire up ext4 to support inline encryption via the helper functions which fs/crypto/ now provides. This includes:
- Adding a mount option 'inlinecrypt' which enables inline encryption on encrypted files where it can be used.
- Setting the bio_crypt_ctx on bios that will be submitted to an inline-encrypted file.
Note: submit_bh_wbc() in fs/buffer.c also needed to be patched for this part, since ext4 sometimes uses ll_rw_block() on file data.
- Not adding logically discontiguous data to bios that will be submitted to an inline-encrypted file.
- Not doing filesystem-layer crypto on inline-encrypted files.
--- Documentation/admin-guide/ext4.rst | 6 ++++++ fs/buffer.c | 7 ++++--- fs/ext4/inode.c | 4 ++-- fs/ext4/page-io.c | 6 ++++-- fs/ext4/readpage.c | 11 ++++++++--- fs/ext4/super.c | 9 +++++++++ 6 files changed, 33 insertions(+), 10 deletions(-)
Documentation/admin-guide/ext4.rst +6 -0
@@ -395,6 +395,12 @@ When mounting an ext4 filesystem, the following option are accepted: Documentation/filesystems/dax.txt. Note that this option is incompatible with data=journal. + inlinecrypt + Encrypt/decrypt the contents of encrypted files using the blk-crypto + framework rather than filesystem-layer encryption. This allows the use + of inline encryption hardware. The on-disk format is unaffected. For + more details, see Documentation/block/inline-encryption.rst. + Data Mode ========= There are 3 different data modes:
fs/buffer.c +4 -3
@@ -328,9 +328,8 @@ static void decrypt_bh(struct work_struct *work)static void end_buffer_async_read_io (struct buffer_head *bh,int uptodate) {/* Decrypt if needed */ -if (uptodate &&IS_ENABLED (CONFIG_FS_ENCRYPTION ) && -IS_ENCRYPTED (bh->b_page->mapping->host) && -S_ISREG (bh->b_page->mapping->host->i_mode)) { +if (uptodate && +fscrypt_inode_uses_fs_layer_crypto (bh->b_page->mapping->host)) {struct decrypt_bh_ctx *ctx =kmalloc (sizeof (*ctx),GFP_ATOMIC );if (ctx) {
@@ -3036,6 +3035,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,*/ bio =bio_alloc (GFP_NOIO ,1 ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); + bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_write_hint = write_hint;
fs/ext4/inode.c +2 -2
@@ -1088,7 +1088,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len, }if (unlikely (err)) {page_zero_new_buffers (page, from, to); - }else if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) { + }else if (fscrypt_inode_uses_fs_layer_crypto (inode)) {for (i =0 ; i < nr_wait; i++) {int err2;
@@ -3738,7 +3738,7 @@ static int __ext4_block_zero_page_range(handle_t *handle,/* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate (bh)) goto unlock; -if (S_ISREG (inode->i_mode) &&IS_ENCRYPTED (inode)) { +if (fscrypt_inode_uses_fs_layer_crypto (inode)) {/* We expect the key to be set. */ BUG_ON (!fscrypt_has_encryption_key (inode)); err = fscrypt_decrypt_pagecache_blocks(page, blocksize,
fs/ext4/page-io.c +4 -2
@@ -402,6 +402,7 @@ static void io_submit_init_bio(struct ext4_io_submit *io,* __GFP_DIRECT_RECLAIM is set, see comments for bio_alloc_bioset(). */ bio =bio_alloc (GFP_NOIO ,BIO_MAX_PAGES ); +fscrypt_set_bio_crypt_ctx_bh (bio, bh,GFP_NOIO ); bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >>9 );bio_set_dev (bio, bh->b_bdev); bio->bi_end_io = ext4_end_bio;
@@ -418,7 +419,8 @@ static void io_submit_add_bh(struct ext4_io_submit *io, {int ret; -if (io->io_bio && bh->b_blocknr != io->io_next_block) { +if (io->io_bio && (bh->b_blocknr != io->io_next_block || + !fscrypt_mergeable_bio_bh (io->io_bio, bh))) { submit_and_retry:ext4_io_submit (io); }
@@ -506,7 +508,7 @@ int ext4_bio_write_page(struct ext4_io_submit *io,* (e.g. holes) to be unnecessarily encrypted, but this is rare and * can't happen in the common case of blocksize == PAGE_SIZE. */ -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode) && nr_to_submit) { +if (fscrypt_inode_uses_fs_layer_crypto (inode) && nr_to_submit) {gfp_t gfp_flags =GFP_NOFS ;unsigned int enc_bytes =round_up (len,i_blocksize (inode));
fs/ext4/readpage.c +8 -3
@@ -195,7 +195,7 @@ static void ext4_set_bio_post_read_ctx(struct bio *bio, {unsigned int post_read_steps =0 ; -if (IS_ENCRYPTED (inode) &&S_ISREG (inode->i_mode)) +if (fscrypt_inode_uses_fs_layer_crypto (inode)) post_read_steps |=1 <<STEP_DECRYPT ;if (ext4_need_verity (inode, first_idx))
@@ -232,6 +232,7 @@ int ext4_mpage_readpages(struct address_space *mapping,const unsigned blkbits = inode->i_blkbits;const unsigned blocks_per_page =PAGE_SIZE >> blkbits;const unsigned blocksize =1 << blkbits; +sector_t next_block;sector_t block_in_file;sector_t last_block;sector_t last_block_in_file;
@@ -264,7 +265,8 @@ int ext4_mpage_readpages(struct address_space *mapping,if (page_has_buffers (page)) goto confused; - block_in_file = (sector_t )page->index << (PAGE_SHIFT - blkbits); + block_in_file = next_block = + (sector_t )page->index << (PAGE_SHIFT - blkbits); last_block = block_in_file + nr_pages * blocks_per_page; last_block_in_file = (ext4_readpage_limit (inode) + blocksize -1 ) >> blkbits;
@@ -364,7 +366,8 @@ int ext4_mpage_readpages(struct address_space *mapping,* This page will go to BIO. Do we need to send this * BIO off first? */ -if (bio && (last_block_in_bio != blocks[0 ] -1 )) { +if (bio && (last_block_in_bio != blocks[0 ] -1 || + !fscrypt_mergeable_bio (bio, inode, next_block))) { submit_and_realloc:submit_bio (bio); bio =NULL ;
@@ -376,6 +379,8 @@ int ext4_mpage_readpages(struct address_space *mapping,*/ bio =bio_alloc (GFP_KERNEL ,min_t (int, nr_pages,BIO_MAX_PAGES )); +fscrypt_set_bio_crypt_ctx (bio, inode, next_block, +GFP_KERNEL );ext4_set_bio_post_read_ctx (bio, inode, page->index);bio_set_dev (bio, bdev); bio->bi_iter.bi_sector = blocks[0 ] << (blkbits -9 );
fs/ext4/super.c +9 -0
@@ -1511,6 +1511,7 @@ enum { Opt_journal_path, Opt_journal_checksum, Opt_journal_async_commit, Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, Opt_data_err_abort, Opt_data_err_ignore, Opt_test_dummy_encryption, + Opt_inlinecrypt, Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, Opt_noquota, Opt_barrier, Opt_nobarrier, Opt_err,
@@ -1608,6 +1609,7 @@ static const match_table_t tokens = { {Opt_noinit_itable, "noinit_itable"}, {Opt_max_dir_size_kb, "max_dir_size_kb=%u"}, {Opt_test_dummy_encryption, "test_dummy_encryption"}, + {Opt_inlinecrypt, "inlinecrypt"}, {Opt_nombcache, "nombcache"}, {Opt_nombcache, "no_mbcache"}, /* for backward compatibility */ {Opt_removed, "check=none"}, /* mount option from ext2/3 */
@@ -1895,6 +1897,13 @@ static int handle_mount_opt(struct super_block *sb, char *opt, int token,case Opt_nolazytime: sb->s_flags &= ~SB_LAZYTIME ;return 1 ; +case Opt_inlinecrypt: +#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT + sb->s_flags |=SB_INLINECRYPT ; +#else +ext4_msg (sb,KERN_ERR ,"inline encryption not supported" ); +#endif +return 1 ; }for (m = ext4_mount_opts; m->token != Opt_err; m++)
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Eric Biggers
maintainerQuoted 3 lines · Satya Tangirala
This patch series adds support for Inline Encryption to the block layer, UFS, fscrypt, f2fs and ext4.
This patch series can also be retrieved from
Repo: https://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git
Tag: inline-encryption-v10
- Eric