Library APIs for AES encryption modes

This series can also be retrieved from:

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git aes-modes-v2

This series adds library APIs for almost all AES encryption modes that are used in the kernel, both authenticated and unauthenticated: AES-ECB, AES-CBC, AES-CBC-CTS, AES-CTR, AES-XCTR, AES-XTS, AES-GCM, and AES-CCM.

Patches 2-7 add the library APIs, and patches 8-13 wire them up to the traditional crypto API (with just priority 110 for now). I'm planning to take these through libcrypto-next for 7.3.

Proof-of-concept patches that convert users of the traditional crypto API to use the new library APIs can be found in patches 14-33 of the v1 series
(https://lore.kernel.org/linux-crypto/[email protected]/).
Those additional patches are intended for 7.4 or later.

Apologies for the large patch series, but it ended up being easiest to handle all the AES modes at once rather than try to do them one by one. This ensures a consistent design for them, which is really important. Additionally, the arch-optimized AES code tends to intermingle different modes, and many depend on each other anyway.

But as a result, to keep this series manageable it doesn't include the usual KUnit tests or the integration of the architecture-optimized code. Those will be sent separately in follow-up series.

Changed in v2:
- Reduced the series to just the patches targeting 7.3.
- Made lots of documentation and comment improvements.
- Made CRYPTO_AES select the library kconfig options correctly.
- Adjusted the parameter order of some functions for better consistency:
aes_ccm_init(), aes_{ccm,gcm}_encrypt(), aes_{ccm,gcm}_decrypt().
- Made the CCM implementation support ad_len up to 2^64 - 1, as per the spec, instead of artifically capping it at 2^32.
- Made the CCM implementation check for incorrect call order, making it consistent with the GCM implementation.
- Optimized how aes_ccm_init() validates data_len.
- Made the GCM implementation check for the maximum ad_len.
- Made aes_{gcm,ccm}_decrypt_final() return an error code if either of the lengths was incorrect, instead of just warning.
- Shared more code between encryption and decryption for GCM and CCM in crypto/aes.c.
- Removed unnecessary casts in inc_be128_ctr().
- Avoided memset() with dst == NULL.
- Lots of other small cleanups.

Eric Biggers (13):
crypto: xts - Split out __xts_verify_key() helper lib/crypto: aes: Add ECB support
lib/crypto: aes: Add CBC and CBC-CTS support lib/crypto: aes: Add CTR and XCTR support lib/crypto: aes: Add XTS support
lib/crypto: aes: Add GCM support
lib/crypto: aes: Add CCM support
crypto: aes - Add ECB support using library
crypto: aes - Add CBC and CBC-CTS support using library
crypto: aes - Add CTR and XCTR support using library
crypto: aes - Add XTS support using library
crypto: aes - Add GCM support using library
crypto: aes - Add CCM support using library

 .../crypto/libcrypto-auth-encryption.rst      |   20 +
 .../crypto/libcrypto-unauth-encryption.rst    |   49 +
 Documentation/crypto/libcrypto.rst            |    2 +
 crypto/Kconfig                                |   11 +
 crypto/aes.c                                  |  892 ++++++++++++-
 include/crypto/aes-cbc.h                      |   77 ++
 include/crypto/aes-ccm.h                      |  266 ++++
 include/crypto/aes-ctr.h                      |   65 +
 include/crypto/aes-ecb.h                      |   49 +
 include/crypto/aes-gcm.h                      |  260 ++++
 include/crypto/aes-xts.h                      |   94 ++
 include/crypto/gcm.h                          |    4 +-
 include/crypto/xts.h                          |   18 +-
 lib/crypto/Kconfig                            |   40 +
 lib/crypto/aes.c                              | 1163 +++++++++++++++++
 lib/crypto/tests/Kconfig                      |    6 +
 16 files changed, 3009 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/crypto/libcrypto-auth-encryption.rst
 create mode 100644 Documentation/crypto/libcrypto-unauth-encryption.rst
 create mode 100644 include/crypto/aes-cbc.h
 create mode 100644 include/crypto/aes-ccm.h
 create mode 100644 include/crypto/aes-ctr.h
 create mode 100644 include/crypto/aes-ecb.h
 create mode 100644 include/crypto/aes-gcm.h
 create mode 100644 include/crypto/aes-xts.h

base-commit: e073f1238ecaea366f53e98724c40b31856da56a

2.55.0

Make the AES-XTS key verification code callable by the crypto library by splitting out a helper function that doesn't use crypto_skcipher.

---
 include/crypto/xts.h | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

index 15b16c4853d8..16aef89f021f 100644
@@ -7,9 +7,9 @@
 #include <linux/fips.h>
 
 #define XTS_BLOCK_SIZE 16
+#define XTS_FORBID_WEAK_KEYS (1 << 0)
 
-static inline int xts_verify_key(struct crypto_skcipher *tfm,
-				 const u8 *key, unsigned int keylen)
+static inline int __xts_verify_key(const u8 *key, size_t keylen, int flags)
 {
 	/*
 	 * key consists of keys of equal size concatenated, therefore

@@ -29,12 +29,22 @@ static inline int xts_verify_key(struct crypto_skcipher *tfm,
 	 * Ensure that the AES and tweak key are not identical when
 	 * in FIPS mode or the FORBID_WEAK_KEYS flag is set.
 	 */
-	if ((fips_enabled || (crypto_skcipher_get_flags(tfm) &
-			      CRYPTO_TFM_REQ_FORBID_WEAK_KEYS)) &&
+	if ((fips_enabled || (flags & XTS_FORBID_WEAK_KEYS)) &&
 	    !crypto_memneq(key, key + (keylen / 2), keylen / 2))
 		return -EINVAL;
 
 	return 0;
 }
 
+static inline int xts_verify_key(struct crypto_skcipher *tfm, const u8 *key,
+				 unsigned int keylen)
+{
+	int flags = (crypto_skcipher_get_flags(tfm) &
+		     CRYPTO_TFM_REQ_FORBID_WEAK_KEYS) ?
+			    XTS_FORBID_WEAK_KEYS :
+			    0;
+
+	return __xts_verify_key(key, keylen, flags);
+}
+
 #endif  /* _CRYPTO_XTS_H */

2.55.0

Add support for AES-ECB to the crypto library.

This will be used to provide a streamlined implementation of the "ecb(aes)" crypto_skcipher algorithm. fs/crypto/keysetup_v1.c will also use aes_ecb_encrypt() directly.

As usual, the architecture-optimized AES-ECB code will be migrated into the library as well (using the hooks provided in this commit), eliminating lots of repetitive boilerplate code.

ECB is obsolete of course, but we need this for parity with the traditional API and to support some odd users of ECB in the kernel.

Initial test coverage is provided by the crypto_skcipher support added in a later commit. I'm planning a KUnit test suite as well.

Create a documentation file libcrypto-unauth-encryption.rst to hold the documentation for this and other unauthenticated encryption modes.

---
 .../crypto/libcrypto-unauth-encryption.rst    | 28 ++++++++++
 Documentation/crypto/libcrypto.rst            |  1 +
 include/crypto/aes-ecb.h                      | 49 ++++++++++++++++
 lib/crypto/Kconfig                            |  6 ++
 lib/crypto/aes.c                              | 56 +++++++++++++++++++
 lib/crypto/tests/Kconfig                      |  1 +
 6 files changed, 141 insertions(+)
 create mode 100644 Documentation/crypto/libcrypto-unauth-encryption.rst
 create mode 100644 include/crypto/aes-ecb.h

new file mode 100644
index 000000000000..6a3397a8adae
@@ -0,0 +1,28 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Unauthenticated encryption
+==========================
+
+These APIs provide support for unauthenticated encryption and decryption,
+including bare stream ciphers and other length-preserving algorithms such as
+block ciphers in XTS mode.  The legitimate use cases for these algorithms are:
+
+- Support for legacy protocols that really should have chosen an authenticated
+  mode (or even another primitive entirely) but didn't.
+
+- Internal components of authenticated modes.  For example, AES-CTR is used by
+  AES-GCM and AES-CCM internally.
+
+- Storage encryption that cannot accommodate ciphertext expansion.  Usually
+  AES-XTS is used for this.
+
+- Stream ciphers for key derivation and random number generation.
+
+Besides the above, these shouldn't be used.
+
+AES-ECB
+-------
+
+This API provides support for AES in the ECB mode of operation.
+
+.. kernel-doc:: include/crypto/aes-ecb.h

index 0733e603d229..33312c3ffad4 100644
@@ -162,5 +162,6 @@ API documentation
    libcrypto-blockcipher
    libcrypto-hash
    libcrypto-signature
+   libcrypto-unauth-encryption
    libcrypto-utils
    sha3

index 83d4c95e079e..26514c181a7f 100644
@@ -35,6 +35,12 @@ config CRYPTO_LIB_AES_CBC_MACS
 	  this if your module uses any of the functions from
 	  <crypto/aes-cbc-macs.h>.
 
+config CRYPTO_LIB_AES_ECB
+	tristate
+	select CRYPTO_LIB_AES
+	help
+	  The AES-ECB library functions.
+
 config CRYPTO_LIB_AESGCM
 	tristate
 	select CRYPTO_LIB_AES

index 9409c1a935c3..a57e87dbb1b1 100644
@@ -145,6 +145,7 @@ config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
 	tristate "Enable all crypto library code for KUnit tests"
 	depends on KUNIT
 	select CRYPTO_LIB_AES_CBC_MACS
+	select CRYPTO_LIB_AES_ECB
 	select CRYPTO_LIB_BLAKE2B
 	select CRYPTO_LIB_CHACHA20POLY1305
 	select CRYPTO_LIB_CURVE25519

2.55.0

Add support for AES-CBC and AES-CBC-CTS to the crypto library.

These will be used to provide streamlined implementations of the "cbc(aes)" and "cts(cbc(aes))" crypto_skcipher algorithms. Most users of these crypto_skcipher algorithms will also be able to switch to the library, which as usual will be simpler and faster, e.g.:

- block/blk-crypto-fallback.c (for AES-128-CBC-ESSIV)
- fs/crypto/crypto.c (for AES-128-CBC-ESSIV)
- fs/crypto/fname.c (for AES-256-CTS and AES-128-CBC)
- kernel/bpf/crypto.c
- net/ceph/crypto.c
- security/keys/encrypted-keys/encrypted.c

As usual, the architecture-optimized AES-CBC and AES-CBC-CTS code will be migrated into the library as well (using the hooks provided in this commit), eliminating lots of repetitive boilerplate code.

Initial test coverage is provided by the crypto_skcipher support added in a later commit. I'm planning a KUnit test suite as well.

---
 .../crypto/libcrypto-unauth-encryption.rst    |   7 +
 include/crypto/aes-cbc.h                      |  77 ++++++++
 lib/crypto/Kconfig                            |   6 +
 lib/crypto/aes.c                              | 187 ++++++++++++++++++
 lib/crypto/tests/Kconfig                      |   1 +
 5 files changed, 278 insertions(+)
 create mode 100644 include/crypto/aes-cbc.h

index 6a3397a8adae..82bdb0d1b93e 100644
@@ -20,6 +20,13 @@ block ciphers in XTS mode.  The legitimate use cases for these algorithms are:
 
 Besides the above, these shouldn't be used.
 
+AES-CBC and AES-CBC-CTS
+-----------------------
+
+This API provides support for AES in the CBC and CBC-CTS modes of operation.
+
+.. kernel-doc:: include/crypto/aes-cbc.h
+
 AES-ECB
 -------
 

index 26514c181a7f..c64cc3e12b57 100644
@@ -27,6 +27,12 @@ config CRYPTO_LIB_AESCFB
 	select CRYPTO_LIB_AES
 	select CRYPTO_LIB_UTILS
 
+config CRYPTO_LIB_AES_CBC
+	tristate
+	select CRYPTO_LIB_AES
+	help
+	  The AES-CBC and AES-CBC-CTS library functions.
+
 config CRYPTO_LIB_AES_CBC_MACS
 	tristate
 	select CRYPTO_LIB_AES

index a57e87dbb1b1..e78086f3c954 100644
@@ -144,6 +144,7 @@ config CRYPTO_LIB_SM3_KUNIT_TEST
 config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
 	tristate "Enable all crypto library code for KUnit tests"
 	depends on KUNIT
+	select CRYPTO_LIB_AES_CBC
 	select CRYPTO_LIB_AES_CBC_MACS
 	select CRYPTO_LIB_AES_ECB
 	select CRYPTO_LIB_BLAKE2B

2.55.0

Add support for AES-CTR and AES-XCTR to the crypto library.

These will be used to provide streamlined implementations of the "ctr(aes)" and "xctr(aes)" crypto_skcipher algorithms. Most users of "ctr(aes)" will also be able to switch to the library, which as usual will be simpler and faster, e.g.:

- net/mac80211/fils_aead.c
- net/mac802154/llsec.c

As usual, the architecture-optimized AES-CTR and AES-XCTR code will be migrated into the library as well (using the hooks provided in this commit), eliminating lots of repetitive boilerplate code.

This is also a prerequisite for supporting AES-GCM, AES-CCM, and AES-HCTR2 in the crypto library.

Initial test coverage is provided by the crypto_skcipher support added in a later commit. I'm planning a KUnit test suite as well.

---
 .../crypto/libcrypto-unauth-encryption.rst    |  7 ++
 include/crypto/aes-ctr.h                      | 65 +++++++++++++
 lib/crypto/Kconfig                            |  6 ++
 lib/crypto/aes.c                              | 96 +++++++++++++++++++
 lib/crypto/tests/Kconfig                      |  1 +
 5 files changed, 175 insertions(+)
 create mode 100644 include/crypto/aes-ctr.h

index 82bdb0d1b93e..4d3555ee81b7 100644
@@ -27,6 +27,13 @@ This API provides support for AES in the CBC and CBC-CTS modes of operation.
 
 .. kernel-doc:: include/crypto/aes-cbc.h
 
+AES-CTR and AES-XCTR
+--------------------
+
+This API provides support for AES in the CTR and XCTR modes of operation.
+
+.. kernel-doc:: include/crypto/aes-ctr.h
+
 AES-ECB
 -------
 

index c64cc3e12b57..67a44e82309d 100644
@@ -41,6 +41,12 @@ config CRYPTO_LIB_AES_CBC_MACS
 	  this if your module uses any of the functions from
 	  <crypto/aes-cbc-macs.h>.
 
+config CRYPTO_LIB_AES_CTR
+	tristate
+	select CRYPTO_LIB_AES
+	help
+	  The AES-CTR and AES-XCTR library functions.
+
 config CRYPTO_LIB_AES_ECB
 	tristate
 	select CRYPTO_LIB_AES

index e78086f3c954..9284d0134d77 100644
@@ -146,6 +146,7 @@ config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
 	depends on KUNIT
 	select CRYPTO_LIB_AES_CBC
 	select CRYPTO_LIB_AES_CBC_MACS
+	select CRYPTO_LIB_AES_CTR
 	select CRYPTO_LIB_AES_ECB
 	select CRYPTO_LIB_BLAKE2B
 	select CRYPTO_LIB_CHACHA20POLY1305

2.55.0

Add support for AES-XTS to the crypto library.

This will be used to provide a streamlined implementation of the "xts(aes)" crypto_skcipher algorithm. I'm also planning to use this directly in fscrypt and blk-crypto-fallback.

As usual, the architecture-optimized AES-XTS code will be migrated into the library as well (using the hooks provided in this commit), eliminating lots of repetitive boilerplate code. Compared to direct implementation of "xts(aes)", I've also eliminated the requirement for architectures to implement ciphertext stealing, as the library just handles it portably instead. That will simplify things considerably.

Initial test coverage is provided by the crypto_skcipher support added in a later commit. I'm planning a KUnit test suite as well.

---
 .../crypto/libcrypto-unauth-encryption.rst    |   7 +
 include/crypto/aes-xts.h                      |  94 +++++++
 lib/crypto/Kconfig                            |   6 +
 lib/crypto/aes.c                              | 231 ++++++++++++++++++
 lib/crypto/tests/Kconfig                      |   1 +
 5 files changed, 339 insertions(+)
 create mode 100644 include/crypto/aes-xts.h

index 4d3555ee81b7..b4639b8927c6 100644
@@ -40,3 +40,10 @@ AES-ECB
 This API provides support for AES in the ECB mode of operation.
 
 .. kernel-doc:: include/crypto/aes-ecb.h
+
+AES-XTS
+-------
+
+This API provides support for AES in the XTS mode of operation.
+
+.. kernel-doc:: include/crypto/aes-xts.h

index 67a44e82309d..6ec47cc328c8 100644
@@ -53,6 +53,12 @@ config CRYPTO_LIB_AES_ECB
 	help
 	  The AES-ECB library functions.
 
+config CRYPTO_LIB_AES_XTS
+	tristate
+	select CRYPTO_LIB_AES
+	help
+	  The AES-XTS library functions.
+
 config CRYPTO_LIB_AESGCM
 	tristate
 	select CRYPTO_LIB_AES

index 9284d0134d77..b559e7c79e76 100644
@@ -148,6 +148,7 @@ config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
 	select CRYPTO_LIB_AES_CBC_MACS
 	select CRYPTO_LIB_AES_CTR
 	select CRYPTO_LIB_AES_ECB
+	select CRYPTO_LIB_AES_XTS
 	select CRYPTO_LIB_BLAKE2B
 	select CRYPTO_LIB_CHACHA20POLY1305
 	select CRYPTO_LIB_CURVE25519

2.55.0

Add support for AES-GCM to the crypto library.

This will be used to provide streamlined implementations of the "gcm(aes)" and "rfc4106(gcm(aes))" crypto_aead algorithms. Most users of these will also be able to switch to the library, which as usual will be faster and simpler, e.g.:

- drivers/net/macsec.c
- fs/smb/client/
- fs/smb/server/
- net/ceph/messenger_v2.c
- net/mac80211/ (for both GMAC and GCMP)
- net/tipc/crypto.c
- security/keys/trusted-keys/trusted_dcp.c

(I've already written proof-of-concept patches for all the above, and they helped inform the API design.)

As usual, the architecture-optimized AES-GCM code will be migrated into the library as well (using the hooks provided in this commit as well as the GHASH ones), eliminating lots of repetitive boilerplate code.

Incremental en/decryption is supported. Incremental operation is a bit controversial in AEAD APIs because users have to be careful not to consume any decrypted data that hasn't been authenticated yet. But I do think it's the right choice here. It's not fundamentally different from the existing incremental MAC APIs, and it's the only approach that's general enough to work well for all users in the kernel:

- An array of virtually-addressed buffers (like that used by BoringSSL's EVP_AEAD_CTX_sealv() and EVP_AEAD_CTX_openv()) doesn't work in the kernel in general, since in some cases the data for a single AES-GCM message is contained in a large number of highmem pages that each need to be mapped into memory individually. That can be done efficiently only by using CPU-local mappings, but there is a limited number of those.

Ceph messenger v2 is a great example, as it can send or receive up to 32 MiB in a single AES-GCM message. And it needs the en/decrypted data to go into a (potentially large) number of bvecs provided by a custom iterator, as well as into four virtually-addressed buffers, two of which can be large buffers in the vmalloc region.

Even just allocating an array big enough to store all the pointers can be problematic in the kernel. There are cases in which decryption runs in GFP_NOIO context or even in softirq context, where memory allocations are not as reliable as they normally are.

- Meanwhile, 'struct scatterlist' (the choice of crypto_aead) has turned out to be really inconvenient for anyone who *does* just have virtually-addressed buffers. This is especially true if they can be in the vmalloc region, including the stack, as in that case the conversion to a scatterlist has to be done page-by-page.

And even for users who have all of their data in bare 'struct page', none of them actually use 'struct scatterlist' as their native data structure anyway. They actually use skbs, bvecs, or other formats.

- iov_iter is attractive, but ultimately not general enough either (considering the Ceph case for example), but also too general in some ways (like having support for userspace addresses). Additional iter types like ITER_SKB would help a bit, but bloating iov_iter with more types would reduce performance elsewhere in the kernel.

Initial test coverage is provided by the crypto_aead support added in a later commit. I'm planning a KUnit test suite as well.

---
 .../crypto/libcrypto-auth-encryption.rst      |  13 +
 Documentation/crypto/libcrypto.rst            |   1 +
 include/crypto/aes-gcm.h                      | 260 ++++++++++++++++
 include/crypto/gcm.h                          |   4 +-
 lib/crypto/Kconfig                            |   8 +
 lib/crypto/aes.c                              | 280 ++++++++++++++++++
 lib/crypto/tests/Kconfig                      |   1 +
 7 files changed, 565 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/crypto/libcrypto-auth-encryption.rst
 create mode 100644 include/crypto/aes-gcm.h

new file mode 100644
index 000000000000..06c94796b5f4
@@ -0,0 +1,13 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Authenticated encryption
+========================
+
+These APIs provide support for authenticated encryption and decryption.
+
+AES-GCM
+-------
+
+This API provides support for AES in the GCM mode of operation.
+
+.. kernel-doc:: include/crypto/aes-gcm.h

index 33312c3ffad4..e911e0521597 100644
@@ -159,6 +159,7 @@ API documentation
 .. toctree::
    :maxdepth: 2
 
+   libcrypto-auth-encryption
    libcrypto-blockcipher
    libcrypto-hash
    libcrypto-signature

index 1d5f39ff1dc4..7fd7892ad818 100644
@@ -13,7 +13,7 @@
 /*
  * validate authentication tag for GCM
  */
-static inline int crypto_gcm_check_authsize(unsigned int authsize)
+static inline int crypto_gcm_check_authsize(size_t authsize)
 {
 	switch (authsize) {
 	case 4:

@@ -34,7 +34,7 @@ static inline int crypto_gcm_check_authsize(unsigned int authsize)
 /*
  * validate authentication tag for RFC4106
  */
-static inline int crypto_rfc4106_check_authsize(unsigned int authsize)
+static inline int crypto_rfc4106_check_authsize(size_t authsize)
 {
 	switch (authsize) {
 	case 8:

index 6ec47cc328c8..b48b0b2299a8 100644
@@ -53,6 +53,14 @@ config CRYPTO_LIB_AES_ECB
 	help
 	  The AES-ECB library functions.
 
+config CRYPTO_LIB_AES_GCM
+	tristate
+	select CRYPTO_LIB_AES
+	select CRYPTO_LIB_AES_CTR
+	select CRYPTO_LIB_GF128HASH
+	help
+	  The AES-GCM library functions.
+
 config CRYPTO_LIB_AES_XTS
 	tristate
 	select CRYPTO_LIB_AES

index b559e7c79e76..51183ffabbef 100644
@@ -148,6 +148,7 @@ config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
 	select CRYPTO_LIB_AES_CBC_MACS
 	select CRYPTO_LIB_AES_CTR
 	select CRYPTO_LIB_AES_ECB
+	select CRYPTO_LIB_AES_GCM
 	select CRYPTO_LIB_AES_XTS
 	select CRYPTO_LIB_BLAKE2B
 	select CRYPTO_LIB_CHACHA20POLY1305

2.55.0

Sorry for asking ignorant questions, but which later commit is this? I couldn't spot it :-/

Anyway, the previous AES-GCM code in lib/crypto/aesgcm.c featured some self-tests in libaesgcm_init() ... would it maybe make sense to add those here, too?

Thomas

It's "crypto: aes - Add GCM support using library". That adds a "gcm(aes)" crypto_aead algorithm that is implemented on top of these new AES-GCM library functions. That makes it be tested by the self-tests in crypto/testmgr.c when CONFIG_CRYPTO_SELFTESTS_FULL=y.

The KUnit test suite I haven't sent out, but I'm going to try to get out in the next week or two. I've just had a lot going on.

The crypto library uses KUnit tests, instead of ad-hoc tests run in the initcalls (except for the minimal tests required by FIPS 140-3). See
https://docs.kernel.org/crypto/libcrypto.html#testing

We could migrate the old libaesgcm_init() tests directly into a KUnit test suite. But I'd like to write something more complete that tests the incremental API and various other aspects of the API too.

- Eric

Add support for AES-CCM to the crypto library.

This will be used to provide a streamlined implementation of the "ccm(aes)" crypto_aead algorithm. Most users of "ccm(aes)" will also be able to switch to the library, which as usual will be faster and simpler, e.g.:

- fs/smb/client/
- fs/smb/server/
- net/mac80211/
- net/mac802154/

(I've already written proof-of-concept patches for all the above, and they helped inform the API design.)

As in the AES-GCM API, incremental operation is supported. It has to be used carefully, especially when decrypting, but it makes the API general enough to work well for all users.

The AES-CCM library code calls aes_cbcmac_blocks() directly, bypassing the higher-level aes_cbcmac_init(), aes_cbcmac_update(), and aes_cbcmac_final(). The latter set of functions is useful only for AES-CCM, so they don't make sense to keep around and will be removed once the "ccm(aes)" crypto_aead starts using the AES-CCM library.

Initial test coverage is provided by the crypto_aead support added in a later commit. I'm planning a KUnit test suite as well.

---
 .../crypto/libcrypto-auth-encryption.rst      |   7 +
 include/crypto/aes-ccm.h                      | 266 +++++++++++++++
 lib/crypto/Kconfig                            |   8 +
 lib/crypto/aes.c                              | 313 ++++++++++++++++++
 lib/crypto/tests/Kconfig                      |   1 +
 5 files changed, 595 insertions(+)
 create mode 100644 include/crypto/aes-ccm.h

index 06c94796b5f4..1e527685a42f 100644
@@ -5,6 +5,13 @@ Authenticated encryption
 
 These APIs provide support for authenticated encryption and decryption.
 
+AES-CCM
+-------
+
+This API provides support for AES in the CCM mode of operation.
+
+.. kernel-doc:: include/crypto/aes-ccm.h
+
 AES-GCM
 -------
 

index b48b0b2299a8..65a478f69715 100644
@@ -41,6 +41,14 @@ config CRYPTO_LIB_AES_CBC_MACS
 	  this if your module uses any of the functions from
 	  <crypto/aes-cbc-macs.h>.
 
+config CRYPTO_LIB_AES_CCM
+	tristate
+	select CRYPTO_LIB_AES
+	select CRYPTO_LIB_AES_CBC_MACS
+	select CRYPTO_LIB_AES_CTR
+	help
+	  The AES-CCM library functions.
+
 config CRYPTO_LIB_AES_CTR
 	tristate
 	select CRYPTO_LIB_AES

index 51183ffabbef..bc084dde424f 100644
@@ -146,6 +146,7 @@ config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
 	depends on KUNIT
 	select CRYPTO_LIB_AES_CBC
 	select CRYPTO_LIB_AES_CBC_MACS
+	select CRYPTO_LIB_AES_CCM
 	select CRYPTO_LIB_AES_CTR
 	select CRYPTO_LIB_AES_ECB
 	select CRYPTO_LIB_AES_GCM

2.55.0

Implement the "ecb(aes)" crypto_skcipher algorithm using the corresponding library functions.

Among other benefits, this allows the architecture-optimized AES-ECB code to be migrated into the library while still leaving it accessible via crypto_skcipher, eliminating lots of boilerplate code.

For now the cra_priority is set to just 110, since the architecture-optimized implementations of this algorithm haven't yet been migrated into the library. It will be boosted once that happens.

---
 crypto/Kconfig |   5 ++
 crypto/aes.c   | 164 ++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 168 insertions(+), 1 deletion(-)

index b61401bd3ef6..1888ae9d3fa3 100644
@@ -359,7 +359,12 @@ config CRYPTO_AES
 	select CRYPTO_ALGAPI
 	select CRYPTO_LIB_AES
 	select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
+	select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
 	select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
+	# CRYPTO_SKCIPHER should be selected only if a mode that needs it is
+	# enabled, but that doesn't work due to a recursive dependency caused by
+	# CRYPTO_SKCIPHER selecting CRYPTO_ECB.  So just always select it.
+	select CRYPTO_SKCIPHER
 	help
 	  AES cipher algorithms (Rijndael)(FIPS-197, ISO/IEC 18033-3)
 

2.55.0

Implement the "cbc(aes)" and "cts(cbc(aes))" crypto_skcipher algorithms using the corresponding library functions.

Among other benefits, this allows the architecture-optimized AES-CBC and AES-CBC-CTS code to be migrated into the library while still leaving it accessible via crypto_skcipher, eliminating lots of boilerplate code.

For now the cra_priority is set to just 110, since the architecture-optimized implementations of these algorithms haven't yet been migrated into the library. It will be boosted once that happens.

---
 crypto/Kconfig |   1 +
 crypto/aes.c   | 169 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 170 insertions(+)

index 1888ae9d3fa3..f413cfc9a3e2 100644
@@ -358,6 +358,7 @@ config CRYPTO_AES
 	tristate "AES (Advanced Encryption Standard)"
 	select CRYPTO_ALGAPI
 	select CRYPTO_LIB_AES
+	select CRYPTO_LIB_AES_CBC if CRYPTO_CBC != n || CRYPTO_CTS != n
 	select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
 	select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n

2.55.0

Implement the "ctr(aes)" and "xctr(aes)" crypto_skcipher algorithms using the corresponding library functions.

Among other benefits, this allows the architecture-optimized AES-CTR and AES-XCTR code to be migrated into the library while still leaving it accessible via crypto_skcipher, eliminating lots of boilerplate code.

For now the cra_priority is set to just 110, since the architecture-optimized implementations of these algorithms haven't yet been migrated into the library. It will be boosted once that happens.

---
 crypto/Kconfig |  1 +
 crypto/aes.c   | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 69 insertions(+)

index f413cfc9a3e2..5a198275e4fc 100644
@@ -360,6 +360,7 @@ config CRYPTO_AES
 	select CRYPTO_LIB_AES
 	select CRYPTO_LIB_AES_CBC if CRYPTO_CBC != n || CRYPTO_CTS != n
 	select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
+	select CRYPTO_LIB_AES_CTR if CRYPTO_CTR != n || CRYPTO_XCTR != n
 	select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
 	select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	# CRYPTO_SKCIPHER should be selected only if a mode that needs it is

2.55.0

Implement the "xts(aes)" crypto_skcipher algorithm using the corresponding library functions.

Among other benefits, this allows the architecture-optimized AES-XTS code to be migrated into the library while still leaving it accessible via crypto_skcipher, eliminating lots of boilerplate code.

Fast paths similar to what x86_64 uses (to eliminate the scatterlist walking overhead) are included. So we'll get that optimization for all architectures.

For now the cra_priority is set to just 110, since the architecture-optimized implementations of this algorithm haven't yet been migrated into the library. It will be boosted once that happens.

---
 crypto/Kconfig |   1 +
 crypto/aes.c   | 117 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 118 insertions(+)

index 5a198275e4fc..567b719c8a44 100644
@@ -362,6 +362,7 @@ config CRYPTO_AES
 	select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	select CRYPTO_LIB_AES_CTR if CRYPTO_CTR != n || CRYPTO_XCTR != n
 	select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
+	select CRYPTO_LIB_AES_XTS if CRYPTO_XTS != n
 	select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	# CRYPTO_SKCIPHER should be selected only if a mode that needs it is
 	# enabled, but that doesn't work due to a recursive dependency caused by

2.55.0

Implement the "gcm(aes)" and "rfc4106(gcm(aes))" crypto_aead algorithms using the corresponding library functions.

Among other benefits, this allows the architecture-optimized AES-GCM code to be migrated into the library while still leaving it accessible via crypto_aead, eliminating lots of boilerplate code.

For now the cra_priority is set to just 110, since the architecture-optimized implementations of these algorithms haven't yet been migrated into the library. It will be boosted once that happens.

---
 crypto/Kconfig |   2 +
 crypto/aes.c   | 245 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 247 insertions(+)

index 567b719c8a44..bbd314c07fd7 100644
@@ -362,7 +362,9 @@ config CRYPTO_AES
 	select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	select CRYPTO_LIB_AES_CTR if CRYPTO_CTR != n || CRYPTO_XCTR != n
 	select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
+	select CRYPTO_LIB_AES_GCM if CRYPTO_GCM != n
 	select CRYPTO_LIB_AES_XTS if CRYPTO_XTS != n
+	select CRYPTO_AEAD if CRYPTO_GCM != n
 	select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	# CRYPTO_SKCIPHER should be selected only if a mode that needs it is
 	# enabled, but that doesn't work due to a recursive dependency caused by

2.55.0

Implement the "ccm(aes)" crypto_aead algorithm using the corresponding library functions.

Among other benefits, this allows the architecture-optimized AES-CCM code to be migrated into the library while still leaving it accessible via crypto_aead, eliminating lots of boilerplate code.

For now the cra_priority is set to just 110, since the architecture-optimized implementations of this algorithm haven't yet been migrated into the library. It will be boosted once that happens.

---
 crypto/Kconfig |   3 +-
 crypto/aes.c   | 129 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 131 insertions(+), 1 deletion(-)

index bbd314c07fd7..981d5dc422d4 100644
@@ -360,11 +360,12 @@ config CRYPTO_AES
 	select CRYPTO_LIB_AES
 	select CRYPTO_LIB_AES_CBC if CRYPTO_CBC != n || CRYPTO_CTS != n
 	select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
+	select CRYPTO_LIB_AES_CCM if CRYPTO_CCM != n
 	select CRYPTO_LIB_AES_CTR if CRYPTO_CTR != n || CRYPTO_XCTR != n
 	select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
 	select CRYPTO_LIB_AES_GCM if CRYPTO_GCM != n
 	select CRYPTO_LIB_AES_XTS if CRYPTO_XTS != n
-	select CRYPTO_AEAD if CRYPTO_GCM != n
+	select CRYPTO_AEAD if CRYPTO_GCM != n || CRYPTO_CCM != n
 	select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
 	# CRYPTO_SKCIPHER should be selected only if a mode that needs it is
 	# enabled, but that doesn't work due to a recursive dependency caused by

2.55.0

Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next

But as always, more reviews and acks would be greatly appreciated.

- Eric