Skip to contentSkip to conversation

Add support for more AES modes in TI DTHEv2

PATCH Superseded T Pratham 2 participants 8 messages Jan 20 – Mar 2

Subsystems (2)

v8 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

T Pratham

author maintainer
#0 lore raw

v8 cover Add support for more AES modes in TI DTHEv2

DTHEv2 is a new cryptography engine introduced in TI AM62L SoC. The features of DTHEv2 and details of AES modes supported were detailed in [1]. Additional hardware details available in SoC TRM [2].

This patch series adds support for the following AES modes:
- AES-CTR
- AES-GCM
- AES-CCM

The driver is tested using full kernel crypto selftests (CRYPTO_SELFTESTS_FULL) which all pass successfully [3].

---
[1]: [PATCH v7 0/2] Add support for Texas Instruments DTHEv2 Crypto Engine

[2]: Section 14.6.3 (DMA Control Registers -> DMASS_DTHE)

[3]: DTHEv2 AES Engine kernel self-tests logs

Change log:
v8:
- Removed scatterlist chaining from AES-CTR, along with accompanying helper functions added in v6. Replaced with sending only complete blocks to hardware and handling the last partial block in software.
v7:
- Moved padding buffer to inside request ctx.
- Removed already merged AES-XTS patch.
- Moved dthe_copy_sg() helper from CTR patch to GCM patch, where it is being used for first time.
v6:
- Removed memory alloc calls on the data path (CTR padding in aes_run), replaced with scatterlist chaining for added a pad buffer. Added two accompanying helpers dthe_chain_pad_sg() and dthe_unchain_padded_sg().
- Replaced GFP_KERNEL to GFP_ATOMIC in AEAD src and dst scatterlist prep functions to avoid deadlock in data path.
- Added fallback to software in AEADs on failure.
v5:
- Simplified AES-XTS fallback allocation, directly using xts(aes) for alg_name
- Changed fallback to sync and allocated on stack
v4:
- Return -EINVAL in AES-XTS when cryptlen = 0
- Added software fallback for AES-XTS when ciphertext stealing is required (cryptlen is not multiple of AES_BLOCK_SIZE)
- Changed DTHE_MAX_KEYSIZE definition to use AES_MAX_KEY_SIZE instead of AES_KEYSIZE_256
- In AES-CTR, also pad dst scatterlist when padding src scatterlist
- Changed polling for TAG ready to use readl_relaxed_poll_timeout()
- Used crypto API functions to access struct members instead of directly accessing them (crypto_aead_tfm and aead_request_flags)
- Allocated padding buffers in AEAD algos on the stack.
- Changed helper functions dthe_aead_prep_* to return ERR_PTR on error
- Changed some error labels in dthe_aead_run to improve clarity
- Moved iv_in[] declaration from middle of the function to the top
- Corrected setting CCM M value in the hardware register
- Added checks for CCM L value input in the algorithm from IV.
- Added more fallback cases for CCM where hardware has limitations
v3:
- Added header files to remove implicit declaration error.
- Corrected assignment of src_nents and dst_nents in dthe_aead_run (Ran the lkp kernel test bot script locally to ensure no more such errors are present)
v2:
- Corrected assignment of variable unpadded_cryptlen in dthe_aead_run.
- Removed some if conditions which are always false, and documented the cases in comments.
- Moved polling of TAG ready register to a separate function and returning -ETIMEDOUT on poll timeout.
- Corrected comments to adhere to kernel coding guidelines.

Link to previous version:

v7: https://lore.kernel.org/all/[email protected]/
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
---

T Pratham (3):
crypto: ti - Add support for AES-CTR in DTHEv2 driver
crypto: ti - Add support for AES-GCM in DTHEv2 driver
crypto: ti - Add support for AES-CCM in DTHEv2 driver

 drivers/crypto/ti/Kconfig         |   4 +
 drivers/crypto/ti/dthev2-aes.c    | 860 +++++++++++++++++++++++++++++-
 drivers/crypto/ti/dthev2-common.c |  19 +
 drivers/crypto/ti/dthev2-common.h |  27 +-
 4 files changed, 889 insertions(+), 21 deletions(-)
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v8 1/3 crypto: ti - Add support for AES-CTR in DTHEv2 driver

Add support for CTR mode of operation for AES algorithm in the AES Engine of the DTHEv2 hardware cryptographic engine.

---
 drivers/crypto/ti/Kconfig         |   1 +
 drivers/crypto/ti/dthev2-aes.c    | 168 ++++++++++++++++++++++++++----
 drivers/crypto/ti/dthev2-common.h |   3 +
 3 files changed, 153 insertions(+), 19 deletions(-)
drivers/crypto/ti/Kconfig +1 -0
index a3692ceec49bc..6027e12de279d 100644
@@ -6,6 +6,7 @@ config CRYPTO_DEV_TI_DTHEV2
 	select CRYPTO_SKCIPHER
 	select CRYPTO_ECB
 	select CRYPTO_CBC
+	select CRYPTO_CTR
 	select CRYPTO_XTS
 	help
 	  This enables support for the TI DTHE V2 hw cryptography engine
drivers/crypto/ti/dthev2-aes.c +149 -19

282 lines not on this page · raw message

drivers/crypto/ti/dthev2-common.h +3 -0
index c7a06a4c353ff..efbcbbb741a14 100644
@@ -36,6 +36,7 @@
 enum dthe_aes_mode {
 	DTHE_AES_ECB = 0,
 	DTHE_AES_CBC,
+	DTHE_AES_CTR,
 	DTHE_AES_XTS,
 };
 
@@ -92,10 +93,12 @@ struct dthe_tfm_ctx {
 /**
  * struct dthe_aes_req_ctx - AES engine req ctx struct
  * @enc: flag indicating encryption or decryption operation
+ * @padding: padding buffer for handling unaligned data
  * @aes_compl: Completion variable for use in manual completion in case of DMA callback failure
  */
 struct dthe_aes_req_ctx {
 	int enc;
+	u8 padding[AES_BLOCK_SIZE];
 	struct completion aes_compl;
 };
 
Signature · 1 line
2.34.1
maintainer
#2 lore raw

Re: [PATCH v8 1/3] crypto: ti - Add support for AES-CTR in DTHEv2 driver

Quoted 15 lines · T Pratham
drivers/crypto/ti/dthev2-aes.c +14 -0
+	/*
+	 * CTR mode can operate on any input length, but the hardware
+	 * requires input length to be a multiple of the block size.
+	 * We need to handle the padding in the driver.
+	 */
+	if (ctx->aes_mode == DTHE_AES_CTR && req->cryptlen % AES_BLOCK_SIZE) {
+		struct scatterlist *sg;
+		int i = 0;
+		unsigned int curr_len = 0;
+
+		len -= req->cryptlen % AES_BLOCK_SIZE;
+		src_nents = sg_nents_for_len(req->src, len);
+		dst_nents = sg_nents_for_len(req->dst, len);
+

drivers/crypto/ti/dthev2-aes.c static int dthe_aes_run(struct crypto_engine *engine, void *areq) · line 361

Quoted 10 lines
drivers/crypto/ti/dthev2-aes.c +10 -0
+		/*
+		 * Need to truncate the src and dst to len, else DMA complains.
+		 * Lengths restored at end
+		 */
+		for_each_sg(req->src, sg, src_nents - 1, i) {
+			curr_len += sg->length;
+		}
+		curr_len += sg->length;
+		src_bkup_len = sg->length;
+		sg->length -= curr_len % AES_BLOCK_SIZE;

Please don't modify the SG lists since they may be used elsewhere. There is no harm in mapping a bit more data than what you will end up using.

Just truncate the length written to the hardware instead.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author maintainer
#3 lore raw

Re: [PATCH v8 1/3] crypto: ti - Add support for AES-CTR in DTHEv2 driver

Quoted 22 lines · Herbert Xu
Quoted 21 lines · T Pratham
drivers/crypto/ti/dthev2-aes.c +20 -0
+	/*
+	 * CTR mode can operate on any input length, but the hardware
+	 * requires input length to be a multiple of the block size.
+	 * We need to handle the padding in the driver.
+	 */
+	if (ctx->aes_mode == DTHE_AES_CTR && req->cryptlen % AES_BLOCK_SIZE) {
+		struct scatterlist *sg;
+		int i = 0;
+		unsigned int curr_len = 0;
+
+		len -= req->cryptlen % AES_BLOCK_SIZE;
+		src_nents = sg_nents_for_len(req->src, len);
+		dst_nents = sg_nents_for_len(req->dst, len);
+
+		/*
+		 * Need to truncate the src and dst to len, else DMA complains.
+		 * Lengths restored at end
+		 */
+		for_each_sg(req->src, sg, src_nents - 1, i) {
+			curr_len += sg->length;
Quoted 8 lines

drivers/crypto/ti/dthev2-aes.c static int dthe_aes_run(struct crypto_engine *engine, void *areq) · line 361

Quoted 4 lines
drivers/crypto/ti/dthev2-aes.c +4 -0
+		}
+		curr_len += sg->length;
+		src_bkup_len = sg->length;
+		sg->length -= curr_len % AES_BLOCK_SIZE;

Please don't modify the SG lists since they may be used elsewhere. There is no harm in mapping a bit more data than what you will end up using.

The DMA (at least UDMA in K3 SoCs) sends/receives all the mapped length. If I have the dst sg mapped with length (len + x) but write len in crypto hardware, the DMA gets stuck waiting for hardware to send the extra x len. Similar issue in reverse as well.

Also, FWIW, I'm restoring the len in SG list correctly at the end.

Quoted 4 lines

Just truncate the length written to the hardware instead.

Thanks,

Signature · 2 lines
Regards
T Pratham <[email protected]>
maintainer
#4 lore raw

Re: [PATCH v8 1/3] crypto: ti - Add support for AES-CTR in DTHEv2 driver

Quoted 7 lines · T Pratham

The DMA (at least UDMA in K3 SoCs) sends/receives all the mapped length. If I have the dst sg mapped with length (len + x) but write len in crypto hardware, the DMA gets stuck waiting for hardware to send the extra x len. Similar issue in reverse as well.

Also, FWIW, I'm restoring the len in SG list correctly at the end.

If that's the case wouldn't it also break if the caller supplied an SG list that's longer than req->cryptlen? There is no requirement that the SG list must be exactly as long as req->cryptlen and it can contain extra data at the end.

Perhaps you should use sg_split like sa2ul.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author maintainer
#5 lore raw

Re: [PATCH v8 1/3] crypto: ti - Add support for AES-CTR in DTHEv2 driver

Quoted 9 lines · Herbert Xu
Quoted 7 lines · T Pratham

The DMA (at least UDMA in K3 SoCs) sends/receives all the mapped length. If I have the dst sg mapped with length (len + x) but write len in crypto hardware, the DMA gets stuck waiting for hardware to send the extra x len. Similar issue in reverse as well.

Also, FWIW, I'm restoring the len in SG list correctly at the end.

Quoted 8 lines

If that's the case wouldn't it also break if the caller supplied an SG list that's longer than req->cryptlen? There is no requirement that the SG list must be exactly as long as req->cryptlen and it can contain extra data at the end.

Perhaps you should use sg_split like sa2ul.

Thanks,

Understood. Will update accordingly.
Also, sent an updated series twice with other changes. Here, for reference, is the latest one:
https://lore.kernel.org/all/[email protected]/

Signature · 2 lines
Regards
T Pratham <[email protected]>
author maintainer
#6 lore raw

PATCH v8 2/3 crypto: ti - Add support for AES-GCM in DTHEv2 driver

In reply to T Pratham

AES-GCM is an AEAD algorithm supporting both encryption and authentication of data. This patch introduces support for AES-GCM as the first AEAD algorithm supported by the DTHEv2 driver.

---
 drivers/crypto/ti/Kconfig         |   2 +
 drivers/crypto/ti/dthev2-aes.c    | 595 +++++++++++++++++++++++++++++-
 drivers/crypto/ti/dthev2-common.c |  19 +
 drivers/crypto/ti/dthev2-common.h |  23 +-
 4 files changed, 637 insertions(+), 2 deletions(-)
drivers/crypto/ti/Kconfig +2 -0
index 6027e12de279d..221e483737439 100644
@@ -8,6 +8,8 @@ config CRYPTO_DEV_TI_DTHEV2
 	select CRYPTO_CBC
 	select CRYPTO_CTR
 	select CRYPTO_XTS
+	select CRYPTO_GCM
+	select SG_SPLIT
 	help
 	  This enables support for the TI DTHE V2 hw cryptography engine
 	  which can be found on TI K3 SOCs. Selecting this enables use
drivers/crypto/ti/dthev2-aes.c +594 -1

648 lines not on this page · raw message

drivers/crypto/ti/dthev2-common.c +19 -0
index c39d37933b9ee..a2ad79bec105a 100644
@@ -48,6 +48,25 @@ struct dthe_data *dthe_get_dev(struct dthe_tfm_ctx *ctx)
 	return dev_data;
 }
 
+struct scatterlist *dthe_copy_sg(struct scatterlist *dst,
+				 struct scatterlist *src,
+				 int buflen)
+{
+	struct scatterlist *from_sg, *to_sg;
+	int sglen;
+
+	for (to_sg = dst, from_sg = src; buflen && from_sg; buflen -= sglen) {
+		sglen = from_sg->length;
+		if (sglen > buflen)
+			sglen = buflen;
+		sg_set_buf(to_sg, sg_virt(from_sg), sglen);
+		from_sg = sg_next(from_sg);
+		to_sg = sg_next(to_sg);
+	}
+
+	return to_sg;
+}
+
 static int dthe_dma_init(struct dthe_data *dev_data)
 {
 	int ret;
drivers/crypto/ti/dthev2-common.h +22 -1
index efbcbbb741a14..0aaecf02258e0 100644
@@ -38,6 +38,7 @@ enum dthe_aes_mode {
 	DTHE_AES_CBC,
 	DTHE_AES_CTR,
 	DTHE_AES_XTS,
+	DTHE_AES_GCM,
 };
 
 /* Driver specific struct definitions */
@@ -78,16 +79,22 @@ struct dthe_list {
  * struct dthe_tfm_ctx - Transform ctx struct containing ctx for all sub-components of DTHE V2
  * @dev_data: Device data struct pointer
  * @keylen: AES key length
+ * @authsize: Authentication size for modes with authentication
  * @key: AES key
  * @aes_mode: AES mode
+ * @aead_fb: Fallback crypto aead handle
  * @skcipher_fb: Fallback crypto skcipher handle for AES-XTS mode
  */
 struct dthe_tfm_ctx {
 	struct dthe_data *dev_data;
 	unsigned int keylen;
+	unsigned int authsize;
 	u32 key[DTHE_MAX_KEYSIZE / sizeof(u32)];
 	enum dthe_aes_mode aes_mode;
-	struct crypto_sync_skcipher *skcipher_fb;
+	union {
+		struct crypto_sync_aead *aead_fb;
+		struct crypto_sync_skcipher *skcipher_fb;
+	};
 };
 
 /**
@@ -106,6 +113,20 @@ struct dthe_aes_req_ctx {
 
 struct dthe_data *dthe_get_dev(struct dthe_tfm_ctx *ctx);
 
+/**
+ * dthe_copy_sg - Copy sg entries from src to dst
+ * @dst: Destination sg to be filled
+ * @src: Source sg to be copied from
+ * @buflen: Number of bytes to be copied
+ *
+ * Description:
+ *   Copy buflen bytes of data from src to dst.
+ *
+ **/
+struct scatterlist *dthe_copy_sg(struct scatterlist *dst,
+				 struct scatterlist *src,
+				 int buflen);
+
 int dthe_register_aes_algs(void);
 void dthe_unregister_aes_algs(void);
 
Signature · 1 line
2.34.1
author maintainer
#7 lore raw

PATCH v8 3/3 crypto: ti - Add support for AES-CCM in DTHEv2 driver

In reply to T Pratham

AES-CCM is an AEAD algorithm supporting both encryption and authentication of data. This patch introduces support for AES-CCM AEAD algorithm in the DTHEv2 driver.

---
 drivers/crypto/ti/Kconfig         |   1 +
 drivers/crypto/ti/dthev2-aes.c    | 129 ++++++++++++++++++++++++++----
 drivers/crypto/ti/dthev2-common.h |   1 +
 3 files changed, 115 insertions(+), 16 deletions(-)
drivers/crypto/ti/Kconfig +1 -0
index 221e483737439..1a3a571ac8cef 100644
@@ -9,6 +9,7 @@ config CRYPTO_DEV_TI_DTHEV2
 	select CRYPTO_CTR
 	select CRYPTO_XTS
 	select CRYPTO_GCM
+	select CRYPTO_CCM
 	select SG_SPLIT
 	help
 	  This enables support for the TI DTHE V2 hw cryptography engine
drivers/crypto/ti/dthev2-aes.c +113 -16

206 lines not on this page · raw message

drivers/crypto/ti/dthev2-common.h +1 -0
index 0aaecf02258e0..3b6f97356e1b3 100644
@@ -39,6 +39,7 @@ enum dthe_aes_mode {
 	DTHE_AES_CTR,
 	DTHE_AES_XTS,
 	DTHE_AES_GCM,
+	DTHE_AES_CCM,
 };
 
 /* Driver specific struct definitions */
Signature · 1 line
2.34.1