Skip to contentSkip to conversation

fixes and improvements to cnxk crytpo PMD

PATCH In review Tejasree Kondoj 2 participants 19 messages Dec 20, 2022 – Jan 4, 2023 Crypto API

lore
Share & tools
lore
#0 lore raw

cover fixes and improvements to cnxk crytpo PMD

This series adds improvements and support for SHA3, IPsec scatter gather mode in cnxk crypto PMD.

Aakash Sasidharan (1):
common/cnxk: generate opad and ipad in driver

Anoob Joseph (3):
common/cnxk: perform LF fini ops only when allocated crypto/cnxk: remove null check of session priv common/cnxk: remove salt from session

Archana Muniganti (1):
crypto/cnxk: add CN9K IPsec SG support

Gowrishankar Muthukrishnan (1):
crypto/cnxk: fix incorrect digest for an empty input data

Tejasree Kondoj (5):
crypto/cnxk: support truncated digest length crypto/octeontx: support truncated digest size crypto/cnxk: set device ops to null in PCI remove crypto/cnxk: add CTX for non IPsec operations crypto/cnxk: set salt in dptr as part of IV

Vidya Sagar Velumuri (2):
crypto/cnxk: update resp len calculation for IPv6 crypto/cnxk: update crypto completion code handling

Volodymyr Fialko (4):
crypto/cnxk: add context to passthrough instruction crypto/cnxk: add queue pair check to meta set crypto/cnxk: add support for SHA3 hash common/cnxk: skip hmac hash precomputation

 doc/guides/cryptodevs/cnxk.rst                |   8 +
 doc/guides/cryptodevs/features/cn10k.ini      |   8 +
 doc/guides/cryptodevs/features/cn9k.ini       |   8 +
 drivers/common/cnxk/cnxk_security.c           |   8 +-
 drivers/common/cnxk/roc_api.h                 |   1 +
 drivers/common/cnxk/roc_cpt.c                 |   3 +
 drivers/common/cnxk/roc_cpt_sg.h              |  37 ++
 drivers/common/cnxk/roc_hash.c                |  12 +-
 drivers/common/cnxk/roc_hash.h                |   2 +-
 drivers/common/cnxk/roc_ie_on.h               |   9 +-
 drivers/common/cnxk/roc_platform.h            |   3 +-
 drivers/common/cnxk/roc_se.c                  | 150 ++++-
 drivers/common/cnxk/roc_se.h                  |  70 +--
 drivers/common/cnxk/version.map               |   1 +
 drivers/common/cpt/cpt_ucode.h                |  17 +-
 drivers/crypto/cnxk/cn10k_cryptodev.c         |   1 +
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c     |  51 +-
 drivers/crypto/cnxk/cn10k_ipsec.c             |  11 +-
 drivers/crypto/cnxk/cn10k_ipsec.h             |   1 +
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h      |   1 +
 drivers/crypto/cnxk/cn9k_cryptodev.c          |   1 +
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c      |  89 +--
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h       | 207 ++++++-
 drivers/crypto/cnxk/cnxk_cryptodev.h          |   2 +-
 .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 212 ++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c      |  65 ++-
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h      |  32 +-
 drivers/crypto/cnxk/cnxk_se.h                 | 551 +++++-------------
 drivers/crypto/cnxk/cnxk_sg.h                 | 273 +++++++++
 .../octeontx/otx_cryptodev_capabilities.c     |  48 +-
 30 files changed, 1224 insertions(+), 658 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_cpt_sg.h
 create mode 100644 drivers/crypto/cnxk/cnxk_sg.h
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/17 common/cnxk: perform LF fini ops only when allocated

From: Anoob Joseph <[email protected]>

If LFs are not attached, LF detach and free ops should not be attempted.

---
 drivers/common/cnxk/roc_cpt.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/common/cnxk/roc_cpt.c +3 -0
index fb97ec89b2..bf216b5545 100644
@@ -861,6 +861,9 @@ roc_cpt_dev_clear(struct roc_cpt *roc_cpt)
 	if (cpt == NULL)
 		return;
 
+	if (roc_cpt->nb_lf == 0)
+		return;
+
 	for (i = 0; i < roc_cpt->nb_lf; i++)
 		cpt->lf_msix_off[i] = 0;
 
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/17 common/cnxk: generate opad and ipad in driver

In reply to Tejasree Kondoj

From: Aakash Sasidharan <[email protected]>

Generate opad and ipad in driver for SHA based crypto algos

---
 drivers/common/cnxk/cnxk_security.c |  4 +--
 drivers/common/cnxk/roc_hash.c      | 12 ++++---
 drivers/common/cnxk/roc_hash.h      |  2 +-
 drivers/common/cnxk/roc_se.c        | 52 +++++++++++++++++++++++++++--
 4 files changed, 60 insertions(+), 10 deletions(-)
drivers/common/cnxk/cnxk_security.c +2 -2
index 85105472a1..5034c76938 100644
@@ -37,8 +37,8 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]);
 		break;
 	case RTE_CRYPTO_AUTH_SHA256_HMAC:
-		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
-		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
+		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0], 256);
+		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64], 256);
 		break;
 	case RTE_CRYPTO_AUTH_SHA384_HMAC:
 		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 384);
drivers/common/cnxk/roc_hash.c +8 -4
index 1b9030e693..8c451440b1 100644
@@ -232,7 +232,7 @@ roc_hash_sha1_gen(uint8_t *msg, uint32_t *hash)
  * Based on implementation from RFC 3174
  */
 void
-roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash)
+roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash, int hash_size)
 {
 	const uint32_t _K[] = {
 		/* Round Constants defined in SHA-256   */
@@ -250,13 +250,17 @@ roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash)
 		0x682e6ff3, 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208,
 		0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2};
 
-	const uint32_t _H[] = {/* Initial Hash constants defined in SHA-256 */
-			       0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
-			       0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19};
+	const uint32_t _H224[] = {/* Initial Hash constants defined in SHA-224 */
+				  0xc1059ed8, 0x367cd507, 0x3070dd17, 0xf70e5939,
+				  0xffc00b31, 0x68581511, 0x64f98fa7, 0xbefa4fa4};
+	const uint32_t _H256[] = {/* Initial Hash constants defined in SHA-256 */
+				  0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
+				  0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19};
 	int i;
 	uint32_t temp[4], S0, S1;	 /* Temporary word value */
 	uint32_t W[64];			 /* Word sequence */
 	uint32_t A, B, C, D, E, F, G, H; /* Word buffers */
+	const uint32_t *_H = (hash_size == 224) ? _H224 : _H256;
 
 	/* Initialize the first 16 words in the array W */
 	memcpy(&W[0], msg, 16 * sizeof(W[0]));
drivers/common/cnxk/roc_hash.h +1 -1
index 8940faa6eb..c3ddb9407b 100644
@@ -11,7 +11,7 @@
  */
 void __roc_api roc_hash_md5_gen(uint8_t *msg, uint32_t *hash);
 void __roc_api roc_hash_sha1_gen(uint8_t *msg, uint32_t *hash);
-void __roc_api roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash);
+void __roc_api roc_hash_sha256_gen(uint8_t *msg, uint32_t *hash, int hash_size);
 void __roc_api roc_hash_sha512_gen(uint8_t *msg, uint64_t *hash, int hash_size);
 
 #endif /* _ROC_HASH_H_ */
drivers/common/cnxk/roc_se.c +49 -3

64 lines not on this page · raw message

Signature · 1 line
2.25.1
#3 lore raw

PATCH 3/17 crypto/cnxk: update resp len calculation for IPv6

In reply to Tejasree Kondoj

From: Vidya Sagar Velumuri <[email protected]>

In IPsec lookaside path, response length for an encryption is pre calculated and updated in the mbuf as the packet length. Response length dpepends on the layer-3 header length.
As IPsec security context does not provide any information about the IP version, the layer-3 header length is calculated in data path based on IP version and extension headers.

---
 drivers/common/cnxk/cnxk_security.c     |  4 +--
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 36 ++++++++++++++++++++++---
 2 files changed, 35 insertions(+), 5 deletions(-)
drivers/common/cnxk/cnxk_security.c +2 -2
index 5034c76938..79427d48fe 100644
@@ -19,7 +19,7 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 	uint32_t i;
 
 	/* HMAC OPAD and IPAD */
-	for (i = 0; i < 127 && i < length; i++) {
+	for (i = 0; i < 128 && i < length; i++) {
 		opad[i] = opad[i] ^ key[i];
 		ipad[i] = ipad[i] ^ key[i];
 	}
@@ -1344,7 +1344,7 @@ cnxk_on_ipsec_outb_sa_create(struct rte_security_ipsec_xform *ipsec,
 	} else
 		ctx_len += sizeof(template->ip4);
 
-	ctx_len += RTE_ALIGN_CEIL(ctx_len, 8);
+	ctx_len = RTE_ALIGN_CEIL(ctx_len, 8);
 
 	if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
 		uint8_t *hmac_opad_ipad = (uint8_t *)&out_sa->sha2;
drivers/crypto/cnxk/cn9k_ipsec_la_ops.h +33 -3
index 724fc525ad..f1298017ce 100644
@@ -13,14 +13,44 @@
 #include "cnxk_security_ar.h"
 
 static __rte_always_inline int32_t
-ipsec_po_out_rlen_get(struct cn9k_sec_session *sess, uint32_t plen)
+ipsec_po_out_rlen_get(struct cn9k_sec_session *sess, uint32_t plen, struct rte_mbuf *m_src)
 {
 	uint32_t enc_payload_len;
 	int adj_len = 0;
 
-	if (sess->sa.out_sa.common_sa.ctl.ipsec_mode == ROC_IE_SA_MODE_TRANSPORT)
+	if (sess->sa.out_sa.common_sa.ctl.ipsec_mode == ROC_IE_SA_MODE_TRANSPORT) {
 		adj_len = ROC_CPT_TUNNEL_IPV4_HDR_LEN;
 
+		uintptr_t data = (uintptr_t)m_src->buf_addr + m_src->data_off;
+		struct rte_ipv4_hdr *ip = (struct rte_ipv4_hdr *)data;
+
+		if (unlikely(((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) != IPVERSION)) {
+			struct rte_ipv6_hdr *ip6 = (struct rte_ipv6_hdr *)ip;
+			uint8_t *nxt_hdr = (uint8_t *)ip6;
+			uint8_t dest_op_cnt = 0;
+			int nh = ip6->proto;
+
+			PLT_ASSERT(((ip->version_ihl & 0xf0) >> RTE_IPV4_IHL_MULTIPLIER) == 6);
+
+			adj_len = ROC_CPT_TUNNEL_IPV6_HDR_LEN;
+			nxt_hdr += ROC_CPT_TUNNEL_IPV6_HDR_LEN;
+			while (nh != -EINVAL) {
+				size_t ext_len = 0;
+
+				nh = rte_ipv6_get_next_ext(nxt_hdr, nh, &ext_len);
+				/* With multiple dest ops headers, the ESP hdr will be before
+				 * the 2nd dest ops and after the first dest ops header
+				 */
+				if ((nh == IPPROTO_DSTOPTS) && dest_op_cnt)
+					break;
+				else if (nh == IPPROTO_DSTOPTS)
+					dest_op_cnt++;
+				adj_len += ext_len;
+				nxt_hdr += ext_len;
+			}
+		}
+	}
+
 	enc_payload_len =
 		RTE_ALIGN_CEIL(plen + sess->rlens.roundup_len - adj_len, sess->rlens.roundup_byte);
 
@@ -41,7 +71,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_sec_session *sess, struct
 
 	pkt_len = rte_pktmbuf_pkt_len(m_src);
 	dlen = pkt_len + hdr_len;
-	rlen = ipsec_po_out_rlen_get(sess, pkt_len);
+	rlen = ipsec_po_out_rlen_get(sess, pkt_len, m_src);
 
 	extend_tail = rlen - dlen;
 	if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) {
Signature · 1 line
2.25.1
#4 lore raw

PATCH 4/17 crypto/cnxk: add context to passthrough instruction

In reply to Tejasree Kondoj

From: Volodymyr Fialko <[email protected]>

Attach valid context of last packet instruction to the passthrough instruction to match hardware requirements.

---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +10 -3
index 7dad370047..2c750d19bf 100644
@@ -36,6 +36,7 @@ struct ops_burst {
 struct vec_request {
 	struct cpt_inflight_req *req;
 	struct rte_event_vector *vec;
+	union cpt_inst_w7 w7;
 	uint64_t w2;
 };
 
@@ -387,7 +388,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, struct cnxk_cpt_qp **qp, ui
 
 static inline void
 cn10k_cpt_vec_inst_fill(struct vec_request *vec_req, struct cpt_inst_s *inst,
-			struct cnxk_cpt_qp *qp)
+			struct cnxk_cpt_qp *qp, union cpt_inst_w7 w7)
 {
 	const union cpt_res_s res = {.cn10k.compcode = CPT_COMP_NOT_DONE};
 	struct cpt_inflight_req *infl_req = vec_req->req;
@@ -400,6 +401,8 @@ cn10k_cpt_vec_inst_fill(struct vec_request *vec_req, struct cpt_inst_s *inst,
 		.s.dlen = 0,
 	};
 
+	w7.s.egrp = ROC_CPT_DFLT_ENG_GRP_SE;
+
 	infl_req->vec = vec_req->vec;
 	infl_req->qp = qp;
 
@@ -410,7 +413,7 @@ cn10k_cpt_vec_inst_fill(struct vec_request *vec_req, struct cpt_inst_s *inst,
 	inst->w2.u64 = vec_req->w2;
 	inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req);
 	inst->w4.u64 = w4.u64;
-	inst->w7.u64 = ROC_CPT_DFLT_ENG_GRP_SE << 61;
+	inst->w7.u64 = w7.u64;
 }
 
 static void
@@ -451,7 +454,7 @@ cn10k_cpt_vec_submit(struct vec_request vec_tbl[], uint16_t vec_tbl_len, struct
 again:
 	burst_size = RTE_MIN(PKTS_PER_STEORL, vec_tbl_len);
 	for (i = 0; i < burst_size; i++)
-		cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp);
+		cn10k_cpt_vec_inst_fill(&vec_tbl[i], &inst[i * 2], qp, vec_tbl[0].w7);
 
 	do {
 		fc.u64[0] = __atomic_load_n(fc_addr, __ATOMIC_RELAXED);
@@ -590,6 +593,10 @@ next_op:;
 		roc_lmt_submit_steorl(lmt_arg, io_addr);
 	}
 
+	/* Store w7 of last successfully filled instruction */
+	inst = &inst_base[2 * (i - 1)];
+	vec_tbl[0].w7 = inst->w7;
+
 	rte_io_wmb();
 
 put:
Signature · 1 line
2.25.1
#5 lore raw

PATCH 5/17 crypto/cnxk: support truncated digest length

In reply to Tejasree Kondoj

Adding truncated digest length support.

---
 .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 48 +++++++++----------
 drivers/crypto/cnxk/cnxk_se.h                 | 19 ++++----
 2 files changed, 35 insertions(+), 32 deletions(-)
drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +24 -24
index 6c28f8942e..9dfbf875ec 100644
@@ -108,9 +108,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 20,
+					.min = 1,
 					.max = 20,
-					.increment = 0
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -128,9 +128,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 1
 				},
 				.digest_size = {
-					.min = 12,
+					.min = 1,
 					.max = 20,
-					.increment = 8
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -148,9 +148,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 28,
+					.min = 1,
 					.max = 28,
-					.increment = 0
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -168,9 +168,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 1
 				},
 				.digest_size = {
-					.min = 28,
+					.min = 1,
 					.max = 28,
-					.increment = 0
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -188,9 +188,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 32,
+					.min = 1,
 					.max = 32,
-					.increment = 0
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -208,9 +208,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 1
 				},
 				.digest_size = {
-					.min = 16,
+					.min = 1,
 					.max = 32,
-					.increment = 16
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -228,9 +228,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 48,
+					.min = 1,
 					.max = 48,
-					.increment = 0
+					.increment = 1
 					},
 			}, }
 		}, }
@@ -248,9 +248,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 1
 				},
 				.digest_size = {
-					.min = 24,
+					.min = 1,
 					.max = 48,
-					.increment = 24
+					.increment = 1
 					},
 			}, }
 		}, }
@@ -268,9 +268,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 64,
+					.min = 1,
 					.max = 64,
-					.increment = 0
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -288,9 +288,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 1
 				},
 				.digest_size = {
-					.min = 32,
+					.min = 1,
 					.max = 64,
-					.increment = 32
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -308,9 +308,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 16,
+					.min = 1,
 					.max = 16,
-					.increment = 0
+					.increment = 1
 				},
 			}, }
 		}, }
@@ -328,9 +328,9 @@ static const struct rte_cryptodev_capabilities caps_sha1_sha2[] = {
 					.increment = 8
 				},
 				.digest_size = {
-					.min = 12,
+					.min = 1,
 					.max = 16,
-					.increment = 4
+					.increment = 1
 				},
 			}, }
 		}, }
drivers/crypto/cnxk/cnxk_se.h +11 -8
index b07fc22858..ce4c2bb585 100644
@@ -148,30 +148,33 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth)
 	uint16_t mac_len = auth->digest_length;
 	int ret;
 
+	if ((auth->algo != RTE_CRYPTO_AUTH_NULL) && (mac_len == 0))
+		return -1;
+
 	switch (auth->algo) {
 	case RTE_CRYPTO_AUTH_MD5:
 	case RTE_CRYPTO_AUTH_MD5_HMAC:
-		ret = (mac_len == 16) ? 0 : -1;
+		ret = (mac_len <= 16) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA1:
 	case RTE_CRYPTO_AUTH_SHA1_HMAC:
-		ret = (mac_len == 20) ? 0 : -1;
+		ret = (mac_len <= 20) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA224:
 	case RTE_CRYPTO_AUTH_SHA224_HMAC:
-		ret = (mac_len == 28) ? 0 : -1;
+		ret = (mac_len <= 28) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA256:
 	case RTE_CRYPTO_AUTH_SHA256_HMAC:
-		ret = (mac_len == 32) ? 0 : -1;
+		ret = (mac_len <= 32) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA384:
 	case RTE_CRYPTO_AUTH_SHA384_HMAC:
-		ret = (mac_len == 48) ? 0 : -1;
+		ret = (mac_len <= 48) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA512:
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
-		ret = (mac_len == 64) ? 0 : -1;
+		ret = (mac_len <= 64) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_NULL:
 		ret = 0;
@@ -838,7 +841,7 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa
 
 	/*GP op header */
 	cpt_inst_w4.s.opcode_minor = 0;
-	cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8);
+	cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8) | mac_len;
 	if (ctx->hmac) {
 		cpt_inst_w4.s.opcode_major =
 			ROC_SE_MAJOR_OP_HMAC | ROC_SE_DMA_MODE;
@@ -969,7 +972,7 @@ cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa
 
 	/*GP op header */
 	cpt_inst_w4.s.opcode_minor = 0;
-	cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8);
+	cpt_inst_w4.s.param2 = ((uint16_t)hash_type << 8) | mac_len;
 	if (ctx->hmac) {
 		cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_HMAC;
 		cpt_inst_w4.s.param1 = key_len;
Signature · 1 line
2.25.1
#6 lore raw

PATCH 6/17 crypto/cnxk: add queue pair check to meta set

In reply to Tejasree Kondoj

From: Volodymyr Fialko <[email protected]>

Check that queue pair provided by metadata information is enabled (added to crypto adapter).

---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +3 -0
index 2c750d19bf..1caa321112 100644
@@ -296,6 +296,9 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, vo
 	qp_id = ec_mdata->request_info.queue_pair_id;
 	qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id];
 
+	if (!qp->ca.enabled)
+		return -EINVAL;
+
 	/* Prepare w2 */
 	tag_type = qp->ca.vector_sz ? RTE_EVENT_TYPE_CRYPTODEV_VECTOR : RTE_EVENT_TYPE_CRYPTODEV;
 	rsp_info = &ec_mdata->response_info;
Signature · 1 line
2.25.1
#7 lore raw

PATCH 7/17 crypto/cnxk: update crypto completion code handling

In reply to Tejasree Kondoj

From: Vidya Sagar Velumuri <[email protected]>

Update crypto and IPsec completion handling as per microcode version OCPT-04-IE-IPSEC-MC-30-01-28-00

ci: skip_checkformat

---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 18 +++++++++---------
 drivers/crypto/cnxk/cn10k_ipsec.c         |  7 +++++--
 drivers/crypto/cnxk/cn10k_ipsec.h         |  1 +
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  |  1 +
 4 files changed, 16 insertions(+), 11 deletions(-)
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +9 -9
index 1caa321112..5a098ffcf2 100644
@@ -801,13 +801,11 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re
 	struct rte_mbuf *mbuf = cop->sym->m_src;
 	const uint16_t m_len = res->rlen;
 
-	mbuf->data_len = m_len;
-	mbuf->pkt_len = m_len;
-
 	switch (res->uc_compcode) {
 	case ROC_IE_OT_UCC_SUCCESS:
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_IP_BADCSUM:
+		mbuf->ol_flags &= ~RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_L4_GOODCSUM:
@@ -819,15 +817,17 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re
 				  RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_IP_GOODCSUM:
-		mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST:
+	case ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_AGAIN:
 		cop->aux_flags = RTE_CRYPTO_OP_AUX_FLAGS_IPSEC_SOFT_EXPIRY;
 		break;
 	default:
-		plt_dp_err("Success with unknown microcode completion code");
-		break;
+		cop->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
 	}
+	mbuf->data_len = m_len;
+	mbuf->pkt_len = m_len;
 }
 
 static inline void
@@ -843,7 +843,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 
 	if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC &&
 	    cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
-		if (likely(compcode == CPT_COMP_WARN)) {
+		if (likely(compcode == CPT_COMP_GOOD || compcode == CPT_COMP_WARN)) {
 			/* Success with additional info */
 			cn10k_cpt_sec_post_process(cop, res);
 		} else {
@@ -860,7 +860,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 		return;
 	}
 
-	if (likely(compcode == CPT_COMP_GOOD || compcode == CPT_COMP_WARN)) {
+	if (likely(compcode == CPT_COMP_GOOD)) {
 		if (unlikely(uc_compcode)) {
 			if (uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE)
 				cop->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
@@ -964,7 +964,7 @@ cn10k_cpt_crypto_adapter_vector_dequeue(uintptr_t get_work1)
 
 #ifdef CNXK_CRYPTODEV_DEBUG
 	res.u64[0] = __atomic_load_n(&vec_infl_req->res.u64[0], __ATOMIC_RELAXED);
-	PLT_ASSERT(res.cn10k.compcode == CPT_COMP_WARN);
+	PLT_ASSERT(res.cn10k.compcode == CPT_COMP_GOOD);
 	PLT_ASSERT(res.cn10k.uc_compcode == 0);
 #endif
 
drivers/crypto/cnxk/cn10k_ipsec.c +5 -2
index 1740a73c36..aafd461436 100644
@@ -200,9 +200,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Disable IP checksum verification by default */
 	param1.s.ip_csum_disable = ROC_IE_OT_SA_INNER_PKT_IP_CSUM_DISABLE;
 
+	/* Set the ip chksum flag in mbuf before enqueue.
+	 * Reset the flag in post process in case of errors
+	 */
 	if (ipsec_xfrm->options.ip_csum_enable) {
-		param1.s.ip_csum_disable =
-			ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE;
+		param1.s.ip_csum_disable = ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE;
+		sec_sess->ip_csum = RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 	}
 
 	/* Disable L4 checksum verification by default */
drivers/crypto/cnxk/cn10k_ipsec.h +1 -0
index 044fe33046..23d7a4fac4 100644
@@ -33,6 +33,7 @@ struct cn10k_sec_session {
 	uint16_t max_extended_len;
 	uint16_t iv_offset;
 	uint8_t iv_length;
+	uint8_t ip_csum;
 	bool is_outbound;
 	/** Queue pair */
 	struct cnxk_cpt_qp *qp;
drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +1 -0
index 084198b5bb..f2761a55a5 100644
@@ -98,6 +98,7 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, struct
 	inst->w4.u64 = sess->inst.w4 | rte_pktmbuf_pkt_len(m_src);
 	dptr = rte_pktmbuf_mtod(m_src, uint64_t);
 	inst->dptr = dptr;
+	m_src->ol_flags |= (uint64_t)sess->ip_csum;
 
 	return 0;
 }
Signature · 1 line
2.25.1
#8 lore raw

PATCH 8/17 crypto/cnxk: fix incorrect digest for an empty input data

In reply to Tejasree Kondoj

From: Gowrishankar Muthukrishnan <[email protected]>

For an empty input data, digest value is incorrectly generated from previous auth op, in case executed. This is due to incorrect gather component used.

Fixes 7c19abdd0cf1 ("common/cnxk: support 103XX CPT")
---
 drivers/crypto/cnxk/cnxk_se.h | 46 ++++++++++++++---------------------
 1 file changed, 18 insertions(+), 28 deletions(-)
drivers/crypto/cnxk/cnxk_se.h +18 -28
index ce4c2bb585..2944d3c287 100644
@@ -247,7 +247,7 @@ fill_sg_comp_from_iov(struct roc_se_sglist_comp *list, uint32_t i,
 	uint32_t extra_len = extra_buf ? extra_buf->size : 0;
 	uint32_t size = *psize;
 
-	for (j = 0; (j < from->buf_cnt) && size; j++) {
+	for (j = 0; j < from->buf_cnt; j++) {
 		struct roc_se_sglist_comp *to = &list[i >> 2];
 		uint32_t buf_sz = from->bufs[j].size;
 		void *vaddr = from->bufs[j].vaddr;
@@ -314,6 +314,9 @@ fill_sg_comp_from_iov(struct roc_se_sglist_comp *list, uint32_t i,
 		if (extra_offset)
 			extra_offset -= size;
 		i++;
+
+		if (unlikely(!size))
+			break;
 	}
 
 	*psize = size;
@@ -370,7 +373,9 @@ fill_sg2_comp_from_iov(struct roc_se_sg2list_comp *list, uint32_t i, struct roc_
 	uint32_t extra_len = extra_buf ? extra_buf->size : 0;
 	uint32_t size = *psize;
 
-	for (j = 0; (j < from->buf_cnt) && size; j++) {
+	rte_prefetch2(psize);
+
+	for (j = 0; j < from->buf_cnt; j++) {
 		struct roc_se_sg2list_comp *to = &list[i / 3];
 		uint32_t buf_sz = from->bufs[j].size;
 		void *vaddr = from->bufs[j].vaddr;
@@ -436,6 +441,9 @@ fill_sg2_comp_from_iov(struct roc_se_sg2list_comp *list, uint32_t i, struct roc_
 		if (extra_offset)
 			extra_offset -= size;
 		i++;
+
+		if (unlikely(!size))
+			break;
 	}
 
 	*psize = size;
@@ -887,20 +895,10 @@ cpt_digest_gen_sg_ver1_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa
 
 	/* input data */
 	size = data_len;
-	if (size) {
-		i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, 0,
-					  &size, NULL, 0);
-		if (unlikely(size)) {
-			plt_dp_err("Insufficient dst IOV size, short by %dB",
-				   size);
-			return -1;
-		}
-	} else {
-		/*
-		 * Looks like we need to support zero data
-		 * gather ptr in case of hash & hmac
-		 */
-		i++;
+	i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, 0, &size, NULL, 0);
+	if (unlikely(size)) {
+		plt_dp_err("Insufficient dst IOV size, short by %dB", size);
+		return -1;
 	}
 	((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i);
 	g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_se_sglist_comp);
@@ -1011,18 +1009,10 @@ cpt_digest_gen_sg_ver2_prep(uint32_t flags, uint64_t d_lens, struct roc_se_fc_pa
 
 	/* input data */
 	size = data_len;
-	if (size) {
-		i = fill_sg2_comp_from_iov(gather_comp, i, params->src_iov, 0, &size, NULL, 0);
-		if (unlikely(size)) {
-			plt_dp_err("Insufficient dst IOV size, short by %dB", size);
-			return -1;
-		}
-	} else {
-		/*
-		 * Looks like we need to support zero data
-		 * gather ptr in case of hash & hmac
-		 */
-		i++;
+	i = fill_sg2_comp_from_iov(gather_comp, i, params->src_iov, 0, &size, NULL, 0);
+	if (unlikely(size)) {
+		plt_dp_err("Insufficient dst IOV size, short by %dB", size);
+		return -1;
 	}
 	cpt_inst_w5.s.gather_sz = ((i + 2) / 3);
 
Signature · 1 line
2.25.1
#9 lore raw

PATCH 9/17 crypto/cnxk: add CN9K IPsec SG support

In reply to Tejasree Kondoj

From: Archana Muniganti <[email protected]>

Added IPsec SG support in CN9K

---
 drivers/common/cnxk/roc_api.h            |   1 +
 drivers/common/cnxk/roc_cpt_sg.h         |  37 +++
 drivers/common/cnxk/roc_ie_on.h          |   9 +-
 drivers/common/cnxk/roc_se.h             |  28 --
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c |  75 ++---
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h  | 171 ++++++++--
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c |  25 +-
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h |  19 ++
 drivers/crypto/cnxk/cnxk_se.h            | 387 +++--------------------
 drivers/crypto/cnxk/cnxk_sg.h            | 273 ++++++++++++++++
 10 files changed, 594 insertions(+), 431 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_cpt_sg.h
 create mode 100644 drivers/crypto/cnxk/cnxk_sg.h
drivers/common/cnxk/roc_api.h +1 -0
index 072f16d77d..14a11321e0 100644
@@ -86,6 +86,7 @@
 /* CPT microcode */
 #include "roc_ae.h"
 #include "roc_ae_fpm_tables.h"
+#include "roc_cpt_sg.h"
 #include "roc_ie.h"
 #include "roc_ie_on.h"
 #include "roc_ie_ot.h"
drivers/common/cnxk/roc_cpt_sg.h new file +37 -0
new file mode 100644
index 0000000000..8a97e1aa5b
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#ifndef _ROC_CPT_SG_H_
+#define _ROC_CPT_SG_H_
+
+#define ROC_DMA_MODE_SG (1 << 7)
+
+#define ROC_MAX_SG_IN_OUT_CNT 32
+#define ROC_MAX_SG_CNT	      (ROC_MAX_SG_IN_OUT_CNT / 2)
+
+#define ROC_SG_LIST_HDR_SIZE (8u)
+#define ROC_SG_ENTRY_SIZE    sizeof(struct roc_sglist_comp)
+
+struct roc_sglist_comp {
+	union {
+		uint64_t len;
+		struct {
+			uint16_t len[4];
+		} s;
+	} u;
+	uint64_t ptr[4];
+};
+
+struct roc_sg2list_comp {
+	union {
+		uint64_t len;
+		struct {
+			uint16_t len[3];
+			uint16_t valid_segs;
+		} s;
+	} u;
+	uint64_t ptr[3];
+};
+
+#endif /* _ROC_CPT_SG_H_ */
drivers/common/cnxk/roc_ie_on.h +5 -4
index 057ff95362..9933ffa148 100644
@@ -25,10 +25,11 @@ enum roc_ie_on_ucc_ipsec {
 };
 
 /* Helper macros */
-#define ROC_IE_ON_INB_RPTR_HDR 16
-#define ROC_IE_ON_MAX_IV_LEN   16
-#define ROC_IE_ON_PER_PKT_IV   BIT(43)
-#define ROC_IE_ON_INPLACE_BIT  BIT(6)
+#define ROC_IE_ON_OUTB_DPTR_HDR 16
+#define ROC_IE_ON_INB_RPTR_HDR	16
+#define ROC_IE_ON_MAX_IV_LEN	16
+#define ROC_IE_ON_PER_PKT_IV	BIT(43)
+#define ROC_IE_ON_INPLACE_BIT	BIT(6)
 
 enum {
 	ROC_IE_ON_SA_ENC_NULL = 0,
drivers/common/cnxk/roc_se.h +0 -28
index c357c19c0b..a8f0f49479 100644
@@ -27,13 +27,6 @@
 #define ROC_SE_MAX_MAC_LEN  64
 
 #define ROC_SE_OFF_CTRL_LEN 8
-#define ROC_SE_DMA_MODE	    (1 << 7)
-
-#define ROC_SE_MAX_SG_IN_OUT_CNT 32
-#define ROC_SE_MAX_SG_CNT	 (ROC_SE_MAX_SG_IN_OUT_CNT / 2)
-
-#define ROC_SE_SG_LIST_HDR_SIZE (8u)
-#define ROC_SE_SG_ENTRY_SIZE	sizeof(struct roc_se_sglist_comp)
 
 #define ROC_SE_ZS_EA 0x1
 #define ROC_SE_ZS_IA 0x2
@@ -173,27 +166,6 @@ typedef enum {
 	ROC_SE_PDCP_MAC_LEN_128_BIT = 0x3
 } roc_se_pdcp_mac_len_type;
 
-struct roc_se_sglist_comp {
-	union {
-		uint64_t len;
-		struct {
-			uint16_t len[4];
-		} s;
-	} u;
-	uint64_t ptr[4];
-};
-
-struct roc_se_sg2list_comp {
-	union {
-		uint64_t len;
-		struct {
-			uint16_t len[3];
-			uint16_t valid_segs;
-		} s;
-	} u;
-	uint64_t ptr[3];
-};
-
 struct roc_se_enc_context {
 	uint64_t iv_source : 1;
 	uint64_t aes_key : 2;
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +38 -37

125 lines not on this page · raw message

drivers/crypto/cnxk/cn9k_ipsec_la_ops.h +150 -21

216 lines not on this page · raw message

drivers/crypto/cnxk/cnxk_cryptodev_ops.c +21 -4
index a9c42205e6..eb2ed0d103 100644
@@ -30,10 +30,22 @@ cnxk_cpt_get_mlen(void)
 	/* For PDCP_CHAIN passthrough alignment */
 	len += 8;
 	len += ROC_SE_OFF_CTRL_LEN + ROC_CPT_AES_CBC_IV_LEN;
-	len += RTE_ALIGN_CEIL(
-		(ROC_SE_SG_LIST_HDR_SIZE +
-		 (RTE_ALIGN_CEIL(ROC_SE_MAX_SG_IN_OUT_CNT, 4) >> 2) * ROC_SE_SG_ENTRY_SIZE),
-		8);
+	len += RTE_ALIGN_CEIL((ROC_SG_LIST_HDR_SIZE +
+			       (RTE_ALIGN_CEIL(ROC_MAX_SG_IN_OUT_CNT, 4) >> 2) * ROC_SG_ENTRY_SIZE),
+			      8);
+
+	return len;
+}
+
+static int
+cnxk_cpt_sec_get_mlen(void)
+{
+	uint32_t len;
+
+	len = ROC_IE_ON_OUTB_DPTR_HDR + ROC_IE_ON_MAX_IV_LEN;
+	len += RTE_ALIGN_CEIL((ROC_SG_LIST_HDR_SIZE +
+			       (RTE_ALIGN_CEIL(ROC_MAX_SG_IN_OUT_CNT, 4) >> 2) * ROC_SG_ENTRY_SIZE),
+			      8);
 
 	return len;
 }
@@ -196,6 +208,11 @@ cnxk_cpt_metabuf_mempool_create(const struct rte_cryptodev *dev,
 		mlen = cnxk_cpt_get_mlen();
 	}
 
+	if (dev->feature_flags & RTE_CRYPTODEV_FF_SECURITY) {
+		/* Get meta len for security operations */
+		mlen = cnxk_cpt_sec_get_mlen();
+	}
+
 	if (dev->feature_flags & RTE_CRYPTODEV_FF_ASYMMETRIC_CRYPTO) {
 
 		/* Get meta len required for asymmetric operations */
drivers/crypto/cnxk/cnxk_cryptodev_ops.h +19 -0
index 13c90444d6..5153d334ba 100644
@@ -158,4 +158,23 @@ pending_queue_free_cnt(uint64_t head, uint64_t tail, const uint64_t mask)
 	return mask - pending_queue_infl_cnt(head, tail, mask);
 }
 
+static __rte_always_inline void *
+alloc_op_meta(struct roc_se_buf_ptr *buf, int32_t len, struct rte_mempool *cpt_meta_pool,
+	      struct cpt_inflight_req *infl_req)
+{
+	uint8_t *mdata;
+
+	if (unlikely(rte_mempool_get(cpt_meta_pool, (void **)&mdata) < 0))
+		return NULL;
+
+	if (likely(buf)) {
+		buf->vaddr = mdata;
+		buf->size = len;
+	}
+
+	infl_req->mdata = mdata;
+	infl_req->op_flags |= CPT_OP_FLAGS_METABUF;
+
+	return mdata;
+}
 #endif /* _CNXK_CRYPTODEV_OPS_H_ */
drivers/crypto/cnxk/cnxk_se.h +50 -337

606 lines not on this page · raw message

drivers/crypto/cnxk/cnxk_sg.h new file +273 -0

273 lines not on this page · raw message

Signature · 1 line
2.25.1
#10 lore raw

PATCH 10/17 crypto/cnxk: add support for SHA3 hash

In reply to Tejasree Kondoj

From: Volodymyr Fialko <[email protected]>

Add support for SHA3 family hash and hmac operations.

---
 doc/guides/cryptodevs/cnxk.rst                |   8 +
 doc/guides/cryptodevs/features/cn10k.ini      |   8 +
 doc/guides/cryptodevs/features/cn9k.ini       |   8 +
 drivers/crypto/cnxk/cnxk_cryptodev.h          |   2 +-
 .../crypto/cnxk/cnxk_cryptodev_capabilities.c | 164 ++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c      |   2 +-
 drivers/crypto/cnxk/cnxk_se.h                 |  33 +++-
 7 files changed, 222 insertions(+), 3 deletions(-)
doc/guides/cryptodevs/cnxk.rst +8 -0
index baf0e3c4fd..9b01e04e5f 100644
@@ -59,6 +59,14 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA384_HMAC``
 * ``RTE_CRYPTO_AUTH_SHA512``
 * ``RTE_CRYPTO_AUTH_SHA512_HMAC``
+* ``RTE_CRYPTO_AUTH_SHA3_224``
+* ``RTE_CRYPTO_AUTH_SHA3_224_HMAC``
+* ``RTE_CRYPTO_AUTH_SHA3_256``
+* ``RTE_CRYPTO_AUTH_SHA3_256_HMAC``
+* ``RTE_CRYPTO_AUTH_SHA3_384``
+* ``RTE_CRYPTO_AUTH_SHA3_384_HMAC``
+* ``RTE_CRYPTO_AUTH_SHA3_512``
+* ``RTE_CRYPTO_AUTH_SHA3_512_HMAC``
 * ``RTE_CRYPTO_AUTH_SNOW3G_UIA2``
 * ``RTE_CRYPTO_AUTH_ZUC_EIA3``
 * ``RTE_CRYPTO_AUTH_AES_CMAC``
doc/guides/cryptodevs/features/cn10k.ini +8 -0
index 6e4e0e0095..44b61663fc 100644
@@ -63,6 +63,14 @@ ZUC EIA3        = Y
 AES CMAC (128)  = Y
 AES CMAC (192)  = Y
 AES CMAC (256)  = Y
+SHA3_224        = Y
+SHA3_224 HMAC   = Y
+SHA3_256        = Y
+SHA3_256 HMAC   = Y
+SHA3_384        = Y
+SHA3_384 HMAC   = Y
+SHA3_512        = Y
+SHA3_512 HMAC   = Y
 
 ;
 ; Supported AEAD algorithms of 'cn10k' crypto driver.
doc/guides/cryptodevs/features/cn9k.ini +8 -0
index f9c896f6bd..e7b287db26 100644
@@ -64,6 +64,14 @@ ZUC EIA3        = Y
 AES CMAC (128)  = Y
 AES CMAC (192)  = Y
 AES CMAC (256)  = Y
+SHA3_224        = Y
+SHA3_224 HMAC   = Y
+SHA3_256        = Y
+SHA3_256 HMAC   = Y
+SHA3_384        = Y
+SHA3_384 HMAC   = Y
+SHA3_512        = Y
+SHA3_512 HMAC   = Y
 
 ;
 ; Supported AEAD algorithms of 'cn9k' crypto driver.
drivers/crypto/cnxk/cnxk_cryptodev.h +1 -1
index 48bd6e144c..8241ee67d0 100644
@@ -10,7 +10,7 @@
 
 #include "roc_cpt.h"
 
-#define CNXK_CPT_MAX_CAPS	 37
+#define CNXK_CPT_MAX_CAPS	 45
 #define CNXK_SEC_CRYPTO_MAX_CAPS 16
 #define CNXK_SEC_MAX_CAPS	 9
 #define CNXK_AE_EC_ID_MAX	 8
drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c +164 -0

176 lines not on this page · raw message

drivers/crypto/cnxk/cnxk_cryptodev_ops.c +1 -1
index eb2ed0d103..92e8755671 100644
@@ -627,7 +627,7 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt)
 	inst_w7.s.cptr = (uint64_t)&sess->roc_se_ctx.se_ctx;
 
 	/* Set the engine group */
-	if (sess->zsk_flag || sess->aes_ctr_eea2)
+	if (sess->zsk_flag || sess->aes_ctr_eea2 || sess->is_sha3)
 		inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_SE];
 	else
 		inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_IE];
drivers/crypto/cnxk/cnxk_se.h +32 -1
index 88049ac431..092cdd88e7 100644
@@ -39,6 +39,8 @@ struct cnxk_se_sess {
 	uint16_t zs_auth : 4;
 	uint16_t dp_thr_type : 8;
 	uint16_t aad_length;
+	uint8_t is_sha3 : 1;
+	uint8_t rsvd : 7;
 	uint8_t mac_len;
 	uint8_t iv_length;
 	uint8_t auth_iv_length;
@@ -163,18 +165,26 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth)
 		break;
 	case RTE_CRYPTO_AUTH_SHA224:
 	case RTE_CRYPTO_AUTH_SHA224_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_224:
+	case RTE_CRYPTO_AUTH_SHA3_224_HMAC:
 		ret = (mac_len <= 28) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA256:
 	case RTE_CRYPTO_AUTH_SHA256_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_256:
+	case RTE_CRYPTO_AUTH_SHA3_256_HMAC:
 		ret = (mac_len <= 32) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA384:
 	case RTE_CRYPTO_AUTH_SHA384_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_384:
+	case RTE_CRYPTO_AUTH_SHA3_384_HMAC:
 		ret = (mac_len <= 48) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA512:
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_512:
+	case RTE_CRYPTO_AUTH_SHA3_512_HMAC:
 		ret = (mac_len <= 64) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_NULL:
@@ -1848,7 +1858,7 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 static __rte_always_inline int
 fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 {
-	uint8_t zsk_flag = 0, zs_auth = 0, aes_gcm = 0, is_null = 0;
+	uint8_t zsk_flag = 0, zs_auth = 0, aes_gcm = 0, is_null = 0, is_sha3 = 0;
 	struct rte_crypto_auth_xform *a_form;
 	roc_se_auth_type auth_type = 0; /* NULL Auth type */
 
@@ -1900,6 +1910,26 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	case RTE_CRYPTO_AUTH_SHA384:
 		auth_type = ROC_SE_SHA2_SHA384;
 		break;
+	case RTE_CRYPTO_AUTH_SHA3_224_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_224:
+		is_sha3 = 1;
+		auth_type = ROC_SE_SHA3_SHA224;
+		break;
+	case RTE_CRYPTO_AUTH_SHA3_256_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_256:
+		is_sha3 = 1;
+		auth_type = ROC_SE_SHA3_SHA256;
+		break;
+	case RTE_CRYPTO_AUTH_SHA3_384_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_384:
+		is_sha3 = 1;
+		auth_type = ROC_SE_SHA3_SHA384;
+		break;
+	case RTE_CRYPTO_AUTH_SHA3_512_HMAC:
+	case RTE_CRYPTO_AUTH_SHA3_512:
+		is_sha3 = 1;
+		auth_type = ROC_SE_SHA3_SHA512;
+		break;
 	case RTE_CRYPTO_AUTH_MD5_HMAC:
 	case RTE_CRYPTO_AUTH_MD5:
 		auth_type = ROC_SE_MD5_TYPE;
@@ -1959,6 +1989,7 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	sess->aes_gcm = aes_gcm;
 	sess->mac_len = a_form->digest_length;
 	sess->is_null = is_null;
+	sess->is_sha3 = is_sha3;
 	if (zsk_flag) {
 		sess->auth_iv_offset = a_form->iv.offset;
 		sess->auth_iv_length = a_form->iv.length;
Signature · 1 line
2.25.1
#11 lore raw

PATCH 11/17 common/cnxk: skip hmac hash precomputation

In reply to Tejasree Kondoj

From: Volodymyr Fialko <[email protected]>

Operations with FC opcode requires precomputed ipad and opad hashes, but for auth only (HMAC opcode) this is not required, thus could be skipped.

---
 drivers/common/cnxk/roc_se.c | 33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)
drivers/common/cnxk/roc_se.c +20 -13
index 22df61f5f0..aba7f9416d 100644
@@ -308,6 +308,7 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type,
 	struct roc_se_context *fctx;
 	uint8_t opcode_minor;
 	uint8_t pdcp_alg;
+	bool chained_op;
 	int ret;
 
 	if (se_ctx == NULL)
@@ -318,12 +319,12 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type,
 	k_ctx = &se_ctx->se_ctx.k_ctx;
 	fctx = &se_ctx->se_ctx.fctx;
 
+	chained_op = se_ctx->ciph_then_auth || se_ctx->auth_then_ciph;
+
 	if ((type >= ROC_SE_ZUC_EIA3) && (type <= ROC_SE_KASUMI_F9_ECB)) {
 		uint8_t *zuc_const;
 		uint32_t keyx[4];
 		uint8_t *ci_key;
-		bool chained_op =
-			se_ctx->ciph_then_auth || se_ctx->auth_then_ciph;
 
 		if (!key_len)
 			return -1;
@@ -470,19 +471,25 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type,
 	se_ctx->mac_len = mac_len;
 
 	if (key_len) {
-		se_ctx->hmac = 1;
-
-		se_ctx->auth_key = plt_zmalloc(key_len, 8);
-		if (se_ctx->auth_key == NULL)
-			return -1;
+		/*
+		 * Chained operation (FC opcode) requires precomputed ipad and opad hashes, but for
+		 * auth only (HMAC opcode) this is not required
+		 */
+		if (chained_op) {
+			memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad));
+			memset(fctx->hmac.opad, 0, sizeof(fctx->hmac.opad));
+			cpt_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac);
+			fctx->enc.auth_input_type = 0;
+		} else {
+			se_ctx->hmac = 1;
 
-		memcpy(se_ctx->auth_key, key, key_len);
-		se_ctx->auth_key_len = key_len;
-		memset(fctx->hmac.ipad, 0, sizeof(fctx->hmac.ipad));
-		memset(fctx->hmac.opad, 0, sizeof(fctx->hmac.opad));
+			se_ctx->auth_key = plt_zmalloc(key_len, 8);
+			if (se_ctx->auth_key == NULL)
+				return -1;
 
-		cpt_hmac_opad_ipad_gen(type, key, key_len, &fctx->hmac);
-		fctx->enc.auth_input_type = 0;
+			memcpy(se_ctx->auth_key, key, key_len);
+			se_ctx->auth_key_len = key_len;
+		}
 	}
 	return 0;
 }
Signature · 1 line
2.25.1
#12 lore raw

PATCH 12/17 crypto/octeontx: support truncated digest size

In reply to Tejasree Kondoj

Support truncated digest size for auth only mode.

---
 drivers/common/cpt/cpt_ucode.h                | 17 ++++---
 .../octeontx/otx_cryptodev_capabilities.c     | 48 +++++++++----------
 2 files changed, 34 insertions(+), 31 deletions(-)
drivers/common/cpt/cpt_ucode.h +10 -7
index 22aabab6ac..b393be4cf6 100644
@@ -41,30 +41,33 @@ cpt_mac_len_verify(struct rte_crypto_auth_xform *auth)
 	uint16_t mac_len = auth->digest_length;
 	int ret;
 
+	if ((auth->algo != RTE_CRYPTO_AUTH_NULL) && (mac_len == 0))
+		return -1;
+
 	switch (auth->algo) {
 	case RTE_CRYPTO_AUTH_MD5:
 	case RTE_CRYPTO_AUTH_MD5_HMAC:
-		ret = (mac_len == 16) ? 0 : -1;
+		ret = (mac_len <= 16) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA1:
 	case RTE_CRYPTO_AUTH_SHA1_HMAC:
-		ret = (mac_len == 20) ? 0 : -1;
+		ret = (mac_len <= 20) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA224:
 	case RTE_CRYPTO_AUTH_SHA224_HMAC:
-		ret = (mac_len == 28) ? 0 : -1;
+		ret = (mac_len <= 28) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA256:
 	case RTE_CRYPTO_AUTH_SHA256_HMAC:
-		ret = (mac_len == 32) ? 0 : -1;
+		ret = (mac_len <= 32) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA384:
 	case RTE_CRYPTO_AUTH_SHA384_HMAC:
-		ret = (mac_len == 48) ? 0 : -1;
+		ret = (mac_len <= 48) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_SHA512:
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
-		ret = (mac_len == 64) ? 0 : -1;
+		ret = (mac_len <= 64) ? 0 : -1;
 		break;
 	case RTE_CRYPTO_AUTH_NULL:
 		ret = 0;
@@ -523,7 +526,7 @@ cpt_digest_gen_prep(uint32_t flags,
 
 	/*GP op header */
 	vq_cmd_w0.s.opcode.minor = 0;
-	vq_cmd_w0.s.param2 = ((uint16_t)hash_type << 8);
+	vq_cmd_w0.s.param2 = ((uint16_t)hash_type << 8) | mac_len;
 	if (ctx->hmac) {
 		vq_cmd_w0.s.opcode.major = CPT_MAJOR_OP_HMAC | CPT_DMA_MODE;
 		vq_cmd_w0.s.param1 = key_len;
drivers/crypto/octeontx/otx_cryptodev_capabilities.c +24 -24

132 lines not on this page · raw message

Signature · 1 line
2.25.1
#13 lore raw

PATCH 13/17 crypto/cnxk: set device ops to null in PCI remove

In reply to Tejasree Kondoj

Setting dev_ops to NULL in pci_remove to avoid
device close after removal.

---
 drivers/crypto/cnxk/cn10k_cryptodev.c | 1 +
 drivers/crypto/cnxk/cn9k_cryptodev.c  | 1 +
 2 files changed, 2 insertions(+)
drivers/crypto/cnxk/cn10k_cryptodev.c +1 -0
index 52de9b9657..2fd4df3c5d 100644
@@ -138,6 +138,7 @@ cn10k_cpt_pci_remove(struct rte_pci_device *pci_dev)
 	cnxk_crypto_sec_ctx_destroy(dev);
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+		dev->dev_ops = NULL;
 		vf = dev->data->dev_private;
 		ret = roc_cpt_dev_fini(&vf->cpt);
 		if (ret)
drivers/crypto/cnxk/cn9k_cryptodev.c +1 -0
index 4cfc1f2150..47b0874185 100644
@@ -138,6 +138,7 @@ cn9k_cpt_pci_remove(struct rte_pci_device *pci_dev)
 	cnxk_crypto_sec_ctx_destroy(dev);
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+		dev->dev_ops = NULL;
 		vf = dev->data->dev_private;
 		ret = roc_cpt_dev_fini(&vf->cpt);
 		if (ret)
Signature · 1 line
2.25.1
#14 lore raw

PATCH 14/17 crypto/cnxk: add CTX for non IPsec operations

In reply to Tejasree Kondoj

Support context cache with non IPsec operations.

---
 drivers/common/cnxk/roc_platform.h        |  3 +-
 drivers/common/cnxk/roc_se.c              | 47 +++++++++++++++++++++++
 drivers/common/cnxk/roc_se.h              | 42 +++++++++++++-------
 drivers/common/cnxk/version.map           |  1 +
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 17 ++++----
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  | 14 +++----
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c  | 38 +++++++++++++-----
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h  | 13 +++----
 drivers/crypto/cnxk/cnxk_se.h             | 10 +++--
 9 files changed, 134 insertions(+), 51 deletions(-)
drivers/common/cnxk/roc_platform.h +2 -1
index 1a48ff3db4..8ba28e69fa 100644
@@ -57,9 +57,10 @@
 #define PLT_ALIGN		 RTE_ALIGN
 #define PLT_ALIGN_MUL_CEIL	 RTE_ALIGN_MUL_CEIL
 #define PLT_MODEL_MZ_NAME	 "roc_model_mz"
-#define PLT_CACHE_LINE_SIZE      RTE_CACHE_LINE_SIZE
+#define PLT_CACHE_LINE_SIZE	 RTE_CACHE_LINE_SIZE
 #define BITMASK_ULL		 GENMASK_ULL
 #define PLT_ALIGN_CEIL		 RTE_ALIGN_CEIL
+#define PLT_ALIGN_FLOOR		 RTE_ALIGN_FLOOR
 #define PLT_INIT		 RTE_INIT
 #define PLT_MAX_ETHPORTS	 RTE_MAX_ETHPORTS
 #define PLT_TAILQ_FOREACH_SAFE	 RTE_TAILQ_FOREACH_SAFE
drivers/common/cnxk/roc_se.c +47 -0

50 lines not on this page · raw message

drivers/common/cnxk/roc_se.h +29 -13

57 lines not on this page · raw message

drivers/common/cnxk/version.map +1 -0
index 17f0ec6b48..ee283d2392 100644
@@ -99,6 +99,7 @@ INTERNAL {
 	roc_model;
 	roc_se_auth_key_set;
 	roc_se_ciph_key_set;
+	roc_se_ctx_init;
 	roc_nix_bpf_alloc;
 	roc_nix_bpf_config;
 	roc_nix_bpf_connect;
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +8 -9
index 5a098ffcf2..57b44210c2 100644
@@ -52,14 +52,13 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 	if (rte_mempool_get(qp->sess_mp, (void **)&sess) < 0)
 		return NULL;
 
-	ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform,
-				    sess);
+	ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess, true);
 	if (ret) {
 		rte_mempool_put(qp->sess_mp, (void *)sess);
 		goto sess_put;
 	}
 
-	priv = (void *)sess->driver_priv_data;
+	priv = (void *)sess;
 	sym_op->session = sess;
 
 	return priv;
@@ -121,13 +120,13 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct
 
 	if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) {
 		if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
-			sec_sess = (struct cn10k_sec_session *)(sym_op->session);
+			sec_sess = (struct cn10k_sec_session *)sym_op->session;
 			ret = cpt_sec_inst_fill(qp, op, sec_sess, &inst[0]);
 			if (unlikely(ret))
 				return 0;
 			w7 = sec_sess->inst.w7;
 		} else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) {
-			sess = CRYPTODEV_GET_SYM_SESS_PRIV(sym_op->session);
+			sess = (struct cnxk_se_sess *)(sym_op->session);
 			ret = cpt_sym_inst_fill(qp, op, sess, infl_req, &inst[0], is_sg_ver2);
 			if (unlikely(ret))
 				return 0;
@@ -141,7 +140,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], struct
 
 			ret = cpt_sym_inst_fill(qp, op, sess, infl_req, &inst[0], is_sg_ver2);
 			if (unlikely(ret)) {
-				sym_session_clear(op->sym->session);
+				sym_session_clear(op->sym->session, true);
 				rte_mempool_put(qp->sess_mp, op->sym->session);
 				return 0;
 			}
@@ -316,7 +315,7 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, vo
 		} else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) {
 			struct cnxk_se_sess *priv;
 
-			priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess);
+			priv = (struct cnxk_se_sess *)sess;
 			priv->qp = qp;
 			priv->cpt_inst_w2 = w2;
 		} else
@@ -351,7 +350,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, struct cnxk_cpt_qp **qp, ui
 		} else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) {
 			struct cnxk_se_sess *priv;
 
-			priv = CRYPTODEV_GET_SYM_SESS_PRIV(op->sym->session);
+			priv = (struct cnxk_se_sess *)op->sym->session;
 			*qp = priv->qp;
 			*w2 = priv->cpt_inst_w2;
 		} else {
@@ -914,7 +913,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 temp_sess_free:
 	if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) {
 		if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) {
-			sym_session_clear(cop->sym->session);
+			sym_session_clear(cop->sym->session, true);
 			rte_mempool_put(qp->sess_mp, cop->sym->session);
 			cop->sym->session = NULL;
 		}
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +7 -7
index cfe1e08dff..e3784e34c9 100644
@@ -49,11 +49,11 @@ cn9k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 	if (rte_mempool_get(qp->sess_mp, (void **)&sess) < 0)
 		return NULL;
 
-	ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess);
+	ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess, true);
 	if (ret)
 		goto sess_put;
 
-	priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess);
+	priv = (struct cnxk_se_sess *)sess;
 
 	sym_op->session = sess;
 
@@ -76,7 +76,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
 
 		if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) {
 			sym_op = op->sym;
-			sess = CRYPTODEV_GET_SYM_SESS_PRIV(sym_op->session);
+			sess = (struct cnxk_se_sess *)sym_op->session;
 			ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst, false);
 			inst->w7.u64 = sess->cpt_inst_w7;
 		} else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION)
@@ -90,7 +90,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
 
 			ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst, false);
 			if (unlikely(ret)) {
-				sym_session_clear(op->sym->session);
+				sym_session_clear(op->sym->session, true);
 				rte_mempool_put(qp->sess_mp, op->sym->session);
 			}
 			inst->w7.u64 = sess->cpt_inst_w7;
@@ -326,7 +326,7 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused,
 		} else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) {
 			struct cnxk_se_sess *priv;
 
-			priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess);
+			priv = (struct cnxk_se_sess *)sess;
 			priv->qp = qp;
 			priv->cpt_inst_w2 = w2;
 		} else
@@ -361,7 +361,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op,
 		} else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) {
 			struct cnxk_se_sess *priv;
 
-			priv = CRYPTODEV_GET_SYM_SESS_PRIV(op->sym->session);
+			priv = (struct cnxk_se_sess *)op->sym->session;
 			*qp = priv->qp;
 			inst->w2.u64 = priv->cpt_inst_w2;
 		} else {
@@ -630,7 +630,7 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,
 temp_sess_free:
 	if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) {
 		if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) {
-			sym_session_clear(cop->sym->session);
+			sym_session_clear(cop->sym->session, true);
 			rte_mempool_put(qp->sess_mp, cop->sym->session);
 			cop->sym->session = NULL;
 		}
drivers/crypto/cnxk/cnxk_cryptodev_ops.c +28 -10
index 92e8755671..2e845afce9 100644
@@ -626,6 +626,11 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt)
 
 	inst_w7.s.cptr = (uint64_t)&sess->roc_se_ctx.se_ctx;
 
+	if (roc_cpt->cpt_revision == ROC_CPT_REVISION_ID_106XX)
+		inst_w7.s.ctx_val = 1;
+	else
+		inst_w7.s.cptr += 8;
+
 	/* Set the engine group */
 	if (sess->zsk_flag || sess->aes_ctr_eea2 || sess->is_sha3)
 		inst_w7.s.egrp = roc_cpt->eng_grp[CPT_ENG_TYPE_SE];
@@ -636,19 +641,22 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt)
 }
 
 int
-sym_session_configure(struct roc_cpt *roc_cpt,
-		      struct rte_crypto_sym_xform *xform,
-		      struct rte_cryptodev_sym_session *sess)
+sym_session_configure(struct roc_cpt *roc_cpt, struct rte_crypto_sym_xform *xform,
+		      struct rte_cryptodev_sym_session *sess, bool is_session_less)
 {
 	enum cpt_dp_thread_type thr_type;
-	struct cnxk_se_sess *sess_priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess);
+	struct cnxk_se_sess *sess_priv = (struct cnxk_se_sess *)sess;
 	int ret;
 
-	memset(sess_priv, 0, sizeof(struct cnxk_se_sess));
+	if (is_session_less)
+		memset(sess_priv, 0, sizeof(struct cnxk_se_sess));
+
 	ret = cnxk_sess_fill(roc_cpt, xform, sess_priv);
 	if (ret)
 		goto priv_put;
 
+	sess_priv->lf = roc_cpt->lf[0];
+
 	if (sess_priv->cpt_op & ROC_SE_OP_CIPHER_MASK) {
 		switch (sess_priv->roc_se_ctx.fc_type) {
 		case ROC_SE_FC_GEN:
@@ -690,6 +698,10 @@ sym_session_configure(struct roc_cpt *roc_cpt,
 	}
 
 	sess_priv->cpt_inst_w7 = cnxk_cpt_inst_w7_get(sess_priv, roc_cpt);
+
+	if (roc_cpt->cpt_revision == ROC_CPT_REVISION_ID_106XX)
+		roc_se_ctx_init(&sess_priv->roc_se_ctx);
+
 	return 0;
 
 priv_put:
@@ -704,25 +716,31 @@ cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev,
 	struct cnxk_cpt_vf *vf = dev->data->dev_private;
 	struct roc_cpt *roc_cpt = &vf->cpt;
 
-	return sym_session_configure(roc_cpt, xform, sess);
+	return sym_session_configure(roc_cpt, xform, sess, false);
 }
 
 void
-sym_session_clear(struct rte_cryptodev_sym_session *sess)
+sym_session_clear(struct rte_cryptodev_sym_session *sess, bool is_session_less)
 {
-	struct cnxk_se_sess *sess_priv = CRYPTODEV_GET_SYM_SESS_PRIV(sess);
+	struct cnxk_se_sess *sess_priv = (struct cnxk_se_sess *)sess;
+
+	/* Trigger CTX flush + invalidate to remove from CTX_CACHE */
+	roc_cpt_lf_ctx_flush(sess_priv->lf, &sess_priv->roc_se_ctx.se_ctx, true);
+
+	plt_delay_ms(1);
 
 	if (sess_priv->roc_se_ctx.auth_key != NULL)
 		plt_free(sess_priv->roc_se_ctx.auth_key);
 
-	memset(sess_priv, 0, cnxk_cpt_sym_session_get_size(NULL));
+	if (is_session_less)
+		memset(sess_priv, 0, cnxk_cpt_sym_session_get_size(NULL));
 }
 
 void
 cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev __rte_unused,
 			   struct rte_cryptodev_sym_session *sess)
 {
-	return sym_session_clear(sess);
+	return sym_session_clear(sess, false);
 }
 
 unsigned int
drivers/crypto/cnxk/cnxk_cryptodev_ops.h +5 -8
index 5153d334ba..f91ad368ea 100644
@@ -107,18 +107,15 @@ int cnxk_cpt_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id);
 
 unsigned int cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev);
 
-int cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev,
-				   struct rte_crypto_sym_xform *xform,
+int cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform,
 				   struct rte_cryptodev_sym_session *sess);
 
-int sym_session_configure(struct roc_cpt *roc_cpt,
-			  struct rte_crypto_sym_xform *xform,
-			  struct rte_cryptodev_sym_session *sess);
+int sym_session_configure(struct roc_cpt *roc_cpt, struct rte_crypto_sym_xform *xform,
+			  struct rte_cryptodev_sym_session *sess, bool is_session_less);
 
-void cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev,
-		struct rte_cryptodev_sym_session *sess);
+void cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess);
 
-void sym_session_clear(struct rte_cryptodev_sym_session *sess);
+void sym_session_clear(struct rte_cryptodev_sym_session *sess, bool is_session_less);
 
 unsigned int cnxk_ae_session_size_get(struct rte_cryptodev *dev __rte_unused);
 
drivers/crypto/cnxk/cnxk_se.h +7 -3
index 092cdd88e7..774efd5879 100644
@@ -6,6 +6,8 @@
 #define _CNXK_SE_H_
 #include <stdbool.h>
 
+#include <rte_cryptodev.h>
+
 #include "cnxk_cryptodev.h"
 #include "cnxk_cryptodev_ops.h"
 #include "cnxk_sg.h"
@@ -25,6 +27,7 @@ enum cpt_dp_thread_type {
 };
 
 struct cnxk_se_sess {
+	struct rte_cryptodev_sym_session rte_sess;
 	uint16_t cpt_op : 4;
 	uint16_t zsk_flag : 4;
 	uint16_t aes_gcm : 1;
@@ -51,10 +54,11 @@ struct cnxk_se_sess {
 	uint64_t cpt_inst_w2;
 	struct cnxk_cpt_qp *qp;
 	struct roc_se_ctx roc_se_ctx;
-} __rte_cache_aligned;
+	struct roc_cpt_lf *lf;
+} __rte_aligned(ROC_ALIGN);
 
-static __rte_always_inline int
-fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess);
+static __rte_always_inline int fill_sess_gmac(struct rte_crypto_sym_xform *xform,
+					      struct cnxk_se_sess *sess);
 
 static inline void
 cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst)
Signature · 1 line
2.25.1
#15 lore raw

PATCH 15/17 crypto/cnxk: set salt in dptr as part of IV

In reply to Tejasree Kondoj

Set salt as part of 16B IV in dptr to avoid race
condition.

---
 drivers/common/cnxk/roc_se.c  |  7 +++-
 drivers/crypto/cnxk/cnxk_se.h | 76 +++++++++++++++++++----------------
 2 files changed, 47 insertions(+), 36 deletions(-)
drivers/common/cnxk/roc_se.c +6 -1
index 8c19c5fccc..f335c2367f 100644
@@ -462,8 +462,10 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type,
 		return -1;
 
 	/* For GMAC auth, cipher must be NULL */
-	if (type == ROC_SE_GMAC_TYPE)
+	if (type == ROC_SE_GMAC_TYPE) {
 		fctx->enc.enc_cipher = 0;
+		se_ctx->template_w4.s.opcode_minor = BIT(5);
+	}
 
 	fctx->enc.hash_type = type;
 	se_ctx->hash_type = type;
@@ -530,6 +532,9 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type,
 			return 0;
 	}
 
+	if (type == ROC_SE_AES_GCM)
+		se_ctx->template_w4.s.opcode_minor = BIT(5);
+
 	ret = cpt_ciph_type_set(type, se_ctx, key_len);
 	if (unlikely(ret))
 		return -1;
drivers/crypto/cnxk/cnxk_se.h +41 -35

136 lines not on this page · raw message

Signature · 1 line
2.25.1
#16 lore raw

PATCH 16/17 crypto/cnxk: remove null check of session priv

In reply to Tejasree Kondoj

From: Anoob Joseph <[email protected]>

Session private data is at an offset of session. This can never be NULL. Remove check for the same.

---
 drivers/crypto/cnxk/cn10k_ipsec.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/crypto/cnxk/cn10k_ipsec.c +1 -3
index aafd461436..ffd3f50eed 100644
@@ -422,14 +422,12 @@ cn10k_sec_session_update(void *device, struct rte_security_session *sess,
 			 struct rte_security_session_conf *conf)
 {
 	struct rte_cryptodev *crypto_dev = device;
-	struct cn10k_sec_session *priv;
 	struct roc_cpt *roc_cpt;
 	struct cnxk_cpt_qp *qp;
 	struct cnxk_cpt_vf *vf;
 	int ret;
 
-	priv = SECURITY_GET_SESS_PRIV(sess);
-	if (priv == NULL)
+	if (sess == NULL)
 		return -EINVAL;
 
 	qp = crypto_dev->data->queue_pairs[0];
Signature · 1 line
2.25.1
#17 lore raw

PATCH 17/17 common/cnxk: remove salt from session

In reply to Tejasree Kondoj

From: Anoob Joseph <[email protected]>

Complete 16B IV can be passed in DPTR to microcode. Passing salt in CPTR cannot be supported as the salt need to be given in different formats when CTX is enabled.

---
 drivers/common/cnxk/roc_se.c  | 17 +++--------------
 drivers/common/cnxk/roc_se.h  |  2 +-
 drivers/crypto/cnxk/cnxk_se.h |  6 +++---
 3 files changed, 7 insertions(+), 18 deletions(-)
drivers/common/cnxk/roc_se.c +3 -14
index f335c2367f..6f0a81b22c 100644
@@ -497,8 +497,9 @@ roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type type,
 }
 
 int
-roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type,
-		    const uint8_t *key, uint16_t key_len, uint8_t *salt)
+roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type, const uint8_t *key,
+		    uint16_t key_len)
+
 {
 	bool chained_op = se_ctx->ciph_then_auth || se_ctx->auth_then_ciph;
 	struct roc_se_zuc_snow3g_ctx *zs_ctx = &se_ctx->se_ctx.zs_ctx;
@@ -520,18 +521,6 @@ roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type,
 		zuc_const = zs_ctx->zuc.otk_ctx.zuc_const;
 	}
 
-	/* For AES-GCM, salt is taken from ctx even if IV source
-	 * is from DPTR
-	 */
-	if ((salt != NULL) && (type == ROC_SE_AES_GCM)) {
-		memcpy(fctx->enc.encr_iv, salt, 4);
-		/* Assuming it was just salt update
-		 * and nothing else
-		 */
-		if (key == NULL)
-			return 0;
-	}
-
 	if (type == ROC_SE_AES_GCM)
 		se_ctx->template_w4.s.opcode_minor = BIT(5);
 
drivers/common/cnxk/roc_se.h +1 -1
index d1ef71577c..e9415f21a5 100644
@@ -370,7 +370,7 @@ int __roc_api roc_se_auth_key_set(struct roc_se_ctx *se_ctx, roc_se_auth_type ty
 				  const uint8_t *key, uint16_t key_len, uint16_t mac_len);
 
 int __roc_api roc_se_ciph_key_set(struct roc_se_ctx *se_ctx, roc_se_cipher_type type,
-				  const uint8_t *key, uint16_t key_len, uint8_t *salt);
+				  const uint8_t *key, uint16_t key_len);
 
 void __roc_api roc_se_ctx_swap(struct roc_se_ctx *se_ctx);
 void __roc_api roc_se_ctx_init(struct roc_se_ctx *se_ctx);
drivers/crypto/cnxk/cnxk_se.h +3 -3
index 1827b0d7b3..0e5d2dde39 100644
@@ -1708,7 +1708,7 @@ fill_sess_aead(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	}
 
 	if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, aead_form->key.data,
-					 aead_form->key.length, NULL)))
+					 aead_form->key.length)))
 		return -1;
 
 	if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, NULL, 0,
@@ -1866,7 +1866,7 @@ fill_sess_cipher(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 		}
 
 	if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, c_form->key.data,
-					 c_form->key.length, NULL)))
+					 c_form->key.length)))
 		return -1;
 
 	if ((enc_type >= ROC_SE_ZUC_EEA3) && (enc_type <= ROC_SE_AES_CTR_EEA2))
@@ -2072,7 +2072,7 @@ fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	}
 
 	if (unlikely(roc_se_ciph_key_set(&sess->roc_se_ctx, enc_type, a_form->key.data,
-					 a_form->key.length, NULL)))
+					 a_form->key.length)))
 		return -1;
 
 	if (unlikely(roc_se_auth_key_set(&sess->roc_se_ctx, auth_type, NULL, 0,
Signature · 1 line
2.25.1
#18 lore raw
In reply to Tejasree Kondoj
Quoted 30 lines

Subject: [PATCH 00/17] fixes and improvements to cnxk crytpo PMD

This series adds improvements and support for SHA3, IPsec scatter gather mode in cnxk crypto PMD.

Aakash Sasidharan (1):
common/cnxk: generate opad and ipad in driver

Anoob Joseph (3):
common/cnxk: perform LF fini ops only when allocated crypto/cnxk: remove null check of session priv common/cnxk: remove salt from session

Archana Muniganti (1):
crypto/cnxk: add CN9K IPsec SG support

Gowrishankar Muthukrishnan (1):
crypto/cnxk: fix incorrect digest for an empty input data

Tejasree Kondoj (5):
crypto/cnxk: support truncated digest length crypto/octeontx: support truncated digest size crypto/cnxk: set device ops to null in PCI remove crypto/cnxk: add CTX for non IPsec operations crypto/cnxk: set salt in dptr as part of IV

Vidya Sagar Velumuri (2):
crypto/cnxk: update resp len calculation for IPv6 crypto/cnxk: update crypto completion code handling

Quoted 6 lines

Volodymyr Fialko (4):
crypto/cnxk: add context to passthrough instruction crypto/cnxk: add queue pair check to meta set crypto/cnxk: add support for SHA3 hash common/cnxk: skip hmac hash precomputation

Series Acked-by: Akhil Goyal <[email protected]>

Fixed couple of formatting issues while applying.
Applied to dpdk-next-crypto