Skip to contentSkip to conversation

cryptodev: add data-path service APIs

PATCH Superseded Fan Zhang 5 participants 84 messages Aug 18, 2020 – Oct 12, 2020 Page 1 of 2

Subsystems (2)
lore
Share & tools
lore

Fan Zhang

author
#0 lore raw

v6 cover cryptodev: add data-path service APIs

Direct crypto data-path service are a set of APIs that especially provided for the external libraries/applications who want to take advantage of the rich features provided by cryptodev, but not necessarily depend on cryptodev operations, mempools, or mbufs in the their data-path implementations.

The direct crypto data-path service has the following advantages:
- Supports raw data pointer and physical addresses as input.
- Do not require specific data structure allocated from heap, such as cryptodev operation.
- Enqueue in a burst or single operation. The service allow enqueuing in a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only enqueue one job at a time but maintaining necessary context data locally for next single job enqueue operation. The latter method is especially helpful when the user application's crypto operations are clustered into a burst. Allowing enqueue one operation at a time helps reducing one additional loop and also reduced the cache misses during the double "looping" situation.
- Customerizable dequeue count. Instead of dequeue maximum possible operations as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the user to provide a callback function to decide how many operations to be dequeued. This is especially helpful when the expected dequeue count is hidden inside the opaque data stored during enqueue. The user can provide the callback function to parse the opaque data structure.
- Abandon enqueue and dequeue anytime. One of the drawbacks of ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst`` operations are: once an operation is enqueued/dequeued there is no way to undo the operation. The service make the operation abandon possible by creating a local copy of the queue operation data in the service context data. The data will be written back to driver maintained operation data when enqueue or dequeue done function is called.

The cryptodev data-path service uses

Cryptodev PMDs who supports this feature will have ``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should be called to get the data path service context data size. The user should creates a local buffer at least this size long and initialize it using ``rte_cryptodev_dp_configure_service`` function call.

The ``rte_cryptodev_dp_configure_service`` function call initialize or updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the driver specific queue pair data pointer and service context buffer, and a set of function pointers to enqueue and dequeue different algorithms' operations. The ``rte_cryptodev_dp_configure_service`` should be called when:

- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
- When different cryptodev session, security session, or session-less xform is used (set ``is_update`` parameter to 1).

Two different enqueue functions are provided.

- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in the ``rte_crypto_sym_vec`` structure.
- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

Either enqueue functions will not command the crypto device to start processing until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user shall expect the driver only stores the necessory context data in the ``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user wants to abandon the submitted operations, simply call ``rte_cryptodev_dp_configure_service`` function instead with the parameter ``is_update`` set to 0. The driver will recover the service context data to the previous state.

To dequeue the operations the user also have two operations:

- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The user needs to provide the callback function for the driver to get the dequeue count and perform post processing such as write the status field.
- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.

Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to merge user's local service context data with the driver's queue operation data. Also to abandon the dequeue operation (still keep the operations in the queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call but calling ``rte_cryptodev_dp_configure_service`` function with the parameter ``is_update`` set to 0.

There are a few limitations to the data path service:

* Only support in-place operations.
* APIs are NOT thread-safe.
* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or vice versa.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: add crypto data-path service APIs crypto/qat: add crypto data-path service API support test/crypto: add unit-test for cryptodev direct APIs
doc: add cryptodev service APIs guide

 app/test/test_cryptodev.c                     | 354 ++++++-
 app/test/test_cryptodev.h                     |   6 +
 app/test/test_cryptodev_blockcipher.c         |  50 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  90 ++
 drivers/common/qat/Makefile                   |   1 +
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 923 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +
 lib/librte_cryptodev/rte_cryptodev.h          | 335 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 +-
 .../rte_cryptodev_version.map                 |  10 +
 15 files changed, 1889 insertions(+), 48 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH v6 1/4 cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 481 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +9 -0
index fd5ef3a87..f009be9af 100644
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct rte_crypto_op *op,
 	return 0;
 }
 
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_crypto_sym.h +38 -6

56 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.c +45 -0

51 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +334 -1

347 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +45 -2

59 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 02f6dcf72..d384382d3 100644
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;
+	rte_cryptodev_get_dp_service_ctx_data_size;
+	rte_cryptodev_dp_submit_single_job;
+	rte_cryptodev_dp_sym_submit_vec;
+	rte_cryptodev_dp_submit_done;
+	rte_cryptodev_dp_sym_dequeue;
+	rte_cryptodev_dp_sym_dequeue_single_job;
+	rte_cryptodev_dp_dequeue_done;
 };
Signature · 1 line
2.20.1
author
#2 lore raw

PATCH v6 2/4 crypto/qat: add crypto data-path service API support

In reply to Fan Zhang

This patch updates QAT PMD to add crypto service API support.

---
 drivers/common/qat/Makefile        |   1 +
 drivers/crypto/qat/meson.build     |   1 +
 drivers/crypto/qat/qat_sym.h       |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c | 923 +++++++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c   |   9 +-
 5 files changed, 945 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
drivers/common/qat/Makefile +1 -0
index 85d420709..1b71bbbab 100644
@@ -42,6 +42,7 @@ endif
 	SRCS-y += qat_sym.c
 	SRCS-y += qat_sym_session.c
 	SRCS-y += qat_sym_pmd.c
+	SRCS-y += qat_sym_hw_dp.c
 	build_qat = yes
 endif
 endif
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +13 -0
index 1a9748849..2d6316130 100644
@@ -264,6 +264,18 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_dp_configure_service_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_dp_service_ctx *service_ctx,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	uint8_t is_update);
+
+int
+qat_sym_get_service_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +288,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +923 -0

923 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..bef08c3bc 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Data plane service related operations */
+		.get_drv_ctx_size = qat_sym_get_service_ctx_size,
+		.configure_service = qat_sym_dp_configure_service_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_DATA_PLANE_SERVICE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#3 lore raw

PATCH v6 3/4 test/crypto: add unit-test for cryptodev direct APIs

In reply to Fan Zhang

This patch adds the QAT test to use cryptodev symmetric crypto direct APIs.

---
 app/test/test_cryptodev.c             | 354 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |   6 +
 app/test/test_cryptodev_blockcipher.c |  50 ++--
 3 files changed, 373 insertions(+), 37 deletions(-)
app/test/test_cryptodev.c +333 -21
index 70bf6fe2c..d6909984d 100644
@@ -49,6 +49,8 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +59,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+int hw_dp_test;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +151,153 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_data iv_vec, aad_vec, digest_vec;
+	union rte_crypto_sym_ofs ofs;
+	int32_t status;
+	uint32_t min_ofs, max_len;
+	union rte_cryptodev_session_ctx sess;
+	enum rte_crypto_dp_service service_type;
+	uint32_t count = 0;
+	uint8_t service_data[MAX_DRV_SERVICE_CTX_SIZE] = {0};
+	struct rte_crypto_dp_service_ctx *ctx = (void *)service_data;
+	int ctx_service_size;
+
+	sop = op->sym;
+
+	sess.crypto_sess = sop->session;
+
+	if (is_cipher && is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_CHAIN;
+		min_ofs = RTE_MIN(sop->cipher.data.offset,
+				sop->auth.data.offset);
+		max_len = RTE_MAX(sop->cipher.data.length,
+				sop->auth.data.length);
+	} else if (is_cipher) {
+		service_type = RTE_CRYPTO_DP_SYM_CIPHER_ONLY;
+		min_ofs = sop->cipher.data.offset;
+		max_len = sop->cipher.data.length;
+	} else if (is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_AUTH_ONLY;
+		min_ofs = sop->auth.data.offset;
+		max_len = sop->auth.data.length;
+	} else { /* aead */
+		service_type = RTE_CRYPTO_DP_SYM_AEAD;
+		min_ofs = sop->aead.data.offset;
+		max_len = sop->aead.data.length;
+	}
+
+	if (len_in_bits) {
+		max_len = max_len >> 3;
+		min_ofs = min_ofs >> 3;
+	}
+
+	ctx_service_size = rte_cryptodev_get_dp_service_ctx_data_size(dev_id);
+	assert(ctx_service_size <= MAX_DRV_SERVICE_CTX_SIZE &&
+			ctx_service_size > 0);
+
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* test update service */
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 1) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, min_ofs + max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n != sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ofs.raw = 0;
+
+	iv_vec.base = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	iv_vec.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	switch (service_type) {
+	case RTE_CRYPTO_DP_SYM_AEAD:
+		ofs.ofs.cipher.head = sop->cipher.data.offset;
+		aad_vec.base = (void *)sop->aead.aad.data;
+		aad_vec.iova = sop->aead.aad.phys_addr;
+		digest_vec.base = (void *)sop->aead.digest.data;
+		digest_vec.iova = sop->aead.digest.phys_addr;
+		if (len_in_bits) {
+			ofs.ofs.cipher.head >>= 3;
+			ofs.ofs.cipher.tail >>= 3;
+		}
+		break;
+	case RTE_CRYPTO_DP_SYM_CIPHER_ONLY:
+		ofs.ofs.cipher.head = sop->cipher.data.offset;
+		if (len_in_bits) {
+			ofs.ofs.cipher.head >>= 3;
+			ofs.ofs.cipher.tail >>= 3;
+		}
+		break;
+	case RTE_CRYPTO_DP_SYM_AUTH_ONLY:
+		ofs.ofs.auth.head = sop->auth.data.offset;
+		digest_vec.base = (void *)sop->auth.digest.data;
+		digest_vec.iova = sop->auth.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CHAIN:
+		ofs.ofs.cipher.head =
+			sop->cipher.data.offset - sop->auth.data.offset;
+		ofs.ofs.cipher.tail =
+			(sop->auth.data.offset + sop->auth.data.length) -
+			(sop->cipher.data.offset + sop->cipher.data.length);
+		if (len_in_bits) {
+			ofs.ofs.cipher.head >>= 3;
+			ofs.ofs.cipher.tail >>= 3;
+		}
+		digest_vec.base = (void *)sop->auth.digest.data;
+		digest_vec.iova = sop->auth.digest.phys_addr;
+		break;
+	default:
+		break;
+	}
+
+	status = rte_cryptodev_dp_submit_single_job(ctx, data_vec, n, ofs,
+		&iv_vec, &digest_vec, &aad_vec, (void *)op);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	rte_cryptodev_dp_submit_done(ctx, 1);
+
+	status = -1;
+	while (count++ < 1024 && status == -1) {
+		status = rte_cryptodev_dp_sym_dequeue_single_job(ctx,
+				(void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}
+
+	if (status != -1)
+		rte_cryptodev_dp_dequeue_done(ctx, 1);
+
+	if (count == 1025 || status != 1 || ret_op != op) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	op->status = status == 1 ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -2470,7 +2621,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2549,7 +2704,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2619,6 +2778,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2690,7 +2852,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2897,8 +3063,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2983,7 +3153,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3306,7 +3480,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3381,7 +3559,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3756,7 +3938,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3924,7 +4110,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4019,7 +4209,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4155,7 +4349,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4344,7 +4542,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4526,7 +4728,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4716,7 +4922,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4857,7 +5067,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4944,7 +5158,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5031,7 +5249,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5119,7 +5341,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5251,7 +5477,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5437,7 +5667,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -7043,6 +7277,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8540,6 +8777,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (hw_dp_test == 1)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -11480,6 +11720,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (oop == IN_PLACE && hw_dp_test == 1)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13041,6 +13284,75 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static struct unit_test_suite cryptodev_sym_direct_api_testsuite = {
+	.suite_name = "Crypto Sym direct API Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_encryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_decryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_auth_cipher_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_auth_cipher_verify_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_hash_generate_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_hash_verify_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_encryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_decryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_CCM_authenticated_encryption_test_case_128_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_CCM_authenticated_decryption_test_case_128_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_authenticated_encryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_authenticated_decryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_encryption_test_case_192_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_decryption_test_case_192_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_encryption_test_case_256_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_decryption_test_case_256_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_encrypt_SGL_in_place_1500B),
+		TEST_CASES_END() /**< NULL terminate unit test array */
+	}
+};
+
+static int
+test_qat_sym_direct_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	hw_dp_test = 1;
+	ret = unit_test_suite_runner(&cryptodev_sym_direct_api_testsuite);
+	hw_dp_test = 0;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_sym_api_autotest, test_qat_sym_direct_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +6 -0
index 41542e055..c382c12c4 100644
@@ -71,6 +71,8 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+extern int hw_dp_test;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +211,8 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +34 -16

59 lines not on this page · raw message

Signature · 1 line
2.20.1
author
#4 lore raw

PATCH v6 4/4 doc: add cryptodev service APIs guide

In reply to Fan Zhang

This patch updates programmer's guide to demonstrate the usage and limitations of cryptodev symmetric crypto data-path service APIs.

---
 doc/guides/prog_guide/cryptodev_lib.rst | 90 +++++++++++++++++++++++++
 1 file changed, 90 insertions(+)
doc/guides/prog_guide/cryptodev_lib.rst +90 -0
index c14f750fa..77521c959 100644
@@ -631,6 +631,96 @@ a call argument. Status different than zero must be treated as error.
 For more details, e.g. how to convert an mbuf to an SGL, please refer to an
 example usage in the IPsec library implementation.
 
+Cryptodev Direct Data-plane Service API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Direct crypto data-path service are a set of APIs that especially provided for
+the external libraries/applications who want to take advantage of the rich
+features provided by cryptodev, but not necessarily depend on cryptodev
+operations, mempools, or mbufs in the their data-path implementations.
+
+The direct crypto data-path service has the following advantages:
+- Supports raw data pointer and physical addresses as input.
+- Do not require specific data structure allocated from heap, such as
+  cryptodev operation.
+- Enqueue in a burst or single operation. The service allow enqueuing in
+  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
+  enqueue one job at a time but maintaining necessary context data locally for
+  next single job enqueue operation. The latter method is especially helpful
+  when the user application's crypto operations are clustered into a burst.
+  Allowing enqueue one operation at a time helps reducing one additional loop
+  and also reduced the cache misses during the double "looping" situation.
+- Customerizable dequeue count. Instead of dequeue maximum possible operations
+  as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the
+  user to provide a callback function to decide how many operations to be
+  dequeued. This is especially helpful when the expected dequeue count is
+  hidden inside the opaque data stored during enqueue. The user can provide
+  the callback function to parse the opaque data structure.
+- Abandon enqueue and dequeue anytime. One of the drawbacks of
+  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
+  operations are: once an operation is enqueued/dequeued there is no way to
+  undo the operation. The service make the operation abandon possible by
+  creating a local copy of the queue operation data in the service context
+  data. The data will be written back to driver maintained operation data
+  when enqueue or dequeue done function is called.
+
+The cryptodev data-path service uses
+
+Cryptodev PMDs who supports this feature will have
+``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this
+feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should
+be called to get the data path service context data size. The user should
+creates a local buffer at least this size long and initialize it using
+``rte_cryptodev_dp_configure_service`` function call.
+
+The ``rte_cryptodev_dp_configure_service`` function call initialize or
+updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the
+driver specific queue pair data pointer and service context buffer, and a
+set of function pointers to enqueue and dequeue different algorithms'
+operations. The ``rte_cryptodev_dp_configure_service`` should be called when:
+
+- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
+- When different cryptodev session, security session, or session-less xform
+  is used (set ``is_update`` parameter to 1).
+
+Two different enqueue functions are provided.
+
+- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in
+  the ``rte_crypto_sym_vec`` structure.
+- ``rte_cryptodev_dp_submit_single_job``: submit single operation.
+
+Either enqueue functions will not command the crypto device to start processing
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user
+shall expect the driver only stores the necessory context data in the
+``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user
+wants to abandon the submitted operations, simply call
+``rte_cryptodev_dp_configure_service`` function instead with the parameter
+``is_update`` set to 0. The driver will recover the service context data to
+the previous state.
+
+To dequeue the operations the user also have two operations:
+
+- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The
+  user needs to provide the callback function for the driver to get the
+  dequeue count and perform post processing such as write the status field.
+- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.
+
+Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to
+merge user's local service context data with the driver's queue operation
+data. Also to abandon the dequeue operation (still keep the operations in the
+queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call
+but calling ``rte_cryptodev_dp_configure_service`` function with the parameter
+``is_update`` set to 0.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
 
Signature · 1 line
2.20.1
author
#5 lore raw

v7 cover cryptodev: add data-path service APIs

In reply to Fan Zhang

Direct crypto data-path service are a set of APIs that especially provided for the external libraries/applications who want to take advantage of the rich features provided by cryptodev, but not necessarily depend on cryptodev operations, mempools, or mbufs in the their data-path implementations.

The direct crypto data-path service has the following advantages:
- Supports raw data pointer and physical addresses as input.
- Do not require specific data structure allocated from heap, such as cryptodev operation.
- Enqueue in a burst or single operation. The service allow enqueuing in a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only enqueue one job at a time but maintaining necessary context data locally for next single job enqueue operation. The latter method is especially helpful when the user application's crypto operations are clustered into a burst. Allowing enqueue one operation at a time helps reducing one additional loop and also reduced the cache misses during the double "looping" situation.
- Customerizable dequeue count. Instead of dequeue maximum possible operations as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the user to provide a callback function to decide how many operations to be dequeued. This is especially helpful when the expected dequeue count is hidden inside the opaque data stored during enqueue. The user can provide the callback function to parse the opaque data structure.
- Abandon enqueue and dequeue anytime. One of the drawbacks of ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst`` operations are: once an operation is enqueued/dequeued there is no way to undo the operation. The service make the operation abandon possible by creating a local copy of the queue operation data in the service context data. The data will be written back to driver maintained operation data when enqueue or dequeue done function is called.

The cryptodev data-path service uses

Cryptodev PMDs who supports this feature will have ``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should be called to get the data path service context data size. The user should creates a local buffer at least this size long and initialize it using ``rte_cryptodev_dp_configure_service`` function call.

The ``rte_cryptodev_dp_configure_service`` function call initialize or updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the driver specific queue pair data pointer and service context buffer, and a set of function pointers to enqueue and dequeue different algorithms' operations. The ``rte_cryptodev_dp_configure_service`` should be called when:

- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
- When different cryptodev session, security session, or session-less xform is used (set ``is_update`` parameter to 1).

Two different enqueue functions are provided.

- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in the ``rte_crypto_sym_vec`` structure.
- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

Either enqueue functions will not command the crypto device to start processing until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user shall expect the driver only stores the necessory context data in the ``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user wants to abandon the submitted operations, simply call ``rte_cryptodev_dp_configure_service`` function instead with the parameter ``is_update`` set to 0. The driver will recover the service context data to the previous state.

To dequeue the operations the user also have two operations:

- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The user needs to provide the callback function for the driver to get the dequeue count and perform post processing such as write the status field.
- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.

Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to merge user's local service context data with the driver's queue operation data. Also to abandon the dequeue operation (still keep the operations in the queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call but calling ``rte_cryptodev_dp_configure_service`` function with the parameter ``is_update`` set to 0.

There are a few limitations to the data path service:

* Only support in-place operations.
* APIs are NOT thread-safe.
* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or vice versa.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: add crypto data-path service APIs crypto/qat: add crypto data-path service API support test/crypto: add unit-test for cryptodev direct APIs
doc: add cryptodev service APIs guide

 app/test/test_cryptodev.c                     | 354 ++++++-
 app/test/test_cryptodev.h                     |   6 +
 app/test/test_cryptodev_blockcipher.c         |  50 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  90 ++
 drivers/common/qat/Makefile                   |   1 +
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 926 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +
 lib/librte_cryptodev/rte_cryptodev.h          | 335 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 +-
 .../rte_cryptodev_version.map                 |  10 +
 15 files changed, 1892 insertions(+), 48 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#6 lore raw

PATCH v7 1/4 cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 481 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +9 -0
index fd5ef3a87..f009be9af 100644
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct rte_crypto_op *op,
 	return 0;
 }
 
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_crypto_sym.h +38 -6

56 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.c +45 -0

51 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +334 -1

347 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +45 -2

59 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 02f6dcf72..d384382d3 100644
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;
+	rte_cryptodev_get_dp_service_ctx_data_size;
+	rte_cryptodev_dp_submit_single_job;
+	rte_cryptodev_dp_sym_submit_vec;
+	rte_cryptodev_dp_submit_done;
+	rte_cryptodev_dp_sym_dequeue;
+	rte_cryptodev_dp_sym_dequeue_single_job;
+	rte_cryptodev_dp_dequeue_done;
 };
Signature · 1 line
2.20.1
#7 lore raw

Re: [dpdk-dev] [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

Hi Fan, Piotrek,

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: piątek, 28 sierpnia 2020 14:58
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Zhang, Roy Fan <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 481 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +5 -0
index fd5ef3a87..f009be9af 100644
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct rte_crypto_op *op,
 	return 0;
 }
 
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,

[Arek] - if it is auth-cipher/cipher-auth will be decided thanks to sym_session/xform?
+ RTE_CRYPTO_DP_SYM_AEAD,
+ RTE_CRYPTO_DP_N_SERVICE
+};
+
#ifdef __cplusplus
}
#endif

lib/librte_cryptodev/rte_crypto_sym.h +38 -6

55 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.c +38 -0
index 1dd795bcb..8a28511f9 100644
@@ -1914,6 +1914,51 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,
 	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);  }
 
+int
+rte_cryptodev_get_dp_service_ctx_data_size(uint8_t dev_id) {
+	struct rte_cryptodev *dev;
+	int32_t size = sizeof(struct rte_crypto_dp_service_ctx);
+	int32_t priv_size;
+
+	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+		return -1;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+	if (*dev->dev_ops->get_drv_ctx_size == NULL ||
+		!(dev->feature_flags & RTE_CRYPTODEV_FF_DATA_PLANE_SERVICE)) {
+		return -1;
+	}
+
+	priv_size = (*dev->dev_ops->get_drv_ctx_size)(dev);
+	if (priv_size < 0)
+		return -1;
+
+	return RTE_ALIGN_CEIL((size + priv_size), 8); }
+
+int
+rte_cryptodev_dp_configure_service(uint8_t dev_id, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx, uint8_t is_update) {
+	struct rte_cryptodev *dev;
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -1;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_DATA_PLANE_SERVICE)
+			|| dev->dev_ops->configure_service == NULL)
+		return -1;
+

[Arek] - Why to change order of arguments between rte_cryptodev_dp_configure_service and configure_service pointer? Except of dev and dev_id they all are the same.
+ return (*dev->dev_ops->configure_service)(dev, qp_id, ctx,
+ service_type, sess_type, session_ctx, is_update); }
+
/** Initialise rte_crypto_op mempool element */ static void rte_crypto_op_init(struct rte_mempool *mempool, diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 7b3ebc20f..9c97846f3 100644

lib/librte_cryptodev/rte_cryptodev.h +2 -1
@@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum rte_crypto_asym_xform_type *xform_enum,  /**< Support symmetric session-less operations */
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL << 23)
 /**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_DATA_PLANE_SERVICE		(1ULL << 24)
+/**< Support accelerated specific raw data as input */
 
 /**
  * Get the name of a crypto device feature flag @@ -1351,6 +1352,338 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,
 	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,

struct rte_crypto_sym_vec *vec);

+/**
+ * Get the size of the data-path service context for all registered drivers.
+ *
+ * @param dev_id The device identifier.
+ *
+ * @return
+ * - If the device supports data-path service, return the context size.
+ * - If the device does not support the data-plane service, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_get_dp_service_ctx_data_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less +xform
+ * pointer.
+ */
+union rte_cryptodev_session_ctx {
+ struct rte_cryptodev_sym_session *crypto_sess;
+ struct rte_crypto_sym_xform *xform;
+ struct rte_security_session *sec_sess; };
+
+/**
+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called. [Arek] " until ``rte_cryptodev_dp_submit_done`` function is called " ?

+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param vec The array of job vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param opaque The array of opaque data for dequeue.
+ * @return
+ * - The number of jobs successfully submitted.
+ */
+typedef uint32_t (*cryptodev_dp_sym_submit_vec_t)(
+ void *qp, uint8_t *service_data, struct rte_crypto_sym_vec *vec,
+ union rte_crypto_sym_ofs ofs, void **opaque);
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called. [Arek] until ``rte_cryptodev_dp_submit_done`` function is called " as above.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param data The buffer vector.
+ * @param n_data_vecs Number of buffer vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param iv IV data.
+ * @param digest Digest data.
+ * @param aad AAD data.
+ * @param opaque The opaque data for dequeue.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
[Arek] How can we distinguish between malformed packet and full queue?
+ */
+typedef int (*cryptodev_dp_submit_single_job_t)(
+ void *qp, uint8_t *service_data, struct rte_crypto_vec *data,
+ uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+ struct rte_crypto_data *iv, struct rte_crypto_data *digest,
+ struct rte_crypto_data *aad, void *opaque);
+
+/**
+ * Inform the queue pair to start processing or finish dequeuing all
+ * submitted/dequeued jobs.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param n The total number of submitted jobs.
+ */
+typedef void (*cryptodev_dp_sym_operation_done_t)(void *qp,
+ uint8_t *service_data, uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from the +opaque
+ * data stored in the first processed job.
+ *
+ * @param opaque Dequeued opaque data.
+ **/
+typedef uint32_t (*rte_cryptodev_get_dequeue_count_t)(void *opaque); +
+/**
+ * Typedef that the user provided to deal with post dequeue operation, +such
+ * as filling status.
+ *
+ * @param opaque Dequeued opaque data. In case
+ * RTE_CRYPTO_HW_DP_FF_GET_OPAQUE_ARRAY bit is
+ * set, this value will be the opaque data stored
+ * in the specific processed jobs referenced by
+ * index, otherwise it will be the opaque data
+ * stored in the first processed job in the burst.
+ * @param index Index number of the processed job.
+ * @param is_op_success Driver filled operation status.
+ **/
+typedef void (*rte_cryptodev_post_dequeue_t)(void *opaque, uint32_t index,
+ uint8_t is_op_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param get_dequeue_count User provided callback function to
+ * obtain dequeue count.
+ * @param post_dequeue User provided callback function to
+ * post-process a dequeued operation.
+ * @param out_opaque Opaque pointer array to be retrieve from
+ * device queue. In case of
+ * *is_opaque_array* is set there should
+ * be enough room to store all opaque data.
+ * @param is_opaque_array Set 1 if every dequeued job will be
+ * written the opaque data into
+ * *out_opaque* array.
+ * @param n_success_jobs Driver written value to specific the
+ * total successful operations count.
+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */
+typedef uint32_t (*cryptodev_dp_sym_dequeue_t)(void *qp, uint8_t *service_data,
+ rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+ rte_cryptodev_post_dequeue_t post_dequeue,
+ void **out_opaque, uint8_t is_opaque_array,
+ uint32_t *n_success_jobs);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param out_opaque Opaque pointer to be retrieve from
+ * device queue.
+ *
+ * @return

+ *   - 1 if the job is dequeued and the operation is a success.
+ *   - 0 if the job is dequeued but the operation is failed.
+ *   - -1 if no job is dequeued.

+ */
+typedef int (*cryptodev_dp_sym_dequeue_single_job_t)(
+ void *qp, uint8_t *service_data, void **out_opaque);
+
+/**
+ * Context data for asynchronous crypto process.
+ */
+struct rte_crypto_dp_service_ctx {
+ void *qp_data;
+
+ union {
[Arek] - Why union? It will be extended by other structs in future?
[Arek] - unnamed union and struct, C11 extension.
+ /* Supposed to be used for symmetric crypto service */
+ struct {
+ cryptodev_dp_submit_single_job_t submit_single_job;
+ cryptodev_dp_sym_submit_vec_t submit_vec;
+ cryptodev_dp_sym_operation_done_t submit_done;
+ cryptodev_dp_sym_dequeue_t dequeue_opaque;
+ cryptodev_dp_sym_dequeue_single_job_t dequeue_single;
+ cryptodev_dp_sym_operation_done_t dequeue_done;
+ };
+ };
+
+ /* Driver specific service data */
+ uint8_t drv_service_data[];
[Arek] - flexible array, C99 so again extension
+};
+
+/**
+ * Configure one DP service context data. Calling this function for the
+first
+ * time the user should unset the *is_update* parameter and the driver
+will
+ * fill necessary operation data into ctx buffer. Only when
+ * rte_cryptodev_dp_submit_done() is called the data stored in the ctx
+buffer
+ * will not be effective.
+ *
+ * @param dev_id The device identifier.
+ * @param qp_id The index of the queue pair from which to
+ * retrieve processed packets. The value must be
+ * in the range [0, nb_queue_pair - 1] previously
+ * supplied to rte_cryptodev_configure().
+ * @param service_type Type of the service requested.
+ * @param sess_type session type.
+ * @param session_ctx Session context data.
+ * @param ctx The data-path service context data.
+ * @param is_update Set 1 if ctx is pre-initialized but need
+ * update to different service type or session,
+ * but the rest driver data remains the same.
[Arek] - if user will call it only once with is_update == 1 will there be any error shown about ctx not set when started processing or is it undefined behavior?
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_configure_service(uint8_t dev_id, uint16_t qp_id,
+ enum rte_crypto_dp_service service_type,
+ enum rte_crypto_op_sess_type sess_type,
+ union rte_cryptodev_session_ctx session_ctx,
+ struct rte_crypto_dp_service_ctx *ctx, uint8_t is_update);
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
+ *
+ * @param ctx The initialized data-path service context data.
+ * @param data The buffer vector.
+ * @param n_data_vecs Number of buffer vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param iv IV data.
+ * @param digest Digest data.
+ * @param aad AAD data.
+ * @param opaque The array of opaque data for dequeue.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */
+__rte_experimental
+static __rte_always_inline int
+rte_cryptodev_dp_submit_single_job(struct rte_crypto_dp_service_ctx *ctx,
+ struct rte_crypto_vec *data, uint16_t n_data_vecs,
+ union rte_crypto_sym_ofs ofs,
+ struct rte_crypto_data *iv, struct rte_crypto_data *digest,
+ struct rte_crypto_data *aad, void *opaque) {
+ return (*ctx->submit_single_job)(ctx->qp_data, ctx->drv_service_data,
+ data, n_data_vecs, ofs, iv, digest, aad, opaque); }
+
+/**
+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
+ *
+ * @param ctx The initialized data-path service context data.
+ * @param vec The array of job vectors.
+ * @param ofs Start and stop offsets for auth and cipher operations.
+ * @param opaque The array of opaque data for dequeue.
+ * @return
+ * - The number of jobs successfully submitted.
+ */
+__rte_experimental
+static __rte_always_inline uint32_t
+rte_cryptodev_dp_sym_submit_vec(struct rte_crypto_dp_service_ctx *ctx,
+ struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+ void **opaque)
+{
+ return (*ctx->submit_vec)(ctx->qp_data, ctx->drv_service_data, vec,
+ ofs, opaque);
+}
+
+/**
+ * Command the queue pair to start processing all submitted jobs from
+last
+ * rte_cryptodev_init_dp_service() call.
+ *
+ * @param ctx The initialized data-path service context data.
+ * @param n The total number of submitted jobs.
+ */
+__rte_experimental
+static __rte_always_inline void
+rte_cryptodev_dp_submit_done(struct rte_crypto_dp_service_ctx *ctx,
+uint32_t n) {
+ (*ctx->submit_done)(ctx->qp_data, ctx->drv_service_data, n); }
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param ctx The initialized data-path service
+ * context data.
+ * @param get_dequeue_count User provided callback function to
+ * obtain dequeue count.
+ * @param post_dequeue User provided callback function to
+ * post-process a dequeued operation.
+ * @param out_opaque Opaque pointer array to be retrieve from
+ * device queue. In case of
+ * *is_opaque_array* is set there should
+ * be enough room to store all opaque data.
+ * @param is_opaque_array Set 1 if every dequeued job will be
+ * written the opaque data into
+ * *out_opaque* array.
+ * @param n_success_jobs Driver written value to specific the
+ * total successful operations count.
+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */
+__rte_experimental
+static __rte_always_inline uint32_t
+rte_cryptodev_dp_sym_dequeue(struct rte_crypto_dp_service_ctx *ctx,
+ rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+ rte_cryptodev_post_dequeue_t post_dequeue,
+ void **out_opaque, uint8_t is_opaque_array,
+ uint32_t *n_success_jobs)
+{
+ return (*ctx->dequeue_opaque)(ctx->qp_data, ctx->drv_service_data,
+ get_dequeue_count, post_dequeue, out_opaque, is_opaque_array,
+ n_success_jobs);
+}
+
+/**
+ * Dequeue Single symmetric crypto processing of user provided data.
+ *
+ * @param ctx The initialized data-path service
+ * context data.
+ * @param out_opaque Opaque pointer to be retrieve from
+ * device queue. The driver shall support
+ * NULL input of this parameter.
+ *
+ * @return

+ *   - 1 if the job is dequeued and the operation is a success.
+ *   - 0 if the job is dequeued but the operation is failed.
+ *   - -1 if no job is dequeued.

+ */
+__rte_experimental
+static __rte_always_inline int +rte_cryptodev_dp_sym_dequeue_single_job(struct rte_crypto_dp_service_ctx *ctx,
+ void **out_opaque)
+{
+ return (*ctx->dequeue_single)(ctx->qp_data, ctx->drv_service_data,
+ out_opaque);
+}
+
+/**
+ * Inform the queue pair dequeue jobs finished.
+ *
+ * @param ctx The initialized data-path service context data.
+ * @param n The total number of jobs already dequeued.
+ */
+__rte_experimental
+static __rte_always_inline void +rte_cryptodev_dp_dequeue_done(struct rte_crypto_dp_service_ctx *ctx, +uint32_t n) {
+ (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_service_data, n); }
+
#ifdef __cplusplus
}
#endif

lib/librte_cryptodev/rte_cryptodev_pmd.h +35 -0
index 81975d72b..bf0260c87 100644
@@ -316,6 +316,41 @@ typedef uint32_t (*cryptodev_sym_cpu_crypto_process_t)
 	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
 	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
 
+/**
+ * Typedef that the driver provided to get service context private date size.
+ *
+ * @param	dev	Crypto device pointer.
+ *
+ * @return
+ *   - On success return the size of the device's service context private data.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_get_service_ctx_size_t)(
+	struct rte_cryptodev *dev);
+
+/**
+ * Typedef that the driver provided to configure data-path service.
+ *
+ * @param	dev		Crypto device pointer.
+ * @param	qp_id		Crypto device queue pair index.
+ * @param	ctx		The data-path service context data.
+ * @param	service_type	Type of the service requested.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @param	is_update	Set 1 if ctx is pre-initialized but need
+ *				update to different service type or session,
+ *				but the rest driver data remains the same.
+ *				buffer will always be one.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_configure_service_t)(
+	struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_dp_service_ctx *ctx,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx, uint8_t is_update);
 
 /** Crypto device operations function pointer table */  struct rte_cryptodev_ops { @@ -348,8 +383,16 @@ struct rte_cryptodev_ops {
 	/**< Clear a Crypto sessions private data. */

cryptodev_asym_free_session_t asym_session_clear; /**< Clear a Crypto sessions private data. */
- cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
- /**< process input data synchronously (cpu-crypto). */
+ union {
+ cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
+ /**< process input data synchronously (cpu-crypto). */
+ struct {
+ cryptodev_dp_get_service_ctx_size_t get_drv_ctx_size;
+ /**< Get data path service context data size. */
+ cryptodev_dp_configure_service_t configure_service;
+ /**< Initialize crypto service ctx data. */
+ };
+ };
};

lib/librte_cryptodev/rte_cryptodev_version.map +4 -0
index 02f6dcf72..d384382d3 100644
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;
+	rte_cryptodev_get_dp_service_ctx_data_size;
 

[Arek] - I know its experimental but following 6 functions are not only static but __always_inline__ too, I doubt there will be any symbol generated, should they be placed inside map file then? [Arek] - Another thing since symbol is not created these funcs are outside of abidifftool scope of interest either I think.

+ rte_cryptodev_dp_submit_single_job;
+ rte_cryptodev_dp_sym_submit_vec;
+ rte_cryptodev_dp_submit_done;
+ rte_cryptodev_dp_sym_dequeue;
+ rte_cryptodev_dp_sym_dequeue_single_job;
+ rte_cryptodev_dp_dequeue_done;
};

[Arek] - Two small things:
- rte_crypto_data - there is many thing in asymmetric/symmetric crypto that can be called like that, and this is basically symmetric encrypted/authenticated data pointer, maybe some more specific name could be better.
- can iv, digest, aad be grouped into one? All share the same features, and there would less arguments to functions.
--
2.20.1

#8 lore raw

Re: [dpdk-dev] [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

Hi Arek,

Thank you very much to review.

Quoted 10 lines

-----Original Message-----
From: Kusztal, ArkadiuszX <[email protected]>
Sent: Monday, August 31, 2020 7:24 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: RE: [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

Quoted 5 lines

[Arek] - Two small things:
- rte_crypto_data - there is many thing in asymmetric/symmetric
crypto that can be called like that, and this is basically symmetric encrypted/authenticated data pointer, maybe some more specific name could be better.

[Fan] Any good name to propose?

Quoted 4 lines

- can iv, digest, aad be grouped into one? All share the same features,
and there would less arguments to functions.
--
2.20.1

Something like
struct rte_crypto_sym_<GOOD_NAME_HERE>

{
	void * iv, *digest, *aad;
	rte_iova_t iv_iova, digest_iova, aad_iova;
};

?

Definitely can do.

Regards,
Fan

#9 lore raw

Re: [dpdk-dev] [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

In reply to Kusztal, ArkadiuszX

Hi Arek,

Quoted 41 lines

-----Original Message-----
From: Kusztal, ArkadiuszX <[email protected]>
Sent: Monday, August 31, 2020 7:24 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: RE: [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

Hi Fan, Piotrek,

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: piątek, 28 sierpnia 2020 14:58
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Zhang, Roy Fan <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 481 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +5 -0
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
Quoted 2 lines

[Arek] - if it is auth-cipher/cipher-auth will be decided thanks to sym_session/xform?

[Fan] - yes.

Quoted 2 lines
(quoted diff) +2 -0
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE

...

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.c +3 -0
+			|| dev->dev_ops->configure_service == NULL)
+		return -1;
+

[Arek] - Why to change order of arguments between rte_cryptodev_dp_configure_service and configure_service pointer? Except of dev and dev_id they all are the same.

[Fan] - I will update.

Quoted 3 lines

+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called. [Arek] " until ``rte_cryptodev_dp_submit_done`` function is called " ?

[Fan] - Yes, will update. Sorry for it.

Quoted 19 lines

+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param vec The array of job vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.
+ * @param opaque The array of opaque data for
dequeue.
+ * @return
+ * - The number of jobs successfully submitted.
+ */

(quoted diff) +7 -0
+typedef uint32_t (*cryptodev_dp_sym_submit_vec_t)(
+	void *qp, uint8_t *service_data, struct rte_crypto_sym_vec *vec,
+	union rte_crypto_sym_ofs ofs, void **opaque);
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
Quoted 1 line

[Arek] until ``rte_cryptodev_dp_submit_done`` function is called " as above.

[Fan] - Yes, will update. Sorry for it.

Quoted 7 lines

+ *
+ * @param qp Driver specific queue pair data.
+ * @param service_data Driver specific service data.
+ * @param data The buffer vector.
+ * @param n_data_vecs Number of buffer vectors.
+ * @param ofs Start and stop offsets for auth and cipher
+ * operations.

Quoted 8 lines

+ * @param iv IV data.
+ * @param digest Digest data.
+ * @param aad AAD data.
+ * @param opaque The opaque data for dequeue.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
[Arek] How can we distinguish between malformed packet and full queue?

[Fan] We may have to rely on the application to track the inflight ops. We want to avoid use enqueue_burst same as rte_cryptodev_enqueue_burst, as the caller application and the PMD has to loop the same job burst 2 times to write and read the job content (cause >5% perf loss). The best option is providing an inline function to enqueue one job at a time and provides the capability of abandoning the enqueue if not all expected jobs are enqueued. So the idea is to create a shadow copy of the device's queue stored in the user application and then have the capability to abandon the enqueue when not all expected jobs are enqueued. The necessary check has to be made by the driver when merging the user's local queue data into PMD's queue data.

Quoted 14 lines

+ */

(quoted diff) +11 -0
+typedef int (*cryptodev_dp_submit_single_job_t)(
+	void *qp, uint8_t *service_data, struct rte_crypto_vec *data,
+	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_data *iv, struct rte_crypto_data *digest,
+	struct rte_crypto_data *aad, void *opaque);
+
+/**
 
 
+struct rte_crypto_dp_service_ctx {
+	void *qp_data;
+
+	union {
Quoted 1 line

[Arek] - Why union? It will be extended by other structs in future?

[Fan] - yes, maybe used by asymmetric crypto in the future.

Quoted 1 line

[Arek] - unnamed union and struct, C11 extension.

[Fan] Will correct it.

Quoted 16 lines
(quoted diff) +3 -0
+		/* Supposed to be used for symmetric crypto service */
+		struct {
+			cryptodev_dp_submit_single_job_t

submit_single_job;

(quoted diff) +4 -0
+			cryptodev_dp_sym_submit_vec_t submit_vec;
+			cryptodev_dp_sym_operation_done_t submit_done;
+			cryptodev_dp_sym_dequeue_t dequeue_opaque;
+			cryptodev_dp_sym_dequeue_single_job_t

dequeue_single;

(quoted diff) +1 -0
+			cryptodev_dp_sym_operation_done_t

dequeue_done;

lib/librte_cryptodev/rte_crypto_sym.h +5 -0
+		};
+	};
+
+	/* Driver specific service data */
+	uint8_t drv_service_data[];
Quoted 1 line

[Arek] - flexible array, C99 so again extension

[Fan] Will correct it.

Quoted 20 lines
(quoted diff) +15 -0
+};
+
+/**
+ * Configure one DP service context data. Calling this function for the
+first
+ * time the user should unset the *is_update* parameter and the driver
+will
+ * fill necessary operation data into ctx buffer. Only when
+ * rte_cryptodev_dp_submit_done() is called the data stored in the ctx
+buffer
+ * will not be effective.
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must

be
+ * in the range [0, nb_queue_pair - 1]
previously
+ * supplied to rte_cryptodev_configure().
+ * @param service_type Type of the service requested.

Quoted 8 lines

+ * @param sess_type session type.
+ * @param session_ctx Session context data.
+ * @param ctx The data-path service context data.
+ * @param is_update Set 1 if ctx is pre-initialized but need
+ * update to different service type or session,
+ * but the rest driver data remains the same.
[Arek] - if user will call it only once with is_update == 1 will there be any error shown about ctx not set when started processing or is it undefined behavior?

[Fan] - the driver will not know the difference so we may have to rely on the user
to set this correctly

Quoted 7 lines
lib/librte_cryptodev/rte_cryptodev_version.map +2 -0
+	rte_cryptodev_dp_configure_service;
+	rte_cryptodev_get_dp_service_ctx_data_size;

[Arek] - I know its experimental but following 6 functions are not only static but __always_inline__ too, I doubt there will be any symbol generated, should they be placed inside map file then?

[Fan] the symbols are not generated. I also have the same question - do we need to place them in the map file?

Quoted 11 lines

[Arek] - Another thing since symbol is not created these funcs are outside of abidifftool scope of interest either I think.

(quoted diff) +6 -0
+	rte_cryptodev_dp_submit_single_job;
+	rte_cryptodev_dp_sym_submit_vec;
+	rte_cryptodev_dp_submit_done;
+	rte_cryptodev_dp_sym_dequeue;
+	rte_cryptodev_dp_sym_dequeue_single_job;
+	rte_cryptodev_dp_dequeue_done;

};

Quoted 7 lines

[Arek] - Two small things:
- rte_crypto_data - there is many thing in asymmetric/symmetric
crypto that can be called like that, and this is basically symmetric encrypted/authenticated data pointer, maybe some more specific name could be better.
- can iv, digest, aad be grouped into one? All share the same features,
and there would less arguments to functions.

[Fan] - answered in the last email. Grouping is a good idea but may limit the usage. We need a structure that does not necessarily contains the length info for all data pre-defined when creating the sessions.

Quoted 2 lines

--
2.20.1

author
#10 lore raw

PATCH v7 2/4 crypto/qat: add crypto data-path service API support

In reply to Fan Zhang

This patch updates QAT PMD to add crypto service API support.

---
 drivers/common/qat/Makefile        |   1 +
 drivers/crypto/qat/meson.build     |   1 +
 drivers/crypto/qat/qat_sym.h       |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c | 926 +++++++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c   |   9 +-
 5 files changed, 948 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
drivers/common/qat/Makefile +1 -0
index 85d420709..1b71bbbab 100644
@@ -42,6 +42,7 @@ endif
 	SRCS-y += qat_sym.c
 	SRCS-y += qat_sym_session.c
 	SRCS-y += qat_sym_pmd.c
+	SRCS-y += qat_sym_hw_dp.c
 	build_qat = yes
 endif
 endif
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +13 -0
index 1a9748849..2d6316130 100644
@@ -264,6 +264,18 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_dp_configure_service_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_dp_service_ctx *service_ctx,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	uint8_t is_update);
+
+int
+qat_sym_get_service_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +288,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +926 -0

926 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..bef08c3bc 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Data plane service related operations */
+		.get_drv_ctx_size = qat_sym_get_service_ctx_size,
+		.configure_service = qat_sym_dp_configure_service_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_DATA_PLANE_SERVICE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#11 lore raw

PATCH v7 3/4 test/crypto: add unit-test for cryptodev direct APIs

In reply to Fan Zhang

This patch adds the QAT test to use cryptodev symmetric crypto direct APIs.

---
 app/test/test_cryptodev.c             | 354 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |   6 +
 app/test/test_cryptodev_blockcipher.c |  50 ++--
 3 files changed, 373 insertions(+), 37 deletions(-)
app/test/test_cryptodev.c +333 -21
index 70bf6fe2c..d6909984d 100644
@@ -49,6 +49,8 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +59,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+int hw_dp_test;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +151,153 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_data iv_vec, aad_vec, digest_vec;
+	union rte_crypto_sym_ofs ofs;
+	int32_t status;
+	uint32_t min_ofs, max_len;
+	union rte_cryptodev_session_ctx sess;
+	enum rte_crypto_dp_service service_type;
+	uint32_t count = 0;
+	uint8_t service_data[MAX_DRV_SERVICE_CTX_SIZE] = {0};
+	struct rte_crypto_dp_service_ctx *ctx = (void *)service_data;
+	int ctx_service_size;
+
+	sop = op->sym;
+
+	sess.crypto_sess = sop->session;
+
+	if (is_cipher && is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_CHAIN;
+		min_ofs = RTE_MIN(sop->cipher.data.offset,
+				sop->auth.data.offset);
+		max_len = RTE_MAX(sop->cipher.data.length,
+				sop->auth.data.length);
+	} else if (is_cipher) {
+		service_type = RTE_CRYPTO_DP_SYM_CIPHER_ONLY;
+		min_ofs = sop->cipher.data.offset;
+		max_len = sop->cipher.data.length;
+	} else if (is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_AUTH_ONLY;
+		min_ofs = sop->auth.data.offset;
+		max_len = sop->auth.data.length;
+	} else { /* aead */
+		service_type = RTE_CRYPTO_DP_SYM_AEAD;
+		min_ofs = sop->aead.data.offset;
+		max_len = sop->aead.data.length;
+	}
+
+	if (len_in_bits) {
+		max_len = max_len >> 3;
+		min_ofs = min_ofs >> 3;
+	}
+
+	ctx_service_size = rte_cryptodev_get_dp_service_ctx_data_size(dev_id);
+	assert(ctx_service_size <= MAX_DRV_SERVICE_CTX_SIZE &&
+			ctx_service_size > 0);
+
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* test update service */
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 1) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, min_ofs + max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n != sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ofs.raw = 0;
+
+	iv_vec.base = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	iv_vec.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	switch (service_type) {
+	case RTE_CRYPTO_DP_SYM_AEAD:
+		ofs.ofs.cipher.head = sop->cipher.data.offset;
+		aad_vec.base = (void *)sop->aead.aad.data;
+		aad_vec.iova = sop->aead.aad.phys_addr;
+		digest_vec.base = (void *)sop->aead.digest.data;
+		digest_vec.iova = sop->aead.digest.phys_addr;
+		if (len_in_bits) {
+			ofs.ofs.cipher.head >>= 3;
+			ofs.ofs.cipher.tail >>= 3;
+		}
+		break;
+	case RTE_CRYPTO_DP_SYM_CIPHER_ONLY:
+		ofs.ofs.cipher.head = sop->cipher.data.offset;
+		if (len_in_bits) {
+			ofs.ofs.cipher.head >>= 3;
+			ofs.ofs.cipher.tail >>= 3;
+		}
+		break;
+	case RTE_CRYPTO_DP_SYM_AUTH_ONLY:
+		ofs.ofs.auth.head = sop->auth.data.offset;
+		digest_vec.base = (void *)sop->auth.digest.data;
+		digest_vec.iova = sop->auth.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CHAIN:
+		ofs.ofs.cipher.head =
+			sop->cipher.data.offset - sop->auth.data.offset;
+		ofs.ofs.cipher.tail =
+			(sop->auth.data.offset + sop->auth.data.length) -
+			(sop->cipher.data.offset + sop->cipher.data.length);
+		if (len_in_bits) {
+			ofs.ofs.cipher.head >>= 3;
+			ofs.ofs.cipher.tail >>= 3;
+		}
+		digest_vec.base = (void *)sop->auth.digest.data;
+		digest_vec.iova = sop->auth.digest.phys_addr;
+		break;
+	default:
+		break;
+	}
+
+	status = rte_cryptodev_dp_submit_single_job(ctx, data_vec, n, ofs,
+		&iv_vec, &digest_vec, &aad_vec, (void *)op);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	rte_cryptodev_dp_submit_done(ctx, 1);
+
+	status = -1;
+	while (count++ < 1024 && status == -1) {
+		status = rte_cryptodev_dp_sym_dequeue_single_job(ctx,
+				(void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}
+
+	if (status != -1)
+		rte_cryptodev_dp_dequeue_done(ctx, 1);
+
+	if (count == 1025 || status != 1 || ret_op != op) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	op->status = status == 1 ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -2470,7 +2621,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2549,7 +2704,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2619,6 +2778,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2690,7 +2852,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2897,8 +3063,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2983,7 +3153,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3306,7 +3480,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3381,7 +3559,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3756,7 +3938,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3924,7 +4110,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4019,7 +4209,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4155,7 +4349,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4344,7 +4542,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4526,7 +4728,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4716,7 +4922,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4857,7 +5067,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4944,7 +5158,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5031,7 +5249,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5119,7 +5341,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5251,7 +5477,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5437,7 +5667,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -7043,6 +7277,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (hw_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8540,6 +8777,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (hw_dp_test == 1)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -11480,6 +11720,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (oop == IN_PLACE && hw_dp_test == 1)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13041,6 +13284,75 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static struct unit_test_suite cryptodev_sym_direct_api_testsuite = {
+	.suite_name = "Crypto Sym direct API Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_encryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_decryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_auth_cipher_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_snow3g_auth_cipher_verify_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_hash_generate_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_hash_verify_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_encryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_kasumi_decryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_CCM_authenticated_encryption_test_case_128_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_CCM_authenticated_decryption_test_case_128_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_authenticated_encryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_authenticated_decryption_test_case_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_encryption_test_case_192_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_decryption_test_case_192_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_encryption_test_case_256_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_decryption_test_case_256_1),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_AES_GCM_auth_encrypt_SGL_in_place_1500B),
+		TEST_CASES_END() /**< NULL terminate unit test array */
+	}
+};
+
+static int
+test_qat_sym_direct_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	hw_dp_test = 1;
+	ret = unit_test_suite_runner(&cryptodev_sym_direct_api_testsuite);
+	hw_dp_test = 0;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_sym_api_autotest, test_qat_sym_direct_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +6 -0
index 41542e055..c382c12c4 100644
@@ -71,6 +71,8 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+extern int hw_dp_test;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +211,8 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +34 -16

59 lines not on this page · raw message

Signature · 1 line
2.20.1
author
#12 lore raw

PATCH v7 4/4 doc: add cryptodev service APIs guide

In reply to Fan Zhang

This patch updates programmer's guide to demonstrate the usage and limitations of cryptodev symmetric crypto data-path service APIs.

---
 doc/guides/prog_guide/cryptodev_lib.rst | 90 +++++++++++++++++++++++++
 1 file changed, 90 insertions(+)
doc/guides/prog_guide/cryptodev_lib.rst +90 -0
index c14f750fa..77521c959 100644
@@ -631,6 +631,96 @@ a call argument. Status different than zero must be treated as error.
 For more details, e.g. how to convert an mbuf to an SGL, please refer to an
 example usage in the IPsec library implementation.
 
+Cryptodev Direct Data-plane Service API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Direct crypto data-path service are a set of APIs that especially provided for
+the external libraries/applications who want to take advantage of the rich
+features provided by cryptodev, but not necessarily depend on cryptodev
+operations, mempools, or mbufs in the their data-path implementations.
+
+The direct crypto data-path service has the following advantages:
+- Supports raw data pointer and physical addresses as input.
+- Do not require specific data structure allocated from heap, such as
+  cryptodev operation.
+- Enqueue in a burst or single operation. The service allow enqueuing in
+  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
+  enqueue one job at a time but maintaining necessary context data locally for
+  next single job enqueue operation. The latter method is especially helpful
+  when the user application's crypto operations are clustered into a burst.
+  Allowing enqueue one operation at a time helps reducing one additional loop
+  and also reduced the cache misses during the double "looping" situation.
+- Customerizable dequeue count. Instead of dequeue maximum possible operations
+  as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the
+  user to provide a callback function to decide how many operations to be
+  dequeued. This is especially helpful when the expected dequeue count is
+  hidden inside the opaque data stored during enqueue. The user can provide
+  the callback function to parse the opaque data structure.
+- Abandon enqueue and dequeue anytime. One of the drawbacks of
+  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
+  operations are: once an operation is enqueued/dequeued there is no way to
+  undo the operation. The service make the operation abandon possible by
+  creating a local copy of the queue operation data in the service context
+  data. The data will be written back to driver maintained operation data
+  when enqueue or dequeue done function is called.
+
+The cryptodev data-path service uses
+
+Cryptodev PMDs who supports this feature will have
+``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this
+feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should
+be called to get the data path service context data size. The user should
+creates a local buffer at least this size long and initialize it using
+``rte_cryptodev_dp_configure_service`` function call.
+
+The ``rte_cryptodev_dp_configure_service`` function call initialize or
+updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the
+driver specific queue pair data pointer and service context buffer, and a
+set of function pointers to enqueue and dequeue different algorithms'
+operations. The ``rte_cryptodev_dp_configure_service`` should be called when:
+
+- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
+- When different cryptodev session, security session, or session-less xform
+  is used (set ``is_update`` parameter to 1).
+
+Two different enqueue functions are provided.
+
+- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in
+  the ``rte_crypto_sym_vec`` structure.
+- ``rte_cryptodev_dp_submit_single_job``: submit single operation.
+
+Either enqueue functions will not command the crypto device to start processing
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user
+shall expect the driver only stores the necessory context data in the
+``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user
+wants to abandon the submitted operations, simply call
+``rte_cryptodev_dp_configure_service`` function instead with the parameter
+``is_update`` set to 0. The driver will recover the service context data to
+the previous state.
+
+To dequeue the operations the user also have two operations:
+
+- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The
+  user needs to provide the callback function for the driver to get the
+  dequeue count and perform post processing such as write the status field.
+- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.
+
+Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to
+merge user's local service context data with the driver's queue operation
+data. Also to abandon the dequeue operation (still keep the operations in the
+queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call
+but calling ``rte_cryptodev_dp_configure_service`` function with the parameter
+``is_update`` set to 0.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
 
Signature · 1 line
2.20.1
author
#13 lore raw

v8 cover cryptodev: add data-path service APIs

In reply to Fan Zhang

Direct crypto data-path service are a set of APIs that especially provided for the external libraries/applications who want to take advantage of the rich features provided by cryptodev, but not necessarily depend on cryptodev operations, mempools, or mbufs in the their data-path implementations.

The direct crypto data-path service has the following advantages:
- Supports raw data pointer and physical addresses as input.
- Do not require specific data structure allocated from heap, such as cryptodev operation.
- Enqueue in a burst or single operation. The service allow enqueuing in a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only enqueue one job at a time but maintaining necessary context data locally for next single job enqueue operation. The latter method is especially helpful when the user application's crypto operations are clustered into a burst. Allowing enqueue one operation at a time helps reducing one additional loop and also reduced the cache misses during the double "looping" situation.
- Customerizable dequeue count. Instead of dequeue maximum possible operations as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the user to provide a callback function to decide how many operations to be dequeued. This is especially helpful when the expected dequeue count is hidden inside the opaque data stored during enqueue. The user can provide the callback function to parse the opaque data structure.
- Abandon enqueue and dequeue anytime. One of the drawbacks of ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst`` operations are: once an operation is enqueued/dequeued there is no way to undo the operation. The service make the operation abandon possible by creating a local copy of the queue operation data in the service context data. The data will be written back to driver maintained operation data when enqueue or dequeue done function is called.

The cryptodev data-path service uses

Cryptodev PMDs who supports this feature will have ``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should be called to get the data path service context data size. The user should creates a local buffer at least this size long and initialize it using ``rte_cryptodev_dp_configure_service`` function call.

The ``rte_cryptodev_dp_configure_service`` function call initialize or updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the driver specific queue pair data pointer and service context buffer, and a set of function pointers to enqueue and dequeue different algorithms' operations. The ``rte_cryptodev_dp_configure_service`` should be called when:

- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
- When different cryptodev session, security session, or session-less xform is used (set ``is_update`` parameter to 1).

Two different enqueue functions are provided.

- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in the ``rte_crypto_sym_vec`` structure.
- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

Either enqueue functions will not command the crypto device to start processing until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user shall expect the driver only stores the necessory context data in the ``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user wants to abandon the submitted operations, simply call ``rte_cryptodev_dp_configure_service`` function instead with the parameter ``is_update`` set to 0. The driver will recover the service context data to the previous state.

To dequeue the operations the user also have two operations:

- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The user needs to provide the callback function for the driver to get the dequeue count and perform post processing such as write the status field.
- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.

Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to merge user's local service context data with the driver's queue operation data. Also to abandon the dequeue operation (still keep the operations in the queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call but calling ``rte_cryptodev_dp_configure_service`` function with the parameter ``is_update`` set to 0.

There are a few limitations to the data path service:

* Only support in-place operations.
* APIs are NOT thread-safe.
* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or vice versa.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: add crypto data-path service APIs crypto/qat: add crypto data-path service API support test/crypto: add unit-test for cryptodev direct APIs
doc: add cryptodev service APIs guide

 app/test/test_cryptodev.c                     | 452 ++++++++-
 app/test/test_cryptodev.h                     |   7 +
 app/test/test_cryptodev_blockcipher.c         |  51 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  90 ++
 drivers/common/qat/Makefile                   |   1 +
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 941 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  49 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  98 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 332 +++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  48 +-
 .../rte_cryptodev_version.map                 |  10 +
 15 files changed, 2037 insertions(+), 74 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#14 lore raw

PATCH v8 1/4 cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  49 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  98 ++++++
 lib/librte_cryptodev/rte_cryptodev.h          | 332 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  48 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 537 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +9 -0
index fd5ef3a87..f009be9af 100644
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct rte_crypto_op *op,
 	return 0;
 }
 
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_crypto_sym.h +43 -6

61 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.c +98 -0

104 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +331 -1

344 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +46 -2

60 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 02f6dcf72..10388ae90 100644
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;
+	rte_cryptodev_dp_get_service_ctx_data_size;
+	rte_cryptodev_dp_sym_dequeue;
+	rte_cryptodev_dp_sym_dequeue_done;
+	rte_cryptodev_dp_sym_dequeue_single_job;
+	rte_cryptodev_dp_sym_submit_done;
+	rte_cryptodev_dp_sym_submit_single_job;
+	rte_cryptodev_dp_sym_submit_vec;
 };
Signature · 1 line
2.20.1
Acked-by on v8 1/4
#15 lore raw

Re: [dpdk-dev] [dpdk-dev v8 1/4] cryptodev: add crypto data-path service APIs

Quoted 10 lines

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: Friday, 4 September, 2020 17:26
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Zhang, Roy Fan <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: [dpdk-dev v8 1/4] cryptodev: add crypto data-path service APIs

Quoted 6 lines

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

Series

author
#16 lore raw

PATCH v8 2/4 crypto/qat: add crypto data-path service API support

In reply to Fan Zhang

This patch updates QAT PMD to add crypto service API support.

---
 drivers/common/qat/Makefile        |   1 +
 drivers/crypto/qat/meson.build     |   1 +
 drivers/crypto/qat/qat_sym.h       |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c | 941 +++++++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c   |   9 +-
 5 files changed, 963 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
drivers/common/qat/Makefile +1 -0
index 85d420709..1b71bbbab 100644
@@ -42,6 +42,7 @@ endif
 	SRCS-y += qat_sym.c
 	SRCS-y += qat_sym_session.c
 	SRCS-y += qat_sym_pmd.c
+	SRCS-y += qat_sym_hw_dp.c
 	build_qat = yes
 endif
 endif
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +13 -0
index 1a9748849..ea2db0ca0 100644
@@ -264,6 +264,18 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_dp_configure_service_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *service_ctx,
+	uint8_t is_update);
+
+int
+qat_sym_get_service_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +288,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +941 -0

941 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..aaaf3e3f1 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Data plane service related operations */
+		.get_drv_ctx_size = qat_sym_get_service_ctx_size,
+		.configure_service = qat_sym_dp_configure_service_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_DATA_PATH_SERVICE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#17 lore raw

PATCH v8 3/4 test/crypto: add unit-test for cryptodev direct APIs

In reply to Fan Zhang

This patch adds the QAT test to use cryptodev symmetric crypto direct APIs.

---
 app/test/test_cryptodev.c             | 452 +++++++++++++++++++++++---
 app/test/test_cryptodev.h             |   7 +
 app/test/test_cryptodev_blockcipher.c |  51 ++-
 3 files changed, 447 insertions(+), 63 deletions(-)
app/test/test_cryptodev.c +405 -47
index 70bf6fe2c..387a3cf15 100644
@@ -49,6 +49,8 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +59,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+int cryptodev_dp_test;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +151,173 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	union rte_crypto_sym_additional_data a_data;
+	union rte_crypto_sym_ofs ofs;
+	int32_t status;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	enum rte_crypto_dp_service service_type;
+	uint32_t count = 0;
+	uint8_t service_data[MAX_DRV_SERVICE_CTX_SIZE] = {0};
+	struct rte_crypto_dp_service_ctx *ctx = (void *)service_data;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int ctx_service_size;
+
+	sop = op->sym;
+
+	sess.crypto_sess = sop->session;
+
+	if (is_cipher && is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_CHAIN;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+	} else if (is_cipher) {
+		service_type = RTE_CRYPTO_DP_SYM_CIPHER_ONLY;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+	} else if (is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_AUTH_ONLY;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+	} else { /* aead */
+		service_type = RTE_CRYPTO_DP_SYM_AEAD;
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+	}
+
+	if (len_in_bits) {
+		max_len = max_len >> 3;
+		cipher_offset = cipher_offset >> 3;
+		auth_offset = auth_offset >> 3;
+		cipher_len = cipher_len >> 3;
+		auth_len = auth_len >> 3;
+	}
+
+	ctx_service_size = rte_cryptodev_dp_get_service_ctx_data_size(dev_id);
+	assert(ctx_service_size <= MAX_DRV_SERVICE_CTX_SIZE &&
+			ctx_service_size > 0);
+
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* test update service */
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 1) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ofs.raw = 0;
+
+	switch (service_type) {
+	case RTE_CRYPTO_DP_SYM_AEAD:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.aead.iv_ptr = rte_crypto_op_ctod_offset(op, void *,
+				IV_OFFSET);
+		a_data.aead.iv_iova = rte_crypto_op_ctophys_offset(op,
+				IV_OFFSET);
+		a_data.aead.aad_ptr = (void *)sop->aead.aad.data;
+		a_data.aead.aad_iova = sop->aead.aad.phys_addr;
+		a_data.aead.digest_ptr = (void *)sop->aead.digest.data;
+		a_data.aead.digest_iova = sop->aead.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CIPHER_ONLY:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.cipher_auth.cipher_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		break;
+	case RTE_CRYPTO_DP_SYM_AUTH_ONLY:
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.auth_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop->auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop->auth.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CHAIN:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.cipher_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		a_data.cipher_auth.auth_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop->auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop->auth.digest.phys_addr;
+		break;
+	default:
+		break;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_single_job(ctx, data_vec, n, ofs,
+		&a_data, (void *)op);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	rte_cryptodev_dp_sym_submit_done(ctx, 1);
+
+	status = -1;
+	while (count++ < 65535 && status == -1) {
+		status = rte_cryptodev_dp_sym_dequeue_single_job(ctx,
+				(void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}
+
+	if (status != -1)
+		rte_cryptodev_dp_sym_dequeue_done(ctx, 1);
+
+	if (count == 65536 || status != 1 || ret_op != op) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	op->status = status == 1 ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1656,6 +1827,9 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params->valid_devs[0],
@@ -1710,12 +1884,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2470,7 +2650,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2549,7 +2733,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2619,6 +2807,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2690,7 +2881,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2897,8 +3092,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2983,7 +3182,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3026,8 +3229,9 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create KASUMI session */
@@ -3107,8 +3311,9 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3192,8 +3397,9 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create KASUMI session */
@@ -3306,7 +3512,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3381,7 +3591,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3419,7 +3633,7 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create SNOW 3G session */
@@ -3502,7 +3716,7 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3621,7 +3835,7 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create SNOW 3G session */
@@ -3756,7 +3970,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3791,7 +4009,7 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create SNOW 3G session */
@@ -3924,7 +4142,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4019,7 +4241,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4087,6 +4313,8 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 	}
 
 	/* Create SNOW 3G session */
@@ -4155,7 +4383,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4266,6 +4498,8 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4344,7 +4578,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4453,6 +4691,8 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	uint64_t feat_flags = dev_info.feature_flags;
 
 	if (op_mode == OUT_OF_PLACE) {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4526,7 +4766,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4638,6 +4882,8 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4716,7 +4962,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4857,7 +5107,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4944,7 +5198,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5031,7 +5289,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5119,7 +5381,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5177,6 +5443,8 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5251,7 +5519,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5359,6 +5631,8 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5437,7 +5711,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5580,6 +5858,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5779,6 +6060,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5793,6 +6077,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6146,10 +6433,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6161,6 +6447,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
 		retval = create_wireless_algo_cipher_auth_session(
@@ -6192,9 +6481,11 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}
 
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6235,18 +6526,17 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6337,10 +6627,9 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6440,20 +6729,18 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -7043,6 +7330,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8540,6 +8830,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8833,6 +9126,9 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
+	/* Data-path service does not support OOP */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
@@ -8923,8 +9219,9 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and data-path service*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create AEAD session */
@@ -9151,8 +9448,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
 
@@ -9472,6 +9774,9 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9530,6 +9835,9 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10098,6 +10406,9 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10215,6 +10526,9 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10780,7 +11094,10 @@ test_authentication_verify_fail_when_data_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10851,7 +11168,10 @@ test_authentication_verify_GMAC_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10926,7 +11246,10 @@ test_authenticated_decryption_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -11021,6 +11344,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11141,6 +11467,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -11285,6 +11614,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Data path service does not support OOP */
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
@@ -11480,6 +11812,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13041,6 +13376,29 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static int
+test_qat_sym_direct_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	cryptodev_dp_test = 1;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	cryptodev_dp_test = 0;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_sym_api_autotest, test_qat_sym_direct_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +7 -0
index 41542e055..e4e4c7626 100644
@@ -71,6 +71,8 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+extern int cryptodev_dp_test;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +211,9 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +35 -16

60 lines not on this page · raw message

Signature · 1 line
2.20.1
author
#18 lore raw

PATCH v8 4/4 doc: add cryptodev service APIs guide

In reply to Fan Zhang

This patch updates programmer's guide to demonstrate the usage and limitations of cryptodev symmetric crypto data-path service APIs.

---
 doc/guides/prog_guide/cryptodev_lib.rst | 90 +++++++++++++++++++++++++
 1 file changed, 90 insertions(+)
doc/guides/prog_guide/cryptodev_lib.rst +90 -0
index c14f750fa..1321e4c5d 100644
@@ -631,6 +631,96 @@ a call argument. Status different than zero must be treated as error.
 For more details, e.g. how to convert an mbuf to an SGL, please refer to an
 example usage in the IPsec library implementation.
 
+Cryptodev Direct Data-path Service API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Direct crypto data-path service are a set of APIs that especially provided for
+the external libraries/applications who want to take advantage of the rich
+features provided by cryptodev, but not necessarily depend on cryptodev
+operations, mempools, or mbufs in the their data-path implementations.
+
+The direct crypto data-path service has the following advantages:
+- Supports raw data pointer and physical addresses as input.
+- Do not require specific data structure allocated from heap, such as
+  cryptodev operation.
+- Enqueue in a burst or single operation. The service allow enqueuing in
+  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
+  enqueue one job at a time but maintaining necessary context data locally for
+  next single job enqueue operation. The latter method is especially helpful
+  when the user application's crypto operations are clustered into a burst.
+  Allowing enqueue one operation at a time helps reducing one additional loop
+  and also reduced the cache misses during the double "looping" situation.
+- Customerizable dequeue count. Instead of dequeue maximum possible operations
+  as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the
+  user to provide a callback function to decide how many operations to be
+  dequeued. This is especially helpful when the expected dequeue count is
+  hidden inside the opaque data stored during enqueue. The user can provide
+  the callback function to parse the opaque data structure.
+- Abandon enqueue and dequeue anytime. One of the drawbacks of
+  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
+  operations are: once an operation is enqueued/dequeued there is no way to
+  undo the operation. The service make the operation abandon possible by
+  creating a local copy of the queue operation data in the service context
+  data. The data will be written back to driver maintained operation data
+  when enqueue or dequeue done function is called.
+
+The cryptodev data-path service uses
+
+Cryptodev PMDs who supports this feature will have
+``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this
+feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should
+be called to get the data path service context data size. The user should
+creates a local buffer at least this size long and initialize it using
+``rte_cryptodev_dp_configure_service`` function call.
+
+The ``rte_cryptodev_dp_configure_service`` function call initialize or
+updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the
+driver specific queue pair data pointer and service context buffer, and a
+set of function pointers to enqueue and dequeue different algorithms'
+operations. The ``rte_cryptodev_dp_configure_service`` should be called when:
+
+- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
+- When different cryptodev session, security session, or session-less xform
+  is used (set ``is_update`` parameter to 1).
+
+Two different enqueue functions are provided.
+
+- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in
+  the ``rte_crypto_sym_vec`` structure.
+- ``rte_cryptodev_dp_submit_single_job``: submit single operation.
+
+Either enqueue functions will not command the crypto device to start processing
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user
+shall expect the driver only stores the necessory context data in the
+``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user
+wants to abandon the submitted operations, simply call
+``rte_cryptodev_dp_configure_service`` function instead with the parameter
+``is_update`` set to 0. The driver will recover the service context data to
+the previous state.
+
+To dequeue the operations the user also have two operations:
+
+- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The
+  user needs to provide the callback function for the driver to get the
+  dequeue count and perform post processing such as write the status field.
+- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.
+
+Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to
+merge user's local service context data with the driver's queue operation
+data. Also to abandon the dequeue operation (still keep the operations in the
+queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call
+but calling ``rte_cryptodev_dp_configure_service`` function with the parameter
+``is_update`` set to 0.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
 
Signature · 1 line
2.20.1
author
#19 lore raw

v9 cover cryptodev: add data-path service APIs

In reply to Fan Zhang

Direct crypto data-path service are a set of APIs that especially provided for the external libraries/applications who want to take advantage of the rich features provided by cryptodev, but not necessarily depend on cryptodev operations, mempools, or mbufs in the their data-path implementations.

The direct crypto data-path service has the following advantages:
- Supports raw data pointer and physical addresses as input.
- Do not require specific data structure allocated from heap, such as cryptodev operation.
- Enqueue in a burst or single operation. The service allow enqueuing in a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only enqueue one job at a time but maintaining necessary context data locally for next single job enqueue operation. The latter method is especially helpful when the user application's crypto operations are clustered into a burst. Allowing enqueue one operation at a time helps reducing one additional loop and also reduced the cache misses during the double "looping" situation.
- Customerizable dequeue count. Instead of dequeue maximum possible operations as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the user to provide a callback function to decide how many operations to be dequeued. This is especially helpful when the expected dequeue count is hidden inside the opaque data stored during enqueue. The user can provide the callback function to parse the opaque data structure.
- Abandon enqueue and dequeue anytime. One of the drawbacks of ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst`` operations are: once an operation is enqueued/dequeued there is no way to undo the operation. The service make the operation abandon possible by creating a local copy of the queue operation data in the service context data. The data will be written back to driver maintained operation data when enqueue or dequeue done function is called.

The cryptodev data-path service uses

Cryptodev PMDs who supports this feature will have ``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should be called to get the data path service context data size. The user should creates a local buffer at least this size long and initialize it using ``rte_cryptodev_dp_configure_service`` function call.

The ``rte_cryptodev_dp_configure_service`` function call initialize or updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the driver specific queue pair data pointer and service context buffer, and a set of function pointers to enqueue and dequeue different algorithms' operations. The ``rte_cryptodev_dp_configure_service`` should be called when:

- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
- When different cryptodev session, security session, or session-less xform is used (set ``is_update`` parameter to 1).

Two different enqueue functions are provided.

- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in the ``rte_crypto_sym_vec`` structure.
- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

Either enqueue functions will not command the crypto device to start processing until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user shall expect the driver only stores the necessory context data in the ``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user wants to abandon the submitted operations, simply call ``rte_cryptodev_dp_configure_service`` function instead with the parameter ``is_update`` set to 0. The driver will recover the service context data to the previous state.

To dequeue the operations the user also have two operations:

- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The user needs to provide the callback function for the driver to get the dequeue count and perform post processing such as write the status field.
- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.

Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to merge user's local service context data with the driver's queue operation data. Also to abandon the dequeue operation (still keep the operations in the queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call but calling ``rte_cryptodev_dp_configure_service`` function with the parameter ``is_update`` set to 0.

There are a few limitations to the data path service:

* Only support in-place operations.
* APIs are NOT thread-safe.
* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or vice versa.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: add crypto data-path service APIs crypto/qat: add crypto data-path service API support test/crypto: add unit-test for cryptodev direct APIs
doc: add cryptodev service APIs guide

 app/test/test_cryptodev.c                     | 461 ++++++++-
 app/test/test_cryptodev.h                     |   7 +
 app/test/test_cryptodev_blockcipher.c         |  51 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  90 ++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 drivers/common/qat/Makefile                   |   1 +
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 947 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  49 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  98 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  48 +-
 .../rte_cryptodev_version.map                 |  10 +
 16 files changed, 2062 insertions(+), 74 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#20 lore raw

PATCH v9 1/4 cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  49 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  98 +++++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  48 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 540 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +9 -0
index fd5ef3a87..f009be9af 100644
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct rte_crypto_op *op,
 	return 0;
 }
 
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_crypto_sym.h +43 -6

61 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.c +98 -0

104 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +334 -1

347 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +46 -2

60 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 02f6dcf72..10388ae90 100644
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;
+	rte_cryptodev_dp_get_service_ctx_data_size;
+	rte_cryptodev_dp_sym_dequeue;
+	rte_cryptodev_dp_sym_dequeue_done;
+	rte_cryptodev_dp_sym_dequeue_single_job;
+	rte_cryptodev_dp_sym_submit_done;
+	rte_cryptodev_dp_sym_submit_single_job;
+	rte_cryptodev_dp_sym_submit_vec;
 };
Signature · 1 line
2.20.1
#21 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 24 lines

Subject: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  49 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  98 +++++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  48 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 540 insertions(+), 9 deletions(-)
lib/librte_cryptodev/rte_crypto.h +0 -0
index fd5ef3a87..f009be9af 100644
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct
 rte_crypto_op *op,
 	return 0;

lib/librte_cryptodev/rte_crypto.h rte_crypto_op_attach_asym_session(struct · line 449

Quoted 10 lines
lib/librte_cryptodev/rte_crypto.h +8 -0
 }
 
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};

Comments missing for this enum.
Do we really need this enum?
Can we not have this info in the driver from the xform list?
And if we really want to add this, why to have it specific to raw data path APIs?

Quoted 10 lines
lib/librte_cryptodev/rte_crypto.h +1 -0
+
 #ifdef __cplusplus
 }

#endif

lib/librte_cryptodev/rte_crypto_sym.h +0 -0
index f29c98051..376412e94 100644

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 57

Quoted 8 lines
lib/librte_cryptodev/rte_crypto_sym.h +5 -0
@@ -50,6 +50,30 @@ struct rte_crypto_sgl {
 	uint32_t num;
 };
 
+/**
+ * Symmetri Crypto Addtional Data other than src and destination data.
+ * Supposed to be used to pass IV/digest/aad data buffers with lengths
+ * defined when creating crypto session.
+ */

Fix typo

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 66

Quoted 9 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -0
+union rte_crypto_sym_additional_data {
+	struct {
+		void *cipher_iv_ptr;
+		rte_iova_t cipher_iv_iova;
+		void *auth_iv_ptr;
+		rte_iova_t auth_iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+	} cipher_auth;

Should be chain instead of cipher_auth

Quoted 25 lines
lib/librte_cryptodev/rte_crypto_sym.h +12 -6
+	struct {
+		void *iv_ptr;
+		rte_iova_t iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+		void *aad_ptr;
+		rte_iova_t aad_iova;
+	} aead;
+};
+
 /**
  * Synchronous operation descriptor.
  * Supposed to be used with CPU crypto API call.
@@ -57,12 +81,25 @@ struct rte_crypto_sgl {
 struct rte_crypto_sym_vec {
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
-	/** array of pointers to digest */
-	void **digest;
+
+	union {

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 95

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+
+		/* Supposed to be used with CPU crypto API call. */
+		struct {
+			/** array of pointers to IV */
+			void **iv;
+			/** array of pointers to AAD */
+			void **aad;
+			/** array of pointers to digest */
+			void **digest;
+		};

Can we also name this struct?
Probably we should split this as a separate patch.

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 102

Quoted 8 lines
lib/librte_cryptodev/rte_crypto_sym.h +7 -0
+
+		/* Supposed to be used with
 rte_cryptodev_dp_sym_submit_vec()
+		 * call.
+		 */
+		union rte_crypto_sym_additional_data *additional_data;
+	};
+

Can we get rid of this unnecessary union rte_crypto_sym_additional_data And place chain and aead directly in the union? At any point, only one of the three would be used.

Quoted 20 lines
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
 	/**
 	 * array of statuses for each operation:
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
 	 *  - 0 on success
lib/librte_cryptodev/rte_cryptodev.c +7 -0
index 1dd795bcb..4f59cf800 100644
@@ -1914,6 +1914,104 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t
 dev_id,
 	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);
 }
 
+int
+rte_cryptodev_dp_get_service_ctx_data_size(uint8_t dev_id)
+{
+	struct rte_cryptodev *dev;
+	int32_t size = sizeof(struct rte_crypto_dp_service_ctx);
+	int32_t priv_size;
+

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t · line 1934

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +9 -0
+	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+		return -1;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+	if (*dev->dev_ops->get_drv_ctx_size == NULL ||
+		!(dev->feature_flags &
 RTE_CRYPTODEV_FF_DATA_PATH_SERVICE)) {
+		return -1;
+	}

I have some suggestions for the naming of the APIs / flags in the doc patch, Please check that and make changes in this patch.
Also, you have missed adding this feature flag in the doc/guides/cryptodevs/features/default.ini file.
And Subsequently in the doc/guides/cryptodevs/features/qat.ini file.

Quoted 14 lines
lib/librte_cryptodev/rte_cryptodev.c +14 -0
+
+	priv_size = (*dev->dev_ops->get_drv_ctx_size)(dev);
+	if (priv_size < 0)
+		return -1;
+
+	return RTE_ALIGN_CEIL((size + priv_size), 8);
+}
+
+int
+rte_cryptodev_dp_configure_service(uint8_t dev_id, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx, uint8_t is_update)

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1956

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+{
+	struct rte_cryptodev *dev;
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -1;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_DATA_PATH_SERVICE)
+			|| dev->dev_ops->configure_service == NULL)
+		return -1;

It would be better to return actual error number like ENOTSUP/EINVAL. It would be helpful in debugging.

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1968

Quoted 13 lines
lib/librte_cryptodev/rte_cryptodev.c +12 -0
+
+	return (*dev->dev_ops->configure_service)(dev, qp_id, service_type,
+			sess_type, session_ctx, ctx, is_update);
+}
+
+int
+rte_cryptodev_dp_sym_submit_single_job(struct rte_crypto_dp_service_ctx
 *ctx,
+		struct rte_crypto_vec *data, uint16_t n_data_vecs,
+		union rte_crypto_sym_ofs ofs,
+		union rte_crypto_sym_additional_data *additional_data,
+		void *opaque)
+{

Can we have some debug checks for NULL checking.

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1970

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.c +2 -0
+	return _cryptodev_dp_submit_single_job(ctx, data, n_data_vecs, ofs,
+			additional_data, opaque);

Unnecessary function call _cryptodev_dp_submit_single_job. You can directly call
return (*ctx->submit_single_job)(ctx->qp_data, ctx->drv_service_data,

		data, n_data_vecs, ofs, additional_data, opaque);
Quoted 14 lines
lib/librte_cryptodev/rte_cryptodev.c +13 -0
+}
+
+uint32_t
+rte_cryptodev_dp_sym_submit_vec(struct rte_crypto_dp_service_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **opaque)
+{
+	return (*ctx->submit_vec)(ctx->qp_data, ctx->drv_service_data, vec,
+			ofs, opaque);
+}
+
+int
+rte_cryptodev_dp_sym_dequeue_single_job(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1986

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev.c +3 -0
+		void **out_opaque)
+{
+	return _cryptodev_dp_sym_dequeue_single_job(ctx, out_opaque);

Same here.

Quoted 38 lines
lib/librte_cryptodev/rte_cryptodev.c +25 -0
+}
+
+int
+rte_cryptodev_dp_sym_submit_done(struct rte_crypto_dp_service_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->submit_done)(ctx->qp_data, ctx->drv_service_data, n);
+}
+
+int
+rte_cryptodev_dp_sym_dequeue_done(struct rte_crypto_dp_service_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_service_data, n);
+}
+
+uint32_t
+rte_cryptodev_dp_sym_dequeue(struct rte_crypto_dp_service_ctx *ctx,
+	rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_post_dequeue_t post_dequeue,
+	void **out_opaque, uint8_t is_opaque_array,
+	uint32_t *n_success_jobs)
+{
+	return (*ctx->dequeue_opaque)(ctx->qp_data, ctx->drv_service_data,
+		get_dequeue_count, post_dequeue, out_opaque,

is_opaque_array,

lib/librte_cryptodev/rte_cryptodev.c +3 -0
+		n_success_jobs);
+}
+
 /** Initialise rte_crypto_op mempool element */
 static void
 rte_crypto_op_init(struct rte_mempool *mempool,
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index 7b3ebc20f..4da0389d1 100644

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_asym_get_xform_enum(enum · line 472

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +2 -1
@@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum
 rte_crypto_asym_xform_type *xform_enum,
 /**< Support symmetric session-less operations */
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL
 << 23)
 /**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_DATA_PATH_SERVICE		(1ULL << 24)
+/**< Support accelerated specific raw data as input */

Support data path APIs for raw data as input.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t · line 1357

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 /**

* Get the name of a crypto device feature flag

lib/librte_cryptodev/rte_cryptodev.h +2 -0
@@ -1351,6 +1352,338 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t
 dev_id,
 	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
 	struct rte_crypto_sym_vec *vec);
 
+/**
+ * Get the size of the data-path service context for all registered drivers.

For all drivers ? or for a device?

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ *
+ * @param	dev_id		The device identifier.
+ *
+ * @return

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1370

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   - If the device supports data-path service, return the context size.
+ *   - If the device does not support the data-dath service, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_get_service_ctx_data_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less xform
+ * pointer.

Union of different symmetric crypto session types ..

Quoted 7 lines
lib/librte_cryptodev/rte_cryptodev.h +7 -0
+ */
+union rte_cryptodev_session_ctx {
+	struct rte_cryptodev_sym_session *crypto_sess;
+	struct rte_crypto_sym_xform *xform;
+	struct rte_security_session *sec_sess;
+};
+

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1387

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+/**
+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	service_data	Driver specific service data.
+ * @param	vec		The array of job vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	opaque		The array of opaque data for dequeue.

Can you elaborate the usage of opaque here?

Quoted 59 lines
lib/librte_cryptodev/rte_cryptodev.h +46 -0
+ * @return
+ *   - The number of jobs successfully submitted.
+ */
+typedef uint32_t (*cryptodev_dp_sym_submit_vec_t)(
+	void *qp, uint8_t *service_data, struct rte_crypto_sym_vec *vec,
+	union rte_crypto_sym_ofs ofs, void **opaque);
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	service_data	Driver specific service data.
+ * @param	data		The buffer vector.
+ * @param	n_data_vecs	Number of buffer vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	additional_data	IV, digest, and aad data.
+ * @param	opaque		The opaque data for dequeue.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_submit_single_job_t)(
+	void *qp, uint8_t *service_data, struct rte_crypto_vec *data,
+	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+	union rte_crypto_sym_additional_data *additional_data,
+	void *opaque);
+
+/**
+ * Inform the queue pair to start processing or finish dequeuing all
+ * submitted/dequeued jobs.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	service_data	Driver specific service data.
+ * @param	n		The total number of submitted jobs.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_sym_operation_done_t)(void *qp,
+		uint8_t *service_data, uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from the

opaque
+ * data stored in the first processed job.
+ *
+ * @param opaque Dequeued opaque data.
+ **/

lib/librte_cryptodev/rte_cryptodev.h +8 -0
+typedef uint32_t (*rte_cryptodev_get_dequeue_count_t)(void *opaque);
+
+/**
+ * Typedef that the user provided to deal with post dequeue operation, such
+ * as filling status.
+ *
+ * @param	opaque		Dequeued opaque data. In case
+ *
Quoted 5 lines

RTE_CRYPTO_HW_DP_FF_GET_OPAQUE_ARRAY bit is
+ * set, this value will be the opaque data stored
+ * in the specific processed jobs referenced by
+ * index, otherwise it will be the opaque data
+ * stored in the first processed job in the burst.

What is RTE_CRYPTO_HW_DP_FF_GET_OPAQUE_ARRAY, I did not find this in the series.

Quoted 62 lines

+ * @param index Index number of the processed job.
+ * @param is_op_success Driver filled operation status.
+ **/

lib/librte_cryptodev/rte_cryptodev.h +13 -0
+typedef void (*rte_cryptodev_post_dequeue_t)(void *opaque, uint32_t index,
+		uint8_t is_op_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	service_data		Driver specific service data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_opaque		Opaque pointer array to be retrieve

from
+ * device queue. In case of
+ * *is_opaque_array* is set there should
+ * be enough room to store all opaque
data.
+ * @param is_opaque_array Set 1 if every dequeued job will
be
+ * written the opaque data into
+ * *out_opaque* array.
+ * @param n_success_jobs Driver written value to specific the
+ * total successful operations count.
+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */
+typedef uint32_t (*cryptodev_dp_sym_dequeue_t)(void *qp, uint8_t *service_data,

lib/librte_cryptodev/rte_cryptodev.h +29 -0
+	rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_post_dequeue_t post_dequeue,
+	void **out_opaque, uint8_t is_opaque_array,
+	uint32_t *n_success_jobs);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	service_data		Driver specific service data.
+ * @param	out_opaque		Opaque pointer to be retrieve from
+ *					device queue.
+ *
+ * @return
+ *   - 1 if the job is dequeued and the operation is a success.
+ *   - 0 if the job is dequeued but the operation is failed.
+ *   - -1 if no job is dequeued.
+ */
+typedef int (*cryptodev_dp_sym_dequeue_single_job_t)(
+		void *qp, uint8_t *service_data, void **out_opaque);
+
+/**
+ * Context data for asynchronous crypto process.
+ */
+struct rte_crypto_dp_service_ctx {
+	void *qp_data;
+
+	struct {
+		cryptodev_dp_submit_single_job_t submit_single_job;

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1517

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+		cryptodev_dp_sym_submit_vec_t submit_vec;
+		cryptodev_dp_sym_operation_done_t submit_done;
+		cryptodev_dp_sym_dequeue_t dequeue_opaque;
+		cryptodev_dp_sym_dequeue_single_job_t dequeue_single;
+		cryptodev_dp_sym_operation_done_t dequeue_done;
+	};
+
+	/* Driver specific service data */
+	__extension__ uint8_t drv_service_data[];
+};

Comments missing for structure params.
Struct name can be rte_crypto_raw_dp_ctx.

Who allocate and free this structure?

Quoted 252 lines
lib/librte_cryptodev/rte_cryptodev.h +50 -0
+
+/**
+ * Configure one DP service context data. Calling this function for the first
+ * time the user should unset the *is_update* parameter and the driver will
+ * fill necessary operation data into ctx buffer. Only when
+ * rte_cryptodev_dp_submit_done() is called the data stored in the ctx buffer
+ * will not be effective.
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must be
+ *				in the range [0, nb_queue_pair - 1] previously
+ *				supplied to rte_cryptodev_configure().
+ * @param	service_type	Type of the service requested.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @param	ctx		The data-path service context data.
+ * @param	is_update	Set 1 if ctx is pre-initialized but need
+ *				update to different service type or session,
+ *				but the rest driver data remains the same.
+ *				Since service context data buffer is provided
+ *				by user, the driver will not check the
+ *				validity of the buffer nor its content. It is
+ *				the user's obligation to initialize and
+ *				uses the buffer properly by setting this field.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_configure_service(uint8_t dev_id, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx, uint8_t is_update);
+
+static __rte_always_inline int
+_cryptodev_dp_submit_single_job(struct rte_crypto_dp_service_ctx *ctx,
+		struct rte_crypto_vec *data, uint16_t n_data_vecs,
+		union rte_crypto_sym_ofs ofs,
+		union rte_crypto_sym_additional_data *additional_data,
+		void *opaque)
+{
+	return (*ctx->submit_single_job)(ctx->qp_data, ctx->drv_service_data,
+		data, n_data_vecs, ofs, additional_data, opaque);
+}
+
+static __rte_always_inline int
+_cryptodev_dp_sym_dequeue_single_job(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +25 -0
+		void **out_opaque)
+{
+	return (*ctx->dequeue_single)(ctx->qp_data, ctx->drv_service_data,
+		out_opaque);
+}
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_submit_done() is called. This is a
+ * simplified
+ *
+ * @param	ctx		The initialized data-path service context data.
+ * @param	data		The buffer vector.
+ * @param	n_data_vecs	Number of buffer vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	additional_data	IV, digest, and aad
+ * @param	opaque		The array of opaque data for dequeue.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_sym_submit_single_job(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +44 -0
+		struct rte_crypto_vec *data, uint16_t n_data_vecs,
+		union rte_crypto_sym_ofs ofs,
+		union rte_crypto_sym_additional_data *additional_data,
+		void *opaque);
+
+/**
+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_submit_done() is called.
+ *
+ * @param	ctx	The initialized data-path service context data.
+ * @param	vec	The array of job vectors.
+ * @param	ofs	Start and stop offsets for auth and cipher operations.
+ * @param	opaque	The array of opaque data for dequeue.
+ * @return
+ *   - The number of jobs successfully submitted.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_dp_sym_submit_vec(struct rte_crypto_dp_service_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **opaque);
+
+/**
+ * Command the queue pair to start processing all submitted jobs from last
+ * rte_cryptodev_init_dp_service() call.
+ *
+ * @param	ctx	The initialized data-path service context data.
+ * @param	n		The total number of submitted jobs.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_sym_submit_done(struct rte_crypto_dp_service_ctx *ctx,
+		uint32_t n);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	ctx			The initialized data-path service
+ *					context data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_opaque		Opaque pointer array to be retrieve

from
+ * device queue. In case of
+ * *is_opaque_array* is set there should
+ * be enough room to store all opaque
data.
+ * @param is_opaque_array Set 1 if every dequeued job will
be
+ * written the opaque data into
+ * *out_opaque* array.
+ * @param n_success_jobs Driver written value to specific the
+ * total successful operations count.
+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */

lib/librte_cryptodev/rte_cryptodev.h +25 -0
+__rte_experimental
+uint32_t
+rte_cryptodev_dp_sym_dequeue(struct rte_crypto_dp_service_ctx *ctx,
+	rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_post_dequeue_t post_dequeue,
+	void **out_opaque, uint8_t is_opaque_array,
+	uint32_t *n_success_jobs);
+
+/**
+ * Dequeue Single symmetric crypto processing of user provided data.
+ *
+ * @param	ctx			The initialized data-path service
+ *					context data.
+ * @param	out_opaque		Opaque pointer to be retrieve from
+ *					device queue. The driver shall support
+ *					NULL input of this parameter.
+ *
+ * @return
+ *   - 1 if the job is dequeued and the operation is a success.
+ *   - 0 if the job is dequeued but the operation is failed.
+ *   - -1 if no job is dequeued.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_sym_dequeue_single_job(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +13 -0
+		void **out_opaque);
+
+/**
+ * Inform the queue pair dequeue jobs finished.
+ *
+ * @param	ctx	The initialized data-path service context data.
+ * @param	n	The total number of jobs already dequeued.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_sym_dequeue_done(struct rte_crypto_dp_service_ctx *ctx,
+		uint32_t n);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +36 -0
index 81975d72b..e19de458c 100644
@@ -316,6 +316,42 @@ typedef uint32_t
 (*cryptodev_sym_cpu_crypto_process_t)
 	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
 	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
 
+/**
+ * Typedef that the driver provided to get service context private date size.
+ *
+ * @param	dev	Crypto device pointer.
+ *
+ * @return
+ *   - On success return the size of the device's service context private data.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_get_service_ctx_size_t)(
+	struct rte_cryptodev *dev);
+
+/**
+ * Typedef that the driver provided to configure data-path service.
+ *
+ * @param	dev		Crypto device pointer.
+ * @param	qp_id		Crypto device queue pair index.
+ * @param	service_type	Type of the service requested.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @param	ctx		The data-path service context data.
+ * @param	is_update	Set 1 if ctx is pre-initialized but need
+ *				update to different service type or session,
+ *				but the rest driver data remains the same.
+ *				buffer will always be one.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_configure_service_t)(
+	struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx,
+	uint8_t is_update);
 
 /** Crypto device operations function pointer table */

struct rte_cryptodev_ops {

lib/librte_cryptodev/rte_cryptodev_pmd.h +10 -2
@@ -348,8 +384,16 @@ struct rte_cryptodev_ops {
 	/**< Clear a Crypto sessions private data. */
 	cryptodev_asym_free_session_t asym_session_clear;
 	/**< Clear a Crypto sessions private data. */
-	cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
-	/**< process input data synchronously (cpu-crypto). */
+	union {
+		cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
+		/**< process input data synchronously (cpu-crypto). */
+		struct {
+			cryptodev_dp_get_service_ctx_size_t get_drv_ctx_size;
+			/**< Get data path service context data size. */
+			cryptodev_dp_configure_service_t configure_service;
+			/**< Initialize crypto service ctx data. */
+		};
+	};
 };
 
 
lib/librte_cryptodev/rte_cryptodev_version.map +0 -0
index 02f6dcf72..10388ae90 100644

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 110

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev_version.map +3 -0
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;

Rte_cryptodev_configure_raw_dp

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 111

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_get_service_ctx_data_size;

Rte_cryptodev_get_raw_dp_ctx_size

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 112

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_dequeue;

rte_cryptodev_raw_dequeue_burst

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 113

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_dequeue_done;

rte_cryptodev_raw_dequeue_done

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 114

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_dequeue_single_job;

rte_cryptodev_raw_dequeue

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 115

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_submit_done;

rte_cryptodev_raw_enqueue_done

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 116

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_submit_single_job;

rte_cryptodev_raw_enqueue

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 117

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_submit_vec;

rte_cryptodev_raw_enqueue_burst

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 118

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +0 -0
 };

Please use above names for the APIs.
No need for keyword dp in enq/deq APIs as it is implicit that enq/deq APIs are data path APIs.

I could not complete the review of this patch specifically as I see a lot of issues in the current version Please send reply to my queries so that review can be completed.

Regards,
Akhil

#22 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Akhil,

Thanks for the review!

Quoted 39 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Friday, September 18, 2020 10:50 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 17 lines

Subject: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

This patch adds data-path service APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors and operation mode.

---
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  49 ++-
 lib/librte_cryptodev/rte_cryptodev.c          |  98 +++++
 lib/librte_cryptodev/rte_cryptodev.h          | 335 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  48 ++-
 .../rte_cryptodev_version.map                 |  10 +
 6 files changed, 540 insertions(+), 9 deletions(-)

Review

Quoted 1 line
(quoted diff) +0 -0

b/lib/librte_cryptodev/rte_crypto.h

Quoted 8 lines

index fd5ef3a87..f009be9af 100644

lib/librte_cryptodev/rte_crypto.h +0 -0
@@ -438,6 +438,15 @@ rte_crypto_op_attach_asym_session(struct
 rte_crypto_op *op,
 	return 0;
 }
 
Quoted 14 lines

lib/librte_cryptodev/rte_crypto.h rte_crypto_op_attach_asym_session(struct rte_crypto_op *op, · line 448

Quoted 8 lines
lib/librte_cryptodev/rte_crypto.h +8 -0
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};

Comments missing for this enum.
Do we really need this enum?
Can we not have this info in the driver from the xform list?
And if we really want to add this, why to have it specific to raw data path APIs?

Will add comments to this enum.
Unless the driver will store xform data in certain way (in fact QAT has it) the driver may not know which data-path to choose from. The purpose of having this enum is that the driver knows to attach the correct handler into the service data structure fast.

Quoted 13 lines
Quoted 13 lines
lib/librte_cryptodev/rte_crypto.h +1 -0
+
 #ifdef __cplusplus
 }

#endif

lib/librte_cryptodev/rte_crypto_sym.h +0 -0
index f29c98051..376412e94 100644
@@ -50,6 +50,30 @@ struct rte_crypto_sgl {
 	uint32_t num;
 };
 
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 57

Quoted 5 lines
lib/librte_cryptodev/rte_crypto_sym.h +5 -0
+/**
+ * Symmetri Crypto Addtional Data other than src and destination data.
+ * Supposed to be used to pass IV/digest/aad data buffers with lengths
+ * defined when creating crypto session.
+ */

Fix typo

Thanks will change.

Quoted 12 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 66

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -0
+union rte_crypto_sym_additional_data {
+	struct {
+		void *cipher_iv_ptr;
+		rte_iova_t cipher_iv_iova;
+		void *auth_iv_ptr;
+		rte_iova_t auth_iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+	} cipher_auth;

Should be chain instead of cipher_auth

This field is used for cipher only, auth only, or chain use-cases so I believe this is a better name for it.

Quoted 31 lines
Quoted 31 lines
lib/librte_cryptodev/rte_crypto_sym.h +17 -6
+	struct {
+		void *iv_ptr;
+		rte_iova_t iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+		void *aad_ptr;
+		rte_iova_t aad_iova;
+	} aead;
+};
+
 /**
  * Synchronous operation descriptor.
  * Supposed to be used with CPU crypto API call.
@@ -57,12 +81,25 @@ struct rte_crypto_sgl {
 struct rte_crypto_sym_vec {
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
-	/** array of pointers to digest */
-	void **digest;
+
+	union {
+
+		/* Supposed to be used with CPU crypto API call. */
+		struct {
+			/** array of pointers to IV */
+			void **iv;
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 95

Quoted 5 lines
lib/librte_cryptodev/rte_crypto_sym.h +5 -0
+			/** array of pointers to AAD */
+			void **aad;
+			/** array of pointers to digest */
+			void **digest;
+		};

Can we also name this struct?
Probably we should split this as a separate patch.

[Then this is an API break right?]

Quoted 14 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 102

Quoted 9 lines
lib/librte_cryptodev/rte_crypto_sym.h +7 -0
+
+		/* Supposed to be used with
 rte_cryptodev_dp_sym_submit_vec()
+		 * call.
+		 */
+		union rte_crypto_sym_additional_data *additional_data;
+	};
+

Can we get rid of this unnecessary union rte_crypto_sym_additional_data And place chain and aead directly in the union? At any point, only one of the three
would be used.

We have 2 main different uses cases, 1 for cpu crypto and 1 for data path APIs. Within each main uses case there are 4 types of algo (cipher only/auth only/aead/chain), one requiring HW address and virtual address, the other doesn't. It seems to causing too much confusion to include these many union into the structure that initially was designed for cpu crypto only.
I suggest better to use different structure than squeeze all into a big union.

Quoted 30 lines

lib/librte_cryptodev/rte_cryptodev.c

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
 	/**
 	 * array of statuses for each operation:
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
 	 *  - 0 on success
lib/librte_cryptodev/rte_cryptodev.c +0 -0
index 1dd795bcb..4f59cf800 100644
@@ -1914,6 +1914,104 @@

rte_cryptodev_sym_cpu_crypto_process(uint8_t

Quoted 19 lines
lib/librte_cryptodev/rte_cryptodev.c +14 -0
 dev_id,
 	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);
 }
 
+int
+rte_cryptodev_dp_get_service_ctx_data_size(uint8_t dev_id)
+{
+	struct rte_cryptodev *dev;
+	int32_t size = sizeof(struct rte_crypto_dp_service_ctx);
+	int32_t priv_size;
+
+	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+		return -1;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+	if (*dev->dev_ops->get_drv_ctx_size == NULL ||
+		!(dev->feature_flags &
 RTE_CRYPTODEV_FF_DATA_PATH_SERVICE)) {
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.c line 1934

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.c +2 -0
+		return -1;
+	}

I have some suggestions for the naming of the APIs / flags in the doc patch, Please check that and make changes in this patch.
Also, you have missed adding this feature flag in the doc/guides/cryptodevs/features/default.ini file.
And Subsequently in the doc/guides/cryptodevs/features/qat.ini file.

Will update. Thanks a lot!

Quoted 23 lines
Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.c +12 -0
+
+	priv_size = (*dev->dev_ops->get_drv_ctx_size)(dev);
+	if (priv_size < 0)
+		return -1;
+
+	return RTE_ALIGN_CEIL((size + priv_size), 8);
+}
+
+int
+rte_cryptodev_dp_configure_service(uint8_t dev_id, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx, uint8_t is_update)
+{
+	struct rte_cryptodev *dev;
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -1;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags &

RTE_CRYPTODEV_FF_DATA_PATH_SERVICE)

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1956

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.c +2 -0
+			|| dev->dev_ops->configure_service == NULL)
+		return -1;

It would be better to return actual error number like ENOTSUP/EINVAL. It would be helpful in debugging.

Will change.

Quoted 10 lines

lib/librte_cryptodev/rte_cryptodev.c line 1960

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev.c +2 -0
+
+	return (*dev->dev_ops->configure_service)(dev, qp_id,

service_type,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.c +5 -0
+			sess_type, session_ctx, ctx, is_update);
+}
+
+int
+rte_cryptodev_dp_sym_submit_single_job(struct

rte_crypto_dp_service_ctx

Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1968

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.c +5 -0
 *ctx,
+		struct rte_crypto_vec *data, uint16_t n_data_vecs,
+		union rte_crypto_sym_ofs ofs,
+		union rte_crypto_sym_additional_data *additional_data,
+		void *opaque)
+{

Can we have some debug checks for NULL checking.

Will do.

Quoted 10 lines

lib/librte_cryptodev/rte_cryptodev.c line 1972

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.c +1 -0
+	return _cryptodev_dp_submit_single_job(ctx, data, n_data_vecs,

ofs,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1970

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.c +1 -0
+			additional_data, opaque);

Unnecessary function call _cryptodev_dp_submit_single_job. You can directly call
return (*ctx->submit_single_job)(ctx->qp_data, ctx->drv_service_data,

		data, n_data_vecs, ofs, additional_data, opaque);

Will change.

Quoted 73 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.c +4 -0
+}
+
+uint32_t
+rte_cryptodev_dp_sym_submit_vec(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 9 lines
lib/librte_cryptodev/rte_cryptodev.c +9 -0
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **opaque)
+{
+	return (*ctx->submit_vec)(ctx->qp_data, ctx->drv_service_data, vec,
+			ofs, opaque);
+}
+
+int
+rte_cryptodev_dp_sym_dequeue_single_job(struct

rte_crypto_dp_service_ctx

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1986

Quoted 4 lines

*ctx,

lib/librte_cryptodev/rte_cryptodev.c +3 -0
+		void **out_opaque)
+{
+	return _cryptodev_dp_sym_dequeue_single_job(ctx, out_opaque);

Same here.

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.c +4 -0
+}
+
+int
+rte_cryptodev_dp_sym_submit_done(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev.c +3 -0
+		uint32_t n)
+{
+	return (*ctx->submit_done)(ctx->qp_data, ctx->drv_service_data,

n);

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.c +4 -0
+}
+
+int
+rte_cryptodev_dp_sym_dequeue_done(struct

rte_crypto_dp_service_ctx *ctx,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev.c +3 -0
+		uint32_t n)
+{
+	return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_service_data,

n);

Quoted 25 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+}
+
+uint32_t
+rte_cryptodev_dp_sym_dequeue(struct rte_crypto_dp_service_ctx *ctx,
+	rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_post_dequeue_t post_dequeue,
+	void **out_opaque, uint8_t is_opaque_array,
+	uint32_t *n_success_jobs)
+{
+	return (*ctx->dequeue_opaque)(ctx->qp_data, ctx-

drv_service_data,

lib/librte_cryptodev/rte_cryptodev.c +1 -0
+		get_dequeue_count, post_dequeue, out_opaque,

is_opaque_array,

lib/librte_cryptodev/rte_cryptodev.c +3 -0
+		n_success_jobs);
+}
+
 /** Initialise rte_crypto_op mempool element */
 static void
 rte_crypto_op_init(struct rte_mempool *mempool,
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index 7b3ebc20f..4da0389d1 100644

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_asym_get_xform_enum(enum · line 471

Quoted 7 lines
lib/librte_cryptodev/rte_cryptodev.h +1 -1
@@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum
 rte_crypto_asym_xform_type *xform_enum,
 /**< Support symmetric session-less operations */
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL
 << 23)
 /**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_DATA_PATH_SERVICE		(1ULL

<< 24)

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_asym_get_xform_enum(enum rte_crypto_asym_xform_type *xform_enum, · line 470

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.h +1 -0
+/**< Support accelerated specific raw data as input */

Support data path APIs for raw data as input.

Will update.

Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.h

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 /**

* Get the name of a crypto device feature flag

lib/librte_cryptodev/rte_cryptodev.h +0 -0
@@ -1351,6 +1352,338 @@

rte_cryptodev_sym_cpu_crypto_process(uint8_t

lib/librte_cryptodev/rte_cryptodev.h line 1353

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 dev_id,
 	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs
Quoted 7 lines

ofs,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1356

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +2 -0
 	struct rte_crypto_sym_vec *vec);
 
+/**
+ * Get the size of the data-path service context for all registered drivers.

For all drivers ? or for a device?

For a device - sorry for the typo.

Quoted 17 lines
Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ *
+ * @param	dev_id		The device identifier.
+ *
+ * @return

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1370

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   - If the device supports data-path service, return the context size.
+ *   - If the device does not support the data-dath service, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_get_service_ctx_data_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less xform
+ * pointer.

Union of different symmetric crypto session types ..

Sorry, will change.

Quoted 20 lines
Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +7 -0
+ */
+union rte_cryptodev_session_ctx {
+	struct rte_cryptodev_sym_session *crypto_sess;
+	struct rte_crypto_sym_xform *xform;
+	struct rte_security_session *sec_sess;
+};
+

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+/**
+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	service_data	Driver specific service data.
+ * @param	vec		The array of job vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	opaque		The array of opaque data for

dequeue.

Quoted 1 line

Can you elaborate the usage of opaque here?

User provided data that wants to retrieve when dequeue. Will do.

Quoted 69 lines
Quoted 37 lines
lib/librte_cryptodev/rte_cryptodev.h +36 -0
+ * @return
+ *   - The number of jobs successfully submitted.
+ */
+typedef uint32_t (*cryptodev_dp_sym_submit_vec_t)(
+	void *qp, uint8_t *service_data, struct rte_crypto_sym_vec *vec,
+	union rte_crypto_sym_ofs ofs, void **opaque);
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_sym_submit_vec() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	service_data	Driver specific service data.
+ * @param	data		The buffer vector.
+ * @param	n_data_vecs	Number of buffer vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	additional_data	IV, digest, and aad data.
+ * @param	opaque		The opaque data for dequeue.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_submit_single_job_t)(
+	void *qp, uint8_t *service_data, struct rte_crypto_vec *data,
+	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+	union rte_crypto_sym_additional_data *additional_data,
+	void *opaque);
+
+/**
+ * Inform the queue pair to start processing or finish dequeuing all
+ * submitted/dequeued jobs.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	service_data	Driver specific service data.
+ * @param	n		The total number of submitted jobs.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_sym_operation_done_t)(void *qp,
+		uint8_t *service_data, uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from

the

Quoted 6 lines

opaque
+ * data stored in the first processed job.
+ *
+ * @param opaque Dequeued opaque data.
+ **/
+typedef uint32_t (*rte_cryptodev_get_dequeue_count_t)(void

*opaque);

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev.h +3 -0
+
+/**
+ * Typedef that the user provided to deal with post dequeue operation,

such

Quoted 9 lines

+ * as filling status.
+ *
+ * @param opaque Dequeued opaque data. In case
+ *
RTE_CRYPTO_HW_DP_FF_GET_OPAQUE_ARRAY bit is
+ * set, this value will be the opaque data stored
+ * in the specific processed jobs referenced by
+ * index, otherwise it will be the opaque data
+ * stored in the first processed job in the burst.

Quoted 2 lines

What is RTE_CRYPTO_HW_DP_FF_GET_OPAQUE_ARRAY, I did not find this in the series.

Will remove.

Quoted 81 lines
Quoted 5 lines

+ * @param index Index number of the processed job.
+ * @param is_op_success Driver filled operation status.
+ **/
+typedef void (*rte_cryptodev_post_dequeue_t)(void *opaque, uint32_t

index,

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.h +12 -0
+		uint8_t is_op_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	service_data		Driver specific service data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_opaque		Opaque pointer array to be retrieve
Quoted 3 lines

from
+ * device queue. In case of
+ * *is_opaque_array* is set there

should

Quoted 3 lines

+ * be enough room to store all opaque
data.
+ * @param is_opaque_array Set 1 if every dequeued job

will

Quoted 4 lines

be
+ * written the opaque data into
+ * *out_opaque* array.
+ * @param n_success_jobs Driver written value to

specific the

Quoted 36 lines

+ * total successful operations count.
+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */
+typedef uint32_t (*cryptodev_dp_sym_dequeue_t)(void *qp, uint8_t *service_data,

lib/librte_cryptodev/rte_cryptodev.h +29 -0
+	rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_post_dequeue_t post_dequeue,
+	void **out_opaque, uint8_t is_opaque_array,
+	uint32_t *n_success_jobs);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	service_data		Driver specific service data.
+ * @param	out_opaque		Opaque pointer to be retrieve from
+ *					device queue.
+ *
+ * @return
+ *   - 1 if the job is dequeued and the operation is a success.
+ *   - 0 if the job is dequeued but the operation is failed.
+ *   - -1 if no job is dequeued.
+ */
+typedef int (*cryptodev_dp_sym_dequeue_single_job_t)(
+		void *qp, uint8_t *service_data, void **out_opaque);
+
+/**
+ * Context data for asynchronous crypto process.
+ */
+struct rte_crypto_dp_service_ctx {
+	void *qp_data;
+
+	struct {
+		cryptodev_dp_submit_single_job_t submit_single_job;

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1517

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+		cryptodev_dp_sym_submit_vec_t submit_vec;
+		cryptodev_dp_sym_operation_done_t submit_done;
+		cryptodev_dp_sym_dequeue_t dequeue_opaque;
+		cryptodev_dp_sym_dequeue_single_job_t dequeue_single;
+		cryptodev_dp_sym_operation_done_t dequeue_done;
+	};
+
+	/* Driver specific service data */
+	__extension__ uint8_t drv_service_data[];
+};

Comments missing for structure params.
Struct name can be rte_crypto_raw_dp_ctx.

Quoted 1 line

Who allocate and free this structure?

Same as crypto session, the user need to query the driver specific service data Size and allocate the buffer accordingly. The difference is it does not have to Be from mempool as it can be reused.

Quoted 309 lines

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+
+/**
+ * Configure one DP service context data. Calling this function for the first
+ * time the user should unset the *is_update* parameter and the driver

will

Quoted 2 lines

+ * fill necessary operation data into ctx buffer. Only when
+ * rte_cryptodev_dp_submit_done() is called the data stored in the ctx

buffer

Quoted 5 lines

+ * will not be effective.
+ *
+ * @param dev_id The device identifier.
+ * @param qp_id The index of the queue pair from which to
+ * retrieve processed packets. The value must

be

Quoted 1 line

+ * in the range [0, nb_queue_pair - 1]

previously

Quoted 17 lines

+ * supplied to rte_cryptodev_configure().
+ * @param service_type Type of the service requested.
+ * @param sess_type session type.
+ * @param session_ctx Session context data.
+ * @param ctx The data-path service context data.
+ * @param is_update Set 1 if ctx is pre-initialized but need
+ * update to different service type or session,
+ * but the rest driver data remains the same.
+ * Since service context data buffer is provided
+ * by user, the driver will not check the
+ * validity of the buffer nor its content. It is
+ * the user's obligation to initialize and
+ * uses the buffer properly by setting this field.
+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+__rte_experimental
+int
+rte_cryptodev_dp_configure_service(uint8_t dev_id, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx, uint8_t is_update);
+
+static __rte_always_inline int
+_cryptodev_dp_submit_single_job(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.h +6 -0
+		struct rte_crypto_vec *data, uint16_t n_data_vecs,
+		union rte_crypto_sym_ofs ofs,
+		union rte_crypto_sym_additional_data *additional_data,
+		void *opaque)
+{
+	return (*ctx->submit_single_job)(ctx->qp_data, ctx-

drv_service_data,

lib/librte_cryptodev/rte_cryptodev.h +5 -0
+		data, n_data_vecs, ofs, additional_data, opaque);
+}
+
+static __rte_always_inline int
+_cryptodev_dp_sym_dequeue_single_job(struct

rte_crypto_dp_service_ctx

Quoted 9 lines

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +8 -0
+		void **out_opaque)
+{
+	return (*ctx->dequeue_single)(ctx->qp_data, ctx->drv_service_data,
+		out_opaque);
+}
+
+/**
+ * Submit single job into device queue but the driver will not start

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ * processing until rte_cryptodev_dp_submit_done() is called. This is a
+ * simplified
+ *
+ * @param	ctx		The initialized data-path service context data.
+ * @param	data		The buffer vector.
+ * @param	n_data_vecs	Number of buffer vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	additional_data	IV, digest, and aad
+ * @param	opaque		The array of opaque data for

dequeue.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 7 lines

+ * @return
+ * - On success return 0.
+ * - On failure return negative integer.
+ */

lib/librte_cryptodev/rte_cryptodev.h +3 -0
+__rte_experimental
+int
+rte_cryptodev_dp_sym_submit_single_job(struct

rte_crypto_dp_service_ctx

Quoted 10 lines

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +9 -0
+		struct rte_crypto_vec *data, uint16_t n_data_vecs,
+		union rte_crypto_sym_ofs ofs,
+		union rte_crypto_sym_additional_data *additional_data,
+		void *opaque);
+
+/**
+ * Submit a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_submit_done() is called.
+ *

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ * @param	ctx	The initialized data-path service context data.
+ * @param	vec	The array of job vectors.
+ * @param	ofs	Start and stop offsets for auth and cipher operations.
+ * @param	opaque	The array of opaque data for dequeue.
+ * @return
+ *   - The number of jobs successfully submitted.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_dp_sym_submit_vec(struct rte_crypto_dp_service_ctx

*ctx,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **opaque);
+
+/**
+ * Command the queue pair to start processing all submitted jobs from

last

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 8 lines

+ * rte_cryptodev_init_dp_service() call.
+ *
+ * @param ctx The initialized data-path service context data.
+ * @param n The total number of submitted jobs.
+ */

lib/librte_cryptodev/rte_cryptodev.h +3 -0
+__rte_experimental
+int
+rte_cryptodev_dp_sym_submit_done(struct rte_crypto_dp_service_ctx

*ctx,

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.h +12 -0
+		uint32_t n);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	ctx			The initialized data-path service
+ *					context data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_opaque		Opaque pointer array to be retrieve
Quoted 3 lines

from
+ * device queue. In case of
+ * *is_opaque_array* is set there

should

Quoted 3 lines

+ * be enough room to store all opaque
data.
+ * @param is_opaque_array Set 1 if every dequeued job

will

Quoted 4 lines

be
+ * written the opaque data into
+ * *out_opaque* array.
+ * @param n_success_jobs Driver written value to

specific the

Quoted 10 lines

+ * total successful operations count.
+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */

lib/librte_cryptodev/rte_cryptodev.h +5 -0
+__rte_experimental
+uint32_t
+rte_cryptodev_dp_sym_dequeue(struct rte_crypto_dp_service_ctx *ctx,
+	rte_cryptodev_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_post_dequeue_t post_dequeue,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+	void **out_opaque, uint8_t is_opaque_array,
+	uint32_t *n_success_jobs);
+
+/**
+ * Dequeue Single symmetric crypto processing of user provided data.
+ *
+ * @param	ctx			The initialized data-path service
+ *					context data.
+ * @param	out_opaque		Opaque pointer to be retrieve from
+ *					device queue. The driver shall

support

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines

+ * NULL input of this parameter.
+ *
+ * @return

+ *   - 1 if the job is dequeued and the operation is a success.
+ *   - 0 if the job is dequeued but the operation is failed.
+ *   - -1 if no job is dequeued.

+ */

lib/librte_cryptodev/rte_cryptodev.h +3 -0
+__rte_experimental
+int
+rte_cryptodev_dp_sym_dequeue_single_job(struct

rte_crypto_dp_service_ctx

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 12 lines

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +11 -0
+		void **out_opaque);
+
+/**
+ * Inform the queue pair dequeue jobs finished.
+ *
+ * @param	ctx	The initialized data-path service context data.
+ * @param	n	The total number of jobs already dequeued.
+ */
+__rte_experimental
+int
+rte_cryptodev_dp_sym_dequeue_done(struct

rte_crypto_dp_service_ctx *ctx,

Quoted 11 lines
lib/librte_cryptodev/rte_cryptodev.h +2 -0
+		uint32_t n);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index 81975d72b..e19de458c 100644

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef uint32_t · line 321

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +2 -0
@@ -316,6 +316,42 @@ typedef uint32_t
 (*cryptodev_sym_cpu_crypto_process_t)
 	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
 	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
 
+/**
+ * Typedef that the driver provided to get service context private date

size.

Quoted 38 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +34 -0
+ *
+ * @param	dev	Crypto device pointer.
+ *
+ * @return
+ *   - On success return the size of the device's service context private data.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_get_service_ctx_size_t)(
+	struct rte_cryptodev *dev);
+
+/**
+ * Typedef that the driver provided to configure data-path service.
+ *
+ * @param	dev		Crypto device pointer.
+ * @param	qp_id		Crypto device queue pair index.
+ * @param	service_type	Type of the service requested.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @param	ctx		The data-path service context data.
+ * @param	is_update	Set 1 if ctx is pre-initialized but need
+ *				update to different service type or session,
+ *				but the rest driver data remains the same.
+ *				buffer will always be one.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_configure_service_t)(
+	struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *ctx,
+	uint8_t is_update);
 
 /** Crypto device operations function pointer table */

struct rte_cryptodev_ops {

lib/librte_cryptodev/rte_cryptodev_pmd.h struct rte_cryptodev_ops { · line 391

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +5 -2
@@ -348,8 +384,16 @@ struct rte_cryptodev_ops {
 	/**< Clear a Crypto sessions private data. */
 	cryptodev_asym_free_session_t asym_session_clear;
 	/**< Clear a Crypto sessions private data. */
-	cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
-	/**< process input data synchronously (cpu-crypto). */
+	union {
+		cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
+		/**< process input data synchronously (cpu-crypto). */
+		struct {
+			cryptodev_dp_get_service_ctx_size_t

get_drv_ctx_size;

lib/librte_cryptodev/rte_cryptodev_pmd.h struct rte_cryptodev_ops { · line 393

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +2 -0
+			/**< Get data path service context data size. */
+			cryptodev_dp_configure_service_t

configure_service;

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+			/**< Initialize crypto service ctx data. */
+		};
+	};
 };
 
 
lib/librte_cryptodev/rte_cryptodev_version.map +0 -0
index 02f6dcf72..10388ae90 100644

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 110

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev_version.map +3 -0
@@ -105,4 +105,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_dp_configure_service;

Rte_cryptodev_configure_raw_dp

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 111

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_get_service_ctx_data_size;

Rte_cryptodev_get_raw_dp_ctx_size

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 112

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_dequeue;

rte_cryptodev_raw_dequeue_burst

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 113

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_dequeue_done;

rte_cryptodev_raw_dequeue_done

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 114

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_dequeue_single_job;

rte_cryptodev_raw_dequeue

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 115

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_submit_done;

rte_cryptodev_raw_enqueue_done

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 116

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_submit_single_job;

rte_cryptodev_raw_enqueue

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 117

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +1 -0
+	rte_cryptodev_dp_sym_submit_vec;

rte_cryptodev_raw_enqueue_burst

Quoted 6 lines

lib/librte_cryptodev/rte_cryptodev_version.map EXPERIMENTAL { · line 118

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev_version.map +0 -0
 };

Please use above names for the APIs.
No need for keyword dp in enq/deq APIs as it is implicit that enq/deq APIs are data path APIs.

Will do.

Quoted 7 lines

I could not complete the review of this patch specifically as I see a lot of issues in the current version
Please send reply to my queries so that review can be completed.

Regards,
Akhil

#23 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 15 lines
Quoted 15 lines

lib/librte_cryptodev/rte_crypto.h rte_crypto_op_attach_asym_session(struct rte_crypto_op *op, · line 448

Quoted 9 lines
lib/librte_cryptodev/rte_crypto.h +8 -0
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};

Comments missing for this enum.
Do we really need this enum?
Can we not have this info in the driver from the xform list?
And if we really want to add this, why to have it specific to raw data path APIs?

Quoted 6 lines

Will add comments to this enum.
Unless the driver will store xform data in certain way (in fact QAT has it) the driver may not know which data-path to choose from.
The purpose of having this enum is that the driver knows to attach the correct handler into the service data structure fast.

I believe all drivers are storing that information already in some way in the session private data. This enum is maintained inside driver as of current implementation. This is not specific to raw Data path APIs. If you are introducing this enum in library, then it should be generic for the legacy Case as well.

Quoted 12 lines
Quoted 12 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 66

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -0
+union rte_crypto_sym_additional_data {
+	struct {
+		void *cipher_iv_ptr;
+		rte_iova_t cipher_iv_iova;
+		void *auth_iv_ptr;
+		rte_iova_t auth_iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+	} cipher_auth;

Should be chain instead of cipher_auth

Quoted 2 lines

This field is used for cipher only, auth only, or chain use-cases so I believe this is a better name for it.

Agreed that this struct will be used for all 3 cases, that is what is happening in Other crypto cases. We use chain for all these three cases in legacy codepath. Chain can be of one or two xforms and ordering can be anything - Cipher only, auth only, cipher auth and auth cipher.

Quoted 39 lines
Quoted 31 lines
Quoted 31 lines
lib/librte_cryptodev/rte_crypto_sym.h +17 -6
+	struct {
+		void *iv_ptr;
+		rte_iova_t iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+		void *aad_ptr;
+		rte_iova_t aad_iova;
+	} aead;
+};
+
 /**
  * Synchronous operation descriptor.
  * Supposed to be used with CPU crypto API call.
@@ -57,12 +81,25 @@ struct rte_crypto_sgl {
 struct rte_crypto_sym_vec {
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
-	/** array of pointers to digest */
-	void **digest;
+
+	union {
+
+		/* Supposed to be used with CPU crypto API call. */
+		struct {
+			/** array of pointers to IV */
+			void **iv;
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 95

Quoted 5 lines
lib/librte_cryptodev/rte_crypto_sym.h +5 -0
+			/** array of pointers to AAD */
+			void **aad;
+			/** array of pointers to digest */
+			void **digest;
+		};

Can we also name this struct?
Probably we should split this as a separate patch.

Quoted 1 line

[Then this is an API break right?]

Since this an LTS release, I am ok to take this change. But others can comment on this.
@Ananyev, Konstantin, @Thomas Monjalon
Can you comment on this?

Quoted 14 lines
Quoted 14 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 102

Quoted 9 lines
lib/librte_cryptodev/rte_crypto_sym.h +7 -0
+
+		/* Supposed to be used with
 rte_cryptodev_dp_sym_submit_vec()
+		 * call.
+		 */
+		union rte_crypto_sym_additional_data *additional_data;
+	};
+

Can we get rid of this unnecessary union rte_crypto_sym_additional_data And place chain and aead directly in the union? At any point, only one of the three
would be used.

Quoted 8 lines

We have 2 main different uses cases, 1 for cpu crypto and 1 for data path APIs. Within each main uses case there are 4 types of algo (cipher only/auth only/aead/chain), one requiring HW address and virtual address, the other doesn't.
It seems to causing too much confusion to include these many union into the structure that initially was designed for cpu crypto only.
I suggest better to use different structure than squeeze all into a big union.

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name> 
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};
Quoted 22 lines
Quoted 22 lines
Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +8 -0
+/**
+ * Context data for asynchronous crypto process.
+ */
+struct rte_crypto_dp_service_ctx {
+	void *qp_data;
+
+	struct {
+		cryptodev_dp_submit_single_job_t submit_single_job;

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1517

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+		cryptodev_dp_sym_submit_vec_t submit_vec;
+		cryptodev_dp_sym_operation_done_t submit_done;
+		cryptodev_dp_sym_dequeue_t dequeue_opaque;
+		cryptodev_dp_sym_dequeue_single_job_t dequeue_single;
+		cryptodev_dp_sym_operation_done_t dequeue_done;
+	};
+
+	/* Driver specific service data */
+	__extension__ uint8_t drv_service_data[];
+};

Comments missing for structure params.
Struct name can be rte_crypto_raw_dp_ctx.

Quoted 4 lines
Quoted 1 line

Who allocate and free this structure?

Same as crypto session, the user need to query the driver specific service data Size and allocate the buffer accordingly. The difference is it does not have to Be from mempool as it can be reused.

So this structure is saved and filled by the lib/driver and not the application. Right? This struct is opaque to application and will be part of session private data. Right? Assignment and calling appropriate driver's call backs will be hidden inside library and will be opaque to the application. In other words, the structure is not exposed to the application.
Please add relevant comments on top of this structure.

Quoted 14 lines
Quoted 14 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+static __rte_always_inline int
+_cryptodev_dp_sym_dequeue_single_job(struct

rte_crypto_dp_service_ctx

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1577

Quoted 11 lines

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +10 -0
+		void **out_opaque)
+{
+	return (*ctx->dequeue_single)(ctx->qp_data, ctx->drv_service_data,
+		out_opaque);
+}
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_submit_done() is called. This is a
+ * simplified

Comment not complete.

Regards,
Akhil

#24 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Akhil,

Quoted 37 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Monday, September 21, 2020 1:00 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 14 lines
Quoted 14 lines

lib/librte_cryptodev/rte_crypto.h rte_crypto_op_attach_asym_session(struct rte_crypto_op *op, · line 448

Quoted 9 lines
lib/librte_cryptodev/rte_crypto.h +8 -0
+/** Crypto data-path service types */
+enum rte_crypto_dp_service {
+	RTE_CRYPTO_DP_SYM_CIPHER_ONLY = 0,
+	RTE_CRYPTO_DP_SYM_AUTH_ONLY,
+	RTE_CRYPTO_DP_SYM_CHAIN,
+	RTE_CRYPTO_DP_SYM_AEAD,
+	RTE_CRYPTO_DP_N_SERVICE
+};

Comments missing for this enum.
Do we really need this enum?
Can we not have this info in the driver from the xform list? And if we really want to add this, why to have it specific to raw data path

APIs?

Quoted 5 lines
Quoted 1 line

Will add comments to this enum.
Unless the driver will store xform data in certain way (in fact QAT has it) the driver may not know which data-path to choose from.
The purpose of having this enum is that the driver knows to attach the

correct

Quoted 2 lines

handler into the service data structure fast.

Quoted 8 lines

I believe all drivers are storing that information already in some way in the session private data.
This enum is maintained inside driver as of current implementation. This is not specific to raw
Data path APIs. If you are introducing this enum in library, then it should be generic for the legacy
Case as well.

[Fan: I am not sure other drivers (that's part of the reason why it is here), but indeed QAT does, Ok]

Quoted 121 lines
Quoted 13 lines
Quoted 13 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 66

Quoted 11 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -0
+union rte_crypto_sym_additional_data {
+	struct {
+		void *cipher_iv_ptr;
+		rte_iova_t cipher_iv_iova;
+		void *auth_iv_ptr;
+		rte_iova_t auth_iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+	} cipher_auth;

Should be chain instead of cipher_auth

Quoted 1 line

This field is used for cipher only, auth only, or chain use-cases so I believe

this is

Quoted 1 line

a better name for it.

Agreed that this struct will be used for all 3 cases, that is what is happening in Other crypto cases. We use chain for all these three cases in legacy codepath. Chain can be of one or two xforms and ordering can be anything - Cipher only, auth only, cipher auth and auth cipher.

Quoted 39 lines
Quoted 31 lines
Quoted 31 lines
lib/librte_cryptodev/rte_crypto_sym.h +17 -6
+	struct {
+		void *iv_ptr;
+		rte_iova_t iv_iova;
+		void *digest_ptr;
+		rte_iova_t digest_iova;
+		void *aad_ptr;
+		rte_iova_t aad_iova;
+	} aead;
+};
+
 /**
  * Synchronous operation descriptor.
  * Supposed to be used with CPU crypto API call.
@@ -57,12 +81,25 @@ struct rte_crypto_sgl {
 struct rte_crypto_sym_vec {
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
-	/** array of pointers to digest */
-	void **digest;
+
+	union {
+
+		/* Supposed to be used with CPU crypto API call. */
+		struct {
+			/** array of pointers to IV */
+			void **iv;
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 95

Quoted 5 lines
lib/librte_cryptodev/rte_crypto_sym.h +5 -0
+			/** array of pointers to AAD */
+			void **aad;
+			/** array of pointers to digest */
+			void **digest;
+		};

Can we also name this struct?
Probably we should split this as a separate patch.

Quoted 1 line

[Then this is an API break right?]

Since this an LTS release, I am ok to take this change. But others can comment on this.
@Ananyev, Konstantin, @Thomas Monjalon
Can you comment on this?

Quoted 11 lines
Quoted 11 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 102

Quoted 9 lines
lib/librte_cryptodev/rte_crypto_sym.h +7 -0
+
+		/* Supposed to be used with
 rte_cryptodev_dp_sym_submit_vec()
+		 * call.
+		 */
+		union rte_crypto_sym_additional_data *additional_data;
+	};
+

Can we get rid of this unnecessary union

rte_crypto_sym_additional_data

Quoted 1 line
Quoted 1 line

And place chain and aead directly in the union? At any point, only one of

the

Quoted 3 lines
Quoted 2 lines

three
would be used.

We have 2 main different uses cases, 1 for cpu crypto and 1 for data path

APIs.

Quoted 4 lines

Within each main uses case there are 4 types of algo (cipher only/auth only/aead/chain), one requiring HW address and virtual address, the other doesn't.
It seems to causing too much confusion to include these many union into

the

Quoted 3 lines

structure that initially was designed for cpu crypto only.
I suggest better to use different structure than squeeze all into a big union.

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name>
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
Quoted 10 lines
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};

[Structure looks good to me thanks!]

Quoted 30 lines
Quoted 23 lines
Quoted 23 lines
Quoted 9 lines
lib/librte_cryptodev/rte_cryptodev.h +8 -0
+/**
+ * Context data for asynchronous crypto process.
+ */
+struct rte_crypto_dp_service_ctx {
+	void *qp_data;
+
+	struct {
+		cryptodev_dp_submit_single_job_t submit_single_job;

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1517

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+		cryptodev_dp_sym_submit_vec_t submit_vec;
+		cryptodev_dp_sym_operation_done_t submit_done;
+		cryptodev_dp_sym_dequeue_t dequeue_opaque;
+		cryptodev_dp_sym_dequeue_single_job_t dequeue_single;
+		cryptodev_dp_sym_operation_done_t dequeue_done;
+	};
+
+	/* Driver specific service data */
+	__extension__ uint8_t drv_service_data[];
+};

Comments missing for structure params.
Struct name can be rte_crypto_raw_dp_ctx.

Quoted 2 lines
Quoted 1 line

Who allocate and free this structure?

Same as crypto session, the user need to query the driver specific service

data

Quoted 1 line

Size and allocate the buffer accordingly. The difference is it does not have

to

Quoted 1 line

Be from mempool as it can be reused.

Quoted 11 lines

So this structure is saved and filled by the lib/driver and not the application. Right?
This struct is opaque to application and will be part of session private data. Right?
Assignment and calling appropriate driver's call backs will be hidden inside library
and will be opaque to the application. In other words, the structure is not exposed
to the application.
Please add relevant comments on top of this structure.

[Fan: will do]

Quoted 20 lines
Quoted 4 lines
Quoted 4 lines

Review

Quoted 3 lines
(quoted diff) +2 -0
+static __rte_always_inline int
+_cryptodev_dp_sym_dequeue_single_job(struct

rte_crypto_dp_service_ctx

Quoted 11 lines
Quoted 11 lines

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1577

Quoted 11 lines

*ctx,

lib/librte_cryptodev/rte_cryptodev.h +10 -0
+		void **out_opaque)
+{
+	return (*ctx->dequeue_single)(ctx->qp_data, ctx->drv_service_data,
+		out_opaque);
+}
+
+/**
+ * Submit single job into device queue but the driver will not start
+ * processing until rte_cryptodev_dp_submit_done() is called. This is a
+ * simplified

Comment not complete.

Regards,
Akhil

#25 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

In reply to Akhil Goyal

Hi AKhil

Quoted 12 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Monday, September 21, 2020 1:00 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

...

Quoted 25 lines

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name>
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
Quoted 10 lines
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};

The above structure cannot support the array of multiple jobs but a single job. So we have to use something like

struct {
	void **cipher_iv_ptr;
	rtei_iova_t *cipher_iv_iova;
	...
} hw_chain;
struct {
	void **iv_ptr;
	rte_iova_t *iv_iova;
	...
} hw_aead;

Is it ok?

Regards,
Fan

#26 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 39 lines

Hi AKhil

...

Quoted 25 lines

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name>
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
Quoted 10 lines
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};
Quoted 1 line

The above structure cannot support the array of multiple jobs but a single job.

So was your previous structure. Was it not tested before?

Quoted 4 lines

So we have to use something like

struct {
	void **cipher_iv_ptr;

You can even drop _ptr from the name of each of them.

Quoted 13 lines
	rtei_iova_t *cipher_iv_iova;
	...
} hw_chain;
struct {
	void **iv_ptr;
	rte_iova_t *iv_iova;
	...
} hw_aead;

Is it ok?

Regards,
Fan

#27 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Akhil,

Quoted 52 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Monday, September 21, 2020 4:49 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 39 lines

Hi AKhil

...

Quoted 35 lines

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name>
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};
Quoted 4 lines
Quoted 1 line

The above structure cannot support the array of multiple jobs but a single

job.

So was your previous structure. Was it not tested before?

Of course it was tested in both DPDK and VPP. The previous structure is an array to the union additional_data " union rte_crypto_sym_additional_data *additional_data;". That's why the union was declared in the first place. I am ok with the way you propose with "*" in the beginning of every member.

Quoted 21 lines
Quoted 5 lines

So we have to use something like

struct {
	void **cipher_iv_ptr;

You can even drop _ptr from the name of each of them.

Quoted 13 lines
	rtei_iova_t *cipher_iv_iova;
	...
} hw_chain;
struct {
	void **iv_ptr;
	rte_iova_t *iv_iova;
	...
} hw_aead;

Is it ok?

Regards,
Fan

Regards,
Fan

#28 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

In reply to Akhil Goyal

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and documentation.
4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};

5. Remove enum rte_crypto_dp_service, let the PMDs using the session private data to decide function handler.
6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single".
I am ok to remove "submit_single" function. In VPP we can use rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to avoid double looping. But we have to put the rte_cryptodev_dp_sym_submit_vec() as an inline function - this will cause the API not traced in version map.

Any ideas?

Regards,
Fan

Quoted 77 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Monday, September 21, 2020 4:49 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 39 lines

Hi AKhil

...

Quoted 25 lines

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name>
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
Quoted 10 lines
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};
Quoted 1 line

The above structure cannot support the array of multiple jobs but a single

job.

So was your previous structure. Was it not tested before?

Quoted 4 lines

So we have to use something like

struct {
	void **cipher_iv_ptr;

You can even drop _ptr from the name of each of them.

Quoted 13 lines
	rtei_iova_t *cipher_iv_iova;
	...
} hw_chain;
struct {
	void **iv_ptr;
	rte_iova_t *iv_iova;
	...
} hw_aead;

Is it ok?

Regards,
Fan

#29 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi lads,

Quoted 57 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and documentation.
4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
Quoted 8 lines
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};

As I understand you just need to add pointers to iova[] for iv, aad and digest, correct? If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

Quoted 91 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the session private data to decide function handler.
6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single".
I am ok to remove "submit_single" function. In VPP we can use rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to avoid double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an inline function - this will cause the API not traced in version map.

Any ideas?

Regards,
Fan

Quoted 77 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Monday, September 21, 2020 4:49 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 39 lines

Hi AKhil

...

Quoted 25 lines

IMO, the following union can clarify all doubts. @Ananyev, Konstantin: Any suggestions from your side?

/** IV and aad information for various use cases. */

union {
        /** Supposed to be used with CPU crypto API call. */
        struct {
                /** array of pointers to IV */
                void **iv;
                /** array of pointers to AAD */
                void **aad;
                /** array of pointers to digest */
                void **digest;
        } cpu_crypto;  < or any other useful name>
        /* Supposed to be used with HW raw crypto API call. */
        struct {
                void *cipher_iv_ptr;
                rte_iova_t cipher_iv_iova;
                void *auth_iv_ptr;
                rte_iova_t auth_iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
        } hw_chain;
        /* Supposed to be used with HW raw crypto API call. */
        struct {
Quoted 10 lines
                void *iv_ptr;
                rte_iova_t iv_iova;
                void *digest_ptr;
                rte_iova_t digest_iova;
                void *aad_ptr;
                rte_iova_t aad_iova;
        } hw_aead;
};
Quoted 1 line

The above structure cannot support the array of multiple jobs but a single

job.

So was your previous structure. Was it not tested before?

Quoted 4 lines

So we have to use something like

struct {
	void **cipher_iv_ptr;

You can even drop _ptr from the name of each of them.

Quoted 13 lines
	rtei_iova_t *cipher_iv_iova;
	...
} hw_chain;
struct {
	void **iv_ptr;
	rte_iova_t *iv_iova;
	...
} hw_aead;

Is it ok?

Regards,
Fan

#30 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Konstantin,

Quoted 97 lines

Hi lads,

Quoted 57 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and documentation.
4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
Quoted 8 lines
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};

As I understand you just need to add pointers to iova[] for iv, aad and digest, correct?
If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};
Quoted 4 lines

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

This is a good suggestion. This will make some changes in the cpu-crypto support as well And should be a separate patch.
We can take the API and ABI breakage in this release. That is not an issue.

Quoted 4 lines

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

This will create some duplicate code. Would not prefer that.

Quoted 17 lines
Quoted 2 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the session private

data to decide function handler.

Quoted 4 lines

6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single". I am ok to remove "submit_single" function. In VPP we can use

rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to avoid

Quoted 2 lines

double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an inline

function - this will cause the API not traced in version map.

Quoted 6 lines

Any ideas?

Regards,
Fan

#31 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Akhil and Konstantin,

Quoted 83 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Tuesday, September 22, 2020 10:06 AM
To: Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Konstantin,

Quoted 11 lines

Hi lads,

Quoted 9 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and

documentation.

Quoted 58 lines
Quoted 56 lines

4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};
Quoted 15 lines
Quoted 1 line

As I understand you just need to add pointers to iova[] for iv, aad and

digest,

Quoted 13 lines

correct?
If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;

We will need 2 IV here, one for cipher and one for auth (GMAC for example).

Quoted 49 lines
Quoted 14 lines
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};
Quoted 4 lines

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

This is a good suggestion. This will make some changes in the cpu-crypto support as well
And should be a separate patch.
We can take the API and ABI breakage in this release. That is not an issue.

Quoted 4 lines

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

This will create some duplicate code. Would not prefer that.

Quoted 2 lines
Quoted 2 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the session

private

Quoted 15 lines

data to decide function handler.

Quoted 4 lines

6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single". I am ok to remove "submit_single" function. In VPP we can use

rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to avoid

Quoted 2 lines

double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an inline

function - this will cause the API not traced in version map.

Quoted 6 lines

Any ideas?

Regards,
Fan

#32 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Quoted 98 lines

Hi Akhil and Konstantin,

Quoted 93 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Tuesday, September 22, 2020 10:06 AM
To: Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Konstantin,

Quoted 11 lines

Hi lads,

Quoted 9 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and

documentation.

Quoted 58 lines
Quoted 48 lines

4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
Quoted 8 lines
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};
Quoted 1 line

As I understand you just need to add pointers to iova[] for iv, aad and

digest,

Quoted 8 lines

correct?
If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;

We will need 2 IV here, one for cipher and one for auth (GMAC for example).

Hmm, why do we need to different IVs for GMAC?
And if so how does it work now with either rte_crypto_op or with rte_crypto_sym_vec?

Quoted 50 lines
Quoted 50 lines
Quoted 15 lines
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};
Quoted 4 lines

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

This is a good suggestion. This will make some changes in the cpu-crypto support as well
And should be a separate patch.
We can take the API and ABI breakage in this release. That is not an issue.

Quoted 4 lines

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

This will create some duplicate code. Would not prefer that.

Quoted 2 lines
Quoted 2 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the session

private

Quoted 15 lines

data to decide function handler.

Quoted 4 lines

6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single". I am ok to remove "submit_single" function. In VPP we can use

rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to avoid

Quoted 2 lines

double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an inline

function - this will cause the API not traced in version map.

Quoted 6 lines

Any ideas?

Regards,
Fan

#33 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Konstantin,

Quoted 118 lines

-----Original Message-----
From: Ananyev, Konstantin <[email protected]>
Sent: Tuesday, September 22, 2020 11:18 AM
To: Zhang, Roy Fan <[email protected]>; Akhil Goyal <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Quoted 7 lines

Hi Akhil and Konstantin,

Quoted 4 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Tuesday, September 22, 2020 10:06 AM
To: Ananyev, Konstantin <[email protected]>; Zhang, Roy

Fan

Quoted 7 lines
Quoted 7 lines

<[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service

APIs

Quoted 88 lines
Quoted 87 lines

Hi Konstantin,

Quoted 11 lines

Hi lads,

Quoted 9 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and

documentation.

Quoted 58 lines
Quoted 48 lines

4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
Quoted 8 lines
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};
Quoted 1 line

As I understand you just need to add pointers to iova[] for iv, aad and

digest,

Quoted 13 lines

correct?
If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;
Quoted 7 lines
Quoted 1 line

We will need 2 IV here, one for cipher and one for auth (GMAC for

example).

Hmm, why do we need to different IVs for GMAC?
And if so how does it work now with either rte_crypto_op or with rte_crypto_sym_vec?

Not only GMAC, the wireless chain algos such as SNOW3G requires IV in auth field (test_snow3g_cipher_auth() in unit test).
Rte_crypto_sym_op has auth_iv.offset to indicate where the iv is stored in crypto op, so the virtual and physical addresses of it can be deducted through the offset, but not yet for rte_crypto_sym_vec.

Quoted 53 lines
Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};
Quoted 4 lines

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

This is a good suggestion. This will make some changes in the cpu-crypto support as well
And should be a separate patch.
We can take the API and ABI breakage in this release. That is not an issue.

Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

This will create some duplicate code. Would not prefer that.

Quoted 2 lines
Quoted 2 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the

session

Quoted 7 lines
Quoted 7 lines

private

Quoted 6 lines

data to decide function handler.

Quoted 4 lines

6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single". I am ok to remove "submit_single" function. In VPP we can use

rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to

avoid

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an

inline

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

function - this will cause the API not traced in version map.

Quoted 6 lines

Any ideas?

Regards,
Fan

#34 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

In reply to Ananyev, Konstantin

Hi Akhil,

Konstantin and I had an off-line discussion. Is this structure ok for you?

/**
* Crypto virtual and IOVA address descriptor. Used to describe cryptographic
* data without
*
*/

struct rte_crypto_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};

/**
* Raw data operation descriptor.
* Supposed to be used with synchronous CPU crypto API call or asynchronous
* RAW data path API call.
*/

struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to cipher IV */
        struct rte_crypto_va_iova_ptr *iv;
        /** array of pointers to digest */
        struct rte_crypto_va_iova_ptr *digest;

        __extension__
        union {
        	/** array of pointers to auth IV, used for chain operation */
	struct rte_crypto_va_iova_ptr *auth_iv;
	/** array of pointers to AAD, used for AEAD operation */
	struct rte_crypto_va_iova_ptr *aad;
        };

        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};

Regards,
Fan

Quoted 178 lines

-----Original Message-----
From: Ananyev, Konstantin <[email protected]>
Sent: Tuesday, September 22, 2020 11:18 AM
To: Zhang, Roy Fan <[email protected]>; Akhil Goyal <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Quoted 7 lines

Hi Akhil and Konstantin,

Quoted 4 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Tuesday, September 22, 2020 10:06 AM
To: Ananyev, Konstantin <[email protected]>; Zhang, Roy

Fan

Quoted 7 lines
Quoted 7 lines

<[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service

APIs

Quoted 82 lines
Quoted 82 lines

Hi Konstantin,

Quoted 11 lines

Hi lads,

Quoted 9 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and

documentation.

Quoted 58 lines
Quoted 48 lines

4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call. */
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
Quoted 8 lines
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};
Quoted 1 line

As I understand you just need to add pointers to iova[] for iv, aad and

digest,

Quoted 8 lines

correct?
If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;

We will need 2 IV here, one for cipher and one for auth (GMAC for

example).

Hmm, why do we need to different IVs for GMAC?
And if so how does it work now with either rte_crypto_op or with rte_crypto_sym_vec?

Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};
Quoted 4 lines

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

This is a good suggestion. This will make some changes in the cpu-crypto support as well
And should be a separate patch.
We can take the API and ABI breakage in this release. That is not an issue.

Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

This will create some duplicate code. Would not prefer that.

Quoted 2 lines
Quoted 2 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the

session

Quoted 7 lines
Quoted 7 lines

private

Quoted 6 lines

data to decide function handler.

Quoted 4 lines

6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single". I am ok to remove "submit_single" function. In VPP we can use

rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to

avoid

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an

inline

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

function - this will cause the API not traced in version map.

Quoted 6 lines

Any ideas?

Regards,
Fan

#35 lore raw

Re: [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Hi Fan,

Quoted 7 lines

Hi Akhil,

Konstantin and I had an off-line discussion. Is this structure ok for you?

/**
* Crypto virtual and IOVA address descriptor. Used to describe cryptographic
* data without

The comment is incomplete, however the structure is fine.

Quoted 221 lines

*
*/

struct rte_crypto_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};

/**
* Raw data operation descriptor.
* Supposed to be used with synchronous CPU crypto API call or asynchronous
* RAW data path API call.
*/

struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to cipher IV */
        struct rte_crypto_va_iova_ptr *iv;
        /** array of pointers to digest */
        struct rte_crypto_va_iova_ptr *digest;

        __extension__
        union {
        	/** array of pointers to auth IV, used for chain operation */
	struct rte_crypto_va_iova_ptr *auth_iv;
	/** array of pointers to AAD, used for AEAD operation */
	struct rte_crypto_va_iova_ptr *aad;
        };

        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};

Regards,
Fan

Quoted 22 lines

-----Original Message-----
From: Ananyev, Konstantin <[email protected]>
Sent: Tuesday, September 22, 2020 11:18 AM
To: Zhang, Roy Fan <[email protected]>; Akhil Goyal <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service APIs

Quoted 7 lines

Hi Akhil and Konstantin,

Quoted 4 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Tuesday, September 22, 2020 10:06 AM
To: Ananyev, Konstantin <[email protected]>; Zhang, Roy

Fan

Quoted 38 lines
Quoted 7 lines
Quoted 7 lines

<[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Bronowski, PiotrX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 1/4] cryptodev: add crypto data-path service

APIs

Quoted 30 lines
Quoted 30 lines

Hi Konstantin,

Quoted 11 lines

Hi lads,

Quoted 9 lines

Hi Akhil,

Thanks again for the review!
To summarize, the following places to be changed for v10.

1. Documentation update and reviewed internally in Intel first.
2. Add the missing comments to the structure.
3. Change the name "dp_service" to "raw_dp" to all APIs and

documentation.

Quoted 16 lines
Quoted 16 lines

4. Change the structure

struct rte_crypto_sym_vec {
	/** array of SGL vectors */
	struct rte_crypto_sgl *sgl;

	union {
		/** Supposed to be used with CPU crypto API call. */
		struct {
			/** array of pointers to IV */
			void **iv;
			/** array of pointers to AAD */
			void **aad;
			/** array of pointers to digest */
			void **digest;
		} cpu_crypto;
		/** Supposed to be used with HW raw crypto API call.

*/

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
		struct {
			/** array of pointers to cipher IV */
			void **cipher_iv_ptr;
			/** array of IOVA addresses to cipher IV */
			rte_iova_t *cipher_iv_iova;
			/** array of pointers to auth IV */
			void **auth_iv_ptr;
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
			/** array of IOVA addresses to auth IV */
			rte_iova_t *auth_iv_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_chain;
		/** Supposed to be used with HW raw crypto API call.

*/

Quoted 103 lines
Quoted 37 lines
Quoted 37 lines
Quoted 27 lines
Quoted 17 lines
		struct {
			/** array of pointers to AEAD IV */
			void **iv_ptr;
			/** array of IOVA addresses to AEAD IV */
			rte_iova_t *iv_iova;
			/** array of pointers to AAD */
			void **aad_ptr;
			/** array of IOVA addresses to AAD */
			rte_iova_t *aad_iova;
			/** array of pointers to digest */
			void **digest_ptr;
			/** array of IOVA addresses to digest */
			rte_iova_t *digest_iova;
		} hw_aead;
	};

	/**
Quoted 8 lines
	 * array of statuses for each operation:
	 *  - 0 on success
	 *  - errno on error
	 */
	int32_t *status;
	/** number of operations to perform */
	uint32_t num;
};
Quoted 1 line

As I understand you just need to add pointers to iova[] for iv, aad and

digest,

Quoted 8 lines

correct?
If so, why not simply:

struct rte_va_iova_ptr {
	void *va;
	rte_iova_t *iova;
};
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
struct rte_crypto_sym_vec {
        /** array of SGL vectors */
        struct rte_crypto_sgl *sgl;
        /** array of pointers to IV */
        struct rte_va_iova_ptr iv;

We will need 2 IV here, one for cipher and one for auth (GMAC for

example).

Hmm, why do we need to different IVs for GMAC?
And if so how does it work now with either rte_crypto_op or with rte_crypto_sym_vec?

Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
        /** array of pointers to AAD */
        struct rte_va_iova_ptr aad;
        /** array of pointers to digest */
        struct rte_va_iova_ptr digest;
        /**
         * array of statuses for each operation:
         *  - 0 on success
         *  - errno on error
         */
        int32_t *status;
        /** number of operations to perform */
        uint32_t num;
};
Quoted 4 lines

BTW, it would be both ABI and API breakage,
though all functions using this struct are marked as experimental, plus it is an LTS release, so it seems to be ok.
Though I think it needs to be flagged in RN.

This is a good suggestion. This will make some changes in the cpu-crypto support as well
And should be a separate patch.
We can take the API and ABI breakage in this release. That is not an issue.

Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Another option obviously - introduce completely new structure for it and leave existing one unaffected.

This will create some duplicate code. Would not prefer that.

Quoted 2 lines
Quoted 2 lines

5. Remove enum rte_crypto_dp_service, let the PMDs using the

session

Quoted 7 lines
Quoted 7 lines

private

Quoted 6 lines

data to decide function handler.

Quoted 4 lines

6. Remove is_update parameter.

The main point that is uncertain is the existance of "submit_single". I am ok to remove "submit_single" function. In VPP we can use

rte_cryptodev_dp_sym_submit_vec() with vec.num=1 each time to

avoid

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

double looping.
But we have to put the rte_cryptodev_dp_sym_submit_vec() as an

inline

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

function - this will cause the API not traced in version map.

Quoted 6 lines

Any ideas?

Regards,
Fan

author
#36 lore raw

PATCH v9 2/4 crypto/qat: add crypto data-path service API support

In reply to Fan Zhang

This patch updates QAT PMD to add crypto service API support.

---
 drivers/common/qat/Makefile        |   1 +
 drivers/crypto/qat/meson.build     |   1 +
 drivers/crypto/qat/qat_sym.h       |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c | 947 +++++++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c   |   9 +-
 5 files changed, 969 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
drivers/common/qat/Makefile +1 -0
index 85d420709..1b71bbbab 100644
@@ -42,6 +42,7 @@ endif
 	SRCS-y += qat_sym.c
 	SRCS-y += qat_sym_session.c
 	SRCS-y += qat_sym_pmd.c
+	SRCS-y += qat_sym_hw_dp.c
 	build_qat = yes
 endif
 endif
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +13 -0
index 1a9748849..ea2db0ca0 100644
@@ -264,6 +264,18 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_dp_configure_service_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_dp_service service_type,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_dp_service_ctx *service_ctx,
+	uint8_t is_update);
+
+int
+qat_sym_get_service_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +288,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +947 -0

947 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..aaaf3e3f1 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Data plane service related operations */
+		.get_drv_ctx_size = qat_sym_get_service_ctx_size,
+		.configure_service = qat_sym_dp_configure_service_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_DATA_PATH_SERVICE;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#37 lore raw

PATCH v9 3/4 test/crypto: add unit-test for cryptodev direct APIs

In reply to Fan Zhang

This patch adds the QAT test to use cryptodev symmetric crypto direct APIs.

---
 app/test/test_cryptodev.c             | 461 +++++++++++++++++++++++---
 app/test/test_cryptodev.h             |   7 +
 app/test/test_cryptodev_blockcipher.c |  51 ++-
 3 files changed, 456 insertions(+), 63 deletions(-)
app/test/test_cryptodev.c +414 -47
index 70bf6fe2c..13f642e0e 100644
@@ -49,6 +49,8 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +59,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+int cryptodev_dp_test;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +151,182 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	union rte_crypto_sym_additional_data a_data;
+	union rte_crypto_sym_ofs ofs;
+	int32_t status;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	enum rte_crypto_dp_service service_type;
+	uint32_t count = 0;
+	uint8_t service_data[MAX_DRV_SERVICE_CTX_SIZE] = {0};
+	struct rte_crypto_dp_service_ctx *ctx = (void *)service_data;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int ctx_service_size;
+
+	sop = op->sym;
+
+	sess.crypto_sess = sop->session;
+
+	if (is_cipher && is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_CHAIN;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+	} else if (is_cipher) {
+		service_type = RTE_CRYPTO_DP_SYM_CIPHER_ONLY;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+	} else if (is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_AUTH_ONLY;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+	} else { /* aead */
+		service_type = RTE_CRYPTO_DP_SYM_AEAD;
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+	}
+
+	if (len_in_bits) {
+		max_len = max_len >> 3;
+		cipher_offset = cipher_offset >> 3;
+		auth_offset = auth_offset >> 3;
+		cipher_len = cipher_len >> 3;
+		auth_len = auth_len >> 3;
+	}
+
+	ctx_service_size = rte_cryptodev_dp_get_service_ctx_data_size(dev_id);
+	assert(ctx_service_size <= MAX_DRV_SERVICE_CTX_SIZE &&
+			ctx_service_size > 0);
+
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* test update service */
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 1) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ofs.raw = 0;
+
+	switch (service_type) {
+	case RTE_CRYPTO_DP_SYM_AEAD:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.aead.iv_ptr = rte_crypto_op_ctod_offset(op, void *,
+				IV_OFFSET);
+		a_data.aead.iv_iova = rte_crypto_op_ctophys_offset(op,
+				IV_OFFSET);
+		a_data.aead.aad_ptr = (void *)sop->aead.aad.data;
+		a_data.aead.aad_iova = sop->aead.aad.phys_addr;
+		a_data.aead.digest_ptr = (void *)sop->aead.digest.data;
+		a_data.aead.digest_iova = sop->aead.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CIPHER_ONLY:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.cipher_auth.cipher_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		break;
+	case RTE_CRYPTO_DP_SYM_AUTH_ONLY:
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.auth_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop->auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop->auth.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CHAIN:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.cipher_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		a_data.cipher_auth.auth_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop->auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop->auth.digest.phys_addr;
+		break;
+	default:
+		break;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_single_job(ctx, data_vec, n, ofs,
+		&a_data, (void *)op);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_done(ctx, 1);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+
+	status = -1;
+	while (count++ < 65535 && status == -1) {
+		status = rte_cryptodev_dp_sym_dequeue_single_job(ctx,
+				(void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}
+
+	if (status != -1) {
+		if (rte_cryptodev_dp_sym_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			return;
+		}
+	}
+
+	if (count == 65536 || status != 1 || ret_op != op) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	op->status = status == 1 ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1656,6 +1836,9 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params->valid_devs[0],
@@ -1710,12 +1893,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2470,7 +2659,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2549,7 +2742,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2619,6 +2816,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2690,7 +2890,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2897,8 +3101,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2983,7 +3191,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3026,8 +3238,9 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create KASUMI session */
@@ -3107,8 +3320,9 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3192,8 +3406,9 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create KASUMI session */
@@ -3306,7 +3521,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3381,7 +3600,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3419,7 +3642,7 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create SNOW 3G session */
@@ -3502,7 +3725,7 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3621,7 +3844,7 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create SNOW 3G session */
@@ -3756,7 +3979,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3791,7 +4018,7 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create SNOW 3G session */
@@ -3924,7 +4151,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4019,7 +4250,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4087,6 +4322,8 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 	}
 
 	/* Create SNOW 3G session */
@@ -4155,7 +4392,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4266,6 +4507,8 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4344,7 +4587,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4453,6 +4700,8 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	uint64_t feat_flags = dev_info.feature_flags;
 
 	if (op_mode == OUT_OF_PLACE) {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4526,7 +4775,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4638,6 +4891,8 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4716,7 +4971,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4857,7 +5116,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4944,7 +5207,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5031,7 +5298,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5119,7 +5390,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5177,6 +5452,8 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5251,7 +5528,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5359,6 +5640,8 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5437,7 +5720,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5580,6 +5867,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5779,6 +6069,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5793,6 +6086,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6146,10 +6442,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6161,6 +6456,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
 		retval = create_wireless_algo_cipher_auth_session(
@@ -6192,9 +6490,11 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}
 
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6235,18 +6535,17 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6337,10 +6636,9 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6440,20 +6738,18 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -7043,6 +7339,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8540,6 +8839,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8833,6 +9135,9 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
+	/* Data-path service does not support OOP */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
@@ -8923,8 +9228,9 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and data-path service*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			cryptodev_dp_test)
 		return -ENOTSUP;
 
 	/* Create AEAD session */
@@ -9151,8 +9457,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
 
@@ -9472,6 +9783,9 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9530,6 +9844,9 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10098,6 +10415,9 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10215,6 +10535,9 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10780,7 +11103,10 @@ test_authentication_verify_fail_when_data_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10851,7 +11177,10 @@ test_authentication_verify_GMAC_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10926,7 +11255,10 @@ test_authenticated_decryption_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -11021,6 +11353,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11141,6 +11476,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -11285,6 +11623,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Data path service does not support OOP */
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
@@ -11480,6 +11821,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13041,6 +13385,29 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static int
+test_qat_sym_direct_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	cryptodev_dp_test = 1;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	cryptodev_dp_test = 0;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_sym_api_autotest, test_qat_sym_direct_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +7 -0
index 41542e055..e4e4c7626 100644
@@ -71,6 +71,8 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+extern int cryptodev_dp_test;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +211,9 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +35 -16

60 lines not on this page · raw message

Signature · 1 line
2.20.1
#38 lore raw

Re: [dpdk-dev] [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct APIs

Hi Fan,

Quoted 26 lines

Subject: [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct APIs

This patch adds the QAT test to use cryptodev symmetric crypto direct APIs.

---
 app/test/test_cryptodev.c             | 461 +++++++++++++++++++++++---
 app/test/test_cryptodev.h             |   7 +
 app/test/test_cryptodev_blockcipher.c |  51 ++-
 3 files changed, 456 insertions(+), 63 deletions(-)
app/test/test_cryptodev.c +2 -0
index 70bf6fe2c..13f642e0e 100644
@@ -49,6 +49,8 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 

app/test/test_cryptodev.c static int gbl_driver_id; · line 63

Quoted 5 lines
app/test/test_cryptodev.c +2 -0
@@ -57,6 +59,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+int cryptodev_dp_test;
+

Why do we need this? We should make decision based on Feature flag of crypto device.

Quoted 66 lines
app/test/test_cryptodev.c +58 -0
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +151,182 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op
 *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	union rte_crypto_sym_additional_data a_data;
+	union rte_crypto_sym_ofs ofs;
+	int32_t status;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	enum rte_crypto_dp_service service_type;
+	uint32_t count = 0;
+	uint8_t service_data[MAX_DRV_SERVICE_CTX_SIZE] = {0};
+	struct rte_crypto_dp_service_ctx *ctx = (void *)service_data;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int ctx_service_size;
+
+	sop = op->sym;
+
+	sess.crypto_sess = sop->session;
+
+	if (is_cipher && is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_CHAIN;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+	} else if (is_cipher) {
+		service_type = RTE_CRYPTO_DP_SYM_CIPHER_ONLY;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+	} else if (is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_AUTH_ONLY;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+	} else { /* aead */
+		service_type = RTE_CRYPTO_DP_SYM_AEAD;
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+	}
+
+	if (len_in_bits) {
+		max_len = max_len >> 3;
+		cipher_offset = cipher_offset >> 3;
+		auth_offset = auth_offset >> 3;
+		cipher_len = cipher_len >> 3;
+		auth_len = auth_len >> 3;
+	}

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 222

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+
+	ctx_service_size = rte_cryptodev_dp_get_service_ctx_data_size(dev_id);
+	assert(ctx_service_size <= MAX_DRV_SERVICE_CTX_SIZE &&
+			ctx_service_size > 0);
+
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}

**_dp_configure_service does not provide context of the API. What does service mean here? Can we rename it to rte_cryptodev_configure_raw_dp?
This gives better readability to have an API to configure crypto device for raw data path.

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 225

Quoted 4 lines
app/test/test_cryptodev.c +4 -0
+
+	/* test update service */
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 1) < 0) {

Do we really need an extra parameter to specify update? Can we not call again same API for update? Anyway the implementation will be copying the complete information from the sess.

Quoted 51 lines
app/test/test_cryptodev.c +51 -0
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ofs.raw = 0;
+
+	switch (service_type) {
+	case RTE_CRYPTO_DP_SYM_AEAD:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.aead.iv_ptr = rte_crypto_op_ctod_offset(op, void *,
+				IV_OFFSET);
+		a_data.aead.iv_iova = rte_crypto_op_ctophys_offset(op,
+				IV_OFFSET);
+		a_data.aead.aad_ptr = (void *)sop->aead.aad.data;
+		a_data.aead.aad_iova = sop->aead.aad.phys_addr;
+		a_data.aead.digest_ptr = (void *)sop->aead.digest.data;
+		a_data.aead.digest_iova = sop->aead.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CIPHER_ONLY:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.cipher_auth.cipher_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		break;
+	case RTE_CRYPTO_DP_SYM_AUTH_ONLY:
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.auth_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop->auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop->auth.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CHAIN:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.cipher_iv_ptr = rte_crypto_op_ctod_offset(

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 286

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		a_data.cipher_auth.auth_iv_ptr = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop->auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop->auth.digest.phys_addr;

Instead of cipher_auth, it should be chain. Can we also support "hash then cipher" case also?

Quoted 25 lines
app/test/test_cryptodev.c +25 -0
+		break;
+	default:
+		break;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_single_job(ctx, data_vec, n, ofs,
+		&a_data, (void *)op);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_done(ctx, 1);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+
+	status = -1;
+	while (count++ < 65535 && status == -1) {
+		status = rte_cryptodev_dp_sym_dequeue_single_job(ctx,
+				(void **)&ret_op);
+		if (status == -1)
+			rte_pause();

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 321

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+	}
+
+	if (status != -1) {
+		if (rte_cryptodev_dp_sym_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			return;
+		}
+	}
+
+	if (count == 65536 || status != 1 || ret_op != op) {

Remove hardcode 65536

Quoted 14 lines
app/test/test_cryptodev.c +8 -0
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	op->status = status == 1 ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)

{

app/test/test_cryptodev.c +0 -0
@@ -1656,6 +1836,9 @@

test_AES_CBC_HMAC_SHA512_decrypt_perform(struct
rte_cryptodev_sym_session *sess,

app/test/test_cryptodev.c test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess, · line 1841

Quoted 6 lines
app/test/test_cryptodev.c +3 -0
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);

Can we rename process_sym_hw_api_op to process_sym_hw_raw_op?

Quoted 536 lines
app/test/test_cryptodev.c +0 -0
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +11 -1
@@ -1710,12 +1893,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2470,7 +2659,11 @@ test_snow3g_authentication(const struct
 snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +5 -1
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2549,7 +2742,11 @@ test_snow3g_authentication_verify(const struct
 snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +3 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2619,6 +2816,9 @@ test_kasumi_authentication(const struct
 kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +5 -1
 			ut_params->op);
@@ -2690,7 +2890,11 @@ test_kasumi_authentication_verify(const struct
 kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +6 -2
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2897,8 +3101,12 @@ test_kasumi_encryption(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
app/test/test_cryptodev.c +5 -1
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2983,7 +3191,11 @@ test_kasumi_encryption_sgl(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +2 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3026,8 +3238,9 @@ test_kasumi_encryption_oop(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +2 -1
 	/* Create KASUMI session */
@@ -3107,8 +3320,9 @@ test_kasumi_encryption_oop_sgl(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +2 -1
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3192,8 +3406,9 @@ test_kasumi_decryption_oop(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +5 -1
 	/* Create KASUMI session */
@@ -3306,7 +3521,11 @@ test_kasumi_decryption(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3381,7 +3600,11 @@ test_snow3g_encryption(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +1 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3419,7 +3642,7 @@ test_snow3g_encryption_oop(const struct
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +1 -1
 	/* Create SNOW 3G session */
@@ -3502,7 +3725,7 @@ test_snow3g_encryption_oop_sgl(const struct
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +1 -1
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3621,7 +3844,7 @@ test_snow3g_encryption_offset_oop(const struct
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -3756,7 +3979,11 @@ static int test_snow3g_decryption(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +1 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3791,7 +4018,7 @@ static int test_snow3g_decryption_oop(const struct
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -3924,7 +4151,11 @@ test_zuc_cipher_auth(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4019,7 +4250,11 @@ test_snow3g_cipher_auth(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +2 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4087,6 +4322,8 @@ test_snow3g_auth_cipher(const struct
 snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 	}
 
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -4155,7 +4392,11 @@ test_snow3g_auth_cipher(const struct
 snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +2 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4266,6 +4507,8 @@ test_snow3g_auth_cipher_sgl(const struct
 snow3g_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -4344,7 +4587,11 @@ test_snow3g_auth_cipher_sgl(const struct
 snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +2 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4453,6 +4700,8 @@ test_kasumi_auth_cipher(const struct
 kasumi_test_data *tdata,
 	uint64_t feat_flags = dev_info.feature_flags;
 
 	if (op_mode == OUT_OF_PLACE) {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
app/test/test_cryptodev.c +5 -1
 			return -ENOTSUP;
@@ -4526,7 +4775,11 @@ test_kasumi_auth_cipher(const struct
 kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +2 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4638,6 +4891,8 @@ test_kasumi_auth_cipher_sgl(const struct
 kasumi_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -4716,7 +4971,11 @@ test_kasumi_auth_cipher_sgl(const struct
 kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4857,7 +5116,11 @@ test_kasumi_cipher_auth(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4944,7 +5207,11 @@ test_zuc_encryption(const struct wireless_test_data
 *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5031,7 +5298,11 @@ test_zuc_encryption_sgl(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5119,7 +5390,11 @@ test_zuc_authentication(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +2 -0
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5177,6 +5452,8 @@ test_zuc_auth_cipher(const struct wireless_test_data
 *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -5251,7 +5528,11 @@ test_zuc_auth_cipher(const struct
 wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +2 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5359,6 +5640,8 @@ test_zuc_auth_cipher_sgl(const struct
 wireless_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -5437,7 +5720,11 @@ test_zuc_auth_cipher_sgl(const struct
 wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +10 -2
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5580,6 +5867,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5779,6 +6069,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5793,6 +6086,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6146,10 +6442,9 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +3 -0
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6161,6 +6456,9 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
app/test/test_cryptodev.c +0 -0
 		retval = create_wireless_algo_cipher_auth_session(

app/test/test_cryptodev.c test_mixed_auth_cipher(const struct · line 6498

Quoted 10 lines
app/test/test_cryptodev.c +3 -1
@@ -6192,9 +6490,11 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}

Unnecessary change.

Quoted 12 lines
app/test/test_cryptodev.c +1 -1
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6235,18 +6535,17 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 

app/test/test_cryptodev.c test_mixed_auth_cipher(const struct · line 6250

Quoted 10 lines
app/test/test_cryptodev.c +1 -2
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;

Unnecessary change

Quoted 24 lines
app/test/test_cryptodev.c +1 -2
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6337,10 +6636,9 @@ test_mixed_auth_cipher_sgl(const struct
 mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +1 -1
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6440,20 +6738,18 @@ test_mixed_auth_cipher_sgl(const struct
 mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);

app/test/test_cryptodev.c test_mixed_auth_cipher_sgl(const struct · line 6752

Quoted 10 lines
app/test/test_cryptodev.c +1 -1
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}

Unnecessary change

Quoted 223 lines
app/test/test_cryptodev.c +3 -2
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -7043,6 +7339,9 @@ test_authenticated_encryption(const struct
 aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params-
 >op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
app/test/test_cryptodev.c +3 -0
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8540,6 +8839,9 @@ test_authenticated_decryption(const struct
 aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params-
 >op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
app/test/test_cryptodev.c +3 -0
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8833,6 +9135,9 @@ test_authenticated_encryption_oop(const struct
 aead_test_data *tdata)
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
+	/* Data-path service does not support OOP */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 
 	/* not supported with CPU crypto */
app/test/test_cryptodev.c +3 -2
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
@@ -8923,8 +9228,9 @@ test_authenticated_decryption_oop(const struct
 aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and data-path service*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +10 -2
 	/* Create AEAD session */
@@ -9151,8 +9457,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params-
 >valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
@@ -9472,6 +9783,9 @@ test_MD5_HMAC_generate(const struct
 HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +3 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9530,6 +9844,9 @@ test_MD5_HMAC_verify(const struct
 HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +3 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10098,6 +10415,9 @@ test_AES_GMAC_authentication(const struct
 gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +3 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10215,6 +10535,9 @@ test_AES_GMAC_authentication_verify(const
 struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +0 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10780,7 +11103,10 @@

test_authentication_verify_fail_when_data_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +0 -0
@@ -10851,7 +11177,10 @@

test_authentication_verify_GMAC_fail_when_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +0 -0
@@ -10926,7 +11255,10 @@

test_authenticated_decryption_fail_when_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +6 -0
@@ -11021,6 +11353,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11141,6 +11476,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +3 -0
 			ut_params->op);
@@ -11285,6 +11623,9 @@ test_authenticated_encryption_SGL(const struct
 aead_test_data *tdata,
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Data path service does not support OOP */
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))

app/test/test_cryptodev.c +3 -0
@@ -11480,6 +11821,9 @@ test_authenticated_encryption_SGL(const struct
 aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type ==
 RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params-
 >op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
app/test/test_cryptodev.c +10 -0
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13041,6 +13385,29 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static int
+test_qat_sym_direct_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that

both "

app/test/test_cryptodev.c +1 -0
+		"CONFIG_RTE_LIBRTE_PMD_QAT and

CONFIG_RTE_LIBRTE_PMD_QAT_SYM "

app/test/test_cryptodev.c test_cryptodev_nitrox(void) · line 13403

Quoted 5 lines
app/test/test_cryptodev.c +5 -0
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	cryptodev_dp_test = 1;

Cryptodev_dp_test cannot be set blindly. You should check for feature flag of the device If the feature is supported or not.

Can we also rename this flag as "test_raw_crypto_dp"

Quoted 8 lines
app/test/test_cryptodev.c +7 -0
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	cryptodev_dp_test = 0;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_sym_api_autotest,

test_qat_sym_direct_api);

It would be better to name the test string as test_cryptodev_qat_raw_dp

Quoted 101 lines

REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat); REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest,
test_cryptodev_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,

app/test/test_cryptodev.h +4 -0
index 41542e055..e4e4c7626 100644
@@ -71,6 +71,8 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+extern int cryptodev_dp_test;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +211,9 @@ create_segmented_mbuf(struct rte_mempool
 *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op

*op,

app/test/test_cryptodev.h +3 -0
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +35 -17
index 221262341..311b34c15 100644
@@ -462,25 +462,44 @@ test_blockcipher_one_case(const struct
 blockcipher_test_case *t,
 	}
 
 	/* Process crypto operation */
-	if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) {
-		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
-			"line %u FAILED: %s",
-			__LINE__, "Error sending packet for encryption");
-		status = TEST_FAILED;
-		goto error_exit;
-	}
+	if (cryptodev_dp_test) {
+		uint8_t is_cipher = 0, is_auth = 0;
 
-	op = NULL;
+		if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_OOP) {
+			RTE_LOG(DEBUG, USER1,
+			"QAT direct API does not support OOP, Test
 Skipped.\n");
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
 "SKIPPED");
+			status = TEST_SUCCESS;
+			goto error_exit;
+		}
+		if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
+			is_cipher = 1;
+		if (t->op_mask & BLOCKCIPHER_TEST_OP_AUTH)
+			is_auth = 1;
 
-	while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0)
-		rte_pause();
+		process_sym_hw_api_op(dev_id, 0, op, is_cipher, is_auth, 0,
+				tdata->iv.len);
+	} else {
+		if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) {
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
+				"line %u FAILED: %s",
+				__LINE__, "Error sending packet for
 encryption");
+			status = TEST_FAILED;
+			goto error_exit;
+		}
 
-	if (!op) {
-		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
-			"line %u FAILED: %s",
-			__LINE__, "Failed to process sym crypto op");
-		status = TEST_FAILED;
-		goto error_exit;
+		op = NULL;
+
+		while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0)
+			rte_pause();
+
+		if (!op) {
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
+				"line %u FAILED: %s",
+				__LINE__, "Failed to process sym crypto op");
+			status = TEST_FAILED;
+			goto error_exit;
+		}
 	}
 
 	debug_hexdump(stdout, "m_src(after):",
--

2.20.1

#39 lore raw

Re: [dpdk-dev] [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct APIs

Hi Akhil,

Comments inline.

Quoted 42 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Friday, September 18, 2020 9:03 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>
Subject: RE: [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct APIs

Hi Fan,

Quoted 1 line

Subject: [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct

APIs

Quoted 28 lines

This patch adds the QAT test to use cryptodev symmetric crypto direct APIs.

---
 app/test/test_cryptodev.c             | 461 +++++++++++++++++++++++---
 app/test/test_cryptodev.h             |   7 +
 app/test/test_cryptodev_blockcipher.c |  51 ++-
 3 files changed, 456 insertions(+), 63 deletions(-)
app/test/test_cryptodev.c +2 -0
index 70bf6fe2c..13f642e0e 100644
@@ -49,6 +49,8 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +59,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
Quoted 5 lines

app/test/test_cryptodev.c static int gbl_driver_id; · line 63

Quoted 2 lines
app/test/test_cryptodev.c +2 -0
+int cryptodev_dp_test;
+

Why do we need this? We should make decision based on Feature flag of crypto device.

Even the feature flag is supported we may want to test both cryptodev_op Datapath and the new data pathes. The parameter is set by the command Only.

Quoted 72 lines

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 155

Quoted 11 lines
app/test/test_cryptodev.c +2 -0
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +151,182 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct

rte_crypto_op

Quoted 49 lines
app/test/test_cryptodev.c +48 -0
 *op,
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	union rte_crypto_sym_additional_data a_data;
+	union rte_crypto_sym_ofs ofs;
+	int32_t status;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	enum rte_crypto_dp_service service_type;
+	uint32_t count = 0;
+	uint8_t service_data[MAX_DRV_SERVICE_CTX_SIZE] = {0};
+	struct rte_crypto_dp_service_ctx *ctx = (void *)service_data;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int ctx_service_size;
+
+	sop = op->sym;
+
+	sess.crypto_sess = sop->session;
+
+	if (is_cipher && is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_CHAIN;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+	} else if (is_cipher) {
+		service_type = RTE_CRYPTO_DP_SYM_CIPHER_ONLY;
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+	} else if (is_auth) {
+		service_type = RTE_CRYPTO_DP_SYM_AUTH_ONLY;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+	} else { /* aead */
+		service_type = RTE_CRYPTO_DP_SYM_AEAD;
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+	}

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+
+	if (len_in_bits) {
+		max_len = max_len >> 3;
+		cipher_offset = cipher_offset >> 3;
+		auth_offset = auth_offset >> 3;
+		cipher_len = cipher_len >> 3;
+		auth_len = auth_len >> 3;
+	}
+
+	ctx_service_size =

rte_cryptodev_dp_get_service_ctx_data_size(dev_id);

Quoted 14 lines

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 221

Quoted 8 lines
app/test/test_cryptodev.c +8 -0
+	assert(ctx_service_size <= MAX_DRV_SERVICE_CTX_SIZE &&
+			ctx_service_size > 0);
+
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}

**_dp_configure_service does not provide context of the API. What does service mean here?
Can we rename it to rte_cryptodev_configure_raw_dp?
This gives better readability to have an API to configure crypto device for raw data path.

Great name. Thanks. Will change.

Quoted 11 lines

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 225

Quoted 5 lines
app/test/test_cryptodev.c +4 -0
+
+	/* test update service */
+	if (rte_cryptodev_dp_configure_service(dev_id, qp_id, service_type,
+			RTE_CRYPTO_OP_WITH_SESSION, sess, ctx, 1) < 0) {

Do we really need an extra parameter to specify update? Can we not call again same API
for update? Anyway the implementation will be copying the complete information from
the sess.

Explained in the last email replying your comments to the 4th patch.

Quoted 69 lines
Quoted 21 lines
app/test/test_cryptodev.c +20 -0
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ofs.raw = 0;
+
+	switch (service_type) {
+	case RTE_CRYPTO_DP_SYM_AEAD:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.aead.iv_ptr = rte_crypto_op_ctod_offset(op, void *,
+				IV_OFFSET);
+		a_data.aead.iv_iova = rte_crypto_op_ctophys_offset(op,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+				IV_OFFSET);
+		a_data.aead.aad_ptr = (void *)sop->aead.aad.data;
+		a_data.aead.aad_iova = sop->aead.aad.phys_addr;
+		a_data.aead.digest_ptr = (void *)sop->aead.digest.data;
+		a_data.aead.digest_iova = sop->aead.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CIPHER_ONLY:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		a_data.cipher_auth.cipher_iv_ptr =

rte_crypto_op_ctod_offset(

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 3 lines
app/test/test_cryptodev.c +3 -0
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op,

IV_OFFSET);

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 5 lines
app/test/test_cryptodev.c +5 -0
+		break;
+	case RTE_CRYPTO_DP_SYM_AUTH_ONLY:
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.auth_iv_ptr =

rte_crypto_op_ctod_offset(

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 3 lines
app/test/test_cryptodev.c +3 -0
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 12 lines
app/test/test_cryptodev.c +10 -0
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop-
 auth.digest.data;
+		a_data.cipher_auth.digest_iova = sop-
 auth.digest.phys_addr;
+		break;
+	case RTE_CRYPTO_DP_SYM_CHAIN:
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		a_data.cipher_auth.cipher_iv_ptr =

rte_crypto_op_ctod_offset(

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 3 lines
app/test/test_cryptodev.c +3 -0
+				op, void *, IV_OFFSET);
+		a_data.cipher_auth.cipher_iv_iova =
+				rte_crypto_op_ctophys_offset(op,

IV_OFFSET);

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 283

Quoted 1 line
app/test/test_cryptodev.c +1 -0
+		a_data.cipher_auth.auth_iv_ptr =

rte_crypto_op_ctod_offset(

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 3 lines
app/test/test_cryptodev.c +3 -0
+				op, void *, IV_OFFSET + cipher_iv_len);
+		a_data.cipher_auth.auth_iv_iova =
+				rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

Quoted 8 lines
Quoted 5 lines
app/test/test_cryptodev.c +2 -0
+						cipher_iv_len);
+		a_data.cipher_auth.digest_ptr = (void *)sop-

auth.digest.data;

app/test/test_cryptodev.c +1 -0
+		a_data.cipher_auth.digest_iova = sop-

auth.digest.phys_addr;

Instead of cipher_auth, it should be chain. Can we also support "hash then cipher" case also?

Cipher_auth is a name to describe the fields needed for cipher only, auth only, and chain Operation. The "cipher" is shown before "auth" is merely because of alphabetical order. Of course hash then cipher is supported - as long as a session can be created successfully by a PMD, the API supports it (but not OOP).

Quoted 31 lines

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits)

Quoted 7 lines
app/test/test_cryptodev.c +6 -0
+		break;
+	default:
+		break;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_single_job(ctx, data_vec, n,

ofs,

Quoted 23 lines
app/test/test_cryptodev.c +23 -0
+		&a_data, (void *)op);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	status = rte_cryptodev_dp_sym_submit_done(ctx, 1);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+
+	status = -1;
+	while (count++ < 65535 && status == -1) {
+		status = rte_cryptodev_dp_sym_dequeue_single_job(ctx,
+				(void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}
+
+	if (status != -1) {
+		if (rte_cryptodev_dp_sym_dequeue_done(ctx, 1) < 0) {
Quoted 8 lines

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 321

Quoted 6 lines
app/test/test_cryptodev.c +6 -0
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			return;
+		}
+	}
+
+	if (count == 65536 || status != 1 || ret_op != op) {

Remove hardcode 65536

Will do.

Quoted 15 lines
Quoted 15 lines
app/test/test_cryptodev.c +8 -0
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	op->status = status == 1 ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1656,6 +1836,9 @@

test_AES_CBC_HMAC_SHA512_decrypt_perform(struct
rte_cryptodev_sym_session *sess,

Quoted 8 lines

app/test/test_cryptodev.c test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess, · line 1841

Quoted 6 lines
app/test/test_cryptodev.c +3 -0
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);

Can we rename process_sym_hw_api_op to process_sym_hw_raw_op?

Yes.

Quoted 1,007 lines

app/test/test_cryptodev.c test_snow3g_encryption_offset_oop(const

Quoted 207 lines
app/test/test_cryptodev.c +0 -0
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +11 -1
@@ -1710,12 +1893,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2470,7 +2659,11 @@ test_snow3g_authentication(const struct
 snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +5 -1
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2549,7 +2742,11 @@ test_snow3g_authentication_verify(const struct
 snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +3 -0
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2619,6 +2816,9 @@ test_kasumi_authentication(const struct
 kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +5 -1
 			ut_params->op);
@@ -2690,7 +2890,11 @@ test_kasumi_authentication_verify(const struct
 kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +6 -2
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2897,8 +3101,12 @@ test_kasumi_encryption(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
-						ut_params->op);
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
+				ut_params->op);
app/test/test_cryptodev.c +5 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2983,7 +3191,11 @@ test_kasumi_encryption_sgl(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +2 -1
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3026,8 +3238,9 @@ test_kasumi_encryption_oop(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +2 -1
 	/* Create KASUMI session */
@@ -3107,8 +3320,9 @@ test_kasumi_encryption_oop_sgl(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +2 -1
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3192,8 +3406,9 @@ test_kasumi_decryption_oop(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +5 -1
 	/* Create KASUMI session */
@@ -3306,7 +3521,11 @@ test_kasumi_decryption(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +5 -1
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3381,7 +3600,11 @@ test_snow3g_encryption(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +1 -1
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3419,7 +3642,7 @@ test_snow3g_encryption_oop(const struct
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +1 -1
 	/* Create SNOW 3G session */
@@ -3502,7 +3725,7 @@ test_snow3g_encryption_oop_sgl(const struct
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +0 -0
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -3621,7 +3844,7 @@ test_snow3g_encryption_offset_oop(const

struct

app/test/test_cryptodev.c static int test_snow3g_decryption_oop(const

Quoted 26 lines
app/test/test_cryptodev.c +1 -1
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -3756,7 +3979,11 @@ static int test_snow3g_decryption(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3791,7 +4018,7 @@ static int test_snow3g_decryption_oop(const

struct

Quoted 68 lines
app/test/test_cryptodev.c +1 -1
 snow3g_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
-			&cap_idx) == NULL)
+			&cap_idx) == NULL || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -3924,7 +4151,11 @@ test_zuc_cipher_auth(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +5 -1
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4019,7 +4250,11 @@ test_snow3g_cipher_auth(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +2 -0
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4087,6 +4322,8 @@ test_snow3g_auth_cipher(const struct
 snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 	}
 
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -4155,7 +4392,11 @@ test_snow3g_auth_cipher(const struct
 snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");

app/test/test_cryptodev.c test_snow3g_auth_cipher_sgl(const struct · line 4513

Quoted 7 lines
app/test/test_cryptodev.c +2 -0
@@ -4266,6 +4507,8 @@ test_snow3g_auth_cipher_sgl(const struct
 snow3g_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

Quoted 21 lines
app/test/test_cryptodev.c +0 -0
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -4344,7 +4587,11 @@ test_snow3g_auth_cipher_sgl(const struct
 snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");

app/test/test_cryptodev.c test_kasumi_auth_cipher(const struct · line 4706

Quoted 7 lines
app/test/test_cryptodev.c +2 -0
@@ -4453,6 +4700,8 @@ test_kasumi_auth_cipher(const struct
 kasumi_test_data *tdata,
 	uint64_t feat_flags = dev_info.feature_flags;
 
 	if (op_mode == OUT_OF_PLACE) {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED))

{

Quoted 19 lines
app/test/test_cryptodev.c +0 -0
 			printf("Device doesn't support digest encrypted.\n");
app/test/test_cryptodev.c +5 -1
 			return -ENOTSUP;
@@ -4526,7 +4775,11 @@ test_kasumi_auth_cipher(const struct
 kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");

app/test/test_cryptodev.c test_kasumi_auth_cipher_sgl(const struct · line 4897

Quoted 7 lines
app/test/test_cryptodev.c +2 -0
@@ -4638,6 +4891,8 @@ test_kasumi_auth_cipher_sgl(const struct
 kasumi_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

app/test/test_cryptodev.c test_zuc_encryption(const struct

Quoted 37 lines
app/test/test_cryptodev.c +0 -0
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -4716,7 +4971,11 @@ test_kasumi_auth_cipher_sgl(const struct
 kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +5 -1
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4857,7 +5116,11 @@ test_kasumi_cipher_auth(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4944,7 +5207,11 @@ test_zuc_encryption(const struct

wireless_test_data

app/test/test_cryptodev.c test_zuc_auth_cipher(const struct

Quoted 48 lines
app/test/test_cryptodev.c +5 -1
 *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +5 -1
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5031,7 +5298,11 @@ test_zuc_encryption_sgl(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +5 -1
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5119,7 +5390,11 @@ test_zuc_authentication(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5177,6 +5452,8 @@ test_zuc_auth_cipher(const struct

wireless_test_data

app/test/test_cryptodev.c test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,

Quoted 7 lines
app/test/test_cryptodev.c +2 -0
 *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

Quoted 21 lines
app/test/test_cryptodev.c +0 -0
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -5251,7 +5528,11 @@ test_zuc_auth_cipher(const struct
 wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");

app/test/test_cryptodev.c test_zuc_auth_cipher_sgl(const struct · line 5646

Quoted 7 lines
app/test/test_cryptodev.c +2 -0
@@ -5359,6 +5640,8 @@ test_zuc_auth_cipher_sgl(const struct
 wireless_test_data *tdata,
 			return -ENOTSUP;
 		}
 	} else {
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (!(feat_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

Quoted 21 lines
app/test/test_cryptodev.c +0 -0
 {

printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -5437,7 +5720,11 @@ test_zuc_auth_cipher_sgl(const struct
 wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");

app/test/test_cryptodev.c test_kasumi_decryption_test_case_2(void) · line 5870

Quoted 4 lines
app/test/test_cryptodev.c +1 -0
@@ -5580,6 +5867,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build

*/

app/test/test_cryptodev.c

Quoted 6 lines
app/test/test_cryptodev.c +2 -0
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5779,6 +6069,9 @@

test_snow3g_auth_cipher_part_digest_enc_oop(void)

app/test/test_cryptodev.c test_snow3g_auth_cipher_part_digest_enc_oop(void)

Quoted 4 lines
app/test/test_cryptodev.c +1 -0
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build

*/

app/test/test_cryptodev.c

Quoted 6 lines
app/test/test_cryptodev.c +2 -0
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5793,6 +6086,9 @@

test_snow3g_auth_cipher_test_case_3_oop_sgl(void)

app/test/test_cryptodev.c test_snow3g_auth_cipher_test_case_3_oop_sgl(void)

Quoted 4 lines
app/test/test_cryptodev.c +1 -0
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build

*/

Quoted 30 lines
app/test/test_cryptodev.c +3 -2
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6146,10 +6442,9 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +3 -0
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6161,6 +6456,9 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
app/test/test_cryptodev.c +0 -0
 		retval = create_wireless_algo_cipher_auth_session(

app/test/test_cryptodev.c test_mixed_auth_cipher(const struct · line 6498

Quoted 10 lines
app/test/test_cryptodev.c +3 -1
@@ -6192,9 +6490,11 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}

Unnecessary change.

Quoted 13 lines
app/test/test_cryptodev.c +1 -1
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6235,18 +6535,17 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
 			ut_params->op);
 

app/test/test_cryptodev.c test_mixed_auth_cipher(const struct · line 6251

Quoted 10 lines
app/test/test_cryptodev.c +1 -2
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;

Unnecessary change

Quoted 25 lines
app/test/test_cryptodev.c +1 -2
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -6337,10 +6636,9 @@ test_mixed_auth_cipher_sgl(const struct
 mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
-	if (test_mixed_check_if_unsupported(tdata))
+	if (test_mixed_check_if_unsupported(tdata) || cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +1 -1
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6440,20 +6738,18 @@ test_mixed_auth_cipher_sgl(const struct
 mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params-
 valid_devs[0],
 			ut_params->op);

app/test/test_cryptodev.c test_mixed_auth_cipher_sgl(const struct · line 6753

Quoted 10 lines
app/test/test_cryptodev.c +1 -1
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}

Unnecessary change

app/test/test_cryptodev.c test_authenticated_encryption(const struct · line 7342

Quoted 11 lines
app/test/test_cryptodev.c +0 -2
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
app/test/test_cryptodev.c +0 -0
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -7043,6 +7339,9 @@ test_authenticated_encryption(const struct
 aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0],

ut_params-

app/test/test_cryptodev.c test_authenticated_decryption(const struct · line 8842

Quoted 12 lines
app/test/test_cryptodev.c +3 -0
 >op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
app/test/test_cryptodev.c +0 -0
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8540,6 +8839,9 @@ test_authenticated_decryption(const struct
 aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0],

ut_params-

Quoted 19 lines
app/test/test_cryptodev.c +3 -0
 >op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
app/test/test_cryptodev.c +3 -0
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8833,6 +9135,9 @@ test_authenticated_encryption_oop(const struct
 aead_test_data *tdata)
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
+	/* Data-path service does not support OOP */
+	if (cryptodev_dp_test)
+		return -ENOTSUP;
 
 	/* not supported with CPU crypto */
app/test/test_cryptodev.c +0 -0
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)

app/test/test_cryptodev.c test_authenticated_decryption_oop(const struct · line 9233

Quoted 8 lines
app/test/test_cryptodev.c +2 -2
@@ -8923,8 +9228,9 @@ test_authenticated_decryption_oop(const struct
 aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and data-path service*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

||

app/test/test_cryptodev.c test_authenticated_encryption_SGL(const

Quoted 132 lines
app/test/test_cryptodev.c +1 -0
+			cryptodev_dp_test)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +10 -2
 	/* Create AEAD session */
@@ -9151,8 +9457,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params-
 >valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
@@ -9472,6 +9783,9 @@ test_MD5_HMAC_generate(const struct
 HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +3 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9530,6 +9844,9 @@ test_MD5_HMAC_verify(const struct
 HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +3 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10098,6 +10415,9 @@ test_AES_GMAC_authentication(const struct
 gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +3 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10215,6 +10535,9 @@ test_AES_GMAC_authentication_verify(const
 struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +0 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10780,7 +11103,10 @@

test_authentication_verify_fail_when_data_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +0 -0
@@ -10851,7 +11177,10 @@

test_authentication_verify_GMAC_fail_when_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +0 -0
@@ -10926,7 +11255,10 @@

test_authenticated_decryption_fail_when_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +6 -0
@@ -11021,6 +11353,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11141,6 +11476,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +0 -0
 			ut_params->op);
@@ -11285,6 +11623,9 @@ test_authenticated_encryption_SGL(const

struct

app/test/test_cryptodev.c test_authenticated_encryption_SGL(const

Quoted 12 lines
app/test/test_cryptodev.c +3 -0
 aead_test_data *tdata,
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Data path service does not support OOP */
+		if (cryptodev_dp_test)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))

app/test/test_cryptodev.c +0 -0
@@ -11480,6 +11821,9 @@ test_authenticated_encryption_SGL(const

struct

app/test/test_cryptodev.c test_authenticated_encryption_SGL(const · line 11825

Quoted 5 lines
app/test/test_cryptodev.c +0 -0
 aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type ==
 RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0],

ut_params-

app/test/test_cryptodev.c test_cryptodev_nitrox(void) · line 13389

Quoted 13 lines
app/test/test_cryptodev.c +3 -0
 >op);
+	else if (cryptodev_dp_test)
+		process_sym_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
app/test/test_cryptodev.c +2 -0
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13041,6 +13385,29 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static int
+test_qat_sym_direct_api(void /*argv __rte_unused, int argc

__rte_unused*/)

Quoted 11 lines
app/test/test_cryptodev.c +8 -0
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that

both "

app/test/test_cryptodev.c +1 -0
+		"CONFIG_RTE_LIBRTE_PMD_QAT and

CONFIG_RTE_LIBRTE_PMD_QAT_SYM "

app/test/test_cryptodev.c test_cryptodev_nitrox(void) · line 13403

Quoted 5 lines
app/test/test_cryptodev.c +5 -0
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	cryptodev_dp_test = 1;

Cryptodev_dp_test cannot be set blindly. You should check for feature flag of the device
If the feature is supported or not.

Can we also rename this flag as "test_raw_crypto_dp"

Quoted 8 lines
app/test/test_cryptodev.c +7 -0
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	cryptodev_dp_test = 0;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_sym_api_autotest,

test_qat_sym_direct_api);

It would be better to name the test string as test_cryptodev_qat_raw_dp

Quoted 1 line

REGISTER_TEST_COMMAND(cryptodev_qat_autotest,

test_cryptodev_qat);

Quoted 17 lines

REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest,
test_cryptodev_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,

app/test/test_cryptodev.h +2 -0
index 41542e055..e4e4c7626 100644
@@ -71,6 +71,8 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+extern int cryptodev_dp_test;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *

app/test/test_cryptodev.h create_segmented_mbuf(struct rte_mempool · line 216

Quoted 6 lines
app/test/test_cryptodev.h +2 -0
@@ -209,4 +211,9 @@ create_segmented_mbuf(struct rte_mempool
 *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_hw_api_op(uint8_t dev_id, uint16_t qp_id, struct

rte_crypto_op

Quoted 53 lines

*op,

app/test/test_cryptodev.h +3 -0
+		uint8_t is_cipher, uint8_t is_auth, uint8_t len_in_bits,
+		uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +23 -10
index 221262341..311b34c15 100644
@@ -462,25 +462,44 @@ test_blockcipher_one_case(const struct
 blockcipher_test_case *t,
 	}
 
 	/* Process crypto operation */
-	if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) {
-		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
-			"line %u FAILED: %s",
-			__LINE__, "Error sending packet for encryption");
-		status = TEST_FAILED;
-		goto error_exit;
-	}
+	if (cryptodev_dp_test) {
+		uint8_t is_cipher = 0, is_auth = 0;
 
-	op = NULL;
+		if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_OOP) {
+			RTE_LOG(DEBUG, USER1,
+			"QAT direct API does not support OOP, Test
 Skipped.\n");
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
 "SKIPPED");
+			status = TEST_SUCCESS;
+			goto error_exit;
+		}
+		if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
+			is_cipher = 1;
+		if (t->op_mask & BLOCKCIPHER_TEST_OP_AUTH)
+			is_auth = 1;
 
-	while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0)
-		rte_pause();
+		process_sym_hw_api_op(dev_id, 0, op, is_cipher, is_auth, 0,
+				tdata->iv.len);
+	} else {
+		if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) {
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
+				"line %u FAILED: %s",
+				__LINE__, "Error sending packet for
 encryption");
+			status = TEST_FAILED;
+			goto error_exit;
+		}

app/test/test_cryptodev_blockcipher.c test_blockcipher_one_case(const struct

Quoted 10 lines
app/test/test_cryptodev_blockcipher.c +3 -6
 
-	if (!op) {
-		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
-			"line %u FAILED: %s",
-			__LINE__, "Failed to process sym crypto op");
-		status = TEST_FAILED;
-		goto error_exit;
+		op = NULL;
+
+		while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) ==

0)

Quoted 14 lines
app/test/test_cryptodev_blockcipher.c +9 -1
+			rte_pause();
+
+		if (!op) {
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
+				"line %u FAILED: %s",
+				__LINE__, "Failed to process sym crypto op");
+			status = TEST_FAILED;
+			goto error_exit;
+		}
 	}
 
 	debug_hexdump(stdout, "m_src(after):",
--

2.20.1

author
#40 lore raw

PATCH v9 4/4 doc: add cryptodev service APIs guide

In reply to Fan Zhang

This patch updates programmer's guide to demonstrate the usage and limitations of cryptodev symmetric crypto data-path service APIs.

---
 doc/guides/prog_guide/cryptodev_lib.rst | 90 +++++++++++++++++++++++++
 doc/guides/rel_notes/release_20_11.rst  |  7 ++
 2 files changed, 97 insertions(+)
doc/guides/prog_guide/cryptodev_lib.rst +90 -0
index c14f750fa..1321e4c5d 100644
@@ -631,6 +631,96 @@ a call argument. Status different than zero must be treated as error.
 For more details, e.g. how to convert an mbuf to an SGL, please refer to an
 example usage in the IPsec library implementation.
 
+Cryptodev Direct Data-path Service API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Direct crypto data-path service are a set of APIs that especially provided for
+the external libraries/applications who want to take advantage of the rich
+features provided by cryptodev, but not necessarily depend on cryptodev
+operations, mempools, or mbufs in the their data-path implementations.
+
+The direct crypto data-path service has the following advantages:
+- Supports raw data pointer and physical addresses as input.
+- Do not require specific data structure allocated from heap, such as
+  cryptodev operation.
+- Enqueue in a burst or single operation. The service allow enqueuing in
+  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
+  enqueue one job at a time but maintaining necessary context data locally for
+  next single job enqueue operation. The latter method is especially helpful
+  when the user application's crypto operations are clustered into a burst.
+  Allowing enqueue one operation at a time helps reducing one additional loop
+  and also reduced the cache misses during the double "looping" situation.
+- Customerizable dequeue count. Instead of dequeue maximum possible operations
+  as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the
+  user to provide a callback function to decide how many operations to be
+  dequeued. This is especially helpful when the expected dequeue count is
+  hidden inside the opaque data stored during enqueue. The user can provide
+  the callback function to parse the opaque data structure.
+- Abandon enqueue and dequeue anytime. One of the drawbacks of
+  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
+  operations are: once an operation is enqueued/dequeued there is no way to
+  undo the operation. The service make the operation abandon possible by
+  creating a local copy of the queue operation data in the service context
+  data. The data will be written back to driver maintained operation data
+  when enqueue or dequeue done function is called.
+
+The cryptodev data-path service uses
+
+Cryptodev PMDs who supports this feature will have
+``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this
+feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should
+be called to get the data path service context data size. The user should
+creates a local buffer at least this size long and initialize it using
+``rte_cryptodev_dp_configure_service`` function call.
+
+The ``rte_cryptodev_dp_configure_service`` function call initialize or
+updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the
+driver specific queue pair data pointer and service context buffer, and a
+set of function pointers to enqueue and dequeue different algorithms'
+operations. The ``rte_cryptodev_dp_configure_service`` should be called when:
+
+- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
+- When different cryptodev session, security session, or session-less xform
+  is used (set ``is_update`` parameter to 1).
+
+Two different enqueue functions are provided.
+
+- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in
+  the ``rte_crypto_sym_vec`` structure.
+- ``rte_cryptodev_dp_submit_single_job``: submit single operation.
+
+Either enqueue functions will not command the crypto device to start processing
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user
+shall expect the driver only stores the necessory context data in the
+``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user
+wants to abandon the submitted operations, simply call
+``rte_cryptodev_dp_configure_service`` function instead with the parameter
+``is_update`` set to 0. The driver will recover the service context data to
+the previous state.
+
+To dequeue the operations the user also have two operations:
+
+- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The
+  user needs to provide the callback function for the driver to get the
+  dequeue count and perform post processing such as write the status field.
+- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.
+
+Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to
+merge user's local service context data with the driver's queue operation
+data. Also to abandon the dequeue operation (still keep the operations in the
+queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call
+but calling ``rte_cryptodev_dp_configure_service`` function with the parameter
+``is_update`` set to 0.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
 
doc/guides/rel_notes/release_20_11.rst +7 -0
index df227a177..159823345 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+   * **Added data-path APIs for cryptodev library.**
+
+     Cryptodev is added data-path APIs to accelerate external libraries or
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.
+
 
 Removed Items
 -------------
Signature · 1 line
2.20.1
#41 lore raw

Re: [dpdk-dev] [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide

Hi Fan,

Quoted 11 lines

Subject: [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide

This patch updates programmer's guide to demonstrate the usage and limitations of cryptodev symmetric crypto data-path service APIs.

---
 doc/guides/prog_guide/cryptodev_lib.rst | 90 +++++++++++++++++++++++++
 doc/guides/rel_notes/release_20_11.rst  |  7 ++
 2 files changed, 97 insertions(+)

We generally do not take separate patches for documentation. Squash it the patches which Implement the feature.

Quoted 11 lines
doc/guides/prog_guide/cryptodev_lib.rst +0 -0
index c14f750fa..1321e4c5d 100644
@@ -631,6 +631,96 @@ a call argument. Status different than zero must be

treated as error.
For more details, e.g. how to convert an mbuf to an SGL, please refer to an example usage in the IPsec library implementation.

Quoted 2 lines

+Cryptodev Direct Data-path Service API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

What do you mean by Direct here. It should be referenced as raw APIs? Moreover, service keyword can also be dropped. We normally use it for Software implementation of a feature which is normally done in hardware.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 640

Quoted 5 lines
doc/guides/prog_guide/cryptodev_lib.rst +5 -0
+
+Direct crypto data-path service are a set of APIs that especially provided for
+the external libraries/applications who want to take advantage of the rich
+features provided by cryptodev, but not necessarily depend on cryptodev
+operations, mempools, or mbufs in the their data-path implementations.

Raw crypto data path is a set of APIs which can be used by external libraries/applications to take advantage of the rich features provided by cryptodev, but not necessarily depend on cryptodev operations, mempools, or mbufs in the data-path implementations.

Quoted 17 lines
doc/guides/prog_guide/cryptodev_lib.rst +13 -0
+
+The direct crypto data-path service has the following advantages:
+- Supports raw data pointer and physical addresses as input.
+- Do not require specific data structure allocated from heap, such as
+  cryptodev operation.
+- Enqueue in a burst or single operation. The service allow enqueuing in
+  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
+  enqueue one job at a time but maintaining necessary context data locally for
+  next single job enqueue operation. The latter method is especially helpful
+  when the user application's crypto operations are clustered into a burst.
+  Allowing enqueue one operation at a time helps reducing one additional loop
+  and also reduced the cache misses during the double "looping" situation.
+- Customerizable dequeue count. Instead of dequeue maximum possible

operations

doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+  as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the
+  user to provide a callback function to decide how many operations to be
+  dequeued. This is especially helpful when the expected dequeue count is

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 666

Quoted 10 lines
doc/guides/prog_guide/cryptodev_lib.rst +10 -0
+  hidden inside the opaque data stored during enqueue. The user can provide
+  the callback function to parse the opaque data structure.
+- Abandon enqueue and dequeue anytime. One of the drawbacks of
+  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
+  operations are: once an operation is enqueued/dequeued there is no way to
+  undo the operation. The service make the operation abandon possible by
+  creating a local copy of the queue operation data in the service context
+  data. The data will be written back to driver maintained operation data
+  when enqueue or dequeue done function is called.
+

The language in the above test need to be re-written. Some sentences does not Make complete sense and has grammatical errors.

I suggest to have an internal review within Intel first before sending the next version.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+The cryptodev data-path service uses
+
+Cryptodev PMDs who supports this feature will have
+``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use

RTE_CRYPTODEV_FF_SYM_HW_RAW_DP looks better.

Quoted 2 lines

this
+feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should

Can be renamed as rte_cryptodev_get_raw_dp_ctx_size

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 673

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+be called to get the data path service context data size. The user should
+creates a local buffer at least this size long and initialize it using

The user should create a local buffer of at least this size and initialize it using

Quoted 1 line

+``rte_cryptodev_dp_configure_service`` function call.

rte_cryptodev_raw_dp_configure or rte_cryptodev_configure _raw_dp can be used here.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 684

Quoted 10 lines
doc/guides/prog_guide/cryptodev_lib.rst +10 -0
+
+The ``rte_cryptodev_dp_configure_service`` function call initialize or
+updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the
+driver specific queue pair data pointer and service context buffer, and a
+set of function pointers to enqueue and dequeue different algorithms'
+operations. The ``rte_cryptodev_dp_configure_service`` should be called when:
+
+- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
+- When different cryptodev session, security session, or session-less xform
+  is used (set ``is_update`` parameter to 1).

The use of is_update is not clear with above text. IMO, we do not need this flag. Whenever an update is required, we change the session information and call the Same API again and driver can copy all information blindly without checking.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 690

Quoted 6 lines
doc/guides/prog_guide/cryptodev_lib.rst +6 -0
+
+Two different enqueue functions are provided.
+
+- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in
+  the ``rte_crypto_sym_vec`` structure.
+- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

What is the meaning of single job here? Can we use multiple buffers/vectors of same session In a single job? Or we can submit only a single buffer/vector in a job?

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 699

Quoted 11 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+
+Either enqueue functions will not command the crypto device to start

processing

doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user
+shall expect the driver only stores the necessory context data in the
+``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the

user

doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+wants to abandon the submitted operations, simply call
+``rte_cryptodev_dp_configure_service`` function instead with the parameter
+``is_update`` set to 0. The driver will recover the service context data to
+the previous state.

Can you explain a use case where this is actually being used? This looks fancy but Do we have this type of requirement in any protocol stacks/specifications?
I believe it to be an extra burden on the application writer if it is not a protocol requirement.

Quoted 37 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+
+To dequeue the operations the user also have two operations:
+
+- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation.

The

doc/guides/prog_guide/cryptodev_lib.rst +20 -0
+  user needs to provide the callback function for the driver to get the
+  dequeue count and perform post processing such as write the status field.
+- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.
+
+Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to
+merge user's local service context data with the driver's queue operation
+data. Also to abandon the dequeue operation (still keep the operations in the
+queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call
+but calling ``rte_cryptodev_dp_configure_service`` function with the parameter
+``is_update`` set to 0.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
doc/guides/rel_notes/release_20_11.rst +0 -0
index df227a177..159823345 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 

doc/guides/rel_notes/release_20_11.rst New Features · line 67

Quoted 10 lines
doc/guides/rel_notes/release_20_11.rst +7 -0
+   * **Added data-path APIs for cryptodev library.**
+
+     Cryptodev is added data-path APIs to accelerate external libraries or
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.
+
 
 Removed Items
 -------------

Regards,
Akhil

#42 lore raw

Re: [dpdk-dev] [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide

Hi Akhil,

Quoted 39 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Friday, September 18, 2020 9:39 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; Anoob Joseph <[email protected]>
Subject: RE: [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide

Hi Fan,

Quoted 9 lines

Subject: [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide

This patch updates programmer's guide to demonstrate the usage and limitations of cryptodev symmetric crypto data-path service APIs.

---
 doc/guides/prog_guide/cryptodev_lib.rst | 90

+++++++++++++++++++++++++

Quoted 2 lines
 doc/guides/rel_notes/release_20_11.rst  |  7 ++
 2 files changed, 97 insertions(+)

We generally do not take separate patches for documentation. Squash it the patches which
Implement the feature.

Quoted 11 lines
doc/guides/prog_guide/cryptodev_lib.rst +0 -0
index c14f750fa..1321e4c5d 100644
@@ -631,6 +631,96 @@ a call argument. Status different than zero must be

treated as error.
For more details, e.g. how to convert an mbuf to an SGL, please refer to an example usage in the IPsec library implementation.

Quoted 7 lines
Quoted 2 lines

+Cryptodev Direct Data-path Service API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

What do you mean by Direct here. It should be referenced as raw APIs? Moreover, service keyword can also be dropped. We normally use it for Software implementation of a feature which is normally done in hardware.

You are right, raw API is a good name. Will remove service keyword.

Quoted 4 lines

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+
+Direct crypto data-path service are a set of APIs that especially provided

for

Quoted 11 lines

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 640

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+the external libraries/applications who want to take advantage of the rich
+features provided by cryptodev, but not necessarily depend on cryptodev
+operations, mempools, or mbufs in the their data-path implementations.

Raw crypto data path is a set of APIs which can be used by external libraries/applications to take advantage of the rich features provided by cryptodev,
but not necessarily depend on cryptodev operations, mempools, or mbufs in the
data-path implementations.

Thanks.

Quoted 78 lines

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 8 lines
doc/guides/prog_guide/cryptodev_lib.rst +8 -0
+
+The direct crypto data-path service has the following advantages:
+- Supports raw data pointer and physical addresses as input.
+- Do not require specific data structure allocated from heap, such as
+  cryptodev operation.
+- Enqueue in a burst or single operation. The service allow enqueuing in
+  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
+  enqueue one job at a time but maintaining necessary context data locally

for

Quoted 1 line

+ next single job enqueue operation. The latter method is especially

helpful

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+  when the user application's crypto operations are clustered into a burst.
+  Allowing enqueue one operation at a time helps reducing one additional

loop

Quoted 4 lines

+ and also reduced the cache misses during the double "looping" situation. +- Customerizable dequeue count. Instead of dequeue maximum possible operations
+ as same as ``rte_cryptodev_dequeue_burst`` operation, the service

allows the

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+  user to provide a callback function to decide how many operations to be
+  dequeued. This is especially helpful when the expected dequeue count

is

Quoted 1 line

+ hidden inside the opaque data stored during enqueue. The user can

provide

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+  the callback function to parse the opaque data structure.
+- Abandon enqueue and dequeue anytime. One of the drawbacks of
+  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
+  operations are: once an operation is enqueued/dequeued there is no

way to

Quoted 1 line

+ undo the operation. The service make the operation abandon possible

by

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 666

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+  creating a local copy of the queue operation data in the service context
+  data. The data will be written back to driver maintained operation data
+  when enqueue or dequeue done function is called.
+

The language in the above test need to be re-written. Some sentences does not
Make complete sense and has grammatical errors.

I suggest to have an internal review within Intel first before sending the next version.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+The cryptodev data-path service uses
+
+Cryptodev PMDs who supports this feature will have
+``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To

use

RTE_CRYPTODEV_FF_SYM_HW_RAW_DP looks better.

Quoted 2 lines

this
+feature the function ``rte_cryptodev_get_dp_service_ctx_data_size``

should

Can be renamed as rte_cryptodev_get_raw_dp_ctx_size

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 673

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+be called to get the data path service context data size. The user should
+creates a local buffer at least this size long and initialize it using

The user should create a local buffer of at least this size and initialize it using

Quoted 1 line

+``rte_cryptodev_dp_configure_service`` function call.

rte_cryptodev_raw_dp_configure or rte_cryptodev_configure _raw_dp can be used here.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+
+The ``rte_cryptodev_dp_configure_service`` function call initialize or
+updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains

the

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+driver specific queue pair data pointer and service context buffer, and a
+set of function pointers to enqueue and dequeue different algorithms'
+operations. The ``rte_cryptodev_dp_configure_service`` should be called

when:

Quoted 3 lines

+

+- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
+- When different cryptodev session, security session, or session-less
Quoted 9 lines

xform

Quoted 1 line

+ is used (set ``is_update`` parameter to 1).

The use of is_update is not clear with above text. IMO, we do not need this flag.
Whenever an update is required, we change the session information and call the
Same API again and driver can copy all information blindly without checking.

The flag is very important
- When flag is not set much queue specific data will be write into driver specific Data field in the service data. This data is written back to the driver when "submit_done" or "dequeue_done" function is called.
- When flag is set the above step is not executed, driver only updates the function handlers attached the service data for different algorithm used. Will update this into the description.

Quoted 5 lines

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+
+Two different enqueue functions are provided.
+
+- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations

stored in

Quoted 7 lines
Quoted 2 lines

+ the ``rte_crypto_sym_vec`` structure.
+- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

What is the meaning of single job here? Can we use multiple buffers/vectors of same session
In a single job? Or we can submit only a single buffer/vector in a job?

Same as CPU crypto, a sym vec will contains one or more jobs with same session. But it is not an inline function as it assumes we are bursting multiple ops up. Single job - the sole purpose is to use it as an inline function that pushes one job into the queue, but not kicking the HW to start processing.
When the raw APIs are used in external application/lib that also works in burst, such as VPP, single job submission is very useful to reduce the cycle cost. Since they also have data structure translation to their specific data structure and also work in burst, you don't want the application loops a burst of 32
ops to translate into a burst of DPDK crypto sym vec first, passing to the driver, the driver then loops the jobs the second time to write to the HW one by one. Use of inline "submit single" API can help reducing the cycles and cache misses, especially when the burst size is 256.

Quoted 12 lines
Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+
+Either enqueue functions will not command the crypto device to start

processing
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then

the user

Quoted 2 lines

+shall expect the driver only stores the necessory context data in the +``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If

the

Quoted 3 lines

user
+wants to abandon the submitted operations, simply call +``rte_cryptodev_dp_configure_service`` function instead with the

parameter

Quoted 8 lines
Quoted 2 lines

+``is_update`` set to 0. The driver will recover the service context data to +the previous state.

Can you explain a use case where this is actually being used? This looks fancy but
Do we have this type of requirement in any protocol stacks/specifications? I believe it to be an extra burden on the application writer if it is not a protocol requirement.

It is not protocol stack specific.
For now all crypto ops need to be translated into HW device/queue operation data in a looped manner. The context between last and current ops is stored in the driver and is updated within enqueue burst function (e.g. for QAT such context is the shadow copy of the QAT queue tail/head).

As mentioned earlier such burst operation introduce more cycles and cache Misses. So we want to introduce single job enqueue in DPDK so we don't loop the same burst twice. However we have to take care of the context between last and current submit single function. The idea is let the user allocate the buffer for that. The "is_update" param is used to tell the driver if the it
needs to write the context to the driver (is_update == 0), or only update context unrelated data inside dp_service buffer (updating function handler to the algo etc). When the burst is processed the user can call submit_done function so the user maintained context is written back to the driver. So in next burst enqueue the user should call ``rte_cryptodev_dp_configure_service`` with "is_update" = 0 before submit any jobs into the driver, or when the session is changed use "is_update" = 1 to only update the function pointer.

Quoted 59 lines

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 5 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+
+To dequeue the operations the user also have two operations:
+
+- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue

operation.

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 3 lines

The

doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+  user needs to provide the callback function for the driver to get the
+  dequeue count and perform post processing such as write the status

field.

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 3 lines

+- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.

doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+
+Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is

used to

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+merge user's local service context data with the driver's queue operation
+data. Also to abandon the dequeue operation (still keep the operations in

the

Quoted 1 line

+queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done``

function call

Quoted 1 line

+but calling ``rte_cryptodev_dp_configure_service`` function with the

parameter

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 7 lines

+``is_update`` set to 0.

doc/guides/prog_guide/cryptodev_lib.rst +6 -0
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with

rte_cryptodev_enqueue_burst, or

Quoted 16 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
doc/guides/rel_notes/release_20_11.rst +0 -0
index df227a177..159823345 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 

doc/guides/rel_notes/release_20_11.rst New Features · line 67

Quoted 10 lines
doc/guides/rel_notes/release_20_11.rst +7 -0
+   * **Added data-path APIs for cryptodev library.**
+
+     Cryptodev is added data-path APIs to accelerate external libraries or
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.
+
 
 Removed Items
 -------------

Regards,
Akhil

#43 lore raw

Re: [dpdk-dev] [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide

In reply to Akhil Goyal

Hi Akhil

Quoted 1 line

-----Original Message-----

...

Quoted 12 lines
Quoted 4 lines
(quoted diff) +2 -0
+
+Either enqueue functions will not command the crypto device to start

processing
+until ``rte_cryptodev_dp_submit_done`` function is called. Before then

the user

Quoted 2 lines

+shall expect the driver only stores the necessory context data in the +``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If

the

Quoted 3 lines

user
+wants to abandon the submitted operations, simply call +``rte_cryptodev_dp_configure_service`` function instead with the

parameter

Quoted 9 lines
Quoted 2 lines

+``is_update`` set to 0. The driver will recover the service context data to +the previous state.

Can you explain a use case where this is actually being used? This looks fancy but
Do we have this type of requirement in any protocol stacks/specifications? I believe it to be an extra burden on the application writer if it is not a protocol requirement.

I missed responding this one.
The requirement comes from cooping with VPP crypto framework.

The reason for this feature is fill the gap of cryptodev enqueue and dequeue operations.
If the user application/library uses the approach similar to " rte_crypto_sym_vec" (such as VPP vnet_crypto_async_frame_t) that clusters multiple crypto ops as a burst, the application requires enqueuing and dequeuing all ops as a whole inside, or nothing.
It is very slow for rte_cryptodev_enqueue/dequeue_burst to achieve this today as the user has no control over how many ops I want to enqueue/dequeue preciously. For example I want to enqueue a " rte_crypto_sym_vec" buffer contains 32 descriptors, and stores " rte_crypto_sym_vec" as opaque data in enqueue, but rte_cryptodev_enqueue_burst returns 31, I have no option but cache the 1 left job for next enqueue attempt (or I manually check the inflight count in every enqueue). Also during dequeue since the number "32" is stored inside rte_crypto_sym_vec.num, I have no way to know how many ops to dequeue, but blindly dequeue them and store in a software ring, parse the the dequeue count from retrieved opaque data, and check the ring count against dequeue count.

With the new way provided we can easily achieve the goal. For HW crypto PMD such implementation is relatively easy, we only need to create a shadow copy to the queue pair data in ``rte_crypto_dp_service_ctx`` and updates in enqueue/dequeue, when "enqueue/dequeue_done" is called the queue is kicked to start processing jobs already set in the queue and merge the shadow copy queue data into driver maintained one.

Regards,
Fan

author
#44 lore raw

v10 cover cryptodev: add raw data-path APIs

In reply to Fan Zhang

The Crypto Raw data-path APIs are a set of APIs are designed to enable externel libraries/applications which want to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but in a manner that is not dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe input data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or dequeue at any time, and operation status translation and set on the fly.

V10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: change crypto symmetric vector structure
cryptodev: add raw crypto data-path APIs crypto/qat: add raw crypto data-path API support test/crypto: add unit-test for cryptodev raw API test

 app/test/test_cryptodev.c                     | 784 ++++++++++++++-
 app/test/test_cryptodev.h                     |  12 +
 app/test/test_cryptodev_blockcipher.c         |  58 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  96 +-
 doc/guides/rel_notes/release_20_11.rst        |  10 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c      |  18 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    |   9 +-
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 951 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto_sym.h         |  40 +-
 lib/librte_cryptodev/rte_cryptodev.c          | 104 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 354 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 +-
 .../rte_cryptodev_version.map                 |  11 +
 lib/librte_ipsec/esp_inb.c                    |  12 +-
 lib/librte_ipsec/esp_outb.c                   |  12 +-
 lib/librte_ipsec/misc.h                       |   6 +-
 19 files changed, 2437 insertions(+), 108 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#45 lore raw

PATCH v10 1/4 cryptodev: change crypto symmetric vector structure

This patch updates ``rte_crypto_sym_vec`` structure to add support for both cpu_crypto synchrounous operation and asynchronous raw data-path APIs. The patch also includes AESNI-MB and AESNI-GCM PMD changes, unit test changes and documentation updates.

---
 app/test/test_cryptodev.c                  | 25 ++++++++------
 doc/guides/prog_guide/cryptodev_lib.rst    |  3 +-
 doc/guides/rel_notes/release_20_11.rst     |  3 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   | 18 +++++-----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  9 +++--
 lib/librte_cryptodev/rte_crypto_sym.h      | 40 ++++++++++++++++------
 lib/librte_ipsec/esp_inb.c                 | 12 +++----
 lib/librte_ipsec/esp_outb.c                | 12 +++----
 lib/librte_ipsec/misc.h                    |  6 ++--
 9 files changed, 79 insertions(+), 49 deletions(-)
app/test/test_cryptodev.c +15 -10
index 70bf6fe2c..99f1eed82 100644
@@ -151,11 +151,11 @@ static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, aad_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -171,13 +171,17 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->aead.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
+	symvec.aad = &aad_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	/* for CPU crypto the IOVA address is not required */
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->aead.digest.data;
+	aad_ptr.va = (void *)sop->aead.aad.data;
+
 	ofs.raw = 0;
 
 	n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
@@ -193,11 +197,11 @@ static void
 process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -213,13 +217,14 @@ process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->auth.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->auth.digest.data;
+
 	ofs.raw = 0;
 	ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
 	ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
doc/guides/prog_guide/cryptodev_lib.rst +2 -1
index c14f750fa..e7ba35c2d 100644
@@ -620,7 +620,8 @@ operation descriptor (``struct rte_crypto_sym_vec``) containing:
   descriptors of performed operations (``struct rte_crypto_sgl``). Each instance
   of ``struct rte_crypto_sgl`` consists of a number of segments and a pointer to
   an array of segment descriptors ``struct rte_crypto_vec``;
-- pointers to arrays of size ``num`` containing IV, AAD and digest information,
+- pointers to arrays of size ``num`` containing IV, AAD and digest information
+  in the ``cpu_crypto`` sub-structure,
 - pointer to an array of size ``num`` where status information will be stored
   for each operation.
 
doc/guides/rel_notes/release_20_11.rst +3 -0
index 73ac08fb0..20ebaef5b 100644
@@ -135,6 +135,9 @@ API Changes
 
 * bpf: ``RTE_BPF_XTYPE_NUM`` has been dropped from ``rte_bpf_xtype``.
 
+* The structure ``rte_crypto_sym_vec`` is updated to support both cpu_crypto
+  synchrounous operation and asynchronous raw data-path APIs.
+
 
 ABI Changes
 -----------
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -8
index 1d2a0ce00..973b61bd6 100644
@@ -464,9 +464,10 @@ aesni_gcm_sgl_encrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -482,9 +483,10 @@ aesni_gcm_sgl_decrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		 vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -505,9 +507,9 @@ aesni_gmac_sgl_generate(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -528,9 +530,9 @@ aesni_gmac_sgl_verify(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +4 -5
index 1bddbcf74..01b3bfc29 100644
@@ -1744,7 +1744,7 @@ generate_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			memcpy(vec->digest[i], dgst[i], len);
+			memcpy(vec->digest[i].va, dgst[i], len);
 			k++;
 		}
 	}
@@ -1760,7 +1760,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			if (memcmp(vec->digest[i], dgst[i], len) != 0)
+			if (memcmp(vec->digest[i].va, dgst[i], len) != 0)
 				vec->status[i] = EBADMSG;
 			else
 				k++;
@@ -1823,9 +1823,8 @@ aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
 		}
 
 		/* Submit job for processing */
-		set_cpu_mb_job_params(job, s, sofs, buf, len,
-			vec->iv[i], vec->aad[i], tmp_dgst[i],
-			&vec->status[i]);
+		set_cpu_mb_job_params(job, s, sofs, buf, len, vec->iv[i].va,
+			vec->aad[i].va, tmp_dgst[i], &vec->status[i]);
 		job = submit_sync_job(mb_mgr);
 		j++;
 
lib/librte_cryptodev/rte_crypto_sym.h +29 -11
index f29c98051..8201189e0 100644
@@ -51,26 +51,44 @@ struct rte_crypto_sgl {
 };
 
 /**
- * Synchronous operation descriptor.
- * Supposed to be used with CPU crypto API call.
+ * Crypto virtual and IOVA address descriptor, used to describe cryptographic
+ * data buffer without the length information. The length information is
+ * normally predefined during session creation.
+ */
+struct rte_crypto_va_iova_ptr {
+	void *va;
+	rte_iova_t *iova;
+};
+
+/**
+ * Raw data operation descriptor.
+ * Supposed to be used with synchronous CPU crypto API call or asynchronous
+ * RAW data path API call.
  */
 struct rte_crypto_sym_vec {
+	/** number of operations to perform */
+	uint32_t num;
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
+	/** array of pointers to cipher IV */
+	struct rte_crypto_va_iova_ptr *iv;
 	/** array of pointers to digest */
-	void **digest;
+	struct rte_crypto_va_iova_ptr *digest;
+
+	__extension__
+	union {
+		/** array of pointers to auth IV, used for chain operation */
+		struct rte_crypto_va_iova_ptr *auth_iv;
+		/** array of pointers to AAD, used for AEAD operation */
+		struct rte_crypto_va_iova_ptr *aad;
+	};
+
 	/**
 	 * array of statuses for each operation:
-	 *  - 0 on success
-	 *  - errno on error
+	 * - 0 on success
+	 * - errno on error
 	 */
 	int32_t *status;
-	/** number of operations to perform */
-	uint32_t num;
 };
 
 /**
lib/librte_ipsec/esp_inb.c +6 -6
index 96eec0131..2b1df6a03 100644
@@ -693,9 +693,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 	struct rte_ipsec_sa *sa;
 	struct replay_sqn *rsn;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -720,9 +720,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 				l4ofs + k, rc, ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/esp_outb.c +6 -6
index fb9d5864c..1e181cf2c 100644
@@ -449,9 +449,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 	uint32_t i, k, n;
 	uint32_t l2, l3;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -488,9 +488,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 				ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/misc.h +4 -2
index 1b543ed87..79b9a2076 100644
@@ -112,7 +112,9 @@ mbuf_cut_seg_ofs(struct rte_mbuf *mb, struct rte_mbuf *ms, uint32_t ofs,
 static inline void
 cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 	union rte_crypto_sym_ofs ofs, struct rte_mbuf *mb[],
-	void *iv[], void *aad[], void *dgst[], uint32_t l4ofs[],
+	struct rte_crypto_va_iova_ptr iv[],
+	struct rte_crypto_va_iova_ptr aad[],
+	struct rte_crypto_va_iova_ptr dgst[], uint32_t l4ofs[],
 	uint32_t clen[], uint32_t num)
 {
 	uint32_t i, j, n;
@@ -136,8 +138,8 @@ cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 			/* fill the request structure */
 			symvec.sgl = &vecpkt[j];
 			symvec.iv = &iv[j];
-			symvec.aad = &aad[j];
 			symvec.digest = &dgst[j];
+			symvec.aad = &aad[j];
 			symvec.status = &st[j];
 			symvec.num = i - j;
 
Signature · 1 line
2.20.1
Acked-by on v10 1/4
#46 lore raw

Re: [dpdk-dev] [dpdk-dev v10 1/4] cryptodev: change crypto symmetric vector structure

Quoted 12 lines

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: Thursday, 24 September, 2020 18:34
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>
Subject: [dpdk-dev v10 1/4] cryptodev: change crypto symmetric vector structure

Quoted 6 lines

This patch updates ``rte_crypto_sym_vec`` structure to add support for both cpu_crypto synchrounous operation and asynchronous raw data-path APIs. The patch also includes AESNI-MB and AESNI-GCM PMD changes, unit test changes and documentation updates.

Acked-by on v10 1/4
#47 lore raw

Re: [dpdk-dev] [dpdk-dev v10 1/4] cryptodev: change crypto symmetric vector structure

In reply to Fan Zhang
Quoted 9 lines

This patch updates ``rte_crypto_sym_vec`` structure to add support for both cpu_crypto synchrounous operation and asynchronous raw data-path APIs. The patch also includes AESNI-MB and AESNI-GCM PMD changes, unit test changes and documentation updates.

---

Quoted 1 line

2.20.1

author
#48 lore raw

PATCH v10 2/4 cryptodev: add raw crypto data-path APIs

In reply to Fan Zhang

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

---
 doc/guides/prog_guide/cryptodev_lib.rst       |  93 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_crypto_sym.h         |   2 +-
 lib/librte_cryptodev/rte_cryptodev.c          | 104 +++++
 lib/librte_cryptodev/rte_cryptodev.h          | 354 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  11 +
 7 files changed, 614 insertions(+), 4 deletions(-)
doc/guides/prog_guide/cryptodev_lib.rst +93 -0
index e7ba35c2d..5fe6c3c24 100644
@@ -632,6 +632,99 @@ a call argument. Status different than zero must be treated as error.
 For more details, e.g. how to convert an mbuf to an SGL, please refer to an
 example usage in the IPsec library implementation.
 
+Cryptodev Raw Data-path APIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Crypto Raw data-path APIs are a set of APIs are designed to enable
+externel libraries/applications which want to leverage the cryptographic
+processing provided by DPDK crypto PMDs through the cryptodev API but in a
+manner that is not dependent on native DPDK data structures (eg. rte_mbuf,
+rte_crypto_op, ... etc) in their data-path implementation.
+
+The raw data-path APIs have the following advantages:
+- External data structure friendly design. The new APIs uses the operation
+  descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
+  IOVA addresses as input. Moreover, the APIs does not require the user to
+  allocate the descriptor from mempool, nor requiring mbufs to describe input
+  data's virtual and IOVA addresses. All these features made the translation
+  from user's own data structure into the descriptor easier and more efficient.
+- Flexible enqueue and dequeue operation. The raw data-path APIs gives the
+  user more control to the enqueue and dequeue operations, including the
+  capability of precious enqueue/dequeue count, abandoning enqueue or dequeue
+  at any time, and operation status translation and set on the fly.
+
+Cryptodev PMDs who supports the raw data-path APIs will have
+``RTE_CRYPTODEV_FF_SYM_HW_RAW_DP`` feature flag presented. To use this
+feature, the user should create a local ``struct rte_crypto_raw_dp_ctx``
+buffer and extend to at least the length returned by
+``rte_cryptodev_raw_get_dp_context_size`` function call. The created buffer
+is then configured using ``rte_cryptodev_raw_configure_dp_context`` function.
+The library and the crypto device driver will then configure the buffer and
+write necessary temporary data into the buffer for later enqueue and dequeue
+operations. The temporary data may be treated as the shadow copy of the
+driver's private queue pair data.
+
+After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is then
+attached either the cryptodev sym session, the rte_security session, or the
+cryptodev xform for session-less operation by
+``rte_cryptodev_raw_attach_session`` function. With the session or xform
+information the driver will set the corresponding enqueue and dequeue function
+handlers to the ``struct rte_crypto_raw_dp_ctx`` buffer.
+
+After the session is attached, the ``struct rte_crypto_raw_dp_ctx`` buffer is
+now ready for enqueue and dequeue operation. There are two different enqueue
+functions: ``rte_cryptodev_raw_enqueue`` to enqueue single descriptor,
+and ``rte_cryptodev_raw_enqueue_burst`` to enqueue multiple descriptors.
+In case of the application uses similar approach to
+``struct rte_crypto_sym_vec`` to manage its data burst but with different
+data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function may be
+less efficient as this is a situation where the application has to loop over
+all crypto descriptors to assemble the ``struct rte_crypto_sym_vec`` buffer
+from its own data structure, and then the driver will loop over them again to
+translate every crypto job to the driver's specific queue data. The
+``rte_cryptodev_raw_enqueue`` should be used to save one loop for each data
+burst instead.
+
+During the enqueue, the cryptodev driver only sets the enqueued descriptors
+into the device queue but not initiates the device to start processing them.
+The temporary queue pair data changes in relation to the enqueued descriptors
+may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the reference
+to the next enqueue function call. When ``rte_cryptodev_raw_enqueue_done`` is
+called, the driver will initiate the processing of all enqueued descriptors and
+merge the temporary queue pair data changes into the driver's private queue
+pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice without
+``rte_cryptodev_dp_enqueue_done`` call in between will invalidate the temporary
+data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is useful
+when the user wants to abandon partially enqueued data for a failed enqueue
+burst operation and try enqueuing in a whole later.
+
+Similar as enqueue, there are two dequeue functions:
+``rte_cryptodev_raw_dequeue`` for dequeing single descriptor, and
+``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of descriptor. The
+dequeue functions only writes back the user data that was passed to the driver
+during inqueue, and inform the application the operation status.
+Different than ``rte_cryptodev_dequeue_burst`` which the user can only
+set an expected dequeue count and needs to read from dequeued cryptodev
+operations' status field, the raw data-path dequeue burst function allows
+the user to provide callback functions to retrieve dequeue
+count from the enqueued user data, and write the expected status value to the
+user data on the fly.
+
+Same as enqueue, both ``rte_cryptodev_raw_dequeue`` and
+``rte_cryptodev_raw_dequeue_burst`` will not wipe the dequeued descriptors
+from cryptodev queue unless ``rte_cryptodev_dp_dequeue_done`` is called. The
+dequeue related temporary queue data will be merged into the driver's private
+queue data in the function call.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
 
doc/guides/rel_notes/release_20_11.rst +7 -0
index 20ebaef5b..d3d9f82f7 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+   * **Added raw data-path APIs for cryptodev library.**
+
+     Cryptodev is added raw data-path APIs to accelerate external libraries or
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.
+
 
 Removed Items
 -------------
lib/librte_cryptodev/rte_crypto_sym.h +1 -1
index 8201189e0..e1f23d303 100644
@@ -57,7 +57,7 @@ struct rte_crypto_sgl {
  */
 struct rte_crypto_va_iova_ptr {
 	void *va;
-	rte_iova_t *iova;
+	rte_iova_t iova;
 };
 
 /**
lib/librte_cryptodev/rte_cryptodev.c +104 -0

110 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +353 -1

366 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +45 -2

59 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +11 -0
index 02f6dcf72..bc4cd1ea5 100644
@@ -105,4 +105,15 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_raw_attach_session;
+	rte_cryptodev_raw_configure_dp_context;
+	rte_cryptodev_raw_get_dp_context_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
Signature · 1 line
2.20.1
Acked-by on v10 2/4
#49 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Quoted 12 lines

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: Thursday, 24 September, 2020 18:34
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Quoted 6 lines

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

#50 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

In reply to Fan Zhang

Hi Fan,

Quoted 27 lines

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

---
 doc/guides/prog_guide/cryptodev_lib.rst       |  93 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_crypto_sym.h         |   2 +-
 lib/librte_cryptodev/rte_cryptodev.c          | 104 +++++
 lib/librte_cryptodev/rte_cryptodev.h          | 354 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  11 +
 7 files changed, 614 insertions(+), 4 deletions(-)
doc/guides/prog_guide/cryptodev_lib.rst +0 -0
index e7ba35c2d..5fe6c3c24 100644
@@ -632,6 +632,99 @@ a call argument. Status different than zero must be

treated as error.
For more details, e.g. how to convert an mbuf to an SGL, please refer to an example usage in the IPsec library implementation.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 638

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+Cryptodev Raw Data-path APIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Crypto Raw data-path APIs are a set of APIs are designed to enable

The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic

Quoted 16 lines
doc/guides/prog_guide/cryptodev_lib.rst +15 -0
+externel libraries/applications which want to leverage the cryptographic
+processing provided by DPDK crypto PMDs through the cryptodev API but in a
+manner that is not dependent on native DPDK data structures (eg. rte_mbuf,
+rte_crypto_op, ... etc) in their data-path implementation.
+
+The raw data-path APIs have the following advantages:
+- External data structure friendly design. The new APIs uses the operation
+  descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
+  IOVA addresses as input. Moreover, the APIs does not require the user to
+  allocate the descriptor from mempool, nor requiring mbufs to describe input
+  data's virtual and IOVA addresses. All these features made the translation
+  from user's own data structure into the descriptor easier and more efficient.
+- Flexible enqueue and dequeue operation. The raw data-path APIs gives the
+  user more control to the enqueue and dequeue operations, including the
+  capability of precious enqueue/dequeue count, abandoning enqueue or

dequeue

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 656

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+  at any time, and operation status translation and set on the fly.
+
+Cryptodev PMDs who supports the raw data-path APIs will have

Cryptodev PMDs which support raw data-path APIs will have

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 660

Quoted 4 lines

+``RTE_CRYPTODEV_FF_SYM_HW_RAW_DP`` feature flag presented. To use this

doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+feature, the user should create a local ``struct rte_crypto_raw_dp_ctx``
+buffer and extend to at least the length returned by
+``rte_cryptodev_raw_get_dp_context_size`` function call. The created buffer

``rte_cryptodev_get_raw_dp_ctx_size``

Quoted 1 line

+is then configured using ``rte_cryptodev_raw_configure_dp_context`` function.

rte_cryptodev _configure_raw_dp_ctx

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 668

Quoted 7 lines
doc/guides/prog_guide/cryptodev_lib.rst +7 -0
+The library and the crypto device driver will then configure the buffer and
+write necessary temporary data into the buffer for later enqueue and dequeue
+operations. The temporary data may be treated as the shadow copy of the
+driver's private queue pair data.
+
+After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is then
+attached either the cryptodev sym session, the rte_security session, or the

attached either with the cryptodev sym session, the rte_security session, or the

Quoted 2 lines

+cryptodev xform for session-less operation by +``rte_cryptodev_raw_attach_session`` function. With the session or xform

``rte_cryptodev_attach_raw_session`` API

Quoted 32 lines

+information the driver will set the corresponding enqueue and dequeue function

doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+handlers to the ``struct rte_crypto_raw_dp_ctx`` buffer.
+
+After the session is attached, the ``struct rte_crypto_raw_dp_ctx`` buffer is
+now ready for enqueue and dequeue operation. There are two different

enqueue

doc/guides/prog_guide/cryptodev_lib.rst +5 -0
+functions: ``rte_cryptodev_raw_enqueue`` to enqueue single descriptor,
+and ``rte_cryptodev_raw_enqueue_burst`` to enqueue multiple descriptors.
+In case of the application uses similar approach to
+``struct rte_crypto_sym_vec`` to manage its data burst but with different
+data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function may

be

doc/guides/prog_guide/cryptodev_lib.rst +11 -0
+less efficient as this is a situation where the application has to loop over
+all crypto descriptors to assemble the ``struct rte_crypto_sym_vec`` buffer
+from its own data structure, and then the driver will loop over them again to
+translate every crypto job to the driver's specific queue data. The
+``rte_cryptodev_raw_enqueue`` should be used to save one loop for each data
+burst instead.
+
+During the enqueue, the cryptodev driver only sets the enqueued descriptors
+into the device queue but not initiates the device to start processing them.
+The temporary queue pair data changes in relation to the enqueued descriptors
+may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the

reference
+to the next enqueue function call. When ``rte_cryptodev_raw_enqueue_done`` is

doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+called, the driver will initiate the processing of all enqueued descriptors and
+merge the temporary queue pair data changes into the driver's private queue
+pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice without
+``rte_cryptodev_dp_enqueue_done`` call in between will invalidate the

temporary

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 699

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is useful
+when the user wants to abandon partially enqueued data for a failed enqueue
+burst operation and try enqueuing in a whole later.

This feature may not be supported by all the HW PMDs, Can there be a way to bypass this done API?

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 705

Quoted 6 lines
doc/guides/prog_guide/cryptodev_lib.rst +6 -0
+
+Similar as enqueue, there are two dequeue functions:
+``rte_cryptodev_raw_dequeue`` for dequeing single descriptor, and
+``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of descriptor. The
+dequeue functions only writes back the user data that was passed to the driver
+during inqueue, and inform the application the operation status.

during enqueue, and inform the operation status to the application.

Quoted 32 lines
doc/guides/prog_guide/cryptodev_lib.rst +10 -0
+Different than ``rte_cryptodev_dequeue_burst`` which the user can only
+set an expected dequeue count and needs to read from dequeued cryptodev
+operations' status field, the raw data-path dequeue burst function allows
+the user to provide callback functions to retrieve dequeue
+count from the enqueued user data, and write the expected status value to the
+user data on the fly.
+
+Same as enqueue, both ``rte_cryptodev_raw_dequeue`` and
+``rte_cryptodev_raw_dequeue_burst`` will not wipe the dequeued descriptors
+from cryptodev queue unless ``rte_cryptodev_dp_dequeue_done`` is called.

The

doc/guides/prog_guide/cryptodev_lib.rst +12 -0
+dequeue related temporary queue data will be merged into the driver's private
+queue data in the function call.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
doc/guides/rel_notes/release_20_11.rst +0 -0
index 20ebaef5b..d3d9f82f7 100644

doc/guides/rel_notes/release_20_11.rst New Features · line 63

Quoted 9 lines
doc/guides/rel_notes/release_20_11.rst +6 -0
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+   * **Added raw data-path APIs for cryptodev library.**
+
+     Cryptodev is added raw data-path APIs to accelerate external libraries or
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.

Raw crypto data path APIs are added to accelerate external libraries or Applications which need to perform crypto processing on raw buffers and Not dependent on rte_mbufs or rte_crypto_op mempools.

Quoted 10 lines
doc/guides/rel_notes/release_20_11.rst +1 -0
+
 
 Removed Items
 -------------
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
index 8201189e0..e1f23d303 100644

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 61

Quoted 6 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -1
@@ -57,7 +57,7 @@ struct rte_crypto_sgl {
  */
 struct rte_crypto_va_iova_ptr {
 	void *va;
-	rte_iova_t *iova;
+	rte_iova_t iova;
 };

This should be part of 1/4 of this patchset.

Quoted 8 lines
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
 /**
lib/librte_cryptodev/rte_cryptodev.c +0 -0
index 1dd795bcb..daeb5f504 100644

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t · line 1919

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.c +2 -0
@@ -1914,6 +1914,110 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t
 dev_id,
 	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);
 }
 
+int
+rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id)

As suggested above raw_dp should be used as keyword in all the APIs Hence it should be rte_cryptodev_get_raw_dp_ctx_size

Quoted 16 lines
lib/librte_cryptodev/rte_cryptodev.c +15 -0
+{
+	struct rte_cryptodev *dev;
+	int32_t size = sizeof(struct rte_crypto_raw_dp_ctx);
+	int32_t priv_size;
+
+	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+	if (*dev->dev_ops->get_drv_ctx_size == NULL ||
+		!(dev->feature_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)) {
+		return -ENOTSUP;
+	}
+

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1943

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+	priv_size = (*dev->dev_ops->get_drv_ctx_size)(dev);
+	if (priv_size < 0)
+		return -ENOTSUP;
+
+	return RTE_ALIGN_CEIL((size + priv_size), 8);
+}
+
+int
+rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx)

rte_cryptodev_configure_raw_dp_ctx

Quoted 23 lines
lib/librte_cryptodev/rte_cryptodev.c +23 -0
+{
+	struct rte_cryptodev *dev;
+	union rte_cryptodev_session_ctx sess_ctx = {NULL};
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id,
+			RTE_CRYPTO_OP_WITH_SESSION, sess_ctx, ctx);
+}
+
+int
+rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx)
+{
+	struct rte_cryptodev *dev;

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1976

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, sess_type,
+			session_ctx, ctx);

What is the difference between rte_cryptodev_raw_configure_dp_context and rte_cryptodev_raw_attach_session?
And if at all it is needed, then it should be rte_cryptodev_attach_raw_dp_session. IMO attach is not needed, I am not clear about it.

You are calling the same dev_ops for both - one with explicit session time and other From an argument.

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1984

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.c +8 -0
+}
+
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **user_data)
+{
+	if (vec->num == 1) {

Why do we need this check? I think user is aware that for enqueuing 1 vector, He should use the other API. Driver will be doing the enqueue operation only one time.

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1993

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +9 -0
+		vec->status[0] = rte_cryptodev_raw_enqueue(ctx, vec->sgl-
 >vec,
+			vec->sgl->num, ofs, vec->iv, vec->digest, vec->aad,
+			user_data[0]);
+		return (vec->status[0] == 0) ? 1 : 0;
+	}
+
+	return (*ctx->enqueue_burst)(ctx->qp_data, ctx->drv_ctx_data, vec,
+			ofs, user_data);
+}

Where are rte_cryptodev_raw_enqueue and rte_cryptodev_raw_dequeue ??

Quoted 36 lines
lib/librte_cryptodev/rte_cryptodev.c +27 -0
+
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success_jobs)
+{
+	return (*ctx->dequeue_burst)(ctx->qp_data, ctx->drv_ctx_data,
+		get_dequeue_count, post_dequeue, out_user_data,
+		is_user_data_array, n_success_jobs);
+}
+

/** Initialise rte_crypto_op mempool element */
static void
rte_crypto_op_init(struct rte_mempool *mempool,

lib/librte_cryptodev/rte_cryptodev.h +0 -0
index 7b3ebc20f..3579ab66e 100644

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_asym_get_xform_enum(enum · line 472

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +1 -1
@@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum
 rte_crypto_asym_xform_type *xform_enum,
 /**< Support symmetric session-less operations */
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL
 << 23)
 /**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_SYM_HW_RAW_DP			(1ULL
 << 24)

RTE_CRYPTODEV_FF_SYM_RAW_DP should be better

Add this in doc/guides/cryptodevs/features/default.ini as well in this patch.

Quoted 164 lines
lib/librte_cryptodev/rte_cryptodev.h +1 -0
+/**< Support accelerated specific raw data-path APIs */

/**
* Get the name of a crypto device feature flag

lib/librte_cryptodev/rte_cryptodev.h +154 -0
@@ -1351,6 +1352,357 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t
 dev_id,
 	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
 	struct rte_crypto_sym_vec *vec);
 
+/**
+ * Get the size of the raw data-path context buffer.
+ *
+ * @param	dev_id		The device identifier.
+ *
+ * @return
+ *   - If the device supports raw data-path APIs, return the context size.
+ *   - If the device does not support the APIs, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less xform
+ * pointer.
+ */
+union rte_cryptodev_session_ctx {
+	struct rte_cryptodev_sym_session *crypto_sess;
+	struct rte_crypto_sym_xform *xform;
+	struct rte_security_session *sec_sess;
+};
+
+/**
+ * Enqueue a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	vec		The array of descriptor vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	user_data	The array of user data for dequeue later.
+ * @return
+ *   - The number of descriptors successfully submitted.
+ */
+typedef uint32_t (*cryptodev_dp_sym_enqueue_burst_t)(
+	void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec,
+	union rte_crypto_sym_ofs ofs, void *user_data[]);
+
+/**
+ * Enqueue single descriptor into device queue but the driver will not start
+ * processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	data_vec	The buffer data vector.
+ * @param	n_data_vecs	Number of buffer data vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	iv		IV virtual and IOVA addresses
+ * @param	digest		digest virtual and IOVA addresses
+ * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA addresses,
+ *				depends on the algorithm used.
+ * @param	user_data	The user data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_sym_enqueue_t)(
+	void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec,
+	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_va_iova_ptr *iv,
+	struct rte_crypto_va_iova_ptr *digest,
+	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+	void *user_data);
+
+/**
+ * Inform the cryptodev queue pair to start processing or finish dequeuing all
+ * enqueued/dequeued descriptors.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	n		The total number of processed descriptors.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_sym_operation_done_t)(void *qp, uint8_t *drv_ctx,
+	uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from the user
+ * data stored in the first processed descriptor.
+ *
+ * @param	user_data	Dequeued user data.
+ **/
+typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void
 *user_data);
+
+/**
+ * Typedef that the user provided to deal with post dequeue operation, such
+ * as filling status.
+ *
+ * @param	user_data	Dequeued user data.
+ * @param	index		Index number of the processed descriptor.
+ * @param	is_op_success	Operation status provided by the driver.
+ **/
+typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
+	uint32_t index, uint8_t is_op_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	drv_ctx			Driver specific context data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_user_data		User data pointer array to be retrieve
+ *					from device queue. In case of
+ *					*is_user_data_array* is set there
+ *					should be enough room to store all
+ *					user data.
+ * @param	is_user_data_array	Set 1 if every dequeued user data will
+ *					be written into out_user_data* array.
+ * @param	n_success		Driver written value to specific the
+ *					total successful operations count.
+ *
+ * @return
+ *  - Returns number of dequeued packets.
+ */
+typedef uint32_t (*cryptodev_dp_sym_dequeue_burst_t)(void *qp,
+	uint8_t *drv_ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	drv_ctx			Driver specific context data.
+ * @param	out_user_data		User data pointer to be retrieve from
+ *					device queue.
+ *
+ * @return
+ *   - 1 if the user_data is dequeued and the operation is a success.
+ *   - 0 if the user_data is dequeued but the operation is failed.
+ *   - -1 if no operation is dequeued.
+ */
+typedef int (*cryptodev_dp_sym_dequeue_t)(
+		void *qp, uint8_t *drv_ctx, void **out_user_data);
+
+/**
+ * Context data for raw data-path API crypto process. The buffer of this
+ * structure is to be allocated by the user application with the size equal
+ * or bigger than rte_cryptodev_raw_get_dp_context_size() returned value.
+ *
+ * NOTE: the buffer is to be used and maintained by the cryptodev driver, the
+ * user should NOT alter the buffer content to avoid application or system
+ * crash.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t · line 1520

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ */
+struct rte_crypto_raw_dp_ctx {
+	void *qp_data;
+
+	cryptodev_dp_sym_enqueue_t enqueue;
+	cryptodev_dp_sym_enqueue_burst_t enqueue_burst;
+	cryptodev_dp_sym_operation_done_t enqueue_done;
+	cryptodev_dp_sym_dequeue_t dequeue;
+	cryptodev_dp_sym_dequeue_burst_t dequeue_burst;
+	cryptodev_dp_sym_operation_done_t dequeue_done;

These function pointers are data path only. Why do we need to add explicit dp in each one of them These should be cryptodev_sym_raw_**

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1530

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.h +12 -0
+
+	/* Driver specific context data */
+	__extension__ uint8_t drv_ctx_data[];
+};
+
+/**
+ * Configure raw data-path context data.
+ *
+ * NOTE:
+ * After the context data is configured, the user should call
+ * rte_cryptodev_raw_attach_session() before using it in
+ * rte_cryptodev_raw_enqueue/dequeue function call.

I am not clear of the purpose of attach API? It looks an overhead to me.

Quoted 119 lines
lib/librte_cryptodev/rte_cryptodev.h +109 -0
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must be
+ *				in the range [0, nb_queue_pair - 1] previously
+ *				supplied to rte_cryptodev_configure().
+ * @param	ctx		The raw data-path context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx);
+
+/**
+ * Attach a cryptodev session to a initialized raw data path context.
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must be
+ *				in the range [0, nb_queue_pair - 1] previously
+ *				supplied to rte_cryptodev_configure().
+ * @param	ctx		The raw data-path context data.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx);
+
+/**
+ * Enqueue single raw data-path descriptor.
+ *
+ * The enqueued descriptor will not be started processing until
+ * rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	ctx		The initialized raw data-path context data.
+ * @param	data_vec	The buffer vector.
+ * @param	n_data_vecs	Number of buffer vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	iv		IV virtual and IOVA addresses
+ * @param	digest		digest virtual and IOVA addresses
+ * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA addresses,
+ *				depends on the algorithm used.
+ * @param	user_data	The user data.
+ * @return
+ *   - The number of descriptors successfully enqueued.
+ */
+__rte_experimental
+static __rte_always_inline int
+rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_vec *data_vec, uint16_t n_data_vecs,
+	union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_va_iova_ptr *iv,
+	struct rte_crypto_va_iova_ptr *digest,
+	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+	void *user_data)
+{
+	return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec,
+		n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data);
+}
+
+/**
+ * Enqueue a data vector of raw data-path descriptors.
+ *
+ * The enqueued descriptors will not be started processing until
+ * rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	ctx		The initialized raw data-path context data.
+ * @param	vec		The array of descriptor vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	user_data	The array of opaque data for dequeue.
+ * @return
+ *   - The number of descriptors successfully enqueued.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **user_data);
+
+/**
+ * Start processing all enqueued descriptors from last
+ * rte_cryptodev_raw_configure_dp_context() call.
+ *
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The total number of submitted descriptors.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
+/**
+ * Dequeue a burst of raw crypto data-path operations and write the previously
+ * enqueued user data into the array provided.
+ *
+ * The dequeued operations, including the user data stored, will not be
+ * wiped out from the device queue until rte_cryptodev_raw_dequeue_done()

is
+ * called.
+ *
+ * @param ctx The initialized raw data-path context
+ * data.
+ * @param get_dequeue_count User provided callback function to
+ * obtain dequeue count.
+ * @param post_dequeue User provided callback function to
+ * post-process a dequeued operation.
+ * @param out_user_data User data pointer array to be retrieve

Quoted 8 lines

+ * from device queue. In case of
+ * *is_user_data_array* is set there
+ * should be enough room to store all
+ * user data.
+ * @param is_user_data_array Set 1 if every dequeued user data will
+ * be written into *out_user_data* array.
+ * @param n_success Driver written value to specific the
+ * total successful operations count.

// to specify the

Quoted 31 lines

+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */

lib/librte_cryptodev/rte_cryptodev.h +14 -0
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success);
+
+/**
+ * Dequeue a raw crypto data-path operation and write the previously
+ * enqueued user data.
+ *
+ * The dequeued operation, including the user data stored, will not be wiped
+ * out from the device queue until rte_cryptodev_raw_dequeue_done() is

called.
+ *
+ * @param ctx The initialized raw data-path context
+ * data.
+ * @param out_user_data User data pointer to be retrieve from
+ * device queue. The driver shall support
+ * NULL input of this parameter.
+ *
+ * @return

+ *   - 1 if the user data is dequeued and the operation is a success.
+ *   - 0 if the user data is dequeued but the operation is failed.
+ *   - -1 if no operation is ready to be dequeued.

+ */

lib/librte_cryptodev/rte_cryptodev.h

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.h +2 -0
+__rte_experimental
+static __rte_always_inline int

Why is this function specifically inline and not others?

Quoted 55 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx,
+	void **out_user_data)
+{
+	return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data,

out_user_data);

lib/librte_cryptodev/rte_cryptodev.h +13 -0
+}
+
+/**
+ * Inform the queue pair dequeue operations finished.
+ *
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The total number of jobs already dequeued.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +24 -0
index 81975d72b..69a2a6d64 100644
@@ -316,6 +316,40 @@ typedef uint32_t
 (*cryptodev_sym_cpu_crypto_process_t)
 	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
 	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
 
+/**
+ * Typedef that the driver provided to get service context private date size.
+ *
+ * @param	dev	Crypto device pointer.
+ *
+ * @return
+ *   - On success return the size of the device's service context private data.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_get_service_ctx_size_t)(
+	struct rte_cryptodev *dev);
+
+/**
+ * Typedef that the driver provided to configure data-path context.
+ *
+ * @param	dev		Crypto device pointer.
+ * @param	qp_id		Crypto device queue pair index.
+ * @param	service_type	Type of the service requested.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data. If NULL the driver
+ *				shall only configure the drv_ctx_data in
+ *				ctx buffer. Otherwise the driver shall only
+ *				parse the session_ctx to set appropriate
+ *				function pointers in ctx.

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef uint32_t · line 353

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +10 -0
+ * @param	ctx		The raw data-path context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_configure_ctx_t)(
+	struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_raw_dp_ctx *ctx);

These typedefs names are not matching with the corresponding API names. Can you fix it for all of them?

Quoted 46 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +11 -2
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
@@ -348,8 +382,17 @@ struct rte_cryptodev_ops {
 	/**< Clear a Crypto sessions private data. */
 	cryptodev_asym_free_session_t asym_session_clear;
 	/**< Clear a Crypto sessions private data. */
-	cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
-	/**< process input data synchronously (cpu-crypto). */
+	union {
+		cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
+		/**< process input data synchronously (cpu-crypto). */
+		__extension__
+		struct {
+			cryptodev_dp_get_service_ctx_size_t get_drv_ctx_size;
+			/**< Get data path service context data size. */
+			cryptodev_dp_configure_ctx_t configure_dp_ctx;
+			/**< Initialize crypto service ctx data. */
+		};
+	};
 };
 
 
lib/librte_cryptodev/rte_cryptodev_version.map +11 -1
index 02f6dcf72..bc4cd1ea5 100644
@@ -105,4 +105,15 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_raw_attach_session;
+	rte_cryptodev_raw_configure_dp_context;
+	rte_cryptodev_raw_get_dp_context_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
--

2.20.1