Skip to contentSkip to conversation

cryptodev: add data-path service APIs

PATCH Superseded Fan Zhang 5 participants 84 messages Aug 18, 2020 – Oct 12, 2020 Page 2 of 2

Subsystems (2)
lore
Share & tools
lore
#51 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

In reply to Akhil Goyal

Hi Akhil,

Thanks a lot for the review. Comments inline.

Regards,
Fan

Quoted 157 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, October 8, 2020 3:26 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: RE: [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Hi Fan,

Quoted 27 lines

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

---
 doc/guides/prog_guide/cryptodev_lib.rst       |  93 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_crypto_sym.h         |   2 +-
 lib/librte_cryptodev/rte_cryptodev.c          | 104 +++++
 lib/librte_cryptodev/rte_cryptodev.h          | 354 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 ++-
 .../rte_cryptodev_version.map                 |  11 +
 7 files changed, 614 insertions(+), 4 deletions(-)
doc/guides/prog_guide/cryptodev_lib.rst +0 -0
index e7ba35c2d..5fe6c3c24 100644
@@ -632,6 +632,99 @@ a call argument. Status different than zero must be

treated as error.
For more details, e.g. how to convert an mbuf to an SGL, please refer to an example usage in the IPsec library implementation.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 638

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+Cryptodev Raw Data-path APIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Crypto Raw data-path APIs are a set of APIs are designed to enable

The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+externel libraries/applications which want to leverage the cryptographic
+processing provided by DPDK crypto PMDs through the cryptodev API but

in a

Quoted 1 line

+manner that is not dependent on native DPDK data structures (eg.

rte_mbuf,

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 5 lines
doc/guides/prog_guide/cryptodev_lib.rst +5 -0
+rte_crypto_op, ... etc) in their data-path implementation.
+
+The raw data-path APIs have the following advantages:
+- External data structure friendly design. The new APIs uses the operation
+  descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer

and

Quoted 1 line

+ IOVA addresses as input. Moreover, the APIs does not require the user

to

Quoted 1 line

+ allocate the descriptor from mempool, nor requiring mbufs to describe

input

Quoted 1 line

+ data's virtual and IOVA addresses. All these features made the

translation

Quoted 1 line

+ from user's own data structure into the descriptor easier and more

efficient.

Quoted 1 line

+- Flexible enqueue and dequeue operation. The raw data-path APIs gives

the

Quoted 1 line

+ user more control to the enqueue and dequeue operations, including

the

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error. · line 656

Quoted 5 lines

+ capability of precious enqueue/dequeue count, abandoning enqueue or dequeue

doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+  at any time, and operation status translation and set on the fly.
+
+Cryptodev PMDs who supports the raw data-path APIs will have

Cryptodev PMDs which support raw data-path APIs will have

Quoted 1 line

+``RTE_CRYPTODEV_FF_SYM_HW_RAW_DP`` feature flag presented. To

use this

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+feature, the user should create a local ``struct rte_crypto_raw_dp_ctx``
+buffer and extend to at least the length returned by
+``rte_cryptodev_raw_get_dp_context_size`` function call. The created

buffer

``rte_cryptodev_get_raw_dp_ctx_size``

Quoted 1 line

+is then configured using ``rte_cryptodev_raw_configure_dp_context``

function.
rte_cryptodev _configure_raw_dp_ctx

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+The library and the crypto device driver will then configure the buffer and
+write necessary temporary data into the buffer for later enqueue and

dequeue

Quoted 1 line

+operations. The temporary data may be treated as the shadow copy of

the

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+driver's private queue pair data.
+
+After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is then
+attached either the cryptodev sym session, the rte_security session, or

the

attached either with the cryptodev sym session, the rte_security session, or the

Quoted 2 lines

+cryptodev xform for session-less operation by +``rte_cryptodev_raw_attach_session`` function. With the session or

xform

``rte_cryptodev_attach_raw_session`` API

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 5 lines

+information the driver will set the corresponding enqueue and dequeue function

doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+handlers to the ``struct rte_crypto_raw_dp_ctx`` buffer.
+
+After the session is attached, the ``struct rte_crypto_raw_dp_ctx`` buffer

is

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 4 lines

+now ready for enqueue and dequeue operation. There are two different enqueue

doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+functions: ``rte_cryptodev_raw_enqueue`` to enqueue single descriptor,
+and ``rte_cryptodev_raw_enqueue_burst`` to enqueue multiple

descriptors.

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+In case of the application uses similar approach to
+``struct rte_crypto_sym_vec`` to manage its data burst but with different
+data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function

may

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 3 lines

be

doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+less efficient as this is a situation where the application has to loop over
+all crypto descriptors to assemble the ``struct rte_crypto_sym_vec``

buffer

Quoted 1 line

+from its own data structure, and then the driver will loop over them again

to

Quoted 2 lines

+translate every crypto job to the driver's specific queue data. The +``rte_cryptodev_raw_enqueue`` should be used to save one loop for

each data

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+burst instead.
+
+During the enqueue, the cryptodev driver only sets the enqueued

descriptors

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+into the device queue but not initiates the device to start processing them.
+The temporary queue pair data changes in relation to the enqueued

descriptors

Quoted 3 lines

+may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the reference
+to the next enqueue function call. When

``rte_cryptodev_raw_enqueue_done``

Quoted 2 lines

is
+called, the driver will initiate the processing of all enqueued descriptors

and

Quoted 1 line

+merge the temporary queue pair data changes into the driver's private

queue

Quoted 1 line

+pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice

without

Quoted 3 lines

+``rte_cryptodev_dp_enqueue_done`` call in between will invalidate the temporary
+data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is

Quoted 8 lines

useful

Quoted 1 line

+when the user wants to abandon partially enqueued data for a failed

enqueue

Quoted 1 line

+burst operation and try enqueuing in a whole later.

This feature may not be supported by all the HW PMDs, Can there be a way to bypass
this done API?

We can add another feature flag "RTE_CRYPTODEV_FF_SYM_HW_RAW_DP_ALLOW_CACHE". The PMDs who do not support this feature can simply return "- ENOTSUP" when calling enqueue_done and dequeue_done function. What do you think?

Quoted 67 lines

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 5 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+
+Similar as enqueue, there are two dequeue functions:
+``rte_cryptodev_raw_dequeue`` for dequeing single descriptor, and
+``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of

descriptor. The

Quoted 1 line

+dequeue functions only writes back the user data that was passed to the

driver

Quoted 1 line

+during inqueue, and inform the application the operation status.

during enqueue, and inform the operation status to the application.

doc/guides/prog_guide/cryptodev_lib.rst

Quoted 2 lines
doc/guides/prog_guide/cryptodev_lib.rst +2 -0
+Different than ``rte_cryptodev_dequeue_burst`` which the user can only
+set an expected dequeue count and needs to read from dequeued

cryptodev

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 3 lines
doc/guides/prog_guide/cryptodev_lib.rst +3 -0
+operations' status field, the raw data-path dequeue burst function allows
+the user to provide callback functions to retrieve dequeue
+count from the enqueued user data, and write the expected status value

to the

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 4 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+user data on the fly.
+
+Same as enqueue, both ``rte_cryptodev_raw_dequeue`` and
+``rte_cryptodev_raw_dequeue_burst`` will not wipe the dequeued

descriptors

Quoted 1 line

+from cryptodev queue unless ``rte_cryptodev_dp_dequeue_done`` is

called.

Quoted 2 lines

The
+dequeue related temporary queue data will be merged into the driver's

private

doc/guides/prog_guide/cryptodev_lib.rst a call argument. Status different than zero must be treated as error.

Quoted 7 lines
doc/guides/prog_guide/cryptodev_lib.rst +7 -0
+queue data in the function call.
+
+There are a few limitations to the data path service:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the direct API's enqueue with

rte_cryptodev_enqueue_burst, or

Quoted 13 lines
doc/guides/prog_guide/cryptodev_lib.rst +4 -0
+  vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
doc/guides/rel_notes/release_20_11.rst +0 -0
index 20ebaef5b..d3d9f82f7 100644

doc/guides/rel_notes/release_20_11.rst New Features · line 60

Quoted 6 lines
doc/guides/rel_notes/release_20_11.rst +3 -0
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+   * **Added raw data-path APIs for cryptodev library.**
+
+     Cryptodev is added raw data-path APIs to accelerate external libraries

or

doc/guides/rel_notes/release_20_11.rst New Features · line 63

Quoted 3 lines
doc/guides/rel_notes/release_20_11.rst +3 -0
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.

Raw crypto data path APIs are added to accelerate external libraries or Applications which need to perform crypto processing on raw buffers and Not dependent on rte_mbufs or rte_crypto_op mempools.

Quoted 2 lines
doc/guides/rel_notes/release_20_11.rst +1 -0
+
 
Quoted 17 lines
Quoted 8 lines
doc/guides/rel_notes/release_20_11.rst +0 -0
 Removed Items
 -------------
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
index 8201189e0..e1f23d303 100644

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_sgl { · line 61

Quoted 6 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -1
@@ -57,7 +57,7 @@ struct rte_crypto_sgl {
  */
 struct rte_crypto_va_iova_ptr {
 	void *va;
-	rte_iova_t *iova;
+	rte_iova_t iova;
 };

This should be part of 1/4 of this patchset.

Sorry missed that, will change.

Quoted 13 lines

lib/librte_cryptodev/rte_cryptodev.c

Quoted 8 lines
lib/librte_cryptodev/rte_crypto_sym.h +0 -0
 /**
lib/librte_cryptodev/rte_cryptodev.c +0 -0
index 1dd795bcb..daeb5f504 100644
@@ -1914,6 +1914,110 @@

rte_cryptodev_sym_cpu_crypto_process(uint8_t

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.c +0 -0
 dev_id,
 	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);
 }
 
Quoted 5 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1918

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.c +2 -0
+int
+rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id)

As suggested above raw_dp should be used as keyword in all the APIs Hence it should be rte_cryptodev_get_raw_dp_ctx_size

Will change

Quoted 72 lines
Quoted 16 lines
lib/librte_cryptodev/rte_cryptodev.c +14 -0
+{
+	struct rte_cryptodev *dev;
+	int32_t size = sizeof(struct rte_crypto_raw_dp_ctx);
+	int32_t priv_size;
+
+	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+	if (*dev->dev_ops->get_drv_ctx_size == NULL ||
+		!(dev->feature_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)) {
+		return -ENOTSUP;
+	}

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+
+	priv_size = (*dev->dev_ops->get_drv_ctx_size)(dev);
+	if (priv_size < 0)
+		return -ENOTSUP;
+
+	return RTE_ALIGN_CEIL((size + priv_size), 8);
+}
+
+int
+rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t

qp_id,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1943

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.c +1 -0
+	struct rte_crypto_raw_dp_ctx *ctx)

rte_cryptodev_configure_raw_dp_ctx

Quoted 23 lines
lib/librte_cryptodev/rte_cryptodev.c +23 -0
+{
+	struct rte_cryptodev *dev;
+	union rte_cryptodev_session_ctx sess_ctx = {NULL};
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id,
+			RTE_CRYPTO_OP_WITH_SESSION, sess_ctx, ctx);
+}
+
+int
+rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx)
+{
+	struct rte_cryptodev *dev;

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1976

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, sess_type,
+			session_ctx, ctx);

What is the difference between rte_cryptodev_raw_configure_dp_context and
rte_cryptodev_raw_attach_session?
And if at all it is needed, then it should be rte_cryptodev_attach_raw_dp_session.
IMO attach is not needed, I am not clear about it.

Quoted 3 lines

You are calling the same dev_ops for both - one with explicit session time and other
From an argument.

rte_cryptodev_raw_configure_dp_context creates a shadow copy of the queue pair data with in ctx, where rte_cryptodev_raw_attach_session sets the function handler based on the session data. Using of the same PMD callback function is to save one function pointer stored in the dev_ops. If you don't like it I can create 2 callback functions no problem.

Quoted 13 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1984

Quoted 9 lines
lib/librte_cryptodev/rte_cryptodev.c +8 -0
+}
+
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **user_data)
+{
+	if (vec->num == 1) {

Why do we need this check? I think user is aware that for enqueuing 1 vector, He should use the other API. Driver will be doing the enqueue operation only one time.

Will remove that.

Quoted 14 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1993

Quoted 11 lines
lib/librte_cryptodev/rte_cryptodev.c +9 -0
+		vec->status[0] = rte_cryptodev_raw_enqueue(ctx, vec->sgl-
 >vec,
+			vec->sgl->num, ofs, vec->iv, vec->digest, vec->aad,
+			user_data[0]);
+		return (vec->status[0] == 0) ? 1 : 0;
+	}
+
+	return (*ctx->enqueue_burst)(ctx->qp_data, ctx->drv_ctx_data, vec,
+			ofs, user_data);
+}

Where are rte_cryptodev_raw_enqueue and
rte_cryptodev_raw_dequeue ??

Defined as inline function in the header file.

Quoted 49 lines
Quoted 38 lines
lib/librte_cryptodev/rte_cryptodev.c +27 -0
+
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success_jobs)
+{
+	return (*ctx->dequeue_burst)(ctx->qp_data, ctx->drv_ctx_data,
+		get_dequeue_count, post_dequeue, out_user_data,
+		is_user_data_array, n_success_jobs);
+}
+

/** Initialise rte_crypto_op mempool element */
static void
rte_crypto_op_init(struct rte_mempool *mempool,

lib/librte_cryptodev/rte_cryptodev.h +0 -0
index 7b3ebc20f..3579ab66e 100644

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_asym_get_xform_enum(enum · line 472

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +1 -1
@@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum
 rte_crypto_asym_xform_type *xform_enum,
 /**< Support symmetric session-less operations */
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL
 << 23)
 /**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_SYM_HW_RAW_DP			(1ULL
 << 24)

RTE_CRYPTODEV_FF_SYM_RAW_DP should be better

Quoted 1 line

Add this in doc/guides/cryptodevs/features/default.ini as well in this patch.

Will change.

Quoted 172 lines

lib/librte_cryptodev/rte_cryptodev.h

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.h +1 -0
+/**< Support accelerated specific raw data-path APIs */

/**
* Get the name of a crypto device feature flag

lib/librte_cryptodev/rte_cryptodev.h +0 -0
@@ -1351,6 +1352,357 @@

rte_cryptodev_sym_cpu_crypto_process(uint8_t

lib/librte_cryptodev/rte_cryptodev.h line 1353

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 dev_id,
 	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs

ofs,

Quoted 45 lines
lib/librte_cryptodev/rte_cryptodev.h +43 -0
 	struct rte_crypto_sym_vec *vec);
 
+/**
+ * Get the size of the raw data-path context buffer.
+ *
+ * @param	dev_id		The device identifier.
+ *
+ * @return
+ *   - If the device supports raw data-path APIs, return the context size.
+ *   - If the device does not support the APIs, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less xform
+ * pointer.
+ */
+union rte_cryptodev_session_ctx {
+	struct rte_cryptodev_sym_session *crypto_sess;
+	struct rte_crypto_sym_xform *xform;
+	struct rte_security_session *sec_sess;
+};
+
+/**
+ * Enqueue a data vector into device queue but the driver will not start
+ * processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	vec		The array of descriptor vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	user_data	The array of user data for dequeue later.
+ * @return
+ *   - The number of descriptors successfully submitted.
+ */
+typedef uint32_t (*cryptodev_dp_sym_enqueue_burst_t)(
+	void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec,
+	union rte_crypto_sym_ofs ofs, void *user_data[]);
+
+/**
+ * Enqueue single descriptor into device queue but the driver will not start
+ * processing until rte_cryptodev_raw_enqueue_done() is called.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	data_vec	The buffer data vector.
+ * @param	n_data_vecs	Number of buffer data vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	iv		IV virtual and IOVA addresses
+ * @param	digest		digest virtual and IOVA addresses
+ * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA

addresses,

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.h +6 -0
+ *				depends on the algorithm used.
+ * @param	user_data	The user data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+typedef int (*cryptodev_dp_sym_enqueue_t)(
+	void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec,
+	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_va_iova_ptr *iv,
+	struct rte_crypto_va_iova_ptr *digest,
+	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+	void *user_data);
+
+/**
+ * Inform the cryptodev queue pair to start processing or finish dequeuing

all

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ * enqueued/dequeued descriptors.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	n		The total number of processed descriptors.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_sym_operation_done_t)(void *qp, uint8_t

*drv_ctx,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+	uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from

the user

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 9 lines
lib/librte_cryptodev/rte_cryptodev.h +8 -0
+ * data stored in the first processed descriptor.
+ *
+ * @param	user_data	Dequeued user data.
+ **/
+typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void
 *user_data);
+
+/**
+ * Typedef that the user provided to deal with post dequeue operation,

such

Quoted 48 lines
lib/librte_cryptodev/rte_cryptodev.h +48 -0
+ * as filling status.
+ *
+ * @param	user_data	Dequeued user data.
+ * @param	index		Index number of the processed descriptor.
+ * @param	is_op_success	Operation status provided by the driver.
+ **/
+typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
+	uint32_t index, uint8_t is_op_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	drv_ctx			Driver specific context data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_user_data		User data pointer array to be retrieve
+ *					from device queue. In case of
+ *					*is_user_data_array* is set there
+ *					should be enough room to store all
+ *					user data.
+ * @param	is_user_data_array	Set 1 if every dequeued user data will
+ *					be written into out_user_data* array.
+ * @param	n_success		Driver written value to specific the
+ *					total successful operations count.
+ *
+ * @return
+ *  - Returns number of dequeued packets.
+ */
+typedef uint32_t (*cryptodev_dp_sym_dequeue_burst_t)(void *qp,
+	uint8_t *drv_ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success);
+
+/**
+ * Dequeue symmetric crypto processing of user provided data.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	drv_ctx			Driver specific context data.
+ * @param	out_user_data		User data pointer to be retrieve from
+ *					device queue.
+ *
+ * @return
+ *   - 1 if the user_data is dequeued and the operation is a success.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   - 0 if the user_data is dequeued but the operation is failed.
+ *   - -1 if no operation is dequeued.
+ */
+typedef int (*cryptodev_dp_sym_dequeue_t)(
+		void *qp, uint8_t *drv_ctx, void **out_user_data);
+
+/**
+ * Context data for raw data-path API crypto process. The buffer of this
+ * structure is to be allocated by the user application with the size equal
+ * or bigger than rte_cryptodev_raw_get_dp_context_size() returned

value.

lib/librte_cryptodev/rte_cryptodev.h line 1508

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev.h +2 -0
+ *
+ * NOTE: the buffer is to be used and maintained by the cryptodev driver,

the

Quoted 17 lines

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1518

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.h +12 -0
+ * user should NOT alter the buffer content to avoid application or system
+ * crash.
+ */
+struct rte_crypto_raw_dp_ctx {
+	void *qp_data;
+
+	cryptodev_dp_sym_enqueue_t enqueue;
+	cryptodev_dp_sym_enqueue_burst_t enqueue_burst;
+	cryptodev_dp_sym_operation_done_t enqueue_done;
+	cryptodev_dp_sym_dequeue_t dequeue;
+	cryptodev_dp_sym_dequeue_burst_t dequeue_burst;
+	cryptodev_dp_sym_operation_done_t dequeue_done;

These function pointers are data path only. Why do we need to add explicit dp in each one of them
These should be cryptodev_sym_raw_**

Good idea.

Quoted 181 lines

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1530

Quoted 12 lines
lib/librte_cryptodev/rte_cryptodev.h +12 -0
+
+	/* Driver specific context data */
+	__extension__ uint8_t drv_ctx_data[];
+};
+
+/**
+ * Configure raw data-path context data.
+ *
+ * NOTE:
+ * After the context data is configured, the user should call
+ * rte_cryptodev_raw_attach_session() before using it in
+ * rte_cryptodev_raw_enqueue/dequeue function call.

I am not clear of the purpose of attach API? It looks an overhead to me.

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must

be

lib/librte_cryptodev/rte_cryptodev.h line 1537

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.h +1 -0
+ *				in the range [0, nb_queue_pair - 1]

previously

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 9 lines
lib/librte_cryptodev/rte_cryptodev.h +9 -0
+ *				supplied to rte_cryptodev_configure().
+ * @param	ctx		The raw data-path context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t

qp_id,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +8 -0
+	struct rte_crypto_raw_dp_ctx *ctx);
+
+/**
+ * Attach a cryptodev session to a initialized raw data path context.
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must

be

lib/librte_cryptodev/rte_cryptodev.h line 1555

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.h +1 -0
+ *				in the range [0, nb_queue_pair - 1]

previously

Quoted 19 lines
lib/librte_cryptodev/rte_cryptodev.h +19 -0
+ *				supplied to rte_cryptodev_configure().
+ * @param	ctx		The raw data-path context data.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx);
+
+/**
+ * Enqueue single raw data-path descriptor.
+ *
+ * The enqueued descriptor will not be started processing until

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ * rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	ctx		The initialized raw data-path context data.
+ * @param	data_vec	The buffer vector.
+ * @param	n_data_vecs	Number of buffer vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	iv		IV virtual and IOVA addresses
+ * @param	digest		digest virtual and IOVA addresses
+ * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA

addresses,

Quoted 43 lines
lib/librte_cryptodev/rte_cryptodev.h +43 -0
+ *				depends on the algorithm used.
+ * @param	user_data	The user data.
+ * @return
+ *   - The number of descriptors successfully enqueued.
+ */
+__rte_experimental
+static __rte_always_inline int
+rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_vec *data_vec, uint16_t n_data_vecs,
+	union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_va_iova_ptr *iv,
+	struct rte_crypto_va_iova_ptr *digest,
+	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+	void *user_data)
+{
+	return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec,
+		n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data);
+}
+
+/**
+ * Enqueue a data vector of raw data-path descriptors.
+ *
+ * The enqueued descriptors will not be started processing until
+ * rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	ctx		The initialized raw data-path context data.
+ * @param	vec		The array of descriptor vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	user_data	The array of opaque data for dequeue.
+ * @return
+ *   - The number of descriptors successfully enqueued.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **user_data);
+
+/**
+ * Start processing all enqueued descriptors from last
+ * rte_cryptodev_raw_configure_dp_context() call.
+ *

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The total number of submitted descriptors.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
+/**
+ * Dequeue a burst of raw crypto data-path operations and write the

previously

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * enqueued user data into the array provided.
+ *
+ * The dequeued operations, including the user data stored, will not be
+ * wiped out from the device queue until

rte_cryptodev_raw_dequeue_done()

Quoted 8 lines

is
+ * called.
+ *
+ * @param ctx The initialized raw data-path context
+ * data.
+ * @param get_dequeue_count User provided callback function to
+ * obtain dequeue count.
+ * @param post_dequeue User provided callback function to

Quoted 8 lines

+ * post-process a dequeued operation.
+ * @param out_user_data User data pointer array to be retrieve
+ * from device queue. In case of
+ * *is_user_data_array* is set there
+ * should be enough room to store all
+ * user data.
+ * @param is_user_data_array Set 1 if every dequeued user data will
+ * be written into *out_user_data*

array.

Quoted 2 lines

+ * @param n_success Driver written value to specific the
+ * total successful operations count.

// to specify the

Quoted 7 lines

+ *
+ * @return
+ * - Returns number of dequeued packets.
+ */

lib/librte_cryptodev/rte_cryptodev.h +3 -0
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success);
+
+/**
+ * Dequeue a raw crypto data-path operation and write the previously
+ * enqueued user data.
+ *
+ * The dequeued operation, including the user data stored, will not be

wiped

Quoted 1 line

+ * out from the device queue until rte_cryptodev_raw_dequeue_done()

is

Quoted 6 lines

called.
+ *
+ * @param ctx The initialized raw data-path context
+ * data.
+ * @param out_user_data User data pointer to be retrieve from
+ * device queue. The driver shall

support

Quoted 11 lines

lib/librte_cryptodev/rte_cryptodev.h

Quoted 9 lines

+ * NULL input of this parameter.
+ *
+ * @return

+ *   - 1 if the user data is dequeued and the operation is a success.
+ *   - 0 if the user data is dequeued but the operation is failed.
+ *   - -1 if no operation is ready to be dequeued.

+ */

lib/librte_cryptodev/rte_cryptodev.h +2 -0
+__rte_experimental
+static __rte_always_inline int

Why is this function specifically inline and not others?

Single op enqueue and dequeue helps ease the double looping performance degradation explained in the prog_guide. But they may have to be inline as the functions are called once for each packet. For bulk enqueue/dequeue and enqueue/dequeue_done, as they are called once per burst,
I don't think we need to inline them.

Quoted 62 lines
Quoted 28 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx,
+	void **out_user_data)
+{
+	return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data,

out_user_data);

lib/librte_cryptodev/rte_cryptodev.h +13 -0
+}
+
+/**
+ * Inform the queue pair dequeue operations finished.
+ *
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The total number of jobs already dequeued.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index 81975d72b..69a2a6d64 100644

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef uint32_t · line 321

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +2 -0
@@ -316,6 +316,40 @@ typedef uint32_t
 (*cryptodev_sym_cpu_crypto_process_t)
 	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
 	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
 
+/**
+ * Typedef that the driver provided to get service context private date

size.

Quoted 27 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +27 -0
+ *
+ * @param	dev	Crypto device pointer.
+ *
+ * @return
+ *   - On success return the size of the device's service context private data.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_dp_get_service_ctx_size_t)(
+	struct rte_cryptodev *dev);
+
+/**
+ * Typedef that the driver provided to configure data-path context.
+ *
+ * @param	dev		Crypto device pointer.
+ * @param	qp_id		Crypto device queue pair index.
+ * @param	service_type	Type of the service requested.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data. If NULL the driver
+ *				shall only configure the drv_ctx_data in
+ *				ctx buffer. Otherwise the driver shall only
+ *				parse the session_ctx to set appropriate
+ *				function pointers in ctx.
+ * @param	ctx		The raw data-path context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef uint32_t (*cryptodev_sym_cpu_crypto_process_t) · line 352

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +5 -0
+typedef int (*cryptodev_dp_configure_ctx_t)(
+	struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_raw_dp_ctx *ctx);

These typedefs names are not matching with the corresponding API names. Can you fix it for all of them?

Will do.

Quoted 48 lines
Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
@@ -348,8 +382,17 @@ struct rte_cryptodev_ops {
 	/**< Clear a Crypto sessions private data. */

lib/librte_cryptodev/rte_cryptodev_pmd.h struct rte_cryptodev_ops { · line 390

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +6 -2
 	cryptodev_asym_free_session_t asym_session_clear;
 	/**< Clear a Crypto sessions private data. */
-	cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
-	/**< process input data synchronously (cpu-crypto). */
+	union {
+		cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
+		/**< process input data synchronously (cpu-crypto). */
+		__extension__
+		struct {
+			cryptodev_dp_get_service_ctx_size_t

get_drv_ctx_size;

Quoted 31 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +5 -0
+			/**< Get data path service context data size. */
+			cryptodev_dp_configure_ctx_t configure_dp_ctx;
+			/**< Initialize crypto service ctx data. */
+		};
+	};
 };
 
 
lib/librte_cryptodev/rte_cryptodev_version.map +11 -1
index 02f6dcf72..bc4cd1ea5 100644
@@ -105,4 +105,15 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_raw_attach_session;
+	rte_cryptodev_raw_configure_dp_context;
+	rte_cryptodev_raw_get_dp_context_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
--

2.20.1

#52 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Quoted 24 lines
Quoted 24 lines
Quoted 1 line

+During the enqueue, the cryptodev driver only sets the enqueued

descriptors

Review

Quoted 2 lines
(quoted diff) +2 -0
+into the device queue but not initiates the device to start processing them.
+The temporary queue pair data changes in relation to the enqueued

descriptors

Quoted 3 lines

+may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the reference
+to the next enqueue function call. When

``rte_cryptodev_raw_enqueue_done``

Quoted 2 lines

is
+called, the driver will initiate the processing of all enqueued descriptors

and

Quoted 1 line

+merge the temporary queue pair data changes into the driver's private

queue

Quoted 1 line

+pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice

without

Quoted 3 lines

+``rte_cryptodev_dp_enqueue_done`` call in between will invalidate the temporary
+data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is

useful

Quoted 1 line

+when the user wants to abandon partially enqueued data for a failed

enqueue

Quoted 1 line

+burst operation and try enqueuing in a whole later.

Quoted 8 lines
Quoted 3 lines

This feature may not be supported by all the HW PMDs, Can there be a way to bypass
this done API?

We can add another feature flag "RTE_CRYPTODEV_FF_SYM_HW_RAW_DP_ALLOW_CACHE". The PMDs who do not support this feature can simply return "- ENOTSUP" when calling enqueue_done and dequeue_done function. What do you think?

Can the enqueue/dequeue API return a flag which decide whether to call done API or not?
Returning ENOTSUP will break the execution.

Quoted 52 lines
Quoted 48 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+int
+rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t

qp_id,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1943

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.c +1 -0
+	struct rte_crypto_raw_dp_ctx *ctx)

rte_cryptodev_configure_raw_dp_ctx

Quoted 23 lines
lib/librte_cryptodev/rte_cryptodev.c +23 -0
+{
+	struct rte_cryptodev *dev;
+	union rte_cryptodev_session_ctx sess_ctx = {NULL};
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id,
+			RTE_CRYPTO_OP_WITH_SESSION, sess_ctx, ctx);
+}
+
+int
+rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx)
+{
+	struct rte_cryptodev *dev;

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1976

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.c +10 -0
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, sess_type,
+			session_ctx, ctx);

What is the difference between rte_cryptodev_raw_configure_dp_context and
rte_cryptodev_raw_attach_session?
And if at all it is needed, then it should be rte_cryptodev_attach_raw_dp_session.
IMO attach is not needed, I am not clear about it.

Quoted 3 lines

You are calling the same dev_ops for both - one with explicit session time and other
From an argument.

Quoted 6 lines

rte_cryptodev_raw_configure_dp_context creates a shadow copy of the queue pair data with in ctx, where rte_cryptodev_raw_attach_session sets the function handler based on the session data. Using of the same PMD callback function is to
save one function pointer stored in the dev_ops. If you don't like it I can create 2 callback functions no problem.

I don't like the idea of having 2 APIs.

Why do you need to create a shadow copy of the queue data? Why it can't be Done in the attach API by the driver? In v9 it was doing that, why is it changed?

#53 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Hi, Good idea. Will do.

Quoted 124 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, October 8, 2020 5:08 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Bronowski, PiotrX <[email protected]>
Subject: RE: [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

+During the enqueue, the cryptodev driver only sets the enqueued

descriptors

Quoted 1 line

+into the device queue but not initiates the device to start processing

them.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

+The temporary queue pair data changes in relation to the enqueued

descriptors

Quoted 3 lines

+may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the reference
+to the next enqueue function call. When

``rte_cryptodev_raw_enqueue_done``

Quoted 2 lines

is
+called, the driver will initiate the processing of all enqueued

descriptors

Quoted 6 lines
Quoted 6 lines

and

Quoted 1 line

+merge the temporary queue pair data changes into the driver's private

queue

Quoted 1 line

+pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice

without

Quoted 1 line

+``rte_cryptodev_dp_enqueue_done`` call in between will invalidate

the

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

temporary
+data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is

useful

Quoted 1 line

+when the user wants to abandon partially enqueued data for a failed

enqueue

Quoted 1 line

+burst operation and try enqueuing in a whole later.

This feature may not be supported by all the HW PMDs, Can there be a

way

Quoted 5 lines
Quoted 2 lines

to bypass
this done API?

We can add another feature flag "RTE_CRYPTODEV_FF_SYM_HW_RAW_DP_ALLOW_CACHE". The PMDs

who

Quoted 2 lines

do not support this feature can simply return "- ENOTSUP" when calling enqueue_done and dequeue_done function. What do you think?

Can the enqueue/dequeue API return a flag which decide whether to call done API or not?
Returning ENOTSUP will break the execution.

Quoted 34 lines
Quoted 34 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+int
+rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t

qp_id,

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1943

Quoted 1 line
lib/librte_cryptodev/rte_cryptodev.c +1 -0
+	struct rte_crypto_raw_dp_ctx *ctx)

rte_cryptodev_configure_raw_dp_ctx

Quoted 27 lines
lib/librte_cryptodev/rte_cryptodev.c +27 -0
+{
+	struct rte_cryptodev *dev;
+	union rte_cryptodev_session_ctx sess_ctx = {NULL};
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id,
+			RTE_CRYPTO_OP_WITH_SESSION, sess_ctx, ctx);
+}
+
+int
+rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx)
+{
+	struct rte_cryptodev *dev;
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.c rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1976

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.c +6 -0
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)
+			|| dev->dev_ops->configure_dp_ctx == NULL)
+		return -ENOTSUP;
+	return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, sess_type,
+			session_ctx, ctx);

What is the difference between

rte_cryptodev_raw_configure_dp_context

Quoted 11 lines
Quoted 9 lines

and
rte_cryptodev_raw_attach_session?
And if at all it is needed, then it should be rte_cryptodev_attach_raw_dp_session.
IMO attach is not needed, I am not clear about it.

You are calling the same dev_ops for both - one with explicit session time and other
From an argument.

rte_cryptodev_raw_configure_dp_context creates a shadow copy of the

queue

Quoted 1 line

pair data with in ctx, where rte_cryptodev_raw_attach_session sets the

function

Quoted 1 line

handler based on the session data. Using of the same PMD callback

function is

Quoted 2 lines

to
save one function pointer stored in the dev_ops. If you don't like it I can

create

Quoted 1 line

2 callback functions no problem.

I don't like the idea of having 2 APIs.

Why do you need to create a shadow copy of the queue data? Why it can't be
Done in the attach API by the driver? In v9 it was doing that, why is it changed?

#54 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

In reply to Akhil Goyal

Hi Akhil,

Quoted 8 lines
Quoted 1 line

rte_cryptodev_raw_configure_dp_context creates a shadow copy of the

queue

Quoted 1 line

pair data with in ctx, where rte_cryptodev_raw_attach_session sets the

function

Quoted 1 line

handler based on the session data. Using of the same PMD callback

function is

Quoted 2 lines

to
save one function pointer stored in the dev_ops. If you don't like it I can

Quoted 9 lines

create

Quoted 1 line

2 callback functions no problem.

I don't like the idea of having 2 APIs.

Why do you need to create a shadow copy of the queue data? Why it can't be
Done in the attach API by the driver? In v9 it was doing that, why is it changed?

The reason for creating shadow copy is for enqueue_done and dequeue_done. As explained if external application uses a data structure similar to rte_crypto_sym_vec and expect all ops or no ops are enqueued/dequeued, it is impossible to do so with rte_cryptodev_enqueue/dequeue_burst. The local queue pair shadow copy helps temporary caching what is already pushed into the HW queue but the driver has yet to issue "start processing command" to the device. Once the application finds out not all ops can be enqueued or dequeued the temp shadow copy can be reset by issuing rte_cryptodev_raw_configure_dp_context again.

In v9 rte_cryptodev_raw_configure_dp_context has another job - to write the function pointers to ctx. So if we are to use the same ctx for AES-CBC and AES-GCM but we don't want to erase the shadow copy data again we need the "is_update" flag to let the driver know not to erase the queue pair shadow data but updating the function pointers only. As you suggested in v9 "is_update" is not needed - to avoid using "is_update" I used 2 APIs instead, one for initializing queue pair shadow copy, one for writing function pointers by parsing the session.

Regards,
Fan

#55 lore raw

Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs

In reply to Fan Zhang

lib/librte_cryptodev/rte_cryptodev.h rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, · line 1633

Quoted 7 lines
lib/librte_cryptodev/rte_cryptodev.h +7 -0
+/**
+ * Start processing all enqueued descriptors from last
+ * rte_cryptodev_raw_configure_dp_context() call.
+ *
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The total number of submitted descriptors.
+

What does this API return?
Check for other comments as well in other APIs

*/

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+__rte_experimental
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
author
#56 lore raw

PATCH v10 3/4 crypto/qat: add raw crypto data-path API support

In reply to Fan Zhang

This patch updates QAT PMD to add raw data-path API support.

---
 drivers/crypto/qat/meson.build     |   1 +
 drivers/crypto/qat/qat_sym.h       |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c | 951 +++++++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c   |   9 +-
 4 files changed, 970 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +11 -0
index 1a9748849..b748c7705 100644
@@ -264,6 +264,16 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_dp_configure_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	struct rte_crypto_raw_dp_ctx *raw_dp_ctx);
+
+int
+qat_sym_get_service_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +286,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +951 -0

951 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..0744e6306 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Raw data-path API related operations */
+		.get_drv_ctx_size = qat_sym_get_service_ctx_size,
+		.configure_dp_ctx = qat_sym_dp_configure_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_SYM_HW_RAW_DP;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
Acked-by on v10 3/4
#57 lore raw

Re: [dpdk-dev] [dpdk-dev v10 3/4] crypto/qat: add raw crypto data-path API support

Quoted 12 lines

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: Thursday, 24 September, 2020 18:34
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>
Subject: [dpdk-dev v10 3/4] crypto/qat: add raw crypto data-path API support

Quoted 3 lines

This patch updates QAT PMD to add raw data-path API support.

author
#58 lore raw

PATCH v10 4/4 test/crypto: add unit-test for cryptodev raw API test

In reply to Fan Zhang

This patch adds the cryptodev raw API test support to unit test. In addtion a new test-case for QAT PMD for the test type is enabled.

---
 app/test/test_cryptodev.c             | 759 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |  12 +
 app/test/test_cryptodev_blockcipher.c |  58 +-
 3 files changed, 775 insertions(+), 54 deletions(-)
app/test/test_cryptodev.c +721 -38
index 99f1eed82..4e7dd1b63 100644
@@ -49,6 +49,10 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
+#define MAX_RAW_DEQUEUE_COUNT	65535
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +61,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+enum cryptodev_api_test_type global_api_test_type = CRYPTODEV_API_TEST;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +153,187 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_raw_hw_api_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop = op->sym;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+	union rte_crypto_sym_ofs ofs;
+	struct rte_crypto_sym_vec vec;
+	struct rte_crypto_sgl sgl;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	uint32_t count = 0;
+	struct rte_crypto_raw_dp_ctx *ctx;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int ctx_service_size;
+	int32_t status = 0;
+
+	ctx_service_size = rte_cryptodev_raw_get_dp_context_size(dev_id);
+	if (ctx_service_size < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ctx = malloc(ctx_service_size);
+	if (!ctx) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	if (rte_cryptodev_raw_configure_dp_context(dev_id, qp_id, ctx) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	sess.crypto_sess = sop->session;
+	if (rte_cryptodev_raw_attach_session(dev_id, qp_id, ctx,
+			op->sess_type, sess) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	cipher_iv.iova = 0;
+	cipher_iv.va = NULL;
+	aad_auth_iv.iova = 0;
+	aad_auth_iv.va = NULL;
+	digest.iova = 0;
+	digest.va = NULL;
+	sgl.vec = data_vec;
+	vec.num = 1;
+	vec.sgl = &sgl;
+	vec.iv = &cipher_iv;
+	vec.digest = &digest;
+	vec.aad = &aad_auth_iv;
+	vec.status = &status;
+
+	ofs.raw = 0;
+
+	if (is_cipher && is_auth) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			auth_offset = auth_offset >> 3;
+			cipher_len = cipher_len >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else if (is_cipher) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	} else if (is_auth) {
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			auth_offset = auth_offset >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else { /* aead */
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = (void *)sop->aead.aad.data;
+		aad_auth_iv.iova = sop->aead.aad.phys_addr;
+		digest.va = (void *)sop->aead.digest.data;
+		digest.iova = sop->aead.digest.phys_addr;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	sgl.num = n;
+
+	if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op)
+			< 1) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+
+	status = -1;
+	while (count++ < MAX_RAW_DEQUEUE_COUNT && status == -1) {
+		status = rte_cryptodev_raw_dequeue(ctx, (void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}
+
+	if (status != -1) {
+		if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			return;
+		}
+	}
+
+	op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op != op ||
+			status != 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+					RTE_CRYPTO_OP_STATUS_SUCCESS;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1661,6 +1848,9 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params->valid_devs[0],
@@ -1715,12 +1905,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2435,6 +2631,12 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2475,7 +2677,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2513,6 +2719,12 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2554,7 +2766,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2580,6 +2796,16 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2624,6 +2850,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2653,6 +2882,16 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2695,7 +2934,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2860,6 +3103,16 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2902,8 +3155,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2959,6 +3216,12 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -2988,7 +3251,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3031,10 +3298,14 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3116,6 +3387,9 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3201,6 +3475,9 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3269,6 +3546,16 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	uint8_t *ciphertext, *plaintext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3311,7 +3598,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3344,6 +3635,16 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3386,7 +3687,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3427,6 +3732,9 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3510,6 +3818,9 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3629,6 +3940,9 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3719,6 +4033,16 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3761,7 +4085,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3799,6 +4127,9 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3886,6 +4217,12 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EEA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_ZUC_EEA3;
@@ -3929,7 +4266,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -3969,6 +4310,16 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4024,7 +4375,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4092,6 +4447,14 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
+	}
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
 	}
 
 	/* Create SNOW 3G session */
@@ -4160,7 +4523,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4270,7 +4637,14 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4349,7 +4723,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4457,7 +4835,15 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 
 	uint64_t feat_flags = dev_info.feature_flags;
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	if (op_mode == OUT_OF_PLACE) {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4531,7 +4917,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4642,7 +5032,14 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4721,7 +5118,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4806,6 +5207,16 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4862,7 +5273,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4905,6 +5320,16 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 
@@ -4949,7 +5374,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5004,6 +5433,12 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	plaintext_len = ceil_byte_length(tdata->plaintext.len);
 
 	/* Append data which is padded to a multiple */
@@ -5036,7 +5471,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5084,6 +5523,12 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EIA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_ZUC_EIA3;
@@ -5124,7 +5569,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5181,7 +5630,15 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5256,7 +5713,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5363,7 +5824,15 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5442,7 +5911,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5585,6 +6058,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5784,6 +6260,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5798,6 +6277,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6151,11 +6633,12 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6166,6 +6649,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
 		retval = create_wireless_algo_cipher_auth_session(
@@ -6197,9 +6683,11 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}
 
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6240,18 +6728,17 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6342,11 +6829,12 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6445,20 +6933,18 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -6999,6 +7485,16 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	uint8_t *ciphertext, *auth_tag;
 	uint16_t plaintext_pad_len;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -7048,6 +7544,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8496,6 +8995,16 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	int retval;
 	uint8_t *plaintext;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -8545,6 +9054,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8839,6 +9351,9 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -8928,8 +9443,9 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and raw data-path APIs*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			global_api_test_type == CRYPTODEV_RAW_API_TEST)
 		return -ENOTSUP;
 
 	/* Create AEAD session */
@@ -9115,6 +9631,12 @@ test_authenticated_decryption_sessionless(
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -9156,8 +9678,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
 
@@ -9449,6 +9976,16 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9477,6 +10014,9 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9509,6 +10049,16 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9535,6 +10085,9 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10040,6 +10593,16 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	int retval;
 
@@ -10103,6 +10666,9 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10161,6 +10727,16 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	int retval;
 	uint32_t plaintext_pad_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	TEST_ASSERT_NOT_EQUAL(tdata->gmac_tag.len, 0,
 			      "No GMAC length in the source data");
@@ -10220,6 +10796,9 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10735,6 +11314,16 @@ test_authentication_verify_fail_when_data_corruption(
 	int retval;
 
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10744,6 +11333,7 @@ test_authentication_verify_fail_when_data_corruption(
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+
 	/* Create session */
 	retval = create_auth_session(ut_params,
 			ts_params->valid_devs[0],
@@ -10785,7 +11375,10 @@ test_authentication_verify_fail_when_data_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10803,6 +11396,16 @@ test_authentication_verify_GMAC_fail_when_corruption(
 {
 	int retval;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10856,7 +11459,10 @@ test_authentication_verify_GMAC_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10875,6 +11481,16 @@ test_authenticated_decryption_fail_when_corruption(
 	int retval;
 
 	uint8_t *ciphertext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10931,7 +11547,10 @@ test_authenticated_decryption_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10952,6 +11571,16 @@ test_authenticated_encryt_with_esn(
 	uint16_t plaintext_pad_len;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11026,6 +11655,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11072,6 +11704,16 @@ test_authenticated_decrypt_with_esn(
 	uint8_t *ciphertext;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11146,6 +11788,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -11286,10 +11931,21 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		if (sgl_in && (!(dev_info.feature_flags &
 				RTE_CRYPTODEV_FF_IN_PLACE_SGL)))
 			return -ENOTSUP;
+
+		uint64_t feat_flags = dev_info.feature_flags;
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Raw data path API does not support OOP */
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
@@ -11485,6 +12141,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13046,6 +13705,30 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static int
+test_cryptodev_qat_raw_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	global_api_test_type = CRYPTODEV_RAW_API_TEST;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	global_api_test_type = CRYPTODEV_API_TEST;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+		test_cryptodev_qat_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +12 -0
index 41542e055..d8fc0db53 100644
@@ -71,6 +71,13 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+enum cryptodev_api_test_type {
+	CRYPTODEV_API_TEST = 0,
+	CRYPTODEV_RAW_API_TEST
+};
+
+extern enum cryptodev_api_test_type global_api_test_type;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +216,9 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_raw_hw_api_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +42 -16

79 lines not on this page · raw message

Signature · 1 line
2.20.1
Acked-by on v10 4/4
#59 lore raw

Re: [dpdk-dev] [dpdk-dev v10 4/4] test/crypto: add unit-test for cryptodev raw API test

Quoted 12 lines

-----Original Message-----
From: Zhang, Roy Fan <[email protected]>
Sent: Thursday, 24 September, 2020 18:34
To: [email protected]
Cc: [email protected]; Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>
Subject: [dpdk-dev v10 4/4] test/crypto: add unit-test for cryptodev raw API test

Quoted 5 lines

This patch adds the cryptodev raw API test support to unit test. In addtion a new test-case for QAT PMD for the test type is enabled.

#60 lore raw

Re: [dpdk-dev] [dpdk-dev v10 4/4] test/crypto: add unit-test for cryptodev raw API test

In reply to Fan Zhang
Quoted 202 lines

This patch adds the cryptodev raw API test support to unit test. In addtion a new test-case for QAT PMD for the test type is enabled.

---
 app/test/test_cryptodev.c             | 759 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |  12 +
 app/test/test_cryptodev_blockcipher.c |  58 +-
 3 files changed, 775 insertions(+), 54 deletions(-)
app/test/test_cryptodev.c +91 -0
index 99f1eed82..4e7dd1b63 100644
@@ -49,6 +49,10 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
+#define MAX_RAW_DEQUEUE_COUNT	65535
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +61,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+enum cryptodev_api_test_type global_api_test_type = CRYPTODEV_API_TEST;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +153,187 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+void
+process_sym_raw_hw_api_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+	int32_t n;
+	struct rte_crypto_sym_op *sop = op->sym;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+	union rte_crypto_sym_ofs ofs;
+	struct rte_crypto_sym_vec vec;
+	struct rte_crypto_sgl sgl;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	uint32_t count = 0;
+	struct rte_crypto_raw_dp_ctx *ctx;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int ctx_service_size;
+	int32_t status = 0;
+
+	ctx_service_size = rte_cryptodev_raw_get_dp_context_size(dev_id);
+	if (ctx_service_size < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ctx = malloc(ctx_service_size);
+	if (!ctx) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	if (rte_cryptodev_raw_configure_dp_context(dev_id, qp_id, ctx) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	sess.crypto_sess = sop->session;
+	if (rte_cryptodev_raw_attach_session(dev_id, qp_id, ctx,
+			op->sess_type, sess) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	cipher_iv.iova = 0;
+	cipher_iv.va = NULL;
+	aad_auth_iv.iova = 0;
+	aad_auth_iv.va = NULL;
+	digest.iova = 0;
+	digest.va = NULL;
+	sgl.vec = data_vec;
+	vec.num = 1;
+	vec.sgl = &sgl;
+	vec.iv = &cipher_iv;
+	vec.digest = &digest;
+	vec.aad = &aad_auth_iv;
+	vec.status = &status;
+
+	ofs.raw = 0;
+
+	if (is_cipher && is_auth) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			auth_offset = auth_offset >> 3;
+			cipher_len = cipher_len >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,
 IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else if (is_cipher) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +16 -0
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	} else if (is_auth) {
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			auth_offset = auth_offset >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else { /* aead */
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +25 -0
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = (void *)sop->aead.aad.data;
+		aad_auth_iv.iova = sop->aead.aad.phys_addr;
+		digest.va = (void *)sop->aead.digest.data;
+		digest.iova = sop->aead.digest.phys_addr;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	sgl.num = n;
+
+	if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op)
+			< 1) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+	if (status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 323

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+		return;
+	}
+
+
+	status = -1;
+	while (count++ < MAX_RAW_DEQUEUE_COUNT && status == -1) {
+		status = rte_cryptodev_raw_dequeue(ctx, (void **)&ret_op);
+		if (status == -1)
+			rte_pause();
+	}

Why don't we do dequeue with the burst API?

Quoted 112 lines
app/test/test_cryptodev.c +9 -0
+
+	if (status != -1) {
+		if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			return;
+		}
+	}
+
+	op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op !=

op ||

app/test/test_cryptodev.c +4 -0
+			status != 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+					RTE_CRYPTO_OP_STATUS_SUCCESS;
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1661,6 +1848,9 @@

test_AES_CBC_HMAC_SHA512_decrypt_perform(struct
rte_cryptodev_sym_session *sess,

app/test/test_cryptodev.c +3 -0
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +12 -0
@@ -1715,12 +1905,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2435,6 +2631,12 @@ test_snow3g_authentication(const struct
 snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
app/test/test_cryptodev.c +5 -1
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2475,7 +2677,11 @@ test_snow3g_authentication(const struct
 snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +6 -0
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2513,6 +2719,12 @@ test_snow3g_authentication_verify(const struct
 snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
app/test/test_cryptodev.c +5 -1
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2554,7 +2766,11 @@ test_snow3g_authentication_verify(const struct
 snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +0 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2580,6 +2796,16 @@ test_kasumi_authentication(const struct
 kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;

app/test/test_cryptodev.c test_kasumi_authentication(const struct · line 2807

Quoted 10 lines
app/test/test_cryptodev.c +7 -0
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {

I believe global_api_test_type is set as CRYPTODEV_RAW_API_TEST after Checking the feature flag. Hence we don't need to check it again here. Right?

Or else we should check it for every case.

Quoted 24 lines
app/test/test_cryptodev.c +3 -0
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2624,6 +2850,9 @@ test_kasumi_authentication(const struct
 kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +1 -0
 			ut_params->op);
@@ -2653,6 +2882,16 @@ test_kasumi_authentication_verify(const struct
 kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;

app/test/test_cryptodev.c test_kasumi_authentication_verify(const struct · line 2896

Quoted 10 lines
app/test/test_cryptodev.c +9 -0
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}

Same comment here as well

Quoted 128 lines
app/test/test_cryptodev.c +0 -0
 	/* Verify the capabilities */
app/test/test_cryptodev.c +5 -1
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2695,7 +2934,11 @@ test_kasumi_authentication_verify(const struct
 kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +10 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2860,6 +3103,16 @@ test_kasumi_encryption(const struct
 kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +6 -2
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2902,8 +3155,12 @@ test_kasumi_encryption(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
app/test/test_cryptodev.c +6 -0
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2959,6 +3216,12 @@ test_kasumi_encryption_sgl(const struct
 kasumi_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Create KASUMI session */
app/test/test_cryptodev.c +5 -1
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -2988,7 +3251,11 @@ test_kasumi_encryption_sgl(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +4 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3031,10 +3298,14 @@ test_kasumi_encryption_oop(const struct
 kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
app/test/test_cryptodev.c +3 -0
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3116,6 +3387,9 @@ test_kasumi_encryption_oop_sgl(const struct
 kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
app/test/test_cryptodev.c +3 -0
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3201,6 +3475,9 @@ test_kasumi_decryption_oop(const struct
 kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
app/test/test_cryptodev.c +1 -0
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3269,6 +3546,16 @@ test_kasumi_decryption(const struct
 kasumi_test_data *tdata)
 	uint8_t *ciphertext, *plaintext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;

app/test/test_cryptodev.c test_kasumi_decryption(const struct · line 3560

Quoted 10 lines
app/test/test_cryptodev.c +9 -0
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}

You are checking the feature flags randomly. Please check if it is really needed as suggested above. Please be consistent.

Quoted 582 lines
app/test/test_cryptodev.c +0 -0
 	/* Verify the capabilities */
app/test/test_cryptodev.c +5 -1
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3311,7 +3598,11 @@ test_kasumi_decryption(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +10 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3344,6 +3635,16 @@ test_snow3g_encryption(const struct
 snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +5 -1
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3386,7 +3687,11 @@ test_snow3g_encryption(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +3 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -3427,6 +3732,9 @@ test_snow3g_encryption_oop(const struct
 snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
app/test/test_cryptodev.c +3 -0
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3510,6 +3818,9 @@ test_snow3g_encryption_oop_sgl(const struct
 snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
app/test/test_cryptodev.c +3 -0
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3629,6 +3940,9 @@ test_snow3g_encryption_offset_oop(const struct
 snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
app/test/test_cryptodev.c +10 -0
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3719,6 +4033,16 @@ static int test_snow3g_decryption(const struct
 snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +5 -1
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3761,7 +4085,11 @@ static int test_snow3g_decryption(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +3 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3799,6 +4127,9 @@ static int test_snow3g_decryption_oop(const struct
 snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
app/test/test_cryptodev.c +6 -0
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3886,6 +4217,12 @@ test_zuc_cipher_auth(const struct
 wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EEA3 */
app/test/test_cryptodev.c +5 -1
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_ZUC_EEA3;
@@ -3929,7 +4266,11 @@ test_zuc_cipher_auth(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +10 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -3969,6 +4310,16 @@ test_snow3g_cipher_auth(const struct
 snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +5 -1
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4024,7 +4375,11 @@ test_snow3g_cipher_auth(const struct
 snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +8 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4092,6 +4447,14 @@ test_snow3g_auth_cipher(const struct
 snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
+	}
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
 	}
app/test/test_cryptodev.c +5 -1
 	/* Create SNOW 3G session */
@@ -4160,7 +4523,11 @@ test_snow3g_auth_cipher(const struct
 snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +7 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4270,7 +4637,14 @@ test_snow3g_auth_cipher_sgl(const struct
 snow3g_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

{
printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -4349,7 +4723,11 @@ test_snow3g_auth_cipher_sgl(const struct
 snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +8 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4457,7 +4835,15 @@ test_kasumi_auth_cipher(const struct
 kasumi_test_data *tdata,
 
 	uint64_t feat_flags = dev_info.feature_flags;
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	if (op_mode == OUT_OF_PLACE) {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
app/test/test_cryptodev.c +5 -1
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4531,7 +4917,11 @@ test_kasumi_auth_cipher(const struct
 kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +7 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4642,7 +5032,14 @@ test_kasumi_auth_cipher_sgl(const struct
 kasumi_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

{
printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -4721,7 +5118,11 @@ test_kasumi_auth_cipher_sgl(const struct
 kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +10 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4806,6 +5207,16 @@ test_kasumi_cipher_auth(const struct
 kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +5 -1
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4862,7 +5273,11 @@ test_kasumi_cipher_auth(const struct
 kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +10 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4905,6 +5320,16 @@ test_zuc_encryption(const struct wireless_test_data
 *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +5 -1
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4949,7 +5374,11 @@ test_zuc_encryption(const struct wireless_test_data
 *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +6 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5004,6 +5433,12 @@ test_zuc_encryption_sgl(const struct
 wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	plaintext_len = ceil_byte_length(tdata->plaintext.len);
app/test/test_cryptodev.c +5 -1
 	/* Append data which is padded to a multiple */
@@ -5036,7 +5471,11 @@ test_zuc_encryption_sgl(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 						ut_params->op);
app/test/test_cryptodev.c +6 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5084,6 +5523,12 @@ test_zuc_authentication(const struct
 wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EIA3 */
app/test/test_cryptodev.c +5 -1
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_ZUC_EIA3;
@@ -5124,7 +5569,11 @@ test_zuc_authentication(const struct
 wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 				ut_params->op);
app/test/test_cryptodev.c +8 -0
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5181,7 +5630,15 @@ test_zuc_auth_cipher(const struct
 wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

{
printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -5256,7 +5713,11 @@ test_zuc_auth_cipher(const struct
 wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +8 -0
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5363,7 +5824,15 @@ test_zuc_auth_cipher_sgl(const struct
 wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))

{
printf("Device doesn't support out-of-place scatter-
gather "

app/test/test_cryptodev.c +5 -1
 					"in both input and output mbufs.\n");
@@ -5442,7 +5911,11 @@ test_zuc_auth_cipher_sgl(const struct
 wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
 			ut_params->op);
app/test/test_cryptodev.c +14 -1
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5585,6 +6058,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5784,6 +6260,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5798,6 +6277,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6151,11 +6633,12 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6166,6 +6649,9 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
app/test/test_cryptodev.c +0 -0
 		retval = create_wireless_algo_cipher_auth_session(

app/test/test_cryptodev.c test_mixed_auth_cipher(const struct · line 6691

Quoted 10 lines
app/test/test_cryptodev.c +3 -1
@@ -6197,9 +6683,11 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}

Unnecessary change.

Quoted 525 lines
app/test/test_cryptodev.c +6 -8
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6240,18 +6728,17 @@ test_mixed_auth_cipher(const struct
 mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -6342,11 +6829,12 @@ test_mixed_auth_cipher_sgl(const struct
 mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -6445,20 +6933,18 @@ test_mixed_auth_cipher_sgl(const struct
 mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
app/test/test_cryptodev.c +10 -0
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -6999,6 +7485,16 @@ test_authenticated_encryption(const struct
 aead_test_data *tdata)
 	uint8_t *ciphertext, *auth_tag;
 	uint16_t plaintext_pad_len;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -7048,6 +7544,9 @@ test_authenticated_encryption(const struct
 aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params-
 >op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
app/test/test_cryptodev.c +10 -0
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8496,6 +8995,16 @@ test_authenticated_decryption(const struct
 aead_test_data *tdata)
 	int retval;
 	uint8_t *plaintext;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -8545,6 +9054,9 @@ test_authenticated_decryption(const struct
 aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params-
 >op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
app/test/test_cryptodev.c +3 -0
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8839,6 +9351,9 @@ test_authenticated_encryption_oop(const struct
 aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
app/test/test_cryptodev.c +3 -2
 		return -ENOTSUP;
@@ -8928,8 +9443,9 @@ test_authenticated_decryption_oop(const struct
 aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and raw data-path APIs*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			global_api_test_type == CRYPTODEV_RAW_API_TEST)
 		return -ENOTSUP;
 
app/test/test_cryptodev.c +6 -0
 	/* Create AEAD session */
@@ -9115,6 +9631,12 @@ test_authenticated_decryption_sessionless(
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
app/test/test_cryptodev.c +17 -2
 		return -ENOTSUP;
@@ -9156,8 +9678,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params-
 >valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
@@ -9449,6 +9976,16 @@ test_MD5_HMAC_generate(const struct
 HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9477,6 +10014,9 @@ test_MD5_HMAC_generate(const struct
 HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +10 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9509,6 +10049,16 @@ test_MD5_HMAC_verify(const struct
 HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9535,6 +10085,9 @@ test_MD5_HMAC_verify(const struct
 HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +10 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10040,6 +10593,16 @@ test_AES_GMAC_authentication(const struct
 gmac_test_data *tdata)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	int retval;
@@ -10103,6 +10666,9 @@ test_AES_GMAC_authentication(const struct
 gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +10 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10161,6 +10727,16 @@ test_AES_GMAC_authentication_verify(const
 struct gmac_test_data *tdata)
 	int retval;
 	uint32_t plaintext_pad_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
app/test/test_cryptodev.c +3 -0
 	TEST_ASSERT_NOT_EQUAL(tdata->gmac_tag.len, 0,
 			      "No GMAC length in the source data");
@@ -10220,6 +10796,9 @@ test_AES_GMAC_authentication_verify(const
 struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
app/test/test_cryptodev.c +0 -0
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10735,6 +11314,16 @@

test_authentication_verify_fail_when_data_corruption(

app/test/test_cryptodev.c +7 -0
 	int retval;
 
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &

RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {

app/test/test_cryptodev.c +3 -0
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10744,6 +11333,7 @@

test_authentication_verify_fail_when_data_corruption(

app/test/test_cryptodev.c +1 -0
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+
 	/* Create session */
 	retval = create_auth_session(ut_params,
 			ts_params->valid_devs[0],
@@ -10785,7 +11375,10 @@

test_authentication_verify_fail_when_data_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +0 -0
@@ -10803,6 +11396,16 @@

test_authentication_verify_GMAC_fail_when_corruption(
{

app/test/test_cryptodev.c +7 -0
 	int retval;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &

RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {

app/test/test_cryptodev.c +3 -0
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10856,7 +11459,10 @@

test_authentication_verify_GMAC_fail_when_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +0 -0
@@ -10875,6 +11481,16 @@

test_authenticated_decryption_fail_when_corruption(

app/test/test_cryptodev.c +7 -0
 	int retval;
 
 	uint8_t *ciphertext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &

RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {

app/test/test_cryptodev.c +3 -0
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10931,7 +11547,10 @@

test_authenticated_decryption_fail_when_corruption(

app/test/test_cryptodev.c +4 -1
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params-
Quoted 1 line

valid_devs[0],

app/test/test_cryptodev.c +0 -0
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not

failed");

app/test/test_cryptodev.c +10 -0
@@ -10952,6 +11571,16 @@ test_authenticated_encryt_with_esn(
 	uint16_t plaintext_pad_len;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
app/test/test_cryptodev.c +13 -0
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11026,6 +11655,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11072,6 +11704,16 @@ test_authenticated_decrypt_with_esn(
 	uint8_t *ciphertext;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
app/test/test_cryptodev.c +3 -0
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11146,6 +11788,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params-
 >valid_devs[0],
app/test/test_cryptodev.c +11 -0
 			ut_params->op);
@@ -11286,10 +11931,21 @@ test_authenticated_encryption_SGL(const struct
 aead_test_data *tdata,
 		if (sgl_in && (!(dev_info.feature_flags &
 				RTE_CRYPTODEV_FF_IN_PLACE_SGL)))
 			return -ENOTSUP;
+
+		uint64_t feat_flags = dev_info.feature_flags;
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags &
 RTE_CRYPTODEV_FF_SYM_HW_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Raw data path API does not support OOP */
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {

if (!(dev_info.feature_flags &

RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))

app/test/test_cryptodev.c +3 -0
@@ -11485,6 +12141,9 @@ test_authenticated_encryption_SGL(const struct
 aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type ==
 RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params-
 >op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_hw_api_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
app/test/test_cryptodev.c +10 -0
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -13046,6 +13705,30 @@ test_cryptodev_nitrox(void)
 	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
 }
 
+static int
+test_cryptodev_qat_raw_api(void /*argv __rte_unused, int argc
 __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that

both "

app/test/test_cryptodev.c +1 -0
+		"CONFIG_RTE_LIBRTE_PMD_QAT and

CONFIG_RTE_LIBRTE_PMD_QAT_SYM "

app/test/test_cryptodev.c test_cryptodev_nitrox(void) · line 13725

Quoted 7 lines
app/test/test_cryptodev.c +7 -0
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	global_api_test_type = CRYPTODEV_RAW_API_TEST;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	global_api_test_type = CRYPTODEV_API_TEST;

OK I got your point of checking feature flag in each test. Please disregard my previous comment above. However, double check that None of the test is left without checking the feature flag.

Quoted 135 lines
app/test/test_cryptodev.c +6 -0
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+		test_cryptodev_qat_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest,

test_cryptodev_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,

app/test/test_cryptodev.h +12 -0
index 41542e055..d8fc0db53 100644
@@ -71,6 +71,13 @@
 #define CRYPTODEV_NAME_CAAM_JR_PMD	crypto_caam_jr
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 
+enum cryptodev_api_test_type {
+	CRYPTODEV_API_TEST = 0,
+	CRYPTODEV_RAW_API_TEST
+};
+
+extern enum cryptodev_api_test_type global_api_test_type;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -209,4 +216,9 @@ create_segmented_mbuf(struct rte_mempool
 *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_raw_hw_api_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len);
+

#endif /* TEST_CRYPTODEV_H_ */

app/test/test_cryptodev_blockcipher.c +8 -0
index 221262341..f675a2c92 100644
@@ -136,6 +136,14 @@ test_blockcipher_one_case(const struct
 blockcipher_test_case *t,
 		nb_segs = 3;
 	}
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST &&
+		!(feat_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)) {
+		printf("Device doesn't support raw data-path APIs. "
+			"Test Skipped.\n");
+		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "SKIPPED");
+		return TEST_SKIPPED;
+	}
+
 	if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_OOP) {
 		uint64_t oop_flags =

RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |

app/test/test_cryptodev_blockcipher.c +7 -0
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
@@ -148,6 +156,13 @@ test_blockcipher_one_case(const struct
 blockcipher_test_case *t,
 				"SKIPPED");
 			return TEST_SKIPPED;
 		}
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST) {
+			printf("Raw Data Path APIs do not support OOP, "
+				"Test Skipped.\n");
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
 "SKIPPED");
+			status = TEST_SUCCESS;
+			goto error_exit;
+		}
 	}
app/test/test_cryptodev_blockcipher.c +27 -16
 	if (tdata->cipher_key.len)
@@ -462,25 +477,36 @@ test_blockcipher_one_case(const struct
 blockcipher_test_case *t,
 	}
 
 	/* Process crypto operation */
-	if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) {
-		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
-			"line %u FAILED: %s",
-			__LINE__, "Error sending packet for encryption");
-		status = TEST_FAILED;
-		goto error_exit;
-	}
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST) {
+		uint8_t is_cipher = 0, is_auth = 0;
+		if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
+			is_cipher = 1;
+		if (t->op_mask & BLOCKCIPHER_TEST_OP_AUTH)
+			is_auth = 1;
 
-	op = NULL;
+		process_sym_raw_hw_api_op(dev_id, 0, op, is_cipher, is_auth,
 0,
+				tdata->iv.len);
+	} else {
+		if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) {
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
+				"line %u FAILED: %s",
+				__LINE__, "Error sending packet for
 encryption");
+			status = TEST_FAILED;
+			goto error_exit;
+		}
 
-	while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0)
-		rte_pause();
+		op = NULL;
 
-	if (!op) {
-		snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
-			"line %u FAILED: %s",
-			__LINE__, "Failed to process sym crypto op");
-		status = TEST_FAILED;
-		goto error_exit;
+		while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0)
+			rte_pause();
+
+		if (!op) {
+			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
+				"line %u FAILED: %s",
+				__LINE__, "Failed to process sym crypto op");
+			status = TEST_FAILED;
+			goto error_exit;
+		}
app/test/test_cryptodev_blockcipher.c +0 -1
 	}
 
 	debug_hexdump(stdout, "m_src(after):",
--

2.20.1

#61 lore raw

Re: [dpdk-dev] [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs

In reply to Fan Zhang

Hi Fan,

Quoted 18 lines

The Crypto Raw data-path APIs are a set of APIs are designed to enable externel libraries/applications which want to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but in a manner that is not dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe input data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or dequeue at any time, and operation status translation and set on the fly.

Quoted 10 lines

V10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

I have some comment on the V10, please revert asap so that these can be merged in RC1.

@[email protected]: Could you please have a look at the 2/4 patch as well?

Regards,
Akhil

#62 lore raw

Re: [dpdk-dev] [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs

Ok. Will do ASAP.

Quoted 56 lines

-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, October 8, 2020 4:04 PM
To: Zhang, Roy Fan <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>
Cc: Trahe, Fiona <[email protected]>; Kusztal, ArkadiuszX <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]
Subject: RE: [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs

Hi Fan,

Quoted 3 lines

The Crypto Raw data-path APIs are a set of APIs are designed to enable externel libraries/applications which want to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but

in a

Quoted 1 line

manner that is not dependent on native DPDK data structures (eg.

rte_mbuf,

Quoted 5 lines

rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer

and

Quoted 2 lines

IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe

input

Quoted 2 lines

data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more

efficient.

Quoted 1 line

- Flexible enqueue and dequeue operation. The raw data-path APIs gives

the

Quoted 2 lines

user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or

dequeue

Quoted 4 lines

at any time, and operation status translation and set on the fly.

V10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto

and

Quoted 8 lines

async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

I have some comment on the V10, please revert asap so that these can be merged in RC1.

@[email protected]: Could you please have a look at the 2/4 patch as well?

Regards,
Akhil

author
#63 lore raw

v11 cover cryptodev: add raw data-path APIs

In reply to Fan Zhang

The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but in a manner that is not dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe input data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or dequeue at any time, and operation status translation and set on the fly.

v11:
- Rebased on top of latest master.
- API changed followed by the discussion results.
- Fixed a few grammar error thanks to Akhil.
- Reverted attach session API changes.
- Fixed QAT driver bugs.

v10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: change crypto symmetric vector structure
cryptodev: add raw crypto data-path APIs crypto/qat: add raw crypto data-path API support test/crypto: add unit-test for cryptodev raw API test

 app/test/test_cryptodev.c                     | 805 ++++++++++++++-
 app/test/test_cryptodev.h                     |  12 +
 app/test/test_cryptodev_blockcipher.c         |  58 +-
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/cryptodevs/features/qat.ini        |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       | 100 +-
 doc/guides/rel_notes/release_20_11.rst        |  10 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c      |  18 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    |   9 +-
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 959 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto_sym.h         |  40 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 367 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 +-
 .../rte_cryptodev_version.map                 |  10 +
 lib/librte_ipsec/esp_inb.c                    |  12 +-
 lib/librte_ipsec/esp_outb.c                   |  12 +-
 lib/librte_ipsec/misc.h                       |   6 +-
 21 files changed, 2464 insertions(+), 108 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#64 lore raw

PATCH v11 1/4 cryptodev: change crypto symmetric vector structure

This patch updates ``rte_crypto_sym_vec`` structure to add support for both cpu_crypto synchrounous operation and asynchronous raw data-path APIs. The patch also includes AESNI-MB and AESNI-GCM PMD changes, unit test changes and documentation updates.

---
 app/test/test_cryptodev.c                  | 25 ++++++++------
 doc/guides/prog_guide/cryptodev_lib.rst    |  3 +-
 doc/guides/rel_notes/release_20_11.rst     |  3 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   | 18 +++++-----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  9 +++--
 lib/librte_cryptodev/rte_crypto_sym.h      | 40 ++++++++++++++++------
 lib/librte_ipsec/esp_inb.c                 | 12 +++----
 lib/librte_ipsec/esp_outb.c                | 12 +++----
 lib/librte_ipsec/misc.h                    |  6 ++--
 9 files changed, 79 insertions(+), 49 deletions(-)
app/test/test_cryptodev.c +15 -10
index ac2a36bc2..62a265520 100644
@@ -151,11 +151,11 @@ static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, aad_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -171,13 +171,17 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->aead.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
+	symvec.aad = &aad_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	/* for CPU crypto the IOVA address is not required */
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->aead.digest.data;
+	aad_ptr.va = (void *)sop->aead.aad.data;
+
 	ofs.raw = 0;
 
 	n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
@@ -193,11 +197,11 @@ static void
 process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -213,13 +217,14 @@ process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->auth.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->auth.digest.data;
+
 	ofs.raw = 0;
 	ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
 	ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
doc/guides/prog_guide/cryptodev_lib.rst +2 -1
index c14f750fa..e7ba35c2d 100644
@@ -620,7 +620,8 @@ operation descriptor (``struct rte_crypto_sym_vec``) containing:
   descriptors of performed operations (``struct rte_crypto_sgl``). Each instance
   of ``struct rte_crypto_sgl`` consists of a number of segments and a pointer to
   an array of segment descriptors ``struct rte_crypto_vec``;
-- pointers to arrays of size ``num`` containing IV, AAD and digest information,
+- pointers to arrays of size ``num`` containing IV, AAD and digest information
+  in the ``cpu_crypto`` sub-structure,
 - pointer to an array of size ``num`` where status information will be stored
   for each operation.
 
doc/guides/rel_notes/release_20_11.rst +3 -0
index 8b911488c..2973b2a33 100644
@@ -302,6 +302,9 @@ API Changes
   ``rte_fpga_lte_fec_configure`` and structure ``fpga_lte_fec_conf`` to
   ``rte_fpga_lte_fec_conf``.
 
+* The structure ``rte_crypto_sym_vec`` is updated to support both
+  cpu_crypto synchrounous operation and asynchronous raw data-path APIs.
+
 
 ABI Changes
 -----------
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -8
index 1d2a0ce00..973b61bd6 100644
@@ -464,9 +464,10 @@ aesni_gcm_sgl_encrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -482,9 +483,10 @@ aesni_gcm_sgl_decrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		 vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -505,9 +507,9 @@ aesni_gmac_sgl_generate(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -528,9 +530,9 @@ aesni_gmac_sgl_verify(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +4 -5
index 34a39ca99..39f90f537 100644
@@ -1877,7 +1877,7 @@ generate_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			memcpy(vec->digest[i], dgst[i], len);
+			memcpy(vec->digest[i].va, dgst[i], len);
 			k++;
 		}
 	}
@@ -1893,7 +1893,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			if (memcmp(vec->digest[i], dgst[i], len) != 0)
+			if (memcmp(vec->digest[i].va, dgst[i], len) != 0)
 				vec->status[i] = EBADMSG;
 			else
 				k++;
@@ -1956,9 +1956,8 @@ aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
 		}
 
 		/* Submit job for processing */
-		set_cpu_mb_job_params(job, s, sofs, buf, len,
-			vec->iv[i], vec->aad[i], tmp_dgst[i],
-			&vec->status[i]);
+		set_cpu_mb_job_params(job, s, sofs, buf, len, vec->iv[i].va,
+			vec->aad[i].va, tmp_dgst[i], &vec->status[i]);
 		job = submit_sync_job(mb_mgr);
 		j++;
 
lib/librte_cryptodev/rte_crypto_sym.h +29 -11
index f29c98051..e1f23d303 100644
@@ -51,26 +51,44 @@ struct rte_crypto_sgl {
 };
 
 /**
- * Synchronous operation descriptor.
- * Supposed to be used with CPU crypto API call.
+ * Crypto virtual and IOVA address descriptor, used to describe cryptographic
+ * data buffer without the length information. The length information is
+ * normally predefined during session creation.
+ */
+struct rte_crypto_va_iova_ptr {
+	void *va;
+	rte_iova_t iova;
+};
+
+/**
+ * Raw data operation descriptor.
+ * Supposed to be used with synchronous CPU crypto API call or asynchronous
+ * RAW data path API call.
  */
 struct rte_crypto_sym_vec {
+	/** number of operations to perform */
+	uint32_t num;
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
+	/** array of pointers to cipher IV */
+	struct rte_crypto_va_iova_ptr *iv;
 	/** array of pointers to digest */
-	void **digest;
+	struct rte_crypto_va_iova_ptr *digest;
+
+	__extension__
+	union {
+		/** array of pointers to auth IV, used for chain operation */
+		struct rte_crypto_va_iova_ptr *auth_iv;
+		/** array of pointers to AAD, used for AEAD operation */
+		struct rte_crypto_va_iova_ptr *aad;
+	};
+
 	/**
 	 * array of statuses for each operation:
-	 *  - 0 on success
-	 *  - errno on error
+	 * - 0 on success
+	 * - errno on error
 	 */
 	int32_t *status;
-	/** number of operations to perform */
-	uint32_t num;
 };
 
 /**
lib/librte_ipsec/esp_inb.c +6 -6
index 96eec0131..2b1df6a03 100644
@@ -693,9 +693,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 	struct rte_ipsec_sa *sa;
 	struct replay_sqn *rsn;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -720,9 +720,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 				l4ofs + k, rc, ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/esp_outb.c +6 -6
index fb9d5864c..1e181cf2c 100644
@@ -449,9 +449,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 	uint32_t i, k, n;
 	uint32_t l2, l3;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -488,9 +488,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 				ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/misc.h +4 -2
index 1b543ed87..79b9a2076 100644
@@ -112,7 +112,9 @@ mbuf_cut_seg_ofs(struct rte_mbuf *mb, struct rte_mbuf *ms, uint32_t ofs,
 static inline void
 cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 	union rte_crypto_sym_ofs ofs, struct rte_mbuf *mb[],
-	void *iv[], void *aad[], void *dgst[], uint32_t l4ofs[],
+	struct rte_crypto_va_iova_ptr iv[],
+	struct rte_crypto_va_iova_ptr aad[],
+	struct rte_crypto_va_iova_ptr dgst[], uint32_t l4ofs[],
 	uint32_t clen[], uint32_t num)
 {
 	uint32_t i, j, n;
@@ -136,8 +138,8 @@ cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 			/* fill the request structure */
 			symvec.sgl = &vecpkt[j];
 			symvec.iv = &iv[j];
-			symvec.aad = &aad[j];
 			symvec.digest = &dgst[j];
+			symvec.aad = &aad[j];
 			symvec.status = &st[j];
 			symvec.num = i - j;
 
Signature · 1 line
2.20.1
author
#65 lore raw

PATCH v11 2/4 cryptodev: add raw crypto data-path APIs

In reply to Fan Zhang

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

---
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/cryptodevs/features/qat.ini        |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       |  97 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++++
 lib/librte_cryptodev/rte_cryptodev.h          | 367 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 ++-
 .../rte_cryptodev_version.map                 |  10 +
 8 files changed, 611 insertions(+), 3 deletions(-)
doc/guides/cryptodevs/features/default.ini +1 -0
index 133a246ee..17b177fc4 100644
@@ -30,6 +30,7 @@ Asymmetric sessionless =
 CPU crypto             =
 Symmetric sessionless  =
 Non-Byte aligned data  =
+Sym raw data path API  =
 
 ;
 ; Supported crypto algorithms of a default crypto driver.
doc/guides/cryptodevs/features/qat.ini +1 -0
index 9e82f2886..6cc09cde7 100644
@@ -17,6 +17,7 @@ Digest encrypted       = Y
 Asymmetric sessionless = Y
 RSA PRIV OP KEY EXP    = Y
 RSA PRIV OP KEY QT     = Y
+Sym raw data path API  = Y
 
 ;
 ; Supported crypto algorithms of the 'qat' crypto driver.
doc/guides/prog_guide/cryptodev_lib.rst +97 -0

103 lines not on this page · raw message

doc/guides/rel_notes/release_20_11.rst +7 -0
index 2973b2a33..947671f09 100644
@@ -165,6 +165,13 @@ New Features
   * Extern objects and functions can be plugged into the pipeline.
   * Transaction-oriented table updates.
 
+* **Added raw data-path APIs for cryptodev library.**
+
+     Cryptodev is added raw data-path APIs to accelerate external libraries or
+     applications those want to avail fast cryptodev enqueue/dequeue
+     operations but does not necessarily depends on mbufs and cryptodev
+     operation mempool.
+
 
 Removed Items
 -------------
lib/librte_cryptodev/rte_cryptodev.c +80 -0

86 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +366 -1

379 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +49 -2

63 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 7727286ac..7e4360ff0 100644
@@ -99,4 +99,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_configure_raw_dp_ctx;
+	rte_cryptodev_get_raw_dp_ctx_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
Signature · 1 line
2.20.1
#66 lore raw

Re: [dpdk-dev] [dpdk-dev v11 2/4] cryptodev: add raw crypto data-path APIs

Hi Fan,

Quoted 8 lines

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

Quoted 11 lines
---
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/cryptodevs/features/qat.ini        |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       |  97 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++++
 lib/librte_cryptodev/rte_cryptodev.h          | 367 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 ++-
 .../rte_cryptodev_version.map                 |  10 +
 8 files changed, 611 insertions(+), 3 deletions(-)

The release notes should be updated just above aesni_mb crypto PMD updates

+* **Added raw data-path APIs for cryptodev library.** +

+  Cryptodev is added with raw data-path APIs to accelerate external
+  libraries or applications which need to avail fast cryptodev
+  enqueue/dequeue operations but does not necessarily depends on
+  mbufs and cryptodev operation mempools.

+

I have following diff which should be incorporated in this patch. Qat.ini file should be updated in the 3/4 patch. Release notes update is also missing for QAT.

doc/guides/cryptodevs/features/qat.ini +0 -1
index 9e82f2886..6cc09cde7 100644
@@ -17,6 +17,7 @@ Digest encrypted       = Y
 Asymmetric sessionless = Y
 RSA PRIV OP KEY EXP    = Y
 RSA PRIV OP KEY QT     = Y
-Sym raw data path API  = Y
 
 ;

; Supported crypto algorithms of the 'qat' crypto driver.

doc/guides/prog_guide/cryptodev_lib.rst +2 -2
index 8ba800122..7fb3022bd 100644
@@ -696,9 +696,9 @@ the status buffer provided by the user):
   are stored. The crypto device will then start enqueuing all of them at
   once.
 
-Calling ``rte_cryptodev_configure_raw_dp_context`` with the parameter
+Calling ``rte_cryptodev_configure_raw_dp_ctx`` with the parameter
 ``is_update`` set as 0 twice without the enqueue function returning status 1 or
-``rte_cryptodev_dp_enqueue_done`` function call in between will invalidate any
+``rte_cryptodev_raw_enqueue_done`` function call in between will invalidate any
 descriptors stored in the device queue but not enqueued. This feature is useful
 when the user wants to abandon partially enqueued data for a failed enqueue
 burst operation and try enqueuing in a whole later.
lib/librte_cryptodev/rte_cryptodev.c +7 -7
index 7a143c4b9..3d95ac6ea 100644
@@ -1833,13 +1833,6 @@ rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
        return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
 }
 
-int
-rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
-               uint32_t n)
-{
-       return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
-}
-
 uint32_t
 rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
        rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
@@ -1852,6 +1845,13 @@ rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
                is_user_data_array, n_success_jobs, status);
 }
 
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+               uint32_t n)
+{
+       return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
 /** Initialise rte_crypto_op mempool element */
 static void
 rte_crypto_op_init(struct rte_mempool *mempool,
lib/librte_cryptodev/rte_cryptodev.h +10 -15
index 840a1c54c..79cfa46c8 100644
@@ -459,7 +459,7 @@ rte_cryptodev_asym_get_xform_enum(enum rte_crypto_asym_xform_type *xform_enum,
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA         (1ULL << 23)
 /**< Support operations on data which is not byte aligned */
 #define RTE_CRYPTODEV_FF_SYM_RAW_DP                    (1ULL << 24)
-/**< Support accelerated specific raw data-path APIs */
+/**< Support accelerator specific symmetric raw data-path APIs */
 
 /**
  * Get the name of a crypto device feature flag
@@ -1344,7 +1344,7 @@ union rte_cryptodev_session_ctx {
 };
 
 /**
- * Enqueue a data vector into device queue but the driver will not start
+ * Enqueue a data vector into device queue but the driver may or may not start
  * processing until rte_cryptodev_raw_enqueue_done() is called.
  *
  * @param      qp              Driver specific queue pair data.
@@ -1357,7 +1357,7 @@ union rte_cryptodev_session_ctx {
  * @return
  *   - The number of descriptors successfully enqueued.
  *   - Possible enqueue status written by the driver:
- *     - 1: The descriptors are enqueued successfully.
+ *     - positive integer: number of descriptors enqueued successfully.
  *     - 0: The descriptors are stored into device queue but are not processed
  *          until rte_cryptodev_raw_enqueue_done() is called.
  *     - negative integer: failure.
@@ -1451,7 +1451,7 @@ typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
  * @return
  *   - The number of descriptors successfully dequeued.
  *   - Possible dequeue status written by the driver:
- *     - 1: The descriptors are dequeued successfully.
+ *     - positive integer: Number of descriptors dequeued successfully.
  *     - 0: The descriptors are fetched from queue pair but are not freed
  *          until rte_cryptodev_raw_dequeue_done() is called.
  *     - negative integer: Error occurred when dequeuing.
@@ -1475,8 +1475,8 @@ typedef uint32_t (*cryptodev_sym_raw_dequeue_burst_t)(void *qp,
  *   - The user data pointer retrieved from device queue or NULL if no
  *     descriptor is ready for dequeue.
  *   - Possible dequeue status written by the driver:
- *     - 1: The descriptors are dequeued successfully.
- *     - 0: The descriptors are fetched from queue pair but are not freed
+ *     - 1: The descriptor is dequeued successfully.
+ *     - 0: The descriptor is fetched from queue pair but is not freed
  *          until rte_cryptodev_raw_dequeue_done() is called.
  *     - negative integer: Error occurred when dequeuing.
  */
@@ -1487,7 +1487,7 @@ typedef void * (*cryptodev_sym_raw_dequeue_t)(
 /**
  * Context data for raw data-path API crypto process. The buffer of this
  * structure is to be allocated by the user application with the size equal
- * or bigger than rte_cryptodev_raw_get_dp_context_size() returned value.
+ * or bigger than rte_cryptodev_get_raw_dp_ctx_size() returned value.
  *
  * NOTE: the buffer is to be used and maintained by the cryptodev driver, the
  * user should NOT alter the buffer content to avoid application or system
@@ -1510,11 +1510,6 @@ struct rte_crypto_raw_dp_ctx {
 /**
  * Configure raw data-path context data.
  *
- * NOTE:
- * After the context data is configured, the user should call
- * rte_cryptodev_raw_attach_session() before using it in
- * rte_cryptodev_raw_enqueue/dequeue function call.
- *
  * @param      dev_id          The device identifier.
  * @param      qp_id           The index of the queue pair from which to
  *                             retrieve processed packets. The value must be
@@ -1596,7 +1591,7 @@ rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
 
 /**
  * Start processing all enqueued descriptors from last
- * rte_cryptodev_raw_configure_dp_context() call.
+ * rte_cryptodev_configure_raw_dp_ctx() call.
  *
  * @param      ctx     The initialized raw data-path context data.
  * @param      n       The total number of submitted descriptors.
@@ -1656,8 +1651,8 @@ rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
  *   - The user data pointer retrieved from device queue or NULL if no
  *     descriptor is ready for dequeue.
  *   - Possible dequeue status written by the driver:
- *     - 1: The descriptors are dequeued successfully.
- *     - 0: The descriptors are fetched from queue pair but are not freed
+ *     - 1: The descriptor is dequeued successfully.
+ *     - 0: The descriptor is fetched from queue pair but is not freed
  *          until rte_cryptodev_raw_dequeue_done() is called.
  *     - negative integer: Error occurred when dequeuing.
  */
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -3
index d6be69903..ea8694ca5 100644
@@ -329,7 +329,7 @@ typedef uint32_t (*cryptodev_sym_cpu_crypto_process_t)
 typedef int (*cryptodev_sym_get_raw_dp_ctx_size_t)(struct rte_cryptodev *dev);
 
 /**
- * Typedef that the driver provided to configure data-path context.
+ * Typedef that the driver provided to configure raw data-path context.
  *
  * @param      dev             Crypto device pointer.
  * @param      qp_id           Crypto device queue pair index.
@@ -392,10 +392,10 @@ struct rte_cryptodev_ops {
                struct {
                        cryptodev_sym_get_raw_dp_ctx_size_t
                                sym_get_raw_dp_ctx_size;
-                       /**< Get data path service context data size. */
+                       /**< Get raw data path context data size. */
                        cryptodev_sym_configure_raw_dp_ctx_t
                                sym_configure_raw_dp_ctx;
-                       /**< Initialize crypto service ctx data. */
+                       /**< Configure raw data path ctx data. */
                };
        };
 };
#67 lore raw

Re: [dpdk-dev] [dpdk-dev v11 2/4] cryptodev: add raw crypto data-path APIs

Hi Akhil,

Will upate in v11.

regards,
Fan

________________________________
发件人: Akhil Goyal <[email protected]>
发送时间: Saturday, October 10, 2020 8:38:39 PM
收件人: Zhang, Roy Fan <[email protected]>; [email protected] <[email protected]>
抄送: Bronowski, PiotrX <[email protected]>; Dybkowski, AdamX <[email protected]>; Ananyev, Konstantin <[email protected]> 主题: RE: [dpdk-dev v11 2/4] cryptodev: add raw crypto data-path APIs

Hi Fan,

Quoted 8 lines

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

Quoted 11 lines
---
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/cryptodevs/features/qat.ini        |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       |  97 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++++
 lib/librte_cryptodev/rte_cryptodev.h          | 367 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 ++-
 .../rte_cryptodev_version.map                 |  10 +
 8 files changed, 611 insertions(+), 3 deletions(-)

The release notes should be updated just above aesni_mb crypto PMD updates

+* **Added raw data-path APIs for cryptodev library.** +

+  Cryptodev is added with raw data-path APIs to accelerate external
+  libraries or applications which need to avail fast cryptodev
+  enqueue/dequeue operations but does not necessarily depends on
+  mbufs and cryptodev operation mempools.

+

I have following diff which should be incorporated in this patch. Qat.ini file should be updated in the 3/4 patch. Release notes update is also missing for QAT.

doc/guides/cryptodevs/features/qat.ini +0 -1
index 9e82f2886..6cc09cde7 100644
@@ -17,6 +17,7 @@ Digest encrypted       = Y
 Asymmetric sessionless = Y
 RSA PRIV OP KEY EXP    = Y
 RSA PRIV OP KEY QT     = Y
-Sym raw data path API  = Y
 
 ;

; Supported crypto algorithms of the 'qat' crypto driver.

doc/guides/prog_guide/cryptodev_lib.rst +2 -2
index 8ba800122..7fb3022bd 100644
@@ -696,9 +696,9 @@ the status buffer provided by the user):
   are stored. The crypto device will then start enqueuing all of them at
   once.
 
-Calling ``rte_cryptodev_configure_raw_dp_context`` with the parameter
+Calling ``rte_cryptodev_configure_raw_dp_ctx`` with the parameter
 ``is_update`` set as 0 twice without the enqueue function returning status 1 or
-``rte_cryptodev_dp_enqueue_done`` function call in between will invalidate any
+``rte_cryptodev_raw_enqueue_done`` function call in between will invalidate any
 descriptors stored in the device queue but not enqueued. This feature is useful
 when the user wants to abandon partially enqueued data for a failed enqueue
 burst operation and try enqueuing in a whole later.
lib/librte_cryptodev/rte_cryptodev.c +7 -7
index 7a143c4b9..3d95ac6ea 100644
@@ -1833,13 +1833,6 @@ rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
        return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
 }
 
-int
-rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
-               uint32_t n)
-{
-       return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
-}
-
 uint32_t
 rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
        rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
@@ -1852,6 +1845,13 @@ rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
                is_user_data_array, n_success_jobs, status);
 }
 
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+               uint32_t n)
+{
+       return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
 /** Initialise rte_crypto_op mempool element */
 static void
 rte_crypto_op_init(struct rte_mempool *mempool,
lib/librte_cryptodev/rte_cryptodev.h +10 -15
index 840a1c54c..79cfa46c8 100644
@@ -459,7 +459,7 @@ rte_cryptodev_asym_get_xform_enum(enum rte_crypto_asym_xform_type *xform_enum,
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA         (1ULL << 23)
 /**< Support operations on data which is not byte aligned */
 #define RTE_CRYPTODEV_FF_SYM_RAW_DP                    (1ULL << 24)
-/**< Support accelerated specific raw data-path APIs */
+/**< Support accelerator specific symmetric raw data-path APIs */
 
 /**
  * Get the name of a crypto device feature flag
@@ -1344,7 +1344,7 @@ union rte_cryptodev_session_ctx {
 };
 
 /**
- * Enqueue a data vector into device queue but the driver will not start
+ * Enqueue a data vector into device queue but the driver may or may not start
  * processing until rte_cryptodev_raw_enqueue_done() is called.
  *
  * @param      qp              Driver specific queue pair data.
@@ -1357,7 +1357,7 @@ union rte_cryptodev_session_ctx {
  * @return
  *   - The number of descriptors successfully enqueued.
  *   - Possible enqueue status written by the driver:
- *     - 1: The descriptors are enqueued successfully.
+ *     - positive integer: number of descriptors enqueued successfully.
  *     - 0: The descriptors are stored into device queue but are not processed
  *          until rte_cryptodev_raw_enqueue_done() is called.
  *     - negative integer: failure.
@@ -1451,7 +1451,7 @@ typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
  * @return
  *   - The number of descriptors successfully dequeued.
  *   - Possible dequeue status written by the driver:
- *     - 1: The descriptors are dequeued successfully.
+ *     - positive integer: Number of descriptors dequeued successfully.
  *     - 0: The descriptors are fetched from queue pair but are not freed
  *          until rte_cryptodev_raw_dequeue_done() is called.
  *     - negative integer: Error occurred when dequeuing.
@@ -1475,8 +1475,8 @@ typedef uint32_t (*cryptodev_sym_raw_dequeue_burst_t)(void *qp,
  *   - The user data pointer retrieved from device queue or NULL if no
  *     descriptor is ready for dequeue.
  *   - Possible dequeue status written by the driver:
- *     - 1: The descriptors are dequeued successfully.
- *     - 0: The descriptors are fetched from queue pair but are not freed
+ *     - 1: The descriptor is dequeued successfully.
+ *     - 0: The descriptor is fetched from queue pair but is not freed
  *          until rte_cryptodev_raw_dequeue_done() is called.
  *     - negative integer: Error occurred when dequeuing.
  */
@@ -1487,7 +1487,7 @@ typedef void * (*cryptodev_sym_raw_dequeue_t)(
 /**
  * Context data for raw data-path API crypto process. The buffer of this
  * structure is to be allocated by the user application with the size equal
- * or bigger than rte_cryptodev_raw_get_dp_context_size() returned value.
+ * or bigger than rte_cryptodev_get_raw_dp_ctx_size() returned value.
  *
  * NOTE: the buffer is to be used and maintained by the cryptodev driver, the
  * user should NOT alter the buffer content to avoid application or system
@@ -1510,11 +1510,6 @@ struct rte_crypto_raw_dp_ctx {
 /**
  * Configure raw data-path context data.
  *
- * NOTE:
- * After the context data is configured, the user should call
- * rte_cryptodev_raw_attach_session() before using it in
- * rte_cryptodev_raw_enqueue/dequeue function call.
- *
  * @param      dev_id          The device identifier.
  * @param      qp_id           The index of the queue pair from which to
  *                             retrieve processed packets. The value must be
@@ -1596,7 +1591,7 @@ rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
 
 /**
  * Start processing all enqueued descriptors from last
- * rte_cryptodev_raw_configure_dp_context() call.
+ * rte_cryptodev_configure_raw_dp_ctx() call.
  *
  * @param      ctx     The initialized raw data-path context data.
  * @param      n       The total number of submitted descriptors.
@@ -1656,8 +1651,8 @@ rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
  *   - The user data pointer retrieved from device queue or NULL if no
  *     descriptor is ready for dequeue.
  *   - Possible dequeue status written by the driver:
- *     - 1: The descriptors are dequeued successfully.
- *     - 0: The descriptors are fetched from queue pair but are not freed
+ *     - 1: The descriptor is dequeued successfully.
+ *     - 0: The descriptor is fetched from queue pair but is not freed
  *          until rte_cryptodev_raw_dequeue_done() is called.
  *     - negative integer: Error occurred when dequeuing.
  */
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -3
index d6be69903..ea8694ca5 100644
@@ -329,7 +329,7 @@ typedef uint32_t (*cryptodev_sym_cpu_crypto_process_t)
 typedef int (*cryptodev_sym_get_raw_dp_ctx_size_t)(struct rte_cryptodev *dev);
 
 /**
- * Typedef that the driver provided to configure data-path context.
+ * Typedef that the driver provided to configure raw data-path context.
  *
  * @param      dev             Crypto device pointer.
  * @param      qp_id           Crypto device queue pair index.
@@ -392,10 +392,10 @@ struct rte_cryptodev_ops {
                struct {
                        cryptodev_sym_get_raw_dp_ctx_size_t
                                sym_get_raw_dp_ctx_size;
-                       /**< Get data path service context data size. */
+                       /**< Get raw data path context data size. */
                        cryptodev_sym_configure_raw_dp_ctx_t
                                sym_configure_raw_dp_ctx;
-                       /**< Initialize crypto service ctx data. */
+                       /**< Configure raw data path ctx data. */
                };
        };
 };
author
#68 lore raw

PATCH v11 3/4 crypto/qat: add raw crypto data-path API support

In reply to Fan Zhang

This patch updates QAT PMD to add raw data-path API support.

---
 drivers/crypto/qat/meson.build     |   1 +
 drivers/crypto/qat/qat_sym.h       |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c | 959 +++++++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c   |   9 +-
 4 files changed, 978 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +11 -0
index 1a9748849..7254f5e3c 100644
@@ -264,6 +264,16 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *raw_dp_ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx, uint8_t is_update);
+
+int
+qat_sym_get_dp_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +286,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +959 -0

959 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..6b189c319 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Raw data-path API related operations */
+		.sym_get_raw_dp_ctx_size = qat_sym_get_dp_ctx_size,
+		.sym_configure_raw_dp_ctx = qat_sym_configure_dp_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_SYM_RAW_DP;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#69 lore raw

PATCH v11 4/4 test/crypto: add unit-test for cryptodev raw API test

In reply to Fan Zhang

This patch adds the cryptodev raw API test support to unit test. In addtion a new test-case for QAT PMD for the test type is enabled.

---
 app/test/test_cryptodev.c             | 780 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |  12 +
 app/test/test_cryptodev_blockcipher.c |  58 +-
 3 files changed, 796 insertions(+), 54 deletions(-)
app/test/test_cryptodev.c +742 -38
index 62a265520..9514a9159 100644
@@ -49,6 +49,10 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
+#define MAX_RAW_DEQUEUE_COUNT	65535
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +61,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+enum cryptodev_api_test_type global_api_test_type = CRYPTODEV_API_TEST;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +153,208 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+static uint32_t
+get_raw_dp_dequeue_count(void *user_data __rte_unused)
+{
+	return 1;
+}
+
+static void
+post_process_raw_dp_op(void *user_data,	uint32_t index __rte_unused,
+		uint8_t is_op_success)
+{
+	struct rte_crypto_op *op = user_data;
+	op->status = is_op_success ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+	struct rte_crypto_sym_op *sop = op->sym;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+	union rte_crypto_sym_ofs ofs;
+	struct rte_crypto_sym_vec vec;
+	struct rte_crypto_sgl sgl;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	uint32_t count = 0;
+	struct rte_crypto_raw_dp_ctx *ctx;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int32_t n;
+	uint32_t n_success;
+	int ctx_service_size;
+	int32_t status = 0;
+
+	ctx_service_size = rte_cryptodev_get_raw_dp_ctx_size(dev_id);
+	if (ctx_service_size < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ctx = malloc(ctx_service_size);
+	if (!ctx) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* Both are enums, setting crypto_sess will suit any session type */
+	sess.crypto_sess = op->sym->session;
+
+	if (rte_cryptodev_configure_raw_dp_ctx(dev_id, qp_id, ctx,
+			op->sess_type, sess, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	cipher_iv.iova = 0;
+	cipher_iv.va = NULL;
+	aad_auth_iv.iova = 0;
+	aad_auth_iv.va = NULL;
+	digest.iova = 0;
+	digest.va = NULL;
+	sgl.vec = data_vec;
+	vec.num = 1;
+	vec.sgl = &sgl;
+	vec.iv = &cipher_iv;
+	vec.digest = &digest;
+	vec.aad = &aad_auth_iv;
+	vec.status = &status;
+
+	ofs.raw = 0;
+
+	if (is_cipher && is_auth) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			auth_offset = auth_offset >> 3;
+			cipher_len = cipher_len >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else if (is_cipher) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	} else if (is_auth) {
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			auth_offset = auth_offset >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else { /* aead */
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = (void *)sop->aead.aad.data;
+		aad_auth_iv.iova = sop->aead.aad.phys_addr;
+		digest.va = (void *)sop->aead.digest.data;
+		digest.iova = sop->aead.digest.phys_addr;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	sgl.num = n;
+
+	if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op,
+			&status)
+			< 1) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	if (status == 0) {
+		status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+		if (status < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	}
+
+	n = n_success = 0;
+	while (count++ < MAX_RAW_DEQUEUE_COUNT && n == 0) {
+		n = rte_cryptodev_raw_dequeue_burst(ctx,
+			get_raw_dp_dequeue_count, post_process_raw_dp_op,
+				(void **)&ret_op, 0, &n_success, &status);
+
+		if (n == 0)
+			rte_pause();
+	}
+
+	if (n == 1 && status == 0) {
+		if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	}
+
+	op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op != op ||
+			n_success < 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+					RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+exit:
+	free(ctx);
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1661,6 +1869,9 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params->valid_devs[0],
@@ -1715,12 +1926,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2435,6 +2652,12 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2475,7 +2698,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2513,6 +2740,12 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2554,7 +2787,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2580,6 +2817,16 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2624,6 +2871,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2653,6 +2903,16 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2695,7 +2955,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2860,6 +3124,16 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2902,8 +3176,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2959,6 +3237,12 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -2988,7 +3272,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3031,10 +3319,14 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3116,6 +3408,9 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3201,6 +3496,9 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3269,6 +3567,16 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	uint8_t *ciphertext, *plaintext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3311,7 +3619,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3344,6 +3656,16 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3386,7 +3708,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3427,6 +3753,9 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3510,6 +3839,9 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3629,6 +3961,9 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3719,6 +4054,16 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3761,7 +4106,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3799,6 +4148,9 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3886,6 +4238,12 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EEA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_ZUC_EEA3;
@@ -3929,7 +4287,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -3969,6 +4331,16 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4024,7 +4396,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4092,6 +4468,14 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
+	}
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
 	}
 
 	/* Create SNOW 3G session */
@@ -4160,7 +4544,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4270,7 +4658,14 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4349,7 +4744,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4457,7 +4856,15 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 
 	uint64_t feat_flags = dev_info.feature_flags;
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	if (op_mode == OUT_OF_PLACE) {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4531,7 +4938,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4642,7 +5053,14 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4721,7 +5139,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4806,6 +5228,16 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4862,7 +5294,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4905,6 +5341,16 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 
@@ -4949,7 +5395,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5004,6 +5454,12 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	plaintext_len = ceil_byte_length(tdata->plaintext.len);
 
 	/* Append data which is padded to a multiple */
@@ -5036,7 +5492,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5084,6 +5544,12 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EIA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_ZUC_EIA3;
@@ -5124,7 +5590,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5181,7 +5651,15 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5256,7 +5734,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5363,7 +5845,15 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5442,7 +5932,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5585,6 +6079,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5784,6 +6281,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5798,6 +6298,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6151,11 +6654,12 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6166,6 +6670,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
 		retval = create_wireless_algo_cipher_auth_session(
@@ -6197,9 +6704,11 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}
 
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6240,18 +6749,17 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6342,11 +6850,12 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6445,20 +6954,18 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -6999,6 +7506,16 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	uint8_t *ciphertext, *auth_tag;
 	uint16_t plaintext_pad_len;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -7048,6 +7565,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8496,6 +9016,16 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	int retval;
 	uint8_t *plaintext;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -8545,6 +9075,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8839,6 +9372,9 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -8928,8 +9464,9 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and raw data-path APIs*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			global_api_test_type == CRYPTODEV_RAW_API_TEST)
 		return -ENOTSUP;
 
 	/* Create AEAD session */
@@ -9115,6 +9652,12 @@ test_authenticated_decryption_sessionless(
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -9156,8 +9699,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
 
@@ -9447,6 +9995,16 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9475,6 +10033,9 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9507,6 +10068,16 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9533,6 +10104,9 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10038,6 +10612,16 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	int retval;
 
@@ -10101,6 +10685,9 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10159,6 +10746,16 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	int retval;
 	uint32_t plaintext_pad_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	TEST_ASSERT_NOT_EQUAL(tdata->gmac_tag.len, 0,
 			      "No GMAC length in the source data");
@@ -10218,6 +10815,9 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10733,6 +11333,16 @@ test_authentication_verify_fail_when_data_corruption(
 	int retval;
 
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10742,6 +11352,7 @@ test_authentication_verify_fail_when_data_corruption(
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+
 	/* Create session */
 	retval = create_auth_session(ut_params,
 			ts_params->valid_devs[0],
@@ -10783,7 +11394,10 @@ test_authentication_verify_fail_when_data_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10801,6 +11415,16 @@ test_authentication_verify_GMAC_fail_when_corruption(
 {
 	int retval;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10854,7 +11478,10 @@ test_authentication_verify_GMAC_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10873,6 +11500,16 @@ test_authenticated_decryption_fail_when_corruption(
 	int retval;
 
 	uint8_t *ciphertext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10929,7 +11566,10 @@ test_authenticated_decryption_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10950,6 +11590,16 @@ test_authenticated_encryt_with_esn(
 	uint16_t plaintext_pad_len;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11024,6 +11674,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11070,6 +11723,16 @@ test_authenticated_decrypt_with_esn(
 	uint8_t *ciphertext;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11144,6 +11807,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -11284,10 +11950,21 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		if (sgl_in && (!(dev_info.feature_flags &
 				RTE_CRYPTODEV_FF_IN_PLACE_SGL)))
 			return -ENOTSUP;
+
+		uint64_t feat_flags = dev_info.feature_flags;
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Raw data path API does not support OOP */
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
@@ -11483,6 +12160,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -12980,6 +13660,30 @@ test_cryptodev_bcmfs(void)
 	return unit_test_suite_runner(&cryptodev_testsuite);
 }
 
+static int
+test_cryptodev_qat_raw_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	global_api_test_type = CRYPTODEV_RAW_API_TEST;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	global_api_test_type = CRYPTODEV_API_TEST;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+		test_cryptodev_qat_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +12 -0
index c58126368..bf4f6c8d6 100644
@@ -72,6 +72,13 @@
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 #define CRYPTODEV_NAME_BCMFS_PMD	crypto_bcmfs
 
+enum cryptodev_api_test_type {
+	CRYPTODEV_API_TEST = 0,
+	CRYPTODEV_RAW_API_TEST
+};
+
+extern enum cryptodev_api_test_type global_api_test_type;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -210,4 +217,9 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +42 -16

79 lines not on this page · raw message

Signature · 1 line
2.20.1
#70 lore raw

Re: [dpdk-dev] [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test

Hi Fan,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 159

Quoted 4 lines
app/test/test_cryptodev.c +4 -0
+static uint32_t
+get_raw_dp_dequeue_count(void *user_data __rte_unused)
+{
+	return 1;

Why is this 1 always? There could be jobs >1 which are processed.

Quoted 159 lines
app/test/test_cryptodev.c +90 -0
+}
+
+static void
+post_process_raw_dp_op(void *user_data,	uint32_t index __rte_unused,
+		uint8_t is_op_success)
+{
+	struct rte_crypto_op *op = user_data;
+	op->status = is_op_success ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+	struct rte_crypto_sym_op *sop = op->sym;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+	union rte_crypto_sym_ofs ofs;
+	struct rte_crypto_sym_vec vec;
+	struct rte_crypto_sgl sgl;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	uint32_t count = 0;
+	struct rte_crypto_raw_dp_ctx *ctx;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int32_t n;
+	uint32_t n_success;
+	int ctx_service_size;
+	int32_t status = 0;
+
+	ctx_service_size = rte_cryptodev_get_raw_dp_ctx_size(dev_id);
+	if (ctx_service_size < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ctx = malloc(ctx_service_size);
+	if (!ctx) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* Both are enums, setting crypto_sess will suit any session type */
+	sess.crypto_sess = op->sym->session;
+
+	if (rte_cryptodev_configure_raw_dp_ctx(dev_id, qp_id, ctx,
+			op->sess_type, sess, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	cipher_iv.iova = 0;
+	cipher_iv.va = NULL;
+	aad_auth_iv.iova = 0;
+	aad_auth_iv.va = NULL;
+	digest.iova = 0;
+	digest.va = NULL;
+	sgl.vec = data_vec;
+	vec.num = 1;
+	vec.sgl = &sgl;
+	vec.iv = &cipher_iv;
+	vec.digest = &digest;
+	vec.aad = &aad_auth_iv;
+	vec.status = &status;
+
+	ofs.raw = 0;
+
+	if (is_cipher && is_auth) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			auth_offset = auth_offset >> 3;
+			cipher_len = cipher_len >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +4 -0
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else if (is_cipher) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +16 -0
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	} else if (is_auth) {
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			auth_offset = auth_offset >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else { /* aead */
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +12 -0
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = (void *)sop->aead.aad.data;
+		aad_auth_iv.iova = sop->aead.aad.phys_addr;
+		digest.va = (void *)sop->aead.digest.data;
+		digest.iova = sop->aead.digest.phys_addr;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 323

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+	}
+
+	sgl.num = n;
+
+	if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op,
+			&status)
+			< 1) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}

This check will always fail in your case. I believe you have not tested it with the Recent changes that I suggested on V10. rte_cryptodev_raw_enqueue_burst will return 0 if rte_cryptodev_raw_enqueue_done need to be called or else the number of successfully enqueued descriptors.

Quoted 14 lines
app/test/test_cryptodev.c +13 -0
+
+	if (status == 0) {
+		status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+		if (status < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	}
+
+	n = n_success = 0;
+	while (count++ < MAX_RAW_DEQUEUE_COUNT && n == 0) {
+		n = rte_cryptodev_raw_dequeue_burst(ctx,
+			get_raw_dp_dequeue_count,

post_process_raw_dp_op,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 341

Quoted 5 lines
app/test/test_cryptodev.c +5 -0
+				(void **)&ret_op, 0, &n_success, &status);
+
+		if (n == 0)
+			rte_pause();
+	}

Same comment here as well.
rte_cryptodev_raw_dequeue_burst will return 0 if dequeue_done need to be called. These checks does not seem to be consistent with the API documentation.

Quoted 17 lines
app/test/test_cryptodev.c +9 -0
+
+	if (n == 1 && status == 0) {
+		if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	}
+
+	op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op !=

op ||

app/test/test_cryptodev.c +7 -0
+			n_success < 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+					RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+exit:
+	free(ctx);
+}
+
#71 lore raw

Re: [dpdk-dev] [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test

Hi Akhil,

For your “always return 1” question:

The way dequeue_burst API works is we may not know how many ops to dequeue without parsing the first user data (e.g. a structure containing n_ops data). It is up to the user to provide a callback function to return the number of ops - either by parsing the data structure, or a constant number - so in our unit test it is always 1 op to process. So returning 1.

For 2nd and 3rd question: enqueue_burst and dequeue_burst have to return 2 values, the number of ops enqueued or stored but not enqueued, and the operation status (enqueued/stored by not enqueued/error). The changed API will return the number of ops enqueued/dequeued, that’s why I made the check here. The operation status (0/1/error code) is stored in the “status” field by the driver. This is explained in the header file comments

“

+ * @return
+ * - The number of descriptors successfully enqueued.
+ * - Possible enqueue status written by the driver:
+ * - 1: The descriptors are enqueued successfully.
+ * - 0: The descriptors are stored into device queue but are not processed
+ * until rte_cryptodev_raw_enqueue_done() is called.
+ * - negative integer: failure.

If you think it is not clear, any suggestions?

Regards,
Fan

________________________________
发件人: Akhil Goyal <[email protected]>
发送时间: Saturday, October 10, 2020 8:55:07 PM
收件人: Zhang, Roy Fan <[email protected]>; [email protected] <[email protected]> 抄送: Dybkowski, AdamX <[email protected]>
主题: RE: [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test

Hi Fan,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 159

Quoted 4 lines
app/test/test_cryptodev.c +4 -0
+static uint32_t
+get_raw_dp_dequeue_count(void *user_data __rte_unused)
+{
+     return 1;

Why is this 1 always? There could be jobs >1 which are processed.

Quoted 159 lines
app/test/test_cryptodev.c +90 -0
+}
+
+static void
+post_process_raw_dp_op(void *user_data,      uint32_t index __rte_unused,
+             uint8_t is_op_success)
+{
+     struct rte_crypto_op *op = user_data;
+     op->status = is_op_success ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+                     RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+             struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+             uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+     struct rte_crypto_sym_op *sop = op->sym;
+     struct rte_crypto_op *ret_op = NULL;
+     struct rte_crypto_vec data_vec[UINT8_MAX];
+     struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+     union rte_crypto_sym_ofs ofs;
+     struct rte_crypto_sym_vec vec;
+     struct rte_crypto_sgl sgl;
+     uint32_t max_len;
+     union rte_cryptodev_session_ctx sess;
+     uint32_t count = 0;
+     struct rte_crypto_raw_dp_ctx *ctx;
+     uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+                     auth_len = 0;
+     int32_t n;
+     uint32_t n_success;
+     int ctx_service_size;
+     int32_t status = 0;
+
+     ctx_service_size = rte_cryptodev_get_raw_dp_ctx_size(dev_id);
+     if (ctx_service_size < 0) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             return;
+     }
+
+     ctx = malloc(ctx_service_size);
+     if (!ctx) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             return;
+     }
+
+     /* Both are enums, setting crypto_sess will suit any session type */
+     sess.crypto_sess = op->sym->session;
+
+     if (rte_cryptodev_configure_raw_dp_ctx(dev_id, qp_id, ctx,
+                     op->sess_type, sess, 0) < 0) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             goto exit;
+     }
+
+     cipher_iv.iova = 0;
+     cipher_iv.va = NULL;
+     aad_auth_iv.iova = 0;
+     aad_auth_iv.va = NULL;
+     digest.iova = 0;
+     digest.va = NULL;
+     sgl.vec = data_vec;
+     vec.num = 1;
+     vec.sgl = &sgl;
+     vec.iv = &cipher_iv;
+     vec.digest = &digest;
+     vec.aad = &aad_auth_iv;
+     vec.status = &status;
+
+     ofs.raw = 0;
+
+     if (is_cipher && is_auth) {
+             cipher_offset = sop->cipher.data.offset;
+             cipher_len = sop->cipher.data.length;
+             auth_offset = sop->auth.data.offset;
+             auth_len = sop->auth.data.length;
+             max_len = RTE_MAX(cipher_offset + cipher_len,
+                             auth_offset + auth_len);
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     cipher_offset = cipher_offset >> 3;
+                     auth_offset = auth_offset >> 3;
+                     cipher_len = cipher_len >> 3;
+                     auth_len = auth_len >> 3;
+             }
+             ofs.ofs.cipher.head = cipher_offset;
+             ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+             ofs.ofs.auth.head = auth_offset;
+             ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+             cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +4 -0
+             cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+             aad_auth_iv.va = rte_crypto_op_ctod_offset(
+                             op, void *, IV_OFFSET + cipher_iv_len);
+             aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+                             cipher_iv_len);
+             digest.va = (void *)sop->auth.digest.data;
+             digest.iova = sop->auth.digest.phys_addr;
+
+     } else if (is_cipher) {
+             cipher_offset = sop->cipher.data.offset;
+             cipher_len = sop->cipher.data.length;
+             max_len = cipher_len + cipher_offset;
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     cipher_offset = cipher_offset >> 3;
+                     cipher_len = cipher_len >> 3;
+             }
+             ofs.ofs.cipher.head = cipher_offset;
+             ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+             cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +16 -0
+             cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+     } else if (is_auth) {
+             auth_offset = sop->auth.data.offset;
+             auth_len = sop->auth.data.length;
+             max_len = auth_len + auth_offset;
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     auth_offset = auth_offset >> 3;
+                     auth_len = auth_len >> 3;
+             }
+             ofs.ofs.auth.head = auth_offset;
+             ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+             aad_auth_iv.va = rte_crypto_op_ctod_offset(
+                             op, void *, IV_OFFSET + cipher_iv_len);
+             aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+                             cipher_iv_len);
+             digest.va = (void *)sop->auth.digest.data;
+             digest.iova = sop->auth.digest.phys_addr;
+
+     } else { /* aead */
+             cipher_offset = sop->aead.data.offset;
+             cipher_len = sop->aead.data.length;
+             max_len = cipher_len + cipher_offset;
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     cipher_offset = cipher_offset >> 3;
+                     cipher_len = cipher_len >> 3;
+             }
+             ofs.ofs.cipher.head = cipher_offset;
+             ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+             cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +12 -0
+             cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+             aad_auth_iv.va = (void *)sop->aead.aad.data;
+             aad_auth_iv.iova = sop->aead.aad.phys_addr;
+             digest.va = (void *)sop->aead.digest.data;
+             digest.iova = sop->aead.digest.phys_addr;
+     }
+
+     n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+                     data_vec, RTE_DIM(data_vec));
+     if (n < 0 || n > sop->m_src->nb_segs) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             goto exit;

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 323

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+     }
+
+     sgl.num = n;
+
+     if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op,
+                     &status)
+                     < 1) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             goto exit;
+     }

This check will always fail in your case. I believe you have not tested it with the Recent changes that I suggested on V10. rte_cryptodev_raw_enqueue_burst will return 0 if rte_cryptodev_raw_enqueue_done need to be called or else the number of successfully enqueued descriptors.

Quoted 14 lines
app/test/test_cryptodev.c +13 -0
+
+     if (status == 0) {
+             status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+             if (status < 0) {
+                     op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+                     goto exit;
+             }
+     }
+
+     n = n_success = 0;
+     while (count++ < MAX_RAW_DEQUEUE_COUNT && n == 0) {
+             n = rte_cryptodev_raw_dequeue_burst(ctx,
+                     get_raw_dp_dequeue_count,

post_process_raw_dp_op,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 341

Quoted 5 lines
app/test/test_cryptodev.c +5 -0
+                             (void **)&ret_op, 0, &n_success, &status);
+
+             if (n == 0)
+                     rte_pause();
+     }

Same comment here as well.
rte_cryptodev_raw_dequeue_burst will return 0 if dequeue_done need to be called. These checks does not seem to be consistent with the API documentation.

Quoted 17 lines
app/test/test_cryptodev.c +9 -0
+
+     if (n == 1 && status == 0) {
+             if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+                     op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+                     goto exit;
+             }
+     }
+
+     op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op !=

op ||

app/test/test_cryptodev.c +7 -0
+                     n_success < 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+                                     RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+exit:
+     free(ctx);
+}
+
#72 lore raw

Re: [dpdk-dev] [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test

Hi Fan,

OK got your point. Review the changes in the documentation of APIs that I suggested in the 2/4 patch according to the below explanation.

Regards,
Akhil

From: Zhang, Roy Fan <[email protected]>
Sent: Sunday, October 11, 2020 2:21 AM
To: Akhil Goyal <[email protected]>; [email protected]
Cc: Dybkowski, AdamX <[email protected]>
Subject: Re: [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test

Hi Akhil,

For your “always return 1” question:

The way dequeue_burst API works is we may not know how many ops to dequeue without parsing the first user data (e.g. a structure containing n_ops data). It is up to the user to provide a callback function to return the number of ops - either by parsing the data structure, or a constant number - so in our unit test it is always 1 op to process. So returning 1.

For 2nd and 3rd question: enqueue_burst and dequeue_burst have to return 2 values, the number of ops enqueued or stored but not enqueued, and the operation status (enqueued/stored by not enqueued/error). The changed API will return the number of ops enqueued/dequeued, that’s why I made the check here. The operation status (0/1/error code) is stored in the “status” field by the driver. This is explained in the header file comments

“

+ * @return

+ * - The number of descriptors successfully enqueued.

+ * - Possible enqueue status written by the driver:

+ * - 1: The descriptors are enqueued successfully.

+ * - 0: The descriptors are stored into device queue but are not processed

+ * until rte_cryptodev_raw_enqueue_done() is called.

+ * - negative integer: failure.
If you think it is not clear, any suggestions?

Regards,
Fan

________________________________
发件人: Akhil Goyal <[email protected]<mailto:[email protected]>>
发送时间: Saturday, October 10, 2020 8:55:07 PM
收件人: Zhang, Roy Fan <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> 抄送: Dybkowski, AdamX <[email protected]<mailto:[email protected]>>
主题: RE: [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test

Hi Fan,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 159

Quoted 4 lines
app/test/test_cryptodev.c +4 -0
+static uint32_t
+get_raw_dp_dequeue_count(void *user_data __rte_unused)
+{
+     return 1;

Why is this 1 always? There could be jobs >1 which are processed.

Quoted 159 lines
app/test/test_cryptodev.c +90 -0
+}
+
+static void
+post_process_raw_dp_op(void *user_data,      uint32_t index __rte_unused,
+             uint8_t is_op_success)
+{
+     struct rte_crypto_op *op = user_data;
+     op->status = is_op_success ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+                     RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+             struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+             uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+     struct rte_crypto_sym_op *sop = op->sym;
+     struct rte_crypto_op *ret_op = NULL;
+     struct rte_crypto_vec data_vec[UINT8_MAX];
+     struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+     union rte_crypto_sym_ofs ofs;
+     struct rte_crypto_sym_vec vec;
+     struct rte_crypto_sgl sgl;
+     uint32_t max_len;
+     union rte_cryptodev_session_ctx sess;
+     uint32_t count = 0;
+     struct rte_crypto_raw_dp_ctx *ctx;
+     uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+                     auth_len = 0;
+     int32_t n;
+     uint32_t n_success;
+     int ctx_service_size;
+     int32_t status = 0;
+
+     ctx_service_size = rte_cryptodev_get_raw_dp_ctx_size(dev_id);
+     if (ctx_service_size < 0) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             return;
+     }
+
+     ctx = malloc(ctx_service_size);
+     if (!ctx) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             return;
+     }
+
+     /* Both are enums, setting crypto_sess will suit any session type */
+     sess.crypto_sess = op->sym->session;
+
+     if (rte_cryptodev_configure_raw_dp_ctx(dev_id, qp_id, ctx,
+                     op->sess_type, sess, 0) < 0) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             goto exit;
+     }
+
+     cipher_iv.iova = 0;
+     cipher_iv.va = NULL;
+     aad_auth_iv.iova = 0;
+     aad_auth_iv.va = NULL;
+     digest.iova = 0;
+     digest.va = NULL;
+     sgl.vec = data_vec;
+     vec.num = 1;
+     vec.sgl = &sgl;
+     vec.iv = &cipher_iv;
+     vec.digest = &digest;
+     vec.aad = &aad_auth_iv;
+     vec.status = &status;
+
+     ofs.raw = 0;
+
+     if (is_cipher && is_auth) {
+             cipher_offset = sop->cipher.data.offset;
+             cipher_len = sop->cipher.data.length;
+             auth_offset = sop->auth.data.offset;
+             auth_len = sop->auth.data.length;
+             max_len = RTE_MAX(cipher_offset + cipher_len,
+                             auth_offset + auth_len);
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     cipher_offset = cipher_offset >> 3;
+                     auth_offset = auth_offset >> 3;
+                     cipher_len = cipher_len >> 3;
+                     auth_len = auth_len >> 3;
+             }
+             ofs.ofs.cipher.head = cipher_offset;
+             ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+             ofs.ofs.auth.head = auth_offset;
+             ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+             cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +4 -0
+             cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+             aad_auth_iv.va = rte_crypto_op_ctod_offset(
+                             op, void *, IV_OFFSET + cipher_iv_len);
+             aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+                             cipher_iv_len);
+             digest.va = (void *)sop->auth.digest.data;
+             digest.iova = sop->auth.digest.phys_addr;
+
+     } else if (is_cipher) {
+             cipher_offset = sop->cipher.data.offset;
+             cipher_len = sop->cipher.data.length;
+             max_len = cipher_len + cipher_offset;
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     cipher_offset = cipher_offset >> 3;
+                     cipher_len = cipher_len >> 3;
+             }
+             ofs.ofs.cipher.head = cipher_offset;
+             ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+             cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +16 -0
+             cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+     } else if (is_auth) {
+             auth_offset = sop->auth.data.offset;
+             auth_len = sop->auth.data.length;
+             max_len = auth_len + auth_offset;
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     auth_offset = auth_offset >> 3;
+                     auth_len = auth_len >> 3;
+             }
+             ofs.ofs.auth.head = auth_offset;
+             ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+             aad_auth_iv.va = rte_crypto_op_ctod_offset(
+                             op, void *, IV_OFFSET + cipher_iv_len);
+             aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op,

IV_OFFSET +

app/test/test_cryptodev.c +16 -0
+                             cipher_iv_len);
+             digest.va = (void *)sop->auth.digest.data;
+             digest.iova = sop->auth.digest.phys_addr;
+
+     } else { /* aead */
+             cipher_offset = sop->aead.data.offset;
+             cipher_len = sop->aead.data.length;
+             max_len = cipher_len + cipher_offset;
+             if (len_in_bits) {
+                     max_len = max_len >> 3;
+                     cipher_offset = cipher_offset >> 3;
+                     cipher_len = cipher_len >> 3;
+             }
+             ofs.ofs.cipher.head = cipher_offset;
+             ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+             cipher_iv.va = rte_crypto_op_ctod_offset(op, void *,

IV_OFFSET);

app/test/test_cryptodev.c +12 -0
+             cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+             aad_auth_iv.va = (void *)sop->aead.aad.data;
+             aad_auth_iv.iova = sop->aead.aad.phys_addr;
+             digest.va = (void *)sop->aead.digest.data;
+             digest.iova = sop->aead.digest.phys_addr;
+     }
+
+     n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+                     data_vec, RTE_DIM(data_vec));
+     if (n < 0 || n > sop->m_src->nb_segs) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             goto exit;

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 323

Quoted 10 lines
app/test/test_cryptodev.c +10 -0
+     }
+
+     sgl.num = n;
+
+     if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op,
+                     &status)
+                     < 1) {
+             op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+             goto exit;
+     }

This check will always fail in your case. I believe you have not tested it with the Recent changes that I suggested on V10. rte_cryptodev_raw_enqueue_burst will return 0 if rte_cryptodev_raw_enqueue_done need to be called or else the number of successfully enqueued descriptors.

Quoted 14 lines
app/test/test_cryptodev.c +13 -0
+
+     if (status == 0) {
+             status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+             if (status < 0) {
+                     op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+                     goto exit;
+             }
+     }
+
+     n = n_success = 0;
+     while (count++ < MAX_RAW_DEQUEUE_COUNT && n == 0) {
+             n = rte_cryptodev_raw_dequeue_burst(ctx,
+                     get_raw_dp_dequeue_count,

post_process_raw_dp_op,

app/test/test_cryptodev.c ceil_byte_length(uint32_t num_bits) · line 341

Quoted 5 lines
app/test/test_cryptodev.c +5 -0
+                             (void **)&ret_op, 0, &n_success, &status);
+
+             if (n == 0)
+                     rte_pause();
+     }

Same comment here as well.
rte_cryptodev_raw_dequeue_burst will return 0 if dequeue_done need to be called. These checks does not seem to be consistent with the API documentation.

Quoted 17 lines
app/test/test_cryptodev.c +9 -0
+
+     if (n == 1 && status == 0) {
+             if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+                     op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+                     goto exit;
+             }
+     }
+
+     op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op !=

op ||

app/test/test_cryptodev.c +7 -0
+                     n_success < 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+                                     RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+exit:
+     free(ctx);
+}
+
author
#73 lore raw

v12 cover cryptodev: add raw data-path APIs

In reply to Fan Zhang

The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but in a manner that is not dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe input data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or dequeue at any time, and operation status translation and set on the fly.

v12:
- Fixed and updated documentation.
- Fixed typo.

v11:
- Rebased on top of latest master.
- API changed followed by the discussion results.
- Fixed a few grammar error thanks to Akhil.
- Reverted attach session API changes.
- Fixed QAT driver bugs.

v10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: change crypto symmetric vector structure
cryptodev: add raw crypto data-path APIs crypto/qat: add raw crypto data-path API support test/crypto: add unit-test for cryptodev raw API test

 app/test/test_cryptodev.c                     | 812 ++++++++++++++-
 app/test/test_cryptodev.h                     |  12 +
 app/test/test_cryptodev_blockcipher.c         |  58 +-
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/cryptodevs/features/qat.ini        |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       | 109 +-
 doc/guides/rel_notes/release_20_11.rst        |  14 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c      |  18 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    |   9 +-
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 959 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto_sym.h         |  40 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 413 +++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 +-
 .../rte_cryptodev_version.map                 |  10 +
 lib/librte_ipsec/esp_inb.c                    |  12 +-
 lib/librte_ipsec/esp_outb.c                   |  12 +-
 lib/librte_ipsec/misc.h                       |   6 +-
 21 files changed, 2530 insertions(+), 108 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#74 lore raw

PATCH v12 1/4 cryptodev: change crypto symmetric vector structure

This patch updates ``rte_crypto_sym_vec`` structure to add support for both cpu_crypto synchrounous operation and asynchronous raw data-path APIs. The patch also includes AESNI-MB and AESNI-GCM PMD changes, unit test changes and documentation updates.

---
 app/test/test_cryptodev.c                  | 25 ++++++++------
 doc/guides/prog_guide/cryptodev_lib.rst    |  3 +-
 doc/guides/rel_notes/release_20_11.rst     |  3 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   | 18 +++++-----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  9 +++--
 lib/librte_cryptodev/rte_crypto_sym.h      | 40 ++++++++++++++++------
 lib/librte_ipsec/esp_inb.c                 | 12 +++----
 lib/librte_ipsec/esp_outb.c                | 12 +++----
 lib/librte_ipsec/misc.h                    |  6 ++--
 9 files changed, 79 insertions(+), 49 deletions(-)
app/test/test_cryptodev.c +15 -10
index ac2a36bc2..62a265520 100644
@@ -151,11 +151,11 @@ static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, aad_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -171,13 +171,17 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->aead.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
+	symvec.aad = &aad_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	/* for CPU crypto the IOVA address is not required */
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->aead.digest.data;
+	aad_ptr.va = (void *)sop->aead.aad.data;
+
 	ofs.raw = 0;
 
 	n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
@@ -193,11 +197,11 @@ static void
 process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -213,13 +217,14 @@ process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->auth.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->auth.digest.data;
+
 	ofs.raw = 0;
 	ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
 	ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
doc/guides/prog_guide/cryptodev_lib.rst +2 -1
index c14f750fa..e7ba35c2d 100644
@@ -620,7 +620,8 @@ operation descriptor (``struct rte_crypto_sym_vec``) containing:
   descriptors of performed operations (``struct rte_crypto_sgl``). Each instance
   of ``struct rte_crypto_sgl`` consists of a number of segments and a pointer to
   an array of segment descriptors ``struct rte_crypto_vec``;
-- pointers to arrays of size ``num`` containing IV, AAD and digest information,
+- pointers to arrays of size ``num`` containing IV, AAD and digest information
+  in the ``cpu_crypto`` sub-structure,
 - pointer to an array of size ``num`` where status information will be stored
   for each operation.
 
doc/guides/rel_notes/release_20_11.rst +3 -0
index 8b911488c..2973b2a33 100644
@@ -302,6 +302,9 @@ API Changes
   ``rte_fpga_lte_fec_configure`` and structure ``fpga_lte_fec_conf`` to
   ``rte_fpga_lte_fec_conf``.
 
+* The structure ``rte_crypto_sym_vec`` is updated to support both
+  cpu_crypto synchrounous operation and asynchronous raw data-path APIs.
+
 
 ABI Changes
 -----------
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -8
index 1d2a0ce00..973b61bd6 100644
@@ -464,9 +464,10 @@ aesni_gcm_sgl_encrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -482,9 +483,10 @@ aesni_gcm_sgl_decrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		 vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -505,9 +507,9 @@ aesni_gmac_sgl_generate(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -528,9 +530,9 @@ aesni_gmac_sgl_verify(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +4 -5
index 34a39ca99..39f90f537 100644
@@ -1877,7 +1877,7 @@ generate_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			memcpy(vec->digest[i], dgst[i], len);
+			memcpy(vec->digest[i].va, dgst[i], len);
 			k++;
 		}
 	}
@@ -1893,7 +1893,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			if (memcmp(vec->digest[i], dgst[i], len) != 0)
+			if (memcmp(vec->digest[i].va, dgst[i], len) != 0)
 				vec->status[i] = EBADMSG;
 			else
 				k++;
@@ -1956,9 +1956,8 @@ aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
 		}
 
 		/* Submit job for processing */
-		set_cpu_mb_job_params(job, s, sofs, buf, len,
-			vec->iv[i], vec->aad[i], tmp_dgst[i],
-			&vec->status[i]);
+		set_cpu_mb_job_params(job, s, sofs, buf, len, vec->iv[i].va,
+			vec->aad[i].va, tmp_dgst[i], &vec->status[i]);
 		job = submit_sync_job(mb_mgr);
 		j++;
 
lib/librte_cryptodev/rte_crypto_sym.h +29 -11
index f29c98051..e1f23d303 100644
@@ -51,26 +51,44 @@ struct rte_crypto_sgl {
 };
 
 /**
- * Synchronous operation descriptor.
- * Supposed to be used with CPU crypto API call.
+ * Crypto virtual and IOVA address descriptor, used to describe cryptographic
+ * data buffer without the length information. The length information is
+ * normally predefined during session creation.
+ */
+struct rte_crypto_va_iova_ptr {
+	void *va;
+	rte_iova_t iova;
+};
+
+/**
+ * Raw data operation descriptor.
+ * Supposed to be used with synchronous CPU crypto API call or asynchronous
+ * RAW data path API call.
  */
 struct rte_crypto_sym_vec {
+	/** number of operations to perform */
+	uint32_t num;
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
+	/** array of pointers to cipher IV */
+	struct rte_crypto_va_iova_ptr *iv;
 	/** array of pointers to digest */
-	void **digest;
+	struct rte_crypto_va_iova_ptr *digest;
+
+	__extension__
+	union {
+		/** array of pointers to auth IV, used for chain operation */
+		struct rte_crypto_va_iova_ptr *auth_iv;
+		/** array of pointers to AAD, used for AEAD operation */
+		struct rte_crypto_va_iova_ptr *aad;
+	};
+
 	/**
 	 * array of statuses for each operation:
-	 *  - 0 on success
-	 *  - errno on error
+	 * - 0 on success
+	 * - errno on error
 	 */
 	int32_t *status;
-	/** number of operations to perform */
-	uint32_t num;
 };
 
 /**
lib/librte_ipsec/esp_inb.c +6 -6
index 96eec0131..2b1df6a03 100644
@@ -693,9 +693,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 	struct rte_ipsec_sa *sa;
 	struct replay_sqn *rsn;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -720,9 +720,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 				l4ofs + k, rc, ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/esp_outb.c +6 -6
index fb9d5864c..1e181cf2c 100644
@@ -449,9 +449,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 	uint32_t i, k, n;
 	uint32_t l2, l3;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -488,9 +488,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 				ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/misc.h +4 -2
index 1b543ed87..79b9a2076 100644
@@ -112,7 +112,9 @@ mbuf_cut_seg_ofs(struct rte_mbuf *mb, struct rte_mbuf *ms, uint32_t ofs,
 static inline void
 cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 	union rte_crypto_sym_ofs ofs, struct rte_mbuf *mb[],
-	void *iv[], void *aad[], void *dgst[], uint32_t l4ofs[],
+	struct rte_crypto_va_iova_ptr iv[],
+	struct rte_crypto_va_iova_ptr aad[],
+	struct rte_crypto_va_iova_ptr dgst[], uint32_t l4ofs[],
 	uint32_t clen[], uint32_t num)
 {
 	uint32_t i, j, n;
@@ -136,8 +138,8 @@ cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 			/* fill the request structure */
 			symvec.sgl = &vecpkt[j];
 			symvec.iv = &iv[j];
-			symvec.aad = &aad[j];
 			symvec.digest = &dgst[j];
+			symvec.aad = &aad[j];
 			symvec.status = &st[j];
 			symvec.num = i - j;
 
Signature · 1 line
2.20.1
author
#75 lore raw

PATCH v12 2/4 cryptodev: add raw crypto data-path APIs

In reply to Fan Zhang

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

---
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       | 106 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++++
 lib/librte_cryptodev/rte_cryptodev.h          | 413 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 ++-
 .../rte_cryptodev_version.map                 |  10 +
 7 files changed, 665 insertions(+), 3 deletions(-)
doc/guides/cryptodevs/features/default.ini +1 -0
index 133a246ee..17b177fc4 100644
@@ -30,6 +30,7 @@ Asymmetric sessionless =
 CPU crypto             =
 Symmetric sessionless  =
 Non-Byte aligned data  =
+Sym raw data path API  =
 
 ;
 ; Supported crypto algorithms of a default crypto driver.
doc/guides/prog_guide/cryptodev_lib.rst +106 -0

112 lines not on this page · raw message

doc/guides/rel_notes/release_20_11.rst +7 -0
index 2973b2a33..85a07d86e 100644
@@ -74,6 +74,13 @@ New Features
 
   * Added SR-IOV PF support
 
+* **Added raw data-path APIs for cryptodev library.**
+
+  Cryptodev is added with raw data-path APIs to accelerate external
+  libraries or applications which need to avail fast cryptodev
+  enqueue/dequeue operations but does not necessarily depends on
+  mbufs and cryptodev operation mempools.
+
 * **Updated the aesni_mb crypto PMD.**
 
   * Added support for AES-ECB 128, 192 and 256.
lib/librte_cryptodev/rte_cryptodev.c +80 -0

86 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +412 -1

425 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +49 -2

63 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 7727286ac..7e4360ff0 100644
@@ -99,4 +99,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_configure_raw_dp_ctx;
+	rte_cryptodev_get_raw_dp_ctx_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
Signature · 1 line
2.20.1
author
#76 lore raw

PATCH v12 3/4 crypto/qat: add raw crypto data-path API support

In reply to Fan Zhang

This patch updates QAT PMD to add raw data-path API support.

---
 doc/guides/cryptodevs/features/qat.ini |   1 +
 doc/guides/rel_notes/release_20_11.rst |   4 +
 drivers/crypto/qat/meson.build         |   1 +
 drivers/crypto/qat/qat_sym.h           |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c     | 959 +++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c       |   9 +-
 6 files changed, 983 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
doc/guides/cryptodevs/features/qat.ini +1 -0
index 9e82f2886..6cc09cde7 100644
@@ -17,6 +17,7 @@ Digest encrypted       = Y
 Asymmetric sessionless = Y
 RSA PRIV OP KEY EXP    = Y
 RSA PRIV OP KEY QT     = Y
+Sym raw data path API  = Y
 
 ;
 ; Supported crypto algorithms of the 'qat' crypto driver.
doc/guides/rel_notes/release_20_11.rst +4 -0
index 85a07d86e..008f4eedc 100644
@@ -104,6 +104,10 @@ New Features
   * Added support for non-HMAC auth algorithms
     (MD5, SHA1, SHA224, SHA256, SHA384, SHA512).
 
+* **Update QAT crypto PMD.**
+
+  * Added Raw Data-path APIs support.
+
 * **Added Intel ACC100 bbdev PMD.**
 
   Added a new ``acc100`` bbdev driver for the Intel\ |reg| ACC100 accelerator
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +11 -0
index 1a9748849..7254f5e3c 100644
@@ -264,6 +264,16 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *raw_dp_ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx, uint8_t is_update);
+
+int
+qat_sym_get_dp_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +286,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +959 -0

959 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..6b189c319 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Raw data-path API related operations */
+		.sym_get_raw_dp_ctx_size = qat_sym_get_dp_ctx_size,
+		.sym_configure_raw_dp_ctx = qat_sym_configure_dp_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_SYM_RAW_DP;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#77 lore raw

PATCH v12 4/4 test/crypto: add unit-test for cryptodev raw API test

In reply to Fan Zhang

This patch adds the cryptodev raw API test support to unit test. In addition a new test-case for QAT PMD for the test type is enabled.

---
 app/test/test_cryptodev.c             | 787 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |  12 +
 app/test/test_cryptodev_blockcipher.c |  58 +-
 3 files changed, 803 insertions(+), 54 deletions(-)
app/test/test_cryptodev.c +749 -38
index 62a265520..219373e10 100644
@@ -49,6 +49,10 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
+#define MAX_RAW_DEQUEUE_COUNT	65535
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +61,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+enum cryptodev_api_test_type global_api_test_type = CRYPTODEV_API_TEST;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +153,215 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+static uint32_t
+get_raw_dp_dequeue_count(void *user_data __rte_unused)
+{
+	return 1;
+}
+
+static void
+post_process_raw_dp_op(void *user_data,	uint32_t index __rte_unused,
+		uint8_t is_op_success)
+{
+	struct rte_crypto_op *op = user_data;
+	op->status = is_op_success ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+	struct rte_crypto_sym_op *sop = op->sym;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+	union rte_crypto_sym_ofs ofs;
+	struct rte_crypto_sym_vec vec;
+	struct rte_crypto_sgl sgl;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	uint32_t count = 0;
+	struct rte_crypto_raw_dp_ctx *ctx;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int32_t n;
+	uint32_t n_success;
+	int ctx_service_size;
+	int32_t status = 0;
+	int enqueue_status, dequeue_status;
+
+	ctx_service_size = rte_cryptodev_get_raw_dp_ctx_size(dev_id);
+	if (ctx_service_size < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ctx = malloc(ctx_service_size);
+	if (!ctx) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* Both are enums, setting crypto_sess will suit any session type */
+	sess.crypto_sess = op->sym->session;
+
+	if (rte_cryptodev_configure_raw_dp_ctx(dev_id, qp_id, ctx,
+			op->sess_type, sess, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	cipher_iv.iova = 0;
+	cipher_iv.va = NULL;
+	aad_auth_iv.iova = 0;
+	aad_auth_iv.va = NULL;
+	digest.iova = 0;
+	digest.va = NULL;
+	sgl.vec = data_vec;
+	vec.num = 1;
+	vec.sgl = &sgl;
+	vec.iv = &cipher_iv;
+	vec.digest = &digest;
+	vec.aad = &aad_auth_iv;
+	vec.status = &status;
+
+	ofs.raw = 0;
+
+	if (is_cipher && is_auth) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			auth_offset = auth_offset >> 3;
+			cipher_len = cipher_len >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else if (is_cipher) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	} else if (is_auth) {
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			auth_offset = auth_offset >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else { /* aead */
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = (void *)sop->aead.aad.data;
+		aad_auth_iv.iova = sop->aead.aad.phys_addr;
+		digest.va = (void *)sop->aead.digest.data;
+		digest.iova = sop->aead.digest.phys_addr;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	sgl.num = n;
+
+	if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op,
+			&enqueue_status) < 1) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	if (enqueue_status == 0) {
+		status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+		if (status < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	} else if (enqueue_status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	n = n_success = 0;
+	while (count++ < MAX_RAW_DEQUEUE_COUNT && n == 0) {
+		n = rte_cryptodev_raw_dequeue_burst(ctx,
+			get_raw_dp_dequeue_count, post_process_raw_dp_op,
+				(void **)&ret_op, 0, &n_success,
+				&dequeue_status);
+		if (dequeue_status < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+		if (n == 0)
+			rte_pause();
+	}
+
+	if (n == 1 && dequeue_status == 0) {
+		if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	}
+
+	op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op != op ||
+			n_success < 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+					RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+exit:
+	free(ctx);
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1661,6 +1876,9 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params->valid_devs[0],
@@ -1715,12 +1933,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2435,6 +2659,12 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2475,7 +2705,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2513,6 +2747,12 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2554,7 +2794,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2580,6 +2824,16 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2624,6 +2878,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2653,6 +2910,16 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2695,7 +2962,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2860,6 +3131,16 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2902,8 +3183,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2959,6 +3244,12 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -2988,7 +3279,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3031,10 +3326,14 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3116,6 +3415,9 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3201,6 +3503,9 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3269,6 +3574,16 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	uint8_t *ciphertext, *plaintext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3311,7 +3626,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3344,6 +3663,16 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3386,7 +3715,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3427,6 +3760,9 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3510,6 +3846,9 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3629,6 +3968,9 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3719,6 +4061,16 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3761,7 +4113,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3799,6 +4155,9 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3886,6 +4245,12 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EEA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_ZUC_EEA3;
@@ -3929,7 +4294,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -3969,6 +4338,16 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4024,7 +4403,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4092,6 +4475,14 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
+	}
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
 	}
 
 	/* Create SNOW 3G session */
@@ -4160,7 +4551,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4270,7 +4665,14 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4349,7 +4751,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4457,7 +4863,15 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 
 	uint64_t feat_flags = dev_info.feature_flags;
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	if (op_mode == OUT_OF_PLACE) {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4531,7 +4945,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4642,7 +5060,14 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4721,7 +5146,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4806,6 +5235,16 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4862,7 +5301,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4905,6 +5348,16 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 
@@ -4949,7 +5402,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5004,6 +5461,12 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	plaintext_len = ceil_byte_length(tdata->plaintext.len);
 
 	/* Append data which is padded to a multiple */
@@ -5036,7 +5499,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5084,6 +5551,12 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EIA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_ZUC_EIA3;
@@ -5124,7 +5597,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5181,7 +5658,15 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5256,7 +5741,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5363,7 +5852,15 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5442,7 +5939,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5585,6 +6086,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5784,6 +6288,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5798,6 +6305,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6151,11 +6661,12 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6166,6 +6677,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
 		retval = create_wireless_algo_cipher_auth_session(
@@ -6197,9 +6711,11 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}
 
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6240,18 +6756,17 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6342,11 +6857,12 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6445,20 +6961,18 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -6999,6 +7513,16 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	uint8_t *ciphertext, *auth_tag;
 	uint16_t plaintext_pad_len;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -7048,6 +7572,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8496,6 +9023,16 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	int retval;
 	uint8_t *plaintext;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -8545,6 +9082,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8839,6 +9379,9 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -8928,8 +9471,9 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and raw data-path APIs*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			global_api_test_type == CRYPTODEV_RAW_API_TEST)
 		return -ENOTSUP;
 
 	/* Create AEAD session */
@@ -9115,6 +9659,12 @@ test_authenticated_decryption_sessionless(
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -9156,8 +9706,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
 
@@ -9447,6 +10002,16 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9475,6 +10040,9 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9507,6 +10075,16 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9533,6 +10111,9 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10038,6 +10619,16 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	int retval;
 
@@ -10101,6 +10692,9 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10159,6 +10753,16 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	int retval;
 	uint32_t plaintext_pad_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	TEST_ASSERT_NOT_EQUAL(tdata->gmac_tag.len, 0,
 			      "No GMAC length in the source data");
@@ -10218,6 +10822,9 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10733,6 +11340,16 @@ test_authentication_verify_fail_when_data_corruption(
 	int retval;
 
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10742,6 +11359,7 @@ test_authentication_verify_fail_when_data_corruption(
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+
 	/* Create session */
 	retval = create_auth_session(ut_params,
 			ts_params->valid_devs[0],
@@ -10783,7 +11401,10 @@ test_authentication_verify_fail_when_data_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10801,6 +11422,16 @@ test_authentication_verify_GMAC_fail_when_corruption(
 {
 	int retval;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10854,7 +11485,10 @@ test_authentication_verify_GMAC_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10873,6 +11507,16 @@ test_authenticated_decryption_fail_when_corruption(
 	int retval;
 
 	uint8_t *ciphertext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10929,7 +11573,10 @@ test_authenticated_decryption_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10950,6 +11597,16 @@ test_authenticated_encryt_with_esn(
 	uint16_t plaintext_pad_len;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11024,6 +11681,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11070,6 +11730,16 @@ test_authenticated_decrypt_with_esn(
 	uint8_t *ciphertext;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11144,6 +11814,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -11284,10 +11957,21 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		if (sgl_in && (!(dev_info.feature_flags &
 				RTE_CRYPTODEV_FF_IN_PLACE_SGL)))
 			return -ENOTSUP;
+
+		uint64_t feat_flags = dev_info.feature_flags;
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Raw data path API does not support OOP */
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
@@ -11483,6 +12167,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -12980,6 +13667,30 @@ test_cryptodev_bcmfs(void)
 	return unit_test_suite_runner(&cryptodev_testsuite);
 }
 
+static int
+test_cryptodev_qat_raw_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	global_api_test_type = CRYPTODEV_RAW_API_TEST;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	global_api_test_type = CRYPTODEV_API_TEST;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+		test_cryptodev_qat_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +12 -0
index c58126368..bf4f6c8d6 100644
@@ -72,6 +72,13 @@
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 #define CRYPTODEV_NAME_BCMFS_PMD	crypto_bcmfs
 
+enum cryptodev_api_test_type {
+	CRYPTODEV_API_TEST = 0,
+	CRYPTODEV_RAW_API_TEST
+};
+
+extern enum cryptodev_api_test_type global_api_test_type;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -210,4 +217,9 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +42 -16

79 lines not on this page · raw message

Signature · 1 line
2.20.1
author
#78 lore raw

v13 cover cryptodev: add raw data-path APIs

In reply to Fan Zhang

The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but in a manner that is not dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe input data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or dequeue at any time, and operation status translation and set on the fly.

v13:
- Fixed a typo.

v12:
- Fixed and updated documentation.
- Fixed typo.

v11:
- Rebased on top of latest master.
- API changed followed by the discussion results.
- Fixed a few grammar error thanks to Akhil.
- Reverted attach session API changes.
- Fixed QAT driver bugs.

v10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
cryptodev: change crypto symmetric vector structure
cryptodev: add raw crypto data-path APIs crypto/qat: add raw crypto data-path API support test/crypto: add unit-test for cryptodev raw API test

 app/test/test_cryptodev.c                     | 812 ++++++++++++++-
 app/test/test_cryptodev.h                     |  12 +
 app/test/test_cryptodev_blockcipher.c         |  58 +-
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/cryptodevs/features/qat.ini        |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       | 109 +-
 doc/guides/rel_notes/release_20_11.rst        |  14 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c      |  18 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    |   9 +-
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 959 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto_sym.h         |  40 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++
 lib/librte_cryptodev/rte_cryptodev.h          | 413 +++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 +-
 .../rte_cryptodev_version.map                 |  10 +
 lib/librte_ipsec/esp_inb.c                    |  12 +-
 lib/librte_ipsec/esp_outb.c                   |  12 +-
 lib/librte_ipsec/misc.h                       |   6 +-
 21 files changed, 2530 insertions(+), 108 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
Signature · 1 line
2.20.1
author
#79 lore raw

PATCH v13 1/4 cryptodev: change crypto symmetric vector structure

This patch updates ``rte_crypto_sym_vec`` structure to add support for both cpu_crypto synchrounous operation and asynchronous raw data-path APIs. The patch also includes AESNI-MB and AESNI-GCM PMD changes, unit test changes and documentation updates.

---
 app/test/test_cryptodev.c                  | 25 ++++++++------
 doc/guides/prog_guide/cryptodev_lib.rst    |  3 +-
 doc/guides/rel_notes/release_20_11.rst     |  3 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   | 18 +++++-----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  9 +++--
 lib/librte_cryptodev/rte_crypto_sym.h      | 40 ++++++++++++++++------
 lib/librte_ipsec/esp_inb.c                 | 12 +++----
 lib/librte_ipsec/esp_outb.c                | 12 +++----
 lib/librte_ipsec/misc.h                    |  6 ++--
 9 files changed, 79 insertions(+), 49 deletions(-)
app/test/test_cryptodev.c +15 -10
index ac2a36bc2..62a265520 100644
@@ -151,11 +151,11 @@ static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, aad_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -171,13 +171,17 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->aead.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
+	symvec.aad = &aad_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	/* for CPU crypto the IOVA address is not required */
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->aead.digest.data;
+	aad_ptr.va = (void *)sop->aead.aad.data;
+
 	ofs.raw = 0;
 
 	n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
@@ -193,11 +197,11 @@ static void
 process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
 	int32_t n, st;
-	void *iv;
 	struct rte_crypto_sym_op *sop;
 	union rte_crypto_sym_ofs ofs;
 	struct rte_crypto_sgl sgl;
 	struct rte_crypto_sym_vec symvec;
+	struct rte_crypto_va_iova_ptr iv_ptr, digest_ptr;
 	struct rte_crypto_vec vec[UINT8_MAX];
 
 	sop = op->sym;
@@ -213,13 +217,14 @@ process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
 	sgl.vec = vec;
 	sgl.num = n;
 	symvec.sgl = &sgl;
-	iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
-	symvec.iv = &iv;
-	symvec.aad = (void **)&sop->aead.aad.data;
-	symvec.digest = (void **)&sop->auth.digest.data;
+	symvec.iv = &iv_ptr;
+	symvec.digest = &digest_ptr;
 	symvec.status = &st;
 	symvec.num = 1;
 
+	iv_ptr.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+	digest_ptr.va = (void *)sop->auth.digest.data;
+
 	ofs.raw = 0;
 	ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
 	ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
doc/guides/prog_guide/cryptodev_lib.rst +2 -1
index c14f750fa..e7ba35c2d 100644
@@ -620,7 +620,8 @@ operation descriptor (``struct rte_crypto_sym_vec``) containing:
   descriptors of performed operations (``struct rte_crypto_sgl``). Each instance
   of ``struct rte_crypto_sgl`` consists of a number of segments and a pointer to
   an array of segment descriptors ``struct rte_crypto_vec``;
-- pointers to arrays of size ``num`` containing IV, AAD and digest information,
+- pointers to arrays of size ``num`` containing IV, AAD and digest information
+  in the ``cpu_crypto`` sub-structure,
 - pointer to an array of size ``num`` where status information will be stored
   for each operation.
 
doc/guides/rel_notes/release_20_11.rst +3 -0
index 8b911488c..2973b2a33 100644
@@ -302,6 +302,9 @@ API Changes
   ``rte_fpga_lte_fec_configure`` and structure ``fpga_lte_fec_conf`` to
   ``rte_fpga_lte_fec_conf``.
 
+* The structure ``rte_crypto_sym_vec`` is updated to support both
+  cpu_crypto synchrounous operation and asynchronous raw data-path APIs.
+
 
 ABI Changes
 -----------
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -8
index 1d2a0ce00..973b61bd6 100644
@@ -464,9 +464,10 @@ aesni_gcm_sgl_encrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -482,9 +483,10 @@ aesni_gcm_sgl_decrypt(struct aesni_gcm_session *s,
 	processed = 0;
 	for (i = 0; i < vec->num; ++i) {
 		aesni_gcm_process_gcm_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i], vec->aad[i]);
+			&vec->sgl[i], vec->iv[i].va,
+			vec->aad[i].va);
 		 vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -505,9 +507,9 @@ aesni_gmac_sgl_generate(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_encryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
@@ -528,9 +530,9 @@ aesni_gmac_sgl_verify(struct aesni_gcm_session *s,
 		}
 
 		aesni_gcm_process_gmac_sgl_op(s, gdata_ctx,
-			&vec->sgl[i], vec->iv[i]);
+			&vec->sgl[i], vec->iv[i].va);
 		vec->status[i] = aesni_gcm_sgl_op_finalize_decryption(s,
-			gdata_ctx, vec->digest[i]);
+			gdata_ctx, vec->digest[i].va);
 		processed += (vec->status[i] == 0);
 	}
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +4 -5
index 34a39ca99..39f90f537 100644
@@ -1877,7 +1877,7 @@ generate_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			memcpy(vec->digest[i], dgst[i], len);
+			memcpy(vec->digest[i].va, dgst[i], len);
 			k++;
 		}
 	}
@@ -1893,7 +1893,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec,
 
 	for (i = 0, k = 0; i != vec->num; i++) {
 		if (vec->status[i] == 0) {
-			if (memcmp(vec->digest[i], dgst[i], len) != 0)
+			if (memcmp(vec->digest[i].va, dgst[i], len) != 0)
 				vec->status[i] = EBADMSG;
 			else
 				k++;
@@ -1956,9 +1956,8 @@ aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
 		}
 
 		/* Submit job for processing */
-		set_cpu_mb_job_params(job, s, sofs, buf, len,
-			vec->iv[i], vec->aad[i], tmp_dgst[i],
-			&vec->status[i]);
+		set_cpu_mb_job_params(job, s, sofs, buf, len, vec->iv[i].va,
+			vec->aad[i].va, tmp_dgst[i], &vec->status[i]);
 		job = submit_sync_job(mb_mgr);
 		j++;
 
lib/librte_cryptodev/rte_crypto_sym.h +29 -11
index f29c98051..e1f23d303 100644
@@ -51,26 +51,44 @@ struct rte_crypto_sgl {
 };
 
 /**
- * Synchronous operation descriptor.
- * Supposed to be used with CPU crypto API call.
+ * Crypto virtual and IOVA address descriptor, used to describe cryptographic
+ * data buffer without the length information. The length information is
+ * normally predefined during session creation.
+ */
+struct rte_crypto_va_iova_ptr {
+	void *va;
+	rte_iova_t iova;
+};
+
+/**
+ * Raw data operation descriptor.
+ * Supposed to be used with synchronous CPU crypto API call or asynchronous
+ * RAW data path API call.
  */
 struct rte_crypto_sym_vec {
+	/** number of operations to perform */
+	uint32_t num;
 	/** array of SGL vectors */
 	struct rte_crypto_sgl *sgl;
-	/** array of pointers to IV */
-	void **iv;
-	/** array of pointers to AAD */
-	void **aad;
+	/** array of pointers to cipher IV */
+	struct rte_crypto_va_iova_ptr *iv;
 	/** array of pointers to digest */
-	void **digest;
+	struct rte_crypto_va_iova_ptr *digest;
+
+	__extension__
+	union {
+		/** array of pointers to auth IV, used for chain operation */
+		struct rte_crypto_va_iova_ptr *auth_iv;
+		/** array of pointers to AAD, used for AEAD operation */
+		struct rte_crypto_va_iova_ptr *aad;
+	};
+
 	/**
 	 * array of statuses for each operation:
-	 *  - 0 on success
-	 *  - errno on error
+	 * - 0 on success
+	 * - errno on error
 	 */
 	int32_t *status;
-	/** number of operations to perform */
-	uint32_t num;
 };
 
 /**
lib/librte_ipsec/esp_inb.c +6 -6
index 96eec0131..2b1df6a03 100644
@@ -693,9 +693,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 	struct rte_ipsec_sa *sa;
 	struct replay_sqn *rsn;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -720,9 +720,9 @@ cpu_inb_pkt_prepare(const struct rte_ipsec_session *ss,
 				l4ofs + k, rc, ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/esp_outb.c +6 -6
index fb9d5864c..1e181cf2c 100644
@@ -449,9 +449,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 	uint32_t i, k, n;
 	uint32_t l2, l3;
 	union sym_op_data icv;
-	void *iv[num];
-	void *aad[num];
-	void *dgst[num];
+	struct rte_crypto_va_iova_ptr iv[num];
+	struct rte_crypto_va_iova_ptr aad[num];
+	struct rte_crypto_va_iova_ptr dgst[num];
 	uint32_t dr[num];
 	uint32_t l4ofs[num];
 	uint32_t clen[num];
@@ -488,9 +488,9 @@ cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
 				ivbuf[k]);
 
 			/* fill iv, digest and aad */
-			iv[k] = ivbuf[k];
-			aad[k] = icv.va + sa->icv_len;
-			dgst[k++] = icv.va;
+			iv[k].va = ivbuf[k];
+			aad[k].va = icv.va + sa->icv_len;
+			dgst[k++].va = icv.va;
 		} else {
 			dr[i - k] = i;
 			rte_errno = -rc;
lib/librte_ipsec/misc.h +4 -2
index 1b543ed87..79b9a2076 100644
@@ -112,7 +112,9 @@ mbuf_cut_seg_ofs(struct rte_mbuf *mb, struct rte_mbuf *ms, uint32_t ofs,
 static inline void
 cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 	union rte_crypto_sym_ofs ofs, struct rte_mbuf *mb[],
-	void *iv[], void *aad[], void *dgst[], uint32_t l4ofs[],
+	struct rte_crypto_va_iova_ptr iv[],
+	struct rte_crypto_va_iova_ptr aad[],
+	struct rte_crypto_va_iova_ptr dgst[], uint32_t l4ofs[],
 	uint32_t clen[], uint32_t num)
 {
 	uint32_t i, j, n;
@@ -136,8 +138,8 @@ cpu_crypto_bulk(const struct rte_ipsec_session *ss,
 			/* fill the request structure */
 			symvec.sgl = &vecpkt[j];
 			symvec.iv = &iv[j];
-			symvec.aad = &aad[j];
 			symvec.digest = &dgst[j];
+			symvec.aad = &aad[j];
 			symvec.status = &st[j];
 			symvec.num = i - j;
 
Signature · 1 line
2.20.1
author
#80 lore raw

PATCH v13 2/4 cryptodev: add raw crypto data-path APIs

In reply to Fan Zhang

This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors.

---
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       | 106 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++++
 lib/librte_cryptodev/rte_cryptodev.h          | 413 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 ++-
 .../rte_cryptodev_version.map                 |  10 +
 7 files changed, 665 insertions(+), 3 deletions(-)
doc/guides/cryptodevs/features/default.ini +1 -0
index 133a246ee..17b177fc4 100644
@@ -30,6 +30,7 @@ Asymmetric sessionless =
 CPU crypto             =
 Symmetric sessionless  =
 Non-Byte aligned data  =
+Sym raw data path API  =
 
 ;
 ; Supported crypto algorithms of a default crypto driver.
doc/guides/prog_guide/cryptodev_lib.rst +106 -0

112 lines not on this page · raw message

doc/guides/rel_notes/release_20_11.rst +7 -0
index 2973b2a33..85a07d86e 100644
@@ -74,6 +74,13 @@ New Features
 
   * Added SR-IOV PF support
 
+* **Added raw data-path APIs for cryptodev library.**
+
+  Cryptodev is added with raw data-path APIs to accelerate external
+  libraries or applications which need to avail fast cryptodev
+  enqueue/dequeue operations but does not necessarily depends on
+  mbufs and cryptodev operation mempools.
+
 * **Updated the aesni_mb crypto PMD.**
 
   * Added support for AES-ECB 128, 192 and 256.
lib/librte_cryptodev/rte_cryptodev.c +80 -0

86 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev.h +412 -1

425 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +49 -2

63 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_version.map +10 -0
index 7727286ac..7e4360ff0 100644
@@ -99,4 +99,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_configure_raw_dp_ctx;
+	rte_cryptodev_get_raw_dp_ctx_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
Signature · 1 line
2.20.1
author
#81 lore raw

PATCH v13 3/4 crypto/qat: add raw crypto data-path API support

In reply to Fan Zhang

This patch updates QAT PMD to add raw data-path API support.

---
 doc/guides/cryptodevs/features/qat.ini |   1 +
 doc/guides/rel_notes/release_20_11.rst |   4 +
 drivers/crypto/qat/meson.build         |   1 +
 drivers/crypto/qat/qat_sym.h           |  11 +
 drivers/crypto/qat/qat_sym_hw_dp.c     | 959 +++++++++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c       |   9 +-
 6 files changed, 983 insertions(+), 2 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c
doc/guides/cryptodevs/features/qat.ini +1 -0
index 9e82f2886..6cc09cde7 100644
@@ -17,6 +17,7 @@ Digest encrypted       = Y
 Asymmetric sessionless = Y
 RSA PRIV OP KEY EXP    = Y
 RSA PRIV OP KEY QT     = Y
+Sym raw data path API  = Y
 
 ;
 ; Supported crypto algorithms of the 'qat' crypto driver.
doc/guides/rel_notes/release_20_11.rst +4 -0
index 85a07d86e..008f4eedc 100644
@@ -104,6 +104,10 @@ New Features
   * Added support for non-HMAC auth algorithms
     (MD5, SHA1, SHA224, SHA256, SHA384, SHA512).
 
+* **Update QAT crypto PMD.**
+
+  * Added Raw Data-path APIs support.
+
 * **Added Intel ACC100 bbdev PMD.**
 
   Added a new ``acc100`` bbdev driver for the Intel\ |reg| ACC100 accelerator
drivers/crypto/qat/meson.build +1 -0
index a225f374a..bc90ec44c 100644
@@ -15,6 +15,7 @@ if dep.found()
 	qat_sources += files('qat_sym_pmd.c',
 			     'qat_sym.c',
 			     'qat_sym_session.c',
+			     'qat_sym_hw_dp.c',
 			     'qat_asym_pmd.c',
 			     'qat_asym.c')
 	qat_ext_deps += dep
drivers/crypto/qat/qat_sym.h +11 -0
index 1a9748849..7254f5e3c 100644
@@ -264,6 +264,16 @@ qat_sym_process_response(void **op, uint8_t *resp)
 	}
 	*op = (void *)rx_op;
 }
+
+int
+qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *raw_dp_ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx, uint8_t is_update);
+
+int
+qat_sym_get_dp_ctx_size(struct rte_cryptodev *dev);
+
 #else
 
 static inline void
@@ -276,5 +286,6 @@ static inline void
 qat_sym_process_response(void **op __rte_unused, uint8_t *resp __rte_unused)
 {
 }
+
 #endif
 #endif /* _QAT_SYM_H_ */
drivers/crypto/qat/qat_sym_hw_dp.c new file +959 -0

959 lines not on this page · raw message

drivers/crypto/qat/qat_sym_pmd.c +7 -2
index 314742f53..6b189c319 100644
@@ -258,7 +258,11 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
 		/* Crypto related operations */
 		.sym_session_get_size	= qat_sym_session_get_private_size,
 		.sym_session_configure	= qat_sym_session_configure,
-		.sym_session_clear	= qat_sym_session_clear
+		.sym_session_clear	= qat_sym_session_clear,
+
+		/* Raw data-path API related operations */
+		.sym_get_raw_dp_ctx_size = qat_sym_get_dp_ctx_size,
+		.sym_configure_raw_dp_ctx = qat_sym_configure_dp_ctx,
 };
 
 #ifdef RTE_LIBRTE_SECURITY
@@ -376,7 +380,8 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev,
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT |
 			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED;
+			RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED |
+			RTE_CRYPTODEV_FF_SYM_RAW_DP;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
Signature · 1 line
2.20.1
author
#82 lore raw

PATCH v13 4/4 test/crypto: add unit-test for cryptodev raw API test

In reply to Fan Zhang

This patch adds the cryptodev raw API test support to unit test. In addition a new test-case for QAT PMD for the test type is enabled.

---
 app/test/test_cryptodev.c             | 787 ++++++++++++++++++++++++--
 app/test/test_cryptodev.h             |  12 +
 app/test/test_cryptodev_blockcipher.c |  58 +-
 3 files changed, 803 insertions(+), 54 deletions(-)
app/test/test_cryptodev.c +749 -38
index 62a265520..219373e10 100644
@@ -49,6 +49,10 @@
 #define VDEV_ARGS_SIZE 100
 #define MAX_NB_SESSIONS 4
 
+#define MAX_DRV_SERVICE_CTX_SIZE 256
+
+#define MAX_RAW_DEQUEUE_COUNT	65535
+
 #define IN_PLACE 0
 #define OUT_OF_PLACE 1
 
@@ -57,6 +61,8 @@ static int gbl_driver_id;
 static enum rte_security_session_action_type gbl_action_type =
 	RTE_SECURITY_ACTION_TYPE_NONE;
 
+enum cryptodev_api_test_type global_api_test_type = CRYPTODEV_API_TEST;
+
 struct crypto_testsuite_params {
 	struct rte_mempool *mbuf_pool;
 	struct rte_mempool *large_mbuf_pool;
@@ -147,6 +153,215 @@ ceil_byte_length(uint32_t num_bits)
 		return (num_bits >> 3);
 }
 
+static uint32_t
+get_raw_dp_dequeue_count(void *user_data __rte_unused)
+{
+	return 1;
+}
+
+static void
+post_process_raw_dp_op(void *user_data,	uint32_t index __rte_unused,
+		uint8_t is_op_success)
+{
+	struct rte_crypto_op *op = user_data;
+	op->status = is_op_success ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+			RTE_CRYPTO_OP_STATUS_ERROR;
+}
+
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len)
+{
+	struct rte_crypto_sym_op *sop = op->sym;
+	struct rte_crypto_op *ret_op = NULL;
+	struct rte_crypto_vec data_vec[UINT8_MAX];
+	struct rte_crypto_va_iova_ptr cipher_iv, digest, aad_auth_iv;
+	union rte_crypto_sym_ofs ofs;
+	struct rte_crypto_sym_vec vec;
+	struct rte_crypto_sgl sgl;
+	uint32_t max_len;
+	union rte_cryptodev_session_ctx sess;
+	uint32_t count = 0;
+	struct rte_crypto_raw_dp_ctx *ctx;
+	uint32_t cipher_offset = 0, cipher_len = 0, auth_offset = 0,
+			auth_len = 0;
+	int32_t n;
+	uint32_t n_success;
+	int ctx_service_size;
+	int32_t status = 0;
+	int enqueue_status, dequeue_status;
+
+	ctx_service_size = rte_cryptodev_get_raw_dp_ctx_size(dev_id);
+	if (ctx_service_size < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	ctx = malloc(ctx_service_size);
+	if (!ctx) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
+	/* Both are enums, setting crypto_sess will suit any session type */
+	sess.crypto_sess = op->sym->session;
+
+	if (rte_cryptodev_configure_raw_dp_ctx(dev_id, qp_id, ctx,
+			op->sess_type, sess, 0) < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	cipher_iv.iova = 0;
+	cipher_iv.va = NULL;
+	aad_auth_iv.iova = 0;
+	aad_auth_iv.va = NULL;
+	digest.iova = 0;
+	digest.va = NULL;
+	sgl.vec = data_vec;
+	vec.num = 1;
+	vec.sgl = &sgl;
+	vec.iv = &cipher_iv;
+	vec.digest = &digest;
+	vec.aad = &aad_auth_iv;
+	vec.status = &status;
+
+	ofs.raw = 0;
+
+	if (is_cipher && is_auth) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = RTE_MAX(cipher_offset + cipher_len,
+				auth_offset + auth_len);
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			auth_offset = auth_offset >> 3;
+			cipher_len = cipher_len >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else if (is_cipher) {
+		cipher_offset = sop->cipher.data.offset;
+		cipher_len = sop->cipher.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+
+	} else if (is_auth) {
+		auth_offset = sop->auth.data.offset;
+		auth_len = sop->auth.data.length;
+		max_len = auth_len + auth_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			auth_offset = auth_offset >> 3;
+			auth_len = auth_len >> 3;
+		}
+		ofs.ofs.auth.head = auth_offset;
+		ofs.ofs.auth.tail = max_len - auth_offset - auth_len;
+		aad_auth_iv.va = rte_crypto_op_ctod_offset(
+				op, void *, IV_OFFSET + cipher_iv_len);
+		aad_auth_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET +
+				cipher_iv_len);
+		digest.va = (void *)sop->auth.digest.data;
+		digest.iova = sop->auth.digest.phys_addr;
+
+	} else { /* aead */
+		cipher_offset = sop->aead.data.offset;
+		cipher_len = sop->aead.data.length;
+		max_len = cipher_len + cipher_offset;
+		if (len_in_bits) {
+			max_len = max_len >> 3;
+			cipher_offset = cipher_offset >> 3;
+			cipher_len = cipher_len >> 3;
+		}
+		ofs.ofs.cipher.head = cipher_offset;
+		ofs.ofs.cipher.tail = max_len - cipher_offset - cipher_len;
+		cipher_iv.va = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
+		cipher_iv.iova = rte_crypto_op_ctophys_offset(op, IV_OFFSET);
+		aad_auth_iv.va = (void *)sop->aead.aad.data;
+		aad_auth_iv.iova = sop->aead.aad.phys_addr;
+		digest.va = (void *)sop->aead.digest.data;
+		digest.iova = sop->aead.digest.phys_addr;
+	}
+
+	n = rte_crypto_mbuf_to_vec(sop->m_src, 0, max_len,
+			data_vec, RTE_DIM(data_vec));
+	if (n < 0 || n > sop->m_src->nb_segs) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	sgl.num = n;
+
+	if (rte_cryptodev_raw_enqueue_burst(ctx, &vec, ofs, (void **)&op,
+			&enqueue_status) < 1) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	if (enqueue_status == 0) {
+		status = rte_cryptodev_raw_enqueue_done(ctx, 1);
+		if (status < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	} else if (enqueue_status < 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		goto exit;
+	}
+
+	n = n_success = 0;
+	while (count++ < MAX_RAW_DEQUEUE_COUNT && n == 0) {
+		n = rte_cryptodev_raw_dequeue_burst(ctx,
+			get_raw_dp_dequeue_count, post_process_raw_dp_op,
+				(void **)&ret_op, 0, &n_success,
+				&dequeue_status);
+		if (dequeue_status < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+		if (n == 0)
+			rte_pause();
+	}
+
+	if (n == 1 && dequeue_status == 0) {
+		if (rte_cryptodev_raw_dequeue_done(ctx, 1) < 0) {
+			op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+			goto exit;
+		}
+	}
+
+	op->status = (count == MAX_RAW_DEQUEUE_COUNT + 1 || ret_op != op ||
+			n_success < 1) ? RTE_CRYPTO_OP_STATUS_ERROR :
+					RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+exit:
+	free(ctx);
+}
+
 static void
 process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
 {
@@ -1661,6 +1876,9 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 				process_crypto_request(ts_params->valid_devs[0],
@@ -1715,12 +1933,18 @@ test_AES_cipheronly_all(void)
 static int
 test_AES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
 }
 
 static int
 test_DES_docsis_all(void)
 {
+	/* Data-path service does not support DOCSIS yet */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
 }
 
@@ -2435,6 +2659,12 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2475,7 +2705,11 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -2513,6 +2747,12 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -2554,7 +2794,11 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2580,6 +2824,16 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2624,6 +2878,9 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -2653,6 +2910,16 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2695,7 +2962,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -2860,6 +3131,16 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -2902,8 +3183,12 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
-						ut_params->op);
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+				ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -2959,6 +3244,12 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -2988,7 +3279,11 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3031,10 +3326,14 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
+	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3116,6 +3415,9 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3201,6 +3503,9 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3269,6 +3574,16 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	uint8_t *ciphertext, *plaintext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3311,7 +3626,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3344,6 +3663,16 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3386,7 +3715,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -3427,6 +3760,9 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3510,6 +3846,9 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
 	uint64_t feat_flags = dev_info.feature_flags;
@@ -3629,6 +3968,9 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_ENCRYPT,
@@ -3719,6 +4061,16 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned ciphertext_pad_len;
 	unsigned ciphertext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3761,7 +4113,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_dst;
@@ -3799,6 +4155,9 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
 					RTE_CRYPTO_CIPHER_OP_DECRYPT,
@@ -3886,6 +4245,12 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EEA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_ZUC_EEA3;
@@ -3929,7 +4294,11 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -3969,6 +4338,16 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4024,7 +4403,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 	ut_params->obuf = ut_params->op->sym->m_src;
@@ -4092,6 +4475,14 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
 		}
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
+	}
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
 	}
 
 	/* Create SNOW 3G session */
@@ -4160,7 +4551,11 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4270,7 +4665,14 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4349,7 +4751,11 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4457,7 +4863,15 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 
 	uint64_t feat_flags = dev_info.feature_flags;
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	if (op_mode == OUT_OF_PLACE) {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
 			return -ENOTSUP;
@@ -4531,7 +4945,11 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4642,7 +5060,14 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -4721,7 +5146,11 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -4806,6 +5235,16 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4862,7 +5301,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -4905,6 +5348,16 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	uint8_t *plaintext, *ciphertext;
 	unsigned plaintext_pad_len;
 	unsigned plaintext_len;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 
@@ -4949,7 +5402,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5004,6 +5461,12 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	plaintext_len = ceil_byte_length(tdata->plaintext.len);
 
 	/* Append data which is padded to a multiple */
@@ -5036,7 +5499,11 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 0, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 						ut_params->op);
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -5084,6 +5551,12 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* Check if device supports ZUC EIA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_ZUC_EIA3;
@@ -5124,7 +5597,11 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 1, 0);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 				ut_params->op);
 	ut_params->obuf = ut_params->op->sym->m_src;
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5181,7 +5658,15 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5256,7 +5741,11 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5363,7 +5852,15 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 					"in both input and output mbufs.\n");
 			return -ENOTSUP;
 		}
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
@@ -5442,7 +5939,11 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 1, tdata->cipher_iv.len);
+	else
+		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
@@ -5585,6 +6086,9 @@ test_kasumi_decryption_test_case_2(void)
 static int
 test_kasumi_decryption_test_case_3(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -5784,6 +6288,9 @@ test_snow3g_auth_cipher_part_digest_enc_oop(void)
 static int
 test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -5798,6 +6305,9 @@ test_snow3g_auth_cipher_test_case_3_oop_sgl(void)
 static int
 test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
+	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -6151,11 +6661,12 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	unsigned int ciphertext_len;
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms separately */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6166,6 +6677,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 		return -ENOTSUP;
 	}
 
+	if (op_mode == OUT_OF_PLACE)
+		return -ENOTSUP;
+
 	/* Create the session */
 	if (verify)
 		retval = create_wireless_algo_cipher_auth_session(
@@ -6197,9 +6711,11 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	/* clear mbuf payload */
 	memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
 		rte_pktmbuf_tailroom(ut_params->ibuf));
-	if (op_mode == OUT_OF_PLACE)
+	if (op_mode == OUT_OF_PLACE) {
+
 		memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
 				rte_pktmbuf_tailroom(ut_params->obuf));
+	}
 
 	ciphertext_len = ceil_byte_length(tdata->ciphertext.len_bits);
 	plaintext_len = ceil_byte_length(tdata->plaintext.len_bits);
@@ -6240,18 +6756,17 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
-	ut_params->op = op;
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
@@ -6342,11 +6857,12 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	uint8_t digest_buffer[10000];
 
 	struct rte_cryptodev_info dev_info;
-	struct rte_crypto_op *op;
 
 	/* Check if device supports particular algorithms */
 	if (test_mixed_check_if_unsupported(tdata))
 		return -ENOTSUP;
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6445,20 +6961,18 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 	if (retval < 0)
 		return retval;
 
-	op = process_crypto_request(ts_params->valid_devs[0],
+	ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 
 	/* Check if the op failed because the device doesn't */
 	/* support this particular combination of algorithms */
-	if (op == NULL && ut_params->op->status ==
+	if (ut_params->op == NULL && ut_params->op->status ==
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
 		return -ENOTSUP;
 	}
 
-	ut_params->op = op;
-
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
 
 	ut_params->obuf = (op_mode == IN_PLACE ?
@@ -6999,6 +7513,16 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	uint8_t *ciphertext, *auth_tag;
 	uint16_t plaintext_pad_len;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -7048,6 +7572,9 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8496,6 +9023,16 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	int retval;
 	uint8_t *plaintext;
 	uint32_t i;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -8545,6 +9082,9 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	/* Process crypto operation */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -8839,6 +9379,9 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		return -ENOTSUP;
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -8928,8 +9471,9 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
-	/* not supported with CPU crypto */
-	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+	/* not supported with CPU crypto and raw data-path APIs*/
+	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
+			global_api_test_type == CRYPTODEV_RAW_API_TEST)
 		return -ENOTSUP;
 
 	/* Create AEAD session */
@@ -9115,6 +9659,12 @@ test_authenticated_decryption_sessionless(
 		return -ENOTSUP;
 	}
 
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
+
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		return -ENOTSUP;
@@ -9156,8 +9706,13 @@ test_authenticated_decryption_sessionless(
 			"crypto op session type not sessionless");
 
 	/* Process crypto operation */
-	TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
-			ut_params->op), "failed to process sym crypto op");
+	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
+	else
+		TEST_ASSERT_NOT_NULL(process_crypto_request(
+			ts_params->valid_devs[0], ut_params->op),
+				"failed to process sym crypto op");
 
 	TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
 
@@ -9447,6 +10002,16 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9475,6 +10040,9 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -9507,6 +10075,16 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -9533,6 +10111,9 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10038,6 +10619,16 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	int retval;
 
@@ -10101,6 +10692,9 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10159,6 +10753,16 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	int retval;
 	uint32_t plaintext_pad_len;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	TEST_ASSERT_NOT_EQUAL(tdata->gmac_tag.len, 0,
 			      "No GMAC length in the source data");
@@ -10218,6 +10822,9 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -10733,6 +11340,16 @@ test_authentication_verify_fail_when_data_corruption(
 	int retval;
 
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10742,6 +11359,7 @@ test_authentication_verify_fail_when_data_corruption(
 			&cap_idx) == NULL)
 		return -ENOTSUP;
 
+
 	/* Create session */
 	retval = create_auth_session(ut_params,
 			ts_params->valid_devs[0],
@@ -10783,7 +11401,10 @@ test_authentication_verify_fail_when_data_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10801,6 +11422,16 @@ test_authentication_verify_GMAC_fail_when_corruption(
 {
 	int retval;
 	uint8_t *plaintext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10854,7 +11485,10 @@ test_authentication_verify_GMAC_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10873,6 +11507,16 @@ test_authenticated_decryption_fail_when_corruption(
 	int retval;
 
 	uint8_t *ciphertext;
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10929,7 +11573,10 @@ test_authenticated_decryption_fail_when_corruption(
 		TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
 			RTE_CRYPTO_OP_STATUS_SUCCESS,
 			"authentication not failed");
-	} else {
+	} else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
+	else {
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
 		TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
@@ -10950,6 +11597,16 @@ test_authenticated_encryt_with_esn(
 	uint16_t plaintext_pad_len;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11024,6 +11681,9 @@ test_authenticated_encryt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(
 			ts_params->valid_devs[0], ut_params->op);
@@ -11070,6 +11730,16 @@ test_authenticated_decrypt_with_esn(
 	uint8_t *ciphertext;
 	uint8_t cipher_key[reference->cipher_key.len + 1];
 	uint8_t auth_key[reference->auth_key.len + 1];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+	uint64_t feat_flags = dev_info.feature_flags;
+
+	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		printf("Device doesn't support RAW data-path APIs.\n");
+		return -ENOTSUP;
+	}
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -11144,6 +11814,9 @@ test_authenticated_decrypt_with_esn(
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_crypt_auth_op(ts_params->valid_devs[0],
 			ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 1, 1, 0, 0);
 	else
 		ut_params->op = process_crypto_request(ts_params->valid_devs[0],
 			ut_params->op);
@@ -11284,10 +11957,21 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		if (sgl_in && (!(dev_info.feature_flags &
 				RTE_CRYPTODEV_FF_IN_PLACE_SGL)))
 			return -ENOTSUP;
+
+		uint64_t feat_flags = dev_info.feature_flags;
+
+		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+			printf("Device doesn't support RAW data-path APIs.\n");
+			return -ENOTSUP;
+		}
 	} else {
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		unsigned int sgl_out = (fragsz_oop ? fragsz_oop : fragsz) <
 				tdata->plaintext.len;
+		/* Raw data path API does not support OOP */
+		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+			return -ENOTSUP;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
@@ -11483,6 +12167,9 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	if (oop == IN_PLACE &&
 			gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
 		process_cpu_aead_op(ts_params->valid_devs[0], ut_params->op);
+	else if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
+		process_sym_raw_dp_op(ts_params->valid_devs[0], 0,
+				ut_params->op, 0, 0, 0, 0);
 	else
 		TEST_ASSERT_NOT_NULL(
 			process_crypto_request(ts_params->valid_devs[0],
@@ -12980,6 +13667,30 @@ test_cryptodev_bcmfs(void)
 	return unit_test_suite_runner(&cryptodev_testsuite);
 }
 
+static int
+test_cryptodev_qat_raw_api(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+	int ret;
+
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
+
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check that both "
+		"CONFIG_RTE_LIBRTE_PMD_QAT and CONFIG_RTE_LIBRTE_PMD_QAT_SYM "
+		"are enabled in config file to run this testsuite.\n");
+		return TEST_SKIPPED;
+	}
+
+	global_api_test_type = CRYPTODEV_RAW_API_TEST;
+	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	global_api_test_type = CRYPTODEV_API_TEST;
+
+	return ret;
+}
+
+REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+		test_cryptodev_qat_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
 REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
 REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
app/test/test_cryptodev.h +12 -0
index c58126368..bf4f6c8d6 100644
@@ -72,6 +72,13 @@
 #define CRYPTODEV_NAME_NITROX_PMD	crypto_nitrox_sym
 #define CRYPTODEV_NAME_BCMFS_PMD	crypto_bcmfs
 
+enum cryptodev_api_test_type {
+	CRYPTODEV_API_TEST = 0,
+	CRYPTODEV_RAW_API_TEST
+};
+
+extern enum cryptodev_api_test_type global_api_test_type;
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -210,4 +217,9 @@ create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
 	return NULL;
 }
 
+void
+process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
+		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
+		uint8_t len_in_bits, uint8_t cipher_iv_len);
+
 #endif /* TEST_CRYPTODEV_H_ */
app/test/test_cryptodev_blockcipher.c +42 -16

79 lines not on this page · raw message

Signature · 1 line
2.20.1
Acked-by on the v13 series Applied
#83 lore raw

Re: [dpdk-dev] [dpdk-dev v13 0/4] cryptodev: add raw data-path APIs

In reply to Fan Zhang
Quoted 70 lines

Subject: [dpdk-dev v13 0/4] cryptodev: add raw data-path APIs

The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic processing provided by DPDK crypto PMDs through the cryptodev API but in a manner that is not dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) in their data-path implementation.

The raw data-path APIs have the following advantages:
- External data structure friendly design. The new APIs uses the operation descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and IOVA addresses as input. Moreover, the APIs does not require the user to allocate the descriptor from mempool, nor requiring mbufs to describe input data's virtual and IOVA addresses. All these features made the translation from user's own data structure into the descriptor easier and more efficient.
- Flexible enqueue and dequeue operation. The raw data-path APIs gives the user more control to the enqueue and dequeue operations, including the capability of precious enqueue/dequeue count, abandoning enqueue or
dequeue
at any time, and operation status translation and set on the fly.

v13:
- Fixed a typo.

v12:
- Fixed and updated documentation.
- Fixed typo.

v11:
- Rebased on top of latest master.
- API changed followed by the discussion results.
- Fixed a few grammar error thanks to Akhil.
- Reverted attach session API changes.
- Fixed QAT driver bugs.

v10:
- Changed rte_crypto_sym_vec structure to support both sync cpu_crypto and async raw data-path API.
- Changed documentation.
- Changed API names.
- Changed the way data-path context is initialized.
- Added new API to attach session or xform to existing context.
- Changed QAT PMD accordingly with new APIs.
- Changed unit test to use the device feature flag for the raw API tests.

v9:
- Changed return types of submit_done() and dequeue_done() APIs.
- Added release note update.

v8:
- Updated following by comments.
- Fixed a few bugs.
- Fixed ARM build error.
- Updated the unit test covering all tests.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

Quoted 8 lines

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Added Konstantin's ACK on 1/4 patch.
Fixed a couple of compilation issues in documentation. Fixed a couple of typos in comments and patch description.

Applied to dpdk-next-crypto

Thanks.